2026-02-17T05:38:44.975 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-17T05:38:44.988 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T05:38:45.067 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059 branch: wip-bharath19-testing-2026-02-13-0903-squid description: rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/one workloads/snaps-few-objects} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '51059' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath19-testing-2026-02-13-0903-squid ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: mon ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async mgr: debug mgr: 20 debug monc: 10 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos service trim min: 5 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(POOL_APP_NOT_ENABLED\) sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath19-testing-2026-02-13-0903-squid sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 3603 sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath19-testing-2026-02-13-0903-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e targets: trial078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgwdKViKCYrGxwNdAmVRuhqhondgpFoGla/qr6gDvfH8kK6aW5nt3zxsBfGZCjZo3uOLclmxW23haLeTP94Fx8= trial115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCbkb9SCDeN/JWz5/4NjaOdwJvaDHnC1NPlFLVpn44KgRwjhu/SogqSyZ2V6PYwLbDQ4NXUr/Bq4Ike+8C+JUzo= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 20 thrash_delay: 1 - rados: clients: - client.0 objects: 50 op_weights: copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-17_04:03:08 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T05:38:45.067 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa; will attempt to use it 2026-02-17T05:38:45.068 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks 2026-02-17T05:38:45.068 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T05:38:45.069 INFO:teuthology.task.internal:Checking packages... 2026-02-17T05:38:45.069 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e' 2026-02-17T05:38:45.069 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T05:38:45.069 INFO:teuthology.packaging:ref: None 2026-02-17T05:38:45.069 INFO:teuthology.packaging:tag: None 2026-02-17T05:38:45.070 INFO:teuthology.packaging:branch: wip-bharath19-testing-2026-02-13-0903-squid 2026-02-17T05:38:45.070 INFO:teuthology.packaging:sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:38:45.070 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath19-testing-2026-02-13-0903-squid 2026-02-17T05:38:45.188 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-705.g04cf6dd4 2026-02-17T05:38:45.190 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T05:38:45.201 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T05:38:45.201 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T05:38:45.215 INFO:teuthology.task.internal:Saving configuration 2026-02-17T05:38:45.227 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T05:38:45.238 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T05:38:45.313 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial078.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 05:35:18.874615', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgwdKViKCYrGxwNdAmVRuhqhondgpFoGla/qr6gDvfH8kK6aW5nt3zxsBfGZCjZo3uOLclmxW23haLeTP94Fx8='} 2026-02-17T05:38:45.369 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial115.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 05:35:18.873680', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCbkb9SCDeN/JWz5/4NjaOdwJvaDHnC1NPlFLVpn44KgRwjhu/SogqSyZ2V6PYwLbDQ4NXUr/Bq4Ike+8C+JUzo='} 2026-02-17T05:38:45.369 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T05:38:45.380 INFO:teuthology.task.internal:roles: ubuntu@trial078.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2026-02-17T05:38:45.380 INFO:teuthology.task.internal:roles: ubuntu@trial115.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2026-02-17T05:38:45.380 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T05:38:45.514 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7727601ed580>, signals=[15]) 2026-02-17T05:38:45.514 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T05:38:45.525 INFO:teuthology.task.internal:Opening connections... 2026-02-17T05:38:45.525 DEBUG:teuthology.task.internal:connecting to ubuntu@trial078.front.sepia.ceph.com 2026-02-17T05:38:45.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:38:45.596 DEBUG:teuthology.task.internal:connecting to ubuntu@trial115.front.sepia.ceph.com 2026-02-17T05:38:45.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:38:45.658 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T05:38:45.669 DEBUG:teuthology.orchestra.run.trial078:> uname -m 2026-02-17T05:38:45.685 INFO:teuthology.orchestra.run.trial078.stdout:x86_64 2026-02-17T05:38:45.685 DEBUG:teuthology.orchestra.run.trial078:> cat /etc/os-release 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:NAME="CentOS Stream" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:VERSION="9" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:ID="centos" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:ID_LIKE="rhel fedora" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:VERSION_ID="9" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:PLATFORM_ID="platform:el9" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:ANSI_COLOR="0;31" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:LOGO="fedora-logo-icon" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:HOME_URL="https://centos.org/" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T05:38:45.741 INFO:teuthology.orchestra.run.trial078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T05:38:45.742 INFO:teuthology.orchestra.run.trial078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T05:38:45.742 INFO:teuthology.lock.ops:Updating trial078.front.sepia.ceph.com on lock server 2026-02-17T05:38:45.811 DEBUG:teuthology.orchestra.run.trial115:> uname -m 2026-02-17T05:38:45.826 INFO:teuthology.orchestra.run.trial115.stdout:x86_64 2026-02-17T05:38:45.826 DEBUG:teuthology.orchestra.run.trial115:> cat /etc/os-release 2026-02-17T05:38:45.881 INFO:teuthology.orchestra.run.trial115.stdout:NAME="CentOS Stream" 2026-02-17T05:38:45.881 INFO:teuthology.orchestra.run.trial115.stdout:VERSION="9" 2026-02-17T05:38:45.881 INFO:teuthology.orchestra.run.trial115.stdout:ID="centos" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:ID_LIKE="rhel fedora" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:VERSION_ID="9" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:PLATFORM_ID="platform:el9" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:ANSI_COLOR="0;31" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:LOGO="fedora-logo-icon" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:HOME_URL="https://centos.org/" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T05:38:45.882 INFO:teuthology.orchestra.run.trial115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T05:38:45.882 INFO:teuthology.lock.ops:Updating trial115.front.sepia.ceph.com on lock server 2026-02-17T05:38:45.958 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T05:38:45.969 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T05:38:45.980 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T05:38:45.980 DEBUG:teuthology.orchestra.run.trial078:> test '!' -e /home/ubuntu/cephtest 2026-02-17T05:38:45.981 DEBUG:teuthology.orchestra.run.trial115:> test '!' -e /home/ubuntu/cephtest 2026-02-17T05:38:45.995 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T05:38:46.006 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T05:38:46.006 DEBUG:teuthology.orchestra.run.trial078:> test -z $(ls -A /var/lib/ceph) 2026-02-17T05:38:46.037 DEBUG:teuthology.orchestra.run.trial115:> test -z $(ls -A /var/lib/ceph) 2026-02-17T05:38:46.053 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T05:38:46.181 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T05:38:46.201 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-17T05:38:46.202 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-17T05:38:46.202 DEBUG:teuthology.orchestra.run.trial078:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T05:38:46.202 DEBUG:teuthology.orchestra.run.trial115:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T05:38:46.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:38:46.217 DEBUG:teuthology.orchestra.run.trial078:> uname -r 2026-02-17T05:38:46.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:38:46.218 DEBUG:teuthology.orchestra.run.trial115:> uname -r 2026-02-17T05:38:46.273 INFO:teuthology.orchestra.run.trial078.stdout:5.14.0-645.el9.x86_64 2026-02-17T05:38:46.273 INFO:teuthology.task.kernel:Running kernel on trial078: 5.14.0-645.el9.x86_64 2026-02-17T05:38:46.273 DEBUG:teuthology.orchestra.run.trial078:> sudo yum install -y kernel 2026-02-17T05:38:46.273 INFO:teuthology.orchestra.run.trial115.stdout:5.14.0-645.el9.x86_64 2026-02-17T05:38:46.274 INFO:teuthology.task.kernel:Running kernel on trial115: 5.14.0-645.el9.x86_64 2026-02-17T05:38:46.274 DEBUG:teuthology.orchestra.run.trial115:> sudo yum install -y kernel 2026-02-17T05:38:47.315 INFO:teuthology.orchestra.run.trial078.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-17T05:38:47.880 INFO:teuthology.orchestra.run.trial115.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.9 MB 00:01 2026-02-17T05:38:50.911 INFO:teuthology.orchestra.run.trial115.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-17T05:38:54.310 INFO:teuthology.orchestra.run.trial115.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 7.8 MB 00:01 2026-02-17T05:38:55.517 INFO:teuthology.orchestra.run.trial115.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 20 kB 00:00 2026-02-17T05:38:56.372 INFO:teuthology.orchestra.run.trial115.stdout:Extra Packages for Enterprise Linux 26 MB/s | 20 MB 00:00 2026-02-17T05:38:59.281 INFO:teuthology.orchestra.run.trial078.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 27 MB 00:11 2026-02-17T05:38:59.623 INFO:teuthology.orchestra.run.trial115.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-17T05:39:00.634 INFO:teuthology.orchestra.run.trial115.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T05:39:00.655 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:39:00.656 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:39:00.656 INFO:teuthology.orchestra.run.trial115.stdout: Package Architecture Version Repository Size 2026-02-17T05:39:00.656 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:39:00.656 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:39:00.656 INFO:teuthology.orchestra.run.trial115.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T05:39:00.656 INFO:teuthology.orchestra.run.trial115.stdout:Installing dependencies: 2026-02-17T05:39:00.656 INFO:teuthology.orchestra.run.trial115.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T05:39:00.656 INFO:teuthology.orchestra.run.trial115.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout:Install 4 Packages 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 88 M 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 135 M 2026-02-17T05:39:00.657 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:39:00.913 INFO:teuthology.orchestra.run.trial115.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 4.0 MB/s | 803 kB 00:00 2026-02-17T05:39:01.087 INFO:teuthology.orchestra.run.trial115.stdout:(2/4): kernel-core-5.14.0-681.el9.x86_64.rpm 46 MB/s | 17 MB 00:00 2026-02-17T05:39:01.258 INFO:teuthology.orchestra.run.trial115.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 89 MB/s | 31 MB 00:00 2026-02-17T05:39:01.313 INFO:teuthology.orchestra.run.trial115.stdout:(4/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 66 MB/s | 40 MB 00:00 2026-02-17T05:39:01.313 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:39:01.313 INFO:teuthology.orchestra.run.trial115.stdout:Total 134 MB/s | 88 MB 00:00 2026-02-17T05:39:01.504 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:39:01.706 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:39:01.706 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:39:02.054 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:39:02.055 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:39:02.594 INFO:teuthology.orchestra.run.trial078.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 7.8 MB 00:00 2026-02-17T05:39:03.143 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:39:03.430 INFO:teuthology.orchestra.run.trial115.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:39:03.448 INFO:teuthology.orchestra.run.trial115.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:39:03.493 INFO:teuthology.orchestra.run.trial078.stdout:CentOS Stream 9 - Extras packages 126 kB/s | 20 kB 00:00 2026-02-17T05:39:04.442 INFO:teuthology.orchestra.run.trial078.stdout:Extra Packages for Enterprise Linux 23 MB/s | 20 MB 00:00 2026-02-17T05:39:05.218 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:39:05.246 INFO:teuthology.orchestra.run.trial115.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:39:07.601 INFO:teuthology.orchestra.run.trial078.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-17T05:39:08.265 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:39:08.271 INFO:teuthology.orchestra.run.trial115.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:08.614 INFO:teuthology.orchestra.run.trial078.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T05:39:08.634 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:39:08.635 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:39:08.636 INFO:teuthology.orchestra.run.trial078.stdout: Package Architecture Version Repository Size 2026-02-17T05:39:08.636 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:39:08.636 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:39:08.636 INFO:teuthology.orchestra.run.trial078.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T05:39:08.636 INFO:teuthology.orchestra.run.trial078.stdout:Installing dependencies: 2026-02-17T05:39:08.636 INFO:teuthology.orchestra.run.trial078.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T05:39:08.636 INFO:teuthology.orchestra.run.trial078.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T05:39:08.637 INFO:teuthology.orchestra.run.trial078.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T05:39:08.637 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:39:08.637 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:39:08.637 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:39:08.637 INFO:teuthology.orchestra.run.trial078.stdout:Install 4 Packages 2026-02-17T05:39:08.637 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:39:08.637 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 88 M 2026-02-17T05:39:08.638 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 135 M 2026-02-17T05:39:08.638 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:39:08.965 INFO:teuthology.orchestra.run.trial078.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 30 MB/s | 803 kB 00:00 2026-02-17T05:39:09.112 INFO:teuthology.orchestra.run.trial078.stdout:(2/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 229 MB/s | 40 MB 00:00 2026-02-17T05:39:09.167 INFO:teuthology.orchestra.run.trial078.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 153 MB/s | 31 MB 00:00 2026-02-17T05:39:09.334 INFO:teuthology.orchestra.run.trial078.stdout:(4/4): kernel-core-5.14.0-681.el9.x86_64.rpm 43 MB/s | 17 MB 00:00 2026-02-17T05:39:09.334 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:39:09.334 INFO:teuthology.orchestra.run.trial078.stdout:Total 126 MB/s | 88 MB 00:00 2026-02-17T05:39:09.525 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:39:09.734 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:39:09.734 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:39:10.081 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:39:10.081 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:39:11.158 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:39:11.439 INFO:teuthology.orchestra.run.trial078.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:39:11.459 INFO:teuthology.orchestra.run.trial078.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:39:11.508 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:13.237 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:39:13.266 INFO:teuthology.orchestra.run.trial078.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:39:16.295 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:39:16.302 INFO:teuthology.orchestra.run.trial078.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:19.339 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:41.794 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:41.815 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:41.879 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:41.879 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:39:41.879 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:39:41.879 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:39:41.931 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:41.931 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:39:41.932 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:39:41.932 INFO:teuthology.orchestra.run.trial115.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T05:39:41.932 INFO:teuthology.orchestra.run.trial115.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T05:39:41.932 INFO:teuthology.orchestra.run.trial115.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T05:39:41.932 INFO:teuthology.orchestra.run.trial115.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T05:39:41.932 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:39:41.932 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:39:41.990 DEBUG:teuthology.orchestra.run.trial115:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:39:42.014 INFO:teuthology.orchestra.run.trial115.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:39:42.015 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:39:42.015 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T05:39:42.015 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T05:39:42.015 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T05:39:42.015 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2026-02-17T05:39:42.015 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial115.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T05:39:42.015 DEBUG:teuthology.orchestra.run.trial115:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:39:42.078 INFO:teuthology.orchestra.run.trial115.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:39:42.079 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:39:42.080 DEBUG:teuthology.orchestra.run.trial115:> sudo rpm -qi grub2-tools 2026-02-17T05:39:42.149 INFO:teuthology.orchestra.run.trial115.stdout:Name : grub2-tools 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Epoch : 1 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Version : 2.06 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Release : 118.el9 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Architecture: x86_64 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Group : Unspecified 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Size : 8393909 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:License : GPLv3+ 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T05:39:42.150 INFO:teuthology.orchestra.run.trial115.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:Packager : builder@centos.org 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:Vendor : CentOS 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:Summary : Support tools for GRUB. 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:Description : 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T05:39:42.151 INFO:teuthology.orchestra.run.trial115.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T05:39:42.152 INFO:teuthology.orchestra.run.trial115.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T05:39:42.152 INFO:teuthology.orchestra.run.trial115.stdout:hardware devices. 2026-02-17T05:39:42.152 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:39:42.152 INFO:teuthology.orchestra.run.trial115.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T05:39:42.153 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T05:39:42.153 INFO:teuthology.task.kernel:Updating grub on trial115 to boot 5.14.0-681.el9.x86_64 2026-02-17T05:39:42.153 DEBUG:teuthology.orchestra.run.trial115:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T05:39:42.218 INFO:teuthology.orchestra.run.trial115.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:39:42.219 DEBUG:teuthology.orchestra.run.trial115:> sudo command -v grubby && echo yes || echo no 2026-02-17T05:39:42.284 INFO:teuthology.orchestra.run.trial115.stdout:/sbin/grubby 2026-02-17T05:39:42.285 INFO:teuthology.orchestra.run.trial115.stdout:yes 2026-02-17T05:39:42.286 DEBUG:teuthology.orchestra.run.trial115:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T05:39:42.349 INFO:teuthology.orchestra.run.trial115.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:39:42.349 DEBUG:teuthology.orchestra.run.trial115:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T05:39:42.421 DEBUG:teuthology.orchestra.run.trial115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T05:39:42.628 INFO:teuthology.orchestra.run.trial115.stderr:Generating grub configuration file ... 2026-02-17T05:39:42.691 INFO:teuthology.orchestra.run.trial115.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T05:39:42.697 INFO:teuthology.orchestra.run.trial115.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T05:39:42.943 INFO:teuthology.orchestra.run.trial115.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T05:39:42.949 INFO:teuthology.orchestra.run.trial115.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T05:39:42.980 INFO:teuthology.orchestra.run.trial115.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T05:39:42.986 INFO:teuthology.orchestra.run.trial115.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T05:39:43.009 INFO:teuthology.orchestra.run.trial115.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T05:39:43.016 INFO:teuthology.orchestra.run.trial115.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T05:39:43.097 INFO:teuthology.orchestra.run.trial115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T05:39:43.105 INFO:teuthology.orchestra.run.trial115.stderr:done 2026-02-17T05:39:43.107 DEBUG:teuthology.orchestra.run.trial115:> test -d /sys/firmware/efi 2026-02-17T05:39:43.122 DEBUG:teuthology.orchestra.run.trial115:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T05:39:43.188 INFO:teuthology.orchestra.run.trial115.stdout:/boot/efi/EFI/BOOT 2026-02-17T05:39:43.188 INFO:teuthology.orchestra.run.trial115.stdout:/boot/efi/EFI/centos 2026-02-17T05:39:43.189 DEBUG:teuthology.orchestra.run.trial115:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T05:39:43.254 INFO:teuthology.orchestra.run.trial115.stdout:no 2026-02-17T05:39:43.255 DEBUG:teuthology.orchestra.run.trial115:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T05:39:43.318 INFO:teuthology.orchestra.run.trial115.stdout:yes 2026-02-17T05:39:43.319 DEBUG:teuthology.orchestra.run.trial115:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T05:39:43.384 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial115 2026-02-17T05:39:43.384 DEBUG:teuthology.orchestra.run.trial115:> sudo bash -c ' 2026-02-17T05:39:43.385 DEBUG:teuthology.orchestra.run.trial115:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T05:39:43.385 DEBUG:teuthology.orchestra.run.trial115:> ' 2026-02-17T05:39:49.630 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:49.652 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:49.823 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:49.823 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T05:39:49.823 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T05:39:49.823 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T05:39:49.874 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T05:39:49.875 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:39:49.875 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:39:49.875 INFO:teuthology.orchestra.run.trial078.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T05:39:49.875 INFO:teuthology.orchestra.run.trial078.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T05:39:49.875 INFO:teuthology.orchestra.run.trial078.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T05:39:49.875 INFO:teuthology.orchestra.run.trial078.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T05:39:49.875 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:39:49.875 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:39:49.932 DEBUG:teuthology.orchestra.run.trial078:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:39:49.958 INFO:teuthology.orchestra.run.trial078.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:39:49.959 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:39:49.959 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T05:39:49.959 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T05:39:49.959 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T05:39:49.959 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T05:39:49.960 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial078.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T05:39:49.960 DEBUG:teuthology.orchestra.run.trial078:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T05:39:50.022 INFO:teuthology.orchestra.run.trial078.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T05:39:50.023 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T05:39:50.023 DEBUG:teuthology.orchestra.run.trial078:> sudo rpm -qi grub2-tools 2026-02-17T05:39:50.092 INFO:teuthology.orchestra.run.trial078.stdout:Name : grub2-tools 2026-02-17T05:39:50.092 INFO:teuthology.orchestra.run.trial078.stdout:Epoch : 1 2026-02-17T05:39:50.092 INFO:teuthology.orchestra.run.trial078.stdout:Version : 2.06 2026-02-17T05:39:50.092 INFO:teuthology.orchestra.run.trial078.stdout:Release : 118.el9 2026-02-17T05:39:50.092 INFO:teuthology.orchestra.run.trial078.stdout:Architecture: x86_64 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Group : Unspecified 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Size : 8393909 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:License : GPLv3+ 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Packager : builder@centos.org 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:Vendor : CentOS 2026-02-17T05:39:50.093 INFO:teuthology.orchestra.run.trial078.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout:Summary : Support tools for GRUB. 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout:Description : 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout:hardware devices. 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:39:50.094 INFO:teuthology.orchestra.run.trial078.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T05:39:50.095 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T05:39:50.095 INFO:teuthology.task.kernel:Updating grub on trial078 to boot 5.14.0-681.el9.x86_64 2026-02-17T05:39:50.096 DEBUG:teuthology.orchestra.run.trial078:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T05:39:50.159 INFO:teuthology.orchestra.run.trial078.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:39:50.160 DEBUG:teuthology.orchestra.run.trial078:> sudo command -v grubby && echo yes || echo no 2026-02-17T05:39:50.224 INFO:teuthology.orchestra.run.trial078.stdout:/sbin/grubby 2026-02-17T05:39:50.225 INFO:teuthology.orchestra.run.trial078.stdout:yes 2026-02-17T05:39:50.226 DEBUG:teuthology.orchestra.run.trial078:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T05:39:50.291 INFO:teuthology.orchestra.run.trial078.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T05:39:50.291 DEBUG:teuthology.orchestra.run.trial078:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T05:39:50.364 DEBUG:teuthology.orchestra.run.trial078:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T05:39:50.567 INFO:teuthology.orchestra.run.trial078.stderr:Generating grub configuration file ... 2026-02-17T05:39:50.632 INFO:teuthology.orchestra.run.trial078.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T05:39:50.638 INFO:teuthology.orchestra.run.trial078.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T05:39:50.886 INFO:teuthology.orchestra.run.trial078.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T05:39:50.892 INFO:teuthology.orchestra.run.trial078.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T05:39:50.923 INFO:teuthology.orchestra.run.trial078.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T05:39:50.930 INFO:teuthology.orchestra.run.trial078.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T05:39:50.953 INFO:teuthology.orchestra.run.trial078.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T05:39:50.959 INFO:teuthology.orchestra.run.trial078.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T05:39:51.040 INFO:teuthology.orchestra.run.trial078.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T05:39:51.048 INFO:teuthology.orchestra.run.trial078.stderr:done 2026-02-17T05:39:51.050 DEBUG:teuthology.orchestra.run.trial078:> test -d /sys/firmware/efi 2026-02-17T05:39:51.065 DEBUG:teuthology.orchestra.run.trial078:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T05:39:51.129 INFO:teuthology.orchestra.run.trial078.stdout:/boot/efi/EFI/BOOT 2026-02-17T05:39:51.130 INFO:teuthology.orchestra.run.trial078.stdout:/boot/efi/EFI/centos 2026-02-17T05:39:51.131 DEBUG:teuthology.orchestra.run.trial078:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T05:39:51.195 INFO:teuthology.orchestra.run.trial078.stdout:no 2026-02-17T05:39:51.195 DEBUG:teuthology.orchestra.run.trial078:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T05:39:51.259 INFO:teuthology.orchestra.run.trial078.stdout:yes 2026-02-17T05:39:51.260 DEBUG:teuthology.orchestra.run.trial078:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T05:39:51.325 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial078 2026-02-17T05:39:51.325 DEBUG:teuthology.orchestra.run.trial078:> sudo bash -c ' 2026-02-17T05:39:51.326 DEBUG:teuthology.orchestra.run.trial078:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T05:39:51.326 DEBUG:teuthology.orchestra.run.trial078:> ' 2026-02-17T05:40:13.427 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2026-02-17T05:40:13.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial115.front.sepia.ceph.com' 2026-02-17T05:40:13.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:40:21.367 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T05:40:21.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial078.front.sepia.ceph.com' 2026-02-17T05:40:21.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:40:27.712 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.115 2026-02-17T05:40:35.648 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.78 2026-02-17T05:40:36.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial115.front.sepia.ceph.com' 2026-02-17T05:40:36.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:40:44.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial078.front.sepia.ceph.com' 2026-02-17T05:40:44.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:41:12.837 DEBUG:teuthology.orchestra.run.trial115:> true 2026-02-17T05:41:12.986 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial115.front.sepia.ceph.com' 2026-02-17T05:41:12.986 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-681.el9.x86_64"... 2026-02-17T05:41:12.986 DEBUG:teuthology.orchestra.run.trial115:> uname -r 2026-02-17T05:41:13.042 INFO:teuthology.orchestra.run.trial115.stdout:5.14.0-681.el9.x86_64 2026-02-17T05:41:13.042 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T05:41:13.042 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T05:41:13.042 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T05:41:14.043 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2026-02-17T05:41:14.044 DEBUG:teuthology.orchestra.run.trial115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T05:41:14.074 INFO:teuthology.orchestra.run.trial115.stdout:ttyS1 2026-02-17T05:41:14.100 DEBUG:teuthology.parallel:result is None 2026-02-17T05:41:20.482 DEBUG:teuthology.orchestra.run.trial078:> true 2026-02-17T05:41:20.646 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial078.front.sepia.ceph.com' 2026-02-17T05:41:20.646 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-681.el9.x86_64"... 2026-02-17T05:41:20.646 DEBUG:teuthology.orchestra.run.trial078:> uname -r 2026-02-17T05:41:20.705 INFO:teuthology.orchestra.run.trial078.stdout:5.14.0-681.el9.x86_64 2026-02-17T05:41:20.705 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T05:41:20.706 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T05:41:20.706 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T05:41:21.707 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T05:41:21.707 DEBUG:teuthology.orchestra.run.trial078:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T05:41:21.738 INFO:teuthology.orchestra.run.trial078.stdout:ttyS1 2026-02-17T05:41:21.766 DEBUG:teuthology.parallel:result is None 2026-02-17T05:41:21.766 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T05:41:21.777 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T05:41:21.778 DEBUG:teuthology.orchestra.run.trial078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T05:41:21.807 DEBUG:teuthology.orchestra.run.trial115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T05:41:21.826 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T05:41:21.838 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T05:41:21.851 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T05:41:21.852 DEBUG:teuthology.orchestra.run.trial078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T05:41:21.865 DEBUG:teuthology.orchestra.run.trial115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T05:41:21.891 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T05:41:21.902 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T05:41:21.903 DEBUG:teuthology.orchestra.run.trial078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T05:41:21.924 DEBUG:teuthology.orchestra.run.trial115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T05:41:21.946 INFO:teuthology.orchestra.run.trial078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:41:21.948 INFO:teuthology.orchestra.run.trial115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:41:21.954 INFO:teuthology.orchestra.run.trial078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:41:21.957 INFO:teuthology.orchestra.run.trial115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:41:21.959 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T05:41:21.971 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T05:41:21.971 DEBUG:teuthology.orchestra.run.trial078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T05:41:21.998 DEBUG:teuthology.orchestra.run.trial115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T05:41:22.023 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T05:41:22.040 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T05:41:22.041 DEBUG:teuthology.orchestra.run.trial078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T05:41:22.063 DEBUG:teuthology.orchestra.run.trial115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T05:41:22.079 DEBUG:teuthology.orchestra.run.trial078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:41:22.244 DEBUG:teuthology.orchestra.run.trial078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:41:22.269 DEBUG:teuthology.orchestra.run.trial078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:41:22.420 DEBUG:teuthology.orchestra.run.trial078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:41:22.444 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:41:22.444 DEBUG:teuthology.orchestra.run.trial078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T05:41:22.508 DEBUG:teuthology.orchestra.run.trial115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:41:22.626 DEBUG:teuthology.orchestra.run.trial115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:41:22.652 DEBUG:teuthology.orchestra.run.trial115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:41:22.795 DEBUG:teuthology.orchestra.run.trial115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:41:22.819 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:41:22.819 DEBUG:teuthology.orchestra.run.trial115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T05:41:22.884 DEBUG:teuthology.orchestra.run.trial078:> sudo service rsyslog restart 2026-02-17T05:41:22.886 DEBUG:teuthology.orchestra.run.trial115:> sudo service rsyslog restart 2026-02-17T05:41:22.913 INFO:teuthology.orchestra.run.trial078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T05:41:22.953 INFO:teuthology.orchestra.run.trial115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T05:41:23.475 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T05:41:23.492 INFO:teuthology.task.internal:Starting timer... 2026-02-17T05:41:23.493 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T05:41:23.515 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T05:41:23.532 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2026-02-17T05:41:23.533 DEBUG:teuthology.orchestra.run.trial078:> sudo service auditd rotate 2026-02-17T05:41:23.562 INFO:teuthology.orchestra.run.trial078.stdout:Rotating logs: 2026-02-17T05:41:23.565 DEBUG:teuthology.orchestra.run.trial115:> sudo service auditd rotate 2026-02-17T05:41:23.594 INFO:teuthology.orchestra.run.trial115.stdout:Rotating logs: 2026-02-17T05:41:23.596 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T05:41:23.596 DEBUG:teuthology.orchestra.run.trial078:> /usr/sbin/getenforce 2026-02-17T05:41:23.621 INFO:teuthology.orchestra.run.trial078.stdout:Permissive 2026-02-17T05:41:23.621 DEBUG:teuthology.orchestra.run.trial115:> /usr/sbin/getenforce 2026-02-17T05:41:23.652 INFO:teuthology.orchestra.run.trial115.stdout:Permissive 2026-02-17T05:41:23.652 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial078.front.sepia.ceph.com': 'permissive', 'ubuntu@trial115.front.sepia.ceph.com': 'permissive'} 2026-02-17T05:41:23.653 DEBUG:teuthology.orchestra.run.trial078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T05:41:23.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:41:23.686 DEBUG:teuthology.orchestra.run.trial115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T05:41:23.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:41:23.719 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T05:41:23.719 DEBUG:teuthology.orchestra.run.trial078:> sudo /usr/sbin/setenforce permissive 2026-02-17T05:41:23.749 DEBUG:teuthology.orchestra.run.trial115:> sudo /usr/sbin/setenforce permissive 2026-02-17T05:41:23.783 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T05:41:23.800 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T05:41:23.812 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T05:41:23.813 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial078.front.sepia.ceph.com,trial115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T05:44:19.544 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial078.front.sepia.ceph.com'), Remote(name='ubuntu@trial115.front.sepia.ceph.com')] 2026-02-17T05:44:19.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial078.front.sepia.ceph.com' 2026-02-17T05:44:19.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:44:19.660 DEBUG:teuthology.orchestra.run.trial078:> true 2026-02-17T05:44:19.736 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial078.front.sepia.ceph.com' 2026-02-17T05:44:19.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial115.front.sepia.ceph.com' 2026-02-17T05:44:19.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:44:19.830 DEBUG:teuthology.orchestra.run.trial115:> true 2026-02-17T05:44:19.902 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial115.front.sepia.ceph.com' 2026-02-17T05:44:19.903 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T05:44:19.912 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T05:44:19.912 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T05:44:19.913 DEBUG:teuthology.orchestra.run.trial078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T05:44:19.915 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T05:44:19.915 DEBUG:teuthology.orchestra.run.trial115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T05:44:19.942 INFO:teuthology.orchestra.run.trial078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T05:44:19.952 INFO:teuthology.orchestra.run.trial078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T05:44:19.969 INFO:teuthology.orchestra.run.trial115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T05:44:19.972 INFO:teuthology.orchestra.run.trial078.stderr:sudo: ntpd: command not found 2026-02-17T05:44:19.981 INFO:teuthology.orchestra.run.trial115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T05:44:19.982 INFO:teuthology.orchestra.run.trial078.stdout:506 Cannot talk to daemon 2026-02-17T05:44:19.993 INFO:teuthology.orchestra.run.trial078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T05:44:20.001 INFO:teuthology.orchestra.run.trial115.stderr:sudo: ntpd: command not found 2026-02-17T05:44:20.004 INFO:teuthology.orchestra.run.trial078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T05:44:20.011 INFO:teuthology.orchestra.run.trial115.stdout:506 Cannot talk to daemon 2026-02-17T05:44:20.023 INFO:teuthology.orchestra.run.trial115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T05:44:20.034 INFO:teuthology.orchestra.run.trial115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T05:44:20.050 INFO:teuthology.orchestra.run.trial078.stderr:bash: line 1: ntpq: command not found 2026-02-17T05:44:20.065 INFO:teuthology.orchestra.run.trial078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T05:44:20.065 INFO:teuthology.orchestra.run.trial078.stdout:=============================================================================== 2026-02-17T05:44:20.065 INFO:teuthology.orchestra.run.trial078.stdout:^? li1210-167.members.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:44:20.065 INFO:teuthology.orchestra.run.trial078.stdout:^? 209.205.228.50 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:44:20.065 INFO:teuthology.orchestra.run.trial078.stdout:^? time1.lshiy.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:44:20.065 INFO:teuthology.orchestra.run.trial078.stdout:^? 170-187-142-180.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:44:20.079 INFO:teuthology.orchestra.run.trial115.stderr:bash: line 1: ntpq: command not found 2026-02-17T05:44:20.099 INFO:teuthology.orchestra.run.trial115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T05:44:20.099 INFO:teuthology.orchestra.run.trial115.stdout:=============================================================================== 2026-02-17T05:44:20.099 INFO:teuthology.orchestra.run.trial115.stdout:^? lithium.constant.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:44:20.099 INFO:teuthology.orchestra.run.trial115.stdout:^? 172-232-15-202.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:44:20.099 INFO:teuthology.orchestra.run.trial115.stdout:^? 72-46-53-234.dia-static.> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:44:20.099 INFO:teuthology.orchestra.run.trial115.stdout:^? nyc3.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T05:44:20.100 INFO:teuthology.run_tasks:Running task install... 2026-02-17T05:44:20.112 DEBUG:teuthology.task.install:project ceph 2026-02-17T05:44:20.112 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'}} 2026-02-17T05:44:20.112 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'} 2026-02-17T05:44:20.112 INFO:teuthology.task.install:Using flavor: default 2026-02-17T05:44:20.116 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T05:44:20.117 INFO:teuthology.task.install:extra packages: [] 2026-02-17T05:44:20.117 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'tag': None, 'wait_for_package': False} 2026-02-17T05:44:20.118 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:44:20.120 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'tag': None, 'wait_for_package': False} 2026-02-17T05:44:20.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:44:20.293 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/ 2026-02-17T05:44:20.293 INFO:teuthology.task.install.rpm:Package version is 19.2.3-705.g04cf6dd4 2026-02-17T05:44:20.301 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/ 2026-02-17T05:44:20.302 INFO:teuthology.task.install.rpm:Package version is 19.2.3-705.g04cf6dd4 2026-02-17T05:44:20.448 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T05:44:20.448 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:44:20.448 DEBUG:teuthology.orchestra.run.trial078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T05:44:20.460 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T05:44:20.460 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:44:20.460 DEBUG:teuthology.orchestra.run.trial115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T05:44:20.473 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-17T05:44:20.473 DEBUG:teuthology.orchestra.run.trial078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T05:44:20.485 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-17T05:44:20.485 DEBUG:teuthology.orchestra.run.trial115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T05:44:20.537 DEBUG:teuthology.orchestra.run.trial078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T05:44:20.551 DEBUG:teuthology.orchestra.run.trial115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T05:44:20.612 DEBUG:teuthology.orchestra.run.trial078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T05:44:20.622 DEBUG:teuthology.orchestra.run.trial115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T05:44:20.675 INFO:teuthology.orchestra.run.trial078.stdout:check_obsoletes = 1 2026-02-17T05:44:20.677 DEBUG:teuthology.orchestra.run.trial078:> sudo yum clean all 2026-02-17T05:44:20.687 INFO:teuthology.orchestra.run.trial115.stdout:check_obsoletes = 1 2026-02-17T05:44:20.689 DEBUG:teuthology.orchestra.run.trial115:> sudo yum clean all 2026-02-17T05:44:20.826 INFO:teuthology.orchestra.run.trial078.stdout:41 files removed 2026-02-17T05:44:20.835 INFO:teuthology.orchestra.run.trial115.stdout:41 files removed 2026-02-17T05:44:20.839 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-radosgw 2026-02-17T05:44:20.848 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-radosgw 2026-02-17T05:44:21.115 INFO:teuthology.orchestra.run.trial115.stdout:ceph packages for x86_64 687 kB/s | 86 kB 00:00 2026-02-17T05:44:21.117 INFO:teuthology.orchestra.run.trial078.stdout:ceph packages for x86_64 640 kB/s | 86 kB 00:00 2026-02-17T05:44:21.246 INFO:teuthology.orchestra.run.trial078.stdout:ceph noarch packages 116 kB/s | 12 kB 00:00 2026-02-17T05:44:21.247 INFO:teuthology.orchestra.run.trial115.stdout:ceph noarch packages 112 kB/s | 12 kB 00:00 2026-02-17T05:44:21.377 INFO:teuthology.orchestra.run.trial078.stdout:ceph source packages 17 kB/s | 1.9 kB 00:00 2026-02-17T05:44:21.378 INFO:teuthology.orchestra.run.trial115.stdout:ceph source packages 17 kB/s | 1.9 kB 00:00 2026-02-17T05:44:21.642 INFO:teuthology.orchestra.run.trial115.stdout:CentOS Stream 9 - BaseOS 37 MB/s | 8.9 MB 00:00 2026-02-17T05:44:22.749 INFO:teuthology.orchestra.run.trial078.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.9 MB 00:01 2026-02-17T05:44:24.822 INFO:teuthology.orchestra.run.trial115.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-17T05:44:28.080 INFO:teuthology.orchestra.run.trial115.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 7.8 MB 00:00 2026-02-17T05:44:28.943 INFO:teuthology.orchestra.run.trial115.stdout:CentOS Stream 9 - Extras packages 139 kB/s | 20 kB 00:00 2026-02-17T05:44:29.253 INFO:teuthology.orchestra.run.trial115.stdout:Extra Packages for Enterprise Linux 82 MB/s | 20 MB 00:00 2026-02-17T05:44:31.701 INFO:teuthology.orchestra.run.trial078.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 27 MB 00:08 2026-02-17T05:44:32.475 INFO:teuthology.orchestra.run.trial115.stdout:lab-extras 1.1 MB/s | 50 kB 00:00 2026-02-17T05:44:33.687 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:44:33.688 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:44:33.688 INFO:teuthology.orchestra.run.trial115.stdout: Package Arch Version Repository Size 2026-02-17T05:44:33.688 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: ceph-radosgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout:Upgrading: 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: librados2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.4 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: librbd1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.2 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout:Installing dependencies: 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: ceph-base x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.5 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 22 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: ceph-selinux x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 25 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: libcephfs2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.0 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: libradosstriper1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 503 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: librgw2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.4 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: python3-ceph-argparse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 45 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: python3-ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 154 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: python3-cephfs x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 165 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: python3-rados x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 323 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: python3-rbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 303 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: python3-rgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 100 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout:Installing weak dependencies: 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T05:44:33.689 INFO:teuthology.orchestra.run.trial115.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T05:44:33.690 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:33.690 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:44:33.690 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:44:33.690 INFO:teuthology.orchestra.run.trial115.stdout:Install 33 Packages 2026-02-17T05:44:33.690 INFO:teuthology.orchestra.run.trial115.stdout:Upgrade 2 Packages 2026-02-17T05:44:33.690 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:33.690 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 63 M 2026-02-17T05:44:33.690 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:44:34.302 INFO:teuthology.orchestra.run.trial115.stdout:(1/35): ceph-base-19.2.3-705.g04cf6dd4.el9.x86_ 16 MB/s | 5.5 MB 00:00 2026-02-17T05:44:34.342 INFO:teuthology.orchestra.run.trial115.stdout:(2/35): ceph-selinux-19.2.3-705.g04cf6dd4.el9.x 624 kB/s | 25 kB 00:00 2026-02-17T05:44:34.403 INFO:teuthology.orchestra.run.trial115.stdout:(3/35): libcephfs2-19.2.3-705.g04cf6dd4.el9.x86 16 MB/s | 1.0 MB 00:00 2026-02-17T05:44:34.457 INFO:teuthology.orchestra.run.trial115.stdout:(4/35): libradosstriper1-19.2.3-705.g04cf6dd4.e 9.0 MB/s | 503 kB 00:00 2026-02-17T05:44:34.477 INFO:teuthology.orchestra.run.trial115.stdout:(5/35): ceph-radosgw-19.2.3-705.g04cf6dd4.el9.x 20 MB/s | 11 MB 00:00 2026-02-17T05:44:34.511 INFO:teuthology.orchestra.run.trial115.stdout:(6/35): python3-ceph-argparse-19.2.3-705.g04cf6 1.3 MB/s | 45 kB 00:00 2026-02-17T05:44:34.555 INFO:teuthology.orchestra.run.trial115.stdout:(7/35): python3-ceph-common-19.2.3-705.g04cf6dd 3.4 MB/s | 154 kB 00:00 2026-02-17T05:44:34.602 INFO:teuthology.orchestra.run.trial115.stdout:(8/35): python3-cephfs-19.2.3-705.g04cf6dd4.el9 3.5 MB/s | 165 kB 00:00 2026-02-17T05:44:34.653 INFO:teuthology.orchestra.run.trial115.stdout:(9/35): python3-rados-19.2.3-705.g04cf6dd4.el9. 6.1 MB/s | 323 kB 00:00 2026-02-17T05:44:34.678 INFO:teuthology.orchestra.run.trial115.stdout:(10/35): librgw2-19.2.3-705.g04cf6dd4.el9.x86_6 24 MB/s | 5.4 MB 00:00 2026-02-17T05:44:34.694 INFO:teuthology.orchestra.run.trial115.stdout:(11/35): python3-rbd-19.2.3-705.g04cf6dd4.el9.x 7.3 MB/s | 303 kB 00:00 2026-02-17T05:44:34.711 INFO:teuthology.orchestra.run.trial115.stdout:(12/35): python3-rgw-19.2.3-705.g04cf6dd4.el9.x 3.0 MB/s | 100 kB 00:00 2026-02-17T05:44:34.841 INFO:teuthology.orchestra.run.trial115.stdout:(13/35): ceph-common-19.2.3-705.g04cf6dd4.el9.x 24 MB/s | 22 MB 00:00 2026-02-17T05:44:34.957 INFO:teuthology.orchestra.run.trial115.stdout:(14/35): mailcap-2.1.49-5.el9.noarch.rpm 126 kB/s | 33 kB 00:00 2026-02-17T05:44:35.200 INFO:teuthology.orchestra.run.trial115.stdout:(15/35): boost-program-options-1.75.0-13.el9.x8 429 kB/s | 104 kB 00:00 2026-02-17T05:44:35.229 INFO:teuthology.orchestra.run.trial115.stdout:(16/35): pciutils-3.7.0-7.el9.x86_64.rpm 240 kB/s | 93 kB 00:00 2026-02-17T05:44:35.368 INFO:teuthology.orchestra.run.trial115.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 972 kB/s | 164 kB 00:00 2026-02-17T05:44:35.379 INFO:teuthology.orchestra.run.trial115.stdout:(18/35): nvme-cli-2.13-1.el9.x86_64.rpm 1.6 MB/s | 1.1 MB 00:00 2026-02-17T05:44:35.411 INFO:teuthology.orchestra.run.trial115.stdout:(19/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 880 kB/s | 160 kB 00:00 2026-02-17T05:44:35.454 INFO:teuthology.orchestra.run.trial115.stdout:(20/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 526 kB/s | 45 kB 00:00 2026-02-17T05:44:35.488 INFO:teuthology.orchestra.run.trial115.stdout:(21/35): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2026-02-17T05:44:35.567 INFO:teuthology.orchestra.run.trial115.stdout:(22/35): qatlib-service-25.08.0-2.el9.x86_64.rp 471 kB/s | 37 kB 00:00 2026-02-17T05:44:35.578 INFO:teuthology.orchestra.run.trial115.stdout:(23/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2026-02-17T05:44:35.579 INFO:teuthology.orchestra.run.trial115.stdout:(24/35): qatlib-25.08.0-2.el9.x86_64.rpm 1.9 MB/s | 240 kB 00:00 2026-02-17T05:44:35.647 INFO:teuthology.orchestra.run.trial115.stdout:(25/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 824 kB/s | 66 kB 00:00 2026-02-17T05:44:35.976 INFO:teuthology.orchestra.run.trial115.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 75 kB/s | 25 kB 00:00 2026-02-17T05:44:36.122 INFO:teuthology.orchestra.run.trial115.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 336 kB/s | 49 kB 00:00 2026-02-17T05:44:36.250 INFO:teuthology.orchestra.run.trial115.stdout:(28/35): libunwind-1.6.2-1.el9.x86_64.rpm 527 kB/s | 67 kB 00:00 2026-02-17T05:44:36.269 INFO:teuthology.orchestra.run.trial115.stdout:(29/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 445 kB/s | 308 kB 00:00 2026-02-17T05:44:36.456 INFO:teuthology.orchestra.run.trial115.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2026-02-17T05:44:36.527 INFO:teuthology.orchestra.run.trial078.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 7.8 MB 00:02 2026-02-17T05:44:36.956 INFO:teuthology.orchestra.run.trial115.stdout:(31/35): libarrow-9.0.0-15.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2026-02-17T05:44:37.072 INFO:teuthology.orchestra.run.trial115.stdout:(32/35): librados2-19.2.3-705.g04cf6dd4.el9.x86 30 MB/s | 3.4 MB 00:00 2026-02-17T05:44:37.131 INFO:teuthology.orchestra.run.trial115.stdout:(33/35): librbd1-19.2.3-705.g04cf6dd4.el9.x86_6 53 MB/s | 3.2 MB 00:00 2026-02-17T05:44:37.181 INFO:teuthology.orchestra.run.trial115.stdout:(34/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 900 kB/s | 838 kB 00:00 2026-02-17T05:44:37.209 INFO:teuthology.orchestra.run.trial115.stdout:(35/35): thrift-0.15.0-4.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2026-02-17T05:44:37.210 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:44:37.210 INFO:teuthology.orchestra.run.trial115.stdout:Total 18 MB/s | 63 MB 00:03 2026-02-17T05:44:37.307 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:44:37.325 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:44:37.326 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:44:37.522 INFO:teuthology.orchestra.run.trial078.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 20 kB 00:00 2026-02-17T05:44:37.563 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:44:37.563 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:44:37.793 INFO:teuthology.orchestra.run.trial078.stdout:Extra Packages for Enterprise Linux 100 MB/s | 20 MB 00:00 2026-02-17T05:44:37.854 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:44:37.862 INFO:teuthology.orchestra.run.trial115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-17T05:44:37.864 INFO:teuthology.orchestra.run.trial115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-17T05:44:37.876 INFO:teuthology.orchestra.run.trial115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-17T05:44:38.026 INFO:teuthology.orchestra.run.trial115.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-17T05:44:38.031 INFO:teuthology.orchestra.run.trial115.stdout: Upgrading : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:44:38.058 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:44:38.100 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:44:38.101 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:44:38.113 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:44:38.116 INFO:teuthology.orchestra.run.trial115.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-17T05:44:38.124 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 9/37 2026-02-17T05:44:38.147 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:44:38.147 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:44:38.159 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:44:38.191 INFO:teuthology.orchestra.run.trial115.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-17T05:44:38.195 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-17T05:44:38.201 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-17T05:44:38.203 INFO:teuthology.orchestra.run.trial115.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-17T05:44:38.206 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-17T05:44:38.211 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-17T05:44:38.351 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:44:38.352 INFO:teuthology.orchestra.run.trial115.stdout: Upgrading : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:44:38.380 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:44:38.382 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 20/37 2026-02-17T05:44:38.389 INFO:teuthology.orchestra.run.trial115.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-17T05:44:38.615 INFO:teuthology.orchestra.run.trial115.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-17T05:44:38.616 INFO:teuthology.orchestra.run.trial115.stdout: Installing : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:44:38.632 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:44:38.636 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 24/37 2026-02-17T05:44:38.640 INFO:teuthology.orchestra.run.trial115.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-17T05:44:38.642 INFO:teuthology.orchestra.run.trial115.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-17T05:44:38.643 INFO:teuthology.orchestra.run.trial115.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:44:38.661 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:44:38.773 INFO:teuthology.orchestra.run.trial115.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-17T05:44:38.775 INFO:teuthology.orchestra.run.trial115.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:44:39.054 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:44:39.054 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T05:44:39.054 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:39.091 INFO:teuthology.orchestra.run.trial115.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-17T05:44:39.093 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 31/37 2026-02-17T05:44:40.076 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:44:40.078 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:44:40.360 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:44:40.363 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:44:40.402 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:44:40.402 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T05:44:40.402 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T05:44:40.402 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:40.406 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:44:40.971 INFO:teuthology.orchestra.run.trial078.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-17T05:44:42.189 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:44:42.191 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:44:42.191 INFO:teuthology.orchestra.run.trial078.stdout: Package Arch Version Repository Size 2026-02-17T05:44:42.191 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:44:42.191 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:44:42.191 INFO:teuthology.orchestra.run.trial078.stdout: ceph-radosgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 11 M 2026-02-17T05:44:42.191 INFO:teuthology.orchestra.run.trial078.stdout:Upgrading: 2026-02-17T05:44:42.191 INFO:teuthology.orchestra.run.trial078.stdout: librados2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.4 M 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: librbd1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.2 M 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout:Installing dependencies: 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: ceph-base x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.5 M 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 22 M 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: ceph-selinux x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 25 k 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: libcephfs2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.0 M 2026-02-17T05:44:42.192 INFO:teuthology.orchestra.run.trial078.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: libradosstriper1 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 503 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: librgw2 x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 5.4 M 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T05:44:42.193 INFO:teuthology.orchestra.run.trial078.stdout: python3-ceph-argparse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 45 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: python3-ceph-common x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 154 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: python3-cephfs x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 165 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: python3-rados x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 323 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: python3-rbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 303 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: python3-rgw x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 100 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout:Installing weak dependencies: 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T05:44:42.194 INFO:teuthology.orchestra.run.trial078.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T05:44:42.195 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:42.195 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:44:42.195 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:44:42.195 INFO:teuthology.orchestra.run.trial078.stdout:Install 33 Packages 2026-02-17T05:44:42.195 INFO:teuthology.orchestra.run.trial078.stdout:Upgrade 2 Packages 2026-02-17T05:44:42.195 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:42.196 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 63 M 2026-02-17T05:44:42.196 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:44:43.055 INFO:teuthology.orchestra.run.trial078.stdout:(1/35): ceph-base-19.2.3-705.g04cf6dd4.el9.x86_ 17 MB/s | 5.5 MB 00:00 2026-02-17T05:44:43.091 INFO:teuthology.orchestra.run.trial078.stdout:(2/35): ceph-selinux-19.2.3-705.g04cf6dd4.el9.x 685 kB/s | 25 kB 00:00 2026-02-17T05:44:43.149 INFO:teuthology.orchestra.run.trial078.stdout:(3/35): libcephfs2-19.2.3-705.g04cf6dd4.el9.x86 17 MB/s | 1.0 MB 00:00 2026-02-17T05:44:43.198 INFO:teuthology.orchestra.run.trial078.stdout:(4/35): libradosstriper1-19.2.3-705.g04cf6dd4.e 9.9 MB/s | 503 kB 00:00 2026-02-17T05:44:43.424 INFO:teuthology.orchestra.run.trial078.stdout:(5/35): ceph-radosgw-19.2.3-705.g04cf6dd4.el9.x 15 MB/s | 11 MB 00:00 2026-02-17T05:44:43.467 INFO:teuthology.orchestra.run.trial078.stdout:(6/35): python3-ceph-argparse-19.2.3-705.g04cf6 1.0 MB/s | 45 kB 00:00 2026-02-17T05:44:43.478 INFO:teuthology.orchestra.run.trial078.stdout:(7/35): librgw2-19.2.3-705.g04cf6dd4.el9.x86_64 19 MB/s | 5.4 MB 00:00 2026-02-17T05:44:43.489 INFO:teuthology.orchestra.run.trial078.stdout:(8/35): python3-ceph-common-19.2.3-705.g04cf6dd 6.8 MB/s | 154 kB 00:00 2026-02-17T05:44:43.503 INFO:teuthology.orchestra.run.trial078.stdout:(9/35): python3-cephfs-19.2.3-705.g04cf6dd4.el9 6.5 MB/s | 165 kB 00:00 2026-02-17T05:44:43.516 INFO:teuthology.orchestra.run.trial078.stdout:(10/35): python3-rados-19.2.3-705.g04cf6dd4.el9 12 MB/s | 323 kB 00:00 2026-02-17T05:44:43.532 INFO:teuthology.orchestra.run.trial078.stdout:(11/35): python3-rbd-19.2.3-705.g04cf6dd4.el9.x 10 MB/s | 303 kB 00:00 2026-02-17T05:44:43.541 INFO:teuthology.orchestra.run.trial078.stdout:(12/35): python3-rgw-19.2.3-705.g04cf6dd4.el9.x 3.9 MB/s | 100 kB 00:00 2026-02-17T05:44:43.559 INFO:teuthology.orchestra.run.trial078.stdout:(13/35): mailcap-2.1.49-5.el9.noarch.rpm 1.2 MB/s | 33 kB 00:00 2026-02-17T05:44:43.567 INFO:teuthology.orchestra.run.trial078.stdout:(14/35): pciutils-3.7.0-7.el9.x86_64.rpm 11 MB/s | 93 kB 00:00 2026-02-17T05:44:43.577 INFO:teuthology.orchestra.run.trial078.stdout:(15/35): nvme-cli-2.13-1.el9.x86_64.rpm 30 MB/s | 1.1 MB 00:00 2026-02-17T05:44:43.623 INFO:teuthology.orchestra.run.trial078.stdout:(16/35): ceph-common-19.2.3-705.g04cf6dd4.el9.x 24 MB/s | 22 MB 00:00 2026-02-17T05:44:43.656 INFO:teuthology.orchestra.run.trial078.stdout:(17/35): boost-program-options-1.75.0-13.el9.x8 1.2 MB/s | 104 kB 00:00 2026-02-17T05:44:43.678 INFO:teuthology.orchestra.run.trial078.stdout:(18/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-17T05:44:43.687 INFO:teuthology.orchestra.run.trial078.stdout:(19/35): libnbd-1.20.3-4.el9.x86_64.rpm 1.5 MB/s | 164 kB 00:00 2026-02-17T05:44:43.708 INFO:teuthology.orchestra.run.trial078.stdout:(20/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.8 MB/s | 160 kB 00:00 2026-02-17T05:44:43.734 INFO:teuthology.orchestra.run.trial078.stdout:(21/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.1 MB/s | 292 kB 00:00 2026-02-17T05:44:43.742 INFO:teuthology.orchestra.run.trial078.stdout:(22/35): librdkafka-1.6.1-102.el9.x86_64.rpm 10 MB/s | 662 kB 00:00 2026-02-17T05:44:43.755 INFO:teuthology.orchestra.run.trial078.stdout:(23/35): qatlib-25.08.0-2.el9.x86_64.rpm 5.0 MB/s | 240 kB 00:00 2026-02-17T05:44:43.756 INFO:teuthology.orchestra.run.trial078.stdout:(24/35): qatlib-service-25.08.0-2.el9.x86_64.rp 1.7 MB/s | 37 kB 00:00 2026-02-17T05:44:43.765 INFO:teuthology.orchestra.run.trial078.stdout:(25/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.8 MB/s | 66 kB 00:00 2026-02-17T05:44:43.787 INFO:teuthology.orchestra.run.trial078.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.1 MB/s | 25 kB 00:00 2026-02-17T05:44:43.792 INFO:teuthology.orchestra.run.trial078.stdout:(27/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 8.3 MB/s | 308 kB 00:00 2026-02-17T05:44:43.795 INFO:teuthology.orchestra.run.trial078.stdout:(28/35): liboath-2.6.12-1.el9.x86_64.rpm 6.6 MB/s | 49 kB 00:00 2026-02-17T05:44:43.796 INFO:teuthology.orchestra.run.trial078.stdout:(29/35): libunwind-1.6.2-1.el9.x86_64.rpm 18 MB/s | 67 kB 00:00 2026-02-17T05:44:43.805 INFO:teuthology.orchestra.run.trial078.stdout:(30/35): libarrow-9.0.0-15.el9.x86_64.rpm 89 MB/s | 4.4 MB 00:00 2026-02-17T05:44:43.807 INFO:teuthology.orchestra.run.trial078.stdout:(31/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 66 MB/s | 838 kB 00:00 2026-02-17T05:44:43.808 INFO:teuthology.orchestra.run.trial078.stdout:(32/35): re2-20211101-20.el9.x86_64.rpm 16 MB/s | 191 kB 00:00 2026-02-17T05:44:43.814 INFO:teuthology.orchestra.run.trial078.stdout:(33/35): thrift-0.15.0-4.el9.x86_64.rpm 195 MB/s | 1.6 MB 00:00 2026-02-17T05:44:43.918 INFO:teuthology.orchestra.run.trial078.stdout:(34/35): librados2-19.2.3-705.g04cf6dd4.el9.x86 31 MB/s | 3.4 MB 00:00 2026-02-17T05:44:43.926 INFO:teuthology.orchestra.run.trial078.stdout:(35/35): librbd1-19.2.3-705.g04cf6dd4.el9.x86_6 27 MB/s | 3.2 MB 00:00 2026-02-17T05:44:43.929 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:44:43.929 INFO:teuthology.orchestra.run.trial078.stdout:Total 36 MB/s | 63 MB 00:01 2026-02-17T05:44:44.033 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:44:44.052 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:44:44.052 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:44:44.289 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:44:44.290 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:44:44.582 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:44:44.590 INFO:teuthology.orchestra.run.trial078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-17T05:44:44.592 INFO:teuthology.orchestra.run.trial078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-17T05:44:44.604 INFO:teuthology.orchestra.run.trial078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-17T05:44:44.752 INFO:teuthology.orchestra.run.trial078.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-17T05:44:44.756 INFO:teuthology.orchestra.run.trial078.stdout: Upgrading : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:44:44.783 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:44:44.825 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:44:44.826 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:44:44.838 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:44:44.842 INFO:teuthology.orchestra.run.trial078.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-17T05:44:44.852 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 9/37 2026-02-17T05:44:44.874 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:44:44.875 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:44:44.889 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:44:44.919 INFO:teuthology.orchestra.run.trial078.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-17T05:44:44.922 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-17T05:44:44.929 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-17T05:44:44.931 INFO:teuthology.orchestra.run.trial078.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-17T05:44:44.934 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-17T05:44:44.938 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-17T05:44:44.950 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:44:44.951 INFO:teuthology.orchestra.run.trial115.stdout:skipping the directory /sys 2026-02-17T05:44:44.951 INFO:teuthology.orchestra.run.trial115.stdout:skipping the directory /proc 2026-02-17T05:44:44.951 INFO:teuthology.orchestra.run.trial115.stdout:skipping the directory /mnt 2026-02-17T05:44:44.951 INFO:teuthology.orchestra.run.trial115.stdout:skipping the directory /var/tmp 2026-02-17T05:44:44.951 INFO:teuthology.orchestra.run.trial115.stdout:skipping the directory /home 2026-02-17T05:44:44.951 INFO:teuthology.orchestra.run.trial115.stdout:skipping the directory /root 2026-02-17T05:44:44.951 INFO:teuthology.orchestra.run.trial115.stdout:skipping the directory /tmp 2026-02-17T05:44:44.951 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:45.078 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:44:45.078 INFO:teuthology.orchestra.run.trial078.stdout: Upgrading : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:44:45.103 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 19/37 2026-02-17T05:44:45.107 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 20/37 2026-02-17T05:44:45.113 INFO:teuthology.orchestra.run.trial078.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-17T05:44:45.337 INFO:teuthology.orchestra.run.trial078.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-17T05:44:45.338 INFO:teuthology.orchestra.run.trial078.stdout: Installing : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:44:45.355 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 23/37 2026-02-17T05:44:45.359 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 24/37 2026-02-17T05:44:45.364 INFO:teuthology.orchestra.run.trial078.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-17T05:44:45.367 INFO:teuthology.orchestra.run.trial078.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-17T05:44:45.367 INFO:teuthology.orchestra.run.trial078.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:44:45.387 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T05:44:45.407 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:44:45.431 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:44:45.431 INFO:teuthology.orchestra.run.trial115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:44:45.431 INFO:teuthology.orchestra.run.trial115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T05:44:45.431 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:44:45.431 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:44:45.431 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:45.432 INFO:teuthology.orchestra.run.trial115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:44:45.444 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:44:45.444 INFO:teuthology.orchestra.run.trial115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:44:45.497 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:44:45.498 INFO:teuthology.orchestra.run.trial115.stdout:Creating group 'qat' with GID 992. 2026-02-17T05:44:45.498 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:45.498 INFO:teuthology.orchestra.run.trial078.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-17T05:44:45.500 INFO:teuthology.orchestra.run.trial078.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:44:45.771 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 8/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 9/37 2026-02-17T05:44:45.772 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 12/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 13/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 15/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-17T05:44:45.773 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:44:45.774 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-17T05:44:45.775 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 36/37 2026-02-17T05:44:45.779 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T05:44:45.779 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T05:44:45.779 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:45.819 INFO:teuthology.orchestra.run.trial078.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-17T05:44:45.821 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 31/37 2026-02-17T05:44:45.831 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout:Upgraded: 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.832 INFO:teuthology.orchestra.run.trial115.stdout: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.833 INFO:teuthology.orchestra.run.trial115.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T05:44:45.833 INFO:teuthology.orchestra.run.trial115.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T05:44:45.833 INFO:teuthology.orchestra.run.trial115.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T05:44:45.833 INFO:teuthology.orchestra.run.trial115.stdout: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.833 INFO:teuthology.orchestra.run.trial115.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T05:44:45.833 INFO:teuthology.orchestra.run.trial115.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T05:44:45.833 INFO:teuthology.orchestra.run.trial115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T05:44:45.833 INFO:teuthology.orchestra.run.trial115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.834 INFO:teuthology.orchestra.run.trial115.stdout: python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:45.835 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:44:45.873 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-test 2026-02-17T05:44:46.190 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:14 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:44:46.272 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:44:46.273 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:44:46.273 INFO:teuthology.orchestra.run.trial115.stdout: Package Arch Version Repository Size 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout: ceph-test x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 50 M 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout:Installing dependencies: 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout: libcephsqlite x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 163 k 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:44:46.274 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:44:46.275 INFO:teuthology.orchestra.run.trial115.stdout:Install 5 Packages 2026-02-17T05:44:46.275 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:46.275 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 50 M 2026-02-17T05:44:46.275 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 212 M 2026-02-17T05:44:46.275 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:44:46.446 INFO:teuthology.orchestra.run.trial115.stdout:(1/5): libcephsqlite-19.2.3-705.g04cf6dd4.el9.x 1.9 MB/s | 163 kB 00:00 2026-02-17T05:44:46.803 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:44:46.806 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:44:47.088 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 32/37 2026-02-17T05:44:47.091 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:44:47.104 INFO:teuthology.orchestra.run.trial115.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 314 kB/s | 233 kB 00:00 2026-02-17T05:44:47.131 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 33/37 2026-02-17T05:44:47.131 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T05:44:47.131 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T05:44:47.131 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:47.135 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:44:47.271 INFO:teuthology.orchestra.run.trial115.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 383 kB/s | 64 kB 00:00 2026-02-17T05:44:47.275 INFO:teuthology.orchestra.run.trial115.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 366 kB/s | 303 kB 00:00 2026-02-17T05:44:47.398 INFO:teuthology.orchestra.run.trial115.stdout:(5/5): ceph-test-19.2.3-705.g04cf6dd4.el9.x86_6 48 MB/s | 50 MB 00:01 2026-02-17T05:44:47.399 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:44:47.399 INFO:teuthology.orchestra.run.trial115.stdout:Total 45 MB/s | 50 MB 00:01 2026-02-17T05:44:47.426 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:44:47.436 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:44:47.436 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:44:47.623 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:44:47.623 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:44:47.835 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:44:47.840 INFO:teuthology.orchestra.run.trial115.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-17T05:44:47.844 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-17T05:44:47.852 INFO:teuthology.orchestra.run.trial115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-17T05:44:47.853 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:44:50.149 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:44:50.167 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:44:50.393 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:44:50.394 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/5 2026-02-17T05:44:50.394 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/5 2026-02-17T05:44:50.394 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-17T05:44:50.394 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-17T05:44:50.447 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-17T05:44:50.447 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:50.447 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:44:50.447 INFO:teuthology.orchestra.run.trial115.stdout: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:50.447 INFO:teuthology.orchestra.run.trial115.stdout: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:50.447 INFO:teuthology.orchestra.run.trial115.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T05:44:50.448 INFO:teuthology.orchestra.run.trial115.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T05:44:50.448 INFO:teuthology.orchestra.run.trial115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T05:44:50.448 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:50.448 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:44:50.483 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph 2026-02-17T05:44:50.791 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:18 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:44:50.876 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:44:50.879 INFO:teuthology.orchestra.run.trial115.stdout:====================================================================================== 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: Package Arch Version Repository Size 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout:====================================================================================== 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ceph x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.5 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout:Installing dependencies: 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ceph-grafana-dashboards noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 31 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mds x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 2.4 M 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-modules-core noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 253 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mon x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 4.7 M 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ceph-osd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 17 M 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ceph-prometheus-alerts noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 17 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 769 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T05:44:50.880 INFO:teuthology.orchestra.run.trial115.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T05:44:50.881 INFO:teuthology.orchestra.run.trial115.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T05:44:50.882 INFO:teuthology.orchestra.run.trial115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-17T05:44:50.883 INFO:teuthology.orchestra.run.trial115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout:Installing weak dependencies: 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 150 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-dashboard noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 3.8 M 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 7.4 M 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 22 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-rook noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 49 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: ceph-volume noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 299 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout:====================================================================================== 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout:Install 99 Packages 2026-02-17T05:44:50.884 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:44:50.885 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 95 M 2026-02-17T05:44:50.885 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 520 M 2026-02-17T05:44:50.885 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:44:51.627 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:44:51.627 INFO:teuthology.orchestra.run.trial078.stdout:skipping the directory /sys 2026-02-17T05:44:51.627 INFO:teuthology.orchestra.run.trial078.stdout:skipping the directory /proc 2026-02-17T05:44:51.627 INFO:teuthology.orchestra.run.trial078.stdout:skipping the directory /mnt 2026-02-17T05:44:51.627 INFO:teuthology.orchestra.run.trial078.stdout:skipping the directory /var/tmp 2026-02-17T05:44:51.627 INFO:teuthology.orchestra.run.trial078.stdout:skipping the directory /home 2026-02-17T05:44:51.628 INFO:teuthology.orchestra.run.trial078.stdout:skipping the directory /root 2026-02-17T05:44:51.628 INFO:teuthology.orchestra.run.trial078.stdout:skipping the directory /tmp 2026-02-17T05:44:51.628 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:51.744 INFO:teuthology.orchestra.run.trial115.stdout:(1/99): ceph-19.2.3-705.g04cf6dd4.el9.x86_64.rp 56 kB/s | 6.5 kB 00:00 2026-02-17T05:44:51.879 INFO:teuthology.orchestra.run.trial115.stdout:(2/99): ceph-mds-19.2.3-705.g04cf6dd4.el9.x86_6 9.7 MB/s | 2.4 MB 00:00 2026-02-17T05:44:52.073 INFO:teuthology.orchestra.run.trial115.stdout:(3/99): ceph-mon-19.2.3-705.g04cf6dd4.el9.x86_6 14 MB/s | 4.7 MB 00:00 2026-02-17T05:44:52.081 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:44:52.104 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 35/37 2026-02-17T05:44:52.104 INFO:teuthology.orchestra.run.trial078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:44:52.104 INFO:teuthology.orchestra.run.trial078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T05:44:52.104 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:44:52.104 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:44:52.104 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:52.106 INFO:teuthology.orchestra.run.trial078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:44:52.113 INFO:teuthology.orchestra.run.trial115.stdout:(4/99): ceph-grafana-dashboards-19.2.3-705.g04c 785 kB/s | 31 kB 00:00 2026-02-17T05:44:52.119 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T05:44:52.119 INFO:teuthology.orchestra.run.trial078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:44:52.158 INFO:teuthology.orchestra.run.trial115.stdout:(5/99): ceph-mgr-cephadm-19.2.3-705.g04cf6dd4.e 3.3 MB/s | 150 kB 00:00 2026-02-17T05:44:52.173 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:44:52.173 INFO:teuthology.orchestra.run.trial078.stdout:Creating group 'qat' with GID 992. 2026-02-17T05:44:52.173 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:52.264 INFO:teuthology.orchestra.run.trial115.stdout:(6/99): ceph-mgr-19.2.3-705.g04cf6dd4.el9.x86_6 1.7 MB/s | 1.1 MB 00:00 2026-02-17T05:44:52.341 INFO:teuthology.orchestra.run.trial115.stdout:(7/99): ceph-osd-19.2.3-705.g04cf6dd4.el9.x86_6 37 MB/s | 17 MB 00:00 2026-02-17T05:44:52.353 INFO:teuthology.orchestra.run.trial115.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-705.g04cf6dd4 1.9 MB/s | 22 kB 00:00 2026-02-17T05:44:52.357 INFO:teuthology.orchestra.run.trial115.stdout:(9/99): ceph-mgr-dashboard-19.2.3-705.g04cf6dd4 19 MB/s | 3.8 MB 00:00 2026-02-17T05:44:52.365 INFO:teuthology.orchestra.run.trial115.stdout:(10/99): ceph-mgr-modules-core-19.2.3-705.g04cf 21 MB/s | 253 kB 00:00 2026-02-17T05:44:52.369 INFO:teuthology.orchestra.run.trial115.stdout:(11/99): ceph-mgr-rook-19.2.3-705.g04cf6dd4.el9 4.3 MB/s | 49 kB 00:00 2026-02-17T05:44:52.376 INFO:teuthology.orchestra.run.trial115.stdout:(12/99): ceph-prometheus-alerts-19.2.3-705.g04c 1.5 MB/s | 17 kB 00:00 2026-02-17T05:44:52.381 INFO:teuthology.orchestra.run.trial115.stdout:(13/99): ceph-volume-19.2.3-705.g04cf6dd4.el9.n 24 MB/s | 299 kB 00:00 2026-02-17T05:44:52.393 INFO:teuthology.orchestra.run.trial115.stdout:(14/99): cephadm-19.2.3-705.g04cf6dd4.el9.noarc 46 MB/s | 769 kB 00:00 2026-02-17T05:44:52.444 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/37 2026-02-17T05:44:52.444 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 6/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 7/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x 8/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86 9/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 10/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 11/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 12/37 2026-02-17T05:44:52.445 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 13/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 15/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-17T05:44:52.446 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 34/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-17T05:44:52.447 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 36/37 2026-02-17T05:44:52.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T05:44:52.504 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout:Upgraded: 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: ceph-radosgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: ceph-selinux-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T05:44:52.505 INFO:teuthology.orchestra.run.trial078.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: libradosstriper1-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: librgw2-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T05:44:52.506 INFO:teuthology.orchestra.run.trial078.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: python3-ceph-argparse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: python3-ceph-common-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T05:44:52.507 INFO:teuthology.orchestra.run.trial078.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T05:44:52.508 INFO:teuthology.orchestra.run.trial078.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T05:44:52.508 INFO:teuthology.orchestra.run.trial078.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T05:44:52.508 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:52.508 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:44:52.547 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-test 2026-02-17T05:44:52.657 INFO:teuthology.orchestra.run.trial115.stdout:(15/99): ceph-mgr-diskprediction-local-19.2.3-7 19 MB/s | 7.4 MB 00:00 2026-02-17T05:44:52.670 INFO:teuthology.orchestra.run.trial115.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 146 kB/s | 40 kB 00:00 2026-02-17T05:44:52.850 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:12 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:44:52.928 INFO:teuthology.orchestra.run.trial115.stdout:(17/99): cryptsetup-2.8.1-2.el9.x86_64.rpm 640 kB/s | 350 kB 00:00 2026-02-17T05:44:52.933 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:44:52.933 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:44:52.933 INFO:teuthology.orchestra.run.trial078.stdout: Package Arch Version Repository Size 2026-02-17T05:44:52.933 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:44:52.933 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:44:52.933 INFO:teuthology.orchestra.run.trial078.stdout: ceph-test x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 50 M 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout:Installing dependencies: 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout: libcephsqlite x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 163 k 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout:Install 5 Packages 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:52.934 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 50 M 2026-02-17T05:44:52.935 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 212 M 2026-02-17T05:44:52.935 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:44:53.012 INFO:teuthology.orchestra.run.trial115.stdout:(18/99): libconfig-1.7.2-9.el9.x86_64.rpm 203 kB/s | 72 kB 00:00 2026-02-17T05:44:53.034 INFO:teuthology.orchestra.run.trial115.stdout:(19/99): libquadmath-11.5.0-14.el9.x86_64.rpm 1.7 MB/s | 184 kB 00:00 2026-02-17T05:44:53.104 INFO:teuthology.orchestra.run.trial078.stdout:(1/5): libcephsqlite-19.2.3-705.g04cf6dd4.el9.x 1.9 MB/s | 163 kB 00:00 2026-02-17T05:44:53.105 INFO:teuthology.orchestra.run.trial115.stdout:(20/99): libgfortran-11.5.0-14.el9.x86_64.rpm 1.8 MB/s | 794 kB 00:00 2026-02-17T05:44:53.108 INFO:teuthology.orchestra.run.trial078.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 2.6 MB/s | 233 kB 00:00 2026-02-17T05:44:53.131 INFO:teuthology.orchestra.run.trial078.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.8 MB/s | 64 kB 00:00 2026-02-17T05:44:53.189 INFO:teuthology.orchestra.run.trial115.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2026-02-17T05:44:53.208 INFO:teuthology.orchestra.run.trial078.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 2.8 MB/s | 303 kB 00:00 2026-02-17T05:44:53.265 INFO:teuthology.orchestra.run.trial115.stdout:(22/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2026-02-17T05:44:53.276 INFO:teuthology.orchestra.run.trial115.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 1.5 MB/s | 135 kB 00:00 2026-02-17T05:44:53.281 INFO:teuthology.orchestra.run.trial115.stdout:(24/99): python3-cryptography-36.0.1-5.el9.x86_ 5.1 MB/s | 1.2 MB 00:00 2026-02-17T05:44:53.361 INFO:teuthology.orchestra.run.trial115.stdout:(25/99): python3-requests-2.25.1-10.el9.noarch. 1.4 MB/s | 126 kB 00:00 2026-02-17T05:44:53.368 INFO:teuthology.orchestra.run.trial115.stdout:(26/99): python3-urllib3-1.26.5-7.el9.noarch.rp 2.5 MB/s | 218 kB 00:00 2026-02-17T05:44:53.371 INFO:teuthology.orchestra.run.trial115.stdout:(27/99): python3-pyparsing-2.4.7-9.el9.noarch.r 1.4 MB/s | 150 kB 00:00 2026-02-17T05:44:53.437 INFO:teuthology.orchestra.run.trial115.stdout:(28/99): flexiblas-3.0.4-9.el9.x86_64.rpm 451 kB/s | 30 kB 00:00 2026-02-17T05:44:53.450 INFO:teuthology.orchestra.run.trial115.stdout:(29/99): unzip-6.0-59.el9.x86_64.rpm 2.0 MB/s | 182 kB 00:00 2026-02-17T05:44:53.456 INFO:teuthology.orchestra.run.trial115.stdout:(30/99): zip-3.0-35.el9.x86_64.rpm 2.9 MB/s | 266 kB 00:00 2026-02-17T05:44:53.508 INFO:teuthology.orchestra.run.trial115.stdout:(31/99): flexiblas-openblas-openmp-3.0.4-9.el9. 252 kB/s | 15 kB 00:00 2026-02-17T05:44:53.571 INFO:teuthology.orchestra.run.trial115.stdout:(32/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 22 MB/s | 3.0 MB 00:00 2026-02-17T05:44:53.572 INFO:teuthology.orchestra.run.trial115.stdout:(33/99): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 188 kB 00:00 2026-02-17T05:44:53.588 INFO:teuthology.orchestra.run.trial115.stdout:(34/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 1.8 MB/s | 246 kB 00:00 2026-02-17T05:44:54.048 INFO:teuthology.orchestra.run.trial115.stdout:(35/99): openblas-0.3.29-1.el9.x86_64.rpm 88 kB/s | 42 kB 00:00 2026-02-17T05:44:54.133 INFO:teuthology.orchestra.run.trial115.stdout:(36/99): python3-babel-2.9.1-2.el9.noarch.rpm 70 MB/s | 6.0 MB 00:00 2026-02-17T05:44:54.144 INFO:teuthology.orchestra.run.trial115.stdout:(37/99): protobuf-3.14.0-17.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2026-02-17T05:44:54.156 INFO:teuthology.orchestra.run.trial115.stdout:(38/99): python3-devel-3.9.25-2.el9.x86_64.rpm 10 MB/s | 244 kB 00:00 2026-02-17T05:44:54.170 INFO:teuthology.orchestra.run.trial115.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 9.5 MB/s | 249 kB 00:00 2026-02-17T05:44:54.178 INFO:teuthology.orchestra.run.trial115.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 7.8 MB/s | 177 kB 00:00 2026-02-17T05:44:54.204 INFO:teuthology.orchestra.run.trial115.stdout:(41/99): python3-mako-1.1.4-6.el9.noarch.rpm 6.8 MB/s | 172 kB 00:00 2026-02-17T05:44:54.224 INFO:teuthology.orchestra.run.trial115.stdout:(42/99): python3-markupsafe-1.1.1-12.el9.x86_64 1.7 MB/s | 35 kB 00:00 2026-02-17T05:44:54.226 INFO:teuthology.orchestra.run.trial115.stdout:(43/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 22 MB/s | 1.2 MB 00:00 2026-02-17T05:44:54.254 INFO:teuthology.orchestra.run.trial115.stdout:(44/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 16 MB/s | 442 kB 00:00 2026-02-17T05:44:54.278 INFO:teuthology.orchestra.run.trial115.stdout:(45/99): python3-packaging-20.9-5.el9.noarch.rp 3.0 MB/s | 77 kB 00:00 2026-02-17T05:44:54.288 INFO:teuthology.orchestra.run.trial115.stdout:(46/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 7.4 MB/s | 5.3 MB 00:00 2026-02-17T05:44:54.296 INFO:teuthology.orchestra.run.trial115.stdout:(47/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 85 MB/s | 6.1 MB 00:00 2026-02-17T05:44:54.303 INFO:teuthology.orchestra.run.trial115.stdout:(48/99): python3-protobuf-3.14.0-17.el9.noarch. 10 MB/s | 267 kB 00:00 2026-02-17T05:44:54.311 INFO:teuthology.orchestra.run.trial115.stdout:(49/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 6.8 MB/s | 159 kB 00:00 2026-02-17T05:44:54.319 INFO:teuthology.orchestra.run.trial115.stdout:(50/99): python3-pyasn1-modules-0.4.8-6.el9.noa 11 MB/s | 279 kB 00:00 2026-02-17T05:44:54.328 INFO:teuthology.orchestra.run.trial115.stdout:(51/99): python3-requests-oauthlib-1.3.0-12.el9 2.2 MB/s | 54 kB 00:00 2026-02-17T05:44:54.341 INFO:teuthology.orchestra.run.trial115.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 1.9 MB/s | 42 kB 00:00 2026-02-17T05:44:54.352 INFO:teuthology.orchestra.run.trial115.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 7.6 MB/s | 189 kB 00:00 2026-02-17T05:44:54.363 INFO:teuthology.orchestra.run.trial115.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 4.1 MB/s | 90 kB 00:00 2026-02-17T05:44:54.553 INFO:teuthology.orchestra.run.trial078.stdout:(5/5): ceph-test-19.2.3-705.g04cf6dd4.el9.x86_6 32 MB/s | 50 MB 00:01 2026-02-17T05:44:54.555 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:44:54.555 INFO:teuthology.orchestra.run.trial078.stdout:Total 31 MB/s | 50 MB 00:01 2026-02-17T05:44:54.580 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:44:54.589 INFO:teuthology.orchestra.run.trial115.stdout:(55/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 69 MB/s | 19 MB 00:00 2026-02-17T05:44:54.590 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:44:54.590 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:44:54.627 INFO:teuthology.orchestra.run.trial115.stdout:(56/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 15 MB/s | 551 kB 00:00 2026-02-17T05:44:54.632 INFO:teuthology.orchestra.run.trial115.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 4.2 MB/s | 19 kB 00:00 2026-02-17T05:44:54.637 INFO:teuthology.orchestra.run.trial115.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 25 MB/s | 151 kB 00:00 2026-02-17T05:44:54.646 INFO:teuthology.orchestra.run.trial115.stdout:(59/99): python3-asyncssh-2.13.2-5.el9.noarch.r 65 MB/s | 548 kB 00:00 2026-02-17T05:44:54.651 INFO:teuthology.orchestra.run.trial115.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 5.9 MB/s | 29 kB 00:00 2026-02-17T05:44:54.656 INFO:teuthology.orchestra.run.trial115.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 11 MB/s | 60 kB 00:00 2026-02-17T05:44:54.661 INFO:teuthology.orchestra.run.trial115.stdout:(62/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 8.5 MB/s | 43 kB 00:00 2026-02-17T05:44:54.666 INFO:teuthology.orchestra.run.trial115.stdout:(63/99): python3-cachetools-4.2.4-1.el9.noarch. 6.1 MB/s | 32 kB 00:00 2026-02-17T05:44:54.671 INFO:teuthology.orchestra.run.trial115.stdout:(64/99): python3-certifi-2023.05.07-4.el9.noarc 3.0 MB/s | 14 kB 00:00 2026-02-17T05:44:54.678 INFO:teuthology.orchestra.run.trial115.stdout:(65/99): python3-cheroot-10.0.1-4.el9.noarch.rp 25 MB/s | 173 kB 00:00 2026-02-17T05:44:54.690 INFO:teuthology.orchestra.run.trial115.stdout:(66/99): python3-cherrypy-18.6.1-2.el9.noarch.r 31 MB/s | 358 kB 00:00 2026-02-17T05:44:54.697 INFO:teuthology.orchestra.run.trial115.stdout:(67/99): python3-google-auth-2.45.0-1.el9.noarc 35 MB/s | 254 kB 00:00 2026-02-17T05:44:54.743 INFO:teuthology.orchestra.run.trial115.stdout:(68/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 44 MB/s | 2.0 MB 00:00 2026-02-17T05:44:54.750 INFO:teuthology.orchestra.run.trial115.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 21 MB/s | 144 kB 00:00 2026-02-17T05:44:54.757 INFO:teuthology.orchestra.run.trial115.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 19 MB/s | 139 kB 00:00 2026-02-17T05:44:54.763 INFO:teuthology.orchestra.run.trial115.stdout:(71/99): python3-isodate-0.6.1-3.el9.noarch.rpm 10 MB/s | 56 kB 00:00 2026-02-17T05:44:54.767 INFO:teuthology.orchestra.run.trial115.stdout:(72/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 2.3 MB/s | 11 kB 00:00 2026-02-17T05:44:54.772 INFO:teuthology.orchestra.run.trial115.stdout:(73/99): python3-jaraco-classes-3.2.1-5.el9.noa 3.6 MB/s | 18 kB 00:00 2026-02-17T05:44:54.775 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:44:54.775 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:44:54.777 INFO:teuthology.orchestra.run.trial115.stdout:(74/99): python3-jaraco-collections-3.0.0-8.el9 4.6 MB/s | 23 kB 00:00 2026-02-17T05:44:54.782 INFO:teuthology.orchestra.run.trial115.stdout:(75/99): python3-jaraco-context-6.0.1-3.el9.noa 4.0 MB/s | 20 kB 00:00 2026-02-17T05:44:54.787 INFO:teuthology.orchestra.run.trial115.stdout:(76/99): python3-jaraco-functools-3.5.0-2.el9.n 4.0 MB/s | 19 kB 00:00 2026-02-17T05:44:54.792 INFO:teuthology.orchestra.run.trial115.stdout:(77/99): python3-jaraco-text-4.0.0-2.el9.noarch 5.4 MB/s | 26 kB 00:00 2026-02-17T05:44:54.803 INFO:teuthology.orchestra.run.trial115.stdout:(78/99): python3-kubernetes-26.1.0-3.el9.noarch 93 MB/s | 1.0 MB 00:00 2026-02-17T05:44:54.814 INFO:teuthology.orchestra.run.trial115.stdout:(79/99): python3-logutils-0.3.5-21.el9.noarch.r 4.2 MB/s | 46 kB 00:00 2026-02-17T05:44:54.820 INFO:teuthology.orchestra.run.trial115.stdout:(80/99): python3-more-itertools-8.12.0-2.el9.no 14 MB/s | 79 kB 00:00 2026-02-17T05:44:54.825 INFO:teuthology.orchestra.run.trial115.stdout:(81/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 17 MB/s | 86 kB 00:00 2026-02-17T05:44:54.830 INFO:teuthology.orchestra.run.trial115.stdout:(82/99): python3-natsort-7.1.1-5.el9.noarch.rpm 11 MB/s | 58 kB 00:00 2026-02-17T05:44:54.836 INFO:teuthology.orchestra.run.trial115.stdout:(83/99): lua-devel-5.4.4-4.el9.x86_64.rpm 46 kB/s | 22 kB 00:00 2026-02-17T05:44:54.838 INFO:teuthology.orchestra.run.trial115.stdout:(84/99): python3-pecan-1.4.2-3.el9.noarch.rpm 36 MB/s | 272 kB 00:00 2026-02-17T05:44:54.844 INFO:teuthology.orchestra.run.trial115.stdout:(85/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 15 MB/s | 90 kB 00:00 2026-02-17T05:44:54.849 INFO:teuthology.orchestra.run.trial115.stdout:(86/99): python3-portend-3.1.0-2.el9.noarch.rpm 1.3 MB/s | 16 kB 00:00 2026-02-17T05:44:54.849 INFO:teuthology.orchestra.run.trial115.stdout:(87/99): python3-repoze-lru-0.7-16.el9.noarch.r 6.0 MB/s | 31 kB 00:00 2026-02-17T05:44:54.854 INFO:teuthology.orchestra.run.trial115.stdout:(88/99): python3-rsa-4.9-2.el9.noarch.rpm 11 MB/s | 59 kB 00:00 2026-02-17T05:44:54.861 INFO:teuthology.orchestra.run.trial115.stdout:(89/99): python3-saml-1.16.0-1.el9.noarch.rpm 19 MB/s | 125 kB 00:00 2026-02-17T05:44:54.865 INFO:teuthology.orchestra.run.trial115.stdout:(90/99): python3-routes-2.5.1-5.el9.noarch.rpm 12 MB/s | 188 kB 00:00 2026-02-17T05:44:54.866 INFO:teuthology.orchestra.run.trial115.stdout:(91/99): python3-tempora-5.0.0-2.el9.noarch.rpm 7.3 MB/s | 36 kB 00:00 2026-02-17T05:44:54.871 INFO:teuthology.orchestra.run.trial115.stdout:(92/99): python3-typing-extensions-4.15.0-1.el9 13 MB/s | 86 kB 00:00 2026-02-17T05:44:54.873 INFO:teuthology.orchestra.run.trial115.stdout:(93/99): python3-webob-1.8.8-2.el9.noarch.rpm 32 MB/s | 230 kB 00:00 2026-02-17T05:44:54.877 INFO:teuthology.orchestra.run.trial115.stdout:(94/99): python3-websocket-client-1.2.3-2.el9.n 14 MB/s | 90 kB 00:00 2026-02-17T05:44:54.881 INFO:teuthology.orchestra.run.trial115.stdout:(95/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 51 MB/s | 427 kB 00:00 2026-02-17T05:44:54.883 INFO:teuthology.orchestra.run.trial115.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 8.3 MB/s | 48 kB 00:00 2026-02-17T05:44:54.886 INFO:teuthology.orchestra.run.trial115.stdout:(97/99): python3-xmltodict-0.12.0-15.el9.noarch 4.3 MB/s | 22 kB 00:00 2026-02-17T05:44:54.889 INFO:teuthology.orchestra.run.trial115.stdout:(98/99): python3-zc-lockfile-2.0-10.el9.noarch. 3.4 MB/s | 20 kB 00:00 2026-02-17T05:44:54.986 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:44:54.991 INFO:teuthology.orchestra.run.trial078.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-17T05:44:54.995 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-17T05:44:55.003 INFO:teuthology.orchestra.run.trial078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-17T05:44:55.004 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:44:55.900 INFO:teuthology.orchestra.run.trial115.stdout:(99/99): protobuf-compiler-3.14.0-17.el9.x86_64 561 kB/s | 862 kB 00:01 2026-02-17T05:44:55.902 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:44:55.903 INFO:teuthology.orchestra.run.trial115.stdout:Total 19 MB/s | 95 MB 00:05 2026-02-17T05:44:56.341 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:44:56.380 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:44:56.380 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:44:56.794 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:44:56.794 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:44:57.238 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:44:57.240 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-17T05:44:57.271 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-17T05:44:57.276 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-17T05:44:57.297 INFO:teuthology.orchestra.run.trial115.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-17T05:44:57.303 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-17T05:44:57.336 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-17T05:44:57.345 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-17T05:44:57.350 INFO:teuthology.orchestra.run.trial115.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-17T05:44:57.353 INFO:teuthology.orchestra.run.trial115.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-17T05:44:57.357 INFO:teuthology.orchestra.run.trial115.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-17T05:44:57.374 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/5 2026-02-17T05:44:57.380 INFO:teuthology.orchestra.run.trial115.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-17T05:44:57.392 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:44:57.393 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-17T05:44:57.396 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-17T05:44:57.402 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-17T05:44:57.404 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-17T05:44:57.415 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-17T05:44:57.421 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-17T05:44:57.445 INFO:teuthology.orchestra.run.trial115.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-17T05:44:57.448 INFO:teuthology.orchestra.run.trial115.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-17T05:44:57.455 INFO:teuthology.orchestra.run.trial115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-17T05:44:57.479 INFO:teuthology.orchestra.run.trial115.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-17T05:44:57.531 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-17T05:44:57.543 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-17T05:44:57.545 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-17T05:44:57.550 INFO:teuthology.orchestra.run.trial115.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-17T05:44:57.556 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-17T05:44:57.560 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-17T05:44:57.564 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-17T05:44:57.578 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-17T05:44:57.599 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-17T05:44:57.604 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-17T05:44:57.609 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-17T05:44:57.620 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-17T05:44:57.622 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/5 2026-02-17T05:44:57.622 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/5 2026-02-17T05:44:57.622 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/5 2026-02-17T05:44:57.622 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-17T05:44:57.622 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-17T05:44:57.629 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-17T05:44:57.635 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-17T05:44:57.655 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-17T05:44:57.664 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-17T05:44:57.675 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-17T05:44:57.675 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:57.675 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:44:57.675 INFO:teuthology.orchestra.run.trial078.stdout: ceph-test-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:57.675 INFO:teuthology.orchestra.run.trial078.stdout: libcephsqlite-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:44:57.675 INFO:teuthology.orchestra.run.trial078.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T05:44:57.675 INFO:teuthology.orchestra.run.trial078.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T05:44:57.675 INFO:teuthology.orchestra.run.trial078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T05:44:57.676 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:57.676 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:44:57.708 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph 2026-02-17T05:44:57.718 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-17T05:44:57.731 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-17T05:44:57.746 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-17T05:44:57.751 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-17T05:44:57.759 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-17T05:44:57.799 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-17T05:44:58.014 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:18 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:44:58.100 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:44:58.103 INFO:teuthology.orchestra.run.trial078.stdout:====================================================================================== 2026-02-17T05:44:58.103 INFO:teuthology.orchestra.run.trial078.stdout: Package Arch Version Repository Size 2026-02-17T05:44:58.103 INFO:teuthology.orchestra.run.trial078.stdout:====================================================================================== 2026-02-17T05:44:58.103 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:44:58.103 INFO:teuthology.orchestra.run.trial078.stdout: ceph x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 6.5 k 2026-02-17T05:44:58.103 INFO:teuthology.orchestra.run.trial078.stdout:Installing dependencies: 2026-02-17T05:44:58.103 INFO:teuthology.orchestra.run.trial078.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T05:44:58.103 INFO:teuthology.orchestra.run.trial078.stdout: ceph-grafana-dashboards noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 31 k 2026-02-17T05:44:58.104 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mds x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 2.4 M 2026-02-17T05:44:58.104 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.1 M 2026-02-17T05:44:58.104 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-modules-core noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 253 k 2026-02-17T05:44:58.104 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mon x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 4.7 M 2026-02-17T05:44:58.104 INFO:teuthology.orchestra.run.trial078.stdout: ceph-osd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 17 M 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: ceph-prometheus-alerts noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 17 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 769 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T05:44:58.105 INFO:teuthology.orchestra.run.trial078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T05:44:58.106 INFO:teuthology.orchestra.run.trial078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T05:44:58.107 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T05:44:58.108 INFO:teuthology.orchestra.run.trial078.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T05:44:58.109 INFO:teuthology.orchestra.run.trial078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-17T05:44:58.110 INFO:teuthology.orchestra.run.trial078.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout:Installing weak dependencies: 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-cephadm noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 150 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-dashboard noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 3.8 M 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 7.4 M 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 22 k 2026-02-17T05:44:58.111 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-rook noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 49 k 2026-02-17T05:44:58.112 INFO:teuthology.orchestra.run.trial078.stdout: ceph-volume noarch 2:19.2.3-705.g04cf6dd4.el9 ceph-noarch 299 k 2026-02-17T05:44:58.112 INFO:teuthology.orchestra.run.trial078.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T05:44:58.112 INFO:teuthology.orchestra.run.trial078.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T05:44:58.112 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:58.112 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:44:58.112 INFO:teuthology.orchestra.run.trial078.stdout:====================================================================================== 2026-02-17T05:44:58.112 INFO:teuthology.orchestra.run.trial078.stdout:Install 99 Packages 2026-02-17T05:44:58.112 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:44:58.113 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 95 M 2026-02-17T05:44:58.113 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 520 M 2026-02-17T05:44:58.113 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:44:58.124 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-17T05:44:58.134 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-17T05:44:58.138 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-17T05:44:58.144 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-17T05:44:58.146 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-17T05:44:58.174 INFO:teuthology.orchestra.run.trial115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-17T05:44:58.217 INFO:teuthology.orchestra.run.trial115.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-17T05:44:58.224 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-17T05:44:58.227 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-17T05:44:58.233 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-17T05:44:58.237 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-17T05:44:58.244 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-17T05:44:58.247 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-17T05:44:58.275 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-17T05:44:58.285 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-17T05:44:58.320 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-17T05:44:58.502 INFO:teuthology.orchestra.run.trial078.stdout:(1/99): ceph-19.2.3-705.g04cf6dd4.el9.x86_64.rp 123 kB/s | 6.5 kB 00:00 2026-02-17T05:44:58.546 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 60/99 2026-02-17T05:44:58.569 INFO:teuthology.orchestra.run.trial078.stdout:(2/99): ceph-mgr-19.2.3-705.g04cf6dd4.el9.x86_6 9.0 MB/s | 1.1 MB 00:00 2026-02-17T05:44:58.570 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-17T05:44:58.574 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-17T05:44:58.633 INFO:teuthology.orchestra.run.trial115.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-17T05:44:58.635 INFO:teuthology.orchestra.run.trial115.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-17T05:44:58.636 INFO:teuthology.orchestra.run.trial078.stdout:(3/99): ceph-mds-19.2.3-705.g04cf6dd4.el9.x86_6 13 MB/s | 2.4 MB 00:00 2026-02-17T05:44:58.656 INFO:teuthology.orchestra.run.trial115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-17T05:44:58.673 INFO:teuthology.orchestra.run.trial078.stdout:(4/99): ceph-grafana-dashboards-19.2.3-705.g04c 854 kB/s | 31 kB 00:00 2026-02-17T05:44:58.714 INFO:teuthology.orchestra.run.trial078.stdout:(5/99): ceph-mgr-cephadm-19.2.3-705.g04cf6dd4.e 3.6 MB/s | 150 kB 00:00 2026-02-17T05:44:58.803 INFO:teuthology.orchestra.run.trial078.stdout:(6/99): ceph-mon-19.2.3-705.g04cf6dd4.el9.x86_6 16 MB/s | 4.7 MB 00:00 2026-02-17T05:44:58.942 INFO:teuthology.orchestra.run.trial078.stdout:(7/99): ceph-mgr-dashboard-19.2.3-705.g04cf6dd4 17 MB/s | 3.8 MB 00:00 2026-02-17T05:44:58.984 INFO:teuthology.orchestra.run.trial115.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-17T05:44:58.990 INFO:teuthology.orchestra.run.trial078.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-705.g04cf6dd4 464 kB/s | 22 kB 00:00 2026-02-17T05:44:59.045 INFO:teuthology.orchestra.run.trial078.stdout:(9/99): ceph-mgr-modules-core-19.2.3-705.g04cf6 4.5 MB/s | 253 kB 00:00 2026-02-17T05:44:59.057 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-17T05:44:59.098 INFO:teuthology.orchestra.run.trial078.stdout:(10/99): ceph-mgr-rook-19.2.3-705.g04cf6dd4.el9 931 kB/s | 49 kB 00:00 2026-02-17T05:44:59.168 INFO:teuthology.orchestra.run.trial078.stdout:(11/99): ceph-osd-19.2.3-705.g04cf6dd4.el9.x86_ 29 MB/s | 17 MB 00:00 2026-02-17T05:44:59.168 INFO:teuthology.orchestra.run.trial078.stdout:(12/99): ceph-prometheus-alerts-19.2.3-705.g04c 237 kB/s | 17 kB 00:00 2026-02-17T05:44:59.188 INFO:teuthology.orchestra.run.trial078.stdout:(13/99): ceph-mgr-diskprediction-local-19.2.3-7 19 MB/s | 7.4 MB 00:00 2026-02-17T05:44:59.190 INFO:teuthology.orchestra.run.trial078.stdout:(14/99): ceph-volume-19.2.3-705.g04cf6dd4.el9.n 13 MB/s | 299 kB 00:00 2026-02-17T05:44:59.191 INFO:teuthology.orchestra.run.trial078.stdout:(15/99): cephadm-19.2.3-705.g04cf6dd4.el9.noarc 32 MB/s | 769 kB 00:00 2026-02-17T05:44:59.257 INFO:teuthology.orchestra.run.trial078.stdout:(16/99): cryptsetup-2.8.1-2.el9.x86_64.rpm 5.0 MB/s | 350 kB 00:00 2026-02-17T05:44:59.260 INFO:teuthology.orchestra.run.trial078.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 1.0 MB/s | 72 kB 00:00 2026-02-17T05:44:59.286 INFO:teuthology.orchestra.run.trial078.stdout:(18/99): libgfortran-11.5.0-14.el9.x86_64.rpm 27 MB/s | 794 kB 00:00 2026-02-17T05:44:59.287 INFO:teuthology.orchestra.run.trial078.stdout:(19/99): libquadmath-11.5.0-14.el9.x86_64.rpm 6.8 MB/s | 184 kB 00:00 2026-02-17T05:44:59.288 INFO:teuthology.orchestra.run.trial078.stdout:(20/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 412 kB/s | 40 kB 00:00 2026-02-17T05:44:59.311 INFO:teuthology.orchestra.run.trial078.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 4.5 MB/s | 106 kB 00:00 2026-02-17T05:44:59.312 INFO:teuthology.orchestra.run.trial078.stdout:(22/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 9.4 MB/s | 253 kB 00:00 2026-02-17T05:44:59.320 INFO:teuthology.orchestra.run.trial078.stdout:(23/99): python3-cryptography-36.0.1-5.el9.x86_ 38 MB/s | 1.2 MB 00:00 2026-02-17T05:44:59.338 INFO:teuthology.orchestra.run.trial078.stdout:(24/99): python3-pycparser-2.20-6.el9.noarch.rp 5.0 MB/s | 135 kB 00:00 2026-02-17T05:44:59.340 INFO:teuthology.orchestra.run.trial078.stdout:(25/99): python3-pyparsing-2.4.7-9.el9.noarch.r 5.3 MB/s | 150 kB 00:00 2026-02-17T05:44:59.345 INFO:teuthology.orchestra.run.trial078.stdout:(26/99): python3-requests-2.25.1-10.el9.noarch. 4.8 MB/s | 126 kB 00:00 2026-02-17T05:44:59.367 INFO:teuthology.orchestra.run.trial078.stdout:(27/99): python3-urllib3-1.26.5-7.el9.noarch.rp 7.5 MB/s | 218 kB 00:00 2026-02-17T05:44:59.375 INFO:teuthology.orchestra.run.trial078.stdout:(28/99): unzip-6.0-59.el9.x86_64.rpm 5.2 MB/s | 182 kB 00:00 2026-02-17T05:44:59.381 INFO:teuthology.orchestra.run.trial078.stdout:(29/99): zip-3.0-35.el9.x86_64.rpm 7.3 MB/s | 266 kB 00:00 2026-02-17T05:44:59.454 INFO:teuthology.orchestra.run.trial078.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 202 kB/s | 15 kB 00:00 2026-02-17T05:44:59.459 INFO:teuthology.orchestra.run.trial078.stdout:(31/99): flexiblas-3.0.4-9.el9.x86_64.rpm 322 kB/s | 30 kB 00:00 2026-02-17T05:44:59.549 INFO:teuthology.orchestra.run.trial078.stdout:(32/99): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2026-02-17T05:44:59.565 INFO:teuthology.orchestra.run.trial078.stdout:(33/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 2.2 MB/s | 246 kB 00:00 2026-02-17T05:44:59.576 INFO:teuthology.orchestra.run.trial078.stdout:(34/99): openblas-0.3.29-1.el9.x86_64.rpm 1.5 MB/s | 42 kB 00:00 2026-02-17T05:44:59.623 INFO:teuthology.orchestra.run.trial078.stdout:(35/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 12 MB/s | 3.0 MB 00:00 2026-02-17T05:44:59.663 INFO:teuthology.orchestra.run.trial078.stdout:(36/99): protobuf-3.14.0-17.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2026-02-17T05:44:59.691 INFO:teuthology.orchestra.run.trial078.stdout:(37/99): python3-devel-3.9.25-2.el9.x86_64.rpm 8.4 MB/s | 244 kB 00:00 2026-02-17T05:44:59.699 INFO:teuthology.orchestra.run.trial078.stdout:(38/99): python3-babel-2.9.1-2.el9.noarch.rpm 79 MB/s | 6.0 MB 00:00 2026-02-17T05:44:59.722 INFO:teuthology.orchestra.run.trial078.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 8.0 MB/s | 249 kB 00:00 2026-02-17T05:44:59.727 INFO:teuthology.orchestra.run.trial078.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 6.2 MB/s | 177 kB 00:00 2026-02-17T05:44:59.733 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-17T05:44:59.741 INFO:teuthology.orchestra.run.trial078.stdout:(41/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 30 MB/s | 5.3 MB 00:00 2026-02-17T05:44:59.750 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-17T05:44:59.758 INFO:teuthology.orchestra.run.trial078.stdout:(42/99): python3-mako-1.1.4-6.el9.noarch.rpm 5.4 MB/s | 172 kB 00:00 2026-02-17T05:44:59.760 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-17T05:44:59.765 INFO:teuthology.orchestra.run.trial078.stdout:(43/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 28 MB/s | 1.2 MB 00:00 2026-02-17T05:44:59.766 INFO:teuthology.orchestra.run.trial078.stdout:(44/99): python3-markupsafe-1.1.1-12.el9.x86_64 1.3 MB/s | 35 kB 00:00 2026-02-17T05:44:59.767 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-17T05:44:59.780 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-17T05:44:59.793 INFO:teuthology.orchestra.run.trial078.stdout:(45/99): python3-packaging-20.9-5.el9.noarch.rp 2.8 MB/s | 77 kB 00:00 2026-02-17T05:44:59.795 INFO:teuthology.orchestra.run.trial078.stdout:(46/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 15 MB/s | 442 kB 00:00 2026-02-17T05:44:59.796 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-17T05:44:59.826 INFO:teuthology.orchestra.run.trial078.stdout:(47/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 90 MB/s | 6.1 MB 00:00 2026-02-17T05:44:59.827 INFO:teuthology.orchestra.run.trial078.stdout:(48/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 4.9 MB/s | 159 kB 00:00 2026-02-17T05:44:59.828 INFO:teuthology.orchestra.run.trial078.stdout:(49/99): python3-protobuf-3.14.0-17.el9.noarch. 7.6 MB/s | 267 kB 00:00 2026-02-17T05:44:59.853 INFO:teuthology.orchestra.run.trial078.stdout:(50/99): python3-requests-oauthlib-1.3.0-12.el9 2.0 MB/s | 54 kB 00:00 2026-02-17T05:44:59.855 INFO:teuthology.orchestra.run.trial078.stdout:(51/99): python3-pyasn1-modules-0.4.8-6.el9.noa 9.6 MB/s | 279 kB 00:00 2026-02-17T05:44:59.872 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-17T05:44:59.879 INFO:teuthology.orchestra.run.trial078.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 1.6 MB/s | 42 kB 00:00 2026-02-17T05:44:59.880 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-17T05:44:59.884 INFO:teuthology.orchestra.run.trial078.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 6.3 MB/s | 189 kB 00:00 2026-02-17T05:44:59.904 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-17T05:44:59.904 INFO:teuthology.orchestra.run.trial078.stdout:(54/99): lua-devel-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 22 kB 00:00 2026-02-17T05:44:59.905 INFO:teuthology.orchestra.run.trial078.stdout:(55/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.5 MB/s | 90 kB 00:00 2026-02-17T05:44:59.924 INFO:teuthology.orchestra.run.trial078.stdout:(56/99): protobuf-compiler-3.14.0-17.el9.x86_64 44 MB/s | 862 kB 00:00 2026-02-17T05:44:59.934 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-17T05:44:59.935 INFO:teuthology.orchestra.run.trial078.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 1.7 MB/s | 19 kB 00:00 2026-02-17T05:44:59.948 INFO:teuthology.orchestra.run.trial078.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 11 MB/s | 151 kB 00:00 2026-02-17T05:44:59.951 INFO:teuthology.orchestra.run.trial078.stdout:(59/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 12 MB/s | 551 kB 00:00 2026-02-17T05:44:59.957 INFO:teuthology.orchestra.run.trial078.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 5.2 MB/s | 29 kB 00:00 2026-02-17T05:44:59.965 INFO:teuthology.orchestra.run.trial078.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 7.4 MB/s | 60 kB 00:00 2026-02-17T05:44:59.969 INFO:teuthology.orchestra.run.trial078.stdout:(62/99): python3-asyncssh-2.13.2-5.el9.noarch.r 26 MB/s | 548 kB 00:00 2026-02-17T05:44:59.971 INFO:teuthology.orchestra.run.trial078.stdout:(63/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 6.9 MB/s | 43 kB 00:00 2026-02-17T05:44:59.975 INFO:teuthology.orchestra.run.trial078.stdout:(64/99): python3-cachetools-4.2.4-1.el9.noarch. 5.8 MB/s | 32 kB 00:00 2026-02-17T05:44:59.978 INFO:teuthology.orchestra.run.trial078.stdout:(65/99): python3-certifi-2023.05.07-4.el9.noarc 2.2 MB/s | 14 kB 00:00 2026-02-17T05:44:59.982 INFO:teuthology.orchestra.run.trial078.stdout:(66/99): python3-cheroot-10.0.1-4.el9.noarch.rp 25 MB/s | 173 kB 00:00 2026-02-17T05:44:59.985 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-17T05:44:59.988 INFO:teuthology.orchestra.run.trial078.stdout:(67/99): python3-cherrypy-18.6.1-2.el9.noarch.r 35 MB/s | 358 kB 00:00 2026-02-17T05:44:59.991 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-17T05:44:59.992 INFO:teuthology.orchestra.run.trial078.stdout:(68/99): python3-google-auth-2.45.0-1.el9.noarc 26 MB/s | 254 kB 00:00 2026-02-17T05:44:59.995 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-17T05:44:59.996 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-17T05:45:00.007 INFO:teuthology.orchestra.run.trial078.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 9.0 MB/s | 144 kB 00:00 2026-02-17T05:45:00.015 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:45:00.015 INFO:teuthology.orchestra.run.trial115.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T05:45:00.015 INFO:teuthology.orchestra.run.trial115.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T05:45:00.015 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:00.015 INFO:teuthology.orchestra.run.trial078.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 19 MB/s | 139 kB 00:00 2026-02-17T05:45:00.020 INFO:teuthology.orchestra.run.trial078.stdout:(71/99): python3-isodate-0.6.1-3.el9.noarch.rpm 10 MB/s | 56 kB 00:00 2026-02-17T05:45:00.025 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:45:00.025 INFO:teuthology.orchestra.run.trial078.stdout:(72/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 2.2 MB/s | 11 kB 00:00 2026-02-17T05:45:00.030 INFO:teuthology.orchestra.run.trial078.stdout:(73/99): python3-jaraco-classes-3.2.1-5.el9.noa 3.7 MB/s | 18 kB 00:00 2026-02-17T05:45:00.035 INFO:teuthology.orchestra.run.trial078.stdout:(74/99): python3-jaraco-collections-3.0.0-8.el9 4.3 MB/s | 23 kB 00:00 2026-02-17T05:45:00.040 INFO:teuthology.orchestra.run.trial078.stdout:(75/99): python3-jaraco-context-6.0.1-3.el9.noa 3.8 MB/s | 20 kB 00:00 2026-02-17T05:45:00.048 INFO:teuthology.orchestra.run.trial078.stdout:(76/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 34 MB/s | 2.0 MB 00:00 2026-02-17T05:45:00.048 INFO:teuthology.orchestra.run.trial078.stdout:(77/99): python3-jaraco-functools-3.5.0-2.el9.n 2.5 MB/s | 19 kB 00:00 2026-02-17T05:45:00.053 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:45:00.053 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T05:45:00.053 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:00.054 INFO:teuthology.orchestra.run.trial078.stdout:(78/99): python3-jaraco-text-4.0.0-2.el9.noarch 4.2 MB/s | 26 kB 00:00 2026-02-17T05:45:00.060 INFO:teuthology.orchestra.run.trial078.stdout:(79/99): python3-kubernetes-26.1.0-3.el9.noarch 83 MB/s | 1.0 MB 00:00 2026-02-17T05:45:00.061 INFO:teuthology.orchestra.run.trial078.stdout:(80/99): python3-logutils-0.3.5-21.el9.noarch.r 6.2 MB/s | 46 kB 00:00 2026-02-17T05:45:00.073 INFO:teuthology.orchestra.run.trial078.stdout:(81/99): python3-more-itertools-8.12.0-2.el9.no 6.4 MB/s | 79 kB 00:00 2026-02-17T05:45:00.073 INFO:teuthology.orchestra.run.trial078.stdout:(82/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 7.2 MB/s | 86 kB 00:00 2026-02-17T05:45:00.079 INFO:teuthology.orchestra.run.trial078.stdout:(83/99): python3-natsort-7.1.1-5.el9.noarch.rpm 9.6 MB/s | 58 kB 00:00 2026-02-17T05:45:00.080 INFO:teuthology.orchestra.run.trial078.stdout:(84/99): python3-pecan-1.4.2-3.el9.noarch.rpm 39 MB/s | 272 kB 00:00 2026-02-17T05:45:00.085 INFO:teuthology.orchestra.run.trial078.stdout:(85/99): python3-portend-3.1.0-2.el9.noarch.rpm 2.4 MB/s | 16 kB 00:00 2026-02-17T05:45:00.086 INFO:teuthology.orchestra.run.trial078.stdout:(86/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 14 MB/s | 90 kB 00:00 2026-02-17T05:45:00.090 INFO:teuthology.orchestra.run.trial115.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-17T05:45:00.093 INFO:teuthology.orchestra.run.trial078.stdout:(87/99): python3-routes-2.5.1-5.el9.noarch.rpm 31 MB/s | 188 kB 00:00 2026-02-17T05:45:00.093 INFO:teuthology.orchestra.run.trial078.stdout:(88/99): python3-repoze-lru-0.7-16.el9.noarch.r 4.1 MB/s | 31 kB 00:00 2026-02-17T05:45:00.100 INFO:teuthology.orchestra.run.trial078.stdout:(89/99): python3-saml-1.16.0-1.el9.noarch.rpm 19 MB/s | 125 kB 00:00 2026-02-17T05:45:00.100 INFO:teuthology.orchestra.run.trial078.stdout:(90/99): python3-rsa-4.9-2.el9.noarch.rpm 7.8 MB/s | 59 kB 00:00 2026-02-17T05:45:00.105 INFO:teuthology.orchestra.run.trial078.stdout:(91/99): python3-typing-extensions-4.15.0-1.el9 17 MB/s | 86 kB 00:00 2026-02-17T05:45:00.106 INFO:teuthology.orchestra.run.trial078.stdout:(92/99): python3-tempora-5.0.0-2.el9.noarch.rpm 5.5 MB/s | 36 kB 00:00 2026-02-17T05:45:00.112 INFO:teuthology.orchestra.run.trial078.stdout:(93/99): python3-webob-1.8.8-2.el9.noarch.rpm 33 MB/s | 230 kB 00:00 2026-02-17T05:45:00.114 INFO:teuthology.orchestra.run.trial078.stdout:(94/99): python3-websocket-client-1.2.3-2.el9.n 11 MB/s | 90 kB 00:00 2026-02-17T05:45:00.120 INFO:teuthology.orchestra.run.trial078.stdout:(95/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 55 MB/s | 427 kB 00:00 2026-02-17T05:45:00.120 INFO:teuthology.orchestra.run.trial078.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 7.6 MB/s | 48 kB 00:00 2026-02-17T05:45:00.126 INFO:teuthology.orchestra.run.trial078.stdout:(97/99): python3-xmltodict-0.12.0-15.el9.noarch 3.4 MB/s | 22 kB 00:00 2026-02-17T05:45:00.128 INFO:teuthology.orchestra.run.trial078.stdout:(98/99): python3-zc-lockfile-2.0-10.el9.noarch. 2.8 MB/s | 20 kB 00:00 2026-02-17T05:45:00.154 INFO:teuthology.orchestra.run.trial115.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 84/99 2026-02-17T05:45:00.156 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:45:00.170 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:45:00.170 INFO:teuthology.orchestra.run.trial115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:00.170 INFO:teuthology.orchestra.run.trial115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T05:45:00.170 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:00.268 INFO:teuthology.orchestra.run.trial078.stdout:(99/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 44 MB/s | 19 MB 00:00 2026-02-17T05:45:00.271 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:00.271 INFO:teuthology.orchestra.run.trial078.stdout:Total 44 MB/s | 95 MB 00:02 2026-02-17T05:45:00.708 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:45:00.746 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:45:00.746 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:45:00.866 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:45:00.892 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:45:00.892 INFO:teuthology.orchestra.run.trial115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:00.892 INFO:teuthology.orchestra.run.trial115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T05:45:00.892 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:45:00.892 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:45:00.892 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:00.948 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:45:00.950 INFO:teuthology.orchestra.run.trial115.stdout: Installing : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:45:00.955 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 88/99 2026-02-17T05:45:00.975 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 89/99 2026-02-17T05:45:00.977 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:45:01.161 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:45:01.162 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:45:01.473 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:45:01.476 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:45:01.603 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:45:01.605 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-17T05:45:01.636 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-17T05:45:01.642 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-17T05:45:01.663 INFO:teuthology.orchestra.run.trial078.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-17T05:45:01.669 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-17T05:45:01.702 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-17T05:45:01.711 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-17T05:45:01.716 INFO:teuthology.orchestra.run.trial078.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-17T05:45:01.719 INFO:teuthology.orchestra.run.trial078.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-17T05:45:01.723 INFO:teuthology.orchestra.run.trial078.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-17T05:45:01.746 INFO:teuthology.orchestra.run.trial078.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-17T05:45:01.759 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-17T05:45:01.763 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-17T05:45:01.771 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-17T05:45:01.773 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-17T05:45:01.784 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-17T05:45:01.790 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-17T05:45:01.814 INFO:teuthology.orchestra.run.trial078.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-17T05:45:01.817 INFO:teuthology.orchestra.run.trial078.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-17T05:45:01.824 INFO:teuthology.orchestra.run.trial078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-17T05:45:01.849 INFO:teuthology.orchestra.run.trial078.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-17T05:45:01.900 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-17T05:45:01.913 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-17T05:45:01.915 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-17T05:45:01.920 INFO:teuthology.orchestra.run.trial078.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-17T05:45:01.925 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-17T05:45:01.930 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-17T05:45:01.933 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-17T05:45:01.936 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:45:01.937 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:45:01.948 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-17T05:45:01.951 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:45:01.952 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:45:01.969 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-17T05:45:01.973 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-17T05:45:01.978 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-17T05:45:01.990 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-17T05:45:01.999 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-17T05:45:02.005 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-17T05:45:02.008 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:45:02.025 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-17T05:45:02.034 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-17T05:45:02.058 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 94/99 2026-02-17T05:45:02.059 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:45:02.081 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:45:02.081 INFO:teuthology.orchestra.run.trial115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:02.081 INFO:teuthology.orchestra.run.trial115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T05:45:02.081 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:45:02.081 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:45:02.082 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:02.088 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-17T05:45:02.093 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:45:02.100 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-17T05:45:02.116 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-17T05:45:02.120 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-17T05:45:02.128 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-17T05:45:02.168 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-17T05:45:02.310 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:45:02.312 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:45:02.334 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:45:02.335 INFO:teuthology.orchestra.run.trial115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:02.335 INFO:teuthology.orchestra.run.trial115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T05:45:02.335 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:45:02.335 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:45:02.335 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:02.436 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:45:02.456 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:45:02.457 INFO:teuthology.orchestra.run.trial115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:02.457 INFO:teuthology.orchestra.run.trial115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T05:45:02.457 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:45:02.457 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:45:02.457 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:02.475 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:45:02.494 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-17T05:45:02.504 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-17T05:45:02.508 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-17T05:45:02.513 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-17T05:45:02.515 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-17T05:45:02.543 INFO:teuthology.orchestra.run.trial078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-17T05:45:02.587 INFO:teuthology.orchestra.run.trial078.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-17T05:45:02.593 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-17T05:45:02.597 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-17T05:45:02.602 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-17T05:45:02.606 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-17T05:45:02.613 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-17T05:45:02.616 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-17T05:45:02.644 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-17T05:45:02.654 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-17T05:45:02.689 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-17T05:45:02.915 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 60/99 2026-02-17T05:45:02.939 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-17T05:45:02.943 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-17T05:45:03.001 INFO:teuthology.orchestra.run.trial078.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-17T05:45:03.003 INFO:teuthology.orchestra.run.trial078.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-17T05:45:03.024 INFO:teuthology.orchestra.run.trial078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-17T05:45:03.155 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:45:03.155 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 6/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 7/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 8/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 9/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 10/99 2026-02-17T05:45:03.156 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 11/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 12/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 13/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 14/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 15/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-17T05:45:03.157 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-17T05:45:03.158 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-17T05:45:03.159 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-17T05:45:03.160 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-17T05:45:03.161 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-17T05:45:03.162 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-17T05:45:03.163 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.218 INFO:teuthology.orchestra.run.trial115.stdout: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T05:45:03.219 INFO:teuthology.orchestra.run.trial115.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T05:45:03.220 INFO:teuthology.orchestra.run.trial115.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T05:45:03.221 INFO:teuthology.orchestra.run.trial115.stdout: zip-3.0-35.el9.x86_64 2026-02-17T05:45:03.221 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:03.221 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:03.273 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-base 2026-02-17T05:45:03.349 INFO:teuthology.orchestra.run.trial078.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-17T05:45:03.423 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-17T05:45:03.584 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:03.649 INFO:teuthology.orchestra.run.trial115.stdout:Package ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:03.666 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:03.667 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:03.667 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:03.688 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install cephadm 2026-02-17T05:45:04.005 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:04.069 INFO:teuthology.orchestra.run.trial115.stdout:Package cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:04.086 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:04.087 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:04.087 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:04.095 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-17T05:45:04.108 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T05:45:04.113 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-17T05:45:04.123 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-17T05:45:04.130 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-17T05:45:04.143 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-17T05:45:04.158 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-17T05:45:04.234 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-17T05:45:04.242 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-17T05:45:04.266 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-17T05:45:04.296 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-17T05:45:04.346 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-17T05:45:04.352 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-17T05:45:04.356 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-17T05:45:04.357 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-17T05:45:04.374 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:45:04.374 INFO:teuthology.orchestra.run.trial078.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T05:45:04.375 INFO:teuthology.orchestra.run.trial078.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T05:45:04.375 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:04.382 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:45:04.408 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T05:45:04.408 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T05:45:04.408 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:04.416 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:04.442 INFO:teuthology.orchestra.run.trial078.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-17T05:45:04.498 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:04.499 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:04.499 INFO:teuthology.orchestra.run.trial115.stdout: Package Arch Version Repo Size 2026-02-17T05:45:04.499 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 145 k 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout:Install 1 Package 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 145 k 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 431 k 2026-02-17T05:45:04.500 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:45:04.505 INFO:teuthology.orchestra.run.trial078.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 84/99 2026-02-17T05:45:04.507 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:45:04.521 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 85/99 2026-02-17T05:45:04.521 INFO:teuthology.orchestra.run.trial078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:04.521 INFO:teuthology.orchestra.run.trial078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T05:45:04.521 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:04.585 INFO:teuthology.orchestra.run.trial115.stdout:ceph-immutable-object-cache-19.2.3-705.g04cf6dd 1.7 MB/s | 145 kB 00:00 2026-02-17T05:45:04.585 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:04.585 INFO:teuthology.orchestra.run.trial115.stdout:Total 1.6 MB/s | 145 kB 00:00 2026-02-17T05:45:04.585 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:45:04.589 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:45:04.589 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:45:04.619 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:45:04.619 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:45:04.674 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:45:04.675 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:45:04.696 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:45:04.697 INFO:teuthology.orchestra.run.trial115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:04.697 INFO:teuthology.orchestra.run.trial115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T05:45:04.697 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:05.009 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:45:05.009 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:05.010 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:45:05.010 INFO:teuthology.orchestra.run.trial115.stdout: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:05.010 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:05.010 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:05.038 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-mgr 2026-02-17T05:45:05.219 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:45:05.243 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 86/99 2026-02-17T05:45:05.244 INFO:teuthology.orchestra.run.trial078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:05.244 INFO:teuthology.orchestra.run.trial078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T05:45:05.244 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:45:05.244 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T05:45:05.244 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:05.300 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:45:05.302 INFO:teuthology.orchestra.run.trial078.stdout: Installing : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 87/99 2026-02-17T05:45:05.307 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 88/99 2026-02-17T05:45:05.327 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 89/99 2026-02-17T05:45:05.328 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:45:05.348 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:05.413 INFO:teuthology.orchestra.run.trial115.stdout:Package ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:05.430 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:05.430 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:05.430 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:05.452 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T05:45:05.769 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:05.820 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 90/99 2026-02-17T05:45:05.823 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:45:05.833 INFO:teuthology.orchestra.run.trial115.stdout:Package ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:05.850 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:05.851 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:05.851 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:05.872 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T05:45:06.185 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:06.249 INFO:teuthology.orchestra.run.trial115.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:06.267 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:06.268 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:06.268 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:06.280 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 91/99 2026-02-17T05:45:06.281 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:45:06.289 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-mgr-rook 2026-02-17T05:45:06.293 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 92/99 2026-02-17T05:45:06.294 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:45:06.351 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 93/99 2026-02-17T05:45:06.400 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 94/99 2026-02-17T05:45:06.401 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:45:06.423 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 95/99 2026-02-17T05:45:06.424 INFO:teuthology.orchestra.run.trial078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:06.424 INFO:teuthology.orchestra.run.trial078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T05:45:06.424 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:45:06.424 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T05:45:06.424 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:06.435 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:45:06.597 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:06.655 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 96/99 2026-02-17T05:45:06.656 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:45:06.660 INFO:teuthology.orchestra.run.trial115.stdout:Package ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:06.678 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 97/99 2026-02-17T05:45:06.678 INFO:teuthology.orchestra.run.trial078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:06.678 INFO:teuthology.orchestra.run.trial078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T05:45:06.678 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:45:06.678 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T05:45:06.678 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:06.679 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:06.679 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:06.679 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:06.700 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T05:45:06.780 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:45:06.800 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 98/99 2026-02-17T05:45:06.800 INFO:teuthology.orchestra.run.trial078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:06.800 INFO:teuthology.orchestra.run.trial078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T05:45:06.800 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:45:06.800 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T05:45:06.801 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:06.818 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:45:07.013 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:07.077 INFO:teuthology.orchestra.run.trial115.stdout:Package ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:07.095 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:07.095 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:07.095 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:07.117 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-fuse 2026-02-17T05:45:07.430 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:07.502 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 99/99 2026-02-17T05:45:07.502 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/99 2026-02-17T05:45:07.502 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2/99 2026-02-17T05:45:07.502 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 3/99 2026-02-17T05:45:07.502 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 4/99 2026-02-17T05:45:07.502 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 5/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9 6/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 7/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noar 8/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6d 9/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noar 10/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.n 11/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 12/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9. 13/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 14/99 2026-02-17T05:45:07.503 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 15/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-17T05:45:07.504 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-17T05:45:07.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-17T05:45:07.506 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-17T05:45:07.507 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-17T05:45:07.508 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-17T05:45:07.509 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-17T05:45:07.510 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-17T05:45:07.511 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-17T05:45:07.511 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-17T05:45:07.511 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-17T05:45:07.511 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-17T05:45:07.511 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-17T05:45:07.511 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-17T05:45:07.511 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-17T05:45:07.511 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-17T05:45:07.513 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:07.514 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:07.514 INFO:teuthology.orchestra.run.trial115.stdout: Package Architecture Version Repository Size 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout: ceph-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.2 M 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout:Install 1 Package 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:07.515 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 1.2 M 2026-02-17T05:45:07.516 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 3.6 M 2026-02-17T05:45:07.516 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:45:07.563 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-grafana-dashboards-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mds-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-k8sevents-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-modules-core-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-mon-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-osd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-prometheus-alerts-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T05:45:07.564 INFO:teuthology.orchestra.run.trial078.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T05:45:07.565 INFO:teuthology.orchestra.run.trial078.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: zip-3.0-35.el9.x86_64 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:07.566 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:07.621 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-base 2026-02-17T05:45:07.658 INFO:teuthology.orchestra.run.trial115.stdout:ceph-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 8.0 MB/s | 1.2 MB 00:00 2026-02-17T05:45:07.659 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:07.659 INFO:teuthology.orchestra.run.trial115.stdout:Total 8.0 MB/s | 1.2 MB 00:00 2026-02-17T05:45:07.659 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:45:07.662 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:45:07.662 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:45:07.696 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:45:07.696 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:45:07.796 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:45:07.813 INFO:teuthology.orchestra.run.trial115.stdout: Installing : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:07.936 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:27 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:08.000 INFO:teuthology.orchestra.run.trial078.stdout:Package ceph-base-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:08.011 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:08.018 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:08.019 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:08.019 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:08.041 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install cephadm 2026-02-17T05:45:08.068 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:08.068 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:08.069 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:45:08.069 INFO:teuthology.orchestra.run.trial115.stdout: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:08.069 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:08.069 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:08.096 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install ceph-volume 2026-02-17T05:45:08.353 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:08.412 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:08.418 INFO:teuthology.orchestra.run.trial078.stdout:Package cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:08.435 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:08.435 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:08.436 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:08.456 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T05:45:08.476 INFO:teuthology.orchestra.run.trial115.stdout:Package ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:08.494 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:08.495 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:08.495 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:08.517 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install librados-devel 2026-02-17T05:45:08.766 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:08.828 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:08.848 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout: Package Arch Version Repo Size 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 145 k 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:Install 1 Package 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 145 k 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 431 k 2026-02-17T05:45:08.849 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout: Package Arch Version Repo Size 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout: librados-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 126 k 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:Install 1 Package 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 126 k 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 456 k 2026-02-17T05:45:08.911 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:45:08.936 INFO:teuthology.orchestra.run.trial078.stdout:ceph-immutable-object-cache-19.2.3-705.g04cf6dd 1.6 MB/s | 145 kB 00:00 2026-02-17T05:45:08.937 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:08.937 INFO:teuthology.orchestra.run.trial078.stdout:Total 1.6 MB/s | 145 kB 00:00 2026-02-17T05:45:08.937 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:45:08.941 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:45:08.941 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:45:08.971 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:45:08.972 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:45:08.997 INFO:teuthology.orchestra.run.trial115.stdout:librados-devel-19.2.3-705.g04cf6dd4.el9.x86_64. 1.4 MB/s | 126 kB 00:00 2026-02-17T05:45:08.998 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:08.998 INFO:teuthology.orchestra.run.trial115.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-17T05:45:08.998 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:45:09.001 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:45:09.001 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:45:09.027 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:45:09.028 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:45:09.032 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:45:09.032 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:45:09.049 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:45:09.049 INFO:teuthology.orchestra.run.trial078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:09.050 INFO:teuthology.orchestra.run.trial078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T05:45:09.050 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:09.088 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:45:09.104 INFO:teuthology.orchestra.run.trial115.stdout: Installing : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:09.189 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:09.245 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:09.245 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:09.246 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:45:09.246 INFO:teuthology.orchestra.run.trial115.stdout: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:09.246 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:09.246 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:09.273 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install libcephfs2 2026-02-17T05:45:09.365 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.e 1/1 2026-02-17T05:45:09.365 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:09.365 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:45:09.365 INFO:teuthology.orchestra.run.trial078.stdout: ceph-immutable-object-cache-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:09.365 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:09.365 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:09.392 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-mgr 2026-02-17T05:45:09.596 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:09.661 INFO:teuthology.orchestra.run.trial115.stdout:Package libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:09.678 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:09.679 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:09.679 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:09.701 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install libcephfs-devel 2026-02-17T05:45:09.707 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:09.772 INFO:teuthology.orchestra.run.trial078.stdout:Package ceph-mgr-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:09.790 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:09.790 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:09.791 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:09.812 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T05:45:10.009 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:10.091 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:10.092 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:10.092 INFO:teuthology.orchestra.run.trial115.stdout: Package Arch Version Repo Size 2026-02-17T05:45:10.092 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:10.092 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:45:10.092 INFO:teuthology.orchestra.run.trial115.stdout: libcephfs-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 34 k 2026-02-17T05:45:10.092 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:10.092 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:45:10.092 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:10.093 INFO:teuthology.orchestra.run.trial115.stdout:Install 1 Package 2026-02-17T05:45:10.093 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:10.093 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 34 k 2026-02-17T05:45:10.093 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 153 k 2026-02-17T05:45:10.093 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:45:10.127 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:10.193 INFO:teuthology.orchestra.run.trial115.stdout:libcephfs-devel-19.2.3-705.g04cf6dd4.el9.x86_64 330 kB/s | 34 kB 00:00 2026-02-17T05:45:10.194 INFO:teuthology.orchestra.run.trial078.stdout:Package ceph-mgr-dashboard-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:10.195 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:10.195 INFO:teuthology.orchestra.run.trial115.stdout:Total 327 kB/s | 34 kB 00:00 2026-02-17T05:45:10.195 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:45:10.196 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:45:10.197 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:45:10.199 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:45:10.199 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:45:10.211 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:10.212 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:10.212 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:10.220 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:45:10.236 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T05:45:10.237 INFO:teuthology.orchestra.run.trial115.stdout: Installing : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:10.289 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:10.344 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:10.345 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:10.345 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:45:10.345 INFO:teuthology.orchestra.run.trial115.stdout: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:10.345 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:10.345 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:10.371 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install librados2 2026-02-17T05:45:10.545 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:10.610 INFO:teuthology.orchestra.run.trial078.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:10.627 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:10.628 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:10.628 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:10.649 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-mgr-rook 2026-02-17T05:45:10.676 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:10.741 INFO:teuthology.orchestra.run.trial115.stdout:Package librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:10.758 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:10.759 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:10.759 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:10.780 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install librbd1 2026-02-17T05:45:10.954 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:11.018 INFO:teuthology.orchestra.run.trial078.stdout:Package ceph-mgr-rook-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:11.035 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:11.036 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:11.036 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:11.056 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T05:45:11.088 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:11.153 INFO:teuthology.orchestra.run.trial115.stdout:Package librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:11.170 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:11.170 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:11.171 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:11.193 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install python3-rados 2026-02-17T05:45:11.365 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:11.429 INFO:teuthology.orchestra.run.trial078.stdout:Package ceph-mgr-cephadm-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:11.447 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:11.447 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:11.447 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:11.467 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-fuse 2026-02-17T05:45:11.505 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:11.570 INFO:teuthology.orchestra.run.trial115.stdout:Package python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:11.587 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:11.588 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:11.588 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:11.610 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install python3-rgw 2026-02-17T05:45:11.774 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:11.856 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout: Package Architecture Version Repository Size 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout: ceph-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 1.2 M 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:Install 1 Package 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 1.2 M 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 3.6 M 2026-02-17T05:45:11.857 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:45:11.917 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:11.981 INFO:teuthology.orchestra.run.trial115.stdout:Package python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:11.998 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:11.998 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:11.999 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:12.020 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install python3-cephfs 2026-02-17T05:45:12.254 INFO:teuthology.orchestra.run.trial078.stdout:ceph-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 2.9 MB/s | 1.2 MB 00:00 2026-02-17T05:45:12.254 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:12.255 INFO:teuthology.orchestra.run.trial078.stdout:Total 2.9 MB/s | 1.2 MB 00:00 2026-02-17T05:45:12.255 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:45:12.259 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:45:12.259 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:45:12.293 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:45:12.293 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:45:12.331 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:12.394 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:45:12.395 INFO:teuthology.orchestra.run.trial115.stdout:Package python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:12.410 INFO:teuthology.orchestra.run.trial078.stdout: Installing : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:12.413 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:12.414 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:12.414 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:12.436 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install python3-rbd 2026-02-17T05:45:12.627 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:12.684 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:12.684 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:12.685 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:45:12.685 INFO:teuthology.orchestra.run.trial078.stdout: ceph-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:12.685 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:12.685 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:12.713 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install ceph-volume 2026-02-17T05:45:12.746 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:12.809 INFO:teuthology.orchestra.run.trial115.stdout:Package python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:12.827 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:12.827 INFO:teuthology.orchestra.run.trial115.stdout:Nothing to do. 2026-02-17T05:45:12.828 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:12.849 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install rbd-fuse 2026-02-17T05:45:13.027 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:13.092 INFO:teuthology.orchestra.run.trial078.stdout:Package ceph-volume-2:19.2.3-705.g04cf6dd4.el9.noarch is already installed. 2026-02-17T05:45:13.109 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:13.110 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:13.110 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:13.132 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install librados-devel 2026-02-17T05:45:13.158 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:13.240 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:13.241 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:13.241 INFO:teuthology.orchestra.run.trial115.stdout: Package Architecture Version Repository Size 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout: rbd-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 85 k 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout:Install 1 Package 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 85 k 2026-02-17T05:45:13.242 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 227 k 2026-02-17T05:45:13.243 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:45:13.315 INFO:teuthology.orchestra.run.trial115.stdout:rbd-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-17T05:45:13.315 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:13.316 INFO:teuthology.orchestra.run.trial115.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-17T05:45:13.316 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:45:13.318 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:45:13.319 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:45:13.350 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:45:13.350 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:45:13.404 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:45:13.420 INFO:teuthology.orchestra.run.trial115.stdout: Installing : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:13.443 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:13.503 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout: Package Arch Version Repo Size 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout: librados-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 126 k 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:Install 1 Package 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 126 k 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 456 k 2026-02-17T05:45:13.527 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:45:13.559 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:13.559 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:13.559 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:45:13.559 INFO:teuthology.orchestra.run.trial115.stdout: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:13.559 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:13.559 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:13.587 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install rbd-mirror 2026-02-17T05:45:13.612 INFO:teuthology.orchestra.run.trial078.stdout:librados-devel-19.2.3-705.g04cf6dd4.el9.x86_64. 1.5 MB/s | 126 kB 00:00 2026-02-17T05:45:13.612 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:13.612 INFO:teuthology.orchestra.run.trial078.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-17T05:45:13.612 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:45:13.615 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:45:13.615 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:45:13.645 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:45:13.646 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:45:13.700 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:45:13.717 INFO:teuthology.orchestra.run.trial078.stdout: Installing : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:13.796 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:13.853 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:13.853 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:13.853 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:45:13.853 INFO:teuthology.orchestra.run.trial078.stdout: librados-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:13.853 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:13.853 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:13.879 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install libcephfs2 2026-02-17T05:45:13.896 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:13.979 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:13.979 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:13.979 INFO:teuthology.orchestra.run.trial115.stdout: Package Arch Version Repo Size 2026-02-17T05:45:13.979 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:13.979 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:45:13.979 INFO:teuthology.orchestra.run.trial115.stdout: rbd-mirror x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.1 M 2026-02-17T05:45:13.980 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:13.980 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:45:13.980 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:13.980 INFO:teuthology.orchestra.run.trial115.stdout:Install 1 Package 2026-02-17T05:45:13.980 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:13.980 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 3.1 M 2026-02-17T05:45:13.980 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 13 M 2026-02-17T05:45:13.980 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:45:14.130 INFO:teuthology.orchestra.run.trial115.stdout:rbd-mirror-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 21 MB/s | 3.1 MB 00:00 2026-02-17T05:45:14.130 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:14.130 INFO:teuthology.orchestra.run.trial115.stdout:Total 21 MB/s | 3.1 MB 00:00 2026-02-17T05:45:14.130 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:45:14.134 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:45:14.134 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:45:14.176 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:45:14.176 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:45:14.195 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:14.259 INFO:teuthology.orchestra.run.trial078.stdout:Package libcephfs2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:14.277 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:14.277 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:14.277 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:14.298 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install libcephfs-devel 2026-02-17T05:45:14.373 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:45:14.374 INFO:teuthology.orchestra.run.trial115.stdout: Installing : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:14.396 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:14.397 INFO:teuthology.orchestra.run.trial115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:14.397 INFO:teuthology.orchestra.run.trial115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T05:45:14.397 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:45:14.397 INFO:teuthology.orchestra.run.trial115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:45:14.398 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:14.607 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:14.675 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:14.676 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:14.676 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:45:14.676 INFO:teuthology.orchestra.run.trial115.stdout: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:14.676 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:14.676 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:14.689 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout: Package Arch Version Repo Size 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout: libcephfs-devel x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 34 k 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:Install 1 Package 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 34 k 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 153 k 2026-02-17T05:45:14.690 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:45:14.704 DEBUG:teuthology.orchestra.run.trial115:> sudo yum -y install rbd-nbd 2026-02-17T05:45:14.753 INFO:teuthology.orchestra.run.trial078.stdout:libcephfs-devel-19.2.3-705.g04cf6dd4.el9.x86_64 536 kB/s | 34 kB 00:00 2026-02-17T05:45:14.754 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:14.754 INFO:teuthology.orchestra.run.trial078.stdout:Total 529 kB/s | 34 kB 00:00 2026-02-17T05:45:14.754 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:45:14.756 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:45:14.756 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:45:14.758 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:45:14.758 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:45:14.778 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:45:14.793 INFO:teuthology.orchestra.run.trial078.stdout: Installing : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:14.852 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:14.912 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:14.913 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:14.913 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:45:14.913 INFO:teuthology.orchestra.run.trial078.stdout: libcephfs-devel-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:14.913 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:14.913 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:14.940 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install librados2 2026-02-17T05:45:15.013 INFO:teuthology.orchestra.run.trial115.stdout:Last metadata expiration check: 0:00:43 ago on Tue 17 Feb 2026 05:44:32 AM UTC. 2026-02-17T05:45:15.095 INFO:teuthology.orchestra.run.trial115.stdout:Dependencies resolved. 2026-02-17T05:45:15.095 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:15.095 INFO:teuthology.orchestra.run.trial115.stdout: Package Architecture Version Repository Size 2026-02-17T05:45:15.095 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:15.095 INFO:teuthology.orchestra.run.trial115.stdout:Installing: 2026-02-17T05:45:15.095 INFO:teuthology.orchestra.run.trial115.stdout: rbd-nbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 171 k 2026-02-17T05:45:15.095 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:15.095 INFO:teuthology.orchestra.run.trial115.stdout:Transaction Summary 2026-02-17T05:45:15.096 INFO:teuthology.orchestra.run.trial115.stdout:================================================================================ 2026-02-17T05:45:15.096 INFO:teuthology.orchestra.run.trial115.stdout:Install 1 Package 2026-02-17T05:45:15.096 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:15.096 INFO:teuthology.orchestra.run.trial115.stdout:Total download size: 171 k 2026-02-17T05:45:15.096 INFO:teuthology.orchestra.run.trial115.stdout:Installed size: 490 k 2026-02-17T05:45:15.096 INFO:teuthology.orchestra.run.trial115.stdout:Downloading Packages: 2026-02-17T05:45:15.252 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:15.264 INFO:teuthology.orchestra.run.trial115.stdout:rbd-nbd-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 1.0 MB/s | 171 kB 00:00 2026-02-17T05:45:15.265 INFO:teuthology.orchestra.run.trial115.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:15.265 INFO:teuthology.orchestra.run.trial115.stdout:Total 1.0 MB/s | 171 kB 00:00 2026-02-17T05:45:15.266 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction check 2026-02-17T05:45:15.270 INFO:teuthology.orchestra.run.trial115.stdout:Transaction check succeeded. 2026-02-17T05:45:15.270 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction test 2026-02-17T05:45:15.301 INFO:teuthology.orchestra.run.trial115.stdout:Transaction test succeeded. 2026-02-17T05:45:15.302 INFO:teuthology.orchestra.run.trial115.stdout:Running transaction 2026-02-17T05:45:15.316 INFO:teuthology.orchestra.run.trial078.stdout:Package librados2-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:15.334 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:15.334 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:15.335 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:15.357 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install librbd1 2026-02-17T05:45:15.358 INFO:teuthology.orchestra.run.trial115.stdout: Preparing : 1/1 2026-02-17T05:45:15.373 INFO:teuthology.orchestra.run.trial115.stdout: Installing : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:15.451 INFO:teuthology.orchestra.run.trial115.stdout: Running scriptlet: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:15.508 INFO:teuthology.orchestra.run.trial115.stdout: Verifying : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:15.508 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:15.509 INFO:teuthology.orchestra.run.trial115.stdout:Installed: 2026-02-17T05:45:15.509 INFO:teuthology.orchestra.run.trial115.stdout: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:15.509 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:15.509 INFO:teuthology.orchestra.run.trial115.stdout:Complete! 2026-02-17T05:45:15.541 DEBUG:teuthology.parallel:result is None 2026-02-17T05:45:15.667 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:15.733 INFO:teuthology.orchestra.run.trial078.stdout:Package librbd1-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:15.750 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:15.751 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:15.751 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:15.773 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install python3-rados 2026-02-17T05:45:16.085 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:16.148 INFO:teuthology.orchestra.run.trial078.stdout:Package python3-rados-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:16.166 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:16.166 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:16.167 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:16.189 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install python3-rgw 2026-02-17T05:45:16.496 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:16.561 INFO:teuthology.orchestra.run.trial078.stdout:Package python3-rgw-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:16.578 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:16.579 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:16.579 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:16.600 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install python3-cephfs 2026-02-17T05:45:16.910 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:36 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:16.975 INFO:teuthology.orchestra.run.trial078.stdout:Package python3-cephfs-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:16.993 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:16.993 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:16.993 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:17.015 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install python3-rbd 2026-02-17T05:45:17.322 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:17.387 INFO:teuthology.orchestra.run.trial078.stdout:Package python3-rbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 is already installed. 2026-02-17T05:45:17.404 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:17.405 INFO:teuthology.orchestra.run.trial078.stdout:Nothing to do. 2026-02-17T05:45:17.405 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:17.426 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install rbd-fuse 2026-02-17T05:45:17.742 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:37 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:17.825 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:17.826 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:17.826 INFO:teuthology.orchestra.run.trial078.stdout: Package Architecture Version Repository Size 2026-02-17T05:45:17.826 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:17.826 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:45:17.826 INFO:teuthology.orchestra.run.trial078.stdout: rbd-fuse x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 85 k 2026-02-17T05:45:17.826 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:17.827 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:45:17.827 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:17.827 INFO:teuthology.orchestra.run.trial078.stdout:Install 1 Package 2026-02-17T05:45:17.827 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:17.827 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 85 k 2026-02-17T05:45:17.827 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 227 k 2026-02-17T05:45:17.827 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:45:17.923 INFO:teuthology.orchestra.run.trial078.stdout:rbd-fuse-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 875 kB/s | 85 kB 00:00 2026-02-17T05:45:17.924 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:17.924 INFO:teuthology.orchestra.run.trial078.stdout:Total 867 kB/s | 85 kB 00:00 2026-02-17T05:45:17.924 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:45:17.927 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:45:17.927 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:45:17.958 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:45:17.958 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:45:18.010 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:45:18.027 INFO:teuthology.orchestra.run.trial078.stdout: Installing : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:18.104 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:18.165 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:18.166 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:18.166 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:45:18.166 INFO:teuthology.orchestra.run.trial078.stdout: rbd-fuse-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:18.166 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:18.166 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:18.194 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install rbd-mirror 2026-02-17T05:45:18.517 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:18.600 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:18.600 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:18.600 INFO:teuthology.orchestra.run.trial078.stdout: Package Arch Version Repo Size 2026-02-17T05:45:18.600 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:18.600 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:45:18.600 INFO:teuthology.orchestra.run.trial078.stdout: rbd-mirror x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 3.1 M 2026-02-17T05:45:18.600 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:18.600 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:45:18.601 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:18.601 INFO:teuthology.orchestra.run.trial078.stdout:Install 1 Package 2026-02-17T05:45:18.601 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:18.601 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 3.1 M 2026-02-17T05:45:18.601 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 13 M 2026-02-17T05:45:18.601 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:45:18.958 INFO:teuthology.orchestra.run.trial078.stdout:rbd-mirror-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 8.7 MB/s | 3.1 MB 00:00 2026-02-17T05:45:18.958 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:18.958 INFO:teuthology.orchestra.run.trial078.stdout:Total 8.7 MB/s | 3.1 MB 00:00 2026-02-17T05:45:18.958 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:45:18.962 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:45:18.962 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:45:19.004 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:45:19.004 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:45:19.206 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:45:19.208 INFO:teuthology.orchestra.run.trial078.stdout: Installing : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:19.231 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:19.232 INFO:teuthology.orchestra.run.trial078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T05:45:19.232 INFO:teuthology.orchestra.run.trial078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T05:45:19.232 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:45:19.232 INFO:teuthology.orchestra.run.trial078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T05:45:19.232 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:19.559 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:19.559 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:19.560 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:45:19.560 INFO:teuthology.orchestra.run.trial078.stdout: rbd-mirror-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:19.560 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:19.560 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:19.588 DEBUG:teuthology.orchestra.run.trial078:> sudo yum -y install rbd-nbd 2026-02-17T05:45:19.912 INFO:teuthology.orchestra.run.trial078.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Feb 2026 05:44:40 AM UTC. 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:Dependencies resolved. 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout: Package Architecture Version Repository Size 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:Installing: 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout: rbd-nbd x86_64 2:19.2.3-705.g04cf6dd4.el9 ceph 171 k 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:Transaction Summary 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:================================================================================ 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:Install 1 Package 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:Total download size: 171 k 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:Installed size: 490 k 2026-02-17T05:45:19.994 INFO:teuthology.orchestra.run.trial078.stdout:Downloading Packages: 2026-02-17T05:45:20.260 INFO:teuthology.orchestra.run.trial078.stdout:rbd-nbd-19.2.3-705.g04cf6dd4.el9.x86_64.rpm 647 kB/s | 171 kB 00:00 2026-02-17T05:45:20.261 INFO:teuthology.orchestra.run.trial078.stdout:-------------------------------------------------------------------------------- 2026-02-17T05:45:20.261 INFO:teuthology.orchestra.run.trial078.stdout:Total 644 kB/s | 171 kB 00:00 2026-02-17T05:45:20.261 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction check 2026-02-17T05:45:20.264 INFO:teuthology.orchestra.run.trial078.stdout:Transaction check succeeded. 2026-02-17T05:45:20.265 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction test 2026-02-17T05:45:20.295 INFO:teuthology.orchestra.run.trial078.stdout:Transaction test succeeded. 2026-02-17T05:45:20.295 INFO:teuthology.orchestra.run.trial078.stdout:Running transaction 2026-02-17T05:45:20.351 INFO:teuthology.orchestra.run.trial078.stdout: Preparing : 1/1 2026-02-17T05:45:20.367 INFO:teuthology.orchestra.run.trial078.stdout: Installing : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:20.448 INFO:teuthology.orchestra.run.trial078.stdout: Running scriptlet: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:20.505 INFO:teuthology.orchestra.run.trial078.stdout: Verifying : rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 1/1 2026-02-17T05:45:20.505 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:20.505 INFO:teuthology.orchestra.run.trial078.stdout:Installed: 2026-02-17T05:45:20.505 INFO:teuthology.orchestra.run.trial078.stdout: rbd-nbd-2:19.2.3-705.g04cf6dd4.el9.x86_64 2026-02-17T05:45:20.505 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:20.505 INFO:teuthology.orchestra.run.trial078.stdout:Complete! 2026-02-17T05:45:20.536 DEBUG:teuthology.parallel:result is None 2026-02-17T05:45:20.537 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:45:20.715 DEBUG:teuthology.orchestra.run.trial078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T05:45:20.736 INFO:teuthology.orchestra.run.trial078.stdout:19.2.3-705.g04cf6dd4.el9 2026-02-17T05:45:20.737 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-705.g04cf6dd4.el9 2026-02-17T05:45:20.737 INFO:teuthology.task.install:The correct ceph version 19.2.3-705.g04cf6dd4 is installed. 2026-02-17T05:45:20.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:45:20.887 DEBUG:teuthology.orchestra.run.trial115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T05:45:20.908 INFO:teuthology.orchestra.run.trial115.stdout:19.2.3-705.g04cf6dd4.el9 2026-02-17T05:45:20.908 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-705.g04cf6dd4.el9 2026-02-17T05:45:20.908 INFO:teuthology.task.install:The correct ceph version 19.2.3-705.g04cf6dd4 is installed. 2026-02-17T05:45:20.909 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T05:45:20.910 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:20.910 DEBUG:teuthology.orchestra.run.trial078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T05:45:20.937 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:20.938 DEBUG:teuthology.orchestra.run.trial115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T05:45:20.974 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T05:45:20.975 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:20.975 DEBUG:teuthology.orchestra.run.trial078:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T05:45:21.003 DEBUG:teuthology.orchestra.run.trial078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T05:45:21.068 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:21.068 DEBUG:teuthology.orchestra.run.trial115:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T05:45:21.094 DEBUG:teuthology.orchestra.run.trial115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T05:45:21.157 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T05:45:21.157 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:21.157 DEBUG:teuthology.orchestra.run.trial078:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T05:45:21.183 DEBUG:teuthology.orchestra.run.trial078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T05:45:21.246 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:21.246 DEBUG:teuthology.orchestra.run.trial115:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T05:45:21.269 DEBUG:teuthology.orchestra.run.trial115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T05:45:21.333 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T05:45:21.334 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:21.334 DEBUG:teuthology.orchestra.run.trial078:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T05:45:21.359 DEBUG:teuthology.orchestra.run.trial078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T05:45:21.422 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:21.422 DEBUG:teuthology.orchestra.run.trial115:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T05:45:21.448 DEBUG:teuthology.orchestra.run.trial115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T05:45:21.512 INFO:teuthology.run_tasks:Running task ceph... 2026-02-17T05:45:21.622 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-17T05:45:21.622 DEBUG:teuthology.orchestra.run.trial078:> sudo chmod 777 /var/log/ceph 2026-02-17T05:45:21.624 DEBUG:teuthology.orchestra.run.trial115:> sudo chmod 777 /var/log/ceph 2026-02-17T05:45:21.649 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-17T05:45:21.649 DEBUG:teuthology.orchestra.run.trial078:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-17T05:45:21.689 DEBUG:teuthology.orchestra.run.trial115:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-17T05:45:21.714 INFO:tasks.ceph:Creating extra log directories... 2026-02-17T05:45:21.714 DEBUG:teuthology.orchestra.run.trial078:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-17T05:45:21.753 DEBUG:teuthology.orchestra.run.trial115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-17T05:45:21.783 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-17T05:45:21.783 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-17T05:45:21.783 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059', 'branch': 'wip-bharath19-testing-2026-02-13-0903-squid', 'description': 'rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/one workloads/snaps-few-objects}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '51059', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath19-testing-2026-02-13-0903-squid'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath19-testing-2026-02-13-0903-squid', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'}}, 'owner': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 3603, 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath19-testing-2026-02-13-0903-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e', 'targets': {'trial078.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgwdKViKCYrGxwNdAmVRuhqhondgpFoGla/qr6gDvfH8kK6aW5nt3zxsBfGZCjZo3uOLclmxW23haLeTP94Fx8=', 'trial115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCbkb9SCDeN/JWz5/4NjaOdwJvaDHnC1NPlFLVpn44KgRwjhu/SogqSyZ2V6PYwLbDQ4NXUr/Bq4Ike+8C+JUzo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 20, 'thrash_delay': 1}}, {'rados': {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-17_04:03:08', 'tube': 'trial', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682'} 2026-02-17T05:45:21.783 DEBUG:teuthology.orchestra.run.trial078:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-17T05:45:21.815 DEBUG:teuthology.orchestra.run.trial115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-17T05:45:21.834 DEBUG:teuthology.orchestra.run.trial078:> sudo install -d -m0777 -- /var/run/ceph 2026-02-17T05:45:21.872 DEBUG:teuthology.orchestra.run.trial115:> sudo install -d -m0777 -- /var/run/ceph 2026-02-17T05:45:21.898 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:21.898 DEBUG:teuthology.orchestra.run.trial078:> dd if=/scratch_devs of=/dev/stdout 2026-02-17T05:45:21.950 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-17T05:45:21.950 DEBUG:teuthology.orchestra.run.trial078:> stat /dev/vg_nvme/lv_1 2026-02-17T05:45:22.007 INFO:teuthology.orchestra.run.trial078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-17T05:45:22.008 INFO:teuthology.orchestra.run.trial078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:45:22.008 INFO:teuthology.orchestra.run.trial078.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-17T05:45:22.008 INFO:teuthology.orchestra.run.trial078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:45:22.008 INFO:teuthology.orchestra.run.trial078.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:45:22.008 INFO:teuthology.orchestra.run.trial078.stdout:Access: 2026-02-17 05:45:19.382820120 +0000 2026-02-17T05:45:22.008 INFO:teuthology.orchestra.run.trial078.stdout:Modify: 2026-02-17 05:44:45.804986124 +0000 2026-02-17T05:45:22.008 INFO:teuthology.orchestra.run.trial078.stdout:Change: 2026-02-17 05:44:45.804986124 +0000 2026-02-17T05:45:22.008 INFO:teuthology.orchestra.run.trial078.stdout: Birth: 2026-02-17 05:44:45.804986124 +0000 2026-02-17T05:45:22.009 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-17T05:45:22.071 INFO:teuthology.orchestra.run.trial078.stderr:1+0 records in 2026-02-17T05:45:22.072 INFO:teuthology.orchestra.run.trial078.stderr:1+0 records out 2026-02-17T05:45:22.072 INFO:teuthology.orchestra.run.trial078.stderr:512 bytes copied, 0.000176259 s, 2.9 MB/s 2026-02-17T05:45:22.073 DEBUG:teuthology.orchestra.run.trial078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-17T05:45:22.130 DEBUG:teuthology.orchestra.run.trial078:> stat /dev/vg_nvme/lv_2 2026-02-17T05:45:22.187 INFO:teuthology.orchestra.run.trial078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-17T05:45:22.187 INFO:teuthology.orchestra.run.trial078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:45:22.187 INFO:teuthology.orchestra.run.trial078.stdout:Device: 6h/6d Inode: 1297 Links: 1 2026-02-17T05:45:22.187 INFO:teuthology.orchestra.run.trial078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:45:22.187 INFO:teuthology.orchestra.run.trial078.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:45:22.187 INFO:teuthology.orchestra.run.trial078.stdout:Access: 2026-02-17 05:45:19.383820114 +0000 2026-02-17T05:45:22.187 INFO:teuthology.orchestra.run.trial078.stdout:Modify: 2026-02-17 05:44:45.804986124 +0000 2026-02-17T05:45:22.188 INFO:teuthology.orchestra.run.trial078.stdout:Change: 2026-02-17 05:44:45.804986124 +0000 2026-02-17T05:45:22.188 INFO:teuthology.orchestra.run.trial078.stdout: Birth: 2026-02-17 05:44:45.804986124 +0000 2026-02-17T05:45:22.188 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-17T05:45:22.249 INFO:teuthology.orchestra.run.trial078.stderr:1+0 records in 2026-02-17T05:45:22.250 INFO:teuthology.orchestra.run.trial078.stderr:1+0 records out 2026-02-17T05:45:22.250 INFO:teuthology.orchestra.run.trial078.stderr:512 bytes copied, 0.000163335 s, 3.1 MB/s 2026-02-17T05:45:22.251 DEBUG:teuthology.orchestra.run.trial078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-17T05:45:22.308 DEBUG:teuthology.orchestra.run.trial078:> stat /dev/vg_nvme/lv_3 2026-02-17T05:45:22.365 INFO:teuthology.orchestra.run.trial078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-17T05:45:22.365 INFO:teuthology.orchestra.run.trial078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:45:22.365 INFO:teuthology.orchestra.run.trial078.stdout:Device: 6h/6d Inode: 1314 Links: 1 2026-02-17T05:45:22.365 INFO:teuthology.orchestra.run.trial078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:45:22.365 INFO:teuthology.orchestra.run.trial078.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:45:22.365 INFO:teuthology.orchestra.run.trial078.stdout:Access: 2026-02-17 05:45:19.383820114 +0000 2026-02-17T05:45:22.366 INFO:teuthology.orchestra.run.trial078.stdout:Modify: 2026-02-17 05:44:45.805986119 +0000 2026-02-17T05:45:22.366 INFO:teuthology.orchestra.run.trial078.stdout:Change: 2026-02-17 05:44:45.805986119 +0000 2026-02-17T05:45:22.366 INFO:teuthology.orchestra.run.trial078.stdout: Birth: 2026-02-17 05:44:45.805986119 +0000 2026-02-17T05:45:22.366 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-17T05:45:22.428 INFO:teuthology.orchestra.run.trial078.stderr:1+0 records in 2026-02-17T05:45:22.428 INFO:teuthology.orchestra.run.trial078.stderr:1+0 records out 2026-02-17T05:45:22.428 INFO:teuthology.orchestra.run.trial078.stderr:512 bytes copied, 0.000146233 s, 3.5 MB/s 2026-02-17T05:45:22.429 DEBUG:teuthology.orchestra.run.trial078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-17T05:45:22.487 DEBUG:teuthology.orchestra.run.trial078:> stat /dev/vg_nvme/lv_4 2026-02-17T05:45:22.542 INFO:teuthology.orchestra.run.trial078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-17T05:45:22.542 INFO:teuthology.orchestra.run.trial078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:45:22.542 INFO:teuthology.orchestra.run.trial078.stdout:Device: 6h/6d Inode: 1309 Links: 1 2026-02-17T05:45:22.543 INFO:teuthology.orchestra.run.trial078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:45:22.543 INFO:teuthology.orchestra.run.trial078.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:45:22.543 INFO:teuthology.orchestra.run.trial078.stdout:Access: 2026-02-17 05:45:19.383820114 +0000 2026-02-17T05:45:22.543 INFO:teuthology.orchestra.run.trial078.stdout:Modify: 2026-02-17 05:44:45.805986119 +0000 2026-02-17T05:45:22.543 INFO:teuthology.orchestra.run.trial078.stdout:Change: 2026-02-17 05:44:45.805986119 +0000 2026-02-17T05:45:22.543 INFO:teuthology.orchestra.run.trial078.stdout: Birth: 2026-02-17 05:44:45.805986119 +0000 2026-02-17T05:45:22.543 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-17T05:45:22.605 INFO:teuthology.orchestra.run.trial078.stderr:1+0 records in 2026-02-17T05:45:22.606 INFO:teuthology.orchestra.run.trial078.stderr:1+0 records out 2026-02-17T05:45:22.606 INFO:teuthology.orchestra.run.trial078.stderr:512 bytes copied, 0.000145211 s, 3.5 MB/s 2026-02-17T05:45:22.607 DEBUG:teuthology.orchestra.run.trial078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-17T05:45:22.665 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T05:45:22.665 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:22.665 DEBUG:teuthology.orchestra.run.trial115:> dd if=/scratch_devs of=/dev/stdout 2026-02-17T05:45:22.684 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-17T05:45:22.684 DEBUG:teuthology.orchestra.run.trial115:> stat /dev/vg_nvme/lv_1 2026-02-17T05:45:22.741 INFO:teuthology.orchestra.run.trial115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-17T05:45:22.742 INFO:teuthology.orchestra.run.trial115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:45:22.742 INFO:teuthology.orchestra.run.trial115.stdout:Device: 6h/6d Inode: 1322 Links: 1 2026-02-17T05:45:22.742 INFO:teuthology.orchestra.run.trial115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:45:22.742 INFO:teuthology.orchestra.run.trial115.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:45:22.742 INFO:teuthology.orchestra.run.trial115.stdout:Access: 2026-02-17 05:45:14.526792972 +0000 2026-02-17T05:45:22.742 INFO:teuthology.orchestra.run.trial115.stdout:Modify: 2026-02-17 05:44:39.080894653 +0000 2026-02-17T05:45:22.742 INFO:teuthology.orchestra.run.trial115.stdout:Change: 2026-02-17 05:44:39.080894653 +0000 2026-02-17T05:45:22.742 INFO:teuthology.orchestra.run.trial115.stdout: Birth: 2026-02-17 05:44:39.080894653 +0000 2026-02-17T05:45:22.743 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-17T05:45:22.804 INFO:teuthology.orchestra.run.trial115.stderr:1+0 records in 2026-02-17T05:45:22.804 INFO:teuthology.orchestra.run.trial115.stderr:1+0 records out 2026-02-17T05:45:22.805 INFO:teuthology.orchestra.run.trial115.stderr:512 bytes copied, 0.000154406 s, 3.3 MB/s 2026-02-17T05:45:22.806 DEBUG:teuthology.orchestra.run.trial115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-17T05:45:22.863 DEBUG:teuthology.orchestra.run.trial115:> stat /dev/vg_nvme/lv_2 2026-02-17T05:45:22.918 INFO:teuthology.orchestra.run.trial115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-17T05:45:22.918 INFO:teuthology.orchestra.run.trial115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:45:22.918 INFO:teuthology.orchestra.run.trial115.stdout:Device: 6h/6d Inode: 1314 Links: 1 2026-02-17T05:45:22.918 INFO:teuthology.orchestra.run.trial115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:45:22.918 INFO:teuthology.orchestra.run.trial115.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:45:22.918 INFO:teuthology.orchestra.run.trial115.stdout:Access: 2026-02-17 05:45:14.527792940 +0000 2026-02-17T05:45:22.918 INFO:teuthology.orchestra.run.trial115.stdout:Modify: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:22.918 INFO:teuthology.orchestra.run.trial115.stdout:Change: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:22.919 INFO:teuthology.orchestra.run.trial115.stdout: Birth: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:22.919 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-17T05:45:22.982 INFO:teuthology.orchestra.run.trial115.stderr:1+0 records in 2026-02-17T05:45:22.982 INFO:teuthology.orchestra.run.trial115.stderr:1+0 records out 2026-02-17T05:45:22.982 INFO:teuthology.orchestra.run.trial115.stderr:512 bytes copied, 0.000145009 s, 3.5 MB/s 2026-02-17T05:45:22.983 DEBUG:teuthology.orchestra.run.trial115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-17T05:45:23.040 DEBUG:teuthology.orchestra.run.trial115:> stat /dev/vg_nvme/lv_3 2026-02-17T05:45:23.096 INFO:teuthology.orchestra.run.trial115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-17T05:45:23.096 INFO:teuthology.orchestra.run.trial115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:45:23.096 INFO:teuthology.orchestra.run.trial115.stdout:Device: 6h/6d Inode: 1301 Links: 1 2026-02-17T05:45:23.096 INFO:teuthology.orchestra.run.trial115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:45:23.096 INFO:teuthology.orchestra.run.trial115.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:45:23.096 INFO:teuthology.orchestra.run.trial115.stdout:Access: 2026-02-17 05:45:14.527792940 +0000 2026-02-17T05:45:23.097 INFO:teuthology.orchestra.run.trial115.stdout:Modify: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:23.097 INFO:teuthology.orchestra.run.trial115.stdout:Change: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:23.097 INFO:teuthology.orchestra.run.trial115.stdout: Birth: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:23.097 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-17T05:45:23.159 INFO:teuthology.orchestra.run.trial115.stderr:1+0 records in 2026-02-17T05:45:23.159 INFO:teuthology.orchestra.run.trial115.stderr:1+0 records out 2026-02-17T05:45:23.159 INFO:teuthology.orchestra.run.trial115.stderr:512 bytes copied, 0.000162211 s, 3.2 MB/s 2026-02-17T05:45:23.160 DEBUG:teuthology.orchestra.run.trial115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-17T05:45:23.217 DEBUG:teuthology.orchestra.run.trial115:> stat /dev/vg_nvme/lv_4 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout:Device: 6h/6d Inode: 1313 Links: 1 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout:Access: 2026-02-17 05:45:14.527792940 +0000 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout:Modify: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout:Change: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:23.276 INFO:teuthology.orchestra.run.trial115.stdout: Birth: 2026-02-17 05:44:39.079894685 +0000 2026-02-17T05:45:23.277 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-17T05:45:23.340 INFO:teuthology.orchestra.run.trial115.stderr:1+0 records in 2026-02-17T05:45:23.340 INFO:teuthology.orchestra.run.trial115.stderr:1+0 records out 2026-02-17T05:45:23.340 INFO:teuthology.orchestra.run.trial115.stderr:512 bytes copied, 0.000142775 s, 3.6 MB/s 2026-02-17T05:45:23.342 DEBUG:teuthology.orchestra.run.trial115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-17T05:45:23.398 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2026-02-17T05:45:23.399 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial078.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@trial115.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2026-02-17T05:45:23.399 INFO:tasks.ceph:Generating config... 2026-02-17T05:45:23.402 INFO:tasks.ceph:[client] debug monc = 20 2026-02-17T05:45:23.402 INFO:tasks.ceph:[client] debug ms = 1 2026-02-17T05:45:23.402 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-17T05:45:23.402 INFO:tasks.ceph:[global] mon election default strategy = 1 2026-02-17T05:45:23.402 INFO:tasks.ceph:[global] ms inject delay max = 1 2026-02-17T05:45:23.402 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2026-02-17T05:45:23.402 INFO:tasks.ceph:[global] ms inject delay type = mon 2026-02-17T05:45:23.402 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2026-02-17T05:45:23.403 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2026-02-17T05:45:23.403 INFO:tasks.ceph:[global] ms type = async 2026-02-17T05:45:23.403 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-17T05:45:23.403 INFO:tasks.ceph:[mgr] debug monc = 10 2026-02-17T05:45:23.403 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-17T05:45:23.403 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-17T05:45:23.403 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-17T05:45:23.403 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-17T05:45:23.403 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2026-02-17T05:45:23.403 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-17T05:45:23.404 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-17T05:45:23.404 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-17T05:45:23.404 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2026-02-17T05:45:23.404 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2026-02-17T05:45:23.404 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-17T05:45:23.404 INFO:tasks.ceph:[mon] paxos service trim min = 5 2026-02-17T05:45:23.404 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-17T05:45:23.404 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2026-02-17T05:45:23.404 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2026-02-17T05:45:23.404 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-17T05:45:23.404 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-17T05:45:23.404 INFO:tasks.ceph:[osd] debug bluefs = 20 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] debug bluestore = 20 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-17T05:45:23.405 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-17T05:45:23.406 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-17T05:45:23.406 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-17T05:45:23.406 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-17T05:45:23.406 INFO:tasks.ceph:Setting up mon.a... 2026-02-17T05:45:23.406 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-17T05:45:23.441 INFO:teuthology.orchestra.run.trial078.stdout:creating /etc/ceph/ceph.keyring 2026-02-17T05:45:23.444 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-17T05:45:23.521 DEBUG:teuthology.orchestra.run.trial078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T05:45:23.584 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.78'), ('mon.c', '[v2:10.20.193.78:3301,v1:10.20.193.78:6790]'), ('mon.b', '10.20.193.115')] 2026-02-17T05:45:23.585 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.78,[v2:10.20.193.78:3301,v1:10.20.193.78:6790],10.20.193.115', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug monc': 10}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2026-02-17T05:45:23.585 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:23.585 DEBUG:teuthology.orchestra.run.trial078:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-17T05:45:23.641 DEBUG:teuthology.orchestra.run.trial078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.78 --addv c '[v2:10.20.193.78:3301,v1:10.20.193.78:6790]' --add b 10.20.193.115 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:23.713 INFO:teuthology.orchestra.run.trial078.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:23.713 INFO:teuthology.orchestra.run.trial078.stdout:monmaptool: generated fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:23.713 INFO:teuthology.orchestra.run.trial078.stdout:setting min_mon_release = quincy 2026-02-17T05:45:23.713 INFO:teuthology.orchestra.run.trial078.stdout:epoch 0 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:min_mon_release 17 (quincy) 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:election_strategy: 1 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:23.714 INFO:teuthology.orchestra.run.trial078.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-17T05:45:23.714 DEBUG:teuthology.orchestra.run.trial078:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-17T05:45:23.771 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5df7382e-3ebf-45af-b855-6719898a3119... 2026-02-17T05:45:23.772 DEBUG:teuthology.orchestra.run.trial078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-17T05:45:23.814 DEBUG:teuthology.orchestra.run.trial115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-17T05:45:23.853 INFO:teuthology.orchestra.run.trial078.stdout:[global] 2026-02-17T05:45:23.853 INFO:teuthology.orchestra.run.trial078.stdout: chdir = "" 2026-02-17T05:45:23.853 INFO:teuthology.orchestra.run.trial078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-17T05:45:23.853 INFO:teuthology.orchestra.run.trial078.stdout: auth supported = cephx 2026-02-17T05:45:23.854 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.854 INFO:teuthology.orchestra.run.trial078.stdout: filestore xattr use omap = true 2026-02-17T05:45:23.854 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.854 INFO:teuthology.orchestra.run.trial078.stdout: mon clock drift allowed = 1.000 2026-02-17T05:45:23.854 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.854 INFO:teuthology.orchestra.run.trial078.stdout: osd crush chooseleaf type = 0 2026-02-17T05:45:23.855 INFO:teuthology.orchestra.run.trial078.stdout: auth debug = true 2026-02-17T05:45:23.855 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.855 INFO:teuthology.orchestra.run.trial078.stdout: ms die on old message = true 2026-02-17T05:45:23.855 INFO:teuthology.orchestra.run.trial078.stdout: ms die on bug = true 2026-02-17T05:45:23.855 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.855 INFO:teuthology.orchestra.run.trial078.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-17T05:45:23.855 INFO:teuthology.orchestra.run.trial078.stdout: mon pg warn max object skew = 0 2026-02-17T05:45:23.856 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.856 INFO:teuthology.orchestra.run.trial078.stdout: # disable pg_autoscaler by default for new pools 2026-02-17T05:45:23.856 INFO:teuthology.orchestra.run.trial078.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-17T05:45:23.856 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.856 INFO:teuthology.orchestra.run.trial078.stdout: osd pool default size = 2 2026-02-17T05:45:23.856 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.857 INFO:teuthology.orchestra.run.trial078.stdout: mon osd allow primary affinity = true 2026-02-17T05:45:23.857 INFO:teuthology.orchestra.run.trial078.stdout: mon osd allow pg remap = true 2026-02-17T05:45:23.857 INFO:teuthology.orchestra.run.trial078.stdout: mon warn on legacy crush tunables = false 2026-02-17T05:45:23.857 INFO:teuthology.orchestra.run.trial078.stdout: mon warn on crush straw calc version zero = false 2026-02-17T05:45:23.857 INFO:teuthology.orchestra.run.trial078.stdout: mon warn on no sortbitwise = false 2026-02-17T05:45:23.857 INFO:teuthology.orchestra.run.trial078.stdout: mon warn on osd down out interval zero = false 2026-02-17T05:45:23.858 INFO:teuthology.orchestra.run.trial078.stdout: mon warn on too few osds = false 2026-02-17T05:45:23.858 INFO:teuthology.orchestra.run.trial078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-17T05:45:23.858 INFO:teuthology.orchestra.run.trial078.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-17T05:45:23.858 INFO:teuthology.orchestra.run.trial078.stdout: mon_allow_pool_size_one = true 2026-02-17T05:45:23.858 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.858 INFO:teuthology.orchestra.run.trial078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-17T05:45:23.859 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.859 INFO:teuthology.orchestra.run.trial078.stdout: osd default data pool replay window = 5 2026-02-17T05:45:23.859 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.859 INFO:teuthology.orchestra.run.trial078.stdout: mon allow pool delete = true 2026-02-17T05:45:23.859 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.859 INFO:teuthology.orchestra.run.trial078.stdout: mon cluster log file level = debug 2026-02-17T05:45:23.860 INFO:teuthology.orchestra.run.trial078.stdout: debug asserts on shutdown = true 2026-02-17T05:45:23.860 INFO:teuthology.orchestra.run.trial078.stdout: mon health detail to clog = false 2026-02-17T05:45:23.860 INFO:teuthology.orchestra.run.trial078.stdout: mon host = "10.20.193.78,[v2:10.20.193.78:3301,v1:10.20.193.78:6790],10.20.193.115" 2026-02-17T05:45:23.860 INFO:teuthology.orchestra.run.trial078.stdout: mon client directed command retry = 5 2026-02-17T05:45:23.860 INFO:teuthology.orchestra.run.trial078.stdout: mon election default strategy = 1 2026-02-17T05:45:23.860 INFO:teuthology.orchestra.run.trial078.stdout: ms inject delay max = 1 2026-02-17T05:45:23.861 INFO:teuthology.orchestra.run.trial078.stdout: ms inject delay probability = 0.005 2026-02-17T05:45:23.861 INFO:teuthology.orchestra.run.trial078.stdout: ms inject delay type = mon 2026-02-17T05:45:23.861 INFO:teuthology.orchestra.run.trial078.stdout: ms inject internal delays = 0.002 2026-02-17T05:45:23.861 INFO:teuthology.orchestra.run.trial078.stdout: ms inject socket failures = 2500 2026-02-17T05:45:23.861 INFO:teuthology.orchestra.run.trial078.stdout: ms type = async 2026-02-17T05:45:23.861 INFO:teuthology.orchestra.run.trial078.stdout: fsid = 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:23.862 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.862 INFO:teuthology.orchestra.run.trial078.stdout:[osd] 2026-02-17T05:45:23.862 INFO:teuthology.orchestra.run.trial078.stdout: osd journal size = 100 2026-02-17T05:45:23.862 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.862 INFO:teuthology.orchestra.run.trial078.stdout: osd scrub load threshold = 5.0 2026-02-17T05:45:23.862 INFO:teuthology.orchestra.run.trial078.stdout: osd scrub max interval = 600 2026-02-17T05:45:23.863 INFO:teuthology.orchestra.run.trial078.stdout: osd mclock profile = high_recovery_ops 2026-02-17T05:45:23.863 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.863 INFO:teuthology.orchestra.run.trial078.stdout: osd recover clone overlap = true 2026-02-17T05:45:23.863 INFO:teuthology.orchestra.run.trial078.stdout: osd recovery max chunk = 1048576 2026-02-17T05:45:23.863 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.863 INFO:teuthology.orchestra.run.trial078.stdout: osd debug shutdown = true 2026-02-17T05:45:23.863 INFO:teuthology.orchestra.run.trial078.stdout: osd debug op order = true 2026-02-17T05:45:23.864 INFO:teuthology.orchestra.run.trial078.stdout: osd debug verify stray on activate = true 2026-02-17T05:45:23.864 INFO:teuthology.orchestra.run.trial078.stdout: osd debug trim objects = true 2026-02-17T05:45:23.864 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.864 INFO:teuthology.orchestra.run.trial078.stdout: osd open classes on start = true 2026-02-17T05:45:23.864 INFO:teuthology.orchestra.run.trial078.stdout: osd debug pg log writeout = true 2026-02-17T05:45:23.864 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.864 INFO:teuthology.orchestra.run.trial078.stdout: osd deep scrub update digest min age = 30 2026-02-17T05:45:23.865 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.865 INFO:teuthology.orchestra.run.trial078.stdout: osd map max advance = 10 2026-02-17T05:45:23.865 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.865 INFO:teuthology.orchestra.run.trial078.stdout: journal zero on create = true 2026-02-17T05:45:23.865 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.865 INFO:teuthology.orchestra.run.trial078.stdout: filestore ondisk finisher threads = 3 2026-02-17T05:45:23.865 INFO:teuthology.orchestra.run.trial078.stdout: filestore apply finisher threads = 3 2026-02-17T05:45:23.865 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.866 INFO:teuthology.orchestra.run.trial078.stdout: bdev debug aio = true 2026-02-17T05:45:23.866 INFO:teuthology.orchestra.run.trial078.stdout: osd debug misdirected ops = true 2026-02-17T05:45:23.866 INFO:teuthology.orchestra.run.trial078.stdout: bluestore block size = 96636764160 2026-02-17T05:45:23.866 INFO:teuthology.orchestra.run.trial078.stdout: bluestore compression algorithm = snappy 2026-02-17T05:45:23.866 INFO:teuthology.orchestra.run.trial078.stdout: bluestore compression mode = aggressive 2026-02-17T05:45:23.866 INFO:teuthology.orchestra.run.trial078.stdout: bluestore fsck on mount = True 2026-02-17T05:45:23.866 INFO:teuthology.orchestra.run.trial078.stdout: bluestore zero block detection = True 2026-02-17T05:45:23.867 INFO:teuthology.orchestra.run.trial078.stdout: debug bluefs = 20 2026-02-17T05:45:23.867 INFO:teuthology.orchestra.run.trial078.stdout: debug bluestore = 20 2026-02-17T05:45:23.867 INFO:teuthology.orchestra.run.trial078.stdout: debug ms = 1 2026-02-17T05:45:23.867 INFO:teuthology.orchestra.run.trial078.stdout: debug osd = 20 2026-02-17T05:45:23.867 INFO:teuthology.orchestra.run.trial078.stdout: debug rocksdb = 10 2026-02-17T05:45:23.867 INFO:teuthology.orchestra.run.trial078.stdout: mon osd backfillfull_ratio = 0.85 2026-02-17T05:45:23.867 INFO:teuthology.orchestra.run.trial078.stdout: mon osd full ratio = 0.9 2026-02-17T05:45:23.868 INFO:teuthology.orchestra.run.trial078.stdout: mon osd nearfull ratio = 0.8 2026-02-17T05:45:23.868 INFO:teuthology.orchestra.run.trial078.stdout: osd debug verify cached snaps = True 2026-02-17T05:45:23.868 INFO:teuthology.orchestra.run.trial078.stdout: osd debug verify missing on start = True 2026-02-17T05:45:23.868 INFO:teuthology.orchestra.run.trial078.stdout: osd failsafe full ratio = 0.95 2026-02-17T05:45:23.868 INFO:teuthology.orchestra.run.trial078.stdout: osd mclock override recovery settings = True 2026-02-17T05:45:23.868 INFO:teuthology.orchestra.run.trial078.stdout: osd objectstore = bluestore 2026-02-17T05:45:23.868 INFO:teuthology.orchestra.run.trial078.stdout: osd op queue = debug_random 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: osd op queue cut off = debug_random 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout:[mgr] 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: debug ms = 1 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: debug mgr = 20 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: debug mon = 20 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: debug auth = 20 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: mgr/telemetry/nag = false 2026-02-17T05:45:23.869 INFO:teuthology.orchestra.run.trial078.stdout: debug monc = 10 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout:[mon] 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: debug ms = 1 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: debug mon = 20 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: debug paxos = 20 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: debug auth = 20 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: mon data avail warn = 5 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: mon mgr mkfs grace = 240 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: mon osd reporter subtree level = osd 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: mon osd prime pg temp = true 2026-02-17T05:45:23.870 INFO:teuthology.orchestra.run.trial078.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: auth mon ticket ttl = 660 # 11m 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: auth service ticket ttl = 240 # 4m 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: # don't complain about insecure global_id in the test suite 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: # 1m isn't quite enough 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: mon_down_mkfs_grace = 2m 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.871 INFO:teuthology.orchestra.run.trial078.stdout: mon_warn_on_filestore_osds = false 2026-02-17T05:45:23.872 INFO:teuthology.orchestra.run.trial078.stdout: mon min osdmap epochs = 25 2026-02-17T05:45:23.872 INFO:teuthology.orchestra.run.trial078.stdout: mon osdmap full prune interval = 2 2026-02-17T05:45:23.872 INFO:teuthology.orchestra.run.trial078.stdout: mon osdmap full prune min = 15 2026-02-17T05:45:23.872 INFO:teuthology.orchestra.run.trial078.stdout: mon osdmap full prune txsize = 2 2026-02-17T05:45:23.872 INFO:teuthology.orchestra.run.trial078.stdout: mon scrub inject crc mismatch = 0.01 2026-02-17T05:45:23.872 INFO:teuthology.orchestra.run.trial078.stdout: mon scrub inject missing keys = 0.05 2026-02-17T05:45:23.872 INFO:teuthology.orchestra.run.trial078.stdout: mon scrub interval = 300 2026-02-17T05:45:23.872 INFO:teuthology.orchestra.run.trial078.stdout: paxos service trim min = 5 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout:[client] 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout: rgw cache enabled = true 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout: rgw enable ops log = true 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout: rgw enable usage log = true 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout: debug monc = 20 2026-02-17T05:45:23.873 INFO:teuthology.orchestra.run.trial078.stdout: debug ms = 1 2026-02-17T05:45:23.874 INFO:teuthology.orchestra.run.trial078.stdout:[mon.a] 2026-02-17T05:45:23.874 INFO:teuthology.orchestra.run.trial078.stdout:[mon.c] 2026-02-17T05:45:23.874 INFO:teuthology.orchestra.run.trial078.stdout:[mon.b] 2026-02-17T05:45:23.876 INFO:teuthology.orchestra.run.trial115.stdout:[global] 2026-02-17T05:45:23.876 INFO:teuthology.orchestra.run.trial115.stdout: chdir = "" 2026-02-17T05:45:23.876 INFO:teuthology.orchestra.run.trial115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-17T05:45:23.876 INFO:teuthology.orchestra.run.trial115.stdout: auth supported = cephx 2026-02-17T05:45:23.876 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.876 INFO:teuthology.orchestra.run.trial115.stdout: filestore xattr use omap = true 2026-02-17T05:45:23.876 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: mon clock drift allowed = 1.000 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: osd crush chooseleaf type = 0 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: auth debug = true 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: ms die on old message = true 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: ms die on bug = true 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: mon pg warn max object skew = 0 2026-02-17T05:45:23.877 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: # disable pg_autoscaler by default for new pools 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: osd pool default size = 2 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: mon osd allow primary affinity = true 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: mon osd allow pg remap = true 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: mon warn on legacy crush tunables = false 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: mon warn on crush straw calc version zero = false 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: mon warn on no sortbitwise = false 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: mon warn on osd down out interval zero = false 2026-02-17T05:45:23.878 INFO:teuthology.orchestra.run.trial115.stdout: mon warn on too few osds = false 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: mon_allow_pool_size_one = true 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: osd default data pool replay window = 5 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: mon allow pool delete = true 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: mon cluster log file level = debug 2026-02-17T05:45:23.879 INFO:teuthology.orchestra.run.trial115.stdout: debug asserts on shutdown = true 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: mon health detail to clog = false 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: mon host = "10.20.193.78,[v2:10.20.193.78:3301,v1:10.20.193.78:6790],10.20.193.115" 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: mon client directed command retry = 5 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: mon election default strategy = 1 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: ms inject delay max = 1 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: ms inject delay probability = 0.005 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: ms inject delay type = mon 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: ms inject internal delays = 0.002 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: ms inject socket failures = 2500 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: ms type = async 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: fsid = 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.880 INFO:teuthology.orchestra.run.trial115.stdout:[osd] 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd journal size = 100 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd scrub load threshold = 5.0 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd scrub max interval = 600 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd mclock profile = high_recovery_ops 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd recover clone overlap = true 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd recovery max chunk = 1048576 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd debug shutdown = true 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd debug op order = true 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd debug verify stray on activate = true 2026-02-17T05:45:23.881 INFO:teuthology.orchestra.run.trial115.stdout: osd debug trim objects = true 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: osd open classes on start = true 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: osd debug pg log writeout = true 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: osd deep scrub update digest min age = 30 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: osd map max advance = 10 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: journal zero on create = true 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: filestore ondisk finisher threads = 3 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: filestore apply finisher threads = 3 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.882 INFO:teuthology.orchestra.run.trial115.stdout: bdev debug aio = true 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: osd debug misdirected ops = true 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: bluestore block size = 96636764160 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: bluestore compression algorithm = snappy 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: bluestore compression mode = aggressive 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: bluestore fsck on mount = True 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: bluestore zero block detection = True 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: debug bluefs = 20 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: debug bluestore = 20 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: debug ms = 1 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: debug osd = 20 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: debug rocksdb = 10 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: mon osd backfillfull_ratio = 0.85 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: mon osd full ratio = 0.9 2026-02-17T05:45:23.883 INFO:teuthology.orchestra.run.trial115.stdout: mon osd nearfull ratio = 0.8 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: osd debug verify cached snaps = True 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: osd debug verify missing on start = True 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: osd failsafe full ratio = 0.95 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: osd mclock override recovery settings = True 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: osd objectstore = bluestore 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: osd op queue = debug_random 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: osd op queue cut off = debug_random 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout:[mgr] 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: debug ms = 1 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: debug mgr = 20 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: debug mon = 20 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: debug auth = 20 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:45:23.884 INFO:teuthology.orchestra.run.trial115.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: mgr/telemetry/nag = false 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: debug monc = 10 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout:[mon] 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: debug ms = 1 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: debug mon = 20 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: debug paxos = 20 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: debug auth = 20 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: mon data avail warn = 5 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: mon mgr mkfs grace = 240 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: mon reweight min pgs per osd = 4 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: mon osd reporter subtree level = osd 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: mon osd prime pg temp = true 2026-02-17T05:45:23.885 INFO:teuthology.orchestra.run.trial115.stdout: mon reweight min bytes per osd = 10 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: auth mon ticket ttl = 660 # 11m 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: auth service ticket ttl = 240 # 4m 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: # don't complain about insecure global_id in the test suite 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: # 1m isn't quite enough 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: mon_down_mkfs_grace = 2m 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: mon_warn_on_filestore_osds = false 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: mon min osdmap epochs = 25 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: mon osdmap full prune interval = 2 2026-02-17T05:45:23.886 INFO:teuthology.orchestra.run.trial115.stdout: mon osdmap full prune min = 15 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: mon osdmap full prune txsize = 2 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: mon scrub inject crc mismatch = 0.01 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: mon scrub inject missing keys = 0.05 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: mon scrub interval = 300 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: paxos service trim min = 5 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout:[client] 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: rgw cache enabled = true 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: rgw enable ops log = true 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: rgw enable usage log = true 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: debug monc = 20 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout: debug ms = 1 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout:[mon.a] 2026-02-17T05:45:23.887 INFO:teuthology.orchestra.run.trial115.stdout:[mon.c] 2026-02-17T05:45:23.888 INFO:teuthology.orchestra.run.trial115.stdout:[mon.b] 2026-02-17T05:45:23.888 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-17T05:45:23.888 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-17T05:45:23.958 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-17T05:45:23.958 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:23.959 DEBUG:teuthology.orchestra.run.trial078:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-17T05:45:24.016 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:24.016 DEBUG:teuthology.orchestra.run.trial078:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-17T05:45:24.074 INFO:tasks.ceph:Sending monmap to node ubuntu@trial078.front.sepia.ceph.com 2026-02-17T05:45:24.074 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:24.074 DEBUG:teuthology.orchestra.run.trial078:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-17T05:45:24.074 DEBUG:teuthology.orchestra.run.trial078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T05:45:24.145 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:24.146 DEBUG:teuthology.orchestra.run.trial078:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:24.201 INFO:tasks.ceph:Sending monmap to node ubuntu@trial115.front.sepia.ceph.com 2026-02-17T05:45:24.201 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:24.201 DEBUG:teuthology.orchestra.run.trial115:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-17T05:45:24.201 DEBUG:teuthology.orchestra.run.trial115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T05:45:24.234 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:24.234 DEBUG:teuthology.orchestra.run.trial115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:24.290 INFO:tasks.ceph:Setting up mon nodes... 2026-02-17T05:45:24.291 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-17T05:45:24.291 DEBUG:teuthology.orchestra.run.trial115:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-17T05:45:24.376 INFO:teuthology.orchestra.run.trial115.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-17T05:45:24.379 INFO:tasks.ceph:Setting up mds nodes... 2026-02-17T05:45:24.379 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-17T05:45:24.379 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-17T05:45:24.414 INFO:teuthology.orchestra.run.trial115.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-17T05:45:24.425 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-17T05:45:24.426 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial078.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@trial115.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2026-02-17T05:45:24.426 DEBUG:teuthology.orchestra.run.trial078:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-17T05:45:24.451 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T05:45:24.451 INFO:tasks.ceph:role: osd.0 2026-02-17T05:45:24.451 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial078.front.sepia.ceph.com 2026-02-17T05:45:24.451 DEBUG:teuthology.orchestra.run.trial078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:45:24.516 INFO:teuthology.orchestra.run.trial078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:45:25.087 INFO:teuthology.orchestra.run.trial078.stdout:Discarding blocks...Done. 2026-02-17T05:45:25.089 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial078.front.sepia.ceph.com -o noatime 2026-02-17T05:45:25.089 DEBUG:teuthology.orchestra.run.trial078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-17T05:45:25.159 DEBUG:teuthology.orchestra.run.trial078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-17T05:45:25.228 DEBUG:teuthology.orchestra.run.trial078:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-17T05:45:25.295 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T05:45:25.295 INFO:tasks.ceph:role: osd.1 2026-02-17T05:45:25.295 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial078.front.sepia.ceph.com 2026-02-17T05:45:25.296 DEBUG:teuthology.orchestra.run.trial078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-17T05:45:25.360 INFO:teuthology.orchestra.run.trial078.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:45:25.360 INFO:teuthology.orchestra.run.trial078.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:45:25.360 INFO:teuthology.orchestra.run.trial078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:45:25.360 INFO:teuthology.orchestra.run.trial078.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:45:25.361 INFO:teuthology.orchestra.run.trial078.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:45:25.361 INFO:teuthology.orchestra.run.trial078.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:45:25.361 INFO:teuthology.orchestra.run.trial078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:45:25.361 INFO:teuthology.orchestra.run.trial078.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:45:25.361 INFO:teuthology.orchestra.run.trial078.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:45:25.361 INFO:teuthology.orchestra.run.trial078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:45:25.922 INFO:teuthology.orchestra.run.trial078.stdout:Discarding blocks...Done. 2026-02-17T05:45:25.923 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial078.front.sepia.ceph.com -o noatime 2026-02-17T05:45:25.924 DEBUG:teuthology.orchestra.run.trial078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-17T05:45:25.991 DEBUG:teuthology.orchestra.run.trial078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-17T05:45:26.058 DEBUG:teuthology.orchestra.run.trial078:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-17T05:45:26.123 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T05:45:26.123 INFO:tasks.ceph:role: osd.2 2026-02-17T05:45:26.123 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial078.front.sepia.ceph.com 2026-02-17T05:45:26.123 DEBUG:teuthology.orchestra.run.trial078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-17T05:45:26.190 INFO:teuthology.orchestra.run.trial078.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:45:26.190 INFO:teuthology.orchestra.run.trial078.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:45:26.190 INFO:teuthology.orchestra.run.trial078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:45:26.190 INFO:teuthology.orchestra.run.trial078.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:45:26.190 INFO:teuthology.orchestra.run.trial078.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:45:26.190 INFO:teuthology.orchestra.run.trial078.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:45:26.190 INFO:teuthology.orchestra.run.trial078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:45:26.190 INFO:teuthology.orchestra.run.trial078.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:45:26.191 INFO:teuthology.orchestra.run.trial078.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:45:26.191 INFO:teuthology.orchestra.run.trial078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:45:26.750 INFO:teuthology.orchestra.run.trial078.stdout:Discarding blocks...Done. 2026-02-17T05:45:26.752 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial078.front.sepia.ceph.com -o noatime 2026-02-17T05:45:26.752 DEBUG:teuthology.orchestra.run.trial078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-17T05:45:26.820 DEBUG:teuthology.orchestra.run.trial078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-17T05:45:26.886 DEBUG:teuthology.orchestra.run.trial078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:26.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:26.966+0000 7f63ff8fe780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-17T05:45:26.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:26.966+0000 7f63ff8fe780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-17T05:45:26.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:26.966+0000 7f63ff8fe780 -1 bdev(0x55ae4eab1800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:26.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:26.966+0000 7f63ff8fe780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-17T05:45:29.867 DEBUG:teuthology.orchestra.run.trial078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-17T05:45:29.935 DEBUG:teuthology.orchestra.run.trial078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:30.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:30.011+0000 7fd603412780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-17T05:45:30.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:30.012+0000 7fd603412780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-17T05:45:30.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:30.012+0000 7fd603412780 -1 bdev(0x55f7dd3a9800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:30.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:30.012+0000 7fd603412780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-17T05:45:33.163 DEBUG:teuthology.orchestra.run.trial078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-17T05:45:33.228 DEBUG:teuthology.orchestra.run.trial078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:33.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:33.305+0000 7ff90d7a3780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-17T05:45:33.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:33.305+0000 7ff90d7a3780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-17T05:45:33.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:33.305+0000 7ff90d7a3780 -1 bdev(0x5603c1ddb800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:33.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:33.305+0000 7ff90d7a3780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-17T05:45:36.460 DEBUG:teuthology.orchestra.run.trial078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-17T05:45:36.525 DEBUG:teuthology.orchestra.run.trial115:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-17T05:45:36.551 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2026-02-17T05:45:36.551 INFO:tasks.ceph:role: osd.3 2026-02-17T05:45:36.551 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial115.front.sepia.ceph.com 2026-02-17T05:45:36.552 DEBUG:teuthology.orchestra.run.trial115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-17T05:45:36.616 INFO:teuthology.orchestra.run.trial115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:45:36.616 INFO:teuthology.orchestra.run.trial115.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:45:36.616 INFO:teuthology.orchestra.run.trial115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:45:36.616 INFO:teuthology.orchestra.run.trial115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:45:36.616 INFO:teuthology.orchestra.run.trial115.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:45:36.616 INFO:teuthology.orchestra.run.trial115.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:45:36.617 INFO:teuthology.orchestra.run.trial115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:45:36.617 INFO:teuthology.orchestra.run.trial115.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:45:36.617 INFO:teuthology.orchestra.run.trial115.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:45:36.617 INFO:teuthology.orchestra.run.trial115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:45:37.186 INFO:teuthology.orchestra.run.trial115.stdout:Discarding blocks...Done. 2026-02-17T05:45:37.188 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial115.front.sepia.ceph.com -o noatime 2026-02-17T05:45:37.188 DEBUG:teuthology.orchestra.run.trial115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2026-02-17T05:45:37.257 DEBUG:teuthology.orchestra.run.trial115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-17T05:45:37.324 DEBUG:teuthology.orchestra.run.trial115:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-17T05:45:37.388 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2026-02-17T05:45:37.388 INFO:tasks.ceph:role: osd.4 2026-02-17T05:45:37.388 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial115.front.sepia.ceph.com 2026-02-17T05:45:37.388 DEBUG:teuthology.orchestra.run.trial115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:45:37.453 INFO:teuthology.orchestra.run.trial115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:45:38.011 INFO:teuthology.orchestra.run.trial115.stdout:Discarding blocks...Done. 2026-02-17T05:45:38.013 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial115.front.sepia.ceph.com -o noatime 2026-02-17T05:45:38.014 DEBUG:teuthology.orchestra.run.trial115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2026-02-17T05:45:38.086 DEBUG:teuthology.orchestra.run.trial115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2026-02-17T05:45:38.155 DEBUG:teuthology.orchestra.run.trial115:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-17T05:45:38.220 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2026-02-17T05:45:38.221 INFO:tasks.ceph:role: osd.5 2026-02-17T05:45:38.221 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial115.front.sepia.ceph.com 2026-02-17T05:45:38.221 DEBUG:teuthology.orchestra.run.trial115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout: = sunit=0 swidth=0 blks 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T05:45:38.285 INFO:teuthology.orchestra.run.trial115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T05:45:38.845 INFO:teuthology.orchestra.run.trial115.stdout:Discarding blocks...Done. 2026-02-17T05:45:38.847 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial115.front.sepia.ceph.com -o noatime 2026-02-17T05:45:38.847 DEBUG:teuthology.orchestra.run.trial115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2026-02-17T05:45:38.920 DEBUG:teuthology.orchestra.run.trial115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2026-02-17T05:45:38.989 DEBUG:teuthology.orchestra.run.trial115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:39.066 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:39.065+0000 7f93c5007780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-17T05:45:39.066 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:39.065+0000 7f93c5007780 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-17T05:45:39.067 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:39.065+0000 7f93c5007780 -1 bdev(0x558a3c023800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:39.067 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:39.066+0000 7f93c5007780 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-17T05:45:42.219 DEBUG:teuthology.orchestra.run.trial115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-17T05:45:42.284 DEBUG:teuthology.orchestra.run.trial115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:42.362 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:42.360+0000 7fdd5eb84780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-17T05:45:42.362 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:42.360+0000 7fdd5eb84780 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-17T05:45:42.363 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:42.360+0000 7fdd5eb84780 -1 bdev(0x55aaa415f800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:42.363 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:42.361+0000 7fdd5eb84780 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-17T05:45:45.261 DEBUG:teuthology.orchestra.run.trial115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-17T05:45:45.326 DEBUG:teuthology.orchestra.run.trial115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:45.404 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:45.402+0000 7fdca4c0f780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-17T05:45:45.404 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:45.402+0000 7fdca4c0f780 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-17T05:45:45.405 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:45.402+0000 7fdca4c0f780 -1 bdev(0x563d858c9800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:45.405 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:45.403+0000 7fdca4c0f780 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-17T05:45:48.303 DEBUG:teuthology.orchestra.run.trial115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-17T05:45:48.369 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-17T05:45:48.369 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:48.369 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-17T05:45:48.396 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:48.396 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-17T05:45:48.460 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:48.460 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-17T05:45:48.524 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:48.524 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-17T05:45:48.550 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:48.550 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-17T05:45:48.614 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:48.614 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-17T05:45:48.677 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:48.677 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-17T05:45:48.741 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:48.741 DEBUG:teuthology.orchestra.run.trial115:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-17T05:45:48.797 INFO:tasks.ceph:Adding keys to all mons... 2026-02-17T05:45:48.797 DEBUG:teuthology.orchestra.run.trial078:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-17T05:45:48.800 DEBUG:teuthology.orchestra.run.trial115:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-17T05:45:48.841 INFO:teuthology.orchestra.run.trial078.stdout:[osd.0] 2026-02-17T05:45:48.841 INFO:teuthology.orchestra.run.trial078.stdout: key = AQB2AJRpgI+pORAAgzy4KYDaOTyavPLpdEHPHw== 2026-02-17T05:45:48.841 INFO:teuthology.orchestra.run.trial078.stdout:[osd.1] 2026-02-17T05:45:48.841 INFO:teuthology.orchestra.run.trial078.stdout: key = AQB6AJRp9OjGABAAXeLGGc3oJujR+o5aDhATTQ== 2026-02-17T05:45:48.842 INFO:teuthology.orchestra.run.trial078.stdout:[osd.2] 2026-02-17T05:45:48.842 INFO:teuthology.orchestra.run.trial078.stdout: key = AQB9AJRpTnxDEhAACcjyW7ItDjgue94B8nVRuA== 2026-02-17T05:45:48.842 INFO:teuthology.orchestra.run.trial078.stdout:[mgr.x] 2026-02-17T05:45:48.842 INFO:teuthology.orchestra.run.trial078.stdout: key = AQB0AJRpTXhqFhAAZYa+p8MlOmQh68BH6gfo5Q== 2026-02-17T05:45:48.842 INFO:teuthology.orchestra.run.trial078.stdout:[osd.3] 2026-02-17T05:45:48.843 INFO:teuthology.orchestra.run.trial078.stdout: key = AQCDAJRpglDsAxAA3X2oNajDcv2GtJj8MiGk1Q== 2026-02-17T05:45:48.843 INFO:teuthology.orchestra.run.trial078.stdout:[osd.4] 2026-02-17T05:45:48.843 INFO:teuthology.orchestra.run.trial078.stdout: key = AQCGAJRprvaQFRAAQRrxmgxSlpxmVAy0VIVjXQ== 2026-02-17T05:45:48.843 INFO:teuthology.orchestra.run.trial078.stdout:[osd.5] 2026-02-17T05:45:48.843 INFO:teuthology.orchestra.run.trial078.stdout: key = AQCJAJRpFrgQGBAAwJvapzP2n77o/Su8CgDt3Q== 2026-02-17T05:45:48.843 INFO:teuthology.orchestra.run.trial078.stdout:[client.0] 2026-02-17T05:45:48.844 INFO:teuthology.orchestra.run.trial078.stdout: key = AQB0AJRpoK6wGBAAMmglRtqyGliBKFrZZAG/og== 2026-02-17T05:45:48.860 INFO:teuthology.orchestra.run.trial115.stdout:[osd.0] 2026-02-17T05:45:48.860 INFO:teuthology.orchestra.run.trial115.stdout: key = AQB2AJRpgI+pORAAgzy4KYDaOTyavPLpdEHPHw== 2026-02-17T05:45:48.860 INFO:teuthology.orchestra.run.trial115.stdout:[osd.1] 2026-02-17T05:45:48.860 INFO:teuthology.orchestra.run.trial115.stdout: key = AQB6AJRp9OjGABAAXeLGGc3oJujR+o5aDhATTQ== 2026-02-17T05:45:48.861 INFO:teuthology.orchestra.run.trial115.stdout:[osd.2] 2026-02-17T05:45:48.861 INFO:teuthology.orchestra.run.trial115.stdout: key = AQB9AJRpTnxDEhAACcjyW7ItDjgue94B8nVRuA== 2026-02-17T05:45:48.861 INFO:teuthology.orchestra.run.trial115.stdout:[mgr.x] 2026-02-17T05:45:48.861 INFO:teuthology.orchestra.run.trial115.stdout: key = AQB0AJRpTXhqFhAAZYa+p8MlOmQh68BH6gfo5Q== 2026-02-17T05:45:48.861 INFO:teuthology.orchestra.run.trial115.stdout:[osd.3] 2026-02-17T05:45:48.861 INFO:teuthology.orchestra.run.trial115.stdout: key = AQCDAJRpglDsAxAA3X2oNajDcv2GtJj8MiGk1Q== 2026-02-17T05:45:48.861 INFO:teuthology.orchestra.run.trial115.stdout:[osd.4] 2026-02-17T05:45:48.862 INFO:teuthology.orchestra.run.trial115.stdout: key = AQCGAJRprvaQFRAAQRrxmgxSlpxmVAy0VIVjXQ== 2026-02-17T05:45:48.862 INFO:teuthology.orchestra.run.trial115.stdout:[osd.5] 2026-02-17T05:45:48.862 INFO:teuthology.orchestra.run.trial115.stdout: key = AQCJAJRpFrgQGBAAwJvapzP2n77o/Su8CgDt3Q== 2026-02-17T05:45:48.862 INFO:teuthology.orchestra.run.trial115.stdout:[client.0] 2026-02-17T05:45:48.862 INFO:teuthology.orchestra.run.trial115.stdout: key = AQB0AJRpoK6wGBAAMmglRtqyGliBKFrZZAG/og== 2026-02-17T05:45:48.864 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:48.888 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:48.944 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:48.970 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.024 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.049 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.106 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T05:45:49.128 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T05:45:49.184 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.208 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.265 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.286 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.344 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.365 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T05:45:49.423 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-17T05:45:49.445 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-17T05:45:49.505 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-17T05:45:49.505 DEBUG:teuthology.orchestra.run.trial078:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-17T05:45:49.550 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-17T05:45:49.632 DEBUG:teuthology.orchestra.run.trial078:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-17T05:45:49.697 DEBUG:teuthology.orchestra.run.trial078:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-17T05:45:49.761 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-17T05:45:49.843 DEBUG:teuthology.orchestra.run.trial078:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-17T05:45:49.907 DEBUG:teuthology.orchestra.run.trial115:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-17T05:45:49.934 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-17T05:45:50.014 DEBUG:teuthology.orchestra.run.trial115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-17T05:45:50.079 DEBUG:teuthology.orchestra.run.trial078:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:50.081 DEBUG:teuthology.orchestra.run.trial115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-17T05:45:50.135 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-17T05:45:50.135 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:45:50.135 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:45:50.137 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:45:50.138 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-17T05:45:50.138 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-17T05:45:50.139 INFO:tasks.ceph.mon.c:Started 2026-02-17T05:45:50.139 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T05:45:50.139 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T05:45:50.178 INFO:tasks.ceph.mon.b:Started 2026-02-17T05:45:50.178 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-17T05:45:50.178 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-17T05:45:50.178 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-17T05:45:50.181 INFO:tasks.ceph.mgr.x:Started 2026-02-17T05:45:50.181 INFO:tasks.ceph:Setting crush tunables to default 2026-02-17T05:45:50.181 DEBUG:teuthology.orchestra.run.trial078:> sudo ceph --cluster ceph osd crush tunables default 2026-02-17T05:45:50.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.250+0000 7ff6f04e1640 10 monclient: get_monmap_and_config 2026-02-17T05:45:50.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.250+0000 7ff6f04e1640 10 monclient: build_initial_monmap 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.250+0000 7ff6f04e1640 10 monclient: monmap: 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:50.250590+0000 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:50.250590+0000 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:50.253 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.250+0000 7ff6f04e1640 1 Processor -- start 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.250+0000 7ff6f04e1640 1 -- start start 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.250+0000 7ff6f04e1640 10 monclient: init 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff6e8135e70 addr v1:10.20.193.78:6789/0 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient(hunting): picked mon.noname-b con 0x7ff6e80a20a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient(hunting): picked mon.noname-a con 0x7ff6e8136960 addr v2:10.20.193.78:3300/0 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6e806f140 con 0x7ff6e8135e70 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e80739d0 con 0x7ff6e8136960 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e80a2bb0 con 0x7ff6e80a20a0 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6f04e1640 10 monclient(hunting): authenticate will time out at 595.804164s 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6ee256640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff6e8135e70 0x7ff6e8136250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52890/0 (socket says 10.20.193.78:52890) 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6ee256640 1 -- 10.20.193.78:0/2801691372 learned_addr learned my addr 10.20.193.78:0/2801691372 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:45:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 10 monclient(hunting): get_auth_request con 0x7ff6e8136960 auth_method 0 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 10 monclient(hunting): get_auth_request con 0x7ff6e80a20a0 auth_method 0 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 msgr2=0x7ff6e813d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:45:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 msgr2=0x7ff6e813d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:45:50.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:50.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6e813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:45:50.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6e813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:45:50.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:50.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:45:50.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.251+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:45:50.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.253+0000 7ff6ee256640 1 -- 10.20.193.78:0/2801691372 >> v1:10.20.193.78:6789/0 conn(0x7ff6e8135e70 legacy=0x7ff6e8136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:50.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 10 monclient(hunting): get_auth_request con 0x7ff6e8136960 auth_method 0 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 10 monclient(hunting): get_auth_request con 0x7ff6e80a20a0 auth_method 0 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:45:50.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 msgr2=0x7ff6e813d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:45:50.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 msgr2=0x7ff6e813d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:45:50.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:50.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6e813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:45:50.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6e813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:45:50.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:50.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:45:50.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.451+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:45:50.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:50.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 10 monclient(hunting): get_auth_request con 0x7ff6e8136960 auth_method 0 2026-02-17T05:45:50.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 10 monclient(hunting): get_auth_request con 0x7ff6e80a20a0 auth_method 0 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 msgr2=0x7ff6e813d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 msgr2=0x7ff6e813d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:45:50.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6e813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:45:50.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6e813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:45:50.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:50.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:50.852+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:45:51.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567726977 0 0) 0x7ff6d8002c60 con 0x7ff6e8135e70 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient(hunting): _negotiate 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient(hunting): my global_id is 4098 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6d0003620 con 0x7ff6e8135e70 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ee256640 1 --1- 10.20.193.78:0/2801691372 >> v1:10.20.193.78:6789/0 conn(0x7ff6e8135e70 0x7ff6e8136250 :-1 s=OPENED pgs=4 cs=1 l=1).queue_received will delay after 1.77131e+09 on 0x7ff6d8003380 mon_map magic: 0 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3880068330 0 0) 0x7ff6d0003620 con 0x7ff6e8135e70 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6e813b2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:45:51.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6e813b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 msgr2=0x7ff6e813d810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 --2- 10.20.193.78:0/2801691372 >> v2:10.20.193.78:3300/0 conn(0x7ff6e8136960 0x7ff6e813d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient: _finish_hunting 0 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e80a28d0 con 0x7ff6e8135e70 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient: _finish_auth 0 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient: _check_auth_tickets 2026-02-17T05:45:51.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6f04e1640 5 monclient: authenticate success, global_id 4098 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6f04e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7ff6d8003380 con 0x7ff6e8135e70 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient: dump: 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.277 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.264+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 >> v1:10.20.193.78:6789/0 conn(0x7ff6e8135e70 legacy=0x7ff6e8136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6d0003b80 0x7ff6d0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): picked mon.a con 0x7ff6d0003b80 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6d0004900 0x7ff6d0004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): picked mon.b con 0x7ff6d0004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6d000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): picked mon.c con 0x7ff6e80a20a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e80a2bb0 con 0x7ff6d0003b80 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0005670 con 0x7ff6d0004900 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d000c000 con 0x7ff6e80a20a0 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6f04e1640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 <== mon.0 v1:10.20.193.78:6789/0 5 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7ff6d8004dd0 con 0x7ff6e8135e70 2026-02-17T05:45:51.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ed254640 10 monclient(hunting): discarding stray monitor message mon_map magic: 0 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6d000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient(hunting): get_auth_request con 0x7ff6e80a20a0 auth_method 0 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ee256640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6d0003b80 0x7ff6d0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ee256640 10 monclient(hunting): get_auth_request con 0x7ff6d0003b80 auth_method 0 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ee256640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ee256640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ee256640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6d0004900 0x7ff6d0004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eda55640 10 monclient(hunting): get_auth_request con 0x7ff6d0004900 auth_method 0 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eda55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eda55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eda55640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ee256640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ee256640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6ee256640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient(hunting): handle_auth_done global_id 4098 payload 293 2026-02-17T05:45:51.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6d0004900 msgr2=0x7ff6d0004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6d0004900 0x7ff6d0004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6d0003b80 msgr2=0x7ff6d0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6d0003b80 0x7ff6d0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient: _finish_hunting 0 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 1 monclient: found mon.c 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d0003620 con 0x7ff6e80a20a0 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient: _finish_auth 0 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 10 monclient: _check_auth_tickets 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.265+0000 7ff6eea57640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6d000bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:51.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6d000bac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009ee0 tx=0x7ff6e402fbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf3c3b2919beb8a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4009bc0 con 0x7ff6e80a20a0 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6ed254640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6ed254640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6ed254640 10 monclient: dump: 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.283 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6f04e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e4003a90 con 0x7ff6e80a20a0 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.267+0000 7ff6ed254640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6ed254640 1 -- 10.20.193.78:0/2801691372 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4031630 con 0x7ff6e80a20a0 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6ed254640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6ed254640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6ed254640 10 monclient: dump: 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.284 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 4 set_mon_vals no callback set 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 10 monclient: shutdown 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 msgr2=0x7ff6d000bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 1 --2- 10.20.193.78:0/2801691372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e80a20a0 0x7ff6d000bac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009ee0 tx=0x7ff6e402fbf0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6eea57640 1 -- 10.20.193.78:0/2801691372 reap_dead start 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2801691372 shutdown_connections 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2801691372 >> 10.20.193.78:0/2801691372 conn(0x7ff6e80a3190 msgr2=0x7ff6e80a35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2801691372 shutdown_connections 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2801691372 wait complete. 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 10 monclient: build_initial_monmap 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 10 monclient: monmap: 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:51.286 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 1 Processor -- start 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 1 -- start start 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.268+0000 7ff6f04e1640 10 monclient: init 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e81cd5a0 0x7ff6e81cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient(hunting): picked mon.noname-c con 0x7ff6e81cd5a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:51.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6e811e7d0 0x7ff6e811ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient(hunting): picked mon.noname-a con 0x7ff6e811e7d0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6e811f170 0x7ff6e811f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient(hunting): picked mon.noname-b con 0x7ff6e811f170 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e809d220 con 0x7ff6e811e7d0 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e809d0a0 con 0x7ff6e811f170 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e80a0180 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6f04e1640 10 monclient(hunting): authenticate will time out at 596.822338s 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e81cd5a0 0x7ff6e81cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e81cd5a0 0x7ff6e81cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42928/0 (socket says 10.20.193.78:42928) 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6e811e7d0 0x7ff6e811ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 -- 10.20.193.78:0/2834450889 learned_addr learned my addr 10.20.193.78:0/2834450889 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient(hunting): get_auth_request con 0x7ff6e81cd5a0 auth_method 0 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 10 monclient(hunting): get_auth_request con 0x7ff6e811e7d0 auth_method 0 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6e811f170 0x7ff6e811f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eea57640 10 monclient(hunting): get_auth_request con 0x7ff6e811f170 auth_method 0 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eea57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eea57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eea57640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient(hunting): handle_auth_done global_id 4100 payload 824 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 -- 10.20.193.78:0/2834450889 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6e811f170 msgr2=0x7ff6e811f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6e811f170 0x7ff6e811f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 -- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6e811e7d0 msgr2=0x7ff6e811ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6e811e7d0 0x7ff6e811ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient: _finish_hunting 0 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 monclient: found mon.noname-c 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6ee256640 1 -- 10.20.193.78:0/2834450889 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e8123140 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eda55640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6e811e7d0 0x7ff6e811ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.269+0000 7ff6eea57640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6e811f170 0x7ff6e811f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ee256640 1 -- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e81cd5a0 msgr2=0x7ff6e81cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ee256640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e81cd5a0 0x7ff6e81cd980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8004b90 tx=0x7ff6d80087e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 1 -- 10.20.193.78:0/2834450889 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8019070 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: dump: 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.292 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: _finish_auth 0 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: _check_auth_tickets 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6f04e1640 5 monclient: authenticate success, global_id 4100 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 1 -- 10.20.193.78:0/2834450889 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d80080d0 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 1 -- 10.20.193.78:0/2834450889 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8005d70 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 10 monclient: dump: 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.293 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6eca53640 4 set_mon_vals no callback set 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6f04e1640 10 monclient: _renew_subs 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6f04e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e8120fd0 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6f04e1640 10 monclient: _renew_subs 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6f04e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e8134e60 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.271+0000 7ff6ceffd640 1 -- 10.20.193.78:0/2834450889 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff6d801e440 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.272+0000 7ff6ceffd640 1 -- 10.20.193.78:0/2834450889 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff6d8014a70 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.272+0000 7ff6f04e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.272+0000 7ff6f04e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.272+0000 7ff6f04e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.272+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e8068410 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.273+0000 7ff6ceffd640 1 -- 10.20.193.78:0/2834450889 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7ff6e8068410 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.273+0000 7ff6ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.273+0000 7ff6ceffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:51.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.302+0000 7ff6f04e1640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-17T05:45:51.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.302+0000 7ff6f04e1640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-17T05:45:51.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.302+0000 7ff6f04e1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:51.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.302+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff6e80a5aa0 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.310+0000 7ff6ceffd640 1 -- 10.20.193.78:0/2834450889 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ff6e8123140 con 0x7ff6e81cd5a0 2026-02-17T05:45:51.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.310+0000 7ff6ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.310+0000 7ff6ceffd640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:adjusted tunables profile to default 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 10 monclient: shutdown 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e81cd5a0 msgr2=0x7ff6e81cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e81cd5a0 0x7ff6e81cd980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8004b90 tx=0x7ff6d80087e0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 shutdown_connections 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6e811f170 0x7ff6e811f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff6e811e7d0 0x7ff6e811ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 --2- 10.20.193.78:0/2834450889 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff6e81cd5a0 0x7ff6e81cd980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 >> 10.20.193.78:0/2834450889 conn(0x7ff6e80a3190 msgr2=0x7ff6e80a4490 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:51.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 shutdown_connections 2026-02-17T05:45:51.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:51.311+0000 7ff6f04e1640 1 -- 10.20.193.78:0/2834450889 wait complete. 2026-02-17T05:45:51.319 INFO:tasks.ceph:check_enable_crimson: False 2026-02-17T05:45:51.319 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-17T05:45:51.319 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:51.319 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-17T05:45:51.344 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:51.344 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-17T05:45:51.408 DEBUG:teuthology.orchestra.run.trial078:> set -ex 2026-02-17T05:45:51.409 DEBUG:teuthology.orchestra.run.trial078:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-17T05:45:51.473 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:51.473 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-17T05:45:51.498 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:51.498 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-17T05:45:51.562 DEBUG:teuthology.orchestra.run.trial115:> set -ex 2026-02-17T05:45:51.563 DEBUG:teuthology.orchestra.run.trial115:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-17T05:45:51.620 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.619+0000 7f11af41e140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-17T05:45:51.628 DEBUG:teuthology.orchestra.run.trial115:> sudo ceph --cluster ceph osd new 6e0537ff-96b9-44ed-87db-8e297f066844 0 2026-02-17T05:45:51.736 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.732+0000 7ff8bf33e640 10 monclient: get_monmap_and_config 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.732+0000 7ff8bf33e640 10 monclient: build_initial_monmap 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.732+0000 7ff8bf33e640 10 monclient: monmap: 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:51.733686+0000 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:51.733686+0000 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.737 INFO:teuthology.orchestra.run.trial115.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.732+0000 7ff8bf33e640 1 Processor -- start 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.732+0000 7ff8bf33e640 1 -- start start 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.732+0000 7ff8bf33e640 10 monclient: init 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff8b80a1fe0 addr v1:10.20.193.78:6789/0 2026-02-17T05:45:51.738 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff8b80a0ab0 0x7ff8b813d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 10 monclient(hunting): picked mon.noname-a con 0x7ff8b80a0ab0 addr v2:10.20.193.78:3300/0 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff8b813d8d0 addr v1:10.20.193.115:6789/0 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b80a48e0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b8070a30 con 0x7ff8b813d8d0 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b80a46a0 con 0x7ff8b80a0ab0 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bf33e640 10 monclient(hunting): authenticate will time out at 604.448293s 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bd8b4640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7ff8b813d8d0 0x7ff8b813fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.115:54816/0 (socket says 10.20.193.115:54816) 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bd8b4640 1 -- 10.20.193.115:0/143735884 learned_addr learned my addr 10.20.193.115:0/143735884 (peer_addr_for_me v1:10.20.193.115:0/0) 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 1 --2- 10.20.193.115:0/143735884 >> v2:10.20.193.78:3300/0 conn(0x7ff8b80a0ab0 0x7ff8b813d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.739 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient(hunting): get_auth_request con 0x7ff8b80a0ab0 auth_method 0 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bd0b3640 1 -- 10.20.193.115:0/143735884 >> v1:10.20.193.78:6789/0 conn(0x7ff8b80a1fe0 legacy=0x7ff8b80a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bd0b3640 1 -- 10.20.193.115:0/143735884 >> v1:10.20.193.78:6789/0 conn(0x7ff8b80a1fe0 legacy=0x7ff8b80a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bd0b3640 1 --1- 10.20.193.115:0/143735884 >> v1:10.20.193.78:6789/0 conn(0x7ff8b80a1fe0 0x7ff8b80a23c0 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient(hunting): handle_auth_done global_id 4110 payload 274 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/143735884 >> v1:10.20.193.115:6789/0 conn(0x7ff8b813d8d0 legacy=0x7ff8b813fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:51.740 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/143735884 >> v1:10.20.193.78:6789/0 conn(0x7ff8b80a1fe0 legacy=0x7ff8b80a23c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:45:51.741 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient: _finish_hunting 0 2026-02-17T05:45:51.741 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 1 monclient: found mon.noname-a 2026-02-17T05:45:51.741 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:51.741 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:45:51.741 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.733+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/143735884 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b80a4cf0 con 0x7ff8b80a0ab0 2026-02-17T05:45:51.747 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/143735884 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 msgr2=0x7ff8b813d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:51.747 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bc8b2640 1 --2- 10.20.193.115:0/143735884 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 0x7ff8b813d390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0098a0 tx=0x7ff8ac02ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c8afd87fffe92f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 1 -- 10.20.193.115:0/143735884 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac03c070 con 0x7ff8b80a0ab0 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: dump: 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.748 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: _finish_auth 0 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 1 -- 10.20.193.115:0/143735884 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac02fce0 con 0x7ff8b80a0ab0 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 5 monclient: authenticate success, global_id 4110 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:51.749 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 1 -- 10.20.193.115:0/143735884 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac037660 con 0x7ff8b80a0ab0 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 4 set_mon_vals no callback set 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8a7fff640 10 monclient: dump: 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.750 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 10 monclient: shutdown 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 1 -- 10.20.193.115:0/143735884 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 msgr2=0x7ff8b813d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 1 --2- 10.20.193.115:0/143735884 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 0x7ff8b813d390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0098a0 tx=0x7ff8ac02ed80 comp rx=0 tx=0).stop 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 1 -- 10.20.193.115:0/143735884 shutdown_connections 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 1 --2- 10.20.193.115:0/143735884 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 0x7ff8b813d390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.736+0000 7ff8bf33e640 1 -- 10.20.193.115:0/143735884 >> 10.20.193.115:0/143735884 conn(0x7ff8b806ccb0 msgr2=0x7ff8b806d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.737+0000 7ff8bf33e640 1 -- 10.20.193.115:0/143735884 shutdown_connections 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.737+0000 7ff8bf33e640 1 -- 10.20.193.115:0/143735884 wait complete. 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.737+0000 7ff8bf33e640 10 monclient: build_initial_monmap 2026-02-17T05:45:51.751 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.737+0000 7ff8bf33e640 10 monclient: monmap: 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 0.000000 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:created 0.000000 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.737+0000 7ff8bf33e640 1 Processor -- start 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.737+0000 7ff8bf33e640 1 -- start start 2026-02-17T05:45:51.752 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.737+0000 7ff8bf33e640 10 monclient: init 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 0x7ff8b81cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient(hunting): picked mon.noname-a con 0x7ff8b80a0ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8b80a1fe0 0x7ff8b81cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient(hunting): picked mon.noname-b con 0x7ff8b80a1fe0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8b813d8d0 0x7ff8b81d0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient(hunting): picked mon.noname-c con 0x7ff8b813d8d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8074410 con 0x7ff8b80a0ab0 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b80a4330 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.753 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b8127f70 con 0x7ff8b813d8d0 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bf33e640 10 monclient(hunting): authenticate will time out at 604.453080s 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8b80a1fe0 0x7ff8b81cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8b80a1fe0 0x7ff8b81cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.115:56818/0 (socket says 10.20.193.115:56818) 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/2742939113 learned_addr learned my addr 10.20.193.115:0/2742939113 (peer_addr_for_me v2:10.20.193.115:0/0) 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient(hunting): get_auth_request con 0x7ff8b80a1fe0 auth_method 0 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.754 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd0b3640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 0x7ff8b81cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd0b3640 10 monclient(hunting): get_auth_request con 0x7ff8b80a0ab0 auth_method 0 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd0b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd0b3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd0b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd8b4640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8b813d8d0 0x7ff8b81d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd8b4640 10 monclient(hunting): get_auth_request con 0x7ff8b813d8d0 auth_method 0 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd8b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd8b4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.755 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd8b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient(hunting): handle_auth_done global_id 4108 payload 824 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8b813d8d0 msgr2=0x7ff8b81d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8b813d8d0 0x7ff8b81d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 msgr2=0x7ff8b81cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 0x7ff8b81cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient: _finish_hunting 0 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 monclient: found mon.noname-b 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/2742939113 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b80a69b0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.738+0000 7ff8bd0b3640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 0x7ff8b81cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bc8b2640 1 -- 10.20.193.115:0/2742939113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8b80a1fe0 msgr2=0x7ff8b81cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:51.756 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bc8b2640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8b80a1fe0 0x7ff8b81cfef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac02f290 tx=0x7ff8ac0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 1 -- 10.20.193.115:0/2742939113 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac046070 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: dump: 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.757 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 1 -- 10.20.193.115:0/2742939113 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac041870 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bf33e640 5 monclient: authenticate success, global_id 4108 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bf33e640 10 monclient: _renew_subs 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bf33e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b809f8a0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 1 -- 10.20.193.115:0/2742939113 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac03c040 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.758 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a77fe640 4 set_mon_vals no callback set 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 10 monclient: dump: 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.759 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bf33e640 10 monclient: _renew_subs 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bf33e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b81295f0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 1 -- 10.20.193.115:0/2742939113 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff8ac041c60 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.740+0000 7ff8a5ffb640 1 -- 10.20.193.115:0/2742939113 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7ff8ac0423f0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.741+0000 7ff8bf33e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.741+0000 7ff8bf33e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.741+0000 7ff8bf33e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.741+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b81299e0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.742+0000 7ff8a5ffb640 1 -- 10.20.193.115:0/2742939113 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7ff8b80a69b0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.742+0000 7ff8a5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.760 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.742+0000 7ff8a5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:51.761 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.742+0000 7f11af41e140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-17T05:45:51.771 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.769+0000 7f11af41e140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-17T05:45:51.775 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.774+0000 7ff8bf33e640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "6e0537ff-96b9-44ed-87db-8e297f066844", "id": 0}] 2026-02-17T05:45:51.775 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.774+0000 7ff8bf33e640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "6e0537ff-96b9-44ed-87db-8e297f066844", "id": 0}] 2026-02-17T05:45:51.776 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.774+0000 7ff8bf33e640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.776 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.774+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6e0537ff-96b9-44ed-87db-8e297f066844", "id": 0} v 0) -- 0x7ff8b80a23c0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.778 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.776+0000 7ff8a5ffb640 1 -- 10.20.193.115:0/2742939113 <== mon.1 v2:10.20.193.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e0537ff-96b9-44ed-87db-8e297f066844", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7ff8ac02fba0 con 0x7ff8b80a1fe0 2026-02-17T05:45:51.778 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.776+0000 7ff8a5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "6e0537ff-96b9-44ed-87db-8e297f066844", "id": 0}] 2026-02-17T05:45:51.778 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.776+0000 7ff8a5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:45:51.778 INFO:teuthology.orchestra.run.trial115.stdout:0 2026-02-17T05:45:51.779 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.776+0000 7ff8bf33e640 10 monclient: shutdown 2026-02-17T05:45:51.779 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.776+0000 7ff8bf33e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:51.779 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.776+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8b80a1fe0 msgr2=0x7ff8b81cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.779 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.776+0000 7ff8bf33e640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8b80a1fe0 0x7ff8b81cfef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac02f290 tx=0x7ff8ac0047c0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.779 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.777+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 shutdown_connections 2026-02-17T05:45:51.780 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.777+0000 7ff8bf33e640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8b813d8d0 0x7ff8b81d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.780 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.777+0000 7ff8bf33e640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8b80a1fe0 0x7ff8b81cfef0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.780 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.777+0000 7ff8bf33e640 1 --2- 10.20.193.115:0/2742939113 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8b80a0ab0 0x7ff8b81cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.780 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.777+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 >> 10.20.193.115:0/2742939113 conn(0x7ff8b806ccb0 msgr2=0x7ff8b80724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:51.780 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.777+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 shutdown_connections 2026-02-17T05:45:51.780 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.777+0000 7ff8bf33e640 1 -- 10.20.193.115:0/2742939113 wait complete. 2026-02-17T05:45:51.786 DEBUG:teuthology.orchestra.run.trial115:> sudo ceph --cluster ceph osd new f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d 1 2026-02-17T05:45:51.804 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.803+0000 7f11af41e140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-17T05:45:51.827 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.826+0000 7f11af41e140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-17T05:45:51.853 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.852+0000 7f11af41e140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-17T05:45:51.856 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient: get_monmap_and_config 2026-02-17T05:45:51.856 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient: build_initial_monmap 2026-02-17T05:45:51.856 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient: monmap: 2026-02-17T05:45:51.856 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:51.856 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.856 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:51.853371+0000 2026-02-17T05:45:51.856 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:51.853371+0000 2026-02-17T05:45:51.856 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 1 Processor -- start 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 1 -- start start 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient: init 2026-02-17T05:45:51.857 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f62b8135e70 addr v1:10.20.193.115:6789/0 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f62b80a20a0 0x7f62b813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient(hunting): picked mon.noname-c con 0x7f62b80a20a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f62b8136960 0x7f62b813d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient(hunting): picked mon.noname-a con 0x7f62b8136960 addr v2:10.20.193.78:3300/0 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b806f140 con 0x7f62b8135e70 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f62b80a2bb0 con 0x7f62b8136960 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f62b809fe60 con 0x7f62b80a20a0 2026-02-17T05:45:51.858 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.852+0000 7f62be7e4640 10 monclient(hunting): authenticate will time out at 604.567954s 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f62b80a20a0 0x7f62b813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b7fff640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f62b8135e70 0x7f62b8136250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.115:54818/0 (socket says 10.20.193.115:54818) 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b7fff640 1 -- 10.20.193.115:0/3017389820 learned_addr learned my addr 10.20.193.115:0/3017389820 (peer_addr_for_me v1:10.20.193.115:0/0) 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient(hunting): get_auth_request con 0x7f62b80a20a0 auth_method 0 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62bcd5a640 1 --2- 10.20.193.115:0/3017389820 >> v2:10.20.193.78:3300/0 conn(0x7f62b8136960 0x7f62b813d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.859 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62bcd5a640 10 monclient(hunting): get_auth_request con 0x7f62b8136960 auth_method 0 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62bcd5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62bcd5a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62bcd5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62bcd5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62bcd5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62bcd5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient(hunting): handle_auth_done global_id 4111 payload 274 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 >> v2:10.20.193.78:3300/0 conn(0x7f62b8136960 msgr2=0x7f62b813d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 1 --2- 10.20.193.115:0/3017389820 >> v2:10.20.193.78:3300/0 conn(0x7f62b8136960 0x7f62b813d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 >> v1:10.20.193.115:6789/0 conn(0x7f62b8135e70 legacy=0x7f62b8136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:51.860 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:45:51.861 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 1 monclient: found mon.noname-c 2026-02-17T05:45:51.861 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:51.861 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.861 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.853+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b80a28d0 con 0x7f62b80a20a0 2026-02-17T05:45:51.869 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 msgr2=0x7f62b813b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:51.869 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b77fe640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 0x7f62b813b2c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62b806dc00 tx=0x7f62a802ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb0037e70aaf20f6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:51.869 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 msgr2=0x7f62b813b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-17T05:45:51.869 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 msgr2=0x7f62b813b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b77fe640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 0x7f62b813b2c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62b806dc00 tx=0x7f62a802ed50 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b77fe640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 0x7f62b813b2c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62b806dc00 tx=0x7f62a802ed50 comp rx=0 tx=0).stop 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 10 monclient: ms_handle_reset current mon [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 1 -- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 msgr2=0x7f62b813b2c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 0x7f62b813b2c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 10 monclient: _add_conns ranks=[0,2] 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f629c003600 addr v1:10.20.193.78:6789/0 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 0x7f629c0099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f629c007530 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 1 -- 10.20.193.115:0/3017389820 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f629c00a2e0 con 0x7f629c003600 2026-02-17T05:45:51.870 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 1 -- 10.20.193.115:0/3017389820 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f629c00b530 con 0x7f629c007530 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.855+0000 7f62b6ffd640 10 monclient(hunting): ms_handle_reset stray mon [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 0x7f629c0099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient(hunting): get_auth_request con 0x7f629c007530 auth_method 0 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:45:51.871 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient(hunting): handle_auth_done global_id 4111 payload 293 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 >> v1:10.20.193.78:6789/0 conn(0x7f629c003600 legacy=0x7f629c005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 1 monclient: found mon.noname-b 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.856+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c00b7f0 con 0x7f629c007530 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b77fe640 1 -- 10.20.193.115:0/3017389820 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 msgr2=0x7f629c0099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b77fe640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 0x7f629c0099f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f62b806dae0 tx=0x7f62a8009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28cab7ad06974c26 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 1 -- 10.20.193.115:0/3017389820 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a803c070 con 0x7f629c007530 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: dump: 2026-02-17T05:45:51.872 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:45:51.873 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62be7e4640 5 monclient: authenticate success, global_id 4111 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62be7e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 1 -- 10.20.193.115:0/3017389820 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a8004240 con 0x7f629c007530 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 1 -- 10.20.193.115:0/3017389820 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a802fbe0 con 0x7f629c007530 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62b6ffd640 10 monclient: dump: 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.874 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62be7e4640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.858+0000 7f62be7e4640 4 set_mon_vals no callback set 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 10 monclient: shutdown 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 1 -- 10.20.193.115:0/3017389820 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 msgr2=0x7f629c0099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.875 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 0x7f629c0099f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f62b806dae0 tx=0x7f62a8009290 comp rx=0 tx=0).stop 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62bcd5a640 1 -- 10.20.193.115:0/3017389820 reap_dead start 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 1 -- 10.20.193.115:0/3017389820 shutdown_connections 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 1 --2- 10.20.193.115:0/3017389820 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b80a20a0 0x7f62b813b2c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 1 -- 10.20.193.115:0/3017389820 >> 10.20.193.115:0/3017389820 conn(0x7f62b80a3190 msgr2=0x7f62b80a35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 1 -- 10.20.193.115:0/3017389820 shutdown_connections 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 1 -- 10.20.193.115:0/3017389820 wait complete. 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 10 monclient: build_initial_monmap 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.859+0000 7f62be7e4640 10 monclient: monmap: 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 0.000000 2026-02-17T05:45:51.876 INFO:teuthology.orchestra.run.trial115.stderr:created 0.000000 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 1 Processor -- start 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 1 -- start start 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient: init 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f62b80a20a0 0x7f62b81ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.877 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient(hunting): picked mon.noname-a con 0x7f62b80a20a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 0x7f62b81cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient(hunting): picked mon.noname-c con 0x7f629c007530 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b81cd7b0 0x7f62b81349e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient(hunting): picked mon.noname-b con 0x7f62b81cd7b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62b809d220 con 0x7f62b80a20a0 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f62b809d0a0 con 0x7f62b81cd7b0 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f62b80a0180 con 0x7f629c007530 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:51.878 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62be7e4640 10 monclient(hunting): authenticate will time out at 604.575359s 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b81cd7b0 0x7f62b81349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b81cd7b0 0x7f62b81349e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.115:56840/0 (socket says 10.20.193.115:56840) 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 -- 10.20.193.115:0/1863225298 learned_addr learned my addr 10.20.193.115:0/1863225298 (peer_addr_for_me v2:10.20.193.115:0/0) 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient(hunting): get_auth_request con 0x7f62b81cd7b0 auth_method 0 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:51.879 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b77fe640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 0x7f62b81cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b7fff640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f62b80a20a0 0x7f62b81ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b77fe640 10 monclient(hunting): get_auth_request con 0x7f629c007530 auth_method 0 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b7fff640 10 monclient(hunting): get_auth_request con 0x7f62b80a20a0 auth_method 0 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient(hunting): handle_auth_done global_id 4114 payload 824 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 -- 10.20.193.115:0/1863225298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 msgr2=0x7f62b81cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.880 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 0x7f62b81cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 -- 10.20.193.115:0/1863225298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f62b80a20a0 msgr2=0x7f62b81ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f62b80a20a0 0x7f62b81ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient: _finish_hunting 0 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 monclient: found mon.noname-b 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.860+0000 7f62bcd5a640 1 -- 10.20.193.115:0/1863225298 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b809d880 con 0x7f62b81cd7b0 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.862+0000 7f62bcd5a640 1 -- 10.20.193.115:0/1863225298 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b81cd7b0 msgr2=0x7f62b81349e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.862+0000 7f62bcd5a640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b81cd7b0 0x7f62b81349e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f62b80a4540 tx=0x7f62ac00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.862+0000 7f62b4ff9640 1 -- 10.20.193.115:0/1863225298 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 1 -- 10.20.193.115:0/1863225298 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac018020 con 0x7f62b81cd7b0 2026-02-17T05:45:51.881 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: dump: 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.882 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: _finish_auth 0 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 1 -- 10.20.193.115:0/1863225298 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ac0146a0 con 0x7f62b81cd7b0 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62be7e4640 5 monclient: authenticate success, global_id 4114 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 1 -- 10.20.193.115:0/1863225298 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac014a90 con 0x7f62b81cd7b0 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b67fc640 4 set_mon_vals no callback set 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 10 monclient: dump: 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:51.883 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62be7e4640 10 monclient: _renew_subs 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62be7e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b813cd80 con 0x7f62b81cd7b0 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62be7e4640 10 monclient: _renew_subs 2026-02-17T05:45:51.884 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62be7e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b811e640 con 0x7f62b81cd7b0 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 1 -- 10.20.193.115:0/1863225298 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f62ac01d400 con 0x7f62b81cd7b0 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.864+0000 7f62b4ff9640 1 -- 10.20.193.115:0/1863225298 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f62ac013b30 con 0x7f62b81cd7b0 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.865+0000 7f62be7e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.865+0000 7f62be7e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.865+0000 7f62be7e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.865+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b8077c20 con 0x7f62b81cd7b0 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.866+0000 7f62b4ff9640 1 -- 10.20.193.115:0/1863225298 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f62ac022030 con 0x7f62b81cd7b0 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.866+0000 7f62b4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:51.885 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.866+0000 7f62b4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:51.899 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.898+0000 7f62be7e4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d", "id": 1}] 2026-02-17T05:45:51.900 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.898+0000 7f62be7e4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d", "id": 1}] 2026-02-17T05:45:51.900 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.898+0000 7f62be7e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:51.900 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.898+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d", "id": 1} v 0) -- 0x7f62b8136250 con 0x7f62b81cd7b0 2026-02-17T05:45:51.901 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62b4ff9640 1 -- 10.20.193.115:0/1863225298 <== mon.1 v2:10.20.193.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f62ac010040 con 0x7f62b81cd7b0 2026-02-17T05:45:51.901 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62b4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d", "id": 1}] 2026-02-17T05:45:51.901 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62b4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:45:51.902 INFO:teuthology.orchestra.run.trial115.stdout:1 2026-02-17T05:45:51.903 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 10 monclient: shutdown 2026-02-17T05:45:51.903 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:51.903 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b81cd7b0 msgr2=0x7f62b81349e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:51.903 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b81cd7b0 0x7f62b81349e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f62b80a4540 tx=0x7f62ac00c490 comp rx=0 tx=0).stop 2026-02-17T05:45:51.903 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 shutdown_connections 2026-02-17T05:45:51.903 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f62b81cd7b0 0x7f62b81349e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.903 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f629c007530 0x7f62b81cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.904 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 --2- 10.20.193.115:0/1863225298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f62b80a20a0 0x7f62b81ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:51.904 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 >> 10.20.193.115:0/1863225298 conn(0x7f62b80a3190 msgr2=0x7f62b80a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:51.904 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 shutdown_connections 2026-02-17T05:45:51.904 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:51.900+0000 7f62be7e4640 1 -- 10.20.193.115:0/1863225298 wait complete. 2026-02-17T05:45:51.909 DEBUG:teuthology.orchestra.run.trial115:> sudo ceph --cluster ceph osd new 9f613d91-6a9c-4d90-bd09-a2c84fc35597 2 2026-02-17T05:45:51.910 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.908+0000 7f11af41e140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-17T05:45:51.970 INFO:tasks.ceph.mgr.x.trial115.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-17T05:45:51.971 INFO:tasks.ceph.mgr.x.trial115.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-17T05:45:51.971 INFO:tasks.ceph.mgr.x.trial115.stderr: from numpy import show_config as show_numpy_config 2026-02-17T05:45:51.974 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.972+0000 7f11af41e140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-17T05:45:51.999 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:51.997+0000 7f11af41e140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-17T05:45:52.017 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.013+0000 7f593187d640 10 monclient: get_monmap_and_config 2026-02-17T05:45:52.017 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.013+0000 7f593187d640 10 monclient: build_initial_monmap 2026-02-17T05:45:52.017 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.013+0000 7f593187d640 10 monclient: monmap: 2026-02-17T05:45:52.017 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:52.017 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.017 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:52.014592+0000 2026-02-17T05:45:52.017 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:52.014592+0000 2026-02-17T05:45:52.017 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.013+0000 7f593187d640 1 Processor -- start 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.013+0000 7f593187d640 1 -- start start 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.013+0000 7f593187d640 10 monclient: init 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2026-02-17T05:45:52.018 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f592c135e70 0x7f592c136250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 10 monclient(hunting): picked mon.noname-c con 0x7f592c135e70 addr v2:10.20.193.115:3300/0 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f592c136790 addr v1:10.20.193.78:6789/0 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f592c0a17a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592c06f140 con 0x7f592c136790 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592c09fe60 con 0x7f592c0a17a0 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f592c0a2bd0 con 0x7f592c135e70 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f593187d640 10 monclient(hunting): authenticate will time out at 604.729166s 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f592c135e70 0x7f592c136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592b7fe640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f592c0a17a0 0x7f592c0a1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.115:54832/0 (socket says 10.20.193.115:54832) 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592b7fe640 1 -- 10.20.193.115:0/3572493750 learned_addr learned my addr 10.20.193.115:0/3572493750 (peer_addr_for_me v1:10.20.193.115:0/0) 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient(hunting): get_auth_request con 0x7f592c135e70 auth_method 0 2026-02-17T05:45:52.019 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient(hunting): handle_auth_done global_id 4117 payload 274 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 1 -- 10.20.193.115:0/3572493750 >> v1:10.20.193.115:6789/0 conn(0x7f592c0a17a0 legacy=0x7f592c0a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 1 -- 10.20.193.115:0/3572493750 >> v1:10.20.193.78:6789/0 conn(0x7f592c136790 legacy=0x7f592c0a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient: _finish_hunting 0 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 1 monclient: found mon.noname-c 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.020 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.014+0000 7f592affd640 1 -- 10.20.193.115:0/3572493750 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c0a2300 con 0x7f592c135e70 2026-02-17T05:45:52.027 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.016+0000 7f592affd640 1 -- 10.20.193.115:0/3572493750 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 msgr2=0x7f592c136250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:52.027 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.016+0000 7f592affd640 1 --2- 10.20.193.115:0/3572493750 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c136250 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f59200098a0 tx=0x7f592002ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb8e95a00dff6f6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:52.027 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 1 -- 10.20.193.115:0/3572493750 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592003c070 con 0x7f592c135e70 2026-02-17T05:45:52.027 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.027 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: dump: 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.028 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: _finish_auth 0 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 5 monclient: authenticate success, global_id 4117 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 1 -- 10.20.193.115:0/3572493750 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592002fce0 con 0x7f592c135e70 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 1 -- 10.20.193.115:0/3572493750 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920037660 con 0x7f592c135e70 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 4 set_mon_vals no callback set 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f5929ffb640 10 monclient: dump: 2026-02-17T05:45:52.029 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 10 monclient: shutdown 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:52.030 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 1 -- 10.20.193.115:0/3572493750 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 msgr2=0x7f592c136250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 1 --2- 10.20.193.115:0/3572493750 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c136250 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f59200098a0 tx=0x7f592002ed80 comp rx=0 tx=0).stop 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 1 -- 10.20.193.115:0/3572493750 shutdown_connections 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 1 --2- 10.20.193.115:0/3572493750 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c136250 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 1 -- 10.20.193.115:0/3572493750 >> 10.20.193.115:0/3572493750 conn(0x7f592c0a3190 msgr2=0x7f592c0a35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 1 -- 10.20.193.115:0/3572493750 shutdown_connections 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 1 -- 10.20.193.115:0/3572493750 wait complete. 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 10 monclient: build_initial_monmap 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 10 monclient: monmap: 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:52.031 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 0.000000 2026-02-17T05:45:52.032 INFO:teuthology.orchestra.run.trial115.stderr:created 0.000000 2026-02-17T05:45:52.032 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:52.032 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.032 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:52.032 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:52.032 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:52.036 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.036 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.017+0000 7f593187d640 1 Processor -- start 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 1 -- start start 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient: init 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f592c0a17a0 0x7f592c1cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient(hunting): picked mon.noname-a con 0x7f592c0a17a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c123e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient(hunting): picked mon.noname-b con 0x7f592c135e70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 0x7f592c1243a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.037 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient(hunting): picked mon.noname-c con 0x7f592c136790 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f592c0a3010 con 0x7f592c0a17a0 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f592c0a2e90 con 0x7f592c135e70 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f592c09d0a0 con 0x7f592c136790 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f593187d640 10 monclient(hunting): authenticate will time out at 604.733266s 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c123e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c123e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.115:56858/0 (socket says 10.20.193.115:56858) 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 1 -- 10.20.193.115:0/3119089577 learned_addr learned my addr 10.20.193.115:0/3119089577 (peer_addr_for_me v2:10.20.193.115:0/0) 2026-02-17T05:45:52.038 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 10 monclient(hunting): get_auth_request con 0x7f592c135e70 auth_method 0 2026-02-17T05:45:52.039 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.039 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.039 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.039 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 1 -- 10.20.193.115:0/3119089577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 msgr2=0x7f592c123e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:45:52.039 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 1 -- 10.20.193.115:0/3119089577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 msgr2=0x7f592c123e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:45:52.039 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c123e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:52.039 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592a7fc640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c123e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:45:52.039 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f592c0a17a0 0x7f592c1cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 0x7f592c1243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient(hunting): get_auth_request con 0x7f592c0a17a0 auth_method 0 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 10 monclient(hunting): get_auth_request con 0x7f592c136790 auth_method 0 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:52.040 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 1 -- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 msgr2=0x7f592c1243a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 1 -- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 msgr2=0x7f592c1243a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 0x7f592c1243a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592b7fe640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 0x7f592c1243a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient(hunting): handle_auth_done global_id 4122 payload 824 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 1 -- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 msgr2=0x7f592c1243a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 0x7f592c1243a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 1 -- 10.20.193.115:0/3119089577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 msgr2=0x7f592c123e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c123e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.041 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient: _finish_hunting 0 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 1 monclient: found mon.noname-a 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.018+0000 7f592affd640 1 -- 10.20.193.115:0/3119089577 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c09d880 con 0x7f592c0a17a0 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.020+0000 7f592affd640 1 -- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f592c0a17a0 msgr2=0x7f592c1cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.020+0000 7f592affd640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f592c0a17a0 0x7f592c1cf6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5920036000 tx=0x7f59200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 1 -- 10.20.193.115:0/3119089577 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920046070 con 0x7f592c0a17a0 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: dump: 2026-02-17T05:45:52.042 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: _finish_auth 0 2026-02-17T05:45:52.043 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: _check_auth_tickets 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 1 -- 10.20.193.115:0/3119089577 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59200418b0 con 0x7f592c0a17a0 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 1 -- 10.20.193.115:0/3119089577 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592003c040 con 0x7f592c0a17a0 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 5 monclient: authenticate success, global_id 4122 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f59297fa640 4 set_mon_vals no callback set 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 10 monclient: _renew_subs 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592c127ac0 con 0x7f592c0a17a0 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:45:52.044 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 10 monclient: dump: 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 10 monclient: _renew_subs 2026-02-17T05:45:52.045 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592c135220 con 0x7f592c0a17a0 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 1 -- 10.20.193.115:0/3119089577 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5920041ca0 con 0x7f592c0a17a0 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f5903fff640 1 -- 10.20.193.115:0/3119089577 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f592002fba0 con 0x7f592c0a17a0 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.021+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592c0a0210 con 0x7f592c0a17a0 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.023+0000 7f5903fff640 1 -- 10.20.193.115:0/3119089577 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f592c06f140 con 0x7f592c0a17a0 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.023+0000 7f5903fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.046 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.023+0000 7f5903fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:52.056 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.052+0000 7f11af41e140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-17T05:45:52.057 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.055+0000 7f593187d640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9f613d91-6a9c-4d90-bd09-a2c84fc35597", "id": 2}] 2026-02-17T05:45:52.057 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.055+0000 7f593187d640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9f613d91-6a9c-4d90-bd09-a2c84fc35597", "id": 2}] 2026-02-17T05:45:52.057 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.055+0000 7f593187d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:45:52.057 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.055+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9f613d91-6a9c-4d90-bd09-a2c84fc35597", "id": 2} v 0) -- 0x7f592c069870 con 0x7f592c0a17a0 2026-02-17T05:45:52.058 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.056+0000 7f5903fff640 1 -- 10.20.193.115:0/3119089577 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f613d91-6a9c-4d90-bd09-a2c84fc35597", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f592004a400 con 0x7f592c0a17a0 2026-02-17T05:45:52.058 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.056+0000 7f5903fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9f613d91-6a9c-4d90-bd09-a2c84fc35597", "id": 2}] 2026-02-17T05:45:52.058 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.056+0000 7f5903fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:45:52.058 INFO:teuthology.orchestra.run.trial115.stdout:2 2026-02-17T05:45:52.059 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 10 monclient: shutdown 2026-02-17T05:45:52.059 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:52.059 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f592c0a17a0 msgr2=0x7f592c1cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.060 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f592c0a17a0 0x7f592c1cf6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5920036000 tx=0x7f59200047c0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.060 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 shutdown_connections 2026-02-17T05:45:52.060 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f592c136790 0x7f592c1243a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.060 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f592c135e70 0x7f592c123e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.060 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 --2- 10.20.193.115:0/3119089577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f592c0a17a0 0x7f592c1cf6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.060 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 >> 10.20.193.115:0/3119089577 conn(0x7f592c0a3190 msgr2=0x7f592c09e620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:52.060 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 shutdown_connections 2026-02-17T05:45:52.060 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.057+0000 7f593187d640 1 -- 10.20.193.115:0/3119089577 wait complete. 2026-02-17T05:45:52.068 DEBUG:teuthology.orchestra.run.trial115:> sudo ceph --cluster ceph osd new 4fe4aca4-d018-4166-953a-0bef54c014f4 3 2026-02-17T05:45:52.096 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.095+0000 7f11af41e140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-17T05:45:52.122 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.121+0000 7f11af41e140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-17T05:45:52.146 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.145+0000 7f11af41e140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-17T05:45:52.163 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.161+0000 7ff9f7fff640 10 monclient: get_monmap_and_config 2026-02-17T05:45:52.163 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.161+0000 7ff9f7fff640 10 monclient: build_initial_monmap 2026-02-17T05:45:52.163 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.161+0000 7ff9f7fff640 10 monclient: monmap: 2026-02-17T05:45:52.163 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:52.163 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.163 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:52.162769+0000 2026-02-17T05:45:52.163 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:52.162769+0000 2026-02-17T05:45:52.164 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:52.164 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.164 INFO:teuthology.orchestra.run.trial115.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:52.164 INFO:teuthology.orchestra.run.trial115.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:52.164 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:52.164 INFO:teuthology.orchestra.run.trial115.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:52.164 INFO:teuthology.orchestra.run.trial115.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:52.164 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.173 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 1 Processor -- start 2026-02-17T05:45:52.173 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 1 -- start start 2026-02-17T05:45:52.173 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient: init 2026-02-17T05:45:52.173 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff9f8138430 0x7ff9f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient(hunting): picked mon.noname-c con 0x7ff9f8138430 addr v2:10.20.193.115:3300/0 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff9f80a6690 addr v1:10.20.193.115:6789/0 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff9f813d320 0x7ff9f813f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient(hunting): picked mon.noname-a con 0x7ff9f813d320 addr v2:10.20.193.78:3300/0 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f8138f20 con 0x7ff9f80a6690 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f80a5e30 con 0x7ff9f813d320 2026-02-17T05:45:52.174 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f80a7040 con 0x7ff9f8138430 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f7fff640 10 monclient(hunting): authenticate will time out at 604.877405s 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f67fc640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7ff9f80a6690 0x7ff9f8138810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.115:54848/0 (socket says 10.20.193.115:54848) 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff9f8138430 0x7ff9f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f67fc640 1 -- 10.20.193.115:0/3659028390 learned_addr learned my addr 10.20.193.115:0/3659028390 (peer_addr_for_me v1:10.20.193.115:0/0) 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient(hunting): get_auth_request con 0x7ff9f8138430 auth_method 0 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:52.175 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f77fe640 1 --2- 10.20.193.115:0/3659028390 >> v2:10.20.193.78:3300/0 conn(0x7ff9f813d320 0x7ff9f813f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f77fe640 10 monclient(hunting): get_auth_request con 0x7ff9f813d320 auth_method 0 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient(hunting): handle_auth_done global_id 4120 payload 274 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/3659028390 >> v2:10.20.193.78:3300/0 conn(0x7ff9f813d320 msgr2=0x7ff9f813f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 1 --2- 10.20.193.115:0/3659028390 >> v2:10.20.193.78:3300/0 conn(0x7ff9f813d320 0x7ff9f813f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/3659028390 >> v1:10.20.193.115:6789/0 conn(0x7ff9f80a6690 legacy=0x7ff9f8138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:45:52.176 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 1 monclient: found mon.noname-c 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.162+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/3659028390 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f80a6280 con 0x7ff9f8138430 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/3659028390 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f8138430 msgr2=0x7ff9f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f6ffd640 1 --2- 10.20.193.115:0/3659028390 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f8138430 0x7ff9f80a58f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8069200 tx=0x7ff9ec02ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52010dc9be5db51b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 1 -- 10.20.193.115:0/3659028390 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec03c070 con 0x7ff9f8138430 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: dump: 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.177 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 1 -- 10.20.193.115:0/3659028390 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ec02fbd0 con 0x7ff9f8138430 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 5 monclient: authenticate success, global_id 4120 2026-02-17T05:45:52.178 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 4 set_mon_vals no callback set 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 1 -- 10.20.193.115:0/3659028390 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec037440 con 0x7ff9f8138430 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f5ffb640 10 monclient: dump: 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.179 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 10 monclient: shutdown 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 1 -- 10.20.193.115:0/3659028390 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f8138430 msgr2=0x7ff9f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 1 --2- 10.20.193.115:0/3659028390 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f8138430 0x7ff9f80a58f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8069200 tx=0x7ff9ec02ed50 comp rx=0 tx=0).stop 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 1 -- 10.20.193.115:0/3659028390 shutdown_connections 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 1 --2- 10.20.193.115:0/3659028390 >> v2:10.20.193.78:3300/0 conn(0x7ff9f813d320 0x7ff9f813f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 1 --2- 10.20.193.115:0/3659028390 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f8138430 0x7ff9f80a58f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.180 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 1 -- 10.20.193.115:0/3659028390 >> 10.20.193.115:0/3659028390 conn(0x7ff9f80aab00 msgr2=0x7ff9f80aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 1 -- 10.20.193.115:0/3659028390 shutdown_connections 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.165+0000 7ff9f7fff640 1 -- 10.20.193.115:0/3659028390 wait complete. 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient: build_initial_monmap 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient: monmap: 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 0.000000 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:created 0.000000 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:52.181 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 1 Processor -- start 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 1 -- start start 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient: init 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 0x7ff9f81cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient(hunting): picked mon.noname-b con 0x7ff9f80a6690 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9f8138430 0x7ff9f8137340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient(hunting): picked mon.noname-a con 0x7ff9f8138430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:52.182 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9f813d320 0x7ff9f8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient(hunting): picked mon.noname-c con 0x7ff9f813d320 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f813fdd0 con 0x7ff9f8138430 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f813fc50 con 0x7ff9f80a6690 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f813ff50 con 0x7ff9f813d320 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f7fff640 10 monclient(hunting): authenticate will time out at 604.881529s 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 0x7ff9f81cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 0x7ff9f81cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.115:56870/0 (socket says 10.20.193.115:56870) 2026-02-17T05:45:52.183 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/2700431573 learned_addr learned my addr 10.20.193.115:0/2700431573 (peer_addr_for_me v2:10.20.193.115:0/0) 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient(hunting): get_auth_request con 0x7ff9f80a6690 auth_method 0 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient(hunting): handle_auth_done global_id 4123 payload 824 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/2700431573 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9f813d320 msgr2=0x7ff9f8071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 --2- 10.20.193.115:0/2700431573 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9f813d320 0x7ff9f8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/2700431573 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9f8138430 msgr2=0x7ff9f8137340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 --2- 10.20.193.115:0/2700431573 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9f8138430 0x7ff9f8137340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.184 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 monclient: found mon.noname-b 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.166+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/2700431573 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f8121cd0 con 0x7ff9f80a6690 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.168+0000 7ff9f6ffd640 1 -- 10.20.193.115:0/2700431573 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 msgr2=0x7ff9f81cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.168+0000 7ff9f6ffd640 1 --2- 10.20.193.115:0/2700431573 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 0x7ff9f81cee80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0099a0 tx=0x7ff9ec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.168+0000 7ff9f6ffd640 1 --2- 10.20.193.115:0/2700431573 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 0x7ff9f81cee80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0099a0 tx=0x7ff9ec0047c0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7ff9ec03c040 mon_map magic: 0 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.168+0000 7ff9cffff640 1 -- 10.20.193.115:0/2700431573 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec03c040 con 0x7ff9f80a6690 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.168+0000 7ff9cffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 10 monclient: dump: 2026-02-17T05:45:52.185 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 10 monclient: _finish_auth 0 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 10 monclient: _check_auth_tickets 2026-02-17T05:45:52.186 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 1 -- 10.20.193.115:0/2700431573 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ec047070 con 0x7ff9f80a6690 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 5 monclient: authenticate success, global_id 4123 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 10 monclient: _renew_subs 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f80a8b40 con 0x7ff9f80a6690 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f57fa640 4 set_mon_vals no callback set 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 10 monclient: _renew_subs 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f80a1b10 con 0x7ff9f80a6690 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 1 -- 10.20.193.115:0/2700431573 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec040a60 con 0x7ff9f80a6690 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.187 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 10 monclient: dump: 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9cffff640 1 -- 10.20.193.115:0/2700431573 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff9ec03f030 con 0x7ff9f80a6690 2026-02-17T05:45:52.188 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.189 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.189 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.189 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.169+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f80a58f0 con 0x7ff9f80a6690 2026-02-17T05:45:52.189 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.170+0000 7f11af41e140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-17T05:45:52.189 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.170+0000 7ff9cffff640 1 -- 10.20.193.115:0/2700431573 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ff9ec042440 con 0x7ff9f80a6690 2026-02-17T05:45:52.194 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.192+0000 7f11af41e140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-17T05:45:52.389 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.387+0000 7f11af41e140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-17T05:45:52.443 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.441+0000 7f11af41e140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-17T05:45:52.464 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.463+0000 7f11af41e140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-17T05:45:52.487 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.485+0000 7f11af41e140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-17T05:45:52.540 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.538+0000 7f11af41e140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-17T05:45:52.560 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.558+0000 7f11af41e140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-17T05:45:52.611 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:52.610+0000 7f11af41e140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-17T05:45:52.793 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.791+0000 7ff9cffff640 1 -- 10.20.193.115:0/2700431573 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7ff9ec040c00 con 0x7ff9f80a6690 2026-02-17T05:45:52.793 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.791+0000 7ff9cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.793 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.791+0000 7ff9cffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:52.825 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.823+0000 7ff9f7fff640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "4fe4aca4-d018-4166-953a-0bef54c014f4", "id": 3}] 2026-02-17T05:45:52.825 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.823+0000 7ff9f7fff640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "4fe4aca4-d018-4166-953a-0bef54c014f4", "id": 3}] 2026-02-17T05:45:52.825 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.823+0000 7ff9f7fff640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.826 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.823+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4fe4aca4-d018-4166-953a-0bef54c014f4", "id": 3} v 0) -- 0x7ff9f81215c0 con 0x7ff9f80a6690 2026-02-17T05:45:52.827 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.825+0000 7ff9cffff640 1 -- 10.20.193.115:0/2700431573 <== mon.1 v2:10.20.193.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4fe4aca4-d018-4166-953a-0bef54c014f4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ff9ec04c460 con 0x7ff9f80a6690 2026-02-17T05:45:52.827 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.825+0000 7ff9cffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "4fe4aca4-d018-4166-953a-0bef54c014f4", "id": 3}] 2026-02-17T05:45:52.827 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.825+0000 7ff9cffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:45:52.828 INFO:teuthology.orchestra.run.trial115.stdout:3 2026-02-17T05:45:52.829 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 10 monclient: shutdown 2026-02-17T05:45:52.829 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:52.829 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 msgr2=0x7ff9f81cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.829 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 --2- 10.20.193.115:0/2700431573 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 0x7ff9f81cee80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0099a0 tx=0x7ff9ec0047c0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.829 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 shutdown_connections 2026-02-17T05:45:52.829 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 --2- 10.20.193.115:0/2700431573 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9f813d320 0x7ff9f8071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.829 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 --2- 10.20.193.115:0/2700431573 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9f8138430 0x7ff9f8137340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.830 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 --2- 10.20.193.115:0/2700431573 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9f80a6690 0x7ff9f81cee80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.830 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 >> 10.20.193.115:0/2700431573 conn(0x7ff9f80aab00 msgr2=0x7ff9f80a0330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:52.830 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 shutdown_connections 2026-02-17T05:45:52.830 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.826+0000 7ff9f7fff640 1 -- 10.20.193.115:0/2700431573 wait complete. 2026-02-17T05:45:52.837 DEBUG:teuthology.orchestra.run.trial115:> sudo ceph --cluster ceph osd new faad6f1c-d3bf-43dd-b7be-66d0c85bab5e 4 2026-02-17T05:45:52.952 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient: get_monmap_and_config 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient: build_initial_monmap 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient: monmap: 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:52.943178+0000 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:52.943178+0000 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:52.953 INFO:teuthology.orchestra.run.trial115.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 1 Processor -- start 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 1 -- start start 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient: init 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c0135e70 0x7f80c0136250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient(hunting): picked mon.noname-b con 0x7f80c0135e70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:52.954 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f80c0136790 0x7f80c00a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient(hunting): picked mon.noname-c con 0x7f80c0136790 addr v2:10.20.193.115:3300/0 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f80c00a15d0 0x7f80c00a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient(hunting): picked mon.noname-a con 0x7f80c00a15d0 addr v2:10.20.193.78:3300/0 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f80c00a2bb0 con 0x7f80c00a15d0 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f80c006f140 con 0x7f80c0136790 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f80c009fe60 con 0x7f80c0135e70 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80c4a29640 10 monclient(hunting): authenticate will time out at 605.657740s 2026-02-17T05:45:52.955 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f80c0136790 0x7f80c00a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f80c0136790 0x7f80c00a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.115:56880/0 (socket says 10.20.193.115:56880) 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 1 -- 10.20.193.115:0/2553012038 learned_addr learned my addr 10.20.193.115:0/2553012038 (peer_addr_for_me v2:10.20.193.115:0/0) 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 10 monclient(hunting): get_auth_request con 0x7f80c0136790 auth_method 0 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80be575640 1 --2- 10.20.193.115:0/2553012038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c0135e70 0x7f80c0136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bed76640 1 --2- 10.20.193.115:0/2553012038 >> v2:10.20.193.78:3300/0 conn(0x7f80c00a15d0 0x7f80c00a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.942+0000 7f80bdd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:52.956 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80be575640 10 monclient(hunting): get_auth_request con 0x7f80c0135e70 auth_method 0 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80be575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80be575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80be575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bed76640 10 monclient(hunting): get_auth_request con 0x7f80c00a15d0 auth_method 0 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 10 monclient(hunting): handle_auth_done global_id 4126 payload 274 2026-02-17T05:45:52.957 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 1 -- 10.20.193.115:0/2553012038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c0135e70 msgr2=0x7f80c0136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 1 --2- 10.20.193.115:0/2553012038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c0135e70 0x7f80c0136250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 1 -- 10.20.193.115:0/2553012038 >> v2:10.20.193.78:3300/0 conn(0x7f80c00a15d0 msgr2=0x7f80c00a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 1 --2- 10.20.193.115:0/2553012038 >> v2:10.20.193.78:3300/0 conn(0x7f80c00a15d0 0x7f80c00a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 10 monclient: _finish_hunting 0 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 1 monclient: found mon.noname-c 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80bdd74640 1 -- 10.20.193.115:0/2553012038 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c00a28d0 con 0x7f80c0136790 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.943+0000 7f80be575640 1 --2- 10.20.193.115:0/2553012038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c0135e70 0x7f80c0136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bdd74640 1 -- 10.20.193.115:0/2553012038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0136790 msgr2=0x7f80c00a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bdd74640 1 --2- 10.20.193.115:0/2553012038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0136790 0x7f80c00a1090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f80c006de10 tx=0x7f80a802ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429f424140f1b4b9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 1 -- 10.20.193.115:0/2553012038 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a803c070 con 0x7f80c0136790 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: dump: 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.958 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: _finish_auth 0 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: _check_auth_tickets 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 1 -- 10.20.193.115:0/2553012038 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a802fca0 con 0x7f80c0136790 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80c4a29640 5 monclient: authenticate success, global_id 4126 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80c4a29640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 1 -- 10.20.193.115:0/2553012038 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8037660 con 0x7f80c0136790 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80c4a29640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80c4a29640 4 set_mon_vals no callback set 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.945+0000 7f80bd573640 10 monclient: dump: 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient: shutdown 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- 10.20.193.115:0/2553012038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0136790 msgr2=0x7f80c00a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 --2- 10.20.193.115:0/2553012038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0136790 0x7f80c00a1090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f80c006de10 tx=0x7f80a802ee60 comp rx=0 tx=0).stop 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- 10.20.193.115:0/2553012038 shutdown_connections 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 --2- 10.20.193.115:0/2553012038 >> v2:10.20.193.78:3300/0 conn(0x7f80c00a15d0 0x7f80c00a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 --2- 10.20.193.115:0/2553012038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0136790 0x7f80c00a1090 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 --2- 10.20.193.115:0/2553012038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c0135e70 0x7f80c0136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- 10.20.193.115:0/2553012038 >> 10.20.193.115:0/2553012038 conn(0x7f80c00a3190 msgr2=0x7f80c00a35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- 10.20.193.115:0/2553012038 shutdown_connections 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- 10.20.193.115:0/2553012038 wait complete. 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient: build_initial_monmap 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient: monmap: 2026-02-17T05:45:52.959 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 0.000000 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:created 0.000000 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 Processor -- start 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- start start 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient: init 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c00a15d0 0x7f80c01cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient(hunting): picked mon.noname-c con 0x7f80c00a15d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0135e70 0x7f80c01cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient(hunting): picked mon.noname-b con 0x7f80c0135e70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f80c0136790 0x7f80c009dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient(hunting): picked mon.noname-a con 0x7f80c0136790 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0136b70 con 0x7f80c0136790 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80c009d0a0 con 0x7f80c0135e70 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f80c00a5d70 con 0x7f80c00a15d0 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.946+0000 7f80c4a29640 10 monclient(hunting): authenticate will time out at 605.661974s 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0135e70 0x7f80c01cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0135e70 0x7f80c01cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.115:56896/0 (socket says 10.20.193.115:56896) 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 -- 10.20.193.115:0/2114702722 learned_addr learned my addr 10.20.193.115:0/2114702722 (peer_addr_for_me v2:10.20.193.115:0/0) 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient(hunting): get_auth_request con 0x7f80c0135e70 auth_method 0 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:52.960 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient(hunting): handle_auth_done global_id 4129 payload 824 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 -- 10.20.193.115:0/2114702722 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c00a15d0 msgr2=0x7f80c01cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c00a15d0 0x7f80c01cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 -- 10.20.193.115:0/2114702722 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f80c0136790 msgr2=0x7f80c009dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bed76640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f80c0136790 0x7f80c009dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f80c0136790 0x7f80c009dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient: _finish_hunting 0 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 monclient: found mon.noname-b 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bdd74640 1 -- 10.20.193.115:0/2114702722 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c009d880 con 0x7f80c0135e70 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bed76640 10 monclient: get_auth_request con 0x7f80c0136790 auth_method 0 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.947+0000 7f80bed76640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f80c0136790 0x7f80c009dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80bdd74640 1 -- 10.20.193.115:0/2114702722 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0135e70 msgr2=0x7f80c01cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80bdd74640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0135e70 0x7f80c01cf7b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80a8002410 tx=0x7f80a8004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 1 -- 10.20.193.115:0/2114702722 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8047070 con 0x7f80c0135e70 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: dump: 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 1 -- 10.20.193.115:0/2114702722 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a802fe50 con 0x7f80c0135e70 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 1 -- 10.20.193.115:0/2114702722 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a803c040 con 0x7f80c0135e70 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 10 monclient: dump: 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:52.961 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80bcd72640 4 set_mon_vals no callback set 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 5 monclient: authenticate success, global_id 4129 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 10 monclient: _renew_subs 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c0135ae0 con 0x7f80c0135e70 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 10 monclient: _renew_subs 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c0127b20 con 0x7f80c0135e70 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 1 -- 10.20.193.115:0/2114702722 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f80a803fab0 con 0x7f80c0135e70 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80a6ffd640 1 -- 10.20.193.115:0/2114702722 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f80a802fc40 con 0x7f80c0135e70 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.949+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c00a02a0 con 0x7f80c0135e70 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.951+0000 7f80a6ffd640 1 -- 10.20.193.115:0/2114702722 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f80a803fdd0 con 0x7f80c0135e70 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.951+0000 7f80a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:52.962 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.951+0000 7f80a6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:52.985 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.984+0000 7f80c4a29640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "faad6f1c-d3bf-43dd-b7be-66d0c85bab5e", "id": 4}] 2026-02-17T05:45:52.985 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.984+0000 7f80c4a29640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "faad6f1c-d3bf-43dd-b7be-66d0c85bab5e", "id": 4}] 2026-02-17T05:45:52.985 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.984+0000 7f80c4a29640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:52.985 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:52.984+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "faad6f1c-d3bf-43dd-b7be-66d0c85bab5e", "id": 4} v 0) -- 0x7f80c0069870 con 0x7f80c0135e70 2026-02-17T05:45:53.018 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:53.017+0000 7f11af41e140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-17T05:45:53.040 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:53.039+0000 7f11af41e140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-17T05:45:53.377 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:53.376+0000 7f11af41e140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-17T05:45:53.429 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.427+0000 7f80a6ffd640 1 -- 10.20.193.115:0/2114702722 <== mon.1 v2:10.20.193.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "faad6f1c-d3bf-43dd-b7be-66d0c85bab5e", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f80a8004020 con 0x7f80c0135e70 2026-02-17T05:45:53.429 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.427+0000 7f80a6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "faad6f1c-d3bf-43dd-b7be-66d0c85bab5e", "id": 4}] 2026-02-17T05:45:53.429 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.427+0000 7f80a6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:45:53.429 INFO:teuthology.orchestra.run.trial115.stdout:4 2026-02-17T05:45:53.430 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 10 monclient: shutdown 2026-02-17T05:45:53.430 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0135e70 msgr2=0x7f80c01cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0135e70 0x7f80c01cf7b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80a8002410 tx=0x7f80a8004830 comp rx=0 tx=0).stop 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 shutdown_connections 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f80c0136790 0x7f80c009dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f80c0135e70 0x7f80c01cf7b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 --2- 10.20.193.115:0/2114702722 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f80c00a15d0 0x7f80c01cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 >> 10.20.193.115:0/2114702722 conn(0x7f80c00a3190 msgr2=0x7f80c011cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 shutdown_connections 2026-02-17T05:45:53.431 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.429+0000 7f80c4a29640 1 -- 10.20.193.115:0/2114702722 wait complete. 2026-02-17T05:45:53.440 DEBUG:teuthology.orchestra.run.trial115:> sudo ceph --cluster ceph osd new 907eb455-f636-4e41-8761-c13502c9c730 5 2026-02-17T05:45:53.508 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.504+0000 7f94b9155640 10 monclient: get_monmap_and_config 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.504+0000 7f94b9155640 10 monclient: build_initial_monmap 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.504+0000 7f94b9155640 10 monclient: monmap: 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:53.505797+0000 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:53.505797+0000 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:53.509 INFO:teuthology.orchestra.run.trial115.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 1 Processor -- start 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 1 -- start start 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient: init 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f94b40a31b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f94b413a930 addr v1:10.20.193.115:6789/0 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f94b413e440 0x7f94b4140830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient(hunting): picked mon.noname-c con 0x7f94b413e440 addr v2:10.20.193.115:3300/0 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b40a5000 con 0x7f94b40a31b0 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b409f9f0 con 0x7f94b413a930 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:53.510 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f94b40a3af0 con 0x7f94b413e440 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b9155640 10 monclient(hunting): authenticate will time out at 606.220480s 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f94b413e440 0x7f94b4140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b2575640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f94b413a930 0x7f94b413cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.115:54864/0 (socket says 10.20.193.115:54864) 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b2575640 1 -- 10.20.193.115:0/67632726 learned_addr learned my addr 10.20.193.115:0/67632726 (peer_addr_for_me v1:10.20.193.115:0/0) 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient(hunting): get_auth_request con 0x7f94b413e440 auth_method 0 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient(hunting): handle_auth_done global_id 4132 payload 274 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 1 -- 10.20.193.115:0/67632726 >> v1:10.20.193.115:6789/0 conn(0x7f94b413a930 legacy=0x7f94b413cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 1 -- 10.20.193.115:0/67632726 >> v1:10.20.193.78:6789/0 conn(0x7f94b40a31b0 legacy=0x7f94b409fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:53.511 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient: _finish_hunting 0 2026-02-17T05:45:53.512 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 1 monclient: found mon.noname-c 2026-02-17T05:45:53.512 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:53.512 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:45:53.512 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.505+0000 7f94b3577640 1 -- 10.20.193.115:0/67632726 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b409f150 con 0x7f94b413e440 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b3577640 1 -- 10.20.193.115:0/67632726 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 msgr2=0x7f94b4140830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b3577640 1 --2- 10.20.193.115:0/67632726 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b4140830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009d90 tx=0x7f94a402f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d415bf2f6f21fc44 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 1 -- 10.20.193.115:0/67632726 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a403c070 con 0x7f94b413e440 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: dump: 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:53.517 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: _finish_auth 0 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: _check_auth_tickets 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 1 -- 10.20.193.115:0/67632726 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a4037640 con 0x7f94b413e440 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 5 monclient: authenticate success, global_id 4132 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 1 -- 10.20.193.115:0/67632726 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4037a10 con 0x7f94b413e440 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b1d74640 10 monclient: dump: 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:53.518 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 4 set_mon_vals no callback set 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 10 monclient: shutdown 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 1 -- 10.20.193.115:0/67632726 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 msgr2=0x7f94b4140830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 1 --2- 10.20.193.115:0/67632726 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b4140830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009d90 tx=0x7f94a402f1e0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 1 -- 10.20.193.115:0/67632726 shutdown_connections 2026-02-17T05:45:53.519 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 1 --2- 10.20.193.115:0/67632726 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b4140830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 1 -- 10.20.193.115:0/67632726 >> 10.20.193.115:0/67632726 conn(0x7f94b40a42a0 msgr2=0x7f94b40a46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 1 -- 10.20.193.115:0/67632726 shutdown_connections 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.508+0000 7f94b9155640 1 -- 10.20.193.115:0/67632726 wait complete. 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient: build_initial_monmap 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient: monmap: 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:epoch 0 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 0.000000 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:created 0.000000 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 1 Processor -- start 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 1 -- start start 2026-02-17T05:45:53.520 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient: init 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94b40a31b0 0x7f94b41cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient(hunting): picked mon.noname-a con 0x7f94b40a31b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94b413a930 0x7f94b4073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient(hunting): picked mon.noname-c con 0x7f94b413a930 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b41cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient(hunting): picked mon.noname-b con 0x7f94b413e440 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4140d70 con 0x7f94b40a31b0 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94b409f490 con 0x7f94b413e440 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f94b4140ef0 con 0x7f94b413a930 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b9155640 10 monclient(hunting): authenticate will time out at 606.224533s 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b41cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b41cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.115:56914/0 (socket says 10.20.193.115:56914) 2026-02-17T05:45:53.521 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 -- 10.20.193.115:0/1425285130 learned_addr learned my addr 10.20.193.115:0/1425285130 (peer_addr_for_me v2:10.20.193.115:0/0) 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient(hunting): get_auth_request con 0x7f94b413e440 auth_method 0 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b2d76640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94b40a31b0 0x7f94b41cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b2d76640 10 monclient(hunting): get_auth_request con 0x7f94b40a31b0 auth_method 0 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient(hunting): handle_auth_done global_id 4135 payload 824 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b2575640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94b413a930 0x7f94b4073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 -- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94b413a930 msgr2=0x7f94b4073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94b413a930 0x7f94b4073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 -- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94b40a31b0 msgr2=0x7f94b41cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:53.522 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94b40a31b0 0x7f94b41cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient: _finish_hunting 0 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 monclient: found mon.noname-b 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b3577640 1 -- 10.20.193.115:0/1425285130 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b413a300 con 0x7f94b413e440 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b2575640 10 monclient: get_auth_request con 0x7f94b413a930 auth_method 0 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.509+0000 7f94b2575640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94b413a930 0x7f94b4073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.511+0000 7f94b3577640 1 -- 10.20.193.115:0/1425285130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 msgr2=0x7f94b41cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.511+0000 7f94b3577640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b41cbf00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009ec0 tx=0x7f94a4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.511+0000 7f94937fe640 1 -- 10.20.193.115:0/1425285130 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4047020 con 0x7f94b413e440 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: dump: 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:53.523 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: _finish_auth 0 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: _check_auth_tickets 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 1 -- 10.20.193.115:0/1425285130 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a40428a0 con 0x7f94b413e440 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 5 monclient: authenticate success, global_id 4135 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 10 monclient: _renew_subs 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b413a5e0 con 0x7f94b413e440 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 1 -- 10.20.193.115:0/1425285130 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a403c040 con 0x7f94b413e440 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 10 monclient: dump: 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:epoch 1 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:53.524 INFO:teuthology.orchestra.run.trial115.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:election_strategy: 1 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b1573640 4 set_mon_vals no callback set 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 10 monclient: _renew_subs 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b41cfdf0 con 0x7f94b413e440 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 1 -- 10.20.193.115:0/1425285130 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f94a40437b0 con 0x7f94b413e440 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94937fe640 1 -- 10.20.193.115:0/1425285130 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f94a404caa0 con 0x7f94b413e440 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.512+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b4139030 con 0x7f94b413e440 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.513+0000 7f94937fe640 1 -- 10.20.193.115:0/1425285130 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f94b4139030 con 0x7f94b413e440 2026-02-17T05:45:53.525 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.513+0000 7f94937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:53.526 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.513+0000 7f94937fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:53.547 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.546+0000 7f94b9155640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "907eb455-f636-4e41-8761-c13502c9c730", "id": 5}] 2026-02-17T05:45:53.547 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.546+0000 7f94b9155640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "907eb455-f636-4e41-8761-c13502c9c730", "id": 5}] 2026-02-17T05:45:53.547 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.546+0000 7f94b9155640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:45:53.547 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.546+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "907eb455-f636-4e41-8761-c13502c9c730", "id": 5} v 0) -- 0x7f94b40a1550 con 0x7f94b413e440 2026-02-17T05:45:53.907 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.905+0000 7f94937fe640 1 -- 10.20.193.115:0/1425285130 <== mon.1 v2:10.20.193.115:3300/0 7 ==== mgrmap(e 2) ==== 50049+0+0 (secure 0 0 0) 0x7f94a4043cb0 con 0x7f94b413e440 2026-02-17T05:45:53.908 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.907+0000 7f94937fe640 1 -- 10.20.193.115:0/1425285130 <== mon.1 v2:10.20.193.115:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "907eb455-f636-4e41-8761-c13502c9c730", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f94a4051030 con 0x7f94b413e440 2026-02-17T05:45:53.909 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.907+0000 7f94937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "907eb455-f636-4e41-8761-c13502c9c730", "id": 5}] 2026-02-17T05:45:53.909 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.907+0000 7f94937fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:45:53.909 INFO:teuthology.orchestra.run.trial115.stdout:5 2026-02-17T05:45:53.910 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.907+0000 7f94b9155640 10 monclient: shutdown 2026-02-17T05:45:53.910 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.907+0000 7f94b9155640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:53.910 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.907+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 msgr2=0x7f94b41cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:53.910 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.907+0000 7f94b9155640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b41cbf00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009ec0 tx=0x7f94a4004b40 comp rx=0 tx=0).stop 2026-02-17T05:45:53.910 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.908+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 shutdown_connections 2026-02-17T05:45:53.911 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.908+0000 7f94b9155640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f94b413e440 0x7f94b41cbf00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.911 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.908+0000 7f94b9155640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94b413a930 0x7f94b4073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.911 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.908+0000 7f94b9155640 1 --2- 10.20.193.115:0/1425285130 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94b40a31b0 0x7f94b41cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:53.911 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.908+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 >> 10.20.193.115:0/1425285130 conn(0x7f94b40a42a0 msgr2=0x7f94b40a0090 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:53.911 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.908+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 shutdown_connections 2026-02-17T05:45:53.911 INFO:teuthology.orchestra.run.trial115.stderr:2026-02-17T05:45:53.908+0000 7f94b9155640 1 -- 10.20.193.115:0/1425285130 wait complete. 2026-02-17T05:45:53.917 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-17T05:45:53.918 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-17T05:45:53.920 INFO:tasks.ceph.osd.0:Started 2026-02-17T05:45:53.920 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-17T05:45:53.920 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-17T05:45:53.922 INFO:tasks.ceph.osd.1:Started 2026-02-17T05:45:53.923 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-17T05:45:53.923 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-17T05:45:53.925 INFO:tasks.ceph.osd.2:Started 2026-02-17T05:45:53.925 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-17T05:45:53.925 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-17T05:45:53.927 INFO:tasks.ceph.osd.3:Started 2026-02-17T05:45:53.928 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-17T05:45:53.928 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2026-02-17T05:45:53.930 INFO:tasks.ceph.osd.4:Started 2026-02-17T05:45:53.930 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-17T05:45:53.931 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2026-02-17T05:45:53.933 INFO:tasks.ceph.osd.5:Started 2026-02-17T05:45:53.933 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.000+0000 7f60f6eb0640 10 monclient: get_monmap_and_config 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.000+0000 7f60f6eb0640 10 monclient: build_initial_monmap 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.000+0000 7f60f6eb0640 10 monclient: monmap: 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:54.002112+0000 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:54.002112+0000 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:54.002 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:54.003 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:54.003 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:54.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 1 Processor -- start 2026-02-17T05:45:54.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 1 -- start start 2026-02-17T05:45:54.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient: init 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f60f006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f60f009d430 0x7f60f013d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient(hunting): picked mon.noname-c con 0x7f60f009d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f60f009bf40 0x7f60f013fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient(hunting): picked mon.noname-a con 0x7f60f009bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f0073600 con 0x7f60f006c890 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f60f009ca50 con 0x7f60f009bf40 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f60f006dab0 con 0x7f60f009d430 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f6eb0640 10 monclient(hunting): authenticate will time out at 599.555682s 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f4c25640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f60f006c890 0x7f60f006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52926/0 (socket says 10.20.193.78:52926) 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f4c25640 1 -- 10.20.193.78:0/808724587 learned_addr learned my addr 10.20.193.78:0/808724587 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 1 --2- 10.20.193.78:0/808724587 >> v2:10.20.193.78:3300/0 conn(0x7f60f009bf40 0x7f60f013fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient(hunting): get_auth_request con 0x7f60f009bf40 auth_method 0 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60e7fff640 1 --2- 10.20.193.78:0/808724587 >> v2:10.20.193.115:3300/0 conn(0x7f60f009d430 0x7f60f013d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60e7fff640 10 monclient(hunting): get_auth_request con 0x7f60f009d430 auth_method 0 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient(hunting): handle_auth_done global_id 4146 payload 274 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 1 -- 10.20.193.78:0/808724587 >> v2:10.20.193.115:3300/0 conn(0x7f60f009d430 msgr2=0x7f60f013d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 1 --2- 10.20.193.78:0/808724587 >> v2:10.20.193.115:3300/0 conn(0x7f60f009d430 0x7f60f013d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 1 -- 10.20.193.78:0/808724587 >> v1:10.20.193.78:6789/0 conn(0x7f60f006c890 legacy=0x7f60f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient: _finish_hunting 0 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 1 monclient: found mon.noname-a 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:45:54.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.001+0000 7f60f5426640 1 -- 10.20.193.78:0/808724587 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f009c770 con 0x7f60f009bf40 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f5426640 1 -- 10.20.193.78:0/808724587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f009bf40 msgr2=0x7f60f013fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f5426640 1 --2- 10.20.193.78:0/808724587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f009bf40 0x7f60f013fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f60f00a0ac0 tx=0x7f60ec02f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f7a2519da85c08 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 1 -- 10.20.193.78:0/808724587 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec03c070 con 0x7f60f009bf40 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: dump: 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:54.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: _finish_auth 0 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 5 monclient: authenticate success, global_id 4146 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 1 -- 10.20.193.78:0/808724587 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec037440 con 0x7f60f009bf40 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 1 -- 10.20.193.78:0/808724587 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec037810 con 0x7f60f009bf40 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60e77fe640 10 monclient: dump: 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 4 set_mon_vals no callback set 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 10 monclient: shutdown 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 1 -- 10.20.193.78:0/808724587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f009bf40 msgr2=0x7f60f013fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 1 --2- 10.20.193.78:0/808724587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f009bf40 0x7f60f013fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f60f00a0ac0 tx=0x7f60ec02f130 comp rx=0 tx=0).stop 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 1 -- 10.20.193.78:0/808724587 shutdown_connections 2026-02-17T05:45:54.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 1 --2- 10.20.193.78:0/808724587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f009bf40 0x7f60f013fb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 1 --2- 10.20.193.78:0/808724587 >> v2:10.20.193.115:3300/0 conn(0x7f60f009d430 0x7f60f013d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 1 -- 10.20.193.78:0/808724587 >> 10.20.193.78:0/808724587 conn(0x7f60f00a6620 msgr2=0x7f60f00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 1 -- 10.20.193.78:0/808724587 shutdown_connections 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.004+0000 7f60f6eb0640 1 -- 10.20.193.78:0/808724587 wait complete. 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient: build_initial_monmap 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient: monmap: 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 1 Processor -- start 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 1 -- start start 2026-02-17T05:45:54.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient: init 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f006c890 0x7f60f013e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient(hunting): picked mon.noname-a con 0x7f60f006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60f009bf40 0x7f60f013ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient(hunting): picked mon.noname-c con 0x7f60f009bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60f009d430 0x7f60f01395b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient(hunting): picked mon.noname-b con 0x7f60f009d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60f006d380 con 0x7f60f006c890 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0073410 con 0x7f60f009d430 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f60f006d500 con 0x7f60f009bf40 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f6eb0640 10 monclient(hunting): authenticate will time out at 599.559606s 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f4c25640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f006c890 0x7f60f013e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60f009bf40 0x7f60f013ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60f009bf40 0x7f60f013ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42950/0 (socket says 10.20.193.78:42950) 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 -- 10.20.193.78:0/1226485817 learned_addr learned my addr 10.20.193.78:0/1226485817 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient(hunting): get_auth_request con 0x7f60f009bf40 auth_method 0 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f4c25640 10 monclient(hunting): get_auth_request con 0x7f60f006c890 auth_method 0 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f4c25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f4c25640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f4c25640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f4c25640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f4c25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f5426640 1 --2- 10.20.193.78:0/1226485817 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60f009d430 0x7f60f01395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f4c25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f5426640 10 monclient(hunting): get_auth_request con 0x7f60f009d430 auth_method 0 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f5426640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f5426640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60f5426640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient(hunting): handle_auth_done global_id 4133 payload 824 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 -- 10.20.193.78:0/1226485817 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60f009d430 msgr2=0x7f60f01395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 --2- 10.20.193.78:0/1226485817 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60f009d430 0x7f60f01395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 -- 10.20.193.78:0/1226485817 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f006c890 msgr2=0x7f60f013e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 --2- 10.20.193.78:0/1226485817 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f006c890 0x7f60f013e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 monclient: found mon.noname-c 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:45:54.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.005+0000 7f60e7fff640 1 -- 10.20.193.78:0/1226485817 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f01360f0 con 0x7f60f009bf40 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.007+0000 7f60e7fff640 1 -- 10.20.193.78:0/1226485817 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60f009bf40 msgr2=0x7f60f013ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.007+0000 7f60e7fff640 1 --2- 10.20.193.78:0/1226485817 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60f009bf40 0x7f60f013ef00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60d800e7c0 tx=0x7f60d800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 1 -- 10.20.193.78:0/1226485817 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8017070 con 0x7f60f009bf40 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: dump: 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: _finish_auth 0 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: _check_auth_tickets 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 1 -- 10.20.193.78:0/1226485817 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d800cae0 con 0x7f60f009bf40 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 1 -- 10.20.193.78:0/1226485817 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d800f040 con 0x7f60f009bf40 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 10 monclient: dump: 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:54.012 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e6ffd640 4 set_mon_vals no callback set 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 5 monclient: authenticate success, global_id 4133 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 10 monclient: _renew_subs 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f013c480 con 0x7f60f009bf40 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 10 monclient: _renew_subs 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f009d810 con 0x7f60f009bf40 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 1 -- 10.20.193.78:0/1226485817 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 2) ==== 50049+0+0 (secure 0 0 0) 0x7f60d8012050 con 0x7f60f009bf40 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f0074980 con 0x7f60f009bf40 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.008+0000 7f60e57fa640 1 -- 10.20.193.78:0/1226485817 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f60d8051410 con 0x7f60f009bf40 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.009+0000 7f60e57fa640 1 -- 10.20.193.78:0/1226485817 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f60d8013410 con 0x7f60f009bf40 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.009+0000 7f60e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:54.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.009+0000 7f60e57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:54.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.036+0000 7f60f6eb0640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:45:54.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.036+0000 7f60f6eb0640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:45:54.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.036+0000 7f60f6eb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:45:54.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.036+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60f006cc70 con 0x7f60f009bf40 2026-02-17T05:45:54.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.036+0000 7f60e57fa640 1 -- 10.20.193.78:0/1226485817 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (secure 0 0 0) 0x7f60d8023d80 con 0x7f60f009bf40 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.036+0000 7f60e57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.036+0000 7f60e57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stdout:{"epoch":8,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T05:45:53.905739+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 10 monclient: shutdown 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60f009bf40 msgr2=0x7f60f013ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 --2- 10.20.193.78:0/1226485817 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60f009bf40 0x7f60f013ef00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60d800e7c0 tx=0x7f60d800ec90 comp rx=0 tx=0).stop 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 shutdown_connections 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 --2- 10.20.193.78:0/1226485817 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60f009d430 0x7f60f01395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 --2- 10.20.193.78:0/1226485817 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60f009bf40 0x7f60f013ef00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 --2- 10.20.193.78:0/1226485817 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60f006c890 0x7f60f013e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 >> 10.20.193.78:0/1226485817 conn(0x7f60f00a6620 msgr2=0x7f60f00a27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 shutdown_connections 2026-02-17T05:45:54.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:45:54.037+0000 7f60f6eb0640 1 -- 10.20.193.78:0/1226485817 wait complete. 2026-02-17T05:45:54.045 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-17T05:45:54.045 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-17T05:45:54.649 DEBUG:teuthology.orchestra.run.trial078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:45:54.721 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.710+0000 7fe7bebcc640 10 monclient: get_monmap_and_config 2026-02-17T05:45:54.721 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.710+0000 7fe7bebcc640 10 monclient: build_initial_monmap 2026-02-17T05:45:54.721 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.710+0000 7fe7bebcc640 10 monclient: monmap: 2026-02-17T05:45:54.721 INFO:teuthology.misc.health.trial078.stderr:epoch 0 2026-02-17T05:45:54.721 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.721 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:54.711741+0000 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:54.711741+0000 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.710+0000 7fe7bebcc640 1 Processor -- start 2026-02-17T05:45:54.722 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.710+0000 7fe7bebcc640 1 -- start start 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.710+0000 7fe7bebcc640 10 monclient: init 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe7b809fb60 0x7fe7b8138680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient(hunting): picked mon.noname-a con 0x7fe7b809fb60 addr v2:10.20.193.78:3300/0 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe7b809ff40 addr v1:10.20.193.115:6789/0 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b80a0380 0x7fe7b80a1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient(hunting): picked mon.noname-b con 0x7fe7b80a0380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b80a20b0 con 0x7fe7b809ff40 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.723 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b80a2390 con 0x7fe7b809fb60 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b8138bc0 con 0x7fe7b80a0380 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bebcc640 10 monclient(hunting): authenticate will time out at 600.265320s 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe7b809fb60 0x7fe7b8138680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe7b809fb60 0x7fe7b8138680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58724/0 (socket says 10.20.193.78:58724) 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 1 -- 10.20.193.78:0/3706305595 learned_addr learned my addr 10.20.193.78:0/3706305595 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient(hunting): get_auth_request con 0x7fe7b809fb60 auth_method 0 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.724 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bd142640 1 --2- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b80a0380 0x7fe7b80a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bd142640 10 monclient(hunting): get_auth_request con 0x7fe7b80a0380 auth_method 0 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bd142640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bd142640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bd142640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bd142640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bd142640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bd142640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient(hunting): handle_auth_done global_id 4152 payload 274 2026-02-17T05:45:54.725 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 1 -- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b80a0380 msgr2=0x7fe7b80a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 1 --2- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b80a0380 0x7fe7b80a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 1 -- 10.20.193.78:0/3706305595 >> v1:10.20.193.115:6789/0 conn(0x7fe7b809ff40 legacy=0x7fe7b80a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient: _finish_hunting 0 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 1 monclient: found mon.noname-a 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.711+0000 7fe7bc941640 1 -- 10.20.193.78:0/3706305595 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b806d5a0 con 0x7fe7b809fb60 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bc941640 1 -- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 msgr2=0x7fe7b8138680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bc941640 1 --2- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b8138680 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009970 tx=0x7fe7a802ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbd37736214f0ad server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 1 -- 10.20.193.78:0/3706305595 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a803c070 con 0x7fe7b809fb60 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:54.726 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: dump: 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: _finish_auth 0 2026-02-17T05:45:54.727 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 5 monclient: authenticate success, global_id 4152 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 1 -- 10.20.193.78:0/3706305595 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a802fcd0 con 0x7fe7b809fb60 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 1 -- 10.20.193.78:0/3706305595 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8037660 con 0x7fe7b809fb60 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 10 monclient: get_monmap_and_config success 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 4 set_mon_vals no callback set 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:45:54.728 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7af7fe640 10 monclient: dump: 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 10 monclient: shutdown 2026-02-17T05:45:54.729 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 1 -- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 msgr2=0x7fe7b8138680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 1 --2- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b8138680 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009970 tx=0x7fe7a802ee90 comp rx=0 tx=0).stop 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 1 -- 10.20.193.78:0/3706305595 shutdown_connections 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 1 --2- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b80a0380 0x7fe7b80a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 1 --2- 10.20.193.78:0/3706305595 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b8138680 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 1 -- 10.20.193.78:0/3706305595 >> 10.20.193.78:0/3706305595 conn(0x7fe7b806c810 msgr2=0x7fe7b806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 1 -- 10.20.193.78:0/3706305595 shutdown_connections 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 1 -- 10.20.193.78:0/3706305595 wait complete. 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 10 monclient: build_initial_monmap 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.714+0000 7fe7bebcc640 10 monclient: monmap: 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:epoch 0 2026-02-17T05:45:54.730 INFO:teuthology.misc.health.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:last_changed 0.000000 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:created 0.000000 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 1 Processor -- start 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 1 -- start start 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient: init 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient: _reopen_session rank -1 2026-02-17T05:45:54.731 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b813bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient(hunting): picked mon.noname-a con 0x7fe7b809fb60 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b809ff40 0x7fe7b813c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient(hunting): picked mon.noname-c con 0x7fe7b809ff40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe7b80a0380 0x7fe7b813ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient(hunting): picked mon.noname-b con 0x7fe7b80a0380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8140680 con 0x7fe7b809fb60 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8140500 con 0x7fe7b80a0380 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:45:54.732 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b8140800 con 0x7fe7b809ff40 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient(hunting): _renew_subs 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bebcc640 10 monclient(hunting): authenticate will time out at 600.269456s 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b813bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b813bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58738/0 (socket says 10.20.193.78:58738) 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 -- 10.20.193.78:0/4190966358 learned_addr learned my addr 10.20.193.78:0/4190966358 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient(hunting): get_auth_request con 0x7fe7b809fb60 auth_method 0 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7affff640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b809ff40 0x7fe7b813c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7affff640 10 monclient(hunting): get_auth_request con 0x7fe7b809ff40 auth_method 0 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:45:54.733 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7affff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient(hunting): handle_auth_done global_id 4155 payload 824 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bd142640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe7b80a0380 0x7fe7b813ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 -- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b809ff40 msgr2=0x7fe7b813c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b809ff40 0x7fe7b813c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.734 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 -- 10.20.193.78:0/4190966358 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe7b80a0380 msgr2=0x7fe7b813ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe7b80a0380 0x7fe7b813ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient: _finish_hunting 0 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 monclient: found mon.noname-a 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bc941640 1 -- 10.20.193.78:0/4190966358 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b813fb20 con 0x7fe7b809fb60 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bd142640 10 monclient: get_auth_request con 0x7fe7b80a0380 auth_method 0 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.715+0000 7fe7bd142640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe7b80a0380 0x7fe7b813ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bc941640 1 -- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 msgr2=0x7fe7b813bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bc941640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b813bf90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009aa0 tx=0x7fe7a8004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 1 -- 10.20.193.78:0/4190966358 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8047070 con 0x7fe7b809fb60 2026-02-17T05:45:54.735 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: dump: 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:54.736 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: _finish_auth 0 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bebcc640 5 monclient: authenticate success, global_id 4155 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 1 -- 10.20.193.78:0/4190966358 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a803f790 con 0x7fe7b809fb60 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 1 -- 10.20.193.78:0/4190966358 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a803c040 con 0x7fe7b809fb60 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7aeffd640 4 set_mon_vals no callback set 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7ad7fa640 10 monclient: dump: 2026-02-17T05:45:54.737 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:45:54.738 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:45:54.738 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.738 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:45:54.738 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:45:54.738 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:45:54.738 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:45:54.738 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bebcc640 10 monclient: _renew_subs 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bebcc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b8132910 con 0x7fe7b809fb60 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bebcc640 10 monclient: _renew_subs 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bebcc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.717+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b813e390 con 0x7fe7b809fb60 2026-02-17T05:45:54.739 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.718+0000 7fe7bebcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:54.740 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.718+0000 7fe7bebcc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:54.740 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.718+0000 7fe7bebcc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:45:54.740 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.718+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b8073130 con 0x7fe7b809fb60 2026-02-17T05:45:54.740 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.718+0000 7fe7ad7fa640 1 -- 10.20.193.78:0/4190966358 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 2) ==== 50049+0+0 (secure 0 0 0) 0x7fe7a803fb80 con 0x7fe7b809fb60 2026-02-17T05:45:54.740 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.718+0000 7fe7ad7fa640 1 -- 10.20.193.78:0/4190966358 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe7a80760b0 con 0x7fe7b809fb60 2026-02-17T05:45:54.740 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.719+0000 7fe7ad7fa640 1 -- 10.20.193.78:0/4190966358 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7fe7a80628b0 con 0x7fe7b809fb60 2026-02-17T05:45:54.740 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.719+0000 7fe7ad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:45:54.740 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.719+0000 7fe7ad7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:45:54.750 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.749+0000 7fe7bebcc640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:45:54.751 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.749+0000 7fe7bebcc640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:45:54.751 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.749+0000 7fe7bebcc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:45:54.751 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.749+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7b8138660 con 0x7fe7b809fb60 2026-02-17T05:45:54.751 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.749+0000 7fe7ad7fa640 1 -- 10.20.193.78:0/4190966358 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (secure 0 0 0) 0x7fe7a80413d0 con 0x7fe7b809fb60 2026-02-17T05:45:54.751 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.749+0000 7fe7ad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:45:54.751 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.749+0000 7fe7ad7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:45:54.751 INFO:teuthology.misc.health.trial078.stdout: 2026-02-17T05:45:54.752 INFO:teuthology.misc.health.trial078.stdout:{"epoch":8,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T05:45:53.905739+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 10 monclient: shutdown 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 msgr2=0x7fe7b813bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b813bf90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009aa0 tx=0x7fe7a8004830 comp rx=0 tx=0).stop 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 shutdown_connections 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe7b80a0380 0x7fe7b813ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe7b809ff40 0x7fe7b813c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 --2- 10.20.193.78:0/4190966358 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe7b809fb60 0x7fe7b813bf90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 >> 10.20.193.78:0/4190966358 conn(0x7fe7b806c810 msgr2=0x7fe7b80a5860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:45:54.753 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 shutdown_connections 2026-02-17T05:45:54.754 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:45:54.750+0000 7fe7bebcc640 1 -- 10.20.193.78:0/4190966358 wait complete. 2026-02-17T05:45:54.758 DEBUG:teuthology.misc:0 of 6 OSDs are up 2026-02-17T05:45:54.925 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:54.923+0000 7f10cbed1640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-17T05:45:54.925 INFO:tasks.ceph.mgr.x.trial115.stderr:2026-02-17T05:45:54.923+0000 7f10cbed1640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-17T05:45:55.339 INFO:tasks.ceph.osd.4.trial115.stderr:2026-02-17T05:45:55.337+0000 7fbb2a03f780 -1 Falling back to public interface 2026-02-17T05:45:55.571 INFO:tasks.ceph.osd.0.trial078.stderr:2026-02-17T05:45:55.570+0000 7ffa7a73d780 -1 Falling back to public interface 2026-02-17T05:45:55.601 INFO:tasks.ceph.osd.1.trial078.stderr:2026-02-17T05:45:55.600+0000 7f028401f780 -1 Falling back to public interface 2026-02-17T05:45:55.609 INFO:tasks.ceph.osd.2.trial078.stderr:2026-02-17T05:45:55.608+0000 7efc5285f780 -1 Falling back to public interface 2026-02-17T05:45:55.828 INFO:tasks.ceph.osd.3.trial115.stderr:2026-02-17T05:45:55.826+0000 7f398be0e780 -1 Falling back to public interface 2026-02-17T05:45:55.828 INFO:tasks.ceph.osd.5.trial115.stderr:2026-02-17T05:45:55.826+0000 7f5281c93780 -1 Falling back to public interface 2026-02-17T05:45:59.090 INFO:tasks.ceph.osd.4.trial115.stderr:2026-02-17T05:45:59.088+0000 7fbb2a03f780 -1 osd.4 0 log_to_monitors true 2026-02-17T05:45:59.315 INFO:tasks.ceph.osd.0.trial078.stderr:2026-02-17T05:45:59.314+0000 7ffa7a73d780 -1 osd.0 0 log_to_monitors true 2026-02-17T05:45:59.590 INFO:tasks.ceph.osd.5.trial115.stderr:2026-02-17T05:45:59.588+0000 7f5281c93780 -1 osd.5 0 log_to_monitors true 2026-02-17T05:45:59.605 INFO:tasks.ceph.osd.1.trial078.stderr:2026-02-17T05:45:59.604+0000 7f028401f780 -1 osd.1 0 log_to_monitors true 2026-02-17T05:45:59.658 INFO:tasks.ceph.osd.2.trial078.stderr:2026-02-17T05:45:59.656+0000 7efc5285f780 -1 osd.2 0 log_to_monitors true 2026-02-17T05:46:00.073 INFO:tasks.ceph.osd.3.trial115.stderr:2026-02-17T05:46:00.071+0000 7f398be0e780 -1 osd.3 0 log_to_monitors true 2026-02-17T05:46:00.429 INFO:tasks.ceph.osd.4.trial115.stderr:2026-02-17T05:46:00.428+0000 7fbb25fcc640 -1 osd.4 0 waiting for initial osdmap 2026-02-17T05:46:00.444 INFO:tasks.ceph.osd.4.trial115.stderr:2026-02-17T05:46:00.443+0000 7fbb1ddee640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:46:01.163 INFO:tasks.ceph.osd.0.trial078.stderr:2026-02-17T05:46:01.162+0000 7ffa766ca640 -1 osd.0 0 waiting for initial osdmap 2026-02-17T05:46:01.164 INFO:tasks.ceph.osd.1.trial078.stderr:2026-02-17T05:46:01.162+0000 7f027fa03640 -1 osd.1 0 waiting for initial osdmap 2026-02-17T05:46:01.164 INFO:tasks.ceph.osd.2.trial078.stderr:2026-02-17T05:46:01.162+0000 7efc4e7ee640 -1 osd.2 0 waiting for initial osdmap 2026-02-17T05:46:01.165 INFO:tasks.ceph.osd.0.trial078.stderr:2026-02-17T05:46:01.164+0000 7ffa724f4640 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:46:01.166 INFO:tasks.ceph.osd.1.trial078.stderr:2026-02-17T05:46:01.164+0000 7f027b82d640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:46:01.166 INFO:tasks.ceph.osd.2.trial078.stderr:2026-02-17T05:46:01.164+0000 7efc4a618640 -1 osd.2 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:46:01.167 INFO:tasks.ceph.osd.3.trial115.stderr:2026-02-17T05:46:01.166+0000 7f39877f4640 -1 osd.3 0 waiting for initial osdmap 2026-02-17T05:46:01.169 INFO:tasks.ceph.osd.3.trial115.stderr:2026-02-17T05:46:01.168+0000 7f398361e640 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:46:01.363 DEBUG:teuthology.orchestra.run.trial078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:46:01.426 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.422+0000 7f03aad5e640 10 monclient: get_monmap_and_config 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.422+0000 7f03aad5e640 10 monclient: build_initial_monmap 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.422+0000 7f03aad5e640 10 monclient: monmap: 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:epoch 0 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:46:01.423643+0000 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:46:01.423643+0000 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:01.427 INFO:teuthology.misc.health.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.422+0000 7f03aad5e640 1 Processor -- start 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.422+0000 7f03aad5e640 1 -- start start 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.422+0000 7f03aad5e640 10 monclient: init 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2026-02-17T05:46:01.428 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f03a4072d20 0x7f03a4136330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient(hunting): picked mon.noname-a con 0x7f03a4072d20 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f03a4073100 0x7f03a40a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient(hunting): picked mon.noname-c con 0x7f03a4073100 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a40a1250 0x7f03a40a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient(hunting): picked mon.noname-b con 0x7f03a40a1250 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f03a4136870 con 0x7f03a4072d20 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f03a4073b80 con 0x7f03a4073100 2026-02-17T05:46:01.429 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f03a40a2040 con 0x7f03a40a1250 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03aad5e640 10 monclient(hunting): authenticate will time out at 606.977186s 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f03a4072d20 0x7f03a4136330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f03a4072d20 0x7f03a4136330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40044/0 (socket says 10.20.193.78:40044) 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a92d4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a40a1250 0x7f03a40a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 -- 10.20.193.78:0/2958858910 learned_addr learned my addr 10.20.193.78:0/2958858910 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient(hunting): get_auth_request con 0x7f03a4072d20 auth_method 0 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:01.430 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a92d4640 10 monclient(hunting): get_auth_request con 0x7f03a40a1250 auth_method 0 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a92d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a92d4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a92d4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a92d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a92d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:01.431 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a92d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f039bfff640 1 --2- 10.20.193.78:0/2958858910 >> v2:10.20.193.115:3300/0 conn(0x7f03a4073100 0x7f03a40a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f039bfff640 10 monclient(hunting): get_auth_request con 0x7f03a4073100 auth_method 0 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f039bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f039bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f039bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient(hunting): handle_auth_done global_id 4173 payload 274 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 -- 10.20.193.78:0/2958858910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a40a1250 msgr2=0x7f03a40a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 --2- 10.20.193.78:0/2958858910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a40a1250 0x7f03a40a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 -- 10.20.193.78:0/2958858910 >> v2:10.20.193.115:3300/0 conn(0x7f03a4073100 msgr2=0x7f03a40a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 --2- 10.20.193.78:0/2958858910 >> v2:10.20.193.115:3300/0 conn(0x7f03a4073100 0x7f03a40a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient: _finish_hunting 0 2026-02-17T05:46:01.432 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 monclient: found mon.noname-a 2026-02-17T05:46:01.433 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:01.433 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:01.433 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.423+0000 7f03a8ad3640 1 -- 10.20.193.78:0/2958858910 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a40a2320 con 0x7f03a4072d20 2026-02-17T05:46:01.440 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03a8ad3640 1 -- 10.20.193.78:0/2958858910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a4072d20 msgr2=0x7f03a4136330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:01.441 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03a8ad3640 1 --2- 10.20.193.78:0/2958858910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a4072d20 0x7f03a4136330 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f03a40a0380 tx=0x7f039402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245fa2bbf3ebe6f4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:01.441 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 1 -- 10.20.193.78:0/2958858910 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039403c070 con 0x7f03a4072d20 2026-02-17T05:46:01.441 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:01.441 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:01.441 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: dump: 2026-02-17T05:46:01.441 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:46:01.441 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:01.442 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 1 -- 10.20.193.78:0/2958858910 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039402fca0 con 0x7f03a4072d20 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 5 monclient: authenticate success, global_id 4173 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 1 -- 10.20.193.78:0/2958858910 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394037660 con 0x7f03a4072d20 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 4 set_mon_vals no callback set 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f039b7fe640 10 monclient: dump: 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:01.443 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 10 monclient: shutdown 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 -- 10.20.193.78:0/2958858910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a4072d20 msgr2=0x7f03a4136330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 --2- 10.20.193.78:0/2958858910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a4072d20 0x7f03a4136330 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f03a40a0380 tx=0x7f039402ee60 comp rx=0 tx=0).stop 2026-02-17T05:46:01.444 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 -- 10.20.193.78:0/2958858910 shutdown_connections 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 --2- 10.20.193.78:0/2958858910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a40a1250 0x7f03a40a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 --2- 10.20.193.78:0/2958858910 >> v2:10.20.193.115:3300/0 conn(0x7f03a4073100 0x7f03a40a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 --2- 10.20.193.78:0/2958858910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a4072d20 0x7f03a4136330 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 -- 10.20.193.78:0/2958858910 >> 10.20.193.78:0/2958858910 conn(0x7f03a406c810 msgr2=0x7f03a406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 -- 10.20.193.78:0/2958858910 shutdown_connections 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 1 -- 10.20.193.78:0/2958858910 wait complete. 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 10 monclient: build_initial_monmap 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.426+0000 7f03aad5e640 10 monclient: monmap: 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:epoch 0 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:01.445 INFO:teuthology.misc.health.trial078.stderr:created 0.000000 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 1 Processor -- start 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 1 -- start start 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient: init 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03a4072d20 0x7f03a41cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:01.446 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient(hunting): picked mon.noname-b con 0x7f03a4072d20 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a4073100 0x7f03a4133050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient(hunting): picked mon.noname-c con 0x7f03a4073100 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a40a1250 0x7f03a4133590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient(hunting): picked mon.noname-a con 0x7f03a40a1250 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4073570 con 0x7f03a40a1250 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03a40a28e0 con 0x7f03a4072d20 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f03a40736f0 con 0x7f03a4073100 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:01.447 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03aad5e640 10 monclient(hunting): authenticate will time out at 606.981466s 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a40a1250 0x7f03a4133590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a4073100 0x7f03a4133050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a40a1250 0x7f03a4133590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40060/0 (socket says 10.20.193.78:40060) 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 -- 10.20.193.78:0/3096070806 learned_addr learned my addr 10.20.193.78:0/3096070806 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a4073100 0x7f03a4133050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40250/0 (socket says 10.20.193.78:40250) 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient(hunting): get_auth_request con 0x7f03a40a1250 auth_method 0 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 10 monclient(hunting): get_auth_request con 0x7f03a4073100 auth_method 0 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:01.448 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a8ad3640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03a4072d20 0x7f03a41cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a8ad3640 10 monclient(hunting): get_auth_request con 0x7f03a4072d20 auth_method 0 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a8ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a8ad3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a8ad3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:01.449 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient(hunting): handle_auth_done global_id 4176 payload 824 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 -- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a4073100 msgr2=0x7f03a4133050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a4073100 0x7f03a4133050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 -- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03a4072d20 msgr2=0x7f03a41cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03a4072d20 0x7f03a41cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient: _finish_hunting 0 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 monclient: found mon.noname-a 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a92d4640 1 -- 10.20.193.78:0/3096070806 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a4124780 con 0x7f03a40a1250 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f039bfff640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a4073100 0x7f03a4133050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.427+0000 7f03a8ad3640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03a4072d20 0x7f03a41cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:46:01.450 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03a92d4640 1 -- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a40a1250 msgr2=0x7f03a4133590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03a92d4640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a40a1250 0x7f03a4133590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009dd0 tx=0x7f03a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 1 -- 10.20.193.78:0/3096070806 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0018020 con 0x7f03a40a1250 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: dump: 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:01.451 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: _finish_auth 0 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: _check_auth_tickets 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03aad5e640 5 monclient: authenticate success, global_id 4176 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 1 -- 10.20.193.78:0/3096070806 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a0014680 con 0x7f03a40a1250 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 1 -- 10.20.193.78:0/3096070806 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0014a70 con 0x7f03a40a1250 2026-02-17T05:46:01.452 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03aad5e640 10 monclient: _renew_subs 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03aad5e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a4125470 con 0x7f03a40a1250 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f039affd640 4 set_mon_vals no callback set 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.429+0000 7f03997fa640 10 monclient: dump: 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:01.453 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03aad5e640 10 monclient: _renew_subs 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03aad5e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a41261d0 con 0x7f03a40a1250 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03aad5e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03aad5e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03aad5e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a406d680 con 0x7f03a40a1250 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03997fa640 1 -- 10.20.193.78:0/3096070806 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 4) ==== 50282+0+0 (secure 0 0 0) 0x7f03a0014c10 con 0x7f03a40a1250 2026-02-17T05:46:01.454 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.430+0000 7f03997fa640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f037403de20 0x7f03740402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:01.455 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.431+0000 7f03997fa640 1 -- 10.20.193.78:0/3096070806 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2203+0+0 (secure 0 0 0) 0x7f03a0052d40 con 0x7f03a40a1250 2026-02-17T05:46:01.455 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.431+0000 7f03a8ad3640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f037403de20 0x7f03740402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:01.455 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.431+0000 7f03a8ad3640 10 monclient: get_auth_request con 0x7f037403de20 auth_method 0 2026-02-17T05:46:01.455 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.433+0000 7f03997fa640 1 -- 10.20.193.78:0/3096070806 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f03a0022030 con 0x7f03a40a1250 2026-02-17T05:46:01.455 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.433+0000 7f03997fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:01.455 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.433+0000 7f03997fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:01.455 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.433+0000 7f03a8ad3640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f037403de20 0x7f03740402e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0394036000 tx=0x7f0394039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:01.456 INFO:tasks.ceph.osd.5.trial115.stderr:2026-02-17T05:46:01.443+0000 7f527dc20640 -1 osd.5 0 waiting for initial osdmap 2026-02-17T05:46:01.460 INFO:tasks.ceph.osd.5.trial115.stderr:2026-02-17T05:46:01.459+0000 7f5275a42640 -1 osd.5 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T05:46:01.517 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.516+0000 7f03aad5e640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:01.517 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.516+0000 7f03aad5e640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:01.517 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.516+0000 7f03aad5e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:01.518 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.516+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03a406d0f0 con 0x7f03a40a1250 2026-02-17T05:46:01.518 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.516+0000 7f03997fa640 1 -- 10.20.193.78:0/3096070806 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+6115 (secure 0 0 0) 0x7f03a0010040 con 0x7f03a40a1250 2026-02-17T05:46:01.518 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.516+0000 7f03997fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:01.518 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.516+0000 7f03997fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:01.518 INFO:teuthology.misc.health.trial078.stdout: 2026-02-17T05:46:01.518 INFO:teuthology.misc.health.trial078.stdout:{"epoch":11,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T05:46:01.149490+0000","last_up_change":"2026-02-17T05:46:01.149490+0000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6802","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6803","nonce":2725813951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6804","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6805","nonce":2725813951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6808","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6809","nonce":2725813951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6806","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6807","nonce":2725813951}]},"public_addr":"10.20.193.115:6803/2725813951","cluster_addr":"10.20.193.115:6805/2725813951","heartbeat_back_addr":"10.20.193.115:6809/2725813951","heartbeat_front_addr":"10.20.193.115:6807/2725813951","state":["exists","up"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:46:01.519 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f037403de20 msgr2=0x7f03740402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f037403de20 0x7f03740402e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0394036000 tx=0x7f0394039040 comp rx=0 tx=0).stop 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 10 monclient: shutdown 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a40a1250 msgr2=0x7f03a4133590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a40a1250 0x7f03a4133590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009dd0 tx=0x7f03a000c490 comp rx=0 tx=0).stop 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 shutdown_connections 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03a40a1250 0x7f03a4133590 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03a4073100 0x7f03a4133050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f037403de20 0x7f03740402e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 --2- 10.20.193.78:0/3096070806 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03a4072d20 0x7f03a41cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:01.520 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 >> 10.20.193.78:0/3096070806 conn(0x7f03a406c810 msgr2=0x7f03a4134b70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:01.521 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 shutdown_connections 2026-02-17T05:46:01.521 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:01.517+0000 7f03aad5e640 1 -- 10.20.193.78:0/3096070806 wait complete. 2026-02-17T05:46:01.525 DEBUG:teuthology.misc:1 of 6 OSDs are up 2026-02-17T05:46:08.129 DEBUG:teuthology.orchestra.run.trial078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T05:46:08.189 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient: get_monmap_and_config 2026-02-17T05:46:08.189 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient: build_initial_monmap 2026-02-17T05:46:08.189 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient: monmap: 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:epoch 0 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:46:08.189394+0000 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:46:08.189394+0000 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:08.190 INFO:teuthology.misc.health.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:08.191 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:08.200 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 1 Processor -- start 2026-02-17T05:46:08.200 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 1 -- start start 2026-02-17T05:46:08.200 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient: init 2026-02-17T05:46:08.200 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:08.200 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2026-02-17T05:46:08.200 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0bf8072d20 0x7f0bf8136330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient(hunting): picked mon.noname-a con 0x7f0bf8072d20 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073100 0x7f0bf80a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient(hunting): picked mon.noname-b con 0x7f0bf8073100 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0bf8073540 0x7f0bf80a1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient(hunting): picked mon.noname-c con 0x7f0bf8073540 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf8073b80 con 0x7f0bf8072d20 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf80a2040 con 0x7f0bf8073540 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.201 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf80a2320 con 0x7f0bf8073100 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.188+0000 7f0bff2dc640 10 monclient(hunting): authenticate will time out at 613.742945s 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073100 0x7f0bf80a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd051640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0bf8072d20 0x7f0bf8136330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073100 0x7f0bf80a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47596/0 (socket says 10.20.193.78:47596) 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 -- 10.20.193.78:0/561982531 learned_addr learned my addr 10.20.193.78:0/561982531 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient(hunting): get_auth_request con 0x7f0bf8073100 auth_method 0 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:08.202 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd051640 10 monclient(hunting): get_auth_request con 0x7f0bf8072d20 auth_method 0 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd051640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd051640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd051640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd852640 1 --2- 10.20.193.78:0/561982531 >> v2:10.20.193.115:3300/0 conn(0x7f0bf8073540 0x7f0bf80a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd051640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd051640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd051640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd852640 10 monclient(hunting): get_auth_request con 0x7f0bf8073540 auth_method 0 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd852640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd852640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.203 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfd852640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient(hunting): handle_auth_done global_id 4163 payload 274 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 -- 10.20.193.78:0/561982531 >> v2:10.20.193.115:3300/0 conn(0x7f0bf8073540 msgr2=0x7f0bf80a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 --2- 10.20.193.78:0/561982531 >> v2:10.20.193.115:3300/0 conn(0x7f0bf8073540 0x7f0bf80a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 -- 10.20.193.78:0/561982531 >> v2:10.20.193.78:3300/0 conn(0x7f0bf8072d20 msgr2=0x7f0bf8136330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 --2- 10.20.193.78:0/561982531 >> v2:10.20.193.78:3300/0 conn(0x7f0bf8072d20 0x7f0bf8136330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient: _finish_hunting 0 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 monclient: found mon.noname-b 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.189+0000 7f0bfc850640 1 -- 10.20.193.78:0/561982531 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf80a2600 con 0x7f0bf8073100 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0bfc850640 1 -- 10.20.193.78:0/561982531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073100 msgr2=0x7f0bf80a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:08.204 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0bfc850640 1 --2- 10.20.193.78:0/561982531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073100 0x7f0bf80a0d10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80a1f70 tx=0x7f0bec02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0068e65fcc6c15b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 1 -- 10.20.193.78:0/561982531 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec03c070 con 0x7f0bf8073100 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: dump: 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:08.205 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: _finish_auth 0 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 1 -- 10.20.193.78:0/561982531 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bec02fca0 con 0x7f0bf8073100 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0bff2dc640 5 monclient: authenticate success, global_id 4163 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0bff2dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 1 -- 10.20.193.78:0/561982531 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec037660 con 0x7f0bf8073100 2026-02-17T05:46:08.206 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0be7fff640 10 monclient: dump: 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:08.207 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0bff2dc640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.191+0000 7f0bff2dc640 4 set_mon_vals no callback set 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 10 monclient: shutdown 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 -- 10.20.193.78:0/561982531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073100 msgr2=0x7f0bf80a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/561982531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073100 0x7f0bf80a0d10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80a1f70 tx=0x7f0bec02ee60 comp rx=0 tx=0).stop 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 -- 10.20.193.78:0/561982531 shutdown_connections 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/561982531 >> v2:10.20.193.115:3300/0 conn(0x7f0bf8073540 0x7f0bf80a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/561982531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073100 0x7f0bf80a0d10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/561982531 >> v2:10.20.193.78:3300/0 conn(0x7f0bf8072d20 0x7f0bf8136330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 -- 10.20.193.78:0/561982531 >> 10.20.193.78:0/561982531 conn(0x7f0bf806c810 msgr2=0x7f0bf806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 -- 10.20.193.78:0/561982531 shutdown_connections 2026-02-17T05:46:08.208 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 -- 10.20.193.78:0/561982531 wait complete. 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 10 monclient: build_initial_monmap 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 10 monclient: monmap: 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:epoch 0 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:created 0.000000 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 Processor -- start 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 1 -- start start 2026-02-17T05:46:08.209 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.192+0000 7f0bff2dc640 10 monclient: init 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bf8072d20 0x7f0bf8133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient(hunting): picked mon.noname-a con 0x7f0bf8072d20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bf8073100 0x7f0bf8134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient(hunting): picked mon.noname-b con 0x7f0bf8073100 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073540 0x7f0bf811d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient(hunting): picked mon.noname-c con 0x7f0bf8073540 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf806d690 con 0x7f0bf8072d20 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf806d510 con 0x7f0bf8073100 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.210 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf80a1790 con 0x7f0bf8073540 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bff2dc640 10 monclient(hunting): authenticate will time out at 613.747271s 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bf8072d20 0x7f0bf8133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bf8072d20 0x7f0bf8133ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52508/0 (socket says 10.20.193.78:52508) 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 -- 10.20.193.78:0/2108832197 learned_addr learned my addr 10.20.193.78:0/2108832197 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient(hunting): get_auth_request con 0x7f0bf8072d20 auth_method 0 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd852640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073540 0x7f0bf811d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd852640 10 monclient(hunting): get_auth_request con 0x7f0bf8073540 auth_method 0 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd852640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.211 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd852640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd852640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd852640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd852640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd852640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient(hunting): handle_auth_done global_id 4188 payload 824 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 -- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073540 msgr2=0x7f0bf811d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073540 0x7f0bf811d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfc850640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bf8073100 0x7f0bf8134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 -- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bf8073100 msgr2=0x7f0bf8134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bf8073100 0x7f0bf8134220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.212 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient: _finish_hunting 0 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 monclient: found mon.noname-a 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfd051640 1 -- 10.20.193.78:0/2108832197 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf81213c0 con 0x7f0bf8072d20 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfc850640 10 monclient: get_auth_request con 0x7f0bf8073100 auth_method 0 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.193+0000 7f0bfc850640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bf8073100 0x7f0bf8134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bfd051640 1 -- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bf8072d20 msgr2=0x7f0bf8133ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bfd051640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bf8072d20 0x7f0bf8133ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0be800ebe0 tx=0x7f0be800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 1 -- 10.20.193.78:0/2108832197 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8017070 con 0x7f0bf8072d20 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: dump: 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.213 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 1 -- 10.20.193.78:0/2108832197 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be800ce50 con 0x7f0bf8072d20 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 1 -- 10.20.193.78:0/2108832197 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be80136b0 con 0x7f0bf8072d20 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bff2dc640 5 monclient: authenticate success, global_id 4188 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bff2dc640 10 monclient: _renew_subs 2026-02-17T05:46:08.214 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bff2dc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf81216a0 con 0x7f0bf8072d20 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be77fe640 4 set_mon_vals no callback set 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0be5ffb640 10 monclient: dump: 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:epoch 1 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr: 2026-02-17T05:46:08.215 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bff2dc640 10 monclient: _renew_subs 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bff2dc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.195+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf81358f0 con 0x7f0bf8072d20 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0bff2dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0bff2dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0bff2dc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf8050a10 con 0x7f0bf8072d20 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0be5ffb640 1 -- 10.20.193.78:0/2108832197 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0be8024060 con 0x7f0bf8072d20 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0be5ffb640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0bc803deb0 0x7f0bc8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0be5ffb640 1 -- 10.20.193.78:0/2108832197 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3989+0+0 (secure 0 0 0) 0x7f0be8052680 con 0x7f0bf8072d20 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0bfc850640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0bc803deb0 0x7f0bc8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.216 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.196+0000 7f0bfc850640 10 monclient: get_auth_request con 0x7f0bc803deb0 auth_method 0 2026-02-17T05:46:08.217 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.199+0000 7f0be5ffb640 1 -- 10.20.193.78:0/2108832197 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0bf8050a10 con 0x7f0bf8072d20 2026-02-17T05:46:08.217 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.199+0000 7f0be5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:08.217 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.199+0000 7f0be5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:08.217 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.199+0000 7f0bfc850640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0bc803deb0 0x7f0bc8040370 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bec036000 tx=0x7f0bec039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:08.286 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.284+0000 7f0bff2dc640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:08.286 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.284+0000 7f0bff2dc640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:08.286 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.284+0000 7f0bff2dc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:08.286 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.284+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bf809d500 con 0x7f0bf8072d20 2026-02-17T05:46:08.286 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.284+0000 7f0be5ffb640 1 -- 10.20.193.78:0/2108832197 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+10497 (secure 0 0 0) 0x7f0be80523f0 con 0x7f0bf8072d20 2026-02-17T05:46:08.287 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.284+0000 7f0be5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:08.287 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.284+0000 7f0be5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:08.287 INFO:teuthology.misc.health.trial078.stdout: 2026-02-17T05:46:08.287 INFO:teuthology.misc.health.trial078.stdout:{"epoch":16,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T05:46:06.166179+0000","last_up_change":"2026-02-17T05:46:02.153660+0000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:46:03.931310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6800","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6801","nonce":3807268031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6802","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6803","nonce":3807268031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6806","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6807","nonce":3807268031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6804","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6805","nonce":3807268031}]},"public_addr":"10.20.193.78:6801/3807268031","cluster_addr":"10.20.193.78:6803/3807268031","heartbeat_back_addr":"10.20.193.78:6807/3807268031","heartbeat_front_addr":"10.20.193.78:6805/3807268031","state":["exists","up"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6808","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6809","nonce":1094981820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6810","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6811","nonce":1094981820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6814","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6815","nonce":1094981820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6812","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6813","nonce":1094981820}]},"public_addr":"10.20.193.78:6809/1094981820","cluster_addr":"10.20.193.78:6811/1094981820","heartbeat_back_addr":"10.20.193.78:6815/1094981820","heartbeat_front_addr":"10.20.193.78:6813/1094981820","state":["exists","up"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6816","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6817","nonce":707008561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6818","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6819","nonce":707008561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6822","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6823","nonce":707008561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6820","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6821","nonce":707008561}]},"public_addr":"10.20.193.78:6817/707008561","cluster_addr":"10.20.193.78:6819/707008561","heartbeat_back_addr":"10.20.193.78:6823/707008561","heartbeat_front_addr":"10.20.193.78:6821/707008561","state":["exists","up"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6810","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6812","nonce":2268846355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6814","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6816","nonce":2268846355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6822","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6824","nonce":2268846355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6818","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6820","nonce":2268846355}]},"public_addr":"10.20.193.115:6812/2268846355","cluster_addr":"10.20.193.115:6816/2268846355","heartbeat_back_addr":"10.20.193.115:6824/2268846355","heartbeat_front_addr":"10.20.193.115:6820/2268846355","state":["exists","up"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6802","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6803","nonce":2725813951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6804","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6805","nonce":2725813951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6808","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6809","nonce":2725813951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6806","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6807","nonce":2725813951}]},"public_addr":"10.20.193.115:6803/2725813951","cluster_addr":"10.20.193.115:6805/2725813951","heartbeat_back_addr":"10.20.193.115:6809/2725813951","heartbeat_front_addr":"10.20.193.115:6807/2725813951","state":["exists","up"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6811","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6813","nonce":2171295268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6815","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6817","nonce":2171295268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6823","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6825","nonce":2171295268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6819","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6821","nonce":2171295268}]},"public_addr":"10.20.193.115:6813/2171295268","cluster_addr":"10.20.193.115:6817/2171295268","heartbeat_back_addr":"10.20.193.115:6825/2171295268","heartbeat_front_addr":"10.20.193.115:6821/2171295268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.280056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.641140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.705253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:01.037254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.075730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.553045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:46:08.288 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0bc803deb0 msgr2=0x7f0bc8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0bc803deb0 0x7f0bc8040370 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bec036000 tx=0x7f0bec039040 comp rx=0 tx=0).stop 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 10 monclient: shutdown 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bf8072d20 msgr2=0x7f0bf8133ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bf8072d20 0x7f0bf8133ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0be800ebe0 tx=0x7f0be800c520 comp rx=0 tx=0).stop 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 shutdown_connections 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bf8073540 0x7f0bf811d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0bc803deb0 0x7f0bc8040370 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bf8073100 0x7f0bf8134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.289 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 --2- 10.20.193.78:0/2108832197 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bf8072d20 0x7f0bf8133ce0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.290 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 >> 10.20.193.78:0/2108832197 conn(0x7f0bf806c810 msgr2=0x7f0bf8071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:08.290 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 shutdown_connections 2026-02-17T05:46:08.290 INFO:teuthology.misc.health.trial078.stderr:2026-02-17T05:46:08.285+0000 7f0bff2dc640 1 -- 10.20.193.78:0/2108832197 wait complete. 2026-02-17T05:46:08.293 DEBUG:teuthology.misc:6 of 6 OSDs are up 2026-02-17T05:46:08.294 INFO:tasks.ceph:Creating RBD pool 2026-02-17T05:46:08.294 DEBUG:teuthology.orchestra.run.trial078:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-17T05:46:08.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.357+0000 7f9ec05a7640 10 monclient: get_monmap_and_config 2026-02-17T05:46:08.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.357+0000 7f9ec05a7640 10 monclient: build_initial_monmap 2026-02-17T05:46:08.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.357+0000 7f9ec05a7640 10 monclient: monmap: 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:08.358849+0000 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:08.358849+0000 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:08.363 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 1 Processor -- start 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 1 -- start start 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient: init 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f9eb8073100 0x7f9eb80734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient(hunting): picked mon.noname-a con 0x7f9eb8073100 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:08.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8074470 0x7f9eb8073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient(hunting): picked mon.noname-b con 0x7f9eb8074470 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9eb80a1740 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb80a1fd0 con 0x7f9eb80a1740 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb8073f60 con 0x7f9eb8073100 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb80a22b0 con 0x7f9eb8074470 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ec05a7640 10 monclient(hunting): authenticate will time out at 613.912479s 2026-02-17T05:46:08.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f9eb8073100 0x7f9eb80734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f9eb8073100 0x7f9eb80734e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52512/0 (socket says 10.20.193.78:52512) 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 1 -- 10.20.193.78:0/4049740355 learned_addr learned my addr 10.20.193.78:0/4049740355 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient(hunting): get_auth_request con 0x7f9eb8073100 auth_method 0 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebdb1b640 1 --2- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8074470 0x7f9eb8073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebdb1b640 10 monclient(hunting): get_auth_request con 0x7f9eb8074470 auth_method 0 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebdb1b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebdb1b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebdb1b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebdb1b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebdb1b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebdb1b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient(hunting): handle_auth_done global_id 4191 payload 274 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 1 -- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8074470 msgr2=0x7f9eb8073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:08.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 1 --2- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8074470 0x7f9eb8073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 1 -- 10.20.193.78:0/4049740355 >> v1:10.20.193.115:6789/0 conn(0x7f9eb80a1740 legacy=0x7f9eb8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient: _finish_hunting 0 2026-02-17T05:46:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 1 monclient: found mon.noname-a 2026-02-17T05:46:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.358+0000 7f9ebe31c640 1 -- 10.20.193.78:0/4049740355 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb809d0a0 con 0x7f9eb8073100 2026-02-17T05:46:08.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebe31c640 1 -- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8073100 msgr2=0x7f9eb80734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:08.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebe31c640 1 --2- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8073100 0x7f9eb80734e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009970 tx=0x7f9ea802ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1f0017b203c67e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:08.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 1 -- 10.20.193.78:0/4049740355 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea803c070 con 0x7f9eb8073100 2026-02-17T05:46:08.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:08.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:08.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: dump: 2026-02-17T05:46:08.376 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:08.376 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: _finish_auth 0 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: _check_auth_tickets 2026-02-17T05:46:08.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 1 -- 10.20.193.78:0/4049740355 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea802fcd0 con 0x7f9eb8073100 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 5 monclient: authenticate success, global_id 4191 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 1 -- 10.20.193.78:0/4049740355 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8037660 con 0x7f9eb8073100 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ebd31a640 10 monclient: dump: 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.378 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 4 set_mon_vals no callback set 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 10 monclient: shutdown 2026-02-17T05:46:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 1 -- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8073100 msgr2=0x7f9eb80734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8073100 0x7f9eb80734e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009970 tx=0x7f9ea802ee90 comp rx=0 tx=0).stop 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 1 -- 10.20.193.78:0/4049740355 shutdown_connections 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8074470 0x7f9eb8073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/4049740355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8073100 0x7f9eb80734e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 1 -- 10.20.193.78:0/4049740355 >> 10.20.193.78:0/4049740355 conn(0x7f9eb80a3300 msgr2=0x7f9eb80a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 1 -- 10.20.193.78:0/4049740355 shutdown_connections 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.361+0000 7f9ec05a7640 1 -- 10.20.193.78:0/4049740355 wait complete. 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient: build_initial_monmap 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient: monmap: 2026-02-17T05:46:08.380 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 1 Processor -- start 2026-02-17T05:46:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 1 -- start start 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient: init 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8073100 0x7f9eb81cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient(hunting): picked mon.noname-c con 0x7f9eb8073100 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9eb80a1740 0x7f9eb81cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient(hunting): picked mon.noname-b con 0x7f9eb80a1740 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8121830 0x7f9eb8121cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient(hunting): picked mon.noname-a con 0x7f9eb8121830 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8135500 con 0x7f9eb8121830 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8135680 con 0x7f9eb80a1740 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb8135800 con 0x7f9eb8073100 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ec05a7640 10 monclient(hunting): authenticate will time out at 613.916798s 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8073100 0x7f9eb81cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebeb1d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8121830 0x7f9eb8121cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8073100 0x7f9eb81cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47630/0 (socket says 10.20.193.78:47630) 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 1 -- 10.20.193.78:0/3605065362 learned_addr learned my addr 10.20.193.78:0/3605065362 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 10 monclient(hunting): get_auth_request con 0x7f9eb8073100 auth_method 0 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebeb1d640 10 monclient(hunting): get_auth_request con 0x7f9eb8121830 auth_method 0 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebeb1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebeb1d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebeb1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebeb1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebeb1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebeb1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.362+0000 7f9ebe31c640 10 monclient(hunting): handle_auth_done global_id 4172 payload 824 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 1 -- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9eb80a1740 msgr2=0x7f9eb81cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9eb80a1740 0x7f9eb81cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 1 -- 10.20.193.78:0/3605065362 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8121830 msgr2=0x7f9eb8121cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8121830 0x7f9eb8121cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 10 monclient: _finish_hunting 0 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 1 monclient: found mon.noname-c 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.363+0000 7f9ebe31c640 1 -- 10.20.193.78:0/3605065362 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb8123fb0 con 0x7f9eb8073100 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ebe31c640 1 -- 10.20.193.78:0/3605065362 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8073100 msgr2=0x7f9eb81cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ebe31c640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8073100 0x7f9eb81cf450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009970 tx=0x7f9ea8004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 1 -- 10.20.193.78:0/3605065362 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea803c040 con 0x7f9eb8073100 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: dump: 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.386 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: _finish_auth 0 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ec05a7640 5 monclient: authenticate success, global_id 4172 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 1 -- 10.20.193.78:0/3605065362 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea8047070 con 0x7f9eb8073100 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ec05a7640 10 monclient: _renew_subs 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ec05a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb8125460 con 0x7f9eb8073100 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 1 -- 10.20.193.78:0/3605065362 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8042890 con 0x7f9eb8073100 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ebcb19640 4 set_mon_vals no callback set 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9e9effd640 10 monclient: dump: 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:08.388 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ec05a7640 10 monclient: _renew_subs 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ec05a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.365+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb80a5040 con 0x7f9eb8073100 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9e9effd640 1 -- 10.20.193.78:0/3605065362 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9ea8042a30 con 0x7f9eb8073100 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9ec05a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9ec05a7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9ec05a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb80a2ea0 con 0x7f9eb8073100 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9e9effd640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9e8803deb0 0x7f9e88040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9e9effd640 1 -- 10.20.193.78:0/3605065362 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 3989+0+0 (secure 0 0 0) 0x7f9ea8077710 con 0x7f9eb8073100 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9ebdb1b640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9e8803deb0 0x7f9e88040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.366+0000 7f9ebdb1b640 10 monclient: get_auth_request con 0x7f9e8803deb0 auth_method 0 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.369+0000 7f9e9effd640 1 -- 10.20.193.78:0/3605065362 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9ea8035340 con 0x7f9eb8073100 2026-02-17T05:46:08.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.369+0000 7f9e9effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:08.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.369+0000 7f9e9effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:08.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.369+0000 7f9ebdb1b640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9e8803deb0 0x7f9e88040370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9eac006b90 tx=0x7f9eac006b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.452+0000 7f9ec05a7640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-17T05:46:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.452+0000 7f9ec05a7640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-17T05:46:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.452+0000 7f9ec05a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:08.452+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f9eb8134dc0 con 0x7f9eb8073100 2026-02-17T05:46:09.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.174+0000 7f9e9effd640 1 -- 10.20.193.78:0/3605065362 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) ==== 109+0+0 (secure 0 0 0) 0x7f9ea8042e00 con 0x7f9eb8073100 2026-02-17T05:46:09.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.174+0000 7f9e9effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-17T05:46:09.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.174+0000 7f9e9effd640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2026-02-17T05:46:09.176 INFO:teuthology.orchestra.run.trial078.stderr:pool 'rbd' created 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9e8803deb0 msgr2=0x7f9e88040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9e8803deb0 0x7f9e88040370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9eac006b90 tx=0x7f9eac006b20 comp rx=0 tx=0).stop 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 10 monclient: shutdown 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8073100 msgr2=0x7f9eb81cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8073100 0x7f9eb81cf450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009970 tx=0x7f9ea8004830 comp rx=0 tx=0).stop 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 shutdown_connections 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9eb8121830 0x7f9eb8121cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9e8803deb0 0x7f9e88040370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9eb80a1740 0x7f9eb81cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 --2- 10.20.193.78:0/3605065362 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9eb8073100 0x7f9eb81cf450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 >> 10.20.193.78:0/3605065362 conn(0x7f9eb80a3300 msgr2=0x7f9eb809e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 shutdown_connections 2026-02-17T05:46:09.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.177+0000 7f9ec05a7640 1 -- 10.20.193.78:0/3605065362 wait complete. 2026-02-17T05:46:09.186 DEBUG:teuthology.orchestra.run.trial078:> rbd --cluster ceph pool init rbd 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.214+0000 7fad9f4f70c0 10 monclient: get_monmap_and_config 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.214+0000 7fad9f4f70c0 10 monclient: build_initial_monmap 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.214+0000 7fad9f4f70c0 10 monclient: monmap: 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:09.215456+0000 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:09.215456+0000 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:09.215 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.214+0000 7fad9f4f70c0 1 Processor -- start 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.214+0000 7fad9f4f70c0 1 -- start start 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.214+0000 7fad9f4f70c0 10 monclient: init 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.214+0000 7fad9f4f70c0 10 monclient: _reopen_session rank -1 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 10 monclient: _add_conns ranks=[3,1,4,2,0] 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 1 --2- >> v2:10.20.193.78:3300/0 conn(0x55cbe0819530 0x55cbe0819910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 10 monclient(hunting): picked mon.noname-a con 0x55cbe0819530 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 10 monclient(hunting): picked mon.noname-c-legacy con 0x55cbe0819e50 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 1 --2- >> v2:10.20.193.115:3300/0 conn(0x55cbe08ca8c0 0x55cbe08d5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 10 monclient(hunting): picked mon.noname-c con 0x55cbe08ca8c0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:09.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55cbe0683d20 con 0x55cbe0819e50 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x55cbe06b0720 con 0x55cbe0819530 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x55cbe0704e80 con 0x55cbe08ca8c0 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 10 monclient(hunting): _renew_subs 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f4f70c0 10 monclient(hunting): authenticate will time out at 614.769061s 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x55cbe0819530 0x55cbe0819910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x55cbe0819530 0x55cbe0819910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52526/0 (socket says 10.20.193.78:52526) 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 1 -- 10.20.193.78:0/674935234 learned_addr learned my addr 10.20.193.78:0/674935234 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient(hunting): get_auth_request con 0x55cbe0819530 auth_method 0 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f246640 1 --2- 10.20.193.78:0/674935234 >> v2:10.20.193.115:3300/0 conn(0x55cbe08ca8c0 0x55cbe08d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f246640 10 monclient(hunting): get_auth_request con 0x55cbe08ca8c0 auth_method 0 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f246640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f246640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9f246640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient(hunting): handle_auth_done global_id 4197 payload 274 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 1 -- 10.20.193.78:0/674935234 >> v2:10.20.193.115:3300/0 conn(0x55cbe08ca8c0 msgr2=0x55cbe08d5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 1 --2- 10.20.193.78:0/674935234 >> v2:10.20.193.115:3300/0 conn(0x55cbe08ca8c0 0x55cbe08d5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 1 -- 10.20.193.78:0/674935234 >> v1:10.20.193.115:6789/0 conn(0x55cbe0819e50 legacy=0x55cbe08ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient: _finish_hunting 0 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 1 monclient: found mon.noname-a 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:09.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.215+0000 7fad9dfbd640 1 -- 10.20.193.78:0/674935234 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe082daf0 con 0x55cbe0819530 2026-02-17T05:46:09.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9dfbd640 1 -- 10.20.193.78:0/674935234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819530 msgr2=0x55cbe0819910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:09.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9dfbd640 1 --2- 10.20.193.78:0/674935234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819530 0x55cbe0819910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x55cbe082ee00 tx=0x7fad940584e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fb801ee3455abc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 1 -- 10.20.193.78:0/674935234 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9406d070 con 0x55cbe0819530 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: dump: 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: _finish_auth 0 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 1 -- 10.20.193.78:0/674935234 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad94060bd0 con 0x55cbe0819530 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 1 -- 10.20.193.78:0/674935234 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94068440 con 0x55cbe0819530 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9f4f70c0 5 monclient: authenticate success, global_id 4197 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9f4f70c0 10 monclient: get_monmap_and_config success 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9f4f70c0 4 set_mon_vals no callback set 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9cfbb640 10 monclient: dump: 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:09.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9f4f70c0 10 monclient: shutdown 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9f4f70c0 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/674935234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819530 msgr2=0x55cbe0819910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.218+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/674935234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819530 0x55cbe0819910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x55cbe082ee00 tx=0x7fad940584e0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/674935234 shutdown_connections 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/674935234 >> v2:10.20.193.115:3300/0 conn(0x55cbe08ca8c0 0x55cbe08d5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/674935234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819530 0x55cbe0819910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/674935234 >> 10.20.193.78:0/674935234 conn(0x55cbe081a680 msgr2=0x55cbe081aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/674935234 shutdown_connections 2026-02-17T05:46:09.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/674935234 wait complete. 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient: build_initial_monmap 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient: monmap: 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 Processor -- start 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 -- start start 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient: init 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient: _reopen_session rank -1 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x55cbe0819530 0x55cbe08b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient(hunting): picked mon.noname-c con 0x55cbe0819530 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819e50 0x55cbe08b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient(hunting): picked mon.noname-a con 0x55cbe0819e50 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x55cbe08ca8c0 0x55cbe08b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient(hunting): picked mon.noname-b con 0x55cbe08ca8c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x55cbe069fe50 con 0x55cbe0819e50 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x55cbe0700db0 con 0x55cbe08ca8c0 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x55cbe08b67b0 con 0x55cbe0819530 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient(hunting): _renew_subs 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.219+0000 7fad9f4f70c0 10 monclient(hunting): authenticate will time out at 614.773970s 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819e50 0x55cbe08b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:09.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9dfbd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x55cbe0819530 0x55cbe08b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819e50 0x55cbe08b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52530/0 (socket says 10.20.193.78:52530) 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 -- 10.20.193.78:0/3560770327 learned_addr learned my addr 10.20.193.78:0/3560770327 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient(hunting): get_auth_request con 0x55cbe0819e50 auth_method 0 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9dfbd640 10 monclient(hunting): get_auth_request con 0x55cbe0819530 auth_method 0 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9dfbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9dfbd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9dfbd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9dfbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9dfbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9dfbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9f246640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x55cbe08ca8c0 0x55cbe08b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9f246640 10 monclient(hunting): get_auth_request con 0x55cbe08ca8c0 auth_method 0 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9f246640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9f246640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9f246640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient(hunting): handle_auth_done global_id 4200 payload 824 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 -- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x55cbe0819530 msgr2=0x55cbe08b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x55cbe0819530 0x55cbe08b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 -- 10.20.193.78:0/3560770327 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x55cbe08ca8c0 msgr2=0x55cbe08b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x55cbe08ca8c0 0x55cbe08b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient: _finish_hunting 0 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 monclient: found mon.noname-a 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:09.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.220+0000 7fad9d7bc640 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe08b5020 con 0x55cbe0819e50 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9d7bc640 1 -- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819e50 msgr2=0x55cbe08b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9d7bc640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819e50 0x55cbe08b3580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad8800bf40 tx=0x7fad88009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88013070 con 0x55cbe0819e50 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: dump: 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8800f850 con 0x55cbe0819e50 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9f4f70c0 5 monclient: authenticate success, global_id 4200 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9f4f70c0 10 monclient: _renew_subs 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9f4f70c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cbe08b64d0 con 0x55cbe0819e50 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88009e60 con 0x55cbe0819e50 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:09.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8dffb640 10 monclient: dump: 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad8ffff640 4 set_mon_vals no callback set 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9f4f70c0 10 monclient: _renew_subs 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9f4f70c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.222+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe08338f0 con 0x55cbe0819e50 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fad8800fcc0 con 0x55cbe0819e50 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad8dffb640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad7c03c6a0 0x7fad7c03eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4331+0+0 (secure 0 0 0) 0x7fad880511d0 con 0x55cbe0819e50 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad9f4f70c0 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad9f4f70c0 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad9f4f70c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad9dfbd640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad7c03c6a0 0x7fad7c03eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x55cbe08cdff0 con 0x55cbe0819e50 2026-02-17T05:46:09.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.223+0000 7fad9dfbd640 10 monclient: get_auth_request con 0x7fad7c03c6a0 auth_method 0 2026-02-17T05:46:09.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:09.226+0000 7fad9dfbd640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad7c03c6a0 0x7fad7c03eb60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x55cbe082e700 tx=0x7fad94064000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:10.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.176+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v18) ==== 141+0+0 (secure 0 0 0) 0x7fad8803ce20 con 0x55cbe0819e50 2026-02-17T05:46:10.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.176+0000 7fad8dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-17T05:46:10.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.176+0000 7fad8dffb640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2026-02-17T05:46:10.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.176+0000 7fad9f4f70c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:10.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.176+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad94060bd0 con 0x55cbe0819e50 2026-02-17T05:46:10.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fad8800d4d0 con 0x55cbe0819e50 2026-02-17T05:46:10.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad8dffb640 10 monclient: handle_get_version_reply finishing 1 version 18 2026-02-17T05:46:10.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad8ffff640 10 monclient: _renew_subs 2026-02-17T05:46:10.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad8ffff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:10.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad8ffff640 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=18}) -- 0x7fad78002ae0 con 0x55cbe0819e50 2026-02-17T05:46:10.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 8 ==== osd_map(18..18 src has 1..18) ==== 674+0+0 (secure 0 0 0) 0x7fad8801fd90 con 0x55cbe0819e50 2026-02-17T05:46:10.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x55cbe08ddc10 0x55cbe08e6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:10.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) -- 0x55cbe08e68a0 con 0x55cbe08ddc10 2026-02-17T05:46:10.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad9f246640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x55cbe08ddc10 0x55cbe08e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:10.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.177+0000 7fad9f246640 10 monclient: get_auth_request con 0x55cbe08ddc10 auth_method 0 2026-02-17T05:46:10.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.181+0000 7fad9f246640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x55cbe08ddc10 0x55cbe08e6360 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:10.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.183+0000 7fad9f246640 1 -- 10.20.193.78:0/3560770327 <== osd.1 v2:10.20.193.78:6808/1094981820 1 ==== osd_op_reply(1 rbd_trash [create] v18'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7fad9000c7e0 con 0x55cbe08ddc10 2026-02-17T05:46:10.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.184+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x55cbe0939990 0x55cbe093be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:10.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.184+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) -- 0x55cbe093c380 con 0x55cbe0939990 2026-02-17T05:46:10.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.184+0000 7fad9dfbd640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x55cbe0939990 0x55cbe093be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:10.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.184+0000 7fad9dfbd640 10 monclient: get_auth_request con 0x55cbe0939990 auth_method 0 2026-02-17T05:46:10.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.188+0000 7fad9dfbd640 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x55cbe0939990 0x55cbe093be20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:10.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.189+0000 7fad9dfbd640 1 -- 10.20.193.78:0/3560770327 <== osd.0 v2:10.20.193.78:6800/3807268031 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7fad9407b040 con 0x55cbe0939990 2026-02-17T05:46:10.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.189+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) -- 0x55cbe0938420 con 0x55cbe0939990 2026-02-17T05:46:10.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.189+0000 7fad9dfbd640 1 -- 10.20.193.78:0/3560770327 <== osd.0 v2:10.20.193.78:6800/3807268031 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7fad9407b040 con 0x55cbe0939990 2026-02-17T05:46:10.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.189+0000 7fad8f7fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:10.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:10.189+0000 7fad8f7fe640 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7fad70001f00 con 0x55cbe0819e50 2026-02-17T05:46:11.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.179+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v19) ==== 55+0+0 (secure 0 0 0) 0x7fad88020d70 con 0x55cbe0819e50 2026-02-17T05:46:11.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.179+0000 7fad8dffb640 10 monclient: _renew_subs 2026-02-17T05:46:11.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.179+0000 7fad8dffb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:11.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.179+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=19}) -- 0x7fad7c042900 con 0x55cbe0819e50 2026-02-17T05:46:11.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.179+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 10 ==== osd_map(19..19 src has 1..19) ==== 626+0+0 (secure 0 0 0) 0x7fad8801f4e0 con 0x55cbe0819e50 2026-02-17T05:46:11.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.179+0000 7fad8ffff640 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19) -- 0x7fad78005790 con 0x55cbe0939990 2026-02-17T05:46:11.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.181+0000 7fad9dfbd640 1 -- 10.20.193.78:0/3560770327 <== osd.0 v2:10.20.193.78:6800/3807268031 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v19'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fad9407b040 con 0x55cbe0939990 2026-02-17T05:46:11.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.181+0000 7fad8f7fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:11.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:11.181+0000 7fad8f7fe640 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v19) -- 0x7fad70003350 con 0x55cbe0819e50 2026-02-17T05:46:12.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.181+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 11 ==== pool_op_reply(tid 6 (0) Success v20) ==== 43+0+0 (secure 0 0 0) 0x7fad8801f6e0 con 0x55cbe0819e50 2026-02-17T05:46:12.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.181+0000 7fad8dffb640 10 monclient: _renew_subs 2026-02-17T05:46:12.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.181+0000 7fad8dffb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.181+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=20}) -- 0x7fad7c0429b0 con 0x55cbe0819e50 2026-02-17T05:46:12.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.181+0000 7fad8dffb640 1 -- 10.20.193.78:0/3560770327 <== mon.0 v2:10.20.193.78:3300/0 12 ==== osd_map(20..20 src has 1..20) ==== 654+0+0 (secure 0 0 0) 0x7fad8800e8d0 con 0x55cbe0819e50 2026-02-17T05:46:12.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.181+0000 7fad8ffff640 1 -- 10.20.193.78:0/3560770327 --> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20) -- 0x7fad78005b30 con 0x55cbe0939990 2026-02-17T05:46:12.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9dfbd640 1 -- 10.20.193.78:0/3560770327 <== osd.0 v2:10.20.193.78:6800/3807268031 4 ==== osd_op_reply(7 rbd_info [write 0~19] v20'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad9407b040 con 0x55cbe0939990 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x55cbe0939990 msgr2=0x55cbe093be20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x55cbe0939990 0x55cbe093be20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x55cbe08ddc10 msgr2=0x55cbe08e6360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x55cbe08ddc10 0x55cbe08e6360 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad7c03c6a0 msgr2=0x7fad7c03eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad7c03c6a0 0x7fad7c03eb60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x55cbe082e700 tx=0x7fad94064000 comp rx=0 tx=0).stop 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 10 monclient: shutdown 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819e50 msgr2=0x55cbe08b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 --2- 10.20.193.78:0/3560770327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x55cbe0819e50 0x55cbe08b3580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad8800bf40 tx=0x7fad88009530 comp rx=0 tx=0).stop 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f246640 1 -- 10.20.193.78:0/3560770327 reap_dead start 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f246640 1 -- 10.20.193.78:0/3560770327 reap_dead start 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 shutdown_connections 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 >> 10.20.193.78:0/3560770327 conn(0x55cbe081a680 msgr2=0x55cbe083b600 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 shutdown_connections 2026-02-17T05:46:12.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.183+0000 7fad9f4f70c0 1 -- 10.20.193.78:0/3560770327 wait complete. 2026-02-17T05:46:12.187 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-17T05:46:12.187 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-17T05:46:12.187 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-17T05:46:12.188 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-17T05:46:12.188 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-17T05:46:12.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.293+0000 7f024f561640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.293+0000 7f024f561640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.293+0000 7f024f561640 10 monclient: monmap: 2026-02-17T05:46:12.295 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.295174+0000 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.295174+0000 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.296 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 1 Processor -- start 2026-02-17T05:46:12.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 1 -- start start 2026-02-17T05:46:12.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient: init 2026-02-17T05:46:12.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2026-02-17T05:46:12.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f024809ea90 0x7f02480a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient(hunting): picked mon.noname-c con 0x7f024809ea90 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:12.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f024809ee70 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f024813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient(hunting): picked mon.noname-b con 0x7f024813d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02480a0a20 con 0x7f024809ee70 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f02480a0d00 con 0x7f024809ea90 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f02480a0460 con 0x7f024813d640 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024f561640 10 monclient(hunting): authenticate will time out at 617.848767s 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f024813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f024813fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47640/0 (socket says 10.20.193.78:47640) 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 1 -- 10.20.193.78:0/1016506106 learned_addr learned my addr 10.20.193.78:0/1016506106 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 10 monclient(hunting): get_auth_request con 0x7f024813d640 auth_method 0 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024d2d6640 1 --2- 10.20.193.78:0/1016506106 >> v2:10.20.193.115:3300/0 conn(0x7f024809ea90 0x7f02480a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024d2d6640 10 monclient(hunting): get_auth_request con 0x7f024809ea90 auth_method 0 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024d2d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024d2d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024d2d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.294+0000 7f024dad7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 10 monclient(hunting): handle_auth_done global_id 4178 payload 274 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 1 -- 10.20.193.78:0/1016506106 >> v2:10.20.193.115:3300/0 conn(0x7f024809ea90 msgr2=0x7f02480a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 1 --2- 10.20.193.78:0/1016506106 >> v2:10.20.193.115:3300/0 conn(0x7f024809ea90 0x7f02480a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 1 -- 10.20.193.78:0/1016506106 >> v1:10.20.193.115:6789/0 conn(0x7f024809ee70 legacy=0x7f024809f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 1 monclient: found mon.noname-b 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.295+0000 7f024dad7640 1 -- 10.20.193.78:0/1016506106 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02480a0740 con 0x7f024813d640 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f024dad7640 1 -- 10.20.193.78:0/1016506106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 msgr2=0x7f024813fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f024dad7640 1 --2- 10.20.193.78:0/1016506106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f024813fa30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0244009e50 tx=0x7f024402f260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72340cfda5b3372a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 1 -- 10.20.193.78:0/1016506106 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02440373f0 con 0x7f024813d640 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: dump: 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.310 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f024f561640 5 monclient: authenticate success, global_id 4178 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 1 -- 10.20.193.78:0/1016506106 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0244037590 con 0x7f024813d640 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f024f561640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 1 -- 10.20.193.78:0/1016506106 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244037960 con 0x7f024813d640 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f024f561640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f024f561640 4 set_mon_vals no callback set 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.297+0000 7f0237fff640 10 monclient: dump: 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.312 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient: shutdown 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- 10.20.193.78:0/1016506106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 msgr2=0x7f024813fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 --2- 10.20.193.78:0/1016506106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f024813fa30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0244009e50 tx=0x7f024402f260 comp rx=0 tx=0).stop 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- 10.20.193.78:0/1016506106 shutdown_connections 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 --2- 10.20.193.78:0/1016506106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f024813fa30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 --2- 10.20.193.78:0/1016506106 >> v2:10.20.193.115:3300/0 conn(0x7f024809ea90 0x7f02480a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- 10.20.193.78:0/1016506106 >> 10.20.193.78:0/1016506106 conn(0x7f02480a7520 msgr2=0x7f02480a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- 10.20.193.78:0/1016506106 shutdown_connections 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- 10.20.193.78:0/1016506106 wait complete. 2026-02-17T05:46:12.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient: monmap: 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 Processor -- start 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- start start 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient: init 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f024809ea90 0x7f02481cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient(hunting): picked mon.noname-a con 0x7f024809ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f024809ee70 0x7f02480736b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient(hunting): picked mon.noname-b con 0x7f024809ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f0248073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient(hunting): picked mon.noname-c con 0x7f024813d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02480a6820 con 0x7f024809ea90 2026-02-17T05:46:12.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f02480a66a0 con 0x7f024809ee70 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f024813ff70 con 0x7f024813d640 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024f561640 10 monclient(hunting): authenticate will time out at 617.852932s 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.298+0000 7f024d2d6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f024809ea90 0x7f02481cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f024809ea90 0x7f02481cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52548/0 (socket says 10.20.193.78:52548) 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 1 -- 10.20.193.78:0/4207611937 learned_addr learned my addr 10.20.193.78:0/4207611937 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 10 monclient(hunting): get_auth_request con 0x7f024809ea90 auth_method 0 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f0248073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient(hunting): get_auth_request con 0x7f024813d640 auth_method 0 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024cad5640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f024809ee70 0x7f02480736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024cad5640 10 monclient(hunting): get_auth_request con 0x7f024809ee70 auth_method 0 2026-02-17T05:46:12.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024cad5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024cad5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024cad5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient(hunting): handle_auth_done global_id 4181 payload 824 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 1 -- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f024809ee70 msgr2=0x7f02480736b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f024809ee70 0x7f02480736b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 1 -- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f024809ea90 msgr2=0x7f02481cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f024809ea90 0x7f02481cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 1 monclient: found mon.noname-c 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024dad7640 1 -- 10.20.193.78:0/4207611937 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02480a9a70 con 0x7f024813d640 2026-02-17T05:46:12.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.299+0000 7f024d2d6640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f024809ea90 0x7f02481cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024dad7640 1 -- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 msgr2=0x7f0248073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024dad7640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f0248073bf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f02480a0e80 tx=0x7f0244004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 1 -- 10.20.193.78:0/4207611937 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244047020 con 0x7f024813d640 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: dump: 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.319 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 1 -- 10.20.193.78:0/4207611937 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0244037e50 con 0x7f024813d640 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 1 -- 10.20.193.78:0/4207611937 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02440373f0 con 0x7f024813d640 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024f561640 5 monclient: authenticate success, global_id 4181 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024f561640 10 monclient: _renew_subs 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024f561640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f02377fe640 4 set_mon_vals no callback set 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02480a6070 con 0x7f024813d640 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f0235ffb640 10 monclient: dump: 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024f561640 10 monclient: _renew_subs 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024f561640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.301+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02480708d0 con 0x7f024813d640 2026-02-17T05:46:12.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f024f561640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f024f561640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f024f561640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02480a13d0 con 0x7f024813d640 2026-02-17T05:46:12.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f0235ffb640 1 -- 10.20.193.78:0/4207611937 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0244043470 con 0x7f024813d640 2026-02-17T05:46:12.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f0235ffb640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f021803df00 0x7f02180403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f0235ffb640 1 -- 10.20.193.78:0/4207611937 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f0244078770 con 0x7f024813d640 2026-02-17T05:46:12.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f024d2d6640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f021803df00 0x7f02180403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.302+0000 7f024d2d6640 10 monclient: get_auth_request con 0x7f021803df00 auth_method 0 2026-02-17T05:46:12.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.305+0000 7f0235ffb640 1 -- 10.20.193.78:0/4207611937 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0244051030 con 0x7f024813d640 2026-02-17T05:46:12.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.305+0000 7f0235ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.305+0000 7f0235ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:12.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.305+0000 7f024d2d6640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f021803df00 0x7f02180403c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f024813d130 tx=0x7f0238008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.410+0000 7f024f561640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2026-02-17T05:46:12.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.410+0000 7f024f561640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-17T05:46:12.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.410+0000 7f024f561640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.410+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f02480710d0 con 0x7f024813d640 2026-02-17T05:46:12.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.411+0000 7f0235ffb640 1 -- 10.20.193.78:0/4207611937 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+96883 (secure 0 0 0) 0x7f02480a9a70 con 0x7f024813d640 2026-02-17T05:46:12.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.411+0000 7f0235ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-17T05:46:12.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.411+0000 7f0235ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:12.414 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:12.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f021803df00 msgr2=0x7f02180403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f021803df00 0x7f02180403c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f024813d130 tx=0x7f0238008040 comp rx=0 tx=0).stop 2026-02-17T05:46:12.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 10 monclient: shutdown 2026-02-17T05:46:12.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 msgr2=0x7f0248073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f0248073bf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f02480a0e80 tx=0x7f0244004b40 comp rx=0 tx=0).stop 2026-02-17T05:46:12.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 shutdown_connections 2026-02-17T05:46:12.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f024813d640 0x7f0248073bf0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f021803df00 0x7f02180403c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f024809ee70 0x7f02480736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 --2- 10.20.193.78:0/4207611937 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f024809ea90 0x7f02481cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 >> 10.20.193.78:0/4207611937 conn(0x7f02480a7520 msgr2=0x7f0248136cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 shutdown_connections 2026-02-17T05:46:12.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.412+0000 7f024f561640 1 -- 10.20.193.78:0/4207611937 wait complete. 2026-02-17T05:46:12.421 INFO:teuthology.orchestra.run.trial078.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6800","nonce":2827515952},{"type":"v1","addr":"10.20.193.115:6801","nonce":2827515952}]},"active_addr":"10.20.193.115:6801/2827515952","active_change":"2026-02-17T05:45:53.430830+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.115:0","nonce":4217099978}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.115:0","nonce":1122571765}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.115:0","nonce":2563794074}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.115:0","nonce":2771094107}]}]} 2026-02-17T05:46:12.425 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-17T05:46:12.426 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-17T05:46:12.426 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T05:46:12.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.492+0000 7fb1052da640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.492+0000 7fb1052da640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.492+0000 7fb1052da640 10 monclient: monmap: 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.493855+0000 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.493855+0000 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.494 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.495 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.495 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.495 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.495 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 1 Processor -- start 2026-02-17T05:46:12.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 1 -- start start 2026-02-17T05:46:12.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient: init 2026-02-17T05:46:12.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb10006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10009c4a0 0x7fb10013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient(hunting): picked mon.noname-b con 0x7fb10009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb10006d380 0x7fb10013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient(hunting): picked mon.noname-c con 0x7fb10006d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10006daf0 con 0x7fb10006c890 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb1000a0880 con 0x7fb10006d380 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb10009cc80 con 0x7fb10009c4a0 2026-02-17T05:46:12.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb1052da640 10 monclient(hunting): authenticate will time out at 618.047420s 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0feffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb10006c890 0x7fb10006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41102/0 (socket says 10.20.193.78:41102) 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0feffd640 1 -- 10.20.193.78:0/2342255865 learned_addr learned my addr 10.20.193.78:0/2342255865 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 1 --2- 10.20.193.78:0/2342255865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10009c4a0 0x7fb10013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient(hunting): get_auth_request con 0x7fb10009c4a0 auth_method 0 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0ff7fe640 1 --2- 10.20.193.78:0/2342255865 >> v2:10.20.193.115:3300/0 conn(0x7fb10006d380 0x7fb10013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0ff7fe640 10 monclient(hunting): get_auth_request con 0x7fb10006d380 auth_method 0 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_done global_id 4184 payload 274 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/2342255865 >> v2:10.20.193.115:3300/0 conn(0x7fb10006d380 msgr2=0x7fb10013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 1 --2- 10.20.193.78:0/2342255865 >> v2:10.20.193.115:3300/0 conn(0x7fb10006d380 0x7fb10013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/2342255865 >> v1:10.20.193.78:6789/0 conn(0x7fb10006c890 legacy=0x7fb10006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 1 monclient: found mon.noname-b 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.493+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/2342255865 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10009cf60 con 0x7fb10009c4a0 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/2342255865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10009c4a0 msgr2=0x7fb10013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fe7fc640 1 --2- 10.20.193.78:0/2342255865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10009c4a0 0x7fb10013b3a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80098a0 tx=0x7fb0e802ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b060c7628220431 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 1 -- 10.20.193.78:0/2342255865 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e803c070 con 0x7fb10009c4a0 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: dump: 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.507 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 1 -- 10.20.193.78:0/2342255865 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e802fc00 con 0x7fb10009c4a0 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 5 monclient: authenticate success, global_id 4184 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 1 -- 10.20.193.78:0/2342255865 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8037440 con 0x7fb10009c4a0 2026-02-17T05:46:12.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 4 set_mon_vals no callback set 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb0fdffb640 10 monclient: dump: 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.509 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 10 monclient: shutdown 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 1 -- 10.20.193.78:0/2342255865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10009c4a0 msgr2=0x7fb10013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 1 --2- 10.20.193.78:0/2342255865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10009c4a0 0x7fb10013b3a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80098a0 tx=0x7fb0e802ed80 comp rx=0 tx=0).stop 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 1 -- 10.20.193.78:0/2342255865 shutdown_connections 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 1 --2- 10.20.193.78:0/2342255865 >> v2:10.20.193.115:3300/0 conn(0x7fb10006d380 0x7fb10013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 1 --2- 10.20.193.78:0/2342255865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10009c4a0 0x7fb10013b3a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 1 -- 10.20.193.78:0/2342255865 >> 10.20.193.78:0/2342255865 conn(0x7fb1000a6620 msgr2=0x7fb1000a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 1 -- 10.20.193.78:0/2342255865 shutdown_connections 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.496+0000 7fb1052da640 1 -- 10.20.193.78:0/2342255865 wait complete. 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient: monmap: 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.511 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 1 Processor -- start 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 1 -- start start 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient: init 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10006c890 0x7fb1001cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient(hunting): picked mon.noname-c con 0x7fb10006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb10006d380 0x7fb1001363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient(hunting): picked mon.noname-a con 0x7fb10006d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb10009c4a0 0x7fb1001cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient(hunting): picked mon.noname-b con 0x7fb10009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb10009bfa0 con 0x7fb10006d380 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb10009be20 con 0x7fb10009c4a0 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb10013de30 con 0x7fb10006c890 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb1052da640 10 monclient(hunting): authenticate will time out at 618.051608s 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0feffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10006c890 0x7fb1001cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb10006d380 0x7fb1001363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb10006d380 0x7fb1001363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52556/0 (socket says 10.20.193.78:52556) 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/3257715992 learned_addr learned my addr 10.20.193.78:0/3257715992 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0feffd640 10 monclient(hunting): get_auth_request con 0x7fb10006c890 auth_method 0 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0feffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0feffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient(hunting): get_auth_request con 0x7fb10006d380 auth_method 0 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0ff7fe640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb10009c4a0 0x7fb1001cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0ff7fe640 10 monclient(hunting): get_auth_request con 0x7fb10009c4a0 auth_method 0 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient(hunting): handle_auth_done global_id 4206 payload 824 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/3257715992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10006c890 msgr2=0x7fb1001cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10006c890 0x7fb1001cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb10009c4a0 msgr2=0x7fb1001cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb10009c4a0 0x7fb1001cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 monclient: found mon.noname-a 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.497+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/3257715992 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1001cfc40 con 0x7fb10006d380 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0fe7fc640 1 -- 10.20.193.78:0/3257715992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb10006d380 msgr2=0x7fb1001363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0fe7fc640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb10006d380 0x7fb1001363d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb0e803a040 tx=0x7fb0e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 1 -- 10.20.193.78:0/3257715992 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8045020 con 0x7fb10006d380 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: dump: 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.516 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 1 -- 10.20.193.78:0/3257715992 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e803d040 con 0x7fb10006d380 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 1 -- 10.20.193.78:0/3257715992 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e803c070 con 0x7fb10006d380 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0fd7fa640 4 set_mon_vals no callback set 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 10 monclient: dump: 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.518 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 5 monclient: authenticate success, global_id 4206 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 10 monclient: _renew_subs 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb100122ac0 con 0x7fb10006d380 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 10 monclient: _renew_subs 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10013d9d0 con 0x7fb10006d380 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10006f730 con 0x7fb10006d380 2026-02-17T05:46:12.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.500+0000 7fb0df7fe640 1 -- 10.20.193.78:0/3257715992 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb0e8041ca0 con 0x7fb10006d380 2026-02-17T05:46:12.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.501+0000 7fb0df7fe640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb0cc03dee0 0x7fb0cc0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.501+0000 7fb0df7fe640 1 -- 10.20.193.78:0/3257715992 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7fb0e8077940 con 0x7fb10006d380 2026-02-17T05:46:12.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.501+0000 7fb0feffd640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb0cc03dee0 0x7fb0cc0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.501+0000 7fb0feffd640 10 monclient: get_auth_request con 0x7fb0cc03dee0 auth_method 0 2026-02-17T05:46:12.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.503+0000 7fb0df7fe640 1 -- 10.20.193.78:0/3257715992 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb0e804f030 con 0x7fb10006d380 2026-02-17T05:46:12.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.503+0000 7fb0df7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.503+0000 7fb0df7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:12.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.503+0000 7fb0feffd640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb0cc03dee0 0x7fb0cc0403a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4007d90 tx=0x7fb0f4007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.585+0000 7fb1052da640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:12.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.585+0000 7fb1052da640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:12.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.585+0000 7fb1052da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.585+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb10013d7b0 con 0x7fb10006d380 2026-02-17T05:46:12.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb0df7fe640 1 -- 10.20.193.78:0/3257715992 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+12403 (secure 0 0 0) 0x7fb10013d7b0 con 0x7fb10006d380 2026-02-17T05:46:12.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb0df7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:12.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb0df7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:12.588 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:12.588 INFO:teuthology.orchestra.run.trial078.stdout:{"epoch":20,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T05:46:12.179931+0000","last_up_change":"2026-02-17T05:46:02.153660+0000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:46:03.931310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T05:46:08.458376+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6800","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6801","nonce":3807268031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6802","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6803","nonce":3807268031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6806","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6807","nonce":3807268031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6804","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6805","nonce":3807268031}]},"public_addr":"10.20.193.78:6801/3807268031","cluster_addr":"10.20.193.78:6803/3807268031","heartbeat_back_addr":"10.20.193.78:6807/3807268031","heartbeat_front_addr":"10.20.193.78:6805/3807268031","state":["exists","up"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6808","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6809","nonce":1094981820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6810","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6811","nonce":1094981820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6814","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6815","nonce":1094981820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6812","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6813","nonce":1094981820}]},"public_addr":"10.20.193.78:6809/1094981820","cluster_addr":"10.20.193.78:6811/1094981820","heartbeat_back_addr":"10.20.193.78:6815/1094981820","heartbeat_front_addr":"10.20.193.78:6813/1094981820","state":["exists","up"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6816","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6817","nonce":707008561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6818","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6819","nonce":707008561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6822","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6823","nonce":707008561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6820","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6821","nonce":707008561}]},"public_addr":"10.20.193.78:6817/707008561","cluster_addr":"10.20.193.78:6819/707008561","heartbeat_back_addr":"10.20.193.78:6823/707008561","heartbeat_front_addr":"10.20.193.78:6821/707008561","state":["exists","up"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6810","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6812","nonce":2268846355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6814","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6816","nonce":2268846355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6822","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6824","nonce":2268846355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6818","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6820","nonce":2268846355}]},"public_addr":"10.20.193.115:6812/2268846355","cluster_addr":"10.20.193.115:6816/2268846355","heartbeat_back_addr":"10.20.193.115:6824/2268846355","heartbeat_front_addr":"10.20.193.115:6820/2268846355","state":["exists","up"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6802","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6803","nonce":2725813951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6804","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6805","nonce":2725813951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6808","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6809","nonce":2725813951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6806","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6807","nonce":2725813951}]},"public_addr":"10.20.193.115:6803/2725813951","cluster_addr":"10.20.193.115:6805/2725813951","heartbeat_back_addr":"10.20.193.115:6809/2725813951","heartbeat_front_addr":"10.20.193.115:6807/2725813951","state":["exists","up"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6811","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6813","nonce":2171295268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6815","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6817","nonce":2171295268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6823","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6825","nonce":2171295268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6819","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6821","nonce":2171295268}]},"public_addr":"10.20.193.115:6813/2171295268","cluster_addr":"10.20.193.115:6817/2171295268","heartbeat_back_addr":"10.20.193.115:6825/2171295268","heartbeat_front_addr":"10.20.193.115:6821/2171295268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.280056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.641140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.705253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:01.037254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.075730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.553045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:46:12.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb0cc03dee0 msgr2=0x7fb0cc0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb1052da640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb0cc03dee0 0x7fb0cc0403a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4007d90 tx=0x7fb0f4007ce0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb1052da640 10 monclient: shutdown 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb1052da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb10006d380 msgr2=0x7fb1001363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.586+0000 7fb1052da640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb10006d380 0x7fb1001363d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb0e803a040 tx=0x7fb0e80047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.587+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 shutdown_connections 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.587+0000 7fb1052da640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb0cc03dee0 0x7fb0cc0403a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.587+0000 7fb1052da640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb10009c4a0 0x7fb1001cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.587+0000 7fb1052da640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb10006d380 0x7fb1001363d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.587+0000 7fb1052da640 1 --2- 10.20.193.78:0/3257715992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb10006c890 0x7fb1001cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.587+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 >> 10.20.193.78:0/3257715992 conn(0x7fb1000a6620 msgr2=0x7fb1000a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.587+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 shutdown_connections 2026-02-17T05:46:12.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.587+0000 7fb1052da640 1 -- 10.20.193.78:0/3257715992 wait complete. 2026-02-17T05:46:12.596 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-17T05:46:12.597 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T05:46:12.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient: monmap: 2026-02-17T05:46:12.667 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.667 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.667 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.664350+0000 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.664350+0000 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 1 Processor -- start 2026-02-17T05:46:12.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 1 -- start start 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient: init 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f902006c890 0x7f902006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient(hunting): picked mon.noname-a con 0x7f902006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902009c4a0 0x7f902006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient(hunting): picked mon.noname-b con 0x7f902009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9020134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f902006daf0 con 0x7f9020134df0 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f90200a0880 con 0x7f902006c890 2026-02-17T05:46:12.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f902006d6f0 con 0x7f902009c4a0 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9026d23640 10 monclient(hunting): authenticate will time out at 618.217865s 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9024a98640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f902006c890 0x7f902006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9017fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902009c4a0 0x7f902006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9024a98640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f902006c890 0x7f902006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52568/0 (socket says 10.20.193.78:52568) 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9025299640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f9020134df0 0x7f9020135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41110/0 (socket says 10.20.193.78:41110) 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9024a98640 1 -- 10.20.193.78:0/1078751383 learned_addr learned my addr 10.20.193.78:0/1078751383 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9017fff640 10 monclient(hunting): get_auth_request con 0x7f902009c4a0 auth_method 0 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9017fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9017fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.663+0000 7f9017fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9024a98640 10 monclient(hunting): get_auth_request con 0x7f902006c890 auth_method 0 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9024a98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9024a98640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9024a98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9024a98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9024a98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9024a98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 10 monclient(hunting): handle_auth_done global_id 4190 payload 274 2026-02-17T05:46:12.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 1 -- 10.20.193.78:0/1078751383 >> v2:10.20.193.78:3300/0 conn(0x7f902006c890 msgr2=0x7f902006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 1 --2- 10.20.193.78:0/1078751383 >> v2:10.20.193.78:3300/0 conn(0x7f902006c890 0x7f902006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 1 -- 10.20.193.78:0/1078751383 >> v1:10.20.193.78:6789/0 conn(0x7f9020134df0 legacy=0x7f9020135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 1 monclient: found mon.noname-b 2026-02-17T05:46:12.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9017fff640 1 -- 10.20.193.78:0/1078751383 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902009cc80 con 0x7f902009c4a0 2026-02-17T05:46:12.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.664+0000 7f9024a98640 1 --2- 10.20.193.78:0/1078751383 >> v2:10.20.193.78:3300/0 conn(0x7f902006c890 0x7f902006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f9017fff640 1 -- 10.20.193.78:0/1078751383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902009c4a0 msgr2=0x7f902006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f9017fff640 1 --2- 10.20.193.78:0/1078751383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902009c4a0 0x7f902006d1b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f902006d870 tx=0x7f900802ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f16332558986179 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 1 -- 10.20.193.78:0/1078751383 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900803c070 con 0x7f902009c4a0 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: dump: 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 1 -- 10.20.193.78:0/1078751383 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f900802fc00 con 0x7f902009c4a0 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f9026d23640 5 monclient: authenticate success, global_id 4190 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f9026d23640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f9026d23640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 1 -- 10.20.193.78:0/1078751383 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008037440 con 0x7f902009c4a0 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f9026d23640 4 set_mon_vals no callback set 2026-02-17T05:46:12.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.666+0000 7f90177fe640 10 monclient: dump: 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient: shutdown 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- 10.20.193.78:0/1078751383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902009c4a0 msgr2=0x7f902006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 --2- 10.20.193.78:0/1078751383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902009c4a0 0x7f902006d1b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f902006d870 tx=0x7f900802ed80 comp rx=0 tx=0).stop 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- 10.20.193.78:0/1078751383 shutdown_connections 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 --2- 10.20.193.78:0/1078751383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902009c4a0 0x7f902006d1b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 --2- 10.20.193.78:0/1078751383 >> v2:10.20.193.78:3300/0 conn(0x7f902006c890 0x7f902006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- 10.20.193.78:0/1078751383 >> 10.20.193.78:0/1078751383 conn(0x7f90200a6620 msgr2=0x7f90200a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- 10.20.193.78:0/1078751383 shutdown_connections 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- 10.20.193.78:0/1078751383 wait complete. 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient: monmap: 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 Processor -- start 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- start start 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient: init 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902006c890 0x7f90201cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient(hunting): picked mon.noname-c con 0x7f902006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f902009c4a0 0x7f902009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient(hunting): picked mon.noname-b con 0x7f902009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9020134df0 0x7f902009e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient(hunting): picked mon.noname-a con 0x7f9020134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90200a01a0 con 0x7f9020134df0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90200a0020 con 0x7f902009c4a0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f902009be20 con 0x7f902006c890 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.667+0000 7f9026d23640 10 monclient(hunting): authenticate will time out at 618.221997s 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902006c890 0x7f90201cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902006c890 0x7f90201cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47672/0 (socket says 10.20.193.78:47672) 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 -- 10.20.193.78:0/114324994 learned_addr learned my addr 10.20.193.78:0/114324994 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient(hunting): get_auth_request con 0x7f902006c890 auth_method 0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9020134df0 0x7f902009e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 10 monclient(hunting): get_auth_request con 0x7f9020134df0 auth_method 0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient(hunting): handle_auth_done global_id 4193 payload 824 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9017fff640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f902009c4a0 0x7f902009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 -- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f902009c4a0 msgr2=0x7f902009db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f902009c4a0 0x7f902009db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 -- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9020134df0 msgr2=0x7f902009e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9020134df0 0x7f902009e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 monclient: found mon.noname-c 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9024a98640 1 -- 10.20.193.78:0/114324994 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902009d810 con 0x7f902006c890 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9025299640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9020134df0 0x7f902009e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9017fff640 10 monclient: get_auth_request con 0x7f902009c4a0 auth_method 0 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.668+0000 7f9017fff640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f902009c4a0 0x7f902009db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9024a98640 1 -- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902006c890 msgr2=0x7f90201cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9024a98640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902006c890 0x7f90201cf7a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f901000ec10 tx=0x7f901000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 1 -- 10.20.193.78:0/114324994 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010017070 con 0x7f902006c890 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: dump: 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 1 -- 10.20.193.78:0/114324994 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f901000cdf0 con 0x7f902006c890 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 5 monclient: authenticate success, global_id 4193 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 10 monclient: _renew_subs 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90201278f0 con 0x7f902006c890 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 1 -- 10.20.193.78:0/114324994 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010013690 con 0x7f902006c890 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9016ffd640 4 set_mon_vals no callback set 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f90157fa640 10 monclient: dump: 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 10 monclient: _renew_subs 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9020127330 con 0x7f902006c890 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.670+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9020074980 con 0x7f902006c890 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.671+0000 7f90157fa640 1 -- 10.20.193.78:0/114324994 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9010024060 con 0x7f902006c890 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.671+0000 7f90157fa640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8ff003df00 0x7f8ff00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.671+0000 7f90157fa640 1 -- 10.20.193.78:0/114324994 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f9010052620 con 0x7f902006c890 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.671+0000 7f9017fff640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8ff003df00 0x7f8ff00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.671+0000 7f9017fff640 10 monclient: get_auth_request con 0x7f8ff003df00 auth_method 0 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.673+0000 7f90157fa640 1 -- 10.20.193.78:0/114324994 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f902009d810 con 0x7f902006c890 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.673+0000 7f90157fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.673+0000 7f90157fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:12.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.673+0000 7f9017fff640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8ff003df00 0x7f8ff00403c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f90200a7490 tx=0x7f9008033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.755+0000 7f9026d23640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:12.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.755+0000 7f9026d23640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:12.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.755+0000 7f9026d23640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.755+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9020135a70 con 0x7f902006c890 2026-02-17T05:46:12.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.756+0000 7f90157fa640 1 -- 10.20.193.78:0/114324994 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+12403 (secure 0 0 0) 0x7f9010013d50 con 0x7f902006c890 2026-02-17T05:46:12.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.756+0000 7f90157fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:12.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.756+0000 7f90157fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:12.758 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:12.758 INFO:teuthology.orchestra.run.trial078.stdout:{"epoch":20,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T05:46:12.179931+0000","last_up_change":"2026-02-17T05:46:02.153660+0000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:46:03.931310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T05:46:08.458376+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6800","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6801","nonce":3807268031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6802","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6803","nonce":3807268031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6806","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6807","nonce":3807268031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6804","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6805","nonce":3807268031}]},"public_addr":"10.20.193.78:6801/3807268031","cluster_addr":"10.20.193.78:6803/3807268031","heartbeat_back_addr":"10.20.193.78:6807/3807268031","heartbeat_front_addr":"10.20.193.78:6805/3807268031","state":["exists","up"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6808","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6809","nonce":1094981820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6810","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6811","nonce":1094981820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6814","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6815","nonce":1094981820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6812","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6813","nonce":1094981820}]},"public_addr":"10.20.193.78:6809/1094981820","cluster_addr":"10.20.193.78:6811/1094981820","heartbeat_back_addr":"10.20.193.78:6815/1094981820","heartbeat_front_addr":"10.20.193.78:6813/1094981820","state":["exists","up"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6816","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6817","nonce":707008561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6818","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6819","nonce":707008561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6822","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6823","nonce":707008561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6820","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6821","nonce":707008561}]},"public_addr":"10.20.193.78:6817/707008561","cluster_addr":"10.20.193.78:6819/707008561","heartbeat_back_addr":"10.20.193.78:6823/707008561","heartbeat_front_addr":"10.20.193.78:6821/707008561","state":["exists","up"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6810","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6812","nonce":2268846355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6814","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6816","nonce":2268846355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6822","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6824","nonce":2268846355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6818","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6820","nonce":2268846355}]},"public_addr":"10.20.193.115:6812/2268846355","cluster_addr":"10.20.193.115:6816/2268846355","heartbeat_back_addr":"10.20.193.115:6824/2268846355","heartbeat_front_addr":"10.20.193.115:6820/2268846355","state":["exists","up"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6802","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6803","nonce":2725813951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6804","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6805","nonce":2725813951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6808","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6809","nonce":2725813951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6806","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6807","nonce":2725813951}]},"public_addr":"10.20.193.115:6803/2725813951","cluster_addr":"10.20.193.115:6805/2725813951","heartbeat_back_addr":"10.20.193.115:6809/2725813951","heartbeat_front_addr":"10.20.193.115:6807/2725813951","state":["exists","up"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6811","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6813","nonce":2171295268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6815","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6817","nonce":2171295268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6823","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6825","nonce":2171295268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6819","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6821","nonce":2171295268}]},"public_addr":"10.20.193.115:6813/2171295268","cluster_addr":"10.20.193.115:6817/2171295268","heartbeat_back_addr":"10.20.193.115:6825/2171295268","heartbeat_front_addr":"10.20.193.115:6821/2171295268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.280056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.641140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.705253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:01.037254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.075730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.553045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:46:12.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8ff003df00 msgr2=0x7f8ff00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8ff003df00 0x7f8ff00403c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f90200a7490 tx=0x7f9008033000 comp rx=0 tx=0).stop 2026-02-17T05:46:12.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 10 monclient: shutdown 2026-02-17T05:46:12.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902006c890 msgr2=0x7f90201cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902006c890 0x7f90201cf7a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f901000ec10 tx=0x7f901000c520 comp rx=0 tx=0).stop 2026-02-17T05:46:12.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 shutdown_connections 2026-02-17T05:46:12.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9020134df0 0x7f902009e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8ff003df00 0x7f8ff00403c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f902009c4a0 0x7f902009db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 --2- 10.20.193.78:0/114324994 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f902006c890 0x7f90201cf7a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 >> 10.20.193.78:0/114324994 conn(0x7f90200a6620 msgr2=0x7f9020071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 shutdown_connections 2026-02-17T05:46:12.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.757+0000 7f9026d23640 1 -- 10.20.193.78:0/114324994 wait complete. 2026-02-17T05:46:12.768 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-17T05:46:12.768 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-17T05:46:12.769 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-17T05:46:12.769 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-17T05:46:12.770 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-17T05:46:12.770 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-17T05:46:12.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.839+0000 7fae02bc1640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.839+0000 7fae02bc1640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.839+0000 7fae02bc1640 10 monclient: monmap: 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.841164+0000 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.841164+0000 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.842 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.843 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.843 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fdb2cd3e640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fdb2cd3e640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fdb2cd3e640 10 monclient: monmap: 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.841398+0000 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.841398+0000 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.844 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.845 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.845 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.845 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f6e045ee640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f6e045ee640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f6e045ee640 10 monclient: monmap: 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.841455+0000 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.841455+0000 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.846 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.847 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f04f8fa9640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f04f8fa9640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f04f8fa9640 10 monclient: monmap: 2026-02-17T05:46:12.847 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.847 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.841545+0000 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.841545+0000 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.848 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 1 Processor -- start 2026-02-17T05:46:12.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 1 -- start start 2026-02-17T05:46:12.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient: init 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f9d31379640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f9d31379640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f9d31379640 10 monclient: monmap: 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.841730+0000 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.841730+0000 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.850 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.851 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.851 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.851 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.851 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.851 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.851 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f4c0f476640 10 monclient: get_monmap_and_config 2026-02-17T05:46:12.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f4c0f476640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f4c0f476640 10 monclient: monmap: 2026-02-17T05:46:12.852 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.852 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.852 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:12.841758+0000 2026-02-17T05:46:12.852 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:12.841758+0000 2026-02-17T05:46:12.853 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.853 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.853 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:12.853 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:12.853 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:12.853 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:12.853 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:12.853 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f6e045ee640 1 Processor -- start 2026-02-17T05:46:12.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fdb2cd3e640 1 Processor -- start 2026-02-17T05:46:12.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f6e045ee640 1 -- start start 2026-02-17T05:46:12.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f6e045ee640 10 monclient: init 2026-02-17T05:46:12.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2026-02-17T05:46:12.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc09ea90 0x7fadfc0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient(hunting): picked mon.noname-b con 0x7fadfc09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fadfc09ee70 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fadfc13d640 0x7fadfc13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient(hunting): picked mon.noname-c con 0x7fadfc13d640 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadfc0a0a20 con 0x7fadfc09ee70 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fadfc0a0d00 con 0x7fadfc13d640 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fadfc0a0460 con 0x7fadfc09ea90 2026-02-17T05:46:12.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fdb2cd3e640 1 -- start start 2026-02-17T05:46:12.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fdb2cd3e640 10 monclient: init 2026-02-17T05:46:12.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae02bc1640 10 monclient(hunting): authenticate will time out at 618.394782s 2026-02-17T05:46:12.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f04f8fa9640 1 Processor -- start 2026-02-17T05:46:12.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f04f8fa9640 1 -- start start 2026-02-17T05:46:12.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f04f8fa9640 10 monclient: init 2026-02-17T05:46:12.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae00936640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc09ea90 0x7fadfc0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fadf3fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fadfc09ee70 0x7fadfc09f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41118/0 (socket says 10.20.193.78:41118) 2026-02-17T05:46:12.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fadf3fff640 1 -- 10.20.193.78:0/4076201172 learned_addr learned my addr 10.20.193.78:0/4076201172 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:12.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae00936640 10 monclient(hunting): get_auth_request con 0x7fadfc09ea90 auth_method 0 2026-02-17T05:46:12.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae00936640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae00936640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae00936640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f9d31379640 1 Processor -- start 2026-02-17T05:46:12.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f6e045ee640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae00936640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7fae00936640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.840+0000 7f6e045ee640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2026-02-17T05:46:12.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06c890 0x7f6dfc0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 10 monclient(hunting): picked mon.noname-b con 0x7f6dfc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 1 -- start start 2026-02-17T05:46:12.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient: init 2026-02-17T05:46:12.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6dfc06cc70 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:12.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6dfc13a240 0x7f6dfc13c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 10 monclient(hunting): picked mon.noname-a con 0x7f6dfc13a240 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:12.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 1 Processor -- start 2026-02-17T05:46:12.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae01137640 1 --2- 10.20.193.78:0/4076201172 >> v2:10.20.193.115:3300/0 conn(0x7fadfc13d640 0x7fadfc13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dfc074150 con 0x7f6dfc06cc70 2026-02-17T05:46:12.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6dfc09ef70 con 0x7f6dfc13a240 2026-02-17T05:46:12.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc09f250 con 0x7f6dfc06c890 2026-02-17T05:46:12.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e045ee640 10 monclient(hunting): authenticate will time out at 618.395064s 2026-02-17T05:46:12.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae01137640 10 monclient(hunting): get_auth_request con 0x7fadfc13d640 auth_method 0 2026-02-17T05:46:12.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae01137640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae01137640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae01137640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2026-02-17T05:46:12.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809ea90 0x7fdb280a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient(hunting): picked mon.noname-b con 0x7fdb2809ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fdb2809ee70 0x7fdb2809f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient(hunting): picked mon.noname-c con 0x7fdb2809ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:12.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 1 -- start start 2026-02-17T05:46:12.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdb2809d980 0x7fdb2809ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient: init 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient(hunting): picked mon.noname-a con 0x7fdb2809d980 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fdb280a0d00 con 0x7fdb2809d980 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fdb280a0a20 con 0x7fdb2809ee70 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdb280a0460 con 0x7fdb2809ea90 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb2cd3e640 10 monclient(hunting): authenticate will time out at 618.395119s 2026-02-17T05:46:12.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06c890 0x7f6dfc0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06c890 0x7f6dfc0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47698/0 (socket says 10.20.193.78:47698) 2026-02-17T05:46:12.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 1 -- 10.20.193.78:0/709044194 learned_addr learned my addr 10.20.193.78:0/709044194 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient(hunting): get_auth_request con 0x7f6dfc06c890 auth_method 0 2026-02-17T05:46:12.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 1 --2- 10.20.193.78:0/709044194 >> v2:10.20.193.78:3300/0 conn(0x7f6dfc13a240 0x7f6dfc13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 10 monclient(hunting): handle_auth_done global_id 4196 payload 274 2026-02-17T05:46:12.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 1 -- 10.20.193.78:0/4076201172 >> v2:10.20.193.115:3300/0 conn(0x7fadfc13d640 msgr2=0x7fadfc13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 1 --2- 10.20.193.78:0/4076201172 >> v2:10.20.193.115:3300/0 conn(0x7fadfc13d640 0x7fadfc13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 10 monclient(hunting): get_auth_request con 0x7f6dfc13a240 auth_method 0 2026-02-17T05:46:12.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 1 -- 10.20.193.78:0/4076201172 >> v1:10.20.193.78:6789/0 conn(0x7fadfc09ee70 legacy=0x7fadfc09f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2026-02-17T05:46:12.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 1 monclient: found mon.noname-b 2026-02-17T05:46:12.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdb2809d980 0x7fdb2809ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fae00936640 1 -- 10.20.193.78:0/4076201172 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc0a0740 con 0x7fadfc09ea90 2026-02-17T05:46:12.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ea90 0x7f04f40a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient(hunting): picked mon.noname-b con 0x7f04f409ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f04f409ee70 0x7f04f409f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdb2809d980 0x7fdb2809ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52590/0 (socket says 10.20.193.78:52590) 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 -- 10.20.193.78:0/4042744786 learned_addr learned my addr 10.20.193.78:0/4042744786 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient(hunting): get_auth_request con 0x7fdb2809d980 auth_method 0 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26575640 1 --2- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809ea90 0x7fdb280a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient(hunting): picked mon.noname-c con 0x7f04f409ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:12.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26575640 10 monclient(hunting): get_auth_request con 0x7fdb2809ea90 auth_method 0 2026-02-17T05:46:12.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f04f409d980 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:12.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f40a0a20 con 0x7f04f409d980 2026-02-17T05:46:12.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f04f40a0d00 con 0x7f04f409ee70 2026-02-17T05:46:12.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f04f40a0460 con 0x7f04f409ea90 2026-02-17T05:46:12.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f8fa9640 10 monclient(hunting): authenticate will time out at 618.395302s 2026-02-17T05:46:12.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T05:46:12.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient(hunting): picked mon.noname-b con 0x7f9d2c09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb25d74640 1 --2- 10.20.193.78:0/4042744786 >> v2:10.20.193.115:3300/0 conn(0x7fdb2809ee70 0x7fdb2809f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ea90 0x7f04f40a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9d2c09ee70 0x7f9d2c09f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient(hunting): picked mon.noname-c con 0x7f9d2c09ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:12.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f9d2c09d980 0x7f9d2c09ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb25d74640 10 monclient(hunting): get_auth_request con 0x7fdb2809ee70 auth_method 0 2026-02-17T05:46:12.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb25d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb25d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb25d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient(hunting): picked mon.noname-a con 0x7f9d2c09d980 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:12.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f9d2c0a0d00 con 0x7f9d2c09d980 2026-02-17T05:46:12.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f9d2c0a0a20 con 0x7f9d2c09ee70 2026-02-17T05:46:12.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient(hunting): handle_auth_done global_id 4199 payload 274 2026-02-17T05:46:12.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ea90 0x7f04f40a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47708/0 (socket says 10.20.193.78:47708) 2026-02-17T05:46:12.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 1 -- 10.20.193.78:0/2407506704 learned_addr learned my addr 10.20.193.78:0/2407506704 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient(hunting): get_auth_request con 0x7f04f409ea90 auth_method 0 2026-02-17T05:46:12.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c0a0460 con 0x7f9d2c09ea90 2026-02-17T05:46:12.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 1 -- 10.20.193.78:0/709044194 >> v2:10.20.193.78:3300/0 conn(0x7f6dfc13a240 msgr2=0x7f6dfc13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 1 --2- 10.20.193.78:0/709044194 >> v2:10.20.193.78:3300/0 conn(0x7f6dfc13a240 0x7f6dfc13c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 1 -- 10.20.193.78:0/709044194 >> v1:10.20.193.115:6789/0 conn(0x7f6dfc06cc70 legacy=0x7f6dfc06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d31379640 10 monclient(hunting): authenticate will time out at 618.395341s 2026-02-17T05:46:12.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 1 monclient: found mon.noname-b 2026-02-17T05:46:12.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02363640 1 -- 10.20.193.78:0/709044194 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc09df40 con 0x7f6dfc06c890 2026-02-17T05:46:12.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f6e02b64640 1 --2- 10.20.193.78:0/709044194 >> v2:10.20.193.78:3300/0 conn(0x7f6dfc13a240 0x7f6dfc13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:12.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb25d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb25d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb25d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c0a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47722/0 (socket says 10.20.193.78:47722) 2026-02-17T05:46:12.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient(hunting): handle_auth_done global_id 4218 payload 274 2026-02-17T05:46:12.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 -- 10.20.193.78:0/2558396616 learned_addr learned my addr 10.20.193.78:0/2558396616 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient(hunting): get_auth_request con 0x7f9d2c09ea90 auth_method 0 2026-02-17T05:46:12.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 -- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809ea90 msgr2=0x7fdb280a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 --2- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809ea90 0x7fdb280a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 -- 10.20.193.78:0/4042744786 >> v2:10.20.193.115:3300/0 conn(0x7fdb2809ee70 msgr2=0x7fdb2809f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 --2- 10.20.193.78:0/4042744786 >> v2:10.20.193.115:3300/0 conn(0x7fdb2809ee70 0x7fdb2809f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 monclient: found mon.noname-a 2026-02-17T05:46:12.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7fdb26d76640 1 -- 10.20.193.78:0/4042744786 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb280a0740 con 0x7fdb2809d980 2026-02-17T05:46:12.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2026-02-17T05:46:12.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4c0809ea90 0x7f4c080a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient(hunting): handle_auth_done global_id 4205 payload 274 2026-02-17T05:46:12.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 1 -- 10.20.193.78:0/2407506704 >> v2:10.20.193.115:3300/0 conn(0x7f04f409ee70 msgr2=0x7f04f409f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient(hunting): picked mon.noname-a con 0x7f4c0809ea90 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:12.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/2558396616 >> v2:10.20.193.78:3300/0 conn(0x7f9d2c09d980 0x7f9d2c09ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 1 --2- 10.20.193.78:0/2407506704 >> v2:10.20.193.115:3300/0 conn(0x7f04f409ee70 0x7f04f409f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 1 -- 10.20.193.78:0/2407506704 >> v1:10.20.193.78:6789/0 conn(0x7f04f409d980 legacy=0x7f04f409ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 1 monclient: found mon.noname-b 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f04f2575640 1 -- 10.20.193.78:0/2407506704 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f40a0740 con 0x7f04f409ea90 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809ee70 0x7f4c0809f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient(hunting): picked mon.noname-b con 0x7f4c0809ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2b7fe640 10 monclient(hunting): get_auth_request con 0x7f9d2c09d980 auth_method 0 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4c0809d980 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:12.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c080a0a20 con 0x7f4c0809d980 2026-02-17T05:46:12.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f4c080a0d00 con 0x7f4c0809ea90 2026-02-17T05:46:12.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4c080a0460 con 0x7f4c0809ee70 2026-02-17T05:46:12.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0f476640 10 monclient(hunting): authenticate will time out at 618.395558s 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient(hunting): handle_auth_done global_id 4208 payload 274 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d9ec640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f4c0809d980 0x7f4c0809ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41144/0 (socket says 10.20.193.78:41144) 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d9ec640 1 -- 10.20.193.78:0/400371112 learned_addr learned my addr 10.20.193.78:0/400371112 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4c0809ea90 0x7f4c080a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 -- 10.20.193.78:0/2558396616 >> v2:10.20.193.115:3300/0 conn(0x7f9d2c09ee70 msgr2=0x7f9d2c09f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 --2- 10.20.193.78:0/2558396616 >> v2:10.20.193.115:3300/0 conn(0x7f9d2c09ee70 0x7f9d2c09f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 -- 10.20.193.78:0/2558396616 >> v2:10.20.193.78:3300/0 conn(0x7f9d2c09d980 msgr2=0x7f9d2c09ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient(hunting): get_auth_request con 0x7f4c0809ea90 auth_method 0 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0c9ea640 1 --2- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809ee70 0x7f4c0809f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 --2- 10.20.193.78:0/2558396616 >> v2:10.20.193.78:3300/0 conn(0x7f9d2c09d980 0x7f9d2c09ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 monclient: found mon.noname-b 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0c9ea640 10 monclient(hunting): get_auth_request con 0x7f4c0809ee70 auth_method 0 2026-02-17T05:46:12.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0c9ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f9d2affd640 1 -- 10.20.193.78:0/2558396616 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c0a0740 con 0x7f9d2c09ea90 2026-02-17T05:46:12.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0c9ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0c9ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient(hunting): handle_auth_done global_id 4224 payload 274 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809ee70 msgr2=0x7f4c0809f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 1 --2- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809ee70 0x7f4c0809f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/400371112 >> v1:10.20.193.78:6789/0 conn(0x7f4c0809d980 legacy=0x7f4c0809ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 1 monclient: found mon.noname-a 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.841+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/400371112 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c080a0740 con 0x7f4c0809ea90 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fae00936640 1 -- 10.20.193.78:0/4076201172 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc09ea90 msgr2=0x7fadfc0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fae00936640 1 --2- 10.20.193.78:0/4076201172 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc09ea90 0x7fadfc0a13d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098a0 tx=0x7fadec02ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc4e69f9cef4ce9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 1 -- 10.20.193.78:0/4076201172 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec03c070 con 0x7fadfc09ea90 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: dump: 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 1 -- 10.20.193.78:0/4076201172 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadec02fc00 con 0x7fadfc09ea90 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 1 -- 10.20.193.78:0/4076201172 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec037440 con 0x7fadfc09ea90 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fae02bc1640 5 monclient: authenticate success, global_id 4196 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fae02bc1640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fae02bc1640 4 set_mon_vals no callback set 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7fadf37fe640 10 monclient: dump: 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7f6e02363640 1 -- 10.20.193.78:0/709044194 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06c890 msgr2=0x7f6dfc0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7f6e02363640 1 --2- 10.20.193.78:0/709044194 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06c890 0x7f6dfc0a01f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc06e270 tx=0x7f6dec02ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a110d2ec2af462a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7f04f2575640 1 -- 10.20.193.78:0/2407506704 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ea90 msgr2=0x7f04f40a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.843+0000 7f04f2575640 1 --2- 10.20.193.78:0/2407506704 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ea90 0x7f04f40a13d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009970 tx=0x7f04e802ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b47f57cf599cf6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 1 -- 10.20.193.78:0/2407506704 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e803c070 con 0x7f04f409ea90 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: dump: 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.899 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 5 monclient: authenticate success, global_id 4205 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 1 -- 10.20.193.78:0/2407506704 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e802fdb0 con 0x7f04f409ea90 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 1 -- 10.20.193.78:0/2407506704 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8037660 con 0x7f04f409ea90 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 4 set_mon_vals no callback set 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f1573640 10 monclient: dump: 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 10 monclient: shutdown 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 -- 10.20.193.78:0/4076201172 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc09ea90 msgr2=0x7fadfc0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 --2- 10.20.193.78:0/4076201172 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc09ea90 0x7fadfc0a13d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098a0 tx=0x7fadec02ed80 comp rx=0 tx=0).stop 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 -- 10.20.193.78:0/4076201172 shutdown_connections 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 --2- 10.20.193.78:0/4076201172 >> v2:10.20.193.115:3300/0 conn(0x7fadfc13d640 0x7fadfc13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 --2- 10.20.193.78:0/4076201172 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc09ea90 0x7fadfc0a13d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 -- 10.20.193.78:0/4076201172 >> 10.20.193.78:0/4076201172 conn(0x7fadfc0a7520 msgr2=0x7fadfc0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 -- 10.20.193.78:0/4076201172 shutdown_connections 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 -- 10.20.193.78:0/4076201172 wait complete. 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4c0809ea90 msgr2=0x7f4c080a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0d1eb640 1 --2- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4c0809ea90 0x7f4c080a13d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80098a0 tx=0x7f4bf802ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a561fa38f3f657 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 10 monclient: shutdown 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2407506704 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ea90 msgr2=0x7f04f40a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 1 --2- 10.20.193.78:0/2407506704 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ea90 0x7f04f40a13d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009970 tx=0x7f04e802ee90 comp rx=0 tx=0).stop 2026-02-17T05:46:12.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 1 -- 10.20.193.78:0/400371112 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf803c070 con 0x7f4c0809ea90 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: dump: 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 1 -- 10.20.193.78:0/400371112 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf802fc00 con 0x7f4c0809ea90 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 1 -- 10.20.193.78:0/400371112 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8037440 con 0x7f4c0809ea90 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 5 monclient: authenticate success, global_id 4224 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 4 set_mon_vals no callback set 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4bf7fff640 10 monclient: dump: 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2407506704 shutdown_connections 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 1 --2- 10.20.193.78:0/2407506704 >> v2:10.20.193.115:3300/0 conn(0x7f04f409ee70 0x7f04f409f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 1 --2- 10.20.193.78:0/2407506704 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ea90 0x7f04f40a13d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2407506704 >> 10.20.193.78:0/2407506704 conn(0x7f04f40a7520 msgr2=0x7f04f40a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2407506704 shutdown_connections 2026-02-17T05:46:12.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2407506704 wait complete. 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 10 monclient: monmap: 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 10 monclient: shutdown 2026-02-17T05:46:12.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 1 -- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4c0809ea90 msgr2=0x7f4c080a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 1 --2- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4c0809ea90 0x7f4c080a13d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80098a0 tx=0x7f4bf802ed80 comp rx=0 tx=0).stop 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 1 -- 10.20.193.78:0/400371112 shutdown_connections 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 1 --2- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809ee70 0x7f4c0809f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 1 --2- 10.20.193.78:0/400371112 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4c0809ea90 0x7f4c080a13d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 1 -- 10.20.193.78:0/400371112 >> 10.20.193.78:0/400371112 conn(0x7f4c080a7520 msgr2=0x7f4c080a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 1 -- 10.20.193.78:0/400371112 shutdown_connections 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f4c0f476640 1 -- 10.20.193.78:0/400371112 wait complete. 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 Processor -- start 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7f04f8fa9640 10 monclient: monmap: 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.906 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 1 -- start start 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.844+0000 7fae02bc1640 10 monclient: init 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 1 Processor -- start 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 1 -- start start 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient: init 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient: monmap: 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.908 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadfc09ea90 0x7fadfc073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient(hunting): picked mon.noname-a con 0x7fadfc09ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fadfc09ee70 0x7fadfc13bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient(hunting): picked mon.noname-b con 0x7fadfc09ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc13d640 0x7fadfc13c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient(hunting): picked mon.noname-c con 0x7fadfc13d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc0a6820 con 0x7fadfc09ea90 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc0a66a0 con 0x7fadfc09ee70 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fadfc13ff70 con 0x7fadfc13d640 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae02bc1640 10 monclient(hunting): authenticate will time out at 618.399221s 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc13d640 0x7fadfc13c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc13d640 0x7fadfc13c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47742/0 (socket says 10.20.193.78:47742) 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 -- 10.20.193.78:0/2936039916 learned_addr learned my addr 10.20.193.78:0/2936039916 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient(hunting): get_auth_request con 0x7fadfc13d640 auth_method 0 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f04f409d980 0x7f04f412fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient(hunting): picked mon.noname-b con 0x7f04f409d980 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 1 Processor -- start 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fadf3fff640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fadfc09ee70 0x7fadfc13bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f04f409ea90 0x7f04f4130020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient(hunting): picked mon.noname-a con 0x7f04f409ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ee70 0x7f04f4130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fadf3fff640 10 monclient(hunting): get_auth_request con 0x7fadfc09ee70 auth_method 0 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fadf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fadf3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fadf3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient(hunting): picked mon.noname-c con 0x7f04f409ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04f40a66a0 con 0x7f04f409ea90 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f04f409f7d0 con 0x7f04f409d980 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f04f40a6820 con 0x7f04f409ee70 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f8fa9640 10 monclient(hunting): authenticate will time out at 618.399425s 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient(hunting): handle_auth_done global_id 4214 payload 824 2026-02-17T05:46:12.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 -- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fadfc09ee70 msgr2=0x7fadfc13bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fadfc09ee70 0x7fadfc13bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 -- 10.20.193.78:0/2936039916 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadfc09ea90 msgr2=0x7fadfc073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadfc09ea90 0x7fadfc073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 monclient: found mon.noname-c 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fae01137640 1 -- 10.20.193.78:0/2936039916 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc070b40 con 0x7fadfc13d640 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 1 -- start start 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient: init 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ee70 0x7f04f4130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ee70 0x7f04f4130560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47752/0 (socket says 10.20.193.78:47752) 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 1 -- 10.20.193.78:0/2591087775 learned_addr learned my addr 10.20.193.78:0/2591087775 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 10 monclient(hunting): get_auth_request con 0x7f04f409ee70 auth_method 0 2026-02-17T05:46:12.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2575640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f04f409d980 0x7f04f412fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2575640 10 monclient(hunting): get_auth_request con 0x7f04f409d980 auth_method 0 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809d980 0x7f4c081cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient(hunting): picked mon.noname-c con 0x7f4c0809d980 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4c0809ea90 0x7f4c081cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient(hunting): picked mon.noname-a con 0x7f4c0809ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4c0809ee70 0x7f4c08072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient(hunting): picked mon.noname-b con 0x7f4c0809ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4c080a66a0 con 0x7f4c0809ea90 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0809f7d0 con 0x7f4c0809ee70 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4c080a6820 con 0x7f4c0809d980 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0f476640 10 monclient(hunting): authenticate will time out at 618.399796s 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809d980 0x7f4c081cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809d980 0x7f4c081cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47760/0 (socket says 10.20.193.78:47760) 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/1174054658 learned_addr learned my addr 10.20.193.78:0/1174054658 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient(hunting): get_auth_request con 0x7f4c0809d980 auth_method 0 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fdb26d76640 1 -- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809d980 msgr2=0x7fdb2809ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7fdb26d76640 1 --2- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809d980 0x7fdb2809ddd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb2806ed80 tx=0x7fdb1802f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82824775643bd83b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient(hunting): handle_auth_done global_id 4220 payload 824 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f1d74640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f04f409ea90 0x7f04f4130020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/1174054658 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4c0809ee70 msgr2=0x7f4c08072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4c0809ee70 0x7f4c08072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4c0809ea90 msgr2=0x7f4c081cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4c0809ea90 0x7f4c081cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 monclient: found mon.noname-c 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f1d74640 10 monclient(hunting): get_auth_request con 0x7f04f409ea90 auth_method 0 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f04f1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.845+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/1174054658 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c081304f0 con 0x7f4c0809d980 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 10 monclient(hunting): handle_auth_done global_id 4231 payload 824 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d2affd640 1 -- 10.20.193.78:0/2558396616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 msgr2=0x7f9d2c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d2affd640 1 --2- 10.20.193.78:0/2558396616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c0a13d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c06ed80 tx=0x7f9d1c02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8f078a26340c0f4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 1 -- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ee70 msgr2=0x7f04f4130560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f04f409ee70 0x7f04f4130560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 1 -- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f04f409ea90 msgr2=0x7f04f4130020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f04f409ea90 0x7f04f4130020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 1 monclient: found mon.noname-b 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f04f2575640 1 -- 10.20.193.78:0/2591087775 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f4131c90 con 0x7f04f409d980 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 1 -- 10.20.193.78:0/709044194 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec03c070 con 0x7f6dfc06c890 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: dump: 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 1 -- 10.20.193.78:0/2558396616 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c03c070 con 0x7f9d2c09ea90 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 1 -- 10.20.193.78:0/709044194 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dec02fde0 con 0x7f6dfc06c890 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 1 -- 10.20.193.78:0/709044194 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec037660 con 0x7f6dfc06c890 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 5 monclient: authenticate success, global_id 4199 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 4 set_mon_vals no callback set 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: dump: 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 1 -- 10.20.193.78:0/2558396616 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1c02fca0 con 0x7f9d2c09ea90 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e01361640 10 monclient: dump: 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 5 monclient: authenticate success, global_id 4208 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 1 -- 10.20.193.78:0/2558396616 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c037660 con 0x7f9d2c09ea90 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d29ffb640 10 monclient: dump: 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.919 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 10 monclient: shutdown 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 1 -- 10.20.193.78:0/709044194 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06c890 msgr2=0x7f6dfc0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 1 --2- 10.20.193.78:0/709044194 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06c890 0x7f6dfc0a01f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc06e270 tx=0x7f6dec02ef60 comp rx=0 tx=0).stop 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 4 set_mon_vals no callback set 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 1 -- 10.20.193.78:0/709044194 shutdown_connections 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 1 --2- 10.20.193.78:0/709044194 >> v2:10.20.193.78:3300/0 conn(0x7f6dfc13a240 0x7f6dfc13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 1 --2- 10.20.193.78:0/709044194 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06c890 0x7f6dfc0a01f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 1 -- 10.20.193.78:0/709044194 >> 10.20.193.78:0/709044194 conn(0x7f6dfc0a6620 msgr2=0x7f6dfc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 1 -- 10.20.193.78:0/709044194 shutdown_connections 2026-02-17T05:46:12.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 1 -- 10.20.193.78:0/709044194 wait complete. 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 10 monclient: shutdown 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 -- 10.20.193.78:0/2558396616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 msgr2=0x7f9d2c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 --2- 10.20.193.78:0/2558396616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c0a13d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c06ed80 tx=0x7f9d1c02ee60 comp rx=0 tx=0).stop 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 -- 10.20.193.78:0/2558396616 shutdown_connections 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 --2- 10.20.193.78:0/2558396616 >> v2:10.20.193.78:3300/0 conn(0x7f9d2c09d980 0x7f9d2c09ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 --2- 10.20.193.78:0/2558396616 >> v2:10.20.193.115:3300/0 conn(0x7f9d2c09ee70 0x7f9d2c09f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 --2- 10.20.193.78:0/2558396616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c0a13d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 -- 10.20.193.78:0/2558396616 >> 10.20.193.78:0/2558396616 conn(0x7f9d2c0a7520 msgr2=0x7f9d2c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 -- 10.20.193.78:0/2558396616 shutdown_connections 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f9d31379640 1 -- 10.20.193.78:0/2558396616 wait complete. 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.846+0000 7f6e045ee640 10 monclient: monmap: 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.921 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 1 Processor -- start 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 1 -- start start 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient: init 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient: monmap: 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 1 Processor -- start 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:46:12.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 1 -- start start 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6dfc06c890 0x7f6dfc1cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient(hunting): picked mon.noname-a con 0x7f6dfc06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient: init 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06cc70 0x7f6dfc1355e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient(hunting): picked mon.noname-c con 0x7f6dfc06cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6dfc13a240 0x7f6dfc1cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient(hunting): picked mon.noname-b con 0x7f6dfc13a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc13ccf0 con 0x7f6dfc06c890 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc13cb70 con 0x7f6dfc13a240 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc13ce70 con 0x7f6dfc06cc70 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e045ee640 10 monclient(hunting): authenticate will time out at 618.401548s 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7fae01137640 1 -- 10.20.193.78:0/2936039916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc13d640 msgr2=0x7fadfc13c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7fae01137640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc13d640 0x7fadfc13c0e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fadf8007e30 tx=0x7fadf800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6dfc06c890 0x7f6dfc1cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6dfc06c890 0x7f6dfc1cf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52638/0 (socket says 10.20.193.78:52638) 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 -- 10.20.193.78:0/2366171409 learned_addr learned my addr 10.20.193.78:0/2366171409 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient(hunting): get_auth_request con 0x7f6dfc06c890 auth_method 0 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02b64640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6dfc13a240 0x7f6dfc1cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02b64640 10 monclient(hunting): get_auth_request con 0x7f6dfc13a240 auth_method 0 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02b64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02b64640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02b64640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09d980 0x7f9d2c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient(hunting): picked mon.noname-c con 0x7f9d2c09d980 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ea90 0x7f9d2c1cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient(hunting): picked mon.noname-a con 0x7f9d2c09ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c1cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient(hunting): picked mon.noname-b con 0x7f9d2c09ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c0a66a0 con 0x7f9d2c09ea90 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c09f7d0 con 0x7f9d2c09ee70 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c0a6820 con 0x7f9d2c09d980 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d31379640 10 monclient(hunting): authenticate will time out at 618.401853s 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02b64640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02b64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02b64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient(hunting): handle_auth_done global_id 4230 payload 824 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ea90 0x7f9d2c1cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 -- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06cc70 msgr2=0x7f6dfc1355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6dfc06cc70 0x7f6dfc1355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 -- 10.20.193.78:0/2366171409 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6dfc13a240 msgr2=0x7f6dfc1cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6dfc13a240 0x7f6dfc1cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ea90 0x7f9d2c1cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52654/0 (socket says 10.20.193.78:52654) 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 1 -- 10.20.193.78:0/2717716608 learned_addr learned my addr 10.20.193.78:0/2717716608 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 10 monclient(hunting): get_auth_request con 0x7f9d2c09ea90 auth_method 0 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 monclient: found mon.noname-a 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f6e02363640 1 -- 10.20.193.78:0/2366171409 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc0a3c10 con 0x7f6dfc06c890 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.847+0000 7f9d2a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c1cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2b7fe640 10 monclient(hunting): get_auth_request con 0x7f9d2c09ee70 auth_method 0 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 10 monclient(hunting): handle_auth_done global_id 4233 payload 824 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 1 -- 10.20.193.78:0/4042744786 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1802fe60 con 0x7fdb2809d980 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: dump: 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 1 -- 10.20.193.78:0/4042744786 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb180373f0 con 0x7fdb2809d980 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 1 -- 10.20.193.78:0/4042744786 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180377c0 con 0x7fdb2809d980 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 5 monclient: authenticate success, global_id 4218 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 4 set_mon_vals no callback set 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb25573640 10 monclient: dump: 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.927 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 1 -- 10.20.193.78:0/2717716608 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09d980 msgr2=0x7f9d2c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09d980 0x7f9d2c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 1 -- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c09ee70 msgr2=0x7f9d2c1cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c1cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 1 monclient: found mon.noname-a 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f9d2a7fc640 1 -- 10.20.193.78:0/2717716608 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c131f40 con 0x7f9d2c09ea90 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0d1eb640 1 -- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809d980 msgr2=0x7f4c081cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0d1eb640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809d980 0x7f4c081cf5d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80099d0 tx=0x7f4bf80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8045020 con 0x7f4c0809d980 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:12.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: dump: 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf803d040 con 0x7f4c0809d980 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf803c070 con 0x7f4c0809d980 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 10 monclient: dump: 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf77fe640 4 set_mon_vals no callback set 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 5 monclient: authenticate success, global_id 4220 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 10 monclient: _renew_subs 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c08130bb0 con 0x7f4c0809d980 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 10 monclient: _renew_subs 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c080a96e0 con 0x7f4c0809d980 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 10 monclient: shutdown 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809d980 msgr2=0x7fdb2809ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 --2- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809d980 0x7fdb2809ddd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb2806ed80 tx=0x7fdb1802f230 comp rx=0 tx=0).stop 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f2575640 1 -- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f04f409d980 msgr2=0x7f04f412fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f2575640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f04f409d980 0x7f04f412fae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04e8006fd0 tx=0x7f04e80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/4042744786 shutdown_connections 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 --2- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809d980 0x7fdb2809ddd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 --2- 10.20.193.78:0/4042744786 >> v2:10.20.193.115:3300/0 conn(0x7fdb2809ee70 0x7fdb2809f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 --2- 10.20.193.78:0/4042744786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809ea90 0x7fdb280a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/4042744786 >> 10.20.193.78:0/4042744786 conn(0x7fdb280a7520 msgr2=0x7fdb280a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8046070 con 0x7f04f409d980 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/4042744786 shutdown_connections 2026-02-17T05:46:12.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/4042744786 wait complete. 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: dump: 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e8037e30 con 0x7f04f409d980 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e803c040 con 0x7f04f409d980 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04daffd640 10 monclient: dump: 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f0d72640 4 set_mon_vals no callback set 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f8fa9640 5 monclient: authenticate success, global_id 4231 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f8fa9640 10 monclient: _renew_subs 2026-02-17T05:46:12.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f8fa9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f4131f70 con 0x7f04f409d980 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f8fa9640 10 monclient: _renew_subs 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f8fa9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f4073170 con 0x7f04f409d980 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c080a9190 con 0x7f4c0809d980 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.848+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4bf8041c80 con 0x7f4c0809d980 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient: build_initial_monmap 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient: monmap: 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4bf5ffb640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4bd803df00 0x7f4bd80403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04f8fa9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:46:12.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04f40a13d0 con 0x7f04f409d980 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f4bf8077be0 con 0x7f4c0809d980 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4bf5ffb640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f4bd8043d80 0x7f4bd80461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 --> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bd8046890 con 0x7f4bd8043d80 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4bf807c2a0 con 0x7f4c0809d980 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4bf5ffb640 10 monclient: handle_get_version_reply finishing 1 version 20 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 1 Processor -- start 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 1 -- start start 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient: init 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4c0d9ec640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f4bd8043d80 0x7f4bd80461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4c0c9ea640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4bd803df00 0x7f4bd80403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4c0d9ec640 10 monclient: get_auth_request con 0x7f4bd8043d80 auth_method 0 2026-02-17T05:46:12.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f4c0c9ea640 10 monclient: get_auth_request con 0x7f4bd803df00 auth_method 0 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809d980 0x7fdb28072d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient(hunting): picked mon.noname-c con 0x7fdb2809d980 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb2809ea90 0x7fdb2812fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient(hunting): picked mon.noname-b con 0x7fdb2809ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809ee70 0x7fdb28130020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient(hunting): picked mon.noname-a con 0x7fdb2809ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb280a66a0 con 0x7fdb2809ee70 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2809f7d0 con 0x7fdb2809ea90 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdb280a6820 con 0x7fdb2809d980 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb2cd3e640 10 monclient(hunting): authenticate will time out at 618.403585s 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f04e80608a0 con 0x7f04f409d980 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809ee70 0x7fdb28130020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809ee70 0x7fdb28130020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52668/0 (socket says 10.20.193.78:52668) 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 -- 10.20.193.78:0/2719062433 learned_addr learned my addr 10.20.193.78:0/2719062433 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809d980 0x7fdb28072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient(hunting): get_auth_request con 0x7fdb2809ee70 auth_method 0 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26575640 10 monclient(hunting): get_auth_request con 0x7fdb2809d980 auth_method 0 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:12.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8013790 con 0x7fadfc13d640 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: dump: 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf8019070 con 0x7fadfc13d640 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8013b70 con 0x7fadfc13d640 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf17fa640 10 monclient: dump: 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fadf2ffd640 4 set_mon_vals no callback set 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fae02bc1640 5 monclient: authenticate success, global_id 4214 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fae02bc1640 10 monclient: _renew_subs 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fae02bc1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadfc0a9dc0 con 0x7fadfc13d640 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient(hunting): handle_auth_done global_id 4236 payload 824 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fae02bc1640 10 monclient: _renew_subs 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fae02bc1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 -- 10.20.193.78:0/2719062433 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809d980 msgr2=0x7fdb28072d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fadfc071100 con 0x7fadfc13d640 2026-02-17T05:46:12.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb25d74640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb2809ea90 0x7fdb2812fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb2809d980 0x7fdb28072d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 -- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb2809ea90 msgr2=0x7fdb2812fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb2809ea90 0x7fdb2812fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient: _finish_hunting 0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 monclient: found mon.noname-a 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb26d76640 1 -- 10.20.193.78:0/2719062433 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb280a90a0 con 0x7fdb2809ee70 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb25d74640 10 monclient: get_auth_request con 0x7fdb2809ea90 auth_method 0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7fdb25d74640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb2809ea90 0x7fdb2812fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04daffd640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f04c003deb0 0x7f04c0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f04e8078360 con 0x7f04f409d980 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04daffd640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f04c0043cb0 0x7f04c00460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 --> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04e8050e80 con 0x7f04c0043cb0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f04e8078640 con 0x7f04f409d980 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04daffd640 10 monclient: handle_get_version_reply finishing 1 version 20 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04f1d74640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f04c003deb0 0x7f04c0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04f1d74640 10 monclient: get_auth_request con 0x7f04c003deb0 auth_method 0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.849+0000 7f04f2d76640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f04c0043cb0 0x7f04c00460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f04f2d76640 10 monclient: get_auth_request con 0x7f04c0043cb0 auth_method 0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e02363640 1 -- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6dfc06c890 msgr2=0x7f6dfc1cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e02363640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6dfc06c890 0x7f6dfc1cf230 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6dec002410 tx=0x7f6dec004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fae02bc1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:12.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fadfc06f0f0 con 0x7fadfc13d640 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d2a7fc640 1 -- 10.20.193.78:0/2717716608 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ea90 msgr2=0x7f9d2c1cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d2a7fc640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ea90 0x7f9d2c1cf360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800b780 tx=0x7f9d1800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec046020 con 0x7f6dfc06c890 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: dump: 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dec042420 con 0x7f6dfc06c890 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 5 monclient: authenticate success, global_id 4230 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 10 monclient: _renew_subs 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc139bc0 con 0x7f6dfc06c890 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fadf801e460 con 0x7fadfc13d640 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec03c040 con 0x7f6dfc06c890 2026-02-17T05:46:12.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d180026e0 con 0x7f9d2c09ea90 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: dump: 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e00b60640 4 set_mon_vals no callback set 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 10 monclient: dump: 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d18010070 con 0x7f9d2c09ea90 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18002ac0 con 0x7f9d2c09ea90 2026-02-17T05:46:12.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 10 monclient: _renew_subs 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc122fa0 con 0x7f6dfc06c890 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d297fa640 4 set_mon_vals no callback set 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d237fe640 10 monclient: dump: 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d31379640 5 monclient: authenticate success, global_id 4233 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d31379640 10 monclient: _renew_subs 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d31379640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2c0a9720 con 0x7f9d2c09ea90 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d31379640 10 monclient: _renew_subs 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d31379640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2c0a1c10 con 0x7f9d2c09ea90 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf17fa640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadcc03df00 0x7fadcc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7fadf8052f10 con 0x7fadfc13d640 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf17fa640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7fadcc043d80 0x7fadcc0461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 --> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadf8015ec0 con 0x7fadcc043d80 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fadf80554e0 con 0x7fadfc13d640 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf17fa640 10 monclient: handle_get_version_reply finishing 1 version 20 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf3fff640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7fadcc043d80 0x7fadcc0461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7fadf3fff640 10 monclient: get_auth_request con 0x7fadcc043d80 auth_method 0 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6dfc0a01f0 con 0x7f6dfc06c890 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.850+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6dec03f030 con 0x7f6dfc06c890 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9d1801d070 con 0x7f9d2c09ea90 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d31379640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d2c0a13d0 con 0x7f9d2c09ea90 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f6de2ffd640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dcc03df00 0x7f6dcc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d237fe640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9d0003df30 0x7f9d000403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f6dec0776f0 con 0x7f6dfc06c890 2026-02-17T05:46:12.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f9d18052050 con 0x7f9d2c09ea90 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f6de2ffd640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f6dcc043df0 0x7f6dcc046230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 --> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6dec046e80 con 0x7f6dcc043df0 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6dec07e050 con 0x7f6dfc06c890 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f6de2ffd640 10 monclient: handle_get_version_reply finishing 1 version 20 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d237fe640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f9d00043db0 0x7f9d000461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 --> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d1800de50 con 0x7f9d00043db0 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9d18059050 con 0x7f9d2c09ea90 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d237fe640 10 monclient: handle_get_version_reply finishing 1 version 20 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f6e02b64640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f6dcc043df0 0x7f6dcc046230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f6e02b64640 10 monclient: get_auth_request con 0x7f6dcc043df0 auth_method 0 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f9d00043db0 0x7f9d000461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f9d2b7fe640 10 monclient: get_auth_request con 0x7f9d00043db0 auth_method 0 2026-02-17T05:46:12.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7f4c0c9ea640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4bd803df00 0x7f4bd80403c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc006fd0 tx=0x7f4bfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7fdb26d76640 1 -- 10.20.193.78:0/2719062433 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809ee70 msgr2=0x7fdb28130020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.851+0000 7fdb26d76640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809ee70 0x7fdb28130020 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb18039040 tx=0x7fdb18004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18046020 con 0x7fdb2809ee70 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: dump: 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: _finish_auth 0 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb18037e40 con 0x7fdb2809ee70 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 5 monclient: authenticate success, global_id 4236 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 10 monclient: _renew_subs 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb280a1c10 con 0x7fdb2809ee70 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18041a90 con 0x7fdb2809ee70 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: dump: 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:12.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb24d72640 4 set_mon_vals no callback set 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 10 monclient: _renew_subs 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb28135330 con 0x7fdb2809ee70 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb28131b70 con 0x7fdb2809ee70 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fdb18041c30 con 0x7fdb2809ee70 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7f04f1d74640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f04c003deb0 0x7f04c0040370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04f41d03b0 tx=0x7f04dc009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fae00936640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadcc03df00 0x7fadcc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fae00936640 10 monclient: get_auth_request con 0x7fadcc03df00 auth_method 0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdaf403deb0 0x7fdaf4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7fdb180783c0 con 0x7fdb2809ee70 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fdaf4043cb0 0x7fdaf40460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 --> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb1803be50 con 0x7fdaf4043cb0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdb1807f050 con 0x7fdb2809ee70 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.852+0000 7fdb0effd640 10 monclient: handle_get_version_reply finishing 1 version 20 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.853+0000 7fdb25d74640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fdaf4043cb0 0x7fdaf40460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.853+0000 7fdb25d74640 10 monclient: get_auth_request con 0x7fdaf4043cb0 auth_method 0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.853+0000 7f4c0d9ec640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f4bd8043d80 0x7f4bd80461c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.854+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 <== osd.2 v2:10.20.193.78:6816/707008561 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f4bd8046890 con 0x7f4bd8043d80 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.854+0000 7f04f2d76640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f04c0043cb0 0x7f04c00460f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.854+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 <== osd.1 v2:10.20.193.78:6808/1094981820 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f04e8050e80 con 0x7f04c0043cb0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.855+0000 7fae00936640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadcc03df00 0x7fadcc0403c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fadfc06ed80 tx=0x7fadec031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.855+0000 7fadf3fff640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7fadcc043d80 0x7fadcc0461c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.855+0000 7f6e02b64640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f6dcc043df0 0x7f6dcc046230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.855+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 <== osd.4 v2:10.20.193.115:6802/2725813951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fadf8015ec0 con 0x7fadcc043d80 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.856+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f9d00043db0 0x7f9d000461f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.856+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 <== osd.3 v2:10.20.193.115:6810/2268846355 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f9d1800de50 con 0x7f9d00043db0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.856+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 <== osd.0 v2:10.20.193.78:6800/3807268031 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f6dec046e80 con 0x7f6dcc043df0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.857+0000 7fdb25d74640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fdaf4043cb0 0x7fdaf40460f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.857+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 <== osd.5 v2:10.20.193.115:6811/2171295268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fdb1803be50 con 0x7fdaf4043cb0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.861+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 --> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04f4135260 con 0x7f04c0043cb0 2026-02-17T05:46:12.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04daffd640 1 -- 10.20.193.78:0/2591087775 <== osd.1 v2:10.20.193.78:6808/1094981820 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f04f4135260 con 0x7f04c0043cb0 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 --> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4c080a13d0 con 0x7f4bd8043d80 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f04c0043cb0 msgr2=0x7f04c00460f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f04c0043cb0 0x7f04c00460f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f04c003deb0 msgr2=0x7f04c0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f04c003deb0 0x7f04c0040370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04f41d03b0 tx=0x7f04dc009210 comp rx=0 tx=0).stop 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 10 monclient: shutdown 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f04f409d980 msgr2=0x7f04f412fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f9d2affd640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9d0003df30 0x7f9d000403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f9d2affd640 10 monclient: get_auth_request con 0x7f9d0003df30 auth_method 0 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fdb26575640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdaf403deb0 0x7fdaf4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f6e01b62640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dcc03df00 0x7f6dcc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 --2- 10.20.193.78:0/2591087775 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f04f409d980 0x7f04f412fae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04e8006fd0 tx=0x7f04e80047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f2d76640 1 -- 10.20.193.78:0/2591087775 reap_dead start 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fdb26575640 10 monclient: get_auth_request con 0x7fdaf403deb0 auth_method 0 2026-02-17T05:46:12.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f6e01b62640 10 monclient: get_auth_request con 0x7f6dcc03df00 auth_method 0 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 shutdown_connections 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 >> 10.20.193.78:0/2591087775 conn(0x7f04f40a7520 msgr2=0x7f04f4138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4bf5ffb640 1 -- 10.20.193.78:0/1174054658 <== osd.2 v2:10.20.193.78:6816/707008561 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4c080a13d0 con 0x7f4bd8043d80 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 shutdown_connections 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 --> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fadfc0a13d0 con 0x7fadcc043d80 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f04f8fa9640 1 -- 10.20.193.78:0/2591087775 wait complete. 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f4bd8043d80 msgr2=0x7f4bd80461c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f4bd8043d80 0x7f4bd80461c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4bd803df00 msgr2=0x7f4bd80403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4bd803df00 0x7f4bd80403c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc006fd0 tx=0x7f4bfc008040 comp rx=0 tx=0).stop 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 10 monclient: shutdown 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809d980 msgr2=0x7f4c081cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 --2- 10.20.193.78:0/1174054658 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4c0809d980 0x7f4c081cf5d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80099d0 tx=0x7f4bf80047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0d9ec640 1 -- 10.20.193.78:0/1174054658 reap_dead start 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fadf17fa640 1 -- 10.20.193.78:0/2936039916 <== osd.4 v2:10.20.193.115:6802/2725813951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fadfc0a13d0 con 0x7fadcc043d80 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 shutdown_connections 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 >> 10.20.193.78:0/1174054658 conn(0x7f4c080a7520 msgr2=0x7f4c08138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 shutdown_connections 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7f4c0f476640 1 -- 10.20.193.78:0/1174054658 wait complete. 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7fadcc043d80 msgr2=0x7fadcc0461c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7fadcc043d80 0x7fadcc0461c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadcc03df00 msgr2=0x7fadcc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadcc03df00 0x7fadcc0403c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fadfc06ed80 tx=0x7fadec031000 comp rx=0 tx=0).stop 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 10 monclient: shutdown 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc13d640 msgr2=0x7fadfc13c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.862+0000 7fae02bc1640 1 --2- 10.20.193.78:0/2936039916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadfc13d640 0x7fadfc13c0e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fadf8007e30 tx=0x7fadf800e4b0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7fae01137640 1 -- 10.20.193.78:0/2936039916 reap_dead start 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 shutdown_connections 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 >> 10.20.193.78:0/2936039916 conn(0x7fadfc0a7520 msgr2=0x7fadfc0a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 shutdown_connections 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7fae02bc1640 1 -- 10.20.193.78:0/2936039916 wait complete. 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 --> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6dfc121810 con 0x7f6dcc043df0 2026-02-17T05:46:12.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6de2ffd640 1 -- 10.20.193.78:0/2366171409 <== osd.0 v2:10.20.193.78:6800/3807268031 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6dfc121810 con 0x7f6dcc043df0 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f6dcc043df0 msgr2=0x7f6dcc046230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f6dcc043df0 0x7f6dcc046230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dcc03df00 msgr2=0x7f6dcc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dcc03df00 0x7f6dcc0403c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc1225a0 tx=0x7f6df0007ab0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 10 monclient: shutdown 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6dfc06c890 msgr2=0x7f6dfc1cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 --2- 10.20.193.78:0/2366171409 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6dfc06c890 0x7f6dfc1cf230 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6dec002410 tx=0x7f6dec004830 comp rx=0 tx=0).stop 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e02b64640 1 -- 10.20.193.78:0/2366171409 reap_dead start 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 shutdown_connections 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 >> 10.20.193.78:0/2366171409 conn(0x7f6dfc0a6620 msgr2=0x7f6dfc138230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 shutdown_connections 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.863+0000 7f6e045ee640 1 -- 10.20.193.78:0/2366171409 wait complete. 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.864+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 --> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9d2c1d0030 con 0x7f9d00043db0 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.864+0000 7f9d237fe640 1 -- 10.20.193.78:0/2717716608 <== osd.3 v2:10.20.193.115:6810/2268846355 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9d2c1d0030 con 0x7f9d00043db0 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.864+0000 7fdb26575640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdaf403deb0 0x7fdaf4040370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009bc0 tx=0x7fdb1c009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.864+0000 7f9d2affd640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9d0003df30 0x7f9d000403f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0a0160 tx=0x7f9d1c039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f9d00043db0 msgr2=0x7f9d000461f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f9d00043db0 0x7f9d000461f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9d0003df30 msgr2=0x7f9d000403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9d0003df30 0x7f9d000403f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0a0160 tx=0x7f9d1c039040 comp rx=0 tx=0).stop 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 10 monclient: shutdown 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ea90 msgr2=0x7f9d2c1cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 --2- 10.20.193.78:0/2717716608 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ea90 0x7f9d2c1cf360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800b780 tx=0x7f9d1800bc50 comp rx=0 tx=0).stop 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/2717716608 reap_dead start 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 shutdown_connections 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 >> 10.20.193.78:0/2717716608 conn(0x7f9d2c0a7520 msgr2=0x7f9d2c138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 shutdown_connections 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7f9d31379640 1 -- 10.20.193.78:0/2717716608 wait complete. 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 --> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb28130560 con 0x7fdaf4043cb0 2026-02-17T05:46:12.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb0effd640 1 -- 10.20.193.78:0/2719062433 <== osd.5 v2:10.20.193.115:6811/2171295268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb28130560 con 0x7fdaf4043cb0 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fdaf4043cb0 msgr2=0x7fdaf40460f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fdaf4043cb0 0x7fdaf40460f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdaf403deb0 msgr2=0x7fdaf4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdaf403deb0 0x7fdaf4040370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009bc0 tx=0x7fdb1c009290 comp rx=0 tx=0).stop 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 10 monclient: shutdown 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809ee70 msgr2=0x7fdb28130020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb2cd3e640 1 --2- 10.20.193.78:0/2719062433 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb2809ee70 0x7fdb28130020 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb18039040 tx=0x7fdb18004b40 comp rx=0 tx=0).stop 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.865+0000 7fdb26d76640 1 -- 10.20.193.78:0/2719062433 reap_dead start 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.866+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 shutdown_connections 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.866+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 >> 10.20.193.78:0/2719062433 conn(0x7fdb280a7520 msgr2=0x7fdb280a3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.866+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 shutdown_connections 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:12.866+0000 7fdb2cd3e640 1 -- 10.20.193.78:0/2719062433 wait complete. 2026-02-17T05:46:12.951 INFO:teuthology.orchestra.run.trial078.stdout:47244640262 2026-02-17T05:46:12.951 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-17T05:46:12.952 INFO:teuthology.orchestra.run.trial078.stdout:51539607556 2026-02-17T05:46:12.952 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-17T05:46:12.952 INFO:teuthology.orchestra.run.trial078.stdout:51539607556 2026-02-17T05:46:12.952 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-17T05:46:12.952 INFO:teuthology.orchestra.run.trial078.stdout:51539607557 2026-02-17T05:46:12.952 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-17T05:46:12.953 INFO:teuthology.orchestra.run.trial078.stdout:51539607556 2026-02-17T05:46:12.953 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-17T05:46:12.953 INFO:teuthology.orchestra.run.trial078.stdout:51539607556 2026-02-17T05:46:12.953 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-17T05:46:13.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7ff94aa0c640 10 monclient: get_monmap_and_config 2026-02-17T05:46:13.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7ff94aa0c640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7ff94aa0c640 10 monclient: monmap: 2026-02-17T05:46:13.064 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.064 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.064 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:13.063890+0000 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:13.063890+0000 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:13.065 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7f093465b640 10 monclient: get_monmap_and_config 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7f093465b640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7f093465b640 10 monclient: monmap: 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:13.063889+0000 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:13.063889+0000 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.066 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:13.067 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:13.067 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:13.067 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:13.067 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:13.067 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7f227576a640 10 monclient: get_monmap_and_config 2026-02-17T05:46:13.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7f227576a640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7f227576a640 10 monclient: monmap: 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:13.064006+0000 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:13.064006+0000 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:13.068 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:13.069 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7fc61dd61640 10 monclient: get_monmap_and_config 2026-02-17T05:46:13.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7fc61dd61640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7fc61dd61640 10 monclient: monmap: 2026-02-17T05:46:13.069 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.069 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.069 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:13.064086+0000 2026-02-17T05:46:13.069 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:13.064086+0000 2026-02-17T05:46:13.070 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.070 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.070 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:13.070 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:13.070 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:13.070 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:13.070 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:13.070 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7efeb9561640 10 monclient: get_monmap_and_config 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7efeb9561640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.062+0000 7efeb9561640 10 monclient: monmap: 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:13.064157+0000 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:13.064157+0000 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.071 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:13.072 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:13.072 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:13.072 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:13.072 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:13.072 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 1 Processor -- start 2026-02-17T05:46:13.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 1 -- start start 2026-02-17T05:46:13.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient: init 2026-02-17T05:46:13.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 1 Processor -- start 2026-02-17T05:46:13.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 1 Processor -- start 2026-02-17T05:46:13.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 1 -- start start 2026-02-17T05:46:13.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient: init 2026-02-17T05:46:13.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 1 -- start start 2026-02-17T05:46:13.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient: init 2026-02-17T05:46:13.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7fc61dd61640 1 Processor -- start 2026-02-17T05:46:13.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 1 Processor -- start 2026-02-17T05:46:13.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7fc61dd61640 1 -- start start 2026-02-17T05:46:13.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7fc61dd61640 10 monclient: init 2026-02-17T05:46:13.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 1 -- start start 2026-02-17T05:46:13.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient: init 2026-02-17T05:46:13.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2026-02-17T05:46:13.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2026-02-17T05:46:13.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f092c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:13.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f092c09c4a0 0x7f092c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient(hunting): picked mon.noname-a con 0x7f092c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:13.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff94409ea90 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:13.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff94409ee70 0x7ff94409f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient(hunting): picked mon.noname-c con 0x7ff94409ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:13.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f092c06d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff94413d640 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9440a0a20 con 0x7ff94413d640 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f092c06daf0 con 0x7f092c06c890 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f092c09cc80 con 0x7f092c06d380 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f092c0a0880 con 0x7f092c09c4a0 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9440a0460 con 0x7ff94409ea90 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff9440a0d00 con 0x7ff94409ee70 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff94aa0c640 10 monclient(hunting): authenticate will time out at 618.617632s 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f093465b640 10 monclient(hunting): authenticate will time out at 618.617625s 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f227006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f227006d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f22700a01f0 0x7f227013d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient(hunting): picked mon.noname-c con 0x7f22700a01f0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22700a0600 con 0x7f227006d380 2026-02-17T05:46:13.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f0931bcf640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f092c09c4a0 0x7f092c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2270074150 con 0x7f227006c890 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f227009ef70 con 0x7f22700a01f0 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f09323d0640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f092c06c890 0x7f092c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41152/0 (socket says 10.20.193.78:41152) 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f09323d0640 1 -- 10.20.193.78:0/2495194908 learned_addr learned my addr 10.20.193.78:0/2495194908 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f227576a640 10 monclient(hunting): authenticate will time out at 618.617735s 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff948f82640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff94413d640 0x7ff94413fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41166/0 (socket says 10.20.193.78:41166) 2026-02-17T05:46:13.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff948f82640 1 -- 10.20.193.78:0/2876213763 learned_addr learned my addr 10.20.193.78:0/2876213763 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f0931bcf640 10 monclient(hunting): get_auth_request con 0x7f092c09c4a0 auth_method 0 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f0931bcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f0931bcf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f0931bcf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f226e7fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f227006d380 0x7f227013a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41170/0 (socket says 10.20.193.78:41170) 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f226e7fc640 1 -- 10.20.193.78:0/1198313666 learned_addr learned my addr 10.20.193.78:0/1198313666 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff9437fe640 1 --2- 10.20.193.78:0/2876213763 >> v2:10.20.193.115:3300/0 conn(0x7ff94409ee70 0x7ff94409f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f0931bcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f0931bcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff9437fe640 10 monclient(hunting): get_auth_request con 0x7ff94409ee70 auth_method 0 2026-02-17T05:46:13.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff9437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff9437fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7ff9437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f0931bcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7f226f7fe640 1 --2- 10.20.193.78:0/1198313666 >> v2:10.20.193.115:3300/0 conn(0x7f22700a01f0 0x7f227013d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7fc61dd61640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7fc61dd61640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efeb41385a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb40a0a70 0x7efeb40a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient(hunting): picked mon.noname-b con 0x7efeb40a0a70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efeb413d390 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb40a1630 con 0x7efeb41385a0 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb40a2380 con 0x7efeb413d390 2026-02-17T05:46:13.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7fc61dd61640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618138980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efeb40a20a0 con 0x7efeb40a0a70 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7efeb9561640 10 monclient(hunting): authenticate will time out at 618.617971s 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7fc61dd61640 10 monclient(hunting): picked mon.noname-b con 0x7fc6181385a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.063+0000 7fc61dd61640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc6180a0a70 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc61dd61640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc61813d390 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc61dd61640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6180a1630 con 0x7fc61813d390 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc61dd61640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6180a2380 con 0x7fc6180a0a70 2026-02-17T05:46:13.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc61dd61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc61dd61640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc6180a20a0 con 0x7fc6181385a0 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc61dd61640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc61dd61640 10 monclient(hunting): authenticate will time out at 618.618032s 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7f226f7fe640 10 monclient(hunting): get_auth_request con 0x7f22700a01f0 auth_method 0 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7f226f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7f226f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7f226f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb40a0a70 0x7efeb40a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb40a0a70 0x7efeb40a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47802/0 (socket says 10.20.193.78:47802) 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 1 -- 10.20.193.78:0/932364272 learned_addr learned my addr 10.20.193.78:0/932364272 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient(hunting): get_auth_request con 0x7efeb40a0a70 auth_method 0 2026-02-17T05:46:13.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618138980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618138980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47810/0 (socket says 10.20.193.78:47810) 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 1 -- 10.20.193.78:0/1337137719 learned_addr learned my addr 10.20.193.78:0/1337137719 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient(hunting): get_auth_request con 0x7fc6181385a0 auth_method 0 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient(hunting): handle_auth_done global_id 4226 payload 274 2026-02-17T05:46:13.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 1 -- 10.20.193.78:0/932364272 >> v1:10.20.193.115:6789/0 conn(0x7efeb413d390 legacy=0x7efeb413f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 1 -- 10.20.193.78:0/932364272 >> v1:10.20.193.78:6789/0 conn(0x7efeb41385a0 legacy=0x7efeb4138980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 1 monclient: found mon.noname-b 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7efeb27fc640 1 -- 10.20.193.78:0/932364272 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb40a3860 con 0x7efeb40a0a70 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient(hunting): handle_auth_done global_id 4229 payload 274 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 1 -- 10.20.193.78:0/1337137719 >> v1:10.20.193.115:6789/0 conn(0x7fc6180a0a70 legacy=0x7fc6180a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 1 -- 10.20.193.78:0/1337137719 >> v1:10.20.193.78:6789/0 conn(0x7fc61813d390 legacy=0x7fc61813f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 1 monclient: found mon.noname-b 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fc6177fe640 1 -- 10.20.193.78:0/1337137719 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618138ec0 con 0x7fc6181385a0 2026-02-17T05:46:13.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 10 monclient: get_monmap_and_config 2026-02-17T05:46:13.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 10 monclient: monmap: 2026-02-17T05:46:13.088 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.088 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.088 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:13.065536+0000 2026-02-17T05:46:13.088 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:13.065536+0000 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 1 Processor -- start 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 1 -- start start 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 10 monclient: init 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe9b809ea90 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b809ee70 0x7fe9b809f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 10 monclient(hunting): picked mon.noname-b con 0x7fe9b809ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.064+0000 7fe9c0949640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fe9b813d640 0x7fe9b813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7fe9c0949640 10 monclient(hunting): picked mon.noname-c con 0x7fe9b813d640 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7fe9c0949640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b80a0a20 con 0x7fe9b809ea90 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7fe9c0949640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7fe9c0949640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b80a0d00 con 0x7fe9b813d640 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7fe9c0949640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7fe9c0949640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b80a0460 con 0x7fe9b809ee70 2026-02-17T05:46:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7fe9c0949640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7fe9c0949640 10 monclient(hunting): authenticate will time out at 618.619019s 2026-02-17T05:46:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7f09323d0640 1 -- 10.20.193.78:0/2495194908 >> v1:10.20.193.78:6789/0 conn(0x7f092c06c890 legacy=0x7f092c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7ff948f82640 1 -- 10.20.193.78:0/2876213763 >> v1:10.20.193.78:6789/0 conn(0x7ff94413d640 legacy=0x7ff94413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.065+0000 7f226e7fc640 1 -- 10.20.193.78:0/1198313666 >> v1:10.20.193.78:6789/0 conn(0x7f227006d380 legacy=0x7f227013a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0931bcf640 10 monclient(hunting): handle_auth_done global_id 4239 payload 274 2026-02-17T05:46:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0932bd1640 1 -- 10.20.193.78:0/2495194908 >> v1:10.20.193.115:6789/0 conn(0x7f092c06d380 legacy=0x7f092c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226effd640 1 -- 10.20.193.78:0/1198313666 >> v1:10.20.193.115:6789/0 conn(0x7f227006c890 legacy=0x7f227006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff943fff640 1 -- 10.20.193.78:0/2876213763 >> v1:10.20.193.115:6789/0 conn(0x7ff94409ea90 legacy=0x7ff9440a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0931bcf640 1 -- 10.20.193.78:0/2495194908 >> v1:10.20.193.115:6789/0 conn(0x7f092c06d380 legacy=0x7f092c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0931bcf640 1 -- 10.20.193.78:0/2495194908 >> v1:10.20.193.78:6789/0 conn(0x7f092c06c890 legacy=0x7f092c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0931bcf640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0931bcf640 1 monclient: found mon.noname-a 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0931bcf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0931bcf640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f0931bcf640 1 -- 10.20.193.78:0/2495194908 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c09cf60 con 0x7f092c09c4a0 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9be6be640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fe9b809ea90 0x7fe9b80a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41204/0 (socket says 10.20.193.78:41204) 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9be6be640 1 -- 10.20.193.78:0/2527615888 learned_addr learned my addr 10.20.193.78:0/2527615888 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 1 --2- 10.20.193.78:0/2527615888 >> v2:10.20.193.115:3300/0 conn(0x7fe9b813d640 0x7fe9b813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient(hunting): get_auth_request con 0x7fe9b813d640 auth_method 0 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 1 -- 10.20.193.78:0/2876213763 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1899938346 0 0) 0x7ff9440a0a20 con 0x7ff94413d640 2026-02-17T05:46:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 10 monclient(hunting): _negotiate 2026-02-17T05:46:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 10 monclient(hunting): my global_id is 4245 2026-02-17T05:46:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226dffb640 1 -- 10.20.193.78:0/1198313666 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908550252 0 0) 0x7f2270074150 con 0x7f227006c890 2026-02-17T05:46:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 1 -- 10.20.193.78:0/2876213763 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff928003620 con 0x7ff94413d640 2026-02-17T05:46:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 1 -- 10.20.193.78:0/2876213763 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3202980958 0 0) 0x7ff9440a0460 con 0x7ff94409ea90 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 10 monclient(hunting): _negotiate 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 10 monclient(hunting): my global_id is 4249 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff942ffd640 1 -- 10.20.193.78:0/2876213763 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9440a0a20 con 0x7ff94409ea90 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226dffb640 10 monclient(hunting): _negotiate 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226dffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226dffb640 10 monclient(hunting): my global_id is 4246 2026-02-17T05:46:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 10 monclient(hunting): handle_auth_done global_id 4240 payload 274 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 1 -- 10.20.193.78:0/2876213763 >> v1:10.20.193.115:6789/0 conn(0x7ff94409ea90 legacy=0x7ff9440a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226dffb640 1 -- 10.20.193.78:0/1198313666 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f225c003620 con 0x7f227006c890 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 1 -- 10.20.193.78:0/2876213763 >> v1:10.20.193.78:6789/0 conn(0x7ff94413d640 legacy=0x7ff94413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 1 monclient: found mon.noname-c 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7ff9437fe640 1 -- 10.20.193.78:0/2876213763 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9440a0740 con 0x7ff94409ee70 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 10 monclient(hunting): handle_auth_done global_id 4243 payload 274 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 1 -- 10.20.193.78:0/1198313666 >> v1:10.20.193.115:6789/0 conn(0x7f227006c890 legacy=0x7f227006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 1 -- 10.20.193.78:0/1198313666 >> v1:10.20.193.78:6789/0 conn(0x7f227006d380 legacy=0x7f227013a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 1 monclient: found mon.noname-c 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:46:13.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7f226f7fe640 1 -- 10.20.193.78:0/1198313666 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227009f250 con 0x7f22700a01f0 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient(hunting): handle_auth_done global_id 4255 payload 274 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 1 -- 10.20.193.78:0/2527615888 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b809ee70 msgr2=0x7fe9b809f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 1 --2- 10.20.193.78:0/2527615888 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b809ee70 0x7fe9b809f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 1 -- 10.20.193.78:0/2527615888 >> v1:10.20.193.78:6789/0 conn(0x7fe9b809ea90 legacy=0x7fe9b80a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 1 monclient: found mon.noname-c 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fe9beebf640 1 -- 10.20.193.78:0/2527615888 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b80a0740 con 0x7fe9b813d640 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb27fc640 1 -- 10.20.193.78:0/932364272 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb40a0a70 msgr2=0x7efeb40a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb27fc640 1 --2- 10.20.193.78:0/932364272 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb40a0a70 0x7efeb40a3320 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efeb40a67e0 tx=0x7efe9c02eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=debfe7ae170ab8eb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6177fe640 1 -- 10.20.193.78:0/1337137719 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 msgr2=0x7fc618138980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6177fe640 1 --2- 10.20.193.78:0/1337137719 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618138980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc6180a1110 tx=0x7fc60402eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11dfa4989c0b89d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 1 -- 10.20.193.78:0/932364272 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c03c070 con 0x7efeb40a0a70 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: dump: 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.099 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 1 -- 10.20.193.78:0/932364272 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c02fdd0 con 0x7efeb40a0a70 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 1 -- 10.20.193.78:0/932364272 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c037660 con 0x7efeb40a0a70 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb9561640 5 monclient: authenticate success, global_id 4226 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb9561640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb9561640 4 set_mon_vals no callback set 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7efeb1ffb640 10 monclient: dump: 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.100 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 1 -- 10.20.193.78:0/1337137719 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60403c070 con 0x7fc6181385a0 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: dump: 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.101 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 1 -- 10.20.193.78:0/1337137719 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60402fdd0 con 0x7fc6181385a0 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 1 -- 10.20.193.78:0/1337137719 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604037660 con 0x7fc6181385a0 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc61dd61640 5 monclient: authenticate success, global_id 4229 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc61dd61640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc61dd61640 4 set_mon_vals no callback set 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.066+0000 7fc6167fc640 10 monclient: dump: 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.102 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 10 monclient: shutdown 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 -- 10.20.193.78:0/932364272 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb40a0a70 msgr2=0x7efeb40a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 --2- 10.20.193.78:0/932364272 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb40a0a70 0x7efeb40a3320 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efeb40a67e0 tx=0x7efe9c02eeb0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 10 monclient: shutdown 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 1 -- 10.20.193.78:0/1337137719 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 msgr2=0x7fc618138980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 1 --2- 10.20.193.78:0/1337137719 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618138980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc6180a1110 tx=0x7fc60402eeb0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 -- 10.20.193.78:0/932364272 shutdown_connections 2026-02-17T05:46:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 --2- 10.20.193.78:0/932364272 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb40a0a70 0x7efeb40a3320 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 -- 10.20.193.78:0/932364272 >> 10.20.193.78:0/932364272 conn(0x7efeb40aad80 msgr2=0x7efeb40ab190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 -- 10.20.193.78:0/932364272 shutdown_connections 2026-02-17T05:46:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 1 -- 10.20.193.78:0/1337137719 shutdown_connections 2026-02-17T05:46:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 1 --2- 10.20.193.78:0/1337137719 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618138980 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 1 -- 10.20.193.78:0/1337137719 >> 10.20.193.78:0/1337137719 conn(0x7fc6180aad80 msgr2=0x7fc6180ab190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 -- 10.20.193.78:0/932364272 wait complete. 2026-02-17T05:46:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 1 -- 10.20.193.78:0/1337137719 shutdown_connections 2026-02-17T05:46:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 1 -- 10.20.193.78:0/1337137719 wait complete. 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 10 monclient: monmap: 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 10 monclient: monmap: 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:13.106 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 Processor -- start 2026-02-17T05:46:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 1 -- start start 2026-02-17T05:46:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7efeb9561640 10 monclient: init 2026-02-17T05:46:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.067+0000 7fc61dd61640 1 Processor -- start 2026-02-17T05:46:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 1 -- start start 2026-02-17T05:46:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient: init 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efeb40a0a70 0x7efeb41cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient(hunting): picked mon.noname-a con 0x7efeb40a0a70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb41385a0 0x7efeb41373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient(hunting): picked mon.noname-c con 0x7efeb41385a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efeb413d390 0x7efeb406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient(hunting): picked mon.noname-b con 0x7efeb413d390 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4141230 con 0x7efeb40a0a70 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41410b0 con 0x7efeb413d390 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efeb41413b0 con 0x7efeb41385a0 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb9561640 10 monclient(hunting): authenticate will time out at 618.622209s 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efeb40a0a70 0x7efeb41cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb41385a0 0x7efeb41373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb41385a0 0x7efeb41373c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47816/0 (socket says 10.20.193.78:47816) 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 -- 10.20.193.78:0/1885259984 learned_addr learned my addr 10.20.193.78:0/1885259984 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient(hunting): get_auth_request con 0x7efeb41385a0 auth_method 0 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efeb40a0a70 0x7efeb41cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52688/0 (socket says 10.20.193.78:52688) 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 10 monclient(hunting): get_auth_request con 0x7efeb40a0a70 auth_method 0 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6180a0a70 0x7fc618137140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient(hunting): picked mon.noname-a con 0x7fc6180a0a70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618137680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient(hunting): picked mon.noname-c con 0x7fc6181385a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc61813d390 0x7fc6181cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient(hunting): picked mon.noname-b con 0x7fc61813d390 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc618141230 con 0x7fc6180a0a70 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181410b0 con 0x7fc61813d390 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc6181413b0 con 0x7fc6181385a0 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc61dd61640 10 monclient(hunting): authenticate will time out at 618.622366s 2026-02-17T05:46:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient(hunting): handle_auth_done global_id 4232 payload 824 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6180a0a70 0x7fc618137140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 -- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efeb413d390 msgr2=0x7efeb406fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efeb413d390 0x7efeb406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 -- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efeb40a0a70 msgr2=0x7efeb41cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efeb40a0a70 0x7efeb41cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6180a0a70 0x7fc618137140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52692/0 (socket says 10.20.193.78:52692) 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 1 -- 10.20.193.78:0/2713489570 learned_addr learned my addr 10.20.193.78:0/2713489570 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 10 monclient(hunting): get_auth_request con 0x7fc6180a0a70 auth_method 0 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618137680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 monclient: found mon.noname-c 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7efeb27fc640 1 -- 10.20.193.78:0/1885259984 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb4073a70 con 0x7efeb41385a0 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient(hunting): get_auth_request con 0x7fc6181385a0 auth_method 0 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc6177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient(hunting): handle_auth_done global_id 4235 payload 824 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 1 -- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc61813d390 msgr2=0x7fc6181cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc61813d390 0x7fc6181cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 1 -- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6180a0a70 msgr2=0x7fc618137140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6180a0a70 0x7fc618137140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 1 monclient: found mon.noname-c 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7fc616ffd640 1 -- 10.20.193.78:0/2713489570 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618073720 con 0x7fc6181385a0 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7ff9437fe640 1 -- 10.20.193.78:0/2876213763 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 msgr2=0x7ff94409f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7ff9437fe640 1 --2- 10.20.193.78:0/2876213763 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 0x7ff94409f2c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9340099a0 tx=0x7ff93402eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e755ed9dd7720774 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7f226f7fe640 1 -- 10.20.193.78:0/1198313666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f22700a01f0 msgr2=0x7f227013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.068+0000 7f226f7fe640 1 --2- 10.20.193.78:0/1198313666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f22700a01f0 0x7f227013d970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2260009e90 tx=0x7f226002f370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e745abed8135fb2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 1 -- 10.20.193.78:0/2876213763 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93403c070 con 0x7ff94409ee70 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: dump: 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.112 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 5 monclient: authenticate success, global_id 4240 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 1 -- 10.20.193.78:0/1198313666 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226003c070 con 0x7f22700a01f0 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: dump: 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.113 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 1 -- 10.20.193.78:0/2876213763 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93402fe00 con 0x7ff94409ee70 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 1 -- 10.20.193.78:0/2876213763 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934037640 con 0x7ff94409ee70 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 5 monclient: authenticate success, global_id 4243 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 4 set_mon_vals no callback set 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff942ffd640 10 monclient: dump: 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.114 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9beebf640 1 -- 10.20.193.78:0/2527615888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b813d640 msgr2=0x7fe9b813fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9beebf640 1 --2- 10.20.193.78:0/2527615888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b813d640 0x7fe9b813fa30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009d90 tx=0x7fe9ac02f190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ad45d91cd45d3e9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 1 -- 10.20.193.78:0/1198313666 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2260007740 con 0x7f22700a01f0 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 1 -- 10.20.193.78:0/1198313666 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260007af0 con 0x7f22700a01f0 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f226dffb640 10 monclient: dump: 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.115 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 4 set_mon_vals no callback set 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 1 -- 10.20.193.78:0/2527615888 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac03c070 con 0x7fe9b813d640 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: dump: 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 1 -- 10.20.193.78:0/2527615888 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac037640 con 0x7fe9b813d640 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 1 -- 10.20.193.78:0/2527615888 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac037a10 con 0x7fe9b813d640 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 5 monclient: authenticate success, global_id 4255 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 4 set_mon_vals no callback set 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9bd6bc640 10 monclient: dump: 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 10 monclient: shutdown 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 1 -- 10.20.193.78:0/2876213763 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 msgr2=0x7ff94409f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 1 --2- 10.20.193.78:0/2876213763 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 0x7ff94409f2c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9340099a0 tx=0x7ff93402eee0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 1 -- 10.20.193.78:0/2876213763 shutdown_connections 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 1 --2- 10.20.193.78:0/2876213763 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 0x7ff94409f2c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 1 -- 10.20.193.78:0/2876213763 >> 10.20.193.78:0/2876213763 conn(0x7ff9440a7520 msgr2=0x7ff9440a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 1 -- 10.20.193.78:0/2876213763 shutdown_connections 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 1 -- 10.20.193.78:0/2876213763 wait complete. 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 10 monclient: shutdown 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 1 -- 10.20.193.78:0/1198313666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f22700a01f0 msgr2=0x7f227013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 1 --2- 10.20.193.78:0/1198313666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f22700a01f0 0x7f227013d970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2260009e90 tx=0x7f226002f370 comp rx=0 tx=0).stop 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 1 -- 10.20.193.78:0/1198313666 shutdown_connections 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 1 --2- 10.20.193.78:0/1198313666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f22700a01f0 0x7f227013d970 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 1 -- 10.20.193.78:0/1198313666 >> 10.20.193.78:0/1198313666 conn(0x7f22700a6620 msgr2=0x7f22700a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 1 -- 10.20.193.78:0/1198313666 shutdown_connections 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7f227576a640 1 -- 10.20.193.78:0/1198313666 wait complete. 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 10 monclient: shutdown 2026-02-17T05:46:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 1 -- 10.20.193.78:0/2527615888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b813d640 msgr2=0x7fe9b813fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 1 --2- 10.20.193.78:0/2527615888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b813d640 0x7fe9b813fa30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009d90 tx=0x7fe9ac02f190 comp rx=0 tx=0).stop 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 1 -- 10.20.193.78:0/2527615888 shutdown_connections 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 1 --2- 10.20.193.78:0/2527615888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b813d640 0x7fe9b813fa30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 1 --2- 10.20.193.78:0/2527615888 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b809ee70 0x7fe9b809f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 1 -- 10.20.193.78:0/2527615888 >> 10.20.193.78:0/2527615888 conn(0x7fe9b80a7520 msgr2=0x7fe9b80a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 1 -- 10.20.193.78:0/2527615888 shutdown_connections 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7fe9c0949640 1 -- 10.20.193.78:0/2527615888 wait complete. 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.069+0000 7ff94aa0c640 10 monclient: monmap: 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient: monmap: 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:13.119 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fe9c0949640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fe9c0949640 10 monclient: monmap: 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 1 Processor -- start 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 1 Processor -- start 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 1 -- start start 2026-02-17T05:46:13.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient: init 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 1 -- start start 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient: init 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f0931bcf640 1 -- 10.20.193.78:0/2495194908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 msgr2=0x7f092c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f0931bcf640 1 --2- 10.20.193.78:0/2495194908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 0x7f092c13b3a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09200099a0 tx=0x7f092002eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2177ef408c19cfc6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fe9c0949640 1 Processor -- start 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 0x7ff9441cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:46:13.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient(hunting): picked mon.noname-c con 0x7ff94409ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f227006c890 0x7f2270135220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 0x7ff9440736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient(hunting): picked mon.noname-c con 0x7f227006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient(hunting): picked mon.noname-b con 0x7ff94409ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f227006d380 0x7f227013ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient(hunting): picked mon.noname-b con 0x7f227006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb27fc640 1 -- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb41385a0 msgr2=0x7efeb41373c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb27fc640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb41385a0 0x7efeb41373c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0098f0 tx=0x7efe9c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff94413d640 0x7ff944073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient(hunting): picked mon.noname-a con 0x7ff94413d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9440a6820 con 0x7ff94413d640 2026-02-17T05:46:13.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f22700a01f0 0x7f22701404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9440a66a0 con 0x7ff94409ee70 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff944141150 con 0x7ff94409ea90 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff94aa0c640 10 monclient(hunting): authenticate will time out at 618.624639s 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient(hunting): picked mon.noname-a con 0x7f22700a01f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f227009d360 con 0x7f22700a01f0 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f227009d1e0 con 0x7f227006d380 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 1 -- 10.20.193.78:0/2495194908 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092003c070 con 0x7f092c09c4a0 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f227009be20 con 0x7f227006c890 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f227576a640 10 monclient(hunting): authenticate will time out at 618.624645s 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: dump: 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 1 -- 10.20.193.78:0/2495194908 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0920037440 con 0x7f092c09c4a0 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 1 -- 10.20.193.78:0/2495194908 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920037810 con 0x7f092c09c4a0 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f093465b640 5 monclient: authenticate success, global_id 4239 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f093465b640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f093465b640 4 set_mon_vals no callback set 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 0x7ff9441cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f09313ce640 10 monclient: dump: 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff948f82640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff94413d640 0x7ff944073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 0x7ff9441cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47836/0 (socket says 10.20.193.78:47836) 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226effd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f227006c890 0x7f2270135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f22700a01f0 0x7f22701404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 -- 10.20.193.78:0/1105802531 learned_addr learned my addr 10.20.193.78:0/1105802531 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient(hunting): get_auth_request con 0x7ff94409ea90 auth_method 0 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff948f82640 10 monclient(hunting): get_auth_request con 0x7ff94413d640 auth_method 0 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff948f82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff948f82640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff948f82640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f22700a01f0 0x7f22701404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52712/0 (socket says 10.20.193.78:52712) 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 -- 10.20.193.78:0/2957048248 learned_addr learned my addr 10.20.193.78:0/2957048248 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient(hunting): get_auth_request con 0x7f22700a01f0 auth_method 0 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226effd640 10 monclient(hunting): get_auth_request con 0x7f227006c890 auth_method 0 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fe9c0949640 1 -- start start 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fe9c0949640 10 monclient: init 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226e7fc640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f227006d380 0x7f227013ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 1 -- 10.20.193.78:0/1885259984 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c046020 con 0x7efeb41385a0 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: dump: 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226e7fc640 10 monclient(hunting): get_auth_request con 0x7f227006d380 auth_method 0 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 1 -- 10.20.193.78:0/1885259984 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c042620 con 0x7efeb41385a0 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 1 -- 10.20.193.78:0/1885259984 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c03c040 con 0x7efeb41385a0 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efe937fe640 10 monclient: dump: 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.127 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb17fa640 4 set_mon_vals no callback set 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff9437fe640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 0x7ff9440736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff948f82640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff948f82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff948f82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff9437fe640 10 monclient(hunting): get_auth_request con 0x7ff94409ee70 auth_method 0 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb9561640 5 monclient: authenticate success, global_id 4232 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb9561640 10 monclient: _renew_subs 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff9437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff9437fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb9561640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb40a8890 con 0x7efeb41385a0 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fc616ffd640 1 -- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 msgr2=0x7fc618137680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fc616ffd640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618137680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b550 tx=0x7fc60800ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff9437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient(hunting): handle_auth_done global_id 4238 payload 824 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient(hunting): handle_auth_done global_id 4257 payload 824 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 -- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 msgr2=0x7ff9440736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 0x7ff9440736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 -- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff94413d640 msgr2=0x7ff944073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff94413d640 0x7ff944073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 -- 10.20.193.78:0/2957048248 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f227006c890 msgr2=0x7f2270135220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f227006c890 0x7f2270135220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb9561640 10 monclient: _renew_subs 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb9561640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 monclient: found mon.noname-c 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7ff943fff640 1 -- 10.20.193.78:0/1105802531 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9440a1c10 con 0x7ff94409ea90 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 -- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f227006d380 msgr2=0x7f227013ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f227006d380 0x7f227013ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb4072340 con 0x7efeb41385a0 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 monclient: found mon.noname-a 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f226f7fe640 1 -- 10.20.193.78:0/2957048248 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2270141b50 con 0x7f22700a01f0 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f093465b640 10 monclient: shutdown 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f093465b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fc5f3fff640 1 -- 10.20.193.78:0/2713489570 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608002780 con 0x7fc6181385a0 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7fc5f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 10 monclient: dump: 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 1 -- 10.20.193.78:0/2713489570 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc608015070 con 0x7fc6181385a0 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 5 monclient: authenticate success, global_id 4235 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 10 monclient: _renew_subs 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6180a8890 con 0x7fc6181385a0 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f093465b640 1 -- 10.20.193.78:0/2495194908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 msgr2=0x7f092c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.070+0000 7f093465b640 1 --2- 10.20.193.78:0/2495194908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 0x7f092c13b3a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09200099a0 tx=0x7f092002eee0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 1 -- 10.20.193.78:0/2495194908 shutdown_connections 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 1 --2- 10.20.193.78:0/2495194908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 0x7f092c13b3a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 1 -- 10.20.193.78:0/2713489570 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608002b60 con 0x7fc6181385a0 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc615ffb640 4 set_mon_vals no callback set 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 1 -- 10.20.193.78:0/2495194908 >> 10.20.193.78:0/2495194908 conn(0x7f092c0a6620 msgr2=0x7f092c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 10 monclient: dump: 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 10 monclient: _renew_subs 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6180722d0 con 0x7fc6181385a0 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 1 -- 10.20.193.78:0/2495194908 shutdown_connections 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 1 -- 10.20.193.78:0/2495194908 wait complete. 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe9b809ea90 0x7fe9b8073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient(hunting): picked mon.noname-a con 0x7fe9b809ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b809ee70 0x7fe9b813bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient(hunting): picked mon.noname-b con 0x7fe9b809ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b813d640 0x7fe9b813c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient(hunting): picked mon.noname-c con 0x7fe9b813d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80a6820 con 0x7fe9b809ea90 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80a66a0 con 0x7fe9b809ee70 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b813ff70 con 0x7fe9b813d640 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9c0949640 10 monclient(hunting): authenticate will time out at 618.625148s 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b813d640 0x7fe9b813c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b813d640 0x7fe9b813c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47852/0 (socket says 10.20.193.78:47852) 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 -- 10.20.193.78:0/1205156130 learned_addr learned my addr 10.20.193.78:0/1205156130 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient(hunting): get_auth_request con 0x7fe9b813d640 auth_method 0 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9bdebd640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b809ee70 0x7fe9b813bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9bdebd640 10 monclient(hunting): get_auth_request con 0x7fe9b809ee70 auth_method 0 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9bdebd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9bdebd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9bdebd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient(hunting): handle_auth_done global_id 4244 payload 824 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 -- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b809ee70 msgr2=0x7fe9b813bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b809ee70 0x7fe9b813bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 -- 10.20.193.78:0/1205156130 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe9b809ea90 msgr2=0x7fe9b8073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe9b809ea90 0x7fe9b8073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 monclient: found mon.noname-c 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9beebf640 1 -- 10.20.193.78:0/1205156130 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b806f870 con 0x7fe9b813d640 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fe9bdebd640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b809ee70 0x7fe9b813bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efe937fe640 1 -- 10.20.193.78:0/1885259984 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7efe9c03f030 con 0x7efeb41385a0 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efeb9561640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efeb9561640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efeb9561640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb413fe90 con 0x7efeb41385a0 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 10 monclient: build_initial_monmap 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 10 monclient: monmap: 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efe937fe640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efe8003df00 0x7efe800403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618138980 con 0x7fc6181385a0 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efe937fe640 1 -- 10.20.193.78:0/1885259984 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7efe9c077800 con 0x7efeb41385a0 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 1 -- 10.20.193.78:0/2713489570 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc608010480 con 0x7fc6181385a0 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 1 Processor -- start 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 1 -- start start 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7f093465b640 10 monclient: init 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efeb2ffd640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efe8003df00 0x7efe800403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7efeb2ffd640 10 monclient: get_auth_request con 0x7efe8003df00 auth_method 0 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.071+0000 7fc5f3fff640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df30 0x7fc5e80403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7fc5f3fff640 1 -- 10.20.193.78:0/2713489570 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7fc608052460 con 0x7fc6181385a0 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7fc6177fe640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df30 0x7fc5e80403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7fc6177fe640 10 monclient: get_auth_request con 0x7fc5e803df30 auth_method 0 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f092c06c890 0x7f092c1331b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient(hunting): picked mon.noname-b con 0x7f092c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f092c06d380 0x7f092c1336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient(hunting): picked mon.noname-c con 0x7f092c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 0x7f092c0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient(hunting): picked mon.noname-a con 0x7f092c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f092c09bfa0 con 0x7f092c09c4a0 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f092c09be20 con 0x7f092c06c890 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f092c13f010 con 0x7f092c06d380 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f093465b640 10 monclient(hunting): authenticate will time out at 618.626251s 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f092c06c890 0x7f092c1331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f092c06c890 0x7f092c1331b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:34896/0 (socket says 10.20.193.78:34896) 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 -- 10.20.193.78:0/283318225 learned_addr learned my addr 10.20.193.78:0/283318225 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient(hunting): get_auth_request con 0x7f092c06c890 auth_method 0 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient(hunting): handle_auth_done global_id 4267 payload 824 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 -- 10.20.193.78:0/283318225 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f092c06d380 msgr2=0x7f092c1336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f092c06d380 0x7f092c1336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 -- 10.20.193.78:0/283318225 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 msgr2=0x7f092c0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 0x7f092c0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient: _finish_hunting 0 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 monclient: found mon.noname-b 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:46:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.072+0000 7f09323d0640 1 -- 10.20.193.78:0/283318225 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c11e890 con 0x7f092c06c890 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f226f7fe640 1 -- 10.20.193.78:0/2957048248 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f22700a01f0 msgr2=0x7f22701404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f226f7fe640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f22700a01f0 0x7f22701404c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2260009e90 tx=0x7f2260004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 1 -- 10.20.193.78:0/2957048248 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260045020 con 0x7f22700a01f0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: dump: 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 1 -- 10.20.193.78:0/2957048248 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226003d040 con 0x7f22700a01f0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 5 monclient: authenticate success, global_id 4257 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 10 monclient: _renew_subs 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227009d810 con 0x7f22700a01f0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f226d7fa640 4 set_mon_vals no callback set 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 10 monclient: _renew_subs 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 1 -- 10.20.193.78:0/2957048248 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226003c070 con 0x7f22700a01f0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22700a2860 con 0x7f22700a01f0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 10 monclient: dump: 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.138 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7ff943fff640 1 -- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 msgr2=0x7ff9441cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7ff943fff640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 0x7ff9441cf330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff93000c700 tx=0x7ff93000cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7fe9beebf640 1 -- 10.20.193.78:0/1205156130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b813d640 msgr2=0x7fe9b813c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7fe9beebf640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b813d640 0x7fe9b813c260 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009ec0 tx=0x7fe9ac004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f224f7fe640 1 -- 10.20.193.78:0/2957048248 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2260004250 con 0x7f22700a01f0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.073+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f227006cc70 con 0x7f22700a01f0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7f224f7fe640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 0x7f223c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7f224f7fe640 1 -- 10.20.193.78:0/2957048248 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f2260077f50 con 0x7f22700a01f0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7efeb2ffd640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efe8003df00 0x7efe800403c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efea80046e0 tx=0x7efea80091c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7fc5f3fff640 1 -- 10.20.193.78:0/2713489570 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc608019ce0 con 0x7fc6181385a0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7fc5f3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7fc5f3fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7fc6177fe640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df30 0x7fc5e80403f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc60402eef0 tx=0x7fc604033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7f226effd640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 0x7f223c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7f226effd640 10 monclient: get_auth_request con 0x7f223c03deb0 auth_method 0 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7f09323d0640 1 -- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f092c06c890 msgr2=0x7f092c1331b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.074+0000 7f09323d0640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f092c06c890 0x7f092c1331b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f091c0036c0 tx=0x7f091c00c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 1 -- 10.20.193.78:0/283318225 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c017070 con 0x7f092c06c890 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: dump: 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 5 monclient: authenticate success, global_id 4267 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 1 -- 10.20.193.78:0/283318225 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c007070 con 0x7f092c06c890 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 10 monclient: _renew_subs 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092c11d160 con 0x7f092c06c890 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 1 -- 10.20.193.78:0/283318225 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c0074c0 con 0x7f092c06c890 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0930bcd640 4 set_mon_vals no callback set 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 10 monclient: dump: 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 10 monclient: _renew_subs 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092c09e480 con 0x7f092c06c890 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092c073060 con 0x7f092c06c890 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f0912ffd640 1 -- 10.20.193.78:0/283318225 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f091c007660 con 0x7f092c06c890 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f226effd640 1 -- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 msgr2=0x7f223c040370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f226effd640 1 -- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 msgr2=0x7f223c040370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f226effd640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 0x7f223c040370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.075+0000 7f226effd640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 0x7f223c040370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7f0912ffd640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f08fc03deb0 0x7f08fc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7f0912ffd640 1 -- 10.20.193.78:0/283318225 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f091c051ce0 con 0x7f092c06c890 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7f0931bcf640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f08fc03deb0 0x7f08fc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7f0931bcf640 10 monclient: get_auth_request con 0x7f08fc03deb0 auth_method 0 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 1 -- 10.20.193.78:0/1205156130 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac047020 con 0x7fe9b813d640 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: dump: 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff943fff640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 0x7ff9441cf330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff93000c700 tx=0x7ff93000cbd0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7ff930017070 mon_map magic: 0 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 1 -- 10.20.193.78:0/1205156130 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac037e10 con 0x7fe9b813d640 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9c0949640 5 monclient: authenticate success, global_id 4244 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9c0949640 10 monclient: _renew_subs 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9c0949640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b80a9ae0 con 0x7fe9b813d640 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 1 -- 10.20.193.78:0/1105802531 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930017070 con 0x7ff94409ea90 2026-02-17T05:46:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 10 monclient: dump: 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 1 -- 10.20.193.78:0/1205156130 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac03c040 con 0x7fe9b813d640 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9bcebb640 4 set_mon_vals no callback set 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9a2ffd640 10 monclient: dump: 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 10 monclient: _finish_auth 0 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 1 -- 10.20.193.78:0/1105802531 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff930005cd0 con 0x7ff94409ea90 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff940ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff9427fc640 4 set_mon_vals no callback set 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff94aa0c640 5 monclient: authenticate success, global_id 4238 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff94aa0c640 10 monclient: _renew_subs 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff94aa0c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9440a97e0 con 0x7ff94409ea90 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9c0949640 10 monclient: _renew_subs 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9c0949640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7efe937fe640 1 -- 10.20.193.78:0/1885259984 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efeb413fe90 con 0x7efeb41385a0 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7efe937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7efe937fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b80a1c10 con 0x7fe9b813d640 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff94aa0c640 10 monclient: _renew_subs 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff94aa0c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.076+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94406feb0 con 0x7ff94409ea90 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7f224f7fe640 1 -- 10.20.193.78:0/2957048248 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f226004f030 con 0x7f22700a01f0 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7f224f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7f224f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff940ff9640 1 -- 10.20.193.78:0/1105802531 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930007270 con 0x7ff94409ea90 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9c0949640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9c0949640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9c0949640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b8136740 con 0x7fe9b813d640 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff940ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff940ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff940ff9640 10 monclient: dump: 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff940ff9640 1 -- 10.20.193.78:0/1105802531 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff930007450 con 0x7ff94409ea90 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff94aa0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff94aa0c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff94aa0c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94409f2c0 con 0x7ff94409ea90 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9a2ffd640 1 -- 10.20.193.78:0/1205156130 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe9ac035b00 con 0x7fe9b813d640 2026-02-17T05:46:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff940ff9640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff91403deb0 0x7ff914040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9a2ffd640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe98803deb0 0x7fe988040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9a2ffd640 1 -- 10.20.193.78:0/1205156130 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7fe9ac078750 con 0x7fe9b813d640 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff9437fe640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff91403deb0 0x7ff914040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7ff9437fe640 10 monclient: get_auth_request con 0x7ff91403deb0 auth_method 0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9be6be640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe98803deb0 0x7fe988040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.077+0000 7fe9be6be640 10 monclient: get_auth_request con 0x7fe98803deb0 auth_method 0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.078+0000 7f0931bcf640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f08fc03deb0 0x7f08fc040370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0920036000 tx=0x7f0920031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.078+0000 7f0912ffd640 1 -- 10.20.193.78:0/283318225 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f091c011360 con 0x7f092c06c890 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.078+0000 7f0912ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.078+0000 7f0912ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.080+0000 7fe9be6be640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe98803deb0 0x7fe988040370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0007fd0 tx=0x7fe9b0007ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.080+0000 7ff9437fe640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff91403deb0 0x7ff914040370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff944074930 tx=0x7ff934033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.081+0000 7fe9a2ffd640 1 -- 10.20.193.78:0/1205156130 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe9ac051030 con 0x7fe9b813d640 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.081+0000 7fe9a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.081+0000 7fe9a2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.083+0000 7ff940ff9640 1 -- 10.20.193.78:0/1105802531 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7ff930051cd0 con 0x7ff94409ea90 2026-02-17T05:46:13.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7f227576a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:46:13.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7f227576a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:46:13.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7f227576a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:13.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2270133220 con 0x7f22700a01f0 2026-02-17T05:46:13.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7f224f7fe640 1 -- 10.20.193.78:0/2957048248 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2260076ad0 con 0x7f22700a01f0 2026-02-17T05:46:13.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7f224f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:46:13.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7f224f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:13.164 INFO:teuthology.orchestra.run.trial078.stdout:51539607555 2026-02-17T05:46:13.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7efeb9561640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:46:13.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7efeb9561640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:46:13.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7efeb9561640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7efeb40a1b60 con 0x7efeb41385a0 2026-02-17T05:46:13.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7efe937fe640 1 -- 10.20.193.78:0/1885259984 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efe9c050030 con 0x7efeb41385a0 2026-02-17T05:46:13.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7efe937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:46:13.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.163+0000 7efe937fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:13.165 INFO:teuthology.orchestra.run.trial078.stdout:51539607554 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 msgr2=0x7f223c040370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 0x7f223c040370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 10 monclient: shutdown 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f22700a01f0 msgr2=0x7f22701404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f22700a01f0 0x7f22701404c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2260009e90 tx=0x7f2260004b40 comp rx=0 tx=0).stop 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 shutdown_connections 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f22700a01f0 0x7f22701404c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f223c03deb0 0x7f223c040370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f227006d380 0x7f227013ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 --2- 10.20.193.78:0/2957048248 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f227006c890 0x7f2270135220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 >> 10.20.193.78:0/2957048248 conn(0x7f22700a6620 msgr2=0x7f2270071470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 shutdown_connections 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7f227576a640 1 -- 10.20.193.78:0/2957048248 wait complete. 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efe8003df00 msgr2=0x7efe800403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efe8003df00 0x7efe800403c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efea80046e0 tx=0x7efea80091c0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 10 monclient: shutdown 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb41385a0 msgr2=0x7efeb41373c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb41385a0 0x7efeb41373c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0098f0 tx=0x7efe9c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 shutdown_connections 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efe8003df00 0x7efe800403c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efeb413d390 0x7efeb406fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efeb41385a0 0x7efeb41373c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 --2- 10.20.193.78:0/1885259984 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efeb40a0a70 0x7efeb41cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 >> 10.20.193.78:0/1885259984 conn(0x7efeb40aad80 msgr2=0x7efeb413f780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 shutdown_connections 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.165+0000 7efeb9561640 1 -- 10.20.193.78:0/1885259984 wait complete. 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.166+0000 7f093465b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.166+0000 7f093465b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.166+0000 7f093465b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.166+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f092c06cc70 con 0x7f092c06c890 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.166+0000 7f0912ffd640 1 -- 10.20.193.78:0/283318225 <== mon.1 v2:10.20.193.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f091c011c80 con 0x7f092c06c890 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.166+0000 7f0912ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.166+0000 7f0912ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:13.167 INFO:teuthology.orchestra.run.trial078.stdout:51539607555 2026-02-17T05:46:13.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.167+0000 7fc61dd61640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:46:13.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.167+0000 7fc61dd61640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:46:13.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.167+0000 7fc61dd61640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.167+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc6180a6530 con 0x7fc6181385a0 2026-02-17T05:46:13.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.167+0000 7fc5f3fff640 1 -- 10.20.193.78:0/2713489570 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc6080520d0 con 0x7fc6181385a0 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.167+0000 7fc5f3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.167+0000 7fc5f3fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stdout:51539607555 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f08fc03deb0 msgr2=0x7f08fc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f08fc03deb0 0x7f08fc040370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0920036000 tx=0x7f0920031000 comp rx=0 tx=0).stop 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 10 monclient: shutdown 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f092c06c890 msgr2=0x7f092c1331b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f092c06c890 0x7f092c1331b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f091c0036c0 tx=0x7f091c00c970 comp rx=0 tx=0).stop 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 shutdown_connections 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f092c09c4a0 0x7f092c0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f092c06d380 0x7f092c1336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f08fc03deb0 0x7f08fc040370 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 --2- 10.20.193.78:0/283318225 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f092c06c890 0x7f092c1331b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 >> 10.20.193.78:0/283318225 conn(0x7f092c0a6620 msgr2=0x7f092c136120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 shutdown_connections 2026-02-17T05:46:13.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7f093465b640 1 -- 10.20.193.78:0/283318225 wait complete. 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7fe9c0949640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7fe9c0949640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7fe9c0949640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe9b80715e0 con 0x7fe9b813d640 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7fe9a2ffd640 1 -- 10.20.193.78:0/1205156130 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe9ac053300 con 0x7fe9b813d640 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7fe9a2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.168+0000 7fe9a2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stdout:51539607555 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.169+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df30 msgr2=0x7fc5e80403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.169+0000 7fc61dd61640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df30 0x7fc5e80403f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc60402eef0 tx=0x7fc604033000 comp rx=0 tx=0).stop 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 10 monclient: shutdown 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 msgr2=0x7fc618137680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618137680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b550 tx=0x7fc60800ba20 comp rx=0 tx=0).stop 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 shutdown_connections 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df30 0x7fc5e80403f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc61813d390 0x7fc6181cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181385a0 0x7fc618137680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 --2- 10.20.193.78:0/2713489570 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6180a0a70 0x7fc618137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 >> 10.20.193.78:0/2713489570 conn(0x7fc6180aad80 msgr2=0x7fc6180a5290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 shutdown_connections 2026-02-17T05:46:13.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.170+0000 7fc61dd61640 1 -- 10.20.193.78:0/2713489570 wait complete. 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe98803deb0 msgr2=0x7fe988040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe98803deb0 0x7fe988040370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0007fd0 tx=0x7fe9b0007ba0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 10 monclient: shutdown 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b813d640 msgr2=0x7fe9b813c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b813d640 0x7fe9b813c260 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009ec0 tx=0x7fe9ac004b40 comp rx=0 tx=0).stop 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 shutdown_connections 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe9b813d640 0x7fe9b813c260 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe98803deb0 0x7fe988040370 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe9b809ee70 0x7fe9b813bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 --2- 10.20.193.78:0/1205156130 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe9b809ea90 0x7fe9b8073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 >> 10.20.193.78:0/1205156130 conn(0x7fe9b80a7520 msgr2=0x7fe9b8136d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 shutdown_connections 2026-02-17T05:46:13.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.171+0000 7fe9c0949640 1 -- 10.20.193.78:0/1205156130 wait complete. 2026-02-17T05:46:13.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.2 2026-02-17T05:46:13.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607554 for osd.3 2026-02-17T05:46:13.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.1 2026-02-17T05:46:13.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.0 2026-02-17T05:46:13.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607555 for osd.5 2026-02-17T05:46:13.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.905+0000 7ff940ff9640 1 -- 10.20.193.78:0/1105802531 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff930050850 con 0x7ff94409ea90 2026-02-17T05:46:13.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.905+0000 7ff940ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:13.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.905+0000 7ff940ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:13.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.987+0000 7ff94aa0c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:46:13.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.987+0000 7ff94aa0c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:46:13.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.987+0000 7ff94aa0c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:13.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.987+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff94406f9b0 con 0x7ff94409ea90 2026-02-17T05:46:13.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.987+0000 7ff940ff9640 1 -- 10.20.193.78:0/1105802531 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff93000ad90 con 0x7ff94409ea90 2026-02-17T05:46:13.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff940ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:46:13.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff940ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:13.991 INFO:teuthology.orchestra.run.trial078.stdout:47244640260 2026-02-17T05:46:13.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff91403deb0 msgr2=0x7ff914040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff91403deb0 0x7ff914040370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff944074930 tx=0x7ff934033000 comp rx=0 tx=0).stop 2026-02-17T05:46:13.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 10 monclient: shutdown 2026-02-17T05:46:13.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:13.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 msgr2=0x7ff9441cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 0x7ff9441cf330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff93000c700 tx=0x7ff93000cbd0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 shutdown_connections 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff94413d640 0x7ff944073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff91403deb0 0x7ff914040370 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff94409ee70 0x7ff9440736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 --2- 10.20.193.78:0/1105802531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff94409ea90 0x7ff9441cf330 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 >> 10.20.193.78:0/1105802531 conn(0x7ff9440a7520 msgr2=0x7ff944138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.988+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 shutdown_connections 2026-02-17T05:46:13.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:13.989+0000 7ff94aa0c640 1 -- 10.20.193.78:0/1105802531 wait complete. 2026-02-17T05:46:13.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.4 2026-02-17T05:46:14.175 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-17T05:46:14.175 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-17T05:46:14.178 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-17T05:46:14.180 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-17T05:46:14.180 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-17T05:46:14.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.245+0000 7f7de6464640 10 monclient: get_monmap_and_config 2026-02-17T05:46:14.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.245+0000 7f7de6464640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.245+0000 7f7de6464640 10 monclient: monmap: 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:14.246779+0000 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:14.246779+0000 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:14.247 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.245+0000 7f7de6464640 1 Processor -- start 2026-02-17T05:46:14.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.245+0000 7f7de6464640 1 -- start start 2026-02-17T05:46:14.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.245+0000 7f7de6464640 10 monclient: init 2026-02-17T05:46:14.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2026-02-17T05:46:14.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de00686e0 0x7f7de0132e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient(hunting): picked mon.noname-b con 0x7f7de00686e0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient(hunting): picked mon.noname-a con 0x7f7de0071980 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7de00a1e70 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de00a3830 con 0x7f7de00a1e70 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7de0068ac0 con 0x7f7de0071980 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7de01329c0 con 0x7f7de00686e0 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de6464640 10 monclient(hunting): authenticate will time out at 619.800324s 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52744/0 (socket says 10.20.193.78:52744) 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7de49da640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f7de00a1e70 0x7f7de00a2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41206/0 (socket says 10.20.193.78:41206) 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/2473500811 learned_addr learned my addr 10.20.193.78:0/2473500811 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient(hunting): get_auth_request con 0x7f7de0071980 auth_method 0 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddffff640 1 --2- 10.20.193.78:0/2473500811 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de00686e0 0x7f7de0132e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddffff640 10 monclient(hunting): get_auth_request con 0x7f7de00686e0 auth_method 0 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient(hunting): handle_auth_done global_id 4263 payload 274 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/2473500811 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de00686e0 msgr2=0x7f7de0132e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 --2- 10.20.193.78:0/2473500811 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de00686e0 0x7f7de0132e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/2473500811 >> v1:10.20.193.78:6789/0 conn(0x7f7de00a1e70 legacy=0x7f7de00a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 monclient: found mon.noname-a 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/2473500811 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de0072870 con 0x7f7de0071980 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7feda73a0640 10 monclient: get_monmap_and_config 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7feda73a0640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7feda73a0640 10 monclient: monmap: 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:14.247817+0000 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:14.247817+0000 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:14.252 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 msgr2=0x7f7de0133350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:46:14.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 msgr2=0x7f7de0133350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:14.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 --2- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f7dd0009970 tx=0x7f7dd002ee90 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:14.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.246+0000 7f7ddf7fe640 1 --2- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:14.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 1 Processor -- start 2026-02-17T05:46:14.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 1 -- start start 2026-02-17T05:46:14.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient: init 2026-02-17T05:46:14.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7feda009ea90 0x7feda00a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient(hunting): picked mon.noname-c con 0x7feda009ea90 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7feda009ee70 0x7feda009f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient(hunting): picked mon.noname-a con 0x7feda009ee70 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feda009d980 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda00a0a20 con 0x7feda009d980 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7feda00a0d00 con 0x7feda009ee70 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7feda00a0460 con 0x7feda009ea90 2026-02-17T05:46:14.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda73a0640 10 monclient(hunting): authenticate will time out at 619.801363s 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5916640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7feda009d980 0x7feda009dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41210/0 (socket says 10.20.193.78:41210) 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5916640 1 -- 10.20.193.78:0/586008487 learned_addr learned my addr 10.20.193.78:0/586008487 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 1 --2- 10.20.193.78:0/586008487 >> v2:10.20.193.115:3300/0 conn(0x7feda009ea90 0x7feda00a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient(hunting): get_auth_request con 0x7feda009ea90 auth_method 0 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient(hunting): handle_auth_done global_id 4270 payload 274 2026-02-17T05:46:14.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 1 -- 10.20.193.78:0/586008487 >> v2:10.20.193.78:3300/0 conn(0x7feda009ee70 msgr2=0x7feda009f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 1 --2- 10.20.193.78:0/586008487 >> v2:10.20.193.78:3300/0 conn(0x7feda009ee70 0x7feda009f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 1 -- 10.20.193.78:0/586008487 >> v1:10.20.193.78:6789/0 conn(0x7feda009d980 legacy=0x7feda009dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 1 monclient: found mon.noname-c 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.247+0000 7feda5115640 1 -- 10.20.193.78:0/586008487 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda00a0740 con 0x7feda009ea90 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda5115640 1 -- 10.20.193.78:0/586008487 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ea90 msgr2=0x7feda00a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda5115640 1 --2- 10.20.193.78:0/586008487 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ea90 0x7feda00a13d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feda006ecd0 tx=0x7fed9002ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=412bcaa33f5b4590 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 1 -- 10.20.193.78:0/586008487 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9003c070 con 0x7feda009ea90 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: dump: 2026-02-17T05:46:14.261 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 1 -- 10.20.193.78:0/586008487 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9002fcb0 con 0x7feda009ea90 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 5 monclient: authenticate success, global_id 4270 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 1 -- 10.20.193.78:0/586008487 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9002fe60 con 0x7feda009ea90 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7fed8ffff640 10 monclient: dump: 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 4 set_mon_vals no callback set 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 10 monclient: shutdown 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 1 -- 10.20.193.78:0/586008487 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ea90 msgr2=0x7feda00a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 1 --2- 10.20.193.78:0/586008487 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ea90 0x7feda00a13d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feda006ecd0 tx=0x7fed9002ed50 comp rx=0 tx=0).stop 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 1 -- 10.20.193.78:0/586008487 shutdown_connections 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 1 --2- 10.20.193.78:0/586008487 >> v2:10.20.193.78:3300/0 conn(0x7feda009ee70 0x7feda009f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 1 --2- 10.20.193.78:0/586008487 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ea90 0x7feda00a13d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 1 -- 10.20.193.78:0/586008487 >> 10.20.193.78:0/586008487 conn(0x7feda00a7520 msgr2=0x7feda00a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 1 -- 10.20.193.78:0/586008487 shutdown_connections 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.250+0000 7feda73a0640 1 -- 10.20.193.78:0/586008487 wait complete. 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient: monmap: 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:14.264 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 1 Processor -- start 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 1 -- start start 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient: init 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feda009d980 0x7feda0134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient(hunting): picked mon.noname-c con 0x7feda009d980 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feda009ea90 0x7feda0135310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient(hunting): picked mon.noname-a con 0x7feda009ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ee70 0x7feda01cf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient(hunting): picked mon.noname-b con 0x7feda009ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7feda00a66a0 con 0x7feda009ea90 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7feda00a1910 con 0x7feda009ee70 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7feda00a6820 con 0x7feda009d980 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda73a0640 10 monclient(hunting): authenticate will time out at 619.805770s 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feda009d980 0x7feda0134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feda009ea90 0x7feda0135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feda009d980 0x7feda0134dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47882/0 (socket says 10.20.193.78:47882) 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 1 -- 10.20.193.78:0/2303403246 learned_addr learned my addr 10.20.193.78:0/2303403246 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feda009ea90 0x7feda0135310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52762/0 (socket says 10.20.193.78:52762) 2026-02-17T05:46:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 10 monclient(hunting): get_auth_request con 0x7feda009d980 auth_method 0 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 10 monclient(hunting): get_auth_request con 0x7feda009ea90 auth_method 0 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda4914640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5916640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ee70 0x7feda01cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5916640 10 monclient(hunting): get_auth_request con 0x7feda009ee70 auth_method 0 2026-02-17T05:46:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5916640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5916640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5916640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 10 monclient(hunting): handle_auth_done global_id 4250 payload 824 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 1 -- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ee70 msgr2=0x7feda01cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ee70 0x7feda01cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 1 -- 10.20.193.78:0/2303403246 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feda009ea90 msgr2=0x7feda0135310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.251+0000 7feda5115640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feda009ea90 0x7feda0135310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.252+0000 7feda5115640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.252+0000 7feda5115640 1 monclient: found mon.noname-c 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.252+0000 7feda5115640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.252+0000 7feda5115640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.252+0000 7feda5115640 1 -- 10.20.193.78:0/2303403246 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda0073c90 con 0x7feda009d980 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda5115640 1 -- 10.20.193.78:0/2303403246 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feda009d980 msgr2=0x7feda0134dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda5115640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feda009d980 0x7feda0134dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed90002410 tx=0x7fed900047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 1 -- 10.20.193.78:0/2303403246 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90046070 con 0x7feda009d980 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: dump: 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda73a0640 5 monclient: authenticate success, global_id 4250 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda73a0640 10 monclient: _renew_subs 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda73a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda00a5c00 con 0x7feda009d980 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 1 -- 10.20.193.78:0/2303403246 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed90041890 con 0x7feda009d980 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 1 -- 10.20.193.78:0/2303403246 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9003c040 con 0x7feda009d980 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8f7fe640 4 set_mon_vals no callback set 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 10 monclient: dump: 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.270 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda73a0640 10 monclient: _renew_subs 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda73a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feda0132890 con 0x7feda009d980 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.254+0000 7fed8dffb640 1 -- 10.20.193.78:0/2303403246 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fed90041cc0 con 0x7feda009d980 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.255+0000 7feda73a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.255+0000 7feda73a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.255+0000 7feda73a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.255+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda00a1730 con 0x7feda009d980 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.255+0000 7fed8dffb640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fed7003deb0 0x7fed70040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.255+0000 7fed8dffb640 1 -- 10.20.193.78:0/2303403246 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7fed90077e60 con 0x7feda009d980 2026-02-17T05:46:14.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.255+0000 7feda4914640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fed7003deb0 0x7fed70040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.255+0000 7feda4914640 10 monclient: get_auth_request con 0x7fed7003deb0 auth_method 0 2026-02-17T05:46:14.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.257+0000 7feda4914640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fed7003deb0 0x7fed70040370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feda00741c0 tx=0x7fed94009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.257+0000 7fed8dffb640 1 -- 10.20.193.78:0/2303403246 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7feda00a1730 con 0x7feda009d980 2026-02-17T05:46:14.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.258+0000 7fed8dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.258+0000 7fed8dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:14.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7f77c1fa9640 10 monclient: get_monmap_and_config 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7f77c1fa9640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7f77c1fa9640 10 monclient: monmap: 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:14.287801+0000 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:14.287801+0000 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:14.288 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7fc50d549640 10 monclient: get_monmap_and_config 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7fc50d549640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7fc50d549640 10 monclient: monmap: 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:14.287894+0000 2026-02-17T05:46:14.289 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:14.287894+0000 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7ffb09da9640 10 monclient: get_monmap_and_config 2026-02-17T05:46:14.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7ffb09da9640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7ffb09da9640 10 monclient: monmap: 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:14.287910+0000 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:14.287910+0000 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:14.291 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.286+0000 7f77c1fa9640 1 Processor -- start 2026-02-17T05:46:14.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 1 Processor -- start 2026-02-17T05:46:14.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 1 -- start start 2026-02-17T05:46:14.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient: init 2026-02-17T05:46:14.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 1 Processor -- start 2026-02-17T05:46:14.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 1 -- start start 2026-02-17T05:46:14.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient: init 2026-02-17T05:46:14.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 1 -- start start 2026-02-17T05:46:14.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient: init 2026-02-17T05:46:14.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f77bc09be20 0x7f77bc09c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient(hunting): picked mon.noname-a con 0x7f77bc09be20 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc09f110 0x7f77bc09c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient(hunting): picked mon.noname-b con 0x7f77bc09f110 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f77bc13c8d0 0x7f77bc13cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient(hunting): picked mon.noname-c con 0x7f77bc13c8d0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f77bc09e770 con 0x7f77bc09be20 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f77bc09cc80 con 0x7f77bc13c8d0 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc09d2c0 con 0x7f77bc09f110 2026-02-17T05:46:14.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77c1fa9640 10 monclient(hunting): authenticate will time out at 619.841389s 2026-02-17T05:46:14.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f77bc09be20 0x7f77bc09c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc09f110 0x7f77bc09c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f77bc09be20 0x7f77bc09c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52764/0 (socket says 10.20.193.78:52764) 2026-02-17T05:46:14.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc09f110 0x7f77bc09c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47896/0 (socket says 10.20.193.78:47896) 2026-02-17T05:46:14.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 -- 10.20.193.78:0/1095216291 learned_addr learned my addr 10.20.193.78:0/1095216291 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:14.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2026-02-17T05:46:14.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient(hunting): get_auth_request con 0x7f77bc09be20 auth_method 0 2026-02-17T05:46:14.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 10 monclient(hunting): get_auth_request con 0x7f77bc09f110 auth_method 0 2026-02-17T05:46:14.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc50809ea90 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:14.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 0x7fc50809f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bbfff640 1 --2- 10.20.193.78:0/1095216291 >> v2:10.20.193.115:3300/0 conn(0x7f77bc13c8d0 0x7f77bc13cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient(hunting): picked mon.noname-a con 0x7fc50809ee70 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:14.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc50813d640 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:14.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bbfff640 10 monclient(hunting): get_auth_request con 0x7f77bc13c8d0 auth_method 0 2026-02-17T05:46:14.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5080a0a20 con 0x7fc50809ea90 2026-02-17T05:46:14.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5080a0460 con 0x7fc50813d640 2026-02-17T05:46:14.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc5080a0d00 con 0x7fc50809ee70 2026-02-17T05:46:14.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc50d549640 10 monclient(hunting): authenticate will time out at 619.841631s 2026-02-17T05:46:14.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T05:46:14.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc506ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc50809ea90 0x7fc5080a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41216/0 (socket says 10.20.193.78:41216) 2026-02-17T05:46:14.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc506ffd640 1 -- 10.20.193.78:0/2177718236 learned_addr learned my addr 10.20.193.78:0/2177718236 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:14.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ffb040730c0 0x7ffb040734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient(hunting): picked mon.noname-c con 0x7ffb040730c0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:14.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 1 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 0x7fc50809f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb041384c0 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:14.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient(hunting): get_auth_request con 0x7fc50809ee70 auth_method 0 2026-02-17T05:46:14.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040a0a40 0x7ffb04140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient(hunting): picked mon.noname-b con 0x7ffb040a0a40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb040a30c0 con 0x7ffb041384c0 2026-02-17T05:46:14.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ffb040a22f0 con 0x7ffb040730c0 2026-02-17T05:46:14.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ffb040a35a0 con 0x7ffb040a0a40 2026-02-17T05:46:14.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb09da9640 10 monclient(hunting): authenticate will time out at 619.841722s 2026-02-17T05:46:14.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient(hunting): handle_auth_done global_id 4269 payload 274 2026-02-17T05:46:14.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 -- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc09f110 msgr2=0x7f77bc09c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 --2- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc09f110 0x7f77bc09c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 -- 10.20.193.78:0/1095216291 >> v2:10.20.193.115:3300/0 conn(0x7f77bc13c8d0 msgr2=0x7f77bc13cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 --2- 10.20.193.78:0/1095216291 >> v2:10.20.193.115:3300/0 conn(0x7f77bc13c8d0 0x7f77bc13cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 monclient: found mon.noname-a 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77bb7fe640 1 -- 10.20.193.78:0/1095216291 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc09d500 con 0x7f77bc09be20 2026-02-17T05:46:14.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7f77baffd640 1 --2- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc09f110 0x7f77bc09c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:14.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb02ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ffb041384c0 0x7ffb040a0330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41230/0 (socket says 10.20.193.78:41230) 2026-02-17T05:46:14.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb02ffd640 1 -- 10.20.193.78:0/3045896676 learned_addr learned my addr 10.20.193.78:0/3045896676 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:14.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 1 --2- 10.20.193.78:0/3045896676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040a0a40 0x7ffb04140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient(hunting): get_auth_request con 0x7ffb040a0a40 auth_method 0 2026-02-17T05:46:14.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb037fe640 1 --2- 10.20.193.78:0/3045896676 >> v2:10.20.193.115:3300/0 conn(0x7ffb040730c0 0x7ffb040734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient(hunting): handle_auth_done global_id 4272 payload 274 2026-02-17T05:46:14.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 1 -- 10.20.193.78:0/2177718236 >> v1:10.20.193.115:6789/0 conn(0x7fc50813d640 legacy=0x7fc50813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 1 -- 10.20.193.78:0/2177718236 >> v1:10.20.193.78:6789/0 conn(0x7fc50809ea90 legacy=0x7fc5080a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb037fe640 10 monclient(hunting): get_auth_request con 0x7ffb040730c0 auth_method 0 2026-02-17T05:46:14.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 1 monclient: found mon.noname-a 2026-02-17T05:46:14.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7fc5067fc640 1 -- 10.20.193.78:0/2177718236 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5080a0740 con 0x7fc50809ee70 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient(hunting): handle_auth_done global_id 4256 payload 274 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 1 -- 10.20.193.78:0/3045896676 >> v2:10.20.193.115:3300/0 conn(0x7ffb040730c0 msgr2=0x7ffb040734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 1 --2- 10.20.193.78:0/3045896676 >> v2:10.20.193.115:3300/0 conn(0x7ffb040730c0 0x7ffb040734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 1 -- 10.20.193.78:0/3045896676 >> v1:10.20.193.78:6789/0 conn(0x7ffb041384c0 legacy=0x7ffb040a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 1 monclient: found mon.noname-b 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.287+0000 7ffb03fff640 1 -- 10.20.193.78:0/3045896676 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb04068950 con 0x7ffb040a0a40 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.288+0000 7fc5067fc640 1 -- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 msgr2=0x7fc50809f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.288+0000 7fc5067fc640 1 -- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 msgr2=0x7fc50809f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.288+0000 7fc5067fc640 1 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 0x7fc50809f2c0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7fc4f00098a0 tx=0x7fc4f002ed80 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:14.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.288+0000 7fc5067fc640 1 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 0x7fc50809f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb03fff640 1 -- 10.20.193.78:0/3045896676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040a0a40 msgr2=0x7ffb04140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb03fff640 1 --2- 10.20.193.78:0/3045896676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040a0a40 0x7ffb04140340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffb040a8010 tx=0x7ffaf802f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7722df2a4849043c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 1 -- 10.20.193.78:0/3045896676 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf803c070 con 0x7ffb040a0a40 2026-02-17T05:46:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: dump: 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 1 -- 10.20.193.78:0/3045896676 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf8037440 con 0x7ffb040a0a40 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 5 monclient: authenticate success, global_id 4256 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 1 -- 10.20.193.78:0/3045896676 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8037810 con 0x7ffb040a0a40 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 4 set_mon_vals no callback set 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb027fc640 10 monclient: dump: 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 10 monclient: shutdown 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 1 -- 10.20.193.78:0/3045896676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040a0a40 msgr2=0x7ffb04140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 1 --2- 10.20.193.78:0/3045896676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040a0a40 0x7ffb04140340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffb040a8010 tx=0x7ffaf802f150 comp rx=0 tx=0).stop 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 1 -- 10.20.193.78:0/3045896676 shutdown_connections 2026-02-17T05:46:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 1 --2- 10.20.193.78:0/3045896676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040a0a40 0x7ffb04140340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 1 --2- 10.20.193.78:0/3045896676 >> v2:10.20.193.115:3300/0 conn(0x7ffb040730c0 0x7ffb040734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 1 -- 10.20.193.78:0/3045896676 >> 10.20.193.78:0/3045896676 conn(0x7ffb040a7370 msgr2=0x7ffb040a7780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 1 -- 10.20.193.78:0/3045896676 shutdown_connections 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.290+0000 7ffb09da9640 1 -- 10.20.193.78:0/3045896676 wait complete. 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient: monmap: 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 1 Processor -- start 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 1 -- start start 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient: init 2026-02-17T05:46:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040730c0 0x7ffb04134b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient(hunting): picked mon.noname-c con 0x7ffb040730c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb040a0a40 0x7ffb04131bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient(hunting): picked mon.noname-a con 0x7ffb040a0a40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb041384c0 0x7ffb04132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient(hunting): picked mon.noname-b con 0x7ffb041384c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb040a2940 con 0x7ffb040a0a40 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb040a2ac0 con 0x7ffb041384c0 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ffb040a0ed0 con 0x7ffb040730c0 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.291+0000 7ffb09da9640 10 monclient(hunting): authenticate will time out at 619.845969s 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040730c0 0x7ffb04134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb02ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb040a0a40 0x7ffb04131bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040730c0 0x7ffb04134b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47928/0 (socket says 10.20.193.78:47928) 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 -- 10.20.193.78:0/966141298 learned_addr learned my addr 10.20.193.78:0/966141298 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient(hunting): get_auth_request con 0x7ffb040730c0 auth_method 0 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb02ffd640 10 monclient(hunting): get_auth_request con 0x7ffb040a0a40 auth_method 0 2026-02-17T05:46:14.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb02ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb02ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77bb7fe640 1 -- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 msgr2=0x7f77bc09c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77bb7fe640 1 --2- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc09c200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77bc06cf30 tx=0x7f77a802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280ef205ba2b727a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb02ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb02ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb02ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb02ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient(hunting): handle_auth_done global_id 4259 payload 824 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 -- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb041384c0 msgr2=0x7ffb04132100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb041384c0 0x7ffb04132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 -- 10.20.193.78:0/966141298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb040a0a40 msgr2=0x7ffb04131bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb040a0a40 0x7ffb04131bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 monclient: found mon.noname-c 2026-02-17T05:46:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 1 -- 10.20.193.78:0/1095216291 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a803c070 con 0x7f77bc09be20 2026-02-17T05:46:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7ffb037fe640 1 -- 10.20.193.78:0/966141298 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb041202b0 con 0x7ffb040730c0 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: dump: 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 1 -- 10.20.193.78:0/1095216291 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a802fca0 con 0x7f77bc09be20 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 5 monclient: authenticate success, global_id 4269 2026-02-17T05:46:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 1 -- 10.20.193.78:0/1095216291 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8037660 con 0x7f77bc09be20 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 4 set_mon_vals no callback set 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77ba7fc640 10 monclient: dump: 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 10 monclient: shutdown 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 -- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 msgr2=0x7f77bc09c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc09c200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77bc06cf30 tx=0x7f77a802ee60 comp rx=0 tx=0).stop 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 -- 10.20.193.78:0/1095216291 shutdown_connections 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/1095216291 >> v2:10.20.193.115:3300/0 conn(0x7f77bc13c8d0 0x7f77bc13cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc09f110 0x7f77bc09c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/1095216291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc09c200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 -- 10.20.193.78:0/1095216291 >> 10.20.193.78:0/1095216291 conn(0x7f77bc0a6620 msgr2=0x7f77bc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 -- 10.20.193.78:0/1095216291 shutdown_connections 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 1 -- 10.20.193.78:0/1095216291 wait complete. 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.292+0000 7f77c1fa9640 10 monclient: monmap: 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 1 Processor -- start 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 1 -- start start 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient: init 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc1d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient(hunting): picked mon.noname-a con 0x7f77bc09be20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77bc09f110 0x7f77bc1d4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient(hunting): picked mon.noname-b con 0x7f77bc09f110 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc13c8d0 0x7f77bc1291b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient(hunting): picked mon.noname-c con 0x7f77bc13c8d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc0a09b0 con 0x7f77bc09be20 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc06cb00 con 0x7f77bc09f110 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc12ce60 con 0x7f77bc13c8d0 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77c1fa9640 10 monclient(hunting): authenticate will time out at 619.847357s 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc1d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc1d3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52792/0 (socket says 10.20.193.78:52792) 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 -- 10.20.193.78:0/2362713997 learned_addr learned my addr 10.20.193.78:0/2362713997 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient(hunting): get_auth_request con 0x7f77bc09be20 auth_method 0 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc13c8d0 0x7f77bc1291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 10 monclient(hunting): get_auth_request con 0x7f77bc13c8d0 auth_method 0 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77baffd640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77bc09f110 0x7f77bc1d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77baffd640 10 monclient(hunting): get_auth_request con 0x7f77bc09f110 auth_method 0 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient(hunting): handle_auth_done global_id 4278 payload 824 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 -- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc13c8d0 msgr2=0x7f77bc1291b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc13c8d0 0x7f77bc1291b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 -- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77bc09f110 msgr2=0x7f77bc1d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77bc09f110 0x7f77bc1d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 monclient: found mon.noname-a 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bb7fe640 1 -- 10.20.193.78:0/2362713997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc12b9b0 con 0x7f77bc09be20 2026-02-17T05:46:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.293+0000 7f77bbfff640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc13c8d0 0x7f77bc1291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb037fe640 1 -- 10.20.193.78:0/966141298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040730c0 msgr2=0x7ffb04134b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb037fe640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040730c0 0x7ffb04134b60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000e890 tx=0x7ffaf000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 1 -- 10.20.193.78:0/966141298 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0017070 con 0x7ffb040730c0 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: dump: 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.330 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 1 -- 10.20.193.78:0/966141298 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf000cad0 con 0x7ffb040730c0 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 1 -- 10.20.193.78:0/966141298 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0013470 con 0x7ffb040730c0 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 10 monclient: dump: 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb01ffb640 4 set_mon_vals no callback set 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 5 monclient: authenticate success, global_id 4259 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 10 monclient: _renew_subs 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb04120590 con 0x7ffb040730c0 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 10 monclient: _renew_subs 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb04136020 con 0x7ffb040730c0 2026-02-17T05:46:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffadffff640 1 -- 10.20.193.78:0/966141298 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ffaf0024060 con 0x7ffb040730c0 2026-02-17T05:46:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.294+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb040734a0 con 0x7ffb040730c0 2026-02-17T05:46:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7ffadffff640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffad403deb0 0x7ffad4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7ffadffff640 1 -- 10.20.193.78:0/966141298 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7ffaf00525e0 con 0x7ffb040730c0 2026-02-17T05:46:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7ffb02ffd640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffad403deb0 0x7ffad4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7ffb02ffd640 10 monclient: get_auth_request con 0x7ffad403deb0 auth_method 0 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77bb7fe640 1 -- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 msgr2=0x7f77bc1d3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77bb7fe640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc1d3b10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77a8002410 tx=0x7f77a8004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 1 -- 10.20.193.78:0/2362713997 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8047070 con 0x7f77bc09be20 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: dump: 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 1 -- 10.20.193.78:0/2362713997 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a802fe50 con 0x7f77bc09be20 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 1 -- 10.20.193.78:0/2362713997 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a803c040 con 0x7f77bc09be20 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f7797fff640 10 monclient: dump: 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77b9ffb640 4 set_mon_vals no callback set 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77c1fa9640 5 monclient: authenticate success, global_id 4278 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77c1fa9640 10 monclient: _renew_subs 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77c1fa9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc09d810 con 0x7f77bc09be20 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77c1fa9640 10 monclient: _renew_subs 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77c1fa9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.295+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc12cb80 con 0x7f77bc09be20 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.296+0000 7f77bb7fe640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc1d3b10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77a8002410 tx=0x7f77a8004830 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f77a803f9f0 mgrmap(e 5) 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.296+0000 7f77c1fa9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.296+0000 7f77c1fa9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.296+0000 7f77c1fa9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.296+0000 7f7797fff640 1 -- 10.20.193.78:0/2362713997 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f77a803f9f0 con 0x7f77bc09be20 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.296+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc13cd40 con 0x7f77bc09be20 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.296+0000 7f7797fff640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f778c03dee0 0x7f778c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.297+0000 7ffadffff640 1 -- 10.20.193.78:0/966141298 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ffaf0051160 con 0x7ffb040730c0 2026-02-17T05:46:14.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.297+0000 7ffadffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.297+0000 7ffadffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:14.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.297+0000 7ffb02ffd640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffad403deb0 0x7ffad4040370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4007d90 tx=0x7ffaf4007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.297+0000 7f77baffd640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f778c03dee0 0x7f778c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.297+0000 7f77baffd640 10 monclient: get_auth_request con 0x7f778c03dee0 auth_method 0 2026-02-17T05:46:14.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.299+0000 7f7797fff640 1 -- 10.20.193.78:0/2362713997 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f77a8076f00 con 0x7f77bc09be20 2026-02-17T05:46:14.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.300+0000 7f77baffd640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f778c03dee0 0x7f778c0403a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f77bc12aa50 tx=0x7f77ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.342+0000 7feda73a0640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:46:14.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.342+0000 7feda73a0640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:46:14.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.342+0000 7feda73a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.342+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feda00a4d00 con 0x7feda009d980 2026-02-17T05:46:14.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.342+0000 7fed8dffb640 1 -- 10.20.193.78:0/2303403246 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feda00a4d00 con 0x7feda009d980 2026-02-17T05:46:14.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.342+0000 7fed8dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-17T05:46:14.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.342+0000 7fed8dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:14.343 INFO:teuthology.orchestra.run.trial078.stdout:51539607557 2026-02-17T05:46:14.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fed7003deb0 msgr2=0x7fed70040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fed7003deb0 0x7fed70040370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feda00741c0 tx=0x7fed94009210 comp rx=0 tx=0).stop 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 10 monclient: shutdown 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feda009d980 msgr2=0x7feda0134dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feda009d980 0x7feda0134dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed90002410 tx=0x7fed900047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 shutdown_connections 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fed7003deb0 0x7fed70040370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feda009ee70 0x7feda01cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feda009ea90 0x7feda0135310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 --2- 10.20.193.78:0/2303403246 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feda009d980 0x7feda0134dd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 >> 10.20.193.78:0/2303403246 conn(0x7feda00a7520 msgr2=0x7feda00a3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 shutdown_connections 2026-02-17T05:46:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.343+0000 7feda73a0640 1 -- 10.20.193.78:0/2303403246 wait complete. 2026-02-17T05:46:14.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607557 for osd.2 2026-02-17T05:46:14.353 DEBUG:teuthology.parallel:result is None 2026-02-17T05:46:14.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.379+0000 7ffb09da9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:46:14.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.379+0000 7ffb09da9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:46:14.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.379+0000 7ffb09da9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.379+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ffb0411da50 con 0x7ffb040730c0 2026-02-17T05:46:14.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.379+0000 7ffadffff640 1 -- 10.20.193.78:0/966141298 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffaf00123d0 con 0x7ffb040730c0 2026-02-17T05:46:14.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.379+0000 7ffadffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-17T05:46:14.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.379+0000 7ffadffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:14.381 INFO:teuthology.orchestra.run.trial078.stdout:51539607557 2026-02-17T05:46:14.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffad403deb0 msgr2=0x7ffad4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffad403deb0 0x7ffad4040370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4007d90 tx=0x7ffaf4007ce0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 10 monclient: shutdown 2026-02-17T05:46:14.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:14.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040730c0 msgr2=0x7ffb04134b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040730c0 0x7ffb04134b60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000e890 tx=0x7ffaf000ed60 comp rx=0 tx=0).stop 2026-02-17T05:46:14.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 shutdown_connections 2026-02-17T05:46:14.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffad403deb0 0x7ffad4040370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb041384c0 0x7ffb04132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb040a0a40 0x7ffb04131bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 --2- 10.20.193.78:0/966141298 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb040730c0 0x7ffb04134b60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 >> 10.20.193.78:0/966141298 conn(0x7ffb040a7370 msgr2=0x7ffb041c1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:14.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 shutdown_connections 2026-02-17T05:46:14.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.380+0000 7ffb09da9640 1 -- 10.20.193.78:0/966141298 wait complete. 2026-02-17T05:46:14.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.5 2026-02-17T05:46:14.391 DEBUG:teuthology.parallel:result is None 2026-02-17T05:46:14.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddf7fe640 1 --2- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddf7fe640 10 monclient: get_auth_request con 0x7f7de0071980 auth_method 0 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddf7fe640 0 --2- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddf7fe640 1 --2- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 10 monclient: ms_handle_reset current mon v2:10.20.193.78:3300/0 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 1 -- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 msgr2=0x7f7de0133350 unknown :-1 s=STATE_CLOSED l=0).mark_down 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 1 --2- 10.20.193.78:0/2473500811 >> v2:10.20.193.78:3300/0 conn(0x7f7de0071980 0x7f7de0133350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 10 monclient: _add_conns ranks=[4,1] 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 1 --2- 10.20.193.78:0/2473500811 >> v2:10.20.193.115:3300/0 conn(0x7f7dc4003600 0x7f7dc4005a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 10 monclient(hunting): picked mon.noname-c con 0x7f7dc4003600 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:14.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7dc40063b0 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 1 -- 10.20.193.78:0/2473500811 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc400a360 con 0x7f7dc40063b0 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 1 -- 10.20.193.78:0/2473500811 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc400b560 con 0x7f7dc4003600 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddeffd640 10 monclient(hunting): ms_handle_reset stray mon v2:10.20.193.78:3300/0 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 1 --2- 10.20.193.78:0/2473500811 >> v2:10.20.193.115:3300/0 conn(0x7f7dc4003600 0x7f7dc4005a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient(hunting): get_auth_request con 0x7f7dc4003600 auth_method 0 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:46:14.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient(hunting): handle_auth_done global_id 4263 payload 293 2026-02-17T05:46:14.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 1 -- 10.20.193.78:0/2473500811 >> v1:10.20.193.115:6789/0 conn(0x7f7dc40063b0 legacy=0x7f7dc4008870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 1 monclient: found mon.noname-c 2026-02-17T05:46:14.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:46:14.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.448+0000 7f7ddffff640 1 -- 10.20.193.78:0/2473500811 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc400fd30 con 0x7f7dc4003600 2026-02-17T05:46:14.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddffff640 1 -- 10.20.193.78:0/2473500811 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 msgr2=0x7f7dc4005a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddffff640 1 --2- 10.20.193.78:0/2473500811 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 0x7f7dc4005a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009b10 tx=0x7f7dd0009340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724407db57b15567 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 1 -- 10.20.193.78:0/2473500811 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc017070 con 0x7f7dc4003600 2026-02-17T05:46:14.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: dump: 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.460 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 5 monclient: authenticate success, global_id 4263 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 1 -- 10.20.193.78:0/2473500811 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dcc00ec00 con 0x7f7dc4003600 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 1 -- 10.20.193.78:0/2473500811 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc009790 con 0x7f7dc4003600 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 4 set_mon_vals no callback set 2026-02-17T05:46:14.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7ddeffd640 10 monclient: dump: 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.462 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 10 monclient: shutdown 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 1 -- 10.20.193.78:0/2473500811 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 msgr2=0x7f7dc4005a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 1 --2- 10.20.193.78:0/2473500811 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 0x7f7dc4005a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009b10 tx=0x7f7dd0009340 comp rx=0 tx=0).stop 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de49da640 1 -- 10.20.193.78:0/2473500811 reap_dead start 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 1 -- 10.20.193.78:0/2473500811 shutdown_connections 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 1 --2- 10.20.193.78:0/2473500811 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 0x7f7dc4005a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 1 -- 10.20.193.78:0/2473500811 >> 10.20.193.78:0/2473500811 conn(0x7f7de00a7370 msgr2=0x7f7de00a7780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 1 -- 10.20.193.78:0/2473500811 shutdown_connections 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.450+0000 7f7de6464640 1 -- 10.20.193.78:0/2473500811 wait complete. 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.451+0000 7f7de6464640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.451+0000 7f7de6464640 10 monclient: monmap: 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.451+0000 7f7de6464640 1 Processor -- start 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.451+0000 7f7de6464640 1 -- start start 2026-02-17T05:46:14.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.451+0000 7f7de6464640 10 monclient: init 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.451+0000 7f7de6464640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 0x7f7de00a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient(hunting): picked mon.noname-b con 0x7f7dc4003600 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7de0131d90 0x7f7de011d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient(hunting): picked mon.noname-a con 0x7f7de0131d90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de01321b0 0x7f7de011dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient(hunting): picked mon.noname-c con 0x7f7de01321b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7de00a2a10 con 0x7f7de0131d90 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0072e30 con 0x7f7dc4003600 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7de00a2b90 con 0x7f7de01321b0 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de6464640 10 monclient(hunting): authenticate will time out at 620.006068s 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7de0131d90 0x7f7de011d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de49da640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de01321b0 0x7f7de011dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7de0131d90 0x7f7de011d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52816/0 (socket says 10.20.193.78:52816) 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/4104380602 learned_addr learned my addr 10.20.193.78:0/4104380602 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient(hunting): get_auth_request con 0x7f7de0131d90 auth_method 0 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de49da640 10 monclient(hunting): get_auth_request con 0x7f7de01321b0 auth_method 0 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de49da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de49da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de49da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddffff640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 0x7f7de00a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddffff640 10 monclient(hunting): get_auth_request con 0x7f7dc4003600 auth_method 0 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de49da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de49da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7de49da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient(hunting): handle_auth_done global_id 4281 payload 824 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/4104380602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de01321b0 msgr2=0x7f7de011dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de01321b0 0x7f7de011dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 msgr2=0x7f7de00a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 0x7f7de00a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 monclient: found mon.noname-a 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/4104380602 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de00740c0 con 0x7f7de0131d90 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.452+0000 7f7ddffff640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 0x7f7de00a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddf7fe640 1 -- 10.20.193.78:0/4104380602 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7de0131d90 msgr2=0x7f7de011d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddf7fe640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7de0131d90 0x7f7de011d5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7dd002eed0 tx=0x7f7dd00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 1 -- 10.20.193.78:0/4104380602 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0037df0 con 0x7f7de0131d90 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: dump: 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.469 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 1 -- 10.20.193.78:0/4104380602 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd0004030 con 0x7f7de0131d90 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7de6464640 5 monclient: authenticate success, global_id 4281 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7de6464640 10 monclient: _renew_subs 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7de6464640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de0120760 con 0x7f7de0131d90 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 1 -- 10.20.193.78:0/4104380602 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0004420 con 0x7f7de0131d90 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7dde7fc640 4 set_mon_vals no callback set 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7ddcff9640 10 monclient: dump: 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.471 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7de6464640 10 monclient: _renew_subs 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7de6464640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.454+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de0120480 con 0x7f7de0131d90 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7ddcff9640 1 -- 10.20.193.78:0/4104380602 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7dd0049070 con 0x7f7de0131d90 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7de6464640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7de6464640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7de6464640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de0120280 con 0x7f7de0131d90 2026-02-17T05:46:14.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7ddcff9640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7db003dee0 0x7f7db00403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7ddcff9640 1 -- 10.20.193.78:0/4104380602 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7f7dd0076ea0 con 0x7f7de0131d90 2026-02-17T05:46:14.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7ddffff640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7db003dee0 0x7f7db00403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.455+0000 7f7ddffff640 10 monclient: get_auth_request con 0x7f7db003dee0 auth_method 0 2026-02-17T05:46:14.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.457+0000 7f7ddcff9640 1 -- 10.20.193.78:0/4104380602 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7de00740c0 con 0x7f7de0131d90 2026-02-17T05:46:14.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.457+0000 7f7ddcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.457+0000 7f7ddcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:14.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.457+0000 7f7ddffff640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7db003dee0 0x7f7db00403a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0090b0 tx=0x7f7dcc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 1 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 0x7fc50809f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient: get_auth_request con 0x7fc50809ee70 auth_method 0 2026-02-17T05:46:14.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 0 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 0x7fc50809f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-17T05:46:14.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 1 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 0x7fc50809f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient: ms_handle_reset current mon v2:10.20.193.78:3300/0 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 1 -- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 msgr2=0x7fc50809f2c0 unknown :-1 s=STATE_CLOSED l=0).mark_down 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 1 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.78:3300/0 conn(0x7fc50809ee70 0x7fc50809f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient: _add_conns ranks=[4,2] 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 1 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.115:3300/0 conn(0x7fc4f4003600 0x7fc4f4005a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient(hunting): picked mon.noname-c con 0x7fc4f4003600 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 1 --2- 10.20.193.78:0/2177718236 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4f4006350 0x7fc4f4008810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient(hunting): picked mon.noname-b con 0x7fc4f4006350 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 1 -- 10.20.193.78:0/2177718236 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f4009190 con 0x7fc4f4003600 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 1 -- 10.20.193.78:0/2177718236 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f4009340 con 0x7fc4f4006350 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc505ffb640 10 monclient(hunting): ms_handle_reset stray mon v2:10.20.193.78:3300/0 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 1 --2- 10.20.193.78:0/2177718236 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4f4006350 0x7fc4f4008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient(hunting): get_auth_request con 0x7fc4f4006350 auth_method 0 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient(hunting): handle_auth_done global_id 4272 payload 293 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 1 -- 10.20.193.78:0/2177718236 >> v2:10.20.193.115:3300/0 conn(0x7fc4f4003600 msgr2=0x7fc4f4005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:14.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 1 --2- 10.20.193.78:0/2177718236 >> v2:10.20.193.115:3300/0 conn(0x7fc4f4003600 0x7fc4f4005a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 1 monclient: found mon.noname-b 2026-02-17T05:46:14.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.489+0000 7fc5067fc640 1 -- 10.20.193.78:0/2177718236 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f4009600 con 0x7fc4f4006350 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc5067fc640 1 -- 10.20.193.78:0/2177718236 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4f4006350 msgr2=0x7fc4f4008810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc5067fc640 1 --2- 10.20.193.78:0/2177718236 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4f4006350 0x7fc4f4008810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc508069da0 tx=0x7fc4f0009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509b6c5ed56349be server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 1 -- 10.20.193.78:0/2177718236 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f003e020 con 0x7fc4f4006350 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: dump: 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.502 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 1 -- 10.20.193.78:0/2177718236 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f0004230 con 0x7fc4f4006350 2026-02-17T05:46:14.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 1 -- 10.20.193.78:0/2177718236 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f002fc10 con 0x7fc4f4006350 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc50d549640 5 monclient: authenticate success, global_id 4272 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc50d549640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc50d549640 4 set_mon_vals no callback set 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.491+0000 7fc505ffb640 10 monclient: dump: 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.504 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 10 monclient: shutdown 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 1 -- 10.20.193.78:0/2177718236 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4f4006350 msgr2=0x7fc4f4008810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 1 --2- 10.20.193.78:0/2177718236 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4f4006350 0x7fc4f4008810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc508069da0 tx=0x7fc4f0009290 comp rx=0 tx=0).stop 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc5077fe640 1 -- 10.20.193.78:0/2177718236 reap_dead start 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 1 -- 10.20.193.78:0/2177718236 shutdown_connections 2026-02-17T05:46:14.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 1 -- 10.20.193.78:0/2177718236 >> 10.20.193.78:0/2177718236 conn(0x7fc5080a7520 msgr2=0x7fc5080a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 1 -- 10.20.193.78:0/2177718236 shutdown_connections 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 1 -- 10.20.193.78:0/2177718236 wait complete. 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 10 monclient: build_initial_monmap 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.492+0000 7fc50d549640 10 monclient: monmap: 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:14.506 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 1 Processor -- start 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 1 -- start start 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient: init 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 0x7fc50813a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient(hunting): picked mon.noname-c con 0x7fc508139fb0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc50813a8d0 0x7fc50813acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient(hunting): picked mon.noname-b con 0x7fc50813a8d0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc5081d3fc0 0x7fc5081d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient(hunting): picked mon.noname-a con 0x7fc5081d3fc0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:14.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5080a6820 con 0x7fc5081d3fc0 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc5080a66a0 con 0x7fc50813a8d0 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc508141150 con 0x7fc508139fb0 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc50d549640 10 monclient(hunting): authenticate will time out at 620.047468s 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 0x7fc50813a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 0x7fc50813a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47958/0 (socket says 10.20.193.78:47958) 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 -- 10.20.193.78:0/61631881 learned_addr learned my addr 10.20.193.78:0/61631881 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient(hunting): get_auth_request con 0x7fc508139fb0 auth_method 0 2026-02-17T05:46:14.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5077fe640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc5081d3fc0 0x7fc5081d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5077fe640 10 monclient(hunting): get_auth_request con 0x7fc5081d3fc0 auth_method 0 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5077fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5067fc640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc50813a8d0 0x7fc50813acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5067fc640 10 monclient(hunting): get_auth_request con 0x7fc50813a8d0 auth_method 0 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5067fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5067fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5067fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc5077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient(hunting): handle_auth_done global_id 4268 payload 824 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 -- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc50813a8d0 msgr2=0x7fc50813acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc50813a8d0 0x7fc50813acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 -- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc5081d3fc0 msgr2=0x7fc5081d63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc5081d3fc0 0x7fc5081d63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:46:14.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 monclient: found mon.noname-c 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.493+0000 7fc506ffd640 1 -- 10.20.193.78:0/61631881 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50813c2a0 con 0x7fc508139fb0 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc506ffd640 1 -- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 msgr2=0x7fc50813a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc506ffd640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 0x7fc50813a390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00c7b0 tx=0x7fc4fc00cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 1 -- 10.20.193.78:0/61631881 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc017070 con 0x7fc508139fb0 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc506ffd640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 0x7fc50813a390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00c7b0 tx=0x7fc4fc00cc80 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fc4fc005e00 config(0 keys) 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 10 monclient: dump: 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.511 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 1 -- 10.20.193.78:0/61631881 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4fc005e00 con 0x7fc508139fb0 2026-02-17T05:46:14.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc4e77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc50d549640 5 monclient: authenticate success, global_id 4268 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc5057fa640 4 set_mon_vals no callback set 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc50d549640 10 monclient: _renew_subs 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc50d549640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5080701a0 con 0x7fc508139fb0 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc50d549640 10 monclient: _renew_subs 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc50d549640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.495+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5080a9e80 con 0x7fc508139fb0 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc50d549640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc50d549640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc50d549640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc508140ce0 con 0x7fc508139fb0 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc4e77fe640 1 -- 10.20.193.78:0/61631881 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0072b0 con 0x7fc508139fb0 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc4e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:14.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc4e77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc4e77fe640 10 monclient: dump: 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc4e77fe640 1 -- 10.20.193.78:0/61631881 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc4fc007490 con 0x7fc508139fb0 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.496+0000 7fc4e77fe640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4d403de60 0x7fc4d4040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:14.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.497+0000 7fc5067fc640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4d403de60 0x7fc4d4040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:14.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.497+0000 7fc5067fc640 10 monclient: get_auth_request con 0x7fc4d403de60 auth_method 0 2026-02-17T05:46:14.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.499+0000 7fc4e77fe640 1 -- 10.20.193.78:0/61631881 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7fc4fc052020 con 0x7fc508139fb0 2026-02-17T05:46:14.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.499+0000 7fc5067fc640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4d403de60 0x7fc4d4040320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0004000 tx=0x7fc4f0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:14.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.539+0000 7f7de6464640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:46:14.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.539+0000 7f7de6464640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:46:14.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.539+0000 7f7de6464640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:14.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.539+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7de00a2300 con 0x7f7de0131d90 2026-02-17T05:46:14.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7ddcff9640 1 -- 10.20.193.78:0/4104380602 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7dd0075a20 con 0x7f7de0131d90 2026-02-17T05:46:14.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7ddcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-17T05:46:14.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7ddcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:14.542 INFO:teuthology.orchestra.run.trial078.stdout:51539607556 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7db003dee0 msgr2=0x7f7db00403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7db003dee0 0x7f7db00403a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0090b0 tx=0x7f7dcc008030 comp rx=0 tx=0).stop 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 10 monclient: shutdown 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7de0131d90 msgr2=0x7f7de011d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7de0131d90 0x7f7de011d5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7dd002eed0 tx=0x7f7dd00047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 shutdown_connections 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7de01321b0 0x7f7de011dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7de0131d90 0x7f7de011d5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7db003dee0 0x7f7db00403a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 --2- 10.20.193.78:0/4104380602 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dc4003600 0x7f7de00a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:14.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 >> 10.20.193.78:0/4104380602 conn(0x7f7de00a7370 msgr2=0x7f7de00a4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:14.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 shutdown_connections 2026-02-17T05:46:14.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:14.540+0000 7f7de6464640 1 -- 10.20.193.78:0/4104380602 wait complete. 2026-02-17T05:46:14.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.3 2026-02-17T05:46:14.550 DEBUG:teuthology.parallel:result is None 2026-02-17T05:46:14.999 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 10 monclient: get_monmap_and_config 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 10 monclient: build_initial_monmap 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 10 monclient: monmap: 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:15.069482+0000 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:15.069482+0000 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:15.069 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:15.070 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:15.070 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:15.070 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 1 Processor -- start 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 1 -- start start 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 10 monclient: init 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 10 monclient: _add_conns ranks=[0,1,3,2,4] 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc3c406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc3c409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.068+0000 7fc3cb75e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc3c406d380 0x7fc3c413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3cb75e640 10 monclient(hunting): picked mon.noname-a con 0x7fc3c406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3cb75e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c406daf0 con 0x7fc3c406c890 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3cb75e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c409cc80 con 0x7fc3c409c4a0 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3cb75e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3cb75e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c40a0880 con 0x7fc3c406d380 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3cb75e640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3cb75e640 10 monclient(hunting): authenticate will time out at 620.623029s 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c94d3640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc3c406c890 0x7fc3c406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41242/0 (socket says 10.20.193.78:41242) 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c94d3640 1 -- 10.20.193.78:0/2546846202 learned_addr learned my addr 10.20.193.78:0/2546846202 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:15.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc3c406d380 0x7fc3c413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient(hunting): get_auth_request con 0x7fc3c406d380 auth_method 0 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient(hunting): handle_auth_done global_id 4287 payload 274 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 1 -- 10.20.193.78:0/2546846202 >> v1:10.20.193.115:6789/0 conn(0x7fc3c409c4a0 legacy=0x7fc3c413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 1 -- 10.20.193.78:0/2546846202 >> v1:10.20.193.78:6789/0 conn(0x7fc3c406c890 legacy=0x7fc3c406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient: _finish_hunting 0 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 1 monclient: found mon.noname-a 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.069+0000 7fc3c9cd4640 1 -- 10.20.193.78:0/2546846202 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c409cf60 con 0x7fc3c406d380 2026-02-17T05:46:15.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3c9cd4640 1 -- 10.20.193.78:0/2546846202 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 msgr2=0x7fc3c413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:15.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3c9cd4640 1 --2- 10.20.193.78:0/2546846202 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c413ead0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4074de0 tx=0x7fc3c002f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1d1857addf257d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3b3fff640 1 -- 10.20.193.78:0/2546846202 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c003c070 con 0x7fc3c406d380 2026-02-17T05:46:15.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3b3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3b3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:15.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3b3fff640 10 monclient: dump: 2026-02-17T05:46:15.080 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.080 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.080 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3b3fff640 10 monclient: _finish_auth 0 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3b3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.071+0000 7fc3b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 5 monclient: authenticate success, global_id 4287 2026-02-17T05:46:15.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3b3fff640 1 -- 10.20.193.78:0/2546846202 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c0037640 con 0x7fc3c406d380 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3b3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3b3fff640 1 -- 10.20.193.78:0/2546846202 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0037a10 con 0x7fc3c406d380 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 4 set_mon_vals no callback set 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3b3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3b3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3b3fff640 10 monclient: dump: 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.082 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 10 monclient: shutdown 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 1 -- 10.20.193.78:0/2546846202 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 msgr2=0x7fc3c413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 1 --2- 10.20.193.78:0/2546846202 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c413ead0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4074de0 tx=0x7fc3c002f130 comp rx=0 tx=0).stop 2026-02-17T05:46:15.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 1 -- 10.20.193.78:0/2546846202 shutdown_connections 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 1 --2- 10.20.193.78:0/2546846202 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c413ead0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 1 -- 10.20.193.78:0/2546846202 >> 10.20.193.78:0/2546846202 conn(0x7fc3c40a6620 msgr2=0x7fc3c40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 1 -- 10.20.193.78:0/2546846202 shutdown_connections 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.072+0000 7fc3cb75e640 1 -- 10.20.193.78:0/2546846202 wait complete. 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient: build_initial_monmap 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient: monmap: 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:15.084 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 1 Processor -- start 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 1 -- start start 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient: init 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c406c890 0x7fc3c413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient(hunting): picked mon.noname-b con 0x7fc3c406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c4132190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient(hunting): picked mon.noname-a con 0x7fc3c406d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c409c4a0 0x7fc3c409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient(hunting): picked mon.noname-c con 0x7fc3c409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c409bfa0 con 0x7fc3c406d380 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c409be20 con 0x7fc3c406c890 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c413f010 con 0x7fc3c409c4a0 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3cb75e640 10 monclient(hunting): authenticate will time out at 620.627528s 2026-02-17T05:46:15.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c4132190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c4132190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52852/0 (socket says 10.20.193.78:52852) 2026-02-17T05:46:15.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 -- 10.20.193.78:0/3718900810 learned_addr learned my addr 10.20.193.78:0/3718900810 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:15.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c409c4a0 0x7fc3c409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient(hunting): get_auth_request con 0x7fc3c406d380 auth_method 0 2026-02-17T05:46:15.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 10 monclient(hunting): get_auth_request con 0x7fc3c409c4a0 auth_method 0 2026-02-17T05:46:15.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c94d3640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c406c890 0x7fc3c413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c94d3640 10 monclient(hunting): get_auth_request con 0x7fc3c406c890 auth_method 0 2026-02-17T05:46:15.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c94d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c94d3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c94d3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c94d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c94d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c94d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient(hunting): handle_auth_done global_id 4290 payload 824 2026-02-17T05:46:15.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 -- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c409c4a0 msgr2=0x7fc3c409db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c409c4a0 0x7fc3c409db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 -- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c406c890 msgr2=0x7fc3c413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c406c890 0x7fc3c413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient: _finish_hunting 0 2026-02-17T05:46:15.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 monclient: found mon.noname-a 2026-02-17T05:46:15.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:15.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c8cd2640 1 -- 10.20.193.78:0/3718900810 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c411d850 con 0x7fc3c406d380 2026-02-17T05:46:15.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.073+0000 7fc3c9cd4640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c409c4a0 0x7fc3c409db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:15.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3c8cd2640 1 -- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 msgr2=0x7fc3c4132190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:15.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3c8cd2640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c4132190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c5d0 tx=0x7fc3b800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 1 -- 10.20.193.78:0/3718900810 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8016020 con 0x7fc3c406d380 2026-02-17T05:46:15.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:15.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: dump: 2026-02-17T05:46:15.093 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.094 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.094 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.094 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.094 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.094 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.094 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.094 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.095 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.095 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:46:15.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:15.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:15.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 1 -- 10.20.193.78:0/3718900810 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8012a60 con 0x7fc3c406d380 2026-02-17T05:46:15.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 5 monclient: authenticate success, global_id 4290 2026-02-17T05:46:15.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 10 monclient: _renew_subs 2026-02-17T05:46:15.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c409f4e0 con 0x7fc3c406d380 2026-02-17T05:46:15.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:15.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 1 -- 10.20.193.78:0/3718900810 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8012e50 con 0x7fc3c406d380 2026-02-17T05:46:15.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:15.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b37fe640 4 set_mon_vals no callback set 2026-02-17T05:46:15.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 10 monclient: dump: 2026-02-17T05:46:15.097 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.097 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.097 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.097 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.098 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.098 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.098 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.098 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.098 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.098 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 10 monclient: _renew_subs 2026-02-17T05:46:15.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c411d570 con 0x7fc3c406d380 2026-02-17T05:46:15.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c406cc70 con 0x7fc3c406d380 2026-02-17T05:46:15.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.076+0000 7fc3b1ffb640 1 -- 10.20.193.78:0/3718900810 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc3b801b400 con 0x7fc3c406d380 2026-02-17T05:46:15.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.077+0000 7fc3b1ffb640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403e0a0 0x7fc394040560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.077+0000 7fc3b1ffb640 1 -- 10.20.193.78:0/3718900810 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4398+0+0 (secure 0 0 0) 0x7fc3b80526e0 con 0x7fc3c406d380 2026-02-17T05:46:15.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.077+0000 7fc3c94d3640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403e0a0 0x7fc394040560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.077+0000 7fc3c94d3640 10 monclient: get_auth_request con 0x7fc39403e0a0 auth_method 0 2026-02-17T05:46:15.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.079+0000 7fc3b1ffb640 1 -- 10.20.193.78:0/3718900810 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc3b8020030 con 0x7fc3c406d380 2026-02-17T05:46:15.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.079+0000 7fc3b1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.079+0000 7fc3b1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:15.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.079+0000 7fc3c94d3640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403e0a0 0x7fc394040560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4007d90 tx=0x7fc3b4007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.124+0000 7f7797fff640 1 -- 10.20.193.78:0/2362713997 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f77a8075a80 con 0x7f77bc09be20 2026-02-17T05:46:15.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.124+0000 7f7797fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.124+0000 7f7797fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:15.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.161+0000 7fc3cb75e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:46:15.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.161+0000 7fc3cb75e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:46:15.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.161+0000 7fc3cb75e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.161+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc3c409f260 con 0x7fc3c406d380 2026-02-17T05:46:15.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.162+0000 7fc3b1ffb640 1 -- 10.20.193.78:0/3718900810 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3b801b900 con 0x7fc3c406d380 2026-02-17T05:46:15.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.162+0000 7fc3b1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-17T05:46:15.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.162+0000 7fc3b1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:15.163 INFO:teuthology.orchestra.run.trial078.stdout:47244640262 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403e0a0 msgr2=0x7fc394040560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403e0a0 0x7fc394040560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4007d90 tx=0x7fc3b4007ce0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 10 monclient: shutdown 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 msgr2=0x7fc3c4132190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c4132190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c5d0 tx=0x7fc3b800caa0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 shutdown_connections 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c409c4a0 0x7fc3c409db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c406d380 0x7fc3c4132190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403e0a0 0x7fc394040560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 --2- 10.20.193.78:0/3718900810 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c406c890 0x7fc3c413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 >> 10.20.193.78:0/3718900810 conn(0x7fc3c40a6620 msgr2=0x7fc3c40a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 shutdown_connections 2026-02-17T05:46:15.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.163+0000 7fc3cb75e640 1 -- 10.20.193.78:0/3718900810 wait complete. 2026-02-17T05:46:15.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.4 2026-02-17T05:46:15.172 DEBUG:teuthology.parallel:result is None 2026-02-17T05:46:15.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.208+0000 7f77c1fa9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:46:15.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.208+0000 7f77c1fa9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:46:15.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.208+0000 7f77c1fa9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.208+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f77bc06cfc0 con 0x7f77bc09be20 2026-02-17T05:46:15.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.208+0000 7f7797fff640 1 -- 10.20.193.78:0/2362713997 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f77a804adb0 con 0x7f77bc09be20 2026-02-17T05:46:15.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.208+0000 7f7797fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-17T05:46:15.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.208+0000 7f7797fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:15.211 INFO:teuthology.orchestra.run.trial078.stdout:51539607557 2026-02-17T05:46:15.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f778c03dee0 msgr2=0x7f778c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f778c03dee0 0x7f778c0403a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f77bc12aa50 tx=0x7f77ac008040 comp rx=0 tx=0).stop 2026-02-17T05:46:15.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 10 monclient: shutdown 2026-02-17T05:46:15.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 msgr2=0x7f77bc1d3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc1d3b10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77a8002410 tx=0x7f77a8004830 comp rx=0 tx=0).stop 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 shutdown_connections 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77bc13c8d0 0x7f77bc1291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f778c03dee0 0x7f778c0403a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77bc09f110 0x7f77bc1d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 --2- 10.20.193.78:0/2362713997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77bc09be20 0x7f77bc1d3b10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 >> 10.20.193.78:0/2362713997 conn(0x7f77bc0a6620 msgr2=0x7f77bc0a1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 shutdown_connections 2026-02-17T05:46:15.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.210+0000 7f77c1fa9640 1 -- 10.20.193.78:0/2362713997 wait complete. 2026-02-17T05:46:15.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607557 for osd.0 2026-02-17T05:46:15.220 DEBUG:teuthology.parallel:result is None 2026-02-17T05:46:15.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.324+0000 7fc4e77fe640 1 -- 10.20.193.78:0/61631881 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc4fc050ba0 con 0x7fc508139fb0 2026-02-17T05:46:15.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.324+0000 7fc4e77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.324+0000 7fc4e77fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:15.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.413+0000 7fc50d549640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:46:15.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.413+0000 7fc50d549640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:46:15.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.413+0000 7fc50d549640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:15.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.413+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc50809f2c0 con 0x7fc508139fb0 2026-02-17T05:46:15.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.413+0000 7fc4e77fe640 1 -- 10.20.193.78:0/61631881 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4fc01b3b0 con 0x7fc508139fb0 2026-02-17T05:46:15.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.413+0000 7fc4e77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-17T05:46:15.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.413+0000 7fc4e77fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:15.417 INFO:teuthology.orchestra.run.trial078.stdout:51539607557 2026-02-17T05:46:15.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4d403de60 msgr2=0x7fc4d4040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4d403de60 0x7fc4d4040320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0004000 tx=0x7fc4f0033000 comp rx=0 tx=0).stop 2026-02-17T05:46:15.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 10 monclient: shutdown 2026-02-17T05:46:15.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 msgr2=0x7fc50813a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 0x7fc50813a390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00c7b0 tx=0x7fc4fc00cc80 comp rx=0 tx=0).stop 2026-02-17T05:46:15.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 shutdown_connections 2026-02-17T05:46:15.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc5081d3fc0 0x7fc5081d63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4d403de60 0x7fc4d4040320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc50813a8d0 0x7fc50813acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 --2- 10.20.193.78:0/61631881 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc508139fb0 0x7fc50813a390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 >> 10.20.193.78:0/61631881 conn(0x7fc5080a7520 msgr2=0x7fc50813c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 shutdown_connections 2026-02-17T05:46:15.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.414+0000 7fc50d549640 1 -- 10.20.193.78:0/61631881 wait complete. 2026-02-17T05:46:15.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607557 for osd.1 2026-02-17T05:46:15.425 DEBUG:teuthology.parallel:result is None 2026-02-17T05:46:15.426 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-17T05:46:15.426 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T05:46:15.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.492+0000 7f9127c32640 10 monclient: get_monmap_and_config 2026-02-17T05:46:15.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.492+0000 7f9127c32640 10 monclient: build_initial_monmap 2026-02-17T05:46:15.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.492+0000 7f9127c32640 10 monclient: monmap: 2026-02-17T05:46:15.494 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:15.494 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.494 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:15.493914+0000 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:15.493914+0000 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 1 Processor -- start 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 1 -- start start 2026-02-17T05:46:15.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient: init 2026-02-17T05:46:15.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:15.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f912006c890 0x7f912006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient(hunting): picked mon.noname-a con 0x7f912006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f912009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d1b0 0x7f9120135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient(hunting): picked mon.noname-b con 0x7f912006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912006daf0 con 0x7f912009c4a0 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f91200a0880 con 0x7f912006c890 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f912009cc80 con 0x7f912006d1b0 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f9127c32640 10 monclient(hunting): authenticate will time out at 621.047437s 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f912006c890 0x7f912006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d1b0 0x7f9120135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d1b0 0x7f9120135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47980/0 (socket says 10.20.193.78:47980) 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f912006c890 0x7f912006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52868/0 (socket says 10.20.193.78:52868) 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 1 -- 10.20.193.78:0/1096100265 learned_addr learned my addr 10.20.193.78:0/1096100265 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 10 monclient(hunting): get_auth_request con 0x7f912006c890 auth_method 0 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient(hunting): get_auth_request con 0x7f912006d1b0 auth_method 0 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient(hunting): handle_auth_done global_id 4274 payload 274 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 1 -- 10.20.193.78:0/1096100265 >> v2:10.20.193.78:3300/0 conn(0x7f912006c890 msgr2=0x7f912006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 1 --2- 10.20.193.78:0/1096100265 >> v2:10.20.193.78:3300/0 conn(0x7f912006c890 0x7f912006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 1 -- 10.20.193.78:0/1096100265 >> v1:10.20.193.115:6789/0 conn(0x7f912009c4a0 legacy=0x7f9120134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient: _finish_hunting 0 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 1 monclient: found mon.noname-b 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91261a8640 1 -- 10.20.193.78:0/1096100265 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912009cf60 con 0x7f912006d1b0 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.493+0000 7f91259a7640 1 --2- 10.20.193.78:0/1096100265 >> v2:10.20.193.78:3300/0 conn(0x7f912006c890 0x7f912006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91261a8640 1 -- 10.20.193.78:0/1096100265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d1b0 msgr2=0x7f9120135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91261a8640 1 --2- 10.20.193.78:0/1096100265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d1b0 0x7f9120135500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f912006dcc0 tx=0x7f911c02f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b696ebc582490012 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 1 -- 10.20.193.78:0/1096100265 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c0373f0 con 0x7f912006d1b0 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:15.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: dump: 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: _finish_auth 0 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: _check_auth_tickets 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 1 -- 10.20.193.78:0/1096100265 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911c037590 con 0x7f912006d1b0 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 5 monclient: authenticate success, global_id 4274 2026-02-17T05:46:15.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 1 -- 10.20.193.78:0/1096100265 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c037960 con 0x7f912006d1b0 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 4 set_mon_vals no callback set 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f91249a5640 10 monclient: dump: 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 10 monclient: shutdown 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 1 -- 10.20.193.78:0/1096100265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d1b0 msgr2=0x7f9120135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 1 --2- 10.20.193.78:0/1096100265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d1b0 0x7f9120135500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f912006dcc0 tx=0x7f911c02f210 comp rx=0 tx=0).stop 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 1 -- 10.20.193.78:0/1096100265 shutdown_connections 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 1 --2- 10.20.193.78:0/1096100265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d1b0 0x7f9120135500 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 1 --2- 10.20.193.78:0/1096100265 >> v2:10.20.193.78:3300/0 conn(0x7f912006c890 0x7f912006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 1 -- 10.20.193.78:0/1096100265 >> 10.20.193.78:0/1096100265 conn(0x7f91200a6620 msgr2=0x7f91200a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 1 -- 10.20.193.78:0/1096100265 shutdown_connections 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.496+0000 7f9127c32640 1 -- 10.20.193.78:0/1096100265 wait complete. 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient: build_initial_monmap 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient: monmap: 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:15.509 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 1 Processor -- start 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 1 -- start start 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient: init 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006c890 0x7f91201cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient(hunting): picked mon.noname-c con 0x7f912006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006d1b0 0x7f91201cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient(hunting): picked mon.noname-b con 0x7f912006d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f9120122510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient(hunting): picked mon.noname-a con 0x7f912009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91200a01a0 con 0x7f912009c4a0 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f91200a0020 con 0x7f912006d1b0 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f912009be20 con 0x7f912006c890 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f9127c32640 10 monclient(hunting): authenticate will time out at 621.051769s 2026-02-17T05:46:15.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006c890 0x7f91201cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91261a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f9120122510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006c890 0x7f91201cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47996/0 (socket says 10.20.193.78:47996) 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 -- 10.20.193.78:0/2083508462 learned_addr learned my addr 10.20.193.78:0/2083508462 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient(hunting): get_auth_request con 0x7f912006c890 auth_method 0 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91261a8640 10 monclient(hunting): get_auth_request con 0x7f912009c4a0 auth_method 0 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91261a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91261a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91261a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91261a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91261a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91261a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91251a6640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006d1b0 0x7f91201cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91251a6640 10 monclient(hunting): get_auth_request con 0x7f912006d1b0 auth_method 0 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91251a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91251a6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91251a6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient(hunting): handle_auth_done global_id 4277 payload 824 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 -- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006d1b0 msgr2=0x7f91201cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006d1b0 0x7f91201cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 -- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 msgr2=0x7f9120122510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f9120122510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient: _finish_hunting 0 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 monclient: found mon.noname-c 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.497+0000 7f91259a7640 1 -- 10.20.193.78:0/2083508462 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9120124650 con 0x7f912006c890 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.498+0000 7f91261a8640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f9120122510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f91259a7640 1 -- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006c890 msgr2=0x7f91201cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f91259a7640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006c890 0x7f91201cd330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f911000ebe0 tx=0x7f911000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 1 -- 10.20.193.78:0/2083508462 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110017070 con 0x7f912006c890 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: dump: 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 1 -- 10.20.193.78:0/2083508462 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911000ce50 con 0x7f912006c890 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 1 -- 10.20.193.78:0/2083508462 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110013690 con 0x7f912006c890 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:15.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910ffff640 4 set_mon_vals no callback set 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 10 monclient: dump: 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 5 monclient: authenticate success, global_id 4277 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 10 monclient: _renew_subs 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9120124d10 con 0x7f912006c890 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 10 monclient: _renew_subs 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9120136ea0 con 0x7f912006c890 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9120074980 con 0x7f912006c890 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.500+0000 7f910e7fc640 1 -- 10.20.193.78:0/2083508462 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9110024060 con 0x7f912006c890 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.501+0000 7f910e7fc640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f90f003df30 0x7f90f00403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.501+0000 7f910e7fc640 1 -- 10.20.193.78:0/2083508462 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7f91100526d0 con 0x7f912006c890 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.501+0000 7f91251a6640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f90f003df30 0x7f90f00403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.501+0000 7f91251a6640 10 monclient: get_auth_request con 0x7f90f003df30 auth_method 0 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.503+0000 7f910e7fc640 1 -- 10.20.193.78:0/2083508462 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9110013d50 con 0x7f912006c890 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.503+0000 7f910e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.503+0000 7f910e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:15.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.504+0000 7f91251a6640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f90f003df30 0x7f90f00403f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9120123db0 tx=0x7f9114009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.584+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91200744d0 con 0x7f90f003df30 2026-02-17T05:46:15.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.585+0000 7f910e7fc640 1 -- 10.20.193.78:0/2083508462 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34165 (secure 0 0 0) 0x7f91200744d0 con 0x7f90f003df30 2026-02-17T05:46:15.587 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:15.588 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T05:46:15.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.585+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f90f003df30 msgr2=0x7f90f00403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.585+0000 7f9127c32640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f90f003df30 0x7f90f00403f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9120123db0 tx=0x7f9114009040 comp rx=0 tx=0).stop 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.585+0000 7f9127c32640 10 monclient: shutdown 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.585+0000 7f9127c32640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.585+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006c890 msgr2=0x7f91201cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.585+0000 7f9127c32640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006c890 0x7f91201cd330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f911000ebe0 tx=0x7f911000c520 comp rx=0 tx=0).stop 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.586+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 shutdown_connections 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.586+0000 7f9127c32640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f9120122510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.586+0000 7f9127c32640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f90f003df30 0x7f90f00403f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.586+0000 7f9127c32640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006d1b0 0x7f91201cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.586+0000 7f9127c32640 1 --2- 10.20.193.78:0/2083508462 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006c890 0x7f91201cd330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.586+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 >> 10.20.193.78:0/2083508462 conn(0x7f91200a6620 msgr2=0x7f9120136680 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.586+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 shutdown_connections 2026-02-17T05:46:15.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.586+0000 7f9127c32640 1 -- 10.20.193.78:0/2083508462 wait complete. 2026-02-17T05:46:15.595 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-17T05:46:15.199961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167196,"kb_used_data":1220,"kb_used_omap":9,"kb_used_meta":165814,"kb_avail":566063844,"statfs":{"total":579820584960,"available":579649376256,"internally_reserved":0,"allocated":1249280,"data_stored":1432448,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":169794214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.290162"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183122+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T05:46:12.183122+0000","last_peered":"2026-02-17T05:46:12.183122+0000","last_clean":"2026-02-17T05:46:12.183122+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T05:46:12.183122+0000","last_undegraded":"2026-02-17T05:46:12.183122+0000","last_fullsized":"2026-02-17T05:46:12.183122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.184594+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T05:46:12.184594+0000","last_peered":"2026-02-17T05:46:12.184594+0000","last_clean":"2026-02-17T05:46:12.184594+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T05:46:12.184594+0000","last_undegraded":"2026-02-17T05:46:12.184594+0000","last_fullsized":"2026-02-17T05:46:12.184594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183276+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T05:46:12.183276+0000","last_peered":"2026-02-17T05:46:12.183276+0000","last_clean":"2026-02-17T05:46:12.183276+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T05:46:12.183276+0000","last_undegraded":"2026-02-17T05:46:12.183276+0000","last_fullsized":"2026-02-17T05:46:12.183276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.217637+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T05:46:12.217637+0000","last_peered":"2026-02-17T05:46:12.217637+0000","last_clean":"2026-02-17T05:46:12.217637+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T05:46:12.217637+0000","last_undegraded":"2026-02-17T05:46:12.217637+0000","last_fullsized":"2026-02-17T05:46:12.217637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.184443+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T05:46:12.184443+0000","last_peered":"2026-02-17T05:46:12.184443+0000","last_clean":"2026-02-17T05:46:12.184443+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T05:46:12.184443+0000","last_undegraded":"2026-02-17T05:46:12.184443+0000","last_fullsized":"2026-02-17T05:46:12.184443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.182358+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T05:46:12.182358+0000","last_peered":"2026-02-17T05:46:12.182358+0000","last_clean":"2026-02-17T05:46:12.182358+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T05:46:12.182358+0000","last_undegraded":"2026-02-17T05:46:12.182358+0000","last_fullsized":"2026-02-17T05:46:12.182358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183042+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T05:46:12.183042+0000","last_peered":"2026-02-17T05:46:12.183042+0000","last_clean":"2026-02-17T05:46:12.183042+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T05:46:12.183042+0000","last_undegraded":"2026-02-17T05:46:12.183042+0000","last_fullsized":"2026-02-17T05:46:12.183042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183095+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T05:46:12.183095+0000","last_peered":"2026-02-17T05:46:12.183095+0000","last_clean":"2026-02-17T05:46:12.183095+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T05:46:12.183095+0000","last_undegraded":"2026-02-17T05:46:12.183095+0000","last_fullsized":"2026-02-17T05:46:12.183095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'112","reported_seq":156,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.182832+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T05:46:12.182832+0000","last_peered":"2026-02-17T05:46:12.182832+0000","last_clean":"2026-02-17T05:46:12.182832+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T05:46:12.182832+0000","last_undegraded":"2026-02-17T05:46:12.182832+0000","last_fullsized":"2026-02-17T05:46:12.182832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":249856,"data_stored":592181,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":192512,"data_stored":63272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":176128,"data_stored":58289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":188416,"data_stored":63253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":249856,"data_stored":592181,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":192512,"data_stored":63272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T05:46:15.596 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.661+0000 7f976223b640 10 monclient: get_monmap_and_config 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.661+0000 7f976223b640 10 monclient: build_initial_monmap 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.661+0000 7f976223b640 10 monclient: monmap: 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:15.663174+0000 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:15.663174+0000 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:15.663 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 1 Processor -- start 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 1 -- start start 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient: init 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f975c06c890 0x7f975c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient(hunting): picked mon.noname-a con 0x7f975c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c06cc70 0x7f975c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient(hunting): picked mon.noname-b con 0x7f975c06cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f975c134df0 0x7f975c135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient(hunting): picked mon.noname-c con 0x7f975c134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f975c074150 con 0x7f975c06c890 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f975c09ef70 con 0x7f975c134df0 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f975c09f250 con 0x7f975c06cc70 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f976223b640 10 monclient(hunting): authenticate will time out at 621.216712s 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f975c06c890 0x7f975c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f975c06c890 0x7f975c0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52884/0 (socket says 10.20.193.78:52884) 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 1 -- 10.20.193.78:0/1623778570 learned_addr learned my addr 10.20.193.78:0/1623778570 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 10 monclient(hunting): get_auth_request con 0x7f975c06c890 auth_method 0 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c06cc70 0x7f975c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975affd640 10 monclient(hunting): get_auth_request con 0x7f975c06cc70 auth_method 0 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975bfff640 1 --2- 10.20.193.78:0/1623778570 >> v2:10.20.193.115:3300/0 conn(0x7f975c134df0 0x7f975c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975bfff640 10 monclient(hunting): get_auth_request con 0x7f975c134df0 auth_method 0 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 10 monclient(hunting): handle_auth_done global_id 4299 payload 274 2026-02-17T05:46:15.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.662+0000 7f975b7fe640 1 -- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c06cc70 msgr2=0x7f975c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975b7fe640 1 --2- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c06cc70 0x7f975c06d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975b7fe640 1 -- 10.20.193.78:0/1623778570 >> v2:10.20.193.115:3300/0 conn(0x7f975c134df0 msgr2=0x7f975c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975b7fe640 1 --2- 10.20.193.78:0/1623778570 >> v2:10.20.193.115:3300/0 conn(0x7f975c134df0 0x7f975c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975b7fe640 1 monclient: found mon.noname-a 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975b7fe640 1 -- 10.20.193.78:0/1623778570 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c09df40 con 0x7f975c06c890 2026-02-17T05:46:15.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.663+0000 7f975affd640 1 --2- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c06cc70 0x7f975c06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975b7fe640 1 -- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06c890 msgr2=0x7f975c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975b7fe640 1 --2- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06c890 0x7f975c0a01f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f975c06e270 tx=0x7f9748031220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581457e8dedf054c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 1 -- 10.20.193.78:0/1623778570 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974803e070 con 0x7f975c06c890 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: dump: 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.667 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 5 monclient: authenticate success, global_id 4299 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 1 -- 10.20.193.78:0/1623778570 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9748039640 con 0x7f975c06c890 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 1 -- 10.20.193.78:0/1623778570 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748031e60 con 0x7f975c06c890 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 4 set_mon_vals no callback set 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f975a7fc640 10 monclient: dump: 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.668 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 10 monclient: shutdown 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 1 -- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06c890 msgr2=0x7f975c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 1 --2- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06c890 0x7f975c0a01f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f975c06e270 tx=0x7f9748031220 comp rx=0 tx=0).stop 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 1 -- 10.20.193.78:0/1623778570 shutdown_connections 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 1 --2- 10.20.193.78:0/1623778570 >> v2:10.20.193.115:3300/0 conn(0x7f975c134df0 0x7f975c135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 1 --2- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c06cc70 0x7f975c06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 1 --2- 10.20.193.78:0/1623778570 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06c890 0x7f975c0a01f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 1 -- 10.20.193.78:0/1623778570 >> 10.20.193.78:0/1623778570 conn(0x7f975c0a6620 msgr2=0x7f975c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.665+0000 7f976223b640 1 -- 10.20.193.78:0/1623778570 shutdown_connections 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 -- 10.20.193.78:0/1623778570 wait complete. 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient: build_initial_monmap 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient: monmap: 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:15.669 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 Processor -- start 2026-02-17T05:46:15.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 -- start start 2026-02-17T05:46:15.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient: init 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f975c06c890 0x7f975c133f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient(hunting): picked mon.noname-b con 0x7f975c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06cc70 0x7f975c1207c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient(hunting): picked mon.noname-a con 0x7f975c06cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c134df0 0x7f975c120d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient(hunting): picked mon.noname-c con 0x7f975c134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f975c09d360 con 0x7f975c06cc70 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f975c09d1e0 con 0x7f975c06c890 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f975c135780 con 0x7f975c134df0 2026-02-17T05:46:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f976223b640 10 monclient(hunting): authenticate will time out at 621.220834s 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c134df0 0x7f975c120d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06cc70 0x7f975c1207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c134df0 0x7f975c120d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48006/0 (socket says 10.20.193.78:48006) 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 1 -- 10.20.193.78:0/875602137 learned_addr learned my addr 10.20.193.78:0/875602137 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975affd640 10 monclient(hunting): get_auth_request con 0x7f975c06cc70 auth_method 0 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 10 monclient(hunting): get_auth_request con 0x7f975c134df0 auth_method 0 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.666+0000 7f975bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975b7fe640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f975c06c890 0x7f975c133f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975b7fe640 10 monclient(hunting): get_auth_request con 0x7f975c06c890 auth_method 0 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 10 monclient(hunting): handle_auth_done global_id 4302 payload 824 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 1 -- 10.20.193.78:0/875602137 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c134df0 msgr2=0x7f975c120d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c134df0 0x7f975c120d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 1 -- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f975c06c890 msgr2=0x7f975c133f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f975c06c890 0x7f975c133f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 10 monclient: _finish_hunting 0 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 1 monclient: found mon.noname-a 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.667+0000 7f975affd640 1 -- 10.20.193.78:0/875602137 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c1bd330 con 0x7f975c06cc70 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f975affd640 1 -- 10.20.193.78:0/875602137 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06cc70 msgr2=0x7f975c1207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f975affd640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06cc70 0x7f975c1207c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f974c00ebe0 tx=0x7f974c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 1 -- 10.20.193.78:0/875602137 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c017070 con 0x7f975c06cc70 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: dump: 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: _finish_auth 0 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 5 monclient: authenticate success, global_id 4302 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 1 -- 10.20.193.78:0/875602137 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974c00ce50 con 0x7f975c06cc70 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 1 -- 10.20.193.78:0/875602137 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c013690 con 0x7f975c06cc70 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9759ffb640 4 set_mon_vals no callback set 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f9737fff640 10 monclient: dump: 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 10 monclient: _renew_subs 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c1228b0 con 0x7f975c06cc70 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 10 monclient: _renew_subs 2026-02-17T05:46:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c0a3900 con 0x7f975c06cc70 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.669+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c074980 con 0x7f975c06cc70 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.670+0000 7f9737fff640 1 -- 10.20.193.78:0/875602137 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f974c024060 con 0x7f975c06cc70 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.670+0000 7f9737fff640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f972c03dee0 0x7f972c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.670+0000 7f9737fff640 1 -- 10.20.193.78:0/875602137 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7f974c052600 con 0x7f975c06cc70 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.670+0000 7f975b7fe640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f972c03dee0 0x7f972c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.670+0000 7f975b7fe640 10 monclient: get_auth_request con 0x7f972c03dee0 auth_method 0 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.672+0000 7f9737fff640 1 -- 10.20.193.78:0/875602137 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f975c1bd330 con 0x7f975c06cc70 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.672+0000 7f9737fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.672+0000 7f9737fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.673+0000 7f975b7fe640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f972c03dee0 0x7f972c0403a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9748038000 tx=0x7f974803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.752+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f975c072e20 con 0x7f972c03dee0 2026-02-17T05:46:15.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f9737fff640 1 -- 10.20.193.78:0/875602137 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34165 (secure 0 0 0) 0x7f975c072e20 con 0x7f972c03dee0 2026-02-17T05:46:15.756 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:15.757 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T05:46:15.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f972c03dee0 msgr2=0x7f972c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f972c03dee0 0x7f972c0403a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9748038000 tx=0x7f974803b040 comp rx=0 tx=0).stop 2026-02-17T05:46:15.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 10 monclient: shutdown 2026-02-17T05:46:15.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06cc70 msgr2=0x7f975c1207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06cc70 0x7f975c1207c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f974c00ebe0 tx=0x7f974c00c520 comp rx=0 tx=0).stop 2026-02-17T05:46:15.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 shutdown_connections 2026-02-17T05:46:15.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f975c134df0 0x7f975c120d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f975c06cc70 0x7f975c1207c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f972c03dee0 0x7f972c0403a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 --2- 10.20.193.78:0/875602137 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f975c06c890 0x7f975c133f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 >> 10.20.193.78:0/875602137 conn(0x7f975c0a6620 msgr2=0x7f975c0732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 shutdown_connections 2026-02-17T05:46:15.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.754+0000 7f976223b640 1 -- 10.20.193.78:0/875602137 wait complete. 2026-02-17T05:46:15.763 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-17T05:46:15.199961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167196,"kb_used_data":1220,"kb_used_omap":9,"kb_used_meta":165814,"kb_avail":566063844,"statfs":{"total":579820584960,"available":579649376256,"internally_reserved":0,"allocated":1249280,"data_stored":1432448,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":169794214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.290162"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183122+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T05:46:12.183122+0000","last_peered":"2026-02-17T05:46:12.183122+0000","last_clean":"2026-02-17T05:46:12.183122+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T05:46:12.183122+0000","last_undegraded":"2026-02-17T05:46:12.183122+0000","last_fullsized":"2026-02-17T05:46:12.183122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.184594+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T05:46:12.184594+0000","last_peered":"2026-02-17T05:46:12.184594+0000","last_clean":"2026-02-17T05:46:12.184594+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T05:46:12.184594+0000","last_undegraded":"2026-02-17T05:46:12.184594+0000","last_fullsized":"2026-02-17T05:46:12.184594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183276+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T05:46:12.183276+0000","last_peered":"2026-02-17T05:46:12.183276+0000","last_clean":"2026-02-17T05:46:12.183276+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T05:46:12.183276+0000","last_undegraded":"2026-02-17T05:46:12.183276+0000","last_fullsized":"2026-02-17T05:46:12.183276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.217637+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T05:46:12.217637+0000","last_peered":"2026-02-17T05:46:12.217637+0000","last_clean":"2026-02-17T05:46:12.217637+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T05:46:12.217637+0000","last_undegraded":"2026-02-17T05:46:12.217637+0000","last_fullsized":"2026-02-17T05:46:12.217637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.184443+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T05:46:12.184443+0000","last_peered":"2026-02-17T05:46:12.184443+0000","last_clean":"2026-02-17T05:46:12.184443+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T05:46:12.184443+0000","last_undegraded":"2026-02-17T05:46:12.184443+0000","last_fullsized":"2026-02-17T05:46:12.184443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.182358+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T05:46:12.182358+0000","last_peered":"2026-02-17T05:46:12.182358+0000","last_clean":"2026-02-17T05:46:12.182358+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T05:46:12.182358+0000","last_undegraded":"2026-02-17T05:46:12.182358+0000","last_fullsized":"2026-02-17T05:46:12.182358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183042+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T05:46:12.183042+0000","last_peered":"2026-02-17T05:46:12.183042+0000","last_clean":"2026-02-17T05:46:12.183042+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T05:46:12.183042+0000","last_undegraded":"2026-02-17T05:46:12.183042+0000","last_fullsized":"2026-02-17T05:46:12.183042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183095+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T05:46:12.183095+0000","last_peered":"2026-02-17T05:46:12.183095+0000","last_clean":"2026-02-17T05:46:12.183095+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T05:46:12.183095+0000","last_undegraded":"2026-02-17T05:46:12.183095+0000","last_fullsized":"2026-02-17T05:46:12.183095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'112","reported_seq":156,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.182832+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T05:46:12.182832+0000","last_peered":"2026-02-17T05:46:12.182832+0000","last_clean":"2026-02-17T05:46:12.182832+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T05:46:12.182832+0000","last_undegraded":"2026-02-17T05:46:12.182832+0000","last_fullsized":"2026-02-17T05:46:12.182832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":249856,"data_stored":592181,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":192512,"data_stored":63272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":176128,"data_stored":58289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":188416,"data_stored":63253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":249856,"data_stored":592181,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":192512,"data_stored":63272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T05:46:15.765 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-17T05:46:15.765 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-17T05:46:15.765 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-17T05:46:15.765 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T05:46:15.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.829+0000 7fed1ef53640 10 monclient: get_monmap_and_config 2026-02-17T05:46:15.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.829+0000 7fed1ef53640 10 monclient: build_initial_monmap 2026-02-17T05:46:15.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.829+0000 7fed1ef53640 10 monclient: monmap: 2026-02-17T05:46:15.831 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:15.831 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.831 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:15.830699+0000 2026-02-17T05:46:15.831 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:15.830699+0000 2026-02-17T05:46:15.831 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:15.832 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.832 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:15.832 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:15.832 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:15.832 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:15.832 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:15.832 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 1 Processor -- start 2026-02-17T05:46:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 1 -- start start 2026-02-17T05:46:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient: init 2026-02-17T05:46:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2026-02-17T05:46:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fed1809ea90 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fed1809ee70 0x7fed1809f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient(hunting): picked mon.noname-a con 0x7fed1809ee70 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fed1813d640 0x7fed1813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient(hunting): picked mon.noname-c con 0x7fed1813d640 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed180a0a20 con 0x7fed1809ea90 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fed180a0d00 con 0x7fed1809ee70 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fed180a0460 con 0x7fed1813d640 2026-02-17T05:46:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ef53640 10 monclient(hunting): authenticate will time out at 621.384434s 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fed1809ee70 0x7fed1809f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ccc8640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fed1809ea90 0x7fed180a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41248/0 (socket says 10.20.193.78:41248) 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1ccc8640 1 -- 10.20.193.78:0/2288686163 learned_addr learned my addr 10.20.193.78:0/2288686163 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient(hunting): get_auth_request con 0x7fed1809ee70 auth_method 0 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1d4c9640 1 --2- 10.20.193.78:0/2288686163 >> v2:10.20.193.115:3300/0 conn(0x7fed1813d640 0x7fed1813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1d4c9640 10 monclient(hunting): get_auth_request con 0x7fed1813d640 auth_method 0 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1d4c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1d4c9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed1d4c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient(hunting): handle_auth_done global_id 4305 payload 274 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 1 -- 10.20.193.78:0/2288686163 >> v2:10.20.193.115:3300/0 conn(0x7fed1813d640 msgr2=0x7fed1813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 1 --2- 10.20.193.78:0/2288686163 >> v2:10.20.193.115:3300/0 conn(0x7fed1813d640 0x7fed1813fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 1 -- 10.20.193.78:0/2288686163 >> v1:10.20.193.78:6789/0 conn(0x7fed1809ea90 legacy=0x7fed180a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 1 monclient: found mon.noname-a 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.830+0000 7fed0ffff640 1 -- 10.20.193.78:0/2288686163 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed180a0740 con 0x7fed1809ee70 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0ffff640 1 -- 10.20.193.78:0/2288686163 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ee70 msgr2=0x7fed1809f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0ffff640 1 --2- 10.20.193.78:0/2288686163 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ee70 0x7fed1809f2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed18074930 tx=0x7fed0002ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4933cddc2b526747 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 1 -- 10.20.193.78:0/2288686163 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0003c070 con 0x7fed1809ee70 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: dump: 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.845 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 1 -- 10.20.193.78:0/2288686163 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0002fbd0 con 0x7fed1809ee70 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 5 monclient: authenticate success, global_id 4305 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 1 -- 10.20.193.78:0/2288686163 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00037440 con 0x7fed1809ee70 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 4 set_mon_vals no callback set 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed0f7fe640 10 monclient: dump: 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.847 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 10 monclient: shutdown 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 1 -- 10.20.193.78:0/2288686163 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ee70 msgr2=0x7fed1809f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2288686163 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ee70 0x7fed1809f2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed18074930 tx=0x7fed0002ed50 comp rx=0 tx=0).stop 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 1 -- 10.20.193.78:0/2288686163 shutdown_connections 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2288686163 >> v2:10.20.193.115:3300/0 conn(0x7fed1813d640 0x7fed1813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2288686163 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ee70 0x7fed1809f2c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 1 -- 10.20.193.78:0/2288686163 >> 10.20.193.78:0/2288686163 conn(0x7fed180a7520 msgr2=0x7fed180a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 1 -- 10.20.193.78:0/2288686163 shutdown_connections 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.833+0000 7fed1ef53640 1 -- 10.20.193.78:0/2288686163 wait complete. 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient: build_initial_monmap 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient: monmap: 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 1 Processor -- start 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 1 -- start start 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient: init 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ea90 0x7fed180a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient(hunting): picked mon.noname-a con 0x7fed1809ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fed1809ee70 0x7fed180a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient(hunting): picked mon.noname-c con 0x7fed1809ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fed1813d640 0x7fed180a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient(hunting): picked mon.noname-b con 0x7fed1813d640 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed180a6820 con 0x7fed1809ea90 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fed180a66a0 con 0x7fed1813d640 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fed1813ff70 con 0x7fed1809ee70 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ef53640 10 monclient(hunting): authenticate will time out at 621.388652s 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ea90 0x7fed180a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ea90 0x7fed180a30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52920/0 (socket says 10.20.193.78:52920) 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 -- 10.20.193.78:0/2711948669 learned_addr learned my addr 10.20.193.78:0/2711948669 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient(hunting): get_auth_request con 0x7fed1809ea90 auth_method 0 2026-02-17T05:46:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fed1809ee70 0x7fed180a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 10 monclient(hunting): get_auth_request con 0x7fed1809ee70 auth_method 0 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1d4c9640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fed1813d640 0x7fed180a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1d4c9640 10 monclient(hunting): get_auth_request con 0x7fed1813d640 auth_method 0 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1d4c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1d4c9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1d4c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient(hunting): handle_auth_done global_id 4308 payload 824 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 -- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fed1809ee70 msgr2=0x7fed180a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fed1809ee70 0x7fed180a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 -- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fed1813d640 msgr2=0x7fed180a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fed1813d640 0x7fed180a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient: _finish_hunting 0 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 monclient: found mon.noname-a 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed1ccc8640 1 -- 10.20.193.78:0/2711948669 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed180a9ce0 con 0x7fed1809ea90 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.834+0000 7fed0ffff640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fed1809ee70 0x7fed180a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ccc8640 1 -- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ea90 msgr2=0x7fed180a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ccc8640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ea90 0x7fed180a30a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c750 tx=0x7fed0800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 1 -- 10.20.193.78:0/2711948669 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08005de0 con 0x7fed1809ea90 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: dump: 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.854 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 1 -- 10.20.193.78:0/2711948669 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed08007070 con 0x7fed1809ea90 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 5 monclient: authenticate success, global_id 4308 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:15.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 1 -- 10.20.193.78:0/2711948669 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08007440 con 0x7fed1809ea90 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0effd640 4 set_mon_vals no callback set 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 10 monclient: dump: 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 10 monclient: _renew_subs 2026-02-17T05:46:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1813fcf0 con 0x7fed1809ea90 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 10 monclient: _renew_subs 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed180a4c40 con 0x7fed1809ea90 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1809f2c0 con 0x7fed1809ea90 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.837+0000 7fed0d7fa640 1 -- 10.20.193.78:0/2711948669 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fed08007620 con 0x7fed1809ea90 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.838+0000 7fed0d7fa640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fece803df00 0x7fece80403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.838+0000 7fed0d7fa640 1 -- 10.20.193.78:0/2711948669 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7fed08052390 con 0x7fed1809ea90 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.838+0000 7fed0ffff640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fece803df00 0x7fece80403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.838+0000 7fed0ffff640 10 monclient: get_auth_request con 0x7fece803df00 auth_method 0 2026-02-17T05:46:15.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.840+0000 7fed0d7fa640 1 -- 10.20.193.78:0/2711948669 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fed0800a870 con 0x7fed1809ea90 2026-02-17T05:46:15.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.840+0000 7fed0d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:15.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.840+0000 7fed0d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:15.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.840+0000 7fed0ffff640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fece803df00 0x7fece80403c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fed1806ecd0 tx=0x7fed00033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:15.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.943+0000 7fed1ef53640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-17T05:46:15.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.943+0000 7fed1ef53640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:46:15.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.943+0000 7fed1ef53640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:15.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.943+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fed180707e0 con 0x7fed1809ea90 2026-02-17T05:46:15.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.943+0000 7fed0d7fa640 1 -- 10.20.193.78:0/2711948669 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fed080520d0 con 0x7fed1809ea90 2026-02-17T05:46:15.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.943+0000 7fed0d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-17T05:46:15.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.943+0000 7fed0d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:15.946 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:15.947 INFO:teuthology.orchestra.run.trial078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fece803df00 msgr2=0x7fece80403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fece803df00 0x7fece80403c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fed1806ecd0 tx=0x7fed00033000 comp rx=0 tx=0).stop 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 10 monclient: shutdown 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ea90 msgr2=0x7fed180a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ea90 0x7fed180a30a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c750 tx=0x7fed0800cc20 comp rx=0 tx=0).stop 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 shutdown_connections 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fece803df00 0x7fece80403c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fed1813d640 0x7fed180a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fed1809ee70 0x7fed180a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 --2- 10.20.193.78:0/2711948669 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fed1809ea90 0x7fed180a30a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 >> 10.20.193.78:0/2711948669 conn(0x7fed180a7520 msgr2=0x7fed180aa230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 shutdown_connections 2026-02-17T05:46:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:15.944+0000 7fed1ef53640 1 -- 10.20.193.78:0/2711948669 wait complete. 2026-02-17T05:46:15.953 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-17T05:46:15.954 INFO:teuthology.run_tasks:Running task mon_thrash... 2026-02-17T05:46:15.965 INFO:tasks.mon_thrash:Beginning mon_thrash... 2026-02-17T05:46:15.965 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T05:46:16.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.031+0000 7fd66c733640 10 monclient: get_monmap_and_config 2026-02-17T05:46:16.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.031+0000 7fd66c733640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.031+0000 7fd66c733640 10 monclient: monmap: 2026-02-17T05:46:16.036 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.036 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.036 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:16.033050+0000 2026-02-17T05:46:16.036 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:16.033050+0000 2026-02-17T05:46:16.036 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 1 Processor -- start 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 1 -- start start 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient: init 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd66406c890 0x7fd66406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient(hunting): picked mon.noname-a con 0x7fd66406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd66406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient(hunting): picked mon.noname-b con 0x7fd66409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd664134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd66406daf0 con 0x7fd664134df0 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd6640a0880 con 0x7fd66406c890 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd66406d6f0 con 0x7fd66409c4a0 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66c733640 10 monclient(hunting): authenticate will time out at 621.586676s 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd66406c890 0x7fd66406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd66406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd66406c890 0x7fd66406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52924/0 (socket says 10.20.193.78:52924) 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 1 -- 10.20.193.78:0/2282138084 learned_addr learned my addr 10.20.193.78:0/2282138084 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient(hunting): get_auth_request con 0x7fd66406c890 auth_method 0 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 10 monclient(hunting): get_auth_request con 0x7fd66409c4a0 auth_method 0 2026-02-17T05:46:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient(hunting): handle_auth_done global_id 4311 payload 274 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 1 -- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 msgr2=0x7fd66406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 1 --2- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd66406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 1 -- 10.20.193.78:0/2282138084 >> v1:10.20.193.78:6789/0 conn(0x7fd664134df0 legacy=0x7fd664135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 1 monclient: found mon.noname-a 2026-02-17T05:46:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd66a4a8640 1 -- 10.20.193.78:0/2282138084 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66409cc80 con 0x7fd66406c890 2026-02-17T05:46:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.032+0000 7fd669ca7640 1 --2- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd66406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66a4a8640 1 -- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 msgr2=0x7fd66406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66a4a8640 1 --2- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd66406cc70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd66406d870 tx=0x7fd65402ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce09102e31e99b20 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 1 -- 10.20.193.78:0/2282138084 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65403c070 con 0x7fd66406c890 2026-02-17T05:46:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: dump: 2026-02-17T05:46:16.047 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 1 -- 10.20.193.78:0/2282138084 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65402fc60 con 0x7fd66406c890 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 5 monclient: authenticate success, global_id 4311 2026-02-17T05:46:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 1 -- 10.20.193.78:0/2282138084 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654037660 con 0x7fd66406c890 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd6694a6640 10 monclient: dump: 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.049 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 4 set_mon_vals no callback set 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 10 monclient: shutdown 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 1 -- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 msgr2=0x7fd66406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 1 --2- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd66406cc70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd66406d870 tx=0x7fd65402ee20 comp rx=0 tx=0).stop 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 1 -- 10.20.193.78:0/2282138084 shutdown_connections 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 1 --2- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd66406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 1 --2- 10.20.193.78:0/2282138084 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd66406cc70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 1 -- 10.20.193.78:0/2282138084 >> 10.20.193.78:0/2282138084 conn(0x7fd6640a6620 msgr2=0x7fd6640a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 1 -- 10.20.193.78:0/2282138084 shutdown_connections 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.035+0000 7fd66c733640 1 -- 10.20.193.78:0/2282138084 wait complete. 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient: monmap: 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 1 Processor -- start 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 1 -- start start 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient: init 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd6641cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient(hunting): picked mon.noname-a con 0x7fd66406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd664133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient(hunting): picked mon.noname-c con 0x7fd66409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 0x7fd664134380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient(hunting): picked mon.noname-b con 0x7fd664134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6640a01a0 con 0x7fd66406c890 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6640a0020 con 0x7fd664134df0 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd66409be20 con 0x7fd66409c4a0 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66c733640 10 monclient(hunting): authenticate will time out at 621.590752s 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd6641cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd664133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd6641cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52930/0 (socket says 10.20.193.78:52930) 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 1 -- 10.20.193.78:0/2072619252 learned_addr learned my addr 10.20.193.78:0/2072619252 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 10 monclient(hunting): get_auth_request con 0x7fd66406c890 auth_method 0 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient(hunting): get_auth_request con 0x7fd66409c4a0 auth_method 0 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient(hunting): handle_auth_done global_id 4292 payload 824 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 1 -- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 msgr2=0x7fd664134380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66aca9640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 0x7fd664134380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66aca9640 1 -- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 msgr2=0x7fd664134380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66aca9640 1 -- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 msgr2=0x7fd664134380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66aca9640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 0x7fd664134380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66aca9640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 0x7fd664134380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 0x7fd664134380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 1 -- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 msgr2=0x7fd6641cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd6641cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 1 monclient: found mon.noname-c 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd669ca7640 1 -- 10.20.193.78:0/2072619252 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66409f7f0 con 0x7fd66409c4a0 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.036+0000 7fd66a4a8640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd6641cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd669ca7640 1 -- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 msgr2=0x7fd664133e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd669ca7640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd664133e40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd65800ebe0 tx=0x7fd65800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 1 -- 10.20.193.78:0/2072619252 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658017070 con 0x7fd66409c4a0 2026-02-17T05:46:16.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: dump: 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 1 -- 10.20.193.78:0/2072619252 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65800ce50 con 0x7fd66409c4a0 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 1 -- 10.20.193.78:0/2072619252 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658013690 con 0x7fd66409c4a0 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 10 monclient: dump: 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd668ca5640 4 set_mon_vals no callback set 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 5 monclient: authenticate success, global_id 4292 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 10 monclient: _renew_subs 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6640a1780 con 0x7fd66409c4a0 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 10 monclient: _renew_subs 2026-02-17T05:46:16.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd664127860 con 0x7fd66409c4a0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd64affd640 1 -- 10.20.193.78:0/2072619252 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd658024060 con 0x7fd66409c4a0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.039+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd664074980 con 0x7fd66409c4a0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.040+0000 7fd64affd640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd63403deb0 0x7fd634040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.040+0000 7fd64affd640 1 -- 10.20.193.78:0/2072619252 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7fd658052670 con 0x7fd66409c4a0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.040+0000 7fd66a4a8640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd63403deb0 0x7fd634040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.040+0000 7fd66a4a8640 10 monclient: get_auth_request con 0x7fd63403deb0 auth_method 0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.042+0000 7fd64affd640 1 -- 10.20.193.78:0/2072619252 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd658051200 con 0x7fd66409c4a0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.042+0000 7fd64affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.042+0000 7fd64affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:16.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.042+0000 7fd66a4a8640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd63403deb0 0x7fd634040370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd654009730 tx=0x7fd654033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.127+0000 7fd66c733640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:16.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.127+0000 7fd66c733640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:16.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.127+0000 7fd66c733640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.127+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd664135a70 con 0x7fd66409c4a0 2026-02-17T05:46:16.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.127+0000 7fd64affd640 1 -- 10.20.193.78:0/2072619252 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+12360 (secure 0 0 0) 0x7fd658013dd0 con 0x7fd66409c4a0 2026-02-17T05:46:16.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.127+0000 7fd64affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T05:46:16.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.127+0000 7fd64affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:16.130 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:16.131 INFO:teuthology.orchestra.run.trial078.stdout:{"epoch":21,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T05:46:15.199168+0000","last_up_change":"2026-02-17T05:46:02.153660+0000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:46:03.931310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T05:46:08.458376+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6800","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6801","nonce":3807268031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6802","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6803","nonce":3807268031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6806","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6807","nonce":3807268031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6804","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6805","nonce":3807268031}]},"public_addr":"10.20.193.78:6801/3807268031","cluster_addr":"10.20.193.78:6803/3807268031","heartbeat_back_addr":"10.20.193.78:6807/3807268031","heartbeat_front_addr":"10.20.193.78:6805/3807268031","state":["exists","up"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6808","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6809","nonce":1094981820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6810","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6811","nonce":1094981820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6814","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6815","nonce":1094981820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6812","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6813","nonce":1094981820}]},"public_addr":"10.20.193.78:6809/1094981820","cluster_addr":"10.20.193.78:6811/1094981820","heartbeat_back_addr":"10.20.193.78:6815/1094981820","heartbeat_front_addr":"10.20.193.78:6813/1094981820","state":["exists","up"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6816","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6817","nonce":707008561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6818","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6819","nonce":707008561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6822","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6823","nonce":707008561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6820","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6821","nonce":707008561}]},"public_addr":"10.20.193.78:6817/707008561","cluster_addr":"10.20.193.78:6819/707008561","heartbeat_back_addr":"10.20.193.78:6823/707008561","heartbeat_front_addr":"10.20.193.78:6821/707008561","state":["exists","up"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6810","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6812","nonce":2268846355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6814","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6816","nonce":2268846355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6822","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6824","nonce":2268846355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6818","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6820","nonce":2268846355}]},"public_addr":"10.20.193.115:6812/2268846355","cluster_addr":"10.20.193.115:6816/2268846355","heartbeat_back_addr":"10.20.193.115:6824/2268846355","heartbeat_front_addr":"10.20.193.115:6820/2268846355","state":["exists","up"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6802","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6803","nonce":2725813951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6804","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6805","nonce":2725813951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6808","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6809","nonce":2725813951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6806","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6807","nonce":2725813951}]},"public_addr":"10.20.193.115:6803/2725813951","cluster_addr":"10.20.193.115:6805/2725813951","heartbeat_back_addr":"10.20.193.115:6809/2725813951","heartbeat_front_addr":"10.20.193.115:6807/2725813951","state":["exists","up"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6811","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6813","nonce":2171295268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6815","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6817","nonce":2171295268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6823","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6825","nonce":2171295268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6819","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6821","nonce":2171295268}]},"public_addr":"10.20.193.115:6813/2171295268","cluster_addr":"10.20.193.115:6817/2171295268","heartbeat_back_addr":"10.20.193.115:6825/2171295268","heartbeat_front_addr":"10.20.193.115:6821/2171295268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.280056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.641140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.705253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:01.037254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.075730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.553045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T05:46:16.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd63403deb0 msgr2=0x7fd634040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd63403deb0 0x7fd634040370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd654009730 tx=0x7fd654033000 comp rx=0 tx=0).stop 2026-02-17T05:46:16.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 10 monclient: shutdown 2026-02-17T05:46:16.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 msgr2=0x7fd664133e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd664133e40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd65800ebe0 tx=0x7fd65800c520 comp rx=0 tx=0).stop 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 shutdown_connections 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd63403deb0 0x7fd634040370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd664134df0 0x7fd664134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd66409c4a0 0x7fd664133e40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 --2- 10.20.193.78:0/2072619252 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd66406c890 0x7fd6641cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 >> 10.20.193.78:0/2072619252 conn(0x7fd6640a6620 msgr2=0x7fd664071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 shutdown_connections 2026-02-17T05:46:16.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.128+0000 7fd66c733640 1 -- 10.20.193.78:0/2072619252 wait complete. 2026-02-17T05:46:16.139 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-17T05:46:03.931310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2026-02-17T05:46:08.458376+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-17T05:46:16.139 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2026-02-17T05:46:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.207+0000 7f1cc6a40640 10 monclient: get_monmap_and_config 2026-02-17T05:46:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.207+0000 7f1cc6a40640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.207+0000 7f1cc6a40640 10 monclient: monmap: 2026-02-17T05:46:16.209 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:16.209030+0000 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:16.209030+0000 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:16.210 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 1 Processor -- start 2026-02-17T05:46:16.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 1 -- start start 2026-02-17T05:46:16.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient: init 2026-02-17T05:46:16.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2026-02-17T05:46:16.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f1cc01382f0 0x7f1cc01386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient(hunting): picked mon.noname-c con 0x7f1cc01382f0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:16.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc0138c10 0x7f1cc00a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient(hunting): picked mon.noname-b con 0x7f1cc0138c10 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1cc00a5a70 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cc00a3540 con 0x7f1cc00a5a70 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f1cc00a5000 con 0x7f1cc01382f0 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc00a37e0 con 0x7f1cc0138c10 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc6a40640 10 monclient(hunting): authenticate will time out at 621.762890s 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cbf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc0138c10 0x7f1cc00a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cbf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc0138c10 0x7f1cc00a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48040/0 (socket says 10.20.193.78:48040) 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/1626516429 learned_addr learned my addr 10.20.193.78:0/1626516429 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cc4fb6640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f1cc00a5a70 0x7f1cc00a5f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41266/0 (socket says 10.20.193.78:41266) 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cbf7fe640 10 monclient(hunting): get_auth_request con 0x7f1cc0138c10 auth_method 0 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cbf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.208+0000 7f1cbf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbffff640 1 --2- 10.20.193.78:0/1626516429 >> v2:10.20.193.115:3300/0 conn(0x7f1cc01382f0 0x7f1cc01386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbffff640 10 monclient(hunting): get_auth_request con 0x7f1cc01382f0 auth_method 0 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 10 monclient(hunting): handle_auth_done global_id 4295 payload 274 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/1626516429 >> v2:10.20.193.115:3300/0 conn(0x7f1cc01382f0 msgr2=0x7f1cc01386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 1 --2- 10.20.193.78:0/1626516429 >> v2:10.20.193.115:3300/0 conn(0x7f1cc01382f0 0x7f1cc01386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/1626516429 >> v1:10.20.193.78:6789/0 conn(0x7f1cc00a5a70 legacy=0x7f1cc00a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 1 monclient: found mon.noname-b 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.209+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/1626516429 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc00a21d0 con 0x7f1cc0138c10 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/1626516429 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc0138c10 msgr2=0x7f1cc00a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbf7fe640 1 --2- 10.20.193.78:0/1626516429 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc0138c10 0x7f1cc00a5530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009970 tx=0x7f1cb002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38b2ca02e10eab53 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 1 -- 10.20.193.78:0/1626516429 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb003c070 con 0x7f1cc0138c10 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: dump: 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.224 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 1 -- 10.20.193.78:0/1626516429 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb002fcd0 con 0x7f1cc0138c10 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cc6a40640 5 monclient: authenticate success, global_id 4295 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cc6a40640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 1 -- 10.20.193.78:0/1626516429 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0037660 con 0x7f1cc0138c10 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cc6a40640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cc6a40640 4 set_mon_vals no callback set 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.211+0000 7f1cbeffd640 10 monclient: dump: 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.226 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient: shutdown 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- 10.20.193.78:0/1626516429 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc0138c10 msgr2=0x7f1cc00a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/1626516429 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc0138c10 0x7f1cc00a5530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009970 tx=0x7f1cb002ee90 comp rx=0 tx=0).stop 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- 10.20.193.78:0/1626516429 shutdown_connections 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/1626516429 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc0138c10 0x7f1cc00a5530 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/1626516429 >> v2:10.20.193.115:3300/0 conn(0x7f1cc01382f0 0x7f1cc01386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- 10.20.193.78:0/1626516429 >> 10.20.193.78:0/1626516429 conn(0x7f1cc00a7a50 msgr2=0x7f1cc00a7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- 10.20.193.78:0/1626516429 shutdown_connections 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- 10.20.193.78:0/1626516429 wait complete. 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient: monmap: 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:16.228 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 Processor -- start 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- start start 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient: init 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc00a5a70 0x7f1cc01cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient(hunting): picked mon.noname-c con 0x7f1cc00a5a70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc01382f0 0x7f1cc01cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient(hunting): picked mon.noname-a con 0x7f1cc01382f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1cc0138c10 0x7f1cc0124ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient(hunting): picked mon.noname-b con 0x7f1cc0138c10 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0138ff0 con 0x7f1cc01382f0 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00a1970 con 0x7f1cc0138c10 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc0127ea0 con 0x7f1cc00a5a70 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cc6a40640 10 monclient(hunting): authenticate will time out at 621.766897s 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc01382f0 0x7f1cc01cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc01382f0 0x7f1cc01cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52938/0 (socket says 10.20.193.78:52938) 2026-02-17T05:46:16.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/3890349976 learned_addr learned my addr 10.20.193.78:0/3890349976 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 10 monclient(hunting): get_auth_request con 0x7f1cc01382f0 auth_method 0 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbffff640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc00a5a70 0x7f1cc01cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbffff640 10 monclient(hunting): get_auth_request con 0x7f1cc00a5a70 auth_method 0 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.212+0000 7f1cbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 10 monclient(hunting): handle_auth_done global_id 4317 payload 824 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc00a5a70 msgr2=0x7f1cc01cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc00a5a70 0x7f1cc01cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1cc0138c10 msgr2=0x7f1cc0124ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1cc0138c10 0x7f1cc0124ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 1 monclient: found mon.noname-a 2026-02-17T05:46:16.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.213+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/3890349976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc01272e0 con 0x7f1cc01382f0 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbf7fe640 1 -- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc01382f0 msgr2=0x7f1cc01cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbf7fe640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc01382f0 0x7f1cc01cfc80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00090b0 tx=0x7f1cb00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 1 -- 10.20.193.78:0/3890349976 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0044020 con 0x7f1cc01382f0 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: dump: 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.233 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 1 -- 10.20.193.78:0/3890349976 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb0040a00 con 0x7f1cc01382f0 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 1 -- 10.20.193.78:0/3890349976 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb003c040 con 0x7f1cc01382f0 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbe7fc640 4 set_mon_vals no callback set 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cbcff9640 10 monclient: dump: 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.235 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 5 monclient: authenticate success, global_id 4317 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 10 monclient: _renew_subs 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc0128a70 con 0x7f1cc01382f0 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 10 monclient: _renew_subs 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc0073650 con 0x7f1cc01382f0 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.215+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc01386d0 con 0x7f1cc01382f0 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.216+0000 7f1cbcff9640 1 -- 10.20.193.78:0/3890349976 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1cb003f050 con 0x7f1cc01382f0 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.216+0000 7f1cbcff9640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c9003df00 0x7f1c900403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.216+0000 7f1cbcff9640 1 -- 10.20.193.78:0/3890349976 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7f1cb0077620 con 0x7f1cc01382f0 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.216+0000 7f1cbffff640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c9003df00 0x7f1c900403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.216+0000 7f1cbffff640 10 monclient: get_auth_request con 0x7f1c9003df00 auth_method 0 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.218+0000 7f1cbcff9640 1 -- 10.20.193.78:0/3890349976 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1cb004e030 con 0x7f1cc01382f0 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.218+0000 7f1cbcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.218+0000 7f1cbcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:16.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.219+0000 7f1cbffff640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c9003df00 0x7f1c900403c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009a50 tx=0x7f1cac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.300+0000 7f1cc6a40640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2026-02-17T05:46:16.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.300+0000 7f1cc6a40640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2026-02-17T05:46:16.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.300+0000 7f1cc6a40640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.300+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cc0126780 con 0x7f1cc01382f0 2026-02-17T05:46:16.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.300+0000 7f1cbcff9640 1 -- 10.20.193.78:0/3890349976 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f1cb0077020 con 0x7f1cc01382f0 2026-02-17T05:46:16.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.300+0000 7f1cbcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2026-02-17T05:46:16.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.300+0000 7f1cbcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stdout:pg_num: 1 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c9003df00 msgr2=0x7f1c900403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c9003df00 0x7f1c900403c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009a50 tx=0x7f1cac008040 comp rx=0 tx=0).stop 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 10 monclient: shutdown 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc01382f0 msgr2=0x7f1cc01cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc01382f0 0x7f1cc01cfc80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00090b0 tx=0x7f1cb00047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 shutdown_connections 2026-02-17T05:46:16.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c9003df00 0x7f1c900403c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1cc0138c10 0x7f1cc0124ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc01382f0 0x7f1cc01cfc80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 --2- 10.20.193.78:0/3890349976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc00a5a70 0x7f1cc01cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 >> 10.20.193.78:0/3890349976 conn(0x7f1cc00a7a50 msgr2=0x7f1cc00aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 shutdown_connections 2026-02-17T05:46:16.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.301+0000 7f1cc6a40640 1 -- 10.20.193.78:0/3890349976 wait complete. 2026-02-17T05:46:16.310 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.378+0000 7f741bc6f640 10 monclient: get_monmap_and_config 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.378+0000 7f741bc6f640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.378+0000 7f741bc6f640 10 monclient: monmap: 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:16.379953+0000 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:16.379953+0000 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:16.380 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 1 Processor -- start 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 1 -- start start 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient: init 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f74141382f0 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f74140a2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient(hunting): picked mon.noname-b con 0x7f7414138de0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f741413d530 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74140a4c90 con 0x7f741413d530 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74140a1a60 con 0x7f74141382f0 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f74140a2390 con 0x7f7414138de0 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741bc6f640 10 monclient(hunting): authenticate will time out at 621.933573s 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741a1e5640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f741413d530 0x7f741413f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41272/0 (socket says 10.20.193.78:41272) 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f741a1e5640 1 -- 10.20.193.78:0/2540576273 learned_addr learned my addr 10.20.193.78:0/2540576273 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 1 --2- 10.20.193.78:0/2540576273 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f74140a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient(hunting): get_auth_request con 0x7f7414138de0 auth_method 0 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient(hunting): handle_auth_done global_id 4301 payload 274 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 1 -- 10.20.193.78:0/2540576273 >> v1:10.20.193.115:6789/0 conn(0x7f74141382f0 legacy=0x7f74141386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 1 -- 10.20.193.78:0/2540576273 >> v1:10.20.193.78:6789/0 conn(0x7f741413d530 legacy=0x7f741413f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 1 monclient: found mon.noname-b 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.379+0000 7f74191e3640 1 -- 10.20.193.78:0/2540576273 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74140a1d40 con 0x7f7414138de0 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74191e3640 1 -- 10.20.193.78:0/2540576273 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 msgr2=0x7f74140a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74191e3640 1 --2- 10.20.193.78:0/2540576273 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f74140a2720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f74080098d0 tx=0x7f740802edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4fee4bd40c1754 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 1 -- 10.20.193.78:0/2540576273 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740803c070 con 0x7f7414138de0 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: dump: 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.384 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 1 -- 10.20.193.78:0/2540576273 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740802fd20 con 0x7f7414138de0 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 1 -- 10.20.193.78:0/2540576273 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408037660 con 0x7f7414138de0 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f74189e2640 10 monclient: dump: 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 5 monclient: authenticate success, global_id 4301 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 4 set_mon_vals no callback set 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 10 monclient: shutdown 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 1 -- 10.20.193.78:0/2540576273 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 msgr2=0x7f74140a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 1 --2- 10.20.193.78:0/2540576273 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f74140a2720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f74080098d0 tx=0x7f740802edc0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 1 -- 10.20.193.78:0/2540576273 shutdown_connections 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 1 --2- 10.20.193.78:0/2540576273 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f74140a2720 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 1 -- 10.20.193.78:0/2540576273 >> 10.20.193.78:0/2540576273 conn(0x7f741406ccb0 msgr2=0x7f741406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 1 -- 10.20.193.78:0/2540576273 shutdown_connections 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.382+0000 7f741bc6f640 1 -- 10.20.193.78:0/2540576273 wait complete. 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient: monmap: 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 1 Processor -- start 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 1 -- start start 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient: init 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74141382f0 0x7f741406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient(hunting): picked mon.noname-a con 0x7f74141382f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f741406f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient(hunting): picked mon.noname-c con 0x7f7414138de0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f741413d530 0x7f74141cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient(hunting): picked mon.noname-b con 0x7f741413d530 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74140a4440 con 0x7f74141382f0 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f74140a42c0 con 0x7f741413d530 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7414141040 con 0x7f7414138de0 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f741bc6f640 10 monclient(hunting): authenticate will time out at 621.937863s 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f741406f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74199e4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74141382f0 0x7f741406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f741406f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48068/0 (socket says 10.20.193.78:48068) 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 1 -- 10.20.193.78:0/116835427 learned_addr learned my addr 10.20.193.78:0/116835427 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 10 monclient(hunting): get_auth_request con 0x7f7414138de0 auth_method 0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74199e4640 10 monclient(hunting): get_auth_request con 0x7f74141382f0 auth_method 0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74199e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74199e4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74199e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74191e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74199e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74199e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.383+0000 7f74199e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 10 monclient(hunting): handle_auth_done global_id 4304 payload 824 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f741a1e5640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f741413d530 0x7f74141cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 1 -- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f741413d530 msgr2=0x7f74141cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f741413d530 0x7f74141cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 1 -- 10.20.193.78:0/116835427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74141382f0 msgr2=0x7f741406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74141382f0 0x7f741406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 1 monclient: found mon.noname-c 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.384+0000 7f74191e3640 1 -- 10.20.193.78:0/116835427 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74140a3d30 con 0x7f7414138de0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74191e3640 1 -- 10.20.193.78:0/116835427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 msgr2=0x7f741406f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74191e3640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f741406f500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f74080098d0 tx=0x7f74080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 1 -- 10.20.193.78:0/116835427 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408046070 con 0x7f7414138de0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: dump: 2026-02-17T05:46:16.389 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 1 -- 10.20.193.78:0/116835427 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7408041890 con 0x7f7414138de0 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 1 -- 10.20.193.78:0/116835427 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740803c040 con 0x7f7414138de0 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 10 monclient: dump: 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f7403fff640 4 set_mon_vals no callback set 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 5 monclient: authenticate success, global_id 4304 2026-02-17T05:46:16.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 10 monclient: _renew_subs 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741409f830 con 0x7f7414138de0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 10 monclient: _renew_subs 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f741413cde0 con 0x7f7414138de0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f74027fc640 1 -- 10.20.193.78:0/116835427 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7408041c80 con 0x7f7414138de0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.386+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74140a2720 con 0x7f7414138de0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.387+0000 7f74027fc640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f73e403deb0 0x7f73e4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.387+0000 7f74027fc640 1 -- 10.20.193.78:0/116835427 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7f7408077ea0 con 0x7f7414138de0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.387+0000 7f74199e4640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f73e403deb0 0x7f73e4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.387+0000 7f74199e4640 10 monclient: get_auth_request con 0x7f73e403deb0 auth_method 0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.389+0000 7f74027fc640 1 -- 10.20.193.78:0/116835427 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7408037e10 con 0x7f7414138de0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.389+0000 7f74027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.389+0000 7f74027fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:16.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.389+0000 7f74199e4640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f73e403deb0 0x7f73e4040370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7404005160 tx=0x7f7404007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.470+0000 7f741bc6f640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2026-02-17T05:46:16.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.470+0000 7f741bc6f640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.470+0000 7f741bc6f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.470+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) -- 0x7f741406fa40 con 0x7f7414138de0 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.470+0000 7f74027fc640 1 -- 10.20.193.78:0/116835427 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) ==== 92+0+10 (secure 0 0 0) 0x7f7408076a30 con 0x7f7414138de0 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.470+0000 7f74027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.470+0000 7f74027fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stdout:pg_num: 8 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f73e403deb0 msgr2=0x7f73e4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f73e403deb0 0x7f73e4040370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7404005160 tx=0x7f7404007f30 comp rx=0 tx=0).stop 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 10 monclient: shutdown 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 msgr2=0x7f741406f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f741406f500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f74080098d0 tx=0x7f74080047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 shutdown_connections 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f73e403deb0 0x7f73e4040370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f741413d530 0x7f74141cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7414138de0 0x7f741406f500 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 --2- 10.20.193.78:0/116835427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74141382f0 0x7f741406efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 >> 10.20.193.78:0/116835427 conn(0x7f741406ccb0 msgr2=0x7f7414073e80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 shutdown_connections 2026-02-17T05:46:16.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.471+0000 7f741bc6f640 1 -- 10.20.193.78:0/116835427 wait complete. 2026-02-17T05:46:16.480 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2026-02-17T05:46:16.480 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T05:46:16.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.546+0000 7f8af28bf640 10 monclient: get_monmap_and_config 2026-02-17T05:46:16.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.546+0000 7f8af28bf640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.546+0000 7f8af28bf640 10 monclient: monmap: 2026-02-17T05:46:16.551 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.551 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.551 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:16.548184+0000 2026-02-17T05:46:16.551 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:16.548184+0000 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 1 Processor -- start 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 1 -- start start 2026-02-17T05:46:16.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient: init 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8aec06c890 0x7f8aec06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient(hunting): picked mon.noname-a con 0x7f8aec06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec09c4a0 0x7f8aec06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient(hunting): picked mon.noname-b con 0x7f8aec09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8aec134df0 0x7f8aec135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient(hunting): picked mon.noname-c con 0x7f8aec134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f8aec0a0880 con 0x7f8aec06c890 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f8aec06d6f0 con 0x7f8aec134df0 2026-02-17T05:46:16.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec06daf0 con 0x7f8aec09c4a0 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af28bf640 10 monclient(hunting): authenticate will time out at 622.101724s 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8aec06c890 0x7f8aec06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aeb7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec09c4a0 0x7f8aec06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8aec06c890 0x7f8aec06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52960/0 (socket says 10.20.193.78:52960) 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 1 -- 10.20.193.78:0/3567460214 learned_addr learned my addr 10.20.193.78:0/3567460214 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 10 monclient(hunting): get_auth_request con 0x7f8aec06c890 auth_method 0 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aeb7fe640 10 monclient(hunting): get_auth_request con 0x7f8aec09c4a0 auth_method 0 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aeb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aeb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af0e35640 1 --2- 10.20.193.78:0/3567460214 >> v2:10.20.193.115:3300/0 conn(0x7f8aec134df0 0x7f8aec135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af0e35640 10 monclient(hunting): get_auth_request con 0x7f8aec134df0 auth_method 0 2026-02-17T05:46:16.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af0e35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af0e35640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8af0e35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 10 monclient(hunting): handle_auth_done global_id 4323 payload 274 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 1 -- 10.20.193.78:0/3567460214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec09c4a0 msgr2=0x7f8aec06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.547+0000 7f8aebfff640 1 --2- 10.20.193.78:0/3567460214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec09c4a0 0x7f8aec06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.548+0000 7f8aebfff640 1 -- 10.20.193.78:0/3567460214 >> v2:10.20.193.115:3300/0 conn(0x7f8aec134df0 msgr2=0x7f8aec135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.548+0000 7f8aebfff640 1 --2- 10.20.193.78:0/3567460214 >> v2:10.20.193.115:3300/0 conn(0x7f8aec134df0 0x7f8aec135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.548+0000 7f8aebfff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.548+0000 7f8aebfff640 1 monclient: found mon.noname-a 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.548+0000 7f8aebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.548+0000 7f8aebfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.548+0000 7f8aebfff640 1 -- 10.20.193.78:0/3567460214 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec09cc80 con 0x7f8aec06c890 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.550+0000 7f8aebfff640 1 -- 10.20.193.78:0/3567460214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 msgr2=0x7f8aec06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.550+0000 7f8aebfff640 1 --2- 10.20.193.78:0/3567460214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec06cc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0a7490 tx=0x7f8ad802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c3f022e2892366 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 1 -- 10.20.193.78:0/3567460214 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad803c070 con 0x7f8aec06c890 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: dump: 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.564 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8af28bf640 5 monclient: authenticate success, global_id 4323 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 1 -- 10.20.193.78:0/3567460214 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad802fca0 con 0x7f8aec06c890 2026-02-17T05:46:16.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8af28bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 1 -- 10.20.193.78:0/3567460214 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8037660 con 0x7f8aec06c890 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8af28bf640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8af28bf640 4 set_mon_vals no callback set 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8aeaffd640 10 monclient: dump: 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.566 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8af28bf640 10 monclient: shutdown 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8af28bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8af28bf640 1 -- 10.20.193.78:0/3567460214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 msgr2=0x7f8aec06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.552+0000 7f8af28bf640 1 --2- 10.20.193.78:0/3567460214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec06cc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0a7490 tx=0x7f8ad802ee60 comp rx=0 tx=0).stop 2026-02-17T05:46:16.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 -- 10.20.193.78:0/3567460214 shutdown_connections 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 --2- 10.20.193.78:0/3567460214 >> v2:10.20.193.115:3300/0 conn(0x7f8aec134df0 0x7f8aec135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 --2- 10.20.193.78:0/3567460214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec09c4a0 0x7f8aec06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 --2- 10.20.193.78:0/3567460214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec06cc70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 -- 10.20.193.78:0/3567460214 >> 10.20.193.78:0/3567460214 conn(0x7f8aec0a6620 msgr2=0x7f8aec0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 -- 10.20.193.78:0/3567460214 shutdown_connections 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 -- 10.20.193.78:0/3567460214 wait complete. 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient: monmap: 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:16.568 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 Processor -- start 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 -- start start 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient: init 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec1cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient(hunting): picked mon.noname-a con 0x7f8aec06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8aec09c4a0 0x7f8aec1cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient(hunting): picked mon.noname-b con 0x7f8aec09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec134df0 0x7f8aec122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient(hunting): picked mon.noname-c con 0x7f8aec134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0a01a0 con 0x7f8aec06c890 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0a0020 con 0x7f8aec09c4a0 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec09be20 con 0x7f8aec134df0 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af28bf640 10 monclient(hunting): authenticate will time out at 622.107882s 2026-02-17T05:46:16.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec1cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec1cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52968/0 (socket says 10.20.193.78:52968) 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 1 -- 10.20.193.78:0/2560453998 learned_addr learned my addr 10.20.193.78:0/2560453998 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af0e35640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec134df0 0x7f8aec122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 10 monclient(hunting): get_auth_request con 0x7f8aec06c890 auth_method 0 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af0e35640 10 monclient(hunting): get_auth_request con 0x7f8aec134df0 auth_method 0 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af0e35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af0e35640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af0e35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8aebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af0e35640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.553+0000 7f8af0e35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8af0e35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aeb7fe640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8aec09c4a0 0x7f8aec1cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aeb7fe640 10 monclient(hunting): get_auth_request con 0x7f8aec09c4a0 auth_method 0 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aeb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aeb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 10 monclient(hunting): handle_auth_done global_id 4326 payload 824 2026-02-17T05:46:16.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 1 -- 10.20.193.78:0/2560453998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec134df0 msgr2=0x7f8aec122e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec134df0 0x7f8aec122e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 1 -- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8aec09c4a0 msgr2=0x7f8aec1cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8aec09c4a0 0x7f8aec1cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 1 monclient: found mon.noname-a 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.554+0000 7f8aebfff640 1 -- 10.20.193.78:0/2560453998 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec126cf0 con 0x7f8aec06c890 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8aebfff640 1 -- 10.20.193.78:0/2560453998 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 msgr2=0x7f8aec1cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8aebfff640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec1cf710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8002410 tx=0x7f8ad8004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 1 -- 10.20.193.78:0/2560453998 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8047070 con 0x7f8aec06c890 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: dump: 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 1 -- 10.20.193.78:0/2560453998 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad802fe50 con 0x7f8aec06c890 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 1 -- 10.20.193.78:0/2560453998 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad803c040 con 0x7f8aec06c890 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 10 monclient: dump: 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.575 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8aea7fc640 4 set_mon_vals no callback set 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 5 monclient: authenticate success, global_id 4326 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 10 monclient: _renew_subs 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec0a16a0 con 0x7f8aec06c890 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 10 monclient: _renew_subs 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec126a10 con 0x7f8aec06c890 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec074980 con 0x7f8aec06c890 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.556+0000 7f8ae8ff9640 1 -- 10.20.193.78:0/2560453998 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8ad8037ce0 con 0x7f8aec06c890 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.557+0000 7f8ae8ff9640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8abc03df30 0x7f8abc0403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.557+0000 7f8ae8ff9640 1 -- 10.20.193.78:0/2560453998 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7f8ad8076a90 con 0x7f8aec06c890 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.557+0000 7f8aeb7fe640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8abc03df30 0x7f8abc0403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.557+0000 7f8aeb7fe640 10 monclient: get_auth_request con 0x7f8abc03df30 auth_method 0 2026-02-17T05:46:16.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.559+0000 7f8ae8ff9640 1 -- 10.20.193.78:0/2560453998 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8ad802fb60 con 0x7f8aec06c890 2026-02-17T05:46:16.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.559+0000 7f8ae8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.559+0000 7f8ae8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:16.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.559+0000 7f8aeb7fe640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8abc03df30 0x7f8abc0403f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8aec125830 tx=0x7f8adc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.641+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8aec127660 con 0x7f8abc03df30 2026-02-17T05:46:16.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.641+0000 7f8ae8ff9640 1 -- 10.20.193.78:0/2560453998 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34165 (secure 0 0 0) 0x7f8aec127660 con 0x7f8abc03df30 2026-02-17T05:46:16.644 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:16.645 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T05:46:16.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8abc03df30 msgr2=0x7f8abc0403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8abc03df30 0x7f8abc0403f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8aec125830 tx=0x7f8adc008040 comp rx=0 tx=0).stop 2026-02-17T05:46:16.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 10 monclient: shutdown 2026-02-17T05:46:16.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 msgr2=0x7f8aec1cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec1cf710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8002410 tx=0x7f8ad8004830 comp rx=0 tx=0).stop 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 shutdown_connections 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8aec134df0 0x7f8aec122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8abc03df30 0x7f8abc0403f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8aec09c4a0 0x7f8aec1cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 --2- 10.20.193.78:0/2560453998 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8aec06c890 0x7f8aec1cf710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 >> 10.20.193.78:0/2560453998 conn(0x7f8aec0a6620 msgr2=0x7f8aec135810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 shutdown_connections 2026-02-17T05:46:16.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.642+0000 7f8af28bf640 1 -- 10.20.193.78:0/2560453998 wait complete. 2026-02-17T05:46:16.651 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2026-02-17T05:46:15.910347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167196,"kb_used_data":1220,"kb_used_omap":9,"kb_used_meta":165814,"kb_avail":566063844,"statfs":{"total":579820584960,"available":579649376256,"internally_reserved":0,"allocated":1249280,"data_stored":1432448,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":169794214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.733105"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183122+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T05:46:12.183122+0000","last_peered":"2026-02-17T05:46:12.183122+0000","last_clean":"2026-02-17T05:46:12.183122+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T05:46:12.183122+0000","last_undegraded":"2026-02-17T05:46:12.183122+0000","last_fullsized":"2026-02-17T05:46:12.183122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.184594+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T05:46:12.184594+0000","last_peered":"2026-02-17T05:46:12.184594+0000","last_clean":"2026-02-17T05:46:12.184594+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T05:46:12.184594+0000","last_undegraded":"2026-02-17T05:46:12.184594+0000","last_fullsized":"2026-02-17T05:46:12.184594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183276+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T05:46:12.183276+0000","last_peered":"2026-02-17T05:46:12.183276+0000","last_clean":"2026-02-17T05:46:12.183276+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T05:46:12.183276+0000","last_undegraded":"2026-02-17T05:46:12.183276+0000","last_fullsized":"2026-02-17T05:46:12.183276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.217637+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T05:46:12.217637+0000","last_peered":"2026-02-17T05:46:12.217637+0000","last_clean":"2026-02-17T05:46:12.217637+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T05:46:12.217637+0000","last_undegraded":"2026-02-17T05:46:12.217637+0000","last_fullsized":"2026-02-17T05:46:12.217637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.184443+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T05:46:12.184443+0000","last_peered":"2026-02-17T05:46:12.184443+0000","last_clean":"2026-02-17T05:46:12.184443+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T05:46:12.184443+0000","last_undegraded":"2026-02-17T05:46:12.184443+0000","last_fullsized":"2026-02-17T05:46:12.184443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.182358+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T05:46:12.182358+0000","last_peered":"2026-02-17T05:46:12.182358+0000","last_clean":"2026-02-17T05:46:12.182358+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T05:46:12.182358+0000","last_undegraded":"2026-02-17T05:46:12.182358+0000","last_fullsized":"2026-02-17T05:46:12.182358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183042+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T05:46:12.183042+0000","last_peered":"2026-02-17T05:46:12.183042+0000","last_clean":"2026-02-17T05:46:12.183042+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T05:46:12.183042+0000","last_undegraded":"2026-02-17T05:46:12.183042+0000","last_fullsized":"2026-02-17T05:46:12.183042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183095+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T05:46:12.183095+0000","last_peered":"2026-02-17T05:46:12.183095+0000","last_clean":"2026-02-17T05:46:12.183095+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T05:46:12.183095+0000","last_undegraded":"2026-02-17T05:46:12.183095+0000","last_fullsized":"2026-02-17T05:46:12.183095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'112","reported_seq":156,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.182832+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T05:46:12.182832+0000","last_peered":"2026-02-17T05:46:12.182832+0000","last_clean":"2026-02-17T05:46:12.182832+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T05:46:12.182832+0000","last_undegraded":"2026-02-17T05:46:12.182832+0000","last_fullsized":"2026-02-17T05:46:12.182832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":249856,"data_stored":592181,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":192512,"data_stored":63272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":176128,"data_stored":58289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":188416,"data_stored":63253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":249856,"data_stored":592181,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":192512,"data_stored":63272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T05:46:16.653 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T05:46:16.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.718+0000 7f9e32e5b640 10 monclient: get_monmap_and_config 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.718+0000 7f9e32e5b640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.718+0000 7f9e32e5b640 10 monclient: monmap: 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:16.719498+0000 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:16.719498+0000 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:16.722 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.718+0000 7f9e32e5b640 1 Processor -- start 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.718+0000 7f9e32e5b640 1 -- start start 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.718+0000 7f9e32e5b640 10 monclient: init 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.718+0000 7f9e32e5b640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.718+0000 7f9e32e5b640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9e2c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c09c4a0 0x7f9e2c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 10 monclient(hunting): picked mon.noname-b con 0x7f9e2c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9e2c06d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e2c06daf0 con 0x7f9e2c06d380 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e2c09cc80 con 0x7f9e2c06c890 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2c0a0880 con 0x7f9e2c09c4a0 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e32e5b640 10 monclient(hunting): authenticate will time out at 622.273053s 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e313d1640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f9e2c06d380 0x7f9e2c13d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41286/0 (socket says 10.20.193.78:41286) 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e313d1640 1 -- 10.20.193.78:0/2781155345 learned_addr learned my addr 10.20.193.78:0/2781155345 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 1 --2- 10.20.193.78:0/2781155345 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c09c4a0 0x7f9e2c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient(hunting): get_auth_request con 0x7f9e2c09c4a0 auth_method 0 2026-02-17T05:46:16.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient(hunting): handle_auth_done global_id 4313 payload 274 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 1 -- 10.20.193.78:0/2781155345 >> v1:10.20.193.115:6789/0 conn(0x7f9e2c06c890 legacy=0x7f9e2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 1 -- 10.20.193.78:0/2781155345 >> v1:10.20.193.78:6789/0 conn(0x7f9e2c06d380 legacy=0x7f9e2c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 1 monclient: found mon.noname-b 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.719+0000 7f9e23fff640 1 -- 10.20.193.78:0/2781155345 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c09cf60 con 0x7f9e2c09c4a0 2026-02-17T05:46:16.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e23fff640 1 -- 10.20.193.78:0/2781155345 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c09c4a0 msgr2=0x7f9e2c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e23fff640 1 --2- 10.20.193.78:0/2781155345 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c09c4a0 0x7f9e2c13b3a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098d0 tx=0x7f9e1402edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7efafa6622b09fa0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 1 -- 10.20.193.78:0/2781155345 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1403c070 con 0x7f9e2c09c4a0 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: dump: 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.732 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 1 -- 10.20.193.78:0/2781155345 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e1402fd20 con 0x7f9e2c09c4a0 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 1 -- 10.20.193.78:0/2781155345 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14037660 con 0x7f9e2c09c4a0 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e32e5b640 5 monclient: authenticate success, global_id 4313 2026-02-17T05:46:16.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e32e5b640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e32e5b640 4 set_mon_vals no callback set 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.721+0000 7f9e237fe640 10 monclient: dump: 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.734 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient: shutdown 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2781155345 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c09c4a0 msgr2=0x7f9e2c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 --2- 10.20.193.78:0/2781155345 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c09c4a0 0x7f9e2c13b3a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098d0 tx=0x7f9e1402edc0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2781155345 shutdown_connections 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 --2- 10.20.193.78:0/2781155345 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c09c4a0 0x7f9e2c13b3a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2781155345 >> 10.20.193.78:0/2781155345 conn(0x7f9e2c0a6620 msgr2=0x7f9e2c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2781155345 shutdown_connections 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2781155345 wait complete. 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient: monmap: 2026-02-17T05:46:16.735 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 Processor -- start 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- start start 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient: init 2026-02-17T05:46:16.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c06c890 0x7f9e2c131dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient(hunting): picked mon.noname-c con 0x7f9e2c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9e2c06d380 0x7f9e2c132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient(hunting): picked mon.noname-a con 0x7f9e2c06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9e2c09c4a0 0x7f9e2c132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient(hunting): picked mon.noname-b con 0x7f9e2c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c09bfa0 con 0x7f9e2c06d380 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c09be20 con 0x7f9e2c09c4a0 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2c13f010 con 0x7f9e2c06c890 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e32e5b640 10 monclient(hunting): authenticate will time out at 622.276923s 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e23fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9e2c06d380 0x7f9e2c132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e30bd0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c06c890 0x7f9e2c131dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e23fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9e2c06d380 0x7f9e2c132300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52978/0 (socket says 10.20.193.78:52978) 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e23fff640 1 -- 10.20.193.78:0/2663669439 learned_addr learned my addr 10.20.193.78:0/2663669439 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient(hunting): get_auth_request con 0x7f9e2c06d380 auth_method 0 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.722+0000 7f9e30bd0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c06c890 0x7f9e2c131dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48106/0 (socket says 10.20.193.78:48106) 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e30bd0640 10 monclient(hunting): get_auth_request con 0x7f9e2c06c890 auth_method 0 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e30bd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e30bd0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e30bd0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e30bd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e30bd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e313d1640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9e2c09c4a0 0x7f9e2c132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e30bd0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e313d1640 10 monclient(hunting): get_auth_request con 0x7f9e2c09c4a0 auth_method 0 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e313d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e313d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e313d1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient(hunting): handle_auth_done global_id 4329 payload 824 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 1 -- 10.20.193.78:0/2663669439 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c06c890 msgr2=0x7f9e2c131dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c06c890 0x7f9e2c131dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 1 -- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9e2c09c4a0 msgr2=0x7f9e2c132840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9e2c09c4a0 0x7f9e2c132840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 1 monclient: found mon.noname-a 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.723+0000 7f9e23fff640 1 -- 10.20.193.78:0/2663669439 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c09d810 con 0x7f9e2c06d380 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e23fff640 1 -- 10.20.193.78:0/2663669439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9e2c06d380 msgr2=0x7f9e2c132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e23fff640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9e2c06d380 0x7f9e2c132300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e1403a040 tx=0x7f9e140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 1 -- 10.20.193.78:0/2663669439 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14045020 con 0x7f9e2c06d380 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: dump: 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.741 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 1 -- 10.20.193.78:0/2663669439 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e1403d040 con 0x7f9e2c06d380 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e32e5b640 5 monclient: authenticate success, global_id 4329 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 1 -- 10.20.193.78:0/2663669439 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1403c070 con 0x7f9e2c06d380 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e22ffd640 4 set_mon_vals no callback set 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e217fa640 10 monclient: dump: 2026-02-17T05:46:16.742 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e32e5b640 10 monclient: _renew_subs 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e32e5b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c13cef0 con 0x7f9e2c06d380 2026-02-17T05:46:16.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e32e5b640 10 monclient: _renew_subs 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e32e5b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.725+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c13cc10 con 0x7f9e2c06d380 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e32e5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e32e5b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e32e5b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2c13ac00 con 0x7f9e2c06d380 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e217fa640 1 -- 10.20.193.78:0/2663669439 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9e14041c80 con 0x7f9e2c06d380 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e217fa640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9dfc03deb0 0x7f9dfc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e217fa640 1 -- 10.20.193.78:0/2663669439 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7f9e140779e0 con 0x7f9e2c06d380 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e30bd0640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9dfc03deb0 0x7f9dfc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.726+0000 7f9e30bd0640 10 monclient: get_auth_request con 0x7f9dfc03deb0 auth_method 0 2026-02-17T05:46:16.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.728+0000 7f9e217fa640 1 -- 10.20.193.78:0/2663669439 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9e1404f030 con 0x7f9e2c06d380 2026-02-17T05:46:16.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.728+0000 7f9e217fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.728+0000 7f9e217fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:16.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.728+0000 7f9e30bd0640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9dfc03deb0 0x7f9dfc040370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005160 tx=0x7f9e1c007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.808+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e2c06cc70 con 0x7f9dfc03deb0 2026-02-17T05:46:16.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e217fa640 1 -- 10.20.193.78:0/2663669439 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34165 (secure 0 0 0) 0x7f9e2c06cc70 con 0x7f9dfc03deb0 2026-02-17T05:46:16.812 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T05:46:16.813 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T05:46:16.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9dfc03deb0 msgr2=0x7f9dfc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9dfc03deb0 0x7f9dfc040370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005160 tx=0x7f9e1c007f30 comp rx=0 tx=0).stop 2026-02-17T05:46:16.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 10 monclient: shutdown 2026-02-17T05:46:16.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9e2c06d380 msgr2=0x7f9e2c132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9e2c06d380 0x7f9e2c132300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e1403a040 tx=0x7f9e140047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 shutdown_connections 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9dfc03deb0 0x7f9dfc040370 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9e2c09c4a0 0x7f9e2c132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9e2c06d380 0x7f9e2c132300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 --2- 10.20.193.78:0/2663669439 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9e2c06c890 0x7f9e2c131dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 >> 10.20.193.78:0/2663669439 conn(0x7f9e2c0a6620 msgr2=0x7f9e2c132fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 shutdown_connections 2026-02-17T05:46:16.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.810+0000 7f9e32e5b640 1 -- 10.20.193.78:0/2663669439 wait complete. 2026-02-17T05:46:16.819 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2026-02-17T05:46:15.910347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167196,"kb_used_data":1220,"kb_used_omap":9,"kb_used_meta":165814,"kb_avail":566063844,"statfs":{"total":579820584960,"available":579649376256,"internally_reserved":0,"allocated":1249280,"data_stored":1432448,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":169794214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.733105"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183122+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T05:46:12.183122+0000","last_peered":"2026-02-17T05:46:12.183122+0000","last_clean":"2026-02-17T05:46:12.183122+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T05:46:12.183122+0000","last_undegraded":"2026-02-17T05:46:12.183122+0000","last_fullsized":"2026-02-17T05:46:12.183122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.184594+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T05:46:12.184594+0000","last_peered":"2026-02-17T05:46:12.184594+0000","last_clean":"2026-02-17T05:46:12.184594+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T05:46:12.184594+0000","last_undegraded":"2026-02-17T05:46:12.184594+0000","last_fullsized":"2026-02-17T05:46:12.184594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183276+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T05:46:12.183276+0000","last_peered":"2026-02-17T05:46:12.183276+0000","last_clean":"2026-02-17T05:46:12.183276+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T05:46:12.183276+0000","last_undegraded":"2026-02-17T05:46:12.183276+0000","last_fullsized":"2026-02-17T05:46:12.183276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.217637+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T05:46:12.217637+0000","last_peered":"2026-02-17T05:46:12.217637+0000","last_clean":"2026-02-17T05:46:12.217637+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T05:46:12.217637+0000","last_undegraded":"2026-02-17T05:46:12.217637+0000","last_fullsized":"2026-02-17T05:46:12.217637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.184443+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T05:46:12.184443+0000","last_peered":"2026-02-17T05:46:12.184443+0000","last_clean":"2026-02-17T05:46:12.184443+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T05:46:12.184443+0000","last_undegraded":"2026-02-17T05:46:12.184443+0000","last_fullsized":"2026-02-17T05:46:12.184443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.182358+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T05:46:12.182358+0000","last_peered":"2026-02-17T05:46:12.182358+0000","last_clean":"2026-02-17T05:46:12.182358+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T05:46:12.182358+0000","last_undegraded":"2026-02-17T05:46:12.182358+0000","last_fullsized":"2026-02-17T05:46:12.182358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183042+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T05:46:12.183042+0000","last_peered":"2026-02-17T05:46:12.183042+0000","last_clean":"2026-02-17T05:46:12.183042+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T05:46:12.183042+0000","last_undegraded":"2026-02-17T05:46:12.183042+0000","last_fullsized":"2026-02-17T05:46:12.183042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.183095+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T05:46:12.183095+0000","last_peered":"2026-02-17T05:46:12.183095+0000","last_clean":"2026-02-17T05:46:12.183095+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T05:46:12.183095+0000","last_undegraded":"2026-02-17T05:46:12.183095+0000","last_fullsized":"2026-02-17T05:46:12.183095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'112","reported_seq":156,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-17T05:46:12.182832+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T05:46:12.182832+0000","last_peered":"2026-02-17T05:46:12.182832+0000","last_clean":"2026-02-17T05:46:12.182832+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T05:46:12.182832+0000","last_undegraded":"2026-02-17T05:46:12.182832+0000","last_fullsized":"2026-02-17T05:46:12.182832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":249856,"data_stored":592181,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":192512,"data_stored":63272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":176128,"data_stored":58289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":188416,"data_stored":63253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":249856,"data_stored":592181,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":192512,"data_stored":63272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T05:46:16.820 INFO:tasks.mon_thrash.ceph_manager:clean! 2026-02-17T05:46:16.821 DEBUG:tasks.mon_thrash:Yielding 2026-02-17T05:46:16.821 INFO:teuthology.run_tasks:Running task rados... 2026-02-17T05:46:16.842 INFO:tasks.rados:Beginning rados... 2026-02-17T05:46:16.842 DEBUG:teuthology.run_tasks:Unwinding manager rados 2026-02-17T05:46:16.857 INFO:tasks.rados:joining rados 2026-02-17T05:46:16.857 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2026-02-17T05:46:16.857 INFO:tasks.mon_thrash.mon_thrasher:seed: 1771307176, revive delay: 20.0, thrash delay: 1.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2026-02-17T05:46:16.857 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:46:16.857 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:46:16.858 INFO:tasks.rados:clients are ['client.0'] 2026-02-17T05:46:16.858 INFO:tasks.rados:starting run 0 out of 1 2026-02-17T05:46:16.858 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2026-02-17T05:46:16.859 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2026-02-17T05:46:16.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.925+0000 7fa1192d8640 10 monclient: get_monmap_and_config 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.925+0000 7fa1192d8640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.925+0000 7fa1192d8640 10 monclient: monmap: 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:16.926943+0000 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:16.926943+0000 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:16.930 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 1 Processor -- start 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 1 -- start start 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient: init 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114135f20 0x7fa114136300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient(hunting): picked mon.noname-b con 0x7fa114135f20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa114136840 0x7fa11406c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient(hunting): picked mon.noname-a con 0x7fa114136840 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa11406cdd0 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa11409d140 con 0x7fa11406cdd0 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa11409cb10 con 0x7fa114136840 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa114135c60 con 0x7fa114135f20 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1192d8640 10 monclient(hunting): authenticate will time out at 622.480546s 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114135f20 0x7fa114136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa118ad7640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fa11406cdd0 0x7fa11413cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41288/0 (socket says 10.20.193.78:41288) 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa118ad7640 1 -- 10.20.193.78:0/2990204268 learned_addr learned my addr 10.20.193.78:0/2990204268 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient(hunting): get_auth_request con 0x7fa114135f20 auth_method 0 2026-02-17T05:46:16.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1137fe640 1 --2- 10.20.193.78:0/2990204268 >> v2:10.20.193.78:3300/0 conn(0x7fa114136840 0x7fa11406c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1137fe640 10 monclient(hunting): get_auth_request con 0x7fa114136840 auth_method 0 2026-02-17T05:46:16.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1137fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa1137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient(hunting): handle_auth_done global_id 4319 payload 274 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 1 -- 10.20.193.78:0/2990204268 >> v2:10.20.193.78:3300/0 conn(0x7fa114136840 msgr2=0x7fa11406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 1 --2- 10.20.193.78:0/2990204268 >> v2:10.20.193.78:3300/0 conn(0x7fa114136840 0x7fa11406c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 1 -- 10.20.193.78:0/2990204268 >> v1:10.20.193.78:6789/0 conn(0x7fa11406cdd0 legacy=0x7fa11413cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 1 monclient: found mon.noname-b 2026-02-17T05:46:16.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.926+0000 7fa113fff640 1 -- 10.20.193.78:0/2990204268 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11409c330 con 0x7fa114135f20 2026-02-17T05:46:16.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.927+0000 7fc61fe91640 10 monclient: get_monmap_and_config 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient: monmap: 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:16.929241+0000 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:16.929241+0000 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:16.938 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 1 Processor -- start 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 1 -- start start 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient: init 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc6181382f0 0x7fc6181386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient(hunting): picked mon.noname-a con 0x7fc6181382f0 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc618138c10 0x7fc6180a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient(hunting): picked mon.noname-b con 0x7fc618138c10 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc6180a5a70 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6180a3540 con 0x7fc6180a5a70 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc6180a5000 con 0x7fc6181382f0 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc6180a37e0 con 0x7fc618138c10 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61fe91640 10 monclient(hunting): authenticate will time out at 622.482852s 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc618138c10 0x7fc6180a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61dc06640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc6181382f0 0x7fc6181386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc618138c10 0x7fc6180a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48124/0 (socket says 10.20.193.78:48124) 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61e407640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc6180a5a70 0x7fc6180a5f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41298/0 (socket says 10.20.193.78:41298) 2026-02-17T05:46:16.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 1 -- 10.20.193.78:0/4111886050 learned_addr learned my addr 10.20.193.78:0/4111886050 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient(hunting): get_auth_request con 0x7fc618138c10 auth_method 0 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61dc06640 10 monclient(hunting): get_auth_request con 0x7fc6181382f0 auth_method 0 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61dc06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61dc06640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61dc06640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61dc06640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61dc06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61dc06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient(hunting): handle_auth_done global_id 4322 payload 274 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 1 -- 10.20.193.78:0/4111886050 >> v2:10.20.193.78:3300/0 conn(0x7fc6181382f0 msgr2=0x7fc6181386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 1 --2- 10.20.193.78:0/4111886050 >> v2:10.20.193.78:3300/0 conn(0x7fc6181382f0 0x7fc6181386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 1 -- 10.20.193.78:0/4111886050 >> v1:10.20.193.78:6789/0 conn(0x7fc6180a5a70 legacy=0x7fc6180a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 1 monclient: found mon.noname-b 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fc61d405640 1 -- 10.20.193.78:0/4111886050 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6180a21d0 con 0x7fc618138c10 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa113fff640 1 -- 10.20.193.78:0/2990204268 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114135f20 msgr2=0x7fa114136300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa113fff640 1 --2- 10.20.193.78:0/2990204268 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114135f20 0x7fa114136300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1000098a0 tx=0x7fa10002ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d80784544b0c1b1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 1 -- 10.20.193.78:0/2990204268 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10003c070 con 0x7fa114135f20 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: dump: 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.947 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 1 -- 10.20.193.78:0/2990204268 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10002fc00 con 0x7fa114135f20 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 1 -- 10.20.193.78:0/2990204268 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100037440 con 0x7fa114135f20 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 5 monclient: authenticate success, global_id 4319 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 4 set_mon_vals no callback set 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa112ffd640 10 monclient: dump: 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.949 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 10 monclient: shutdown 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 1 -- 10.20.193.78:0/2990204268 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114135f20 msgr2=0x7fa114136300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 1 --2- 10.20.193.78:0/2990204268 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114135f20 0x7fa114136300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1000098a0 tx=0x7fa10002ed80 comp rx=0 tx=0).stop 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 1 -- 10.20.193.78:0/2990204268 shutdown_connections 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 1 --2- 10.20.193.78:0/2990204268 >> v2:10.20.193.78:3300/0 conn(0x7fa114136840 0x7fa11406c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 1 --2- 10.20.193.78:0/2990204268 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114135f20 0x7fa114136300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 1 -- 10.20.193.78:0/2990204268 >> 10.20.193.78:0/2990204268 conn(0x7fa1140a65f0 msgr2=0x7fa1140a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 1 -- 10.20.193.78:0/2990204268 shutdown_connections 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.929+0000 7fa1192d8640 1 -- 10.20.193.78:0/2990204268 wait complete. 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient: monmap: 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:16.951 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 1 Processor -- start 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 1 -- start start 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient: init 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa11406cdd0 0x7fa1141cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient(hunting): picked mon.noname-b con 0x7fa11406cdd0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa114135f20 0x7fa114134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient(hunting): picked mon.noname-a con 0x7fa114135f20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114136840 0x7fa114135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient(hunting): picked mon.noname-c con 0x7fa114136840 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa11409ff90 con 0x7fa114135f20 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa11409fe10 con 0x7fa11406cdd0 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa1140a02f0 con 0x7fa114136840 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1192d8640 10 monclient(hunting): authenticate will time out at 622.484907s 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1137fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa114135f20 0x7fa114134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa118ad7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114136840 0x7fa114135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1137fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa114135f20 0x7fa114134f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:53002/0 (socket says 10.20.193.78:53002) 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1137fe640 1 -- 10.20.193.78:0/657690291 learned_addr learned my addr 10.20.193.78:0/657690291 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1137fe640 10 monclient(hunting): get_auth_request con 0x7fa114135f20 auth_method 0 2026-02-17T05:46:16.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1137fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa118ad7640 10 monclient(hunting): get_auth_request con 0x7fa114136840 auth_method 0 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa118ad7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa118ad7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa118ad7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.930+0000 7fa1137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa118ad7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa118ad7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa118ad7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 10 monclient(hunting): handle_auth_done global_id 4338 payload 824 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa113fff640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa11406cdd0 0x7fa1141cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 1 -- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114136840 msgr2=0x7fa114135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114136840 0x7fa114135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 1 -- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa11406cdd0 msgr2=0x7fa1141cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa11406cdd0 0x7fa1141cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 1 monclient: found mon.noname-a 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa1137fe640 1 -- 10.20.193.78:0/657690291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa114128390 con 0x7fa114135f20 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa113fff640 10 monclient: get_auth_request con 0x7fa11406cdd0 auth_method 0 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa118ad7640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114136840 0x7fa114135470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:16.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fa113fff640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa11406cdd0 0x7fa1141cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:46:16.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 1 -- 10.20.193.78:0/4111886050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc618138c10 msgr2=0x7fc6180a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 1 --2- 10.20.193.78:0/4111886050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc618138c10 0x7fc6180a5530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009970 tx=0x7fc60c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47900c9766167a4c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 1 -- 10.20.193.78:0/4111886050 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c03c070 con 0x7fc618138c10 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: dump: 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.960 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 1 -- 10.20.193.78:0/4111886050 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c02fcd0 con 0x7fc618138c10 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 1 -- 10.20.193.78:0/4111886050 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c037660 con 0x7fc618138c10 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 5 monclient: authenticate success, global_id 4322 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 4 set_mon_vals no callback set 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61cc04640 10 monclient: dump: 2026-02-17T05:46:16.961 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient: shutdown 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- 10.20.193.78:0/4111886050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc618138c10 msgr2=0x7fc6180a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 --2- 10.20.193.78:0/4111886050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc618138c10 0x7fc6180a5530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009970 tx=0x7fc60c02ee90 comp rx=0 tx=0).stop 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- 10.20.193.78:0/4111886050 shutdown_connections 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 --2- 10.20.193.78:0/4111886050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc618138c10 0x7fc6180a5530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 --2- 10.20.193.78:0/4111886050 >> v2:10.20.193.78:3300/0 conn(0x7fc6181382f0 0x7fc6181386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- 10.20.193.78:0/4111886050 >> 10.20.193.78:0/4111886050 conn(0x7fc6180a7a50 msgr2=0x7fc6180a7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- 10.20.193.78:0/4111886050 shutdown_connections 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- 10.20.193.78:0/4111886050 wait complete. 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient: build_initial_monmap 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient: monmap: 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 Processor -- start 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- start start 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient: init 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6180a5a70 0x7fc6181cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient(hunting): picked mon.noname-b con 0x7fc6180a5a70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181382f0 0x7fc6181cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient(hunting): picked mon.noname-c con 0x7fc6181382f0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc618138c10 0x7fc618124980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient(hunting): picked mon.noname-a con 0x7fc618138c10 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc618138ff0 con 0x7fc618138c10 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6180a1970 con 0x7fc6180a5a70 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc6181273a0 con 0x7fc6181382f0 2026-02-17T05:46:16.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61fe91640 10 monclient(hunting): authenticate will time out at 622.486911s 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181382f0 0x7fc6181cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61e407640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc618138c10 0x7fc618124980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181382f0 0x7fc6181cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48148/0 (socket says 10.20.193.78:48148) 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 1 -- 10.20.193.78:0/2851544675 learned_addr learned my addr 10.20.193.78:0/2851544675 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 10 monclient(hunting): get_auth_request con 0x7fc6181382f0 auth_method 0 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61d405640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.932+0000 7fc61e407640 10 monclient(hunting): get_auth_request con 0x7fc618138c10 auth_method 0 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61e407640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61e407640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61e407640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61e407640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61e407640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61e407640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61dc06640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6180a5a70 0x7fc6181cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61dc06640 10 monclient(hunting): get_auth_request con 0x7fc6180a5a70 auth_method 0 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61dc06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61dc06640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61dc06640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 10 monclient(hunting): handle_auth_done global_id 4328 payload 824 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 1 -- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6180a5a70 msgr2=0x7fc6181cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6180a5a70 0x7fc6181cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 1 -- 10.20.193.78:0/2851544675 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc618138c10 msgr2=0x7fc618124980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc618138c10 0x7fc618124980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 10 monclient: _finish_hunting 0 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 1 monclient: found mon.noname-c 2026-02-17T05:46:16.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:16.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fc61d405640 1 -- 10.20.193.78:0/2851544675 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618128910 con 0x7fc6181382f0 2026-02-17T05:46:16.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1137fe640 1 -- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa114135f20 msgr2=0x7fa114134f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1137fe640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa114135f20 0x7fa114134f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa10400ebe0 tx=0x7fa10400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 1 -- 10.20.193.78:0/657690291 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104017070 con 0x7fa114135f20 2026-02-17T05:46:16.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: dump: 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 1 -- 10.20.193.78:0/657690291 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10400ce50 con 0x7fa114135f20 2026-02-17T05:46:16.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 5 monclient: authenticate success, global_id 4338 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 10 monclient: _renew_subs 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa114128670 con 0x7fa114135f20 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 1 -- 10.20.193.78:0/657690291 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1040136b0 con 0x7fa114135f20 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1127fc640 4 set_mon_vals no callback set 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 10 monclient: dump: 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.970 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 10 monclient: _renew_subs 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa114072990 con 0x7fa114135f20 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa114136300 con 0x7fa114135f20 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.933+0000 7fa110ff9640 1 -- 10.20.193.78:0/657690291 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa104024060 con 0x7fa114135f20 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.934+0000 7fa110ff9640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0e403deb0 0x7fa0e4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.934+0000 7fa110ff9640 1 -- 10.20.193.78:0/657690291 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7fa104052800 con 0x7fa114135f20 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.934+0000 7fa113fff640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0e403deb0 0x7fa0e4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.934+0000 7fa113fff640 10 monclient: get_auth_request con 0x7fa0e403deb0 auth_method 0 2026-02-17T05:46:16.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61d405640 1 -- 10.20.193.78:0/2851544675 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181382f0 msgr2=0x7fc6181cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61d405640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181382f0 0x7fc6181cfc80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009aa0 tx=0x7fc60c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 1 -- 10.20.193.78:0/2851544675 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c045020 con 0x7fc6181382f0 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: dump: 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:16.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 1 -- 10.20.193.78:0/2851544675 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c03d040 con 0x7fc6181382f0 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 1 -- 10.20.193.78:0/2851544675 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c03c070 con 0x7fc6181382f0 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 10 monclient: dump: 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc607fff640 4 set_mon_vals no callback set 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 5 monclient: authenticate success, global_id 4328 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 10 monclient: _renew_subs 2026-02-17T05:46:16.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc618073650 con 0x7fc6181382f0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 10 monclient: _renew_subs 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc618128350 con 0x7fc6181382f0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618127180 con 0x7fc6181382f0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.935+0000 7fc6067fc640 1 -- 10.20.193.78:0/2851544675 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc60c041c80 con 0x7fc6181382f0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.936+0000 7fc6067fc640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df00 0x7fc5e80403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.936+0000 7fc6067fc640 1 -- 10.20.193.78:0/2851544675 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (secure 0 0 0) 0x7fc60c077960 con 0x7fc6181382f0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.936+0000 7fc61dc06640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df00 0x7fc5e80403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.936+0000 7fc61dc06640 10 monclient: get_auth_request con 0x7fc5e803df00 auth_method 0 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.936+0000 7fa110ff9640 1 -- 10.20.193.78:0/657690291 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa104013850 con 0x7fa114135f20 2026-02-17T05:46:16.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.936+0000 7fa110ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.936+0000 7fa110ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:16.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.936+0000 7fa113fff640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0e403deb0 0x7fa0e4040370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa114136c40 tx=0x7fa100033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.939+0000 7fc61dc06640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df00 0x7fc5e80403c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc608009bd0 tx=0x7fc608008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:16.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.939+0000 7fc6067fc640 1 -- 10.20.193.78:0/2851544675 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc60c04f030 con 0x7fc6181382f0 2026-02-17T05:46:16.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.939+0000 7fc6067fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:16.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:16.939+0000 7fc6067fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:17.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.026+0000 7fc61fe91640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2026-02-17T05:46:17.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.026+0000 7fc61fe91640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2026-02-17T05:46:17.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.026+0000 7fc61fe91640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.026+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} v 0) -- 0x7fc6181386d0 con 0x7fc6181382f0 2026-02-17T05:46:17.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:46:17.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:46:17.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:17.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa114127680 con 0x7fa114135f20 2026-02-17T05:46:17.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa110ff9640 1 -- 10.20.193.78:0/657690291 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fa104013b30 con 0x7fa114135f20 2026-02-17T05:46:17.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa110ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:46:17.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa110ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:17.054 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:46:17.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0e403deb0 msgr2=0x7fa0e4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0e403deb0 0x7fa0e4040370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa114136c40 tx=0x7fa100033000 comp rx=0 tx=0).stop 2026-02-17T05:46:17.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 10 monclient: shutdown 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa114135f20 msgr2=0x7fa114134f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa114135f20 0x7fa114134f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa10400ebe0 tx=0x7fa10400c520 comp rx=0 tx=0).stop 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 shutdown_connections 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa114136840 0x7fa114135470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa114135f20 0x7fa114134f30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0e403deb0 0x7fa0e4040370 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 --2- 10.20.193.78:0/657690291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa11406cdd0 0x7fa1141cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 >> 10.20.193.78:0/657690291 conn(0x7fa1140a65f0 msgr2=0x7fa1140a2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 shutdown_connections 2026-02-17T05:46:17.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.053+0000 7fa1192d8640 1 -- 10.20.193.78:0/657690291 wait complete. 2026-02-17T05:46:17.063 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:46:17.064 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:46:17.064 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:46:17.064 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:46:17.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.169+0000 7ff31b0a7640 10 monclient: get_monmap_and_config 2026-02-17T05:46:17.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.169+0000 7ff31b0a7640 10 monclient: build_initial_monmap 2026-02-17T05:46:17.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.169+0000 7ff31b0a7640 10 monclient: monmap: 2026-02-17T05:46:17.171 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:17.171 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.171 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:17.171135+0000 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:17.171135+0000 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:17.172 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 1 Processor -- start 2026-02-17T05:46:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 1 -- start start 2026-02-17T05:46:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient: init 2026-02-17T05:46:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff31406c890 0x7ff31406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient(hunting): picked mon.noname-a con 0x7ff31406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff31409d430 0x7ff31406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient(hunting): picked mon.noname-c con 0x7ff31409d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff31409bf40 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff314073600 con 0x7ff31409bf40 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff31409ca50 con 0x7ff31406c890 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff31406d6f0 con 0x7ff31409d430 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff31b0a7640 10 monclient(hunting): authenticate will time out at 622.724736s 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff31406c890 0x7ff31406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff31406c890 0x7ff31406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:53030/0 (socket says 10.20.193.78:53030) 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 1 -- 10.20.193.78:0/227309132 learned_addr learned my addr 10.20.193.78:0/227309132 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 10 monclient(hunting): get_auth_request con 0x7ff31406c890 auth_method 0 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff30bfff640 1 --2- 10.20.193.78:0/227309132 >> v2:10.20.193.115:3300/0 conn(0x7ff31409d430 0x7ff31406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff30bfff640 10 monclient(hunting): get_auth_request con 0x7ff31409d430 auth_method 0 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff30bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff30bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.170+0000 7ff30bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 10 monclient(hunting): handle_auth_done global_id 4344 payload 274 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 1 -- 10.20.193.78:0/227309132 >> v2:10.20.193.115:3300/0 conn(0x7ff31409d430 msgr2=0x7ff31406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 1 --2- 10.20.193.78:0/227309132 >> v2:10.20.193.115:3300/0 conn(0x7ff31409d430 0x7ff31406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 1 -- 10.20.193.78:0/227309132 >> v1:10.20.193.115:6789/0 conn(0x7ff31409bf40 legacy=0x7ff314134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 10 monclient: _finish_hunting 0 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 1 monclient: found mon.noname-a 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.171+0000 7ff318e1c640 1 -- 10.20.193.78:0/227309132 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31406dab0 con 0x7ff31406c890 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff318e1c640 1 -- 10.20.193.78:0/227309132 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31406c890 msgr2=0x7ff31406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff318e1c640 1 --2- 10.20.193.78:0/227309132 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31406c890 0x7ff31406cc70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3140a0ac0 tx=0x7ff30402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcc5b3983c3deb2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 1 -- 10.20.193.78:0/227309132 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30403c070 con 0x7ff31406c890 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: dump: 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.185 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 1 -- 10.20.193.78:0/227309132 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30402fca0 con 0x7ff31406c890 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 1 -- 10.20.193.78:0/227309132 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304037660 con 0x7ff31406c890 2026-02-17T05:46:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff31b0a7640 5 monclient: authenticate success, global_id 4344 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff31b0a7640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff31b0a7640 4 set_mon_vals no callback set 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff30b7fe640 10 monclient: dump: 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.187 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff31b0a7640 10 monclient: shutdown 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff31b0a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.173+0000 7ff31b0a7640 1 -- 10.20.193.78:0/227309132 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31406c890 msgr2=0x7ff31406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 --2- 10.20.193.78:0/227309132 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31406c890 0x7ff31406cc70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3140a0ac0 tx=0x7ff30402ee60 comp rx=0 tx=0).stop 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 -- 10.20.193.78:0/227309132 shutdown_connections 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 --2- 10.20.193.78:0/227309132 >> v2:10.20.193.115:3300/0 conn(0x7ff31409d430 0x7ff31406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 --2- 10.20.193.78:0/227309132 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31406c890 0x7ff31406cc70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 -- 10.20.193.78:0/227309132 >> 10.20.193.78:0/227309132 conn(0x7ff3140a6620 msgr2=0x7ff3140a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 -- 10.20.193.78:0/227309132 shutdown_connections 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 -- 10.20.193.78:0/227309132 wait complete. 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 10 monclient: build_initial_monmap 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 10 monclient: monmap: 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:17.189 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 Processor -- start 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 1 -- start start 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.174+0000 7ff31b0a7640 10 monclient: init 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff31406c890 0x7ff3141cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient(hunting): picked mon.noname-b con 0x7ff31406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff31409bf40 0x7ff3141cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient(hunting): picked mon.noname-c con 0x7ff31409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31409d430 0x7ff314121660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient(hunting): picked mon.noname-a con 0x7ff31409d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff31409c800 con 0x7ff31409d430 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff314073410 con 0x7ff31406c890 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff314135500 con 0x7ff31409bf40 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31b0a7640 10 monclient(hunting): authenticate will time out at 622.729103s 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31409d430 0x7ff314121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31409d430 0x7ff314121660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:53038/0 (socket says 10.20.193.78:53038) 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 -- 10.20.193.78:0/358904690 learned_addr learned my addr 10.20.193.78:0/358904690 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient(hunting): get_auth_request con 0x7ff31409d430 auth_method 0 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff30bfff640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff31409bf40 0x7ff3141cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff30bfff640 10 monclient(hunting): get_auth_request con 0x7ff31409bf40 auth_method 0 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff30bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff30bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff30bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff318e1c640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff31406c890 0x7ff3141cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff30bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff30bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff30bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff318e1c640 10 monclient(hunting): get_auth_request con 0x7ff31406c890 auth_method 0 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff318e1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff318e1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff318e1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient(hunting): handle_auth_done global_id 4347 payload 824 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff31409bf40 msgr2=0x7ff3141cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff31409bf40 0x7ff3141cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff31406c890 msgr2=0x7ff3141cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff31406c890 0x7ff3141cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient: _finish_hunting 0 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 monclient: found mon.noname-a 2026-02-17T05:46:17.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.175+0000 7ff31961d640 1 -- 10.20.193.78:0/358904690 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3141232b0 con 0x7ff31409d430 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31961d640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31409d430 msgr2=0x7ff314121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31961d640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31409d430 0x7ff314121660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff31000cb60 tx=0x7ff31000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310016020 con 0x7ff31409d430 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: dump: 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.194 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: _finish_auth 0 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: _check_auth_tickets 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff310012420 con 0x7ff31409d430 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31b0a7640 5 monclient: authenticate success, global_id 4347 2026-02-17T05:46:17.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31b0a7640 10 monclient: _renew_subs 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31b0a7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3140a1a10 con 0x7ff31409d430 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310012810 con 0x7ff31409d430 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff3097fa640 10 monclient: dump: 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.196 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff30affd640 4 set_mon_vals no callback set 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31b0a7640 10 monclient: _renew_subs 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31b0a7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.177+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3141252c0 con 0x7ff31409d430 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff31b0a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff31b0a7640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff314125f80 0x7ff314122360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff31b0a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff314074980 con 0x7ff314125f80 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff314125f80 0x7ff314122360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 10 monclient: get_auth_request con 0x7ff314125f80 auth_method 0 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff31961d640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31409d430 0x7ff314121660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff31000cb60 tx=0x7ff31000a430 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7ff310053050 osd_map(21..21 src has 1..21) 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff310012a30 con 0x7ff31409d430 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 10 monclient(hunting): handle_auth_done global_id 4350 payload 824 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff318e1c640 1 -- 10.20.193.78:0/358904690 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff314135890 con 0x7ff314125f80 2026-02-17T05:46:17.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff3097fa640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff2e403deb0 0x7ff2e4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff30bfff640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff2e403deb0 0x7ff2e4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.178+0000 7ff30bfff640 10 monclient: get_auth_request con 0x7ff2e403deb0 auth_method 0 2026-02-17T05:46:17.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff318e1c640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff314125f80 msgr2=0x7ff314122360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff318e1c640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff314125f80 0x7ff314122360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff314138100 tx=0x7ff304004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30403c040 con 0x7ff314125f80 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 10 monclient: dump: 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.202 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff30402fe50 con 0x7ff314125f80 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff314125f80 msgr2=0x7ff314122360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff314125f80 0x7ff314122360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff314138100 tx=0x7ff304004830 comp rx=0 tx=0).stop 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff314125f80 msgr2=0x7ff314122360 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.180+0000 7ff3097fa640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff314125f80 0x7ff314122360 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff314138100 tx=0x7ff304004830 comp rx=0 tx=0).stop 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.181+0000 7ff30bfff640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff2e403deb0 0x7ff2e4040370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff3141346e0 tx=0x7ff2fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff31b0a7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:46:17.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff31b0a7640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31411cc70 0x7ff3141da820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff31b0a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff314134c40 con 0x7ff31411cc70 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31411cc70 0x7ff3141da820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 10 monclient: get_auth_request con 0x7ff31411cc70 auth_method 0 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 10 monclient(hunting): handle_auth_done global_id 4353 payload 824 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.184+0000 7ff318e1c640 1 -- 10.20.193.78:0/358904690 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff314125600 con 0x7ff31411cc70 2026-02-17T05:46:17.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff318e1c640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31411cc70 msgr2=0x7ff3141da820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff318e1c640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31411cc70 0x7ff3141da820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff314074b00 tx=0x7ff3040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30403c040 con 0x7ff31411cc70 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 10 monclient: dump: 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.205 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7ff304003e60 con 0x7ff31411cc70 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31411cc70 msgr2=0x7ff3141da820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31411cc70 0x7ff3141da820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff314074b00 tx=0x7ff3040047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31411cc70 msgr2=0x7ff3141da820 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:17.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.186+0000 7ff3097fa640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31411cc70 0x7ff3141da820 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff314074b00 tx=0x7ff3040047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 4, 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 26990, 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:46:17.207 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 25, 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.208 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.209 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:46:17.210 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.211 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.212 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:46:17.213 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:46:17.214 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.215 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:46:17.216 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:46:17.217 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff2e403deb0 msgr2=0x7ff2e4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff2e403deb0 0x7ff2e4040370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff3141346e0 tx=0x7ff2fc008040 comp rx=0 tx=0).stop 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 10 monclient: shutdown 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31409d430 msgr2=0x7ff314121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 1 --2- 10.20.193.78:0/358904690 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff31409d430 0x7ff314121660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff31000cb60 tx=0x7ff31000a430 comp rx=0 tx=0).stop 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31961d640 1 -- 10.20.193.78:0/358904690 reap_dead start 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31961d640 1 -- 10.20.193.78:0/358904690 reap_dead start 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff3097fa640 1 -- 10.20.193.78:0/358904690 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 4370+0+0 (unknown 0 0 0) 0x7ff310053050 con 0x7ff31409d430 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 shutdown_connections 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 >> 10.20.193.78:0/358904690 conn(0x7ff3140a6620 msgr2=0x7ff314136910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 shutdown_connections 2026-02-17T05:46:17.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.187+0000 7ff31b0a7640 1 -- 10.20.193.78:0/358904690 wait complete. 2026-02-17T05:46:17.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:46:17.220 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:46:17.220 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:46:17.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.205+0000 7fc6067fc640 1 -- 10.20.193.78:0/2851544675 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]=0 pool 'unique_pool_0' created v22) ==== 130+0+0 (secure 0 0 0) 0x7fc60c03fd80 con 0x7fc6181382f0 2026-02-17T05:46:17.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.205+0000 7fc6067fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.205+0000 7fc6067fc640 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' created 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:pool 'unique_pool_0' created 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df00 msgr2=0x7fc5e80403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df00 0x7fc5e80403c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc608009bd0 tx=0x7fc608008040 comp rx=0 tx=0).stop 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 10 monclient: shutdown 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181382f0 msgr2=0x7fc6181cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181382f0 0x7fc6181cfc80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009aa0 tx=0x7fc60c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 shutdown_connections 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc618138c10 0x7fc618124980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6181382f0 0x7fc6181cfc80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5e803df00 0x7fc5e80403c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 --2- 10.20.193.78:0/2851544675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6180a5a70 0x7fc6181cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 >> 10.20.193.78:0/2851544675 conn(0x7fc6180a7a50 msgr2=0x7fc6180aa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 shutdown_connections 2026-02-17T05:46:17.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.207+0000 7fc61fe91640 1 -- 10.20.193.78:0/2851544675 wait complete. 2026-02-17T05:46:17.223 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2026-02-17T05:46:17.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.329+0000 7f5eca064640 10 monclient: get_monmap_and_config 2026-02-17T05:46:17.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.329+0000 7f5eca064640 10 monclient: build_initial_monmap 2026-02-17T05:46:17.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.329+0000 7f5eca064640 10 monclient: monmap: 2026-02-17T05:46:17.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:17.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:17.330499+0000 2026-02-17T05:46:17.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:17.330499+0000 2026-02-17T05:46:17.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:17.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.329+0000 7f5eca064640 1 Processor -- start 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.329+0000 7f5eca064640 1 -- start start 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.329+0000 7f5eca064640 10 monclient: init 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2026-02-17T05:46:17.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5ec406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5ec409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec406d380 0x7f5ec413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 10 monclient(hunting): picked mon.noname-b con 0x7f5ec406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec406daf0 con 0x7f5ec406c890 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec409cc80 con 0x7f5ec409c4a0 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec40a0880 con 0x7f5ec406d380 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5eca064640 10 monclient(hunting): authenticate will time out at 622.884175s 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec37fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f5ec406c890 0x7f5ec406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33324/0 (socket says 10.20.193.78:33324) 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec37fe640 1 -- 10.20.193.78:0/700953937 learned_addr learned my addr 10.20.193.78:0/700953937 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:17.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 1 --2- 10.20.193.78:0/700953937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec406d380 0x7f5ec413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient(hunting): get_auth_request con 0x7f5ec406d380 auth_method 0 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient(hunting): handle_auth_done global_id 4334 payload 274 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 1 -- 10.20.193.78:0/700953937 >> v1:10.20.193.115:6789/0 conn(0x7f5ec409c4a0 legacy=0x7f5ec413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 1 -- 10.20.193.78:0/700953937 >> v1:10.20.193.78:6789/0 conn(0x7f5ec406c890 legacy=0x7f5ec406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 1 monclient: found mon.noname-b 2026-02-17T05:46:17.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:17.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.330+0000 7f5ec3fff640 1 -- 10.20.193.78:0/700953937 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec409cf60 con 0x7f5ec406d380 2026-02-17T05:46:17.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient: get_monmap_and_config 2026-02-17T05:46:17.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient: build_initial_monmap 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient: monmap: 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:17.332381+0000 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:17.332381+0000 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:17.340 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 1 Processor -- start 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 1 -- start start 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient: init 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5a380a4d50 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5a380a3850 0x7f5a3813d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient(hunting): picked mon.noname-c con 0x7f5a380a3850 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a3cc0 0x7f5a3813fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient(hunting): picked mon.noname-b con 0x7f5a380a3cc0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3809e6a0 con 0x7f5a380a4d50 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5a380a5130 con 0x7f5a380a3850 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3809b7a0 con 0x7f5a380a3cc0 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a3cdfb640 10 monclient(hunting): authenticate will time out at 622.885933s 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.331+0000 7f5a36d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a3cc0 0x7f5a3813fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a3cc0 0x7f5a3813fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47804/0 (socket says 10.20.193.78:47804) 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 1 -- 10.20.193.78:0/2543781445 learned_addr learned my addr 10.20.193.78:0/2543781445 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient(hunting): get_auth_request con 0x7f5a380a3cc0 auth_method 0 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a35d74640 1 --2- 10.20.193.78:0/2543781445 >> v2:10.20.193.115:3300/0 conn(0x7f5a380a3850 0x7f5a3813d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a35d74640 10 monclient(hunting): get_auth_request con 0x7f5a380a3850 auth_method 0 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a35d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a35d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a35d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient(hunting): handle_auth_done global_id 4337 payload 274 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 1 -- 10.20.193.78:0/2543781445 >> v2:10.20.193.115:3300/0 conn(0x7f5a380a3850 msgr2=0x7f5a3813d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 1 --2- 10.20.193.78:0/2543781445 >> v2:10.20.193.115:3300/0 conn(0x7f5a380a3850 0x7f5a3813d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 1 -- 10.20.193.78:0/2543781445 >> v1:10.20.193.115:6789/0 conn(0x7f5a380a4d50 legacy=0x7f5a380a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:17.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient: _finish_hunting 0 2026-02-17T05:46:17.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 1 monclient: found mon.noname-b 2026-02-17T05:46:17.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:17.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5a36d76640 1 -- 10.20.193.78:0/2543781445 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3806d070 con 0x7f5a380a3cc0 2026-02-17T05:46:17.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec3fff640 1 -- 10.20.193.78:0/700953937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec406d380 msgr2=0x7f5ec413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec3fff640 1 --2- 10.20.193.78:0/700953937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec406d380 0x7f5ec413ead0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009dc0 tx=0x7f5eb802f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87f1c596042c8cf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 1 -- 10.20.193.78:0/700953937 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb803c070 con 0x7f5ec406d380 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: dump: 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.349 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 1 -- 10.20.193.78:0/700953937 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8037640 con 0x7f5ec406d380 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 5 monclient: authenticate success, global_id 4334 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 1 -- 10.20.193.78:0/700953937 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8037a10 con 0x7f5ec406d380 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 4 set_mon_vals no callback set 2026-02-17T05:46:17.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5ec27fc640 10 monclient: dump: 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.351 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 10 monclient: shutdown 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 -- 10.20.193.78:0/700953937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec406d380 msgr2=0x7f5ec413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 --2- 10.20.193.78:0/700953937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec406d380 0x7f5ec413ead0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009dc0 tx=0x7f5eb802f1c0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 -- 10.20.193.78:0/700953937 shutdown_connections 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 --2- 10.20.193.78:0/700953937 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec406d380 0x7f5ec413ead0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 -- 10.20.193.78:0/700953937 >> 10.20.193.78:0/700953937 conn(0x7f5ec40a6620 msgr2=0x7f5ec40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 -- 10.20.193.78:0/700953937 shutdown_connections 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 -- 10.20.193.78:0/700953937 wait complete. 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 10 monclient: build_initial_monmap 2026-02-17T05:46:17.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 10 monclient: monmap: 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 Processor -- start 2026-02-17T05:46:17.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 1 -- start start 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.333+0000 7f5eca064640 10 monclient: init 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ec406c890 0x7f5ec411cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient(hunting): picked mon.noname-a con 0x7f5ec406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ec406d380 0x7f5ec411d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient(hunting): picked mon.noname-b con 0x7f5ec406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec409c4a0 0x7f5ec411d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient(hunting): picked mon.noname-c con 0x7f5ec409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec409bfa0 con 0x7f5ec406c890 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec409be20 con 0x7f5ec406d380 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec413f010 con 0x7f5ec409c4a0 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5eca064640 10 monclient(hunting): authenticate will time out at 622.888215s 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec409c4a0 0x7f5ec411d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec409c4a0 0x7f5ec411d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47806/0 (socket says 10.20.193.78:47806) 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 -- 10.20.193.78:0/4079255524 learned_addr learned my addr 10.20.193.78:0/4079255524 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient(hunting): get_auth_request con 0x7f5ec409c4a0 auth_method 0 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec37fe640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ec406c890 0x7f5ec411cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec37fe640 10 monclient(hunting): get_auth_request con 0x7f5ec406c890 auth_method 0 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec2ffd640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ec406d380 0x7f5ec411d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec2ffd640 10 monclient(hunting): get_auth_request con 0x7f5ec406d380 auth_method 0 2026-02-17T05:46:17.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient(hunting): handle_auth_done global_id 4340 payload 824 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ec406d380 msgr2=0x7f5ec411d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ec406d380 0x7f5ec411d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ec406c890 msgr2=0x7f5ec411cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ec406c890 0x7f5ec411cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 monclient: found mon.noname-c 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5ec3fff640 1 -- 10.20.193.78:0/4079255524 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec409d810 con 0x7f5ec409c4a0 2026-02-17T05:46:17.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a36d76640 1 -- 10.20.193.78:0/2543781445 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a3cc0 msgr2=0x7f5a3813fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a36d76640 1 --2- 10.20.193.78:0/2543781445 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a3cc0 0x7f5a3813fe30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a3806d920 tx=0x7f5a2802f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec88240c09fbaad server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 1 -- 10.20.193.78:0/2543781445 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2802fe60 con 0x7f5a380a3cc0 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: dump: 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: _finish_auth 0 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: _check_auth_tickets 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a3cdfb640 5 monclient: authenticate success, global_id 4337 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 1 -- 10.20.193.78:0/2543781445 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a280373f0 con 0x7f5a380a3cc0 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a3cdfb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a3cdfb640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 1 -- 10.20.193.78:0/2543781445 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a280377c0 con 0x7f5a380a3cc0 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a3cdfb640 4 set_mon_vals no callback set 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.334+0000 7f5a35573640 10 monclient: dump: 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.364 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient: shutdown 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/2543781445 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a3cc0 msgr2=0x7f5a3813fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/2543781445 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a3cc0 0x7f5a3813fe30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a3806d920 tx=0x7f5a2802f230 comp rx=0 tx=0).stop 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/2543781445 shutdown_connections 2026-02-17T05:46:17.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/2543781445 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a3cc0 0x7f5a3813fe30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/2543781445 >> v2:10.20.193.115:3300/0 conn(0x7f5a380a3850 0x7f5a3813d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/2543781445 >> 10.20.193.78:0/2543781445 conn(0x7f5a380a4230 msgr2=0x7f5a380a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/2543781445 shutdown_connections 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/2543781445 wait complete. 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient: build_initial_monmap 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient: monmap: 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:17.366 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 Processor -- start 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- start start 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient: init 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5a380a3850 0x7f5a3813f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient(hunting): picked mon.noname-b con 0x7f5a380a3850 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5a380a3cc0 0x7f5a38137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient(hunting): picked mon.noname-a con 0x7f5a380a3cc0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a4d50 0x7f5a38138460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient(hunting): picked mon.noname-c con 0x7f5a380a4d50 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3809b1d0 con 0x7f5a380a3cc0 2026-02-17T05:46:17.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3809bf60 con 0x7f5a380a3850 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5a38140370 con 0x7f5a380a4d50 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.335+0000 7f5a3cdfb640 10 monclient(hunting): authenticate will time out at 622.889970s 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a4d50 0x7f5a38138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5a380a3cc0 0x7f5a38137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a4d50 0x7f5a38138460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47812/0 (socket says 10.20.193.78:47812) 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5a380a3cc0 0x7f5a38137f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:50554/0 (socket says 10.20.193.78:50554) 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 -- 10.20.193.78:0/3802867093 learned_addr learned my addr 10.20.193.78:0/3802867093 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient(hunting): get_auth_request con 0x7f5a380a4d50 auth_method 0 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 10 monclient(hunting): get_auth_request con 0x7f5a380a3cc0 auth_method 0 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36575640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5a380a3850 0x7f5a3813f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient(hunting): handle_auth_done global_id 4343 payload 824 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 -- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5a380a3850 msgr2=0x7f5a3813f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5a380a3850 0x7f5a3813f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 -- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5a380a3cc0 msgr2=0x7f5a38137f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5a380a3cc0 0x7f5a38137f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient: _finish_hunting 0 2026-02-17T05:46:17.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 monclient: found mon.noname-c 2026-02-17T05:46:17.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:17.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36d76640 1 -- 10.20.193.78:0/3802867093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a380a6bb0 con 0x7f5a380a4d50 2026-02-17T05:46:17.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36575640 10 monclient: get_auth_request con 0x7f5a380a3850 auth_method 0 2026-02-17T05:46:17.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a36575640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5a380a3850 0x7f5a3813f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:46:17.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5a35d74640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5a380a3cc0 0x7f5a38137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:17.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5ec3fff640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec409c4a0 msgr2=0x7f5ec411d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5ec3fff640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec409c4a0 0x7f5ec411d6f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5eb803a040 tx=0x7f5eb8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8047020 con 0x7f5ec409c4a0 2026-02-17T05:46:17.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: dump: 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: _finish_auth 0 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8042880 con 0x7f5ec409c4a0 2026-02-17T05:46:17.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5eca064640 5 monclient: authenticate success, global_id 4340 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5eca064640 10 monclient: _renew_subs 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5eca064640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec411f5c0 con 0x7f5ec409c4a0 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb803c040 con 0x7f5ec409c4a0 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5e9ffff640 10 monclient: dump: 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.374 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5ec1ffb640 4 set_mon_vals no callback set 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5eca064640 10 monclient: _renew_subs 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5eca064640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.336+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec411e3f0 con 0x7f5ec409c4a0 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5eca064640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5eca064640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec4135a90 0x7f5ec4135e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5eca064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec411e730 con 0x7f5ec4135a90 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec4135a90 0x7f5ec4135e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 10 monclient: get_auth_request con 0x7f5ec4135a90 auth_method 0 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5eb8043420 con 0x7f5ec409c4a0 2026-02-17T05:46:17.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_done global_id 4346 payload 824 2026-02-17T05:46:17.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec37fe640 1 -- 10.20.193.78:0/4079255524 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ec411fc70 con 0x7f5ec4135a90 2026-02-17T05:46:17.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5e9ffff640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5e9403deb0 0x7f5e94040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4694+0+0 (secure 0 0 0) 0x7f5eb8078650 con 0x7f5ec409c4a0 2026-02-17T05:46:17.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec2ffd640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5e9403deb0 0x7f5e94040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.337+0000 7f5ec2ffd640 10 monclient: get_auth_request con 0x7f5e9403deb0 auth_method 0 2026-02-17T05:46:17.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a36d76640 1 -- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a4d50 msgr2=0x7f5a38138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a36d76640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a4d50 0x7f5a38138460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009f50 tx=0x7f5a28004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 1 -- 10.20.193.78:0/3802867093 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28046020 con 0x7f5a380a4d50 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: dump: 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: _finish_auth 0 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 1 -- 10.20.193.78:0/3802867093 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a28041870 con 0x7f5a380a4d50 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 5 monclient: authenticate success, global_id 4343 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 10 monclient: _renew_subs 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3809e920 con 0x7f5a380a4d50 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 1 -- 10.20.193.78:0/3802867093 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28041c40 con 0x7f5a380a4d50 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 10 monclient: dump: 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.379 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a34d72640 4 set_mon_vals no callback set 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 10 monclient: _renew_subs 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3813e4d0 con 0x7f5a380a4d50 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a1effd640 1 -- 10.20.193.78:0/3802867093 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5a2804b400 con 0x7f5a380a4d50 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.338+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3813b850 con 0x7f5a380a4d50 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.339+0000 7f5a1effd640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5a0403df30 0x7f5a040403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.339+0000 7f5a1effd640 1 -- 10.20.193.78:0/3802867093 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4694+0+0 (secure 0 0 0) 0x7f5a28078a10 con 0x7f5a380a4d50 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.339+0000 7f5a36575640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5a0403df30 0x7f5a040403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.339+0000 7f5a36575640 10 monclient: get_auth_request con 0x7f5a0403df30 auth_method 0 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.339+0000 7f5ec37fe640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec4135a90 msgr2=0x7f5ec4135e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.339+0000 7f5ec37fe640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec4135a90 0x7f5ec4135e70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005030 tx=0x7f5eb0005430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.340+0000 7f5ec2ffd640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5e9403deb0 0x7f5e94040370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4136d00 tx=0x7f5eb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0006050 con 0x7f5ec4135a90 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 10 monclient: dump: 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.381 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5a1effd640 1 -- 10.20.193.78:0/3802867093 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5a3813b850 con 0x7f5a380a4d50 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5a1effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5a1effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f5eb0018070 con 0x7f5ec4135a90 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec4135a90 msgr2=0x7f5ec4135e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec4135a90 0x7f5ec4135e70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005030 tx=0x7f5eb0005430 comp rx=0 tx=0).stop 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec4135a90 msgr2=0x7f5ec4135e70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5e9ffff640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec4135a90 0x7f5ec4135e70 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005030 tx=0x7f5eb0005430 comp rx=0 tx=0).stop 2026-02-17T05:46:17.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.341+0000 7f5a36575640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5a0403df30 0x7f5a040403f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c004020 tx=0x7f5a2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5eca064640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5eca064640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec41363b0 0x7f5ec41d8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5eca064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec406fd40 con 0x7f5ec41363b0 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec41363b0 0x7f5ec41d8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 10 monclient: get_auth_request con 0x7f5ec41363b0 auth_method 0 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 10 monclient(hunting): handle_auth_done global_id 4349 payload 824 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.344+0000 7f5ec37fe640 1 -- 10.20.193.78:0/4079255524 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5ec406cc70 con 0x7f5ec41363b0 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.347+0000 7f5ec37fe640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec41363b0 msgr2=0x7f5ec41d8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.347+0000 7f5ec37fe640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec41363b0 0x7f5ec41d8490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005030 tx=0x7f5eb0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.347+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00067d0 con 0x7f5ec41363b0 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.347+0000 7f5e9ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.347+0000 7f5e9ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.347+0000 7f5e9ffff640 10 monclient: dump: 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.384 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4707 (secure 0 0 0) 0x7f5eb00067d0 con 0x7f5ec41363b0 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5e9ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5e9ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec41363b0 msgr2=0x7f5ec41d8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5e9ffff640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec41363b0 0x7f5ec41d8490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005030 tx=0x7f5eb0004a40 comp rx=0 tx=0).stop 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5e9ffff640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec41363b0 msgr2=0x7f5ec41d8490 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5e9ffff640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec41363b0 0x7f5ec41d8490 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005030 tx=0x7f5eb0004a40 comp rx=0 tx=0).stop 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:46:17.385 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 4, 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 27152, 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 26, 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.386 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:17.387 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.388 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.389 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.390 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:46:17.391 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:46:17.392 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5e9403deb0 msgr2=0x7f5e94040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5e9403deb0 0x7f5e94040370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4136d00 tx=0x7f5eb4009040 comp rx=0 tx=0).stop 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 10 monclient: shutdown 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec409c4a0 msgr2=0x7f5ec411d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec409c4a0 0x7f5ec411d6f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5eb803a040 tx=0x7f5eb8004b40 comp rx=0 tx=0).stop 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5ec3fff640 1 -- 10.20.193.78:0/4079255524 reap_dead start 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 shutdown_connections 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 --2- 10.20.193.78:0/4079255524 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ec409c4a0 0x7f5ec411d6f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 >> 10.20.193.78:0/4079255524 conn(0x7f5ec40a6620 msgr2=0x7f5ec4070290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 shutdown_connections 2026-02-17T05:46:17.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.348+0000 7f5eca064640 1 -- 10.20.193.78:0/4079255524 wait complete. 2026-02-17T05:46:17.394 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:46:17.394 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:46:17.394 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:46:17.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.425+0000 7f5a3cdfb640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2026-02-17T05:46:17.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.425+0000 7f5a3cdfb640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2026-02-17T05:46:17.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.425+0000 7f5a3cdfb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.425+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} v 0) -- 0x7f5a3813fe70 con 0x7f5a380a4d50 2026-02-17T05:46:17.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.491+0000 7f4fd0040640 10 monclient: get_monmap_and_config 2026-02-17T05:46:17.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.491+0000 7f4fd0040640 10 monclient: build_initial_monmap 2026-02-17T05:46:17.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.491+0000 7f4fd0040640 10 monclient: monmap: 2026-02-17T05:46:17.493 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:17.493 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.493 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:17.493010+0000 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:17.493010+0000 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 1 Processor -- start 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 1 -- start start 2026-02-17T05:46:17.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient: init 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc806c890 0x7f4fc806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient(hunting): picked mon.noname-b con 0x7f4fc806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4fc809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f4fc806d1b0 0x7f4fc8135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient(hunting): picked mon.noname-c con 0x7f4fc806d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc806daf0 con 0x7f4fc809c4a0 2026-02-17T05:46:17.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f4fc80a0880 con 0x7f4fc806d1b0 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc809cc80 con 0x7f4fc806c890 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fd0040640 10 monclient(hunting): authenticate will time out at 623.046583s 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc806c890 0x7f4fc806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcd5b4640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f4fc809c4a0 0x7f4fc8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33336/0 (socket says 10.20.193.78:33336) 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcd5b4640 1 -- 10.20.193.78:0/439999412 learned_addr learned my addr 10.20.193.78:0/439999412 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient(hunting): get_auth_request con 0x7f4fc806c890 auth_method 0 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fce5b6640 1 --2- 10.20.193.78:0/439999412 >> v2:10.20.193.115:3300/0 conn(0x7f4fc806d1b0 0x7f4fc8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fce5b6640 10 monclient(hunting): get_auth_request con 0x7f4fc806d1b0 auth_method 0 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fce5b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fce5b6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fce5b6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_done global_id 4352 payload 274 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 1 -- 10.20.193.78:0/439999412 >> v2:10.20.193.115:3300/0 conn(0x7f4fc806d1b0 msgr2=0x7f4fc8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 1 --2- 10.20.193.78:0/439999412 >> v2:10.20.193.115:3300/0 conn(0x7f4fc806d1b0 0x7f4fc8135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 1 -- 10.20.193.78:0/439999412 >> v1:10.20.193.78:6789/0 conn(0x7f4fc809c4a0 legacy=0x7f4fc8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient: _finish_hunting 0 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 1 monclient: found mon.noname-b 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.492+0000 7f4fcddb5640 1 -- 10.20.193.78:0/439999412 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc809cf60 con 0x7f4fc806c890 2026-02-17T05:46:17.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fcddb5640 1 -- 10.20.193.78:0/439999412 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc806c890 msgr2=0x7f4fc806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fcddb5640 1 --2- 10.20.193.78:0/439999412 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc806c890 0x7f4fc806cc70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098a0 tx=0x7f4fb802ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d79f3cc4d047b47 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 1 -- 10.20.193.78:0/439999412 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb803c070 con 0x7f4fc806c890 2026-02-17T05:46:17.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:17.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: dump: 2026-02-17T05:46:17.503 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: _finish_auth 0 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: _check_auth_tickets 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 1 -- 10.20.193.78:0/439999412 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb802fc00 con 0x7f4fc806c890 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 1 -- 10.20.193.78:0/439999412 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8037440 con 0x7f4fc806c890 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fccdb3640 10 monclient: dump: 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 5 monclient: authenticate success, global_id 4352 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:17.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 4 set_mon_vals no callback set 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 10 monclient: shutdown 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 1 -- 10.20.193.78:0/439999412 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc806c890 msgr2=0x7f4fc806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 1 --2- 10.20.193.78:0/439999412 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc806c890 0x7f4fc806cc70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098a0 tx=0x7f4fb802ed80 comp rx=0 tx=0).stop 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 1 -- 10.20.193.78:0/439999412 shutdown_connections 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 1 --2- 10.20.193.78:0/439999412 >> v2:10.20.193.115:3300/0 conn(0x7f4fc806d1b0 0x7f4fc8135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 1 --2- 10.20.193.78:0/439999412 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc806c890 0x7f4fc806cc70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.495+0000 7f4fd0040640 1 -- 10.20.193.78:0/439999412 >> 10.20.193.78:0/439999412 conn(0x7f4fc80a6620 msgr2=0x7f4fc80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 -- 10.20.193.78:0/439999412 shutdown_connections 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 -- 10.20.193.78:0/439999412 wait complete. 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient: build_initial_monmap 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient: monmap: 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:17.506 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 Processor -- start 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 -- start start 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient: init 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4fc806c890 0x7f4fc81cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient(hunting): picked mon.noname-a con 0x7f4fc806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc806d1b0 0x7f4fc81cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient(hunting): picked mon.noname-b con 0x7f4fc806d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc809c4a0 0x7f4fc8123520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient(hunting): picked mon.noname-c con 0x7f4fc809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80a01a0 con 0x7f4fc806c890 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80a0020 con 0x7f4fc806d1b0 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc809be20 con 0x7f4fc809c4a0 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fd0040640 10 monclient(hunting): authenticate will time out at 623.050934s 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fcddb5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4fc806c890 0x7f4fc81cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.496+0000 7f4fce5b6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc809c4a0 0x7f4fc8123520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4fc806c890 0x7f4fc81cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:50568/0 (socket says 10.20.193.78:50568) 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc809c4a0 0x7f4fc8123520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47850/0 (socket says 10.20.193.78:47850) 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 1 -- 10.20.193.78:0/2583571454 learned_addr learned my addr 10.20.193.78:0/2583571454 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 10 monclient(hunting): get_auth_request con 0x7f4fc806c890 auth_method 0 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient(hunting): get_auth_request con 0x7f4fc809c4a0 auth_method 0 2026-02-17T05:46:17.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcd5b4640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc806d1b0 0x7f4fc81cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcd5b4640 10 monclient(hunting): get_auth_request con 0x7f4fc806d1b0 auth_method 0 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcd5b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcd5b4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcd5b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient(hunting): handle_auth_done global_id 4355 payload 824 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc806d1b0 msgr2=0x7f4fc81cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc806d1b0 0x7f4fc81cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4fc806c890 msgr2=0x7f4fc81cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4fc806c890 0x7f4fc81cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient: _finish_hunting 0 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 1 monclient: found mon.noname-c 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fce5b6640 1 -- 10.20.193.78:0/2583571454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc8125c40 con 0x7f4fc809c4a0 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.497+0000 7f4fcddb5640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4fc806c890 0x7f4fc81cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fce5b6640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc809c4a0 msgr2=0x7f4fc8123520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fce5b6640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc809c4a0 0x7f4fc8123520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4010fa0 tx=0x7f4fc40077d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4003a90 con 0x7f4fc809c4a0 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: dump: 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.510 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: _finish_auth 0 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc4003c50 con 0x7f4fc809c4a0 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400e550 con 0x7f4fc809c4a0 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fd0040640 5 monclient: authenticate success, global_id 4355 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb7fff640 4 set_mon_vals no callback set 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fd0040640 10 monclient: _renew_subs 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fd0040640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc81270f0 con 0x7f4fc809c4a0 2026-02-17T05:46:17.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fb67fc640 10 monclient: dump: 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fd0040640 10 monclient: _renew_subs 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fd0040640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.499+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc8126e10 con 0x7f4fc809c4a0 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4fc400e730 con 0x7f4fc809c4a0 2026-02-17T05:46:17.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fd0040640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fd0040640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8123a60 0x7f4fc8123e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fd0040640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8074980 con 0x7f4fc8123a60 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fb67fc640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f9803deb0 0x7f4f98040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8123a60 0x7f4fc8123e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4694+0+0 (secure 0 0 0) 0x7f4fc4056a60 con 0x7f4fc809c4a0 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 10 monclient: get_auth_request con 0x7f4fc8123a60 auth_method 0 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcd5b4640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f9803deb0 0x7f4f98040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcd5b4640 10 monclient: get_auth_request con 0x7f4f9803deb0 auth_method 0 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_done global_id 4342 payload 824 2026-02-17T05:46:17.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.500+0000 7f4fcddb5640 1 -- 10.20.193.78:0/2583571454 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4fc806cc70 con 0x7f4fc8123a60 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.502+0000 7f4fcddb5640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8123a60 msgr2=0x7f4fc8123e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.502+0000 7f4fcddb5640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8123a60 0x7f4fc8123e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098a0 tx=0x7f4fb80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8048070 con 0x7f4fc8123a60 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 10 monclient: dump: 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.516 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fcd5b4640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f9803deb0 0x7f4f98040370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8124ce0 tx=0x7f4fbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4fb8041850 con 0x7f4fc8123a60 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8123a60 msgr2=0x7f4fc8123e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8123a60 0x7f4fc8123e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098a0 tx=0x7f4fb80047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8123a60 msgr2=0x7f4fc8123e40 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.503+0000 7f4fb67fc640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8123a60 0x7f4fc8123e40 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098a0 tx=0x7f4fb80047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.506+0000 7f4fd0040640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:46:17.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.506+0000 7f4fd0040640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8124380 0x7f4fc81d6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.506+0000 7f4fd0040640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.506+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc809fb30 con 0x7f4fc8124380 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8124380 0x7f4fc81d6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 10 monclient: get_auth_request con 0x7f4fc8124380 auth_method 0 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 10 monclient(hunting): handle_auth_done global_id 4345 payload 824 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.507+0000 7f4fcddb5640 1 -- 10.20.193.78:0/2583571454 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4fc80744d0 con 0x7f4fc8124380 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fcddb5640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8124380 msgr2=0x7f4fc81d6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fcddb5640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8124380 0x7f4fc81d6320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8074b00 tx=0x7f4fb8009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb803c040 con 0x7f4fc8124380 2026-02-17T05:46:17.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 10 monclient: dump: 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4874 (secure 0 0 0) 0x7f4fb80403a0 con 0x7f4fc8124380 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8124380 msgr2=0x7f4fc81d6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8124380 0x7f4fc81d6320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8074b00 tx=0x7f4fb8009290 comp rx=0 tx=0).stop 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8124380 msgr2=0x7f4fc81d6320 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.509+0000 7f4fb67fc640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4fc8124380 0x7f4fc81d6320 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8074b00 tx=0x7f4fb8009290 comp rx=0 tx=0).stop 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 4, 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 27277, 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:46:17.520 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 26, 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.521 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.522 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:46:17.523 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:46:17.524 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.525 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:46:17.526 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:17.527 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:17.528 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.529 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.530 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:17.531 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:46:17.532 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:46:17.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f9803deb0 msgr2=0x7f4f98040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f9803deb0 0x7f4f98040370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8124ce0 tx=0x7f4fbc009040 comp rx=0 tx=0).stop 2026-02-17T05:46:17.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 10 monclient: shutdown 2026-02-17T05:46:17.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:17.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc809c4a0 msgr2=0x7f4fc8123520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:17.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 1 --2- 10.20.193.78:0/2583571454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4fc809c4a0 0x7f4fc8123520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4010fa0 tx=0x7f4fc40077d0 comp rx=0 tx=0).stop 2026-02-17T05:46:17.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fce5b6640 1 -- 10.20.193.78:0/2583571454 reap_dead start 2026-02-17T05:46:17.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fce5b6640 1 -- 10.20.193.78:0/2583571454 reap_dead start 2026-02-17T05:46:17.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 shutdown_connections 2026-02-17T05:46:17.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 >> 10.20.193.78:0/2583571454 conn(0x7f4fc80a6620 msgr2=0x7f4fc8072240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:17.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 shutdown_connections 2026-02-17T05:46:17.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:17.510+0000 7f4fd0040640 1 -- 10.20.193.78:0/2583571454 wait complete. 2026-02-17T05:46:17.536 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:46:17.536 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T05:46:17.536 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2026-02-17T05:46:17.536 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T05:46:17.536 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T05:46:17.537 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T05:46:17.537 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:46:17.546 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T05:46:17.546 INFO:tasks.mon_thrash:Sending STOP to mon c 2026-02-17T05:46:17.547 INFO:tasks.ceph.mon.c:Sent signal 19 2026-02-17T05:46:17.547 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T05:46:20.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:20.336+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:20.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:20.336+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:20.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:20.336+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:20.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:20.336+0000 7f5a1dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:21.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:21.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:21.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:21.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:21.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:21.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:22.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:22.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:22.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:22.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:22.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:22.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:23.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:23.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:23.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:23.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:24.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:24.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:24.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:24.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:24.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:24.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:25.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:25.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:25.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:25.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:25.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:25.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:26.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:26.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:26.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:26.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:26.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:26.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:27.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:27.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:27.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:27.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:27.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:27.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:28.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:28.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:28.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:28.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:28.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:28.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:29.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:29.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:29.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:29.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:29.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:30.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:30.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:30.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:30.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:30.337+0000 7f5a1dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:31.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:31.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:31.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:32.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:32.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:32.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:32.547 INFO:tasks.mon_thrash:Sending CONT to mon c 2026-02-17T05:46:32.548 INFO:tasks.ceph.mon.c:Sent signal 18 2026-02-17T05:46:32.548 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:46:32.548 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.616+0000 7fc6a47a8640 10 monclient: get_monmap_and_config 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.616+0000 7fc6a47a8640 10 monclient: build_initial_monmap 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient: monmap: 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:32.617216+0000 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:32.617216+0000 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:32.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 1 Processor -- start 2026-02-17T05:46:32.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 1 -- start start 2026-02-17T05:46:32.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient: init 2026-02-17T05:46:32.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:32.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2026-02-17T05:46:32.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:32.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient(hunting): picked mon.noname-b con 0x7fc69c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:32.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 0x7fc69c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient(hunting): picked mon.noname-c con 0x7fc69c09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient(hunting): picked mon.noname-a con 0x7fc69c134df0 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c0a0880 con 0x7fc69c134df0 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c06d6f0 con 0x7fc69c09c4a0 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c06daf0 con 0x7fc69c06c890 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a47a8640 10 monclient(hunting): authenticate will time out at 638.170762s 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a1d1c640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 msgr2=0x7fc69c06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a1d1c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 0x7fc69c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36454/0 (socket says 10.20.193.78:36454) 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 learned_addr learned my addr 10.20.193.78:0/2028782340 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47388/0 (socket says 10.20.193.78:47388) 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 10 monclient(hunting): get_auth_request con 0x7fc69c06c890 auth_method 0 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:32.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 10 monclient(hunting): get_auth_request con 0x7fc69c134df0 auth_method 0 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.617+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:32.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a1d1c640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 msgr2=0x7fc69c06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:46:32.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a1d1c640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 0x7fc69c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:46:32.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:32.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 10 monclient(hunting): get_auth_request con 0x7fc69c06c890 auth_method 0 2026-02-17T05:46:32.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:32.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:32.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:32.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:32.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 10 monclient(hunting): get_auth_request con 0x7fc69c134df0 auth_method 0 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:32.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:32.818+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:46:33.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:33.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 10 monclient(hunting): get_auth_request con 0x7fc69c134df0 auth_method 0 2026-02-17T05:46:33.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:33.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:33.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:33.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:33.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a1d1c640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 msgr2=0x7fc69c06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 10 monclient(hunting): get_auth_request con 0x7fc69c06c890 auth_method 0 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a1d1c640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 0x7fc69c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:33.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.219+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:46:33.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:33.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:33.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:33.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:34.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:34.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a1d1c640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 msgr2=0x7fc69c06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:46:34.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 10 monclient(hunting): get_auth_request con 0x7fc69c134df0 auth_method 0 2026-02-17T05:46:34.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a1d1c640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 0x7fc69c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 10 monclient(hunting): get_auth_request con 0x7fc69c06c890 auth_method 0 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:46:34.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.020+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:46:34.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:34.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:34.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:34.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:35.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 10 monclient(hunting): tick 2026-02-17T05:46:35.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:46:35.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 monclient(hunting): continuing hunt 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 msgr2=0x7fc69c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc69c06c890 0x7fc69c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 msgr2=0x7fc69c06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.115:3300/0 conn(0x7fc69c09c4a0 0x7fc69c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 -- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 msgr2=0x7fc69c135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 --2- 10.20.193.78:0/2028782340 >> v2:10.20.193.78:3300/0 conn(0x7fc69c134df0 0x7fc69c135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 10 monclient: _add_conns ranks=[1,0] 2026-02-17T05:46:35.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc67c003770 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc67c0074c0 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 -- 10.20.193.78:0/2028782340 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c09cc80 con 0x7fc67c0074c0 2026-02-17T05:46:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 1 -- 10.20.193.78:0/2028782340 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc67c00c200 con 0x7fc67c003770 2026-02-17T05:46:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc683fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.617+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> v1:10.20.193.115:6789/0 conn(0x7fc67c003770 legacy=0x7fc67c005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:46:35.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.620+0000 7fc6a1d1c640 1 -- 10.20.193.78:0/2028782340 >> v1:10.20.193.78:6789/0 conn(0x7fc67c0074c0 legacy=0x7fc67c009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:35.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:35.818+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> v1:10.20.193.115:6789/0 conn(0x7fc67c003770 legacy=0x7fc67c005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:46:36.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:36.219+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> v1:10.20.193.115:6789/0 conn(0x7fc67c003770 legacy=0x7fc67c005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:46:36.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:36.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:36.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:36.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:36.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:36.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.020+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> v1:10.20.193.115:6789/0 conn(0x7fc67c003770 legacy=0x7fc67c005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:46:37.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:37.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:37.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.556+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 106110655 0 0) 0x7fc69c09cc80 con 0x7fc67c0074c0 2026-02-17T05:46:37.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.556+0000 7fc6a151b640 10 monclient(hunting): _negotiate 2026-02-17T05:46:37.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.556+0000 7fc6a151b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.556+0000 7fc6a151b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.556+0000 7fc6a151b640 10 monclient(hunting): my global_id is 4365 2026-02-17T05:46:37.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.556+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc684003620 con 0x7fc67c0074c0 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.557+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2338252603 0 0) 0x7fc684003620 con 0x7fc67c0074c0 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.557+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 >> v1:10.20.193.115:6789/0 conn(0x7fc67c003770 legacy=0x7fc67c005bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: _finish_hunting 0 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c00d460 con 0x7fc67c0074c0 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: _finish_auth 0 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: _check_auth_tickets 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fc690004600 con 0x7fc67c0074c0 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: dump: 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 >> v1:10.20.193.78:6789/0 conn(0x7fc67c0074c0 legacy=0x7fc67c009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 reap_dead start 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc684003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient(hunting): picked mon.a con 0x7fc684003ae0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc684004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient(hunting): picked mon.c con 0x7fc684004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 0x7fc68400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient(hunting): picked mon.b con 0x7fc6840054c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc68400c320 con 0x7fc684003ae0 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc68400c4d0 con 0x7fc6840054c0 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc68400c6d0 con 0x7fc684004900 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a151b640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc684003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a1d1c640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc684004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 msgr2=0x7fc68400bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient(hunting): get_auth_request con 0x7fc684003ae0 auth_method 0 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 0x7fc68400bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a1d1c640 10 monclient(hunting): get_auth_request con 0x7fc684004900 auth_method 0 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a1d1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a1d1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a1d1c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a1d1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a1d1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a1d1c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient(hunting): handle_auth_done global_id 4365 payload 293 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 msgr2=0x7fc684004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:37.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc684004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 msgr2=0x7fc68400bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 0x7fc68400bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient: _finish_hunting 0 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 1 monclient: found mon.a 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc684003620 con 0x7fc684003ae0 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient: _finish_auth 0 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 10 monclient: _check_auth_tickets 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a251d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a47a8640 5 monclient: authenticate success, global_id 4365 2026-02-17T05:46:37.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.558+0000 7fc6a47a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.560+0000 7fc6a251d640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 msgr2=0x7fc684003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.560+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc684003ec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc68c006fd0 tx=0x7fc68c02fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b61baf96cf0238c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c036070 con 0x7fc684003ae0 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 10 monclient: dump: 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.569 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c030650 con 0x7fc684003ae0 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 1 -- 10.20.193.78:0/2028782340 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c003750 con 0x7fc684003ae0 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 4 set_mon_vals no callback set 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a151b640 10 monclient: dump: 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.570 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 10 monclient: shutdown 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 msgr2=0x7fc684003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc684003ec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc68c006fd0 tx=0x7fc68c02fe30 comp rx=0 tx=0).stop 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2028782340 shutdown_connections 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 0x7fc68400bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc684004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2028782340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc684003ec0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2028782340 >> 10.20.193.78:0/2028782340 conn(0x7fc69c0a6620 msgr2=0x7fc69c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2028782340 shutdown_connections 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2028782340 wait complete. 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 10 monclient: build_initial_monmap 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.561+0000 7fc6a47a8640 10 monclient: monmap: 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:37.571 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 1 Processor -- start 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 1 -- start start 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient: init 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc69c134050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient(hunting): picked mon.noname-a con 0x7fc684003ae0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc69c134590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient(hunting): picked mon.noname-c con 0x7fc684004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 0x7fc69c11eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient(hunting): picked mon.noname-b con 0x7fc6840054c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0a01a0 con 0x7fc684003ae0 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0a0020 con 0x7fc6840054c0 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c09be20 con 0x7fc684004900 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a47a8640 10 monclient(hunting): authenticate will time out at 643.115433s 2026-02-17T05:46:37.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc69c134050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc69c134590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc69c134050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39678/0 (socket says 10.20.193.78:39678) 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 -- 10.20.193.78:0/2363990666 learned_addr learned my addr 10.20.193.78:0/2363990666 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient(hunting): get_auth_request con 0x7fc684003ae0 auth_method 0 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 10 monclient(hunting): get_auth_request con 0x7fc684004900 auth_method 0 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a2d1e640 1 -- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 msgr2=0x7fc69c11eb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a2d1e640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 0x7fc69c11eb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient(hunting): handle_auth_done global_id 4368 payload 824 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 -- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 msgr2=0x7fc69c134590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc69c134590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 -- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 msgr2=0x7fc69c11eb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 0x7fc69c11eb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient: _finish_hunting 0 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 monclient: found mon.noname-a 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a251d640 1 -- 10.20.193.78:0/2363990666 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c122740 con 0x7fc684003ae0 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.562+0000 7fc6a1d1c640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc69c134590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc6a251d640 1 -- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 msgr2=0x7fc69c134050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc6a251d640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc69c134050 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc68c02fe70 tx=0x7fc68c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 1 -- 10.20.193.78:0/2363990666 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c048020 con 0x7fc684003ae0 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: dump: 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.574 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: _finish_auth 0 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 1 -- 10.20.193.78:0/2363990666 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c003520 con 0x7fc684003ae0 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 1 -- 10.20.193.78:0/2363990666 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c030e60 con 0x7fc684003ae0 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc682ffd640 10 monclient: dump: 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc6a0d1a640 4 set_mon_vals no callback set 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc6a47a8640 5 monclient: authenticate success, global_id 4368 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc6a47a8640 10 monclient: _renew_subs 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc6a47a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.564+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c122a20 con 0x7fc684003ae0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc6a47a8640 10 monclient: _renew_subs 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc6a47a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c0a1240 con 0x7fc684003ae0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc6a47a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc6a47a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc6a47a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c074980 con 0x7fc684003ae0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc682ffd640 1 -- 10.20.193.78:0/2363990666 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc68c002870 con 0x7fc684003ae0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc682ffd640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc66c03de60 0x7fc66c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.565+0000 7fc682ffd640 1 -- 10.20.193.78:0/2363990666 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4707+0+0 (secure 0 0 0) 0x7fc68c078400 con 0x7fc684003ae0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.566+0000 7fc6a1d1c640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc66c03de60 0x7fc66c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.566+0000 7fc6a1d1c640 10 monclient: get_auth_request con 0x7fc66c03de60 auth_method 0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.568+0000 7fc6a1d1c640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc66c03de60 0x7fc66c040320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc690003d60 tx=0x7fc690011040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.568+0000 7fc682ffd640 1 -- 10.20.193.78:0/2363990666 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc68c052030 con 0x7fc684003ae0 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.568+0000 7fc682ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:37.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.568+0000 7fc682ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:37.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.682+0000 7fc6a47a8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:46:37.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.682+0000 7fc6a47a8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:46:37.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.682+0000 7fc6a47a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.682+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc69c135240 con 0x7fc684003ae0 2026-02-17T05:46:37.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.682+0000 7fc682ffd640 1 -- 10.20.193.78:0/2363990666 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1549 (secure 0 0 0) 0x7fc68c0423a0 con 0x7fc684003ae0 2026-02-17T05:46:37.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.682+0000 7fc682ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:46:37.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.682+0000 7fc682ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:37.684 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":6,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc66c03de60 msgr2=0x7fc66c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc66c03de60 0x7fc66c040320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc690003d60 tx=0x7fc690011040 comp rx=0 tx=0).stop 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 10 monclient: shutdown 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 msgr2=0x7fc69c134050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc69c134050 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc68c02fe70 tx=0x7fc68c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 shutdown_connections 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc66c03de60 0x7fc66c040320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6840054c0 0x7fc69c11eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc684004900 0x7fc69c134590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 --2- 10.20.193.78:0/2363990666 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc684003ae0 0x7fc69c134050 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 >> 10.20.193.78:0/2363990666 conn(0x7fc69c0a6620 msgr2=0x7fc69c135450 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:37.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 shutdown_connections 2026-02-17T05:46:37.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.683+0000 7fc6a47a8640 1 -- 10.20.193.78:0/2363990666 wait complete. 2026-02-17T05:46:37.692 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:46:37.692 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T05:46:37.693 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:46:37.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient: get_monmap_and_config 2026-02-17T05:46:37.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient: build_initial_monmap 2026-02-17T05:46:37.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient: monmap: 2026-02-17T05:46:37.765 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:37.765 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.765 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:37.762424+0000 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:37.762424+0000 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 1 Processor -- start 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 1 -- start start 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient: init 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fee5406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee5413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient(hunting): picked mon.noname-b con 0x7fee5409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fee5406d380 0x7fee5413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient(hunting): picked mon.noname-a con 0x7fee5406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5406daf0 con 0x7fee5406c890 2026-02-17T05:46:37.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fee540a0880 con 0x7fee5406d380 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fee5409cc80 con 0x7fee5409c4a0 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5c42c640 10 monclient(hunting): authenticate will time out at 643.315953s 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.762+0000 7fee5a1a1640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fee5406c890 legacy=0x7fee5406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee5413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fee5406d380 0x7fee5413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee5413b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60904/0 (socket says 10.20.193.78:60904) 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fee5406d380 0x7fee5413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39694/0 (socket says 10.20.193.78:39694) 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 1 -- 10.20.193.78:0/1631839230 learned_addr learned my addr 10.20.193.78:0/1631839230 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:37.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient(hunting): get_auth_request con 0x7fee5409c4a0 auth_method 0 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 10 monclient(hunting): get_auth_request con 0x7fee5406d380 auth_method 0 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee5a9a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient(hunting): handle_auth_done global_id 4361 payload 274 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 1 -- 10.20.193.78:0/1631839230 >> v2:10.20.193.78:3300/0 conn(0x7fee5406d380 msgr2=0x7fee5413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 1 --2- 10.20.193.78:0/1631839230 >> v2:10.20.193.78:3300/0 conn(0x7fee5406d380 0x7fee5413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 1 -- 10.20.193.78:0/1631839230 >> v1:10.20.193.115:6789/0 conn(0x7fee5406c890 legacy=0x7fee5406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient: _finish_hunting 0 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 1 monclient: found mon.noname-b 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:46:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.763+0000 7fee599a0640 1 -- 10.20.193.78:0/1631839230 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5409cf60 con 0x7fee5409c4a0 2026-02-17T05:46:37.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee599a0640 1 -- 10.20.193.78:0/1631839230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 msgr2=0x7fee5413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee599a0640 1 --2- 10.20.193.78:0/1631839230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee5413b3a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee48009970 tx=0x7fee4802ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=201592e45067c61d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 1 -- 10.20.193.78:0/1631839230 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4803c070 con 0x7fee5409c4a0 2026-02-17T05:46:37.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:46:37.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: dump: 2026-02-17T05:46:37.781 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.781 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.781 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: _finish_auth 0 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: _check_auth_tickets 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5c42c640 5 monclient: authenticate success, global_id 4361 2026-02-17T05:46:37.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 1 -- 10.20.193.78:0/1631839230 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4802fcd0 con 0x7fee5409c4a0 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5c42c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 1 -- 10.20.193.78:0/1631839230 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48037660 con 0x7fee5409c4a0 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5c42c640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5c42c640 4 set_mon_vals no callback set 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.765+0000 7fee5919f640 10 monclient: dump: 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.783 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient: shutdown 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- 10.20.193.78:0/1631839230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 msgr2=0x7fee5413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 --2- 10.20.193.78:0/1631839230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee5413b3a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee48009970 tx=0x7fee4802ee90 comp rx=0 tx=0).stop 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- 10.20.193.78:0/1631839230 shutdown_connections 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 --2- 10.20.193.78:0/1631839230 >> v2:10.20.193.78:3300/0 conn(0x7fee5406d380 0x7fee5413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 --2- 10.20.193.78:0/1631839230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee5413b3a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- 10.20.193.78:0/1631839230 >> 10.20.193.78:0/1631839230 conn(0x7fee540a6620 msgr2=0x7fee540a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- 10.20.193.78:0/1631839230 shutdown_connections 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- 10.20.193.78:0/1631839230 wait complete. 2026-02-17T05:46:37.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient: build_initial_monmap 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient: monmap: 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 Processor -- start 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- start start 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient: init 2026-02-17T05:46:37.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5406c890 0x7fee541ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient(hunting): picked mon.noname-a con 0x7fee5406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee5406d380 0x7fee541360c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient(hunting): picked mon.noname-b con 0x7fee5406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee54136600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient(hunting): picked mon.noname-c con 0x7fee5409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee5409bfa0 con 0x7fee5406c890 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fee5409be20 con 0x7fee5406d380 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fee5413de30 con 0x7fee5409c4a0 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.766+0000 7fee5c42c640 10 monclient(hunting): authenticate will time out at 643.320004s 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5406c890 0x7fee541ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee599a0640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee5406d380 msgr2=0x7fee541360c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee599a0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee5406d380 0x7fee541360c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a9a2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee54136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5406c890 0x7fee541ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39700/0 (socket says 10.20.193.78:39700) 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 -- 10.20.193.78:0/2956265334 learned_addr learned my addr 10.20.193.78:0/2956265334 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient(hunting): get_auth_request con 0x7fee5406c890 auth_method 0 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a9a2640 10 monclient(hunting): get_auth_request con 0x7fee5409c4a0 auth_method 0 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a9a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a9a2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a9a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a9a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a9a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a9a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient(hunting): handle_auth_done global_id 4374 payload 824 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 msgr2=0x7fee54136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee5409c4a0 0x7fee54136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee5406d380 msgr2=0x7fee541360c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee5406d380 0x7fee541360c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient: _finish_hunting 0 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 monclient: found mon.noname-a 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.767+0000 7fee5a1a1640 1 -- 10.20.193.78:0/2956265334 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee541227b0 con 0x7fee5406c890 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5a1a1640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5406c890 msgr2=0x7fee541ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5a1a1640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5406c890 0x7fee541ceea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fee4400b780 tx=0x7fee4400bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440052d0 con 0x7fee5406c890 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: dump: 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: _finish_auth 0 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee44010070 con 0x7fee5406c890 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5c42c640 5 monclient: authenticate success, global_id 4374 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5c42c640 10 monclient: _renew_subs 2026-02-17T05:46:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5c42c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee541cf3e0 con 0x7fee5406c890 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440056d0 con 0x7fee5406c890 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5899e640 4 set_mon_vals no callback set 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee3affd640 10 monclient: dump: 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5c42c640 10 monclient: _renew_subs 2026-02-17T05:46:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5c42c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.769+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5413db90 con 0x7fee5406c890 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee5c42c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee5c42c640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411eb50 0x7fee5411ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fee4401d030 con 0x7fee5406c890 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee5c42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee5406f7c0 con 0x7fee5411eb50 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411eb50 0x7fee5411ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 10 monclient: get_auth_request con 0x7fee5411eb50 auth_method 0 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 10 monclient(hunting): handle_auth_done global_id 4377 payload 824 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee599a0640 1 -- 10.20.193.78:0/2956265334 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee54074980 con 0x7fee5411eb50 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee3affd640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee2403de60 0x7fee24040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4707+0+0 (secure 0 0 0) 0x7fee44051d20 con 0x7fee5406c890 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee5a9a2640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee2403de60 0x7fee24040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.770+0000 7fee5a9a2640 10 monclient: get_auth_request con 0x7fee2403de60 auth_method 0 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee599a0640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411eb50 msgr2=0x7fee5411ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee599a0640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411eb50 0x7fee5411ef30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fee5406a480 tx=0x7fee48004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48048070 con 0x7fee5411eb50 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 10 monclient: dump: 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.792 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fee4803d040 con 0x7fee5411eb50 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411eb50 msgr2=0x7fee5411ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411eb50 0x7fee5411ef30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fee5406a480 tx=0x7fee48004830 comp rx=0 tx=0).stop 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411eb50 msgr2=0x7fee5411ef30 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.772+0000 7fee3affd640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411eb50 0x7fee5411ef30 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fee5406a480 tx=0x7fee48004830 comp rx=0 tx=0).stop 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.773+0000 7fee5a9a2640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee2403de60 0x7fee24040320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee500113e0 tx=0x7fee5000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee5c42c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee5c42c640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411f470 0x7fee541da250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee5c42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee54121080 con 0x7fee5411f470 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee599a0640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411f470 0x7fee541da250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee599a0640 10 monclient: get_auth_request con 0x7fee5411f470 auth_method 0 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee599a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee599a0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee599a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.775+0000 7fee599a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.776+0000 7fee599a0640 10 monclient(hunting): handle_auth_done global_id 4380 payload 824 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.776+0000 7fee599a0640 1 -- 10.20.193.78:0/2956265334 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fee540744d0 con 0x7fee5411f470 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee599a0640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411f470 msgr2=0x7fee541da250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee599a0640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411f470 0x7fee541da250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee48009970 tx=0x7fee480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48048040 con 0x7fee5411f470 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 10 monclient: dump: 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7fee4804a3d0 con 0x7fee5411f470 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411f470 msgr2=0x7fee541da250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411f470 0x7fee541da250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee48009970 tx=0x7fee480047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411f470 msgr2=0x7fee541da250 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee3affd640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5411f470 0x7fee541da250 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee48009970 tx=0x7fee480047c0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 6, 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 47582, 2026-02-17T05:46:37.795 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:37.796 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:46:37.797 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:37.798 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:46:37.799 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:46:37.800 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.801 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:37.802 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:46:37.803 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee2403de60 msgr2=0x7fee24040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee2403de60 0x7fee24040320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee500113e0 tx=0x7fee5000b040 comp rx=0 tx=0).stop 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 10 monclient: shutdown 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5406c890 msgr2=0x7fee541ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5a9a2640 1 -- 10.20.193.78:0/2956265334 reap_dead start 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5406c890 0x7fee541ceea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fee4400b780 tx=0x7fee4400bc50 comp rx=0 tx=0).stop 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 shutdown_connections 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 --2- 10.20.193.78:0/2956265334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee5406c890 0x7fee541ceea0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 >> 10.20.193.78:0/2956265334 conn(0x7fee540a6620 msgr2=0x7fee540a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 shutdown_connections 2026-02-17T05:46:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.778+0000 7fee5c42c640 1 -- 10.20.193.78:0/2956265334 wait complete. 2026-02-17T05:46:37.806 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:46:37.806 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:46:37.806 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.873+0000 7f41a293e640 10 monclient: get_monmap_and_config 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.873+0000 7f41a293e640 10 monclient: build_initial_monmap 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.873+0000 7f41a293e640 10 monclient: monmap: 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:46:37.873672+0000 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:46:37.873672+0000 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:46:37.876 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.873+0000 7f41a293e640 1 Processor -- start 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.873+0000 7f41a293e640 1 -- start start 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.873+0000 7f41a293e640 10 monclient: init 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f419c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f419c09d430 addr v1:10.20.193.115:6789/0 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f419c09bf40 0x7f419c140d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 10 monclient(hunting): picked mon.noname-a con 0x7f419c09bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:46:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f419c073600 con 0x7f419c06c890 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f419c06dab0 con 0x7f419c09d430 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f419c09ca50 con 0x7f419c09bf40 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a293e640 10 monclient(hunting): authenticate will time out at 643.427247s 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f419b7fe640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f419c09d430 legacy=0x7f419c13d5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f419bfff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f419c06c890 0x7f419c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41846/0 (socket says 10.20.193.78:41846) 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f419bfff640 1 -- 10.20.193.78:0/2968262817 learned_addr learned my addr 10.20.193.78:0/2968262817 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 1 --2- 10.20.193.78:0/2968262817 >> v2:10.20.193.78:3300/0 conn(0x7f419c09bf40 0x7f419c140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient(hunting): get_auth_request con 0x7f419c09bf40 auth_method 0 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient(hunting): handle_auth_done global_id 4383 payload 274 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 1 -- 10.20.193.78:0/2968262817 >> v1:10.20.193.115:6789/0 conn(0x7f419c09d430 legacy=0x7f419c13d5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 1 -- 10.20.193.78:0/2968262817 >> v1:10.20.193.78:6789/0 conn(0x7f419c06c890 legacy=0x7f419c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient: _finish_hunting 0 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 1 monclient: found mon.noname-a 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.874+0000 7f41a0eb4640 1 -- 10.20.193.78:0/2968262817 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c09c770 con 0x7f419c09bf40 2026-02-17T05:46:37.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a0eb4640 1 -- 10.20.193.78:0/2968262817 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09bf40 msgr2=0x7f419c140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a0eb4640 1 --2- 10.20.193.78:0/2968262817 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09bf40 0x7f419c140d00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f419c09cbd0 tx=0x7f419002f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49484145284b021 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 1 -- 10.20.193.78:0/2968262817 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419003c070 con 0x7f419c09bf40 2026-02-17T05:46:37.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: dump: 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: _finish_auth 0 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: _check_auth_tickets 2026-02-17T05:46:37.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 1 -- 10.20.193.78:0/2968262817 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4190037640 con 0x7f419c09bf40 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 5 monclient: authenticate success, global_id 4383 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 1 -- 10.20.193.78:0/2968262817 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190037a10 con 0x7f419c09bf40 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 10 monclient: get_monmap_and_config success 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 4 set_mon_vals no callback set 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f419affd640 10 monclient: dump: 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.887 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 10 monclient: shutdown 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 1 -- 10.20.193.78:0/2968262817 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09bf40 msgr2=0x7f419c140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 1 --2- 10.20.193.78:0/2968262817 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09bf40 0x7f419c140d00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f419c09cbd0 tx=0x7f419002f150 comp rx=0 tx=0).stop 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 1 -- 10.20.193.78:0/2968262817 shutdown_connections 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 1 --2- 10.20.193.78:0/2968262817 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09bf40 0x7f419c140d00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 1 -- 10.20.193.78:0/2968262817 >> 10.20.193.78:0/2968262817 conn(0x7f419c0a6620 msgr2=0x7f419c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 1 -- 10.20.193.78:0/2968262817 shutdown_connections 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 1 -- 10.20.193.78:0/2968262817 wait complete. 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 10 monclient: build_initial_monmap 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.877+0000 7f41a293e640 10 monclient: monmap: 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 1 Processor -- start 2026-02-17T05:46:37.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 1 -- start start 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient: init 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient: _reopen_session rank -1 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f419c06c890 0x7f419c12e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient(hunting): picked mon.noname-b con 0x7f419c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c09bf40 0x7f419c12e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient(hunting): picked mon.noname-c con 0x7f419c09bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09d430 0x7f419c1340a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient(hunting): picked mon.noname-a con 0x7f419c09d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f419c06d380 con 0x7f419c09d430 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f419c073410 con 0x7f419c06c890 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f419c06d500 con 0x7f419c09bf40 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient(hunting): _renew_subs 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a293e640 10 monclient(hunting): authenticate will time out at 643.431409s 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419bfff640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f419c06c890 msgr2=0x7f419c12e080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c09bf40 0x7f419c12e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419bfff640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f419c06c890 0x7f419c12e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09d430 0x7f419c1340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c09bf40 0x7f419c12e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60930/0 (socket says 10.20.193.78:60930) 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 1 -- 10.20.193.78:0/3838382024 learned_addr learned my addr 10.20.193.78:0/3838382024 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09d430 0x7f419c1340a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39732/0 (socket says 10.20.193.78:39732) 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 10 monclient(hunting): get_auth_request con 0x7f419c09bf40 auth_method 0 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient(hunting): get_auth_request con 0x7f419c09d430 auth_method 0 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient(hunting): handle_auth_done global_id 4386 payload 824 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c09bf40 msgr2=0x7f419c12e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c09bf40 0x7f419c12e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f419c06c890 msgr2=0x7f419c12e080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f419c06c890 0x7f419c12e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient: _finish_hunting 0 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 1 monclient: found mon.noname-a 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:46:37.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f41a0eb4640 1 -- 10.20.193.78:0/3838382024 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c135d00 con 0x7f419c09d430 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.878+0000 7f419b7fe640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c09bf40 0x7f419c12e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f41a0eb4640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09d430 msgr2=0x7f419c1340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f41a0eb4640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09d430 0x7f419c1340a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4190039000 tx=0x7f4190004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190047020 con 0x7f419c09d430 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: dump: 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.893 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: _finish_auth 0 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419003d040 con 0x7f419c09d430 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419003c070 con 0x7f419c09d430 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f4198ff9640 10 monclient: dump: 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f419a7fc640 4 set_mon_vals no callback set 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f41a293e640 5 monclient: authenticate success, global_id 4386 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f41a293e640 10 monclient: _renew_subs 2026-02-17T05:46:37.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f41a293e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.880+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419c09d810 con 0x7f419c09d430 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f41a293e640 10 monclient: _renew_subs 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f41a293e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419c13baf0 con 0x7f419c09d430 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f41a293e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f41a293e640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c1345e0 0x7f419c1349c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f41a293e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f419c06cc70 con 0x7f419c1345e0 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c1345e0 0x7f419c1349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 10 monclient: get_auth_request con 0x7f419c1345e0 auth_method 0 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f419004c410 con 0x7f419c09d430 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 10 monclient(hunting): handle_auth_done global_id 4370 payload 824 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f419bfff640 1 -- 10.20.193.78:0/3838382024 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f419c0008d0 con 0x7f419c1345e0 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f4198ff9640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f416c03deb0 0x7f416c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.881+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4707+0+0 (secure 0 0 0) 0x7f41900782d0 con 0x7f419c09d430 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.882+0000 7f419b7fe640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f416c03deb0 0x7f416c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.882+0000 7f419b7fe640 10 monclient: get_auth_request con 0x7f416c03deb0 auth_method 0 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.883+0000 7f419bfff640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c1345e0 msgr2=0x7f419c1349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.883+0000 7f419bfff640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c1345e0 0x7f419c1349c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4188004970 tx=0x7f418800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.883+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188017070 con 0x7f419c1345e0 2026-02-17T05:46:37.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.883+0000 7f4198ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.883+0000 7f4198ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.883+0000 7f4198ff9640 10 monclient: dump: 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.884+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f41880072f0 con 0x7f419c1345e0 2026-02-17T05:46:37.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.884+0000 7f4198ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:46:37.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.884+0000 7f4198ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:46:37.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.884+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c1345e0 msgr2=0x7f419c1349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.884+0000 7f4198ff9640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c1345e0 0x7f419c1349c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4188004970 tx=0x7f418800c8f0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.884+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c1345e0 msgr2=0x7f419c1349c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:37.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.884+0000 7f4198ff9640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c1345e0 0x7f419c1349c0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4188004970 tx=0x7f418800c8f0 comp rx=0 tx=0).stop 2026-02-17T05:46:37.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.884+0000 7f419b7fe640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f416c03deb0 0x7f416c040370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f418c006fd0 tx=0x7f418c006f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f41a293e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:46:37.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f41a293e640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c134f00 0x7f419c1da580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:46:37.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f41a293e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:46:37.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f419c140510 con 0x7f419c134f00 2026-02-17T05:46:37.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c134f00 0x7f419c1da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:46:37.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 10 monclient: get_auth_request con 0x7f419c134f00 auth_method 0 2026-02-17T05:46:37.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:46:37.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 10 monclient(hunting): handle_auth_done global_id 4373 payload 824 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.887+0000 7f419bfff640 1 -- 10.20.193.78:0/3838382024 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f419c12eb00 con 0x7f419c134f00 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f419bfff640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c134f00 msgr2=0x7f419c1da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f419bfff640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c134f00 0x7f419c1da580 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4188004970 tx=0x7f4188004a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188017070 con 0x7f419c134f00 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 10 monclient: dump: 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:46:37.902 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4695 (secure 0 0 0) 0x7f4188003030 con 0x7f419c134f00 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c134f00 msgr2=0x7f419c1da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c134f00 0x7f419c1da580 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4188004970 tx=0x7f4188004a70 comp rx=0 tx=0).stop 2026-02-17T05:46:37.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c134f00 msgr2=0x7f419c1da580 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f4198ff9640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f419c134f00 0x7f419c1da580 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4188004970 tx=0x7f4188004a70 comp rx=0 tx=0).stop 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 6, 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 47694, 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:46:37.904 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:37.905 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:37.906 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:37.907 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:46:37.908 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.909 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:37.910 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.911 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.912 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:46:37.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:46:37.914 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.915 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:46:37.916 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f416c03deb0 msgr2=0x7f416c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f416c03deb0 0x7f416c040370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f418c006fd0 tx=0x7f418c006f00 comp rx=0 tx=0).stop 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 10 monclient: shutdown 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09d430 msgr2=0x7f419c1340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 1 --2- 10.20.193.78:0/3838382024 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f419c09d430 0x7f419c1340a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4190039000 tx=0x7f4190004b40 comp rx=0 tx=0).stop 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a0eb4640 1 -- 10.20.193.78:0/3838382024 reap_dead start 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a0eb4640 1 -- 10.20.193.78:0/3838382024 reap_dead start 2026-02-17T05:46:37.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 shutdown_connections 2026-02-17T05:46:37.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 >> 10.20.193.78:0/3838382024 conn(0x7f419c0a6620 msgr2=0x7f419c136a50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:37.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 shutdown_connections 2026-02-17T05:46:37.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:37.889+0000 7f41a293e640 1 -- 10.20.193.78:0/3838382024 wait complete. 2026-02-17T05:46:37.919 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:46:37.919 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:46:38.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.337+0000 7f5a1dffb640 10 monclient: tick 2026-02-17T05:46:38.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.337+0000 7f5a1dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:46:38.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.337+0000 7f5a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:46:38.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.555+0000 7f5a1effd640 1 -- 10.20.193.78:0/3802867093 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]=0 enabled application 'rados' on pool 'unique_pool_0' v24) ==== 197+0+0 (secure 0 0 0) 0x7f5a28050020 con 0x7f5a380a4d50 2026-02-17T05:46:38.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.555+0000 7f5a1effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2026-02-17T05:46:38.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.555+0000 7f5a1effd640 10 monclient: _finish_command 2 = system:0 enabled application 'rados' on pool 'unique_pool_0' 2026-02-17T05:46:38.556 INFO:teuthology.orchestra.run.trial078.stderr:enabled application 'rados' on pool 'unique_pool_0' 2026-02-17T05:46:38.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5a0403df30 msgr2=0x7f5a040403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:38.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5a0403df30 0x7f5a040403f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c004020 tx=0x7f5a2c009040 comp rx=0 tx=0).stop 2026-02-17T05:46:38.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 10 monclient: shutdown 2026-02-17T05:46:38.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:46:38.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a4d50 msgr2=0x7f5a38138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:46:38.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a4d50 0x7f5a38138460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009f50 tx=0x7f5a28004b40 comp rx=0 tx=0).stop 2026-02-17T05:46:38.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 shutdown_connections 2026-02-17T05:46:38.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5a380a4d50 0x7f5a38138460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:38.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5a380a3cc0 0x7f5a38137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:38.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5a0403df30 0x7f5a040403f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:38.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 --2- 10.20.193.78:0/3802867093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5a380a3850 0x7f5a3813f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:46:38.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 >> 10.20.193.78:0/3802867093 conn(0x7f5a380a4230 msgr2=0x7f5a380a5530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:46:38.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 shutdown_connections 2026-02-17T05:46:38.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:46:38.557+0000 7f5a3cdfb640 1 -- 10.20.193.78:0/3802867093 wait complete. 2026-02-17T05:46:39.567 DEBUG:teuthology.orchestra.run.trial115:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0 2026-02-17T05:46:39.595 INFO:tasks.rados.rados.0.trial115.stdout:adding op weight read -> 100 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:adding op weight write -> 50 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:adding op weight delete -> 50 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:adding op weight snap_create -> 50 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:adding op weight snap_remove -> 50 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:adding op weight rollback -> 50 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:adding op weight copy_from -> 50 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:adding op weight write_excl -> 50 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:ceph version 19.2.3-705-g04cf6dd4 (04cf6dd4be446d06026adad18a5fbf5da1fa8c0e) squid (stable) 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout:Configuration: 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout: Number of operations: 4000 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout: Number of objects: 50 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout: Max in flight operations: 16 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout: Object size (in bytes): 4000000 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout: Write stride min: 400000 2026-02-17T05:46:39.596 INFO:tasks.rados.rados.0.trial115.stdout: Write stride max: 800000 2026-02-17T05:46:40.557 INFO:tasks.rados.rados.0.trial115.stdout:1: write initial oid 1 2026-02-17T05:46:40.557 INFO:tasks.rados.rados.0.trial115.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2026-02-17T05:46:40.560 INFO:tasks.rados.rados.0.trial115.stdout:1: writing trial11545420-1 from 416807 to 892056 tid 1 2026-02-17T05:46:40.565 INFO:tasks.rados.rados.0.trial115.stdout:1: writing trial11545420-1 from 1542129 to 2016807 tid 2 2026-02-17T05:46:40.565 INFO:tasks.rados.rados.0.trial115.stdout:2: write initial oid 2 2026-02-17T05:46:40.565 INFO:tasks.rados.rados.0.trial115.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2026-02-17T05:46:40.568 INFO:tasks.rados.rados.0.trial115.stdout:2: writing trial11545420-2 from 416807 to 892056 tid 1 2026-02-17T05:46:40.573 INFO:tasks.rados.rados.0.trial115.stdout:2: writing trial11545420-2 from 1542129 to 2016807 tid 2 2026-02-17T05:46:40.573 INFO:tasks.rados.rados.0.trial115.stdout:3: write initial oid 3 2026-02-17T05:46:40.573 INFO:tasks.rados.rados.0.trial115.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2026-02-17T05:46:40.577 INFO:tasks.rados.rados.0.trial115.stdout:3: writing trial11545420-3 from 433614 to 984112 tid 1 2026-02-17T05:46:40.581 INFO:tasks.rados.rados.0.trial115.stdout:3: writing trial11545420-3 from 1600611 to 2033614 tid 2 2026-02-17T05:46:40.581 INFO:tasks.rados.rados.0.trial115.stdout:4: write initial oid 4 2026-02-17T05:46:40.581 INFO:tasks.rados.rados.0.trial115.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2026-02-17T05:46:40.585 INFO:tasks.rados.rados.0.trial115.stdout:4: writing trial11545420-4 from 450421 to 1076168 tid 1 2026-02-17T05:46:40.589 INFO:tasks.rados.rados.0.trial115.stdout:4: writing trial11545420-4 from 1659093 to 2050421 tid 2 2026-02-17T05:46:40.589 INFO:tasks.rados.rados.0.trial115.stdout:5: write initial oid 5 2026-02-17T05:46:40.589 INFO:tasks.rados.rados.0.trial115.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2026-02-17T05:46:40.593 INFO:tasks.rados.rados.0.trial115.stdout:5: writing trial11545420-5 from 467228 to 1168224 tid 1 2026-02-17T05:46:40.596 INFO:tasks.rados.rados.0.trial115.stdout:5: writing trial11545420-5 from 1717575 to 2067228 tid 2 2026-02-17T05:46:40.596 INFO:tasks.rados.rados.0.trial115.stdout:6: write initial oid 6 2026-02-17T05:46:40.597 INFO:tasks.rados.rados.0.trial115.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2026-02-17T05:46:40.602 INFO:tasks.rados.rados.0.trial115.stdout:6: writing trial11545420-6 from 484035 to 1260280 tid 1 2026-02-17T05:46:40.605 INFO:tasks.rados.rados.0.trial115.stdout:6: writing trial11545420-6 from 2059704 to 2084035 tid 2 2026-02-17T05:46:40.605 INFO:tasks.rados.rados.0.trial115.stdout:7: write initial oid 7 2026-02-17T05:46:40.605 INFO:tasks.rados.rados.0.trial115.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2026-02-17T05:46:40.608 INFO:tasks.rados.rados.0.trial115.stdout:7: writing trial11545420-7 from 500842 to 952336 tid 1 2026-02-17T05:46:40.613 INFO:tasks.rados.rados.0.trial115.stdout:7: writing trial11545420-7 from 1718186 to 2100842 tid 2 2026-02-17T05:46:40.613 INFO:tasks.rados.rados.0.trial115.stdout:8: write initial oid 8 2026-02-17T05:46:40.613 INFO:tasks.rados.rados.0.trial115.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2026-02-17T05:46:40.616 INFO:tasks.rados.rados.0.trial115.stdout:8: writing trial11545420-8 from 517649 to 1044392 tid 1 2026-02-17T05:46:40.621 INFO:tasks.rados.rados.0.trial115.stdout:8: writing trial11545420-8 from 1776668 to 2117649 tid 2 2026-02-17T05:46:40.621 INFO:tasks.rados.rados.0.trial115.stdout:9: write initial oid 9 2026-02-17T05:46:40.621 INFO:tasks.rados.rados.0.trial115.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2026-02-17T05:46:40.626 INFO:tasks.rados.rados.0.trial115.stdout:9: writing trial11545420-9 from 534456 to 1252801 tid 1 2026-02-17T05:46:40.629 INFO:tasks.rados.rados.0.trial115.stdout:9: writing trial11545420-9 from 1951503 to 2134456 tid 2 2026-02-17T05:46:40.630 INFO:tasks.rados.rados.0.trial115.stdout:10: write initial oid 10 2026-02-17T05:46:40.630 INFO:tasks.rados.rados.0.trial115.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2026-02-17T05:46:40.634 INFO:tasks.rados.rados.0.trial115.stdout:10: writing trial11545420-10 from 551263 to 1344857 tid 1 2026-02-17T05:46:40.637 INFO:tasks.rados.rados.0.trial115.stdout:10: writing trial11545420-10 from 1893632 to 2151263 tid 2 2026-02-17T05:46:40.638 INFO:tasks.rados.rados.0.trial115.stdout:11: write initial oid 11 2026-02-17T05:46:40.638 INFO:tasks.rados.rados.0.trial115.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2026-02-17T05:46:40.642 INFO:tasks.rados.rados.0.trial115.stdout:11: writing trial11545420-11 from 568070 to 1036913 tid 1 2026-02-17T05:46:40.647 INFO:tasks.rados.rados.0.trial115.stdout:11: writing trial11545420-11 from 1552114 to 2168070 tid 2 2026-02-17T05:46:40.647 INFO:tasks.rados.rados.0.trial115.stdout:12: write initial oid 12 2026-02-17T05:46:40.647 INFO:tasks.rados.rados.0.trial115.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2026-02-17T05:46:40.651 INFO:tasks.rados.rados.0.trial115.stdout:12: writing trial11545420-12 from 584877 to 1128969 tid 1 2026-02-17T05:46:40.656 INFO:tasks.rados.rados.0.trial115.stdout:12: writing trial11545420-12 from 1610596 to 2172599 tid 2 2026-02-17T05:46:40.656 INFO:tasks.rados.rados.0.trial115.stdout:13: write initial oid 13 2026-02-17T05:46:40.656 INFO:tasks.rados.rados.0.trial115.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2026-02-17T05:46:40.661 INFO:tasks.rados.rados.0.trial115.stdout:13: writing trial11545420-13 from 601684 to 1221025 tid 1 2026-02-17T05:46:40.665 INFO:tasks.rados.rados.0.trial115.stdout:13: writing trial11545420-13 from 1669078 to 2201684 tid 2 2026-02-17T05:46:40.665 INFO:tasks.rados.rados.0.trial115.stdout:14: write initial oid 14 2026-02-17T05:46:40.665 INFO:tasks.rados.rados.0.trial115.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2026-02-17T05:46:40.670 INFO:tasks.rados.rados.0.trial115.stdout:14: writing trial11545420-14 from 618491 to 1313081 tid 1 2026-02-17T05:46:40.673 INFO:tasks.rados.rados.0.trial115.stdout:14: writing trial11545420-14 from 2011207 to 2218491 tid 2 2026-02-17T05:46:40.673 INFO:tasks.rados.rados.0.trial115.stdout:15: write initial oid 15 2026-02-17T05:46:40.673 INFO:tasks.rados.rados.0.trial115.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2026-02-17T05:46:40.679 INFO:tasks.rados.rados.0.trial115.stdout:15: writing trial11545420-15 from 635298 to 1405137 tid 1 2026-02-17T05:46:40.683 INFO:tasks.rados.rados.0.trial115.stdout:15: writing trial11545420-15 from 2069689 to 2235298 tid 2 2026-02-17T05:46:40.683 INFO:tasks.rados.rados.0.trial115.stdout:16: write initial oid 16 2026-02-17T05:46:40.683 INFO:tasks.rados.rados.0.trial115.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2026-02-17T05:46:40.687 INFO:tasks.rados.rados.0.trial115.stdout:16: writing trial11545420-16 from 652105 to 1097193 tid 1 2026-02-17T05:46:40.692 INFO:tasks.rados.rados.0.trial115.stdout:16: writing trial11545420-16 from 1728171 to 2181159 tid 2 2026-02-17T05:46:40.692 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: finishing write tid 1 to trial11545420-1 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: finishing write tid 2 to trial11545420-1 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: finishing write tid 3 to trial11545420-1 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: finishing write tid 4 to trial11545420-1 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: oid 1 updating version 0 to 1 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: oid 1 updating version 1 to 2 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: oid 1 updating version 2 to 3 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:2: finishing write tid 1 to trial11545420-2 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:1: done (15 left) 2026-02-17T05:46:40.693 INFO:tasks.rados.rados.0.trial115.stdout:17: write initial oid 17 2026-02-17T05:46:40.694 INFO:tasks.rados.rados.0.trial115.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2026-02-17T05:46:40.697 INFO:tasks.rados.rados.0.trial115.stdout:17: writing trial11545420-17 from 668912 to 1305602 tid 1 2026-02-17T05:46:40.701 INFO:tasks.rados.rados.0.trial115.stdout:17: writing trial11545420-17 from 1903006 to 2268912 tid 2 2026-02-17T05:46:40.701 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.701 INFO:tasks.rados.rados.0.trial115.stdout:2: finishing write tid 2 to trial11545420-2 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:2: finishing write tid 3 to trial11545420-2 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:2: finishing write tid 4 to trial11545420-2 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:2: oid 2 updating version 0 to 2 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:2: oid 2 version 2 is already newer than 1 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:2: oid 2 updating version 2 to 3 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:3: finishing write tid 1 to trial11545420-3 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:3: finishing write tid 2 to trial11545420-3 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:3: finishing write tid 3 to trial11545420-3 2026-02-17T05:46:40.702 INFO:tasks.rados.rados.0.trial115.stdout:3: finishing write tid 4 to trial11545420-3 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:3: oid 3 updating version 0 to 6 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:3: oid 3 version 6 is already newer than 4 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:3: oid 3 version 6 is already newer than 5 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 6 (ObjNum 2 snap 0 seq_num 2) dirty exists 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:4: finishing write tid 1 to trial11545420-4 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:4: finishing write tid 2 to trial11545420-4 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:4: finishing write tid 3 to trial11545420-4 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:4: finishing write tid 4 to trial11545420-4 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:4: oid 4 updating version 0 to 2 2026-02-17T05:46:40.703 INFO:tasks.rados.rados.0.trial115.stdout:4: oid 4 updating version 2 to 3 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:4: oid 4 version 3 is already newer than 1 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:5: finishing write tid 1 to trial11545420-5 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:5: finishing write tid 2 to trial11545420-5 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:5: finishing write tid 3 to trial11545420-5 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:5: finishing write tid 4 to trial11545420-5 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:5: oid 5 updating version 0 to 8 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:5: oid 5 updating version 8 to 9 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:5: oid 5 version 9 is already newer than 7 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 9 (ObjNum 4 snap 0 seq_num 4) dirty exists 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2026-02-17T05:46:40.704 INFO:tasks.rados.rados.0.trial115.stdout:6: finishing write tid 1 to trial11545420-6 2026-02-17T05:46:40.705 INFO:tasks.rados.rados.0.trial115.stdout:2: done (15 left) 2026-02-17T05:46:40.705 INFO:tasks.rados.rados.0.trial115.stdout:3: done (14 left) 2026-02-17T05:46:40.705 INFO:tasks.rados.rados.0.trial115.stdout:4: done (13 left) 2026-02-17T05:46:40.705 INFO:tasks.rados.rados.0.trial115.stdout:5: done (12 left) 2026-02-17T05:46:40.705 INFO:tasks.rados.rados.0.trial115.stdout:18: write initial oid 18 2026-02-17T05:46:40.705 INFO:tasks.rados.rados.0.trial115.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2026-02-17T05:46:40.706 INFO:tasks.rados.rados.0.trial115.stdout:18: writing trial11545420-18 from 685719 to 1397658 tid 1 2026-02-17T05:46:40.710 INFO:tasks.rados.rados.0.trial115.stdout:18: writing trial11545420-18 from 1845135 to 2285719 tid 2 2026-02-17T05:46:40.710 INFO:tasks.rados.rados.0.trial115.stdout:19: write initial oid 19 2026-02-17T05:46:40.710 INFO:tasks.rados.rados.0.trial115.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2026-02-17T05:46:40.716 INFO:tasks.rados.rados.0.trial115.stdout:19: writing trial11545420-19 from 702526 to 1489714 tid 1 2026-02-17T05:46:40.719 INFO:tasks.rados.rados.0.trial115.stdout:19: writing trial11545420-19 from 1903617 to 2302526 tid 2 2026-02-17T05:46:40.719 INFO:tasks.rados.rados.0.trial115.stdout:20: write initial oid 20 2026-02-17T05:46:40.719 INFO:tasks.rados.rados.0.trial115.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2026-02-17T05:46:40.723 INFO:tasks.rados.rados.0.trial115.stdout:20: writing trial11545420-20 from 719333 to 1181770 tid 1 2026-02-17T05:46:40.728 INFO:tasks.rados.rados.0.trial115.stdout:20: writing trial11545420-20 from 1962099 to 2319333 tid 2 2026-02-17T05:46:40.728 INFO:tasks.rados.rados.0.trial115.stdout:21: write initial oid 21 2026-02-17T05:46:40.728 INFO:tasks.rados.rados.0.trial115.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2026-02-17T05:46:40.733 INFO:tasks.rados.rados.0.trial115.stdout:21: writing trial11545420-21 from 736140 to 1273826 tid 1 2026-02-17T05:46:40.737 INFO:tasks.rados.rados.0.trial115.stdout:21: writing trial11545420-21 from 2020581 to 2336140 tid 2 2026-02-17T05:46:40.737 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.737 INFO:tasks.rados.rados.0.trial115.stdout:6: finishing write tid 2 to trial11545420-6 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:6: finishing write tid 3 to trial11545420-6 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:6: finishing write tid 4 to trial11545420-6 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:6: oid 6 updating version 0 to 1 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:6: oid 6 updating version 1 to 3 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:6: oid 6 version 3 is already newer than 2 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:7: finishing write tid 1 to trial11545420-7 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:7: finishing write tid 2 to trial11545420-7 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:7: finishing write tid 3 to trial11545420-7 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:7: finishing write tid 4 to trial11545420-7 2026-02-17T05:46:40.738 INFO:tasks.rados.rados.0.trial115.stdout:7: oid 7 updating version 0 to 3 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:7: oid 7 version 3 is already newer than 1 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:7: oid 7 version 3 is already newer than 2 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:8: finishing write tid 1 to trial11545420-8 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:6: done (15 left) 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:7: done (14 left) 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:22: write initial oid 22 2026-02-17T05:46:40.739 INFO:tasks.rados.rados.0.trial115.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2026-02-17T05:46:40.742 INFO:tasks.rados.rados.0.trial115.stdout:22: writing trial11545420-22 from 752947 to 1365882 tid 1 2026-02-17T05:46:40.746 INFO:tasks.rados.rados.0.trial115.stdout:22: writing trial11545420-22 from 1962710 to 2352947 tid 2 2026-02-17T05:46:40.746 INFO:tasks.rados.rados.0.trial115.stdout:23: write initial oid 23 2026-02-17T05:46:40.746 INFO:tasks.rados.rados.0.trial115.stdout:8: finishing write tid 2 to trial11545420-8 2026-02-17T05:46:40.746 INFO:tasks.rados.rados.0.trial115.stdout:8: finishing write tid 3 to trial11545420-8 2026-02-17T05:46:40.746 INFO:tasks.rados.rados.0.trial115.stdout:8: finishing write tid 4 to trial11545420-8 2026-02-17T05:46:40.746 INFO:tasks.rados.rados.0.trial115.stdout:8: oid 8 updating version 0 to 3 2026-02-17T05:46:40.746 INFO:tasks.rados.rados.0.trial115.stdout:8: oid 8 version 3 is already newer than 2 2026-02-17T05:46:40.747 INFO:tasks.rados.rados.0.trial115.stdout:8: oid 8 version 3 is already newer than 1 2026-02-17T05:46:40.747 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2026-02-17T05:46:40.747 INFO:tasks.rados.rados.0.trial115.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2026-02-17T05:46:40.747 INFO:tasks.rados.rados.0.trial115.stdout:9: finishing write tid 1 to trial11545420-9 2026-02-17T05:46:40.747 INFO:tasks.rados.rados.0.trial115.stdout:9: finishing write tid 2 to trial11545420-9 2026-02-17T05:46:40.747 INFO:tasks.rados.rados.0.trial115.stdout:9: finishing write tid 3 to trial11545420-9 2026-02-17T05:46:40.747 INFO:tasks.rados.rados.0.trial115.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2026-02-17T05:46:40.751 INFO:tasks.rados.rados.0.trial115.stdout:23: writing trial11545420-23 from 769754 to 1457938 tid 1 2026-02-17T05:46:40.755 INFO:tasks.rados.rados.0.trial115.stdout:23: writing trial11545420-23 from 2021192 to 2369754 tid 2 2026-02-17T05:46:40.755 INFO:tasks.rados.rados.0.trial115.stdout:8: done (15 left) 2026-02-17T05:46:40.755 INFO:tasks.rados.rados.0.trial115.stdout:24: write initial oid 24 2026-02-17T05:46:40.755 INFO:tasks.rados.rados.0.trial115.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2026-02-17T05:46:40.759 INFO:tasks.rados.rados.0.trial115.stdout:24: writing trial11545420-24 from 786561 to 1266347 tid 1 2026-02-17T05:46:40.764 INFO:tasks.rados.rados.0.trial115.stdout:24: writing trial11545420-24 from 1796027 to 2263691 tid 2 2026-02-17T05:46:40.764 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.764 INFO:tasks.rados.rados.0.trial115.stdout:9: finishing write tid 4 to trial11545420-9 2026-02-17T05:46:40.764 INFO:tasks.rados.rados.0.trial115.stdout:9: oid 9 updating version 0 to 1 2026-02-17T05:46:40.764 INFO:tasks.rados.rados.0.trial115.stdout:9: oid 9 updating version 1 to 2 2026-02-17T05:46:40.764 INFO:tasks.rados.rados.0.trial115.stdout:9: oid 9 updating version 2 to 3 2026-02-17T05:46:40.764 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 3 (ObjNum 8 snap 0 seq_num 8) dirty exists 2026-02-17T05:46:40.764 INFO:tasks.rados.rados.0.trial115.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:10: finishing write tid 1 to trial11545420-10 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:10: finishing write tid 2 to trial11545420-10 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:10: finishing write tid 3 to trial11545420-10 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:10: finishing write tid 4 to trial11545420-10 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:10: oid 10 updating version 0 to 4 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:10: oid 10 updating version 4 to 5 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:10: oid 10 updating version 5 to 6 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 6 (ObjNum 9 snap 0 seq_num 9) dirty exists 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:11: finishing write tid 1 to trial11545420-11 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:11: finishing write tid 2 to trial11545420-11 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:11: finishing write tid 3 to trial11545420-11 2026-02-17T05:46:40.765 INFO:tasks.rados.rados.0.trial115.stdout:9: done (15 left) 2026-02-17T05:46:40.766 INFO:tasks.rados.rados.0.trial115.stdout:10: done (14 left) 2026-02-17T05:46:40.766 INFO:tasks.rados.rados.0.trial115.stdout:25: write initial oid 25 2026-02-17T05:46:40.766 INFO:tasks.rados.rados.0.trial115.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2026-02-17T05:46:40.767 INFO:tasks.rados.rados.0.trial115.stdout:25: writing trial11545420-25 from 403368 to 958403 tid 1 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:25: writing trial11545420-25 from 1454509 to 2182184 tid 2 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:26: write initial oid 26 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:11: finishing write tid 4 to trial11545420-11 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:11: oid 11 updating version 0 to 9 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:11: oid 11 version 9 is already newer than 8 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:11: oid 11 version 9 is already newer than 7 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 9 (ObjNum 10 snap 0 seq_num 10) dirty exists 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:12: finishing write tid 1 to trial11545420-12 2026-02-17T05:46:40.773 INFO:tasks.rados.rados.0.trial115.stdout:12: finishing write tid 2 to trial11545420-12 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:12: finishing write tid 3 to trial11545420-12 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:12: finishing write tid 4 to trial11545420-12 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:12: oid 12 updating version 0 to 1 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:12: oid 12 updating version 1 to 2 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:12: oid 12 updating version 2 to 3 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 3 (ObjNum 11 snap 0 seq_num 11) dirty exists 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:13: finishing write tid 1 to trial11545420-13 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:13: finishing write tid 2 to trial11545420-13 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:13: finishing write tid 3 to trial11545420-13 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:13: finishing write tid 4 to trial11545420-13 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:13: oid 13 updating version 0 to 1 2026-02-17T05:46:40.774 INFO:tasks.rados.rados.0.trial115.stdout:13: oid 13 updating version 1 to 2 2026-02-17T05:46:40.775 INFO:tasks.rados.rados.0.trial115.stdout:13: oid 13 updating version 2 to 3 2026-02-17T05:46:40.775 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 3 (ObjNum 12 snap 0 seq_num 12) dirty exists 2026-02-17T05:46:40.775 INFO:tasks.rados.rados.0.trial115.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2026-02-17T05:46:40.775 INFO:tasks.rados.rados.0.trial115.stdout:14: finishing write tid 1 to trial11545420-14 2026-02-17T05:46:40.775 INFO:tasks.rados.rados.0.trial115.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2026-02-17T05:46:40.776 INFO:tasks.rados.rados.0.trial115.stdout:26: writing trial11545420-26 from 420175 to 1050459 tid 1 2026-02-17T05:46:40.781 INFO:tasks.rados.rados.0.trial115.stdout:26: writing trial11545420-26 from 1796638 to 2267971 tid 2 2026-02-17T05:46:40.781 INFO:tasks.rados.rados.0.trial115.stdout:11: done (15 left) 2026-02-17T05:46:40.781 INFO:tasks.rados.rados.0.trial115.stdout:12: done (14 left) 2026-02-17T05:46:40.781 INFO:tasks.rados.rados.0.trial115.stdout:13: done (13 left) 2026-02-17T05:46:40.781 INFO:tasks.rados.rados.0.trial115.stdout:27: write initial oid 27 2026-02-17T05:46:40.781 INFO:tasks.rados.rados.0.trial115.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2026-02-17T05:46:40.785 INFO:tasks.rados.rados.0.trial115.stdout:27: writing trial11545420-27 from 436982 to 1142515 tid 1 2026-02-17T05:46:40.790 INFO:tasks.rados.rados.0.trial115.stdout:27: writing trial11545420-27 from 1855120 to 2436982 tid 2 2026-02-17T05:46:40.790 INFO:tasks.rados.rados.0.trial115.stdout:28: write initial oid 28 2026-02-17T05:46:40.790 INFO:tasks.rados.rados.0.trial115.stdout:14: finishing write tid 2 to trial11545420-14 2026-02-17T05:46:40.790 INFO:tasks.rados.rados.0.trial115.stdout:14: finishing write tid 3 to trial11545420-14 2026-02-17T05:46:40.791 INFO:tasks.rados.rados.0.trial115.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2026-02-17T05:46:40.795 INFO:tasks.rados.rados.0.trial115.stdout:28: writing trial11545420-28 from 453789 to 1234571 tid 1 2026-02-17T05:46:40.799 INFO:tasks.rados.rados.0.trial115.stdout:28: writing trial11545420-28 from 1913602 to 2388604 tid 2 2026-02-17T05:46:40.799 INFO:tasks.rados.rados.0.trial115.stdout:29: write initial oid 29 2026-02-17T05:46:40.799 INFO:tasks.rados.rados.0.trial115.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2026-02-17T05:46:40.802 INFO:tasks.rados.rados.0.trial115.stdout:29: writing trial11545420-29 from 470596 to 926627 tid 1 2026-02-17T05:46:40.808 INFO:tasks.rados.rados.0.trial115.stdout:29: writing trial11545420-29 from 1572084 to 2190744 tid 2 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:14: finishing write tid 4 to trial11545420-14 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:14: oid 14 updating version 0 to 5 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:14: oid 14 updating version 5 to 6 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:14: oid 14 version 6 is already newer than 4 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:15: finishing write tid 1 to trial11545420-15 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:15: finishing write tid 2 to trial11545420-15 2026-02-17T05:46:40.809 INFO:tasks.rados.rados.0.trial115.stdout:15: finishing write tid 3 to trial11545420-15 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:15: finishing write tid 4 to trial11545420-15 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:15: oid 15 updating version 0 to 10 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:15: oid 15 updating version 10 to 11 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:15: oid 15 updating version 11 to 12 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 12 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:16: finishing write tid 1 to trial11545420-16 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:16: finishing write tid 2 to trial11545420-16 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:16: finishing write tid 3 to trial11545420-16 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:16: finishing write tid 4 to trial11545420-16 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:16: oid 16 updating version 0 to 1 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:16: oid 16 updating version 1 to 2 2026-02-17T05:46:40.810 INFO:tasks.rados.rados.0.trial115.stdout:16: oid 16 updating version 2 to 3 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 3 (ObjNum 15 snap 0 seq_num 15) dirty exists 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:17: finishing write tid 1 to trial11545420-17 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:17: finishing write tid 2 to trial11545420-17 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:17: finishing write tid 3 to trial11545420-17 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:17: finishing write tid 4 to trial11545420-17 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:17: oid 17 updating version 0 to 1 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:17: oid 17 updating version 1 to 3 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:17: oid 17 version 3 is already newer than 2 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 3 (ObjNum 16 snap 0 seq_num 16) dirty exists 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:18: finishing write tid 1 to trial11545420-18 2026-02-17T05:46:40.811 INFO:tasks.rados.rados.0.trial115.stdout:14: done (15 left) 2026-02-17T05:46:40.812 INFO:tasks.rados.rados.0.trial115.stdout:15: done (14 left) 2026-02-17T05:46:40.812 INFO:tasks.rados.rados.0.trial115.stdout:16: done (13 left) 2026-02-17T05:46:40.812 INFO:tasks.rados.rados.0.trial115.stdout:17: done (12 left) 2026-02-17T05:46:40.812 INFO:tasks.rados.rados.0.trial115.stdout:30: write initial oid 30 2026-02-17T05:46:40.812 INFO:tasks.rados.rados.0.trial115.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2026-02-17T05:46:40.812 INFO:tasks.rados.rados.0.trial115.stdout:30: writing trial11545420-30 from 487403 to 1018683 tid 1 2026-02-17T05:46:40.815 INFO:tasks.rados.rados.0.trial115.stdout:30: writing trial11545420-30 from 1514213 to 1992884 tid 2 2026-02-17T05:46:40.815 INFO:tasks.rados.rados.0.trial115.stdout:31: write initial oid 31 2026-02-17T05:46:40.815 INFO:tasks.rados.rados.0.trial115.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2026-02-17T05:46:40.819 INFO:tasks.rados.rados.0.trial115.stdout:31: writing trial11545420-31 from 504210 to 1110739 tid 1 2026-02-17T05:46:40.823 INFO:tasks.rados.rados.0.trial115.stdout:31: writing trial11545420-31 from 1572695 to 2195024 tid 2 2026-02-17T05:46:40.824 INFO:tasks.rados.rados.0.trial115.stdout:32: write initial oid 32 2026-02-17T05:46:40.824 INFO:tasks.rados.rados.0.trial115.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2026-02-17T05:46:40.828 INFO:tasks.rados.rados.0.trial115.stdout:32: writing trial11545420-32 from 521017 to 1319148 tid 1 2026-02-17T05:46:40.832 INFO:tasks.rados.rados.0.trial115.stdout:32: writing trial11545420-32 from 1747530 to 2229870 tid 2 2026-02-17T05:46:40.832 INFO:tasks.rados.rados.0.trial115.stdout:33: write initial oid 33 2026-02-17T05:46:40.832 INFO:tasks.rados.rados.0.trial115.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2026-02-17T05:46:40.836 INFO:tasks.rados.rados.0.trial115.stdout:33: writing trial11545420-33 from 537824 to 1011204 tid 1 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:33: writing trial11545420-33 from 1806012 to 2432010 tid 2 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:18: finishing write tid 2 to trial11545420-18 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:18: finishing write tid 3 to trial11545420-18 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:18: finishing write tid 4 to trial11545420-18 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:18: oid 18 updating version 0 to 3 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:18: oid 18 version 3 is already newer than 2 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:18: oid 18 version 3 is already newer than 1 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 3 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-17T05:46:40.842 INFO:tasks.rados.rados.0.trial115.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2026-02-17T05:46:40.843 INFO:tasks.rados.rados.0.trial115.stdout:19: finishing write tid 1 to trial11545420-19 2026-02-17T05:46:40.843 INFO:tasks.rados.rados.0.trial115.stdout:18: done (15 left) 2026-02-17T05:46:40.843 INFO:tasks.rados.rados.0.trial115.stdout:34: write initial oid 34 2026-02-17T05:46:40.843 INFO:tasks.rados.rados.0.trial115.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2026-02-17T05:46:40.845 INFO:tasks.rados.rados.0.trial115.stdout:34: writing trial11545420-34 from 554631 to 1103260 tid 1 2026-02-17T05:46:40.850 INFO:tasks.rados.rados.0.trial115.stdout:34: writing trial11545420-34 from 1748141 to 2234150 tid 2 2026-02-17T05:46:40.850 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.850 INFO:tasks.rados.rados.0.trial115.stdout:19: finishing write tid 2 to trial11545420-19 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:19: finishing write tid 3 to trial11545420-19 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:19: finishing write tid 4 to trial11545420-19 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:19: oid 19 updating version 0 to 5 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:19: oid 19 version 5 is already newer than 4 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:19: oid 19 updating version 5 to 6 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 6 (ObjNum 18 snap 0 seq_num 18) dirty exists 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:20: finishing write tid 1 to trial11545420-20 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:19: done (15 left) 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:35: write initial oid 35 2026-02-17T05:46:40.851 INFO:tasks.rados.rados.0.trial115.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2026-02-17T05:46:40.854 INFO:tasks.rados.rados.0.trial115.stdout:35: writing trial11545420-35 from 571438 to 1195316 tid 1 2026-02-17T05:46:40.859 INFO:tasks.rados.rados.0.trial115.stdout:35: writing trial11545420-35 from 1806623 to 2436290 tid 2 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:20: finishing write tid 2 to trial11545420-20 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:20: finishing write tid 3 to trial11545420-20 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:20: finishing write tid 4 to trial11545420-20 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:20: oid 20 updating version 0 to 6 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:20: oid 20 version 6 is already newer than 4 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:20: oid 20 version 6 is already newer than 5 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:21: finishing write tid 1 to trial11545420-21 2026-02-17T05:46:40.860 INFO:tasks.rados.rados.0.trial115.stdout:20: done (15 left) 2026-02-17T05:46:40.861 INFO:tasks.rados.rados.0.trial115.stdout:36: write initial oid 36 2026-02-17T05:46:40.861 INFO:tasks.rados.rados.0.trial115.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2026-02-17T05:46:40.864 INFO:tasks.rados.rados.0.trial115.stdout:36: writing trial11545420-36 from 588245 to 1287372 tid 1 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout:36: writing trial11545420-36 from 1865105 to 2354783 tid 2 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout:21: finishing write tid 2 to trial11545420-21 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout:21: finishing write tid 3 to trial11545420-21 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout:21: finishing write tid 4 to trial11545420-21 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout:21: oid 21 updating version 0 to 4 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout:21: oid 21 updating version 4 to 6 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout:21: oid 21 version 6 is already newer than 5 2026-02-17T05:46:40.869 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2026-02-17T05:46:40.870 INFO:tasks.rados.rados.0.trial115.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2026-02-17T05:46:40.870 INFO:tasks.rados.rados.0.trial115.stdout:22: finishing write tid 1 to trial11545420-22 2026-02-17T05:46:40.870 INFO:tasks.rados.rados.0.trial115.stdout:21: done (15 left) 2026-02-17T05:46:40.870 INFO:tasks.rados.rados.0.trial115.stdout:37: write initial oid 37 2026-02-17T05:46:40.870 INFO:tasks.rados.rados.0.trial115.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2026-02-17T05:46:40.874 INFO:tasks.rados.rados.0.trial115.stdout:37: writing trial11545420-37 from 605052 to 1379428 tid 1 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout:37: writing trial11545420-37 from 1923587 to 2556923 tid 2 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout:22: finishing write tid 2 to trial11545420-22 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout:22: finishing write tid 3 to trial11545420-22 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout:22: finishing write tid 4 to trial11545420-22 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout:22: oid 22 updating version 0 to 1 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout:22: oid 22 updating version 1 to 3 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout:22: oid 22 version 3 is already newer than 2 2026-02-17T05:46:40.879 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 3 (ObjNum 21 snap 0 seq_num 21) dirty exists 2026-02-17T05:46:40.880 INFO:tasks.rados.rados.0.trial115.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2026-02-17T05:46:40.880 INFO:tasks.rados.rados.0.trial115.stdout:23: finishing write tid 1 to trial11545420-23 2026-02-17T05:46:40.880 INFO:tasks.rados.rados.0.trial115.stdout:23: finishing write tid 2 to trial11545420-23 2026-02-17T05:46:40.880 INFO:tasks.rados.rados.0.trial115.stdout:22: done (15 left) 2026-02-17T05:46:40.880 INFO:tasks.rados.rados.0.trial115.stdout:38: write initial oid 38 2026-02-17T05:46:40.880 INFO:tasks.rados.rados.0.trial115.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2026-02-17T05:46:40.882 INFO:tasks.rados.rados.0.trial115.stdout:38: writing trial11545420-38 from 621859 to 1071484 tid 1 2026-02-17T05:46:40.889 INFO:tasks.rados.rados.0.trial115.stdout:38: writing trial11545420-38 from 1865716 to 2621859 tid 2 2026-02-17T05:46:40.889 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.889 INFO:tasks.rados.rados.0.trial115.stdout:23: finishing write tid 3 to trial11545420-23 2026-02-17T05:46:40.889 INFO:tasks.rados.rados.0.trial115.stdout:23: finishing write tid 4 to trial11545420-23 2026-02-17T05:46:40.889 INFO:tasks.rados.rados.0.trial115.stdout:23: oid 23 updating version 0 to 4 2026-02-17T05:46:40.889 INFO:tasks.rados.rados.0.trial115.stdout:23: oid 23 updating version 4 to 5 2026-02-17T05:46:40.889 INFO:tasks.rados.rados.0.trial115.stdout:23: oid 23 updating version 5 to 6 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: finishing write tid 1 to trial11545420-24 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:25: finishing write tid 1 to trial11545420-25 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: finishing write tid 2 to trial11545420-24 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: finishing write tid 3 to trial11545420-24 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: finishing write tid 4 to trial11545420-24 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: oid 24 updating version 0 to 5 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: oid 24 updating version 5 to 6 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: oid 24 version 6 is already newer than 4 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:25: finishing write tid 2 to trial11545420-25 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:23: done (15 left) 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:24: done (14 left) 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:39: write initial oid 39 2026-02-17T05:46:40.890 INFO:tasks.rados.rados.0.trial115.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2026-02-17T05:46:40.893 INFO:tasks.rados.rados.0.trial115.stdout:39: writing trial11545420-39 from 638666 to 1163540 tid 1 2026-02-17T05:46:40.898 INFO:tasks.rados.rados.0.trial115.stdout:39: writing trial11545420-39 from 1924198 to 2561203 tid 2 2026-02-17T05:46:40.898 INFO:tasks.rados.rados.0.trial115.stdout:40: write initial oid 40 2026-02-17T05:46:40.898 INFO:tasks.rados.rados.0.trial115.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2026-02-17T05:46:40.904 INFO:tasks.rados.rados.0.trial115.stdout:40: writing trial11545420-40 from 655473 to 1371949 tid 1 2026-02-17T05:46:40.910 INFO:tasks.rados.rados.0.trial115.stdout:40: writing trial11545420-40 from 2099033 to 2655473 tid 2 2026-02-17T05:46:40.910 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.910 INFO:tasks.rados.rados.0.trial115.stdout:25: finishing write tid 3 to trial11545420-25 2026-02-17T05:46:40.910 INFO:tasks.rados.rados.0.trial115.stdout:25: finishing write tid 4 to trial11545420-25 2026-02-17T05:46:40.910 INFO:tasks.rados.rados.0.trial115.stdout:25: oid 25 updating version 0 to 7 2026-02-17T05:46:40.910 INFO:tasks.rados.rados.0.trial115.stdout:25: oid 25 updating version 7 to 9 2026-02-17T05:46:40.910 INFO:tasks.rados.rados.0.trial115.stdout:25: oid 25 version 9 is already newer than 8 2026-02-17T05:46:40.911 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 9 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-17T05:46:40.911 INFO:tasks.rados.rados.0.trial115.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2026-02-17T05:46:40.911 INFO:tasks.rados.rados.0.trial115.stdout:26: finishing write tid 1 to trial11545420-26 2026-02-17T05:46:40.911 INFO:tasks.rados.rados.0.trial115.stdout:25: done (15 left) 2026-02-17T05:46:40.911 INFO:tasks.rados.rados.0.trial115.stdout:41: write initial oid 41 2026-02-17T05:46:40.911 INFO:tasks.rados.rados.0.trial115.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2026-02-17T05:46:40.915 INFO:tasks.rados.rados.0.trial115.stdout:41: writing trial11545420-41 from 672280 to 1464005 tid 1 2026-02-17T05:46:40.920 INFO:tasks.rados.rados.0.trial115.stdout:41: writing trial11545420-41 from 2157515 to 2672280 tid 2 2026-02-17T05:46:40.920 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.920 INFO:tasks.rados.rados.0.trial115.stdout:26: finishing write tid 2 to trial11545420-26 2026-02-17T05:46:40.920 INFO:tasks.rados.rados.0.trial115.stdout:26: finishing write tid 3 to trial11545420-26 2026-02-17T05:46:40.920 INFO:tasks.rados.rados.0.trial115.stdout:26: finishing write tid 4 to trial11545420-26 2026-02-17T05:46:40.920 INFO:tasks.rados.rados.0.trial115.stdout:26: oid 26 updating version 0 to 3 2026-02-17T05:46:40.920 INFO:tasks.rados.rados.0.trial115.stdout:26: oid 26 version 3 is already newer than 1 2026-02-17T05:46:40.921 INFO:tasks.rados.rados.0.trial115.stdout:26: oid 26 version 3 is already newer than 2 2026-02-17T05:46:40.921 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 3 (ObjNum 25 snap 0 seq_num 25) dirty exists 2026-02-17T05:46:40.921 INFO:tasks.rados.rados.0.trial115.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2026-02-17T05:46:40.921 INFO:tasks.rados.rados.0.trial115.stdout:27: finishing write tid 1 to trial11545420-27 2026-02-17T05:46:40.921 INFO:tasks.rados.rados.0.trial115.stdout:26: done (15 left) 2026-02-17T05:46:40.921 INFO:tasks.rados.rados.0.trial115.stdout:42: write initial oid 42 2026-02-17T05:46:40.921 INFO:tasks.rados.rados.0.trial115.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2026-02-17T05:46:40.924 INFO:tasks.rados.rados.0.trial115.stdout:42: writing trial11545420-42 from 689087 to 1156061 tid 1 2026-02-17T05:46:40.929 INFO:tasks.rados.rados.0.trial115.stdout:42: writing trial11545420-42 from 1699644 to 2483976 tid 2 2026-02-17T05:46:40.929 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:27: finishing write tid 2 to trial11545420-27 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:27: finishing write tid 3 to trial11545420-27 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:27: finishing write tid 4 to trial11545420-27 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:27: oid 27 updating version 0 to 4 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:27: oid 27 updating version 4 to 6 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:27: oid 27 version 6 is already newer than 5 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:28: finishing write tid 1 to trial11545420-28 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:27: done (15 left) 2026-02-17T05:46:40.930 INFO:tasks.rados.rados.0.trial115.stdout:43: write initial oid 43 2026-02-17T05:46:40.931 INFO:tasks.rados.rados.0.trial115.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2026-02-17T05:46:40.934 INFO:tasks.rados.rados.0.trial115.stdout:43: writing trial11545420-43 from 705894 to 1248117 tid 1 2026-02-17T05:46:40.939 INFO:tasks.rados.rados.0.trial115.stdout:43: writing trial11545420-43 from 1758126 to 2402469 tid 2 2026-02-17T05:46:40.939 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.939 INFO:tasks.rados.rados.0.trial115.stdout:28: finishing write tid 2 to trial11545420-28 2026-02-17T05:46:40.939 INFO:tasks.rados.rados.0.trial115.stdout:28: finishing write tid 3 to trial11545420-28 2026-02-17T05:46:40.939 INFO:tasks.rados.rados.0.trial115.stdout:28: finishing write tid 4 to trial11545420-28 2026-02-17T05:46:40.939 INFO:tasks.rados.rados.0.trial115.stdout:28: oid 28 updating version 0 to 14 2026-02-17T05:46:40.940 INFO:tasks.rados.rados.0.trial115.stdout:28: oid 28 updating version 14 to 15 2026-02-17T05:46:40.940 INFO:tasks.rados.rados.0.trial115.stdout:28: oid 28 version 15 is already newer than 13 2026-02-17T05:46:40.940 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 15 (ObjNum 27 snap 0 seq_num 27) dirty exists 2026-02-17T05:46:40.940 INFO:tasks.rados.rados.0.trial115.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2026-02-17T05:46:40.940 INFO:tasks.rados.rados.0.trial115.stdout:29: finishing write tid 1 to trial11545420-29 2026-02-17T05:46:40.940 INFO:tasks.rados.rados.0.trial115.stdout:28: done (15 left) 2026-02-17T05:46:40.940 INFO:tasks.rados.rados.0.trial115.stdout:44: write initial oid 44 2026-02-17T05:46:40.940 INFO:tasks.rados.rados.0.trial115.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2026-02-17T05:46:40.944 INFO:tasks.rados.rados.0.trial115.stdout:44: writing trial11545420-44 from 722701 to 1340173 tid 1 2026-02-17T05:46:40.949 INFO:tasks.rados.rados.0.trial115.stdout:44: writing trial11545420-44 from 1816608 to 2604609 tid 2 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout:29: finishing write tid 2 to trial11545420-29 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout:29: finishing write tid 3 to trial11545420-29 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout:29: finishing write tid 4 to trial11545420-29 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout:29: oid 29 updating version 0 to 5 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout:29: oid 29 updating version 5 to 6 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout:29: oid 29 version 6 is already newer than 4 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2026-02-17T05:46:40.950 INFO:tasks.rados.rados.0.trial115.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: finishing write tid 1 to trial11545420-30 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: finishing write tid 2 to trial11545420-30 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: finishing write tid 3 to trial11545420-30 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: finishing write tid 4 to trial11545420-30 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: oid 30 updating version 0 to 7 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: oid 30 updating version 7 to 9 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: oid 30 version 9 is already newer than 8 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 9 (ObjNum 29 snap 0 seq_num 29) dirty exists 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:31: finishing write tid 1 to trial11545420-31 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:29: done (15 left) 2026-02-17T05:46:40.951 INFO:tasks.rados.rados.0.trial115.stdout:30: done (14 left) 2026-02-17T05:46:40.952 INFO:tasks.rados.rados.0.trial115.stdout:45: write initial oid 45 2026-02-17T05:46:40.952 INFO:tasks.rados.rados.0.trial115.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2026-02-17T05:46:40.954 INFO:tasks.rados.rados.0.trial115.stdout:45: writing trial11545420-45 from 739508 to 1432229 tid 1 2026-02-17T05:46:40.958 INFO:tasks.rados.rados.0.trial115.stdout:45: writing trial11545420-45 from 1875090 to 2523102 tid 2 2026-02-17T05:46:40.958 INFO:tasks.rados.rados.0.trial115.stdout:46: write initial oid 46 2026-02-17T05:46:40.958 INFO:tasks.rados.rados.0.trial115.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2026-02-17T05:46:40.963 INFO:tasks.rados.rados.0.trial115.stdout:46: writing trial11545420-46 from 756315 to 1524285 tid 1 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout:46: writing trial11545420-46 from 1933572 to 2725242 tid 2 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout:31: finishing write tid 2 to trial11545420-31 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout:31: finishing write tid 3 to trial11545420-31 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout:31: finishing write tid 4 to trial11545420-31 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout:31: oid 31 updating version 0 to 12 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout:31: oid 31 version 12 is already newer than 11 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout:31: oid 31 version 12 is already newer than 10 2026-02-17T05:46:40.969 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 12 (ObjNum 30 snap 0 seq_num 30) dirty exists 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:32: finishing write tid 1 to trial11545420-32 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:32: finishing write tid 2 to trial11545420-32 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:32: finishing write tid 3 to trial11545420-32 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:32: finishing write tid 4 to trial11545420-32 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:32: oid 32 updating version 0 to 2 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:32: oid 32 updating version 2 to 3 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:32: oid 32 version 3 is already newer than 1 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 3 (ObjNum 31 snap 0 seq_num 31) dirty exists 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:33: finishing write tid 1 to trial11545420-33 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:33: finishing write tid 2 to trial11545420-33 2026-02-17T05:46:40.970 INFO:tasks.rados.rados.0.trial115.stdout:33: finishing write tid 3 to trial11545420-33 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:33: finishing write tid 4 to trial11545420-33 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:33: oid 33 updating version 0 to 7 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:33: oid 33 updating version 7 to 9 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:33: oid 33 version 9 is already newer than 8 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 9 (ObjNum 32 snap 0 seq_num 32) dirty exists 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:34: finishing write tid 1 to trial11545420-34 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:34: finishing write tid 2 to trial11545420-34 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:34: finishing write tid 3 to trial11545420-34 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:31: done (15 left) 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:32: done (14 left) 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:33: done (13 left) 2026-02-17T05:46:40.971 INFO:tasks.rados.rados.0.trial115.stdout:47: write initial oid 47 2026-02-17T05:46:40.972 INFO:tasks.rados.rados.0.trial115.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2026-02-17T05:46:40.973 INFO:tasks.rados.rados.0.trial115.stdout:47: writing trial11545420-47 from 773122 to 1332694 tid 1 2026-02-17T05:46:40.978 INFO:tasks.rados.rados.0.trial115.stdout:47: writing trial11545420-47 from 1992054 to 2643735 tid 2 2026-02-17T05:46:40.978 INFO:tasks.rados.rados.0.trial115.stdout:48: write initial oid 48 2026-02-17T05:46:40.978 INFO:tasks.rados.rados.0.trial115.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2026-02-17T05:46:40.983 INFO:tasks.rados.rados.0.trial115.stdout:48: writing trial11545420-48 from 789929 to 1424750 tid 1 2026-02-17T05:46:40.988 INFO:tasks.rados.rados.0.trial115.stdout:48: writing trial11545420-48 from 2050536 to 2789929 tid 2 2026-02-17T05:46:40.989 INFO:tasks.rados.rados.0.trial115.stdout:49: write initial oid 49 2026-02-17T05:46:40.989 INFO:tasks.rados.rados.0.trial115.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2026-02-17T05:46:40.992 INFO:tasks.rados.rados.0.trial115.stdout:49: writing trial11545420-49 from 406736 to 1116806 tid 1 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout:49: writing trial11545420-49 from 1709018 to 2364368 tid 2 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 16 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout:34: finishing write tid 4 to trial11545420-34 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout:34: oid 34 updating version 0 to 5 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout:34: oid 34 version 5 is already newer than 4 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout:34: oid 34 updating version 5 to 6 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2026-02-17T05:46:40.998 INFO:tasks.rados.rados.0.trial115.stdout:35: finishing write tid 1 to trial11545420-35 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:35: finishing write tid 2 to trial11545420-35 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:35: finishing write tid 3 to trial11545420-35 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:35: finishing write tid 4 to trial11545420-35 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:35: oid 35 updating version 0 to 8 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:35: oid 35 version 8 is already newer than 7 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:35: oid 35 updating version 8 to 9 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 9 (ObjNum 34 snap 0 seq_num 34) dirty exists 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:36: finishing write tid 1 to trial11545420-36 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:34: done (15 left) 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:35: done (14 left) 2026-02-17T05:46:40.999 INFO:tasks.rados.rados.0.trial115.stdout:50: write initial oid 50 2026-02-17T05:46:41.000 INFO:tasks.rados.rados.0.trial115.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2026-02-17T05:46:41.001 INFO:tasks.rados.rados.0.trial115.stdout:50: writing trial11545420-50 from 423543 to 1208862 tid 1 2026-02-17T05:46:41.008 INFO:tasks.rados.rados.0.trial115.stdout:50: writing trial11545420-50 from 1767500 to 2566508 tid 2 2026-02-17T05:46:41.008 INFO:tasks.rados.rados.0.trial115.stdout:51: copy_from oid 24 from oid 33 current snap is 0 2026-02-17T05:46:41.008 INFO:tasks.rados.rados.0.trial115.stdout:36: finishing write tid 2 to trial11545420-36 2026-02-17T05:46:41.008 INFO:tasks.rados.rados.0.trial115.stdout:36: finishing write tid 3 to trial11545420-36 2026-02-17T05:46:41.008 INFO:tasks.rados.rados.0.trial115.stdout:36: finishing write tid 4 to trial11545420-36 2026-02-17T05:46:41.008 INFO:tasks.rados.rados.0.trial115.stdout:36: oid 36 updating version 0 to 10 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:36: oid 36 updating version 10 to 11 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:36: oid 36 updating version 11 to 12 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 12 (ObjNum 35 snap 0 seq_num 35) dirty exists 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:37: finishing write tid 1 to trial11545420-37 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:37: finishing write tid 2 to trial11545420-37 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:37: finishing write tid 3 to trial11545420-37 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:37: finishing write tid 4 to trial11545420-37 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:37: oid 37 updating version 0 to 9 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:37: oid 37 version 9 is already newer than 7 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:37: oid 37 version 9 is already newer than 8 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 9 (ObjNum 36 snap 0 seq_num 36) dirty exists 2026-02-17T05:46:41.009 INFO:tasks.rados.rados.0.trial115.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:38: finishing write tid 1 to trial11545420-38 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:38: finishing write tid 2 to trial11545420-38 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:36: done (15 left) 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:37: done (14 left) 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:52: rollback oid 27 current snap is 0 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:52: done (14 left) 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:53: write oid 29 current snap is 0 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:39: finishing write tid 1 to trial11545420-39 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:38: finishing write tid 3 to trial11545420-38 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:38: finishing write tid 4 to trial11545420-38 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:38: oid 38 updating version 0 to 12 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:38: oid 38 version 12 is already newer than 10 2026-02-17T05:46:41.010 INFO:tasks.rados.rados.0.trial115.stdout:38: oid 38 version 12 is already newer than 11 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:39: finishing write tid 2 to trial11545420-39 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:39: finishing write tid 3 to trial11545420-39 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:39: finishing write tid 4 to trial11545420-39 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:39: oid 39 updating version 0 to 7 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:39: oid 39 updating version 7 to 8 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:39: oid 39 updating version 8 to 9 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 9 (ObjNum 38 snap 0 seq_num 38) dirty exists 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:40: finishing write tid 1 to trial11545420-40 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:40: finishing write tid 2 to trial11545420-40 2026-02-17T05:46:41.011 INFO:tasks.rados.rados.0.trial115.stdout:40: finishing write tid 3 to trial11545420-40 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:40: finishing write tid 4 to trial11545420-40 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:40: oid 40 updating version 0 to 4 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:40: oid 40 updating version 4 to 6 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:40: oid 40 version 6 is already newer than 5 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:41: finishing write tid 1 to trial11545420-41 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:53: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2026-02-17T05:46:41.012 INFO:tasks.rados.rados.0.trial115.stdout:53: writing trial11545420-29 from 440350 to 900918 tid 1 2026-02-17T05:46:41.013 INFO:tasks.rados.rados.0.trial115.stdout:53: writing trial11545420-29 from 1309629 to 1852295 tid 2 2026-02-17T05:46:41.017 INFO:tasks.rados.rados.0.trial115.stdout:53: writing trial11545420-29 from 2323973 to 2840350 tid 3 2026-02-17T05:46:41.017 INFO:tasks.rados.rados.0.trial115.stdout:38: done (14 left) 2026-02-17T05:46:41.017 INFO:tasks.rados.rados.0.trial115.stdout:39: done (13 left) 2026-02-17T05:46:41.018 INFO:tasks.rados.rados.0.trial115.stdout:40: done (12 left) 2026-02-17T05:46:41.018 INFO:tasks.rados.rados.0.trial115.stdout:54: write (excl) oid 15 current snap is 0 2026-02-17T05:46:41.018 INFO:tasks.rados.rados.0.trial115.stdout:54: seq_num 51 ranges {457157=535817,1768111=402677} 2026-02-17T05:46:41.021 INFO:tasks.rados.rados.0.trial115.stdout:54: writing trial11545420-15 from 457157 to 992974 tid 1 2026-02-17T05:46:41.026 INFO:tasks.rados.rados.0.trial115.stdout:54: writing trial11545420-15 from 1768111 to 2170788 tid 2 2026-02-17T05:46:41.026 INFO:tasks.rados.rados.0.trial115.stdout:55: read oid 21 snap -1 2026-02-17T05:46:41.026 INFO:tasks.rados.rados.0.trial115.stdout:55: expect (ObjNum 20 snap 0 seq_num 20) 2026-02-17T05:46:41.026 INFO:tasks.rados.rados.0.trial115.stdout:56: write oid 11 current snap is 0 2026-02-17T05:46:41.026 INFO:tasks.rados.rados.0.trial115.stdout:41: finishing write tid 2 to trial11545420-41 2026-02-17T05:46:41.027 INFO:tasks.rados.rados.0.trial115.stdout:41: finishing write tid 3 to trial11545420-41 2026-02-17T05:46:41.027 INFO:tasks.rados.rados.0.trial115.stdout:41: finishing write tid 4 to trial11545420-41 2026-02-17T05:46:41.027 INFO:tasks.rados.rados.0.trial115.stdout:41: oid 41 updating version 0 to 11 2026-02-17T05:46:41.027 INFO:tasks.rados.rados.0.trial115.stdout:41: oid 41 updating version 11 to 12 2026-02-17T05:46:41.027 INFO:tasks.rados.rados.0.trial115.stdout:41: oid 41 version 12 is already newer than 10 2026-02-17T05:46:41.027 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 12 (ObjNum 40 snap 0 seq_num 40) dirty exists 2026-02-17T05:46:41.027 INFO:tasks.rados.rados.0.trial115.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2026-02-17T05:46:41.027 INFO:tasks.rados.rados.0.trial115.stdout:56: seq_num 52 ranges {473964=611066,1826593=546335,2778819=95145} 2026-02-17T05:46:41.029 INFO:tasks.rados.rados.0.trial115.stdout:56: writing trial11545420-11 from 473964 to 1085030 tid 1 2026-02-17T05:46:41.034 INFO:tasks.rados.rados.0.trial115.stdout:56: writing trial11545420-11 from 1826593 to 2372928 tid 2 2026-02-17T05:46:41.038 INFO:tasks.rados.rados.0.trial115.stdout:56: writing trial11545420-11 from 2778819 to 2873964 tid 3 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:41: done (14 left) 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:57: read oid 38 snap -1 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:57: expect (ObjNum 37 snap 0 seq_num 37) 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:58: delete oid 28 current snap is 0 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:42: finishing write tid 1 to trial11545420-42 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:42: finishing write tid 2 to trial11545420-42 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:42: finishing write tid 3 to trial11545420-42 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:42: finishing write tid 4 to trial11545420-42 2026-02-17T05:46:41.039 INFO:tasks.rados.rados.0.trial115.stdout:42: oid 42 updating version 0 to 10 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:42: oid 42 updating version 10 to 12 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:42: oid 42 version 12 is already newer than 11 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:43: finishing write tid 1 to trial11545420-43 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:43: finishing write tid 2 to trial11545420-43 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:43: finishing write tid 3 to trial11545420-43 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:43: finishing write tid 4 to trial11545420-43 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:43: oid 43 updating version 0 to 12 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:43: oid 43 version 12 is already newer than 10 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:43: oid 43 version 12 is already newer than 11 2026-02-17T05:46:41.040 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 12 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:44: finishing write tid 1 to trial11545420-44 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:44: finishing write tid 2 to trial11545420-44 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:44: finishing write tid 3 to trial11545420-44 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:44: finishing write tid 4 to trial11545420-44 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:44: oid 44 updating version 0 to 4 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:44: oid 44 updating version 4 to 6 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:44: oid 44 version 6 is already newer than 5 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:45: finishing write tid 1 to trial11545420-45 2026-02-17T05:46:41.041 INFO:tasks.rados.rados.0.trial115.stdout:45: finishing write tid 2 to trial11545420-45 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:45: finishing write tid 3 to trial11545420-45 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:45: finishing write tid 4 to trial11545420-45 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:45: oid 45 updating version 0 to 14 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:45: oid 45 updating version 14 to 15 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:45: oid 45 version 15 is already newer than 13 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 15 (ObjNum 44 snap 0 seq_num 44) dirty exists 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:46: finishing write tid 1 to trial11545420-46 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:46: finishing write tid 2 to trial11545420-46 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:46: finishing write tid 3 to trial11545420-46 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:46: finishing write tid 4 to trial11545420-46 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:46: oid 46 updating version 0 to 17 2026-02-17T05:46:41.042 INFO:tasks.rados.rados.0.trial115.stdout:46: oid 46 updating version 17 to 18 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:46: oid 46 version 18 is already newer than 16 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 18 (ObjNum 45 snap 0 seq_num 45) dirty exists 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:47: finishing write tid 1 to trial11545420-47 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:47: finishing write tid 2 to trial11545420-47 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:47: finishing write tid 3 to trial11545420-47 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:47: finishing write tid 4 to trial11545420-47 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:47: oid 47 updating version 0 to 15 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:47: oid 47 version 15 is already newer than 14 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:47: oid 47 version 15 is already newer than 13 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 15 (ObjNum 46 snap 0 seq_num 46) dirty exists 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:48: finishing write tid 1 to trial11545420-48 2026-02-17T05:46:41.043 INFO:tasks.rados.rados.0.trial115.stdout:48: finishing write tid 2 to trial11545420-48 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:48: finishing write tid 3 to trial11545420-48 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:48: finishing write tid 4 to trial11545420-48 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:48: oid 48 updating version 0 to 13 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:48: oid 48 updating version 13 to 14 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:48: oid 48 updating version 14 to 15 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 15 (ObjNum 47 snap 0 seq_num 47) dirty exists 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:49: finishing write tid 1 to trial11545420-49 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:49: finishing write tid 2 to trial11545420-49 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:49: finishing write tid 3 to trial11545420-49 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:49: finishing write tid 4 to trial11545420-49 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:49: oid 49 updating version 0 to 7 2026-02-17T05:46:41.044 INFO:tasks.rados.rados.0.trial115.stdout:49: oid 49 updating version 7 to 9 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:49: oid 49 version 9 is already newer than 8 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:50: finishing write tid 1 to trial11545420-50 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:50: finishing write tid 2 to trial11545420-50 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:50: finishing write tid 3 to trial11545420-50 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:50: finishing write tid 4 to trial11545420-50 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:50: oid 50 updating version 0 to 12 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:50: oid 50 version 12 is already newer than 11 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:50: oid 50 version 12 is already newer than 10 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 12 (ObjNum 49 snap 0 seq_num 49) dirty exists 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2026-02-17T05:46:41.045 INFO:tasks.rados.rados.0.trial115.stdout:53: finishing write tid 1 to trial11545420-29 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: finishing write tid 2 to trial11545420-29 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:51: finishing copy_from racing read to trial11545420-24 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:51: finishing copy_from to trial11545420-24 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 9 (ObjNum 32 snap 0 seq_num 32) dirty exists 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: finishing write tid 3 to trial11545420-29 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: finishing write tid 4 to trial11545420-29 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: finishing write tid 5 to trial11545420-29 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: oid 29 updating version 0 to 7 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: oid 29 updating version 7 to 8 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: oid 29 updating version 8 to 9 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: oid 29 updating version 9 to 10 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 10 (ObjNum 50 snap 0 seq_num 50) dirty exists 2026-02-17T05:46:41.046 INFO:tasks.rados.rados.0.trial115.stdout:53: left oid 29 (ObjNum 50 snap 0 seq_num 50) 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:54: finishing write tid 1 to trial11545420-15 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:54: finishing write tid 2 to trial11545420-15 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:54: finishing write tid 3 to trial11545420-15 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:54: finishing write tid 4 to trial11545420-15 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:54: oid 15 updating version 0 to 20 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:54: oid 15 updating version 20 to 21 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:54: oid 15 version 21 is already newer than 19 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 21 (ObjNum 51 snap 0 seq_num 51) dirty exists 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:54: left oid 15 (ObjNum 51 snap 0 seq_num 51) 2026-02-17T05:46:41.047 INFO:tasks.rados.rados.0.trial115.stdout:55: expect (ObjNum 20 snap 0 seq_num 20) 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:56: finishing write tid 1 to trial11545420-11 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:56: finishing write tid 2 to trial11545420-11 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:42: done (15 left) 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:43: done (14 left) 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:44: done (13 left) 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:45: done (12 left) 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:46: done (11 left) 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:47: done (10 left) 2026-02-17T05:46:41.094 INFO:tasks.rados.rados.0.trial115.stdout:48: done (9 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:49: done (8 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:50: done (7 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:51: done (6 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:53: done (5 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:54: done (4 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:55: done (3 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:58: done (2 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:59: rollback oid 25 current snap is 0 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:59: done (2 left) 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:60: write oid 3 current snap is 0 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:56: finishing write tid 3 to trial11545420-11 2026-02-17T05:46:41.095 INFO:tasks.rados.rados.0.trial115.stdout:56: finishing write tid 4 to trial11545420-11 2026-02-17T05:46:41.096 INFO:tasks.rados.rados.0.trial115.stdout:56: finishing write tid 5 to trial11545420-11 2026-02-17T05:46:41.096 INFO:tasks.rados.rados.0.trial115.stdout:56: oid 11 updating version 0 to 23 2026-02-17T05:46:41.096 INFO:tasks.rados.rados.0.trial115.stdout:56: oid 11 version 23 is already newer than 22 2026-02-17T05:46:41.096 INFO:tasks.rados.rados.0.trial115.stdout:56: oid 11 updating version 23 to 25 2026-02-17T05:46:41.096 INFO:tasks.rados.rados.0.trial115.stdout:56: oid 11 version 25 is already newer than 24 2026-02-17T05:46:41.096 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 25 (ObjNum 52 snap 0 seq_num 52) dirty exists 2026-02-17T05:46:41.096 INFO:tasks.rados.rados.0.trial115.stdout:56: left oid 11 (ObjNum 52 snap 0 seq_num 52) 2026-02-17T05:46:41.096 INFO:tasks.rados.rados.0.trial115.stdout:57: expect (ObjNum 37 snap 0 seq_num 37) 2026-02-17T05:46:41.154 INFO:tasks.rados.rados.0.trial115.stdout:60: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2026-02-17T05:46:41.158 INFO:tasks.rados.rados.0.trial115.stdout:60: writing trial11545420-3 from 507578 to 985495 tid 1 2026-02-17T05:46:41.162 INFO:tasks.rados.rados.0.trial115.stdout:60: writing trial11545420-3 from 1543557 to 2093561 tid 2 2026-02-17T05:46:41.165 INFO:tasks.rados.rados.0.trial115.stdout:60: writing trial11545420-3 from 2833665 to 2907578 tid 3 2026-02-17T05:46:41.165 INFO:tasks.rados.rados.0.trial115.stdout:56: done (2 left) 2026-02-17T05:46:41.165 INFO:tasks.rados.rados.0.trial115.stdout:57: done (1 left) 2026-02-17T05:46:41.166 INFO:tasks.rados.rados.0.trial115.stdout:61: read oid 10 snap -1 2026-02-17T05:46:41.166 INFO:tasks.rados.rados.0.trial115.stdout:61: expect (ObjNum 9 snap 0 seq_num 9) 2026-02-17T05:46:41.166 INFO:tasks.rados.rados.0.trial115.stdout:62: snap_create 2026-02-17T05:46:41.166 INFO:tasks.rados.rados.0.trial115.stdout:60: finishing write tid 1 to trial11545420-3 2026-02-17T05:46:41.166 INFO:tasks.rados.rados.0.trial115.stdout:60: finishing write tid 2 to trial11545420-3 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:60: finishing write tid 3 to trial11545420-3 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:60: finishing write tid 4 to trial11545420-3 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:60: finishing write tid 5 to trial11545420-3 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:60: oid 3 updating version 0 to 17 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:60: oid 3 version 17 is already newer than 16 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:60: oid 3 updating version 17 to 18 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:60: oid 3 updating version 18 to 19 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 19 (ObjNum 54 snap 0 seq_num 54) dirty exists 2026-02-17T05:46:41.167 INFO:tasks.rados.rados.0.trial115.stdout:60: left oid 3 (ObjNum 54 snap 0 seq_num 54) 2026-02-17T05:46:41.168 INFO:tasks.rados.rados.0.trial115.stdout:61: expect (ObjNum 9 snap 0 seq_num 9) 2026-02-17T05:46:41.560 INFO:tasks.rados.rados.0.trial115.stdout:60: done (2 left) 2026-02-17T05:46:41.560 INFO:tasks.rados.rados.0.trial115.stdout:61: done (1 left) 2026-02-17T05:46:41.561 INFO:tasks.rados.rados.0.trial115.stdout:62: done (0 left) 2026-02-17T05:46:41.561 INFO:tasks.rados.rados.0.trial115.stdout:63: read oid 28 snap -1 2026-02-17T05:46:41.561 INFO:tasks.rados.rados.0.trial115.stdout:63: expect deleted 2026-02-17T05:46:41.561 INFO:tasks.rados.rados.0.trial115.stdout:64: write (excl) oid 41 current snap is 1 2026-02-17T05:46:41.561 INFO:tasks.rados.rados.0.trial115.stdout:64: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2026-02-17T05:46:41.564 INFO:tasks.rados.rados.0.trial115.stdout:64: writing trial11545420-41 from 524385 to 1077551 tid 1 2026-02-17T05:46:41.567 INFO:tasks.rados.rados.0.trial115.stdout:64: writing trial11545420-41 from 1602039 to 2012054 tid 2 2026-02-17T05:46:41.571 INFO:tasks.rados.rados.0.trial115.stdout:64: writing trial11545420-41 from 2577441 to 2924385 tid 3 2026-02-17T05:46:41.571 INFO:tasks.rados.rados.0.trial115.stdout:65: copy_from oid 50 from oid 43 current snap is 1 2026-02-17T05:46:41.571 INFO:tasks.rados.rados.0.trial115.stdout:64: finishing write tid 1 to trial11545420-41 2026-02-17T05:46:41.571 INFO:tasks.rados.rados.0.trial115.stdout:64: finishing write tid 2 to trial11545420-41 2026-02-17T05:46:41.571 INFO:tasks.rados.rados.0.trial115.stdout:63: done (2 left) 2026-02-17T05:46:41.572 INFO:tasks.rados.rados.0.trial115.stdout:66: snap_remove snap 0 2026-02-17T05:46:42.570 INFO:tasks.rados.rados.0.trial115.stdout:66: done (2 left) 2026-02-17T05:46:42.570 INFO:tasks.rados.rados.0.trial115.stdout:67: copy_from oid 35 from oid 18 current snap is 1 2026-02-17T05:46:42.570 INFO:tasks.rados.rados.0.trial115.stdout:64: finishing write tid 3 to trial11545420-41 2026-02-17T05:46:42.570 INFO:tasks.rados.rados.0.trial115.stdout:64: finishing write tid 4 to trial11545420-41 2026-02-17T05:46:42.570 INFO:tasks.rados.rados.0.trial115.stdout:64: finishing write tid 5 to trial11545420-41 2026-02-17T05:46:42.570 INFO:tasks.rados.rados.0.trial115.stdout:64: oid 41 updating version 0 to 15 2026-02-17T05:46:42.570 INFO:tasks.rados.rados.0.trial115.stdout:64: oid 41 updating version 15 to 17 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:64: oid 41 version 17 is already newer than 14 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:64: oid 41 version 17 is already newer than 16 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 17 (ObjNum 55 snap 1 seq_num 55) dirty exists 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:64: left oid 41 (ObjNum 55 snap 1 seq_num 55) 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:65: finishing copy_from racing read to trial11545420-50 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:65: finishing copy_from to trial11545420-50 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 12 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:64: done (2 left) 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:65: done (1 left) 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:68: read oid 30 snap -1 2026-02-17T05:46:42.571 INFO:tasks.rados.rados.0.trial115.stdout:68: expect (ObjNum 29 snap 0 seq_num 29) 2026-02-17T05:46:42.572 INFO:tasks.rados.rados.0.trial115.stdout:69: delete oid 8 current snap is 1 2026-02-17T05:46:42.572 INFO:tasks.rados.rados.0.trial115.stdout:69: done (2 left) 2026-02-17T05:46:42.572 INFO:tasks.rados.rados.0.trial115.stdout:70: copy_from oid 45 from oid 23 current snap is 1 2026-02-17T05:46:42.572 INFO:tasks.rados.rados.0.trial115.stdout:71: snap_create 2026-02-17T05:46:42.574 INFO:tasks.rados.rados.0.trial115.stdout:68: expect (ObjNum 29 snap 0 seq_num 29) 2026-02-17T05:46:42.634 INFO:tasks.rados.rados.0.trial115.stdout:67: finishing copy_from racing read to trial11545420-35 2026-02-17T05:46:42.634 INFO:tasks.rados.rados.0.trial115.stdout:67: finishing copy_from to trial11545420-35 2026-02-17T05:46:42.635 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 3 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-17T05:46:42.635 INFO:tasks.rados.rados.0.trial115.stdout:70: finishing copy_from racing read to trial11545420-45 2026-02-17T05:46:42.635 INFO:tasks.rados.rados.0.trial115.stdout:70: finishing copy_from to trial11545420-45 2026-02-17T05:46:42.635 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2026-02-17T05:46:43.571 INFO:tasks.rados.rados.0.trial115.stdout:67: done (3 left) 2026-02-17T05:46:43.571 INFO:tasks.rados.rados.0.trial115.stdout:68: done (2 left) 2026-02-17T05:46:43.571 INFO:tasks.rados.rados.0.trial115.stdout:70: done (1 left) 2026-02-17T05:46:43.571 INFO:tasks.rados.rados.0.trial115.stdout:71: done (0 left) 2026-02-17T05:46:43.571 INFO:tasks.rados.rados.0.trial115.stdout:72: delete oid 14 current snap is 2 2026-02-17T05:46:43.574 INFO:tasks.rados.rados.0.trial115.stdout:72: done (0 left) 2026-02-17T05:46:43.574 INFO:tasks.rados.rados.0.trial115.stdout:73: read oid 7 snap 1 2026-02-17T05:46:43.574 INFO:tasks.rados.rados.0.trial115.stdout:73: expect (ObjNum 6 snap 0 seq_num 6) 2026-02-17T05:46:43.574 INFO:tasks.rados.rados.0.trial115.stdout:74: write oid 42 current snap is 2 2026-02-17T05:46:43.574 INFO:tasks.rados.rados.0.trial115.stdout:74: seq_num 58 ranges {574806=778913,2061132=557342} 2026-02-17T05:46:43.580 INFO:tasks.rados.rados.0.trial115.stdout:74: writing trial11545420-42 from 574806 to 1353719 tid 1 2026-02-17T05:46:43.585 INFO:tasks.rados.rados.0.trial115.stdout:74: writing trial11545420-42 from 2061132 to 2618474 tid 2 2026-02-17T05:46:43.585 INFO:tasks.rados.rados.0.trial115.stdout:75: read oid 28 snap -1 2026-02-17T05:46:43.585 INFO:tasks.rados.rados.0.trial115.stdout:75: expect deleted 2026-02-17T05:46:43.585 INFO:tasks.rados.rados.0.trial115.stdout:76: write (excl) oid 45 current snap is 2 2026-02-17T05:46:43.585 INFO:tasks.rados.rados.0.trial115.stdout:76: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2026-02-17T05:46:43.588 INFO:tasks.rados.rados.0.trial115.stdout:76: writing trial11545420-45 from 591613 to 1045775 tid 1 2026-02-17T05:46:43.592 INFO:tasks.rados.rados.0.trial115.stdout:76: writing trial11545420-45 from 1719614 to 2136967 tid 2 2026-02-17T05:46:43.596 INFO:tasks.rados.rados.0.trial115.stdout:76: writing trial11545420-45 from 2570780 to 2991613 tid 3 2026-02-17T05:46:43.596 INFO:tasks.rados.rados.0.trial115.stdout:77: read oid 33 snap -1 2026-02-17T05:46:43.596 INFO:tasks.rados.rados.0.trial115.stdout:77: expect (ObjNum 32 snap 0 seq_num 32) 2026-02-17T05:46:43.596 INFO:tasks.rados.rados.0.trial115.stdout:78: rollback oid 38 current snap is 2 2026-02-17T05:46:43.596 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 38 to 1 2026-02-17T05:46:43.596 INFO:tasks.rados.rados.0.trial115.stdout:73: expect (ObjNum 6 snap 0 seq_num 6) 2026-02-17T05:46:43.646 INFO:tasks.rados.rados.0.trial115.stdout:74: finishing write tid 1 to trial11545420-42 2026-02-17T05:46:43.647 INFO:tasks.rados.rados.0.trial115.stdout:73: done (5 left) 2026-02-17T05:46:43.647 INFO:tasks.rados.rados.0.trial115.stdout:79: write (excl) oid 30 current snap is 2 2026-02-17T05:46:43.647 INFO:tasks.rados.rados.0.trial115.stdout:79: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2026-02-17T05:46:43.651 INFO:tasks.rados.rados.0.trial115.stdout:79: writing trial11545420-30 from 608420 to 1137831 tid 1 2026-02-17T05:46:43.655 INFO:tasks.rados.rados.0.trial115.stdout:79: writing trial11545420-30 from 1778096 to 2339107 tid 2 2026-02-17T05:46:43.658 INFO:tasks.rados.rados.0.trial115.stdout:79: writing trial11545420-30 from 2881850 to 3008420 tid 3 2026-02-17T05:46:43.658 INFO:tasks.rados.rados.0.trial115.stdout:80: write (excl) oid 48 current snap is 2 2026-02-17T05:46:43.658 INFO:tasks.rados.rados.0.trial115.stdout:80: seq_num 61 ranges {625227=721013,1952931=704669} 2026-02-17T05:46:43.663 INFO:tasks.rados.rados.0.trial115.stdout:80: writing trial11545420-48 from 625227 to 1346240 tid 1 2026-02-17T05:46:43.668 INFO:tasks.rados.rados.0.trial115.stdout:80: writing trial11545420-48 from 1952931 to 2657600 tid 2 2026-02-17T05:46:43.668 INFO:tasks.rados.rados.0.trial115.stdout:81: rollback oid 25 current snap is 2 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 1 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:74: finishing write tid 2 to trial11545420-42 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:74: finishing write tid 3 to trial11545420-42 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:74: finishing write tid 4 to trial11545420-42 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:74: oid 42 updating version 0 to 21 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:74: oid 42 version 21 is already newer than 19 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:74: oid 42 version 21 is already newer than 20 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 21 (ObjNum 58 snap 2 seq_num 58) dirty exists 2026-02-17T05:46:43.669 INFO:tasks.rados.rados.0.trial115.stdout:74: left oid 42 (ObjNum 58 snap 2 seq_num 58) 2026-02-17T05:46:43.670 INFO:tasks.rados.rados.0.trial115.stdout:76: finishing write tid 1 to trial11545420-45 2026-02-17T05:46:43.670 INFO:tasks.rados.rados.0.trial115.stdout:76: finishing write tid 2 to trial11545420-45 2026-02-17T05:46:43.670 INFO:tasks.rados.rados.0.trial115.stdout:76: finishing write tid 3 to trial11545420-45 2026-02-17T05:46:43.670 INFO:tasks.rados.rados.0.trial115.stdout:74: done (7 left) 2026-02-17T05:46:43.670 INFO:tasks.rados.rados.0.trial115.stdout:75: done (6 left) 2026-02-17T05:46:43.670 INFO:tasks.rados.rados.0.trial115.stdout:82: write (excl) oid 8 current snap is 2 2026-02-17T05:46:43.670 INFO:tasks.rados.rados.0.trial115.stdout:82: seq_num 62 ranges {642034=796262,1895060=564680} 2026-02-17T05:46:43.673 INFO:tasks.rados.rados.0.trial115.stdout:82: writing trial11545420-8 from 642034 to 1438296 tid 1 2026-02-17T05:46:43.677 INFO:tasks.rados.rados.0.trial115.stdout:82: writing trial11545420-8 from 1895060 to 2459740 tid 2 2026-02-17T05:46:43.677 INFO:tasks.rados.rados.0.trial115.stdout:83: read oid 35 snap 1 2026-02-17T05:46:43.677 INFO:tasks.rados.rados.0.trial115.stdout:83: expect (ObjNum 17 snap 0 seq_num 17) 2026-02-17T05:46:43.677 INFO:tasks.rados.rados.0.trial115.stdout:84: write (excl) oid 31 current snap is 2 2026-02-17T05:46:43.678 INFO:tasks.rados.rados.0.trial115.stdout:84: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2026-02-17T05:46:43.681 INFO:tasks.rados.rados.0.trial115.stdout:84: writing trial11545420-31 from 658841 to 1130352 tid 1 2026-02-17T05:46:43.685 INFO:tasks.rados.rados.0.trial115.stdout:84: writing trial11545420-31 from 1553542 to 2261880 tid 2 2026-02-17T05:46:43.689 INFO:tasks.rados.rados.0.trial115.stdout:84: writing trial11545420-31 from 2964119 to 3058841 tid 3 2026-02-17T05:46:43.689 INFO:tasks.rados.rados.0.trial115.stdout:85: read oid 16 snap 1 2026-02-17T05:46:43.689 INFO:tasks.rados.rados.0.trial115.stdout:85: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-17T05:46:43.689 INFO:tasks.rados.rados.0.trial115.stdout:86: rollback oid 40 current snap is 2 2026-02-17T05:46:43.689 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 1 2026-02-17T05:46:43.689 INFO:tasks.rados.rados.0.trial115.stdout:76: finishing write tid 4 to trial11545420-45 2026-02-17T05:46:43.689 INFO:tasks.rados.rados.0.trial115.stdout:76: finishing write tid 5 to trial11545420-45 2026-02-17T05:46:43.690 INFO:tasks.rados.rados.0.trial115.stdout:76: oid 45 updating version 0 to 25 2026-02-17T05:46:43.690 INFO:tasks.rados.rados.0.trial115.stdout:76: oid 45 version 25 is already newer than 24 2026-02-17T05:46:43.690 INFO:tasks.rados.rados.0.trial115.stdout:76: oid 45 updating version 25 to 26 2026-02-17T05:46:43.690 INFO:tasks.rados.rados.0.trial115.stdout:76: oid 45 version 26 is already newer than 23 2026-02-17T05:46:43.690 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 26 (ObjNum 59 snap 2 seq_num 59) dirty exists 2026-02-17T05:46:43.690 INFO:tasks.rados.rados.0.trial115.stdout:76: left oid 45 (ObjNum 59 snap 2 seq_num 59) 2026-02-17T05:46:43.690 INFO:tasks.rados.rados.0.trial115.stdout:77: expect (ObjNum 32 snap 0 seq_num 32) 2026-02-17T05:46:43.750 INFO:tasks.rados.rados.0.trial115.stdout:78: finishing rollback tid 0 to trial11545420-38 2026-02-17T05:46:43.750 INFO:tasks.rados.rados.0.trial115.stdout:78: finishing rollback tid 1 to trial11545420-38 2026-02-17T05:46:43.750 INFO:tasks.rados.rados.0.trial115.stdout:78: finishing rollback tid 2 to trial11545420-38 2026-02-17T05:46:43.750 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 20 (ObjNum 37 snap 0 seq_num 37) dirty exists 2026-02-17T05:46:43.750 INFO:tasks.rados.rados.0.trial115.stdout:76: done (10 left) 2026-02-17T05:46:43.751 INFO:tasks.rados.rados.0.trial115.stdout:77: done (9 left) 2026-02-17T05:46:43.751 INFO:tasks.rados.rados.0.trial115.stdout:78: done (8 left) 2026-02-17T05:46:43.751 INFO:tasks.rados.rados.0.trial115.stdout:87: snap_create 2026-02-17T05:46:43.751 INFO:tasks.rados.rados.0.trial115.stdout:79: finishing write tid 1 to trial11545420-30 2026-02-17T05:46:43.751 INFO:tasks.rados.rados.0.trial115.stdout:79: finishing write tid 2 to trial11545420-30 2026-02-17T05:46:43.751 INFO:tasks.rados.rados.0.trial115.stdout:79: finishing write tid 3 to trial11545420-30 2026-02-17T05:46:43.752 INFO:tasks.rados.rados.0.trial115.stdout:79: finishing write tid 4 to trial11545420-30 2026-02-17T05:46:43.752 INFO:tasks.rados.rados.0.trial115.stdout:79: finishing write tid 5 to trial11545420-30 2026-02-17T05:46:43.752 INFO:tasks.rados.rados.0.trial115.stdout:79: oid 30 updating version 0 to 19 2026-02-17T05:46:43.752 INFO:tasks.rados.rados.0.trial115.stdout:79: oid 30 version 19 is already newer than 18 2026-02-17T05:46:43.752 INFO:tasks.rados.rados.0.trial115.stdout:79: oid 30 updating version 19 to 20 2026-02-17T05:46:43.752 INFO:tasks.rados.rados.0.trial115.stdout:79: oid 30 updating version 20 to 21 2026-02-17T05:46:43.753 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 21 (ObjNum 60 snap 2 seq_num 60) dirty exists 2026-02-17T05:46:43.753 INFO:tasks.rados.rados.0.trial115.stdout:79: left oid 30 (ObjNum 60 snap 2 seq_num 60) 2026-02-17T05:46:43.753 INFO:tasks.rados.rados.0.trial115.stdout:80: finishing write tid 1 to trial11545420-48 2026-02-17T05:46:43.753 INFO:tasks.rados.rados.0.trial115.stdout:80: finishing write tid 2 to trial11545420-48 2026-02-17T05:46:43.753 INFO:tasks.rados.rados.0.trial115.stdout:80: finishing write tid 3 to trial11545420-48 2026-02-17T05:46:43.753 INFO:tasks.rados.rados.0.trial115.stdout:80: finishing write tid 4 to trial11545420-48 2026-02-17T05:46:43.753 INFO:tasks.rados.rados.0.trial115.stdout:80: oid 48 updating version 0 to 22 2026-02-17T05:46:43.754 INFO:tasks.rados.rados.0.trial115.stdout:80: oid 48 updating version 22 to 24 2026-02-17T05:46:43.754 INFO:tasks.rados.rados.0.trial115.stdout:80: oid 48 version 24 is already newer than 23 2026-02-17T05:46:43.754 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 24 (ObjNum 61 snap 2 seq_num 61) dirty exists 2026-02-17T05:46:43.754 INFO:tasks.rados.rados.0.trial115.stdout:80: left oid 48 (ObjNum 61 snap 2 seq_num 61) 2026-02-17T05:46:43.754 INFO:tasks.rados.rados.0.trial115.stdout:81: finishing rollback tid 0 to trial11545420-25 2026-02-17T05:46:43.754 INFO:tasks.rados.rados.0.trial115.stdout:81: finishing rollback tid 1 to trial11545420-25 2026-02-17T05:46:43.754 INFO:tasks.rados.rados.0.trial115.stdout:81: finishing rollback tid 2 to trial11545420-25 2026-02-17T05:46:43.755 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 23 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-17T05:46:43.755 INFO:tasks.rados.rados.0.trial115.stdout:82: finishing write tid 1 to trial11545420-8 2026-02-17T05:46:43.755 INFO:tasks.rados.rados.0.trial115.stdout:82: finishing write tid 2 to trial11545420-8 2026-02-17T05:46:43.755 INFO:tasks.rados.rados.0.trial115.stdout:82: finishing write tid 3 to trial11545420-8 2026-02-17T05:46:43.755 INFO:tasks.rados.rados.0.trial115.stdout:82: finishing write tid 4 to trial11545420-8 2026-02-17T05:46:43.755 INFO:tasks.rados.rados.0.trial115.stdout:82: oid 8 updating version 0 to 5 2026-02-17T05:46:43.756 INFO:tasks.rados.rados.0.trial115.stdout:82: oid 8 updating version 5 to 7 2026-02-17T05:46:43.756 INFO:tasks.rados.rados.0.trial115.stdout:82: oid 8 version 7 is already newer than 6 2026-02-17T05:46:43.756 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 7 (ObjNum 62 snap 2 seq_num 62) dirty exists 2026-02-17T05:46:43.756 INFO:tasks.rados.rados.0.trial115.stdout:82: left oid 8 (ObjNum 62 snap 2 seq_num 62) 2026-02-17T05:46:43.756 INFO:tasks.rados.rados.0.trial115.stdout:84: finishing write tid 1 to trial11545420-31 2026-02-17T05:46:43.756 INFO:tasks.rados.rados.0.trial115.stdout:83: expect (ObjNum 17 snap 0 seq_num 17) 2026-02-17T05:46:43.801 INFO:tasks.rados.rados.0.trial115.stdout:84: finishing write tid 2 to trial11545420-31 2026-02-17T05:46:43.801 INFO:tasks.rados.rados.0.trial115.stdout:84: finishing write tid 3 to trial11545420-31 2026-02-17T05:46:43.801 INFO:tasks.rados.rados.0.trial115.stdout:84: finishing write tid 4 to trial11545420-31 2026-02-17T05:46:43.801 INFO:tasks.rados.rados.0.trial115.stdout:84: finishing write tid 5 to trial11545420-31 2026-02-17T05:46:43.801 INFO:tasks.rados.rados.0.trial115.stdout:84: oid 31 updating version 0 to 22 2026-02-17T05:46:43.801 INFO:tasks.rados.rados.0.trial115.stdout:84: oid 31 version 22 is already newer than 21 2026-02-17T05:46:43.801 INFO:tasks.rados.rados.0.trial115.stdout:84: oid 31 updating version 22 to 24 2026-02-17T05:46:43.802 INFO:tasks.rados.rados.0.trial115.stdout:84: oid 31 version 24 is already newer than 23 2026-02-17T05:46:43.802 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 24 (ObjNum 63 snap 2 seq_num 63) dirty exists 2026-02-17T05:46:43.802 INFO:tasks.rados.rados.0.trial115.stdout:84: left oid 31 (ObjNum 63 snap 2 seq_num 63) 2026-02-17T05:46:43.802 INFO:tasks.rados.rados.0.trial115.stdout:86: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:46:43.802 INFO:tasks.rados.rados.0.trial115.stdout:85: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-17T05:46:43.855 INFO:tasks.rados.rados.0.trial115.stdout:86: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:46:43.855 INFO:tasks.rados.rados.0.trial115.stdout:86: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:46:43.855 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 13 (ObjNum 39 snap 0 seq_num 39) dirty exists 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:79: done (8 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:80: done (7 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:81: done (6 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:82: done (5 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:83: done (4 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:84: done (3 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:85: done (2 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:86: done (1 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:87: done (0 left) 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:88: write oid 48 current snap is 3 2026-02-17T05:46:44.575 INFO:tasks.rados.rados.0.trial115.stdout:88: seq_num 64 ranges {675648=546760,2012024=568349} 2026-02-17T05:46:44.579 INFO:tasks.rados.rados.0.trial115.stdout:88: writing trial11545420-48 from 675648 to 1222408 tid 1 2026-02-17T05:46:44.585 INFO:tasks.rados.rados.0.trial115.stdout:88: writing trial11545420-48 from 2012024 to 2580373 tid 2 2026-02-17T05:46:44.586 INFO:tasks.rados.rados.0.trial115.stdout:89: rollback oid 28 current snap is 3 2026-02-17T05:46:44.586 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 28 to 2 2026-02-17T05:46:44.586 INFO:tasks.rados.rados.0.trial115.stdout:88: finishing write tid 1 to trial11545420-48 2026-02-17T05:46:44.586 INFO:tasks.rados.rados.0.trial115.stdout:90: copy_from oid 32 from oid 21 current snap is 3 2026-02-17T05:46:44.586 INFO:tasks.rados.rados.0.trial115.stdout:91: rollback oid 42 current snap is 3 2026-02-17T05:46:44.586 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 2 2026-02-17T05:46:44.586 INFO:tasks.rados.rados.0.trial115.stdout:92: write oid 34 current snap is 3 2026-02-17T05:46:44.586 INFO:tasks.rados.rados.0.trial115.stdout:92: seq_num 65 ranges {692455=622009,2070506=712007} 2026-02-17T05:46:44.590 INFO:tasks.rados.rados.0.trial115.stdout:92: writing trial11545420-34 from 692455 to 1314464 tid 1 2026-02-17T05:46:44.596 INFO:tasks.rados.rados.0.trial115.stdout:92: writing trial11545420-34 from 2070506 to 2782513 tid 2 2026-02-17T05:46:44.596 INFO:tasks.rados.rados.0.trial115.stdout:93: snap_create 2026-02-17T05:46:44.596 INFO:tasks.rados.rados.0.trial115.stdout:88: finishing write tid 2 to trial11545420-48 2026-02-17T05:46:44.597 INFO:tasks.rados.rados.0.trial115.stdout:89: finishing rollback tid 1 to trial11545420-28 2026-02-17T05:46:44.597 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 0 (ObjNum 1701146144 snap 778331492 seq_num 304) dirty dne 2026-02-17T05:46:44.597 INFO:tasks.rados.rados.0.trial115.stdout:88: finishing write tid 3 to trial11545420-48 2026-02-17T05:46:44.597 INFO:tasks.rados.rados.0.trial115.stdout:88: finishing write tid 4 to trial11545420-48 2026-02-17T05:46:44.597 INFO:tasks.rados.rados.0.trial115.stdout:88: oid 48 updating version 0 to 27 2026-02-17T05:46:44.597 INFO:tasks.rados.rados.0.trial115.stdout:88: oid 48 updating version 27 to 28 2026-02-17T05:46:44.597 INFO:tasks.rados.rados.0.trial115.stdout:88: oid 48 version 28 is already newer than 26 2026-02-17T05:46:44.597 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 28 (ObjNum 64 snap 3 seq_num 64) dirty exists 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:88: left oid 48 (ObjNum 64 snap 3 seq_num 64) 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:91: finishing rollback tid 0 to trial11545420-42 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:91: finishing rollback tid 1 to trial11545420-42 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:91: finishing rollback tid 2 to trial11545420-42 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 30 (ObjNum 58 snap 2 seq_num 58) dirty exists 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:92: finishing write tid 1 to trial11545420-34 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:90: finishing copy_from racing read to trial11545420-32 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:90: finishing copy_from to trial11545420-32 2026-02-17T05:46:44.598 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2026-02-17T05:46:44.600 INFO:tasks.rados.rados.0.trial115.stdout:92: finishing write tid 2 to trial11545420-34 2026-02-17T05:46:44.600 INFO:tasks.rados.rados.0.trial115.stdout:92: finishing write tid 3 to trial11545420-34 2026-02-17T05:46:44.600 INFO:tasks.rados.rados.0.trial115.stdout:92: finishing write tid 4 to trial11545420-34 2026-02-17T05:46:44.600 INFO:tasks.rados.rados.0.trial115.stdout:92: oid 34 updating version 0 to 10 2026-02-17T05:46:44.601 INFO:tasks.rados.rados.0.trial115.stdout:92: oid 34 version 10 is already newer than 9 2026-02-17T05:46:44.601 INFO:tasks.rados.rados.0.trial115.stdout:92: oid 34 version 10 is already newer than 8 2026-02-17T05:46:44.601 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 10 (ObjNum 65 snap 3 seq_num 65) dirty exists 2026-02-17T05:46:44.601 INFO:tasks.rados.rados.0.trial115.stdout:92: left oid 34 (ObjNum 65 snap 3 seq_num 65) 2026-02-17T05:46:45.578 INFO:tasks.rados.rados.0.trial115.stdout:88: done (5 left) 2026-02-17T05:46:45.579 INFO:tasks.rados.rados.0.trial115.stdout:89: done (4 left) 2026-02-17T05:46:45.579 INFO:tasks.rados.rados.0.trial115.stdout:90: done (3 left) 2026-02-17T05:46:45.579 INFO:tasks.rados.rados.0.trial115.stdout:91: done (2 left) 2026-02-17T05:46:45.579 INFO:tasks.rados.rados.0.trial115.stdout:92: done (1 left) 2026-02-17T05:46:45.579 INFO:tasks.rados.rados.0.trial115.stdout:93: done (0 left) 2026-02-17T05:46:45.579 INFO:tasks.rados.rados.0.trial115.stdout:94: copy_from oid 34 from oid 13 current snap is 4 2026-02-17T05:46:45.579 INFO:tasks.rados.rados.0.trial115.stdout:95: write (excl) oid 21 current snap is 4 2026-02-17T05:46:45.579 INFO:tasks.rados.rados.0.trial115.stdout:95: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2026-02-17T05:46:45.584 INFO:tasks.rados.rados.0.trial115.stdout:95: writing trial11545420-21 from 709262 to 1406520 tid 1 2026-02-17T05:46:45.588 INFO:tasks.rados.rados.0.trial115.stdout:95: writing trial11545420-21 from 2012635 to 2584653 tid 2 2026-02-17T05:46:45.590 INFO:tasks.rados.rados.0.trial115.stdout:95: writing trial11545420-21 from 3046388 to 3109262 tid 3 2026-02-17T05:46:45.590 INFO:tasks.rados.rados.0.trial115.stdout:96: delete oid 47 current snap is 4 2026-02-17T05:46:45.591 INFO:tasks.rados.rados.0.trial115.stdout:94: finishing copy_from racing read to trial11545420-34 2026-02-17T05:46:45.591 INFO:tasks.rados.rados.0.trial115.stdout:95: finishing write tid 1 to trial11545420-21 2026-02-17T05:46:45.591 INFO:tasks.rados.rados.0.trial115.stdout:95: finishing write tid 2 to trial11545420-21 2026-02-17T05:46:45.592 INFO:tasks.rados.rados.0.trial115.stdout:94: finishing copy_from to trial11545420-34 2026-02-17T05:46:45.592 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 3 (ObjNum 12 snap 0 seq_num 12) dirty exists 2026-02-17T05:46:45.592 INFO:tasks.rados.rados.0.trial115.stdout:95: finishing write tid 3 to trial11545420-21 2026-02-17T05:46:45.592 INFO:tasks.rados.rados.0.trial115.stdout:95: finishing write tid 4 to trial11545420-21 2026-02-17T05:46:45.592 INFO:tasks.rados.rados.0.trial115.stdout:95: finishing write tid 5 to trial11545420-21 2026-02-17T05:46:45.592 INFO:tasks.rados.rados.0.trial115.stdout:95: oid 21 updating version 0 to 26 2026-02-17T05:46:45.592 INFO:tasks.rados.rados.0.trial115.stdout:95: oid 21 updating version 26 to 27 2026-02-17T05:46:45.593 INFO:tasks.rados.rados.0.trial115.stdout:95: oid 21 version 27 is already newer than 25 2026-02-17T05:46:45.593 INFO:tasks.rados.rados.0.trial115.stdout:95: oid 21 updating version 27 to 28 2026-02-17T05:46:45.593 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 28 (ObjNum 66 snap 4 seq_num 66) dirty exists 2026-02-17T05:46:45.593 INFO:tasks.rados.rados.0.trial115.stdout:95: left oid 21 (ObjNum 66 snap 4 seq_num 66) 2026-02-17T05:46:45.593 INFO:tasks.rados.rados.0.trial115.stdout:94: done (2 left) 2026-02-17T05:46:45.594 INFO:tasks.rados.rados.0.trial115.stdout:95: done (1 left) 2026-02-17T05:46:45.594 INFO:tasks.rados.rados.0.trial115.stdout:96: done (0 left) 2026-02-17T05:46:45.594 INFO:tasks.rados.rados.0.trial115.stdout:97: write oid 39 current snap is 4 2026-02-17T05:46:45.594 INFO:tasks.rados.rados.0.trial115.stdout:97: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2026-02-17T05:46:45.597 INFO:tasks.rados.rados.0.trial115.stdout:97: writing trial11545420-39 from 742876 to 1190632 tid 1 2026-02-17T05:46:45.601 INFO:tasks.rados.rados.0.trial115.stdout:97: writing trial11545420-39 from 1729599 to 2305286 tid 2 2026-02-17T05:46:45.605 INFO:tasks.rados.rados.0.trial115.stdout:97: writing trial11545420-39 from 3101234 to 3142876 tid 3 2026-02-17T05:46:45.605 INFO:tasks.rados.rados.0.trial115.stdout:98: copy_from oid 48 from oid 29 current snap is 4 2026-02-17T05:46:45.605 INFO:tasks.rados.rados.0.trial115.stdout:97: finishing write tid 1 to trial11545420-39 2026-02-17T05:46:45.605 INFO:tasks.rados.rados.0.trial115.stdout:97: finishing write tid 2 to trial11545420-39 2026-02-17T05:46:45.605 INFO:tasks.rados.rados.0.trial115.stdout:99: delete oid 4 current snap is 4 2026-02-17T05:46:45.606 INFO:tasks.rados.rados.0.trial115.stdout:97: finishing write tid 3 to trial11545420-39 2026-02-17T05:46:45.606 INFO:tasks.rados.rados.0.trial115.stdout:97: finishing write tid 4 to trial11545420-39 2026-02-17T05:46:45.607 INFO:tasks.rados.rados.0.trial115.stdout:97: finishing write tid 5 to trial11545420-39 2026-02-17T05:46:45.607 INFO:tasks.rados.rados.0.trial115.stdout:97: oid 39 updating version 0 to 33 2026-02-17T05:46:45.607 INFO:tasks.rados.rados.0.trial115.stdout:97: oid 39 updating version 33 to 35 2026-02-17T05:46:45.607 INFO:tasks.rados.rados.0.trial115.stdout:97: oid 39 version 35 is already newer than 34 2026-02-17T05:46:45.607 INFO:tasks.rados.rados.0.trial115.stdout:97: oid 39 version 35 is already newer than 32 2026-02-17T05:46:45.607 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 35 (ObjNum 68 snap 4 seq_num 68) dirty exists 2026-02-17T05:46:45.607 INFO:tasks.rados.rados.0.trial115.stdout:97: left oid 39 (ObjNum 68 snap 4 seq_num 68) 2026-02-17T05:46:45.608 INFO:tasks.rados.rados.0.trial115.stdout:97: done (2 left) 2026-02-17T05:46:45.608 INFO:tasks.rados.rados.0.trial115.stdout:99: done (1 left) 2026-02-17T05:46:45.608 INFO:tasks.rados.rados.0.trial115.stdout:100: snap_remove snap 3 2026-02-17T05:46:46.581 INFO:tasks.rados.rados.0.trial115.stdout:100: done (1 left) 2026-02-17T05:46:46.581 INFO:tasks.rados.rados.0.trial115.stdout:101: snap_create 2026-02-17T05:46:46.581 INFO:tasks.rados.rados.0.trial115.stdout:98: finishing copy_from racing read to trial11545420-48 2026-02-17T05:46:46.581 INFO:tasks.rados.rados.0.trial115.stdout:98: finishing copy_from to trial11545420-48 2026-02-17T05:46:46.581 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 10 (ObjNum 50 snap 0 seq_num 50) dirty exists 2026-02-17T05:46:47.582 INFO:tasks.rados.rados.0.trial115.stdout:98: done (1 left) 2026-02-17T05:46:47.582 INFO:tasks.rados.rados.0.trial115.stdout:101: done (0 left) 2026-02-17T05:46:47.582 INFO:tasks.rados.rados.0.trial115.stdout:102: rollback oid 25 current snap is 5 2026-02-17T05:46:47.582 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 4 2026-02-17T05:46:47.582 INFO:tasks.rados.rados.0.trial115.stdout:103: delete oid 34 current snap is 5 2026-02-17T05:46:47.585 INFO:tasks.rados.rados.0.trial115.stdout:103: done (1 left) 2026-02-17T05:46:47.585 INFO:tasks.rados.rados.0.trial115.stdout:104: snap_remove snap 2 2026-02-17T05:46:48.585 INFO:tasks.rados.rados.0.trial115.stdout:104: done (1 left) 2026-02-17T05:46:48.585 INFO:tasks.rados.rados.0.trial115.stdout:105: snap_create 2026-02-17T05:46:48.585 INFO:tasks.rados.rados.0.trial115.stdout:102: finishing rollback tid 0 to trial11545420-25 2026-02-17T05:46:48.585 INFO:tasks.rados.rados.0.trial115.stdout:102: finishing rollback tid 1 to trial11545420-25 2026-02-17T05:46:48.586 INFO:tasks.rados.rados.0.trial115.stdout:102: finishing rollback tid 2 to trial11545420-25 2026-02-17T05:46:48.586 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 34 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-17T05:46:49.586 INFO:tasks.rados.rados.0.trial115.stdout:102: done (1 left) 2026-02-17T05:46:49.586 INFO:tasks.rados.rados.0.trial115.stdout:105: done (0 left) 2026-02-17T05:46:49.586 INFO:tasks.rados.rados.0.trial115.stdout:106: rollback oid 29 current snap is 6 2026-02-17T05:46:49.586 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 4 2026-02-17T05:46:49.586 INFO:tasks.rados.rados.0.trial115.stdout:107: write oid 14 current snap is 6 2026-02-17T05:46:49.586 INFO:tasks.rados.rados.0.trial115.stdout:107: seq_num 71 ranges {793297=789856,2305045=723014} 2026-02-17T05:46:49.592 INFO:tasks.rados.rados.0.trial115.stdout:107: writing trial11545420-14 from 793297 to 1583153 tid 1 2026-02-17T05:46:49.597 INFO:tasks.rados.rados.0.trial115.stdout:107: writing trial11545420-14 from 2305045 to 3028059 tid 2 2026-02-17T05:46:49.598 INFO:tasks.rados.rados.0.trial115.stdout:108: delete oid 21 current snap is 6 2026-02-17T05:46:49.598 INFO:tasks.rados.rados.0.trial115.stdout:106: finishing rollback tid 0 to trial11545420-29 2026-02-17T05:46:49.598 INFO:tasks.rados.rados.0.trial115.stdout:106: finishing rollback tid 1 to trial11545420-29 2026-02-17T05:46:49.598 INFO:tasks.rados.rados.0.trial115.stdout:106: finishing rollback tid 2 to trial11545420-29 2026-02-17T05:46:49.598 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 14 (ObjNum 50 snap 0 seq_num 50) dirty exists 2026-02-17T05:46:49.598 INFO:tasks.rados.rados.0.trial115.stdout:107: finishing write tid 1 to trial11545420-14 2026-02-17T05:46:49.599 INFO:tasks.rados.rados.0.trial115.stdout:106: done (2 left) 2026-02-17T05:46:49.599 INFO:tasks.rados.rados.0.trial115.stdout:108: done (1 left) 2026-02-17T05:46:49.600 INFO:tasks.rados.rados.0.trial115.stdout:109: read oid 13 snap 5 2026-02-17T05:46:49.600 INFO:tasks.rados.rados.0.trial115.stdout:109: expect (ObjNum 12 snap 0 seq_num 12) 2026-02-17T05:46:49.600 INFO:tasks.rados.rados.0.trial115.stdout:110: snap_create 2026-02-17T05:46:49.600 INFO:tasks.rados.rados.0.trial115.stdout:107: finishing write tid 2 to trial11545420-14 2026-02-17T05:46:49.600 INFO:tasks.rados.rados.0.trial115.stdout:107: finishing write tid 3 to trial11545420-14 2026-02-17T05:46:49.601 INFO:tasks.rados.rados.0.trial115.stdout:107: finishing write tid 4 to trial11545420-14 2026-02-17T05:46:49.601 INFO:tasks.rados.rados.0.trial115.stdout:107: oid 14 updating version 0 to 15 2026-02-17T05:46:49.601 INFO:tasks.rados.rados.0.trial115.stdout:107: oid 14 updating version 15 to 17 2026-02-17T05:46:49.601 INFO:tasks.rados.rados.0.trial115.stdout:107: oid 14 version 17 is already newer than 16 2026-02-17T05:46:49.601 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 17 (ObjNum 71 snap 6 seq_num 71) dirty exists 2026-02-17T05:46:49.601 INFO:tasks.rados.rados.0.trial115.stdout:107: left oid 14 (ObjNum 71 snap 6 seq_num 71) 2026-02-17T05:46:49.601 INFO:tasks.rados.rados.0.trial115.stdout:109: expect (ObjNum 12 snap 0 seq_num 12) 2026-02-17T05:46:50.204 INFO:tasks.rados.rados.0.trial115.stdout:107: done (2 left) 2026-02-17T05:46:50.204 INFO:tasks.rados.rados.0.trial115.stdout:109: done (1 left) 2026-02-17T05:46:50.204 INFO:tasks.rados.rados.0.trial115.stdout:110: done (0 left) 2026-02-17T05:46:50.204 INFO:tasks.rados.rados.0.trial115.stdout:111: rollback oid 45 current snap is 7 2026-02-17T05:46:50.204 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 5 2026-02-17T05:46:50.204 INFO:tasks.rados.rados.0.trial115.stdout:112: snap_create 2026-02-17T05:46:50.207 INFO:tasks.rados.rados.0.trial115.stdout:111: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:46:50.207 INFO:tasks.rados.rados.0.trial115.stdout:111: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:46:50.207 INFO:tasks.rados.rados.0.trial115.stdout:111: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:46:50.207 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 45 (ObjNum 59 snap 2 seq_num 59) dirty exists 2026-02-17T05:46:51.204 INFO:tasks.rados.rados.0.trial115.stdout:111: done (1 left) 2026-02-17T05:46:51.204 INFO:tasks.rados.rados.0.trial115.stdout:112: done (0 left) 2026-02-17T05:46:51.204 INFO:tasks.rados.rados.0.trial115.stdout:113: rollback oid 34 current snap is 8 2026-02-17T05:46:51.205 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 4 2026-02-17T05:46:51.205 INFO:tasks.rados.rados.0.trial115.stdout:114: write (excl) oid 28 current snap is 8 2026-02-17T05:46:51.205 INFO:tasks.rados.rados.0.trial115.stdout:114: seq_num 73 ranges {426911=540354,1622009=726683,3121996=104915} 2026-02-17T05:46:51.208 INFO:tasks.rados.rados.0.trial115.stdout:114: writing trial11545420-28 from 426911 to 967265 tid 1 2026-02-17T05:46:51.213 INFO:tasks.rados.rados.0.trial115.stdout:114: writing trial11545420-28 from 1622009 to 2348692 tid 2 2026-02-17T05:46:51.217 INFO:tasks.rados.rados.0.trial115.stdout:114: writing trial11545420-28 from 3121996 to 3226911 tid 3 2026-02-17T05:46:51.217 INFO:tasks.rados.rados.0.trial115.stdout:115: write (excl) oid 13 current snap is 8 2026-02-17T05:46:51.217 INFO:tasks.rados.rados.0.trial115.stdout:115: seq_num 74 ranges {443718=615603,1564138=470341,2633066=610652} 2026-02-17T05:46:51.220 INFO:tasks.rados.rados.0.trial115.stdout:115: writing trial11545420-13 from 443718 to 1059321 tid 1 2026-02-17T05:46:51.224 INFO:tasks.rados.rados.0.trial115.stdout:115: writing trial11545420-13 from 1564138 to 2034479 tid 2 2026-02-17T05:46:51.229 INFO:tasks.rados.rados.0.trial115.stdout:115: writing trial11545420-13 from 2633066 to 3243718 tid 3 2026-02-17T05:46:51.229 INFO:tasks.rados.rados.0.trial115.stdout:116: snap_remove snap 6 2026-02-17T05:46:51.229 INFO:tasks.rados.rados.0.trial115.stdout:113: finishing rollback tid 1 to trial11545420-34 2026-02-17T05:46:51.229 INFO:tasks.rados.rados.0.trial115.stdout:113: finishing rollback tid 2 to trial11545420-34 2026-02-17T05:46:51.229 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 20 (ObjNum 12 snap 0 seq_num 12) dirty exists 2026-02-17T05:46:51.229 INFO:tasks.rados.rados.0.trial115.stdout:114: finishing write tid 1 to trial11545420-28 2026-02-17T05:46:52.206 INFO:tasks.rados.rados.0.trial115.stdout:113: done (3 left) 2026-02-17T05:46:52.207 INFO:tasks.rados.rados.0.trial115.stdout:116: done (2 left) 2026-02-17T05:46:52.207 INFO:tasks.rados.rados.0.trial115.stdout:117: write (excl) oid 32 current snap is 8 2026-02-17T05:46:52.207 INFO:tasks.rados.rados.0.trial115.stdout:117: seq_num 75 ranges {460525=690852,1622620=730352,3060489=200036} 2026-02-17T05:46:52.211 INFO:tasks.rados.rados.0.trial115.stdout:117: writing trial11545420-32 from 460525 to 1151377 tid 1 2026-02-17T05:46:52.215 INFO:tasks.rados.rados.0.trial115.stdout:117: writing trial11545420-32 from 1622620 to 2352972 tid 2 2026-02-17T05:46:52.219 INFO:tasks.rados.rados.0.trial115.stdout:117: writing trial11545420-32 from 3060489 to 3260525 tid 3 2026-02-17T05:46:52.219 INFO:tasks.rados.rados.0.trial115.stdout:118: write oid 9 current snap is 8 2026-02-17T05:46:52.219 INFO:tasks.rados.rados.0.trial115.stdout:118: seq_num 76 ranges {477332=766101,1681102=474010,2687912=589420} 2026-02-17T05:46:52.223 INFO:tasks.rados.rados.0.trial115.stdout:118: writing trial11545420-9 from 477332 to 1243433 tid 1 2026-02-17T05:46:52.226 INFO:tasks.rados.rados.0.trial115.stdout:118: writing trial11545420-9 from 1681102 to 2155112 tid 2 2026-02-17T05:46:52.231 INFO:tasks.rados.rados.0.trial115.stdout:118: writing trial11545420-9 from 2687912 to 3277332 tid 3 2026-02-17T05:46:52.231 INFO:tasks.rados.rados.0.trial115.stdout:119: write oid 48 current snap is 8 2026-02-17T05:46:52.231 INFO:tasks.rados.rados.0.trial115.stdout:119: seq_num 77 ranges {494139=557703,1455937=734021,2948041=346098} 2026-02-17T05:46:52.234 INFO:tasks.rados.rados.0.trial115.stdout:119: writing trial11545420-48 from 494139 to 1051842 tid 1 2026-02-17T05:46:52.238 INFO:tasks.rados.rados.0.trial115.stdout:119: writing trial11545420-48 from 1455937 to 2189958 tid 2 2026-02-17T05:46:52.243 INFO:tasks.rados.rados.0.trial115.stdout:119: writing trial11545420-48 from 2948041 to 3294139 tid 3 2026-02-17T05:46:52.243 INFO:tasks.rados.rados.0.trial115.stdout:120: write oid 34 current snap is 8 2026-02-17T05:46:52.243 INFO:tasks.rados.rados.0.trial115.stdout:120: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2026-02-17T05:46:52.247 INFO:tasks.rados.rados.0.trial115.stdout:120: writing trial11545420-34 from 510946 to 1143898 tid 1 2026-02-17T05:46:52.251 INFO:tasks.rados.rados.0.trial115.stdout:120: writing trial11545420-34 from 1798066 to 2275745 tid 2 2026-02-17T05:46:52.255 INFO:tasks.rados.rados.0.trial115.stdout:120: writing trial11545420-34 from 2742758 to 3199975 tid 3 2026-02-17T05:46:52.255 INFO:tasks.rados.rados.0.trial115.stdout:121: snap_remove snap 4 2026-02-17T05:46:53.209 INFO:tasks.rados.rados.0.trial115.stdout:121: done (6 left) 2026-02-17T05:46:53.209 INFO:tasks.rados.rados.0.trial115.stdout:122: copy_from oid 45 from oid 42 current snap is 8 2026-02-17T05:46:53.209 INFO:tasks.rados.rados.0.trial115.stdout:114: finishing write tid 2 to trial11545420-28 2026-02-17T05:46:53.209 INFO:tasks.rados.rados.0.trial115.stdout:114: finishing write tid 3 to trial11545420-28 2026-02-17T05:46:53.209 INFO:tasks.rados.rados.0.trial115.stdout:114: finishing write tid 4 to trial11545420-28 2026-02-17T05:46:53.209 INFO:tasks.rados.rados.0.trial115.stdout:114: finishing write tid 5 to trial11545420-28 2026-02-17T05:46:53.209 INFO:tasks.rados.rados.0.trial115.stdout:114: oid 28 updating version 0 to 28 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:114: oid 28 updating version 28 to 31 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:114: oid 28 version 31 is already newer than 30 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:114: oid 28 version 31 is already newer than 29 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 31 (ObjNum 73 snap 8 seq_num 73) dirty exists 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:114: left oid 28 (ObjNum 73 snap 8 seq_num 73) 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: finishing write tid 1 to trial11545420-13 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:114: done (6 left) 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:123: delete oid 12 current snap is 8 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: finishing write tid 2 to trial11545420-13 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: finishing write tid 3 to trial11545420-13 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: finishing write tid 4 to trial11545420-13 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: finishing write tid 5 to trial11545420-13 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: oid 13 updating version 0 to 18 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: oid 13 version 18 is already newer than 17 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: oid 13 updating version 18 to 19 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: oid 13 version 19 is already newer than 16 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 19 (ObjNum 74 snap 8 seq_num 74) dirty exists 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:115: left oid 13 (ObjNum 74 snap 8 seq_num 74) 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: finishing write tid 1 to trial11545420-32 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: finishing write tid 2 to trial11545420-32 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: finishing write tid 3 to trial11545420-32 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: finishing write tid 4 to trial11545420-32 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: finishing write tid 5 to trial11545420-32 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: oid 32 updating version 0 to 13 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: oid 32 updating version 13 to 15 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: oid 32 version 15 is already newer than 12 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: oid 32 version 15 is already newer than 14 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 15 (ObjNum 75 snap 8 seq_num 75) dirty exists 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:117: left oid 32 (ObjNum 75 snap 8 seq_num 75) 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: finishing write tid 1 to trial11545420-9 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: finishing write tid 2 to trial11545420-9 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: finishing write tid 3 to trial11545420-9 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: finishing write tid 4 to trial11545420-9 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: finishing write tid 5 to trial11545420-9 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: oid 9 updating version 0 to 34 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: oid 9 version 34 is already newer than 33 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: oid 9 updating version 34 to 36 2026-02-17T05:46:53.210 INFO:tasks.rados.rados.0.trial115.stdout:118: oid 9 version 36 is already newer than 35 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 36 (ObjNum 76 snap 8 seq_num 76) dirty exists 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:118: left oid 9 (ObjNum 76 snap 8 seq_num 76) 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: finishing write tid 1 to trial11545420-48 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: finishing write tid 2 to trial11545420-48 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: finishing write tid 3 to trial11545420-48 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: finishing write tid 4 to trial11545420-48 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: finishing write tid 5 to trial11545420-48 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: oid 48 updating version 0 to 37 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: oid 48 version 37 is already newer than 36 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: oid 48 updating version 37 to 38 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: oid 48 updating version 38 to 39 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 39 (ObjNum 77 snap 8 seq_num 77) dirty exists 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:119: left oid 48 (ObjNum 77 snap 8 seq_num 77) 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: finishing write tid 1 to trial11545420-34 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: finishing write tid 2 to trial11545420-34 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: finishing write tid 3 to trial11545420-34 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: finishing write tid 4 to trial11545420-34 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: finishing write tid 5 to trial11545420-34 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: oid 34 updating version 0 to 23 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: oid 34 updating version 23 to 24 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: oid 34 version 24 is already newer than 21 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: oid 34 version 24 is already newer than 22 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 24 (ObjNum 78 snap 8 seq_num 78) dirty exists 2026-02-17T05:46:53.211 INFO:tasks.rados.rados.0.trial115.stdout:120: left oid 34 (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:46:53.214 INFO:tasks.rados.rados.0.trial115.stdout:115: done (6 left) 2026-02-17T05:46:53.214 INFO:tasks.rados.rados.0.trial115.stdout:117: done (5 left) 2026-02-17T05:46:53.214 INFO:tasks.rados.rados.0.trial115.stdout:118: done (4 left) 2026-02-17T05:46:53.214 INFO:tasks.rados.rados.0.trial115.stdout:119: done (3 left) 2026-02-17T05:46:53.214 INFO:tasks.rados.rados.0.trial115.stdout:120: done (2 left) 2026-02-17T05:46:53.215 INFO:tasks.rados.rados.0.trial115.stdout:123: done (1 left) 2026-02-17T05:46:53.215 INFO:tasks.rados.rados.0.trial115.stdout:124: read oid 41 snap 5 2026-02-17T05:46:53.215 INFO:tasks.rados.rados.0.trial115.stdout:124: expect (ObjNum 55 snap 1 seq_num 55) 2026-02-17T05:46:53.215 INFO:tasks.rados.rados.0.trial115.stdout:125: read oid 33 snap -1 2026-02-17T05:46:53.215 INFO:tasks.rados.rados.0.trial115.stdout:125: expect (ObjNum 32 snap 0 seq_num 32) 2026-02-17T05:46:53.215 INFO:tasks.rados.rados.0.trial115.stdout:126: delete oid 50 current snap is 8 2026-02-17T05:46:53.216 INFO:tasks.rados.rados.0.trial115.stdout:125: expect (ObjNum 32 snap 0 seq_num 32) 2026-02-17T05:46:53.277 INFO:tasks.rados.rados.0.trial115.stdout:122: finishing copy_from racing read to trial11545420-45 2026-02-17T05:46:53.277 INFO:tasks.rados.rados.0.trial115.stdout:122: finishing copy_from to trial11545420-45 2026-02-17T05:46:53.277 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 30 (ObjNum 58 snap 2 seq_num 58) dirty exists 2026-02-17T05:46:53.277 INFO:tasks.rados.rados.0.trial115.stdout:124: expect (ObjNum 55 snap 1 seq_num 55) 2026-02-17T05:46:53.365 INFO:tasks.rados.rados.0.trial115.stdout:122: done (3 left) 2026-02-17T05:46:53.365 INFO:tasks.rados.rados.0.trial115.stdout:124: done (2 left) 2026-02-17T05:46:53.365 INFO:tasks.rados.rados.0.trial115.stdout:125: done (1 left) 2026-02-17T05:46:53.365 INFO:tasks.rados.rados.0.trial115.stdout:126: done (0 left) 2026-02-17T05:46:53.366 INFO:tasks.rados.rados.0.trial115.stdout:127: write (excl) oid 29 current snap is 8 2026-02-17T05:46:53.366 INFO:tasks.rados.rados.0.trial115.stdout:127: seq_num 81 ranges {561367=458699,1573512=741359,2941380=419987} 2026-02-17T05:46:53.368 INFO:tasks.rados.rados.0.trial115.stdout:127: writing trial11545420-29 from 561367 to 1020066 tid 1 2026-02-17T05:46:53.373 INFO:tasks.rados.rados.0.trial115.stdout:127: writing trial11545420-29 from 1573512 to 2314871 tid 2 2026-02-17T05:46:53.378 INFO:tasks.rados.rados.0.trial115.stdout:127: writing trial11545420-29 from 2941380 to 3361367 tid 3 2026-02-17T05:46:53.378 INFO:tasks.rados.rados.0.trial115.stdout:128: copy_from oid 41 from oid 43 current snap is 8 2026-02-17T05:46:53.378 INFO:tasks.rados.rados.0.trial115.stdout:127: finishing write tid 1 to trial11545420-29 2026-02-17T05:46:53.378 INFO:tasks.rados.rados.0.trial115.stdout:127: finishing write tid 2 to trial11545420-29 2026-02-17T05:46:53.378 INFO:tasks.rados.rados.0.trial115.stdout:129: snap_remove snap 7 2026-02-17T05:46:54.212 INFO:tasks.rados.rados.0.trial115.stdout:129: done (2 left) 2026-02-17T05:46:54.212 INFO:tasks.rados.rados.0.trial115.stdout:130: read oid 45 snap -1 2026-02-17T05:46:54.212 INFO:tasks.rados.rados.0.trial115.stdout:130: expect (ObjNum 58 snap 2 seq_num 58) 2026-02-17T05:46:54.212 INFO:tasks.rados.rados.0.trial115.stdout:131: rollback oid 35 current snap is 8 2026-02-17T05:46:54.212 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 35 to 1 2026-02-17T05:46:54.212 INFO:tasks.rados.rados.0.trial115.stdout:127: finishing write tid 3 to trial11545420-29 2026-02-17T05:46:54.212 INFO:tasks.rados.rados.0.trial115.stdout:127: finishing write tid 4 to trial11545420-29 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:127: finishing write tid 5 to trial11545420-29 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:127: oid 29 updating version 0 to 28 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:127: oid 29 updating version 28 to 30 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:127: oid 29 version 30 is already newer than 27 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:127: oid 29 version 30 is already newer than 29 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 30 (ObjNum 81 snap 8 seq_num 81) dirty exists 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:127: left oid 29 (ObjNum 81 snap 8 seq_num 81) 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:128: finishing copy_from racing read to trial11545420-41 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:128: finishing copy_from to trial11545420-41 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 12 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:127: done (3 left) 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:128: done (2 left) 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:132: write (excl) oid 28 current snap is 8 2026-02-17T05:46:54.213 INFO:tasks.rados.rados.0.trial115.stdout:132: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2026-02-17T05:46:54.216 INFO:tasks.rados.rados.0.trial115.stdout:132: writing trial11545420-28 from 578174 to 1112122 tid 1 2026-02-17T05:46:54.220 INFO:tasks.rados.rados.0.trial115.stdout:132: writing trial11545420-28 from 1515641 to 2000658 tid 2 2026-02-17T05:46:54.225 INFO:tasks.rados.rados.0.trial115.stdout:132: writing trial11545420-28 from 2736097 to 3238402 tid 3 2026-02-17T05:46:54.225 INFO:tasks.rados.rados.0.trial115.stdout:133: write (excl) oid 29 current snap is 8 2026-02-17T05:46:54.225 INFO:tasks.rados.rados.0.trial115.stdout:133: seq_num 83 ranges {594981=609197,1974123=745028,3279873=115108} 2026-02-17T05:46:54.230 INFO:tasks.rados.rados.0.trial115.stdout:133: writing trial11545420-29 from 594981 to 1204178 tid 1 2026-02-17T05:46:54.236 INFO:tasks.rados.rados.0.trial115.stdout:133: writing trial11545420-29 from 1974123 to 2719151 tid 2 2026-02-17T05:46:54.239 INFO:tasks.rados.rados.0.trial115.stdout:133: writing trial11545420-29 from 3279873 to 3394981 tid 3 2026-02-17T05:46:54.239 INFO:tasks.rados.rados.0.trial115.stdout:134: write (excl) oid 5 current snap is 8 2026-02-17T05:46:54.239 INFO:tasks.rados.rados.0.trial115.stdout:134: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2026-02-17T05:46:54.242 INFO:tasks.rados.rados.0.trial115.stdout:134: writing trial11545420-5 from 611788 to 1012587 tid 1 2026-02-17T05:46:54.247 INFO:tasks.rados.rados.0.trial115.stdout:134: writing trial11545420-5 from 1748958 to 2237644 tid 2 2026-02-17T05:46:54.252 INFO:tasks.rados.rados.0.trial115.stdout:134: writing trial11545420-5 from 2907296 to 3411788 tid 3 2026-02-17T05:46:54.252 INFO:tasks.rados.rados.0.trial115.stdout:135: copy_from oid 33 from oid 27 current snap is 8 2026-02-17T05:46:54.252 INFO:tasks.rados.rados.0.trial115.stdout:131: finishing rollback tid 0 to trial11545420-35 2026-02-17T05:46:54.252 INFO:tasks.rados.rados.0.trial115.stdout:131: finishing rollback tid 1 to trial11545420-35 2026-02-17T05:46:54.253 INFO:tasks.rados.rados.0.trial115.stdout:131: finishing rollback tid 2 to trial11545420-35 2026-02-17T05:46:54.253 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 45 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-17T05:46:54.253 INFO:tasks.rados.rados.0.trial115.stdout:131: done (5 left) 2026-02-17T05:46:54.253 INFO:tasks.rados.rados.0.trial115.stdout:136: snap_remove snap 5 2026-02-17T05:46:55.205 INFO:tasks.rados.rados.0.trial115.stdout:136: done (5 left) 2026-02-17T05:46:55.206 INFO:tasks.rados.rados.0.trial115.stdout:137: delete oid 46 current snap is 8 2026-02-17T05:46:55.206 INFO:tasks.rados.rados.0.trial115.stdout:130: expect (ObjNum 58 snap 2 seq_num 58) 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: finishing write tid 1 to trial11545420-28 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: finishing write tid 2 to trial11545420-28 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: finishing write tid 3 to trial11545420-28 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: finishing write tid 4 to trial11545420-28 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:130: done (5 left) 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:137: done (4 left) 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:138: write oid 31 current snap is 8 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: finishing write tid 5 to trial11545420-28 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: oid 28 updating version 0 to 42 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: oid 28 version 42 is already newer than 41 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: oid 28 updating version 42 to 44 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: oid 28 version 44 is already newer than 43 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 44 (ObjNum 82 snap 8 seq_num 82) dirty exists 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:132: left oid 28 (ObjNum 82 snap 8 seq_num 82) 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:133: finishing write tid 1 to trial11545420-29 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:133: finishing write tid 2 to trial11545420-29 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:133: finishing write tid 3 to trial11545420-29 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:133: finishing write tid 4 to trial11545420-29 2026-02-17T05:46:55.302 INFO:tasks.rados.rados.0.trial115.stdout:133: finishing write tid 5 to trial11545420-29 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:133: oid 29 updating version 0 to 36 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:133: oid 29 updating version 36 to 37 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:133: oid 29 version 37 is already newer than 35 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:133: oid 29 updating version 37 to 38 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 38 (ObjNum 83 snap 8 seq_num 83) dirty exists 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:133: left oid 29 (ObjNum 83 snap 8 seq_num 83) 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:134: finishing write tid 1 to trial11545420-5 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:134: finishing write tid 2 to trial11545420-5 2026-02-17T05:46:55.303 INFO:tasks.rados.rados.0.trial115.stdout:138: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2026-02-17T05:46:55.306 INFO:tasks.rados.rados.0.trial115.stdout:138: writing trial11545420-31 from 645402 to 1196699 tid 1 2026-02-17T05:46:55.310 INFO:tasks.rados.rados.0.trial115.stdout:138: writing trial11545420-31 from 1749569 to 2241924 tid 2 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:138: writing trial11545420-31 from 2845789 to 3445402 tid 3 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:132: done (4 left) 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:133: done (3 left) 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:139: rollback oid 20 current snap is 8 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 1 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:134: finishing write tid 3 to trial11545420-5 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:134: finishing write tid 4 to trial11545420-5 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:134: finishing write tid 5 to trial11545420-5 2026-02-17T05:46:55.315 INFO:tasks.rados.rados.0.trial115.stdout:134: oid 5 updating version 0 to 33 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:134: oid 5 updating version 33 to 35 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:134: oid 5 version 35 is already newer than 34 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:134: oid 5 version 35 is already newer than 32 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 35 (ObjNum 84 snap 8 seq_num 84) dirty exists 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:134: left oid 5 (ObjNum 84 snap 8 seq_num 84) 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:135: finishing copy_from racing read to trial11545420-33 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:135: finishing copy_from to trial11545420-33 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:138: finishing write tid 1 to trial11545420-31 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:138: finishing write tid 2 to trial11545420-31 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:134: done (3 left) 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:135: done (2 left) 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:140: read oid 27 snap -1 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:140: expect (ObjNum 26 snap 0 seq_num 26) 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:141: write (excl) oid 37 current snap is 8 2026-02-17T05:46:55.316 INFO:tasks.rados.rados.0.trial115.stdout:141: seq_num 87 ranges {662209=626546,1808051=636013,2873212=588997} 2026-02-17T05:46:55.321 INFO:tasks.rados.rados.0.trial115.stdout:141: writing trial11545420-37 from 662209 to 1288755 tid 1 2026-02-17T05:46:55.325 INFO:tasks.rados.rados.0.trial115.stdout:141: writing trial11545420-37 from 1808051 to 2444064 tid 2 2026-02-17T05:46:55.330 INFO:tasks.rados.rados.0.trial115.stdout:141: writing trial11545420-37 from 2873212 to 3462209 tid 3 2026-02-17T05:46:55.330 INFO:tasks.rados.rados.0.trial115.stdout:142: read oid 10 snap 1 2026-02-17T05:46:55.330 INFO:tasks.rados.rados.0.trial115.stdout:142: expect (ObjNum 9 snap 0 seq_num 9) 2026-02-17T05:46:55.330 INFO:tasks.rados.rados.0.trial115.stdout:143: copy_from oid 40 from oid 25 current snap is 8 2026-02-17T05:46:55.330 INFO:tasks.rados.rados.0.trial115.stdout:139: finishing rollback tid 0 to trial11545420-20 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:138: finishing write tid 3 to trial11545420-31 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:139: finishing rollback tid 1 to trial11545420-20 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:138: finishing write tid 4 to trial11545420-31 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:139: finishing rollback tid 2 to trial11545420-20 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 16 (ObjNum 19 snap 0 seq_num 19) dirty exists 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:138: finishing write tid 5 to trial11545420-31 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:138: oid 31 updating version 0 to 38 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:138: oid 31 updating version 38 to 41 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:138: oid 31 version 41 is already newer than 39 2026-02-17T05:46:55.331 INFO:tasks.rados.rados.0.trial115.stdout:138: oid 31 version 41 is already newer than 40 2026-02-17T05:46:55.332 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 41 (ObjNum 86 snap 8 seq_num 86) dirty exists 2026-02-17T05:46:55.332 INFO:tasks.rados.rados.0.trial115.stdout:138: left oid 31 (ObjNum 86 snap 8 seq_num 86) 2026-02-17T05:46:55.332 INFO:tasks.rados.rados.0.trial115.stdout:140: expect (ObjNum 26 snap 0 seq_num 26) 2026-02-17T05:46:55.385 INFO:tasks.rados.rados.0.trial115.stdout:141: finishing write tid 1 to trial11545420-37 2026-02-17T05:46:55.385 INFO:tasks.rados.rados.0.trial115.stdout:141: finishing write tid 2 to trial11545420-37 2026-02-17T05:46:55.385 INFO:tasks.rados.rados.0.trial115.stdout:138: done (5 left) 2026-02-17T05:46:55.385 INFO:tasks.rados.rados.0.trial115.stdout:139: done (4 left) 2026-02-17T05:46:55.385 INFO:tasks.rados.rados.0.trial115.stdout:140: done (3 left) 2026-02-17T05:46:55.385 INFO:tasks.rados.rados.0.trial115.stdout:144: snap_create 2026-02-17T05:46:55.385 INFO:tasks.rados.rados.0.trial115.stdout:142: expect (ObjNum 9 snap 0 seq_num 9) 2026-02-17T05:46:55.434 INFO:tasks.rados.rados.0.trial115.stdout:141: finishing write tid 3 to trial11545420-37 2026-02-17T05:46:55.434 INFO:tasks.rados.rados.0.trial115.stdout:141: finishing write tid 4 to trial11545420-37 2026-02-17T05:46:55.434 INFO:tasks.rados.rados.0.trial115.stdout:141: finishing write tid 5 to trial11545420-37 2026-02-17T05:46:55.434 INFO:tasks.rados.rados.0.trial115.stdout:141: oid 37 updating version 0 to 21 2026-02-17T05:46:55.434 INFO:tasks.rados.rados.0.trial115.stdout:141: oid 37 version 21 is already newer than 18 2026-02-17T05:46:55.435 INFO:tasks.rados.rados.0.trial115.stdout:141: oid 37 version 21 is already newer than 19 2026-02-17T05:46:55.435 INFO:tasks.rados.rados.0.trial115.stdout:141: oid 37 version 21 is already newer than 20 2026-02-17T05:46:55.435 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 21 (ObjNum 87 snap 8 seq_num 87) dirty exists 2026-02-17T05:46:55.435 INFO:tasks.rados.rados.0.trial115.stdout:141: left oid 37 (ObjNum 87 snap 8 seq_num 87) 2026-02-17T05:46:55.435 INFO:tasks.rados.rados.0.trial115.stdout:143: finishing copy_from racing read to trial11545420-40 2026-02-17T05:46:55.435 INFO:tasks.rados.rados.0.trial115.stdout:143: finishing copy_from to trial11545420-40 2026-02-17T05:46:55.435 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 34 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-17T05:46:56.216 INFO:tasks.rados.rados.0.trial115.stdout:141: done (3 left) 2026-02-17T05:46:56.217 INFO:tasks.rados.rados.0.trial115.stdout:142: done (2 left) 2026-02-17T05:46:56.217 INFO:tasks.rados.rados.0.trial115.stdout:143: done (1 left) 2026-02-17T05:46:56.217 INFO:tasks.rados.rados.0.trial115.stdout:144: done (0 left) 2026-02-17T05:46:56.217 INFO:tasks.rados.rados.0.trial115.stdout:145: copy_from oid 41 from oid 43 current snap is 9 2026-02-17T05:46:56.217 INFO:tasks.rados.rados.0.trial115.stdout:146: delete oid 31 current snap is 9 2026-02-17T05:46:56.219 INFO:tasks.rados.rados.0.trial115.stdout:146: done (1 left) 2026-02-17T05:46:56.219 INFO:tasks.rados.rados.0.trial115.stdout:147: snap_remove snap 1 2026-02-17T05:46:57.219 INFO:tasks.rados.rados.0.trial115.stdout:147: done (1 left) 2026-02-17T05:46:57.219 INFO:tasks.rados.rados.0.trial115.stdout:148: rollback oid 30 current snap is 9 2026-02-17T05:46:57.220 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 8 2026-02-17T05:46:57.220 INFO:tasks.rados.rados.0.trial115.stdout:145: finishing copy_from racing read to trial11545420-41 2026-02-17T05:46:57.220 INFO:tasks.rados.rados.0.trial115.stdout:145: finishing copy_from to trial11545420-41 2026-02-17T05:46:57.220 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 12 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-17T05:46:57.220 INFO:tasks.rados.rados.0.trial115.stdout:145: done (1 left) 2026-02-17T05:46:57.220 INFO:tasks.rados.rados.0.trial115.stdout:149: write oid 27 current snap is 9 2026-02-17T05:46:57.220 INFO:tasks.rados.rados.0.trial115.stdout:149: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2026-02-17T05:46:57.225 INFO:tasks.rados.rados.0.trial115.stdout:149: writing trial11545420-27 from 695823 to 1472867 tid 1 2026-02-17T05:46:57.229 INFO:tasks.rados.rados.0.trial115.stdout:149: writing trial11545420-27 from 1925015 to 2564697 tid 2 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:149: writing trial11545420-27 from 3328058 to 3495823 tid 3 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:150: read oid 16 snap -1 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:150: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:151: read oid 1 snap -1 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:151: expect (ObjNum 0 snap 0 seq_num 0) 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:152: snap_create 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:148: finishing rollback tid 0 to trial11545420-30 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:148: finishing rollback tid 1 to trial11545420-30 2026-02-17T05:46:57.234 INFO:tasks.rados.rados.0.trial115.stdout:148: finishing rollback tid 2 to trial11545420-30 2026-02-17T05:46:57.235 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 31 (ObjNum 60 snap 2 seq_num 60) dirty exists 2026-02-17T05:46:57.235 INFO:tasks.rados.rados.0.trial115.stdout:149: finishing write tid 1 to trial11545420-27 2026-02-17T05:46:57.235 INFO:tasks.rados.rados.0.trial115.stdout:149: finishing write tid 2 to trial11545420-27 2026-02-17T05:46:57.236 INFO:tasks.rados.rados.0.trial115.stdout:149: finishing write tid 3 to trial11545420-27 2026-02-17T05:46:57.236 INFO:tasks.rados.rados.0.trial115.stdout:149: finishing write tid 4 to trial11545420-27 2026-02-17T05:46:57.236 INFO:tasks.rados.rados.0.trial115.stdout:149: finishing write tid 5 to trial11545420-27 2026-02-17T05:46:57.236 INFO:tasks.rados.rados.0.trial115.stdout:149: oid 27 updating version 0 to 37 2026-02-17T05:46:57.236 INFO:tasks.rados.rados.0.trial115.stdout:149: oid 27 updating version 37 to 39 2026-02-17T05:46:57.236 INFO:tasks.rados.rados.0.trial115.stdout:149: oid 27 updating version 39 to 40 2026-02-17T05:46:57.236 INFO:tasks.rados.rados.0.trial115.stdout:149: oid 27 version 40 is already newer than 38 2026-02-17T05:46:57.237 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 40 (ObjNum 89 snap 9 seq_num 89) dirty exists 2026-02-17T05:46:57.237 INFO:tasks.rados.rados.0.trial115.stdout:149: left oid 27 (ObjNum 89 snap 9 seq_num 89) 2026-02-17T05:46:57.237 INFO:tasks.rados.rados.0.trial115.stdout:150: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-17T05:46:57.292 INFO:tasks.rados.rados.0.trial115.stdout:151: expect (ObjNum 0 snap 0 seq_num 0) 2026-02-17T05:46:57.920 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T05:46:57.921 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T05:46:57.921 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T05:46:57.921 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T05:46:57.924 INFO:tasks.ceph.mon.b:Started 2026-02-17T05:46:57.924 INFO:tasks.mon_thrash:Sending STOP to mon c 2026-02-17T05:46:57.924 INFO:tasks.ceph.mon.c:Sent signal 19 2026-02-17T05:46:57.925 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T05:47:03.262 INFO:tasks.rados.rados.0.trial115.stdout:148: done (4 left) 2026-02-17T05:47:03.262 INFO:tasks.rados.rados.0.trial115.stdout:149: done (3 left) 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:150: done (2 left) 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:151: done (1 left) 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:152: done (0 left) 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:153: read oid 34 snap -1 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:153: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:154: read oid 22 snap -1 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:154: expect (ObjNum 21 snap 0 seq_num 21) 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:155: read oid 47 snap -1 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:155: expect deleted 2026-02-17T05:47:03.263 INFO:tasks.rados.rados.0.trial115.stdout:156: snap_create 2026-02-17T05:47:03.269 INFO:tasks.rados.rados.0.trial115.stdout:153: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:47:03.379 INFO:tasks.rados.rados.0.trial115.stdout:154: expect (ObjNum 21 snap 0 seq_num 21) 2026-02-17T05:47:04.264 INFO:tasks.rados.rados.0.trial115.stdout:153: done (3 left) 2026-02-17T05:47:04.265 INFO:tasks.rados.rados.0.trial115.stdout:154: done (2 left) 2026-02-17T05:47:04.265 INFO:tasks.rados.rados.0.trial115.stdout:156: done (1 left) 2026-02-17T05:47:04.265 INFO:tasks.rados.rados.0.trial115.stdout:157: read oid 2 snap -1 2026-02-17T05:47:04.265 INFO:tasks.rados.rados.0.trial115.stdout:157: expect (ObjNum 1 snap 0 seq_num 1) 2026-02-17T05:47:04.265 INFO:tasks.rados.rados.0.trial115.stdout:158: copy_from oid 11 from oid 41 current snap is 11 2026-02-17T05:47:04.265 INFO:tasks.rados.rados.0.trial115.stdout:159: rollback oid 24 current snap is 11 2026-02-17T05:47:04.265 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 9 2026-02-17T05:47:04.265 INFO:tasks.rados.rados.0.trial115.stdout:160: snap_create 2026-02-17T05:47:05.205 INFO:tasks.rados.rados.0.trial115.stdout:155: done (4 left) 2026-02-17T05:47:05.205 INFO:tasks.rados.rados.0.trial115.stdout:160: done (3 left) 2026-02-17T05:47:05.205 INFO:tasks.rados.rados.0.trial115.stdout:161: read oid 39 snap -1 2026-02-17T05:47:05.205 INFO:tasks.rados.rados.0.trial115.stdout:161: expect (ObjNum 68 snap 4 seq_num 68) 2026-02-17T05:47:05.206 INFO:tasks.rados.rados.0.trial115.stdout:162: write oid 30 current snap is 12 2026-02-17T05:47:05.206 INFO:tasks.rados.rados.0.trial115.stdout:162: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2026-02-17T05:47:05.209 INFO:tasks.rados.rados.0.trial115.stdout:162: writing trial11545420-30 from 712630 to 1164923 tid 1 2026-02-17T05:47:05.213 INFO:tasks.rados.rados.0.trial115.stdout:162: writing trial11545420-30 from 1583497 to 2083190 tid 2 2026-02-17T05:47:05.216 INFO:tasks.rados.rados.0.trial115.stdout:162: writing trial11545420-30 from 2555481 to 2980668 tid 3 2026-02-17T05:47:05.216 INFO:tasks.rados.rados.0.trial115.stdout:163: snap_create 2026-02-17T05:47:05.217 INFO:tasks.rados.rados.0.trial115.stdout:162: finishing write tid 1 to trial11545420-30 2026-02-17T05:47:05.217 INFO:tasks.rados.rados.0.trial115.stdout:162: finishing write tid 2 to trial11545420-30 2026-02-17T05:47:05.218 INFO:tasks.rados.rados.0.trial115.stdout:162: finishing write tid 3 to trial11545420-30 2026-02-17T05:47:05.218 INFO:tasks.rados.rados.0.trial115.stdout:162: finishing write tid 4 to trial11545420-30 2026-02-17T05:47:05.218 INFO:tasks.rados.rados.0.trial115.stdout:162: finishing write tid 5 to trial11545420-30 2026-02-17T05:47:05.218 INFO:tasks.rados.rados.0.trial115.stdout:162: oid 30 updating version 0 to 43 2026-02-17T05:47:05.219 INFO:tasks.rados.rados.0.trial115.stdout:162: oid 30 updating version 43 to 44 2026-02-17T05:47:05.219 INFO:tasks.rados.rados.0.trial115.stdout:162: oid 30 updating version 44 to 45 2026-02-17T05:47:05.219 INFO:tasks.rados.rados.0.trial115.stdout:162: oid 30 version 45 is already newer than 42 2026-02-17T05:47:05.219 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 45 (ObjNum 90 snap 12 seq_num 90) dirty exists 2026-02-17T05:47:05.219 INFO:tasks.rados.rados.0.trial115.stdout:162: left oid 30 (ObjNum 90 snap 12 seq_num 90) 2026-02-17T05:47:05.417 INFO:tasks.rados.rados.0.trial115.stdout:157: expect (ObjNum 1 snap 0 seq_num 1) 2026-02-17T05:47:05.463 INFO:tasks.rados.rados.0.trial115.stdout:159: finishing rollback tid 0 to trial11545420-24 2026-02-17T05:47:05.463 INFO:tasks.rados.rados.0.trial115.stdout:159: finishing rollback tid 1 to trial11545420-24 2026-02-17T05:47:05.464 INFO:tasks.rados.rados.0.trial115.stdout:159: finishing rollback tid 2 to trial11545420-24 2026-02-17T05:47:05.464 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 73 (ObjNum 32 snap 0 seq_num 32) dirty exists 2026-02-17T05:47:05.464 INFO:tasks.rados.rados.0.trial115.stdout:161: expect (ObjNum 68 snap 4 seq_num 68) 2026-02-17T05:47:05.565 INFO:tasks.rados.rados.0.trial115.stdout:158: finishing copy_from racing read to trial11545420-11 2026-02-17T05:47:05.566 INFO:tasks.rados.rados.0.trial115.stdout:158: finishing copy_from to trial11545420-11 2026-02-17T05:47:05.566 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 12 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-17T05:47:06.207 INFO:tasks.rados.rados.0.trial115.stdout:157: done (5 left) 2026-02-17T05:47:06.207 INFO:tasks.rados.rados.0.trial115.stdout:158: done (4 left) 2026-02-17T05:47:06.207 INFO:tasks.rados.rados.0.trial115.stdout:159: done (3 left) 2026-02-17T05:47:06.207 INFO:tasks.rados.rados.0.trial115.stdout:161: done (2 left) 2026-02-17T05:47:06.207 INFO:tasks.rados.rados.0.trial115.stdout:162: done (1 left) 2026-02-17T05:47:06.208 INFO:tasks.rados.rados.0.trial115.stdout:163: done (0 left) 2026-02-17T05:47:06.208 INFO:tasks.rados.rados.0.trial115.stdout:164: read oid 10 snap 9 2026-02-17T05:47:06.208 INFO:tasks.rados.rados.0.trial115.stdout:164: expect (ObjNum 9 snap 0 seq_num 9) 2026-02-17T05:47:06.208 INFO:tasks.rados.rados.0.trial115.stdout:165: write oid 2 current snap is 13 2026-02-17T05:47:06.208 INFO:tasks.rados.rados.0.trial115.stdout:165: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2026-02-17T05:47:06.212 INFO:tasks.rados.rados.0.trial115.stdout:165: writing trial11545420-2 from 729437 to 1256979 tid 1 2026-02-17T05:47:06.217 INFO:tasks.rados.rados.0.trial115.stdout:165: writing trial11545420-2 from 1925626 to 2568977 tid 2 2026-02-17T05:47:06.221 INFO:tasks.rados.rados.0.trial115.stdout:165: writing trial11545420-2 from 3266551 to 3529437 tid 3 2026-02-17T05:47:06.221 INFO:tasks.rados.rados.0.trial115.stdout:166: delete oid 14 current snap is 13 2026-02-17T05:47:06.224 INFO:tasks.rados.rados.0.trial115.stdout:166: done (2 left) 2026-02-17T05:47:06.225 INFO:tasks.rados.rados.0.trial115.stdout:167: copy_from oid 39 from oid 40 current snap is 13 2026-02-17T05:47:06.225 INFO:tasks.rados.rados.0.trial115.stdout:168: read oid 11 snap -1 2026-02-17T05:47:06.225 INFO:tasks.rados.rados.0.trial115.stdout:168: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:47:06.225 INFO:tasks.rados.rados.0.trial115.stdout:169: write (excl) oid 35 current snap is 13 2026-02-17T05:47:06.225 INFO:tasks.rados.rados.0.trial115.stdout:169: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2026-02-17T05:47:06.230 INFO:tasks.rados.rados.0.trial115.stdout:169: writing trial11545420-35 from 763051 to 1557444 tid 1 2026-02-17T05:47:06.234 INFO:tasks.rados.rados.0.trial115.stdout:169: writing trial11545420-35 from 2158943 to 2805963 tid 2 2026-02-17T05:47:06.237 INFO:tasks.rados.rados.0.trial115.stdout:169: writing trial11545420-35 from 3437750 to 3563051 tid 3 2026-02-17T05:47:06.237 INFO:tasks.rados.rados.0.trial115.stdout:170: delete oid 29 current snap is 13 2026-02-17T05:47:06.237 INFO:tasks.rados.rados.0.trial115.stdout:169: finishing write tid 1 to trial11545420-35 2026-02-17T05:47:06.237 INFO:tasks.rados.rados.0.trial115.stdout:169: finishing write tid 2 to trial11545420-35 2026-02-17T05:47:06.238 INFO:tasks.rados.rados.0.trial115.stdout:169: finishing write tid 3 to trial11545420-35 2026-02-17T05:47:06.238 INFO:tasks.rados.rados.0.trial115.stdout:169: finishing write tid 4 to trial11545420-35 2026-02-17T05:47:06.239 INFO:tasks.rados.rados.0.trial115.stdout:169: finishing write tid 5 to trial11545420-35 2026-02-17T05:47:06.239 INFO:tasks.rados.rados.0.trial115.stdout:169: oid 35 updating version 0 to 60 2026-02-17T05:47:06.239 INFO:tasks.rados.rados.0.trial115.stdout:169: oid 35 updating version 60 to 61 2026-02-17T05:47:06.239 INFO:tasks.rados.rados.0.trial115.stdout:169: oid 35 updating version 61 to 62 2026-02-17T05:47:06.239 INFO:tasks.rados.rados.0.trial115.stdout:169: oid 35 version 62 is already newer than 59 2026-02-17T05:47:06.239 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 62 (ObjNum 93 snap 13 seq_num 93) dirty exists 2026-02-17T05:47:06.239 INFO:tasks.rados.rados.0.trial115.stdout:169: left oid 35 (ObjNum 93 snap 13 seq_num 93) 2026-02-17T05:47:06.240 INFO:tasks.rados.rados.0.trial115.stdout:169: done (5 left) 2026-02-17T05:47:06.240 INFO:tasks.rados.rados.0.trial115.stdout:170: done (4 left) 2026-02-17T05:47:06.240 INFO:tasks.rados.rados.0.trial115.stdout:171: read oid 50 snap -1 2026-02-17T05:47:06.240 INFO:tasks.rados.rados.0.trial115.stdout:171: expect deleted 2026-02-17T05:47:06.240 INFO:tasks.rados.rados.0.trial115.stdout:172: delete oid 35 current snap is 13 2026-02-17T05:47:06.241 INFO:tasks.rados.rados.0.trial115.stdout:171: done (5 left) 2026-02-17T05:47:06.241 INFO:tasks.rados.rados.0.trial115.stdout:172: done (4 left) 2026-02-17T05:47:06.241 INFO:tasks.rados.rados.0.trial115.stdout:173: copy_from oid 19 from oid 26 current snap is 13 2026-02-17T05:47:06.241 INFO:tasks.rados.rados.0.trial115.stdout:174: write (excl) oid 49 current snap is 13 2026-02-17T05:47:06.241 INFO:tasks.rados.rados.0.trial115.stdout:174: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2026-02-17T05:47:06.244 INFO:tasks.rados.rados.0.trial115.stdout:174: writing trial11545420-49 from 413472 to 1033612 tid 1 2026-02-17T05:47:06.249 INFO:tasks.rados.rados.0.trial115.stdout:174: writing trial11545420-49 from 1818036 to 2328736 tid 2 2026-02-17T05:47:06.254 INFO:tasks.rados.rados.0.trial115.stdout:174: writing trial11545420-49 from 3120019 to 3613472 tid 3 2026-02-17T05:47:06.254 INFO:tasks.rados.rados.0.trial115.stdout:175: snap_create 2026-02-17T05:47:06.254 INFO:tasks.rados.rados.0.trial115.stdout:174: finishing write tid 1 to trial11545420-49 2026-02-17T05:47:06.254 INFO:tasks.rados.rados.0.trial115.stdout:174: finishing write tid 2 to trial11545420-49 2026-02-17T05:47:06.254 INFO:tasks.rados.rados.0.trial115.stdout:173: finishing copy_from racing read to trial11545420-19 2026-02-17T05:47:06.254 INFO:tasks.rados.rados.0.trial115.stdout:173: finishing copy_from to trial11545420-19 2026-02-17T05:47:06.254 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 3 (ObjNum 25 snap 0 seq_num 25) dirty exists 2026-02-17T05:47:06.255 INFO:tasks.rados.rados.0.trial115.stdout:174: finishing write tid 3 to trial11545420-49 2026-02-17T05:47:06.256 INFO:tasks.rados.rados.0.trial115.stdout:174: finishing write tid 4 to trial11545420-49 2026-02-17T05:47:06.256 INFO:tasks.rados.rados.0.trial115.stdout:174: finishing write tid 5 to trial11545420-49 2026-02-17T05:47:06.256 INFO:tasks.rados.rados.0.trial115.stdout:174: oid 49 updating version 0 to 37 2026-02-17T05:47:06.256 INFO:tasks.rados.rados.0.trial115.stdout:174: oid 49 version 37 is already newer than 36 2026-02-17T05:47:06.256 INFO:tasks.rados.rados.0.trial115.stdout:174: oid 49 version 37 is already newer than 35 2026-02-17T05:47:06.256 INFO:tasks.rados.rados.0.trial115.stdout:174: oid 49 updating version 37 to 38 2026-02-17T05:47:06.256 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 38 (ObjNum 96 snap 13 seq_num 96) dirty exists 2026-02-17T05:47:06.256 INFO:tasks.rados.rados.0.trial115.stdout:174: left oid 49 (ObjNum 96 snap 13 seq_num 96) 2026-02-17T05:47:06.799 INFO:tasks.rados.rados.0.trial115.stdout:164: expect (ObjNum 9 snap 0 seq_num 9) 2026-02-17T05:47:06.849 INFO:tasks.rados.rados.0.trial115.stdout:165: finishing write tid 1 to trial11545420-2 2026-02-17T05:47:06.849 INFO:tasks.rados.rados.0.trial115.stdout:165: finishing write tid 2 to trial11545420-2 2026-02-17T05:47:06.849 INFO:tasks.rados.rados.0.trial115.stdout:165: finishing write tid 3 to trial11545420-2 2026-02-17T05:47:06.849 INFO:tasks.rados.rados.0.trial115.stdout:165: finishing write tid 4 to trial11545420-2 2026-02-17T05:47:06.849 INFO:tasks.rados.rados.0.trial115.stdout:165: finishing write tid 5 to trial11545420-2 2026-02-17T05:47:06.849 INFO:tasks.rados.rados.0.trial115.stdout:165: oid 2 updating version 0 to 52 2026-02-17T05:47:06.849 INFO:tasks.rados.rados.0.trial115.stdout:165: oid 2 updating version 52 to 53 2026-02-17T05:47:06.849 INFO:tasks.rados.rados.0.trial115.stdout:165: oid 2 version 53 is already newer than 51 2026-02-17T05:47:06.850 INFO:tasks.rados.rados.0.trial115.stdout:165: oid 2 updating version 53 to 54 2026-02-17T05:47:06.850 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 54 (ObjNum 91 snap 13 seq_num 91) dirty exists 2026-02-17T05:47:06.850 INFO:tasks.rados.rados.0.trial115.stdout:165: left oid 2 (ObjNum 91 snap 13 seq_num 91) 2026-02-17T05:47:06.850 INFO:tasks.rados.rados.0.trial115.stdout:168: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:47:06.912 INFO:tasks.rados.rados.0.trial115.stdout:167: finishing copy_from racing read to trial11545420-39 2026-02-17T05:47:06.912 INFO:tasks.rados.rados.0.trial115.stdout:167: finishing copy_from to trial11545420-39 2026-02-17T05:47:06.912 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 34 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-17T05:47:07.210 INFO:tasks.rados.rados.0.trial115.stdout:164: done (6 left) 2026-02-17T05:47:07.210 INFO:tasks.rados.rados.0.trial115.stdout:165: done (5 left) 2026-02-17T05:47:07.211 INFO:tasks.rados.rados.0.trial115.stdout:167: done (4 left) 2026-02-17T05:47:07.211 INFO:tasks.rados.rados.0.trial115.stdout:168: done (3 left) 2026-02-17T05:47:07.211 INFO:tasks.rados.rados.0.trial115.stdout:173: done (2 left) 2026-02-17T05:47:07.211 INFO:tasks.rados.rados.0.trial115.stdout:174: done (1 left) 2026-02-17T05:47:07.211 INFO:tasks.rados.rados.0.trial115.stdout:175: done (0 left) 2026-02-17T05:47:07.211 INFO:tasks.rados.rados.0.trial115.stdout:176: delete oid 48 current snap is 14 2026-02-17T05:47:07.213 INFO:tasks.rados.rados.0.trial115.stdout:176: done (0 left) 2026-02-17T05:47:07.213 INFO:tasks.rados.rados.0.trial115.stdout:177: read oid 24 snap -1 2026-02-17T05:47:07.213 INFO:tasks.rados.rados.0.trial115.stdout:177: expect (ObjNum 32 snap 0 seq_num 32) 2026-02-17T05:47:07.213 INFO:tasks.rados.rados.0.trial115.stdout:178: delete oid 7 current snap is 14 2026-02-17T05:47:07.901 INFO:tasks.rados.rados.0.trial115.stdout:177: expect (ObjNum 32 snap 0 seq_num 32) 2026-02-17T05:47:07.961 INFO:tasks.rados.rados.0.trial115.stdout:177: done (1 left) 2026-02-17T05:47:07.961 INFO:tasks.rados.rados.0.trial115.stdout:178: done (0 left) 2026-02-17T05:47:07.961 INFO:tasks.rados.rados.0.trial115.stdout:179: write oid 14 current snap is 14 2026-02-17T05:47:07.961 INFO:tasks.rados.rados.0.trial115.stdout:179: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2026-02-17T05:47:07.965 INFO:tasks.rados.rados.0.trial115.stdout:179: writing trial11545420-14 from 463893 to 1026133 tid 1 2026-02-17T05:47:07.969 INFO:tasks.rados.rados.0.trial115.stdout:179: writing trial11545420-14 from 1593482 to 2251509 tid 2 2026-02-17T05:47:07.973 INFO:tasks.rados.rados.0.trial115.stdout:179: writing trial11545420-14 from 2685935 to 3245276 tid 3 2026-02-17T05:47:07.974 INFO:tasks.rados.rados.0.trial115.stdout:180: read oid 18 snap -1 2026-02-17T05:47:07.974 INFO:tasks.rados.rados.0.trial115.stdout:180: expect (ObjNum 17 snap 0 seq_num 17) 2026-02-17T05:47:07.974 INFO:tasks.rados.rados.0.trial115.stdout:181: write (excl) oid 9 current snap is 14 2026-02-17T05:47:07.974 INFO:tasks.rados.rados.0.trial115.stdout:181: seq_num 100 ranges {480700=637489,1651964=401685,2713358=770613} 2026-02-17T05:47:07.977 INFO:tasks.rados.rados.0.trial115.stdout:181: writing trial11545420-9 from 480700 to 1118189 tid 1 2026-02-17T05:47:07.981 INFO:tasks.rados.rados.0.trial115.stdout:181: writing trial11545420-9 from 1651964 to 2053649 tid 2 2026-02-17T05:47:07.987 INFO:tasks.rados.rados.0.trial115.stdout:181: writing trial11545420-9 from 2713358 to 3483971 tid 3 2026-02-17T05:47:07.987 INFO:tasks.rados.rados.0.trial115.stdout:182: snap_create 2026-02-17T05:47:07.987 INFO:tasks.rados.rados.0.trial115.stdout:179: finishing write tid 1 to trial11545420-14 2026-02-17T05:47:07.987 INFO:tasks.rados.rados.0.trial115.stdout:179: finishing write tid 2 to trial11545420-14 2026-02-17T05:47:07.987 INFO:tasks.rados.rados.0.trial115.stdout:180: expect (ObjNum 17 snap 0 seq_num 17) 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:179: finishing write tid 3 to trial11545420-14 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:179: finishing write tid 4 to trial11545420-14 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:179: finishing write tid 5 to trial11545420-14 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:179: oid 14 updating version 0 to 31 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:179: oid 14 version 31 is already newer than 29 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:179: oid 14 version 31 is already newer than 30 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:179: oid 14 version 31 is already newer than 28 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 31 (ObjNum 99 snap 14 seq_num 99) dirty exists 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:179: left oid 14 (ObjNum 99 snap 14 seq_num 99) 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:181: finishing write tid 1 to trial11545420-9 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:181: finishing write tid 2 to trial11545420-9 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:181: finishing write tid 3 to trial11545420-9 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:181: finishing write tid 4 to trial11545420-9 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:181: finishing write tid 5 to trial11545420-9 2026-02-17T05:47:08.039 INFO:tasks.rados.rados.0.trial115.stdout:181: oid 9 updating version 0 to 56 2026-02-17T05:47:08.040 INFO:tasks.rados.rados.0.trial115.stdout:181: oid 9 updating version 56 to 57 2026-02-17T05:47:08.040 INFO:tasks.rados.rados.0.trial115.stdout:181: oid 9 updating version 57 to 59 2026-02-17T05:47:08.040 INFO:tasks.rados.rados.0.trial115.stdout:181: oid 9 version 59 is already newer than 58 2026-02-17T05:47:08.040 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 59 (ObjNum 100 snap 14 seq_num 100) dirty exists 2026-02-17T05:47:08.040 INFO:tasks.rados.rados.0.trial115.stdout:181: left oid 9 (ObjNum 100 snap 14 seq_num 100) 2026-02-17T05:47:08.212 INFO:tasks.rados.rados.0.trial115.stdout:179: done (3 left) 2026-02-17T05:47:08.212 INFO:tasks.rados.rados.0.trial115.stdout:180: done (2 left) 2026-02-17T05:47:08.212 INFO:tasks.rados.rados.0.trial115.stdout:181: done (1 left) 2026-02-17T05:47:08.212 INFO:tasks.rados.rados.0.trial115.stdout:182: done (0 left) 2026-02-17T05:47:08.212 INFO:tasks.rados.rados.0.trial115.stdout:183: write (excl) oid 25 current snap is 15 2026-02-17T05:47:08.212 INFO:tasks.rados.rados.0.trial115.stdout:183: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2026-02-17T05:47:08.216 INFO:tasks.rados.rados.0.trial115.stdout:183: writing trial11545420-25 from 497507 to 1210245 tid 1 2026-02-17T05:47:08.220 INFO:tasks.rados.rados.0.trial115.stdout:183: writing trial11545420-25 from 1710446 to 2372142 tid 2 2026-02-17T05:47:08.225 INFO:tasks.rados.rados.0.trial115.stdout:183: writing trial11545420-25 from 3140781 to 3697507 tid 3 2026-02-17T05:47:08.225 INFO:tasks.rados.rados.0.trial115.stdout:184: write oid 18 current snap is 15 2026-02-17T05:47:08.225 INFO:tasks.rados.rados.0.trial115.stdout:183: finishing write tid 1 to trial11545420-25 2026-02-17T05:47:08.225 INFO:tasks.rados.rados.0.trial115.stdout:183: finishing write tid 2 to trial11545420-25 2026-02-17T05:47:08.225 INFO:tasks.rados.rados.0.trial115.stdout:184: seq_num 102 ranges {514314=787987,1768928=405354,2768204=509510} 2026-02-17T05:47:08.230 INFO:tasks.rados.rados.0.trial115.stdout:184: writing trial11545420-18 from 514314 to 1302301 tid 1 2026-02-17T05:47:08.233 INFO:tasks.rados.rados.0.trial115.stdout:184: writing trial11545420-18 from 1768928 to 2174282 tid 2 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:184: writing trial11545420-18 from 2768204 to 3277714 tid 3 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:185: copy_from oid 2 from oid 36 current snap is 15 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: finishing write tid 3 to trial11545420-25 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: finishing write tid 4 to trial11545420-25 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: finishing write tid 5 to trial11545420-25 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: oid 25 updating version 0 to 63 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: oid 25 version 63 is already newer than 62 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: oid 25 updating version 63 to 64 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: oid 25 updating version 64 to 65 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 65 (ObjNum 101 snap 15 seq_num 101) dirty exists 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: left oid 25 (ObjNum 101 snap 15 seq_num 101) 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:184: finishing write tid 1 to trial11545420-18 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:184: finishing write tid 2 to trial11545420-18 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:183: done (2 left) 2026-02-17T05:47:08.238 INFO:tasks.rados.rados.0.trial115.stdout:186: read oid 28 snap -1 2026-02-17T05:47:08.239 INFO:tasks.rados.rados.0.trial115.stdout:186: expect (ObjNum 82 snap 8 seq_num 82) 2026-02-17T05:47:08.239 INFO:tasks.rados.rados.0.trial115.stdout:187: read oid 1 snap -1 2026-02-17T05:47:08.239 INFO:tasks.rados.rados.0.trial115.stdout:187: expect (ObjNum 0 snap 0 seq_num 0) 2026-02-17T05:47:08.239 INFO:tasks.rados.rados.0.trial115.stdout:188: snap_remove snap 10 2026-02-17T05:47:09.215 INFO:tasks.rados.rados.0.trial115.stdout:188: done (4 left) 2026-02-17T05:47:09.215 INFO:tasks.rados.rados.0.trial115.stdout:189: write (excl) oid 48 current snap is 15 2026-02-17T05:47:09.215 INFO:tasks.rados.rados.0.trial115.stdout:189: seq_num 103 ranges {531121=463236,1711057=665365,3079274=651847} 2026-02-17T05:47:09.219 INFO:tasks.rados.rados.0.trial115.stdout:189: writing trial11545420-48 from 531121 to 994357 tid 1 2026-02-17T05:47:09.224 INFO:tasks.rados.rados.0.trial115.stdout:189: writing trial11545420-48 from 1711057 to 2376422 tid 2 2026-02-17T05:47:09.229 INFO:tasks.rados.rados.0.trial115.stdout:189: writing trial11545420-48 from 3079274 to 3731121 tid 3 2026-02-17T05:47:09.229 INFO:tasks.rados.rados.0.trial115.stdout:190: write (excl) oid 44 current snap is 15 2026-02-17T05:47:09.229 INFO:tasks.rados.rados.0.trial115.stdout:190: seq_num 104 ranges {547928=538485,1769539=409023,2706697=532054} 2026-02-17T05:47:09.233 INFO:tasks.rados.rados.0.trial115.stdout:190: writing trial11545420-44 from 547928 to 1086413 tid 1 2026-02-17T05:47:09.237 INFO:tasks.rados.rados.0.trial115.stdout:190: writing trial11545420-44 from 1769539 to 2178562 tid 2 2026-02-17T05:47:09.241 INFO:tasks.rados.rados.0.trial115.stdout:190: writing trial11545420-44 from 2706697 to 3238751 tid 3 2026-02-17T05:47:09.241 INFO:tasks.rados.rados.0.trial115.stdout:191: rollback oid 45 current snap is 15 2026-02-17T05:47:09.242 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 14 2026-02-17T05:47:09.242 INFO:tasks.rados.rados.0.trial115.stdout:184: finishing write tid 3 to trial11545420-18 2026-02-17T05:47:09.242 INFO:tasks.rados.rados.0.trial115.stdout:184: finishing write tid 4 to trial11545420-18 2026-02-17T05:47:09.242 INFO:tasks.rados.rados.0.trial115.stdout:187: expect (ObjNum 0 snap 0 seq_num 0) 2026-02-17T05:47:09.288 INFO:tasks.rados.rados.0.trial115.stdout:184: finishing write tid 5 to trial11545420-18 2026-02-17T05:47:09.288 INFO:tasks.rados.rados.0.trial115.stdout:184: oid 18 updating version 0 to 67 2026-02-17T05:47:09.288 INFO:tasks.rados.rados.0.trial115.stdout:184: oid 18 updating version 67 to 70 2026-02-17T05:47:09.288 INFO:tasks.rados.rados.0.trial115.stdout:184: oid 18 version 70 is already newer than 69 2026-02-17T05:47:09.289 INFO:tasks.rados.rados.0.trial115.stdout:184: oid 18 version 70 is already newer than 68 2026-02-17T05:47:09.289 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 70 (ObjNum 102 snap 15 seq_num 102) dirty exists 2026-02-17T05:47:09.289 INFO:tasks.rados.rados.0.trial115.stdout:184: left oid 18 (ObjNum 102 snap 15 seq_num 102) 2026-02-17T05:47:09.289 INFO:tasks.rados.rados.0.trial115.stdout:186: expect (ObjNum 82 snap 8 seq_num 82) 2026-02-17T05:47:09.398 INFO:tasks.rados.rados.0.trial115.stdout:185: finishing copy_from racing read to trial11545420-2 2026-02-17T05:47:09.398 INFO:tasks.rados.rados.0.trial115.stdout:185: finishing copy_from to trial11545420-2 2026-02-17T05:47:09.398 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 12 (ObjNum 35 snap 0 seq_num 35) dirty exists 2026-02-17T05:47:09.398 INFO:tasks.rados.rados.0.trial115.stdout:189: finishing write tid 1 to trial11545420-48 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:189: finishing write tid 2 to trial11545420-48 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:184: done (6 left) 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:185: done (5 left) 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:186: done (4 left) 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:187: done (3 left) 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:192: read oid 26 snap 13 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:192: expect (ObjNum 25 snap 0 seq_num 25) 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:193: write (excl) oid 28 current snap is 15 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:189: finishing write tid 3 to trial11545420-48 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:189: finishing write tid 4 to trial11545420-48 2026-02-17T05:47:09.399 INFO:tasks.rados.rados.0.trial115.stdout:189: finishing write tid 5 to trial11545420-48 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:189: oid 48 updating version 0 to 74 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:189: oid 48 updating version 74 to 77 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:189: oid 48 version 77 is already newer than 75 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:189: oid 48 version 77 is already newer than 76 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 77 (ObjNum 103 snap 15 seq_num 103) dirty exists 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:189: left oid 48 (ObjNum 103 snap 15 seq_num 103) 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:190: finishing write tid 1 to trial11545420-44 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:190: finishing write tid 2 to trial11545420-44 2026-02-17T05:47:09.400 INFO:tasks.rados.rados.0.trial115.stdout:193: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2026-02-17T05:47:09.402 INFO:tasks.rados.rados.0.trial115.stdout:193: writing trial11545420-28 from 564735 to 1178469 tid 1 2026-02-17T05:47:09.406 INFO:tasks.rados.rados.0.trial115.stdout:193: writing trial11545420-28 from 1828021 to 2497055 tid 2 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:193: writing trial11545420-28 from 3134120 to 3764735 tid 3 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:189: done (4 left) 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:194: snap_create 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:190: finishing write tid 3 to trial11545420-44 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:190: finishing write tid 4 to trial11545420-44 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:190: finishing write tid 5 to trial11545420-44 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:190: oid 44 updating version 0 to 28 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:190: oid 44 version 28 is already newer than 26 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:190: oid 44 version 28 is already newer than 25 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:190: oid 44 version 28 is already newer than 27 2026-02-17T05:47:09.412 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 28 (ObjNum 104 snap 15 seq_num 104) dirty exists 2026-02-17T05:47:09.413 INFO:tasks.rados.rados.0.trial115.stdout:190: left oid 44 (ObjNum 104 snap 15 seq_num 104) 2026-02-17T05:47:09.413 INFO:tasks.rados.rados.0.trial115.stdout:192: expect (ObjNum 25 snap 0 seq_num 25) 2026-02-17T05:47:09.516 INFO:tasks.rados.rados.0.trial115.stdout:191: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:47:09.516 INFO:tasks.rados.rados.0.trial115.stdout:191: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:47:09.516 INFO:tasks.rados.rados.0.trial115.stdout:191: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:47:09.516 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 79 (ObjNum 58 snap 2 seq_num 58) dirty exists 2026-02-17T05:47:09.518 INFO:tasks.rados.rados.0.trial115.stdout:193: finishing write tid 1 to trial11545420-28 2026-02-17T05:47:09.518 INFO:tasks.rados.rados.0.trial115.stdout:193: finishing write tid 2 to trial11545420-28 2026-02-17T05:47:09.518 INFO:tasks.rados.rados.0.trial115.stdout:193: finishing write tid 3 to trial11545420-28 2026-02-17T05:47:09.518 INFO:tasks.rados.rados.0.trial115.stdout:193: finishing write tid 4 to trial11545420-28 2026-02-17T05:47:09.519 INFO:tasks.rados.rados.0.trial115.stdout:193: finishing write tid 5 to trial11545420-28 2026-02-17T05:47:09.519 INFO:tasks.rados.rados.0.trial115.stdout:193: oid 28 updating version 0 to 63 2026-02-17T05:47:09.519 INFO:tasks.rados.rados.0.trial115.stdout:193: oid 28 version 63 is already newer than 61 2026-02-17T05:47:09.519 INFO:tasks.rados.rados.0.trial115.stdout:193: oid 28 version 63 is already newer than 62 2026-02-17T05:47:09.519 INFO:tasks.rados.rados.0.trial115.stdout:193: oid 28 updating version 63 to 64 2026-02-17T05:47:09.519 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 64 (ObjNum 105 snap 15 seq_num 105) dirty exists 2026-02-17T05:47:09.519 INFO:tasks.rados.rados.0.trial115.stdout:193: left oid 28 (ObjNum 105 snap 15 seq_num 105) 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:190: done (4 left) 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:191: done (3 left) 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:192: done (2 left) 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:193: done (1 left) 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:194: done (0 left) 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:195: read oid 12 snap -1 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:195: expect deleted 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:196: write (excl) oid 19 current snap is 16 2026-02-17T05:47:10.227 INFO:tasks.rados.rados.0.trial115.stdout:196: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2026-02-17T05:47:10.231 INFO:tasks.rados.rados.0.trial115.stdout:196: writing trial11545420-19 from 581542 to 1270525 tid 1 2026-02-17T05:47:10.235 INFO:tasks.rados.rados.0.trial115.stdout:196: writing trial11545420-19 from 1886503 to 2299195 tid 2 2026-02-17T05:47:10.241 INFO:tasks.rados.rados.0.trial115.stdout:196: writing trial11545420-19 from 2761543 to 3432494 tid 3 2026-02-17T05:47:10.241 INFO:tasks.rados.rados.0.trial115.stdout:197: read oid 11 snap 15 2026-02-17T05:47:10.241 INFO:tasks.rados.rados.0.trial115.stdout:197: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:47:10.241 INFO:tasks.rados.rados.0.trial115.stdout:198: copy_from oid 37 from oid 34 current snap is 16 2026-02-17T05:47:10.241 INFO:tasks.rados.rados.0.trial115.stdout:196: finishing write tid 1 to trial11545420-19 2026-02-17T05:47:10.241 INFO:tasks.rados.rados.0.trial115.stdout:196: finishing write tid 2 to trial11545420-19 2026-02-17T05:47:10.241 INFO:tasks.rados.rados.0.trial115.stdout:199: write oid 22 current snap is 16 2026-02-17T05:47:10.241 INFO:tasks.rados.rados.0.trial115.stdout:199: seq_num 107 ranges {598349=480585,1544985=672703,2905319=482223} 2026-02-17T05:47:10.244 INFO:tasks.rados.rados.0.trial115.stdout:199: writing trial11545420-22 from 598349 to 1078934 tid 1 2026-02-17T05:47:10.248 INFO:tasks.rados.rados.0.trial115.stdout:199: writing trial11545420-22 from 1544985 to 2217688 tid 2 2026-02-17T05:47:10.253 INFO:tasks.rados.rados.0.trial115.stdout:199: writing trial11545420-22 from 2905319 to 3387542 tid 3 2026-02-17T05:47:10.253 INFO:tasks.rados.rados.0.trial115.stdout:200: read oid 39 snap -1 2026-02-17T05:47:10.253 INFO:tasks.rados.rados.0.trial115.stdout:200: expect (ObjNum 24 snap 0 seq_num 24) 2026-02-17T05:47:10.254 INFO:tasks.rados.rados.0.trial115.stdout:201: write (excl) oid 43 current snap is 16 2026-02-17T05:47:10.254 INFO:tasks.rados.rados.0.trial115.stdout:201: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2026-02-17T05:47:10.257 INFO:tasks.rados.rados.0.trial115.stdout:201: writing trial11545420-43 from 615156 to 1170990 tid 1 2026-02-17T05:47:10.260 INFO:tasks.rados.rados.0.trial115.stdout:201: writing trial11545420-43 from 1603467 to 2019828 tid 2 2026-02-17T05:47:10.266 INFO:tasks.rados.rados.0.trial115.stdout:201: writing trial11545420-43 from 2816389 to 3509884 tid 3 2026-02-17T05:47:10.266 INFO:tasks.rados.rados.0.trial115.stdout:202: delete oid 35 current snap is 16 2026-02-17T05:47:10.267 INFO:tasks.rados.rados.0.trial115.stdout:196: finishing write tid 3 to trial11545420-19 2026-02-17T05:47:10.267 INFO:tasks.rados.rados.0.trial115.stdout:196: finishing write tid 4 to trial11545420-19 2026-02-17T05:47:10.267 INFO:tasks.rados.rados.0.trial115.stdout:196: finishing write tid 5 to trial11545420-19 2026-02-17T05:47:10.267 INFO:tasks.rados.rados.0.trial115.stdout:196: oid 19 updating version 0 to 79 2026-02-17T05:47:10.267 INFO:tasks.rados.rados.0.trial115.stdout:196: oid 19 updating version 79 to 81 2026-02-17T05:47:10.267 INFO:tasks.rados.rados.0.trial115.stdout:196: oid 19 updating version 81 to 82 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:196: oid 19 version 82 is already newer than 80 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 82 (ObjNum 106 snap 16 seq_num 106) dirty exists 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:196: left oid 19 (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: finishing write tid 1 to trial11545420-22 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: finishing write tid 2 to trial11545420-22 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: finishing write tid 3 to trial11545420-22 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: finishing write tid 4 to trial11545420-22 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: finishing write tid 5 to trial11545420-22 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: oid 22 updating version 0 to 52 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: oid 22 version 52 is already newer than 49 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: oid 22 version 52 is already newer than 50 2026-02-17T05:47:10.268 INFO:tasks.rados.rados.0.trial115.stdout:199: oid 22 version 52 is already newer than 51 2026-02-17T05:47:10.269 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 52 (ObjNum 107 snap 16 seq_num 107) dirty exists 2026-02-17T05:47:10.269 INFO:tasks.rados.rados.0.trial115.stdout:199: left oid 22 (ObjNum 107 snap 16 seq_num 107) 2026-02-17T05:47:10.269 INFO:tasks.rados.rados.0.trial115.stdout:196: done (7 left) 2026-02-17T05:47:10.269 INFO:tasks.rados.rados.0.trial115.stdout:199: done (6 left) 2026-02-17T05:47:10.270 INFO:tasks.rados.rados.0.trial115.stdout:202: done (5 left) 2026-02-17T05:47:10.270 INFO:tasks.rados.rados.0.trial115.stdout:203: snap_create 2026-02-17T05:47:10.469 INFO:tasks.rados.rados.0.trial115.stdout:201: finishing write tid 1 to trial11545420-43 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: finishing write tid 2 to trial11545420-43 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: finishing write tid 3 to trial11545420-43 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: finishing write tid 4 to trial11545420-43 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: finishing write tid 5 to trial11545420-43 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: oid 43 updating version 0 to 32 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: oid 43 updating version 32 to 34 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: oid 43 version 34 is already newer than 33 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: oid 43 version 34 is already newer than 31 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 34 (ObjNum 108 snap 16 seq_num 108) dirty exists 2026-02-17T05:47:10.470 INFO:tasks.rados.rados.0.trial115.stdout:201: left oid 43 (ObjNum 108 snap 16 seq_num 108) 2026-02-17T05:47:10.474 INFO:tasks.rados.rados.0.trial115.stdout:198: finishing copy_from racing read to trial11545420-37 2026-02-17T05:47:10.475 INFO:tasks.rados.rados.0.trial115.stdout:198: finishing copy_from to trial11545420-37 2026-02-17T05:47:10.476 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 24 (ObjNum 78 snap 8 seq_num 78) dirty exists 2026-02-17T05:47:11.230 INFO:tasks.rados.rados.0.trial115.stdout:198: done (5 left) 2026-02-17T05:47:11.230 INFO:tasks.rados.rados.0.trial115.stdout:201: done (4 left) 2026-02-17T05:47:11.230 INFO:tasks.rados.rados.0.trial115.stdout:203: done (3 left) 2026-02-17T05:47:11.230 INFO:tasks.rados.rados.0.trial115.stdout:204: snap_remove snap 13 2026-02-17T05:47:12.585 INFO:tasks.rados.rados.0.trial115.stdout:204: done (3 left) 2026-02-17T05:47:12.585 INFO:tasks.rados.rados.0.trial115.stdout:205: rollback oid 22 current snap is 17 2026-02-17T05:47:12.586 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 22 to 12 2026-02-17T05:47:12.586 INFO:tasks.rados.rados.0.trial115.stdout:197: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:47:12.649 INFO:tasks.rados.rados.0.trial115.stdout:200: expect (ObjNum 24 snap 0 seq_num 24) 2026-02-17T05:47:12.703 INFO:tasks.rados.rados.0.trial115.stdout:205: finishing rollback tid 0 to trial11545420-22 2026-02-17T05:47:12.703 INFO:tasks.rados.rados.0.trial115.stdout:205: finishing rollback tid 1 to trial11545420-22 2026-02-17T05:47:12.703 INFO:tasks.rados.rados.0.trial115.stdout:195: done (3 left) 2026-02-17T05:47:12.704 INFO:tasks.rados.rados.0.trial115.stdout:197: done (2 left) 2026-02-17T05:47:12.704 INFO:tasks.rados.rados.0.trial115.stdout:200: done (1 left) 2026-02-17T05:47:12.704 INFO:tasks.rados.rados.0.trial115.stdout:206: snap_create 2026-02-17T05:47:12.704 INFO:tasks.rados.rados.0.trial115.stdout:205: finishing rollback tid 2 to trial11545420-22 2026-02-17T05:47:12.704 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 56 (ObjNum 21 snap 0 seq_num 21) dirty exists 2026-02-17T05:47:12.925 INFO:tasks.mon_thrash:Sending CONT to mon c 2026-02-17T05:47:12.926 INFO:tasks.ceph.mon.c:Sent signal 18 2026-02-17T05:47:12.926 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:47:12.926 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.994+0000 7f89cd711640 10 monclient: get_monmap_and_config 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.994+0000 7f89cd711640 10 monclient: build_initial_monmap 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.994+0000 7f89cd711640 10 monclient: monmap: 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:12.995625+0000 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:12.995625+0000 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:12.995 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:12.996 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:12.996 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:12.996 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:12.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.994+0000 7f89cd711640 1 Processor -- start 2026-02-17T05:47:12.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.994+0000 7f89cd711640 1 -- start start 2026-02-17T05:47:12.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.994+0000 7f89cd711640 10 monclient: init 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f89c806c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient(hunting): picked mon.noname-a con 0x7f89c809d430 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient(hunting): picked mon.noname-c con 0x7f89c809bf40 addr v2:10.20.193.115:3300/0 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c8073600 con 0x7f89c806c890 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f89c809ca50 con 0x7f89c809d430 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f89c806dab0 con 0x7f89c809bf40 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89cd711640 10 monclient(hunting): authenticate will time out at 678.549177s 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48166/0 (socket says 10.20.193.78:48166) 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 1 -- 10.20.193.78:0/789718929 learned_addr learned my addr 10.20.193.78:0/789718929 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 10 monclient(hunting): get_auth_request con 0x7f89c809d430 auth_method 0 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 msgr2=0x7f89c813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 msgr2=0x7f89c813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 10 monclient(hunting): get_auth_request con 0x7f89c809bf40 auth_method 0 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:12.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:12.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 msgr2=0x7f89c813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:47:12.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 msgr2=0x7f89c813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:12.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:12.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.995+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:12.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:12.997+0000 7f89c6ffd640 1 -- 10.20.193.78:0/789718929 >> v1:10.20.193.78:6789/0 conn(0x7f89c806c890 legacy=0x7f89c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 10 monclient(hunting): get_auth_request con 0x7f89c809d430 auth_method 0 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 msgr2=0x7f89c813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 msgr2=0x7f89c813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 10 monclient(hunting): get_auth_request con 0x7f89c809bf40 auth_method 0 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 msgr2=0x7f89c813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 msgr2=0x7f89c813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:13.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.196+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:47:13.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 10 monclient(hunting): get_auth_request con 0x7f89c809d430 auth_method 0 2026-02-17T05:47:13.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:13.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 msgr2=0x7f89c813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:47:13.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 msgr2=0x7f89c813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:13.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:13.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 10 monclient(hunting): get_auth_request con 0x7f89c809bf40 auth_method 0 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 msgr2=0x7f89c813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 msgr2=0x7f89c813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:13.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.596+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:47:13.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.753+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2725200593 0 0) 0x7f89c8073600 con 0x7f89c806c890 2026-02-17T05:47:13.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.753+0000 7f89c5ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:47:13.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.753+0000 7f89c5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.753+0000 7f89c5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.753+0000 7f89c5ffb640 10 monclient(hunting): my global_id is 4395 2026-02-17T05:47:13.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.753+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89b4003620 con 0x7f89c806c890 2026-02-17T05:47:13.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 251291708 0 0) 0x7f89b4003620 con 0x7f89c806c890 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 msgr2=0x7f89c813fb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.115:3300/0 conn(0x7f89c809bf40 0x7f89c813fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 msgr2=0x7f89c813d5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 --2- 10.20.193.78:0/789718929 >> v2:10.20.193.78:3300/0 conn(0x7f89c809d430 0x7f89c813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: _finish_hunting 0 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c809c770 con 0x7f89c806c890 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f89bc003200 con 0x7f89c806c890 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89cd711640 5 monclient: authenticate success, global_id 4395 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89cd711640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: dump: 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 >> v1:10.20.193.78:6789/0 conn(0x7f89c806c890 legacy=0x7f89c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89b4003c20 0x7f89b4004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient(hunting): picked mon.c con 0x7f89b4003c20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89b4004900 0x7f89b4004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient(hunting): picked mon.b con 0x7f89b4004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89b400bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient(hunting): picked mon.a con 0x7f89b40054a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89c809ca50 con 0x7f89b40054a0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89c806dab0 con 0x7f89b4004900 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f89b400c460 con 0x7f89b4003c20 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c5ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89cd711640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89b400bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient(hunting): get_auth_request con 0x7f89b40054a0 auth_method 0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89b4004900 0x7f89b4004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c6ffd640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89b4003c20 0x7f89b4004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c6ffd640 10 monclient(hunting): get_auth_request con 0x7f89b4003c20 auth_method 0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient(hunting): handle_auth_done global_id 4395 payload 293 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89b4003c20 msgr2=0x7f89b4004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89b4003c20 0x7f89b4004040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89b4004900 msgr2=0x7f89b4004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89b4004900 0x7f89b4004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 1 monclient: found mon.a 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b4003620 con 0x7f89b40054a0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient: _finish_auth 0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c67fc640 10 monclient: get_auth_request con 0x7f89b4004900 auth_method 0 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.754+0000 7f89c67fc640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89b4004900 0x7f89b4004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:47:13.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.755+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 reap_dead start 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c77fe640 1 -- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 msgr2=0x7f89b400bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c77fe640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89b400bb00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009fd0 tx=0x7f89b802fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de2557c352d34dc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8039070 con 0x7f89b40054a0 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 10 monclient: dump: 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b8035850 con 0x7f89b40054a0 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 1 -- 10.20.193.78:0/789718929 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8009bd0 con 0x7f89b40054a0 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89c5ffb640 10 monclient: dump: 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.760 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 4 set_mon_vals no callback set 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 10 monclient: shutdown 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 1 -- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 msgr2=0x7f89b400bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89b400bb00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009fd0 tx=0x7f89b802fb80 comp rx=0 tx=0).stop 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 1 -- 10.20.193.78:0/789718929 shutdown_connections 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 1 --2- 10.20.193.78:0/789718929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89b400bb00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 1 -- 10.20.193.78:0/789718929 >> 10.20.193.78:0/789718929 conn(0x7f89c80a6620 msgr2=0x7f89c80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 1 -- 10.20.193.78:0/789718929 shutdown_connections 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.757+0000 7f89cd711640 1 -- 10.20.193.78:0/789718929 wait complete. 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient: build_initial_monmap 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient: monmap: 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 1 Processor -- start 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 1 -- start start 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient: init 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89c81cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient(hunting): picked mon.noname-a con 0x7f89b40054a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89c81cf830 0x7f89c81cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient(hunting): picked mon.noname-c con 0x7f89c81cf830 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89c81222e0 0x7f89c8122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient(hunting): picked mon.noname-b con 0x7f89c81222e0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89c806cc70 con 0x7f89b40054a0 2026-02-17T05:47:13.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8073410 con 0x7f89c81222e0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f89c806cdf0 con 0x7f89c81cf830 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89cd711640 10 monclient(hunting): authenticate will time out at 679.312512s 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89c81cf830 0x7f89c81cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89c81cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89c81cf830 0x7f89c81cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39834/0 (socket says 10.20.193.78:39834) 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 -- 10.20.193.78:0/2651086495 learned_addr learned my addr 10.20.193.78:0/2651086495 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient(hunting): get_auth_request con 0x7f89c81cf830 auth_method 0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 10 monclient(hunting): get_auth_request con 0x7f89b40054a0 auth_method 0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c77fe640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89c81222e0 0x7f89c8122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c77fe640 10 monclient(hunting): get_auth_request con 0x7f89c81222e0 auth_method 0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient(hunting): handle_auth_done global_id 4379 payload 824 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 -- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89c81222e0 msgr2=0x7f89c8122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89c81222e0 0x7f89c8122790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 -- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 msgr2=0x7f89c81cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89c81cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient: _finish_hunting 0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 monclient: found mon.noname-c 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c67fc640 1 -- 10.20.193.78:0/2651086495 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c8125f40 con 0x7f89c81cf830 2026-02-17T05:47:13.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.758+0000 7f89c6ffd640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89c81cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.760+0000 7f89c67fc640 1 -- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89c81cf830 msgr2=0x7f89c81cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.760+0000 7f89c67fc640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89c81cf830 0x7f89c81cfc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89b0012690 tx=0x7f89b0012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.760+0000 7f89a77fe640 1 -- 10.20.193.78:0/2651086495 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b001a070 con 0x7f89c81cf830 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.760+0000 7f89a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.760+0000 7f89a77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.760+0000 7f89a77fe640 10 monclient: dump: 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 10 monclient: _finish_auth 0 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:13.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 5 monclient: authenticate success, global_id 4379 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 1 -- 10.20.193.78:0/2651086495 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b00099f0 con 0x7f89c81cf830 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 1 -- 10.20.193.78:0/2651086495 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0009da0 con 0x7f89c81cf830 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89c57fa640 4 set_mon_vals no callback set 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 10 monclient: dump: 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 10 monclient: _renew_subs 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c80a29d0 con 0x7f89c81cf830 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 10 monclient: _renew_subs 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c8125c60 con 0x7f89c81cf830 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 1 -- 10.20.193.78:0/2651086495 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f89b00153f0 con 0x7f89c81cf830 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c806d120 con 0x7f89c81cf830 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f899403deb0 0x7f8994040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.761+0000 7f89a77fe640 1 -- 10.20.193.78:0/2651086495 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 4779+0+0 (secure 0 0 0) 0x7f89b0053050 con 0x7f89c81cf830 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.762+0000 7f89c6ffd640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f899403deb0 0x7f8994040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.762+0000 7f89c6ffd640 10 monclient: get_auth_request con 0x7f899403deb0 auth_method 0 2026-02-17T05:47:13.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.764+0000 7f89a77fe640 1 -- 10.20.193.78:0/2651086495 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f89b0051a50 con 0x7f89c81cf830 2026-02-17T05:47:13.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.764+0000 7f89a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:13.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.764+0000 7f89a77fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:13.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.764+0000 7f89c6ffd640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f899403deb0 0x7f8994040370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89bc002d00 tx=0x7f89bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:13.806 INFO:tasks.rados.rados.0.trial115.stdout:205: done (1 left) 2026-02-17T05:47:13.806 INFO:tasks.rados.rados.0.trial115.stdout:206: done (0 left) 2026-02-17T05:47:13.806 INFO:tasks.rados.rados.0.trial115.stdout:207: rollback oid 46 current snap is 18 2026-02-17T05:47:13.806 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 46 to 9 2026-02-17T05:47:13.806 INFO:tasks.rados.rados.0.trial115.stdout:208: snap_remove snap 17 2026-02-17T05:47:13.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.877+0000 7f89cd711640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:47:13.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.877+0000 7f89cd711640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:47:13.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.877+0000 7f89cd711640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:13.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.877+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f89c80744d0 con 0x7f89c81cf830 2026-02-17T05:47:13.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.877+0000 7f89a77fe640 1 -- 10.20.193.78:0/2651086495 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f89b001dc60 con 0x7f89c81cf830 2026-02-17T05:47:13.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.877+0000 7f89a77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:47:13.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.877+0000 7f89a77fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:13.879 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":12,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:47:13.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f899403deb0 msgr2=0x7f8994040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f899403deb0 0x7f8994040370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89bc002d00 tx=0x7f89bc009040 comp rx=0 tx=0).stop 2026-02-17T05:47:13.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 10 monclient: shutdown 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89c81cf830 msgr2=0x7f89c81cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89c81cf830 0x7f89c81cfc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89b0012690 tx=0x7f89b0012b60 comp rx=0 tx=0).stop 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 shutdown_connections 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f899403deb0 0x7f8994040370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f89c81222e0 0x7f89c8122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89c81cf830 0x7f89c81cfc90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 --2- 10.20.193.78:0/2651086495 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89b40054a0 0x7f89c81cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 >> 10.20.193.78:0/2651086495 conn(0x7f89c80a6620 msgr2=0x7f89c8137580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:13.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 shutdown_connections 2026-02-17T05:47:13.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.878+0000 7f89cd711640 1 -- 10.20.193.78:0/2651086495 wait complete. 2026-02-17T05:47:13.888 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:47:13.888 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:47:13.889 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:47:13.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.955+0000 7fc49c6d3640 10 monclient: get_monmap_and_config 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.955+0000 7fc49c6d3640 10 monclient: build_initial_monmap 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.955+0000 7fc49c6d3640 10 monclient: monmap: 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:13.956832+0000 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:13.956832+0000 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 1 Processor -- start 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 1 -- start start 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient: init 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc49406c890 0x7fc4940a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient(hunting): picked mon.noname-c con 0x7fc49406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc49406cc70 0x7fc49406d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient(hunting): picked mon.noname-a con 0x7fc49406cc70 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc494134df0 0x7fc494135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient(hunting): picked mon.noname-b con 0x7fc494134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc494074150 con 0x7fc49406cc70 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc49409ef70 con 0x7fc49406c890 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc49409f250 con 0x7fc494134df0 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49c6d3640 10 monclient(hunting): authenticate will time out at 679.510560s 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc49406cc70 0x7fc49406d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc494134df0 0x7fc494135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc49406cc70 0x7fc49406d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48204/0 (socket says 10.20.193.78:48204) 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc494134df0 0x7fc494135220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39838/0 (socket says 10.20.193.78:39838) 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 -- 10.20.193.78:0/932593813 learned_addr learned my addr 10.20.193.78:0/932593813 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient(hunting): get_auth_request con 0x7fc49406cc70 auth_method 0 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 10 monclient(hunting): get_auth_request con 0x7fc494134df0 auth_method 0 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49a448640 1 --2- 10.20.193.78:0/932593813 >> v2:10.20.193.115:3300/0 conn(0x7fc49406c890 0x7fc4940a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49a448640 10 monclient(hunting): get_auth_request con 0x7fc49406c890 auth_method 0 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49a448640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49a448640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49a448640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49ac49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient(hunting): handle_auth_done global_id 4401 payload 274 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 -- 10.20.193.78:0/932593813 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc494134df0 msgr2=0x7fc494135220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 --2- 10.20.193.78:0/932593813 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc494134df0 0x7fc494135220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 -- 10.20.193.78:0/932593813 >> v2:10.20.193.115:3300/0 conn(0x7fc49406c890 msgr2=0x7fc4940a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 --2- 10.20.193.78:0/932593813 >> v2:10.20.193.115:3300/0 conn(0x7fc49406c890 0x7fc4940a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient: _finish_hunting 0 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 monclient: found mon.noname-a 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:13.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.956+0000 7fc499c47640 1 -- 10.20.193.78:0/932593813 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49409df40 con 0x7fc49406cc70 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499c47640 1 -- 10.20.193.78:0/932593813 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406cc70 msgr2=0x7fc49406d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499c47640 1 --2- 10.20.193.78:0/932593813 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406cc70 0x7fc49406d080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc49406a480 tx=0x7fc48802ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f423d6cc694a892d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 1 -- 10.20.193.78:0/932593813 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48803c070 con 0x7fc49406cc70 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: dump: 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: _finish_auth 0 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: _check_auth_tickets 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 1 -- 10.20.193.78:0/932593813 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48802fde0 con 0x7fc49406cc70 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 1 -- 10.20.193.78:0/932593813 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488037660 con 0x7fc49406cc70 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc499446640 10 monclient: dump: 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 5 monclient: authenticate success, global_id 4401 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 4 set_mon_vals no callback set 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 10 monclient: shutdown 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 -- 10.20.193.78:0/932593813 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406cc70 msgr2=0x7fc49406d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 --2- 10.20.193.78:0/932593813 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406cc70 0x7fc49406d080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc49406a480 tx=0x7fc48802ef60 comp rx=0 tx=0).stop 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 -- 10.20.193.78:0/932593813 shutdown_connections 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 --2- 10.20.193.78:0/932593813 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc494134df0 0x7fc494135220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 --2- 10.20.193.78:0/932593813 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406cc70 0x7fc49406d080 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 --2- 10.20.193.78:0/932593813 >> v2:10.20.193.115:3300/0 conn(0x7fc49406c890 0x7fc4940a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 -- 10.20.193.78:0/932593813 >> 10.20.193.78:0/932593813 conn(0x7fc4940a6620 msgr2=0x7fc4940a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 -- 10.20.193.78:0/932593813 shutdown_connections 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.959+0000 7fc49c6d3640 1 -- 10.20.193.78:0/932593813 wait complete. 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient: build_initial_monmap 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient: monmap: 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 1 Processor -- start 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 1 -- start start 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient: init 2026-02-17T05:47:13.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc49406c890 0x7fc4941cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient(hunting): picked mon.noname-b con 0x7fc49406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc49406cc70 0x7fc4941cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient(hunting): picked mon.noname-c con 0x7fc49406cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc494134df0 0x7fc49406eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient(hunting): picked mon.noname-a con 0x7fc494134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc49409d360 con 0x7fc494134df0 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc49409d1e0 con 0x7fc49406c890 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc494135760 con 0x7fc49406cc70 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49c6d3640 10 monclient(hunting): authenticate will time out at 679.514918s 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc499c47640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc49406cc70 0x7fc4941cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc499c47640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc49406cc70 0x7fc4941cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39844/0 (socket says 10.20.193.78:39844) 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc499c47640 1 -- 10.20.193.78:0/2084100803 learned_addr learned my addr 10.20.193.78:0/2084100803 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49ac49640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc494134df0 0x7fc49406eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc499c47640 10 monclient(hunting): get_auth_request con 0x7fc49406cc70 auth_method 0 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc499c47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc499c47640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc499c47640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49ac49640 10 monclient(hunting): get_auth_request con 0x7fc494134df0 auth_method 0 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49ac49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49ac49640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.960+0000 7fc49ac49640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49ac49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49ac49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49ac49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49a448640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc49406c890 0x7fc4941cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49a448640 10 monclient(hunting): get_auth_request con 0x7fc49406c890 auth_method 0 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49a448640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49a448640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49a448640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 10 monclient(hunting): handle_auth_done global_id 4385 payload 824 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc49406c890 msgr2=0x7fc4941cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc49406c890 0x7fc4941cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc494134df0 msgr2=0x7fc49406eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc494134df0 0x7fc49406eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 10 monclient: _finish_hunting 0 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 1 monclient: found mon.noname-c 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc499c47640 1 -- 10.20.193.78:0/2084100803 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49409d810 con 0x7fc49406cc70 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.961+0000 7fc49ac49640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc494134df0 0x7fc49406eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc499c47640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc49406cc70 msgr2=0x7fc4941cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc499c47640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc49406cc70 0x7fc4941cfb00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc488009a70 tx=0x7fc488031070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:13.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488047070 con 0x7fc49406cc70 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: dump: 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: _finish_auth 0 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: _check_auth_tickets 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48803f7b0 con 0x7fc49406cc70 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 5 monclient: authenticate success, global_id 4385 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 10 monclient: _renew_subs 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4940a3970 con 0x7fc49406cc70 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48803c040 con 0x7fc49406cc70 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc498c45640 4 set_mon_vals no callback set 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc47affd640 10 monclient: dump: 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 10 monclient: _renew_subs 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4941285e0 con 0x7fc49406cc70 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:13.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f020 0x7fc49406f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.963+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc494074980 con 0x7fc49406f020 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f020 0x7fc49406f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc48803fba0 con 0x7fc49406cc70 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 10 monclient: get_auth_request con 0x7fc49406f020 auth_method 0 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 10 monclient(hunting): handle_auth_done global_id 4407 payload 824 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49a448640 1 -- 10.20.193.78:0/2084100803 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc494135b00 con 0x7fc49406f020 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc47affd640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc46403deb0 0x7fc464040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 4751+0+0 (secure 0 0 0) 0x7fc4880775e0 con 0x7fc49406cc70 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49ac49640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc46403deb0 0x7fc464040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.964+0000 7fc49ac49640 10 monclient: get_auth_request con 0x7fc46403deb0 auth_method 0 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc49a448640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f020 msgr2=0x7fc49406f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc49a448640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f020 0x7fc49406f400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc484011830 tx=0x7fc484011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48400fe50 con 0x7fc49406f020 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 10 monclient: dump: 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fc48400fe50 con 0x7fc49406f020 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f020 msgr2=0x7fc49406f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f020 0x7fc49406f400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc484011830 tx=0x7fc484011d00 comp rx=0 tx=0).stop 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f020 msgr2=0x7fc49406f400 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.966+0000 7fc47affd640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f020 0x7fc49406f400 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc484011830 tx=0x7fc484011d00 comp rx=0 tx=0).stop 2026-02-17T05:47:13.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.967+0000 7fc49ac49640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc46403deb0 0x7fc464040370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc49000e740 tx=0x7fc49000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:13.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49c6d3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:47:13.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49c6d3640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f940 0x7fc4941da2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:13.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49c6d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:13.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940744d0 con 0x7fc49406f940 2026-02-17T05:47:13.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f940 0x7fc4941da2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:13.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 10 monclient: get_auth_request con 0x7fc49406f940 auth_method 0 2026-02-17T05:47:13.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:13.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:13.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:13.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:13.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:13.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:13.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 10 monclient(hunting): handle_auth_done global_id 4410 payload 824 2026-02-17T05:47:13.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.970+0000 7fc49a448640 1 -- 10.20.193.78:0/2084100803 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc49406fd50 con 0x7fc49406f940 2026-02-17T05:47:13.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49a448640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f940 msgr2=0x7fc4941da2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:13.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49a448640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f940 0x7fc4941da2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc484011830 tx=0x7fc4840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:13.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484003920 con 0x7fc49406f940 2026-02-17T05:47:13.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:13.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 10 monclient: dump: 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7fc484003920 con 0x7fc49406f940 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:13.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f940 msgr2=0x7fc4941da2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f940 0x7fc4941da2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc484011830 tx=0x7fc4840047c0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f940 msgr2=0x7fc4941da2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc47affd640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc49406f940 0x7fc4941da2e0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc484011830 tx=0x7fc4840047c0 comp rx=0 tx=0).stop 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 12, 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 83776, 2026-02-17T05:47:13.976 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:13.977 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:13.978 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:47:13.979 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:13.980 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:13.981 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.982 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:13.983 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:13.984 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:47:13.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:47:13.986 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:47:13.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc46403deb0 msgr2=0x7fc464040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49c6d3640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc46403deb0 0x7fc464040370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc49000e740 tx=0x7fc49000b040 comp rx=0 tx=0).stop 2026-02-17T05:47:13.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49c6d3640 10 monclient: shutdown 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49c6d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc49406cc70 msgr2=0x7fc4941cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49c6d3640 1 --2- 10.20.193.78:0/2084100803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc49406cc70 0x7fc4941cfb00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc488009a70 tx=0x7fc488031070 comp rx=0 tx=0).stop 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49ac49640 1 -- 10.20.193.78:0/2084100803 reap_dead start 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49ac49640 1 -- 10.20.193.78:0/2084100803 reap_dead start 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.972+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 shutdown_connections 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.973+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 >> 10.20.193.78:0/2084100803 conn(0x7fc4940a6620 msgr2=0x7fc49409be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.973+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 shutdown_connections 2026-02-17T05:47:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:13.973+0000 7fc49c6d3640 1 -- 10.20.193.78:0/2084100803 wait complete. 2026-02-17T05:47:13.988 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:47:13.988 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:47:13.989 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:47:14.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient: get_monmap_and_config 2026-02-17T05:47:14.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient: build_initial_monmap 2026-02-17T05:47:14.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient: monmap: 2026-02-17T05:47:14.058 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:14.058 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:14.055304+0000 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:14.055304+0000 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 1 Processor -- start 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 1 -- start start 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient: init 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4e7406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4e7409c4a0 0x7f4e7413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient(hunting): picked mon.noname-a con 0x7f4e7409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e7413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient(hunting): picked mon.noname-b con 0x7f4e7406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7406daf0 con 0x7f4e7406c890 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f4e740a0880 con 0x7f4e7409c4a0 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7409cc80 con 0x7f4e7406d380 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:14.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e78a3f640 10 monclient(hunting): authenticate will time out at 679.608885s 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e71d74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4e7409c4a0 0x7f4e7413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.054+0000 7f4e72d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e7413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4e7409c4a0 0x7f4e7413b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48236/0 (socket says 10.20.193.78:48236) 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 1 -- 10.20.193.78:0/3109254136 learned_addr learned my addr 10.20.193.78:0/3109254136 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient(hunting): get_auth_request con 0x7f4e7409c4a0 auth_method 0 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e72d76640 10 monclient(hunting): get_auth_request con 0x7f4e7406d380 auth_method 0 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e72d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e72d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e72d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e72d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e72d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e72d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient(hunting): handle_auth_done global_id 4413 payload 274 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 1 -- 10.20.193.78:0/3109254136 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 msgr2=0x7f4e7413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 1 --2- 10.20.193.78:0/3109254136 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e7413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 1 -- 10.20.193.78:0/3109254136 >> v1:10.20.193.115:6789/0 conn(0x7f4e7406c890 legacy=0x7f4e7406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient: _finish_hunting 0 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 1 monclient: found mon.noname-a 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:14.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.055+0000 7f4e71d74640 1 -- 10.20.193.78:0/3109254136 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7409cf60 con 0x7f4e7409c4a0 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71d74640 1 -- 10.20.193.78:0/3109254136 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 msgr2=0x7f4e7413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71d74640 1 --2- 10.20.193.78:0/3109254136 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 0x7f4e7413b3a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009970 tx=0x7f4e5c02ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388e288297d33a88 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 1 -- 10.20.193.78:0/3109254136 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c03c070 con 0x7f4e7409c4a0 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: dump: 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.068 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: _finish_auth 0 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: _check_auth_tickets 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 1 -- 10.20.193.78:0/3109254136 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c02fcd0 con 0x7f4e7409c4a0 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 1 -- 10.20.193.78:0/3109254136 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c037660 con 0x7f4e7409c4a0 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e78a3f640 5 monclient: authenticate success, global_id 4413 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e78a3f640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e78a3f640 4 set_mon_vals no callback set 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.057+0000 7f4e71573640 10 monclient: dump: 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.069 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 10 monclient: shutdown 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 -- 10.20.193.78:0/3109254136 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 msgr2=0x7f4e7413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 --2- 10.20.193.78:0/3109254136 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 0x7f4e7413b3a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009970 tx=0x7f4e5c02ee90 comp rx=0 tx=0).stop 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 -- 10.20.193.78:0/3109254136 shutdown_connections 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 --2- 10.20.193.78:0/3109254136 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e7413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 --2- 10.20.193.78:0/3109254136 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 0x7f4e7413b3a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 -- 10.20.193.78:0/3109254136 >> 10.20.193.78:0/3109254136 conn(0x7f4e740a6620 msgr2=0x7f4e740a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 -- 10.20.193.78:0/3109254136 shutdown_connections 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 -- 10.20.193.78:0/3109254136 wait complete. 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 10 monclient: build_initial_monmap 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 10 monclient: monmap: 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:14.070 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 Processor -- start 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 1 -- start start 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.058+0000 7f4e78a3f640 10 monclient: init 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e7406c890 0x7f4e741cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient(hunting): picked mon.noname-b con 0x7f4e7406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e741302b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient(hunting): picked mon.noname-c con 0x7f4e7406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 0x7f4e741307f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient(hunting): picked mon.noname-a con 0x7f4e7409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7409bfa0 con 0x7f4e7409c4a0 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7409be20 con 0x7f4e7406c890 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7413de30 con 0x7f4e7406d380 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e78a3f640 10 monclient(hunting): authenticate will time out at 679.613171s 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e741302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e741302b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39858/0 (socket says 10.20.193.78:39858) 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 0x7f4e741307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 -- 10.20.193.78:0/401457442 learned_addr learned my addr 10.20.193.78:0/401457442 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient(hunting): get_auth_request con 0x7f4e7406d380 auth_method 0 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 10 monclient(hunting): get_auth_request con 0x7f4e7409c4a0 auth_method 0 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72575640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e7406c890 0x7f4e741cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72575640 10 monclient(hunting): get_auth_request con 0x7f4e7406c890 auth_method 0 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient(hunting): handle_auth_done global_id 4391 payload 824 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e7406c890 msgr2=0x7f4e741cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e7406c890 0x7f4e741cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 msgr2=0x7f4e741307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 0x7f4e741307f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient: _finish_hunting 0 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 monclient: found mon.noname-c 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e71d74640 1 -- 10.20.193.78:0/401457442 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7409e930 con 0x7f4e7406d380 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.059+0000 7f4e72d76640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7409c4a0 0x7f4e741307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e71d74640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 msgr2=0x7f4e741302b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e71d74640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e741302b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009970 tx=0x7f4e5c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c03c040 con 0x7f4e7406d380 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: dump: 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.074 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: _finish_auth 0 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: _check_auth_tickets 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e78a3f640 5 monclient: authenticate success, global_id 4391 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c047070 con 0x7f4e7406d380 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e78a3f640 10 monclient: _renew_subs 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e78a3f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7409dbb0 con 0x7f4e7406d380 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c042890 con 0x7f4e7406d380 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e5affd640 10 monclient: dump: 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e70d72640 4 set_mon_vals no callback set 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e78a3f640 10 monclient: _renew_subs 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e78a3f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7409f600 con 0x7f4e7406d380 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.061+0000 7f4e78a3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e78a3f640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413bb00 0x7f4e7413bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e78a3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7409e0c0 con 0x7f4e7413bb00 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413bb00 0x7f4e7413bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 10 monclient: get_auth_request con 0x7f4e7413bb00 auth_method 0 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 10 monclient(hunting): handle_auth_done global_id 4394 payload 824 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72575640 1 -- 10.20.193.78:0/401457442 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e7406f7f0 con 0x7f4e7413bb00 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4e5c042a30 con 0x7f4e7406d380 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e5affd640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4003deb0 0x7f4e40040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 4751+0+0 (secure 0 0 0) 0x7f4e5c077a30 con 0x7f4e7406d380 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72d76640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4003deb0 0x7f4e40040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.062+0000 7f4e72d76640 10 monclient: get_auth_request con 0x7f4e4003deb0 auth_method 0 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e72575640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413bb00 msgr2=0x7f4e7413bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e72575640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413bb00 0x7f4e7413bee0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800e730 tx=0x7f4e6800ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68006ae0 con 0x7f4e7413bb00 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 10 monclient: dump: 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.077 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4e68006ae0 con 0x7f4e7413bb00 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413bb00 msgr2=0x7f4e7413bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413bb00 0x7f4e7413bee0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800e730 tx=0x7f4e6800ec00 comp rx=0 tx=0).stop 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413bb00 msgr2=0x7f4e7413bee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.064+0000 7f4e5affd640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413bb00 0x7f4e7413bee0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800e730 tx=0x7f4e6800ec00 comp rx=0 tx=0).stop 2026-02-17T05:47:14.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.065+0000 7f4e72d76640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4003deb0 0x7f4e40040370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400e740 tx=0x7f4e6400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e78a3f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e78a3f640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413c420 0x7f4e741da520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e78a3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7406cc70 con 0x7f4e7413c420 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413c420 0x7f4e741da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 10 monclient: get_auth_request con 0x7f4e7413c420 auth_method 0 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 10 monclient(hunting): handle_auth_done global_id 4397 payload 824 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.067+0000 7f4e72575640 1 -- 10.20.193.78:0/401457442 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4e7409e490 con 0x7f4e7413c420 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e72575640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413c420 msgr2=0x7f4e741da520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e72575640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413c420 0x7f4e741da520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800e730 tx=0x7f4e68003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800c930 con 0x7f4e7413c420 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 10 monclient: dump: 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4707 (secure 0 0 0) 0x7f4e6800c930 con 0x7f4e7413c420 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413c420 msgr2=0x7f4e741da520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413c420 0x7f4e741da520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800e730 tx=0x7f4e68003860 comp rx=0 tx=0).stop 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413c420 msgr2=0x7f4e741da520 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e5affd640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7413c420 0x7f4e741da520 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800e730 tx=0x7f4e68003860 comp rx=0 tx=0).stop 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 12, 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 83875, 2026-02-17T05:47:14.082 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:14.083 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:47:14.085 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:14.086 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:14.087 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:47:14.088 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.089 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:47:14.090 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:47:14.091 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4003deb0 msgr2=0x7f4e40040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4003deb0 0x7f4e40040370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400e740 tx=0x7f4e6400b040 comp rx=0 tx=0).stop 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 10 monclient: shutdown 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 msgr2=0x7f4e741302b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e741302b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009970 tx=0x7f4e5c004830 comp rx=0 tx=0).stop 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e72d76640 1 -- 10.20.193.78:0/401457442 reap_dead start 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 shutdown_connections 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 --2- 10.20.193.78:0/401457442 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7406d380 0x7f4e741302b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 >> 10.20.193.78:0/401457442 conn(0x7f4e740a6620 msgr2=0x7f4e740a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 shutdown_connections 2026-02-17T05:47:14.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.070+0000 7f4e78a3f640 1 -- 10.20.193.78:0/401457442 wait complete. 2026-02-17T05:47:14.093 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:47:14.093 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:47:14.093 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.158+0000 7f9cd9eb6640 10 monclient: get_monmap_and_config 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.158+0000 7f9cd9eb6640 10 monclient: build_initial_monmap 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.158+0000 7f9cd9eb6640 10 monclient: monmap: 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:14.159661+0000 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:14.159661+0000 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.160 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:14.161 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:14.161 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:14.161 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:14.161 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:14.161 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.158+0000 7f9cd9eb6640 1 Processor -- start 2026-02-17T05:47:14.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.158+0000 7f9cd9eb6640 1 -- start start 2026-02-17T05:47:14.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.158+0000 7f9cd9eb6640 10 monclient: init 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9cd406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9cd409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 10 monclient(hunting): picked mon.noname-b con 0x7f9cd406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd406daf0 con 0x7f9cd406c890 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd409cc80 con 0x7f9cd409c4a0 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd40a0880 con 0x7f9cd406d380 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd9eb6640 10 monclient(hunting): authenticate will time out at 679.713308s 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd37fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f9cd406c890 0x7f9cd406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56166/0 (socket says 10.20.193.78:56166) 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd37fe640 1 -- 10.20.193.78:0/2022442367 learned_addr learned my addr 10.20.193.78:0/2022442367 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:47:14.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 1 --2- 10.20.193.78:0/2022442367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient(hunting): get_auth_request con 0x7f9cd406d380 auth_method 0 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient(hunting): handle_auth_done global_id 4400 payload 274 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 1 -- 10.20.193.78:0/2022442367 >> v1:10.20.193.115:6789/0 conn(0x7f9cd409c4a0 legacy=0x7f9cd413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 1 -- 10.20.193.78:0/2022442367 >> v1:10.20.193.78:6789/0 conn(0x7f9cd406c890 legacy=0x7f9cd406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 1 monclient: found mon.noname-b 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:47:14.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.159+0000 7f9cd3fff640 1 -- 10.20.193.78:0/2022442367 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd409cf60 con 0x7f9cd406d380 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.162+0000 7f9cd3fff640 1 -- 10.20.193.78:0/2022442367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 msgr2=0x7f9cd413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.162+0000 7f9cd3fff640 1 --2- 10.20.193.78:0/2022442367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd413ead0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009dc0 tx=0x7f9cc802f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71106f4917b44252 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 1 -- 10.20.193.78:0/2022442367 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc803c070 con 0x7f9cd406d380 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: dump: 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.166 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: _finish_auth 0 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 1 -- 10.20.193.78:0/2022442367 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc8037640 con 0x7f9cd406d380 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 5 monclient: authenticate success, global_id 4400 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 1 -- 10.20.193.78:0/2022442367 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8037a10 con 0x7f9cd406d380 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 4 set_mon_vals no callback set 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd27fc640 10 monclient: dump: 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.167 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 10 monclient: shutdown 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/2022442367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 msgr2=0x7f9cd413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 1 --2- 10.20.193.78:0/2022442367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd413ead0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009dc0 tx=0x7f9cc802f1c0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/2022442367 shutdown_connections 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 1 --2- 10.20.193.78:0/2022442367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd413ead0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/2022442367 >> 10.20.193.78:0/2022442367 conn(0x7f9cd40a6620 msgr2=0x7f9cd40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/2022442367 shutdown_connections 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.164+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/2022442367 wait complete. 2026-02-17T05:47:14.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient: build_initial_monmap 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient: monmap: 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:14.169 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 1 Processor -- start 2026-02-17T05:47:14.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 1 -- start start 2026-02-17T05:47:14.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient: init 2026-02-17T05:47:14.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:14.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:47:14.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cd406c890 0x7f9cd413d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient(hunting): picked mon.noname-a con 0x7f9cd406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:14.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd4136460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient(hunting): picked mon.noname-c con 0x7f9cd406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409c4a0 0x7f9cd409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient(hunting): picked mon.noname-b con 0x7f9cd409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd409bfa0 con 0x7f9cd406c890 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd409be20 con 0x7f9cd409c4a0 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd413f010 con 0x7f9cd406d380 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd9eb6640 10 monclient(hunting): authenticate will time out at 679.719650s 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd4136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cd406c890 0x7f9cd413d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd4136460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39900/0 (socket says 10.20.193.78:39900) 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 -- 10.20.193.78:0/1345149403 learned_addr learned my addr 10.20.193.78:0/1345149403 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient(hunting): get_auth_request con 0x7f9cd406d380 auth_method 0 2026-02-17T05:47:14.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd37fe640 10 monclient(hunting): get_auth_request con 0x7f9cd406c890 auth_method 0 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_done global_id 4403 payload 824 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409c4a0 msgr2=0x7f9cd409db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409c4a0 0x7f9cd409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cd406c890 msgr2=0x7f9cd413d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cd406c890 0x7f9cd413d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 monclient: found mon.noname-c 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.165+0000 7f9cd2ffd640 1 -- 10.20.193.78:0/1345149403 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd413e060 con 0x7f9cd406d380 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.167+0000 7f9cd2ffd640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 msgr2=0x7f9cd4136460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.167+0000 7f9cd2ffd640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd4136460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c600 tx=0x7f9cc400cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4012a40 con 0x7f9cd406d380 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: dump: 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.176 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: _finish_auth 0 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: _check_auth_tickets 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc4017070 con 0x7f9cd406d380 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 5 monclient: authenticate success, global_id 4403 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 10 monclient: _renew_subs 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd4121210 con 0x7f9cd406d380 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4012e20 con 0x7f9cd406d380 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd1ffb640 4 set_mon_vals no callback set 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 10 monclient: dump: 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.177 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 10 monclient: _renew_subs 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd4121560 con 0x7f9cd406d380 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409f2a0 0x7f9cd409e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd406cc70 con 0x7f9cd409f2a0 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9cc401c460 con 0x7f9cd406d380 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd37fe640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409f2a0 0x7f9cd409e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd37fe640 10 monclient: get_auth_request con 0x7f9cd409f2a0 auth_method 0 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.168+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.169+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.169+0000 7f9caffff640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9ca403deb0 0x7f9ca4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.169+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 4751+0+0 (secure 0 0 0) 0x7f9cc4052520 con 0x7f9cd406d380 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.169+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_done global_id 34111 payload 824 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.169+0000 7f9cd37fe640 1 -- 10.20.193.78:0/1345149403 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cd41218a0 con 0x7f9cd409f2a0 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.169+0000 7f9cd3fff640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9ca403deb0 0x7f9ca4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.169+0000 7f9cd3fff640 10 monclient: get_auth_request con 0x7f9ca403deb0 auth_method 0 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9cd37fe640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409f2a0 msgr2=0x7f9cd409e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9cd37fe640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409f2a0 0x7f9cd409e060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000eaa0 tx=0x7f9cc000ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000cbf0 con 0x7f9cd409f2a0 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 10 monclient: dump: 2026-02-17T05:47:14.179 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f9cc0019070 con 0x7f9cd409f2a0 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409f2a0 msgr2=0x7f9cd409e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409f2a0 0x7f9cd409e060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000eaa0 tx=0x7f9cc000ef70 comp rx=0 tx=0).stop 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409f2a0 msgr2=0x7f9cd409e060 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.171+0000 7f9caffff640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd409f2a0 0x7f9cd409e060 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000eaa0 tx=0x7f9cc000ef70 comp rx=0 tx=0).stop 2026-02-17T05:47:14.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.172+0000 7f9cd3fff640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9ca403deb0 0x7f9ca4040370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9cd406a480 tx=0x7f9cc8031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd9eb6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd9eb6640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd41369a0 0x7f9cd41da4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd9eb6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40744d0 con 0x7f9cd41369a0 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd41369a0 0x7f9cd41da4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 10 monclient: get_auth_request con 0x7f9cd41369a0 auth_method 0 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_done global_id 34114 payload 824 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.175+0000 7f9cd37fe640 1 -- 10.20.193.78:0/1345149403 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9cd409e5a0 con 0x7f9cd41369a0 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd37fe640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd41369a0 msgr2=0x7f9cd41da4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd37fe640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd41369a0 0x7f9cd41da4b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0016040 tx=0x7f9cc0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc001edb0 con 0x7f9cd41369a0 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 10 monclient: dump: 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f9cc001ed90 con 0x7f9cd41369a0 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd41369a0 msgr2=0x7f9cd41da4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd41369a0 0x7f9cd41da4b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0016040 tx=0x7f9cc0004a40 comp rx=0 tx=0).stop 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd41369a0 msgr2=0x7f9cd41da4b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9caffff640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cd41369a0 0x7f9cd41da4b0 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0016040 tx=0x7f9cc0004a40 comp rx=0 tx=0).stop 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 12, 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 16194, 2026-02-17T05:47:14.184 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:14.185 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.186 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:14.187 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:14.188 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:14.189 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.190 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:14.191 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:47:14.192 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:47:14.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9ca403deb0 msgr2=0x7f9ca4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9ca403deb0 0x7f9ca4040370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9cd406a480 tx=0x7f9cc8031000 comp rx=0 tx=0).stop 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 10 monclient: shutdown 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 msgr2=0x7f9cd4136460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 1 --2- 10.20.193.78:0/1345149403 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cd406d380 0x7f9cd4136460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c600 tx=0x7f9cc400cad0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd3fff640 1 -- 10.20.193.78:0/1345149403 reap_dead start 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd3fff640 1 -- 10.20.193.78:0/1345149403 reap_dead start 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 shutdown_connections 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 >> 10.20.193.78:0/1345149403 conn(0x7f9cd40a6620 msgr2=0x7f9cd4070290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 shutdown_connections 2026-02-17T05:47:14.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.178+0000 7f9cd9eb6640 1 -- 10.20.193.78:0/1345149403 wait complete. 2026-02-17T05:47:14.194 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:47:14.194 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:47:14.194 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.260+0000 7fae33fff640 10 monclient: get_monmap_and_config 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.260+0000 7fae33fff640 10 monclient: build_initial_monmap 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.260+0000 7fae33fff640 10 monclient: monmap: 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:14.262142+0000 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:14.262142+0000 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:14.262 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 1 Processor -- start 2026-02-17T05:47:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 1 -- start start 2026-02-17T05:47:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient: init 2026-02-17T05:47:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:14.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient: _add_conns ranks=[4,1,3,2,0] 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fae3409ddd0 0x7fae3409e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient(hunting): picked mon.noname-c con 0x7fae3409ddd0 addr v2:10.20.193.115:3300/0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fae340a12d0 addr v1:10.20.193.115:6789/0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae340a16e0 0x7fae340a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient(hunting): picked mon.noname-a con 0x7fae340a16e0 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae340a04c0 con 0x7fae340a12d0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fae3409f560 con 0x7fae340a16e0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fae3409ee60 con 0x7fae3409ddd0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae33fff640 10 monclient(hunting): authenticate will time out at 679.815678s 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae340a16e0 0x7fae340a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae340a16e0 0x7fae340a6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48272/0 (socket says 10.20.193.78:48272) 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 1 -- 10.20.193.78:0/2932409244 learned_addr learned my addr 10.20.193.78:0/2932409244 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 10 monclient(hunting): get_auth_request con 0x7fae340a16e0 auth_method 0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae32ffd640 1 --2- 10.20.193.78:0/2932409244 >> v2:10.20.193.115:3300/0 conn(0x7fae3409ddd0 0x7fae3409e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae32ffd640 10 monclient(hunting): get_auth_request con 0x7fae3409ddd0 auth_method 0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae32ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae32ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 10 monclient(hunting): handle_auth_done global_id 4422 payload 274 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 1 -- 10.20.193.78:0/2932409244 >> v2:10.20.193.115:3300/0 conn(0x7fae3409ddd0 msgr2=0x7fae3409e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.261+0000 7fae337fe640 1 --2- 10.20.193.78:0/2932409244 >> v2:10.20.193.115:3300/0 conn(0x7fae3409ddd0 0x7fae3409e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.262+0000 7fae337fe640 1 -- 10.20.193.78:0/2932409244 >> v1:10.20.193.115:6789/0 conn(0x7fae340a12d0 legacy=0x7fae340a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.262+0000 7fae337fe640 10 monclient: _finish_hunting 0 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.262+0000 7fae337fe640 1 monclient: found mon.noname-a 2026-02-17T05:47:14.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.262+0000 7fae337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.262+0000 7fae337fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.262+0000 7fae337fe640 1 -- 10.20.193.78:0/2932409244 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3409ff00 con 0x7fae340a16e0 2026-02-17T05:47:14.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae337fe640 1 -- 10.20.193.78:0/2932409244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a16e0 msgr2=0x7fae340a6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae337fe640 1 --2- 10.20.193.78:0/2932409244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a16e0 0x7fae340a6b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae340a1be0 tx=0x7fae2402f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7c727d749575d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 1 -- 10.20.193.78:0/2932409244 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2402fe50 con 0x7fae340a16e0 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: dump: 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.266 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: _finish_auth 0 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 1 -- 10.20.193.78:0/2932409244 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae240373f0 con 0x7fae340a16e0 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 5 monclient: authenticate success, global_id 4422 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 1 -- 10.20.193.78:0/2932409244 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae240377c0 con 0x7fae340a16e0 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 4 set_mon_vals no callback set 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae31ffb640 10 monclient: dump: 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.268 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 10 monclient: shutdown 2026-02-17T05:47:14.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:14.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 1 -- 10.20.193.78:0/2932409244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a16e0 msgr2=0x7fae340a6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 1 --2- 10.20.193.78:0/2932409244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a16e0 0x7fae340a6b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae340a1be0 tx=0x7fae2402f1e0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 1 -- 10.20.193.78:0/2932409244 shutdown_connections 2026-02-17T05:47:14.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 1 --2- 10.20.193.78:0/2932409244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a16e0 0x7fae340a6b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.264+0000 7fae33fff640 1 --2- 10.20.193.78:0/2932409244 >> v2:10.20.193.115:3300/0 conn(0x7fae3409ddd0 0x7fae3409e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 -- 10.20.193.78:0/2932409244 >> 10.20.193.78:0/2932409244 conn(0x7fae340a7520 msgr2=0x7fae340a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 -- 10.20.193.78:0/2932409244 shutdown_connections 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 -- 10.20.193.78:0/2932409244 wait complete. 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient: build_initial_monmap 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient: monmap: 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:14.274 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 Processor -- start 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 -- start start 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient: init 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae3409ddd0 0x7fae341cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient(hunting): picked mon.noname-c con 0x7fae3409ddd0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a12d0 0x7fae341cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient(hunting): picked mon.noname-a con 0x7fae340a12d0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae340a16e0 0x7fae34130b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient(hunting): picked mon.noname-b con 0x7fae340a16e0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae340a82b0 con 0x7fae340a12d0 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae3409e6f0 con 0x7fae340a16e0 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae340a8430 con 0x7fae3409ddd0 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae33fff640 10 monclient(hunting): authenticate will time out at 679.819901s 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae327fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a12d0 0x7fae341cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae32ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae3409ddd0 0x7fae341cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae327fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a12d0 0x7fae341cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48278/0 (socket says 10.20.193.78:48278) 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae327fc640 1 -- 10.20.193.78:0/2858465141 learned_addr learned my addr 10.20.193.78:0/2858465141 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae32ffd640 10 monclient(hunting): get_auth_request con 0x7fae3409ddd0 auth_method 0 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae32ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae32ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae327fc640 10 monclient(hunting): get_auth_request con 0x7fae340a12d0 auth_method 0 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae327fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae327fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.265+0000 7fae327fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae32ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae32ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae337fe640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae340a16e0 0x7fae34130b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae32ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae337fe640 10 monclient(hunting): get_auth_request con 0x7fae340a16e0 auth_method 0 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae337fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 10 monclient(hunting): handle_auth_done global_id 4425 payload 824 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 1 -- 10.20.193.78:0/2858465141 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae3409ddd0 msgr2=0x7fae341cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae3409ddd0 0x7fae341cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 1 -- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae340a16e0 msgr2=0x7fae34130b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae340a16e0 0x7fae34130b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 10 monclient: _finish_hunting 0 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 1 monclient: found mon.noname-a 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.266+0000 7fae327fc640 1 -- 10.20.193.78:0/2858465141 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae341342f0 con 0x7fae340a12d0 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae327fc640 1 -- 10.20.193.78:0/2858465141 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a12d0 msgr2=0x7fae341cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae327fc640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a12d0 0x7fae341cf4e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00c5d0 tx=0x7fae1c00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 1 -- 10.20.193.78:0/2858465141 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c016020 con 0x7fae340a12d0 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: dump: 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.278 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: _finish_auth 0 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: _check_auth_tickets 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 1 -- 10.20.193.78:0/2858465141 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1c012a60 con 0x7fae340a12d0 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 5 monclient: authenticate success, global_id 4425 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 10 monclient: _renew_subs 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:14.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae341345d0 con 0x7fae340a12d0 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 1 -- 10.20.193.78:0/2858465141 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c012e50 con 0x7fae340a12d0 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae317fa640 4 set_mon_vals no callback set 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae137fe640 10 monclient: dump: 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:14.280 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 10 monclient: _renew_subs 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae341322f0 con 0x7fae340a12d0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.268+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3409d820 con 0x7fae340a12d0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.269+0000 7fae137fe640 1 -- 10.20.193.78:0/2858465141 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fae1c01b400 con 0x7fae340a12d0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.269+0000 7fae137fe640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae0003e080 0x7fae00040540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.269+0000 7fae137fe640 1 -- 10.20.193.78:0/2858465141 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4751+0+0 (secure 0 0 0) 0x7fae1c0528a0 con 0x7fae340a12d0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.269+0000 7fae32ffd640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae0003e080 0x7fae00040540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.269+0000 7fae32ffd640 10 monclient: get_auth_request con 0x7fae0003e080 auth_method 0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.271+0000 7fae137fe640 1 -- 10.20.193.78:0/2858465141 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fae3409d820 con 0x7fae340a12d0 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.271+0000 7fae137fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:14.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.271+0000 7fae137fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:14.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.272+0000 7fae32ffd640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae0003e080 0x7fae00040540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae28009a50 tx=0x7fae28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:14.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.385+0000 7fae33fff640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:47:14.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.385+0000 7fae33fff640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:47:14.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.385+0000 7fae33fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:14.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.385+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fae340a71a0 con 0x7fae340a12d0 2026-02-17T05:47:14.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.385+0000 7fae137fe640 1 -- 10.20.193.78:0/2858465141 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fae1c020030 con 0x7fae340a12d0 2026-02-17T05:47:14.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.385+0000 7fae137fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:47:14.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.385+0000 7fae137fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:14.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae0003e080 msgr2=0x7fae00040540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae0003e080 0x7fae00040540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae28009a50 tx=0x7fae28008040 comp rx=0 tx=0).stop 2026-02-17T05:47:14.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 10 monclient: shutdown 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a12d0 msgr2=0x7fae341cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a12d0 0x7fae341cf4e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00c5d0 tx=0x7fae1c00caa0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 shutdown_connections 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae0003e080 0x7fae00040540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae340a16e0 0x7fae34130b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae340a12d0 0x7fae341cf4e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 --2- 10.20.193.78:0/2858465141 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae3409ddd0 0x7fae341cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 >> 10.20.193.78:0/2858465141 conn(0x7fae340a7520 msgr2=0x7fae340a2640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 shutdown_connections 2026-02-17T05:47:14.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:14.386+0000 7fae33fff640 1 -- 10.20.193.78:0/2858465141 wait complete. 2026-02-17T05:47:14.395 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:47:14.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.672+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.672+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=8,config=2,health=12,kv=11,logm=67} crc {auth=1561265925,config=2435484489,health=861809967,kv=1121147495,logm=4123231035}) 2026-02-17T05:47:14.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.672+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=10,logm=66} crc {auth=2448445663,config=692619927,health=861809967,kv=394596124,logm=1864004873}) 2026-02-17T05:47:14.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.672+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.672+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=8,config=2,health=12,kv=11,logm=67} crc {auth=1561265925,config=2435484489,health=861809967,kv=1121147495,logm=4123231035}) 2026-02-17T05:47:14.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.672+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=1,health=10,kv=9,logm=70} crc {auth=2448445663,config=1228700967,health=4177358965,kv=3076176908,logm=2159148290}) 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1617713925}) 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=438669911}) 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1617713925}) 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272520210}) 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985256335}) 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2036342039}) 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985256335}) 2026-02-17T05:47:14.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.673+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1687207643}) 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114612483}) 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3625328176}) 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114612483}) 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1835137348}) 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=25,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=6,osdmap=27} crc {logm=3167816359,mdsmap=1160443688,mgr=3003080605,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2984988131,mon_config_key=4028116958,monmap=3140514009,osd_metadata=1501630441,osd_pg_creating=379183070,osd_snap=595986708,osdmap=270300021}) 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=27,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=6,osdmap=22} crc {logm=1061658152,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3656682377,mon_config_key=1765318643,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=379183070,osd_snap=595986708,osdmap=3725671577}) 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=25,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=6,osdmap=27} crc {logm=3167816359,mdsmap=1160443688,mgr=3003080605,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2984988131,mon_config_key=4028116958,monmap=3140514009,osd_metadata=1501630441,osd_pg_creating=379183070,osd_snap=595986708,osdmap=270300021}) 2026-02-17T05:47:14.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.674+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=25,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=6,osdmap=26} crc {logm=2448812785,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4018125704,mon_config_key=1912222953,monmap=1228700967,osd_metadata=2997046373,osd_pg_creating=379183070,osd_snap=595986708,osdmap=1762260071}) 2026-02-17T05:47:14.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.675+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.675+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=62} crc {osdmap=1667561109}) 2026-02-17T05:47:14.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.675+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2323652595}) 2026-02-17T05:47:14.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.675+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:47:14.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.675+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=62} crc {osdmap=1667561109}) 2026-02-17T05:47:14.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:47:14.675+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=62} crc {osdmap=557560570}) 2026-02-17T05:47:14.808 INFO:tasks.rados.rados.0.trial115.stdout:208: done (1 left) 2026-02-17T05:47:14.808 INFO:tasks.rados.rados.0.trial115.stdout:209: write oid 7 current snap is 18 2026-02-17T05:47:14.809 INFO:tasks.rados.rados.0.trial115.stdout:207: finishing rollback tid 1 to trial11545420-46 2026-02-17T05:47:14.809 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 0 (ObjNum 81 snap 0 seq_num 1561655257) dirty dne 2026-02-17T05:47:14.809 INFO:tasks.rados.rados.0.trial115.stdout:209: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2026-02-17T05:47:14.814 INFO:tasks.rados.rados.0.trial115.stdout:209: writing trial11545420-7 from 648770 to 1355102 tid 1 2026-02-17T05:47:14.818 INFO:tasks.rados.rados.0.trial115.stdout:209: writing trial11545420-7 from 2120431 to 2540461 tid 2 2026-02-17T05:47:14.824 INFO:tasks.rados.rados.0.trial115.stdout:209: writing trial11545420-7 from 3271235 to 3703627 tid 3 2026-02-17T05:47:14.824 INFO:tasks.rados.rados.0.trial115.stdout:207: done (1 left) 2026-02-17T05:47:14.824 INFO:tasks.rados.rados.0.trial115.stdout:210: write (excl) oid 45 current snap is 18 2026-02-17T05:47:14.824 INFO:tasks.rados.rados.0.trial115.stdout:210: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2026-02-17T05:47:14.829 INFO:tasks.rados.rados.0.trial115.stdout:210: writing trial11545420-45 from 665577 to 1447158 tid 1 2026-02-17T05:47:14.833 INFO:tasks.rados.rados.0.trial115.stdout:210: writing trial11545420-45 from 2062560 to 2626248 tid 2 2026-02-17T05:47:14.838 INFO:tasks.rados.rados.0.trial115.stdout:210: writing trial11545420-45 from 3182305 to 3825969 tid 3 2026-02-17T05:47:14.838 INFO:tasks.rados.rados.0.trial115.stdout:211: read oid 22 snap 8 2026-02-17T05:47:14.838 INFO:tasks.rados.rados.0.trial115.stdout:211: expect (ObjNum 21 snap 0 seq_num 21) 2026-02-17T05:47:14.839 INFO:tasks.rados.rados.0.trial115.stdout:212: snap_remove snap 14 2026-02-17T05:47:15.206 INFO:tasks.rados.rados.0.trial115.stdout:212: done (3 left) 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:213: snap_create 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: finishing write tid 1 to trial11545420-7 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: finishing write tid 2 to trial11545420-7 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: finishing write tid 3 to trial11545420-7 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: finishing write tid 4 to trial11545420-7 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: finishing write tid 5 to trial11545420-7 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: oid 7 updating version 0 to 44 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: oid 7 updating version 44 to 46 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: oid 7 version 46 is already newer than 45 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:209: oid 7 version 46 is already newer than 43 2026-02-17T05:47:15.207 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 46 (ObjNum 110 snap 18 seq_num 110) dirty exists 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:209: left oid 7 (ObjNum 110 snap 18 seq_num 110) 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: finishing write tid 1 to trial11545420-45 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: finishing write tid 2 to trial11545420-45 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: finishing write tid 3 to trial11545420-45 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: finishing write tid 4 to trial11545420-45 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: finishing write tid 5 to trial11545420-45 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: oid 45 updating version 0 to 88 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: oid 45 updating version 88 to 90 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: oid 45 version 90 is already newer than 89 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: oid 45 version 90 is already newer than 87 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 90 (ObjNum 111 snap 18 seq_num 111) dirty exists 2026-02-17T05:47:15.208 INFO:tasks.rados.rados.0.trial115.stdout:210: left oid 45 (ObjNum 111 snap 18 seq_num 111) 2026-02-17T05:47:15.209 INFO:tasks.rados.rados.0.trial115.stdout:211: expect (ObjNum 21 snap 0 seq_num 21) 2026-02-17T05:47:15.397 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:47:15.397 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.466+0000 7f1f89c04640 10 monclient: get_monmap_and_config 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.466+0000 7f1f89c04640 10 monclient: build_initial_monmap 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.466+0000 7f1f89c04640 10 monclient: monmap: 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:15.467696+0000 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:15.467696+0000 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:15.467 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:15.468 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:15.468 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.466+0000 7f1f89c04640 1 Processor -- start 2026-02-17T05:47:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.466+0000 7f1f89c04640 1 -- start start 2026-02-17T05:47:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.466+0000 7f1f89c04640 10 monclient: init 2026-02-17T05:47:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f1f8406c890 0x7f1f8406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient(hunting): picked mon.noname-c con 0x7f1f8406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1f8409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1f8406d1b0 0x7f1f84135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient(hunting): picked mon.noname-a con 0x7f1f8406d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8406daf0 con 0x7f1f8409c4a0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f1f840a0880 con 0x7f1f8406d1b0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f1f8409cc80 con 0x7f1f8406c890 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f89c04640 10 monclient(hunting): authenticate will time out at 681.021227s 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f82ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f1f8409c4a0 0x7f1f84134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56176/0 (socket says 10.20.193.78:56176) 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f82ffd640 1 -- 10.20.193.78:0/3335087820 learned_addr learned my addr 10.20.193.78:0/3335087820 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 1 --2- 10.20.193.78:0/3335087820 >> v2:10.20.193.78:3300/0 conn(0x7f1f8406d1b0 0x7f1f84135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient(hunting): get_auth_request con 0x7f1f8406d1b0 auth_method 0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f837fe640 1 --2- 10.20.193.78:0/3335087820 >> v2:10.20.193.115:3300/0 conn(0x7f1f8406c890 0x7f1f8406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f837fe640 10 monclient(hunting): get_auth_request con 0x7f1f8406c890 auth_method 0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f837fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient(hunting): handle_auth_done global_id 4428 payload 274 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 1 -- 10.20.193.78:0/3335087820 >> v2:10.20.193.115:3300/0 conn(0x7f1f8406c890 msgr2=0x7f1f8406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 1 --2- 10.20.193.78:0/3335087820 >> v2:10.20.193.115:3300/0 conn(0x7f1f8406c890 0x7f1f8406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 1 -- 10.20.193.78:0/3335087820 >> v1:10.20.193.78:6789/0 conn(0x7f1f8409c4a0 legacy=0x7f1f84134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient: _finish_hunting 0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 1 monclient: found mon.noname-a 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.467+0000 7f1f83fff640 1 -- 10.20.193.78:0/3335087820 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8409cf60 con 0x7f1f8406d1b0 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f83fff640 1 -- 10.20.193.78:0/3335087820 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8406d1b0 msgr2=0x7f1f84135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f83fff640 1 --2- 10.20.193.78:0/3335087820 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8406d1b0 0x7f1f84135500 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1f8406a480 tx=0x7f1f7802f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a4ea8fedb5323c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 1 -- 10.20.193.78:0/3335087820 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7803c070 con 0x7f1f8406d1b0 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: dump: 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: _finish_auth 0 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: _check_auth_tickets 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 1 -- 10.20.193.78:0/3335087820 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f78037440 con 0x7f1f8406d1b0 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 1 -- 10.20.193.78:0/3335087820 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78037810 con 0x7f1f8406d1b0 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f827fc640 10 monclient: dump: 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 5 monclient: authenticate success, global_id 4428 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 4 set_mon_vals no callback set 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 10 monclient: shutdown 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 1 -- 10.20.193.78:0/3335087820 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8406d1b0 msgr2=0x7f1f84135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3335087820 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8406d1b0 0x7f1f84135500 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1f8406a480 tx=0x7f1f7802f130 comp rx=0 tx=0).stop 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 1 -- 10.20.193.78:0/3335087820 shutdown_connections 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3335087820 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8406d1b0 0x7f1f84135500 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3335087820 >> v2:10.20.193.115:3300/0 conn(0x7f1f8406c890 0x7f1f8406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 1 -- 10.20.193.78:0/3335087820 >> 10.20.193.78:0/3335087820 conn(0x7f1f840a6620 msgr2=0x7f1f840a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 1 -- 10.20.193.78:0/3335087820 shutdown_connections 2026-02-17T05:47:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.470+0000 7f1f89c04640 1 -- 10.20.193.78:0/3335087820 wait complete. 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient: build_initial_monmap 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient: monmap: 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:15.473 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 1 Processor -- start 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 1 -- start start 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient: init 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1f8406c890 0x7f1f841cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient(hunting): picked mon.noname-c con 0x7f1f8406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1f8406d1b0 0x7f1f841cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient(hunting): picked mon.noname-b con 0x7f1f8406d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8409c4a0 0x7f1f84125c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient(hunting): picked mon.noname-a con 0x7f1f8409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f840a01a0 con 0x7f1f8409c4a0 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1f840a0020 con 0x7f1f8406d1b0 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8409be20 con 0x7f1f8406c890 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f89c04640 10 monclient(hunting): authenticate will time out at 681.025681s 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8409c4a0 0x7f1f84125c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f837fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1f8406c890 0x7f1f841cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8409c4a0 0x7f1f84125c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48302/0 (socket says 10.20.193.78:48302) 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 -- 10.20.193.78:0/3104726962 learned_addr learned my addr 10.20.193.78:0/3104726962 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient(hunting): get_auth_request con 0x7f1f8409c4a0 auth_method 0 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f837fe640 10 monclient(hunting): get_auth_request con 0x7f1f8406c890 auth_method 0 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f837fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f82ffd640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1f8406d1b0 0x7f1f841cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f82ffd640 10 monclient(hunting): get_auth_request con 0x7f1f8406d1b0 auth_method 0 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f82ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient(hunting): handle_auth_done global_id 4431 payload 824 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 -- 10.20.193.78:0/3104726962 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1f8406c890 msgr2=0x7f1f841cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1f8406c890 0x7f1f841cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 -- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1f8406d1b0 msgr2=0x7f1f841cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1f8406d1b0 0x7f1f841cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient: _finish_hunting 0 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 monclient: found mon.noname-a 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.471+0000 7f1f83fff640 1 -- 10.20.193.78:0/3104726962 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f841297a0 con 0x7f1f8409c4a0 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f83fff640 1 -- 10.20.193.78:0/3104726962 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8409c4a0 msgr2=0x7f1f84125c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f83fff640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8409c4a0 0x7f1f84125c60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009e90 tx=0x7f1f78004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 1 -- 10.20.193.78:0/3104726962 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7803c050 con 0x7f1f8409c4a0 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: dump: 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: _finish_auth 0 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 1 -- 10.20.193.78:0/3104726962 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f78048070 con 0x7f1f8409c4a0 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 1 -- 10.20.193.78:0/3104726962 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78041c60 con 0x7f1f8409c4a0 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f5ffff640 10 monclient: dump: 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f81ffb640 4 set_mon_vals no callback set 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 5 monclient: authenticate success, global_id 4431 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 10 monclient: _renew_subs 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f84072710 con 0x7f1f8409c4a0 2026-02-17T05:47:15.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 10 monclient: _renew_subs 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f841291e0 con 0x7f1f8409c4a0 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.474+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f84074980 con 0x7f1f8409c4a0 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.475+0000 7f1f5ffff640 1 -- 10.20.193.78:0/3104726962 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1f7804d460 con 0x7f1f8409c4a0 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.475+0000 7f1f5ffff640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1f5403deb0 0x7f1f54040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.475+0000 7f1f5ffff640 1 -- 10.20.193.78:0/3104726962 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4807+0+0 (secure 0 0 0) 0x7f1f78078790 con 0x7f1f8409c4a0 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.475+0000 7f1f837fe640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1f5403deb0 0x7f1f54040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.475+0000 7f1f837fe640 10 monclient: get_auth_request con 0x7f1f5403deb0 auth_method 0 2026-02-17T05:47:15.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.477+0000 7f1f5ffff640 1 -- 10.20.193.78:0/3104726962 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1f78040da0 con 0x7f1f8409c4a0 2026-02-17T05:47:15.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.477+0000 7f1f5ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.477+0000 7f1f5ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:15.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.478+0000 7f1f837fe640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1f5403deb0 0x7f1f54040370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009a50 tx=0x7f1f70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.589+0000 7f1f89c04640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:47:15.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.589+0000 7f1f89c04640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:47:15.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.589+0000 7f1f89c04640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.589+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1f8406cc70 con 0x7f1f8409c4a0 2026-02-17T05:47:15.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.589+0000 7f1f5ffff640 1 -- 10.20.193.78:0/3104726962 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f1f7804d780 con 0x7f1f8409c4a0 2026-02-17T05:47:15.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.589+0000 7f1f5ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:47:15.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.589+0000 7f1f5ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:15.592 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":12,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:47:15.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1f5403deb0 msgr2=0x7f1f54040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1f5403deb0 0x7f1f54040370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009a50 tx=0x7f1f70008040 comp rx=0 tx=0).stop 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 10 monclient: shutdown 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8409c4a0 msgr2=0x7f1f84125c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8409c4a0 0x7f1f84125c60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009e90 tx=0x7f1f78004b40 comp rx=0 tx=0).stop 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 shutdown_connections 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1f8409c4a0 0x7f1f84125c60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1f5403deb0 0x7f1f54040370 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1f8406d1b0 0x7f1f841cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 --2- 10.20.193.78:0/3104726962 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1f8406c890 0x7f1f841cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 >> 10.20.193.78:0/3104726962 conn(0x7f1f840a6620 msgr2=0x7f1f840a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 shutdown_connections 2026-02-17T05:47:15.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.590+0000 7f1f89c04640 1 -- 10.20.193.78:0/3104726962 wait complete. 2026-02-17T05:47:15.599 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:47:15.599 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:47:15.599 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:47:15.599 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.665+0000 7f228d149640 10 monclient: get_monmap_and_config 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.665+0000 7f228d149640 10 monclient: build_initial_monmap 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient: monmap: 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:15.667224+0000 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:15.667224+0000 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:15.667 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 1 Processor -- start 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 1 -- start start 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient: init 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient: _add_conns ranks=[0,3,1,4,2] 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f228806c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f228806d380 0x7f22880a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient(hunting): picked mon.noname-a con 0x7f228806d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:15.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f228813a240 addr v1:10.20.193.115:6789/0 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2288074150 con 0x7f228806c890 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228809ef70 con 0x7f228813a240 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f228809f250 con 0x7f228806d380 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f228d149640 10 monclient(hunting): authenticate will time out at 681.220782s 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286d76640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f228806c890 0x7f228806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56188/0 (socket says 10.20.193.78:56188) 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286d76640 1 -- 10.20.193.78:0/2246408312 learned_addr learned my addr 10.20.193.78:0/2246408312 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286575640 1 --2- 10.20.193.78:0/2246408312 >> v2:10.20.193.78:3300/0 conn(0x7f228806d380 0x7f22880a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286575640 10 monclient(hunting): get_auth_request con 0x7f228806d380 auth_method 0 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.666+0000 7f2286575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.667+0000 7f2286575640 10 monclient(hunting): handle_auth_done global_id 4434 payload 274 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.667+0000 7f2286575640 1 -- 10.20.193.78:0/2246408312 >> v1:10.20.193.115:6789/0 conn(0x7f228813a240 legacy=0x7f228813c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.667+0000 7f2286575640 1 -- 10.20.193.78:0/2246408312 >> v1:10.20.193.78:6789/0 conn(0x7f228806c890 legacy=0x7f228806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.667+0000 7f2286575640 10 monclient: _finish_hunting 0 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.667+0000 7f2286575640 1 monclient: found mon.noname-a 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.667+0000 7f2286575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.667+0000 7f2286575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.667+0000 7f2286575640 1 -- 10.20.193.78:0/2246408312 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228809df40 con 0x7f228806d380 2026-02-17T05:47:15.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2286575640 1 -- 10.20.193.78:0/2246408312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228806d380 msgr2=0x7f22880a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2286575640 1 --2- 10.20.193.78:0/2246408312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228806d380 0x7f22880a01f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f22700098a0 tx=0x7f22700310a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4acd13b492cee server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 1 -- 10.20.193.78:0/2246408312 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227003e070 con 0x7f228806d380 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: dump: 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: _finish_auth 0 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: _check_auth_tickets 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 1 -- 10.20.193.78:0/2246408312 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2270039440 con 0x7f228806d380 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 5 monclient: authenticate success, global_id 4434 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 1 -- 10.20.193.78:0/2246408312 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270039810 con 0x7f228806d380 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f2285d74640 10 monclient: dump: 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.671 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 4 set_mon_vals no callback set 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 10 monclient: shutdown 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 1 -- 10.20.193.78:0/2246408312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228806d380 msgr2=0x7f22880a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 1 --2- 10.20.193.78:0/2246408312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228806d380 0x7f22880a01f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f22700098a0 tx=0x7f22700310a0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 1 -- 10.20.193.78:0/2246408312 shutdown_connections 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 1 --2- 10.20.193.78:0/2246408312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228806d380 0x7f22880a01f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 1 -- 10.20.193.78:0/2246408312 >> 10.20.193.78:0/2246408312 conn(0x7f22880a6620 msgr2=0x7f22880a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.669+0000 7f228d149640 1 -- 10.20.193.78:0/2246408312 shutdown_connections 2026-02-17T05:47:15.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 1 -- 10.20.193.78:0/2246408312 wait complete. 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 10 monclient: build_initial_monmap 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 10 monclient: monmap: 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:15.673 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 1 Processor -- start 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 1 -- start start 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 10 monclient: init 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.670+0000 7f228d149640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f228806c890 0x7f228811d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 10 monclient(hunting): picked mon.noname-b con 0x7f228806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228806d380 0x7f228811dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 10 monclient(hunting): picked mon.noname-c con 0x7f228806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228813a240 0x7f228811e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 10 monclient(hunting): picked mon.noname-a con 0x7f228813a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f228809bfa0 con 0x7f228813a240 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f228809be20 con 0x7f228806c890 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f228809c120 con 0x7f228806d380 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f228d149640 10 monclient(hunting): authenticate will time out at 681.225040s 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228813a240 0x7f228811e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228813a240 0x7f228811e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48316/0 (socket says 10.20.193.78:48316) 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228806d380 0x7f228811dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 -- 10.20.193.78:0/2369844333 learned_addr learned my addr 10.20.193.78:0/2369844333 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient(hunting): get_auth_request con 0x7f228813a240 auth_method 0 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286575640 10 monclient(hunting): get_auth_request con 0x7f228806d380 auth_method 0 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286d76640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f228806c890 0x7f228811d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286d76640 10 monclient(hunting): get_auth_request con 0x7f228806c890 auth_method 0 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient(hunting): handle_auth_done global_id 4437 payload 824 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228806d380 msgr2=0x7f228811dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228806d380 0x7f228811dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f228806c890 msgr2=0x7f228811d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f228806c890 0x7f228811d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient: _finish_hunting 0 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 monclient: found mon.noname-a 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.671+0000 7f2287577640 1 -- 10.20.193.78:0/2369844333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228811fa10 con 0x7f228813a240 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f2287577640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228813a240 msgr2=0x7f228811e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f2287577640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228813a240 0x7f228811e1d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f227800cb60 tx=0x7f227800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278016020 con 0x7f228813a240 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: dump: 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.676 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: _finish_auth 0 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: _check_auth_tickets 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2278012420 con 0x7f228813a240 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f228d149640 5 monclient: authenticate success, global_id 4437 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f228d149640 10 monclient: _renew_subs 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f228d149640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2288121420 con 0x7f228813a240 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278012810 con 0x7f228813a240 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f22677fe640 10 monclient: dump: 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f2285573640 4 set_mon_vals no callback set 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f228d149640 10 monclient: _renew_subs 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f228d149640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.673+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22880a2640 con 0x7f228813a240 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f228d149640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f228d149640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288120d50 0x7f2288071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f228d149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2288071890 con 0x7f2288120d50 2026-02-17T05:47:15.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288120d50 0x7f2288071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 10 monclient: get_auth_request con 0x7f2288120d50 auth_method 0 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2278012a30 con 0x7f228813a240 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 10 monclient(hunting): handle_auth_done global_id 4440 payload 824 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286d76640 1 -- 10.20.193.78:0/2369844333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22880008d0 con 0x7f2288120d50 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f22677fe640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225403dee0 0x7f22540403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4807+0+0 (secure 0 0 0) 0x7f22780531b0 con 0x7f228813a240 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286575640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225403dee0 0x7f22540403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.674+0000 7f2286575640 10 monclient: get_auth_request con 0x7f225403dee0 auth_method 0 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f2286d76640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288120d50 msgr2=0x7f2288071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f2286d76640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288120d50 0x7f2288071350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f227c00eaa0 tx=0x7f227c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c00cbf0 con 0x7f2288120d50 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 10 monclient: dump: 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f227c019070 con 0x7f2288120d50 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288120d50 msgr2=0x7f2288071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288120d50 0x7f2288071350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f227c00eaa0 tx=0x7f227c00ef70 comp rx=0 tx=0).stop 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288120d50 msgr2=0x7f2288071350 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.676+0000 7f22677fe640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288120d50 0x7f2288071350 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f227c00eaa0 tx=0x7f227c00ef70 comp rx=0 tx=0).stop 2026-02-17T05:47:15.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.677+0000 7f2286575640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225403dee0 0x7f22540403a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f228806a480 tx=0x7f2270035000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.679+0000 7f228d149640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f228d149640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288063880 0x7f2288063c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f228d149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22880a01d0 con 0x7f2288063880 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288063880 0x7f2288063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 10 monclient: get_auth_request con 0x7f2288063880 auth_method 0 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 10 monclient(hunting): handle_auth_done global_id 4443 payload 824 2026-02-17T05:47:15.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.680+0000 7f2286d76640 1 -- 10.20.193.78:0/2369844333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2288121130 con 0x7f2288063880 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f2286d76640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288063880 msgr2=0x7f2288063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f2286d76640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288063880 0x7f2288063c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f227c00eaa0 tx=0x7f227c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c01e250 con 0x7f2288063880 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 10 monclient: dump: 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f227c01e250 con 0x7f2288063880 2026-02-17T05:47:15.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288063880 msgr2=0x7f2288063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288063880 0x7f2288063c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f227c00eaa0 tx=0x7f227c004a40 comp rx=0 tx=0).stop 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288063880 msgr2=0x7f2288063c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f22677fe640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2288063880 0x7f2288063c60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 12, 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 85486, 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:47:15.684 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:47:15.685 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:15.686 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.687 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:47:15.688 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:47:15.689 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:47:15.690 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.691 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:47:15.692 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225403dee0 msgr2=0x7f22540403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225403dee0 0x7f22540403a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f228806a480 tx=0x7f2270035000 comp rx=0 tx=0).stop 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 10 monclient: shutdown 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228813a240 msgr2=0x7f228811e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228813a240 0x7f228811e1d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f227800cb60 tx=0x7f227800a430 comp rx=0 tx=0).stop 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f2287577640 1 -- 10.20.193.78:0/2369844333 reap_dead start 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 shutdown_connections 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 --2- 10.20.193.78:0/2369844333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228813a240 0x7f228811e1d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 >> 10.20.193.78:0/2369844333 conn(0x7f22880a6620 msgr2=0x7f228806eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 shutdown_connections 2026-02-17T05:47:15.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.682+0000 7f228d149640 1 -- 10.20.193.78:0/2369844333 wait complete. 2026-02-17T05:47:15.694 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:47:15.694 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:47:15.694 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:47:15.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient: get_monmap_and_config 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient: build_initial_monmap 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient: monmap: 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:15.761483+0000 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:15.761483+0000 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:15.764 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 1 Processor -- start 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 1 -- start start 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient: init 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806c890 0x7f184806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient(hunting): picked mon.noname-b con 0x7f184806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f184809c4a0 0x7f184806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient(hunting): picked mon.noname-a con 0x7f184809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f1848134df0 0x7f1848135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient(hunting): picked mon.noname-c con 0x7f1848134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f18480a0880 con 0x7f184809c4a0 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f184806d6f0 con 0x7f1848134df0 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f184806daf0 con 0x7f184806c890 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:15.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.760+0000 7f184df61640 10 monclient(hunting): authenticate will time out at 681.314983s 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f184809c4a0 0x7f184806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f18477fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806c890 0x7f184806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f184809c4a0 0x7f184806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48342/0 (socket says 10.20.193.78:48342) 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 -- 10.20.193.78:0/1089064920 learned_addr learned my addr 10.20.193.78:0/1089064920 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient(hunting): get_auth_request con 0x7f184809c4a0 auth_method 0 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f18477fe640 10 monclient(hunting): get_auth_request con 0x7f184806c890 auth_method 0 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f18477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f18477fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f18477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1847fff640 1 --2- 10.20.193.78:0/1089064920 >> v2:10.20.193.115:3300/0 conn(0x7f1848134df0 0x7f1848135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1847fff640 10 monclient(hunting): get_auth_request con 0x7f1848134df0 auth_method 0 2026-02-17T05:47:15.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1847fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1847fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1847fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f18477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f18477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f18477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient(hunting): handle_auth_done global_id 4446 payload 274 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 -- 10.20.193.78:0/1089064920 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806c890 msgr2=0x7f184806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 --2- 10.20.193.78:0/1089064920 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806c890 0x7f184806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 -- 10.20.193.78:0/1089064920 >> v2:10.20.193.115:3300/0 conn(0x7f1848134df0 msgr2=0x7f1848135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 --2- 10.20.193.78:0/1089064920 >> v2:10.20.193.115:3300/0 conn(0x7f1848134df0 0x7f1848135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 monclient: found mon.noname-a 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.761+0000 7f1846ffd640 1 -- 10.20.193.78:0/1089064920 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184809cc80 con 0x7f184809c4a0 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f1846ffd640 1 -- 10.20.193.78:0/1089064920 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184809c4a0 msgr2=0x7f184806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f1846ffd640 1 --2- 10.20.193.78:0/1089064920 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184809c4a0 0x7f184806d1b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f184809c880 tx=0x7f183802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908ee2d46762aec0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 1 -- 10.20.193.78:0/1089064920 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183803c070 con 0x7f184809c4a0 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: dump: 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.773 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: _finish_auth 0 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: _check_auth_tickets 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 1 -- 10.20.193.78:0/1089064920 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183802fca0 con 0x7f184809c4a0 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 1 -- 10.20.193.78:0/1089064920 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838037660 con 0x7f184809c4a0 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f184df61640 5 monclient: authenticate success, global_id 4446 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f184df61640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f184df61640 4 set_mon_vals no callback set 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.763+0000 7f18467fc640 10 monclient: dump: 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.774 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 10 monclient: shutdown 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 -- 10.20.193.78:0/1089064920 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184809c4a0 msgr2=0x7f184806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 --2- 10.20.193.78:0/1089064920 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184809c4a0 0x7f184806d1b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f184809c880 tx=0x7f183802ee60 comp rx=0 tx=0).stop 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 -- 10.20.193.78:0/1089064920 shutdown_connections 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 --2- 10.20.193.78:0/1089064920 >> v2:10.20.193.115:3300/0 conn(0x7f1848134df0 0x7f1848135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 --2- 10.20.193.78:0/1089064920 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184809c4a0 0x7f184806d1b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 --2- 10.20.193.78:0/1089064920 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806c890 0x7f184806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 -- 10.20.193.78:0/1089064920 >> 10.20.193.78:0/1089064920 conn(0x7f18480a6620 msgr2=0x7f18480a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 -- 10.20.193.78:0/1089064920 shutdown_connections 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 1 -- 10.20.193.78:0/1089064920 wait complete. 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 10 monclient: build_initial_monmap 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.764+0000 7f184df61640 10 monclient: monmap: 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:15.775 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 1 Processor -- start 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 1 -- start start 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient: init 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184806c890 0x7f18481cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient(hunting): picked mon.noname-a con 0x7f184806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f184809c4a0 0x7f18481cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient(hunting): picked mon.noname-b con 0x7f184809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:15.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1848134df0 0x7f1848122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient(hunting): picked mon.noname-c con 0x7f1848134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18480a01a0 con 0x7f184806c890 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18480a0020 con 0x7f184809c4a0 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f184809be20 con 0x7f1848134df0 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f184df61640 10 monclient(hunting): authenticate will time out at 681.319324s 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184806c890 0x7f18481cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184806c890 0x7f18481cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48354/0 (socket says 10.20.193.78:48354) 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1848134df0 0x7f1848122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 -- 10.20.193.78:0/2029455584 learned_addr learned my addr 10.20.193.78:0/2029455584 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient(hunting): get_auth_request con 0x7f184806c890 auth_method 0 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 10 monclient(hunting): get_auth_request con 0x7f1848134df0 auth_method 0 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1846ffd640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f184809c4a0 0x7f18481cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient(hunting): handle_auth_done global_id 4449 payload 824 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1848134df0 msgr2=0x7f1848122bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1848134df0 0x7f1848122bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f184809c4a0 msgr2=0x7f18481cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f184809c4a0 0x7f18481cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient: _finish_hunting 0 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 monclient: found mon.noname-a 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:15.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f18477fe640 1 -- 10.20.193.78:0/2029455584 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1848126ac0 con 0x7f184806c890 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.765+0000 7f1847fff640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1848134df0 0x7f1848122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f18477fe640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184806c890 msgr2=0x7f18481cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f18477fe640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184806c890 0x7f18481cf350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f183400ebe0 tx=0x7f183400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834017070 con 0x7f184806c890 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: dump: 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: _finish_auth 0 2026-02-17T05:47:15.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: _check_auth_tickets 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183400ce50 con 0x7f184806c890 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18340136b0 con 0x7f184806c890 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1823fff640 10 monclient: dump: 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f1845ffb640 4 set_mon_vals no callback set 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f184df61640 5 monclient: authenticate success, global_id 4449 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f184df61640 10 monclient: _renew_subs 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f184df61640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18480a1610 con 0x7f184806c890 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f184df61640 10 monclient: _renew_subs 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f184df61640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.767+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18480725b0 con 0x7f184806c890 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f184df61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f184df61640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18481265c0 0x7f18481238b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f184df61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1848074980 con 0x7f18481265c0 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18481265c0 0x7f18481238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 10 monclient: get_auth_request con 0x7f18481265c0 auth_method 0 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 10 monclient(hunting): handle_auth_done global_id 4421 payload 824 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1834024060 con 0x7f184806c890 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1846ffd640 1 -- 10.20.193.78:0/2029455584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1848135240 con 0x7f18481265c0 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1823fff640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f181803deb0 0x7f1818040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4807+0+0 (secure 0 0 0) 0x7f18340529b0 con 0x7f184806c890 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1847fff640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f181803deb0 0x7f1818040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.768+0000 7f1847fff640 10 monclient: get_auth_request con 0x7f181803deb0 auth_method 0 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1846ffd640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18481265c0 msgr2=0x7f18481238b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1846ffd640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18481265c0 0x7f18481238b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18480a7600 tx=0x7f1838004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838046070 con 0x7f18481265c0 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 10 monclient: dump: 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.782 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f183802fe50 con 0x7f18481265c0 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18481265c0 msgr2=0x7f18481238b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18481265c0 0x7f18481238b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18480a7600 tx=0x7f1838004830 comp rx=0 tx=0).stop 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18481265c0 msgr2=0x7f18481238b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.770+0000 7f1823fff640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18481265c0 0x7f18481238b0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18480a7600 tx=0x7f1838004830 comp rx=0 tx=0).stop 2026-02-17T05:47:15.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.771+0000 7f1847fff640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f181803deb0 0x7f1818040370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f183c009fd0 tx=0x7f183c009770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f184df61640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f184df61640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806f410 0x7f18481da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f184df61640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f18481250b0 con 0x7f184806f410 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806f410 0x7f18481da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 10 monclient: get_auth_request con 0x7f184806f410 auth_method 0 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 10 monclient(hunting): handle_auth_done global_id 4424 payload 824 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.774+0000 7f1846ffd640 1 -- 10.20.193.78:0/2029455584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f18480744d0 con 0x7f184806f410 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1846ffd640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806f410 msgr2=0x7f18481da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1846ffd640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806f410 0x7f18481da560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f183802eea0 tx=0x7f18380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183803c040 con 0x7f184806f410 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 10 monclient: dump: 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4707 (secure 0 0 0) 0x7f183804bad0 con 0x7f184806f410 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:15.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806f410 msgr2=0x7f18481da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806f410 0x7f18481da560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f183802eea0 tx=0x7f18380047c0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806f410 msgr2=0x7f18481da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1823fff640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f184806f410 0x7f18481da560 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f183802eea0 tx=0x7f18380047c0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 12, 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 85581, 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:47:15.787 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.788 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:47:15.789 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.790 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.791 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.792 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.793 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:47:15.794 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.796 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:47:15.796 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f181803deb0 msgr2=0x7f1818040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f184df61640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f181803deb0 0x7f1818040370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f183c009fd0 tx=0x7f183c009770 comp rx=0 tx=0).stop 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f184df61640 10 monclient: shutdown 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f184df61640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184806c890 msgr2=0x7f18481cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f184df61640 1 --2- 10.20.193.78:0/2029455584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f184806c890 0x7f18481cf350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f183400ebe0 tx=0x7f183400c520 comp rx=0 tx=0).stop 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1847fff640 1 -- 10.20.193.78:0/2029455584 reap_dead start 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.776+0000 7f1847fff640 1 -- 10.20.193.78:0/2029455584 reap_dead start 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.777+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 shutdown_connections 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.777+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 >> 10.20.193.78:0/2029455584 conn(0x7f18480a6620 msgr2=0x7f18480a2820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.777+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 shutdown_connections 2026-02-17T05:47:15.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.777+0000 7f184df61640 1 -- 10.20.193.78:0/2029455584 wait complete. 2026-02-17T05:47:15.797 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:47:15.797 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:47:15.798 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:47:15.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.863+0000 7effa676a640 10 monclient: get_monmap_and_config 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.863+0000 7effa676a640 10 monclient: build_initial_monmap 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.863+0000 7effa676a640 10 monclient: monmap: 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:15.864989+0000 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:15.864989+0000 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:15.865 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 1 Processor -- start 2026-02-17T05:47:15.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 1 -- start start 2026-02-17T05:47:15.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient: init 2026-02-17T05:47:15.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:15.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2026-02-17T05:47:15.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7effa006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:47:15.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7effa006d380 0x7effa00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient(hunting): picked mon.noname-a con 0x7effa006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7effa013a240 addr v1:10.20.193.78:6789/0 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa0074150 con 0x7effa013a240 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa009ef70 con 0x7effa006c890 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7effa009f250 con 0x7effa006d380 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa676a640 10 monclient(hunting): authenticate will time out at 681.418526s 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7effa006d380 0x7effa00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7effa006d380 0x7effa00a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48358/0 (socket says 10.20.193.78:48358) 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 1 -- 10.20.193.78:0/2025694501 learned_addr learned my addr 10.20.193.78:0/2025694501 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient(hunting): get_auth_request con 0x7effa006d380 auth_method 0 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7effa4ce0640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7effa013a240 0x7effa013c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56194/0 (socket says 10.20.193.78:56194) 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_done global_id 4452 payload 274 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 1 -- 10.20.193.78:0/2025694501 >> v1:10.20.193.115:6789/0 conn(0x7effa006c890 legacy=0x7effa006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 1 -- 10.20.193.78:0/2025694501 >> v1:10.20.193.78:6789/0 conn(0x7effa013a240 legacy=0x7effa013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 1 monclient: found mon.noname-a 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.864+0000 7eff9f7fe640 1 -- 10.20.193.78:0/2025694501 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa009df40 con 0x7effa006d380 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9f7fe640 1 -- 10.20.193.78:0/2025694501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006d380 msgr2=0x7effa00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9f7fe640 1 --2- 10.20.193.78:0/2025694501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006d380 0x7effa00a01f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff90009970 tx=0x7eff9002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9be2b8f0145900d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 1 -- 10.20.193.78:0/2025694501 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9003c070 con 0x7effa006d380 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: dump: 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.874 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: _finish_auth 0 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: _check_auth_tickets 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 1 -- 10.20.193.78:0/2025694501 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9002fe50 con 0x7effa006d380 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 1 -- 10.20.193.78:0/2025694501 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90037660 con 0x7effa006d380 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7eff9effd640 10 monclient: dump: 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.875 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 5 monclient: authenticate success, global_id 4452 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 4 set_mon_vals no callback set 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 10 monclient: shutdown 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 1 -- 10.20.193.78:0/2025694501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006d380 msgr2=0x7effa00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 1 --2- 10.20.193.78:0/2025694501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006d380 0x7effa00a01f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff90009970 tx=0x7eff9002eef0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 1 -- 10.20.193.78:0/2025694501 shutdown_connections 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 1 --2- 10.20.193.78:0/2025694501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006d380 0x7effa00a01f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 1 -- 10.20.193.78:0/2025694501 >> 10.20.193.78:0/2025694501 conn(0x7effa00a6620 msgr2=0x7effa00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 1 -- 10.20.193.78:0/2025694501 shutdown_connections 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.867+0000 7effa676a640 1 -- 10.20.193.78:0/2025694501 wait complete. 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient: build_initial_monmap 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient: monmap: 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 1 Processor -- start 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 1 -- start start 2026-02-17T05:47:15.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient: init 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006c890 0x7effa0134b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient(hunting): picked mon.noname-a con 0x7effa006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006d380 0x7effa0135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient(hunting): picked mon.noname-b con 0x7effa006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7effa013a240 0x7effa0121970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient(hunting): picked mon.noname-c con 0x7effa013a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7effa009bfa0 con 0x7effa006c890 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7effa009be20 con 0x7effa006d380 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7effa009c120 con 0x7effa013a240 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa676a640 10 monclient(hunting): authenticate will time out at 681.422617s 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006c890 0x7effa0134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006c890 0x7effa0134b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48364/0 (socket says 10.20.193.78:48364) 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 -- 10.20.193.78:0/3889351214 learned_addr learned my addr 10.20.193.78:0/3889351214 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient(hunting): get_auth_request con 0x7effa006c890 auth_method 0 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa4ce0640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7effa013a240 0x7effa0121970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa4ce0640 10 monclient(hunting): get_auth_request con 0x7effa013a240 auth_method 0 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa4ce0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa4ce0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa4ce0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa4ce0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa4ce0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7effa4ce0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9f7fe640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006d380 0x7effa0135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient(hunting): handle_auth_done global_id 4455 payload 824 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7effa013a240 msgr2=0x7effa0121970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7effa013a240 0x7effa0121970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006d380 msgr2=0x7effa0135080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006d380 0x7effa0135080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 monclient: found mon.noname-a 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9ffff640 1 -- 10.20.193.78:0/3889351214 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa0124130 con 0x7effa006c890 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9f7fe640 10 monclient: get_auth_request con 0x7effa006d380 auth_method 0 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.868+0000 7eff9f7fe640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006d380 0x7effa0135080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.870+0000 7eff9ffff640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006c890 msgr2=0x7effa0134b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.870+0000 7eff9ffff640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006c890 0x7effa0134b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c5d0 tx=0x7eff8c00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c016020 con 0x7effa006c890 2026-02-17T05:47:15.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: dump: 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: _finish_auth 0 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: _check_auth_tickets 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8c012a60 con 0x7effa006c890 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 5 monclient: authenticate success, global_id 4455 2026-02-17T05:47:15.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 10 monclient: _renew_subs 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01255e0 con 0x7effa006c890 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c012e50 con 0x7effa006c890 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9e7fc640 4 set_mon_vals no callback set 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 10 monclient: dump: 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.882 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 10 monclient: _renew_subs 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa00a2640 con 0x7effa006c890 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa0122670 0x7effa0122a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7effa00008d0 con 0x7effa0122670 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7eff8c01b400 con 0x7effa006c890 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9f7fe640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa0122670 0x7effa0122a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9f7fe640 10 monclient: get_auth_request con 0x7effa0122670 auth_method 0 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.871+0000 7eff9cff9640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7eff7003e0a0 0x7eff70040560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.872+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4807+0+0 (secure 0 0 0) 0x7eff8c052840 con 0x7effa006c890 2026-02-17T05:47:15.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.872+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_done global_id 34135 payload 824 2026-02-17T05:47:15.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.872+0000 7eff9f7fe640 1 -- 10.20.193.78:0/3889351214 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effa0125820 con 0x7effa0122670 2026-02-17T05:47:15.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.872+0000 7effa4ce0640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7eff7003e0a0 0x7eff70040560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.872+0000 7effa4ce0640 10 monclient: get_auth_request con 0x7eff7003e0a0 auth_method 0 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9f7fe640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa0122670 msgr2=0x7effa0122a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9f7fe640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa0122670 0x7effa0122a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7effa0069d40 tx=0x7eff900047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90046070 con 0x7effa0122670 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 10 monclient: dump: 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.886 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7eff90037d50 con 0x7effa0122670 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa0122670 msgr2=0x7effa0122a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa0122670 0x7effa0122a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7effa0069d40 tx=0x7eff900047c0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa0122670 msgr2=0x7effa0122a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7eff9cff9640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa0122670 0x7effa0122a50 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7effa0069d40 tx=0x7eff900047c0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.874+0000 7effa4ce0640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7eff7003e0a0 0x7eff70040560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effa00a23f0 tx=0x7eff94005380 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.877+0000 7effa676a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.877+0000 7effa676a640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006fac0 0x7effa01da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.877+0000 7effa676a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:15.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.877+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7effa00a01d0 con 0x7effa006fac0 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006fac0 0x7effa01da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 10 monclient: get_auth_request con 0x7effa006fac0 auth_method 0 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 10 monclient(hunting): handle_auth_done global_id 34138 payload 824 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.878+0000 7eff9f7fe640 1 -- 10.20.193.78:0/3889351214 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7effa0125020 con 0x7effa006fac0 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.880+0000 7eff9f7fe640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006fac0 msgr2=0x7effa01da7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.880+0000 7eff9f7fe640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006fac0 0x7effa01da7d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7eff90006d80 tx=0x7eff90009340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.880+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9003c040 con 0x7effa006fac0 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.880+0000 7eff9cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.880+0000 7eff9cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.880+0000 7eff9cff9640 10 monclient: dump: 2026-02-17T05:47:15.888 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7eff90041670 con 0x7effa006fac0 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7eff9cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7eff9cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006fac0 msgr2=0x7effa01da7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7eff9cff9640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006fac0 0x7effa01da7d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7eff90006d80 tx=0x7eff90009340 comp rx=0 tx=0).stop 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7eff9cff9640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006fac0 msgr2=0x7effa01da7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:15.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7eff9cff9640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7effa006fac0 0x7effa01da7d0 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7eff90006d80 tx=0x7eff90009340 comp rx=0 tx=0).stop 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 12, 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 17897, 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:47:15.890 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:47:15.891 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:47:15.892 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:15.893 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.894 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.895 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.896 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:15.897 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:15.898 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:15.899 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:47:15.899 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7eff7003e0a0 msgr2=0x7eff70040560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7eff7003e0a0 0x7eff70040560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effa00a23f0 tx=0x7eff94005380 comp rx=0 tx=0).stop 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 10 monclient: shutdown 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006c890 msgr2=0x7effa0134b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006c890 0x7effa0134b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c5d0 tx=0x7eff8c00caa0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa4ce0640 1 -- 10.20.193.78:0/3889351214 reap_dead start 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 shutdown_connections 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 --2- 10.20.193.78:0/3889351214 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7effa006c890 0x7effa0134b40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 >> 10.20.193.78:0/3889351214 conn(0x7effa00a6620 msgr2=0x7effa0138c50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 shutdown_connections 2026-02-17T05:47:15.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:15.881+0000 7effa676a640 1 -- 10.20.193.78:0/3889351214 wait complete. 2026-02-17T05:47:15.901 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:47:15.901 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T05:47:15.901 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2026-02-17T05:47:15.901 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T05:47:15.901 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T05:47:15.901 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T05:47:15.901 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:47:15.909 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T05:47:15.909 INFO:tasks.mon_thrash:Sending STOP to mon a 2026-02-17T05:47:15.909 INFO:tasks.ceph.mon.a:Sent signal 19 2026-02-17T05:47:15.909 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T05:47:30.910 INFO:tasks.mon_thrash:Sending CONT to mon a 2026-02-17T05:47:30.911 INFO:tasks.ceph.mon.a:Sent signal 18 2026-02-17T05:47:30.911 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:47:30.911 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:47:30.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.978+0000 7f31a8a29640 10 monclient: get_monmap_and_config 2026-02-17T05:47:30.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.978+0000 7f31a8a29640 10 monclient: build_initial_monmap 2026-02-17T05:47:30.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.978+0000 7f31a8a29640 10 monclient: monmap: 2026-02-17T05:47:30.982 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:30.982 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:30.982 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:30.979777+0000 2026-02-17T05:47:30.982 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:30.979777+0000 2026-02-17T05:47:30.982 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.978+0000 7f31a8a29640 1 Processor -- start 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 1 -- start start 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient: init 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:30.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 0x7f31a406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient(hunting): picked mon.noname-c con 0x7f31a406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f31a409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient(hunting): picked mon.noname-b con 0x7f31a406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a406daf0 con 0x7f31a409c4a0 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f31a40a0880 con 0x7f31a406c890 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f31a409cc80 con 0x7f31a406d1b0 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:30.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a8a29640 10 monclient(hunting): authenticate will time out at 696.533367s 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a1d74640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f31a409c4a0 legacy=0x7f31a4134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2575640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 msgr2=0x7f31a406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2575640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 0x7f31a406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37720/0 (socket says 10.20.193.78:37720) 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 learned_addr learned my addr 10.20.193.78:0/217226718 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 10 monclient(hunting): get_auth_request con 0x7f31a406d1b0 auth_method 0 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:30.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:30.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:47:30.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:30.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:30.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:30.979+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 msgr2=0x7f31a406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a1d74640 1 -- 10.20.193.78:0/217226718 >> v1:10.20.193.115:6789/0 conn(0x7f31a409c4a0 legacy=0x7f31a4134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 0x7f31a406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 10 monclient(hunting): get_auth_request con 0x7f31a406d1b0 auth_method 0 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:31.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.180+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:47:31.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 msgr2=0x7f31a406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:31.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a1d74640 1 -- 10.20.193.78:0/217226718 >> v1:10.20.193.115:6789/0 conn(0x7f31a409c4a0 legacy=0x7f31a4134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:47:31.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 0x7f31a406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:47:31.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:31.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 10 monclient(hunting): get_auth_request con 0x7f31a406d1b0 auth_method 0 2026-02-17T05:47:31.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:31.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:31.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:31.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:47:31.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:31.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:31.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:31.580+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:47:32.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 msgr2=0x7f31a406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:32.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a1d74640 1 -- 10.20.193.78:0/217226718 >> v1:10.20.193.115:6789/0 conn(0x7f31a409c4a0 legacy=0x7f31a4134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:47:32.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 0x7f31a406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:47:32.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:32.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 10 monclient(hunting): get_auth_request con 0x7f31a406d1b0 auth_method 0 2026-02-17T05:47:32.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:32.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:32.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:32.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:47:32.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:32.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:32.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:32.381+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:47:33.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 10 monclient(hunting): tick 2026-02-17T05:47:33.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:47:33.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 monclient(hunting): continuing hunt 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 msgr2=0x7f31a4135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a406d1b0 0x7f31a4135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 msgr2=0x7f31a406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.115:3300/0 conn(0x7f31a406c890 0x7f31a406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 -- 10.20.193.78:0/217226718 >> v1:10.20.193.115:6789/0 conn(0x7f31a409c4a0 legacy=0x7f31a4134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 10 monclient: _add_conns ranks=[3,0] 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 10 monclient(hunting): picked mon.noname-a con 0x7f31800037a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3180006310 addr v1:10.20.193.78:6789/0 2026-02-17T05:47:33.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 -- 10.20.193.78:0/217226718 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a409cf60 con 0x7f3180006310 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 1 -- 10.20.193.78:0/217226718 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f31a409cc80 con 0x7f31800037a0 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f318bfff640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 10 monclient(hunting): get_auth_request con 0x7f31800037a0 auth_method 0 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:33.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.980+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:33.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:33.981+0000 7f31a1d74640 1 -- 10.20.193.78:0/217226718 >> v1:10.20.193.78:6789/0 conn(0x7f3180006310 legacy=0x7f31800087d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:34.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:34.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 10 monclient(hunting): get_auth_request con 0x7f31800037a0 auth_method 0 2026-02-17T05:47:34.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:34.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:34.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:34.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:47:34.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:34.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:34.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.180+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:47:34.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:34.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 10 monclient(hunting): get_auth_request con 0x7f31800037a0 auth_method 0 2026-02-17T05:47:34.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:34.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:34.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:34.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:47:34.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:34.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:34.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:34.581+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:47:35.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:35.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 10 monclient(hunting): get_auth_request con 0x7f31800037a0 auth_method 0 2026-02-17T05:47:35.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:35.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:35.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:35.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:47:35.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:47:35.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:47:35.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.382+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:47:35.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.918+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731805134 0 0) 0x7f31a409cf60 con 0x7f3180006310 2026-02-17T05:47:35.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.918+0000 7f31a1573640 10 monclient(hunting): _negotiate 2026-02-17T05:47:35.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.918+0000 7f31a1573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:35.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.918+0000 7f31a1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:35.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.918+0000 7f31a1573640 10 monclient(hunting): my global_id is 4458 2026-02-17T05:47:35.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.918+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3190003620 con 0x7f3180006310 2026-02-17T05:47:35.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3543726895 0 0) 0x7f3190003620 con 0x7f3180006310 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 msgr2=0x7f3180005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 --2- 10.20.193.78:0/217226718 >> v2:10.20.193.78:3300/0 conn(0x7f31800037a0 0x7f3180005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: _finish_hunting 0 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a406daf0 con 0x7f3180006310 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: _finish_auth 0 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: _check_auth_tickets 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f318c0044f0 con 0x7f3180006310 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: dump: 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:35.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 >> v1:10.20.193.78:6789/0 conn(0x7f3180006310 legacy=0x7f31800087d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 1 -- 10.20.193.78:0/217226718 reap_dead start 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190003ab0 0x7f3190003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient(hunting): picked mon.b con 0x7f3190003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190004900 0x7f3190004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient(hunting): picked mon.a con 0x7f3190004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190005390 0x7f319000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient(hunting): picked mon.c con 0x7f3190005390 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:35.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a409cc80 con 0x7f3190004900 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f319000c2e0 con 0x7f3190003ab0 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f319000c4e0 con 0x7f3190005390 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1573640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190004900 0x7f3190004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2575640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190003ab0 msgr2=0x7f3190003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2575640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190003ab0 0x7f3190003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient(hunting): get_auth_request con 0x7f3190004900 auth_method 0 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:35.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190005390 0x7f319000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 10 monclient(hunting): get_auth_request con 0x7f3190005390 auth_method 0 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:35.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a2d76640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient(hunting): handle_auth_done global_id 4458 payload 293 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190005390 msgr2=0x7f319000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190005390 0x7f319000bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190003ab0 msgr2=0x7f3190003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190003ab0 0x7f3190003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient: _finish_hunting 0 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 1 monclient: found mon.a 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 1 -- 10.20.193.78:0/217226718 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3190003620 con 0x7f3190004900 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient: _finish_auth 0 2026-02-17T05:47:35.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 10 monclient: _check_auth_tickets 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a1d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a8a29640 5 monclient: authenticate success, global_id 4458 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.920+0000 7f31a8a29640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1d74640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190004900 msgr2=0x7f3190004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1d74640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190004900 0x7f3190004d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f318c003a30 tx=0x7f318c02f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe01c3c853de095 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00c120 con 0x7f3190004900 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 10 monclient: dump: 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:35.933 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c006270 con 0x7f3190004900 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:35.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 1 -- 10.20.193.78:0/217226718 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c004790 con 0x7f3190004900 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a1573640 10 monclient: dump: 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:35.935 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 4 set_mon_vals no callback set 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 10 monclient: shutdown 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 -- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190004900 msgr2=0x7f3190004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190004900 0x7f3190004d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f318c003a30 tx=0x7f318c02f6f0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 -- 10.20.193.78:0/217226718 shutdown_connections 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190005390 0x7f319000bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190004900 0x7f3190004d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 --2- 10.20.193.78:0/217226718 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190003ab0 0x7f3190003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 -- 10.20.193.78:0/217226718 >> 10.20.193.78:0/217226718 conn(0x7f31a40a6620 msgr2=0x7f31a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 -- 10.20.193.78:0/217226718 shutdown_connections 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.923+0000 7f31a8a29640 1 -- 10.20.193.78:0/217226718 wait complete. 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient: build_initial_monmap 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient: monmap: 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:35.937 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 1 Processor -- start 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 1 -- start start 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient: init 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190003ab0 0x7f31a409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient(hunting): picked mon.noname-c con 0x7f3190003ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190004900 0x7f31a409e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient(hunting): picked mon.noname-b con 0x7f3190004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:35.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190005390 0x7f31a409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient(hunting): picked mon.noname-a con 0x7f3190005390 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40a01a0 con 0x7f3190005390 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40a0020 con 0x7f3190004900 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f31a409be20 con 0x7f3190003ab0 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a8a29640 10 monclient(hunting): authenticate will time out at 701.478821s 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190003ab0 0x7f31a409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:35.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190005390 0x7f31a409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190005390 0x7f31a409e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35056/0 (socket says 10.20.193.78:35056) 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190003ab0 0x7f31a409db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37770/0 (socket says 10.20.193.78:37770) 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 1 -- 10.20.193.78:0/4101006125 learned_addr learned my addr 10.20.193.78:0/4101006125 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a1d74640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190004900 msgr2=0x7f31a409e090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 10 monclient(hunting): get_auth_request con 0x7f3190005390 auth_method 0 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 10 monclient(hunting): get_auth_request con 0x7f3190003ab0 auth_method 0 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:35.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a1d74640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190004900 0x7f31a409e090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 10 monclient(hunting): handle_auth_done global_id 4461 payload 824 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 1 -- 10.20.193.78:0/4101006125 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190003ab0 msgr2=0x7f31a409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190003ab0 0x7f31a409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 1 -- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190004900 msgr2=0x7f31a409e090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.924+0000 7f31a2d76640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190004900 0x7f31a409e090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:35.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.925+0000 7f31a2d76640 10 monclient: _finish_hunting 0 2026-02-17T05:47:35.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.925+0000 7f31a2d76640 1 monclient: found mon.noname-a 2026-02-17T05:47:35.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.925+0000 7f31a2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:35.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.925+0000 7f31a2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:35.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.925+0000 7f31a2d76640 1 -- 10.20.193.78:0/4101006125 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a409d810 con 0x7f3190005390 2026-02-17T05:47:35.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a2d76640 1 -- 10.20.193.78:0/4101006125 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190005390 msgr2=0x7f31a409e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:35.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a2d76640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190005390 0x7f31a409e5d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f31940096d0 tx=0x7f3194015d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:35.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 1 -- 10.20.193.78:0/4101006125 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319401c070 con 0x7f3190005390 2026-02-17T05:47:35.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:35.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: dump: 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:35.945 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: _finish_auth 0 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: _check_auth_tickets 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 1 -- 10.20.193.78:0/4101006125 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3194013810 con 0x7f3190005390 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 1 -- 10.20.193.78:0/4101006125 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940092d0 con 0x7f3190005390 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f318affd640 10 monclient: dump: 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:35.946 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a0d72640 4 set_mon_vals no callback set 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a8a29640 5 monclient: authenticate success, global_id 4461 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a8a29640 10 monclient: _renew_subs 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a8a29640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:35.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a4128160 con 0x7f3190005390 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a8a29640 10 monclient: _renew_subs 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a8a29640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.927+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a41278c0 con 0x7f3190005390 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f318affd640 1 -- 10.20.193.78:0/4101006125 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3194029060 con 0x7f3190005390 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f31a8a29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f31a8a29640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f31a8a29640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a4074980 con 0x7f3190005390 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f318affd640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f317003de90 0x7f3170040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f318affd640 1 -- 10.20.193.78:0/4101006125 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4751+0+0 (secure 0 0 0) 0x7f31940531c0 con 0x7f3190005390 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f31a2575640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f317003de90 0x7f3170040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.928+0000 7f31a2575640 10 monclient: get_auth_request con 0x7f317003de90 auth_method 0 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.930+0000 7f318affd640 1 -- 10.20.193.78:0/4101006125 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f31a409d810 con 0x7f3190005390 2026-02-17T05:47:35.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.930+0000 7f318affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:35.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.930+0000 7f318affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:35.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:35.931+0000 7f31a2575640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f317003de90 0x7f3170040350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3198003a30 tx=0x7f319800d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.045+0000 7f31a8a29640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:47:36.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.045+0000 7f31a8a29640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:47:36.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.045+0000 7f31a8a29640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:36.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.045+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f31a4127c60 con 0x7f3190005390 2026-02-17T05:47:36.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.045+0000 7f318affd640 1 -- 10.20.193.78:0/4101006125 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f3194051be0 con 0x7f3190005390 2026-02-17T05:47:36.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.045+0000 7f318affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:47:36.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.045+0000 7f318affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:36.047 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":16,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:47:36.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f317003de90 msgr2=0x7f3170040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f317003de90 0x7f3170040350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3198003a30 tx=0x7f319800d040 comp rx=0 tx=0).stop 2026-02-17T05:47:36.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 10 monclient: shutdown 2026-02-17T05:47:36.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:36.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190005390 msgr2=0x7f31a409e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190005390 0x7f31a409e5d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f31940096d0 tx=0x7f3194015d10 comp rx=0 tx=0).stop 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 shutdown_connections 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3190005390 0x7f31a409e5d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f317003de90 0x7f3170040350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3190004900 0x7f31a409e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 --2- 10.20.193.78:0/4101006125 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3190003ab0 0x7f31a409db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 >> 10.20.193.78:0/4101006125 conn(0x7f31a40a6620 msgr2=0x7f31a4136030 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 shutdown_connections 2026-02-17T05:47:36.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.046+0000 7f31a8a29640 1 -- 10.20.193.78:0/4101006125 wait complete. 2026-02-17T05:47:36.056 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:47:36.056 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T05:47:36.056 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:47:36.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.122+0000 7f92f7fde640 10 monclient: get_monmap_and_config 2026-02-17T05:47:36.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.122+0000 7f92f7fde640 10 monclient: build_initial_monmap 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.122+0000 7f92f7fde640 10 monclient: monmap: 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:36.124179+0000 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:36.124179+0000 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:36.125 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:36.126 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 1 Processor -- start 2026-02-17T05:47:36.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 1 -- start start 2026-02-17T05:47:36.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient: init 2026-02-17T05:47:36.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:36.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f92f006c890 0x7f92f006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient(hunting): picked mon.noname-c con 0x7f92f006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f92f009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f92f006d1b0 0x7f92f0135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient(hunting): picked mon.noname-a con 0x7f92f006d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f006daf0 con 0x7f92f009c4a0 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f92f00a0880 con 0x7f92f006d1b0 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f92f009cc80 con 0x7f92f006c890 2026-02-17T05:47:36.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f7fde640 10 monclient(hunting): authenticate will time out at 701.677754s 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f5d53640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f92f006c890 msgr2=0x7f92f006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f5d53640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f92f006c890 0x7f92f006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f6554640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f92f006d1b0 0x7f92f0135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f5552640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f92f009c4a0 0x7f92f0134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50784/0 (socket says 10.20.193.78:50784) 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f5552640 1 -- 10.20.193.78:0/2149334888 learned_addr learned my addr 10.20.193.78:0/2149334888 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f6554640 10 monclient(hunting): get_auth_request con 0x7f92f006d1b0 auth_method 0 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f6554640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f6554640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f6554640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f6554640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f6554640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.123+0000 7f92f6554640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 10 monclient(hunting): handle_auth_done global_id 4464 payload 274 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 1 -- 10.20.193.78:0/2149334888 >> v2:10.20.193.115:3300/0 conn(0x7f92f006c890 msgr2=0x7f92f006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 1 --2- 10.20.193.78:0/2149334888 >> v2:10.20.193.115:3300/0 conn(0x7f92f006c890 0x7f92f006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 1 -- 10.20.193.78:0/2149334888 >> v1:10.20.193.78:6789/0 conn(0x7f92f009c4a0 legacy=0x7f92f0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 10 monclient: _finish_hunting 0 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 1 monclient: found mon.noname-a 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.124+0000 7f92f6554640 1 -- 10.20.193.78:0/2149334888 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f009cf60 con 0x7f92f006d1b0 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f6554640 1 -- 10.20.193.78:0/2149334888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 msgr2=0x7f92f0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:36.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f6554640 1 --2- 10.20.193.78:0/2149334888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f0135500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92f006a480 tx=0x7f92ec02f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74be9c7549679f8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 1 -- 10.20.193.78:0/2149334888 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec03c070 con 0x7f92f006d1b0 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: dump: 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.140 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: _finish_auth 0 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: _check_auth_tickets 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f7fde640 5 monclient: authenticate success, global_id 4464 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 1 -- 10.20.193.78:0/2149334888 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec037640 con 0x7f92f006d1b0 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f7fde640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 1 -- 10.20.193.78:0/2149334888 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec037a10 con 0x7f92f006d1b0 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f7fde640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:36.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f7fde640 4 set_mon_vals no callback set 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.126+0000 7f92f4d51640 10 monclient: dump: 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.142 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 10 monclient: shutdown 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 -- 10.20.193.78:0/2149334888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 msgr2=0x7f92f0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 --2- 10.20.193.78:0/2149334888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f0135500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92f006a480 tx=0x7f92ec02f150 comp rx=0 tx=0).stop 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 -- 10.20.193.78:0/2149334888 shutdown_connections 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 --2- 10.20.193.78:0/2149334888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f0135500 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 --2- 10.20.193.78:0/2149334888 >> v2:10.20.193.115:3300/0 conn(0x7f92f006c890 0x7f92f006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 -- 10.20.193.78:0/2149334888 >> 10.20.193.78:0/2149334888 conn(0x7f92f00a6620 msgr2=0x7f92f00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 -- 10.20.193.78:0/2149334888 shutdown_connections 2026-02-17T05:47:36.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 -- 10.20.193.78:0/2149334888 wait complete. 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 10 monclient: build_initial_monmap 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 10 monclient: monmap: 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:36.144 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 Processor -- start 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 1 -- start start 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.127+0000 7f92f7fde640 10 monclient: init 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92f006c890 0x7f92f01cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient(hunting): picked mon.noname-c con 0x7f92f006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f01cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient(hunting): picked mon.noname-a con 0x7f92f006d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92f009c4a0 0x7f92f01cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient(hunting): picked mon.noname-b con 0x7f92f009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:36.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92f00a01a0 con 0x7f92f006d1b0 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92f00a0020 con 0x7f92f009c4a0 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f92f009be20 con 0x7f92f006c890 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f7fde640 10 monclient(hunting): authenticate will time out at 701.682175s 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f01cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f01cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35072/0 (socket says 10.20.193.78:35072) 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 -- 10.20.193.78:0/1769491062 learned_addr learned my addr 10.20.193.78:0/1769491062 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient(hunting): get_auth_request con 0x7f92f006d1b0 auth_method 0 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f6554640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92f009c4a0 msgr2=0x7f92f01cdb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f6554640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92f009c4a0 0x7f92f01cdb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5d53640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92f006c890 0x7f92f01cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5d53640 10 monclient(hunting): get_auth_request con 0x7f92f006c890 auth_method 0 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5d53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5d53640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5d53640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient(hunting): handle_auth_done global_id 4467 payload 824 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92f006c890 msgr2=0x7f92f01cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92f006c890 0x7f92f01cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92f009c4a0 msgr2=0x7f92f01cdb40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92f009c4a0 0x7f92f01cdb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient: _finish_hunting 0 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 monclient: found mon.noname-a 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.128+0000 7f92f5552640 1 -- 10.20.193.78:0/1769491062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f0127090 con 0x7f92f006d1b0 2026-02-17T05:47:36.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f5552640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 msgr2=0x7f92f01cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f5552640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f01cd600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c750 tx=0x7f92e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005de0 con 0x7f92f006d1b0 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: dump: 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.149 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: _finish_auth 0 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e4007070 con 0x7f92f006d1b0 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f7fde640 5 monclient: authenticate success, global_id 4467 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4007440 con 0x7f92f006d1b0 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92de7fc640 10 monclient: dump: 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92dffff640 4 set_mon_vals no callback set 2026-02-17T05:47:36.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f7fde640 10 monclient: _renew_subs 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f7fde640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f0125ec0 con 0x7f92f006d1b0 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f7fde640 10 monclient: _renew_subs 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f7fde640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.130+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f0136ea0 con 0x7f92f006d1b0 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f7fde640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f7fde640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011ec80 0x7f92f011f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f7fde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01275b0 con 0x7f92f011ec80 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f92e4007660 con 0x7f92f006d1b0 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011ec80 0x7f92f011f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 10 monclient: get_auth_request con 0x7f92f011ec80 auth_method 0 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 10 monclient(hunting): handle_auth_done global_id 4470 payload 824 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f5d53640 1 -- 10.20.193.78:0/1769491062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92f006cc70 con 0x7f92f011ec80 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92de7fc640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f92c003de60 0x7f92c0040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4751+0+0 (secure 0 0 0) 0x7f92e4052570 con 0x7f92f006d1b0 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f6554640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f92c003de60 0x7f92c0040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.131+0000 7f92f6554640 10 monclient: get_auth_request con 0x7f92c003de60 auth_method 0 2026-02-17T05:47:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92f5d53640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011ec80 msgr2=0x7f92f011f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92f5d53640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011ec80 0x7f92f011f060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f92e0003a30 tx=0x7f92e0005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0016070 con 0x7f92f011ec80 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 10 monclient: dump: 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.154 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f92e000eb60 con 0x7f92f011ec80 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011ec80 msgr2=0x7f92f011f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011ec80 0x7f92f011f060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f92e0003a30 tx=0x7f92e0005a60 comp rx=0 tx=0).stop 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011ec80 msgr2=0x7f92f011f060 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.133+0000 7f92de7fc640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011ec80 0x7f92f011f060 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f92e0003a30 tx=0x7f92e0005a60 comp rx=0 tx=0).stop 2026-02-17T05:47:36.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.134+0000 7f92f6554640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f92c003de60 0x7f92c0040320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92f009c980 tx=0x7f92ec033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f7fde640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f7fde640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011f5a0 0x7f92f01d8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f7fde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92f00744d0 con 0x7f92f011f5a0 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011f5a0 0x7f92f01d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 10 monclient: get_auth_request con 0x7f92f011f5a0 auth_method 0 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 10 monclient(hunting): handle_auth_done global_id 4473 payload 824 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.137+0000 7f92f5d53640 1 -- 10.20.193.78:0/1769491062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f92f0126db0 con 0x7f92f011f5a0 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92f5d53640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011f5a0 msgr2=0x7f92f01d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92f5d53640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011f5a0 0x7f92f01d8540 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92e0003a30 tx=0x7f92e0005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0016040 con 0x7f92f011f5a0 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 10 monclient: dump: 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.159 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7f92e001b450 con 0x7f92f011f5a0 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011f5a0 msgr2=0x7f92f01d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011f5a0 0x7f92f01d8540 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92e0003a30 tx=0x7f92e0005ad0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011f5a0 msgr2=0x7f92f01d8540 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:36.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.139+0000 7f92de7fc640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f011f5a0 0x7f92f01d8540 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92e0003a30 tx=0x7f92e0005ad0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 16, 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 105943, 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:47:36.161 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:36.162 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:36.163 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:47:36.164 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:36.165 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:36.166 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:36.167 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:36.168 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:36.169 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.170 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:47:36.171 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:47:36.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f92c003de60 msgr2=0x7f92c0040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f92c003de60 0x7f92c0040320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92f009c980 tx=0x7f92ec033000 comp rx=0 tx=0).stop 2026-02-17T05:47:36.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 10 monclient: shutdown 2026-02-17T05:47:36.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 msgr2=0x7f92f01cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f01cd600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c750 tx=0x7f92e400cc20 comp rx=0 tx=0).stop 2026-02-17T05:47:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f6554640 1 -- 10.20.193.78:0/1769491062 reap_dead start 2026-02-17T05:47:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 shutdown_connections 2026-02-17T05:47:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 --2- 10.20.193.78:0/1769491062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92f006d1b0 0x7f92f01cd600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 >> 10.20.193.78:0/1769491062 conn(0x7f92f00a6620 msgr2=0x7f92f00a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 shutdown_connections 2026-02-17T05:47:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.140+0000 7f92f7fde640 1 -- 10.20.193.78:0/1769491062 wait complete. 2026-02-17T05:47:36.173 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:47:36.173 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:47:36.173 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:47:36.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.240+0000 7fd197bc1640 10 monclient: get_monmap_and_config 2026-02-17T05:47:36.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.240+0000 7fd197bc1640 10 monclient: build_initial_monmap 2026-02-17T05:47:36.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.240+0000 7fd197bc1640 10 monclient: monmap: 2026-02-17T05:47:36.244 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:36.244 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.244 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:47:36.241637+0000 2026-02-17T05:47:36.244 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:47:36.241637+0000 2026-02-17T05:47:36.244 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.240+0000 7fd197bc1640 1 Processor -- start 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.240+0000 7fd197bc1640 1 -- start start 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.240+0000 7fd197bc1640 10 monclient: init 2026-02-17T05:47:36.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19006eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient(hunting): picked mon.noname-b con 0x7fd1900a3140 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd1900a3520 0x7fd1900a3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient(hunting): picked mon.noname-c con 0x7fd1900a3520 addr v2:10.20.193.115:3300/0 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd19013c410 addr v1:10.20.193.78:6789/0 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1900a48a0 con 0x7fd19013c410 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd19006ffc0 con 0x7fd1900a3520 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd19006f970 con 0x7fd1900a3140 2026-02-17T05:47:36.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd197bc1640 10 monclient(hunting): authenticate will time out at 701.795177s 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195135640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fd1900a3520 msgr2=0x7fd1900a3900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195135640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd1900a3520 0x7fd1900a3900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19006eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd196137640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd19013c410 0x7fd19013e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50794/0 (socket says 10.20.193.78:50794) 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd196137640 1 -- 10.20.193.78:0/2008696595 learned_addr learned my addr 10.20.193.78:0/2008696595 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient(hunting): get_auth_request con 0x7fd1900a3140 auth_method 0 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient(hunting): handle_auth_done global_id 4436 payload 274 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 1 -- 10.20.193.78:0/2008696595 >> v2:10.20.193.115:3300/0 conn(0x7fd1900a3520 msgr2=0x7fd1900a3900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 1 --2- 10.20.193.78:0/2008696595 >> v2:10.20.193.115:3300/0 conn(0x7fd1900a3520 0x7fd1900a3900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 1 -- 10.20.193.78:0/2008696595 >> v1:10.20.193.78:6789/0 conn(0x7fd19013c410 legacy=0x7fd19013e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient: _finish_hunting 0 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 1 monclient: found mon.noname-b 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:47:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.241+0000 7fd195936640 1 -- 10.20.193.78:0/2008696595 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19006fc50 con 0x7fd1900a3140 2026-02-17T05:47:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd195936640 1 -- 10.20.193.78:0/2008696595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 msgr2=0x7fd19006eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd195936640 1 --2- 10.20.193.78:0/2008696595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19006eb10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd190070140 tx=0x7fd18002eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a644da644bab04a5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 1 -- 10.20.193.78:0/2008696595 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18003c070 con 0x7fd1900a3140 2026-02-17T05:47:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:47:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: dump: 2026-02-17T05:47:36.257 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.257 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: _finish_auth 0 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: _check_auth_tickets 2026-02-17T05:47:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 1 -- 10.20.193.78:0/2008696595 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd18002fd00 con 0x7fd1900a3140 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 5 monclient: authenticate success, global_id 4436 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 1 -- 10.20.193.78:0/2008696595 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180037660 con 0x7fd1900a3140 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd194934640 10 monclient: dump: 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.259 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 10 monclient: get_monmap_and_config success 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 4 set_mon_vals no callback set 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 10 monclient: shutdown 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 1 -- 10.20.193.78:0/2008696595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 msgr2=0x7fd19006eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 1 --2- 10.20.193.78:0/2008696595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19006eb10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd190070140 tx=0x7fd18002eda0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 1 -- 10.20.193.78:0/2008696595 shutdown_connections 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 1 --2- 10.20.193.78:0/2008696595 >> v2:10.20.193.115:3300/0 conn(0x7fd1900a3520 0x7fd1900a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 1 --2- 10.20.193.78:0/2008696595 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19006eb10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 1 -- 10.20.193.78:0/2008696595 >> 10.20.193.78:0/2008696595 conn(0x7fd19009c000 msgr2=0x7fd1900a8740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 1 -- 10.20.193.78:0/2008696595 shutdown_connections 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 1 -- 10.20.193.78:0/2008696595 wait complete. 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 10 monclient: build_initial_monmap 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.244+0000 7fd197bc1640 10 monclient: monmap: 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:47:36.261 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 1 Processor -- start 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 1 -- start start 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient: init 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient: _reopen_session rank -1 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19013b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient(hunting): picked mon.noname-c con 0x7fd1900a3140 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:47:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1900a3520 0x7fd19013b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient(hunting): picked mon.noname-a con 0x7fd1900a3520 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd19013c410 0x7fd1901232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient(hunting): picked mon.noname-b con 0x7fd19013c410 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1900a9300 con 0x7fd1900a3520 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1900a5070 con 0x7fd19013c410 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd1900a9480 con 0x7fd1900a3140 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient(hunting): _renew_subs 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd197bc1640 10 monclient(hunting): authenticate will time out at 701.799463s 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19013b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1900a3520 0x7fd19013b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19013b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37814/0 (socket says 10.20.193.78:37814) 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 -- 10.20.193.78:0/1123050925 learned_addr learned my addr 10.20.193.78:0/1123050925 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient(hunting): get_auth_request con 0x7fd1900a3140 auth_method 0 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 10 monclient(hunting): get_auth_request con 0x7fd1900a3520 auth_method 0 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd196137640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd19013c410 msgr2=0x7fd1901232a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd196137640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd19013c410 0x7fd1901232a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient(hunting): handle_auth_done global_id 4439 payload 824 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd19013c410 msgr2=0x7fd1901232a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd19013c410 0x7fd1901232a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1900a3520 msgr2=0x7fd19013b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1900a3520 0x7fd19013b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient: _finish_hunting 0 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 monclient: found mon.noname-c 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195936640 1 -- 10.20.193.78:0/1123050925 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd190125bf0 con 0x7fd1900a3140 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.245+0000 7fd195135640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1900a3520 0x7fd19013b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd195936640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 msgr2=0x7fd19013b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd195936640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19013b450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd180002410 tx=0x7fd1800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180046070 con 0x7fd1900a3140 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: dump: 2026-02-17T05:47:36.266 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:47:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd197bc1640 5 monclient: authenticate success, global_id 4439 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd197bc1640 10 monclient: _renew_subs 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd180041890 con 0x7fd1900a3140 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd197bc1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd190124a20 con 0x7fd1900a3140 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18003c040 con 0x7fd1900a3140 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17e7fc640 10 monclient: dump: 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.268 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd17ffff640 4 set_mon_vals no callback set 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd197bc1640 10 monclient: _renew_subs 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd197bc1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.247+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1900a5fb0 con 0x7fd1900a3140 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd197bc1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd197bc1640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013c7f0 0x7fd19013cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd197bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd190069d50 con 0x7fd19013c7f0 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013c7f0 0x7fd19013cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 10 monclient: get_auth_request con 0x7fd19013c7f0 auth_method 0 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd180041cc0 con 0x7fd1900a3140 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 10 monclient(hunting): handle_auth_done global_id 4442 payload 824 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd195135640 1 -- 10.20.193.78:0/1123050925 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd19006eb10 con 0x7fd19013c7f0 2026-02-17T05:47:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd17e7fc640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd16003de60 0x7fd160040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.248+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 4751+0+0 (secure 0 0 0) 0x7fd180077d00 con 0x7fd1900a3140 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.249+0000 7fd196137640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd16003de60 0x7fd160040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.249+0000 7fd196137640 10 monclient: get_auth_request con 0x7fd16003de60 auth_method 0 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.250+0000 7fd195135640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013c7f0 msgr2=0x7fd19013cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.250+0000 7fd195135640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013c7f0 0x7fd19013cbd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd190070d40 tx=0x7fd184003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184016070 con 0x7fd19013c7f0 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 10 monclient: dump: 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.271 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd18400ee10 con 0x7fd19013c7f0 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013c7f0 msgr2=0x7fd19013cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013c7f0 0x7fd19013cbd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd190070d40 tx=0x7fd184003ec0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013c7f0 msgr2=0x7fd19013cbd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd17e7fc640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013c7f0 0x7fd19013cbd0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd190070d40 tx=0x7fd184003ec0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.251+0000 7fd196137640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd16003de60 0x7fd160040320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd19006f690 tx=0x7fd18c0050d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd197bc1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd197bc1640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013d110 0x7fd19013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd197bc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd190124c40 con 0x7fd19013d110 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013d110 0x7fd19013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 10 monclient: get_auth_request con 0x7fd19013d110 auth_method 0 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 10 monclient(hunting): handle_auth_done global_id 4445 payload 824 2026-02-17T05:47:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.254+0000 7fd195135640 1 -- 10.20.193.78:0/1123050925 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd190073e90 con 0x7fd19013d110 2026-02-17T05:47:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.256+0000 7fd195135640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013d110 msgr2=0x7fd19013d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.256+0000 7fd195135640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013d110 0x7fd19013d520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd184018000 tx=0x7fd184005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184016040 con 0x7fd19013d110 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 10 monclient: dump: 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:47:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7fd18401b660 con 0x7fd19013d110 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013d110 msgr2=0x7fd19013d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013d110 0x7fd19013d520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd184018000 tx=0x7fd184005ac0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013d110 msgr2=0x7fd19013d520 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd17e7fc640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd19013d110 0x7fd19013d520 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd184018000 tx=0x7fd184005ac0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 16, 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 106062, 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:47:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:47:36.280 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.281 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.282 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.283 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:47:36.284 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:47:36.285 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:47:36.286 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd16003de60 msgr2=0x7fd160040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd16003de60 0x7fd160040320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd19006f690 tx=0x7fd18c0050d0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 10 monclient: shutdown 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 msgr2=0x7fd19013b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19013b450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd180002410 tx=0x7fd1800047c0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd196137640 1 -- 10.20.193.78:0/1123050925 reap_dead start 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 shutdown_connections 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 --2- 10.20.193.78:0/1123050925 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1900a3140 0x7fd19013b450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:47:36.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 >> 10.20.193.78:0/1123050925 conn(0x7fd19009c000 msgr2=0x7fd1900a23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:47:36.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 shutdown_connections 2026-02-17T05:47:36.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:47:36.257+0000 7fd197bc1640 1 -- 10.20.193.78:0/1123050925 wait complete. 2026-02-17T05:47:36.288 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:47:36.288 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:47:37.538 INFO:tasks.rados.rados.0.trial115.stdout:209: done (3 left) 2026-02-17T05:47:37.538 INFO:tasks.rados.rados.0.trial115.stdout:210: done (2 left) 2026-02-17T05:47:37.538 INFO:tasks.rados.rados.0.trial115.stdout:211: done (1 left) 2026-02-17T05:47:37.539 INFO:tasks.rados.rados.0.trial115.stdout:213: done (0 left) 2026-02-17T05:47:37.539 INFO:tasks.rados.rados.0.trial115.stdout:214: write (excl) oid 5 current snap is 19 2026-02-17T05:47:37.539 INFO:tasks.rados.rados.0.trial115.stdout:214: seq_num 112 ranges {682384=456830,1721042=423699,2809728=454936,3731357=151027} 2026-02-17T05:47:37.542 INFO:tasks.rados.rados.0.trial115.stdout:214: writing trial11545420-5 from 682384 to 1139214 tid 1 2026-02-17T05:47:37.546 INFO:tasks.rados.rados.0.trial115.stdout:214: writing trial11545420-5 from 1721042 to 2144741 tid 2 2026-02-17T05:47:37.551 INFO:tasks.rados.rados.0.trial115.stdout:214: writing trial11545420-5 from 2809728 to 3264664 tid 3 2026-02-17T05:47:37.553 INFO:tasks.rados.rados.0.trial115.stdout:214: writing trial11545420-5 from 3731357 to 3882384 tid 4 2026-02-17T05:47:37.554 INFO:tasks.rados.rados.0.trial115.stdout:214: finishing write tid 1 to trial11545420-5 2026-02-17T05:47:37.554 INFO:tasks.rados.rados.0.trial115.stdout:214: finishing write tid 2 to trial11545420-5 2026-02-17T05:47:37.554 INFO:tasks.rados.rados.0.trial115.stdout:214: finishing write tid 3 to trial11545420-5 2026-02-17T05:47:37.554 INFO:tasks.rados.rados.0.trial115.stdout:215: read oid 38 snap -1 2026-02-17T05:47:37.554 INFO:tasks.rados.rados.0.trial115.stdout:215: expect (ObjNum 37 snap 0 seq_num 37) 2026-02-17T05:47:37.554 INFO:tasks.rados.rados.0.trial115.stdout:216: snap_remove snap 15 2026-02-17T05:47:38.539 INFO:tasks.rados.rados.0.trial115.stdout:216: done (2 left) 2026-02-17T05:47:38.539 INFO:tasks.rados.rados.0.trial115.stdout:217: snap_remove snap 11 2026-02-17T05:47:39.542 INFO:tasks.rados.rados.0.trial115.stdout:217: done (2 left) 2026-02-17T05:47:39.543 INFO:tasks.rados.rados.0.trial115.stdout:218: write oid 27 current snap is 19 2026-02-17T05:47:39.543 INFO:tasks.rados.rados.0.trial115.stdout:218: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2026-02-17T05:47:39.547 INFO:tasks.rados.rados.0.trial115.stdout:218: writing trial11545420-27 from 699191 to 1231270 tid 1 2026-02-17T05:47:39.551 INFO:tasks.rados.rados.0.trial115.stdout:218: writing trial11545420-27 from 1779524 to 2346881 tid 2 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:218: writing trial11545420-27 from 2837151 to 3503359 tid 3 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:219: rollback oid 30 current snap is 19 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 8 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:214: finishing write tid 4 to trial11545420-5 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:214: finishing write tid 5 to trial11545420-5 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:214: finishing write tid 6 to trial11545420-5 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:214: oid 5 updating version 0 to 67 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:214: oid 5 updating version 67 to 68 2026-02-17T05:47:39.556 INFO:tasks.rados.rados.0.trial115.stdout:214: oid 5 version 68 is already newer than 64 2026-02-17T05:47:39.557 INFO:tasks.rados.rados.0.trial115.stdout:214: oid 5 version 68 is already newer than 65 2026-02-17T05:47:39.557 INFO:tasks.rados.rados.0.trial115.stdout:214: oid 5 version 68 is already newer than 66 2026-02-17T05:47:39.557 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 68 (ObjNum 112 snap 19 seq_num 112) dirty exists 2026-02-17T05:47:39.557 INFO:tasks.rados.rados.0.trial115.stdout:214: left oid 5 (ObjNum 112 snap 19 seq_num 112) 2026-02-17T05:47:39.557 INFO:tasks.rados.rados.0.trial115.stdout:215: expect (ObjNum 37 snap 0 seq_num 37) 2026-02-17T05:47:39.616 INFO:tasks.rados.rados.0.trial115.stdout:218: finishing write tid 1 to trial11545420-27 2026-02-17T05:47:39.617 INFO:tasks.rados.rados.0.trial115.stdout:214: done (3 left) 2026-02-17T05:47:39.617 INFO:tasks.rados.rados.0.trial115.stdout:215: done (2 left) 2026-02-17T05:47:39.617 INFO:tasks.rados.rados.0.trial115.stdout:220: snap_remove snap 9 2026-02-17T05:47:40.546 INFO:tasks.rados.rados.0.trial115.stdout:220: done (2 left) 2026-02-17T05:47:40.546 INFO:tasks.rados.rados.0.trial115.stdout:221: copy_from oid 21 from oid 3 current snap is 19 2026-02-17T05:47:40.546 INFO:tasks.rados.rados.0.trial115.stdout:218: finishing write tid 2 to trial11545420-27 2026-02-17T05:47:40.546 INFO:tasks.rados.rados.0.trial115.stdout:218: finishing write tid 3 to trial11545420-27 2026-02-17T05:47:40.546 INFO:tasks.rados.rados.0.trial115.stdout:218: finishing write tid 4 to trial11545420-27 2026-02-17T05:47:40.546 INFO:tasks.rados.rados.0.trial115.stdout:219: finishing rollback tid 0 to trial11545420-30 2026-02-17T05:47:40.546 INFO:tasks.rados.rados.0.trial115.stdout:218: finishing write tid 5 to trial11545420-27 2026-02-17T05:47:40.546 INFO:tasks.rados.rados.0.trial115.stdout:218: oid 27 updating version 0 to 69 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:218: oid 27 version 69 is already newer than 68 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:218: oid 27 updating version 69 to 71 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:218: oid 27 version 71 is already newer than 70 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 71 (ObjNum 113 snap 19 seq_num 113) dirty exists 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:218: left oid 27 (ObjNum 113 snap 19 seq_num 113) 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:219: finishing rollback tid 1 to trial11545420-30 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:219: finishing rollback tid 2 to trial11545420-30 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 75 (ObjNum 60 snap 2 seq_num 60) dirty exists 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:218: done (2 left) 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:219: done (1 left) 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:222: write (excl) oid 1 current snap is 19 2026-02-17T05:47:40.547 INFO:tasks.rados.rados.0.trial115.stdout:222: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2026-02-17T05:47:40.551 INFO:tasks.rados.rados.0.trial115.stdout:222: writing trial11545420-1 from 715998 to 1323326 tid 1 2026-02-17T05:47:40.554 INFO:tasks.rados.rados.0.trial115.stdout:222: writing trial11545420-1 from 1838006 to 2265374 tid 2 2026-02-17T05:47:40.558 INFO:tasks.rados.rados.0.trial115.stdout:222: writing trial11545420-1 from 2864574 to 3342054 tid 3 2026-02-17T05:47:40.561 INFO:tasks.rados.rados.0.trial115.stdout:222: writing trial11545420-1 from 3863835 to 3915998 tid 4 2026-02-17T05:47:40.561 INFO:tasks.rados.rados.0.trial115.stdout:223: write (excl) oid 24 current snap is 19 2026-02-17T05:47:40.561 INFO:tasks.rados.rados.0.trial115.stdout:222: finishing write tid 1 to trial11545420-1 2026-02-17T05:47:40.561 INFO:tasks.rados.rados.0.trial115.stdout:222: finishing write tid 2 to trial11545420-1 2026-02-17T05:47:40.561 INFO:tasks.rados.rados.0.trial115.stdout:222: finishing write tid 3 to trial11545420-1 2026-02-17T05:47:40.561 INFO:tasks.rados.rados.0.trial115.stdout:223: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2026-02-17T05:47:40.566 INFO:tasks.rados.rados.0.trial115.stdout:223: writing trial11545420-24 from 732805 to 1531735 tid 1 2026-02-17T05:47:40.571 INFO:tasks.rados.rados.0.trial115.stdout:223: writing trial11545420-24 from 2296488 to 2867514 tid 2 2026-02-17T05:47:40.574 INFO:tasks.rados.rados.0.trial115.stdout:223: writing trial11545420-24 from 3291997 to 3697102 tid 3 2026-02-17T05:47:40.575 INFO:tasks.rados.rados.0.trial115.stdout:224: snap_remove snap 8 2026-02-17T05:47:40.915 INFO:tasks.rados.rados.0.trial115.stdout:224: done (3 left) 2026-02-17T05:47:40.915 INFO:tasks.rados.rados.0.trial115.stdout:225: write oid 39 current snap is 19 2026-02-17T05:47:40.916 INFO:tasks.rados.rados.0.trial115.stdout:225: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2026-02-17T05:47:40.920 INFO:tasks.rados.rados.0.trial115.stdout:225: writing trial11545420-39 from 749612 to 1223791 tid 1 2026-02-17T05:47:40.924 INFO:tasks.rados.rados.0.trial115.stdout:225: writing trial11545420-39 from 1954970 to 2386007 tid 2 2026-02-17T05:47:40.929 INFO:tasks.rados.rados.0.trial115.stdout:225: writing trial11545420-39 from 2919420 to 3535797 tid 3 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:226: read oid 46 snap -1 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:226: expect deleted 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:227: copy_from oid 22 from oid 50 current snap is 19 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:221: finishing copy_from racing read to trial11545420-21 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:221: finishing copy_from to trial11545420-21 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 19 (ObjNum 54 snap 0 seq_num 54) dirty exists 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:222: finishing write tid 4 to trial11545420-1 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:222: finishing write tid 5 to trial11545420-1 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:222: finishing write tid 6 to trial11545420-1 2026-02-17T05:47:40.930 INFO:tasks.rados.rados.0.trial115.stdout:222: oid 1 updating version 0 to 39 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:222: oid 1 updating version 39 to 40 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:222: oid 1 updating version 40 to 41 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:222: oid 1 updating version 41 to 42 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:222: oid 1 updating version 42 to 43 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 43 (ObjNum 114 snap 19 seq_num 114) dirty exists 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:222: left oid 1 (ObjNum 114 snap 19 seq_num 114) 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:223: finishing write tid 1 to trial11545420-24 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:223: finishing write tid 2 to trial11545420-24 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:223: finishing write tid 3 to trial11545420-24 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:223: finishing write tid 4 to trial11545420-24 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:223: finishing write tid 5 to trial11545420-24 2026-02-17T05:47:40.931 INFO:tasks.rados.rados.0.trial115.stdout:223: oid 24 updating version 0 to 109 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:223: oid 24 version 109 is already newer than 108 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:223: oid 24 version 109 is already newer than 107 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:223: oid 24 version 109 is already newer than 106 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 109 (ObjNum 115 snap 19 seq_num 115) dirty exists 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:223: left oid 24 (ObjNum 115 snap 19 seq_num 115) 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:225: finishing write tid 1 to trial11545420-39 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:225: finishing write tid 2 to trial11545420-39 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:221: done (5 left) 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:222: done (4 left) 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:223: done (3 left) 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:228: write oid 25 current snap is 19 2026-02-17T05:47:40.932 INFO:tasks.rados.rados.0.trial115.stdout:228: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2026-02-17T05:47:40.934 INFO:tasks.rados.rados.0.trial115.stdout:228: writing trial11545420-25 from 766419 to 1315847 tid 1 2026-02-17T05:47:40.938 INFO:tasks.rados.rados.0.trial115.stdout:228: writing trial11545420-25 from 2013452 to 2588147 tid 2 2026-02-17T05:47:40.943 INFO:tasks.rados.rados.0.trial115.stdout:228: writing trial11545420-25 from 3346843 to 3774492 tid 3 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:229: rollback oid 16 current snap is 19 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 16 to 16 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:227: finishing copy_from to trial11545420-22 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:227: got expected ENOENT (src dne) 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:227: finishing copy_from racing read to trial11545420-22 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:225: finishing write tid 3 to trial11545420-39 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:226: done (4 left) 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:227: done (3 left) 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:230: write oid 12 current snap is 19 2026-02-17T05:47:40.944 INFO:tasks.rados.rados.0.trial115.stdout:230: seq_num 118 ranges {783226=624677,2071934=434706,2974266=638921} 2026-02-17T05:47:40.948 INFO:tasks.rados.rados.0.trial115.stdout:230: writing trial11545420-12 from 783226 to 1407903 tid 1 2026-02-17T05:47:40.952 INFO:tasks.rados.rados.0.trial115.stdout:230: writing trial11545420-12 from 2071934 to 2506640 tid 2 2026-02-17T05:47:40.958 INFO:tasks.rados.rados.0.trial115.stdout:230: writing trial11545420-12 from 2974266 to 3613187 tid 3 2026-02-17T05:47:40.958 INFO:tasks.rados.rados.0.trial115.stdout:231: delete oid 28 current snap is 19 2026-02-17T05:47:40.958 INFO:tasks.rados.rados.0.trial115.stdout:225: finishing write tid 4 to trial11545420-39 2026-02-17T05:47:40.958 INFO:tasks.rados.rados.0.trial115.stdout:225: finishing write tid 5 to trial11545420-39 2026-02-17T05:47:40.958 INFO:tasks.rados.rados.0.trial115.stdout:225: oid 39 updating version 0 to 111 2026-02-17T05:47:40.958 INFO:tasks.rados.rados.0.trial115.stdout:225: oid 39 updating version 111 to 120 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:225: oid 39 version 120 is already newer than 119 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:225: oid 39 version 120 is already newer than 118 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 120 (ObjNum 116 snap 19 seq_num 116) dirty exists 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:225: left oid 39 (ObjNum 116 snap 19 seq_num 116) 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:228: finishing write tid 1 to trial11545420-25 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:228: finishing write tid 2 to trial11545420-25 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:229: finishing rollback tid 0 to trial11545420-16 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:228: finishing write tid 3 to trial11545420-25 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:228: finishing write tid 4 to trial11545420-25 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:229: finishing rollback tid 1 to trial11545420-16 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:228: finishing write tid 5 to trial11545420-25 2026-02-17T05:47:40.959 INFO:tasks.rados.rados.0.trial115.stdout:228: oid 25 updating version 0 to 99 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:228: oid 25 updating version 99 to 102 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:228: oid 25 version 102 is already newer than 101 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:228: oid 25 version 102 is already newer than 100 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 102 (ObjNum 117 snap 19 seq_num 117) dirty exists 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:228: left oid 25 (ObjNum 117 snap 19 seq_num 117) 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:229: finishing rollback tid 2 to trial11545420-16 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 113 (ObjNum 15 snap 0 seq_num 15) dirty exists 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:230: finishing write tid 1 to trial11545420-12 2026-02-17T05:47:40.960 INFO:tasks.rados.rados.0.trial115.stdout:230: finishing write tid 2 to trial11545420-12 2026-02-17T05:47:40.961 INFO:tasks.rados.rados.0.trial115.stdout:230: finishing write tid 3 to trial11545420-12 2026-02-17T05:47:40.961 INFO:tasks.rados.rados.0.trial115.stdout:230: finishing write tid 4 to trial11545420-12 2026-02-17T05:47:40.961 INFO:tasks.rados.rados.0.trial115.stdout:230: finishing write tid 5 to trial11545420-12 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:230: oid 12 updating version 0 to 122 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:230: oid 12 version 122 is already newer than 121 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:230: oid 12 updating version 122 to 124 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:230: oid 12 version 124 is already newer than 123 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 124 (ObjNum 118 snap 19 seq_num 118) dirty exists 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:230: left oid 12 (ObjNum 118 snap 19 seq_num 118) 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:225: done (4 left) 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:228: done (3 left) 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:229: done (2 left) 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:230: done (1 left) 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:231: done (0 left) 2026-02-17T05:47:40.962 INFO:tasks.rados.rados.0.trial115.stdout:232: write (excl) oid 42 current snap is 19 2026-02-17T05:47:40.963 INFO:tasks.rados.rados.0.trial115.stdout:232: seq_num 120 ranges {416840=775175,1672545=344295} 2026-02-17T05:47:40.964 INFO:tasks.rados.rados.0.trial115.stdout:232: writing trial11545420-42 from 416840 to 1192015 tid 1 2026-02-17T05:47:40.968 INFO:tasks.rados.rados.0.trial115.stdout:232: writing trial11545420-42 from 1672545 to 2016840 tid 2 2026-02-17T05:47:40.968 INFO:tasks.rados.rados.0.trial115.stdout:233: copy_from oid 23 from oid 40 current snap is 19 2026-02-17T05:47:40.968 INFO:tasks.rados.rados.0.trial115.stdout:232: finishing write tid 1 to trial11545420-42 2026-02-17T05:47:40.969 INFO:tasks.rados.rados.0.trial115.stdout:234: copy_from oid 27 from oid 12 current snap is 19 2026-02-17T05:47:40.969 INFO:tasks.rados.rados.0.trial115.stdout:235: write oid 26 current snap is 19 2026-02-17T05:47:40.969 INFO:tasks.rados.rados.0.trial115.stdout:235: seq_num 121 ranges {433647=450424,1331027=582033} 2026-02-17T05:47:40.971 INFO:tasks.rados.rados.0.trial115.stdout:235: writing trial11545420-26 from 433647 to 884071 tid 1 2026-02-17T05:47:40.975 INFO:tasks.rados.rados.0.trial115.stdout:235: writing trial11545420-26 from 1331027 to 1913060 tid 2 2026-02-17T05:47:40.975 INFO:tasks.rados.rados.0.trial115.stdout:236: snap_remove snap 18 2026-02-17T05:47:40.975 INFO:tasks.rados.rados.0.trial115.stdout:232: finishing write tid 2 to trial11545420-42 2026-02-17T05:47:40.976 INFO:tasks.rados.rados.0.trial115.stdout:232: finishing write tid 3 to trial11545420-42 2026-02-17T05:47:40.976 INFO:tasks.rados.rados.0.trial115.stdout:232: finishing write tid 4 to trial11545420-42 2026-02-17T05:47:40.976 INFO:tasks.rados.rados.0.trial115.stdout:232: oid 42 updating version 0 to 126 2026-02-17T05:47:40.976 INFO:tasks.rados.rados.0.trial115.stdout:232: oid 42 updating version 126 to 127 2026-02-17T05:47:40.976 INFO:tasks.rados.rados.0.trial115.stdout:232: oid 42 updating version 127 to 128 2026-02-17T05:47:40.976 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 128 (ObjNum 120 snap 19 seq_num 120) dirty exists 2026-02-17T05:47:40.976 INFO:tasks.rados.rados.0.trial115.stdout:232: left oid 42 (ObjNum 120 snap 19 seq_num 120) 2026-02-17T05:47:40.976 INFO:tasks.rados.rados.0.trial115.stdout:235: finishing write tid 1 to trial11545420-26 2026-02-17T05:47:41.918 INFO:tasks.rados.rados.0.trial115.stdout:232: done (4 left) 2026-02-17T05:47:41.918 INFO:tasks.rados.rados.0.trial115.stdout:236: done (3 left) 2026-02-17T05:47:41.918 INFO:tasks.rados.rados.0.trial115.stdout:237: read oid 11 snap 16 2026-02-17T05:47:41.918 INFO:tasks.rados.rados.0.trial115.stdout:237: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:238: snap_create 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:234: finishing copy_from racing read to trial11545420-27 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:234: finishing copy_from to trial11545420-27 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 124 (ObjNum 118 snap 19 seq_num 118) dirty exists 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:235: finishing write tid 2 to trial11545420-26 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:235: finishing write tid 3 to trial11545420-26 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:235: finishing write tid 4 to trial11545420-26 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:235: oid 26 updating version 0 to 6 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:235: oid 26 updating version 6 to 7 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:235: oid 26 version 7 is already newer than 5 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 7 (ObjNum 121 snap 19 seq_num 121) dirty exists 2026-02-17T05:47:41.919 INFO:tasks.rados.rados.0.trial115.stdout:235: left oid 26 (ObjNum 121 snap 19 seq_num 121) 2026-02-17T05:47:41.920 INFO:tasks.rados.rados.0.trial115.stdout:233: finishing copy_from racing read to trial11545420-23 2026-02-17T05:47:41.920 INFO:tasks.rados.rados.0.trial115.stdout:233: finishing copy_from to trial11545420-23 2026-02-17T05:47:41.920 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 34 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-17T05:47:41.925 INFO:tasks.rados.rados.0.trial115.stdout:237: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:47:42.919 INFO:tasks.rados.rados.0.trial115.stdout:233: done (4 left) 2026-02-17T05:47:42.919 INFO:tasks.rados.rados.0.trial115.stdout:234: done (3 left) 2026-02-17T05:47:42.920 INFO:tasks.rados.rados.0.trial115.stdout:235: done (2 left) 2026-02-17T05:47:42.920 INFO:tasks.rados.rados.0.trial115.stdout:237: done (1 left) 2026-02-17T05:47:42.920 INFO:tasks.rados.rados.0.trial115.stdout:238: done (0 left) 2026-02-17T05:47:42.920 INFO:tasks.rados.rados.0.trial115.stdout:239: snap_remove snap 19 2026-02-17T05:47:43.920 INFO:tasks.rados.rados.0.trial115.stdout:239: done (0 left) 2026-02-17T05:47:43.920 INFO:tasks.rados.rados.0.trial115.stdout:240: write oid 3 current snap is 20 2026-02-17T05:47:43.920 INFO:tasks.rados.rados.0.trial115.stdout:240: seq_num 122 ranges {450454=642026,1505862=544592} 2026-02-17T05:47:43.924 INFO:tasks.rados.rados.0.trial115.stdout:240: writing trial11545420-3 from 450454 to 1092480 tid 1 2026-02-17T05:47:43.928 INFO:tasks.rados.rados.0.trial115.stdout:240: writing trial11545420-3 from 1505862 to 2050454 tid 2 2026-02-17T05:47:43.928 INFO:tasks.rados.rados.0.trial115.stdout:241: delete oid 42 current snap is 20 2026-02-17T05:47:43.928 INFO:tasks.rados.rados.0.trial115.stdout:240: finishing write tid 1 to trial11545420-3 2026-02-17T05:47:43.929 INFO:tasks.rados.rados.0.trial115.stdout:241: done (1 left) 2026-02-17T05:47:43.929 INFO:tasks.rados.rados.0.trial115.stdout:242: write (excl) oid 15 current snap is 20 2026-02-17T05:47:43.930 INFO:tasks.rados.rados.0.trial115.stdout:242: seq_num 124 ranges {484068=792524,1906473=177595} 2026-02-17T05:47:43.934 INFO:tasks.rados.rados.0.trial115.stdout:242: writing trial11545420-15 from 484068 to 1276592 tid 1 2026-02-17T05:47:43.937 INFO:tasks.rados.rados.0.trial115.stdout:242: writing trial11545420-15 from 1906473 to 2084068 tid 2 2026-02-17T05:47:43.937 INFO:tasks.rados.rados.0.trial115.stdout:243: read oid 22 snap 16 2026-02-17T05:47:43.938 INFO:tasks.rados.rados.0.trial115.stdout:243: expect (ObjNum 107 snap 16 seq_num 107) 2026-02-17T05:47:43.938 INFO:tasks.rados.rados.0.trial115.stdout:244: snap_remove snap 12 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:244: done (3 left) 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:245: copy_from oid 10 from oid 50 current snap is 20 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:240: finishing write tid 2 to trial11545420-3 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:240: finishing write tid 3 to trial11545420-3 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:240: finishing write tid 4 to trial11545420-3 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:240: oid 3 updating version 0 to 90 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:240: oid 3 version 90 is already newer than 88 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:240: oid 3 version 90 is already newer than 89 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 90 (ObjNum 122 snap 20 seq_num 122) dirty exists 2026-02-17T05:47:44.924 INFO:tasks.rados.rados.0.trial115.stdout:240: left oid 3 (ObjNum 122 snap 20 seq_num 122) 2026-02-17T05:47:44.925 INFO:tasks.rados.rados.0.trial115.stdout:242: finishing write tid 1 to trial11545420-15 2026-02-17T05:47:44.925 INFO:tasks.rados.rados.0.trial115.stdout:242: finishing write tid 2 to trial11545420-15 2026-02-17T05:47:44.925 INFO:tasks.rados.rados.0.trial115.stdout:242: finishing write tid 3 to trial11545420-15 2026-02-17T05:47:44.925 INFO:tasks.rados.rados.0.trial115.stdout:240: done (3 left) 2026-02-17T05:47:44.925 INFO:tasks.rados.rados.0.trial115.stdout:246: write oid 49 current snap is 20 2026-02-17T05:47:44.925 INFO:tasks.rados.rados.0.trial115.stdout:246: seq_num 125 ranges {500875=467773,1564955=535920} 2026-02-17T05:47:44.927 INFO:tasks.rados.rados.0.trial115.stdout:246: writing trial11545420-49 from 500875 to 968648 tid 1 2026-02-17T05:47:44.932 INFO:tasks.rados.rados.0.trial115.stdout:246: writing trial11545420-49 from 1564955 to 2100875 tid 2 2026-02-17T05:47:44.932 INFO:tasks.rados.rados.0.trial115.stdout:247: read oid 43 snap -1 2026-02-17T05:47:44.932 INFO:tasks.rados.rados.0.trial115.stdout:247: expect (ObjNum 108 snap 16 seq_num 108) 2026-02-17T05:47:44.932 INFO:tasks.rados.rados.0.trial115.stdout:248: delete oid 33 current snap is 20 2026-02-17T05:47:44.932 INFO:tasks.rados.rados.0.trial115.stdout:242: finishing write tid 4 to trial11545420-15 2026-02-17T05:47:44.932 INFO:tasks.rados.rados.0.trial115.stdout:242: oid 15 updating version 0 to 98 2026-02-17T05:47:44.932 INFO:tasks.rados.rados.0.trial115.stdout:242: oid 15 version 98 is already newer than 96 2026-02-17T05:47:44.933 INFO:tasks.rados.rados.0.trial115.stdout:242: oid 15 version 98 is already newer than 97 2026-02-17T05:47:44.933 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 98 (ObjNum 124 snap 20 seq_num 124) dirty exists 2026-02-17T05:47:44.933 INFO:tasks.rados.rados.0.trial115.stdout:242: left oid 15 (ObjNum 124 snap 20 seq_num 124) 2026-02-17T05:47:44.933 INFO:tasks.rados.rados.0.trial115.stdout:243: expect (ObjNum 107 snap 16 seq_num 107) 2026-02-17T05:47:45.054 INFO:tasks.rados.rados.0.trial115.stdout:245: finishing copy_from to trial11545420-10 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:245: got expected ENOENT (src dne) 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:245: finishing copy_from racing read to trial11545420-10 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:246: finishing write tid 1 to trial11545420-49 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:242: done (5 left) 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:243: done (4 left) 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:245: done (3 left) 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:248: done (2 left) 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:249: read oid 50 snap -1 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:249: expect deleted 2026-02-17T05:47:45.055 INFO:tasks.rados.rados.0.trial115.stdout:250: read oid 39 snap -1 2026-02-17T05:47:45.056 INFO:tasks.rados.rados.0.trial115.stdout:250: expect (ObjNum 116 snap 19 seq_num 116) 2026-02-17T05:47:45.056 INFO:tasks.rados.rados.0.trial115.stdout:251: snap_remove snap 16 2026-02-17T05:47:45.916 INFO:tasks.rados.rados.0.trial115.stdout:251: done (4 left) 2026-02-17T05:47:45.917 INFO:tasks.rados.rados.0.trial115.stdout:252: write oid 8 current snap is 20 2026-02-17T05:47:45.917 INFO:tasks.rados.rados.0.trial115.stdout:246: finishing write tid 2 to trial11545420-49 2026-02-17T05:47:45.917 INFO:tasks.rados.rados.0.trial115.stdout:252: seq_num 127 ranges {534489=618271,1565566=568923} 2026-02-17T05:47:45.921 INFO:tasks.rados.rados.0.trial115.stdout:252: writing trial11545420-8 from 534489 to 1152760 tid 1 2026-02-17T05:47:45.925 INFO:tasks.rados.rados.0.trial115.stdout:252: writing trial11545420-8 from 1565566 to 2134489 tid 2 2026-02-17T05:47:45.925 INFO:tasks.rados.rados.0.trial115.stdout:253: write (excl) oid 47 current snap is 20 2026-02-17T05:47:45.925 INFO:tasks.rados.rados.0.trial115.stdout:253: seq_num 128 ranges {551296=693520,2024048=127248} 2026-02-17T05:47:45.930 INFO:tasks.rados.rados.0.trial115.stdout:253: writing trial11545420-47 from 551296 to 1244816 tid 1 2026-02-17T05:47:45.934 INFO:tasks.rados.rados.0.trial115.stdout:253: writing trial11545420-47 from 2024048 to 2151296 tid 2 2026-02-17T05:47:45.934 INFO:tasks.rados.rados.0.trial115.stdout:254: read oid 1 snap -1 2026-02-17T05:47:45.934 INFO:tasks.rados.rados.0.trial115.stdout:254: expect (ObjNum 114 snap 19 seq_num 114) 2026-02-17T05:47:45.934 INFO:tasks.rados.rados.0.trial115.stdout:255: read oid 2 snap -1 2026-02-17T05:47:45.934 INFO:tasks.rados.rados.0.trial115.stdout:255: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:256: read oid 35 snap -1 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:256: expect deleted 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:257: delete oid 45 current snap is 20 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:246: finishing write tid 3 to trial11545420-49 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:246: finishing write tid 4 to trial11545420-49 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:246: oid 49 updating version 0 to 42 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:246: oid 49 updating version 42 to 43 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:246: oid 49 version 43 is already newer than 41 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 43 (ObjNum 125 snap 20 seq_num 125) dirty exists 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:246: left oid 49 (ObjNum 125 snap 20 seq_num 125) 2026-02-17T05:47:45.935 INFO:tasks.rados.rados.0.trial115.stdout:247: expect (ObjNum 108 snap 16 seq_num 108) 2026-02-17T05:47:46.057 INFO:tasks.rados.rados.0.trial115.stdout:250: expect (ObjNum 116 snap 19 seq_num 116) 2026-02-17T05:47:46.182 INFO:tasks.rados.rados.0.trial115.stdout:252: finishing write tid 1 to trial11545420-8 2026-02-17T05:47:46.182 INFO:tasks.rados.rados.0.trial115.stdout:252: finishing write tid 2 to trial11545420-8 2026-02-17T05:47:46.182 INFO:tasks.rados.rados.0.trial115.stdout:252: finishing write tid 3 to trial11545420-8 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:252: finishing write tid 4 to trial11545420-8 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:252: oid 8 updating version 0 to 8 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:252: oid 8 updating version 8 to 10 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:252: oid 8 version 10 is already newer than 9 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 10 (ObjNum 127 snap 20 seq_num 127) dirty exists 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:252: left oid 8 (ObjNum 127 snap 20 seq_num 127) 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:253: finishing write tid 1 to trial11545420-47 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:253: finishing write tid 2 to trial11545420-47 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:253: finishing write tid 3 to trial11545420-47 2026-02-17T05:47:46.183 INFO:tasks.rados.rados.0.trial115.stdout:253: finishing write tid 4 to trial11545420-47 2026-02-17T05:47:46.184 INFO:tasks.rados.rados.0.trial115.stdout:253: oid 47 updating version 0 to 102 2026-02-17T05:47:46.184 INFO:tasks.rados.rados.0.trial115.stdout:253: oid 47 version 102 is already newer than 101 2026-02-17T05:47:46.184 INFO:tasks.rados.rados.0.trial115.stdout:253: oid 47 updating version 102 to 103 2026-02-17T05:47:46.184 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 103 (ObjNum 128 snap 20 seq_num 128) dirty exists 2026-02-17T05:47:46.184 INFO:tasks.rados.rados.0.trial115.stdout:253: left oid 47 (ObjNum 128 snap 20 seq_num 128) 2026-02-17T05:47:46.184 INFO:tasks.rados.rados.0.trial115.stdout:255: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-17T05:47:46.242 INFO:tasks.rados.rados.0.trial115.stdout:254: expect (ObjNum 114 snap 19 seq_num 114) 2026-02-17T05:47:46.366 INFO:tasks.rados.rados.0.trial115.stdout:246: done (9 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:247: done (8 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:249: done (7 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:250: done (6 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:252: done (5 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:253: done (4 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:254: done (3 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:255: done (2 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:256: done (1 left) 2026-02-17T05:47:46.367 INFO:tasks.rados.rados.0.trial115.stdout:257: done (0 left) 2026-02-17T05:47:46.368 INFO:tasks.rados.rados.0.trial115.stdout:258: copy_from oid 33 from oid 41 current snap is 20 2026-02-17T05:47:46.368 INFO:tasks.rados.rados.0.trial115.stdout:259: copy_from oid 23 from oid 6 current snap is 20 2026-02-17T05:47:46.368 INFO:tasks.rados.rados.0.trial115.stdout:260: read oid 47 snap -1 2026-02-17T05:47:46.368 INFO:tasks.rados.rados.0.trial115.stdout:260: expect (ObjNum 128 snap 20 seq_num 128) 2026-02-17T05:47:46.368 INFO:tasks.rados.rados.0.trial115.stdout:261: snap_create 2026-02-17T05:47:46.369 INFO:tasks.rados.rados.0.trial115.stdout:260: expect (ObjNum 128 snap 20 seq_num 128) 2026-02-17T05:47:46.421 INFO:tasks.rados.rados.0.trial115.stdout:258: finishing copy_from racing read to trial11545420-33 2026-02-17T05:47:46.421 INFO:tasks.rados.rados.0.trial115.stdout:259: finishing copy_from racing read to trial11545420-23 2026-02-17T05:47:46.421 INFO:tasks.rados.rados.0.trial115.stdout:258: finishing copy_from to trial11545420-33 2026-02-17T05:47:46.421 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 12 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-17T05:47:46.421 INFO:tasks.rados.rados.0.trial115.stdout:259: finishing copy_from to trial11545420-23 2026-02-17T05:47:46.421 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2026-02-17T05:47:46.925 INFO:tasks.rados.rados.0.trial115.stdout:258: done (3 left) 2026-02-17T05:47:46.925 INFO:tasks.rados.rados.0.trial115.stdout:259: done (2 left) 2026-02-17T05:47:46.925 INFO:tasks.rados.rados.0.trial115.stdout:260: done (1 left) 2026-02-17T05:47:46.925 INFO:tasks.rados.rados.0.trial115.stdout:261: done (0 left) 2026-02-17T05:47:46.925 INFO:tasks.rados.rados.0.trial115.stdout:262: copy_from oid 13 from oid 18 current snap is 21 2026-02-17T05:47:46.925 INFO:tasks.rados.rados.0.trial115.stdout:263: read oid 20 snap -1 2026-02-17T05:47:46.925 INFO:tasks.rados.rados.0.trial115.stdout:263: expect (ObjNum 19 snap 0 seq_num 19) 2026-02-17T05:47:46.925 INFO:tasks.rados.rados.0.trial115.stdout:264: snap_create 2026-02-17T05:47:46.931 INFO:tasks.rados.rados.0.trial115.stdout:263: expect (ObjNum 19 snap 0 seq_num 19) 2026-02-17T05:47:46.987 INFO:tasks.rados.rados.0.trial115.stdout:262: finishing copy_from racing read to trial11545420-13 2026-02-17T05:47:46.987 INFO:tasks.rados.rados.0.trial115.stdout:262: finishing copy_from to trial11545420-13 2026-02-17T05:47:46.987 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 70 (ObjNum 102 snap 15 seq_num 102) dirty exists 2026-02-17T05:47:47.926 INFO:tasks.rados.rados.0.trial115.stdout:262: done (2 left) 2026-02-17T05:47:47.927 INFO:tasks.rados.rados.0.trial115.stdout:263: done (1 left) 2026-02-17T05:47:47.927 INFO:tasks.rados.rados.0.trial115.stdout:264: done (0 left) 2026-02-17T05:47:47.927 INFO:tasks.rados.rados.0.trial115.stdout:265: write oid 2 current snap is 22 2026-02-17T05:47:47.927 INFO:tasks.rados.rados.0.trial115.stdout:265: seq_num 130 ranges {584910=560371,1857365=327545} 2026-02-17T05:47:47.931 INFO:tasks.rados.rados.0.trial115.stdout:265: writing trial11545420-2 from 584910 to 1145281 tid 1 2026-02-17T05:47:47.936 INFO:tasks.rados.rados.0.trial115.stdout:265: writing trial11545420-2 from 1857365 to 2184910 tid 2 2026-02-17T05:47:47.936 INFO:tasks.rados.rados.0.trial115.stdout:266: rollback oid 18 current snap is 22 2026-02-17T05:47:47.936 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 20 2026-02-17T05:47:47.936 INFO:tasks.rados.rados.0.trial115.stdout:265: finishing write tid 1 to trial11545420-2 2026-02-17T05:47:47.936 INFO:tasks.rados.rados.0.trial115.stdout:267: write (excl) oid 26 current snap is 22 2026-02-17T05:47:47.936 INFO:tasks.rados.rados.0.trial115.stdout:267: seq_num 131 ranges {601717=635620,1799494=402223} 2026-02-17T05:47:47.940 INFO:tasks.rados.rados.0.trial115.stdout:267: writing trial11545420-26 from 601717 to 1237337 tid 1 2026-02-17T05:47:47.944 INFO:tasks.rados.rados.0.trial115.stdout:267: writing trial11545420-26 from 1799494 to 2201717 tid 2 2026-02-17T05:47:47.944 INFO:tasks.rados.rados.0.trial115.stdout:268: read oid 38 snap -1 2026-02-17T05:47:47.944 INFO:tasks.rados.rados.0.trial115.stdout:268: expect (ObjNum 37 snap 0 seq_num 37) 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:269: copy_from oid 16 from oid 14 current snap is 22 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:265: finishing write tid 2 to trial11545420-2 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:265: finishing write tid 3 to trial11545420-2 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:265: finishing write tid 4 to trial11545420-2 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:265: oid 2 updating version 0 to 105 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:265: oid 2 updating version 105 to 107 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:265: oid 2 version 107 is already newer than 106 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 107 (ObjNum 130 snap 22 seq_num 130) dirty exists 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:265: left oid 2 (ObjNum 130 snap 22 seq_num 130) 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:266: finishing rollback tid 0 to trial11545420-18 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:266: finishing rollback tid 1 to trial11545420-18 2026-02-17T05:47:47.945 INFO:tasks.rados.rados.0.trial115.stdout:265: done (4 left) 2026-02-17T05:47:47.946 INFO:tasks.rados.rados.0.trial115.stdout:270: read oid 30 snap 21 2026-02-17T05:47:47.946 INFO:tasks.rados.rados.0.trial115.stdout:270: expect (ObjNum 60 snap 2 seq_num 60) 2026-02-17T05:47:47.946 INFO:tasks.rados.rados.0.trial115.stdout:271: write oid 1 current snap is 22 2026-02-17T05:47:47.946 INFO:tasks.rados.rados.0.trial115.stdout:266: finishing rollback tid 2 to trial11545420-18 2026-02-17T05:47:47.946 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 114 (ObjNum 102 snap 15 seq_num 102) dirty exists 2026-02-17T05:47:47.946 INFO:tasks.rados.rados.0.trial115.stdout:267: finishing write tid 1 to trial11545420-26 2026-02-17T05:47:47.946 INFO:tasks.rados.rados.0.trial115.stdout:271: seq_num 132 ranges {618524=710869,1857976=360548} 2026-02-17T05:47:47.949 INFO:tasks.rados.rados.0.trial115.stdout:271: writing trial11545420-1 from 618524 to 1329393 tid 1 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:271: writing trial11545420-1 from 1857976 to 2218524 tid 2 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:266: done (5 left) 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:272: delete oid 18 current snap is 22 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:267: finishing write tid 2 to trial11545420-26 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:267: finishing write tid 3 to trial11545420-26 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:267: finishing write tid 4 to trial11545420-26 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:267: oid 26 updating version 0 to 17 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:267: oid 26 version 17 is already newer than 15 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:267: oid 26 version 17 is already newer than 16 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 17 (ObjNum 131 snap 22 seq_num 131) dirty exists 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:267: left oid 26 (ObjNum 131 snap 22 seq_num 131) 2026-02-17T05:47:47.953 INFO:tasks.rados.rados.0.trial115.stdout:268: expect (ObjNum 37 snap 0 seq_num 37) 2026-02-17T05:47:48.015 INFO:tasks.rados.rados.0.trial115.stdout:270: expect (ObjNum 60 snap 2 seq_num 60) 2026-02-17T05:47:48.110 INFO:tasks.rados.rados.0.trial115.stdout:269: finishing copy_from racing read to trial11545420-16 2026-02-17T05:47:48.110 INFO:tasks.rados.rados.0.trial115.stdout:267: done (5 left) 2026-02-17T05:47:48.110 INFO:tasks.rados.rados.0.trial115.stdout:268: done (4 left) 2026-02-17T05:47:48.110 INFO:tasks.rados.rados.0.trial115.stdout:270: done (3 left) 2026-02-17T05:47:48.110 INFO:tasks.rados.rados.0.trial115.stdout:272: done (2 left) 2026-02-17T05:47:48.110 INFO:tasks.rados.rados.0.trial115.stdout:273: snap_remove snap 20 2026-02-17T05:47:48.931 INFO:tasks.rados.rados.0.trial115.stdout:273: done (2 left) 2026-02-17T05:47:48.931 INFO:tasks.rados.rados.0.trial115.stdout:274: snap_create 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:269: finishing copy_from to trial11545420-16 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 31 (ObjNum 99 snap 14 seq_num 99) dirty exists 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:271: finishing write tid 1 to trial11545420-1 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:271: finishing write tid 2 to trial11545420-1 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:271: finishing write tid 3 to trial11545420-1 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:271: finishing write tid 4 to trial11545420-1 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:271: oid 1 updating version 0 to 63 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:271: oid 1 updating version 63 to 64 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:271: oid 1 version 64 is already newer than 62 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 64 (ObjNum 132 snap 22 seq_num 132) dirty exists 2026-02-17T05:47:48.932 INFO:tasks.rados.rados.0.trial115.stdout:271: left oid 1 (ObjNum 132 snap 22 seq_num 132) 2026-02-17T05:47:49.933 INFO:tasks.rados.rados.0.trial115.stdout:269: done (2 left) 2026-02-17T05:47:49.934 INFO:tasks.rados.rados.0.trial115.stdout:271: done (1 left) 2026-02-17T05:47:49.934 INFO:tasks.rados.rados.0.trial115.stdout:274: done (0 left) 2026-02-17T05:47:49.934 INFO:tasks.rados.rados.0.trial115.stdout:275: snap_remove snap 22 2026-02-17T05:47:50.915 INFO:tasks.rados.rados.0.trial115.stdout:275: done (0 left) 2026-02-17T05:47:50.916 INFO:tasks.rados.rados.0.trial115.stdout:276: delete oid 5 current snap is 23 2026-02-17T05:47:50.918 INFO:tasks.rados.rados.0.trial115.stdout:276: done (0 left) 2026-02-17T05:47:50.918 INFO:tasks.rados.rados.0.trial115.stdout:277: read oid 46 snap 21 2026-02-17T05:47:50.918 INFO:tasks.rados.rados.0.trial115.stdout:277: expect deleted 2026-02-17T05:47:50.919 INFO:tasks.rados.rados.0.trial115.stdout:278: write oid 17 current snap is 23 2026-02-17T05:47:50.919 INFO:tasks.rados.rados.0.trial115.stdout:278: seq_num 135 ranges {668945=536616,1633422=491363} 2026-02-17T05:47:50.923 INFO:tasks.rados.rados.0.trial115.stdout:278: writing trial11545420-17 from 668945 to 1205561 tid 1 2026-02-17T05:47:50.927 INFO:tasks.rados.rados.0.trial115.stdout:278: writing trial11545420-17 from 1633422 to 2124785 tid 2 2026-02-17T05:47:50.927 INFO:tasks.rados.rados.0.trial115.stdout:279: delete oid 15 current snap is 23 2026-02-17T05:47:50.929 INFO:tasks.rados.rados.0.trial115.stdout:277: done (2 left) 2026-02-17T05:47:50.929 INFO:tasks.rados.rados.0.trial115.stdout:279: done (1 left) 2026-02-17T05:47:50.929 INFO:tasks.rados.rados.0.trial115.stdout:280: snap_create 2026-02-17T05:47:50.929 INFO:tasks.rados.rados.0.trial115.stdout:278: finishing write tid 1 to trial11545420-17 2026-02-17T05:47:50.929 INFO:tasks.rados.rados.0.trial115.stdout:278: finishing write tid 2 to trial11545420-17 2026-02-17T05:47:50.930 INFO:tasks.rados.rados.0.trial115.stdout:278: finishing write tid 3 to trial11545420-17 2026-02-17T05:47:50.930 INFO:tasks.rados.rados.0.trial115.stdout:278: finishing write tid 4 to trial11545420-17 2026-02-17T05:47:50.930 INFO:tasks.rados.rados.0.trial115.stdout:278: oid 17 updating version 0 to 37 2026-02-17T05:47:50.930 INFO:tasks.rados.rados.0.trial115.stdout:278: oid 17 version 37 is already newer than 35 2026-02-17T05:47:50.930 INFO:tasks.rados.rados.0.trial115.stdout:278: oid 17 version 37 is already newer than 36 2026-02-17T05:47:50.930 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 37 (ObjNum 135 snap 23 seq_num 135) dirty exists 2026-02-17T05:47:50.930 INFO:tasks.rados.rados.0.trial115.stdout:278: left oid 17 (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:47:51.918 INFO:tasks.rados.rados.0.trial115.stdout:278: done (1 left) 2026-02-17T05:47:51.918 INFO:tasks.rados.rados.0.trial115.stdout:280: done (0 left) 2026-02-17T05:47:51.918 INFO:tasks.rados.rados.0.trial115.stdout:281: snap_create 2026-02-17T05:47:52.919 INFO:tasks.rados.rados.0.trial115.stdout:281: done (0 left) 2026-02-17T05:47:52.919 INFO:tasks.rados.rados.0.trial115.stdout:282: write oid 47 current snap is 25 2026-02-17T05:47:52.919 INFO:tasks.rados.rados.0.trial115.stdout:282: seq_num 137 ranges {702559=403467,1750386=495032} 2026-02-17T05:47:52.923 INFO:tasks.rados.rados.0.trial115.stdout:282: writing trial11545420-47 from 702559 to 1106026 tid 1 2026-02-17T05:47:52.928 INFO:tasks.rados.rados.0.trial115.stdout:282: writing trial11545420-47 from 1750386 to 2245418 tid 2 2026-02-17T05:47:52.928 INFO:tasks.rados.rados.0.trial115.stdout:283: snap_remove snap 24 2026-02-17T05:47:53.921 INFO:tasks.rados.rados.0.trial115.stdout:283: done (1 left) 2026-02-17T05:47:53.922 INFO:tasks.rados.rados.0.trial115.stdout:284: write oid 31 current snap is 25 2026-02-17T05:47:53.922 INFO:tasks.rados.rados.0.trial115.stdout:284: seq_num 138 ranges {719366=478716,1808868=510498} 2026-02-17T05:47:53.927 INFO:tasks.rados.rados.0.trial115.stdout:284: writing trial11545420-31 from 719366 to 1198082 tid 1 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:284: writing trial11545420-31 from 1808868 to 2319366 tid 2 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:285: read oid 3 snap 21 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:285: expect (ObjNum 122 snap 20 seq_num 122) 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:286: copy_from oid 21 from oid 9 current snap is 25 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:282: finishing write tid 1 to trial11545420-47 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:282: finishing write tid 2 to trial11545420-47 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:282: finishing write tid 3 to trial11545420-47 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:282: finishing write tid 4 to trial11545420-47 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:282: oid 47 updating version 0 to 115 2026-02-17T05:47:53.932 INFO:tasks.rados.rados.0.trial115.stdout:282: oid 47 version 115 is already newer than 114 2026-02-17T05:47:53.933 INFO:tasks.rados.rados.0.trial115.stdout:282: oid 47 version 115 is already newer than 113 2026-02-17T05:47:53.933 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 115 (ObjNum 137 snap 25 seq_num 137) dirty exists 2026-02-17T05:47:53.933 INFO:tasks.rados.rados.0.trial115.stdout:282: left oid 47 (ObjNum 137 snap 25 seq_num 137) 2026-02-17T05:47:53.933 INFO:tasks.rados.rados.0.trial115.stdout:284: finishing write tid 1 to trial11545420-31 2026-02-17T05:47:53.933 INFO:tasks.rados.rados.0.trial115.stdout:282: done (3 left) 2026-02-17T05:47:53.933 INFO:tasks.rados.rados.0.trial115.stdout:287: snap_create 2026-02-17T05:47:53.934 INFO:tasks.rados.rados.0.trial115.stdout:284: finishing write tid 2 to trial11545420-31 2026-02-17T05:47:53.934 INFO:tasks.rados.rados.0.trial115.stdout:284: finishing write tid 3 to trial11545420-31 2026-02-17T05:47:53.934 INFO:tasks.rados.rados.0.trial115.stdout:284: finishing write tid 4 to trial11545420-31 2026-02-17T05:47:53.934 INFO:tasks.rados.rados.0.trial115.stdout:284: oid 31 updating version 0 to 116 2026-02-17T05:47:53.934 INFO:tasks.rados.rados.0.trial115.stdout:284: oid 31 updating version 116 to 119 2026-02-17T05:47:53.934 INFO:tasks.rados.rados.0.trial115.stdout:284: oid 31 updating version 119 to 120 2026-02-17T05:47:53.935 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 120 (ObjNum 138 snap 25 seq_num 138) dirty exists 2026-02-17T05:47:53.935 INFO:tasks.rados.rados.0.trial115.stdout:284: left oid 31 (ObjNum 138 snap 25 seq_num 138) 2026-02-17T05:47:53.935 INFO:tasks.rados.rados.0.trial115.stdout:285: expect (ObjNum 122 snap 20 seq_num 122) 2026-02-17T05:47:53.986 INFO:tasks.rados.rados.0.trial115.stdout:286: finishing copy_from racing read to trial11545420-21 2026-02-17T05:47:53.986 INFO:tasks.rados.rados.0.trial115.stdout:286: finishing copy_from to trial11545420-21 2026-02-17T05:47:53.986 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 59 (ObjNum 100 snap 14 seq_num 100) dirty exists 2026-02-17T05:47:54.924 INFO:tasks.rados.rados.0.trial115.stdout:284: done (3 left) 2026-02-17T05:47:54.925 INFO:tasks.rados.rados.0.trial115.stdout:285: done (2 left) 2026-02-17T05:47:54.925 INFO:tasks.rados.rados.0.trial115.stdout:286: done (1 left) 2026-02-17T05:47:54.925 INFO:tasks.rados.rados.0.trial115.stdout:287: done (0 left) 2026-02-17T05:47:54.925 INFO:tasks.rados.rados.0.trial115.stdout:288: delete oid 35 current snap is 26 2026-02-17T05:47:54.927 INFO:tasks.rados.rados.0.trial115.stdout:288: done (0 left) 2026-02-17T05:47:54.927 INFO:tasks.rados.rados.0.trial115.stdout:289: read oid 25 snap -1 2026-02-17T05:47:54.927 INFO:tasks.rados.rados.0.trial115.stdout:289: expect (ObjNum 117 snap 19 seq_num 117) 2026-02-17T05:47:54.927 INFO:tasks.rados.rados.0.trial115.stdout:290: snap_create 2026-02-17T05:47:54.936 INFO:tasks.rados.rados.0.trial115.stdout:289: expect (ObjNum 117 snap 19 seq_num 117) 2026-02-17T05:47:55.917 INFO:tasks.rados.rados.0.trial115.stdout:289: done (1 left) 2026-02-17T05:47:55.917 INFO:tasks.rados.rados.0.trial115.stdout:290: done (0 left) 2026-02-17T05:47:55.917 INFO:tasks.rados.rados.0.trial115.stdout:291: snap_remove snap 25 2026-02-17T05:47:56.289 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T05:47:56.289 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T05:47:56.289 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T05:47:56.290 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T05:47:56.293 INFO:tasks.ceph.mon.b:Started 2026-02-17T05:47:56.293 INFO:tasks.mon_thrash:Sending STOP to mon a 2026-02-17T05:47:56.293 INFO:tasks.ceph.mon.a:Sent signal 19 2026-02-17T05:47:56.293 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T05:48:02.033 INFO:tasks.rados.rados.0.trial115.stdout:291: done (0 left) 2026-02-17T05:48:02.034 INFO:tasks.rados.rados.0.trial115.stdout:292: delete oid 27 current snap is 27 2026-02-17T05:48:04.169 INFO:tasks.rados.rados.0.trial115.stdout:292: done (0 left) 2026-02-17T05:48:04.169 INFO:tasks.rados.rados.0.trial115.stdout:293: delete oid 46 current snap is 27 2026-02-17T05:48:04.170 INFO:tasks.rados.rados.0.trial115.stdout:293: done (0 left) 2026-02-17T05:48:04.170 INFO:tasks.rados.rados.0.trial115.stdout:294: write (excl) oid 10 current snap is 27 2026-02-17T05:48:04.170 INFO:tasks.rados.rados.0.trial115.stdout:294: seq_num 142 ranges {786594=779712,2326443=60151} 2026-02-17T05:48:04.175 INFO:tasks.rados.rados.0.trial115.stdout:294: writing trial11545420-10 from 786594 to 1566306 tid 1 2026-02-17T05:48:04.179 INFO:tasks.rados.rados.0.trial115.stdout:294: writing trial11545420-10 from 2326443 to 2386594 tid 2 2026-02-17T05:48:04.179 INFO:tasks.rados.rados.0.trial115.stdout:295: snap_create 2026-02-17T05:48:04.179 INFO:tasks.rados.rados.0.trial115.stdout:294: finishing write tid 1 to trial11545420-10 2026-02-17T05:48:04.179 INFO:tasks.rados.rados.0.trial115.stdout:294: finishing write tid 2 to trial11545420-10 2026-02-17T05:48:04.180 INFO:tasks.rados.rados.0.trial115.stdout:294: finishing write tid 3 to trial11545420-10 2026-02-17T05:48:04.180 INFO:tasks.rados.rados.0.trial115.stdout:294: finishing write tid 4 to trial11545420-10 2026-02-17T05:48:04.180 INFO:tasks.rados.rados.0.trial115.stdout:294: oid 10 updating version 0 to 115 2026-02-17T05:48:04.180 INFO:tasks.rados.rados.0.trial115.stdout:294: oid 10 version 115 is already newer than 114 2026-02-17T05:48:04.180 INFO:tasks.rados.rados.0.trial115.stdout:294: oid 10 version 115 is already newer than 113 2026-02-17T05:48:04.180 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 115 (ObjNum 142 snap 27 seq_num 142) dirty exists 2026-02-17T05:48:04.180 INFO:tasks.rados.rados.0.trial115.stdout:294: left oid 10 (ObjNum 142 snap 27 seq_num 142) 2026-02-17T05:48:04.231 INFO:tasks.rados.rados.0.trial115.stdout:294: done (1 left) 2026-02-17T05:48:04.231 INFO:tasks.rados.rados.0.trial115.stdout:295: done (0 left) 2026-02-17T05:48:04.231 INFO:tasks.rados.rados.0.trial115.stdout:296: delete oid 10 current snap is 28 2026-02-17T05:48:04.234 INFO:tasks.rados.rados.0.trial115.stdout:296: done (0 left) 2026-02-17T05:48:04.234 INFO:tasks.rados.rados.0.trial115.stdout:297: write (excl) oid 45 current snap is 28 2026-02-17T05:48:04.234 INFO:tasks.rados.rados.0.trial115.stdout:297: seq_num 144 ranges {420208=530210,1527054=766050} 2026-02-17T05:48:04.238 INFO:tasks.rados.rados.0.trial115.stdout:297: writing trial11545420-45 from 420208 to 950418 tid 1 2026-02-17T05:48:04.243 INFO:tasks.rados.rados.0.trial115.stdout:297: writing trial11545420-45 from 1527054 to 2293104 tid 2 2026-02-17T05:48:04.244 INFO:tasks.rados.rados.0.trial115.stdout:298: snap_remove snap 21 2026-02-17T05:48:05.235 INFO:tasks.rados.rados.0.trial115.stdout:298: done (1 left) 2026-02-17T05:48:05.235 INFO:tasks.rados.rados.0.trial115.stdout:299: copy_from oid 6 from oid 32 current snap is 28 2026-02-17T05:48:05.235 INFO:tasks.rados.rados.0.trial115.stdout:297: finishing write tid 1 to trial11545420-45 2026-02-17T05:48:05.235 INFO:tasks.rados.rados.0.trial115.stdout:297: finishing write tid 2 to trial11545420-45 2026-02-17T05:48:05.235 INFO:tasks.rados.rados.0.trial115.stdout:297: finishing write tid 3 to trial11545420-45 2026-02-17T05:48:05.235 INFO:tasks.rados.rados.0.trial115.stdout:297: finishing write tid 4 to trial11545420-45 2026-02-17T05:48:05.235 INFO:tasks.rados.rados.0.trial115.stdout:297: oid 45 updating version 0 to 154 2026-02-17T05:48:05.236 INFO:tasks.rados.rados.0.trial115.stdout:297: oid 45 version 154 is already newer than 153 2026-02-17T05:48:05.236 INFO:tasks.rados.rados.0.trial115.stdout:297: oid 45 updating version 154 to 155 2026-02-17T05:48:05.236 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 155 (ObjNum 144 snap 28 seq_num 144) dirty exists 2026-02-17T05:48:05.236 INFO:tasks.rados.rados.0.trial115.stdout:297: left oid 45 (ObjNum 144 snap 28 seq_num 144) 2026-02-17T05:48:05.236 INFO:tasks.rados.rados.0.trial115.stdout:297: done (1 left) 2026-02-17T05:48:05.236 INFO:tasks.rados.rados.0.trial115.stdout:300: delete oid 38 current snap is 28 2026-02-17T05:48:05.496 INFO:tasks.rados.rados.0.trial115.stdout:300: done (1 left) 2026-02-17T05:48:05.496 INFO:tasks.rados.rados.0.trial115.stdout:301: delete oid 50 current snap is 28 2026-02-17T05:48:05.497 INFO:tasks.rados.rados.0.trial115.stdout:299: finishing copy_from racing read to trial11545420-6 2026-02-17T05:48:05.498 INFO:tasks.rados.rados.0.trial115.stdout:301: done (1 left) 2026-02-17T05:48:05.498 INFO:tasks.rados.rados.0.trial115.stdout:302: copy_from oid 36 from oid 42 current snap is 28 2026-02-17T05:48:05.498 INFO:tasks.rados.rados.0.trial115.stdout:303: read oid 43 snap -1 2026-02-17T05:48:05.499 INFO:tasks.rados.rados.0.trial115.stdout:303: expect (ObjNum 108 snap 16 seq_num 108) 2026-02-17T05:48:05.499 INFO:tasks.rados.rados.0.trial115.stdout:304: write (excl) oid 3 current snap is 28 2026-02-17T05:48:05.499 INFO:tasks.rados.rados.0.trial115.stdout:304: seq_num 147 ranges {470629=472310,1418853=513377} 2026-02-17T05:48:05.500 INFO:tasks.rados.rados.0.trial115.stdout:304: writing trial11545420-3 from 470629 to 942939 tid 1 2026-02-17T05:48:05.505 INFO:tasks.rados.rados.0.trial115.stdout:304: writing trial11545420-3 from 1418853 to 1932230 tid 2 2026-02-17T05:48:05.505 INFO:tasks.rados.rados.0.trial115.stdout:305: read oid 26 snap -1 2026-02-17T05:48:05.506 INFO:tasks.rados.rados.0.trial115.stdout:305: expect (ObjNum 131 snap 22 seq_num 131) 2026-02-17T05:48:05.506 INFO:tasks.rados.rados.0.trial115.stdout:306: copy_from oid 45 from oid 47 current snap is 28 2026-02-17T05:48:05.506 INFO:tasks.rados.rados.0.trial115.stdout:302: finishing copy_from to trial11545420-36 2026-02-17T05:48:05.506 INFO:tasks.rados.rados.0.trial115.stdout:302: got expected ENOENT (src dne) 2026-02-17T05:48:05.506 INFO:tasks.rados.rados.0.trial115.stdout:302: finishing copy_from racing read to trial11545420-36 2026-02-17T05:48:05.506 INFO:tasks.rados.rados.0.trial115.stdout:299: finishing copy_from to trial11545420-6 2026-02-17T05:48:05.507 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 15 (ObjNum 75 snap 8 seq_num 75) dirty exists 2026-02-17T05:48:05.507 INFO:tasks.rados.rados.0.trial115.stdout:303: expect (ObjNum 108 snap 16 seq_num 108) 2026-02-17T05:48:05.628 INFO:tasks.rados.rados.0.trial115.stdout:304: finishing write tid 1 to trial11545420-3 2026-02-17T05:48:05.628 INFO:tasks.rados.rados.0.trial115.stdout:306: finishing copy_from racing read to trial11545420-45 2026-02-17T05:48:05.628 INFO:tasks.rados.rados.0.trial115.stdout:305: expect (ObjNum 131 snap 22 seq_num 131) 2026-02-17T05:48:05.697 INFO:tasks.rados.rados.0.trial115.stdout:306: finishing copy_from to trial11545420-45 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 115 (ObjNum 137 snap 25 seq_num 137) dirty exists 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:304: finishing write tid 2 to trial11545420-3 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:304: finishing write tid 3 to trial11545420-3 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:304: finishing write tid 4 to trial11545420-3 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:304: oid 3 updating version 0 to 129 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:304: oid 3 version 129 is already newer than 128 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:304: oid 3 updating version 129 to 130 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 130 (ObjNum 147 snap 28 seq_num 147) dirty exists 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:304: left oid 3 (ObjNum 147 snap 28 seq_num 147) 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:299: done (5 left) 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:302: done (4 left) 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:303: done (3 left) 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:304: done (2 left) 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:305: done (1 left) 2026-02-17T05:48:05.698 INFO:tasks.rados.rados.0.trial115.stdout:306: done (0 left) 2026-02-17T05:48:05.699 INFO:tasks.rados.rados.0.trial115.stdout:307: copy_from oid 6 from oid 42 current snap is 28 2026-02-17T05:48:05.699 INFO:tasks.rados.rados.0.trial115.stdout:308: delete oid 32 current snap is 28 2026-02-17T05:48:05.699 INFO:tasks.rados.rados.0.trial115.stdout:308: done (1 left) 2026-02-17T05:48:05.699 INFO:tasks.rados.rados.0.trial115.stdout:309: write (excl) oid 29 current snap is 28 2026-02-17T05:48:05.699 INFO:tasks.rados.rados.0.trial115.stdout:309: seq_num 149 ranges {504243=622808,1819464=517046} 2026-02-17T05:48:05.703 INFO:tasks.rados.rados.0.trial115.stdout:309: writing trial11545420-29 from 504243 to 1127051 tid 1 2026-02-17T05:48:05.708 INFO:tasks.rados.rados.0.trial115.stdout:309: writing trial11545420-29 from 1819464 to 2336510 tid 2 2026-02-17T05:48:05.708 INFO:tasks.rados.rados.0.trial115.stdout:310: read oid 41 snap -1 2026-02-17T05:48:05.708 INFO:tasks.rados.rados.0.trial115.stdout:310: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:48:05.709 INFO:tasks.rados.rados.0.trial115.stdout:311: write (excl) oid 30 current snap is 28 2026-02-17T05:48:05.709 INFO:tasks.rados.rados.0.trial115.stdout:307: finishing copy_from to trial11545420-6 2026-02-17T05:48:05.709 INFO:tasks.rados.rados.0.trial115.stdout:307: got expected ENOENT (src dne) 2026-02-17T05:48:05.709 INFO:tasks.rados.rados.0.trial115.stdout:307: finishing copy_from racing read to trial11545420-6 2026-02-17T05:48:05.709 INFO:tasks.rados.rados.0.trial115.stdout:309: finishing write tid 1 to trial11545420-29 2026-02-17T05:48:05.709 INFO:tasks.rados.rados.0.trial115.stdout:311: seq_num 150 ranges {521050=698057,1877946=643104} 2026-02-17T05:48:05.713 INFO:tasks.rados.rados.0.trial115.stdout:311: writing trial11545420-30 from 521050 to 1219107 tid 1 2026-02-17T05:48:05.717 INFO:tasks.rados.rados.0.trial115.stdout:311: writing trial11545420-30 from 1877946 to 2521050 tid 2 2026-02-17T05:48:05.718 INFO:tasks.rados.rados.0.trial115.stdout:307: done (3 left) 2026-02-17T05:48:05.718 INFO:tasks.rados.rados.0.trial115.stdout:312: write (excl) oid 12 current snap is 28 2026-02-17T05:48:05.718 INFO:tasks.rados.rados.0.trial115.stdout:312: seq_num 151 ranges {537857=773306,1936428=520715} 2026-02-17T05:48:05.722 INFO:tasks.rados.rados.0.trial115.stdout:312: writing trial11545420-12 from 537857 to 1311163 tid 1 2026-02-17T05:48:05.727 INFO:tasks.rados.rados.0.trial115.stdout:312: writing trial11545420-12 from 1936428 to 2457143 tid 2 2026-02-17T05:48:05.727 INFO:tasks.rados.rados.0.trial115.stdout:313: read oid 24 snap -1 2026-02-17T05:48:05.727 INFO:tasks.rados.rados.0.trial115.stdout:313: expect (ObjNum 115 snap 19 seq_num 115) 2026-02-17T05:48:05.727 INFO:tasks.rados.rados.0.trial115.stdout:314: write (excl) oid 43 current snap is 28 2026-02-17T05:48:05.727 INFO:tasks.rados.rados.0.trial115.stdout:314: seq_num 152 ranges {554664=448555,1478557=664373} 2026-02-17T05:48:05.731 INFO:tasks.rados.rados.0.trial115.stdout:314: writing trial11545420-43 from 554664 to 1003219 tid 1 2026-02-17T05:48:05.735 INFO:tasks.rados.rados.0.trial115.stdout:314: writing trial11545420-43 from 1478557 to 2142930 tid 2 2026-02-17T05:48:05.735 INFO:tasks.rados.rados.0.trial115.stdout:315: rollback oid 8 current snap is 28 2026-02-17T05:48:05.735 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 8 to 26 2026-02-17T05:48:05.735 INFO:tasks.rados.rados.0.trial115.stdout:310: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:48:05.800 INFO:tasks.rados.rados.0.trial115.stdout:309: finishing write tid 2 to trial11545420-29 2026-02-17T05:48:05.800 INFO:tasks.rados.rados.0.trial115.stdout:309: finishing write tid 3 to trial11545420-29 2026-02-17T05:48:05.800 INFO:tasks.rados.rados.0.trial115.stdout:309: finishing write tid 4 to trial11545420-29 2026-02-17T05:48:05.800 INFO:tasks.rados.rados.0.trial115.stdout:309: oid 29 updating version 0 to 73 2026-02-17T05:48:05.800 INFO:tasks.rados.rados.0.trial115.stdout:309: oid 29 version 73 is already newer than 72 2026-02-17T05:48:05.800 INFO:tasks.rados.rados.0.trial115.stdout:309: oid 29 version 73 is already newer than 71 2026-02-17T05:48:05.800 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 73 (ObjNum 149 snap 28 seq_num 149) dirty exists 2026-02-17T05:48:05.800 INFO:tasks.rados.rados.0.trial115.stdout:309: left oid 29 (ObjNum 149 snap 28 seq_num 149) 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:311: finishing write tid 1 to trial11545420-30 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:311: finishing write tid 2 to trial11545420-30 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:311: finishing write tid 3 to trial11545420-30 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:311: finishing write tid 4 to trial11545420-30 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:311: oid 30 updating version 0 to 126 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:311: oid 30 updating version 126 to 128 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:311: oid 30 version 128 is already newer than 127 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 128 (ObjNum 150 snap 28 seq_num 150) dirty exists 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:311: left oid 30 (ObjNum 150 snap 28 seq_num 150) 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:312: finishing write tid 1 to trial11545420-12 2026-02-17T05:48:05.801 INFO:tasks.rados.rados.0.trial115.stdout:312: finishing write tid 2 to trial11545420-12 2026-02-17T05:48:05.802 INFO:tasks.rados.rados.0.trial115.stdout:312: finishing write tid 3 to trial11545420-12 2026-02-17T05:48:05.802 INFO:tasks.rados.rados.0.trial115.stdout:312: finishing write tid 4 to trial11545420-12 2026-02-17T05:48:05.802 INFO:tasks.rados.rados.0.trial115.stdout:312: oid 12 updating version 0 to 158 2026-02-17T05:48:05.802 INFO:tasks.rados.rados.0.trial115.stdout:312: oid 12 updating version 158 to 159 2026-02-17T05:48:05.802 INFO:tasks.rados.rados.0.trial115.stdout:312: oid 12 updating version 159 to 160 2026-02-17T05:48:05.802 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 160 (ObjNum 151 snap 28 seq_num 151) dirty exists 2026-02-17T05:48:05.802 INFO:tasks.rados.rados.0.trial115.stdout:312: left oid 12 (ObjNum 151 snap 28 seq_num 151) 2026-02-17T05:48:05.802 INFO:tasks.rados.rados.0.trial115.stdout:313: expect (ObjNum 115 snap 19 seq_num 115) 2026-02-17T05:48:05.925 INFO:tasks.rados.rados.0.trial115.stdout:314: finishing write tid 1 to trial11545420-43 2026-02-17T05:48:05.925 INFO:tasks.rados.rados.0.trial115.stdout:315: finishing rollback tid 0 to trial11545420-8 2026-02-17T05:48:05.925 INFO:tasks.rados.rados.0.trial115.stdout:315: finishing rollback tid 1 to trial11545420-8 2026-02-17T05:48:05.925 INFO:tasks.rados.rados.0.trial115.stdout:314: finishing write tid 2 to trial11545420-43 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:315: finishing rollback tid 2 to trial11545420-8 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 14 (ObjNum 127 snap 20 seq_num 127) dirty exists 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:314: finishing write tid 3 to trial11545420-43 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:309: done (6 left) 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:310: done (5 left) 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:311: done (4 left) 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:312: done (3 left) 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:313: done (2 left) 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:315: done (1 left) 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:316: write oid 1 current snap is 28 2026-02-17T05:48:05.926 INFO:tasks.rados.rados.0.trial115.stdout:314: finishing write tid 4 to trial11545420-43 2026-02-17T05:48:05.927 INFO:tasks.rados.rados.0.trial115.stdout:314: oid 43 updating version 0 to 81 2026-02-17T05:48:05.927 INFO:tasks.rados.rados.0.trial115.stdout:314: oid 43 version 81 is already newer than 80 2026-02-17T05:48:05.927 INFO:tasks.rados.rados.0.trial115.stdout:314: oid 43 updating version 81 to 82 2026-02-17T05:48:05.927 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 82 (ObjNum 152 snap 28 seq_num 152) dirty exists 2026-02-17T05:48:05.927 INFO:tasks.rados.rados.0.trial115.stdout:314: left oid 43 (ObjNum 152 snap 28 seq_num 152) 2026-02-17T05:48:05.927 INFO:tasks.rados.rados.0.trial115.stdout:316: seq_num 153 ranges {571471=640157,1653392=524384} 2026-02-17T05:48:05.929 INFO:tasks.rados.rados.0.trial115.stdout:316: writing trial11545420-1 from 571471 to 1211628 tid 1 2026-02-17T05:48:05.933 INFO:tasks.rados.rados.0.trial115.stdout:316: writing trial11545420-1 from 1653392 to 2177776 tid 2 2026-02-17T05:48:05.933 INFO:tasks.rados.rados.0.trial115.stdout:314: done (1 left) 2026-02-17T05:48:05.933 INFO:tasks.rados.rados.0.trial115.stdout:317: delete oid 50 current snap is 28 2026-02-17T05:48:05.933 INFO:tasks.rados.rados.0.trial115.stdout:316: finishing write tid 1 to trial11545420-1 2026-02-17T05:48:05.933 INFO:tasks.rados.rados.0.trial115.stdout:317: done (1 left) 2026-02-17T05:48:05.934 INFO:tasks.rados.rados.0.trial115.stdout:318: write oid 15 current snap is 28 2026-02-17T05:48:05.934 INFO:tasks.rados.rados.0.trial115.stdout:318: seq_num 155 ranges {605085=790655,2170356=434729} 2026-02-17T05:48:05.939 INFO:tasks.rados.rados.0.trial115.stdout:318: writing trial11545420-15 from 605085 to 1395740 tid 1 2026-02-17T05:48:05.943 INFO:tasks.rados.rados.0.trial115.stdout:318: writing trial11545420-15 from 2170356 to 2605085 tid 2 2026-02-17T05:48:05.943 INFO:tasks.rados.rados.0.trial115.stdout:319: read oid 31 snap -1 2026-02-17T05:48:05.944 INFO:tasks.rados.rados.0.trial115.stdout:319: expect (ObjNum 138 snap 25 seq_num 138) 2026-02-17T05:48:05.944 INFO:tasks.rados.rados.0.trial115.stdout:320: write (excl) oid 45 current snap is 28 2026-02-17T05:48:05.944 INFO:tasks.rados.rados.0.trial115.stdout:320: seq_num 156 ranges {621892=465904,1712485=671711} 2026-02-17T05:48:05.947 INFO:tasks.rados.rados.0.trial115.stdout:320: writing trial11545420-45 from 621892 to 1087796 tid 1 2026-02-17T05:48:05.953 INFO:tasks.rados.rados.0.trial115.stdout:320: writing trial11545420-45 from 1712485 to 2384196 tid 2 2026-02-17T05:48:05.953 INFO:tasks.rados.rados.0.trial115.stdout:321: read oid 29 snap 27 2026-02-17T05:48:05.953 INFO:tasks.rados.rados.0.trial115.stdout:321: expect deleted 2026-02-17T05:48:05.953 INFO:tasks.rados.rados.0.trial115.stdout:322: rollback oid 26 current snap is 28 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 26 to 23 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:316: finishing write tid 2 to trial11545420-1 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:316: finishing write tid 3 to trial11545420-1 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:316: finishing write tid 4 to trial11545420-1 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:316: oid 1 updating version 0 to 71 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:316: oid 1 updating version 71 to 72 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:316: oid 1 version 72 is already newer than 70 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 72 (ObjNum 153 snap 28 seq_num 153) dirty exists 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:316: left oid 1 (ObjNum 153 snap 28 seq_num 153) 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:318: finishing write tid 1 to trial11545420-15 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:318: finishing write tid 2 to trial11545420-15 2026-02-17T05:48:05.954 INFO:tasks.rados.rados.0.trial115.stdout:318: finishing write tid 3 to trial11545420-15 2026-02-17T05:48:05.955 INFO:tasks.rados.rados.0.trial115.stdout:318: finishing write tid 4 to trial11545420-15 2026-02-17T05:48:05.955 INFO:tasks.rados.rados.0.trial115.stdout:318: oid 15 updating version 0 to 122 2026-02-17T05:48:05.955 INFO:tasks.rados.rados.0.trial115.stdout:318: oid 15 updating version 122 to 124 2026-02-17T05:48:05.955 INFO:tasks.rados.rados.0.trial115.stdout:318: oid 15 version 124 is already newer than 123 2026-02-17T05:48:05.955 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 124 (ObjNum 155 snap 28 seq_num 155) dirty exists 2026-02-17T05:48:05.955 INFO:tasks.rados.rados.0.trial115.stdout:318: left oid 15 (ObjNum 155 snap 28 seq_num 155) 2026-02-17T05:48:05.955 INFO:tasks.rados.rados.0.trial115.stdout:319: expect (ObjNum 138 snap 25 seq_num 138) 2026-02-17T05:48:06.008 INFO:tasks.rados.rados.0.trial115.stdout:320: finishing write tid 1 to trial11545420-45 2026-02-17T05:48:06.008 INFO:tasks.rados.rados.0.trial115.stdout:320: finishing write tid 2 to trial11545420-45 2026-02-17T05:48:06.008 INFO:tasks.rados.rados.0.trial115.stdout:320: finishing write tid 3 to trial11545420-45 2026-02-17T05:48:06.008 INFO:tasks.rados.rados.0.trial115.stdout:320: finishing write tid 4 to trial11545420-45 2026-02-17T05:48:06.008 INFO:tasks.rados.rados.0.trial115.stdout:320: oid 45 updating version 0 to 163 2026-02-17T05:48:06.008 INFO:tasks.rados.rados.0.trial115.stdout:320: oid 45 version 163 is already newer than 162 2026-02-17T05:48:06.008 INFO:tasks.rados.rados.0.trial115.stdout:320: oid 45 version 163 is already newer than 161 2026-02-17T05:48:06.008 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 163 (ObjNum 156 snap 28 seq_num 156) dirty exists 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:320: left oid 45 (ObjNum 156 snap 28 seq_num 156) 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:322: finishing rollback tid 0 to trial11545420-26 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:322: finishing rollback tid 1 to trial11545420-26 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:322: finishing rollback tid 2 to trial11545420-26 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 25 (ObjNum 131 snap 22 seq_num 131) dirty exists 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:316: done (5 left) 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:318: done (4 left) 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:319: done (3 left) 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:320: done (2 left) 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:321: done (1 left) 2026-02-17T05:48:06.009 INFO:tasks.rados.rados.0.trial115.stdout:322: done (0 left) 2026-02-17T05:48:06.010 INFO:tasks.rados.rados.0.trial115.stdout:323: snap_create 2026-02-17T05:48:06.239 INFO:tasks.rados.rados.0.trial115.stdout:323: done (0 left) 2026-02-17T05:48:06.239 INFO:tasks.rados.rados.0.trial115.stdout:324: copy_from oid 38 from oid 23 current snap is 29 2026-02-17T05:48:06.239 INFO:tasks.rados.rados.0.trial115.stdout:325: read oid 15 snap -1 2026-02-17T05:48:06.239 INFO:tasks.rados.rados.0.trial115.stdout:325: expect (ObjNum 155 snap 28 seq_num 155) 2026-02-17T05:48:06.239 INFO:tasks.rados.rados.0.trial115.stdout:326: read oid 34 snap -1 2026-02-17T05:48:06.240 INFO:tasks.rados.rados.0.trial115.stdout:326: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:48:06.240 INFO:tasks.rados.rados.0.trial115.stdout:327: write oid 20 current snap is 29 2026-02-17T05:48:06.240 INFO:tasks.rados.rados.0.trial115.stdout:327: seq_num 157 ranges {638699=541153,1770967=531722} 2026-02-17T05:48:06.243 INFO:tasks.rados.rados.0.trial115.stdout:327: writing trial11545420-20 from 638699 to 1179852 tid 1 2026-02-17T05:48:06.247 INFO:tasks.rados.rados.0.trial115.stdout:327: writing trial11545420-20 from 1770967 to 2302689 tid 2 2026-02-17T05:48:06.248 INFO:tasks.rados.rados.0.trial115.stdout:328: read oid 48 snap -1 2026-02-17T05:48:06.248 INFO:tasks.rados.rados.0.trial115.stdout:328: expect (ObjNum 103 snap 15 seq_num 103) 2026-02-17T05:48:06.248 INFO:tasks.rados.rados.0.trial115.stdout:329: read oid 43 snap -1 2026-02-17T05:48:06.248 INFO:tasks.rados.rados.0.trial115.stdout:329: expect (ObjNum 152 snap 28 seq_num 152) 2026-02-17T05:48:06.248 INFO:tasks.rados.rados.0.trial115.stdout:330: delete oid 8 current snap is 29 2026-02-17T05:48:06.248 INFO:tasks.rados.rados.0.trial115.stdout:325: expect (ObjNum 155 snap 28 seq_num 155) 2026-02-17T05:48:06.307 INFO:tasks.rados.rados.0.trial115.stdout:327: finishing write tid 1 to trial11545420-20 2026-02-17T05:48:06.307 INFO:tasks.rados.rados.0.trial115.stdout:326: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:48:06.417 INFO:tasks.rados.rados.0.trial115.stdout:327: finishing write tid 2 to trial11545420-20 2026-02-17T05:48:06.417 INFO:tasks.rados.rados.0.trial115.stdout:327: finishing write tid 3 to trial11545420-20 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:325: done (6 left) 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:326: done (5 left) 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:330: done (4 left) 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:331: delete oid 32 current snap is 29 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:327: finishing write tid 4 to trial11545420-20 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:327: oid 20 updating version 0 to 84 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:327: oid 20 updating version 84 to 86 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:327: oid 20 version 86 is already newer than 85 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 86 (ObjNum 157 snap 29 seq_num 157) dirty exists 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:327: left oid 20 (ObjNum 157 snap 29 seq_num 157) 2026-02-17T05:48:06.418 INFO:tasks.rados.rados.0.trial115.stdout:329: expect (ObjNum 152 snap 28 seq_num 152) 2026-02-17T05:48:06.510 INFO:tasks.rados.rados.0.trial115.stdout:324: finishing copy_from racing read to trial11545420-38 2026-02-17T05:48:06.510 INFO:tasks.rados.rados.0.trial115.stdout:328: expect (ObjNum 103 snap 15 seq_num 103) 2026-02-17T05:48:06.597 INFO:tasks.rados.rados.0.trial115.stdout:324: finishing copy_from to trial11545420-38 2026-02-17T05:48:06.597 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2026-02-17T05:48:06.597 INFO:tasks.rados.rados.0.trial115.stdout:324: done (4 left) 2026-02-17T05:48:06.597 INFO:tasks.rados.rados.0.trial115.stdout:327: done (3 left) 2026-02-17T05:48:06.597 INFO:tasks.rados.rados.0.trial115.stdout:328: done (2 left) 2026-02-17T05:48:06.597 INFO:tasks.rados.rados.0.trial115.stdout:329: done (1 left) 2026-02-17T05:48:06.597 INFO:tasks.rados.rados.0.trial115.stdout:331: done (0 left) 2026-02-17T05:48:06.598 INFO:tasks.rados.rados.0.trial115.stdout:332: read oid 45 snap -1 2026-02-17T05:48:06.598 INFO:tasks.rados.rados.0.trial115.stdout:332: expect (ObjNum 156 snap 28 seq_num 156) 2026-02-17T05:48:06.598 INFO:tasks.rados.rados.0.trial115.stdout:333: write oid 50 current snap is 29 2026-02-17T05:48:06.598 INFO:tasks.rados.rados.0.trial115.stdout:333: seq_num 160 ranges {689120=483253,1946413=679049} 2026-02-17T05:48:06.600 INFO:tasks.rados.rados.0.trial115.stdout:333: writing trial11545420-50 from 689120 to 1172373 tid 1 2026-02-17T05:48:06.606 INFO:tasks.rados.rados.0.trial115.stdout:333: writing trial11545420-50 from 1946413 to 2625462 tid 2 2026-02-17T05:48:06.606 INFO:tasks.rados.rados.0.trial115.stdout:334: read oid 6 snap 27 2026-02-17T05:48:06.606 INFO:tasks.rados.rados.0.trial115.stdout:334: expect (ObjNum 5 snap 0 seq_num 5) 2026-02-17T05:48:06.607 INFO:tasks.rados.rados.0.trial115.stdout:335: read oid 19 snap 26 2026-02-17T05:48:06.607 INFO:tasks.rados.rados.0.trial115.stdout:335: expect (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:48:06.607 INFO:tasks.rados.rados.0.trial115.stdout:336: write oid 35 current snap is 29 2026-02-17T05:48:06.607 INFO:tasks.rados.rados.0.trial115.stdout:336: seq_num 161 ranges {705927=558502,2004895=422707} 2026-02-17T05:48:06.610 INFO:tasks.rados.rados.0.trial115.stdout:336: writing trial11545420-35 from 705927 to 1264429 tid 1 2026-02-17T05:48:06.615 INFO:tasks.rados.rados.0.trial115.stdout:336: writing trial11545420-35 from 2004895 to 2427602 tid 2 2026-02-17T05:48:06.615 INFO:tasks.rados.rados.0.trial115.stdout:337: copy_from oid 43 from oid 15 current snap is 29 2026-02-17T05:48:06.615 INFO:tasks.rados.rados.0.trial115.stdout:332: expect (ObjNum 156 snap 28 seq_num 156) 2026-02-17T05:48:06.715 INFO:tasks.rados.rados.0.trial115.stdout:333: finishing write tid 1 to trial11545420-50 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:333: finishing write tid 2 to trial11545420-50 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:333: finishing write tid 3 to trial11545420-50 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:333: finishing write tid 4 to trial11545420-50 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:333: oid 50 updating version 0 to 131 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:333: oid 50 version 131 is already newer than 129 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:333: oid 50 version 131 is already newer than 130 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 131 (ObjNum 160 snap 29 seq_num 160) dirty exists 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:333: left oid 50 (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:48:06.716 INFO:tasks.rados.rados.0.trial115.stdout:335: expect (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:48:06.839 INFO:tasks.rados.rados.0.trial115.stdout:336: finishing write tid 1 to trial11545420-35 2026-02-17T05:48:06.839 INFO:tasks.rados.rados.0.trial115.stdout:336: finishing write tid 2 to trial11545420-35 2026-02-17T05:48:06.839 INFO:tasks.rados.rados.0.trial115.stdout:336: finishing write tid 3 to trial11545420-35 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:336: finishing write tid 4 to trial11545420-35 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:336: oid 35 updating version 0 to 137 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:336: oid 35 updating version 137 to 138 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:336: oid 35 version 138 is already newer than 136 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 138 (ObjNum 161 snap 29 seq_num 161) dirty exists 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:336: left oid 35 (ObjNum 161 snap 29 seq_num 161) 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:337: finishing copy_from racing read to trial11545420-43 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:332: done (5 left) 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:333: done (4 left) 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:335: done (3 left) 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:336: done (2 left) 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:338: write (excl) oid 18 current snap is 29 2026-02-17T05:48:06.840 INFO:tasks.rados.rados.0.trial115.stdout:338: seq_num 162 ranges {722734=633751,2063377=659357} 2026-02-17T05:48:06.844 INFO:tasks.rados.rados.0.trial115.stdout:338: writing trial11545420-18 from 722734 to 1356485 tid 1 2026-02-17T05:48:06.849 INFO:tasks.rados.rados.0.trial115.stdout:338: writing trial11545420-18 from 2063377 to 2722734 tid 2 2026-02-17T05:48:06.849 INFO:tasks.rados.rados.0.trial115.stdout:339: snap_remove snap 23 2026-02-17T05:48:07.358 INFO:tasks.rados.rados.0.trial115.stdout:339: done (3 left) 2026-02-17T05:48:07.358 INFO:tasks.rados.rados.0.trial115.stdout:340: write oid 29 current snap is 29 2026-02-17T05:48:07.358 INFO:tasks.rados.rados.0.trial115.stdout:340: seq_num 163 ranges {739541=709000,2121859=426376} 2026-02-17T05:48:07.364 INFO:tasks.rados.rados.0.trial115.stdout:340: writing trial11545420-29 from 739541 to 1448541 tid 1 2026-02-17T05:48:07.368 INFO:tasks.rados.rados.0.trial115.stdout:340: writing trial11545420-29 from 2121859 to 2548235 tid 2 2026-02-17T05:48:07.368 INFO:tasks.rados.rados.0.trial115.stdout:341: write oid 9 current snap is 29 2026-02-17T05:48:07.368 INFO:tasks.rados.rados.0.trial115.stdout:341: seq_num 164 ranges {756348=784249,2063988=686387} 2026-02-17T05:48:07.373 INFO:tasks.rados.rados.0.trial115.stdout:341: writing trial11545420-9 from 756348 to 1540597 tid 1 2026-02-17T05:48:07.378 INFO:tasks.rados.rados.0.trial115.stdout:341: writing trial11545420-9 from 2063988 to 2750375 tid 2 2026-02-17T05:48:07.378 INFO:tasks.rados.rados.0.trial115.stdout:342: snap_remove snap 26 2026-02-17T05:48:08.360 INFO:tasks.rados.rados.0.trial115.stdout:342: done (5 left) 2026-02-17T05:48:08.360 INFO:tasks.rados.rados.0.trial115.stdout:343: rollback oid 22 current snap is 29 2026-02-17T05:48:08.361 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 22 to 27 2026-02-17T05:48:08.361 INFO:tasks.rados.rados.0.trial115.stdout:334: expect (ObjNum 5 snap 0 seq_num 5) 2026-02-17T05:48:08.412 INFO:tasks.rados.rados.0.trial115.stdout:337: finishing copy_from to trial11545420-43 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 124 (ObjNum 155 snap 28 seq_num 155) dirty exists 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:338: finishing write tid 1 to trial11545420-18 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:334: done (5 left) 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:337: done (4 left) 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:344: rollback oid 11 current snap is 29 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 28 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:338: finishing write tid 2 to trial11545420-18 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:338: finishing write tid 3 to trial11545420-18 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:338: finishing write tid 4 to trial11545420-18 2026-02-17T05:48:08.413 INFO:tasks.rados.rados.0.trial115.stdout:338: oid 18 updating version 0 to 126 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:338: oid 18 version 126 is already newer than 124 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:338: oid 18 version 126 is already newer than 125 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 126 (ObjNum 162 snap 29 seq_num 162) dirty exists 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:338: left oid 18 (ObjNum 162 snap 29 seq_num 162) 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:340: finishing write tid 1 to trial11545420-29 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:340: finishing write tid 2 to trial11545420-29 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:340: finishing write tid 3 to trial11545420-29 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:340: finishing write tid 4 to trial11545420-29 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:340: oid 29 updating version 0 to 76 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:340: oid 29 version 76 is already newer than 75 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:340: oid 29 version 76 is already newer than 74 2026-02-17T05:48:08.414 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 76 (ObjNum 163 snap 29 seq_num 163) dirty exists 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:340: left oid 29 (ObjNum 163 snap 29 seq_num 163) 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:341: finishing write tid 1 to trial11545420-9 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:341: finishing write tid 2 to trial11545420-9 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:341: finishing write tid 3 to trial11545420-9 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:338: done (4 left) 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:340: done (3 left) 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:345: write oid 24 current snap is 29 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:341: finishing write tid 4 to trial11545420-9 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:341: oid 9 updating version 0 to 130 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:341: oid 9 version 130 is already newer than 129 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:341: oid 9 version 130 is already newer than 128 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 130 (ObjNum 164 snap 29 seq_num 164) dirty exists 2026-02-17T05:48:08.415 INFO:tasks.rados.rados.0.trial115.stdout:341: left oid 9 (ObjNum 164 snap 29 seq_num 164) 2026-02-17T05:48:08.416 INFO:tasks.rados.rados.0.trial115.stdout:343: finishing rollback tid 0 to trial11545420-22 2026-02-17T05:48:08.416 INFO:tasks.rados.rados.0.trial115.stdout:343: finishing rollback tid 1 to trial11545420-22 2026-02-17T05:48:08.416 INFO:tasks.rados.rados.0.trial115.stdout:343: finishing rollback tid 2 to trial11545420-22 2026-02-17T05:48:08.416 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 134 (ObjNum 21 snap 0 seq_num 21) dirty exists 2026-02-17T05:48:08.416 INFO:tasks.rados.rados.0.trial115.stdout:345: seq_num 165 ranges {773155=459498,1722470=430045,2648676=124479} 2026-02-17T05:48:08.416 INFO:tasks.rados.rados.0.trial115.stdout:345: writing trial11545420-24 from 773155 to 1232653 tid 1 2026-02-17T05:48:08.420 INFO:tasks.rados.rados.0.trial115.stdout:345: writing trial11545420-24 from 1722470 to 2152515 tid 2 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:345: writing trial11545420-24 from 2648676 to 2773155 tid 3 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:341: done (3 left) 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:343: done (2 left) 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:346: read oid 1 snap 28 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:346: expect (ObjNum 153 snap 28 seq_num 153) 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:347: rollback oid 35 current snap is 29 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 35 to 28 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:344: finishing rollback tid 0 to trial11545420-11 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:348: read oid 6 snap -1 2026-02-17T05:48:08.423 INFO:tasks.rados.rados.0.trial115.stdout:348: expect (ObjNum 75 snap 8 seq_num 75) 2026-02-17T05:48:08.424 INFO:tasks.rados.rados.0.trial115.stdout:349: snap_remove snap 27 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:344: finishing rollback tid 1 to trial11545420-11 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:349: done (5 left) 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:350: copy_from oid 18 from oid 34 current snap is 29 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:344: finishing rollback tid 2 to trial11545420-11 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 138 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:345: finishing write tid 1 to trial11545420-24 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:345: finishing write tid 2 to trial11545420-24 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:345: finishing write tid 3 to trial11545420-24 2026-02-17T05:48:09.364 INFO:tasks.rados.rados.0.trial115.stdout:347: finishing rollback tid 0 to trial11545420-35 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:345: finishing write tid 4 to trial11545420-24 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:347: finishing rollback tid 1 to trial11545420-35 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 144 (ObjNum 49 snap 0 seq_num 1334097728) dirty dne 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:345: finishing write tid 5 to trial11545420-24 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:345: oid 24 updating version 0 to 171 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:345: oid 24 updating version 171 to 172 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:345: oid 24 version 172 is already newer than 169 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:345: oid 24 version 172 is already newer than 170 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 172 (ObjNum 165 snap 29 seq_num 165) dirty exists 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:345: left oid 24 (ObjNum 165 snap 29 seq_num 165) 2026-02-17T05:48:09.365 INFO:tasks.rados.rados.0.trial115.stdout:348: expect (ObjNum 75 snap 8 seq_num 75) 2026-02-17T05:48:09.464 INFO:tasks.rados.rados.0.trial115.stdout:346: expect (ObjNum 153 snap 28 seq_num 153) 2026-02-17T05:48:09.564 INFO:tasks.rados.rados.0.trial115.stdout:350: finishing copy_from racing read to trial11545420-18 2026-02-17T05:48:09.564 INFO:tasks.rados.rados.0.trial115.stdout:344: done (5 left) 2026-02-17T05:48:09.564 INFO:tasks.rados.rados.0.trial115.stdout:345: done (4 left) 2026-02-17T05:48:09.564 INFO:tasks.rados.rados.0.trial115.stdout:346: done (3 left) 2026-02-17T05:48:09.564 INFO:tasks.rados.rados.0.trial115.stdout:347: done (2 left) 2026-02-17T05:48:09.564 INFO:tasks.rados.rados.0.trial115.stdout:348: done (1 left) 2026-02-17T05:48:09.564 INFO:tasks.rados.rados.0.trial115.stdout:351: snap_create 2026-02-17T05:48:09.565 INFO:tasks.rados.rados.0.trial115.stdout:350: finishing copy_from to trial11545420-18 2026-02-17T05:48:09.565 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 24 (ObjNum 78 snap 8 seq_num 78) dirty exists 2026-02-17T05:48:10.367 INFO:tasks.rados.rados.0.trial115.stdout:350: done (1 left) 2026-02-17T05:48:10.367 INFO:tasks.rados.rados.0.trial115.stdout:351: done (0 left) 2026-02-17T05:48:10.367 INFO:tasks.rados.rados.0.trial115.stdout:352: write oid 20 current snap is 30 2026-02-17T05:48:10.367 INFO:tasks.rados.rados.0.trial115.stdout:352: seq_num 166 ranges {789962=534747,1780952=690056} 2026-02-17T05:48:10.371 INFO:tasks.rados.rados.0.trial115.stdout:352: writing trial11545420-20 from 789962 to 1324709 tid 1 2026-02-17T05:48:10.376 INFO:tasks.rados.rados.0.trial115.stdout:352: writing trial11545420-20 from 1780952 to 2471008 tid 2 2026-02-17T05:48:10.376 INFO:tasks.rados.rados.0.trial115.stdout:353: write oid 47 current snap is 30 2026-02-17T05:48:10.376 INFO:tasks.rados.rados.0.trial115.stdout:353: seq_num 167 ranges {406769=609996,1439434=433714,2303522=503247} 2026-02-17T05:48:10.379 INFO:tasks.rados.rados.0.trial115.stdout:353: writing trial11545420-47 from 406769 to 1016765 tid 1 2026-02-17T05:48:10.382 INFO:tasks.rados.rados.0.trial115.stdout:353: writing trial11545420-47 from 1439434 to 1873148 tid 2 2026-02-17T05:48:10.386 INFO:tasks.rados.rados.0.trial115.stdout:353: writing trial11545420-47 from 2303522 to 2806769 tid 3 2026-02-17T05:48:10.386 INFO:tasks.rados.rados.0.trial115.stdout:354: delete oid 28 current snap is 30 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:352: finishing write tid 1 to trial11545420-20 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:352: finishing write tid 2 to trial11545420-20 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:352: finishing write tid 3 to trial11545420-20 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:352: finishing write tid 4 to trial11545420-20 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:352: oid 20 updating version 0 to 125 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:352: oid 20 updating version 125 to 126 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:352: oid 20 updating version 126 to 127 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 127 (ObjNum 166 snap 30 seq_num 166) dirty exists 2026-02-17T05:48:10.387 INFO:tasks.rados.rados.0.trial115.stdout:352: left oid 20 (ObjNum 166 snap 30 seq_num 166) 2026-02-17T05:48:10.388 INFO:tasks.rados.rados.0.trial115.stdout:353: finishing write tid 1 to trial11545420-47 2026-02-17T05:48:10.388 INFO:tasks.rados.rados.0.trial115.stdout:353: finishing write tid 2 to trial11545420-47 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:352: done (2 left) 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:354: done (1 left) 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:355: read oid 33 snap 29 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:355: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:356: copy_from oid 45 from oid 24 current snap is 30 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:357: read oid 50 snap -1 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:357: expect (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:358: write (excl) oid 42 current snap is 30 2026-02-17T05:48:10.389 INFO:tasks.rados.rados.0.trial115.stdout:358: seq_num 169 ranges {440383=476847,1556398=437383,2474721=365662} 2026-02-17T05:48:10.391 INFO:tasks.rados.rados.0.trial115.stdout:358: writing trial11545420-42 from 440383 to 917230 tid 1 2026-02-17T05:48:10.395 INFO:tasks.rados.rados.0.trial115.stdout:358: writing trial11545420-42 from 1556398 to 1993781 tid 2 2026-02-17T05:48:10.398 INFO:tasks.rados.rados.0.trial115.stdout:358: writing trial11545420-42 from 2474721 to 2840383 tid 3 2026-02-17T05:48:10.398 INFO:tasks.rados.rados.0.trial115.stdout:359: write (excl) oid 30 current snap is 30 2026-02-17T05:48:10.398 INFO:tasks.rados.rados.0.trial115.stdout:359: seq_num 170 ranges {457190=552096,1614880=581041,2785791=71399} 2026-02-17T05:48:10.402 INFO:tasks.rados.rados.0.trial115.stdout:359: writing trial11545420-30 from 457190 to 1009286 tid 1 2026-02-17T05:48:10.406 INFO:tasks.rados.rados.0.trial115.stdout:359: writing trial11545420-30 from 1614880 to 2195921 tid 2 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:359: writing trial11545420-30 from 2785791 to 2857190 tid 3 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:360: rollback oid 46 current snap is 30 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 46 to 29 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:353: finishing write tid 3 to trial11545420-47 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:353: finishing write tid 4 to trial11545420-47 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:353: finishing write tid 5 to trial11545420-47 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:353: oid 47 updating version 0 to 141 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:353: oid 47 updating version 141 to 143 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:353: oid 47 version 143 is already newer than 140 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:353: oid 47 version 143 is already newer than 142 2026-02-17T05:48:10.410 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 143 (ObjNum 167 snap 30 seq_num 167) dirty exists 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:353: left oid 47 (ObjNum 167 snap 30 seq_num 167) 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:356: finishing copy_from racing read to trial11545420-45 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:356: finishing copy_from to trial11545420-45 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 172 (ObjNum 165 snap 29 seq_num 165) dirty exists 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:358: finishing write tid 1 to trial11545420-42 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:358: finishing write tid 2 to trial11545420-42 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:358: finishing write tid 3 to trial11545420-42 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:358: finishing write tid 4 to trial11545420-42 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:353: done (6 left) 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:356: done (5 left) 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:361: read oid 10 snap 28 2026-02-17T05:48:10.411 INFO:tasks.rados.rados.0.trial115.stdout:361: expect deleted 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:362: rollback oid 9 current snap is 30 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 9 to 28 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:358: finishing write tid 5 to trial11545420-42 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:358: oid 42 updating version 0 to 181 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:358: oid 42 version 181 is already newer than 179 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:358: oid 42 updating version 181 to 182 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:358: oid 42 version 182 is already newer than 180 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 182 (ObjNum 169 snap 30 seq_num 169) dirty exists 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:358: left oid 42 (ObjNum 169 snap 30 seq_num 169) 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:358: done (6 left) 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:363: write (excl) oid 20 current snap is 30 2026-02-17T05:48:10.412 INFO:tasks.rados.rados.0.trial115.stdout:363: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2026-02-17T05:48:10.413 INFO:tasks.rados.rados.0.trial115.stdout:363: writing trial11545420-20 from 473997 to 1101342 tid 1 2026-02-17T05:48:10.417 INFO:tasks.rados.rados.0.trial115.stdout:363: writing trial11545420-20 from 1673362 to 2114414 tid 2 2026-02-17T05:48:10.420 INFO:tasks.rados.rados.0.trial115.stdout:363: writing trial11545420-20 from 2529567 to 2873997 tid 3 2026-02-17T05:48:10.420 INFO:tasks.rados.rados.0.trial115.stdout:364: copy_from oid 48 from oid 14 current snap is 30 2026-02-17T05:48:10.420 INFO:tasks.rados.rados.0.trial115.stdout:360: finishing rollback tid 1 to trial11545420-46 2026-02-17T05:48:10.420 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 0 (ObjNum 33 snap 0 seq_num 152318468) dirty dne 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:362: finishing rollback tid 0 to trial11545420-9 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:362: finishing rollback tid 1 to trial11545420-9 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:362: finishing rollback tid 2 to trial11545420-9 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 150 (ObjNum 100 snap 14 seq_num 100) dirty exists 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:363: finishing write tid 1 to trial11545420-20 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:363: finishing write tid 2 to trial11545420-20 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:360: done (7 left) 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:361: done (6 left) 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:362: done (5 left) 2026-02-17T05:48:10.421 INFO:tasks.rados.rados.0.trial115.stdout:365: delete oid 15 current snap is 30 2026-02-17T05:48:10.422 INFO:tasks.rados.rados.0.trial115.stdout:365: done (5 left) 2026-02-17T05:48:10.422 INFO:tasks.rados.rados.0.trial115.stdout:366: snap_remove snap 28 2026-02-17T05:48:11.294 INFO:tasks.mon_thrash:Sending CONT to mon a 2026-02-17T05:48:11.295 INFO:tasks.ceph.mon.a:Sent signal 18 2026-02-17T05:48:11.295 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:48:11.295 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:11.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.362+0000 7f95b8839640 10 monclient: get_monmap_and_config 2026-02-17T05:48:11.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.362+0000 7f95b8839640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.362+0000 7f95b8839640 10 monclient: monmap: 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:11.364189+0000 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:11.364189+0000 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:11.367 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 1 Processor -- start 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 1 -- start start 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient: init 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f95b009ea90 0x7f95b00a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient(hunting): picked mon.noname-a con 0x7f95b009ea90 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:11.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f95b009ee70 0x7f95b009f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient(hunting): picked mon.noname-c con 0x7f95b009ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b009ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient(hunting): picked mon.noname-b con 0x7f95b009d980 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f95b00a0d00 con 0x7f95b009ea90 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f95b00a0a20 con 0x7f95b009ee70 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f95b00a0460 con 0x7f95b009d980 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b8839640 10 monclient(hunting): authenticate will time out at 736.917808s 2026-02-17T05:48:11.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b009ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b65ae640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f95b009ea90 0x7f95b00a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b009ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57234/0 (socket says 10.20.193.78:57234) 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 1 -- 10.20.193.78:0/1264680072 learned_addr learned my addr 10.20.193.78:0/1264680072 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 10 monclient(hunting): get_auth_request con 0x7f95b009d980 auth_method 0 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b65ae640 10 monclient(hunting): get_auth_request con 0x7f95b009ea90 auth_method 0 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b65ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b65ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b65ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.363+0000 7f95b6daf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b65ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b65ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b65ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b5dad640 1 --2- 10.20.193.78:0/1264680072 >> v2:10.20.193.115:3300/0 conn(0x7f95b009ee70 0x7f95b009f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b5dad640 10 monclient(hunting): get_auth_request con 0x7f95b009ee70 auth_method 0 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b5dad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b5dad640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b5dad640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 10 monclient(hunting): handle_auth_done global_id 4448 payload 274 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 1 -- 10.20.193.78:0/1264680072 >> v2:10.20.193.115:3300/0 conn(0x7f95b009ee70 msgr2=0x7f95b009f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 1 --2- 10.20.193.78:0/1264680072 >> v2:10.20.193.115:3300/0 conn(0x7f95b009ee70 0x7f95b009f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 1 -- 10.20.193.78:0/1264680072 >> v2:10.20.193.78:3300/0 conn(0x7f95b009ea90 msgr2=0x7f95b00a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 1 --2- 10.20.193.78:0/1264680072 >> v2:10.20.193.78:3300/0 conn(0x7f95b009ea90 0x7f95b00a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 1 monclient: found mon.noname-b 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b6daf640 1 -- 10.20.193.78:0/1264680072 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b00a0740 con 0x7f95b009d980 2026-02-17T05:48:11.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.364+0000 7f95b65ae640 1 --2- 10.20.193.78:0/1264680072 >> v2:10.20.193.78:3300/0 conn(0x7f95b009ea90 0x7f95b00a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:366: done (5 left) 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:367: rollback oid 8 current snap is 30 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 8 to 29 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:363: finishing write tid 3 to trial11545420-20 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:363: finishing write tid 4 to trial11545420-20 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:363: finishing write tid 5 to trial11545420-20 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:363: oid 20 updating version 0 to 128 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:363: oid 20 updating version 128 to 131 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:363: oid 20 version 131 is already newer than 130 2026-02-17T05:48:11.374 INFO:tasks.rados.rados.0.trial115.stdout:363: oid 20 version 131 is already newer than 129 2026-02-17T05:48:11.375 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 131 (ObjNum 171 snap 30 seq_num 171) dirty exists 2026-02-17T05:48:11.375 INFO:tasks.rados.rados.0.trial115.stdout:363: left oid 20 (ObjNum 171 snap 30 seq_num 171) 2026-02-17T05:48:11.375 INFO:tasks.rados.rados.0.trial115.stdout:357: expect (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:48:11.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b6daf640 1 -- 10.20.193.78:0/1264680072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 msgr2=0x7f95b009ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b6daf640 1 --2- 10.20.193.78:0/1264680072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b009ddb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95b006e270 tx=0x7f95a402f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ecf9dde65ee564 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 1 -- 10.20.193.78:0/1264680072 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a402fe50 con 0x7f95b009d980 2026-02-17T05:48:11.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: dump: 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.383 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 1 -- 10.20.193.78:0/1264680072 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a40373f0 con 0x7f95b009d980 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b8839640 5 monclient: authenticate success, global_id 4448 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b8839640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b8839640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 1 -- 10.20.193.78:0/1264680072 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40377c0 con 0x7f95b009d980 2026-02-17T05:48:11.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b8839640 4 set_mon_vals no callback set 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.366+0000 7f95b55ac640 10 monclient: dump: 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.385 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 10 monclient: shutdown 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 -- 10.20.193.78:0/1264680072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 msgr2=0x7f95b009ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 --2- 10.20.193.78:0/1264680072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b009ddb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95b006e270 tx=0x7f95a402f1e0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 -- 10.20.193.78:0/1264680072 shutdown_connections 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 --2- 10.20.193.78:0/1264680072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b009ddb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 --2- 10.20.193.78:0/1264680072 >> v2:10.20.193.115:3300/0 conn(0x7f95b009ee70 0x7f95b009f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 --2- 10.20.193.78:0/1264680072 >> v2:10.20.193.78:3300/0 conn(0x7f95b009ea90 0x7f95b00a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 -- 10.20.193.78:0/1264680072 >> 10.20.193.78:0/1264680072 conn(0x7f95b00a7520 msgr2=0x7f95b00a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 -- 10.20.193.78:0/1264680072 shutdown_connections 2026-02-17T05:48:11.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 -- 10.20.193.78:0/1264680072 wait complete. 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 10 monclient: monmap: 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:11.387 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 Processor -- start 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 1 -- start start 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.367+0000 7f95b8839640 10 monclient: init 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b012fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient(hunting): picked mon.noname-c con 0x7f95b009d980 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95b009ea90 0x7f95b0130020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient(hunting): picked mon.noname-a con 0x7f95b009ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95b009ee70 0x7f95b0130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient(hunting): picked mon.noname-b con 0x7f95b009ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:11.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95b00a66a0 con 0x7f95b009ea90 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95b00a1910 con 0x7f95b009ee70 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f95b00a6820 con 0x7f95b009d980 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b8839640 10 monclient(hunting): authenticate will time out at 736.922083s 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95b009ea90 0x7f95b0130020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95b009ea90 0x7f95b0130020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45798/0 (socket says 10.20.193.78:45798) 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 -- 10.20.193.78:0/382584324 learned_addr learned my addr 10.20.193.78:0/382584324 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient(hunting): get_auth_request con 0x7f95b009ea90 auth_method 0 2026-02-17T05:48:11.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b65ae640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b012fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b65ae640 10 monclient(hunting): get_auth_request con 0x7f95b009d980 auth_method 0 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b65ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b65ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b65ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b65ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b65ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b65ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient(hunting): handle_auth_done global_id 4482 payload 824 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b6daf640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95b009ee70 0x7f95b0130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 -- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 msgr2=0x7f95b012fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b012fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 -- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95b009ee70 msgr2=0x7f95b0130560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95b009ee70 0x7f95b0130560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 monclient: found mon.noname-a 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b5dad640 1 -- 10.20.193.78:0/382584324 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b0131960 con 0x7f95b009ea90 2026-02-17T05:48:11.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b6daf640 10 monclient: get_auth_request con 0x7f95b009ee70 auth_method 0 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.368+0000 7f95b6daf640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95b009ee70 0x7f95b0130560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b5dad640 1 -- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95b009ea90 msgr2=0x7f95b0130020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b5dad640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95b009ea90 0x7f95b0130020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e7c0 tx=0x7f959c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 1 -- 10.20.193.78:0/382584324 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c017070 con 0x7f95b009ea90 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: dump: 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.392 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 1 -- 10.20.193.78:0/382584324 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c00cae0 con 0x7f95b009ea90 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b8839640 5 monclient: authenticate success, global_id 4482 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b8839640 10 monclient: _renew_subs 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b8839640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b0135330 con 0x7f95b009ea90 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 1 -- 10.20.193.78:0/382584324 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00f040 con 0x7f95b009ea90 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b4dab640 4 set_mon_vals no callback set 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f959affd640 10 monclient: dump: 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.394 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b8839640 10 monclient: _renew_subs 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b8839640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.370+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b01321a0 con 0x7f95b009ea90 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f95b8839640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f95b8839640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f95b8839640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b00a13d0 con 0x7f95b009ea90 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f959affd640 1 -- 10.20.193.78:0/382584324 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f959c012050 con 0x7f95b009ea90 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f959affd640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f958003deb0 0x7f9580040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f959affd640 1 -- 10.20.193.78:0/382584324 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4795+0+0 (secure 0 0 0) 0x7f959c052630 con 0x7f95b009ea90 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f95b65ae640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f958003deb0 0x7f9580040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.371+0000 7f95b65ae640 10 monclient: get_auth_request con 0x7f958003deb0 auth_method 0 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.374+0000 7f959affd640 1 -- 10.20.193.78:0/382584324 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f95b0131960 con 0x7f95b009ea90 2026-02-17T05:48:11.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.374+0000 7f959affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.374+0000 7f959affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:11.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.374+0000 7f95b65ae640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f958003deb0 0x7f9580040370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009660 tx=0x7f95a8009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.431 INFO:tasks.rados.rados.0.trial115.stdout:355: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:48:11.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.489+0000 7f95b8839640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:48:11.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.489+0000 7f95b8839640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:11.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.489+0000 7f95b8839640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.489+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f95b0130d50 con 0x7f95b009ea90 2026-02-17T05:48:11.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.489+0000 7f959affd640 1 -- 10.20.193.78:0/382584324 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f959c023d80 con 0x7f95b009ea90 2026-02-17T05:48:11.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.489+0000 7f959affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:11.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.489+0000 7f959affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:11.492 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:48:11.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f958003deb0 msgr2=0x7f9580040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f958003deb0 0x7f9580040370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009660 tx=0x7f95a8009290 comp rx=0 tx=0).stop 2026-02-17T05:48:11.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 10 monclient: shutdown 2026-02-17T05:48:11.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95b009ea90 msgr2=0x7f95b0130020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95b009ea90 0x7f95b0130020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e7c0 tx=0x7f959c00ec90 comp rx=0 tx=0).stop 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 shutdown_connections 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f958003deb0 0x7f9580040370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95b009ee70 0x7f95b0130560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95b009ea90 0x7f95b0130020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 --2- 10.20.193.78:0/382584324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95b009d980 0x7f95b012fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 >> 10.20.193.78:0/382584324 conn(0x7f95b00a7520 msgr2=0x7f95b0136e20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 shutdown_connections 2026-02-17T05:48:11.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.490+0000 7f95b8839640 1 -- 10.20.193.78:0/382584324 wait complete. 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: finishing write tid 1 to trial11545420-30 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: finishing write tid 2 to trial11545420-30 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: finishing write tid 3 to trial11545420-30 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: finishing write tid 4 to trial11545420-30 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: finishing write tid 5 to trial11545420-30 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: oid 30 updating version 0 to 135 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: oid 30 updating version 135 to 137 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: oid 30 version 137 is already newer than 136 2026-02-17T05:48:11.496 INFO:tasks.rados.rados.0.trial115.stdout:359: oid 30 updating version 137 to 138 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 138 (ObjNum 170 snap 30 seq_num 170) dirty exists 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:359: left oid 30 (ObjNum 170 snap 30 seq_num 170) 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:364: finishing copy_from racing read to trial11545420-48 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:364: finishing copy_from to trial11545420-48 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 31 (ObjNum 99 snap 14 seq_num 99) dirty exists 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:367: finishing rollback tid 1 to trial11545420-8 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 16 (ObjNum 289 snap 0 seq_num 308995056) dirty dne 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:355: done (5 left) 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:357: done (4 left) 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:359: done (3 left) 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:363: done (2 left) 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:364: done (1 left) 2026-02-17T05:48:11.497 INFO:tasks.rados.rados.0.trial115.stdout:367: done (0 left) 2026-02-17T05:48:11.498 INFO:tasks.rados.rados.0.trial115.stdout:368: read oid 42 snap -1 2026-02-17T05:48:11.498 INFO:tasks.rados.rados.0.trial115.stdout:368: expect (ObjNum 169 snap 30 seq_num 169) 2026-02-17T05:48:11.498 INFO:tasks.rados.rados.0.trial115.stdout:369: write oid 3 current snap is 30 2026-02-17T05:48:11.498 INFO:tasks.rados.rados.0.trial115.stdout:369: seq_num 173 ranges {507611=777843,2073973=444721} 2026-02-17T05:48:11.500 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:48:11.500 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:48:11.500 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:48:11.501 INFO:tasks.rados.rados.0.trial115.stdout:369: writing trial11545420-3 from 507611 to 1285454 tid 1 2026-02-17T05:48:11.505 INFO:tasks.rados.rados.0.trial115.stdout:369: writing trial11545420-3 from 2073973 to 2518694 tid 2 2026-02-17T05:48:11.505 INFO:tasks.rados.rados.0.trial115.stdout:370: read oid 37 snap -1 2026-02-17T05:48:11.505 INFO:tasks.rados.rados.0.trial115.stdout:370: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:48:11.505 INFO:tasks.rados.rados.0.trial115.stdout:371: copy_from oid 6 from oid 10 current snap is 30 2026-02-17T05:48:11.505 INFO:tasks.rados.rados.0.trial115.stdout:368: expect (ObjNum 169 snap 30 seq_num 169) 2026-02-17T05:48:11.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.566+0000 7fb57af96640 10 monclient: get_monmap_and_config 2026-02-17T05:48:11.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.566+0000 7fb57af96640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.566+0000 7fb57af96640 10 monclient: monmap: 2026-02-17T05:48:11.568 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:11.568133+0000 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:11.568133+0000 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 1 Processor -- start 2026-02-17T05:48:11.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 1 -- start start 2026-02-17T05:48:11.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient: init 2026-02-17T05:48:11.578 INFO:tasks.rados.rados.0.trial115.stdout:371: finishing copy_from to trial11545420-6 2026-02-17T05:48:11.578 INFO:tasks.rados.rados.0.trial115.stdout:371: got expected ENOENT (src dne) 2026-02-17T05:48:11.578 INFO:tasks.rados.rados.0.trial115.stdout:371: finishing copy_from racing read to trial11545420-6 2026-02-17T05:48:11.578 INFO:tasks.rados.rados.0.trial115.stdout:369: finishing write tid 1 to trial11545420-3 2026-02-17T05:48:11.578 INFO:tasks.rados.rados.0.trial115.stdout:369: finishing write tid 2 to trial11545420-3 2026-02-17T05:48:11.578 INFO:tasks.rados.rados.0.trial115.stdout:368: done (3 left) 2026-02-17T05:48:11.578 INFO:tasks.rados.rados.0.trial115.stdout:371: done (2 left) 2026-02-17T05:48:11.578 INFO:tasks.rados.rados.0.trial115.stdout:372: delete oid 45 current snap is 30 2026-02-17T05:48:11.579 INFO:tasks.rados.rados.0.trial115.stdout:369: finishing write tid 3 to trial11545420-3 2026-02-17T05:48:11.579 INFO:tasks.rados.rados.0.trial115.stdout:369: finishing write tid 4 to trial11545420-3 2026-02-17T05:48:11.579 INFO:tasks.rados.rados.0.trial115.stdout:369: oid 3 updating version 0 to 148 2026-02-17T05:48:11.579 INFO:tasks.rados.rados.0.trial115.stdout:369: oid 3 updating version 148 to 149 2026-02-17T05:48:11.579 INFO:tasks.rados.rados.0.trial115.stdout:369: oid 3 version 149 is already newer than 147 2026-02-17T05:48:11.579 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 149 (ObjNum 173 snap 30 seq_num 173) dirty exists 2026-02-17T05:48:11.579 INFO:tasks.rados.rados.0.trial115.stdout:369: left oid 3 (ObjNum 173 snap 30 seq_num 173) 2026-02-17T05:48:11.579 INFO:tasks.rados.rados.0.trial115.stdout:370: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:48:11.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2026-02-17T05:48:11.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb57406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:11.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb57409c4a0 0x7fb57413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient(hunting): picked mon.noname-a con 0x7fb57409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:11.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb57406d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb57406daf0 con 0x7fb57406d380 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb57409cc80 con 0x7fb57406c890 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb5740a0880 con 0x7fb57409c4a0 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57af96640 10 monclient(hunting): authenticate will time out at 737.121684s 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb57409c4a0 0x7fb57413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57950c640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb57406d380 0x7fb57413d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33712/0 (socket says 10.20.193.78:33712) 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb57950c640 1 -- 10.20.193.78:0/1094066009 learned_addr learned my addr 10.20.193.78:0/1094066009 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient(hunting): get_auth_request con 0x7fb57409c4a0 auth_method 0 2026-02-17T05:48:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient(hunting): handle_auth_done global_id 4485 payload 274 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 1 -- 10.20.193.78:0/1094066009 >> v1:10.20.193.115:6789/0 conn(0x7fb57406c890 legacy=0x7fb57406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 1 -- 10.20.193.78:0/1094066009 >> v1:10.20.193.78:6789/0 conn(0x7fb57406d380 legacy=0x7fb57413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 1 monclient: found mon.noname-a 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.567+0000 7fb56bfff640 1 -- 10.20.193.78:0/1094066009 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57409cf60 con 0x7fb57409c4a0 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56bfff640 1 -- 10.20.193.78:0/1094066009 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57409c4a0 msgr2=0x7fb57413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56bfff640 1 --2- 10.20.193.78:0/1094066009 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57409c4a0 0x7fb57413b3a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb57409c980 tx=0x7fb55c02ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811aec8d98d71960 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 1 -- 10.20.193.78:0/1094066009 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c03c070 con 0x7fb57409c4a0 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: dump: 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.585 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 1 -- 10.20.193.78:0/1094066009 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c02fcb0 con 0x7fb57409c4a0 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 5 monclient: authenticate success, global_id 4485 2026-02-17T05:48:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 1 -- 10.20.193.78:0/1094066009 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c02fe60 con 0x7fb57409c4a0 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 4 set_mon_vals no callback set 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb56b7fe640 10 monclient: dump: 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.587 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 10 monclient: shutdown 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 1 -- 10.20.193.78:0/1094066009 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57409c4a0 msgr2=0x7fb57413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 1 --2- 10.20.193.78:0/1094066009 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57409c4a0 0x7fb57413b3a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb57409c980 tx=0x7fb55c02ed50 comp rx=0 tx=0).stop 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 1 -- 10.20.193.78:0/1094066009 shutdown_connections 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 1 --2- 10.20.193.78:0/1094066009 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57409c4a0 0x7fb57413b3a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 1 -- 10.20.193.78:0/1094066009 >> 10.20.193.78:0/1094066009 conn(0x7fb5740a6620 msgr2=0x7fb5740a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 1 -- 10.20.193.78:0/1094066009 shutdown_connections 2026-02-17T05:48:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.570+0000 7fb57af96640 1 -- 10.20.193.78:0/1094066009 wait complete. 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient: monmap: 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:11.589 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 1 Processor -- start 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 1 -- start start 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient: init 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406c890 0x7fb5741cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient(hunting): picked mon.noname-a con 0x7fb57406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb57406d380 0x7fb574131030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient(hunting): picked mon.noname-c con 0x7fb57406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb57409c4a0 0x7fb574131570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient(hunting): picked mon.noname-b con 0x7fb57409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb57409bfa0 con 0x7fb57406c890 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb57409be20 con 0x7fb57409c4a0 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb57413f010 con 0x7fb57406d380 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57af96640 10 monclient(hunting): authenticate will time out at 737.125681s 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406c890 0x7fb5741cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406c890 0x7fb5741cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45806/0 (socket says 10.20.193.78:45806) 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 -- 10.20.193.78:0/3697684411 learned_addr learned my addr 10.20.193.78:0/3697684411 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient(hunting): get_auth_request con 0x7fb57406c890 auth_method 0 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb56bfff640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb57406d380 0x7fb574131030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb56bfff640 10 monclient(hunting): get_auth_request con 0x7fb57406d380 auth_method 0 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb56bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb56bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb56bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient(hunting): handle_auth_done global_id 4488 payload 824 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb57406d380 msgr2=0x7fb574131030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb57406d380 0x7fb574131030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57950c640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb57409c4a0 0x7fb574131570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb57409c4a0 msgr2=0x7fb574131570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb57409c4a0 0x7fb574131570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 monclient: found mon.noname-a 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb578d0b640 1 -- 10.20.193.78:0/3697684411 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5740718c0 con 0x7fb57406c890 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57950c640 10 monclient: get_auth_request con 0x7fb57409c4a0 auth_method 0 2026-02-17T05:48:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.571+0000 7fb57950c640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb57409c4a0 0x7fb574131570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb578d0b640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406c890 msgr2=0x7fb5741cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb578d0b640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406c890 0x7fb5741cf140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c600 tx=0x7fb56400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564012a40 con 0x7fb57406c890 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: dump: 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.594 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 5 monclient: authenticate success, global_id 4488 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb564017070 con 0x7fb57406c890 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564012e20 con 0x7fb57406c890 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56affd640 4 set_mon_vals no callback set 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 10 monclient: dump: 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.596 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 10 monclient: _renew_subs 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57409e430 con 0x7fb57406c890 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 10 monclient: _renew_subs 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb574072d10 con 0x7fb57406c890 2026-02-17T05:48:11.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb574072f10 0x7fb57406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb57406cc70 con 0x7fb574072f10 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb574072f10 0x7fb57406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 10 monclient: get_auth_request con 0x7fb574072f10 auth_method 0 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb578d0b640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406c890 0x7fb5741cf140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c600 tx=0x7fb56400cad0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fb5640526a0 osd_map(86..86 src has 1..86) 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb56401c460 con 0x7fb57406c890 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 10 monclient(hunting): handle_auth_done global_id 4491 payload 824 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb56bfff640 1 -- 10.20.193.78:0/3697684411 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5740732f0 con 0x7fb574072f10 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.574+0000 7fb5697fa640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb54403deb0 0x7fb544040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.575+0000 7fb57950c640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb54403deb0 0x7fb544040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.575+0000 7fb57950c640 10 monclient: get_auth_request con 0x7fb54403deb0 auth_method 0 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.576+0000 7fb56bfff640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb574072f10 msgr2=0x7fb57406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.576+0000 7fb56bfff640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb574072f10 0x7fb57406f410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb574074de0 tx=0x7fb55c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.576+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c03c040 con 0x7fb574072f10 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.576+0000 7fb5697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:11.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 10 monclient: dump: 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fb55c041410 con 0x7fb574072f10 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb574072f10 msgr2=0x7fb57406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb574072f10 0x7fb57406f410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb574074de0 tx=0x7fb55c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb574072f10 msgr2=0x7fb57406f410 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb5697fa640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb574072f10 0x7fb57406f410 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb574074de0 tx=0x7fb55c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.577+0000 7fb57950c640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb54403deb0 0x7fb544040370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb570005c60 tx=0x7fb570009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb57af96640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb57af96640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406f950 0x7fb5741da490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb57af96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5740744d0 con 0x7fb57406f950 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406f950 0x7fb5741da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 10 monclient: get_auth_request con 0x7fb57406f950 auth_method 0 2026-02-17T05:48:11.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 10 monclient(hunting): handle_auth_done global_id 4494 payload 824 2026-02-17T05:48:11.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.580+0000 7fb56bfff640 1 -- 10.20.193.78:0/3697684411 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb57406fd30 con 0x7fb57406f950 2026-02-17T05:48:11.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb56bfff640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406f950 msgr2=0x7fb5741da490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb56bfff640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406f950 0x7fb5741da490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb55c048000 tx=0x7fb55c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c03c040 con 0x7fb57406f950 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 10 monclient: dump: 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.604 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7fb55c051940 con 0x7fb57406f950 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406f950 msgr2=0x7fb5741da490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406f950 0x7fb5741da490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb55c048000 tx=0x7fb55c008030 comp rx=0 tx=0).stop 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406f950 msgr2=0x7fb5741da490 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406f950 0x7fb5741da490 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb55c048000 tx=0x7fb55c008030 comp rx=0 tx=0).stop 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:11.605 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 24, 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 141386, 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.606 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.607 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:48:11.608 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.609 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.610 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.611 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.612 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.613 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.614 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.615 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:48:11.616 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb54403deb0 msgr2=0x7fb544040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb54403deb0 0x7fb544040370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb570005c60 tx=0x7fb570009040 comp rx=0 tx=0).stop 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 10 monclient: shutdown 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406c890 msgr2=0x7fb5741cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 1 --2- 10.20.193.78:0/3697684411 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb57406c890 0x7fb5741cf140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c600 tx=0x7fb56400cad0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57950c640 1 -- 10.20.193.78:0/3697684411 reap_dead start 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb5697fa640 1 -- 10.20.193.78:0/3697684411 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4795+0+0 (unknown 0 0 0) 0x7fb5640526a0 con 0x7fb57406c890 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57950c640 1 -- 10.20.193.78:0/3697684411 reap_dead start 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 shutdown_connections 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 >> 10.20.193.78:0/3697684411 conn(0x7fb5740a6620 msgr2=0x7fb57413d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 shutdown_connections 2026-02-17T05:48:11.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.582+0000 7fb57af96640 1 -- 10.20.193.78:0/3697684411 wait complete. 2026-02-17T05:48:11.618 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:48:11.618 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:48:11.618 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:48:11.681 INFO:tasks.rados.rados.0.trial115.stdout:369: done (2 left) 2026-02-17T05:48:11.681 INFO:tasks.rados.rados.0.trial115.stdout:370: done (1 left) 2026-02-17T05:48:11.681 INFO:tasks.rados.rados.0.trial115.stdout:372: done (0 left) 2026-02-17T05:48:11.681 INFO:tasks.rados.rados.0.trial115.stdout:373: write (excl) oid 47 current snap is 30 2026-02-17T05:48:11.681 INFO:tasks.rados.rados.0.trial115.stdout:373: seq_num 175 ranges {541225=644694,1907290=448390} 2026-02-17T05:48:11.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient: get_monmap_and_config 2026-02-17T05:48:11.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient: monmap: 2026-02-17T05:48:11.685 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.685 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:11.685457+0000 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:11.685457+0000 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:11.686 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.687 INFO:tasks.rados.rados.0.trial115.stdout:373: writing trial11545420-47 from 541225 to 1185919 tid 1 2026-02-17T05:48:11.694 INFO:tasks.rados.rados.0.trial115.stdout:373: writing trial11545420-47 from 1907290 to 2355680 tid 2 2026-02-17T05:48:11.694 INFO:tasks.rados.rados.0.trial115.stdout:374: read oid 44 snap -1 2026-02-17T05:48:11.694 INFO:tasks.rados.rados.0.trial115.stdout:374: expect (ObjNum 104 snap 15 seq_num 104) 2026-02-17T05:48:11.694 INFO:tasks.rados.rados.0.trial115.stdout:375: read oid 15 snap -1 2026-02-17T05:48:11.694 INFO:tasks.rados.rados.0.trial115.stdout:375: expect deleted 2026-02-17T05:48:11.694 INFO:tasks.rados.rados.0.trial115.stdout:376: delete oid 28 current snap is 30 2026-02-17T05:48:11.694 INFO:tasks.rados.rados.0.trial115.stdout:373: finishing write tid 1 to trial11545420-47 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:375: done (3 left) 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:376: done (2 left) 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:377: snap_create 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:373: finishing write tid 2 to trial11545420-47 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:373: finishing write tid 3 to trial11545420-47 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:373: finishing write tid 4 to trial11545420-47 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:373: oid 47 updating version 0 to 151 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:373: oid 47 updating version 151 to 152 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:373: oid 47 version 152 is already newer than 150 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 152 (ObjNum 175 snap 30 seq_num 175) dirty exists 2026-02-17T05:48:11.695 INFO:tasks.rados.rados.0.trial115.stdout:373: left oid 47 (ObjNum 175 snap 30 seq_num 175) 2026-02-17T05:48:11.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 1 Processor -- start 2026-02-17T05:48:11.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 1 -- start start 2026-02-17T05:48:11.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient: init 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f90a006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f90a009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a006d380 0x7f90a013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 10 monclient(hunting): picked mon.noname-b con 0x7f90a006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.684+0000 7f90a4bf5640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a006daf0 con 0x7f90a009c4a0 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f90a4bf5640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a009cc80 con 0x7f90a006c890 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f90a4bf5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f90a4bf5640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f90a00a0880 con 0x7f90a006d380 2026-02-17T05:48:11.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f90a4bf5640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f90a4bf5640 10 monclient(hunting): authenticate will time out at 737.239021s 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909dd74640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f90a009c4a0 0x7f90a013b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33716/0 (socket says 10.20.193.78:33716) 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909dd74640 1 -- 10.20.193.78:0/3323307506 learned_addr learned my addr 10.20.193.78:0/3323307506 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a006d380 0x7f90a013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient(hunting): get_auth_request con 0x7f90a006d380 auth_method 0 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient(hunting): handle_auth_done global_id 4457 payload 274 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 1 -- 10.20.193.78:0/3323307506 >> v1:10.20.193.115:6789/0 conn(0x7f90a006c890 legacy=0x7f90a006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 1 -- 10.20.193.78:0/3323307506 >> v1:10.20.193.78:6789/0 conn(0x7f90a009c4a0 legacy=0x7f90a013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 1 monclient: found mon.noname-b 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.685+0000 7f909ed76640 1 -- 10.20.193.78:0/3323307506 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a009cf60 con 0x7f90a006d380 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.687+0000 7f909ed76640 1 -- 10.20.193.78:0/3323307506 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a006d380 msgr2=0x7f90a013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.687+0000 7f909ed76640 1 --2- 10.20.193.78:0/3323307506 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a006d380 0x7f90a013ead0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9090009dc0 tx=0x7f909002f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655dc9924b43aa6c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 1 -- 10.20.193.78:0/3323307506 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909003c070 con 0x7f90a006d380 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: dump: 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 5 monclient: authenticate success, global_id 4457 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 1 -- 10.20.193.78:0/3323307506 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9090037640 con 0x7f90a006d380 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 1 -- 10.20.193.78:0/3323307506 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090037a10 con 0x7f90a006d380 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f909d573640 10 monclient: dump: 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.703 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 4 set_mon_vals no callback set 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 10 monclient: shutdown 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 1 -- 10.20.193.78:0/3323307506 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a006d380 msgr2=0x7f90a013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 1 --2- 10.20.193.78:0/3323307506 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a006d380 0x7f90a013ead0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9090009dc0 tx=0x7f909002f1c0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 1 -- 10.20.193.78:0/3323307506 shutdown_connections 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 1 --2- 10.20.193.78:0/3323307506 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a006d380 0x7f90a013ead0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 1 -- 10.20.193.78:0/3323307506 >> 10.20.193.78:0/3323307506 conn(0x7f90a00a6620 msgr2=0x7f90a00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 1 -- 10.20.193.78:0/3323307506 shutdown_connections 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 1 -- 10.20.193.78:0/3323307506 wait complete. 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.688+0000 7f90a4bf5640 10 monclient: monmap: 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:11.705 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 1 Processor -- start 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 1 -- start start 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient: init 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f90a006c890 0x7f90a01364d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient(hunting): picked mon.noname-a con 0x7f90a006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f90a006d380 0x7f90a0136a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient(hunting): picked mon.noname-b con 0x7f90a006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a009c4a0 0x7f90a011df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient(hunting): picked mon.noname-c con 0x7f90a009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90a009bfa0 con 0x7f90a006c890 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90a009be20 con 0x7f90a006d380 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f90a013f010 con 0x7f90a009c4a0 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f90a4bf5640 10 monclient(hunting): authenticate will time out at 737.243367s 2026-02-17T05:48:11.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f90a006c890 0x7f90a01364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a009c4a0 0x7f90a011df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f90a006c890 0x7f90a01364d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45836/0 (socket says 10.20.193.78:45836) 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a009c4a0 0x7f90a011df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57288/0 (socket says 10.20.193.78:57288) 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 1 -- 10.20.193.78:0/1113406264 learned_addr learned my addr 10.20.193.78:0/1113406264 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 10 monclient(hunting): get_auth_request con 0x7f90a006c890 auth_method 0 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient(hunting): get_auth_request con 0x7f90a009c4a0 auth_method 0 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909dd74640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f90a006d380 0x7f90a0136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909dd74640 10 monclient(hunting): get_auth_request con 0x7f90a006d380 auth_method 0 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient(hunting): handle_auth_done global_id 4460 payload 824 2026-02-17T05:48:11.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f90a006d380 msgr2=0x7f90a0136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f90a006d380 0x7f90a0136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f90a006c890 msgr2=0x7f90a01364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f90a006c890 0x7f90a01364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 1 monclient: found mon.noname-c 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909ed76640 1 -- 10.20.193.78:0/1113406264 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a0121bb0 con 0x7f90a009c4a0 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.689+0000 7f909e575640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f90a006c890 0x7f90a01364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f909ed76640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a009c4a0 msgr2=0x7f90a011df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f909ed76640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a009c4a0 0x7f90a011df00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9090036000 tx=0x7f9090004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090047020 con 0x7f90a009c4a0 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: dump: 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.711 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9090042880 con 0x7f90a009c4a0 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909003c040 con 0x7f90a009c4a0 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f90a4bf5640 5 monclient: authenticate success, global_id 4460 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f90a4bf5640 10 monclient: _renew_subs 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f909cd72640 4 set_mon_vals no callback set 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f90a4bf5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a0120480 con 0x7f90a009c4a0 2026-02-17T05:48:11.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.691+0000 7f9086ffd640 10 monclient: dump: 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.713 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f90a4bf5640 10 monclient: _renew_subs 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f90a4bf5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a0072960 con 0x7f90a009c4a0 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f90a4bf5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f90a4bf5640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a01cf3d0 0x7f90a011e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f90a4bf5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f90a011fba0 con 0x7f90a01cf3d0 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a01cf3d0 0x7f90a011e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 10 monclient: get_auth_request con 0x7f90a01cf3d0 auth_method 0 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9090042cb0 con 0x7f90a009c4a0 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 10 monclient(hunting): handle_auth_done global_id 4463 payload 824 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f909e575640 1 -- 10.20.193.78:0/1113406264 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90a006fdd0 con 0x7f90a01cf3d0 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f9086ffd640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f906c03df00 0x7f906c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.692+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(86..86 src has 1..86) ==== 4795+0+0 (secure 0 0 0) 0x7f90900785a0 con 0x7f90a009c4a0 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.693+0000 7f909dd74640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f906c03df00 0x7f906c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.693+0000 7f909dd74640 10 monclient: get_auth_request con 0x7f906c03df00 auth_method 0 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.694+0000 7f909e575640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a01cf3d0 msgr2=0x7f90a011e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.694+0000 7f909e575640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a01cf3d0 0x7f90a011e440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f909400fc30 tx=0x7f909400d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094018020 con 0x7f90a01cf3d0 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 10 monclient: dump: 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.716 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f909400fe70 con 0x7f90a01cf3d0 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a01cf3d0 msgr2=0x7f90a011e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a01cf3d0 0x7f90a011e440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f909400fc30 tx=0x7f909400d520 comp rx=0 tx=0).stop 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a01cf3d0 msgr2=0x7f90a011e440 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f9086ffd640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a01cf3d0 0x7f90a011e440 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f909400fc30 tx=0x7f909400d520 comp rx=0 tx=0).stop 2026-02-17T05:48:11.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.695+0000 7f909dd74640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f906c03df00 0x7f906c0403c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f90a011f7a0 tx=0x7f9088009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.718 INFO:tasks.rados.rados.0.trial115.stdout:374: expect (ObjNum 104 snap 15 seq_num 104) 2026-02-17T05:48:11.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f90a4bf5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f90a4bf5640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a011e980 0x7f90a01da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f90a4bf5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f90a011ed90 con 0x7f90a011e980 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a011e980 0x7f90a01da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 10 monclient: get_auth_request con 0x7f90a011e980 auth_method 0 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 10 monclient(hunting): handle_auth_done global_id 4466 payload 824 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.698+0000 7f909e575640 1 -- 10.20.193.78:0/1113406264 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f90a006cc70 con 0x7f90a011e980 2026-02-17T05:48:11.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f909e575640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a011e980 msgr2=0x7f90a01da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f909e575640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a011e980 0x7f90a01da4d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9094019000 tx=0x7f9094003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094018070 con 0x7f90a011e980 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f9086ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f9086ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f9086ffd640 10 monclient: dump: 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f9094014ca0 con 0x7f90a011e980 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f9086ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.700+0000 7f9086ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a011e980 msgr2=0x7f90a01da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f9086ffd640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a011e980 0x7f90a01da4d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9094019000 tx=0x7f9094003860 comp rx=0 tx=0).stop 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f9086ffd640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a011e980 msgr2=0x7f90a01da4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f9086ffd640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a011e980 0x7f90a01da4d0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9094019000 tx=0x7f9094003860 comp rx=0 tx=0).stop 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:48:11.722 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 24, 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 141505, 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.723 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.724 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:11.725 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.726 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:48:11.727 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.728 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:48:11.729 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.730 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:48:11.731 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f906c03df00 msgr2=0x7f906c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f906c03df00 0x7f906c0403c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f90a011f7a0 tx=0x7f9088009040 comp rx=0 tx=0).stop 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 10 monclient: shutdown 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a009c4a0 msgr2=0x7f90a011df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a009c4a0 0x7f90a011df00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9090036000 tx=0x7f9090004b40 comp rx=0 tx=0).stop 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f909ed76640 1 -- 10.20.193.78:0/1113406264 reap_dead start 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 shutdown_connections 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 --2- 10.20.193.78:0/1113406264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90a009c4a0 0x7f90a011df00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 >> 10.20.193.78:0/1113406264 conn(0x7f90a00a6620 msgr2=0x7f90a00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 shutdown_connections 2026-02-17T05:48:11.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.701+0000 7f90a4bf5640 1 -- 10.20.193.78:0/1113406264 wait complete. 2026-02-17T05:48:11.733 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:48:11.734 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:48:11.734 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:48:11.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.800+0000 7f297cd55640 10 monclient: get_monmap_and_config 2026-02-17T05:48:11.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.800+0000 7f297cd55640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.800+0000 7f297cd55640 10 monclient: monmap: 2026-02-17T05:48:11.801 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.801 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.801 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:11.801451+0000 2026-02-17T05:48:11.801 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:11.801451+0000 2026-02-17T05:48:11.802 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.802 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.802 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:11.802 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:11.802 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:11.802 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:11.802 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:11.802 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.800+0000 7f297cd55640 1 Processor -- start 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.800+0000 7f297cd55640 1 -- start start 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.800+0000 7f297cd55640 10 monclient: init 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f297809ea90 0x7f29780a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient(hunting): picked mon.noname-c con 0x7f297809ea90 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f297809ee70 0x7f297809f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient(hunting): picked mon.noname-a con 0x7f297809ee70 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809d980 0x7f297809ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient(hunting): picked mon.noname-b con 0x7f297809d980 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f29780a0d00 con 0x7f297809ee70 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f29780a0a20 con 0x7f297809ea90 2026-02-17T05:48:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f29780a0460 con 0x7f297809d980 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f297cd55640 10 monclient(hunting): authenticate will time out at 737.355098s 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809d980 0x7f297809ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2975d74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f297809ee70 0x7f297809f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809d980 0x7f297809ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57312/0 (socket says 10.20.193.78:57312) 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 -- 10.20.193.78:0/1036348984 learned_addr learned my addr 10.20.193.78:0/1036348984 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient(hunting): get_auth_request con 0x7f297809d980 auth_method 0 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2975d74640 10 monclient(hunting): get_auth_request con 0x7f297809ee70 auth_method 0 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2975d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2975d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2975d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976575640 1 --2- 10.20.193.78:0/1036348984 >> v2:10.20.193.115:3300/0 conn(0x7f297809ea90 0x7f29780a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976575640 10 monclient(hunting): get_auth_request con 0x7f297809ea90 auth_method 0 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2975d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2975d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2975d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient(hunting): handle_auth_done global_id 4469 payload 274 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 -- 10.20.193.78:0/1036348984 >> v2:10.20.193.115:3300/0 conn(0x7f297809ea90 msgr2=0x7f29780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 --2- 10.20.193.78:0/1036348984 >> v2:10.20.193.115:3300/0 conn(0x7f297809ea90 0x7f29780a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 -- 10.20.193.78:0/1036348984 >> v2:10.20.193.78:3300/0 conn(0x7f297809ee70 msgr2=0x7f297809f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 --2- 10.20.193.78:0/1036348984 >> v2:10.20.193.78:3300/0 conn(0x7f297809ee70 0x7f297809f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 monclient: found mon.noname-b 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.801+0000 7f2976d76640 1 -- 10.20.193.78:0/1036348984 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29780a0740 con 0x7f297809d980 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2976d76640 1 -- 10.20.193.78:0/1036348984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809d980 msgr2=0x7f297809ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2976d76640 1 --2- 10.20.193.78:0/1036348984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809d980 0x7f297809ddb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f297806ecd0 tx=0x7f296802f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefdbbc7ec736718 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 1 -- 10.20.193.78:0/1036348984 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296802fe60 con 0x7f297809d980 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: dump: 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.809 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f297cd55640 5 monclient: authenticate success, global_id 4469 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f297cd55640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 1 -- 10.20.193.78:0/1036348984 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29680373f0 con 0x7f297809d980 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 1 -- 10.20.193.78:0/1036348984 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29680377c0 con 0x7f297809d980 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f297cd55640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f297cd55640 4 set_mon_vals no callback set 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:11.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.803+0000 7f2975573640 10 monclient: dump: 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 10 monclient: shutdown 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 -- 10.20.193.78:0/1036348984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809d980 msgr2=0x7f297809ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 --2- 10.20.193.78:0/1036348984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809d980 0x7f297809ddb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f297806ecd0 tx=0x7f296802f230 comp rx=0 tx=0).stop 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 -- 10.20.193.78:0/1036348984 shutdown_connections 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 --2- 10.20.193.78:0/1036348984 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809d980 0x7f297809ddb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 --2- 10.20.193.78:0/1036348984 >> v2:10.20.193.78:3300/0 conn(0x7f297809ee70 0x7f297809f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 --2- 10.20.193.78:0/1036348984 >> v2:10.20.193.115:3300/0 conn(0x7f297809ea90 0x7f29780a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 -- 10.20.193.78:0/1036348984 >> 10.20.193.78:0/1036348984 conn(0x7f29780a7520 msgr2=0x7f29780a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 -- 10.20.193.78:0/1036348984 shutdown_connections 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 1 -- 10.20.193.78:0/1036348984 wait complete. 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.804+0000 7f297cd55640 10 monclient: monmap: 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:11.812 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 1 Processor -- start 2026-02-17T05:48:11.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 1 -- start start 2026-02-17T05:48:11.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient: init 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297809d980 0x7f2978072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient(hunting): picked mon.noname-b con 0x7f297809d980 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809ea90 0x7f29780731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient(hunting): picked mon.noname-c con 0x7f297809ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f297809ee70 0x7f29780736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient(hunting): picked mon.noname-a con 0x7f297809ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29780a66a0 con 0x7f297809ee70 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29780a1910 con 0x7f297809d980 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f29780a6820 con 0x7f297809ea90 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f297cd55640 10 monclient(hunting): authenticate will time out at 737.359386s 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809ea90 0x7f29780731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f297809ee70 0x7f29780736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809ea90 0x7f29780731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57328/0 (socket says 10.20.193.78:57328) 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f297809ee70 0x7f29780736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45858/0 (socket says 10.20.193.78:45858) 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 -- 10.20.193.78:0/871367508 learned_addr learned my addr 10.20.193.78:0/871367508 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient(hunting): get_auth_request con 0x7f297809ea90 auth_method 0 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 10 monclient(hunting): get_auth_request con 0x7f297809ee70 auth_method 0 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976575640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297809d980 0x7f2978072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976575640 10 monclient(hunting): get_auth_request con 0x7f297809d980 auth_method 0 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient(hunting): handle_auth_done global_id 4472 payload 824 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297809d980 msgr2=0x7f2978072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297809d980 0x7f2978072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f297809ee70 msgr2=0x7f29780736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f297809ee70 0x7f29780736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 monclient: found mon.noname-c 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2975d74640 1 -- 10.20.193.78:0/871367508 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2978131c90 con 0x7f297809ea90 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.805+0000 7f2976d76640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f297809ee70 0x7f29780736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f2975d74640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809ea90 msgr2=0x7f29780731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f2975d74640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809ea90 0x7f29780731a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f296000ebe0 tx=0x7f296000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960017070 con 0x7f297809ea90 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: dump: 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.821 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296000ce50 con 0x7f297809ea90 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f297cd55640 5 monclient: authenticate success, global_id 4472 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29600136b0 con 0x7f297809ea90 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f2974d72640 4 set_mon_vals no callback set 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f295effd640 10 monclient: dump: 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.822 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f297cd55640 10 monclient: _renew_subs 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f297cd55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2978131f70 con 0x7f297809ea90 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f297cd55640 10 monclient: _renew_subs 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f297cd55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.807+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29780a94e0 con 0x7f297809ea90 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f297cd55640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f297cd55640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806ef90 0x7f297806f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f297cd55640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29780a1c10 con 0x7f297806ef90 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806ef90 0x7f297806f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 10 monclient: get_auth_request con 0x7f297806ef90 auth_method 0 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2960024060 con 0x7f297809ea90 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 10 monclient(hunting): handle_auth_done global_id 44110 payload 824 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f2976575640 1 -- 10.20.193.78:0/871367508 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29780a13d0 con 0x7f297806ef90 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f295effd640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f294403df00 0x7f29440403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.808+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(86..86 src has 1..86) ==== 4795+0+0 (secure 0 0 0) 0x7f29600528a0 con 0x7f297809ea90 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.809+0000 7f2976d76640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f294403df00 0x7f29440403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.809+0000 7f2976d76640 10 monclient: get_auth_request con 0x7f294403df00 auth_method 0 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.810+0000 7f2976575640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806ef90 msgr2=0x7f297806f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.810+0000 7f2976575640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806ef90 0x7f297806f370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f296c010830 tx=0x7f296c010d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c01a030 con 0x7f297806ef90 2026-02-17T05:48:11.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 10 monclient: dump: 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f296c00eb80 con 0x7f297806ef90 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806ef90 msgr2=0x7f297806f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806ef90 0x7f297806f370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f296c010830 tx=0x7f296c010d00 comp rx=0 tx=0).stop 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806ef90 msgr2=0x7f297806f370 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f295effd640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806ef90 0x7f297806f370 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f296c010830 tx=0x7f296c010d00 comp rx=0 tx=0).stop 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.811+0000 7f2976d76640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f294403df00 0x7f29440403c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f297806e270 tx=0x7f2968033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f297cd55640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f297cd55640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806f8b0 0x7f29781d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f297cd55640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29780744d0 con 0x7f297806f8b0 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806f8b0 0x7f29781d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 10 monclient: get_auth_request con 0x7f297806f8b0 auth_method 0 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 10 monclient(hunting): handle_auth_done global_id 44113 payload 824 2026-02-17T05:48:11.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.814+0000 7f2976575640 1 -- 10.20.193.78:0/871367508 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2978073c20 con 0x7f297806f8b0 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f2976575640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806f8b0 msgr2=0x7f29781d93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f2976575640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806f8b0 0x7f29781d93b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f296c010830 tx=0x7f296c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c00f040 con 0x7f297806f8b0 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 10 monclient: dump: 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.828 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f296c004030 con 0x7f297806f8b0 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806f8b0 msgr2=0x7f29781d93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806f8b0 0x7f29781d93b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f296c010830 tx=0x7f296c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806f8b0 msgr2=0x7f29781d93b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f295effd640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f297806f8b0 0x7f29781d93b0 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f296c010830 tx=0x7f296c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:11.829 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 24, 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 15465, 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.830 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.831 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:48:11.832 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:11.833 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.834 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.835 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:11.836 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:11.837 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:11.838 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:11.838 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:11.838 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:11.838 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:11.838 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:11.838 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:48:11.838 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f294403df00 msgr2=0x7f29440403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f294403df00 0x7f29440403c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f297806e270 tx=0x7f2968033000 comp rx=0 tx=0).stop 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 10 monclient: shutdown 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809ea90 msgr2=0x7f29780731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809ea90 0x7f29780731a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f296000ebe0 tx=0x7f296000c520 comp rx=0 tx=0).stop 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f2976d76640 1 -- 10.20.193.78:0/871367508 reap_dead start 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 shutdown_connections 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 --2- 10.20.193.78:0/871367508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f297809ea90 0x7f29780731a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 >> 10.20.193.78:0/871367508 conn(0x7f29780a7520 msgr2=0x7f2978138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 shutdown_connections 2026-02-17T05:48:11.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.817+0000 7f297cd55640 1 -- 10.20.193.78:0/871367508 wait complete. 2026-02-17T05:48:11.840 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:48:11.840 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:48:11.840 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:48:11.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.904+0000 7f50cc4de640 10 monclient: get_monmap_and_config 2026-02-17T05:48:11.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.904+0000 7f50cc4de640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.904+0000 7f50cc4de640 10 monclient: monmap: 2026-02-17T05:48:11.908 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.908 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.908 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:11.906198+0000 2026-02-17T05:48:11.908 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:11.906198+0000 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 1 Processor -- start 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 1 -- start start 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient: init 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f50c406c890 0x7f50c406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient(hunting): picked mon.noname-a con 0x7f50c406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f50c409c4a0 0x7f50c406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient(hunting): picked mon.noname-c con 0x7f50c409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:11.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f50c4134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50c406daf0 con 0x7f50c4134df0 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f50c40a0880 con 0x7f50c406c890 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f50c406d6f0 con 0x7f50c409c4a0 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50cc4de640 10 monclient(hunting): authenticate will time out at 737.459761s 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f50c406c890 0x7f50c406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f50c406c890 0x7f50c406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45872/0 (socket says 10.20.193.78:45872) 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 1 -- 10.20.193.78:0/802094189 learned_addr learned my addr 10.20.193.78:0/802094189 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 10 monclient(hunting): get_auth_request con 0x7f50c406c890 auth_method 0 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50c9a52640 1 --2- 10.20.193.78:0/802094189 >> v2:10.20.193.115:3300/0 conn(0x7f50c409c4a0 0x7f50c406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50c9a52640 10 monclient(hunting): get_auth_request con 0x7f50c409c4a0 auth_method 0 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50c9a52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50c9a52640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50c9a52640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.905+0000 7f50ca253640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50c9a52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50c9a52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50c9a52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 10 monclient(hunting): handle_auth_done global_id 4506 payload 274 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 1 -- 10.20.193.78:0/802094189 >> v2:10.20.193.115:3300/0 conn(0x7f50c409c4a0 msgr2=0x7f50c406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 1 --2- 10.20.193.78:0/802094189 >> v2:10.20.193.115:3300/0 conn(0x7f50c409c4a0 0x7f50c406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 1 -- 10.20.193.78:0/802094189 >> v1:10.20.193.115:6789/0 conn(0x7f50c4134df0 legacy=0x7f50c4135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 1 monclient: found mon.noname-a 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.906+0000 7f50ca253640 1 -- 10.20.193.78:0/802094189 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c409cc80 con 0x7f50c406c890 2026-02-17T05:48:11.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50ca253640 1 -- 10.20.193.78:0/802094189 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c406c890 msgr2=0x7f50c406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50ca253640 1 --2- 10.20.193.78:0/802094189 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c406c890 0x7f50c406cc70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50c409c980 tx=0x7f50b402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67749b0f930f7358 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 1 -- 10.20.193.78:0/802094189 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b403c070 con 0x7f50c406c890 2026-02-17T05:48:11.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:48:11.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: dump: 2026-02-17T05:48:11.915 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 1 -- 10.20.193.78:0/802094189 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b402fca0 con 0x7f50c406c890 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50cc4de640 5 monclient: authenticate success, global_id 4506 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 1 -- 10.20.193.78:0/802094189 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4037660 con 0x7f50c406c890 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50cc4de640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50cc4de640 4 set_mon_vals no callback set 2026-02-17T05:48:11.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.908+0000 7f50c9251640 10 monclient: dump: 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 10 monclient: shutdown 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 -- 10.20.193.78:0/802094189 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c406c890 msgr2=0x7f50c406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 --2- 10.20.193.78:0/802094189 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c406c890 0x7f50c406cc70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50c409c980 tx=0x7f50b402ee60 comp rx=0 tx=0).stop 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 -- 10.20.193.78:0/802094189 shutdown_connections 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 --2- 10.20.193.78:0/802094189 >> v2:10.20.193.115:3300/0 conn(0x7f50c409c4a0 0x7f50c406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 --2- 10.20.193.78:0/802094189 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c406c890 0x7f50c406cc70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 -- 10.20.193.78:0/802094189 >> 10.20.193.78:0/802094189 conn(0x7f50c40a6620 msgr2=0x7f50c40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 -- 10.20.193.78:0/802094189 shutdown_connections 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 -- 10.20.193.78:0/802094189 wait complete. 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 10 monclient: build_initial_monmap 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 10 monclient: monmap: 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 Processor -- start 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 1 -- start start 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.909+0000 7f50cc4de640 10 monclient: init 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:11.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50c406c890 0x7f50c41cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient(hunting): picked mon.noname-b con 0x7f50c406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50c409c4a0 0x7f50c41cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient(hunting): picked mon.noname-c con 0x7f50c409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c4134df0 0x7f50c4121620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient(hunting): picked mon.noname-a con 0x7f50c4134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50c40a01a0 con 0x7f50c4134df0 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50c40a0020 con 0x7f50c406c890 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f50c409be20 con 0x7f50c409c4a0 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50cc4de640 10 monclient(hunting): authenticate will time out at 737.464245s 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c4134df0 0x7f50c4121620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c4134df0 0x7f50c4121620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45886/0 (socket says 10.20.193.78:45886) 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 -- 10.20.193.78:0/1286124168 learned_addr learned my addr 10.20.193.78:0/1286124168 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient(hunting): get_auth_request con 0x7f50c4134df0 auth_method 0 2026-02-17T05:48:11.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50c9a52640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50c409c4a0 0x7f50c41cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50c9a52640 10 monclient(hunting): get_auth_request con 0x7f50c409c4a0 auth_method 0 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50c9a52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50c9a52640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50c9a52640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50c9a52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50c9a52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50c9a52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50ca253640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50c406c890 0x7f50c41cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50ca253640 10 monclient(hunting): get_auth_request con 0x7f50c406c890 auth_method 0 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50ca253640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50ca253640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50ca253640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient(hunting): handle_auth_done global_id 4509 payload 824 2026-02-17T05:48:11.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 -- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50c409c4a0 msgr2=0x7f50c41cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50c409c4a0 0x7f50c41cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 -- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50c406c890 msgr2=0x7f50c41cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50c406c890 0x7f50c41cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient: _finish_hunting 0 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 monclient: found mon.noname-a 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.910+0000 7f50caa54640 1 -- 10.20.193.78:0/1286124168 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c4125290 con 0x7f50c4134df0 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50caa54640 1 -- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c4134df0 msgr2=0x7f50c4121620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50caa54640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c4134df0 0x7f50c4121620 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50c000cb90 tx=0x7f50c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 1 -- 10.20.193.78:0/1286124168 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0016020 con 0x7f50c4134df0 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: dump: 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.921 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: _finish_auth 0 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 1 -- 10.20.193.78:0/1286124168 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c0012420 con 0x7f50c4134df0 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50cc4de640 5 monclient: authenticate success, global_id 4509 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 1 -- 10.20.193.78:0/1286124168 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0012810 con 0x7f50c4134df0 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50c8a50640 4 set_mon_vals no callback set 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50aaffd640 10 monclient: dump: 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:11.922 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50cc4de640 10 monclient: _renew_subs 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50cc4de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c4123de0 con 0x7f50c4134df0 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50cc4de640 10 monclient: _renew_subs 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50cc4de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.912+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c40a1770 con 0x7f50c4134df0 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50cc4de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50cc4de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50cc4de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c4074980 con 0x7f50c4134df0 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50aaffd640 1 -- 10.20.193.78:0/1286124168 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f50c00129b0 con 0x7f50c4134df0 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50aaffd640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f509403deb0 0x7f5094040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50aaffd640 1 -- 10.20.193.78:0/1286124168 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4795+0+0 (secure 0 0 0) 0x7f50c00531a0 con 0x7f50c4134df0 2026-02-17T05:48:11.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50ca253640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f509403deb0 0x7f5094040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:11.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.913+0000 7f50ca253640 10 monclient: get_auth_request con 0x7f509403deb0 auth_method 0 2026-02-17T05:48:11.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.916+0000 7f50aaffd640 1 -- 10.20.193.78:0/1286124168 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f50c0020020 con 0x7f50c4134df0 2026-02-17T05:48:11.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.916+0000 7f50aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:11.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.916+0000 7f50aaffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:11.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:11.916+0000 7f50ca253640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f509403deb0 0x7f5094040370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50b4036000 tx=0x7f50b4039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:12.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.032+0000 7f50cc4de640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:48:12.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.032+0000 7f50cc4de640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:48:12.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.032+0000 7f50cc4de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:12.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.032+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f50c4135a20 con 0x7f50c4134df0 2026-02-17T05:48:12.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.032+0000 7f50aaffd640 1 -- 10.20.193.78:0/1286124168 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f50c0012cd0 con 0x7f50c4134df0 2026-02-17T05:48:12.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.032+0000 7f50aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:48:12.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.032+0000 7f50aaffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:12.033 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.032+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.033 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.032+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=10,logm=67} crc {auth=1514644883,config=692619927,health=3897446751,kv=584707085,logm=3585635748}) 2026-02-17T05:48:12.033 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.032+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=2940381945,config=692619927,health=1625394897,kv=449989585,logm=2916380544}) 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.032+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.032+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=10,logm=67} crc {auth=1514644883,config=692619927,health=3897446751,kv=584707085,logm=3585635748}) 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.032+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=12,kv=12,logm=64} crc {auth=2284341679,config=692619927,health=3897446751,kv=449989585,logm=2877828485}) 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2272773195}) 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001640313}) 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2272773195}) 2026-02-17T05:48:12.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f509403deb0 msgr2=0x7f5094040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:12.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f509403deb0 0x7f5094040370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50b4036000 tx=0x7f50b4039040 comp rx=0 tx=0).stop 2026-02-17T05:48:12.034 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=395624467}) 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 10 monclient: shutdown 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c4134df0 msgr2=0x7f50c4121620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c4134df0 0x7f50c4121620 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50c000cb90 tx=0x7f50c000a430 comp rx=0 tx=0).stop 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 shutdown_connections 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50c4134df0 0x7f50c4121620 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50c409c4a0 0x7f50c41cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f509403deb0 0x7f5094040370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 --2- 10.20.193.78:0/1286124168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50c406c890 0x7f50c41cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 >> 10.20.193.78:0/1286124168 conn(0x7f50c40a6620 msgr2=0x7f50c4071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 shutdown_connections 2026-02-17T05:48:12.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:12.033+0000 7f50cc4de640 1 -- 10.20.193.78:0/1286124168 wait complete. 2026-02-17T05:48:12.035 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259247066}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=796702165}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259247066}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.033+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=823038144}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707744065}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2415008395}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707744065}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=413041000}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3067742543}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1096553978}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3067742543}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280262194}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069529202}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=239941695}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1069529202}) 2026-02-17T05:48:12.036 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.034+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1620798829}) 2026-02-17T05:48:12.042 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:48:12.305 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.303+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.305 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.303+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=21,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=13,osdmap=23} crc {logm=1414571956,mdsmap=1160443688,mgr=69100650,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1524734769,mon_config_key=501681677,monmap=1784110042,osd_metadata=278982410,osd_pg_creating=2248316729,osd_snap=315486445,osdmap=4193592366}) 2026-02-17T05:48:12.305 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.303+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=23,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=13,osdmap=18} crc {logm=2550183954,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1524734769,mon_config_key=624035425,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2248316729,osd_snap=315486445,osdmap=2024526112}) 2026-02-17T05:48:12.305 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.303+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.305 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.303+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=21,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=13,osdmap=23} crc {logm=1414571956,mdsmap=1160443688,mgr=69100650,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1524734769,mon_config_key=501681677,monmap=1784110042,osd_metadata=278982410,osd_pg_creating=2248316729,osd_snap=315486445,osdmap=4193592366}) 2026-02-17T05:48:12.306 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.303+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=13,osdmap=21} crc {logm=784809653,mdsmap=1160443688,mgr=3987721426,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4149926120,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2248316729,osd_snap=315486445,osdmap=1534370790}) 2026-02-17T05:48:12.367 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.366+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.368 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.366+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=694027973}) 2026-02-17T05:48:12.368 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.366+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162213568}) 2026-02-17T05:48:12.368 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.366+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.368 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.366+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=694027973}) 2026-02-17T05:48:12.368 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.366+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2488786310}) 2026-02-17T05:48:12.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.368+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.368+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=21} crc {osdmap=707968890}) 2026-02-17T05:48:12.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.368+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=2490402782}) 2026-02-17T05:48:12.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.368+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:48:12.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.368+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=21} crc {osdmap=707968890}) 2026-02-17T05:48:12.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:48:12.368+0000 7efed8c03640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=22} crc {osdmap=670690806}) 2026-02-17T05:48:13.043 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:48:13.043 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:13.048 INFO:tasks.rados.rados.0.trial115.stdout:373: done (2 left) 2026-02-17T05:48:13.048 INFO:tasks.rados.rados.0.trial115.stdout:374: done (1 left) 2026-02-17T05:48:13.049 INFO:tasks.rados.rados.0.trial115.stdout:377: done (0 left) 2026-02-17T05:48:13.049 INFO:tasks.rados.rados.0.trial115.stdout:378: delete oid 10 current snap is 31 2026-02-17T05:48:13.049 INFO:tasks.rados.rados.0.trial115.stdout:378: done (0 left) 2026-02-17T05:48:13.050 INFO:tasks.rados.rados.0.trial115.stdout:379: snap_remove snap 29 2026-02-17T05:48:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.110+0000 7f0bce779640 10 monclient: get_monmap_and_config 2026-02-17T05:48:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.110+0000 7f0bce779640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.110+0000 7f0bce779640 10 monclient: monmap: 2026-02-17T05:48:13.114 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.114 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:13.111788+0000 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:13.111788+0000 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.110+0000 7f0bce779640 1 Processor -- start 2026-02-17T05:48:13.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 1 -- start start 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient: init 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0bc806c890 0x7f0bc806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient(hunting): picked mon.noname-c con 0x7f0bc806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0bc809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806d1b0 0x7f0bc8135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient(hunting): picked mon.noname-b con 0x7f0bc806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc806daf0 con 0x7f0bc809c4a0 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc80a0880 con 0x7f0bc806c890 2026-02-17T05:48:13.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc809cc80 con 0x7f0bc806d1b0 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bce779640 10 monclient(hunting): authenticate will time out at 738.665349s 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806d1b0 0x7f0bc8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806d1b0 0x7f0bc8135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57338/0 (socket says 10.20.193.78:57338) 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 1 -- 10.20.193.78:0/3050084892 learned_addr learned my addr 10.20.193.78:0/3050084892 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient(hunting): get_auth_request con 0x7f0bc806d1b0 auth_method 0 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bc7fff640 1 --2- 10.20.193.78:0/3050084892 >> v2:10.20.193.115:3300/0 conn(0x7f0bc806c890 0x7f0bc806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bc7fff640 10 monclient(hunting): get_auth_request con 0x7f0bc806c890 auth_method 0 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bc7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bc7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bc7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient(hunting): handle_auth_done global_id 4478 payload 274 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 1 -- 10.20.193.78:0/3050084892 >> v2:10.20.193.115:3300/0 conn(0x7f0bc806c890 msgr2=0x7f0bc806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 1 --2- 10.20.193.78:0/3050084892 >> v2:10.20.193.115:3300/0 conn(0x7f0bc806c890 0x7f0bc806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 1 -- 10.20.193.78:0/3050084892 >> v1:10.20.193.115:6789/0 conn(0x7f0bc809c4a0 legacy=0x7f0bc8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 1 monclient: found mon.noname-b 2026-02-17T05:48:13.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.111+0000 7f0bcccef640 1 -- 10.20.193.78:0/3050084892 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc809cf60 con 0x7f0bc806d1b0 2026-02-17T05:48:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bcccef640 1 -- 10.20.193.78:0/3050084892 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806d1b0 msgr2=0x7f0bc8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bcccef640 1 --2- 10.20.193.78:0/3050084892 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806d1b0 0x7f0bc8135500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0bc806dcc0 tx=0x7f0bbc02f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e07c2b826e419e2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 1 -- 10.20.193.78:0/3050084892 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0373f0 con 0x7f0bc806d1b0 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: dump: 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.127 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 1 -- 10.20.193.78:0/3050084892 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc037590 con 0x7f0bc806d1b0 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 5 monclient: authenticate success, global_id 4478 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 1 -- 10.20.193.78:0/3050084892 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc037960 con 0x7f0bc806d1b0 2026-02-17T05:48:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 4 set_mon_vals no callback set 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bc6ffd640 10 monclient: dump: 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.129 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 10 monclient: shutdown 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 1 -- 10.20.193.78:0/3050084892 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806d1b0 msgr2=0x7f0bc8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 1 --2- 10.20.193.78:0/3050084892 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806d1b0 0x7f0bc8135500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0bc806dcc0 tx=0x7f0bbc02f200 comp rx=0 tx=0).stop 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 1 -- 10.20.193.78:0/3050084892 shutdown_connections 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 1 --2- 10.20.193.78:0/3050084892 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806d1b0 0x7f0bc8135500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 1 --2- 10.20.193.78:0/3050084892 >> v2:10.20.193.115:3300/0 conn(0x7f0bc806c890 0x7f0bc806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 1 -- 10.20.193.78:0/3050084892 >> 10.20.193.78:0/3050084892 conn(0x7f0bc80a6620 msgr2=0x7f0bc80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 1 -- 10.20.193.78:0/3050084892 shutdown_connections 2026-02-17T05:48:13.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.114+0000 7f0bce779640 1 -- 10.20.193.78:0/3050084892 wait complete. 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient: monmap: 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:13.131 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 1 Processor -- start 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 1 -- start start 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient: init 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806c890 0x7f0bc81cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient(hunting): picked mon.noname-c con 0x7f0bc806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc806d1b0 0x7f0bc81cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient(hunting): picked mon.noname-b con 0x7f0bc806d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc809c4a0 0x7f0bc809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient(hunting): picked mon.noname-a con 0x7f0bc809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:13.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc80a01a0 con 0x7f0bc809c4a0 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc80a0020 con 0x7f0bc806d1b0 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc809be20 con 0x7f0bc806c890 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bce779640 10 monclient(hunting): authenticate will time out at 738.669684s 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806c890 0x7f0bc81cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806c890 0x7f0bc81cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57346/0 (socket says 10.20.193.78:57346) 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 -- 10.20.193.78:0/2593694869 learned_addr learned my addr 10.20.193.78:0/2593694869 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient(hunting): get_auth_request con 0x7f0bc806c890 auth_method 0 2026-02-17T05:48:13.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc809c4a0 0x7f0bc809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 10 monclient(hunting): get_auth_request con 0x7f0bc809c4a0 auth_method 0 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc77fe640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc806d1b0 0x7f0bc81cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient(hunting): handle_auth_done global_id 4481 payload 824 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 -- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc806d1b0 msgr2=0x7f0bc81cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc806d1b0 0x7f0bc81cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 -- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc809c4a0 msgr2=0x7f0bc809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc809c4a0 0x7f0bc809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 monclient: found mon.noname-c 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc7fff640 1 -- 10.20.193.78:0/2593694869 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc806fd20 con 0x7f0bc806c890 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc77fe640 10 monclient: get_auth_request con 0x7f0bc806d1b0 auth_method 0 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bc77fe640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc806d1b0 0x7f0bc81cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.115+0000 7f0bcccef640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc809c4a0 0x7f0bc809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc7fff640 1 -- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806c890 msgr2=0x7f0bc81cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc7fff640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806c890 0x7f0bc81cf350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e820 tx=0x7f0bb400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 1 -- 10.20.193.78:0/2593694869 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4017070 con 0x7f0bc806c890 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: dump: 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.136 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 1 -- 10.20.193.78:0/2593694869 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb400cae0 con 0x7f0bc806c890 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 5 monclient: authenticate success, global_id 4481 2026-02-17T05:48:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 10 monclient: _renew_subs 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc8128eb0 con 0x7f0bc806c890 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 1 -- 10.20.193.78:0/2593694869 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400f040 con 0x7f0bc806c890 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc67fc640 4 set_mon_vals no callback set 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 10 monclient: dump: 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.138 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 10 monclient: _renew_subs 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc8127fc0 con 0x7f0bc806c890 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc8074980 con 0x7f0bc806c890 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 1 -- 10.20.193.78:0/2593694869 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0bb4012050 con 0x7f0bc806c890 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9803deb0 0x7f0b98040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.118+0000 7f0bc4ff9640 1 -- 10.20.193.78:0/2593694869 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 4767+0+0 (secure 0 0 0) 0x7f0bb4052650 con 0x7f0bc806c890 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.119+0000 7f0bc77fe640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9803deb0 0x7f0b98040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.119+0000 7f0bc77fe640 10 monclient: get_auth_request con 0x7f0b9803deb0 auth_method 0 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.121+0000 7f0bc4ff9640 1 -- 10.20.193.78:0/2593694869 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0bb4023d80 con 0x7f0bc806c890 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.121+0000 7f0bc4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.121+0000 7f0bc4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.121+0000 7f0bc77fe640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9803deb0 0x7f0b98040370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8051ad0 tx=0x7f0bb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.240+0000 7f0bce779640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:48:13.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.240+0000 7f0bce779640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:13.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.240+0000 7f0bce779640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.240+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0bc809d760 con 0x7f0bc806c890 2026-02-17T05:48:13.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.240+0000 7f0bc4ff9640 1 -- 10.20.193.78:0/2593694869 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f0bb4013c10 con 0x7f0bc806c890 2026-02-17T05:48:13.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.240+0000 7f0bc4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:13.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.240+0000 7f0bc4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:13.243 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:48:13.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9803deb0 msgr2=0x7f0b98040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9803deb0 0x7f0b98040370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8051ad0 tx=0x7f0bb8009040 comp rx=0 tx=0).stop 2026-02-17T05:48:13.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 10 monclient: shutdown 2026-02-17T05:48:13.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806c890 msgr2=0x7f0bc81cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806c890 0x7f0bc81cf350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e820 tx=0x7f0bb400ecf0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 shutdown_connections 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc809c4a0 0x7f0bc809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9803deb0 0x7f0b98040370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc806d1b0 0x7f0bc81cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 --2- 10.20.193.78:0/2593694869 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc806c890 0x7f0bc81cf350 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 >> 10.20.193.78:0/2593694869 conn(0x7f0bc80a6620 msgr2=0x7f0bc8072130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 shutdown_connections 2026-02-17T05:48:13.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.242+0000 7f0bce779640 1 -- 10.20.193.78:0/2593694869 wait complete. 2026-02-17T05:48:13.251 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:48:13.251 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:48:13.252 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:48:13.252 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:48:13.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient: get_monmap_and_config 2026-02-17T05:48:13.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient: monmap: 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:13.321388+0000 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:13.321388+0000 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:13.324 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 1 Processor -- start 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 1 -- start start 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient: init 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient(hunting): picked mon.noname-b con 0x7ff4c406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff4c409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff4c406d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c4135c10 con 0x7ff4c409c4a0 2026-02-17T05:48:13.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c406daf0 con 0x7ff4c406d1b0 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c40a0880 con 0x7ff4c406c890 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4cc61a640 10 monclient(hunting): authenticate will time out at 738.874921s 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4c9b8e640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff4c409c4a0 0x7ff4c4134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33732/0 (socket says 10.20.193.78:33732) 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4c9b8e640 1 -- 10.20.193.78:0/1951533621 learned_addr learned my addr 10.20.193.78:0/1951533621 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.320+0000 7ff4ca38f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient(hunting): get_auth_request con 0x7ff4c406c890 auth_method 0 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient(hunting): handle_auth_done global_id 4484 payload 274 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 1 -- 10.20.193.78:0/1951533621 >> v1:10.20.193.115:6789/0 conn(0x7ff4c406d1b0 legacy=0x7ff4c4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 1 -- 10.20.193.78:0/1951533621 >> v1:10.20.193.78:6789/0 conn(0x7ff4c409c4a0 legacy=0x7ff4c4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 1 monclient: found mon.noname-b 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.321+0000 7ff4ca38f640 1 -- 10.20.193.78:0/1951533621 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c409cc80 con 0x7ff4c406c890 2026-02-17T05:48:13.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4ca38f640 1 -- 10.20.193.78:0/1951533621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 msgr2=0x7ff4c406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4ca38f640 1 --2- 10.20.193.78:0/1951533621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c406cc70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098d0 tx=0x7ff4b402ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0a226818896284e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 1 -- 10.20.193.78:0/1951533621 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b403c070 con 0x7ff4c406c890 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: dump: 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.337 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 1 -- 10.20.193.78:0/1951533621 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b402fcc0 con 0x7ff4c406c890 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4cc61a640 5 monclient: authenticate success, global_id 4484 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4cc61a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 1 -- 10.20.193.78:0/1951533621 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4037660 con 0x7ff4c406c890 2026-02-17T05:48:13.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4c938d640 10 monclient: dump: 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.339 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4cc61a640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.323+0000 7ff4cc61a640 4 set_mon_vals no callback set 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 10 monclient: shutdown 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 -- 10.20.193.78:0/1951533621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 msgr2=0x7ff4c406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 --2- 10.20.193.78:0/1951533621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c406cc70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098d0 tx=0x7ff4b402ed60 comp rx=0 tx=0).stop 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 -- 10.20.193.78:0/1951533621 shutdown_connections 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 --2- 10.20.193.78:0/1951533621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c406cc70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 -- 10.20.193.78:0/1951533621 >> 10.20.193.78:0/1951533621 conn(0x7ff4c40a6620 msgr2=0x7ff4c40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 -- 10.20.193.78:0/1951533621 shutdown_connections 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 -- 10.20.193.78:0/1951533621 wait complete. 2026-02-17T05:48:13.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 10 monclient: monmap: 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 Processor -- start 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 1 -- start start 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.324+0000 7ff4cc61a640 10 monclient: init 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c4133f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient(hunting): picked mon.noname-c con 0x7ff4c406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4c406d1b0 0x7ff4c41344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient(hunting): picked mon.noname-b con 0x7ff4c406d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c409c4a0 0x7ff4c4121a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient(hunting): picked mon.noname-a con 0x7ff4c409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c40a0020 con 0x7ff4c409c4a0 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c406d600 con 0x7ff4c406d1b0 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c40a01a0 con 0x7ff4c406c890 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cc61a640 10 monclient(hunting): authenticate will time out at 738.879105s 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c4133f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c4133f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57360/0 (socket says 10.20.193.78:57360) 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 -- 10.20.193.78:0/2679558386 learned_addr learned my addr 10.20.193.78:0/2679558386 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient(hunting): get_auth_request con 0x7ff4c406c890 auth_method 0 2026-02-17T05:48:13.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cab90640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c409c4a0 0x7ff4c4121a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cab90640 10 monclient(hunting): get_auth_request con 0x7ff4c409c4a0 auth_method 0 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cab90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cab90640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cab90640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cab90640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cab90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4cab90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4c9b8e640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4c406d1b0 0x7ff4c41344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient(hunting): handle_auth_done global_id 4487 payload 824 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4c406d1b0 msgr2=0x7ff4c41344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4c406d1b0 0x7ff4c41344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c409c4a0 msgr2=0x7ff4c4121a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c409c4a0 0x7ff4c4121a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 monclient: found mon.noname-c 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.325+0000 7ff4ca38f640 1 -- 10.20.193.78:0/2679558386 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c4134f10 con 0x7ff4c406c890 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4ca38f640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 msgr2=0x7ff4c4133f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4ca38f640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c4133f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098d0 tx=0x7ff4b40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4046070 con 0x7ff4c406c890 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: dump: 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.346 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4041870 con 0x7ff4c406c890 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b403c040 con 0x7ff4c406c890 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4aaffd640 10 monclient: dump: 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.347 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4c8b8c640 4 set_mon_vals no callback set 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 5 monclient: authenticate success, global_id 4487 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 10 monclient: _renew_subs 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c4124420 con 0x7ff4c406c890 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 10 monclient: _renew_subs 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c409d810 con 0x7ff4c406c890 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.327+0000 7ff4cc61a640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c41cd3a0 0x7ff4c4121f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4cc61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4125af0 con 0x7ff4c41cd3a0 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c41cd3a0 0x7ff4c4121f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 10 monclient: get_auth_request con 0x7ff4c41cd3a0 auth_method 0 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff4b4041c60 con 0x7ff4c406c890 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 10 monclient(hunting): handle_auth_done global_id 4518 payload 824 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4c9b8e640 1 -- 10.20.193.78:0/2679558386 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4c4068410 con 0x7ff4c41cd3a0 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4aaffd640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49403df00 0x7ff4940403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 4767+0+0 (secure 0 0 0) 0x7ff4b4078030 con 0x7ff4c406c890 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4cab90640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49403df00 0x7ff4940403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.328+0000 7ff4cab90640 10 monclient: get_auth_request con 0x7ff49403df00 auth_method 0 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4c9b8e640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c41cd3a0 msgr2=0x7ff4c4121f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4c9b8e640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c41cd3a0 0x7ff4c4121f40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff4c41232a0 tx=0x7ff4b800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8007070 con 0x7ff4c41cd3a0 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 10 monclient: dump: 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.351 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff4b8017070 con 0x7ff4c41cd3a0 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c41cd3a0 msgr2=0x7ff4c4121f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c41cd3a0 0x7ff4c4121f40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff4c41232a0 tx=0x7ff4b800d960 comp rx=0 tx=0).stop 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c41cd3a0 msgr2=0x7ff4c4121f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.330+0000 7ff4aaffd640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c41cd3a0 0x7ff4c4121f40 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff4c41232a0 tx=0x7ff4b800d960 comp rx=0 tx=0).stop 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.331+0000 7ff4cab90640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49403df00 0x7ff4940403c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4073e30 tx=0x7ff4c0004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4cc61a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:48:13.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4cc61a640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c4122480 0x7ff4c41d8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4cc61a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4122890 con 0x7ff4c4122480 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c4122480 0x7ff4c41d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 10 monclient: get_auth_request con 0x7ff4c4122480 auth_method 0 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 10 monclient(hunting): handle_auth_done global_id 4521 payload 824 2026-02-17T05:48:13.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.333+0000 7ff4c9b8e640 1 -- 10.20.193.78:0/2679558386 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff4c406cc70 con 0x7ff4c4122480 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.335+0000 7ff4c9b8e640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c4122480 msgr2=0x7ff4c41d8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.335+0000 7ff4c9b8e640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c4122480 0x7ff4c41d8480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4c41232a0 tx=0x7ff4b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8012410 con 0x7ff4c4122480 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 10 monclient: dump: 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.355 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7ff4b8012410 con 0x7ff4c4122480 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c4122480 msgr2=0x7ff4c41d8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c4122480 0x7ff4c41d8480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4c41232a0 tx=0x7ff4b8004a40 comp rx=0 tx=0).stop 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c4122480 msgr2=0x7ff4c41d8480 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:13.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4aaffd640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4c4122480 0x7ff4c41d8480 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4c41232a0 tx=0x7ff4b8004a40 comp rx=0 tx=0).stop 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 24, 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 143140, 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:48:13.357 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.358 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.359 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:48:13.360 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.361 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.362 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:13.363 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:48:13.364 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:48:13.365 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.366 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.367 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:48:13.368 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:48:13.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49403df00 msgr2=0x7ff4940403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49403df00 0x7ff4940403c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4073e30 tx=0x7ff4c0004200 comp rx=0 tx=0).stop 2026-02-17T05:48:13.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 10 monclient: shutdown 2026-02-17T05:48:13.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 msgr2=0x7ff4c4133f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cab90640 1 -- 10.20.193.78:0/2679558386 reap_dead start 2026-02-17T05:48:13.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c4133f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098d0 tx=0x7ff4b40047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 shutdown_connections 2026-02-17T05:48:13.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 --2- 10.20.193.78:0/2679558386 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4c406c890 0x7ff4c4133f90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 >> 10.20.193.78:0/2679558386 conn(0x7ff4c40a6620 msgr2=0x7ff4c4135460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 shutdown_connections 2026-02-17T05:48:13.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.336+0000 7ff4cc61a640 1 -- 10.20.193.78:0/2679558386 wait complete. 2026-02-17T05:48:13.370 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:48:13.370 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:48:13.370 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.435+0000 7fa562eda640 10 monclient: get_monmap_and_config 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.435+0000 7fa562eda640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.435+0000 7fa562eda640 10 monclient: monmap: 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:13.437078+0000 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:13.437078+0000 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:13.437 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 1 Processor -- start 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 1 -- start start 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient: init 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa55c06c890 0x7fa55c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient(hunting): picked mon.noname-a con 0x7fa55c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa55c06cc70 0x7fa55c06d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient(hunting): picked mon.noname-c con 0x7fa55c06cc70 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa55c134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa55c074150 con 0x7fa55c134df0 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa55c09ef70 con 0x7fa55c06c890 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa55c09f250 con 0x7fa55c06cc70 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa562eda640 10 monclient(hunting): authenticate will time out at 738.990607s 2026-02-17T05:48:13.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa55c06c890 0x7fa55c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa55c06c890 0x7fa55c0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45920/0 (socket says 10.20.193.78:45920) 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 1 -- 10.20.193.78:0/3722399854 learned_addr learned my addr 10.20.193.78:0/3722399854 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient(hunting): get_auth_request con 0x7fa55c06c890 auth_method 0 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa553fff640 1 --2- 10.20.193.78:0/3722399854 >> v2:10.20.193.115:3300/0 conn(0x7fa55c06cc70 0x7fa55c06d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa553fff640 10 monclient(hunting): get_auth_request con 0x7fa55c06cc70 auth_method 0 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa553fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa553fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa553fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient(hunting): handle_auth_done global_id 4524 payload 274 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 1 -- 10.20.193.78:0/3722399854 >> v2:10.20.193.115:3300/0 conn(0x7fa55c06cc70 msgr2=0x7fa55c06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 1 --2- 10.20.193.78:0/3722399854 >> v2:10.20.193.115:3300/0 conn(0x7fa55c06cc70 0x7fa55c06d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 1 -- 10.20.193.78:0/3722399854 >> v1:10.20.193.115:6789/0 conn(0x7fa55c134df0 legacy=0x7fa55c135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 1 monclient: found mon.noname-a 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:48:13.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.436+0000 7fa560c4f640 1 -- 10.20.193.78:0/3722399854 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c09df40 con 0x7fa55c06c890 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa560c4f640 1 -- 10.20.193.78:0/3722399854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 msgr2=0x7fa55c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa560c4f640 1 --2- 10.20.193.78:0/3722399854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 0x7fa55c0a01f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa55c074e00 tx=0x7fa54c02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb80f8a35286eaff server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 1 -- 10.20.193.78:0/3722399854 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c03c070 con 0x7fa55c06c890 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: dump: 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.440 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 1 -- 10.20.193.78:0/3722399854 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c02fd50 con 0x7fa55c06c890 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 5 monclient: authenticate success, global_id 4524 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 1 -- 10.20.193.78:0/3722399854 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c037660 con 0x7fa55c06c890 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 4 set_mon_vals no callback set 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa5537fe640 10 monclient: dump: 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 10 monclient: shutdown 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 1 -- 10.20.193.78:0/3722399854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 msgr2=0x7fa55c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 1 --2- 10.20.193.78:0/3722399854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 0x7fa55c0a01f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa55c074e00 tx=0x7fa54c02eed0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 1 -- 10.20.193.78:0/3722399854 shutdown_connections 2026-02-17T05:48:13.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 1 --2- 10.20.193.78:0/3722399854 >> v2:10.20.193.115:3300/0 conn(0x7fa55c06cc70 0x7fa55c06d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 1 --2- 10.20.193.78:0/3722399854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 0x7fa55c0a01f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 1 -- 10.20.193.78:0/3722399854 >> 10.20.193.78:0/3722399854 conn(0x7fa55c0a6620 msgr2=0x7fa55c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 1 -- 10.20.193.78:0/3722399854 shutdown_connections 2026-02-17T05:48:13.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.439+0000 7fa562eda640 1 -- 10.20.193.78:0/3722399854 wait complete. 2026-02-17T05:48:13.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient: monmap: 2026-02-17T05:48:13.442 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 1 Processor -- start 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 1 -- start start 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient: init 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 0x7fa55c1cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient(hunting): picked mon.noname-a con 0x7fa55c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c06cc70 0x7fa55c134050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient(hunting): picked mon.noname-c con 0x7fa55c06cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa55c134df0 0x7fa55c134590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient(hunting): picked mon.noname-b con 0x7fa55c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c09d360 con 0x7fa55c06c890 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c09d1e0 con 0x7fa55c134df0 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c135910 con 0x7fa55c06cc70 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa562eda640 10 monclient(hunting): authenticate will time out at 738.994859s 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 0x7fa55c1cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 0x7fa55c1cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45930/0 (socket says 10.20.193.78:45930) 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 1 -- 10.20.193.78:0/4242508718 learned_addr learned my addr 10.20.193.78:0/4242508718 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 10 monclient(hunting): get_auth_request con 0x7fa55c06c890 auth_method 0 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa553fff640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c06cc70 0x7fa55c134050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa560c4f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa553fff640 10 monclient(hunting): get_auth_request con 0x7fa55c06cc70 auth_method 0 2026-02-17T05:48:13.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa553fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa553fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.440+0000 7fa553fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 10 monclient(hunting): handle_auth_done global_id 4527 payload 824 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c06cc70 msgr2=0x7fa55c134050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c06cc70 0x7fa55c134050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa55c134df0 msgr2=0x7fa55c134590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa55c134df0 0x7fa55c134590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 1 monclient: found mon.noname-a 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.441+0000 7fa560c4f640 1 -- 10.20.193.78:0/4242508718 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c1287a0 con 0x7fa55c06c890 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa560c4f640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 msgr2=0x7fa55c1cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa560c4f640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 0x7fa55c1cf170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa54c002410 tx=0x7fa54c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c03c040 con 0x7fa55c06c890 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: dump: 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.444 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c047070 con 0x7fa55c06c890 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa562eda640 5 monclient: authenticate success, global_id 4527 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa562eda640 10 monclient: _renew_subs 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa562eda640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c1cf6b0 con 0x7fa55c06c890 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c041800 con 0x7fa55c06c890 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa552ffd640 4 set_mon_vals no callback set 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa5517fa640 10 monclient: dump: 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa562eda640 10 monclient: _renew_subs 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa562eda640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:13.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c0a3a90 con 0x7fa55c06c890 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.443+0000 7fa562eda640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa562eda640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c1283c0 0x7fa55c124af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa562eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c068410 con 0x7fa55c1283c0 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa54c0419e0 con 0x7fa55c06c890 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c1283c0 0x7fa55c124af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 10 monclient: get_auth_request con 0x7fa55c1283c0 auth_method 0 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 10 monclient(hunting): handle_auth_done global_id 4493 payload 824 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa553fff640 1 -- 10.20.193.78:0/4242508718 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa55c074980 con 0x7fa55c1283c0 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa5517fa640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa52c03df30 0x7fa52c0403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 4767+0+0 (secure 0 0 0) 0x7fa54c077b10 con 0x7fa55c06c890 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa561450640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa52c03df30 0x7fa52c0403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.444+0000 7fa561450640 10 monclient: get_auth_request con 0x7fa52c03df30 auth_method 0 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa553fff640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c1283c0 msgr2=0x7fa55c124af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa553fff640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c1283c0 0x7fa55c124af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa55c126390 tx=0x7fa544011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54400f040 con 0x7fa55c1283c0 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 10 monclient: dump: 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.447 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa54400fe50 con 0x7fa55c1283c0 2026-02-17T05:48:13.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:13.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c1283c0 msgr2=0x7fa55c124af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c1283c0 0x7fa55c124af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa55c126390 tx=0x7fa544011ca0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c1283c0 msgr2=0x7fa55c124af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:13.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.446+0000 7fa5517fa640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c1283c0 0x7fa55c124af0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa55c126390 tx=0x7fa544011ca0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.447+0000 7fa561450640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa52c03df30 0x7fa52c0403f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa558004980 tx=0x7fa558003fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.449+0000 7fa562eda640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:48:13.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.449+0000 7fa562eda640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c125030 0x7fa55c1da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.449+0000 7fa562eda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.449+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c09d810 con 0x7fa55c125030 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.449+0000 7fa553fff640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c125030 0x7fa55c1da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 10 monclient: get_auth_request con 0x7fa55c125030 auth_method 0 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 10 monclient(hunting): handle_auth_done global_id 4496 payload 824 2026-02-17T05:48:13.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.450+0000 7fa553fff640 1 -- 10.20.193.78:0/4242508718 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa55c0a01d0 con 0x7fa55c125030 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa553fff640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c125030 msgr2=0x7fa55c1da4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa553fff640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c125030 0x7fa55c1da4e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa55c126390 tx=0x7fa544004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544015660 con 0x7fa55c125030 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 10 monclient: dump: 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7fa544003e30 con 0x7fa55c125030 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c125030 msgr2=0x7fa55c1da4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c125030 0x7fa55c1da4e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa55c126390 tx=0x7fa544004830 comp rx=0 tx=0).stop 2026-02-17T05:48:13.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c125030 msgr2=0x7fa55c1da4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa5517fa640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa55c125030 0x7fa55c1da4e0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa55c126390 tx=0x7fa544004830 comp rx=0 tx=0).stop 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 24, 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 143257, 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.454 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:13.455 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.456 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.457 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:48:13.458 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:48:13.459 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa52c03df30 msgr2=0x7fa52c0403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa52c03df30 0x7fa52c0403f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa558004980 tx=0x7fa558003fc0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 10 monclient: shutdown 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 msgr2=0x7fa55c1cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 1 --2- 10.20.193.78:0/4242508718 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa55c06c890 0x7fa55c1cf170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa54c002410 tx=0x7fa54c004830 comp rx=0 tx=0).stop 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa561450640 1 -- 10.20.193.78:0/4242508718 reap_dead start 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa561450640 1 -- 10.20.193.78:0/4242508718 reap_dead start 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 shutdown_connections 2026-02-17T05:48:13.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 >> 10.20.193.78:0/4242508718 conn(0x7fa55c0a6620 msgr2=0x7fa55c0735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 shutdown_connections 2026-02-17T05:48:13.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.452+0000 7fa562eda640 1 -- 10.20.193.78:0/4242508718 wait complete. 2026-02-17T05:48:13.461 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:48:13.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:48:13.461 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:48:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient: get_monmap_and_config 2026-02-17T05:48:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient: monmap: 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:13.529444+0000 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:13.529444+0000 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 1 Processor -- start 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 1 -- start start 2026-02-17T05:48:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient: init 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf0c06c890 0x7faf0c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient(hunting): picked mon.noname-a con 0x7faf0c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient(hunting): picked mon.noname-b con 0x7faf0c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7faf0c134df0 0x7faf0c135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient(hunting): picked mon.noname-c con 0x7faf0c134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7faf0c0a0880 con 0x7faf0c06c890 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7faf0c06d6f0 con 0x7faf0c134df0 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf0c06daf0 con 0x7faf0c09c4a0 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.528+0000 7faf147f5640 10 monclient(hunting): authenticate will time out at 739.082966s 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf1256a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf0c06c890 0x7faf0c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57408/0 (socket says 10.20.193.78:57408) 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 -- 10.20.193.78:0/1038417516 learned_addr learned my addr 10.20.193.78:0/1038417516 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient(hunting): get_auth_request con 0x7faf0c09c4a0 auth_method 0 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf1256a640 10 monclient(hunting): get_auth_request con 0x7faf0c06c890 auth_method 0 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf1256a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf1256a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf1256a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf12d6b640 1 --2- 10.20.193.78:0/1038417516 >> v2:10.20.193.115:3300/0 conn(0x7faf0c134df0 0x7faf0c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf12d6b640 10 monclient(hunting): get_auth_request con 0x7faf0c134df0 auth_method 0 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf12d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf12d6b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf12d6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient(hunting): handle_auth_done global_id 4499 payload 274 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 -- 10.20.193.78:0/1038417516 >> v2:10.20.193.115:3300/0 conn(0x7faf0c134df0 msgr2=0x7faf0c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 --2- 10.20.193.78:0/1038417516 >> v2:10.20.193.115:3300/0 conn(0x7faf0c134df0 0x7faf0c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 -- 10.20.193.78:0/1038417516 >> v2:10.20.193.78:3300/0 conn(0x7faf0c06c890 msgr2=0x7faf0c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 --2- 10.20.193.78:0/1038417516 >> v2:10.20.193.78:3300/0 conn(0x7faf0c06c890 0x7faf0c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 monclient: found mon.noname-b 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.529+0000 7faf11d69640 1 -- 10.20.193.78:0/1038417516 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c09cc80 con 0x7faf0c09c4a0 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11d69640 1 -- 10.20.193.78:0/1038417516 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 msgr2=0x7faf0c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11d69640 1 --2- 10.20.193.78:0/1038417516 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c06d1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0a7490 tx=0x7faf0002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eeae89344ec6f28 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 1 -- 10.20.193.78:0/1038417516 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0003c070 con 0x7faf0c09c4a0 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: dump: 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.538 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 1 -- 10.20.193.78:0/1038417516 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0002fca0 con 0x7faf0c09c4a0 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 1 -- 10.20.193.78:0/1038417516 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00037660 con 0x7faf0c09c4a0 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf147f5640 5 monclient: authenticate success, global_id 4499 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf147f5640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf147f5640 4 set_mon_vals no callback set 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.531+0000 7faf11568640 10 monclient: dump: 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 10 monclient: shutdown 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 -- 10.20.193.78:0/1038417516 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 msgr2=0x7faf0c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 --2- 10.20.193.78:0/1038417516 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c06d1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0a7490 tx=0x7faf0002ee60 comp rx=0 tx=0).stop 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 -- 10.20.193.78:0/1038417516 shutdown_connections 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 --2- 10.20.193.78:0/1038417516 >> v2:10.20.193.115:3300/0 conn(0x7faf0c134df0 0x7faf0c135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 --2- 10.20.193.78:0/1038417516 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c06d1b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 --2- 10.20.193.78:0/1038417516 >> v2:10.20.193.78:3300/0 conn(0x7faf0c06c890 0x7faf0c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 -- 10.20.193.78:0/1038417516 >> 10.20.193.78:0/1038417516 conn(0x7faf0c0a6620 msgr2=0x7faf0c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 -- 10.20.193.78:0/1038417516 shutdown_connections 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 -- 10.20.193.78:0/1038417516 wait complete. 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 10 monclient: monmap: 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 Processor -- start 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 1 -- start start 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 10 monclient: init 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.532+0000 7faf147f5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0c06c890 0x7faf0c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 10 monclient(hunting): picked mon.noname-a con 0x7faf0c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 10 monclient(hunting): picked mon.noname-c con 0x7faf0c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c134df0 0x7faf0c122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 10 monclient(hunting): picked mon.noname-b con 0x7faf0c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c0a01a0 con 0x7faf0c06c890 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c0a0020 con 0x7faf0c134df0 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf0c09be20 con 0x7faf0c09c4a0 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf147f5640 10 monclient(hunting): authenticate will time out at 739.087036s 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf1256a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0c06c890 0x7faf0c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c1cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57414/0 (socket says 10.20.193.78:57414) 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 -- 10.20.193.78:0/416811230 learned_addr learned my addr 10.20.193.78:0/416811230 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient(hunting): get_auth_request con 0x7faf0c09c4a0 auth_method 0 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf1256a640 10 monclient(hunting): get_auth_request con 0x7faf0c06c890 auth_method 0 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf1256a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf1256a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf1256a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf12d6b640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c134df0 0x7faf0c122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient(hunting): handle_auth_done global_id 4502 payload 824 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c134df0 msgr2=0x7faf0c122c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c134df0 0x7faf0c122c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0c06c890 msgr2=0x7faf0c1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0c06c890 0x7faf0c1cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 monclient: found mon.noname-c 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.533+0000 7faf11d69640 1 -- 10.20.193.78:0/416811230 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c1265a0 con 0x7faf0c09c4a0 2026-02-17T05:48:13.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf11d69640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 msgr2=0x7faf0c1cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf11d69640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c1cf890 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf00009a70 tx=0x7faf00004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00047070 con 0x7faf0c09c4a0 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: dump: 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0002fe50 con 0x7faf0c09c4a0 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 5 monclient: authenticate success, global_id 4502 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0003c040 con 0x7faf0c09c4a0 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 10 monclient: _renew_subs 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0c1253d0 con 0x7faf0c09c4a0 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf10d67640 4 set_mon_vals no callback set 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faef2ffd640 10 monclient: dump: 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 10 monclient: _renew_subs 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0c0a1730 con 0x7faf0c09c4a0 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c063880 0x7faf0c063c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.535+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c074980 con 0x7faf0c063880 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7faf0003f9f0 con 0x7faf0c09c4a0 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c063880 0x7faf0c063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 10 monclient: get_auth_request con 0x7faf0c063880 auth_method 0 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faef2ffd640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faedc03deb0 0x7faedc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 4767+0+0 (secure 0 0 0) 0x7faf000774d0 con 0x7faf0c09c4a0 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 10 monclient(hunting): handle_auth_done global_id 44131 payload 824 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf1256a640 1 -- 10.20.193.78:0/416811230 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf0c135240 con 0x7faf0c063880 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf12d6b640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faedc03deb0 0x7faedc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.536+0000 7faf12d6b640 10 monclient: get_auth_request con 0x7faedc03deb0 auth_method 0 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.538+0000 7faf1256a640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c063880 msgr2=0x7faf0c063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.538+0000 7faf1256a640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c063880 0x7faf0c063c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faefc0119d0 tx=0x7faefc011ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.538+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc019070 con 0x7faf0c063880 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.538+0000 7faef2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.538+0000 7faef2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.538+0000 7faef2ffd640 10 monclient: dump: 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.548 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.538+0000 7faf12d6b640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faedc03deb0 0x7faedc040370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf0c125000 tx=0x7faf0800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.539+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7faefc004030 con 0x7faf0c063880 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.539+0000 7faef2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.539+0000 7faef2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.539+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c063880 msgr2=0x7faf0c063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.539+0000 7faef2ffd640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c063880 0x7faf0c063c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faefc0119d0 tx=0x7faefc011ea0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.539+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c063880 msgr2=0x7faf0c063c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.539+0000 7faef2ffd640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c063880 0x7faf0c063c60 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faefc0119d0 tx=0x7faefc011ea0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf147f5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf147f5640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c06f410 0x7faf0c1da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf147f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c0744d0 con 0x7faf0c06f410 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c06f410 0x7faf0c1da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 10 monclient: get_auth_request con 0x7faf0c06f410 auth_method 0 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 10 monclient(hunting): handle_auth_done global_id 44134 payload 824 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.542+0000 7faf1256a640 1 -- 10.20.193.78:0/416811230 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faf0c1262c0 con 0x7faf0c06f410 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf1256a640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c06f410 msgr2=0x7faf0c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf1256a640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c06f410 0x7faf0c1da560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faefc01b000 tx=0x7faefc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc019040 con 0x7faf0c06f410 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 10 monclient: dump: 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.550 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7faefc003e10 con 0x7faf0c06f410 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c06f410 msgr2=0x7faf0c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c06f410 0x7faf0c1da560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faefc01b000 tx=0x7faefc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c06f410 msgr2=0x7faf0c1da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faef2ffd640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0c06f410 0x7faf0c1da560 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faefc01b000 tx=0x7faefc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 24, 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 17194, 2026-02-17T05:48:13.551 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:48:13.552 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:48:13.553 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:13.554 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.555 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.556 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.557 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:13.558 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:13.559 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:13.559 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:13.559 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:13.559 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:48:13.559 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faedc03deb0 msgr2=0x7faedc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faedc03deb0 0x7faedc040370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf0c125000 tx=0x7faf0800b040 comp rx=0 tx=0).stop 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 10 monclient: shutdown 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 msgr2=0x7faf0c1cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c1cf890 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf00009a70 tx=0x7faf00004830 comp rx=0 tx=0).stop 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf12d6b640 1 -- 10.20.193.78:0/416811230 reap_dead start 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 shutdown_connections 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 --2- 10.20.193.78:0/416811230 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0c09c4a0 0x7faf0c1cf890 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 >> 10.20.193.78:0/416811230 conn(0x7faf0c0a6620 msgr2=0x7faf0c0a2900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 shutdown_connections 2026-02-17T05:48:13.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.545+0000 7faf147f5640 1 -- 10.20.193.78:0/416811230 wait complete. 2026-02-17T05:48:13.560 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:48:13.560 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:48:13.561 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:48:13.561 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:48:13.561 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:48:13.561 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:48:13.561 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:48:13.571 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:48:13.571 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:48:13.571 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:13.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.676+0000 7f891786c640 10 monclient: get_monmap_and_config 2026-02-17T05:48:13.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.676+0000 7f891786c640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.676+0000 7f891786c640 10 monclient: monmap: 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:13.677523+0000 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:13.677523+0000 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:13.678 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.676+0000 7f891786c640 1 Processor -- start 2026-02-17T05:48:13.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.676+0000 7f891786c640 1 -- start start 2026-02-17T05:48:13.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.676+0000 7f891786c640 10 monclient: init 2026-02-17T05:48:13.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2026-02-17T05:48:13.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f891009ea90 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:13.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f891009ee70 0x7f891009f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 10 monclient(hunting): picked mon.noname-a con 0x7f891009ee70 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f891013d640 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89100a0a20 con 0x7f891009ea90 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89100a0460 con 0x7f891013d640 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f89100a0d00 con 0x7f891009ee70 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f891786c640 10 monclient(hunting): authenticate will time out at 739.231232s 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f89155e1640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f891009ea90 legacy=0x7f89100a13d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f8914de0640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f891009ee70 msgr2=0x7f891009f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f8914de0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f891009ee70 0x7f891009f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f8915de2640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f891013d640 0x7f891013fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:34064/0 (socket says 10.20.193.78:34064) 2026-02-17T05:48:13.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.677+0000 7f8915de2640 1 -- 10.20.193.78:0/1505277947 learned_addr learned my addr 10.20.193.78:0/1505277947 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:48:13.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.679+0000 7f8915de2640 1 -- 10.20.193.78:0/1505277947 >> v1:10.20.193.115:6789/0 conn(0x7f891013d640 legacy=0x7f891013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.679+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 778508888 0 0) 0x7f89100a0460 con 0x7f891013d640 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.679+0000 7f88fffff640 10 monclient(hunting): _negotiate 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.679+0000 7f88fffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.679+0000 7f88fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.679+0000 7f88fffff640 10 monclient(hunting): my global_id is 44137 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.679+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88f8003620 con 0x7f891013d640 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1650230704 0 0) 0x7f88f8003620 con 0x7f891013d640 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 >> v2:10.20.193.78:3300/0 conn(0x7f891009ee70 msgr2=0x7f891009f2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 --2- 10.20.193.78:0/1505277947 >> v2:10.20.193.78:3300/0 conn(0x7f891009ee70 0x7f891009f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 >> v1:10.20.193.78:6789/0 conn(0x7f891009ea90 legacy=0x7f89100a13d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:48:13.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89100a0740 con 0x7f891013d640 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f890c003530 con 0x7f891013d640 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: dump: 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.691 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 >> v1:10.20.193.115:6789/0 conn(0x7f891013d640 legacy=0x7f891013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8003ab0 0x7f88f8003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient(hunting): picked mon.b con 0x7f88f8003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:13.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88f8004900 0x7f88f8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient(hunting): picked mon.c con 0x7f88f8004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88f80054c0 0x7f88f800bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient(hunting): picked mon.a con 0x7f88f80054c0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89100a0d00 con 0x7f88f80054c0 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f88f800c400 con 0x7f88f8003ab0 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f88f800c600 con 0x7f88f8004900 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f88fffff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8915de2640 1 -- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88f80054c0 msgr2=0x7f88f800bb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8915de2640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88f80054c0 0x7f88f800bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:13.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88f8004900 0x7f88f8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient(hunting): get_auth_request con 0x7f88f8004900 auth_method 0 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f89155e1640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8003ab0 0x7f88f8003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f89155e1640 10 monclient(hunting): get_auth_request con 0x7f88f8003ab0 auth_method 0 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f89155e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f89155e1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f89155e1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f89155e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f89155e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f89155e1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient(hunting): handle_auth_done global_id 44137 payload 293 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 1 -- 10.20.193.78:0/1505277947 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8003ab0 msgr2=0x7f88f8003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8003ab0 0x7f88f8003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 1 -- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88f80054c0 msgr2=0x7f88f800bb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88f80054c0 0x7f88f800bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 1 monclient: found mon.c 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8915de2640 1 -- 10.20.193.78:0/1505277947 reap_dead start 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 1 -- 10.20.193.78:0/1505277947 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f8003620 con 0x7f88f8004900 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f8914de0640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f891786c640 5 monclient: authenticate success, global_id 44137 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.680+0000 7f891786c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.682+0000 7f8914de0640 1 -- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88f8004900 msgr2=0x7f88f8004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.682+0000 7f8914de0640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88f8004900 0x7f88f8004d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8904009e30 tx=0x7f890402f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e53646dd080a8bd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904009a30 con 0x7f88f8004900 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 10 monclient: dump: 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.696 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8904003cb0 con 0x7f88f8004900 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 1 -- 10.20.193.78:0/1505277947 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904003e60 con 0x7f88f8004900 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f88fffff640 10 monclient: dump: 2026-02-17T05:48:13.697 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 4 set_mon_vals no callback set 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 10 monclient: shutdown 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 1 -- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88f8004900 msgr2=0x7f88f8004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88f8004900 0x7f88f8004d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8904009e30 tx=0x7f890402f660 comp rx=0 tx=0).stop 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 1 -- 10.20.193.78:0/1505277947 shutdown_connections 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 1 --2- 10.20.193.78:0/1505277947 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88f8004900 0x7f88f8004d80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 1 -- 10.20.193.78:0/1505277947 >> 10.20.193.78:0/1505277947 conn(0x7f89100a7520 msgr2=0x7f89100a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 1 -- 10.20.193.78:0/1505277947 shutdown_connections 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 1 -- 10.20.193.78:0/1505277947 wait complete. 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 10 monclient: build_initial_monmap 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 10 monclient: monmap: 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:13.699 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.683+0000 7f891786c640 1 Processor -- start 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 1 -- start start 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient: init 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8004900 0x7f89101cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient(hunting): picked mon.noname-b con 0x7f88f8004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89101cfed0 0x7f891006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient(hunting): picked mon.noname-c con 0x7f89101cfed0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89101d0350 0x7f891006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient(hunting): picked mon.noname-a con 0x7f89101d0350 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89100a1550 con 0x7f89101d0350 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89100a13d0 con 0x7f88f8004900 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f89100a16d0 con 0x7f89101cfed0 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f891786c640 10 monclient(hunting): authenticate will time out at 739.238338s 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89101cfed0 0x7f891006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8915de2640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89101d0350 msgr2=0x7f891006ff60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89101cfed0 0x7f891006fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57438/0 (socket says 10.20.193.78:57438) 2026-02-17T05:48:13.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 -- 10.20.193.78:0/1197305124 learned_addr learned my addr 10.20.193.78:0/1197305124 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient(hunting): get_auth_request con 0x7f89101cfed0 auth_method 0 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8915de2640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89101d0350 0x7f891006ff60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f89155e1640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8004900 0x7f89101cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f89155e1640 10 monclient(hunting): get_auth_request con 0x7f88f8004900 auth_method 0 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f89155e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f89155e1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:13.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f89155e1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient(hunting): handle_auth_done global_id 4505 payload 824 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 -- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8004900 msgr2=0x7f89101cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8004900 0x7f89101cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 -- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89101d0350 msgr2=0x7f891006ff60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89101d0350 0x7f891006ff60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient: _finish_hunting 0 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 monclient: found mon.noname-c 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.684+0000 7f8914de0640 1 -- 10.20.193.78:0/1197305124 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89100a9e00 con 0x7f89101cfed0 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f8914de0640 1 -- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89101cfed0 msgr2=0x7f891006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f8914de0640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89101cfed0 0x7f891006fa20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f890402fb70 tx=0x7f8904005b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 1 -- 10.20.193.78:0/1197305124 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904048070 con 0x7f89101cfed0 2026-02-17T05:48:13.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: dump: 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: _finish_auth 0 2026-02-17T05:48:13.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: _check_auth_tickets 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 1 -- 10.20.193.78:0/1197305124 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f890403d040 con 0x7f89101cfed0 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f891786c640 5 monclient: authenticate success, global_id 4505 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 1 -- 10.20.193.78:0/1197305124 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904009a30 con 0x7f89101cfed0 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88ff7fe640 4 set_mon_vals no callback set 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f88fdffb640 10 monclient: dump: 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:13.705 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f891786c640 10 monclient: _renew_subs 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f891786c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8910073920 con 0x7f89101cfed0 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f891786c640 10 monclient: _renew_subs 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f891786c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.686+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8910071aa0 con 0x7f89101cfed0 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f891786c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f891786c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f891786c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891009f2c0 con 0x7f89101cfed0 2026-02-17T05:48:13.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f88fdffb640 1 -- 10.20.193.78:0/1197305124 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8904037430 con 0x7f89101cfed0 2026-02-17T05:48:13.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f88fdffb640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f88e003dee0 0x7f88e00403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:13.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f88fdffb640 1 -- 10.20.193.78:0/1197305124 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 4767+0+0 (secure 0 0 0) 0x7f8904077e00 con 0x7f89101cfed0 2026-02-17T05:48:13.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f89155e1640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f88e003dee0 0x7f88e00403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:13.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.687+0000 7f89155e1640 10 monclient: get_auth_request con 0x7f88e003dee0 auth_method 0 2026-02-17T05:48:13.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.690+0000 7f88fdffb640 1 -- 10.20.193.78:0/1197305124 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f890407c070 con 0x7f89101cfed0 2026-02-17T05:48:13.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.690+0000 7f88fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:13.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.690+0000 7f88fdffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:13.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.690+0000 7f89155e1640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f88e003dee0 0x7f88e00403a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8910075790 tx=0x7f8900009520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:13.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.807+0000 7f891786c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:48:13.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.807+0000 7f891786c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:13.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.807+0000 7f891786c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:13.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.807+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f89100aa350 con 0x7f89101cfed0 2026-02-17T05:48:13.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.807+0000 7f88fdffb640 1 -- 10.20.193.78:0/1197305124 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f89100a9e00 con 0x7f89101cfed0 2026-02-17T05:48:13.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.807+0000 7f88fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:13.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.807+0000 7f88fdffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:13.810 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:48:13.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f88e003dee0 msgr2=0x7f88e00403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f88e003dee0 0x7f88e00403a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8910075790 tx=0x7f8900009520 comp rx=0 tx=0).stop 2026-02-17T05:48:13.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 10 monclient: shutdown 2026-02-17T05:48:13.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89101cfed0 msgr2=0x7f891006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89101cfed0 0x7f891006fa20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f890402fb70 tx=0x7f8904005b10 comp rx=0 tx=0).stop 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 shutdown_connections 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f89101d0350 0x7f891006ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89101cfed0 0x7f891006fa20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f88e003dee0 0x7f88e00403a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 --2- 10.20.193.78:0/1197305124 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88f8004900 0x7f89101cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 >> 10.20.193.78:0/1197305124 conn(0x7f89100a7520 msgr2=0x7f891013cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 shutdown_connections 2026-02-17T05:48:13.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:13.809+0000 7f891786c640 1 -- 10.20.193.78:0/1197305124 wait complete. 2026-02-17T05:48:16.819 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:16.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.886+0000 7fa24fd85640 10 monclient: get_monmap_and_config 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.886+0000 7fa24fd85640 10 monclient: build_initial_monmap 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.886+0000 7fa24fd85640 10 monclient: monmap: 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:16.887639+0000 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:16.887639+0000 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:16.888 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:16.889 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:16.889 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:16.889 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.886+0000 7fa24fd85640 1 Processor -- start 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.886+0000 7fa24fd85640 1 -- start start 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.886+0000 7fa24fd85640 10 monclient: init 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa24806c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa24806d380 0x7fa2480a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient(hunting): picked mon.noname-a con 0x7fa24806d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa24813c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:16.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient(hunting): picked mon.noname-b con 0x7fa24813a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa248074150 con 0x7fa24806c890 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa24809ef70 con 0x7fa24806d380 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa24809f250 con 0x7fa24813a240 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24fd85640 10 monclient(hunting): authenticate will time out at 742.441180s 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24d2f9640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fa24806d380 msgr2=0x7fa2480a01f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24d2f9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa24806d380 0x7fa2480a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa24813c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:16.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa24813c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57446/0 (socket says 10.20.193.78:57446) 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2512078619 learned_addr learned my addr 10.20.193.78:0/2512078619 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient(hunting): get_auth_request con 0x7fa24813a240 auth_method 0 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient(hunting): handle_auth_done global_id 4508 payload 274 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2512078619 >> v2:10.20.193.78:3300/0 conn(0x7fa24806d380 msgr2=0x7fa2480a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 1 --2- 10.20.193.78:0/2512078619 >> v2:10.20.193.78:3300/0 conn(0x7fa24806d380 0x7fa2480a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:16.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2512078619 >> v1:10.20.193.115:6789/0 conn(0x7fa24806c890 legacy=0x7fa24806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient: _finish_hunting 0 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 1 monclient: found mon.noname-b 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.887+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2512078619 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24809df40 con 0x7fa24813a240 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.889+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2512078619 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 msgr2=0x7fa24813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.889+0000 7fa24e2fb640 1 --2- 10.20.193.78:0/2512078619 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa24813c630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa24402f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a50d18296940c85 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 1 -- 10.20.193.78:0/2512078619 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440373f0 con 0x7fa24813a240 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: dump: 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:16.901 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: _finish_auth 0 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: _check_auth_tickets 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:16.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 1 -- 10.20.193.78:0/2512078619 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa244037590 con 0x7fa24813a240 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 5 monclient: authenticate success, global_id 4508 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 1 -- 10.20.193.78:0/2512078619 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244037960 con 0x7fa24813a240 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 4 set_mon_vals no callback set 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24caf8640 10 monclient: dump: 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:16.903 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 10 monclient: shutdown 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 1 -- 10.20.193.78:0/2512078619 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 msgr2=0x7fa24813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2512078619 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa24813c630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa24402f360 comp rx=0 tx=0).stop 2026-02-17T05:48:16.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 1 -- 10.20.193.78:0/2512078619 shutdown_connections 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2512078619 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa24813c630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2512078619 >> v2:10.20.193.78:3300/0 conn(0x7fa24806d380 0x7fa2480a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 1 -- 10.20.193.78:0/2512078619 >> 10.20.193.78:0/2512078619 conn(0x7fa2480a6620 msgr2=0x7fa2480a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 1 -- 10.20.193.78:0/2512078619 shutdown_connections 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 1 -- 10.20.193.78:0/2512078619 wait complete. 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 10 monclient: build_initial_monmap 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.890+0000 7fa24fd85640 10 monclient: monmap: 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:16.905 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 1 Processor -- start 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 1 -- start start 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient: init 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa24806c890 0x7fa2481cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:16.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient(hunting): picked mon.noname-b con 0x7fa24806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa24806d380 0x7fa248135190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient(hunting): picked mon.noname-a con 0x7fa24806d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa2481356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient(hunting): picked mon.noname-c con 0x7fa24813a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa24809bfa0 con 0x7fa24806d380 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa24809be20 con 0x7fa24806c890 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa24809c120 con 0x7fa24813a240 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:16.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24fd85640 10 monclient(hunting): authenticate will time out at 742.445385s 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24d2f9640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa24806d380 msgr2=0x7fa248135190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24d2f9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa24806d380 0x7fa248135190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa2481356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa2481356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57452/0 (socket says 10.20.193.78:57452) 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2671268964 learned_addr learned my addr 10.20.193.78:0/2671268964 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient(hunting): get_auth_request con 0x7fa24813a240 auth_method 0 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:16.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient(hunting): handle_auth_done global_id 4511 payload 824 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa24806c890 msgr2=0x7fa2481cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa24806c890 0x7fa2481cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa24806d380 msgr2=0x7fa248135190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa24806d380 0x7fa248135190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient: _finish_hunting 0 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 monclient: found mon.noname-c 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.891+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2671268964 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa248073630 con 0x7fa24813a240 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24e2fb640 1 -- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 msgr2=0x7fa2481356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:16.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24e2fb640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa2481356d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa244004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 1 -- 10.20.193.78:0/2671268964 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244046020 con 0x7fa24813a240 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: dump: 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:16.910 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: _finish_auth 0 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: _check_auth_tickets 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 1 -- 10.20.193.78:0/2671268964 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa244041870 con 0x7fa24813a240 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24fd85640 5 monclient: authenticate success, global_id 4511 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24fd85640 10 monclient: _renew_subs 2026-02-17T05:48:16.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24fd85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2481cf560 con 0x7fa24813a240 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 1 -- 10.20.193.78:0/2671268964 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244041c60 con 0x7fa24813a240 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa237fff640 4 set_mon_vals no callback set 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa2367fc640 10 monclient: dump: 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:16.912 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24fd85640 10 monclient: _renew_subs 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24fd85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:16.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.893+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24813e980 con 0x7fa24813a240 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa24fd85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa2367fc640 1 -- 10.20.193.78:0/2671268964 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa24404b410 con 0x7fa24813a240 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa24fd85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa24fd85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2480a2150 con 0x7fa24813a240 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa2367fc640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa21803de60 0x7fa218040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa2367fc640 1 -- 10.20.193.78:0/2671268964 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 4767+0+0 (secure 0 0 0) 0x7fa2440785f0 con 0x7fa24813a240 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa24dafa640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa21803de60 0x7fa218040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:16.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.894+0000 7fa24dafa640 10 monclient: get_auth_request con 0x7fa21803de60 auth_method 0 2026-02-17T05:48:16.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.897+0000 7fa24dafa640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa21803de60 0x7fa218040320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa238003ec0 tx=0x7fa238007ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:16.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.897+0000 7fa2367fc640 1 -- 10.20.193.78:0/2671268964 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa244050070 con 0x7fa24813a240 2026-02-17T05:48:16.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.897+0000 7fa2367fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:16.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:16.897+0000 7fa2367fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:17.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.013+0000 7fa24fd85640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:48:17.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.013+0000 7fa24fd85640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:17.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.013+0000 7fa24fd85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:17.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.013+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa2480a01d0 con 0x7fa24813a240 2026-02-17T05:48:17.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.013+0000 7fa24e2fb640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa2481356d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa244004b40 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fa244077000 mon_command_ack([{"prefix": "quorum_status"}]=0 v0) 2026-02-17T05:48:17.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.841+0000 7fa2367fc640 1 -- 10.20.193.78:0/2671268964 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fa244077000 con 0x7fa24813a240 2026-02-17T05:48:17.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.841+0000 7fa2367fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:17.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.841+0000 7fa2367fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:17.843 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:48:17.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa21803de60 msgr2=0x7fa218040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:17.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa21803de60 0x7fa218040320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa238003ec0 tx=0x7fa238007ab0 comp rx=0 tx=0).stop 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 10 monclient: shutdown 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 msgr2=0x7fa2481356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa2481356d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa244004b40 comp rx=0 tx=0).stop 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 shutdown_connections 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa24813a240 0x7fa2481356d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa24806d380 0x7fa248135190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa21803de60 0x7fa218040320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 --2- 10.20.193.78:0/2671268964 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa24806c890 0x7fa2481cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:17.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 >> 10.20.193.78:0/2671268964 conn(0x7fa2480a6620 msgr2=0x7fa2480a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:17.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 shutdown_connections 2026-02-17T05:48:17.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:17.843+0000 7fa24fd85640 1 -- 10.20.193.78:0/2671268964 wait complete. 2026-02-17T05:48:20.856 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:20.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.922+0000 7f19671ba640 10 monclient: get_monmap_and_config 2026-02-17T05:48:20.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.922+0000 7f19671ba640 10 monclient: build_initial_monmap 2026-02-17T05:48:20.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.922+0000 7f19671ba640 10 monclient: monmap: 2026-02-17T05:48:20.924 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:20.924 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:20.924 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:20.924133+0000 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:20.924133+0000 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:20.925 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 1 Processor -- start 2026-02-17T05:48:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 1 -- start start 2026-02-17T05:48:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient: init 2026-02-17T05:48:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2026-02-17T05:48:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196006c890 0x7f196006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient(hunting): picked mon.noname-b con 0x7f196006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f196009c4a0 0x7f196006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient(hunting): picked mon.noname-a con 0x7f196009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1960134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196006daf0 con 0x7f1960134df0 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f19600a0880 con 0x7f196009c4a0 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f196006d6f0 con 0x7f196006c890 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f19671ba640 10 monclient(hunting): authenticate will time out at 746.477665s 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f195ffff640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f196009c4a0 msgr2=0x7f196006d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:20.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f195ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f196009c4a0 0x7f196006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196006c890 0x7f196006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196006c890 0x7f196006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57480/0 (socket says 10.20.193.78:57480) 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 1 -- 10.20.193.78:0/2097140694 learned_addr learned my addr 10.20.193.78:0/2097140694 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient(hunting): get_auth_request con 0x7f196006c890 auth_method 0 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:20.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient(hunting): handle_auth_done global_id 4514 payload 274 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 1 -- 10.20.193.78:0/2097140694 >> v2:10.20.193.78:3300/0 conn(0x7f196009c4a0 msgr2=0x7f196006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 1 --2- 10.20.193.78:0/2097140694 >> v2:10.20.193.78:3300/0 conn(0x7f196009c4a0 0x7f196006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 1 -- 10.20.193.78:0/2097140694 >> v1:10.20.193.115:6789/0 conn(0x7f1960134df0 legacy=0x7f1960135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient: _finish_hunting 0 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 1 monclient: found mon.noname-b 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.923+0000 7f1964f2f640 1 -- 10.20.193.78:0/2097140694 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196009cc80 con 0x7f196006c890 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f1964f2f640 1 -- 10.20.193.78:0/2097140694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196006c890 msgr2=0x7f196006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f1964f2f640 1 --2- 10.20.193.78:0/2097140694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196006c890 0x7f196006cc70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1950009e50 tx=0x7f195002f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5fbbe146d0bf5b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 1 -- 10.20.193.78:0/2097140694 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500373f0 con 0x7f196006c890 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: dump: 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 5 monclient: authenticate success, global_id 4514 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 1 -- 10.20.193.78:0/2097140694 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1950037590 con 0x7f196006c890 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 4 set_mon_vals no callback set 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 1 -- 10.20.193.78:0/2097140694 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950037960 con 0x7f196006c890 2026-02-17T05:48:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f195f7fe640 10 monclient: dump: 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 10 monclient: shutdown 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 1 -- 10.20.193.78:0/2097140694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196006c890 msgr2=0x7f196006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 1 --2- 10.20.193.78:0/2097140694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196006c890 0x7f196006cc70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1950009e50 tx=0x7f195002f230 comp rx=0 tx=0).stop 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 1 -- 10.20.193.78:0/2097140694 shutdown_connections 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 1 --2- 10.20.193.78:0/2097140694 >> v2:10.20.193.78:3300/0 conn(0x7f196009c4a0 0x7f196006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 1 --2- 10.20.193.78:0/2097140694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196006c890 0x7f196006cc70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 1 -- 10.20.193.78:0/2097140694 >> 10.20.193.78:0/2097140694 conn(0x7f19600a6620 msgr2=0x7f19600a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 1 -- 10.20.193.78:0/2097140694 shutdown_connections 2026-02-17T05:48:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.926+0000 7f19671ba640 1 -- 10.20.193.78:0/2097140694 wait complete. 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient: build_initial_monmap 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient: monmap: 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:20.942 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 1 Processor -- start 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 1 -- start start 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient: init 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f196006c890 0x7f19601cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient(hunting): picked mon.noname-b con 0x7f196006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196009c4a0 0x7f19601cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient(hunting): picked mon.noname-c con 0x7f196009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1960134df0 0x7f196009dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient(hunting): picked mon.noname-a con 0x7f1960134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19600a01a0 con 0x7f1960134df0 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f19600a0020 con 0x7f196006c890 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f196009be20 con 0x7f196009c4a0 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f19671ba640 10 monclient(hunting): authenticate will time out at 746.481923s 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f1965730640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1960134df0 msgr2=0x7f196009dba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f1965730640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1960134df0 0x7f196009dba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f195ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196009c4a0 0x7f19601cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f195ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196009c4a0 0x7f19601cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57488/0 (socket says 10.20.193.78:57488) 2026-02-17T05:48:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f195ffff640 1 -- 10.20.193.78:0/525174979 learned_addr learned my addr 10.20.193.78:0/525174979 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f195ffff640 10 monclient(hunting): get_auth_request con 0x7f196009c4a0 auth_method 0 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f195ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f195ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.927+0000 7f195ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f1964f2f640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f196006c890 0x7f19601cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f1964f2f640 10 monclient(hunting): get_auth_request con 0x7f196006c890 auth_method 0 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f1964f2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f1964f2f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f1964f2f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 10 monclient(hunting): handle_auth_done global_id 4517 payload 824 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 1 -- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f196006c890 msgr2=0x7f19601cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f196006c890 0x7f19601cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 1 -- 10.20.193.78:0/525174979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1960134df0 msgr2=0x7f196009dba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1960134df0 0x7f196009dba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 1 monclient: found mon.noname-c 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.928+0000 7f195ffff640 1 -- 10.20.193.78:0/525174979 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19601272c0 con 0x7f196009c4a0 2026-02-17T05:48:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195ffff640 1 -- 10.20.193.78:0/525174979 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196009c4a0 msgr2=0x7f19601cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195ffff640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196009c4a0 0x7f19601cf970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f194800b780 tx=0x7f194800bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 1 -- 10.20.193.78:0/525174979 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948005270 con 0x7f196009c4a0 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: dump: 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:20.947 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 1 -- 10.20.193.78:0/525174979 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1948010070 con 0x7f196009c4a0 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 1 -- 10.20.193.78:0/525174979 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948005650 con 0x7f196009c4a0 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195d7fa640 10 monclient: dump: 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f195effd640 4 set_mon_vals no callback set 2026-02-17T05:48:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f19671ba640 5 monclient: authenticate success, global_id 4517 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f19671ba640 10 monclient: _renew_subs 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f19671ba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19601275a0 con 0x7f196009c4a0 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f19671ba640 10 monclient: _renew_subs 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f19671ba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.930+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1960134690 con 0x7f196009c4a0 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f19671ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f19671ba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f19671ba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1960068410 con 0x7f196009c4a0 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f195d7fa640 1 -- 10.20.193.78:0/525174979 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f194800c520 con 0x7f196009c4a0 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f195d7fa640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f193003de60 0x7f1930040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f195d7fa640 1 -- 10.20.193.78:0/525174979 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 4767+0+0 (secure 0 0 0) 0x7f1948052040 con 0x7f196009c4a0 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f1964f2f640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f193003de60 0x7f1930040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.931+0000 7f1964f2f640 10 monclient: get_auth_request con 0x7f193003de60 auth_method 0 2026-02-17T05:48:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.934+0000 7f195d7fa640 1 -- 10.20.193.78:0/525174979 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1960068410 con 0x7f196009c4a0 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.934+0000 7f195d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.934+0000 7f195d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:20.934+0000 7f1964f2f640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f193003de60 0x7f1930040320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f196009c880 tx=0x7f1950033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.048+0000 7f19671ba640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:48:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.048+0000 7f19671ba640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.048+0000 7f19671ba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.048+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f196006cc70 con 0x7f196009c4a0 2026-02-17T05:48:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.049+0000 7f195d7fa640 1 -- 10.20.193.78:0/525174979 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f194800ce10 con 0x7f196009c4a0 2026-02-17T05:48:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.049+0000 7f195d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.049+0000 7f195d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:21.051 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:48:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f193003de60 msgr2=0x7f1930040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f193003de60 0x7f1930040320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f196009c880 tx=0x7f1950033000 comp rx=0 tx=0).stop 2026-02-17T05:48:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 10 monclient: shutdown 2026-02-17T05:48:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196009c4a0 msgr2=0x7f19601cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196009c4a0 0x7f19601cf970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f194800b780 tx=0x7f194800bc50 comp rx=0 tx=0).stop 2026-02-17T05:48:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 shutdown_connections 2026-02-17T05:48:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1960134df0 0x7f196009dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f196009c4a0 0x7f19601cf970 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f193003de60 0x7f1930040320 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 --2- 10.20.193.78:0/525174979 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f196006c890 0x7f19601cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 >> 10.20.193.78:0/525174979 conn(0x7f19600a6620 msgr2=0x7f1960071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 shutdown_connections 2026-02-17T05:48:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:21.050+0000 7f19671ba640 1 -- 10.20.193.78:0/525174979 wait complete. 2026-02-17T05:48:24.059 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.125+0000 7f0c47a8f640 10 monclient: get_monmap_and_config 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.125+0000 7f0c47a8f640 10 monclient: build_initial_monmap 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.125+0000 7f0c47a8f640 10 monclient: monmap: 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:24.126601+0000 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:24.126601+0000 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:24.126 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:24.127 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:24.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.125+0000 7f0c47a8f640 1 Processor -- start 2026-02-17T05:48:24.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.125+0000 7f0c47a8f640 1 -- start start 2026-02-17T05:48:24.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.125+0000 7f0c47a8f640 10 monclient: init 2026-02-17T05:48:24.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:24.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2026-02-17T05:48:24.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:24.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient(hunting): picked mon.noname-b con 0x7f0c4006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient(hunting): picked mon.noname-c con 0x7f0c4009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0c40134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4006daf0 con 0x7f0c40134df0 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0c400a0880 con 0x7f0c4009c4a0 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4006d6f0 con 0x7f0c4006c890 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c47a8f640 10 monclient(hunting): authenticate will time out at 749.680109s 2026-02-17T05:48:24.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57510/0 (socket says 10.20.193.78:57510) 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 learned_addr learned my addr 10.20.193.78:0/2743947802 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 10 monclient(hunting): get_auth_request con 0x7f0c4006c890 auth_method 0 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:24.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 10 monclient(hunting): get_auth_request con 0x7f0c4009c4a0 auth_method 0 2026-02-17T05:48:24.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:24.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:24.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:24.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:24.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:48:24.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:24.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:24.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.126+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:24.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.128+0000 7f0c46005640 1 -- 10.20.193.78:0/2743947802 >> v1:10.20.193.115:6789/0 conn(0x7f0c40134df0 legacy=0x7f0c40135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:24.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:24.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 10 monclient(hunting): get_auth_request con 0x7f0c4006c890 auth_method 0 2026-02-17T05:48:24.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:24.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:24.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:24.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:24.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45003640 10 monclient(hunting): get_auth_request con 0x7f0c4009c4a0 auth_method 0 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45003640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45003640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.326+0000 7f0c45003640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.327+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.327+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.327+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:24.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.327+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:48:24.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 10 monclient(hunting): get_auth_request con 0x7f0c4006c890 auth_method 0 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 10 monclient(hunting): get_auth_request con 0x7f0c4009c4a0 auth_method 0 2026-02-17T05:48:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:24.727+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:48:25.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:25.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 10 monclient(hunting): get_auth_request con 0x7f0c4006c890 auth_method 0 2026-02-17T05:48:25.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:25.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:25.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:25.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:25.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:25.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.528+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.529+0000 7f0c45003640 10 monclient(hunting): get_auth_request con 0x7f0c4009c4a0 auth_method 0 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.529+0000 7f0c45003640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.529+0000 7f0c45003640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.529+0000 7f0c45003640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.529+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.529+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.529+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:25.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:25.529+0000 7f0c45003640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:48:27.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 10 monclient(hunting): tick 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 msgr2=0x7f0c4006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c4006c890 0x7f0c4006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 msgr2=0x7f0c4006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c4009c4a0 0x7f0c4006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 >> v1:10.20.193.115:6789/0 conn(0x7f0c40134df0 legacy=0x7f0c40135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 10 monclient: _add_conns ranks=[3,0] 2026-02-17T05:48:27.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 0x7f0c24005c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f0c24003820 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0c24006370 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4009cc80 con 0x7f0c24006370 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f0c4006d6f0 con 0x7f0c24003820 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c2f7fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 msgr2=0x7f0c24005c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 0x7f0c24005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:27.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.126+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v1:10.20.193.78:6789/0 conn(0x7f0c24006370 legacy=0x7f0c24008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:48:27.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.326+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 msgr2=0x7f0c24005c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:27.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.326+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 0x7f0c24005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:48:27.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.326+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v1:10.20.193.78:6789/0 conn(0x7f0c24006370 legacy=0x7f0c24008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:48:27.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.727+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 msgr2=0x7f0c24005c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:27.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.727+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 0x7f0c24005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:48:27.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:27.727+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v1:10.20.193.78:6789/0 conn(0x7f0c24006370 legacy=0x7f0c24008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:48:27.886 INFO:tasks.rados.rados.0.trial115.stdout:379: done (0 left) 2026-02-17T05:48:27.886 INFO:tasks.rados.rados.0.trial115.stdout:380: snap_create 2026-02-17T05:48:28.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:28.528+0000 7f0c45804640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 msgr2=0x7f0c24005c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:28.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:28.528+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 0x7f0c24005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:48:28.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:28.528+0000 7f0c45003640 1 -- 10.20.193.78:0/2743947802 >> v1:10.20.193.78:6789/0 conn(0x7f0c24006370 legacy=0x7f0c24008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:48:28.890 INFO:tasks.rados.rados.0.trial115.stdout:380: done (0 left) 2026-02-17T05:48:28.890 INFO:tasks.rados.rados.0.trial115.stdout:381: snap_remove snap 30 2026-02-17T05:48:29.890 INFO:tasks.rados.rados.0.trial115.stdout:381: done (0 left) 2026-02-17T05:48:29.891 INFO:tasks.rados.rados.0.trial115.stdout:382: write oid 34 current snap is 32 2026-02-17T05:48:29.891 INFO:tasks.rados.rados.0.trial115.stdout:382: seq_num 178 ranges {591646=470441,1566383=595717,2888822=102824} 2026-02-17T05:48:29.895 INFO:tasks.rados.rados.0.trial115.stdout:382: writing trial11545420-34 from 591646 to 1062087 tid 1 2026-02-17T05:48:29.899 INFO:tasks.rados.rados.0.trial115.stdout:382: writing trial11545420-34 from 1566383 to 2162100 tid 2 2026-02-17T05:48:29.902 INFO:tasks.rados.rados.0.trial115.stdout:382: writing trial11545420-34 from 2888822 to 2991646 tid 3 2026-02-17T05:48:29.902 INFO:tasks.rados.rados.0.trial115.stdout:383: read oid 18 snap -1 2026-02-17T05:48:29.902 INFO:tasks.rados.rados.0.trial115.stdout:383: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:48:29.902 INFO:tasks.rados.rados.0.trial115.stdout:384: delete oid 26 current snap is 32 2026-02-17T05:48:29.902 INFO:tasks.rados.rados.0.trial115.stdout:382: finishing write tid 1 to trial11545420-34 2026-02-17T05:48:29.902 INFO:tasks.rados.rados.0.trial115.stdout:382: finishing write tid 2 to trial11545420-34 2026-02-17T05:48:29.904 INFO:tasks.rados.rados.0.trial115.stdout:384: done (2 left) 2026-02-17T05:48:29.904 INFO:tasks.rados.rados.0.trial115.stdout:385: delete oid 36 current snap is 32 2026-02-17T05:48:29.906 INFO:tasks.rados.rados.0.trial115.stdout:382: finishing write tid 3 to trial11545420-34 2026-02-17T05:48:29.906 INFO:tasks.rados.rados.0.trial115.stdout:382: finishing write tid 4 to trial11545420-34 2026-02-17T05:48:29.906 INFO:tasks.rados.rados.0.trial115.stdout:382: finishing write tid 5 to trial11545420-34 2026-02-17T05:48:29.906 INFO:tasks.rados.rados.0.trial115.stdout:382: oid 34 updating version 0 to 39 2026-02-17T05:48:29.906 INFO:tasks.rados.rados.0.trial115.stdout:382: oid 34 updating version 39 to 41 2026-02-17T05:48:29.906 INFO:tasks.rados.rados.0.trial115.stdout:382: oid 34 version 41 is already newer than 38 2026-02-17T05:48:29.907 INFO:tasks.rados.rados.0.trial115.stdout:382: oid 34 version 41 is already newer than 40 2026-02-17T05:48:29.907 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 41 (ObjNum 178 snap 32 seq_num 178) dirty exists 2026-02-17T05:48:29.907 INFO:tasks.rados.rados.0.trial115.stdout:382: left oid 34 (ObjNum 178 snap 32 seq_num 178) 2026-02-17T05:48:29.907 INFO:tasks.rados.rados.0.trial115.stdout:383: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:48:30.017 INFO:tasks.rados.rados.0.trial115.stdout:382: done (2 left) 2026-02-17T05:48:30.017 INFO:tasks.rados.rados.0.trial115.stdout:383: done (1 left) 2026-02-17T05:48:30.017 INFO:tasks.rados.rados.0.trial115.stdout:385: done (0 left) 2026-02-17T05:48:30.018 INFO:tasks.rados.rados.0.trial115.stdout:386: read oid 30 snap -1 2026-02-17T05:48:30.018 INFO:tasks.rados.rados.0.trial115.stdout:386: expect (ObjNum 170 snap 30 seq_num 170) 2026-02-17T05:48:30.018 INFO:tasks.rados.rados.0.trial115.stdout:387: write (excl) oid 24 current snap is 32 2026-02-17T05:48:30.018 INFO:tasks.rados.rados.0.trial115.stdout:387: seq_num 181 ranges {642067=696188,2025476=459397,2971091=70976} 2026-02-17T05:48:30.022 INFO:tasks.rados.rados.0.trial115.stdout:387: writing trial11545420-24 from 642067 to 1338255 tid 1 2026-02-17T05:48:30.026 INFO:tasks.rados.rados.0.trial115.stdout:387: writing trial11545420-24 from 2025476 to 2484873 tid 2 2026-02-17T05:48:30.028 INFO:tasks.rados.rados.0.trial115.stdout:387: writing trial11545420-24 from 2971091 to 3042067 tid 3 2026-02-17T05:48:30.029 INFO:tasks.rados.rados.0.trial115.stdout:388: read oid 29 snap -1 2026-02-17T05:48:30.029 INFO:tasks.rados.rados.0.trial115.stdout:388: expect (ObjNum 163 snap 29 seq_num 163) 2026-02-17T05:48:30.029 INFO:tasks.rados.rados.0.trial115.stdout:389: write (excl) oid 34 current snap is 32 2026-02-17T05:48:30.029 INFO:tasks.rados.rados.0.trial115.stdout:389: seq_num 182 ranges {658874=771437,2083958=603055} 2026-02-17T05:48:30.034 INFO:tasks.rados.rados.0.trial115.stdout:389: writing trial11545420-34 from 658874 to 1430311 tid 1 2026-02-17T05:48:30.038 INFO:tasks.rados.rados.0.trial115.stdout:389: writing trial11545420-34 from 2083958 to 2687013 tid 2 2026-02-17T05:48:30.039 INFO:tasks.rados.rados.0.trial115.stdout:390: write (excl) oid 47 current snap is 32 2026-02-17T05:48:30.039 INFO:tasks.rados.rados.0.trial115.stdout:390: seq_num 183 ranges {675681=563039,1858793=746713,3025937=49744} 2026-02-17T05:48:30.043 INFO:tasks.rados.rados.0.trial115.stdout:390: writing trial11545420-47 from 675681 to 1238720 tid 1 2026-02-17T05:48:30.048 INFO:tasks.rados.rados.0.trial115.stdout:390: writing trial11545420-47 from 1858793 to 2605506 tid 2 2026-02-17T05:48:30.050 INFO:tasks.rados.rados.0.trial115.stdout:390: writing trial11545420-47 from 3025937 to 3075681 tid 3 2026-02-17T05:48:30.051 INFO:tasks.rados.rados.0.trial115.stdout:391: snap_create 2026-02-17T05:48:30.051 INFO:tasks.rados.rados.0.trial115.stdout:386: expect (ObjNum 170 snap 30 seq_num 170) 2026-02-17T05:48:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): tick 2026-02-17T05:48:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:48:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:48:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 msgr2=0x7f0c24005c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.78:3300/0 conn(0x7f0c24003820 0x7f0c24005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 >> v1:10.20.193.78:6789/0 conn(0x7f0c24006370 legacy=0x7f0c24008830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 1 -- 10.20.193.78:0/2743947802 reap_dead start 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c24009000 0x7f0c24005c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f0c24009000 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0c24009480 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009970 0x7f0c2400d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f0c24009970 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2400b130 con 0x7f0c24009480 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0c4006d6f0 con 0x7f0c24009000 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 1 -- 10.20.193.78:0/2743947802 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2400eb10 con 0x7f0c24009970 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c2f7fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009970 0x7f0c2400d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient(hunting): get_auth_request con 0x7f0c24009970 auth_method 0 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c45804640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c24009000 0x7f0c24005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c45804640 10 monclient(hunting): get_auth_request con 0x7f0c24009000 auth_method 0 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c45804640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c45804640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c45804640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient(hunting): handle_auth_done global_id 4520 payload 274 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 1 -- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c24009000 msgr2=0x7f0c24005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c24009000 0x7f0c24005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 1 -- 10.20.193.78:0/2743947802 >> v1:10.20.193.115:6789/0 conn(0x7f0c24009480 legacy=0x7f0c24008830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient: _finish_hunting 0 2026-02-17T05:48:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 1 monclient: found mon.noname-b 2026-02-17T05:48:30.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:30.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.126+0000 7f0c46005640 1 -- 10.20.193.78:0/2743947802 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4009cc80 con 0x7f0c24009970 2026-02-17T05:48:30.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c46005640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009970 msgr2=0x7f0c2400d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c46005640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009970 0x7f0c2400d570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c40074e00 tx=0x7f0c3c02f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fb100747c678a66 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 1 -- 10.20.193.78:0/2743947802 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c037630 con 0x7f0c24009970 2026-02-17T05:48:30.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: dump: 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.141 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: _finish_auth 0 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: _check_auth_tickets 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 1 -- 10.20.193.78:0/2743947802 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3c0377d0 con 0x7f0c24009970 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 1 -- 10.20.193.78:0/2743947802 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c037b80 con 0x7f0c24009970 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 5 monclient: authenticate success, global_id 4520 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 4 set_mon_vals no callback set 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c44802640 10 monclient: dump: 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.143 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 10 monclient: shutdown 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 1 -- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009970 msgr2=0x7f0c2400d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009970 0x7f0c2400d570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c40074e00 tx=0x7f0c3c02f520 comp rx=0 tx=0).stop 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 1 -- 10.20.193.78:0/2743947802 shutdown_connections 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/2743947802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009970 0x7f0c2400d570 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/2743947802 >> v2:10.20.193.115:3300/0 conn(0x7f0c24009000 0x7f0c24005c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 1 -- 10.20.193.78:0/2743947802 >> 10.20.193.78:0/2743947802 conn(0x7f0c400a6620 msgr2=0x7f0c400a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 1 -- 10.20.193.78:0/2743947802 shutdown_connections 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.129+0000 7f0c47a8f640 1 -- 10.20.193.78:0/2743947802 wait complete. 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.130+0000 7f0c47a8f640 10 monclient: build_initial_monmap 2026-02-17T05:48:30.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.130+0000 7f0c47a8f640 10 monclient: monmap: 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.130+0000 7f0c47a8f640 1 Processor -- start 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.130+0000 7f0c47a8f640 1 -- start start 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.130+0000 7f0c47a8f640 10 monclient: init 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0c24009000 0x7f0c401cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient(hunting): picked mon.noname-a con 0x7f0c24009000 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009480 0x7f0c401cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient(hunting): picked mon.noname-c con 0x7f0c24009480 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0c24009970 0x7f0c4009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient(hunting): picked mon.noname-b con 0x7f0c24009970 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400a01a0 con 0x7f0c24009000 2026-02-17T05:48:30.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400a0020 con 0x7f0c24009970 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4009be20 con 0x7f0c24009480 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c47a8f640 10 monclient(hunting): authenticate will time out at 755.685248s 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45804640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0c24009000 msgr2=0x7f0c401cf350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45804640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0c24009000 0x7f0c401cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009480 0x7f0c401cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009480 0x7f0c401cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38410/0 (socket says 10.20.193.78:38410) 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 -- 10.20.193.78:0/3757553657 learned_addr learned my addr 10.20.193.78:0/3757553657 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:30.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient(hunting): get_auth_request con 0x7f0c24009480 auth_method 0 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c46005640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0c24009970 0x7f0c4009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c46005640 10 monclient(hunting): get_auth_request con 0x7f0c24009970 auth_method 0 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c46005640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c46005640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c46005640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient(hunting): handle_auth_done global_id 4523 payload 824 2026-02-17T05:48:30.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 -- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0c24009970 msgr2=0x7f0c4009db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0c24009970 0x7f0c4009db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 -- 10.20.193.78:0/3757553657 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0c24009000 msgr2=0x7f0c401cf350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0c24009000 0x7f0c401cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient: _finish_hunting 0 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 monclient: found mon.noname-c 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.131+0000 7f0c45003640 1 -- 10.20.193.78:0/3757553657 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c401263c0 con 0x7f0c24009480 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c45003640 1 -- 10.20.193.78:0/3757553657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009480 msgr2=0x7f0c401cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c45003640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009480 0x7f0c401cf890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400c700 tx=0x7f0c3400cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 1 -- 10.20.193.78:0/3757553657 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3401a030 con 0x7f0c24009480 2026-02-17T05:48:30.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: dump: 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c47a8f640 5 monclient: authenticate success, global_id 4523 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 1 -- 10.20.193.78:0/3757553657 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c340138c0 con 0x7f0c24009480 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 1 -- 10.20.193.78:0/3757553657 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400f040 con 0x7f0c24009480 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c47a8f640 10 monclient: _renew_subs 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2ffff640 4 set_mon_vals no callback set 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c47a8f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c40133f00 con 0x7f0c24009480 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c2e7fc640 10 monclient: dump: 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c47a8f640 10 monclient: _renew_subs 2026-02-17T05:48:30.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c47a8f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.133+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4009f7f0 con 0x7f0c24009480 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c2e7fc640 1 -- 10.20.193.78:0/3757553657 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0c34013cb0 con 0x7f0c24009480 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c47a8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c47a8f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c47a8f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c40074980 con 0x7f0c24009480 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c2e7fc640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0c1003de60 0x7f0c10040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c2e7fc640 1 -- 10.20.193.78:0/3757553657 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 4795+0+0 (secure 0 0 0) 0x7f0c34052e30 con 0x7f0c24009480 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c45804640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0c1003de60 0x7f0c10040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.134+0000 7f0c45804640 10 monclient: get_auth_request con 0x7f0c1003de60 auth_method 0 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.137+0000 7f0c45804640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0c1003de60 0x7f0c10040320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c30006fd0 tx=0x7f0c30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.137+0000 7f0c2e7fc640 1 -- 10.20.193.78:0/3757553657 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0c34051820 con 0x7f0c24009480 2026-02-17T05:48:30.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.137+0000 7f0c2e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:30.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.137+0000 7f0c2e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: finishing write tid 1 to trial11545420-24 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: finishing write tid 2 to trial11545420-24 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: finishing write tid 3 to trial11545420-24 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: finishing write tid 4 to trial11545420-24 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: finishing write tid 5 to trial11545420-24 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: oid 24 updating version 0 to 191 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: oid 24 updating version 191 to 193 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: oid 24 version 193 is already newer than 192 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: oid 24 updating version 193 to 194 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 194 (ObjNum 181 snap 32 seq_num 181) dirty exists 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:387: left oid 24 (ObjNum 181 snap 32 seq_num 181) 2026-02-17T05:48:30.156 INFO:tasks.rados.rados.0.trial115.stdout:388: expect (ObjNum 163 snap 29 seq_num 163) 2026-02-17T05:48:30.244 INFO:tasks.rados.rados.0.trial115.stdout:389: finishing write tid 1 to trial11545420-34 2026-02-17T05:48:30.244 INFO:tasks.rados.rados.0.trial115.stdout:389: finishing write tid 2 to trial11545420-34 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:389: finishing write tid 3 to trial11545420-34 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:389: finishing write tid 4 to trial11545420-34 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:389: oid 34 updating version 0 to 42 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:389: oid 34 updating version 42 to 44 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:389: oid 34 version 44 is already newer than 43 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 44 (ObjNum 182 snap 32 seq_num 182) dirty exists 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:389: left oid 34 (ObjNum 182 snap 32 seq_num 182) 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:390: finishing write tid 1 to trial11545420-47 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:390: finishing write tid 2 to trial11545420-47 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:390: finishing write tid 3 to trial11545420-47 2026-02-17T05:48:30.245 INFO:tasks.rados.rados.0.trial115.stdout:390: finishing write tid 4 to trial11545420-47 2026-02-17T05:48:30.246 INFO:tasks.rados.rados.0.trial115.stdout:390: finishing write tid 5 to trial11545420-47 2026-02-17T05:48:30.246 INFO:tasks.rados.rados.0.trial115.stdout:390: oid 47 updating version 0 to 161 2026-02-17T05:48:30.246 INFO:tasks.rados.rados.0.trial115.stdout:390: oid 47 version 161 is already newer than 158 2026-02-17T05:48:30.246 INFO:tasks.rados.rados.0.trial115.stdout:390: oid 47 version 161 is already newer than 159 2026-02-17T05:48:30.246 INFO:tasks.rados.rados.0.trial115.stdout:390: oid 47 version 161 is already newer than 160 2026-02-17T05:48:30.246 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 161 (ObjNum 183 snap 32 seq_num 183) dirty exists 2026-02-17T05:48:30.246 INFO:tasks.rados.rados.0.trial115.stdout:390: left oid 47 (ObjNum 183 snap 32 seq_num 183) 2026-02-17T05:48:30.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.252+0000 7f0c47a8f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:48:30.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.252+0000 7f0c47a8f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:30.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.252+0000 7f0c47a8f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.252+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0c40124f10 con 0x7f0c24009480 2026-02-17T05:48:30.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.252+0000 7f0c2e7fc640 1 -- 10.20.193.78:0/3757553657 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f0c34026d30 con 0x7f0c24009480 2026-02-17T05:48:30.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.252+0000 7f0c2e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:30.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.252+0000 7f0c2e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:30.254 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":26,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:48:30.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0c1003de60 msgr2=0x7f0c10040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0c1003de60 0x7f0c10040320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c30006fd0 tx=0x7f0c30008040 comp rx=0 tx=0).stop 2026-02-17T05:48:30.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 10 monclient: shutdown 2026-02-17T05:48:30.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:30.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009480 msgr2=0x7f0c401cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009480 0x7f0c401cf890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400c700 tx=0x7f0c3400cbd0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 shutdown_connections 2026-02-17T05:48:30.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0c1003de60 0x7f0c10040320 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0c24009970 0x7f0c4009db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0c24009480 0x7f0c401cf890 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 --2- 10.20.193.78:0/3757553657 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0c24009000 0x7f0c401cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 >> 10.20.193.78:0/3757553657 conn(0x7f0c400a6620 msgr2=0x7f0c40135240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:30.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 shutdown_connections 2026-02-17T05:48:30.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.253+0000 7f0c47a8f640 1 -- 10.20.193.78:0/3757553657 wait complete. 2026-02-17T05:48:30.262 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:48:30.262 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:48:30.263 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:48:30.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.329+0000 7f4654934640 10 monclient: get_monmap_and_config 2026-02-17T05:48:30.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.329+0000 7f4654934640 10 monclient: build_initial_monmap 2026-02-17T05:48:30.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.329+0000 7f4654934640 10 monclient: monmap: 2026-02-17T05:48:30.334 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:30.334 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.334 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:30.331111+0000 2026-02-17T05:48:30.334 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:30.331111+0000 2026-02-17T05:48:30.334 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 1 Processor -- start 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 1 -- start start 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient: init 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:30.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T05:48:30.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f464c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:30.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f464c09d430 0x7f464c13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient(hunting): picked mon.noname-c con 0x7f464c09d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:30.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f464c09bf40 0x7f464c13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient(hunting): picked mon.noname-a con 0x7f464c09bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:30.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f464c073600 con 0x7f464c06c890 2026-02-17T05:48:30.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f464c09ca50 con 0x7f464c09bf40 2026-02-17T05:48:30.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f464c06dab0 con 0x7f464c09d430 2026-02-17T05:48:30.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:30.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4654934640 10 monclient(hunting): authenticate will time out at 755.884722s 2026-02-17T05:48:30.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f46526a9640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f464c06c890 legacy=0x7f464c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:48:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4652eaa640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f464c09bf40 msgr2=0x7f464c13fb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4652eaa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f464c09bf40 0x7f464c13fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4651ea8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f464c09d430 0x7f464c13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4651ea8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f464c09d430 0x7f464c13d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:51770/0 (socket says 10.20.193.78:51770) 2026-02-17T05:48:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4651ea8640 1 -- 10.20.193.78:0/3317273928 learned_addr learned my addr 10.20.193.78:0/3317273928 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:30.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4651ea8640 10 monclient(hunting): get_auth_request con 0x7f464c09d430 auth_method 0 2026-02-17T05:48:30.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4651ea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4651ea8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.330+0000 7f4651ea8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 10 monclient(hunting): handle_auth_done global_id 44152 payload 274 2026-02-17T05:48:30.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 1 -- 10.20.193.78:0/3317273928 >> v2:10.20.193.78:3300/0 conn(0x7f464c09bf40 msgr2=0x7f464c13fb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:30.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 1 --2- 10.20.193.78:0/3317273928 >> v2:10.20.193.78:3300/0 conn(0x7f464c09bf40 0x7f464c13fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 1 -- 10.20.193.78:0/3317273928 >> v1:10.20.193.78:6789/0 conn(0x7f464c06c890 legacy=0x7f464c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:30.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 10 monclient: _finish_hunting 0 2026-02-17T05:48:30.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 1 monclient: found mon.noname-c 2026-02-17T05:48:30.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:30.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:48:30.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.331+0000 7f4651ea8640 1 -- 10.20.193.78:0/3317273928 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c09c770 con 0x7f464c09d430 2026-02-17T05:48:30.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.333+0000 7f4651ea8640 1 -- 10.20.193.78:0/3317273928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 msgr2=0x7f464c13d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.333+0000 7f4651ea8640 1 --2- 10.20.193.78:0/3317273928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 0x7f464c13d5d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f464c0a0ac0 tx=0x7f463802ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6660a8e30e9651a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 1 -- 10.20.193.78:0/3317273928 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463803c070 con 0x7f464c09d430 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: dump: 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: _finish_auth 0 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: _check_auth_tickets 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 5 monclient: authenticate success, global_id 44152 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 1 -- 10.20.193.78:0/3317273928 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f463802fda0 con 0x7f464c09d430 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 1 -- 10.20.193.78:0/3317273928 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638037660 con 0x7f464c09d430 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f46516a7640 10 monclient: dump: 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.349 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 4 set_mon_vals no callback set 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 10 monclient: shutdown 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 1 -- 10.20.193.78:0/3317273928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 msgr2=0x7f464c13d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 1 --2- 10.20.193.78:0/3317273928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 0x7f464c13d5d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f464c0a0ac0 tx=0x7f463802ee80 comp rx=0 tx=0).stop 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 1 -- 10.20.193.78:0/3317273928 shutdown_connections 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 1 --2- 10.20.193.78:0/3317273928 >> v2:10.20.193.78:3300/0 conn(0x7f464c09bf40 0x7f464c13fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 1 --2- 10.20.193.78:0/3317273928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 0x7f464c13d5d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 1 -- 10.20.193.78:0/3317273928 >> 10.20.193.78:0/3317273928 conn(0x7f464c0a6620 msgr2=0x7f464c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 1 -- 10.20.193.78:0/3317273928 shutdown_connections 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 1 -- 10.20.193.78:0/3317273928 wait complete. 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 10 monclient: build_initial_monmap 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.334+0000 7f4654934640 10 monclient: monmap: 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 1 Processor -- start 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 1 -- start start 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient: init 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f464c06c890 0x7f464c1cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient(hunting): picked mon.noname-a con 0x7f464c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c09bf40 0x7f464c13eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient(hunting): picked mon.noname-c con 0x7f464c09bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:30.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 0x7f464c13efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient(hunting): picked mon.noname-b con 0x7f464c09d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f464c06d380 con 0x7f464c06c890 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f464c073410 con 0x7f464c09d430 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f464c06d500 con 0x7f464c09bf40 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4654934640 10 monclient(hunting): authenticate will time out at 755.889415s 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f46526a9640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f464c06c890 msgr2=0x7f464c1cf3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f46526a9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f464c06c890 0x7f464c1cf3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c09bf40 0x7f464c13eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c09bf40 0x7f464c13eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38412/0 (socket says 10.20.193.78:38412) 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 -- 10.20.193.78:0/1330516948 learned_addr learned my addr 10.20.193.78:0/1330516948 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient(hunting): get_auth_request con 0x7f464c09bf40 auth_method 0 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4652eaa640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 0x7f464c13efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4652eaa640 10 monclient(hunting): get_auth_request con 0x7f464c09d430 auth_method 0 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4652eaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4652eaa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4652eaa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4652eaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4652eaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4652eaa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient(hunting): handle_auth_done global_id 4526 payload 824 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 msgr2=0x7f464c13efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f464c09d430 0x7f464c13efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f464c06c890 msgr2=0x7f464c1cf3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f464c06c890 0x7f464c1cf3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient: _finish_hunting 0 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 monclient: found mon.noname-c 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:30.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.335+0000 7f4651ea8640 1 -- 10.20.193.78:0/1330516948 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c0a2320 con 0x7f464c09bf40 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.337+0000 7f4651ea8640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c09bf40 msgr2=0x7f464c13eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.337+0000 7f4651ea8640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c09bf40 0x7f464c13eaa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4638009a70 tx=0x7f46380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638046020 con 0x7f464c09bf40 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: dump: 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: _finish_auth 0 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638042620 con 0x7f464c09bf40 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 5 monclient: authenticate success, global_id 4526 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 10 monclient: _renew_subs 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c13af90 con 0x7f464c09bf40 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463803c040 con 0x7f464c09bf40 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4650ea6640 4 set_mon_vals no callback set 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 10 monclient: dump: 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.352 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 10 monclient: _renew_subs 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c13b550 con 0x7f464c09bf40 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c1345a0 0x7f464c134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f464c139bf0 con 0x7f464c1345a0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c1345a0 0x7f464c134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 10 monclient: get_auth_request con 0x7f464c1345a0 auth_method 0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f463803f030 con 0x7f464c09bf40 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 10 monclient(hunting): handle_auth_done global_id 4529 payload 824 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f46526a9640 1 -- 10.20.193.78:0/1330516948 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f464c09d810 con 0x7f464c1345a0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f461c03de60 0x7f461c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.338+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 4795+0+0 (secure 0 0 0) 0x7f46380779e0 con 0x7f464c09bf40 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.339+0000 7f4652eaa640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f461c03de60 0x7f461c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.339+0000 7f4652eaa640 10 monclient: get_auth_request con 0x7f461c03de60 auth_method 0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.340+0000 7f46526a9640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c1345a0 msgr2=0x7f464c134980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.340+0000 7f46526a9640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c1345a0 0x7f464c134980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f464400bcc0 tx=0x7f464400bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644005070 con 0x7f464c1345a0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 10 monclient: dump: 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.353 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4644005070 con 0x7f464c1345a0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c1345a0 msgr2=0x7f464c134980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c1345a0 0x7f464c134980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f464400bcc0 tx=0x7f464400bcf0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c1345a0 msgr2=0x7f464c134980 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4636ffd640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c1345a0 0x7f464c134980 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f464400bcc0 tx=0x7f464400bcf0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.341+0000 7f4652eaa640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f461c03de60 0x7f461c040320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4640003db0 tx=0x7f464000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f4654934640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f4654934640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c134ec0 0x7f464c1da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f4654934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f464c13fa00 con 0x7f464c134ec0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c134ec0 0x7f464c1da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 10 monclient: get_auth_request con 0x7f464c134ec0 auth_method 0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 10 monclient(hunting): handle_auth_done global_id 4532 payload 824 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.344+0000 7f46526a9640 1 -- 10.20.193.78:0/1330516948 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f464c06cc70 con 0x7f464c134ec0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f46526a9640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c134ec0 msgr2=0x7f464c1da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f46526a9640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c134ec0 0x7f464c1da4d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4644014000 tx=0x7f4644005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644005980 con 0x7f464c134ec0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 10 monclient: dump: 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.354 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7f464401a940 con 0x7f464c134ec0 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c134ec0 msgr2=0x7f464c1da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c134ec0 0x7f464c1da4d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4644014000 tx=0x7f4644005b20 comp rx=0 tx=0).stop 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c134ec0 msgr2=0x7f464c1da4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.346+0000 7f4636ffd640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c134ec0 0x7f464c1da4d0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4644014000 tx=0x7f4644005b20 comp rx=0 tx=0).stop 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 26, 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 160151, 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:30.355 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:30.356 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:30.357 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:30.358 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:48:30.359 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f461c03de60 msgr2=0x7f461c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f461c03de60 0x7f461c040320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4640003db0 tx=0x7f464000d040 comp rx=0 tx=0).stop 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 10 monclient: shutdown 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c09bf40 msgr2=0x7f464c13eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c09bf40 0x7f464c13eaa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4638009a70 tx=0x7f46380047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4652eaa640 1 -- 10.20.193.78:0/1330516948 reap_dead start 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 shutdown_connections 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 --2- 10.20.193.78:0/1330516948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f464c09bf40 0x7f464c13eaa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 >> 10.20.193.78:0/1330516948 conn(0x7f464c0a6620 msgr2=0x7f464c0a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 shutdown_connections 2026-02-17T05:48:30.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.347+0000 7f4654934640 1 -- 10.20.193.78:0/1330516948 wait complete. 2026-02-17T05:48:30.360 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:48:30.360 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:48:30.361 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:48:30.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.427+0000 7f77458c7640 10 monclient: get_monmap_and_config 2026-02-17T05:48:30.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.427+0000 7f77458c7640 10 monclient: build_initial_monmap 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.427+0000 7f77458c7640 10 monclient: monmap: 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:30.429146+0000 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:30.429146+0000 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:30.430 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:30.431 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 1 Processor -- start 2026-02-17T05:48:30.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 1 -- start start 2026-02-17T05:48:30.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient: init 2026-02-17T05:48:30.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:30.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2026-02-17T05:48:30.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f774006c890 0x7f77400a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient(hunting): picked mon.noname-c con 0x7f774006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:30.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f774006cc70 0x7f774006d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient(hunting): picked mon.noname-a con 0x7f774006cc70 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7740134df0 0x7f7740135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient(hunting): picked mon.noname-b con 0x7f7740134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7740074150 con 0x7f774006cc70 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f774009ef70 con 0x7f774006c890 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f774009f250 con 0x7f7740134df0 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f77458c7640 10 monclient(hunting): authenticate will time out at 755.982781s 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773e7fc640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f774006cc70 msgr2=0x7f774006d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773e7fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f774006cc70 0x7f774006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7740134df0 0x7f7740135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7740134df0 0x7f7740135220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38428/0 (socket says 10.20.193.78:38428) 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 1 -- 10.20.193.78:0/728511472 learned_addr learned my addr 10.20.193.78:0/728511472 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 10 monclient(hunting): get_auth_request con 0x7f7740134df0 auth_method 0 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773effd640 1 --2- 10.20.193.78:0/728511472 >> v2:10.20.193.115:3300/0 conn(0x7f774006c890 0x7f77400a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773effd640 10 monclient(hunting): get_auth_request con 0x7f774006c890 auth_method 0 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.428+0000 7f773f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 10 monclient(hunting): handle_auth_done global_id 4535 payload 274 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 1 -- 10.20.193.78:0/728511472 >> v2:10.20.193.115:3300/0 conn(0x7f774006c890 msgr2=0x7f77400a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 1 --2- 10.20.193.78:0/728511472 >> v2:10.20.193.115:3300/0 conn(0x7f774006c890 0x7f77400a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 1 -- 10.20.193.78:0/728511472 >> v2:10.20.193.78:3300/0 conn(0x7f774006cc70 msgr2=0x7f774006d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 1 --2- 10.20.193.78:0/728511472 >> v2:10.20.193.78:3300/0 conn(0x7f774006cc70 0x7f774006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 1 monclient: found mon.noname-b 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.429+0000 7f773f7fe640 1 -- 10.20.193.78:0/728511472 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774009df40 con 0x7f7740134df0 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773f7fe640 1 -- 10.20.193.78:0/728511472 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7740134df0 msgr2=0x7f7740135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773f7fe640 1 --2- 10.20.193.78:0/728511472 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7740134df0 0x7f7740135220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f774006a480 tx=0x7f773002f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e4114e00c99734 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 1 -- 10.20.193.78:0/728511472 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300373f0 con 0x7f7740134df0 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: dump: 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: _finish_auth 0 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f77458c7640 5 monclient: authenticate success, global_id 4535 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 1 -- 10.20.193.78:0/728511472 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7730037590 con 0x7f7740134df0 2026-02-17T05:48:30.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f77458c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 1 -- 10.20.193.78:0/728511472 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730037960 con 0x7f7740134df0 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f77458c7640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f77458c7640 4 set_mon_vals no callback set 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.431+0000 7f773dffb640 10 monclient: dump: 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 10 monclient: shutdown 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 -- 10.20.193.78:0/728511472 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7740134df0 msgr2=0x7f7740135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 --2- 10.20.193.78:0/728511472 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7740134df0 0x7f7740135220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f774006a480 tx=0x7f773002f330 comp rx=0 tx=0).stop 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 -- 10.20.193.78:0/728511472 shutdown_connections 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 --2- 10.20.193.78:0/728511472 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7740134df0 0x7f7740135220 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 --2- 10.20.193.78:0/728511472 >> v2:10.20.193.78:3300/0 conn(0x7f774006cc70 0x7f774006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 --2- 10.20.193.78:0/728511472 >> v2:10.20.193.115:3300/0 conn(0x7f774006c890 0x7f77400a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 -- 10.20.193.78:0/728511472 >> 10.20.193.78:0/728511472 conn(0x7f77400a6620 msgr2=0x7f77400a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 -- 10.20.193.78:0/728511472 shutdown_connections 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 -- 10.20.193.78:0/728511472 wait complete. 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 10 monclient: build_initial_monmap 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 10 monclient: monmap: 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 Processor -- start 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 1 -- start start 2026-02-17T05:48:30.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.432+0000 7f77458c7640 10 monclient: init 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774006c890 0x7f7740123650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient(hunting): picked mon.noname-a con 0x7f774006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774006cc70 0x7f7740123b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient(hunting): picked mon.noname-c con 0x7f774006cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7740134df0 0x7f77401240d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient(hunting): picked mon.noname-b con 0x7f7740134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f774009d360 con 0x7f774006c890 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f774009d1e0 con 0x7f7740134df0 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7740135760 con 0x7f774006cc70 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f77458c7640 10 monclient(hunting): authenticate will time out at 755.987214s 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773effd640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774006c890 msgr2=0x7f7740123650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774006c890 0x7f7740123650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774006cc70 0x7f7740123b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774006cc70 0x7f7740123b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38440/0 (socket says 10.20.193.78:38440) 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 -- 10.20.193.78:0/3651728812 learned_addr learned my addr 10.20.193.78:0/3651728812 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:30.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient(hunting): get_auth_request con 0x7f774006cc70 auth_method 0 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient(hunting): handle_auth_done global_id 4538 payload 824 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7740134df0 msgr2=0x7f77401240d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7740134df0 0x7f77401240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774006c890 msgr2=0x7f7740123650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774006c890 0x7f7740123650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 monclient: found mon.noname-c 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.433+0000 7f773e7fc640 1 -- 10.20.193.78:0/3651728812 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7740126f00 con 0x7f774006cc70 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f773e7fc640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774006cc70 msgr2=0x7f7740123b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f773e7fc640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774006cc70 0x7f7740123b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f772800b7b0 tx=0x7f772800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728005270 con 0x7f774006cc70 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: dump: 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.445 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: _finish_auth 0 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: _check_auth_tickets 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f77458c7640 5 monclient: authenticate success, global_id 4538 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7728010070 con 0x7f774006cc70 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728005670 con 0x7f774006cc70 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f77458c7640 10 monclient: _renew_subs 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f773d7fa640 4 set_mon_vals no callback set 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f77458c7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7740126240 con 0x7f774006cc70 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f7717fff640 10 monclient: dump: 2026-02-17T05:48:30.446 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f77458c7640 10 monclient: _renew_subs 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f77458c7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.435+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7740072ce0 con 0x7f774006cc70 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f77458c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f77458c7640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401ccfe0 0x7f77401cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f77458c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77401246d0 con 0x7f77401ccfe0 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401ccfe0 0x7f77401cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 10 monclient: get_auth_request con 0x7f77401ccfe0 auth_method 0 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f772800c520 con 0x7f774006cc70 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 10 monclient(hunting): handle_auth_done global_id 44161 payload 824 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773effd640 1 -- 10.20.193.78:0/3651728812 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7740074980 con 0x7f77401ccfe0 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f7717fff640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771003deb0 0x7f7710040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 4795+0+0 (secure 0 0 0) 0x7f7728052190 con 0x7f774006cc70 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773f7fe640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771003deb0 0x7f7710040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.436+0000 7f773f7fe640 10 monclient: get_auth_request con 0x7f771003deb0 auth_method 0 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f773effd640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401ccfe0 msgr2=0x7f77401cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f773effd640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401ccfe0 0x7f77401cd3c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f773400e8b0 tx=0x7f773400ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734017070 con 0x7f77401ccfe0 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 10 monclient: dump: 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.448 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f773f7fe640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771003deb0 0x7f7710040370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7740074e00 tx=0x7f7730031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f773400cbc0 con 0x7f77401ccfe0 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401ccfe0 msgr2=0x7f77401cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401ccfe0 0x7f77401cd3c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f773400e8b0 tx=0x7f773400ed80 comp rx=0 tx=0).stop 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401ccfe0 msgr2=0x7f77401cd3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.439+0000 7f7717fff640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401ccfe0 0x7f77401cd3c0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f773400e8b0 tx=0x7f773400ed80 comp rx=0 tx=0).stop 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f77458c7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f77458c7640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401cd900 0x7f77401d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f77458c7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f774006d0e0 con 0x7f77401cd900 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401cd900 0x7f77401d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 10 monclient: get_auth_request con 0x7f77401cd900 auth_method 0 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:30.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:30.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:30.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:30.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:30.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 10 monclient(hunting): handle_auth_done global_id 44164 payload 824 2026-02-17T05:48:30.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.442+0000 7f773effd640 1 -- 10.20.193.78:0/3651728812 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7740135a20 con 0x7f77401cd900 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f773effd640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401cd900 msgr2=0x7f77401d82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f773effd640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401cd900 0x7f77401d82c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7734018000 tx=0x7f7734004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734017070 con 0x7f77401cd900 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 10 monclient: dump: 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7f7734004610 con 0x7f77401cd900 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:48:30.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401cd900 msgr2=0x7f77401d82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401cd900 0x7f77401d82c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7734018000 tx=0x7f7734004830 comp rx=0 tx=0).stop 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401cd900 msgr2=0x7f77401d82c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f7717fff640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77401cd900 0x7f77401d82c0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7734018000 tx=0x7f7734004830 comp rx=0 tx=0).stop 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 26, 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 34093, 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.452 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:30.453 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:48:30.454 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:30.455 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.456 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:48:30.457 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:30.458 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:48:30.459 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771003deb0 msgr2=0x7f7710040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771003deb0 0x7f7710040370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7740074e00 tx=0x7f7730031000 comp rx=0 tx=0).stop 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 10 monclient: shutdown 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774006cc70 msgr2=0x7f7740123b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f773f7fe640 1 -- 10.20.193.78:0/3651728812 reap_dead start 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774006cc70 0x7f7740123b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f772800b7b0 tx=0x7f772800bc80 comp rx=0 tx=0).stop 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 shutdown_connections 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 --2- 10.20.193.78:0/3651728812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774006cc70 0x7f7740123b90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 >> 10.20.193.78:0/3651728812 conn(0x7f77400a6620 msgr2=0x7f7740071350 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 shutdown_connections 2026-02-17T05:48:30.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:30.445+0000 7f77458c7640 1 -- 10.20.193.78:0/3651728812 wait complete. 2026-02-17T05:48:30.461 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:48:30.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:48:30.895 INFO:tasks.rados.rados.0.trial115.stdout:386: done (5 left) 2026-02-17T05:48:30.895 INFO:tasks.rados.rados.0.trial115.stdout:387: done (4 left) 2026-02-17T05:48:30.895 INFO:tasks.rados.rados.0.trial115.stdout:388: done (3 left) 2026-02-17T05:48:30.895 INFO:tasks.rados.rados.0.trial115.stdout:389: done (2 left) 2026-02-17T05:48:30.895 INFO:tasks.rados.rados.0.trial115.stdout:390: done (1 left) 2026-02-17T05:48:30.895 INFO:tasks.rados.rados.0.trial115.stdout:391: done (0 left) 2026-02-17T05:48:30.895 INFO:tasks.rados.rados.0.trial115.stdout:392: snap_remove snap 32 2026-02-17T05:48:31.358 INFO:tasks.rados.rados.0.trial115.stdout:392: done (0 left) 2026-02-17T05:48:31.359 INFO:tasks.rados.rados.0.trial115.stdout:393: write (excl) oid 10 current snap is 33 2026-02-17T05:48:31.359 INFO:tasks.rados.rados.0.trial115.stdout:393: seq_num 184 ranges {692488=638288,1917275=606724} 2026-02-17T05:48:31.363 INFO:tasks.rados.rados.0.trial115.stdout:393: writing trial11545420-10 from 692488 to 1330776 tid 1 2026-02-17T05:48:31.368 INFO:tasks.rados.rados.0.trial115.stdout:393: writing trial11545420-10 from 1917275 to 2523999 tid 2 2026-02-17T05:48:31.368 INFO:tasks.rados.rados.0.trial115.stdout:394: write oid 25 current snap is 33 2026-02-17T05:48:31.368 INFO:tasks.rados.rados.0.trial115.stdout:394: seq_num 185 ranges {709295=713537,1859404=750382} 2026-02-17T05:48:31.373 INFO:tasks.rados.rados.0.trial115.stdout:394: writing trial11545420-25 from 709295 to 1422832 tid 1 2026-02-17T05:48:31.378 INFO:tasks.rados.rados.0.trial115.stdout:394: writing trial11545420-25 from 1859404 to 2609786 tid 2 2026-02-17T05:48:31.378 INFO:tasks.rados.rados.0.trial115.stdout:393: finishing write tid 1 to trial11545420-10 2026-02-17T05:48:31.378 INFO:tasks.rados.rados.0.trial115.stdout:395: read oid 8 snap -1 2026-02-17T05:48:31.378 INFO:tasks.rados.rados.0.trial115.stdout:395: expect deleted 2026-02-17T05:48:31.378 INFO:tasks.rados.rados.0.trial115.stdout:396: write oid 33 current snap is 33 2026-02-17T05:48:31.378 INFO:tasks.rados.rados.0.trial115.stdout:393: finishing write tid 2 to trial11545420-10 2026-02-17T05:48:31.378 INFO:tasks.rados.rados.0.trial115.stdout:393: finishing write tid 3 to trial11545420-10 2026-02-17T05:48:31.379 INFO:tasks.rados.rados.0.trial115.stdout:393: finishing write tid 4 to trial11545420-10 2026-02-17T05:48:31.379 INFO:tasks.rados.rados.0.trial115.stdout:393: oid 10 updating version 0 to 167 2026-02-17T05:48:31.379 INFO:tasks.rados.rados.0.trial115.stdout:393: oid 10 version 167 is already newer than 165 2026-02-17T05:48:31.379 INFO:tasks.rados.rados.0.trial115.stdout:393: oid 10 version 167 is already newer than 166 2026-02-17T05:48:31.379 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 167 (ObjNum 184 snap 33 seq_num 184) dirty exists 2026-02-17T05:48:31.379 INFO:tasks.rados.rados.0.trial115.stdout:393: left oid 10 (ObjNum 184 snap 33 seq_num 184) 2026-02-17T05:48:31.379 INFO:tasks.rados.rados.0.trial115.stdout:396: seq_num 186 ranges {726102=788786,1917886=610393,3108206=17896} 2026-02-17T05:48:31.383 INFO:tasks.rados.rados.0.trial115.stdout:396: writing trial11545420-33 from 726102 to 1514888 tid 1 2026-02-17T05:48:31.386 INFO:tasks.rados.rados.0.trial115.stdout:396: writing trial11545420-33 from 1917886 to 2528279 tid 2 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:396: writing trial11545420-33 from 3108206 to 3126102 tid 3 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:393: done (3 left) 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:397: copy_from oid 20 from oid 39 current snap is 33 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:394: finishing write tid 1 to trial11545420-25 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:394: finishing write tid 2 to trial11545420-25 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:394: finishing write tid 3 to trial11545420-25 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:394: finishing write tid 4 to trial11545420-25 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:394: oid 25 updating version 0 to 131 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:394: oid 25 updating version 131 to 132 2026-02-17T05:48:31.390 INFO:tasks.rados.rados.0.trial115.stdout:394: oid 25 updating version 132 to 133 2026-02-17T05:48:31.391 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 133 (ObjNum 185 snap 33 seq_num 185) dirty exists 2026-02-17T05:48:31.391 INFO:tasks.rados.rados.0.trial115.stdout:394: left oid 25 (ObjNum 185 snap 33 seq_num 185) 2026-02-17T05:48:31.391 INFO:tasks.rados.rados.0.trial115.stdout:396: finishing write tid 1 to trial11545420-33 2026-02-17T05:48:31.391 INFO:tasks.rados.rados.0.trial115.stdout:396: finishing write tid 2 to trial11545420-33 2026-02-17T05:48:31.391 INFO:tasks.rados.rados.0.trial115.stdout:394: done (3 left) 2026-02-17T05:48:31.391 INFO:tasks.rados.rados.0.trial115.stdout:395: done (2 left) 2026-02-17T05:48:31.391 INFO:tasks.rados.rados.0.trial115.stdout:398: write (excl) oid 35 current snap is 33 2026-02-17T05:48:31.391 INFO:tasks.rados.rados.0.trial115.stdout:398: seq_num 187 ranges {742909=464035,1976368=754051} 2026-02-17T05:48:31.393 INFO:tasks.rados.rados.0.trial115.stdout:398: writing trial11545420-35 from 742909 to 1206944 tid 1 2026-02-17T05:48:31.399 INFO:tasks.rados.rados.0.trial115.stdout:398: writing trial11545420-35 from 1976368 to 2730419 tid 2 2026-02-17T05:48:31.400 INFO:tasks.rados.rados.0.trial115.stdout:399: read oid 4 snap -1 2026-02-17T05:48:31.400 INFO:tasks.rados.rados.0.trial115.stdout:399: expect deleted 2026-02-17T05:48:31.400 INFO:tasks.rados.rados.0.trial115.stdout:400: write oid 6 current snap is 33 2026-02-17T05:48:31.400 INFO:tasks.rados.rados.0.trial115.stdout:400: seq_num 188 ranges {759716=539284,2034850=614062} 2026-02-17T05:48:31.404 INFO:tasks.rados.rados.0.trial115.stdout:400: writing trial11545420-6 from 759716 to 1299000 tid 1 2026-02-17T05:48:31.409 INFO:tasks.rados.rados.0.trial115.stdout:400: writing trial11545420-6 from 2034850 to 2648912 tid 2 2026-02-17T05:48:31.410 INFO:tasks.rados.rados.0.trial115.stdout:401: read oid 2 snap 31 2026-02-17T05:48:31.410 INFO:tasks.rados.rados.0.trial115.stdout:401: expect (ObjNum 130 snap 22 seq_num 130) 2026-02-17T05:48:31.410 INFO:tasks.rados.rados.0.trial115.stdout:402: read oid 50 snap -1 2026-02-17T05:48:31.410 INFO:tasks.rados.rados.0.trial115.stdout:402: expect (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:48:31.410 INFO:tasks.rados.rados.0.trial115.stdout:403: read oid 46 snap -1 2026-02-17T05:48:31.410 INFO:tasks.rados.rados.0.trial115.stdout:403: expect deleted 2026-02-17T05:48:31.410 INFO:tasks.rados.rados.0.trial115.stdout:404: write (excl) oid 5 current snap is 33 2026-02-17T05:48:31.410 INFO:tasks.rados.rados.0.trial115.stdout:404: seq_num 189 ranges {776523=614533,1976979=757720} 2026-02-17T05:48:31.414 INFO:tasks.rados.rados.0.trial115.stdout:404: writing trial11545420-5 from 776523 to 1391056 tid 1 2026-02-17T05:48:31.420 INFO:tasks.rados.rados.0.trial115.stdout:404: writing trial11545420-5 from 1976979 to 2734699 tid 2 2026-02-17T05:48:31.420 INFO:tasks.rados.rados.0.trial115.stdout:405: read oid 11 snap -1 2026-02-17T05:48:31.420 INFO:tasks.rados.rados.0.trial115.stdout:405: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:48:31.420 INFO:tasks.rados.rados.0.trial115.stdout:406: write oid 30 current snap is 33 2026-02-17T05:48:31.420 INFO:tasks.rados.rados.0.trial115.stdout:396: finishing write tid 3 to trial11545420-33 2026-02-17T05:48:31.420 INFO:tasks.rados.rados.0.trial115.stdout:406: seq_num 190 ranges {793330=689782,2035461=617731,3101545=91785} 2026-02-17T05:48:31.425 INFO:tasks.rados.rados.0.trial115.stdout:406: writing trial11545420-30 from 793330 to 1483112 tid 1 2026-02-17T05:48:31.429 INFO:tasks.rados.rados.0.trial115.stdout:406: writing trial11545420-30 from 2035461 to 2653192 tid 2 2026-02-17T05:48:31.431 INFO:tasks.rados.rados.0.trial115.stdout:406: writing trial11545420-30 from 3101545 to 3193330 tid 3 2026-02-17T05:48:31.431 INFO:tasks.rados.rados.0.trial115.stdout:407: snap_create 2026-02-17T05:48:31.431 INFO:tasks.rados.rados.0.trial115.stdout:396: finishing write tid 4 to trial11545420-33 2026-02-17T05:48:31.431 INFO:tasks.rados.rados.0.trial115.stdout:396: finishing write tid 5 to trial11545420-33 2026-02-17T05:48:31.431 INFO:tasks.rados.rados.0.trial115.stdout:396: oid 33 updating version 0 to 83 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:396: oid 33 updating version 83 to 84 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:396: oid 33 version 84 is already newer than 82 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:396: oid 33 updating version 84 to 85 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 85 (ObjNum 186 snap 33 seq_num 186) dirty exists 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:396: left oid 33 (ObjNum 186 snap 33 seq_num 186) 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:398: finishing write tid 1 to trial11545420-35 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:397: finishing copy_from racing read to trial11545420-20 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:397: finishing copy_from to trial11545420-20 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 120 (ObjNum 116 snap 19 seq_num 116) dirty exists 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:398: finishing write tid 2 to trial11545420-35 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:398: finishing write tid 3 to trial11545420-35 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:398: finishing write tid 4 to trial11545420-35 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:398: oid 35 updating version 0 to 155 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:398: oid 35 version 155 is already newer than 153 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:398: oid 35 version 155 is already newer than 154 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 155 (ObjNum 187 snap 33 seq_num 187) dirty exists 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:398: left oid 35 (ObjNum 187 snap 33 seq_num 187) 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:400: finishing write tid 1 to trial11545420-6 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:400: finishing write tid 2 to trial11545420-6 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:400: finishing write tid 3 to trial11545420-6 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:400: finishing write tid 4 to trial11545420-6 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:400: oid 6 updating version 0 to 46 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:400: oid 6 updating version 46 to 47 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:400: oid 6 updating version 47 to 48 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 48 (ObjNum 188 snap 33 seq_num 188) dirty exists 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:400: left oid 6 (ObjNum 188 snap 33 seq_num 188) 2026-02-17T05:48:31.432 INFO:tasks.rados.rados.0.trial115.stdout:401: expect (ObjNum 130 snap 22 seq_num 130) 2026-02-17T05:48:31.496 INFO:tasks.rados.rados.0.trial115.stdout:402: expect (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:48:31.560 INFO:tasks.rados.rados.0.trial115.stdout:404: finishing write tid 1 to trial11545420-5 2026-02-17T05:48:31.560 INFO:tasks.rados.rados.0.trial115.stdout:404: finishing write tid 2 to trial11545420-5 2026-02-17T05:48:31.560 INFO:tasks.rados.rados.0.trial115.stdout:404: finishing write tid 3 to trial11545420-5 2026-02-17T05:48:31.560 INFO:tasks.rados.rados.0.trial115.stdout:404: finishing write tid 4 to trial11545420-5 2026-02-17T05:48:31.560 INFO:tasks.rados.rados.0.trial115.stdout:404: oid 5 updating version 0 to 163 2026-02-17T05:48:31.561 INFO:tasks.rados.rados.0.trial115.stdout:404: oid 5 version 163 is already newer than 162 2026-02-17T05:48:31.561 INFO:tasks.rados.rados.0.trial115.stdout:404: oid 5 updating version 163 to 164 2026-02-17T05:48:31.561 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 164 (ObjNum 189 snap 33 seq_num 189) dirty exists 2026-02-17T05:48:31.561 INFO:tasks.rados.rados.0.trial115.stdout:404: left oid 5 (ObjNum 189 snap 33 seq_num 189) 2026-02-17T05:48:31.561 INFO:tasks.rados.rados.0.trial115.stdout:406: finishing write tid 1 to trial11545420-30 2026-02-17T05:48:31.561 INFO:tasks.rados.rados.0.trial115.stdout:405: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:48:31.623 INFO:tasks.rados.rados.0.trial115.stdout:406: finishing write tid 2 to trial11545420-30 2026-02-17T05:48:31.623 INFO:tasks.rados.rados.0.trial115.stdout:406: finishing write tid 3 to trial11545420-30 2026-02-17T05:48:31.624 INFO:tasks.rados.rados.0.trial115.stdout:406: finishing write tid 4 to trial11545420-30 2026-02-17T05:48:31.624 INFO:tasks.rados.rados.0.trial115.stdout:406: finishing write tid 5 to trial11545420-30 2026-02-17T05:48:31.624 INFO:tasks.rados.rados.0.trial115.stdout:406: oid 30 updating version 0 to 147 2026-02-17T05:48:31.624 INFO:tasks.rados.rados.0.trial115.stdout:406: oid 30 version 147 is already newer than 146 2026-02-17T05:48:31.624 INFO:tasks.rados.rados.0.trial115.stdout:406: oid 30 updating version 147 to 149 2026-02-17T05:48:31.624 INFO:tasks.rados.rados.0.trial115.stdout:406: oid 30 version 149 is already newer than 148 2026-02-17T05:48:31.624 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 149 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:48:31.624 INFO:tasks.rados.rados.0.trial115.stdout:406: left oid 30 (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:48:32.361 INFO:tasks.rados.rados.0.trial115.stdout:396: done (11 left) 2026-02-17T05:48:32.361 INFO:tasks.rados.rados.0.trial115.stdout:397: done (10 left) 2026-02-17T05:48:32.361 INFO:tasks.rados.rados.0.trial115.stdout:398: done (9 left) 2026-02-17T05:48:32.361 INFO:tasks.rados.rados.0.trial115.stdout:399: done (8 left) 2026-02-17T05:48:32.361 INFO:tasks.rados.rados.0.trial115.stdout:400: done (7 left) 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:401: done (6 left) 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:402: done (5 left) 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:403: done (4 left) 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:404: done (3 left) 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:405: done (2 left) 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:406: done (1 left) 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:407: done (0 left) 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:408: rollback oid 17 current snap is 34 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 33 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:409: write (excl) oid 13 current snap is 34 2026-02-17T05:48:32.362 INFO:tasks.rados.rados.0.trial115.stdout:409: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2026-02-17T05:48:32.363 INFO:tasks.rados.rados.0.trial115.stdout:409: writing trial11545420-13 from 410137 to 891521 tid 1 2026-02-17T05:48:32.368 INFO:tasks.rados.rados.0.trial115.stdout:409: writing trial11545420-13 from 1410296 to 2171685 tid 2 2026-02-17T05:48:32.373 INFO:tasks.rados.rados.0.trial115.stdout:409: writing trial11545420-13 from 2728968 to 3210137 tid 3 2026-02-17T05:48:32.373 INFO:tasks.rados.rados.0.trial115.stdout:410: read oid 48 snap 31 2026-02-17T05:48:32.373 INFO:tasks.rados.rados.0.trial115.stdout:410: expect (ObjNum 99 snap 14 seq_num 99) 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:411: copy_from oid 36 from oid 39 current snap is 34 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:408: finishing rollback tid 0 to trial11545420-17 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:408: finishing rollback tid 1 to trial11545420-17 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:408: finishing rollback tid 2 to trial11545420-17 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 40 (ObjNum 135 snap 23 seq_num 135) dirty exists 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:409: finishing write tid 1 to trial11545420-13 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:409: finishing write tid 2 to trial11545420-13 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:408: done (3 left) 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:412: rollback oid 31 current snap is 34 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 31 to 31 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:413: copy_from oid 33 from oid 3 current snap is 34 2026-02-17T05:48:32.374 INFO:tasks.rados.rados.0.trial115.stdout:414: copy_from oid 25 from oid 17 current snap is 34 2026-02-17T05:48:32.375 INFO:tasks.rados.rados.0.trial115.stdout:415: write oid 44 current snap is 34 2026-02-17T05:48:32.375 INFO:tasks.rados.rados.0.trial115.stdout:415: seq_num 192 ranges {426944=556633,1468778=621400,2872744=354200} 2026-02-17T05:48:32.376 INFO:tasks.rados.rados.0.trial115.stdout:415: writing trial11545420-44 from 426944 to 983577 tid 1 2026-02-17T05:48:32.380 INFO:tasks.rados.rados.0.trial115.stdout:415: writing trial11545420-44 from 1468778 to 2090178 tid 2 2026-02-17T05:48:32.384 INFO:tasks.rados.rados.0.trial115.stdout:415: writing trial11545420-44 from 2872744 to 3226944 tid 3 2026-02-17T05:48:32.384 INFO:tasks.rados.rados.0.trial115.stdout:416: snap_remove snap 33 2026-02-17T05:48:33.362 INFO:tasks.rados.rados.0.trial115.stdout:416: done (7 left) 2026-02-17T05:48:33.362 INFO:tasks.rados.rados.0.trial115.stdout:417: copy_from oid 46 from oid 37 current snap is 34 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: finishing write tid 3 to trial11545420-13 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: finishing write tid 4 to trial11545420-13 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: finishing write tid 5 to trial11545420-13 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: oid 13 updating version 0 to 77 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: oid 13 updating version 77 to 80 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: oid 13 version 80 is already newer than 79 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: oid 13 version 80 is already newer than 78 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 80 (ObjNum 191 snap 34 seq_num 191) dirty exists 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: left oid 13 (ObjNum 191 snap 34 seq_num 191) 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:412: finishing rollback tid 0 to trial11545420-31 2026-02-17T05:48:33.363 INFO:tasks.rados.rados.0.trial115.stdout:409: done (7 left) 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:418: read oid 50 snap -1 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:418: expect (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:419: read oid 2 snap -1 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:419: expect (ObjNum 130 snap 22 seq_num 130) 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:420: read oid 11 snap 31 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:420: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:421: rollback oid 20 current snap is 34 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 31 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:412: finishing rollback tid 1 to trial11545420-31 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:412: finishing rollback tid 2 to trial11545420-31 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 168 (ObjNum 138 snap 25 seq_num 138) dirty exists 2026-02-17T05:48:33.364 INFO:tasks.rados.rados.0.trial115.stdout:410: expect (ObjNum 99 snap 14 seq_num 99) 2026-02-17T05:48:33.447 INFO:tasks.rados.rados.0.trial115.stdout:414: finishing copy_from racing read to trial11545420-25 2026-02-17T05:48:33.447 INFO:tasks.rados.rados.0.trial115.stdout:414: finishing copy_from to trial11545420-25 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 40 (ObjNum 135 snap 23 seq_num 135) dirty exists 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:415: finishing write tid 1 to trial11545420-44 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:415: finishing write tid 2 to trial11545420-44 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:413: finishing copy_from racing read to trial11545420-33 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:410: done (10 left) 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:412: done (9 left) 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:414: done (8 left) 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:422: read oid 13 snap -1 2026-02-17T05:48:33.448 INFO:tasks.rados.rados.0.trial115.stdout:422: expect (ObjNum 191 snap 34 seq_num 191) 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:423: rollback oid 42 current snap is 34 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 31 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:411: finishing copy_from racing read to trial11545420-36 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:415: finishing write tid 3 to trial11545420-44 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:411: finishing copy_from to trial11545420-36 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 120 (ObjNum 116 snap 19 seq_num 116) dirty exists 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:415: finishing write tid 4 to trial11545420-44 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:415: finishing write tid 5 to trial11545420-44 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:415: oid 44 updating version 0 to 54 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:415: oid 44 version 54 is already newer than 52 2026-02-17T05:48:33.449 INFO:tasks.rados.rados.0.trial115.stdout:415: oid 44 version 54 is already newer than 53 2026-02-17T05:48:33.450 INFO:tasks.rados.rados.0.trial115.stdout:415: oid 44 version 54 is already newer than 51 2026-02-17T05:48:33.450 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 54 (ObjNum 192 snap 34 seq_num 192) dirty exists 2026-02-17T05:48:33.450 INFO:tasks.rados.rados.0.trial115.stdout:415: left oid 44 (ObjNum 192 snap 34 seq_num 192) 2026-02-17T05:48:33.450 INFO:tasks.rados.rados.0.trial115.stdout:413: finishing copy_from to trial11545420-33 2026-02-17T05:48:33.450 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 149 (ObjNum 173 snap 30 seq_num 173) dirty exists 2026-02-17T05:48:33.450 INFO:tasks.rados.rados.0.trial115.stdout:419: expect (ObjNum 130 snap 22 seq_num 130) 2026-02-17T05:48:33.511 INFO:tasks.rados.rados.0.trial115.stdout:420: expect (ObjNum 42 snap 0 seq_num 42) 2026-02-17T05:48:33.575 INFO:tasks.rados.rados.0.trial115.stdout:421: finishing rollback tid 0 to trial11545420-20 2026-02-17T05:48:33.575 INFO:tasks.rados.rados.0.trial115.stdout:421: finishing rollback tid 1 to trial11545420-20 2026-02-17T05:48:33.575 INFO:tasks.rados.rados.0.trial115.stdout:421: finishing rollback tid 2 to trial11545420-20 2026-02-17T05:48:33.575 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 134 (ObjNum 171 snap 30 seq_num 171) dirty exists 2026-02-17T05:48:33.575 INFO:tasks.rados.rados.0.trial115.stdout:418: expect (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:48:33.638 INFO:tasks.rados.rados.0.trial115.stdout:417: finishing copy_from racing read to trial11545420-46 2026-02-17T05:48:33.638 INFO:tasks.rados.rados.0.trial115.stdout:417: finishing copy_from to trial11545420-46 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 24 (ObjNum 78 snap 8 seq_num 78) dirty exists 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:423: finishing rollback tid 0 to trial11545420-42 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:423: finishing rollback tid 1 to trial11545420-42 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:411: done (9 left) 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:413: done (8 left) 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:415: done (7 left) 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:417: done (6 left) 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:418: done (5 left) 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:419: done (4 left) 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:420: done (3 left) 2026-02-17T05:48:33.639 INFO:tasks.rados.rados.0.trial115.stdout:421: done (2 left) 2026-02-17T05:48:33.640 INFO:tasks.rados.rados.0.trial115.stdout:424: snap_create 2026-02-17T05:48:33.640 INFO:tasks.rados.rados.0.trial115.stdout:423: finishing rollback tid 2 to trial11545420-42 2026-02-17T05:48:33.640 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 198 (ObjNum 169 snap 30 seq_num 169) dirty exists 2026-02-17T05:48:33.640 INFO:tasks.rados.rados.0.trial115.stdout:422: expect (ObjNum 191 snap 34 seq_num 191) 2026-02-17T05:48:34.365 INFO:tasks.rados.rados.0.trial115.stdout:422: done (2 left) 2026-02-17T05:48:34.365 INFO:tasks.rados.rados.0.trial115.stdout:423: done (1 left) 2026-02-17T05:48:34.365 INFO:tasks.rados.rados.0.trial115.stdout:424: done (0 left) 2026-02-17T05:48:34.365 INFO:tasks.rados.rados.0.trial115.stdout:425: rollback oid 6 current snap is 35 2026-02-17T05:48:34.365 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 6 to 31 2026-02-17T05:48:34.366 INFO:tasks.rados.rados.0.trial115.stdout:426: snap_remove snap 34 2026-02-17T05:48:35.366 INFO:tasks.rados.rados.0.trial115.stdout:426: done (1 left) 2026-02-17T05:48:35.366 INFO:tasks.rados.rados.0.trial115.stdout:427: copy_from oid 7 from oid 44 current snap is 35 2026-02-17T05:48:35.366 INFO:tasks.rados.rados.0.trial115.stdout:425: finishing rollback tid 0 to trial11545420-6 2026-02-17T05:48:35.366 INFO:tasks.rados.rados.0.trial115.stdout:425: finishing rollback tid 1 to trial11545420-6 2026-02-17T05:48:35.366 INFO:tasks.rados.rados.0.trial115.stdout:425: finishing rollback tid 2 to trial11545420-6 2026-02-17T05:48:35.366 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 52 (ObjNum 75 snap 8 seq_num 75) dirty exists 2026-02-17T05:48:35.366 INFO:tasks.rados.rados.0.trial115.stdout:425: done (1 left) 2026-02-17T05:48:35.367 INFO:tasks.rados.rados.0.trial115.stdout:428: delete oid 10 current snap is 35 2026-02-17T05:48:35.369 INFO:tasks.rados.rados.0.trial115.stdout:428: done (1 left) 2026-02-17T05:48:35.370 INFO:tasks.rados.rados.0.trial115.stdout:429: write (excl) oid 47 current snap is 35 2026-02-17T05:48:35.370 INFO:tasks.rados.rados.0.trial115.stdout:429: seq_num 194 ranges {460558=707131,1869389=508716,3094884=165674} 2026-02-17T05:48:35.374 INFO:tasks.rados.rados.0.trial115.stdout:429: writing trial11545420-47 from 460558 to 1167689 tid 1 2026-02-17T05:48:35.380 INFO:tasks.rados.rados.0.trial115.stdout:429: writing trial11545420-47 from 1869389 to 2378105 tid 2 2026-02-17T05:48:35.383 INFO:tasks.rados.rados.0.trial115.stdout:429: writing trial11545420-47 from 3094884 to 3260558 tid 3 2026-02-17T05:48:35.383 INFO:tasks.rados.rados.0.trial115.stdout:430: rollback oid 42 current snap is 35 2026-02-17T05:48:35.384 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 31 2026-02-17T05:48:35.384 INFO:tasks.rados.rados.0.trial115.stdout:429: finishing write tid 1 to trial11545420-47 2026-02-17T05:48:35.384 INFO:tasks.rados.rados.0.trial115.stdout:429: finishing write tid 2 to trial11545420-47 2026-02-17T05:48:35.384 INFO:tasks.rados.rados.0.trial115.stdout:431: snap_create 2026-02-17T05:48:35.385 INFO:tasks.rados.rados.0.trial115.stdout:429: finishing write tid 3 to trial11545420-47 2026-02-17T05:48:35.385 INFO:tasks.rados.rados.0.trial115.stdout:429: finishing write tid 4 to trial11545420-47 2026-02-17T05:48:35.385 INFO:tasks.rados.rados.0.trial115.stdout:429: finishing write tid 5 to trial11545420-47 2026-02-17T05:48:35.385 INFO:tasks.rados.rados.0.trial115.stdout:429: oid 47 updating version 0 to 174 2026-02-17T05:48:35.385 INFO:tasks.rados.rados.0.trial115.stdout:429: oid 47 version 174 is already newer than 171 2026-02-17T05:48:35.385 INFO:tasks.rados.rados.0.trial115.stdout:429: oid 47 version 174 is already newer than 173 2026-02-17T05:48:35.385 INFO:tasks.rados.rados.0.trial115.stdout:429: oid 47 version 174 is already newer than 172 2026-02-17T05:48:35.386 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 174 (ObjNum 194 snap 35 seq_num 194) dirty exists 2026-02-17T05:48:35.386 INFO:tasks.rados.rados.0.trial115.stdout:429: left oid 47 (ObjNum 194 snap 35 seq_num 194) 2026-02-17T05:48:35.386 INFO:tasks.rados.rados.0.trial115.stdout:430: finishing rollback tid 0 to trial11545420-42 2026-02-17T05:48:35.386 INFO:tasks.rados.rados.0.trial115.stdout:430: finishing rollback tid 1 to trial11545420-42 2026-02-17T05:48:35.386 INFO:tasks.rados.rados.0.trial115.stdout:430: finishing rollback tid 2 to trial11545420-42 2026-02-17T05:48:35.386 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 201 (ObjNum 169 snap 30 seq_num 169) dirty exists 2026-02-17T05:48:35.580 INFO:tasks.rados.rados.0.trial115.stdout:427: finishing copy_from racing read to trial11545420-7 2026-02-17T05:48:35.581 INFO:tasks.rados.rados.0.trial115.stdout:427: finishing copy_from to trial11545420-7 2026-02-17T05:48:35.582 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 54 (ObjNum 192 snap 34 seq_num 192) dirty exists 2026-02-17T05:48:36.361 INFO:tasks.rados.rados.0.trial115.stdout:427: done (3 left) 2026-02-17T05:48:36.362 INFO:tasks.rados.rados.0.trial115.stdout:429: done (2 left) 2026-02-17T05:48:36.362 INFO:tasks.rados.rados.0.trial115.stdout:430: done (1 left) 2026-02-17T05:48:36.362 INFO:tasks.rados.rados.0.trial115.stdout:431: done (0 left) 2026-02-17T05:48:36.362 INFO:tasks.rados.rados.0.trial115.stdout:432: copy_from oid 24 from oid 42 current snap is 36 2026-02-17T05:48:36.362 INFO:tasks.rados.rados.0.trial115.stdout:433: read oid 37 snap 31 2026-02-17T05:48:36.362 INFO:tasks.rados.rados.0.trial115.stdout:433: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:48:36.363 INFO:tasks.rados.rados.0.trial115.stdout:434: write (excl) oid 35 current snap is 36 2026-02-17T05:48:36.363 INFO:tasks.rados.rados.0.trial115.stdout:434: seq_num 195 ranges {477365=782380,1927871=768727,3122307=155058} 2026-02-17T05:48:36.365 INFO:tasks.rados.rados.0.trial115.stdout:434: writing trial11545420-35 from 477365 to 1259745 tid 1 2026-02-17T05:48:36.371 INFO:tasks.rados.rados.0.trial115.stdout:434: writing trial11545420-35 from 1927871 to 2696598 tid 2 2026-02-17T05:48:36.373 INFO:tasks.rados.rados.0.trial115.stdout:434: writing trial11545420-35 from 3122307 to 3277365 tid 3 2026-02-17T05:48:36.373 INFO:tasks.rados.rados.0.trial115.stdout:435: snap_create 2026-02-17T05:48:36.373 INFO:tasks.rados.rados.0.trial115.stdout:433: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:48:36.483 INFO:tasks.rados.rados.0.trial115.stdout:432: finishing copy_from racing read to trial11545420-24 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:432: finishing copy_from to trial11545420-24 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 201 (ObjNum 169 snap 30 seq_num 169) dirty exists 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: finishing write tid 1 to trial11545420-35 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: finishing write tid 2 to trial11545420-35 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: finishing write tid 3 to trial11545420-35 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: finishing write tid 4 to trial11545420-35 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: finishing write tid 5 to trial11545420-35 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: oid 35 updating version 0 to 157 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: oid 35 updating version 157 to 160 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: oid 35 version 160 is already newer than 158 2026-02-17T05:48:36.484 INFO:tasks.rados.rados.0.trial115.stdout:434: oid 35 version 160 is already newer than 159 2026-02-17T05:48:36.485 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 160 (ObjNum 195 snap 36 seq_num 195) dirty exists 2026-02-17T05:48:36.485 INFO:tasks.rados.rados.0.trial115.stdout:434: left oid 35 (ObjNum 195 snap 36 seq_num 195) 2026-02-17T05:48:37.370 INFO:tasks.rados.rados.0.trial115.stdout:432: done (3 left) 2026-02-17T05:48:37.370 INFO:tasks.rados.rados.0.trial115.stdout:433: done (2 left) 2026-02-17T05:48:37.370 INFO:tasks.rados.rados.0.trial115.stdout:434: done (1 left) 2026-02-17T05:48:37.370 INFO:tasks.rados.rados.0.trial115.stdout:435: done (0 left) 2026-02-17T05:48:37.371 INFO:tasks.rados.rados.0.trial115.stdout:436: write oid 3 current snap is 37 2026-02-17T05:48:37.371 INFO:tasks.rados.rados.0.trial115.stdout:436: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2026-02-17T05:48:37.374 INFO:tasks.rados.rados.0.trial115.stdout:436: writing trial11545420-3 from 494172 to 951801 tid 1 2026-02-17T05:48:37.379 INFO:tasks.rados.rados.0.trial115.stdout:436: writing trial11545420-3 from 1586353 to 2098738 tid 2 2026-02-17T05:48:37.384 INFO:tasks.rados.rados.0.trial115.stdout:436: writing trial11545420-3 from 2749730 to 3294172 tid 3 2026-02-17T05:48:37.384 INFO:tasks.rados.rados.0.trial115.stdout:437: snap_create 2026-02-17T05:48:37.384 INFO:tasks.rados.rados.0.trial115.stdout:436: finishing write tid 1 to trial11545420-3 2026-02-17T05:48:37.385 INFO:tasks.rados.rados.0.trial115.stdout:436: finishing write tid 2 to trial11545420-3 2026-02-17T05:48:37.386 INFO:tasks.rados.rados.0.trial115.stdout:436: finishing write tid 3 to trial11545420-3 2026-02-17T05:48:37.387 INFO:tasks.rados.rados.0.trial115.stdout:436: finishing write tid 4 to trial11545420-3 2026-02-17T05:48:37.387 INFO:tasks.rados.rados.0.trial115.stdout:436: finishing write tid 5 to trial11545420-3 2026-02-17T05:48:37.387 INFO:tasks.rados.rados.0.trial115.stdout:436: oid 3 updating version 0 to 179 2026-02-17T05:48:37.387 INFO:tasks.rados.rados.0.trial115.stdout:436: oid 3 version 179 is already newer than 176 2026-02-17T05:48:37.387 INFO:tasks.rados.rados.0.trial115.stdout:436: oid 3 version 179 is already newer than 178 2026-02-17T05:48:37.387 INFO:tasks.rados.rados.0.trial115.stdout:436: oid 3 version 179 is already newer than 177 2026-02-17T05:48:37.388 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 179 (ObjNum 196 snap 37 seq_num 196) dirty exists 2026-02-17T05:48:37.388 INFO:tasks.rados.rados.0.trial115.stdout:436: left oid 3 (ObjNum 196 snap 37 seq_num 196) 2026-02-17T05:48:38.372 INFO:tasks.rados.rados.0.trial115.stdout:436: done (1 left) 2026-02-17T05:48:38.373 INFO:tasks.rados.rados.0.trial115.stdout:437: done (0 left) 2026-02-17T05:48:38.373 INFO:tasks.rados.rados.0.trial115.stdout:438: snap_create 2026-02-17T05:48:39.373 INFO:tasks.rados.rados.0.trial115.stdout:438: done (0 left) 2026-02-17T05:48:39.373 INFO:tasks.rados.rados.0.trial115.stdout:439: delete oid 39 current snap is 39 2026-02-17T05:48:39.375 INFO:tasks.rados.rados.0.trial115.stdout:439: done (0 left) 2026-02-17T05:48:39.375 INFO:tasks.rados.rados.0.trial115.stdout:440: read oid 13 snap -1 2026-02-17T05:48:39.375 INFO:tasks.rados.rados.0.trial115.stdout:440: expect (ObjNum 191 snap 34 seq_num 191) 2026-02-17T05:48:39.376 INFO:tasks.rados.rados.0.trial115.stdout:441: copy_from oid 12 from oid 5 current snap is 39 2026-02-17T05:48:39.376 INFO:tasks.rados.rados.0.trial115.stdout:442: rollback oid 17 current snap is 39 2026-02-17T05:48:39.376 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 37 2026-02-17T05:48:39.376 INFO:tasks.rados.rados.0.trial115.stdout:443: delete oid 1 current snap is 39 2026-02-17T05:48:39.378 INFO:tasks.rados.rados.0.trial115.stdout:440: expect (ObjNum 191 snap 34 seq_num 191) 2026-02-17T05:48:39.494 INFO:tasks.rados.rados.0.trial115.stdout:442: finishing rollback tid 0 to trial11545420-17 2026-02-17T05:48:39.494 INFO:tasks.rados.rados.0.trial115.stdout:442: finishing rollback tid 1 to trial11545420-17 2026-02-17T05:48:39.494 INFO:tasks.rados.rados.0.trial115.stdout:440: done (3 left) 2026-02-17T05:48:39.494 INFO:tasks.rados.rados.0.trial115.stdout:443: done (2 left) 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:444: rollback oid 33 current snap is 39 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 38 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:442: finishing rollback tid 2 to trial11545420-17 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 43 (ObjNum 135 snap 23 seq_num 135) dirty exists 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:441: finishing copy_from racing read to trial11545420-12 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:441: finishing copy_from to trial11545420-12 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 164 (ObjNum 189 snap 33 seq_num 189) dirty exists 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:441: done (2 left) 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:442: done (1 left) 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:445: read oid 38 snap -1 2026-02-17T05:48:39.495 INFO:tasks.rados.rados.0.trial115.stdout:445: expect (ObjNum 5 snap 0 seq_num 5) 2026-02-17T05:48:39.496 INFO:tasks.rados.rados.0.trial115.stdout:446: read oid 31 snap -1 2026-02-17T05:48:39.496 INFO:tasks.rados.rados.0.trial115.stdout:446: expect (ObjNum 138 snap 25 seq_num 138) 2026-02-17T05:48:39.496 INFO:tasks.rados.rados.0.trial115.stdout:447: copy_from oid 36 from oid 7 current snap is 39 2026-02-17T05:48:39.496 INFO:tasks.rados.rados.0.trial115.stdout:448: snap_create 2026-02-17T05:48:39.496 INFO:tasks.rados.rados.0.trial115.stdout:444: finishing rollback tid 0 to trial11545420-33 2026-02-17T05:48:39.497 INFO:tasks.rados.rados.0.trial115.stdout:446: expect (ObjNum 138 snap 25 seq_num 138) 2026-02-17T05:48:39.550 INFO:tasks.rados.rados.0.trial115.stdout:445: expect (ObjNum 5 snap 0 seq_num 5) 2026-02-17T05:48:39.600 INFO:tasks.rados.rados.0.trial115.stdout:444: finishing rollback tid 1 to trial11545420-33 2026-02-17T05:48:39.600 INFO:tasks.rados.rados.0.trial115.stdout:444: finishing rollback tid 2 to trial11545420-33 2026-02-17T05:48:39.600 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 153 (ObjNum 173 snap 30 seq_num 173) dirty exists 2026-02-17T05:48:39.600 INFO:tasks.rados.rados.0.trial115.stdout:447: finishing copy_from racing read to trial11545420-36 2026-02-17T05:48:39.600 INFO:tasks.rados.rados.0.trial115.stdout:447: finishing copy_from to trial11545420-36 2026-02-17T05:48:39.600 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 54 (ObjNum 192 snap 34 seq_num 192) dirty exists 2026-02-17T05:48:40.379 INFO:tasks.rados.rados.0.trial115.stdout:444: done (4 left) 2026-02-17T05:48:40.379 INFO:tasks.rados.rados.0.trial115.stdout:445: done (3 left) 2026-02-17T05:48:40.380 INFO:tasks.rados.rados.0.trial115.stdout:446: done (2 left) 2026-02-17T05:48:40.380 INFO:tasks.rados.rados.0.trial115.stdout:447: done (1 left) 2026-02-17T05:48:40.380 INFO:tasks.rados.rados.0.trial115.stdout:448: done (0 left) 2026-02-17T05:48:40.380 INFO:tasks.rados.rados.0.trial115.stdout:449: write oid 34 current snap is 40 2026-02-17T05:48:40.380 INFO:tasks.rados.rados.0.trial115.stdout:449: seq_num 199 ranges {544593=799729,1761799=776065,2948352=396241} 2026-02-17T05:48:40.384 INFO:tasks.rados.rados.0.trial115.stdout:449: writing trial11545420-34 from 544593 to 1344322 tid 1 2026-02-17T05:48:40.388 INFO:tasks.rados.rados.0.trial115.stdout:449: writing trial11545420-34 from 1761799 to 2537864 tid 2 2026-02-17T05:48:40.392 INFO:tasks.rados.rados.0.trial115.stdout:449: writing trial11545420-34 from 2948352 to 3344593 tid 3 2026-02-17T05:48:40.392 INFO:tasks.rados.rados.0.trial115.stdout:450: snap_create 2026-02-17T05:48:40.392 INFO:tasks.rados.rados.0.trial115.stdout:449: finishing write tid 1 to trial11545420-34 2026-02-17T05:48:40.392 INFO:tasks.rados.rados.0.trial115.stdout:449: finishing write tid 2 to trial11545420-34 2026-02-17T05:48:40.393 INFO:tasks.rados.rados.0.trial115.stdout:449: finishing write tid 3 to trial11545420-34 2026-02-17T05:48:40.393 INFO:tasks.rados.rados.0.trial115.stdout:449: finishing write tid 4 to trial11545420-34 2026-02-17T05:48:40.394 INFO:tasks.rados.rados.0.trial115.stdout:449: finishing write tid 5 to trial11545420-34 2026-02-17T05:48:40.394 INFO:tasks.rados.rados.0.trial115.stdout:449: oid 34 updating version 0 to 59 2026-02-17T05:48:40.394 INFO:tasks.rados.rados.0.trial115.stdout:449: oid 34 version 59 is already newer than 58 2026-02-17T05:48:40.394 INFO:tasks.rados.rados.0.trial115.stdout:449: oid 34 version 59 is already newer than 57 2026-02-17T05:48:40.394 INFO:tasks.rados.rados.0.trial115.stdout:449: oid 34 version 59 is already newer than 56 2026-02-17T05:48:40.394 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 59 (ObjNum 199 snap 40 seq_num 199) dirty exists 2026-02-17T05:48:40.394 INFO:tasks.rados.rados.0.trial115.stdout:449: left oid 34 (ObjNum 199 snap 40 seq_num 199) 2026-02-17T05:48:41.360 INFO:tasks.rados.rados.0.trial115.stdout:449: done (1 left) 2026-02-17T05:48:41.360 INFO:tasks.rados.rados.0.trial115.stdout:450: done (0 left) 2026-02-17T05:48:41.360 INFO:tasks.rados.rados.0.trial115.stdout:451: rollback oid 40 current snap is 41 2026-02-17T05:48:41.360 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 38 2026-02-17T05:48:41.360 INFO:tasks.rados.rados.0.trial115.stdout:452: copy_from oid 23 from oid 14 current snap is 41 2026-02-17T05:48:41.360 INFO:tasks.rados.rados.0.trial115.stdout:453: delete oid 1 current snap is 41 2026-02-17T05:48:41.365 INFO:tasks.rados.rados.0.trial115.stdout:453: done (2 left) 2026-02-17T05:48:41.366 INFO:tasks.rados.rados.0.trial115.stdout:454: write (excl) oid 36 current snap is 41 2026-02-17T05:48:41.366 INFO:tasks.rados.rados.0.trial115.stdout:454: seq_num 201 ranges {578207=550227,1762410=779734,3286845=91362} 2026-02-17T05:48:41.369 INFO:tasks.rados.rados.0.trial115.stdout:454: writing trial11545420-36 from 578207 to 1128434 tid 1 2026-02-17T05:48:41.375 INFO:tasks.rados.rados.0.trial115.stdout:454: writing trial11545420-36 from 1762410 to 2542144 tid 2 2026-02-17T05:48:41.379 INFO:tasks.rados.rados.0.trial115.stdout:454: writing trial11545420-36 from 3286845 to 3378207 tid 3 2026-02-17T05:48:41.379 INFO:tasks.rados.rados.0.trial115.stdout:455: snap_remove snap 40 2026-02-17T05:48:42.382 INFO:tasks.rados.rados.0.trial115.stdout:455: done (3 left) 2026-02-17T05:48:42.383 INFO:tasks.rados.rados.0.trial115.stdout:456: write (excl) oid 20 current snap is 41 2026-02-17T05:48:42.383 INFO:tasks.rados.rados.0.trial115.stdout:451: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:48:42.383 INFO:tasks.rados.rados.0.trial115.stdout:451: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:48:42.383 INFO:tasks.rados.rados.0.trial115.stdout:456: seq_num 202 ranges {595014=625476,1820892=523392,2797915=596476} 2026-02-17T05:48:42.387 INFO:tasks.rados.rados.0.trial115.stdout:456: writing trial11545420-20 from 595014 to 1220490 tid 1 2026-02-17T05:48:42.391 INFO:tasks.rados.rados.0.trial115.stdout:456: writing trial11545420-20 from 1820892 to 2344284 tid 2 2026-02-17T05:48:42.396 INFO:tasks.rados.rados.0.trial115.stdout:456: writing trial11545420-20 from 2797915 to 3394391 tid 3 2026-02-17T05:48:42.396 INFO:tasks.rados.rados.0.trial115.stdout:457: read oid 34 snap -1 2026-02-17T05:48:42.396 INFO:tasks.rados.rados.0.trial115.stdout:457: expect (ObjNum 199 snap 40 seq_num 199) 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:458: delete oid 33 current snap is 41 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:451: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 49 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:452: finishing copy_from racing read to trial11545420-23 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:452: finishing copy_from to trial11545420-23 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 31 (ObjNum 99 snap 14 seq_num 99) dirty exists 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:454: finishing write tid 1 to trial11545420-36 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:454: finishing write tid 2 to trial11545420-36 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:454: finishing write tid 3 to trial11545420-36 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:454: finishing write tid 4 to trial11545420-36 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:454: finishing write tid 5 to trial11545420-36 2026-02-17T05:48:42.397 INFO:tasks.rados.rados.0.trial115.stdout:454: oid 36 updating version 0 to 157 2026-02-17T05:48:42.398 INFO:tasks.rados.rados.0.trial115.stdout:454: oid 36 version 157 is already newer than 154 2026-02-17T05:48:42.398 INFO:tasks.rados.rados.0.trial115.stdout:454: oid 36 version 157 is already newer than 156 2026-02-17T05:48:42.398 INFO:tasks.rados.rados.0.trial115.stdout:454: oid 36 version 157 is already newer than 155 2026-02-17T05:48:42.398 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 157 (ObjNum 201 snap 41 seq_num 201) dirty exists 2026-02-17T05:48:42.398 INFO:tasks.rados.rados.0.trial115.stdout:454: left oid 36 (ObjNum 201 snap 41 seq_num 201) 2026-02-17T05:48:42.398 INFO:tasks.rados.rados.0.trial115.stdout:456: finishing write tid 1 to trial11545420-20 2026-02-17T05:48:42.398 INFO:tasks.rados.rados.0.trial115.stdout:456: finishing write tid 2 to trial11545420-20 2026-02-17T05:48:42.399 INFO:tasks.rados.rados.0.trial115.stdout:451: done (5 left) 2026-02-17T05:48:42.399 INFO:tasks.rados.rados.0.trial115.stdout:452: done (4 left) 2026-02-17T05:48:42.399 INFO:tasks.rados.rados.0.trial115.stdout:454: done (3 left) 2026-02-17T05:48:42.399 INFO:tasks.rados.rados.0.trial115.stdout:458: done (2 left) 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:459: snap_create 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:456: finishing write tid 3 to trial11545420-20 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:456: finishing write tid 4 to trial11545420-20 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:456: finishing write tid 5 to trial11545420-20 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:456: oid 20 updating version 0 to 137 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:456: oid 20 updating version 137 to 138 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:456: oid 20 version 138 is already newer than 136 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:456: oid 20 version 138 is already newer than 135 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 138 (ObjNum 202 snap 41 seq_num 202) dirty exists 2026-02-17T05:48:42.400 INFO:tasks.rados.rados.0.trial115.stdout:456: left oid 20 (ObjNum 202 snap 41 seq_num 202) 2026-02-17T05:48:42.401 INFO:tasks.rados.rados.0.trial115.stdout:457: expect (ObjNum 199 snap 40 seq_num 199) 2026-02-17T05:48:43.384 INFO:tasks.rados.rados.0.trial115.stdout:456: done (2 left) 2026-02-17T05:48:43.384 INFO:tasks.rados.rados.0.trial115.stdout:457: done (1 left) 2026-02-17T05:48:43.385 INFO:tasks.rados.rados.0.trial115.stdout:459: done (0 left) 2026-02-17T05:48:43.385 INFO:tasks.rados.rados.0.trial115.stdout:460: snap_remove snap 35 2026-02-17T05:48:44.458 INFO:tasks.rados.rados.0.trial115.stdout:460: done (0 left) 2026-02-17T05:48:44.458 INFO:tasks.rados.rados.0.trial115.stdout:461: delete oid 28 current snap is 42 2026-02-17T05:48:44.461 INFO:tasks.rados.rados.0.trial115.stdout:461: done (0 left) 2026-02-17T05:48:44.461 INFO:tasks.rados.rados.0.trial115.stdout:462: snap_create 2026-02-17T05:48:45.459 INFO:tasks.rados.rados.0.trial115.stdout:462: done (0 left) 2026-02-17T05:48:45.459 INFO:tasks.rados.rados.0.trial115.stdout:463: read oid 25 snap -1 2026-02-17T05:48:45.460 INFO:tasks.rados.rados.0.trial115.stdout:463: expect (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:48:45.460 INFO:tasks.rados.rados.0.trial115.stdout:464: write (excl) oid 8 current snap is 43 2026-02-17T05:48:45.460 INFO:tasks.rados.rados.0.trial115.stdout:464: seq_num 205 ranges {645435=451223,1879985=787072,3280184=165251} 2026-02-17T05:48:45.464 INFO:tasks.rados.rados.0.trial115.stdout:464: writing trial11545420-8 from 645435 to 1096658 tid 1 2026-02-17T05:48:45.470 INFO:tasks.rados.rados.0.trial115.stdout:464: writing trial11545420-8 from 1879985 to 2667057 tid 2 2026-02-17T05:48:45.473 INFO:tasks.rados.rados.0.trial115.stdout:464: writing trial11545420-8 from 3280184 to 3445435 tid 3 2026-02-17T05:48:45.473 INFO:tasks.rados.rados.0.trial115.stdout:465: read oid 7 snap -1 2026-02-17T05:48:45.473 INFO:tasks.rados.rados.0.trial115.stdout:465: expect (ObjNum 192 snap 34 seq_num 192) 2026-02-17T05:48:45.473 INFO:tasks.rados.rados.0.trial115.stdout:466: copy_from oid 1 from oid 35 current snap is 43 2026-02-17T05:48:45.473 INFO:tasks.rados.rados.0.trial115.stdout:463: expect (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:48:45.548 INFO:tasks.rados.rados.0.trial115.stdout:464: finishing write tid 1 to trial11545420-8 2026-02-17T05:48:45.548 INFO:tasks.rados.rados.0.trial115.stdout:463: done (3 left) 2026-02-17T05:48:45.549 INFO:tasks.rados.rados.0.trial115.stdout:467: snap_remove snap 31 2026-02-17T05:48:46.362 INFO:tasks.rados.rados.0.trial115.stdout:467: done (3 left) 2026-02-17T05:48:46.363 INFO:tasks.rados.rados.0.trial115.stdout:468: rollback oid 18 current snap is 43 2026-02-17T05:48:46.363 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 39 2026-02-17T05:48:46.363 INFO:tasks.rados.rados.0.trial115.stdout:465: expect (ObjNum 192 snap 34 seq_num 192) 2026-02-17T05:48:46.487 INFO:tasks.rados.rados.0.trial115.stdout:464: finishing write tid 2 to trial11545420-8 2026-02-17T05:48:46.487 INFO:tasks.rados.rados.0.trial115.stdout:464: finishing write tid 3 to trial11545420-8 2026-02-17T05:48:46.488 INFO:tasks.rados.rados.0.trial115.stdout:465: done (3 left) 2026-02-17T05:48:46.488 INFO:tasks.rados.rados.0.trial115.stdout:469: snap_remove snap 38 2026-02-17T05:48:47.464 INFO:tasks.rados.rados.0.trial115.stdout:469: done (3 left) 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:470: copy_from oid 20 from oid 34 current snap is 43 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:466: finishing copy_from racing read to trial11545420-1 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:464: finishing write tid 4 to trial11545420-8 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:464: finishing write tid 5 to trial11545420-8 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:464: oid 8 updating version 0 to 26 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:464: oid 8 updating version 26 to 29 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:464: oid 8 version 29 is already newer than 27 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:464: oid 8 version 29 is already newer than 28 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 29 (ObjNum 205 snap 43 seq_num 205) dirty exists 2026-02-17T05:48:47.465 INFO:tasks.rados.rados.0.trial115.stdout:464: left oid 8 (ObjNum 205 snap 43 seq_num 205) 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:466: finishing copy_from to trial11545420-1 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 160 (ObjNum 195 snap 36 seq_num 195) dirty exists 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:468: finishing rollback tid 0 to trial11545420-18 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:468: finishing rollback tid 1 to trial11545420-18 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:468: finishing rollback tid 2 to trial11545420-18 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 143 (ObjNum 78 snap 8 seq_num 78) dirty exists 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:464: done (3 left) 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:466: done (2 left) 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:468: done (1 left) 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:471: write oid 16 current snap is 43 2026-02-17T05:48:47.466 INFO:tasks.rados.rados.0.trial115.stdout:471: seq_num 206 ranges {662242=642825,2054820=530730,3307607=154635} 2026-02-17T05:48:47.469 INFO:tasks.rados.rados.0.trial115.stdout:471: writing trial11545420-16 from 662242 to 1305067 tid 1 2026-02-17T05:48:47.474 INFO:tasks.rados.rados.0.trial115.stdout:471: writing trial11545420-16 from 2054820 to 2585550 tid 2 2026-02-17T05:48:47.477 INFO:tasks.rados.rados.0.trial115.stdout:471: writing trial11545420-16 from 3307607 to 3462242 tid 3 2026-02-17T05:48:47.478 INFO:tasks.rados.rados.0.trial115.stdout:472: delete oid 10 current snap is 43 2026-02-17T05:48:47.478 INFO:tasks.rados.rados.0.trial115.stdout:471: finishing write tid 1 to trial11545420-16 2026-02-17T05:48:47.478 INFO:tasks.rados.rados.0.trial115.stdout:470: finishing copy_from racing read to trial11545420-20 2026-02-17T05:48:47.478 INFO:tasks.rados.rados.0.trial115.stdout:471: finishing write tid 2 to trial11545420-16 2026-02-17T05:48:47.478 INFO:tasks.rados.rados.0.trial115.stdout:470: finishing copy_from to trial11545420-20 2026-02-17T05:48:47.478 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 59 (ObjNum 199 snap 40 seq_num 199) dirty exists 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:471: finishing write tid 3 to trial11545420-16 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:471: finishing write tid 4 to trial11545420-16 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:471: finishing write tid 5 to trial11545420-16 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:471: oid 16 updating version 0 to 167 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:471: oid 16 version 167 is already newer than 164 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:471: oid 16 version 167 is already newer than 165 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:471: oid 16 version 167 is already newer than 166 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 167 (ObjNum 206 snap 43 seq_num 206) dirty exists 2026-02-17T05:48:47.479 INFO:tasks.rados.rados.0.trial115.stdout:471: left oid 16 (ObjNum 206 snap 43 seq_num 206) 2026-02-17T05:48:47.480 INFO:tasks.rados.rados.0.trial115.stdout:470: done (2 left) 2026-02-17T05:48:47.480 INFO:tasks.rados.rados.0.trial115.stdout:471: done (1 left) 2026-02-17T05:48:47.480 INFO:tasks.rados.rados.0.trial115.stdout:472: done (0 left) 2026-02-17T05:48:47.480 INFO:tasks.rados.rados.0.trial115.stdout:473: snap_remove snap 42 2026-02-17T05:48:48.467 INFO:tasks.rados.rados.0.trial115.stdout:473: done (0 left) 2026-02-17T05:48:48.467 INFO:tasks.rados.rados.0.trial115.stdout:474: rollback oid 19 current snap is 43 2026-02-17T05:48:48.467 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 19 to 37 2026-02-17T05:48:48.467 INFO:tasks.rados.rados.0.trial115.stdout:475: read oid 12 snap 39 2026-02-17T05:48:48.468 INFO:tasks.rados.rados.0.trial115.stdout:475: expect (ObjNum 189 snap 33 seq_num 189) 2026-02-17T05:48:48.468 INFO:tasks.rados.rados.0.trial115.stdout:476: snap_remove snap 36 2026-02-17T05:48:49.469 INFO:tasks.rados.rados.0.trial115.stdout:476: done (2 left) 2026-02-17T05:48:49.470 INFO:tasks.rados.rados.0.trial115.stdout:477: snap_create 2026-02-17T05:48:49.470 INFO:tasks.rados.rados.0.trial115.stdout:474: finishing rollback tid 0 to trial11545420-19 2026-02-17T05:48:49.470 INFO:tasks.rados.rados.0.trial115.stdout:474: finishing rollback tid 1 to trial11545420-19 2026-02-17T05:48:49.470 INFO:tasks.rados.rados.0.trial115.stdout:474: finishing rollback tid 2 to trial11545420-19 2026-02-17T05:48:49.470 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 171 (ObjNum 106 snap 16 seq_num 106) dirty exists 2026-02-17T05:48:49.470 INFO:tasks.rados.rados.0.trial115.stdout:475: expect (ObjNum 189 snap 33 seq_num 189) 2026-02-17T05:48:50.462 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:48:50.462 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T05:48:50.462 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:48:50.462 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:48:50.465 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:48:50.465 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:48:50.465 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:50.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient: get_monmap_and_config 2026-02-17T05:48:50.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient: build_initial_monmap 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient: monmap: 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:50.533426+0000 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:50.533426+0000 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:50.536 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 1 Processor -- start 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 1 -- start start 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient: init 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f861406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f861409c4a0 0x7f861413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient(hunting): picked mon.noname-a con 0x7f861409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f861406d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861406daf0 con 0x7f861406c890 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861409cc80 con 0x7f861406d380 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f86140a0880 con 0x7f861409c4a0 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.532+0000 7f861c333640 10 monclient(hunting): authenticate will time out at 776.086965s 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f861409c4a0 0x7f861413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f861a0a8640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f861406c890 0x7f861406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56172/0 (socket says 10.20.193.78:56172) 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f861a0a8640 1 -- 10.20.193.78:0/754453592 learned_addr learned my addr 10.20.193.78:0/754453592 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 10 monclient(hunting): get_auth_request con 0x7f861409c4a0 auth_method 0 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 1 -- 10.20.193.78:0/754453592 >> v2:10.20.193.78:3300/0 conn(0x7f861409c4a0 msgr2=0x7f861413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:48:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 1 -- 10.20.193.78:0/754453592 >> v2:10.20.193.78:3300/0 conn(0x7f861409c4a0 msgr2=0x7f861413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 1 --2- 10.20.193.78:0/754453592 >> v2:10.20.193.78:3300/0 conn(0x7f861409c4a0 0x7f861413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.533+0000 7f86198a7640 1 --2- 10.20.193.78:0/754453592 >> v2:10.20.193.78:3300/0 conn(0x7f861409c4a0 0x7f861413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:50.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 1 -- 10.20.193.78:0/754453592 >> v1:10.20.193.78:6789/0 conn(0x7f861406c890 legacy=0x7f861406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a8a9640 1 -- 10.20.193.78:0/754453592 >> v1:10.20.193.115:6789/0 conn(0x7f861406d380 legacy=0x7f861413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2788088280 0 0) 0x7f861409cc80 con 0x7f861406d380 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): _negotiate 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): my global_id is 44167 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85fc003620 con 0x7f861406d380 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3896539831 0 0) 0x7f85fc003620 con 0x7f861406d380 2026-02-17T05:48:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 >> v2:10.20.193.78:3300/0 conn(0x7f861409c4a0 msgr2=0x7f861413b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 --2- 10.20.193.78:0/754453592 >> v2:10.20.193.78:3300/0 conn(0x7f861409c4a0 0x7f861413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 >> v1:10.20.193.78:6789/0 conn(0x7f861406c890 legacy=0x7f861406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: _finish_hunting 0 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861409cf60 con 0x7f861406d380 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: _finish_auth 0 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: _check_auth_tickets 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f8610003570 con 0x7f861406d380 2026-02-17T05:48:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: dump: 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:50.552 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 >> v1:10.20.193.115:6789/0 conn(0x7f861406d380 legacy=0x7f861413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f85fc004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): picked mon.a con 0x7f85fc003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85fc004900 0x7f85fc004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): picked mon.c con 0x7f85fc004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85fc009ab0 0x7f85fc00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): picked mon.b con 0x7f85fc009ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc00c490 con 0x7f85fc003c20 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc00c640 con 0x7f85fc009ab0 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f85fc00c840 con 0x7f85fc004900 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86190a6640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f85fc004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86198a7640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85fc004900 0x7f85fc004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient(hunting): get_auth_request con 0x7f85fc003c20 auth_method 0 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86198a7640 10 monclient(hunting): get_auth_request con 0x7f85fc004900 auth_method 0 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86198a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86198a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86198a7640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86198a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86198a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f86198a7640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient(hunting): handle_auth_done global_id 44167 payload 293 2026-02-17T05:48:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 1 -- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85fc004900 msgr2=0x7f85fc004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85fc004900 0x7f85fc004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 1 -- 10.20.193.78:0/754453592 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85fc009ab0 msgr2=0x7f85fc00bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85fc009ab0 0x7f85fc00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient: _finish_hunting 0 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 1 monclient: found mon.a 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a8a9640 1 -- 10.20.193.78:0/754453592 reap_dead start 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 1 -- 10.20.193.78:0/754453592 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc003620 con 0x7f85fc003c20 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient: _finish_auth 0 2026-02-17T05:48:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 10 monclient: _check_auth_tickets 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861a0a8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861c333640 5 monclient: authenticate success, global_id 44167 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.535+0000 7f861c333640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861a0a8640 1 -- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 msgr2=0x7f85fc004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861a0a8640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f85fc004040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8604003360 tx=0x7f860402f0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebc7f70f7b5637f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604003060 con 0x7f85fc003c20 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 10 monclient: dump: 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.557 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8604033030 con 0x7f85fc003c20 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 1 -- 10.20.193.78:0/754453592 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860402f8f0 con 0x7f85fc003c20 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 4 set_mon_vals no callback set 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f86190a6640 10 monclient: dump: 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:50.559 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 10 monclient: shutdown 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 1 -- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 msgr2=0x7f85fc004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f85fc004040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8604003360 tx=0x7f860402f0d0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 1 -- 10.20.193.78:0/754453592 shutdown_connections 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 1 --2- 10.20.193.78:0/754453592 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f85fc004040 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 1 -- 10.20.193.78:0/754453592 >> 10.20.193.78:0/754453592 conn(0x7f86140a6620 msgr2=0x7f86140a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 1 -- 10.20.193.78:0/754453592 shutdown_connections 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.538+0000 7f861c333640 1 -- 10.20.193.78:0/754453592 wait complete. 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.539+0000 7f861c333640 10 monclient: build_initial_monmap 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.539+0000 7f861c333640 10 monclient: monmap: 2026-02-17T05:48:50.560 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.539+0000 7f861c333640 1 Processor -- start 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.539+0000 7f861c333640 1 -- start start 2026-02-17T05:48:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.539+0000 7f861c333640 10 monclient: init 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f86141cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient(hunting): picked mon.noname-a con 0x7f85fc003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86141cfbe0 0x7f861409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient(hunting): picked mon.noname-c con 0x7f86141cfbe0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f861409e090 0x7f861409e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient(hunting): picked mon.noname-b con 0x7f861409e090 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f861409bfa0 con 0x7f85fc003c20 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f861409be20 con 0x7f861409e090 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f861406cc70 con 0x7f86141cfbe0 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861c333640 10 monclient(hunting): authenticate will time out at 776.094216s 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f86141cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86141cfbe0 0x7f861409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f86141cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40024/0 (socket says 10.20.193.78:40024) 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 1 -- 10.20.193.78:0/3669523050 learned_addr learned my addr 10.20.193.78:0/3669523050 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 10 monclient(hunting): get_auth_request con 0x7f85fc003c20 auth_method 0 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient(hunting): get_auth_request con 0x7f86141cfbe0 auth_method 0 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 1 -- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 msgr2=0x7f86141cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 1 -- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 msgr2=0x7f86141cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f86141cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f861a0a8640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f86141cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:50.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient(hunting): handle_auth_done global_id 4541 payload 824 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 1 -- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f861409e090 msgr2=0x7f861409e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f861409e090 0x7f861409e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 1 -- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 msgr2=0x7f86141cf6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f86141cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient: _finish_hunting 0 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 1 monclient: found mon.noname-c 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.540+0000 7f86198a7640 1 -- 10.20.193.78:0/3669523050 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8614127fd0 con 0x7f86141cfbe0 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f86198a7640 1 -- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86141cfbe0 msgr2=0x7f861409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f86198a7640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86141cfbe0 0x7f861409db50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86080115d0 tx=0x7f8608011aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:50.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 1 -- 10.20.193.78:0/3669523050 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608019020 con 0x7f86141cfbe0 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: dump: 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:50.566 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: _finish_auth 0 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 1 -- 10.20.193.78:0/3669523050 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8608003710 con 0x7f86141cfbe0 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f861c333640 5 monclient: authenticate success, global_id 4541 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f861c333640 10 monclient: _renew_subs 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f861c333640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861409d810 con 0x7f86141cfbe0 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 1 -- 10.20.193.78:0/3669523050 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860800c070 con 0x7f86141cfbe0 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:50.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f85faffd640 10 monclient: dump: 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:50.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f86188a5640 4 set_mon_vals no callback set 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f861c333640 10 monclient: _renew_subs 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f861c333640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.542+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8614128ec0 con 0x7f86141cfbe0 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f85faffd640 1 -- 10.20.193.78:0/3669523050 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8608003b40 con 0x7f86141cfbe0 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f861c333640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f861c333640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f861c333640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861406cff0 con 0x7f86141cfbe0 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f85faffd640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85e403dee0 0x7f85e40403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f85faffd640 1 -- 10.20.193.78:0/3669523050 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(110..110 src has 1..110) ==== 4827+0+0 (secure 0 0 0) 0x7f8608052f90 con 0x7f86141cfbe0 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f861a0a8640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85e403dee0 0x7f85e40403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.543+0000 7f861a0a8640 10 monclient: get_auth_request con 0x7f85e403dee0 auth_method 0 2026-02-17T05:48:50.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.545+0000 7f85faffd640 1 -- 10.20.193.78:0/3669523050 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8608023020 con 0x7f86141cfbe0 2026-02-17T05:48:50.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.545+0000 7f85faffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:50.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.545+0000 7f85faffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:50.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.546+0000 7f861a0a8640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85e403dee0 0x7f85e40403a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8604034040 tx=0x7f8604007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:50.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.660+0000 7f861c333640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:48:50.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.660+0000 7f861c333640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:50.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.660+0000 7f861c333640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:50.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.660+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8614129260 con 0x7f86141cfbe0 2026-02-17T05:48:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.661+0000 7f85faffd640 1 -- 10.20.193.78:0/3669523050 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f86080143e0 con 0x7f86141cfbe0 2026-02-17T05:48:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.661+0000 7f85faffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.661+0000 7f85faffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:50.663 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":26,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:48:50.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85e403dee0 msgr2=0x7f85e40403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:50.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85e403dee0 0x7f85e40403a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8604034040 tx=0x7f8604007000 comp rx=0 tx=0).stop 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 10 monclient: shutdown 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86141cfbe0 msgr2=0x7f861409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86141cfbe0 0x7f861409db50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86080115d0 tx=0x7f8608011aa0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 shutdown_connections 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85e403dee0 0x7f85e40403a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f861409e090 0x7f861409e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86141cfbe0 0x7f861409db50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 --2- 10.20.193.78:0/3669523050 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85fc003c20 0x7f86141cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 >> 10.20.193.78:0/3669523050 conn(0x7f86140a6620 msgr2=0x7f86140721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 shutdown_connections 2026-02-17T05:48:50.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:50.662+0000 7f861c333640 1 -- 10.20.193.78:0/3669523050 wait complete. 2026-02-17T05:48:50.976 INFO:tasks.rados.rados.0.trial115.stdout:474: done (2 left) 2026-02-17T05:48:50.976 INFO:tasks.rados.rados.0.trial115.stdout:475: done (1 left) 2026-02-17T05:48:50.976 INFO:tasks.rados.rados.0.trial115.stdout:477: done (0 left) 2026-02-17T05:48:50.977 INFO:tasks.rados.rados.0.trial115.stdout:478: copy_from oid 12 from oid 20 current snap is 44 2026-02-17T05:48:50.977 INFO:tasks.rados.rados.0.trial115.stdout:479: write oid 18 current snap is 44 2026-02-17T05:48:50.977 INFO:tasks.rados.rados.0.trial115.stdout:479: seq_num 208 ranges {695856=793323,2171784=534399,3362453=133403} 2026-02-17T05:48:50.982 INFO:tasks.rados.rados.0.trial115.stdout:479: writing trial11545420-18 from 695856 to 1489179 tid 1 2026-02-17T05:48:50.987 INFO:tasks.rados.rados.0.trial115.stdout:479: writing trial11545420-18 from 2171784 to 2706183 tid 2 2026-02-17T05:48:50.991 INFO:tasks.rados.rados.0.trial115.stdout:479: writing trial11545420-18 from 3362453 to 3495856 tid 3 2026-02-17T05:48:50.991 INFO:tasks.rados.rados.0.trial115.stdout:480: copy_from oid 9 from oid 19 current snap is 44 2026-02-17T05:48:50.991 INFO:tasks.rados.rados.0.trial115.stdout:479: finishing write tid 1 to trial11545420-18 2026-02-17T05:48:50.991 INFO:tasks.rados.rados.0.trial115.stdout:478: finishing copy_from racing read to trial11545420-12 2026-02-17T05:48:50.991 INFO:tasks.rados.rados.0.trial115.stdout:479: finishing write tid 2 to trial11545420-18 2026-02-17T05:48:50.991 INFO:tasks.rados.rados.0.trial115.stdout:478: finishing copy_from to trial11545420-12 2026-02-17T05:48:50.992 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 59 (ObjNum 199 snap 40 seq_num 199) dirty exists 2026-02-17T05:48:50.992 INFO:tasks.rados.rados.0.trial115.stdout:478: done (2 left) 2026-02-17T05:48:50.992 INFO:tasks.rados.rados.0.trial115.stdout:481: snap_remove snap 37 2026-02-17T05:48:51.362 INFO:tasks.rados.rados.0.trial115.stdout:481: done (2 left) 2026-02-17T05:48:51.362 INFO:tasks.rados.rados.0.trial115.stdout:482: delete oid 11 current snap is 44 2026-02-17T05:48:51.362 INFO:tasks.rados.rados.0.trial115.stdout:479: finishing write tid 3 to trial11545420-18 2026-02-17T05:48:51.362 INFO:tasks.rados.rados.0.trial115.stdout:479: finishing write tid 4 to trial11545420-18 2026-02-17T05:48:51.362 INFO:tasks.rados.rados.0.trial115.stdout:479: finishing write tid 5 to trial11545420-18 2026-02-17T05:48:51.362 INFO:tasks.rados.rados.0.trial115.stdout:479: oid 18 updating version 0 to 151 2026-02-17T05:48:51.363 INFO:tasks.rados.rados.0.trial115.stdout:479: oid 18 updating version 151 to 153 2026-02-17T05:48:51.363 INFO:tasks.rados.rados.0.trial115.stdout:479: oid 18 updating version 153 to 154 2026-02-17T05:48:51.363 INFO:tasks.rados.rados.0.trial115.stdout:479: oid 18 version 154 is already newer than 152 2026-02-17T05:48:51.363 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 154 (ObjNum 208 snap 44 seq_num 208) dirty exists 2026-02-17T05:48:51.363 INFO:tasks.rados.rados.0.trial115.stdout:479: left oid 18 (ObjNum 208 snap 44 seq_num 208) 2026-02-17T05:48:51.363 INFO:tasks.rados.rados.0.trial115.stdout:480: finishing copy_from racing read to trial11545420-9 2026-02-17T05:48:51.363 INFO:tasks.rados.rados.0.trial115.stdout:480: finishing copy_from to trial11545420-9 2026-02-17T05:48:51.363 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 171 (ObjNum 106 snap 16 seq_num 106) dirty exists 2026-02-17T05:48:51.365 INFO:tasks.rados.rados.0.trial115.stdout:479: done (2 left) 2026-02-17T05:48:51.365 INFO:tasks.rados.rados.0.trial115.stdout:480: done (1 left) 2026-02-17T05:48:51.365 INFO:tasks.rados.rados.0.trial115.stdout:482: done (0 left) 2026-02-17T05:48:51.365 INFO:tasks.rados.rados.0.trial115.stdout:483: delete oid 25 current snap is 44 2026-02-17T05:48:51.368 INFO:tasks.rados.rados.0.trial115.stdout:483: done (0 left) 2026-02-17T05:48:51.368 INFO:tasks.rados.rados.0.trial115.stdout:484: copy_from oid 20 from oid 18 current snap is 44 2026-02-17T05:48:51.368 INFO:tasks.rados.rados.0.trial115.stdout:485: read oid 13 snap -1 2026-02-17T05:48:51.368 INFO:tasks.rados.rados.0.trial115.stdout:485: expect (ObjNum 191 snap 34 seq_num 191) 2026-02-17T05:48:51.368 INFO:tasks.rados.rados.0.trial115.stdout:486: write oid 29 current snap is 44 2026-02-17T05:48:51.368 INFO:tasks.rados.rados.0.trial115.stdout:486: seq_num 211 ranges {746277=619070,1830877=681726,2928369=617908} 2026-02-17T05:48:51.373 INFO:tasks.rados.rados.0.trial115.stdout:486: writing trial11545420-29 from 746277 to 1365347 tid 1 2026-02-17T05:48:51.377 INFO:tasks.rados.rados.0.trial115.stdout:486: writing trial11545420-29 from 1830877 to 2512603 tid 2 2026-02-17T05:48:51.381 INFO:tasks.rados.rados.0.trial115.stdout:486: writing trial11545420-29 from 2928369 to 3546277 tid 3 2026-02-17T05:48:51.381 INFO:tasks.rados.rados.0.trial115.stdout:487: read oid 10 snap -1 2026-02-17T05:48:51.381 INFO:tasks.rados.rados.0.trial115.stdout:487: expect deleted 2026-02-17T05:48:51.382 INFO:tasks.rados.rados.0.trial115.stdout:488: write oid 27 current snap is 44 2026-02-17T05:48:51.382 INFO:tasks.rados.rados.0.trial115.stdout:488: seq_num 212 ranges {763084=694319,1889359=541737,2955792=541902} 2026-02-17T05:48:51.386 INFO:tasks.rados.rados.0.trial115.stdout:488: writing trial11545420-27 from 763084 to 1457403 tid 1 2026-02-17T05:48:51.390 INFO:tasks.rados.rados.0.trial115.stdout:488: writing trial11545420-27 from 1889359 to 2431096 tid 2 2026-02-17T05:48:51.394 INFO:tasks.rados.rados.0.trial115.stdout:488: writing trial11545420-27 from 2955792 to 3497694 tid 3 2026-02-17T05:48:51.394 INFO:tasks.rados.rados.0.trial115.stdout:489: read oid 2 snap -1 2026-02-17T05:48:51.394 INFO:tasks.rados.rados.0.trial115.stdout:489: expect (ObjNum 130 snap 22 seq_num 130) 2026-02-17T05:48:51.395 INFO:tasks.rados.rados.0.trial115.stdout:490: write (excl) oid 43 current snap is 44 2026-02-17T05:48:51.395 INFO:tasks.rados.rados.0.trial115.stdout:490: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2026-02-17T05:48:51.399 INFO:tasks.rados.rados.0.trial115.stdout:490: writing trial11545420-43 from 779891 to 1265812 tid 1 2026-02-17T05:48:51.404 INFO:tasks.rados.rados.0.trial115.stdout:490: writing trial11545420-43 from 1947841 to 2633236 tid 2 2026-02-17T05:48:51.408 INFO:tasks.rados.rados.0.trial115.stdout:490: writing trial11545420-43 from 3383215 to 3579891 tid 3 2026-02-17T05:48:51.408 INFO:tasks.rados.rados.0.trial115.stdout:491: write oid 9 current snap is 44 2026-02-17T05:48:51.408 INFO:tasks.rados.rados.0.trial115.stdout:491: seq_num 214 ranges {796698=561170,2006323=545406,3126991=469707} 2026-02-17T05:48:51.412 INFO:tasks.rados.rados.0.trial115.stdout:491: writing trial11545420-9 from 796698 to 1357868 tid 1 2026-02-17T05:48:51.417 INFO:tasks.rados.rados.0.trial115.stdout:491: writing trial11545420-9 from 2006323 to 2551729 tid 2 2026-02-17T05:48:51.421 INFO:tasks.rados.rados.0.trial115.stdout:491: writing trial11545420-9 from 3126991 to 3596698 tid 3 2026-02-17T05:48:51.421 INFO:tasks.rados.rados.0.trial115.stdout:492: snap_remove snap 41 2026-02-17T05:48:52.848 INFO:tasks.rados.rados.0.trial115.stdout:492: done (8 left) 2026-02-17T05:48:52.848 INFO:tasks.rados.rados.0.trial115.stdout:493: snap_remove snap 39 2026-02-17T05:48:53.672 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:53.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.740+0000 7f85de3e7640 10 monclient: get_monmap_and_config 2026-02-17T05:48:53.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.740+0000 7f85de3e7640 10 monclient: build_initial_monmap 2026-02-17T05:48:53.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.740+0000 7f85de3e7640 10 monclient: monmap: 2026-02-17T05:48:53.742 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:53.742 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:53.742 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:53.742003+0000 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:53.742003+0000 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:53.743 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:53.744 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:53.744 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:53.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 1 Processor -- start 2026-02-17T05:48:53.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 1 -- start start 2026-02-17T05:48:53.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient: init 2026-02-17T05:48:53.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:53.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient(hunting): picked mon.noname-b con 0x7f85d806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f85d809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f85d806d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d8135c10 con 0x7f85d806d1b0 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d806daf0 con 0x7f85d809c4a0 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f85d80a0880 con 0x7f85d806c890 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85de3e7640 10 monclient(hunting): authenticate will time out at 779.295546s 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:53.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53950/0 (socket says 10.20.193.78:53950) 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 1 -- 10.20.193.78:0/493859667 learned_addr learned my addr 10.20.193.78:0/493859667 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient(hunting): get_auth_request con 0x7f85d806c890 auth_method 0 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85dc95d640 1 --1- 10.20.193.78:0/493859667 >> v1:10.20.193.78:6789/0 conn(0x7f85d806d1b0 0x7f85d8135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56176/0 (socket says 10.20.193.78:56176) 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient(hunting): handle_auth_done global_id 4544 payload 274 2026-02-17T05:48:53.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 1 -- 10.20.193.78:0/493859667 >> v1:10.20.193.115:6789/0 conn(0x7f85d809c4a0 legacy=0x7f85d8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 1 -- 10.20.193.78:0/493859667 >> v1:10.20.193.78:6789/0 conn(0x7f85d806d1b0 legacy=0x7f85d8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 1 monclient: found mon.noname-b 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.741+0000 7f85d7fff640 1 -- 10.20.193.78:0/493859667 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d809cc80 con 0x7f85d806c890 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d7fff640 1 -- 10.20.193.78:0/493859667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 msgr2=0x7f85d806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d7fff640 1 --2- 10.20.193.78:0/493859667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d806cc70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f85c40099a0 tx=0x7f85c402ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8cb4388ee536b58 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 1 -- 10.20.193.78:0/493859667 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c403c070 con 0x7f85d806c890 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:48:53.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: dump: 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:48:53.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 1 -- 10.20.193.78:0/493859667 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c402fd90 con 0x7f85d806c890 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 5 monclient: authenticate success, global_id 4544 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 1 -- 10.20.193.78:0/493859667 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4037660 con 0x7f85d806c890 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 10 monclient: get_monmap_and_config success 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 4 set_mon_vals no callback set 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85d6ffd640 10 monclient: dump: 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:53.756 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 10 monclient: shutdown 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 1 -- 10.20.193.78:0/493859667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 msgr2=0x7f85d806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 1 --2- 10.20.193.78:0/493859667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d806cc70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f85c40099a0 tx=0x7f85c402ee30 comp rx=0 tx=0).stop 2026-02-17T05:48:53.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 1 -- 10.20.193.78:0/493859667 shutdown_connections 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 1 --2- 10.20.193.78:0/493859667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d806cc70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 1 -- 10.20.193.78:0/493859667 >> 10.20.193.78:0/493859667 conn(0x7f85d80a6620 msgr2=0x7f85d80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 1 -- 10.20.193.78:0/493859667 shutdown_connections 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.744+0000 7f85de3e7640 1 -- 10.20.193.78:0/493859667 wait complete. 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient: build_initial_monmap 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient: monmap: 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:53.758 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 1 Processor -- start 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 1 -- start start 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient: init 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d8134350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient(hunting): picked mon.noname-c con 0x7f85d806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 0x7f85d8134890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:53.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient(hunting): picked mon.noname-a con 0x7f85d806d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85d809c4a0 0x7f85d81cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient(hunting): picked mon.noname-b con 0x7f85d809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85d80a0020 con 0x7f85d806d1b0 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85d806d600 con 0x7f85d809c4a0 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f85d80a01a0 con 0x7f85d806c890 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85de3e7640 10 monclient(hunting): authenticate will time out at 779.299861s 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 0x7f85d8134890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d8134350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:53.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 0x7f85d8134890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40040/0 (socket says 10.20.193.78:40040) 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 1 -- 10.20.193.78:0/3298214601 learned_addr learned my addr 10.20.193.78:0/3298214601 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d8134350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53964/0 (socket says 10.20.193.78:53964) 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 10 monclient(hunting): get_auth_request con 0x7f85d806d1b0 auth_method 0 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 10 monclient(hunting): get_auth_request con 0x7f85d806c890 auth_method 0 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 1 -- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 msgr2=0x7f85d8134890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 1 -- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 msgr2=0x7f85d8134890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:48:53.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d77fe640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 0x7f85d8134890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.745+0000 7f85d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d77fe640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 0x7f85d8134890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85dc95d640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85d809c4a0 0x7f85d81cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85dc95d640 10 monclient(hunting): get_auth_request con 0x7f85d809c4a0 auth_method 0 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85dc95d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85dc95d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85dc95d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 10 monclient(hunting): handle_auth_done global_id 4547 payload 824 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 1 -- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85d809c4a0 msgr2=0x7f85d81cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:53.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85d809c4a0 0x7f85d81cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 1 -- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 msgr2=0x7f85d8134890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 0x7f85d8134890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 1 monclient: found mon.noname-c 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.746+0000 7f85d7fff640 1 -- 10.20.193.78:0/3298214601 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d8125f20 con 0x7f85d806c890 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d7fff640 1 -- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 msgr2=0x7f85d8134350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d7fff640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d8134350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85d806dcc0 tx=0x7f85c40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 1 -- 10.20.193.78:0/3298214601 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4046020 con 0x7f85d806c890 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:48:53.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: dump: 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: _finish_auth 0 2026-02-17T05:48:53.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 1 -- 10.20.193.78:0/3298214601 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c4042660 con 0x7f85d806c890 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 1 -- 10.20.193.78:0/3298214601 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c403c040 con 0x7f85d806c890 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d4ff9640 10 monclient: dump: 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:48:53.765 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85d67fc640 4 set_mon_vals no callback set 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85de3e7640 5 monclient: authenticate success, global_id 4547 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85de3e7640 10 monclient: _renew_subs 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85de3e7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d809d810 con 0x7f85d806c890 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85de3e7640 10 monclient: _renew_subs 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85de3e7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:53.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.748+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d8127a90 con 0x7f85d806c890 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85de3e7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85de3e7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85de3e7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8069a00 con 0x7f85d806c890 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85d4ff9640 1 -- 10.20.193.78:0/3298214601 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f85c403f030 con 0x7f85d806c890 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85d4ff9640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85a803dee0 0x7f85a80403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85d4ff9640 1 -- 10.20.193.78:0/3298214601 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(113..113 src has 1..113) ==== 4779+0+0 (secure 0 0 0) 0x7f85c4077a50 con 0x7f85d806c890 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85d77fe640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85a803dee0 0x7f85a80403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:53.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.749+0000 7f85d77fe640 10 monclient: get_auth_request con 0x7f85a803dee0 auth_method 0 2026-02-17T05:48:53.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.751+0000 7f85d4ff9640 1 -- 10.20.193.78:0/3298214601 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f85d8069a00 con 0x7f85d806c890 2026-02-17T05:48:53.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.751+0000 7f85d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:48:53.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.751+0000 7f85d4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:48:53.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.752+0000 7f85d77fe640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85a803dee0 0x7f85a80403a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85d80a2f00 tx=0x7f85c8004050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:48:53.849 INFO:tasks.rados.rados.0.trial115.stdout:493: done (8 left) 2026-02-17T05:48:53.849 INFO:tasks.rados.rados.0.trial115.stdout:494: copy_from oid 40 from oid 31 current snap is 44 2026-02-17T05:48:53.849 INFO:tasks.rados.rados.0.trial115.stdout:485: expect (ObjNum 191 snap 34 seq_num 191) 2026-02-17T05:48:53.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.869+0000 7f85de3e7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:48:53.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.869+0000 7f85de3e7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:53.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.869+0000 7f85de3e7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:48:53.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.869+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f85d806cc70 con 0x7f85d806c890 2026-02-17T05:48:53.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.869+0000 7f85d4ff9640 1 -- 10.20.193.78:0/3298214601 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f85c4050030 con 0x7f85d806c890 2026-02-17T05:48:53.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.869+0000 7f85d4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:48:53.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.869+0000 7f85d4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:48:53.872 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":26,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":26,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:48:53.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.870+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85a803dee0 msgr2=0x7f85a80403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:53.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.870+0000 7f85de3e7640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85a803dee0 0x7f85a80403a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85d80a2f00 tx=0x7f85c8004050 comp rx=0 tx=0).stop 2026-02-17T05:48:53.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.870+0000 7f85de3e7640 10 monclient: shutdown 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.870+0000 7f85de3e7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.870+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 msgr2=0x7f85d8134350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.870+0000 7f85de3e7640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d8134350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85d806dcc0 tx=0x7f85c40047c0 comp rx=0 tx=0).stop 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.871+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 shutdown_connections 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.871+0000 7f85de3e7640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85a803dee0 0x7f85a80403a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.871+0000 7f85de3e7640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85d809c4a0 0x7f85d81cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.871+0000 7f85de3e7640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85d806d1b0 0x7f85d8134890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.871+0000 7f85de3e7640 1 --2- 10.20.193.78:0/3298214601 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85d806c890 0x7f85d8134350 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.871+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 >> 10.20.193.78:0/3298214601 conn(0x7f85d80a6620 msgr2=0x7f85d8135340 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.871+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 shutdown_connections 2026-02-17T05:48:53.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:53.871+0000 7f85de3e7640 1 -- 10.20.193.78:0/3298214601 wait complete. 2026-02-17T05:48:53.965 INFO:tasks.rados.rados.0.trial115.stdout:486: finishing write tid 1 to trial11545420-29 2026-02-17T05:48:53.965 INFO:tasks.rados.rados.0.trial115.stdout:484: finishing copy_from racing read to trial11545420-20 2026-02-17T05:48:53.966 INFO:tasks.rados.rados.0.trial115.stdout:484: finishing copy_from to trial11545420-20 2026-02-17T05:48:53.966 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 154 (ObjNum 208 snap 44 seq_num 208) dirty exists 2026-02-17T05:48:53.966 INFO:tasks.rados.rados.0.trial115.stdout:486: finishing write tid 2 to trial11545420-29 2026-02-17T05:48:53.966 INFO:tasks.rados.rados.0.trial115.stdout:484: done (8 left) 2026-02-17T05:48:53.966 INFO:tasks.rados.rados.0.trial115.stdout:485: done (7 left) 2026-02-17T05:48:53.966 INFO:tasks.rados.rados.0.trial115.stdout:495: read oid 16 snap -1 2026-02-17T05:48:53.967 INFO:tasks.rados.rados.0.trial115.stdout:495: expect (ObjNum 206 snap 43 seq_num 206) 2026-02-17T05:48:53.967 INFO:tasks.rados.rados.0.trial115.stdout:496: snap_create 2026-02-17T05:48:53.967 INFO:tasks.rados.rados.0.trial115.stdout:486: finishing write tid 3 to trial11545420-29 2026-02-17T05:48:53.967 INFO:tasks.rados.rados.0.trial115.stdout:486: finishing write tid 4 to trial11545420-29 2026-02-17T05:48:53.967 INFO:tasks.rados.rados.0.trial115.stdout:486: finishing write tid 5 to trial11545420-29 2026-02-17T05:48:53.967 INFO:tasks.rados.rados.0.trial115.stdout:486: oid 29 updating version 0 to 84 2026-02-17T05:48:53.967 INFO:tasks.rados.rados.0.trial115.stdout:486: oid 29 updating version 84 to 85 2026-02-17T05:48:53.968 INFO:tasks.rados.rados.0.trial115.stdout:486: oid 29 version 85 is already newer than 82 2026-02-17T05:48:53.968 INFO:tasks.rados.rados.0.trial115.stdout:486: oid 29 version 85 is already newer than 83 2026-02-17T05:48:53.968 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 85 (ObjNum 211 snap 44 seq_num 211) dirty exists 2026-02-17T05:48:53.968 INFO:tasks.rados.rados.0.trial115.stdout:486: left oid 29 (ObjNum 211 snap 44 seq_num 211) 2026-02-17T05:48:53.968 INFO:tasks.rados.rados.0.trial115.stdout:488: finishing write tid 1 to trial11545420-27 2026-02-17T05:48:53.968 INFO:tasks.rados.rados.0.trial115.stdout:488: finishing write tid 2 to trial11545420-27 2026-02-17T05:48:53.968 INFO:tasks.rados.rados.0.trial115.stdout:489: expect (ObjNum 130 snap 22 seq_num 130) 2026-02-17T05:48:54.029 INFO:tasks.rados.rados.0.trial115.stdout:488: finishing write tid 3 to trial11545420-27 2026-02-17T05:48:54.029 INFO:tasks.rados.rados.0.trial115.stdout:488: finishing write tid 4 to trial11545420-27 2026-02-17T05:48:54.029 INFO:tasks.rados.rados.0.trial115.stdout:488: finishing write tid 5 to trial11545420-27 2026-02-17T05:48:54.030 INFO:tasks.rados.rados.0.trial115.stdout:488: oid 27 updating version 0 to 152 2026-02-17T05:48:54.030 INFO:tasks.rados.rados.0.trial115.stdout:488: oid 27 updating version 152 to 155 2026-02-17T05:48:54.030 INFO:tasks.rados.rados.0.trial115.stdout:488: oid 27 version 155 is already newer than 154 2026-02-17T05:48:54.030 INFO:tasks.rados.rados.0.trial115.stdout:488: oid 27 version 155 is already newer than 153 2026-02-17T05:48:54.030 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 155 (ObjNum 212 snap 44 seq_num 212) dirty exists 2026-02-17T05:48:54.030 INFO:tasks.rados.rados.0.trial115.stdout:488: left oid 27 (ObjNum 212 snap 44 seq_num 212) 2026-02-17T05:48:54.031 INFO:tasks.rados.rados.0.trial115.stdout:490: finishing write tid 1 to trial11545420-43 2026-02-17T05:48:54.031 INFO:tasks.rados.rados.0.trial115.stdout:490: finishing write tid 2 to trial11545420-43 2026-02-17T05:48:54.031 INFO:tasks.rados.rados.0.trial115.stdout:490: finishing write tid 3 to trial11545420-43 2026-02-17T05:48:54.031 INFO:tasks.rados.rados.0.trial115.stdout:490: finishing write tid 4 to trial11545420-43 2026-02-17T05:48:54.031 INFO:tasks.rados.rados.0.trial115.stdout:490: finishing write tid 5 to trial11545420-43 2026-02-17T05:48:54.031 INFO:tasks.rados.rados.0.trial115.stdout:490: oid 43 updating version 0 to 157 2026-02-17T05:48:54.031 INFO:tasks.rados.rados.0.trial115.stdout:490: oid 43 version 157 is already newer than 155 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:490: oid 43 updating version 157 to 158 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:490: oid 43 version 158 is already newer than 156 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 158 (ObjNum 213 snap 44 seq_num 213) dirty exists 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:490: left oid 43 (ObjNum 213 snap 44 seq_num 213) 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:491: finishing write tid 1 to trial11545420-9 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:491: finishing write tid 2 to trial11545420-9 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:491: finishing write tid 3 to trial11545420-9 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:491: finishing write tid 4 to trial11545420-9 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:491: finishing write tid 5 to trial11545420-9 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:491: oid 9 updating version 0 to 180 2026-02-17T05:48:54.032 INFO:tasks.rados.rados.0.trial115.stdout:491: oid 9 version 180 is already newer than 178 2026-02-17T05:48:54.033 INFO:tasks.rados.rados.0.trial115.stdout:491: oid 9 version 180 is already newer than 179 2026-02-17T05:48:54.033 INFO:tasks.rados.rados.0.trial115.stdout:491: oid 9 updating version 180 to 181 2026-02-17T05:48:54.033 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 181 (ObjNum 214 snap 44 seq_num 214) dirty exists 2026-02-17T05:48:54.033 INFO:tasks.rados.rados.0.trial115.stdout:491: left oid 9 (ObjNum 214 snap 44 seq_num 214) 2026-02-17T05:48:54.033 INFO:tasks.rados.rados.0.trial115.stdout:494: finishing copy_from racing read to trial11545420-40 2026-02-17T05:48:54.033 INFO:tasks.rados.rados.0.trial115.stdout:494: finishing copy_from to trial11545420-40 2026-02-17T05:48:54.033 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 168 (ObjNum 138 snap 25 seq_num 138) dirty exists 2026-02-17T05:48:54.033 INFO:tasks.rados.rados.0.trial115.stdout:495: expect (ObjNum 206 snap 43 seq_num 206) 2026-02-17T05:48:54.975 INFO:tasks.rados.rados.0.trial115.stdout:486: done (8 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:487: done (7 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:488: done (6 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:489: done (5 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:490: done (4 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:491: done (3 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:494: done (2 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:495: done (1 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:496: done (0 left) 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:497: copy_from oid 27 from oid 33 current snap is 45 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:498: write (excl) oid 41 current snap is 45 2026-02-17T05:48:54.976 INFO:tasks.rados.rados.0.trial115.stdout:498: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2026-02-17T05:48:54.978 INFO:tasks.rados.rados.0.trial115.stdout:498: writing trial11545420-41 from 413505 to 1049924 tid 1 2026-02-17T05:48:54.983 INFO:tasks.rados.rados.0.trial115.stdout:498: writing trial11545420-41 from 1664805 to 2353869 tid 2 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:498: writing trial11545420-41 from 3038061 to 3530132 tid 3 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:499: rollback oid 50 current snap is 45 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 50 to 43 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:497: finishing copy_from to trial11545420-27 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:497: got expected ENOENT (src dne) 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:497: finishing copy_from racing read to trial11545420-27 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:498: finishing write tid 1 to trial11545420-41 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:498: finishing write tid 2 to trial11545420-41 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:497: done (2 left) 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:500: write oid 10 current snap is 45 2026-02-17T05:48:54.988 INFO:tasks.rados.rados.0.trial115.stdout:500: seq_num 216 ranges {430312=711668,1723287=549075,2781837=703343} 2026-02-17T05:48:54.991 INFO:tasks.rados.rados.0.trial115.stdout:500: writing trial11545420-10 from 430312 to 1141980 tid 1 2026-02-17T05:48:54.996 INFO:tasks.rados.rados.0.trial115.stdout:500: writing trial11545420-10 from 1723287 to 2272362 tid 2 2026-02-17T05:48:55.001 INFO:tasks.rados.rados.0.trial115.stdout:500: writing trial11545420-10 from 2781837 to 3485180 tid 3 2026-02-17T05:48:55.001 INFO:tasks.rados.rados.0.trial115.stdout:501: write oid 42 current snap is 45 2026-02-17T05:48:55.002 INFO:tasks.rados.rados.0.trial115.stdout:501: seq_num 217 ranges {447119=786917,1665416=692733,2976554=514615} 2026-02-17T05:48:55.005 INFO:tasks.rados.rados.0.trial115.stdout:501: writing trial11545420-42 from 447119 to 1234036 tid 1 2026-02-17T05:48:55.009 INFO:tasks.rados.rados.0.trial115.stdout:501: writing trial11545420-42 from 1665416 to 2358149 tid 2 2026-02-17T05:48:55.014 INFO:tasks.rados.rados.0.trial115.stdout:501: writing trial11545420-42 from 2976554 to 3491169 tid 3 2026-02-17T05:48:55.014 INFO:tasks.rados.rados.0.trial115.stdout:502: write (excl) oid 29 current snap is 45 2026-02-17T05:48:55.014 INFO:tasks.rados.rados.0.trial115.stdout:502: seq_num 218 ranges {463926=462166,1723898=436391,2603977=725887} 2026-02-17T05:48:55.017 INFO:tasks.rados.rados.0.trial115.stdout:502: writing trial11545420-29 from 463926 to 926092 tid 1 2026-02-17T05:48:55.022 INFO:tasks.rados.rados.0.trial115.stdout:502: writing trial11545420-29 from 1723898 to 2160289 tid 2 2026-02-17T05:48:55.026 INFO:tasks.rados.rados.0.trial115.stdout:502: writing trial11545420-29 from 2603977 to 3329864 tid 3 2026-02-17T05:48:55.027 INFO:tasks.rados.rados.0.trial115.stdout:503: write oid 24 current snap is 45 2026-02-17T05:48:55.027 INFO:tasks.rados.rados.0.trial115.stdout:498: finishing write tid 3 to trial11545420-41 2026-02-17T05:48:55.027 INFO:tasks.rados.rados.0.trial115.stdout:503: seq_num 219 ranges {480733=537415,1782380=696402,3031400=537159} 2026-02-17T05:48:55.030 INFO:tasks.rados.rados.0.trial115.stdout:503: writing trial11545420-24 from 480733 to 1018148 tid 1 2026-02-17T05:48:55.035 INFO:tasks.rados.rados.0.trial115.stdout:503: writing trial11545420-24 from 1782380 to 2478782 tid 2 2026-02-17T05:48:55.040 INFO:tasks.rados.rados.0.trial115.stdout:503: writing trial11545420-24 from 3031400 to 3568559 tid 3 2026-02-17T05:48:55.040 INFO:tasks.rados.rados.0.trial115.stdout:504: write (excl) oid 7 current snap is 45 2026-02-17T05:48:55.040 INFO:tasks.rados.rados.0.trial115.stdout:504: seq_num 220 ranges {497540=612664,1840862=440060,3058823=464784} 2026-02-17T05:48:55.044 INFO:tasks.rados.rados.0.trial115.stdout:504: writing trial11545420-7 from 497540 to 1110204 tid 1 2026-02-17T05:48:55.048 INFO:tasks.rados.rados.0.trial115.stdout:504: writing trial11545420-7 from 1840862 to 2280922 tid 2 2026-02-17T05:48:55.053 INFO:tasks.rados.rados.0.trial115.stdout:504: writing trial11545420-7 from 3058823 to 3523607 tid 3 2026-02-17T05:48:55.053 INFO:tasks.rados.rados.0.trial115.stdout:505: write oid 43 current snap is 45 2026-02-17T05:48:55.053 INFO:tasks.rados.rados.0.trial115.stdout:505: seq_num 221 ranges {514347=404266,1499344=700071,2686246=676056} 2026-02-17T05:48:55.056 INFO:tasks.rados.rados.0.trial115.stdout:505: writing trial11545420-43 from 514347 to 918613 tid 1 2026-02-17T05:48:55.061 INFO:tasks.rados.rados.0.trial115.stdout:505: writing trial11545420-43 from 1499344 to 2199415 tid 2 2026-02-17T05:48:55.065 INFO:tasks.rados.rados.0.trial115.stdout:505: writing trial11545420-43 from 2686246 to 3362302 tid 3 2026-02-17T05:48:55.065 INFO:tasks.rados.rados.0.trial115.stdout:506: read oid 35 snap -1 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:506: expect (ObjNum 195 snap 36 seq_num 195) 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:507: snap_remove snap 44 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:498: finishing write tid 4 to trial11545420-41 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:498: finishing write tid 5 to trial11545420-41 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:498: oid 41 updating version 0 to 168 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:498: oid 41 updating version 168 to 170 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:498: oid 41 updating version 170 to 171 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:498: oid 41 version 171 is already newer than 169 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 171 (ObjNum 215 snap 45 seq_num 215) dirty exists 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:498: left oid 41 (ObjNum 215 snap 45 seq_num 215) 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:499: finishing rollback tid 0 to trial11545420-50 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:499: finishing rollback tid 1 to trial11545420-50 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:499: finishing rollback tid 2 to trial11545420-50 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 159 (ObjNum 160 snap 29 seq_num 160) dirty exists 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: finishing write tid 1 to trial11545420-10 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: finishing write tid 2 to trial11545420-10 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: finishing write tid 3 to trial11545420-10 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: finishing write tid 4 to trial11545420-10 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: finishing write tid 5 to trial11545420-10 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: oid 10 updating version 0 to 187 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: oid 10 version 187 is already newer than 186 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: oid 10 updating version 187 to 189 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: oid 10 version 189 is already newer than 188 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 189 (ObjNum 216 snap 45 seq_num 216) dirty exists 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:500: left oid 10 (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:501: finishing write tid 1 to trial11545420-42 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:501: finishing write tid 2 to trial11545420-42 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:501: finishing write tid 3 to trial11545420-42 2026-02-17T05:48:55.066 INFO:tasks.rados.rados.0.trial115.stdout:501: finishing write tid 4 to trial11545420-42 2026-02-17T05:48:55.856 INFO:tasks.rados.rados.0.trial115.stdout:498: done (9 left) 2026-02-17T05:48:55.856 INFO:tasks.rados.rados.0.trial115.stdout:499: done (8 left) 2026-02-17T05:48:55.856 INFO:tasks.rados.rados.0.trial115.stdout:500: done (7 left) 2026-02-17T05:48:55.856 INFO:tasks.rados.rados.0.trial115.stdout:507: done (6 left) 2026-02-17T05:48:55.856 INFO:tasks.rados.rados.0.trial115.stdout:508: write oid 8 current snap is 45 2026-02-17T05:48:55.856 INFO:tasks.rados.rados.0.trial115.stdout:508: seq_num 222 ranges {531154=479515,1557826=443729,2713669=487328,3679295=51859} 2026-02-17T05:48:55.859 INFO:tasks.rados.rados.0.trial115.stdout:508: writing trial11545420-8 from 531154 to 1010669 tid 1 2026-02-17T05:48:55.863 INFO:tasks.rados.rados.0.trial115.stdout:508: writing trial11545420-8 from 1557826 to 2001555 tid 2 2026-02-17T05:48:55.867 INFO:tasks.rados.rados.0.trial115.stdout:508: writing trial11545420-8 from 2713669 to 3200997 tid 3 2026-02-17T05:48:55.870 INFO:tasks.rados.rados.0.trial115.stdout:508: writing trial11545420-8 from 3679295 to 3731154 tid 4 2026-02-17T05:48:55.870 INFO:tasks.rados.rados.0.trial115.stdout:509: read oid 34 snap 43 2026-02-17T05:48:55.870 INFO:tasks.rados.rados.0.trial115.stdout:509: expect (ObjNum 199 snap 40 seq_num 199) 2026-02-17T05:48:55.870 INFO:tasks.rados.rados.0.trial115.stdout:510: write oid 14 current snap is 45 2026-02-17T05:48:55.870 INFO:tasks.rados.rados.0.trial115.stdout:501: finishing write tid 5 to trial11545420-42 2026-02-17T05:48:55.870 INFO:tasks.rados.rados.0.trial115.stdout:501: oid 42 updating version 0 to 242 2026-02-17T05:48:55.870 INFO:tasks.rados.rados.0.trial115.stdout:501: oid 42 version 242 is already newer than 241 2026-02-17T05:48:55.871 INFO:tasks.rados.rados.0.trial115.stdout:501: oid 42 version 242 is already newer than 239 2026-02-17T05:48:55.871 INFO:tasks.rados.rados.0.trial115.stdout:501: oid 42 version 242 is already newer than 240 2026-02-17T05:48:55.871 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 242 (ObjNum 217 snap 45 seq_num 217) dirty exists 2026-02-17T05:48:55.871 INFO:tasks.rados.rados.0.trial115.stdout:501: left oid 42 (ObjNum 217 snap 45 seq_num 217) 2026-02-17T05:48:55.871 INFO:tasks.rados.rados.0.trial115.stdout:502: finishing write tid 1 to trial11545420-29 2026-02-17T05:48:55.871 INFO:tasks.rados.rados.0.trial115.stdout:510: seq_num 223 ranges {547961=554764,1616308=703740,2741092=698600} 2026-02-17T05:48:55.873 INFO:tasks.rados.rados.0.trial115.stdout:510: writing trial11545420-14 from 547961 to 1102725 tid 1 2026-02-17T05:48:55.878 INFO:tasks.rados.rados.0.trial115.stdout:510: writing trial11545420-14 from 1616308 to 2320048 tid 2 2026-02-17T05:48:55.883 INFO:tasks.rados.rados.0.trial115.stdout:510: writing trial11545420-14 from 2741092 to 3439692 tid 3 2026-02-17T05:48:55.883 INFO:tasks.rados.rados.0.trial115.stdout:501: done (8 left) 2026-02-17T05:48:55.883 INFO:tasks.rados.rados.0.trial115.stdout:511: delete oid 44 current snap is 45 2026-02-17T05:48:55.883 INFO:tasks.rados.rados.0.trial115.stdout:502: finishing write tid 2 to trial11545420-29 2026-02-17T05:48:55.883 INFO:tasks.rados.rados.0.trial115.stdout:502: finishing write tid 3 to trial11545420-29 2026-02-17T05:48:55.883 INFO:tasks.rados.rados.0.trial115.stdout:502: finishing write tid 4 to trial11545420-29 2026-02-17T05:48:55.883 INFO:tasks.rados.rados.0.trial115.stdout:502: finishing write tid 5 to trial11545420-29 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:502: oid 29 updating version 0 to 94 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:502: oid 29 version 94 is already newer than 93 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:502: oid 29 version 94 is already newer than 91 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:502: oid 29 version 94 is already newer than 92 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 94 (ObjNum 218 snap 45 seq_num 218) dirty exists 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:502: left oid 29 (ObjNum 218 snap 45 seq_num 218) 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:503: finishing write tid 1 to trial11545420-24 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:503: finishing write tid 2 to trial11545420-24 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:503: finishing write tid 3 to trial11545420-24 2026-02-17T05:48:55.884 INFO:tasks.rados.rados.0.trial115.stdout:503: finishing write tid 4 to trial11545420-24 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:503: finishing write tid 5 to trial11545420-24 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:503: oid 24 updating version 0 to 246 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:503: oid 24 version 246 is already newer than 244 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:503: oid 24 version 246 is already newer than 245 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:503: oid 24 updating version 246 to 247 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 247 (ObjNum 219 snap 45 seq_num 219) dirty exists 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:503: left oid 24 (ObjNum 219 snap 45 seq_num 219) 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:504: finishing write tid 1 to trial11545420-7 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:504: finishing write tid 2 to trial11545420-7 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:504: finishing write tid 3 to trial11545420-7 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:504: finishing write tid 4 to trial11545420-7 2026-02-17T05:48:55.885 INFO:tasks.rados.rados.0.trial115.stdout:504: finishing write tid 5 to trial11545420-7 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:504: oid 7 updating version 0 to 161 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:504: oid 7 updating version 161 to 162 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:504: oid 7 version 162 is already newer than 159 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:504: oid 7 version 162 is already newer than 160 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 162 (ObjNum 220 snap 45 seq_num 220) dirty exists 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:504: left oid 7 (ObjNum 220 snap 45 seq_num 220) 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:505: finishing write tid 1 to trial11545420-43 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:505: finishing write tid 2 to trial11545420-43 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:505: finishing write tid 3 to trial11545420-43 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:505: finishing write tid 4 to trial11545420-43 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:505: finishing write tid 5 to trial11545420-43 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:505: oid 43 updating version 0 to 165 2026-02-17T05:48:55.886 INFO:tasks.rados.rados.0.trial115.stdout:505: oid 43 version 165 is already newer than 164 2026-02-17T05:48:55.887 INFO:tasks.rados.rados.0.trial115.stdout:505: oid 43 updating version 165 to 166 2026-02-17T05:48:55.887 INFO:tasks.rados.rados.0.trial115.stdout:505: oid 43 updating version 166 to 167 2026-02-17T05:48:55.887 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 167 (ObjNum 221 snap 45 seq_num 221) dirty exists 2026-02-17T05:48:55.887 INFO:tasks.rados.rados.0.trial115.stdout:505: left oid 43 (ObjNum 221 snap 45 seq_num 221) 2026-02-17T05:48:55.887 INFO:tasks.rados.rados.0.trial115.stdout:506: expect (ObjNum 195 snap 36 seq_num 195) 2026-02-17T05:48:55.986 INFO:tasks.rados.rados.0.trial115.stdout:508: finishing write tid 1 to trial11545420-8 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: finishing write tid 2 to trial11545420-8 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: finishing write tid 3 to trial11545420-8 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: finishing write tid 4 to trial11545420-8 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: finishing write tid 5 to trial11545420-8 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: finishing write tid 6 to trial11545420-8 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: oid 8 updating version 0 to 35 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: oid 8 version 35 is already newer than 31 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: oid 8 version 35 is already newer than 32 2026-02-17T05:48:55.987 INFO:tasks.rados.rados.0.trial115.stdout:508: oid 8 version 35 is already newer than 33 2026-02-17T05:48:55.988 INFO:tasks.rados.rados.0.trial115.stdout:508: oid 8 version 35 is already newer than 34 2026-02-17T05:48:55.988 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 35 (ObjNum 222 snap 45 seq_num 222) dirty exists 2026-02-17T05:48:55.988 INFO:tasks.rados.rados.0.trial115.stdout:508: left oid 8 (ObjNum 222 snap 45 seq_num 222) 2026-02-17T05:48:55.988 INFO:tasks.rados.rados.0.trial115.stdout:509: expect (ObjNum 199 snap 40 seq_num 199) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: finishing write tid 1 to trial11545420-14 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: finishing write tid 2 to trial11545420-14 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: finishing write tid 3 to trial11545420-14 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: finishing write tid 4 to trial11545420-14 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:502: done (8 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:503: done (7 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:504: done (6 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:505: done (5 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:506: done (4 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:508: done (3 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:509: done (2 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:511: done (1 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:512: copy_from oid 2 from oid 42 current snap is 45 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: finishing write tid 5 to trial11545420-14 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: oid 14 updating version 0 to 162 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: oid 14 version 162 is already newer than 161 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: oid 14 updating version 162 to 163 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: oid 14 updating version 163 to 164 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 164 (ObjNum 223 snap 45 seq_num 223) dirty exists 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: left oid 14 (ObjNum 223 snap 45 seq_num 223) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:510: done (1 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:513: delete oid 40 current snap is 45 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:513: done (1 left) 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:514: copy_from oid 46 from oid 15 current snap is 45 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:515: copy_from oid 38 from oid 17 current snap is 45 2026-02-17T05:48:56.177 INFO:tasks.rados.rados.0.trial115.stdout:516: snap_remove snap 43 2026-02-17T05:48:56.363 INFO:tasks.rados.rados.0.trial115.stdout:516: done (3 left) 2026-02-17T05:48:56.363 INFO:tasks.rados.rados.0.trial115.stdout:517: read oid 27 snap -1 2026-02-17T05:48:56.363 INFO:tasks.rados.rados.0.trial115.stdout:517: expect (ObjNum 212 snap 44 seq_num 212) 2026-02-17T05:48:56.363 INFO:tasks.rados.rados.0.trial115.stdout:518: rollback oid 19 current snap is 45 2026-02-17T05:48:56.363 INFO:tasks.rados.rados.0.trial115.stdout:518: done (4 left) 2026-02-17T05:48:56.363 INFO:tasks.rados.rados.0.trial115.stdout:519: copy_from oid 8 from oid 19 current snap is 45 2026-02-17T05:48:56.363 INFO:tasks.rados.rados.0.trial115.stdout:514: finishing copy_from to trial11545420-46 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:514: got expected ENOENT (src dne) 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:514: finishing copy_from racing read to trial11545420-46 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:515: finishing copy_from racing read to trial11545420-38 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:515: finishing copy_from to trial11545420-38 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 43 (ObjNum 135 snap 23 seq_num 135) dirty exists 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:512: finishing copy_from racing read to trial11545420-2 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:512: finishing copy_from to trial11545420-2 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 242 (ObjNum 217 snap 45 seq_num 217) dirty exists 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:512: done (4 left) 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:514: done (3 left) 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:515: done (2 left) 2026-02-17T05:48:56.364 INFO:tasks.rados.rados.0.trial115.stdout:520: delete oid 2 current snap is 45 2026-02-17T05:48:56.366 INFO:tasks.rados.rados.0.trial115.stdout:520: done (2 left) 2026-02-17T05:48:56.366 INFO:tasks.rados.rados.0.trial115.stdout:521: rollback oid 47 current snap is 45 2026-02-17T05:48:56.366 INFO:tasks.rados.rados.0.trial115.stdout:521: done (2 left) 2026-02-17T05:48:56.366 INFO:tasks.rados.rados.0.trial115.stdout:522: copy_from oid 39 from oid 10 current snap is 45 2026-02-17T05:48:56.367 INFO:tasks.rados.rados.0.trial115.stdout:523: snap_create 2026-02-17T05:48:56.371 INFO:tasks.rados.rados.0.trial115.stdout:517: expect (ObjNum 212 snap 44 seq_num 212) 2026-02-17T05:48:56.453 INFO:tasks.rados.rados.0.trial115.stdout:519: finishing copy_from racing read to trial11545420-8 2026-02-17T05:48:56.453 INFO:tasks.rados.rados.0.trial115.stdout:522: finishing copy_from racing read to trial11545420-39 2026-02-17T05:48:56.453 INFO:tasks.rados.rados.0.trial115.stdout:519: finishing copy_from to trial11545420-8 2026-02-17T05:48:56.453 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 171 (ObjNum 106 snap 16 seq_num 106) dirty exists 2026-02-17T05:48:56.453 INFO:tasks.rados.rados.0.trial115.stdout:522: finishing copy_from to trial11545420-39 2026-02-17T05:48:56.453 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 189 (ObjNum 216 snap 45 seq_num 216) dirty exists 2026-02-17T05:48:56.881 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.949+0000 7f647e291640 10 monclient: get_monmap_and_config 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.949+0000 7f647e291640 10 monclient: build_initial_monmap 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.949+0000 7f647e291640 10 monclient: monmap: 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:48:56.950337+0000 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:48:56.950337+0000 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:48:56.950 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.949+0000 7f647e291640 1 Processor -- start 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.949+0000 7f647e291640 1 -- start start 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.949+0000 7f647e291640 10 monclient: init 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 10 monclient: _reopen_session rank -1 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 10 monclient(hunting): picked mon.noname-a con 0x7f647809ea90 addr v2:10.20.193.78:3300/0 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f647809ee70 addr v1:10.20.193.78:6789/0 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f647813d640 addr v1:10.20.193.115:6789/0 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64780a0a20 con 0x7f647809ee70 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64780a0460 con 0x7f647813d640 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f64780a0d00 con 0x7f647809ea90 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f647e291640 10 monclient(hunting): authenticate will time out at 782.504091s 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40054/0 (socket says 10.20.193.78:40054) 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 learned_addr learned my addr 10.20.193.78:0/72817148 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f647809ea90 auth_method 0 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:56.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:56.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:56.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:56.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.950+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:56.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.952+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v1:10.20.193.78:6789/0 conn(0x7f647809ee70 legacy=0x7f647809f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:56.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:56.952+0000 7f647c807640 1 -- 10.20.193.78:0/72817148 >> v1:10.20.193.115:6789/0 conn(0x7f647813d640 legacy=0x7f647813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:48:57.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:57.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f647809ea90 auth_method 0 2026-02-17T05:48:57.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:57.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:57.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:57.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:57.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:57.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:57.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.150+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:48:57.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:57.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f647809ea90 auth_method 0 2026-02-17T05:48:57.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:57.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:57.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:57.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:57.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:57.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:57.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:57.551+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:48:58.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:58.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f647809ea90 auth_method 0 2026-02-17T05:48:58.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:58.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:58.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:48:58.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:58.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:58.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:58.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:58.352+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:48:59.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient(hunting): tick 2026-02-17T05:48:59.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:48:59.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 monclient(hunting): continuing hunt 2026-02-17T05:48:59.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:48:59.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 msgr2=0x7f64780a13d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:48:59.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.78:3300/0 conn(0x7f647809ea90 0x7f64780a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 >> v1:10.20.193.115:6789/0 conn(0x7f647813d640 legacy=0x7f647813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 >> v1:10.20.193.78:6789/0 conn(0x7f647809ee70 legacy=0x7f647809f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient: _add_conns ranks=[2,4] 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f6454003910 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f6454006470 addr v2:10.20.193.115:3300/0 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f64780a0d00 con 0x7f6454006470 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6454008fd0 con 0x7f6454003910 2026-02-17T05:48:59.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6475ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f6454003910 auth_method 0 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 10 monclient(hunting): get_auth_request con 0x7f6454006470 auth_method 0 2026-02-17T05:48:59.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:48:59.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:48:59.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:48:59.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:48:59.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:48:59.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:48:59.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:48:59.950+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:00.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f6454003910 auth_method 0 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.150+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:00.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 10 monclient(hunting): get_auth_request con 0x7f6454006470 auth_method 0 2026-02-17T05:49:00.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:00.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:00.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:00.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:00.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:00.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:00.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.151+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f6454003910 auth_method 0 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:00.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 10 monclient(hunting): get_auth_request con 0x7f6454006470 auth_method 0 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:00.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:00.551+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:49:01.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:01.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f6454003910 auth_method 0 2026-02-17T05:49:01.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:01.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:01.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:01.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:01.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:01.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:01.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.353+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:49:01.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:01.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 10 monclient(hunting): get_auth_request con 0x7f6454006470 auth_method 0 2026-02-17T05:49:01.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:01.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:01.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:01.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:01.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:01.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:01.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:01.354+0000 7f64777fe640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): tick 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 monclient(hunting): continuing hunt 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 msgr2=0x7f6454005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003910 0x7f6454005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 msgr2=0x7f6454008910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f6454006470 0x7f6454008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 1 -- 10.20.193.78:0/72817148 reap_dead start 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2026-02-17T05:49:02.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f6454003d30 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6454006d90 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f64540089f0 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64780a0740 con 0x7f6454006d90 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f64780a0d00 con 0x7f64540089f0 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 1 -- 10.20.193.78:0/72817148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6454008fd0 con 0x7f6454003d30 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6475ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f6454003d30 auth_method 0 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:02.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 msgr2=0x7f6454006850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 msgr2=0x7f6454006850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 10 monclient(hunting): get_auth_request con 0x7f64540089f0 auth_method 0 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 msgr2=0x7f645400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:49:02.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 msgr2=0x7f645400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:02.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:02.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.951+0000 7f647c807640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:02.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:02.953+0000 7f64777fe640 1 -- 10.20.193.78:0/72817148 >> v1:10.20.193.78:6789/0 conn(0x7f6454006d90 legacy=0x7f6454007210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f6454003d30 auth_method 0 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 msgr2=0x7f6454006850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 msgr2=0x7f6454006850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f647c807640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.151+0000 7f647c807640 10 monclient(hunting): get_auth_request con 0x7f64540089f0 auth_method 0 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.152+0000 7f647c807640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.152+0000 7f647c807640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.152+0000 7f647c807640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:03.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.152+0000 7f647c807640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 msgr2=0x7f645400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:03.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.152+0000 7f647c807640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 msgr2=0x7f645400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:03.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.152+0000 7f647c807640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:03.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.152+0000 7f647c807640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:49:03.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f6454003d30 auth_method 0 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient(hunting): handle_auth_done global_id 4550 payload 274 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 msgr2=0x7f645400d450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> v1:10.20.193.78:6789/0 conn(0x7f6454006d90 legacy=0x7f6454007210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:03.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 1 monclient: found mon.noname-b 2026-02-17T05:49:03.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:03.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.552+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645400ec70 con 0x7f6454003d30 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6477fff640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 msgr2=0x7f6454006850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6477fff640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6464006fd0 tx=0x7f6464031170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c60a19faea3a4f2f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 1 -- 10.20.193.78:0/72817148 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646403e070 con 0x7f6454003d30 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: dump: 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.562 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: _finish_auth 0 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:03.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 1 -- 10.20.193.78:0/72817148 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6464031bf0 con 0x7f6454003d30 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 1 -- 10.20.193.78:0/72817148 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464039440 con 0x7f6454003d30 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 5 monclient: authenticate success, global_id 4550 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 4 set_mon_vals no callback set 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f6476ffd640 10 monclient: dump: 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.564 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 10 monclient: shutdown 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 1 -- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 msgr2=0x7f6454006850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6464006fd0 tx=0x7f6464031170 comp rx=0 tx=0).stop 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 1 -- 10.20.193.78:0/72817148 shutdown_connections 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 1 --2- 10.20.193.78:0/72817148 >> v2:10.20.193.115:3300/0 conn(0x7f64540089f0 0x7f645400d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 1 --2- 10.20.193.78:0/72817148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454003d30 0x7f6454006850 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 1 -- 10.20.193.78:0/72817148 >> 10.20.193.78:0/72817148 conn(0x7f64780a7520 msgr2=0x7f64780a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 1 -- 10.20.193.78:0/72817148 shutdown_connections 2026-02-17T05:49:03.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.555+0000 7f647e291640 1 -- 10.20.193.78:0/72817148 wait complete. 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient: build_initial_monmap 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient: monmap: 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 1 Processor -- start 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 1 -- start start 2026-02-17T05:49:03.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient: init 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 0x7f647806fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient(hunting): picked mon.noname-a con 0x7f6454003d30 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454006d90 0x7f6478070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient(hunting): picked mon.noname-c con 0x7f6454006d90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64540089f0 0x7f6478070860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient(hunting): picked mon.noname-b con 0x7f64540089f0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f647809f440 con 0x7f6454003d30 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f647809f2c0 con 0x7f64540089f0 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f647809f5c0 con 0x7f6454006d90 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:03.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.556+0000 7f647e291640 10 monclient(hunting): authenticate will time out at 789.109998s 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454006d90 0x7f6478070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454006d90 0x7f6478070320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39858/0 (socket says 10.20.193.78:39858) 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 0x7f647806fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 -- 10.20.193.78:0/3973315000 learned_addr learned my addr 10.20.193.78:0/3973315000 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient(hunting): get_auth_request con 0x7f6454006d90 auth_method 0 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f6454003d30 auth_method 0 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 1 -- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 msgr2=0x7f647806fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 1 -- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 msgr2=0x7f647806fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 0x7f647806fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f6477fff640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 0x7f647806fde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f647c807640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64540089f0 0x7f6478070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient(hunting): handle_auth_done global_id 4553 payload 824 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 -- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64540089f0 msgr2=0x7f6478070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64540089f0 0x7f6478070860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 -- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 msgr2=0x7f647806fde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 0x7f647806fde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 monclient: found mon.noname-c 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:03.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f64777fe640 1 -- 10.20.193.78:0/3973315000 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6478072ec0 con 0x7f6454006d90 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f647c807640 10 monclient: get_auth_request con 0x7f64540089f0 auth_method 0 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.557+0000 7f647c807640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64540089f0 0x7f6478070860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f64777fe640 1 -- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454006d90 msgr2=0x7f6478070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f64777fe640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454006d90 0x7f6478070320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6468007a80 tx=0x7f64680031d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 1 -- 10.20.193.78:0/3973315000 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646801a020 con 0x7f6454006d90 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: dump: 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.570 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: _finish_auth 0 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 1 -- 10.20.193.78:0/3973315000 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468016870 con 0x7f6454006d90 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 1 -- 10.20.193.78:0/3973315000 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468016c40 con 0x7f6454006d90 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f6474ff9640 10 monclient: dump: 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.571 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f64767fc640 4 set_mon_vals no callback set 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f647e291640 5 monclient: authenticate success, global_id 4553 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f647e291640 10 monclient: _renew_subs 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f647e291640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6478073180 con 0x7f6454006d90 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f647e291640 10 monclient: _renew_subs 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f647e291640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.559+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64780a9f70 con 0x7f6454006d90 2026-02-17T05:49:03.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f6474ff9640 1 -- 10.20.193.78:0/3973315000 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f646801f400 con 0x7f6454006d90 2026-02-17T05:49:03.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f647e291640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:03.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f647e291640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:03.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f647e291640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64780a13d0 con 0x7f6454006d90 2026-02-17T05:49:03.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f6474ff9640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f644803df00 0x7f64480403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f6474ff9640 1 -- 10.20.193.78:0/3973315000 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(117..117 src has 1..117) ==== 4839+0+0 (secure 0 0 0) 0x7f6468053430 con 0x7f6454006d90 2026-02-17T05:49:03.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f6477fff640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f644803df00 0x7f64480403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.560+0000 7f6477fff640 10 monclient: get_auth_request con 0x7f644803df00 auth_method 0 2026-02-17T05:49:03.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.562+0000 7f6474ff9640 1 -- 10.20.193.78:0/3973315000 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6468024020 con 0x7f6454006d90 2026-02-17T05:49:03.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.562+0000 7f6474ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:03.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.562+0000 7f6474ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:03.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.562+0000 7f6477fff640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f644803df00 0x7f64480403c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6454006410 tx=0x7f6464035000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.674+0000 7f647e291640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:03.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.674+0000 7f647e291640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:03.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.674+0000 7f647e291640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.674+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6478071b80 con 0x7f6454006d90 2026-02-17T05:49:03.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f6474ff9640 1 -- 10.20.193.78:0/3973315000 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f6468016e20 con 0x7f6454006d90 2026-02-17T05:49:03.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f6474ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:03.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f6474ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:03.677 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:03.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f644803df00 msgr2=0x7f64480403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f644803df00 0x7f64480403c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6454006410 tx=0x7f6464035000 comp rx=0 tx=0).stop 2026-02-17T05:49:03.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 10 monclient: shutdown 2026-02-17T05:49:03.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454006d90 msgr2=0x7f6478070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454006d90 0x7f6478070320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6468007a80 tx=0x7f64680031d0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 shutdown_connections 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f644803df00 0x7f64480403c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64540089f0 0x7f6478070860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6454006d90 0x7f6478070320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 --2- 10.20.193.78:0/3973315000 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6454003d30 0x7f647806fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 >> 10.20.193.78:0/3973315000 conn(0x7f64780a7520 msgr2=0x7f647813cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.675+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 shutdown_connections 2026-02-17T05:49:03.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.676+0000 7f647e291640 1 -- 10.20.193.78:0/3973315000 wait complete. 2026-02-17T05:49:03.684 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:49:03.684 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:49:03.684 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:49:03.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.752+0000 7f79c98d3640 10 monclient: get_monmap_and_config 2026-02-17T05:49:03.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.752+0000 7f79c98d3640 10 monclient: build_initial_monmap 2026-02-17T05:49:03.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.752+0000 7f79c98d3640 10 monclient: monmap: 2026-02-17T05:49:03.755 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:03.755 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.755 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:03.752622+0000 2026-02-17T05:49:03.755 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:03.752622+0000 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.752+0000 7f79c98d3640 1 Processor -- start 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.752+0000 7f79c98d3640 1 -- start start 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.752+0000 7f79c98d3640 10 monclient: init 2026-02-17T05:49:03.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 0x7f79c406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient(hunting): picked mon.noname-a con 0x7f79c406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f79c409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c406d1b0 0x7f79c4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient(hunting): picked mon.noname-b con 0x7f79c406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c406daf0 con 0x7f79c409c4a0 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f79c40a0880 con 0x7f79c406c890 2026-02-17T05:49:03.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f79c409cc80 con 0x7f79c406d1b0 2026-02-17T05:49:03.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:03.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c98d3640 10 monclient(hunting): authenticate will time out at 789.306155s 2026-02-17T05:49:03.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 0x7f79c406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c406d1b0 0x7f79c4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 0x7f79c406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36888/0 (socket says 10.20.193.78:36888) 2026-02-17T05:49:03.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 1 -- 10.20.193.78:0/24599657 learned_addr learned my addr 10.20.193.78:0/24599657 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 10 monclient(hunting): get_auth_request con 0x7f79c406c890 auth_method 0 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient(hunting): get_auth_request con 0x7f79c406d1b0 auth_method 0 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 1 -- 10.20.193.78:0/24599657 >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 msgr2=0x7f79c406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:03.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 1 -- 10.20.193.78:0/24599657 >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 msgr2=0x7f79c406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 1 --2- 10.20.193.78:0/24599657 >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 0x7f79c406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c2ffd640 1 --2- 10.20.193.78:0/24599657 >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 0x7f79c406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient(hunting): handle_auth_done global_id 4556 payload 274 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 1 -- 10.20.193.78:0/24599657 >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 msgr2=0x7f79c406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 1 --2- 10.20.193.78:0/24599657 >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 0x7f79c406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 1 -- 10.20.193.78:0/24599657 >> v1:10.20.193.115:6789/0 conn(0x7f79c409c4a0 legacy=0x7f79c4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 1 monclient: found mon.noname-b 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:03.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.753+0000 7f79c37fe640 1 -- 10.20.193.78:0/24599657 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c409cf60 con 0x7f79c406d1b0 2026-02-17T05:49:03.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.755+0000 7f79c37fe640 1 -- 10.20.193.78:0/24599657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c406d1b0 msgr2=0x7f79c4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.755+0000 7f79c37fe640 1 --2- 10.20.193.78:0/24599657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c406d1b0 0x7f79c4135500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79c406dcc0 tx=0x7f79b402f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6fec5ec24f0b94c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 1 -- 10.20.193.78:0/24599657 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b403c070 con 0x7f79c406d1b0 2026-02-17T05:49:03.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:03.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: dump: 2026-02-17T05:49:03.770 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.770 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.770 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:03.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:03.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 5 monclient: authenticate success, global_id 4556 2026-02-17T05:49:03.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:03.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 1 -- 10.20.193.78:0/24599657 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b4037440 con 0x7f79c406d1b0 2026-02-17T05:49:03.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:03.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 1 -- 10.20.193.78:0/24599657 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4037810 con 0x7f79c406d1b0 2026-02-17T05:49:03.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:03.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 4 set_mon_vals no callback set 2026-02-17T05:49:03.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:03.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c1ffb640 10 monclient: dump: 2026-02-17T05:49:03.773 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.773 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.773 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.774 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.774 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.774 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.774 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.774 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.774 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.774 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 10 monclient: shutdown 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 1 -- 10.20.193.78:0/24599657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c406d1b0 msgr2=0x7f79c4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 1 --2- 10.20.193.78:0/24599657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c406d1b0 0x7f79c4135500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79c406dcc0 tx=0x7f79b402f150 comp rx=0 tx=0).stop 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 1 -- 10.20.193.78:0/24599657 shutdown_connections 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 1 --2- 10.20.193.78:0/24599657 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c406d1b0 0x7f79c4135500 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 1 --2- 10.20.193.78:0/24599657 >> v2:10.20.193.78:3300/0 conn(0x7f79c406c890 0x7f79c406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 1 -- 10.20.193.78:0/24599657 >> 10.20.193.78:0/24599657 conn(0x7f79c40a6620 msgr2=0x7f79c40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 1 -- 10.20.193.78:0/24599657 shutdown_connections 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.757+0000 7f79c98d3640 1 -- 10.20.193.78:0/24599657 wait complete. 2026-02-17T05:49:03.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient: build_initial_monmap 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient: monmap: 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 1 Processor -- start 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 1 -- start start 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient: init 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f79c406c890 0x7f79c41cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient(hunting): picked mon.noname-b con 0x7f79c406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c406d1b0 0x7f79c41cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient(hunting): picked mon.noname-a con 0x7f79c406d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c409c4a0 0x7f79c409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient(hunting): picked mon.noname-c con 0x7f79c409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79c40a01a0 con 0x7f79c406d1b0 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f79c40a0020 con 0x7f79c406c890 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f79c409be20 con 0x7f79c409c4a0 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.758+0000 7f79c98d3640 10 monclient(hunting): authenticate will time out at 789.311993s 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c409c4a0 0x7f79c409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c406d1b0 0x7f79c41cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c409c4a0 0x7f79c409db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39866/0 (socket says 10.20.193.78:39866) 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient(hunting): get_auth_request con 0x7f79c406d1b0 auth_method 0 2026-02-17T05:49:03.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 1 -- 10.20.193.78:0/2653521974 learned_addr learned my addr 10.20.193.78:0/2653521974 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 10 monclient(hunting): get_auth_request con 0x7f79c409c4a0 auth_method 0 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c2ffd640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f79c406c890 0x7f79c41cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c2ffd640 10 monclient(hunting): get_auth_request con 0x7f79c406c890 auth_method 0 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient(hunting): handle_auth_done global_id 54099 payload 824 2026-02-17T05:49:03.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c409c4a0 msgr2=0x7f79c409db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f79c409c4a0 0x7f79c409db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f79c406c890 msgr2=0x7f79c41cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f79c406c890 0x7f79c41cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient: _finish_hunting 0 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 1 monclient: found mon.noname-a 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.759+0000 7f79c27fc640 1 -- 10.20.193.78:0/2653521974 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c4129850 con 0x7f79c406d1b0 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c27fc640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c406d1b0 msgr2=0x7f79c41cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c27fc640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c406d1b0 0x7f79c41cfc50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00c450 tx=0x7f79ac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac016020 con 0x7f79c406d1b0 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: dump: 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.782 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: _finish_auth 0 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ac012a40 con 0x7f79c406d1b0 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c98d3640 5 monclient: authenticate success, global_id 54099 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c98d3640 10 monclient: _renew_subs 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c98d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c4136e30 con 0x7f79c406d1b0 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac011380 con 0x7f79c406d1b0 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f799bfff640 10 monclient: dump: 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c17fa640 4 set_mon_vals no callback set 2026-02-17T05:49:03.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c98d3640 10 monclient: _renew_subs 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c98d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.761+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c4129290 con 0x7f79c406d1b0 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c98d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c98d3640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e060 0x7f79c409e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c98d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4128fb0 con 0x7f79c409e060 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f79ac0115e0 con 0x7f79c406d1b0 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e060 0x7f79c409e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 10 monclient: get_auth_request con 0x7f79c409e060 auth_method 0 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 10 monclient(hunting): handle_auth_done global_id 54102 payload 824 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c2ffd640 1 -- 10.20.193.78:0/2653521974 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79c4074980 con 0x7f79c409e060 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f799bfff640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f799403deb0 0x7f7994040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 4839+0+0 (secure 0 0 0) 0x7f79ac052b80 con 0x7f79c406d1b0 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c37fe640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f799403deb0 0x7f7994040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.762+0000 7f79c37fe640 10 monclient: get_auth_request con 0x7f799403deb0 auth_method 0 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f79c2ffd640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e060 msgr2=0x7f79c409e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f79c2ffd640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e060 0x7f79c409e440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f79b800f550 tx=0x7f79b800fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8019020 con 0x7f79c409e060 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 10 monclient: dump: 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f79b8014b60 con 0x7f79c409e060 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e060 msgr2=0x7f79c409e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e060 0x7f79c409e440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f79b800f550 tx=0x7f79b800fa20 comp rx=0 tx=0).stop 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e060 msgr2=0x7f79c409e440 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.764+0000 7f799bfff640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e060 0x7f79c409e440 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f79b800f550 tx=0x7f79b800fa20 comp rx=0 tx=0).stop 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.765+0000 7f79c37fe640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f799403deb0 0x7f7994040370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79b402f190 tx=0x7f79b4033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c98d3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c98d3640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e980 0x7f79c41da430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c98d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79c406cc70 con 0x7f79c409e980 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e980 0x7f79c41da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 10 monclient: get_auth_request con 0x7f79c409e980 auth_method 0 2026-02-17T05:49:03.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 10 monclient(hunting): handle_auth_done global_id 54105 payload 824 2026-02-17T05:49:03.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.768+0000 7f79c2ffd640 1 -- 10.20.193.78:0/2653521974 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f79c4127e40 con 0x7f79c409e980 2026-02-17T05:49:03.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c2ffd640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e980 msgr2=0x7f79c41da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c2ffd640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e980 0x7f79c41da430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f79b800f550 tx=0x7f79b8004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8019060 con 0x7f79c409e980 2026-02-17T05:49:03.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:03.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 10 monclient: dump: 2026-02-17T05:49:03.791 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.791 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f79b8003230 con 0x7f79c409e980 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e980 msgr2=0x7f79c41da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e980 0x7f79c41da430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f79b800f550 tx=0x7f79b8004830 comp rx=0 tx=0).stop 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e980 msgr2=0x7f79c41da430 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:03.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f799bfff640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c409e980 0x7f79c41da430 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f79b800f550 tx=0x7f79b8004830 comp rx=0 tx=0).stop 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 34, 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 13244, 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:03.793 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:03.794 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.795 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:03.796 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.797 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:03.798 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:03.799 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.800 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:03.801 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:49:03.802 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.802 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.802 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.802 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:03.802 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:03.803 INFO:tasks.rados.rados.0.trial115.stdout:517: done (3 left) 2026-02-17T05:49:03.803 INFO:tasks.rados.rados.0.trial115.stdout:519: done (2 left) 2026-02-17T05:49:03.803 INFO:tasks.rados.rados.0.trial115.stdout:522: done (1 left) 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:523: done (0 left) 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: write oid 48 current snap is 46 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: writing trial11545420-48 from 615189 to 1070949 tid 1 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: writing trial11545420-48 from 1733883 to 2444961 tid 2 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: writing trial11545420-48 from 3134431 to 3594472 tid 3 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:525: delete oid 44 current snap is 46 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: finishing write tid 1 to trial11545420-48 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: finishing write tid 2 to trial11545420-48 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: finishing write tid 3 to trial11545420-48 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: finishing write tid 4 to trial11545420-48 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: finishing write tid 5 to trial11545420-48 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: oid 48 updating version 0 to 197 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: oid 48 version 197 is already newer than 196 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: oid 48 updating version 197 to 198 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: oid 48 version 198 is already newer than 195 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 198 (ObjNum 227 snap 46 seq_num 227) dirty exists 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: left oid 48 (ObjNum 227 snap 46 seq_num 227) 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:524: done (1 left) 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:525: done (0 left) 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:526: read oid 16 snap -1 2026-02-17T05:49:03.804 INFO:tasks.rados.rados.0.trial115.stdout:526: expect (ObjNum 206 snap 43 seq_num 206) 2026-02-17T05:49:03.805 INFO:tasks.rados.rados.0.trial115.stdout:527: write (excl) oid 45 current snap is 46 2026-02-17T05:49:03.805 INFO:tasks.rados.rados.0.trial115.stdout:527: seq_num 229 ranges {648803=722611,1967200=714747,3421983=426820} 2026-02-17T05:49:03.805 INFO:tasks.rados.rados.0.trial115.stdout:527: writing trial11545420-45 from 648803 to 1371414 tid 1 2026-02-17T05:49:03.805 INFO:tasks.rados.rados.0.trial115.stdout:527: writing trial11545420-45 from 1967200 to 2681947 tid 2 2026-02-17T05:49:03.805 INFO:tasks.rados.rados.0.trial115.stdout:527: writing trial11545420-45 from 3421983 to 3848803 tid 3 2026-02-17T05:49:03.805 INFO:tasks.rados.rados.0.trial115.stdout:528: snap_remove snap 45 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f799403deb0 msgr2=0x7f7994040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f799403deb0 0x7f7994040370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79b402f190 tx=0x7f79b4033000 comp rx=0 tx=0).stop 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 10 monclient: shutdown 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c406d1b0 msgr2=0x7f79c41cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 1 --2- 10.20.193.78:0/2653521974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f79c406d1b0 0x7f79c41cfc50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00c450 tx=0x7f79ac00c920 comp rx=0 tx=0).stop 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c37fe640 1 -- 10.20.193.78:0/2653521974 reap_dead start 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c37fe640 1 -- 10.20.193.78:0/2653521974 reap_dead start 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 shutdown_connections 2026-02-17T05:49:03.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 >> 10.20.193.78:0/2653521974 conn(0x7f79c40a6620 msgr2=0x7f79c40a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:03.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 shutdown_connections 2026-02-17T05:49:03.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.770+0000 7f79c98d3640 1 -- 10.20.193.78:0/2653521974 wait complete. 2026-02-17T05:49:03.806 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:03.806 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:49:03.806 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:49:03.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.873+0000 7fba80e3b640 10 monclient: get_monmap_and_config 2026-02-17T05:49:03.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.873+0000 7fba80e3b640 10 monclient: build_initial_monmap 2026-02-17T05:49:03.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.873+0000 7fba80e3b640 10 monclient: monmap: 2026-02-17T05:49:03.875 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:03.875 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.875 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:03.873709+0000 2026-02-17T05:49:03.875 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:03.873709+0000 2026-02-17T05:49:03.875 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.873+0000 7fba80e3b640 1 Processor -- start 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.873+0000 7fba80e3b640 1 -- start start 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.873+0000 7fba80e3b640 10 monclient: init 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient(hunting): picked mon.noname-b con 0x7fba7c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fba7c09c4a0 0x7fba7c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient(hunting): picked mon.noname-c con 0x7fba7c09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fba7c134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:03.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7c06daf0 con 0x7fba7c134df0 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fba7c0a0880 con 0x7fba7c09c4a0 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c06d6f0 con 0x7fba7c06c890 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba80e3b640 10 monclient(hunting): authenticate will time out at 789.427289s 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39882/0 (socket says 10.20.193.78:39882) 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 1 -- 10.20.193.78:0/1134559981 learned_addr learned my addr 10.20.193.78:0/1134559981 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient(hunting): get_auth_request con 0x7fba7c06c890 auth_method 0 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7ad76640 1 --1- 10.20.193.78:0/1134559981 >> v1:10.20.193.78:6789/0 conn(0x7fba7c134df0 0x7fba7c135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38574/0 (socket says 10.20.193.78:38574) 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba79d74640 1 --2- 10.20.193.78:0/1134559981 >> v2:10.20.193.115:3300/0 conn(0x7fba7c09c4a0 0x7fba7c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba79d74640 10 monclient(hunting): get_auth_request con 0x7fba7c09c4a0 auth_method 0 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba79d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba79d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba79d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient(hunting): handle_auth_done global_id 4562 payload 274 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 1 -- 10.20.193.78:0/1134559981 >> v2:10.20.193.115:3300/0 conn(0x7fba7c09c4a0 msgr2=0x7fba7c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 1 --2- 10.20.193.78:0/1134559981 >> v2:10.20.193.115:3300/0 conn(0x7fba7c09c4a0 0x7fba7c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 1 -- 10.20.193.78:0/1134559981 >> v1:10.20.193.78:6789/0 conn(0x7fba7c134df0 legacy=0x7fba7c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient: _finish_hunting 0 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 1 monclient: found mon.noname-b 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.874+0000 7fba7a575640 1 -- 10.20.193.78:0/1134559981 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c09cc80 con 0x7fba7c06c890 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba7a575640 1 -- 10.20.193.78:0/1134559981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 msgr2=0x7fba7c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba7a575640 1 --2- 10.20.193.78:0/1134559981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c06cc70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba70009970 tx=0x7fba7002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad65ab55a6130644 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 1 -- 10.20.193.78:0/1134559981 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7003c070 con 0x7fba7c06c890 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: dump: 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: _finish_auth 0 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: _check_auth_tickets 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 1 -- 10.20.193.78:0/1134559981 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7002fcd0 con 0x7fba7c06c890 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 5 monclient: authenticate success, global_id 4562 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 1 -- 10.20.193.78:0/1134559981 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70037660 con 0x7fba7c06c890 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba79573640 10 monclient: dump: 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.883 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 4 set_mon_vals no callback set 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 10 monclient: shutdown 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 1 -- 10.20.193.78:0/1134559981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 msgr2=0x7fba7c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 1 --2- 10.20.193.78:0/1134559981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c06cc70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba70009970 tx=0x7fba7002ee90 comp rx=0 tx=0).stop 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 1 -- 10.20.193.78:0/1134559981 shutdown_connections 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 1 --2- 10.20.193.78:0/1134559981 >> v2:10.20.193.115:3300/0 conn(0x7fba7c09c4a0 0x7fba7c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 1 --2- 10.20.193.78:0/1134559981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c06cc70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 1 -- 10.20.193.78:0/1134559981 >> 10.20.193.78:0/1134559981 conn(0x7fba7c0a6620 msgr2=0x7fba7c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 1 -- 10.20.193.78:0/1134559981 shutdown_connections 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.877+0000 7fba80e3b640 1 -- 10.20.193.78:0/1134559981 wait complete. 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient: build_initial_monmap 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient: monmap: 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 1 Processor -- start 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 1 -- start start 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient: init 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:03.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c1cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient(hunting): picked mon.noname-c con 0x7fba7c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba7c09c4a0 0x7fba7c122d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient(hunting): picked mon.noname-a con 0x7fba7c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba7c134df0 0x7fba7c123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient(hunting): picked mon.noname-b con 0x7fba7c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c0a01a0 con 0x7fba7c09c4a0 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c0a0020 con 0x7fba7c134df0 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c09be20 con 0x7fba7c06c890 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba80e3b640 10 monclient(hunting): authenticate will time out at 789.431607s 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c1cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c1cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39884/0 (socket says 10.20.193.78:39884) 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 -- 10.20.193.78:0/1511524667 learned_addr learned my addr 10.20.193.78:0/1511524667 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient(hunting): get_auth_request con 0x7fba7c06c890 auth_method 0 2026-02-17T05:49:03.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba79d74640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba7c09c4a0 0x7fba7c122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba79d74640 10 monclient(hunting): get_auth_request con 0x7fba7c09c4a0 auth_method 0 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba79d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba79d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba79d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient(hunting): handle_auth_done global_id 4565 payload 824 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba7c134df0 msgr2=0x7fba7c123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba7c134df0 0x7fba7c123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba7c09c4a0 msgr2=0x7fba7c122d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba7c09c4a0 0x7fba7c122d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient: _finish_hunting 0 2026-02-17T05:49:03.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 monclient: found mon.noname-c 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.878+0000 7fba7a575640 1 -- 10.20.193.78:0/1511524667 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c1266d0 con 0x7fba7c06c890 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.880+0000 7fba7a575640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 msgr2=0x7fba7c1cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.880+0000 7fba7a575640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c1cf710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fba70009970 tx=0x7fba70004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7003c040 con 0x7fba7c06c890 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: dump: 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: _finish_auth 0 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba70047070 con 0x7fba7c06c890 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70042890 con 0x7fba7c06c890 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba78d72640 4 set_mon_vals no callback set 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 10 monclient: dump: 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 5 monclient: authenticate success, global_id 4565 2026-02-17T05:49:03.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 10 monclient: _renew_subs 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c125500 con 0x7fba7c06c890 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 10 monclient: _renew_subs 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c125280 con 0x7fba7c06c890 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c1cfc50 0x7fba7c123780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c127690 con 0x7fba7c1cfc50 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c1cfc50 0x7fba7c123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 10 monclient: get_auth_request con 0x7fba7c1cfc50 auth_method 0 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fba70042a30 con 0x7fba7c06c890 2026-02-17T05:49:03.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 10 monclient(hunting): handle_auth_done global_id 4568 payload 824 2026-02-17T05:49:03.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba79d74640 1 -- 10.20.193.78:0/1511524667 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba7c074980 con 0x7fba7c1cfc50 2026-02-17T05:49:03.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba4803df00 0x7fba480403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.881+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 4767+0+0 (secure 0 0 0) 0x7fba70077bd0 con 0x7fba7c06c890 2026-02-17T05:49:03.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.882+0000 7fba7ad76640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba4803df00 0x7fba480403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.882+0000 7fba7ad76640 10 monclient: get_auth_request con 0x7fba4803df00 auth_method 0 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.883+0000 7fba79d74640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c1cfc50 msgr2=0x7fba7c123780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.883+0000 7fba79d74640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c1cfc50 0x7fba7c123780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1345b0 tx=0x7fba64011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.883+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6400f040 con 0x7fba7c1cfc50 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.883+0000 7fba62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 10 monclient: dump: 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fba6400fe20 con 0x7fba7c1cfc50 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c1cfc50 msgr2=0x7fba7c123780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c1cfc50 0x7fba7c123780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1345b0 tx=0x7fba64011ca0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c1cfc50 msgr2=0x7fba7c123780 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba62ffd640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c1cfc50 0x7fba7c123780 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1345b0 tx=0x7fba64011ca0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.884+0000 7fba7ad76640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba4803df00 0x7fba480403c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba7c126ea0 tx=0x7fba6c005140 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba80e3b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba80e3b640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c123cc0 0x7fba7c1da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba80e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c072660 con 0x7fba7c123cc0 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c123cc0 0x7fba7c1da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 10 monclient: get_auth_request con 0x7fba7c123cc0 auth_method 0 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 10 monclient(hunting): handle_auth_done global_id 4571 payload 824 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.887+0000 7fba79d74640 1 -- 10.20.193.78:0/1511524667 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fba7c1359e0 con 0x7fba7c123cc0 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba79d74640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c123cc0 msgr2=0x7fba7c1da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba79d74640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c123cc0 0x7fba7c1da3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1345b0 tx=0x7fba64004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64015460 con 0x7fba7c123cc0 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 10 monclient: dump: 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7fba64015460 con 0x7fba7c123cc0 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:03.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c123cc0 msgr2=0x7fba7c1da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c123cc0 0x7fba7c1da3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1345b0 tx=0x7fba64004830 comp rx=0 tx=0).stop 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c123cc0 msgr2=0x7fba7c1da3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba62ffd640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c123cc0 0x7fba7c1da3c0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1345b0 tx=0x7fba64004830 comp rx=0 tx=0).stop 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 34, 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 193693, 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:03.896 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:03.897 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:03.898 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:03.899 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.900 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.901 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:03.902 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:03.903 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:03.904 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:03.905 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:03.905 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba4803df00 msgr2=0x7fba480403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.889+0000 7fba80e3b640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba4803df00 0x7fba480403c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba7c126ea0 tx=0x7fba6c005140 comp rx=0 tx=0).stop 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba80e3b640 10 monclient: shutdown 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba80e3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 msgr2=0x7fba7c1cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba80e3b640 1 --2- 10.20.193.78:0/1511524667 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba7c06c890 0x7fba7c1cf710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fba70009970 tx=0x7fba70004830 comp rx=0 tx=0).stop 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba7ad76640 1 -- 10.20.193.78:0/1511524667 reap_dead start 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba7ad76640 1 -- 10.20.193.78:0/1511524667 reap_dead start 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 shutdown_connections 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 >> 10.20.193.78:0/1511524667 conn(0x7fba7c0a6620 msgr2=0x7fba7c135bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 shutdown_connections 2026-02-17T05:49:03.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.890+0000 7fba80e3b640 1 -- 10.20.193.78:0/1511524667 wait complete. 2026-02-17T05:49:03.907 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:03.907 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:49:03.907 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.974+0000 7f8da1551640 10 monclient: get_monmap_and_config 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.974+0000 7f8da1551640 10 monclient: build_initial_monmap 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.974+0000 7f8da1551640 10 monclient: monmap: 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:03.974758+0000 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:03.974758+0000 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:03.977 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.974+0000 7f8da1551640 1 Processor -- start 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.974+0000 7f8da1551640 1 -- start start 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.974+0000 7f8da1551640 10 monclient: init 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8d9c06c890 0x7f8d9c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient(hunting): picked mon.noname-c con 0x7f8d9c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8d9c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient(hunting): picked mon.noname-b con 0x7f8d9c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d9c06daf0 con 0x7f8d9c09c4a0 2026-02-17T05:49:03.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f8d9c0a0880 con 0x7f8d9c06c890 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9c09cc80 con 0x7f8d9c06d1b0 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8da1551640 10 monclient(hunting): authenticate will time out at 789.528341s 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39902/0 (socket says 10.20.193.78:39902) 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 1 -- 10.20.193.78:0/2239128366 learned_addr learned my addr 10.20.193.78:0/2239128366 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:03.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient(hunting): get_auth_request con 0x7f8d9c06d1b0 auth_method 0 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9affd640 1 --2- 10.20.193.78:0/2239128366 >> v2:10.20.193.115:3300/0 conn(0x7f8d9c06c890 0x7f8d9c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9affd640 10 monclient(hunting): get_auth_request con 0x7f8d9c06c890 auth_method 0 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient(hunting): handle_auth_done global_id 4574 payload 274 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 1 -- 10.20.193.78:0/2239128366 >> v2:10.20.193.115:3300/0 conn(0x7f8d9c06c890 msgr2=0x7f8d9c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 1 --2- 10.20.193.78:0/2239128366 >> v2:10.20.193.115:3300/0 conn(0x7f8d9c06c890 0x7f8d9c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 1 -- 10.20.193.78:0/2239128366 >> v1:10.20.193.115:6789/0 conn(0x7f8d9c09c4a0 legacy=0x7f8d9c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:03.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:03.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 1 monclient: found mon.noname-b 2026-02-17T05:49:03.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:03.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:03.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.975+0000 7f8d9b7fe640 1 -- 10.20.193.78:0/2239128366 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c09cf60 con 0x7f8d9c06d1b0 2026-02-17T05:49:03.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.977+0000 7f8d9b7fe640 1 -- 10.20.193.78:0/2239128366 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 msgr2=0x7f8d9c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:03.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.977+0000 7f8d9b7fe640 1 --2- 10.20.193.78:0/2239128366 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c135500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c06dcc0 tx=0x7f8d8c02f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903bbf91c44f75b2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:03.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 1 -- 10.20.193.78:0/2239128366 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c0373f0 con 0x7f8d9c06d1b0 2026-02-17T05:49:03.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: dump: 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.994 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 1 -- 10.20.193.78:0/2239128366 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d8c037590 con 0x7f8d9c06d1b0 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:03.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 1 -- 10.20.193.78:0/2239128366 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c037960 con 0x7f8d9c06d1b0 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 5 monclient: authenticate success, global_id 4574 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 4 set_mon_vals no callback set 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8d99ffb640 10 monclient: dump: 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.996 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 10 monclient: shutdown 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:03.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 1 -- 10.20.193.78:0/2239128366 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 msgr2=0x7f8d9c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 1 --2- 10.20.193.78:0/2239128366 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c135500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c06dcc0 tx=0x7f8d8c02f200 comp rx=0 tx=0).stop 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 1 -- 10.20.193.78:0/2239128366 shutdown_connections 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 1 --2- 10.20.193.78:0/2239128366 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c135500 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 1 --2- 10.20.193.78:0/2239128366 >> v2:10.20.193.115:3300/0 conn(0x7f8d9c06c890 0x7f8d9c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 1 -- 10.20.193.78:0/2239128366 >> 10.20.193.78:0/2239128366 conn(0x7f8d9c0a6620 msgr2=0x7f8d9c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 1 -- 10.20.193.78:0/2239128366 shutdown_connections 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 1 -- 10.20.193.78:0/2239128366 wait complete. 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 10 monclient: build_initial_monmap 2026-02-17T05:49:03.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.978+0000 7f8da1551640 10 monclient: monmap: 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:03.999 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 1 Processor -- start 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 1 -- start start 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient: init 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8d9c06c890 0x7f8d9c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient(hunting): picked mon.noname-a con 0x7f8d9c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient(hunting): picked mon.noname-c con 0x7f8d9c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09c4a0 0x7f8d9c09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient(hunting): picked mon.noname-b con 0x7f8d9c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c0a01a0 con 0x7f8d9c06c890 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c0a0020 con 0x7f8d9c09c4a0 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9c09be20 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8da1551640 10 monclient(hunting): authenticate will time out at 789.532501s 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8d9c06c890 0x7f8d9c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c1cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39912/0 (socket says 10.20.193.78:39912) 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 -- 10.20.193.78:0/1661155868 learned_addr learned my addr 10.20.193.78:0/1661155868 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient(hunting): get_auth_request con 0x7f8d9c06d1b0 auth_method 0 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:04.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9affd640 10 monclient(hunting): get_auth_request con 0x7f8d9c06c890 auth_method 0 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:04.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient(hunting): handle_auth_done global_id 4577 payload 824 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09c4a0 msgr2=0x7f8d9c09db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09c4a0 0x7f8d9c09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8d9c06c890 msgr2=0x7f8d9c1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8d9c06c890 0x7f8d9c1cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 monclient: found mon.noname-c 2026-02-17T05:49:04.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.979+0000 7f8d9a7fc640 1 -- 10.20.193.78:0/1661155868 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c06fd20 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d9a7fc640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 msgr2=0x7f8d9c1cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d9a7fc640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c1cf890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c5b0 tx=0x7f8d8400ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84016020 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: dump: 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:04.005 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:49:04.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d84012a40 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84012e30 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d997fa640 4 set_mon_vals no callback set 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8d7b7fe640 10 monclient: dump: 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:04.007 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8da1551640 5 monclient: authenticate success, global_id 4577 2026-02-17T05:49:04.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8da1551640 10 monclient: _renew_subs 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8da1551640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.981+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c128eb0 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8da1551640 10 monclient: _renew_subs 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8da1551640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c127fc0 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8da1551640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8da1551640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c136b60 0x7f8d9c09e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8da1551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c09d760 con 0x7f8d9c136b60 2026-02-17T05:49:04.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c136b60 0x7f8d9c09e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 10 monclient: get_auth_request con 0x7f8d9c136b60 auth_method 0 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8d8401b400 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 10 monclient(hunting): handle_auth_done global_id 44182 payload 824 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d9affd640 1 -- 10.20.193.78:0/1661155868 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d9c074980 con 0x7f8d9c136b60 2026-02-17T05:49:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d7b7fe640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d6803df00 0x7f8d680403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.982+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 4767+0+0 (secure 0 0 0) 0x7f8d84052ad0 con 0x7f8d9c06d1b0 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.983+0000 7f8d9b7fe640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d6803df00 0x7f8d680403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.983+0000 7f8d9b7fe640 10 monclient: get_auth_request con 0x7f8d6803df00 auth_method 0 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d9affd640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c136b60 msgr2=0x7f8d9c09e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d9affd640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c136b60 0x7f8d9c09e090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8d90011c00 tx=0x7f8d9000f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90019070 con 0x7f8d9c136b60 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 10 monclient: dump: 2026-02-17T05:49:04.011 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f8d9000fe80 con 0x7f8d9c136b60 2026-02-17T05:49:04.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c136b60 msgr2=0x7f8d9c09e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c136b60 0x7f8d9c09e090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8d90011c00 tx=0x7f8d9000f610 comp rx=0 tx=0).stop 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c136b60 msgr2=0x7f8d9c09e090 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.985+0000 7f8d7b7fe640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c136b60 0x7f8d9c09e090 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8d90011c00 tx=0x7f8d9000f610 comp rx=0 tx=0).stop 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.986+0000 7f8d9b7fe640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d6803df00 0x7f8d680403c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c09c980 tx=0x7f8d8c031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8da1551640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8da1551640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09e5d0 0x7f8d9c1da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8da1551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c0744d0 con 0x7f8d9c09e5d0 2026-02-17T05:49:04.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8d9affd640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09e5d0 0x7f8d9c1da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8d9affd640 10 monclient: get_auth_request con 0x7f8d9c09e5d0 auth_method 0 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8d9affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8d9affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8d9affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.988+0000 7f8d9affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.989+0000 7f8d9affd640 10 monclient(hunting): handle_auth_done global_id 44185 payload 824 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.989+0000 7f8d9affd640 1 -- 10.20.193.78:0/1661155868 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8d9c1cfdd0 con 0x7f8d9c09e5d0 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d9affd640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09e5d0 msgr2=0x7f8d9c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:04.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d9affd640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09e5d0 0x7f8d9c1da560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8d90011c00 tx=0x7f8d900047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90019040 con 0x7f8d9c09e5d0 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 10 monclient: dump: 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:04.015 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4874 (secure 0 0 0) 0x7f8d90003e10 con 0x7f8d9c09e5d0 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09e5d0 msgr2=0x7f8d9c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09e5d0 0x7f8d9c1da560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8d90011c00 tx=0x7f8d900047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09e5d0 msgr2=0x7f8d9c1da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:04.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.991+0000 7f8d7b7fe640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8d9c09e5d0 0x7f8d9c1da560 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8d90011c00 tx=0x7f8d900047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 34, 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 67639, 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:04.017 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:04.018 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:04.019 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:04.020 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:04.021 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.022 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:04.023 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:04.024 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:04.025 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.026 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:04.027 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.028 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:04.029 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d6803df00 msgr2=0x7f8d680403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d6803df00 0x7f8d680403c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c09c980 tx=0x7f8d8c031000 comp rx=0 tx=0).stop 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 10 monclient: shutdown 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 msgr2=0x7f8d9c1cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 1 --2- 10.20.193.78:0/1661155868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8d9c06d1b0 0x7f8d9c1cf890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c5b0 tx=0x7f8d8400ca80 comp rx=0 tx=0).stop 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8d9b7fe640 1 -- 10.20.193.78:0/1661155868 reap_dead start 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8d9b7fe640 1 -- 10.20.193.78:0/1661155868 reap_dead start 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 shutdown_connections 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 >> 10.20.193.78:0/1661155868 conn(0x7f8d9c0a6620 msgr2=0x7f8d9c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 shutdown_connections 2026-02-17T05:49:04.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:03.992+0000 7f8da1551640 1 -- 10.20.193.78:0/1661155868 wait complete. 2026-02-17T05:49:04.032 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:04.032 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:49:04.032 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.098+0000 7f5772761640 10 monclient: get_monmap_and_config 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.098+0000 7f5772761640 10 monclient: build_initial_monmap 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.098+0000 7f5772761640 10 monclient: monmap: 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:04.099197+0000 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:04.099197+0000 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.099 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:04.100 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:04.100 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:04.100 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:04.100 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:04.100 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 1 Processor -- start 2026-02-17T05:49:04.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 1 -- start start 2026-02-17T05:49:04.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient: init 2026-02-17T05:49:04.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f576c06c890 0x7f576c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient(hunting): picked mon.noname-a con 0x7f576c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c09c4a0 0x7f576c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient(hunting): picked mon.noname-b con 0x7f576c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f576c134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f576c06daf0 con 0x7f576c134df0 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f576c0a0880 con 0x7f576c06c890 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f576c06d6f0 con 0x7f576c09c4a0 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5772761640 10 monclient(hunting): authenticate will time out at 789.652695s 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c09c4a0 0x7f576c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576bfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f576c06c890 0x7f576c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c09c4a0 0x7f576c06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39916/0 (socket says 10.20.193.78:39916) 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 1 -- 10.20.193.78:0/788698731 learned_addr learned my addr 10.20.193.78:0/788698731 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:04.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient(hunting): get_auth_request con 0x7f576c09c4a0 auth_method 0 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f5770cd7640 1 --1- 10.20.193.78:0/788698731 >> v1:10.20.193.78:6789/0 conn(0x7f576c134df0 0x7f576c135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38586/0 (socket says 10.20.193.78:38586) 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576bfff640 10 monclient(hunting): get_auth_request con 0x7f576c06c890 auth_method 0 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient(hunting): handle_auth_done global_id 4580 payload 274 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 1 -- 10.20.193.78:0/788698731 >> v2:10.20.193.78:3300/0 conn(0x7f576c06c890 msgr2=0x7f576c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 1 --2- 10.20.193.78:0/788698731 >> v2:10.20.193.78:3300/0 conn(0x7f576c06c890 0x7f576c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 1 -- 10.20.193.78:0/788698731 >> v1:10.20.193.78:6789/0 conn(0x7f576c134df0 legacy=0x7f576c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:04.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 1 monclient: found mon.noname-b 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.099+0000 7f576b7fe640 1 -- 10.20.193.78:0/788698731 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c09cc80 con 0x7f576c09c4a0 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576b7fe640 1 -- 10.20.193.78:0/788698731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c09c4a0 msgr2=0x7f576c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576b7fe640 1 --2- 10.20.193.78:0/788698731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c09c4a0 0x7f576c06d1b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f576c06d870 tx=0x7f575c02ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf292793513e889 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 1 -- 10.20.193.78:0/788698731 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c03c070 con 0x7f576c09c4a0 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: dump: 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: _finish_auth 0 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 1 -- 10.20.193.78:0/788698731 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575c02fcc0 con 0x7f576c09c4a0 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 5 monclient: authenticate success, global_id 4580 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 1 -- 10.20.193.78:0/788698731 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c037660 con 0x7f576c09c4a0 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 4 set_mon_vals no callback set 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f576affd640 10 monclient: dump: 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 10 monclient: shutdown 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 1 -- 10.20.193.78:0/788698731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c09c4a0 msgr2=0x7f576c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 1 --2- 10.20.193.78:0/788698731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c09c4a0 0x7f576c06d1b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f576c06d870 tx=0x7f575c02ee80 comp rx=0 tx=0).stop 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 1 -- 10.20.193.78:0/788698731 shutdown_connections 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 1 --2- 10.20.193.78:0/788698731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c09c4a0 0x7f576c06d1b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 1 --2- 10.20.193.78:0/788698731 >> v2:10.20.193.78:3300/0 conn(0x7f576c06c890 0x7f576c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 1 -- 10.20.193.78:0/788698731 >> 10.20.193.78:0/788698731 conn(0x7f576c0a6620 msgr2=0x7f576c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:04.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 1 -- 10.20.193.78:0/788698731 shutdown_connections 2026-02-17T05:49:04.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.102+0000 7f5772761640 1 -- 10.20.193.78:0/788698731 wait complete. 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient: build_initial_monmap 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient: monmap: 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 1 Processor -- start 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 1 -- start start 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient: init 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576c06c890 0x7f576c1cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient(hunting): picked mon.noname-b con 0x7f576c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576c09c4a0 0x7f576c121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient(hunting): picked mon.noname-a con 0x7f576c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c134df0 0x7f576c121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient(hunting): picked mon.noname-c con 0x7f576c134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0a01a0 con 0x7f576c09c4a0 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0a0020 con 0x7f576c06c890 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f576c09be20 con 0x7f576c134df0 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5772761640 10 monclient(hunting): authenticate will time out at 789.656813s 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576c09c4a0 0x7f576c121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5770cd7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c134df0 0x7f576c121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576c09c4a0 0x7f576c121760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36942/0 (socket says 10.20.193.78:36942) 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 -- 10.20.193.78:0/4013110428 learned_addr learned my addr 10.20.193.78:0/4013110428 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient(hunting): get_auth_request con 0x7f576c09c4a0 auth_method 0 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5770cd7640 10 monclient(hunting): get_auth_request con 0x7f576c134df0 auth_method 0 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5770cd7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5770cd7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5770cd7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5770cd7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5770cd7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f5770cd7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:04.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient(hunting): handle_auth_done global_id 54117 payload 824 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576bfff640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576c06c890 0x7f576c1cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 -- 10.20.193.78:0/4013110428 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c134df0 msgr2=0x7f576c121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c134df0 0x7f576c121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 -- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576c06c890 msgr2=0x7f576c1cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576c06c890 0x7f576c1cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576b7fe640 1 -- 10.20.193.78:0/4013110428 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c1260d0 con 0x7f576c09c4a0 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576bfff640 10 monclient: get_auth_request con 0x7f576c06c890 auth_method 0 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.103+0000 7f576bfff640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576c06c890 0x7f576c1cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f576b7fe640 1 -- 10.20.193.78:0/4013110428 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576c09c4a0 msgr2=0x7f576c121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f576b7fe640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576c09c4a0 0x7f576c121760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f575c009a70 tx=0x7f575c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 1 -- 10.20.193.78:0/4013110428 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c047070 con 0x7f576c09c4a0 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: dump: 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:04.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: _finish_auth 0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 1 -- 10.20.193.78:0/4013110428 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575c03f7b0 con 0x7f576c09c4a0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 1 -- 10.20.193.78:0/4013110428 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c03c040 con 0x7f576c09c4a0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 10 monclient: dump: 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f576a7fc640 4 set_mon_vals no callback set 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 5 monclient: authenticate success, global_id 54117 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 10 monclient: _renew_subs 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c125410 con 0x7f576c09c4a0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 10 monclient: _renew_subs 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c072760 con 0x7f576c09c4a0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 1 -- 10.20.193.78:0/4013110428 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f575c03fba0 con 0x7f576c09c4a0 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:04.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576c074980 con 0x7f576c09c4a0 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573c03deb0 0x7f573c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.106+0000 7f5768ff9640 1 -- 10.20.193.78:0/4013110428 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4767+0+0 (secure 0 0 0) 0x7f575c077660 con 0x7f576c09c4a0 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.107+0000 7f576bfff640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573c03deb0 0x7f573c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.107+0000 7f576bfff640 10 monclient: get_auth_request con 0x7f573c03deb0 auth_method 0 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.109+0000 7f576bfff640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573c03deb0 0x7f573c040370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5758009b20 tx=0x7f5758008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.109+0000 7f5768ff9640 1 -- 10.20.193.78:0/4013110428 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f576c1260d0 con 0x7f576c09c4a0 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.109+0000 7f5768ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:04.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.109+0000 7f5768ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.230+0000 7f5772761640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:49:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.230+0000 7f5772761640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:49:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.230+0000 7f5772761640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.230+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f576c0a17b0 con 0x7f576c09c4a0 2026-02-17T05:49:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.231+0000 7f5768ff9640 1 -- 10.20.193.78:0/4013110428 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f575c076070 con 0x7f576c09c4a0 2026-02-17T05:49:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.231+0000 7f5768ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:49:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.231+0000 7f5768ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.231+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.231+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=1789523239,config=692619927,health=2157093694,kv=449989585,logm=549619699}) 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.231+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=991029821,config=692619927,health=1216614704,kv=449989585,logm=3031078878}) 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.231+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.231+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=1789523239,config=692619927,health=2157093694,kv=449989585,logm=549619699}) 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.231+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=11,config=1,health=12,kv=10,logm=66} crc {auth=3459990436,config=3678701417,health=2157093694,kv=565707359,logm=3114849139}) 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=475816255}) 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2202643592}) 2026-02-17T05:49:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=475816255}) 2026-02-17T05:49:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3134164454}) 2026-02-17T05:49:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573c03deb0 msgr2=0x7f573c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573c03deb0 0x7f573c040370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5758009b20 tx=0x7f5758008040 comp rx=0 tx=0).stop 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 10 monclient: shutdown 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576c09c4a0 msgr2=0x7f576c121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576c09c4a0 0x7f576c121760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f575c009a70 tx=0x7f575c004830 comp rx=0 tx=0).stop 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 shutdown_connections 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576c134df0 0x7f576c121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576c09c4a0 0x7f576c121760 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573c03deb0 0x7f573c040370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 --2- 10.20.193.78:0/4013110428 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576c06c890 0x7f576c1cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 >> 10.20.193.78:0/4013110428 conn(0x7f576c0a6620 msgr2=0x7f576c135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 shutdown_connections 2026-02-17T05:49:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:04.232+0000 7f5772761640 1 -- 10.20.193.78:0/4013110428 wait complete. 2026-02-17T05:49:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3138682129}) 2026-02-17T05:49:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3112736056}) 2026-02-17T05:49:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3138682129}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3578494012}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=855504938}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002679241}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=855504938}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.232+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3638580346}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=763596531}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2803592336}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=763596531}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1068785436}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2920022766}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3968571614}) 2026-02-17T05:49:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2920022766}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.233+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1384570232}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=974436992}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2687248728}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=974436992}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175264816}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189623102}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3336229607}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189623102}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=94,mdsmap=3,mgr=3} crc {logm=3958886430,mdsmap=1160443688,mgr=2195335647}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=20,osdmap=35} crc {mdsmap=1160443688,mgr=3487925938,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3673079771,mon_config_key=1253202243,monmap=1784110042,osd_metadata=1246192198,osd_pg_creating=1099985362,osd_snap=11539598,osdmap=2316637448}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.234+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=37} crc {mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3673079770,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=1099985362,osd_snap=880093294,osdmap=1006111846}) 2026-02-17T05:49:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=20,osdmap=35} crc {mdsmap=1160443688,mgr=3487925938,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3673079771,mon_config_key=1253202243,monmap=1784110042,osd_metadata=1246192198,osd_pg_creating=1099985362,osd_snap=11539598,osdmap=2316637448}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=6,osd_snap=18,osdmap=37} crc {mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3673079770,mon_config_key=1987855637,monmap=1784110042,osd_metadata=2997046373,osd_snap=4073734895,osdmap=2201075179}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2610345863}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4476834}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2610345863}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.235+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2839991539}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.236+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.236+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=50} crc {osdmap=1872094033}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.236+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=50} crc {osdmap=3621370133}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.236+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.236+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=50} crc {osdmap=1872094033}) 2026-02-17T05:49:04.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:04.236+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=50} crc {osdmap=401330112}) 2026-02-17T05:49:04.242 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:49:05.165 INFO:tasks.rados.rados.0.trial115.stdout:528: done (2 left) 2026-02-17T05:49:05.165 INFO:tasks.rados.rados.0.trial115.stdout:529: write (excl) oid 29 current snap is 46 2026-02-17T05:49:05.165 INFO:tasks.rados.rados.0.trial115.stdout:526: expect (ObjNum 206 snap 43 seq_num 206) 2026-02-17T05:49:05.243 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:49:05.243 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:05.271 INFO:tasks.rados.rados.0.trial115.stdout:527: finishing write tid 1 to trial11545420-45 2026-02-17T05:49:05.272 INFO:tasks.rados.rados.0.trial115.stdout:527: finishing write tid 2 to trial11545420-45 2026-02-17T05:49:05.272 INFO:tasks.rados.rados.0.trial115.stdout:529: seq_num 230 ranges {665610=797860,1909329=458405,2816700=410210} 2026-02-17T05:49:05.276 INFO:tasks.rados.rados.0.trial115.stdout:529: writing trial11545420-29 from 665610 to 1463470 tid 1 2026-02-17T05:49:05.280 INFO:tasks.rados.rados.0.trial115.stdout:529: writing trial11545420-29 from 1909329 to 2367734 tid 2 2026-02-17T05:49:05.283 INFO:tasks.rados.rados.0.trial115.stdout:529: writing trial11545420-29 from 2816700 to 3226910 tid 3 2026-02-17T05:49:05.284 INFO:tasks.rados.rados.0.trial115.stdout:526: done (2 left) 2026-02-17T05:49:05.284 INFO:tasks.rados.rados.0.trial115.stdout:530: write oid 5 current snap is 46 2026-02-17T05:49:05.284 INFO:tasks.rados.rados.0.trial115.stdout:527: finishing write tid 3 to trial11545420-45 2026-02-17T05:49:05.284 INFO:tasks.rados.rados.0.trial115.stdout:530: seq_num 231 ranges {682417=473109,1567811=602063,2844123=621482} 2026-02-17T05:49:05.287 INFO:tasks.rados.rados.0.trial115.stdout:530: writing trial11545420-5 from 682417 to 1155526 tid 1 2026-02-17T05:49:05.291 INFO:tasks.rados.rados.0.trial115.stdout:530: writing trial11545420-5 from 1567811 to 2169874 tid 2 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:530: writing trial11545420-5 from 2844123 to 3465605 tid 3 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:531: write (excl) oid 20 current snap is 46 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:527: finishing write tid 4 to trial11545420-45 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:527: finishing write tid 5 to trial11545420-45 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:527: oid 45 updating version 0 to 262 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:527: oid 45 version 262 is already newer than 259 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:527: oid 45 version 262 is already newer than 260 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:527: oid 45 version 262 is already newer than 261 2026-02-17T05:49:05.297 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 262 (ObjNum 229 snap 46 seq_num 229) dirty exists 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:527: left oid 45 (ObjNum 229 snap 46 seq_num 229) 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: finishing write tid 1 to trial11545420-29 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: finishing write tid 2 to trial11545420-29 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: finishing write tid 3 to trial11545420-29 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: finishing write tid 4 to trial11545420-29 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: finishing write tid 5 to trial11545420-29 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: oid 29 updating version 0 to 102 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: oid 29 version 102 is already newer than 101 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: oid 29 version 102 is already newer than 100 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: oid 29 version 102 is already newer than 99 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 102 (ObjNum 230 snap 46 seq_num 230) dirty exists 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:529: left oid 29 (ObjNum 230 snap 46 seq_num 230) 2026-02-17T05:49:05.298 INFO:tasks.rados.rados.0.trial115.stdout:530: finishing write tid 1 to trial11545420-5 2026-02-17T05:49:05.299 INFO:tasks.rados.rados.0.trial115.stdout:530: finishing write tid 2 to trial11545420-5 2026-02-17T05:49:05.299 INFO:tasks.rados.rados.0.trial115.stdout:531: seq_num 232 ranges {699224=548358,2026293=462074,3271546=432754} 2026-02-17T05:49:05.301 INFO:tasks.rados.rados.0.trial115.stdout:531: writing trial11545420-20 from 699224 to 1247582 tid 1 2026-02-17T05:49:05.305 INFO:tasks.rados.rados.0.trial115.stdout:531: writing trial11545420-20 from 2026293 to 2488367 tid 2 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:531: writing trial11545420-20 from 3271546 to 3704300 tid 3 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:527: done (3 left) 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:529: done (2 left) 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:532: read oid 50 snap -1 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:532: expect (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:533: read oid 33 snap -1 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:533: expect deleted 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:534: delete oid 12 current snap is 46 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:530: finishing write tid 3 to trial11545420-5 2026-02-17T05:49:05.311 INFO:tasks.rados.rados.0.trial115.stdout:530: finishing write tid 4 to trial11545420-5 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:530: finishing write tid 5 to trial11545420-5 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:530: oid 5 updating version 0 to 246 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:530: oid 5 version 246 is already newer than 245 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:530: oid 5 updating version 246 to 247 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:530: oid 5 version 247 is already newer than 244 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 247 (ObjNum 231 snap 46 seq_num 231) dirty exists 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:530: left oid 5 (ObjNum 231 snap 46 seq_num 231) 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:531: finishing write tid 1 to trial11545420-20 2026-02-17T05:49:05.312 INFO:tasks.rados.rados.0.trial115.stdout:531: finishing write tid 2 to trial11545420-20 2026-02-17T05:49:05.314 INFO:tasks.rados.rados.0.trial115.stdout:530: done (4 left) 2026-02-17T05:49:05.314 INFO:tasks.rados.rados.0.trial115.stdout:534: done (3 left) 2026-02-17T05:49:05.314 INFO:tasks.rados.rados.0.trial115.stdout:535: snap_create 2026-02-17T05:49:05.314 INFO:tasks.rados.rados.0.trial115.stdout:531: finishing write tid 3 to trial11545420-20 2026-02-17T05:49:05.314 INFO:tasks.rados.rados.0.trial115.stdout:531: finishing write tid 4 to trial11545420-20 2026-02-17T05:49:05.314 INFO:tasks.rados.rados.0.trial115.stdout:531: finishing write tid 5 to trial11545420-20 2026-02-17T05:49:05.314 INFO:tasks.rados.rados.0.trial115.stdout:531: oid 20 updating version 0 to 178 2026-02-17T05:49:05.315 INFO:tasks.rados.rados.0.trial115.stdout:531: oid 20 updating version 178 to 181 2026-02-17T05:49:05.315 INFO:tasks.rados.rados.0.trial115.stdout:531: oid 20 version 181 is already newer than 180 2026-02-17T05:49:05.315 INFO:tasks.rados.rados.0.trial115.stdout:531: oid 20 version 181 is already newer than 179 2026-02-17T05:49:05.315 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 181 (ObjNum 232 snap 46 seq_num 232) dirty exists 2026-02-17T05:49:05.315 INFO:tasks.rados.rados.0.trial115.stdout:531: left oid 20 (ObjNum 232 snap 46 seq_num 232) 2026-02-17T05:49:05.320 INFO:tasks.rados.rados.0.trial115.stdout:532: expect (ObjNum 160 snap 29 seq_num 160) 2026-02-17T05:49:05.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.312+0000 7fe546585640 10 monclient: get_monmap_and_config 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.312+0000 7fe546585640 10 monclient: build_initial_monmap 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.312+0000 7fe546585640 10 monclient: monmap: 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:05.312497+0000 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:05.312497+0000 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.325 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.312+0000 7fe546585640 1 Processor -- start 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.312+0000 7fe546585640 1 -- start start 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.312+0000 7fe546585640 10 monclient: init 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2026-02-17T05:49:05.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5400a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient(hunting): picked mon.noname-b con 0x7fe54009ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe54009ee70 0x7fe54009f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient(hunting): picked mon.noname-a con 0x7fe54009ee70 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe54009d980 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5400a0a20 con 0x7fe54009d980 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fe5400a0d00 con 0x7fe54009ee70 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400a0460 con 0x7fe54009ea90 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:05.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe546585640 10 monclient(hunting): authenticate will time out at 790.866185s 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5400a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5400a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39930/0 (socket says 10.20.193.78:39930) 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 1 -- 10.20.193.78:0/1742403041 learned_addr learned my addr 10.20.193.78:0/1742403041 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient(hunting): get_auth_request con 0x7fe54009ea90 auth_method 0 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53f7fe640 1 --2- 10.20.193.78:0/1742403041 >> v2:10.20.193.78:3300/0 conn(0x7fe54009ee70 0x7fe54009f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53f7fe640 10 monclient(hunting): get_auth_request con 0x7fe54009ee70 auth_method 0 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient(hunting): handle_auth_done global_id 4586 payload 274 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 1 -- 10.20.193.78:0/1742403041 >> v2:10.20.193.78:3300/0 conn(0x7fe54009ee70 msgr2=0x7fe54009f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:05.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 1 --2- 10.20.193.78:0/1742403041 >> v2:10.20.193.78:3300/0 conn(0x7fe54009ee70 0x7fe54009f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 1 -- 10.20.193.78:0/1742403041 >> v1:10.20.193.115:6789/0 conn(0x7fe54009d980 legacy=0x7fe54009ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 1 monclient: found mon.noname-b 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.313+0000 7fe53ffff640 1 -- 10.20.193.78:0/1742403041 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5400a0740 con 0x7fe54009ea90 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.315+0000 7fe53ffff640 1 -- 10.20.193.78:0/1742403041 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 msgr2=0x7fe5400a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.315+0000 7fe53ffff640 1 --2- 10.20.193.78:0/1742403041 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5400a13d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009970 tx=0x7fe52c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bfdeead088ae9f7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.315+0000 7fe53effd640 1 -- 10.20.193.78:0/1742403041 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c03c070 con 0x7fe54009ea90 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.315+0000 7fe53effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.315+0000 7fe53effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.315+0000 7fe53effd640 10 monclient: dump: 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.331 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 10 monclient: _finish_auth 0 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 1 -- 10.20.193.78:0/1742403041 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c02fcd0 con 0x7fe54009ea90 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 1 -- 10.20.193.78:0/1742403041 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c037660 con 0x7fe54009ea90 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe53effd640 10 monclient: dump: 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.332 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 5 monclient: authenticate success, global_id 4586 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 4 set_mon_vals no callback set 2026-02-17T05:49:05.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 10 monclient: shutdown 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 1 -- 10.20.193.78:0/1742403041 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 msgr2=0x7fe5400a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 1 --2- 10.20.193.78:0/1742403041 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5400a13d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009970 tx=0x7fe52c02ee90 comp rx=0 tx=0).stop 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 1 -- 10.20.193.78:0/1742403041 shutdown_connections 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 1 --2- 10.20.193.78:0/1742403041 >> v2:10.20.193.78:3300/0 conn(0x7fe54009ee70 0x7fe54009f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 1 --2- 10.20.193.78:0/1742403041 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5400a13d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 1 -- 10.20.193.78:0/1742403041 >> 10.20.193.78:0/1742403041 conn(0x7fe5400a7520 msgr2=0x7fe5400a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 1 -- 10.20.193.78:0/1742403041 shutdown_connections 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 1 -- 10.20.193.78:0/1742403041 wait complete. 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 10 monclient: build_initial_monmap 2026-02-17T05:49:05.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.316+0000 7fe546585640 10 monclient: monmap: 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 1 Processor -- start 2026-02-17T05:49:05.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 1 -- start start 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient: init 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe54009d980 0x7fe5401cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient(hunting): picked mon.noname-a con 0x7fe54009d980 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5401cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient(hunting): picked mon.noname-c con 0x7fe54009ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe54009ee70 0x7fe54012fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient(hunting): picked mon.noname-b con 0x7fe54009ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400a66a0 con 0x7fe54009d980 2026-02-17T05:49:05.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe54009f7d0 con 0x7fe54009ee70 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400a6820 con 0x7fe54009ea90 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe546585640 10 monclient(hunting): authenticate will time out at 790.870586s 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe54009d980 0x7fe5401cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5401cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe54009d980 0x7fe5401cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36960/0 (socket says 10.20.193.78:36960) 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 -- 10.20.193.78:0/1454276035 learned_addr learned my addr 10.20.193.78:0/1454276035 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 10 monclient(hunting): get_auth_request con 0x7fe54009ea90 auth_method 0 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient(hunting): get_auth_request con 0x7fe54009d980 auth_method 0 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe544afb640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe54009ee70 0x7fe54012fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient(hunting): handle_auth_done global_id 54123 payload 824 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 -- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 msgr2=0x7fe5401cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5401cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 -- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe54009ee70 msgr2=0x7fe54012fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe54009ee70 0x7fe54012fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 monclient: found mon.noname-a 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53ffff640 1 -- 10.20.193.78:0/1454276035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe540137d40 con 0x7fe54009d980 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe53f7fe640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5401cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe544afb640 10 monclient: get_auth_request con 0x7fe54009ee70 auth_method 0 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.317+0000 7fe544afb640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe54009ee70 0x7fe54012fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53ffff640 1 -- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe54009d980 msgr2=0x7fe5401cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53ffff640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe54009d980 0x7fe5401cf6b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009970 tx=0x7fe52c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 1 -- 10.20.193.78:0/1454276035 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c03c040 con 0x7fe54009d980 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: dump: 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: _finish_auth 0 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: _check_auth_tickets 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 1 -- 10.20.193.78:0/1454276035 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c047070 con 0x7fe54009d980 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 5 monclient: authenticate success, global_id 54123 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 10 monclient: _renew_subs 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe540132480 con 0x7fe54009d980 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 1 -- 10.20.193.78:0/1454276035 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c042890 con 0x7fe54009d980 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53e7fc640 4 set_mon_vals no callback set 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 10 monclient: dump: 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 10 monclient: _renew_subs 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5400a1c10 con 0x7fe54009d980 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 1 -- 10.20.193.78:0/1454276035 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe52c042a30 con 0x7fe54009d980 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5400a13d0 con 0x7fe54009d980 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe51003deb0 0x7fe510040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.320+0000 7fe53cff9640 1 -- 10.20.193.78:0/1454276035 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4795+0+0 (secure 0 0 0) 0x7fe52c077a10 con 0x7fe54009d980 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.321+0000 7fe53f7fe640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe51003deb0 0x7fe510040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.321+0000 7fe53f7fe640 10 monclient: get_auth_request con 0x7fe51003deb0 auth_method 0 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.323+0000 7fe53f7fe640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe51003deb0 0x7fe510040370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe540132690 tx=0x7fe530009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.323+0000 7fe53cff9640 1 -- 10.20.193.78:0/1454276035 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe540137d40 con 0x7fe54009d980 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.323+0000 7fe53cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.323+0000 7fe53cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:05.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.435+0000 7fe546585640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:05.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.435+0000 7fe546585640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:05.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.435+0000 7fe546585640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.435+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe5401d0130 con 0x7fe54009d980 2026-02-17T05:49:05.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe53cff9640 1 -- 10.20.193.78:0/1454276035 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fe52c035340 con 0x7fe54009d980 2026-02-17T05:49:05.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe53cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:05.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe53cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:05.439 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:05.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe51003deb0 msgr2=0x7fe510040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe546585640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe51003deb0 0x7fe510040370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe540132690 tx=0x7fe530009290 comp rx=0 tx=0).stop 2026-02-17T05:49:05.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe546585640 10 monclient: shutdown 2026-02-17T05:49:05.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe546585640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:05.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe54009d980 msgr2=0x7fe5401cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.436+0000 7fe546585640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe54009d980 0x7fe5401cf6b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009970 tx=0x7fe52c004830 comp rx=0 tx=0).stop 2026-02-17T05:49:05.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.437+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 shutdown_connections 2026-02-17T05:49:05.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.437+0000 7fe546585640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe51003deb0 0x7fe510040370 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.437+0000 7fe546585640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe54009ee70 0x7fe54012fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.437+0000 7fe546585640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe54009ea90 0x7fe5401cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.437+0000 7fe546585640 1 --2- 10.20.193.78:0/1454276035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe54009d980 0x7fe5401cf6b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.437+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 >> 10.20.193.78:0/1454276035 conn(0x7fe5400a7520 msgr2=0x7fe540138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:05.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.437+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 shutdown_connections 2026-02-17T05:49:05.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.437+0000 7fe546585640 1 -- 10.20.193.78:0/1454276035 wait complete. 2026-02-17T05:49:05.445 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:49:05.446 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:49:05.446 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:49:05.446 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:49:05.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.513+0000 7f7e2c90a640 10 monclient: get_monmap_and_config 2026-02-17T05:49:05.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.513+0000 7f7e2c90a640 10 monclient: build_initial_monmap 2026-02-17T05:49:05.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.513+0000 7f7e2c90a640 10 monclient: monmap: 2026-02-17T05:49:05.515 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:05.515 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.515 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:05.513668+0000 2026-02-17T05:49:05.515 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:05.513668+0000 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.513+0000 7f7e2c90a640 1 Processor -- start 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.513+0000 7f7e2c90a640 1 -- start start 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.513+0000 7f7e2c90a640 10 monclient: init 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:05.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7e2406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7e2406d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7e240a01f0 0x7f7e2413d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 10 monclient(hunting): picked mon.noname-a con 0x7f7e240a01f0 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e240a0600 con 0x7f7e2406d380 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e24074150 con 0x7f7e2406c890 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2409ef70 con 0x7f7e240a01f0 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2c90a640 10 monclient(hunting): authenticate will time out at 791.067222s 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7e240a01f0 0x7f7e2413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e29e7e640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f7e2406d380 0x7f7e2413a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38600/0 (socket says 10.20.193.78:38600) 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e29e7e640 1 -- 10.20.193.78:0/19823776 learned_addr learned my addr 10.20.193.78:0/19823776 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient(hunting): get_auth_request con 0x7f7e240a01f0 auth_method 0 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient(hunting): handle_auth_done global_id 54126 payload 274 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 1 -- 10.20.193.78:0/19823776 >> v1:10.20.193.115:6789/0 conn(0x7f7e2406c890 legacy=0x7f7e2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 1 -- 10.20.193.78:0/19823776 >> v1:10.20.193.78:6789/0 conn(0x7f7e2406d380 legacy=0x7f7e2413a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient: _finish_hunting 0 2026-02-17T05:49:05.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 1 monclient: found mon.noname-a 2026-02-17T05:49:05.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:05.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.514+0000 7f7e2ae80640 1 -- 10.20.193.78:0/19823776 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2409f250 con 0x7f7e240a01f0 2026-02-17T05:49:05.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2ae80640 1 -- 10.20.193.78:0/19823776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 msgr2=0x7f7e2413d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2ae80640 1 --2- 10.20.193.78:0/19823776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e2413d970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009d90 tx=0x7f7e1802f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc948df1c7ce318 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 1 -- 10.20.193.78:0/19823776 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1803c070 con 0x7f7e240a01f0 2026-02-17T05:49:05.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: dump: 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: _finish_auth 0 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: _check_auth_tickets 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 1 -- 10.20.193.78:0/19823776 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e18037640 con 0x7f7e240a01f0 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 1 -- 10.20.193.78:0/19823776 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18037a10 con 0x7f7e240a01f0 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2967d640 10 monclient: dump: 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.528 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 5 monclient: authenticate success, global_id 54126 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 4 set_mon_vals no callback set 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 10 monclient: shutdown 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 1 -- 10.20.193.78:0/19823776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 msgr2=0x7f7e2413d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 1 --2- 10.20.193.78:0/19823776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e2413d970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009d90 tx=0x7f7e1802f1e0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 1 -- 10.20.193.78:0/19823776 shutdown_connections 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 1 --2- 10.20.193.78:0/19823776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e2413d970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 1 -- 10.20.193.78:0/19823776 >> 10.20.193.78:0/19823776 conn(0x7f7e240a6620 msgr2=0x7f7e240a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 1 -- 10.20.193.78:0/19823776 shutdown_connections 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.517+0000 7f7e2c90a640 1 -- 10.20.193.78:0/19823776 wait complete. 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient: build_initial_monmap 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient: monmap: 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:05.530 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 1 Processor -- start 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 1 -- start start 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient: init 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7e2406c890 0x7f7e24072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient(hunting): picked mon.noname-c con 0x7f7e2406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:05.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7e2406d380 0x7f7e24133240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient(hunting): picked mon.noname-b con 0x7f7e2406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e24133780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient(hunting): picked mon.noname-a con 0x7f7e240a01f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2409d360 con 0x7f7e240a01f0 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2409d1e0 con 0x7f7e2406d380 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2413deb0 con 0x7f7e2406c890 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2c90a640 10 monclient(hunting): authenticate will time out at 791.071679s 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e24133780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7e2406c890 0x7f7e24072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e24133780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36970/0 (socket says 10.20.193.78:36970) 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7e2406c890 0x7f7e24072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39938/0 (socket says 10.20.193.78:39938) 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 -- 10.20.193.78:0/157040452 learned_addr learned my addr 10.20.193.78:0/157040452 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient(hunting): get_auth_request con 0x7f7e240a01f0 auth_method 0 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 10 monclient(hunting): get_auth_request con 0x7f7e2406c890 auth_method 0 2026-02-17T05:49:05.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient(hunting): handle_auth_done global_id 54129 payload 824 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7e2406c890 msgr2=0x7f7e24072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7e2406c890 0x7f7e24072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7e2406d380 msgr2=0x7f7e24133240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e29e7e640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7e2406d380 0x7f7e24133240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7e2406d380 0x7f7e24133240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient: _finish_hunting 0 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 monclient: found mon.noname-a 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e2ae80640 1 -- 10.20.193.78:0/157040452 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e241cf730 con 0x7f7e240a01f0 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e29e7e640 10 monclient: get_auth_request con 0x7f7e2406d380 auth_method 0 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.518+0000 7f7e29e7e640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7e2406d380 0x7f7e24133240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2ae80640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 msgr2=0x7f7e24133780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2ae80640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e24133780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009d90 tx=0x7f7e18004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18047020 con 0x7f7e240a01f0 2026-02-17T05:49:05.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: dump: 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.536 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: _finish_auth 0 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e18042680 con 0x7f7e240a01f0 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 5 monclient: authenticate success, global_id 54129 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 10 monclient: _renew_subs 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2411e9d0 con 0x7f7e240a01f0 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1803c040 con 0x7f7e240a01f0 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 10 monclient: dump: 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0ffff640 4 set_mon_vals no callback set 2026-02-17T05:49:05.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 10 monclient: _renew_subs 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2411fbf0 con 0x7f7e240a01f0 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411cc70 0x7f7e2411d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2406cc70 con 0x7f7e2411cc70 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411cc70 0x7f7e2411d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 10 monclient: get_auth_request con 0x7f7e2411cc70 auth_method 0 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_done global_id 54132 payload 824 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e2a67f640 1 -- 10.20.193.78:0/157040452 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e240008d0 con 0x7f7e2411cc70 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.521+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7e18042af0 con 0x7f7e240a01f0 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.522+0000 7f7e0effd640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7df403deb0 0x7f7df4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.522+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4795+0+0 (secure 0 0 0) 0x7f7e180786b0 con 0x7f7e240a01f0 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.522+0000 7f7e29e7e640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7df403deb0 0x7f7df4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.522+0000 7f7e29e7e640 10 monclient: get_auth_request con 0x7f7df403deb0 auth_method 0 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e2a67f640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411cc70 msgr2=0x7f7e2411d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e2a67f640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411cc70 0x7f7e2411d050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00f770 tx=0x7f7e1c00fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c018030 con 0x7f7e2411cc70 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 10 monclient: dump: 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.541 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7e1c00dbc0 con 0x7f7e2411cc70 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411cc70 msgr2=0x7f7e2411d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411cc70 0x7f7e2411d050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00f770 tx=0x7f7e1c00fc40 comp rx=0 tx=0).stop 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411cc70 msgr2=0x7f7e2411d050 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.524+0000 7f7e0effd640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411cc70 0x7f7e2411d050 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00f770 tx=0x7f7e1c00fc40 comp rx=0 tx=0).stop 2026-02-17T05:49:05.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.525+0000 7f7e29e7e640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7df403deb0 0x7f7df4040370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e2411de50 tx=0x7f7e10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.545 INFO:tasks.rados.rados.0.trial115.stdout:531: done (3 left) 2026-02-17T05:49:05.545 INFO:tasks.rados.rados.0.trial115.stdout:532: done (2 left) 2026-02-17T05:49:05.545 INFO:tasks.rados.rados.0.trial115.stdout:533: done (1 left) 2026-02-17T05:49:05.545 INFO:tasks.rados.rados.0.trial115.stdout:535: done (0 left) 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:536: copy_from oid 6 from oid 24 current snap is 47 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:537: delete oid 5 current snap is 47 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:537: done (1 left) 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:538: rollback oid 41 current snap is 47 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 46 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:539: read oid 23 snap -1 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:539: expect (ObjNum 99 snap 14 seq_num 99) 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:540: read oid 1 snap -1 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:540: expect (ObjNum 195 snap 36 seq_num 195) 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:541: read oid 38 snap -1 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:541: expect (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:542: copy_from oid 4 from oid 37 current snap is 47 2026-02-17T05:49:05.546 INFO:tasks.rados.rados.0.trial115.stdout:543: delete oid 11 current snap is 47 2026-02-17T05:49:05.547 INFO:tasks.rados.rados.0.trial115.stdout:543: done (6 left) 2026-02-17T05:49:05.547 INFO:tasks.rados.rados.0.trial115.stdout:544: delete oid 8 current snap is 47 2026-02-17T05:49:05.547 INFO:tasks.rados.rados.0.trial115.stdout:544: done (6 left) 2026-02-17T05:49:05.547 INFO:tasks.rados.rados.0.trial115.stdout:545: write oid 42 current snap is 47 2026-02-17T05:49:05.547 INFO:tasks.rados.rados.0.trial115.stdout:545: seq_num 237 ranges {783259=640956,1918703=613070,3008661=405467,3821939=161320} 2026-02-17T05:49:05.547 INFO:tasks.rados.rados.0.trial115.stdout:545: writing trial11545420-42 from 783259 to 1424215 tid 1 2026-02-17T05:49:05.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2c90a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:05.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2c90a640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411d590 0x7f7e241da460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2c90a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2411e460 con 0x7f7e2411d590 2026-02-17T05:49:05.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411d590 0x7f7e241da460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 10 monclient: get_auth_request con 0x7f7e2411d590 auth_method 0 2026-02-17T05:49:05.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 10 monclient(hunting): handle_auth_done global_id 54135 payload 824 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.527+0000 7f7e2a67f640 1 -- 10.20.193.78:0/157040452 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7e2411f350 con 0x7f7e2411d590 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.529+0000 7f7e2a67f640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411d590 msgr2=0x7f7e241da460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.529+0000 7f7e2a67f640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411d590 0x7f7e241da460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00f770 tx=0x7f7e1c003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c005260 con 0x7f7e2411d590 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 10 monclient: dump: 2026-02-17T05:49:05.548 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f7e1c005260 con 0x7f7e2411d590 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411d590 msgr2=0x7f7e241da460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411d590 0x7f7e241da460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00f770 tx=0x7f7e1c003860 comp rx=0 tx=0).stop 2026-02-17T05:49:05.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411d590 msgr2=0x7f7e241da460 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e0effd640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e2411d590 0x7f7e241da460 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00f770 tx=0x7f7e1c003860 comp rx=0 tx=0).stop 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 34, 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 15004, 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:05.550 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.551 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.552 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:05.553 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.554 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.555 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:05.556 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.557 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.558 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:05.559 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:05.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7df403deb0 msgr2=0x7f7df4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7df403deb0 0x7f7df4040370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e2411de50 tx=0x7f7e10009040 comp rx=0 tx=0).stop 2026-02-17T05:49:05.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 10 monclient: shutdown 2026-02-17T05:49:05.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:05.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 msgr2=0x7f7e24133780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e24133780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009d90 tx=0x7f7e18004b40 comp rx=0 tx=0).stop 2026-02-17T05:49:05.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2ae80640 1 -- 10.20.193.78:0/157040452 reap_dead start 2026-02-17T05:49:05.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 shutdown_connections 2026-02-17T05:49:05.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 --2- 10.20.193.78:0/157040452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7e240a01f0 0x7f7e24133780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 >> 10.20.193.78:0/157040452 conn(0x7f7e240a6620 msgr2=0x7f7e2413d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:05.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 shutdown_connections 2026-02-17T05:49:05.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.530+0000 7f7e2c90a640 1 -- 10.20.193.78:0/157040452 wait complete. 2026-02-17T05:49:05.561 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:05.561 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:49:05.561 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:49:05.562 INFO:tasks.rados.rados.0.trial115.stdout:545: writing trial11545420-42 from 1918703 to 2531773 tid 2 2026-02-17T05:49:05.562 INFO:tasks.rados.rados.0.trial115.stdout:545: writing trial11545420-42 from 3008661 to 3414128 tid 3 2026-02-17T05:49:05.562 INFO:tasks.rados.rados.0.trial115.stdout:545: writing trial11545420-42 from 3821939 to 3983259 tid 4 2026-02-17T05:49:05.562 INFO:tasks.rados.rados.0.trial115.stdout:546: copy_from oid 45 from oid 36 current snap is 47 2026-02-17T05:49:05.562 INFO:tasks.rados.rados.0.trial115.stdout:538: finishing rollback tid 0 to trial11545420-41 2026-02-17T05:49:05.562 INFO:tasks.rados.rados.0.trial115.stdout:538: finishing rollback tid 1 to trial11545420-41 2026-02-17T05:49:05.563 INFO:tasks.rados.rados.0.trial115.stdout:541: expect (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:49:05.629 INFO:tasks.rados.rados.0.trial115.stdout:536: finishing copy_from racing read to trial11545420-6 2026-02-17T05:49:05.629 INFO:tasks.rados.rados.0.trial115.stdout:540: expect (ObjNum 195 snap 36 seq_num 195) 2026-02-17T05:49:05.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient: get_monmap_and_config 2026-02-17T05:49:05.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient: build_initial_monmap 2026-02-17T05:49:05.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient: monmap: 2026-02-17T05:49:05.632 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:05.632 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.632 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:05.630425+0000 2026-02-17T05:49:05.632 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:05.630425+0000 2026-02-17T05:49:05.632 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 1 Processor -- start 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 1 -- start start 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient: init 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4e7c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient(hunting): picked mon.noname-b con 0x7f4e7c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4e7c06d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:05.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7c06daf0 con 0x7f4e7c06c890 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7c09cc80 con 0x7f4e7c06d380 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c0a0880 con 0x7f4e7c09c4a0 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.630+0000 7f4e82a0c640 10 monclient(hunting): authenticate will time out at 791.183976s 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7bfff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f4e7c06c890 0x7f4e7c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38614/0 (socket says 10.20.193.78:38614) 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7bfff640 1 -- 10.20.193.78:0/985688695 learned_addr learned my addr 10.20.193.78:0/985688695 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 1 --2- 10.20.193.78:0/985688695 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient(hunting): get_auth_request con 0x7f4e7c09c4a0 auth_method 0 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_done global_id 4595 payload 274 2026-02-17T05:49:05.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 1 -- 10.20.193.78:0/985688695 >> v1:10.20.193.115:6789/0 conn(0x7f4e7c06d380 legacy=0x7f4e7c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:05.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 1 -- 10.20.193.78:0/985688695 >> v1:10.20.193.78:6789/0 conn(0x7f4e7c06c890 legacy=0x7f4e7c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:05.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:05.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 1 monclient: found mon.noname-b 2026-02-17T05:49:05.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:05.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:05.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.631+0000 7f4e7b7fe640 1 -- 10.20.193.78:0/985688695 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c09cf60 con 0x7f4e7c09c4a0 2026-02-17T05:49:05.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7b7fe640 1 -- 10.20.193.78:0/985688695 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 msgr2=0x7f4e7c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7b7fe640 1 --2- 10.20.193.78:0/985688695 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c13b3a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0098d0 tx=0x7f4e6c02edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff2ba76d994f87ad server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 1 -- 10.20.193.78:0/985688695 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c03c070 con 0x7f4e7c09c4a0 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: dump: 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: _finish_auth 0 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 1 -- 10.20.193.78:0/985688695 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6c02fd20 con 0x7f4e7c09c4a0 2026-02-17T05:49:05.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 1 -- 10.20.193.78:0/985688695 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c037660 con 0x7f4e7c09c4a0 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e82a0c640 5 monclient: authenticate success, global_id 4595 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e82a0c640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e82a0c640 4 set_mon_vals no callback set 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.633+0000 7f4e7affd640 10 monclient: dump: 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 10 monclient: shutdown 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 1 -- 10.20.193.78:0/985688695 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 msgr2=0x7f4e7c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 1 --2- 10.20.193.78:0/985688695 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c13b3a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0098d0 tx=0x7f4e6c02edc0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 1 -- 10.20.193.78:0/985688695 shutdown_connections 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 1 --2- 10.20.193.78:0/985688695 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c13b3a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 1 -- 10.20.193.78:0/985688695 >> 10.20.193.78:0/985688695 conn(0x7f4e7c0a6620 msgr2=0x7f4e7c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 1 -- 10.20.193.78:0/985688695 shutdown_connections 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 1 -- 10.20.193.78:0/985688695 wait complete. 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 10 monclient: build_initial_monmap 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 10 monclient: monmap: 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.642 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.634+0000 7f4e82a0c640 1 Processor -- start 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 1 -- start start 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient: init 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7c06c890 0x7f4e7c1363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient(hunting): picked mon.noname-a con 0x7f4e7c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e7c06d380 0x7f4e7c136910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient(hunting): picked mon.noname-b con 0x7f4e7c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient(hunting): picked mon.noname-c con 0x7f4e7c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c09bfa0 con 0x7f4e7c06c890 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c09be20 con 0x7f4e7c06d380 2026-02-17T05:49:05.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c13f010 con 0x7f4e7c09c4a0 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e82a0c640 10 monclient(hunting): authenticate will time out at 791.188410s 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7c06c890 0x7f4e7c1363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7c06c890 0x7f4e7c1363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36990/0 (socket says 10.20.193.78:36990) 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 -- 10.20.193.78:0/3035304111 learned_addr learned my addr 10.20.193.78:0/3035304111 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient(hunting): get_auth_request con 0x7f4e7c06c890 auth_method 0 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 10 monclient(hunting): get_auth_request con 0x7f4e7c09c4a0 auth_method 0 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7b7fe640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e7c06d380 0x7f4e7c136910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7b7fe640 10 monclient(hunting): get_auth_request con 0x7f4e7c06d380 auth_method 0 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient(hunting): handle_auth_done global_id 54138 payload 824 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 msgr2=0x7f4e7c11cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c11cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e7c06d380 msgr2=0x7f4e7c136910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e7c06d380 0x7f4e7c136910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 monclient: found mon.noname-a 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e7bfff640 1 -- 10.20.193.78:0/3035304111 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c13e1d0 con 0x7f4e7c06c890 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.635+0000 7f4e80f82640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c09c4a0 0x7f4e7c11cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e7bfff640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7c06c890 msgr2=0x7f4e7c1363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e7bfff640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7c06c890 0x7f4e7c1363d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c780 tx=0x7f4e6800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68005de0 con 0x7f4e7c06c890 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: dump: 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: _finish_auth 0 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e82a0c640 5 monclient: authenticate success, global_id 54138 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e68007070 con 0x7f4e7c06c890 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68007440 con 0x7f4e7c06c890 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e7a7fc640 4 set_mon_vals no callback set 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e78ff9640 10 monclient: dump: 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.647 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e82a0c640 10 monclient: _renew_subs 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e82a0c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.637+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c06fd90 con 0x7f4e7c06c890 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e82a0c640 10 monclient: _renew_subs 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e82a0c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c11fd50 con 0x7f4e7c06c890 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e82a0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e82a0c640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c11d1b0 0x7f4e7c11d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e82a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c11dad0 con 0x7f4e7c11d1b0 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c11d1b0 0x7f4e7c11d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 10 monclient: get_auth_request con 0x7f4e7c11d1b0 auth_method 0 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_done global_id 4601 payload 824 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4e68007620 con 0x7f4e7c06c890 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e7b7fe640 1 -- 10.20.193.78:0/3035304111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e7c06d130 con 0x7f4e7c11d1b0 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e78ff9640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4c03deb0 0x7f4e4c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4795+0+0 (secure 0 0 0) 0x7f4e680525f0 con 0x7f4e7c06c890 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e80f82640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4c03deb0 0x7f4e4c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.638+0000 7f4e80f82640 10 monclient: get_auth_request con 0x7f4e4c03deb0 auth_method 0 2026-02-17T05:49:05.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.640+0000 7f4e7b7fe640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c11d1b0 msgr2=0x7f4e7c11d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.640+0000 7f4e7b7fe640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c11d1b0 0x7f4e7c11d590 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c039000 tx=0x7f4e6c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.640+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c047070 con 0x7f4e7c11d1b0 2026-02-17T05:49:05.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.640+0000 7f4e78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.640+0000 7f4e78ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:05.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.640+0000 7f4e78ff9640 10 monclient: dump: 2026-02-17T05:49:05.651 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.652 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.652 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.652 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.652 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.652 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.652 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.652 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.652 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.641+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4e6c04c450 con 0x7f4e7c11d1b0 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.641+0000 7f4e78ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.641+0000 7f4e78ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.641+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c11d1b0 msgr2=0x7f4e7c11d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.641+0000 7f4e78ff9640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c11d1b0 0x7f4e7c11d590 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c039000 tx=0x7f4e6c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.641+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c11d1b0 msgr2=0x7f4e7c11d590 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.641+0000 7f4e78ff9640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c11d1b0 0x7f4e7c11d590 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.641+0000 7f4e80f82640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4c03deb0 0x7f4e4c040370 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000d5e0 tx=0x7f4e70004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.643+0000 7f4e82a0c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e82a0c640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c139390 0x7f4e7c1d8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e82a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c06cc70 con 0x7f4e7c139390 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c139390 0x7f4e7c1d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 10 monclient: get_auth_request con 0x7f4e7c139390 auth_method 0 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 10 monclient(hunting): handle_auth_done global_id 4604 payload 824 2026-02-17T05:49:05.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.644+0000 7f4e7b7fe640 1 -- 10.20.193.78:0/3035304111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4e7c11f890 con 0x7f4e7c139390 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e7b7fe640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c139390 msgr2=0x7f4e7c1d8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e7b7fe640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c139390 0x7f4e7c1d8430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c002410 tx=0x7f4e6c008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c03c040 con 0x7f4e7c139390 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 10 monclient: dump: 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f4e6c04c810 con 0x7f4e7c139390 2026-02-17T05:49:05.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c139390 msgr2=0x7f4e7c1d8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c139390 0x7f4e7c1d8430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c002410 tx=0x7f4e6c008030 comp rx=0 tx=0).stop 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c139390 msgr2=0x7f4e7c1d8430 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e78ff9640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e7c139390 0x7f4e7c1d8430 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c002410 tx=0x7f4e6c008030 comp rx=0 tx=0).stop 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:05.656 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 34, 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 195450, 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.657 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.658 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:05.659 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.660 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:05.661 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.662 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.664 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.665 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.666 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:05.666 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4c03deb0 msgr2=0x7f4e4c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e82a0c640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4e4c03deb0 0x7f4e4c040370 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000d5e0 tx=0x7f4e70004200 comp rx=0 tx=0).stop 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e82a0c640 10 monclient: shutdown 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e82a0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7c06c890 msgr2=0x7f4e7c1363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e82a0c640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7c06c890 0x7f4e7c1363d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c780 tx=0x7f4e6800cc50 comp rx=0 tx=0).stop 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.646+0000 7f4e80f82640 1 -- 10.20.193.78:0/3035304111 reap_dead start 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.647+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 shutdown_connections 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.647+0000 7f4e82a0c640 1 --2- 10.20.193.78:0/3035304111 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e7c06c890 0x7f4e7c1363d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.647+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 >> 10.20.193.78:0/3035304111 conn(0x7f4e7c0a6620 msgr2=0x7f4e7c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.647+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 shutdown_connections 2026-02-17T05:49:05.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.647+0000 7f4e82a0c640 1 -- 10.20.193.78:0/3035304111 wait complete. 2026-02-17T05:49:05.668 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:05.668 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:49:05.668 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:49:05.733 INFO:tasks.rados.rados.0.trial115.stdout:539: expect (ObjNum 99 snap 14 seq_num 99) 2026-02-17T05:49:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.734+0000 7f130991b640 10 monclient: get_monmap_and_config 2026-02-17T05:49:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.734+0000 7f130991b640 10 monclient: build_initial_monmap 2026-02-17T05:49:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.734+0000 7f130991b640 10 monclient: monmap: 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:05.735102+0000 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:05.735102+0000 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:05.735 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 1 Processor -- start 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 1 -- start start 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient: init 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f130406c890 0x7f130406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient(hunting): picked mon.noname-a con 0x7f130406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f130409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f130406d1b0 0x7f1304135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient(hunting): picked mon.noname-c con 0x7f130406d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f130406daf0 con 0x7f130409c4a0 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f13040a0880 con 0x7f130406c890 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f130409cc80 con 0x7f130406d1b0 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f130991b640 10 monclient(hunting): authenticate will time out at 791.288628s 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f130406c890 0x7f130406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f13027fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f130409c4a0 0x7f1304134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38620/0 (socket says 10.20.193.78:38620) 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f13027fc640 1 -- 10.20.193.78:0/866670833 learned_addr learned my addr 10.20.193.78:0/866670833 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient(hunting): get_auth_request con 0x7f130406c890 auth_method 0 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f13037fe640 1 --2- 10.20.193.78:0/866670833 >> v2:10.20.193.115:3300/0 conn(0x7f130406d1b0 0x7f1304135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f13037fe640 10 monclient(hunting): get_auth_request con 0x7f130406d1b0 auth_method 0 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f13037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f13037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f13037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient(hunting): handle_auth_done global_id 54141 payload 274 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 1 -- 10.20.193.78:0/866670833 >> v2:10.20.193.115:3300/0 conn(0x7f130406d1b0 msgr2=0x7f1304135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 1 --2- 10.20.193.78:0/866670833 >> v2:10.20.193.115:3300/0 conn(0x7f130406d1b0 0x7f1304135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 1 -- 10.20.193.78:0/866670833 >> v1:10.20.193.78:6789/0 conn(0x7f130409c4a0 legacy=0x7f1304134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 1 monclient: found mon.noname-a 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f1302ffd640 1 -- 10.20.193.78:0/866670833 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130409cf60 con 0x7f130406c890 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.735+0000 7f13037fe640 1 --2- 10.20.193.78:0/866670833 >> v2:10.20.193.115:3300/0 conn(0x7f130406d1b0 0x7f1304135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1302ffd640 1 -- 10.20.193.78:0/866670833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 msgr2=0x7f130406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1302ffd640 1 --2- 10.20.193.78:0/866670833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 0x7f130406cc70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1304074de0 tx=0x7f12f802ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4898fe8bc67c47e3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 1 -- 10.20.193.78:0/866670833 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f803c070 con 0x7f130406c890 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: dump: 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 5 monclient: authenticate success, global_id 54141 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 1 -- 10.20.193.78:0/866670833 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f802fbd0 con 0x7f130406c890 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 1 -- 10.20.193.78:0/866670833 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8037440 con 0x7f130406c890 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 4 set_mon_vals no callback set 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f1301ffb640 10 monclient: dump: 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.743 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 10 monclient: shutdown 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 1 -- 10.20.193.78:0/866670833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 msgr2=0x7f130406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 1 --2- 10.20.193.78:0/866670833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 0x7f130406cc70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1304074de0 tx=0x7f12f802ed50 comp rx=0 tx=0).stop 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 1 -- 10.20.193.78:0/866670833 shutdown_connections 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 1 --2- 10.20.193.78:0/866670833 >> v2:10.20.193.115:3300/0 conn(0x7f130406d1b0 0x7f1304135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 1 --2- 10.20.193.78:0/866670833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 0x7f130406cc70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 1 -- 10.20.193.78:0/866670833 >> 10.20.193.78:0/866670833 conn(0x7f13040a6620 msgr2=0x7f13040a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 1 -- 10.20.193.78:0/866670833 shutdown_connections 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.738+0000 7f130991b640 1 -- 10.20.193.78:0/866670833 wait complete. 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient: build_initial_monmap 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient: monmap: 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 1 Processor -- start 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 1 -- start start 2026-02-17T05:49:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient: init 2026-02-17T05:49:05.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:05.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:49:05.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 0x7f13041cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient(hunting): picked mon.noname-a con 0x7f130406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f130406d1b0 0x7f1304124dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient(hunting): picked mon.noname-b con 0x7f130406d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f130409c4a0 0x7f1304125300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient(hunting): picked mon.noname-c con 0x7f130409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13040a01a0 con 0x7f130406c890 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13040a0020 con 0x7f130406d1b0 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f130409be20 con 0x7f130409c4a0 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f130991b640 10 monclient(hunting): authenticate will time out at 791.292750s 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 0x7f13041cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f130409c4a0 0x7f1304125300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 0x7f13041cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:37018/0 (socket says 10.20.193.78:37018) 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 -- 10.20.193.78:0/441425864 learned_addr learned my addr 10.20.193.78:0/441425864 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 10 monclient(hunting): get_auth_request con 0x7f130409c4a0 auth_method 0 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient(hunting): get_auth_request con 0x7f130406c890 auth_method 0 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13027fc640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f130406d1b0 0x7f1304124dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13027fc640 10 monclient(hunting): get_auth_request con 0x7f130406d1b0 auth_method 0 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13027fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient(hunting): handle_auth_done global_id 54144 payload 824 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f130409c4a0 msgr2=0x7f1304125300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f130409c4a0 0x7f1304125300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f130406d1b0 msgr2=0x7f1304124dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f130406d1b0 0x7f1304124dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 monclient: found mon.noname-a 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f1302ffd640 1 -- 10.20.193.78:0/441425864 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13041296e0 con 0x7f130406c890 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.739+0000 7f13037fe640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f130409c4a0 0x7f1304125300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f1302ffd640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 msgr2=0x7f13041cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f1302ffd640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 0x7f13041cf6a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12f8002410 tx=0x7f12f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f803c040 con 0x7f130406c890 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: dump: 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: _finish_auth 0 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f8047070 con 0x7f130406c890 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 5 monclient: authenticate success, global_id 54144 2026-02-17T05:49:05.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 10 monclient: _renew_subs 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1304128a20 con 0x7f130406c890 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8040a60 con 0x7f130406c890 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f13017fa640 4 set_mon_vals no callback set 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f12dbfff640 10 monclient: dump: 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.755 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 10 monclient: _renew_subs 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1304072f90 con 0x7f130406c890 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13041cfbe0 0x7f1304125840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1304074980 con 0x7f13041cfbe0 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f13027fc640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13041cfbe0 0x7f1304125840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f13027fc640 10 monclient: get_auth_request con 0x7f13041cfbe0 auth_method 0 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f13027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f13027fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.742+0000 7f13027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f12f803f030 con 0x7f130406c890 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f13027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f13027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f13027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f13027fc640 10 monclient(hunting): handle_auth_done global_id 44197 payload 824 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f12dbfff640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f12d403deb0 0x7f12d4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f13027fc640 1 -- 10.20.193.78:0/441425864 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f130406cc70 con 0x7f13041cfbe0 2026-02-17T05:49:05.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4795+0+0 (secure 0 0 0) 0x7f12f8077c00 con 0x7f130406c890 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f13037fe640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f12d403deb0 0x7f12d4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.743+0000 7f13037fe640 10 monclient: get_auth_request con 0x7f12d403deb0 auth_method 0 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f13027fc640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13041cfbe0 msgr2=0x7f1304125840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f13027fc640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13041cfbe0 0x7f1304125840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1304126840 tx=0x7f12ec00eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec005be0 con 0x7f13041cfbe0 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 10 monclient: dump: 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f12ec018070 con 0x7f13041cfbe0 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13041cfbe0 msgr2=0x7f1304125840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13041cfbe0 0x7f1304125840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1304126840 tx=0x7f12ec00eba0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13041cfbe0 msgr2=0x7f1304125840 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f12dbfff640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13041cfbe0 0x7f1304125840 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1304126840 tx=0x7f12ec00eba0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.745+0000 7f13037fe640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f12d403deb0 0x7f12d4040370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12f4003f80 tx=0x7f12f40097a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f130991b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f130991b640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 0x7f13041da350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f130991b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1304126190 con 0x7f1304125d80 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 0x7f13041da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 10 monclient: get_auth_request con 0x7f1304125d80 auth_method 0 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 msgr2=0x7f13041da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 msgr2=0x7f13041da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 0x7f13041da350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:05.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.749+0000 7f13027fc640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 0x7f13041da350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:538: finishing rollback tid 2 to trial11545420-41 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 183 (ObjNum 215 snap 45 seq_num 215) dirty exists 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:542: finishing copy_from racing read to trial11545420-4 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:536: finishing copy_from to trial11545420-6 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 247 (ObjNum 219 snap 45 seq_num 219) dirty exists 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:545: finishing write tid 1 to trial11545420-42 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:536: done (7 left) 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:538: done (6 left) 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:539: done (5 left) 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:540: done (4 left) 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:541: done (3 left) 2026-02-17T05:49:05.817 INFO:tasks.rados.rados.0.trial115.stdout:547: snap_remove snap 46 2026-02-17T05:49:05.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.949+0000 7f13027fc640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 0x7f13041da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:05.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.949+0000 7f13027fc640 10 monclient: get_auth_request con 0x7f1304125d80 auth_method 0 2026-02-17T05:49:05.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.949+0000 7f13027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.949+0000 7f13027fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.949+0000 7f13027fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.950+0000 7f13027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.950+0000 7f13027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.950+0000 7f13027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.950+0000 7f13027fc640 10 monclient(hunting): handle_auth_done global_id 44203 payload 824 2026-02-17T05:49:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.950+0000 7f13027fc640 1 -- 10.20.193.78:0/441425864 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f13041272f0 con 0x7f1304125d80 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.952+0000 7f13027fc640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 msgr2=0x7f13041da350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.952+0000 7f13027fc640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 0x7f13041da350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12ec017000 tx=0x7f12ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.952+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec01dde0 con 0x7f1304125d80 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.952+0000 7f12dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.952+0000 7f12dbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.952+0000 7f12dbfff640 10 monclient: dump: 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:05.953 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4874 (secure 0 0 0) 0x7f12ec01dde0 con 0x7f1304125d80 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f12dbfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f12dbfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 msgr2=0x7f13041da350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f12dbfff640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 0x7f13041da350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12ec017000 tx=0x7f12ec004a40 comp rx=0 tx=0).stop 2026-02-17T05:49:05.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f12dbfff640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 msgr2=0x7f13041da350 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:05.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f12dbfff640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1304125d80 0x7f13041da350 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12ec017000 tx=0x7f12ec004a40 comp rx=0 tx=0).stop 2026-02-17T05:49:05.955 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:05.955 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:05.955 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:05.955 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:05.955 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 34, 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 69600, 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:05.956 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.958 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.962 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:05.967 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.968 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:05.969 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:05.970 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:05.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f12d403deb0 msgr2=0x7f12d4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f12d403deb0 0x7f12d4040370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12f4003f80 tx=0x7f12f40097a0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 10 monclient: shutdown 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 msgr2=0x7f13041cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 1 --2- 10.20.193.78:0/441425864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f130406c890 0x7f13041cf6a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12f8002410 tx=0x7f12f80047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f13037fe640 1 -- 10.20.193.78:0/441425864 reap_dead start 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f13037fe640 1 -- 10.20.193.78:0/441425864 reap_dead start 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 shutdown_connections 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 >> 10.20.193.78:0/441425864 conn(0x7f13040a6620 msgr2=0x7f13040700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 shutdown_connections 2026-02-17T05:49:05.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:05.953+0000 7f130991b640 1 -- 10.20.193.78:0/441425864 wait complete. 2026-02-17T05:49:05.973 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:05.973 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-17T05:49:05.973 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:49:05.973 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-17T05:49:05.973 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T05:49:05.973 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-17T05:49:05.974 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:49:05.983 INFO:tasks.ceph.mon.c:Stopped 2026-02-17T05:49:05.983 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:49:05.983 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:06.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.091+0000 7fee9a0b7640 10 monclient: get_monmap_and_config 2026-02-17T05:49:06.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.091+0000 7fee9a0b7640 10 monclient: build_initial_monmap 2026-02-17T05:49:06.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.091+0000 7fee9a0b7640 10 monclient: monmap: 2026-02-17T05:49:06.094 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:06.094 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:06.094 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:06.091776+0000 2026-02-17T05:49:06.094 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:06.091776+0000 2026-02-17T05:49:06.094 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.091+0000 7fee9a0b7640 1 Processor -- start 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 1 -- start start 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient: init 2026-02-17T05:49:06.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fee9406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9409d430 0x7fee9413d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient(hunting): picked mon.noname-b con 0x7fee9409d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fee9409bf40 0x7fee9413fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient(hunting): picked mon.noname-c con 0x7fee9409bf40 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee94073600 con 0x7fee9406c890 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fee9409ca50 con 0x7fee9409bf40 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fee9406dab0 con 0x7fee9409d430 2026-02-17T05:49:06.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee9a0b7640 10 monclient(hunting): authenticate will time out at 791.645368s 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee92ffd640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9409d430 msgr2=0x7fee9413d5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee92ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9409d430 0x7fee9413d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee937fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fee9406c890 0x7fee9406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38634/0 (socket says 10.20.193.78:38634) 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee937fe640 1 -- 10.20.193.78:0/3475513030 learned_addr learned my addr 10.20.193.78:0/3475513030 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee93fff640 1 --2- 10.20.193.78:0/3475513030 >> v2:10.20.193.115:3300/0 conn(0x7fee9409bf40 0x7fee9413fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee93fff640 10 monclient(hunting): get_auth_request con 0x7fee9409bf40 auth_method 0 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee93fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee93fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:06.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.092+0000 7fee93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 10 monclient(hunting): handle_auth_done global_id 44206 payload 274 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 1 -- 10.20.193.78:0/3475513030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9409d430 msgr2=0x7fee9413d5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 1 --2- 10.20.193.78:0/3475513030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9409d430 0x7fee9413d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 1 -- 10.20.193.78:0/3475513030 >> v1:10.20.193.78:6789/0 conn(0x7fee9406c890 legacy=0x7fee9406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 1 monclient: found mon.noname-c 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:49:06.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.093+0000 7fee93fff640 1 -- 10.20.193.78:0/3475513030 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9409c770 con 0x7fee9409bf40 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee93fff640 1 -- 10.20.193.78:0/3475513030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 msgr2=0x7fee9413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee93fff640 1 --2- 10.20.193.78:0/3475513030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 0x7fee9413fb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee9409c670 tx=0x7fee8802f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19f76e9a9f1fa618 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 1 -- 10.20.193.78:0/3475513030 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8803c070 con 0x7fee9409bf40 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: dump: 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:06.105 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: _finish_auth 0 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: _check_auth_tickets 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 1 -- 10.20.193.78:0/3475513030 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88037640 con 0x7fee9409bf40 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee9a0b7640 5 monclient: authenticate success, global_id 44206 2026-02-17T05:49:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 1 -- 10.20.193.78:0/3475513030 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88037a10 con 0x7fee9409bf40 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee9a0b7640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee9a0b7640 4 set_mon_vals no callback set 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.095+0000 7fee927fc640 10 monclient: dump: 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:06.107 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 10 monclient: shutdown 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 1 -- 10.20.193.78:0/3475513030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 msgr2=0x7fee9413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/3475513030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 0x7fee9413fb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee9409c670 tx=0x7fee8802f150 comp rx=0 tx=0).stop 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 1 -- 10.20.193.78:0/3475513030 shutdown_connections 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/3475513030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 0x7fee9413fb20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/3475513030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9409d430 0x7fee9413d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 1 -- 10.20.193.78:0/3475513030 >> 10.20.193.78:0/3475513030 conn(0x7fee940a6620 msgr2=0x7fee940a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:06.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 1 -- 10.20.193.78:0/3475513030 shutdown_connections 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 1 -- 10.20.193.78:0/3475513030 wait complete. 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 10 monclient: build_initial_monmap 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.096+0000 7fee9a0b7640 10 monclient: monmap: 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:06.109 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 1 Processor -- start 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 1 -- start start 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient: init 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9406c890 0x7fee9413a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient(hunting): picked mon.noname-c con 0x7fee9406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 0x7fee9413a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:06.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient(hunting): picked mon.noname-b con 0x7fee9409bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 0x7fee9413ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient(hunting): picked mon.noname-a con 0x7fee9409d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee9406d500 con 0x7fee9409d430 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fee9406d380 con 0x7fee9409bf40 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fee9406d680 con 0x7fee9406c890 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee9a0b7640 10 monclient(hunting): authenticate will time out at 791.650360s 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee937fe640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9406c890 msgr2=0x7fee9413a260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee937fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9406c890 0x7fee9413a260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:06.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 0x7fee9413ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 0x7fee9413ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:37050/0 (socket says 10.20.193.78:37050) 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 -- 10.20.193.78:0/2823888472 learned_addr learned my addr 10.20.193.78:0/2823888472 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient(hunting): get_auth_request con 0x7fee9409d430 auth_method 0 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient(hunting): handle_auth_done global_id 54147 payload 824 2026-02-17T05:49:06.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 -- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9406c890 msgr2=0x7fee9413a260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9406c890 0x7fee9413a260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 -- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 msgr2=0x7fee9413a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 0x7fee9413a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 monclient: found mon.noname-a 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.097+0000 7fee93fff640 1 -- 10.20.193.78:0/2823888472 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9413e1f0 con 0x7fee9409d430 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee93fff640 1 -- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 msgr2=0x7fee9413ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee93fff640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 0x7fee9413ace0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee88009e90 tx=0x7fee88004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 1 -- 10.20.193.78:0/2823888472 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88047020 con 0x7fee9409d430 2026-02-17T05:49:06.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 10 monclient: dump: 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:06.114 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 10 monclient: _finish_auth 0 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee93fff640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 0x7fee9413ace0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee88009e90 tx=0x7fee88004b40 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fee8803c040 mon_map magic: 0 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee9a0b7640 5 monclient: authenticate success, global_id 54147 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee9a0b7640 10 monclient: _renew_subs 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee9a0b7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9413ca60 con 0x7fee9409d430 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 1 -- 10.20.193.78:0/2823888472 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88042880 con 0x7fee9409d430 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee6ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee91ffb640 4 set_mon_vals no callback set 2026-02-17T05:49:06.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee9a0b7640 10 monclient: _renew_subs 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee9a0b7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.099+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9413f560 con 0x7fee9409d430 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee6ffff640 1 -- 10.20.193.78:0/2823888472 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8803c040 con 0x7fee9409d430 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee6ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee6ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee6ffff640 10 monclient: dump: 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:06.116 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee6ffff640 1 -- 10.20.193.78:0/2823888472 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fee88042cb0 con 0x7fee9409d430 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee9a0b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee9a0b7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee9a0b7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9406cc70 con 0x7fee9409d430 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee6ffff640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee6403de60 0x7fee64040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee937fe640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee6403de60 0x7fee64040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.100+0000 7fee937fe640 10 monclient: get_auth_request con 0x7fee6403de60 auth_method 0 2026-02-17T05:49:06.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.103+0000 7fee937fe640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee6403de60 0x7fee64040320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee80004910 tx=0x7fee80007c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:06.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.103+0000 7fee6ffff640 1 -- 10.20.193.78:0/2823888472 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4795+0+0 (secure 0 0 0) 0x7fee88078cb0 con 0x7fee9409d430 2026-02-17T05:49:06.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.722+0000 7fee6ffff640 1 -- 10.20.193.78:0/2823888472 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fee88051020 con 0x7fee9409d430 2026-02-17T05:49:06.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.722+0000 7fee6ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:06.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.722+0000 7fee6ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.836+0000 7fee9a0b7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.836+0000 7fee9a0b7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.836+0000 7fee9a0b7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.836+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fee940744d0 con 0x7fee9409d430 2026-02-17T05:49:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.836+0000 7fee6ffff640 1 -- 10.20.193.78:0/2823888472 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fee88054da0 con 0x7fee9409d430 2026-02-17T05:49:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.836+0000 7fee6ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.836+0000 7fee6ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:06.838 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:06.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee6403de60 msgr2=0x7fee64040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:06.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee6403de60 0x7fee64040320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee80004910 tx=0x7fee80007c60 comp rx=0 tx=0).stop 2026-02-17T05:49:06.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 10 monclient: shutdown 2026-02-17T05:49:06.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:06.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 msgr2=0x7fee9413ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:06.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 0x7fee9413ace0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee88009e90 tx=0x7fee88004b40 comp rx=0 tx=0).stop 2026-02-17T05:49:06.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 shutdown_connections 2026-02-17T05:49:06.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fee9409d430 0x7fee9413ace0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fee6403de60 0x7fee64040320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fee9409bf40 0x7fee9413a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 --2- 10.20.193.78:0/2823888472 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fee9406c890 0x7fee9413a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:06.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 >> 10.20.193.78:0/2823888472 conn(0x7fee940a6620 msgr2=0x7fee94137720 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:06.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 shutdown_connections 2026-02-17T05:49:06.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:06.837+0000 7fee9a0b7640 1 -- 10.20.193.78:0/2823888472 wait complete. 2026-02-17T05:49:09.847 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:09.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 10 monclient: get_monmap_and_config 2026-02-17T05:49:09.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 10 monclient: build_initial_monmap 2026-02-17T05:49:09.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 10 monclient: monmap: 2026-02-17T05:49:09.918 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:09.918 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:09.918 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:09.916452+0000 2026-02-17T05:49:09.918 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:09.916452+0000 2026-02-17T05:49:09.918 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:09.918 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 1 Processor -- start 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 1 -- start start 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 10 monclient: init 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2026-02-17T05:49:09.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f34bc06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f34bc09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.916+0000 7f34c3923640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 0x7f34bc13ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c3923640 10 monclient(hunting): picked mon.noname-b con 0x7f34bc06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c3923640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34bc06daf0 con 0x7f34bc09c4a0 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c3923640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34bc09cc80 con 0x7f34bc06c890 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c3923640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c3923640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc0a0880 con 0x7f34bc06d380 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c3923640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c3923640 10 monclient(hunting): authenticate will time out at 795.470027s 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c1e99640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 msgr2=0x7f34bc13ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c1e99640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 0x7f34bc13ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:09.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c0e97640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f34bc09c4a0 0x7f34bc13b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52350/0 (socket says 10.20.193.78:52350) 2026-02-17T05:49:09.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.917+0000 7f34c0e97640 1 -- 10.20.193.78:0/876923679 learned_addr learned my addr 10.20.193.78:0/876923679 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:09.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c0e97640 1 -- 10.20.193.78:0/876923679 >> v1:10.20.193.78:6789/0 conn(0x7f34bc09c4a0 legacy=0x7f34bc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173055636 0 0) 0x7f34bc06daf0 con 0x7f34bc09c4a0 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): _negotiate 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): my global_id is 54150 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34a4003620 con 0x7f34bc09c4a0 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 29695664 0 0) 0x7f34a4003620 con 0x7f34bc09c4a0 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 msgr2=0x7f34bc13ead0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 1 -- 10.20.193.78:0/876923679 >> v1:10.20.193.115:6789/0 conn(0x7f34bc06c890 legacy=0x7f34bc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 0x7f34bc13ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:09.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 >> v1:10.20.193.115:6789/0 conn(0x7f34bc06c890 legacy=0x7f34bc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc09cf60 con 0x7f34bc09c4a0 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: _finish_auth 0 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f34b00031b0 con 0x7f34bc09c4a0 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c3923640 5 monclient: authenticate success, global_id 54150 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c3923640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:09.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: dump: 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:09.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 >> v1:10.20.193.78:6789/0 conn(0x7f34bc09c4a0 legacy=0x7f34bc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34a4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): picked mon.a con 0x7f34a4003b30 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34a4004900 0x7f34a4004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): picked mon.b con 0x7f34a4004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 0x7f34a400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): picked mon.c con 0x7f34bc06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34a400c000 con 0x7f34a4003b30 2026-02-17T05:49:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f34a400c180 con 0x7f34a4004900 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f34a400c300 con 0x7f34bc06d380 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34abfff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c3923640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1e99640 1 -- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 msgr2=0x7f34a400bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34a4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1e99640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 0x7f34a400bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient(hunting): get_auth_request con 0x7f34a4003b30 auth_method 0 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c0e97640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34a4004900 0x7f34a4004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient(hunting): handle_auth_done global_id 54150 payload 293 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 1 -- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 msgr2=0x7f34a400bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc06d380 0x7f34a400bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 1 -- 10.20.193.78:0/876923679 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34a4004900 msgr2=0x7f34a4004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34a4004900 0x7f34a4004d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient: _finish_hunting 0 2026-02-17T05:49:09.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 1 monclient: found mon.a 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 1 -- 10.20.193.78:0/876923679 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a4003620 con 0x7f34a4003b30 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient: _finish_auth 0 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 10 monclient: _check_auth_tickets 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.919+0000 7f34c1698640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c1698640 1 -- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 msgr2=0x7f34a4003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c1698640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34a4003f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0032a0 tx=0x7f34ac02efe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e470eed5d2e006ac server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac002fa0 con 0x7f34a4003b30 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:09.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 10 monclient: dump: 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ac033030 con 0x7f34a4003b30 2026-02-17T05:49:09.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 1 -- 10.20.193.78:0/876923679 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac005d20 con 0x7f34a4003b30 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 4 set_mon_vals no callback set 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34abfff640 10 monclient: dump: 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:09.939 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 10 monclient: shutdown 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 1 -- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 msgr2=0x7f34a4003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 1 --2- 10.20.193.78:0/876923679 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34a4003f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0032a0 tx=0x7f34ac02efe0 comp rx=0 tx=0).stop 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c1e99640 1 -- 10.20.193.78:0/876923679 reap_dead start 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 1 -- 10.20.193.78:0/876923679 shutdown_connections 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 1 -- 10.20.193.78:0/876923679 >> 10.20.193.78:0/876923679 conn(0x7f34bc0a6620 msgr2=0x7f34bc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:09.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 1 -- 10.20.193.78:0/876923679 shutdown_connections 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.922+0000 7f34c3923640 1 -- 10.20.193.78:0/876923679 wait complete. 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient: build_initial_monmap 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient: monmap: 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:09.941 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 1 Processor -- start 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 1 -- start start 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient: init 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34bc1ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient(hunting): picked mon.noname-a con 0x7f34a4003b30 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34bc131220 0x7f34bc131680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient(hunting): picked mon.noname-b con 0x7f34bc131220 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc1384c0 0x7f34bc138940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient(hunting): picked mon.noname-c con 0x7f34bc1384c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:09.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc09bfa0 con 0x7f34a4003b30 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc09be20 con 0x7f34bc131220 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc13cac0 con 0x7f34bc1384c0 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c3923640 10 monclient(hunting): authenticate will time out at 795.476865s 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1e99640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc1384c0 msgr2=0x7f34bc138940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1e99640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc1384c0 0x7f34bc138940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34bc1ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34bc1ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57502/0 (socket says 10.20.193.78:57502) 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 1 -- 10.20.193.78:0/3938186500 learned_addr learned my addr 10.20.193.78:0/3938186500 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:09.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 10 monclient(hunting): get_auth_request con 0x7f34a4003b30 auth_method 0 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c1698640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.923+0000 7f34c0e97640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34bc131220 0x7f34bc131680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c0e97640 10 monclient(hunting): get_auth_request con 0x7f34bc131220 auth_method 0 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c0e97640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c0e97640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c0e97640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:09.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 10 monclient(hunting): handle_auth_done global_id 54153 payload 824 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 1 -- 10.20.193.78:0/3938186500 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc1384c0 msgr2=0x7f34bc138940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc1384c0 0x7f34bc138940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 1 -- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34bc131220 msgr2=0x7f34bc131680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34bc131220 0x7f34bc131680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 10 monclient: _finish_hunting 0 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 1 monclient: found mon.noname-a 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.924+0000 7f34c1698640 1 -- 10.20.193.78:0/3938186500 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc13c170 con 0x7f34a4003b30 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c1698640 1 -- 10.20.193.78:0/3938186500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 msgr2=0x7f34bc1ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c1698640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34bc1ccda0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34ac002920 tx=0x7f34ac002cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 1 -- 10.20.193.78:0/3938186500 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac002fa0 con 0x7f34a4003b30 2026-02-17T05:49:09.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: dump: 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:09.946 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 1 -- 10.20.193.78:0/3938186500 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ac02fc20 con 0x7f34a4003b30 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 1 -- 10.20.193.78:0/3938186500 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac007070 con 0x7f34a4003b30 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34ab7fe640 4 set_mon_vals no callback set 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 10 monclient: dump: 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:09.947 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 5 monclient: authenticate success, global_id 54153 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 10 monclient: _renew_subs 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc13aa40 con 0x7f34a4003b30 2026-02-17T05:49:09.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 10 monclient: _renew_subs 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc06f6c0 con 0x7f34a4003b30 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc06cc70 con 0x7f34a4003b30 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.926+0000 7f34a9ffb640 1 -- 10.20.193.78:0/3938186500 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f34ac007230 con 0x7f34a4003b30 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.927+0000 7f34a9ffb640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f348c03de60 0x7f348c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.927+0000 7f34a9ffb640 1 -- 10.20.193.78:0/3938186500 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4795+0+0 (secure 0 0 0) 0x7f34ac076b70 con 0x7f34a4003b30 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.927+0000 7f34c0e97640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f348c03de60 0x7f348c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.927+0000 7f34c0e97640 10 monclient: get_auth_request con 0x7f348c03de60 auth_method 0 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.929+0000 7f34a9ffb640 1 -- 10.20.193.78:0/3938186500 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f34ac04b340 con 0x7f34a4003b30 2026-02-17T05:49:09.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.929+0000 7f34a9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:09.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.929+0000 7f34a9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:09.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:09.929+0000 7f34c0e97640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f348c03de60 0x7f348c040320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34b00036e0 tx=0x7f34bc09d130 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:10.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.042+0000 7f34c3923640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:10.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.042+0000 7f34c3923640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:10.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.042+0000 7f34c3923640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:10.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.042+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f34bc1cd4c0 con 0x7f34a4003b30 2026-02-17T05:49:10.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.042+0000 7f34a9ffb640 1 -- 10.20.193.78:0/3938186500 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f34ac0433c0 con 0x7f34a4003b30 2026-02-17T05:49:10.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.042+0000 7f34a9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:10.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.042+0000 7f34a9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:10.044 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:10.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f348c03de60 msgr2=0x7f348c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:10.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f348c03de60 0x7f348c040320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34b00036e0 tx=0x7f34bc09d130 comp rx=0 tx=0).stop 2026-02-17T05:49:10.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 10 monclient: shutdown 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 msgr2=0x7f34bc1ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34bc1ccda0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34ac002920 tx=0x7f34ac002cc0 comp rx=0 tx=0).stop 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 shutdown_connections 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f34bc1384c0 0x7f34bc138940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f348c03de60 0x7f348c040320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f34bc131220 0x7f34bc131680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 --2- 10.20.193.78:0/3938186500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34a4003b30 0x7f34bc1ccda0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 >> 10.20.193.78:0/3938186500 conn(0x7f34bc0a6620 msgr2=0x7f34bc072750 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 shutdown_connections 2026-02-17T05:49:10.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:10.043+0000 7f34c3923640 1 -- 10.20.193.78:0/3938186500 wait complete. 2026-02-17T05:49:13.051 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 10 monclient: get_monmap_and_config 2026-02-17T05:49:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 10 monclient: build_initial_monmap 2026-02-17T05:49:13.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 10 monclient: monmap: 2026-02-17T05:49:13.123 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:13.123 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:13.123 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:13.120431+0000 2026-02-17T05:49:13.123 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:13.120431+0000 2026-02-17T05:49:13.124 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:13.124 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:13.124 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:13.124 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:13.124 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:13.124 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:13.124 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:13.124 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 1 Processor -- start 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 1 -- start start 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 10 monclient: init 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa0a406c890 0x7fa0a406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 10 monclient(hunting): picked mon.noname-a con 0x7fa0a406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.120+0000 7fa0a9edc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa0a409c4a0 0x7fa0a406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 10 monclient(hunting): picked mon.noname-c con 0x7fa0a409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa0a4134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:13.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a406daf0 con 0x7fa0a4134df0 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a40a0880 con 0x7fa0a406c890 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a406d6f0 con 0x7fa0a409c4a0 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a9edc640 10 monclient(hunting): authenticate will time out at 798.674029s 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a37fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa0a406c890 0x7fa0a406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a37fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa0a406c890 0x7fa0a406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57514/0 (socket says 10.20.193.78:57514) 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a37fe640 1 -- 10.20.193.78:0/4108964265 learned_addr learned my addr 10.20.193.78:0/4108964265 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a3fff640 1 --1- 10.20.193.78:0/4108964265 >> v1:10.20.193.78:6789/0 conn(0x7fa0a4134df0 0x7fa0a4135280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52364/0 (socket says 10.20.193.78:52364) 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a37fe640 10 monclient(hunting): get_auth_request con 0x7fa0a406c890 auth_method 0 2026-02-17T05:49:13.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 1 --2- 10.20.193.78:0/4108964265 >> v2:10.20.193.115:3300/0 conn(0x7fa0a409c4a0 0x7fa0a406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient(hunting): get_auth_request con 0x7fa0a409c4a0 auth_method 0 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:13.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient(hunting): handle_auth_done global_id 44212 payload 274 2026-02-17T05:49:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 1 -- 10.20.193.78:0/4108964265 >> v2:10.20.193.78:3300/0 conn(0x7fa0a406c890 msgr2=0x7fa0a406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 1 --2- 10.20.193.78:0/4108964265 >> v2:10.20.193.78:3300/0 conn(0x7fa0a406c890 0x7fa0a406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 1 -- 10.20.193.78:0/4108964265 >> v1:10.20.193.78:6789/0 conn(0x7fa0a4134df0 legacy=0x7fa0a4135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:49:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 1 monclient: found mon.noname-c 2026-02-17T05:49:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:49:13.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.121+0000 7fa0a2ffd640 1 -- 10.20.193.78:0/4108964265 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a409cc80 con 0x7fa0a409c4a0 2026-02-17T05:49:13.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a2ffd640 1 -- 10.20.193.78:0/4108964265 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 msgr2=0x7fa0a406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a2ffd640 1 --2- 10.20.193.78:0/4108964265 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 0x7fa0a406d1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0a409c980 tx=0x7fa09402ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c22037689b6b45 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 1 -- 10.20.193.78:0/4108964265 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09403c070 con 0x7fa0a409c4a0 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: dump: 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:13.137 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: _finish_auth 0 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 1 -- 10.20.193.78:0/4108964265 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09402fbd0 con 0x7fa0a409c4a0 2026-02-17T05:49:13.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 5 monclient: authenticate success, global_id 44212 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 1 -- 10.20.193.78:0/4108964265 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094037440 con 0x7fa0a409c4a0 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a27fc640 10 monclient: dump: 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:13.139 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 4 set_mon_vals no callback set 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 10 monclient: shutdown 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:13.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 1 -- 10.20.193.78:0/4108964265 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 msgr2=0x7fa0a406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/4108964265 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 0x7fa0a406d1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0a409c980 tx=0x7fa09402ed50 comp rx=0 tx=0).stop 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 1 -- 10.20.193.78:0/4108964265 shutdown_connections 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/4108964265 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 0x7fa0a406d1b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/4108964265 >> v2:10.20.193.78:3300/0 conn(0x7fa0a406c890 0x7fa0a406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 1 -- 10.20.193.78:0/4108964265 >> 10.20.193.78:0/4108964265 conn(0x7fa0a40a6620 msgr2=0x7fa0a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 1 -- 10.20.193.78:0/4108964265 shutdown_connections 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.124+0000 7fa0a9edc640 1 -- 10.20.193.78:0/4108964265 wait complete. 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient: build_initial_monmap 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient: monmap: 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:13.141 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 1 Processor -- start 2026-02-17T05:49:13.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 1 -- start start 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient: init 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0a406c890 0x7fa0a41cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient(hunting): picked mon.noname-a con 0x7fa0a406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 0x7fa0a4133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient(hunting): picked mon.noname-b con 0x7fa0a409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0a4134df0 0x7fa0a41344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient(hunting): picked mon.noname-c con 0x7fa0a4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:13.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40a01a0 con 0x7fa0a406c890 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40a0020 con 0x7fa0a409c4a0 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a409be20 con 0x7fa0a4134df0 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a9edc640 10 monclient(hunting): authenticate will time out at 798.678519s 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a3fff640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0a4134df0 msgr2=0x7fa0a41344b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0a4134df0 0x7fa0a41344b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0a406c890 0x7fa0a41cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:13.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0a406c890 0x7fa0a41cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57522/0 (socket says 10.20.193.78:57522) 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 -- 10.20.193.78:0/2311523333 learned_addr learned my addr 10.20.193.78:0/2311523333 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient(hunting): get_auth_request con 0x7fa0a406c890 auth_method 0 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a2ffd640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 0x7fa0a4133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a2ffd640 10 monclient(hunting): get_auth_request con 0x7fa0a409c4a0 auth_method 0 2026-02-17T05:49:13.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient(hunting): handle_auth_done global_id 54159 payload 824 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 -- 10.20.193.78:0/2311523333 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0a4134df0 msgr2=0x7fa0a41344b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0a4134df0 0x7fa0a41344b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 -- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 msgr2=0x7fa0a4133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 0x7fa0a4133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:13.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.125+0000 7fa0a37fe640 1 -- 10.20.193.78:0/2311523333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40a1780 con 0x7fa0a406c890 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.127+0000 7fa0a37fe640 1 -- 10.20.193.78:0/2311523333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0a406c890 msgr2=0x7fa0a41cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.127+0000 7fa0a37fe640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0a406c890 0x7fa0a41cf430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa09000d800 tx=0x7fa09000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 1 -- 10.20.193.78:0/2311523333 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090018020 con 0x7fa0a406c890 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: dump: 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:13.147 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: _finish_auth 0 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:13.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 1 -- 10.20.193.78:0/2311523333 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09000bae0 con 0x7fa0a406c890 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 5 monclient: authenticate success, global_id 54159 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 10 monclient: _renew_subs 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a409d810 con 0x7fa0a406c890 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 1 -- 10.20.193.78:0/2311523333 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090014410 con 0x7fa0a406c890 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 10 monclient: dump: 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:13.149 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a1ffb640 4 set_mon_vals no callback set 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 10 monclient: _renew_subs 2026-02-17T05:49:13.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a409f7a0 con 0x7fa0a406c890 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a4074980 con 0x7fa0a406c890 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.128+0000 7fa07ffff640 1 -- 10.20.193.78:0/2311523333 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa090013050 con 0x7fa0a406c890 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.129+0000 7fa07ffff640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa07403de60 0x7fa074040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.129+0000 7fa07ffff640 1 -- 10.20.193.78:0/2311523333 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4795+0+0 (secure 0 0 0) 0x7fa090052b30 con 0x7fa0a406c890 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.129+0000 7fa0a2ffd640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa07403de60 0x7fa074040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:13.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.129+0000 7fa0a2ffd640 10 monclient: get_auth_request con 0x7fa07403de60 auth_method 0 2026-02-17T05:49:13.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.131+0000 7fa07ffff640 1 -- 10.20.193.78:0/2311523333 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa090022030 con 0x7fa0a406c890 2026-02-17T05:49:13.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.131+0000 7fa07ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:13.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.131+0000 7fa07ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:13.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.131+0000 7fa0a2ffd640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa07403de60 0x7fa074040320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4074de0 tx=0x7fa094033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:13.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.247+0000 7fa0a9edc640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:13.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.247+0000 7fa0a9edc640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:13.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.247+0000 7fa0a9edc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:13.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.247+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa0a40744d0 con 0x7fa0a406c890 2026-02-17T05:49:13.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.248+0000 7fa07ffff640 1 -- 10.20.193.78:0/2311523333 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa0900123e0 con 0x7fa0a406c890 2026-02-17T05:49:13.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.248+0000 7fa07ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:13.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.248+0000 7fa07ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:13.248 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:13.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa07403de60 msgr2=0x7fa074040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:13.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa07403de60 0x7fa074040320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4074de0 tx=0x7fa094033000 comp rx=0 tx=0).stop 2026-02-17T05:49:13.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 10 monclient: shutdown 2026-02-17T05:49:13.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:13.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0a406c890 msgr2=0x7fa0a41cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:13.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0a406c890 0x7fa0a41cf430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa09000d800 tx=0x7fa09000dcd0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 shutdown_connections 2026-02-17T05:49:13.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0a4134df0 0x7fa0a41344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa07403de60 0x7fa074040320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0a409c4a0 0x7fa0a4133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 --2- 10.20.193.78:0/2311523333 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0a406c890 0x7fa0a41cf430 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:13.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 >> 10.20.193.78:0/2311523333 conn(0x7fa0a40a6620 msgr2=0x7fa0a40a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:13.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 shutdown_connections 2026-02-17T05:49:13.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:13.249+0000 7fa0a9edc640 1 -- 10.20.193.78:0/2311523333 wait complete. 2026-02-17T05:49:16.258 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:16.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.327+0000 7f7583ae1640 10 monclient: get_monmap_and_config 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.327+0000 7f7583ae1640 10 monclient: build_initial_monmap 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.327+0000 7f7583ae1640 10 monclient: monmap: 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:16.329106+0000 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:16.329106+0000 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:16.331 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 1 Processor -- start 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 1 -- start start 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient: init 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 0x7f757c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient(hunting): picked mon.noname-b con 0x7f757c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:16.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f757c09d430 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient(hunting): picked mon.noname-a con 0x7f757c09bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c073600 con 0x7f757c09d430 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f757c09ca50 con 0x7f757c09bf40 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f757c06d6f0 con 0x7f757c06c890 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7583ae1640 10 monclient(hunting): authenticate will time out at 801.882721s 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7581856640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 msgr2=0x7f757c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7581856640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 0x7f757c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:16.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57530/0 (socket says 10.20.193.78:57530) 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 learned_addr learned my addr 10.20.193.78:0/4084526850 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 10 monclient(hunting): get_auth_request con 0x7f757c09bf40 auth_method 0 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:16.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.328+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:16.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.331+0000 7f7581055640 1 -- 10.20.193.78:0/4084526850 >> v1:10.20.193.115:6789/0 conn(0x7f757c09d430 legacy=0x7f757c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:16.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.528+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 msgr2=0x7f757c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:16.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.528+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 0x7f757c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 10 monclient(hunting): get_auth_request con 0x7f757c09bf40 auth_method 0 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:16.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.529+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:49:16.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 msgr2=0x7f757c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 0x7f757c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:49:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 10 monclient(hunting): get_auth_request con 0x7f757c09bf40 auth_method 0 2026-02-17T05:49:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:16.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:16.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:16.930+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:49:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 msgr2=0x7f757c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 0x7f757c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 10 monclient(hunting): get_auth_request con 0x7f757c09bf40 auth_method 0 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:17.731+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:49:19.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 10 monclient(hunting): tick 2026-02-17T05:49:19.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:49:19.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:49:19.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:49:19.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 msgr2=0x7f757c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f757c06c890 0x7f757c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 msgr2=0x7f757c06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.78:3300/0 conn(0x7f757c09bf40 0x7f757c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 -- 10.20.193.78:0/4084526850 >> v1:10.20.193.115:6789/0 conn(0x7f757c09d430 legacy=0x7f757c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 10 monclient: _add_conns ranks=[4,0] 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f7560003840 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7560006390 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 -- 10.20.193.78:0/4084526850 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c06dab0 con 0x7f7560006390 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 1 -- 10.20.193.78:0/4084526850 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f757c09ca50 con 0x7f7560003840 2026-02-17T05:49:19.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f756b7fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.328+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.329+0000 7f7581856640 10 monclient(hunting): get_auth_request con 0x7f7560003840 auth_method 0 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.329+0000 7f7581856640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.329+0000 7f7581856640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.329+0000 7f7581856640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.329+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 msgr2=0x7f7560005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.329+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 msgr2=0x7f7560005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.329+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:19.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.329+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:19.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.331+0000 7f7581055640 1 -- 10.20.193.78:0/4084526850 >> v1:10.20.193.78:6789/0 conn(0x7f7560006390 legacy=0x7f7560008850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:19.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:19.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 10 monclient(hunting): get_auth_request con 0x7f7560003840 auth_method 0 2026-02-17T05:49:19.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:19.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:19.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:19.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 msgr2=0x7f7560005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:19.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 msgr2=0x7f7560005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:19.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:19.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.530+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:49:19.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:19.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 10 monclient(hunting): get_auth_request con 0x7f7560003840 auth_method 0 2026-02-17T05:49:19.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:19.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:19.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:19.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 msgr2=0x7f7560005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:49:19.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 msgr2=0x7f7560005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:19.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:19.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:19.931+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:49:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.540+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2381844094 0 0) 0x7f757c06dab0 con 0x7f7560006390 2026-02-17T05:49:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.540+0000 7f7580854640 10 monclient(hunting): _negotiate 2026-02-17T05:49:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.540+0000 7f7580854640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.540+0000 7f7580854640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.540+0000 7f7580854640 10 monclient(hunting): my global_id is 54162 2026-02-17T05:49:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.540+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7564003620 con 0x7f7560006390 2026-02-17T05:49:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1237797419 0 0) 0x7f7564003620 con 0x7f7560006390 2026-02-17T05:49:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 msgr2=0x7f7560005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 --2- 10.20.193.78:0/4084526850 >> v2:10.20.193.115:3300/0 conn(0x7f7560003840 0x7f7560005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: _finish_hunting 0 2026-02-17T05:49:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756000b160 con 0x7f7560006390 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: _finish_auth 0 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: _check_auth_tickets 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f7570003210 con 0x7f7560006390 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: dump: 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 >> v1:10.20.193.78:6789/0 conn(0x7f7560006390 legacy=0x7f7560008850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 reap_dead start 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564003ab0 0x7f7564003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient(hunting): picked mon.b con 0x7f7564003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 0x7f7564004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient(hunting): picked mon.c con 0x7f7564004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564005390 0x7f756400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient(hunting): picked mon.a con 0x7f7564005390 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f757c09ca50 con 0x7f7564005390 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f756400c2e0 con 0x7f7564003ab0 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f756400c4e0 con 0x7f7564004900 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7580854640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7581055640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 msgr2=0x7f7564004d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7581055640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 0x7f7564004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564005390 0x7f756400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient(hunting): get_auth_request con 0x7f7564005390 auth_method 0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient(hunting): handle_auth_done global_id 54162 payload 293 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 msgr2=0x7f7564004d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 0x7f7564004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564003ab0 0x7f7564003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564003ab0 msgr2=0x7f7564003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564003ab0 0x7f7564003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient: _finish_hunting 0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 1 monclient: found mon.a 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7564003620 con 0x7f7564005390 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient: _finish_auth 0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 10 monclient: _check_auth_tickets 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7582057640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7583ae1640 5 monclient: authenticate success, global_id 54162 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7583ae1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7581856640 10 monclient: get_auth_request con 0x7f7564003ab0 auth_method 0 2026-02-17T05:49:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.541+0000 7f7581856640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564003ab0 0x7f7564003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7582057640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564005390 msgr2=0x7f756400bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7582057640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564005390 0x7f756400bac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f757c09c470 tx=0x7f757802fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0926240d29a1a0a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75780096d0 con 0x7f7564005390 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 10 monclient: dump: 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757803c070 con 0x7f7564005390 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 1 -- 10.20.193.78:0/4084526850 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578036800 con 0x7f7564005390 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7580854640 10 monclient: dump: 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 4 set_mon_vals no callback set 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 10 monclient: shutdown 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 -- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564005390 msgr2=0x7f756400bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564005390 0x7f756400bac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f757c09c470 tx=0x7f757802fa90 comp rx=0 tx=0).stop 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 -- 10.20.193.78:0/4084526850 shutdown_connections 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564005390 0x7f756400bac0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 0x7f7564004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 --2- 10.20.193.78:0/4084526850 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564003ab0 0x7f7564003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 -- 10.20.193.78:0/4084526850 >> 10.20.193.78:0/4084526850 conn(0x7f757c0a6620 msgr2=0x7f757c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 -- 10.20.193.78:0/4084526850 shutdown_connections 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.544+0000 7f7583ae1640 1 -- 10.20.193.78:0/4084526850 wait complete. 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient: build_initial_monmap 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient: monmap: 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 1 Processor -- start 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 1 -- start start 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient: init 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564003ab0 0x7f757c1cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient(hunting): picked mon.noname-a con 0x7f7564003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 0x7f757c1354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient(hunting): picked mon.noname-c con 0x7f7564004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564005390 0x7f757c1ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient(hunting): picked mon.noname-b con 0x7f7564005390 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f757c073380 con 0x7f7564003ab0 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f757c073200 con 0x7f7564005390 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f757c06cc70 con 0x7f7564004900 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7583ae1640 10 monclient(hunting): authenticate will time out at 806.099912s 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581055640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 msgr2=0x7f757c1354a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581055640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 0x7f757c1354a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564003ab0 0x7f757c1cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564003ab0 0x7f757c1cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55450/0 (socket says 10.20.193.78:55450) 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 1 -- 10.20.193.78:0/470893559 learned_addr learned my addr 10.20.193.78:0/470893559 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 10 monclient(hunting): get_auth_request con 0x7f7564003ab0 auth_method 0 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.545+0000 7f7581856640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 10 monclient(hunting): handle_auth_done global_id 54165 payload 824 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 1 -- 10.20.193.78:0/470893559 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 msgr2=0x7f757c1354a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 0x7f757c1354a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 1 -- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564005390 msgr2=0x7f757c1ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564005390 0x7f757c1ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 10 monclient: _finish_hunting 0 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 1 monclient: found mon.noname-a 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.546+0000 7f7581856640 1 -- 10.20.193.78:0/470893559 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c121b80 con 0x7f7564003ab0 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7581856640 1 -- 10.20.193.78:0/470893559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564003ab0 msgr2=0x7f757c1cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7581856640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564003ab0 0x7f757c1cb470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f756c003990 tx=0x7f756c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 1 -- 10.20.193.78:0/470893559 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c016610 con 0x7f7564003ab0 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: dump: 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.549 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 1 -- 10.20.193.78:0/470893559 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c01b070 con 0x7f7564003ab0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 5 monclient: authenticate success, global_id 54165 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 10 monclient: _renew_subs 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757c121390 con 0x7f7564003ab0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 1 -- 10.20.193.78:0/470893559 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0169d0 con 0x7f7564003ab0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756bfff640 4 set_mon_vals no callback set 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 10 monclient: dump: 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 10 monclient: _renew_subs 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757c1235d0 con 0x7f7564003ab0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c074980 con 0x7f7564003ab0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.548+0000 7f756a7fc640 1 -- 10.20.193.78:0/470893559 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f756c016b70 con 0x7f7564003ab0 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.549+0000 7f756a7fc640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f754c03deb0 0x7f754c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.549+0000 7f756a7fc640 1 -- 10.20.193.78:0/470893559 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4795+0+0 (secure 0 0 0) 0x7f756c053460 con 0x7f7564003ab0 2026-02-17T05:49:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.549+0000 7f7581055640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f754c03deb0 0x7f754c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.549+0000 7f7581055640 10 monclient: get_auth_request con 0x7f754c03deb0 auth_method 0 2026-02-17T05:49:20.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.551+0000 7f756a7fc640 1 -- 10.20.193.78:0/470893559 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f756c005c30 con 0x7f7564003ab0 2026-02-17T05:49:20.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.551+0000 7f756a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:20.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.551+0000 7f756a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:20.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.551+0000 7f7581055640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f754c03deb0 0x7f754c040370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75700036c0 tx=0x7f7570003610 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.664+0000 7f7583ae1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:20.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.664+0000 7f7583ae1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:20.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.664+0000 7f7583ae1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.664+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f757c06d000 con 0x7f7564003ab0 2026-02-17T05:49:20.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.664+0000 7f756a7fc640 1 -- 10.20.193.78:0/470893559 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f756c014d50 con 0x7f7564003ab0 2026-02-17T05:49:20.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.664+0000 7f756a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:20.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.664+0000 7f756a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:20.667 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":38,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:20.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f754c03deb0 msgr2=0x7f754c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f754c03deb0 0x7f754c040370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75700036c0 tx=0x7f7570003610 comp rx=0 tx=0).stop 2026-02-17T05:49:20.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 10 monclient: shutdown 2026-02-17T05:49:20.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:20.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564003ab0 msgr2=0x7f757c1cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564003ab0 0x7f757c1cb470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f756c003990 tx=0x7f756c0050b0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 shutdown_connections 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f754c03deb0 0x7f754c040370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7564005390 0x7f757c1ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7564004900 0x7f757c1354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 --2- 10.20.193.78:0/470893559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7564003ab0 0x7f757c1cb470 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 >> 10.20.193.78:0/470893559 conn(0x7f757c0a6620 msgr2=0x7f757c1365d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 shutdown_connections 2026-02-17T05:49:20.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.665+0000 7f7583ae1640 1 -- 10.20.193.78:0/470893559 wait complete. 2026-02-17T05:49:20.674 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:49:20.674 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T05:49:20.675 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:49:20.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.742+0000 7f2bc7d79640 10 monclient: get_monmap_and_config 2026-02-17T05:49:20.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.742+0000 7f2bc7d79640 10 monclient: build_initial_monmap 2026-02-17T05:49:20.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.742+0000 7f2bc7d79640 10 monclient: monmap: 2026-02-17T05:49:20.746 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:20.746 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.746 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:20.743906+0000 2026-02-17T05:49:20.746 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:20.743906+0000 2026-02-17T05:49:20.746 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:20.746 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 1 Processor -- start 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 1 -- start start 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient: init 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:20.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2bc006c890 0x7f2bc006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient(hunting): picked mon.noname-c con 0x7f2bc006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2bc009c4a0 0x7f2bc006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient(hunting): picked mon.noname-a con 0x7f2bc009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2bc0134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc006daf0 con 0x7f2bc0134df0 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f2bc00a0880 con 0x7f2bc009c4a0 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2bc006d6f0 con 0x7f2bc006c890 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:20.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc7d79640 10 monclient(hunting): authenticate will time out at 806.297444s 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2bc009c4a0 0x7f2bc006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2bc009c4a0 0x7f2bc006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55462/0 (socket says 10.20.193.78:55462) 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 1 -- 10.20.193.78:0/3948247164 learned_addr learned my addr 10.20.193.78:0/3948247164 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient(hunting): get_auth_request con 0x7f2bc009c4a0 auth_method 0 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc5aee640 1 --2- 10.20.193.78:0/3948247164 >> v2:10.20.193.115:3300/0 conn(0x7f2bc006c890 0x7f2bc006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc5aee640 10 monclient(hunting): get_auth_request con 0x7f2bc006c890 auth_method 0 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc5aee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc5aee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc5aee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient(hunting): handle_auth_done global_id 54168 payload 274 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 1 -- 10.20.193.78:0/3948247164 >> v2:10.20.193.115:3300/0 conn(0x7f2bc006c890 msgr2=0x7f2bc006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 1 --2- 10.20.193.78:0/3948247164 >> v2:10.20.193.115:3300/0 conn(0x7f2bc006c890 0x7f2bc006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 1 -- 10.20.193.78:0/3948247164 >> v1:10.20.193.115:6789/0 conn(0x7f2bc0134df0 legacy=0x7f2bc0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient: _finish_hunting 0 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 1 monclient: found mon.noname-a 2026-02-17T05:49:20.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:20.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.743+0000 7f2bc52ed640 1 -- 10.20.193.78:0/3948247164 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc009cc80 con 0x7f2bc009c4a0 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc52ed640 1 -- 10.20.193.78:0/3948247164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc009c4a0 msgr2=0x7f2bc006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc52ed640 1 --2- 10.20.193.78:0/3948247164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc009c4a0 0x7f2bc006d1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bc006a480 tx=0x7f2bb402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31848661a11372c6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 1 -- 10.20.193.78:0/3948247164 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb403c070 con 0x7f2bc009c4a0 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: dump: 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.761 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: _finish_auth 0 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: _check_auth_tickets 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:20.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 1 -- 10.20.193.78:0/3948247164 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb402fca0 con 0x7f2bc009c4a0 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 5 monclient: authenticate success, global_id 54168 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 1 -- 10.20.193.78:0/3948247164 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4037660 con 0x7f2bc009c4a0 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 4 set_mon_vals no callback set 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc4aec640 10 monclient: dump: 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.763 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 10 monclient: shutdown 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 1 -- 10.20.193.78:0/3948247164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc009c4a0 msgr2=0x7f2bc006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 1 --2- 10.20.193.78:0/3948247164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc009c4a0 0x7f2bc006d1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bc006a480 tx=0x7f2bb402ee60 comp rx=0 tx=0).stop 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 1 -- 10.20.193.78:0/3948247164 shutdown_connections 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 1 --2- 10.20.193.78:0/3948247164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc009c4a0 0x7f2bc006d1b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 1 --2- 10.20.193.78:0/3948247164 >> v2:10.20.193.115:3300/0 conn(0x7f2bc006c890 0x7f2bc006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 1 -- 10.20.193.78:0/3948247164 >> 10.20.193.78:0/3948247164 conn(0x7f2bc00a6620 msgr2=0x7f2bc00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 1 -- 10.20.193.78:0/3948247164 shutdown_connections 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.746+0000 7f2bc7d79640 1 -- 10.20.193.78:0/3948247164 wait complete. 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient: build_initial_monmap 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient: monmap: 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:20.765 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 1 Processor -- start 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 1 -- start start 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient: init 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2bc006c890 0x7f2bc01cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient(hunting): picked mon.noname-c con 0x7f2bc006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2bc009c4a0 0x7f2bc0133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient(hunting): picked mon.noname-b con 0x7f2bc009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0134df0 0x7f2bc01344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient(hunting): picked mon.noname-a con 0x7f2bc0134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00a01a0 con 0x7f2bc0134df0 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00a0020 con 0x7f2bc009c4a0 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc009be20 con 0x7f2bc006c890 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc7d79640 10 monclient(hunting): authenticate will time out at 806.301766s 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc5aee640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2bc006c890 msgr2=0x7f2bc01cf430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc5aee640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2bc006c890 0x7f2bc01cf430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0134df0 0x7f2bc01344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0134df0 0x7f2bc01344b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55464/0 (socket says 10.20.193.78:55464) 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 -- 10.20.193.78:0/1699744001 learned_addr learned my addr 10.20.193.78:0/1699744001 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient(hunting): get_auth_request con 0x7f2bc0134df0 auth_method 0 2026-02-17T05:49:20.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc52ed640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2bc009c4a0 0x7f2bc0133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc52ed640 10 monclient(hunting): get_auth_request con 0x7f2bc009c4a0 auth_method 0 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc52ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc52ed640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc52ed640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient(hunting): handle_auth_done global_id 54171 payload 824 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2bc006c890 msgr2=0x7f2bc01cf430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2bc006c890 0x7f2bc01cf430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2bc009c4a0 msgr2=0x7f2bc0133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2bc009c4a0 0x7f2bc0133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient: _finish_hunting 0 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 monclient: found mon.noname-a 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc62ef640 1 -- 10.20.193.78:0/1699744001 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc00a1780 con 0x7f2bc0134df0 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.747+0000 7f2bc52ed640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2bc009c4a0 0x7f2bc0133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc62ef640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0134df0 msgr2=0x7f2bc01344b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc62ef640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0134df0 0x7f2bc01344b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00cb60 tx=0x7f2bbc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc016020 con 0x7f2bc0134df0 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: dump: 2026-02-17T05:49:20.769 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: _finish_auth 0 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bbc012420 con 0x7f2bc0134df0 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:20.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc012810 con 0x7f2bc0134df0 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 5 monclient: authenticate success, global_id 54171 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 10 monclient: _renew_subs 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2baffff640 4 set_mon_vals no callback set 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc009d810 con 0x7f2bc0134df0 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 10 monclient: dump: 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.771 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 10 monclient: _renew_subs 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc009f7a0 con 0x7f2bc0134df0 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0126200 0x7f2bc009e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0074980 con 0x7f2bc0126200 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0126200 0x7f2bc009e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 10 monclient: get_auth_request con 0x7f2bc0126200 auth_method 0 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2bbc012a30 con 0x7f2bc0134df0 2026-02-17T05:49:20.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_done global_id 54174 payload 824 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.750+0000 7f2bc5aee640 1 -- 10.20.193.78:0/1699744001 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bc0135a20 con 0x7f2bc0126200 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.751+0000 7f2bae7fc640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2b9003de60 0x7f2b90040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.751+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4795+0+0 (secure 0 0 0) 0x7f2bbc0532d0 con 0x7f2bc0134df0 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.751+0000 7f2bc52ed640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2b9003de60 0x7f2b90040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.751+0000 7f2bc52ed640 10 monclient: get_auth_request con 0x7f2b9003de60 auth_method 0 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bc5aee640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0126200 msgr2=0x7f2bc009e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bc5aee640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0126200 0x7f2bc009e2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e8b0 tx=0x7f2bb000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0017070 con 0x7f2bc0126200 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 10 monclient: dump: 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.773 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2bb000cbc0 con 0x7f2bc0126200 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0126200 msgr2=0x7f2bc009e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0126200 0x7f2bc009e2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e8b0 tx=0x7f2bb000ed80 comp rx=0 tx=0).stop 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0126200 msgr2=0x7f2bc009e2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bae7fc640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0126200 0x7f2bc009e2e0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e8b0 tx=0x7f2bb000ed80 comp rx=0 tx=0).stop 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.753+0000 7f2bc52ed640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2b9003de60 0x7f2b90040320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4036000 tx=0x7f2bb4039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc7d79640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc7d79640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc011cc70 0x7f2bc01da530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc7d79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc00744d0 con 0x7f2bc011cc70 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc011cc70 0x7f2bc01da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 10 monclient: get_auth_request con 0x7f2bc011cc70 auth_method 0 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 10 monclient(hunting): handle_auth_done global_id 54177 payload 824 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.756+0000 7f2bc5aee640 1 -- 10.20.193.78:0/1699744001 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2bc009e820 con 0x7f2bc011cc70 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.758+0000 7f2bc5aee640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc011cc70 msgr2=0x7f2bc01da530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.758+0000 7f2bc5aee640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc011cc70 0x7f2bc01da530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e8b0 tx=0x7f2bb0004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0017070 con 0x7f2bc011cc70 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 10 monclient: dump: 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7f2bb0003db0 con 0x7f2bc011cc70 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc011cc70 msgr2=0x7f2bc01da530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc011cc70 0x7f2bc01da530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e8b0 tx=0x7f2bb0004830 comp rx=0 tx=0).stop 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc011cc70 msgr2=0x7f2bc01da530 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bae7fc640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc011cc70 0x7f2bc01da530 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e8b0 tx=0x7f2bb0004830 comp rx=0 tx=0).stop 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 38, 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 30233, 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:20.775 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:20.776 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.777 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.778 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.779 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:20.780 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2b9003de60 msgr2=0x7f2b90040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2b9003de60 0x7f2b90040320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4036000 tx=0x7f2bb4039040 comp rx=0 tx=0).stop 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 10 monclient: shutdown 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0134df0 msgr2=0x7f2bc01344b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0134df0 0x7f2bc01344b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00cb60 tx=0x7f2bbc00a430 comp rx=0 tx=0).stop 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc62ef640 1 -- 10.20.193.78:0/1699744001 reap_dead start 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 shutdown_connections 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 --2- 10.20.193.78:0/1699744001 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2bc0134df0 0x7f2bc01344b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 >> 10.20.193.78:0/1699744001 conn(0x7f2bc00a6620 msgr2=0x7f2bc00a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 shutdown_connections 2026-02-17T05:49:20.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.759+0000 7f2bc7d79640 1 -- 10.20.193.78:0/1699744001 wait complete. 2026-02-17T05:49:20.781 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:49:20.781 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:49:20.781 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.848+0000 7fba13696640 10 monclient: get_monmap_and_config 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.848+0000 7fba13696640 10 monclient: build_initial_monmap 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.848+0000 7fba13696640 10 monclient: monmap: 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:20.850033+0000 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:20.850033+0000 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:20.849 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:20.850 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 1 Processor -- start 2026-02-17T05:49:20.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 1 -- start start 2026-02-17T05:49:20.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient: init 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fba0c06c890 0x7fba0c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient(hunting): picked mon.noname-a con 0x7fba0c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fba0c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 0x7fba0c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient(hunting): picked mon.noname-b con 0x7fba0c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba0c06daf0 con 0x7fba0c09c4a0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fba0c0a0880 con 0x7fba0c06c890 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fba0c09cc80 con 0x7fba0c06d1b0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba13696640 10 monclient(hunting): authenticate will time out at 806.403663s 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba11c0c640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 msgr2=0x7fba0c135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fba0c06c890 0x7fba0c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba11c0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 0x7fba0c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fba0c06c890 0x7fba0c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55496/0 (socket says 10.20.193.78:55496) 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 1 -- 10.20.193.78:0/2037471624 learned_addr learned my addr 10.20.193.78:0/2037471624 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient(hunting): get_auth_request con 0x7fba0c06c890 auth_method 0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient(hunting): handle_auth_done global_id 54180 payload 274 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 1 -- 10.20.193.78:0/2037471624 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 msgr2=0x7fba0c135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 1 --2- 10.20.193.78:0/2037471624 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 0x7fba0c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 1 -- 10.20.193.78:0/2037471624 >> v1:10.20.193.115:6789/0 conn(0x7fba0c09c4a0 legacy=0x7fba0c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient: _finish_hunting 0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 1 monclient: found mon.noname-a 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.849+0000 7fba1140b640 1 -- 10.20.193.78:0/2037471624 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c09cf60 con 0x7fba0c06c890 2026-02-17T05:49:20.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba1140b640 1 -- 10.20.193.78:0/2037471624 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 msgr2=0x7fba0c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba1140b640 1 --2- 10.20.193.78:0/2037471624 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 0x7fba0c06cc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fba0c06dcc0 tx=0x7fb9fc02eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b086b4fbf5a6983 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 1 -- 10.20.193.78:0/2037471624 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc03c070 con 0x7fba0c06c890 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: dump: 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: _finish_auth 0 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:20.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 5 monclient: authenticate success, global_id 54180 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 1 -- 10.20.193.78:0/2037471624 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc02fdd0 con 0x7fba0c06c890 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 1 -- 10.20.193.78:0/2037471624 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc037660 con 0x7fba0c06c890 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 4 set_mon_vals no callback set 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fb9fbfff640 10 monclient: dump: 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 10 monclient: shutdown 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 1 -- 10.20.193.78:0/2037471624 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 msgr2=0x7fba0c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 1 --2- 10.20.193.78:0/2037471624 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 0x7fba0c06cc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fba0c06dcc0 tx=0x7fb9fc02eeb0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 1 -- 10.20.193.78:0/2037471624 shutdown_connections 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 1 --2- 10.20.193.78:0/2037471624 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 0x7fba0c135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 1 --2- 10.20.193.78:0/2037471624 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 0x7fba0c06cc70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 1 -- 10.20.193.78:0/2037471624 >> 10.20.193.78:0/2037471624 conn(0x7fba0c0a6620 msgr2=0x7fba0c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 1 -- 10.20.193.78:0/2037471624 shutdown_connections 2026-02-17T05:49:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.852+0000 7fba13696640 1 -- 10.20.193.78:0/2037471624 wait complete. 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient: build_initial_monmap 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient: monmap: 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 1 Processor -- start 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 1 -- start start 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient: init 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 0x7fba0c1347d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient(hunting): picked mon.noname-a con 0x7fba0c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 0x7fba0c134d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient(hunting): picked mon.noname-c con 0x7fba0c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c09c4a0 0x7fba0c1242e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient(hunting): picked mon.noname-b con 0x7fba0c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c0a01a0 con 0x7fba0c06c890 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c0a0020 con 0x7fba0c09c4a0 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fba0c09be20 con 0x7fba0c06d1b0 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba13696640 10 monclient(hunting): authenticate will time out at 806.407932s 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba10c0a640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 msgr2=0x7fba0c134d10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba10c0a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 0x7fba0c134d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba1140b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 0x7fba0c1347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba1140b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 0x7fba0c1347d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55508/0 (socket says 10.20.193.78:55508) 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba1140b640 1 -- 10.20.193.78:0/3921013670 learned_addr learned my addr 10.20.193.78:0/3921013670 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba1140b640 10 monclient(hunting): get_auth_request con 0x7fba0c06c890 auth_method 0 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.853+0000 7fba1140b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 10 monclient(hunting): handle_auth_done global_id 54183 payload 824 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 msgr2=0x7fba0c134d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fba0c06d1b0 0x7fba0c134d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c09c4a0 msgr2=0x7fba0c1242e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c09c4a0 0x7fba0c1242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 10 monclient: _finish_hunting 0 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 1 monclient: found mon.noname-a 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.854+0000 7fba1140b640 1 -- 10.20.193.78:0/3921013670 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c1281f0 con 0x7fba0c06c890 2026-02-17T05:49:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba1140b640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 msgr2=0x7fba0c1347d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba1140b640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 0x7fba0c1347d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc02f3c0 tx=0x7fb9fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc046020 con 0x7fba0c06c890 2026-02-17T05:49:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: dump: 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc042620 con 0x7fba0c06c890 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc03c040 con 0x7fba0c06c890 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9fb7fe640 4 set_mon_vals no callback set 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fb9f9ffb640 10 monclient: dump: 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 5 monclient: authenticate success, global_id 54183 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 10 monclient: _renew_subs 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0c0a2030 con 0x7fba0c06c890 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 10 monclient: _renew_subs 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0c127c30 con 0x7fba0c06c890 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:20.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c1cf6d0 0x7fba0c071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.856+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c135ad0 con 0x7fba0c1cf6d0 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb9fc03f030 con 0x7fba0c06c890 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c1cf6d0 0x7fba0c071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 10 monclient: get_auth_request con 0x7fba0c1cf6d0 auth_method 0 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 10 monclient(hunting): handle_auth_done global_id 44224 payload 824 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fb9f9ffb640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb9dc03deb0 0x7fb9dc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba10c0a640 1 -- 10.20.193.78:0/3921013670 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba0c073310 con 0x7fba0c1cf6d0 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4795+0+0 (secure 0 0 0) 0x7fb9fc0777f0 con 0x7fba0c06c890 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba11c0c640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb9dc03deb0 0x7fb9dc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.857+0000 7fba11c0c640 10 monclient: get_auth_request con 0x7fb9dc03deb0 auth_method 0 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fba10c0a640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c1cf6d0 msgr2=0x7fba0c071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fba10c0a640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c1cf6d0 0x7fba0c071420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fba0c126980 tx=0x7fba0000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00017070 con 0x7fba0c1cf6d0 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 10 monclient: dump: 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fba00007070 con 0x7fba0c1cf6d0 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c1cf6d0 msgr2=0x7fba0c071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c1cf6d0 0x7fba0c071420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fba0c126980 tx=0x7fba0000c8d0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c1cf6d0 msgr2=0x7fba0c071420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:20.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.859+0000 7fb9f9ffb640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c1cf6d0 0x7fba0c071420 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fba0c126980 tx=0x7fba0000c8d0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.860+0000 7fba11c0c640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb9dc03deb0 0x7fb9dc040370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba08003e00 tx=0x7fba08009230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba13696640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:20.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba13696640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c071960 0x7fba0c1da810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:20.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba13696640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:20.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c0a6a00 con 0x7fba0c071960 2026-02-17T05:49:20.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c071960 0x7fba0c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:20.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 10 monclient: get_auth_request con 0x7fba0c071960 auth_method 0 2026-02-17T05:49:20.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:20.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:20.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:20.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:20.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:20.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:20.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 10 monclient(hunting): handle_auth_done global_id 44227 payload 824 2026-02-17T05:49:20.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.863+0000 7fba10c0a640 1 -- 10.20.193.78:0/3921013670 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fba0c126540 con 0x7fba0c071960 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.865+0000 7fba10c0a640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c071960 msgr2=0x7fba0c1da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.865+0000 7fba10c0a640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c071960 0x7fba0c1da810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba0c126980 tx=0x7fba000038e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00017040 con 0x7fba0c071960 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 10 monclient: dump: 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:20.866 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4863 (secure 0 0 0) 0x7fba00003030 con 0x7fba0c071960 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c071960 msgr2=0x7fba0c1da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c071960 0x7fba0c1da810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba0c126980 tx=0x7fba000038e0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c071960 msgr2=0x7fba0c1da810 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fb9f9ffb640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fba0c071960 0x7fba0c1da810 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba0c126980 tx=0x7fba000038e0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 38, 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 84515, 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:20.867 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:20.868 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:20.869 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:20.870 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:20.871 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb9dc03deb0 msgr2=0x7fb9dc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb9dc03deb0 0x7fb9dc040370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba08003e00 tx=0x7fba08009230 comp rx=0 tx=0).stop 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 10 monclient: shutdown 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 msgr2=0x7fba0c1347d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 1 --2- 10.20.193.78:0/3921013670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fba0c06c890 0x7fba0c1347d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc02f3c0 tx=0x7fb9fc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba11c0c640 1 -- 10.20.193.78:0/3921013670 reap_dead start 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba11c0c640 1 -- 10.20.193.78:0/3921013670 reap_dead start 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 shutdown_connections 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 >> 10.20.193.78:0/3921013670 conn(0x7fba0c0a6620 msgr2=0x7fba0c0a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 shutdown_connections 2026-02-17T05:49:20.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:20.866+0000 7fba13696640 1 -- 10.20.193.78:0/3921013670 wait complete. 2026-02-17T05:49:20.873 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:49:20.873 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:49:21.542 INFO:tasks.rados.rados.0.trial115.stdout:547: done (3 left) 2026-02-17T05:49:21.542 INFO:tasks.rados.rados.0.trial115.stdout:548: read oid 1 snap -1 2026-02-17T05:49:21.542 INFO:tasks.rados.rados.0.trial115.stdout:548: expect (ObjNum 195 snap 36 seq_num 195) 2026-02-17T05:49:21.542 INFO:tasks.rados.rados.0.trial115.stdout:549: copy_from oid 14 from oid 30 current snap is 47 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:542: finishing copy_from to trial11545420-4 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 24 (ObjNum 78 snap 8 seq_num 78) dirty exists 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:545: finishing write tid 2 to trial11545420-42 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:545: finishing write tid 3 to trial11545420-42 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:542: done (4 left) 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:550: rollback oid 23 current snap is 47 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:550: done (4 left) 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:551: write oid 12 current snap is 47 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:545: finishing write tid 4 to trial11545420-42 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:545: finishing write tid 5 to trial11545420-42 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:545: finishing write tid 6 to trial11545420-42 2026-02-17T05:49:21.543 INFO:tasks.rados.rados.0.trial115.stdout:545: oid 42 updating version 0 to 266 2026-02-17T05:49:21.544 INFO:tasks.rados.rados.0.trial115.stdout:545: oid 42 updating version 266 to 269 2026-02-17T05:49:21.544 INFO:tasks.rados.rados.0.trial115.stdout:545: oid 42 version 269 is already newer than 268 2026-02-17T05:49:21.544 INFO:tasks.rados.rados.0.trial115.stdout:545: oid 42 version 269 is already newer than 267 2026-02-17T05:49:21.544 INFO:tasks.rados.rados.0.trial115.stdout:545: oid 42 version 269 is already newer than 265 2026-02-17T05:49:21.544 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 269 (ObjNum 237 snap 47 seq_num 237) dirty exists 2026-02-17T05:49:21.544 INFO:tasks.rados.rados.0.trial115.stdout:545: left oid 42 (ObjNum 237 snap 47 seq_num 237) 2026-02-17T05:49:21.544 INFO:tasks.rados.rados.0.trial115.stdout:546: finishing copy_from racing read to trial11545420-45 2026-02-17T05:49:21.544 INFO:tasks.rados.rados.0.trial115.stdout:551: seq_num 238 ranges {400066=716205,1860832=139234} 2026-02-17T05:49:21.546 INFO:tasks.rados.rados.0.trial115.stdout:551: writing trial11545420-12 from 400066 to 1116271 tid 1 2026-02-17T05:49:21.549 INFO:tasks.rados.rados.0.trial115.stdout:551: writing trial11545420-12 from 1860832 to 2000066 tid 2 2026-02-17T05:49:21.550 INFO:tasks.rados.rados.0.trial115.stdout:545: done (4 left) 2026-02-17T05:49:21.550 INFO:tasks.rados.rados.0.trial115.stdout:552: write oid 20 current snap is 47 2026-02-17T05:49:21.550 INFO:tasks.rados.rados.0.trial115.stdout:546: finishing copy_from to trial11545420-45 2026-02-17T05:49:21.550 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 157 (ObjNum 201 snap 41 seq_num 201) dirty exists 2026-02-17T05:49:21.550 INFO:tasks.rados.rados.0.trial115.stdout:548: expect (ObjNum 195 snap 36 seq_num 195) 2026-02-17T05:49:21.653 INFO:tasks.rados.rados.0.trial115.stdout:549: finishing copy_from racing read to trial11545420-14 2026-02-17T05:49:21.653 INFO:tasks.rados.rados.0.trial115.stdout:551: finishing write tid 1 to trial11545420-12 2026-02-17T05:49:21.653 INFO:tasks.rados.rados.0.trial115.stdout:552: seq_num 239 ranges {416873=791454,1919314=97559} 2026-02-17T05:49:21.657 INFO:tasks.rados.rados.0.trial115.stdout:552: writing trial11545420-20 from 416873 to 1208327 tid 1 2026-02-17T05:49:21.660 INFO:tasks.rados.rados.0.trial115.stdout:552: writing trial11545420-20 from 1919314 to 2016873 tid 2 2026-02-17T05:49:21.660 INFO:tasks.rados.rados.0.trial115.stdout:546: done (4 left) 2026-02-17T05:49:21.660 INFO:tasks.rados.rados.0.trial115.stdout:548: done (3 left) 2026-02-17T05:49:21.660 INFO:tasks.rados.rados.0.trial115.stdout:553: write oid 28 current snap is 47 2026-02-17T05:49:21.660 INFO:tasks.rados.rados.0.trial115.stdout:553: seq_num 240 ranges {433680=466703,1577796=455884} 2026-02-17T05:49:21.664 INFO:tasks.rados.rados.0.trial115.stdout:553: writing trial11545420-28 from 433680 to 900383 tid 1 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:553: writing trial11545420-28 from 1577796 to 2033680 tid 2 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:554: delete oid 24 current snap is 47 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:551: finishing write tid 2 to trial11545420-12 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:551: finishing write tid 3 to trial11545420-12 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:551: finishing write tid 4 to trial11545420-12 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:551: oid 12 updating version 0 to 278 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:551: oid 12 version 278 is already newer than 272 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:551: oid 12 version 278 is already newer than 277 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 278 (ObjNum 238 snap 47 seq_num 238) dirty exists 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:551: left oid 12 (ObjNum 238 snap 47 seq_num 238) 2026-02-17T05:49:21.668 INFO:tasks.rados.rados.0.trial115.stdout:549: finishing copy_from to trial11545420-14 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 149 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:552: finishing write tid 1 to trial11545420-20 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:552: finishing write tid 2 to trial11545420-20 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:552: finishing write tid 3 to trial11545420-20 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:552: finishing write tid 4 to trial11545420-20 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:552: oid 20 updating version 0 to 184 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:552: oid 20 version 184 is already newer than 182 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:552: oid 20 version 184 is already newer than 183 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 184 (ObjNum 239 snap 47 seq_num 239) dirty exists 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:552: left oid 20 (ObjNum 239 snap 47 seq_num 239) 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:553: finishing write tid 1 to trial11545420-28 2026-02-17T05:49:21.669 INFO:tasks.rados.rados.0.trial115.stdout:553: finishing write tid 2 to trial11545420-28 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:553: finishing write tid 3 to trial11545420-28 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:549: done (4 left) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:551: done (3 left) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:552: done (2 left) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:554: done (1 left) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:555: rollback oid 20 current snap is 47 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:555: done (1 left) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:556: read oid 37 snap -1 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:556: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:557: read oid 27 snap -1 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:557: expect (ObjNum 212 snap 44 seq_num 212) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:558: copy_from oid 5 from oid 8 current snap is 47 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:553: finishing write tid 4 to trial11545420-28 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:553: oid 28 updating version 0 to 196 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:553: oid 28 version 196 is already newer than 195 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:553: oid 28 updating version 196 to 197 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 197 (ObjNum 240 snap 47 seq_num 240) dirty exists 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:553: left oid 28 (ObjNum 240 snap 47 seq_num 240) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:553: done (3 left) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:559: read oid 4 snap -1 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:559: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:560: snap_create 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:558: finishing copy_from to trial11545420-5 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:558: got expected ENOENT (src dne) 2026-02-17T05:49:21.670 INFO:tasks.rados.rados.0.trial115.stdout:558: finishing copy_from racing read to trial11545420-5 2026-02-17T05:49:21.675 INFO:tasks.rados.rados.0.trial115.stdout:557: expect (ObjNum 212 snap 44 seq_num 212) 2026-02-17T05:49:21.757 INFO:tasks.rados.rados.0.trial115.stdout:556: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:49:21.866 INFO:tasks.rados.rados.0.trial115.stdout:559: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:49:22.545 INFO:tasks.rados.rados.0.trial115.stdout:556: done (4 left) 2026-02-17T05:49:22.545 INFO:tasks.rados.rados.0.trial115.stdout:557: done (3 left) 2026-02-17T05:49:22.545 INFO:tasks.rados.rados.0.trial115.stdout:558: done (2 left) 2026-02-17T05:49:22.545 INFO:tasks.rados.rados.0.trial115.stdout:559: done (1 left) 2026-02-17T05:49:22.546 INFO:tasks.rados.rados.0.trial115.stdout:560: done (0 left) 2026-02-17T05:49:22.546 INFO:tasks.rados.rados.0.trial115.stdout:561: write (excl) oid 32 current snap is 48 2026-02-17T05:49:22.546 INFO:tasks.rados.rados.0.trial115.stdout:561: seq_num 242 ranges {467294=617201,1578407=488887} 2026-02-17T05:49:22.549 INFO:tasks.rados.rados.0.trial115.stdout:561: writing trial11545420-32 from 467294 to 1084495 tid 1 2026-02-17T05:49:22.553 INFO:tasks.rados.rados.0.trial115.stdout:561: writing trial11545420-32 from 1578407 to 2067294 tid 2 2026-02-17T05:49:22.553 INFO:tasks.rados.rados.0.trial115.stdout:562: rollback oid 2 current snap is 48 2026-02-17T05:49:22.553 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 2 to 47 2026-02-17T05:49:22.553 INFO:tasks.rados.rados.0.trial115.stdout:561: finishing write tid 1 to trial11545420-32 2026-02-17T05:49:22.553 INFO:tasks.rados.rados.0.trial115.stdout:563: write (excl) oid 5 current snap is 48 2026-02-17T05:49:22.553 INFO:tasks.rados.rados.0.trial115.stdout:563: seq_num 243 ranges {484101=692450,1636889=447212} 2026-02-17T05:49:22.557 INFO:tasks.rados.rados.0.trial115.stdout:563: writing trial11545420-5 from 484101 to 1176551 tid 1 2026-02-17T05:49:22.560 INFO:tasks.rados.rados.0.trial115.stdout:563: writing trial11545420-5 from 1636889 to 2084101 tid 2 2026-02-17T05:49:22.561 INFO:tasks.rados.rados.0.trial115.stdout:564: write oid 25 current snap is 48 2026-02-17T05:49:22.561 INFO:tasks.rados.rados.0.trial115.stdout:564: seq_num 244 ranges {500908=484052,1411724=689184} 2026-02-17T05:49:22.564 INFO:tasks.rados.rados.0.trial115.stdout:564: writing trial11545420-25 from 500908 to 984960 tid 1 2026-02-17T05:49:22.569 INFO:tasks.rados.rados.0.trial115.stdout:564: writing trial11545420-25 from 1411724 to 2100908 tid 2 2026-02-17T05:49:22.569 INFO:tasks.rados.rados.0.trial115.stdout:565: write oid 50 current snap is 48 2026-02-17T05:49:22.569 INFO:tasks.rados.rados.0.trial115.stdout:565: seq_num 245 ranges {517715=559301,1870206=247509} 2026-02-17T05:49:22.572 INFO:tasks.rados.rados.0.trial115.stdout:565: writing trial11545420-50 from 517715 to 1077016 tid 1 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:565: writing trial11545420-50 from 1870206 to 2117715 tid 2 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:566: rollback oid 36 current snap is 48 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:562: finishing rollback tid 1 to trial11545420-2 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 0 (ObjNum 177 snap 0 seq_num 1333683344) dirty dne 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:561: finishing write tid 2 to trial11545420-32 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:561: finishing write tid 3 to trial11545420-32 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 36 to 47 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:561: finishing write tid 4 to trial11545420-32 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:561: oid 32 updating version 0 to 64 2026-02-17T05:49:22.578 INFO:tasks.rados.rados.0.trial115.stdout:561: oid 32 updating version 64 to 65 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:561: oid 32 updating version 65 to 66 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 66 (ObjNum 242 snap 48 seq_num 242) dirty exists 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:561: left oid 32 (ObjNum 242 snap 48 seq_num 242) 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:563: finishing write tid 1 to trial11545420-5 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:563: finishing write tid 2 to trial11545420-5 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:563: finishing write tid 3 to trial11545420-5 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:563: finishing write tid 4 to trial11545420-5 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:563: oid 5 updating version 0 to 249 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:563: oid 5 updating version 249 to 250 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:563: oid 5 updating version 250 to 251 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 251 (ObjNum 243 snap 48 seq_num 243) dirty exists 2026-02-17T05:49:22.579 INFO:tasks.rados.rados.0.trial115.stdout:563: left oid 5 (ObjNum 243 snap 48 seq_num 243) 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:564: finishing write tid 1 to trial11545420-25 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:564: finishing write tid 2 to trial11545420-25 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:564: finishing write tid 3 to trial11545420-25 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:564: finishing write tid 4 to trial11545420-25 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:564: oid 25 updating version 0 to 186 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:564: oid 25 updating version 186 to 188 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:564: oid 25 version 188 is already newer than 187 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 188 (ObjNum 244 snap 48 seq_num 244) dirty exists 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:564: left oid 25 (ObjNum 244 snap 48 seq_num 244) 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:565: finishing write tid 1 to trial11545420-50 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:561: done (5 left) 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:562: done (4 left) 2026-02-17T05:49:22.580 INFO:tasks.rados.rados.0.trial115.stdout:563: done (3 left) 2026-02-17T05:49:22.581 INFO:tasks.rados.rados.0.trial115.stdout:564: done (2 left) 2026-02-17T05:49:22.581 INFO:tasks.rados.rados.0.trial115.stdout:567: rollback oid 35 current snap is 48 2026-02-17T05:49:22.581 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 35 to 47 2026-02-17T05:49:22.581 INFO:tasks.rados.rados.0.trial115.stdout:568: write (excl) oid 47 current snap is 48 2026-02-17T05:49:22.581 INFO:tasks.rados.rados.0.trial115.stdout:568: seq_num 246 ranges {534522=634550,1812335=322187} 2026-02-17T05:49:22.581 INFO:tasks.rados.rados.0.trial115.stdout:568: writing trial11545420-47 from 534522 to 1169072 tid 1 2026-02-17T05:49:22.585 INFO:tasks.rados.rados.0.trial115.stdout:568: writing trial11545420-47 from 1812335 to 2134522 tid 2 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:569: snap_create 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:565: finishing write tid 2 to trial11545420-50 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:565: finishing write tid 3 to trial11545420-50 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:565: finishing write tid 4 to trial11545420-50 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:565: oid 50 updating version 0 to 165 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:565: oid 50 updating version 165 to 166 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:565: oid 50 updating version 166 to 167 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 167 (ObjNum 245 snap 48 seq_num 245) dirty exists 2026-02-17T05:49:22.586 INFO:tasks.rados.rados.0.trial115.stdout:565: left oid 50 (ObjNum 245 snap 48 seq_num 245) 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:567: finishing rollback tid 0 to trial11545420-35 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:566: finishing rollback tid 0 to trial11545420-36 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:567: finishing rollback tid 1 to trial11545420-35 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:566: finishing rollback tid 1 to trial11545420-36 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:567: finishing rollback tid 2 to trial11545420-35 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 204 (ObjNum 195 snap 36 seq_num 195) dirty exists 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:566: finishing rollback tid 2 to trial11545420-36 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 167 (ObjNum 201 snap 41 seq_num 201) dirty exists 2026-02-17T05:49:22.587 INFO:tasks.rados.rados.0.trial115.stdout:568: finishing write tid 1 to trial11545420-47 2026-02-17T05:49:22.588 INFO:tasks.rados.rados.0.trial115.stdout:568: finishing write tid 2 to trial11545420-47 2026-02-17T05:49:22.588 INFO:tasks.rados.rados.0.trial115.stdout:568: finishing write tid 3 to trial11545420-47 2026-02-17T05:49:22.588 INFO:tasks.rados.rados.0.trial115.stdout:568: finishing write tid 4 to trial11545420-47 2026-02-17T05:49:22.588 INFO:tasks.rados.rados.0.trial115.stdout:568: oid 47 updating version 0 to 253 2026-02-17T05:49:22.588 INFO:tasks.rados.rados.0.trial115.stdout:568: oid 47 updating version 253 to 254 2026-02-17T05:49:22.588 INFO:tasks.rados.rados.0.trial115.stdout:568: oid 47 version 254 is already newer than 252 2026-02-17T05:49:22.589 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 254 (ObjNum 246 snap 48 seq_num 246) dirty exists 2026-02-17T05:49:22.589 INFO:tasks.rados.rados.0.trial115.stdout:568: left oid 47 (ObjNum 246 snap 48 seq_num 246) 2026-02-17T05:49:23.546 INFO:tasks.rados.rados.0.trial115.stdout:565: done (4 left) 2026-02-17T05:49:23.546 INFO:tasks.rados.rados.0.trial115.stdout:566: done (3 left) 2026-02-17T05:49:23.546 INFO:tasks.rados.rados.0.trial115.stdout:567: done (2 left) 2026-02-17T05:49:23.546 INFO:tasks.rados.rados.0.trial115.stdout:568: done (1 left) 2026-02-17T05:49:23.546 INFO:tasks.rados.rados.0.trial115.stdout:569: done (0 left) 2026-02-17T05:49:23.546 INFO:tasks.rados.rados.0.trial115.stdout:570: snap_remove snap 48 2026-02-17T05:49:24.550 INFO:tasks.rados.rados.0.trial115.stdout:570: done (0 left) 2026-02-17T05:49:24.550 INFO:tasks.rados.rados.0.trial115.stdout:571: read oid 15 snap -1 2026-02-17T05:49:24.550 INFO:tasks.rados.rados.0.trial115.stdout:571: expect deleted 2026-02-17T05:49:24.550 INFO:tasks.rados.rados.0.trial115.stdout:572: snap_remove snap 47 2026-02-17T05:49:26.162 INFO:tasks.rados.rados.0.trial115.stdout:572: done (1 left) 2026-02-17T05:49:26.162 INFO:tasks.rados.rados.0.trial115.stdout:573: read oid 24 snap -1 2026-02-17T05:49:26.162 INFO:tasks.rados.rados.0.trial115.stdout:573: expect deleted 2026-02-17T05:49:26.162 INFO:tasks.rados.rados.0.trial115.stdout:574: write oid 1 current snap is 49 2026-02-17T05:49:26.162 INFO:tasks.rados.rados.0.trial115.stdout:574: seq_num 247 ranges {551329=709799,1870817=280512} 2026-02-17T05:49:26.167 INFO:tasks.rados.rados.0.trial115.stdout:574: writing trial11545420-1 from 551329 to 1261128 tid 1 2026-02-17T05:49:26.171 INFO:tasks.rados.rados.0.trial115.stdout:574: writing trial11545420-1 from 1870817 to 2151329 tid 2 2026-02-17T05:49:26.172 INFO:tasks.rados.rados.0.trial115.stdout:575: write oid 2 current snap is 49 2026-02-17T05:49:26.172 INFO:tasks.rados.rados.0.trial115.stdout:575: seq_num 248 ranges {568136=785048,1929299=238837} 2026-02-17T05:49:26.177 INFO:tasks.rados.rados.0.trial115.stdout:575: writing trial11545420-2 from 568136 to 1353184 tid 1 2026-02-17T05:49:26.180 INFO:tasks.rados.rados.0.trial115.stdout:575: writing trial11545420-2 from 1929299 to 2168136 tid 2 2026-02-17T05:49:26.180 INFO:tasks.rados.rados.0.trial115.stdout:576: write oid 21 current snap is 49 2026-02-17T05:49:26.180 INFO:tasks.rados.rados.0.trial115.stdout:576: seq_num 249 ranges {584943=460297,1587781=597162} 2026-02-17T05:49:26.184 INFO:tasks.rados.rados.0.trial115.stdout:576: writing trial11545420-21 from 584943 to 1045240 tid 1 2026-02-17T05:49:26.189 INFO:tasks.rados.rados.0.trial115.stdout:576: writing trial11545420-21 from 1587781 to 2184943 tid 2 2026-02-17T05:49:26.189 INFO:tasks.rados.rados.0.trial115.stdout:571: done (4 left) 2026-02-17T05:49:26.189 INFO:tasks.rados.rados.0.trial115.stdout:577: write (excl) oid 15 current snap is 49 2026-02-17T05:49:26.189 INFO:tasks.rados.rados.0.trial115.stdout:577: seq_num 250 ranges {601750=535546,1929910=271840} 2026-02-17T05:49:26.193 INFO:tasks.rados.rados.0.trial115.stdout:577: writing trial11545420-15 from 601750 to 1137296 tid 1 2026-02-17T05:49:26.197 INFO:tasks.rados.rados.0.trial115.stdout:577: writing trial11545420-15 from 1929910 to 2201750 tid 2 2026-02-17T05:49:26.197 INFO:tasks.rados.rados.0.trial115.stdout:578: write (excl) oid 47 current snap is 49 2026-02-17T05:49:26.197 INFO:tasks.rados.rados.0.trial115.stdout:578: seq_num 251 ranges {618557=727148,2104745=113812} 2026-02-17T05:49:26.202 INFO:tasks.rados.rados.0.trial115.stdout:578: writing trial11545420-47 from 618557 to 1345705 tid 1 2026-02-17T05:49:26.206 INFO:tasks.rados.rados.0.trial115.stdout:578: writing trial11545420-47 from 2104745 to 2218557 tid 2 2026-02-17T05:49:26.206 INFO:tasks.rados.rados.0.trial115.stdout:579: rollback oid 48 current snap is 49 2026-02-17T05:49:26.206 INFO:tasks.rados.rados.0.trial115.stdout:579: done (6 left) 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:580: delete oid 22 current snap is 49 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:574: finishing write tid 1 to trial11545420-1 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:574: finishing write tid 2 to trial11545420-1 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:574: finishing write tid 3 to trial11545420-1 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:574: finishing write tid 4 to trial11545420-1 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:574: oid 1 updating version 0 to 169 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:574: oid 1 updating version 169 to 170 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:574: oid 1 version 170 is already newer than 168 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 170 (ObjNum 247 snap 49 seq_num 247) dirty exists 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:574: left oid 1 (ObjNum 247 snap 49 seq_num 247) 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:575: finishing write tid 1 to trial11545420-2 2026-02-17T05:49:26.207 INFO:tasks.rados.rados.0.trial115.stdout:575: finishing write tid 2 to trial11545420-2 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:575: finishing write tid 3 to trial11545420-2 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:575: finishing write tid 4 to trial11545420-2 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:575: oid 2 updating version 0 to 255 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:575: oid 2 updating version 255 to 256 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:575: oid 2 updating version 256 to 257 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 257 (ObjNum 248 snap 49 seq_num 248) dirty exists 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:575: left oid 2 (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:576: finishing write tid 1 to trial11545420-21 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:576: finishing write tid 2 to trial11545420-21 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:576: finishing write tid 3 to trial11545420-21 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:576: finishing write tid 4 to trial11545420-21 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:576: oid 21 updating version 0 to 189 2026-02-17T05:49:26.208 INFO:tasks.rados.rados.0.trial115.stdout:576: oid 21 updating version 189 to 190 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:576: oid 21 updating version 190 to 191 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 191 (ObjNum 249 snap 49 seq_num 249) dirty exists 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:576: left oid 21 (ObjNum 249 snap 49 seq_num 249) 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:577: finishing write tid 1 to trial11545420-15 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:577: finishing write tid 2 to trial11545420-15 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:577: finishing write tid 3 to trial11545420-15 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:577: finishing write tid 4 to trial11545420-15 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:577: oid 15 updating version 0 to 200 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:577: oid 15 version 200 is already newer than 199 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:577: oid 15 version 200 is already newer than 198 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 200 (ObjNum 250 snap 49 seq_num 250) dirty exists 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:577: left oid 15 (ObjNum 250 snap 49 seq_num 250) 2026-02-17T05:49:26.209 INFO:tasks.rados.rados.0.trial115.stdout:578: finishing write tid 1 to trial11545420-47 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:578: finishing write tid 2 to trial11545420-47 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:573: done (6 left) 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:574: done (5 left) 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:575: done (4 left) 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:576: done (3 left) 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:577: done (2 left) 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:580: done (1 left) 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:581: write oid 27 current snap is 49 2026-02-17T05:49:26.210 INFO:tasks.rados.rados.0.trial115.stdout:581: seq_num 253 ranges {652171=477646,1821709=430462} 2026-02-17T05:49:26.211 INFO:tasks.rados.rados.0.trial115.stdout:581: writing trial11545420-27 from 652171 to 1129817 tid 1 2026-02-17T05:49:26.215 INFO:tasks.rados.rados.0.trial115.stdout:581: writing trial11545420-27 from 1821709 to 2252171 tid 2 2026-02-17T05:49:26.215 INFO:tasks.rados.rados.0.trial115.stdout:582: snap_create 2026-02-17T05:49:26.215 INFO:tasks.rados.rados.0.trial115.stdout:578: finishing write tid 3 to trial11545420-47 2026-02-17T05:49:26.215 INFO:tasks.rados.rados.0.trial115.stdout:578: finishing write tid 4 to trial11545420-47 2026-02-17T05:49:26.215 INFO:tasks.rados.rados.0.trial115.stdout:578: oid 47 updating version 0 to 260 2026-02-17T05:49:26.215 INFO:tasks.rados.rados.0.trial115.stdout:578: oid 47 version 260 is already newer than 259 2026-02-17T05:49:26.215 INFO:tasks.rados.rados.0.trial115.stdout:578: oid 47 version 260 is already newer than 258 2026-02-17T05:49:26.215 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 260 (ObjNum 251 snap 49 seq_num 251) dirty exists 2026-02-17T05:49:26.216 INFO:tasks.rados.rados.0.trial115.stdout:578: left oid 47 (ObjNum 251 snap 49 seq_num 251) 2026-02-17T05:49:26.216 INFO:tasks.rados.rados.0.trial115.stdout:581: finishing write tid 1 to trial11545420-27 2026-02-17T05:49:26.216 INFO:tasks.rados.rados.0.trial115.stdout:581: finishing write tid 2 to trial11545420-27 2026-02-17T05:49:26.217 INFO:tasks.rados.rados.0.trial115.stdout:581: finishing write tid 3 to trial11545420-27 2026-02-17T05:49:26.217 INFO:tasks.rados.rados.0.trial115.stdout:581: finishing write tid 4 to trial11545420-27 2026-02-17T05:49:26.217 INFO:tasks.rados.rados.0.trial115.stdout:581: oid 27 updating version 0 to 172 2026-02-17T05:49:26.217 INFO:tasks.rados.rados.0.trial115.stdout:581: oid 27 version 172 is already newer than 171 2026-02-17T05:49:26.217 INFO:tasks.rados.rados.0.trial115.stdout:581: oid 27 updating version 172 to 173 2026-02-17T05:49:26.217 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 173 (ObjNum 253 snap 49 seq_num 253) dirty exists 2026-02-17T05:49:26.217 INFO:tasks.rados.rados.0.trial115.stdout:581: left oid 27 (ObjNum 253 snap 49 seq_num 253) 2026-02-17T05:49:27.517 INFO:tasks.rados.rados.0.trial115.stdout:578: done (2 left) 2026-02-17T05:49:27.517 INFO:tasks.rados.rados.0.trial115.stdout:581: done (1 left) 2026-02-17T05:49:27.517 INFO:tasks.rados.rados.0.trial115.stdout:582: done (0 left) 2026-02-17T05:49:27.517 INFO:tasks.rados.rados.0.trial115.stdout:583: write oid 8 current snap is 50 2026-02-17T05:49:27.517 INFO:tasks.rados.rados.0.trial115.stdout:583: seq_num 254 ranges {668978=552895,1763838=505140} 2026-02-17T05:49:27.522 INFO:tasks.rados.rados.0.trial115.stdout:583: writing trial11545420-8 from 668978 to 1221873 tid 1 2026-02-17T05:49:27.526 INFO:tasks.rados.rados.0.trial115.stdout:583: writing trial11545420-8 from 1763838 to 2268978 tid 2 2026-02-17T05:49:27.526 INFO:tasks.rados.rados.0.trial115.stdout:584: rollback oid 23 current snap is 50 2026-02-17T05:49:27.526 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 23 to 49 2026-02-17T05:49:27.526 INFO:tasks.rados.rados.0.trial115.stdout:583: finishing write tid 1 to trial11545420-8 2026-02-17T05:49:27.526 INFO:tasks.rados.rados.0.trial115.stdout:585: write (excl) oid 6 current snap is 50 2026-02-17T05:49:27.526 INFO:tasks.rados.rados.0.trial115.stdout:585: seq_num 255 ranges {685785=628144,1822320=463465} 2026-02-17T05:49:27.531 INFO:tasks.rados.rados.0.trial115.stdout:585: writing trial11545420-6 from 685785 to 1313929 tid 1 2026-02-17T05:49:27.534 INFO:tasks.rados.rados.0.trial115.stdout:585: writing trial11545420-6 from 1822320 to 2285785 tid 2 2026-02-17T05:49:27.534 INFO:tasks.rados.rados.0.trial115.stdout:586: snap_create 2026-02-17T05:49:27.534 INFO:tasks.rados.rados.0.trial115.stdout:583: finishing write tid 2 to trial11545420-8 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:583: finishing write tid 3 to trial11545420-8 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:583: finishing write tid 4 to trial11545420-8 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:583: oid 8 updating version 0 to 174 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:583: oid 8 updating version 174 to 175 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:583: oid 8 version 175 is already newer than 173 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 175 (ObjNum 254 snap 50 seq_num 254) dirty exists 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:583: left oid 8 (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:584: finishing rollback tid 0 to trial11545420-23 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:584: finishing rollback tid 1 to trial11545420-23 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:584: finishing rollback tid 2 to trial11545420-23 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 61 (ObjNum 99 snap 14 seq_num 99) dirty exists 2026-02-17T05:49:27.535 INFO:tasks.rados.rados.0.trial115.stdout:585: finishing write tid 1 to trial11545420-6 2026-02-17T05:49:27.536 INFO:tasks.rados.rados.0.trial115.stdout:585: finishing write tid 2 to trial11545420-6 2026-02-17T05:49:27.536 INFO:tasks.rados.rados.0.trial115.stdout:585: finishing write tid 3 to trial11545420-6 2026-02-17T05:49:27.536 INFO:tasks.rados.rados.0.trial115.stdout:585: finishing write tid 4 to trial11545420-6 2026-02-17T05:49:27.536 INFO:tasks.rados.rados.0.trial115.stdout:585: oid 6 updating version 0 to 249 2026-02-17T05:49:27.537 INFO:tasks.rados.rados.0.trial115.stdout:585: oid 6 version 249 is already newer than 248 2026-02-17T05:49:27.537 INFO:tasks.rados.rados.0.trial115.stdout:585: oid 6 updating version 249 to 250 2026-02-17T05:49:27.537 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 250 (ObjNum 255 snap 50 seq_num 255) dirty exists 2026-02-17T05:49:27.537 INFO:tasks.rados.rados.0.trial115.stdout:585: left oid 6 (ObjNum 255 snap 50 seq_num 255) 2026-02-17T05:49:28.521 INFO:tasks.rados.rados.0.trial115.stdout:583: done (3 left) 2026-02-17T05:49:28.521 INFO:tasks.rados.rados.0.trial115.stdout:584: done (2 left) 2026-02-17T05:49:28.521 INFO:tasks.rados.rados.0.trial115.stdout:585: done (1 left) 2026-02-17T05:49:28.521 INFO:tasks.rados.rados.0.trial115.stdout:586: done (0 left) 2026-02-17T05:49:28.521 INFO:tasks.rados.rados.0.trial115.stdout:587: copy_from oid 4 from oid 32 current snap is 51 2026-02-17T05:49:28.521 INFO:tasks.rados.rados.0.trial115.stdout:588: snap_remove snap 50 2026-02-17T05:49:29.523 INFO:tasks.rados.rados.0.trial115.stdout:588: done (1 left) 2026-02-17T05:49:29.523 INFO:tasks.rados.rados.0.trial115.stdout:589: write oid 44 current snap is 51 2026-02-17T05:49:29.523 INFO:tasks.rados.rados.0.trial115.stdout:589: seq_num 256 ranges {702592=703393,1880802=421790} 2026-02-17T05:49:29.528 INFO:tasks.rados.rados.0.trial115.stdout:589: writing trial11545420-44 from 702592 to 1405985 tid 1 2026-02-17T05:49:29.531 INFO:tasks.rados.rados.0.trial115.stdout:589: writing trial11545420-44 from 1880802 to 2302592 tid 2 2026-02-17T05:49:29.532 INFO:tasks.rados.rados.0.trial115.stdout:590: delete oid 25 current snap is 51 2026-02-17T05:49:29.532 INFO:tasks.rados.rados.0.trial115.stdout:587: finishing copy_from racing read to trial11545420-4 2026-02-17T05:49:29.532 INFO:tasks.rados.rados.0.trial115.stdout:587: finishing copy_from to trial11545420-4 2026-02-17T05:49:29.532 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 66 (ObjNum 242 snap 48 seq_num 242) dirty exists 2026-02-17T05:49:29.533 INFO:tasks.rados.rados.0.trial115.stdout:589: finishing write tid 1 to trial11545420-44 2026-02-17T05:49:29.533 INFO:tasks.rados.rados.0.trial115.stdout:587: done (2 left) 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:590: done (1 left) 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:591: read oid 39 snap -1 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:591: expect (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:592: delete oid 40 current snap is 51 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:589: finishing write tid 2 to trial11545420-44 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:589: finishing write tid 3 to trial11545420-44 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:589: finishing write tid 4 to trial11545420-44 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:589: oid 44 updating version 0 to 67 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:589: oid 44 updating version 67 to 68 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:589: oid 44 updating version 68 to 69 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 69 (ObjNum 256 snap 51 seq_num 256) dirty exists 2026-02-17T05:49:29.534 INFO:tasks.rados.rados.0.trial115.stdout:589: left oid 44 (ObjNum 256 snap 51 seq_num 256) 2026-02-17T05:49:29.535 INFO:tasks.rados.rados.0.trial115.stdout:589: done (2 left) 2026-02-17T05:49:29.535 INFO:tasks.rados.rados.0.trial115.stdout:592: done (1 left) 2026-02-17T05:49:29.535 INFO:tasks.rados.rados.0.trial115.stdout:593: write oid 9 current snap is 51 2026-02-17T05:49:29.535 INFO:tasks.rados.rados.0.trial115.stdout:593: seq_num 259 ranges {753013=645493,2056248=296765} 2026-02-17T05:49:29.540 INFO:tasks.rados.rados.0.trial115.stdout:593: writing trial11545420-9 from 753013 to 1398506 tid 1 2026-02-17T05:49:29.544 INFO:tasks.rados.rados.0.trial115.stdout:593: writing trial11545420-9 from 2056248 to 2353013 tid 2 2026-02-17T05:49:29.544 INFO:tasks.rados.rados.0.trial115.stdout:594: snap_remove snap 49 2026-02-17T05:49:30.528 INFO:tasks.rados.rados.0.trial115.stdout:594: done (2 left) 2026-02-17T05:49:30.528 INFO:tasks.rados.rados.0.trial115.stdout:595: read oid 28 snap -1 2026-02-17T05:49:30.528 INFO:tasks.rados.rados.0.trial115.stdout:595: expect (ObjNum 240 snap 47 seq_num 240) 2026-02-17T05:49:30.528 INFO:tasks.rados.rados.0.trial115.stdout:596: copy_from oid 16 from oid 47 current snap is 51 2026-02-17T05:49:30.528 INFO:tasks.rados.rados.0.trial115.stdout:591: expect (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:49:30.610 INFO:tasks.rados.rados.0.trial115.stdout:593: finishing write tid 1 to trial11545420-9 2026-02-17T05:49:30.610 INFO:tasks.rados.rados.0.trial115.stdout:593: finishing write tid 2 to trial11545420-9 2026-02-17T05:49:30.610 INFO:tasks.rados.rados.0.trial115.stdout:593: finishing write tid 3 to trial11545420-9 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:591: done (3 left) 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:597: snap_create 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:593: finishing write tid 4 to trial11545420-9 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:593: oid 9 updating version 0 to 202 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:593: oid 9 updating version 202 to 203 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:593: oid 9 updating version 203 to 204 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 204 (ObjNum 259 snap 51 seq_num 259) dirty exists 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:593: left oid 9 (ObjNum 259 snap 51 seq_num 259) 2026-02-17T05:49:30.611 INFO:tasks.rados.rados.0.trial115.stdout:595: expect (ObjNum 240 snap 47 seq_num 240) 2026-02-17T05:49:30.658 INFO:tasks.rados.rados.0.trial115.stdout:596: finishing copy_from racing read to trial11545420-16 2026-02-17T05:49:30.658 INFO:tasks.rados.rados.0.trial115.stdout:596: finishing copy_from to trial11545420-16 2026-02-17T05:49:30.658 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 260 (ObjNum 251 snap 49 seq_num 251) dirty exists 2026-02-17T05:49:31.534 INFO:tasks.rados.rados.0.trial115.stdout:593: done (3 left) 2026-02-17T05:49:31.534 INFO:tasks.rados.rados.0.trial115.stdout:595: done (2 left) 2026-02-17T05:49:31.534 INFO:tasks.rados.rados.0.trial115.stdout:596: done (1 left) 2026-02-17T05:49:31.534 INFO:tasks.rados.rados.0.trial115.stdout:597: done (0 left) 2026-02-17T05:49:31.534 INFO:tasks.rados.rados.0.trial115.stdout:598: rollback oid 45 current snap is 52 2026-02-17T05:49:31.534 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 51 2026-02-17T05:49:31.534 INFO:tasks.rados.rados.0.trial115.stdout:599: write (excl) oid 23 current snap is 52 2026-02-17T05:49:31.534 INFO:tasks.rados.rados.0.trial115.stdout:599: seq_num 260 ranges {769820=720742,2114730=255090} 2026-02-17T05:49:31.539 INFO:tasks.rados.rados.0.trial115.stdout:599: writing trial11545420-23 from 769820 to 1490562 tid 1 2026-02-17T05:49:31.543 INFO:tasks.rados.rados.0.trial115.stdout:599: writing trial11545420-23 from 2114730 to 2369820 tid 2 2026-02-17T05:49:31.543 INFO:tasks.rados.rados.0.trial115.stdout:600: copy_from oid 26 from oid 41 current snap is 52 2026-02-17T05:49:31.543 INFO:tasks.rados.rados.0.trial115.stdout:598: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:49:31.543 INFO:tasks.rados.rados.0.trial115.stdout:598: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:49:31.543 INFO:tasks.rados.rados.0.trial115.stdout:598: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:49:31.543 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 283 (ObjNum 201 snap 41 seq_num 201) dirty exists 2026-02-17T05:49:31.543 INFO:tasks.rados.rados.0.trial115.stdout:598: done (2 left) 2026-02-17T05:49:31.543 INFO:tasks.rados.rados.0.trial115.stdout:601: snap_remove snap 51 2026-02-17T05:49:32.537 INFO:tasks.rados.rados.0.trial115.stdout:601: done (2 left) 2026-02-17T05:49:32.537 INFO:tasks.rados.rados.0.trial115.stdout:602: write oid 42 current snap is 52 2026-02-17T05:49:32.538 INFO:tasks.rados.rados.0.trial115.stdout:602: seq_num 261 ranges {786627=795991,2173212=213415} 2026-02-17T05:49:32.543 INFO:tasks.rados.rados.0.trial115.stdout:602: writing trial11545420-42 from 786627 to 1582618 tid 1 2026-02-17T05:49:32.547 INFO:tasks.rados.rados.0.trial115.stdout:602: writing trial11545420-42 from 2173212 to 2386627 tid 2 2026-02-17T05:49:32.547 INFO:tasks.rados.rados.0.trial115.stdout:603: snap_create 2026-02-17T05:49:32.547 INFO:tasks.rados.rados.0.trial115.stdout:599: finishing write tid 1 to trial11545420-23 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:599: finishing write tid 2 to trial11545420-23 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:599: finishing write tid 3 to trial11545420-23 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:599: finishing write tid 4 to trial11545420-23 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:599: oid 23 updating version 0 to 67 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:599: oid 23 version 67 is already newer than 65 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:599: oid 23 version 67 is already newer than 66 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 67 (ObjNum 260 snap 52 seq_num 260) dirty exists 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:599: left oid 23 (ObjNum 260 snap 52 seq_num 260) 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:600: finishing copy_from racing read to trial11545420-26 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:600: finishing copy_from to trial11545420-26 2026-02-17T05:49:32.548 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 183 (ObjNum 215 snap 45 seq_num 215) dirty exists 2026-02-17T05:49:32.549 INFO:tasks.rados.rados.0.trial115.stdout:602: finishing write tid 1 to trial11545420-42 2026-02-17T05:49:32.550 INFO:tasks.rados.rados.0.trial115.stdout:602: finishing write tid 2 to trial11545420-42 2026-02-17T05:49:32.550 INFO:tasks.rados.rados.0.trial115.stdout:602: finishing write tid 3 to trial11545420-42 2026-02-17T05:49:32.550 INFO:tasks.rados.rados.0.trial115.stdout:602: finishing write tid 4 to trial11545420-42 2026-02-17T05:49:32.550 INFO:tasks.rados.rados.0.trial115.stdout:602: oid 42 updating version 0 to 284 2026-02-17T05:49:32.550 INFO:tasks.rados.rados.0.trial115.stdout:602: oid 42 updating version 284 to 288 2026-02-17T05:49:32.550 INFO:tasks.rados.rados.0.trial115.stdout:602: oid 42 version 288 is already newer than 287 2026-02-17T05:49:32.550 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 288 (ObjNum 261 snap 52 seq_num 261) dirty exists 2026-02-17T05:49:32.550 INFO:tasks.rados.rados.0.trial115.stdout:602: left oid 42 (ObjNum 261 snap 52 seq_num 261) 2026-02-17T05:49:33.538 INFO:tasks.rados.rados.0.trial115.stdout:599: done (3 left) 2026-02-17T05:49:33.538 INFO:tasks.rados.rados.0.trial115.stdout:600: done (2 left) 2026-02-17T05:49:33.538 INFO:tasks.rados.rados.0.trial115.stdout:602: done (1 left) 2026-02-17T05:49:33.538 INFO:tasks.rados.rados.0.trial115.stdout:603: done (0 left) 2026-02-17T05:49:33.538 INFO:tasks.rados.rados.0.trial115.stdout:604: write (excl) oid 29 current snap is 53 2026-02-17T05:49:33.538 INFO:tasks.rados.rados.0.trial115.stdout:604: seq_num 262 ranges {403434=471240,1315341=400756} 2026-02-17T05:49:33.541 INFO:tasks.rados.rados.0.trial115.stdout:604: writing trial11545420-29 from 403434 to 874674 tid 1 2026-02-17T05:49:33.544 INFO:tasks.rados.rados.0.trial115.stdout:604: writing trial11545420-29 from 1315341 to 1716097 tid 2 2026-02-17T05:49:33.544 INFO:tasks.rados.rados.0.trial115.stdout:605: copy_from oid 43 from oid 42 current snap is 53 2026-02-17T05:49:33.544 INFO:tasks.rados.rados.0.trial115.stdout:606: write (excl) oid 46 current snap is 53 2026-02-17T05:49:33.544 INFO:tasks.rados.rados.0.trial115.stdout:606: seq_num 263 ranges {420241=546489,1373823=544414} 2026-02-17T05:49:33.548 INFO:tasks.rados.rados.0.trial115.stdout:606: writing trial11545420-46 from 420241 to 966730 tid 1 2026-02-17T05:49:33.551 INFO:tasks.rados.rados.0.trial115.stdout:606: writing trial11545420-46 from 1373823 to 1918237 tid 2 2026-02-17T05:49:33.551 INFO:tasks.rados.rados.0.trial115.stdout:607: write oid 28 current snap is 53 2026-02-17T05:49:33.551 INFO:tasks.rados.rados.0.trial115.stdout:604: finishing write tid 1 to trial11545420-29 2026-02-17T05:49:33.551 INFO:tasks.rados.rados.0.trial115.stdout:604: finishing write tid 2 to trial11545420-29 2026-02-17T05:49:33.551 INFO:tasks.rados.rados.0.trial115.stdout:604: finishing write tid 3 to trial11545420-29 2026-02-17T05:49:33.551 INFO:tasks.rados.rados.0.trial115.stdout:604: finishing write tid 4 to trial11545420-29 2026-02-17T05:49:33.552 INFO:tasks.rados.rados.0.trial115.stdout:604: oid 29 updating version 0 to 106 2026-02-17T05:49:33.552 INFO:tasks.rados.rados.0.trial115.stdout:604: oid 29 version 106 is already newer than 105 2026-02-17T05:49:33.552 INFO:tasks.rados.rados.0.trial115.stdout:604: oid 29 version 106 is already newer than 104 2026-02-17T05:49:33.552 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 106 (ObjNum 262 snap 53 seq_num 262) dirty exists 2026-02-17T05:49:33.552 INFO:tasks.rados.rados.0.trial115.stdout:604: left oid 29 (ObjNum 262 snap 53 seq_num 262) 2026-02-17T05:49:33.552 INFO:tasks.rados.rados.0.trial115.stdout:606: finishing write tid 1 to trial11545420-46 2026-02-17T05:49:33.552 INFO:tasks.rados.rados.0.trial115.stdout:607: seq_num 264 ranges {437048=621738,1832305=404425} 2026-02-17T05:49:33.555 INFO:tasks.rados.rados.0.trial115.stdout:607: writing trial11545420-28 from 437048 to 1058786 tid 1 2026-02-17T05:49:33.560 INFO:tasks.rados.rados.0.trial115.stdout:607: writing trial11545420-28 from 1832305 to 2236730 tid 2 2026-02-17T05:49:33.560 INFO:tasks.rados.rados.0.trial115.stdout:604: done (3 left) 2026-02-17T05:49:33.560 INFO:tasks.rados.rados.0.trial115.stdout:608: snap_create 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:605: finishing copy_from racing read to trial11545420-43 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:606: finishing write tid 2 to trial11545420-46 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:606: finishing write tid 3 to trial11545420-46 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:606: finishing write tid 4 to trial11545420-46 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:606: oid 46 updating version 0 to 210 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:606: oid 46 version 210 is already newer than 208 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:606: oid 46 version 210 is already newer than 209 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 210 (ObjNum 263 snap 53 seq_num 263) dirty exists 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:606: left oid 46 (ObjNum 263 snap 53 seq_num 263) 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:605: finishing copy_from to trial11545420-43 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 288 (ObjNum 261 snap 52 seq_num 261) dirty exists 2026-02-17T05:49:33.561 INFO:tasks.rados.rados.0.trial115.stdout:607: finishing write tid 1 to trial11545420-28 2026-02-17T05:49:33.562 INFO:tasks.rados.rados.0.trial115.stdout:607: finishing write tid 2 to trial11545420-28 2026-02-17T05:49:33.562 INFO:tasks.rados.rados.0.trial115.stdout:607: finishing write tid 3 to trial11545420-28 2026-02-17T05:49:33.562 INFO:tasks.rados.rados.0.trial115.stdout:607: finishing write tid 4 to trial11545420-28 2026-02-17T05:49:33.562 INFO:tasks.rados.rados.0.trial115.stdout:607: oid 28 updating version 0 to 214 2026-02-17T05:49:33.562 INFO:tasks.rados.rados.0.trial115.stdout:607: oid 28 version 214 is already newer than 213 2026-02-17T05:49:33.562 INFO:tasks.rados.rados.0.trial115.stdout:607: oid 28 version 214 is already newer than 212 2026-02-17T05:49:33.562 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 214 (ObjNum 264 snap 53 seq_num 264) dirty exists 2026-02-17T05:49:33.562 INFO:tasks.rados.rados.0.trial115.stdout:607: left oid 28 (ObjNum 264 snap 53 seq_num 264) 2026-02-17T05:49:34.544 INFO:tasks.rados.rados.0.trial115.stdout:605: done (3 left) 2026-02-17T05:49:34.544 INFO:tasks.rados.rados.0.trial115.stdout:606: done (2 left) 2026-02-17T05:49:34.544 INFO:tasks.rados.rados.0.trial115.stdout:607: done (1 left) 2026-02-17T05:49:34.544 INFO:tasks.rados.rados.0.trial115.stdout:608: done (0 left) 2026-02-17T05:49:34.544 INFO:tasks.rados.rados.0.trial115.stdout:609: rollback oid 15 current snap is 54 2026-02-17T05:49:34.544 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 15 to 52 2026-02-17T05:49:34.544 INFO:tasks.rados.rados.0.trial115.stdout:610: write oid 22 current snap is 54 2026-02-17T05:49:34.544 INFO:tasks.rados.rados.0.trial115.stdout:610: seq_num 265 ranges {453855=696987,1890787=548083} 2026-02-17T05:49:34.548 INFO:tasks.rados.rados.0.trial115.stdout:610: writing trial11545420-22 from 453855 to 1150842 tid 1 2026-02-17T05:49:34.553 INFO:tasks.rados.rados.0.trial115.stdout:610: writing trial11545420-22 from 1890787 to 2438870 tid 2 2026-02-17T05:49:34.553 INFO:tasks.rados.rados.0.trial115.stdout:611: snap_remove snap 53 2026-02-17T05:49:35.534 INFO:tasks.rados.rados.0.trial115.stdout:611: done (2 left) 2026-02-17T05:49:35.534 INFO:tasks.rados.rados.0.trial115.stdout:612: read oid 37 snap -1 2026-02-17T05:49:35.534 INFO:tasks.rados.rados.0.trial115.stdout:612: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:49:35.534 INFO:tasks.rados.rados.0.trial115.stdout:613: write oid 12 current snap is 54 2026-02-17T05:49:35.534 INFO:tasks.rados.rados.0.trial115.stdout:613: seq_num 266 ranges {470662=772236,1832916=408094} 2026-02-17T05:49:35.538 INFO:tasks.rados.rados.0.trial115.stdout:613: writing trial11545420-12 from 470662 to 1242898 tid 1 2026-02-17T05:49:35.542 INFO:tasks.rados.rados.0.trial115.stdout:613: writing trial11545420-12 from 1832916 to 2241010 tid 2 2026-02-17T05:49:35.542 INFO:tasks.rados.rados.0.trial115.stdout:614: write (excl) oid 20 current snap is 54 2026-02-17T05:49:35.542 INFO:tasks.rados.rados.0.trial115.stdout:614: seq_num 267 ranges {487469=563838,1607751=551752} 2026-02-17T05:49:35.546 INFO:tasks.rados.rados.0.trial115.stdout:614: writing trial11545420-20 from 487469 to 1051307 tid 1 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:614: writing trial11545420-20 from 1607751 to 2159503 tid 2 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:615: copy_from oid 8 from oid 40 current snap is 54 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:609: finishing rollback tid 0 to trial11545420-15 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:609: finishing rollback tid 1 to trial11545420-15 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:609: finishing rollback tid 2 to trial11545420-15 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 218 (ObjNum 250 snap 49 seq_num 250) dirty exists 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:610: finishing write tid 1 to trial11545420-22 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:610: finishing write tid 2 to trial11545420-22 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:610: finishing write tid 3 to trial11545420-22 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:610: finishing write tid 4 to trial11545420-22 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:610: oid 22 updating version 0 to 175 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:610: oid 22 version 175 is already newer than 174 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:610: oid 22 updating version 175 to 176 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 176 (ObjNum 265 snap 54 seq_num 265) dirty exists 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:610: left oid 22 (ObjNum 265 snap 54 seq_num 265) 2026-02-17T05:49:35.550 INFO:tasks.rados.rados.0.trial115.stdout:609: done (5 left) 2026-02-17T05:49:35.551 INFO:tasks.rados.rados.0.trial115.stdout:610: done (4 left) 2026-02-17T05:49:35.551 INFO:tasks.rados.rados.0.trial115.stdout:616: write oid 44 current snap is 54 2026-02-17T05:49:35.551 INFO:tasks.rados.rados.0.trial115.stdout:616: seq_num 268 ranges {504276=639087,1666233=695410} 2026-02-17T05:49:35.554 INFO:tasks.rados.rados.0.trial115.stdout:616: writing trial11545420-44 from 504276 to 1143363 tid 1 2026-02-17T05:49:35.559 INFO:tasks.rados.rados.0.trial115.stdout:616: writing trial11545420-44 from 1666233 to 2361643 tid 2 2026-02-17T05:49:35.559 INFO:tasks.rados.rados.0.trial115.stdout:617: write (excl) oid 9 current snap is 54 2026-02-17T05:49:35.559 INFO:tasks.rados.rados.0.trial115.stdout:617: seq_num 269 ranges {521083=714336,1724715=555421} 2026-02-17T05:49:35.563 INFO:tasks.rados.rados.0.trial115.stdout:617: writing trial11545420-9 from 521083 to 1235419 tid 1 2026-02-17T05:49:35.567 INFO:tasks.rados.rados.0.trial115.stdout:617: writing trial11545420-9 from 1724715 to 2280136 tid 2 2026-02-17T05:49:35.567 INFO:tasks.rados.rados.0.trial115.stdout:618: write (excl) oid 36 current snap is 54 2026-02-17T05:49:35.567 INFO:tasks.rados.rados.0.trial115.stdout:612: expect (ObjNum 78 snap 8 seq_num 78) 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:613: finishing write tid 1 to trial11545420-12 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:613: finishing write tid 2 to trial11545420-12 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:613: finishing write tid 3 to trial11545420-12 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:613: finishing write tid 4 to trial11545420-12 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:613: oid 12 updating version 0 to 291 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:613: oid 12 version 291 is already newer than 290 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:613: oid 12 updating version 291 to 292 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 292 (ObjNum 266 snap 54 seq_num 266) dirty exists 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:613: left oid 12 (ObjNum 266 snap 54 seq_num 266) 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:614: finishing write tid 1 to trial11545420-20 2026-02-17T05:49:35.677 INFO:tasks.rados.rados.0.trial115.stdout:618: seq_num 270 ranges {537890=789585,1783197=699079} 2026-02-17T05:49:35.681 INFO:tasks.rados.rados.0.trial115.stdout:618: writing trial11545420-36 from 537890 to 1327475 tid 1 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:618: writing trial11545420-36 from 1783197 to 2482276 tid 2 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:612: done (6 left) 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:613: done (5 left) 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:619: copy_from oid 6 from oid 45 current snap is 54 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:614: finishing write tid 2 to trial11545420-20 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:615: finishing copy_from to trial11545420-8 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:615: got expected ENOENT (src dne) 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:615: finishing copy_from racing read to trial11545420-8 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:614: finishing write tid 3 to trial11545420-20 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:614: finishing write tid 4 to trial11545420-20 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:614: oid 20 updating version 0 to 290 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:614: oid 20 version 290 is already newer than 289 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:614: oid 20 updating version 290 to 291 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 291 (ObjNum 267 snap 54 seq_num 267) dirty exists 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:614: left oid 20 (ObjNum 267 snap 54 seq_num 267) 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:616: finishing write tid 1 to trial11545420-44 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:616: finishing write tid 2 to trial11545420-44 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:616: finishing write tid 3 to trial11545420-44 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:616: finishing write tid 4 to trial11545420-44 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:616: oid 44 updating version 0 to 73 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:616: oid 44 version 73 is already newer than 71 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:616: oid 44 version 73 is already newer than 72 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 73 (ObjNum 268 snap 54 seq_num 268) dirty exists 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:616: left oid 44 (ObjNum 268 snap 54 seq_num 268) 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:617: finishing write tid 1 to trial11545420-9 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:617: finishing write tid 2 to trial11545420-9 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:617: finishing write tid 3 to trial11545420-9 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:617: finishing write tid 4 to trial11545420-9 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:617: oid 9 updating version 0 to 224 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:617: oid 9 version 224 is already newer than 222 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:617: oid 9 version 224 is already newer than 223 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 224 (ObjNum 269 snap 54 seq_num 269) dirty exists 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:617: left oid 9 (ObjNum 269 snap 54 seq_num 269) 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:614: done (5 left) 2026-02-17T05:49:35.688 INFO:tasks.rados.rados.0.trial115.stdout:615: done (4 left) 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:616: done (3 left) 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:617: done (2 left) 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:620: read oid 23 snap -1 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:620: expect (ObjNum 260 snap 52 seq_num 260) 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:621: rollback oid 16 current snap is 54 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 16 to 52 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:622: read oid 27 snap -1 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:622: expect (ObjNum 253 snap 49 seq_num 253) 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:623: rollback oid 14 current snap is 54 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 52 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:624: delete oid 34 current snap is 54 2026-02-17T05:49:35.689 INFO:tasks.rados.rados.0.trial115.stdout:618: finishing write tid 1 to trial11545420-36 2026-02-17T05:49:35.690 INFO:tasks.rados.rados.0.trial115.stdout:624: done (6 left) 2026-02-17T05:49:35.690 INFO:tasks.rados.rados.0.trial115.stdout:625: snap_create 2026-02-17T05:49:35.691 INFO:tasks.rados.rados.0.trial115.stdout:621: finishing rollback tid 0 to trial11545420-16 2026-02-17T05:49:35.692 INFO:tasks.rados.rados.0.trial115.stdout:622: expect (ObjNum 253 snap 49 seq_num 253) 2026-02-17T05:49:35.758 INFO:tasks.rados.rados.0.trial115.stdout:621: finishing rollback tid 1 to trial11545420-16 2026-02-17T05:49:35.758 INFO:tasks.rados.rados.0.trial115.stdout:621: finishing rollback tid 2 to trial11545420-16 2026-02-17T05:49:35.758 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 263 (ObjNum 251 snap 49 seq_num 251) dirty exists 2026-02-17T05:49:35.758 INFO:tasks.rados.rados.0.trial115.stdout:619: finishing copy_from racing read to trial11545420-6 2026-02-17T05:49:35.759 INFO:tasks.rados.rados.0.trial115.stdout:620: expect (ObjNum 260 snap 52 seq_num 260) 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:618: finishing write tid 2 to trial11545420-36 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:618: finishing write tid 3 to trial11545420-36 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:618: finishing write tid 4 to trial11545420-36 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:618: oid 36 updating version 0 to 173 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:618: oid 36 version 173 is already newer than 172 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:618: oid 36 version 173 is already newer than 171 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 173 (ObjNum 270 snap 54 seq_num 270) dirty exists 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:618: left oid 36 (ObjNum 270 snap 54 seq_num 270) 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:623: finishing rollback tid 0 to trial11545420-14 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:623: finishing rollback tid 1 to trial11545420-14 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:623: finishing rollback tid 2 to trial11545420-14 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 176 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:619: finishing copy_from to trial11545420-6 2026-02-17T05:49:35.829 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 283 (ObjNum 201 snap 41 seq_num 201) dirty exists 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:618: done (6 left) 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:619: done (5 left) 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:620: done (4 left) 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:621: done (3 left) 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:622: done (2 left) 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:623: done (1 left) 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:625: done (0 left) 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:626: read oid 43 snap -1 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:626: expect (ObjNum 261 snap 52 seq_num 261) 2026-02-17T05:49:36.548 INFO:tasks.rados.rados.0.trial115.stdout:627: snap_create 2026-02-17T05:49:36.553 INFO:tasks.rados.rados.0.trial115.stdout:626: expect (ObjNum 261 snap 52 seq_num 261) 2026-02-17T05:49:37.550 INFO:tasks.rados.rados.0.trial115.stdout:626: done (1 left) 2026-02-17T05:49:37.550 INFO:tasks.rados.rados.0.trial115.stdout:627: done (0 left) 2026-02-17T05:49:37.550 INFO:tasks.rados.rados.0.trial115.stdout:628: copy_from oid 7 from oid 14 current snap is 56 2026-02-17T05:49:37.550 INFO:tasks.rados.rados.0.trial115.stdout:629: write oid 46 current snap is 56 2026-02-17T05:49:37.550 INFO:tasks.rados.rados.0.trial115.stdout:629: seq_num 272 ranges {571504=540083,1783808=702748} 2026-02-17T05:49:37.554 INFO:tasks.rados.rados.0.trial115.stdout:629: writing trial11545420-46 from 571504 to 1111587 tid 1 2026-02-17T05:49:37.559 INFO:tasks.rados.rados.0.trial115.stdout:629: writing trial11545420-46 from 1783808 to 2486556 tid 2 2026-02-17T05:49:37.559 INFO:tasks.rados.rados.0.trial115.stdout:630: snap_remove snap 54 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:630: done (2 left) 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:631: rollback oid 43 current snap is 56 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 55 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:629: finishing write tid 1 to trial11545420-46 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:628: finishing copy_from racing read to trial11545420-7 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:628: finishing copy_from to trial11545420-7 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 176 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:629: finishing write tid 2 to trial11545420-46 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:629: finishing write tid 3 to trial11545420-46 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:629: finishing write tid 4 to trial11545420-46 2026-02-17T05:49:38.553 INFO:tasks.rados.rados.0.trial115.stdout:629: oid 46 updating version 0 to 226 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:629: oid 46 updating version 226 to 227 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:629: oid 46 updating version 227 to 228 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 228 (ObjNum 272 snap 56 seq_num 272) dirty exists 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:629: left oid 46 (ObjNum 272 snap 56 seq_num 272) 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:628: done (2 left) 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:629: done (1 left) 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:632: copy_from oid 44 from oid 16 current snap is 56 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:633: read oid 41 snap -1 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:633: expect (ObjNum 215 snap 45 seq_num 215) 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:634: write oid 31 current snap is 56 2026-02-17T05:49:38.554 INFO:tasks.rados.rados.0.trial115.stdout:634: seq_num 273 ranges {588311=615332,1842290=562759} 2026-02-17T05:49:38.558 INFO:tasks.rados.rados.0.trial115.stdout:634: writing trial11545420-31 from 588311 to 1203643 tid 1 2026-02-17T05:49:38.563 INFO:tasks.rados.rados.0.trial115.stdout:634: writing trial11545420-31 from 1842290 to 2405049 tid 2 2026-02-17T05:49:38.563 INFO:tasks.rados.rados.0.trial115.stdout:635: snap_create 2026-02-17T05:49:38.563 INFO:tasks.rados.rados.0.trial115.stdout:631: finishing rollback tid 0 to trial11545420-43 2026-02-17T05:49:38.563 INFO:tasks.rados.rados.0.trial115.stdout:631: finishing rollback tid 1 to trial11545420-43 2026-02-17T05:49:38.563 INFO:tasks.rados.rados.0.trial115.stdout:631: finishing rollback tid 2 to trial11545420-43 2026-02-17T05:49:38.563 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 294 (ObjNum 261 snap 52 seq_num 261) dirty exists 2026-02-17T05:49:38.563 INFO:tasks.rados.rados.0.trial115.stdout:634: finishing write tid 1 to trial11545420-31 2026-02-17T05:49:38.564 INFO:tasks.rados.rados.0.trial115.stdout:632: finishing copy_from racing read to trial11545420-44 2026-02-17T05:49:38.564 INFO:tasks.rados.rados.0.trial115.stdout:633: expect (ObjNum 215 snap 45 seq_num 215) 2026-02-17T05:49:38.681 INFO:tasks.rados.rados.0.trial115.stdout:632: finishing copy_from to trial11545420-44 2026-02-17T05:49:38.681 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 263 (ObjNum 251 snap 49 seq_num 251) dirty exists 2026-02-17T05:49:38.682 INFO:tasks.rados.rados.0.trial115.stdout:634: finishing write tid 2 to trial11545420-31 2026-02-17T05:49:38.682 INFO:tasks.rados.rados.0.trial115.stdout:634: finishing write tid 3 to trial11545420-31 2026-02-17T05:49:38.682 INFO:tasks.rados.rados.0.trial115.stdout:634: finishing write tid 4 to trial11545420-31 2026-02-17T05:49:38.682 INFO:tasks.rados.rados.0.trial115.stdout:634: oid 31 updating version 0 to 263 2026-02-17T05:49:38.682 INFO:tasks.rados.rados.0.trial115.stdout:634: oid 31 version 263 is already newer than 262 2026-02-17T05:49:38.682 INFO:tasks.rados.rados.0.trial115.stdout:634: oid 31 version 263 is already newer than 261 2026-02-17T05:49:38.682 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 263 (ObjNum 273 snap 56 seq_num 273) dirty exists 2026-02-17T05:49:38.682 INFO:tasks.rados.rados.0.trial115.stdout:634: left oid 31 (ObjNum 273 snap 56 seq_num 273) 2026-02-17T05:49:39.563 INFO:tasks.rados.rados.0.trial115.stdout:631: done (4 left) 2026-02-17T05:49:39.563 INFO:tasks.rados.rados.0.trial115.stdout:632: done (3 left) 2026-02-17T05:49:39.563 INFO:tasks.rados.rados.0.trial115.stdout:633: done (2 left) 2026-02-17T05:49:39.564 INFO:tasks.rados.rados.0.trial115.stdout:634: done (1 left) 2026-02-17T05:49:39.564 INFO:tasks.rados.rados.0.trial115.stdout:635: done (0 left) 2026-02-17T05:49:39.564 INFO:tasks.rados.rados.0.trial115.stdout:636: write oid 24 current snap is 57 2026-02-17T05:49:39.564 INFO:tasks.rados.rados.0.trial115.stdout:636: seq_num 274 ranges {605118=406934,1617125=706417} 2026-02-17T05:49:39.567 INFO:tasks.rados.rados.0.trial115.stdout:636: writing trial11545420-24 from 605118 to 1012052 tid 1 2026-02-17T05:49:39.572 INFO:tasks.rados.rados.0.trial115.stdout:636: writing trial11545420-24 from 1617125 to 2323542 tid 2 2026-02-17T05:49:39.572 INFO:tasks.rados.rados.0.trial115.stdout:637: read oid 11 snap -1 2026-02-17T05:49:39.572 INFO:tasks.rados.rados.0.trial115.stdout:637: expect deleted 2026-02-17T05:49:39.572 INFO:tasks.rados.rados.0.trial115.stdout:638: read oid 26 snap -1 2026-02-17T05:49:39.573 INFO:tasks.rados.rados.0.trial115.stdout:638: expect (ObjNum 215 snap 45 seq_num 215) 2026-02-17T05:49:39.573 INFO:tasks.rados.rados.0.trial115.stdout:639: write oid 21 current snap is 57 2026-02-17T05:49:39.573 INFO:tasks.rados.rados.0.trial115.stdout:639: seq_num 275 ranges {621925=482183,1559254=566428} 2026-02-17T05:49:39.576 INFO:tasks.rados.rados.0.trial115.stdout:639: writing trial11545420-21 from 621925 to 1104108 tid 1 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:639: writing trial11545420-21 from 1559254 to 2125682 tid 2 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:640: rollback oid 2 current snap is 57 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 2 to 52 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:636: finishing write tid 1 to trial11545420-24 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:636: finishing write tid 2 to trial11545420-24 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:636: finishing write tid 3 to trial11545420-24 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:636: finishing write tid 4 to trial11545420-24 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:636: oid 24 updating version 0 to 293 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:636: oid 24 updating version 293 to 295 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:636: oid 24 version 295 is already newer than 294 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 295 (ObjNum 274 snap 57 seq_num 274) dirty exists 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:636: left oid 24 (ObjNum 274 snap 57 seq_num 274) 2026-02-17T05:49:39.580 INFO:tasks.rados.rados.0.trial115.stdout:638: expect (ObjNum 215 snap 45 seq_num 215) 2026-02-17T05:49:39.699 INFO:tasks.rados.rados.0.trial115.stdout:639: finishing write tid 1 to trial11545420-21 2026-02-17T05:49:39.699 INFO:tasks.rados.rados.0.trial115.stdout:636: done (4 left) 2026-02-17T05:49:39.699 INFO:tasks.rados.rados.0.trial115.stdout:637: done (3 left) 2026-02-17T05:49:39.699 INFO:tasks.rados.rados.0.trial115.stdout:638: done (2 left) 2026-02-17T05:49:39.699 INFO:tasks.rados.rados.0.trial115.stdout:641: snap_remove snap 56 2026-02-17T05:49:40.535 INFO:tasks.rados.rados.0.trial115.stdout:641: done (2 left) 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:642: snap_create 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:640: finishing rollback tid 0 to trial11545420-2 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:640: finishing rollback tid 1 to trial11545420-2 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:640: finishing rollback tid 2 to trial11545420-2 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 266 (ObjNum 248 snap 49 seq_num 248) dirty exists 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:639: finishing write tid 2 to trial11545420-21 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:639: finishing write tid 3 to trial11545420-21 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:639: finishing write tid 4 to trial11545420-21 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:639: oid 21 updating version 0 to 198 2026-02-17T05:49:40.536 INFO:tasks.rados.rados.0.trial115.stdout:639: oid 21 updating version 198 to 199 2026-02-17T05:49:40.537 INFO:tasks.rados.rados.0.trial115.stdout:639: oid 21 version 199 is already newer than 197 2026-02-17T05:49:40.537 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 199 (ObjNum 275 snap 57 seq_num 275) dirty exists 2026-02-17T05:49:40.537 INFO:tasks.rados.rados.0.trial115.stdout:639: left oid 21 (ObjNum 275 snap 57 seq_num 275) 2026-02-17T05:49:40.874 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T05:49:40.874 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-17T05:49:40.874 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-17T05:49:40.874 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-17T05:49:40.878 INFO:tasks.ceph.mon.c:Started 2026-02-17T05:49:40.878 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:49:40.878 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.946+0000 7f505e914640 10 monclient: get_monmap_and_config 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.946+0000 7f505e914640 10 monclient: build_initial_monmap 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.946+0000 7f505e914640 10 monclient: monmap: 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:40.947969+0000 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:40.947969+0000 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:40.947 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:40.949 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:40.949 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:40.949 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:40.950 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:40.950 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:40.950 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:40.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 1 Processor -- start 2026-02-17T05:49:40.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 1 -- start start 2026-02-17T05:49:40.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient: init 2026-02-17T05:49:40.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:40.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T05:49:40.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f505809ea90 0x7f50580a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:40.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient(hunting): picked mon.noname-c con 0x7f505809ea90 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f505809ee70 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 0x7f505813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient(hunting): picked mon.noname-b con 0x7f505813d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50580a0a20 con 0x7f505809ee70 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f50580a0d00 con 0x7f505809ea90 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f50580a0460 con 0x7f505813d640 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:40.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505e914640 10 monclient(hunting): authenticate will time out at 826.501638s 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 0x7f505813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 0x7f505813fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43800/0 (socket says 10.20.193.78:43800) 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 1 -- 10.20.193.78:0/1880323804 learned_addr learned my addr 10.20.193.78:0/1880323804 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 10 monclient(hunting): get_auth_request con 0x7f505813d640 auth_method 0 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 1 -- 10.20.193.78:0/1880323804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 msgr2=0x7f505813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 1 -- 10.20.193.78:0/1880323804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 msgr2=0x7f505813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 1 --2- 10.20.193.78:0/1880323804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 0x7f505813fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:40.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f505ce8a640 1 --2- 10.20.193.78:0/1880323804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 0x7f505813fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f5057fff640 1 --2- 10.20.193.78:0/1880323804 >> v2:10.20.193.115:3300/0 conn(0x7f505809ea90 0x7f50580a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f5057fff640 10 monclient(hunting): get_auth_request con 0x7f505809ea90 auth_method 0 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f5057fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f5057fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.947+0000 7f5057fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 10 monclient(hunting): handle_auth_done global_id 44230 payload 274 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 1 -- 10.20.193.78:0/1880323804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 msgr2=0x7f505813fa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 1 --2- 10.20.193.78:0/1880323804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 0x7f505813fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:40.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 1 -- 10.20.193.78:0/1880323804 >> v1:10.20.193.115:6789/0 conn(0x7f505809ee70 legacy=0x7f505809f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 1 monclient: found mon.noname-c 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.948+0000 7f5057fff640 1 -- 10.20.193.78:0/1880323804 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50580a0740 con 0x7f505809ea90 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5057fff640 1 -- 10.20.193.78:0/1880323804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505809ea90 msgr2=0x7f50580a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5057fff640 1 --2- 10.20.193.78:0/1880323804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505809ea90 0x7f50580a13d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50440098a0 tx=0x7f504402ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb054fd12764c89f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 1 -- 10.20.193.78:0/1880323804 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504403c070 con 0x7f505809ea90 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: dump: 2026-02-17T05:49:40.964 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: _finish_auth 0 2026-02-17T05:49:40.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 5 monclient: authenticate success, global_id 44230 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 1 -- 10.20.193.78:0/1880323804 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504402fc00 con 0x7f505809ea90 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 1 -- 10.20.193.78:0/1880323804 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044037440 con 0x7f505809ea90 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 4 set_mon_vals no callback set 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f5056ffd640 10 monclient: dump: 2026-02-17T05:49:40.966 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 10 monclient: shutdown 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:40.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 1 -- 10.20.193.78:0/1880323804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505809ea90 msgr2=0x7f50580a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 1 --2- 10.20.193.78:0/1880323804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505809ea90 0x7f50580a13d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50440098a0 tx=0x7f504402ed80 comp rx=0 tx=0).stop 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 1 -- 10.20.193.78:0/1880323804 shutdown_connections 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 1 --2- 10.20.193.78:0/1880323804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505813d640 0x7f505813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 1 --2- 10.20.193.78:0/1880323804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505809ea90 0x7f50580a13d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 1 -- 10.20.193.78:0/1880323804 >> 10.20.193.78:0/1880323804 conn(0x7f50580a7520 msgr2=0x7f50580a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 1 -- 10.20.193.78:0/1880323804 shutdown_connections 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.951+0000 7f505e914640 1 -- 10.20.193.78:0/1880323804 wait complete. 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient: build_initial_monmap 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient: monmap: 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:40.968 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 1 Processor -- start 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 1 -- start start 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient: init 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:40.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 0x7f50581cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient(hunting): picked mon.noname-c con 0x7f505809ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f505809ee70 0x7f5058073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient(hunting): picked mon.noname-a con 0x7f505809ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505813d640 0x7f5058073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient(hunting): picked mon.noname-b con 0x7f505813d640 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50580a6820 con 0x7f505809ee70 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50580a66a0 con 0x7f505813d640 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:40.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f505813ff70 con 0x7f505809ea90 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f505e914640 10 monclient(hunting): authenticate will time out at 826.506743s 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 0x7f50581cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 0x7f50581cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43816/0 (socket says 10.20.193.78:43816) 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 1 -- 10.20.193.78:0/421674062 learned_addr learned my addr 10.20.193.78:0/421674062 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 10 monclient(hunting): get_auth_request con 0x7f505809ea90 auth_method 0 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f50577fe640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f505809ee70 0x7f5058073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f50577fe640 10 monclient(hunting): get_auth_request con 0x7f505809ee70 auth_method 0 2026-02-17T05:49:40.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f50577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f50577fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f50577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 1 -- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 msgr2=0x7f50581cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 1 -- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 msgr2=0x7f50581cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 0x7f50581cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f5057fff640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 0x7f50581cf340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f50577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f50577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.952+0000 7f50577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f505ce8a640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505813d640 0x7f5058073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f505ce8a640 10 monclient(hunting): get_auth_request con 0x7f505813d640 auth_method 0 2026-02-17T05:49:40.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f505ce8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f505ce8a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f505ce8a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 10 monclient(hunting): handle_auth_done global_id 54186 payload 824 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 1 -- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 msgr2=0x7f50581cf340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 0x7f50581cf340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 1 -- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505813d640 msgr2=0x7f5058073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505813d640 0x7f5058073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:40.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.953+0000 7f50577fe640 1 -- 10.20.193.78:0/421674062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5058070910 con 0x7f505809ee70 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f50577fe640 1 -- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f505809ee70 msgr2=0x7f5058073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f50577fe640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f505809ee70 0x7f5058073680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f504800c450 tx=0x7f504800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 1 -- 10.20.193.78:0/421674062 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048016020 con 0x7f505809ee70 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: dump: 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:40.974 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: _finish_auth 0 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 1 -- 10.20.193.78:0/421674062 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5048012a40 con 0x7f505809ee70 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 5 monclient: authenticate success, global_id 54186 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 1 -- 10.20.193.78:0/421674062 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048012e30 con 0x7f505809ee70 2026-02-17T05:49:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f5054ff9640 10 monclient: dump: 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:40.976 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f50567fc640 4 set_mon_vals no callback set 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 10 monclient: _renew_subs 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5058070b80 con 0x7f505809ee70 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 10 monclient: _renew_subs 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5058071140 con 0x7f505809ee70 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:40.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.955+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f505806f460 con 0x7f505809ee70 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.956+0000 7f5054ff9640 1 -- 10.20.193.78:0/421674062 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f504801b400 con 0x7f505809ee70 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.956+0000 7f5054ff9640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803df00 0x7f50280403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.956+0000 7f5054ff9640 1 -- 10.20.193.78:0/421674062 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4807+0+0 (secure 0 0 0) 0x7f50480528f0 con 0x7f505809ee70 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.956+0000 7f5057fff640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803df00 0x7f50280403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.956+0000 7f5057fff640 10 monclient: get_auth_request con 0x7f502803df00 auth_method 0 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.958+0000 7f5054ff9640 1 -- 10.20.193.78:0/421674062 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5048020030 con 0x7f505809ee70 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.958+0000 7f5054ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.958+0000 7f5054ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:40.959+0000 7f5057fff640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803df00 0x7f50280403c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f504403a040 tx=0x7f5044033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.078+0000 7f505e914640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.078+0000 7f505e914640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.078+0000 7f505e914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.078+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f50580a13b0 con 0x7f505809ee70 2026-02-17T05:49:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.078+0000 7f5054ff9640 1 -- 10.20.193.78:0/421674062 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f504801b900 con 0x7f505809ee70 2026-02-17T05:49:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.078+0000 7f5054ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.078+0000 7f5054ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:41.079 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":38,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:41.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803df00 msgr2=0x7f50280403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:41.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803df00 0x7f50280403c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f504403a040 tx=0x7f5044033000 comp rx=0 tx=0).stop 2026-02-17T05:49:41.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 10 monclient: shutdown 2026-02-17T05:49:41.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f505809ee70 msgr2=0x7f5058073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f505809ee70 0x7f5058073680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f504800c450 tx=0x7f504800c920 comp rx=0 tx=0).stop 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 shutdown_connections 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803df00 0x7f50280403c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f505813d640 0x7f5058073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f505809ee70 0x7f5058073680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 --2- 10.20.193.78:0/421674062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505809ea90 0x7f50581cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 >> 10.20.193.78:0/421674062 conn(0x7f50580a7520 msgr2=0x7f505813c860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 shutdown_connections 2026-02-17T05:49:41.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:41.079+0000 7f505e914640 1 -- 10.20.193.78:0/421674062 wait complete. 2026-02-17T05:49:41.536 INFO:tasks.rados.rados.0.trial115.stdout:639: done (2 left) 2026-02-17T05:49:41.536 INFO:tasks.rados.rados.0.trial115.stdout:640: done (1 left) 2026-02-17T05:49:41.536 INFO:tasks.rados.rados.0.trial115.stdout:642: done (0 left) 2026-02-17T05:49:41.536 INFO:tasks.rados.rados.0.trial115.stdout:643: read oid 39 snap 57 2026-02-17T05:49:41.537 INFO:tasks.rados.rados.0.trial115.stdout:643: expect (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:49:41.537 INFO:tasks.rados.rados.0.trial115.stdout:644: read oid 29 snap -1 2026-02-17T05:49:41.537 INFO:tasks.rados.rados.0.trial115.stdout:644: expect (ObjNum 262 snap 53 seq_num 262) 2026-02-17T05:49:41.537 INFO:tasks.rados.rados.0.trial115.stdout:645: write oid 22 current snap is 58 2026-02-17T05:49:41.537 INFO:tasks.rados.rados.0.trial115.stdout:645: seq_num 276 ranges {638732=557432,1617736=710086} 2026-02-17T05:49:41.541 INFO:tasks.rados.rados.0.trial115.stdout:645: writing trial11545420-22 from 638732 to 1196164 tid 1 2026-02-17T05:49:41.545 INFO:tasks.rados.rados.0.trial115.stdout:645: writing trial11545420-22 from 1617736 to 2327822 tid 2 2026-02-17T05:49:41.545 INFO:tasks.rados.rados.0.trial115.stdout:646: snap_create 2026-02-17T05:49:41.546 INFO:tasks.rados.rados.0.trial115.stdout:643: expect (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:49:41.627 INFO:tasks.rados.rados.0.trial115.stdout:644: expect (ObjNum 262 snap 53 seq_num 262) 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:645: finishing write tid 1 to trial11545420-22 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:645: finishing write tid 2 to trial11545420-22 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:645: finishing write tid 3 to trial11545420-22 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:645: finishing write tid 4 to trial11545420-22 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:645: oid 22 updating version 0 to 180 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:645: oid 22 version 180 is already newer than 178 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:645: oid 22 version 180 is already newer than 179 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 180 (ObjNum 276 snap 58 seq_num 276) dirty exists 2026-02-17T05:49:41.747 INFO:tasks.rados.rados.0.trial115.stdout:645: left oid 22 (ObjNum 276 snap 58 seq_num 276) 2026-02-17T05:49:42.536 INFO:tasks.rados.rados.0.trial115.stdout:643: done (3 left) 2026-02-17T05:49:42.537 INFO:tasks.rados.rados.0.trial115.stdout:644: done (2 left) 2026-02-17T05:49:42.537 INFO:tasks.rados.rados.0.trial115.stdout:645: done (1 left) 2026-02-17T05:49:42.537 INFO:tasks.rados.rados.0.trial115.stdout:646: done (0 left) 2026-02-17T05:49:42.537 INFO:tasks.rados.rados.0.trial115.stdout:647: delete oid 17 current snap is 59 2026-02-17T05:49:42.539 INFO:tasks.rados.rados.0.trial115.stdout:647: done (0 left) 2026-02-17T05:49:42.539 INFO:tasks.rados.rados.0.trial115.stdout:648: snap_create 2026-02-17T05:49:43.537 INFO:tasks.rados.rados.0.trial115.stdout:648: done (0 left) 2026-02-17T05:49:43.537 INFO:tasks.rados.rados.0.trial115.stdout:649: write oid 33 current snap is 60 2026-02-17T05:49:43.537 INFO:tasks.rados.rados.0.trial115.stdout:649: seq_num 278 ranges {672346=707930,2134700=537646} 2026-02-17T05:49:43.542 INFO:tasks.rados.rados.0.trial115.stdout:649: writing trial11545420-33 from 672346 to 1380276 tid 1 2026-02-17T05:49:43.547 INFO:tasks.rados.rados.0.trial115.stdout:649: writing trial11545420-33 from 2134700 to 2672346 tid 2 2026-02-17T05:49:43.547 INFO:tasks.rados.rados.0.trial115.stdout:650: snap_create 2026-02-17T05:49:43.547 INFO:tasks.rados.rados.0.trial115.stdout:649: finishing write tid 1 to trial11545420-33 2026-02-17T05:49:43.549 INFO:tasks.rados.rados.0.trial115.stdout:649: finishing write tid 2 to trial11545420-33 2026-02-17T05:49:43.549 INFO:tasks.rados.rados.0.trial115.stdout:649: finishing write tid 3 to trial11545420-33 2026-02-17T05:49:43.549 INFO:tasks.rados.rados.0.trial115.stdout:649: finishing write tid 4 to trial11545420-33 2026-02-17T05:49:43.549 INFO:tasks.rados.rados.0.trial115.stdout:649: oid 33 updating version 0 to 179 2026-02-17T05:49:43.549 INFO:tasks.rados.rados.0.trial115.stdout:649: oid 33 version 179 is already newer than 178 2026-02-17T05:49:43.549 INFO:tasks.rados.rados.0.trial115.stdout:649: oid 33 version 179 is already newer than 177 2026-02-17T05:49:43.549 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 179 (ObjNum 278 snap 60 seq_num 278) dirty exists 2026-02-17T05:49:43.549 INFO:tasks.rados.rados.0.trial115.stdout:649: left oid 33 (ObjNum 278 snap 60 seq_num 278) 2026-02-17T05:49:44.088 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:44.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 10 monclient: get_monmap_and_config 2026-02-17T05:49:44.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 10 monclient: build_initial_monmap 2026-02-17T05:49:44.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 10 monclient: monmap: 2026-02-17T05:49:44.154 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:44.154 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:44.154 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:44.155519+0000 2026-02-17T05:49:44.154 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:44.155519+0000 2026-02-17T05:49:44.154 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:44.155 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:44.155 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:44.155 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:44.155 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:44.155 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:44.155 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:44.155 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:44.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 1 Processor -- start 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 1 -- start start 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 10 monclient: init 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f28d006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.154+0000 7f28d61cc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f28d009c4a0 0x7f28d013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 10 monclient(hunting): picked mon.noname-a con 0x7f28d009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f28d006d380 0x7f28d013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 10 monclient(hunting): picked mon.noname-c con 0x7f28d006d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:44.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28d006daf0 con 0x7f28d006c890 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f28d00a0880 con 0x7f28d009c4a0 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f28d009cc80 con 0x7f28d006d380 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28d61cc640 10 monclient(hunting): authenticate will time out at 829.709030s 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cf7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f28d006c890 0x7f28d006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:47980/0 (socket says 10.20.193.78:47980) 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cf7fe640 1 -- 10.20.193.78:0/2087169409 learned_addr learned my addr 10.20.193.78:0/2087169409 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28ceffd640 1 --2- 10.20.193.78:0/2087169409 >> v2:10.20.193.78:3300/0 conn(0x7f28d009c4a0 0x7f28d013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28ceffd640 10 monclient(hunting): get_auth_request con 0x7f28d009c4a0 auth_method 0 2026-02-17T05:49:44.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 1 --2- 10.20.193.78:0/2087169409 >> v2:10.20.193.115:3300/0 conn(0x7f28d006d380 0x7f28d013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient(hunting): get_auth_request con 0x7f28d006d380 auth_method 0 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient(hunting): handle_auth_done global_id 44236 payload 274 2026-02-17T05:49:44.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 1 -- 10.20.193.78:0/2087169409 >> v2:10.20.193.78:3300/0 conn(0x7f28d009c4a0 msgr2=0x7f28d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 1 --2- 10.20.193.78:0/2087169409 >> v2:10.20.193.78:3300/0 conn(0x7f28d009c4a0 0x7f28d013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 1 -- 10.20.193.78:0/2087169409 >> v1:10.20.193.78:6789/0 conn(0x7f28d006c890 legacy=0x7f28d006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 1 monclient: found mon.noname-c 2026-02-17T05:49:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:49:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.155+0000 7f28cffff640 1 -- 10.20.193.78:0/2087169409 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d009cf60 con 0x7f28d006d380 2026-02-17T05:49:44.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28cffff640 1 -- 10.20.193.78:0/2087169409 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 msgr2=0x7f28d013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28cffff640 1 --2- 10.20.193.78:0/2087169409 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 0x7f28d013d8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f28d006dcc0 tx=0x7f28c402f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f56cbbcf97b5a6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 1 -- 10.20.193.78:0/2087169409 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c403c070 con 0x7f28d006d380 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: dump: 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:44.168 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: _finish_auth 0 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 5 monclient: authenticate success, global_id 44236 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 1 -- 10.20.193.78:0/2087169409 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c4037440 con 0x7f28d006d380 2026-02-17T05:49:44.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 1 -- 10.20.193.78:0/2087169409 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4037810 con 0x7f28d006d380 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 4 set_mon_vals no callback set 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28ce7fc640 10 monclient: dump: 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:44.170 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 10 monclient: shutdown 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 1 -- 10.20.193.78:0/2087169409 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 msgr2=0x7f28d013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 1 --2- 10.20.193.78:0/2087169409 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 0x7f28d013d8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f28d006dcc0 tx=0x7f28c402f150 comp rx=0 tx=0).stop 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 1 -- 10.20.193.78:0/2087169409 shutdown_connections 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 1 --2- 10.20.193.78:0/2087169409 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 0x7f28d013d8f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 1 --2- 10.20.193.78:0/2087169409 >> v2:10.20.193.78:3300/0 conn(0x7f28d009c4a0 0x7f28d013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 1 -- 10.20.193.78:0/2087169409 >> 10.20.193.78:0/2087169409 conn(0x7f28d00a6620 msgr2=0x7f28d00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 1 -- 10.20.193.78:0/2087169409 shutdown_connections 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.158+0000 7f28d61cc640 1 -- 10.20.193.78:0/2087169409 wait complete. 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient: build_initial_monmap 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient: monmap: 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:44.172 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 1 Processor -- start 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 1 -- start start 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient: init 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28d006c890 0x7f28d00728b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient(hunting): picked mon.noname-a con 0x7f28d006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 0x7f28d011cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient(hunting): picked mon.noname-b con 0x7f28d006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 0x7f28d011d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:44.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient(hunting): picked mon.noname-c con 0x7f28d009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28d009bfa0 con 0x7f28d006c890 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f28d009be20 con 0x7f28d006d380 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f28d013de30 con 0x7f28d009c4a0 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28d61cc640 10 monclient(hunting): authenticate will time out at 829.713702s 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28d006c890 0x7f28d00728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28d006c890 0x7f28d00728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:43532/0 (socket says 10.20.193.78:43532) 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 -- 10.20.193.78:0/3116865381 learned_addr learned my addr 10.20.193.78:0/3116865381 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:44.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient(hunting): get_auth_request con 0x7f28d006c890 auth_method 0 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 0x7f28d011d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 10 monclient(hunting): get_auth_request con 0x7f28d009c4a0 auth_method 0 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:44.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 1 -- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 msgr2=0x7f28d011d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 1 -- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 msgr2=0x7f28d011d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 0x7f28d011d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cffff640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 0x7f28d011d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28ceffd640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 0x7f28d011cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28ceffd640 10 monclient(hunting): get_auth_request con 0x7f28d006d380 auth_method 0 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient(hunting): handle_auth_done global_id 54192 payload 824 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 -- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 msgr2=0x7f28d011d1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 0x7f28d011d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 -- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 msgr2=0x7f28d011cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 0x7f28d011cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.159+0000 7f28cf7fe640 1 -- 10.20.193.78:0/3116865381 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d009deb0 con 0x7f28d006c890 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28cf7fe640 1 -- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28d006c890 msgr2=0x7f28d00728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28cf7fe640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28d006c890 0x7f28d00728b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c750 tx=0x7f28bc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 1 -- 10.20.193.78:0/3116865381 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc005de0 con 0x7f28d006c890 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: dump: 2026-02-17T05:49:44.177 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: _finish_auth 0 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:44.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 1 -- 10.20.193.78:0/3116865381 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28bc007070 con 0x7f28d006c890 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 5 monclient: authenticate success, global_id 54192 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 10 monclient: _renew_subs 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d009d810 con 0x7f28d006c890 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 1 -- 10.20.193.78:0/3116865381 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc007440 con 0x7f28d006c890 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28cdffb640 4 set_mon_vals no callback set 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 10 monclient: dump: 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:44.179 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 10 monclient: _renew_subs 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d011f4f0 con 0x7f28d006c890 2026-02-17T05:49:44.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d0073b30 con 0x7f28d006c890 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.162+0000 7f28abfff640 1 -- 10.20.193.78:0/3116865381 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f28bc007660 con 0x7f28d006c890 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.163+0000 7f28abfff640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f28a003deb0 0x7f28a0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.163+0000 7f28abfff640 1 -- 10.20.193.78:0/3116865381 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4751+0+0 (secure 0 0 0) 0x7f28bc052690 con 0x7f28d006c890 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.163+0000 7f28ceffd640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f28a003deb0 0x7f28a0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.163+0000 7f28ceffd640 10 monclient: get_auth_request con 0x7f28a003deb0 auth_method 0 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.165+0000 7f28ceffd640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f28a003deb0 0x7f28a0040370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28d011e090 tx=0x7f28c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.165+0000 7f28abfff640 1 -- 10.20.193.78:0/3116865381 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f28d0073b30 con 0x7f28d006c890 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.165+0000 7f28abfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:44.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.165+0000 7f28abfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:44.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.279+0000 7f28d61cc640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:44.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.279+0000 7f28d61cc640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:44.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.279+0000 7f28d61cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:44.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.279+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f28d0074980 con 0x7f28d006c890 2026-02-17T05:49:44.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.279+0000 7f28abfff640 1 -- 10.20.193.78:0/3116865381 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f28bc010de0 con 0x7f28d006c890 2026-02-17T05:49:44.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.279+0000 7f28abfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:44.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.279+0000 7f28abfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:44.281 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":38,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f28a003deb0 msgr2=0x7f28a0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f28a003deb0 0x7f28a0040370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28d011e090 tx=0x7f28c0008040 comp rx=0 tx=0).stop 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 10 monclient: shutdown 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28d006c890 msgr2=0x7f28d00728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28d006c890 0x7f28d00728b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c750 tx=0x7f28bc00cc20 comp rx=0 tx=0).stop 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 shutdown_connections 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28d009c4a0 0x7f28d011d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f28a003deb0 0x7f28a0040370 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28d006d380 0x7f28d011cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 --2- 10.20.193.78:0/3116865381 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28d006c890 0x7f28d00728b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:44.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 >> 10.20.193.78:0/3116865381 conn(0x7f28d00a6620 msgr2=0x7f28d0133ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:44.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 shutdown_connections 2026-02-17T05:49:44.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:44.280+0000 7f28d61cc640 1 -- 10.20.193.78:0/3116865381 wait complete. 2026-02-17T05:49:44.539 INFO:tasks.rados.rados.0.trial115.stdout:649: done (1 left) 2026-02-17T05:49:44.539 INFO:tasks.rados.rados.0.trial115.stdout:650: done (0 left) 2026-02-17T05:49:44.539 INFO:tasks.rados.rados.0.trial115.stdout:651: copy_from oid 27 from oid 10 current snap is 61 2026-02-17T05:49:44.539 INFO:tasks.rados.rados.0.trial115.stdout:652: write oid 17 current snap is 61 2026-02-17T05:49:44.539 INFO:tasks.rados.rados.0.trial115.stdout:652: seq_num 279 ranges {689153=783179,2076829=457413} 2026-02-17T05:49:44.544 INFO:tasks.rados.rados.0.trial115.stdout:652: writing trial11545420-17 from 689153 to 1472332 tid 1 2026-02-17T05:49:44.549 INFO:tasks.rados.rados.0.trial115.stdout:652: writing trial11545420-17 from 2076829 to 2534242 tid 2 2026-02-17T05:49:44.549 INFO:tasks.rados.rados.0.trial115.stdout:653: snap_create 2026-02-17T05:49:44.549 INFO:tasks.rados.rados.0.trial115.stdout:651: finishing copy_from racing read to trial11545420-27 2026-02-17T05:49:44.549 INFO:tasks.rados.rados.0.trial115.stdout:651: finishing copy_from to trial11545420-27 2026-02-17T05:49:44.549 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 189 (ObjNum 216 snap 45 seq_num 216) dirty exists 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:652: finishing write tid 1 to trial11545420-17 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:652: finishing write tid 2 to trial11545420-17 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:652: finishing write tid 3 to trial11545420-17 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:652: finishing write tid 4 to trial11545420-17 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:652: oid 17 updating version 0 to 73 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:652: oid 17 updating version 73 to 74 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:652: oid 17 version 74 is already newer than 72 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 74 (ObjNum 279 snap 61 seq_num 279) dirty exists 2026-02-17T05:49:44.556 INFO:tasks.rados.rados.0.trial115.stdout:652: left oid 17 (ObjNum 279 snap 61 seq_num 279) 2026-02-17T05:49:45.536 INFO:tasks.rados.rados.0.trial115.stdout:651: done (2 left) 2026-02-17T05:49:45.536 INFO:tasks.rados.rados.0.trial115.stdout:652: done (1 left) 2026-02-17T05:49:45.536 INFO:tasks.rados.rados.0.trial115.stdout:653: done (0 left) 2026-02-17T05:49:45.536 INFO:tasks.rados.rados.0.trial115.stdout:654: delete oid 27 current snap is 62 2026-02-17T05:49:45.542 INFO:tasks.rados.rados.0.trial115.stdout:654: done (0 left) 2026-02-17T05:49:45.542 INFO:tasks.rados.rados.0.trial115.stdout:655: copy_from oid 22 from oid 34 current snap is 62 2026-02-17T05:49:45.542 INFO:tasks.rados.rados.0.trial115.stdout:656: rollback oid 40 current snap is 62 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 60 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:657: read oid 23 snap -1 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:657: expect (ObjNum 260 snap 52 seq_num 260) 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:658: delete oid 27 current snap is 62 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:656: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 0 (ObjNum 129 snap 0 seq_num 1333687048) dirty dne 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:655: finishing copy_from to trial11545420-22 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:655: got expected ENOENT (src dne) 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:655: finishing copy_from racing read to trial11545420-22 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:655: done (3 left) 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:656: done (2 left) 2026-02-17T05:49:45.543 INFO:tasks.rados.rados.0.trial115.stdout:658: done (1 left) 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:659: read oid 22 snap 59 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:659: expect (ObjNum 276 snap 58 seq_num 276) 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:660: read oid 10 snap -1 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:660: expect (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:661: read oid 11 snap -1 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:661: expect deleted 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:662: rollback oid 44 current snap is 62 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 44 to 57 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:663: rollback oid 24 current snap is 62 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 55 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:664: read oid 3 snap 60 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:664: expect (ObjNum 196 snap 37 seq_num 196) 2026-02-17T05:49:45.544 INFO:tasks.rados.rados.0.trial115.stdout:665: read oid 40 snap -1 2026-02-17T05:49:45.545 INFO:tasks.rados.rados.0.trial115.stdout:665: expect deleted 2026-02-17T05:49:45.545 INFO:tasks.rados.rados.0.trial115.stdout:666: rollback oid 41 current snap is 62 2026-02-17T05:49:45.545 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 52 2026-02-17T05:49:45.545 INFO:tasks.rados.rados.0.trial115.stdout:667: write (excl) oid 35 current snap is 62 2026-02-17T05:49:45.545 INFO:tasks.rados.rados.0.trial115.stdout:667: seq_num 282 ranges {739574=725279,1968628=721093} 2026-02-17T05:49:45.547 INFO:tasks.rados.rados.0.trial115.stdout:667: writing trial11545420-35 from 739574 to 1464853 tid 1 2026-02-17T05:49:45.552 INFO:tasks.rados.rados.0.trial115.stdout:667: writing trial11545420-35 from 1968628 to 2689721 tid 2 2026-02-17T05:49:45.552 INFO:tasks.rados.rados.0.trial115.stdout:668: copy_from oid 9 from oid 19 current snap is 62 2026-02-17T05:49:45.552 INFO:tasks.rados.rados.0.trial115.stdout:662: finishing rollback tid 0 to trial11545420-44 2026-02-17T05:49:45.552 INFO:tasks.rados.rados.0.trial115.stdout:662: finishing rollback tid 1 to trial11545420-44 2026-02-17T05:49:45.552 INFO:tasks.rados.rados.0.trial115.stdout:662: finishing rollback tid 2 to trial11545420-44 2026-02-17T05:49:45.553 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 266 (ObjNum 251 snap 49 seq_num 251) dirty exists 2026-02-17T05:49:45.553 INFO:tasks.rados.rados.0.trial115.stdout:657: expect (ObjNum 260 snap 52 seq_num 260) 2026-02-17T05:49:45.622 INFO:tasks.rados.rados.0.trial115.stdout:660: expect (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:49:45.704 INFO:tasks.rados.rados.0.trial115.stdout:664: expect (ObjNum 196 snap 37 seq_num 196) 2026-02-17T05:49:45.836 INFO:tasks.rados.rados.0.trial115.stdout:663: finishing rollback tid 0 to trial11545420-24 2026-02-17T05:49:45.836 INFO:tasks.rados.rados.0.trial115.stdout:663: finishing rollback tid 1 to trial11545420-24 2026-02-17T05:49:45.836 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 298 (ObjNum 33 snap 0 seq_num 311723719) dirty dne 2026-02-17T05:49:45.836 INFO:tasks.rados.rados.0.trial115.stdout:659: expect (ObjNum 276 snap 58 seq_num 276) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:667: finishing write tid 1 to trial11545420-35 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:666: finishing rollback tid 0 to trial11545420-41 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:657: done (10 left) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:659: done (9 left) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:660: done (8 left) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:661: done (7 left) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:662: done (6 left) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:663: done (5 left) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:664: done (4 left) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:665: done (3 left) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:669: snap_create 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:666: finishing rollback tid 1 to trial11545420-41 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:666: finishing rollback tid 2 to trial11545420-41 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 205 (ObjNum 215 snap 45 seq_num 215) dirty exists 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:667: finishing write tid 2 to trial11545420-35 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:667: finishing write tid 3 to trial11545420-35 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:667: finishing write tid 4 to trial11545420-35 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:667: oid 35 updating version 0 to 264 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:667: oid 35 updating version 264 to 265 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:667: oid 35 updating version 265 to 266 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 266 (ObjNum 282 snap 62 seq_num 282) dirty exists 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:667: left oid 35 (ObjNum 282 snap 62 seq_num 282) 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:668: finishing copy_from racing read to trial11545420-9 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:668: finishing copy_from to trial11545420-9 2026-02-17T05:49:45.920 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 171 (ObjNum 106 snap 16 seq_num 106) dirty exists 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:666: done (3 left) 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:667: done (2 left) 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:668: done (1 left) 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:669: done (0 left) 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:670: read oid 8 snap -1 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:670: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:671: read oid 45 snap 55 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:671: expect (ObjNum 201 snap 41 seq_num 201) 2026-02-17T05:49:46.543 INFO:tasks.rados.rados.0.trial115.stdout:672: delete oid 1 current snap is 63 2026-02-17T05:49:46.545 INFO:tasks.rados.rados.0.trial115.stdout:671: expect (ObjNum 201 snap 41 seq_num 201) 2026-02-17T05:49:46.708 INFO:tasks.rados.rados.0.trial115.stdout:670: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:49:46.761 INFO:tasks.rados.rados.0.trial115.stdout:670: done (2 left) 2026-02-17T05:49:46.761 INFO:tasks.rados.rados.0.trial115.stdout:671: done (1 left) 2026-02-17T05:49:46.761 INFO:tasks.rados.rados.0.trial115.stdout:672: done (0 left) 2026-02-17T05:49:46.761 INFO:tasks.rados.rados.0.trial115.stdout:673: write (excl) oid 22 current snap is 63 2026-02-17T05:49:46.761 INFO:tasks.rados.rados.0.trial115.stdout:673: seq_num 284 ranges {773188=475777,1969239=724762} 2026-02-17T05:49:46.765 INFO:tasks.rados.rados.0.trial115.stdout:673: writing trial11545420-22 from 773188 to 1248965 tid 1 2026-02-17T05:49:46.771 INFO:tasks.rados.rados.0.trial115.stdout:673: writing trial11545420-22 from 1969239 to 2694001 tid 2 2026-02-17T05:49:46.771 INFO:tasks.rados.rados.0.trial115.stdout:674: snap_create 2026-02-17T05:49:46.771 INFO:tasks.rados.rados.0.trial115.stdout:673: finishing write tid 1 to trial11545420-22 2026-02-17T05:49:46.773 INFO:tasks.rados.rados.0.trial115.stdout:673: finishing write tid 2 to trial11545420-22 2026-02-17T05:49:46.773 INFO:tasks.rados.rados.0.trial115.stdout:673: finishing write tid 3 to trial11545420-22 2026-02-17T05:49:46.773 INFO:tasks.rados.rados.0.trial115.stdout:673: finishing write tid 4 to trial11545420-22 2026-02-17T05:49:46.773 INFO:tasks.rados.rados.0.trial115.stdout:673: oid 22 updating version 0 to 191 2026-02-17T05:49:46.773 INFO:tasks.rados.rados.0.trial115.stdout:673: oid 22 updating version 191 to 193 2026-02-17T05:49:46.774 INFO:tasks.rados.rados.0.trial115.stdout:673: oid 22 version 193 is already newer than 192 2026-02-17T05:49:46.774 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 193 (ObjNum 284 snap 63 seq_num 284) dirty exists 2026-02-17T05:49:46.774 INFO:tasks.rados.rados.0.trial115.stdout:673: left oid 22 (ObjNum 284 snap 63 seq_num 284) 2026-02-17T05:49:47.289 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:47.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient: get_monmap_and_config 2026-02-17T05:49:47.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient: build_initial_monmap 2026-02-17T05:49:47.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient: monmap: 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:47.358253+0000 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:47.358253+0000 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:47.360 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 1 Processor -- start 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 1 -- start start 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient: init 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f705806c890 0x7f705806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient(hunting): picked mon.noname-a con 0x7f705806c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f705809c4a0 0x7f705806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient(hunting): picked mon.noname-c con 0x7f705809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:47.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 0x7f7058135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient(hunting): picked mon.noname-b con 0x7f7058134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f70580a0880 con 0x7f705806c890 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f705806d6f0 con 0x7f705809c4a0 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f705806daf0 con 0x7f7058134df0 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f705dca0640 10 monclient(hunting): authenticate will time out at 832.911814s 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f705806c890 0x7f705806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f7057fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 0x7f7058135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f705806c890 0x7f705806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52612/0 (socket says 10.20.193.78:52612) 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 1 -- 10.20.193.78:0/1963207354 learned_addr learned my addr 10.20.193.78:0/1963207354 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 10 monclient(hunting): get_auth_request con 0x7f705806c890 auth_method 0 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f7057fff640 10 monclient(hunting): get_auth_request con 0x7f7058134df0 auth_method 0 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f7057fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f7057fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f7057fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f70577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:47.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f7057fff640 1 -- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 msgr2=0x7f7058135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f7057fff640 1 -- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 msgr2=0x7f7058135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.357+0000 7f7056ffd640 1 --2- 10.20.193.78:0/1963207354 >> v2:10.20.193.115:3300/0 conn(0x7f705809c4a0 0x7f705806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f7057fff640 1 --2- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 0x7f7058135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f7056ffd640 10 monclient(hunting): get_auth_request con 0x7f705809c4a0 auth_method 0 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f7056ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f7056ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f7056ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f7057fff640 1 --2- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 0x7f7058135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 10 monclient(hunting): handle_auth_done global_id 54195 payload 274 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 1 -- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 msgr2=0x7f7058135280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:47.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 1 --2- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 0x7f7058135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 1 -- 10.20.193.78:0/1963207354 >> v2:10.20.193.115:3300/0 conn(0x7f705809c4a0 msgr2=0x7f705806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:47.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 1 --2- 10.20.193.78:0/1963207354 >> v2:10.20.193.115:3300/0 conn(0x7f705809c4a0 0x7f705806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:47.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:47.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:47.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f70577fe640 1 -- 10.20.193.78:0/1963207354 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705809cc80 con 0x7f705806c890 2026-02-17T05:49:47.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.358+0000 7f7056ffd640 1 --2- 10.20.193.78:0/1963207354 >> v2:10.20.193.115:3300/0 conn(0x7f705809c4a0 0x7f705806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:49:47.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70577fe640 1 -- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705806c890 msgr2=0x7f705806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70577fe640 1 --2- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705806c890 0x7f705806cc70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f705809c980 tx=0x7f704402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5872dc885bbf72 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 1 -- 10.20.193.78:0/1963207354 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704403c070 con 0x7f705806c890 2026-02-17T05:49:47.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: dump: 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: _finish_auth 0 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: _check_auth_tickets 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 1 -- 10.20.193.78:0/1963207354 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704402fca0 con 0x7f705806c890 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f705dca0640 5 monclient: authenticate success, global_id 54195 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f705dca0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 1 -- 10.20.193.78:0/1963207354 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044037660 con 0x7f705806c890 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f705dca0640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f705dca0640 4 set_mon_vals no callback set 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:47.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.360+0000 7f70567fc640 10 monclient: dump: 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient: shutdown 2026-02-17T05:49:47.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705806c890 msgr2=0x7f705806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 --2- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705806c890 0x7f705806cc70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f705809c980 tx=0x7f704402ee60 comp rx=0 tx=0).stop 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- 10.20.193.78:0/1963207354 shutdown_connections 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 --2- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7058134df0 0x7f7058135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 --2- 10.20.193.78:0/1963207354 >> v2:10.20.193.115:3300/0 conn(0x7f705809c4a0 0x7f705806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 --2- 10.20.193.78:0/1963207354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705806c890 0x7f705806cc70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- 10.20.193.78:0/1963207354 >> 10.20.193.78:0/1963207354 conn(0x7f70580a6620 msgr2=0x7f70580a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- 10.20.193.78:0/1963207354 shutdown_connections 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- 10.20.193.78:0/1963207354 wait complete. 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient: build_initial_monmap 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient: monmap: 2026-02-17T05:49:47.375 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:47.376 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 Processor -- start 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- start start 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient: init 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 0x7f70581cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient(hunting): picked mon.noname-c con 0x7f705806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705809c4a0 0x7f7058133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient(hunting): picked mon.noname-a con 0x7f705809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7058134df0 0x7f7058134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient(hunting): picked mon.noname-b con 0x7f7058134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:47.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70580a01a0 con 0x7f705809c4a0 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70580a0020 con 0x7f7058134df0 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f705809be20 con 0x7f705806c890 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f705dca0640 10 monclient(hunting): authenticate will time out at 832.915933s 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f70577fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 0x7f70581cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f7056ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705809c4a0 0x7f7058133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f70577fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 0x7f70581cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36636/0 (socket says 10.20.193.78:36636) 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f70577fe640 1 -- 10.20.193.78:0/4172926038 learned_addr learned my addr 10.20.193.78:0/4172926038 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:47.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.361+0000 7f70577fe640 10 monclient(hunting): get_auth_request con 0x7f705806c890 auth_method 0 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f70577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f70577fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f70577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient(hunting): get_auth_request con 0x7f705809c4a0 auth_method 0 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f70577fe640 1 -- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 msgr2=0x7f70581cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f70577fe640 1 -- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 msgr2=0x7f70581cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f70577fe640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 0x7f70581cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:47.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f70577fe640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 0x7f70581cf150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient(hunting): handle_auth_done global_id 54198 payload 824 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7057fff640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7058134df0 0x7f7058134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 1 -- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 msgr2=0x7f70581cf150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 0x7f70581cf150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 1 -- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7058134df0 msgr2=0x7f7058134400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7058134df0 0x7f7058134400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 1 monclient: found mon.noname-a 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:47.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7056ffd640 1 -- 10.20.193.78:0/4172926038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7058127600 con 0x7f705809c4a0 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7057fff640 10 monclient: get_auth_request con 0x7f7058134df0 auth_method 0 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.362+0000 7f7057fff640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7058134df0 0x7f7058134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7056ffd640 1 -- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705809c4a0 msgr2=0x7f7058133ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7056ffd640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705809c4a0 0x7f7058133ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f704800e770 tx=0x7f704800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 1 -- 10.20.193.78:0/4172926038 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704800f040 con 0x7f705809c4a0 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: dump: 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.381 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: _finish_auth 0 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 1 -- 10.20.193.78:0/4172926038 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704800cae0 con 0x7f705809c4a0 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f705dca0640 5 monclient: authenticate success, global_id 54198 2026-02-17T05:49:47.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f705dca0640 10 monclient: _renew_subs 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f705dca0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70581cf690 con 0x7f705809c4a0 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 1 -- 10.20.193.78:0/4172926038 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048004030 con 0x7f705809c4a0 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7055ffb640 4 set_mon_vals no callback set 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f7033fff640 10 monclient: dump: 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.383 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f705dca0640 10 monclient: _renew_subs 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f705dca0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.364+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70580a1730 con 0x7f705809c4a0 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f705dca0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f705dca0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:47.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f705dca0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f7033fff640 1 -- 10.20.193.78:0/4172926038 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7048012050 con 0x7f705809c4a0 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7058068410 con 0x7f705809c4a0 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f7033fff640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f702803deb0 0x7f7028040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f7033fff640 1 -- 10.20.193.78:0/4172926038 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 4707+0+0 (secure 0 0 0) 0x7f7048052350 con 0x7f705809c4a0 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f70577fe640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f702803deb0 0x7f7028040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.365+0000 7f70577fe640 10 monclient: get_auth_request con 0x7f702803deb0 auth_method 0 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.367+0000 7f7033fff640 1 -- 10.20.193.78:0/4172926038 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7058068410 con 0x7f705809c4a0 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.367+0000 7f7033fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.367+0000 7f7033fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:47.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.367+0000 7f70577fe640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f702803deb0 0x7f7028040370 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f70440090b0 tx=0x7f7044039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.481+0000 7f705dca0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:47.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.481+0000 7f705dca0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:47.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.481+0000 7f705dca0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.481+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7058074980 con 0x7f705809c4a0 2026-02-17T05:49:47.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.482+0000 7f7033fff640 1 -- 10.20.193.78:0/4172926038 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f7048017020 con 0x7f705809c4a0 2026-02-17T05:49:47.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.482+0000 7f7033fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:47.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.482+0000 7f7033fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:47.483 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:47.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f702803deb0 msgr2=0x7f7028040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f702803deb0 0x7f7028040370 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f70440090b0 tx=0x7f7044039040 comp rx=0 tx=0).stop 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 10 monclient: shutdown 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705809c4a0 msgr2=0x7f7058133ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705809c4a0 0x7f7058133ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f704800e770 tx=0x7f704800ec40 comp rx=0 tx=0).stop 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 shutdown_connections 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f702803deb0 0x7f7028040370 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7058134df0 0x7f7058134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f705809c4a0 0x7f7058133ec0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 --2- 10.20.193.78:0/4172926038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f705806c890 0x7f70581cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 >> 10.20.193.78:0/4172926038 conn(0x7f70580a6620 msgr2=0x7f70580a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:47.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 shutdown_connections 2026-02-17T05:49:47.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.483+0000 7f705dca0640 1 -- 10.20.193.78:0/4172926038 wait complete. 2026-02-17T05:49:47.491 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:49:47.491 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:49:47.491 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:49:47.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.557+0000 7f19098e8640 10 monclient: get_monmap_and_config 2026-02-17T05:49:47.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.557+0000 7f19098e8640 10 monclient: build_initial_monmap 2026-02-17T05:49:47.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.557+0000 7f19098e8640 10 monclient: monmap: 2026-02-17T05:49:47.560 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:47.559007+0000 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:47.559007+0000 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:47.561 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 1 Processor -- start 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 1 -- start start 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient: init 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f190406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f190409c4a0 0x7f190413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient(hunting): picked mon.noname-a con 0x7f190409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f190406d380 0x7f190413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient(hunting): picked mon.noname-c con 0x7f190406d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190406daf0 con 0x7f190406c890 2026-02-17T05:49:47.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f19040a0880 con 0x7f190409c4a0 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f190409cc80 con 0x7f190406d380 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19098e8640 10 monclient(hunting): authenticate will time out at 833.112588s 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f1902ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f190406c890 0x7f190406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49116/0 (socket says 10.20.193.78:49116) 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f1902ffd640 1 -- 10.20.193.78:0/3327849045 learned_addr learned my addr 10.20.193.78:0/3327849045 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19027fc640 1 --2- 10.20.193.78:0/3327849045 >> v2:10.20.193.78:3300/0 conn(0x7f190409c4a0 0x7f190413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19027fc640 10 monclient(hunting): get_auth_request con 0x7f190409c4a0 auth_method 0 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19027fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19037fe640 1 --2- 10.20.193.78:0/3327849045 >> v2:10.20.193.115:3300/0 conn(0x7f190406d380 0x7f190413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19037fe640 10 monclient(hunting): get_auth_request con 0x7f190406d380 auth_method 0 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.558+0000 7f19037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 10 monclient(hunting): handle_auth_done global_id 44245 payload 274 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 1 -- 10.20.193.78:0/3327849045 >> v2:10.20.193.78:3300/0 conn(0x7f190409c4a0 msgr2=0x7f190413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 1 --2- 10.20.193.78:0/3327849045 >> v2:10.20.193.78:3300/0 conn(0x7f190409c4a0 0x7f190413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 1 -- 10.20.193.78:0/3327849045 >> v1:10.20.193.78:6789/0 conn(0x7f190406c890 legacy=0x7f190406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:47.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:47.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 1 monclient: found mon.noname-c 2026-02-17T05:49:47.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:47.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:49:47.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.559+0000 7f19037fe640 1 -- 10.20.193.78:0/3327849045 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190409cf60 con 0x7f190406d380 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f19037fe640 1 -- 10.20.193.78:0/3327849045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f190406d380 msgr2=0x7f190413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f19037fe640 1 --2- 10.20.193.78:0/3327849045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f190406d380 0x7f190413d8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f190406dcc0 tx=0x7f18f402f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643e108a3c6914f5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 1 -- 10.20.193.78:0/3327849045 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f403c070 con 0x7f190406d380 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: dump: 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.574 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 1 -- 10.20.193.78:0/3327849045 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f4037440 con 0x7f190406d380 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f19098e8640 5 monclient: authenticate success, global_id 44245 2026-02-17T05:49:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f19098e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 1 -- 10.20.193.78:0/3327849045 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4037810 con 0x7f190406d380 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f19098e8640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f19098e8640 4 set_mon_vals no callback set 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.561+0000 7f1901ffb640 10 monclient: dump: 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.576 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient: shutdown 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- 10.20.193.78:0/3327849045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f190406d380 msgr2=0x7f190413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 --2- 10.20.193.78:0/3327849045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f190406d380 0x7f190413d8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f190406dcc0 tx=0x7f18f402f150 comp rx=0 tx=0).stop 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- 10.20.193.78:0/3327849045 shutdown_connections 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 --2- 10.20.193.78:0/3327849045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f190406d380 0x7f190413d8f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 --2- 10.20.193.78:0/3327849045 >> v2:10.20.193.78:3300/0 conn(0x7f190409c4a0 0x7f190413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- 10.20.193.78:0/3327849045 >> 10.20.193.78:0/3327849045 conn(0x7f19040a6620 msgr2=0x7f19040a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- 10.20.193.78:0/3327849045 shutdown_connections 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- 10.20.193.78:0/3327849045 wait complete. 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient: build_initial_monmap 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient: monmap: 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:47.578 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 Processor -- start 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- start start 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient: init 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f190406c890 0x7f1904136540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient(hunting): picked mon.noname-b con 0x7f190406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f190406d380 0x7f1904136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient(hunting): picked mon.noname-c con 0x7f190406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:47.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190409c4a0 0x7f190411ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient(hunting): picked mon.noname-a con 0x7f190409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f190409bfa0 con 0x7f190409c4a0 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f190409be20 con 0x7f190406c890 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f190413de30 con 0x7f190406d380 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.562+0000 7f19098e8640 10 monclient(hunting): authenticate will time out at 833.116999s 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190409c4a0 0x7f190411ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f190406d380 0x7f1904136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190409c4a0 0x7f190411ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52654/0 (socket says 10.20.193.78:52654) 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 -- 10.20.193.78:0/1710244731 learned_addr learned my addr 10.20.193.78:0/1710244731 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient(hunting): get_auth_request con 0x7f190409c4a0 auth_method 0 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 10 monclient(hunting): get_auth_request con 0x7f190406d380 auth_method 0 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:47.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f190406d380 msgr2=0x7f1904136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f190406d380 msgr2=0x7f1904136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f190406d380 0x7f1904136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19027fc640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f190406d380 0x7f1904136a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient(hunting): handle_auth_done global_id 54204 payload 824 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f190406d380 msgr2=0x7f1904136a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f190406d380 0x7f1904136a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f190406c890 msgr2=0x7f1904136540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f190406c890 0x7f1904136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:47.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.563+0000 7f19037fe640 1 -- 10.20.193.78:0/1710244731 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1904122930 con 0x7f190409c4a0 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19037fe640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190409c4a0 msgr2=0x7f190411ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19037fe640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190409c4a0 0x7f190411ef60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18f403a040 tx=0x7f18f4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f403c050 con 0x7f190409c4a0 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: dump: 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.583 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: _finish_auth 0 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f4048070 con 0x7f190409c4a0 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19098e8640 5 monclient: authenticate success, global_id 54204 2026-02-17T05:49:47.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19098e8640 10 monclient: _renew_subs 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19098e8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19041214e0 con 0x7f190409c4a0 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4041c80 con 0x7f190409c4a0 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19017fa640 4 set_mon_vals no callback set 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f18dbfff640 10 monclient: dump: 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.585 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19098e8640 10 monclient: _renew_subs 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19098e8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.565+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190413da10 con 0x7f190409c4a0 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f19098e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f19098e8640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f19041cf290 0x7f190411f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f19098e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1904121760 con 0x7f19041cf290 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f19041cf290 0x7f190411f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f18f4043450 con 0x7f190409c4a0 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 10 monclient: get_auth_request con 0x7f19041cf290 auth_method 0 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 10 monclient(hunting): handle_auth_done global_id 54207 payload 824 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f1902ffd640 1 -- 10.20.193.78:0/1710244731 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1904074980 con 0x7f19041cf290 2026-02-17T05:49:47.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f18dbfff640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18d403dee0 0x7f18d40403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 4707+0+0 (secure 0 0 0) 0x7f18f40782d0 con 0x7f190409c4a0 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f19027fc640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18d403dee0 0x7f18d40403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.566+0000 7f19027fc640 10 monclient: get_auth_request con 0x7f18d403dee0 auth_method 0 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f1902ffd640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f19041cf290 msgr2=0x7f190411f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f1902ffd640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f19041cf290 0x7f190411f4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c3d0 tx=0x7f18f800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8017070 con 0x7f19041cf290 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 10 monclient: dump: 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.588 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f18f8007270 con 0x7f19041cf290 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f19041cf290 msgr2=0x7f190411f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f19041cf290 0x7f190411f4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c3d0 tx=0x7f18f800c8a0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f19041cf290 msgr2=0x7f190411f4a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f18dbfff640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f19041cf290 0x7f190411f4a0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c3d0 tx=0x7f18f800c8a0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.568+0000 7f19027fc640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18d403dee0 0x7f18d40403a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19041223d0 tx=0x7f18ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f19098e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f19098e8640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190411f9e0 0x7f19041da370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f19098e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f190411fdf0 con 0x7f190411f9e0 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190411f9e0 0x7f19041da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 10 monclient: get_auth_request con 0x7f190411f9e0 auth_method 0 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:47.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:47.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:47.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 10 monclient(hunting): handle_auth_done global_id 54210 payload 824 2026-02-17T05:49:47.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.571+0000 7f1902ffd640 1 -- 10.20.193.78:0/1710244731 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f190406cc70 con 0x7f190411f9e0 2026-02-17T05:49:47.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f1902ffd640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190411f9e0 msgr2=0x7f19041da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f1902ffd640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190411f9e0 0x7f19041da370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c3d0 tx=0x7f18f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8017030 con 0x7f190411f9e0 2026-02-17T05:49:47.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 10 monclient: dump: 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f18f8022410 con 0x7f190411f9e0 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:47.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190411f9e0 msgr2=0x7f19041da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190411f9e0 0x7f19041da370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c3d0 tx=0x7f18f8004a40 comp rx=0 tx=0).stop 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190411f9e0 msgr2=0x7f19041da370 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f18dbfff640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190411f9e0 0x7f19041da370 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c3d0 tx=0x7f18f8004a40 comp rx=0 tx=0).stop 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:49:47.594 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 40, 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 57048, 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:47.595 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:47.596 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:47.597 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:47.598 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:47.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:47.600 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:47.601 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:47.602 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:47.603 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:47.604 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:47.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18d403dee0 msgr2=0x7f18d40403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18d403dee0 0x7f18d40403a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19041223d0 tx=0x7f18ec008040 comp rx=0 tx=0).stop 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 10 monclient: shutdown 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190409c4a0 msgr2=0x7f190411ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19037fe640 1 -- 10.20.193.78:0/1710244731 reap_dead start 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190409c4a0 0x7f190411ef60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18f403a040 tx=0x7f18f4004b40 comp rx=0 tx=0).stop 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 shutdown_connections 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 --2- 10.20.193.78:0/1710244731 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f190409c4a0 0x7f190411ef60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 >> 10.20.193.78:0/1710244731 conn(0x7f19040a6620 msgr2=0x7f19040a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 shutdown_connections 2026-02-17T05:49:47.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.574+0000 7f19098e8640 1 -- 10.20.193.78:0/1710244731 wait complete. 2026-02-17T05:49:47.606 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:47.606 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:49:47.607 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:49:47.645 INFO:tasks.rados.rados.0.trial115.stdout:673: done (1 left) 2026-02-17T05:49:47.645 INFO:tasks.rados.rados.0.trial115.stdout:674: done (0 left) 2026-02-17T05:49:47.645 INFO:tasks.rados.rados.0.trial115.stdout:675: rollback oid 29 current snap is 64 2026-02-17T05:49:47.645 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 61 2026-02-17T05:49:47.645 INFO:tasks.rados.rados.0.trial115.stdout:676: snap_remove snap 63 2026-02-17T05:49:47.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.674+0000 7f846ae04640 10 monclient: get_monmap_and_config 2026-02-17T05:49:47.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.674+0000 7f846ae04640 10 monclient: build_initial_monmap 2026-02-17T05:49:47.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.674+0000 7f846ae04640 10 monclient: monmap: 2026-02-17T05:49:47.675 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:47.675 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.675 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:47.676132+0000 2026-02-17T05:49:47.675 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:47.676132+0000 2026-02-17T05:49:47.676 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:47.676 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.676 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:47.676 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:47.676 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:47.676 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:47.676 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:47.676 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 1 Processor -- start 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 1 -- start start 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient: init 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f846406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f846409c4a0 0x7f846413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient(hunting): picked mon.noname-a con 0x7f846409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 0x7f846413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient(hunting): picked mon.noname-b con 0x7f846406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846406daf0 con 0x7f846406c890 2026-02-17T05:49:47.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f84640a0880 con 0x7f846409c4a0 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f846409cc80 con 0x7f846406d380 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846ae04640 10 monclient(hunting): authenticate will time out at 833.229741s 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f8468b79640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f846406c890 0x7f846406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49128/0 (socket says 10.20.193.78:49128) 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f8468b79640 1 -- 10.20.193.78:0/959573729 learned_addr learned my addr 10.20.193.78:0/959573729 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f845bfff640 1 --2- 10.20.193.78:0/959573729 >> v2:10.20.193.78:3300/0 conn(0x7f846409c4a0 0x7f846413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 1 --2- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 0x7f846413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f845bfff640 10 monclient(hunting): get_auth_request con 0x7f846409c4a0 auth_method 0 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f845bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f845bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f845bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 10 monclient(hunting): get_auth_request con 0x7f846406d380 auth_method 0 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 1 -- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 msgr2=0x7f846413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 1 -- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 msgr2=0x7f846413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 1 --2- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 0x7f846413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.675+0000 7f846937a640 1 --2- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 0x7f846413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.677+0000 7f8468b79640 1 -- 10.20.193.78:0/959573729 >> v1:10.20.193.78:6789/0 conn(0x7f846406c890 legacy=0x7f846406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.677+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.677+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.677+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:47.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 10 monclient(hunting): handle_auth_done global_id 54213 payload 274 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 1 -- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 msgr2=0x7f846413d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 1 --2- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 0x7f846413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 1 -- 10.20.193.78:0/959573729 >> v1:10.20.193.78:6789/0 conn(0x7f846406c890 legacy=0x7f846406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 1 monclient: found mon.noname-a 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845b7fe640 1 -- 10.20.193.78:0/959573729 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1158877744 0 0) 0x7f846406daf0 con 0x7f846406c890 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845bfff640 1 -- 10.20.193.78:0/959573729 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846409cf60 con 0x7f846409c4a0 2026-02-17T05:49:47.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.678+0000 7f845b7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T05:49:47.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845bfff640 1 -- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846409c4a0 msgr2=0x7f846413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845bfff640 1 --2- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846409c4a0 0x7f846413b3a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f844c009920 tx=0x7f844c02ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35e3cfcd276816f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 1 -- 10.20.193.78:0/959573729 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c03c070 con 0x7f846409c4a0 2026-02-17T05:49:47.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:47.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: dump: 2026-02-17T05:49:47.688 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 1 -- 10.20.193.78:0/959573729 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c02fce0 con 0x7f846409c4a0 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f846ae04640 5 monclient: authenticate success, global_id 54213 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f846ae04640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 1 -- 10.20.193.78:0/959573729 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c037660 con 0x7f846409c4a0 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f846ae04640 4 set_mon_vals no callback set 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.680+0000 7f845b7fe640 10 monclient: dump: 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 10 monclient: shutdown 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 -- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846409c4a0 msgr2=0x7f846413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 --2- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846409c4a0 0x7f846413b3a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f844c009920 tx=0x7f844c02ee10 comp rx=0 tx=0).stop 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 -- 10.20.193.78:0/959573729 shutdown_connections 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 --2- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846406d380 0x7f846413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 --2- 10.20.193.78:0/959573729 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846409c4a0 0x7f846413b3a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 -- 10.20.193.78:0/959573729 >> 10.20.193.78:0/959573729 conn(0x7f84640a6620 msgr2=0x7f84640a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 -- 10.20.193.78:0/959573729 shutdown_connections 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 -- 10.20.193.78:0/959573729 wait complete. 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 10 monclient: build_initial_monmap 2026-02-17T05:49:47.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 10 monclient: monmap: 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 Processor -- start 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 -- start start 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 10 monclient: init 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846406c890 0x7f84641363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 10 monclient(hunting): picked mon.noname-a con 0x7f846406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.681+0000 7f846ae04640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f846406d380 0x7f8464136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 10 monclient(hunting): picked mon.noname-b con 0x7f846406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846409c4a0 0x7f846411ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 10 monclient(hunting): picked mon.noname-c con 0x7f846409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f846409bfa0 con 0x7f846406c890 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f846409be20 con 0x7f846406d380 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f846413de30 con 0x7f846409c4a0 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846ae04640 10 monclient(hunting): authenticate will time out at 833.236022s 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846406c890 0x7f84641363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846406c890 0x7f84641363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52692/0 (socket says 10.20.193.78:52692) 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846409c4a0 0x7f846411ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 -- 10.20.193.78:0/198556314 learned_addr learned my addr 10.20.193.78:0/198556314 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient(hunting): get_auth_request con 0x7f846406c890 auth_method 0 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 10 monclient(hunting): get_auth_request con 0x7f846409c4a0 auth_method 0 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:47.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846409c4a0 msgr2=0x7f846411ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846409c4a0 msgr2=0x7f846411ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846409c4a0 0x7f846411ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f846937a640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846409c4a0 0x7f846411ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f846406d380 0x7f8464136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient(hunting): handle_auth_done global_id 54219 payload 824 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846409c4a0 msgr2=0x7f846411ee00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846409c4a0 0x7f846411ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f846406d380 msgr2=0x7f8464136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f846406d380 0x7f8464136920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient: _finish_hunting 0 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 monclient: found mon.noname-a 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f8468b79640 1 -- 10.20.193.78:0/198556314 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84641227d0 con 0x7f846406c890 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f845bfff640 10 monclient: get_auth_request con 0x7f846406d380 auth_method 0 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.682+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f846406d380 0x7f8464136920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f8468b79640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846406c890 msgr2=0x7f84641363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f8468b79640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846406c890 0x7f84641363e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f845400c730 tx=0x7f845400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454018070 con 0x7f846406c890 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: dump: 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: _finish_auth 0 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: _check_auth_tickets 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8454004d10 con 0x7f846406c890 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f846ae04640 5 monclient: authenticate success, global_id 54219 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f846ae04640 10 monclient: _renew_subs 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f846ae04640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846406f550 con 0x7f846406c890 2026-02-17T05:49:47.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454007310 con 0x7f846406c890 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f845affd640 4 set_mon_vals no callback set 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f84597fa640 10 monclient: dump: 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f846ae04640 10 monclient: _renew_subs 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f846ae04640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.684+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8464122ab0 con 0x7f846406c890 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f846ae04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f846ae04640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f846ae04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8464074980 con 0x7f84641cf0f0 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8454007530 con 0x7f846406c890 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 10 monclient: get_auth_request con 0x7f84641cf0f0 auth_method 0 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f84597fa640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f843403df00 0x7f84340403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 4707+0+0 (secure 0 0 0) 0x7f8454052030 con 0x7f846406c890 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f846937a640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f843403df00 0x7f84340403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.685+0000 7f846937a640 10 monclient: get_auth_request con 0x7f843403df00 auth_method 0 2026-02-17T05:49:47.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.688+0000 7f846937a640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f843403df00 0x7f84340403c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8460009610 tx=0x7f846000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 10 monclient: get_auth_request con 0x7f84641cf0f0 auth_method 0 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:47.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:47.885+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:49:48.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:48.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 10 monclient: get_auth_request con 0x7f84641cf0f0 auth_method 0 2026-02-17T05:49:48.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:48.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:48.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:48.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:49:48.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:48.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:48.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:48.286+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:49:48.648 INFO:tasks.rados.rados.0.trial115.stdout:676: done (1 left) 2026-02-17T05:49:48.648 INFO:tasks.rados.rados.0.trial115.stdout:677: write oid 46 current snap is 64 2026-02-17T05:49:48.648 INFO:tasks.rados.rados.0.trial115.stdout:675: finishing rollback tid 0 to trial11545420-29 2026-02-17T05:49:48.648 INFO:tasks.rados.rados.0.trial115.stdout:675: finishing rollback tid 1 to trial11545420-29 2026-02-17T05:49:48.648 INFO:tasks.rados.rados.0.trial115.stdout:675: finishing rollback tid 2 to trial11545420-29 2026-02-17T05:49:48.648 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 110 (ObjNum 262 snap 53 seq_num 262) dirty exists 2026-02-17T05:49:48.648 INFO:tasks.rados.rados.0.trial115.stdout:677: seq_num 285 ranges {789995=551026,2027721=468420} 2026-02-17T05:49:48.653 INFO:tasks.rados.rados.0.trial115.stdout:677: writing trial11545420-46 from 789995 to 1341021 tid 1 2026-02-17T05:49:48.658 INFO:tasks.rados.rados.0.trial115.stdout:677: writing trial11545420-46 from 2027721 to 2496141 tid 2 2026-02-17T05:49:48.658 INFO:tasks.rados.rados.0.trial115.stdout:675: done (1 left) 2026-02-17T05:49:48.658 INFO:tasks.rados.rados.0.trial115.stdout:678: write (excl) oid 18 current snap is 64 2026-02-17T05:49:48.658 INFO:tasks.rados.rados.0.trial115.stdout:678: seq_num 286 ranges {406802=626275,1686203=728431} 2026-02-17T05:49:48.661 INFO:tasks.rados.rados.0.trial115.stdout:678: writing trial11545420-18 from 406802 to 1033077 tid 1 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:678: writing trial11545420-18 from 1686203 to 2414634 tid 2 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:679: copy_from oid 33 from oid 34 current snap is 64 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: finishing write tid 1 to trial11545420-46 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: finishing write tid 2 to trial11545420-46 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: finishing write tid 3 to trial11545420-46 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: finishing write tid 4 to trial11545420-46 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: oid 46 updating version 0 to 235 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: oid 46 updating version 235 to 236 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: oid 46 version 236 is already newer than 234 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 236 (ObjNum 285 snap 64 seq_num 285) dirty exists 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: left oid 46 (ObjNum 285 snap 64 seq_num 285) 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:678: finishing write tid 1 to trial11545420-18 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:677: done (2 left) 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:680: snap_create 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:679: finishing copy_from to trial11545420-33 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:679: got expected ENOENT (src dne) 2026-02-17T05:49:48.668 INFO:tasks.rados.rados.0.trial115.stdout:679: finishing copy_from racing read to trial11545420-33 2026-02-17T05:49:48.669 INFO:tasks.rados.rados.0.trial115.stdout:678: finishing write tid 2 to trial11545420-18 2026-02-17T05:49:48.669 INFO:tasks.rados.rados.0.trial115.stdout:678: finishing write tid 3 to trial11545420-18 2026-02-17T05:49:48.670 INFO:tasks.rados.rados.0.trial115.stdout:678: finishing write tid 4 to trial11545420-18 2026-02-17T05:49:48.670 INFO:tasks.rados.rados.0.trial115.stdout:678: oid 18 updating version 0 to 209 2026-02-17T05:49:48.670 INFO:tasks.rados.rados.0.trial115.stdout:678: oid 18 version 209 is already newer than 207 2026-02-17T05:49:48.670 INFO:tasks.rados.rados.0.trial115.stdout:678: oid 18 version 209 is already newer than 208 2026-02-17T05:49:48.670 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 209 (ObjNum 286 snap 64 seq_num 286) dirty exists 2026-02-17T05:49:48.670 INFO:tasks.rados.rados.0.trial115.stdout:678: left oid 18 (ObjNum 286 snap 64 seq_num 286) 2026-02-17T05:49:49.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:49.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 10 monclient: get_auth_request con 0x7f84641cf0f0 auth_method 0 2026-02-17T05:49:49.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:49.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:49.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:49.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:49:49.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:49.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:49.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:49.087+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:49:49.657 INFO:tasks.rados.rados.0.trial115.stdout:678: done (2 left) 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:679: done (1 left) 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:680: done (0 left) 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:681: rollback oid 34 current snap is 65 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 61 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:682: read oid 9 snap 55 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:682: expect (ObjNum 269 snap 54 seq_num 269) 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:683: read oid 33 snap -1 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:683: expect (ObjNum 278 snap 60 seq_num 278) 2026-02-17T05:49:49.658 INFO:tasks.rados.rados.0.trial115.stdout:684: snap_create 2026-02-17T05:49:49.660 INFO:tasks.rados.rados.0.trial115.stdout:681: finishing rollback tid 1 to trial11545420-34 2026-02-17T05:49:49.660 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 251 (ObjNum 49 snap 0 seq_num 1634300532) dirty dne 2026-02-17T05:49:49.661 INFO:tasks.rados.rados.0.trial115.stdout:682: expect (ObjNum 269 snap 54 seq_num 269) 2026-02-17T05:49:49.748 INFO:tasks.rados.rados.0.trial115.stdout:683: expect (ObjNum 278 snap 60 seq_num 278) 2026-02-17T05:49:50.660 INFO:tasks.rados.rados.0.trial115.stdout:681: done (3 left) 2026-02-17T05:49:50.660 INFO:tasks.rados.rados.0.trial115.stdout:682: done (2 left) 2026-02-17T05:49:50.660 INFO:tasks.rados.rados.0.trial115.stdout:683: done (1 left) 2026-02-17T05:49:50.660 INFO:tasks.rados.rados.0.trial115.stdout:684: done (0 left) 2026-02-17T05:49:50.660 INFO:tasks.rados.rados.0.trial115.stdout:685: write oid 38 current snap is 66 2026-02-17T05:49:50.661 INFO:tasks.rados.rados.0.trial115.stdout:685: seq_num 287 ranges {423609=701524,1628332=472089,2648987=174622} 2026-02-17T05:49:50.665 INFO:tasks.rados.rados.0.trial115.stdout:685: writing trial11545420-38 from 423609 to 1125133 tid 1 2026-02-17T05:49:50.668 INFO:tasks.rados.rados.0.trial115.stdout:685: writing trial11545420-38 from 1628332 to 2100421 tid 2 2026-02-17T05:49:50.671 INFO:tasks.rados.rados.0.trial115.stdout:685: writing trial11545420-38 from 2648987 to 2823609 tid 3 2026-02-17T05:49:50.671 INFO:tasks.rados.rados.0.trial115.stdout:686: rollback oid 42 current snap is 66 2026-02-17T05:49:50.671 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 57 2026-02-17T05:49:50.672 INFO:tasks.rados.rados.0.trial115.stdout:685: finishing write tid 1 to trial11545420-38 2026-02-17T05:49:50.672 INFO:tasks.rados.rados.0.trial115.stdout:685: finishing write tid 2 to trial11545420-38 2026-02-17T05:49:50.672 INFO:tasks.rados.rados.0.trial115.stdout:687: snap_create 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:685: finishing write tid 3 to trial11545420-38 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:685: finishing write tid 4 to trial11545420-38 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:685: finishing write tid 5 to trial11545420-38 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:685: oid 38 updating version 0 to 269 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:685: oid 38 version 269 is already newer than 268 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:685: oid 38 updating version 269 to 270 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:685: oid 38 version 270 is already newer than 267 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 270 (ObjNum 287 snap 66 seq_num 287) dirty exists 2026-02-17T05:49:50.673 INFO:tasks.rados.rados.0.trial115.stdout:685: left oid 38 (ObjNum 287 snap 66 seq_num 287) 2026-02-17T05:49:50.675 INFO:tasks.rados.rados.0.trial115.stdout:686: finishing rollback tid 0 to trial11545420-42 2026-02-17T05:49:50.675 INFO:tasks.rados.rados.0.trial115.stdout:686: finishing rollback tid 1 to trial11545420-42 2026-02-17T05:49:50.675 INFO:tasks.rados.rados.0.trial115.stdout:686: finishing rollback tid 2 to trial11545420-42 2026-02-17T05:49:50.675 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 302 (ObjNum 261 snap 52 seq_num 261) dirty exists 2026-02-17T05:49:50.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.682+0000 7f8433fff640 10 monclient: tick 2026-02-17T05:49:50.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.682+0000 7f8433fff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:50.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.682+0000 7f8433fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:50.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.682+0000 7f8433fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:50.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.688+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:50.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.688+0000 7f845bfff640 10 monclient: get_auth_request con 0x7f84641cf0f0 auth_method 0 2026-02-17T05:49:50.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.688+0000 7f845bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:50.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.688+0000 7f845bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:50.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.688+0000 7f845bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:49:50.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.689+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:49:50.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.689+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:49:50.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.689+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:49:50.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:50.689+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2026-02-17T05:49:51.661 INFO:tasks.rados.rados.0.trial115.stdout:685: done (2 left) 2026-02-17T05:49:51.661 INFO:tasks.rados.rados.0.trial115.stdout:686: done (1 left) 2026-02-17T05:49:51.661 INFO:tasks.rados.rados.0.trial115.stdout:687: done (0 left) 2026-02-17T05:49:51.662 INFO:tasks.rados.rados.0.trial115.stdout:688: snap_create 2026-02-17T05:49:51.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 10 monclient: tick 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 5 monclient: _check_tell_commands timeout tell command 1 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f842c003640 0x7f842c005ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 msgr2=0x7f846411f340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84641cf0f0 0x7f846411f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f8433fff640 1 -- 10.20.193.78:0/198556314 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8464074980 con 0x7f842c003640 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f842c003640 0x7f842c005ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 10 monclient: get_auth_request con 0x7f842c003640 auth_method 0 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 10 monclient(hunting): handle_auth_done global_id 64100 payload 824 2026-02-17T05:49:51.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.682+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f842c0061b0 con 0x7f842c003640 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f842c003640 msgr2=0x7f842c005ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f842c003640 0x7f842c005ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f844c039000 tx=0x7f844c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c03c040 con 0x7f842c003640 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 10 monclient: dump: 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.687 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f844c041610 con 0x7f842c003640 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:51.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f842c003640 msgr2=0x7f842c005ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f842c003640 0x7f842c005ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f844c039000 tx=0x7f844c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f842c003640 msgr2=0x7f842c005ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.684+0000 7f84597fa640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f842c003640 0x7f842c005ad0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f844c039000 tx=0x7f844c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f846ae04640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f846ae04640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846411f880 0x7f84641da210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f846ae04640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f846411fc60 con 0x7f846411f880 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846411f880 0x7f84641da210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 10 monclient: get_auth_request con 0x7f846411f880 auth_method 0 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 10 monclient(hunting): handle_auth_done global_id 64103 payload 824 2026-02-17T05:49:51.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.688+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8464121380 con 0x7f846411f880 2026-02-17T05:49:51.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f845bfff640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846411f880 msgr2=0x7f84641da210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:51.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f845bfff640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846411f880 0x7f84641da210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f844c009920 tx=0x7f844c009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c03c040 con 0x7f846411f880 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 10 monclient: dump: 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.691 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f844c051b50 con 0x7f846411f880 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846411f880 msgr2=0x7f84641da210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846411f880 0x7f84641da210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f844c009920 tx=0x7f844c009290 comp rx=0 tx=0).stop 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846411f880 msgr2=0x7f84641da210 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f84597fa640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f846411f880 0x7f84641da210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f846937a640 1 -- 10.20.193.78:0/198556314 reap_dead start 2026-02-17T05:49:51.693 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:51.693 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:51.693 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:51.693 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:51.693 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 40, 2026-02-17T05:49:51.693 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 10753, 2026-02-17T05:49:51.693 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:51.693 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:51.694 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:51.695 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.696 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:51.697 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:51.698 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:51.699 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.700 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:51.701 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.702 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.703 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:51.704 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:51.705 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:51.706 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:51.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f843403df00 msgr2=0x7f84340403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f846ae04640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f843403df00 0x7f84340403c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8460009610 tx=0x7f846000b040 comp rx=0 tx=0).stop 2026-02-17T05:49:51.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f846ae04640 10 monclient: shutdown 2026-02-17T05:49:51.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f846ae04640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:51.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846406c890 msgr2=0x7f84641363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.690+0000 7f846ae04640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846406c890 0x7f84641363e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f845400c730 tx=0x7f845400cc00 comp rx=0 tx=0).stop 2026-02-17T05:49:51.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.691+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 shutdown_connections 2026-02-17T05:49:51.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.691+0000 7f846ae04640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f843403df00 0x7f84340403c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.691+0000 7f846ae04640 1 --2- 10.20.193.78:0/198556314 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f846406c890 0x7f84641363e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.691+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 >> 10.20.193.78:0/198556314 conn(0x7f84640a6620 msgr2=0x7f84640a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:51.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.691+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 shutdown_connections 2026-02-17T05:49:51.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.691+0000 7f846ae04640 1 -- 10.20.193.78:0/198556314 wait complete. 2026-02-17T05:49:51.710 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:51.710 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:49:51.710 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:49:51.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.776+0000 7f7c235e4640 10 monclient: get_monmap_and_config 2026-02-17T05:49:51.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.776+0000 7f7c235e4640 10 monclient: build_initial_monmap 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.776+0000 7f7c235e4640 10 monclient: monmap: 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:51.777254+0000 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:51.777254+0000 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:51.777 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:51.778 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.776+0000 7f7c235e4640 1 Processor -- start 2026-02-17T05:49:51.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.776+0000 7f7c235e4640 1 -- start start 2026-02-17T05:49:51.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.776+0000 7f7c235e4640 10 monclient: init 2026-02-17T05:49:51.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:51.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient: _add_conns ranks=[2,3,0,4,1] 2026-02-17T05:49:51.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09ea90 0x7f7c1c0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient(hunting): picked mon.noname-b con 0x7f7c1c09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7c1c09ee70 0x7f7c1c09f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient(hunting): picked mon.noname-a con 0x7f7c1c09ee70 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7c1c09d980 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1c0a0a20 con 0x7f7c1c09d980 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1c0a0d00 con 0x7f7c1c09ee70 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1c0a0460 con 0x7f7c1c09ea90 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c235e4640 10 monclient(hunting): authenticate will time out at 837.331119s 2026-02-17T05:49:51.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09ea90 0x7f7c1c0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7c1c09ee70 0x7f7c1c09f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09ea90 0x7f7c1c0a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36734/0 (socket says 10.20.193.78:36734) 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 1 -- 10.20.193.78:0/3971597336 learned_addr learned my addr 10.20.193.78:0/3971597336 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21b5a640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f7c1c09d980 0x7f7c1c09ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49136/0 (socket says 10.20.193.78:49136) 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient(hunting): get_auth_request con 0x7f7c1c09ea90 auth_method 0 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 10 monclient(hunting): get_auth_request con 0x7f7c1c09ee70 auth_method 0 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient(hunting): handle_auth_done global_id 64106 payload 274 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 1 -- 10.20.193.78:0/3971597336 >> v2:10.20.193.78:3300/0 conn(0x7f7c1c09ee70 msgr2=0x7f7c1c09f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 1 --2- 10.20.193.78:0/3971597336 >> v2:10.20.193.78:3300/0 conn(0x7f7c1c09ee70 0x7f7c1c09f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 1 -- 10.20.193.78:0/3971597336 >> v1:10.20.193.78:6789/0 conn(0x7f7c1c09d980 legacy=0x7f7c1c09ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:51.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient: _finish_hunting 0 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 1 monclient: found mon.noname-b 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c21359640 1 -- 10.20.193.78:0/3971597336 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c0a0740 con 0x7f7c1c09ea90 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.777+0000 7f7c20b58640 1 --2- 10.20.193.78:0/3971597336 >> v2:10.20.193.78:3300/0 conn(0x7f7c1c09ee70 0x7f7c1c09f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c21359640 1 -- 10.20.193.78:0/3971597336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09ea90 msgr2=0x7f7c1c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c21359640 1 --2- 10.20.193.78:0/3971597336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09ea90 0x7f7c1c0a13d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009970 tx=0x7f7c0c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d0b5387b14ade server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 1 -- 10.20.193.78:0/3971597336 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c03c070 con 0x7f7c1c09ea90 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: dump: 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.793 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: _finish_auth 0 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:51.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c235e4640 5 monclient: authenticate success, global_id 64106 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 1 -- 10.20.193.78:0/3971597336 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c02fcd0 con 0x7f7c1c09ea90 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c235e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 1 -- 10.20.193.78:0/3971597336 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c037660 con 0x7f7c1c09ea90 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c235e4640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c235e4640 4 set_mon_vals no callback set 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.779+0000 7f7c0bfff640 10 monclient: dump: 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.795 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 10 monclient: shutdown 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 -- 10.20.193.78:0/3971597336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09ea90 msgr2=0x7f7c1c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 --2- 10.20.193.78:0/3971597336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09ea90 0x7f7c1c0a13d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009970 tx=0x7f7c0c02ee90 comp rx=0 tx=0).stop 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 -- 10.20.193.78:0/3971597336 shutdown_connections 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 --2- 10.20.193.78:0/3971597336 >> v2:10.20.193.78:3300/0 conn(0x7f7c1c09ee70 0x7f7c1c09f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 --2- 10.20.193.78:0/3971597336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09ea90 0x7f7c1c0a13d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 -- 10.20.193.78:0/3971597336 >> 10.20.193.78:0/3971597336 conn(0x7f7c1c0a7520 msgr2=0x7f7c1c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 -- 10.20.193.78:0/3971597336 shutdown_connections 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 -- 10.20.193.78:0/3971597336 wait complete. 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 10 monclient: build_initial_monmap 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 10 monclient: monmap: 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:51.797 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 Processor -- start 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 1 -- start start 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.780+0000 7f7c235e4640 10 monclient: init 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09d980 0x7f7c1c1cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient(hunting): picked mon.noname-c con 0x7f7c1c09d980 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c09ea90 0x7f7c1c1cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient(hunting): picked mon.noname-b con 0x7f7c1c09ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c1c09ee70 0x7f7c1c1cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient(hunting): picked mon.noname-a con 0x7f7c1c09ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c0a66a0 con 0x7f7c1c09ee70 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c09f7d0 con 0x7f7c1c09ea90 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1c0a6820 con 0x7f7c1c09d980 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c235e4640 10 monclient(hunting): authenticate will time out at 837.335218s 2026-02-17T05:49:51.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09d980 0x7f7c1c1cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09d980 0x7f7c1c1cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36748/0 (socket says 10.20.193.78:36748) 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 -- 10.20.193.78:0/827977558 learned_addr learned my addr 10.20.193.78:0/827977558 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient(hunting): get_auth_request con 0x7f7c1c09d980 auth_method 0 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21b5a640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c1c09ee70 0x7f7c1c1cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21b5a640 10 monclient(hunting): get_auth_request con 0x7f7c1c09ee70 auth_method 0 2026-02-17T05:49:51.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21b5a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21b5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21b5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21b5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21b5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient(hunting): handle_auth_done global_id 64109 payload 824 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c09ea90 msgr2=0x7f7c1c1cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c09ea90 0x7f7c1c1cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c1c09ee70 msgr2=0x7f7c1c1cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c1c09ee70 0x7f7c1c1cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient: _finish_hunting 0 2026-02-17T05:49:51.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 monclient: found mon.noname-c 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.781+0000 7f7c21359640 1 -- 10.20.193.78:0/827977558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c0737e0 con 0x7f7c1c09d980 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c21359640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09d980 msgr2=0x7f7c1c1cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c21359640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09d980 0x7f7c1c1cf360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009970 tx=0x7f7c0c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c03c040 con 0x7f7c1c09d980 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: dump: 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.802 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c047070 con 0x7f7c1c09d980 2026-02-17T05:49:51.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c235e4640 5 monclient: authenticate success, global_id 64109 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c235e4640 10 monclient: _renew_subs 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c235e4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c132440 con 0x7f7c1c09d980 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c042890 con 0x7f7c1c09d980 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c0b7fe640 4 set_mon_vals no callback set 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c09ffb640 10 monclient: dump: 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.804 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c235e4640 10 monclient: _renew_subs 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c235e4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.783+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c135940 con 0x7f7c1c09d980 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c235e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c235e4640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c131de0 0x7f7c1c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c235e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c0a1c10 con 0x7f7c1c131de0 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7c0c042a30 con 0x7f7c1c09d980 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c131de0 0x7f7c1c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 10 monclient: get_auth_request con 0x7f7c1c131de0 auth_method 0 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c09ffb640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7bec03dee0 0x7f7bec0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(151..151 src has 117..151) ==== 4735+0+0 (secure 0 0 0) 0x7f7c0c0773e0 con 0x7f7c1c09d980 2026-02-17T05:49:51.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 10 monclient(hunting): handle_auth_done global_id 44248 payload 824 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c20b58640 1 -- 10.20.193.78:0/827977558 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c1c0a13d0 con 0x7f7c1c131de0 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c21b5a640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7bec03dee0 0x7f7bec0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.784+0000 7f7c21b5a640 10 monclient: get_auth_request con 0x7f7bec03dee0 auth_method 0 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.786+0000 7f7c20b58640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c131de0 msgr2=0x7f7c1c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.786+0000 7f7c20b58640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c131de0 0x7f7c1c06ef90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0a1df0 tx=0x7f7c1000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1000cb10 con 0x7f7c1c131de0 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 10 monclient: dump: 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7c1000cb10 con 0x7f7c1c131de0 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c131de0 msgr2=0x7f7c1c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c131de0 0x7f7c1c06ef90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0a1df0 tx=0x7f7c1000eca0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c131de0 msgr2=0x7f7c1c06ef90 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c09ffb640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c131de0 0x7f7c1c06ef90 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0a1df0 tx=0x7f7c1000eca0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.787+0000 7f7c21b5a640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7bec03dee0 0x7f7bec0403a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c073f70 tx=0x7f7c18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c235e4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:51.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c235e4640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c06f4d0 0x7f7c1c1da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c235e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c0744d0 con 0x7f7c1c06f4d0 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c06f4d0 0x7f7c1c1da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 10 monclient: get_auth_request con 0x7f7c1c06f4d0 auth_method 0 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 10 monclient(hunting): handle_auth_done global_id 44251 payload 824 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.790+0000 7f7c20b58640 1 -- 10.20.193.78:0/827977558 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7c1c130c60 con 0x7f7c1c06f4d0 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c20b58640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c06f4d0 msgr2=0x7f7c1c1da7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:51.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c20b58640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c06f4d0 0x7f7c1c1da7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c10018000 tx=0x7f7c100047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10003b20 con 0x7f7c1c06f4d0 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 10 monclient: dump: 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f7c10005e40 con 0x7f7c1c06f4d0 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c06f4d0 msgr2=0x7f7c1c1da7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c06f4d0 0x7f7c1c1da7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c10018000 tx=0x7f7c100047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c06f4d0 msgr2=0x7f7c1c1da7e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c09ffb640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c1c06f4d0 0x7f7c1c1da7e0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c10018000 tx=0x7f7c100047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:51.813 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 40, 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 115442, 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:51.814 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:51.815 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:51.816 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:51.817 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.818 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:51.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:51.821 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:51.822 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:51.823 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7bec03dee0 msgr2=0x7f7bec0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7bec03dee0 0x7f7bec0403a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c073f70 tx=0x7f7c18009040 comp rx=0 tx=0).stop 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 10 monclient: shutdown 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09d980 msgr2=0x7f7c1c1cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 1 --2- 10.20.193.78:0/827977558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c1c09d980 0x7f7c1c1cf360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009970 tx=0x7f7c0c004830 comp rx=0 tx=0).stop 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c21b5a640 1 -- 10.20.193.78:0/827977558 reap_dead start 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c21b5a640 1 -- 10.20.193.78:0/827977558 reap_dead start 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 shutdown_connections 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 >> 10.20.193.78:0/827977558 conn(0x7f7c1c0a7520 msgr2=0x7f7c1c138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 shutdown_connections 2026-02-17T05:49:51.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.793+0000 7f7c235e4640 1 -- 10.20.193.78:0/827977558 wait complete. 2026-02-17T05:49:51.825 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:51.825 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:49:51.826 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.892+0000 7f0aff33e640 10 monclient: get_monmap_and_config 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.892+0000 7f0aff33e640 10 monclient: build_initial_monmap 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.892+0000 7f0aff33e640 10 monclient: monmap: 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:51.893782+0000 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:51.893782+0000 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:51.895 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.892+0000 7f0aff33e640 1 Processor -- start 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 1 -- start start 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient: init 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0af806c890 0x7f0af806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient(hunting): picked mon.noname-c con 0x7f0af806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0af809c4a0 0x7f0af806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient(hunting): picked mon.noname-a con 0x7f0af809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0af8134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af806daf0 con 0x7f0af8134df0 2026-02-17T05:49:51.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f0af80a0880 con 0x7f0af809c4a0 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0af806d6f0 con 0x7f0af806c890 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0aff33e640 10 monclient(hunting): authenticate will time out at 837.447370s 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0af809c4a0 0x7f0af806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd8b4640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f0af8134df0 0x7f0af8135280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49148/0 (socket says 10.20.193.78:49148) 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd8b4640 1 -- 10.20.193.78:0/3615142654 learned_addr learned my addr 10.20.193.78:0/3615142654 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient(hunting): get_auth_request con 0x7f0af809c4a0 auth_method 0 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd0b3640 1 --2- 10.20.193.78:0/3615142654 >> v2:10.20.193.115:3300/0 conn(0x7f0af806c890 0x7f0af806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd0b3640 10 monclient(hunting): get_auth_request con 0x7f0af806c890 auth_method 0 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd0b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd0b3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd0b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd0b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd0b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afd0b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient(hunting): handle_auth_done global_id 54228 payload 274 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 1 -- 10.20.193.78:0/3615142654 >> v2:10.20.193.115:3300/0 conn(0x7f0af806c890 msgr2=0x7f0af806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 1 --2- 10.20.193.78:0/3615142654 >> v2:10.20.193.115:3300/0 conn(0x7f0af806c890 0x7f0af806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 1 -- 10.20.193.78:0/3615142654 >> v1:10.20.193.78:6789/0 conn(0x7f0af8134df0 legacy=0x7f0af8135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient: _finish_hunting 0 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 1 monclient: found mon.noname-a 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:51.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.893+0000 7f0afc8b2640 1 -- 10.20.193.78:0/3615142654 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af809cc80 con 0x7f0af809c4a0 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0afc8b2640 1 -- 10.20.193.78:0/3615142654 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 msgr2=0x7f0af806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0afc8b2640 1 --2- 10.20.193.78:0/3615142654 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af806d1b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0af806a480 tx=0x7f0aec02ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85768d623ed2dd9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 1 -- 10.20.193.78:0/3615142654 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec03c070 con 0x7f0af809c4a0 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: dump: 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.903 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: _finish_auth 0 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 1 -- 10.20.193.78:0/3615142654 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aec02fbd0 con 0x7f0af809c4a0 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 5 monclient: authenticate success, global_id 54228 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 1 -- 10.20.193.78:0/3615142654 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec037440 con 0x7f0af809c4a0 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 4 set_mon_vals no callback set 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0ae7fff640 10 monclient: dump: 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 10 monclient: shutdown 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 1 -- 10.20.193.78:0/3615142654 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 msgr2=0x7f0af806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 1 --2- 10.20.193.78:0/3615142654 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af806d1b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0af806a480 tx=0x7f0aec02ed50 comp rx=0 tx=0).stop 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 1 -- 10.20.193.78:0/3615142654 shutdown_connections 2026-02-17T05:49:51.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 1 --2- 10.20.193.78:0/3615142654 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af806d1b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 1 --2- 10.20.193.78:0/3615142654 >> v2:10.20.193.115:3300/0 conn(0x7f0af806c890 0x7f0af806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 1 -- 10.20.193.78:0/3615142654 >> 10.20.193.78:0/3615142654 conn(0x7f0af80a6620 msgr2=0x7f0af80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 1 -- 10.20.193.78:0/3615142654 shutdown_connections 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.896+0000 7f0aff33e640 1 -- 10.20.193.78:0/3615142654 wait complete. 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient: build_initial_monmap 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient: monmap: 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 1 Processor -- start 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 1 -- start start 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient: init 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0af806c890 0x7f0af81cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient(hunting): picked mon.noname-c con 0x7f0af806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af81cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient(hunting): picked mon.noname-a con 0x7f0af809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0af8134df0 0x7f0af8121780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient(hunting): picked mon.noname-b con 0x7f0af8134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0af80a01a0 con 0x7f0af809c4a0 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0af80a0020 con 0x7f0af8134df0 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0af809be20 con 0x7f0af806c890 2026-02-17T05:49:51.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0aff33e640 10 monclient(hunting): authenticate will time out at 837.451683s 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af81cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af81cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52738/0 (socket says 10.20.193.78:52738) 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 -- 10.20.193.78:0/2841218778 learned_addr learned my addr 10.20.193.78:0/2841218778 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient(hunting): get_auth_request con 0x7f0af809c4a0 auth_method 0 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afd0b3640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0af806c890 0x7f0af81cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afd0b3640 10 monclient(hunting): get_auth_request con 0x7f0af806c890 auth_method 0 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afd0b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afd0b3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afd0b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afd0b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afd0b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afd0b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient(hunting): handle_auth_done global_id 54231 payload 824 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 -- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0af806c890 msgr2=0x7f0af81cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0af806c890 0x7f0af81cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 -- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0af8134df0 msgr2=0x7f0af8121780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0af8134df0 0x7f0af8121780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient: _finish_hunting 0 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 monclient: found mon.noname-a 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.897+0000 7f0afc8b2640 1 -- 10.20.193.78:0/2841218778 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af8072770 con 0x7f0af809c4a0 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0afc8b2640 1 -- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 msgr2=0x7f0af81cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0afc8b2640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af81cd870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aec002410 tx=0x7f0aec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 1 -- 10.20.193.78:0/2841218778 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec03c040 con 0x7f0af809c4a0 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: dump: 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 1 -- 10.20.193.78:0/2841218778 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aec047070 con 0x7f0af809c4a0 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:51.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 1 -- 10.20.193.78:0/2841218778 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec040a60 con 0x7f0af809c4a0 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 10 monclient: dump: 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae77fe640 4 set_mon_vals no callback set 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 5 monclient: authenticate success, global_id 54231 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 10 monclient: _renew_subs 2026-02-17T05:49:51.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af8125100 con 0x7f0af809c4a0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 10 monclient: _renew_subs 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af8123f30 con 0x7f0af809c4a0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af8074980 con 0x7f0af809c4a0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 1 -- 10.20.193.78:0/2841218778 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0aec03f030 con 0x7f0af809c4a0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ac803deb0 0x7f0ac8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.900+0000 7f0ae5ffb640 1 -- 10.20.193.78:0/2841218778 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(151..151 src has 117..151) ==== 4735+0+0 (secure 0 0 0) 0x7f0aec077b60 con 0x7f0af809c4a0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.901+0000 7f0afd0b3640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ac803deb0 0x7f0ac8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:51.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.901+0000 7f0afd0b3640 10 monclient: get_auth_request con 0x7f0ac803deb0 auth_method 0 2026-02-17T05:49:51.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.903+0000 7f0ae5ffb640 1 -- 10.20.193.78:0/2841218778 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0af8072770 con 0x7f0af809c4a0 2026-02-17T05:49:51.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.903+0000 7f0ae5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:51.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.903+0000 7f0ae5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:51.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:51.903+0000 7f0afd0b3640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ac803deb0 0x7f0ac8040370 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009ba0 tx=0x7f0ae8009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:52.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.016+0000 7f0aff33e640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:49:52.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.016+0000 7f0aff33e640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:49:52.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.016+0000 7f0aff33e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:52.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.016+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0af8135a20 con 0x7f0af809c4a0 2026-02-17T05:49:52.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=1,health=12,kv=12,logm=65} crc {auth=2227171214,config=3678701417,health=2157093694,kv=449989585,logm=471797010}) 2026-02-17T05:49:52.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=12,config=2,health=10,kv=11,logm=65} crc {auth=4058076638,config=692619927,health=894172243,kv=2049492640,logm=4019998302}) 2026-02-17T05:49:52.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.017+0000 7f0ae5ffb640 1 -- 10.20.193.78:0/2841218778 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f0aec076590 con 0x7f0af809c4a0 2026-02-17T05:49:52.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.017+0000 7f0ae5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:49:52.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.017+0000 7f0ae5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=1,health=12,kv=12,logm=65} crc {auth=2227171214,config=3678701417,health=2157093694,kv=449989585,logm=471797010}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=13,config=2,health=12,kv=11,logm=62} crc {auth=3036457387,config=692619927,health=2157093694,kv=449989585,logm=3409853651}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2063333119}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1645870783}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2063333119}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.017+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=883463252}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=242601894}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1098328601}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=242601894}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1803268901}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495674914}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=615651493}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495674914}) 2026-02-17T05:49:52.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2245677692}) 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ac803deb0 msgr2=0x7f0ac8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ac803deb0 0x7f0ac8040370 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009ba0 tx=0x7f0ae8009290 comp rx=0 tx=0).stop 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 10 monclient: shutdown 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 msgr2=0x7f0af81cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af81cd870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aec002410 tx=0x7f0aec0047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 shutdown_connections 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ac803deb0 0x7f0ac8040370 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0af8134df0 0x7f0af8121780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0af809c4a0 0x7f0af81cd870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 --2- 10.20.193.78:0/2841218778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0af806c890 0x7f0af81cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 >> 10.20.193.78:0/2841218778 conn(0x7f0af80a6620 msgr2=0x7f0af80a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 shutdown_connections 2026-02-17T05:49:52.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:52.018+0000 7f0aff33e640 1 -- 10.20.193.78:0/2841218778 wait complete. 2026-02-17T05:49:52.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3870965852}) 2026-02-17T05:49:52.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3469215622}) 2026-02-17T05:49:52.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.018+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3870965852}) 2026-02-17T05:49:52.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.019+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3444576274}) 2026-02-17T05:49:52.026 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:49:52.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3525845143}) 2026-02-17T05:49:52.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1591194045}) 2026-02-17T05:49:52.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3525845143}) 2026-02-17T05:49:52.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3769563575}) 2026-02-17T05:49:52.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1878100535}) 2026-02-17T05:49:52.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1062100495}) 2026-02-17T05:49:52.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1878100535}) 2026-02-17T05:49:52.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.161+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3480518222}) 2026-02-17T05:49:52.288 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.288+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.289 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.288+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578258104}) 2026-02-17T05:49:52.289 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.288+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2651289069}) 2026-02-17T05:49:52.289 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.288+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.289 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.288+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578258104}) 2026-02-17T05:49:52.289 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.288+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072871777}) 2026-02-17T05:49:52.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.662+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.662+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1330022147}) 2026-02-17T05:49:52.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.662+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3060726079}) 2026-02-17T05:49:52.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.662+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.662+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1330022147}) 2026-02-17T05:49:52.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.662+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2446791030}) 2026-02-17T05:49:52.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.664+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.664+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=91,mdsmap=2,mgr=7} crc {logm=2146807168,mdsmap=3140514009,mgr=267573432}) 2026-02-17T05:49:52.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.664+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1010892907,mdsmap=1160443688,mgr=787438293}) 2026-02-17T05:49:52.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.665+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.665+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=91,mdsmap=2,mgr=7} crc {logm=2146807168,mdsmap=3140514009,mgr=267573432}) 2026-02-17T05:49:52.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.665+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=91,mdsmap=3,mgr=6} crc {logm=3893638500,mdsmap=1160443688,mgr=2030165184}) 2026-02-17T05:49:52.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.666+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.666+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=24,osdmap=41} crc {mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3906473370,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2132305197,osd_snap=1296379714,osdmap=2085559843}) 2026-02-17T05:49:52.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.666+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=24,osdmap=43} crc {mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3337868927,mon_config_key=634641127,monmap=1784110042,osd_metadata=2240493509,osd_pg_creating=2132305197,osd_snap=1608360715,osdmap=3246098574}) 2026-02-17T05:49:52.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.666+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.666+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=24,osdmap=41} crc {mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3906473370,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2132305197,osd_snap=1296379714,osdmap=2085559843}) 2026-02-17T05:49:52.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.666+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=26,osdmap=41} crc {mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=966110815,mon_config_key=501681677,monmap=3287446063,osd_metadata=2997046373,osd_pg_creating=2132305197,osd_snap=3583409214,osdmap=64943200}) 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:688: done (0 left) 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:689: read oid 11 snap -1 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:689: expect deleted 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:690: copy_from oid 20 from oid 24 current snap is 68 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:691: read oid 45 snap -1 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:691: expect (ObjNum 201 snap 41 seq_num 201) 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:692: read oid 9 snap -1 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:692: expect (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:49:52.667 INFO:tasks.rados.rados.0.trial115.stdout:693: delete oid 21 current snap is 68 2026-02-17T05:49:52.670 INFO:tasks.rados.rados.0.trial115.stdout:689: done (4 left) 2026-02-17T05:49:52.670 INFO:tasks.rados.rados.0.trial115.stdout:693: done (3 left) 2026-02-17T05:49:52.670 INFO:tasks.rados.rados.0.trial115.stdout:694: read oid 8 snap -1 2026-02-17T05:49:52.670 INFO:tasks.rados.rados.0.trial115.stdout:694: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:49:52.670 INFO:tasks.rados.rados.0.trial115.stdout:695: delete oid 1 current snap is 68 2026-02-17T05:49:52.671 INFO:tasks.rados.rados.0.trial115.stdout:695: done (4 left) 2026-02-17T05:49:52.671 INFO:tasks.rados.rados.0.trial115.stdout:696: write (excl) oid 10 current snap is 68 2026-02-17T05:49:52.671 INFO:tasks.rados.rados.0.trial115.stdout:696: seq_num 290 ranges {474030=643624,1520131=735769} 2026-02-17T05:49:52.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.674+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.674+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=25} crc {osdmap=3155275097}) 2026-02-17T05:49:52.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.674+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=29} crc {osdmap=12915675}) 2026-02-17T05:49:52.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.674+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:49:52.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.674+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=25} crc {osdmap=3155275097}) 2026-02-17T05:49:52.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:49:52.674+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=28} crc {osdmap=3181342211}) 2026-02-17T05:49:52.675 INFO:tasks.rados.rados.0.trial115.stdout:696: writing trial11545420-10 from 474030 to 1117654 tid 1 2026-02-17T05:49:52.680 INFO:tasks.rados.rados.0.trial115.stdout:696: writing trial11545420-10 from 1520131 to 2255900 tid 2 2026-02-17T05:49:52.680 INFO:tasks.rados.rados.0.trial115.stdout:697: write oid 30 current snap is 68 2026-02-17T05:49:52.680 INFO:tasks.rados.rados.0.trial115.stdout:697: seq_num 291 ranges {490837=718873,1862260=479427,2875032=15805} 2026-02-17T05:49:52.684 INFO:tasks.rados.rados.0.trial115.stdout:697: writing trial11545420-30 from 490837 to 1209710 tid 1 2026-02-17T05:49:52.689 INFO:tasks.rados.rados.0.trial115.stdout:697: writing trial11545420-30 from 1862260 to 2341687 tid 2 2026-02-17T05:49:52.691 INFO:tasks.rados.rados.0.trial115.stdout:697: writing trial11545420-30 from 2875032 to 2890837 tid 3 2026-02-17T05:49:52.691 INFO:tasks.rados.rados.0.trial115.stdout:698: write oid 33 current snap is 68 2026-02-17T05:49:52.691 INFO:tasks.rados.rados.0.trial115.stdout:698: seq_num 292 ranges {507644=794122,1920742=623085} 2026-02-17T05:49:52.695 INFO:tasks.rados.rados.0.trial115.stdout:698: writing trial11545420-33 from 507644 to 1301766 tid 1 2026-02-17T05:49:52.700 INFO:tasks.rados.rados.0.trial115.stdout:698: writing trial11545420-33 from 1920742 to 2543827 tid 2 2026-02-17T05:49:52.700 INFO:tasks.rados.rados.0.trial115.stdout:699: snap_remove snap 61 2026-02-17T05:49:53.026 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:49:53.027 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:53.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient: get_monmap_and_config 2026-02-17T05:49:53.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient: monmap: 2026-02-17T05:49:53.096 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.096 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:53.097338+0000 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:53.097338+0000 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:53.097 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 1 Processor -- start 2026-02-17T05:49:53.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 1 -- start start 2026-02-17T05:49:53.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient: init 2026-02-17T05:49:53.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2026-02-17T05:49:53.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406c890 0x7f03d406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient(hunting): picked mon.noname-b con 0x7f03d406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f03d409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f03d406d1b0 0x7f03d4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient(hunting): picked mon.noname-c con 0x7f03d406d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d406daf0 con 0x7f03d409c4a0 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f03d40a0880 con 0x7f03d406d1b0 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f03d409cc80 con 0x7f03d406c890 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03daa6f640 10 monclient(hunting): authenticate will time out at 838.650869s 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03d3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406c890 0x7f03d406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03d37fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f03d409c4a0 0x7f03d4134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49152/0 (socket says 10.20.193.78:49152) 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03d37fe640 1 -- 10.20.193.78:0/3602700077 learned_addr learned my addr 10.20.193.78:0/3602700077 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03d3fff640 10 monclient(hunting): get_auth_request con 0x7f03d406c890 auth_method 0 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03d3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.096+0000 7f03d3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d8fe5640 1 --2- 10.20.193.78:0/3602700077 >> v2:10.20.193.115:3300/0 conn(0x7f03d406d1b0 0x7f03d4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d8fe5640 10 monclient(hunting): get_auth_request con 0x7f03d406d1b0 auth_method 0 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d8fe5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d8fe5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d8fe5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 10 monclient(hunting): handle_auth_done global_id 64115 payload 274 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 1 -- 10.20.193.78:0/3602700077 >> v2:10.20.193.115:3300/0 conn(0x7f03d406d1b0 msgr2=0x7f03d4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 1 --2- 10.20.193.78:0/3602700077 >> v2:10.20.193.115:3300/0 conn(0x7f03d406d1b0 0x7f03d4135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 1 -- 10.20.193.78:0/3602700077 >> v1:10.20.193.78:6789/0 conn(0x7f03d409c4a0 legacy=0x7f03d4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 1 monclient: found mon.noname-b 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:49:53.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.097+0000 7f03d3fff640 1 -- 10.20.193.78:0/3602700077 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d409cf60 con 0x7f03d406c890 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d3fff640 1 -- 10.20.193.78:0/3602700077 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406c890 msgr2=0x7f03d406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d3fff640 1 --2- 10.20.193.78:0/3602700077 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406c890 0x7f03d406cc70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f03c00098a0 tx=0x7f03c002ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc9b2a39eb79b81 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 1 -- 10.20.193.78:0/3602700077 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c003c070 con 0x7f03d406c890 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: dump: 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.109 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 1 -- 10.20.193.78:0/3602700077 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c002fc00 con 0x7f03d406c890 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03daa6f640 5 monclient: authenticate success, global_id 64115 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 1 -- 10.20.193.78:0/3602700077 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0037440 con 0x7f03d406c890 2026-02-17T05:49:53.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03d2ffd640 10 monclient: dump: 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03daa6f640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.099+0000 7f03daa6f640 4 set_mon_vals no callback set 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 10 monclient: shutdown 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 -- 10.20.193.78:0/3602700077 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406c890 msgr2=0x7f03d406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 --2- 10.20.193.78:0/3602700077 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406c890 0x7f03d406cc70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f03c00098a0 tx=0x7f03c002ed80 comp rx=0 tx=0).stop 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 -- 10.20.193.78:0/3602700077 shutdown_connections 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 --2- 10.20.193.78:0/3602700077 >> v2:10.20.193.115:3300/0 conn(0x7f03d406d1b0 0x7f03d4135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 --2- 10.20.193.78:0/3602700077 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406c890 0x7f03d406cc70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 -- 10.20.193.78:0/3602700077 >> 10.20.193.78:0/3602700077 conn(0x7f03d40a6620 msgr2=0x7f03d40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 -- 10.20.193.78:0/3602700077 shutdown_connections 2026-02-17T05:49:53.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 -- 10.20.193.78:0/3602700077 wait complete. 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 10 monclient: monmap: 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:53.113 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 Processor -- start 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 1 -- start start 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.100+0000 7f03daa6f640 10 monclient: init 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03d406c890 0x7f03d41cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient(hunting): picked mon.noname-a con 0x7f03d406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406d1b0 0x7f03d4125b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient(hunting): picked mon.noname-c con 0x7f03d406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03d409c4a0 0x7f03d41260b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient(hunting): picked mon.noname-b con 0x7f03d409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03d40a01a0 con 0x7f03d406c890 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03d40a0020 con 0x7f03d409c4a0 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f03d409be20 con 0x7f03d406d1b0 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03daa6f640 10 monclient(hunting): authenticate will time out at 838.655188s 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03d406c890 0x7f03d41cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406d1b0 0x7f03d4125b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03d406c890 0x7f03d41cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52748/0 (socket says 10.20.193.78:52748) 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 1 -- 10.20.193.78:0/304937259 learned_addr learned my addr 10.20.193.78:0/304937259 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 10 monclient(hunting): get_auth_request con 0x7f03d406c890 auth_method 0 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient(hunting): get_auth_request con 0x7f03d406d1b0 auth_method 0 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient(hunting): handle_auth_done global_id 64118 payload 824 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 1 -- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03d409c4a0 msgr2=0x7f03d41260b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03d409c4a0 0x7f03d41260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 1 -- 10.20.193.78:0/304937259 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03d406c890 msgr2=0x7f03d41cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03d406c890 0x7f03d41cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 1 monclient: found mon.noname-c 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.101+0000 7f03d37fe640 1 -- 10.20.193.78:0/304937259 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d412a4e0 con 0x7f03d406d1b0 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d37fe640 1 -- 10.20.193.78:0/304937259 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406d1b0 msgr2=0x7f03d4125b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d37fe640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406d1b0 0x7f03d4125b70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f03c400c4d0 tx=0x7f03c400c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 1 -- 10.20.193.78:0/304937259 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4007070 con 0x7f03d406d1b0 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: dump: 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.118 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 1 -- 10.20.193.78:0/304937259 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c4007230 con 0x7f03d406d1b0 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 1 -- 10.20.193.78:0/304937259 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4007620 con 0x7f03d406d1b0 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d0ff9640 10 monclient: dump: 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.119 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03d27fc640 4 set_mon_vals no callback set 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03daa6f640 5 monclient: authenticate success, global_id 64118 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03daa6f640 10 monclient: _renew_subs 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03daa6f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:53.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d4129540 con 0x7f03d406d1b0 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03daa6f640 10 monclient: _renew_subs 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03daa6f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.103+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d4072770 con 0x7f03d406d1b0 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03d0ff9640 1 -- 10.20.193.78:0/304937259 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f03c401b400 con 0x7f03d406d1b0 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03daa6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03daa6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03daa6f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d4074980 con 0x7f03d406d1b0 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03d0ff9640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f03a403deb0 0x7f03a4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03d0ff9640 1 -- 10.20.193.78:0/304937259 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7f03c40524d0 con 0x7f03d406d1b0 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03d3fff640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f03a403deb0 0x7f03a4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.104+0000 7f03d3fff640 10 monclient: get_auth_request con 0x7f03a403deb0 auth_method 0 2026-02-17T05:49:53.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.107+0000 7f03d3fff640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f03a403deb0 0x7f03a4040370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f03c002edc0 tx=0x7f03c0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.107+0000 7f03d0ff9640 1 -- 10.20.193.78:0/304937259 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f03d412a4e0 con 0x7f03d406d1b0 2026-02-17T05:49:53.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.107+0000 7f03d0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.107+0000 7f03d0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:53.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.221+0000 7f03daa6f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:53.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.221+0000 7f03daa6f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:53.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.221+0000 7f03daa6f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:49:53.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.221+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f03d406cc70 con 0x7f03d406d1b0 2026-02-17T05:49:53.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.221+0000 7f03d0ff9640 1 -- 10.20.193.78:0/304937259 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f03c401ac60 con 0x7f03d406d1b0 2026-02-17T05:49:53.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.221+0000 7f03d0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:53.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.221+0000 7f03d0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:53.223 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f03a403deb0 msgr2=0x7f03a4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f03a403deb0 0x7f03a4040370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f03c002edc0 tx=0x7f03c0033000 comp rx=0 tx=0).stop 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 10 monclient: shutdown 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406d1b0 msgr2=0x7f03d4125b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406d1b0 0x7f03d4125b70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f03c400c4d0 tx=0x7f03c400c9a0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 shutdown_connections 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f03a403deb0 0x7f03a4040370 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03d409c4a0 0x7f03d41260b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03d406d1b0 0x7f03d4125b70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 --2- 10.20.193.78:0/304937259 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03d406c890 0x7f03d41cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 >> 10.20.193.78:0/304937259 conn(0x7f03d40a6620 msgr2=0x7f03d4071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 shutdown_connections 2026-02-17T05:49:53.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.222+0000 7f03daa6f640 1 -- 10.20.193.78:0/304937259 wait complete. 2026-02-17T05:49:53.230 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:49:53.231 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:49:53.231 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:49:53.231 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.297+0000 7f2c13fa0640 10 monclient: get_monmap_and_config 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.297+0000 7f2c13fa0640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.297+0000 7f2c13fa0640 10 monclient: monmap: 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:53.299019+0000 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:53.299019+0000 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:53.298 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:53.299 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:53.299 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 1 Processor -- start 2026-02-17T05:49:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 1 -- start start 2026-02-17T05:49:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient: init 2026-02-17T05:49:53.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2c0c06c890 0x7f2c0c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient(hunting): picked mon.noname-a con 0x7f2c0c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2c0c06cc70 0x7f2c0c06d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient(hunting): picked mon.noname-c con 0x7f2c0c06cc70 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c134df0 0x7f2c0c135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient(hunting): picked mon.noname-b con 0x7f2c0c134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f2c0c074150 con 0x7f2c0c06c890 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2c0c09ef70 con 0x7f2c0c06cc70 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0c09f250 con 0x7f2c0c134df0 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c13fa0640 10 monclient(hunting): authenticate will time out at 838.852526s 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2c0c06c890 0x7f2c0c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2c0c06c890 0x7f2c0c0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52750/0 (socket says 10.20.193.78:52750) 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 -- 10.20.193.78:0/1514737030 learned_addr learned my addr 10.20.193.78:0/1514737030 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient(hunting): get_auth_request con 0x7f2c0c06c890 auth_method 0 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 1 --2- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c134df0 0x7f2c0c135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 10 monclient(hunting): get_auth_request con 0x7f2c0c134df0 auth_method 0 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11514640 1 --2- 10.20.193.78:0/1514737030 >> v2:10.20.193.115:3300/0 conn(0x7f2c0c06cc70 0x7f2c0c06d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11514640 10 monclient(hunting): get_auth_request con 0x7f2c0c06cc70 auth_method 0 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11514640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11514640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11514640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient(hunting): handle_auth_done global_id 54237 payload 274 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 -- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c134df0 msgr2=0x7f2c0c135220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 --2- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c134df0 0x7f2c0c135220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 -- 10.20.193.78:0/1514737030 >> v2:10.20.193.115:3300/0 conn(0x7f2c0c06cc70 msgr2=0x7f2c0c06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 --2- 10.20.193.78:0/1514737030 >> v2:10.20.193.115:3300/0 conn(0x7f2c0c06cc70 0x7f2c0c06d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 monclient: found mon.noname-a 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c11d15640 1 -- 10.20.193.78:0/1514737030 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c09df40 con 0x7f2c0c06c890 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.298+0000 7f2c12516640 1 --2- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c134df0 0x7f2c0c135220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c11d15640 1 -- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06c890 msgr2=0x7f2c0c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c11d15640 1 --2- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06c890 0x7f2c0c0a01f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c074e00 tx=0x7f2bfc0317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f40c0ce6b622b5f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 1 -- 10.20.193.78:0/1514737030 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc036920 con 0x7f2c0c06c890 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: dump: 2026-02-17T05:49:53.312 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 5 monclient: authenticate success, global_id 54237 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 1 -- 10.20.193.78:0/1514737030 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc036ac0 con 0x7f2c0c06c890 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 1 -- 10.20.193.78:0/1514737030 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc038400 con 0x7f2c0c06c890 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c10d13640 10 monclient: dump: 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.314 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 4 set_mon_vals no callback set 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 10 monclient: shutdown 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 -- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06c890 msgr2=0x7f2c0c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 --2- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06c890 0x7f2c0c0a01f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c074e00 tx=0x7f2bfc0317a0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 -- 10.20.193.78:0/1514737030 shutdown_connections 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 --2- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c134df0 0x7f2c0c135220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 --2- 10.20.193.78:0/1514737030 >> v2:10.20.193.115:3300/0 conn(0x7f2c0c06cc70 0x7f2c0c06d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 --2- 10.20.193.78:0/1514737030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06c890 0x7f2c0c0a01f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 -- 10.20.193.78:0/1514737030 >> 10.20.193.78:0/1514737030 conn(0x7f2c0c0a6620 msgr2=0x7f2c0c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 -- 10.20.193.78:0/1514737030 shutdown_connections 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.301+0000 7f2c13fa0640 1 -- 10.20.193.78:0/1514737030 wait complete. 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient: monmap: 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:53.316 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 1 Processor -- start 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 1 -- start start 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient: init 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:49:53.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c06c890 0x7f2c0c1cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient(hunting): picked mon.noname-c con 0x7f2c0c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06cc70 0x7f2c0c1cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient(hunting): picked mon.noname-a con 0x7f2c0c06cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2c0c134df0 0x7f2c0c120380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient(hunting): picked mon.noname-b con 0x7f2c0c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c09d360 con 0x7f2c0c06cc70 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c09d1e0 con 0x7f2c0c134df0 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0c135760 con 0x7f2c0c06c890 2026-02-17T05:49:53.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c13fa0640 10 monclient(hunting): authenticate will time out at 838.856778s 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c06c890 0x7f2c0c1cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06cc70 0x7f2c0c1cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c06c890 0x7f2c0c1cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36796/0 (socket says 10.20.193.78:36796) 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 1 -- 10.20.193.78:0/2325052786 learned_addr learned my addr 10.20.193.78:0/2325052786 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 10 monclient(hunting): get_auth_request con 0x7f2c0c06c890 auth_method 0 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient(hunting): get_auth_request con 0x7f2c0c06cc70 auth_method 0 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c12516640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2c0c134df0 0x7f2c0c120380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient(hunting): handle_auth_done global_id 54240 payload 824 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c06c890 msgr2=0x7f2c0c1cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2c0c06c890 0x7f2c0c1cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2c0c134df0 msgr2=0x7f2c0c120380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2c0c134df0 0x7f2c0c120380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 1 monclient: found mon.noname-a 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.302+0000 7f2c11514640 1 -- 10.20.193.78:0/2325052786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c1224c0 con 0x7f2c0c06cc70 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11514640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06cc70 msgr2=0x7f2c0c1cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11514640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06cc70 0x7f2c0c1cb700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000e810 tx=0x7f2c0000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00017070 con 0x7f2c0c06cc70 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: dump: 2026-02-17T05:49:53.321 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0000cae0 con 0x7f2c0c06cc70 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000f040 con 0x7f2c0c06cc70 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfbfff640 4 set_mon_vals no callback set 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 10 monclient: dump: 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.323 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 5 monclient: authenticate success, global_id 54240 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 10 monclient: _renew_subs 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c123d50 con 0x7f2c0c06cc70 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 10 monclient: _renew_subs 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c124030 con 0x7f2c0c06cc70 2026-02-17T05:49:53.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1208c0 0x7f2c0c120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c074980 con 0x7f2c0c1208c0 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11d15640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1208c0 0x7f2c0c120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11d15640 10 monclient: get_auth_request con 0x7f2c0c1208c0 auth_method 0 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11d15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11d15640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11d15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11514640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06cc70 0x7f2c0c1cb700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000e810 tx=0x7f2c0000ece0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f2c00012050 mgrmap(e 5) 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.305+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2c00012050 con 0x7f2c0c06cc70 2026-02-17T05:49:53.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.306+0000 7f2c11d15640 10 monclient(hunting): handle_auth_done global_id 54243 payload 824 2026-02-17T05:49:53.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.306+0000 7f2c11d15640 1 -- 10.20.193.78:0/2325052786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c0c072d60 con 0x7f2c0c1208c0 2026-02-17T05:49:53.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.306+0000 7f2bfa7fc640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bdc03deb0 0x7f2bdc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.306+0000 7f2c12516640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bdc03deb0 0x7f2bdc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.306+0000 7f2c12516640 10 monclient: get_auth_request con 0x7f2bdc03deb0 auth_method 0 2026-02-17T05:49:53.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2c11d15640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1208c0 msgr2=0x7f2c0c120ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2c11d15640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1208c0 0x7f2c0c120ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00b340 tx=0x7f2bfc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc038cd0 con 0x7f2c0c1208c0 2026-02-17T05:49:53.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 10 monclient: dump: 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2bfc038cd0 con 0x7f2c0c1208c0 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1208c0 msgr2=0x7f2c0c120ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1208c0 0x7f2c0c120ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00b340 tx=0x7f2bfc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1208c0 msgr2=0x7f2c0c120ca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2bfa7fc640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1208c0 0x7f2c0c120ca0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00b340 tx=0x7f2bfc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.308+0000 7f2c12516640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bdc03deb0 0x7f2bdc040370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c123880 tx=0x7f2c0800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c13fa0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c13fa0640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1211e0 0x7f2c0c1d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c13fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c135b90 con 0x7f2c0c1211e0 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1211e0 0x7f2c0c1d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 10 monclient: get_auth_request con 0x7f2c0c1211e0 auth_method 0 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 10 monclient(hunting): handle_auth_done global_id 54246 payload 824 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.311+0000 7f2c11d15640 1 -- 10.20.193.78:0/2325052786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2c0c0744d0 con 0x7f2c0c1211e0 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c11d15640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1211e0 msgr2=0x7f2c0c1d61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c11d15640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1211e0 0x7f2c0c1d61c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc039000 tx=0x7f2bfc037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc056070 con 0x7f2c0c1211e0 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 10 monclient: dump: 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f2bfc003710 con 0x7f2c0c1211e0 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1211e0 msgr2=0x7f2c0c1d61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1211e0 0x7f2c0c1d61c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc039000 tx=0x7f2bfc037000 comp rx=0 tx=0).stop 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1211e0 msgr2=0x7f2c0c1d61c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c1211e0 0x7f2c0c1d61c0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc039000 tx=0x7f2bfc037000 comp rx=0 tx=0).stop 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 40, 2026-02-17T05:49:53.334 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 62789, 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.335 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.336 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:53.337 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.338 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.339 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.340 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.341 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.342 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.343 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.344 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.345 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:53.345 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bdc03deb0 msgr2=0x7f2bdc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bdc03deb0 0x7f2bdc040370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c123880 tx=0x7f2c0800b040 comp rx=0 tx=0).stop 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 10 monclient: shutdown 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06cc70 msgr2=0x7f2c0c1cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 1 --2- 10.20.193.78:0/2325052786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2c0c06cc70 0x7f2c0c1cb700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000e810 tx=0x7f2c0000ece0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c12516640 1 -- 10.20.193.78:0/2325052786 reap_dead start 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2bfa7fc640 1 -- 10.20.193.78:0/2325052786 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (unknown 0 0 0) 0x7f2c00052570 con 0x7f2c0c06cc70 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c12516640 1 -- 10.20.193.78:0/2325052786 reap_dead start 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 shutdown_connections 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 >> 10.20.193.78:0/2325052786 conn(0x7f2c0c0a6620 msgr2=0x7f2c0c070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 shutdown_connections 2026-02-17T05:49:53.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.314+0000 7f2c13fa0640 1 -- 10.20.193.78:0/2325052786 wait complete. 2026-02-17T05:49:53.347 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:53.347 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:49:53.347 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:49:53.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.414+0000 7ff581476640 10 monclient: get_monmap_and_config 2026-02-17T05:49:53.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.414+0000 7ff581476640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.414+0000 7ff581476640 10 monclient: monmap: 2026-02-17T05:49:53.414 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.414 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.414 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:53.415506+0000 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:53.415506+0000 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:53.415 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.414+0000 7ff581476640 1 Processor -- start 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.414+0000 7ff581476640 1 -- start start 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.414+0000 7ff581476640 10 monclient: init 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff57c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff57c09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff57c06d380 0x7ff57c13ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 10 monclient(hunting): picked mon.noname-c con 0x7ff57c06d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57c06daf0 con 0x7ff57c09c4a0 2026-02-17T05:49:53.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57c09cc80 con 0x7ff57c06c890 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff57c0a0880 con 0x7ff57c06d380 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff581476640 10 monclient(hunting): authenticate will time out at 838.969112s 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57a7fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff57c09c4a0 0x7ff57c13b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49160/0 (socket says 10.20.193.78:49160) 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57a7fc640 1 -- 10.20.193.78:0/66839238 learned_addr learned my addr 10.20.193.78:0/66839238 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 1 --2- 10.20.193.78:0/66839238 >> v2:10.20.193.115:3300/0 conn(0x7ff57c06d380 0x7ff57c13ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient(hunting): get_auth_request con 0x7ff57c06d380 auth_method 0 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient(hunting): handle_auth_done global_id 44263 payload 274 2026-02-17T05:49:53.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 1 -- 10.20.193.78:0/66839238 >> v1:10.20.193.115:6789/0 conn(0x7ff57c06c890 legacy=0x7ff57c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 1 -- 10.20.193.78:0/66839238 >> v1:10.20.193.78:6789/0 conn(0x7ff57c09c4a0 legacy=0x7ff57c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 1 monclient: found mon.noname-c 2026-02-17T05:49:53.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:49:53.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.415+0000 7ff57b7fe640 1 -- 10.20.193.78:0/66839238 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c09cf60 con 0x7ff57c06d380 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff57b7fe640 1 -- 10.20.193.78:0/66839238 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff57c06d380 msgr2=0x7ff57c13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff57b7fe640 1 --2- 10.20.193.78:0/66839238 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff57c06d380 0x7ff57c13ead0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff57c074de0 tx=0x7ff56c02f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a8763fd134f13a6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 1 -- 10.20.193.78:0/66839238 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c03c070 con 0x7ff57c06d380 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: dump: 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.422 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff581476640 5 monclient: authenticate success, global_id 44263 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff581476640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 1 -- 10.20.193.78:0/66839238 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56c037640 con 0x7ff57c06d380 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 1 -- 10.20.193.78:0/66839238 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c037a10 con 0x7ff57c06d380 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff581476640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:53.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff581476640 4 set_mon_vals no callback set 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff579ffb640 10 monclient: dump: 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.418+0000 7ff581476640 10 monclient: shutdown 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- 10.20.193.78:0/66839238 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff57c06d380 msgr2=0x7ff57c13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 --2- 10.20.193.78:0/66839238 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff57c06d380 0x7ff57c13ead0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff57c074de0 tx=0x7ff56c02f130 comp rx=0 tx=0).stop 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- 10.20.193.78:0/66839238 shutdown_connections 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 --2- 10.20.193.78:0/66839238 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff57c06d380 0x7ff57c13ead0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- 10.20.193.78:0/66839238 >> 10.20.193.78:0/66839238 conn(0x7ff57c0a6620 msgr2=0x7ff57c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- 10.20.193.78:0/66839238 shutdown_connections 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- 10.20.193.78:0/66839238 wait complete. 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient: monmap: 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:53.425 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 Processor -- start 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- start start 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient: init 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c06c890 0x7ff57c131dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient(hunting): picked mon.noname-c con 0x7ff57c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff57c06d380 0x7ff57c132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient(hunting): picked mon.noname-b con 0x7ff57c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff57c09c4a0 0x7ff57c132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient(hunting): picked mon.noname-a con 0x7ff57c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c09bfa0 con 0x7ff57c09c4a0 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c09be20 con 0x7ff57c06d380 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff57c13f010 con 0x7ff57c06c890 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.419+0000 7ff581476640 10 monclient(hunting): authenticate will time out at 838.973962s 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff57c09c4a0 0x7ff57c132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c06c890 0x7ff57c131dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff57c09c4a0 0x7ff57c132840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52768/0 (socket says 10.20.193.78:52768) 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c06c890 0x7ff57c131dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36802/0 (socket says 10.20.193.78:36802) 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 -- 10.20.193.78:0/3024102134 learned_addr learned my addr 10.20.193.78:0/3024102134 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient(hunting): get_auth_request con 0x7ff57c09c4a0 auth_method 0 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 10 monclient(hunting): get_auth_request con 0x7ff57c06c890 auth_method 0 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient(hunting): handle_auth_done global_id 54249 payload 824 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c06c890 msgr2=0x7ff57c131dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c06c890 0x7ff57c131dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff57c06d380 msgr2=0x7ff57c132300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff57c06d380 0x7ff57c132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57b7fe640 1 -- 10.20.193.78:0/3024102134 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c13cce0 con 0x7ff57c09c4a0 2026-02-17T05:49:53.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.420+0000 7ff57affd640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c06c890 0x7ff57c131dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:49:53.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff57b7fe640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff57c09c4a0 msgr2=0x7ff57c132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff57b7fe640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff57c09c4a0 0x7ff57c132840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009e90 tx=0x7ff56c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c047020 con 0x7ff57c09c4a0 2026-02-17T05:49:53.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: dump: 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff581476640 5 monclient: authenticate success, global_id 54249 2026-02-17T05:49:53.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff581476640 10 monclient: _renew_subs 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff581476640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56c042880 con 0x7ff57c09c4a0 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57c06fcc0 con 0x7ff57c09c4a0 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c03c040 con 0x7ff57c09c4a0 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff55b7fe640 10 monclient: dump: 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.434 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff5797fa640 4 set_mon_vals no callback set 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff581476640 10 monclient: _renew_subs 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff581476640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.422+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57c13ca00 con 0x7ff57c09c4a0 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff581476640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff581476640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c063880 0x7ff57c063c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff581476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff57c06cc70 con 0x7ff57c063880 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff56c043420 con 0x7ff57c09c4a0 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c063880 0x7ff57c063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 10 monclient: get_auth_request con 0x7ff57c063880 auth_method 0 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 10 monclient(hunting): handle_auth_done global_id 64130 payload 824 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57affd640 1 -- 10.20.193.78:0/3024102134 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff57c13b5b0 con 0x7ff57c063880 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff55b7fe640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff54803deb0 0x7ff548040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7ff56c078670 con 0x7ff57c09c4a0 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57a7fc640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff54803deb0 0x7ff548040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.423+0000 7ff57a7fc640 10 monclient: get_auth_request con 0x7ff54803deb0 auth_method 0 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff57affd640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c063880 msgr2=0x7ff57c063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff57affd640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c063880 0x7ff57c063c60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff57000fb10 tx=0x7ff57000d150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570018020 con 0x7ff57c063880 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 10 monclient: dump: 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.436 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff570014470 con 0x7ff57c063880 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c063880 msgr2=0x7ff57c063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c063880 0x7ff57c063c60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff57000fb10 tx=0x7ff57000d150 comp rx=0 tx=0).stop 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c063880 msgr2=0x7ff57c063c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.425+0000 7ff55b7fe640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c063880 0x7ff57c063c60 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff57000fb10 tx=0x7ff57000d150 comp rx=0 tx=0).stop 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.426+0000 7ff57a7fc640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff54803deb0 0x7ff548040370 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff564007fd0 tx=0x7ff564007ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.428+0000 7ff581476640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.428+0000 7ff581476640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c133ff0 0x7ff57c1d8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.428+0000 7ff581476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.428+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff57c13d020 con 0x7ff57c133ff0 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c133ff0 0x7ff57c1d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 10 monclient: get_auth_request con 0x7ff57c133ff0 auth_method 0 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 10 monclient(hunting): handle_auth_done global_id 64133 payload 824 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.429+0000 7ff57affd640 1 -- 10.20.193.78:0/3024102134 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff57c0744d0 con 0x7ff57c133ff0 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff57affd640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c133ff0 msgr2=0x7ff57c1d8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff57affd640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c133ff0 0x7ff57c1d8420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff570019000 tx=0x7ff570003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570018030 con 0x7ff57c133ff0 2026-02-17T05:49:53.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 10 monclient: dump: 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7ff570014a30 con 0x7ff57c133ff0 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c133ff0 msgr2=0x7ff57c1d8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c133ff0 0x7ff57c1d8420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff570019000 tx=0x7ff570003860 comp rx=0 tx=0).stop 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c133ff0 msgr2=0x7ff57c1d8420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff55b7fe640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff57c133ff0 0x7ff57c1d8420 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff570019000 tx=0x7ff570003860 comp rx=0 tx=0).stop 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 40, 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 12494, 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:53.440 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.441 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.442 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:53.443 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.444 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:53.445 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.446 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.447 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.448 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:53.449 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:53.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff54803deb0 msgr2=0x7ff548040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff54803deb0 0x7ff548040370 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff564007fd0 tx=0x7ff564007ba0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 10 monclient: shutdown 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff57c09c4a0 msgr2=0x7ff57c132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 1 --2- 10.20.193.78:0/3024102134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff57c09c4a0 0x7ff57c132840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009e90 tx=0x7ff56c004b40 comp rx=0 tx=0).stop 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff57b7fe640 1 -- 10.20.193.78:0/3024102134 reap_dead start 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff57b7fe640 1 -- 10.20.193.78:0/3024102134 reap_dead start 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 shutdown_connections 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 >> 10.20.193.78:0/3024102134 conn(0x7ff57c0a6620 msgr2=0x7ff57c0a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 shutdown_connections 2026-02-17T05:49:53.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.431+0000 7ff581476640 1 -- 10.20.193.78:0/3024102134 wait complete. 2026-02-17T05:49:53.451 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:53.451 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:49:53.452 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.518+0000 7f03116c4640 10 monclient: get_monmap_and_config 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.518+0000 7f03116c4640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.518+0000 7f03116c4640 10 monclient: monmap: 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:53.519569+0000 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:53.519569+0000 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:53.518 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.518+0000 7f03116c4640 1 Processor -- start 2026-02-17T05:49:53.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.518+0000 7f03116c4640 1 -- start start 2026-02-17T05:49:53.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.518+0000 7f03116c4640 10 monclient: init 2026-02-17T05:49:53.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T05:49:53.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f030c09ea90 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:53.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f030c09ee70 0x7f030c09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient(hunting): picked mon.noname-c con 0x7f030c09ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f030c13d640 0x7f030c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient(hunting): picked mon.noname-a con 0x7f030c13d640 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030c0a0a20 con 0x7f030c09ea90 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f030c0a0d00 con 0x7f030c13d640 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f030c0a0460 con 0x7f030c09ee70 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f03116c4640 10 monclient(hunting): authenticate will time out at 839.073242s 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030affd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f030c09ea90 0x7f030c0a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49164/0 (socket says 10.20.193.78:49164) 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030affd640 1 -- 10.20.193.78:0/587937855 learned_addr learned my addr 10.20.193.78:0/587937855 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 1 --2- 10.20.193.78:0/587937855 >> v2:10.20.193.78:3300/0 conn(0x7f030c13d640 0x7f030c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient(hunting): get_auth_request con 0x7f030c13d640 auth_method 0 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030a7fc640 1 --2- 10.20.193.78:0/587937855 >> v2:10.20.193.115:3300/0 conn(0x7f030c09ee70 0x7f030c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030a7fc640 10 monclient(hunting): get_auth_request con 0x7f030c09ee70 auth_method 0 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient(hunting): handle_auth_done global_id 54252 payload 274 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 1 -- 10.20.193.78:0/587937855 >> v2:10.20.193.115:3300/0 conn(0x7f030c09ee70 msgr2=0x7f030c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 1 --2- 10.20.193.78:0/587937855 >> v2:10.20.193.115:3300/0 conn(0x7f030c09ee70 0x7f030c09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 1 -- 10.20.193.78:0/587937855 >> v1:10.20.193.78:6789/0 conn(0x7f030c09ea90 legacy=0x7f030c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.519+0000 7f030b7fe640 1 -- 10.20.193.78:0/587937855 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c0a0740 con 0x7f030c13d640 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f030b7fe640 1 -- 10.20.193.78:0/587937855 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 msgr2=0x7f030c13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f030b7fe640 1 --2- 10.20.193.78:0/587937855 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 0x7f030c13fa30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f030c06e270 tx=0x7f02fc02f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b28e0e384ed5a7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 1 -- 10.20.193.78:0/587937855 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc03c070 con 0x7f030c13d640 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: dump: 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 5 monclient: authenticate success, global_id 54252 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 1 -- 10.20.193.78:0/587937855 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02fc037440 con 0x7f030c13d640 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 1 -- 10.20.193.78:0/587937855 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc037810 con 0x7f030c13d640 2026-02-17T05:49:53.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 4 set_mon_vals no callback set 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f0309ffb640 10 monclient: dump: 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.523 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 10 monclient: shutdown 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 1 -- 10.20.193.78:0/587937855 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 msgr2=0x7f030c13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 1 --2- 10.20.193.78:0/587937855 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 0x7f030c13fa30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f030c06e270 tx=0x7f02fc02f130 comp rx=0 tx=0).stop 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 1 -- 10.20.193.78:0/587937855 shutdown_connections 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 1 --2- 10.20.193.78:0/587937855 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 0x7f030c13fa30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 1 --2- 10.20.193.78:0/587937855 >> v2:10.20.193.115:3300/0 conn(0x7f030c09ee70 0x7f030c09f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 1 -- 10.20.193.78:0/587937855 >> 10.20.193.78:0/587937855 conn(0x7f030c0a7520 msgr2=0x7f030c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 1 -- 10.20.193.78:0/587937855 shutdown_connections 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 1 -- 10.20.193.78:0/587937855 wait complete. 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.522+0000 7f03116c4640 10 monclient: monmap: 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 1 Processor -- start 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 1 -- start start 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient: init 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f030c09ea90 0x7f030c13b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient(hunting): picked mon.noname-c con 0x7f030c09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c09ee70 0x7f030c13be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient(hunting): picked mon.noname-b con 0x7f030c09ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 0x7f030c13c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient(hunting): picked mon.noname-a con 0x7f030c13d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f030c0a6820 con 0x7f030c13d640 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f030c0a66a0 con 0x7f030c09ee70 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f030c13ff70 con 0x7f030c09ea90 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f03116c4640 10 monclient(hunting): authenticate will time out at 839.077400s 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 0x7f030c13c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f030c09ea90 0x7f030c13b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 0x7f030c13c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52788/0 (socket says 10.20.193.78:52788) 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 -- 10.20.193.78:0/1770170335 learned_addr learned my addr 10.20.193.78:0/1770170335 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient(hunting): get_auth_request con 0x7f030c13d640 auth_method 0 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030affd640 10 monclient(hunting): get_auth_request con 0x7f030c09ea90 auth_method 0 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030a7fc640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c09ee70 0x7f030c13be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient(hunting): handle_auth_done global_id 54255 payload 824 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f030c09ea90 msgr2=0x7f030c13b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f030c09ea90 0x7f030c13b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c09ee70 msgr2=0x7f030c13be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c09ee70 0x7f030c13be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 monclient: found mon.noname-a 2026-02-17T05:49:53.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030b7fe640 1 -- 10.20.193.78:0/1770170335 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c0a1c10 con 0x7f030c13d640 2026-02-17T05:49:53.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030a7fc640 10 monclient: get_auth_request con 0x7f030c09ee70 auth_method 0 2026-02-17T05:49:53.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.523+0000 7f030a7fc640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c09ee70 0x7f030c13be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f030b7fe640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 msgr2=0x7f030c13c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f030b7fe640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 0x7f030c13c3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02fc03a040 tx=0x7f02fc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc03c050 con 0x7f030c13d640 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: dump: 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02fc048070 con 0x7f030c13d640 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f03116c4640 5 monclient: authenticate success, global_id 54255 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f03116c4640 10 monclient: _renew_subs 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f03116c4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c0a9b50 con 0x7f030c13d640 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc041c60 con 0x7f030c13d640 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f03097fa640 4 set_mon_vals no callback set 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f02eb7fe640 10 monclient: dump: 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.527 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f03116c4640 10 monclient: _renew_subs 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f03116c4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.525+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c070d90 con 0x7f030c13d640 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f03116c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f03116c4640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cefc0 0x7f030c1cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f03116c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f030c070b90 con 0x7f030c1cefc0 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f02fc04d460 con 0x7f030c13d640 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cefc0 0x7f030c1cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 10 monclient: get_auth_request con 0x7f030c1cefc0 auth_method 0 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f02eb7fe640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02d803deb0 0x7f02d8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7f02fc078750 con 0x7f030c13d640 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030a7fc640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02d803deb0 0x7f02d8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 10 monclient(hunting): handle_auth_done global_id 44269 payload 824 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030affd640 1 -- 10.20.193.78:0/1770170335 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f030c09f2c0 con 0x7f030c1cefc0 2026-02-17T05:49:53.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.526+0000 7f030a7fc640 10 monclient: get_auth_request con 0x7f02d803deb0 auth_method 0 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f030affd640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cefc0 msgr2=0x7f030c1cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f030affd640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cefc0 0x7f030c1cf3a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f030000eaa0 tx=0x7f030000ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f030a7fc640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02d803deb0 0x7f02d8040370 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f030c074480 tx=0x7f02f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030000cbf0 con 0x7f030c1cefc0 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 10 monclient: dump: 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.529 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f0300019070 con 0x7f030c1cefc0 2026-02-17T05:49:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cefc0 msgr2=0x7f030c1cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cefc0 0x7f030c1cf3a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f030000eaa0 tx=0x7f030000ef70 comp rx=0 tx=0).stop 2026-02-17T05:49:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cefc0 msgr2=0x7f030c1cf3a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.529+0000 7f02eb7fe640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cefc0 0x7f030c1cf3a0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f030000eaa0 tx=0x7f030000ef70 comp rx=0 tx=0).stop 2026-02-17T05:49:53.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.532+0000 7f03116c4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:49:53.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.532+0000 7f03116c4640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cf8e0 0x7f030c1da310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.532+0000 7f03116c4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.532+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f030c13d3c0 con 0x7f030c1cf8e0 2026-02-17T05:49:53.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.532+0000 7f030affd640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cf8e0 0x7f030c1da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 10 monclient: get_auth_request con 0x7f030c1cf8e0 auth_method 0 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 10 monclient(hunting): handle_auth_done global_id 44272 payload 824 2026-02-17T05:49:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.533+0000 7f030affd640 1 -- 10.20.193.78:0/1770170335 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f030c0744d0 con 0x7f030c1cf8e0 2026-02-17T05:49:53.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.535+0000 7f030affd640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cf8e0 msgr2=0x7f030c1da310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.535+0000 7f030affd640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cf8e0 0x7f030c1da310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f030000eaa0 tx=0x7f0300004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.535+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030001e250 con 0x7f030c1cf8e0 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 10 monclient: dump: 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f030001e250 con 0x7f030c1cf8e0 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cf8e0 msgr2=0x7f030c1da310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cf8e0 0x7f030c1da310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f030000eaa0 tx=0x7f0300004a40 comp rx=0 tx=0).stop 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cf8e0 msgr2=0x7f030c1da310 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:49:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f02eb7fe640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f030c1cf8e0 0x7f030c1da310 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f030000eaa0 tx=0x7f0300004a40 comp rx=0 tx=0).stop 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 40, 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 117184, 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:49:53.537 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:49:53.538 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.539 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:49:53.540 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:49:53.541 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02d803deb0 msgr2=0x7f02d8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02d803deb0 0x7f02d8040370 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f030c074480 tx=0x7f02f4008040 comp rx=0 tx=0).stop 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 10 monclient: shutdown 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 msgr2=0x7f030c13c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 1 --2- 10.20.193.78:0/1770170335 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f030c13d640 0x7f030c13c3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02fc03a040 tx=0x7f02fc004b40 comp rx=0 tx=0).stop 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f030b7fe640 1 -- 10.20.193.78:0/1770170335 reap_dead start 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f030b7fe640 1 -- 10.20.193.78:0/1770170335 reap_dead start 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 shutdown_connections 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 >> 10.20.193.78:0/1770170335 conn(0x7f030c0a7520 msgr2=0x7f030c136d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 shutdown_connections 2026-02-17T05:49:53.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.536+0000 7f03116c4640 1 -- 10.20.193.78:0/1770170335 wait complete. 2026-02-17T05:49:53.542 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:49:53.543 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-17T05:49:53.543 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:49:53.543 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-17T05:49:53.543 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T05:49:53.543 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-17T05:49:53.543 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:49:53.551 INFO:tasks.ceph.mon.c:Stopped 2026-02-17T05:49:53.551 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:49:53.551 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.658+0000 7fb2e29a9640 10 monclient: get_monmap_and_config 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.658+0000 7fb2e29a9640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.658+0000 7fb2e29a9640 10 monclient: monmap: 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:53.659564+0000 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:53.659564+0000 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.658 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:53.659 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:53.659 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:53.659 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:53.659 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:53.659 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.658+0000 7fb2e29a9640 1 Processor -- start 2026-02-17T05:49:53.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.658+0000 7fb2e29a9640 1 -- start start 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.658+0000 7fb2e29a9640 10 monclient: init 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb2dc06c890 0x7fb2dc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient(hunting): picked mon.noname-c con 0x7fb2dc06c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 0x7fb2dc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient(hunting): picked mon.noname-b con 0x7fb2dc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb2dc134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2dc06daf0 con 0x7fb2dc134df0 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb2dc0a0880 con 0x7fb2dc06c890 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb2dc06d6f0 con 0x7fb2dc09c4a0 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e29a9640 10 monclient(hunting): authenticate will time out at 839.213115s 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2db7fe640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 msgr2=0x7fb2dc06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2db7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 0x7fb2dc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e0f1f640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb2dc134df0 0x7fb2dc135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49176/0 (socket says 10.20.193.78:49176) 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2e0f1f640 1 -- 10.20.193.78:0/2256625552 learned_addr learned my addr 10.20.193.78:0/2256625552 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 1 --2- 10.20.193.78:0/2256625552 >> v2:10.20.193.115:3300/0 conn(0x7fb2dc06c890 0x7fb2dc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient(hunting): get_auth_request con 0x7fb2dc06c890 auth_method 0 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient(hunting): handle_auth_done global_id 44275 payload 274 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 1 -- 10.20.193.78:0/2256625552 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 msgr2=0x7fb2dc06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 1 --2- 10.20.193.78:0/2256625552 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 0x7fb2dc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 1 -- 10.20.193.78:0/2256625552 >> v1:10.20.193.78:6789/0 conn(0x7fb2dc134df0 legacy=0x7fb2dc135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 1 monclient: found mon.noname-c 2026-02-17T05:49:53.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:49:53.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.659+0000 7fb2dbfff640 1 -- 10.20.193.78:0/2256625552 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc09cc80 con 0x7fb2dc06c890 2026-02-17T05:49:53.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2dbfff640 1 -- 10.20.193.78:0/2256625552 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 msgr2=0x7fb2dc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2dbfff640 1 --2- 10.20.193.78:0/2256625552 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 0x7fb2dc06cc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc06d870 tx=0x7fb2c802ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e021be4a0e45ac2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 1 -- 10.20.193.78:0/2256625552 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c803c070 con 0x7fb2dc06c890 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: dump: 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 5 monclient: authenticate success, global_id 44275 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 1 -- 10.20.193.78:0/2256625552 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c802fca0 con 0x7fb2dc06c890 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 1 -- 10.20.193.78:0/2256625552 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8037660 con 0x7fb2dc06c890 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2daffd640 10 monclient: dump: 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:53.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 4 set_mon_vals no callback set 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 10 monclient: shutdown 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 1 -- 10.20.193.78:0/2256625552 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 msgr2=0x7fb2dc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/2256625552 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 0x7fb2dc06cc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc06d870 tx=0x7fb2c802ed40 comp rx=0 tx=0).stop 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 1 -- 10.20.193.78:0/2256625552 shutdown_connections 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/2256625552 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 0x7fb2dc06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/2256625552 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 0x7fb2dc06cc70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 1 -- 10.20.193.78:0/2256625552 >> 10.20.193.78:0/2256625552 conn(0x7fb2dc0a6620 msgr2=0x7fb2dc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 1 -- 10.20.193.78:0/2256625552 shutdown_connections 2026-02-17T05:49:53.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.662+0000 7fb2e29a9640 1 -- 10.20.193.78:0/2256625552 wait complete. 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient: build_initial_monmap 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient: monmap: 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 1 Processor -- start 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 1 -- start start 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient: init 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 0x7fb2dc1cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient(hunting): picked mon.noname-b con 0x7fb2dc06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 0x7fb2dc1cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient(hunting): picked mon.noname-c con 0x7fb2dc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb2dc134df0 0x7fb2dc09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient(hunting): picked mon.noname-a con 0x7fb2dc134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc0a01a0 con 0x7fb2dc134df0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc0a0020 con 0x7fb2dc06c890 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb2dc09be20 con 0x7fb2dc09c4a0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e29a9640 10 monclient(hunting): authenticate will time out at 839.217741s 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2db7fe640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 msgr2=0x7fb2dc1cfbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2db7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 0x7fb2dc1cfbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb2dc134df0 0x7fb2dc09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb2dc134df0 0x7fb2dc09db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52798/0 (socket says 10.20.193.78:52798) 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 1 -- 10.20.193.78:0/1629666888 learned_addr learned my addr 10.20.193.78:0/1629666888 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 10 monclient(hunting): get_auth_request con 0x7fb2dc134df0 auth_method 0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2dbfff640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 0x7fb2dc1cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2dbfff640 10 monclient(hunting): get_auth_request con 0x7fb2dc06c890 auth_method 0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2e0f1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.663+0000 7fb2dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 10 monclient(hunting): handle_auth_done global_id 54258 payload 824 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 1 -- 10.20.193.78:0/1629666888 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 msgr2=0x7fb2dc1cfbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 0x7fb2dc1cfbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 1 -- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 msgr2=0x7fb2dc1cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 0x7fb2dc1cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 10 monclient: _finish_hunting 0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 1 monclient: found mon.noname-a 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.664+0000 7fb2e0f1f640 1 -- 10.20.193.78:0/1629666888 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc1281a0 con 0x7fb2dc134df0 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e0f1f640 1 -- 10.20.193.78:0/1629666888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb2dc134df0 msgr2=0x7fb2dc09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e0f1f640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb2dc134df0 0x7fb2dc09db50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000ca70 tx=0x7fb2d000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 1 -- 10.20.193.78:0/1629666888 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000ae50 con 0x7fb2dc134df0 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: dump: 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.667 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: _finish_auth 0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 1 -- 10.20.193.78:0/1629666888 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d0012400 con 0x7fb2dc134df0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 1 -- 10.20.193.78:0/1629666888 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00127d0 con 0x7fb2dc134df0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 10 monclient: dump: 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 5 monclient: authenticate success, global_id 54258 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2da7fc640 4 set_mon_vals no callback set 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 10 monclient: _renew_subs 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc127200 con 0x7fb2dc134df0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 10 monclient: _renew_subs 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc09f7f0 con 0x7fb2dc134df0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc135a70 con 0x7fb2dc134df0 2026-02-17T05:49:53.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.666+0000 7fb2d8ff9640 1 -- 10.20.193.78:0/1629666888 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb2d00129f0 con 0x7fb2dc134df0 2026-02-17T05:49:53.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.667+0000 7fb2d8ff9640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb2ac03e000 0x7fb2ac0404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:53.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.667+0000 7fb2d8ff9640 1 -- 10.20.193.78:0/1629666888 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7fb2d0052d00 con 0x7fb2dc134df0 2026-02-17T05:49:53.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.667+0000 7fb2dbfff640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb2ac03e000 0x7fb2ac0404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:53.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.667+0000 7fb2dbfff640 10 monclient: get_auth_request con 0x7fb2ac03e000 auth_method 0 2026-02-17T05:49:53.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.669+0000 7fb2d8ff9640 1 -- 10.20.193.78:0/1629666888 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb2dc1281a0 con 0x7fb2dc134df0 2026-02-17T05:49:53.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.669+0000 7fb2d8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:53.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.669+0000 7fb2d8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:53.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.670+0000 7fb2dbfff640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb2ac03e000 0x7fb2ac0404c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0a7490 tx=0x7fb2c8033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.783+0000 7fb2e29a9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.783+0000 7fb2e29a9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.783+0000 7fb2e29a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.783+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb2dc0744d0 con 0x7fb2dc134df0 2026-02-17T05:49:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.783+0000 7fb2d8ff9640 1 -- 10.20.193.78:0/1629666888 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fb2d0007af0 con 0x7fb2dc134df0 2026-02-17T05:49:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.783+0000 7fb2d8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.783+0000 7fb2d8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:53.785 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb2ac03e000 msgr2=0x7fb2ac0404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb2ac03e000 0x7fb2ac0404c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0a7490 tx=0x7fb2c8033000 comp rx=0 tx=0).stop 2026-02-17T05:49:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 10 monclient: shutdown 2026-02-17T05:49:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb2dc134df0 msgr2=0x7fb2dc09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb2dc134df0 0x7fb2dc09db50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000ca70 tx=0x7fb2d000cf40 comp rx=0 tx=0).stop 2026-02-17T05:49:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 shutdown_connections 2026-02-17T05:49:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb2dc134df0 0x7fb2dc09db50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb2dc09c4a0 0x7fb2dc1cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb2ac03e000 0x7fb2ac0404c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 --2- 10.20.193.78:0/1629666888 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb2dc06c890 0x7fb2dc1cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 >> 10.20.193.78:0/1629666888 conn(0x7fb2dc0a6620 msgr2=0x7fb2dc071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 shutdown_connections 2026-02-17T05:49:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:53.784+0000 7fb2e29a9640 1 -- 10.20.193.78:0/1629666888 wait complete. 2026-02-17T05:49:56.793 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.861+0000 7f0120dbe640 10 monclient: get_monmap_and_config 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.861+0000 7f0120dbe640 10 monclient: build_initial_monmap 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.861+0000 7f0120dbe640 10 monclient: monmap: 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:49:56.862541+0000 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:49:56.862541+0000 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:49:56.861 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.861+0000 7f0120dbe640 1 Processor -- start 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.861+0000 7f0120dbe640 1 -- start start 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.861+0000 7f0120dbe640 10 monclient: init 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f011c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f011c09c4a0 0x7f011c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 10 monclient(hunting): picked mon.noname-a con 0x7f011c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f011c06d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f011c06daf0 con 0x7f011c06d380 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f011c09cc80 con 0x7f011c06c890 2026-02-17T05:49:56.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f011c0a0880 con 0x7f011c09c4a0 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0120dbe640 10 monclient(hunting): authenticate will time out at 842.416137s 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f011c09c4a0 0x7f011c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f011c09c4a0 0x7f011c13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52802/0 (socket says 10.20.193.78:52802) 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 1 -- 10.20.193.78:0/890027397 learned_addr learned my addr 10.20.193.78:0/890027397 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient(hunting): get_auth_request con 0x7f011c09c4a0 auth_method 0 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f011ad76640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f011c06d380 0x7f011c13d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:49180/0 (socket says 10.20.193.78:49180) 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient(hunting): handle_auth_done global_id 54261 payload 274 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 1 -- 10.20.193.78:0/890027397 >> v1:10.20.193.115:6789/0 conn(0x7f011c06c890 legacy=0x7f011c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 1 -- 10.20.193.78:0/890027397 >> v1:10.20.193.78:6789/0 conn(0x7f011c06d380 legacy=0x7f011c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient: _finish_hunting 0 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 1 monclient: found mon.noname-a 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.862+0000 7f0119d74640 1 -- 10.20.193.78:0/890027397 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c09cf60 con 0x7f011c09c4a0 2026-02-17T05:49:56.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119d74640 1 -- 10.20.193.78:0/890027397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 msgr2=0x7f011c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:56.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119d74640 1 --2- 10.20.193.78:0/890027397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c13b3a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f011c09c980 tx=0x7f010402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2ba169a236fac0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 1 -- 10.20.193.78:0/890027397 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010403c070 con 0x7f011c09c4a0 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: dump: 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: _finish_auth 0 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: _check_auth_tickets 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 1 -- 10.20.193.78:0/890027397 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010402fd80 con 0x7f011c09c4a0 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 1 -- 10.20.193.78:0/890027397 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104037660 con 0x7f011c09c4a0 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0120dbe640 5 monclient: authenticate success, global_id 54261 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0120dbe640 10 monclient: get_monmap_and_config success 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0120dbe640 4 set_mon_vals no callback set 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.864+0000 7f0119573640 10 monclient: dump: 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:56.865 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 10 monclient: shutdown 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 1 -- 10.20.193.78:0/890027397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 msgr2=0x7f011c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 1 --2- 10.20.193.78:0/890027397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c13b3a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f011c09c980 tx=0x7f010402ee60 comp rx=0 tx=0).stop 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 1 -- 10.20.193.78:0/890027397 shutdown_connections 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 1 --2- 10.20.193.78:0/890027397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c13b3a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 1 -- 10.20.193.78:0/890027397 >> 10.20.193.78:0/890027397 conn(0x7f011c0a6620 msgr2=0x7f011c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 1 -- 10.20.193.78:0/890027397 shutdown_connections 2026-02-17T05:49:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 1 -- 10.20.193.78:0/890027397 wait complete. 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 10 monclient: build_initial_monmap 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 10 monclient: monmap: 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.865+0000 7f0120dbe640 1 Processor -- start 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 1 -- start start 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient: init 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient: _reopen_session rank -1 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f011c06c890 0x7f011c1cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient(hunting): picked mon.noname-c con 0x7f011c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f011c06d380 0x7f011c1cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient(hunting): picked mon.noname-b con 0x7f011c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c1cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient(hunting): picked mon.noname-a con 0x7f011c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f011c09bfa0 con 0x7f011c09c4a0 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f011c09be20 con 0x7f011c06d380 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f011c13f010 con 0x7f011c06c890 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient(hunting): _renew_subs 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0120dbe640 10 monclient(hunting): authenticate will time out at 842.420301s 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011a575640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f011c06c890 msgr2=0x7f011c1cf240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:49:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f011c06c890 0x7f011c1cf240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c1cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c1cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52810/0 (socket says 10.20.193.78:52810) 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 -- 10.20.193.78:0/3353040405 learned_addr learned my addr 10.20.193.78:0/3353040405 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient(hunting): get_auth_request con 0x7f011c09c4a0 auth_method 0 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0119d74640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f011c06d380 0x7f011c1cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient(hunting): handle_auth_done global_id 54264 payload 824 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 -- 10.20.193.78:0/3353040405 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f011c06c890 msgr2=0x7f011c1cf240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f011c06c890 0x7f011c1cf240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 -- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f011c06d380 msgr2=0x7f011c1cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f011c06d380 0x7f011c1cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient: _finish_hunting 0 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 monclient: found mon.noname-a 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f011ad76640 1 -- 10.20.193.78:0/3353040405 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c121ee0 con 0x7f011c09c4a0 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0119d74640 10 monclient: get_auth_request con 0x7f011c06d380 auth_method 0 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.866+0000 7f0119d74640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f011c06d380 0x7f011c1cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:49:56.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f011ad76640 1 -- 10.20.193.78:0/3353040405 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 msgr2=0x7f011c1cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f011ad76640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c1cfcc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f010c00caf0 tx=0x7f010c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 1 -- 10.20.193.78:0/3353040405 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c017070 con 0x7f011c09c4a0 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: dump: 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: _finish_auth 0 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 1 -- 10.20.193.78:0/3353040405 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010c012400 con 0x7f011c09c4a0 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0120dbe640 5 monclient: authenticate success, global_id 54264 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 1 -- 10.20.193.78:0/3353040405 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c0127f0 con 0x7f011c09c4a0 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0102ffd640 10 monclient: dump: 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0118d72640 4 set_mon_vals no callback set 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0120dbe640 10 monclient: _renew_subs 2026-02-17T05:49:56.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0120dbe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c09d810 con 0x7f011c09c4a0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0120dbe640 10 monclient: _renew_subs 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0120dbe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.868+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c121c00 con 0x7f011c09c4a0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f0102ffd640 1 -- 10.20.193.78:0/3353040405 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f010c007070 con 0x7f011c09c4a0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f0120dbe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f0120dbe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f0120dbe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011c121270 con 0x7f011c09c4a0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f0102ffd640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f00e803de60 0x7f00e8040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f0102ffd640 1 -- 10.20.193.78:0/3353040405 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7f010c052940 con 0x7f011c09c4a0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f011a575640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f00e803de60 0x7f00e8040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:49:56.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.869+0000 7f011a575640 10 monclient: get_auth_request con 0x7f00e803de60 auth_method 0 2026-02-17T05:49:56.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.872+0000 7f0102ffd640 1 -- 10.20.193.78:0/3353040405 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f011c121270 con 0x7f011c09c4a0 2026-02-17T05:49:56.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.872+0000 7f0102ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:49:56.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.872+0000 7f0102ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:49:56.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.872+0000 7f011a575640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f00e803de60 0x7f00e8040320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0110003870 tx=0x7f0110007b10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:49:56.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.986+0000 7f0120dbe640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:49:56.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.986+0000 7f0120dbe640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:56.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.986+0000 7f0120dbe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:49:56.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.986+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f011c06cc70 con 0x7f011c09c4a0 2026-02-17T05:49:56.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.986+0000 7f0102ffd640 1 -- 10.20.193.78:0/3353040405 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f010c007620 con 0x7f011c09c4a0 2026-02-17T05:49:56.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.986+0000 7f0102ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:49:56.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.986+0000 7f0102ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:49:56.988 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:49:56.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f00e803de60 msgr2=0x7f00e8040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:56.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f00e803de60 0x7f00e8040320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0110003870 tx=0x7f0110007b10 comp rx=0 tx=0).stop 2026-02-17T05:49:56.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 10 monclient: shutdown 2026-02-17T05:49:56.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:49:56.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 msgr2=0x7f011c1cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:49:56.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c1cfcc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f010c00caf0 tx=0x7f010c00a060 comp rx=0 tx=0).stop 2026-02-17T05:49:56.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 shutdown_connections 2026-02-17T05:49:56.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f011c09c4a0 0x7f011c1cfcc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:56.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f00e803de60 0x7f00e8040320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:56.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f011c06d380 0x7f011c1cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:56.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 --2- 10.20.193.78:0/3353040405 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f011c06c890 0x7f011c1cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:49:56.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 >> 10.20.193.78:0/3353040405 conn(0x7f011c0a6620 msgr2=0x7f011c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:49:56.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 shutdown_connections 2026-02-17T05:49:56.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:49:56.987+0000 7f0120dbe640 1 -- 10.20.193.78:0/3353040405 wait complete. 2026-02-17T05:49:59.996 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.067+0000 7f2deb99e640 10 monclient: get_monmap_and_config 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.067+0000 7f2deb99e640 10 monclient: build_initial_monmap 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.067+0000 7f2deb99e640 10 monclient: monmap: 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:00.068797+0000 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:00.068797+0000 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:00.067 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:00.068 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:00.068 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:00.068 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:00.068 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:00.068 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:00.068 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.067+0000 7f2deb99e640 1 Processor -- start 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 1 -- start start 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient: init 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2de406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2de409c4a0 0x7f2de413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient(hunting): picked mon.noname-c con 0x7f2de409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2de406d380 0x7f2de413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient(hunting): picked mon.noname-a con 0x7f2de406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de406daf0 con 0x7f2de406c890 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f2de40a0880 con 0x7f2de406d380 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2de409cc80 con 0x7f2de409c4a0 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2deb99e640 10 monclient(hunting): authenticate will time out at 845.622367s 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9713640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f2de406c890 0x7f2de406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41580/0 (socket says 10.20.193.78:41580) 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9713640 1 -- 10.20.193.78:0/1149583501 learned_addr learned my addr 10.20.193.78:0/1149583501 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 1 --2- 10.20.193.78:0/1149583501 >> v2:10.20.193.78:3300/0 conn(0x7f2de406d380 0x7f2de413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient(hunting): get_auth_request con 0x7f2de406d380 auth_method 0 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de8f12640 1 --2- 10.20.193.78:0/1149583501 >> v2:10.20.193.115:3300/0 conn(0x7f2de409c4a0 0x7f2de413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de8f12640 10 monclient(hunting): get_auth_request con 0x7f2de409c4a0 auth_method 0 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de8f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de8f12640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de8f12640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de8f12640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de8f12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de8f12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient(hunting): handle_auth_done global_id 54267 payload 274 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 1 -- 10.20.193.78:0/1149583501 >> v2:10.20.193.115:3300/0 conn(0x7f2de409c4a0 msgr2=0x7f2de413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:00.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 1 --2- 10.20.193.78:0/1149583501 >> v2:10.20.193.115:3300/0 conn(0x7f2de409c4a0 0x7f2de413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 1 -- 10.20.193.78:0/1149583501 >> v1:10.20.193.78:6789/0 conn(0x7f2de406c890 legacy=0x7f2de406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:00.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient: _finish_hunting 0 2026-02-17T05:50:00.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 1 monclient: found mon.noname-a 2026-02-17T05:50:00.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:00.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:00.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.068+0000 7f2de9f14640 1 -- 10.20.193.78:0/1149583501 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de409cf60 con 0x7f2de406d380 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2de9f14640 1 -- 10.20.193.78:0/1149583501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 msgr2=0x7f2de413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2de9f14640 1 --2- 10.20.193.78:0/1149583501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de413d8f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2de406dcc0 tx=0x7f2de002f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b3bdadf6063bc7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 1 -- 10.20.193.78:0/1149583501 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de003c070 con 0x7f2de406d380 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: dump: 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:00.071 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: _finish_auth 0 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 1 -- 10.20.193.78:0/1149583501 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de0037440 con 0x7f2de406d380 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 1 -- 10.20.193.78:0/1149583501 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0037810 con 0x7f2de406d380 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 5 monclient: authenticate success, global_id 54267 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 4 set_mon_vals no callback set 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2dd3fff640 10 monclient: dump: 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 10 monclient: shutdown 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 1 -- 10.20.193.78:0/1149583501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 msgr2=0x7f2de413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 1 --2- 10.20.193.78:0/1149583501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de413d8f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2de406dcc0 tx=0x7f2de002f150 comp rx=0 tx=0).stop 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 1 -- 10.20.193.78:0/1149583501 shutdown_connections 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 1 --2- 10.20.193.78:0/1149583501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de413d8f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 1 --2- 10.20.193.78:0/1149583501 >> v2:10.20.193.115:3300/0 conn(0x7f2de409c4a0 0x7f2de413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 1 -- 10.20.193.78:0/1149583501 >> 10.20.193.78:0/1149583501 conn(0x7f2de40a6620 msgr2=0x7f2de40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 1 -- 10.20.193.78:0/1149583501 shutdown_connections 2026-02-17T05:50:00.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.071+0000 7f2deb99e640 1 -- 10.20.193.78:0/1149583501 wait complete. 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient: build_initial_monmap 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient: monmap: 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:00.073 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 1 Processor -- start 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 1 -- start start 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient: init 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2de406c890 0x7f2de413c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient(hunting): picked mon.noname-c con 0x7f2de406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient(hunting): picked mon.noname-a con 0x7f2de406d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2de409c4a0 0x7f2de409e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient(hunting): picked mon.noname-b con 0x7f2de409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2de409bfa0 con 0x7f2de406d380 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2de409be20 con 0x7f2de409c4a0 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2de413de30 con 0x7f2de406c890 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2deb99e640 10 monclient(hunting): authenticate will time out at 845.626717s 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de9713640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2de406c890 msgr2=0x7f2de413c720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de9713640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2de406c890 0x7f2de413c720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de409db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:46804/0 (socket says 10.20.193.78:46804) 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 1 -- 10.20.193.78:0/2425727463 learned_addr learned my addr 10.20.193.78:0/2425727463 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 10 monclient(hunting): get_auth_request con 0x7f2de406d380 auth_method 0 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de9f14640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2de409c4a0 0x7f2de409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de9f14640 10 monclient(hunting): get_auth_request con 0x7f2de409c4a0 auth_method 0 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de9f14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de9f14640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de9f14640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:00.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 10 monclient(hunting): handle_auth_done global_id 54270 payload 824 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 1 -- 10.20.193.78:0/2425727463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2de406c890 msgr2=0x7f2de413c720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2de406c890 0x7f2de413c720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 1 -- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2de409c4a0 msgr2=0x7f2de409e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.072+0000 7f2de8f12640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2de409c4a0 0x7f2de409e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.073+0000 7f2de8f12640 10 monclient: _finish_hunting 0 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.073+0000 7f2de8f12640 1 monclient: found mon.noname-a 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.073+0000 7f2de8f12640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.073+0000 7f2de8f12640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.073+0000 7f2de8f12640 1 -- 10.20.193.78:0/2425727463 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de413b7f0 con 0x7f2de406d380 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.073+0000 7f2de9f14640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2de409c4a0 0x7f2de409e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2de8f12640 1 -- 10.20.193.78:0/2425727463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 msgr2=0x7f2de409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2de8f12640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de409db50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800ec10 tx=0x7f2dd800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 1 -- 10.20.193.78:0/2425727463 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8017070 con 0x7f2de406d380 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: dump: 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:00.075 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 1 -- 10.20.193.78:0/2425727463 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd800ce50 con 0x7f2de406d380 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 1 -- 10.20.193.78:0/2425727463 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80136b0 con 0x7f2de406d380 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd37fe640 4 set_mon_vals no callback set 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2dd1ffb640 10 monclient: dump: 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 5 monclient: authenticate success, global_id 54270 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 10 monclient: _renew_subs 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de413bad0 con 0x7f2de406d380 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 10 monclient: _renew_subs 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:00.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de413cc60 con 0x7f2de406d380 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.075+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de4074980 con 0x7f2de406d380 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.076+0000 7f2dd1ffb640 1 -- 10.20.193.78:0/2425727463 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2dd8024060 con 0x7f2de406d380 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.076+0000 7f2dd1ffb640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2db403de60 0x7f2db4040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.076+0000 7f2dd1ffb640 1 -- 10.20.193.78:0/2425727463 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7f2dd8052900 con 0x7f2de406d380 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.076+0000 7f2de9713640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2db403de60 0x7f2db4040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:00.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.076+0000 7f2de9713640 10 monclient: get_auth_request con 0x7f2db403de60 auth_method 0 2026-02-17T05:50:00.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.078+0000 7f2dd1ffb640 1 -- 10.20.193.78:0/2425727463 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2dd8013900 con 0x7f2de406d380 2026-02-17T05:50:00.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.078+0000 7f2dd1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:00.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.078+0000 7f2dd1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:00.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.079+0000 7f2de9713640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2db403de60 0x7f2db4040320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4004910 tx=0x7f2dd4007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:00.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.194+0000 7f2deb99e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:00.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.194+0000 7f2deb99e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:00.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.194+0000 7f2deb99e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:00.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.194+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2de413db80 con 0x7f2de406d380 2026-02-17T05:50:00.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.195+0000 7f2dd1ffb640 1 -- 10.20.193.78:0/2425727463 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f2dd8013b80 con 0x7f2de406d380 2026-02-17T05:50:00.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.195+0000 7f2dd1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:00.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.195+0000 7f2dd1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:00.196 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:00.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2db403de60 msgr2=0x7f2db4040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:00.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2db403de60 0x7f2db4040320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4004910 tx=0x7f2dd4007c20 comp rx=0 tx=0).stop 2026-02-17T05:50:00.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 10 monclient: shutdown 2026-02-17T05:50:00.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:00.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 msgr2=0x7f2de409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:00.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de409db50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800ec10 tx=0x7f2dd800c520 comp rx=0 tx=0).stop 2026-02-17T05:50:00.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 shutdown_connections 2026-02-17T05:50:00.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2db403de60 0x7f2db4040320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2de409c4a0 0x7f2de409e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2de406d380 0x7f2de409db50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 --2- 10.20.193.78:0/2425727463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2de406c890 0x7f2de413c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:00.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 >> 10.20.193.78:0/2425727463 conn(0x7f2de40a6620 msgr2=0x7f2de40a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:00.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 shutdown_connections 2026-02-17T05:50:00.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:00.196+0000 7f2deb99e640 1 -- 10.20.193.78:0/2425727463 wait complete. 2026-02-17T05:50:03.206 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:03.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.275+0000 7f606f23f640 10 monclient: get_monmap_and_config 2026-02-17T05:50:03.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.275+0000 7f606f23f640 10 monclient: build_initial_monmap 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.275+0000 7f606f23f640 10 monclient: monmap: 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:03.275545+0000 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:03.275545+0000 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:03.275 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:03.276 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:03.276 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:03.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.275+0000 7f606f23f640 1 Processor -- start 2026-02-17T05:50:03.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.275+0000 7f606f23f640 1 -- start start 2026-02-17T05:50:03.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.275+0000 7f606f23f640 10 monclient: init 2026-02-17T05:50:03.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:03.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2026-02-17T05:50:03.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 10 monclient(hunting): picked mon.noname-a con 0x7f60680a3220 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f60680704c0 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f60680a3b40 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6068074000 con 0x7f60680a3b40 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60680a4af0 con 0x7f60680704c0 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6068070130 con 0x7f60680a3220 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606f23f640 10 monclient(hunting): authenticate will time out at 848.829389s 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:46812/0 (socket says 10.20.193.78:46812) 2026-02-17T05:50:03.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 learned_addr learned my addr 10.20.193.78:0/2174655786 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606d7b5640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f60680a3b40 0x7f606813fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41594/0 (socket says 10.20.193.78:41594) 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 10 monclient(hunting): get_auth_request con 0x7f60680a3220 auth_method 0 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.276+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.278+0000 7f606d7b5640 1 -- 10.20.193.78:0/2174655786 >> v1:10.20.193.78:6789/0 conn(0x7f60680a3b40 legacy=0x7f606813fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:03.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.278+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> v1:10.20.193.115:6789/0 conn(0x7f60680704c0 legacy=0x7f606813c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:03.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:03.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 10 monclient(hunting): get_auth_request con 0x7f60680a3220 auth_method 0 2026-02-17T05:50:03.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:03.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:03.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:03.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:03.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:03.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:03.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.476+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 10 monclient(hunting): get_auth_request con 0x7f60680a3220 auth_method 0 2026-02-17T05:50:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:03.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:03.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:03.877+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:04.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:04.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 10 monclient(hunting): get_auth_request con 0x7f60680a3220 auth_method 0 2026-02-17T05:50:04.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:04.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:04.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:04.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:04.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:04.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:04.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:04.678+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:50:06.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient(hunting): tick 2026-02-17T05:50:06.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 monclient(hunting): continuing hunt 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 msgr2=0x7f60680a3600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.78:3300/0 conn(0x7f60680a3220 0x7f60680a3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 -- 10.20.193.78:0/2174655786 >> v1:10.20.193.115:6789/0 conn(0x7f60680704c0 legacy=0x7f606813c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 -- 10.20.193.78:0/2174655786 >> v1:10.20.193.78:6789/0 conn(0x7f60680a3b40 legacy=0x7f606813fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient: _add_conns ranks=[2,4] 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 0x7f604c005d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient(hunting): picked mon.noname-b con 0x7f604c003910 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient(hunting): picked mon.noname-c con 0x7f604c006450 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 -- 10.20.193.78:0/2174655786 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6068070130 con 0x7f604c006450 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 1 -- 10.20.193.78:0/2174655786 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f604c008fd0 con 0x7f604c003910 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605e7fc640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 msgr2=0x7f604c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 0x7f604c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 10 monclient(hunting): get_auth_request con 0x7f604c006450 auth_method 0 2026-02-17T05:50:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 msgr2=0x7f604c008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:50:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 msgr2=0x7f604c008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.276+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:06.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.476+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 msgr2=0x7f604c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:50:06.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.476+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 0x7f604c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 10 monclient(hunting): get_auth_request con 0x7f604c006450 auth_method 0 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 msgr2=0x7f604c008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 msgr2=0x7f604c008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:06.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.477+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:06.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.877+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 msgr2=0x7f604c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:50:06.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.877+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 0x7f604c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:06.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:06.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 10 monclient(hunting): get_auth_request con 0x7f604c006450 auth_method 0 2026-02-17T05:50:06.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:06.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:06.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:06.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 msgr2=0x7f604c008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:06.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 msgr2=0x7f604c008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:06.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:06.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:06.878+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.678+0000 7f606cfb4640 1 -- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 msgr2=0x7f604c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:50:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.678+0000 7f606cfb4640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 0x7f604c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:50:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> v2:10.20.193.115:3300/0 conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient(hunting): get_auth_request con 0x7f604c006450 auth_method 0 2026-02-17T05:50:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient(hunting): handle_auth_done global_id 44287 payload 274 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 msgr2=0x7f604c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003910 0x7f604c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 1 monclient: found mon.noname-c 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:50:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.679+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606806f800 con 0x7f604c006450 2026-02-17T05:50:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605ffff640 1 -- 10.20.193.78:0/2174655786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c006450 msgr2=0x7f604c008910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605ffff640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c006450 0x7f604c008910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6050007bf0 tx=0x7f605002f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a4a0eb61f1cb02 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 1 -- 10.20.193.78:0/2174655786 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605003b070 con 0x7f604c006450 2026-02-17T05:50:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: dump: 2026-02-17T05:50:07.688 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:50:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 1 -- 10.20.193.78:0/2174655786 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6050005b90 con 0x7f604c006450 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 1 -- 10.20.193.78:0/2174655786 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500070d0 con 0x7f604c006450 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 5 monclient: authenticate success, global_id 44287 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 4 set_mon_vals no callback set 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f605f7fe640 10 monclient: dump: 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.690 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 10 monclient: shutdown 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 1 -- 10.20.193.78:0/2174655786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c006450 msgr2=0x7f604c008910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c006450 0x7f604c008910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6050007bf0 tx=0x7f605002f710 comp rx=0 tx=0).stop 2026-02-17T05:50:07.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606d7b5640 1 -- 10.20.193.78:0/2174655786 reap_dead start 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 1 -- 10.20.193.78:0/2174655786 shutdown_connections 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 1 --2- 10.20.193.78:0/2174655786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c006450 0x7f604c008910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 1 -- 10.20.193.78:0/2174655786 >> 10.20.193.78:0/2174655786 conn(0x7f606809c000 msgr2=0x7f60680a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 1 -- 10.20.193.78:0/2174655786 shutdown_connections 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.682+0000 7f606f23f640 1 -- 10.20.193.78:0/2174655786 wait complete. 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient: build_initial_monmap 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient: monmap: 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:07.692 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 1 Processor -- start 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 1 -- start start 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient: init 2026-02-17T05:50:07.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c006450 0x7f60681cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient(hunting): picked mon.noname-a con 0x7f604c006450 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6068140810 0x7f6068140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient(hunting): picked mon.noname-b con 0x7f6068140810 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6068126240 0x7f60681266c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient(hunting): picked mon.noname-c con 0x7f6068126240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60680a9680 con 0x7f604c006450 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60680a9500 con 0x7f6068140810 2026-02-17T05:50:07.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f60680a4f70 con 0x7f6068126240 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.683+0000 7f606f23f640 10 monclient(hunting): authenticate will time out at 853.236989s 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c006450 0x7f60681cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606d7b5640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6068126240 msgr2=0x7f60681266c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606d7b5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6068126240 0x7f60681266c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c006450 0x7f60681cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35594/0 (socket says 10.20.193.78:35594) 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 -- 10.20.193.78:0/3651235035 learned_addr learned my addr 10.20.193.78:0/3651235035 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient(hunting): get_auth_request con 0x7f604c006450 auth_method 0 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:07.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f605ffff640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6068140810 0x7f6068140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f605ffff640 10 monclient(hunting): get_auth_request con 0x7f6068140810 auth_method 0 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f605ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f605ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f605ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient(hunting): handle_auth_done global_id 54273 payload 824 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 -- 10.20.193.78:0/3651235035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6068126240 msgr2=0x7f60681266c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:07.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6068126240 0x7f60681266c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 -- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6068140810 msgr2=0x7f6068140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6068140810 0x7f6068140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient: _finish_hunting 0 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 monclient: found mon.noname-a 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.684+0000 7f606cfb4640 1 -- 10.20.193.78:0/3651235035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6068127ea0 con 0x7f604c006450 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606cfb4640 1 -- 10.20.193.78:0/3651235035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c006450 msgr2=0x7f60681cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606cfb4640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c006450 0x7f60681cf4f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6058013590 tx=0x7f6058013a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 1 -- 10.20.193.78:0/3651235035 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605801c070 con 0x7f604c006450 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:07.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: dump: 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:50:07.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606f23f640 5 monclient: authenticate success, global_id 54273 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 1 -- 10.20.193.78:0/3651235035 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60580158a0 con 0x7f604c006450 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606f23f640 10 monclient: _renew_subs 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606f23f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606809fa10 con 0x7f604c006450 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 1 -- 10.20.193.78:0/3651235035 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058015c90 con 0x7f604c006450 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605d7fa640 10 monclient: dump: 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.699 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f605effd640 4 set_mon_vals no callback set 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606f23f640 10 monclient: _renew_subs 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606f23f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.686+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606812a190 con 0x7f604c006450 2026-02-17T05:50:07.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f606f23f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:07.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f606f23f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:07.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f606f23f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60680a2ce0 con 0x7f604c006450 2026-02-17T05:50:07.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f605d7fa640 1 -- 10.20.193.78:0/3651235035 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6058016460 con 0x7f604c006450 2026-02-17T05:50:07.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f605d7fa640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603803deb0 0x7f6038040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f605d7fa640 1 -- 10.20.193.78:0/3651235035 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7f6058052fa0 con 0x7f604c006450 2026-02-17T05:50:07.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f605ffff640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603803deb0 0x7f6038040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.687+0000 7f605ffff640 10 monclient: get_auth_request con 0x7f603803deb0 auth_method 0 2026-02-17T05:50:07.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.689+0000 7f605d7fa640 1 -- 10.20.193.78:0/3651235035 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f60680a2ce0 con 0x7f604c006450 2026-02-17T05:50:07.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.689+0000 7f605d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:07.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.689+0000 7f605d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:07.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.690+0000 7f605ffff640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603803deb0 0x7f6038040370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f604c006420 tx=0x7f6050007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.803+0000 7f606f23f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.803+0000 7f606f23f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.803+0000 7f606f23f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.803+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f60680a74c0 con 0x7f604c006450 2026-02-17T05:50:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f605d7fa640 1 -- 10.20.193.78:0/3651235035 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f60580203b0 con 0x7f604c006450 2026-02-17T05:50:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f605d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f605d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:07.804 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603803deb0 msgr2=0x7f6038040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603803deb0 0x7f6038040370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f604c006420 tx=0x7f6050007000 comp rx=0 tx=0).stop 2026-02-17T05:50:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 10 monclient: shutdown 2026-02-17T05:50:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c006450 msgr2=0x7f60681cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c006450 0x7f60681cf4f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6058013590 tx=0x7f6058013a60 comp rx=0 tx=0).stop 2026-02-17T05:50:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 shutdown_connections 2026-02-17T05:50:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6068126240 0x7f60681266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603803deb0 0x7f6038040370 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6068140810 0x7f6068140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 --2- 10.20.193.78:0/3651235035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c006450 0x7f60681cf4f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 >> 10.20.193.78:0/3651235035 conn(0x7f606809c000 msgr2=0x7f6068071930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 shutdown_connections 2026-02-17T05:50:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.804+0000 7f606f23f640 1 -- 10.20.193.78:0/3651235035 wait complete. 2026-02-17T05:50:07.812 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:50:07.812 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T05:50:07.812 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:50:07.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.882+0000 7ff7e250a640 10 monclient: get_monmap_and_config 2026-02-17T05:50:07.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.882+0000 7ff7e250a640 10 monclient: build_initial_monmap 2026-02-17T05:50:07.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.882+0000 7ff7e250a640 10 monclient: monmap: 2026-02-17T05:50:07.884 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:07.884 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:07.883132+0000 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:07.883132+0000 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:07.885 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 1 Processor -- start 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 1 -- start start 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient: init 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff7dc06c890 0x7ff7dc0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient(hunting): picked mon.noname-a con 0x7ff7dc06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff7dc06cc70 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff7dc13a240 0x7ff7dc13c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient(hunting): picked mon.noname-c con 0x7ff7dc13a240 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7dc074150 con 0x7ff7dc06cc70 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff7dc09ef70 con 0x7ff7dc06c890 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff7dc09f250 con 0x7ff7dc13a240 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e250a640 10 monclient(hunting): authenticate will time out at 853.436700s 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff7dc06c890 0x7ff7dc0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff7dc06c890 0x7ff7dc0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35606/0 (socket says 10.20.193.78:35606) 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4058592245 learned_addr learned my addr 10.20.193.78:0/4058592245 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 10 monclient(hunting): get_auth_request con 0x7ff7dc06c890 auth_method 0 2026-02-17T05:50:07.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e0a80640 1 --2- 10.20.193.78:0/4058592245 >> v2:10.20.193.115:3300/0 conn(0x7ff7dc13a240 0x7ff7dc13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e0a80640 10 monclient(hunting): get_auth_request con 0x7ff7dc13a240 auth_method 0 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e0a80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e0a80640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.883+0000 7ff7e0a80640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 10 monclient(hunting): handle_auth_done global_id 54276 payload 274 2026-02-17T05:50:07.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4058592245 >> v2:10.20.193.115:3300/0 conn(0x7ff7dc13a240 msgr2=0x7ff7dc13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:07.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 1 --2- 10.20.193.78:0/4058592245 >> v2:10.20.193.115:3300/0 conn(0x7ff7dc13a240 0x7ff7dc13c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4058592245 >> v1:10.20.193.115:6789/0 conn(0x7ff7dc06cc70 legacy=0x7ff7dc06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:07.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:07.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 1 monclient: found mon.noname-a 2026-02-17T05:50:07.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:07.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.884+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4058592245 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc09df40 con 0x7ff7dc06c890 2026-02-17T05:50:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4058592245 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 msgr2=0x7ff7dc0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7dbfff640 1 --2- 10.20.193.78:0/4058592245 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc0a01f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc069d40 tx=0x7ff7c802eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30388942de95ab21 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 1 -- 10.20.193.78:0/4058592245 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c803c070 con 0x7ff7dc06c890 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: dump: 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.902 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: _finish_auth 0 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 1 -- 10.20.193.78:0/4058592245 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c802fd50 con 0x7ff7dc06c890 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7e250a640 5 monclient: authenticate success, global_id 54276 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7e250a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:07.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 1 -- 10.20.193.78:0/4058592245 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8037660 con 0x7ff7dc06c890 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7e250a640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7e250a640 4 set_mon_vals no callback set 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.886+0000 7ff7daffd640 10 monclient: dump: 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.904 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient: shutdown 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- 10.20.193.78:0/4058592245 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 msgr2=0x7ff7dc0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 --2- 10.20.193.78:0/4058592245 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc0a01f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc069d40 tx=0x7ff7c802eed0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- 10.20.193.78:0/4058592245 shutdown_connections 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 --2- 10.20.193.78:0/4058592245 >> v2:10.20.193.115:3300/0 conn(0x7ff7dc13a240 0x7ff7dc13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 --2- 10.20.193.78:0/4058592245 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc0a01f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- 10.20.193.78:0/4058592245 >> 10.20.193.78:0/4058592245 conn(0x7ff7dc0a6620 msgr2=0x7ff7dc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:07.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- 10.20.193.78:0/4058592245 shutdown_connections 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- 10.20.193.78:0/4058592245 wait complete. 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient: build_initial_monmap 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient: monmap: 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:07.906 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 Processor -- start 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- start start 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient: init 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient(hunting): picked mon.noname-a con 0x7ff7dc06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff7dc06cc70 0x7ff7dc073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient(hunting): picked mon.noname-b con 0x7ff7dc06cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff7dc13a240 0x7ff7dc133110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient(hunting): picked mon.noname-c con 0x7ff7dc13a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc13ccf0 con 0x7ff7dc06c890 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc13cb70 con 0x7ff7dc06cc70 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff7dc13ce70 con 0x7ff7dc13a240 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e250a640 10 monclient(hunting): authenticate will time out at 853.440914s 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e0a80640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff7dc13a240 msgr2=0x7ff7dc133110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:50:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7e0a80640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff7dc13a240 0x7ff7dc133110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7dbfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7dbfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc072f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35616/0 (socket says 10.20.193.78:35616) 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4073015534 learned_addr learned my addr 10.20.193.78:0/4073015534 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7dbfff640 10 monclient(hunting): get_auth_request con 0x7ff7dc06c890 auth_method 0 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.887+0000 7ff7dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 10 monclient(hunting): handle_auth_done global_id 54279 payload 824 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff7dc13a240 msgr2=0x7ff7dc133110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff7dc13a240 0x7ff7dc133110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff7dc06cc70 msgr2=0x7ff7dc073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff7dc06cc70 0x7ff7dc073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 1 monclient: found mon.noname-a 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.888+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4073015534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc09d810 con 0x7ff7dc06c890 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7dbfff640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 msgr2=0x7ff7dc072f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7dbfff640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc072f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8002410 tx=0x7ff7c8004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c803c040 con 0x7ff7dc06c890 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: dump: 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.911 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: _finish_auth 0 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c8047070 con 0x7ff7dc06c890 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7e250a640 5 monclient: authenticate success, global_id 54279 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7e250a640 10 monclient: _renew_subs 2026-02-17T05:50:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7e250a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7dc134cb0 con 0x7ff7dc06c890 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8041800 con 0x7ff7dc06c890 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7da7fc640 4 set_mon_vals no callback set 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7d8ff9640 10 monclient: dump: 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.913 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7e250a640 10 monclient: _renew_subs 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7e250a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.890+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7dc1cd290 con 0x7ff7dc06c890 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7e250a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7e250a640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc133650 0x7ff7dc133a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7e250a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc13c940 con 0x7ff7dc133650 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff7c80419e0 con 0x7ff7dc06c890 2026-02-17T05:50:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc133650 0x7ff7dc133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 10 monclient: get_auth_request con 0x7ff7dc133650 auth_method 0 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7d8ff9640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff7ac03dee0 0x7ff7ac0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7ff7c8077520 con 0x7ff7dc06c890 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 10 monclient(hunting): handle_auth_done global_id 54282 payload 824 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7db7fe640 1 -- 10.20.193.78:0/4073015534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7dc133f70 con 0x7ff7dc133650 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7e0a80640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff7ac03dee0 0x7ff7ac0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.891+0000 7ff7e0a80640 10 monclient: get_auth_request con 0x7ff7ac03dee0 auth_method 0 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7db7fe640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc133650 msgr2=0x7ff7dc133a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7db7fe640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc133650 0x7ff7dc133a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc134930 tx=0x7ff7cc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc011a80 con 0x7ff7dc133650 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 10 monclient: dump: 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.916 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff7cc017070 con 0x7ff7dc133650 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc133650 msgr2=0x7ff7dc133a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc133650 0x7ff7dc133a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc134930 tx=0x7ff7cc00c800 comp rx=0 tx=0).stop 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc133650 msgr2=0x7ff7dc133a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7d8ff9640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc133650 0x7ff7dc133a30 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc134930 tx=0x7ff7cc00c800 comp rx=0 tx=0).stop 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.893+0000 7ff7e0a80640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff7ac03dee0 0x7ff7ac0403a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc135fe0 tx=0x7ff7d0009950 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7e250a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7e250a640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc137230 0x7ff7dc1d81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7e250a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc06f790 con 0x7ff7dc137230 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc137230 0x7ff7dc1d81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 10 monclient: get_auth_request con 0x7ff7dc137230 auth_method 0 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 10 monclient(hunting): handle_auth_done global_id 54285 payload 824 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.897+0000 7ff7db7fe640 1 -- 10.20.193.78:0/4073015534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff7dc0a1340 con 0x7ff7dc137230 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7db7fe640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc137230 msgr2=0x7ff7dc1d81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7db7fe640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc137230 0x7ff7dc1d81d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc134930 tx=0x7ff7cc003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc01cae0 con 0x7ff7dc137230 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:07.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 10 monclient: dump: 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7ff7cc01cae0 con 0x7ff7dc137230 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:07.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc137230 msgr2=0x7ff7dc1d81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc137230 0x7ff7dc1d81d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc134930 tx=0x7ff7cc003860 comp rx=0 tx=0).stop 2026-02-17T05:50:07.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc137230 msgr2=0x7ff7dc1d81d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:07.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.899+0000 7ff7d8ff9640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc137230 0x7ff7dc1d81d0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc134930 tx=0x7ff7cc003860 comp rx=0 tx=0).stop 2026-02-17T05:50:07.920 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:50:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 44, 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 77373, 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:07.925 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:07.926 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:07.927 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:07.928 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:07.929 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:07.930 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:07.930 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:07.930 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:07.930 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:07.930 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:07.930 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:07.930 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:07.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff7ac03dee0 msgr2=0x7ff7ac0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff7ac03dee0 0x7ff7ac0403a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc135fe0 tx=0x7ff7d0009950 comp rx=0 tx=0).stop 2026-02-17T05:50:07.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 10 monclient: shutdown 2026-02-17T05:50:07.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:07.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 msgr2=0x7ff7dc072f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:07.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e0a80640 1 -- 10.20.193.78:0/4073015534 reap_dead start 2026-02-17T05:50:07.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc072f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8002410 tx=0x7ff7c8004830 comp rx=0 tx=0).stop 2026-02-17T05:50:07.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 shutdown_connections 2026-02-17T05:50:07.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 --2- 10.20.193.78:0/4073015534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff7dc06c890 0x7ff7dc072f50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:07.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 >> 10.20.193.78:0/4073015534 conn(0x7ff7dc0a6620 msgr2=0x7ff7dc09be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:07.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 shutdown_connections 2026-02-17T05:50:07.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:07.900+0000 7ff7e250a640 1 -- 10.20.193.78:0/4073015534 wait complete. 2026-02-17T05:50:07.932 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:50:07.932 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:50:07.932 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:50:08.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient: get_monmap_and_config 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient: build_initial_monmap 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient: monmap: 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:08.002367+0000 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:08.002367+0000 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:08.004 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 1 Processor -- start 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 1 -- start start 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient: init 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f774406c890 0x7f774406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient(hunting): picked mon.noname-a con 0x7f774406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f774409d430 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f774409bf40 0x7f774406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient(hunting): picked mon.noname-c con 0x7f774409bf40 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7744073600 con 0x7f774409d430 2026-02-17T05:50:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f774409ca50 con 0x7f774406c890 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f774406d6f0 con 0x7f774409bf40 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774c2f2640 10 monclient(hunting): authenticate will time out at 853.555916s 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.002+0000 7f774a067640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f774406c890 0x7f774406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f774406c890 0x7f774406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35648/0 (socket says 10.20.193.78:35648) 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 1 -- 10.20.193.78:0/3383300764 learned_addr learned my addr 10.20.193.78:0/3383300764 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient(hunting): get_auth_request con 0x7f774406c890 auth_method 0 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a868640 1 --2- 10.20.193.78:0/3383300764 >> v2:10.20.193.115:3300/0 conn(0x7f774409bf40 0x7f774406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a868640 10 monclient(hunting): get_auth_request con 0x7f774409bf40 auth_method 0 2026-02-17T05:50:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a868640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a868640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient(hunting): handle_auth_done global_id 54288 payload 274 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 1 -- 10.20.193.78:0/3383300764 >> v2:10.20.193.115:3300/0 conn(0x7f774409bf40 msgr2=0x7f774406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 1 --2- 10.20.193.78:0/3383300764 >> v2:10.20.193.115:3300/0 conn(0x7f774409bf40 0x7f774406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 1 -- 10.20.193.78:0/3383300764 >> v1:10.20.193.115:6789/0 conn(0x7f774409d430 legacy=0x7f7744134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient: _finish_hunting 0 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 1 monclient: found mon.noname-a 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a067640 1 -- 10.20.193.78:0/3383300764 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774406dab0 con 0x7f774406c890 2026-02-17T05:50:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.003+0000 7f774a868640 1 --2- 10.20.193.78:0/3383300764 >> v2:10.20.193.115:3300/0 conn(0x7f774409bf40 0x7f774406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:50:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f774a067640 1 -- 10.20.193.78:0/3383300764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774406c890 msgr2=0x7f774406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f774a067640 1 --2- 10.20.193.78:0/3383300764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774406c890 0x7f774406cc70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7744138100 tx=0x7f773402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fb8cc153bf75b3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 1 -- 10.20.193.78:0/3383300764 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773403c070 con 0x7f774406c890 2026-02-17T05:50:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: dump: 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: _finish_auth 0 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: _check_auth_tickets 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 1 -- 10.20.193.78:0/3383300764 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773402fca0 con 0x7f774406c890 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 1 -- 10.20.193.78:0/3383300764 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734037660 con 0x7f774406c890 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f774c2f2640 5 monclient: authenticate success, global_id 54288 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f7749065640 10 monclient: dump: 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f774c2f2640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.005+0000 7f774c2f2640 4 set_mon_vals no callback set 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 10 monclient: shutdown 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 -- 10.20.193.78:0/3383300764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774406c890 msgr2=0x7f774406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 --2- 10.20.193.78:0/3383300764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774406c890 0x7f774406cc70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7744138100 tx=0x7f773402ee60 comp rx=0 tx=0).stop 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 -- 10.20.193.78:0/3383300764 shutdown_connections 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 --2- 10.20.193.78:0/3383300764 >> v2:10.20.193.115:3300/0 conn(0x7f774409bf40 0x7f774406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 --2- 10.20.193.78:0/3383300764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774406c890 0x7f774406cc70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 -- 10.20.193.78:0/3383300764 >> 10.20.193.78:0/3383300764 conn(0x7f77440a6620 msgr2=0x7f77440a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 -- 10.20.193.78:0/3383300764 shutdown_connections 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 -- 10.20.193.78:0/3383300764 wait complete. 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 10 monclient: build_initial_monmap 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 10 monclient: monmap: 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:08.015 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 Processor -- start 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 1 -- start start 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.006+0000 7f774c2f2640 10 monclient: init 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774406c890 0x7f77441cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient(hunting): picked mon.noname-c con 0x7f774406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f774409bf40 0x7f77441cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient(hunting): picked mon.noname-b con 0x7f774409bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:08.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774409d430 0x7f77441cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient(hunting): picked mon.noname-a con 0x7f774409d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f774409c800 con 0x7f774409d430 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7744073410 con 0x7f774409bf40 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7744135500 con 0x7f774406c890 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774c2f2640 10 monclient(hunting): authenticate will time out at 853.560145s 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a067640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774406c890 msgr2=0x7f77441cf2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a067640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774406c890 0x7f77441cf2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774409d430 0x7f77441cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774409d430 0x7f77441cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35664/0 (socket says 10.20.193.78:35664) 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 -- 10.20.193.78:0/1139975042 learned_addr learned my addr 10.20.193.78:0/1139975042 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient(hunting): get_auth_request con 0x7f774409d430 auth_method 0 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:08.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f7749866640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f774409bf40 0x7f77441cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f7749866640 10 monclient(hunting): get_auth_request con 0x7f774409bf40 auth_method 0 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f7749866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f7749866640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f7749866640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient(hunting): handle_auth_done global_id 54291 payload 824 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774406c890 msgr2=0x7f77441cf2f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f774406c890 0x7f77441cf2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f774409bf40 msgr2=0x7f77441cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f774409bf40 0x7f77441cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient: _finish_hunting 0 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 monclient: found mon.noname-a 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:08.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.007+0000 7f774a868640 1 -- 10.20.193.78:0/1139975042 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7744123160 con 0x7f774409d430 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.009+0000 7f774a868640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774409d430 msgr2=0x7f77441cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.009+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.009+0000 7f774a868640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774409d430 0x7f77441cfd70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7740007e60 tx=0x7f774000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774001a070 con 0x7f774409d430 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: dump: 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:08.019 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:08.020 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: _finish_auth 0 2026-02-17T05:50:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7740003cb0 con 0x7f774409d430 2026-02-17T05:50:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740003e60 con 0x7f774409d430 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f772affd640 10 monclient: dump: 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:08.021 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f7748864640 4 set_mon_vals no callback set 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f774c2f2640 5 monclient: authenticate success, global_id 54291 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f774c2f2640 10 monclient: _renew_subs 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f774c2f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7744125450 con 0x7f774409d430 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f774c2f2640 10 monclient: _renew_subs 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f774c2f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7744125170 con 0x7f774409d430 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.011+0000 7f774c2f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:08.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774c2f2640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744122680 0x7f7744063880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77440a1a80 con 0x7f7744122680 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744122680 0x7f7744063880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 10 monclient: get_auth_request con 0x7f7744122680 auth_method 0 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f77400127e0 con 0x7f774409d430 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 10 monclient(hunting): handle_auth_done global_id 44302 payload 824 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f774a067640 1 -- 10.20.193.78:0/1139975042 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f774409d810 con 0x7f7744122680 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f772affd640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771403deb0 0x7f7714040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:08.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(152..152 src has 117..152) ==== 4735+0+0 (secure 0 0 0) 0x7f7740052e10 con 0x7f774409d430 2026-02-17T05:50:08.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f7749866640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771403deb0 0x7f7714040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:08.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.012+0000 7f7749866640 10 monclient: get_auth_request con 0x7f771403deb0 auth_method 0 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.014+0000 7f774a067640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744122680 msgr2=0x7f7744063880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.014+0000 7f774a067640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744122680 0x7f7744063880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f77440a0ac0 tx=0x7f7734004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.015+0000 7f7749866640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771403deb0 0x7f7714040370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f77441246e0 tx=0x7f7738009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.016+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734048070 con 0x7f7744122680 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.016+0000 7f772affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.016+0000 7f772affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.016+0000 7f772affd640 10 monclient: dump: 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:08.026 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.017+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f773403d040 con 0x7f7744122680 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.017+0000 7f772affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.017+0000 7f772affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.017+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744122680 msgr2=0x7f7744063880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.017+0000 7f772affd640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744122680 0x7f7744063880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f77440a0ac0 tx=0x7f7734004830 comp rx=0 tx=0).stop 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.017+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744122680 msgr2=0x7f7744063880 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.017+0000 7f772affd640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744122680 0x7f7744063880 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f77440a0ac0 tx=0x7f7734004830 comp rx=0 tx=0).stop 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774c2f2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774c2f2640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744063dc0 0x7f77441da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7744074980 con 0x7f7744063dc0 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744063dc0 0x7f77441da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 10 monclient: get_auth_request con 0x7f7744063dc0 auth_method 0 2026-02-17T05:50:08.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 10 monclient(hunting): handle_auth_done global_id 44305 payload 824 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.020+0000 7f774a067640 1 -- 10.20.193.78:0/1139975042 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7744135890 con 0x7f7744063dc0 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774a067640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744063dc0 msgr2=0x7f77441da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774a067640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744063dc0 0x7f77441da560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7734002410 tx=0x7f77340047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734048040 con 0x7f7744063dc0 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 10 monclient: dump: 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:08.028 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7f7734003d50 con 0x7f7744063dc0 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744063dc0 msgr2=0x7f77441da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744063dc0 0x7f77441da560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7734002410 tx=0x7f77340047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744063dc0 msgr2=0x7f77441da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f772affd640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7744063dc0 0x7f77441da560 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7734002410 tx=0x7f77340047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:08.029 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 44, 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 131669, 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:08.030 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:08.031 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:08.032 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:08.033 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:08.034 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:08.035 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:08.036 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:08.037 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:08.038 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:08.039 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:08.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771403deb0 msgr2=0x7f7714040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:08.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f771403deb0 0x7f7714040370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f77441246e0 tx=0x7f7738009040 comp rx=0 tx=0).stop 2026-02-17T05:50:08.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 10 monclient: shutdown 2026-02-17T05:50:08.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:08.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774409d430 msgr2=0x7f77441cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:08.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 1 --2- 10.20.193.78:0/1139975042 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f774409d430 0x7f77441cfd70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7740007e60 tx=0x7f774000d4a0 comp rx=0 tx=0).stop 2026-02-17T05:50:08.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774a868640 1 -- 10.20.193.78:0/1139975042 reap_dead start 2026-02-17T05:50:08.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774a868640 1 -- 10.20.193.78:0/1139975042 reap_dead start 2026-02-17T05:50:08.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 shutdown_connections 2026-02-17T05:50:08.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 >> 10.20.193.78:0/1139975042 conn(0x7f77440a6620 msgr2=0x7f7744136810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:08.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 shutdown_connections 2026-02-17T05:50:08.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:08.023+0000 7f774c2f2640 1 -- 10.20.193.78:0/1139975042 wait complete. 2026-02-17T05:50:08.041 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:50:08.041 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:50:08.676 INFO:tasks.rados.rados.0.trial115.stdout:699: done (7 left) 2026-02-17T05:50:08.676 INFO:tasks.rados.rados.0.trial115.stdout:700: copy_from oid 48 from oid 42 current snap is 68 2026-02-17T05:50:08.676 INFO:tasks.rados.rados.0.trial115.stdout:691: expect (ObjNum 201 snap 41 seq_num 201) 2026-02-17T05:50:08.841 INFO:tasks.rados.rados.0.trial115.stdout:694: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:50:08.894 INFO:tasks.rados.rados.0.trial115.stdout:690: finishing copy_from to trial11545420-20 2026-02-17T05:50:08.894 INFO:tasks.rados.rados.0.trial115.stdout:690: got expected ENOENT (src dne) 2026-02-17T05:50:08.894 INFO:tasks.rados.rados.0.trial115.stdout:690: finishing copy_from racing read to trial11545420-20 2026-02-17T05:50:08.894 INFO:tasks.rados.rados.0.trial115.stdout:692: expect (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:50:09.017 INFO:tasks.rados.rados.0.trial115.stdout:696: finishing write tid 1 to trial11545420-10 2026-02-17T05:50:09.018 INFO:tasks.rados.rados.0.trial115.stdout:690: done (7 left) 2026-02-17T05:50:09.018 INFO:tasks.rados.rados.0.trial115.stdout:691: done (6 left) 2026-02-17T05:50:09.018 INFO:tasks.rados.rados.0.trial115.stdout:692: done (5 left) 2026-02-17T05:50:09.018 INFO:tasks.rados.rados.0.trial115.stdout:694: done (4 left) 2026-02-17T05:50:09.018 INFO:tasks.rados.rados.0.trial115.stdout:701: write (excl) oid 36 current snap is 68 2026-02-17T05:50:09.018 INFO:tasks.rados.rados.0.trial115.stdout:696: finishing write tid 2 to trial11545420-10 2026-02-17T05:50:09.018 INFO:tasks.rados.rados.0.trial115.stdout:701: seq_num 293 ranges {524451=469371,1579224=483096,2529878=394573} 2026-02-17T05:50:09.021 INFO:tasks.rados.rados.0.trial115.stdout:701: writing trial11545420-36 from 524451 to 993822 tid 1 2026-02-17T05:50:09.025 INFO:tasks.rados.rados.0.trial115.stdout:701: writing trial11545420-36 from 1579224 to 2062320 tid 2 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:701: writing trial11545420-36 from 2529878 to 2924451 tid 3 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:702: delete oid 19 current snap is 68 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:696: finishing write tid 3 to trial11545420-10 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:696: finishing write tid 4 to trial11545420-10 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:696: oid 10 updating version 0 to 240 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:696: oid 10 version 240 is already newer than 239 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:696: oid 10 version 240 is already newer than 238 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 240 (ObjNum 290 snap 68 seq_num 290) dirty exists 2026-02-17T05:50:09.029 INFO:tasks.rados.rados.0.trial115.stdout:696: left oid 10 (ObjNum 290 snap 68 seq_num 290) 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: finishing write tid 1 to trial11545420-30 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: finishing write tid 2 to trial11545420-30 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: finishing write tid 3 to trial11545420-30 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: finishing write tid 4 to trial11545420-30 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: finishing write tid 5 to trial11545420-30 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: oid 30 updating version 0 to 197 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: oid 30 version 197 is already newer than 195 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: oid 30 version 197 is already newer than 196 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: oid 30 version 197 is already newer than 194 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 197 (ObjNum 291 snap 68 seq_num 291) dirty exists 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:697: left oid 30 (ObjNum 291 snap 68 seq_num 291) 2026-02-17T05:50:09.030 INFO:tasks.rados.rados.0.trial115.stdout:698: finishing write tid 1 to trial11545420-33 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:698: finishing write tid 2 to trial11545420-33 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:698: finishing write tid 3 to trial11545420-33 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:698: finishing write tid 4 to trial11545420-33 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:698: oid 33 updating version 0 to 183 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:698: oid 33 version 183 is already newer than 181 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:698: oid 33 version 183 is already newer than 182 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 183 (ObjNum 292 snap 68 seq_num 292) dirty exists 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:698: left oid 33 (ObjNum 292 snap 68 seq_num 292) 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:700: finishing copy_from racing read to trial11545420-48 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:700: finishing copy_from to trial11545420-48 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 302 (ObjNum 261 snap 52 seq_num 261) dirty exists 2026-02-17T05:50:09.031 INFO:tasks.rados.rados.0.trial115.stdout:701: finishing write tid 1 to trial11545420-36 2026-02-17T05:50:09.032 INFO:tasks.rados.rados.0.trial115.stdout:701: finishing write tid 2 to trial11545420-36 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:701: finishing write tid 3 to trial11545420-36 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:701: finishing write tid 4 to trial11545420-36 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:701: finishing write tid 5 to trial11545420-36 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:701: oid 36 updating version 0 to 187 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:701: oid 36 version 187 is already newer than 186 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:701: oid 36 version 187 is already newer than 185 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:701: oid 36 version 187 is already newer than 184 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 187 (ObjNum 293 snap 68 seq_num 293) dirty exists 2026-02-17T05:50:09.033 INFO:tasks.rados.rados.0.trial115.stdout:701: left oid 36 (ObjNum 293 snap 68 seq_num 293) 2026-02-17T05:50:09.034 INFO:tasks.rados.rados.0.trial115.stdout:696: done (5 left) 2026-02-17T05:50:09.034 INFO:tasks.rados.rados.0.trial115.stdout:697: done (4 left) 2026-02-17T05:50:09.034 INFO:tasks.rados.rados.0.trial115.stdout:698: done (3 left) 2026-02-17T05:50:09.034 INFO:tasks.rados.rados.0.trial115.stdout:700: done (2 left) 2026-02-17T05:50:09.034 INFO:tasks.rados.rados.0.trial115.stdout:701: done (1 left) 2026-02-17T05:50:09.034 INFO:tasks.rados.rados.0.trial115.stdout:702: done (0 left) 2026-02-17T05:50:09.034 INFO:tasks.rados.rados.0.trial115.stdout:703: write oid 35 current snap is 68 2026-02-17T05:50:09.034 INFO:tasks.rados.rados.0.trial115.stdout:703: seq_num 295 ranges {558065=619869,1579835=486765,2468371=489694} 2026-02-17T05:50:09.035 INFO:tasks.rados.rados.0.trial115.stdout:703: writing trial11545420-35 from 558065 to 1177934 tid 1 2026-02-17T05:50:09.038 INFO:tasks.rados.rados.0.trial115.stdout:703: writing trial11545420-35 from 1579835 to 2066600 tid 2 2026-02-17T05:50:09.041 INFO:tasks.rados.rados.0.trial115.stdout:703: writing trial11545420-35 from 2468371 to 2958065 tid 3 2026-02-17T05:50:09.041 INFO:tasks.rados.rados.0.trial115.stdout:704: rollback oid 12 current snap is 68 2026-02-17T05:50:09.042 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 62 2026-02-17T05:50:09.042 INFO:tasks.rados.rados.0.trial115.stdout:703: finishing write tid 1 to trial11545420-35 2026-02-17T05:50:09.042 INFO:tasks.rados.rados.0.trial115.stdout:703: finishing write tid 2 to trial11545420-35 2026-02-17T05:50:09.042 INFO:tasks.rados.rados.0.trial115.stdout:705: delete oid 23 current snap is 68 2026-02-17T05:50:09.043 INFO:tasks.rados.rados.0.trial115.stdout:703: finishing write tid 3 to trial11545420-35 2026-02-17T05:50:09.043 INFO:tasks.rados.rados.0.trial115.stdout:703: finishing write tid 4 to trial11545420-35 2026-02-17T05:50:09.043 INFO:tasks.rados.rados.0.trial115.stdout:704: finishing rollback tid 0 to trial11545420-12 2026-02-17T05:50:09.044 INFO:tasks.rados.rados.0.trial115.stdout:705: done (2 left) 2026-02-17T05:50:09.044 INFO:tasks.rados.rados.0.trial115.stdout:706: snap_remove snap 65 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:706: done (2 left) 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:707: copy_from oid 27 from oid 43 current snap is 68 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:703: finishing write tid 5 to trial11545420-35 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:703: oid 35 updating version 0 to 304 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:703: oid 35 updating version 304 to 307 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:703: oid 35 version 307 is already newer than 305 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:703: oid 35 version 307 is already newer than 306 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 307 (ObjNum 295 snap 68 seq_num 295) dirty exists 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:703: left oid 35 (ObjNum 295 snap 68 seq_num 295) 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:704: finishing rollback tid 1 to trial11545420-12 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:704: finishing rollback tid 2 to trial11545420-12 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 310 (ObjNum 266 snap 54 seq_num 266) dirty exists 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:703: done (2 left) 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:704: done (1 left) 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:708: rollback oid 40 current snap is 68 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 52 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:709: read oid 38 snap 57 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:709: expect (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:710: read oid 34 snap 59 2026-02-17T05:50:09.678 INFO:tasks.rados.rados.0.trial115.stdout:710: expect deleted 2026-02-17T05:50:09.679 INFO:tasks.rados.rados.0.trial115.stdout:711: snap_create 2026-02-17T05:50:09.681 INFO:tasks.rados.rados.0.trial115.stdout:708: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:50:09.681 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 0 (ObjNum 305 snap 0 seq_num 1715449376) dirty dne 2026-02-17T05:50:09.683 INFO:tasks.rados.rados.0.trial115.stdout:709: expect (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:50:09.759 INFO:tasks.rados.rados.0.trial115.stdout:707: finishing copy_from racing read to trial11545420-27 2026-02-17T05:50:09.759 INFO:tasks.rados.rados.0.trial115.stdout:707: finishing copy_from to trial11545420-27 2026-02-17T05:50:09.759 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 294 (ObjNum 261 snap 52 seq_num 261) dirty exists 2026-02-17T05:50:10.539 INFO:tasks.rados.rados.0.trial115.stdout:707: done (4 left) 2026-02-17T05:50:10.539 INFO:tasks.rados.rados.0.trial115.stdout:708: done (3 left) 2026-02-17T05:50:10.539 INFO:tasks.rados.rados.0.trial115.stdout:709: done (2 left) 2026-02-17T05:50:10.539 INFO:tasks.rados.rados.0.trial115.stdout:710: done (1 left) 2026-02-17T05:50:10.540 INFO:tasks.rados.rados.0.trial115.stdout:711: done (0 left) 2026-02-17T05:50:10.540 INFO:tasks.rados.rados.0.trial115.stdout:712: delete oid 30 current snap is 69 2026-02-17T05:50:10.541 INFO:tasks.rados.rados.0.trial115.stdout:712: done (0 left) 2026-02-17T05:50:10.541 INFO:tasks.rados.rados.0.trial115.stdout:713: rollback oid 18 current snap is 69 2026-02-17T05:50:10.541 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 66 2026-02-17T05:50:10.542 INFO:tasks.rados.rados.0.trial115.stdout:714: read oid 48 snap -1 2026-02-17T05:50:10.542 INFO:tasks.rados.rados.0.trial115.stdout:714: expect (ObjNum 261 snap 52 seq_num 261) 2026-02-17T05:50:10.542 INFO:tasks.rados.rados.0.trial115.stdout:715: read oid 3 snap -1 2026-02-17T05:50:10.542 INFO:tasks.rados.rados.0.trial115.stdout:715: expect (ObjNum 196 snap 37 seq_num 196) 2026-02-17T05:50:10.542 INFO:tasks.rados.rados.0.trial115.stdout:716: snap_create 2026-02-17T05:50:10.543 INFO:tasks.rados.rados.0.trial115.stdout:715: expect (ObjNum 196 snap 37 seq_num 196) 2026-02-17T05:50:10.676 INFO:tasks.rados.rados.0.trial115.stdout:713: finishing rollback tid 0 to trial11545420-18 2026-02-17T05:50:10.676 INFO:tasks.rados.rados.0.trial115.stdout:713: finishing rollback tid 1 to trial11545420-18 2026-02-17T05:50:10.676 INFO:tasks.rados.rados.0.trial115.stdout:714: expect (ObjNum 261 snap 52 seq_num 261) 2026-02-17T05:50:10.787 INFO:tasks.rados.rados.0.trial115.stdout:713: finishing rollback tid 2 to trial11545420-18 2026-02-17T05:50:10.787 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 223 (ObjNum 286 snap 64 seq_num 286) dirty exists 2026-02-17T05:50:11.541 INFO:tasks.rados.rados.0.trial115.stdout:713: done (3 left) 2026-02-17T05:50:11.541 INFO:tasks.rados.rados.0.trial115.stdout:714: done (2 left) 2026-02-17T05:50:11.542 INFO:tasks.rados.rados.0.trial115.stdout:715: done (1 left) 2026-02-17T05:50:11.542 INFO:tasks.rados.rados.0.trial115.stdout:716: done (0 left) 2026-02-17T05:50:11.542 INFO:tasks.rados.rados.0.trial115.stdout:717: snap_remove snap 64 2026-02-17T05:50:12.543 INFO:tasks.rados.rados.0.trial115.stdout:717: done (0 left) 2026-02-17T05:50:12.543 INFO:tasks.rados.rados.0.trial115.stdout:718: write (excl) oid 30 current snap is 70 2026-02-17T05:50:12.543 INFO:tasks.rados.rados.0.trial115.stdout:718: seq_num 298 ranges {608486=561969,1871634=634092,2950640=57846} 2026-02-17T05:50:12.548 INFO:tasks.rados.rados.0.trial115.stdout:718: writing trial11545420-30 from 608486 to 1170455 tid 1 2026-02-17T05:50:12.553 INFO:tasks.rados.rados.0.trial115.stdout:718: writing trial11545420-30 from 1871634 to 2505726 tid 2 2026-02-17T05:50:12.555 INFO:tasks.rados.rados.0.trial115.stdout:718: writing trial11545420-30 from 2950640 to 3008486 tid 3 2026-02-17T05:50:12.555 INFO:tasks.rados.rados.0.trial115.stdout:719: copy_from oid 33 from oid 20 current snap is 70 2026-02-17T05:50:12.555 INFO:tasks.rados.rados.0.trial115.stdout:718: finishing write tid 1 to trial11545420-30 2026-02-17T05:50:12.555 INFO:tasks.rados.rados.0.trial115.stdout:720: delete oid 29 current snap is 70 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:718: finishing write tid 2 to trial11545420-30 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:718: finishing write tid 3 to trial11545420-30 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:718: finishing write tid 4 to trial11545420-30 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:718: finishing write tid 5 to trial11545420-30 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:718: oid 30 updating version 0 to 297 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:718: oid 30 updating version 297 to 299 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:718: oid 30 version 299 is already newer than 298 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:718: oid 30 version 299 is already newer than 296 2026-02-17T05:50:12.556 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 299 (ObjNum 298 snap 70 seq_num 298) dirty exists 2026-02-17T05:50:12.557 INFO:tasks.rados.rados.0.trial115.stdout:718: left oid 30 (ObjNum 298 snap 70 seq_num 298) 2026-02-17T05:50:12.557 INFO:tasks.rados.rados.0.trial115.stdout:718: done (2 left) 2026-02-17T05:50:12.557 INFO:tasks.rados.rados.0.trial115.stdout:720: done (1 left) 2026-02-17T05:50:12.557 INFO:tasks.rados.rados.0.trial115.stdout:721: snap_create 2026-02-17T05:50:12.561 INFO:tasks.rados.rados.0.trial115.stdout:719: finishing copy_from racing read to trial11545420-33 2026-02-17T05:50:12.563 INFO:tasks.rados.rados.0.trial115.stdout:719: finishing copy_from to trial11545420-33 2026-02-17T05:50:12.563 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 291 (ObjNum 267 snap 54 seq_num 267) dirty exists 2026-02-17T05:50:13.546 INFO:tasks.rados.rados.0.trial115.stdout:719: done (1 left) 2026-02-17T05:50:13.546 INFO:tasks.rados.rados.0.trial115.stdout:721: done (0 left) 2026-02-17T05:50:13.546 INFO:tasks.rados.rados.0.trial115.stdout:722: copy_from oid 1 from oid 44 current snap is 71 2026-02-17T05:50:13.546 INFO:tasks.rados.rados.0.trial115.stdout:723: read oid 27 snap 57 2026-02-17T05:50:13.547 INFO:tasks.rados.rados.0.trial115.stdout:723: expect (ObjNum 253 snap 49 seq_num 253) 2026-02-17T05:50:13.547 INFO:tasks.rados.rados.0.trial115.stdout:724: read oid 19 snap 57 2026-02-17T05:50:13.547 INFO:tasks.rados.rados.0.trial115.stdout:724: expect (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:50:13.547 INFO:tasks.rados.rados.0.trial115.stdout:725: snap_create 2026-02-17T05:50:13.552 INFO:tasks.rados.rados.0.trial115.stdout:723: expect (ObjNum 253 snap 49 seq_num 253) 2026-02-17T05:50:13.619 INFO:tasks.rados.rados.0.trial115.stdout:724: expect (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:50:13.741 INFO:tasks.rados.rados.0.trial115.stdout:722: finishing copy_from racing read to trial11545420-1 2026-02-17T05:50:13.742 INFO:tasks.rados.rados.0.trial115.stdout:722: finishing copy_from to trial11545420-1 2026-02-17T05:50:13.742 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 266 (ObjNum 251 snap 49 seq_num 251) dirty exists 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:722: done (3 left) 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:723: done (2 left) 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:724: done (1 left) 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:725: done (0 left) 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:726: read oid 5 snap 70 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:726: expect (ObjNum 243 snap 48 seq_num 243) 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:727: read oid 4 snap -1 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:727: expect (ObjNum 242 snap 48 seq_num 242) 2026-02-17T05:50:14.548 INFO:tasks.rados.rados.0.trial115.stdout:728: delete oid 28 current snap is 72 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:728: done (2 left) 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:729: rollback oid 18 current snap is 72 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 55 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:730: read oid 6 snap 59 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:730: expect (ObjNum 201 snap 41 seq_num 201) 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:731: read oid 20 snap -1 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:731: expect (ObjNum 267 snap 54 seq_num 267) 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:732: write oid 17 current snap is 72 2026-02-17T05:50:14.551 INFO:tasks.rados.rados.0.trial115.stdout:732: seq_num 301 ranges {658907=787716,1930727=497772,3032909=25998} 2026-02-17T05:50:14.557 INFO:tasks.rados.rados.0.trial115.stdout:732: writing trial11545420-17 from 658907 to 1446623 tid 1 2026-02-17T05:50:14.561 INFO:tasks.rados.rados.0.trial115.stdout:732: writing trial11545420-17 from 1930727 to 2428499 tid 2 2026-02-17T05:50:14.563 INFO:tasks.rados.rados.0.trial115.stdout:732: writing trial11545420-17 from 3032909 to 3058907 tid 3 2026-02-17T05:50:14.563 INFO:tasks.rados.rados.0.trial115.stdout:733: write (excl) oid 45 current snap is 72 2026-02-17T05:50:14.563 INFO:tasks.rados.rados.0.trial115.stdout:733: seq_num 302 ranges {675714=462965,1589209=641430,2943979=131735} 2026-02-17T05:50:14.567 INFO:tasks.rados.rados.0.trial115.stdout:733: writing trial11545420-45 from 675714 to 1138679 tid 1 2026-02-17T05:50:14.572 INFO:tasks.rados.rados.0.trial115.stdout:733: writing trial11545420-45 from 1589209 to 2230639 tid 2 2026-02-17T05:50:14.576 INFO:tasks.rados.rados.0.trial115.stdout:733: writing trial11545420-45 from 2943979 to 3075714 tid 3 2026-02-17T05:50:14.576 INFO:tasks.rados.rados.0.trial115.stdout:734: snap_remove snap 60 2026-02-17T05:50:15.540 INFO:tasks.rados.rados.0.trial115.stdout:734: done (7 left) 2026-02-17T05:50:15.540 INFO:tasks.rados.rados.0.trial115.stdout:735: write oid 11 current snap is 72 2026-02-17T05:50:15.540 INFO:tasks.rados.rados.0.trial115.stdout:735: seq_num 303 ranges {692521=538214,1931338=785088} 2026-02-17T05:50:15.544 INFO:tasks.rados.rados.0.trial115.stdout:735: writing trial11545420-11 from 692521 to 1230735 tid 1 2026-02-17T05:50:15.551 INFO:tasks.rados.rados.0.trial115.stdout:735: writing trial11545420-11 from 1931338 to 2716426 tid 2 2026-02-17T05:50:15.551 INFO:tasks.rados.rados.0.trial115.stdout:726: expect (ObjNum 243 snap 48 seq_num 243) 2026-02-17T05:50:15.597 INFO:tasks.rados.rados.0.trial115.stdout:727: expect (ObjNum 242 snap 48 seq_num 242) 2026-02-17T05:50:15.644 INFO:tasks.rados.rados.0.trial115.stdout:729: finishing rollback tid 0 to trial11545420-18 2026-02-17T05:50:15.644 INFO:tasks.rados.rados.0.trial115.stdout:736: read oid 31 snap 52 2026-02-17T05:50:15.644 INFO:tasks.rados.rados.0.trial115.stdout:736: expect (ObjNum 138 snap 25 seq_num 138) 2026-02-17T05:50:15.645 INFO:tasks.rados.rados.0.trial115.stdout:726: done (8 left) 2026-02-17T05:50:15.645 INFO:tasks.rados.rados.0.trial115.stdout:727: done (7 left) 2026-02-17T05:50:15.645 INFO:tasks.rados.rados.0.trial115.stdout:737: copy_from oid 5 from oid 36 current snap is 72 2026-02-17T05:50:15.645 INFO:tasks.rados.rados.0.trial115.stdout:729: finishing rollback tid 1 to trial11545420-18 2026-02-17T05:50:15.645 INFO:tasks.rados.rados.0.trial115.stdout:729: finishing rollback tid 2 to trial11545420-18 2026-02-17T05:50:15.645 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 231 (ObjNum 208 snap 44 seq_num 208) dirty exists 2026-02-17T05:50:15.645 INFO:tasks.rados.rados.0.trial115.stdout:731: expect (ObjNum 267 snap 54 seq_num 267) 2026-02-17T05:50:15.752 INFO:tasks.rados.rados.0.trial115.stdout:730: expect (ObjNum 201 snap 41 seq_num 201) 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: finishing write tid 1 to trial11545420-17 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:729: done (7 left) 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:730: done (6 left) 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:731: done (5 left) 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:738: snap_create 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: finishing write tid 2 to trial11545420-17 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: finishing write tid 3 to trial11545420-17 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: finishing write tid 4 to trial11545420-17 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: finishing write tid 5 to trial11545420-17 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: oid 17 updating version 0 to 84 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: oid 17 version 84 is already newer than 83 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: oid 17 version 84 is already newer than 82 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: oid 17 updating version 84 to 85 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 85 (ObjNum 301 snap 72 seq_num 301) dirty exists 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:732: left oid 17 (ObjNum 301 snap 72 seq_num 301) 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: finishing write tid 1 to trial11545420-45 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: finishing write tid 2 to trial11545420-45 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: finishing write tid 3 to trial11545420-45 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: finishing write tid 4 to trial11545420-45 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: finishing write tid 5 to trial11545420-45 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: oid 45 updating version 0 to 320 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: oid 45 updating version 320 to 322 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: oid 45 updating version 322 to 323 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:733: oid 45 version 323 is already newer than 321 2026-02-17T05:50:15.916 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 323 (ObjNum 302 snap 72 seq_num 302) dirty exists 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:733: left oid 45 (ObjNum 302 snap 72 seq_num 302) 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:735: finishing write tid 1 to trial11545420-11 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:735: finishing write tid 2 to trial11545420-11 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:735: finishing write tid 3 to trial11545420-11 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:735: finishing write tid 4 to trial11545420-11 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:735: oid 11 updating version 0 to 262 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:735: oid 11 updating version 262 to 263 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:735: oid 11 version 263 is already newer than 253 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 263 (ObjNum 303 snap 72 seq_num 303) dirty exists 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:735: left oid 11 (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:50:15.917 INFO:tasks.rados.rados.0.trial115.stdout:736: expect (ObjNum 138 snap 25 seq_num 138) 2026-02-17T05:50:15.970 INFO:tasks.rados.rados.0.trial115.stdout:737: finishing copy_from racing read to trial11545420-5 2026-02-17T05:50:15.971 INFO:tasks.rados.rados.0.trial115.stdout:737: finishing copy_from to trial11545420-5 2026-02-17T05:50:15.971 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 187 (ObjNum 293 snap 68 seq_num 293) dirty exists 2026-02-17T05:50:16.555 INFO:tasks.rados.rados.0.trial115.stdout:732: done (5 left) 2026-02-17T05:50:16.555 INFO:tasks.rados.rados.0.trial115.stdout:733: done (4 left) 2026-02-17T05:50:16.555 INFO:tasks.rados.rados.0.trial115.stdout:735: done (3 left) 2026-02-17T05:50:16.555 INFO:tasks.rados.rados.0.trial115.stdout:736: done (2 left) 2026-02-17T05:50:16.556 INFO:tasks.rados.rados.0.trial115.stdout:737: done (1 left) 2026-02-17T05:50:16.556 INFO:tasks.rados.rados.0.trial115.stdout:738: done (0 left) 2026-02-17T05:50:16.556 INFO:tasks.rados.rados.0.trial115.stdout:739: rollback oid 46 current snap is 73 2026-02-17T05:50:16.556 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 46 to 57 2026-02-17T05:50:16.556 INFO:tasks.rados.rados.0.trial115.stdout:740: copy_from oid 16 from oid 6 current snap is 73 2026-02-17T05:50:16.556 INFO:tasks.rados.rados.0.trial115.stdout:741: read oid 9 snap -1 2026-02-17T05:50:16.557 INFO:tasks.rados.rados.0.trial115.stdout:741: expect (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:50:16.557 INFO:tasks.rados.rados.0.trial115.stdout:742: read oid 1 snap 68 2026-02-17T05:50:16.557 INFO:tasks.rados.rados.0.trial115.stdout:742: expect deleted 2026-02-17T05:50:16.557 INFO:tasks.rados.rados.0.trial115.stdout:743: delete oid 24 current snap is 73 2026-02-17T05:50:16.558 INFO:tasks.rados.rados.0.trial115.stdout:742: done (4 left) 2026-02-17T05:50:16.558 INFO:tasks.rados.rados.0.trial115.stdout:743: done (3 left) 2026-02-17T05:50:16.558 INFO:tasks.rados.rados.0.trial115.stdout:744: write oid 27 current snap is 73 2026-02-17T05:50:16.559 INFO:tasks.rados.rados.0.trial115.stdout:744: seq_num 305 ranges {726135=405065,1764655=788757,3026248=99887} 2026-02-17T05:50:16.560 INFO:tasks.rados.rados.0.trial115.stdout:744: writing trial11545420-27 from 726135 to 1131200 tid 1 2026-02-17T05:50:16.566 INFO:tasks.rados.rados.0.trial115.stdout:744: writing trial11545420-27 from 1764655 to 2553412 tid 2 2026-02-17T05:50:16.568 INFO:tasks.rados.rados.0.trial115.stdout:744: writing trial11545420-27 from 3026248 to 3126135 tid 3 2026-02-17T05:50:16.568 INFO:tasks.rados.rados.0.trial115.stdout:745: snap_remove snap 62 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:745: done (4 left) 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:746: read oid 24 snap 55 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:746: expect deleted 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:747: rollback oid 32 current snap is 73 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 69 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:739: finishing rollback tid 0 to trial11545420-46 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:739: finishing rollback tid 1 to trial11545420-46 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:739: finishing rollback tid 2 to trial11545420-46 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 267 (ObjNum 272 snap 56 seq_num 272) dirty exists 2026-02-17T05:50:17.557 INFO:tasks.rados.rados.0.trial115.stdout:741: expect (ObjNum 106 snap 16 seq_num 106) 2026-02-17T05:50:17.681 INFO:tasks.rados.rados.0.trial115.stdout:740: finishing copy_from racing read to trial11545420-16 2026-02-17T05:50:17.681 INFO:tasks.rados.rados.0.trial115.stdout:740: finishing copy_from to trial11545420-16 2026-02-17T05:50:17.681 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 283 (ObjNum 201 snap 41 seq_num 201) dirty exists 2026-02-17T05:50:17.681 INFO:tasks.rados.rados.0.trial115.stdout:744: finishing write tid 1 to trial11545420-27 2026-02-17T05:50:17.681 INFO:tasks.rados.rados.0.trial115.stdout:739: done (5 left) 2026-02-17T05:50:17.681 INFO:tasks.rados.rados.0.trial115.stdout:740: done (4 left) 2026-02-17T05:50:17.681 INFO:tasks.rados.rados.0.trial115.stdout:741: done (3 left) 2026-02-17T05:50:17.682 INFO:tasks.rados.rados.0.trial115.stdout:748: write oid 19 current snap is 73 2026-02-17T05:50:17.682 INFO:tasks.rados.rados.0.trial115.stdout:748: seq_num 306 ranges {742942=480314,1823137=648768} 2026-02-17T05:50:17.685 INFO:tasks.rados.rados.0.trial115.stdout:748: writing trial11545420-19 from 742942 to 1223256 tid 1 2026-02-17T05:50:17.691 INFO:tasks.rados.rados.0.trial115.stdout:748: writing trial11545420-19 from 1823137 to 2471905 tid 2 2026-02-17T05:50:17.691 INFO:tasks.rados.rados.0.trial115.stdout:749: copy_from oid 14 from oid 49 current snap is 73 2026-02-17T05:50:17.691 INFO:tasks.rados.rados.0.trial115.stdout:744: finishing write tid 2 to trial11545420-27 2026-02-17T05:50:17.691 INFO:tasks.rados.rados.0.trial115.stdout:744: finishing write tid 3 to trial11545420-27 2026-02-17T05:50:17.691 INFO:tasks.rados.rados.0.trial115.stdout:744: finishing write tid 4 to trial11545420-27 2026-02-17T05:50:17.691 INFO:tasks.rados.rados.0.trial115.stdout:744: finishing write tid 5 to trial11545420-27 2026-02-17T05:50:17.691 INFO:tasks.rados.rados.0.trial115.stdout:744: oid 27 updating version 0 to 300 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:744: oid 27 updating version 300 to 303 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:744: oid 27 version 303 is already newer than 302 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:744: oid 27 version 303 is already newer than 301 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 303 (ObjNum 305 snap 73 seq_num 305) dirty exists 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:744: left oid 27 (ObjNum 305 snap 73 seq_num 305) 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:744: done (4 left) 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:750: copy_from oid 15 from oid 17 current snap is 73 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:747: finishing rollback tid 0 to trial11545420-32 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:747: finishing rollback tid 1 to trial11545420-32 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:747: finishing rollback tid 2 to trial11545420-32 2026-02-17T05:50:17.692 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 269 (ObjNum 242 snap 48 seq_num 242) dirty exists 2026-02-17T05:50:17.693 INFO:tasks.rados.rados.0.trial115.stdout:748: finishing write tid 1 to trial11545420-19 2026-02-17T05:50:17.693 INFO:tasks.rados.rados.0.trial115.stdout:746: done (4 left) 2026-02-17T05:50:17.693 INFO:tasks.rados.rados.0.trial115.stdout:747: done (3 left) 2026-02-17T05:50:17.693 INFO:tasks.rados.rados.0.trial115.stdout:751: snap_remove snap 72 2026-02-17T05:50:18.560 INFO:tasks.rados.rados.0.trial115.stdout:751: done (3 left) 2026-02-17T05:50:18.560 INFO:tasks.rados.rados.0.trial115.stdout:752: snap_remove snap 70 2026-02-17T05:50:19.560 INFO:tasks.rados.rados.0.trial115.stdout:752: done (3 left) 2026-02-17T05:50:19.560 INFO:tasks.rados.rados.0.trial115.stdout:753: write (excl) oid 41 current snap is 73 2026-02-17T05:50:19.560 INFO:tasks.rados.rados.0.trial115.stdout:753: seq_num 307 ranges {759749=555563,1765266=792426,2964741=195008} 2026-02-17T05:50:19.566 INFO:tasks.rados.rados.0.trial115.stdout:753: writing trial11545420-41 from 759749 to 1315312 tid 1 2026-02-17T05:50:19.571 INFO:tasks.rados.rados.0.trial115.stdout:753: writing trial11545420-41 from 1765266 to 2557692 tid 2 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:753: writing trial11545420-41 from 2964741 to 3159749 tid 3 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:754: copy_from oid 40 from oid 34 current snap is 73 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:748: finishing write tid 2 to trial11545420-19 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:748: finishing write tid 3 to trial11545420-19 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:748: finishing write tid 4 to trial11545420-19 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:748: oid 19 updating version 0 to 309 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:748: oid 19 version 309 is already newer than 308 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:748: oid 19 updating version 309 to 310 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 310 (ObjNum 306 snap 73 seq_num 306) dirty exists 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:748: left oid 19 (ObjNum 306 snap 73 seq_num 306) 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:749: finishing copy_from racing read to trial11545420-14 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:748: done (4 left) 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:755: write (excl) oid 25 current snap is 73 2026-02-17T05:50:19.574 INFO:tasks.rados.rados.0.trial115.stdout:755: seq_num 308 ranges {776556=630812,1823748=652437,3108517=68039} 2026-02-17T05:50:19.578 INFO:tasks.rados.rados.0.trial115.stdout:755: writing trial11545420-25 from 776556 to 1407368 tid 1 2026-02-17T05:50:19.582 INFO:tasks.rados.rados.0.trial115.stdout:755: writing trial11545420-25 from 1823748 to 2476185 tid 2 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:755: writing trial11545420-25 from 3108517 to 3176556 tid 3 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:756: delete oid 46 current snap is 73 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:750: finishing copy_from racing read to trial11545420-15 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:750: finishing copy_from to trial11545420-15 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 85 (ObjNum 301 snap 72 seq_num 301) dirty exists 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:749: finishing copy_from to trial11545420-14 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 43 (ObjNum 125 snap 20 seq_num 125) dirty exists 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:753: finishing write tid 1 to trial11545420-41 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:753: finishing write tid 2 to trial11545420-41 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:753: finishing write tid 3 to trial11545420-41 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:753: finishing write tid 4 to trial11545420-41 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:753: finishing write tid 5 to trial11545420-41 2026-02-17T05:50:19.586 INFO:tasks.rados.rados.0.trial115.stdout:753: oid 41 updating version 0 to 247 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:753: oid 41 version 247 is already newer than 246 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:753: oid 41 version 247 is already newer than 243 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:753: oid 41 updating version 247 to 248 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 248 (ObjNum 307 snap 73 seq_num 307) dirty exists 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:753: left oid 41 (ObjNum 307 snap 73 seq_num 307) 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:754: finishing copy_from to trial11545420-40 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:754: got expected ENOENT (src dne) 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:754: finishing copy_from racing read to trial11545420-40 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:755: finishing write tid 1 to trial11545420-25 2026-02-17T05:50:19.587 INFO:tasks.rados.rados.0.trial115.stdout:755: finishing write tid 2 to trial11545420-25 2026-02-17T05:50:19.588 INFO:tasks.rados.rados.0.trial115.stdout:755: finishing write tid 3 to trial11545420-25 2026-02-17T05:50:19.588 INFO:tasks.rados.rados.0.trial115.stdout:755: finishing write tid 4 to trial11545420-25 2026-02-17T05:50:19.588 INFO:tasks.rados.rados.0.trial115.stdout:755: finishing write tid 5 to trial11545420-25 2026-02-17T05:50:19.588 INFO:tasks.rados.rados.0.trial115.stdout:755: oid 25 updating version 0 to 249 2026-02-17T05:50:19.588 INFO:tasks.rados.rados.0.trial115.stdout:755: oid 25 updating version 249 to 251 2026-02-17T05:50:19.588 INFO:tasks.rados.rados.0.trial115.stdout:755: oid 25 updating version 251 to 252 2026-02-17T05:50:19.588 INFO:tasks.rados.rados.0.trial115.stdout:755: oid 25 version 252 is already newer than 250 2026-02-17T05:50:19.588 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 252 (ObjNum 308 snap 73 seq_num 308) dirty exists 2026-02-17T05:50:19.589 INFO:tasks.rados.rados.0.trial115.stdout:755: left oid 25 (ObjNum 308 snap 73 seq_num 308) 2026-02-17T05:50:19.589 INFO:tasks.rados.rados.0.trial115.stdout:749: done (5 left) 2026-02-17T05:50:19.589 INFO:tasks.rados.rados.0.trial115.stdout:750: done (4 left) 2026-02-17T05:50:19.590 INFO:tasks.rados.rados.0.trial115.stdout:753: done (3 left) 2026-02-17T05:50:19.590 INFO:tasks.rados.rados.0.trial115.stdout:754: done (2 left) 2026-02-17T05:50:19.590 INFO:tasks.rados.rados.0.trial115.stdout:755: done (1 left) 2026-02-17T05:50:19.590 INFO:tasks.rados.rados.0.trial115.stdout:756: done (0 left) 2026-02-17T05:50:19.590 INFO:tasks.rados.rados.0.trial115.stdout:757: copy_from oid 18 from oid 33 current snap is 73 2026-02-17T05:50:19.590 INFO:tasks.rados.rados.0.trial115.stdout:758: write (excl) oid 35 current snap is 73 2026-02-17T05:50:19.590 INFO:tasks.rados.rados.0.trial115.stdout:758: seq_num 310 ranges {410170=781310,1940712=656106,3163363=46807} 2026-02-17T05:50:19.593 INFO:tasks.rados.rados.0.trial115.stdout:758: writing trial11545420-35 from 410170 to 1191480 tid 1 2026-02-17T05:50:19.598 INFO:tasks.rados.rados.0.trial115.stdout:758: writing trial11545420-35 from 1940712 to 2596818 tid 2 2026-02-17T05:50:19.601 INFO:tasks.rados.rados.0.trial115.stdout:758: writing trial11545420-35 from 3163363 to 3210170 tid 3 2026-02-17T05:50:19.601 INFO:tasks.rados.rados.0.trial115.stdout:759: snap_remove snap 55 2026-02-17T05:50:20.540 INFO:tasks.rados.rados.0.trial115.stdout:759: done (2 left) 2026-02-17T05:50:20.540 INFO:tasks.rados.rados.0.trial115.stdout:760: write oid 16 current snap is 73 2026-02-17T05:50:20.540 INFO:tasks.rados.rados.0.trial115.stdout:760: seq_num 311 ranges {426977=456559,1482841=799764,2958080=268897} 2026-02-17T05:50:20.543 INFO:tasks.rados.rados.0.trial115.stdout:760: writing trial11545420-16 from 426977 to 883536 tid 1 2026-02-17T05:50:20.549 INFO:tasks.rados.rados.0.trial115.stdout:760: writing trial11545420-16 from 1482841 to 2282605 tid 2 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:760: writing trial11545420-16 from 2958080 to 3226977 tid 3 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:761: read oid 28 snap -1 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:761: expect deleted 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:762: copy_from oid 36 from oid 8 current snap is 73 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:757: finishing copy_from racing read to trial11545420-18 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:757: finishing copy_from to trial11545420-18 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 291 (ObjNum 267 snap 54 seq_num 267) dirty exists 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:758: finishing write tid 1 to trial11545420-35 2026-02-17T05:50:20.553 INFO:tasks.rados.rados.0.trial115.stdout:758: finishing write tid 2 to trial11545420-35 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:757: done (4 left) 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:763: copy_from oid 37 from oid 25 current snap is 73 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: finishing write tid 3 to trial11545420-35 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: finishing write tid 4 to trial11545420-35 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: finishing write tid 5 to trial11545420-35 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: oid 35 updating version 0 to 311 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: oid 35 updating version 311 to 313 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: oid 35 updating version 313 to 314 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: oid 35 version 314 is already newer than 312 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 314 (ObjNum 310 snap 73 seq_num 310) dirty exists 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: left oid 35 (ObjNum 310 snap 73 seq_num 310) 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:760: finishing write tid 1 to trial11545420-16 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:758: done (4 left) 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:764: rollback oid 4 current snap is 73 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 57 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:760: finishing write tid 2 to trial11545420-16 2026-02-17T05:50:20.554 INFO:tasks.rados.rados.0.trial115.stdout:765: write (excl) oid 38 current snap is 73 2026-02-17T05:50:20.555 INFO:tasks.rados.rados.0.trial115.stdout:765: seq_num 312 ranges {443784=648161,1657676=659775,2818209=425575} 2026-02-17T05:50:20.557 INFO:tasks.rados.rados.0.trial115.stdout:765: writing trial11545420-38 from 443784 to 1091945 tid 1 2026-02-17T05:50:20.561 INFO:tasks.rados.rados.0.trial115.stdout:765: writing trial11545420-38 from 1657676 to 2317451 tid 2 2026-02-17T05:50:20.565 INFO:tasks.rados.rados.0.trial115.stdout:765: writing trial11545420-38 from 2818209 to 3243784 tid 3 2026-02-17T05:50:20.565 INFO:tasks.rados.rados.0.trial115.stdout:766: copy_from oid 2 from oid 40 current snap is 73 2026-02-17T05:50:20.565 INFO:tasks.rados.rados.0.trial115.stdout:764: finishing rollback tid 0 to trial11545420-4 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: finishing write tid 3 to trial11545420-16 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:764: finishing rollback tid 1 to trial11545420-4 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:764: finishing rollback tid 2 to trial11545420-4 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 172 (ObjNum 242 snap 48 seq_num 242) dirty exists 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: finishing write tid 4 to trial11545420-16 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: finishing write tid 5 to trial11545420-16 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: oid 16 updating version 0 to 315 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: oid 16 updating version 315 to 317 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: oid 16 version 317 is already newer than 316 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: oid 16 updating version 317 to 318 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 318 (ObjNum 311 snap 73 seq_num 311) dirty exists 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: left oid 16 (ObjNum 311 snap 73 seq_num 311) 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:762: finishing copy_from racing read to trial11545420-36 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:763: finishing copy_from racing read to trial11545420-37 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:765: finishing write tid 1 to trial11545420-38 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:760: done (6 left) 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:761: done (5 left) 2026-02-17T05:50:20.566 INFO:tasks.rados.rados.0.trial115.stdout:764: done (4 left) 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:767: copy_from oid 3 from oid 17 current snap is 73 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:763: finishing copy_from to trial11545420-37 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 252 (ObjNum 308 snap 73 seq_num 308) dirty exists 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:762: finishing copy_from to trial11545420-36 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 175 (ObjNum 254 snap 50 seq_num 254) dirty exists 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:762: done (4 left) 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:763: done (3 left) 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:768: snap_create 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:766: finishing copy_from to trial11545420-2 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:766: got expected ENOENT (src dne) 2026-02-17T05:50:20.567 INFO:tasks.rados.rados.0.trial115.stdout:766: finishing copy_from racing read to trial11545420-2 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: finishing write tid 2 to trial11545420-38 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: finishing write tid 3 to trial11545420-38 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: finishing write tid 4 to trial11545420-38 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: finishing write tid 5 to trial11545420-38 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: oid 38 updating version 0 to 321 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: oid 38 version 321 is already newer than 319 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: oid 38 version 321 is already newer than 320 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: oid 38 updating version 321 to 322 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 322 (ObjNum 312 snap 73 seq_num 312) dirty exists 2026-02-17T05:50:20.568 INFO:tasks.rados.rados.0.trial115.stdout:765: left oid 38 (ObjNum 312 snap 73 seq_num 312) 2026-02-17T05:50:20.571 INFO:tasks.rados.rados.0.trial115.stdout:767: finishing copy_from racing read to trial11545420-3 2026-02-17T05:50:20.571 INFO:tasks.rados.rados.0.trial115.stdout:767: finishing copy_from to trial11545420-3 2026-02-17T05:50:20.571 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 85 (ObjNum 301 snap 72 seq_num 301) dirty exists 2026-02-17T05:50:21.566 INFO:tasks.rados.rados.0.trial115.stdout:765: done (3 left) 2026-02-17T05:50:21.566 INFO:tasks.rados.rados.0.trial115.stdout:766: done (2 left) 2026-02-17T05:50:21.566 INFO:tasks.rados.rados.0.trial115.stdout:767: done (1 left) 2026-02-17T05:50:21.566 INFO:tasks.rados.rados.0.trial115.stdout:768: done (0 left) 2026-02-17T05:50:21.567 INFO:tasks.rados.rados.0.trial115.stdout:769: read oid 16 snap -1 2026-02-17T05:50:21.567 INFO:tasks.rados.rados.0.trial115.stdout:769: expect (ObjNum 311 snap 73 seq_num 311) 2026-02-17T05:50:21.567 INFO:tasks.rados.rados.0.trial115.stdout:770: write (excl) oid 43 current snap is 74 2026-02-17T05:50:21.567 INFO:tasks.rados.rados.0.trial115.stdout:770: seq_num 313 ranges {460591=723410,1716158=403433,2729279=440140} 2026-02-17T05:50:21.570 INFO:tasks.rados.rados.0.trial115.stdout:770: writing trial11545420-43 from 460591 to 1184001 tid 1 2026-02-17T05:50:21.573 INFO:tasks.rados.rados.0.trial115.stdout:770: writing trial11545420-43 from 1716158 to 2119591 tid 2 2026-02-17T05:50:21.578 INFO:tasks.rados.rados.0.trial115.stdout:770: writing trial11545420-43 from 2729279 to 3169419 tid 3 2026-02-17T05:50:21.578 INFO:tasks.rados.rados.0.trial115.stdout:771: write (excl) oid 4 current snap is 74 2026-02-17T05:50:21.578 INFO:tasks.rados.rados.0.trial115.stdout:771: seq_num 314 ranges {477398=798659,1774640=663444,2873055=404343} 2026-02-17T05:50:21.582 INFO:tasks.rados.rados.0.trial115.stdout:771: writing trial11545420-4 from 477398 to 1276057 tid 1 2026-02-17T05:50:21.586 INFO:tasks.rados.rados.0.trial115.stdout:771: writing trial11545420-4 from 1774640 to 2438084 tid 2 2026-02-17T05:50:21.590 INFO:tasks.rados.rados.0.trial115.stdout:771: writing trial11545420-4 from 2873055 to 3277398 tid 3 2026-02-17T05:50:21.590 INFO:tasks.rados.rados.0.trial115.stdout:772: rollback oid 38 current snap is 74 2026-02-17T05:50:21.590 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 38 to 52 2026-02-17T05:50:21.590 INFO:tasks.rados.rados.0.trial115.stdout:769: expect (ObjNum 311 snap 73 seq_num 311) 2026-02-17T05:50:21.775 INFO:tasks.rados.rados.0.trial115.stdout:770: finishing write tid 1 to trial11545420-43 2026-02-17T05:50:21.775 INFO:tasks.rados.rados.0.trial115.stdout:769: done (3 left) 2026-02-17T05:50:21.775 INFO:tasks.rados.rados.0.trial115.stdout:773: snap_create 2026-02-17T05:50:21.775 INFO:tasks.rados.rados.0.trial115.stdout:770: finishing write tid 2 to trial11545420-43 2026-02-17T05:50:21.775 INFO:tasks.rados.rados.0.trial115.stdout:770: finishing write tid 3 to trial11545420-43 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:770: finishing write tid 4 to trial11545420-43 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:770: finishing write tid 5 to trial11545420-43 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:770: oid 43 updating version 0 to 295 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:770: oid 43 updating version 295 to 297 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:770: oid 43 updating version 297 to 298 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:770: oid 43 version 298 is already newer than 296 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 298 (ObjNum 313 snap 74 seq_num 313) dirty exists 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:770: left oid 43 (ObjNum 313 snap 74 seq_num 313) 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:771: finishing write tid 1 to trial11545420-4 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:771: finishing write tid 2 to trial11545420-4 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:771: finishing write tid 3 to trial11545420-4 2026-02-17T05:50:21.776 INFO:tasks.rados.rados.0.trial115.stdout:771: finishing write tid 4 to trial11545420-4 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:771: finishing write tid 5 to trial11545420-4 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:771: oid 4 updating version 0 to 175 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:771: oid 4 version 175 is already newer than 173 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:771: oid 4 updating version 175 to 176 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:771: oid 4 version 176 is already newer than 174 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 176 (ObjNum 314 snap 74 seq_num 314) dirty exists 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:771: left oid 4 (ObjNum 314 snap 74 seq_num 314) 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:772: finishing rollback tid 0 to trial11545420-38 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:772: finishing rollback tid 1 to trial11545420-38 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:772: finishing rollback tid 2 to trial11545420-38 2026-02-17T05:50:21.777 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 325 (ObjNum 135 snap 23 seq_num 135) dirty exists 2026-02-17T05:50:22.569 INFO:tasks.rados.rados.0.trial115.stdout:770: done (3 left) 2026-02-17T05:50:22.569 INFO:tasks.rados.rados.0.trial115.stdout:771: done (2 left) 2026-02-17T05:50:22.569 INFO:tasks.rados.rados.0.trial115.stdout:772: done (1 left) 2026-02-17T05:50:22.569 INFO:tasks.rados.rados.0.trial115.stdout:773: done (0 left) 2026-02-17T05:50:22.569 INFO:tasks.rados.rados.0.trial115.stdout:774: write oid 6 current snap is 75 2026-02-17T05:50:22.569 INFO:tasks.rados.rados.0.trial115.stdout:774: seq_num 315 ranges {494205=473908,1433122=407102,2384125=462684} 2026-02-17T05:50:22.572 INFO:tasks.rados.rados.0.trial115.stdout:774: writing trial11545420-6 from 494205 to 968113 tid 1 2026-02-17T05:50:22.576 INFO:tasks.rados.rados.0.trial115.stdout:774: writing trial11545420-6 from 1433122 to 1840224 tid 2 2026-02-17T05:50:22.580 INFO:tasks.rados.rados.0.trial115.stdout:774: writing trial11545420-6 from 2384125 to 2846809 tid 3 2026-02-17T05:50:22.580 INFO:tasks.rados.rados.0.trial115.stdout:775: snap_create 2026-02-17T05:50:22.580 INFO:tasks.rados.rados.0.trial115.stdout:774: finishing write tid 1 to trial11545420-6 2026-02-17T05:50:22.580 INFO:tasks.rados.rados.0.trial115.stdout:774: finishing write tid 2 to trial11545420-6 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:774: finishing write tid 3 to trial11545420-6 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:774: finishing write tid 4 to trial11545420-6 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:774: finishing write tid 5 to trial11545420-6 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:774: oid 6 updating version 0 to 287 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:774: oid 6 version 287 is already newer than 284 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:774: oid 6 version 287 is already newer than 286 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:774: oid 6 version 287 is already newer than 285 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 287 (ObjNum 315 snap 75 seq_num 315) dirty exists 2026-02-17T05:50:22.582 INFO:tasks.rados.rados.0.trial115.stdout:774: left oid 6 (ObjNum 315 snap 75 seq_num 315) 2026-02-17T05:50:23.571 INFO:tasks.rados.rados.0.trial115.stdout:774: done (1 left) 2026-02-17T05:50:23.571 INFO:tasks.rados.rados.0.trial115.stdout:775: done (0 left) 2026-02-17T05:50:23.571 INFO:tasks.rados.rados.0.trial115.stdout:776: write oid 21 current snap is 76 2026-02-17T05:50:23.571 INFO:tasks.rados.rados.0.trial115.stdout:776: seq_num 316 ranges {511012=549157,1775251=550760,3095195=215817} 2026-02-17T05:50:23.575 INFO:tasks.rados.rados.0.trial115.stdout:776: writing trial11545420-21 from 511012 to 1060169 tid 1 2026-02-17T05:50:23.579 INFO:tasks.rados.rados.0.trial115.stdout:776: writing trial11545420-21 from 1775251 to 2326011 tid 2 2026-02-17T05:50:23.583 INFO:tasks.rados.rados.0.trial115.stdout:776: writing trial11545420-21 from 3095195 to 3311012 tid 3 2026-02-17T05:50:23.583 INFO:tasks.rados.rados.0.trial115.stdout:777: delete oid 6 current snap is 76 2026-02-17T05:50:23.583 INFO:tasks.rados.rados.0.trial115.stdout:776: finishing write tid 1 to trial11545420-21 2026-02-17T05:50:23.583 INFO:tasks.rados.rados.0.trial115.stdout:776: finishing write tid 2 to trial11545420-21 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:776: finishing write tid 3 to trial11545420-21 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:776: finishing write tid 4 to trial11545420-21 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:776: finishing write tid 5 to trial11545420-21 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:776: oid 21 updating version 0 to 292 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:776: oid 21 updating version 292 to 294 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:776: oid 21 updating version 294 to 295 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:776: oid 21 version 295 is already newer than 293 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 295 (ObjNum 316 snap 76 seq_num 316) dirty exists 2026-02-17T05:50:23.585 INFO:tasks.rados.rados.0.trial115.stdout:776: left oid 21 (ObjNum 316 snap 76 seq_num 316) 2026-02-17T05:50:23.586 INFO:tasks.rados.rados.0.trial115.stdout:776: done (1 left) 2026-02-17T05:50:23.586 INFO:tasks.rados.rados.0.trial115.stdout:777: done (0 left) 2026-02-17T05:50:23.586 INFO:tasks.rados.rados.0.trial115.stdout:778: rollback oid 27 current snap is 76 2026-02-17T05:50:23.586 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 73 2026-02-17T05:50:23.586 INFO:tasks.rados.rados.0.trial115.stdout:779: delete oid 16 current snap is 76 2026-02-17T05:50:23.588 INFO:tasks.rados.rados.0.trial115.stdout:778: finishing rollback tid 0 to trial11545420-27 2026-02-17T05:50:23.588 INFO:tasks.rados.rados.0.trial115.stdout:779: done (1 left) 2026-02-17T05:50:23.588 INFO:tasks.rados.rados.0.trial115.stdout:780: snap_create 2026-02-17T05:50:23.588 INFO:tasks.rados.rados.0.trial115.stdout:778: finishing rollback tid 1 to trial11545420-27 2026-02-17T05:50:23.588 INFO:tasks.rados.rados.0.trial115.stdout:778: finishing rollback tid 2 to trial11545420-27 2026-02-17T05:50:23.588 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 306 (ObjNum 305 snap 73 seq_num 305) dirty exists 2026-02-17T05:50:24.572 INFO:tasks.rados.rados.0.trial115.stdout:778: done (1 left) 2026-02-17T05:50:24.573 INFO:tasks.rados.rados.0.trial115.stdout:780: done (0 left) 2026-02-17T05:50:24.573 INFO:tasks.rados.rados.0.trial115.stdout:781: write oid 40 current snap is 77 2026-02-17T05:50:24.573 INFO:tasks.rados.rados.0.trial115.stdout:781: seq_num 319 ranges {561433=774904,1950697=414440,2777464=583969} 2026-02-17T05:50:24.578 INFO:tasks.rados.rados.0.trial115.stdout:781: writing trial11545420-40 from 561433 to 1336337 tid 1 2026-02-17T05:50:24.582 INFO:tasks.rados.rados.0.trial115.stdout:781: writing trial11545420-40 from 1950697 to 2365137 tid 2 2026-02-17T05:50:24.586 INFO:tasks.rados.rados.0.trial115.stdout:781: writing trial11545420-40 from 2777464 to 3361433 tid 3 2026-02-17T05:50:24.586 INFO:tasks.rados.rados.0.trial115.stdout:782: write (excl) oid 48 current snap is 77 2026-02-17T05:50:24.586 INFO:tasks.rados.rados.0.trial115.stdout:781: finishing write tid 1 to trial11545420-40 2026-02-17T05:50:24.586 INFO:tasks.rados.rados.0.trial115.stdout:781: finishing write tid 2 to trial11545420-40 2026-02-17T05:50:24.586 INFO:tasks.rados.rados.0.trial115.stdout:782: seq_num 320 ranges {578240=566506,1609179=558098,2804887=551750} 2026-02-17T05:50:24.590 INFO:tasks.rados.rados.0.trial115.stdout:782: writing trial11545420-48 from 578240 to 1144746 tid 1 2026-02-17T05:50:24.594 INFO:tasks.rados.rados.0.trial115.stdout:782: writing trial11545420-48 from 1609179 to 2167277 tid 2 2026-02-17T05:50:24.599 INFO:tasks.rados.rados.0.trial115.stdout:782: writing trial11545420-48 from 2804887 to 3356637 tid 3 2026-02-17T05:50:24.599 INFO:tasks.rados.rados.0.trial115.stdout:783: read oid 30 snap -1 2026-02-17T05:50:24.599 INFO:tasks.rados.rados.0.trial115.stdout:783: expect (ObjNum 298 snap 70 seq_num 298) 2026-02-17T05:50:24.599 INFO:tasks.rados.rados.0.trial115.stdout:784: read oid 8 snap -1 2026-02-17T05:50:24.599 INFO:tasks.rados.rados.0.trial115.stdout:784: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:50:24.599 INFO:tasks.rados.rados.0.trial115.stdout:785: rollback oid 42 current snap is 77 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 58 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: finishing write tid 3 to trial11545420-40 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: finishing write tid 4 to trial11545420-40 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: finishing write tid 5 to trial11545420-40 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: oid 40 updating version 0 to 180 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: oid 40 version 180 is already newer than 177 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: oid 40 version 180 is already newer than 178 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: oid 40 version 180 is already newer than 179 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 180 (ObjNum 319 snap 77 seq_num 319) dirty exists 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: left oid 40 (ObjNum 319 snap 77 seq_num 319) 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:782: finishing write tid 1 to trial11545420-48 2026-02-17T05:50:24.600 INFO:tasks.rados.rados.0.trial115.stdout:781: done (4 left) 2026-02-17T05:50:24.601 INFO:tasks.rados.rados.0.trial115.stdout:786: snap_create 2026-02-17T05:50:24.601 INFO:tasks.rados.rados.0.trial115.stdout:782: finishing write tid 2 to trial11545420-48 2026-02-17T05:50:24.601 INFO:tasks.rados.rados.0.trial115.stdout:782: finishing write tid 3 to trial11545420-48 2026-02-17T05:50:24.602 INFO:tasks.rados.rados.0.trial115.stdout:782: finishing write tid 4 to trial11545420-48 2026-02-17T05:50:24.602 INFO:tasks.rados.rados.0.trial115.stdout:782: finishing write tid 5 to trial11545420-48 2026-02-17T05:50:24.602 INFO:tasks.rados.rados.0.trial115.stdout:782: oid 48 updating version 0 to 330 2026-02-17T05:50:24.602 INFO:tasks.rados.rados.0.trial115.stdout:782: oid 48 version 330 is already newer than 327 2026-02-17T05:50:24.602 INFO:tasks.rados.rados.0.trial115.stdout:782: oid 48 version 330 is already newer than 329 2026-02-17T05:50:24.602 INFO:tasks.rados.rados.0.trial115.stdout:782: oid 48 version 330 is already newer than 328 2026-02-17T05:50:24.602 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 330 (ObjNum 320 snap 77 seq_num 320) dirty exists 2026-02-17T05:50:24.602 INFO:tasks.rados.rados.0.trial115.stdout:782: left oid 48 (ObjNum 320 snap 77 seq_num 320) 2026-02-17T05:50:24.603 INFO:tasks.rados.rados.0.trial115.stdout:784: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:50:24.655 INFO:tasks.rados.rados.0.trial115.stdout:785: finishing rollback tid 0 to trial11545420-42 2026-02-17T05:50:24.655 INFO:tasks.rados.rados.0.trial115.stdout:785: finishing rollback tid 1 to trial11545420-42 2026-02-17T05:50:24.655 INFO:tasks.rados.rados.0.trial115.stdout:785: finishing rollback tid 2 to trial11545420-42 2026-02-17T05:50:24.655 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 350 (ObjNum 261 snap 52 seq_num 261) dirty exists 2026-02-17T05:50:24.655 INFO:tasks.rados.rados.0.trial115.stdout:783: expect (ObjNum 298 snap 70 seq_num 298) 2026-02-17T05:50:25.541 INFO:tasks.rados.rados.0.trial115.stdout:782: done (4 left) 2026-02-17T05:50:25.541 INFO:tasks.rados.rados.0.trial115.stdout:783: done (3 left) 2026-02-17T05:50:25.541 INFO:tasks.rados.rados.0.trial115.stdout:784: done (2 left) 2026-02-17T05:50:25.541 INFO:tasks.rados.rados.0.trial115.stdout:785: done (1 left) 2026-02-17T05:50:25.541 INFO:tasks.rados.rados.0.trial115.stdout:786: done (0 left) 2026-02-17T05:50:25.541 INFO:tasks.rados.rados.0.trial115.stdout:787: read oid 27 snap -1 2026-02-17T05:50:25.541 INFO:tasks.rados.rados.0.trial115.stdout:787: expect (ObjNum 305 snap 73 seq_num 305) 2026-02-17T05:50:25.541 INFO:tasks.rados.rados.0.trial115.stdout:788: delete oid 15 current snap is 78 2026-02-17T05:50:25.545 INFO:tasks.rados.rados.0.trial115.stdout:788: done (1 left) 2026-02-17T05:50:25.545 INFO:tasks.rados.rados.0.trial115.stdout:789: copy_from oid 2 from oid 34 current snap is 78 2026-02-17T05:50:25.545 INFO:tasks.rados.rados.0.trial115.stdout:790: read oid 29 snap 75 2026-02-17T05:50:25.545 INFO:tasks.rados.rados.0.trial115.stdout:790: expect deleted 2026-02-17T05:50:25.545 INFO:tasks.rados.rados.0.trial115.stdout:791: write (excl) oid 50 current snap is 78 2026-02-17T05:50:25.545 INFO:tasks.rados.rados.0.trial115.stdout:791: seq_num 322 ranges {611854=717004,2126143=561767,3259733=152121} 2026-02-17T05:50:25.550 INFO:tasks.rados.rados.0.trial115.stdout:791: writing trial11545420-50 from 611854 to 1328858 tid 1 2026-02-17T05:50:25.555 INFO:tasks.rados.rados.0.trial115.stdout:791: writing trial11545420-50 from 2126143 to 2687910 tid 2 2026-02-17T05:50:25.558 INFO:tasks.rados.rados.0.trial115.stdout:791: writing trial11545420-50 from 3259733 to 3411854 tid 3 2026-02-17T05:50:25.558 INFO:tasks.rados.rados.0.trial115.stdout:792: rollback oid 3 current snap is 78 2026-02-17T05:50:25.558 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 3 to 68 2026-02-17T05:50:25.558 INFO:tasks.rados.rados.0.trial115.stdout:787: expect (ObjNum 305 snap 73 seq_num 305) 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:789: finishing copy_from to trial11545420-2 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:789: got expected ENOENT (src dne) 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:789: finishing copy_from racing read to trial11545420-2 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:791: finishing write tid 1 to trial11545420-50 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:791: finishing write tid 2 to trial11545420-50 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:787: done (4 left) 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:789: done (3 left) 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:790: done (2 left) 2026-02-17T05:50:25.688 INFO:tasks.rados.rados.0.trial115.stdout:793: write oid 23 current snap is 78 2026-02-17T05:50:25.689 INFO:tasks.rados.rados.0.trial115.stdout:793: seq_num 323 ranges {628661=792253,2184625=421778,3403509=25152} 2026-02-17T05:50:25.694 INFO:tasks.rados.rados.0.trial115.stdout:793: writing trial11545420-23 from 628661 to 1420914 tid 1 2026-02-17T05:50:25.698 INFO:tasks.rados.rados.0.trial115.stdout:793: writing trial11545420-23 from 2184625 to 2606403 tid 2 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:793: writing trial11545420-23 from 3403509 to 3428661 tid 3 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:794: copy_from oid 29 from oid 36 current snap is 78 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: finishing write tid 3 to trial11545420-50 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: finishing write tid 4 to trial11545420-50 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: finishing write tid 5 to trial11545420-50 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: oid 50 updating version 0 to 309 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: oid 50 updating version 309 to 310 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: oid 50 version 310 is already newer than 307 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: oid 50 version 310 is already newer than 308 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 310 (ObjNum 322 snap 78 seq_num 322) dirty exists 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: left oid 50 (ObjNum 322 snap 78 seq_num 322) 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:792: finishing rollback tid 0 to trial11545420-3 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:792: finishing rollback tid 1 to trial11545420-3 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:791: done (3 left) 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:795: delete oid 14 current snap is 78 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:792: finishing rollback tid 2 to trial11545420-3 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 269 (ObjNum 196 snap 37 seq_num 196) dirty exists 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:793: finishing write tid 1 to trial11545420-23 2026-02-17T05:50:25.702 INFO:tasks.rados.rados.0.trial115.stdout:793: finishing write tid 2 to trial11545420-23 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:793: finishing write tid 3 to trial11545420-23 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:793: finishing write tid 4 to trial11545420-23 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:793: finishing write tid 5 to trial11545420-23 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:793: oid 23 updating version 0 to 99 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:793: oid 23 updating version 99 to 100 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:793: oid 23 updating version 100 to 101 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:793: oid 23 version 101 is already newer than 98 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 101 (ObjNum 323 snap 78 seq_num 323) dirty exists 2026-02-17T05:50:25.703 INFO:tasks.rados.rados.0.trial115.stdout:793: left oid 23 (ObjNum 323 snap 78 seq_num 323) 2026-02-17T05:50:25.704 INFO:tasks.rados.rados.0.trial115.stdout:792: done (3 left) 2026-02-17T05:50:25.704 INFO:tasks.rados.rados.0.trial115.stdout:793: done (2 left) 2026-02-17T05:50:25.704 INFO:tasks.rados.rados.0.trial115.stdout:795: done (1 left) 2026-02-17T05:50:25.704 INFO:tasks.rados.rados.0.trial115.stdout:796: copy_from oid 13 from oid 30 current snap is 78 2026-02-17T05:50:25.704 INFO:tasks.rados.rados.0.trial115.stdout:797: copy_from oid 27 from oid 18 current snap is 78 2026-02-17T05:50:25.704 INFO:tasks.rados.rados.0.trial115.stdout:798: delete oid 37 current snap is 78 2026-02-17T05:50:25.705 INFO:tasks.rados.rados.0.trial115.stdout:798: done (3 left) 2026-02-17T05:50:25.705 INFO:tasks.rados.rados.0.trial115.stdout:799: rollback oid 24 current snap is 78 2026-02-17T05:50:25.705 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 76 2026-02-17T05:50:25.706 INFO:tasks.rados.rados.0.trial115.stdout:800: write oid 35 current snap is 78 2026-02-17T05:50:25.706 INFO:tasks.rados.rados.0.trial115.stdout:800: seq_num 326 ranges {679082=618000,1843718=569105,2853072=626010} 2026-02-17T05:50:25.710 INFO:tasks.rados.rados.0.trial115.stdout:800: writing trial11545420-35 from 679082 to 1297082 tid 1 2026-02-17T05:50:25.715 INFO:tasks.rados.rados.0.trial115.stdout:800: writing trial11545420-35 from 1843718 to 2412823 tid 2 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:800: writing trial11545420-35 from 2853072 to 3479082 tid 3 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:801: copy_from oid 46 from oid 38 current snap is 78 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:799: finishing rollback tid 1 to trial11545420-24 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 298 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:794: finishing copy_from racing read to trial11545420-29 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:794: finishing copy_from to trial11545420-29 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 175 (ObjNum 254 snap 50 seq_num 254) dirty exists 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:797: finishing copy_from racing read to trial11545420-27 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:797: finishing copy_from to trial11545420-27 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 291 (ObjNum 267 snap 54 seq_num 267) dirty exists 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:800: finishing write tid 1 to trial11545420-35 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:796: finishing copy_from racing read to trial11545420-13 2026-02-17T05:50:25.719 INFO:tasks.rados.rados.0.trial115.stdout:794: done (5 left) 2026-02-17T05:50:25.720 INFO:tasks.rados.rados.0.trial115.stdout:797: done (4 left) 2026-02-17T05:50:25.720 INFO:tasks.rados.rados.0.trial115.stdout:799: done (3 left) 2026-02-17T05:50:25.720 INFO:tasks.rados.rados.0.trial115.stdout:802: snap_create 2026-02-17T05:50:25.720 INFO:tasks.rados.rados.0.trial115.stdout:796: finishing copy_from to trial11545420-13 2026-02-17T05:50:25.720 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 299 (ObjNum 298 snap 70 seq_num 298) dirty exists 2026-02-17T05:50:25.720 INFO:tasks.rados.rados.0.trial115.stdout:800: finishing write tid 2 to trial11545420-35 2026-02-17T05:50:25.722 INFO:tasks.rados.rados.0.trial115.stdout:800: finishing write tid 3 to trial11545420-35 2026-02-17T05:50:25.722 INFO:tasks.rados.rados.0.trial115.stdout:800: finishing write tid 4 to trial11545420-35 2026-02-17T05:50:25.722 INFO:tasks.rados.rados.0.trial115.stdout:800: finishing write tid 5 to trial11545420-35 2026-02-17T05:50:25.722 INFO:tasks.rados.rados.0.trial115.stdout:800: oid 35 updating version 0 to 331 2026-02-17T05:50:25.722 INFO:tasks.rados.rados.0.trial115.stdout:800: oid 35 updating version 331 to 332 2026-02-17T05:50:25.723 INFO:tasks.rados.rados.0.trial115.stdout:800: oid 35 updating version 332 to 334 2026-02-17T05:50:25.723 INFO:tasks.rados.rados.0.trial115.stdout:800: oid 35 version 334 is already newer than 333 2026-02-17T05:50:25.723 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 334 (ObjNum 326 snap 78 seq_num 326) dirty exists 2026-02-17T05:50:25.723 INFO:tasks.rados.rados.0.trial115.stdout:800: left oid 35 (ObjNum 326 snap 78 seq_num 326) 2026-02-17T05:50:25.724 INFO:tasks.rados.rados.0.trial115.stdout:801: finishing copy_from racing read to trial11545420-46 2026-02-17T05:50:25.724 INFO:tasks.rados.rados.0.trial115.stdout:801: finishing copy_from to trial11545420-46 2026-02-17T05:50:25.725 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 325 (ObjNum 135 snap 23 seq_num 135) dirty exists 2026-02-17T05:50:26.579 INFO:tasks.rados.rados.0.trial115.stdout:796: done (3 left) 2026-02-17T05:50:26.579 INFO:tasks.rados.rados.0.trial115.stdout:800: done (2 left) 2026-02-17T05:50:26.580 INFO:tasks.rados.rados.0.trial115.stdout:801: done (1 left) 2026-02-17T05:50:26.580 INFO:tasks.rados.rados.0.trial115.stdout:802: done (0 left) 2026-02-17T05:50:26.580 INFO:tasks.rados.rados.0.trial115.stdout:803: snap_create 2026-02-17T05:50:27.579 INFO:tasks.rados.rados.0.trial115.stdout:803: done (0 left) 2026-02-17T05:50:27.579 INFO:tasks.rados.rados.0.trial115.stdout:804: write (excl) oid 25 current snap is 80 2026-02-17T05:50:27.579 INFO:tasks.rados.rados.0.trial115.stdout:804: seq_num 327 ranges {695889=409602,1618553=712763,2996848=499041} 2026-02-17T05:50:27.584 INFO:tasks.rados.rados.0.trial115.stdout:804: writing trial11545420-25 from 695889 to 1105491 tid 1 2026-02-17T05:50:27.589 INFO:tasks.rados.rados.0.trial115.stdout:804: writing trial11545420-25 from 1618553 to 2331316 tid 2 2026-02-17T05:50:27.593 INFO:tasks.rados.rados.0.trial115.stdout:804: writing trial11545420-25 from 2996848 to 3495889 tid 3 2026-02-17T05:50:27.593 INFO:tasks.rados.rados.0.trial115.stdout:805: rollback oid 1 current snap is 80 2026-02-17T05:50:27.593 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 1 to 77 2026-02-17T05:50:27.593 INFO:tasks.rados.rados.0.trial115.stdout:804: finishing write tid 1 to trial11545420-25 2026-02-17T05:50:27.594 INFO:tasks.rados.rados.0.trial115.stdout:804: finishing write tid 2 to trial11545420-25 2026-02-17T05:50:27.594 INFO:tasks.rados.rados.0.trial115.stdout:806: snap_remove snap 74 2026-02-17T05:50:28.042 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T05:50:28.043 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-17T05:50:28.043 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-17T05:50:28.043 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-17T05:50:28.046 INFO:tasks.ceph.mon.c:Started 2026-02-17T05:50:28.047 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:50:28.047 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.115+0000 7f8495f85640 10 monclient: get_monmap_and_config 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.115+0000 7f8495f85640 10 monclient: build_initial_monmap 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.115+0000 7f8495f85640 10 monclient: monmap: 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:28.116913+0000 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:28.116913+0000 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:28.115 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 1 Processor -- start 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 1 -- start start 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient: init 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f849006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f849009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f849006d380 0x7f849013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient(hunting): picked mon.noname-c con 0x7f849006d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f849006daf0 con 0x7f849006c890 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f849009cc80 con 0x7f849009c4a0 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f84900a0880 con 0x7f849006d380 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f8495f85640 10 monclient(hunting): authenticate will time out at 873.670535s 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f848f7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f849006c890 0x7f849006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57626/0 (socket says 10.20.193.78:57626) 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f848f7fe640 1 -- 10.20.193.78:0/2213249628 learned_addr learned my addr 10.20.193.78:0/2213249628 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:50:28.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f848ffff640 1 --2- 10.20.193.78:0/2213249628 >> v2:10.20.193.115:3300/0 conn(0x7f849006d380 0x7f849013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f848ffff640 10 monclient(hunting): get_auth_request con 0x7f849006d380 auth_method 0 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f848ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f848ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.116+0000 7f848ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 10 monclient(hunting): handle_auth_done global_id 44308 payload 274 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 1 -- 10.20.193.78:0/2213249628 >> v1:10.20.193.115:6789/0 conn(0x7f849009c4a0 legacy=0x7f849013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 1 -- 10.20.193.78:0/2213249628 >> v1:10.20.193.78:6789/0 conn(0x7f849006c890 legacy=0x7f849006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 1 monclient: found mon.noname-c 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:50:28.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.117+0000 7f848ffff640 1 -- 10.20.193.78:0/2213249628 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849009cf60 con 0x7f849006d380 2026-02-17T05:50:28.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848ffff640 1 -- 10.20.193.78:0/2213249628 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 msgr2=0x7f849013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:28.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848ffff640 1 --2- 10.20.193.78:0/2213249628 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 0x7f849013ead0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8490074de0 tx=0x7f848402f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d749188722b19957 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:28.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 1 -- 10.20.193.78:0/2213249628 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848403c070 con 0x7f849006d380 2026-02-17T05:50:28.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: dump: 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 5 monclient: authenticate success, global_id 44308 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 1 -- 10.20.193.78:0/2213249628 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8484037640 con 0x7f849006d380 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 1 -- 10.20.193.78:0/2213249628 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484037a10 con 0x7f849006d380 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f848e7fc640 10 monclient: dump: 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:28.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 4 set_mon_vals no callback set 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 10 monclient: shutdown 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 1 -- 10.20.193.78:0/2213249628 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 msgr2=0x7f849013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 1 --2- 10.20.193.78:0/2213249628 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 0x7f849013ead0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8490074de0 tx=0x7f848402f130 comp rx=0 tx=0).stop 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 1 -- 10.20.193.78:0/2213249628 shutdown_connections 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 1 --2- 10.20.193.78:0/2213249628 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 0x7f849013ead0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 1 -- 10.20.193.78:0/2213249628 >> 10.20.193.78:0/2213249628 conn(0x7f84900a6620 msgr2=0x7f84900a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 1 -- 10.20.193.78:0/2213249628 shutdown_connections 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.120+0000 7f8495f85640 1 -- 10.20.193.78:0/2213249628 wait complete. 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient: build_initial_monmap 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient: monmap: 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:28.122 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 1 Processor -- start 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 1 -- start start 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient: init 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 0x7f84901cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient(hunting): picked mon.noname-c con 0x7f849006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 0x7f8490136560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient(hunting): picked mon.noname-b con 0x7f849006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f849009c4a0 0x7f849011dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient(hunting): picked mon.noname-a con 0x7f849009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f849009bfa0 con 0x7f849009c4a0 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f849009be20 con 0x7f849006d380 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f849013f010 con 0x7f849006c890 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f8495f85640 10 monclient(hunting): authenticate will time out at 873.675649s 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 0x7f84901cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f849009c4a0 0x7f849011dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f849009c4a0 0x7f849011dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42396/0 (socket says 10.20.193.78:42396) 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 0x7f84901cf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44654/0 (socket says 10.20.193.78:44654) 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 -- 10.20.193.78:0/1566257538 learned_addr learned my addr 10.20.193.78:0/1566257538 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient(hunting): get_auth_request con 0x7f849009c4a0 auth_method 0 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 10 monclient(hunting): get_auth_request con 0x7f849006c890 auth_method 0 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:28.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848effd640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 0x7f8490136560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 1 -- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 msgr2=0x7f84901cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 1 -- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 msgr2=0x7f84901cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 0x7f84901cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848effd640 10 monclient(hunting): get_auth_request con 0x7f849006d380 auth_method 0 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848f7fe640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 0x7f84901cf240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient(hunting): handle_auth_done global_id 54294 payload 824 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 -- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 msgr2=0x7f84901cf240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 0x7f84901cf240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 -- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 msgr2=0x7f8490136560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 0x7f8490136560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 monclient: found mon.noname-a 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848ffff640 1 -- 10.20.193.78:0/1566257538 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8490121c20 con 0x7f849009c4a0 2026-02-17T05:50:28.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.121+0000 7f848effd640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 0x7f8490136560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f848ffff640 1 -- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f849009c4a0 msgr2=0x7f849011dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f848ffff640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f849009c4a0 0x7f849011dfc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8484009e90 tx=0x7f8484004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 1 -- 10.20.193.78:0/1566257538 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484047020 con 0x7f849009c4a0 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: dump: 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: _finish_auth 0 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 1 -- 10.20.193.78:0/1566257538 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8484042880 con 0x7f849009c4a0 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 5 monclient: authenticate success, global_id 54294 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 10 monclient: _renew_subs 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84901204f0 con 0x7f849009c4a0 2026-02-17T05:50:28.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 1 -- 10.20.193.78:0/1566257538 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848403c040 con 0x7f849009c4a0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f848dffb640 4 set_mon_vals no callback set 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 10 monclient: dump: 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 10 monclient: _renew_subs 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849013e4f0 con 0x7f849009c4a0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 1 -- 10.20.193.78:0/1566257538 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8484042cb0 con 0x7f849009c4a0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849006cc70 con 0x7f849009c4a0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846003deb0 0x7f8460040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.124+0000 7f846bfff640 1 -- 10.20.193.78:0/1566257538 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(172..172 src has 117..172) ==== 4707+0+0 (secure 0 0 0) 0x7f8484078700 con 0x7f849009c4a0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.125+0000 7f848f7fe640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846003deb0 0x7f8460040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:28.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.125+0000 7f848f7fe640 10 monclient: get_auth_request con 0x7f846003deb0 auth_method 0 2026-02-17T05:50:28.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.127+0000 7f848f7fe640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846003deb0 0x7f8460040370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f847c007fd0 tx=0x7f847c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:28.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.127+0000 7f846bfff640 1 -- 10.20.193.78:0/1566257538 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8484051070 con 0x7f849009c4a0 2026-02-17T05:50:28.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.127+0000 7f846bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:28.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.127+0000 7f846bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:28.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.244+0000 7f8495f85640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:28.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.244+0000 7f8495f85640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:28.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.244+0000 7f8495f85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:28.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.244+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f84900744d0 con 0x7f849009c4a0 2026-02-17T05:50:28.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.244+0000 7f846bfff640 1 -- 10.20.193.78:0/1566257538 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f8484036e20 con 0x7f849009c4a0 2026-02-17T05:50:28.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.244+0000 7f846bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:28.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.244+0000 7f846bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:28.244 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:28.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.245+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846003deb0 msgr2=0x7f8460040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.245+0000 7f8495f85640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846003deb0 0x7f8460040370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f847c007fd0 tx=0x7f847c009040 comp rx=0 tx=0).stop 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.245+0000 7f8495f85640 10 monclient: shutdown 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.245+0000 7f8495f85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.245+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f849009c4a0 msgr2=0x7f849011dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.245+0000 7f8495f85640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f849009c4a0 0x7f849011dfc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8484009e90 tx=0x7f8484004b40 comp rx=0 tx=0).stop 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.246+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 shutdown_connections 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.246+0000 7f8495f85640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f849009c4a0 0x7f849011dfc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.246+0000 7f8495f85640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846003deb0 0x7f8460040370 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.246+0000 7f8495f85640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f849006d380 0x7f8490136560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.246+0000 7f8495f85640 1 --2- 10.20.193.78:0/1566257538 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f849006c890 0x7f84901cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.246+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 >> 10.20.193.78:0/1566257538 conn(0x7f84900a6620 msgr2=0x7f84900a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.246+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 shutdown_connections 2026-02-17T05:50:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:28.246+0000 7f8495f85640 1 -- 10.20.193.78:0/1566257538 wait complete. 2026-02-17T05:50:28.585 INFO:tasks.rados.rados.0.trial115.stdout:806: done (2 left) 2026-02-17T05:50:28.585 INFO:tasks.rados.rados.0.trial115.stdout:807: delete oid 17 current snap is 80 2026-02-17T05:50:28.585 INFO:tasks.rados.rados.0.trial115.stdout:804: finishing write tid 3 to trial11545420-25 2026-02-17T05:50:28.585 INFO:tasks.rados.rados.0.trial115.stdout:805: finishing rollback tid 0 to trial11545420-1 2026-02-17T05:50:28.585 INFO:tasks.rados.rados.0.trial115.stdout:804: finishing write tid 4 to trial11545420-25 2026-02-17T05:50:28.585 INFO:tasks.rados.rados.0.trial115.stdout:804: finishing write tid 5 to trial11545420-25 2026-02-17T05:50:28.585 INFO:tasks.rados.rados.0.trial115.stdout:804: oid 25 updating version 0 to 296 2026-02-17T05:50:28.585 INFO:tasks.rados.rados.0.trial115.stdout:804: oid 25 updating version 296 to 297 2026-02-17T05:50:28.586 INFO:tasks.rados.rados.0.trial115.stdout:804: oid 25 updating version 297 to 298 2026-02-17T05:50:28.586 INFO:tasks.rados.rados.0.trial115.stdout:804: oid 25 updating version 298 to 299 2026-02-17T05:50:28.586 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 299 (ObjNum 327 snap 80 seq_num 327) dirty exists 2026-02-17T05:50:28.586 INFO:tasks.rados.rados.0.trial115.stdout:804: left oid 25 (ObjNum 327 snap 80 seq_num 327) 2026-02-17T05:50:28.586 INFO:tasks.rados.rados.0.trial115.stdout:805: finishing rollback tid 1 to trial11545420-1 2026-02-17T05:50:28.586 INFO:tasks.rados.rados.0.trial115.stdout:805: finishing rollback tid 2 to trial11545420-1 2026-02-17T05:50:28.586 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 295 (ObjNum 251 snap 49 seq_num 251) dirty exists 2026-02-17T05:50:28.588 INFO:tasks.rados.rados.0.trial115.stdout:804: done (2 left) 2026-02-17T05:50:28.589 INFO:tasks.rados.rados.0.trial115.stdout:805: done (1 left) 2026-02-17T05:50:28.589 INFO:tasks.rados.rados.0.trial115.stdout:807: done (0 left) 2026-02-17T05:50:28.589 INFO:tasks.rados.rados.0.trial115.stdout:808: read oid 35 snap -1 2026-02-17T05:50:28.589 INFO:tasks.rados.rados.0.trial115.stdout:808: expect (ObjNum 326 snap 78 seq_num 326) 2026-02-17T05:50:28.589 INFO:tasks.rados.rados.0.trial115.stdout:809: snap_remove snap 76 2026-02-17T05:50:29.585 INFO:tasks.rados.rados.0.trial115.stdout:809: done (1 left) 2026-02-17T05:50:29.586 INFO:tasks.rados.rados.0.trial115.stdout:810: snap_remove snap 71 2026-02-17T05:50:30.540 INFO:tasks.rados.rados.0.trial115.stdout:810: done (1 left) 2026-02-17T05:50:30.541 INFO:tasks.rados.rados.0.trial115.stdout:811: snap_create 2026-02-17T05:50:30.541 INFO:tasks.rados.rados.0.trial115.stdout:808: expect (ObjNum 326 snap 78 seq_num 326) 2026-02-17T05:50:31.254 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:31.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.322+0000 7f46972e2640 10 monclient: get_monmap_and_config 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.322+0000 7f46972e2640 10 monclient: build_initial_monmap 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.322+0000 7f46972e2640 10 monclient: monmap: 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:31.324163+0000 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:31.324163+0000 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 1 Processor -- start 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 1 -- start start 2026-02-17T05:50:31.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient: init 2026-02-17T05:50:31.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:31.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2026-02-17T05:50:31.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f469006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f469009c4a0 0x7f469013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient(hunting): picked mon.noname-c con 0x7f469009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 0x7f469013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient(hunting): picked mon.noname-b con 0x7f469006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469006daf0 con 0x7f469006c890 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f46900a0880 con 0x7f469009c4a0 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f469009cc80 con 0x7f469006d380 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f46972e2640 10 monclient(hunting): authenticate will time out at 876.877675s 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 0x7f469013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 0x7f469013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44670/0 (socket says 10.20.193.78:44670) 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 1 -- 10.20.193.78:0/4171478199 learned_addr learned my addr 10.20.193.78:0/4171478199 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 10 monclient(hunting): get_auth_request con 0x7f469006d380 auth_method 0 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 1 -- 10.20.193.78:0/4171478199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 msgr2=0x7f469013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 1 -- 10.20.193.78:0/4171478199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 msgr2=0x7f469013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 1 --2- 10.20.193.78:0/4171478199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 0x7f469013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4695858640 1 --2- 10.20.193.78:0/4171478199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 0x7f469013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.323+0000 7f4694856640 1 --2- 10.20.193.78:0/4171478199 >> v2:10.20.193.115:3300/0 conn(0x7f469009c4a0 0x7f469013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient(hunting): get_auth_request con 0x7f469009c4a0 auth_method 0 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:31.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient(hunting): handle_auth_done global_id 44314 payload 274 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 1 -- 10.20.193.78:0/4171478199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 msgr2=0x7f469013d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 1 --2- 10.20.193.78:0/4171478199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 0x7f469013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 1 -- 10.20.193.78:0/4171478199 >> v1:10.20.193.115:6789/0 conn(0x7f469006c890 legacy=0x7f469006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient: _finish_hunting 0 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 1 monclient: found mon.noname-c 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.324+0000 7f4694856640 1 -- 10.20.193.78:0/4171478199 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469009cf60 con 0x7f469009c4a0 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f4694856640 1 -- 10.20.193.78:0/4171478199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469009c4a0 msgr2=0x7f469013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:31.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f4694856640 1 --2- 10.20.193.78:0/4171478199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469009c4a0 0x7f469013b3a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46840098a0 tx=0x7f468402ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7998b4dd4ab36580 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 1 -- 10.20.193.78:0/4171478199 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468403c070 con 0x7f469009c4a0 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: dump: 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: _finish_auth 0 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 5 monclient: authenticate success, global_id 44314 2026-02-17T05:50:31.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 1 -- 10.20.193.78:0/4171478199 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f468402fc00 con 0x7f469009c4a0 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 1 -- 10.20.193.78:0/4171478199 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684037440 con 0x7f469009c4a0 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 4 set_mon_vals no callback set 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f467ffff640 10 monclient: dump: 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 10 monclient: shutdown 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 1 -- 10.20.193.78:0/4171478199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469009c4a0 msgr2=0x7f469013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 1 --2- 10.20.193.78:0/4171478199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469009c4a0 0x7f469013b3a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46840098a0 tx=0x7f468402ed80 comp rx=0 tx=0).stop 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 1 -- 10.20.193.78:0/4171478199 shutdown_connections 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 1 --2- 10.20.193.78:0/4171478199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006d380 0x7f469013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 1 --2- 10.20.193.78:0/4171478199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469009c4a0 0x7f469013b3a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 1 -- 10.20.193.78:0/4171478199 >> 10.20.193.78:0/4171478199 conn(0x7f46900a6620 msgr2=0x7f46900a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 1 -- 10.20.193.78:0/4171478199 shutdown_connections 2026-02-17T05:50:31.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.327+0000 7f46972e2640 1 -- 10.20.193.78:0/4171478199 wait complete. 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient: build_initial_monmap 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient: monmap: 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 1 Processor -- start 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 1 -- start start 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient: init 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 0x7f46901cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient(hunting): picked mon.noname-c con 0x7f469006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469006d380 0x7f4690130360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient(hunting): picked mon.noname-b con 0x7f469006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f469009c4a0 0x7f46901308a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient(hunting): picked mon.noname-a con 0x7f469009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:31.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f469009bfa0 con 0x7f469009c4a0 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f469009be20 con 0x7f469006d380 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f469013de30 con 0x7f469006c890 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f46972e2640 10 monclient(hunting): authenticate will time out at 876.882731s 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 0x7f46901cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f469009c4a0 0x7f46901308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 0x7f46901cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44678/0 (socket says 10.20.193.78:44678) 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 1 -- 10.20.193.78:0/3259210312 learned_addr learned my addr 10.20.193.78:0/3259210312 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 10 monclient(hunting): get_auth_request con 0x7f469006c890 auth_method 0 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient(hunting): get_auth_request con 0x7f469009c4a0 auth_method 0 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 1 -- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 msgr2=0x7f46901cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 1 -- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 msgr2=0x7f46901cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 0x7f46901cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695057640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 0x7f46901cf170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4694856640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469006d380 0x7f4690130360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4694856640 10 monclient(hunting): get_auth_request con 0x7f469006d380 auth_method 0 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4694856640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4694856640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:31.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4694856640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient(hunting): handle_auth_done global_id 54297 payload 824 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 1 -- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 msgr2=0x7f46901cf170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 0x7f46901cf170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 1 -- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469006d380 msgr2=0x7f4690130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469006d380 0x7f4690130360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 10 monclient: _finish_hunting 0 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 1 monclient: found mon.noname-a 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.328+0000 7f4695858640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.329+0000 7f4695858640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.329+0000 7f4695858640 1 -- 10.20.193.78:0/3259210312 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469009f580 con 0x7f469009c4a0 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f4695858640 1 -- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f469009c4a0 msgr2=0x7f46901308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f4695858640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f469009c4a0 0x7f46901308a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f468c009fd0 tx=0x7f468c007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 1 -- 10.20.193.78:0/3259210312 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c01a070 con 0x7f469009c4a0 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: dump: 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: _finish_auth 0 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 1 -- 10.20.193.78:0/3259210312 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f468c00ddb0 con 0x7f469009c4a0 2026-02-17T05:50:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 1 -- 10.20.193.78:0/3259210312 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c015640 con 0x7f469009c4a0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 5 monclient: authenticate success, global_id 54297 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 10 monclient: _renew_subs 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467f7fe640 4 set_mon_vals no callback set 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469009d810 con 0x7f469009c4a0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f467dffb640 10 monclient: dump: 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 10 monclient: _renew_subs 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469011cd10 con 0x7f469009c4a0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.331+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4690074980 con 0x7f469009c4a0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.332+0000 7f467dffb640 1 -- 10.20.193.78:0/3259210312 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f468c015820 con 0x7f469009c4a0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.332+0000 7f467dffb640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f466003deb0 0x7f4660040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.332+0000 7f467dffb640 1 -- 10.20.193.78:0/3259210312 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(175..175 src has 117..175) ==== 4795+0+0 (secure 0 0 0) 0x7f468c052640 con 0x7f469009c4a0 2026-02-17T05:50:31.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.332+0000 7f4695057640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f466003deb0 0x7f4660040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:31.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.332+0000 7f4695057640 10 monclient: get_auth_request con 0x7f466003deb0 auth_method 0 2026-02-17T05:50:31.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.334+0000 7f467dffb640 1 -- 10.20.193.78:0/3259210312 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f469009d810 con 0x7f469009c4a0 2026-02-17T05:50:31.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.334+0000 7f467dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:31.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.334+0000 7f467dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:31.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.335+0000 7f4695057640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f466003deb0 0x7f4660040370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4680007e10 tx=0x7f4680009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.449+0000 7f46972e2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.449+0000 7f46972e2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.449+0000 7f46972e2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:31.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.449+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f469006cc70 con 0x7f469009c4a0 2026-02-17T05:50:31.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f467dffb640 1 -- 10.20.193.78:0/3259210312 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f468c010070 con 0x7f469009c4a0 2026-02-17T05:50:31.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f467dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:31.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f467dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:31.451 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:31.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f466003deb0 msgr2=0x7f4660040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f466003deb0 0x7f4660040370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4680007e10 tx=0x7f4680009040 comp rx=0 tx=0).stop 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 10 monclient: shutdown 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f469009c4a0 msgr2=0x7f46901308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f469009c4a0 0x7f46901308a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f468c009fd0 tx=0x7f468c007f00 comp rx=0 tx=0).stop 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 shutdown_connections 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f469009c4a0 0x7f46901308a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f466003deb0 0x7f4660040370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f469006d380 0x7f4690130360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.450+0000 7f46972e2640 1 --2- 10.20.193.78:0/3259210312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f469006c890 0x7f46901cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.451+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 >> 10.20.193.78:0/3259210312 conn(0x7f46900a6620 msgr2=0x7f46900a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.451+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 shutdown_connections 2026-02-17T05:50:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:31.451+0000 7f46972e2640 1 -- 10.20.193.78:0/3259210312 wait complete. 2026-02-17T05:50:31.543 INFO:tasks.rados.rados.0.trial115.stdout:808: done (1 left) 2026-02-17T05:50:31.543 INFO:tasks.rados.rados.0.trial115.stdout:811: done (0 left) 2026-02-17T05:50:31.543 INFO:tasks.rados.rados.0.trial115.stdout:812: delete oid 18 current snap is 81 2026-02-17T05:50:31.545 INFO:tasks.rados.rados.0.trial115.stdout:812: done (0 left) 2026-02-17T05:50:31.546 INFO:tasks.rados.rados.0.trial115.stdout:813: copy_from oid 20 from oid 4 current snap is 81 2026-02-17T05:50:31.546 INFO:tasks.rados.rados.0.trial115.stdout:814: snap_create 2026-02-17T05:50:31.558 INFO:tasks.rados.rados.0.trial115.stdout:813: finishing copy_from racing read to trial11545420-20 2026-02-17T05:50:31.561 INFO:tasks.rados.rados.0.trial115.stdout:813: finishing copy_from to trial11545420-20 2026-02-17T05:50:31.561 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 176 (ObjNum 314 snap 74 seq_num 314) dirty exists 2026-02-17T05:50:32.544 INFO:tasks.rados.rados.0.trial115.stdout:813: done (1 left) 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:814: done (0 left) 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:815: copy_from oid 42 from oid 27 current snap is 82 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:816: read oid 41 snap -1 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:816: expect (ObjNum 307 snap 73 seq_num 307) 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:817: copy_from oid 15 from oid 12 current snap is 82 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:818: read oid 47 snap -1 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:818: expect (ObjNum 251 snap 49 seq_num 251) 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:819: read oid 29 snap -1 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:819: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:50:32.545 INFO:tasks.rados.rados.0.trial115.stdout:820: rollback oid 20 current snap is 82 2026-02-17T05:50:32.546 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 81 2026-02-17T05:50:32.546 INFO:tasks.rados.rados.0.trial115.stdout:821: delete oid 33 current snap is 82 2026-02-17T05:50:32.546 INFO:tasks.rados.rados.0.trial115.stdout:821: done (6 left) 2026-02-17T05:50:32.546 INFO:tasks.rados.rados.0.trial115.stdout:822: delete oid 32 current snap is 82 2026-02-17T05:50:32.548 INFO:tasks.rados.rados.0.trial115.stdout:822: done (6 left) 2026-02-17T05:50:32.548 INFO:tasks.rados.rados.0.trial115.stdout:823: copy_from oid 33 from oid 13 current snap is 82 2026-02-17T05:50:32.548 INFO:tasks.rados.rados.0.trial115.stdout:824: read oid 11 snap -1 2026-02-17T05:50:32.548 INFO:tasks.rados.rados.0.trial115.stdout:824: expect (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:50:32.548 INFO:tasks.rados.rados.0.trial115.stdout:825: rollback oid 31 current snap is 82 2026-02-17T05:50:32.549 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 31 to 58 2026-02-17T05:50:32.549 INFO:tasks.rados.rados.0.trial115.stdout:826: snap_create 2026-02-17T05:50:32.549 INFO:tasks.rados.rados.0.trial115.stdout:820: finishing rollback tid 0 to trial11545420-20 2026-02-17T05:50:32.550 INFO:tasks.rados.rados.0.trial115.stdout:820: finishing rollback tid 1 to trial11545420-20 2026-02-17T05:50:32.550 INFO:tasks.rados.rados.0.trial115.stdout:820: finishing rollback tid 2 to trial11545420-20 2026-02-17T05:50:32.550 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 302 (ObjNum 314 snap 74 seq_num 314) dirty exists 2026-02-17T05:50:32.550 INFO:tasks.rados.rados.0.trial115.stdout:818: expect (ObjNum 251 snap 49 seq_num 251) 2026-02-17T05:50:32.644 INFO:tasks.rados.rados.0.trial115.stdout:819: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:50:32.696 INFO:tasks.rados.rados.0.trial115.stdout:825: finishing rollback tid 0 to trial11545420-31 2026-02-17T05:50:32.696 INFO:tasks.rados.rados.0.trial115.stdout:825: finishing rollback tid 1 to trial11545420-31 2026-02-17T05:50:32.696 INFO:tasks.rados.rados.0.trial115.stdout:825: finishing rollback tid 2 to trial11545420-31 2026-02-17T05:50:32.696 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 272 (ObjNum 273 snap 56 seq_num 273) dirty exists 2026-02-17T05:50:32.696 INFO:tasks.rados.rados.0.trial115.stdout:816: expect (ObjNum 307 snap 73 seq_num 307) 2026-02-17T05:50:32.812 INFO:tasks.rados.rados.0.trial115.stdout:824: expect (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:50:32.885 INFO:tasks.rados.rados.0.trial115.stdout:815: finishing copy_from racing read to trial11545420-42 2026-02-17T05:50:32.885 INFO:tasks.rados.rados.0.trial115.stdout:815: finishing copy_from to trial11545420-42 2026-02-17T05:50:32.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 291 (ObjNum 267 snap 54 seq_num 267) dirty exists 2026-02-17T05:50:32.886 INFO:tasks.rados.rados.0.trial115.stdout:817: finishing copy_from racing read to trial11545420-15 2026-02-17T05:50:32.886 INFO:tasks.rados.rados.0.trial115.stdout:823: finishing copy_from racing read to trial11545420-33 2026-02-17T05:50:32.886 INFO:tasks.rados.rados.0.trial115.stdout:823: finishing copy_from to trial11545420-33 2026-02-17T05:50:32.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 299 (ObjNum 298 snap 70 seq_num 298) dirty exists 2026-02-17T05:50:32.886 INFO:tasks.rados.rados.0.trial115.stdout:817: finishing copy_from to trial11545420-15 2026-02-17T05:50:32.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 310 (ObjNum 266 snap 54 seq_num 266) dirty exists 2026-02-17T05:50:33.545 INFO:tasks.rados.rados.0.trial115.stdout:815: done (9 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:816: done (8 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:817: done (7 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:818: done (6 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:819: done (5 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:820: done (4 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:823: done (3 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:824: done (2 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:825: done (1 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:826: done (0 left) 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:827: write (excl) oid 25 current snap is 83 2026-02-17T05:50:33.546 INFO:tasks.rados.rados.0.trial115.stdout:827: seq_num 332 ranges {779924=785847,2078257=580112,3301257=278667} 2026-02-17T05:50:33.551 INFO:tasks.rados.rados.0.trial115.stdout:827: writing trial11545420-25 from 779924 to 1565771 tid 1 2026-02-17T05:50:33.555 INFO:tasks.rados.rados.0.trial115.stdout:827: writing trial11545420-25 from 2078257 to 2658369 tid 2 2026-02-17T05:50:33.559 INFO:tasks.rados.rados.0.trial115.stdout:827: writing trial11545420-25 from 3301257 to 3579924 tid 3 2026-02-17T05:50:33.559 INFO:tasks.rados.rados.0.trial115.stdout:828: read oid 2 snap 57 2026-02-17T05:50:33.559 INFO:tasks.rados.rados.0.trial115.stdout:828: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:50:33.559 INFO:tasks.rados.rados.0.trial115.stdout:829: write (excl) oid 12 current snap is 83 2026-02-17T05:50:33.559 INFO:tasks.rados.rados.0.trial115.stdout:829: seq_num 333 ranges {796731=461096,1736739=723770,2928680=668051} 2026-02-17T05:50:33.563 INFO:tasks.rados.rados.0.trial115.stdout:829: writing trial11545420-12 from 796731 to 1257827 tid 1 2026-02-17T05:50:33.568 INFO:tasks.rados.rados.0.trial115.stdout:829: writing trial11545420-12 from 1736739 to 2460509 tid 2 2026-02-17T05:50:33.572 INFO:tasks.rados.rados.0.trial115.stdout:829: writing trial11545420-12 from 2928680 to 3596731 tid 3 2026-02-17T05:50:33.572 INFO:tasks.rados.rados.0.trial115.stdout:830: write oid 13 current snap is 83 2026-02-17T05:50:33.572 INFO:tasks.rados.rados.0.trial115.stdout:827: finishing write tid 1 to trial11545420-25 2026-02-17T05:50:33.572 INFO:tasks.rados.rados.0.trial115.stdout:827: finishing write tid 2 to trial11545420-25 2026-02-17T05:50:33.572 INFO:tasks.rados.rados.0.trial115.stdout:827: finishing write tid 3 to trial11545420-25 2026-02-17T05:50:33.572 INFO:tasks.rados.rados.0.trial115.stdout:830: seq_num 334 ranges {413538=536345,1395221=583781,2556103=658617} 2026-02-17T05:50:33.575 INFO:tasks.rados.rados.0.trial115.stdout:830: writing trial11545420-13 from 413538 to 949883 tid 1 2026-02-17T05:50:33.579 INFO:tasks.rados.rados.0.trial115.stdout:830: writing trial11545420-13 from 1395221 to 1979002 tid 2 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:830: writing trial11545420-13 from 2556103 to 3214720 tid 3 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:831: rollback oid 3 current snap is 83 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 3 to 68 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:827: finishing write tid 4 to trial11545420-25 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:827: finishing write tid 5 to trial11545420-25 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:827: oid 25 updating version 0 to 302 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:827: oid 25 updating version 302 to 304 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:827: oid 25 version 304 is already newer than 301 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:827: oid 25 version 304 is already newer than 303 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 304 (ObjNum 332 snap 83 seq_num 332) dirty exists 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:827: left oid 25 (ObjNum 332 snap 83 seq_num 332) 2026-02-17T05:50:33.584 INFO:tasks.rados.rados.0.trial115.stdout:828: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:829: finishing write tid 1 to trial11545420-12 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:829: finishing write tid 2 to trial11545420-12 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:829: finishing write tid 3 to trial11545420-12 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:829: finishing write tid 4 to trial11545420-12 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:827: done (4 left) 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:828: done (3 left) 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:832: read oid 1 snap 80 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:832: expect (ObjNum 251 snap 49 seq_num 251) 2026-02-17T05:50:33.636 INFO:tasks.rados.rados.0.trial115.stdout:833: write (excl) oid 16 current snap is 83 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:829: finishing write tid 5 to trial11545420-12 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:829: oid 12 updating version 0 to 361 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:829: oid 12 updating version 361 to 364 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:829: oid 12 version 364 is already newer than 362 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:829: oid 12 version 364 is already newer than 363 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 364 (ObjNum 333 snap 83 seq_num 333) dirty exists 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:829: left oid 12 (ObjNum 333 snap 83 seq_num 333) 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:830: finishing write tid 1 to trial11545420-13 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:830: finishing write tid 2 to trial11545420-13 2026-02-17T05:50:33.637 INFO:tasks.rados.rados.0.trial115.stdout:833: seq_num 335 ranges {430345=727947,1570056=727439,2699879=469889} 2026-02-17T05:50:33.639 INFO:tasks.rados.rados.0.trial115.stdout:833: writing trial11545420-16 from 430345 to 1158292 tid 1 2026-02-17T05:50:33.644 INFO:tasks.rados.rados.0.trial115.stdout:833: writing trial11545420-16 from 1570056 to 2297495 tid 2 2026-02-17T05:50:33.647 INFO:tasks.rados.rados.0.trial115.stdout:833: writing trial11545420-16 from 2699879 to 3169768 tid 3 2026-02-17T05:50:33.648 INFO:tasks.rados.rados.0.trial115.stdout:829: done (4 left) 2026-02-17T05:50:33.648 INFO:tasks.rados.rados.0.trial115.stdout:834: copy_from oid 10 from oid 48 current snap is 83 2026-02-17T05:50:33.648 INFO:tasks.rados.rados.0.trial115.stdout:830: finishing write tid 3 to trial11545420-13 2026-02-17T05:50:33.648 INFO:tasks.rados.rados.0.trial115.stdout:830: finishing write tid 4 to trial11545420-13 2026-02-17T05:50:33.648 INFO:tasks.rados.rados.0.trial115.stdout:830: finishing write tid 5 to trial11545420-13 2026-02-17T05:50:33.648 INFO:tasks.rados.rados.0.trial115.stdout:830: oid 13 updating version 0 to 303 2026-02-17T05:50:33.649 INFO:tasks.rados.rados.0.trial115.stdout:830: oid 13 version 303 is already newer than 302 2026-02-17T05:50:33.649 INFO:tasks.rados.rados.0.trial115.stdout:830: oid 13 version 303 is already newer than 300 2026-02-17T05:50:33.649 INFO:tasks.rados.rados.0.trial115.stdout:830: oid 13 version 303 is already newer than 301 2026-02-17T05:50:33.649 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 303 (ObjNum 334 snap 83 seq_num 334) dirty exists 2026-02-17T05:50:33.649 INFO:tasks.rados.rados.0.trial115.stdout:830: left oid 13 (ObjNum 334 snap 83 seq_num 334) 2026-02-17T05:50:33.649 INFO:tasks.rados.rados.0.trial115.stdout:831: finishing rollback tid 0 to trial11545420-3 2026-02-17T05:50:33.649 INFO:tasks.rados.rados.0.trial115.stdout:831: finishing rollback tid 1 to trial11545420-3 2026-02-17T05:50:33.650 INFO:tasks.rados.rados.0.trial115.stdout:831: finishing rollback tid 2 to trial11545420-3 2026-02-17T05:50:33.650 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 275 (ObjNum 196 snap 37 seq_num 196) dirty exists 2026-02-17T05:50:33.650 INFO:tasks.rados.rados.0.trial115.stdout:832: expect (ObjNum 251 snap 49 seq_num 251) 2026-02-17T05:50:33.742 INFO:tasks.rados.rados.0.trial115.stdout:833: finishing write tid 1 to trial11545420-16 2026-02-17T05:50:33.742 INFO:tasks.rados.rados.0.trial115.stdout:833: finishing write tid 2 to trial11545420-16 2026-02-17T05:50:33.742 INFO:tasks.rados.rados.0.trial115.stdout:830: done (4 left) 2026-02-17T05:50:33.742 INFO:tasks.rados.rados.0.trial115.stdout:831: done (3 left) 2026-02-17T05:50:33.742 INFO:tasks.rados.rados.0.trial115.stdout:832: done (2 left) 2026-02-17T05:50:33.742 INFO:tasks.rados.rados.0.trial115.stdout:835: copy_from oid 43 from oid 21 current snap is 83 2026-02-17T05:50:33.742 INFO:tasks.rados.rados.0.trial115.stdout:833: finishing write tid 3 to trial11545420-16 2026-02-17T05:50:33.742 INFO:tasks.rados.rados.0.trial115.stdout:833: finishing write tid 4 to trial11545420-16 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:833: finishing write tid 5 to trial11545420-16 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:833: oid 16 updating version 0 to 338 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:833: oid 16 version 338 is already newer than 335 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:833: oid 16 version 338 is already newer than 336 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:833: oid 16 version 338 is already newer than 337 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 338 (ObjNum 335 snap 83 seq_num 335) dirty exists 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:833: left oid 16 (ObjNum 335 snap 83 seq_num 335) 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:834: finishing copy_from racing read to trial11545420-10 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:834: finishing copy_from to trial11545420-10 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 330 (ObjNum 320 snap 77 seq_num 320) dirty exists 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:833: done (2 left) 2026-02-17T05:50:33.743 INFO:tasks.rados.rados.0.trial115.stdout:834: done (1 left) 2026-02-17T05:50:33.744 INFO:tasks.rados.rados.0.trial115.stdout:836: delete oid 45 current snap is 83 2026-02-17T05:50:33.744 INFO:tasks.rados.rados.0.trial115.stdout:836: done (1 left) 2026-02-17T05:50:33.744 INFO:tasks.rados.rados.0.trial115.stdout:837: read oid 31 snap -1 2026-02-17T05:50:33.744 INFO:tasks.rados.rados.0.trial115.stdout:837: expect (ObjNum 273 snap 56 seq_num 273) 2026-02-17T05:50:33.745 INFO:tasks.rados.rados.0.trial115.stdout:838: delete oid 10 current snap is 83 2026-02-17T05:50:33.745 INFO:tasks.rados.rados.0.trial115.stdout:838: done (2 left) 2026-02-17T05:50:33.745 INFO:tasks.rados.rados.0.trial115.stdout:839: snap_remove snap 57 2026-02-17T05:50:34.459 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:34.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.526+0000 7f3b1be00640 10 monclient: get_monmap_and_config 2026-02-17T05:50:34.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.526+0000 7f3b1be00640 10 monclient: build_initial_monmap 2026-02-17T05:50:34.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.526+0000 7f3b1be00640 10 monclient: monmap: 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:34.528054+0000 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:34.528054+0000 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:34.529 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 1 Processor -- start 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 1 -- start start 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient: init 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3b1406c890 0x7f3b1406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient(hunting): picked mon.noname-c con 0x7f3b1406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3b1409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3b1406d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:34.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b14135c10 con 0x7f3b1406d1b0 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b1406daf0 con 0x7f3b1409c4a0 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f3b140a0880 con 0x7f3b1406c890 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1be00640 10 monclient(hunting): authenticate will time out at 880.081582s 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1a376640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f3b1406d1b0 0x7f3b14135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57636/0 (socket says 10.20.193.78:57636) 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b1a376640 1 -- 10.20.193.78:0/733953147 learned_addr learned my addr 10.20.193.78:0/733953147 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b19b75640 1 --2- 10.20.193.78:0/733953147 >> v2:10.20.193.115:3300/0 conn(0x7f3b1406c890 0x7f3b1406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b19b75640 10 monclient(hunting): get_auth_request con 0x7f3b1406c890 auth_method 0 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b19b75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:34.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b19b75640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:34.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b19b75640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:34.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b19b75640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:34.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b19b75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:34.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.527+0000 7f3b19b75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:34.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.529+0000 7f3b1a376640 1 -- 10.20.193.78:0/733953147 >> v1:10.20.193.78:6789/0 conn(0x7f3b1406d1b0 legacy=0x7f3b14135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:34.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.529+0000 7f3b18b73640 1 -- 10.20.193.78:0/733953147 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976605337 0 0) 0x7f3b14135c10 con 0x7f3b1406d1b0 2026-02-17T05:50:34.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.529+0000 7f3b18b73640 10 monclient(hunting): _negotiate 2026-02-17T05:50:34.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.529+0000 7f3b18b73640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.529+0000 7f3b18b73640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.529+0000 7f3b18b73640 10 monclient(hunting): my global_id is 54300 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.529+0000 7f3b18b73640 1 -- 10.20.193.78:0/733953147 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3afc003620 con 0x7f3b1406d1b0 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19374640 1 -- 10.20.193.78:0/733953147 >> v1:10.20.193.115:6789/0 conn(0x7f3b1409c4a0 legacy=0x7f3b14134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19b75640 10 monclient(hunting): handle_auth_done global_id 44320 payload 274 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b18b73640 1 -- 10.20.193.78:0/733953147 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3000805359 0 0) 0x7f3afc003620 con 0x7f3b1406d1b0 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19b75640 1 -- 10.20.193.78:0/733953147 >> v1:10.20.193.115:6789/0 conn(0x7f3b1409c4a0 legacy=0x7f3b14134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19b75640 1 -- 10.20.193.78:0/733953147 >> v1:10.20.193.78:6789/0 conn(0x7f3b1406d1b0 legacy=0x7f3b14135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19b75640 10 monclient: _finish_hunting 0 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19b75640 1 monclient: found mon.noname-c 2026-02-17T05:50:34.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19b75640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19b75640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b19b75640 1 -- 10.20.193.78:0/733953147 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1409cc80 con 0x7f3b1406c890 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.530+0000 7f3b18b73640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b19b75640 1 -- 10.20.193.78:0/733953147 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406c890 msgr2=0x7f3b1406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b19b75640 1 --2- 10.20.193.78:0/733953147 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406c890 0x7f3b1406cc70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3b040099a0 tx=0x7f3b0402eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443399429e58a147 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 1 -- 10.20.193.78:0/733953147 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0403c070 con 0x7f3b1406c890 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: dump: 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:34.542 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:34.543 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:34.543 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:34.543 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:34.543 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:34.543 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:34.543 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:34.543 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: _finish_auth 0 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: _check_auth_tickets 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b1be00640 5 monclient: authenticate success, global_id 44320 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b1be00640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 1 -- 10.20.193.78:0/733953147 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0402fb10 con 0x7f3b1406c890 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 1 -- 10.20.193.78:0/733953147 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10003530 con 0x7f3b1406c890 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:34.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b18b73640 10 monclient: dump: 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:34.545 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b1be00640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.532+0000 7f3b1be00640 4 set_mon_vals no callback set 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 10 monclient: shutdown 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 -- 10.20.193.78:0/733953147 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406c890 msgr2=0x7f3b1406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 --2- 10.20.193.78:0/733953147 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406c890 0x7f3b1406cc70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3b040099a0 tx=0x7f3b0402eeb0 comp rx=0 tx=0).stop 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 -- 10.20.193.78:0/733953147 shutdown_connections 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 --2- 10.20.193.78:0/733953147 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406c890 0x7f3b1406cc70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 -- 10.20.193.78:0/733953147 >> 10.20.193.78:0/733953147 conn(0x7f3b140a6620 msgr2=0x7f3b140a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 -- 10.20.193.78:0/733953147 shutdown_connections 2026-02-17T05:50:34.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 -- 10.20.193.78:0/733953147 wait complete. 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 10 monclient: build_initial_monmap 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 10 monclient: monmap: 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:34.547 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 Processor -- start 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 1 -- start start 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.533+0000 7f3b1be00640 10 monclient: init 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 0x7f3b141cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient(hunting): picked mon.noname-a con 0x7f3b1406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406d1b0 0x7f3b141cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient(hunting): picked mon.noname-b con 0x7f3b1406d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 0x7f3b14124820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:34.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient(hunting): picked mon.noname-c con 0x7f3b1409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b140a0020 con 0x7f3b1406c890 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1406d640 con 0x7f3b1406d1b0 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3b140a01a0 con 0x7f3b1409c4a0 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1be00640 10 monclient(hunting): authenticate will time out at 880.088207s 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 0x7f3b141cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 0x7f3b141cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42456/0 (socket says 10.20.193.78:42456) 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 -- 10.20.193.78:0/675481730 learned_addr learned my addr 10.20.193.78:0/675481730 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:34.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient(hunting): get_auth_request con 0x7f3b1406c890 auth_method 0 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 0x7f3b14124820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 10 monclient(hunting): get_auth_request con 0x7f3b1409c4a0 auth_method 0 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:34.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19374640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406d1b0 0x7f3b141cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 1 -- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 msgr2=0x7f3b14124820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 1 -- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 msgr2=0x7f3b14124820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 0x7f3b14124820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b1a376640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 0x7f3b14124820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19374640 10 monclient(hunting): get_auth_request con 0x7f3b1406d1b0 auth_method 0 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19374640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19374640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19374640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient(hunting): handle_auth_done global_id 54303 payload 824 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 -- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 msgr2=0x7f3b14124820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 0x7f3b14124820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:34.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 -- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406d1b0 msgr2=0x7f3b141cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406d1b0 0x7f3b141cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient: _finish_hunting 0 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 monclient: found mon.noname-a 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.534+0000 7f3b19b75640 1 -- 10.20.193.78:0/675481730 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b14129190 con 0x7f3b1406c890 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b19b75640 1 -- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 msgr2=0x7f3b141cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b19b75640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 0x7f3b141cf6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b04002410 tx=0x7f3b040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b19b75640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 0x7f3b141cf6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b04002410 tx=0x7f3b040047c0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f3b04046070 mon_map magic: 0 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 1 -- 10.20.193.78:0/675481730 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04046070 con 0x7f3b1406c890 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:34.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 10 monclient: dump: 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:34.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 10 monclient: _finish_auth 0 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 10 monclient: _check_auth_tickets 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 1 -- 10.20.193.78:0/675481730 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04041660 con 0x7f3b1406c890 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b027fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b1be00640 5 monclient: authenticate success, global_id 54303 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b1be00640 10 monclient: _renew_subs 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b1be00640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b03fff640 4 set_mon_vals no callback set 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b141281f0 con 0x7f3b1406c890 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b1be00640 10 monclient: _renew_subs 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b1be00640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:34.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.536+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b14126da0 con 0x7f3b1406c890 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b027fc640 1 -- 10.20.193.78:0/675481730 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0403c040 con 0x7f3b1406c890 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b027fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b027fc640 10 monclient: dump: 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:34.555 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b027fc640 1 -- 10.20.193.78:0/675481730 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3b04041a30 con 0x7f3b1406c890 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b1be00640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b1be00640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b1be00640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b14072530 con 0x7f3b1406c890 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b027fc640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae403deb0 0x7f3ae4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b19374640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae403deb0 0x7f3ae4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.537+0000 7f3b19374640 10 monclient: get_auth_request con 0x7f3ae403deb0 auth_method 0 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.540+0000 7f3b027fc640 1 -- 10.20.193.78:0/675481730 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(178..178 src has 117..178) ==== 4767+0+0 (secure 0 0 0) 0x7f3b04078190 con 0x7f3b1406c890 2026-02-17T05:50:34.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:34.540+0000 7f3b19374640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae403deb0 0x7f3ae4040370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3b140a6eb0 tx=0x7f3b08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:34.796 INFO:tasks.rados.rados.0.trial115.stdout:839: done (2 left) 2026-02-17T05:50:34.796 INFO:tasks.rados.rados.0.trial115.stdout:840: rollback oid 49 current snap is 83 2026-02-17T05:50:34.796 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 49 to 81 2026-02-17T05:50:34.796 INFO:tasks.rados.rados.0.trial115.stdout:837: expect (ObjNum 273 snap 56 seq_num 273) 2026-02-17T05:50:34.883 INFO:tasks.rados.rados.0.trial115.stdout:835: finishing copy_from racing read to trial11545420-43 2026-02-17T05:50:34.883 INFO:tasks.rados.rados.0.trial115.stdout:835: finishing copy_from to trial11545420-43 2026-02-17T05:50:34.883 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 295 (ObjNum 316 snap 76 seq_num 316) dirty exists 2026-02-17T05:50:34.883 INFO:tasks.rados.rados.0.trial115.stdout:835: done (2 left) 2026-02-17T05:50:34.883 INFO:tasks.rados.rados.0.trial115.stdout:837: done (1 left) 2026-02-17T05:50:34.884 INFO:tasks.rados.rados.0.trial115.stdout:841: snap_create 2026-02-17T05:50:34.884 INFO:tasks.rados.rados.0.trial115.stdout:840: finishing rollback tid 0 to trial11545420-49 2026-02-17T05:50:34.884 INFO:tasks.rados.rados.0.trial115.stdout:840: finishing rollback tid 1 to trial11545420-49 2026-02-17T05:50:34.884 INFO:tasks.rados.rados.0.trial115.stdout:840: finishing rollback tid 2 to trial11545420-49 2026-02-17T05:50:34.884 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 183 (ObjNum 125 snap 20 seq_num 125) dirty exists 2026-02-17T05:50:35.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.363+0000 7f3b027fc640 1 -- 10.20.193.78:0/675481730 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3b14129190 con 0x7f3b1406c890 2026-02-17T05:50:35.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.363+0000 7f3b027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:35.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.363+0000 7f3b027fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:35.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.480+0000 7f3b1be00640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:35.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.480+0000 7f3b1be00640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:35.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.480+0000 7f3b1be00640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:35.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.480+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3b1409bf40 con 0x7f3b1406c890 2026-02-17T05:50:35.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.480+0000 7f3b027fc640 1 -- 10.20.193.78:0/675481730 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f3b04076ba0 con 0x7f3b1406c890 2026-02-17T05:50:35.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.480+0000 7f3b027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:35.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.480+0000 7f3b027fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:35.481 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:35.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae403deb0 msgr2=0x7f3ae4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae403deb0 0x7f3ae4040370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3b140a6eb0 tx=0x7f3b08009040 comp rx=0 tx=0).stop 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 10 monclient: shutdown 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 msgr2=0x7f3b141cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 0x7f3b141cf6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b04002410 tx=0x7f3b040047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 shutdown_connections 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1409c4a0 0x7f3b14124820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae403deb0 0x7f3ae4040370 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b1406d1b0 0x7f3b141cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 --2- 10.20.193.78:0/675481730 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b1406c890 0x7f3b141cf6a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 >> 10.20.193.78:0/675481730 conn(0x7f3b140a6620 msgr2=0x7f3b1406fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:35.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 shutdown_connections 2026-02-17T05:50:35.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.481+0000 7f3b1be00640 1 -- 10.20.193.78:0/675481730 wait complete. 2026-02-17T05:50:35.489 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:50:35.490 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:50:35.490 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:50:35.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.559+0000 7f5c0632c640 10 monclient: get_monmap_and_config 2026-02-17T05:50:35.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.559+0000 7f5c0632c640 10 monclient: build_initial_monmap 2026-02-17T05:50:35.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.559+0000 7f5c0632c640 10 monclient: monmap: 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:35.560622+0000 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:35.560622+0000 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:35.563 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.559+0000 7f5c0632c640 1 Processor -- start 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.559+0000 7f5c0632c640 1 -- start start 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.559+0000 7f5c0632c640 10 monclient: init 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5c0006c890 0x7f5c0006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient(hunting): picked mon.noname-a con 0x7f5c0006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c0006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient(hunting): picked mon.noname-b con 0x7f5c0009d170 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5c0009b200 0x7f5c001388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient(hunting): picked mon.noname-c con 0x7f5c0009b200 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5c0006d6f0 con 0x7f5c0006c890 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5c0009bf60 con 0x7f5c0009b200 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0009c520 con 0x7f5c0009d170 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c0632c640 10 monclient(hunting): authenticate will time out at 881.114163s 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5c0006c890 0x7f5c0006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c0006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5c0006c890 0x7f5c0006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42472/0 (socket says 10.20.193.78:42472) 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 -- 10.20.193.78:0/2152563348 learned_addr learned my addr 10.20.193.78:0/2152563348 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c0006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44704/0 (socket says 10.20.193.78:44704) 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient(hunting): get_auth_request con 0x7f5c0006c890 auth_method 0 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 10 monclient(hunting): get_auth_request con 0x7f5c0009d170 auth_method 0 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c048a2640 1 --2- 10.20.193.78:0/2152563348 >> v2:10.20.193.115:3300/0 conn(0x7f5c0009b200 0x7f5c001388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 1 -- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 msgr2=0x7f5c0006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 1 -- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 msgr2=0x7f5c0006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 1 --2- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c0006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c048a2640 10 monclient(hunting): get_auth_request con 0x7f5c0009b200 auth_method 0 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c048a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c048a2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5c048a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bff7fe640 1 --2- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c0006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient(hunting): handle_auth_done global_id 54306 payload 274 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 -- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 msgr2=0x7f5c0006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 --2- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c0006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 -- 10.20.193.78:0/2152563348 >> v2:10.20.193.115:3300/0 conn(0x7f5c0009b200 msgr2=0x7f5c001388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 --2- 10.20.193.78:0/2152563348 >> v2:10.20.193.115:3300/0 conn(0x7f5c0009b200 0x7f5c001388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 monclient: found mon.noname-a 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:35.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:35.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.560+0000 7f5bfffff640 1 -- 10.20.193.78:0/2152563348 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0009c800 con 0x7f5c0006c890 2026-02-17T05:50:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.562+0000 7f5bfffff640 1 -- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 msgr2=0x7f5c0006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.562+0000 7f5bfffff640 1 --2- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c0006cc70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5c000693c0 tx=0x7f5bec02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189486939bccbc00 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 1 -- 10.20.193.78:0/2152563348 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec03c070 con 0x7f5c0006c890 2026-02-17T05:50:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: dump: 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.580 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: _finish_auth 0 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 1 -- 10.20.193.78:0/2152563348 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec02fca0 con 0x7f5c0006c890 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 1 -- 10.20.193.78:0/2152563348 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec037660 con 0x7f5c0006c890 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 5 monclient: authenticate success, global_id 54306 2026-02-17T05:50:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 4 set_mon_vals no callback set 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5bfeffd640 10 monclient: dump: 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.582 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 10 monclient: shutdown 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 -- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 msgr2=0x7f5c0006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 --2- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c0006cc70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5c000693c0 tx=0x7f5bec02ee60 comp rx=0 tx=0).stop 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 -- 10.20.193.78:0/2152563348 shutdown_connections 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 --2- 10.20.193.78:0/2152563348 >> v2:10.20.193.115:3300/0 conn(0x7f5c0009b200 0x7f5c001388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 --2- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c0006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 --2- 10.20.193.78:0/2152563348 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c0006cc70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 -- 10.20.193.78:0/2152563348 >> 10.20.193.78:0/2152563348 conn(0x7f5c000a4230 msgr2=0x7f5c000a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 -- 10.20.193.78:0/2152563348 shutdown_connections 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 1 -- 10.20.193.78:0/2152563348 wait complete. 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 10 monclient: build_initial_monmap 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.563+0000 7f5c0632c640 10 monclient: monmap: 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:35.584 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 1 Processor -- start 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 1 -- start start 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient: init 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:35.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c000918c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient(hunting): picked mon.noname-a con 0x7f5c0006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5c0009b200 0x7f5c00091e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient(hunting): picked mon.noname-b con 0x7f5c0009b200 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c00092340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient(hunting): picked mon.noname-c con 0x7f5c0009d170 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c000a3530 con 0x7f5c0006c890 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c000a33b0 con 0x7f5c0009b200 2026-02-17T05:50:35.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5c00138df0 con 0x7f5c0009d170 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c0632c640 10 monclient(hunting): authenticate will time out at 881.118416s 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c000918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c000918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42488/0 (socket says 10.20.193.78:42488) 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 -- 10.20.193.78:0/3959838450 learned_addr learned my addr 10.20.193.78:0/3959838450 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient(hunting): get_auth_request con 0x7f5c0006c890 auth_method 0 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c00092340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 10 monclient(hunting): get_auth_request con 0x7f5c0009d170 auth_method 0 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 msgr2=0x7f5c00092340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:50:35.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 msgr2=0x7f5c00092340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c00092340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient(hunting): handle_auth_done global_id 54309 payload 824 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bff7fe640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5c0009b200 0x7f5c00091e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5c048a2640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c00092340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 msgr2=0x7f5c00092340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5c0009d170 0x7f5c00092340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5c0009b200 msgr2=0x7f5c00091e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5c0009b200 0x7f5c00091e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 monclient: found mon.noname-a 2026-02-17T05:50:35.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bfffff640 1 -- 10.20.193.78:0/3959838450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c00094ed0 con 0x7f5c0006c890 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bff7fe640 10 monclient: get_auth_request con 0x7f5c0009b200 auth_method 0 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.564+0000 7f5bff7fe640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5c0009b200 0x7f5c00091e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfffff640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 msgr2=0x7f5c000918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfffff640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c000918c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5bec002410 tx=0x7f5bec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec046070 con 0x7f5c0006c890 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: dump: 2026-02-17T05:50:35.590 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: _finish_auth 0 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: _check_auth_tickets 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec02fe50 con 0x7f5c0006c890 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec03c040 con 0x7f5c0006c890 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfe7fc640 4 set_mon_vals no callback set 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5bfcff9640 10 monclient: dump: 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.592 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5c0632c640 5 monclient: authenticate success, global_id 54309 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5c0632c640 10 monclient: _renew_subs 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5c0632c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c00093020 con 0x7f5c0006c890 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5c0632c640 10 monclient: _renew_subs 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5c0632c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.566+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c000a5be0 con 0x7f5c0006c890 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5c0632c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5c0632c640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00094870 0x7f5c000700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5c0632c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c00077110 con 0x7f5c00094870 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5bec0418c0 con 0x7f5c0006c890 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00094870 0x7f5c000700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 10 monclient: get_auth_request con 0x7f5c00094870 auth_method 0 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 10 monclient(hunting): handle_auth_done global_id 54312 payload 824 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bfcff9640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bd003df00 0x7f5bd00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bff7fe640 1 -- 10.20.193.78:0/3959838450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c00074980 con 0x7f5c00094870 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(179..179 src has 117..179) ==== 4811+0+0 (secure 0 0 0) 0x7f5bec0777c0 con 0x7f5c0006c890 2026-02-17T05:50:35.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5c048a2640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bd003df00 0x7f5bd00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.567+0000 7f5c048a2640 10 monclient: get_auth_request con 0x7f5bd003df00 auth_method 0 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.569+0000 7f5bff7fe640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00094870 msgr2=0x7f5c000700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.569+0000 7f5bff7fe640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00094870 0x7f5c000700f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c00094520 tx=0x7f5bf0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.569+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0017020 con 0x7f5c00094870 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.569+0000 7f5bfcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.569+0000 7f5bfcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.569+0000 7f5bfcff9640 10 monclient: dump: 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.570+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f5bf000e990 con 0x7f5c00094870 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.570+0000 7f5bfcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.570+0000 7f5bfcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.570+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00094870 msgr2=0x7f5c000700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.570+0000 7f5bfcff9640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00094870 0x7f5c000700f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c00094520 tx=0x7f5bf0009ba0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.570+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00094870 msgr2=0x7f5c000700f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.570+0000 7f5bfcff9640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00094870 0x7f5c000700f0 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c00094520 tx=0x7f5bf0009ba0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.570+0000 7f5c048a2640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bd003df00 0x7f5bd00403c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c0009a430 tx=0x7f5bf400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5c0632c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5c0632c640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00070630 0x7f5c00070a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5c0632c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c001388b0 con 0x7f5c00070630 2026-02-17T05:50:35.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00070630 0x7f5c00070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 10 monclient: get_auth_request con 0x7f5c00070630 auth_method 0 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 10 monclient(hunting): handle_auth_done global_id 54315 payload 824 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.573+0000 7f5bff7fe640 1 -- 10.20.193.78:0/3959838450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5c000a3870 con 0x7f5c00070630 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bff7fe640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00070630 msgr2=0x7f5c00070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bff7fe640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00070630 0x7f5c00070a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0004750 tx=0x7f5bf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000f040 con 0x7f5c00070630 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 10 monclient: dump: 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f5bf0003230 con 0x7f5c00070630 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00070630 msgr2=0x7f5c00070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00070630 0x7f5c00070a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0004750 tx=0x7f5bf00047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00070630 msgr2=0x7f5c00070a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5bfcff9640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c00070630 0x7f5c00070a40 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0004750 tx=0x7f5bf00047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 46, 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 105049, 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:35.596 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:35.597 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.598 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:35.599 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:35.600 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:35.601 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bd003df00 msgr2=0x7f5bd00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bd003df00 0x7f5bd00403c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c0009a430 tx=0x7f5bf400b040 comp rx=0 tx=0).stop 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 10 monclient: shutdown 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 msgr2=0x7f5c000918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c048a2640 1 -- 10.20.193.78:0/3959838450 reap_dead start 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c000918c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5bec002410 tx=0x7f5bec0047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 shutdown_connections 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 --2- 10.20.193.78:0/3959838450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5c0006c890 0x7f5c000918c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 >> 10.20.193.78:0/3959838450 conn(0x7f5c000a4230 msgr2=0x7f5c00137110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 shutdown_connections 2026-02-17T05:50:35.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.575+0000 7f5c0632c640 1 -- 10.20.193.78:0/3959838450 wait complete. 2026-02-17T05:50:35.602 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:50:35.602 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:50:35.602 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:50:35.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.669+0000 7f9ba86bf640 10 monclient: get_monmap_and_config 2026-02-17T05:50:35.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.669+0000 7f9ba86bf640 10 monclient: build_initial_monmap 2026-02-17T05:50:35.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.669+0000 7f9ba86bf640 10 monclient: monmap: 2026-02-17T05:50:35.669 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:35.670903+0000 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:35.670903+0000 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:35.670 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 1 Processor -- start 2026-02-17T05:50:35.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 1 -- start start 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient: init 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9ba006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9ba009c4a0 0x7f9ba013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient(hunting): picked mon.noname-c con 0x7f9ba009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 0x7f9ba013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient(hunting): picked mon.noname-b con 0x7f9ba006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba006daf0 con 0x7f9ba006c890 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba00a0880 con 0x7f9ba009c4a0 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba009cc80 con 0x7f9ba006d380 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba86bf640 10 monclient(hunting): authenticate will time out at 881.224493s 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 0x7f9ba013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 0x7f9ba013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44716/0 (socket says 10.20.193.78:44716) 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 1 -- 10.20.193.78:0/378141942 learned_addr learned my addr 10.20.193.78:0/378141942 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:35.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 10 monclient(hunting): get_auth_request con 0x7f9ba006d380 auth_method 0 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6434640 1 --1- 10.20.193.78:0/378141942 >> v1:10.20.193.115:6789/0 conn(0x7f9ba006c890 0x7f9ba006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:53294/0 (socket says 10.20.193.78:53294) 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba5c33640 1 --2- 10.20.193.78:0/378141942 >> v2:10.20.193.115:3300/0 conn(0x7f9ba009c4a0 0x7f9ba013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba5c33640 10 monclient(hunting): get_auth_request con 0x7f9ba009c4a0 auth_method 0 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba5c33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba5c33640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba5c33640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 1 -- 10.20.193.78:0/378141942 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 msgr2=0x7f9ba013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 1 -- 10.20.193.78:0/378141942 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 msgr2=0x7f9ba013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 1 --2- 10.20.193.78:0/378141942 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 0x7f9ba013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:35.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.670+0000 7f9ba6c35640 1 --2- 10.20.193.78:0/378141942 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 0x7f9ba013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.672+0000 7f9ba5c33640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.672+0000 7f9ba5c33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.672+0000 7f9ba6434640 1 -- 10.20.193.78:0/378141942 >> v1:10.20.193.115:6789/0 conn(0x7f9ba006c890 legacy=0x7f9ba006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.672+0000 7f9ba5c33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 10 monclient(hunting): handle_auth_done global_id 44332 payload 274 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 1 -- 10.20.193.78:0/378141942 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 msgr2=0x7f9ba013d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 1 --2- 10.20.193.78:0/378141942 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 0x7f9ba013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 1 -- 10.20.193.78:0/378141942 >> v1:10.20.193.115:6789/0 conn(0x7f9ba006c890 legacy=0x7f9ba006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 10 monclient: _finish_hunting 0 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 1 monclient: found mon.noname-c 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:50:35.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.673+0000 7f9ba5c33640 1 -- 10.20.193.78:0/378141942 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba009cf60 con 0x7f9ba009c4a0 2026-02-17T05:50:35.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5c33640 1 -- 10.20.193.78:0/378141942 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 msgr2=0x7f9ba013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:35.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5c33640 1 --2- 10.20.193.78:0/378141942 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 0x7f9ba013b3a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009920 tx=0x7f9b9402ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d9f4bd393dacdea server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 1 -- 10.20.193.78:0/378141942 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9403c070 con 0x7f9ba009c4a0 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: dump: 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: _finish_auth 0 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: _check_auth_tickets 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 1 -- 10.20.193.78:0/378141942 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9402fce0 con 0x7f9ba009c4a0 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:35.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 1 -- 10.20.193.78:0/378141942 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94037660 con 0x7f9ba009c4a0 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba86bf640 5 monclient: authenticate success, global_id 44332 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba86bf640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba86bf640 4 set_mon_vals no callback set 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.675+0000 7f9ba5432640 10 monclient: dump: 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 10 monclient: shutdown 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 -- 10.20.193.78:0/378141942 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 msgr2=0x7f9ba013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 --2- 10.20.193.78:0/378141942 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 0x7f9ba013b3a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009920 tx=0x7f9b9402ee10 comp rx=0 tx=0).stop 2026-02-17T05:50:35.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 -- 10.20.193.78:0/378141942 shutdown_connections 2026-02-17T05:50:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 --2- 10.20.193.78:0/378141942 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006d380 0x7f9ba013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 --2- 10.20.193.78:0/378141942 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 0x7f9ba013b3a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 -- 10.20.193.78:0/378141942 >> 10.20.193.78:0/378141942 conn(0x7f9ba00a6620 msgr2=0x7f9ba00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 -- 10.20.193.78:0/378141942 shutdown_connections 2026-02-17T05:50:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 -- 10.20.193.78:0/378141942 wait complete. 2026-02-17T05:50:35.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 10 monclient: build_initial_monmap 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 10 monclient: monmap: 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 Processor -- start 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 1 -- start start 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.676+0000 7f9ba86bf640 10 monclient: init 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 0x7f9ba0136280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient(hunting): picked mon.noname-c con 0x7f9ba006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ba006d380 0x7f9ba01367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient(hunting): picked mon.noname-a con 0x7f9ba006d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 0x7f9ba011eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient(hunting): picked mon.noname-b con 0x7f9ba009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba009bfa0 con 0x7f9ba006d380 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba009be20 con 0x7f9ba009c4a0 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba013de30 con 0x7f9ba006c890 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba86bf640 10 monclient(hunting): authenticate will time out at 881.231210s 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 0x7f9ba0136280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 0x7f9ba0136280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44724/0 (socket says 10.20.193.78:44724) 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 learned_addr learned my addr 10.20.193.78:0/2453154235 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 10 monclient(hunting): get_auth_request con 0x7f9ba006c890 auth_method 0 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ba006d380 0x7f9ba01367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient(hunting): get_auth_request con 0x7f9ba006d380 auth_method 0 2026-02-17T05:50:35.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 msgr2=0x7f9ba0136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 msgr2=0x7f9ba0136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 0x7f9ba0136280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 0x7f9ba0136280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6c35640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 0x7f9ba011eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient(hunting): handle_auth_done global_id 54318 payload 824 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 msgr2=0x7f9ba0136280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba006c890 0x7f9ba0136280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 msgr2=0x7f9ba011eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 0x7f9ba011eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient: _finish_hunting 0 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 1 monclient: found mon.noname-a 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba5c33640 1 -- 10.20.193.78:0/2453154235 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba0122670 con 0x7f9ba006d380 2026-02-17T05:50:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6c35640 10 monclient: get_auth_request con 0x7f9ba009c4a0 auth_method 0 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.677+0000 7f9ba6c35640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ba009c4a0 0x7f9ba011eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba5c33640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ba006d380 msgr2=0x7f9ba01367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba5c33640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ba006d380 0x7f9ba01367c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9b94036000 tx=0x7f9b940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94046020 con 0x7f9ba006d380 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: dump: 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: _finish_auth 0 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b94040820 con 0x7f9ba006d380 2026-02-17T05:50:35.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9403c040 con 0x7f9ba006d380 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba4c31640 4 set_mon_vals no callback set 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9b86ffd640 10 monclient: dump: 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba86bf640 5 monclient: authenticate success, global_id 54318 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba86bf640 10 monclient: _renew_subs 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba86bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba0121220 con 0x7f9ba006d380 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba86bf640 10 monclient: _renew_subs 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba86bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.679+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba013da80 con 0x7f9ba006d380 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba86bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba86bf640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba01cf6e0 con 0x7f9ba01cf300 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 10 monclient: get_auth_request con 0x7f9ba01cf300 auth_method 0 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9b9403f030 con 0x7f9ba006d380 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9b86ffd640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b7003dee0 0x7f9b700403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(179..179 src has 117..179) ==== 4811+0+0 (secure 0 0 0) 0x7f9b940773b0 con 0x7f9ba006d380 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6c35640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b7003dee0 0x7f9b700403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.680+0000 7f9ba6c35640 10 monclient: get_auth_request con 0x7f9b7003dee0 auth_method 0 2026-02-17T05:50:35.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.683+0000 7f9ba6c35640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b7003dee0 0x7f9b700403a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0072e0 tx=0x7f9b9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:35.800 INFO:tasks.rados.rados.0.trial115.stdout:840: done (1 left) 2026-02-17T05:50:35.801 INFO:tasks.rados.rados.0.trial115.stdout:841: done (0 left) 2026-02-17T05:50:35.801 INFO:tasks.rados.rados.0.trial115.stdout:842: write oid 47 current snap is 84 2026-02-17T05:50:35.801 INFO:tasks.rados.rados.0.trial115.stdout:842: seq_num 338 ranges {480766=553694,1629149=591119,2782148=420058} 2026-02-17T05:50:35.805 INFO:tasks.rados.rados.0.trial115.stdout:842: writing trial11545420-47 from 480766 to 1034460 tid 1 2026-02-17T05:50:35.809 INFO:tasks.rados.rados.0.trial115.stdout:842: writing trial11545420-47 from 1629149 to 2220268 tid 2 2026-02-17T05:50:35.813 INFO:tasks.rados.rados.0.trial115.stdout:842: writing trial11545420-47 from 2782148 to 3202206 tid 3 2026-02-17T05:50:35.813 INFO:tasks.rados.rados.0.trial115.stdout:843: delete oid 49 current snap is 84 2026-02-17T05:50:35.813 INFO:tasks.rados.rados.0.trial115.stdout:842: finishing write tid 1 to trial11545420-47 2026-02-17T05:50:35.813 INFO:tasks.rados.rados.0.trial115.stdout:842: finishing write tid 2 to trial11545420-47 2026-02-17T05:50:35.814 INFO:tasks.rados.rados.0.trial115.stdout:842: finishing write tid 3 to trial11545420-47 2026-02-17T05:50:35.815 INFO:tasks.rados.rados.0.trial115.stdout:842: finishing write tid 4 to trial11545420-47 2026-02-17T05:50:35.815 INFO:tasks.rados.rados.0.trial115.stdout:842: finishing write tid 5 to trial11545420-47 2026-02-17T05:50:35.815 INFO:tasks.rados.rados.0.trial115.stdout:842: oid 47 updating version 0 to 279 2026-02-17T05:50:35.815 INFO:tasks.rados.rados.0.trial115.stdout:842: oid 47 version 279 is already newer than 277 2026-02-17T05:50:35.815 INFO:tasks.rados.rados.0.trial115.stdout:842: oid 47 version 279 is already newer than 278 2026-02-17T05:50:35.815 INFO:tasks.rados.rados.0.trial115.stdout:842: oid 47 version 279 is already newer than 276 2026-02-17T05:50:35.815 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 279 (ObjNum 338 snap 84 seq_num 338) dirty exists 2026-02-17T05:50:35.815 INFO:tasks.rados.rados.0.trial115.stdout:842: left oid 47 (ObjNum 338 snap 84 seq_num 338) 2026-02-17T05:50:35.816 INFO:tasks.rados.rados.0.trial115.stdout:842: done (1 left) 2026-02-17T05:50:35.816 INFO:tasks.rados.rados.0.trial115.stdout:843: done (0 left) 2026-02-17T05:50:35.816 INFO:tasks.rados.rados.0.trial115.stdout:844: snap_remove snap 69 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 10 monclient: get_auth_request con 0x7f9ba01cf300 auth_method 0 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:35.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:35.880+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 10 monclient: get_auth_request con 0x7f9ba01cf300 auth_method 0 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:36.281+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:36.805 INFO:tasks.rados.rados.0.trial115.stdout:844: done (0 left) 2026-02-17T05:50:36.805 INFO:tasks.rados.rados.0.trial115.stdout:845: rollback oid 43 current snap is 84 2026-02-17T05:50:36.805 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 82 2026-02-17T05:50:36.805 INFO:tasks.rados.rados.0.trial115.stdout:846: snap_remove snap 77 2026-02-17T05:50:37.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:37.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 10 monclient: get_auth_request con 0x7f9ba01cf300 auth_method 0 2026-02-17T05:50:37.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:37.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:37.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:37.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:50:37.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:37.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:37.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:37.082+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:50:37.807 INFO:tasks.rados.rados.0.trial115.stdout:846: done (1 left) 2026-02-17T05:50:37.807 INFO:tasks.rados.rados.0.trial115.stdout:847: write oid 8 current snap is 84 2026-02-17T05:50:37.807 INFO:tasks.rados.rados.0.trial115.stdout:845: finishing rollback tid 0 to trial11545420-43 2026-02-17T05:50:37.807 INFO:tasks.rados.rados.0.trial115.stdout:845: finishing rollback tid 1 to trial11545420-43 2026-02-17T05:50:37.807 INFO:tasks.rados.rados.0.trial115.stdout:845: finishing rollback tid 2 to trial11545420-43 2026-02-17T05:50:37.808 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 305 (ObjNum 313 snap 74 seq_num 313) dirty exists 2026-02-17T05:50:37.808 INFO:tasks.rados.rados.0.trial115.stdout:847: seq_num 340 ranges {514380=704192,1629760=478435,2604288=442602} 2026-02-17T05:50:37.812 INFO:tasks.rados.rados.0.trial115.stdout:847: writing trial11545420-8 from 514380 to 1218572 tid 1 2026-02-17T05:50:37.815 INFO:tasks.rados.rados.0.trial115.stdout:847: writing trial11545420-8 from 1629760 to 2108195 tid 2 2026-02-17T05:50:37.819 INFO:tasks.rados.rados.0.trial115.stdout:847: writing trial11545420-8 from 2604288 to 3046890 tid 3 2026-02-17T05:50:37.819 INFO:tasks.rados.rados.0.trial115.stdout:845: done (1 left) 2026-02-17T05:50:37.819 INFO:tasks.rados.rados.0.trial115.stdout:848: delete oid 35 current snap is 84 2026-02-17T05:50:37.819 INFO:tasks.rados.rados.0.trial115.stdout:847: finishing write tid 1 to trial11545420-8 2026-02-17T05:50:37.819 INFO:tasks.rados.rados.0.trial115.stdout:847: finishing write tid 2 to trial11545420-8 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:847: finishing write tid 3 to trial11545420-8 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:847: finishing write tid 4 to trial11545420-8 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:847: finishing write tid 5 to trial11545420-8 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:847: oid 8 updating version 0 to 177 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:847: oid 8 updating version 177 to 179 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:847: oid 8 version 179 is already newer than 176 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:847: oid 8 version 179 is already newer than 178 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 179 (ObjNum 340 snap 84 seq_num 340) dirty exists 2026-02-17T05:50:37.820 INFO:tasks.rados.rados.0.trial115.stdout:847: left oid 8 (ObjNum 340 snap 84 seq_num 340) 2026-02-17T05:50:37.821 INFO:tasks.rados.rados.0.trial115.stdout:847: done (1 left) 2026-02-17T05:50:37.821 INFO:tasks.rados.rados.0.trial115.stdout:848: done (0 left) 2026-02-17T05:50:37.821 INFO:tasks.rados.rados.0.trial115.stdout:849: read oid 15 snap -1 2026-02-17T05:50:37.821 INFO:tasks.rados.rados.0.trial115.stdout:849: expect (ObjNum 266 snap 54 seq_num 266) 2026-02-17T05:50:37.821 INFO:tasks.rados.rados.0.trial115.stdout:850: write oid 33 current snap is 84 2026-02-17T05:50:37.821 INFO:tasks.rados.rados.0.trial115.stdout:850: seq_num 342 ranges {547994=454690,1746724=482104,2659134=465146,3605976=142018} 2026-02-17T05:50:37.824 INFO:tasks.rados.rados.0.trial115.stdout:850: writing trial11545420-33 from 547994 to 1002684 tid 1 2026-02-17T05:50:37.830 INFO:tasks.rados.rados.0.trial115.stdout:850: writing trial11545420-33 from 1746724 to 2228828 tid 2 2026-02-17T05:50:37.833 INFO:tasks.rados.rados.0.trial115.stdout:850: writing trial11545420-33 from 2659134 to 3124280 tid 3 2026-02-17T05:50:37.835 INFO:tasks.rados.rados.0.trial115.stdout:850: writing trial11545420-33 from 3605976 to 3747994 tid 4 2026-02-17T05:50:37.835 INFO:tasks.rados.rados.0.trial115.stdout:851: snap_remove snap 79 2026-02-17T05:50:38.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.677+0000 7f9b85ffb640 10 monclient: tick 2026-02-17T05:50:38.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.677+0000 7f9b85ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:50:38.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.677+0000 7f9b85ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:38.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.677+0000 7f9b85ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:38.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 10 monclient: get_auth_request con 0x7f9ba01cf300 auth_method 0 2026-02-17T05:50:38.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:38.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 10 monclient(hunting): handle_auth_done global_id 74099 payload 824 2026-02-17T05:50:38.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.684+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ba0074980 con 0x7f9ba01cf300 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005fd0 tx=0x7f9b90003110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9001c070 con 0x7f9ba01cf300 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 10 monclient: dump: 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.686 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f9b90018860 con 0x7f9ba01cf300 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005fd0 tx=0x7f9b90003110 comp rx=0 tx=0).stop 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 msgr2=0x7f9ba01228b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:38.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.686+0000 7f9b86ffd640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba01cf300 0x7f9ba01228b0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005fd0 tx=0x7f9b90003110 comp rx=0 tx=0).stop 2026-02-17T05:50:38.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba86bf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba86bf640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba011f1e0 0x7f9ba01da400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba006cc70 con 0x7f9ba011f1e0 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba011f1e0 0x7f9ba01da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 10 monclient: get_auth_request con 0x7f9ba011f1e0 auth_method 0 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 10 monclient(hunting): handle_auth_done global_id 74102 payload 824 2026-02-17T05:50:38.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.690+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ba00744d0 con 0x7f9ba011f1e0 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba6434640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba011f1e0 msgr2=0x7f9ba01da400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba6434640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba011f1e0 0x7f9ba01da400 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005fd0 tx=0x7f9b90002c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9002c400 con 0x7f9ba011f1e0 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 10 monclient: dump: 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.692 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f9b900173d0 con 0x7f9ba011f1e0 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba011f1e0 msgr2=0x7f9ba01da400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba011f1e0 0x7f9ba01da400 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005fd0 tx=0x7f9b90002c70 comp rx=0 tx=0).stop 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba011f1e0 msgr2=0x7f9ba01da400 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9b86ffd640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ba011f1e0 0x7f9ba01da400 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005fd0 tx=0x7f9b90002c70 comp rx=0 tx=0).stop 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 46, 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 10585, 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:38.693 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:38.694 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:38.695 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:38.696 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:38.697 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:38.698 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:38.699 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:38.700 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:38.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b7003dee0 msgr2=0x7f9b700403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba86bf640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b7003dee0 0x7f9b700403a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0072e0 tx=0x7f9b9c009040 comp rx=0 tx=0).stop 2026-02-17T05:50:38.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba86bf640 10 monclient: shutdown 2026-02-17T05:50:38.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba86bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:38.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ba006d380 msgr2=0x7f9ba01367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba86bf640 1 --2- 10.20.193.78:0/2453154235 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ba006d380 0x7f9ba01367c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9b94036000 tx=0x7f9b940047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba6c35640 1 -- 10.20.193.78:0/2453154235 reap_dead start 2026-02-17T05:50:38.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.692+0000 7f9ba6c35640 1 -- 10.20.193.78:0/2453154235 reap_dead start 2026-02-17T05:50:38.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.693+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 shutdown_connections 2026-02-17T05:50:38.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.693+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 >> 10.20.193.78:0/2453154235 conn(0x7f9ba00a6620 msgr2=0x7f9ba00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:38.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.693+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 shutdown_connections 2026-02-17T05:50:38.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.693+0000 7f9ba86bf640 1 -- 10.20.193.78:0/2453154235 wait complete. 2026-02-17T05:50:38.702 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:50:38.702 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:50:38.702 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:50:38.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.772+0000 7f63686e8640 10 monclient: get_monmap_and_config 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.772+0000 7f63686e8640 10 monclient: build_initial_monmap 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.772+0000 7f63686e8640 10 monclient: monmap: 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:38.773470+0000 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:38.773470+0000 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:38.772 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.772+0000 7f63686e8640 1 Processor -- start 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.772+0000 7f63686e8640 1 -- start start 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.772+0000 7f63686e8640 10 monclient: init 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.772+0000 7f63686e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f636006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636009c4a0 0x7f636013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 10 monclient(hunting): picked mon.noname-b con 0x7f636009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f636006d380 0x7f636013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 10 monclient(hunting): picked mon.noname-a con 0x7f636006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636006daf0 con 0x7f636006c890 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f63600a0880 con 0x7f636006d380 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f636009cc80 con 0x7f636009c4a0 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:38.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f63686e8640 10 monclient(hunting): authenticate will time out at 884.327064s 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f636645d640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f636006c890 0x7f636006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33088/0 (socket says 10.20.193.78:33088) 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f636645d640 1 -- 10.20.193.78:0/290826970 learned_addr learned my addr 10.20.193.78:0/290826970 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 1 --2- 10.20.193.78:0/290826970 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636009c4a0 0x7f636013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient(hunting): get_auth_request con 0x7f636009c4a0 auth_method 0 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6366c5e640 1 --2- 10.20.193.78:0/290826970 >> v2:10.20.193.78:3300/0 conn(0x7f636006d380 0x7f636013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6366c5e640 10 monclient(hunting): get_auth_request con 0x7f636006d380 auth_method 0 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6366c5e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6366c5e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6366c5e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6366c5e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6366c5e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6366c5e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient(hunting): handle_auth_done global_id 74105 payload 274 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 1 -- 10.20.193.78:0/290826970 >> v2:10.20.193.78:3300/0 conn(0x7f636006d380 msgr2=0x7f636013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 1 --2- 10.20.193.78:0/290826970 >> v2:10.20.193.78:3300/0 conn(0x7f636006d380 0x7f636013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 1 -- 10.20.193.78:0/290826970 >> v1:10.20.193.78:6789/0 conn(0x7f636006c890 legacy=0x7f636006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:38.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient: _finish_hunting 0 2026-02-17T05:50:38.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 1 monclient: found mon.noname-b 2026-02-17T05:50:38.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:38.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:50:38.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.773+0000 7f6365c5c640 1 -- 10.20.193.78:0/290826970 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636009cf60 con 0x7f636009c4a0 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.775+0000 7f6365c5c640 1 -- 10.20.193.78:0/290826970 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636009c4a0 msgr2=0x7f636013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.775+0000 7f6365c5c640 1 --2- 10.20.193.78:0/290826970 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636009c4a0 0x7f636013b3a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63540098a0 tx=0x7f635402ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56a2e79c67dab3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.775+0000 7f636545b640 1 -- 10.20.193.78:0/290826970 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635403c070 con 0x7f636009c4a0 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: dump: 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.778 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: _finish_auth 0 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: _check_auth_tickets 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 1 -- 10.20.193.78:0/290826970 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635402fc00 con 0x7f636009c4a0 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 5 monclient: authenticate success, global_id 74105 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 1 -- 10.20.193.78:0/290826970 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354037440 con 0x7f636009c4a0 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 4 set_mon_vals no callback set 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f636545b640 10 monclient: dump: 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.779 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 10 monclient: shutdown 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 -- 10.20.193.78:0/290826970 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636009c4a0 msgr2=0x7f636013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 --2- 10.20.193.78:0/290826970 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636009c4a0 0x7f636013b3a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63540098a0 tx=0x7f635402ed80 comp rx=0 tx=0).stop 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 -- 10.20.193.78:0/290826970 shutdown_connections 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 --2- 10.20.193.78:0/290826970 >> v2:10.20.193.78:3300/0 conn(0x7f636006d380 0x7f636013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 --2- 10.20.193.78:0/290826970 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636009c4a0 0x7f636013b3a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 -- 10.20.193.78:0/290826970 >> 10.20.193.78:0/290826970 conn(0x7f63600a6620 msgr2=0x7f63600a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 -- 10.20.193.78:0/290826970 shutdown_connections 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 -- 10.20.193.78:0/290826970 wait complete. 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 10 monclient: build_initial_monmap 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 10 monclient: monmap: 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.776+0000 7f63686e8640 1 Processor -- start 2026-02-17T05:50:38.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 1 -- start start 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient: init 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636006c890 0x7f63601cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient(hunting): picked mon.noname-c con 0x7f636006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f636006d380 0x7f63601363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient(hunting): picked mon.noname-a con 0x7f636006d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f636009c4a0 0x7f63601cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient(hunting): picked mon.noname-b con 0x7f636009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f636009bfa0 con 0x7f636006d380 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f636009be20 con 0x7f636009c4a0 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f636013de30 con 0x7f636006c890 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f63686e8640 10 monclient(hunting): authenticate will time out at 884.331309s 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f636006d380 0x7f63601363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636006c890 0x7f63601cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f636006d380 0x7f63601363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49008/0 (socket says 10.20.193.78:49008) 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 1 -- 10.20.193.78:0/1795775928 learned_addr learned my addr 10.20.193.78:0/1795775928 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:38.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 10 monclient(hunting): get_auth_request con 0x7f636006d380 auth_method 0 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient(hunting): get_auth_request con 0x7f636006c890 auth_method 0 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient(hunting): handle_auth_done global_id 74108 payload 824 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f636009c4a0 msgr2=0x7f63601cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6366c5e640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f636009c4a0 0x7f63601cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f636009c4a0 0x7f63601cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f636006d380 msgr2=0x7f63601363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f636006d380 0x7f63601363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient: _finish_hunting 0 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 1 monclient: found mon.noname-c 2026-02-17T05:50:38.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:38.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:38.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f636645d640 1 -- 10.20.193.78:0/1795775928 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63601cfc40 con 0x7f636006c890 2026-02-17T05:50:38.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6365c5c640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f636006d380 0x7f63601363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:50:38.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6366c5e640 10 monclient: get_auth_request con 0x7f636009c4a0 auth_method 0 2026-02-17T05:50:38.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.777+0000 7f6366c5e640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f636009c4a0 0x7f63601cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f636645d640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636006c890 msgr2=0x7f63601cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f636645d640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636006c890 0x7f63601cf1e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f635000c4a0 tx=0x7f635000c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350007070 con 0x7f636006c890 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: dump: 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.785 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: _finish_auth 0 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6350007210 con 0x7f636006c890 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350007600 con 0x7f636006c890 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6364c5a640 4 set_mon_vals no callback set 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f6346ffd640 10 monclient: dump: 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.786 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f63686e8640 5 monclient: authenticate success, global_id 74108 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f63686e8640 10 monclient: _renew_subs 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f63686e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.779+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6360122ac0 con 0x7f636006c890 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f63686e8640 10 monclient: _renew_subs 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f63686e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f636013d9d0 con 0x7f636006c890 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f63686e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f63686e8640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6360136910 0x7f63601cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f63686e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f636006f730 con 0x7f6360136910 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f635001b400 con 0x7f636006c890 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6365c5c640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6360136910 0x7f63601cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6365c5c640 10 monclient: get_auth_request con 0x7f6360136910 auth_method 0 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6365c5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6365c5c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6365c5c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6346ffd640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f633003df30 0x7f63300403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(182..182 src has 117..182) ==== 4827+0+0 (secure 0 0 0) 0x7f63500524e0 con 0x7f636006c890 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.780+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.781+0000 7f6366c5e640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f633003df30 0x7f63300403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.781+0000 7f6366c5e640 10 monclient: get_auth_request con 0x7f633003df30 auth_method 0 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.781+0000 7f6365c5c640 10 monclient(hunting): handle_auth_done global_id 44338 payload 824 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.781+0000 7f6365c5c640 1 -- 10.20.193.78:0/1795775928 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f636013d7b0 con 0x7f6360136910 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.783+0000 7f6365c5c640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6360136910 msgr2=0x7f63601cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.783+0000 7f6365c5c640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6360136910 0x7f63601cc220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63540098a0 tx=0x7f63540047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.783+0000 7f6366c5e640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f633003df30 0x7f63300403f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6360121a40 tx=0x7f635c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.783+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354048070 con 0x7f6360136910 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.783+0000 7f6346ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.783+0000 7f6346ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.783+0000 7f6346ffd640 10 monclient: dump: 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.784+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f635403d040 con 0x7f6360136910 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.784+0000 7f6346ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.784+0000 7f6346ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:38.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.784+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6360136910 msgr2=0x7f63601cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.784+0000 7f6346ffd640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6360136910 0x7f63601cc220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63540098a0 tx=0x7f63540047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.784+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6360136910 msgr2=0x7f63601cc220 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:38.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.784+0000 7f6346ffd640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6360136910 0x7f63601cc220 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63540098a0 tx=0x7f63540047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f63686e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f63686e8640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f63601cc760 0x7f63601da570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f63686e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f63601ccb70 con 0x7f63601cc760 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f63601cc760 0x7f63601da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 10 monclient: get_auth_request con 0x7f63601cc760 auth_method 0 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 10 monclient(hunting): handle_auth_done global_id 44341 payload 824 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.787+0000 7f6365c5c640 1 -- 10.20.193.78:0/1795775928 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6360074980 con 0x7f63601cc760 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.789+0000 7f6365c5c640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f63601cc760 msgr2=0x7f63601da570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:38.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.789+0000 7f6365c5c640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f63601cc760 0x7f63601da570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f636006f8b0 tx=0x7f6354009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.789+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635403c040 con 0x7f63601cc760 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.789+0000 7f6346ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.789+0000 7f6346ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.789+0000 7f6346ffd640 10 monclient: dump: 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f6354057390 con 0x7f63601cc760 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6346ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6346ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f63601cc760 msgr2=0x7f63601da570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6346ffd640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f63601cc760 0x7f63601da570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f636006f8b0 tx=0x7f6354009290 comp rx=0 tx=0).stop 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6346ffd640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f63601cc760 msgr2=0x7f63601da570 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:38.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6346ffd640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f63601cc760 0x7f63601da570 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f636006f8b0 tx=0x7f6354009290 comp rx=0 tx=0).stop 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 46, 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 162437, 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:38.792 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.793 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:38.794 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.796 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:38.797 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:38.798 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:38.799 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:38.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f633003df30 msgr2=0x7f63300403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f633003df30 0x7f63300403f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6360121a40 tx=0x7f635c00b040 comp rx=0 tx=0).stop 2026-02-17T05:50:38.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 10 monclient: shutdown 2026-02-17T05:50:38.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:38.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636006c890 msgr2=0x7f63601cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 1 --2- 10.20.193.78:0/1795775928 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f636006c890 0x7f63601cf1e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f635000c4a0 tx=0x7f635000c970 comp rx=0 tx=0).stop 2026-02-17T05:50:38.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6366c5e640 1 -- 10.20.193.78:0/1795775928 reap_dead start 2026-02-17T05:50:38.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f6366c5e640 1 -- 10.20.193.78:0/1795775928 reap_dead start 2026-02-17T05:50:38.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 shutdown_connections 2026-02-17T05:50:38.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 >> 10.20.193.78:0/1795775928 conn(0x7f63600a6620 msgr2=0x7f63600a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:38.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 shutdown_connections 2026-02-17T05:50:38.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.790+0000 7f63686e8640 1 -- 10.20.193.78:0/1795775928 wait complete. 2026-02-17T05:50:38.801 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:50:38.801 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:50:38.801 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:50:38.809 INFO:tasks.rados.rados.0.trial115.stdout:851: done (2 left) 2026-02-17T05:50:38.809 INFO:tasks.rados.rados.0.trial115.stdout:852: snap_create 2026-02-17T05:50:38.809 INFO:tasks.rados.rados.0.trial115.stdout:849: expect (ObjNum 266 snap 54 seq_num 266) 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.869+0000 7f9533f87640 10 monclient: get_monmap_and_config 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.869+0000 7f9533f87640 10 monclient: build_initial_monmap 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.869+0000 7f9533f87640 10 monclient: monmap: 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:38.870850+0000 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:38.870850+0000 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:38.869 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 1 Processor -- start 2026-02-17T05:50:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 1 -- start start 2026-02-17T05:50:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient: init 2026-02-17T05:50:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c06c890 0x7f952c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient(hunting): picked mon.noname-b con 0x7f952c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f952c09c4a0 0x7f952c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient(hunting): picked mon.noname-a con 0x7f952c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f952c134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f952c06daf0 con 0x7f952c134df0 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f952c0a0880 con 0x7f952c09c4a0 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f952c06d6f0 con 0x7f952c06c890 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9533f87640 10 monclient(hunting): authenticate will time out at 884.424394s 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c06c890 0x7f952c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f952c09c4a0 0x7f952c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c06c890 0x7f952c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40108/0 (socket says 10.20.193.78:40108) 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 1 -- 10.20.193.78:0/3014821673 learned_addr learned my addr 10.20.193.78:0/3014821673 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient(hunting): get_auth_request con 0x7f952c06c890 auth_method 0 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 10 monclient(hunting): get_auth_request con 0x7f952c09c4a0 auth_method 0 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient(hunting): handle_auth_done global_id 74111 payload 274 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 1 -- 10.20.193.78:0/3014821673 >> v2:10.20.193.78:3300/0 conn(0x7f952c09c4a0 msgr2=0x7f952c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 1 --2- 10.20.193.78:0/3014821673 >> v2:10.20.193.78:3300/0 conn(0x7f952c09c4a0 0x7f952c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 1 -- 10.20.193.78:0/3014821673 >> v1:10.20.193.115:6789/0 conn(0x7f952c134df0 legacy=0x7f952c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient: _finish_hunting 0 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 1 monclient: found mon.noname-b 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f9531cfc640 1 -- 10.20.193.78:0/3014821673 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c09cc80 con 0x7f952c06c890 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.870+0000 7f95314fb640 1 --2- 10.20.193.78:0/3014821673 >> v2:10.20.193.78:3300/0 conn(0x7f952c09c4a0 0x7f952c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9531cfc640 1 -- 10.20.193.78:0/3014821673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c06c890 msgr2=0x7f952c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9531cfc640 1 --2- 10.20.193.78:0/3014821673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c06c890 0x7f952c06cc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f951c009970 tx=0x7f951c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0981444fe5880fd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 1 -- 10.20.193.78:0/3014821673 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c03c070 con 0x7f952c06c890 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: dump: 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: _finish_auth 0 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: _check_auth_tickets 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 5 monclient: authenticate success, global_id 74111 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 1 -- 10.20.193.78:0/3014821673 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c02fcd0 con 0x7f952c06c890 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 1 -- 10.20.193.78:0/3014821673 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c037660 con 0x7f952c06c890 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9530cfa640 10 monclient: dump: 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.873 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 4 set_mon_vals no callback set 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 10 monclient: shutdown 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 1 -- 10.20.193.78:0/3014821673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c06c890 msgr2=0x7f952c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 1 --2- 10.20.193.78:0/3014821673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c06c890 0x7f952c06cc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f951c009970 tx=0x7f951c02ee90 comp rx=0 tx=0).stop 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 1 -- 10.20.193.78:0/3014821673 shutdown_connections 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 1 --2- 10.20.193.78:0/3014821673 >> v2:10.20.193.78:3300/0 conn(0x7f952c09c4a0 0x7f952c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 1 --2- 10.20.193.78:0/3014821673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c06c890 0x7f952c06cc70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 1 -- 10.20.193.78:0/3014821673 >> 10.20.193.78:0/3014821673 conn(0x7f952c0a6620 msgr2=0x7f952c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 1 -- 10.20.193.78:0/3014821673 shutdown_connections 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.873+0000 7f9533f87640 1 -- 10.20.193.78:0/3014821673 wait complete. 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient: build_initial_monmap 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient: monmap: 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 1 Processor -- start 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 1 -- start start 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient: init 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f952c06c890 0x7f952c1cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient(hunting): picked mon.noname-a con 0x7f952c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c09c4a0 0x7f952c123990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient(hunting): picked mon.noname-c con 0x7f952c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f952c134df0 0x7f952c123ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient(hunting): picked mon.noname-b con 0x7f952c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f952c0a01a0 con 0x7f952c06c890 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f952c0a0020 con 0x7f952c134df0 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f952c09be20 con 0x7f952c09c4a0 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9533f87640 10 monclient(hunting): authenticate will time out at 884.428560s 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f952c06c890 0x7f952c1cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f952c06c890 0x7f952c1cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49024/0 (socket says 10.20.193.78:49024) 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 -- 10.20.193.78:0/1756514450 learned_addr learned my addr 10.20.193.78:0/1756514450 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient(hunting): get_auth_request con 0x7f952c06c890 auth_method 0 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95314fb640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c09c4a0 0x7f952c123990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95314fb640 10 monclient(hunting): get_auth_request con 0x7f952c09c4a0 auth_method 0 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95314fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95314fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:38.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95314fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95314fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95314fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95314fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient(hunting): handle_auth_done global_id 54330 payload 824 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 -- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c09c4a0 msgr2=0x7f952c123990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c09c4a0 0x7f952c123990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 -- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f952c134df0 msgr2=0x7f952c123ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95324fd640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f952c134df0 0x7f952c123ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f952c134df0 0x7f952c123ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient: _finish_hunting 0 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 monclient: found mon.noname-a 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f9531cfc640 1 -- 10.20.193.78:0/1756514450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c127240 con 0x7f952c06c890 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95324fd640 10 monclient: get_auth_request con 0x7f952c134df0 auth_method 0 2026-02-17T05:50:38.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.874+0000 7f95324fd640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f952c134df0 0x7f952c123ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.876+0000 7f9531cfc640 1 -- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f952c06c890 msgr2=0x7f952c1cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.876+0000 7f9531cfc640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f952c06c890 0x7f952c1cf6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f951c009970 tx=0x7f951c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 1 -- 10.20.193.78:0/1756514450 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c03c040 con 0x7f952c06c890 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: dump: 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 5 monclient: authenticate success, global_id 54330 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 1 -- 10.20.193.78:0/1756514450 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c047070 con 0x7f952c06c890 2026-02-17T05:50:38.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 10 monclient: _renew_subs 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952c126000 con 0x7f952c06c890 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 1 -- 10.20.193.78:0/1756514450 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c042890 con 0x7f952c06c890 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951bfff640 4 set_mon_vals no callback set 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 10 monclient: dump: 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 10 monclient: _renew_subs 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f952c0a16d0 con 0x7f952c06c890 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f952c074980 con 0x7f952c06c890 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.877+0000 7f951a7fc640 1 -- 10.20.193.78:0/1756514450 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f951c042a30 con 0x7f952c06c890 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.878+0000 7f951a7fc640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94fc03deb0 0x7f94fc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.878+0000 7f951a7fc640 1 -- 10.20.193.78:0/1756514450 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(183..183 src has 117..183) ==== 4843+0+0 (secure 0 0 0) 0x7f951c077ac0 con 0x7f952c06c890 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.878+0000 7f95314fb640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94fc03deb0 0x7f94fc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:38.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.878+0000 7f95314fb640 10 monclient: get_auth_request con 0x7f94fc03deb0 auth_method 0 2026-02-17T05:50:38.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.880+0000 7f951a7fc640 1 -- 10.20.193.78:0/1756514450 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f951c03f380 con 0x7f952c06c890 2026-02-17T05:50:38.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.880+0000 7f951a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:38.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.880+0000 7f951a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:38.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.880+0000 7f95314fb640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94fc03deb0 0x7f94fc040370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f952c127450 tx=0x7f9520009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: finishing write tid 1 to trial11545420-33 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: finishing write tid 2 to trial11545420-33 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: finishing write tid 3 to trial11545420-33 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: finishing write tid 4 to trial11545420-33 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: finishing write tid 5 to trial11545420-33 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: finishing write tid 6 to trial11545420-33 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: oid 33 updating version 0 to 301 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: oid 33 updating version 301 to 304 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: oid 33 version 304 is already newer than 300 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: oid 33 version 304 is already newer than 303 2026-02-17T05:50:38.890 INFO:tasks.rados.rados.0.trial115.stdout:850: oid 33 version 304 is already newer than 302 2026-02-17T05:50:38.891 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 304 (ObjNum 342 snap 84 seq_num 342) dirty exists 2026-02-17T05:50:38.891 INFO:tasks.rados.rados.0.trial115.stdout:850: left oid 33 (ObjNum 342 snap 84 seq_num 342) 2026-02-17T05:50:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.993+0000 7f9533f87640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:50:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.993+0000 7f9533f87640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:50:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.993+0000 7f9533f87640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.993+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f952c1359e0 con 0x7f952c06c890 2026-02-17T05:50:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.993+0000 7f951a7fc640 1 -- 10.20.193.78:0/1756514450 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f951c042e00 con 0x7f952c06c890 2026-02-17T05:50:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.993+0000 7f951a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:50:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.993+0000 7f951a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:38.994 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.993+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:38.994 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.993+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=4053055469,config=692619927,health=1535303502,kv=449989585,logm=389863974}) 2026-02-17T05:50:38.994 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.993+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=11,kv=11,logm=65} crc {auth=1834656158,config=692619927,health=1647462659,kv=3215367626,logm=2908365694}) 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.993+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.993+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=4053055469,config=692619927,health=1535303502,kv=449989585,logm=389863974}) 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.993+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=12,kv=11,logm=61} crc {auth=4062714869,config=692619927,health=1535303502,kv=449989585,logm=1027794727}) 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1336314364}) 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1036222073}) 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1336314364}) 2026-02-17T05:50:38.995 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148966764}) 2026-02-17T05:50:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94fc03deb0 msgr2=0x7f94fc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94fc03deb0 0x7f94fc040370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f952c127450 tx=0x7f9520009290 comp rx=0 tx=0).stop 2026-02-17T05:50:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 10 monclient: shutdown 2026-02-17T05:50:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f952c06c890 msgr2=0x7f952c1cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f952c06c890 0x7f952c1cf6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f951c009970 tx=0x7f951c004830 comp rx=0 tx=0).stop 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 shutdown_connections 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94fc03deb0 0x7f94fc040370 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f952c134df0 0x7f952c123ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f952c09c4a0 0x7f952c123990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 --2- 10.20.193.78:0/1756514450 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f952c06c890 0x7f952c1cf6a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 >> 10.20.193.78:0/1756514450 conn(0x7f952c0a6620 msgr2=0x7f952c136410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 shutdown_connections 2026-02-17T05:50:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:38.994+0000 7f9533f87640 1 -- 10.20.193.78:0/1756514450 wait complete. 2026-02-17T05:50:38.999 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:38.999 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1752017239}) 2026-02-17T05:50:38.999 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2253197686}) 2026-02-17T05:50:38.999 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:38.999 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1752017239}) 2026-02-17T05:50:38.999 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.994+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1497517541}) 2026-02-17T05:50:38.999 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4269605502}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=135037279}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4269605502}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=909895377}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=185112676}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1988999559}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=185112676}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3673249771}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1384337138}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4184866964}) 2026-02-17T05:50:39.000 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.001 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1384337138}) 2026-02-17T05:50:39.001 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:38.995+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1117174974}) 2026-02-17T05:50:39.002 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:50:39.009 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.009 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3402412630}) 2026-02-17T05:50:39.009 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3482934677}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3402412630}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825246490}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2078263596}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3749261047}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2078263596}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.007+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3715961692}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411368906}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=516997434}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411368906}) 2026-02-17T05:50:39.010 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3466101643}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2834520934}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=585359994}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2834520934}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=342641771}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1338767138}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1066694603}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1338767138}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.008+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3785255366}) 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.011 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1273082005}) 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3449799943}) 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1273082005}) 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4122317387}) 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=14,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32,osdmap=10} crc {logm=2242358996,mdsmap=1160443688,mgr=267573432,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3466381057,mon_config_key=1912222953,monmap=1784110042,osd_metadata=1611097156,osd_pg_creating=3096085446,osd_snap=744179359,osdmap=2676976364}) 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=31,osdmap=10} crc {logm=2242358996,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3850823448,mon_config_key=2905250389,monmap=1784110042,osd_metadata=3878803090,osd_pg_creating=3096085446,osd_snap=1292630215,osdmap=2676976364}) 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=14,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32,osdmap=10} crc {logm=2242358996,mdsmap=1160443688,mgr=267573432,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3466381057,mon_config_key=1912222953,monmap=1784110042,osd_metadata=1611097156,osd_pg_creating=3096085446,osd_snap=744179359,osdmap=2676976364}) 2026-02-17T05:50:39.012 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.009+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=14,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=31,osdmap=8} crc {logm=2655836475,mdsmap=3140514009,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3850823448,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=3096085446,osd_snap=2515656663,osdmap=4019410925}) 2026-02-17T05:50:39.013 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.011+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.013 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.011+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=394357191}) 2026-02-17T05:50:39.013 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.011+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1651701312}) 2026-02-17T05:50:39.013 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.011+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:50:39.013 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.011+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=394357191}) 2026-02-17T05:50:39.013 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:50:39.011+0000 7f2784e8f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=97} crc {osdmap=76245800}) 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:849: done (2 left) 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:850: done (1 left) 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:852: done (0 left) 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:853: read oid 48 snap -1 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:853: expect (ObjNum 320 snap 77 seq_num 320) 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:854: read oid 2 snap -1 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:854: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:855: rollback oid 11 current snap is 85 2026-02-17T05:50:39.813 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 73 2026-02-17T05:50:39.814 INFO:tasks.rados.rados.0.trial115.stdout:856: read oid 32 snap -1 2026-02-17T05:50:39.814 INFO:tasks.rados.rados.0.trial115.stdout:856: expect deleted 2026-02-17T05:50:39.814 INFO:tasks.rados.rados.0.trial115.stdout:857: snap_remove snap 68 2026-02-17T05:50:40.003 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:50:40.004 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:40.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.072+0000 7f6cd7577640 10 monclient: get_monmap_and_config 2026-02-17T05:50:40.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.072+0000 7f6cd7577640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.072+0000 7f6cd7577640 10 monclient: monmap: 2026-02-17T05:50:40.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:40.074049+0000 2026-02-17T05:50:40.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:40.074049+0000 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 1 Processor -- start 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 1 -- start start 2026-02-17T05:50:40.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient: init 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6cd806c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd80a0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient(hunting): picked mon.noname-b con 0x7f6cd806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6cd813a0a0 0x7f6cd813c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient(hunting): picked mon.noname-a con 0x7f6cd813a0a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd809dd50 con 0x7f6cd806c890 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd80741c0 con 0x7f6cd813a0a0 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd809dad0 con 0x7f6cd806d380 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd7577640 10 monclient(hunting): authenticate will time out at 885.627619s 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd80a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6cd813a0a0 0x7f6cd813c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd80a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40132/0 (socket says 10.20.193.78:40132) 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 1 -- 10.20.193.78:0/715446258 learned_addr learned my addr 10.20.193.78:0/715446258 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient(hunting): get_auth_request con 0x7f6cd813a0a0 auth_method 0 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 10 monclient(hunting): get_auth_request con 0x7f6cd806d380 auth_method 0 2026-02-17T05:50:40.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient(hunting): handle_auth_done global_id 54333 payload 274 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 1 -- 10.20.193.78:0/715446258 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 msgr2=0x7f6cd80a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 1 --2- 10.20.193.78:0/715446258 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd80a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 1 -- 10.20.193.78:0/715446258 >> v1:10.20.193.115:6789/0 conn(0x7f6cd806c890 legacy=0x7f6cd806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 1 monclient: found mon.noname-a 2026-02-17T05:50:40.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:40.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.073+0000 7f6cd6d76640 1 -- 10.20.193.78:0/715446258 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd809edf0 con 0x7f6cd813a0a0 2026-02-17T05:50:40.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd6d76640 1 -- 10.20.193.78:0/715446258 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd813a0a0 msgr2=0x7f6cd813c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd6d76640 1 --2- 10.20.193.78:0/715446258 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd813a0a0 0x7f6cd813c490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009d90 tx=0x7f6cc802f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb3a57ad99f8b7d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 1 -- 10.20.193.78:0/715446258 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc803c070 con 0x7f6cd813a0a0 2026-02-17T05:50:40.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:40.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: dump: 2026-02-17T05:50:40.087 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.087 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 1 -- 10.20.193.78:0/715446258 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc8037440 con 0x7f6cd813a0a0 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 5 monclient: authenticate success, global_id 54333 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 1 -- 10.20.193.78:0/715446258 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8037810 con 0x7f6cd813a0a0 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 4 set_mon_vals no callback set 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd5573640 10 monclient: dump: 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.089 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 10 monclient: shutdown 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 1 -- 10.20.193.78:0/715446258 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd813a0a0 msgr2=0x7f6cd813c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 1 --2- 10.20.193.78:0/715446258 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd813a0a0 0x7f6cd813c490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009d90 tx=0x7f6cc802f190 comp rx=0 tx=0).stop 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 1 -- 10.20.193.78:0/715446258 shutdown_connections 2026-02-17T05:50:40.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 1 --2- 10.20.193.78:0/715446258 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd813a0a0 0x7f6cd813c490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 1 --2- 10.20.193.78:0/715446258 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd80a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 1 -- 10.20.193.78:0/715446258 >> 10.20.193.78:0/715446258 conn(0x7f6cd80a65f0 msgr2=0x7f6cd80a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 1 -- 10.20.193.78:0/715446258 shutdown_connections 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.076+0000 7f6cd7577640 1 -- 10.20.193.78:0/715446258 wait complete. 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient: monmap: 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.091 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 1 Processor -- start 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 1 -- start start 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient: init 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd806c890 0x7f6cd8072a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient(hunting): picked mon.noname-a con 0x7f6cd806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd8072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient(hunting): picked mon.noname-c con 0x7f6cd806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cd813a0a0 0x7f6cd80734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient(hunting): picked mon.noname-b con 0x7f6cd813a0a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd813cb50 con 0x7f6cd806c890 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd813c9d0 con 0x7f6cd813a0a0 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd813ccd0 con 0x7f6cd806d380 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd7577640 10 monclient(hunting): authenticate will time out at 885.631789s 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd8072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd6575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd806c890 0x7f6cd8072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd8072f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40146/0 (socket says 10.20.193.78:40146) 2026-02-17T05:50:40.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 1 -- 10.20.193.78:0/3293629058 learned_addr learned my addr 10.20.193.78:0/3293629058 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 10 monclient(hunting): get_auth_request con 0x7f6cd806d380 auth_method 0 2026-02-17T05:50:40.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd6575640 10 monclient(hunting): get_auth_request con 0x7f6cd806c890 auth_method 0 2026-02-17T05:50:40.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 10 monclient(hunting): handle_auth_done global_id 74120 payload 824 2026-02-17T05:50:40.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 1 -- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cd813a0a0 msgr2=0x7f6cd80734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cd813a0a0 0x7f6cd80734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 1 -- 10.20.193.78:0/3293629058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd806c890 msgr2=0x7f6cd8072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.077+0000 7f6cd5d74640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd806c890 0x7f6cd8072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.078+0000 7f6cd5d74640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.078+0000 7f6cd5d74640 1 monclient: found mon.noname-c 2026-02-17T05:50:40.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.078+0000 7f6cd5d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.078+0000 7f6cd5d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.078+0000 7f6cd5d74640 1 -- 10.20.193.78:0/3293629058 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd811e070 con 0x7f6cd806d380 2026-02-17T05:50:40.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd5d74640 1 -- 10.20.193.78:0/3293629058 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 msgr2=0x7f6cd8072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd5d74640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd8072f80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000e8b0 tx=0x7f6cc000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 1 -- 10.20.193.78:0/3293629058 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0017070 con 0x7f6cd806d380 2026-02-17T05:50:40.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:40.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: dump: 2026-02-17T05:50:40.098 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.098 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.098 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.098 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.098 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.098 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.099 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.099 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.099 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.099 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 1 -- 10.20.193.78:0/3293629058 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc000cb50 con 0x7f6cd806d380 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 1 -- 10.20.193.78:0/3293629058 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0013470 con 0x7f6cd806d380 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cbeffd640 10 monclient: dump: 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.100 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.101 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.101 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.101 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.101 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.101 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.101 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.101 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd4d72640 4 set_mon_vals no callback set 2026-02-17T05:50:40.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 5 monclient: authenticate success, global_id 74120 2026-02-17T05:50:40.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 10 monclient: _renew_subs 2026-02-17T05:50:40.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd813c450 con 0x7f6cd806d380 2026-02-17T05:50:40.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 10 monclient: _renew_subs 2026-02-17T05:50:40.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd811e350 con 0x7f6cd806d380 2026-02-17T05:50:40.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.080+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd80a0070 con 0x7f6cd806d380 2026-02-17T05:50:40.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.081+0000 7f6cbeffd640 1 -- 10.20.193.78:0/3293629058 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6cc0024060 con 0x7f6cd806d380 2026-02-17T05:50:40.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.081+0000 7f6cbeffd640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ca403deb0 0x7f6ca4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.081+0000 7f6cbeffd640 1 -- 10.20.193.78:0/3293629058 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(184..184 src has 117..184) ==== 4815+0+0 (secure 0 0 0) 0x7f6cc0052830 con 0x7f6cd806d380 2026-02-17T05:50:40.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.081+0000 7f6cd6575640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ca403deb0 0x7f6ca4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.081+0000 7f6cd6575640 10 monclient: get_auth_request con 0x7f6ca403deb0 auth_method 0 2026-02-17T05:50:40.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.084+0000 7f6cbeffd640 1 -- 10.20.193.78:0/3293629058 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6cc0013e10 con 0x7f6cd806d380 2026-02-17T05:50:40.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.084+0000 7f6cbeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.084+0000 7f6cbeffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:40.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.084+0000 7f6cd6575640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ca403deb0 0x7f6ca4040370 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8073cb0 tx=0x7f6ccc007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.196+0000 7f6cd7577640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:40.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.196+0000 7f6cd7577640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:40.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.196+0000 7f6cd7577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.196+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6cd8135af0 con 0x7f6cd806d380 2026-02-17T05:50:40.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.196+0000 7f6cbeffd640 1 -- 10.20.193.78:0/3293629058 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f6cc00123d0 con 0x7f6cd806d380 2026-02-17T05:50:40.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.196+0000 7f6cbeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:40.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.196+0000 7f6cbeffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:40.198 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ca403deb0 msgr2=0x7f6ca4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ca403deb0 0x7f6ca4040370 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8073cb0 tx=0x7f6ccc007f30 comp rx=0 tx=0).stop 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 10 monclient: shutdown 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 msgr2=0x7f6cd8072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd8072f80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000e8b0 tx=0x7f6cc000ed80 comp rx=0 tx=0).stop 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 shutdown_connections 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ca403deb0 0x7f6ca4040370 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cd813a0a0 0x7f6cd80734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cd806d380 0x7f6cd8072f80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 --2- 10.20.193.78:0/3293629058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cd806c890 0x7f6cd8072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 >> 10.20.193.78:0/3293629058 conn(0x7f6cd80a65f0 msgr2=0x7f6cd813b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 shutdown_connections 2026-02-17T05:50:40.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.197+0000 7f6cd7577640 1 -- 10.20.193.78:0/3293629058 wait complete. 2026-02-17T05:50:40.204 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:50:40.204 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:50:40.204 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:50:40.205 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.272+0000 7f1a84e5b640 10 monclient: get_monmap_and_config 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.272+0000 7f1a84e5b640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.272+0000 7f1a84e5b640 10 monclient: monmap: 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:40.274107+0000 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:40.274107+0000 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.272 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.273 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:40.273 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:40.273 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:40.273 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:40.273 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 1 Processor -- start 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 1 -- start start 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient: init 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f1a8006c890 0x7f1a8006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient(hunting): picked mon.noname-c con 0x7f1a8006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8009c4a0 0x7f1a8006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient(hunting): picked mon.noname-b con 0x7f1a8009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1a80134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8006daf0 con 0x7f1a80134df0 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f1a800a0880 con 0x7f1a8006c890 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8006d6f0 con 0x7f1a8009c4a0 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a84e5b640 10 monclient(hunting): authenticate will time out at 885.827678s 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8009c4a0 0x7f1a8006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8009c4a0 0x7f1a8006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40156/0 (socket says 10.20.193.78:40156) 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7ed76640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f1a80134df0 0x7f1a80135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33096/0 (socket says 10.20.193.78:33096) 2026-02-17T05:50:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 1 -- 10.20.193.78:0/1755101475 learned_addr learned my addr 10.20.193.78:0/1755101475 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient(hunting): get_auth_request con 0x7f1a8009c4a0 auth_method 0 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7e575640 1 --2- 10.20.193.78:0/1755101475 >> v2:10.20.193.115:3300/0 conn(0x7f1a8006c890 0x7f1a8006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_done global_id 74123 payload 274 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 1 -- 10.20.193.78:0/1755101475 >> v2:10.20.193.115:3300/0 conn(0x7f1a8006c890 msgr2=0x7f1a8006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 1 --2- 10.20.193.78:0/1755101475 >> v2:10.20.193.115:3300/0 conn(0x7f1a8006c890 0x7f1a8006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 1 -- 10.20.193.78:0/1755101475 >> v1:10.20.193.78:6789/0 conn(0x7f1a80134df0 legacy=0x7f1a80135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 1 monclient: found mon.noname-b 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.273+0000 7f1a7dd74640 1 -- 10.20.193.78:0/1755101475 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8009cc80 con 0x7f1a8009c4a0 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7dd74640 1 -- 10.20.193.78:0/1755101475 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8009c4a0 msgr2=0x7f1a8006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7dd74640 1 --2- 10.20.193.78:0/1755101475 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8009c4a0 0x7f1a8006d1b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a8006d870 tx=0x7f1a6802ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d866e17d3327ae0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 1 -- 10.20.193.78:0/1755101475 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6803c070 con 0x7f1a8009c4a0 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: dump: 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.276 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 1 -- 10.20.193.78:0/1755101475 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a6802fda0 con 0x7f1a8009c4a0 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 5 monclient: authenticate success, global_id 74123 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 1 -- 10.20.193.78:0/1755101475 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68037660 con 0x7f1a8009c4a0 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a7d573640 10 monclient: dump: 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 4 set_mon_vals no callback set 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 10 monclient: shutdown 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 1 -- 10.20.193.78:0/1755101475 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8009c4a0 msgr2=0x7f1a8006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 1 --2- 10.20.193.78:0/1755101475 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8009c4a0 0x7f1a8006d1b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a8006d870 tx=0x7f1a6802ee80 comp rx=0 tx=0).stop 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 1 -- 10.20.193.78:0/1755101475 shutdown_connections 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 1 --2- 10.20.193.78:0/1755101475 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8009c4a0 0x7f1a8006d1b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 1 --2- 10.20.193.78:0/1755101475 >> v2:10.20.193.115:3300/0 conn(0x7f1a8006c890 0x7f1a8006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 1 -- 10.20.193.78:0/1755101475 >> 10.20.193.78:0/1755101475 conn(0x7f1a800a6620 msgr2=0x7f1a800a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 1 -- 10.20.193.78:0/1755101475 shutdown_connections 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.276+0000 7f1a84e5b640 1 -- 10.20.193.78:0/1755101475 wait complete. 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient: monmap: 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:40.278 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 1 Processor -- start 2026-02-17T05:50:40.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 1 -- start start 2026-02-17T05:50:40.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient: init 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8006c890 0x7f1a801cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient(hunting): picked mon.noname-c con 0x7f1a8006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a8009c4a0 0x7f1a801333c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient(hunting): picked mon.noname-b con 0x7f1a8009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80134df0 0x7f1a80133900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient(hunting): picked mon.noname-a con 0x7f1a80134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800a01a0 con 0x7f1a80134df0 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800a0020 con 0x7f1a8009c4a0 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8009be20 con 0x7f1a8006c890 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a84e5b640 10 monclient(hunting): authenticate will time out at 885.831617s 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8006c890 0x7f1a801cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8006c890 0x7f1a801cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40170/0 (socket says 10.20.193.78:40170) 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 -- 10.20.193.78:0/4227796418 learned_addr learned my addr 10.20.193.78:0/4227796418 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient(hunting): get_auth_request con 0x7f1a8006c890 auth_method 0 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80134df0 0x7f1a80133900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 10 monclient(hunting): get_auth_request con 0x7f1a80134df0 auth_method 0 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80134df0 msgr2=0x7f1a80133900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80134df0 msgr2=0x7f1a80133900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80134df0 0x7f1a80133900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7ed76640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80134df0 0x7f1a80133900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7dd74640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a8009c4a0 0x7f1a801333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient(hunting): handle_auth_done global_id 74126 payload 824 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a8009c4a0 msgr2=0x7f1a801333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a8009c4a0 0x7f1a801333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80134df0 msgr2=0x7f1a80133900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80134df0 0x7f1a80133900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 monclient: found mon.noname-c 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7e575640 1 -- 10.20.193.78:0/4227796418 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a800a17b0 con 0x7f1a8006c890 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7dd74640 10 monclient: get_auth_request con 0x7f1a8009c4a0 auth_method 0 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.277+0000 7f1a7dd74640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a8009c4a0 0x7f1a801333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.279+0000 7f1a7e575640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8006c890 msgr2=0x7f1a801cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.279+0000 7f1a7e575640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8006c890 0x7f1a801cf1c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400c7f0 tx=0x7f1a7400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74015070 con 0x7f1a8006c890 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: dump: 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.282 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a74007c40 con 0x7f1a8006c890 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 5 monclient: authenticate success, global_id 74126 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 10 monclient: _renew_subs 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a80126b00 con 0x7f1a8006c890 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74004020 con 0x7f1a8006c890 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 10 monclient: dump: 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7cd72640 4 set_mon_vals no callback set 2026-02-17T05:50:40.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 10 monclient: _renew_subs 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a80124af0 con 0x7f1a8006c890 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80122ea0 0x7f1a80123280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80074980 con 0x7f1a80122ea0 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80122ea0 0x7f1a80123280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 10 monclient: get_auth_request con 0x7f1a80122ea0 auth_method 0 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_done global_id 54339 payload 824 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a7dd74640 1 -- 10.20.193.78:0/4227796418 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a80126690 con 0x7f1a80122ea0 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1a74004280 con 0x7f1a8006c890 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a4c03df30 0x7f1a4c0403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.280+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(184..184 src has 117..184) ==== 4815+0+0 (secure 0 0 0) 0x7f1a74052830 con 0x7f1a8006c890 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.281+0000 7f1a7ed76640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a4c03df30 0x7f1a4c0403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.281+0000 7f1a7ed76640 10 monclient: get_auth_request con 0x7f1a4c03df30 auth_method 0 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.282+0000 7f1a7dd74640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80122ea0 msgr2=0x7f1a80123280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.282+0000 7f1a7dd74640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80122ea0 0x7f1a80123280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a800a7490 tx=0x7f1a68004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.282+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6803c040 con 0x7f1a80122ea0 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.282+0000 7f1a66ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.282+0000 7f1a66ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.282+0000 7f1a66ffd640 10 monclient: dump: 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.283+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f1a68042620 con 0x7f1a80122ea0 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.283+0000 7f1a66ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.283+0000 7f1a66ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.283+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80122ea0 msgr2=0x7f1a80123280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.283+0000 7f1a66ffd640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80122ea0 0x7f1a80123280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a800a7490 tx=0x7f1a68004830 comp rx=0 tx=0).stop 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.283+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80122ea0 msgr2=0x7f1a80123280 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.283+0000 7f1a66ffd640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a80122ea0 0x7f1a80123280 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a800a7490 tx=0x7f1a68004830 comp rx=0 tx=0).stop 2026-02-17T05:50:40.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.283+0000 7f1a7ed76640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a4c03df30 0x7f1a4c0403f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a80126fa0 tx=0x7f1a70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a84e5b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a84e5b640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a801237c0 0x7f1a801da510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a84e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80125870 con 0x7f1a801237c0 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a801237c0 0x7f1a801da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 10 monclient: get_auth_request con 0x7f1a801237c0 auth_method 0 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 10 monclient(hunting): handle_auth_done global_id 54342 payload 824 2026-02-17T05:50:40.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.286+0000 7f1a7dd74640 1 -- 10.20.193.78:0/4227796418 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1a800744d0 con 0x7f1a801237c0 2026-02-17T05:50:40.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a7dd74640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a801237c0 msgr2=0x7f1a801da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a7dd74640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a801237c0 0x7f1a801da510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a680090b0 tx=0x7f1a680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68053070 con 0x7f1a801237c0 2026-02-17T05:50:40.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:40.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 10 monclient: dump: 2026-02-17T05:50:40.289 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f1a6804cb20 con 0x7f1a801237c0 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a801237c0 msgr2=0x7f1a801da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a801237c0 0x7f1a801da510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a680090b0 tx=0x7f1a680047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a801237c0 msgr2=0x7f1a801da510 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a66ffd640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a801237c0 0x7f1a801da510 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a680090b0 tx=0x7f1a680047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:40.290 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 46, 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 109762, 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.291 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:40.292 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.293 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:40.294 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:40.295 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:40.296 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:40.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a4c03df30 msgr2=0x7f1a4c0403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a4c03df30 0x7f1a4c0403f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a80126fa0 tx=0x7f1a70009040 comp rx=0 tx=0).stop 2026-02-17T05:50:40.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 10 monclient: shutdown 2026-02-17T05:50:40.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8006c890 msgr2=0x7f1a801cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8006c890 0x7f1a801cf1c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400c7f0 tx=0x7f1a7400ccc0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a7ed76640 1 -- 10.20.193.78:0/4227796418 reap_dead start 2026-02-17T05:50:40.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 shutdown_connections 2026-02-17T05:50:40.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 --2- 10.20.193.78:0/4227796418 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a8006c890 0x7f1a801cf1c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 >> 10.20.193.78:0/4227796418 conn(0x7f1a800a6620 msgr2=0x7f1a80135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 shutdown_connections 2026-02-17T05:50:40.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.288+0000 7f1a84e5b640 1 -- 10.20.193.78:0/4227796418 wait complete. 2026-02-17T05:50:40.298 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:50:40.298 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:50:40.298 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.365+0000 7f0db5fba640 10 monclient: get_monmap_and_config 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.365+0000 7f0db5fba640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.365+0000 7f0db5fba640 10 monclient: monmap: 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:40.366552+0000 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:40.366552+0000 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.365+0000 7f0db5fba640 1 Processor -- start 2026-02-17T05:50:40.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.365+0000 7f0db5fba640 1 -- start start 2026-02-17T05:50:40.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.365+0000 7f0db5fba640 10 monclient: init 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0db006c890 0x7f0db006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient(hunting): picked mon.noname-a con 0x7f0db006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0db009c4a0 0x7f0db006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient(hunting): picked mon.noname-c con 0x7f0db009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0db0134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db006daf0 con 0x7f0db0134df0 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f0db00a0880 con 0x7f0db006c890 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0db006d6f0 con 0x7f0db009c4a0 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0db5fba640 10 monclient(hunting): authenticate will time out at 885.920070s 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0db006c890 0x7f0db006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0db006c890 0x7f0db006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49088/0 (socket says 10.20.193.78:49088) 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 1 -- 10.20.193.78:0/3572050752 learned_addr learned my addr 10.20.193.78:0/3572050752 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient(hunting): get_auth_request con 0x7f0db006c890 auth_method 0 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daeffd640 1 --2- 10.20.193.78:0/3572050752 >> v2:10.20.193.115:3300/0 conn(0x7f0db009c4a0 0x7f0db006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daeffd640 10 monclient(hunting): get_auth_request con 0x7f0db009c4a0 auth_method 0 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_done global_id 54345 payload 274 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 1 -- 10.20.193.78:0/3572050752 >> v2:10.20.193.115:3300/0 conn(0x7f0db009c4a0 msgr2=0x7f0db006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 1 --2- 10.20.193.78:0/3572050752 >> v2:10.20.193.115:3300/0 conn(0x7f0db009c4a0 0x7f0db006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 1 -- 10.20.193.78:0/3572050752 >> v1:10.20.193.115:6789/0 conn(0x7f0db0134df0 legacy=0x7f0db0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 1 monclient: found mon.noname-a 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:40.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.366+0000 7f0daf7fe640 1 -- 10.20.193.78:0/3572050752 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db009cc80 con 0x7f0db006c890 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.368+0000 7f0daf7fe640 1 -- 10.20.193.78:0/3572050752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db006c890 msgr2=0x7f0db006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.368+0000 7f0daf7fe640 1 --2- 10.20.193.78:0/3572050752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db006c890 0x7f0db006cc70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0db009c980 tx=0x7f0d9c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3319582e3d7400a8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 1 -- 10.20.193.78:0/3572050752 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c03c070 con 0x7f0db006c890 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: dump: 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 1 -- 10.20.193.78:0/3572050752 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c02fca0 con 0x7f0db006c890 2026-02-17T05:50:40.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 5 monclient: authenticate success, global_id 54345 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 1 -- 10.20.193.78:0/3572050752 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c037660 con 0x7f0db006c890 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0dae7fc640 10 monclient: dump: 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 4 set_mon_vals no callback set 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 10 monclient: shutdown 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 1 -- 10.20.193.78:0/3572050752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db006c890 msgr2=0x7f0db006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 1 --2- 10.20.193.78:0/3572050752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db006c890 0x7f0db006cc70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0db009c980 tx=0x7f0d9c02ee60 comp rx=0 tx=0).stop 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 1 -- 10.20.193.78:0/3572050752 shutdown_connections 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 1 --2- 10.20.193.78:0/3572050752 >> v2:10.20.193.115:3300/0 conn(0x7f0db009c4a0 0x7f0db006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 1 --2- 10.20.193.78:0/3572050752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db006c890 0x7f0db006cc70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 1 -- 10.20.193.78:0/3572050752 >> 10.20.193.78:0/3572050752 conn(0x7f0db00a6620 msgr2=0x7f0db00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 1 -- 10.20.193.78:0/3572050752 shutdown_connections 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 1 -- 10.20.193.78:0/3572050752 wait complete. 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.369+0000 7f0db5fba640 10 monclient: monmap: 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 1 Processor -- start 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 1 -- start start 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient: init 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db006c890 0x7f0db01cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient(hunting): picked mon.noname-c con 0x7f0db006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0db009c4a0 0x7f0db01334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient(hunting): picked mon.noname-b con 0x7f0db009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db0134df0 0x7f0db01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient(hunting): picked mon.noname-a con 0x7f0db0134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00a01a0 con 0x7f0db0134df0 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00a0020 con 0x7f0db009c4a0 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0db009be20 con 0x7f0db006c890 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0db5fba640 10 monclient(hunting): authenticate will time out at 885.924325s 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db006c890 0x7f0db01cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db0134df0 0x7f0db01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db006c890 0x7f0db01cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40182/0 (socket says 10.20.193.78:40182) 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 -- 10.20.193.78:0/584728833 learned_addr learned my addr 10.20.193.78:0/584728833 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient(hunting): get_auth_request con 0x7f0db006c890 auth_method 0 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 10 monclient(hunting): get_auth_request con 0x7f0db0134df0 auth_method 0 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_done global_id 74129 payload 824 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0db009c4a0 msgr2=0x7f0db01334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0db009c4a0 0x7f0db01334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db0134df0 msgr2=0x7f0db01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db0134df0 0x7f0db01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 monclient: found mon.noname-c 2026-02-17T05:50:40.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daf7fe640 1 -- 10.20.193.78:0/584728833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db0126ed0 con 0x7f0db006c890 2026-02-17T05:50:40.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.370+0000 7f0daffff640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0db0134df0 0x7f0db01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0daf7fe640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db006c890 msgr2=0x7f0db01cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0daf7fe640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db006c890 0x7f0db01cf2c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c002410 tx=0x7f0d9c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c03c040 con 0x7f0db006c890 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: dump: 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.376 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c047070 con 0x7f0db006c890 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0db5fba640 5 monclient: authenticate success, global_id 74129 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0db5fba640 10 monclient: _renew_subs 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0db5fba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db0125080 con 0x7f0db006c890 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c041800 con 0x7f0db006c890 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0dadffb640 4 set_mon_vals no callback set 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0d8bfff640 10 monclient: dump: 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.377 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0db5fba640 10 monclient: _renew_subs 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0db5fba640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.372+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db00a1710 con 0x7f0db006c890 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0db5fba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0db5fba640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0122fe0 0x7f0db01233c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0db5fba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0db0073b30 con 0x7f0db0122fe0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0d9c0419e0 con 0x7f0db006c890 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0122fe0 0x7f0db01233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 10 monclient: get_auth_request con 0x7f0db0122fe0 auth_method 0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0d8bfff640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0d8003deb0 0x7f0d80040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 10 monclient(hunting): handle_auth_done global_id 74132 payload 824 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daeffd640 1 -- 10.20.193.78:0/584728833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0db0074980 con 0x7f0db0122fe0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(184..184 src has 117..184) ==== 4815+0+0 (secure 0 0 0) 0x7f0d9c077af0 con 0x7f0db006c890 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daffff640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0d8003deb0 0x7f0d80040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.373+0000 7f0daffff640 10 monclient: get_auth_request con 0x7f0d8003deb0 auth_method 0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.375+0000 7f0daeffd640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0122fe0 msgr2=0x7f0db01233c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.375+0000 7f0daeffd640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0122fe0 0x7f0db01233c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0db01261f0 tx=0x7f0da000ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.375+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0017070 con 0x7f0db0122fe0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.375+0000 7f0d8bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.375+0000 7f0d8bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.375+0000 7f0d8bfff640 10 monclient: dump: 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.378 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.376+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f0da000ee90 con 0x7f0db0122fe0 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.376+0000 7f0d8bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.376+0000 7f0d8bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.376+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0122fe0 msgr2=0x7f0db01233c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.376+0000 7f0d8bfff640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0122fe0 0x7f0db01233c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0db01261f0 tx=0x7f0da000ebb0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.376+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0122fe0 msgr2=0x7f0db01233c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.376+0000 7f0d8bfff640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0122fe0 0x7f0db01233c0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0db01261f0 tx=0x7f0da000ebb0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.376+0000 7f0daffff640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0d8003deb0 0x7f0d80040370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0da400d5e0 tx=0x7f0da4004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0db5fba640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0db5fba640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0123900 0x7f0db01da610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0db5fba640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0db0135a20 con 0x7f0db0123900 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0123900 0x7f0db01da610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 10 monclient: get_auth_request con 0x7f0db0123900 auth_method 0 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 10 monclient(hunting): handle_auth_done global_id 74135 payload 824 2026-02-17T05:50:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.379+0000 7f0daeffd640 1 -- 10.20.193.78:0/584728833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0db0124be0 con 0x7f0db0123900 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0daeffd640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0123900 msgr2=0x7f0db01da610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0daeffd640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0123900 0x7f0db01da610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0da0018000 tx=0x7f0da00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da000f040 con 0x7f0db0123900 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 10 monclient: dump: 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f0da0003db0 con 0x7f0db0123900 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0123900 msgr2=0x7f0db01da610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0123900 0x7f0db01da610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0da0018000 tx=0x7f0da00047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0123900 msgr2=0x7f0db01da610 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.381+0000 7f0d8bfff640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db0123900 0x7f0db01da610 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0da0018000 tx=0x7f0da00047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 46, 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 12274, 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.382 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:40.383 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:40.384 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.385 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.386 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:40.387 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0d8003deb0 msgr2=0x7f0d80040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0d8003deb0 0x7f0d80040370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0da400d5e0 tx=0x7f0da4004200 comp rx=0 tx=0).stop 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 10 monclient: shutdown 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db006c890 msgr2=0x7f0db01cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 1 --2- 10.20.193.78:0/584728833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0db006c890 0x7f0db01cf2c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c002410 tx=0x7f0d9c004830 comp rx=0 tx=0).stop 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0daffff640 1 -- 10.20.193.78:0/584728833 reap_dead start 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0daffff640 1 -- 10.20.193.78:0/584728833 reap_dead start 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 shutdown_connections 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 >> 10.20.193.78:0/584728833 conn(0x7f0db00a6620 msgr2=0x7f0db0135c30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 shutdown_connections 2026-02-17T05:50:40.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.382+0000 7f0db5fba640 1 -- 10.20.193.78:0/584728833 wait complete. 2026-02-17T05:50:40.388 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:50:40.388 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:50:40.388 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:50:40.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.455+0000 7febb6c0e640 10 monclient: get_monmap_and_config 2026-02-17T05:50:40.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.455+0000 7febb6c0e640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.455+0000 7febb6c0e640 10 monclient: monmap: 2026-02-17T05:50:40.455 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.455 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.455 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:40.456748+0000 2026-02-17T05:50:40.455 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:40.456748+0000 2026-02-17T05:50:40.455 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.455+0000 7febb6c0e640 1 Processor -- start 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.455+0000 7febb6c0e640 1 -- start start 2026-02-17T05:50:40.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.455+0000 7febb6c0e640 10 monclient: init 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7febb006c890 0x7febb006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient(hunting): picked mon.noname-a con 0x7febb006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb009c4a0 0x7febb006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient(hunting): picked mon.noname-b con 0x7febb009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7febb0134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb006daf0 con 0x7febb0134df0 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7febb00a0880 con 0x7febb006c890 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7febb006d6f0 con 0x7febb009c4a0 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb6c0e640 10 monclient(hunting): authenticate will time out at 886.010292s 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7febb006c890 0x7febb006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb009c4a0 0x7febb006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7febb006c890 0x7febb006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49114/0 (socket says 10.20.193.78:49114) 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 1 -- 10.20.193.78:0/3564082179 learned_addr learned my addr 10.20.193.78:0/3564082179 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb009c4a0 0x7febb006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40196/0 (socket says 10.20.193.78:40196) 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 10 monclient(hunting): get_auth_request con 0x7febb006c890 auth_method 0 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient(hunting): get_auth_request con 0x7febb009c4a0 auth_method 0 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient(hunting): handle_auth_done global_id 74138 payload 274 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 1 -- 10.20.193.78:0/3564082179 >> v2:10.20.193.78:3300/0 conn(0x7febb006c890 msgr2=0x7febb006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 1 --2- 10.20.193.78:0/3564082179 >> v2:10.20.193.78:3300/0 conn(0x7febb006c890 0x7febb006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 1 -- 10.20.193.78:0/3564082179 >> v1:10.20.193.115:6789/0 conn(0x7febb0134df0 legacy=0x7febb0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 1 monclient: found mon.noname-b 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.456+0000 7feba7fff640 1 -- 10.20.193.78:0/3564082179 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb009cc80 con 0x7febb009c4a0 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.458+0000 7feba7fff640 1 -- 10.20.193.78:0/3564082179 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb009c4a0 msgr2=0x7febb006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.458+0000 7feba7fff640 1 --2- 10.20.193.78:0/3564082179 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb009c4a0 0x7febb006d1b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febb006d870 tx=0x7feb9802ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cece9e3b82b59452 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 1 -- 10.20.193.78:0/3564082179 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9803c070 con 0x7febb009c4a0 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: dump: 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.465 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 1 -- 10.20.193.78:0/3564082179 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9802fbf0 con 0x7febb009c4a0 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 5 monclient: authenticate success, global_id 74138 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 1 -- 10.20.193.78:0/3564082179 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98037440 con 0x7febb009c4a0 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 4 set_mon_vals no callback set 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7feba77fe640 10 monclient: dump: 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 10 monclient: shutdown 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 1 -- 10.20.193.78:0/3564082179 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb009c4a0 msgr2=0x7febb006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 1 --2- 10.20.193.78:0/3564082179 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb009c4a0 0x7febb006d1b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febb006d870 tx=0x7feb9802ed70 comp rx=0 tx=0).stop 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 1 -- 10.20.193.78:0/3564082179 shutdown_connections 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 1 --2- 10.20.193.78:0/3564082179 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb009c4a0 0x7febb006d1b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 1 --2- 10.20.193.78:0/3564082179 >> v2:10.20.193.78:3300/0 conn(0x7febb006c890 0x7febb006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 1 -- 10.20.193.78:0/3564082179 >> 10.20.193.78:0/3564082179 conn(0x7febb00a6620 msgr2=0x7febb00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 1 -- 10.20.193.78:0/3564082179 shutdown_connections 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 1 -- 10.20.193.78:0/3564082179 wait complete. 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.459+0000 7febb6c0e640 10 monclient: monmap: 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 1 Processor -- start 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 1 -- start start 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient: init 2026-02-17T05:50:40.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb006c890 0x7febb01cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient(hunting): picked mon.noname-c con 0x7febb006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7febb009c4a0 0x7febb009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient(hunting): picked mon.noname-a con 0x7febb009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb0134df0 0x7febb009e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient(hunting): picked mon.noname-b con 0x7febb0134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7febb00a01a0 con 0x7febb009c4a0 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7febb00a0020 con 0x7febb0134df0 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7febb009be20 con 0x7febb006c890 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb6c0e640 10 monclient(hunting): authenticate will time out at 886.014429s 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb006c890 0x7febb01cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb006c890 0x7febb01cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40202/0 (socket says 10.20.193.78:40202) 2026-02-17T05:50:40.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 1 -- 10.20.193.78:0/1463854664 learned_addr learned my addr 10.20.193.78:0/1463854664 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7febb009c4a0 0x7febb009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 10 monclient(hunting): get_auth_request con 0x7febb006c890 auth_method 0 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient(hunting): get_auth_request con 0x7febb009c4a0 auth_method 0 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient(hunting): handle_auth_done global_id 54354 payload 824 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb006c890 msgr2=0x7febb01cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb006c890 0x7febb01cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb0134df0 msgr2=0x7febb009e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb0134df0 0x7febb009e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 1 monclient: found mon.noname-a 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7feba7fff640 1 -- 10.20.193.78:0/1463854664 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb009d810 con 0x7febb009c4a0 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.460+0000 7febb4983640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7febb006c890 0x7febb01cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba7fff640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7febb009c4a0 msgr2=0x7febb009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba7fff640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7febb009c4a0 0x7febb009db50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feb98036000 tx=0x7feb980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9803c040 con 0x7febb009c4a0 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: dump: 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.471 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb98047070 con 0x7febb009c4a0 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98040a40 con 0x7febb009c4a0 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba6ffd640 4 set_mon_vals no callback set 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7feba57fa640 10 monclient: dump: 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.472 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7febb6c0e640 5 monclient: authenticate success, global_id 54354 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7febb6c0e640 10 monclient: _renew_subs 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7febb6c0e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb01278f0 con 0x7febb009c4a0 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7febb6c0e640 10 monclient: _renew_subs 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7febb6c0e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.462+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb0127330 con 0x7febb009c4a0 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb6c0e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb6c0e640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb01cfce0 0x7febb009e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb6c0e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7febb00726d0 con 0x7febb01cfce0 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7feb9803f030 con 0x7febb009c4a0 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb4983640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb01cfce0 0x7febb009e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb4983640 10 monclient: get_auth_request con 0x7febb01cfce0 auth_method 0 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb4983640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb4983640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb4983640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7feba57fa640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feb8003deb0 0x7feb80040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.463+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(184..184 src has 117..184) ==== 4815+0+0 (secure 0 0 0) 0x7feb98077b30 con 0x7febb009c4a0 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.464+0000 7febb4983640 10 monclient(hunting): handle_auth_done global_id 44347 payload 824 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.464+0000 7febb4983640 1 -- 10.20.193.78:0/1463854664 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febb01282f0 con 0x7febb01cfce0 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.464+0000 7febb5184640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feb8003deb0 0x7feb80040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.464+0000 7febb5184640 10 monclient: get_auth_request con 0x7feb8003deb0 auth_method 0 2026-02-17T05:50:40.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7febb4983640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb01cfce0 msgr2=0x7febb009e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7febb4983640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb01cfce0 0x7febb009e5d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ef0 tx=0x7feba000f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0004020 con 0x7febb01cfce0 2026-02-17T05:50:40.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:40.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 10 monclient: dump: 2026-02-17T05:50:40.476 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7febb5184640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feb8003deb0 0x7feb80040370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7febb0126a30 tx=0x7febac003fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7feba0004020 con 0x7febb01cfce0 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb01cfce0 msgr2=0x7febb009e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb01cfce0 0x7febb009e5d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ef0 tx=0x7feba000f610 comp rx=0 tx=0).stop 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb01cfce0 msgr2=0x7febb009e5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.466+0000 7feba57fa640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb01cfce0 0x7febb009e5d0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ef0 tx=0x7feba000f610 comp rx=0 tx=0).stop 2026-02-17T05:50:40.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.469+0000 7febb6c0e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.469+0000 7febb6c0e640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb009eb10 0x7febb01da450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.469+0000 7febb6c0e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.469+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7febb009ef20 con 0x7febb009eb10 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.469+0000 7febb4983640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb009eb10 0x7febb01da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 10 monclient: get_auth_request con 0x7febb009eb10 auth_method 0 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 10 monclient(hunting): handle_auth_done global_id 44350 payload 824 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.470+0000 7febb4983640 1 -- 10.20.193.78:0/1463854664 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7febb0074980 con 0x7febb009eb10 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7febb4983640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb009eb10 msgr2=0x7febb01da450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7febb4983640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb009eb10 0x7febb01da450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ef0 tx=0x7feba00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba002b020 con 0x7febb009eb10 2026-02-17T05:50:40.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 10 monclient: dump: 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7feba0003b20 con 0x7febb009eb10 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb009eb10 msgr2=0x7febb01da450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb009eb10 0x7febb01da450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ef0 tx=0x7feba00047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb009eb10 msgr2=0x7febb01da450 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.472+0000 7feba57fa640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7febb009eb10 0x7febb01da450 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feba0009ef0 tx=0x7feba00047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 46, 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 164119, 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:50:40.480 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.481 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:40.482 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:40.483 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.484 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.485 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:40.486 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:40.487 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:40.488 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:40.489 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:40.489 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:40.489 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:40.489 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:40.489 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:40.489 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:40.489 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feb8003deb0 msgr2=0x7feb80040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feb8003deb0 0x7feb80040370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7febb0126a30 tx=0x7febac003fc0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 10 monclient: shutdown 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7febb009c4a0 msgr2=0x7febb009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 1 --2- 10.20.193.78:0/1463854664 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7febb009c4a0 0x7febb009db50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feb98036000 tx=0x7feb980047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb5184640 1 -- 10.20.193.78:0/1463854664 reap_dead start 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb5184640 1 -- 10.20.193.78:0/1463854664 reap_dead start 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 shutdown_connections 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 >> 10.20.193.78:0/1463854664 conn(0x7febb00a6620 msgr2=0x7febb0071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 shutdown_connections 2026-02-17T05:50:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.473+0000 7febb6c0e640 1 -- 10.20.193.78:0/1463854664 wait complete. 2026-02-17T05:50:40.491 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:50:40.491 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:50:40.491 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:50:40.491 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:50:40.491 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:50:40.491 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:50:40.491 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:50:40.500 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:50:40.501 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:50:40.501 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient: get_monmap_and_config 2026-02-17T05:50:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient: monmap: 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:40.608386+0000 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:40.608386+0000 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 1 Processor -- start 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 1 -- start start 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient: init 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc04406c890 0x7fc04406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient(hunting): picked mon.noname-a con 0x7fc04406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc04409c4a0 0x7fc04406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient(hunting): picked mon.noname-c con 0x7fc04409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc044134df0 0x7fc044135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient(hunting): picked mon.noname-b con 0x7fc044134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc0440a0880 con 0x7fc04406c890 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc04406d6f0 con 0x7fc04409c4a0 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc04406daf0 con 0x7fc044134df0 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc049b33640 10 monclient(hunting): authenticate will time out at 886.161914s 2026-02-17T05:50:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc0437fe640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fc04406c890 msgr2=0x7fc04406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.607+0000 7fc0437fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc04406c890 0x7fc04406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc044134df0 0x7fc044135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc044134df0 0x7fc044135280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40234/0 (socket says 10.20.193.78:40234) 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 -- 10.20.193.78:0/624852434 learned_addr learned my addr 10.20.193.78:0/624852434 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient(hunting): get_auth_request con 0x7fc044134df0 auth_method 0 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc042ffd640 1 --2- 10.20.193.78:0/624852434 >> v2:10.20.193.115:3300/0 conn(0x7fc04409c4a0 0x7fc04406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc042ffd640 10 monclient(hunting): get_auth_request con 0x7fc04409c4a0 auth_method 0 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc042ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc042ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc042ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient(hunting): handle_auth_done global_id 74144 payload 274 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 -- 10.20.193.78:0/624852434 >> v2:10.20.193.115:3300/0 conn(0x7fc04409c4a0 msgr2=0x7fc04406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 --2- 10.20.193.78:0/624852434 >> v2:10.20.193.115:3300/0 conn(0x7fc04409c4a0 0x7fc04406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 -- 10.20.193.78:0/624852434 >> v2:10.20.193.78:3300/0 conn(0x7fc04406c890 msgr2=0x7fc04406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 --2- 10.20.193.78:0/624852434 >> v2:10.20.193.78:3300/0 conn(0x7fc04406c890 0x7fc04406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 monclient: found mon.noname-b 2026-02-17T05:50:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.608+0000 7fc043fff640 1 -- 10.20.193.78:0/624852434 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04409cc80 con 0x7fc044134df0 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc043fff640 1 -- 10.20.193.78:0/624852434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc044134df0 msgr2=0x7fc044135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc043fff640 1 --2- 10.20.193.78:0/624852434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc044134df0 0x7fc044135280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc04409c980 tx=0x7fc03802f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b3abd72ce177b1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 1 -- 10.20.193.78:0/624852434 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0380373f0 con 0x7fc044134df0 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: dump: 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.623 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 1 -- 10.20.193.78:0/624852434 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc038037590 con 0x7fc044134df0 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc049b33640 5 monclient: authenticate success, global_id 74144 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc049b33640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 1 -- 10.20.193.78:0/624852434 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038037960 con 0x7fc044134df0 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc049b33640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc049b33640 4 set_mon_vals no callback set 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.610+0000 7fc0427fc640 10 monclient: dump: 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.624 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 10 monclient: shutdown 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 -- 10.20.193.78:0/624852434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc044134df0 msgr2=0x7fc044135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 --2- 10.20.193.78:0/624852434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc044134df0 0x7fc044135280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc04409c980 tx=0x7fc03802f200 comp rx=0 tx=0).stop 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 -- 10.20.193.78:0/624852434 shutdown_connections 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 --2- 10.20.193.78:0/624852434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc044134df0 0x7fc044135280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 --2- 10.20.193.78:0/624852434 >> v2:10.20.193.115:3300/0 conn(0x7fc04409c4a0 0x7fc04406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 --2- 10.20.193.78:0/624852434 >> v2:10.20.193.78:3300/0 conn(0x7fc04406c890 0x7fc04406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 -- 10.20.193.78:0/624852434 >> 10.20.193.78:0/624852434 conn(0x7fc0440a6620 msgr2=0x7fc0440a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 -- 10.20.193.78:0/624852434 shutdown_connections 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 -- 10.20.193.78:0/624852434 wait complete. 2026-02-17T05:50:40.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 10 monclient: build_initial_monmap 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 10 monclient: monmap: 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.611+0000 7fc049b33640 1 Processor -- start 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 1 -- start start 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient: init 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc04406c890 0x7fc0441cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient(hunting): picked mon.noname-c con 0x7fc04406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc04409c4a0 0x7fc0441cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient(hunting): picked mon.noname-a con 0x7fc04409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc044134df0 0x7fc044121440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient(hunting): picked mon.noname-b con 0x7fc044134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440a01a0 con 0x7fc04409c4a0 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440a0020 con 0x7fc044134df0 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc04409be20 con 0x7fc04406c890 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc049b33640 10 monclient(hunting): authenticate will time out at 886.166359s 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc042ffd640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc04409c4a0 msgr2=0x7fc0441cd870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc04406c890 0x7fc0441cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc042ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc04409c4a0 0x7fc0441cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc04406c890 0x7fc0441cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40248/0 (socket says 10.20.193.78:40248) 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 -- 10.20.193.78:0/2982025030 learned_addr learned my addr 10.20.193.78:0/2982025030 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient(hunting): get_auth_request con 0x7fc04406c890 auth_method 0 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc043fff640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc044134df0 0x7fc044121440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc043fff640 10 monclient(hunting): get_auth_request con 0x7fc044134df0 auth_method 0 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc043fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc043fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc043fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient(hunting): handle_auth_done global_id 74147 payload 824 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 -- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc044134df0 msgr2=0x7fc044121440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc044134df0 0x7fc044121440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 -- 10.20.193.78:0/2982025030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc04409c4a0 msgr2=0x7fc0441cd870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc04409c4a0 0x7fc0441cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient: _finish_hunting 0 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 monclient: found mon.noname-c 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.612+0000 7fc0437fe640 1 -- 10.20.193.78:0/2982025030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc044124e10 con 0x7fc04406c890 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc0437fe640 1 -- 10.20.193.78:0/2982025030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc04406c890 msgr2=0x7fc0441cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc0437fe640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc04406c890 0x7fc0441cd330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc03000b7e0 tx=0x7fc03000bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 1 -- 10.20.193.78:0/2982025030 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030005270 con 0x7fc04406c890 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: dump: 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: _finish_auth 0 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 1 -- 10.20.193.78:0/2982025030 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030010070 con 0x7fc04406c890 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc049b33640 5 monclient: authenticate success, global_id 74147 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc049b33640 10 monclient: _renew_subs 2026-02-17T05:50:40.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc049b33640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441250f0 con 0x7fc04406c890 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 1 -- 10.20.193.78:0/2982025030 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030005650 con 0x7fc04406c890 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc041ffb640 4 set_mon_vals no callback set 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc01ffff640 10 monclient: dump: 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc049b33640 10 monclient: _renew_subs 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc049b33640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.614+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc044072560 con 0x7fc04406c890 2026-02-17T05:50:40.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc049b33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc049b33640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc049b33640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc044074980 con 0x7fc04406c890 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc01ffff640 1 -- 10.20.193.78:0/2982025030 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc03001d030 con 0x7fc04406c890 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc01ffff640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc01403de60 0x7fc014040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc01ffff640 1 -- 10.20.193.78:0/2982025030 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(184..184 src has 117..184) ==== 4815+0+0 (secure 0 0 0) 0x7fc030051d00 con 0x7fc04406c890 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc042ffd640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc01403de60 0x7fc014040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.615+0000 7fc042ffd640 10 monclient: get_auth_request con 0x7fc01403de60 auth_method 0 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.618+0000 7fc01ffff640 1 -- 10.20.193.78:0/2982025030 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc03000cd80 con 0x7fc04406c890 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.618+0000 7fc01ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.618+0000 7fc01ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:40.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.618+0000 7fc042ffd640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc01403de60 0x7fc014040320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc044125020 tx=0x7fc034008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:40.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.730+0000 7fc049b33640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.730+0000 7fc049b33640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.730+0000 7fc049b33640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.730+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc044135280 con 0x7fc04406c890 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.730+0000 7fc01ffff640 1 -- 10.20.193.78:0/2982025030 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fc0300057f0 con 0x7fc04406c890 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.730+0000 7fc01ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.730+0000 7fc01ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc01403de60 msgr2=0x7fc014040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc01403de60 0x7fc014040320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc044125020 tx=0x7fc034008040 comp rx=0 tx=0).stop 2026-02-17T05:50:40.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 10 monclient: shutdown 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc04406c890 msgr2=0x7fc0441cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc04406c890 0x7fc0441cd330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc03000b7e0 tx=0x7fc03000bcb0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 shutdown_connections 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc01403de60 0x7fc014040320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc044134df0 0x7fc044121440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc04409c4a0 0x7fc0441cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 --2- 10.20.193.78:0/2982025030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc04406c890 0x7fc0441cd330 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 >> 10.20.193.78:0/2982025030 conn(0x7fc0440a6620 msgr2=0x7fc044135850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 shutdown_connections 2026-02-17T05:50:40.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:40.731+0000 7fc049b33640 1 -- 10.20.193.78:0/2982025030 wait complete. 2026-02-17T05:50:43.739 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.807+0000 7fd640abf640 10 monclient: get_monmap_and_config 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.807+0000 7fd640abf640 10 monclient: build_initial_monmap 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.807+0000 7fd640abf640 10 monclient: monmap: 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:43.809172+0000 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:43.809172+0000 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:43.807 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:43.808 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:43.808 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:43.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 1 Processor -- start 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 1 -- start start 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient: init 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd63c09be20 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd63c09f110 0x7fd63c141b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient(hunting): picked mon.noname-c con 0x7fd63c09f110 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd63c09c910 0x7fd63c1440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient(hunting): picked mon.noname-a con 0x7fd63c09c910 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c09d2c0 con 0x7fd63c09be20 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd63c09e770 con 0x7fd63c09c910 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd63c09d500 con 0x7fd63c09f110 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd640abf640 10 monclient(hunting): authenticate will time out at 889.362739s 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd63ad76640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fd63c09c910 msgr2=0x7fd63c1440d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd63ad76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd63c09c910 0x7fd63c1440d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd63a575640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7fd63c09be20 0x7fd63c09c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:36096/0 (socket says 10.20.193.78:36096) 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.808+0000 7fd63a575640 1 -- 10.20.193.78:0/916424130 learned_addr learned my addr 10.20.193.78:0/916424130 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 1 --2- 10.20.193.78:0/916424130 >> v2:10.20.193.115:3300/0 conn(0x7fd63c09f110 0x7fd63c141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient(hunting): get_auth_request con 0x7fd63c09f110 auth_method 0 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient(hunting): handle_auth_done global_id 44359 payload 274 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 1 -- 10.20.193.78:0/916424130 >> v2:10.20.193.78:3300/0 conn(0x7fd63c09c910 msgr2=0x7fd63c1440d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 1 --2- 10.20.193.78:0/916424130 >> v2:10.20.193.78:3300/0 conn(0x7fd63c09c910 0x7fd63c1440d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:43.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 1 -- 10.20.193.78:0/916424130 >> v1:10.20.193.115:6789/0 conn(0x7fd63c09be20 legacy=0x7fd63c09c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:43.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient: _finish_hunting 0 2026-02-17T05:50:43.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 1 monclient: found mon.noname-c 2026-02-17T05:50:43.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:43.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:50:43.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.809+0000 7fd639d74640 1 -- 10.20.193.78:0/916424130 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c09de40 con 0x7fd63c09f110 2026-02-17T05:50:43.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.811+0000 7fd639d74640 1 -- 10.20.193.78:0/916424130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09f110 msgr2=0x7fd63c141b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:43.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.811+0000 7fd639d74640 1 --2- 10.20.193.78:0/916424130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09f110 0x7fd63c141b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd63c09d490 tx=0x7fd62402ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703fd3ff1bf08018 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:43.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 1 -- 10.20.193.78:0/916424130 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62403c070 con 0x7fd63c09f110 2026-02-17T05:50:43.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:43.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:43.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: dump: 2026-02-17T05:50:43.811 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:43.811 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: _finish_auth 0 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: _check_auth_tickets 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 5 monclient: authenticate success, global_id 44359 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 1 -- 10.20.193.78:0/916424130 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62402fcd0 con 0x7fd63c09f110 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 1 -- 10.20.193.78:0/916424130 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624037660 con 0x7fd63c09f110 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd639573640 10 monclient: dump: 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 4 set_mon_vals no callback set 2026-02-17T05:50:43.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 10 monclient: shutdown 2026-02-17T05:50:43.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:43.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 1 -- 10.20.193.78:0/916424130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09f110 msgr2=0x7fd63c141b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:43.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 1 --2- 10.20.193.78:0/916424130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09f110 0x7fd63c141b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd63c09d490 tx=0x7fd62402ed70 comp rx=0 tx=0).stop 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 1 -- 10.20.193.78:0/916424130 shutdown_connections 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 1 --2- 10.20.193.78:0/916424130 >> v2:10.20.193.78:3300/0 conn(0x7fd63c09c910 0x7fd63c1440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 1 --2- 10.20.193.78:0/916424130 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09f110 0x7fd63c141b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 1 -- 10.20.193.78:0/916424130 >> 10.20.193.78:0/916424130 conn(0x7fd63c0a6620 msgr2=0x7fd63c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 1 -- 10.20.193.78:0/916424130 shutdown_connections 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.812+0000 7fd640abf640 1 -- 10.20.193.78:0/916424130 wait complete. 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient: build_initial_monmap 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient: monmap: 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:43.814 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 1 Processor -- start 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 1 -- start start 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient: init 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09be20 0x7fd63c1d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient(hunting): picked mon.noname-b con 0x7fd63c09be20 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 0x7fd63c13bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient(hunting): picked mon.noname-c con 0x7fd63c09c910 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63c09f110 0x7fd63c140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient(hunting): picked mon.noname-a con 0x7fd63c09f110 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c06cc80 con 0x7fd63c09f110 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c06cb00 con 0x7fd63c09be20 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c144610 con 0x7fd63c09c910 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd640abf640 10 monclient(hunting): authenticate will time out at 889.367606s 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd63ad76640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63c09f110 msgr2=0x7fd63c140c10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd63ad76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63c09f110 0x7fd63c140c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 0x7fd63c13bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 0x7fd63c13bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40262/0 (socket says 10.20.193.78:40262) 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 -- 10.20.193.78:0/414644367 learned_addr learned my addr 10.20.193.78:0/414644367 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient(hunting): get_auth_request con 0x7fd63c09c910 auth_method 0 2026-02-17T05:50:43.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd63a575640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09be20 0x7fd63c1d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient(hunting): handle_auth_done global_id 74150 payload 824 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 -- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09be20 msgr2=0x7fd63c1d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09be20 0x7fd63c1d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 -- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63c09f110 msgr2=0x7fd63c140c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63c09f110 0x7fd63c140c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient: _finish_hunting 0 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 monclient: found mon.noname-c 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd639d74640 1 -- 10.20.193.78:0/414644367 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c1264d0 con 0x7fd63c09c910 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd63a575640 10 monclient: get_auth_request con 0x7fd63c09be20 auth_method 0 2026-02-17T05:50:43.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.813+0000 7fd63a575640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09be20 0x7fd63c1d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd639d74640 1 -- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 msgr2=0x7fd63c13bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd639d74640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 0x7fd63c13bb40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd62402f280 tx=0x7fd6240047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 1 -- 10.20.193.78:0/414644367 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624046070 con 0x7fd63c09c910 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: dump: 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd639d74640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 0x7fd63c13bb40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd62402f280 tx=0x7fd6240047c0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fd624041870 config(0 keys) 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: _finish_auth 0 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:43.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 5 monclient: authenticate success, global_id 74150 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 10 monclient: _renew_subs 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 1 -- 10.20.193.78:0/414644367 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd624041870 con 0x7fd63c09c910 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c0a1320 con 0x7fd63c09c910 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd638d72640 4 set_mon_vals no callback set 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 10 monclient: _renew_subs 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c125f10 con 0x7fd63c09c910 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c125d10 con 0x7fd63c09c910 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 1 -- 10.20.193.78:0/414644367 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62403c040 con 0x7fd63c09c910 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 10 monclient: dump: 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 1 -- 10.20.193.78:0/414644367 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd624041c60 con 0x7fd63c09c910 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.816+0000 7fd622ffd640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60803de60 0x7fd608040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.817+0000 7fd63a575640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60803de60 0x7fd608040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:43.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.817+0000 7fd63a575640 10 monclient: get_auth_request con 0x7fd60803de60 auth_method 0 2026-02-17T05:50:43.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.819+0000 7fd622ffd640 1 -- 10.20.193.78:0/414644367 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(184..184 src has 117..184) ==== 4815+0+0 (secure 0 0 0) 0x7fd624078290 con 0x7fd63c09c910 2026-02-17T05:50:43.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.820+0000 7fd63a575640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60803de60 0x7fd608040320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6300052b0 tx=0x7fd630009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:43.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:43.820+0000 7fd622ffd640 1 -- 10.20.193.78:0/414644367 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:50:44.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.439+0000 7fd622ffd640 1 -- 10.20.193.78:0/414644367 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd624076c70 con 0x7fd63c09c910 2026-02-17T05:50:44.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.439+0000 7fd622ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:44.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.439+0000 7fd622ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:44.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.552+0000 7fd640abf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:44.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.552+0000 7fd640abf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:44.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.552+0000 7fd640abf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:44.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.552+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd63c09c200 con 0x7fd63c09c910 2026-02-17T05:50:44.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.552+0000 7fd622ffd640 1 -- 10.20.193.78:0/414644367 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd624040c10 con 0x7fd63c09c910 2026-02-17T05:50:44.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.552+0000 7fd622ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:44.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.552+0000 7fd622ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:44.553 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:44.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60803de60 msgr2=0x7fd608040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:44.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60803de60 0x7fd608040320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6300052b0 tx=0x7fd630009040 comp rx=0 tx=0).stop 2026-02-17T05:50:44.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 10 monclient: shutdown 2026-02-17T05:50:44.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:44.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 msgr2=0x7fd63c13bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:44.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 0x7fd63c13bb40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd62402f280 tx=0x7fd6240047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:44.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 shutdown_connections 2026-02-17T05:50:44.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63c09f110 0x7fd63c140c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:44.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63c09c910 0x7fd63c13bb40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:44.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60803de60 0x7fd608040320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:44.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 --2- 10.20.193.78:0/414644367 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63c09be20 0x7fd63c1d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:44.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 >> 10.20.193.78:0/414644367 conn(0x7fd63c0a6620 msgr2=0x7fd63c0a30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:44.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 shutdown_connections 2026-02-17T05:50:44.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:44.553+0000 7fd640abf640 1 -- 10.20.193.78:0/414644367 wait complete. 2026-02-17T05:50:47.561 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:50:47.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.629+0000 7f13c8264640 10 monclient: get_monmap_and_config 2026-02-17T05:50:47.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.629+0000 7f13c8264640 10 monclient: build_initial_monmap 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.629+0000 7f13c8264640 10 monclient: monmap: 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:47.630774+0000 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:47.630774+0000 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:47.631 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.629+0000 7f13c8264640 1 Processor -- start 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.629+0000 7f13c8264640 1 -- start start 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.629+0000 7f13c8264640 10 monclient: init 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f13c006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 0x7f13c013d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 10 monclient(hunting): picked mon.noname-a con 0x7f13c009d430 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f13c009bf40 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:47.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c0073600 con 0x7f13c006c890 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c006dab0 con 0x7f13c009bf40 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f13c009ca50 con 0x7f13c009d430 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c8264640 10 monclient(hunting): authenticate will time out at 893.184315s 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c5fd9640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f13c006c890 legacy=0x7f13c006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c57d8640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 msgr2=0x7f13c013d5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c57d8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 0x7f13c013d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c67da640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f13c009bf40 0x7f13c013fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:50508/0 (socket says 10.20.193.78:50508) 2026-02-17T05:50:47.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.630+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 learned_addr learned my addr 10.20.193.78:0/1398951736 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:50:47.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.632+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.115:6789/0 conn(0x7f13c009bf40 legacy=0x7f13c013fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:47.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.830+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13c006c890 legacy=0x7f13c006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:47.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.830+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 msgr2=0x7f13c013d5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:47.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:47.830+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 0x7f13c013d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:48.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:48.231+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13c006c890 legacy=0x7f13c006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:48.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:48.231+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 msgr2=0x7f13c013d5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:48.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:48.231+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 0x7f13c013d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:49.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:49.032+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13c006c890 legacy=0x7f13c006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:49.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:49.032+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 msgr2=0x7f13c013d5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:49.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:49.032+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 0x7f13c013d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient(hunting): tick 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 msgr2=0x7f13c013d5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13c009d430 0x7f13c013d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.115:6789/0 conn(0x7f13c009bf40 legacy=0x7f13c013fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13c006c890 legacy=0x7f13c006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient: _add_conns ranks=[4,2] 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f13a4003840 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f13a4006330 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:50.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f13c009ca50 con 0x7f13a4003840 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f13a4008ea0 con 0x7f13a4006330 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13af7fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 10 monclient(hunting): get_auth_request con 0x7f13a4006330 auth_method 0 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request con 0x7f13a4003840 auth_method 0 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:50.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.630+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:50.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.830+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:50.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c57d8640 10 monclient(hunting): get_auth_request con 0x7f13a4006330 auth_method 0 2026-02-17T05:50:50.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c57d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c57d8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c57d8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request con 0x7f13a4003840 auth_method 0 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:50.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:50.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:50.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:50:50.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:50.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:50.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:50.831+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:51.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:51.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 10 monclient(hunting): get_auth_request con 0x7f13a4006330 auth_method 0 2026-02-17T05:50:51.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:51.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:51.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:51.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:50:51.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:51.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request con 0x7f13a4003840 auth_method 0 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c5fd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.231+0000 7f13c5fd9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.232+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.232+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.232+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:51.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:51.232+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 10 monclient(hunting): get_auth_request con 0x7f13a4006330 auth_method 0 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.032+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request con 0x7f13a4003840 auth_method 0 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:52.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:52.033+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:50:53.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): tick 2026-02-17T05:50:53.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:50:53.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:50:53.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:50:53.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 msgr2=0x7f13a40087f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:53.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4006330 0x7f13a40087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:53.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 msgr2=0x7f13a4005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:53.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a4003840 0x7f13a4005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 reap_dead start 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f13a4003c90 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 0x7f13a4008350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f13a4007ee0 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f13a40088c0 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c009c770 con 0x7f13a4003c90 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f13c009ca50 con 0x7f13a4007ee0 2026-02-17T05:50:53.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 1 -- 10.20.193.78:0/1398951736 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f13a4008ea0 con 0x7f13a40088c0 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13af7fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13a4003c90 legacy=0x7f13a40067c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 msgr2=0x7f13a4008350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 0x7f13a4008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 10 monclient(hunting): get_auth_request con 0x7f13a40088c0 auth_method 0 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 msgr2=0x7f13a400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:50:53.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 msgr2=0x7f13a400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:53.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.630+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:53.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.631+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:53.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13a4003c90 legacy=0x7f13a40067c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:53.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 msgr2=0x7f13a4008350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:53.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 0x7f13a4008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:53.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:53.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 10 monclient(hunting): get_auth_request con 0x7f13a40088c0 auth_method 0 2026-02-17T05:50:53.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:53.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:53.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:53.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 msgr2=0x7f13a400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:53.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 msgr2=0x7f13a400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:53.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:53.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:53.831+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:50:54.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.231+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13a4003c90 legacy=0x7f13a40067c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:54.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.231+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 msgr2=0x7f13a4008350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:54.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.231+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 0x7f13a4008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:54.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:54.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 10 monclient(hunting): get_auth_request con 0x7f13a40088c0 auth_method 0 2026-02-17T05:50:54.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:54.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:54.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:54.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 msgr2=0x7f13a400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:50:54.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 msgr2=0x7f13a400d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:50:54.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:50:54.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:54.232+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:50:54.875 INFO:tasks.rados.rados.0.trial115.stdout:857: done (4 left) 2026-02-17T05:50:54.875 INFO:tasks.rados.rados.0.trial115.stdout:858: rollback oid 45 current snap is 85 2026-02-17T05:50:54.875 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 73 2026-02-17T05:50:54.875 INFO:tasks.rados.rados.0.trial115.stdout:855: finishing rollback tid 0 to trial11545420-11 2026-02-17T05:50:54.875 INFO:tasks.rados.rados.0.trial115.stdout:854: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:50:54.924 INFO:tasks.rados.rados.0.trial115.stdout:855: finishing rollback tid 1 to trial11545420-11 2026-02-17T05:50:54.924 INFO:tasks.rados.rados.0.trial115.stdout:855: finishing rollback tid 2 to trial11545420-11 2026-02-17T05:50:54.924 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 339 (ObjNum 303 snap 72 seq_num 303) dirty exists 2026-02-17T05:50:54.924 INFO:tasks.rados.rados.0.trial115.stdout:853: expect (ObjNum 320 snap 77 seq_num 320) 2026-02-17T05:50:55.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.032+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13a4003c90 legacy=0x7f13a40067c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:55.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.032+0000 7f13c57d8640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 msgr2=0x7f13a4008350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:55.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.032+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 0x7f13a4008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.115:3300/0 conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient(hunting): get_auth_request con 0x7f13a40088c0 auth_method 0 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient(hunting): handle_auth_done global_id 44362 payload 274 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 msgr2=0x7f13a4008350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 0x7f13a4008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> v1:10.20.193.78:6789/0 conn(0x7f13a4003c90 legacy=0x7f13a40067c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient: _finish_hunting 0 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 1 monclient: found mon.noname-c 2026-02-17T05:50:55.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:55.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:50:55.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.033+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c0073600 con 0x7f13a40088c0 2026-02-17T05:50:55.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c67da640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a40088c0 msgr2=0x7f13a400d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c67da640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a40088c0 0x7f13a400d450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f13c009c670 tx=0x7f13bc02fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75654e6146664d59 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 1 -- 10.20.193.78:0/1398951736 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0317f0 con 0x7f13a40088c0 2026-02-17T05:50:55.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: dump: 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: _finish_auth 0 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: _check_auth_tickets 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 1 -- 10.20.193.78:0/1398951736 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc031990 con 0x7f13a40088c0 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 1 -- 10.20.193.78:0/1398951736 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc031d40 con 0x7f13a40088c0 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c8264640 5 monclient: authenticate success, global_id 44362 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c8264640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:55.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c8264640 4 set_mon_vals no callback set 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.036+0000 7f13c4fd7640 10 monclient: dump: 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient: shutdown 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- 10.20.193.78:0/1398951736 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a40088c0 msgr2=0x7f13a400d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a40088c0 0x7f13a400d450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f13c009c670 tx=0x7f13bc02fcd0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- 10.20.193.78:0/1398951736 shutdown_connections 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 --2- 10.20.193.78:0/1398951736 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a40088c0 0x7f13a400d450 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 --2- 10.20.193.78:0/1398951736 >> v2:10.20.193.78:3300/0 conn(0x7f13a4007ee0 0x7f13a4008350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- 10.20.193.78:0/1398951736 >> 10.20.193.78:0/1398951736 conn(0x7f13c00a6620 msgr2=0x7f13c00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- 10.20.193.78:0/1398951736 shutdown_connections 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- 10.20.193.78:0/1398951736 wait complete. 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient: build_initial_monmap 2026-02-17T05:50:55.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient: monmap: 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 Processor -- start 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- start start 2026-02-17T05:50:55.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient: init 2026-02-17T05:50:55.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:55.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4003c90 0x7f13c01cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient(hunting): picked mon.noname-c con 0x7f13a4003c90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a4007ee0 0x7f13c01cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient(hunting): picked mon.noname-b con 0x7f13a4007ee0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13a40088c0 0x7f13c01226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient(hunting): picked mon.noname-a con 0x7f13a40088c0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13c006cc70 con 0x7f13a40088c0 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0073410 con 0x7f13a4007ee0 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f13c006cdf0 con 0x7f13a4003c90 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.037+0000 7f13c8264640 10 monclient(hunting): authenticate will time out at 900.591989s 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c67da640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13a40088c0 msgr2=0x7f13c01226f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c67da640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13a40088c0 0x7f13c01226f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4003c90 0x7f13c01cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4003c90 0x7f13c01cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42360/0 (socket says 10.20.193.78:42360) 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1941158834 learned_addr learned my addr 10.20.193.78:0/1941158834 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request con 0x7f13a4003c90 auth_method 0 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a4007ee0 0x7f13c01cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c57d8640 10 monclient(hunting): get_auth_request con 0x7f13a4007ee0 auth_method 0 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c57d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c57d8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c57d8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient(hunting): handle_auth_done global_id 74153 payload 824 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a4007ee0 msgr2=0x7f13c01cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a4007ee0 0x7f13c01cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1941158834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13a40088c0 msgr2=0x7f13c01226f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13a40088c0 0x7f13c01226f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient: _finish_hunting 0 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 monclient: found mon.noname-c 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.038+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1941158834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c01260c0 con 0x7f13a4003c90 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c5fd9640 1 -- 10.20.193.78:0/1941158834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4003c90 msgr2=0x7f13c01cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c5fd9640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4003c90 0x7f13c01cf560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f13b000ce10 tx=0x7f13b000b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 1 -- 10.20.193.78:0/1941158834 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b001c070 con 0x7f13a4003c90 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: dump: 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.043 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: _finish_auth 0 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 1 -- 10.20.193.78:0/1941158834 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b00056d0 con 0x7f13a4003c90 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 1 -- 10.20.193.78:0/1941158834 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0005ac0 con 0x7f13a4003c90 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13ae7fc640 10 monclient: dump: 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.044 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13affff640 4 set_mon_vals no callback set 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c8264640 5 monclient: authenticate success, global_id 74153 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c8264640 10 monclient: _renew_subs 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c8264640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c01263a0 con 0x7f13a4003c90 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c8264640 10 monclient: _renew_subs 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c8264640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.040+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c01371f0 con 0x7f13a4003c90 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13ae7fc640 1 -- 10.20.193.78:0/1941158834 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f13b00203b0 con 0x7f13a4003c90 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13c8264640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13c8264640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13c8264640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c006d170 con 0x7f13a4003c90 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13ae7fc640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f139003de60 0x7f1390040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13ae7fc640 1 -- 10.20.193.78:0/1941158834 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(185..185 src has 117..185) ==== 4843+0+0 (secure 0 0 0) 0x7f13b0053090 con 0x7f13a4003c90 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f139003de60 0x7f1390040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.041+0000 7f13c57d8640 10 monclient: get_auth_request con 0x7f139003de60 auth_method 0 2026-02-17T05:50:55.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.043+0000 7f13ae7fc640 1 -- 10.20.193.78:0/1941158834 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f13b0051a50 con 0x7f13a4003c90 2026-02-17T05:50:55.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.043+0000 7f13ae7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:55.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.043+0000 7f13ae7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:55.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.043+0000 7f13c57d8640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f139003de60 0x7f1390040320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f13c01265b0 tx=0x7f13b400e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.078 INFO:tasks.rados.rados.0.trial115.stdout:858: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:50:55.078 INFO:tasks.rados.rados.0.trial115.stdout:858: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 394 (ObjNum 302 snap 72 seq_num 302) dirty exists 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:853: done (4 left) 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:854: done (3 left) 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:855: done (2 left) 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:856: done (1 left) 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:858: done (0 left) 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:859: read oid 48 snap 58 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:859: expect (ObjNum 227 snap 46 seq_num 227) 2026-02-17T05:50:55.079 INFO:tasks.rados.rados.0.trial115.stdout:860: snap_create 2026-02-17T05:50:55.085 INFO:tasks.rados.rados.0.trial115.stdout:859: expect (ObjNum 227 snap 46 seq_num 227) 2026-02-17T05:50:55.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.158+0000 7f13c8264640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:50:55.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.158+0000 7f13c8264640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:55.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.158+0000 7f13c8264640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.158+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f13c00744d0 con 0x7f13a4003c90 2026-02-17T05:50:55.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.158+0000 7f13ae7fc640 1 -- 10.20.193.78:0/1941158834 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f13b001dce0 con 0x7f13a4003c90 2026-02-17T05:50:55.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.158+0000 7f13ae7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:50:55.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.158+0000 7f13ae7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:55.159 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":48,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f139003de60 msgr2=0x7f1390040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f139003de60 0x7f1390040320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f13c01265b0 tx=0x7f13b400e040 comp rx=0 tx=0).stop 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 10 monclient: shutdown 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4003c90 msgr2=0x7f13c01cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4003c90 0x7f13c01cf560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f13b000ce10 tx=0x7f13b000b780 comp rx=0 tx=0).stop 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 shutdown_connections 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13a40088c0 0x7f13c01226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f139003de60 0x7f1390040320 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13a4007ee0 0x7f13c01cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 --2- 10.20.193.78:0/1941158834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13a4003c90 0x7f13c01cf560 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 >> 10.20.193.78:0/1941158834 conn(0x7f13c00a6620 msgr2=0x7f13c00a0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:55.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 shutdown_connections 2026-02-17T05:50:55.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.159+0000 7f13c8264640 1 -- 10.20.193.78:0/1941158834 wait complete. 2026-02-17T05:50:55.167 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:50:55.167 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:50:55.167 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient: get_monmap_and_config 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient: build_initial_monmap 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient: monmap: 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:55.238394+0000 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:55.238394+0000 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:55.236 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:55.237 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:55.237 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:55.237 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 1 Processor -- start 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 1 -- start start 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient: init 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f963406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient(hunting): picked mon.noname-b con 0x7f963409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f963406d380 0x7f963413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient(hunting): picked mon.noname-a con 0x7f963406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f963406daf0 con 0x7f963406c890 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f96340a0880 con 0x7f963406d380 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f963409cc80 con 0x7f963409c4a0 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f963ac43640 10 monclient(hunting): authenticate will time out at 900.791947s 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f96389b8640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f963406c890 legacy=0x7f963406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f96391b9640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f963406d380 msgr2=0x7f963413d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.237+0000 7f96391b9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f963406d380 0x7f963413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42374/0 (socket says 10.20.193.78:42374) 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 1 -- 10.20.193.78:0/615851123 learned_addr learned my addr 10.20.193.78:0/615851123 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient(hunting): get_auth_request con 0x7f963409c4a0 auth_method 0 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient(hunting): handle_auth_done global_id 74156 payload 274 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 1 -- 10.20.193.78:0/615851123 >> v2:10.20.193.78:3300/0 conn(0x7f963406d380 msgr2=0x7f963413d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 1 --2- 10.20.193.78:0/615851123 >> v2:10.20.193.78:3300/0 conn(0x7f963406d380 0x7f963413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 1 -- 10.20.193.78:0/615851123 >> v1:10.20.193.78:6789/0 conn(0x7f963406c890 legacy=0x7f963406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:55.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:50:55.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 1 monclient: found mon.noname-b 2026-02-17T05:50:55.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:55.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.238+0000 7f962bfff640 1 -- 10.20.193.78:0/615851123 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963409cf60 con 0x7f963409c4a0 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962bfff640 1 -- 10.20.193.78:0/615851123 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 msgr2=0x7f963413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962bfff640 1 --2- 10.20.193.78:0/615851123 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413b3a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f961c009970 tx=0x7f961c02eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a6345c3cd77f81 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 1 -- 10.20.193.78:0/615851123 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c03c070 con 0x7f963409c4a0 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: dump: 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.242 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f963ac43640 5 monclient: authenticate success, global_id 74156 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 1 -- 10.20.193.78:0/615851123 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961c02fdd0 con 0x7f963409c4a0 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f963ac43640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f963ac43640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f963ac43640 4 set_mon_vals no callback set 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 1 -- 10.20.193.78:0/615851123 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c037660 con 0x7f963409c4a0 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.240+0000 7f962b7fe640 10 monclient: dump: 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.243 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient: shutdown 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- 10.20.193.78:0/615851123 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 msgr2=0x7f963413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 --2- 10.20.193.78:0/615851123 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413b3a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f961c009970 tx=0x7f961c02eeb0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- 10.20.193.78:0/615851123 shutdown_connections 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 --2- 10.20.193.78:0/615851123 >> v2:10.20.193.78:3300/0 conn(0x7f963406d380 0x7f963413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 --2- 10.20.193.78:0/615851123 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413b3a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- 10.20.193.78:0/615851123 >> 10.20.193.78:0/615851123 conn(0x7f96340a6620 msgr2=0x7f96340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- 10.20.193.78:0/615851123 shutdown_connections 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- 10.20.193.78:0/615851123 wait complete. 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient: build_initial_monmap 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient: monmap: 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:55.244 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 Processor -- start 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- start start 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient: init 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f963406c890 0x7f96341cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient(hunting): picked mon.noname-a con 0x7f963406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f963406d380 0x7f9634132f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient(hunting): picked mon.noname-b con 0x7f963406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient(hunting): picked mon.noname-c con 0x7f963409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f963409bfa0 con 0x7f963406c890 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f963409be20 con 0x7f963406d380 2026-02-17T05:50:55.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f963413de30 con 0x7f963409c4a0 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.241+0000 7f963ac43640 10 monclient(hunting): authenticate will time out at 900.795991s 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96389b8640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f963406c890 msgr2=0x7f96341cce80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96389b8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f963406c890 0x7f96341cce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42382/0 (socket says 10.20.193.78:42382) 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 -- 10.20.193.78:0/3289768736 learned_addr learned my addr 10.20.193.78:0/3289768736 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient(hunting): get_auth_request con 0x7f963409c4a0 auth_method 0 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f962bfff640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f963406d380 0x7f9634132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f962bfff640 10 monclient(hunting): get_auth_request con 0x7f963406d380 auth_method 0 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f962bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f962bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f962bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient(hunting): handle_auth_done global_id 74159 payload 824 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f963406d380 msgr2=0x7f9634132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f963406d380 0x7f9634132f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f963406c890 msgr2=0x7f96341cce80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f963406c890 0x7f96341cce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient: _finish_hunting 0 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 monclient: found mon.noname-c 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.242+0000 7f96391b9640 1 -- 10.20.193.78:0/3289768736 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963409d810 con 0x7f963409c4a0 2026-02-17T05:50:55.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96391b9640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 msgr2=0x7f963413a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96391b9640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413a140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f963000be90 tx=0x7f963000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96391b9640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413a140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f963000be90 tx=0x7f963000bf90 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f9630016870 mon_map magic: 0 2026-02-17T05:50:55.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630006e60 con 0x7f963409c4a0 2026-02-17T05:50:55.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 10 monclient: dump: 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 10 monclient: _finish_auth 0 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 10 monclient: _check_auth_tickets 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9630010030 con 0x7f963409c4a0 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f96297fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f963ac43640 5 monclient: authenticate success, global_id 74159 2026-02-17T05:50:55.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f963ac43640 10 monclient: _renew_subs 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f963ac43640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f962affd640 4 set_mon_vals no callback set 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963411cc70 con 0x7f963409c4a0 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f963ac43640 10 monclient: _renew_subs 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f963ac43640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.244+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f963411d330 con 0x7f963409c4a0 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f963ac43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f963ac43640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963413a680 0x7f963413aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f963ac43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f963413d9d0 con 0x7f963413a680 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963413a680 0x7f963413aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 10 monclient: get_auth_request con 0x7f963413a680 auth_method 0 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630016870 con 0x7f963409c4a0 2026-02-17T05:50:55.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96297fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96297fa640 10 monclient: dump: 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9630008070 con 0x7f963409c4a0 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 10 monclient(hunting): handle_auth_done global_id 74162 payload 824 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96389b8640 1 -- 10.20.193.78:0/3289768736 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f963413afa0 con 0x7f963413a680 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f96297fa640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f960403de60 0x7f9604040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f962bfff640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f960403de60 0x7f9604040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.245+0000 7f962bfff640 10 monclient: get_auth_request con 0x7f960403de60 auth_method 0 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96389b8640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963413a680 msgr2=0x7f963413aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96389b8640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963413a680 0x7f963413aa60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f962400bcc0 tx=0x7f962400bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624005070 con 0x7f963413a680 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 10 monclient: dump: 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.253 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f9624005070 con 0x7f963413a680 2026-02-17T05:50:55.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:55.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:55.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963413a680 msgr2=0x7f963413aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963413a680 0x7f963413aa60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f962400bcc0 tx=0x7f962400bcf0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963413a680 msgr2=0x7f963413aa60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:55.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.247+0000 7f96297fa640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963413a680 0x7f963413aa60 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f962400bcc0 tx=0x7f962400bcf0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.248+0000 7f962bfff640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f960403de60 0x7f9604040320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f961c036000 tx=0x7f961c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f963ac43640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f963ac43640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963411d500 0x7f96341d81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f963ac43640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f963411da00 con 0x7f963411d500 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f96389b8640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963411d500 0x7f96341d81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f96389b8640 10 monclient: get_auth_request con 0x7f963411d500 auth_method 0 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f96389b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f96389b8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f96389b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f96389b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f96389b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.250+0000 7f96389b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.251+0000 7f96389b8640 10 monclient(hunting): handle_auth_done global_id 74165 payload 824 2026-02-17T05:50:55.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.251+0000 7f96389b8640 1 -- 10.20.193.78:0/3289768736 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f963406cc70 con 0x7f963411d500 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96389b8640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963411d500 msgr2=0x7f96341d81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96389b8640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963411d500 0x7f96341d81d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f962400bcc0 tx=0x7f9624005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624005220 con 0x7f963411d500 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 10 monclient: dump: 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4696 (secure 0 0 0) 0x7f9624005220 con 0x7f963411d500 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963411d500 msgr2=0x7f96341d81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963411d500 0x7f96341d81d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f962400bcc0 tx=0x7f9624005b20 comp rx=0 tx=0).stop 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963411d500 msgr2=0x7f96341d81d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963411d500 0x7f96341d81d0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f962400bcc0 tx=0x7f9624005b20 comp rx=0 tx=0).stop 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 48, 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 27146, 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:50:55.256 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:55.257 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:55.258 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.259 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:55.260 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f960403de60 msgr2=0x7f9604040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f960403de60 0x7f9604040320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f961c036000 tx=0x7f961c033000 comp rx=0 tx=0).stop 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 10 monclient: shutdown 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 msgr2=0x7f963413a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 1 --2- 10.20.193.78:0/3289768736 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f963409c4a0 0x7f963413a140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f963000be90 tx=0x7f963000bf90 comp rx=0 tx=0).stop 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96391b9640 1 -- 10.20.193.78:0/3289768736 reap_dead start 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96391b9640 1 -- 10.20.193.78:0/3289768736 reap_dead start 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f96297fa640 1 -- 10.20.193.78:0/3289768736 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(185..185 src has 117..185) ==== 4843+0+0 (unknown 0 0 0) 0x7f96300529e0 con 0x7f963409c4a0 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 shutdown_connections 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 >> 10.20.193.78:0/3289768736 conn(0x7f96340a6620 msgr2=0x7f9634070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 shutdown_connections 2026-02-17T05:50:55.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.253+0000 7f963ac43640 1 -- 10.20.193.78:0/3289768736 wait complete. 2026-02-17T05:50:55.261 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:50:55.261 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:50:55.261 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.328+0000 7f149a390640 10 monclient: get_monmap_and_config 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.328+0000 7f149a390640 10 monclient: build_initial_monmap 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient: monmap: 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:50:55.330217+0000 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:50:55.330217+0000 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:50:55.328 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 1 Processor -- start 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 1 -- start start 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient: init 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f149406c890 0x7f149406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient(hunting): picked mon.noname-a con 0x7f149406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f149409c4a0 0x7f149406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient(hunting): picked mon.noname-c con 0x7f149409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1494134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149406daf0 con 0x7f1494134df0 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f14940a0880 con 0x7f149406c890 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f149406d6f0 con 0x7f149409c4a0 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f149a390640 10 monclient(hunting): authenticate will time out at 900.883737s 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f1493fff640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f149406c890 msgr2=0x7f149406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f1493fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f149406c890 0x7f149406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f14937fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f149409c4a0 0x7f149406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f14937fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f149409c4a0 0x7f149406d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:53172/0 (socket says 10.20.193.78:53172) 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f14937fe640 1 -- 10.20.193.78:0/3169523245 learned_addr learned my addr 10.20.193.78:0/3169523245 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f14937fe640 10 monclient(hunting): get_auth_request con 0x7f149409c4a0 auth_method 0 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f14937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f14937fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.329+0000 7f14937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 10 monclient(hunting): handle_auth_done global_id 44371 payload 274 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 1 -- 10.20.193.78:0/3169523245 >> v2:10.20.193.78:3300/0 conn(0x7f149406c890 msgr2=0x7f149406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 1 --2- 10.20.193.78:0/3169523245 >> v2:10.20.193.78:3300/0 conn(0x7f149406c890 0x7f149406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 1 -- 10.20.193.78:0/3169523245 >> v1:10.20.193.115:6789/0 conn(0x7f1494134df0 legacy=0x7f1494135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 10 monclient: _finish_hunting 0 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 1 monclient: found mon.noname-c 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:50:55.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.330+0000 7f14937fe640 1 -- 10.20.193.78:0/3169523245 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149409cc80 con 0x7f149409c4a0 2026-02-17T05:50:55.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.332+0000 7f14937fe640 1 -- 10.20.193.78:0/3169523245 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149409c4a0 msgr2=0x7f149406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.332+0000 7f14937fe640 1 --2- 10.20.193.78:0/3169523245 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149409c4a0 0x7f149406d1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f149409c980 tx=0x7f148402ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1a3f4ecdfe87f9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 1 -- 10.20.193.78:0/3169523245 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148403c070 con 0x7f149409c4a0 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: dump: 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.332 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: _finish_auth 0 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 1 -- 10.20.193.78:0/3169523245 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148402fda0 con 0x7f149409c4a0 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 5 monclient: authenticate success, global_id 44371 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 1 -- 10.20.193.78:0/3169523245 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484037660 con 0x7f149409c4a0 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 10 monclient: get_monmap_and_config success 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 4 set_mon_vals no callback set 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f1492ffd640 10 monclient: dump: 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 10 monclient: shutdown 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 -- 10.20.193.78:0/3169523245 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149409c4a0 msgr2=0x7f149406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 --2- 10.20.193.78:0/3169523245 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149409c4a0 0x7f149406d1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f149409c980 tx=0x7f148402ee80 comp rx=0 tx=0).stop 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 -- 10.20.193.78:0/3169523245 shutdown_connections 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 --2- 10.20.193.78:0/3169523245 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149409c4a0 0x7f149406d1b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 --2- 10.20.193.78:0/3169523245 >> v2:10.20.193.78:3300/0 conn(0x7f149406c890 0x7f149406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 -- 10.20.193.78:0/3169523245 >> 10.20.193.78:0/3169523245 conn(0x7f14940a6620 msgr2=0x7f14940a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 -- 10.20.193.78:0/3169523245 shutdown_connections 2026-02-17T05:50:55.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 -- 10.20.193.78:0/3169523245 wait complete. 2026-02-17T05:50:55.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 10 monclient: build_initial_monmap 2026-02-17T05:50:55.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 10 monclient: monmap: 2026-02-17T05:50:55.334 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 Processor -- start 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 1 -- start start 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.333+0000 7f149a390640 10 monclient: init 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient: _reopen_session rank -1 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f149406c890 0x7f14941cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient(hunting): picked mon.noname-a con 0x7f149406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f149409c4a0 0x7f14941cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient(hunting): picked mon.noname-c con 0x7f149409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494134df0 0x7f149411f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient(hunting): picked mon.noname-b con 0x7f1494134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14940a01a0 con 0x7f149406c890 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14940a0020 con 0x7f1494134df0 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f149409be20 con 0x7f149409c4a0 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient(hunting): _renew_subs 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f149a390640 10 monclient(hunting): authenticate will time out at 900.888255s 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1493fff640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f149406c890 msgr2=0x7f14941cb310 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1493fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f149406c890 0x7f14941cb310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f149409c4a0 0x7f14941cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f149409c4a0 0x7f14941cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42388/0 (socket says 10.20.193.78:42388) 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 -- 10.20.193.78:0/3025358660 learned_addr learned my addr 10.20.193.78:0/3025358660 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient(hunting): get_auth_request con 0x7f149409c4a0 auth_method 0 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1498906640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494134df0 0x7f149411f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1498906640 10 monclient(hunting): get_auth_request con 0x7f1494134df0 auth_method 0 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1498906640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1498906640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1498906640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1498906640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1498906640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f1498906640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient(hunting): handle_auth_done global_id 74168 payload 824 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494134df0 msgr2=0x7f149411f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494134df0 0x7f149411f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f149406c890 msgr2=0x7f14941cb310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f149406c890 0x7f14941cb310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient: _finish_hunting 0 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 monclient: found mon.noname-c 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.334+0000 7f14937fe640 1 -- 10.20.193.78:0/3025358660 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1494121960 con 0x7f149409c4a0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f14937fe640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f149409c4a0 msgr2=0x7f14941cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f14937fe640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f149409c4a0 0x7f14941cb850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1484009a70 tx=0x7f14840047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484046020 con 0x7f149409c4a0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: dump: 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: _finish_auth 0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f149a390640 5 monclient: authenticate success, global_id 74168 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1484042640 con 0x7f149409c4a0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148403c040 con 0x7f149409c4a0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f149a390640 10 monclient: _renew_subs 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f149a390640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1494121be0 con 0x7f149409c4a0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f14927fc640 4 set_mon_vals no callback set 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f1490ff9640 10 monclient: dump: 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.337 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f149a390640 10 monclient: _renew_subs 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f149a390640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.336+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1494072660 con 0x7f149409c4a0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f149a390640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f149a390640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494122ad0 0x7f149411f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f149a390640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1494122eb0 con 0x7f1494122ad0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494122ad0 0x7f149411f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f148403f030 con 0x7f149409c4a0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 10 monclient: get_auth_request con 0x7f1494122ad0 auth_method 0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1490ff9640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f146403de60 0x7f1464040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(185..185 src has 117..185) ==== 4843+0+0 (secure 0 0 0) 0x7f14840777e0 con 0x7f149409c4a0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 10 monclient(hunting): handle_auth_done global_id 44377 payload 824 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1493fff640 1 -- 10.20.193.78:0/3025358660 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14940a16d0 con 0x7f1494122ad0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1498906640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f146403de60 0x7f1464040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.337+0000 7f1498906640 10 monclient: get_auth_request con 0x7f146403de60 auth_method 0 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1493fff640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494122ad0 msgr2=0x7f149411f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1493fff640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494122ad0 0x7f149411f960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f148000bcc0 tx=0x7f148000bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480005070 con 0x7f1494122ad0 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 10 monclient: dump: 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.339 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1498906640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f146403de60 0x7f1464040320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f148800f2e0 tx=0x7f1488004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f1480005070 con 0x7f1494122ad0 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494122ad0 msgr2=0x7f149411f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494122ad0 0x7f149411f960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f148000bcc0 tx=0x7f148000bcf0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494122ad0 msgr2=0x7f149411f960 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:55.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.340+0000 7f1490ff9640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1494122ad0 0x7f149411f960 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f148000bcc0 tx=0x7f148000bcf0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f149a390640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:50:55.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f149a390640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149411fea0 0x7f14941d6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:50:55.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f149a390640 10 monclient(hunting): start opening mon connection 2026-02-17T05:50:55.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1494135a20 con 0x7f149411fea0 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149411fea0 0x7f14941d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 10 monclient: get_auth_request con 0x7f149411fea0 auth_method 0 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 10 monclient(hunting): handle_auth_done global_id 44380 payload 824 2026-02-17T05:50:55.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.343+0000 7f1493fff640 1 -- 10.20.193.78:0/3025358660 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f14940744d0 con 0x7f149411fea0 2026-02-17T05:50:55.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1493fff640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149411fea0 msgr2=0x7f14941d6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:50:55.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1493fff640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149411fea0 0x7f14941d6310 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f148000bcc0 tx=0x7f1480005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:50:55.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148001a730 con 0x7f149411fea0 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 10 monclient: dump: 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7f1480014430 con 0x7f149411fea0 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149411fea0 msgr2=0x7f14941d6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149411fea0 0x7f14941d6310 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f148000bcc0 tx=0x7f1480005ad0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149411fea0 msgr2=0x7f14941d6310 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1490ff9640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f149411fea0 0x7f14941d6310 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f148000bcc0 tx=0x7f1480005ad0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:50:55.346 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 48, 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 178994, 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:55.347 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:50:55.348 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:55.349 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:55.350 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.351 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.352 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:50:55.353 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f146403de60 msgr2=0x7f1464040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f146403de60 0x7f1464040320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f148800f2e0 tx=0x7f1488004200 comp rx=0 tx=0).stop 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 10 monclient: shutdown 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f149409c4a0 msgr2=0x7f14941cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 1 --2- 10.20.193.78:0/3025358660 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f149409c4a0 0x7f14941cb850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1484009a70 tx=0x7f14840047c0 comp rx=0 tx=0).stop 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1498906640 1 -- 10.20.193.78:0/3025358660 reap_dead start 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f1498906640 1 -- 10.20.193.78:0/3025358660 reap_dead start 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 shutdown_connections 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 >> 10.20.193.78:0/3025358660 conn(0x7f14940a6620 msgr2=0x7f149406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 shutdown_connections 2026-02-17T05:50:55.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:50:55.346+0000 7f149a390640 1 -- 10.20.193.78:0/3025358660 wait complete. 2026-02-17T05:50:55.355 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:50:55.355 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:50:55.878 INFO:tasks.rados.rados.0.trial115.stdout:859: done (1 left) 2026-02-17T05:50:55.878 INFO:tasks.rados.rados.0.trial115.stdout:860: done (0 left) 2026-02-17T05:50:55.878 INFO:tasks.rados.rados.0.trial115.stdout:861: delete oid 35 current snap is 86 2026-02-17T05:50:55.881 INFO:tasks.rados.rados.0.trial115.stdout:861: done (0 left) 2026-02-17T05:50:55.881 INFO:tasks.rados.rados.0.trial115.stdout:862: copy_from oid 33 from oid 9 current snap is 86 2026-02-17T05:50:55.881 INFO:tasks.rados.rados.0.trial115.stdout:863: copy_from oid 22 from oid 24 current snap is 86 2026-02-17T05:50:55.881 INFO:tasks.rados.rados.0.trial115.stdout:864: write (excl) oid 29 current snap is 86 2026-02-17T05:50:55.881 INFO:tasks.rados.rados.0.trial115.stdout:864: seq_num 344 ranges {581608=721541,1863688=485773,3113980=604043} 2026-02-17T05:50:55.886 INFO:tasks.rados.rados.0.trial115.stdout:864: writing trial11545420-29 from 581608 to 1303149 tid 1 2026-02-17T05:50:55.890 INFO:tasks.rados.rados.0.trial115.stdout:864: writing trial11545420-29 from 1863688 to 2349461 tid 2 2026-02-17T05:50:55.895 INFO:tasks.rados.rados.0.trial115.stdout:864: writing trial11545420-29 from 3113980 to 3718023 tid 3 2026-02-17T05:50:55.895 INFO:tasks.rados.rados.0.trial115.stdout:865: write oid 8 current snap is 86 2026-02-17T05:50:55.895 INFO:tasks.rados.rados.0.trial115.stdout:865: seq_num 345 ranges {598415=796790,1922170=745784,3141403=415315} 2026-02-17T05:50:55.901 INFO:tasks.rados.rados.0.trial115.stdout:865: writing trial11545420-8 from 598415 to 1395205 tid 1 2026-02-17T05:50:55.905 INFO:tasks.rados.rados.0.trial115.stdout:865: writing trial11545420-8 from 1922170 to 2667954 tid 2 2026-02-17T05:50:55.909 INFO:tasks.rados.rados.0.trial115.stdout:865: writing trial11545420-8 from 3141403 to 3556718 tid 3 2026-02-17T05:50:55.909 INFO:tasks.rados.rados.0.trial115.stdout:866: snap_create 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:863: finishing copy_from to trial11545420-22 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:863: got expected ENOENT (src dne) 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:863: finishing copy_from racing read to trial11545420-22 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:862: finishing copy_from racing read to trial11545420-33 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:864: finishing write tid 1 to trial11545420-29 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:862: finishing copy_from to trial11545420-33 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 171 (ObjNum 106 snap 16 seq_num 106) dirty exists 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:864: finishing write tid 2 to trial11545420-29 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:864: finishing write tid 3 to trial11545420-29 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:864: finishing write tid 4 to trial11545420-29 2026-02-17T05:50:55.910 INFO:tasks.rados.rados.0.trial115.stdout:864: finishing write tid 5 to trial11545420-29 2026-02-17T05:50:55.911 INFO:tasks.rados.rados.0.trial115.stdout:864: oid 29 updating version 0 to 305 2026-02-17T05:50:55.911 INFO:tasks.rados.rados.0.trial115.stdout:864: oid 29 updating version 305 to 307 2026-02-17T05:50:55.911 INFO:tasks.rados.rados.0.trial115.stdout:864: oid 29 version 307 is already newer than 304 2026-02-17T05:50:55.911 INFO:tasks.rados.rados.0.trial115.stdout:864: oid 29 version 307 is already newer than 306 2026-02-17T05:50:55.911 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 307 (ObjNum 344 snap 86 seq_num 344) dirty exists 2026-02-17T05:50:55.911 INFO:tasks.rados.rados.0.trial115.stdout:864: left oid 29 (ObjNum 344 snap 86 seq_num 344) 2026-02-17T05:50:55.911 INFO:tasks.rados.rados.0.trial115.stdout:865: finishing write tid 1 to trial11545420-8 2026-02-17T05:50:55.911 INFO:tasks.rados.rados.0.trial115.stdout:865: finishing write tid 2 to trial11545420-8 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:865: finishing write tid 3 to trial11545420-8 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:865: finishing write tid 4 to trial11545420-8 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:865: finishing write tid 5 to trial11545420-8 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:865: oid 8 updating version 0 to 183 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:865: oid 8 version 183 is already newer than 180 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:865: oid 8 version 183 is already newer than 181 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:865: oid 8 version 183 is already newer than 182 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 183 (ObjNum 345 snap 86 seq_num 345) dirty exists 2026-02-17T05:50:55.912 INFO:tasks.rados.rados.0.trial115.stdout:865: left oid 8 (ObjNum 345 snap 86 seq_num 345) 2026-02-17T05:50:56.568 INFO:tasks.rados.rados.0.trial115.stdout:862: done (4 left) 2026-02-17T05:50:56.569 INFO:tasks.rados.rados.0.trial115.stdout:863: done (3 left) 2026-02-17T05:50:56.569 INFO:tasks.rados.rados.0.trial115.stdout:864: done (2 left) 2026-02-17T05:50:56.569 INFO:tasks.rados.rados.0.trial115.stdout:865: done (1 left) 2026-02-17T05:50:56.569 INFO:tasks.rados.rados.0.trial115.stdout:866: done (0 left) 2026-02-17T05:50:56.569 INFO:tasks.rados.rados.0.trial115.stdout:867: read oid 36 snap -1 2026-02-17T05:50:56.569 INFO:tasks.rados.rados.0.trial115.stdout:867: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:50:56.569 INFO:tasks.rados.rados.0.trial115.stdout:868: snap_create 2026-02-17T05:50:56.575 INFO:tasks.rados.rados.0.trial115.stdout:867: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:50:57.568 INFO:tasks.rados.rados.0.trial115.stdout:867: done (1 left) 2026-02-17T05:50:57.569 INFO:tasks.rados.rados.0.trial115.stdout:868: done (0 left) 2026-02-17T05:50:57.569 INFO:tasks.rados.rados.0.trial115.stdout:869: rollback oid 47 current snap is 88 2026-02-17T05:50:57.569 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 47 to 59 2026-02-17T05:50:57.569 INFO:tasks.rados.rados.0.trial115.stdout:870: delete oid 1 current snap is 88 2026-02-17T05:50:57.572 INFO:tasks.rados.rados.0.trial115.stdout:870: done (1 left) 2026-02-17T05:50:57.572 INFO:tasks.rados.rados.0.trial115.stdout:871: read oid 14 snap -1 2026-02-17T05:50:57.572 INFO:tasks.rados.rados.0.trial115.stdout:871: expect deleted 2026-02-17T05:50:57.572 INFO:tasks.rados.rados.0.trial115.stdout:872: copy_from oid 48 from oid 7 current snap is 88 2026-02-17T05:50:57.572 INFO:tasks.rados.rados.0.trial115.stdout:873: read oid 43 snap 81 2026-02-17T05:50:57.573 INFO:tasks.rados.rados.0.trial115.stdout:873: expect (ObjNum 313 snap 74 seq_num 313) 2026-02-17T05:50:57.573 INFO:tasks.rados.rados.0.trial115.stdout:874: delete oid 33 current snap is 88 2026-02-17T05:50:57.573 INFO:tasks.rados.rados.0.trial115.stdout:869: finishing rollback tid 0 to trial11545420-47 2026-02-17T05:50:57.573 INFO:tasks.rados.rados.0.trial115.stdout:869: finishing rollback tid 1 to trial11545420-47 2026-02-17T05:50:57.573 INFO:tasks.rados.rados.0.trial115.stdout:869: finishing rollback tid 2 to trial11545420-47 2026-02-17T05:50:57.573 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 302 (ObjNum 251 snap 49 seq_num 251) dirty exists 2026-02-17T05:50:57.574 INFO:tasks.rados.rados.0.trial115.stdout:869: done (4 left) 2026-02-17T05:50:57.574 INFO:tasks.rados.rados.0.trial115.stdout:871: done (3 left) 2026-02-17T05:50:57.574 INFO:tasks.rados.rados.0.trial115.stdout:874: done (2 left) 2026-02-17T05:50:57.574 INFO:tasks.rados.rados.0.trial115.stdout:875: write oid 40 current snap is 88 2026-02-17T05:50:57.574 INFO:tasks.rados.rados.0.trial115.stdout:875: seq_num 348 ranges {648836=622537,1981263=493111,3107319=741517} 2026-02-17T05:50:57.579 INFO:tasks.rados.rados.0.trial115.stdout:875: writing trial11545420-40 from 648836 to 1271373 tid 1 2026-02-17T05:50:57.583 INFO:tasks.rados.rados.0.trial115.stdout:875: writing trial11545420-40 from 1981263 to 2474374 tid 2 2026-02-17T05:50:57.588 INFO:tasks.rados.rados.0.trial115.stdout:875: writing trial11545420-40 from 3107319 to 3848836 tid 3 2026-02-17T05:50:57.588 INFO:tasks.rados.rados.0.trial115.stdout:876: snap_remove snap 58 2026-02-17T05:50:58.572 INFO:tasks.rados.rados.0.trial115.stdout:876: done (3 left) 2026-02-17T05:50:58.573 INFO:tasks.rados.rados.0.trial115.stdout:877: read oid 19 snap -1 2026-02-17T05:50:58.573 INFO:tasks.rados.rados.0.trial115.stdout:877: expect (ObjNum 306 snap 73 seq_num 306) 2026-02-17T05:50:58.573 INFO:tasks.rados.rados.0.trial115.stdout:878: write (excl) oid 47 current snap is 88 2026-02-17T05:50:58.573 INFO:tasks.rados.rados.0.trial115.stdout:878: seq_num 349 ranges {665643=697786,2039745=753122,3534742=330901} 2026-02-17T05:50:58.577 INFO:tasks.rados.rados.0.trial115.stdout:878: writing trial11545420-47 from 665643 to 1363429 tid 1 2026-02-17T05:50:58.582 INFO:tasks.rados.rados.0.trial115.stdout:878: writing trial11545420-47 from 2039745 to 2792867 tid 2 2026-02-17T05:50:58.586 INFO:tasks.rados.rados.0.trial115.stdout:878: writing trial11545420-47 from 3534742 to 3865643 tid 3 2026-02-17T05:50:58.587 INFO:tasks.rados.rados.0.trial115.stdout:879: copy_from oid 35 from oid 34 current snap is 88 2026-02-17T05:50:58.587 INFO:tasks.rados.rados.0.trial115.stdout:873: expect (ObjNum 313 snap 74 seq_num 313) 2026-02-17T05:50:58.750 INFO:tasks.rados.rados.0.trial115.stdout:872: finishing copy_from racing read to trial11545420-48 2026-02-17T05:50:58.750 INFO:tasks.rados.rados.0.trial115.stdout:875: finishing write tid 1 to trial11545420-40 2026-02-17T05:50:58.750 INFO:tasks.rados.rados.0.trial115.stdout:873: done (5 left) 2026-02-17T05:50:58.750 INFO:tasks.rados.rados.0.trial115.stdout:880: copy_from oid 37 from oid 32 current snap is 88 2026-02-17T05:50:58.750 INFO:tasks.rados.rados.0.trial115.stdout:875: finishing write tid 2 to trial11545420-40 2026-02-17T05:50:58.750 INFO:tasks.rados.rados.0.trial115.stdout:872: finishing copy_from to trial11545420-48 2026-02-17T05:50:58.750 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 176 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:875: finishing write tid 3 to trial11545420-40 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:875: finishing write tid 4 to trial11545420-40 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:875: finishing write tid 5 to trial11545420-40 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:875: oid 40 updating version 0 to 188 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:875: oid 40 version 188 is already newer than 185 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:875: oid 40 version 188 is already newer than 186 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:875: oid 40 version 188 is already newer than 187 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 188 (ObjNum 348 snap 88 seq_num 348) dirty exists 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:875: left oid 40 (ObjNum 348 snap 88 seq_num 348) 2026-02-17T05:50:58.751 INFO:tasks.rados.rados.0.trial115.stdout:877: expect (ObjNum 306 snap 73 seq_num 306) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: finishing write tid 1 to trial11545420-47 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: finishing write tid 2 to trial11545420-47 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:872: done (5 left) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:875: done (4 left) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:877: done (3 left) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:881: rollback oid 33 current snap is 88 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 81 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:879: finishing copy_from to trial11545420-35 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:879: got expected ENOENT (src dne) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:879: finishing copy_from racing read to trial11545420-35 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: finishing write tid 3 to trial11545420-47 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: finishing write tid 4 to trial11545420-47 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: finishing write tid 5 to trial11545420-47 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: oid 47 updating version 0 to 303 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: oid 47 updating version 303 to 313 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: oid 47 updating version 313 to 314 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: oid 47 version 314 is already newer than 312 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 314 (ObjNum 349 snap 88 seq_num 349) dirty exists 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: left oid 47 (ObjNum 349 snap 88 seq_num 349) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:880: finishing copy_from to trial11545420-37 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:880: got expected ENOENT (src dne) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:880: finishing copy_from racing read to trial11545420-37 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:878: done (3 left) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:879: done (2 left) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:880: done (1 left) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:882: read oid 4 snap -1 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:882: expect (ObjNum 314 snap 74 seq_num 314) 2026-02-17T05:50:58.826 INFO:tasks.rados.rados.0.trial115.stdout:883: copy_from oid 43 from oid 45 current snap is 88 2026-02-17T05:50:58.827 INFO:tasks.rados.rados.0.trial115.stdout:884: snap_create 2026-02-17T05:50:58.827 INFO:tasks.rados.rados.0.trial115.stdout:881: finishing rollback tid 1 to trial11545420-33 2026-02-17T05:50:58.827 INFO:tasks.rados.rados.0.trial115.stdout:881: finishing rollback tid 2 to trial11545420-33 2026-02-17T05:50:58.827 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 308 (ObjNum 267 snap 54 seq_num 267) dirty exists 2026-02-17T05:50:58.830 INFO:tasks.rados.rados.0.trial115.stdout:882: expect (ObjNum 314 snap 74 seq_num 314) 2026-02-17T05:50:58.928 INFO:tasks.rados.rados.0.trial115.stdout:883: finishing copy_from racing read to trial11545420-43 2026-02-17T05:50:58.928 INFO:tasks.rados.rados.0.trial115.stdout:883: finishing copy_from to trial11545420-43 2026-02-17T05:50:58.928 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 394 (ObjNum 302 snap 72 seq_num 302) dirty exists 2026-02-17T05:50:59.575 INFO:tasks.rados.rados.0.trial115.stdout:881: done (3 left) 2026-02-17T05:50:59.575 INFO:tasks.rados.rados.0.trial115.stdout:882: done (2 left) 2026-02-17T05:50:59.575 INFO:tasks.rados.rados.0.trial115.stdout:883: done (1 left) 2026-02-17T05:50:59.575 INFO:tasks.rados.rados.0.trial115.stdout:884: done (0 left) 2026-02-17T05:50:59.576 INFO:tasks.rados.rados.0.trial115.stdout:885: snap_remove snap 73 2026-02-17T05:51:00.576 INFO:tasks.rados.rados.0.trial115.stdout:885: done (0 left) 2026-02-17T05:51:00.577 INFO:tasks.rados.rados.0.trial115.stdout:886: copy_from oid 29 from oid 31 current snap is 89 2026-02-17T05:51:00.577 INFO:tasks.rados.rados.0.trial115.stdout:887: write (excl) oid 42 current snap is 89 2026-02-17T05:51:00.577 INFO:tasks.rados.rados.0.trial115.stdout:887: seq_num 350 ranges {682450=489388,1814580=496780,2878518=788028} 2026-02-17T05:51:00.581 INFO:tasks.rados.rados.0.trial115.stdout:887: writing trial11545420-42 from 682450 to 1171838 tid 1 2026-02-17T05:51:00.585 INFO:tasks.rados.rados.0.trial115.stdout:887: writing trial11545420-42 from 1814580 to 2311360 tid 2 2026-02-17T05:51:00.591 INFO:tasks.rados.rados.0.trial115.stdout:887: writing trial11545420-42 from 2878518 to 3666546 tid 3 2026-02-17T05:51:00.591 INFO:tasks.rados.rados.0.trial115.stdout:888: read oid 24 snap -1 2026-02-17T05:51:00.591 INFO:tasks.rados.rados.0.trial115.stdout:888: expect deleted 2026-02-17T05:51:00.591 INFO:tasks.rados.rados.0.trial115.stdout:889: snap_remove snap 66 2026-02-17T05:51:01.371 INFO:tasks.rados.rados.0.trial115.stdout:889: done (3 left) 2026-02-17T05:51:01.371 INFO:tasks.rados.rados.0.trial115.stdout:890: copy_from oid 4 from oid 3 current snap is 89 2026-02-17T05:51:01.371 INFO:tasks.rados.rados.0.trial115.stdout:887: finishing write tid 1 to trial11545420-42 2026-02-17T05:51:01.371 INFO:tasks.rados.rados.0.trial115.stdout:887: finishing write tid 2 to trial11545420-42 2026-02-17T05:51:01.371 INFO:tasks.rados.rados.0.trial115.stdout:886: finishing copy_from racing read to trial11545420-29 2026-02-17T05:51:01.371 INFO:tasks.rados.rados.0.trial115.stdout:886: finishing copy_from to trial11545420-29 2026-02-17T05:51:01.371 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 272 (ObjNum 273 snap 56 seq_num 273) dirty exists 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:886: done (3 left) 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:888: done (2 left) 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:891: delete oid 7 current snap is 89 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:887: finishing write tid 3 to trial11545420-42 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:887: finishing write tid 4 to trial11545420-42 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:887: finishing write tid 5 to trial11545420-42 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:887: oid 42 updating version 0 to 418 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:887: oid 42 updating version 418 to 419 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:887: oid 42 version 419 is already newer than 415 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:887: oid 42 version 419 is already newer than 410 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 419 (ObjNum 350 snap 89 seq_num 350) dirty exists 2026-02-17T05:51:01.372 INFO:tasks.rados.rados.0.trial115.stdout:887: left oid 42 (ObjNum 350 snap 89 seq_num 350) 2026-02-17T05:51:01.373 INFO:tasks.rados.rados.0.trial115.stdout:887: done (2 left) 2026-02-17T05:51:01.373 INFO:tasks.rados.rados.0.trial115.stdout:891: done (1 left) 2026-02-17T05:51:01.374 INFO:tasks.rados.rados.0.trial115.stdout:892: copy_from oid 44 from oid 19 current snap is 89 2026-02-17T05:51:01.374 INFO:tasks.rados.rados.0.trial115.stdout:893: snap_create 2026-02-17T05:51:01.377 INFO:tasks.rados.rados.0.trial115.stdout:892: finishing copy_from racing read to trial11545420-44 2026-02-17T05:51:01.378 INFO:tasks.rados.rados.0.trial115.stdout:892: finishing copy_from to trial11545420-44 2026-02-17T05:51:01.378 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 310 (ObjNum 306 snap 73 seq_num 306) dirty exists 2026-02-17T05:51:01.381 INFO:tasks.rados.rados.0.trial115.stdout:890: finishing copy_from racing read to trial11545420-4 2026-02-17T05:51:01.382 INFO:tasks.rados.rados.0.trial115.stdout:890: finishing copy_from to trial11545420-4 2026-02-17T05:51:01.382 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 275 (ObjNum 196 snap 37 seq_num 196) dirty exists 2026-02-17T05:51:02.370 INFO:tasks.rados.rados.0.trial115.stdout:890: done (2 left) 2026-02-17T05:51:02.370 INFO:tasks.rados.rados.0.trial115.stdout:892: done (1 left) 2026-02-17T05:51:02.370 INFO:tasks.rados.rados.0.trial115.stdout:893: done (0 left) 2026-02-17T05:51:02.370 INFO:tasks.rados.rados.0.trial115.stdout:894: read oid 35 snap -1 2026-02-17T05:51:02.370 INFO:tasks.rados.rados.0.trial115.stdout:894: expect deleted 2026-02-17T05:51:02.370 INFO:tasks.rados.rados.0.trial115.stdout:895: write (excl) oid 42 current snap is 90 2026-02-17T05:51:02.370 INFO:tasks.rados.rados.0.trial115.stdout:895: seq_num 352 ranges {716064=639886,1815191=500449,2817011=526925} 2026-02-17T05:51:02.375 INFO:tasks.rados.rados.0.trial115.stdout:895: writing trial11545420-42 from 716064 to 1355950 tid 1 2026-02-17T05:51:02.378 INFO:tasks.rados.rados.0.trial115.stdout:895: writing trial11545420-42 from 1815191 to 2315640 tid 2 2026-02-17T05:51:02.383 INFO:tasks.rados.rados.0.trial115.stdout:895: writing trial11545420-42 from 2817011 to 3343936 tid 3 2026-02-17T05:51:02.383 INFO:tasks.rados.rados.0.trial115.stdout:896: read oid 30 snap -1 2026-02-17T05:51:02.384 INFO:tasks.rados.rados.0.trial115.stdout:896: expect (ObjNum 298 snap 70 seq_num 298) 2026-02-17T05:51:02.384 INFO:tasks.rados.rados.0.trial115.stdout:897: write oid 49 current snap is 90 2026-02-17T05:51:02.384 INFO:tasks.rados.rados.0.trial115.stdout:897: seq_num 353 ranges {732871=715135,1873673=644107,3244434=688437} 2026-02-17T05:51:02.388 INFO:tasks.rados.rados.0.trial115.stdout:897: writing trial11545420-49 from 732871 to 1448006 tid 1 2026-02-17T05:51:02.392 INFO:tasks.rados.rados.0.trial115.stdout:897: writing trial11545420-49 from 1873673 to 2517780 tid 2 2026-02-17T05:51:02.397 INFO:tasks.rados.rados.0.trial115.stdout:897: writing trial11545420-49 from 3244434 to 3932871 tid 3 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:898: snap_create 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: finishing write tid 1 to trial11545420-42 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: finishing write tid 2 to trial11545420-42 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: finishing write tid 3 to trial11545420-42 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: finishing write tid 4 to trial11545420-42 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: finishing write tid 5 to trial11545420-42 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: oid 42 updating version 0 to 430 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: oid 42 version 430 is already newer than 429 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: oid 42 version 430 is already newer than 427 2026-02-17T05:51:02.398 INFO:tasks.rados.rados.0.trial115.stdout:895: oid 42 version 430 is already newer than 428 2026-02-17T05:51:02.399 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 430 (ObjNum 352 snap 90 seq_num 352) dirty exists 2026-02-17T05:51:02.399 INFO:tasks.rados.rados.0.trial115.stdout:895: left oid 42 (ObjNum 352 snap 90 seq_num 352) 2026-02-17T05:51:02.399 INFO:tasks.rados.rados.0.trial115.stdout:896: expect (ObjNum 298 snap 70 seq_num 298) 2026-02-17T05:51:02.471 INFO:tasks.rados.rados.0.trial115.stdout:897: finishing write tid 1 to trial11545420-49 2026-02-17T05:51:02.471 INFO:tasks.rados.rados.0.trial115.stdout:897: finishing write tid 2 to trial11545420-49 2026-02-17T05:51:02.471 INFO:tasks.rados.rados.0.trial115.stdout:897: finishing write tid 3 to trial11545420-49 2026-02-17T05:51:02.471 INFO:tasks.rados.rados.0.trial115.stdout:897: finishing write tid 4 to trial11545420-49 2026-02-17T05:51:02.471 INFO:tasks.rados.rados.0.trial115.stdout:897: finishing write tid 5 to trial11545420-49 2026-02-17T05:51:02.471 INFO:tasks.rados.rados.0.trial115.stdout:897: oid 49 updating version 0 to 279 2026-02-17T05:51:02.471 INFO:tasks.rados.rados.0.trial115.stdout:897: oid 49 version 279 is already newer than 278 2026-02-17T05:51:02.471 INFO:tasks.rados.rados.0.trial115.stdout:897: oid 49 version 279 is already newer than 276 2026-02-17T05:51:02.472 INFO:tasks.rados.rados.0.trial115.stdout:897: oid 49 version 279 is already newer than 277 2026-02-17T05:51:02.472 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 279 (ObjNum 353 snap 90 seq_num 353) dirty exists 2026-02-17T05:51:02.472 INFO:tasks.rados.rados.0.trial115.stdout:897: left oid 49 (ObjNum 353 snap 90 seq_num 353) 2026-02-17T05:51:03.370 INFO:tasks.rados.rados.0.trial115.stdout:894: done (4 left) 2026-02-17T05:51:03.371 INFO:tasks.rados.rados.0.trial115.stdout:895: done (3 left) 2026-02-17T05:51:03.371 INFO:tasks.rados.rados.0.trial115.stdout:896: done (2 left) 2026-02-17T05:51:03.371 INFO:tasks.rados.rados.0.trial115.stdout:897: done (1 left) 2026-02-17T05:51:03.371 INFO:tasks.rados.rados.0.trial115.stdout:898: done (0 left) 2026-02-17T05:51:03.371 INFO:tasks.rados.rados.0.trial115.stdout:899: write (excl) oid 6 current snap is 91 2026-02-17T05:51:03.371 INFO:tasks.rados.rados.0.trial115.stdout:899: seq_num 354 ranges {749678=790384,2332155=504118,3271857=549469} 2026-02-17T05:51:03.376 INFO:tasks.rados.rados.0.trial115.stdout:899: writing trial11545420-6 from 749678 to 1540062 tid 1 2026-02-17T05:51:03.381 INFO:tasks.rados.rados.0.trial115.stdout:899: writing trial11545420-6 from 2332155 to 2836273 tid 2 2026-02-17T05:51:03.385 INFO:tasks.rados.rados.0.trial115.stdout:899: writing trial11545420-6 from 3271857 to 3821326 tid 3 2026-02-17T05:51:03.385 INFO:tasks.rados.rados.0.trial115.stdout:900: read oid 4 snap 67 2026-02-17T05:51:03.385 INFO:tasks.rados.rados.0.trial115.stdout:900: expect (ObjNum 242 snap 48 seq_num 242) 2026-02-17T05:51:03.385 INFO:tasks.rados.rados.0.trial115.stdout:901: delete oid 16 current snap is 91 2026-02-17T05:51:03.385 INFO:tasks.rados.rados.0.trial115.stdout:899: finishing write tid 1 to trial11545420-6 2026-02-17T05:51:03.385 INFO:tasks.rados.rados.0.trial115.stdout:899: finishing write tid 2 to trial11545420-6 2026-02-17T05:51:03.386 INFO:tasks.rados.rados.0.trial115.stdout:901: done (2 left) 2026-02-17T05:51:03.386 INFO:tasks.rados.rados.0.trial115.stdout:902: write (excl) oid 37 current snap is 91 2026-02-17T05:51:03.387 INFO:tasks.rados.rados.0.trial115.stdout:902: seq_num 356 ranges {783292=540882,1932766=507787,3210350=572013} 2026-02-17T05:51:03.391 INFO:tasks.rados.rados.0.trial115.stdout:902: writing trial11545420-37 from 783292 to 1324174 tid 1 2026-02-17T05:51:03.395 INFO:tasks.rados.rados.0.trial115.stdout:902: writing trial11545420-37 from 1932766 to 2440553 tid 2 2026-02-17T05:51:03.401 INFO:tasks.rados.rados.0.trial115.stdout:902: writing trial11545420-37 from 3210350 to 3782363 tid 3 2026-02-17T05:51:03.401 INFO:tasks.rados.rados.0.trial115.stdout:903: snap_create 2026-02-17T05:51:03.401 INFO:tasks.rados.rados.0.trial115.stdout:899: finishing write tid 3 to trial11545420-6 2026-02-17T05:51:03.401 INFO:tasks.rados.rados.0.trial115.stdout:899: finishing write tid 4 to trial11545420-6 2026-02-17T05:51:03.401 INFO:tasks.rados.rados.0.trial115.stdout:899: finishing write tid 5 to trial11545420-6 2026-02-17T05:51:03.401 INFO:tasks.rados.rados.0.trial115.stdout:899: oid 6 updating version 0 to 290 2026-02-17T05:51:03.401 INFO:tasks.rados.rados.0.trial115.stdout:899: oid 6 version 290 is already newer than 289 2026-02-17T05:51:03.402 INFO:tasks.rados.rados.0.trial115.stdout:899: oid 6 updating version 290 to 291 2026-02-17T05:51:03.402 INFO:tasks.rados.rados.0.trial115.stdout:899: oid 6 updating version 291 to 292 2026-02-17T05:51:03.402 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 292 (ObjNum 354 snap 91 seq_num 354) dirty exists 2026-02-17T05:51:03.402 INFO:tasks.rados.rados.0.trial115.stdout:899: left oid 6 (ObjNum 354 snap 91 seq_num 354) 2026-02-17T05:51:03.402 INFO:tasks.rados.rados.0.trial115.stdout:900: expect (ObjNum 242 snap 48 seq_num 242) 2026-02-17T05:51:03.447 INFO:tasks.rados.rados.0.trial115.stdout:902: finishing write tid 1 to trial11545420-37 2026-02-17T05:51:03.447 INFO:tasks.rados.rados.0.trial115.stdout:902: finishing write tid 2 to trial11545420-37 2026-02-17T05:51:03.448 INFO:tasks.rados.rados.0.trial115.stdout:902: finishing write tid 3 to trial11545420-37 2026-02-17T05:51:03.448 INFO:tasks.rados.rados.0.trial115.stdout:902: finishing write tid 4 to trial11545420-37 2026-02-17T05:51:03.448 INFO:tasks.rados.rados.0.trial115.stdout:902: finishing write tid 5 to trial11545420-37 2026-02-17T05:51:03.448 INFO:tasks.rados.rados.0.trial115.stdout:902: oid 37 updating version 0 to 399 2026-02-17T05:51:03.448 INFO:tasks.rados.rados.0.trial115.stdout:902: oid 37 version 399 is already newer than 396 2026-02-17T05:51:03.448 INFO:tasks.rados.rados.0.trial115.stdout:902: oid 37 version 399 is already newer than 398 2026-02-17T05:51:03.448 INFO:tasks.rados.rados.0.trial115.stdout:902: oid 37 version 399 is already newer than 397 2026-02-17T05:51:03.449 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 399 (ObjNum 356 snap 91 seq_num 356) dirty exists 2026-02-17T05:51:03.449 INFO:tasks.rados.rados.0.trial115.stdout:902: left oid 37 (ObjNum 356 snap 91 seq_num 356) 2026-02-17T05:51:04.373 INFO:tasks.rados.rados.0.trial115.stdout:899: done (3 left) 2026-02-17T05:51:04.373 INFO:tasks.rados.rados.0.trial115.stdout:900: done (2 left) 2026-02-17T05:51:04.373 INFO:tasks.rados.rados.0.trial115.stdout:902: done (1 left) 2026-02-17T05:51:04.373 INFO:tasks.rados.rados.0.trial115.stdout:903: done (0 left) 2026-02-17T05:51:04.374 INFO:tasks.rados.rados.0.trial115.stdout:904: write (excl) oid 40 current snap is 92 2026-02-17T05:51:04.374 INFO:tasks.rados.rados.0.trial115.stdout:904: seq_num 357 ranges {400099=616131,1591248=408851} 2026-02-17T05:51:04.377 INFO:tasks.rados.rados.0.trial115.stdout:904: writing trial11545420-40 from 400099 to 1016230 tid 1 2026-02-17T05:51:04.381 INFO:tasks.rados.rados.0.trial115.stdout:904: writing trial11545420-40 from 1591248 to 2000099 tid 2 2026-02-17T05:51:04.381 INFO:tasks.rados.rados.0.trial115.stdout:905: write (excl) oid 9 current snap is 92 2026-02-17T05:51:04.381 INFO:tasks.rados.rados.0.trial115.stdout:905: seq_num 358 ranges {416906=407733,1366083=511456} 2026-02-17T05:51:04.383 INFO:tasks.rados.rados.0.trial115.stdout:905: writing trial11545420-9 from 416906 to 824639 tid 1 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:905: writing trial11545420-9 from 1366083 to 1877539 tid 2 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:906: delete oid 33 current snap is 92 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:904: finishing write tid 1 to trial11545420-40 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:904: finishing write tid 2 to trial11545420-40 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:904: finishing write tid 3 to trial11545420-40 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:904: finishing write tid 4 to trial11545420-40 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:904: oid 40 updating version 0 to 282 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:904: oid 40 version 282 is already newer than 280 2026-02-17T05:51:04.388 INFO:tasks.rados.rados.0.trial115.stdout:904: oid 40 version 282 is already newer than 281 2026-02-17T05:51:04.389 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 282 (ObjNum 357 snap 92 seq_num 357) dirty exists 2026-02-17T05:51:04.389 INFO:tasks.rados.rados.0.trial115.stdout:904: left oid 40 (ObjNum 357 snap 92 seq_num 357) 2026-02-17T05:51:04.389 INFO:tasks.rados.rados.0.trial115.stdout:905: finishing write tid 1 to trial11545420-9 2026-02-17T05:51:04.389 INFO:tasks.rados.rados.0.trial115.stdout:904: done (2 left) 2026-02-17T05:51:04.389 INFO:tasks.rados.rados.0.trial115.stdout:906: done (1 left) 2026-02-17T05:51:04.389 INFO:tasks.rados.rados.0.trial115.stdout:907: write oid 39 current snap is 92 2026-02-17T05:51:04.390 INFO:tasks.rados.rados.0.trial115.stdout:907: seq_num 360 ranges {450520=558231,1483047=515125} 2026-02-17T05:51:04.393 INFO:tasks.rados.rados.0.trial115.stdout:907: writing trial11545420-39 from 450520 to 1008751 tid 1 2026-02-17T05:51:04.397 INFO:tasks.rados.rados.0.trial115.stdout:907: writing trial11545420-39 from 1483047 to 1998172 tid 2 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:908: snap_create 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:905: finishing write tid 2 to trial11545420-9 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:905: finishing write tid 3 to trial11545420-9 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:905: finishing write tid 4 to trial11545420-9 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:905: oid 9 updating version 0 to 373 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:905: oid 9 updating version 373 to 374 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:905: oid 9 updating version 374 to 375 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 375 (ObjNum 358 snap 92 seq_num 358) dirty exists 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:905: left oid 9 (ObjNum 358 snap 92 seq_num 358) 2026-02-17T05:51:04.398 INFO:tasks.rados.rados.0.trial115.stdout:907: finishing write tid 1 to trial11545420-39 2026-02-17T05:51:04.399 INFO:tasks.rados.rados.0.trial115.stdout:907: finishing write tid 2 to trial11545420-39 2026-02-17T05:51:04.399 INFO:tasks.rados.rados.0.trial115.stdout:907: finishing write tid 3 to trial11545420-39 2026-02-17T05:51:04.399 INFO:tasks.rados.rados.0.trial115.stdout:907: finishing write tid 4 to trial11545420-39 2026-02-17T05:51:04.400 INFO:tasks.rados.rados.0.trial115.stdout:907: oid 39 updating version 0 to 433 2026-02-17T05:51:04.400 INFO:tasks.rados.rados.0.trial115.stdout:907: oid 39 updating version 433 to 434 2026-02-17T05:51:04.400 INFO:tasks.rados.rados.0.trial115.stdout:907: oid 39 version 434 is already newer than 432 2026-02-17T05:51:04.400 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 434 (ObjNum 360 snap 92 seq_num 360) dirty exists 2026-02-17T05:51:04.400 INFO:tasks.rados.rados.0.trial115.stdout:907: left oid 39 (ObjNum 360 snap 92 seq_num 360) 2026-02-17T05:51:05.375 INFO:tasks.rados.rados.0.trial115.stdout:905: done (2 left) 2026-02-17T05:51:05.375 INFO:tasks.rados.rados.0.trial115.stdout:907: done (1 left) 2026-02-17T05:51:05.375 INFO:tasks.rados.rados.0.trial115.stdout:908: done (0 left) 2026-02-17T05:51:05.375 INFO:tasks.rados.rados.0.trial115.stdout:909: read oid 35 snap -1 2026-02-17T05:51:05.375 INFO:tasks.rados.rados.0.trial115.stdout:909: expect deleted 2026-02-17T05:51:05.375 INFO:tasks.rados.rados.0.trial115.stdout:910: snap_create 2026-02-17T05:51:06.369 INFO:tasks.rados.rados.0.trial115.stdout:909: done (1 left) 2026-02-17T05:51:06.369 INFO:tasks.rados.rados.0.trial115.stdout:910: done (0 left) 2026-02-17T05:51:06.369 INFO:tasks.rados.rados.0.trial115.stdout:911: copy_from oid 5 from oid 11 current snap is 94 2026-02-17T05:51:06.369 INFO:tasks.rados.rados.0.trial115.stdout:912: snap_create 2026-02-17T05:51:06.379 INFO:tasks.rados.rados.0.trial115.stdout:911: finishing copy_from racing read to trial11545420-5 2026-02-17T05:51:06.379 INFO:tasks.rados.rados.0.trial115.stdout:911: finishing copy_from to trial11545420-5 2026-02-17T05:51:06.379 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 339 (ObjNum 303 snap 72 seq_num 303) dirty exists 2026-02-17T05:51:07.371 INFO:tasks.rados.rados.0.trial115.stdout:911: done (1 left) 2026-02-17T05:51:07.371 INFO:tasks.rados.rados.0.trial115.stdout:912: done (0 left) 2026-02-17T05:51:07.371 INFO:tasks.rados.rados.0.trial115.stdout:913: snap_create 2026-02-17T05:51:08.371 INFO:tasks.rados.rados.0.trial115.stdout:913: done (0 left) 2026-02-17T05:51:08.371 INFO:tasks.rados.rados.0.trial115.stdout:914: snap_remove snap 88 2026-02-17T05:51:09.372 INFO:tasks.rados.rados.0.trial115.stdout:914: done (0 left) 2026-02-17T05:51:09.372 INFO:tasks.rados.rados.0.trial115.stdout:915: delete oid 3 current snap is 96 2026-02-17T05:51:09.376 INFO:tasks.rados.rados.0.trial115.stdout:915: done (0 left) 2026-02-17T05:51:09.376 INFO:tasks.rados.rados.0.trial115.stdout:916: read oid 23 snap -1 2026-02-17T05:51:09.376 INFO:tasks.rados.rados.0.trial115.stdout:916: expect (ObjNum 323 snap 78 seq_num 323) 2026-02-17T05:51:09.376 INFO:tasks.rados.rados.0.trial115.stdout:917: write oid 39 current snap is 96 2026-02-17T05:51:09.376 INFO:tasks.rados.rados.0.trial115.stdout:917: seq_num 362 ranges {484134=708729,1883658=200476} 2026-02-17T05:51:09.381 INFO:tasks.rados.rados.0.trial115.stdout:917: writing trial11545420-39 from 484134 to 1192863 tid 1 2026-02-17T05:51:09.385 INFO:tasks.rados.rados.0.trial115.stdout:917: writing trial11545420-39 from 1883658 to 2084134 tid 2 2026-02-17T05:51:09.385 INFO:tasks.rados.rados.0.trial115.stdout:918: rollback oid 29 current snap is 96 2026-02-17T05:51:09.385 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 92 2026-02-17T05:51:09.385 INFO:tasks.rados.rados.0.trial115.stdout:916: expect (ObjNum 323 snap 78 seq_num 323) 2026-02-17T05:51:09.469 INFO:tasks.rados.rados.0.trial115.stdout:917: finishing write tid 1 to trial11545420-39 2026-02-17T05:51:09.469 INFO:tasks.rados.rados.0.trial115.stdout:917: finishing write tid 2 to trial11545420-39 2026-02-17T05:51:09.469 INFO:tasks.rados.rados.0.trial115.stdout:916: done (2 left) 2026-02-17T05:51:09.469 INFO:tasks.rados.rados.0.trial115.stdout:919: read oid 46 snap -1 2026-02-17T05:51:09.469 INFO:tasks.rados.rados.0.trial115.stdout:919: expect (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:51:09.469 INFO:tasks.rados.rados.0.trial115.stdout:920: rollback oid 23 current snap is 96 2026-02-17T05:51:09.469 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 23 to 86 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:917: finishing write tid 3 to trial11545420-39 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:917: finishing write tid 4 to trial11545420-39 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:917: oid 39 updating version 0 to 441 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:917: oid 39 version 441 is already newer than 440 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:917: oid 39 updating version 441 to 442 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 442 (ObjNum 362 snap 96 seq_num 362) dirty exists 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:917: left oid 39 (ObjNum 362 snap 96 seq_num 362) 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:918: finishing rollback tid 0 to trial11545420-29 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:918: finishing rollback tid 1 to trial11545420-29 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:918: finishing rollback tid 2 to trial11545420-29 2026-02-17T05:51:09.470 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 310 (ObjNum 273 snap 56 seq_num 273) dirty exists 2026-02-17T05:51:09.471 INFO:tasks.rados.rados.0.trial115.stdout:917: done (3 left) 2026-02-17T05:51:09.471 INFO:tasks.rados.rados.0.trial115.stdout:918: done (2 left) 2026-02-17T05:51:09.471 INFO:tasks.rados.rados.0.trial115.stdout:921: read oid 30 snap -1 2026-02-17T05:51:09.471 INFO:tasks.rados.rados.0.trial115.stdout:921: expect (ObjNum 298 snap 70 seq_num 298) 2026-02-17T05:51:09.471 INFO:tasks.rados.rados.0.trial115.stdout:922: copy_from oid 47 from oid 49 current snap is 96 2026-02-17T05:51:09.471 INFO:tasks.rados.rados.0.trial115.stdout:923: snap_create 2026-02-17T05:51:09.471 INFO:tasks.rados.rados.0.trial115.stdout:919: expect (ObjNum 135 snap 23 seq_num 135) 2026-02-17T05:51:09.546 INFO:tasks.rados.rados.0.trial115.stdout:921: expect (ObjNum 298 snap 70 seq_num 298) 2026-02-17T05:51:09.618 INFO:tasks.rados.rados.0.trial115.stdout:920: finishing rollback tid 0 to trial11545420-23 2026-02-17T05:51:09.618 INFO:tasks.rados.rados.0.trial115.stdout:920: finishing rollback tid 1 to trial11545420-23 2026-02-17T05:51:09.618 INFO:tasks.rados.rados.0.trial115.stdout:920: finishing rollback tid 2 to trial11545420-23 2026-02-17T05:51:09.619 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 133 (ObjNum 323 snap 78 seq_num 323) dirty exists 2026-02-17T05:51:09.619 INFO:tasks.rados.rados.0.trial115.stdout:922: finishing copy_from racing read to trial11545420-47 2026-02-17T05:51:09.619 INFO:tasks.rados.rados.0.trial115.stdout:922: finishing copy_from to trial11545420-47 2026-02-17T05:51:09.619 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 279 (ObjNum 353 snap 90 seq_num 353) dirty exists 2026-02-17T05:51:10.378 INFO:tasks.rados.rados.0.trial115.stdout:919: done (4 left) 2026-02-17T05:51:10.379 INFO:tasks.rados.rados.0.trial115.stdout:920: done (3 left) 2026-02-17T05:51:10.379 INFO:tasks.rados.rados.0.trial115.stdout:921: done (2 left) 2026-02-17T05:51:10.379 INFO:tasks.rados.rados.0.trial115.stdout:922: done (1 left) 2026-02-17T05:51:10.379 INFO:tasks.rados.rados.0.trial115.stdout:923: done (0 left) 2026-02-17T05:51:10.379 INFO:tasks.rados.rados.0.trial115.stdout:924: snap_remove snap 96 2026-02-17T05:51:11.379 INFO:tasks.rados.rados.0.trial115.stdout:924: done (0 left) 2026-02-17T05:51:11.379 INFO:tasks.rados.rados.0.trial115.stdout:925: rollback oid 34 current snap is 97 2026-02-17T05:51:11.380 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 93 2026-02-17T05:51:11.380 INFO:tasks.rados.rados.0.trial115.stdout:926: write oid 26 current snap is 97 2026-02-17T05:51:11.380 INFO:tasks.rados.rados.0.trial115.stdout:926: seq_num 363 ranges {500941=783978,1942140=158801} 2026-02-17T05:51:11.385 INFO:tasks.rados.rados.0.trial115.stdout:926: writing trial11545420-26 from 500941 to 1284919 tid 1 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:926: writing trial11545420-26 from 1942140 to 2100941 tid 2 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:927: read oid 14 snap -1 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:927: expect deleted 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:928: copy_from oid 30 from oid 3 current snap is 97 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:925: finishing rollback tid 1 to trial11545420-34 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 251 (ObjNum 49 snap 0 seq_num 1634300532) dirty dne 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:925: done (3 left) 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:929: rollback oid 27 current snap is 97 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 95 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:926: finishing write tid 1 to trial11545420-26 2026-02-17T05:51:11.388 INFO:tasks.rados.rados.0.trial115.stdout:930: snap_remove snap 92 2026-02-17T05:51:12.383 INFO:tasks.rados.rados.0.trial115.stdout:930: done (4 left) 2026-02-17T05:51:12.383 INFO:tasks.rados.rados.0.trial115.stdout:931: read oid 31 snap -1 2026-02-17T05:51:12.383 INFO:tasks.rados.rados.0.trial115.stdout:931: expect (ObjNum 273 snap 56 seq_num 273) 2026-02-17T05:51:12.383 INFO:tasks.rados.rados.0.trial115.stdout:932: rollback oid 28 current snap is 97 2026-02-17T05:51:12.383 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 28 to 89 2026-02-17T05:51:12.384 INFO:tasks.rados.rados.0.trial115.stdout:926: finishing write tid 2 to trial11545420-26 2026-02-17T05:51:12.384 INFO:tasks.rados.rados.0.trial115.stdout:927: done (5 left) 2026-02-17T05:51:12.384 INFO:tasks.rados.rados.0.trial115.stdout:933: write oid 17 current snap is 97 2026-02-17T05:51:12.384 INFO:tasks.rados.rados.0.trial115.stdout:933: seq_num 364 ranges {517748=459227,1600622=406110} 2026-02-17T05:51:12.386 INFO:tasks.rados.rados.0.trial115.stdout:933: writing trial11545420-17 from 517748 to 976975 tid 1 2026-02-17T05:51:12.390 INFO:tasks.rados.rados.0.trial115.stdout:933: writing trial11545420-17 from 1600622 to 2006732 tid 2 2026-02-17T05:51:12.390 INFO:tasks.rados.rados.0.trial115.stdout:934: delete oid 25 current snap is 97 2026-02-17T05:51:12.390 INFO:tasks.rados.rados.0.trial115.stdout:926: finishing write tid 3 to trial11545420-26 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:926: finishing write tid 4 to trial11545420-26 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:926: oid 26 updating version 0 to 184 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:926: oid 26 updating version 184 to 185 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:926: oid 26 updating version 185 to 186 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 186 (ObjNum 363 snap 97 seq_num 363) dirty exists 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:926: left oid 26 (ObjNum 363 snap 97 seq_num 363) 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:928: finishing copy_from to trial11545420-30 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:928: got expected ENOENT (src dne) 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:928: finishing copy_from racing read to trial11545420-30 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:929: finishing rollback tid 0 to trial11545420-27 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:929: finishing rollback tid 1 to trial11545420-27 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:929: finishing rollback tid 2 to trial11545420-27 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 313 (ObjNum 267 snap 54 seq_num 267) dirty exists 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:932: finishing rollback tid 1 to trial11545420-28 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 252 (ObjNum 305 snap 0 seq_num 1) dirty dne 2026-02-17T05:51:12.391 INFO:tasks.rados.rados.0.trial115.stdout:931: expect (ObjNum 273 snap 56 seq_num 273) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: finishing write tid 1 to trial11545420-17 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: finishing write tid 2 to trial11545420-17 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: finishing write tid 3 to trial11545420-17 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: finishing write tid 4 to trial11545420-17 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: oid 17 updating version 0 to 134 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: oid 17 updating version 134 to 138 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: oid 17 version 138 is already newer than 137 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 138 (ObjNum 364 snap 97 seq_num 364) dirty exists 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: left oid 17 (ObjNum 364 snap 97 seq_num 364) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:926: done (6 left) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:928: done (5 left) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:929: done (4 left) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:931: done (3 left) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:932: done (2 left) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:933: done (1 left) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:934: done (0 left) 2026-02-17T05:51:12.478 INFO:tasks.rados.rados.0.trial115.stdout:935: delete oid 49 current snap is 97 2026-02-17T05:51:12.479 INFO:tasks.rados.rados.0.trial115.stdout:935: done (0 left) 2026-02-17T05:51:12.479 INFO:tasks.rados.rados.0.trial115.stdout:936: write oid 29 current snap is 97 2026-02-17T05:51:12.479 INFO:tasks.rados.rados.0.trial115.stdout:936: seq_num 367 ranges {568169=401327,1376068=669790} 2026-02-17T05:51:12.484 INFO:tasks.rados.rados.0.trial115.stdout:936: writing trial11545420-29 from 568169 to 969496 tid 1 2026-02-17T05:51:12.488 INFO:tasks.rados.rados.0.trial115.stdout:936: writing trial11545420-29 from 1376068 to 2045858 tid 2 2026-02-17T05:51:12.488 INFO:tasks.rados.rados.0.trial115.stdout:937: write (excl) oid 3 current snap is 97 2026-02-17T05:51:12.488 INFO:tasks.rados.rados.0.trial115.stdout:937: seq_num 368 ranges {584976=476576,1834550=350426} 2026-02-17T05:51:12.492 INFO:tasks.rados.rados.0.trial115.stdout:937: writing trial11545420-3 from 584976 to 1061552 tid 1 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:937: writing trial11545420-3 from 1834550 to 2184976 tid 2 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:938: snap_create 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:936: finishing write tid 1 to trial11545420-29 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:936: finishing write tid 2 to trial11545420-29 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:936: finishing write tid 3 to trial11545420-29 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:936: finishing write tid 4 to trial11545420-29 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:936: oid 29 updating version 0 to 311 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:936: oid 29 updating version 311 to 312 2026-02-17T05:51:12.497 INFO:tasks.rados.rados.0.trial115.stdout:936: oid 29 updating version 312 to 313 2026-02-17T05:51:12.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 313 (ObjNum 367 snap 97 seq_num 367) dirty exists 2026-02-17T05:51:12.498 INFO:tasks.rados.rados.0.trial115.stdout:936: left oid 29 (ObjNum 367 snap 97 seq_num 367) 2026-02-17T05:51:12.498 INFO:tasks.rados.rados.0.trial115.stdout:937: finishing write tid 1 to trial11545420-3 2026-02-17T05:51:12.499 INFO:tasks.rados.rados.0.trial115.stdout:937: finishing write tid 2 to trial11545420-3 2026-02-17T05:51:12.499 INFO:tasks.rados.rados.0.trial115.stdout:937: finishing write tid 3 to trial11545420-3 2026-02-17T05:51:12.499 INFO:tasks.rados.rados.0.trial115.stdout:937: finishing write tid 4 to trial11545420-3 2026-02-17T05:51:12.499 INFO:tasks.rados.rados.0.trial115.stdout:937: oid 3 updating version 0 to 344 2026-02-17T05:51:12.499 INFO:tasks.rados.rados.0.trial115.stdout:937: oid 3 updating version 344 to 345 2026-02-17T05:51:12.499 INFO:tasks.rados.rados.0.trial115.stdout:937: oid 3 version 345 is already newer than 343 2026-02-17T05:51:12.499 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 345 (ObjNum 368 snap 97 seq_num 368) dirty exists 2026-02-17T05:51:12.499 INFO:tasks.rados.rados.0.trial115.stdout:937: left oid 3 (ObjNum 368 snap 97 seq_num 368) 2026-02-17T05:51:13.386 INFO:tasks.rados.rados.0.trial115.stdout:936: done (2 left) 2026-02-17T05:51:13.386 INFO:tasks.rados.rados.0.trial115.stdout:937: done (1 left) 2026-02-17T05:51:13.386 INFO:tasks.rados.rados.0.trial115.stdout:938: done (0 left) 2026-02-17T05:51:13.386 INFO:tasks.rados.rados.0.trial115.stdout:939: read oid 23 snap -1 2026-02-17T05:51:13.386 INFO:tasks.rados.rados.0.trial115.stdout:939: expect (ObjNum 323 snap 78 seq_num 323) 2026-02-17T05:51:13.386 INFO:tasks.rados.rados.0.trial115.stdout:940: write oid 16 current snap is 98 2026-02-17T05:51:13.386 INFO:tasks.rados.rados.0.trial115.stdout:940: seq_num 369 ranges {601783=551825,1776679=425104} 2026-02-17T05:51:13.390 INFO:tasks.rados.rados.0.trial115.stdout:940: writing trial11545420-16 from 601783 to 1153608 tid 1 2026-02-17T05:51:13.394 INFO:tasks.rados.rados.0.trial115.stdout:940: writing trial11545420-16 from 1776679 to 2201783 tid 2 2026-02-17T05:51:13.394 INFO:tasks.rados.rados.0.trial115.stdout:941: read oid 7 snap 80 2026-02-17T05:51:13.394 INFO:tasks.rados.rados.0.trial115.stdout:941: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:51:13.395 INFO:tasks.rados.rados.0.trial115.stdout:942: read oid 33 snap -1 2026-02-17T05:51:13.395 INFO:tasks.rados.rados.0.trial115.stdout:942: expect deleted 2026-02-17T05:51:13.395 INFO:tasks.rados.rados.0.trial115.stdout:943: snap_create 2026-02-17T05:51:13.395 INFO:tasks.rados.rados.0.trial115.stdout:939: expect (ObjNum 323 snap 78 seq_num 323) 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:940: finishing write tid 1 to trial11545420-16 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:940: finishing write tid 2 to trial11545420-16 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:940: finishing write tid 3 to trial11545420-16 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:940: finishing write tid 4 to trial11545420-16 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:940: oid 16 updating version 0 to 406 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:940: oid 16 updating version 406 to 407 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:940: oid 16 updating version 407 to 408 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 408 (ObjNum 369 snap 98 seq_num 369) dirty exists 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:940: left oid 16 (ObjNum 369 snap 98 seq_num 369) 2026-02-17T05:51:13.478 INFO:tasks.rados.rados.0.trial115.stdout:941: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:51:14.389 INFO:tasks.rados.rados.0.trial115.stdout:939: done (4 left) 2026-02-17T05:51:14.389 INFO:tasks.rados.rados.0.trial115.stdout:940: done (3 left) 2026-02-17T05:51:14.389 INFO:tasks.rados.rados.0.trial115.stdout:941: done (2 left) 2026-02-17T05:51:14.389 INFO:tasks.rados.rados.0.trial115.stdout:942: done (1 left) 2026-02-17T05:51:14.389 INFO:tasks.rados.rados.0.trial115.stdout:943: done (0 left) 2026-02-17T05:51:14.389 INFO:tasks.rados.rados.0.trial115.stdout:944: read oid 39 snap 87 2026-02-17T05:51:14.390 INFO:tasks.rados.rados.0.trial115.stdout:944: expect (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:51:14.390 INFO:tasks.rados.rados.0.trial115.stdout:945: write (excl) oid 21 current snap is 99 2026-02-17T05:51:14.390 INFO:tasks.rados.rados.0.trial115.stdout:945: seq_num 370 ranges {618590=627074,1835161=383429} 2026-02-17T05:51:14.393 INFO:tasks.rados.rados.0.trial115.stdout:945: writing trial11545420-21 from 618590 to 1245664 tid 1 2026-02-17T05:51:14.397 INFO:tasks.rados.rados.0.trial115.stdout:945: writing trial11545420-21 from 1835161 to 2218590 tid 2 2026-02-17T05:51:14.397 INFO:tasks.rados.rados.0.trial115.stdout:946: snap_remove snap 89 2026-02-17T05:51:15.355 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:51:15.355 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T05:51:15.355 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:51:15.355 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:51:15.358 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:51:15.358 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:51:15.358 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:15.391 INFO:tasks.rados.rados.0.trial115.stdout:946: done (2 left) 2026-02-17T05:51:15.391 INFO:tasks.rados.rados.0.trial115.stdout:947: read oid 26 snap 97 2026-02-17T05:51:15.392 INFO:tasks.rados.rados.0.trial115.stdout:947: expect (ObjNum 363 snap 97 seq_num 363) 2026-02-17T05:51:15.392 INFO:tasks.rados.rados.0.trial115.stdout:948: delete oid 41 current snap is 99 2026-02-17T05:51:15.392 INFO:tasks.rados.rados.0.trial115.stdout:944: expect (ObjNum 216 snap 45 seq_num 216) 2026-02-17T05:51:15.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.427+0000 7fc776f90640 10 monclient: get_monmap_and_config 2026-02-17T05:51:15.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.427+0000 7fc776f90640 10 monclient: build_initial_monmap 2026-02-17T05:51:15.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.427+0000 7fc776f90640 10 monclient: monmap: 2026-02-17T05:51:15.427 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:15.427 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:15.429016+0000 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:15.429016+0000 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:15.428 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:15.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 1 Processor -- start 2026-02-17T05:51:15.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 1 -- start start 2026-02-17T05:51:15.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient: init 2026-02-17T05:51:15.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:15.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient: _add_conns ranks=[0,3,1,4,2] 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc77006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc77006d380 0x7fc7700a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient(hunting): picked mon.noname-a con 0x7fc77006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc77013a240 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc770074150 con 0x7fc77006c890 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77009ef70 con 0x7fc77013a240 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc77009f250 con 0x7fc77006d380 2026-02-17T05:51:15.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc776f90640 10 monclient(hunting): authenticate will time out at 920.982859s 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc774d05640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc77006c890 0x7fc77006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:59214/0 (socket says 10.20.193.78:59214) 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc774d05640 1 -- 10.20.193.78:0/1003613799 learned_addr learned my addr 10.20.193.78:0/1003613799 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 1 --2- 10.20.193.78:0/1003613799 >> v2:10.20.193.78:3300/0 conn(0x7fc77006d380 0x7fc7700a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 10 monclient(hunting): get_auth_request con 0x7fc77006d380 auth_method 0 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 1 -- 10.20.193.78:0/1003613799 >> v2:10.20.193.78:3300/0 conn(0x7fc77006d380 msgr2=0x7fc7700a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 1 -- 10.20.193.78:0/1003613799 >> v2:10.20.193.78:3300/0 conn(0x7fc77006d380 msgr2=0x7fc7700a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:15.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 1 --2- 10.20.193.78:0/1003613799 >> v2:10.20.193.78:3300/0 conn(0x7fc77006d380 0x7fc7700a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.428+0000 7fc767fff640 1 --2- 10.20.193.78:0/1003613799 >> v2:10.20.193.78:3300/0 conn(0x7fc77006d380 0x7fc7700a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.430+0000 7fc774d05640 1 -- 10.20.193.78:0/1003613799 >> v1:10.20.193.78:6789/0 conn(0x7fc77006c890 legacy=0x7fc77006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc775506640 1 -- 10.20.193.78:0/1003613799 >> v1:10.20.193.115:6789/0 conn(0x7fc77013a240 legacy=0x7fc77013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1424929739 0 0) 0x7fc77009ef70 con 0x7fc77013a240 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): _negotiate 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): my global_id is 44383 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc75c003620 con 0x7fc77013a240 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1647607006 0 0) 0x7fc75c003620 con 0x7fc77013a240 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 >> v2:10.20.193.78:3300/0 conn(0x7fc77006d380 msgr2=0x7fc7700a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:15.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 --2- 10.20.193.78:0/1003613799 >> v2:10.20.193.78:3300/0 conn(0x7fc77006d380 0x7fc7700a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 >> v1:10.20.193.78:6789/0 conn(0x7fc77006c890 legacy=0x7fc77006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: _finish_hunting 0 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77009df40 con 0x7fc77013a240 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: _finish_auth 0 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fc76c003640 con 0x7fc77013a240 2026-02-17T05:51:15.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: dump: 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:15.446 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 >> v1:10.20.193.115:6789/0 conn(0x7fc77013a240 legacy=0x7fc77013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc75c003b50 0x7fc75c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): picked mon.a con 0x7fc75c003b50 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc75c004900 0x7fc75c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): picked mon.c con 0x7fc75c004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c0054c0 0x7fc75c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): picked mon.b con 0x7fc75c0054c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:15.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc77009f250 con 0x7fc75c003b50 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c00c350 con 0x7fc75c0054c0 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c00c550 con 0x7fc75c004900 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc7677fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc75c003b50 0x7fc75c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient(hunting): get_auth_request con 0x7fc75c003b50 auth_method 0 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:15.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc767fff640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc75c004900 0x7fc75c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc767fff640 10 monclient(hunting): get_auth_request con 0x7fc75c004900 auth_method 0 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc767fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc767fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc767fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc767fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc767fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc767fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:51:15.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc775506640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c0054c0 0x7fc75c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc775506640 10 monclient(hunting): get_auth_request con 0x7fc75c0054c0 auth_method 0 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc775506640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc775506640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc775506640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient(hunting): handle_auth_done global_id 44383 payload 293 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 1 -- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc75c004900 msgr2=0x7fc75c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc75c004900 0x7fc75c004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 1 -- 10.20.193.78:0/1003613799 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c0054c0 msgr2=0x7fc75c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c0054c0 0x7fc75c00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient: _finish_hunting 0 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 1 monclient: found mon.a 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc775506640 1 -- 10.20.193.78:0/1003613799 reap_dead start 2026-02-17T05:51:15.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 1 -- 10.20.193.78:0/1003613799 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c003620 con 0x7fc75c003b50 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient: _finish_auth 0 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 10 monclient: _check_auth_tickets 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc774d05640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc776f90640 5 monclient: authenticate success, global_id 44383 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.431+0000 7fc776f90640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc774d05640 1 -- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc75c003b50 msgr2=0x7fc75c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc774d05640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc75c003b50 0x7fc75c003f70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc760003360 tx=0x7fc76002f0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5797be4df9bee9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760003060 con 0x7fc75c003b50 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:15.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 10 monclient: dump: 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:15.452 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc760033030 con 0x7fc75c003b50 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 1 -- 10.20.193.78:0/1003613799 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76002f8f0 con 0x7fc75c003b50 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 4 set_mon_vals no callback set 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc7677fe640 10 monclient: dump: 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.453 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 10 monclient: shutdown 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 1 -- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc75c003b50 msgr2=0x7fc75c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc75c003b50 0x7fc75c003f70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc760003360 tx=0x7fc76002f0d0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 1 -- 10.20.193.78:0/1003613799 shutdown_connections 2026-02-17T05:51:15.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 1 --2- 10.20.193.78:0/1003613799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc75c003b50 0x7fc75c003f70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 1 -- 10.20.193.78:0/1003613799 >> 10.20.193.78:0/1003613799 conn(0x7fc7700a6620 msgr2=0x7fc7700a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:15.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 1 -- 10.20.193.78:0/1003613799 shutdown_connections 2026-02-17T05:51:15.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.434+0000 7fc776f90640 1 -- 10.20.193.78:0/1003613799 wait complete. 2026-02-17T05:51:15.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient: build_initial_monmap 2026-02-17T05:51:15.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient: monmap: 2026-02-17T05:51:15.455 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:15.456 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:15.456 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:15.456 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:15.456 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:15.456 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:15.456 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:15.456 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:15.457 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:15.457 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:15.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 1 Processor -- start 2026-02-17T05:51:15.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 1 -- start start 2026-02-17T05:51:15.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient: init 2026-02-17T05:51:15.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:15.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:51:15.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c003b50 0x7fc7701cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:15.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient(hunting): picked mon.noname-b con 0x7fc75c003b50 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:15.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc7701cfbe0 0x7fc770121630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:15.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient(hunting): picked mon.noname-c con 0x7fc7701cfbe0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:15.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 0x7fc770121b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:15.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient(hunting): picked mon.noname-a con 0x7fc7701d0020 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:15.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:15.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7700a0370 con 0x7fc7701d0020 2026-02-17T05:51:15.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:15.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7700a01f0 con 0x7fc75c003b50 2026-02-17T05:51:15.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:15.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc7700a04f0 con 0x7fc7701cfbe0 2026-02-17T05:51:15.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:15.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc776f90640 10 monclient(hunting): authenticate will time out at 920.989759s 2026-02-17T05:51:15.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc7701cfbe0 0x7fc770121630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:15.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 0x7fc770121b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:15.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc7701cfbe0 0x7fc770121630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49032/0 (socket says 10.20.193.78:49032) 2026-02-17T05:51:15.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 -- 10.20.193.78:0/2522755785 learned_addr learned my addr 10.20.193.78:0/2522755785 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:15.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient(hunting): get_auth_request con 0x7fc7701cfbe0 auth_method 0 2026-02-17T05:51:15.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:15.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:15.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:15.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 10 monclient(hunting): get_auth_request con 0x7fc7701d0020 auth_method 0 2026-02-17T05:51:15.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:15.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:15.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:15.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:15.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:15.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 1 -- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 msgr2=0x7fc770121b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:51:15.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 1 -- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 msgr2=0x7fc770121b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:51:15.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:15.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 0x7fc770121b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:15.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc774d05640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c003b50 0x7fc7701cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:15.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc775506640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 0x7fc770121b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:15.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc774d05640 10 monclient(hunting): get_auth_request con 0x7fc75c003b50 auth_method 0 2026-02-17T05:51:15.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc774d05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:15.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc774d05640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:15.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc774d05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:15.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient(hunting): handle_auth_done global_id 74171 payload 824 2026-02-17T05:51:15.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 -- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c003b50 msgr2=0x7fc7701cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:15.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c003b50 0x7fc7701cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 -- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 msgr2=0x7fc770121b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:15.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 0x7fc770121b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient: _finish_hunting 0 2026-02-17T05:51:15.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 monclient: found mon.noname-c 2026-02-17T05:51:15.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:15.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:15.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.435+0000 7fc767fff640 1 -- 10.20.193.78:0/2522755785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc770123770 con 0x7fc7701cfbe0 2026-02-17T05:51:15.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc767fff640 1 -- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc7701cfbe0 msgr2=0x7fc770121630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:15.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc767fff640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc7701cfbe0 0x7fc770121630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc758011930 tx=0x7fc758011e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:15.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 1 -- 10.20.193.78:0/2522755785 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758019070 con 0x7fc7701cfbe0 2026-02-17T05:51:15.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:15.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:15.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: dump: 2026-02-17T05:51:15.466 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:15.466 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:15.466 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.466 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.466 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:15.467 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:15.467 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:15.467 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:15.467 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:15.467 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:15.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: _finish_auth 0 2026-02-17T05:51:15.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: _check_auth_tickets 2026-02-17T05:51:15.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 1 -- 10.20.193.78:0/2522755785 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75800fd80 con 0x7fc7701cfbe0 2026-02-17T05:51:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 1 -- 10.20.193.78:0/2522755785 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75800f040 con 0x7fc7701cfbe0 2026-02-17T05:51:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 5 monclient: authenticate success, global_id 74171 2026-02-17T05:51:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc766ffd640 4 set_mon_vals no callback set 2026-02-17T05:51:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 10 monclient: _renew_subs 2026-02-17T05:51:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:15.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7701255c0 con 0x7fc7701cfbe0 2026-02-17T05:51:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:15.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 10 monclient: dump: 2026-02-17T05:51:15.469 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:15.469 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:15.469 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.469 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 10 monclient: _renew_subs 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:15.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc770073520 con 0x7fc7701cfbe0 2026-02-17T05:51:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77006cc70 con 0x7fc7701cfbe0 2026-02-17T05:51:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.438+0000 7fc7657fa640 1 -- 10.20.193.78:0/2522755785 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc7580042a0 con 0x7fc7701cfbe0 2026-02-17T05:51:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.439+0000 7fc7657fa640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc74003deb0 0x7fc740040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.439+0000 7fc7657fa640 1 -- 10.20.193.78:0/2522755785 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(206..206 src has 117..206) ==== 4795+0+0 (secure 0 0 0) 0x7fc758052cd0 con 0x7fc7701cfbe0 2026-02-17T05:51:15.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.439+0000 7fc774d05640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc74003deb0 0x7fc740040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.439+0000 7fc774d05640 10 monclient: get_auth_request con 0x7fc74003deb0 auth_method 0 2026-02-17T05:51:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.441+0000 7fc7657fa640 1 -- 10.20.193.78:0/2522755785 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc758024d80 con 0x7fc7701cfbe0 2026-02-17T05:51:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.441+0000 7fc7657fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.441+0000 7fc7657fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:15.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.441+0000 7fc774d05640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc74003deb0 0x7fc740040370 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc760003330 tx=0x7fc76003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:15.473 INFO:tasks.rados.rados.0.trial115.stdout:945: finishing write tid 1 to trial11545420-21 2026-02-17T05:51:15.473 INFO:tasks.rados.rados.0.trial115.stdout:945: finishing write tid 2 to trial11545420-21 2026-02-17T05:51:15.473 INFO:tasks.rados.rados.0.trial115.stdout:945: finishing write tid 3 to trial11545420-21 2026-02-17T05:51:15.473 INFO:tasks.rados.rados.0.trial115.stdout:945: finishing write tid 4 to trial11545420-21 2026-02-17T05:51:15.474 INFO:tasks.rados.rados.0.trial115.stdout:945: oid 21 updating version 0 to 327 2026-02-17T05:51:15.474 INFO:tasks.rados.rados.0.trial115.stdout:945: oid 21 version 327 is already newer than 326 2026-02-17T05:51:15.474 INFO:tasks.rados.rados.0.trial115.stdout:945: oid 21 version 327 is already newer than 325 2026-02-17T05:51:15.474 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 327 (ObjNum 370 snap 99 seq_num 370) dirty exists 2026-02-17T05:51:15.474 INFO:tasks.rados.rados.0.trial115.stdout:945: left oid 21 (ObjNum 370 snap 99 seq_num 370) 2026-02-17T05:51:15.474 INFO:tasks.rados.rados.0.trial115.stdout:947: expect (ObjNum 363 snap 97 seq_num 363) 2026-02-17T05:51:15.538 INFO:tasks.rados.rados.0.trial115.stdout:944: done (3 left) 2026-02-17T05:51:15.538 INFO:tasks.rados.rados.0.trial115.stdout:945: done (2 left) 2026-02-17T05:51:15.538 INFO:tasks.rados.rados.0.trial115.stdout:947: done (1 left) 2026-02-17T05:51:15.539 INFO:tasks.rados.rados.0.trial115.stdout:948: done (0 left) 2026-02-17T05:51:15.539 INFO:tasks.rados.rados.0.trial115.stdout:949: read oid 1 snap 81 2026-02-17T05:51:15.539 INFO:tasks.rados.rados.0.trial115.stdout:949: expect (ObjNum 251 snap 49 seq_num 251) 2026-02-17T05:51:15.539 INFO:tasks.rados.rados.0.trial115.stdout:950: read oid 19 snap -1 2026-02-17T05:51:15.539 INFO:tasks.rados.rados.0.trial115.stdout:950: expect (ObjNum 306 snap 73 seq_num 306) 2026-02-17T05:51:15.539 INFO:tasks.rados.rados.0.trial115.stdout:951: write oid 26 current snap is 99 2026-02-17T05:51:15.539 INFO:tasks.rados.rados.0.trial115.stdout:951: seq_num 372 ranges {652204=777572,1952125=300079} 2026-02-17T05:51:15.543 INFO:tasks.rados.rados.0.trial115.stdout:951: writing trial11545420-26 from 652204 to 1429776 tid 1 2026-02-17T05:51:15.546 INFO:tasks.rados.rados.0.trial115.stdout:951: writing trial11545420-26 from 1952125 to 2252204 tid 2 2026-02-17T05:51:15.546 INFO:tasks.rados.rados.0.trial115.stdout:952: copy_from oid 8 from oid 18 current snap is 99 2026-02-17T05:51:15.546 INFO:tasks.rados.rados.0.trial115.stdout:949: expect (ObjNum 251 snap 49 seq_num 251) 2026-02-17T05:51:15.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.556+0000 7fc776f90640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:15.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.556+0000 7fc776f90640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:15.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.556+0000 7fc776f90640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:15.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.556+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc7700744d0 con 0x7fc7701cfbe0 2026-02-17T05:51:15.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.556+0000 7fc7657fa640 1 -- 10.20.193.78:0/2522755785 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fc758003710 con 0x7fc7701cfbe0 2026-02-17T05:51:15.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.556+0000 7fc7657fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:15.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.556+0000 7fc7657fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:15.556 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":48,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:15.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc74003deb0 msgr2=0x7fc740040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:15.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc74003deb0 0x7fc740040370 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc760003330 tx=0x7fc76003b040 comp rx=0 tx=0).stop 2026-02-17T05:51:15.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 10 monclient: shutdown 2026-02-17T05:51:15.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:15.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc7701cfbe0 msgr2=0x7fc770121630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc7701cfbe0 0x7fc770121630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc758011930 tx=0x7fc758011e00 comp rx=0 tx=0).stop 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 shutdown_connections 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7701d0020 0x7fc770121b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc7701cfbe0 0x7fc770121630 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc74003deb0 0x7fc740040370 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 --2- 10.20.193.78:0/2522755785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc75c003b50 0x7fc7701cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 >> 10.20.193.78:0/2522755785 conn(0x7fc7700a6620 msgr2=0x7fc770139840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 shutdown_connections 2026-02-17T05:51:15.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:15.557+0000 7fc776f90640 1 -- 10.20.193.78:0/2522755785 wait complete. 2026-02-17T05:51:15.641 INFO:tasks.rados.rados.0.trial115.stdout:950: expect (ObjNum 306 snap 73 seq_num 306) 2026-02-17T05:51:15.719 INFO:tasks.rados.rados.0.trial115.stdout:951: finishing write tid 1 to trial11545420-26 2026-02-17T05:51:15.719 INFO:tasks.rados.rados.0.trial115.stdout:951: finishing write tid 2 to trial11545420-26 2026-02-17T05:51:15.719 INFO:tasks.rados.rados.0.trial115.stdout:951: finishing write tid 3 to trial11545420-26 2026-02-17T05:51:15.719 INFO:tasks.rados.rados.0.trial115.stdout:951: finishing write tid 4 to trial11545420-26 2026-02-17T05:51:15.719 INFO:tasks.rados.rados.0.trial115.stdout:951: oid 26 updating version 0 to 188 2026-02-17T05:51:15.719 INFO:tasks.rados.rados.0.trial115.stdout:951: oid 26 version 188 is already newer than 187 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:951: oid 26 updating version 188 to 189 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 189 (ObjNum 372 snap 99 seq_num 372) dirty exists 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:951: left oid 26 (ObjNum 372 snap 99 seq_num 372) 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:952: finishing copy_from to trial11545420-8 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:952: got expected ENOENT (src dne) 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:952: finishing copy_from racing read to trial11545420-8 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:949: done (3 left) 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:950: done (2 left) 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:951: done (1 left) 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:952: done (0 left) 2026-02-17T05:51:15.720 INFO:tasks.rados.rados.0.trial115.stdout:953: rollback oid 9 current snap is 99 2026-02-17T05:51:15.721 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 9 to 87 2026-02-17T05:51:15.721 INFO:tasks.rados.rados.0.trial115.stdout:954: delete oid 13 current snap is 99 2026-02-17T05:51:15.722 INFO:tasks.rados.rados.0.trial115.stdout:953: finishing rollback tid 0 to trial11545420-9 2026-02-17T05:51:15.722 INFO:tasks.rados.rados.0.trial115.stdout:953: finishing rollback tid 1 to trial11545420-9 2026-02-17T05:51:15.722 INFO:tasks.rados.rados.0.trial115.stdout:953: finishing rollback tid 2 to trial11545420-9 2026-02-17T05:51:15.722 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 384 (ObjNum 106 snap 16 seq_num 106) dirty exists 2026-02-17T05:51:15.723 INFO:tasks.rados.rados.0.trial115.stdout:953: done (1 left) 2026-02-17T05:51:15.723 INFO:tasks.rados.rados.0.trial115.stdout:954: done (0 left) 2026-02-17T05:51:15.723 INFO:tasks.rados.rados.0.trial115.stdout:955: rollback oid 21 current snap is 99 2026-02-17T05:51:15.723 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 21 to 59 2026-02-17T05:51:15.723 INFO:tasks.rados.rados.0.trial115.stdout:956: write oid 20 current snap is 99 2026-02-17T05:51:15.723 INFO:tasks.rados.rados.0.trial115.stdout:956: seq_num 374 ranges {685818=644423,2069089=216729} 2026-02-17T05:51:15.725 INFO:tasks.rados.rados.0.trial115.stdout:956: writing trial11545420-20 from 685818 to 1330241 tid 1 2026-02-17T05:51:15.729 INFO:tasks.rados.rados.0.trial115.stdout:956: writing trial11545420-20 from 2069089 to 2285818 tid 2 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:957: read oid 35 snap -1 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:957: expect deleted 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:958: copy_from oid 36 from oid 31 current snap is 99 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:955: finishing rollback tid 0 to trial11545420-21 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:955: finishing rollback tid 1 to trial11545420-21 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:955: finishing rollback tid 2 to trial11545420-21 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 336 (ObjNum 275 snap 57 seq_num 275) dirty exists 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:955: done (3 left) 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:959: copy_from oid 13 from oid 43 current snap is 99 2026-02-17T05:51:15.730 INFO:tasks.rados.rados.0.trial115.stdout:960: write oid 50 current snap is 99 2026-02-17T05:51:15.731 INFO:tasks.rados.rados.0.trial115.stdout:960: seq_num 375 ranges {702625=719672,2127571=175054} 2026-02-17T05:51:15.734 INFO:tasks.rados.rados.0.trial115.stdout:960: writing trial11545420-50 from 702625 to 1422297 tid 1 2026-02-17T05:51:15.738 INFO:tasks.rados.rados.0.trial115.stdout:960: writing trial11545420-50 from 2127571 to 2302625 tid 2 2026-02-17T05:51:15.738 INFO:tasks.rados.rados.0.trial115.stdout:961: copy_from oid 38 from oid 29 current snap is 99 2026-02-17T05:51:15.738 INFO:tasks.rados.rados.0.trial115.stdout:956: finishing write tid 1 to trial11545420-20 2026-02-17T05:51:15.738 INFO:tasks.rados.rados.0.trial115.stdout:956: finishing write tid 2 to trial11545420-20 2026-02-17T05:51:15.738 INFO:tasks.rados.rados.0.trial115.stdout:956: finishing write tid 3 to trial11545420-20 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:956: finishing write tid 4 to trial11545420-20 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:956: oid 20 updating version 0 to 400 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:956: oid 20 updating version 400 to 401 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:956: oid 20 updating version 401 to 402 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 402 (ObjNum 374 snap 99 seq_num 374) dirty exists 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:956: left oid 20 (ObjNum 374 snap 99 seq_num 374) 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:958: finishing copy_from racing read to trial11545420-36 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:958: finishing copy_from to trial11545420-36 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 272 (ObjNum 273 snap 56 seq_num 273) dirty exists 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:960: finishing write tid 1 to trial11545420-50 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:956: done (5 left) 2026-02-17T05:51:15.739 INFO:tasks.rados.rados.0.trial115.stdout:957: done (4 left) 2026-02-17T05:51:15.740 INFO:tasks.rados.rados.0.trial115.stdout:958: done (3 left) 2026-02-17T05:51:15.740 INFO:tasks.rados.rados.0.trial115.stdout:962: write (excl) oid 9 current snap is 99 2026-02-17T05:51:15.740 INFO:tasks.rados.rados.0.trial115.stdout:962: seq_num 376 ranges {719432=794921,2186053=133379} 2026-02-17T05:51:15.743 INFO:tasks.rados.rados.0.trial115.stdout:962: writing trial11545420-9 from 719432 to 1514353 tid 1 2026-02-17T05:51:15.746 INFO:tasks.rados.rados.0.trial115.stdout:962: writing trial11545420-9 from 2186053 to 2319432 tid 2 2026-02-17T05:51:15.747 INFO:tasks.rados.rados.0.trial115.stdout:963: write oid 44 current snap is 99 2026-02-17T05:51:15.747 INFO:tasks.rados.rados.0.trial115.stdout:959: finishing copy_from racing read to trial11545420-13 2026-02-17T05:51:15.747 INFO:tasks.rados.rados.0.trial115.stdout:959: finishing copy_from to trial11545420-13 2026-02-17T05:51:15.747 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 394 (ObjNum 302 snap 72 seq_num 302) dirty exists 2026-02-17T05:51:15.747 INFO:tasks.rados.rados.0.trial115.stdout:960: finishing write tid 2 to trial11545420-50 2026-02-17T05:51:15.747 INFO:tasks.rados.rados.0.trial115.stdout:963: seq_num 377 ranges {736239=470170,1728182=571782} 2026-02-17T05:51:15.751 INFO:tasks.rados.rados.0.trial115.stdout:963: writing trial11545420-44 from 736239 to 1206409 tid 1 2026-02-17T05:51:15.755 INFO:tasks.rados.rados.0.trial115.stdout:963: writing trial11545420-44 from 1728182 to 2299964 tid 2 2026-02-17T05:51:15.755 INFO:tasks.rados.rados.0.trial115.stdout:959: done (4 left) 2026-02-17T05:51:15.755 INFO:tasks.rados.rados.0.trial115.stdout:964: write oid 1 current snap is 99 2026-02-17T05:51:15.755 INFO:tasks.rados.rados.0.trial115.stdout:964: seq_num 378 ranges {753046=545419,1786664=431793} 2026-02-17T05:51:15.760 INFO:tasks.rados.rados.0.trial115.stdout:964: writing trial11545420-1 from 753046 to 1298465 tid 1 2026-02-17T05:51:15.764 INFO:tasks.rados.rados.0.trial115.stdout:964: writing trial11545420-1 from 1786664 to 2218457 tid 2 2026-02-17T05:51:15.764 INFO:tasks.rados.rados.0.trial115.stdout:965: snap_create 2026-02-17T05:51:15.764 INFO:tasks.rados.rados.0.trial115.stdout:960: finishing write tid 3 to trial11545420-50 2026-02-17T05:51:15.764 INFO:tasks.rados.rados.0.trial115.stdout:960: finishing write tid 4 to trial11545420-50 2026-02-17T05:51:15.764 INFO:tasks.rados.rados.0.trial115.stdout:960: oid 50 updating version 0 to 314 2026-02-17T05:51:15.764 INFO:tasks.rados.rados.0.trial115.stdout:960: oid 50 updating version 314 to 316 2026-02-17T05:51:15.764 INFO:tasks.rados.rados.0.trial115.stdout:960: oid 50 version 316 is already newer than 315 2026-02-17T05:51:15.764 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 316 (ObjNum 375 snap 99 seq_num 375) dirty exists 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:960: left oid 50 (ObjNum 375 snap 99 seq_num 375) 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:962: finishing write tid 1 to trial11545420-9 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:962: finishing write tid 2 to trial11545420-9 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:962: finishing write tid 3 to trial11545420-9 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:962: finishing write tid 4 to trial11545420-9 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:962: oid 9 updating version 0 to 387 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:962: oid 9 version 387 is already newer than 386 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:962: oid 9 version 387 is already newer than 385 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 387 (ObjNum 376 snap 99 seq_num 376) dirty exists 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:962: left oid 9 (ObjNum 376 snap 99 seq_num 376) 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:961: finishing copy_from racing read to trial11545420-38 2026-02-17T05:51:15.765 INFO:tasks.rados.rados.0.trial115.stdout:963: finishing write tid 1 to trial11545420-44 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:961: finishing copy_from to trial11545420-38 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 313 (ObjNum 367 snap 97 seq_num 367) dirty exists 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:963: finishing write tid 2 to trial11545420-44 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:963: finishing write tid 3 to trial11545420-44 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:963: finishing write tid 4 to trial11545420-44 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:963: oid 44 updating version 0 to 313 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:963: oid 44 version 313 is already newer than 312 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:963: oid 44 version 313 is already newer than 311 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 313 (ObjNum 377 snap 99 seq_num 377) dirty exists 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:963: left oid 44 (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:51:15.766 INFO:tasks.rados.rados.0.trial115.stdout:964: finishing write tid 1 to trial11545420-1 2026-02-17T05:51:15.767 INFO:tasks.rados.rados.0.trial115.stdout:964: finishing write tid 2 to trial11545420-1 2026-02-17T05:51:15.767 INFO:tasks.rados.rados.0.trial115.stdout:964: finishing write tid 3 to trial11545420-1 2026-02-17T05:51:15.768 INFO:tasks.rados.rados.0.trial115.stdout:964: finishing write tid 4 to trial11545420-1 2026-02-17T05:51:15.768 INFO:tasks.rados.rados.0.trial115.stdout:964: oid 1 updating version 0 to 317 2026-02-17T05:51:15.768 INFO:tasks.rados.rados.0.trial115.stdout:964: oid 1 version 317 is already newer than 316 2026-02-17T05:51:15.768 INFO:tasks.rados.rados.0.trial115.stdout:964: oid 1 updating version 317 to 318 2026-02-17T05:51:15.768 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 318 (ObjNum 378 snap 99 seq_num 378) dirty exists 2026-02-17T05:51:15.768 INFO:tasks.rados.rados.0.trial115.stdout:964: left oid 1 (ObjNum 378 snap 99 seq_num 378) 2026-02-17T05:51:16.371 INFO:tasks.rados.rados.0.trial115.stdout:960: done (5 left) 2026-02-17T05:51:16.371 INFO:tasks.rados.rados.0.trial115.stdout:961: done (4 left) 2026-02-17T05:51:16.371 INFO:tasks.rados.rados.0.trial115.stdout:962: done (3 left) 2026-02-17T05:51:16.371 INFO:tasks.rados.rados.0.trial115.stdout:963: done (2 left) 2026-02-17T05:51:16.371 INFO:tasks.rados.rados.0.trial115.stdout:964: done (1 left) 2026-02-17T05:51:16.371 INFO:tasks.rados.rados.0.trial115.stdout:965: done (0 left) 2026-02-17T05:51:16.371 INFO:tasks.rados.rados.0.trial115.stdout:966: write oid 35 current snap is 100 2026-02-17T05:51:16.371 INFO:tasks.rados.rados.0.trial115.stdout:966: seq_num 379 ranges {769853=620668,1845146=524707} 2026-02-17T05:51:16.376 INFO:tasks.rados.rados.0.trial115.stdout:966: writing trial11545420-35 from 769853 to 1390521 tid 1 2026-02-17T05:51:16.379 INFO:tasks.rados.rados.0.trial115.stdout:966: writing trial11545420-35 from 1845146 to 2369853 tid 2 2026-02-17T05:51:16.379 INFO:tasks.rados.rados.0.trial115.stdout:967: delete oid 10 current snap is 100 2026-02-17T05:51:16.379 INFO:tasks.rados.rados.0.trial115.stdout:966: finishing write tid 1 to trial11545420-35 2026-02-17T05:51:16.381 INFO:tasks.rados.rados.0.trial115.stdout:966: finishing write tid 2 to trial11545420-35 2026-02-17T05:51:16.382 INFO:tasks.rados.rados.0.trial115.stdout:966: finishing write tid 3 to trial11545420-35 2026-02-17T05:51:16.382 INFO:tasks.rados.rados.0.trial115.stdout:966: finishing write tid 4 to trial11545420-35 2026-02-17T05:51:16.382 INFO:tasks.rados.rados.0.trial115.stdout:966: oid 35 updating version 0 to 415 2026-02-17T05:51:16.382 INFO:tasks.rados.rados.0.trial115.stdout:966: oid 35 version 415 is already newer than 414 2026-02-17T05:51:16.382 INFO:tasks.rados.rados.0.trial115.stdout:966: oid 35 version 415 is already newer than 413 2026-02-17T05:51:16.382 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 415 (ObjNum 379 snap 100 seq_num 379) dirty exists 2026-02-17T05:51:16.382 INFO:tasks.rados.rados.0.trial115.stdout:966: left oid 35 (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:51:16.382 INFO:tasks.rados.rados.0.trial115.stdout:966: done (1 left) 2026-02-17T05:51:16.383 INFO:tasks.rados.rados.0.trial115.stdout:967: done (0 left) 2026-02-17T05:51:16.383 INFO:tasks.rados.rados.0.trial115.stdout:968: copy_from oid 8 from oid 23 current snap is 100 2026-02-17T05:51:16.383 INFO:tasks.rados.rados.0.trial115.stdout:969: write (excl) oid 15 current snap is 100 2026-02-17T05:51:16.383 INFO:tasks.rados.rados.0.trial115.stdout:969: seq_num 381 ranges {403467=487519,1562110=579120} 2026-02-17T05:51:16.385 INFO:tasks.rados.rados.0.trial115.stdout:969: writing trial11545420-15 from 403467 to 890986 tid 1 2026-02-17T05:51:16.390 INFO:tasks.rados.rados.0.trial115.stdout:969: writing trial11545420-15 from 1562110 to 2141230 tid 2 2026-02-17T05:51:16.390 INFO:tasks.rados.rados.0.trial115.stdout:970: snap_create 2026-02-17T05:51:16.390 INFO:tasks.rados.rados.0.trial115.stdout:969: finishing write tid 1 to trial11545420-15 2026-02-17T05:51:16.390 INFO:tasks.rados.rados.0.trial115.stdout:968: finishing copy_from racing read to trial11545420-8 2026-02-17T05:51:16.390 INFO:tasks.rados.rados.0.trial115.stdout:968: finishing copy_from to trial11545420-8 2026-02-17T05:51:16.390 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 133 (ObjNum 323 snap 78 seq_num 323) dirty exists 2026-02-17T05:51:16.391 INFO:tasks.rados.rados.0.trial115.stdout:969: finishing write tid 2 to trial11545420-15 2026-02-17T05:51:16.391 INFO:tasks.rados.rados.0.trial115.stdout:969: finishing write tid 3 to trial11545420-15 2026-02-17T05:51:16.391 INFO:tasks.rados.rados.0.trial115.stdout:969: finishing write tid 4 to trial11545420-15 2026-02-17T05:51:16.392 INFO:tasks.rados.rados.0.trial115.stdout:969: oid 15 updating version 0 to 392 2026-02-17T05:51:16.392 INFO:tasks.rados.rados.0.trial115.stdout:969: oid 15 version 392 is already newer than 391 2026-02-17T05:51:16.392 INFO:tasks.rados.rados.0.trial115.stdout:969: oid 15 version 392 is already newer than 390 2026-02-17T05:51:16.392 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 392 (ObjNum 381 snap 100 seq_num 381) dirty exists 2026-02-17T05:51:16.392 INFO:tasks.rados.rados.0.trial115.stdout:969: left oid 15 (ObjNum 381 snap 100 seq_num 381) 2026-02-17T05:51:17.374 INFO:tasks.rados.rados.0.trial115.stdout:968: done (2 left) 2026-02-17T05:51:17.374 INFO:tasks.rados.rados.0.trial115.stdout:969: done (1 left) 2026-02-17T05:51:17.374 INFO:tasks.rados.rados.0.trial115.stdout:970: done (0 left) 2026-02-17T05:51:17.374 INFO:tasks.rados.rados.0.trial115.stdout:971: snap_remove snap 83 2026-02-17T05:51:18.377 INFO:tasks.rados.rados.0.trial115.stdout:971: done (0 left) 2026-02-17T05:51:18.377 INFO:tasks.rados.rados.0.trial115.stdout:972: rollback oid 40 current snap is 101 2026-02-17T05:51:18.378 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 99 2026-02-17T05:51:18.378 INFO:tasks.rados.rados.0.trial115.stdout:973: read oid 1 snap -1 2026-02-17T05:51:18.378 INFO:tasks.rados.rados.0.trial115.stdout:973: expect (ObjNum 378 snap 99 seq_num 378) 2026-02-17T05:51:18.378 INFO:tasks.rados.rados.0.trial115.stdout:974: copy_from oid 13 from oid 12 current snap is 101 2026-02-17T05:51:18.378 INFO:tasks.rados.rados.0.trial115.stdout:975: snap_remove snap 85 2026-02-17T05:51:18.565 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:18.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.636+0000 7fe36d41a640 10 monclient: get_monmap_and_config 2026-02-17T05:51:18.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.636+0000 7fe36d41a640 10 monclient: build_initial_monmap 2026-02-17T05:51:18.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.636+0000 7fe36d41a640 10 monclient: monmap: 2026-02-17T05:51:18.638 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:18.638103+0000 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:18.638103+0000 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 1 Processor -- start 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 1 -- start start 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient: init 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fe36806c890 0x7fe36806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient(hunting): picked mon.noname-c con 0x7fe36806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:18.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe36806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient(hunting): picked mon.noname-b con 0x7fe36809d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 0x7fe368134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient(hunting): picked mon.noname-a con 0x7fe36809bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fe36809ca50 con 0x7fe36809bf40 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fe36806d6f0 con 0x7fe36806c890 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe368073600 con 0x7fe36809d430 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe36d41a640 10 monclient(hunting): authenticate will time out at 924.191636s 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe36806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe36806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42208/0 (socket says 10.20.193.78:42208) 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 -- 10.20.193.78:0/2606466844 learned_addr learned my addr 10.20.193.78:0/2606466844 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 0x7fe368134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient(hunting): get_auth_request con 0x7fe36809d430 auth_method 0 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:18.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 10 monclient(hunting): get_auth_request con 0x7fe36809bf40 auth_method 0 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe366ffd640 1 --2- 10.20.193.78:0/2606466844 >> v2:10.20.193.115:3300/0 conn(0x7fe36806c890 0x7fe36806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe366ffd640 10 monclient(hunting): get_auth_request con 0x7fe36806c890 auth_method 0 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe366ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe366ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe366ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 1 -- 10.20.193.78:0/2606466844 >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 msgr2=0x7fe368134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 1 -- 10.20.193.78:0/2606466844 >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 msgr2=0x7fe368134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 1 --2- 10.20.193.78:0/2606466844 >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 0x7fe368134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3677fe640 1 --2- 10.20.193.78:0/2606466844 >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 0x7fe368134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient(hunting): handle_auth_done global_id 74174 payload 274 2026-02-17T05:51:18.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 -- 10.20.193.78:0/2606466844 >> v2:10.20.193.115:3300/0 conn(0x7fe36806c890 msgr2=0x7fe36806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:18.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 --2- 10.20.193.78:0/2606466844 >> v2:10.20.193.115:3300/0 conn(0x7fe36806c890 0x7fe36806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 -- 10.20.193.78:0/2606466844 >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 msgr2=0x7fe368134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:18.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 --2- 10.20.193.78:0/2606466844 >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 0x7fe368134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient: _finish_hunting 0 2026-02-17T05:51:18.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 monclient: found mon.noname-b 2026-02-17T05:51:18.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:18.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:18.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.637+0000 7fe3667fc640 1 -- 10.20.193.78:0/2606466844 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36806dab0 con 0x7fe36809d430 2026-02-17T05:51:18.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe3667fc640 1 -- 10.20.193.78:0/2606466844 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 msgr2=0x7fe36806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:18.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe3667fc640 1 --2- 10.20.193.78:0/2606466844 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe36806d1b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe36806a480 tx=0x7fe35002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f870a28c73c0b316 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:18.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 1 -- 10.20.193.78:0/2606466844 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35003c070 con 0x7fe36809d430 2026-02-17T05:51:18.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:18.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:18.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: dump: 2026-02-17T05:51:18.646 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: _finish_auth 0 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 1 -- 10.20.193.78:0/2606466844 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35002fca0 con 0x7fe36809d430 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 1 -- 10.20.193.78:0/2606466844 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350037660 con 0x7fe36809d430 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:18.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe365ffb640 10 monclient: dump: 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 5 monclient: authenticate success, global_id 74174 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 4 set_mon_vals no callback set 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 10 monclient: shutdown 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 -- 10.20.193.78:0/2606466844 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 msgr2=0x7fe36806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 --2- 10.20.193.78:0/2606466844 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe36806d1b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe36806a480 tx=0x7fe35002ee60 comp rx=0 tx=0).stop 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 -- 10.20.193.78:0/2606466844 shutdown_connections 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 --2- 10.20.193.78:0/2606466844 >> v2:10.20.193.78:3300/0 conn(0x7fe36809bf40 0x7fe368134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 --2- 10.20.193.78:0/2606466844 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe36806d1b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 --2- 10.20.193.78:0/2606466844 >> v2:10.20.193.115:3300/0 conn(0x7fe36806c890 0x7fe36806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 -- 10.20.193.78:0/2606466844 >> 10.20.193.78:0/2606466844 conn(0x7fe3680a6620 msgr2=0x7fe3680a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 -- 10.20.193.78:0/2606466844 shutdown_connections 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.640+0000 7fe36d41a640 1 -- 10.20.193.78:0/2606466844 wait complete. 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient: build_initial_monmap 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient: monmap: 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 1 Processor -- start 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 1 -- start start 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient: init 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:18.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 0x7fe3681cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient(hunting): picked mon.noname-a con 0x7fe36806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe36809bf40 0x7fe368121720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient(hunting): picked mon.noname-b con 0x7fe36809bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe368121c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient(hunting): picked mon.noname-c con 0x7fe36809d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe368073380 con 0x7fe36806c890 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe368073200 con 0x7fe36809bf40 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe368135330 con 0x7fe36809d430 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe36d41a640 10 monclient(hunting): authenticate will time out at 924.195802s 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 0x7fe3681cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3677fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe368121c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 0x7fe3681cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40726/0 (socket says 10.20.193.78:40726) 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3677fe640 10 monclient(hunting): get_auth_request con 0x7fe36809d430 auth_method 0 2026-02-17T05:51:18.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 1 -- 10.20.193.78:0/348008646 learned_addr learned my addr 10.20.193.78:0/348008646 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3677fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 10 monclient(hunting): get_auth_request con 0x7fe36806c890 auth_method 0 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 1 -- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 msgr2=0x7fe3681cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 1 -- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 msgr2=0x7fe3681cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3667fc640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe36809bf40 0x7fe368121720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 0x7fe3681cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3667fc640 10 monclient(hunting): get_auth_request con 0x7fe36809bf40 auth_method 0 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3667fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe3667fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.641+0000 7fe366ffd640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 0x7fe3681cf8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:18.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 10 monclient(hunting): handle_auth_done global_id 74177 payload 824 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 1 -- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe36809bf40 msgr2=0x7fe368121720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe36809bf40 0x7fe368121720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 1 -- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 msgr2=0x7fe3681cf8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 0x7fe3681cf8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 10 monclient: _finish_hunting 0 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 1 monclient: found mon.noname-c 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.642+0000 7fe3677fe640 1 -- 10.20.193.78:0/348008646 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe368126090 con 0x7fe36809d430 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3677fe640 1 -- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 msgr2=0x7fe368121c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3677fe640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe368121c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe358007d40 tx=0x7fe35800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 1 -- 10.20.193.78:0/348008646 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358003c90 con 0x7fe36809d430 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: dump: 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:18.652 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: _finish_auth 0 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 1 -- 10.20.193.78:0/348008646 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe358003e50 con 0x7fe36809d430 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 5 monclient: authenticate success, global_id 74177 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 10 monclient: _renew_subs 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3681256b0 con 0x7fe36809d430 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 1 -- 10.20.193.78:0/348008646 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358004260 con 0x7fe36809d430 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:18.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3657fa640 4 set_mon_vals no callback set 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe3477fe640 10 monclient: dump: 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 10 monclient: _renew_subs 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3680a15b0 con 0x7fe36809d430 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.644+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe368074980 con 0x7fe36809d430 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.645+0000 7fe3477fe640 1 -- 10.20.193.78:0/348008646 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe358004440 con 0x7fe36809d430 2026-02-17T05:51:18.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.645+0000 7fe3477fe640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe33403deb0 0x7fe334040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:18.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.645+0000 7fe3477fe640 1 -- 10.20.193.78:0/348008646 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(209..209 src has 117..209) ==== 4795+0+0 (secure 0 0 0) 0x7fe358052640 con 0x7fe36809d430 2026-02-17T05:51:18.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.645+0000 7fe366ffd640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe33403deb0 0x7fe334040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:18.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.645+0000 7fe366ffd640 10 monclient: get_auth_request con 0x7fe33403deb0 auth_method 0 2026-02-17T05:51:18.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.647+0000 7fe3477fe640 1 -- 10.20.193.78:0/348008646 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe368126090 con 0x7fe36809d430 2026-02-17T05:51:18.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.647+0000 7fe3477fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:18.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.647+0000 7fe3477fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:18.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.648+0000 7fe366ffd640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe33403deb0 0x7fe334040370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe368122fc0 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:18.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.758+0000 7fe36d41a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:18.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.758+0000 7fe36d41a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:18.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.758+0000 7fe36d41a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:18.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.758+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe368134df0 con 0x7fe36809d430 2026-02-17T05:51:18.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.759+0000 7fe3477fe640 1 -- 10.20.193.78:0/348008646 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fe35801b310 con 0x7fe36809d430 2026-02-17T05:51:18.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.759+0000 7fe3477fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:18.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.759+0000 7fe3477fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:18.760 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":48,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:18.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.759+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe33403deb0 msgr2=0x7fe334040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:18.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.759+0000 7fe36d41a640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe33403deb0 0x7fe334040370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe368122fc0 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2026-02-17T05:51:18.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.759+0000 7fe36d41a640 10 monclient: shutdown 2026-02-17T05:51:18.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.759+0000 7fe36d41a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:18.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.759+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 msgr2=0x7fe368121c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:18.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe368121c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe358007d40 tx=0x7fe35800e4b0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 shutdown_connections 2026-02-17T05:51:18.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe36809d430 0x7fe368121c60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe33403deb0 0x7fe334040370 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe36809bf40 0x7fe368121720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 --2- 10.20.193.78:0/348008646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe36806c890 0x7fe3681cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:18.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 >> 10.20.193.78:0/348008646 conn(0x7fe3680a6620 msgr2=0x7fe368137670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:18.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 shutdown_connections 2026-02-17T05:51:18.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:18.760+0000 7fe36d41a640 1 -- 10.20.193.78:0/348008646 wait complete. 2026-02-17T05:51:19.380 INFO:tasks.rados.rados.0.trial115.stdout:975: done (3 left) 2026-02-17T05:51:19.380 INFO:tasks.rados.rados.0.trial115.stdout:976: read oid 39 snap -1 2026-02-17T05:51:19.381 INFO:tasks.rados.rados.0.trial115.stdout:976: expect (ObjNum 362 snap 96 seq_num 362) 2026-02-17T05:51:19.381 INFO:tasks.rados.rados.0.trial115.stdout:977: write (excl) oid 16 current snap is 101 2026-02-17T05:51:19.381 INFO:tasks.rados.rados.0.trial115.stdout:977: seq_num 382 ranges {420274=562768,1620592=439131} 2026-02-17T05:51:19.384 INFO:tasks.rados.rados.0.trial115.stdout:977: writing trial11545420-16 from 420274 to 983042 tid 1 2026-02-17T05:51:19.388 INFO:tasks.rados.rados.0.trial115.stdout:977: writing trial11545420-16 from 1620592 to 2059723 tid 2 2026-02-17T05:51:19.388 INFO:tasks.rados.rados.0.trial115.stdout:978: write oid 50 current snap is 101 2026-02-17T05:51:19.388 INFO:tasks.rados.rados.0.trial115.stdout:978: seq_num 383 ranges {437081=638017,1679074=582789} 2026-02-17T05:51:19.392 INFO:tasks.rados.rados.0.trial115.stdout:978: writing trial11545420-50 from 437081 to 1075098 tid 1 2026-02-17T05:51:19.398 INFO:tasks.rados.rados.0.trial115.stdout:978: writing trial11545420-50 from 1679074 to 2261863 tid 2 2026-02-17T05:51:19.398 INFO:tasks.rados.rados.0.trial115.stdout:979: read oid 21 snap -1 2026-02-17T05:51:19.398 INFO:tasks.rados.rados.0.trial115.stdout:979: expect (ObjNum 275 snap 57 seq_num 275) 2026-02-17T05:51:19.398 INFO:tasks.rados.rados.0.trial115.stdout:980: read oid 47 snap -1 2026-02-17T05:51:19.398 INFO:tasks.rados.rados.0.trial115.stdout:980: expect (ObjNum 353 snap 90 seq_num 353) 2026-02-17T05:51:19.398 INFO:tasks.rados.rados.0.trial115.stdout:981: read oid 35 snap -1 2026-02-17T05:51:19.398 INFO:tasks.rados.rados.0.trial115.stdout:981: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:51:19.398 INFO:tasks.rados.rados.0.trial115.stdout:982: snap_remove snap 84 2026-02-17T05:51:20.383 INFO:tasks.rados.rados.0.trial115.stdout:982: done (9 left) 2026-02-17T05:51:20.383 INFO:tasks.rados.rados.0.trial115.stdout:983: write oid 30 current snap is 101 2026-02-17T05:51:20.383 INFO:tasks.rados.rados.0.trial115.stdout:983: seq_num 384 ranges {453888=713266,1737556=442800} 2026-02-17T05:51:20.388 INFO:tasks.rados.rados.0.trial115.stdout:983: writing trial11545420-30 from 453888 to 1167154 tid 1 2026-02-17T05:51:20.392 INFO:tasks.rados.rados.0.trial115.stdout:983: writing trial11545420-30 from 1737556 to 2180356 tid 2 2026-02-17T05:51:20.392 INFO:tasks.rados.rados.0.trial115.stdout:984: snap_remove snap 81 2026-02-17T05:51:21.371 INFO:tasks.rados.rados.0.trial115.stdout:984: done (10 left) 2026-02-17T05:51:21.372 INFO:tasks.rados.rados.0.trial115.stdout:985: write oid 4 current snap is 101 2026-02-17T05:51:21.372 INFO:tasks.rados.rados.0.trial115.stdout:985: seq_num 385 ranges {470695=788515,1679685=586458} 2026-02-17T05:51:21.376 INFO:tasks.rados.rados.0.trial115.stdout:985: writing trial11545420-4 from 470695 to 1259210 tid 1 2026-02-17T05:51:21.380 INFO:tasks.rados.rados.0.trial115.stdout:985: writing trial11545420-4 from 1679685 to 2266143 tid 2 2026-02-17T05:51:21.380 INFO:tasks.rados.rados.0.trial115.stdout:986: rollback oid 34 current snap is 101 2026-02-17T05:51:21.380 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 99 2026-02-17T05:51:21.380 INFO:tasks.rados.rados.0.trial115.stdout:972: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:51:21.380 INFO:tasks.rados.rados.0.trial115.stdout:972: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:51:21.380 INFO:tasks.rados.rados.0.trial115.stdout:972: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:51:21.380 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 286 (ObjNum 357 snap 92 seq_num 357) dirty exists 2026-02-17T05:51:21.380 INFO:tasks.rados.rados.0.trial115.stdout:973: expect (ObjNum 378 snap 99 seq_num 378) 2026-02-17T05:51:21.437 INFO:tasks.rados.rados.0.trial115.stdout:974: finishing copy_from racing read to trial11545420-13 2026-02-17T05:51:21.437 INFO:tasks.rados.rados.0.trial115.stdout:974: finishing copy_from to trial11545420-13 2026-02-17T05:51:21.437 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 364 (ObjNum 333 snap 83 seq_num 333) dirty exists 2026-02-17T05:51:21.437 INFO:tasks.rados.rados.0.trial115.stdout:976: expect (ObjNum 362 snap 96 seq_num 362) 2026-02-17T05:51:21.508 INFO:tasks.rados.rados.0.trial115.stdout:977: finishing write tid 1 to trial11545420-16 2026-02-17T05:51:21.508 INFO:tasks.rados.rados.0.trial115.stdout:977: finishing write tid 2 to trial11545420-16 2026-02-17T05:51:21.508 INFO:tasks.rados.rados.0.trial115.stdout:977: finishing write tid 3 to trial11545420-16 2026-02-17T05:51:21.508 INFO:tasks.rados.rados.0.trial115.stdout:977: finishing write tid 4 to trial11545420-16 2026-02-17T05:51:21.508 INFO:tasks.rados.rados.0.trial115.stdout:977: oid 16 updating version 0 to 426 2026-02-17T05:51:21.508 INFO:tasks.rados.rados.0.trial115.stdout:977: oid 16 version 426 is already newer than 425 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:977: oid 16 updating version 426 to 427 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 427 (ObjNum 382 snap 101 seq_num 382) dirty exists 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:977: left oid 16 (ObjNum 382 snap 101 seq_num 382) 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:978: finishing write tid 1 to trial11545420-50 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:978: finishing write tid 2 to trial11545420-50 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:972: done (11 left) 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:973: done (10 left) 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:974: done (9 left) 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:976: done (8 left) 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:977: done (7 left) 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:987: delete oid 17 current snap is 101 2026-02-17T05:51:21.509 INFO:tasks.rados.rados.0.trial115.stdout:978: finishing write tid 3 to trial11545420-50 2026-02-17T05:51:21.510 INFO:tasks.rados.rados.0.trial115.stdout:978: finishing write tid 4 to trial11545420-50 2026-02-17T05:51:21.510 INFO:tasks.rados.rados.0.trial115.stdout:978: oid 50 updating version 0 to 320 2026-02-17T05:51:21.510 INFO:tasks.rados.rados.0.trial115.stdout:978: oid 50 version 320 is already newer than 319 2026-02-17T05:51:21.510 INFO:tasks.rados.rados.0.trial115.stdout:978: oid 50 updating version 320 to 321 2026-02-17T05:51:21.510 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 321 (ObjNum 383 snap 101 seq_num 383) dirty exists 2026-02-17T05:51:21.510 INFO:tasks.rados.rados.0.trial115.stdout:978: left oid 50 (ObjNum 383 snap 101 seq_num 383) 2026-02-17T05:51:21.510 INFO:tasks.rados.rados.0.trial115.stdout:980: expect (ObjNum 353 snap 90 seq_num 353) 2026-02-17T05:51:21.596 INFO:tasks.rados.rados.0.trial115.stdout:981: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:51:21.652 INFO:tasks.rados.rados.0.trial115.stdout:979: expect (ObjNum 275 snap 57 seq_num 275) 2026-02-17T05:51:21.768 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:21.778 INFO:tasks.rados.rados.0.trial115.stdout:983: finishing write tid 1 to trial11545420-30 2026-02-17T05:51:21.778 INFO:tasks.rados.rados.0.trial115.stdout:983: finishing write tid 2 to trial11545420-30 2026-02-17T05:51:21.778 INFO:tasks.rados.rados.0.trial115.stdout:983: finishing write tid 3 to trial11545420-30 2026-02-17T05:51:21.778 INFO:tasks.rados.rados.0.trial115.stdout:983: finishing write tid 4 to trial11545420-30 2026-02-17T05:51:21.779 INFO:tasks.rados.rados.0.trial115.stdout:983: oid 30 updating version 0 to 329 2026-02-17T05:51:21.779 INFO:tasks.rados.rados.0.trial115.stdout:983: oid 30 version 329 is already newer than 328 2026-02-17T05:51:21.779 INFO:tasks.rados.rados.0.trial115.stdout:983: oid 30 version 329 is already newer than 327 2026-02-17T05:51:21.779 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 329 (ObjNum 384 snap 101 seq_num 384) dirty exists 2026-02-17T05:51:21.779 INFO:tasks.rados.rados.0.trial115.stdout:983: left oid 30 (ObjNum 384 snap 101 seq_num 384) 2026-02-17T05:51:21.779 INFO:tasks.rados.rados.0.trial115.stdout:985: finishing write tid 1 to trial11545420-4 2026-02-17T05:51:21.780 INFO:tasks.rados.rados.0.trial115.stdout:986: finishing rollback tid 1 to trial11545420-34 2026-02-17T05:51:21.780 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 251 (ObjNum 81 snap 0 seq_num 308980170) dirty dne 2026-02-17T05:51:21.780 INFO:tasks.rados.rados.0.trial115.stdout:985: finishing write tid 2 to trial11545420-4 2026-02-17T05:51:21.780 INFO:tasks.rados.rados.0.trial115.stdout:985: finishing write tid 3 to trial11545420-4 2026-02-17T05:51:21.780 INFO:tasks.rados.rados.0.trial115.stdout:985: finishing write tid 4 to trial11545420-4 2026-02-17T05:51:21.781 INFO:tasks.rados.rados.0.trial115.stdout:985: oid 4 updating version 0 to 288 2026-02-17T05:51:21.781 INFO:tasks.rados.rados.0.trial115.stdout:985: oid 4 version 288 is already newer than 287 2026-02-17T05:51:21.781 INFO:tasks.rados.rados.0.trial115.stdout:985: oid 4 updating version 288 to 289 2026-02-17T05:51:21.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 289 (ObjNum 385 snap 101 seq_num 385) dirty exists 2026-02-17T05:51:21.781 INFO:tasks.rados.rados.0.trial115.stdout:985: left oid 4 (ObjNum 385 snap 101 seq_num 385) 2026-02-17T05:51:21.781 INFO:tasks.rados.rados.0.trial115.stdout:978: done (7 left) 2026-02-17T05:51:21.782 INFO:tasks.rados.rados.0.trial115.stdout:979: done (6 left) 2026-02-17T05:51:21.782 INFO:tasks.rados.rados.0.trial115.stdout:980: done (5 left) 2026-02-17T05:51:21.782 INFO:tasks.rados.rados.0.trial115.stdout:981: done (4 left) 2026-02-17T05:51:21.782 INFO:tasks.rados.rados.0.trial115.stdout:983: done (3 left) 2026-02-17T05:51:21.782 INFO:tasks.rados.rados.0.trial115.stdout:985: done (2 left) 2026-02-17T05:51:21.783 INFO:tasks.rados.rados.0.trial115.stdout:986: done (1 left) 2026-02-17T05:51:21.783 INFO:tasks.rados.rados.0.trial115.stdout:987: done (0 left) 2026-02-17T05:51:21.783 INFO:tasks.rados.rados.0.trial115.stdout:988: copy_from oid 16 from oid 7 current snap is 101 2026-02-17T05:51:21.783 INFO:tasks.rados.rados.0.trial115.stdout:989: read oid 9 snap -1 2026-02-17T05:51:21.783 INFO:tasks.rados.rados.0.trial115.stdout:989: expect (ObjNum 376 snap 99 seq_num 376) 2026-02-17T05:51:21.783 INFO:tasks.rados.rados.0.trial115.stdout:990: read oid 1 snap -1 2026-02-17T05:51:21.784 INFO:tasks.rados.rados.0.trial115.stdout:990: expect (ObjNum 378 snap 99 seq_num 378) 2026-02-17T05:51:21.784 INFO:tasks.rados.rados.0.trial115.stdout:991: rollback oid 37 current snap is 101 2026-02-17T05:51:21.784 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 93 2026-02-17T05:51:21.784 INFO:tasks.rados.rados.0.trial115.stdout:992: write oid 45 current snap is 101 2026-02-17T05:51:21.784 INFO:tasks.rados.rados.0.trial115.stdout:992: seq_num 387 ranges {504309=539013,1796649=590127} 2026-02-17T05:51:21.786 INFO:tasks.rados.rados.0.trial115.stdout:992: writing trial11545420-45 from 504309 to 1043322 tid 1 2026-02-17T05:51:21.786 INFO:tasks.rados.rados.0.trial115.stdout:992: writing trial11545420-45 from 1796649 to 2386776 tid 2 2026-02-17T05:51:21.786 INFO:tasks.rados.rados.0.trial115.stdout:993: copy_from oid 46 from oid 43 current snap is 101 2026-02-17T05:51:21.786 INFO:tasks.rados.rados.0.trial115.stdout:994: write (excl) oid 38 current snap is 101 2026-02-17T05:51:21.787 INFO:tasks.rados.rados.0.trial115.stdout:994: seq_num 388 ranges {521116=730615,1971484=549632} 2026-02-17T05:51:21.789 INFO:tasks.rados.rados.0.trial115.stdout:994: writing trial11545420-38 from 521116 to 1251731 tid 1 2026-02-17T05:51:21.794 INFO:tasks.rados.rados.0.trial115.stdout:994: writing trial11545420-38 from 1971484 to 2521116 tid 2 2026-02-17T05:51:21.795 INFO:tasks.rados.rados.0.trial115.stdout:995: write (excl) oid 29 current snap is 101 2026-02-17T05:51:21.795 INFO:tasks.rados.rados.0.trial115.stdout:995: seq_num 389 ranges {537923=405864,1513613=593796} 2026-02-17T05:51:21.798 INFO:tasks.rados.rados.0.trial115.stdout:995: writing trial11545420-29 from 537923 to 943787 tid 1 2026-02-17T05:51:21.804 INFO:tasks.rados.rados.0.trial115.stdout:995: writing trial11545420-29 from 1513613 to 2107409 tid 2 2026-02-17T05:51:21.804 INFO:tasks.rados.rados.0.trial115.stdout:996: read oid 2 snap -1 2026-02-17T05:51:21.804 INFO:tasks.rados.rados.0.trial115.stdout:996: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:51:21.804 INFO:tasks.rados.rados.0.trial115.stdout:997: read oid 35 snap -1 2026-02-17T05:51:21.804 INFO:tasks.rados.rados.0.trial115.stdout:997: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:51:21.805 INFO:tasks.rados.rados.0.trial115.stdout:998: delete oid 41 current snap is 101 2026-02-17T05:51:21.805 INFO:tasks.rados.rados.0.trial115.stdout:988: finishing copy_from to trial11545420-16 2026-02-17T05:51:21.805 INFO:tasks.rados.rados.0.trial115.stdout:988: got expected ENOENT (src dne) 2026-02-17T05:51:21.805 INFO:tasks.rados.rados.0.trial115.stdout:988: finishing copy_from racing read to trial11545420-16 2026-02-17T05:51:21.805 INFO:tasks.rados.rados.0.trial115.stdout:989: expect (ObjNum 376 snap 99 seq_num 376) 2026-02-17T05:51:21.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.838+0000 7f6fd86ca640 10 monclient: get_monmap_and_config 2026-02-17T05:51:21.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.838+0000 7f6fd86ca640 10 monclient: build_initial_monmap 2026-02-17T05:51:21.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.838+0000 7f6fd86ca640 10 monclient: monmap: 2026-02-17T05:51:21.841 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:21.841 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:21.841 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:21.839898+0000 2026-02-17T05:51:21.841 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:21.839898+0000 2026-02-17T05:51:21.841 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:21.841 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 1 Processor -- start 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 1 -- start start 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient: init 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:21.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient(hunting): picked mon.noname-b con 0x7f6fd006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6fd009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6fd006d1b0 0x7f6fd0135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient(hunting): picked mon.noname-c con 0x7f6fd006d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd006daf0 con 0x7f6fd009c4a0 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd00a0880 con 0x7f6fd006d1b0 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd009cc80 con 0x7f6fd006c890 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:21.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd86ca640 10 monclient(hunting): authenticate will time out at 927.393457s 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42220/0 (socket says 10.20.193.78:42220) 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 1 -- 10.20.193.78:0/1034603548 learned_addr learned my addr 10.20.193.78:0/1034603548 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient(hunting): get_auth_request con 0x7f6fd006c890 auth_method 0 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd6c40640 1 --2- 10.20.193.78:0/1034603548 >> v2:10.20.193.115:3300/0 conn(0x7f6fd006d1b0 0x7f6fd0135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:21.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd6c40640 10 monclient(hunting): get_auth_request con 0x7f6fd006d1b0 auth_method 0 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd6c40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd6c40640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd6c40640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient(hunting): handle_auth_done global_id 74180 payload 274 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 1 -- 10.20.193.78:0/1034603548 >> v2:10.20.193.115:3300/0 conn(0x7f6fd006d1b0 msgr2=0x7f6fd0135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 1 --2- 10.20.193.78:0/1034603548 >> v2:10.20.193.115:3300/0 conn(0x7f6fd006d1b0 0x7f6fd0135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 1 -- 10.20.193.78:0/1034603548 >> v1:10.20.193.115:6789/0 conn(0x7f6fd009c4a0 legacy=0x7f6fd0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient: _finish_hunting 0 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 1 monclient: found mon.noname-b 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:21.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.839+0000 7f6fd643f640 1 -- 10.20.193.78:0/1034603548 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd009cf60 con 0x7f6fd006c890 2026-02-17T05:51:21.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd643f640 1 -- 10.20.193.78:0/1034603548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 msgr2=0x7f6fd006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:21.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd643f640 1 --2- 10.20.193.78:0/1034603548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd006cc70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009970 tx=0x7f6fc002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cf173192bb213af server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:21.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 1 -- 10.20.193.78:0/1034603548 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc003c070 con 0x7f6fd006c890 2026-02-17T05:51:21.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:21.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:21.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: dump: 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: _finish_auth 0 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: _check_auth_tickets 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 1 -- 10.20.193.78:0/1034603548 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc002fcd0 con 0x7f6fd006c890 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 5 monclient: authenticate success, global_id 74180 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 1 -- 10.20.193.78:0/1034603548 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0037660 con 0x7f6fd006c890 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:21.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd543d640 10 monclient: dump: 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 4 set_mon_vals no callback set 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 10 monclient: shutdown 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 1 -- 10.20.193.78:0/1034603548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 msgr2=0x7f6fd006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/1034603548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd006cc70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009970 tx=0x7f6fc002ee90 comp rx=0 tx=0).stop 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 1 -- 10.20.193.78:0/1034603548 shutdown_connections 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/1034603548 >> v2:10.20.193.115:3300/0 conn(0x7f6fd006d1b0 0x7f6fd0135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/1034603548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd006cc70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 1 -- 10.20.193.78:0/1034603548 >> 10.20.193.78:0/1034603548 conn(0x7f6fd00a6620 msgr2=0x7f6fd00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:21.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 1 -- 10.20.193.78:0/1034603548 shutdown_connections 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.842+0000 7f6fd86ca640 1 -- 10.20.193.78:0/1034603548 wait complete. 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient: build_initial_monmap 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient: monmap: 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 1 Processor -- start 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 1 -- start start 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient: init 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd01cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient(hunting): picked mon.noname-c con 0x7f6fd006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 0x7f6fd01cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:21.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient(hunting): picked mon.noname-a con 0x7f6fd006d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6fd009c4a0 0x7f6fd0123520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient(hunting): picked mon.noname-b con 0x7f6fd009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd00a01a0 con 0x7f6fd006d1b0 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd00a0020 con 0x7f6fd009c4a0 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd009be20 con 0x7f6fd006c890 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd86ca640 10 monclient(hunting): authenticate will time out at 927.397801s 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd01cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd01cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42232/0 (socket says 10.20.193.78:42232) 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 -- 10.20.193.78:0/4217332671 learned_addr learned my addr 10.20.193.78:0/4217332671 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient(hunting): get_auth_request con 0x7f6fd006c890 auth_method 0 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 0x7f6fd01cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:21.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 10 monclient(hunting): get_auth_request con 0x7f6fd006d1b0 auth_method 0 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 1 -- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 msgr2=0x7f6fd01cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 1 -- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 msgr2=0x7f6fd01cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 0x7f6fd01cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient(hunting): handle_auth_done global_id 74183 payload 824 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 -- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6fd009c4a0 msgr2=0x7f6fd0123520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd5c3e640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 0x7f6fd01cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6fd009c4a0 0x7f6fd0123520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 -- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 msgr2=0x7f6fd01cd870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 0x7f6fd01cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient: _finish_hunting 0 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 monclient: found mon.noname-c 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.843+0000 7f6fd643f640 1 -- 10.20.193.78:0/4217332671 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd0125c40 con 0x7f6fd006c890 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.846+0000 7f6fd643f640 1 -- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 msgr2=0x7f6fd01cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.846+0000 7f6fd643f640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd01cd330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009aa0 tx=0x7f6fc0004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.846+0000 7f6fb6ffd640 1 -- 10.20.193.78:0/4217332671 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 1 -- 10.20.193.78:0/4217332671 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0047070 con 0x7f6fd006c890 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:21.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: dump: 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 1 -- 10.20.193.78:0/4217332671 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc003f790 con 0x7f6fd006c890 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 5 monclient: authenticate success, global_id 74183 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 10 monclient: _renew_subs 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd01270f0 con 0x7f6fd006c890 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 1 -- 10.20.193.78:0/4217332671 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc003c040 con 0x7f6fd006c890 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd4c3c640 4 set_mon_vals no callback set 2026-02-17T05:51:21.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 10 monclient: dump: 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 10 monclient: _renew_subs 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd0126e10 con 0x7f6fd006c890 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd0074980 con 0x7f6fd006c890 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.848+0000 7f6fb6ffd640 1 -- 10.20.193.78:0/4217332671 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6fc003fb80 con 0x7f6fd006c890 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.849+0000 7f6fb6ffd640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6fa003df00 0x7f6fa00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.849+0000 7f6fb6ffd640 1 -- 10.20.193.78:0/4217332671 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(212..212 src has 117..212) ==== 4823+0+0 (secure 0 0 0) 0x7f6fc0077730 con 0x7f6fd006c890 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.849+0000 7f6fd5c3e640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6fa003df00 0x7f6fa00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.849+0000 7f6fd5c3e640 10 monclient: get_auth_request con 0x7f6fa003df00 auth_method 0 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.851+0000 7f6fb6ffd640 1 -- 10.20.193.78:0/4217332671 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6fc002fc80 con 0x7f6fd006c890 2026-02-17T05:51:21.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.851+0000 7f6fb6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:21.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.851+0000 7f6fb6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:21.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.852+0000 7f6fd5c3e640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6fa003df00 0x7f6fa00403c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0124ce0 tx=0x7f6fc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:21.895 INFO:tasks.rados.rados.0.trial115.stdout:991: finishing rollback tid 0 to trial11545420-37 2026-02-17T05:51:21.895 INFO:tasks.rados.rados.0.trial115.stdout:991: finishing rollback tid 1 to trial11545420-37 2026-02-17T05:51:21.895 INFO:tasks.rados.rados.0.trial115.stdout:988: done (10 left) 2026-02-17T05:51:21.895 INFO:tasks.rados.rados.0.trial115.stdout:989: done (9 left) 2026-02-17T05:51:21.896 INFO:tasks.rados.rados.0.trial115.stdout:998: done (8 left) 2026-02-17T05:51:21.896 INFO:tasks.rados.rados.0.trial115.stdout:999: delete oid 22 current snap is 101 2026-02-17T05:51:21.896 INFO:tasks.rados.rados.0.trial115.stdout:991: finishing rollback tid 2 to trial11545420-37 2026-02-17T05:51:21.896 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 405 (ObjNum 356 snap 91 seq_num 356) dirty exists 2026-02-17T05:51:21.896 INFO:tasks.rados.rados.0.trial115.stdout:990: expect (ObjNum 378 snap 99 seq_num 378) 2026-02-17T05:51:21.952 INFO:tasks.rados.rados.0.trial115.stdout:992: finishing write tid 1 to trial11545420-45 2026-02-17T05:51:21.952 INFO:tasks.rados.rados.0.trial115.stdout:990: done (8 left) 2026-02-17T05:51:21.952 INFO:tasks.rados.rados.0.trial115.stdout:991: done (7 left) 2026-02-17T05:51:21.952 INFO:tasks.rados.rados.0.trial115.stdout:999: done (6 left) 2026-02-17T05:51:21.952 INFO:tasks.rados.rados.0.trial115.stdout:1000: delete oid 6 current snap is 101 2026-02-17T05:51:21.952 INFO:tasks.rados.rados.0.trial115.stdout:992: finishing write tid 2 to trial11545420-45 2026-02-17T05:51:21.952 INFO:tasks.rados.rados.0.trial115.stdout:992: finishing write tid 3 to trial11545420-45 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:992: finishing write tid 4 to trial11545420-45 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:992: oid 45 updating version 0 to 471 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:992: oid 45 version 471 is already newer than 470 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:992: oid 45 updating version 471 to 472 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 472 (ObjNum 387 snap 101 seq_num 387) dirty exists 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:992: left oid 45 (ObjNum 387 snap 101 seq_num 387) 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:993: finishing copy_from racing read to trial11545420-46 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:993: finishing copy_from to trial11545420-46 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 394 (ObjNum 302 snap 72 seq_num 302) dirty exists 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:994: finishing write tid 1 to trial11545420-38 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:994: finishing write tid 2 to trial11545420-38 2026-02-17T05:51:21.953 INFO:tasks.rados.rados.0.trial115.stdout:994: finishing write tid 3 to trial11545420-38 2026-02-17T05:51:21.954 INFO:tasks.rados.rados.0.trial115.stdout:994: finishing write tid 4 to trial11545420-38 2026-02-17T05:51:21.954 INFO:tasks.rados.rados.0.trial115.stdout:994: oid 38 updating version 0 to 447 2026-02-17T05:51:21.954 INFO:tasks.rados.rados.0.trial115.stdout:994: oid 38 updating version 447 to 448 2026-02-17T05:51:21.954 INFO:tasks.rados.rados.0.trial115.stdout:994: oid 38 updating version 448 to 449 2026-02-17T05:51:21.954 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 449 (ObjNum 388 snap 101 seq_num 388) dirty exists 2026-02-17T05:51:21.954 INFO:tasks.rados.rados.0.trial115.stdout:994: left oid 38 (ObjNum 388 snap 101 seq_num 388) 2026-02-17T05:51:21.954 INFO:tasks.rados.rados.0.trial115.stdout:995: finishing write tid 1 to trial11545420-29 2026-02-17T05:51:21.954 INFO:tasks.rados.rados.0.trial115.stdout:996: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:51:21.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.964+0000 7f6fd86ca640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:21.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.964+0000 7f6fd86ca640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:21.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.964+0000 7f6fd86ca640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:21.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.964+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6fd0125520 con 0x7f6fd006c890 2026-02-17T05:51:21.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.964+0000 7f6fb6ffd640 1 -- 10.20.193.78:0/4217332671 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f6fc0036e20 con 0x7f6fd006c890 2026-02-17T05:51:21.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.964+0000 7f6fb6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:21.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.964+0000 7f6fb6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:21.965 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":48,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":27,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:21.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6fa003df00 msgr2=0x7f6fa00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:21.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6fa003df00 0x7f6fa00403c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0124ce0 tx=0x7f6fc4009040 comp rx=0 tx=0).stop 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 10 monclient: shutdown 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 msgr2=0x7f6fd01cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd01cd330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009aa0 tx=0x7f6fc0004830 comp rx=0 tx=0).stop 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 shutdown_connections 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6fa003df00 0x7f6fa00403c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6fd009c4a0 0x7f6fd0123520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6fd006d1b0 0x7f6fd01cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 --2- 10.20.193.78:0/4217332671 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6fd006c890 0x7f6fd01cd330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:21.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 >> 10.20.193.78:0/4217332671 conn(0x7f6fd00a6620 msgr2=0x7f6fd0072240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:21.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 shutdown_connections 2026-02-17T05:51:21.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:21.965+0000 7f6fd86ca640 1 -- 10.20.193.78:0/4217332671 wait complete. 2026-02-17T05:51:22.002 INFO:tasks.rados.rados.0.trial115.stdout:995: finishing write tid 2 to trial11545420-29 2026-02-17T05:51:22.002 INFO:tasks.rados.rados.0.trial115.stdout:995: finishing write tid 3 to trial11545420-29 2026-02-17T05:51:22.002 INFO:tasks.rados.rados.0.trial115.stdout:992: done (6 left) 2026-02-17T05:51:22.002 INFO:tasks.rados.rados.0.trial115.stdout:993: done (5 left) 2026-02-17T05:51:22.002 INFO:tasks.rados.rados.0.trial115.stdout:994: done (4 left) 2026-02-17T05:51:22.002 INFO:tasks.rados.rados.0.trial115.stdout:996: done (3 left) 2026-02-17T05:51:22.002 INFO:tasks.rados.rados.0.trial115.stdout:1000: done (2 left) 2026-02-17T05:51:22.002 INFO:tasks.rados.rados.0.trial115.stdout:1001: write oid 27 current snap is 101 2026-02-17T05:51:22.003 INFO:tasks.rados.rados.0.trial115.stdout:1001: seq_num 393 ranges {605151=706860,2031188=573963} 2026-02-17T05:51:22.007 INFO:tasks.rados.rados.0.trial115.stdout:1001: writing trial11545420-27 from 605151 to 1312011 tid 1 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:1001: writing trial11545420-27 from 2031188 to 2605151 tid 2 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:1002: read oid 18 snap -1 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:1002: expect deleted 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:1003: delete oid 5 current snap is 101 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:995: finishing write tid 4 to trial11545420-29 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:995: oid 29 updating version 0 to 397 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:995: oid 29 version 397 is already newer than 395 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:995: oid 29 version 397 is already newer than 396 2026-02-17T05:51:22.013 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 397 (ObjNum 389 snap 101 seq_num 389) dirty exists 2026-02-17T05:51:22.014 INFO:tasks.rados.rados.0.trial115.stdout:995: left oid 29 (ObjNum 389 snap 101 seq_num 389) 2026-02-17T05:51:22.014 INFO:tasks.rados.rados.0.trial115.stdout:997: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:51:22.066 INFO:tasks.rados.rados.0.trial115.stdout:1001: finishing write tid 1 to trial11545420-27 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:995: done (4 left) 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:997: done (3 left) 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1003: done (2 left) 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1004: delete oid 22 current snap is 101 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1001: finishing write tid 2 to trial11545420-27 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1001: finishing write tid 3 to trial11545420-27 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1001: finishing write tid 4 to trial11545420-27 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1001: oid 27 updating version 0 to 341 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1001: oid 27 version 341 is already newer than 340 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1001: oid 27 version 341 is already newer than 339 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 341 (ObjNum 393 snap 101 seq_num 393) dirty exists 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1001: left oid 27 (ObjNum 393 snap 101 seq_num 393) 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1001: done (2 left) 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1002: done (1 left) 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1004: done (0 left) 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1005: read oid 42 snap -1 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1005: expect (ObjNum 352 snap 90 seq_num 352) 2026-02-17T05:51:22.067 INFO:tasks.rados.rados.0.trial115.stdout:1006: snap_create 2026-02-17T05:51:22.072 INFO:tasks.rados.rados.0.trial115.stdout:1005: expect (ObjNum 352 snap 90 seq_num 352) 2026-02-17T05:51:22.388 INFO:tasks.rados.rados.0.trial115.stdout:1005: done (1 left) 2026-02-17T05:51:22.388 INFO:tasks.rados.rados.0.trial115.stdout:1006: done (0 left) 2026-02-17T05:51:22.388 INFO:tasks.rados.rados.0.trial115.stdout:1007: rollback oid 17 current snap is 102 2026-02-17T05:51:22.389 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 78 2026-02-17T05:51:22.389 INFO:tasks.rados.rados.0.trial115.stdout:1008: read oid 23 snap 98 2026-02-17T05:51:22.389 INFO:tasks.rados.rados.0.trial115.stdout:1008: expect (ObjNum 323 snap 78 seq_num 323) 2026-02-17T05:51:22.389 INFO:tasks.rados.rados.0.trial115.stdout:1009: delete oid 27 current snap is 102 2026-02-17T05:51:22.398 INFO:tasks.rados.rados.0.trial115.stdout:1007: finishing rollback tid 1 to trial11545420-17 2026-02-17T05:51:22.398 INFO:tasks.rados.rados.0.trial115.stdout:1009: done (2 left) 2026-02-17T05:51:22.398 INFO:tasks.rados.rados.0.trial115.stdout:1010: snap_remove snap 90 2026-02-17T05:51:23.390 INFO:tasks.rados.rados.0.trial115.stdout:1010: done (2 left) 2026-02-17T05:51:23.390 INFO:tasks.rados.rados.0.trial115.stdout:1011: write oid 40 current snap is 102 2026-02-17T05:51:23.390 INFO:tasks.rados.rados.0.trial115.stdout:1011: seq_num 397 ranges {672379=724209,1865116=608472} 2026-02-17T05:51:23.395 INFO:tasks.rados.rados.0.trial115.stdout:1011: writing trial11545420-40 from 672379 to 1396588 tid 1 2026-02-17T05:51:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1011: writing trial11545420-40 from 1865116 to 2473588 tid 2 2026-02-17T05:51:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1012: snap_remove snap 93 2026-02-17T05:51:24.392 INFO:tasks.rados.rados.0.trial115.stdout:1012: done (3 left) 2026-02-17T05:51:24.392 INFO:tasks.rados.rados.0.trial115.stdout:1013: read oid 35 snap -1 2026-02-17T05:51:24.392 INFO:tasks.rados.rados.0.trial115.stdout:1013: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:51:24.392 INFO:tasks.rados.rados.0.trial115.stdout:1014: copy_from oid 16 from oid 41 current snap is 102 2026-02-17T05:51:24.392 INFO:tasks.rados.rados.0.trial115.stdout:1008: expect (ObjNum 323 snap 78 seq_num 323) 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1007: finishing rollback tid 2 to trial11545420-17 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 152 (ObjNum 301 snap 72 seq_num 301) dirty exists 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1011: finishing write tid 1 to trial11545420-40 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1011: finishing write tid 2 to trial11545420-40 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1011: finishing write tid 3 to trial11545420-40 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1011: finishing write tid 4 to trial11545420-40 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1011: oid 40 updating version 0 to 292 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1011: oid 40 version 292 is already newer than 291 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1011: oid 40 version 292 is already newer than 290 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 292 (ObjNum 397 snap 102 seq_num 397) dirty exists 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1011: left oid 40 (ObjNum 397 snap 102 seq_num 397) 2026-02-17T05:51:24.476 INFO:tasks.rados.rados.0.trial115.stdout:1013: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:51:24.530 INFO:tasks.rados.rados.0.trial115.stdout:1014: finishing copy_from to trial11545420-16 2026-02-17T05:51:24.531 INFO:tasks.rados.rados.0.trial115.stdout:1014: got expected ENOENT (src dne) 2026-02-17T05:51:24.531 INFO:tasks.rados.rados.0.trial115.stdout:1007: done (4 left) 2026-02-17T05:51:24.531 INFO:tasks.rados.rados.0.trial115.stdout:1008: done (3 left) 2026-02-17T05:51:24.531 INFO:tasks.rados.rados.0.trial115.stdout:1011: done (2 left) 2026-02-17T05:51:24.531 INFO:tasks.rados.rados.0.trial115.stdout:1013: done (1 left) 2026-02-17T05:51:24.531 INFO:tasks.rados.rados.0.trial115.stdout:1015: snap_create 2026-02-17T05:51:24.531 INFO:tasks.rados.rados.0.trial115.stdout:1014: finishing copy_from racing read to trial11545420-16 2026-02-17T05:51:24.973 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:25.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.043+0000 7f9838b51640 10 monclient: get_monmap_and_config 2026-02-17T05:51:25.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.043+0000 7f9838b51640 10 monclient: build_initial_monmap 2026-02-17T05:51:25.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.043+0000 7f9838b51640 10 monclient: monmap: 2026-02-17T05:51:25.042 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:25.042 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:25.042 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:25.044544+0000 2026-02-17T05:51:25.042 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:25.044544+0000 2026-02-17T05:51:25.042 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:25.043 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:25.043 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:25.043 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:25.043 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:25.043 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:25.043 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:25.043 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:25.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.043+0000 7f9838b51640 1 Processor -- start 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.043+0000 7f9838b51640 1 -- start start 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.043+0000 7f9838b51640 10 monclient: init 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f983406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f983409d430 0x7f983413d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient(hunting): picked mon.noname-c con 0x7f983409d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f983413fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient(hunting): picked mon.noname-b con 0x7f983409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9834073600 con 0x7f983406c890 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f983409ca50 con 0x7f983409d430 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f983406dab0 con 0x7f983409bf40 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9838b51640 10 monclient(hunting): authenticate will time out at 930.598155s 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f983413fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f983413fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42250/0 (socket says 10.20.193.78:42250) 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 1 -- 10.20.193.78:0/950377577 learned_addr learned my addr 10.20.193.78:0/950377577 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient(hunting): get_auth_request con 0x7f983409bf40 auth_method 0 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9831d74640 1 --2- 10.20.193.78:0/950377577 >> v2:10.20.193.115:3300/0 conn(0x7f983409d430 0x7f983413d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9831d74640 10 monclient(hunting): get_auth_request con 0x7f983409d430 auth_method 0 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9831d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9831d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9831d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient(hunting): handle_auth_done global_id 74186 payload 274 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 1 -- 10.20.193.78:0/950377577 >> v2:10.20.193.115:3300/0 conn(0x7f983409d430 msgr2=0x7f983413d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 1 --2- 10.20.193.78:0/950377577 >> v2:10.20.193.115:3300/0 conn(0x7f983409d430 0x7f983413d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 1 -- 10.20.193.78:0/950377577 >> v1:10.20.193.115:6789/0 conn(0x7f983406c890 legacy=0x7f983406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient: _finish_hunting 0 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 1 monclient: found mon.noname-b 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9832d76640 1 -- 10.20.193.78:0/950377577 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983409c770 con 0x7f983409bf40 2026-02-17T05:51:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.044+0000 7f9831d74640 1 --2- 10.20.193.78:0/950377577 >> v2:10.20.193.115:3300/0 conn(0x7f983409d430 0x7f983413d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9832d76640 1 -- 10.20.193.78:0/950377577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 msgr2=0x7f983413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9832d76640 1 --2- 10.20.193.78:0/950377577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f983413fb20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f983409cbd0 tx=0x7f982402f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ecf6f800e31797 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 1 -- 10.20.193.78:0/950377577 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240373f0 con 0x7f983409bf40 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: dump: 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: _finish_auth 0 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: _check_auth_tickets 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 1 -- 10.20.193.78:0/950377577 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9824037590 con 0x7f983409bf40 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 5 monclient: authenticate success, global_id 74186 2026-02-17T05:51:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 1 -- 10.20.193.78:0/950377577 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824037960 con 0x7f983409bf40 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9831573640 10 monclient: dump: 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 4 set_mon_vals no callback set 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 10 monclient: shutdown 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 1 -- 10.20.193.78:0/950377577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 msgr2=0x7f983413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 1 --2- 10.20.193.78:0/950377577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f983413fb20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f983409cbd0 tx=0x7f982402f200 comp rx=0 tx=0).stop 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 1 -- 10.20.193.78:0/950377577 shutdown_connections 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 1 --2- 10.20.193.78:0/950377577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f983413fb20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 1 --2- 10.20.193.78:0/950377577 >> v2:10.20.193.115:3300/0 conn(0x7f983409d430 0x7f983413d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 1 -- 10.20.193.78:0/950377577 >> 10.20.193.78:0/950377577 conn(0x7f98340a6620 msgr2=0x7f98340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 1 -- 10.20.193.78:0/950377577 shutdown_connections 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.047+0000 7f9838b51640 1 -- 10.20.193.78:0/950377577 wait complete. 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient: build_initial_monmap 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient: monmap: 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:25.048 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 1 Processor -- start 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 1 -- start start 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient: init 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 0x7f98341cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient(hunting): picked mon.noname-a con 0x7f983406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f98341cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient(hunting): picked mon.noname-c con 0x7f983409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f983409d430 0x7f9834122610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient(hunting): picked mon.noname-b con 0x7f983409d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f983406d380 con 0x7f983406c890 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9834073410 con 0x7f983409d430 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f983406d500 con 0x7f983409bf40 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9838b51640 10 monclient(hunting): authenticate will time out at 930.602644s 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f98341cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f98341cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:42262/0 (socket says 10.20.193.78:42262) 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 -- 10.20.193.78:0/3216916318 learned_addr learned my addr 10.20.193.78:0/3216916318 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient(hunting): get_auth_request con 0x7f983409bf40 auth_method 0 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 0x7f98341cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 10 monclient(hunting): get_auth_request con 0x7f983406c890 auth_method 0 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 1 -- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 msgr2=0x7f98341cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 1 -- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 msgr2=0x7f98341cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:51:25.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient(hunting): handle_auth_done global_id 74189 payload 824 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 0x7f98341cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 -- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f983409d430 msgr2=0x7f9834122610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f983409d430 0x7f9834122610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9832575640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 0x7f98341cf560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 -- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 msgr2=0x7f98341cf560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 0x7f98341cf560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient: _finish_hunting 0 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 monclient: found mon.noname-c 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.048+0000 7f9831d74640 1 -- 10.20.193.78:0/3216916318 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9834124210 con 0x7f983409bf40 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9831d74640 1 -- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 msgr2=0x7f98341cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9831d74640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f98341cfaa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f981c00e8b0 tx=0x7f981c00ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 1 -- 10.20.193.78:0/3216916318 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c00f040 con 0x7f983409bf40 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: dump: 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:25.050 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: _finish_auth 0 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: _check_auth_tickets 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 1 -- 10.20.193.78:0/3216916318 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c004230 con 0x7f983409bf40 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 1 -- 10.20.193.78:0/3216916318 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c00c860 con 0x7f983409bf40 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 10 monclient: dump: 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9830d72640 4 set_mon_vals no callback set 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 5 monclient: authenticate success, global_id 74189 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 10 monclient: _renew_subs 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9834126500 con 0x7f983409bf40 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 10 monclient: _renew_subs 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:25.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98340a2ac0 con 0x7f983409bf40 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f981affd640 1 -- 10.20.193.78:0/3216916318 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f981c012050 con 0x7f983409bf40 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.051+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98340a2370 con 0x7f983409bf40 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.052+0000 7f981affd640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f980003df00 0x7f98000403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.052+0000 7f981affd640 1 -- 10.20.193.78:0/3216916318 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(215..215 src has 117..215) ==== 4811+0+0 (secure 0 0 0) 0x7f981c0523b0 con 0x7f983409bf40 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.052+0000 7f9832575640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f980003df00 0x7f98000403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:25.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.052+0000 7f9832575640 10 monclient: get_auth_request con 0x7f980003df00 auth_method 0 2026-02-17T05:51:25.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.054+0000 7f981affd640 1 -- 10.20.193.78:0/3216916318 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f98340a2370 con 0x7f983409bf40 2026-02-17T05:51:25.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.054+0000 7f981affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:25.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.054+0000 7f981affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:25.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.054+0000 7f9832575640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f980003df00 0x7f98000403c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9828007d50 tx=0x7f9828009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:25.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.168+0000 7f9838b51640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:25.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.168+0000 7f9838b51640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:25.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.168+0000 7f9838b51640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:25.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.168+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9834074980 con 0x7f983409bf40 2026-02-17T05:51:25.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.168+0000 7f981affd640 1 -- 10.20.193.78:0/3216916318 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f981c017020 con 0x7f983409bf40 2026-02-17T05:51:25.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.168+0000 7f981affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:25.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.168+0000 7f981affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:25.169 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":48,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":30,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:25.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f980003df00 msgr2=0x7f98000403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f980003df00 0x7f98000403c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9828007d50 tx=0x7f9828009040 comp rx=0 tx=0).stop 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 10 monclient: shutdown 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 msgr2=0x7f98341cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f98341cfaa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f981c00e8b0 tx=0x7f981c00ed80 comp rx=0 tx=0).stop 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 shutdown_connections 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f980003df00 0x7f98000403c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f983409d430 0x7f9834122610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f983409bf40 0x7f98341cfaa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 --2- 10.20.193.78:0/3216916318 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f983406c890 0x7f98341cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:25.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 >> 10.20.193.78:0/3216916318 conn(0x7f98340a6620 msgr2=0x7f9834136a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:25.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 shutdown_connections 2026-02-17T05:51:25.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:25.170+0000 7f9838b51640 1 -- 10.20.193.78:0/3216916318 wait complete. 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1014: done (1 left) 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1015: done (0 left) 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1016: copy_from oid 46 from oid 40 current snap is 103 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1017: rollback oid 12 current snap is 103 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 99 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1018: read oid 49 snap -1 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1018: expect deleted 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1019: read oid 2 snap 98 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1019: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1020: write (excl) oid 26 current snap is 103 2026-02-17T05:51:25.397 INFO:tasks.rados.rados.0.trial115.stdout:1020: seq_num 398 ranges {689186=799458,1923598=752130} 2026-02-17T05:51:25.401 INFO:tasks.rados.rados.0.trial115.stdout:1020: writing trial11545420-26 from 689186 to 1488644 tid 1 2026-02-17T05:51:25.405 INFO:tasks.rados.rados.0.trial115.stdout:1020: writing trial11545420-26 from 1923598 to 2675728 tid 2 2026-02-17T05:51:25.405 INFO:tasks.rados.rados.0.trial115.stdout:1021: snap_remove snap 87 2026-02-17T05:51:26.373 INFO:tasks.rados.rados.0.trial115.stdout:1021: done (5 left) 2026-02-17T05:51:26.373 INFO:tasks.rados.rados.0.trial115.stdout:1022: snap_remove snap 100 2026-02-17T05:51:28.178 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:28.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.246+0000 7fd002b3b640 10 monclient: get_monmap_and_config 2026-02-17T05:51:28.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.246+0000 7fd002b3b640 10 monclient: build_initial_monmap 2026-02-17T05:51:28.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.246+0000 7fd002b3b640 10 monclient: monmap: 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:28.247504+0000 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:28.247504+0000 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:28.246 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.246+0000 7fd002b3b640 1 Processor -- start 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.246+0000 7fd002b3b640 1 -- start start 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.246+0000 7fd002b3b640 10 monclient: init 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient(hunting): picked mon.noname-a con 0x7fcffc0a6470 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcffc0a6850 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient(hunting): picked mon.noname-c con 0x7fcffc0a6c60 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcffc0a0730 con 0x7fcffc0a6850 2026-02-17T05:51:28.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fcffc09fa00 con 0x7fcffc0a6470 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fcffc0a19c0 con 0x7fcffc0a6c60 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd002b3b640 10 monclient(hunting): authenticate will time out at 933.801084s 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36894/0 (socket says 10.20.193.78:36894) 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 learned_addr learned my addr 10.20.193.78:0/2851987064 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 10 monclient(hunting): get_auth_request con 0x7fcffc0a6470 auth_method 0 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 10 monclient(hunting): get_auth_request con 0x7fcffc0a6c60 auth_method 0 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:28.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.247+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:28.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.249+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> v1:10.20.193.115:6789/0 conn(0x7fcffc0a6850 legacy=0x7fcffc137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 10 monclient(hunting): get_auth_request con 0x7fcffc0a6470 auth_method 0 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:51:28.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:28.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 10 monclient(hunting): get_auth_request con 0x7fcffc0a6c60 auth_method 0 2026-02-17T05:51:28.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:28.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:28.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:28.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:51:28.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:28.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:28.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.447+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:51:28.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:28.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 10 monclient(hunting): get_auth_request con 0x7fcffc0a6470 auth_method 0 2026-02-17T05:51:28.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:28.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:28.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:28.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:51:28.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 10 monclient(hunting): get_auth_request con 0x7fcffc0a6c60 auth_method 0 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:28.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:28.848+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:51:29.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:29.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 10 monclient(hunting): get_auth_request con 0x7fcffc0a6470 auth_method 0 2026-02-17T05:51:29.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 10 monclient(hunting): get_auth_request con 0x7fcffc0a6c60 auth_method 0 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:29.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:29.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:51:29.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:29.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:29.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:29.649+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:51:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 10 monclient(hunting): tick 2026-02-17T05:51:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:51:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 monclient(hunting): continuing hunt 2026-02-17T05:51:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:51:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 msgr2=0x7fcffc138660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.115:3300/0 conn(0x7fcffc0a6c60 0x7fcffc138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 -- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 msgr2=0x7fcffc0a1480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 --2- 10.20.193.78:0/2851987064 >> v2:10.20.193.78:3300/0 conn(0x7fcffc0a6470 0x7fcffc0a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 -- 10.20.193.78:0/2851987064 >> v1:10.20.193.115:6789/0 conn(0x7fcffc0a6850 legacy=0x7fcffc137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 10 monclient: _add_conns ranks=[0,2] 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcfe00037f0 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 10 monclient(hunting): picked mon.noname-b con 0x7fcfe00074c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 -- 10.20.193.78:0/2851987064 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcffc0a0450 con 0x7fcfe00037f0 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 1 -- 10.20.193.78:0/2851987064 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcffc09fa00 con 0x7fcfe00074c0 2026-02-17T05:51:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff27fc640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 10 monclient(hunting): get_auth_request con 0x7fcfe00074c0 auth_method 0 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.247+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.249+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> v1:10.20.193.78:6789/0 conn(0x7fcfe00037f0 legacy=0x7fcfe0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 10 monclient(hunting): get_auth_request con 0x7fcfe00074c0 auth_method 0 2026-02-17T05:51:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:51:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.447+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 10 monclient(hunting): get_auth_request con 0x7fcfe00074c0 auth_method 0 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:31.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:31.848+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:51:32.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:32.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 10 monclient(hunting): get_auth_request con 0x7fcfe00074c0 auth_method 0 2026-02-17T05:51:32.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:32.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:32.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:32.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:51:32.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:51:32.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:32.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:32.649+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:51:33.991 INFO:tasks.rados.rados.0.trial115.stdout:1022: done (5 left) 2026-02-17T05:51:33.992 INFO:tasks.rados.rados.0.trial115.stdout:1023: rollback oid 28 current snap is 103 2026-02-17T05:51:33.992 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 28 to 99 2026-02-17T05:51:33.992 INFO:tasks.rados.rados.0.trial115.stdout:1019: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3337988391 0 0) 0x7fcffc0a0450 con 0x7fcfe00037f0 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): _negotiate 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): my global_id is 84099 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfe8003620 con 0x7fcfe00037f0 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2055235874 0 0) 0x7fcfe8003620 con 0x7fcfe00037f0 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 msgr2=0x7fcfe0009980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe00074c0 0x7fcfe0009980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe000b220 con 0x7fcfe00037f0 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: _finish_auth 0 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fcfec0096c0 con 0x7fcfe00037f0 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: dump: 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 >> v1:10.20.193.78:6789/0 conn(0x7fcfe00037f0 legacy=0x7fcfe0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 1 -- 10.20.193.78:0/2851987064 reap_dead start 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcfe8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): picked mon.c con 0x7fcfe8003b50 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe8004900 0x7fcfe8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): picked mon.b con 0x7fcfe8004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe80054c0 0x7fcfe800bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): picked mon.a con 0x7fcfe80054c0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc09fa00 con 0x7fcfe80054c0 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe800c3c0 con 0x7fcfe8004900 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe800c5c0 con 0x7fcfe8003b50 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff37fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcfe8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient(hunting): get_auth_request con 0x7fcfe8003b50 auth_method 0 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe80054c0 0x7fcfe800bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 10 monclient(hunting): get_auth_request con 0x7fcfe80054c0 auth_method 0 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0010b1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe8004900 0x7fcfe8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff3fff640 10 monclient(hunting): get_auth_request con 0x7fcfe8004900 auth_method 0 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fcff3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient(hunting): handle_auth_done global_id 84099 payload 293 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe8004900 msgr2=0x7fcfe8004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe8004900 0x7fcfe8004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe80054c0 msgr2=0x7fcfe800bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe80054c0 0x7fcfe800bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient: _finish_hunting 0 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 1 monclient: found mon.c 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe8003620 con 0x7fcfe8003b50 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient: _finish_auth 0 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 10 monclient: _check_auth_tickets 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd0008b0640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd002b3b640 5 monclient: authenticate success, global_id 84099 2026-02-17T05:51:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.995+0000 7fd002b3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd0008b0640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 msgr2=0x7fcfe8003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcfe8003f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006fd0 tx=0x7fcfec0300f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ef7d6dc26cfacf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec036070 con 0x7fcfe8003b50 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 10 monclient: dump: 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec030b50 con 0x7fcfe8003b50 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:34.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 1 -- 10.20.193.78:0/2851987064 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec004040 con 0x7fcfe8003b50 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 4 set_mon_vals no callback set 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fcff37fe640 10 monclient: dump: 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 10 monclient: shutdown 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 -- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 msgr2=0x7fcfe8003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcfe8003f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006fd0 tx=0x7fcfec0300f0 comp rx=0 tx=0).stop 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 -- 10.20.193.78:0/2851987064 shutdown_connections 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe80054c0 0x7fcfe800bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe8004900 0x7fcfe8004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:34.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2851987064 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcfe8003f50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 -- 10.20.193.78:0/2851987064 >> 10.20.193.78:0/2851987064 conn(0x7fcffc0aae40 msgr2=0x7fcffc0ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 -- 10.20.193.78:0/2851987064 shutdown_connections 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.998+0000 7fd002b3b640 1 -- 10.20.193.78:0/2851987064 wait complete. 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient: build_initial_monmap 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient: monmap: 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 1 Processor -- start 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 1 -- start start 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient: init 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcffc1ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient(hunting): picked mon.noname-c con 0x7fcfe8003b50 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:34.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe8004900 0x7fcffc1cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient(hunting): picked mon.noname-a con 0x7fcfe8004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe80054c0 0x7fcffc1cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient(hunting): picked mon.noname-b con 0x7fcfe80054c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc0a1480 con 0x7fcfe8004900 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc0a1600 con 0x7fcfe80054c0 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcffc0a1780 con 0x7fcfe8003b50 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd002b3b640 10 monclient(hunting): authenticate will time out at 939.553747s 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcffc1ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcffc1ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40902/0 (socket says 10.20.193.78:40902) 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe8004900 0x7fcffc1cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 1 -- 10.20.193.78:0/2386357579 learned_addr learned my addr 10.20.193.78:0/2386357579 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 10 monclient(hunting): get_auth_request con 0x7fcfe8003b50 auth_method 0 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:34.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient(hunting): get_auth_request con 0x7fcfe8004900 auth_method 0 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 1 -- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 msgr2=0x7fcffc1ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 1 -- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 msgr2=0x7fcffc1ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcffc1ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcffc1ceeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe80054c0 0x7fcffc1cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient(hunting): handle_auth_done global_id 84102 payload 824 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 1 -- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 msgr2=0x7fcffc1ceeb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcffc1ceeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 1 -- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe80054c0 msgr2=0x7fcffc1cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe80054c0 0x7fcffc1cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 1 monclient: found mon.noname-a 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:34.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fcff3fff640 1 -- 10.20.193.78:0/2386357579 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc136960 con 0x7fcfe8004900 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0010b1640 10 monclient: get_auth_request con 0x7fcfe80054c0 auth_method 0 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:33.999+0000 7fd0010b1640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe80054c0 0x7fcffc1cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff3fff640 1 -- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe8004900 msgr2=0x7fcffc1cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff3fff640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe8004900 0x7fcffc1cf3f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4007bf0 tx=0x7fcfe40031d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 1 -- 10.20.193.78:0/2386357579 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe401b070 con 0x7fcfe8004900 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: dump: 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: _finish_auth 0 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: _check_auth_tickets 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:34.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 1 -- 10.20.193.78:0/2386357579 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe4017910 con 0x7fcfe8004900 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 1 -- 10.20.193.78:0/2386357579 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4017ce0 con 0x7fcfe8004900 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 10 monclient: dump: 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff2ffd640 4 set_mon_vals no callback set 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 5 monclient: authenticate success, global_id 84102 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 10 monclient: _renew_subs 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc0a1c10 con 0x7fcfe8004900 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 10 monclient: _renew_subs 2026-02-17T05:51:34.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc134b90 con 0x7fcfe8004900 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 1 -- 10.20.193.78:0/2386357579 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fcfe40163c0 con 0x7fcfe8004900 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc136760 con 0x7fcfe8004900 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcfcc03df00 0x7fcfcc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.002+0000 7fcff17fa640 1 -- 10.20.193.78:0/2386357579 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(218..218 src has 117..218) ==== 4811+0+0 (secure 0 0 0) 0x7fcfe4053570 con 0x7fcfe8004900 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.003+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcfcc03df00 0x7fcfcc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.003+0000 7fd0008b0640 10 monclient: get_auth_request con 0x7fcfcc03df00 auth_method 0 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.006+0000 7fd0008b0640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcfcc03df00 0x7fcfcc0403c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcfec007010 tx=0x7fcfec036000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.006+0000 7fcff17fa640 1 -- 10.20.193.78:0/2386357579 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcffc136760 con 0x7fcfe8004900 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.006+0000 7fcff17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:34.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.006+0000 7fcff17fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1017: finishing rollback tid 0 to trial11545420-12 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1017: finishing rollback tid 1 to trial11545420-12 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1017: finishing rollback tid 2 to trial11545420-12 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 484 (ObjNum 333 snap 83 seq_num 333) dirty exists 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1016: finishing copy_from racing read to trial11545420-46 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1020: finishing write tid 1 to trial11545420-26 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1016: finishing copy_from to trial11545420-46 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 292 (ObjNum 397 snap 102 seq_num 397) dirty exists 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1020: finishing write tid 2 to trial11545420-26 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1020: finishing write tid 3 to trial11545420-26 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1020: finishing write tid 4 to trial11545420-26 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1020: oid 26 updating version 0 to 191 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1020: oid 26 version 191 is already newer than 190 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1020: oid 26 updating version 191 to 192 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 192 (ObjNum 398 snap 103 seq_num 398) dirty exists 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1020: left oid 26 (ObjNum 398 snap 103 seq_num 398) 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1023: finishing rollback tid 1 to trial11545420-28 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 252 (ObjNum 689 snap 0 seq_num 311479502) dirty dne 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1016: done (5 left) 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1017: done (4 left) 2026-02-17T05:51:34.043 INFO:tasks.rados.rados.0.trial115.stdout:1018: done (3 left) 2026-02-17T05:51:34.044 INFO:tasks.rados.rados.0.trial115.stdout:1019: done (2 left) 2026-02-17T05:51:34.044 INFO:tasks.rados.rados.0.trial115.stdout:1020: done (1 left) 2026-02-17T05:51:34.044 INFO:tasks.rados.rados.0.trial115.stdout:1023: done (0 left) 2026-02-17T05:51:34.044 INFO:tasks.rados.rados.0.trial115.stdout:1024: copy_from oid 38 from oid 25 current snap is 103 2026-02-17T05:51:34.044 INFO:tasks.rados.rados.0.trial115.stdout:1025: rollback oid 32 current snap is 103 2026-02-17T05:51:34.044 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 97 2026-02-17T05:51:34.044 INFO:tasks.rados.rados.0.trial115.stdout:1026: write (excl) oid 17 current snap is 103 2026-02-17T05:51:34.044 INFO:tasks.rados.rados.0.trial115.stdout:1026: seq_num 399 ranges {705993=474707,1582080=612141} 2026-02-17T05:51:34.047 INFO:tasks.rados.rados.0.trial115.stdout:1026: writing trial11545420-17 from 705993 to 1180700 tid 1 2026-02-17T05:51:34.051 INFO:tasks.rados.rados.0.trial115.stdout:1026: writing trial11545420-17 from 1582080 to 2194221 tid 2 2026-02-17T05:51:34.051 INFO:tasks.rados.rados.0.trial115.stdout:1027: snap_remove snap 67 2026-02-17T05:51:34.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.124+0000 7fd002b3b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:34.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.124+0000 7fd002b3b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:34.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.124+0000 7fd002b3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:34.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:34.124+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcffc0a4370 con 0x7fcfe8004900 2026-02-17T05:51:36.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:36.999+0000 7fcfcbfff640 10 monclient: tick 2026-02-17T05:51:36.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:36.999+0000 7fcfcbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:36.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:36.999+0000 7fcfcbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:36.999+0000 7fcfcbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:37.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:37.999+0000 7fcfcbfff640 10 monclient: tick 2026-02-17T05:51:37.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:37.999+0000 7fcfcbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:37.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:37.999+0000 7fcfcbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:38.999+0000 7fcfcbfff640 10 monclient: tick 2026-02-17T05:51:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:38.999+0000 7fcfcbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:38.999+0000 7fcfcbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:39.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.090+0000 7fcff17fa640 1 -- 10.20.193.78:0/2386357579 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7fcffc0a4370 con 0x7fcfe8004900 2026-02-17T05:51:39.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.090+0000 7fcff17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:39.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.090+0000 7fcff17fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:39.090 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":56,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.093+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcfcc03df00 msgr2=0x7fcfcc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.093+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcfcc03df00 0x7fcfcc0403c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcfec007010 tx=0x7fcfec036000 comp rx=0 tx=0).stop 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.093+0000 7fd002b3b640 10 monclient: shutdown 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.093+0000 7fd002b3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.093+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe8004900 msgr2=0x7fcffc1cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.093+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe8004900 0x7fcffc1cf3f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4007bf0 tx=0x7fcfe40031d0 comp rx=0 tx=0).stop 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.094+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 shutdown_connections 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.094+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcfcc03df00 0x7fcfcc0403c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.094+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcfe80054c0 0x7fcffc1cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.094+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcfe8004900 0x7fcffc1cf3f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.094+0000 7fd002b3b640 1 --2- 10.20.193.78:0/2386357579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcfe8003b50 0x7fcffc1ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.094+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 >> 10.20.193.78:0/2386357579 conn(0x7fcffc0aae40 msgr2=0x7fcffc06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.094+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 shutdown_connections 2026-02-17T05:51:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:39.094+0000 7fd002b3b640 1 -- 10.20.193.78:0/2386357579 wait complete. 2026-02-17T05:51:39.147 INFO:tasks.rados.rados.0.trial115.stdout:1027: done (3 left) 2026-02-17T05:51:39.147 INFO:tasks.rados.rados.0.trial115.stdout:1028: snap_remove snap 75 2026-02-17T05:51:42.036 INFO:tasks.rados.rados.0.trial115.stdout:1028: done (3 left) 2026-02-17T05:51:42.036 INFO:tasks.rados.rados.0.trial115.stdout:1029: rollback oid 8 current snap is 103 2026-02-17T05:51:42.036 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 8 to 82 2026-02-17T05:51:42.036 INFO:tasks.rados.rados.0.trial115.stdout:1024: finishing copy_from to trial11545420-38 2026-02-17T05:51:42.036 INFO:tasks.rados.rados.0.trial115.stdout:1024: got expected ENOENT (src dne) 2026-02-17T05:51:42.036 INFO:tasks.rados.rados.0.trial115.stdout:1024: finishing copy_from racing read to trial11545420-38 2026-02-17T05:51:42.036 INFO:tasks.rados.rados.0.trial115.stdout:1025: finishing rollback tid 1 to trial11545420-32 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 270 (ObjNum 193 snap 0 seq_num 106) dirty dne 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1026: finishing write tid 1 to trial11545420-17 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1026: finishing write tid 2 to trial11545420-17 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1026: finishing write tid 3 to trial11545420-17 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1026: finishing write tid 4 to trial11545420-17 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1026: oid 17 updating version 0 to 163 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1026: oid 17 version 163 is already newer than 162 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1026: oid 17 updating version 163 to 164 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 164 (ObjNum 399 snap 103 seq_num 399) dirty exists 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1026: left oid 17 (ObjNum 399 snap 103 seq_num 399) 2026-02-17T05:51:42.037 INFO:tasks.rados.rados.0.trial115.stdout:1024: done (3 left) 2026-02-17T05:51:42.038 INFO:tasks.rados.rados.0.trial115.stdout:1025: done (2 left) 2026-02-17T05:51:42.038 INFO:tasks.rados.rados.0.trial115.stdout:1026: done (1 left) 2026-02-17T05:51:42.038 INFO:tasks.rados.rados.0.trial115.stdout:1030: delete oid 38 current snap is 103 2026-02-17T05:51:42.039 INFO:tasks.rados.rados.0.trial115.stdout:1029: finishing rollback tid 0 to trial11545420-8 2026-02-17T05:51:42.039 INFO:tasks.rados.rados.0.trial115.stdout:1029: finishing rollback tid 1 to trial11545420-8 2026-02-17T05:51:42.039 INFO:tasks.rados.rados.0.trial115.stdout:1029: finishing rollback tid 2 to trial11545420-8 2026-02-17T05:51:42.040 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 186 (ObjNum 254 snap 50 seq_num 254) dirty exists 2026-02-17T05:51:42.046 INFO:tasks.rados.rados.0.trial115.stdout:1029: done (1 left) 2026-02-17T05:51:42.046 INFO:tasks.rados.rados.0.trial115.stdout:1030: done (0 left) 2026-02-17T05:51:42.046 INFO:tasks.rados.rados.0.trial115.stdout:1031: write oid 40 current snap is 103 2026-02-17T05:51:42.046 INFO:tasks.rados.rados.0.trial115.stdout:1031: seq_num 401 ranges {739607=625205,1982691=499457} 2026-02-17T05:51:42.051 INFO:tasks.rados.rados.0.trial115.stdout:1031: writing trial11545420-40 from 739607 to 1364812 tid 1 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1031: writing trial11545420-40 from 1982691 to 2482148 tid 2 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1032: read oid 3 snap -1 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1032: expect (ObjNum 368 snap 97 seq_num 368) 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1033: read oid 39 snap -1 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1033: expect (ObjNum 362 snap 96 seq_num 362) 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1034: read oid 43 snap -1 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1034: expect (ObjNum 302 snap 72 seq_num 302) 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1035: write oid 47 current snap is 103 2026-02-17T05:51:42.056 INFO:tasks.rados.rados.0.trial115.stdout:1035: seq_num 402 ranges {756414=700454,2041173=715241} 2026-02-17T05:51:42.061 INFO:tasks.rados.rados.0.trial115.stdout:1035: writing trial11545420-47 from 756414 to 1456868 tid 1 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1035: writing trial11545420-47 from 2041173 to 2756414 tid 2 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1036: read oid 38 snap -1 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1036: expect deleted 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1037: read oid 29 snap -1 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1037: expect (ObjNum 389 snap 101 seq_num 389) 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1038: copy_from oid 23 from oid 48 current snap is 103 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1031: finishing write tid 1 to trial11545420-40 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1031: finishing write tid 2 to trial11545420-40 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1031: finishing write tid 3 to trial11545420-40 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1031: finishing write tid 4 to trial11545420-40 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1031: oid 40 updating version 0 to 293 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1031: oid 40 updating version 293 to 294 2026-02-17T05:51:42.066 INFO:tasks.rados.rados.0.trial115.stdout:1031: oid 40 updating version 294 to 295 2026-02-17T05:51:42.067 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 295 (ObjNum 401 snap 103 seq_num 401) dirty exists 2026-02-17T05:51:42.067 INFO:tasks.rados.rados.0.trial115.stdout:1031: left oid 40 (ObjNum 401 snap 103 seq_num 401) 2026-02-17T05:51:42.067 INFO:tasks.rados.rados.0.trial115.stdout:1033: expect (ObjNum 362 snap 96 seq_num 362) 2026-02-17T05:51:42.104 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:42.137 INFO:tasks.rados.rados.0.trial115.stdout:1032: expect (ObjNum 368 snap 97 seq_num 368) 2026-02-17T05:51:42.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 10 monclient: get_monmap_and_config 2026-02-17T05:51:42.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 10 monclient: monmap: 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:42.173496+0000 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:42.173496+0000 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:42.175 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 1 Processor -- start 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 1 -- start start 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 10 monclient: init 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f335806c890 0x7f33580a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 10 monclient(hunting): picked mon.noname-c con 0x7f335806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f335806cc70 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.173+0000 7f335d7cc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f335813a240 0x7f335813c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f335d7cc640 10 monclient(hunting): picked mon.noname-a con 0x7f335813a240 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f335d7cc640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3358074150 con 0x7f335806cc70 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f335d7cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f335d7cc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f335809ef70 con 0x7f335813a240 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f335d7cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f335d7cc640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f335809f250 con 0x7f335806c890 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f335d7cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f335d7cc640 10 monclient(hunting): authenticate will time out at 947.727023s 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f335813a240 0x7f335813c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33567fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f335806cc70 0x7f335806d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41206/0 (socket says 10.20.193.78:41206) 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33567fc640 1 -- 10.20.193.78:0/4196774449 learned_addr learned my addr 10.20.193.78:0/4196774449 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient(hunting): get_auth_request con 0x7f335813a240 auth_method 0 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient(hunting): handle_auth_done global_id 84105 payload 274 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f3356ffd640 1 --2- 10.20.193.78:0/4196774449 >> v2:10.20.193.115:3300/0 conn(0x7f335806c890 0x7f33580a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 1 -- 10.20.193.78:0/4196774449 >> v2:10.20.193.115:3300/0 conn(0x7f335806c890 msgr2=0x7f33580a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 1 --2- 10.20.193.78:0/4196774449 >> v2:10.20.193.115:3300/0 conn(0x7f335806c890 0x7f33580a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 1 -- 10.20.193.78:0/4196774449 >> v1:10.20.193.78:6789/0 conn(0x7f335806cc70 legacy=0x7f335806d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 1 monclient: found mon.noname-a 2026-02-17T05:51:42.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f33577fe640 1 -- 10.20.193.78:0/4196774449 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335809df40 con 0x7f335813a240 2026-02-17T05:51:42.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f3356ffd640 10 monclient: get_auth_request con 0x7f335806c890 auth_method 0 2026-02-17T05:51:42.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.174+0000 7f3356ffd640 1 --2- 10.20.193.78:0/4196774449 >> v2:10.20.193.115:3300/0 conn(0x7f335806c890 0x7f33580a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:51:42.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.176+0000 7f33577fe640 1 -- 10.20.193.78:0/4196774449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335813a240 msgr2=0x7f335813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.176+0000 7f33577fe640 1 --2- 10.20.193.78:0/4196774449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335813a240 0x7f335813c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f335806a480 tx=0x7f334802f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571e9eddda26e63 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 1 -- 10.20.193.78:0/4196774449 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334803c070 con 0x7f335813a240 2026-02-17T05:51:42.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:42.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: dump: 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 5 monclient: authenticate success, global_id 84105 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 1 -- 10.20.193.78:0/4196774449 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3348037640 con 0x7f335813a240 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 1 -- 10.20.193.78:0/4196774449 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348037a10 con 0x7f335813a240 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 4 set_mon_vals no callback set 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f3355ffb640 10 monclient: dump: 2026-02-17T05:51:42.188 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 10 monclient: shutdown 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 -- 10.20.193.78:0/4196774449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335813a240 msgr2=0x7f335813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 --2- 10.20.193.78:0/4196774449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335813a240 0x7f335813c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f335806a480 tx=0x7f334802f1c0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 -- 10.20.193.78:0/4196774449 shutdown_connections 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 --2- 10.20.193.78:0/4196774449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335813a240 0x7f335813c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 --2- 10.20.193.78:0/4196774449 >> v2:10.20.193.115:3300/0 conn(0x7f335806c890 0x7f33580a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 -- 10.20.193.78:0/4196774449 >> 10.20.193.78:0/4196774449 conn(0x7f33580a6620 msgr2=0x7f33580a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 -- 10.20.193.78:0/4196774449 shutdown_connections 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 -- 10.20.193.78:0/4196774449 wait complete. 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 10 monclient: monmap: 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:42.190 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 Processor -- start 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 1 -- start start 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.177+0000 7f335d7cc640 10 monclient: init 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f335806c890 0x7f3358135370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient(hunting): picked mon.noname-b con 0x7f335806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335806cc70 0x7f33581358b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient(hunting): picked mon.noname-a con 0x7f335806cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f335813a240 0x7f335811edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient(hunting): picked mon.noname-c con 0x7f335813a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f335813ccf0 con 0x7f335806cc70 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f335813cb70 con 0x7f335806c890 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f335813ce70 con 0x7f335813a240 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f335d7cc640 10 monclient(hunting): authenticate will time out at 947.731203s 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335806cc70 0x7f33581358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335806cc70 0x7f33581358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38416/0 (socket says 10.20.193.78:38416) 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 -- 10.20.193.78:0/3113232991 learned_addr learned my addr 10.20.193.78:0/3113232991 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient(hunting): get_auth_request con 0x7f335806cc70 auth_method 0 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33577fe640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f335813a240 0x7f335811edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33577fe640 10 monclient(hunting): get_auth_request con 0x7f335813a240 auth_method 0 2026-02-17T05:51:42.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33577fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient(hunting): handle_auth_done global_id 84108 payload 824 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f3356ffd640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f335806c890 0x7f3358135370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 -- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f335813a240 msgr2=0x7f335811edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f335813a240 0x7f335811edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 -- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f335806c890 msgr2=0x7f3358135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f335806c890 0x7f3358135370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 monclient: found mon.noname-a 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f33567fc640 1 -- 10.20.193.78:0/3113232991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3358122d30 con 0x7f335806cc70 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f3356ffd640 10 monclient: get_auth_request con 0x7f335806c890 auth_method 0 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.178+0000 7f3356ffd640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f335806c890 0x7f3358135370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33567fc640 1 -- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335806cc70 msgr2=0x7f33581358b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33567fc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335806cc70 0x7f33581358b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f334000c750 tx=0x7f334000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 1 -- 10.20.193.78:0/3113232991 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340005de0 con 0x7f335806cc70 2026-02-17T05:51:42.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: dump: 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.196 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 1 -- 10.20.193.78:0/3113232991 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3340007070 con 0x7f335806cc70 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 1 -- 10.20.193.78:0/3113232991 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340007440 con 0x7f335806cc70 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33377fe640 10 monclient: dump: 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.197 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f33557fa640 4 set_mon_vals no callback set 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f335d7cc640 5 monclient: authenticate success, global_id 84108 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f335d7cc640 10 monclient: _renew_subs 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f335d7cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3358073150 con 0x7f335806cc70 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f335d7cc640 10 monclient: _renew_subs 2026-02-17T05:51:42.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f335d7cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.180+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335809d810 con 0x7f335806cc70 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f33377fe640 1 -- 10.20.193.78:0/3113232991 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3340007620 con 0x7f335806cc70 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f335d7cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f335d7cc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f335d7cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3358074980 con 0x7f335806cc70 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f33377fe640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f332403df00 0x7f33240403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f33377fe640 1 -- 10.20.193.78:0/3113232991 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(220..220 src has 117..220) ==== 4843+0+0 (secure 0 0 0) 0x7f33400525b0 con 0x7f335806cc70 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f3356ffd640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f332403df00 0x7f33240403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.181+0000 7f3356ffd640 10 monclient: get_auth_request con 0x7f332403df00 auth_method 0 2026-02-17T05:51:42.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.183+0000 7f33377fe640 1 -- 10.20.193.78:0/3113232991 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f334000a8d0 con 0x7f335806cc70 2026-02-17T05:51:42.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.183+0000 7f33377fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.183+0000 7f33377fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:42.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.184+0000 7f3356ffd640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f332403df00 0x7f33240403c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f334c006fd0 tx=0x7f334c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.200 INFO:tasks.rados.rados.0.trial115.stdout:1034: expect (ObjNum 302 snap 72 seq_num 302) 2026-02-17T05:51:42.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.298+0000 7f335d7cc640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:42.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.298+0000 7f335d7cc640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:42.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.298+0000 7f335d7cc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.298+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3358121320 con 0x7f335806cc70 2026-02-17T05:51:42.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.298+0000 7f33377fe640 1 -- 10.20.193.78:0/3113232991 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f334000a5e0 con 0x7f335806cc70 2026-02-17T05:51:42.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.298+0000 7f33377fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:42.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.298+0000 7f33377fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:42.298 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f332403df00 msgr2=0x7f33240403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f332403df00 0x7f33240403c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f334c006fd0 tx=0x7f334c008040 comp rx=0 tx=0).stop 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 10 monclient: shutdown 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335806cc70 msgr2=0x7f33581358b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335806cc70 0x7f33581358b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f334000c750 tx=0x7f334000cc20 comp rx=0 tx=0).stop 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 shutdown_connections 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f335813a240 0x7f335811edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f335806cc70 0x7f33581358b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f332403df00 0x7f33240403c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 --2- 10.20.193.78:0/3113232991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f335806c890 0x7f3358135370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 >> 10.20.193.78:0/3113232991 conn(0x7f33580a6620 msgr2=0x7f3358139560 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 shutdown_connections 2026-02-17T05:51:42.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.299+0000 7f335d7cc640 1 -- 10.20.193.78:0/3113232991 wait complete. 2026-02-17T05:51:42.307 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:51:42.307 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:51:42.307 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:51:42.373 INFO:tasks.rados.rados.0.trial115.stdout:1035: finishing write tid 1 to trial11545420-47 2026-02-17T05:51:42.374 INFO:tasks.rados.rados.0.trial115.stdout:1031: done (7 left) 2026-02-17T05:51:42.374 INFO:tasks.rados.rados.0.trial115.stdout:1032: done (6 left) 2026-02-17T05:51:42.374 INFO:tasks.rados.rados.0.trial115.stdout:1033: done (5 left) 2026-02-17T05:51:42.374 INFO:tasks.rados.rados.0.trial115.stdout:1034: done (4 left) 2026-02-17T05:51:42.374 INFO:tasks.rados.rados.0.trial115.stdout:1039: snap_remove snap 99 2026-02-17T05:51:42.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.377+0000 7f64460bf640 10 monclient: get_monmap_and_config 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.377+0000 7f64460bf640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.377+0000 7f64460bf640 10 monclient: monmap: 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:42.377601+0000 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:42.377601+0000 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:42.376 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:42.377 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:42.377 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.377+0000 7f64460bf640 1 Processor -- start 2026-02-17T05:51:42.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.377+0000 7f64460bf640 1 -- start start 2026-02-17T05:51:42.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.377+0000 7f64460bf640 10 monclient: init 2026-02-17T05:51:42.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f644006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f644009d430 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644009bf40 0x7f6440140d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 10 monclient(hunting): picked mon.noname-b con 0x7f644009bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6440073600 con 0x7f644006c890 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644006dab0 con 0x7f644009d430 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f644009ca50 con 0x7f644009bf40 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f64460bf640 10 monclient(hunting): authenticate will time out at 947.931201s 2026-02-17T05:51:42.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643f7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f644006c890 0x7f644006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41212/0 (socket says 10.20.193.78:41212) 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643f7fe640 1 -- 10.20.193.78:0/860536368 learned_addr learned my addr 10.20.193.78:0/860536368 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 1 --2- 10.20.193.78:0/860536368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644009bf40 0x7f6440140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient(hunting): get_auth_request con 0x7f644009bf40 auth_method 0 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient(hunting): handle_auth_done global_id 74195 payload 274 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 1 -- 10.20.193.78:0/860536368 >> v1:10.20.193.115:6789/0 conn(0x7f644009d430 legacy=0x7f644013d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 1 -- 10.20.193.78:0/860536368 >> v1:10.20.193.78:6789/0 conn(0x7f644006c890 legacy=0x7f644006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 1 monclient: found mon.noname-b 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.378+0000 7f643ffff640 1 -- 10.20.193.78:0/860536368 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644009c770 con 0x7f644009bf40 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.380+0000 7f643ffff640 1 -- 10.20.193.78:0/860536368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644009bf40 msgr2=0x7f6440140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.380+0000 7f643ffff640 1 --2- 10.20.193.78:0/860536368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644009bf40 0x7f6440140d00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6434009dc0 tx=0x7f643402f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d157b62de5979a85 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 1 -- 10.20.193.78:0/860536368 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643403c070 con 0x7f644009bf40 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: dump: 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.388 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 1 -- 10.20.193.78:0/860536368 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6434037640 con 0x7f644009bf40 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 5 monclient: authenticate success, global_id 74195 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 1 -- 10.20.193.78:0/860536368 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434037a10 con 0x7f644009bf40 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 4 set_mon_vals no callback set 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f643e7fc640 10 monclient: dump: 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.390 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 10 monclient: shutdown 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 1 -- 10.20.193.78:0/860536368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644009bf40 msgr2=0x7f6440140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 1 --2- 10.20.193.78:0/860536368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644009bf40 0x7f6440140d00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6434009dc0 tx=0x7f643402f1c0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 1 -- 10.20.193.78:0/860536368 shutdown_connections 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 1 --2- 10.20.193.78:0/860536368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644009bf40 0x7f6440140d00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 1 -- 10.20.193.78:0/860536368 >> 10.20.193.78:0/860536368 conn(0x7f64400a6620 msgr2=0x7f64400a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 1 -- 10.20.193.78:0/860536368 shutdown_connections 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 1 -- 10.20.193.78:0/860536368 wait complete. 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.381+0000 7f64460bf640 10 monclient: monmap: 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.392 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 1 Processor -- start 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 1 -- start start 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient: init 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644006c890 0x7f64401cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient(hunting): picked mon.noname-c con 0x7f644006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f644009bf40 0x7f644013f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient(hunting): picked mon.noname-b con 0x7f644009bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f644009d430 0x7f644013fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient(hunting): picked mon.noname-a con 0x7f644009d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f644006d500 con 0x7f644009d430 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f644006d380 con 0x7f644009bf40 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f644006d680 con 0x7f644006c890 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f64460bf640 10 monclient(hunting): authenticate will time out at 947.935585s 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644006c890 0x7f64401cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644006c890 0x7f64401cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45936/0 (socket says 10.20.193.78:45936) 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f644009d430 0x7f644013fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 -- 10.20.193.78:0/149361254 learned_addr learned my addr 10.20.193.78:0/149361254 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient(hunting): get_auth_request con 0x7f644006c890 auth_method 0 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643ffff640 10 monclient(hunting): get_auth_request con 0x7f644009d430 auth_method 0 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient(hunting): handle_auth_done global_id 74198 payload 824 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f644009bf40 msgr2=0x7f644013f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f644009bf40 0x7f644013f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f644009d430 msgr2=0x7f644013fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f644009d430 0x7f644013fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 monclient: found mon.noname-c 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:42.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.382+0000 7f643f7fe640 1 -- 10.20.193.78:0/149361254 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644013b1c0 con 0x7f644006c890 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.384+0000 7f643f7fe640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644006c890 msgr2=0x7f64401cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.384+0000 7f643f7fe640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644006c890 0x7f64401cf100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f642c00b530 tx=0x7f642c00ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c00fa60 con 0x7f644006c890 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: dump: 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.397 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c00fc00 con 0x7f644006c890 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c017400 con 0x7f644006c890 2026-02-17T05:51:42.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 10 monclient: dump: 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643dffb640 4 set_mon_vals no callback set 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 5 monclient: authenticate success, global_id 74198 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 10 monclient: _renew_subs 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644009d810 con 0x7f644006c890 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 10 monclient: _renew_subs 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f644013ac00 con 0x7f644006c890 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134250 0x7f6440134630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f644013ae40 con 0x7f6440134250 2026-02-17T05:51:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f642c00d3e0 con 0x7f644006c890 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134250 0x7f6440134630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 10 monclient: get_auth_request con 0x7f6440134250 auth_method 0 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f641003deb0 0x7f6410040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(220..220 src has 117..220) ==== 4843+0+0 (secure 0 0 0) 0x7f642c0528d0 con 0x7f644006c890 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 10 monclient(hunting): handle_auth_done global_id 84114 payload 824 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.385+0000 7f643effd640 1 -- 10.20.193.78:0/149361254 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64400a3ef0 con 0x7f6440134250 2026-02-17T05:51:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.386+0000 7f643ffff640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f641003deb0 0x7f6410040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.386+0000 7f643ffff640 10 monclient: get_auth_request con 0x7f641003deb0 auth_method 0 2026-02-17T05:51:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.387+0000 7f643effd640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134250 msgr2=0x7f6440134630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.387+0000 7f643effd640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134250 0x7f6440134630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6440139c00 tx=0x7f643000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430011a80 con 0x7f6440134250 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 10 monclient: dump: 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.405 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6430017070 con 0x7f6440134250 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134250 msgr2=0x7f6440134630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134250 0x7f6440134630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6440139c00 tx=0x7f643000c800 comp rx=0 tx=0).stop 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134250 msgr2=0x7f6440134630 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f641bfff640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134250 0x7f6440134630 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6440139c00 tx=0x7f643000c800 comp rx=0 tx=0).stop 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.388+0000 7f643ffff640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f641003deb0 0x7f6410040370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f644006a480 tx=0x7f6434033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f64460bf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:51:42.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f64460bf640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134b70 0x7f64401da450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f64460bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f644006cc70 con 0x7f6440134b70 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134b70 0x7f64401da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 10 monclient: get_auth_request con 0x7f6440134b70 auth_method 0 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 10 monclient(hunting): handle_auth_done global_id 84117 payload 824 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.391+0000 7f643effd640 1 -- 10.20.193.78:0/149361254 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6440139c50 con 0x7f6440134b70 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.393+0000 7f643effd640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134b70 msgr2=0x7f64401da450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.393+0000 7f643effd640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134b70 0x7f64401da450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6440139c00 tx=0x7f6430003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.393+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643001cb00 con 0x7f6440134b70 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.393+0000 7f641bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 10 monclient: dump: 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.408 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f64300068d0 con 0x7f6440134b70 2026-02-17T05:51:42.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:51:42.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:42.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134b70 msgr2=0x7f64401da450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134b70 0x7f64401da450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6440139c00 tx=0x7f6430003860 comp rx=0 tx=0).stop 2026-02-17T05:51:42.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134b70 msgr2=0x7f64401da450 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:42.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f641bfff640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6440134b70 0x7f64401da450 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6440139c00 tx=0x7f6430003860 comp rx=0 tx=0).stop 2026-02-17T05:51:42.409 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 60, 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 26975, 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:51:42.410 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:51:42.411 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:51:42.412 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:51:42.413 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:51:42.414 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.415 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:51:42.416 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:51:42.417 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.417 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.417 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.417 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.417 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.417 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:42.417 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:42.417 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.418 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.418 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.418 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.418 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:51:42.418 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.418 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.418 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.418 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.419 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.420 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.421 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:51:42.422 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:51:42.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f641003deb0 msgr2=0x7f6410040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f641003deb0 0x7f6410040370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f644006a480 tx=0x7f6434033000 comp rx=0 tx=0).stop 2026-02-17T05:51:42.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 10 monclient: shutdown 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644006c890 msgr2=0x7f64401cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f643ffff640 1 -- 10.20.193.78:0/149361254 reap_dead start 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644006c890 0x7f64401cf100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f642c00b530 tx=0x7f642c00ba00 comp rx=0 tx=0).stop 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 shutdown_connections 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 --2- 10.20.193.78:0/149361254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f644006c890 0x7f64401cf100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 >> 10.20.193.78:0/149361254 conn(0x7f64400a6620 msgr2=0x7f64400a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 shutdown_connections 2026-02-17T05:51:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.394+0000 7f64460bf640 1 -- 10.20.193.78:0/149361254 wait complete. 2026-02-17T05:51:42.425 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:51:42.425 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:51:42.425 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.494+0000 7fceb9cda640 10 monclient: get_monmap_and_config 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.494+0000 7fceb9cda640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.494+0000 7fceb9cda640 10 monclient: monmap: 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:42.494812+0000 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:42.494812+0000 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:42.493 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:42.494 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:42.494 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:42.494 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:42.494 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 1 Processor -- start 2026-02-17T05:51:42.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 1 -- start start 2026-02-17T05:51:42.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient: init 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fceb406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fceb409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406d380 0x7fceb413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient(hunting): picked mon.noname-b con 0x7fceb406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb406daf0 con 0x7fceb409c4a0 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb409cc80 con 0x7fceb406c890 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fceb40a0880 con 0x7fceb406d380 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb9cda640 10 monclient(hunting): authenticate will time out at 948.048414s 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb2ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fceb409c4a0 0x7fceb413b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41224/0 (socket says 10.20.193.78:41224) 2026-02-17T05:51:42.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406d380 0x7fceb413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb2ffd640 1 -- 10.20.193.78:0/3156027787 learned_addr learned my addr 10.20.193.78:0/3156027787 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient(hunting): get_auth_request con 0x7fceb406d380 auth_method 0 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient(hunting): handle_auth_done global_id 74201 payload 274 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 1 -- 10.20.193.78:0/3156027787 >> v1:10.20.193.115:6789/0 conn(0x7fceb406c890 legacy=0x7fceb406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 1 -- 10.20.193.78:0/3156027787 >> v1:10.20.193.78:6789/0 conn(0x7fceb409c4a0 legacy=0x7fceb413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 1 monclient: found mon.noname-b 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.495+0000 7fceb3fff640 1 -- 10.20.193.78:0/3156027787 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb409cf60 con 0x7fceb406d380 2026-02-17T05:51:42.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb3fff640 1 -- 10.20.193.78:0/3156027787 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406d380 msgr2=0x7fceb413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb3fff640 1 --2- 10.20.193.78:0/3156027787 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406d380 0x7fceb413ead0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009dc0 tx=0x7fcea802f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffca0ce6550793e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 1 -- 10.20.193.78:0/3156027787 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea803c070 con 0x7fceb406d380 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: dump: 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 1 -- 10.20.193.78:0/3156027787 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea8037640 con 0x7fceb406d380 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 1 -- 10.20.193.78:0/3156027787 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8037a10 con 0x7fceb406d380 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 5 monclient: authenticate success, global_id 74201 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 4 set_mon_vals no callback set 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb27fc640 10 monclient: dump: 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.512 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 10 monclient: shutdown 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 1 -- 10.20.193.78:0/3156027787 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406d380 msgr2=0x7fceb413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 1 --2- 10.20.193.78:0/3156027787 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406d380 0x7fceb413ead0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009dc0 tx=0x7fcea802f1c0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 1 -- 10.20.193.78:0/3156027787 shutdown_connections 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 1 --2- 10.20.193.78:0/3156027787 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406d380 0x7fceb413ead0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 1 -- 10.20.193.78:0/3156027787 >> 10.20.193.78:0/3156027787 conn(0x7fceb40a6620 msgr2=0x7fceb40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 1 -- 10.20.193.78:0/3156027787 shutdown_connections 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.498+0000 7fceb9cda640 1 -- 10.20.193.78:0/3156027787 wait complete. 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient: monmap: 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:42.513 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 1 Processor -- start 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 1 -- start start 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient: init 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406c890 0x7fceb41ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient(hunting): picked mon.noname-c con 0x7fceb406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fceb406d380 0x7fceb41cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient(hunting): picked mon.noname-a con 0x7fceb406d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fceb409c4a0 0x7fceb41cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient(hunting): picked mon.noname-b con 0x7fceb409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fceb409bfa0 con 0x7fceb406d380 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fceb409be20 con 0x7fceb409c4a0 2026-02-17T05:51:42.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fceb413f010 con 0x7fceb406c890 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb9cda640 10 monclient(hunting): authenticate will time out at 948.052511s 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406c890 0x7fceb41ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fceb406d380 0x7fceb41cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406c890 0x7fceb41ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45948/0 (socket says 10.20.193.78:45948) 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 -- 10.20.193.78:0/2784340288 learned_addr learned my addr 10.20.193.78:0/2784340288 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient(hunting): get_auth_request con 0x7fceb406c890 auth_method 0 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb2ffd640 10 monclient(hunting): get_auth_request con 0x7fceb406d380 auth_method 0 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb3fff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fceb409c4a0 0x7fceb41cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient(hunting): handle_auth_done global_id 74204 payload 824 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fceb409c4a0 msgr2=0x7fceb41cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fceb409c4a0 0x7fceb41cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fceb406d380 msgr2=0x7fceb41cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fceb406d380 0x7fceb41cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 monclient: found mon.noname-c 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb37fe640 1 -- 10.20.193.78:0/2784340288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb409d810 con 0x7fceb406c890 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb3fff640 10 monclient: get_auth_request con 0x7fceb409c4a0 auth_method 0 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.499+0000 7fceb3fff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fceb409c4a0 0x7fceb41cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.501+0000 7fceb37fe640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406c890 msgr2=0x7fceb41ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.501+0000 7fceb37fe640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406c890 0x7fceb41ccfc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c5d0 tx=0x7fcea000caa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.501+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0016020 con 0x7fceb406c890 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: dump: 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.516 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea0012a60 con 0x7fceb406c890 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 5 monclient: authenticate success, global_id 74204 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 10 monclient: _renew_subs 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb411f980 con 0x7fceb406c890 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0012e50 con 0x7fceb406c890 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb1ffb640 4 set_mon_vals no callback set 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 10 monclient: dump: 2026-02-17T05:51:42.517 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 10 monclient: _renew_subs 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb4072960 con 0x7fceb406c890 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb4136200 0x7fceb4139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fceb406fdd0 con 0x7fceb4136200 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb4136200 0x7fceb4139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fcea001b400 con 0x7fceb406c890 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 10 monclient: get_auth_request con 0x7fceb4136200 auth_method 0 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_done global_id 74207 payload 824 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fceb2ffd640 1 -- 10.20.193.78:0/2784340288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fceb413ed10 con 0x7fceb4136200 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fce8403e0a0 0x7fce84040560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.502+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(220..220 src has 117..220) ==== 4843+0+0 (secure 0 0 0) 0x7fcea0052860 con 0x7fceb406c890 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.503+0000 7fceb3fff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fce8403e0a0 0x7fce84040560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.503+0000 7fceb3fff640 10 monclient: get_auth_request con 0x7fce8403e0a0 auth_method 0 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.504+0000 7fceb2ffd640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb4136200 msgr2=0x7fceb4139390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.504+0000 7fceb2ffd640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb4136200 0x7fceb4139390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fceb413efe0 tx=0x7fcea400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea400cbc0 con 0x7fceb4136200 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 10 monclient: dump: 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fcea4019070 con 0x7fceb4136200 2026-02-17T05:51:42.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb4136200 msgr2=0x7fceb4139390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb4136200 0x7fceb4139390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fceb413efe0 tx=0x7fcea400ef10 comp rx=0 tx=0).stop 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb4136200 msgr2=0x7fceb4139390 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fce8ffff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb4136200 0x7fceb4139390 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fceb413efe0 tx=0x7fcea400ef10 comp rx=0 tx=0).stop 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.505+0000 7fceb3fff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fce8403e0a0 0x7fce84040560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fceb409c980 tx=0x7fcea8033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb9cda640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb9cda640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb41365e0 0x7fceb41d8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb9cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fceb406cc70 con 0x7fceb41365e0 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb41365e0 0x7fceb41d8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 10 monclient: get_auth_request con 0x7fceb41365e0 auth_method 0 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 10 monclient(hunting): handle_auth_done global_id 74210 payload 824 2026-02-17T05:51:42.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.508+0000 7fceb2ffd640 1 -- 10.20.193.78:0/2784340288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fceb413aa50 con 0x7fceb41365e0 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fceb2ffd640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb41365e0 msgr2=0x7fceb41d8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fceb2ffd640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb41365e0 0x7fceb41d8520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fceb413efe0 tx=0x7fcea4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea401e230 con 0x7fceb41365e0 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 10 monclient: dump: 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.521 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4707 (secure 0 0 0) 0x7fcea401e230 con 0x7fceb41365e0 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb41365e0 msgr2=0x7fceb41d8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb41365e0 0x7fceb41d8520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fceb413efe0 tx=0x7fcea4004a40 comp rx=0 tx=0).stop 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb41365e0 msgr2=0x7fceb41d8520 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.510+0000 7fce8ffff640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb41365e0 0x7fceb41d8520 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fceb413efe0 tx=0x7fcea4004a40 comp rx=0 tx=0).stop 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 60, 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 74402, 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:51:42.522 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.523 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.524 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:51:42.525 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:42.526 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:42.527 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:51:42.528 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fce8403e0a0 msgr2=0x7fce84040560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fce8403e0a0 0x7fce84040560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fceb409c980 tx=0x7fcea8033000 comp rx=0 tx=0).stop 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 10 monclient: shutdown 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406c890 msgr2=0x7fceb41ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 1 --2- 10.20.193.78:0/2784340288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fceb406c890 0x7fceb41ccfc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c5d0 tx=0x7fcea000caa0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb3fff640 1 -- 10.20.193.78:0/2784340288 reap_dead start 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb3fff640 1 -- 10.20.193.78:0/2784340288 reap_dead start 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 shutdown_connections 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 >> 10.20.193.78:0/2784340288 conn(0x7fceb40a6620 msgr2=0x7fceb40715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 shutdown_connections 2026-02-17T05:51:42.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.511+0000 7fceb9cda640 1 -- 10.20.193.78:0/2784340288 wait complete. 2026-02-17T05:51:42.529 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:51:42.530 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:51:42.530 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:51:42.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.599+0000 7f341aeb8640 10 monclient: get_monmap_and_config 2026-02-17T05:51:42.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.599+0000 7f341aeb8640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.599+0000 7f341aeb8640 10 monclient: monmap: 2026-02-17T05:51:42.599 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:42.599628+0000 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:42.599628+0000 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.599+0000 7f341aeb8640 1 Processor -- start 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.599+0000 7f341aeb8640 1 -- start start 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.599+0000 7f341aeb8640 10 monclient: init 2026-02-17T05:51:42.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f341406c890 0x7f34140a3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 10 monclient(hunting): picked mon.noname-c con 0x7f341406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f341406cc70 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f341413d8e0 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f341409c0c0 con 0x7f341413d8e0 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f341409c680 con 0x7f341406cc70 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f341409e7a0 con 0x7f341406c890 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341aeb8640 10 monclient(hunting): authenticate will time out at 948.153253s 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341942e640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f341413d8e0 0x7f341413fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41236/0 (socket says 10.20.193.78:41236) 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f341942e640 1 -- 10.20.193.78:0/3082401847 learned_addr learned my addr 10.20.193.78:0/3082401847 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 1 --2- 10.20.193.78:0/3082401847 >> v2:10.20.193.115:3300/0 conn(0x7f341406c890 0x7f34140a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient(hunting): get_auth_request con 0x7f341406c890 auth_method 0 2026-02-17T05:51:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient(hunting): handle_auth_done global_id 44401 payload 274 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 1 -- 10.20.193.78:0/3082401847 >> v1:10.20.193.115:6789/0 conn(0x7f341406cc70 legacy=0x7f341406d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 1 -- 10.20.193.78:0/3082401847 >> v1:10.20.193.78:6789/0 conn(0x7f341413d8e0 legacy=0x7f341413fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 1 monclient: found mon.noname-c 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:51:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.600+0000 7f3418c2d640 1 -- 10.20.193.78:0/3082401847 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34140a39c0 con 0x7f341406c890 2026-02-17T05:51:42.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f3418c2d640 1 -- 10.20.193.78:0/3082401847 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406c890 msgr2=0x7f34140a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f3418c2d640 1 --2- 10.20.193.78:0/3082401847 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406c890 0x7f34140a3480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f34040098a0 tx=0x7f340402ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c55c57514d799c7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 1 -- 10.20.193.78:0/3082401847 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340403c070 con 0x7f341406c890 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: dump: 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 1 -- 10.20.193.78:0/3082401847 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340402fd70 con 0x7f341406c890 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 1 -- 10.20.193.78:0/3082401847 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404037660 con 0x7f341406c890 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f341aeb8640 5 monclient: authenticate success, global_id 44401 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f340b7fe640 10 monclient: dump: 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.609 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f341aeb8640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.603+0000 7f341aeb8640 4 set_mon_vals no callback set 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 10 monclient: shutdown 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 -- 10.20.193.78:0/3082401847 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406c890 msgr2=0x7f34140a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 --2- 10.20.193.78:0/3082401847 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406c890 0x7f34140a3480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f34040098a0 tx=0x7f340402ee10 comp rx=0 tx=0).stop 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 -- 10.20.193.78:0/3082401847 shutdown_connections 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 --2- 10.20.193.78:0/3082401847 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406c890 0x7f34140a3480 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 -- 10.20.193.78:0/3082401847 >> 10.20.193.78:0/3082401847 conn(0x7f34140a4230 msgr2=0x7f34140a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 -- 10.20.193.78:0/3082401847 shutdown_connections 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 -- 10.20.193.78:0/3082401847 wait complete. 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 10 monclient: monmap: 2026-02-17T05:51:42.610 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 Processor -- start 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 1 -- start start 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.604+0000 7f341aeb8640 10 monclient: init 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f341406c890 0x7f341413cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient(hunting): picked mon.noname-c con 0x7f341406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:42.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406cc70 0x7f341413d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient(hunting): picked mon.noname-b con 0x7f341406cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f341413d8e0 0x7f34140918c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient(hunting): picked mon.noname-a con 0x7f341413d8e0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34141413f0 con 0x7f341413d8e0 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f341409c3a0 con 0x7f341406cc70 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3414141570 con 0x7f341406c890 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341aeb8640 10 monclient(hunting): authenticate will time out at 948.158270s 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f341413d8e0 0x7f34140918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f3418c2d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f341406c890 0x7f341413cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f341413d8e0 0x7f34140918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38468/0 (socket says 10.20.193.78:38468) 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 -- 10.20.193.78:0/2487915661 learned_addr learned my addr 10.20.193.78:0/2487915661 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:42.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient(hunting): get_auth_request con 0x7f341413d8e0 auth_method 0 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f3418c2d640 10 monclient(hunting): get_auth_request con 0x7f341406c890 auth_method 0 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f3418c2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f3418c2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f3418c2d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient(hunting): handle_auth_done global_id 84123 payload 824 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f341406c890 msgr2=0x7f341413cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f341406c890 0x7f341413cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f340bfff640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406cc70 0x7f341413d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406cc70 msgr2=0x7f341413d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406cc70 0x7f341413d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 monclient: found mon.noname-a 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f341942e640 1 -- 10.20.193.78:0/2487915661 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3414093d20 con 0x7f341413d8e0 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f340bfff640 10 monclient: get_auth_request con 0x7f341406cc70 auth_method 0 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.605+0000 7f340bfff640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341406cc70 0x7f341413d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341942e640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f341413d8e0 msgr2=0x7f34140918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341942e640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f341413d8e0 0x7f34140918c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f341000ca40 tx=0x7f341000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341000add0 con 0x7f341413d8e0 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: dump: 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.614 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3410012400 con 0x7f341413d8e0 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34100127f0 con 0x7f341413d8e0 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341aeb8640 5 monclient: authenticate success, global_id 84123 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f340affd640 4 set_mon_vals no callback set 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f34097fa640 10 monclient: dump: 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341aeb8640 10 monclient: _renew_subs 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341aeb8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34140942e0 con 0x7f341413d8e0 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341aeb8640 10 monclient: _renew_subs 2026-02-17T05:51:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341aeb8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.607+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3414140df0 con 0x7f341413d8e0 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f341aeb8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f341aeb8640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3414091e00 0x7f34140921e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f341aeb8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3414074980 con 0x7f3414091e00 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3410012a10 con 0x7f341413d8e0 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3414091e00 0x7f34140921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 10 monclient: get_auth_request con 0x7f3414091e00 auth_method 0 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f34097fa640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33e403deb0 0x7f33e4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(220..220 src has 117..220) ==== 4843+0+0 (secure 0 0 0) 0x7f3410052d50 con 0x7f341413d8e0 2026-02-17T05:51:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 10 monclient(hunting): handle_auth_done global_id 44404 payload 824 2026-02-17T05:51:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f3418c2d640 1 -- 10.20.193.78:0/2487915661 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3414092720 con 0x7f3414091e00 2026-02-17T05:51:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f340bfff640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33e403deb0 0x7f33e4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.608+0000 7f340bfff640 10 monclient: get_auth_request con 0x7f33e403deb0 auth_method 0 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f3418c2d640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3414091e00 msgr2=0x7f34140921e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f3418c2d640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3414091e00 0x7f34140921e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3404039000 tx=0x7f34040047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f340bfff640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33e403deb0 0x7f33e4040370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3414073630 tx=0x7f33fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340403c040 con 0x7f3414091e00 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 10 monclient: dump: 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.620 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f3404037d90 con 0x7f3414091e00 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3414091e00 msgr2=0x7f34140921e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3414091e00 0x7f34140921e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3404039000 tx=0x7f34040047c0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3414091e00 msgr2=0x7f34140921e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.611+0000 7f34097fa640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3414091e00 0x7f34140921e0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3404039000 tx=0x7f34040047c0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f341aeb8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f341aeb8640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341409f8a0 0x7f34141da410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f341aeb8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f34140a3480 con 0x7f341409f8a0 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f3418c2d640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341409f8a0 0x7f34141da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f3418c2d640 10 monclient: get_auth_request con 0x7f341409f8a0 auth_method 0 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f3418c2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f3418c2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.614+0000 7f3418c2d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.615+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.615+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.615+0000 7f3418c2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.615+0000 7f3418c2d640 10 monclient(hunting): handle_auth_done global_id 44407 payload 824 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.615+0000 7f3418c2d640 1 -- 10.20.193.78:0/2487915661 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f341409a1c0 con 0x7f341409f8a0 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f3418c2d640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341409f8a0 msgr2=0x7f34141da410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f3418c2d640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341409f8a0 0x7f34141da410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3404002410 tx=0x7f3404008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404055070 con 0x7f341409f8a0 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 10 monclient: dump: 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f34040513f0 con 0x7f341409f8a0 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341409f8a0 msgr2=0x7f34141da410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341409f8a0 0x7f34141da410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3404002410 tx=0x7f3404008030 comp rx=0 tx=0).stop 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341409f8a0 msgr2=0x7f34141da410 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f34097fa640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f341409f8a0 0x7f34141da410 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3404002410 tx=0x7f3404008030 comp rx=0 tx=0).stop 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 60, 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 226263, 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:51:42.624 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.625 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.626 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:42.627 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:42.628 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.629 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:51:42.630 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.631 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.632 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:51:42.633 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:51:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33e403deb0 msgr2=0x7f33e4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33e403deb0 0x7f33e4040370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3414073630 tx=0x7f33fc009040 comp rx=0 tx=0).stop 2026-02-17T05:51:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 10 monclient: shutdown 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f341413d8e0 msgr2=0x7f34140918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 1 --2- 10.20.193.78:0/2487915661 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f341413d8e0 0x7f34140918c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f341000ca40 tx=0x7f341000cf10 comp rx=0 tx=0).stop 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341942e640 1 -- 10.20.193.78:0/2487915661 reap_dead start 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341942e640 1 -- 10.20.193.78:0/2487915661 reap_dead start 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 shutdown_connections 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 >> 10.20.193.78:0/2487915661 conn(0x7f34140a4230 msgr2=0x7f341409d340 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.617+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 shutdown_connections 2026-02-17T05:51:42.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.618+0000 7f341aeb8640 1 -- 10.20.193.78:0/2487915661 wait complete. 2026-02-17T05:51:42.635 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:51:42.635 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:51:42.635 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.703+0000 7f6cc0cda640 10 monclient: get_monmap_and_config 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.703+0000 7f6cc0cda640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.703+0000 7f6cc0cda640 10 monclient: monmap: 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:42.703491+0000 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:42.703491+0000 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.702 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:42.703 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:42.703 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:42.703 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:42.703 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:42.703 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.703+0000 7f6cc0cda640 1 Processor -- start 2026-02-17T05:51:42.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.703+0000 7f6cc0cda640 1 -- start start 2026-02-17T05:51:42.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.703+0000 7f6cc0cda640 10 monclient: init 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6cbc06c890 0x7f6cbc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient(hunting): picked mon.noname-c con 0x7f6cbc06c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6cbc09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6cbc06d1b0 0x7f6cbc135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient(hunting): picked mon.noname-a con 0x7f6cbc06d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cbc06daf0 con 0x7f6cbc09c4a0 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc0a0880 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc09cc80 con 0x7f6cbc06c890 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cc0cda640 10 monclient(hunting): authenticate will time out at 948.257085s 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6cbc06d1b0 0x7f6cbc135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6cbc06d1b0 0x7f6cbc135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38476/0 (socket says 10.20.193.78:38476) 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 1 -- 10.20.193.78:0/2465065524 learned_addr learned my addr 10.20.193.78:0/2465065524 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient(hunting): get_auth_request con 0x7f6cbc06d1b0 auth_method 0 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cba575640 1 --2- 10.20.193.78:0/2465065524 >> v2:10.20.193.115:3300/0 conn(0x7f6cbc06c890 0x7f6cbc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cba575640 10 monclient(hunting): get_auth_request con 0x7f6cbc06c890 auth_method 0 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cba575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cba575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cba575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient(hunting): handle_auth_done global_id 84126 payload 274 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 1 -- 10.20.193.78:0/2465065524 >> v2:10.20.193.115:3300/0 conn(0x7f6cbc06c890 msgr2=0x7f6cbc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 1 --2- 10.20.193.78:0/2465065524 >> v2:10.20.193.115:3300/0 conn(0x7f6cbc06c890 0x7f6cbc06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 1 -- 10.20.193.78:0/2465065524 >> v1:10.20.193.115:6789/0 conn(0x7f6cbc09c4a0 legacy=0x7f6cbc134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 1 monclient: found mon.noname-a 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.704+0000 7f6cbad76640 1 -- 10.20.193.78:0/2465065524 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc09cf60 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.706+0000 7f6cbad76640 1 -- 10.20.193.78:0/2465065524 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 msgr2=0x7f6cbc135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.706+0000 7f6cbad76640 1 --2- 10.20.193.78:0/2465065524 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc135500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06a480 tx=0x7f6cac02f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60551ba7e5c25ff9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 1 -- 10.20.193.78:0/2465065524 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac02fe60 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: dump: 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 1 -- 10.20.193.78:0/2465065524 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cac0373f0 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 5 monclient: authenticate success, global_id 84126 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 1 -- 10.20.193.78:0/2465065524 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0377c0 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 4 set_mon_vals no callback set 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cb9573640 10 monclient: dump: 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.715 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 10 monclient: shutdown 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 1 -- 10.20.193.78:0/2465065524 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 msgr2=0x7f6cbc135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/2465065524 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc135500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06a480 tx=0x7f6cac02f230 comp rx=0 tx=0).stop 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 1 -- 10.20.193.78:0/2465065524 shutdown_connections 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/2465065524 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc135500 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/2465065524 >> v2:10.20.193.115:3300/0 conn(0x7f6cbc06c890 0x7f6cbc06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 1 -- 10.20.193.78:0/2465065524 >> 10.20.193.78:0/2465065524 conn(0x7f6cbc0a6620 msgr2=0x7f6cbc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 1 -- 10.20.193.78:0/2465065524 shutdown_connections 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 1 -- 10.20.193.78:0/2465065524 wait complete. 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 10 monclient: build_initial_monmap 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.707+0000 7f6cc0cda640 10 monclient: monmap: 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 1 Processor -- start 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 1 -- start start 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient: init 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cbc06c890 0x7f6cbc1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient(hunting): picked mon.noname-c con 0x7f6cbc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient(hunting): picked mon.noname-a con 0x7f6cbc06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cbc09c4a0 0x7f6cbc123550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient(hunting): picked mon.noname-b con 0x7f6cbc09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0a01a0 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0a0020 con 0x7f6cbc09c4a0 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:42.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc09be20 con 0x7f6cbc06c890 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cc0cda640 10 monclient(hunting): authenticate will time out at 948.261489s 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc1cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc1cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38478/0 (socket says 10.20.193.78:38478) 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 -- 10.20.193.78:0/4161469385 learned_addr learned my addr 10.20.193.78:0/4161469385 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient(hunting): get_auth_request con 0x7f6cbc06d1b0 auth_method 0 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cba575640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cbc06c890 0x7f6cbc1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cba575640 10 monclient(hunting): get_auth_request con 0x7f6cbc06c890 auth_method 0 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cba575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cba575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cba575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cba575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cba575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cba575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cbad76640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cbc09c4a0 0x7f6cbc123550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cbad76640 10 monclient(hunting): get_auth_request con 0x7f6cbc09c4a0 auth_method 0 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cbad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cbad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cbad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient(hunting): handle_auth_done global_id 84129 payload 824 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 -- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cbc06c890 msgr2=0x7f6cbc1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cbc06c890 0x7f6cbc1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 -- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cbc09c4a0 msgr2=0x7f6cbc123550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cbc09c4a0 0x7f6cbc123550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient: _finish_hunting 0 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 monclient: found mon.noname-a 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.708+0000 7f6cb9d74640 1 -- 10.20.193.78:0/4161469385 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc125d00 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cb9d74640 1 -- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 msgr2=0x7f6cbc1cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cb9d74640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc1cd870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c5d0 tx=0x7f6ca400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 1 -- 10.20.193.78:0/4161469385 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4016020 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: dump: 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.721 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 1 -- 10.20.193.78:0/4161469385 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca4012a60 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 1 -- 10.20.193.78:0/4161469385 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4012e50 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cc0cda640 5 monclient: authenticate success, global_id 84129 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cb8d72640 4 set_mon_vals no callback set 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cc0cda640 10 monclient: _renew_subs 2026-02-17T05:51:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cc0cda640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc1271b0 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6ca2ffd640 10 monclient: dump: 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cc0cda640 10 monclient: _renew_subs 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cc0cda640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.710+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc127490 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.711+0000 7f6cc0cda640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.711+0000 7f6ca2ffd640 1 -- 10.20.193.78:0/4161469385 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6ca401b400 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.711+0000 7f6cc0cda640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.711+0000 7f6cc0cda640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.711+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc074980 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.711+0000 7f6ca2ffd640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c8803e080 0x7f6c88040540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.711+0000 7f6ca2ffd640 1 -- 10.20.193.78:0/4161469385 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(220..220 src has 117..220) ==== 4843+0+0 (secure 0 0 0) 0x7f6ca40528a0 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.712+0000 7f6cba575640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c8803e080 0x7f6c88040540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.712+0000 7f6cba575640 10 monclient: get_auth_request con 0x7f6c8803e080 auth_method 0 2026-02-17T05:51:42.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.714+0000 7f6ca2ffd640 1 -- 10.20.193.78:0/4161469385 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6ca4020030 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.714+0000 7f6ca2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:42.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.714+0000 7f6ca2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:42.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.714+0000 7f6cba575640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c8803e080 0x7f6c88040540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009a50 tx=0x7f6cb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:42.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.830+0000 7f6cc0cda640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:51:42.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.830+0000 7f6cc0cda640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:51:42.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.830+0000 7f6cc0cda640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:42.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.830+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f6cbc06cc70 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.831+0000 7f6ca2ffd640 1 -- 10.20.193.78:0/4161469385 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f6ca4051260 con 0x7f6cbc06d1b0 2026-02-17T05:51:42.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.831+0000 7f6ca2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:51:42.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.831+0000 7f6ca2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.831+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.831+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=1419114897,config=692619927,health=1361790972,kv=1121147495,logm=2124403507}) 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.831+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=2,health=11,kv=12,logm=62} crc {auth=924059048,config=692619927,health=3602034646,kv=449989585,logm=1263236302}) 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.831+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.831+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=1419114897,config=692619927,health=1361790972,kv=1121147495,logm=2124403507}) 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.831+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=15,config=2,health=12,kv=10,logm=61} crc {auth=3674768144,config=692619927,health=1361790972,kv=596693898,logm=3323064159}) 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1232902646}) 2026-02-17T05:51:42.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3800349287}) 2026-02-17T05:51:42.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1232902646}) 2026-02-17T05:51:42.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2401708619}) 2026-02-17T05:51:42.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c8803e080 msgr2=0x7f6c88040540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c8803e080 0x7f6c88040540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009a50 tx=0x7f6cb0008040 comp rx=0 tx=0).stop 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 10 monclient: shutdown 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 msgr2=0x7f6cbc1cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc1cd870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c5d0 tx=0x7f6ca400caa0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 shutdown_connections 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c8803e080 0x7f6c88040540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6cbc09c4a0 0x7f6cbc123550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6cbc06d1b0 0x7f6cbc1cd870 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 --2- 10.20.193.78:0/4161469385 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6cbc06c890 0x7f6cbc1cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 >> 10.20.193.78:0/4161469385 conn(0x7f6cbc0a6620 msgr2=0x7f6cbc072240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 shutdown_connections 2026-02-17T05:51:42.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:42.832+0000 7f6cc0cda640 1 -- 10.20.193.78:0/4161469385 wait complete. 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2183858416}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2004678851}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2183858416}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.832+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2921175600}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492599900}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1707493880}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492599900}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=186628082}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3846767226}) 2026-02-17T05:51:42.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2245236132}) 2026-02-17T05:51:42.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3846767226}) 2026-02-17T05:51:42.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.833+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1425509656}) 2026-02-17T05:51:42.838 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:51:42.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2573110592}) 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3286008518}) 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2573110592}) 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2597962486}) 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2970306608}) 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2241085452}) 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2970306608}) 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.840+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3276216134}) 2026-02-17T05:51:42.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3470996378}) 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=26179226}) 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3470996378}) 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1121305146}) 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916871207}) 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211504904}) 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916871207}) 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=795832587}) 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2378325707}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.841+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3707949566}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2378325707}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3619314257}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4107930555}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=801939606}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4107930555}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1394399283}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1917851767}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3863168008}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1917851767}) 2026-02-17T05:51:42.844 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.842+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2032057165}) 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1107374958}) 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2062303442}) 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1107374958}) 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=838212209}) 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2601628244}) 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4228491049}) 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2601628244}) 2026-02-17T05:51:42.845 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=668641368}) 2026-02-17T05:51:42.847 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.847 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=6} crc {logm=2303633133,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3452471739,mon_config_key=3503210767,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2739706848,osd_snap=2725921312}) 2026-02-17T05:51:42.847 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=50,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=5} crc {logm=1049816541,mdsmap=3140514009,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1267155819,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2343536789,osd_pg_creating=2739706848,osd_snap=1160776918}) 2026-02-17T05:51:42.847 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.847 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=6} crc {logm=2303633133,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3452471739,mon_config_key=3503210767,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2739706848,osd_snap=2725921312}) 2026-02-17T05:51:42.847 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.843+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=51,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=6} crc {logm=1305770141,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3449624255,mon_config_key=1987855637,monmap=1784110042,osd_metadata=1611097156,osd_pg_creating=2739706848,osd_snap=3881713549}) 2026-02-17T05:51:42.847 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.844+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.847 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.844+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=37,osdmap=63} crc {osd_snap=3169949349,osdmap=2163476424}) 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.844+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=37,osdmap=63} crc {osd_snap=2999287436,osdmap=2961650036}) 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.844+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.844+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=37,osdmap=63} crc {osd_snap=3169949349,osdmap=2163476424}) 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.844+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=38,osdmap=62} crc {osd_snap=441301128,osdmap=405881996}) 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.844+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.844+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=94021921}) 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.845+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3982268320}) 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.845+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.845+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=94021921}) 2026-02-17T05:51:42.848 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:51:42.845+0000 7fc536603640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=895870686}) 2026-02-17T05:51:43.110 INFO:tasks.rados.rados.0.trial115.stdout:1039: done (4 left) 2026-02-17T05:51:43.110 INFO:tasks.rados.rados.0.trial115.stdout:1040: snap_remove snap 59 2026-02-17T05:51:43.839 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:51:43.840 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:43.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient: get_monmap_and_config 2026-02-17T05:51:43.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient: build_initial_monmap 2026-02-17T05:51:43.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient: monmap: 2026-02-17T05:51:43.912 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:43.913 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:43.913 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:43.911279+0000 2026-02-17T05:51:43.913 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:43.911279+0000 2026-02-17T05:51:43.913 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:43.913 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:43.913 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:43.913 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:43.913 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 1 Processor -- start 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 1 -- start start 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient: init 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2026-02-17T05:51:43.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efc3806c890 0x7efc3806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient(hunting): picked mon.noname-a con 0x7efc3806c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7efc3809c4a0 0x7efc3806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient(hunting): picked mon.noname-c con 0x7efc3809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efc38134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.911+0000 7efc3d9e6640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc3806daf0 con 0x7efc38134df0 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc3d9e6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc3d9e6640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7efc380a0880 con 0x7efc3806c890 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc3d9e6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc3d9e6640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7efc3806d6f0 con 0x7efc3809c4a0 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc3d9e6640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc3d9e6640 10 monclient(hunting): authenticate will time out at 949.465013s 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efc3806c890 0x7efc3806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:43.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efc3806c890 0x7efc3806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38496/0 (socket says 10.20.193.78:38496) 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 1 -- 10.20.193.78:0/845671853 learned_addr learned my addr 10.20.193.78:0/845671853 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient(hunting): get_auth_request con 0x7efc3806c890 auth_method 0 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient(hunting): handle_auth_done global_id 84132 payload 274 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc367fc640 1 --2- 10.20.193.78:0/845671853 >> v2:10.20.193.115:3300/0 conn(0x7efc3809c4a0 0x7efc3806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:43.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 1 -- 10.20.193.78:0/845671853 >> v2:10.20.193.115:3300/0 conn(0x7efc3809c4a0 msgr2=0x7efc3806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:43.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 1 --2- 10.20.193.78:0/845671853 >> v2:10.20.193.115:3300/0 conn(0x7efc3809c4a0 0x7efc3806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:43.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 1 -- 10.20.193.78:0/845671853 >> v1:10.20.193.115:6789/0 conn(0x7efc38134df0 legacy=0x7efc38135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:43.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:51:43.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 1 monclient: found mon.noname-a 2026-02-17T05:51:43.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:43.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:43.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.912+0000 7efc36ffd640 1 -- 10.20.193.78:0/845671853 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3809cc80 con 0x7efc3806c890 2026-02-17T05:51:43.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc36ffd640 1 -- 10.20.193.78:0/845671853 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3806c890 msgr2=0x7efc3806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:43.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc36ffd640 1 --2- 10.20.193.78:0/845671853 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3806c890 0x7efc3806cc70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc3809c980 tx=0x7efc2c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5251665573130d7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:43.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 1 -- 10.20.193.78:0/845671853 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c03c070 con 0x7efc3806c890 2026-02-17T05:51:43.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:43.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:43.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 10 monclient: dump: 2026-02-17T05:51:43.925 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 10 monclient: _finish_auth 0 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:51:43.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc3d9e6640 5 monclient: authenticate success, global_id 84132 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 1 -- 10.20.193.78:0/845671853 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2c02fd80 con 0x7efc3806c890 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc3d9e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.914+0000 7efc35ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc35ffb640 1 -- 10.20.193.78:0/845671853 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c037660 con 0x7efc3806c890 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc35ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc35ffb640 10 monclient: dump: 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:43.927 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 4 set_mon_vals no callback set 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 10 monclient: shutdown 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 -- 10.20.193.78:0/845671853 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3806c890 msgr2=0x7efc3806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:43.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/845671853 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3806c890 0x7efc3806cc70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc3809c980 tx=0x7efc2c02ee60 comp rx=0 tx=0).stop 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 -- 10.20.193.78:0/845671853 shutdown_connections 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/845671853 >> v2:10.20.193.115:3300/0 conn(0x7efc3809c4a0 0x7efc3806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/845671853 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3806c890 0x7efc3806cc70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 -- 10.20.193.78:0/845671853 >> 10.20.193.78:0/845671853 conn(0x7efc380a6620 msgr2=0x7efc380a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 -- 10.20.193.78:0/845671853 shutdown_connections 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 -- 10.20.193.78:0/845671853 wait complete. 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 10 monclient: build_initial_monmap 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 10 monclient: monmap: 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:43.929 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 Processor -- start 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 1 -- start start 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.915+0000 7efc3d9e6640 10 monclient: init 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:51:43.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efc3806c890 0x7efc381cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient(hunting): picked mon.noname-b con 0x7efc3806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3809c4a0 0x7efc381cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient(hunting): picked mon.noname-a con 0x7efc3809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efc38134df0 0x7efc38121620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient(hunting): picked mon.noname-c con 0x7efc38134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc380a01a0 con 0x7efc3809c4a0 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc380a0020 con 0x7efc3806c890 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efc3809be20 con 0x7efc38134df0 2026-02-17T05:51:43.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc3d9e6640 10 monclient(hunting): authenticate will time out at 949.469288s 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3809c4a0 0x7efc381cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3809c4a0 0x7efc381cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38512/0 (socket says 10.20.193.78:38512) 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 -- 10.20.193.78:0/1338431948 learned_addr learned my addr 10.20.193.78:0/1338431948 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient(hunting): get_auth_request con 0x7efc3809c4a0 auth_method 0 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc377fe640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efc38134df0 0x7efc38121620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:43.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc377fe640 10 monclient(hunting): get_auth_request con 0x7efc38134df0 auth_method 0 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc377fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc36ffd640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efc3806c890 0x7efc381cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient(hunting): handle_auth_done global_id 84135 payload 824 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 -- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efc38134df0 msgr2=0x7efc38121620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:43.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efc38134df0 0x7efc38121620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 -- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efc3806c890 msgr2=0x7efc381cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efc3806c890 0x7efc381cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient: _finish_hunting 0 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 monclient: found mon.noname-a 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc367fc640 1 -- 10.20.193.78:0/1338431948 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc38125290 con 0x7efc3809c4a0 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc36ffd640 10 monclient: get_auth_request con 0x7efc3806c890 auth_method 0 2026-02-17T05:51:43.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.916+0000 7efc36ffd640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efc3806c890 0x7efc381cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc367fc640 1 -- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3809c4a0 msgr2=0x7efc381cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc367fc640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3809c4a0 0x7efc381cd870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc2000c7f0 tx=0x7efc2000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 1 -- 10.20.193.78:0/1338431948 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20007c40 con 0x7efc3809c4a0 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: dump: 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:43.935 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: _finish_auth 0 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 1 -- 10.20.193.78:0/1338431948 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc20007e00 con 0x7efc3809c4a0 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc3d9e6640 5 monclient: authenticate success, global_id 84135 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc3d9e6640 10 monclient: _renew_subs 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc3d9e6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc38123de0 con 0x7efc3809c4a0 2026-02-17T05:51:43.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 1 -- 10.20.193.78:0/1338431948 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20004260 con 0x7efc3809c4a0 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc0ffff640 10 monclient: dump: 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:43.937 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc357fa640 4 set_mon_vals no callback set 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc3d9e6640 10 monclient: _renew_subs 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc3d9e6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.918+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc380a1770 con 0x7efc3809c4a0 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc3d9e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc3d9e6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc3d9e6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc38074980 con 0x7efc3809c4a0 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc0ffff640 1 -- 10.20.193.78:0/1338431948 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7efc20004400 con 0x7efc3809c4a0 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc0ffff640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc0803df00 0x7efc080403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc0ffff640 1 -- 10.20.193.78:0/1338431948 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(221..221 src has 117..221) ==== 4843+0+0 (secure 0 0 0) 0x7efc20052ed0 con 0x7efc3809c4a0 2026-02-17T05:51:43.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc36ffd640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc0803df00 0x7efc080403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:43.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.919+0000 7efc36ffd640 10 monclient: get_auth_request con 0x7efc0803df00 auth_method 0 2026-02-17T05:51:43.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.922+0000 7efc0ffff640 1 -- 10.20.193.78:0/1338431948 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efc2001e020 con 0x7efc3809c4a0 2026-02-17T05:51:43.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.922+0000 7efc0ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:43.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.922+0000 7efc0ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:43.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:43.922+0000 7efc36ffd640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc0803df00 0x7efc080403c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc2c036000 tx=0x7efc2c039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.034+0000 7efc3d9e6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:44.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.034+0000 7efc3d9e6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:44.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.034+0000 7efc3d9e6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.034+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efc38135a20 con 0x7efc3809c4a0 2026-02-17T05:51:44.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.034+0000 7efc0ffff640 1 -- 10.20.193.78:0/1338431948 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7efc20015030 con 0x7efc3809c4a0 2026-02-17T05:51:44.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.034+0000 7efc0ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:44.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.034+0000 7efc0ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:44.034 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc0803df00 msgr2=0x7efc080403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc0803df00 0x7efc080403c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc2c036000 tx=0x7efc2c039040 comp rx=0 tx=0).stop 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 10 monclient: shutdown 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3809c4a0 msgr2=0x7efc381cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3809c4a0 0x7efc381cd870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc2000c7f0 tx=0x7efc2000ccc0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 shutdown_connections 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efc38134df0 0x7efc38121620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efc3809c4a0 0x7efc381cd870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc0803df00 0x7efc080403c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 --2- 10.20.193.78:0/1338431948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efc3806c890 0x7efc381cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 >> 10.20.193.78:0/1338431948 conn(0x7efc380a6620 msgr2=0x7efc38071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 shutdown_connections 2026-02-17T05:51:44.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.035+0000 7efc3d9e6640 1 -- 10.20.193.78:0/1338431948 wait complete. 2026-02-17T05:51:44.041 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:51:44.041 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:51:44.042 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:51:44.042 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:51:44.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.111+0000 7f8cada76640 10 monclient: get_monmap_and_config 2026-02-17T05:51:44.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.111+0000 7f8cada76640 10 monclient: build_initial_monmap 2026-02-17T05:51:44.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.111+0000 7f8cada76640 10 monclient: monmap: 2026-02-17T05:51:44.110 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:44.110 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.110 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:44.112005+0000 2026-02-17T05:51:44.110 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:44.112005+0000 2026-02-17T05:51:44.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:44.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.111 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:44.111 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:44.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:44.111 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:44.111 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:44.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.112 INFO:tasks.rados.rados.0.trial115.stdout:1040: done (4 left) 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1041: delete oid 30 current snap is 103 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1035: finishing write tid 2 to trial11545420-47 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1035: finishing write tid 3 to trial11545420-47 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1035: finishing write tid 4 to trial11545420-47 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1035: oid 47 updating version 0 to 419 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1035: oid 47 version 419 is already newer than 417 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1035: oid 47 version 419 is already newer than 418 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 419 (ObjNum 402 snap 103 seq_num 402) dirty exists 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1035: left oid 47 (ObjNum 402 snap 103 seq_num 402) 2026-02-17T05:51:44.113 INFO:tasks.rados.rados.0.trial115.stdout:1037: expect (ObjNum 389 snap 101 seq_num 389) 2026-02-17T05:51:44.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 1 Processor -- start 2026-02-17T05:51:44.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 1 -- start start 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient: init 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8ca809ea90 0x7f8ca80a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient(hunting): picked mon.noname-a con 0x7f8ca809ea90 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8ca809ee70 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8ca813d640 0x7f8ca813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient(hunting): picked mon.noname-c con 0x7f8ca813d640 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca80a0a20 con 0x7f8ca809ee70 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca80a0d00 con 0x7f8ca809ea90 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca80a0460 con 0x7f8ca813d640 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8cada76640 10 monclient(hunting): authenticate will time out at 949.665804s 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8ca809ea90 0x7f8ca80a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8ca809ea90 0x7f8ca80a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38518/0 (socket says 10.20.193.78:38518) 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/3422309559 learned_addr learned my addr 10.20.193.78:0/3422309559 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 10 monclient(hunting): get_auth_request con 0x7f8ca809ea90 auth_method 0 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.112+0000 7f8ca6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca77fe640 1 --2- 10.20.193.78:0/3422309559 >> v2:10.20.193.115:3300/0 conn(0x7f8ca813d640 0x7f8ca813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca77fe640 10 monclient(hunting): get_auth_request con 0x7f8ca813d640 auth_method 0 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 10 monclient(hunting): handle_auth_done global_id 84138 payload 274 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/3422309559 >> v2:10.20.193.115:3300/0 conn(0x7f8ca813d640 msgr2=0x7f8ca813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 1 --2- 10.20.193.78:0/3422309559 >> v2:10.20.193.115:3300/0 conn(0x7f8ca813d640 0x7f8ca813fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/3422309559 >> v1:10.20.193.115:6789/0 conn(0x7f8ca809ee70 legacy=0x7f8ca809f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 1 monclient: found mon.noname-a 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.113+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/3422309559 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca80a0740 con 0x7f8ca809ea90 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/3422309559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 msgr2=0x7f8ca80a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca6ffd640 1 --2- 10.20.193.78:0/3422309559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 0x7f8ca80a13d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8135c40 tx=0x7f8c9c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780823eddcbef02a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 1 -- 10.20.193.78:0/3422309559 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c03c070 con 0x7f8ca809ea90 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: dump: 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.129 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:44.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 1 -- 10.20.193.78:0/3422309559 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c02fca0 con 0x7f8ca809ea90 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 1 -- 10.20.193.78:0/3422309559 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c037660 con 0x7f8ca809ea90 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8cada76640 5 monclient: authenticate success, global_id 84138 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8ca5ffb640 10 monclient: dump: 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.131 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8cada76640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.115+0000 7f8cada76640 4 set_mon_vals no callback set 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient: shutdown 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- 10.20.193.78:0/3422309559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 msgr2=0x7f8ca80a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 --2- 10.20.193.78:0/3422309559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 0x7f8ca80a13d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8135c40 tx=0x7f8c9c02ee60 comp rx=0 tx=0).stop 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- 10.20.193.78:0/3422309559 shutdown_connections 2026-02-17T05:51:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 --2- 10.20.193.78:0/3422309559 >> v2:10.20.193.115:3300/0 conn(0x7f8ca813d640 0x7f8ca813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 --2- 10.20.193.78:0/3422309559 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 0x7f8ca80a13d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- 10.20.193.78:0/3422309559 >> 10.20.193.78:0/3422309559 conn(0x7f8ca80a7520 msgr2=0x7f8ca80a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- 10.20.193.78:0/3422309559 shutdown_connections 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- 10.20.193.78:0/3422309559 wait complete. 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient: build_initial_monmap 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient: monmap: 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:44.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 Processor -- start 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- start start 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient: init 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 0x7f8ca813bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient(hunting): picked mon.noname-a con 0x7f8ca809ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8ca809ee70 0x7f8ca813c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient(hunting): picked mon.noname-b con 0x7f8ca809ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8ca813d640 0x7f8ca813c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient(hunting): picked mon.noname-c con 0x7f8ca813d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80a6820 con 0x7f8ca809ea90 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80a66a0 con 0x7f8ca809ee70 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca813ff70 con 0x7f8ca813d640 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.116+0000 7f8cada76640 10 monclient(hunting): authenticate will time out at 949.669976s 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 0x7f8ca813bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 0x7f8ca813bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38528/0 (socket says 10.20.193.78:38528) 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/1530127437 learned_addr learned my addr 10.20.193.78:0/1530127437 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient(hunting): get_auth_request con 0x7f8ca809ea90 auth_method 0 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca77fe640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8ca813d640 0x7f8ca813c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca77fe640 10 monclient(hunting): get_auth_request con 0x7f8ca813d640 auth_method 0 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca67fc640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8ca809ee70 0x7f8ca813c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca67fc640 10 monclient(hunting): get_auth_request con 0x7f8ca809ee70 auth_method 0 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient(hunting): handle_auth_done global_id 84141 payload 824 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8ca813d640 msgr2=0x7f8ca813c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8ca813d640 0x7f8ca813c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8ca809ee70 msgr2=0x7f8ca813c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8ca809ee70 0x7f8ca813c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 monclient: found mon.noname-a 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.117+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/1530127437 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca8071350 con 0x7f8ca809ea90 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8ca6ffd640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 msgr2=0x7f8ca813bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8ca6ffd640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 0x7f8ca813bcb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c002410 tx=0x7f8c9c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c046020 con 0x7f8ca809ea90 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: dump: 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: _finish_auth 0 2026-02-17T05:51:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c02fe50 con 0x7f8ca809ea90 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c03c040 con 0x7f8ca809ea90 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 5 monclient: authenticate success, global_id 84141 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8ca57fa640 4 set_mon_vals no callback set 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 10 monclient: _renew_subs 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca80a9c80 con 0x7f8ca809ea90 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8c7ffff640 10 monclient: dump: 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 10 monclient: _renew_subs 2026-02-17T05:51:44.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca806fd70 con 0x7f8ca809ea90 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cf2b0 0x7f8ca81cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.119+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80a13d0 con 0x7f8ca81cf2b0 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cf2b0 0x7f8ca81cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 10 monclient: get_auth_request con 0x7f8ca81cf2b0 auth_method 0 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 10 monclient(hunting): handle_auth_done global_id 84144 payload 824 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca67fc640 1 -- 10.20.193.78:0/1530127437 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ca80008d0 con 0x7f8ca81cf2b0 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8c9c03f030 con 0x7f8ca809ea90 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8c7ffff640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8c7803df00 0x7f8c780403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(222..222 src has 117..222) ==== 4859+0+0 (secure 0 0 0) 0x7f8c9c0778e0 con 0x7f8ca809ea90 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca77fe640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8c7803df00 0x7f8c780403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.120+0000 7f8ca77fe640 10 monclient: get_auth_request con 0x7f8c7803df00 auth_method 0 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8ca67fc640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cf2b0 msgr2=0x7f8ca81cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8ca67fc640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cf2b0 0x7f8ca81cf690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8071560 tx=0x7f8c9000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90006ae0 con 0x7f8ca81cf2b0 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 10 monclient: dump: 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f8c90006ae0 con 0x7f8ca81cf2b0 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cf2b0 msgr2=0x7f8ca81cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cf2b0 0x7f8ca81cf690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8071560 tx=0x7f8c9000eba0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cf2b0 msgr2=0x7f8ca81cf690 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.122+0000 7f8c7ffff640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cf2b0 0x7f8ca81cf690 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.123+0000 7f8ca77fe640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8c7803df00 0x7f8c780403c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c98003ba0 tx=0x7f8c98009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8cada76640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8cada76640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cfbd0 0x7f8ca81da600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8cada76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8070fd0 con 0x7f8ca81cfbd0 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cfbd0 0x7f8ca81da600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 10 monclient: get_auth_request con 0x7f8ca81cfbd0 auth_method 0 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 10 monclient(hunting): handle_auth_done global_id 84147 payload 824 2026-02-17T05:51:44.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.125+0000 7f8ca67fc640 1 -- 10.20.193.78:0/1530127437 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8ca8071180 con 0x7f8ca81cfbd0 2026-02-17T05:51:44.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.127+0000 7f8ca67fc640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cfbd0 msgr2=0x7f8ca81da600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.127+0000 7f8ca67fc640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cfbd0 0x7f8ca81da600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c90004c50 tx=0x7f8c90003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000c930 con 0x7f8ca81cfbd0 2026-02-17T05:51:44.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:44.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 10 monclient: dump: 2026-02-17T05:51:44.147 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f8c9000c930 con 0x7f8ca81cfbd0 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cfbd0 msgr2=0x7f8ca81da600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cfbd0 0x7f8ca81da600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c90004c50 tx=0x7f8c90003860 comp rx=0 tx=0).stop 2026-02-17T05:51:44.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cfbd0 msgr2=0x7f8ca81da600 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8c7ffff640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca81cfbd0 0x7f8ca81da600 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c90004c50 tx=0x7f8c90003860 comp rx=0 tx=0).stop 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 60, 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 28709, 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:51:44.149 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.150 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.151 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:51:44.152 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.153 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.154 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.155 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.156 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.157 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.158 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.159 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:51:44.159 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.159 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.159 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.159 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:51:44.159 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8c7803df00 msgr2=0x7f8c780403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8c7803df00 0x7f8c780403c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c98003ba0 tx=0x7f8c98009820 comp rx=0 tx=0).stop 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 10 monclient: shutdown 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 msgr2=0x7f8ca813bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 1 --2- 10.20.193.78:0/1530127437 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8ca809ea90 0x7f8ca813bcb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c002410 tx=0x7f8c9c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8ca77fe640 1 -- 10.20.193.78:0/1530127437 reap_dead start 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8ca77fe640 1 -- 10.20.193.78:0/1530127437 reap_dead start 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 shutdown_connections 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 >> 10.20.193.78:0/1530127437 conn(0x7f8ca80a7520 msgr2=0x7f8ca80a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 shutdown_connections 2026-02-17T05:51:44.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.128+0000 7f8cada76640 1 -- 10.20.193.78:0/1530127437 wait complete. 2026-02-17T05:51:44.161 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:51:44.161 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:51:44.161 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:51:44.216 INFO:tasks.rados.rados.0.trial115.stdout:1038: finishing copy_from racing read to trial11545420-23 2026-02-17T05:51:44.216 INFO:tasks.rados.rados.0.trial115.stdout:1038: finishing copy_from to trial11545420-23 2026-02-17T05:51:44.216 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 176 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:51:44.217 INFO:tasks.rados.rados.0.trial115.stdout:1035: done (4 left) 2026-02-17T05:51:44.217 INFO:tasks.rados.rados.0.trial115.stdout:1036: done (3 left) 2026-02-17T05:51:44.217 INFO:tasks.rados.rados.0.trial115.stdout:1037: done (2 left) 2026-02-17T05:51:44.217 INFO:tasks.rados.rados.0.trial115.stdout:1038: done (1 left) 2026-02-17T05:51:44.217 INFO:tasks.rados.rados.0.trial115.stdout:1041: done (0 left) 2026-02-17T05:51:44.217 INFO:tasks.rados.rados.0.trial115.stdout:1042: write oid 18 current snap is 103 2026-02-17T05:51:44.217 INFO:tasks.rados.rados.0.trial115.stdout:1042: seq_num 404 ranges {790028=567305,1874490=763137} 2026-02-17T05:51:44.221 INFO:tasks.rados.rados.0.trial115.stdout:1042: writing trial11545420-18 from 790028 to 1357333 tid 1 2026-02-17T05:51:44.227 INFO:tasks.rados.rados.0.trial115.stdout:1042: writing trial11545420-18 from 1874490 to 2637627 tid 2 2026-02-17T05:51:44.227 INFO:tasks.rados.rados.0.trial115.stdout:1043: read oid 23 snap -1 2026-02-17T05:51:44.227 INFO:tasks.rados.rados.0.trial115.stdout:1043: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:51:44.227 INFO:tasks.rados.rados.0.trial115.stdout:1044: snap_create 2026-02-17T05:51:44.227 INFO:tasks.rados.rados.0.trial115.stdout:1042: finishing write tid 1 to trial11545420-18 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.230+0000 7fea12a56640 10 monclient: get_monmap_and_config 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.230+0000 7fea12a56640 10 monclient: build_initial_monmap 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.230+0000 7fea12a56640 10 monclient: monmap: 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:44.230813+0000 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:44.230813+0000 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.229 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:44.230 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:44.230 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:44.230 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:44.230 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:44.230 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.231 INFO:tasks.rados.rados.0.trial115.stdout:1042: finishing write tid 2 to trial11545420-18 2026-02-17T05:51:44.231 INFO:tasks.rados.rados.0.trial115.stdout:1042: finishing write tid 3 to trial11545420-18 2026-02-17T05:51:44.236 INFO:tasks.rados.rados.0.trial115.stdout:1042: finishing write tid 4 to trial11545420-18 2026-02-17T05:51:44.236 INFO:tasks.rados.rados.0.trial115.stdout:1042: oid 18 updating version 0 to 400 2026-02-17T05:51:44.237 INFO:tasks.rados.rados.0.trial115.stdout:1042: oid 18 version 400 is already newer than 399 2026-02-17T05:51:44.237 INFO:tasks.rados.rados.0.trial115.stdout:1042: oid 18 version 400 is already newer than 398 2026-02-17T05:51:44.237 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 400 (ObjNum 404 snap 103 seq_num 404) dirty exists 2026-02-17T05:51:44.237 INFO:tasks.rados.rados.0.trial115.stdout:1042: left oid 18 (ObjNum 404 snap 103 seq_num 404) 2026-02-17T05:51:44.242 INFO:tasks.rados.rados.0.trial115.stdout:1043: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:51:44.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 1 Processor -- start 2026-02-17T05:51:44.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 1 -- start start 2026-02-17T05:51:44.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient: init 2026-02-17T05:51:44.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:44.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T05:51:44.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c06c890 0x7fea0c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient(hunting): picked mon.noname-b con 0x7fea0c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:44.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fea0c09c4a0 0x7fea0c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient(hunting): picked mon.noname-c con 0x7fea0c09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:44.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fea0c134df0 0x7fea0c135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient(hunting): picked mon.noname-a con 0x7fea0c134df0 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fea0c0a0880 con 0x7fea0c134df0 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fea0c06d6f0 con 0x7fea0c09c4a0 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fea0c06daf0 con 0x7fea0c06c890 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:44.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea12a56640 10 monclient(hunting): authenticate will time out at 949.784731s 2026-02-17T05:51:44.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c06c890 0x7fea0c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c06c890 0x7fea0c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46022/0 (socket says 10.20.193.78:46022) 2026-02-17T05:51:44.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 1 -- 10.20.193.78:0/2541277548 learned_addr learned my addr 10.20.193.78:0/2541277548 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:44.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 10 monclient(hunting): get_auth_request con 0x7fea0c06c890 auth_method 0 2026-02-17T05:51:44.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea10fcc640 1 --2- 10.20.193.78:0/2541277548 >> v2:10.20.193.78:3300/0 conn(0x7fea0c134df0 0x7fea0c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea10fcc640 10 monclient(hunting): get_auth_request con 0x7fea0c134df0 auth_method 0 2026-02-17T05:51:44.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea10fcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea10fcc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea10fcc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea10fcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea10fcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea10fcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0b7fe640 1 --2- 10.20.193.78:0/2541277548 >> v2:10.20.193.115:3300/0 conn(0x7fea0c09c4a0 0x7fea0c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 10 monclient(hunting): handle_auth_done global_id 74225 payload 274 2026-02-17T05:51:44.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 1 -- 10.20.193.78:0/2541277548 >> v2:10.20.193.115:3300/0 conn(0x7fea0c09c4a0 msgr2=0x7fea0c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:44.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 1 --2- 10.20.193.78:0/2541277548 >> v2:10.20.193.115:3300/0 conn(0x7fea0c09c4a0 0x7fea0c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 1 -- 10.20.193.78:0/2541277548 >> v2:10.20.193.78:3300/0 conn(0x7fea0c134df0 msgr2=0x7fea0c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:44.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.231+0000 7fea0bfff640 1 --2- 10.20.193.78:0/2541277548 >> v2:10.20.193.78:3300/0 conn(0x7fea0c134df0 0x7fea0c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.232+0000 7fea0bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:51:44.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.232+0000 7fea0bfff640 1 monclient: found mon.noname-b 2026-02-17T05:51:44.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.232+0000 7fea0bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:44.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.232+0000 7fea0bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:44.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.232+0000 7fea0bfff640 1 -- 10.20.193.78:0/2541277548 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c09cc80 con 0x7fea0c06c890 2026-02-17T05:51:44.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.232+0000 7fea0b7fe640 10 monclient: get_auth_request con 0x7fea0c09c4a0 auth_method 0 2026-02-17T05:51:44.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.232+0000 7fea0b7fe640 1 --2- 10.20.193.78:0/2541277548 >> v2:10.20.193.115:3300/0 conn(0x7fea0c09c4a0 0x7fea0c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:51:44.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.232+0000 7fea10fcc640 1 --2- 10.20.193.78:0/2541277548 >> v2:10.20.193.78:3300/0 conn(0x7fea0c134df0 0x7fea0c135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:51:44.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0bfff640 1 -- 10.20.193.78:0/2541277548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c06c890 msgr2=0x7fea0c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0bfff640 1 --2- 10.20.193.78:0/2541277548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c06c890 0x7fea0c06cc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0a7600 tx=0x7fe9f802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f3b7989b67ca5a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 1 -- 10.20.193.78:0/2541277548 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f803c070 con 0x7fea0c06c890 2026-02-17T05:51:44.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:44.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: dump: 2026-02-17T05:51:44.254 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.255 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: _finish_auth 0 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: _check_auth_tickets 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 1 -- 10.20.193.78:0/2541277548 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f802fca0 con 0x7fea0c06c890 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 5 monclient: authenticate success, global_id 74225 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 1 -- 10.20.193.78:0/2541277548 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8037660 con 0x7fea0c06c890 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 4 set_mon_vals no callback set 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea0affd640 10 monclient: dump: 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.257 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 10 monclient: shutdown 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 -- 10.20.193.78:0/2541277548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c06c890 msgr2=0x7fea0c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 --2- 10.20.193.78:0/2541277548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c06c890 0x7fea0c06cc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0a7600 tx=0x7fe9f802ee60 comp rx=0 tx=0).stop 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 -- 10.20.193.78:0/2541277548 shutdown_connections 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 --2- 10.20.193.78:0/2541277548 >> v2:10.20.193.78:3300/0 conn(0x7fea0c134df0 0x7fea0c135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 --2- 10.20.193.78:0/2541277548 >> v2:10.20.193.115:3300/0 conn(0x7fea0c09c4a0 0x7fea0c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 --2- 10.20.193.78:0/2541277548 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c06c890 0x7fea0c06cc70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 -- 10.20.193.78:0/2541277548 >> 10.20.193.78:0/2541277548 conn(0x7fea0c0a6620 msgr2=0x7fea0c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 -- 10.20.193.78:0/2541277548 shutdown_connections 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.234+0000 7fea12a56640 1 -- 10.20.193.78:0/2541277548 wait complete. 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient: build_initial_monmap 2026-02-17T05:51:44.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient: monmap: 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:44.260 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:44.261 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 1 Processor -- start 2026-02-17T05:51:44.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 1 -- start start 2026-02-17T05:51:44.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient: init 2026-02-17T05:51:44.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:44.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:51:44.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fea0c06c890 0x7fea0c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient(hunting): picked mon.noname-a con 0x7fea0c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c09c4a0 0x7fea0c1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient(hunting): picked mon.noname-c con 0x7fea0c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fea0c134df0 0x7fea0c122be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient(hunting): picked mon.noname-b con 0x7fea0c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c0a01a0 con 0x7fea0c06c890 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c0a0020 con 0x7fea0c134df0 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fea0c09be20 con 0x7fea0c09c4a0 2026-02-17T05:51:44.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea12a56640 10 monclient(hunting): authenticate will time out at 949.788717s 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fea0c06c890 0x7fea0c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c09c4a0 0x7fea0c1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fea0c06c890 0x7fea0c1cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38558/0 (socket says 10.20.193.78:38558) 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 -- 10.20.193.78:0/1954021332 learned_addr learned my addr 10.20.193.78:0/1954021332 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient(hunting): get_auth_request con 0x7fea0c06c890 auth_method 0 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0b7fe640 10 monclient(hunting): get_auth_request con 0x7fea0c09c4a0 auth_method 0 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea10fcc640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fea0c134df0 0x7fea0c122be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea10fcc640 10 monclient(hunting): get_auth_request con 0x7fea0c134df0 auth_method 0 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea10fcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea10fcc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea10fcc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient(hunting): handle_auth_done global_id 84153 payload 824 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c09c4a0 msgr2=0x7fea0c1cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c09c4a0 0x7fea0c1cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fea0c134df0 msgr2=0x7fea0c122be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fea0c134df0 0x7fea0c122be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:51:44.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 monclient: found mon.noname-a 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.235+0000 7fea0bfff640 1 -- 10.20.193.78:0/1954021332 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c125350 con 0x7fea0c06c890 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0bfff640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fea0c06c890 msgr2=0x7fea0c1cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0bfff640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fea0c06c890 0x7fea0c1cf350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8002410 tx=0x7fe9f8004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8047070 con 0x7fea0c06c890 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: dump: 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.266 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: _finish_auth 0 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f802fe50 con 0x7fea0c06c890 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 5 monclient: authenticate success, global_id 84153 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 10 monclient: _renew_subs 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0c126ae0 con 0x7fea0c06c890 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f803c040 con 0x7fea0c06c890 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0a7fc640 4 set_mon_vals no callback set 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 10 monclient: dump: 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 10 monclient: _renew_subs 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0c126520 con 0x7fea0c06c890 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123120 0x7fea0c123500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fea0c0724d0 con 0x7fea0c123120 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123120 0x7fea0c123500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 10 monclient: get_auth_request con 0x7fea0c123120 auth_method 0 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_done global_id 74231 payload 824 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe9f803f9f0 con 0x7fea0c06c890 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.238+0000 7fea0b7fe640 1 -- 10.20.193.78:0/1954021332 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea0c074980 con 0x7fea0c123120 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.239+0000 7fea08ff9640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe9dc03deb0 0x7fe9dc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.239+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(222..222 src has 117..222) ==== 4859+0+0 (secure 0 0 0) 0x7fe9f8077540 con 0x7fea0c06c890 2026-02-17T05:51:44.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.239+0000 7fea10fcc640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe9dc03deb0 0x7fe9dc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.239+0000 7fea10fcc640 10 monclient: get_auth_request con 0x7fe9dc03deb0 auth_method 0 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea0b7fe640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123120 msgr2=0x7fea0c123500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea0b7fe640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123120 0x7fea0c123500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea0c1243f0 tx=0x7fe9fc00ae10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc013440 con 0x7fea0c123120 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 10 monclient: dump: 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fe9fc013440 con 0x7fea0c123120 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123120 msgr2=0x7fea0c123500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123120 0x7fea0c123500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea0c1243f0 tx=0x7fe9fc00ae10 comp rx=0 tx=0).stop 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123120 msgr2=0x7fea0c123500 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea08ff9640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123120 0x7fea0c123500 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea0c1243f0 tx=0x7fe9fc00ae10 comp rx=0 tx=0).stop 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.241+0000 7fea10fcc640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe9dc03deb0 0x7fe9dc040370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea0c073e30 tx=0x7fea0000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea12a56640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea12a56640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123a40 0x7fea0c1da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea12a56640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fea0c1250d0 con 0x7fea0c123a40 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123a40 0x7fea0c1da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 10 monclient: get_auth_request con 0x7fea0c123a40 auth_method 0 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 10 monclient(hunting): handle_auth_done global_id 74234 payload 824 2026-02-17T05:51:44.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.244+0000 7fea0b7fe640 1 -- 10.20.193.78:0/1954021332 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fea0c0744d0 con 0x7fea0c123a40 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.246+0000 7fea0b7fe640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123a40 msgr2=0x7fea0c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.246+0000 7fea0b7fe640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123a40 0x7fea0c1da560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea0c1243f0 tx=0x7fe9fc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.246+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc003030 con 0x7fea0c123a40 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.246+0000 7fea08ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.246+0000 7fea08ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.246+0000 7fea08ff9640 10 monclient: dump: 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4707 (secure 0 0 0) 0x7fe9fc005f40 con 0x7fea0c123a40 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea08ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:51:44.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea08ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123a40 msgr2=0x7fea0c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea08ff9640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123a40 0x7fea0c1da560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea0c1243f0 tx=0x7fe9fc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea08ff9640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123a40 msgr2=0x7fea0c1da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea08ff9640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fea0c123a40 0x7fea0c1da560 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea0c1243f0 tx=0x7fe9fc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 60, 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 76138, 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:51:44.273 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:51:44.274 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:44.275 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.276 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:51:44.277 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:51:44.278 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:51:44.279 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.280 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.281 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.281 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:44.281 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.281 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.281 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.281 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:51:44.281 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe9dc03deb0 msgr2=0x7fe9dc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe9dc03deb0 0x7fe9dc040370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea0c073e30 tx=0x7fea0000b040 comp rx=0 tx=0).stop 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 10 monclient: shutdown 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fea0c06c890 msgr2=0x7fea0c1cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fea0c06c890 0x7fea0c1cf350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8002410 tx=0x7fe9f8004830 comp rx=0 tx=0).stop 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea10fcc640 1 -- 10.20.193.78:0/1954021332 reap_dead start 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 shutdown_connections 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 --2- 10.20.193.78:0/1954021332 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fea0c06c890 0x7fea0c1cf350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 >> 10.20.193.78:0/1954021332 conn(0x7fea0c0a6620 msgr2=0x7fea0c135810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 shutdown_connections 2026-02-17T05:51:44.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.247+0000 7fea12a56640 1 -- 10.20.193.78:0/1954021332 wait complete. 2026-02-17T05:51:44.282 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:51:44.283 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:51:44.283 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.352+0000 7fcba3722640 10 monclient: get_monmap_and_config 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.352+0000 7fcba3722640 10 monclient: build_initial_monmap 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.352+0000 7fcba3722640 10 monclient: monmap: 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:44.352735+0000 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:44.352735+0000 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.352+0000 7fcba3722640 1 Processor -- start 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.352+0000 7fcba3722640 1 -- start start 2026-02-17T05:51:44.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.352+0000 7fcba3722640 10 monclient: init 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcb9c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcb9c09c4a0 0x7fcb9c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 10 monclient(hunting): picked mon.noname-a con 0x7fcb9c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcb9c06d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9c06daf0 con 0x7fcb9c06c890 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9c09cc80 con 0x7fcb9c06d380 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9c0a0880 con 0x7fcb9c09c4a0 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba3722640 10 monclient(hunting): authenticate will time out at 949.906266s 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba1497640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fcb9c06c890 0x7fcb9c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:41246/0 (socket says 10.20.193.78:41246) 2026-02-17T05:51:44.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba1497640 1 -- 10.20.193.78:0/2882115499 learned_addr learned my addr 10.20.193.78:0/2882115499 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba0c96640 1 --2- 10.20.193.78:0/2882115499 >> v2:10.20.193.78:3300/0 conn(0x7fcb9c09c4a0 0x7fcb9c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba0c96640 10 monclient(hunting): get_auth_request con 0x7fcb9c09c4a0 auth_method 0 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba0c96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba0c96640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.353+0000 7fcba0c96640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba1497640 1 -- 10.20.193.78:0/2882115499 >> v1:10.20.193.78:6789/0 conn(0x7fcb9c06c890 legacy=0x7fcb9c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 10 monclient(hunting): handle_auth_done global_id 84156 payload 274 2026-02-17T05:51:44.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba1c98640 1 -- 10.20.193.78:0/2882115499 >> v1:10.20.193.115:6789/0 conn(0x7fcb9c06d380 legacy=0x7fcb9c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 1 -- 10.20.193.78:0/2882115499 >> v1:10.20.193.115:6789/0 conn(0x7fcb9c06d380 legacy=0x7fcb9c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 1 -- 10.20.193.78:0/2882115499 >> v1:10.20.193.78:6789/0 conn(0x7fcb9c06c890 legacy=0x7fcb9c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 10 monclient: _finish_hunting 0 2026-02-17T05:51:44.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 1 monclient: found mon.noname-a 2026-02-17T05:51:44.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:44.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.355+0000 7fcba0c96640 1 -- 10.20.193.78:0/2882115499 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c09cf60 con 0x7fcb9c09c4a0 2026-02-17T05:51:44.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba0c96640 1 -- 10.20.193.78:0/2882115499 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c09c4a0 msgr2=0x7fcb9c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba0c96640 1 --2- 10.20.193.78:0/2882115499 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c09c4a0 0x7fcb9c13b3a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb900099a0 tx=0x7fcb9002eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb0c246fc83d5dd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 1 -- 10.20.193.78:0/2882115499 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9003c070 con 0x7fcb9c09c4a0 2026-02-17T05:51:44.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:44.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: dump: 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: _finish_auth 0 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 1 -- 10.20.193.78:0/2882115499 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb90037440 con 0x7fcb9c09c4a0 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 1 -- 10.20.193.78:0/2882115499 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90037810 con 0x7fcb9c09c4a0 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 5 monclient: authenticate success, global_id 84156 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 4 set_mon_vals no callback set 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcb8bfff640 10 monclient: dump: 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.357 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 10 monclient: shutdown 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 1 -- 10.20.193.78:0/2882115499 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c09c4a0 msgr2=0x7fcb9c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 1 --2- 10.20.193.78:0/2882115499 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c09c4a0 0x7fcb9c13b3a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb900099a0 tx=0x7fcb9002eee0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 1 -- 10.20.193.78:0/2882115499 shutdown_connections 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 1 --2- 10.20.193.78:0/2882115499 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c09c4a0 0x7fcb9c13b3a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 1 -- 10.20.193.78:0/2882115499 >> 10.20.193.78:0/2882115499 conn(0x7fcb9c0a6620 msgr2=0x7fcb9c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 1 -- 10.20.193.78:0/2882115499 shutdown_connections 2026-02-17T05:51:44.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 1 -- 10.20.193.78:0/2882115499 wait complete. 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 10 monclient: build_initial_monmap 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.358+0000 7fcba3722640 10 monclient: monmap: 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:44.359 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 1 Processor -- start 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 1 -- start start 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient: init 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c06c890 0x7fcb9c1ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient(hunting): picked mon.noname-b con 0x7fcb9c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c06d380 0x7fcb9c1cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient(hunting): picked mon.noname-a con 0x7fcb9c06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcb9c09c4a0 0x7fcb9c1cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient(hunting): picked mon.noname-c con 0x7fcb9c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c09bfa0 con 0x7fcb9c06d380 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c09be20 con 0x7fcb9c06c890 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9c13f010 con 0x7fcb9c09c4a0 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba3722640 10 monclient(hunting): authenticate will time out at 949.912567s 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c06d380 0x7fcb9c1cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c06d380 0x7fcb9c1cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38576/0 (socket says 10.20.193.78:38576) 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 -- 10.20.193.78:0/387301668 learned_addr learned my addr 10.20.193.78:0/387301668 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient(hunting): get_auth_request con 0x7fcb9c06d380 auth_method 0 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1c98640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcb9c09c4a0 0x7fcb9c1cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1c98640 10 monclient(hunting): get_auth_request con 0x7fcb9c09c4a0 auth_method 0 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1c98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1c98640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1c98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1497640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c06c890 0x7fcb9c1ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1497640 10 monclient(hunting): get_auth_request con 0x7fcb9c06c890 auth_method 0 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1497640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1497640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1497640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1c98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1c98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1c98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient(hunting): handle_auth_done global_id 84162 payload 824 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcb9c09c4a0 msgr2=0x7fcb9c1cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcb9c09c4a0 0x7fcb9c1cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c06c890 msgr2=0x7fcb9c1ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c06c890 0x7fcb9c1ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient: _finish_hunting 0 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 monclient: found mon.noname-a 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba0c96640 1 -- 10.20.193.78:0/387301668 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c13cfd0 con 0x7fcb9c06d380 2026-02-17T05:51:44.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.359+0000 7fcba1497640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c06c890 0x7fcb9c1ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.361+0000 7fcba0c96640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c06d380 msgr2=0x7fcb9c1cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.361+0000 7fcba0c96640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c06d380 0x7fcb9c1cd500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb9002f410 tx=0x7fcb900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9003c050 con 0x7fcb9c06d380 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: dump: 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.362 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: _finish_auth 0 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 5 monclient: authenticate success, global_id 84162 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 10 monclient: _renew_subs 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c13e480 con 0x7fcb9c06d380 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9003d040 con 0x7fcb9c06d380 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90041d80 con 0x7fcb9c06d380 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 10 monclient: dump: 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb8b7fe640 4 set_mon_vals no callback set 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 10 monclient: _renew_subs 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c09d810 con 0x7fcb9c06d380 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132580 0x7fcb9c134700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c06cc70 con 0x7fcb9c132580 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132580 0x7fcb9c134700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 10 monclient: get_auth_request con 0x7fcb9c132580 auth_method 0 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fcb9002fce0 con 0x7fcb9c06d380 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 10 monclient(hunting): handle_auth_done global_id 44428 payload 824 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcba1497640 1 -- 10.20.193.78:0/387301668 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb9c13ba60 con 0x7fcb9c132580 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.362+0000 7fcb89ffb640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcb6c03deb0 0x7fcb6c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.363+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(222..222 src has 117..222) ==== 4859+0+0 (secure 0 0 0) 0x7fcb90077d80 con 0x7fcb9c06d380 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.363+0000 7fcba1c98640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcb6c03deb0 0x7fcb6c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.363+0000 7fcba1c98640 10 monclient: get_auth_request con 0x7fcb6c03deb0 auth_method 0 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcba1497640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132580 msgr2=0x7fcb9c134700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcba1497640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132580 0x7fcb9c134700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00e700 tx=0x7fcb8c00ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c005b60 con 0x7fcb9c132580 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 10 monclient: dump: 2026-02-17T05:51:44.364 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fcb8c018070 con 0x7fcb9c132580 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132580 msgr2=0x7fcb9c134700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132580 0x7fcb9c134700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00e700 tx=0x7fcb8c00ebd0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132580 msgr2=0x7fcb9c134700 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcb89ffb640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132580 0x7fcb9c134700 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00e700 tx=0x7fcb8c00ebd0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.365+0000 7fcba1c98640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcb6c03deb0 0x7fcb6c040370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb98003a40 tx=0x7fcb98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba3722640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:51:44.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba3722640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132960 0x7fcb9c1d8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba3722640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c135dc0 con 0x7fcb9c132960 2026-02-17T05:51:44.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132960 0x7fcb9c1d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 10 monclient: get_auth_request con 0x7fcb9c132960 auth_method 0 2026-02-17T05:51:44.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 10 monclient(hunting): handle_auth_done global_id 44431 payload 824 2026-02-17T05:51:44.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.368+0000 7fcba1497640 1 -- 10.20.193.78:0/387301668 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcb9c0744d0 con 0x7fcb9c132960 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba1497640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132960 msgr2=0x7fcb9c1d8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba1497640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132960 0x7fcb9c1d8590 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00e700 tx=0x7fcb8c0026e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0070d0 con 0x7fcb9c132960 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 10 monclient: dump: 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7fcb8c0070d0 con 0x7fcb9c132960 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:44.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132960 msgr2=0x7fcb9c1d8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132960 0x7fcb9c1d8590 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00e700 tx=0x7fcb8c0026e0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132960 msgr2=0x7fcb9c1d8590 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcb89ffb640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcb9c132960 0x7fcb9c1d8590 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00e700 tx=0x7fcb8c0026e0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 60, 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 228017, 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.370 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:51:44.371 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.372 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.373 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:51:44.374 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:51:44.375 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcb6c03deb0 msgr2=0x7fcb6c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcb6c03deb0 0x7fcb6c040370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb98003a40 tx=0x7fcb98009040 comp rx=0 tx=0).stop 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 10 monclient: shutdown 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c06d380 msgr2=0x7fcb9c1cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 1 --2- 10.20.193.78:0/387301668 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcb9c06d380 0x7fcb9c1cd500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb9002f410 tx=0x7fcb900047c0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba1c98640 1 -- 10.20.193.78:0/387301668 reap_dead start 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba1c98640 1 -- 10.20.193.78:0/387301668 reap_dead start 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 shutdown_connections 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 >> 10.20.193.78:0/387301668 conn(0x7fcb9c0a6620 msgr2=0x7fcb9c132fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 shutdown_connections 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.371+0000 7fcba3722640 1 -- 10.20.193.78:0/387301668 wait complete. 2026-02-17T05:51:44.376 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:51:44.376 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:51:44.376 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:51:44.376 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:51:44.376 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:51:44.376 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:51:44.376 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:51:44.385 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:51:44.385 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:51:44.385 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:44.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.492+0000 7f854b4a7640 10 monclient: get_monmap_and_config 2026-02-17T05:51:44.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.492+0000 7f854b4a7640 10 monclient: build_initial_monmap 2026-02-17T05:51:44.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.492+0000 7f854b4a7640 10 monclient: monmap: 2026-02-17T05:51:44.493 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:44.492943+0000 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:44.492943+0000 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 1 Processor -- start 2026-02-17T05:51:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 1 -- start start 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient: init 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f854406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient(hunting): picked mon.noname-b con 0x7f854406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f854409c4a0 0x7f854406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient(hunting): picked mon.noname-c con 0x7f854409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8544134df0 0x7f8544135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient(hunting): picked mon.noname-a con 0x7f8544134df0 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f85440a0880 con 0x7f8544134df0 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f854406d6f0 con 0x7f854409c4a0 2026-02-17T05:51:44.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f854406daf0 con 0x7f854406c890 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854b4a7640 10 monclient(hunting): authenticate will time out at 950.046513s 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f8549a1d640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f8544134df0 msgr2=0x7f8544135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f8549a1d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8544134df0 0x7f8544135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f854406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f854406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46068/0 (socket says 10.20.193.78:46068) 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 -- 10.20.193.78:0/2100434737 learned_addr learned my addr 10.20.193.78:0/2100434737 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient(hunting): get_auth_request con 0x7f854406c890 auth_method 0 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f8548a1b640 1 --2- 10.20.193.78:0/2100434737 >> v2:10.20.193.115:3300/0 conn(0x7f854409c4a0 0x7f854406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f8548a1b640 10 monclient(hunting): get_auth_request con 0x7f854409c4a0 auth_method 0 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f8548a1b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f8548a1b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f8548a1b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient(hunting): handle_auth_done global_id 74240 payload 274 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 -- 10.20.193.78:0/2100434737 >> v2:10.20.193.115:3300/0 conn(0x7f854409c4a0 msgr2=0x7f854406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 --2- 10.20.193.78:0/2100434737 >> v2:10.20.193.115:3300/0 conn(0x7f854409c4a0 0x7f854406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 -- 10.20.193.78:0/2100434737 >> v2:10.20.193.78:3300/0 conn(0x7f8544134df0 msgr2=0x7f8544135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 --2- 10.20.193.78:0/2100434737 >> v2:10.20.193.78:3300/0 conn(0x7f8544134df0 0x7f8544135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient: _finish_hunting 0 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 monclient: found mon.noname-b 2026-02-17T05:51:44.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:44.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:44.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.493+0000 7f854921c640 1 -- 10.20.193.78:0/2100434737 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854409cc80 con 0x7f854406c890 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854921c640 1 -- 10.20.193.78:0/2100434737 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 msgr2=0x7f854406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854921c640 1 --2- 10.20.193.78:0/2100434737 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f854406cc70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85440a7600 tx=0x7f853402ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8167e8ad941c5aef server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 1 -- 10.20.193.78:0/2100434737 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853403c070 con 0x7f854406c890 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: dump: 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.504 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: _finish_auth 0 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 1 -- 10.20.193.78:0/2100434737 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853402fcc0 con 0x7f854406c890 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 5 monclient: authenticate success, global_id 74240 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 1 -- 10.20.193.78:0/2100434737 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534037660 con 0x7f854406c890 2026-02-17T05:51:44.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 4 set_mon_vals no callback set 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f8533fff640 10 monclient: dump: 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.506 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 10 monclient: shutdown 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 -- 10.20.193.78:0/2100434737 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 msgr2=0x7f854406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 --2- 10.20.193.78:0/2100434737 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f854406cc70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85440a7600 tx=0x7f853402ee80 comp rx=0 tx=0).stop 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 -- 10.20.193.78:0/2100434737 shutdown_connections 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 --2- 10.20.193.78:0/2100434737 >> v2:10.20.193.78:3300/0 conn(0x7f8544134df0 0x7f8544135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 --2- 10.20.193.78:0/2100434737 >> v2:10.20.193.115:3300/0 conn(0x7f854409c4a0 0x7f854406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 --2- 10.20.193.78:0/2100434737 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f854406cc70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 -- 10.20.193.78:0/2100434737 >> 10.20.193.78:0/2100434737 conn(0x7f85440a6620 msgr2=0x7f85440a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 -- 10.20.193.78:0/2100434737 shutdown_connections 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.496+0000 7f854b4a7640 1 -- 10.20.193.78:0/2100434737 wait complete. 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient: build_initial_monmap 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient: monmap: 2026-02-17T05:51:44.507 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 1 Processor -- start 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 1 -- start start 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient: init 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:44.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f85441cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient(hunting): picked mon.noname-c con 0x7f854406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f854409c4a0 0x7f85441cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient(hunting): picked mon.noname-b con 0x7f854409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8544134df0 0x7f8544121480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient(hunting): picked mon.noname-a con 0x7f8544134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85440a01a0 con 0x7f8544134df0 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85440a0020 con 0x7f854409c4a0 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f854409be20 con 0x7f854406c890 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:44.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854b4a7640 10 monclient(hunting): authenticate will time out at 950.050871s 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f8549a1d640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8544134df0 msgr2=0x7f8544121480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f8549a1d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8544134df0 0x7f8544121480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f85441cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f85441cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46076/0 (socket says 10.20.193.78:46076) 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 1 -- 10.20.193.78:0/1480291056 learned_addr learned my addr 10.20.193.78:0/1480291056 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 10 monclient(hunting): get_auth_request con 0x7f854406c890 auth_method 0 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:44.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.497+0000 7f854921c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 10 monclient(hunting): handle_auth_done global_id 74243 payload 824 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 1 -- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f854409c4a0 msgr2=0x7f85441cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f854409c4a0 0x7f85441cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 1 -- 10.20.193.78:0/1480291056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8544134df0 msgr2=0x7f8544121480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8544134df0 0x7f8544121480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 10 monclient: _finish_hunting 0 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 1 monclient: found mon.noname-c 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.498+0000 7f854921c640 1 -- 10.20.193.78:0/1480291056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8544125410 con 0x7f854406c890 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854921c640 1 -- 10.20.193.78:0/1480291056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 msgr2=0x7f85441cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854921c640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f85441cd480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8534009a40 tx=0x7f8534004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 1 -- 10.20.193.78:0/1480291056 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534047070 con 0x7f854406c890 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: dump: 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: _finish_auth 0 2026-02-17T05:51:44.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 5 monclient: authenticate success, global_id 74243 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 1 -- 10.20.193.78:0/1480291056 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853403f870 con 0x7f854406c890 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 10 monclient: _renew_subs 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85440726c0 con 0x7f854406c890 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 1 -- 10.20.193.78:0/1480291056 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853403c040 con 0x7f854406c890 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f85337fe640 4 set_mon_vals no callback set 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 10 monclient: dump: 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:44.513 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 10 monclient: _renew_subs 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8544124e50 con 0x7f854406c890 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f8531ffb640 1 -- 10.20.193.78:0/1480291056 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f853403fc60 con 0x7f854406c890 2026-02-17T05:51:44.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.500+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8544074980 con 0x7f854406c890 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.501+0000 7f8531ffb640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f851403de60 0x7f8514040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.501+0000 7f8531ffb640 1 -- 10.20.193.78:0/1480291056 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(222..222 src has 117..222) ==== 4859+0+0 (secure 0 0 0) 0x7f8534077340 con 0x7f854406c890 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.501+0000 7f8548a1b640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f851403de60 0x7f8514040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.501+0000 7f8548a1b640 10 monclient: get_auth_request con 0x7f851403de60 auth_method 0 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.503+0000 7f8548a1b640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f851403de60 0x7f8514040320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8538006fd0 tx=0x7f8538008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.504+0000 7f8531ffb640 1 -- 10.20.193.78:0/1480291056 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8534075cf0 con 0x7f854406c890 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.504+0000 7f8531ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:44.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.504+0000 7f8531ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:44.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.620+0000 7f854b4a7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:44.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.620+0000 7f854b4a7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:44.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.620+0000 7f854b4a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:44.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.620+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8544135240 con 0x7f854406c890 2026-02-17T05:51:44.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.620+0000 7f8531ffb640 1 -- 10.20.193.78:0/1480291056 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f85340770d0 con 0x7f854406c890 2026-02-17T05:51:44.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.620+0000 7f8531ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:44.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.620+0000 7f8531ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:44.619 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.621+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f851403de60 msgr2=0x7f8514040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.621+0000 7f854b4a7640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f851403de60 0x7f8514040320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8538006fd0 tx=0x7f8538008040 comp rx=0 tx=0).stop 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.621+0000 7f854b4a7640 10 monclient: shutdown 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.621+0000 7f854b4a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.621+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 msgr2=0x7f85441cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f85441cd480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8534009a40 tx=0x7f8534004830 comp rx=0 tx=0).stop 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 shutdown_connections 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8544134df0 0x7f8544121480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f851403de60 0x7f8514040320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f854409c4a0 0x7f85441cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 --2- 10.20.193.78:0/1480291056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f854406c890 0x7f85441cd480 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 >> 10.20.193.78:0/1480291056 conn(0x7f85440a6620 msgr2=0x7f85440a1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 shutdown_connections 2026-02-17T05:51:44.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:44.622+0000 7f854b4a7640 1 -- 10.20.193.78:0/1480291056 wait complete. 2026-02-17T05:51:47.629 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:47.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.700+0000 7f4a96029640 10 monclient: get_monmap_and_config 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.700+0000 7f4a96029640 10 monclient: build_initial_monmap 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.700+0000 7f4a96029640 10 monclient: monmap: 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:47.700947+0000 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:47.700947+0000 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:47.702 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 1 Processor -- start 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 1 -- start start 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient: init 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2026-02-17T05:51:47.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4a9006c890 0x7f4a900a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient(hunting): picked mon.noname-a con 0x7f4a9006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4a9006cc70 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f4a9013a240 0x7f4a9013c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient(hunting): picked mon.noname-c con 0x7f4a9013a240 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a90074150 con 0x7f4a9006cc70 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9009ef70 con 0x7f4a9006c890 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9009f250 con 0x7f4a9013a240 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a96029640 10 monclient(hunting): authenticate will time out at 953.254551s 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8f7fe640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f4a9006c890 msgr2=0x7f4a900a01f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:47.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8f7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4a9006c890 0x7f4a900a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8effd640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f4a9006cc70 0x7f4a9006d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:37116/0 (socket says 10.20.193.78:37116) 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8effd640 1 -- 10.20.193.78:0/1092054048 learned_addr learned my addr 10.20.193.78:0/1092054048 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8ffff640 1 --2- 10.20.193.78:0/1092054048 >> v2:10.20.193.115:3300/0 conn(0x7f4a9013a240 0x7f4a9013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8ffff640 10 monclient(hunting): get_auth_request con 0x7f4a9013a240 auth_method 0 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:47.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.701+0000 7f4a8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 10 monclient(hunting): handle_auth_done global_id 44437 payload 274 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 1 -- 10.20.193.78:0/1092054048 >> v2:10.20.193.78:3300/0 conn(0x7f4a9006c890 msgr2=0x7f4a900a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 1 --2- 10.20.193.78:0/1092054048 >> v2:10.20.193.78:3300/0 conn(0x7f4a9006c890 0x7f4a900a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 1 -- 10.20.193.78:0/1092054048 >> v1:10.20.193.115:6789/0 conn(0x7f4a9006cc70 legacy=0x7f4a9006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 1 monclient: found mon.noname-c 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:51:47.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.702+0000 7f4a8ffff640 1 -- 10.20.193.78:0/1092054048 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9009df40 con 0x7f4a9013a240 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8ffff640 1 -- 10.20.193.78:0/1092054048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9013a240 msgr2=0x7f4a9013c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8ffff640 1 --2- 10.20.193.78:0/1092054048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9013a240 0x7f4a9013c630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a90069d40 tx=0x7f4a8402f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b12315f2a862327c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 1 -- 10.20.193.78:0/1092054048 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8403c070 con 0x7f4a9013a240 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: dump: 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:47.714 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:47.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 1 -- 10.20.193.78:0/1092054048 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a84037640 con 0x7f4a9013a240 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 1 -- 10.20.193.78:0/1092054048 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84037a10 con 0x7f4a9013a240 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a96029640 5 monclient: authenticate success, global_id 44437 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a96029640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a96029640 4 set_mon_vals no callback set 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.704+0000 7f4a8e7fc640 10 monclient: dump: 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:47.716 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 10 monclient: shutdown 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 -- 10.20.193.78:0/1092054048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9013a240 msgr2=0x7f4a9013c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:47.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 --2- 10.20.193.78:0/1092054048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9013a240 0x7f4a9013c630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a90069d40 tx=0x7f4a8402f1e0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 -- 10.20.193.78:0/1092054048 shutdown_connections 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 --2- 10.20.193.78:0/1092054048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9013a240 0x7f4a9013c630 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 --2- 10.20.193.78:0/1092054048 >> v2:10.20.193.78:3300/0 conn(0x7f4a9006c890 0x7f4a900a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 -- 10.20.193.78:0/1092054048 >> 10.20.193.78:0/1092054048 conn(0x7f4a900a6620 msgr2=0x7f4a900a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 -- 10.20.193.78:0/1092054048 shutdown_connections 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 -- 10.20.193.78:0/1092054048 wait complete. 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 10 monclient: build_initial_monmap 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 10 monclient: monmap: 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:47.718 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 Processor -- start 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 1 -- start start 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.705+0000 7f4a96029640 10 monclient: init 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:51:47.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a9006c890 0x7f4a900731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient(hunting): picked mon.noname-a con 0x7f4a9006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9006cc70 0x7f4a901302b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient(hunting): picked mon.noname-b con 0x7f4a9006cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a9013a240 0x7f4a901307f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient(hunting): picked mon.noname-c con 0x7f4a9013a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9013ccf0 con 0x7f4a9006c890 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9013cb70 con 0x7f4a9006cc70 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9013ce70 con 0x7f4a9013a240 2026-02-17T05:51:47.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a96029640 10 monclient(hunting): authenticate will time out at 953.259199s 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8f7fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a9006c890 msgr2=0x7f4a900731d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a9006c890 0x7f4a900731d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a9013a240 0x7f4a901307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a9013a240 0x7f4a901307f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36348/0 (socket says 10.20.193.78:36348) 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 -- 10.20.193.78:0/3282140521 learned_addr learned my addr 10.20.193.78:0/3282140521 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient(hunting): get_auth_request con 0x7f4a9013a240 auth_method 0 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:47.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8effd640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9006cc70 0x7f4a901302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8effd640 10 monclient(hunting): get_auth_request con 0x7f4a9006cc70 auth_method 0 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient(hunting): handle_auth_done global_id 74246 payload 824 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 -- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9006cc70 msgr2=0x7f4a901302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9006cc70 0x7f4a901302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 -- 10.20.193.78:0/3282140521 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a9006c890 msgr2=0x7f4a900731d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a9006c890 0x7f4a900731d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 monclient: found mon.noname-c 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.706+0000 7f4a8ffff640 1 -- 10.20.193.78:0/3282140521 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9013ba50 con 0x7f4a9013a240 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a8ffff640 1 -- 10.20.193.78:0/3282140521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a9013a240 msgr2=0x7f4a901307f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a8ffff640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a9013a240 0x7f4a901307f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009e90 tx=0x7f4a84004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 1 -- 10.20.193.78:0/3282140521 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84047020 con 0x7f4a9013a240 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:47.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: dump: 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: _finish_auth 0 2026-02-17T05:51:47.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 1 -- 10.20.193.78:0/3282140521 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a84042680 con 0x7f4a9013a240 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a96029640 5 monclient: authenticate success, global_id 74246 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 1 -- 10.20.193.78:0/3282140521 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8403c040 con 0x7f4a9013a240 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a6bfff640 10 monclient: dump: 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:47.725 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a8dffb640 4 set_mon_vals no callback set 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a96029640 10 monclient: _renew_subs 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a96029640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9011d990 con 0x7f4a9013a240 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a96029640 10 monclient: _renew_subs 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a96029640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:47.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.708+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9013be70 con 0x7f4a9013a240 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a6bfff640 1 -- 10.20.193.78:0/3282140521 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4a84042ad0 con 0x7f4a9013a240 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a96029640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a96029640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a96029640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9011d3d0 con 0x7f4a9013a240 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a6bfff640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a6003de60 0x7f4a60040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a6bfff640 1 -- 10.20.193.78:0/3282140521 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(222..222 src has 117..222) ==== 4859+0+0 (secure 0 0 0) 0x7f4a84078560 con 0x7f4a9013a240 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a8f7fe640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a6003de60 0x7f4a60040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.709+0000 7f4a8f7fe640 10 monclient: get_auth_request con 0x7f4a6003de60 auth_method 0 2026-02-17T05:51:47.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.712+0000 7f4a6bfff640 1 -- 10.20.193.78:0/3282140521 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4a84051070 con 0x7f4a9013a240 2026-02-17T05:51:47.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.712+0000 7f4a6bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:47.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.712+0000 7f4a6bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:47.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.712+0000 7f4a8f7fe640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a6003de60 0x7f4a60040320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c003a80 tx=0x7f4a7c009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:47.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.825+0000 7f4a96029640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:47.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.825+0000 7f4a96029640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:47.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.825+0000 7f4a96029640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:47.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.825+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4a900a01f0 con 0x7f4a9013a240 2026-02-17T05:51:47.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.825+0000 7f4a6bfff640 1 -- 10.20.193.78:0/3282140521 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f4a9013ba50 con 0x7f4a9013a240 2026-02-17T05:51:47.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.825+0000 7f4a6bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:47.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.825+0000 7f4a6bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a6003de60 msgr2=0x7f4a60040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a6003de60 0x7f4a60040320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c003a80 tx=0x7f4a7c009270 comp rx=0 tx=0).stop 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 10 monclient: shutdown 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a9013a240 msgr2=0x7f4a901307f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a9013a240 0x7f4a901307f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009e90 tx=0x7f4a84004b40 comp rx=0 tx=0).stop 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 shutdown_connections 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a9013a240 0x7f4a901307f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a6003de60 0x7f4a60040320 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a9006cc70 0x7f4a901302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 --2- 10.20.193.78:0/3282140521 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a9006c890 0x7f4a900731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 >> 10.20.193.78:0/3282140521 conn(0x7f4a900a6620 msgr2=0x7f4a9009c640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 shutdown_connections 2026-02-17T05:51:47.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:47.826+0000 7f4a96029640 1 -- 10.20.193.78:0/3282140521 wait complete. 2026-02-17T05:51:50.832 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:50.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient: get_monmap_and_config 2026-02-17T05:51:50.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient: build_initial_monmap 2026-02-17T05:51:50.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient: monmap: 2026-02-17T05:51:50.903 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:50.902417+0000 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:50.902417+0000 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:50.904 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 1 Processor -- start 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 1 -- start start 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient: init 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient(hunting): picked mon.noname-b con 0x7f32c406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f32c409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f32c406d1b0 0x7f32c4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient(hunting): picked mon.noname-c con 0x7f32c406d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:50.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32c406daf0 con 0x7f32c409c4a0 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f32c40a0880 con 0x7f32c406d1b0 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.902+0000 7f32cc843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32cc843640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f32c409cc80 con 0x7f32c406c890 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32cc843640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32cc843640 10 monclient(hunting): authenticate will time out at 956.456006s 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32c9db7640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f32c409c4a0 legacy=0x7f32c4134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36352/0 (socket says 10.20.193.78:36352) 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 1 -- 10.20.193.78:0/2727768685 learned_addr learned my addr 10.20.193.78:0/2727768685 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient(hunting): get_auth_request con 0x7f32c406c890 auth_method 0 2026-02-17T05:51:50.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32cadb9640 1 --2- 10.20.193.78:0/2727768685 >> v2:10.20.193.115:3300/0 conn(0x7f32c406d1b0 0x7f32c4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32cadb9640 10 monclient(hunting): get_auth_request con 0x7f32c406d1b0 auth_method 0 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32cadb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32cadb9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32cadb9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient(hunting): handle_auth_done global_id 74249 payload 274 2026-02-17T05:51:50.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 1 -- 10.20.193.78:0/2727768685 >> v2:10.20.193.115:3300/0 conn(0x7f32c406d1b0 msgr2=0x7f32c4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:50.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 1 --2- 10.20.193.78:0/2727768685 >> v2:10.20.193.115:3300/0 conn(0x7f32c406d1b0 0x7f32c4135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:50.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 1 -- 10.20.193.78:0/2727768685 >> v1:10.20.193.78:6789/0 conn(0x7f32c409c4a0 legacy=0x7f32c4134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:50.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient: _finish_hunting 0 2026-02-17T05:51:50.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 1 monclient: found mon.noname-b 2026-02-17T05:51:50.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:50.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:50.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.903+0000 7f32ca5b8640 1 -- 10.20.193.78:0/2727768685 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c409cf60 con 0x7f32c406c890 2026-02-17T05:51:50.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.905+0000 7f32ca5b8640 1 -- 10.20.193.78:0/2727768685 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 msgr2=0x7f32c406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:50.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.905+0000 7f32ca5b8640 1 --2- 10.20.193.78:0/2727768685 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c406cc70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009970 tx=0x7f32bc02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92055cf77421e75e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:50.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 1 -- 10.20.193.78:0/2727768685 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc03c070 con 0x7f32c406c890 2026-02-17T05:51:50.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: dump: 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:50.916 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: _finish_auth 0 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: _check_auth_tickets 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 1 -- 10.20.193.78:0/2727768685 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32bc02fcd0 con 0x7f32c406c890 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 5 monclient: authenticate success, global_id 74249 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 1 -- 10.20.193.78:0/2727768685 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc037660 con 0x7f32c406c890 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32c95b6640 10 monclient: dump: 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:50.917 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 4 set_mon_vals no callback set 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 10 monclient: shutdown 2026-02-17T05:51:50.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 1 -- 10.20.193.78:0/2727768685 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 msgr2=0x7f32c406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 1 --2- 10.20.193.78:0/2727768685 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c406cc70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009970 tx=0x7f32bc02ee90 comp rx=0 tx=0).stop 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 1 -- 10.20.193.78:0/2727768685 shutdown_connections 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 1 --2- 10.20.193.78:0/2727768685 >> v2:10.20.193.115:3300/0 conn(0x7f32c406d1b0 0x7f32c4135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 1 --2- 10.20.193.78:0/2727768685 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c406cc70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 1 -- 10.20.193.78:0/2727768685 >> 10.20.193.78:0/2727768685 conn(0x7f32c40a6620 msgr2=0x7f32c40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 1 -- 10.20.193.78:0/2727768685 shutdown_connections 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 1 -- 10.20.193.78:0/2727768685 wait complete. 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 10 monclient: build_initial_monmap 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.906+0000 7f32cc843640 10 monclient: monmap: 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:50.919 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 1 Processor -- start 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 1 -- start start 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient: init 2026-02-17T05:51:50.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c4135520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient(hunting): picked mon.noname-c con 0x7f32c406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f32c406d1b0 0x7f32c4135a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient(hunting): picked mon.noname-b con 0x7f32c406d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32c409c4a0 0x7f32c4124fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient(hunting): picked mon.noname-a con 0x7f32c409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32c40a01a0 con 0x7f32c409c4a0 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32c40a0020 con 0x7f32c406d1b0 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:50.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f32c409be20 con 0x7f32c406c890 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cc843640 10 monclient(hunting): authenticate will time out at 956.460374s 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cadb9640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32c409c4a0 msgr2=0x7f32c4124fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32cadb9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32c409c4a0 0x7f32c4124fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c4135520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c4135520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36356/0 (socket says 10.20.193.78:36356) 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 -- 10.20.193.78:0/1884427588 learned_addr learned my addr 10.20.193.78:0/1884427588 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient(hunting): get_auth_request con 0x7f32c406c890 auth_method 0 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:50.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient(hunting): handle_auth_done global_id 74252 payload 824 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 -- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f32c406d1b0 msgr2=0x7f32c4135a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f32c406d1b0 0x7f32c4135a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 -- 10.20.193.78:0/1884427588 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32c409c4a0 msgr2=0x7f32c4124fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32c409c4a0 0x7f32c4124fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient: _finish_hunting 0 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 monclient: found mon.noname-c 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:50.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.907+0000 7f32ca5b8640 1 -- 10.20.193.78:0/1884427588 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c4136ea0 con 0x7f32c406c890 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32ca5b8640 1 -- 10.20.193.78:0/1884427588 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 msgr2=0x7f32c4135520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32ca5b8640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c4135520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009970 tx=0x7f32bc004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 1 -- 10.20.193.78:0/1884427588 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc03c040 con 0x7f32c406c890 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: dump: 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:50.924 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: _finish_auth 0 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: _check_auth_tickets 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 1 -- 10.20.193.78:0/1884427588 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32bc047070 con 0x7f32c406c890 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 1 -- 10.20.193.78:0/1884427588 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc042890 con 0x7f32c406c890 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32cc843640 5 monclient: authenticate success, global_id 74252 2026-02-17T05:51:50.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32c8db5640 4 set_mon_vals no callback set 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32cc843640 10 monclient: _renew_subs 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32cc843640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c409d810 con 0x7f32c406c890 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32aeffd640 10 monclient: dump: 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:50.926 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32cc843640 10 monclient: _renew_subs 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32cc843640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.909+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c4128980 con 0x7f32c406c890 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.910+0000 7f32aeffd640 1 -- 10.20.193.78:0/1884427588 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f32bc042a30 con 0x7f32c406c890 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.910+0000 7f32cc843640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.910+0000 7f32cc843640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.910+0000 7f32cc843640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.910+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c406f580 con 0x7f32c406c890 2026-02-17T05:51:50.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.910+0000 7f32aeffd640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f329403deb0 0x7f3294040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:50.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.910+0000 7f32aeffd640 1 -- 10.20.193.78:0/1884427588 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(222..222 src has 117..222) ==== 4859+0+0 (secure 0 0 0) 0x7f32bc077c70 con 0x7f32c406c890 2026-02-17T05:51:50.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.911+0000 7f32c9db7640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f329403deb0 0x7f3294040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:50.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.911+0000 7f32c9db7640 10 monclient: get_auth_request con 0x7f329403deb0 auth_method 0 2026-02-17T05:51:50.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.913+0000 7f32aeffd640 1 -- 10.20.193.78:0/1884427588 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f32c406f580 con 0x7f32c406c890 2026-02-17T05:51:50.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.913+0000 7f32aeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:50.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.913+0000 7f32aeffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:50.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:50.913+0000 7f32c9db7640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f329403deb0 0x7f3294040370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f32c4127d10 tx=0x7f32b0006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:51.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.029+0000 7f32cc843640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:51.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.029+0000 7f32cc843640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:51.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.029+0000 7f32cc843640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:51.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.029+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f32c406cc70 con 0x7f32c406c890 2026-02-17T05:51:51.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.029+0000 7f32aeffd640 1 -- 10.20.193.78:0/1884427588 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f32bc04ae20 con 0x7f32c406c890 2026-02-17T05:51:51.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.029+0000 7f32aeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:51.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.029+0000 7f32aeffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:51.029 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:51.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f329403deb0 msgr2=0x7f3294040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:51.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f329403deb0 0x7f3294040370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f32c4127d10 tx=0x7f32b0006d90 comp rx=0 tx=0).stop 2026-02-17T05:51:51.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 10 monclient: shutdown 2026-02-17T05:51:51.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:51.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 msgr2=0x7f32c4135520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:51.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c4135520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009970 tx=0x7f32bc004830 comp rx=0 tx=0).stop 2026-02-17T05:51:51.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 shutdown_connections 2026-02-17T05:51:51.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32c409c4a0 0x7f32c4124fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:51.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f329403deb0 0x7f3294040370 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:51.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f32c406d1b0 0x7f32c4135a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:51.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 --2- 10.20.193.78:0/1884427588 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32c406c890 0x7f32c4135520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:51.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 >> 10.20.193.78:0/1884427588 conn(0x7f32c40a6620 msgr2=0x7f32c4070310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:51.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 shutdown_connections 2026-02-17T05:51:51.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:51.030+0000 7f32cc843640 1 -- 10.20.193.78:0/1884427588 wait complete. 2026-02-17T05:51:54.038 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:51:54.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.108+0000 7fa1027c1640 10 monclient: get_monmap_and_config 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.108+0000 7fa1027c1640 10 monclient: build_initial_monmap 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.108+0000 7fa1027c1640 10 monclient: monmap: 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:54.109023+0000 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:54.109023+0000 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:54.110 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 1 Processor -- start 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 1 -- start start 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient: init 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:54.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa0fc06c890 0x7fa0fc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient(hunting): picked mon.noname-c con 0x7fa0fc06c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa0fc09c4a0 0x7fa0fc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient(hunting): picked mon.noname-a con 0x7fa0fc09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient(hunting): picked mon.noname-b con 0x7fa0fc134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa0fc0a0880 con 0x7fa0fc09c4a0 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa0fc06d6f0 con 0x7fa0fc06c890 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:54.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc06daf0 con 0x7fa0fc134df0 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa1027c1640 10 monclient(hunting): authenticate will time out at 959.662557s 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa0fb7fe640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fa0fc09c4a0 msgr2=0x7fa0fc06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa0fb7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa0fc09c4a0 0x7fa0fc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc135280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36362/0 (socket says 10.20.193.78:36362) 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 -- 10.20.193.78:0/1194779567 learned_addr learned my addr 10.20.193.78:0/1194779567 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient(hunting): get_auth_request con 0x7fa0fc134df0 auth_method 0 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:54.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa0fbfff640 1 --2- 10.20.193.78:0/1194779567 >> v2:10.20.193.115:3300/0 conn(0x7fa0fc06c890 0x7fa0fc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient(hunting): handle_auth_done global_id 74255 payload 274 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 -- 10.20.193.78:0/1194779567 >> v2:10.20.193.115:3300/0 conn(0x7fa0fc06c890 msgr2=0x7fa0fc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 --2- 10.20.193.78:0/1194779567 >> v2:10.20.193.115:3300/0 conn(0x7fa0fc06c890 0x7fa0fc06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 -- 10.20.193.78:0/1194779567 >> v2:10.20.193.78:3300/0 conn(0x7fa0fc09c4a0 msgr2=0x7fa0fc06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 --2- 10.20.193.78:0/1194779567 >> v2:10.20.193.78:3300/0 conn(0x7fa0fc09c4a0 0x7fa0fc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient: _finish_hunting 0 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 monclient: found mon.noname-b 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:54.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.109+0000 7fa100d37640 1 -- 10.20.193.78:0/1194779567 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc09cc80 con 0x7fa0fc134df0 2026-02-17T05:51:54.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa100d37640 1 -- 10.20.193.78:0/1194779567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 msgr2=0x7fa0fc135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:54.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa100d37640 1 --2- 10.20.193.78:0/1194779567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc135280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc06a480 tx=0x7fa0f002f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e81554820083868 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:54.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 1 -- 10.20.193.78:0/1194779567 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00373f0 con 0x7fa0fc134df0 2026-02-17T05:51:54.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:54.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:54.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: dump: 2026-02-17T05:51:54.122 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:54.122 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: _finish_auth 0 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: _check_auth_tickets 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:54.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 1 -- 10.20.193.78:0/1194779567 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0037590 con 0x7fa0fc134df0 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 5 monclient: authenticate success, global_id 74255 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 1 -- 10.20.193.78:0/1194779567 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0037960 con 0x7fa0fc134df0 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 4 set_mon_vals no callback set 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa0faffd640 10 monclient: dump: 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:54.124 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 10 monclient: shutdown 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 -- 10.20.193.78:0/1194779567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 msgr2=0x7fa0fc135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1194779567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc135280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc06a480 tx=0x7fa0f002f200 comp rx=0 tx=0).stop 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 -- 10.20.193.78:0/1194779567 shutdown_connections 2026-02-17T05:51:54.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1194779567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc135280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1194779567 >> v2:10.20.193.78:3300/0 conn(0x7fa0fc09c4a0 0x7fa0fc06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1194779567 >> v2:10.20.193.115:3300/0 conn(0x7fa0fc06c890 0x7fa0fc06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 -- 10.20.193.78:0/1194779567 >> 10.20.193.78:0/1194779567 conn(0x7fa0fc0a6620 msgr2=0x7fa0fc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 -- 10.20.193.78:0/1194779567 shutdown_connections 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.112+0000 7fa1027c1640 1 -- 10.20.193.78:0/1194779567 wait complete. 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient: build_initial_monmap 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient: monmap: 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:54.126 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 1 Processor -- start 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 1 -- start start 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient: init 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0fc06c890 0x7fa0fc1cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:54.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient(hunting): picked mon.noname-a con 0x7fa0fc06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0fc09c4a0 0x7fa0fc122d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient(hunting): picked mon.noname-b con 0x7fa0fc09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc123250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient(hunting): picked mon.noname-c con 0x7fa0fc134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc0a01a0 con 0x7fa0fc06c890 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc0a0020 con 0x7fa0fc09c4a0 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc09be20 con 0x7fa0fc134df0 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:54.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa1027c1640 10 monclient(hunting): authenticate will time out at 959.666713s 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa0fbfff640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0fc06c890 msgr2=0x7fa0fc1cf6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa0fbfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0fc06c890 0x7fa0fc1cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc123250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc123250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36366/0 (socket says 10.20.193.78:36366) 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 -- 10.20.193.78:0/1974551015 learned_addr learned my addr 10.20.193.78:0/1974551015 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient(hunting): get_auth_request con 0x7fa0fc134df0 auth_method 0 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:54.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa0fb7fe640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0fc09c4a0 0x7fa0fc122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa0fb7fe640 10 monclient(hunting): get_auth_request con 0x7fa0fc09c4a0 auth_method 0 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa0fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa0fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa0fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient(hunting): handle_auth_done global_id 74258 payload 824 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 -- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0fc09c4a0 msgr2=0x7fa0fc122d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0fc09c4a0 0x7fa0fc122d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 -- 10.20.193.78:0/1974551015 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0fc06c890 msgr2=0x7fa0fc1cf6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0fc06c890 0x7fa0fc1cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient: _finish_hunting 0 2026-02-17T05:51:54.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 monclient: found mon.noname-c 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.113+0000 7fa100d37640 1 -- 10.20.193.78:0/1974551015 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc127680 con 0x7fa0fc134df0 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa100d37640 1 -- 10.20.193.78:0/1974551015 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 msgr2=0x7fa0fc123250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa100d37640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc123250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0f002f710 tx=0x7fa0f0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 1 -- 10.20.193.78:0/1974551015 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0046020 con 0x7fa0fc134df0 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: dump: 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:54.131 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: _finish_auth 0 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 1 -- 10.20.193.78:0/1974551015 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0041870 con 0x7fa0fc134df0 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:54.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 1 -- 10.20.193.78:0/1974551015 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0041c40 con 0x7fa0fc134df0 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 10 monclient: dump: 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:54.133 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 5 monclient: authenticate success, global_id 74258 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0fa7fc640 4 set_mon_vals no callback set 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 10 monclient: _renew_subs 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc1269c0 con 0x7fa0fc134df0 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 10 monclient: _renew_subs 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc0a1580 con 0x7fa0fc134df0 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:54.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc074980 con 0x7fa0fc134df0 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.116+0000 7fa0f8ff9640 1 -- 10.20.193.78:0/1974551015 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa0f004b400 con 0x7fa0fc134df0 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.117+0000 7fa0f8ff9640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0cc03deb0 0x7fa0cc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.117+0000 7fa0f8ff9640 1 -- 10.20.193.78:0/1974551015 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(222..222 src has 117..222) ==== 4859+0+0 (secure 0 0 0) 0x7fa0f0078ab0 con 0x7fa0fc134df0 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.117+0000 7fa0fbfff640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0cc03deb0 0x7fa0cc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.117+0000 7fa0fbfff640 10 monclient: get_auth_request con 0x7fa0cc03deb0 auth_method 0 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.119+0000 7fa0f8ff9640 1 -- 10.20.193.78:0/1974551015 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa0f0050020 con 0x7fa0fc134df0 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.119+0000 7fa0f8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.119+0000 7fa0f8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:54.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.120+0000 7fa0fbfff640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0cc03deb0 0x7fa0cc040370 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8006fd0 tx=0x7fa0e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:54.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.234+0000 7fa1027c1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:51:54.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.234+0000 7fa1027c1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:54.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.234+0000 7fa1027c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:54.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:54.234+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa0fc125290 con 0x7fa0fc134df0 2026-02-17T05:51:57.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:57.113+0000 7fa0d77fe640 10 monclient: tick 2026-02-17T05:51:57.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:57.113+0000 7fa0d77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:57.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:57.113+0000 7fa0d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:57.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:57.113+0000 7fa0d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:58.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:58.113+0000 7fa0d77fe640 10 monclient: tick 2026-02-17T05:51:58.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:58.113+0000 7fa0d77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:58.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:58.113+0000 7fa0d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:59.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.113+0000 7fa0d77fe640 10 monclient: tick 2026-02-17T05:51:59.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.113+0000 7fa0d77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:59.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.113+0000 7fa0d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:59.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.116+0000 7fa0f8ff9640 1 -- 10.20.193.78:0/1974551015 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7fa0fc127680 con 0x7fa0fc134df0 2026-02-17T05:51:59.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.116+0000 7fa0f8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:51:59.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.116+0000 7fa0f8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:59.116 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":64,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:51:59.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0cc03deb0 msgr2=0x7fa0cc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0cc03deb0 0x7fa0cc040370 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8006fd0 tx=0x7fa0e8008040 comp rx=0 tx=0).stop 2026-02-17T05:51:59.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 10 monclient: shutdown 2026-02-17T05:51:59.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:59.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 msgr2=0x7fa0fc123250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc123250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0f002f710 tx=0x7fa0f0004b40 comp rx=0 tx=0).stop 2026-02-17T05:51:59.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 shutdown_connections 2026-02-17T05:51:59.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa0fc134df0 0x7fa0fc123250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa0cc03deb0 0x7fa0cc040370 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa0fc09c4a0 0x7fa0fc122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 --2- 10.20.193.78:0/1974551015 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa0fc06c890 0x7fa0fc1cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 >> 10.20.193.78:0/1974551015 conn(0x7fa0fc0a6620 msgr2=0x7fa0fc135850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:59.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 shutdown_connections 2026-02-17T05:51:59.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.118+0000 7fa1027c1640 1 -- 10.20.193.78:0/1974551015 wait complete. 2026-02-17T05:51:59.127 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:51:59.127 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:51:59.128 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:1042: done (2 left) 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:1043: done (1 left) 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:1044: done (0 left) 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:1045: rollback oid 2 current snap is 104 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 2 to 82 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:1046: read oid 16 snap -1 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:1046: expect (ObjNum 382 snap 101 seq_num 382) 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:1047: write (excl) oid 4 current snap is 104 2026-02-17T05:51:59.173 INFO:tasks.rados.rados.0.trial115.stdout:1047: seq_num 405 ranges {406835=642554,1532972=506795,2772312=34523} 2026-02-17T05:51:59.176 INFO:tasks.rados.rados.0.trial115.stdout:1047: writing trial11545420-4 from 406835 to 1049389 tid 1 2026-02-17T05:51:59.180 INFO:tasks.rados.rados.0.trial115.stdout:1047: writing trial11545420-4 from 1532972 to 2039767 tid 2 2026-02-17T05:51:59.183 INFO:tasks.rados.rados.0.trial115.stdout:1047: writing trial11545420-4 from 2772312 to 2806835 tid 3 2026-02-17T05:51:59.183 INFO:tasks.rados.rados.0.trial115.stdout:1048: rollback oid 18 current snap is 104 2026-02-17T05:51:59.183 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 103 2026-02-17T05:51:59.183 INFO:tasks.rados.rados.0.trial115.stdout:1045: finishing rollback tid 0 to trial11545420-2 2026-02-17T05:51:59.184 INFO:tasks.rados.rados.0.trial115.stdout:1045: finishing rollback tid 1 to trial11545420-2 2026-02-17T05:51:59.184 INFO:tasks.rados.rados.0.trial115.stdout:1045: finishing rollback tid 2 to trial11545420-2 2026-02-17T05:51:59.184 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 435 (ObjNum 248 snap 49 seq_num 248) dirty exists 2026-02-17T05:51:59.184 INFO:tasks.rados.rados.0.trial115.stdout:1046: expect (ObjNum 382 snap 101 seq_num 382) 2026-02-17T05:51:59.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.199+0000 7f2290fcc640 10 monclient: get_monmap_and_config 2026-02-17T05:51:59.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.199+0000 7f2290fcc640 10 monclient: build_initial_monmap 2026-02-17T05:51:59.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.199+0000 7f2290fcc640 10 monclient: monmap: 2026-02-17T05:51:59.198 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:59.198 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.198 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:59.200086+0000 2026-02-17T05:51:59.198 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:59.200086+0000 2026-02-17T05:51:59.199 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:59.199 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.199 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:59.199 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:59.199 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:59.199 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:59.199 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:59.199 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 1 Processor -- start 2026-02-17T05:51:59.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 1 -- start start 2026-02-17T05:51:59.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient: init 2026-02-17T05:51:59.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:59.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c06c890 0x7f228c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient(hunting): picked mon.noname-b con 0x7f228c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f228c09c4a0 0x7f228c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient(hunting): picked mon.noname-a con 0x7f228c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f228c134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228c06daf0 con 0x7f228c134df0 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f228c0a0880 con 0x7f228c09c4a0 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f228c06d6f0 con 0x7f228c06c890 2026-02-17T05:51:59.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2290fcc640 10 monclient(hunting): authenticate will time out at 964.753693s 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2289d74640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f228c09c4a0 msgr2=0x7f228c06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228ad76640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f228c134df0 legacy=0x7f228c135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f2289d74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f228c09c4a0 0x7f228c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c06c890 0x7f228c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c06c890 0x7f228c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38640/0 (socket says 10.20.193.78:38640) 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 1 -- 10.20.193.78:0/2597541779 learned_addr learned my addr 10.20.193.78:0/2597541779 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient(hunting): get_auth_request con 0x7f228c06c890 auth_method 0 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient(hunting): handle_auth_done global_id 74261 payload 274 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 1 -- 10.20.193.78:0/2597541779 >> v2:10.20.193.78:3300/0 conn(0x7f228c09c4a0 msgr2=0x7f228c06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 1 --2- 10.20.193.78:0/2597541779 >> v2:10.20.193.78:3300/0 conn(0x7f228c09c4a0 0x7f228c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 1 -- 10.20.193.78:0/2597541779 >> v1:10.20.193.78:6789/0 conn(0x7f228c134df0 legacy=0x7f228c135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient: _finish_hunting 0 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 1 monclient: found mon.noname-b 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.200+0000 7f228a575640 1 -- 10.20.193.78:0/2597541779 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c09cc80 con 0x7f228c06c890 2026-02-17T05:51:59.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f228a575640 1 -- 10.20.193.78:0/2597541779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c06c890 msgr2=0x7f228c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f228a575640 1 --2- 10.20.193.78:0/2597541779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c06c890 0x7f228c06cc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2280009970 tx=0x7f228002eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c669366c31465b3a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 1 -- 10.20.193.78:0/2597541779 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228003c070 con 0x7f228c06c890 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: dump: 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.212 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: _finish_auth 0 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: _check_auth_tickets 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 1 -- 10.20.193.78:0/2597541779 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f228002fdd0 con 0x7f228c06c890 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 5 monclient: authenticate success, global_id 74261 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 1 -- 10.20.193.78:0/2597541779 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280037660 con 0x7f228c06c890 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2289573640 10 monclient: dump: 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 4 set_mon_vals no callback set 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 10 monclient: shutdown 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 1 -- 10.20.193.78:0/2597541779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c06c890 msgr2=0x7f228c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 1 --2- 10.20.193.78:0/2597541779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c06c890 0x7f228c06cc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2280009970 tx=0x7f228002eeb0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 1 -- 10.20.193.78:0/2597541779 shutdown_connections 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 1 --2- 10.20.193.78:0/2597541779 >> v2:10.20.193.78:3300/0 conn(0x7f228c09c4a0 0x7f228c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 1 --2- 10.20.193.78:0/2597541779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c06c890 0x7f228c06cc70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 1 -- 10.20.193.78:0/2597541779 >> 10.20.193.78:0/2597541779 conn(0x7f228c0a6620 msgr2=0x7f228c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.203+0000 7f2290fcc640 1 -- 10.20.193.78:0/2597541779 shutdown_connections 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 1 -- 10.20.193.78:0/2597541779 wait complete. 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 10 monclient: build_initial_monmap 2026-02-17T05:51:59.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 10 monclient: monmap: 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 1 Processor -- start 2026-02-17T05:51:59.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 1 -- start start 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 10 monclient: init 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f228c06c890 0x7f228c1cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.204+0000 7f2290fcc640 10 monclient(hunting): picked mon.noname-b con 0x7f228c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09c4a0 0x7f228c134040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 10 monclient(hunting): picked mon.noname-c con 0x7f228c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228c134df0 0x7f228c134580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 10 monclient(hunting): picked mon.noname-a con 0x7f228c134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f228c0a01a0 con 0x7f228c134df0 2026-02-17T05:51:59.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f228c0a0020 con 0x7f228c06c890 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f228c09be20 con 0x7f228c09c4a0 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2290fcc640 10 monclient(hunting): authenticate will time out at 964.758025s 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f228ad76640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228c134df0 msgr2=0x7f228c134580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f228ad76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228c134df0 0x7f228c134580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09c4a0 0x7f228c134040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09c4a0 0x7f228c134040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38652/0 (socket says 10.20.193.78:38652) 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 -- 10.20.193.78:0/3847243199 learned_addr learned my addr 10.20.193.78:0/3847243199 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient(hunting): get_auth_request con 0x7f228c09c4a0 auth_method 0 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f228a575640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f228c06c890 0x7f228c1cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f228a575640 10 monclient(hunting): get_auth_request con 0x7f228c06c890 auth_method 0 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f228a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f228a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f228a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient(hunting): handle_auth_done global_id 74264 payload 824 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f228c06c890 msgr2=0x7f228c1cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f228c06c890 0x7f228c1cf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228c134df0 msgr2=0x7f228c134580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f228c134df0 0x7f228c134580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient: _finish_hunting 0 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 monclient: found mon.noname-c 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.205+0000 7f2289d74640 1 -- 10.20.193.78:0/3847243199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c134ac0 con 0x7f228c09c4a0 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2289d74640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09c4a0 msgr2=0x7f228c134040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2289d74640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09c4a0 0x7f228c134040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f227400b780 tx=0x7f227400bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274005270 con 0x7f228c09c4a0 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: dump: 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: _finish_auth 0 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2274010070 con 0x7f228c09c4a0 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274005650 con 0x7f228c09c4a0 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2272ffd640 10 monclient: dump: 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.222 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2290fcc640 5 monclient: authenticate success, global_id 74264 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2288d72640 4 set_mon_vals no callback set 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2290fcc640 10 monclient: _renew_subs 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2290fcc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c072730 con 0x7f228c09c4a0 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2290fcc640 10 monclient: _renew_subs 2026-02-17T05:51:59.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2290fcc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c1274a0 con 0x7f228c09c4a0 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.207+0000 7f2290fcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f2290fcc640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09db50 0x7f228c09df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f2290fcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f228c1359e0 con 0x7f228c09db50 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09db50 0x7f228c09df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 10 monclient: get_auth_request con 0x7f228c09db50 auth_method 0 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:59.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f227400c520 con 0x7f228c09c4a0 2026-02-17T05:51:59.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:59.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 10 monclient(hunting): handle_auth_done global_id 74267 payload 824 2026-02-17T05:51:59.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228a575640 1 -- 10.20.193.78:0/3847243199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f228c09d760 con 0x7f228c09db50 2026-02-17T05:51:59.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f2272ffd640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225803de60 0x7f2258040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(223..223 src has 117..223) ==== 4831+0+0 (secure 0 0 0) 0x7f2274052090 con 0x7f228c09c4a0 2026-02-17T05:51:59.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228ad76640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225803de60 0x7f2258040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.208+0000 7f228ad76640 10 monclient: get_auth_request con 0x7f225803de60 auth_method 0 2026-02-17T05:51:59.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f228a575640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09db50 msgr2=0x7f228c09df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:59.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f228a575640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09db50 0x7f228c09df30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f228c09c880 tx=0x7f22800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280047070 con 0x7f228c09db50 2026-02-17T05:51:59.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:59.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 10 monclient: dump: 2026-02-17T05:51:59.228 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.228 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f22800376d0 con 0x7f228c09db50 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09db50 msgr2=0x7f228c09df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09db50 0x7f228c09df30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f228c09c880 tx=0x7f22800047c0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09db50 msgr2=0x7f228c09df30 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.210+0000 7f2272ffd640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09db50 0x7f228c09df30 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f228c09c880 tx=0x7f22800047c0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.211+0000 7f228ad76640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225803de60 0x7f2258040320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f227c003d30 tx=0x7f227c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f2290fcc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f2290fcc640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09e470 0x7f228c1da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f2290fcc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f228c125de0 con 0x7f228c09e470 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09e470 0x7f228c1da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 10 monclient: get_auth_request con 0x7f228c09e470 auth_method 0 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:59.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 10 monclient(hunting): handle_auth_done global_id 74270 payload 824 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.214+0000 7f228a575640 1 -- 10.20.193.78:0/3847243199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f228c0744d0 con 0x7f228c09e470 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f228a575640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09e470 msgr2=0x7f228c1da6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f228a575640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09e470 0x7f228c1da6b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f228002eef0 tx=0x7f2280009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228003c040 con 0x7f228c09e470 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 10 monclient: dump: 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.231 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4696 (secure 0 0 0) 0x7f2280043e50 con 0x7f228c09e470 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09e470 msgr2=0x7f228c1da6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09e470 0x7f228c1da6b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f228002eef0 tx=0x7f2280009340 comp rx=0 tx=0).stop 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09e470 msgr2=0x7f228c1da6b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2272ffd640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09e470 0x7f228c1da6b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.232 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 64, 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 91108, 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:51:59.233 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:51:59.234 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:51:59.235 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:59.236 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:59.237 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:59.238 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.239 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:59.240 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.241 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:51:59.242 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225803de60 msgr2=0x7f2258040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f225803de60 0x7f2258040320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f227c003d30 tx=0x7f227c0091e0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 10 monclient: shutdown 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09c4a0 msgr2=0x7f228c134040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 1 --2- 10.20.193.78:0/3847243199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f228c09c4a0 0x7f228c134040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f227400b780 tx=0x7f227400bc50 comp rx=0 tx=0).stop 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f228ad76640 1 -- 10.20.193.78:0/3847243199 reap_dead start 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f228ad76640 1 -- 10.20.193.78:0/3847243199 reap_dead start 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 shutdown_connections 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 >> 10.20.193.78:0/3847243199 conn(0x7f228c0a6620 msgr2=0x7f228c135bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 shutdown_connections 2026-02-17T05:51:59.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.216+0000 7f2290fcc640 1 -- 10.20.193.78:0/3847243199 wait complete. 2026-02-17T05:51:59.243 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:51:59.244 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:51:59.244 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: finishing write tid 1 to trial11545420-4 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: finishing write tid 2 to trial11545420-4 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: finishing write tid 3 to trial11545420-4 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: finishing write tid 4 to trial11545420-4 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: finishing write tid 5 to trial11545420-4 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: oid 4 updating version 0 to 298 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: oid 4 version 298 is already newer than 297 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: oid 4 updating version 298 to 299 2026-02-17T05:51:59.261 INFO:tasks.rados.rados.0.trial115.stdout:1047: oid 4 version 299 is already newer than 296 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 299 (ObjNum 405 snap 104 seq_num 405) dirty exists 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:1047: left oid 4 (ObjNum 405 snap 104 seq_num 405) 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:1048: finishing rollback tid 0 to trial11545420-18 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:1048: finishing rollback tid 1 to trial11545420-18 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:1045: done (3 left) 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:1046: done (2 left) 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:1047: done (1 left) 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:1049: write oid 16 current snap is 104 2026-02-17T05:51:59.262 INFO:tasks.rados.rados.0.trial115.stdout:1049: seq_num 406 ranges {423642=717803,1875101=766806} 2026-02-17T05:51:59.266 INFO:tasks.rados.rados.0.trial115.stdout:1049: writing trial11545420-16 from 423642 to 1141445 tid 1 2026-02-17T05:51:59.272 INFO:tasks.rados.rados.0.trial115.stdout:1049: writing trial11545420-16 from 1875101 to 2641907 tid 2 2026-02-17T05:51:59.272 INFO:tasks.rados.rados.0.trial115.stdout:1050: read oid 27 snap -1 2026-02-17T05:51:59.272 INFO:tasks.rados.rados.0.trial115.stdout:1050: expect deleted 2026-02-17T05:51:59.272 INFO:tasks.rados.rados.0.trial115.stdout:1051: write (excl) oid 7 current snap is 104 2026-02-17T05:51:59.272 INFO:tasks.rados.rados.0.trial115.stdout:1051: seq_num 407 ranges {440449=793052,1933583=510464} 2026-02-17T05:51:59.276 INFO:tasks.rados.rados.0.trial115.stdout:1051: writing trial11545420-7 from 440449 to 1233501 tid 1 2026-02-17T05:51:59.281 INFO:tasks.rados.rados.0.trial115.stdout:1051: writing trial11545420-7 from 1933583 to 2444047 tid 2 2026-02-17T05:51:59.281 INFO:tasks.rados.rados.0.trial115.stdout:1052: delete oid 19 current snap is 104 2026-02-17T05:51:59.281 INFO:tasks.rados.rados.0.trial115.stdout:1048: finishing rollback tid 2 to trial11545420-18 2026-02-17T05:51:59.281 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 404 (ObjNum 404 snap 103 seq_num 404) dirty exists 2026-02-17T05:51:59.281 INFO:tasks.rados.rados.0.trial115.stdout:1049: finishing write tid 1 to trial11545420-16 2026-02-17T05:51:59.281 INFO:tasks.rados.rados.0.trial115.stdout:1049: finishing write tid 2 to trial11545420-16 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1049: finishing write tid 3 to trial11545420-16 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1049: finishing write tid 4 to trial11545420-16 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1049: oid 16 updating version 0 to 501 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1049: oid 16 updating version 501 to 502 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1049: oid 16 updating version 502 to 503 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 503 (ObjNum 406 snap 104 seq_num 406) dirty exists 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1049: left oid 16 (ObjNum 406 snap 104 seq_num 406) 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1051: finishing write tid 1 to trial11545420-7 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1048: done (4 left) 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1049: done (3 left) 2026-02-17T05:51:59.282 INFO:tasks.rados.rados.0.trial115.stdout:1050: done (2 left) 2026-02-17T05:51:59.283 INFO:tasks.rados.rados.0.trial115.stdout:1052: done (1 left) 2026-02-17T05:51:59.283 INFO:tasks.rados.rados.0.trial115.stdout:1053: delete oid 24 current snap is 104 2026-02-17T05:51:59.284 INFO:tasks.rados.rados.0.trial115.stdout:1051: finishing write tid 2 to trial11545420-7 2026-02-17T05:51:59.284 INFO:tasks.rados.rados.0.trial115.stdout:1053: done (1 left) 2026-02-17T05:51:59.284 INFO:tasks.rados.rados.0.trial115.stdout:1054: read oid 10 snap -1 2026-02-17T05:51:59.284 INFO:tasks.rados.rados.0.trial115.stdout:1054: expect deleted 2026-02-17T05:51:59.284 INFO:tasks.rados.rados.0.trial115.stdout:1055: write (excl) oid 27 current snap is 104 2026-02-17T05:51:59.284 INFO:tasks.rados.rados.0.trial115.stdout:1055: seq_num 410 ranges {490870=618799,1592676=774144,2793074=97796} 2026-02-17T05:51:59.288 INFO:tasks.rados.rados.0.trial115.stdout:1055: writing trial11545420-27 from 490870 to 1109669 tid 1 2026-02-17T05:51:59.293 INFO:tasks.rados.rados.0.trial115.stdout:1055: writing trial11545420-27 from 1592676 to 2366820 tid 2 2026-02-17T05:51:59.295 INFO:tasks.rados.rados.0.trial115.stdout:1055: writing trial11545420-27 from 2793074 to 2890870 tid 3 2026-02-17T05:51:59.295 INFO:tasks.rados.rados.0.trial115.stdout:1056: delete oid 13 current snap is 104 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:1051: finishing write tid 3 to trial11545420-7 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:1051: finishing write tid 4 to trial11545420-7 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:1051: oid 7 updating version 0 to 407 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:1051: oid 7 version 407 is already newer than 406 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:1051: oid 7 updating version 407 to 408 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 408 (ObjNum 407 snap 104 seq_num 407) dirty exists 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:1051: left oid 7 (ObjNum 407 snap 104 seq_num 407) 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:1055: finishing write tid 1 to trial11545420-27 2026-02-17T05:51:59.296 INFO:tasks.rados.rados.0.trial115.stdout:1055: finishing write tid 2 to trial11545420-27 2026-02-17T05:51:59.297 INFO:tasks.rados.rados.0.trial115.stdout:1055: finishing write tid 3 to trial11545420-27 2026-02-17T05:51:59.297 INFO:tasks.rados.rados.0.trial115.stdout:1055: finishing write tid 4 to trial11545420-27 2026-02-17T05:51:59.297 INFO:tasks.rados.rados.0.trial115.stdout:1055: finishing write tid 5 to trial11545420-27 2026-02-17T05:51:59.297 INFO:tasks.rados.rados.0.trial115.stdout:1055: oid 27 updating version 0 to 409 2026-02-17T05:51:59.297 INFO:tasks.rados.rados.0.trial115.stdout:1055: oid 27 updating version 409 to 412 2026-02-17T05:51:59.297 INFO:tasks.rados.rados.0.trial115.stdout:1055: oid 27 version 412 is already newer than 411 2026-02-17T05:51:59.298 INFO:tasks.rados.rados.0.trial115.stdout:1055: oid 27 version 412 is already newer than 410 2026-02-17T05:51:59.298 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 412 (ObjNum 410 snap 104 seq_num 410) dirty exists 2026-02-17T05:51:59.298 INFO:tasks.rados.rados.0.trial115.stdout:1055: left oid 27 (ObjNum 410 snap 104 seq_num 410) 2026-02-17T05:51:59.300 INFO:tasks.rados.rados.0.trial115.stdout:1051: done (3 left) 2026-02-17T05:51:59.300 INFO:tasks.rados.rados.0.trial115.stdout:1054: done (2 left) 2026-02-17T05:51:59.300 INFO:tasks.rados.rados.0.trial115.stdout:1055: done (1 left) 2026-02-17T05:51:59.300 INFO:tasks.rados.rados.0.trial115.stdout:1056: done (0 left) 2026-02-17T05:51:59.300 INFO:tasks.rados.rados.0.trial115.stdout:1057: snap_create 2026-02-17T05:51:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient: get_monmap_and_config 2026-02-17T05:51:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient: build_initial_monmap 2026-02-17T05:51:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient: monmap: 2026-02-17T05:51:59.313 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:51:59.315398+0000 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:51:59.315398+0000 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:51:59.314 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 1 Processor -- start 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 1 -- start start 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient: init 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f492406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f492406d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f49240a01f0 0x7f492413d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient(hunting): picked mon.noname-c con 0x7f49240a01f0 addr v2:10.20.193.115:3300/0 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49240a0600 con 0x7f492406c890 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4924074150 con 0x7f492406d380 2026-02-17T05:51:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f492409ef70 con 0x7f49240a01f0 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f49294e8640 10 monclient(hunting): authenticate will time out at 964.868929s 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.315+0000 7f4922ffd640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f492406c890 legacy=0x7f492406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f49240a01f0 0x7f492413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49227fc640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f492406d380 0x7f492413a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:45626/0 (socket says 10.20.193.78:45626) 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49227fc640 1 -- 10.20.193.78:0/2175462565 learned_addr learned my addr 10.20.193.78:0/2175462565 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient(hunting): get_auth_request con 0x7f49240a01f0 auth_method 0 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient(hunting): handle_auth_done global_id 44452 payload 274 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 1 -- 10.20.193.78:0/2175462565 >> v1:10.20.193.115:6789/0 conn(0x7f492406d380 legacy=0x7f492413a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 1 -- 10.20.193.78:0/2175462565 >> v1:10.20.193.78:6789/0 conn(0x7f492406c890 legacy=0x7f492406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient: _finish_hunting 0 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 1 monclient: found mon.noname-c 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.316+0000 7f49237fe640 1 -- 10.20.193.78:0/2175462565 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492409f250 con 0x7f49240a01f0 2026-02-17T05:51:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.318+0000 7f49237fe640 1 -- 10.20.193.78:0/2175462565 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49240a01f0 msgr2=0x7f492413d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.318+0000 7f49237fe640 1 --2- 10.20.193.78:0/2175462565 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49240a01f0 0x7f492413d970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4914009d90 tx=0x7f491402f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d301812ac38bc05c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.318+0000 7f4921ffb640 1 -- 10.20.193.78:0/2175462565 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491403c070 con 0x7f49240a01f0 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.318+0000 7f4921ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 10 monclient: dump: 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.327 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 10 monclient: _finish_auth 0 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 5 monclient: authenticate success, global_id 44452 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 1 -- 10.20.193.78:0/2175462565 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4914037640 con 0x7f49240a01f0 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 1 -- 10.20.193.78:0/2175462565 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914037a10 con 0x7f49240a01f0 2026-02-17T05:51:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f4921ffb640 10 monclient: dump: 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 10 monclient: get_monmap_and_config success 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 4 set_mon_vals no callback set 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 10 monclient: shutdown 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 1 -- 10.20.193.78:0/2175462565 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49240a01f0 msgr2=0x7f492413d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 1 --2- 10.20.193.78:0/2175462565 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49240a01f0 0x7f492413d970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4914009d90 tx=0x7f491402f1e0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 1 -- 10.20.193.78:0/2175462565 shutdown_connections 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 1 --2- 10.20.193.78:0/2175462565 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49240a01f0 0x7f492413d970 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 1 -- 10.20.193.78:0/2175462565 >> 10.20.193.78:0/2175462565 conn(0x7f49240a6620 msgr2=0x7f49240a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 1 -- 10.20.193.78:0/2175462565 shutdown_connections 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.319+0000 7f49294e8640 1 -- 10.20.193.78:0/2175462565 wait complete. 2026-02-17T05:51:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient: build_initial_monmap 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient: monmap: 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 1 Processor -- start 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 1 -- start start 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient: init 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 0x7f49241cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient(hunting): picked mon.noname-c con 0x7f492406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f492406d380 0x7f4924135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient(hunting): picked mon.noname-b con 0x7f492406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f49240a01f0 0x7f4924135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient(hunting): picked mon.noname-a con 0x7f49240a01f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f492409d360 con 0x7f49240a01f0 2026-02-17T05:51:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f492409d1e0 con 0x7f492406d380 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f492409be20 con 0x7f492406c890 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49294e8640 10 monclient(hunting): authenticate will time out at 964.873531s 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49237fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f49240a01f0 msgr2=0x7f4924135640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49237fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f49240a01f0 0x7f4924135640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 0x7f49241cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 0x7f49241cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38672/0 (socket says 10.20.193.78:38672) 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 -- 10.20.193.78:0/130612446 learned_addr learned my addr 10.20.193.78:0/130612446 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient(hunting): get_auth_request con 0x7f492406c890 auth_method 0 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49227fc640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f492406d380 0x7f4924135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49227fc640 10 monclient(hunting): get_auth_request con 0x7f492406d380 auth_method 0 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49227fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49227fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient(hunting): handle_auth_done global_id 74273 payload 824 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f492406d380 msgr2=0x7f4924135100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f492406d380 0x7f4924135100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f49240a01f0 msgr2=0x7f4924135640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f49240a01f0 0x7f4924135640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 monclient: found mon.noname-c 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f4922ffd640 1 -- 10.20.193.78:0/130612446 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4924127840 con 0x7f492406c890 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.320+0000 7f49227fc640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f492406d380 0x7f4924135100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.322+0000 7f4922ffd640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 msgr2=0x7f49241cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.322+0000 7f4922ffd640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 0x7f49241cf3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f491800b780 tx=0x7f491800bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f4922ffd640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 0x7f49241cf3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f491800b780 tx=0x7f491800bc50 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f4918005270 mon_map magic: 0 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918005270 con 0x7f492406c890 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:51:59.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: dump: 2026-02-17T05:51:59.336 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.336 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.336 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.336 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.336 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.336 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.336 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: _finish_auth 0 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: _check_auth_tickets 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4918010070 con 0x7f492406c890 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 5 monclient: authenticate success, global_id 74273 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 10 monclient: _renew_subs 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:59.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49217fa640 4 set_mon_vals no callback set 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492409d810 con 0x7f492406c890 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 10 monclient: _renew_subs 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4924072ea0 con 0x7f492406c890 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4924125ba0 0x7f4924125f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49241cf920 con 0x7f4924125ba0 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918005650 con 0x7f492406c890 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 10 monclient: dump: 2026-02-17T05:51:59.338 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f491801d030 con 0x7f492406c890 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49227fc640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4924125ba0 0x7f4924125f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49227fc640 10 monclient: get_auth_request con 0x7f4924125ba0 auth_method 0 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49227fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49227fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.323+0000 7f49037fe640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48f003de60 0x7f48f0040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.324+0000 7f49227fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.324+0000 7f49227fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.324+0000 7f49227fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.324+0000 7f49237fe640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48f003de60 0x7f48f0040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.324+0000 7f49237fe640 10 monclient: get_auth_request con 0x7f48f003de60 auth_method 0 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.324+0000 7f49227fc640 10 monclient(hunting): handle_auth_done global_id 44458 payload 824 2026-02-17T05:51:59.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.324+0000 7f49227fc640 1 -- 10.20.193.78:0/130612446 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49240a28d0 con 0x7f4924125ba0 2026-02-17T05:51:59.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.326+0000 7f49237fe640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48f003de60 0x7f48f0040320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f492406a480 tx=0x7f4914033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.326+0000 7f49227fc640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4924125ba0 msgr2=0x7f4924125f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:59.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.326+0000 7f49227fc640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4924125ba0 0x7f4924125f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4924128900 tx=0x7f490c00eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.326+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c005be0 con 0x7f4924125ba0 2026-02-17T05:51:59.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.326+0000 7f49037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.326+0000 7f49037fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.326+0000 7f49037fe640 10 monclient: dump: 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.327+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f490c018070 con 0x7f4924125ba0 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.327+0000 7f49037fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:51:59.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.327+0000 7f49037fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.327+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4924125ba0 msgr2=0x7f4924125f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.327+0000 7f49037fe640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4924125ba0 0x7f4924125f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4924128900 tx=0x7f490c00eba0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.327+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4924125ba0 msgr2=0x7f4924125f80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.327+0000 7f49037fe640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4924125ba0 0x7f4924125f80 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4924128900 tx=0x7f490c00eba0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49294e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49294e8640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49241264c0 0x7f49241da4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49294e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f492406cc70 con 0x7f49241264c0 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49241264c0 0x7f49241da4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 10 monclient: get_auth_request con 0x7f49241264c0 auth_method 0 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:51:59.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 10 monclient(hunting): handle_auth_done global_id 44461 payload 824 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.330+0000 7f49227fc640 1 -- 10.20.193.78:0/130612446 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f49240744d0 con 0x7f49241264c0 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.332+0000 7f49227fc640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49241264c0 msgr2=0x7f49241da4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.332+0000 7f49227fc640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49241264c0 0x7f49241da4c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49241291c0 tx=0x7f490c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.332+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c00c900 con 0x7f49241264c0 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.332+0000 7f49037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.332+0000 7f49037fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.332+0000 7f49037fe640 10 monclient: dump: 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:51:59.346 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7f490c01ee50 con 0x7f49241264c0 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49037fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49037fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49241264c0 msgr2=0x7f49241da4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49037fe640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49241264c0 0x7f49241da4c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49241291c0 tx=0x7f490c004a40 comp rx=0 tx=0).stop 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49241264c0 msgr2=0x7f49241da4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49037fe640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f49241264c0 0x7f49241da4c0 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49241291c0 tx=0x7f490c004a40 comp rx=0 tx=0).stop 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:51:59.347 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 64, 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 242979, 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:59.348 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:59.349 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:51:59.350 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:51:59.351 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:51:59.352 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.353 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:51:59.354 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:51:59.355 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:51:59.356 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:51:59.357 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48f003de60 msgr2=0x7f48f0040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48f003de60 0x7f48f0040320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f492406a480 tx=0x7f4914033000 comp rx=0 tx=0).stop 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 10 monclient: shutdown 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 msgr2=0x7f49241cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49237fe640 1 -- 10.20.193.78:0/130612446 reap_dead start 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 0x7f49241cf3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f491800b780 tx=0x7f491800bc50 comp rx=0 tx=0).stop 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49037fe640 1 -- 10.20.193.78:0/130612446 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(223..223 src has 117..223) ==== 4831+0+0 (unknown 0 0 0) 0x7f4918051e20 con 0x7f492406c890 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 shutdown_connections 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 --2- 10.20.193.78:0/130612446 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f492406c890 0x7f49241cf3e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 >> 10.20.193.78:0/130612446 conn(0x7f49240a6620 msgr2=0x7f4924071550 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 shutdown_connections 2026-02-17T05:51:59.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:51:59.333+0000 7f49294e8640 1 -- 10.20.193.78:0/130612446 wait complete. 2026-02-17T05:51:59.359 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:51:59.359 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:52:00.176 INFO:tasks.rados.rados.0.trial115.stdout:1057: done (0 left) 2026-02-17T05:52:00.176 INFO:tasks.rados.rados.0.trial115.stdout:1058: rollback oid 19 current snap is 105 2026-02-17T05:52:00.177 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 19 to 95 2026-02-17T05:52:00.177 INFO:tasks.rados.rados.0.trial115.stdout:1059: delete oid 37 current snap is 105 2026-02-17T05:52:00.178 INFO:tasks.rados.rados.0.trial115.stdout:1058: finishing rollback tid 1 to trial11545420-19 2026-02-17T05:52:00.178 INFO:tasks.rados.rados.0.trial115.stdout:1058: finishing rollback tid 2 to trial11545420-19 2026-02-17T05:52:00.178 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 507 (ObjNum 306 snap 73 seq_num 306) dirty exists 2026-02-17T05:52:00.179 INFO:tasks.rados.rados.0.trial115.stdout:1058: done (1 left) 2026-02-17T05:52:00.179 INFO:tasks.rados.rados.0.trial115.stdout:1059: done (0 left) 2026-02-17T05:52:00.179 INFO:tasks.rados.rados.0.trial115.stdout:1060: snap_create 2026-02-17T05:52:01.178 INFO:tasks.rados.rados.0.trial115.stdout:1060: done (0 left) 2026-02-17T05:52:01.178 INFO:tasks.rados.rados.0.trial115.stdout:1061: delete oid 7 current snap is 106 2026-02-17T05:52:01.180 INFO:tasks.rados.rados.0.trial115.stdout:1061: done (0 left) 2026-02-17T05:52:01.180 INFO:tasks.rados.rados.0.trial115.stdout:1062: read oid 34 snap 86 2026-02-17T05:52:01.180 INFO:tasks.rados.rados.0.trial115.stdout:1062: expect deleted 2026-02-17T05:52:01.180 INFO:tasks.rados.rados.0.trial115.stdout:1063: copy_from oid 9 from oid 19 current snap is 106 2026-02-17T05:52:01.180 INFO:tasks.rados.rados.0.trial115.stdout:1064: rollback oid 1 current snap is 106 2026-02-17T05:52:01.180 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 1 to 103 2026-02-17T05:52:01.180 INFO:tasks.rados.rados.0.trial115.stdout:1065: snap_create 2026-02-17T05:52:01.183 INFO:tasks.rados.rados.0.trial115.stdout:1064: finishing rollback tid 0 to trial11545420-1 2026-02-17T05:52:01.183 INFO:tasks.rados.rados.0.trial115.stdout:1064: finishing rollback tid 1 to trial11545420-1 2026-02-17T05:52:01.183 INFO:tasks.rados.rados.0.trial115.stdout:1064: finishing rollback tid 2 to trial11545420-1 2026-02-17T05:52:01.183 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 378 (ObjNum 378 snap 99 seq_num 378) dirty exists 2026-02-17T05:52:01.189 INFO:tasks.rados.rados.0.trial115.stdout:1063: finishing copy_from racing read to trial11545420-9 2026-02-17T05:52:01.189 INFO:tasks.rados.rados.0.trial115.stdout:1063: finishing copy_from to trial11545420-9 2026-02-17T05:52:01.189 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 507 (ObjNum 306 snap 73 seq_num 306) dirty exists 2026-02-17T05:52:01.374 INFO:tasks.rados.rados.0.trial115.stdout:1062: done (3 left) 2026-02-17T05:52:01.374 INFO:tasks.rados.rados.0.trial115.stdout:1063: done (2 left) 2026-02-17T05:52:01.374 INFO:tasks.rados.rados.0.trial115.stdout:1064: done (1 left) 2026-02-17T05:52:01.374 INFO:tasks.rados.rados.0.trial115.stdout:1065: done (0 left) 2026-02-17T05:52:01.374 INFO:tasks.rados.rados.0.trial115.stdout:1066: write oid 42 current snap is 107 2026-02-17T05:52:01.374 INFO:tasks.rados.rados.0.trial115.stdout:1066: seq_num 414 ranges {558098=636148,1826604=665129} 2026-02-17T05:52:01.378 INFO:tasks.rados.rados.0.trial115.stdout:1066: writing trial11545420-42 from 558098 to 1194246 tid 1 2026-02-17T05:52:01.383 INFO:tasks.rados.rados.0.trial115.stdout:1066: writing trial11545420-42 from 1826604 to 2491733 tid 2 2026-02-17T05:52:01.383 INFO:tasks.rados.rados.0.trial115.stdout:1067: write oid 27 current snap is 107 2026-02-17T05:52:01.383 INFO:tasks.rados.rados.0.trial115.stdout:1067: seq_num 415 ranges {574905=711397,1885086=525140,2930189=44716} 2026-02-17T05:52:01.387 INFO:tasks.rados.rados.0.trial115.stdout:1067: writing trial11545420-27 from 574905 to 1286302 tid 1 2026-02-17T05:52:01.392 INFO:tasks.rados.rados.0.trial115.stdout:1067: writing trial11545420-27 from 1885086 to 2410226 tid 2 2026-02-17T05:52:01.394 INFO:tasks.rados.rados.0.trial115.stdout:1067: writing trial11545420-27 from 2930189 to 2974905 tid 3 2026-02-17T05:52:01.394 INFO:tasks.rados.rados.0.trial115.stdout:1068: write (excl) oid 39 current snap is 107 2026-02-17T05:52:01.394 INFO:tasks.rados.rados.0.trial115.stdout:1068: seq_num 416 ranges {591712=786646,1943568=668798} 2026-02-17T05:52:01.399 INFO:tasks.rados.rados.0.trial115.stdout:1068: writing trial11545420-39 from 591712 to 1378358 tid 1 2026-02-17T05:52:01.405 INFO:tasks.rados.rados.0.trial115.stdout:1068: writing trial11545420-39 from 1943568 to 2612366 tid 2 2026-02-17T05:52:01.405 INFO:tasks.rados.rados.0.trial115.stdout:1069: write (excl) oid 3 current snap is 107 2026-02-17T05:52:01.405 INFO:tasks.rados.rados.0.trial115.stdout:1066: finishing write tid 1 to trial11545420-42 2026-02-17T05:52:01.405 INFO:tasks.rados.rados.0.trial115.stdout:1066: finishing write tid 2 to trial11545420-42 2026-02-17T05:52:01.405 INFO:tasks.rados.rados.0.trial115.stdout:1066: finishing write tid 3 to trial11545420-42 2026-02-17T05:52:01.405 INFO:tasks.rados.rados.0.trial115.stdout:1066: finishing write tid 4 to trial11545420-42 2026-02-17T05:52:01.405 INFO:tasks.rados.rados.0.trial115.stdout:1066: oid 42 updating version 0 to 530 2026-02-17T05:52:01.405 INFO:tasks.rados.rados.0.trial115.stdout:1066: oid 42 updating version 530 to 531 2026-02-17T05:52:01.406 INFO:tasks.rados.rados.0.trial115.stdout:1066: oid 42 version 531 is already newer than 529 2026-02-17T05:52:01.406 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 531 (ObjNum 414 snap 107 seq_num 414) dirty exists 2026-02-17T05:52:01.406 INFO:tasks.rados.rados.0.trial115.stdout:1066: left oid 42 (ObjNum 414 snap 107 seq_num 414) 2026-02-17T05:52:01.406 INFO:tasks.rados.rados.0.trial115.stdout:1067: finishing write tid 1 to trial11545420-27 2026-02-17T05:52:01.406 INFO:tasks.rados.rados.0.trial115.stdout:1069: seq_num 417 ranges {608519=461895,1602050=528809,2585035=423484} 2026-02-17T05:52:01.408 INFO:tasks.rados.rados.0.trial115.stdout:1069: writing trial11545420-3 from 608519 to 1070414 tid 1 2026-02-17T05:52:01.412 INFO:tasks.rados.rados.0.trial115.stdout:1069: writing trial11545420-3 from 1602050 to 2130859 tid 2 2026-02-17T05:52:01.415 INFO:tasks.rados.rados.0.trial115.stdout:1069: writing trial11545420-3 from 2585035 to 3008519 tid 3 2026-02-17T05:52:01.415 INFO:tasks.rados.rados.0.trial115.stdout:1066: done (3 left) 2026-02-17T05:52:01.415 INFO:tasks.rados.rados.0.trial115.stdout:1070: write (excl) oid 34 current snap is 107 2026-02-17T05:52:01.415 INFO:tasks.rados.rados.0.trial115.stdout:1070: seq_num 418 ranges {625326=537144,1944179=672467} 2026-02-17T05:52:01.419 INFO:tasks.rados.rados.0.trial115.stdout:1070: writing trial11545420-34 from 625326 to 1162470 tid 1 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1070: writing trial11545420-34 from 1944179 to 2616646 tid 2 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1071: copy_from oid 40 from oid 7 current snap is 107 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1067: finishing write tid 2 to trial11545420-27 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1067: finishing write tid 3 to trial11545420-27 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1067: finishing write tid 4 to trial11545420-27 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1067: finishing write tid 5 to trial11545420-27 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1067: oid 27 updating version 0 to 415 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1067: oid 27 updating version 415 to 417 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1067: oid 27 version 417 is already newer than 416 2026-02-17T05:52:01.427 INFO:tasks.rados.rados.0.trial115.stdout:1067: oid 27 version 417 is already newer than 414 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 417 (ObjNum 415 snap 107 seq_num 415) dirty exists 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1067: left oid 27 (ObjNum 415 snap 107 seq_num 415) 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1068: finishing write tid 1 to trial11545420-39 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1067: done (4 left) 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1072: read oid 8 snap -1 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1072: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1073: rollback oid 20 current snap is 107 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 105 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1068: finishing write tid 2 to trial11545420-39 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1068: finishing write tid 3 to trial11545420-39 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1068: finishing write tid 4 to trial11545420-39 2026-02-17T05:52:01.428 INFO:tasks.rados.rados.0.trial115.stdout:1068: oid 39 updating version 0 to 534 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1068: oid 39 version 534 is already newer than 533 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1068: oid 39 updating version 534 to 535 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 535 (ObjNum 416 snap 107 seq_num 416) dirty exists 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1068: left oid 39 (ObjNum 416 snap 107 seq_num 416) 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1069: finishing write tid 1 to trial11545420-3 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1069: finishing write tid 2 to trial11545420-3 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1070: finishing write tid 1 to trial11545420-34 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1069: finishing write tid 3 to trial11545420-3 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1069: finishing write tid 4 to trial11545420-3 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1069: finishing write tid 5 to trial11545420-3 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1069: oid 3 updating version 0 to 439 2026-02-17T05:52:01.429 INFO:tasks.rados.rados.0.trial115.stdout:1069: oid 3 version 439 is already newer than 438 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1069: oid 3 version 439 is already newer than 437 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1069: oid 3 updating version 439 to 440 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 440 (ObjNum 417 snap 107 seq_num 417) dirty exists 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1069: left oid 3 (ObjNum 417 snap 107 seq_num 417) 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1068: done (5 left) 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1069: done (4 left) 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1074: read oid 30 snap -1 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1074: expect deleted 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1075: delete oid 25 current snap is 107 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1071: finishing copy_from to trial11545420-40 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1071: got expected ENOENT (src dne) 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1071: finishing copy_from racing read to trial11545420-40 2026-02-17T05:52:01.430 INFO:tasks.rados.rados.0.trial115.stdout:1071: done (5 left) 2026-02-17T05:52:01.431 INFO:tasks.rados.rados.0.trial115.stdout:1074: done (4 left) 2026-02-17T05:52:01.431 INFO:tasks.rados.rados.0.trial115.stdout:1075: done (3 left) 2026-02-17T05:52:01.431 INFO:tasks.rados.rados.0.trial115.stdout:1076: rollback oid 45 current snap is 107 2026-02-17T05:52:01.431 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 78 2026-02-17T05:52:01.431 INFO:tasks.rados.rados.0.trial115.stdout:1077: write (excl) oid 31 current snap is 107 2026-02-17T05:52:01.431 INFO:tasks.rados.rados.0.trial115.stdout:1077: seq_num 420 ranges {658940=403995,1777496=676136,2950951=107989} 2026-02-17T05:52:01.431 INFO:tasks.rados.rados.0.trial115.stdout:1077: writing trial11545420-31 from 658940 to 1062935 tid 1 2026-02-17T05:52:01.435 INFO:tasks.rados.rados.0.trial115.stdout:1077: writing trial11545420-31 from 1777496 to 2453632 tid 2 2026-02-17T05:52:01.438 INFO:tasks.rados.rados.0.trial115.stdout:1077: writing trial11545420-31 from 2950951 to 3058940 tid 3 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1078: read oid 46 snap -1 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1078: expect (ObjNum 397 snap 102 seq_num 397) 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1079: rollback oid 47 current snap is 107 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 47 to 98 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1070: finishing write tid 2 to trial11545420-34 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1070: finishing write tid 3 to trial11545420-34 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1070: finishing write tid 4 to trial11545420-34 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1070: oid 34 updating version 0 to 296 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1070: oid 34 version 296 is already newer than 294 2026-02-17T05:52:01.439 INFO:tasks.rados.rados.0.trial115.stdout:1070: oid 34 version 296 is already newer than 295 2026-02-17T05:52:01.440 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 296 (ObjNum 418 snap 107 seq_num 418) dirty exists 2026-02-17T05:52:01.440 INFO:tasks.rados.rados.0.trial115.stdout:1070: left oid 34 (ObjNum 418 snap 107 seq_num 418) 2026-02-17T05:52:01.440 INFO:tasks.rados.rados.0.trial115.stdout:1073: finishing rollback tid 0 to trial11545420-20 2026-02-17T05:52:01.440 INFO:tasks.rados.rados.0.trial115.stdout:1070: done (6 left) 2026-02-17T05:52:01.440 INFO:tasks.rados.rados.0.trial115.stdout:1080: delete oid 23 current snap is 107 2026-02-17T05:52:01.440 INFO:tasks.rados.rados.0.trial115.stdout:1073: finishing rollback tid 1 to trial11545420-20 2026-02-17T05:52:01.440 INFO:tasks.rados.rados.0.trial115.stdout:1072: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:52:01.491 INFO:tasks.rados.rados.0.trial115.stdout:1073: finishing rollback tid 2 to trial11545420-20 2026-02-17T05:52:01.491 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 413 (ObjNum 374 snap 99 seq_num 374) dirty exists 2026-02-17T05:52:01.491 INFO:tasks.rados.rados.0.trial115.stdout:1076: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1076: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1076: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 539 (ObjNum 302 snap 72 seq_num 302) dirty exists 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1077: finishing write tid 1 to trial11545420-31 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1072: done (6 left) 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1073: done (5 left) 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1076: done (4 left) 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1080: done (3 left) 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1081: write oid 26 current snap is 107 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1077: finishing write tid 2 to trial11545420-31 2026-02-17T05:52:01.492 INFO:tasks.rados.rados.0.trial115.stdout:1081: seq_num 422 ranges {692554=554493,1778107=679805,2889444=203110} 2026-02-17T05:52:01.495 INFO:tasks.rados.rados.0.trial115.stdout:1081: writing trial11545420-26 from 692554 to 1247047 tid 1 2026-02-17T05:52:01.500 INFO:tasks.rados.rados.0.trial115.stdout:1081: writing trial11545420-26 from 1778107 to 2457912 tid 2 2026-02-17T05:52:01.502 INFO:tasks.rados.rados.0.trial115.stdout:1081: writing trial11545420-26 from 2889444 to 3092554 tid 3 2026-02-17T05:52:01.503 INFO:tasks.rados.rados.0.trial115.stdout:1082: write oid 33 current snap is 107 2026-02-17T05:52:01.503 INFO:tasks.rados.rados.0.trial115.stdout:1077: finishing write tid 3 to trial11545420-31 2026-02-17T05:52:01.503 INFO:tasks.rados.rados.0.trial115.stdout:1077: finishing write tid 4 to trial11545420-31 2026-02-17T05:52:01.503 INFO:tasks.rados.rados.0.trial115.stdout:1078: expect (ObjNum 397 snap 102 seq_num 397) 2026-02-17T05:52:01.585 INFO:tasks.rados.rados.0.trial115.stdout:1077: finishing write tid 5 to trial11545420-31 2026-02-17T05:52:01.585 INFO:tasks.rados.rados.0.trial115.stdout:1077: oid 31 updating version 0 to 445 2026-02-17T05:52:01.585 INFO:tasks.rados.rados.0.trial115.stdout:1077: oid 31 version 445 is already newer than 443 2026-02-17T05:52:01.585 INFO:tasks.rados.rados.0.trial115.stdout:1077: oid 31 version 445 is already newer than 444 2026-02-17T05:52:01.585 INFO:tasks.rados.rados.0.trial115.stdout:1077: oid 31 version 445 is already newer than 442 2026-02-17T05:52:01.585 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 445 (ObjNum 420 snap 107 seq_num 420) dirty exists 2026-02-17T05:52:01.586 INFO:tasks.rados.rados.0.trial115.stdout:1077: left oid 31 (ObjNum 420 snap 107 seq_num 420) 2026-02-17T05:52:01.586 INFO:tasks.rados.rados.0.trial115.stdout:1079: finishing rollback tid 0 to trial11545420-47 2026-02-17T05:52:01.586 INFO:tasks.rados.rados.0.trial115.stdout:1079: finishing rollback tid 1 to trial11545420-47 2026-02-17T05:52:01.586 INFO:tasks.rados.rados.0.trial115.stdout:1079: finishing rollback tid 2 to trial11545420-47 2026-02-17T05:52:01.586 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 449 (ObjNum 353 snap 90 seq_num 353) dirty exists 2026-02-17T05:52:01.586 INFO:tasks.rados.rados.0.trial115.stdout:1081: finishing write tid 1 to trial11545420-26 2026-02-17T05:52:01.586 INFO:tasks.rados.rados.0.trial115.stdout:1082: seq_num 423 ranges {709361=629742,1836589=539816,3033220=76141} 2026-02-17T05:52:01.589 INFO:tasks.rados.rados.0.trial115.stdout:1082: writing trial11545420-33 from 709361 to 1339103 tid 1 2026-02-17T05:52:01.593 INFO:tasks.rados.rados.0.trial115.stdout:1082: writing trial11545420-33 from 1836589 to 2376405 tid 2 2026-02-17T05:52:01.597 INFO:tasks.rados.rados.0.trial115.stdout:1082: writing trial11545420-33 from 3033220 to 3109361 tid 3 2026-02-17T05:52:01.597 INFO:tasks.rados.rados.0.trial115.stdout:1077: done (4 left) 2026-02-17T05:52:01.597 INFO:tasks.rados.rados.0.trial115.stdout:1078: done (3 left) 2026-02-17T05:52:01.597 INFO:tasks.rados.rados.0.trial115.stdout:1079: done (2 left) 2026-02-17T05:52:01.597 INFO:tasks.rados.rados.0.trial115.stdout:1083: read oid 48 snap -1 2026-02-17T05:52:01.597 INFO:tasks.rados.rados.0.trial115.stdout:1083: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:52:01.597 INFO:tasks.rados.rados.0.trial115.stdout:1084: write (excl) oid 29 current snap is 107 2026-02-17T05:52:01.597 INFO:tasks.rados.rados.0.trial115.stdout:1084: seq_num 424 ranges {726168=704991,1895071=683474} 2026-02-17T05:52:01.602 INFO:tasks.rados.rados.0.trial115.stdout:1084: writing trial11545420-29 from 726168 to 1431159 tid 1 2026-02-17T05:52:01.607 INFO:tasks.rados.rados.0.trial115.stdout:1084: writing trial11545420-29 from 1895071 to 2578545 tid 2 2026-02-17T05:52:01.607 INFO:tasks.rados.rados.0.trial115.stdout:1085: copy_from oid 10 from oid 27 current snap is 107 2026-02-17T05:52:01.607 INFO:tasks.rados.rados.0.trial115.stdout:1081: finishing write tid 2 to trial11545420-26 2026-02-17T05:52:01.607 INFO:tasks.rados.rados.0.trial115.stdout:1081: finishing write tid 3 to trial11545420-26 2026-02-17T05:52:01.607 INFO:tasks.rados.rados.0.trial115.stdout:1081: finishing write tid 4 to trial11545420-26 2026-02-17T05:52:01.607 INFO:tasks.rados.rados.0.trial115.stdout:1081: finishing write tid 5 to trial11545420-26 2026-02-17T05:52:01.607 INFO:tasks.rados.rados.0.trial115.stdout:1081: oid 26 updating version 0 to 196 2026-02-17T05:52:01.608 INFO:tasks.rados.rados.0.trial115.stdout:1081: oid 26 version 196 is already newer than 194 2026-02-17T05:52:01.608 INFO:tasks.rados.rados.0.trial115.stdout:1081: oid 26 version 196 is already newer than 193 2026-02-17T05:52:01.608 INFO:tasks.rados.rados.0.trial115.stdout:1081: oid 26 version 196 is already newer than 195 2026-02-17T05:52:01.608 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 196 (ObjNum 422 snap 107 seq_num 422) dirty exists 2026-02-17T05:52:01.608 INFO:tasks.rados.rados.0.trial115.stdout:1081: left oid 26 (ObjNum 422 snap 107 seq_num 422) 2026-02-17T05:52:01.608 INFO:tasks.rados.rados.0.trial115.stdout:1081: done (4 left) 2026-02-17T05:52:01.608 INFO:tasks.rados.rados.0.trial115.stdout:1086: write (excl) oid 16 current snap is 107 2026-02-17T05:52:01.608 INFO:tasks.rados.rados.0.trial115.stdout:1086: seq_num 425 ranges {742975=780240,1953553=427132,2971713=171262} 2026-02-17T05:52:01.612 INFO:tasks.rados.rados.0.trial115.stdout:1086: writing trial11545420-16 from 742975 to 1523215 tid 1 2026-02-17T05:52:01.615 INFO:tasks.rados.rados.0.trial115.stdout:1086: writing trial11545420-16 from 1953553 to 2380685 tid 2 2026-02-17T05:52:01.619 INFO:tasks.rados.rados.0.trial115.stdout:1086: writing trial11545420-16 from 2971713 to 3142975 tid 3 2026-02-17T05:52:01.619 INFO:tasks.rados.rados.0.trial115.stdout:1087: read oid 35 snap -1 2026-02-17T05:52:01.619 INFO:tasks.rados.rados.0.trial115.stdout:1087: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:52:01.619 INFO:tasks.rados.rados.0.trial115.stdout:1088: delete oid 20 current snap is 107 2026-02-17T05:52:01.619 INFO:tasks.rados.rados.0.trial115.stdout:1082: finishing write tid 1 to trial11545420-33 2026-02-17T05:52:01.619 INFO:tasks.rados.rados.0.trial115.stdout:1082: finishing write tid 2 to trial11545420-33 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1082: finishing write tid 3 to trial11545420-33 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1082: finishing write tid 4 to trial11545420-33 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1082: finishing write tid 5 to trial11545420-33 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1082: oid 33 updating version 0 to 382 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1082: oid 33 version 382 is already newer than 381 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1082: oid 33 version 382 is already newer than 379 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1082: oid 33 version 382 is already newer than 380 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 382 (ObjNum 423 snap 107 seq_num 423) dirty exists 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1082: left oid 33 (ObjNum 423 snap 107 seq_num 423) 2026-02-17T05:52:01.620 INFO:tasks.rados.rados.0.trial115.stdout:1083: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:52:01.763 INFO:tasks.rados.rados.0.trial115.stdout:1084: finishing write tid 1 to trial11545420-29 2026-02-17T05:52:01.763 INFO:tasks.rados.rados.0.trial115.stdout:1084: finishing write tid 2 to trial11545420-29 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1084: finishing write tid 3 to trial11545420-29 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1084: finishing write tid 4 to trial11545420-29 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1084: oid 29 updating version 0 to 401 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1084: oid 29 version 401 is already newer than 399 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1084: oid 29 version 401 is already newer than 400 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 401 (ObjNum 424 snap 107 seq_num 424) dirty exists 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1084: left oid 29 (ObjNum 424 snap 107 seq_num 424) 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1085: finishing copy_from racing read to trial11545420-10 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1085: finishing copy_from to trial11545420-10 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 417 (ObjNum 415 snap 107 seq_num 415) dirty exists 2026-02-17T05:52:01.764 INFO:tasks.rados.rados.0.trial115.stdout:1086: finishing write tid 1 to trial11545420-16 2026-02-17T05:52:01.765 INFO:tasks.rados.rados.0.trial115.stdout:1086: finishing write tid 2 to trial11545420-16 2026-02-17T05:52:01.765 INFO:tasks.rados.rados.0.trial115.stdout:1082: done (6 left) 2026-02-17T05:52:01.765 INFO:tasks.rados.rados.0.trial115.stdout:1083: done (5 left) 2026-02-17T05:52:01.765 INFO:tasks.rados.rados.0.trial115.stdout:1084: done (4 left) 2026-02-17T05:52:01.765 INFO:tasks.rados.rados.0.trial115.stdout:1085: done (3 left) 2026-02-17T05:52:01.765 INFO:tasks.rados.rados.0.trial115.stdout:1088: done (2 left) 2026-02-17T05:52:01.765 INFO:tasks.rados.rados.0.trial115.stdout:1089: copy_from oid 9 from oid 44 current snap is 107 2026-02-17T05:52:01.765 INFO:tasks.rados.rados.0.trial115.stdout:1087: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:52:01.817 INFO:tasks.rados.rados.0.trial115.stdout:1086: finishing write tid 3 to trial11545420-16 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1086: finishing write tid 4 to trial11545420-16 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1086: finishing write tid 5 to trial11545420-16 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1086: oid 16 updating version 0 to 512 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1086: oid 16 version 512 is already newer than 509 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1086: oid 16 version 512 is already newer than 510 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1086: oid 16 version 512 is already newer than 511 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 512 (ObjNum 425 snap 107 seq_num 425) dirty exists 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1086: left oid 16 (ObjNum 425 snap 107 seq_num 425) 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1086: done (2 left) 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1087: done (1 left) 2026-02-17T05:52:01.818 INFO:tasks.rados.rados.0.trial115.stdout:1090: read oid 23 snap -1 2026-02-17T05:52:01.819 INFO:tasks.rados.rados.0.trial115.stdout:1090: expect deleted 2026-02-17T05:52:01.819 INFO:tasks.rados.rados.0.trial115.stdout:1091: snap_create 2026-02-17T05:52:01.819 INFO:tasks.rados.rados.0.trial115.stdout:1089: finishing copy_from racing read to trial11545420-9 2026-02-17T05:52:01.819 INFO:tasks.rados.rados.0.trial115.stdout:1089: finishing copy_from to trial11545420-9 2026-02-17T05:52:01.819 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 313 (ObjNum 377 snap 99 seq_num 377) dirty exists 2026-02-17T05:52:02.377 INFO:tasks.rados.rados.0.trial115.stdout:1089: done (2 left) 2026-02-17T05:52:02.377 INFO:tasks.rados.rados.0.trial115.stdout:1090: done (1 left) 2026-02-17T05:52:02.377 INFO:tasks.rados.rados.0.trial115.stdout:1091: done (0 left) 2026-02-17T05:52:02.378 INFO:tasks.rados.rados.0.trial115.stdout:1092: read oid 35 snap -1 2026-02-17T05:52:02.378 INFO:tasks.rados.rados.0.trial115.stdout:1092: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:52:02.378 INFO:tasks.rados.rados.0.trial115.stdout:1093: delete oid 34 current snap is 108 2026-02-17T05:52:02.380 INFO:tasks.rados.rados.0.trial115.stdout:1093: done (1 left) 2026-02-17T05:52:02.380 INFO:tasks.rados.rados.0.trial115.stdout:1094: read oid 38 snap -1 2026-02-17T05:52:02.380 INFO:tasks.rados.rados.0.trial115.stdout:1094: expect deleted 2026-02-17T05:52:02.380 INFO:tasks.rados.rados.0.trial115.stdout:1095: read oid 22 snap 78 2026-02-17T05:52:02.380 INFO:tasks.rados.rados.0.trial115.stdout:1095: expect (ObjNum 284 snap 63 seq_num 284) 2026-02-17T05:52:02.380 INFO:tasks.rados.rados.0.trial115.stdout:1096: write oid 39 current snap is 108 2026-02-17T05:52:02.380 INFO:tasks.rados.rados.0.trial115.stdout:1096: seq_num 428 ranges {793396=722340,2128999=690812} 2026-02-17T05:52:02.386 INFO:tasks.rados.rados.0.trial115.stdout:1096: writing trial11545420-39 from 793396 to 1515736 tid 1 2026-02-17T05:52:02.391 INFO:tasks.rados.rados.0.trial115.stdout:1096: writing trial11545420-39 from 2128999 to 2819811 tid 2 2026-02-17T05:52:02.392 INFO:tasks.rados.rados.0.trial115.stdout:1097: read oid 29 snap -1 2026-02-17T05:52:02.392 INFO:tasks.rados.rados.0.trial115.stdout:1097: expect (ObjNum 424 snap 107 seq_num 424) 2026-02-17T05:52:02.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: write oid 25 current snap is 108 2026-02-17T05:52:02.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: seq_num 429 ranges {410203=797589,1787481=434470,2681405=528798} 2026-02-17T05:52:02.395 INFO:tasks.rados.rados.0.trial115.stdout:1098: writing trial11545420-25 from 410203 to 1207792 tid 1 2026-02-17T05:52:02.399 INFO:tasks.rados.rados.0.trial115.stdout:1098: writing trial11545420-25 from 1787481 to 2221951 tid 2 2026-02-17T05:52:02.403 INFO:tasks.rados.rados.0.trial115.stdout:1098: writing trial11545420-25 from 2681405 to 3210203 tid 3 2026-02-17T05:52:02.403 INFO:tasks.rados.rados.0.trial115.stdout:1099: copy_from oid 34 from oid 24 current snap is 108 2026-02-17T05:52:02.404 INFO:tasks.rados.rados.0.trial115.stdout:1092: expect (ObjNum 379 snap 100 seq_num 379) 2026-02-17T05:52:02.457 INFO:tasks.rados.rados.0.trial115.stdout:1095: expect (ObjNum 284 snap 63 seq_num 284) 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1096: finishing write tid 1 to trial11545420-39 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1096: finishing write tid 2 to trial11545420-39 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1096: finishing write tid 3 to trial11545420-39 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1096: finishing write tid 4 to trial11545420-39 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1096: oid 39 updating version 0 to 542 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1096: oid 39 version 542 is already newer than 541 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1096: oid 39 updating version 542 to 543 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 543 (ObjNum 428 snap 108 seq_num 428) dirty exists 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1096: left oid 39 (ObjNum 428 snap 108 seq_num 428) 2026-02-17T05:52:02.552 INFO:tasks.rados.rados.0.trial115.stdout:1097: expect (ObjNum 424 snap 107 seq_num 424) 2026-02-17T05:52:02.665 INFO:tasks.rados.rados.0.trial115.stdout:1098: finishing write tid 1 to trial11545420-25 2026-02-17T05:52:02.665 INFO:tasks.rados.rados.0.trial115.stdout:1092: done (6 left) 2026-02-17T05:52:02.665 INFO:tasks.rados.rados.0.trial115.stdout:1094: done (5 left) 2026-02-17T05:52:02.665 INFO:tasks.rados.rados.0.trial115.stdout:1095: done (4 left) 2026-02-17T05:52:02.665 INFO:tasks.rados.rados.0.trial115.stdout:1096: done (3 left) 2026-02-17T05:52:02.665 INFO:tasks.rados.rados.0.trial115.stdout:1097: done (2 left) 2026-02-17T05:52:02.665 INFO:tasks.rados.rados.0.trial115.stdout:1100: snap_remove snap 106 2026-02-17T05:52:03.378 INFO:tasks.rados.rados.0.trial115.stdout:1100: done (2 left) 2026-02-17T05:52:03.378 INFO:tasks.rados.rados.0.trial115.stdout:1101: write oid 47 current snap is 108 2026-02-17T05:52:03.378 INFO:tasks.rados.rados.0.trial115.stdout:1101: seq_num 430 ranges {427010=472838,1329610=694481,2708828=518182} 2026-02-17T05:52:03.382 INFO:tasks.rados.rados.0.trial115.stdout:1101: writing trial11545420-47 from 427010 to 899848 tid 1 2026-02-17T05:52:03.386 INFO:tasks.rados.rados.0.trial115.stdout:1101: writing trial11545420-47 from 1329610 to 2024091 tid 2 2026-02-17T05:52:03.391 INFO:tasks.rados.rados.0.trial115.stdout:1101: writing trial11545420-47 from 2708828 to 3227010 tid 3 2026-02-17T05:52:03.391 INFO:tasks.rados.rados.0.trial115.stdout:1102: delete oid 50 current snap is 108 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: finishing write tid 2 to trial11545420-25 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1099: finishing copy_from to trial11545420-34 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1099: got expected ENOENT (src dne) 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1099: finishing copy_from racing read to trial11545420-34 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: finishing write tid 3 to trial11545420-25 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: finishing write tid 4 to trial11545420-25 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: finishing write tid 5 to trial11545420-25 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: oid 25 updating version 0 to 407 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: oid 25 updating version 407 to 408 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: oid 25 version 408 is already newer than 406 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: oid 25 updating version 408 to 409 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 409 (ObjNum 429 snap 108 seq_num 429) dirty exists 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1098: left oid 25 (ObjNum 429 snap 108 seq_num 429) 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1101: finishing write tid 1 to trial11545420-47 2026-02-17T05:52:03.392 INFO:tasks.rados.rados.0.trial115.stdout:1101: finishing write tid 2 to trial11545420-47 2026-02-17T05:52:03.393 INFO:tasks.rados.rados.0.trial115.stdout:1098: done (3 left) 2026-02-17T05:52:03.393 INFO:tasks.rados.rados.0.trial115.stdout:1099: done (2 left) 2026-02-17T05:52:03.393 INFO:tasks.rados.rados.0.trial115.stdout:1102: done (1 left) 2026-02-17T05:52:03.393 INFO:tasks.rados.rados.0.trial115.stdout:1103: write (excl) oid 42 current snap is 108 2026-02-17T05:52:03.393 INFO:tasks.rados.rados.0.trial115.stdout:1103: seq_num 432 ranges {460624=623336,1846574=698150,3163674=96950} 2026-02-17T05:52:03.397 INFO:tasks.rados.rados.0.trial115.stdout:1103: writing trial11545420-42 from 460624 to 1083960 tid 1 2026-02-17T05:52:03.402 INFO:tasks.rados.rados.0.trial115.stdout:1103: writing trial11545420-42 from 1846574 to 2544724 tid 2 2026-02-17T05:52:03.405 INFO:tasks.rados.rados.0.trial115.stdout:1103: writing trial11545420-42 from 3163674 to 3260624 tid 3 2026-02-17T05:52:03.405 INFO:tasks.rados.rados.0.trial115.stdout:1104: snap_remove snap 102 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1104: done (2 left) 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1105: copy_from oid 40 from oid 6 current snap is 108 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1101: finishing write tid 3 to trial11545420-47 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1101: finishing write tid 4 to trial11545420-47 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1101: finishing write tid 5 to trial11545420-47 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1101: oid 47 updating version 0 to 458 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1101: oid 47 updating version 458 to 460 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1101: oid 47 version 460 is already newer than 459 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:1101: oid 47 version 460 is already newer than 451 2026-02-17T05:52:04.382 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 460 (ObjNum 430 snap 108 seq_num 430) dirty exists 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1101: left oid 47 (ObjNum 430 snap 108 seq_num 430) 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: finishing write tid 1 to trial11545420-42 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: finishing write tid 2 to trial11545420-42 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: finishing write tid 3 to trial11545420-42 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: finishing write tid 4 to trial11545420-42 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: finishing write tid 5 to trial11545420-42 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: oid 42 updating version 0 to 552 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: oid 42 version 552 is already newer than 551 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: oid 42 updating version 552 to 553 2026-02-17T05:52:04.383 INFO:tasks.rados.rados.0.trial115.stdout:1103: oid 42 updating version 553 to 554 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 554 (ObjNum 432 snap 108 seq_num 432) dirty exists 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:1103: left oid 42 (ObjNum 432 snap 108 seq_num 432) 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:1101: done (2 left) 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:1103: done (1 left) 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:1106: read oid 23 snap -1 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:1106: expect deleted 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:1107: read oid 16 snap -1 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:1107: expect (ObjNum 425 snap 107 seq_num 425) 2026-02-17T05:52:04.384 INFO:tasks.rados.rados.0.trial115.stdout:1108: delete oid 43 current snap is 108 2026-02-17T05:52:04.385 INFO:tasks.rados.rados.0.trial115.stdout:1105: finishing copy_from to trial11545420-40 2026-02-17T05:52:04.385 INFO:tasks.rados.rados.0.trial115.stdout:1105: got expected ENOENT (src dne) 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1105: finishing copy_from racing read to trial11545420-40 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1105: done (3 left) 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1106: done (2 left) 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1108: done (1 left) 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1109: copy_from oid 42 from oid 22 current snap is 108 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1110: read oid 18 snap -1 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1110: expect (ObjNum 404 snap 103 seq_num 404) 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1111: rollback oid 14 current snap is 108 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 105 2026-02-17T05:52:04.386 INFO:tasks.rados.rados.0.trial115.stdout:1112: snap_create 2026-02-17T05:52:04.387 INFO:tasks.rados.rados.0.trial115.stdout:1109: finishing copy_from to trial11545420-42 2026-02-17T05:52:04.387 INFO:tasks.rados.rados.0.trial115.stdout:1109: got expected ENOENT (src dne) 2026-02-17T05:52:04.387 INFO:tasks.rados.rados.0.trial115.stdout:1109: finishing copy_from racing read to trial11545420-42 2026-02-17T05:52:04.388 INFO:tasks.rados.rados.0.trial115.stdout:1107: expect (ObjNum 425 snap 107 seq_num 425) 2026-02-17T05:52:04.512 INFO:tasks.rados.rados.0.trial115.stdout:1110: expect (ObjNum 404 snap 103 seq_num 404) 2026-02-17T05:52:04.577 INFO:tasks.rados.rados.0.trial115.stdout:1111: finishing rollback tid 1 to trial11545420-14 2026-02-17T05:52:04.577 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 292 (ObjNum 65 snap 0 seq_num 1634300532) dirty dne 2026-02-17T05:52:05.383 INFO:tasks.rados.rados.0.trial115.stdout:1107: done (4 left) 2026-02-17T05:52:05.384 INFO:tasks.rados.rados.0.trial115.stdout:1109: done (3 left) 2026-02-17T05:52:05.384 INFO:tasks.rados.rados.0.trial115.stdout:1110: done (2 left) 2026-02-17T05:52:05.384 INFO:tasks.rados.rados.0.trial115.stdout:1111: done (1 left) 2026-02-17T05:52:05.384 INFO:tasks.rados.rados.0.trial115.stdout:1112: done (0 left) 2026-02-17T05:52:05.384 INFO:tasks.rados.rados.0.trial115.stdout:1113: write (excl) oid 17 current snap is 109 2026-02-17T05:52:05.384 INFO:tasks.rados.rados.0.trial115.stdout:1113: seq_num 434 ranges {494238=490187,1563538=701819,2818520=475718} 2026-02-17T05:52:05.387 INFO:tasks.rados.rados.0.trial115.stdout:1113: writing trial11545420-17 from 494238 to 984425 tid 1 2026-02-17T05:52:05.392 INFO:tasks.rados.rados.0.trial115.stdout:1113: writing trial11545420-17 from 1563538 to 2265357 tid 2 2026-02-17T05:52:05.396 INFO:tasks.rados.rados.0.trial115.stdout:1113: writing trial11545420-17 from 2818520 to 3294238 tid 3 2026-02-17T05:52:05.396 INFO:tasks.rados.rados.0.trial115.stdout:1114: write (excl) oid 16 current snap is 109 2026-02-17T05:52:05.396 INFO:tasks.rados.rados.0.trial115.stdout:1114: seq_num 435 ranges {511045=565436,1622020=445477,2729590=556855} 2026-02-17T05:52:05.400 INFO:tasks.rados.rados.0.trial115.stdout:1114: writing trial11545420-16 from 511045 to 1076481 tid 1 2026-02-17T05:52:05.404 INFO:tasks.rados.rados.0.trial115.stdout:1114: writing trial11545420-16 from 1622020 to 2067497 tid 2 2026-02-17T05:52:05.409 INFO:tasks.rados.rados.0.trial115.stdout:1114: writing trial11545420-16 from 2729590 to 3286445 tid 3 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1115: delete oid 3 current snap is 109 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1113: finishing write tid 1 to trial11545420-17 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1113: finishing write tid 2 to trial11545420-17 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1113: finishing write tid 3 to trial11545420-17 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1113: finishing write tid 4 to trial11545420-17 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1113: finishing write tid 5 to trial11545420-17 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1113: oid 17 updating version 0 to 191 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1113: oid 17 version 191 is already newer than 190 2026-02-17T05:52:05.410 INFO:tasks.rados.rados.0.trial115.stdout:1113: oid 17 version 191 is already newer than 189 2026-02-17T05:52:05.411 INFO:tasks.rados.rados.0.trial115.stdout:1113: oid 17 version 191 is already newer than 188 2026-02-17T05:52:05.411 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 191 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:52:05.411 INFO:tasks.rados.rados.0.trial115.stdout:1113: left oid 17 (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:52:05.411 INFO:tasks.rados.rados.0.trial115.stdout:1114: finishing write tid 1 to trial11545420-16 2026-02-17T05:52:05.411 INFO:tasks.rados.rados.0.trial115.stdout:1114: finishing write tid 2 to trial11545420-16 2026-02-17T05:52:05.412 INFO:tasks.rados.rados.0.trial115.stdout:1114: finishing write tid 3 to trial11545420-16 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1114: finishing write tid 4 to trial11545420-16 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1114: finishing write tid 5 to trial11545420-16 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1114: oid 16 updating version 0 to 523 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1114: oid 16 version 523 is already newer than 522 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1114: oid 16 updating version 523 to 525 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1114: oid 16 version 525 is already newer than 524 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 525 (ObjNum 435 snap 109 seq_num 435) dirty exists 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1114: left oid 16 (ObjNum 435 snap 109 seq_num 435) 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1113: done (2 left) 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1114: done (1 left) 2026-02-17T05:52:05.413 INFO:tasks.rados.rados.0.trial115.stdout:1115: done (0 left) 2026-02-17T05:52:05.414 INFO:tasks.rados.rados.0.trial115.stdout:1116: read oid 48 snap -1 2026-02-17T05:52:05.414 INFO:tasks.rados.rados.0.trial115.stdout:1116: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:52:05.414 INFO:tasks.rados.rados.0.trial115.stdout:1117: snap_remove snap 98 2026-02-17T05:52:06.375 INFO:tasks.rados.rados.0.trial115.stdout:1117: done (1 left) 2026-02-17T05:52:06.375 INFO:tasks.rados.rados.0.trial115.stdout:1118: write (excl) oid 14 current snap is 109 2026-02-17T05:52:06.375 INFO:tasks.rados.rados.0.trial115.stdout:1116: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:52:06.518 INFO:tasks.rados.rados.0.trial115.stdout:1118: seq_num 437 ranges {544659=715934,1738984=449146,2784436=560223} 2026-02-17T05:52:06.523 INFO:tasks.rados.rados.0.trial115.stdout:1118: writing trial11545420-14 from 544659 to 1260593 tid 1 2026-02-17T05:52:06.527 INFO:tasks.rados.rados.0.trial115.stdout:1118: writing trial11545420-14 from 1738984 to 2188130 tid 2 2026-02-17T05:52:06.532 INFO:tasks.rados.rados.0.trial115.stdout:1118: writing trial11545420-14 from 2784436 to 3344659 tid 3 2026-02-17T05:52:06.532 INFO:tasks.rados.rados.0.trial115.stdout:1116: done (1 left) 2026-02-17T05:52:06.532 INFO:tasks.rados.rados.0.trial115.stdout:1119: copy_from oid 4 from oid 48 current snap is 109 2026-02-17T05:52:06.532 INFO:tasks.rados.rados.0.trial115.stdout:1118: finishing write tid 1 to trial11545420-14 2026-02-17T05:52:06.532 INFO:tasks.rados.rados.0.trial115.stdout:1118: finishing write tid 2 to trial11545420-14 2026-02-17T05:52:06.532 INFO:tasks.rados.rados.0.trial115.stdout:1120: snap_create 2026-02-17T05:52:06.533 INFO:tasks.rados.rados.0.trial115.stdout:1118: finishing write tid 3 to trial11545420-14 2026-02-17T05:52:06.534 INFO:tasks.rados.rados.0.trial115.stdout:1118: finishing write tid 4 to trial11545420-14 2026-02-17T05:52:06.534 INFO:tasks.rados.rados.0.trial115.stdout:1118: finishing write tid 5 to trial11545420-14 2026-02-17T05:52:06.534 INFO:tasks.rados.rados.0.trial115.stdout:1118: oid 14 updating version 0 to 390 2026-02-17T05:52:06.534 INFO:tasks.rados.rados.0.trial115.stdout:1118: oid 14 updating version 390 to 391 2026-02-17T05:52:06.534 INFO:tasks.rados.rados.0.trial115.stdout:1118: oid 14 version 391 is already newer than 388 2026-02-17T05:52:06.534 INFO:tasks.rados.rados.0.trial115.stdout:1118: oid 14 version 391 is already newer than 389 2026-02-17T05:52:06.534 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 391 (ObjNum 437 snap 109 seq_num 437) dirty exists 2026-02-17T05:52:06.534 INFO:tasks.rados.rados.0.trial115.stdout:1118: left oid 14 (ObjNum 437 snap 109 seq_num 437) 2026-02-17T05:52:06.537 INFO:tasks.rados.rados.0.trial115.stdout:1119: finishing copy_from racing read to trial11545420-4 2026-02-17T05:52:06.539 INFO:tasks.rados.rados.0.trial115.stdout:1119: finishing copy_from to trial11545420-4 2026-02-17T05:52:06.539 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 176 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:52:07.387 INFO:tasks.rados.rados.0.trial115.stdout:1118: done (2 left) 2026-02-17T05:52:07.387 INFO:tasks.rados.rados.0.trial115.stdout:1119: done (1 left) 2026-02-17T05:52:07.388 INFO:tasks.rados.rados.0.trial115.stdout:1120: done (0 left) 2026-02-17T05:52:07.388 INFO:tasks.rados.rados.0.trial115.stdout:1121: read oid 32 snap -1 2026-02-17T05:52:07.388 INFO:tasks.rados.rados.0.trial115.stdout:1121: expect deleted 2026-02-17T05:52:07.388 INFO:tasks.rados.rados.0.trial115.stdout:1122: write (excl) oid 42 current snap is 110 2026-02-17T05:52:07.388 INFO:tasks.rados.rados.0.trial115.stdout:1122: seq_num 438 ranges {561466=791183,2081113=592804,3095506=265960} 2026-02-17T05:52:07.392 INFO:tasks.rados.rados.0.trial115.stdout:1122: writing trial11545420-42 from 561466 to 1352649 tid 1 2026-02-17T05:52:07.397 INFO:tasks.rados.rados.0.trial115.stdout:1122: writing trial11545420-42 from 2081113 to 2673917 tid 2 2026-02-17T05:52:07.400 INFO:tasks.rados.rados.0.trial115.stdout:1122: writing trial11545420-42 from 3095506 to 3361466 tid 3 2026-02-17T05:52:07.400 INFO:tasks.rados.rados.0.trial115.stdout:1123: write oid 3 current snap is 110 2026-02-17T05:52:07.400 INFO:tasks.rados.rados.0.trial115.stdout:1123: seq_num 439 ranges {578273=466432,1739595=452815,2722929=655344} 2026-02-17T05:52:07.404 INFO:tasks.rados.rados.0.trial115.stdout:1123: writing trial11545420-3 from 578273 to 1044705 tid 1 2026-02-17T05:52:07.408 INFO:tasks.rados.rados.0.trial115.stdout:1123: writing trial11545420-3 from 1739595 to 2192410 tid 2 2026-02-17T05:52:07.413 INFO:tasks.rados.rados.0.trial115.stdout:1123: writing trial11545420-3 from 2722929 to 3378273 tid 3 2026-02-17T05:52:07.413 INFO:tasks.rados.rados.0.trial115.stdout:1124: snap_remove snap 52 2026-02-17T05:52:09.142 INFO:tasks.rados.rados.0.trial115.stdout:1124: done (3 left) 2026-02-17T05:52:09.142 INFO:tasks.rados.rados.0.trial115.stdout:1125: rollback oid 18 current snap is 110 2026-02-17T05:52:09.142 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 103 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: finishing write tid 1 to trial11545420-42 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: finishing write tid 2 to trial11545420-42 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: finishing write tid 3 to trial11545420-42 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: finishing write tid 4 to trial11545420-42 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: finishing write tid 5 to trial11545420-42 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: oid 42 updating version 0 to 575 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: oid 42 version 575 is already newer than 574 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: oid 42 version 575 is already newer than 573 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: oid 42 version 575 is already newer than 572 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 575 (ObjNum 438 snap 110 seq_num 438) dirty exists 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1122: left oid 42 (ObjNum 438 snap 110 seq_num 438) 2026-02-17T05:52:09.143 INFO:tasks.rados.rados.0.trial115.stdout:1123: finishing write tid 1 to trial11545420-3 2026-02-17T05:52:09.144 INFO:tasks.rados.rados.0.trial115.stdout:1121: done (3 left) 2026-02-17T05:52:09.144 INFO:tasks.rados.rados.0.trial115.stdout:1122: done (2 left) 2026-02-17T05:52:09.144 INFO:tasks.rados.rados.0.trial115.stdout:1126: write (excl) oid 8 current snap is 110 2026-02-17T05:52:09.144 INFO:tasks.rados.rados.0.trial115.stdout:1126: seq_num 440 ranges {595080=541681,1798077=596473,3150352=244728} 2026-02-17T05:52:09.146 INFO:tasks.rados.rados.0.trial115.stdout:1126: writing trial11545420-8 from 595080 to 1136761 tid 1 2026-02-17T05:52:09.151 INFO:tasks.rados.rados.0.trial115.stdout:1126: writing trial11545420-8 from 1798077 to 2394550 tid 2 2026-02-17T05:52:09.155 INFO:tasks.rados.rados.0.trial115.stdout:1126: writing trial11545420-8 from 3150352 to 3395080 tid 3 2026-02-17T05:52:09.155 INFO:tasks.rados.rados.0.trial115.stdout:1127: snap_remove snap 101 2026-02-17T05:52:10.146 INFO:tasks.rados.rados.0.trial115.stdout:1127: done (3 left) 2026-02-17T05:52:10.146 INFO:tasks.rados.rados.0.trial115.stdout:1128: delete oid 40 current snap is 110 2026-02-17T05:52:10.146 INFO:tasks.rados.rados.0.trial115.stdout:1123: finishing write tid 2 to trial11545420-3 2026-02-17T05:52:10.146 INFO:tasks.rados.rados.0.trial115.stdout:1123: finishing write tid 3 to trial11545420-3 2026-02-17T05:52:10.146 INFO:tasks.rados.rados.0.trial115.stdout:1123: finishing write tid 4 to trial11545420-3 2026-02-17T05:52:10.146 INFO:tasks.rados.rados.0.trial115.stdout:1123: finishing write tid 5 to trial11545420-3 2026-02-17T05:52:10.146 INFO:tasks.rados.rados.0.trial115.stdout:1123: oid 3 updating version 0 to 482 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1123: oid 3 version 482 is already newer than 481 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1123: oid 3 updating version 482 to 483 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1123: oid 3 updating version 483 to 484 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 484 (ObjNum 439 snap 110 seq_num 439) dirty exists 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1123: left oid 3 (ObjNum 439 snap 110 seq_num 439) 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1125: finishing rollback tid 0 to trial11545420-18 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1125: finishing rollback tid 1 to trial11545420-18 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1125: finishing rollback tid 2 to trial11545420-18 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 417 (ObjNum 404 snap 103 seq_num 404) dirty exists 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1126: finishing write tid 1 to trial11545420-8 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1126: finishing write tid 2 to trial11545420-8 2026-02-17T05:52:10.147 INFO:tasks.rados.rados.0.trial115.stdout:1126: finishing write tid 3 to trial11545420-8 2026-02-17T05:52:10.148 INFO:tasks.rados.rados.0.trial115.stdout:1126: finishing write tid 4 to trial11545420-8 2026-02-17T05:52:10.148 INFO:tasks.rados.rados.0.trial115.stdout:1126: finishing write tid 5 to trial11545420-8 2026-02-17T05:52:10.148 INFO:tasks.rados.rados.0.trial115.stdout:1126: oid 8 updating version 0 to 187 2026-02-17T05:52:10.148 INFO:tasks.rados.rados.0.trial115.stdout:1126: oid 8 updating version 187 to 189 2026-02-17T05:52:10.148 INFO:tasks.rados.rados.0.trial115.stdout:1126: oid 8 updating version 189 to 190 2026-02-17T05:52:10.148 INFO:tasks.rados.rados.0.trial115.stdout:1126: oid 8 version 190 is already newer than 188 2026-02-17T05:52:10.148 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 190 (ObjNum 440 snap 110 seq_num 440) dirty exists 2026-02-17T05:52:10.148 INFO:tasks.rados.rados.0.trial115.stdout:1126: left oid 8 (ObjNum 440 snap 110 seq_num 440) 2026-02-17T05:52:10.149 INFO:tasks.rados.rados.0.trial115.stdout:1123: done (3 left) 2026-02-17T05:52:10.149 INFO:tasks.rados.rados.0.trial115.stdout:1125: done (2 left) 2026-02-17T05:52:10.149 INFO:tasks.rados.rados.0.trial115.stdout:1126: done (1 left) 2026-02-17T05:52:10.149 INFO:tasks.rados.rados.0.trial115.stdout:1128: done (0 left) 2026-02-17T05:52:10.149 INFO:tasks.rados.rados.0.trial115.stdout:1129: write oid 30 current snap is 110 2026-02-17T05:52:10.149 INFO:tasks.rados.rados.0.trial115.stdout:1129: seq_num 442 ranges {628694=408532,1515041=600142,2805198=552112} 2026-02-17T05:52:10.152 INFO:tasks.rados.rados.0.trial115.stdout:1129: writing trial11545420-30 from 628694 to 1037226 tid 1 2026-02-17T05:52:10.156 INFO:tasks.rados.rados.0.trial115.stdout:1129: writing trial11545420-30 from 1515041 to 2115183 tid 2 2026-02-17T05:52:10.161 INFO:tasks.rados.rados.0.trial115.stdout:1129: writing trial11545420-30 from 2805198 to 3357310 tid 3 2026-02-17T05:52:10.161 INFO:tasks.rados.rados.0.trial115.stdout:1130: copy_from oid 1 from oid 28 current snap is 110 2026-02-17T05:52:10.162 INFO:tasks.rados.rados.0.trial115.stdout:1129: finishing write tid 1 to trial11545420-30 2026-02-17T05:52:10.162 INFO:tasks.rados.rados.0.trial115.stdout:1129: finishing write tid 2 to trial11545420-30 2026-02-17T05:52:10.162 INFO:tasks.rados.rados.0.trial115.stdout:1131: rollback oid 27 current snap is 110 2026-02-17T05:52:10.162 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 103 2026-02-17T05:52:10.162 INFO:tasks.rados.rados.0.trial115.stdout:1132: delete oid 47 current snap is 110 2026-02-17T05:52:10.162 INFO:tasks.rados.rados.0.trial115.stdout:1130: finishing copy_from to trial11545420-1 2026-02-17T05:52:10.163 INFO:tasks.rados.rados.0.trial115.stdout:1130: got expected ENOENT (src dne) 2026-02-17T05:52:10.163 INFO:tasks.rados.rados.0.trial115.stdout:1130: finishing copy_from racing read to trial11545420-1 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:1129: finishing write tid 3 to trial11545420-30 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:1129: finishing write tid 4 to trial11545420-30 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:1129: finishing write tid 5 to trial11545420-30 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:1129: oid 30 updating version 0 to 449 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:1129: oid 30 version 449 is already newer than 447 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:1129: oid 30 version 449 is already newer than 448 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:1129: oid 30 version 449 is already newer than 446 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 449 (ObjNum 442 snap 110 seq_num 442) dirty exists 2026-02-17T05:52:10.165 INFO:tasks.rados.rados.0.trial115.stdout:1129: left oid 30 (ObjNum 442 snap 110 seq_num 442) 2026-02-17T05:52:10.166 INFO:tasks.rados.rados.0.trial115.stdout:1131: finishing rollback tid 0 to trial11545420-27 2026-02-17T05:52:10.166 INFO:tasks.rados.rados.0.trial115.stdout:1131: finishing rollback tid 1 to trial11545420-27 2026-02-17T05:52:10.166 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 452 (ObjNum 129 snap 0 seq_num 1333748552) dirty dne 2026-02-17T05:52:10.166 INFO:tasks.rados.rados.0.trial115.stdout:1129: done (3 left) 2026-02-17T05:52:10.166 INFO:tasks.rados.rados.0.trial115.stdout:1130: done (2 left) 2026-02-17T05:52:10.166 INFO:tasks.rados.rados.0.trial115.stdout:1131: done (1 left) 2026-02-17T05:52:10.166 INFO:tasks.rados.rados.0.trial115.stdout:1132: done (0 left) 2026-02-17T05:52:10.166 INFO:tasks.rados.rados.0.trial115.stdout:1133: delete oid 24 current snap is 110 2026-02-17T05:52:10.167 INFO:tasks.rados.rados.0.trial115.stdout:1133: done (0 left) 2026-02-17T05:52:10.167 INFO:tasks.rados.rados.0.trial115.stdout:1134: write oid 18 current snap is 110 2026-02-17T05:52:10.167 INFO:tasks.rados.rados.0.trial115.stdout:1134: seq_num 445 ranges {679115=634279,2090487=463822,3003820=475295} 2026-02-17T05:52:10.171 INFO:tasks.rados.rados.0.trial115.stdout:1134: writing trial11545420-18 from 679115 to 1313394 tid 1 2026-02-17T05:52:10.175 INFO:tasks.rados.rados.0.trial115.stdout:1134: writing trial11545420-18 from 2090487 to 2554309 tid 2 2026-02-17T05:52:10.179 INFO:tasks.rados.rados.0.trial115.stdout:1134: writing trial11545420-18 from 3003820 to 3479115 tid 3 2026-02-17T05:52:10.179 INFO:tasks.rados.rados.0.trial115.stdout:1135: snap_remove snap 109 2026-02-17T05:52:11.148 INFO:tasks.rados.rados.0.trial115.stdout:1135: done (1 left) 2026-02-17T05:52:11.148 INFO:tasks.rados.rados.0.trial115.stdout:1136: copy_from oid 4 from oid 34 current snap is 110 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: finishing write tid 1 to trial11545420-18 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: finishing write tid 2 to trial11545420-18 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: finishing write tid 3 to trial11545420-18 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: finishing write tid 4 to trial11545420-18 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: finishing write tid 5 to trial11545420-18 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: oid 18 updating version 0 to 426 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: oid 18 updating version 426 to 427 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: oid 18 version 427 is already newer than 425 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: oid 18 version 427 is already newer than 424 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 427 (ObjNum 445 snap 110 seq_num 445) dirty exists 2026-02-17T05:52:11.149 INFO:tasks.rados.rados.0.trial115.stdout:1134: left oid 18 (ObjNum 445 snap 110 seq_num 445) 2026-02-17T05:52:11.150 INFO:tasks.rados.rados.0.trial115.stdout:1134: done (1 left) 2026-02-17T05:52:11.150 INFO:tasks.rados.rados.0.trial115.stdout:1137: delete oid 22 current snap is 110 2026-02-17T05:52:11.151 INFO:tasks.rados.rados.0.trial115.stdout:1136: finishing copy_from to trial11545420-4 2026-02-17T05:52:11.151 INFO:tasks.rados.rados.0.trial115.stdout:1136: got expected ENOENT (src dne) 2026-02-17T05:52:11.151 INFO:tasks.rados.rados.0.trial115.stdout:1137: done (1 left) 2026-02-17T05:52:11.151 INFO:tasks.rados.rados.0.trial115.stdout:1138: read oid 2 snap 105 2026-02-17T05:52:11.151 INFO:tasks.rados.rados.0.trial115.stdout:1138: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:52:11.151 INFO:tasks.rados.rados.0.trial115.stdout:1139: snap_remove snap 91 2026-02-17T05:52:11.374 INFO:tasks.rados.rados.0.trial115.stdout:1139: done (2 left) 2026-02-17T05:52:11.375 INFO:tasks.rados.rados.0.trial115.stdout:1140: write (excl) oid 21 current snap is 110 2026-02-17T05:52:11.375 INFO:tasks.rados.rados.0.trial115.stdout:1140: seq_num 447 ranges {712729=784777,2091098=467491,3342313=170416} 2026-02-17T05:52:11.381 INFO:tasks.rados.rados.0.trial115.stdout:1140: writing trial11545420-21 from 712729 to 1497506 tid 1 2026-02-17T05:52:11.385 INFO:tasks.rados.rados.0.trial115.stdout:1140: writing trial11545420-21 from 2091098 to 2558589 tid 2 2026-02-17T05:52:11.390 INFO:tasks.rados.rados.0.trial115.stdout:1140: writing trial11545420-21 from 3342313 to 3512729 tid 3 2026-02-17T05:52:11.390 INFO:tasks.rados.rados.0.trial115.stdout:1141: rollback oid 32 current snap is 110 2026-02-17T05:52:11.390 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 105 2026-02-17T05:52:11.390 INFO:tasks.rados.rados.0.trial115.stdout:1136: finishing copy_from racing read to trial11545420-4 2026-02-17T05:52:11.390 INFO:tasks.rados.rados.0.trial115.stdout:1136: done (3 left) 2026-02-17T05:52:11.390 INFO:tasks.rados.rados.0.trial115.stdout:1142: snap_remove snap 97 2026-02-17T05:52:12.378 INFO:tasks.rados.rados.0.trial115.stdout:1142: done (3 left) 2026-02-17T05:52:12.379 INFO:tasks.rados.rados.0.trial115.stdout:1143: delete oid 27 current snap is 110 2026-02-17T05:52:12.379 INFO:tasks.rados.rados.0.trial115.stdout:1138: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:52:12.429 INFO:tasks.rados.rados.0.trial115.stdout:1140: finishing write tid 1 to trial11545420-21 2026-02-17T05:52:12.429 INFO:tasks.rados.rados.0.trial115.stdout:1140: finishing write tid 2 to trial11545420-21 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1138: done (3 left) 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1143: done (2 left) 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1144: snap_create 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1141: finishing rollback tid 1 to trial11545420-32 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 270 (ObjNum 129 snap 0 seq_num 1334568736) dirty dne 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1140: finishing write tid 3 to trial11545420-21 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1140: finishing write tid 4 to trial11545420-21 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1140: finishing write tid 5 to trial11545420-21 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1140: oid 21 updating version 0 to 440 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1140: oid 21 version 440 is already newer than 439 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1140: oid 21 version 440 is already newer than 438 2026-02-17T05:52:12.430 INFO:tasks.rados.rados.0.trial115.stdout:1140: oid 21 version 440 is already newer than 435 2026-02-17T05:52:12.431 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 440 (ObjNum 447 snap 110 seq_num 447) dirty exists 2026-02-17T05:52:12.431 INFO:tasks.rados.rados.0.trial115.stdout:1140: left oid 21 (ObjNum 447 snap 110 seq_num 447) 2026-02-17T05:52:13.379 INFO:tasks.rados.rados.0.trial115.stdout:1140: done (2 left) 2026-02-17T05:52:13.379 INFO:tasks.rados.rados.0.trial115.stdout:1141: done (1 left) 2026-02-17T05:52:13.379 INFO:tasks.rados.rados.0.trial115.stdout:1144: done (0 left) 2026-02-17T05:52:13.380 INFO:tasks.rados.rados.0.trial115.stdout:1145: snap_create 2026-02-17T05:52:14.381 INFO:tasks.rados.rados.0.trial115.stdout:1145: done (0 left) 2026-02-17T05:52:14.381 INFO:tasks.rados.rados.0.trial115.stdout:1146: delete oid 12 current snap is 112 2026-02-17T05:52:14.383 INFO:tasks.rados.rados.0.trial115.stdout:1146: done (0 left) 2026-02-17T05:52:14.383 INFO:tasks.rados.rados.0.trial115.stdout:1147: write oid 15 current snap is 112 2026-02-17T05:52:14.384 INFO:tasks.rados.rados.0.trial115.stdout:1147: seq_num 450 ranges {763150=726877,1982897=614818,3024582=474994} 2026-02-17T05:52:14.389 INFO:tasks.rados.rados.0.trial115.stdout:1147: writing trial11545420-15 from 763150 to 1490027 tid 1 2026-02-17T05:52:14.394 INFO:tasks.rados.rados.0.trial115.stdout:1147: writing trial11545420-15 from 1982897 to 2597715 tid 2 2026-02-17T05:52:14.397 INFO:tasks.rados.rados.0.trial115.stdout:1147: writing trial11545420-15 from 3024582 to 3499576 tid 3 2026-02-17T05:52:14.397 INFO:tasks.rados.rados.0.trial115.stdout:1148: write (excl) oid 40 current snap is 112 2026-02-17T05:52:14.398 INFO:tasks.rados.rados.0.trial115.stdout:1148: seq_num 451 ranges {779957=402126,1925026=758476,3335652=244305} 2026-02-17T05:52:14.401 INFO:tasks.rados.rados.0.trial115.stdout:1148: writing trial11545420-40 from 779957 to 1182083 tid 1 2026-02-17T05:52:14.407 INFO:tasks.rados.rados.0.trial115.stdout:1148: writing trial11545420-40 from 1925026 to 2683502 tid 2 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1148: writing trial11545420-40 from 3335652 to 3579957 tid 3 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1149: rollback oid 23 current snap is 112 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 23 to 111 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: finishing write tid 1 to trial11545420-15 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: finishing write tid 2 to trial11545420-15 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: finishing write tid 3 to trial11545420-15 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: finishing write tid 4 to trial11545420-15 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1150: read oid 24 snap -1 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1150: expect deleted 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1151: copy_from oid 42 from oid 6 current snap is 112 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: finishing write tid 5 to trial11545420-15 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: oid 15 updating version 0 to 519 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: oid 15 version 519 is already newer than 516 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: oid 15 version 519 is already newer than 517 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: oid 15 version 519 is already newer than 518 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 519 (ObjNum 450 snap 112 seq_num 450) dirty exists 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: left oid 15 (ObjNum 450 snap 112 seq_num 450) 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1148: finishing write tid 1 to trial11545420-40 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1148: finishing write tid 2 to trial11545420-40 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1147: done (4 left) 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1152: write (excl) oid 13 current snap is 112 2026-02-17T05:52:14.411 INFO:tasks.rados.rados.0.trial115.stdout:1152: seq_num 452 ranges {796764=477375,1983508=618487,3363075=233689} 2026-02-17T05:52:14.415 INFO:tasks.rados.rados.0.trial115.stdout:1152: writing trial11545420-13 from 796764 to 1274139 tid 1 2026-02-17T05:52:14.421 INFO:tasks.rados.rados.0.trial115.stdout:1152: writing trial11545420-13 from 1983508 to 2601995 tid 2 2026-02-17T05:52:14.425 INFO:tasks.rados.rados.0.trial115.stdout:1152: writing trial11545420-13 from 3363075 to 3596764 tid 3 2026-02-17T05:52:14.425 INFO:tasks.rados.rados.0.trial115.stdout:1153: snap_remove snap 86 2026-02-17T05:52:15.386 INFO:tasks.rados.rados.0.trial115.stdout:1153: done (5 left) 2026-02-17T05:52:15.387 INFO:tasks.rados.rados.0.trial115.stdout:1154: copy_from oid 38 from oid 5 current snap is 112 2026-02-17T05:52:15.387 INFO:tasks.rados.rados.0.trial115.stdout:1151: finishing copy_from to trial11545420-42 2026-02-17T05:52:15.387 INFO:tasks.rados.rados.0.trial115.stdout:1151: got expected ENOENT (src dne) 2026-02-17T05:52:15.387 INFO:tasks.rados.rados.0.trial115.stdout:1151: finishing copy_from racing read to trial11545420-42 2026-02-17T05:52:15.387 INFO:tasks.rados.rados.0.trial115.stdout:1149: finishing rollback tid 1 to trial11545420-23 2026-02-17T05:52:15.387 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 180 (ObjNum 177 snap 0 seq_num 1333854736) dirty dne 2026-02-17T05:52:15.388 INFO:tasks.rados.rados.0.trial115.stdout:1148: finishing write tid 3 to trial11545420-40 2026-02-17T05:52:15.388 INFO:tasks.rados.rados.0.trial115.stdout:1148: finishing write tid 4 to trial11545420-40 2026-02-17T05:52:15.388 INFO:tasks.rados.rados.0.trial115.stdout:1148: finishing write tid 5 to trial11545420-40 2026-02-17T05:52:15.388 INFO:tasks.rados.rados.0.trial115.stdout:1148: oid 40 updating version 0 to 303 2026-02-17T05:52:15.388 INFO:tasks.rados.rados.0.trial115.stdout:1148: oid 40 updating version 303 to 304 2026-02-17T05:52:15.388 INFO:tasks.rados.rados.0.trial115.stdout:1148: oid 40 version 304 is already newer than 301 2026-02-17T05:52:15.389 INFO:tasks.rados.rados.0.trial115.stdout:1148: oid 40 version 304 is already newer than 302 2026-02-17T05:52:15.389 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 304 (ObjNum 451 snap 112 seq_num 451) dirty exists 2026-02-17T05:52:15.389 INFO:tasks.rados.rados.0.trial115.stdout:1148: left oid 40 (ObjNum 451 snap 112 seq_num 451) 2026-02-17T05:52:15.389 INFO:tasks.rados.rados.0.trial115.stdout:1152: finishing write tid 1 to trial11545420-13 2026-02-17T05:52:15.389 INFO:tasks.rados.rados.0.trial115.stdout:1152: finishing write tid 2 to trial11545420-13 2026-02-17T05:52:15.389 INFO:tasks.rados.rados.0.trial115.stdout:1152: finishing write tid 3 to trial11545420-13 2026-02-17T05:52:15.390 INFO:tasks.rados.rados.0.trial115.stdout:1148: done (5 left) 2026-02-17T05:52:15.390 INFO:tasks.rados.rados.0.trial115.stdout:1149: done (4 left) 2026-02-17T05:52:15.390 INFO:tasks.rados.rados.0.trial115.stdout:1150: done (3 left) 2026-02-17T05:52:15.390 INFO:tasks.rados.rados.0.trial115.stdout:1151: done (2 left) 2026-02-17T05:52:15.390 INFO:tasks.rados.rados.0.trial115.stdout:1155: copy_from oid 45 from oid 25 current snap is 112 2026-02-17T05:52:15.390 INFO:tasks.rados.rados.0.trial115.stdout:1152: finishing write tid 4 to trial11545420-13 2026-02-17T05:52:15.390 INFO:tasks.rados.rados.0.trial115.stdout:1152: finishing write tid 5 to trial11545420-13 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1152: oid 13 updating version 0 to 403 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1152: oid 13 updating version 403 to 405 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1152: oid 13 version 405 is already newer than 404 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1152: oid 13 version 405 is already newer than 402 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 405 (ObjNum 452 snap 112 seq_num 452) dirty exists 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1152: left oid 13 (ObjNum 452 snap 112 seq_num 452) 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1152: done (2 left) 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1156: read oid 16 snap 107 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1156: expect (ObjNum 425 snap 107 seq_num 425) 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1157: write (excl) oid 10 current snap is 112 2026-02-17T05:52:15.391 INFO:tasks.rados.rados.0.trial115.stdout:1157: seq_num 453 ranges {413571=552624,1641990=762145,2990498=425163} 2026-02-17T05:52:15.392 INFO:tasks.rados.rados.0.trial115.stdout:1157: writing trial11545420-10 from 413571 to 966195 tid 1 2026-02-17T05:52:15.393 INFO:tasks.rados.rados.0.trial115.stdout:1157: writing trial11545420-10 from 1641990 to 2404135 tid 2 2026-02-17T05:52:15.398 INFO:tasks.rados.rados.0.trial115.stdout:1157: writing trial11545420-10 from 2990498 to 3415661 tid 3 2026-02-17T05:52:15.398 INFO:tasks.rados.rados.0.trial115.stdout:1158: snap_create 2026-02-17T05:52:15.398 INFO:tasks.rados.rados.0.trial115.stdout:1157: finishing write tid 1 to trial11545420-10 2026-02-17T05:52:15.399 INFO:tasks.rados.rados.0.trial115.stdout:1154: finishing copy_from to trial11545420-38 2026-02-17T05:52:15.399 INFO:tasks.rados.rados.0.trial115.stdout:1154: got expected ENOENT (src dne) 2026-02-17T05:52:15.399 INFO:tasks.rados.rados.0.trial115.stdout:1154: finishing copy_from racing read to trial11545420-38 2026-02-17T05:52:15.399 INFO:tasks.rados.rados.0.trial115.stdout:1156: expect (ObjNum 425 snap 107 seq_num 425) 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1155: finishing copy_from racing read to trial11545420-45 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: finishing write tid 2 to trial11545420-10 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1155: finishing copy_from to trial11545420-45 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 409 (ObjNum 429 snap 108 seq_num 429) dirty exists 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: finishing write tid 3 to trial11545420-10 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: finishing write tid 4 to trial11545420-10 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: finishing write tid 5 to trial11545420-10 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: oid 10 updating version 0 to 528 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: oid 10 version 528 is already newer than 525 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: oid 10 updating version 528 to 530 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: oid 10 version 530 is already newer than 529 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 530 (ObjNum 453 snap 112 seq_num 453) dirty exists 2026-02-17T05:52:15.523 INFO:tasks.rados.rados.0.trial115.stdout:1157: left oid 10 (ObjNum 453 snap 112 seq_num 453) 2026-02-17T05:52:16.378 INFO:tasks.rados.rados.0.trial115.stdout:1154: done (4 left) 2026-02-17T05:52:16.378 INFO:tasks.rados.rados.0.trial115.stdout:1155: done (3 left) 2026-02-17T05:52:16.378 INFO:tasks.rados.rados.0.trial115.stdout:1156: done (2 left) 2026-02-17T05:52:16.378 INFO:tasks.rados.rados.0.trial115.stdout:1157: done (1 left) 2026-02-17T05:52:16.378 INFO:tasks.rados.rados.0.trial115.stdout:1158: done (0 left) 2026-02-17T05:52:16.378 INFO:tasks.rados.rados.0.trial115.stdout:1159: delete oid 19 current snap is 113 2026-02-17T05:52:16.381 INFO:tasks.rados.rados.0.trial115.stdout:1159: done (0 left) 2026-02-17T05:52:16.381 INFO:tasks.rados.rados.0.trial115.stdout:1160: snap_create 2026-02-17T05:52:17.376 INFO:tasks.rados.rados.0.trial115.stdout:1160: done (0 left) 2026-02-17T05:52:17.376 INFO:tasks.rados.rados.0.trial115.stdout:1161: snap_create 2026-02-17T05:52:19.056 INFO:tasks.rados.rados.0.trial115.stdout:1161: done (0 left) 2026-02-17T05:52:19.056 INFO:tasks.rados.rados.0.trial115.stdout:1162: snap_remove snap 103 2026-02-17T05:52:19.360 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:52:19.360 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T05:52:19.360 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:52:19.360 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:52:19.363 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:52:19.363 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:52:19.364 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:19.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.433+0000 7ff0abcc4640 10 monclient: get_monmap_and_config 2026-02-17T05:52:19.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.433+0000 7ff0abcc4640 10 monclient: build_initial_monmap 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.433+0000 7ff0abcc4640 10 monclient: monmap: 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:19.433594+0000 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:19.433594+0000 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:19.442 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.433+0000 7ff0abcc4640 1 Processor -- start 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.433+0000 7ff0abcc4640 1 -- start start 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.433+0000 7ff0abcc4640 10 monclient: init 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:19.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff0a40a3220 0x7ff0a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient(hunting): picked mon.noname-c con 0x7ff0a40a3220 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40704c0 0x7ff0a40a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient(hunting): picked mon.noname-b con 0x7ff0a40704c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 0x7ff0a413e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient(hunting): picked mon.noname-a con 0x7ff0a413c4f0 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff0a4070130 con 0x7ff0a413c4f0 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:19.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff0a40a4080 con 0x7ff0a40a3220 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a4074000 con 0x7ff0a40704c0 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0abcc4640 10 monclient(hunting): authenticate will time out at 984.987185s 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40704c0 0x7ff0a40a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 0x7ff0a413e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40704c0 0x7ff0a40a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56262/0 (socket says 10.20.193.78:56262) 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 0x7ff0a413e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57826/0 (socket says 10.20.193.78:57826) 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 -- 10.20.193.78:0/810520228 learned_addr learned my addr 10.20.193.78:0/810520228 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:19.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient(hunting): get_auth_request con 0x7ff0a40704c0 auth_method 0 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 10 monclient(hunting): get_auth_request con 0x7ff0a413c4f0 auth_method 0 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9a39640 1 --2- 10.20.193.78:0/810520228 >> v2:10.20.193.115:3300/0 conn(0x7ff0a40a3220 0x7ff0a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:19.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 1 -- 10.20.193.78:0/810520228 >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 msgr2=0x7ff0a413e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 1 -- 10.20.193.78:0/810520228 >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 msgr2=0x7ff0a413e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 1 --2- 10.20.193.78:0/810520228 >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 0x7ff0a413e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9a39640 10 monclient(hunting): get_auth_request con 0x7ff0a40a3220 auth_method 0 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9a39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9a39640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9a39640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0aa23a640 1 --2- 10.20.193.78:0/810520228 >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 0x7ff0a413e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:19.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient(hunting): handle_auth_done global_id 74276 payload 274 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 -- 10.20.193.78:0/810520228 >> v2:10.20.193.115:3300/0 conn(0x7ff0a40a3220 msgr2=0x7ff0a40a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 --2- 10.20.193.78:0/810520228 >> v2:10.20.193.115:3300/0 conn(0x7ff0a40a3220 0x7ff0a40a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 -- 10.20.193.78:0/810520228 >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 msgr2=0x7ff0a413e8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 --2- 10.20.193.78:0/810520228 >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 0x7ff0a413e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient: _finish_hunting 0 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 monclient: found mon.noname-b 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.434+0000 7ff0a9238640 1 -- 10.20.193.78:0/810520228 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a40a4af0 con 0x7ff0a40704c0 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a9238640 1 -- 10.20.193.78:0/810520228 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40704c0 msgr2=0x7ff0a40a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:19.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a9238640 1 --2- 10.20.193.78:0/810520228 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40704c0 0x7ff0a40a3b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff098009970 tx=0x7ff09802ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e54929d2b7908bac server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 1 -- 10.20.193.78:0/810520228 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09803c070 con 0x7ff0a40704c0 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: dump: 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:19.449 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: _finish_auth 0 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: _check_auth_tickets 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 1 -- 10.20.193.78:0/810520228 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09802fcd0 con 0x7ff0a40704c0 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0abcc4640 5 monclient: authenticate success, global_id 74276 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0abcc4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 1 -- 10.20.193.78:0/810520228 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098037660 con 0x7ff0a40704c0 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0abcc4640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:19.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0abcc4640 4 set_mon_vals no callback set 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.436+0000 7ff0a8a37640 10 monclient: dump: 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:19.451 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 10 monclient: shutdown 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 -- 10.20.193.78:0/810520228 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40704c0 msgr2=0x7ff0a40a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/810520228 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40704c0 0x7ff0a40a3b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff098009970 tx=0x7ff09802ee90 comp rx=0 tx=0).stop 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 -- 10.20.193.78:0/810520228 shutdown_connections 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/810520228 >> v2:10.20.193.78:3300/0 conn(0x7ff0a413c4f0 0x7ff0a413e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/810520228 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40704c0 0x7ff0a40a3b40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/810520228 >> v2:10.20.193.115:3300/0 conn(0x7ff0a40a3220 0x7ff0a40a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 -- 10.20.193.78:0/810520228 >> 10.20.193.78:0/810520228 conn(0x7ff0a409c000 msgr2=0x7ff0a40a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 -- 10.20.193.78:0/810520228 shutdown_connections 2026-02-17T05:52:19.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 -- 10.20.193.78:0/810520228 wait complete. 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 10 monclient: build_initial_monmap 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 10 monclient: monmap: 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:19.453 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 Processor -- start 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 1 -- start start 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.437+0000 7ff0abcc4640 10 monclient: init 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff0a40704c0 0x7ff0a41cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient(hunting): picked mon.noname-b con 0x7ff0a40704c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40a3220 0x7ff0a413ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient(hunting): picked mon.noname-c con 0x7ff0a40a3220 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 0x7ff0a4123440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:19.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient(hunting): picked mon.noname-a con 0x7ff0a413c4f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a40a9680 con 0x7ff0a413c4f0 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a40a9500 con 0x7ff0a40704c0 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a40a4f70 con 0x7ff0a40a3220 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0abcc4640 10 monclient(hunting): authenticate will time out at 984.991229s 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40a3220 0x7ff0a413ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40a3220 0x7ff0a413ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56264/0 (socket says 10.20.193.78:56264) 2026-02-17T05:52:19.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 -- 10.20.193.78:0/3103236895 learned_addr learned my addr 10.20.193.78:0/3103236895 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient(hunting): get_auth_request con 0x7ff0a40a3220 auth_method 0 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 0x7ff0a4123440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 10 monclient(hunting): get_auth_request con 0x7ff0a413c4f0 auth_method 0 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:19.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 1 -- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 msgr2=0x7ff0a4123440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 1 -- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 msgr2=0x7ff0a4123440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 0x7ff0a4123440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9a39640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff0a40704c0 0x7ff0a41cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9a39640 10 monclient(hunting): get_auth_request con 0x7ff0a40704c0 auth_method 0 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9a39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9a39640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9a39640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0aa23a640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 0x7ff0a4123440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient(hunting): handle_auth_done global_id 74279 payload 824 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 -- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff0a40704c0 msgr2=0x7ff0a41cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:19.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff0a40704c0 0x7ff0a41cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 -- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 msgr2=0x7ff0a4123440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 0x7ff0a4123440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient: _finish_hunting 0 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 monclient: found mon.noname-c 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.438+0000 7ff0a9238640 1 -- 10.20.193.78:0/3103236895 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a4126e10 con 0x7ff0a40a3220 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0a9238640 1 -- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40a3220 msgr2=0x7ff0a413ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0a9238640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40a3220 0x7ff0a413ba30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff098009aa0 tx=0x7ff098004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 1 -- 10.20.193.78:0/3103236895 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098047070 con 0x7ff0a40a3220 2026-02-17T05:52:19.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: dump: 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:19.459 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: _finish_auth 0 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: _check_auth_tickets 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 1 -- 10.20.193.78:0/3103236895 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff098004230 con 0x7ff0a40a3220 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 1 -- 10.20.193.78:0/3103236895 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09803c040 con 0x7ff0a40a3220 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0927fc640 10 monclient: dump: 2026-02-17T05:52:19.460 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:19.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff093fff640 4 set_mon_vals no callback set 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0abcc4640 5 monclient: authenticate success, global_id 74279 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0abcc4640 10 monclient: _renew_subs 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0abcc4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a40a6900 con 0x7ff0a40a3220 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0abcc4640 10 monclient: _renew_subs 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0abcc4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.440+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a409fa10 con 0x7ff0a40a3220 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0927fc640 1 -- 10.20.193.78:0/3103236895 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff098003710 con 0x7ff0a40a3220 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0abcc4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0abcc4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:19.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0abcc4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:19.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a40a79b0 con 0x7ff0a40a3220 2026-02-17T05:52:19.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0927fc640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff07403df00 0x7ff0740403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:19.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0927fc640 1 -- 10.20.193.78:0/3103236895 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(243..243 src has 117..243) ==== 4751+0+0 (secure 0 0 0) 0x7ff0980776e0 con 0x7ff0a40a3220 2026-02-17T05:52:19.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0a9a39640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff07403df00 0x7ff0740403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:19.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.441+0000 7ff0a9a39640 10 monclient: get_auth_request con 0x7ff07403df00 auth_method 0 2026-02-17T05:52:19.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.444+0000 7ff0927fc640 1 -- 10.20.193.78:0/3103236895 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff0a40a79b0 con 0x7ff0a40a3220 2026-02-17T05:52:19.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.444+0000 7ff0927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:19.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.444+0000 7ff0927fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:19.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.444+0000 7ff0a9a39640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff07403df00 0x7ff0740403c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff0940099b0 tx=0x7ff094008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:19.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.559+0000 7ff0abcc4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:19.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.559+0000 7ff0abcc4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:19.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.559+0000 7ff0abcc4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:19.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.559+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff0a41259c0 con 0x7ff0a40a3220 2026-02-17T05:52:19.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.559+0000 7ff0927fc640 1 -- 10.20.193.78:0/3103236895 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7ff098076100 con 0x7ff0a40a3220 2026-02-17T05:52:19.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.559+0000 7ff0927fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:19.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.559+0000 7ff0927fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:19.559 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":64,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:19.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff07403df00 msgr2=0x7ff0740403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:19.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff07403df00 0x7ff0740403c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff0940099b0 tx=0x7ff094008040 comp rx=0 tx=0).stop 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 10 monclient: shutdown 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40a3220 msgr2=0x7ff0a413ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40a3220 0x7ff0a413ba30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff098009aa0 tx=0x7ff098004830 comp rx=0 tx=0).stop 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 shutdown_connections 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff0a413c4f0 0x7ff0a4123440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff0a40a3220 0x7ff0a413ba30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff07403df00 0x7ff0740403c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 --2- 10.20.193.78:0/3103236895 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff0a40704c0 0x7ff0a41cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 >> 10.20.193.78:0/3103236895 conn(0x7ff0a409c000 msgr2=0x7ff0a4071110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 shutdown_connections 2026-02-17T05:52:19.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:19.560+0000 7ff0abcc4640 1 -- 10.20.193.78:0/3103236895 wait complete. 2026-02-17T05:52:20.057 INFO:tasks.rados.rados.0.trial115.stdout:1162: done (0 left) 2026-02-17T05:52:20.057 INFO:tasks.rados.rados.0.trial115.stdout:1163: rollback oid 22 current snap is 115 2026-02-17T05:52:20.057 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 22 to 114 2026-02-17T05:52:20.057 INFO:tasks.rados.rados.0.trial115.stdout:1164: rollback oid 13 current snap is 115 2026-02-17T05:52:20.058 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 13 to 104 2026-02-17T05:52:20.058 INFO:tasks.rados.rados.0.trial115.stdout:1165: snap_create 2026-02-17T05:52:20.059 INFO:tasks.rados.rados.0.trial115.stdout:1163: finishing rollback tid 1 to trial11545420-22 2026-02-17T05:52:20.059 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 337 (ObjNum 129 snap 0 seq_num 1333758784) dirty dne 2026-02-17T05:52:20.061 INFO:tasks.rados.rados.0.trial115.stdout:1164: finishing rollback tid 0 to trial11545420-13 2026-02-17T05:52:20.061 INFO:tasks.rados.rados.0.trial115.stdout:1164: finishing rollback tid 1 to trial11545420-13 2026-02-17T05:52:20.062 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 407 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T05:52:21.057 INFO:tasks.rados.rados.0.trial115.stdout:1163: done (2 left) 2026-02-17T05:52:21.058 INFO:tasks.rados.rados.0.trial115.stdout:1164: done (1 left) 2026-02-17T05:52:21.058 INFO:tasks.rados.rados.0.trial115.stdout:1165: done (0 left) 2026-02-17T05:52:21.058 INFO:tasks.rados.rados.0.trial115.stdout:1166: copy_from oid 41 from oid 50 current snap is 116 2026-02-17T05:52:21.058 INFO:tasks.rados.rados.0.trial115.stdout:1167: read oid 18 snap -1 2026-02-17T05:52:21.058 INFO:tasks.rados.rados.0.trial115.stdout:1167: expect (ObjNum 445 snap 110 seq_num 445) 2026-02-17T05:52:21.058 INFO:tasks.rados.rados.0.trial115.stdout:1168: write oid 22 current snap is 116 2026-02-17T05:52:21.058 INFO:tasks.rados.rados.0.trial115.stdout:1168: seq_num 455 ranges {447185=703122,1642601=765814,2928991=447707} 2026-02-17T05:52:21.062 INFO:tasks.rados.rados.0.trial115.stdout:1168: writing trial11545420-22 from 447185 to 1150307 tid 1 2026-02-17T05:52:21.067 INFO:tasks.rados.rados.0.trial115.stdout:1168: writing trial11545420-22 from 1642601 to 2408415 tid 2 2026-02-17T05:52:21.071 INFO:tasks.rados.rados.0.trial115.stdout:1168: writing trial11545420-22 from 2928991 to 3376698 tid 3 2026-02-17T05:52:21.071 INFO:tasks.rados.rados.0.trial115.stdout:1169: read oid 43 snap -1 2026-02-17T05:52:21.071 INFO:tasks.rados.rados.0.trial115.stdout:1169: expect deleted 2026-02-17T05:52:21.071 INFO:tasks.rados.rados.0.trial115.stdout:1170: snap_remove snap 104 2026-02-17T05:52:21.376 INFO:tasks.rados.rados.0.trial115.stdout:1170: done (4 left) 2026-02-17T05:52:21.376 INFO:tasks.rados.rados.0.trial115.stdout:1171: rollback oid 7 current snap is 116 2026-02-17T05:52:21.376 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 78 2026-02-17T05:52:21.376 INFO:tasks.rados.rados.0.trial115.stdout:1166: finishing copy_from to trial11545420-41 2026-02-17T05:52:21.376 INFO:tasks.rados.rados.0.trial115.stdout:1166: got expected ENOENT (src dne) 2026-02-17T05:52:21.377 INFO:tasks.rados.rados.0.trial115.stdout:1166: finishing copy_from racing read to trial11545420-41 2026-02-17T05:52:21.377 INFO:tasks.rados.rados.0.trial115.stdout:1167: expect (ObjNum 445 snap 110 seq_num 445) 2026-02-17T05:52:21.484 INFO:tasks.rados.rados.0.trial115.stdout:1168: finishing write tid 1 to trial11545420-22 2026-02-17T05:52:21.485 INFO:tasks.rados.rados.0.trial115.stdout:1168: finishing write tid 2 to trial11545420-22 2026-02-17T05:52:21.485 INFO:tasks.rados.rados.0.trial115.stdout:1168: finishing write tid 3 to trial11545420-22 2026-02-17T05:52:21.485 INFO:tasks.rados.rados.0.trial115.stdout:1166: done (4 left) 2026-02-17T05:52:21.485 INFO:tasks.rados.rados.0.trial115.stdout:1167: done (3 left) 2026-02-17T05:52:21.485 INFO:tasks.rados.rados.0.trial115.stdout:1172: write oid 1 current snap is 116 2026-02-17T05:52:21.485 INFO:tasks.rados.rados.0.trial115.stdout:1172: seq_num 456 ranges {463992=778371,1701083=625825,2956414=658979} 2026-02-17T05:52:21.489 INFO:tasks.rados.rados.0.trial115.stdout:1172: writing trial11545420-1 from 463992 to 1242363 tid 1 2026-02-17T05:52:21.493 INFO:tasks.rados.rados.0.trial115.stdout:1172: writing trial11545420-1 from 1701083 to 2326908 tid 2 2026-02-17T05:52:21.498 INFO:tasks.rados.rados.0.trial115.stdout:1172: writing trial11545420-1 from 2956414 to 3615393 tid 3 2026-02-17T05:52:21.498 INFO:tasks.rados.rados.0.trial115.stdout:1173: write (excl) oid 32 current snap is 116 2026-02-17T05:52:21.498 INFO:tasks.rados.rados.0.trial115.stdout:1173: seq_num 457 ranges {480799=569973,1475918=769483,2700190=586604} 2026-02-17T05:52:21.503 INFO:tasks.rados.rados.0.trial115.stdout:1173: writing trial11545420-32 from 480799 to 1050772 tid 1 2026-02-17T05:52:21.508 INFO:tasks.rados.rados.0.trial115.stdout:1173: writing trial11545420-32 from 1475918 to 2245401 tid 2 2026-02-17T05:52:21.512 INFO:tasks.rados.rados.0.trial115.stdout:1173: writing trial11545420-32 from 2700190 to 3286794 tid 3 2026-02-17T05:52:21.512 INFO:tasks.rados.rados.0.trial115.stdout:1174: snap_remove snap 95 2026-02-17T05:52:22.380 INFO:tasks.rados.rados.0.trial115.stdout:1174: done (5 left) 2026-02-17T05:52:22.380 INFO:tasks.rados.rados.0.trial115.stdout:1175: snap_remove snap 105 2026-02-17T05:52:22.569 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient: get_monmap_and_config 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient: build_initial_monmap 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient: monmap: 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:22.637401+0000 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:22.637401+0000 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:22.635 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:22.636 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:22.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 1 Processor -- start 2026-02-17T05:52:22.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 1 -- start start 2026-02-17T05:52:22.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient: init 2026-02-17T05:52:22.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:22.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2026-02-17T05:52:22.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06c890 0x7f33ec06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:22.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient(hunting): picked mon.noname-b con 0x7f33ec06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f33ec09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 0x7f33ec135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient(hunting): picked mon.noname-a con 0x7f33ec06d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33ec06daf0 con 0x7f33ec09c4a0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f33ec0a0880 con 0x7f33ec06d1b0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f33ec09cc80 con 0x7f33ec06c890 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.637+0000 7f33f2d1b640 10 monclient(hunting): authenticate will time out at 988.190944s 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06c890 0x7f33ec06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06c890 0x7f33ec06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56318/0 (socket says 10.20.193.78:56318) 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 1 -- 10.20.193.78:0/3435747323 learned_addr learned my addr 10.20.193.78:0/3435747323 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 0x7f33ec135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient(hunting): get_auth_request con 0x7f33ec06c890 auth_method 0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 10 monclient(hunting): get_auth_request con 0x7f33ec06d1b0 auth_method 0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 1 -- 10.20.193.78:0/3435747323 >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 msgr2=0x7f33ec135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 1 -- 10.20.193.78:0/3435747323 >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 msgr2=0x7f33ec135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 1 --2- 10.20.193.78:0/3435747323 >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 0x7f33ec135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f1291640 1 --2- 10.20.193.78:0/3435747323 >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 0x7f33ec135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient(hunting): handle_auth_done global_id 74282 payload 274 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 1 -- 10.20.193.78:0/3435747323 >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 msgr2=0x7f33ec135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 1 --2- 10.20.193.78:0/3435747323 >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 0x7f33ec135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 1 -- 10.20.193.78:0/3435747323 >> v1:10.20.193.115:6789/0 conn(0x7f33ec09c4a0 legacy=0x7f33ec134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient: _finish_hunting 0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 1 monclient: found mon.noname-b 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:52:22.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.638+0000 7f33f0a90640 1 -- 10.20.193.78:0/3435747323 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec09cf60 con 0x7f33ec06c890 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33f0a90640 1 -- 10.20.193.78:0/3435747323 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06c890 msgr2=0x7f33ec06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33f0a90640 1 --2- 10.20.193.78:0/3435747323 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06c890 0x7f33ec06cc70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009970 tx=0x7f33dc02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a6193f98eedcf55 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 1 -- 10.20.193.78:0/3435747323 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc03c070 con 0x7f33ec06c890 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: dump: 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: _finish_auth 0 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 1 -- 10.20.193.78:0/3435747323 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc02fcd0 con 0x7f33ec06c890 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 1 -- 10.20.193.78:0/3435747323 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc037660 con 0x7f33ec06c890 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33f2d1b640 5 monclient: authenticate success, global_id 74282 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33f2d1b640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33f2d1b640 4 set_mon_vals no callback set 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.640+0000 7f33e37fe640 10 monclient: dump: 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:22.639 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:22.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 10 monclient: shutdown 2026-02-17T05:52:22.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3435747323 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06c890 msgr2=0x7f33ec06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3435747323 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06c890 0x7f33ec06cc70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009970 tx=0x7f33dc02ee90 comp rx=0 tx=0).stop 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3435747323 shutdown_connections 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3435747323 >> v2:10.20.193.78:3300/0 conn(0x7f33ec06d1b0 0x7f33ec135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3435747323 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06c890 0x7f33ec06cc70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3435747323 >> 10.20.193.78:0/3435747323 conn(0x7f33ec0a6620 msgr2=0x7f33ec0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3435747323 shutdown_connections 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3435747323 wait complete. 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 10 monclient: build_initial_monmap 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 10 monclient: monmap: 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 Processor -- start 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 1 -- start start 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.641+0000 7f33f2d1b640 10 monclient: init 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f33ec06c890 0x7f33ec09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient(hunting): picked mon.noname-b con 0x7f33ec06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06d1b0 0x7f33ec09e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient(hunting): picked mon.noname-c con 0x7f33ec06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 0x7f33ec09e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient(hunting): picked mon.noname-a con 0x7f33ec09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec0a01a0 con 0x7f33ec09c4a0 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec0a0020 con 0x7f33ec06c890 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f33ec09be20 con 0x7f33ec06d1b0 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f2d1b640 10 monclient(hunting): authenticate will time out at 988.195112s 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06d1b0 0x7f33ec09e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 0x7f33ec09e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:22.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06d1b0 0x7f33ec09e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56330/0 (socket says 10.20.193.78:56330) 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 -- 10.20.193.78:0/3622591550 learned_addr learned my addr 10.20.193.78:0/3622591550 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 10 monclient(hunting): get_auth_request con 0x7f33ec09c4a0 auth_method 0 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient(hunting): get_auth_request con 0x7f33ec06d1b0 auth_method 0 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 1 -- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 msgr2=0x7f33ec09e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 1 -- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 msgr2=0x7f33ec09e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 0x7f33ec09e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f0a90640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f33ec06c890 0x7f33ec09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f1291640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 0x7f33ec09e5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f0a90640 10 monclient(hunting): get_auth_request con 0x7f33ec06c890 auth_method 0 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f0a90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f0a90640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33f0a90640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient(hunting): handle_auth_done global_id 74285 payload 824 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 -- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f33ec06c890 msgr2=0x7f33ec09db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f33ec06c890 0x7f33ec09db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 -- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 msgr2=0x7f33ec09e5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 0x7f33ec09e5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 monclient: found mon.noname-c 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:22.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.642+0000 7f33e3fff640 1 -- 10.20.193.78:0/3622591550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec073020 con 0x7f33ec06d1b0 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e3fff640 1 -- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06d1b0 msgr2=0x7f33ec09e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e3fff640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06d1b0 0x7f33ec09e060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f33d400c5d0 tx=0x7f33d400caa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 1 -- 10.20.193.78:0/3622591550 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4016020 con 0x7f33ec06d1b0 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: dump: 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: _finish_auth 0 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: _check_auth_tickets 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 1 -- 10.20.193.78:0/3622591550 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d4012a60 con 0x7f33ec06d1b0 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 1 -- 10.20.193.78:0/3622591550 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4012e50 con 0x7f33ec06d1b0 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e2ffd640 4 set_mon_vals no callback set 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33e17fa640 10 monclient: dump: 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:22.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33f2d1b640 5 monclient: authenticate success, global_id 74285 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33f2d1b640 10 monclient: _renew_subs 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33f2d1b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ec09d810 con 0x7f33ec06d1b0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33f2d1b640 10 monclient: _renew_subs 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33f2d1b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.644+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ec128280 con 0x7f33ec06d1b0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33e17fa640 1 -- 10.20.193.78:0/3622591550 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f33d401b400 con 0x7f33ec06d1b0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33e17fa640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33bc03e050 0x7f33bc040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33f2d1b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33f2d1b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33f2d1b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33e17fa640 1 -- 10.20.193.78:0/3622591550 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(247..247 src has 117..247) ==== 4779+0+0 (secure 0 0 0) 0x7f33d4052940 con 0x7f33ec06d1b0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ec074980 con 0x7f33ec06d1b0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33f0a90640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33bc03e050 0x7f33bc040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:22.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.645+0000 7f33f0a90640 10 monclient: get_auth_request con 0x7f33bc03e050 auth_method 0 2026-02-17T05:52:22.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.647+0000 7f33e17fa640 1 -- 10.20.193.78:0/3622591550 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f33d4020030 con 0x7f33ec06d1b0 2026-02-17T05:52:22.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.647+0000 7f33e17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:22.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.647+0000 7f33e17fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:22.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.648+0000 7f33f0a90640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33bc03e050 0x7f33bc040510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33dc036000 tx=0x7f33dc039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:22.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.759+0000 7f33f2d1b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:22.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.759+0000 7f33f2d1b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:22.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.759+0000 7f33f2d1b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:22.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.759+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f33ec136740 con 0x7f33ec06d1b0 2026-02-17T05:52:22.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33e17fa640 1 -- 10.20.193.78:0/3622591550 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f33d401b900 con 0x7f33ec06d1b0 2026-02-17T05:52:22.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33e17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:22.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33e17fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:22.758 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":64,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33bc03e050 msgr2=0x7f33bc040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33bc03e050 0x7f33bc040510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33dc036000 tx=0x7f33dc039040 comp rx=0 tx=0).stop 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33f2d1b640 10 monclient: shutdown 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33f2d1b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06d1b0 msgr2=0x7f33ec09e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.760+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06d1b0 0x7f33ec09e060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f33d400c5d0 tx=0x7f33d400caa0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.761+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 shutdown_connections 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.761+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f33ec09c4a0 0x7f33ec09e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.761+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f33ec06d1b0 0x7f33ec09e060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.761+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33bc03e050 0x7f33bc040510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.761+0000 7f33f2d1b640 1 --2- 10.20.193.78:0/3622591550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f33ec06c890 0x7f33ec09db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.761+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 >> 10.20.193.78:0/3622591550 conn(0x7f33ec0a6620 msgr2=0x7f33ec071600 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.761+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 shutdown_connections 2026-02-17T05:52:22.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:22.761+0000 7f33f2d1b640 1 -- 10.20.193.78:0/3622591550 wait complete. 2026-02-17T05:52:23.381 INFO:tasks.rados.rados.0.trial115.stdout:1175: done (5 left) 2026-02-17T05:52:23.381 INFO:tasks.rados.rados.0.trial115.stdout:1176: write (excl) oid 8 current snap is 116 2026-02-17T05:52:23.381 INFO:tasks.rados.rados.0.trial115.stdout:1176: seq_num 458 ranges {497606=645222,1934400=629494,3243966=453640} 2026-02-17T05:52:23.386 INFO:tasks.rados.rados.0.trial115.stdout:1176: writing trial11545420-8 from 497606 to 1142828 tid 1 2026-02-17T05:52:23.392 INFO:tasks.rados.rados.0.trial115.stdout:1176: writing trial11545420-8 from 1934400 to 2563894 tid 2 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:1176: writing trial11545420-8 from 3243966 to 3697606 tid 3 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:1177: rollback oid 44 current snap is 116 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 44 to 114 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:1168: finishing write tid 4 to trial11545420-22 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:1168: finishing write tid 5 to trial11545420-22 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:1168: oid 22 updating version 0 to 484 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:1168: oid 22 updating version 484 to 487 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:1168: oid 22 version 487 is already newer than 485 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:1168: oid 22 version 487 is already newer than 486 2026-02-17T05:52:23.398 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 487 (ObjNum 455 snap 116 seq_num 455) dirty exists 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1168: left oid 22 (ObjNum 455 snap 116 seq_num 455) 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1171: finishing rollback tid 1 to trial11545420-7 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1171: finishing rollback tid 2 to trial11545420-7 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 425 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: finishing write tid 1 to trial11545420-1 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1168: done (6 left) 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1169: done (5 left) 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1171: done (4 left) 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1178: copy_from oid 31 from oid 21 current snap is 116 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: finishing write tid 2 to trial11545420-1 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: finishing write tid 3 to trial11545420-1 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: finishing write tid 4 to trial11545420-1 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: finishing write tid 5 to trial11545420-1 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: oid 1 updating version 0 to 420 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: oid 1 version 420 is already newer than 419 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: oid 1 version 420 is already newer than 418 2026-02-17T05:52:23.399 INFO:tasks.rados.rados.0.trial115.stdout:1172: oid 1 version 420 is already newer than 415 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 420 (ObjNum 456 snap 116 seq_num 456) dirty exists 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1172: left oid 1 (ObjNum 456 snap 116 seq_num 456) 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: finishing write tid 1 to trial11545420-32 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1172: done (4 left) 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1179: copy_from oid 42 from oid 23 current snap is 116 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: finishing write tid 2 to trial11545420-32 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: finishing write tid 3 to trial11545420-32 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: finishing write tid 4 to trial11545420-32 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: finishing write tid 5 to trial11545420-32 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: oid 32 updating version 0 to 315 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: oid 32 updating version 315 to 317 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: oid 32 version 317 is already newer than 314 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: oid 32 version 317 is already newer than 316 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 317 (ObjNum 457 snap 116 seq_num 457) dirty exists 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1173: left oid 32 (ObjNum 457 snap 116 seq_num 457) 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1176: finishing write tid 1 to trial11545420-8 2026-02-17T05:52:23.400 INFO:tasks.rados.rados.0.trial115.stdout:1176: finishing write tid 2 to trial11545420-8 2026-02-17T05:52:23.401 INFO:tasks.rados.rados.0.trial115.stdout:1173: done (4 left) 2026-02-17T05:52:23.401 INFO:tasks.rados.rados.0.trial115.stdout:1180: read oid 25 snap -1 2026-02-17T05:52:23.401 INFO:tasks.rados.rados.0.trial115.stdout:1180: expect (ObjNum 429 snap 108 seq_num 429) 2026-02-17T05:52:23.401 INFO:tasks.rados.rados.0.trial115.stdout:1181: write oid 35 current snap is 116 2026-02-17T05:52:23.401 INFO:tasks.rados.rados.0.trial115.stdout:1181: seq_num 459 ranges {514413=720471,1876529=773152,3438683=275730} 2026-02-17T05:52:23.401 INFO:tasks.rados.rados.0.trial115.stdout:1181: writing trial11545420-35 from 514413 to 1234884 tid 1 2026-02-17T05:52:23.407 INFO:tasks.rados.rados.0.trial115.stdout:1181: writing trial11545420-35 from 1876529 to 2649681 tid 2 2026-02-17T05:52:23.411 INFO:tasks.rados.rados.0.trial115.stdout:1181: writing trial11545420-35 from 3438683 to 3714413 tid 3 2026-02-17T05:52:23.411 INFO:tasks.rados.rados.0.trial115.stdout:1182: write oid 14 current snap is 116 2026-02-17T05:52:23.412 INFO:tasks.rados.rados.0.trial115.stdout:1182: seq_num 460 ranges {531220=795720,1935011=516810,3066106=420420} 2026-02-17T05:52:23.416 INFO:tasks.rados.rados.0.trial115.stdout:1182: writing trial11545420-14 from 531220 to 1326940 tid 1 2026-02-17T05:52:23.421 INFO:tasks.rados.rados.0.trial115.stdout:1182: writing trial11545420-14 from 1935011 to 2451821 tid 2 2026-02-17T05:52:23.425 INFO:tasks.rados.rados.0.trial115.stdout:1182: writing trial11545420-14 from 3066106 to 3486526 tid 3 2026-02-17T05:52:23.425 INFO:tasks.rados.rados.0.trial115.stdout:1183: write (excl) oid 18 current snap is 116 2026-02-17T05:52:23.425 INFO:tasks.rados.rados.0.trial115.stdout:1179: finishing copy_from to trial11545420-42 2026-02-17T05:52:23.425 INFO:tasks.rados.rados.0.trial115.stdout:1179: got expected ENOENT (src dne) 2026-02-17T05:52:23.425 INFO:tasks.rados.rados.0.trial115.stdout:1183: seq_num 461 ranges {548027=470969,1593493=776821,3093529=631692} 2026-02-17T05:52:23.428 INFO:tasks.rados.rados.0.trial115.stdout:1183: writing trial11545420-18 from 548027 to 1018996 tid 1 2026-02-17T05:52:23.433 INFO:tasks.rados.rados.0.trial115.stdout:1183: writing trial11545420-18 from 1593493 to 2370314 tid 2 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1183: writing trial11545420-18 from 3093529 to 3725221 tid 3 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1184: delete oid 49 current snap is 116 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1179: finishing copy_from racing read to trial11545420-42 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1176: finishing write tid 3 to trial11545420-8 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1176: finishing write tid 4 to trial11545420-8 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1176: finishing write tid 5 to trial11545420-8 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1176: oid 8 updating version 0 to 194 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1176: oid 8 version 194 is already newer than 191 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1176: oid 8 version 194 is already newer than 193 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1176: oid 8 version 194 is already newer than 192 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 194 (ObjNum 458 snap 116 seq_num 458) dirty exists 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1176: left oid 8 (ObjNum 458 snap 116 seq_num 458) 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1177: finishing rollback tid 0 to trial11545420-44 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1177: finishing rollback tid 1 to trial11545420-44 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1177: finishing rollback tid 2 to trial11545420-44 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 320 (ObjNum 377 snap 99 seq_num 377) dirty exists 2026-02-17T05:52:23.439 INFO:tasks.rados.rados.0.trial115.stdout:1180: expect (ObjNum 429 snap 108 seq_num 429) 2026-02-17T05:52:23.513 INFO:tasks.rados.rados.0.trial115.stdout:1181: finishing write tid 1 to trial11545420-35 2026-02-17T05:52:23.513 INFO:tasks.rados.rados.0.trial115.stdout:1178: finishing copy_from racing read to trial11545420-31 2026-02-17T05:52:23.513 INFO:tasks.rados.rados.0.trial115.stdout:1178: finishing copy_from to trial11545420-31 2026-02-17T05:52:23.514 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 440 (ObjNum 447 snap 110 seq_num 447) dirty exists 2026-02-17T05:52:23.514 INFO:tasks.rados.rados.0.trial115.stdout:1181: finishing write tid 2 to trial11545420-35 2026-02-17T05:52:23.514 INFO:tasks.rados.rados.0.trial115.stdout:1181: finishing write tid 3 to trial11545420-35 2026-02-17T05:52:23.514 INFO:tasks.rados.rados.0.trial115.stdout:1176: done (8 left) 2026-02-17T05:52:23.514 INFO:tasks.rados.rados.0.trial115.stdout:1177: done (7 left) 2026-02-17T05:52:23.514 INFO:tasks.rados.rados.0.trial115.stdout:1178: done (6 left) 2026-02-17T05:52:23.514 INFO:tasks.rados.rados.0.trial115.stdout:1179: done (5 left) 2026-02-17T05:52:23.514 INFO:tasks.rados.rados.0.trial115.stdout:1180: done (4 left) 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1184: done (3 left) 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1185: read oid 20 snap 110 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1185: expect deleted 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1181: finishing write tid 4 to trial11545420-35 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1181: finishing write tid 5 to trial11545420-35 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1181: oid 35 updating version 0 to 584 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1181: oid 35 version 584 is already newer than 581 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1181: oid 35 version 584 is already newer than 583 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:1181: oid 35 version 584 is already newer than 582 2026-02-17T05:52:23.515 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 584 (ObjNum 459 snap 116 seq_num 459) dirty exists 2026-02-17T05:52:23.516 INFO:tasks.rados.rados.0.trial115.stdout:1181: left oid 35 (ObjNum 459 snap 116 seq_num 459) 2026-02-17T05:52:23.516 INFO:tasks.rados.rados.0.trial115.stdout:1182: finishing write tid 1 to trial11545420-14 2026-02-17T05:52:23.516 INFO:tasks.rados.rados.0.trial115.stdout:1182: finishing write tid 2 to trial11545420-14 2026-02-17T05:52:23.516 INFO:tasks.rados.rados.0.trial115.stdout:1182: finishing write tid 3 to trial11545420-14 2026-02-17T05:52:23.516 INFO:tasks.rados.rados.0.trial115.stdout:1182: finishing write tid 4 to trial11545420-14 2026-02-17T05:52:23.516 INFO:tasks.rados.rados.0.trial115.stdout:1182: finishing write tid 5 to trial11545420-14 2026-02-17T05:52:23.516 INFO:tasks.rados.rados.0.trial115.stdout:1182: oid 14 updating version 0 to 428 2026-02-17T05:52:23.516 INFO:tasks.rados.rados.0.trial115.stdout:1182: oid 14 version 428 is already newer than 426 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1182: oid 14 version 428 is already newer than 427 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1182: oid 14 updating version 428 to 429 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 429 (ObjNum 460 snap 116 seq_num 460) dirty exists 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1182: left oid 14 (ObjNum 460 snap 116 seq_num 460) 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1183: finishing write tid 1 to trial11545420-18 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1183: finishing write tid 2 to trial11545420-18 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1183: finishing write tid 3 to trial11545420-18 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1183: finishing write tid 4 to trial11545420-18 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1183: finishing write tid 5 to trial11545420-18 2026-02-17T05:52:23.517 INFO:tasks.rados.rados.0.trial115.stdout:1183: oid 18 updating version 0 to 473 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1183: oid 18 version 473 is already newer than 471 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1183: oid 18 version 473 is already newer than 470 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1183: oid 18 version 473 is already newer than 472 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 473 (ObjNum 461 snap 116 seq_num 461) dirty exists 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1183: left oid 18 (ObjNum 461 snap 116 seq_num 461) 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1186: read oid 3 snap 107 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1186: expect (ObjNum 417 snap 107 seq_num 417) 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1181: done (4 left) 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1182: done (3 left) 2026-02-17T05:52:23.518 INFO:tasks.rados.rados.0.trial115.stdout:1183: done (2 left) 2026-02-17T05:52:23.519 INFO:tasks.rados.rados.0.trial115.stdout:1187: copy_from oid 46 from oid 7 current snap is 116 2026-02-17T05:52:23.519 INFO:tasks.rados.rados.0.trial115.stdout:1188: read oid 6 snap -1 2026-02-17T05:52:23.519 INFO:tasks.rados.rados.0.trial115.stdout:1188: expect deleted 2026-02-17T05:52:23.519 INFO:tasks.rados.rados.0.trial115.stdout:1189: read oid 15 snap -1 2026-02-17T05:52:23.519 INFO:tasks.rados.rados.0.trial115.stdout:1189: expect (ObjNum 450 snap 112 seq_num 450) 2026-02-17T05:52:23.519 INFO:tasks.rados.rados.0.trial115.stdout:1190: rollback oid 39 current snap is 116 2026-02-17T05:52:23.519 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 39 to 110 2026-02-17T05:52:23.519 INFO:tasks.rados.rados.0.trial115.stdout:1191: snap_remove snap 82 2026-02-17T05:52:24.386 INFO:tasks.rados.rados.0.trial115.stdout:1191: done (6 left) 2026-02-17T05:52:24.387 INFO:tasks.rados.rados.0.trial115.stdout:1192: snap_remove snap 78 2026-02-17T05:52:25.388 INFO:tasks.rados.rados.0.trial115.stdout:1192: done (6 left) 2026-02-17T05:52:25.388 INFO:tasks.rados.rados.0.trial115.stdout:1193: read oid 8 snap 107 2026-02-17T05:52:25.388 INFO:tasks.rados.rados.0.trial115.stdout:1193: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:52:25.388 INFO:tasks.rados.rados.0.trial115.stdout:1194: snap_create 2026-02-17T05:52:25.389 INFO:tasks.rados.rados.0.trial115.stdout:1190: finishing rollback tid 0 to trial11545420-39 2026-02-17T05:52:25.389 INFO:tasks.rados.rados.0.trial115.stdout:1190: finishing rollback tid 1 to trial11545420-39 2026-02-17T05:52:25.389 INFO:tasks.rados.rados.0.trial115.stdout:1190: finishing rollback tid 2 to trial11545420-39 2026-02-17T05:52:25.389 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 658 (ObjNum 428 snap 108 seq_num 428) dirty exists 2026-02-17T05:52:25.389 INFO:tasks.rados.rados.0.trial115.stdout:1186: expect (ObjNum 417 snap 107 seq_num 417) 2026-02-17T05:52:25.481 INFO:tasks.rados.rados.0.trial115.stdout:1189: expect (ObjNum 450 snap 112 seq_num 450) 2026-02-17T05:52:25.596 INFO:tasks.rados.rados.0.trial115.stdout:1187: finishing copy_from racing read to trial11545420-46 2026-02-17T05:52:25.596 INFO:tasks.rados.rados.0.trial115.stdout:1187: finishing copy_from to trial11545420-46 2026-02-17T05:52:25.596 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 425 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:52:25.596 INFO:tasks.rados.rados.0.trial115.stdout:1193: expect (ObjNum 254 snap 50 seq_num 254) 2026-02-17T05:52:25.766 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:25.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.837+0000 7f6b383f7640 10 monclient: get_monmap_and_config 2026-02-17T05:52:25.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.837+0000 7f6b383f7640 10 monclient: build_initial_monmap 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.837+0000 7f6b383f7640 10 monclient: monmap: 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:25.837743+0000 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:25.837743+0000 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:25.839 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.837+0000 7f6b383f7640 1 Processor -- start 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.837+0000 7f6b383f7640 1 -- start start 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.837+0000 7f6b383f7640 10 monclient: init 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient(hunting): picked mon.noname-c con 0x7f6b3006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:25.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient(hunting): picked mon.noname-a con 0x7f6b3009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6b30134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3006daf0 con 0x7f6b30134df0 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6b300a0880 con 0x7f6b3009c4a0 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3006d6f0 con 0x7f6b3006c890 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b383f7640 10 monclient(hunting): authenticate will time out at 991.391250s 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57888/0 (socket says 10.20.193.78:57888) 2026-02-17T05:52:25.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 learned_addr learned my addr 10.20.193.78:0/269960890 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b3009c4a0 auth_method 0 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b3006c890 auth_method 0 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:25.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:25.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:25.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:25.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:25.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:25.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:25.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:25.840+0000 7f6b3696d640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.115:6789/0 conn(0x7f6b30134df0 legacy=0x7f6b30135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:26.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b3009c4a0 auth_method 0 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.038+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b3006c890 auth_method 0 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:26.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:26.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:26.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:26.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:26.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:26.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b3009c4a0 auth_method 0 2026-02-17T05:52:26.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:26.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:26.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:26.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:26.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:26.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:26.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:26.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:26.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b3006c890 auth_method 0 2026-02-17T05:52:26.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:26.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:26.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:26.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:26.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:26.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:26.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:26.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:27.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:27.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b3009c4a0 auth_method 0 2026-02-17T05:52:27.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:27.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:27.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:27.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:27.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:27.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:27.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:27.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:27.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.240+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b3006c890 auth_method 0 2026-02-17T05:52:27.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.241+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:27.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.241+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:27.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.241+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:27.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.241+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:27.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.241+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:27.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.241+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:27.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:27.241+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:28.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 10 monclient(hunting): tick 2026-02-17T05:52:28.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:52:28.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 monclient(hunting): continuing hunt 2026-02-17T05:52:28.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 msgr2=0x7f6b3006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b3006c890 0x7f6b3006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 msgr2=0x7f6b3006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b3009c4a0 0x7f6b3006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.115:6789/0 conn(0x7f6b30134df0 legacy=0x7f6b30135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 10 monclient: _add_conns ranks=[2,0] 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 10 monclient(hunting): picked mon.noname-b con 0x7f6b100037f0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6b10006350 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3009cc80 con 0x7f6b10006350 2026-02-17T05:52:28.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3006d6f0 con 0x7f6b100037f0 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b17fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b100037f0 auth_method 0 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:28.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:28.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:28.840+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.78:6789/0 conn(0x7f6b10006350 legacy=0x7f6b10008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:29.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.038+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:29.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.038+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b100037f0 auth_method 0 2026-02-17T05:52:29.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.038+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:29.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.038+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:29.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.038+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:29.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.039+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:29.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.039+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:29.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:29.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:29.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.428+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.78:6789/0 conn(0x7f6b10006350 legacy=0x7f6b10008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-17T05:52:29.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.428+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.78:6789/0 conn(0x7f6b10006350 legacy=0x7f6b10008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:52:29.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.428+0000 7f6b3596b640 1 --1- 10.20.193.78:0/269960890 >> v1:10.20.193.78:6789/0 conn(0x7f6b10006350 0x7f6b10008810 :-1 s=OPENED pgs=24 cs=1 l=1).handle_message read tag failed 2026-02-17T05:52:29.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.428+0000 7f6b3596b640 1 --1- 10.20.193.78:0/269960890 >> v1:10.20.193.78:6789/0 conn(0x7f6b10006350 0x7f6b10008810 :-1 s=OPENED pgs=24 cs=1 l=1).fault on lossy channel, failing 2026-02-17T05:52:29.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.428+0000 7f6b3516a640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.78:6789/0 2026-02-17T05:52:29.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:29.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b100037f0 auth_method 0 2026-02-17T05:52:29.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:29.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:29.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:29.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:29.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:29.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:29.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:29.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:30.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:30.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b100037f0 auth_method 0 2026-02-17T05:52:30.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:30.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:30.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:30.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:30.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:30.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:30.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:30.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:31.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient(hunting): tick 2026-02-17T05:52:31.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:52:31.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 1 monclient(hunting): continuing hunt 2026-02-17T05:52:31.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:52:31.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 msgr2=0x7f6b10005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:31.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b100037f0 0x7f6b10005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:31.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.78:6789/0 conn(0x7f6b10006350 legacy=0x7f6b10008810 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3696d640 1 -- 10.20.193.78:0/269960890 reap_dead start 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient(hunting): picked mon.noname-b con 0x7f6b1000a100 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6b1000a620 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6b1000ab70 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1000b1c0 con 0x7f6b1000ab70 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1000fdc0 con 0x7f6b1000a620 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3006d6f0 con 0x7f6b1000a100 2026-02-17T05:52:31.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b17fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b1000a100 auth_method 0 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:31.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:31.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.840+0000 7f6b3696d640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.78:6789/0 conn(0x7f6b1000ab70 legacy=0x7f6b1000d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:31.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:31.841+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.115:6789/0 conn(0x7f6b1000a620 legacy=0x7f6b10008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:32.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.038+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:32.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.039+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b1000a100 auth_method 0 2026-02-17T05:52:32.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.039+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:32.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.039+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:32.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.039+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:32.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.039+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:32.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.039+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:32.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:32.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:32.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:32.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b1000a100 auth_method 0 2026-02-17T05:52:32.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:32.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:32.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:32.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:32.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:32.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:32.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:32.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b1000a100 auth_method 0 2026-02-17T05:52:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:33.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:34.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient(hunting): tick 2026-02-17T05:52:34.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:52:34.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 monclient(hunting): continuing hunt 2026-02-17T05:52:34.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:52:34.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 msgr2=0x7f6b10005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:34.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b1000a100 0x7f6b10005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:34.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.115:6789/0 conn(0x7f6b1000a620 legacy=0x7f6b10008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 >> v1:10.20.193.78:6789/0 conn(0x7f6b1000ab70 legacy=0x7f6b1000d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient: _add_conns ranks=[3,4] 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient(hunting): picked mon.noname-a con 0x7f6b10010270 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient(hunting): picked mon.noname-c con 0x7f6b1000ed60 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3006d6f0 con 0x7f6b10010270 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 1 -- 10.20.193.78:0/269960890 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1000f6b0 con 0x7f6b1000ed60 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b17fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:34.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b10010270 auth_method 0 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b1000ed60 auth_method 0 2026-02-17T05:52:34.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:34.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:34.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.838+0000 7f6b3596b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:34.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.839+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 msgr2=0x7f6b1000f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:34.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.839+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 msgr2=0x7f6b1000f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:34.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.839+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:34.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:34.839+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:35.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b10010270 auth_method 0 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:35.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b1000ed60 auth_method 0 2026-02-17T05:52:35.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:35.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:35.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:35.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 msgr2=0x7f6b1000f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:35.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 msgr2=0x7f6b1000f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:35.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:35.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.039+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b10010270 auth_method 0 2026-02-17T05:52:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.439+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b1000ed60 auth_method 0 2026-02-17T05:52:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 msgr2=0x7f6b1000f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 msgr2=0x7f6b1000f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:35.440+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1185: done (7 left) 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1186: done (6 left) 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1187: done (5 left) 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1188: done (4 left) 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1189: done (3 left) 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1190: done (2 left) 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1193: done (1 left) 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1194: done (0 left) 2026-02-17T05:52:35.713 INFO:tasks.rados.rados.0.trial115.stdout:1195: delete oid 36 current snap is 117 2026-02-17T05:52:35.715 INFO:tasks.rados.rados.0.trial115.stdout:1195: done (0 left) 2026-02-17T05:52:35.715 INFO:tasks.rados.rados.0.trial115.stdout:1196: snap_remove snap 112 2026-02-17T05:52:36.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:36.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b10010270 auth_method 0 2026-02-17T05:52:36.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:36.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:36.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:36.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:36.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:36.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:36.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.240+0000 7f6b3616c640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.115:3300/0 conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b1000ed60 auth_method 0 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:36.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient(hunting): handle_auth_done global_id 44473 payload 274 2026-02-17T05:52:36.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 msgr2=0x7f6b10010680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:36.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> v2:10.20.193.78:3300/0 conn(0x7f6b10010270 0x7f6b10010680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:36.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient: _finish_hunting 0 2026-02-17T05:52:36.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 1 monclient: found mon.noname-c 2026-02-17T05:52:36.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:36.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:52:36.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.241+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b100110b0 con 0x7f6b1000ed60 2026-02-17T05:52:36.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3596b640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b1000ed60 msgr2=0x7f6b1000f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:36.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3596b640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b1000ed60 0x7f6b1000f170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6b10010d20 tx=0x7f6b24030b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11a36eebbb02f39 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:36.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 1 -- 10.20.193.78:0/269960890 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2403a070 con 0x7f6b1000ed60 2026-02-17T05:52:36.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:36.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:36.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: dump: 2026-02-17T05:52:36.250 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:36.250 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: _finish_auth 0 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: _check_auth_tickets 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b383f7640 5 monclient: authenticate success, global_id 44473 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b383f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 1 -- 10.20.193.78:0/269960890 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b24036640 con 0x7f6b1000ed60 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 1 -- 10.20.193.78:0/269960890 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24036a10 con 0x7f6b1000ed60 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b3516a640 10 monclient: dump: 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:36.252 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b383f7640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.244+0000 7f6b383f7640 4 set_mon_vals no callback set 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 10 monclient: shutdown 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 1 -- 10.20.193.78:0/269960890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b1000ed60 msgr2=0x7f6b1000f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b1000ed60 0x7f6b1000f170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6b10010d20 tx=0x7f6b24030b90 comp rx=0 tx=0).stop 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b3696d640 1 -- 10.20.193.78:0/269960890 reap_dead start 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 1 -- 10.20.193.78:0/269960890 shutdown_connections 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 1 --2- 10.20.193.78:0/269960890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b1000ed60 0x7f6b1000f170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 1 -- 10.20.193.78:0/269960890 >> 10.20.193.78:0/269960890 conn(0x7f6b300a6620 msgr2=0x7f6b300a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 1 -- 10.20.193.78:0/269960890 shutdown_connections 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 1 -- 10.20.193.78:0/269960890 wait complete. 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.245+0000 7f6b383f7640 10 monclient: build_initial_monmap 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient: monmap: 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:36.254 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 1 Processor -- start 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 1 -- start start 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient: init 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 0x7f6b301cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient(hunting): picked mon.noname-a con 0x7f6b1000ed60 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b301cfcc0 0x7f6b30121d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient(hunting): picked mon.noname-c con 0x7f6b301cfcc0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b301222b0 0x7f6b30122730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient(hunting): picked mon.noname-b con 0x7f6b301222b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300a01a0 con 0x7f6b1000ed60 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300a0020 con 0x7f6b301222b0 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3009be20 con 0x7f6b301cfcc0 2026-02-17T05:52:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b383f7640 10 monclient(hunting): authenticate will time out at 1001.799638s 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b301cfcc0 0x7f6b30121d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 0x7f6b301cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b301cfcc0 0x7f6b30121d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46312/0 (socket says 10.20.193.78:46312) 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 -- 10.20.193.78:0/3629319121 learned_addr learned my addr 10.20.193.78:0/3629319121 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient(hunting): get_auth_request con 0x7f6b301cfcc0 auth_method 0 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 10 monclient(hunting): get_auth_request con 0x7f6b1000ed60 auth_method 0 2026-02-17T05:52:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 1 -- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 msgr2=0x7f6b301cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 1 -- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 msgr2=0x7f6b301cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 0x7f6b301cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3616c640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 0x7f6b301cf780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3696d640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b301222b0 0x7f6b30122730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3696d640 10 monclient(hunting): get_auth_request con 0x7f6b301222b0 auth_method 0 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3696d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3696d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3696d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient(hunting): handle_auth_done global_id 74288 payload 824 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 -- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b301222b0 msgr2=0x7f6b30122730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b301222b0 0x7f6b30122730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 -- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 msgr2=0x7f6b301cf780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 0x7f6b301cf780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient: _finish_hunting 0 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 monclient: found mon.noname-c 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3596b640 1 -- 10.20.193.78:0/3629319121 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b301266e0 con 0x7f6b301cfcc0 2026-02-17T05:52:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.246+0000 7f6b3696d640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b301222b0 0x7f6b30122730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b3596b640 1 -- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b301cfcc0 msgr2=0x7f6b30121d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b3596b640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b301cfcc0 0x7f6b30121d70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b240310a0 tx=0x7f6b24004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 1 -- 10.20.193.78:0/3629319121 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2403a040 con 0x7f6b301cfcc0 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: dump: 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:36.260 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: _finish_auth 0 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 1 -- 10.20.193.78:0/3629319121 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b24002650 con 0x7f6b301cfcc0 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 5 monclient: authenticate success, global_id 74288 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 10 monclient: _renew_subs 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b30125d00 con 0x7f6b301cfcc0 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 1 -- 10.20.193.78:0/3629319121 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24002a20 con 0x7f6b301cfcc0 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 10 monclient: dump: 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:36.262 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b34969640 4 set_mon_vals no callback set 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 10 monclient: _renew_subs 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b300a12b0 con 0x7f6b301cfcc0 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 1 -- 10.20.193.78:0/3629319121 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6b24002bc0 con 0x7f6b301cfcc0 2026-02-17T05:52:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b301242f0 con 0x7f6b301cfcc0 2026-02-17T05:52:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.249+0000 7f6b16ffd640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b0003dee0 0x7f6b000403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.250+0000 7f6b16ffd640 1 -- 10.20.193.78:0/3629319121 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(251..251 src has 117..251) ==== 4783+0+0 (secure 0 0 0) 0x7f6b240797f0 con 0x7f6b301cfcc0 2026-02-17T05:52:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.250+0000 7f6b3616c640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b0003dee0 0x7f6b000403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.250+0000 7f6b3616c640 10 monclient: get_auth_request con 0x7f6b0003dee0 auth_method 0 2026-02-17T05:52:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.252+0000 7f6b3616c640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b0003dee0 0x7f6b000403a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b20006fb0 tx=0x7f6b20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.252+0000 7f6b16ffd640 1 -- 10.20.193.78:0/3629319121 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6b301266e0 con 0x7f6b301cfcc0 2026-02-17T05:52:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.252+0000 7f6b16ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.252+0000 7f6b16ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.365+0000 7f6b383f7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.365+0000 7f6b383f7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.365+0000 7f6b383f7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.365+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6b300744d0 con 0x7f6b301cfcc0 2026-02-17T05:52:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b16ffd640 1 -- 10.20.193.78:0/3629319121 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f6b24045070 con 0x7f6b301cfcc0 2026-02-17T05:52:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b16ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b16ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:36.366 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":70,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b0003dee0 msgr2=0x7f6b000403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b383f7640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b0003dee0 0x7f6b000403a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b20006fb0 tx=0x7f6b20008040 comp rx=0 tx=0).stop 2026-02-17T05:52:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b383f7640 10 monclient: shutdown 2026-02-17T05:52:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b383f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b301cfcc0 msgr2=0x7f6b30121d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.366+0000 7f6b383f7640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b301cfcc0 0x7f6b30121d70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b240310a0 tx=0x7f6b24004830 comp rx=0 tx=0).stop 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.367+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 shutdown_connections 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.367+0000 7f6b383f7640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b0003dee0 0x7f6b000403a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.367+0000 7f6b383f7640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b301222b0 0x7f6b30122730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.367+0000 7f6b383f7640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b301cfcc0 0x7f6b30121d70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.367+0000 7f6b383f7640 1 --2- 10.20.193.78:0/3629319121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b1000ed60 0x7f6b301cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.367+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 >> 10.20.193.78:0/3629319121 conn(0x7f6b300a6620 msgr2=0x7f6b301342a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.367+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 shutdown_connections 2026-02-17T05:52:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:36.367+0000 7f6b383f7640 1 -- 10.20.193.78:0/3629319121 wait complete. 2026-02-17T05:52:36.377 INFO:tasks.rados.rados.0.trial115.stdout:1196: done (0 left) 2026-02-17T05:52:36.377 INFO:tasks.rados.rados.0.trial115.stdout:1197: write (excl) oid 30 current snap is 117 2026-02-17T05:52:36.377 INFO:tasks.rados.rados.0.trial115.stdout:1197: seq_num 464 ranges {598448=413069,1768939=524148,2775798=581861,3781488=16960} 2026-02-17T05:52:36.381 INFO:tasks.rados.rados.0.trial115.stdout:1197: writing trial11545420-30 from 598448 to 1011517 tid 1 2026-02-17T05:52:36.386 INFO:tasks.rados.rados.0.trial115.stdout:1197: writing trial11545420-30 from 1768939 to 2293087 tid 2 2026-02-17T05:52:36.390 INFO:tasks.rados.rados.0.trial115.stdout:1197: writing trial11545420-30 from 2775798 to 3357659 tid 3 2026-02-17T05:52:36.392 INFO:tasks.rados.rados.0.trial115.stdout:1197: writing trial11545420-30 from 3781488 to 3798448 tid 4 2026-02-17T05:52:36.392 INFO:tasks.rados.rados.0.trial115.stdout:1198: delete oid 13 current snap is 117 2026-02-17T05:52:36.392 INFO:tasks.rados.rados.0.trial115.stdout:1197: finishing write tid 1 to trial11545420-30 2026-02-17T05:52:36.392 INFO:tasks.rados.rados.0.trial115.stdout:1197: finishing write tid 2 to trial11545420-30 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1197: finishing write tid 3 to trial11545420-30 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1198: done (1 left) 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1199: snap_create 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1197: finishing write tid 4 to trial11545420-30 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1197: finishing write tid 5 to trial11545420-30 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1197: finishing write tid 6 to trial11545420-30 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1197: oid 30 updating version 0 to 521 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1197: oid 30 updating version 521 to 522 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1197: oid 30 updating version 522 to 525 2026-02-17T05:52:36.394 INFO:tasks.rados.rados.0.trial115.stdout:1197: oid 30 version 525 is already newer than 523 2026-02-17T05:52:36.395 INFO:tasks.rados.rados.0.trial115.stdout:1197: oid 30 version 525 is already newer than 524 2026-02-17T05:52:36.395 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 525 (ObjNum 464 snap 117 seq_num 464) dirty exists 2026-02-17T05:52:36.395 INFO:tasks.rados.rados.0.trial115.stdout:1197: left oid 30 (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:52:38.055 INFO:tasks.rados.rados.0.trial115.stdout:1197: done (1 left) 2026-02-17T05:52:38.055 INFO:tasks.rados.rados.0.trial115.stdout:1199: done (0 left) 2026-02-17T05:52:38.056 INFO:tasks.rados.rados.0.trial115.stdout:1200: write oid 20 current snap is 118 2026-02-17T05:52:38.056 INFO:tasks.rados.rados.0.trial115.stdout:1200: seq_num 466 ranges {632062=563567,1885903=527817,2830644=720758} 2026-02-17T05:52:38.060 INFO:tasks.rados.rados.0.trial115.stdout:1200: writing trial11545420-20 from 632062 to 1195629 tid 1 2026-02-17T05:52:38.065 INFO:tasks.rados.rados.0.trial115.stdout:1200: writing trial11545420-20 from 1885903 to 2413720 tid 2 2026-02-17T05:52:38.069 INFO:tasks.rados.rados.0.trial115.stdout:1200: writing trial11545420-20 from 2830644 to 3551402 tid 3 2026-02-17T05:52:38.069 INFO:tasks.rados.rados.0.trial115.stdout:1201: write oid 19 current snap is 118 2026-02-17T05:52:38.070 INFO:tasks.rados.rados.0.trial115.stdout:1201: seq_num 467 ranges {648869=638816,1828032=787828,3141714=532030} 2026-02-17T05:52:38.074 INFO:tasks.rados.rados.0.trial115.stdout:1201: writing trial11545420-19 from 648869 to 1287685 tid 1 2026-02-17T05:52:38.079 INFO:tasks.rados.rados.0.trial115.stdout:1201: writing trial11545420-19 from 1828032 to 2615860 tid 2 2026-02-17T05:52:38.083 INFO:tasks.rados.rados.0.trial115.stdout:1201: writing trial11545420-19 from 3141714 to 3673744 tid 3 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1202: read oid 11 snap -1 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1202: expect (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1203: snap_create 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1200: finishing write tid 1 to trial11545420-20 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1200: finishing write tid 2 to trial11545420-20 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1200: finishing write tid 3 to trial11545420-20 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1200: finishing write tid 4 to trial11545420-20 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1200: finishing write tid 5 to trial11545420-20 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1200: oid 20 updating version 0 to 459 2026-02-17T05:52:38.084 INFO:tasks.rados.rados.0.trial115.stdout:1200: oid 20 version 459 is already newer than 458 2026-02-17T05:52:38.085 INFO:tasks.rados.rados.0.trial115.stdout:1200: oid 20 updating version 459 to 461 2026-02-17T05:52:38.085 INFO:tasks.rados.rados.0.trial115.stdout:1200: oid 20 version 461 is already newer than 460 2026-02-17T05:52:38.085 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 461 (ObjNum 466 snap 118 seq_num 466) dirty exists 2026-02-17T05:52:38.085 INFO:tasks.rados.rados.0.trial115.stdout:1200: left oid 20 (ObjNum 466 snap 118 seq_num 466) 2026-02-17T05:52:38.085 INFO:tasks.rados.rados.0.trial115.stdout:1201: finishing write tid 1 to trial11545420-19 2026-02-17T05:52:38.085 INFO:tasks.rados.rados.0.trial115.stdout:1201: finishing write tid 2 to trial11545420-19 2026-02-17T05:52:38.086 INFO:tasks.rados.rados.0.trial115.stdout:1201: finishing write tid 3 to trial11545420-19 2026-02-17T05:52:38.086 INFO:tasks.rados.rados.0.trial115.stdout:1201: finishing write tid 4 to trial11545420-19 2026-02-17T05:52:38.086 INFO:tasks.rados.rados.0.trial115.stdout:1201: finishing write tid 5 to trial11545420-19 2026-02-17T05:52:38.086 INFO:tasks.rados.rados.0.trial115.stdout:1201: oid 19 updating version 0 to 608 2026-02-17T05:52:38.087 INFO:tasks.rados.rados.0.trial115.stdout:1201: oid 19 version 608 is already newer than 605 2026-02-17T05:52:38.087 INFO:tasks.rados.rados.0.trial115.stdout:1201: oid 19 version 608 is already newer than 606 2026-02-17T05:52:38.087 INFO:tasks.rados.rados.0.trial115.stdout:1201: oid 19 version 608 is already newer than 607 2026-02-17T05:52:38.087 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 608 (ObjNum 467 snap 118 seq_num 467) dirty exists 2026-02-17T05:52:38.087 INFO:tasks.rados.rados.0.trial115.stdout:1201: left oid 19 (ObjNum 467 snap 118 seq_num 467) 2026-02-17T05:52:38.087 INFO:tasks.rados.rados.0.trial115.stdout:1202: expect (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:52:38.381 INFO:tasks.rados.rados.0.trial115.stdout:1200: done (3 left) 2026-02-17T05:52:38.382 INFO:tasks.rados.rados.0.trial115.stdout:1201: done (2 left) 2026-02-17T05:52:38.382 INFO:tasks.rados.rados.0.trial115.stdout:1202: done (1 left) 2026-02-17T05:52:38.382 INFO:tasks.rados.rados.0.trial115.stdout:1203: done (0 left) 2026-02-17T05:52:38.382 INFO:tasks.rados.rados.0.trial115.stdout:1204: snap_create 2026-02-17T05:52:39.374 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:39.382 INFO:tasks.rados.rados.0.trial115.stdout:1204: done (0 left) 2026-02-17T05:52:39.382 INFO:tasks.rados.rados.0.trial115.stdout:1205: write (excl) oid 25 current snap is 120 2026-02-17T05:52:39.383 INFO:tasks.rados.rados.0.trial115.stdout:1205: seq_num 468 ranges {665676=714065,1886514=531486,3169137=696539} 2026-02-17T05:52:39.387 INFO:tasks.rados.rados.0.trial115.stdout:1205: writing trial11545420-25 from 665676 to 1379741 tid 1 2026-02-17T05:52:39.391 INFO:tasks.rados.rados.0.trial115.stdout:1205: writing trial11545420-25 from 1886514 to 2418000 tid 2 2026-02-17T05:52:39.397 INFO:tasks.rados.rados.0.trial115.stdout:1205: writing trial11545420-25 from 3169137 to 3865676 tid 3 2026-02-17T05:52:39.397 INFO:tasks.rados.rados.0.trial115.stdout:1206: rollback oid 41 current snap is 120 2026-02-17T05:52:39.397 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 115 2026-02-17T05:52:39.397 INFO:tasks.rados.rados.0.trial115.stdout:1205: finishing write tid 1 to trial11545420-25 2026-02-17T05:52:39.397 INFO:tasks.rados.rados.0.trial115.stdout:1205: finishing write tid 2 to trial11545420-25 2026-02-17T05:52:39.397 INFO:tasks.rados.rados.0.trial115.stdout:1207: snap_remove snap 107 2026-02-17T05:52:39.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient: get_monmap_and_config 2026-02-17T05:52:39.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient: build_initial_monmap 2026-02-17T05:52:39.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient: monmap: 2026-02-17T05:52:39.447 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:39.447 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:39.446353+0000 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:39.446353+0000 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 1 Processor -- start 2026-02-17T05:52:39.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 1 -- start start 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient: init 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 0x7f918c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient(hunting): picked mon.noname-a con 0x7f918c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f918c09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c06d1b0 0x7f918c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient(hunting): picked mon.noname-b con 0x7f918c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f918c06daf0 con 0x7f918c09c4a0 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f918c0a0880 con 0x7f918c06c890 2026-02-17T05:52:39.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f918c09cc80 con 0x7f918c06d1b0 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9193264640 10 monclient(hunting): authenticate will time out at 1004.999888s 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9190fd9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 0x7f918c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9183fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f918c09c4a0 0x7f918c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46168/0 (socket says 10.20.193.78:46168) 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9183fff640 1 -- 10.20.193.78:0/1904636862 learned_addr learned my addr 10.20.193.78:0/1904636862 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9190fd9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 0x7f918c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51404/0 (socket says 10.20.193.78:51404) 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9190fd9640 10 monclient(hunting): get_auth_request con 0x7f918c06c890 auth_method 0 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9190fd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9190fd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f9190fd9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:39.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.446+0000 7f91917da640 1 --2- 10.20.193.78:0/1904636862 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c06d1b0 0x7f918c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient(hunting): get_auth_request con 0x7f918c06d1b0 auth_method 0 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f9190fd9640 1 -- 10.20.193.78:0/1904636862 >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 msgr2=0x7f918c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f9190fd9640 1 -- 10.20.193.78:0/1904636862 >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 msgr2=0x7f918c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f9190fd9640 1 --2- 10.20.193.78:0/1904636862 >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 0x7f918c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f9190fd9640 1 --2- 10.20.193.78:0/1904636862 >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 0x7f918c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:39.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient(hunting): handle_auth_done global_id 74291 payload 274 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 1 -- 10.20.193.78:0/1904636862 >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 msgr2=0x7f918c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 1 --2- 10.20.193.78:0/1904636862 >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 0x7f918c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 1 -- 10.20.193.78:0/1904636862 >> v1:10.20.193.78:6789/0 conn(0x7f918c09c4a0 legacy=0x7f918c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient: _finish_hunting 0 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 1 monclient: found mon.noname-b 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:52:39.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.447+0000 7f91917da640 1 -- 10.20.193.78:0/1904636862 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c09cf60 con 0x7f918c06d1b0 2026-02-17T05:52:39.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91917da640 1 -- 10.20.193.78:0/1904636862 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c06d1b0 msgr2=0x7f918c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:39.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91917da640 1 --2- 10.20.193.78:0/1904636862 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c06d1b0 0x7f918c135500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f918c06dcc0 tx=0x7f918802f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4caba5680315f0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:39.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 1 -- 10.20.193.78:0/1904636862 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918803c070 con 0x7f918c06d1b0 2026-02-17T05:52:39.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:39.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:52:39.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: dump: 2026-02-17T05:52:39.461 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:39.461 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:39.461 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: _finish_auth 0 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: _check_auth_tickets 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f9193264640 5 monclient: authenticate success, global_id 74291 2026-02-17T05:52:39.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f9193264640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 1 -- 10.20.193.78:0/1904636862 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9188037440 con 0x7f918c06d1b0 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f9193264640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 1 -- 10.20.193.78:0/1904636862 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188037810 con 0x7f918c06d1b0 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f9193264640 4 set_mon_vals no callback set 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.449+0000 7f91837fe640 10 monclient: dump: 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:39.463 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient: shutdown 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- 10.20.193.78:0/1904636862 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c06d1b0 msgr2=0x7f918c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 --2- 10.20.193.78:0/1904636862 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c06d1b0 0x7f918c135500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f918c06dcc0 tx=0x7f918802f150 comp rx=0 tx=0).stop 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- 10.20.193.78:0/1904636862 shutdown_connections 2026-02-17T05:52:39.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 --2- 10.20.193.78:0/1904636862 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c06d1b0 0x7f918c135500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 --2- 10.20.193.78:0/1904636862 >> v2:10.20.193.78:3300/0 conn(0x7f918c06c890 0x7f918c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- 10.20.193.78:0/1904636862 >> 10.20.193.78:0/1904636862 conn(0x7f918c0a6620 msgr2=0x7f918c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- 10.20.193.78:0/1904636862 shutdown_connections 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- 10.20.193.78:0/1904636862 wait complete. 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient: build_initial_monmap 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient: monmap: 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:39.465 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 Processor -- start 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- start start 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient: init 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f918c06c890 0x7f918c11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient(hunting): picked mon.noname-b con 0x7f918c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:39.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 0x7f918c11d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient(hunting): picked mon.noname-a con 0x7f918c06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c09c4a0 0x7f918c11d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient(hunting): picked mon.noname-c con 0x7f918c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f918c0a01a0 con 0x7f918c06d1b0 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f918c0a0020 con 0x7f918c06c890 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f918c09be20 con 0x7f918c09c4a0 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9193264640 10 monclient(hunting): authenticate will time out at 1005.003914s 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f91917da640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c09c4a0 0x7f918c11d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:39.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9183fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 0x7f918c11d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f91917da640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c09c4a0 0x7f918c11d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48824/0 (socket says 10.20.193.78:48824) 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f91917da640 1 -- 10.20.193.78:0/949117346 learned_addr learned my addr 10.20.193.78:0/949117346 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f91917da640 10 monclient(hunting): get_auth_request con 0x7f918c09c4a0 auth_method 0 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f91917da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f91917da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f91917da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9183fff640 10 monclient(hunting): get_auth_request con 0x7f918c06d1b0 auth_method 0 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9183fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9183fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.450+0000 7f9183fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:39.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f9183fff640 1 -- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 msgr2=0x7f918c11d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f9183fff640 1 -- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 msgr2=0x7f918c11d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f9183fff640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 0x7f918c11d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f9183fff640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 0x7f918c11d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f9190fd9640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f918c06c890 0x7f918c11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 10 monclient(hunting): handle_auth_done global_id 74294 payload 824 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 1 -- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f918c06c890 msgr2=0x7f918c11cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f918c06c890 0x7f918c11cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 1 -- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 msgr2=0x7f918c11d1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 0x7f918c11d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 10 monclient: _finish_hunting 0 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 1 monclient: found mon.noname-c 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f91917da640 1 -- 10.20.193.78:0/949117346 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c120120 con 0x7f918c09c4a0 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f9190fd9640 10 monclient: get_auth_request con 0x7f918c06c890 auth_method 0 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.451+0000 7f9190fd9640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f918c06c890 0x7f918c11cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91917da640 1 -- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c09c4a0 msgr2=0x7f918c11d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91917da640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c09c4a0 0x7f918c11d6f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9188009e90 tx=0x7f9188004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 1 -- 10.20.193.78:0/949117346 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918803c050 con 0x7f918c09c4a0 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:39.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: dump: 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: _finish_auth 0 2026-02-17T05:52:39.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: _check_auth_tickets 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 1 -- 10.20.193.78:0/949117346 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9188048070 con 0x7f918c09c4a0 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f9193264640 5 monclient: authenticate success, global_id 74294 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f9193264640 10 monclient: _renew_subs 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f9193264640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c0a2030 con 0x7f918c09c4a0 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 1 -- 10.20.193.78:0/949117346 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188041c60 con 0x7f918c09c4a0 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f9182ffd640 4 set_mon_vals no callback set 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f91817fa640 10 monclient: dump: 2026-02-17T05:52:39.472 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f9193264640 10 monclient: _renew_subs 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f9193264640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.453+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c11fb60 con 0x7f918c09c4a0 2026-02-17T05:52:39.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f91817fa640 1 -- 10.20.193.78:0/949117346 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f918804d460 con 0x7f918c09c4a0 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f9193264640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f9193264640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f9193264640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c0a3b20 con 0x7f918c09c4a0 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f91817fa640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f915c03deb0 0x7f915c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f91817fa640 1 -- 10.20.193.78:0/949117346 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(255..255 src has 117..255) ==== 4735+0+0 (secure 0 0 0) 0x7f9188078770 con 0x7f918c09c4a0 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f9190fd9640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f915c03deb0 0x7f915c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.454+0000 7f9190fd9640 10 monclient: get_auth_request con 0x7f915c03deb0 auth_method 0 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.456+0000 7f9190fd9640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f915c03deb0 0x7f915c040370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f917c006fb0 tx=0x7f917c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.456+0000 7f91817fa640 1 -- 10.20.193.78:0/949117346 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f918c0a3b20 con 0x7f918c09c4a0 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.456+0000 7f91817fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:39.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.456+0000 7f91817fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.574+0000 7f9193264640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.574+0000 7f9193264640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.574+0000 7f9193264640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.574+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f918c135ad0 con 0x7f918c09c4a0 2026-02-17T05:52:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.574+0000 7f91817fa640 1 -- 10.20.193.78:0/949117346 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f91880771a0 con 0x7f918c09c4a0 2026-02-17T05:52:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.574+0000 7f91817fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.574+0000 7f91817fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:39.574 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":70,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f915c03deb0 msgr2=0x7f915c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f915c03deb0 0x7f915c040370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f917c006fb0 tx=0x7f917c008040 comp rx=0 tx=0).stop 2026-02-17T05:52:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 10 monclient: shutdown 2026-02-17T05:52:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c09c4a0 msgr2=0x7f918c11d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c09c4a0 0x7f918c11d6f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9188009e90 tx=0x7f9188004b40 comp rx=0 tx=0).stop 2026-02-17T05:52:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 shutdown_connections 2026-02-17T05:52:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f918c09c4a0 0x7f918c11d6f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f918c06d1b0 0x7f918c11d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f915c03deb0 0x7f915c040370 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 --2- 10.20.193.78:0/949117346 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f918c06c890 0x7f918c11cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 >> 10.20.193.78:0/949117346 conn(0x7f918c0a6620 msgr2=0x7f918c1346d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 shutdown_connections 2026-02-17T05:52:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:39.575+0000 7f9193264640 1 -- 10.20.193.78:0/949117346 wait complete. 2026-02-17T05:52:40.387 INFO:tasks.rados.rados.0.trial115.stdout:1207: done (2 left) 2026-02-17T05:52:40.388 INFO:tasks.rados.rados.0.trial115.stdout:1208: rollback oid 28 current snap is 120 2026-02-17T05:52:40.388 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 28 to 117 2026-02-17T05:52:40.388 INFO:tasks.rados.rados.0.trial115.stdout:1205: finishing write tid 3 to trial11545420-25 2026-02-17T05:52:40.388 INFO:tasks.rados.rados.0.trial115.stdout:1205: finishing write tid 4 to trial11545420-25 2026-02-17T05:52:40.388 INFO:tasks.rados.rados.0.trial115.stdout:1205: finishing write tid 5 to trial11545420-25 2026-02-17T05:52:40.388 INFO:tasks.rados.rados.0.trial115.stdout:1205: oid 25 updating version 0 to 493 2026-02-17T05:52:40.388 INFO:tasks.rados.rados.0.trial115.stdout:1205: oid 25 updating version 493 to 494 2026-02-17T05:52:40.388 INFO:tasks.rados.rados.0.trial115.stdout:1205: oid 25 version 494 is already newer than 492 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:1205: oid 25 version 494 is already newer than 491 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 494 (ObjNum 468 snap 120 seq_num 468) dirty exists 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:1205: left oid 25 (ObjNum 468 snap 120 seq_num 468) 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:1206: finishing rollback tid 1 to trial11545420-41 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 329 (ObjNum 129 snap 0 seq_num 1333667944) dirty dne 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:1205: done (2 left) 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:1206: done (1 left) 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:1209: write (excl) oid 4 current snap is 120 2026-02-17T05:52:40.389 INFO:tasks.rados.rados.0.trial115.stdout:1209: seq_num 469 ranges {682483=789314,1944996=791497,3196560=554574} 2026-02-17T05:52:40.392 INFO:tasks.rados.rados.0.trial115.stdout:1209: writing trial11545420-4 from 682483 to 1471797 tid 1 2026-02-17T05:52:40.397 INFO:tasks.rados.rados.0.trial115.stdout:1209: writing trial11545420-4 from 1944996 to 2736493 tid 2 2026-02-17T05:52:40.401 INFO:tasks.rados.rados.0.trial115.stdout:1209: writing trial11545420-4 from 3196560 to 3751134 tid 3 2026-02-17T05:52:40.401 INFO:tasks.rados.rados.0.trial115.stdout:1210: snap_create 2026-02-17T05:52:40.401 INFO:tasks.rados.rados.0.trial115.stdout:1208: finishing rollback tid 1 to trial11545420-28 2026-02-17T05:52:40.401 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 0 (ObjNum 401 snap 0 seq_num 1333817632) dirty dne 2026-02-17T05:52:40.401 INFO:tasks.rados.rados.0.trial115.stdout:1209: finishing write tid 1 to trial11545420-4 2026-02-17T05:52:40.401 INFO:tasks.rados.rados.0.trial115.stdout:1209: finishing write tid 2 to trial11545420-4 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: finishing write tid 3 to trial11545420-4 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: finishing write tid 4 to trial11545420-4 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: finishing write tid 5 to trial11545420-4 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: oid 4 updating version 0 to 306 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: oid 4 updating version 306 to 308 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: oid 4 version 308 is already newer than 307 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: oid 4 version 308 is already newer than 305 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 308 (ObjNum 469 snap 120 seq_num 469) dirty exists 2026-02-17T05:52:40.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: left oid 4 (ObjNum 469 snap 120 seq_num 469) 2026-02-17T05:52:42.583 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:42.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient: get_monmap_and_config 2026-02-17T05:52:42.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient: build_initial_monmap 2026-02-17T05:52:42.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient: monmap: 2026-02-17T05:52:42.655 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:42.655 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:42.655 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:42.654359+0000 2026-02-17T05:52:42.655 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:42.654359+0000 2026-02-17T05:52:42.655 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 1 Processor -- start 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 1 -- start start 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient: init 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2026-02-17T05:52:42.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f253006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient(hunting): picked mon.noname-b con 0x7f253009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient(hunting): picked mon.noname-a con 0x7f253006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f253006daf0 con 0x7f253006c890 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f25300a0880 con 0x7f253006d380 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f253009cc80 con 0x7f253009c4a0 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2537e76640 10 monclient(hunting): authenticate will time out at 1008.207891s 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25353ea640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25363ec640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25353ea640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48828/0 (socket says 10.20.193.78:48828) 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 learned_addr learned my addr 10.20.193.78:0/945401287 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25353ea640 10 monclient(hunting): get_auth_request con 0x7f253009c4a0 auth_method 0 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25353ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25353ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25353ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f2535beb640 1 --1- 10.20.193.78:0/945401287 >> v1:10.20.193.78:6789/0 conn(0x7f253006c890 0x7f253006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46180/0 (socket says 10.20.193.78:46180) 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25363ec640 10 monclient(hunting): get_auth_request con 0x7f253006d380 auth_method 0 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25363ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.654+0000 7f25363ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25363ec640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:42.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:42.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:52:42.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:42.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:42.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.655+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:42.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.657+0000 7f2535beb640 1 -- 10.20.193.78:0/945401287 >> v1:10.20.193.78:6789/0 conn(0x7f253006c890 legacy=0x7f253006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:42.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 10 monclient(hunting): get_auth_request con 0x7f253009c4a0 auth_method 0 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 10 monclient(hunting): get_auth_request con 0x7f253006d380 auth_method 0 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:42.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:42.855+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:43.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.255+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.255+0000 7f25353ea640 10 monclient(hunting): get_auth_request con 0x7f253009c4a0 auth_method 0 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.255+0000 7f25353ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.255+0000 7f25353ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.255+0000 7f25353ea640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.255+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:43.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:43.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25363ec640 10 monclient(hunting): get_auth_request con 0x7f253006d380 auth_method 0 2026-02-17T05:52:43.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25363ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:43.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25363ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:43.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25363ec640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:43.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:52:43.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:43.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:43.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:43.256+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:44.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.056+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.056+0000 7f25353ea640 10 monclient(hunting): get_auth_request con 0x7f253009c4a0 auth_method 0 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.056+0000 7f25353ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.056+0000 7f25353ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.056+0000 7f25353ea640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 10 monclient(hunting): get_auth_request con 0x7f253006d380 auth_method 0 2026-02-17T05:52:44.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:44.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:52:44.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:44.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:44.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.057+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.865+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741609908 0 0) 0x7f253006daf0 con 0x7f253006c890 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.865+0000 7f2534be9640 10 monclient(hunting): _negotiate 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.865+0000 7f2534be9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.865+0000 7f2534be9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.865+0000 7f2534be9640 10 monclient(hunting): my global_id is 94098 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.865+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2518003620 con 0x7f253006c890 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2698563514 0 0) 0x7f2518003620 con 0x7f253006c890 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f253013b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f253013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 msgr2=0x7f253013d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 --2- 10.20.193.78:0/945401287 >> v2:10.20.193.78:3300/0 conn(0x7f253006d380 0x7f253013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: _finish_hunting 0 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253009cf60 con 0x7f253006c890 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: _finish_auth 0 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: _check_auth_tickets 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f25200031b0 con 0x7f253006c890 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: dump: 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:44.877 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 >> v1:10.20.193.78:6789/0 conn(0x7f253006c890 legacy=0x7f253006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2518003bd0 0x7f2518003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient(hunting): picked mon.b con 0x7f2518003bd0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f2518004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient(hunting): picked mon.c con 0x7f253009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2518004f60 0x7f251800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient(hunting): picked mon.a con 0x7f2518004f60 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f253009cc80 con 0x7f2518004f60 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2518005750 con 0x7f2518003bd0 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f251800c000 con 0x7f253009c4a0 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2534be9640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f2518004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient(hunting): get_auth_request con 0x7f253009c4a0 auth_method 0 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25363ec640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2518004f60 0x7f251800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25363ec640 10 monclient(hunting): get_auth_request con 0x7f2518004f60 auth_method 0 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25363ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25363ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25363ec640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:44.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2535beb640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2518003bd0 0x7f2518003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2535beb640 10 monclient(hunting): get_auth_request con 0x7f2518003bd0 auth_method 0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2535beb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2535beb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2535beb640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient(hunting): handle_auth_done global_id 94098 payload 293 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2518003bd0 msgr2=0x7f2518003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2518003bd0 0x7f2518003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2518004f60 msgr2=0x7f251800bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2518004f60 0x7f251800bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient: _finish_hunting 0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 1 monclient: found mon.c 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2518003620 con 0x7f253009c4a0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient: _finish_auth 0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 10 monclient: _check_auth_tickets 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f25353ea640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2537e76640 5 monclient: authenticate success, global_id 94098 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.866+0000 7f2537e76640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.868+0000 7f25353ea640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f2518004900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.868+0000 7f25353ea640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f2518004900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2524003b10 tx=0x7f252402fb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e6da708c84034b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524039070 con 0x7f253009c4a0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 10 monclient: dump: 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:44.879 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2524034680 con 0x7f253009c4a0 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 1 -- 10.20.193.78:0/945401287 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524003710 con 0x7f253009c4a0 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2534be9640 10 monclient: dump: 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 4 set_mon_vals no callback set 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 10 monclient: shutdown 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 1 -- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 msgr2=0x7f2518004900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 1 --2- 10.20.193.78:0/945401287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f253009c4a0 0x7f2518004900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2524003b10 tx=0x7f252402fb70 comp rx=0 tx=0).stop 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f25363ec640 1 -- 10.20.193.78:0/945401287 reap_dead start 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 1 -- 10.20.193.78:0/945401287 shutdown_connections 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 1 -- 10.20.193.78:0/945401287 >> 10.20.193.78:0/945401287 conn(0x7f25300a6620 msgr2=0x7f25300a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 1 -- 10.20.193.78:0/945401287 shutdown_connections 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.869+0000 7f2537e76640 1 -- 10.20.193.78:0/945401287 wait complete. 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient: build_initial_monmap 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient: monmap: 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:44.880 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 1 Processor -- start 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 1 -- start start 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient: init 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f2530121180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient(hunting): picked mon.noname-a con 0x7f253009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f25301216c0 0x7f2530121b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient(hunting): picked mon.noname-b con 0x7f25301216c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 0x7f25301cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient(hunting): picked mon.noname-c con 0x7f25301cf4d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f253009bfa0 con 0x7f253009c4a0 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f253009be20 con 0x7f25301216c0 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2530139390 con 0x7f25301cf4d0 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2537e76640 10 monclient(hunting): authenticate will time out at 1010.423576s 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f2530121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 0x7f25301cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f2530121180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51478/0 (socket says 10.20.193.78:51478) 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 1 -- 10.20.193.78:0/1049603185 learned_addr learned my addr 10.20.193.78:0/1049603185 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 10 monclient(hunting): get_auth_request con 0x7f253009c4a0 auth_method 0 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient(hunting): get_auth_request con 0x7f25301cf4d0 auth_method 0 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:44.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f2535beb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient(hunting): handle_auth_done global_id 74297 payload 824 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f25301216c0 msgr2=0x7f2530121b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f25301216c0 0x7f2530121b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 msgr2=0x7f2530121180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f2530121180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient: _finish_hunting 0 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 1 monclient: found mon.noname-c 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.870+0000 7f25363ec640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2530123560 con 0x7f25301cf4d0 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.872+0000 7f25363ec640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 msgr2=0x7f25301cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.872+0000 7f25363ec640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 0x7f25301cf950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f253006d760 tx=0x7f252c004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 msgr2=0x7f25301cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 msgr2=0x7f25301cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 0x7f25301cf950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f253006d760 tx=0x7f252c004a80 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 0x7f25301cf950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f253006d760 tx=0x7f252c004a80 comp rx=0 tx=0).stop 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7f252c01c070 con 0x7f25301cf4d0 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient: dump: 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient: ms_handle_reset current mon [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 msgr2=0x7f25301cf950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f25301cf4d0 0x7f25301cf950 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f25301216c0 0x7f2500005840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient(hunting): picked mon.b con 0x7f25301216c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f25000082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient(hunting): picked mon.a con 0x7f253009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2500008cd0 0x7f250000b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient(hunting): picked mon.c con 0x7f2500008cd0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f250000bc10 con 0x7f253009c4a0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f250000bd90 con 0x7f25301216c0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f250000bf10 con 0x7f2500008cd0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f251e7fc640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f25000082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2500008cd0 0x7f250000b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient(hunting): get_auth_request con 0x7f253009c4a0 auth_method 0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 10 monclient(hunting): get_auth_request con 0x7f2500008cd0 auth_method 0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient(hunting): handle_auth_done global_id 74297 payload 293 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2500008cd0 msgr2=0x7f250000b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2500008cd0 0x7f250000b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f25301216c0 msgr2=0x7f2500005840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f25301216c0 0x7f2500005840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient: _finish_hunting 0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 1 monclient: found mon.a 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=2+}) -- 0x7f250000c180 con 0x7f253009c4a0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient: _finish_auth 0 2026-02-17T05:52:44.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 10 monclient: _check_auth_tickets 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25353ea640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f2537e76640 5 monclient: authenticate success, global_id 74297 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f25363ec640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2500008cd0 0x7f250000b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f2537e76640 10 monclient: _renew_subs 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f2537e76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2530123840 con 0x7f253009c4a0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f2537e76640 10 monclient: _renew_subs 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f2537e76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.873+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2530133de0 con 0x7f253009c4a0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.874+0000 7f2537e76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.874+0000 7f2537e76640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.874+0000 7f2537e76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.874+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2530137630 con 0x7f253009c4a0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f25353ea640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 msgr2=0x7f25000082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f25353ea640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f25000082c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2524006fd0 tx=0x7f252c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524039040 con 0x7f253009c4a0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f251e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f251e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f251e7fc640 10 monclient: dump: 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2524006cf0 con 0x7f253009c4a0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f251e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.875+0000 7f251ffff640 4 set_mon_vals no callback set 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.876+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2524050450 con 0x7f253009c4a0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.876+0000 7f251e7fc640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25301cf4d0 0x7f25000483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.876+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 <== mon.0 v2:10.20.193.78:3300/0 4 ==== osd_map(256..256 src has 117..256) ==== 4779+0+0 (secure 0 0 0) 0x7f2524077370 con 0x7f253009c4a0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.876+0000 7f2535beb640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25301cf4d0 0x7f25000483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.876+0000 7f2535beb640 10 monclient: get_auth_request con 0x7f25301cf4d0 auth_method 0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.878+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 <== mon.0 v2:10.20.193.78:3300/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2524074dc0 con 0x7f253009c4a0 2026-02-17T05:52:44.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.878+0000 7f251e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:44.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.878+0000 7f251e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:44.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.878+0000 7f2535beb640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25301cf4d0 0x7f25000483c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2520007ee0 tx=0x7f25200079f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:44.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.990+0000 7f2537e76640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:44.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.990+0000 7f2537e76640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:44.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.990+0000 7f2537e76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:44.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.990+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f25300744d0 con 0x7f253009c4a0 2026-02-17T05:52:44.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.990+0000 7f251e7fc640 1 -- 10.20.193.78:0/1049603185 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f25240770d0 con 0x7f253009c4a0 2026-02-17T05:52:44.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.990+0000 7f251e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:44.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.990+0000 7f251e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:44.990 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":74,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:44.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25301cf4d0 msgr2=0x7f25000483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25301cf4d0 0x7f25000483c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2520007ee0 tx=0x7f25200079f0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 10 monclient: shutdown 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 msgr2=0x7f25000082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f25000082c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2524006fd0 tx=0x7f252c00b040 comp rx=0 tx=0).stop 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 shutdown_connections 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2500008cd0 0x7f250000b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f253009c4a0 0x7f25000082c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25301cf4d0 0x7f25000483c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 --2- 10.20.193.78:0/1049603185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f25301216c0 0x7f2500005840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:44.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 >> 10.20.193.78:0/1049603185 conn(0x7f25300a6620 msgr2=0x7f25301334b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:44.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 shutdown_connections 2026-02-17T05:52:44.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:44.991+0000 7f2537e76640 1 -- 10.20.193.78:0/1049603185 wait complete. 2026-02-17T05:52:44.998 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:52:44.998 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:52:44.998 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:52:45.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.067+0000 7fda33379640 10 monclient: get_monmap_and_config 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.067+0000 7fda33379640 10 monclient: build_initial_monmap 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.067+0000 7fda33379640 10 monclient: monmap: 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:45.067633+0000 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:45.067633+0000 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:45.066 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:45.067 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:45.067 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:45.067 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.067+0000 7fda33379640 1 Processor -- start 2026-02-17T05:52:45.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.067+0000 7fda33379640 1 -- start start 2026-02-17T05:52:45.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.067+0000 7fda33379640 10 monclient: init 2026-02-17T05:52:45.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fda2c06c890 0x7fda2c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient(hunting): picked mon.noname-c con 0x7fda2c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fda2c09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c06d1b0 0x7fda2c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient(hunting): picked mon.noname-b con 0x7fda2c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda2c06daf0 con 0x7fda2c09c4a0 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fda2c0a0880 con 0x7fda2c06c890 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c09cc80 con 0x7fda2c06d1b0 2026-02-17T05:52:45.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda33379640 10 monclient(hunting): authenticate will time out at 1010.621219s 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c06d1b0 0x7fda2c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda308ed640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fda2c09c4a0 0x7fda2c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46194/0 (socket says 10.20.193.78:46194) 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c06d1b0 0x7fda2c135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48886/0 (socket says 10.20.193.78:48886) 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda308ed640 1 -- 10.20.193.78:0/2226837242 learned_addr learned my addr 10.20.193.78:0/2226837242 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient(hunting): get_auth_request con 0x7fda2c06d1b0 auth_method 0 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda310ee640 1 --2- 10.20.193.78:0/2226837242 >> v2:10.20.193.115:3300/0 conn(0x7fda2c06c890 0x7fda2c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda310ee640 10 monclient(hunting): get_auth_request con 0x7fda2c06c890 auth_method 0 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda310ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda310ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda310ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient(hunting): handle_auth_done global_id 74300 payload 274 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 1 -- 10.20.193.78:0/2226837242 >> v2:10.20.193.115:3300/0 conn(0x7fda2c06c890 msgr2=0x7fda2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 1 --2- 10.20.193.78:0/2226837242 >> v2:10.20.193.115:3300/0 conn(0x7fda2c06c890 0x7fda2c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 1 -- 10.20.193.78:0/2226837242 >> v1:10.20.193.78:6789/0 conn(0x7fda2c09c4a0 legacy=0x7fda2c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient: _finish_hunting 0 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 1 monclient: found mon.noname-b 2026-02-17T05:52:45.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda318ef640 1 -- 10.20.193.78:0/2226837242 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c09cf60 con 0x7fda2c06d1b0 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.068+0000 7fda310ee640 1 --2- 10.20.193.78:0/2226837242 >> v2:10.20.193.115:3300/0 conn(0x7fda2c06c890 0x7fda2c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda318ef640 1 -- 10.20.193.78:0/2226837242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c06d1b0 msgr2=0x7fda2c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda318ef640 1 --2- 10.20.193.78:0/2226837242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c06d1b0 0x7fda2c135500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fda2c06dcc0 tx=0x7fda2802f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bba1fa481ff139d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 1 -- 10.20.193.78:0/2226837242 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2803c070 con 0x7fda2c06d1b0 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: dump: 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.081 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: _finish_auth 0 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 5 monclient: authenticate success, global_id 74300 2026-02-17T05:52:45.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 1 -- 10.20.193.78:0/2226837242 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda28037440 con 0x7fda2c06d1b0 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 1 -- 10.20.193.78:0/2226837242 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28037810 con 0x7fda2c06d1b0 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 4 set_mon_vals no callback set 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda1bfff640 10 monclient: dump: 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.083 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 10 monclient: shutdown 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 1 -- 10.20.193.78:0/2226837242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c06d1b0 msgr2=0x7fda2c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 1 --2- 10.20.193.78:0/2226837242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c06d1b0 0x7fda2c135500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fda2c06dcc0 tx=0x7fda2802f150 comp rx=0 tx=0).stop 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 1 -- 10.20.193.78:0/2226837242 shutdown_connections 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 1 --2- 10.20.193.78:0/2226837242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c06d1b0 0x7fda2c135500 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 1 --2- 10.20.193.78:0/2226837242 >> v2:10.20.193.115:3300/0 conn(0x7fda2c06c890 0x7fda2c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 1 -- 10.20.193.78:0/2226837242 >> 10.20.193.78:0/2226837242 conn(0x7fda2c0a6620 msgr2=0x7fda2c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 1 -- 10.20.193.78:0/2226837242 shutdown_connections 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 1 -- 10.20.193.78:0/2226837242 wait complete. 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 10 monclient: build_initial_monmap 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.071+0000 7fda33379640 10 monclient: monmap: 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.085 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 1 Processor -- start 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 1 -- start start 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient: init 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fda2c06c890 0x7fda2c1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient(hunting): picked mon.noname-b con 0x7fda2c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c06d1b0 0x7fda2c1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient(hunting): picked mon.noname-a con 0x7fda2c06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c09c4a0 0x7fda2c1217d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient(hunting): picked mon.noname-c con 0x7fda2c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c0a01a0 con 0x7fda2c06d1b0 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c0a0020 con 0x7fda2c06c890 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c09be20 con 0x7fda2c09c4a0 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda33379640 10 monclient(hunting): authenticate will time out at 1010.625542s 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c09c4a0 0x7fda2c1217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c06d1b0 0x7fda2c1cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c06d1b0 0x7fda2c1cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51500/0 (socket says 10.20.193.78:51500) 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 1 -- 10.20.193.78:0/298145187 learned_addr learned my addr 10.20.193.78:0/298145187 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c09c4a0 0x7fda2c1217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48888/0 (socket says 10.20.193.78:48888) 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 10 monclient(hunting): get_auth_request con 0x7fda2c06d1b0 auth_method 0 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient(hunting): get_auth_request con 0x7fda2c09c4a0 auth_method 0 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda308ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda310ee640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fda2c06c890 0x7fda2c1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda310ee640 10 monclient(hunting): get_auth_request con 0x7fda2c06c890 auth_method 0 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda310ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda310ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda310ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient(hunting): handle_auth_done global_id 74303 payload 824 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fda2c06c890 msgr2=0x7fda2c1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fda2c06c890 0x7fda2c1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c06d1b0 msgr2=0x7fda2c1cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c06d1b0 0x7fda2c1cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient: _finish_hunting 0 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 1 monclient: found mon.noname-c 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.072+0000 7fda318ef640 1 -- 10.20.193.78:0/298145187 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c0a20c0 con 0x7fda2c09c4a0 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.074+0000 7fda318ef640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c09c4a0 msgr2=0x7fda2c1217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.074+0000 7fda318ef640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c09c4a0 0x7fda2c1217d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda28036000 tx=0x7fda28004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2803c050 con 0x7fda2c09c4a0 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:45.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: dump: 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: _finish_auth 0 2026-02-17T05:52:45.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2803d040 con 0x7fda2c09c4a0 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 5 monclient: authenticate success, global_id 74303 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 10 monclient: _renew_subs 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c09d810 con 0x7fda2c09c4a0 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28041a20 con 0x7fda2c09c4a0 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda1b7fe640 4 set_mon_vals no callback set 2026-02-17T05:52:45.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 10 monclient: dump: 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 10 monclient: _renew_subs 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:45.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c1253e0 con 0x7fda2c09c4a0 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1224d0 0x7fda2c11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c123f30 con 0x7fda2c1224d0 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fda28041c40 con 0x7fda2c09c4a0 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1224d0 0x7fda2c11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 10 monclient: get_auth_request con 0x7fda2c1224d0 auth_method 0 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 10 monclient(hunting): handle_auth_done global_id 94107 payload 824 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda310ee640 1 -- 10.20.193.78:0/298145187 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda2c0a3c40 con 0x7fda2c1224d0 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9fc03df00 0x7fd9fc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.075+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(256..256 src has 117..256) ==== 4779+0+0 (secure 0 0 0) 0x7fda280780b0 con 0x7fda2c09c4a0 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.076+0000 7fda308ed640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9fc03df00 0x7fd9fc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.076+0000 7fda308ed640 10 monclient: get_auth_request con 0x7fd9fc03df00 auth_method 0 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.077+0000 7fda310ee640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1224d0 msgr2=0x7fda2c11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.077+0000 7fda310ee640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1224d0 0x7fda2c11cc70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda1c010790 tx=0x7fda1c010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c01a020 con 0x7fda2c1224d0 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 10 monclient: dump: 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fda1c00ede0 con 0x7fda2c1224d0 2026-02-17T05:52:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1224d0 msgr2=0x7fda2c11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1224d0 0x7fda2c11cc70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda1c010790 tx=0x7fda1c010c60 comp rx=0 tx=0).stop 2026-02-17T05:52:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1224d0 msgr2=0x7fda2c11cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda19ffb640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1224d0 0x7fda2c11cc70 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda1c010790 tx=0x7fda1c010c60 comp rx=0 tx=0).stop 2026-02-17T05:52:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.078+0000 7fda308ed640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9fc03df00 0x7fd9fc0403c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda20007e50 tx=0x7fda20007da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda33379640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:52:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda33379640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1228b0 0x7fda2c1d8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda33379640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c135ad0 con 0x7fda2c1228b0 2026-02-17T05:52:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1228b0 0x7fda2c1d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 10 monclient: get_auth_request con 0x7fda2c1228b0 auth_method 0 2026-02-17T05:52:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 10 monclient(hunting): handle_auth_done global_id 94110 payload 824 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.081+0000 7fda310ee640 1 -- 10.20.193.78:0/298145187 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fda2c134d60 con 0x7fda2c1228b0 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda310ee640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1228b0 msgr2=0x7fda2c1d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda310ee640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1228b0 0x7fda2c1d8540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda1c010790 tx=0x7fda1c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c01a030 con 0x7fda2c1228b0 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 10 monclient: dump: 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.100 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7fda1c015670 con 0x7fda2c1228b0 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1228b0 msgr2=0x7fda2c1d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1228b0 0x7fda2c1d8540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda1c010790 tx=0x7fda1c004830 comp rx=0 tx=0).stop 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1228b0 msgr2=0x7fda2c1d8540 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:45.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda19ffb640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda2c1228b0 0x7fda2c1d8540 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda1c010790 tx=0x7fda1c004830 comp rx=0 tx=0).stop 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 74, 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 25658, 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:52:45.102 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.103 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.104 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:52:45.105 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.106 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.107 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.108 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.109 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.110 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.111 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:52:45.112 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9fc03df00 msgr2=0x7fd9fc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda33379640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9fc03df00 0x7fd9fc0403c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda20007e50 tx=0x7fda20007da0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda33379640 10 monclient: shutdown 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda33379640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c09c4a0 msgr2=0x7fda2c1217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda33379640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c09c4a0 0x7fda2c1217d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda28036000 tx=0x7fda28004b40 comp rx=0 tx=0).stop 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.083+0000 7fda318ef640 1 -- 10.20.193.78:0/298145187 reap_dead start 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.084+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 shutdown_connections 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.084+0000 7fda33379640 1 --2- 10.20.193.78:0/298145187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2c09c4a0 0x7fda2c1217d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.084+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 >> 10.20.193.78:0/298145187 conn(0x7fda2c0a6620 msgr2=0x7fda2c071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.084+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 shutdown_connections 2026-02-17T05:52:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.084+0000 7fda33379640 1 -- 10.20.193.78:0/298145187 wait complete. 2026-02-17T05:52:45.114 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:52:45.114 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:52:45.114 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.181+0000 7ff69b561640 10 monclient: get_monmap_and_config 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.181+0000 7ff69b561640 10 monclient: build_initial_monmap 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.181+0000 7ff69b561640 10 monclient: monmap: 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:45.182030+0000 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:45.182030+0000 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:45.180 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:45.181 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:45.181 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:45.181 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 1 Processor -- start 2026-02-17T05:52:45.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 1 -- start start 2026-02-17T05:52:45.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient: init 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff69406c890 0x7ff69406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient(hunting): picked mon.noname-c con 0x7ff69406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff69409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff69406d1b0 0x7ff694135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient(hunting): picked mon.noname-a con 0x7ff69406d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69406daf0 con 0x7ff69409c4a0 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff6940a0880 con 0x7ff69406d1b0 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff69409cc80 con 0x7ff69406c890 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:45.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff69b561640 10 monclient(hunting): authenticate will time out at 1010.735565s 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff69406d1b0 0x7ff694135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff69406d1b0 0x7ff694135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51524/0 (socket says 10.20.193.78:51524) 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 1 -- 10.20.193.78:0/2227182462 learned_addr learned my addr 10.20.193.78:0/2227182462 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient(hunting): get_auth_request con 0x7ff69406d1b0 auth_method 0 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff6992d6640 1 --2- 10.20.193.78:0/2227182462 >> v2:10.20.193.115:3300/0 conn(0x7ff69406c890 0x7ff69406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff6992d6640 10 monclient(hunting): get_auth_request con 0x7ff69406c890 auth_method 0 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff6992d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff6992d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff6992d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient(hunting): handle_auth_done global_id 94113 payload 274 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 1 -- 10.20.193.78:0/2227182462 >> v2:10.20.193.115:3300/0 conn(0x7ff69406c890 msgr2=0x7ff69406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 1 --2- 10.20.193.78:0/2227182462 >> v2:10.20.193.115:3300/0 conn(0x7ff69406c890 0x7ff69406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 1 -- 10.20.193.78:0/2227182462 >> v1:10.20.193.115:6789/0 conn(0x7ff69409c4a0 legacy=0x7ff694134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient: _finish_hunting 0 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 1 monclient: found mon.noname-a 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.182+0000 7ff699ad7640 1 -- 10.20.193.78:0/2227182462 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69409cf60 con 0x7ff69406d1b0 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff699ad7640 1 -- 10.20.193.78:0/2227182462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 msgr2=0x7ff694135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff699ad7640 1 --2- 10.20.193.78:0/2227182462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff694135500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff69406a480 tx=0x7ff69002f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bae4d9af21a86ea server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 1 -- 10.20.193.78:0/2227182462 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69002fe60 con 0x7ff69406d1b0 2026-02-17T05:52:45.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: dump: 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: _finish_auth 0 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: _check_auth_tickets 2026-02-17T05:52:45.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 1 -- 10.20.193.78:0/2227182462 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6900373f0 con 0x7ff69406d1b0 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 1 -- 10.20.193.78:0/2227182462 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900377c0 con 0x7ff69406d1b0 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 5 monclient: authenticate success, global_id 94113 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff683fff640 10 monclient: dump: 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.191 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 4 set_mon_vals no callback set 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 10 monclient: shutdown 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 1 -- 10.20.193.78:0/2227182462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 msgr2=0x7ff694135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 1 --2- 10.20.193.78:0/2227182462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff694135500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff69406a480 tx=0x7ff69002f230 comp rx=0 tx=0).stop 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 1 -- 10.20.193.78:0/2227182462 shutdown_connections 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 1 --2- 10.20.193.78:0/2227182462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff694135500 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 1 --2- 10.20.193.78:0/2227182462 >> v2:10.20.193.115:3300/0 conn(0x7ff69406c890 0x7ff69406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 1 -- 10.20.193.78:0/2227182462 >> 10.20.193.78:0/2227182462 conn(0x7ff6940a6620 msgr2=0x7ff6940a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 1 -- 10.20.193.78:0/2227182462 shutdown_connections 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.185+0000 7ff69b561640 1 -- 10.20.193.78:0/2227182462 wait complete. 2026-02-17T05:52:45.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient: build_initial_monmap 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient: monmap: 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 1 Processor -- start 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 1 -- start start 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient: init 2026-02-17T05:52:45.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff69406c890 0x7ff6941cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient(hunting): picked mon.noname-b con 0x7ff69406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff6941cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient(hunting): picked mon.noname-a con 0x7ff69406d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff69409c4a0 0x7ff694123560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient(hunting): picked mon.noname-c con 0x7ff69409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6940a01a0 con 0x7ff69406d1b0 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6940a0020 con 0x7ff69406c890 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff69409be20 con 0x7ff69409c4a0 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:45.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff69b561640 10 monclient(hunting): authenticate will time out at 1010.739553s 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff6941cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff6941cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51530/0 (socket says 10.20.193.78:51530) 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 -- 10.20.193.78:0/2594509941 learned_addr learned my addr 10.20.193.78:0/2594509941 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient(hunting): get_auth_request con 0x7ff69406d1b0 auth_method 0 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff699ad7640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff69409c4a0 0x7ff694123560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff699ad7640 10 monclient(hunting): get_auth_request con 0x7ff69409c4a0 auth_method 0 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff699ad7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff699ad7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff699ad7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff699ad7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff699ad7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff699ad7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff6992d6640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff69406c890 0x7ff6941cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff6992d6640 10 monclient(hunting): get_auth_request con 0x7ff69406c890 auth_method 0 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff6992d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff6992d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff6992d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient(hunting): handle_auth_done global_id 94116 payload 824 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff69409c4a0 msgr2=0x7ff694123560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff69409c4a0 0x7ff694123560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff69406c890 msgr2=0x7ff6941cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff69406c890 0x7ff6941cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient: _finish_hunting 0 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 monclient: found mon.noname-a 2026-02-17T05:52:45.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:45.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.186+0000 7ff698ad5640 1 -- 10.20.193.78:0/2594509941 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff694126eb0 con 0x7ff69406d1b0 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.188+0000 7ff698ad5640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 msgr2=0x7ff6941cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.188+0000 7ff698ad5640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff6941cd870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff68800c5d0 tx=0x7ff68800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.188+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688016020 con 0x7ff69406d1b0 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.188+0000 7ff681ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.188+0000 7ff681ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.188+0000 7ff681ffb640 10 monclient: dump: 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.201 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 10 monclient: _finish_auth 0 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff688012a60 con 0x7ff69406d1b0 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 5 monclient: authenticate success, global_id 94116 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688012e50 con 0x7ff69406d1b0 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:45.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6837fe640 4 set_mon_vals no callback set 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 10 monclient: dump: 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 10 monclient: _renew_subs 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff694125c70 con 0x7ff69406d1b0 2026-02-17T05:52:45.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 10 monclient: _renew_subs 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69406fe70 con 0x7ff69406d1b0 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124260 0x7ff69411ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff69409d870 con 0x7ff694124260 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124260 0x7ff69411ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 10 monclient: get_auth_request con 0x7ff694124260 auth_method 0 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff68801b400 con 0x7ff69406d1b0 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 10 monclient(hunting): handle_auth_done global_id 74309 payload 824 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff6992d6640 1 -- 10.20.193.78:0/2594509941 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff694074980 con 0x7ff694124260 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff66403e050 0x7ff664040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.189+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(256..256 src has 117..256) ==== 4779+0+0 (secure 0 0 0) 0x7ff688052870 con 0x7ff69406d1b0 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.190+0000 7ff699ad7640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff66403e050 0x7ff664040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.190+0000 7ff699ad7640 10 monclient: get_auth_request con 0x7ff66403e050 auth_method 0 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.191+0000 7ff6992d6640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124260 msgr2=0x7ff69411ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.191+0000 7ff6992d6640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124260 0x7ff69411ec80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff68400f440 tx=0x7ff68400f910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684018020 con 0x7ff694124260 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 10 monclient: dump: 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.205 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff684014b60 con 0x7ff694124260 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124260 msgr2=0x7ff69411ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124260 0x7ff69411ec80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff68400f440 tx=0x7ff68400f910 comp rx=0 tx=0).stop 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124260 msgr2=0x7ff69411ec80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:45.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff681ffb640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124260 0x7ff69411ec80 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff68400f440 tx=0x7ff68400f910 comp rx=0 tx=0).stop 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.192+0000 7ff699ad7640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff66403e050 0x7ff664040510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff69409c980 tx=0x7ff690033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff69b561640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff69b561640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124640 0x7ff6941d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff69b561640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff69406cc70 con 0x7ff694124640 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124640 0x7ff6941d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 10 monclient: get_auth_request con 0x7ff694124640 auth_method 0 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 10 monclient(hunting): handle_auth_done global_id 74312 payload 824 2026-02-17T05:52:45.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.195+0000 7ff6992d6640 1 -- 10.20.193.78:0/2594509941 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff694120340 con 0x7ff694124640 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff6992d6640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124640 msgr2=0x7ff6941d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff6992d6640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124640 0x7ff6941d8330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff68400f440 tx=0x7ff6840047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684018070 con 0x7ff694124640 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 10 monclient: dump: 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.208 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7ff684029620 con 0x7ff694124640 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124640 msgr2=0x7ff6941d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124640 0x7ff6941d8330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff68400f440 tx=0x7ff6840047c0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124640 msgr2=0x7ff6941d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.197+0000 7ff681ffb640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff694124640 0x7ff6941d8330 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff68400f440 tx=0x7ff6840047c0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:52:45.209 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 74, 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 137089, 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:52:45.210 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.211 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:52:45.212 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.213 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.214 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.215 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.217 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.218 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:52:45.219 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff66403e050 msgr2=0x7ff664040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff66403e050 0x7ff664040510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff69409c980 tx=0x7ff690033000 comp rx=0 tx=0).stop 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 10 monclient: shutdown 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 msgr2=0x7ff6941cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff6941cd870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff68800c5d0 tx=0x7ff68800caa0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff699ad7640 1 -- 10.20.193.78:0/2594509941 reap_dead start 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 shutdown_connections 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 --2- 10.20.193.78:0/2594509941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff69406d1b0 0x7ff6941cd870 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 >> 10.20.193.78:0/2594509941 conn(0x7ff6940a6620 msgr2=0x7ff694071280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:45.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 shutdown_connections 2026-02-17T05:52:45.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.198+0000 7ff69b561640 1 -- 10.20.193.78:0/2594509941 wait complete. 2026-02-17T05:52:45.221 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:52:45.221 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:52:45.221 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.289+0000 7f343165b640 10 monclient: get_monmap_and_config 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.289+0000 7f343165b640 10 monclient: build_initial_monmap 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient: monmap: 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:45.290216+0000 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:45.290216+0000 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:45.288 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 1 Processor -- start 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 1 -- start start 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient: init 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f342c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f342c06d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c13d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient(hunting): picked mon.noname-b con 0x7f342c0a01f0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c074150 con 0x7f342c06d380 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c09ef70 con 0x7f342c06c890 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f342c0a0620 con 0x7f342c0a01f0 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:45.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.290+0000 7f343165b640 10 monclient(hunting): authenticate will time out at 1010.843944s 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c13d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342a7fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f342c06d380 0x7f342c13a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46198/0 (socket says 10.20.193.78:46198) 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342a7fc640 1 -- 10.20.193.78:0/738214860 learned_addr learned my addr 10.20.193.78:0/738214860 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient(hunting): get_auth_request con 0x7f342c0a01f0 auth_method 0 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient(hunting): handle_auth_done global_id 74315 payload 274 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 1 -- 10.20.193.78:0/738214860 >> v1:10.20.193.115:6789/0 conn(0x7f342c06c890 legacy=0x7f342c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 1 -- 10.20.193.78:0/738214860 >> v1:10.20.193.78:6789/0 conn(0x7f342c06d380 legacy=0x7f342c13a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 1 monclient: found mon.noname-b 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:52:45.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.291+0000 7f342b7fe640 1 -- 10.20.193.78:0/738214860 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c09f250 con 0x7f342c0a01f0 2026-02-17T05:52:45.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f342b7fe640 1 -- 10.20.193.78:0/738214860 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 msgr2=0x7f342c13d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f342b7fe640 1 --2- 10.20.193.78:0/738214860 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c13d970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f342c06a480 tx=0x7f341c02f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c31c5e7beaa62a4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 1 -- 10.20.193.78:0/738214860 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c03c070 con 0x7f342c0a01f0 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: dump: 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: _finish_auth 0 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 1 -- 10.20.193.78:0/738214860 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c037640 con 0x7f342c0a01f0 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f343165b640 5 monclient: authenticate success, global_id 74315 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 1 -- 10.20.193.78:0/738214860 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c037a10 con 0x7f342c0a01f0 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f3429ffb640 10 monclient: dump: 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.292 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f343165b640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:45.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.293+0000 7f343165b640 4 set_mon_vals no callback set 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 10 monclient: shutdown 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 -- 10.20.193.78:0/738214860 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 msgr2=0x7f342c13d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 --2- 10.20.193.78:0/738214860 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c13d970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f342c06a480 tx=0x7f341c02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 -- 10.20.193.78:0/738214860 shutdown_connections 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 --2- 10.20.193.78:0/738214860 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c13d970 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 -- 10.20.193.78:0/738214860 >> 10.20.193.78:0/738214860 conn(0x7f342c0a6620 msgr2=0x7f342c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 -- 10.20.193.78:0/738214860 shutdown_connections 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 -- 10.20.193.78:0/738214860 wait complete. 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 10 monclient: build_initial_monmap 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 10 monclient: monmap: 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 Processor -- start 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 1 -- start start 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.294+0000 7f343165b640 10 monclient: init 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f342c06c890 0x7f342c135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient(hunting): picked mon.noname-a con 0x7f342c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c06d380 0x7f342c135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient(hunting): picked mon.noname-b con 0x7f342c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c1cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient(hunting): picked mon.noname-c con 0x7f342c0a01f0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f342c09d360 con 0x7f342c06c890 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f342c09d1e0 con 0x7f342c06d380 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f342c13deb0 con 0x7f342c0a01f0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f343165b640 10 monclient(hunting): authenticate will time out at 1010.848223s 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c1cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f342c06c890 0x7f342c135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c1cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48934/0 (socket says 10.20.193.78:48934) 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 -- 10.20.193.78:0/2923658839 learned_addr learned my addr 10.20.193.78:0/2923658839 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient(hunting): get_auth_request con 0x7f342c0a01f0 auth_method 0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342affd640 10 monclient(hunting): get_auth_request con 0x7f342c06c890 auth_method 0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342a7fc640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c06d380 0x7f342c135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342a7fc640 10 monclient(hunting): get_auth_request con 0x7f342c06d380 auth_method 0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient(hunting): handle_auth_done global_id 74318 payload 824 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c06d380 msgr2=0x7f342c135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c06d380 0x7f342c135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f342c06c890 msgr2=0x7f342c135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f342c06c890 0x7f342c135560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 monclient: found mon.noname-c 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:45.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.295+0000 7f342b7fe640 1 -- 10.20.193.78:0/2923658839 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c06fc90 con 0x7f342c0a01f0 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f342b7fe640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 msgr2=0x7f342c1cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f342b7fe640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c1cf1f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f341c009d90 tx=0x7f341c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c047020 con 0x7f342c0a01f0 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: dump: 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f343165b640 5 monclient: authenticate success, global_id 74318 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c042680 con 0x7f342c0a01f0 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f343165b640 10 monclient: _renew_subs 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f343165b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c073200 con 0x7f342c0a01f0 2026-02-17T05:52:45.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c03c040 con 0x7f342c0a01f0 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f34297fa640 4 set_mon_vals no callback set 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f340b7fe640 10 monclient: dump: 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f343165b640 10 monclient: _renew_subs 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f343165b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.297+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c09d810 con 0x7f342c0a01f0 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f343165b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f343165b640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c121660 0x7f342c138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f343165b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f342c06cc70 con 0x7f342c121660 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f341c042af0 con 0x7f342c0a01f0 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c121660 0x7f342c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 10 monclient: get_auth_request con 0x7f342c121660 auth_method 0 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f340b7fe640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33f803dee0 0x7f33f80403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(256..256 src has 117..256) ==== 4779+0+0 (secure 0 0 0) 0x7f341c0783f0 con 0x7f342c0a01f0 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 10 monclient(hunting): handle_auth_done global_id 44494 payload 824 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342affd640 1 -- 10.20.193.78:0/2923658839 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f342c13dbd0 con 0x7f342c121660 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342a7fc640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33f803dee0 0x7f33f80403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.298+0000 7f342a7fc640 10 monclient: get_auth_request con 0x7f33f803dee0 auth_method 0 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f342affd640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c121660 msgr2=0x7f342c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f342affd640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c121660 0x7f342c138230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f342000f770 tx=0x7f342000fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420018030 con 0x7f342c121660 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 10 monclient: dump: 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f342a7fc640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33f803dee0 0x7f33f80403a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f342c121230 tx=0x7f3414009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f342000dbc0 con 0x7f342c121660 2026-02-17T05:52:45.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:45.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c121660 msgr2=0x7f342c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c121660 0x7f342c138230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f342000f770 tx=0x7f342000fc40 comp rx=0 tx=0).stop 2026-02-17T05:52:45.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c121660 msgr2=0x7f342c138230 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:45.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.301+0000 7f340b7fe640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c121660 0x7f342c138230 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f342000f770 tx=0x7f342000fc40 comp rx=0 tx=0).stop 2026-02-17T05:52:45.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.304+0000 7f343165b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:52:45.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.304+0000 7f343165b640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c138770 0x7f342c1da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.304+0000 7f343165b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.304+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0744d0 con 0x7f342c138770 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c138770 0x7f342c1da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 10 monclient: get_auth_request con 0x7f342c138770 auth_method 0 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 10 monclient(hunting): handle_auth_done global_id 44497 payload 824 2026-02-17T05:52:45.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.305+0000 7f342affd640 1 -- 10.20.193.78:0/2923658839 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f342c06ef90 con 0x7f342c138770 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f342affd640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c138770 msgr2=0x7f342c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f342affd640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c138770 0x7f342c1da560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f342000f770 tx=0x7f3420003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420005260 con 0x7f342c138770 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 10 monclient: dump: 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.306 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f3420005260 con 0x7f342c138770 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c138770 msgr2=0x7f342c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c138770 0x7f342c1da560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f342000f770 tx=0x7f3420003860 comp rx=0 tx=0).stop 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c138770 msgr2=0x7f342c1da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f340b7fe640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f342c138770 0x7f342c1da560 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f342000f770 tx=0x7f3420003860 comp rx=0 tx=0).stop 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:52:45.307 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 74, 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 288954, 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.308 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.309 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:52:45.310 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.311 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:52:45.312 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.313 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:45.314 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.315 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:52:45.316 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:52:45.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33f803dee0 msgr2=0x7f33f80403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f33f803dee0 0x7f33f80403a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f342c121230 tx=0x7f3414009040 comp rx=0 tx=0).stop 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 10 monclient: shutdown 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 msgr2=0x7f342c1cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 1 --2- 10.20.193.78:0/2923658839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f342c0a01f0 0x7f342c1cf1f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f341c009d90 tx=0x7f341c004b40 comp rx=0 tx=0).stop 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f342b7fe640 1 -- 10.20.193.78:0/2923658839 reap_dead start 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f342b7fe640 1 -- 10.20.193.78:0/2923658839 reap_dead start 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 shutdown_connections 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 >> 10.20.193.78:0/2923658839 conn(0x7f342c0a6620 msgr2=0x7f342c09c640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 shutdown_connections 2026-02-17T05:52:45.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.308+0000 7f343165b640 1 -- 10.20.193.78:0/2923658839 wait complete. 2026-02-17T05:52:45.318 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:52:45.318 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:52:45.318 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:52:45.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 10 monclient: get_monmap_and_config 2026-02-17T05:52:45.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 10 monclient: build_initial_monmap 2026-02-17T05:52:45.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 10 monclient: monmap: 2026-02-17T05:52:45.385 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:45.385 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.385 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:45.387367+0000 2026-02-17T05:52:45.385 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:45.387367+0000 2026-02-17T05:52:45.386 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:45.386 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.386 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:45.386 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:45.386 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:45.386 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:45.386 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:45.386 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 1 Processor -- start 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 1 -- start start 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 10 monclient: init 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.387+0000 7fb4f4b87640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb4f006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb4f006d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb4f00a01f0 0x7fb4f013d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 10 monclient(hunting): picked mon.noname-a con 0x7fb4f00a01f0 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f00a0600 con 0x7fb4f006c890 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f0074150 con 0x7fb4f006d380 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f009ef70 con 0x7fb4f00a01f0 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4f4b87640 10 monclient(hunting): authenticate will time out at 1010.941039s 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4ee575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb4f006c890 0x7fb4f006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46212/0 (socket says 10.20.193.78:46212) 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4ee575640 1 -- 10.20.193.78:0/1265686748 learned_addr learned my addr 10.20.193.78:0/1265686748 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 1 --2- 10.20.193.78:0/1265686748 >> v2:10.20.193.78:3300/0 conn(0x7fb4f00a01f0 0x7fb4f013d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient(hunting): get_auth_request con 0x7fb4f00a01f0 auth_method 0 2026-02-17T05:52:45.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient(hunting): handle_auth_done global_id 94122 payload 274 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 1 -- 10.20.193.78:0/1265686748 >> v1:10.20.193.115:6789/0 conn(0x7fb4f006d380 legacy=0x7fb4f013a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 1 -- 10.20.193.78:0/1265686748 >> v1:10.20.193.78:6789/0 conn(0x7fb4f006c890 legacy=0x7fb4f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient: _finish_hunting 0 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 1 monclient: found mon.noname-a 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.388+0000 7fb4eed76640 1 -- 10.20.193.78:0/1265686748 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f009f250 con 0x7fb4f00a01f0 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.390+0000 7fb4eed76640 1 -- 10.20.193.78:0/1265686748 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f00a01f0 msgr2=0x7fb4f013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.390+0000 7fb4eed76640 1 --2- 10.20.193.78:0/1265686748 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f00a01f0 0x7fb4f013d970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009d90 tx=0x7fb4e002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c45db0c6b813ed server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.390+0000 7fb4ed573640 1 -- 10.20.193.78:0/1265686748 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e003c070 con 0x7fb4f00a01f0 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.390+0000 7fb4ed573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.390+0000 7fb4ed573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.390+0000 7fb4ed573640 10 monclient: dump: 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.392 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.390+0000 7fb4ed573640 10 monclient: _finish_auth 0 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.390+0000 7fb4ed573640 10 monclient: _check_auth_tickets 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4ed573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4ed573640 1 -- 10.20.193.78:0/1265686748 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e0037640 con 0x7fb4f00a01f0 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 5 monclient: authenticate success, global_id 94122 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4ed573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4ed573640 1 -- 10.20.193.78:0/1265686748 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0037a10 con 0x7fb4f00a01f0 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 4 set_mon_vals no callback set 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4ed573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4ed573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4ed573640 10 monclient: dump: 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.393 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 10 monclient: shutdown 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1265686748 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f00a01f0 msgr2=0x7fb4f013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 1 --2- 10.20.193.78:0/1265686748 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f00a01f0 0x7fb4f013d970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009d90 tx=0x7fb4e002f1e0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1265686748 shutdown_connections 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 1 --2- 10.20.193.78:0/1265686748 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f00a01f0 0x7fb4f013d970 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1265686748 >> 10.20.193.78:0/1265686748 conn(0x7fb4f00a6620 msgr2=0x7fb4f00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1265686748 shutdown_connections 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1265686748 wait complete. 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 10 monclient: build_initial_monmap 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 10 monclient: monmap: 2026-02-17T05:52:45.394 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.391+0000 7fb4f4b87640 1 Processor -- start 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 1 -- start start 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient: init 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f006c890 0x7fb4f013ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient(hunting): picked mon.noname-a con 0x7fb4f006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4f006d380 0x7fb4f013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient(hunting): picked mon.noname-b con 0x7fb4f006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4f00a01f0 0x7fb4f0140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient(hunting): picked mon.noname-c con 0x7fb4f00a01f0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f009d360 con 0x7fb4f006c890 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f009d1e0 con 0x7fb4f006d380 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f009be20 con 0x7fb4f00a01f0 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4f4b87640 10 monclient(hunting): authenticate will time out at 1010.945289s 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f006c890 0x7fb4f013ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f006c890 0x7fb4f013ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51564/0 (socket says 10.20.193.78:51564) 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 -- 10.20.193.78:0/1093741732 learned_addr learned my addr 10.20.193.78:0/1093741732 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient(hunting): get_auth_request con 0x7fb4f006c890 auth_method 0 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4eed76640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4f00a01f0 0x7fb4f0140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4eed76640 10 monclient(hunting): get_auth_request con 0x7fb4f00a01f0 auth_method 0 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4eed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4eed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4eed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:45.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4eed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4eed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4eed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient(hunting): handle_auth_done global_id 94125 payload 824 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 -- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4f00a01f0 msgr2=0x7fb4f0140440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4f00a01f0 0x7fb4f0140440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4edd74640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4f006d380 0x7fb4f013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 -- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4f006d380 msgr2=0x7fb4f013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4f006d380 0x7fb4f013ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient: _finish_hunting 0 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 monclient: found mon.noname-a 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.392+0000 7fb4ee575640 1 -- 10.20.193.78:0/1093741732 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f01347d0 con 0x7fb4f006c890 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4ee575640 1 -- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f006c890 msgr2=0x7fb4f013ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4ee575640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f006c890 0x7fb4f013ec40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400c750 tx=0x7fb4e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 1 -- 10.20.193.78:0/1093741732 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4005de0 con 0x7fb4f006c890 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: dump: 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.399 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 1 -- 10.20.193.78:0/1093741732 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e4007070 con 0x7fb4f006c890 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4f4b87640 5 monclient: authenticate success, global_id 94125 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 1 -- 10.20.193.78:0/1093741732 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4007440 con 0x7fb4f006c890 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4f4b87640 10 monclient: _renew_subs 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4ecd72640 4 set_mon_vals no callback set 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4f4b87640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f0134ab0 con 0x7fb4f006c890 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4d6ffd640 10 monclient: dump: 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:45.400 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4f4b87640 10 monclient: _renew_subs 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4f4b87640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.394+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f009d810 con 0x7fb4f006c890 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.395+0000 7fb4f4b87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.395+0000 7fb4f4b87640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.395+0000 7fb4f4b87640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.395+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f013f180 con 0x7fb4f006c890 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.395+0000 7fb4d6ffd640 1 -- 10.20.193.78:0/1093741732 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb4e4007660 con 0x7fb4f006c890 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.395+0000 7fb4d6ffd640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb4bc03df00 0x7fb4bc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.395+0000 7fb4d6ffd640 1 -- 10.20.193.78:0/1093741732 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(256..256 src has 117..256) ==== 4779+0+0 (secure 0 0 0) 0x7fb4e4052570 con 0x7fb4f006c890 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.396+0000 7fb4edd74640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb4bc03df00 0x7fb4bc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.396+0000 7fb4edd74640 10 monclient: get_auth_request con 0x7fb4bc03df00 auth_method 0 2026-02-17T05:52:45.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.398+0000 7fb4d6ffd640 1 -- 10.20.193.78:0/1093741732 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb4e400a8d0 con 0x7fb4f006c890 2026-02-17T05:52:45.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.398+0000 7fb4d6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:45.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.398+0000 7fb4d6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:45.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.398+0000 7fb4edd74640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb4bc03df00 0x7fb4bc0403c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0134140 tx=0x7fb4d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:45.402 INFO:tasks.rados.rados.0.trial115.stdout:1208: done (2 left) 2026-02-17T05:52:45.402 INFO:tasks.rados.rados.0.trial115.stdout:1209: done (1 left) 2026-02-17T05:52:45.402 INFO:tasks.rados.rados.0.trial115.stdout:1210: done (0 left) 2026-02-17T05:52:45.402 INFO:tasks.rados.rados.0.trial115.stdout:1211: copy_from oid 18 from oid 15 current snap is 121 2026-02-17T05:52:45.402 INFO:tasks.rados.rados.0.trial115.stdout:1212: snap_remove snap 120 2026-02-17T05:52:45.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.514+0000 7fb4f4b87640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:52:45.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.514+0000 7fb4f4b87640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:52:45.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.514+0000 7fb4f4b87640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:45.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.514+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fb4f006cc70 con 0x7fb4f006c890 2026-02-17T05:52:45.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.514+0000 7fb4d6ffd640 1 -- 10.20.193.78:0/1093741732 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fb4e400a5e0 con 0x7fb4f006c890 2026-02-17T05:52:45.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.514+0000 7fb4d6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:52:45.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.514+0000 7fb4d6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:45.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb4bc03df00 msgr2=0x7fb4bc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb4bc03df00 0x7fb4bc0403c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0134140 tx=0x7fb4d8009040 comp rx=0 tx=0).stop 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 10 monclient: shutdown 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f006c890 msgr2=0x7fb4f013ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f006c890 0x7fb4f013ec40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400c750 tx=0x7fb4e400cc20 comp rx=0 tx=0).stop 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 shutdown_connections 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4f00a01f0 0x7fb4f0140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb4bc03df00 0x7fb4bc0403c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4f006d380 0x7fb4f013ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 --2- 10.20.193.78:0/1093741732 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4f006c890 0x7fb4f013ec40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 >> 10.20.193.78:0/1093741732 conn(0x7fb4f00a6620 msgr2=0x7fb4f00a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 shutdown_connections 2026-02-17T05:52:45.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:45.515+0000 7fb4f4b87640 1 -- 10.20.193.78:0/1093741732 wait complete. 2026-02-17T05:52:45.522 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:52:45.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.653+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:45.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.653+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=931580521,config=692619927,health=2319137676,kv=270111327,logm=2045786660}) 2026-02-17T05:52:45.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.653+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=2,health=10,kv=12,logm=61} crc {auth=412194174,config=692619927,health=1237543367,kv=449989585,logm=2115777912}) 2026-02-17T05:52:45.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.653+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:45.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.653+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=931580521,config=692619927,health=2319137676,kv=270111327,logm=2045786660}) 2026-02-17T05:52:45.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.653+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=12,kv=12,logm=60} crc {auth=1571468930,config=692619927,health=2319137676,kv=449989585,logm=3468094584}) 2026-02-17T05:52:45.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.660+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:45.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.660+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3730809240}) 2026-02-17T05:52:45.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.660+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2681457755}) 2026-02-17T05:52:45.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.660+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:45.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.660+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3730809240}) 2026-02-17T05:52:45.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:45.660+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2840330464}) 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.245+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.245+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3910350056}) 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.245+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2817421802}) 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.245+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.245+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3910350056}) 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.245+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3423869955}) 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624929732}) 2026-02-17T05:52:46.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2889344001}) 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=624929732}) 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3015175856}) 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3677445714}) 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451066287}) 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3677445714}) 2026-02-17T05:52:46.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.246+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1795495497}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.253+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.253+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1181534827}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.253+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=139903553}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.253+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.253+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1181534827}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.253+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1007900038}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2282974322}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2690507952}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2282974322}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3646486820}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=716273294}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3047511792}) 2026-02-17T05:52:46.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=716273294}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.254+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=517831015}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3414043168}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2549729740}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3414043168}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=996071860}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1297326172}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3267432631}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1297326172}) 2026-02-17T05:52:46.253 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3031951562}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2390008445}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1752949334}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2390008445}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.255+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=433167730}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3753046783}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3055651760}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3753046783}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=103458000}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3919661266}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1320185240}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3919661266}) 2026-02-17T05:52:46.254 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1563184580}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1426343045}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3856308069}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1426343045}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.256+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2634824173}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2759779752}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3223288362}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2759779752}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1953651172}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859343170}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3259895665}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859343170}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2159217475}) 2026-02-17T05:52:46.255 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=43502998}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3927801845}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=43502998}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.257+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=802228417}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=12,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=14,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=41} crc {logm=2229639490,mdsmap=1160443688,mgr=1039804757,mgr_metadata=1258482571,mgrstat=3152945090,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2240493509,osd_pg_creating=1778302969,osd_snap=2967925479}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42} crc {logm=2558359866,mdsmap=1160443688,mgr=787438293,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2757186642,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=1778302969,osd_snap=4256334838}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=12,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=14,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=41} crc {logm=2229639490,mdsmap=1160443688,mgr=1039804757,mgr_metadata=1258482571,mgrstat=3152945090,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2240493509,osd_pg_creating=1778302969,osd_snap=2967925479}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=12,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39} crc {logm=2229639490,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2145724076,mon_config_key=2209207285,monmap=1784110042,osd_metadata=2655843256,osd_pg_creating=1778302969,osd_snap=2029165178}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=3,osdmap=97} crc {osd_snap=2888533665,osdmap=194213783}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=3,osdmap=97} crc {osd_snap=2888533665,osdmap=44867322}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=3,osdmap=97} crc {osd_snap=2888533665,osdmap=194213783}) 2026-02-17T05:52:46.256 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.258+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=3,osdmap=97} crc {osd_snap=2888533665,osdmap=3805640877}) 2026-02-17T05:52:46.257 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.259+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.257 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.259+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3629425425}) 2026-02-17T05:52:46.257 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.259+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2275315432}) 2026-02-17T05:52:46.257 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.259+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.257 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.259+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3629425425}) 2026-02-17T05:52:46.257 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.259+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=737757408}) 2026-02-17T05:52:46.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.261+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.261+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=21} crc {osdmap=1650529333}) 2026-02-17T05:52:46.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.261+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=20} crc {osdmap=2361963870}) 2026-02-17T05:52:46.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.261+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:52:46.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.261+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=21} crc {osdmap=1650529333}) 2026-02-17T05:52:46.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:52:46.261+0000 7fa3afe48640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=17} crc {osdmap=60205304}) 2026-02-17T05:52:46.523 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:52:46.523 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:46.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.591+0000 7fc73e9fb640 10 monclient: get_monmap_and_config 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.591+0000 7fc73e9fb640 10 monclient: build_initial_monmap 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.591+0000 7fc73e9fb640 10 monclient: monmap: 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:46.591822+0000 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:46.591822+0000 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:46.590 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 1 Processor -- start 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 1 -- start start 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient: init 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc73809ea90 0x7fc7380a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient(hunting): picked mon.noname-a con 0x7fc73809ea90 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc73809ee70 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:46.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73813d640 0x7fc73813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient(hunting): picked mon.noname-b con 0x7fc73813d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7380a0a20 con 0x7fc73809ee70 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc7380a0d00 con 0x7fc73809ea90 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc7380a0460 con 0x7fc73813d640 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73e9fb640 10 monclient(hunting): authenticate will time out at 1012.145521s 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73813d640 0x7fc73813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc737fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc73809ea90 0x7fc7380a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73813d640 0x7fc73813fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48956/0 (socket says 10.20.193.78:48956) 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 1 -- 10.20.193.78:0/3547040650 learned_addr learned my addr 10.20.193.78:0/3547040650 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient(hunting): get_auth_request con 0x7fc73813d640 auth_method 0 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc737fff640 10 monclient(hunting): get_auth_request con 0x7fc73809ea90 auth_method 0 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc737fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc737fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc737fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc737fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc737fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc737fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient(hunting): handle_auth_done global_id 74324 payload 274 2026-02-17T05:52:46.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 1 -- 10.20.193.78:0/3547040650 >> v2:10.20.193.78:3300/0 conn(0x7fc73809ea90 msgr2=0x7fc7380a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:46.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 1 --2- 10.20.193.78:0/3547040650 >> v2:10.20.193.78:3300/0 conn(0x7fc73809ea90 0x7fc7380a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 1 -- 10.20.193.78:0/3547040650 >> v1:10.20.193.115:6789/0 conn(0x7fc73809ee70 legacy=0x7fc73809f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:46.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient: _finish_hunting 0 2026-02-17T05:52:46.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 1 monclient: found mon.noname-b 2026-02-17T05:52:46.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:46.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.592+0000 7fc73cf71640 1 -- 10.20.193.78:0/3547040650 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7380a0740 con 0x7fc73813d640 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73cf71640 1 -- 10.20.193.78:0/3547040650 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73813d640 msgr2=0x7fc73813fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73cf71640 1 --2- 10.20.193.78:0/3547040650 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73813d640 0x7fc73813fa30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009e50 tx=0x7fc72c02f260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e4de5b5df16f02 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 1 -- 10.20.193.78:0/3547040650 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0373f0 con 0x7fc73813d640 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: dump: 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: _finish_auth 0 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 1 -- 10.20.193.78:0/3547040650 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c037590 con 0x7fc73813d640 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 1 -- 10.20.193.78:0/3547040650 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c037960 con 0x7fc73813d640 2026-02-17T05:52:46.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 5 monclient: authenticate success, global_id 74324 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 4 set_mon_vals no callback set 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc736ffd640 10 monclient: dump: 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 10 monclient: shutdown 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 1 -- 10.20.193.78:0/3547040650 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73813d640 msgr2=0x7fc73813fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/3547040650 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73813d640 0x7fc73813fa30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009e50 tx=0x7fc72c02f260 comp rx=0 tx=0).stop 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 1 -- 10.20.193.78:0/3547040650 shutdown_connections 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/3547040650 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73813d640 0x7fc73813fa30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/3547040650 >> v2:10.20.193.78:3300/0 conn(0x7fc73809ea90 0x7fc7380a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 1 -- 10.20.193.78:0/3547040650 >> 10.20.193.78:0/3547040650 conn(0x7fc7380a7520 msgr2=0x7fc7380a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 1 -- 10.20.193.78:0/3547040650 shutdown_connections 2026-02-17T05:52:46.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.595+0000 7fc73e9fb640 1 -- 10.20.193.78:0/3547040650 wait complete. 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient: build_initial_monmap 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient: monmap: 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 1 Processor -- start 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 1 -- start start 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient: init 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc73809ea90 0x7fc7381cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient(hunting): picked mon.noname-b con 0x7fc73809ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73809ee70 0x7fc7380736b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient(hunting): picked mon.noname-c con 0x7fc73809ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc73813d640 0x7fc738073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient(hunting): picked mon.noname-a con 0x7fc73813d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7380a6820 con 0x7fc73813d640 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7380a66a0 con 0x7fc73809ea90 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc73813ff70 con 0x7fc73809ee70 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73e9fb640 10 monclient(hunting): authenticate will time out at 1012.149727s 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73809ee70 0x7fc7380736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73809ee70 0x7fc7380736b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48960/0 (socket says 10.20.193.78:48960) 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 -- 10.20.193.78:0/2490455689 learned_addr learned my addr 10.20.193.78:0/2490455689 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient(hunting): get_auth_request con 0x7fc73809ee70 auth_method 0 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73cf71640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc73813d640 0x7fc738073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73cf71640 10 monclient(hunting): get_auth_request con 0x7fc73813d640 auth_method 0 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73cf71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73cf71640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73cf71640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73cf71640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73cf71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc73cf71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient(hunting): handle_auth_done global_id 74327 payload 824 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 -- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc73809ea90 msgr2=0x7fc7381cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc73809ea90 0x7fc7381cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 -- 10.20.193.78:0/2490455689 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc73813d640 msgr2=0x7fc738073bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc73813d640 0x7fc738073bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient: _finish_hunting 0 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 monclient: found mon.noname-c 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.596+0000 7fc7377fe640 1 -- 10.20.193.78:0/2490455689 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7380a9a70 con 0x7fc73809ee70 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc7377fe640 1 -- 10.20.193.78:0/2490455689 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73809ee70 msgr2=0x7fc7380736b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc7377fe640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73809ee70 0x7fc7380736b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c600 tx=0x7fc72800cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 1 -- 10.20.193.78:0/2490455689 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728012a40 con 0x7fc73809ee70 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: dump: 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.599 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: _finish_auth 0 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 1 -- 10.20.193.78:0/2490455689 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc728017070 con 0x7fc73809ee70 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 1 -- 10.20.193.78:0/2490455689 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728012e20 con 0x7fc73809ee70 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc7367fc640 4 set_mon_vals no callback set 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 10 monclient: dump: 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 5 monclient: authenticate success, global_id 74327 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 10 monclient: _renew_subs 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7380a6070 con 0x7fc73809ee70 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 10 monclient: _renew_subs 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7380708d0 con 0x7fc73809ee70 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7380a13d0 con 0x7fc73809ee70 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.599+0000 7fc734ff9640 1 -- 10.20.193.78:0/2490455689 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc72801c460 con 0x7fc73809ee70 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.600+0000 7fc734ff9640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc70803deb0 0x7fc708040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.600+0000 7fc734ff9640 1 -- 10.20.193.78:0/2490455689 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(258..258 src has 117..258) ==== 4795+0+0 (secure 0 0 0) 0x7fc728052650 con 0x7fc73809ee70 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.600+0000 7fc737fff640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc70803deb0 0x7fc708040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.600+0000 7fc737fff640 10 monclient: get_auth_request con 0x7fc70803deb0 auth_method 0 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.602+0000 7fc734ff9640 1 -- 10.20.193.78:0/2490455689 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc7380a9a70 con 0x7fc73809ee70 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.602+0000 7fc734ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.602+0000 7fc734ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:46.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.602+0000 7fc737fff640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc70803deb0 0x7fc708040370 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc724006b90 tx=0x7fc724006b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.633 INFO:tasks.rados.rados.0.trial115.stdout:1212: done (1 left) 2026-02-17T05:52:46.633 INFO:tasks.rados.rados.0.trial115.stdout:1213: write oid 31 current snap is 121 2026-02-17T05:52:46.633 INFO:tasks.rados.rados.0.trial115.stdout:1213: seq_num 470 ranges {699290=464563,1603478=535155,2823983=765846} 2026-02-17T05:52:46.638 INFO:tasks.rados.rados.0.trial115.stdout:1213: writing trial11545420-31 from 699290 to 1163853 tid 1 2026-02-17T05:52:46.641 INFO:tasks.rados.rados.0.trial115.stdout:1213: writing trial11545420-31 from 1603478 to 2138633 tid 2 2026-02-17T05:52:46.647 INFO:tasks.rados.rados.0.trial115.stdout:1213: writing trial11545420-31 from 2823983 to 3589829 tid 3 2026-02-17T05:52:46.648 INFO:tasks.rados.rados.0.trial115.stdout:1214: copy_from oid 6 from oid 46 current snap is 121 2026-02-17T05:52:46.648 INFO:tasks.rados.rados.0.trial115.stdout:1211: finishing copy_from racing read to trial11545420-18 2026-02-17T05:52:46.648 INFO:tasks.rados.rados.0.trial115.stdout:1211: finishing copy_from to trial11545420-18 2026-02-17T05:52:46.648 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 519 (ObjNum 450 snap 112 seq_num 450) dirty exists 2026-02-17T05:52:46.648 INFO:tasks.rados.rados.0.trial115.stdout:1213: finishing write tid 1 to trial11545420-31 2026-02-17T05:52:46.648 INFO:tasks.rados.rados.0.trial115.stdout:1213: finishing write tid 2 to trial11545420-31 2026-02-17T05:52:46.648 INFO:tasks.rados.rados.0.trial115.stdout:1211: done (2 left) 2026-02-17T05:52:46.648 INFO:tasks.rados.rados.0.trial115.stdout:1215: snap_remove snap 80 2026-02-17T05:52:46.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.714+0000 7fc73e9fb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:46.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.714+0000 7fc73e9fb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:46.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.714+0000 7fc73e9fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.714+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc7380710d0 con 0x7fc73809ee70 2026-02-17T05:52:46.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.715+0000 7fc734ff9640 1 -- 10.20.193.78:0/2490455689 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fc72801ac60 con 0x7fc73809ee70 2026-02-17T05:52:46.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.715+0000 7fc734ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:46.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.715+0000 7fc734ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:46.713 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":74,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc70803deb0 msgr2=0x7fc708040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc70803deb0 0x7fc708040370 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc724006b90 tx=0x7fc724006b20 comp rx=0 tx=0).stop 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 10 monclient: shutdown 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73809ee70 msgr2=0x7fc7380736b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73809ee70 0x7fc7380736b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c600 tx=0x7fc72800cad0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 shutdown_connections 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc73813d640 0x7fc738073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc73809ee70 0x7fc7380736b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc70803deb0 0x7fc708040370 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 --2- 10.20.193.78:0/2490455689 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc73809ea90 0x7fc7381cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 >> 10.20.193.78:0/2490455689 conn(0x7fc7380a7520 msgr2=0x7fc738136cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 shutdown_connections 2026-02-17T05:52:46.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.716+0000 7fc73e9fb640 1 -- 10.20.193.78:0/2490455689 wait complete. 2026-02-17T05:52:46.721 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:52:46.721 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:52:46.721 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:52:46.722 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:52:46.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 10 monclient: get_monmap_and_config 2026-02-17T05:52:46.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 10 monclient: build_initial_monmap 2026-02-17T05:52:46.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 10 monclient: monmap: 2026-02-17T05:52:46.789 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:46.789 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.789 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:46.791495+0000 2026-02-17T05:52:46.789 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:46.791495+0000 2026-02-17T05:52:46.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:46.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.790 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:46.790 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:46.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:46.790 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:46.790 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:46.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 1 Processor -- start 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 1 -- start start 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 10 monclient: init 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5ce406c890 0x7f5ce406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.791+0000 7f5cec200640 10 monclient(hunting): picked mon.noname-c con 0x7f5ce406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5ce409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5ce406d1b0 0x7f5ce4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 10 monclient(hunting): picked mon.noname-a con 0x7f5ce406d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce406daf0 con 0x7f5ce409c4a0 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce40a0880 con 0x7f5ce406d1b0 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce409cc80 con 0x7f5ce406c890 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cec200640 10 monclient(hunting): authenticate will time out at 1012.345060s 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5ce406d1b0 0x7f5ce4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5ce406d1b0 0x7f5ce4135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51590/0 (socket says 10.20.193.78:51590) 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 1 -- 10.20.193.78:0/2118964771 learned_addr learned my addr 10.20.193.78:0/2118964771 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient(hunting): get_auth_request con 0x7f5ce406d1b0 auth_method 0 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5ce9f75640 1 --2- 10.20.193.78:0/2118964771 >> v2:10.20.193.115:3300/0 conn(0x7f5ce406c890 0x7f5ce406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5ce9f75640 10 monclient(hunting): get_auth_request con 0x7f5ce406c890 auth_method 0 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5ce9f75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5ce9f75640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5ce9f75640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient(hunting): handle_auth_done global_id 94134 payload 274 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 1 -- 10.20.193.78:0/2118964771 >> v2:10.20.193.115:3300/0 conn(0x7f5ce406c890 msgr2=0x7f5ce406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 1 --2- 10.20.193.78:0/2118964771 >> v2:10.20.193.115:3300/0 conn(0x7f5ce406c890 0x7f5ce406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 1 -- 10.20.193.78:0/2118964771 >> v1:10.20.193.115:6789/0 conn(0x7f5ce409c4a0 legacy=0x7f5ce4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient: _finish_hunting 0 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 1 monclient: found mon.noname-a 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:52:46.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.792+0000 7f5cea776640 1 -- 10.20.193.78:0/2118964771 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce409cf60 con 0x7f5ce406d1b0 2026-02-17T05:52:46.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5cea776640 1 -- 10.20.193.78:0/2118964771 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 msgr2=0x7f5ce4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5cea776640 1 --2- 10.20.193.78:0/2118964771 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce4135500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ce406a480 tx=0x7f5ce002f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f14a785cdda40a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 1 -- 10.20.193.78:0/2118964771 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce002fe60 con 0x7f5ce406d1b0 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: dump: 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: _finish_auth 0 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: _check_auth_tickets 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 1 -- 10.20.193.78:0/2118964771 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce00373f0 con 0x7f5ce406d1b0 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5cec200640 5 monclient: authenticate success, global_id 94134 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5cec200640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 1 -- 10.20.193.78:0/2118964771 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00377c0 con 0x7f5ce406d1b0 2026-02-17T05:52:46.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5ce8f73640 10 monclient: dump: 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5cec200640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.794+0000 7f5cec200640 4 set_mon_vals no callback set 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 10 monclient: shutdown 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 1 -- 10.20.193.78:0/2118964771 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 msgr2=0x7f5ce4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 1 --2- 10.20.193.78:0/2118964771 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce4135500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ce406a480 tx=0x7f5ce002f230 comp rx=0 tx=0).stop 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 1 -- 10.20.193.78:0/2118964771 shutdown_connections 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 1 --2- 10.20.193.78:0/2118964771 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce4135500 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 1 --2- 10.20.193.78:0/2118964771 >> v2:10.20.193.115:3300/0 conn(0x7f5ce406c890 0x7f5ce406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 1 -- 10.20.193.78:0/2118964771 >> 10.20.193.78:0/2118964771 conn(0x7f5ce40a6620 msgr2=0x7f5ce40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:46.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 1 -- 10.20.193.78:0/2118964771 shutdown_connections 2026-02-17T05:52:46.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 1 -- 10.20.193.78:0/2118964771 wait complete. 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 10 monclient: build_initial_monmap 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.795+0000 7f5cec200640 10 monclient: monmap: 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 1 Processor -- start 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 1 -- start start 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient: init 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ce406c890 0x7f5ce41cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient(hunting): picked mon.noname-c con 0x7f5ce406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:46.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce41cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient(hunting): picked mon.noname-a con 0x7f5ce406d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ce409c4a0 0x7f5ce409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient(hunting): picked mon.noname-b con 0x7f5ce409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce40a01a0 con 0x7f5ce406d1b0 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce40a0020 con 0x7f5ce409c4a0 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce409be20 con 0x7f5ce406c890 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cec200640 10 monclient(hunting): authenticate will time out at 1012.349403s 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce41cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce41cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51598/0 (socket says 10.20.193.78:51598) 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 -- 10.20.193.78:0/363264 learned_addr learned my addr 10.20.193.78:0/363264 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient(hunting): get_auth_request con 0x7f5ce406d1b0 auth_method 0 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9f75640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ce406c890 0x7f5ce41cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9f75640 10 monclient(hunting): get_auth_request con 0x7f5ce406c890 auth_method 0 2026-02-17T05:52:46.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9f75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9f75640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9f75640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cea776640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ce409c4a0 0x7f5ce409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient(hunting): handle_auth_done global_id 94137 payload 824 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ce406c890 msgr2=0x7f5ce41cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ce406c890 0x7f5ce41cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ce409c4a0 msgr2=0x7f5ce409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ce409c4a0 0x7f5ce409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient: _finish_hunting 0 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 monclient: found mon.noname-a 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5ce9774640 1 -- 10.20.193.78:0/363264 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce4136ea0 con 0x7f5ce406d1b0 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cea776640 10 monclient: get_auth_request con 0x7f5ce409c4a0 auth_method 0 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.796+0000 7f5cea776640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ce409c4a0 0x7f5ce409db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5ce9774640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 msgr2=0x7f5ce41cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5ce9774640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce41cf890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800c5d0 tx=0x7f5cd800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8016020 con 0x7f5ce406d1b0 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:52:46.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: dump: 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: _finish_auth 0 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cec200640 5 monclient: authenticate success, global_id 94137 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd8012a60 con 0x7f5ce406d1b0 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cec200640 10 monclient: _renew_subs 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cec200640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce406fdd0 con 0x7f5ce406d1b0 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8012e50 con 0x7f5ce406d1b0 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd3fff640 4 set_mon_vals no callback set 2026-02-17T05:52:46.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cd27fc640 10 monclient: dump: 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cec200640 10 monclient: _renew_subs 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cec200640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.798+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce409f7a0 con 0x7f5ce406d1b0 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5cec200640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5cec200640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409e850 0x7f5ce411ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5cec200640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4074980 con 0x7f5ce409e850 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409e850 0x7f5ce411ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 10 monclient: get_auth_request con 0x7f5ce409e850 auth_method 0 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_done global_id 94140 payload 824 2026-02-17T05:52:46.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9774640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce41cf890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800c5d0 tx=0x7f5cd800caa0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f5cd80529a0 osd_map(258..258 src has 117..258) 2026-02-17T05:52:46.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5ce9f75640 1 -- 10.20.193.78:0/363264 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ce406cc70 con 0x7f5ce409e850 2026-02-17T05:52:46.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5cd801b400 con 0x7f5ce406d1b0 2026-02-17T05:52:46.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5cd27fc640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cb403e050 0x7f5cb4040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.799+0000 7f5cea776640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cb403e050 0x7f5cb4040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.800+0000 7f5cea776640 10 monclient: get_auth_request con 0x7f5cb403e050 auth_method 0 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.801+0000 7f5ce9f75640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409e850 msgr2=0x7f5ce411ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.801+0000 7f5ce9f75640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409e850 0x7f5ce411ec80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4011830 tx=0x7f5cd4011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.801+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400f040 con 0x7f5ce409e850 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.801+0000 7f5cd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.801+0000 7f5cd27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.801+0000 7f5cd27fc640 10 monclient: dump: 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.805 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.802+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f5cd400fe50 con 0x7f5ce409e850 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.802+0000 7f5cd27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.802+0000 7f5cd27fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.802+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409e850 msgr2=0x7f5ce411ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.802+0000 7f5cd27fc640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409e850 0x7f5ce411ec80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4011830 tx=0x7f5cd4011d00 comp rx=0 tx=0).stop 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.802+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409e850 msgr2=0x7f5ce411ec80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.802+0000 7f5cd27fc640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409e850 0x7f5ce411ec80 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4011830 tx=0x7f5cd4011d00 comp rx=0 tx=0).stop 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.802+0000 7f5cea776640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cb403e050 0x7f5cb4040510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ce409c980 tx=0x7f5ce0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5cec200640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5cec200640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409ec30 0x7f5ce41da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5cec200640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4136b70 con 0x7f5ce409ec30 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409ec30 0x7f5ce41da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 10 monclient: get_auth_request con 0x7f5ce409ec30 auth_method 0 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 10 monclient(hunting): handle_auth_done global_id 94143 payload 824 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.805+0000 7f5ce9f75640 1 -- 10.20.193.78:0/363264 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5ce4120340 con 0x7f5ce409ec30 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5ce9f75640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409ec30 msgr2=0x7f5ce41da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5ce9f75640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409ec30 0x7f5ce41da560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4011830 tx=0x7f5cd4004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4015660 con 0x7f5ce409ec30 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 10 monclient: dump: 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f5cd4015660 con 0x7f5ce409ec30 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409ec30 msgr2=0x7f5ce41da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409ec30 0x7f5ce41da560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4011830 tx=0x7f5cd4004830 comp rx=0 tx=0).stop 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409ec30 msgr2=0x7f5ce41da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:46.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce409ec30 0x7f5ce41da560 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4011830 tx=0x7f5cd4004830 comp rx=0 tx=0).stop 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 74, 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 27382, 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.809 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.810 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.811 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:46.812 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.813 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:52:46.814 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cb403e050 msgr2=0x7f5cb4040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cec200640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cb403e050 0x7f5cb4040510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ce409c980 tx=0x7f5ce0033000 comp rx=0 tx=0).stop 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cec200640 10 monclient: shutdown 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cec200640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 msgr2=0x7f5ce41cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cec200640 1 --2- 10.20.193.78:0/363264 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ce406d1b0 0x7f5ce41cf890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800c5d0 tx=0x7f5cd800caa0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cea776640 1 -- 10.20.193.78:0/363264 reap_dead start 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cd27fc640 1 -- 10.20.193.78:0/363264 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(258..258 src has 117..258) ==== 4795+0+0 (unknown 0 0 0) 0x7f5cd80529a0 con 0x7f5ce406d1b0 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cea776640 1 -- 10.20.193.78:0/363264 reap_dead start 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 shutdown_connections 2026-02-17T05:52:46.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.807+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 >> 10.20.193.78:0/363264 conn(0x7f5ce40a6620 msgr2=0x7f5ce4072240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:46.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.808+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 shutdown_connections 2026-02-17T05:52:46.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.808+0000 7f5cec200640 1 -- 10.20.193.78:0/363264 wait complete. 2026-02-17T05:52:46.816 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:52:46.816 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:52:46.816 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:52:46.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.883+0000 7efcb0832640 10 monclient: get_monmap_and_config 2026-02-17T05:52:46.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.883+0000 7efcb0832640 10 monclient: build_initial_monmap 2026-02-17T05:52:46.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.883+0000 7efcb0832640 10 monclient: monmap: 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:46.883628+0000 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:46.883628+0000 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:46.882 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.883+0000 7efcb0832640 1 Processor -- start 2026-02-17T05:52:46.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.883+0000 7efcb0832640 1 -- start start 2026-02-17T05:52:46.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.883+0000 7efcb0832640 10 monclient: init 2026-02-17T05:52:46.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:46.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2026-02-17T05:52:46.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efca806c890 0x7efca806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient(hunting): picked mon.noname-a con 0x7efca806c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7efca809c4a0 0x7efca806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient(hunting): picked mon.noname-c con 0x7efca809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8134df0 0x7efca8135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient(hunting): picked mon.noname-b con 0x7efca8134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7efca80a0880 con 0x7efca806c890 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7efca806d6f0 con 0x7efca809c4a0 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efca806daf0 con 0x7efca8134df0 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcb0832640 10 monclient(hunting): authenticate will time out at 1012.437220s 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efca806c890 0x7efca806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8134df0 0x7efca8135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efca806c890 0x7efca806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51634/0 (socket says 10.20.193.78:51634) 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 1 -- 10.20.193.78:0/3178139119 learned_addr learned my addr 10.20.193.78:0/3178139119 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 10 monclient(hunting): get_auth_request con 0x7efca806c890 auth_method 0 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient(hunting): get_auth_request con 0x7efca8134df0 auth_method 0 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcadda6640 1 --2- 10.20.193.78:0/3178139119 >> v2:10.20.193.115:3300/0 conn(0x7efca809c4a0 0x7efca806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcae5a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcadda6640 10 monclient(hunting): get_auth_request con 0x7efca809c4a0 auth_method 0 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcadda6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcadda6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcadda6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient(hunting): handle_auth_done global_id 74333 payload 274 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 1 -- 10.20.193.78:0/3178139119 >> v2:10.20.193.115:3300/0 conn(0x7efca809c4a0 msgr2=0x7efca806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 1 --2- 10.20.193.78:0/3178139119 >> v2:10.20.193.115:3300/0 conn(0x7efca809c4a0 0x7efca806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 1 -- 10.20.193.78:0/3178139119 >> v2:10.20.193.78:3300/0 conn(0x7efca806c890 msgr2=0x7efca806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 1 --2- 10.20.193.78:0/3178139119 >> v2:10.20.193.78:3300/0 conn(0x7efca806c890 0x7efca806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient: _finish_hunting 0 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 1 monclient: found mon.noname-b 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.884+0000 7efcaeda8640 1 -- 10.20.193.78:0/3178139119 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca809cc80 con 0x7efca8134df0 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.886+0000 7efcaeda8640 1 -- 10.20.193.78:0/3178139119 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8134df0 msgr2=0x7efca8135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.886+0000 7efcaeda8640 1 --2- 10.20.193.78:0/3178139119 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8134df0 0x7efca8135280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efca809c980 tx=0x7efc9c02f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc2a6e609b00a9d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 1 -- 10.20.193.78:0/3178139119 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c03c070 con 0x7efca8134df0 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: dump: 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.887 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: _finish_auth 0 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: _check_auth_tickets 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 1 -- 10.20.193.78:0/3178139119 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc9c037440 con 0x7efca8134df0 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 5 monclient: authenticate success, global_id 74333 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 1 -- 10.20.193.78:0/3178139119 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c037810 con 0x7efca8134df0 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 4 set_mon_vals no callback set 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcad5a5640 10 monclient: dump: 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 10 monclient: shutdown 2026-02-17T05:52:46.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 -- 10.20.193.78:0/3178139119 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8134df0 msgr2=0x7efca8135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 --2- 10.20.193.78:0/3178139119 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8134df0 0x7efca8135280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efca809c980 tx=0x7efc9c02f130 comp rx=0 tx=0).stop 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 -- 10.20.193.78:0/3178139119 shutdown_connections 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 --2- 10.20.193.78:0/3178139119 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8134df0 0x7efca8135280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 --2- 10.20.193.78:0/3178139119 >> v2:10.20.193.115:3300/0 conn(0x7efca809c4a0 0x7efca806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 --2- 10.20.193.78:0/3178139119 >> v2:10.20.193.78:3300/0 conn(0x7efca806c890 0x7efca806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 -- 10.20.193.78:0/3178139119 >> 10.20.193.78:0/3178139119 conn(0x7efca80a6620 msgr2=0x7efca80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 -- 10.20.193.78:0/3178139119 shutdown_connections 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 1 -- 10.20.193.78:0/3178139119 wait complete. 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 10 monclient: build_initial_monmap 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.887+0000 7efcb0832640 10 monclient: monmap: 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 1 Processor -- start 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 1 -- start start 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient: init 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca806c890 0x7efca81cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient(hunting): picked mon.noname-c con 0x7efca806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:46.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efca809c4a0 0x7efca81cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient(hunting): picked mon.noname-b con 0x7efca809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efca8134df0 0x7efca809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient(hunting): picked mon.noname-a con 0x7efca8134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7efca80a01a0 con 0x7efca8134df0 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7efca80a0020 con 0x7efca809c4a0 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efca809be20 con 0x7efca806c890 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcb0832640 10 monclient(hunting): authenticate will time out at 1012.441395s 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca806c890 0x7efca81cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efca8134df0 0x7efca809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca806c890 0x7efca81cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48998/0 (socket says 10.20.193.78:48998) 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 -- 10.20.193.78:0/1558682464 learned_addr learned my addr 10.20.193.78:0/1558682464 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient(hunting): get_auth_request con 0x7efca806c890 auth_method 0 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 10 monclient(hunting): get_auth_request con 0x7efca8134df0 auth_method 0 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient(hunting): handle_auth_done global_id 74336 payload 824 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efca809c4a0 msgr2=0x7efca81cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efca809c4a0 0x7efca81cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efca8134df0 msgr2=0x7efca809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efca8134df0 0x7efca809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient: _finish_hunting 0 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 monclient: found mon.noname-c 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcae5a7640 1 -- 10.20.193.78:0/1558682464 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca809d810 con 0x7efca806c890 2026-02-17T05:52:46.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.888+0000 7efcaeda8640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efca8134df0 0x7efca809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:52:46.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcae5a7640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca806c890 msgr2=0x7efca81cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcae5a7640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca806c890 0x7efca81cf350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efca000e8c0 tx=0x7efca000ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0017070 con 0x7efca806c890 2026-02-17T05:52:46.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: dump: 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: _finish_auth 0 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca000cc10 con 0x7efca806c890 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0013470 con 0x7efca806c890 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcacda4640 4 set_mon_vals no callback set 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efc92ffd640 10 monclient: dump: 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.893 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcb0832640 5 monclient: authenticate success, global_id 74336 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcb0832640 10 monclient: _renew_subs 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcb0832640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca81344d0 con 0x7efca806c890 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcb0832640 10 monclient: _renew_subs 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcb0832640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.890+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efca80a1580 con 0x7efca806c890 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcb0832640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcb0832640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8127130 0x7efca809e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcb0832640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efca809f850 con 0x7efca8127130 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8127130 0x7efca809e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 10 monclient: get_auth_request con 0x7efca8127130 auth_method 0 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 10 monclient(hunting): handle_auth_done global_id 74339 payload 824 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efcadda6640 1 -- 10.20.193.78:0/1558682464 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efca8074980 con 0x7efca8127130 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7efca0024060 con 0x7efca806c890 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efc92ffd640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc7803df00 0x7efc780403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.891+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(258..258 src has 117..258) ==== 4795+0+0 (secure 0 0 0) 0x7efca0052860 con 0x7efca806c890 2026-02-17T05:52:46.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.892+0000 7efcaeda8640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc7803df00 0x7efc780403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.892+0000 7efcaeda8640 10 monclient: get_auth_request con 0x7efc7803df00 auth_method 0 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.893+0000 7efcadda6640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8127130 msgr2=0x7efca809e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.893+0000 7efcadda6640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8127130 0x7efca809e850 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efca809f2d0 tx=0x7efc9400dba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.893+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94018020 con 0x7efca8127130 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.893+0000 7efc92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.893+0000 7efc92ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.893+0000 7efc92ffd640 10 monclient: dump: 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.894+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7efc9400b940 con 0x7efca8127130 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.894+0000 7efc92ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.894+0000 7efc92ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.894+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8127130 msgr2=0x7efca809e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.894+0000 7efc92ffd640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8127130 0x7efca809e850 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efca809f2d0 tx=0x7efc9400dba0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.894+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8127130 msgr2=0x7efca809e850 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.894+0000 7efc92ffd640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca8127130 0x7efca809e850 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efca809f2d0 tx=0x7efc9400dba0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.894+0000 7efcaeda8640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc7803df00 0x7efc780403c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efca806a480 tx=0x7efc9c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcb0832640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcb0832640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca809ed90 0x7efca81da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcb0832640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efca8135280 con 0x7efca809ed90 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca809ed90 0x7efca81da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 10 monclient: get_auth_request con 0x7efca809ed90 auth_method 0 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 10 monclient(hunting): handle_auth_done global_id 74342 payload 824 2026-02-17T05:52:46.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.897+0000 7efcadda6640 1 -- 10.20.193.78:0/1558682464 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efca8125dc0 con 0x7efca809ed90 2026-02-17T05:52:46.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcadda6640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca809ed90 msgr2=0x7efca81da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcadda6640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca809ed90 0x7efca81da560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc94009fd0 tx=0x7efc940047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94018040 con 0x7efca809ed90 2026-02-17T05:52:46.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:46.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 10 monclient: dump: 2026-02-17T05:52:46.897 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7efc9401dde0 con 0x7efca809ed90 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca809ed90 msgr2=0x7efca81da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca809ed90 0x7efca81da560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc94009fd0 tx=0x7efc940047c0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca809ed90 msgr2=0x7efca81da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:46.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efc92ffd640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca809ed90 0x7efca81da560 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc94009fd0 tx=0x7efc940047c0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 74, 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 138791, 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.899 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:52:46.900 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:52:46.901 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:46.902 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:46.903 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:52:46.904 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc7803df00 msgr2=0x7efc780403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcb0832640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc7803df00 0x7efc780403c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efca806a480 tx=0x7efc9c033000 comp rx=0 tx=0).stop 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcb0832640 10 monclient: shutdown 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcb0832640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca806c890 msgr2=0x7efca81cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcb0832640 1 --2- 10.20.193.78:0/1558682464 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efca806c890 0x7efca81cf350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efca000e8c0 tx=0x7efca000ed90 comp rx=0 tx=0).stop 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.899+0000 7efcaeda8640 1 -- 10.20.193.78:0/1558682464 reap_dead start 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.900+0000 7efcaeda8640 1 -- 10.20.193.78:0/1558682464 reap_dead start 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.900+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 shutdown_connections 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.900+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 >> 10.20.193.78:0/1558682464 conn(0x7efca80a6620 msgr2=0x7efca80a1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.900+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 shutdown_connections 2026-02-17T05:52:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.900+0000 7efcb0832640 1 -- 10.20.193.78:0/1558682464 wait complete. 2026-02-17T05:52:46.906 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:52:46.906 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:52:46.906 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:52:46.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.973+0000 7f07508a6640 10 monclient: get_monmap_and_config 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.973+0000 7f07508a6640 10 monclient: build_initial_monmap 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.973+0000 7f07508a6640 10 monclient: monmap: 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:46.973817+0000 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:46.973817+0000 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 1 Processor -- start 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 1 -- start start 2026-02-17T05:52:46.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient: init 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f074c06c890 0x7f074c0a0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient(hunting): picked mon.noname-c con 0x7f074c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f074c06cc70 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f074c13a0a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f074c09dd50 con 0x7f074c06cc70 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f074c09dad0 con 0x7f074c13a0a0 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f074c0741c0 con 0x7f074c06c890 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f07508a6640 10 monclient(hunting): authenticate will time out at 1012.527405s 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074a575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f074c06cc70 0x7f074c06d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46224/0 (socket says 10.20.193.78:46224) 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074a575640 1 -- 10.20.193.78:0/1093238478 learned_addr learned my addr 10.20.193.78:0/1093238478 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 1 --2- 10.20.193.78:0/1093238478 >> v2:10.20.193.115:3300/0 conn(0x7f074c06c890 0x7f074c0a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient(hunting): get_auth_request con 0x7f074c06c890 auth_method 0 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient(hunting): handle_auth_done global_id 44506 payload 274 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 1 -- 10.20.193.78:0/1093238478 >> v1:10.20.193.115:6789/0 conn(0x7f074c13a0a0 legacy=0x7f074c13c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 1 -- 10.20.193.78:0/1093238478 >> v1:10.20.193.78:6789/0 conn(0x7f074c06cc70 legacy=0x7f074c06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient: _finish_hunting 0 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 1 monclient: found mon.noname-c 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:52:46.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.974+0000 7f074ad76640 1 -- 10.20.193.78:0/1093238478 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c09edf0 con 0x7f074c06c890 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f074ad76640 1 -- 10.20.193.78:0/1093238478 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06c890 msgr2=0x7f074c0a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f074ad76640 1 --2- 10.20.193.78:0/1093238478 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06c890 0x7f074c0a0070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f07400098a0 tx=0x7f074002ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87583f6ac7f82aa8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 1 -- 10.20.193.78:0/1093238478 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074003c070 con 0x7f074c06c890 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: dump: 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: _finish_auth 0 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: _check_auth_tickets 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 1 -- 10.20.193.78:0/1093238478 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074002fce0 con 0x7f074c06c890 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 5 monclient: authenticate success, global_id 44506 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 1 -- 10.20.193.78:0/1093238478 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740037660 con 0x7f074c06c890 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 4 set_mon_vals no callback set 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f0749d74640 10 monclient: dump: 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.982 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 10 monclient: shutdown 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 1 -- 10.20.193.78:0/1093238478 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06c890 msgr2=0x7f074c0a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 1 --2- 10.20.193.78:0/1093238478 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06c890 0x7f074c0a0070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f07400098a0 tx=0x7f074002ed80 comp rx=0 tx=0).stop 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 1 -- 10.20.193.78:0/1093238478 shutdown_connections 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 1 --2- 10.20.193.78:0/1093238478 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06c890 0x7f074c0a0070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 1 -- 10.20.193.78:0/1093238478 >> 10.20.193.78:0/1093238478 conn(0x7f074c0a65f0 msgr2=0x7f074c0a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 1 -- 10.20.193.78:0/1093238478 shutdown_connections 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.977+0000 7f07508a6640 1 -- 10.20.193.78:0/1093238478 wait complete. 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient: build_initial_monmap 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient: monmap: 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:46.983 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 1 Processor -- start 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 1 -- start start 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient: init 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f074c06c890 0x7f074c1cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient(hunting): picked mon.noname-c con 0x7f074c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06cc70 0x7f074c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient(hunting): picked mon.noname-b con 0x7f074c06cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f074c13a0a0 0x7f074c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient(hunting): picked mon.noname-a con 0x7f074c13a0a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f074c09bfa0 con 0x7f074c13a0a0 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f074c09be20 con 0x7f074c06cc70 2026-02-17T05:52:46.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f074c09c120 con 0x7f074c06c890 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f07508a6640 10 monclient(hunting): authenticate will time out at 1012.531763s 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f074c13a0a0 0x7f074c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f074c06c890 0x7f074c1cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f074c13a0a0 0x7f074c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51658/0 (socket says 10.20.193.78:51658) 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 1 -- 10.20.193.78:0/152724561 learned_addr learned my addr 10.20.193.78:0/152724561 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 10 monclient(hunting): get_auth_request con 0x7f074c13a0a0 auth_method 0 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074ad76640 10 monclient(hunting): get_auth_request con 0x7f074c06c890 auth_method 0 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074a575640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06cc70 0x7f074c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074a575640 10 monclient(hunting): get_auth_request con 0x7f074c06cc70 auth_method 0 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.978+0000 7f074b577640 10 monclient(hunting): handle_auth_done global_id 94152 payload 824 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f074c06c890 msgr2=0x7f074c1cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f074c06c890 0x7f074c1cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06cc70 msgr2=0x7f074c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c06cc70 0x7f074c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 10 monclient: _finish_hunting 0 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 1 monclient: found mon.noname-a 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.979+0000 7f074b577640 1 -- 10.20.193.78:0/152724561 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c06f880 con 0x7f074c13a0a0 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f074b577640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f074c13a0a0 msgr2=0x7f074c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f074b577640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f074c13a0a0 0x7f074c0735f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f073c00cc40 tx=0x7f073c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c016020 con 0x7f074c13a0a0 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: dump: 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073c012620 con 0x7f074c13a0a0 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c00ce60 con 0x7f074c13a0a0 2026-02-17T05:52:46.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f072b7fe640 10 monclient: dump: 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f0749573640 4 set_mon_vals no callback set 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 5 monclient: authenticate success, global_id 94152 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 10 monclient: _renew_subs 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074c06fb60 con 0x7f074c13a0a0 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 10 monclient: _renew_subs 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:52:46.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074c0a1f70 con 0x7f074c13a0a0 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c132f60 0x7f074c133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.981+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f074c09d660 con 0x7f074c132f60 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c132f60 0x7f074c133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f073c012a50 con 0x7f074c13a0a0 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 10 monclient: get_auth_request con 0x7f074c132f60 auth_method 0 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f072b7fe640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f071803deb0 0x7f0718040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(258..258 src has 117..258) ==== 4795+0+0 (secure 0 0 0) 0x7f073c053270 con 0x7f074c13a0a0 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074a575640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f071803deb0 0x7f0718040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 10 monclient(hunting): handle_auth_done global_id 44512 payload 824 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074ad76640 1 -- 10.20.193.78:0/152724561 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f074c0a3d30 con 0x7f074c132f60 2026-02-17T05:52:46.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.982+0000 7f074a575640 10 monclient: get_auth_request con 0x7f071803deb0 auth_method 0 2026-02-17T05:52:46.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.984+0000 7f074a575640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f071803deb0 0x7f0718040370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f074c09cbf0 tx=0x7f0734009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.984+0000 7f074ad76640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c132f60 msgr2=0x7f074c133340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.984+0000 7f074ad76640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c132f60 0x7f074c133340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0740039000 tx=0x7f07400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.984+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074003c040 con 0x7f074c132f60 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.984+0000 7f072b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 10 monclient: dump: 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f0740037e50 con 0x7f074c132f60 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c132f60 msgr2=0x7f074c133340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c132f60 0x7f074c133340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0740039000 tx=0x7f07400047c0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c132f60 msgr2=0x7f074c133340 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:46.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.985+0000 7f072b7fe640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c132f60 0x7f074c133340 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0740039000 tx=0x7f07400047c0 comp rx=0 tx=0).stop 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f07508a6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f07508a6640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c133880 0x7f074c1d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f07508a6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f074c0a0070 con 0x7f074c133880 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c133880 0x7f074c1d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 10 monclient: get_auth_request con 0x7f074c133880 auth_method 0 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 10 monclient(hunting): handle_auth_done global_id 44515 payload 824 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.988+0000 7f074ad76640 1 -- 10.20.193.78:0/152724561 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f074c1354e0 con 0x7f074c133880 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.990+0000 7f074ad76640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c133880 msgr2=0x7f074c1d81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.990+0000 7f074ad76640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c133880 0x7f074c1d81e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f074002edc0 tx=0x7f0740008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740055070 con 0x7f074c133880 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:46.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 10 monclient: dump: 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f07400513f0 con 0x7f074c133880 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c133880 msgr2=0x7f074c1d81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c133880 0x7f074c1d81e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f074002edc0 tx=0x7f0740008030 comp rx=0 tx=0).stop 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c133880 msgr2=0x7f074c1d81e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:52:46.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f072b7fe640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f074c133880 0x7f074c1d81e0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f074002edc0 tx=0x7f0740008030 comp rx=0 tx=0).stop 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 74, 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 290636, 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.995 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.996 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:52:46.997 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.998 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:46.999 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:52:47.000 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:47.001 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:47.002 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:52:47.003 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:52:47.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f071803deb0 msgr2=0x7f0718040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:47.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f071803deb0 0x7f0718040370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f074c09cbf0 tx=0x7f0734009040 comp rx=0 tx=0).stop 2026-02-17T05:52:47.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 10 monclient: shutdown 2026-02-17T05:52:47.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:47.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f074c13a0a0 msgr2=0x7f074c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:47.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f074c13a0a0 0x7f074c0735f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f073c00cc40 tx=0x7f073c00a430 comp rx=0 tx=0).stop 2026-02-17T05:52:47.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f074b577640 1 -- 10.20.193.78:0/152724561 reap_dead start 2026-02-17T05:52:47.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 shutdown_connections 2026-02-17T05:52:47.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 --2- 10.20.193.78:0/152724561 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f074c13a0a0 0x7f074c0735f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 >> 10.20.193.78:0/152724561 conn(0x7f074c0a65f0 msgr2=0x7f074c0a0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:47.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 shutdown_connections 2026-02-17T05:52:47.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:46.991+0000 7f07508a6640 1 -- 10.20.193.78:0/152724561 wait complete. 2026-02-17T05:52:47.005 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:52:47.005 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:52:47.005 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:52:47.005 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:52:47.005 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:52:47.005 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:52:47.005 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:52:47.013 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:52:47.013 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:52:47.013 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.120+0000 7f6f948b0640 10 monclient: get_monmap_and_config 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.120+0000 7f6f948b0640 10 monclient: build_initial_monmap 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.120+0000 7f6f948b0640 10 monclient: monmap: 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:47.120957+0000 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:47.120957+0000 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:47.119 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 1 Processor -- start 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 1 -- start start 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient: init 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6f8c06c890 0x7f6f8c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient(hunting): picked mon.noname-a con 0x7f6f8c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6f8c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6f8c06d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c135c10 con 0x7f6f8c06d1b0 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c06daf0 con 0x7f6f8c09c4a0 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8c0a0880 con 0x7f6f8c06c890 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f948b0640 10 monclient(hunting): authenticate will time out at 1012.674506s 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f92625640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f6f8c06c890 msgr2=0x7f6f8c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f92625640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6f8c06c890 0x7f6f8c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f92e26640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f6f8c06d1b0 legacy=0x7f6f8c135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f91e24640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f6f8c09c4a0 0x7f6f8c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:55342/0 (socket says 10.20.193.78:55342) 2026-02-17T05:52:47.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.121+0000 7f6f91e24640 1 -- 10.20.193.78:0/1575328324 learned_addr learned my addr 10.20.193.78:0/1575328324 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:52:47.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.123+0000 7f6f91e24640 1 -- 10.20.193.78:0/1575328324 >> v1:10.20.193.115:6789/0 conn(0x7f6f8c09c4a0 legacy=0x7f6f8c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:47.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.123+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1632346976 0 0) 0x7f6f8c06daf0 con 0x7f6f8c09c4a0 2026-02-17T05:52:47.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.123+0000 7f6f91623640 10 monclient(hunting): _negotiate 2026-02-17T05:52:47.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.123+0000 7f6f91623640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:47.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.123+0000 7f6f91623640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:47.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.123+0000 7f6f91623640 10 monclient(hunting): my global_id is 44518 2026-02-17T05:52:47.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f7c003620 con 0x7f6f8c09c4a0 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2253795665 0 0) 0x7f6f7c003620 con 0x7f6f8c09c4a0 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 >> v2:10.20.193.78:3300/0 conn(0x7f6f8c06c890 msgr2=0x7f6f8c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 --2- 10.20.193.78:0/1575328324 >> v2:10.20.193.78:3300/0 conn(0x7f6f8c06c890 0x7f6f8c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 >> v1:10.20.193.78:6789/0 conn(0x7f6f8c06d1b0 legacy=0x7f6f8c135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: _finish_hunting 0 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c09cc80 con 0x7f6f8c09c4a0 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: _finish_auth 0 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: _check_auth_tickets 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f6f78003270 con 0x7f6f8c09c4a0 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: dump: 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:47.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 >> v1:10.20.193.115:6789/0 conn(0x7f6f8c09c4a0 legacy=0x7f6f8c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f7c003ab0 0x7f6f7c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient(hunting): picked mon.c con 0x7f6f7c003ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c004900 0x7f6f7c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient(hunting): picked mon.b con 0x7f6f7c004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f7c005470 0x7f6f7c00bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient(hunting): picked mon.a con 0x7f6f7c005470 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c00c390 con 0x7f6f7c005470 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c00c540 con 0x7f6f7c004900 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7c00c740 con 0x7f6f7c003ab0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91623640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92e26640 1 -- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f7c005470 msgr2=0x7f6f7c00bae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92e26640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f7c005470 0x7f6f7c00bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f7c003ab0 0x7f6f7c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient(hunting): get_auth_request con 0x7f6f7c003ab0 auth_method 0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91e24640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c004900 0x7f6f7c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91e24640 10 monclient(hunting): get_auth_request con 0x7f6f7c004900 auth_method 0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91e24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91e24640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f91e24640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient(hunting): handle_auth_done global_id 44518 payload 293 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 1 -- 10.20.193.78:0/1575328324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c004900 msgr2=0x7f6f7c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c004900 0x7f6f7c004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 1 -- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f7c005470 msgr2=0x7f6f7c00bae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f7c005470 0x7f6f7c00bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient: _finish_hunting 0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 1 monclient: found mon.c 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92e26640 1 -- 10.20.193.78:0/1575328324 reap_dead start 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 1 -- 10.20.193.78:0/1575328324 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c003620 con 0x7f6f7c003ab0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient: _finish_auth 0 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 10 monclient: _check_auth_tickets 2026-02-17T05:52:47.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f92625640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f948b0640 5 monclient: authenticate success, global_id 44518 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.124+0000 7f6f948b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f92625640 1 -- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f7c003ab0 msgr2=0x7f6f7c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f92625640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f7c003ab0 0x7f6f7c003eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009e30 tx=0x7f6f8402f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3ef62ac22d432f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84009a30 con 0x7f6f7c003ab0 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 10 monclient: dump: 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f84003cb0 con 0x7f6f7c003ab0 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:47.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 1 -- 10.20.193.78:0/1575328324 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84003e60 con 0x7f6f7c003ab0 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f91623640 10 monclient: dump: 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 4 set_mon_vals no callback set 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 10 monclient: shutdown 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 1 -- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f7c003ab0 msgr2=0x7f6f7c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f7c003ab0 0x7f6f7c003eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009e30 tx=0x7f6f8402f660 comp rx=0 tx=0).stop 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 1 -- 10.20.193.78:0/1575328324 shutdown_connections 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 1 --2- 10.20.193.78:0/1575328324 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f7c003ab0 0x7f6f7c003eb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 1 -- 10.20.193.78:0/1575328324 >> 10.20.193.78:0/1575328324 conn(0x7f6f8c0a6620 msgr2=0x7f6f8c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 1 -- 10.20.193.78:0/1575328324 shutdown_connections 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 1 -- 10.20.193.78:0/1575328324 wait complete. 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 10 monclient: build_initial_monmap 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.127+0000 7f6f948b0640 10 monclient: monmap: 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 1 Processor -- start 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 1 -- start start 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient: init 2026-02-17T05:52:47.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c003ab0 0x7f6f8c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient(hunting): picked mon.noname-b con 0x7f6f7c003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f8c1cf890 0x7f6f8c09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient(hunting): picked mon.noname-c con 0x7f6f8c1cf890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f8c1cfcd0 0x7f6f8c09e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient(hunting): picked mon.noname-a con 0x7f6f8c1cfcd0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0a0020 con 0x7f6f8c1cfcd0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c06d640 con 0x7f6f7c003ab0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c0a01a0 con 0x7f6f8c1cf890 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f948b0640 10 monclient(hunting): authenticate will time out at 1012.681550s 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f92e26640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f8c1cfcd0 msgr2=0x7f6f8c09e090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f92e26640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f8c1cfcd0 0x7f6f8c09e090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f8c1cf890 0x7f6f8c09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f8c1cf890 0x7f6f8c09db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49044/0 (socket says 10.20.193.78:49044) 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 -- 10.20.193.78:0/1041318821 learned_addr learned my addr 10.20.193.78:0/1041318821 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient(hunting): get_auth_request con 0x7f6f8c1cf890 auth_method 0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient(hunting): handle_auth_done global_id 74348 payload 824 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 -- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c003ab0 msgr2=0x7f6f8c1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f92625640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c003ab0 0x7f6f8c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c003ab0 0x7f6f8c1cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 -- 10.20.193.78:0/1041318821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f8c1cfcd0 msgr2=0x7f6f8c09e090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f8c1cfcd0 0x7f6f8c09e090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient: _finish_hunting 0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 monclient: found mon.noname-c 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f91e24640 1 -- 10.20.193.78:0/1041318821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c134610 con 0x7f6f8c1cf890 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f92625640 10 monclient: get_auth_request con 0x7f6f7c003ab0 auth_method 0 2026-02-17T05:52:47.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.128+0000 7f6f92625640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c003ab0 0x7f6f8c1cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.130+0000 7f6f91e24640 1 -- 10.20.193.78:0/1041318821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f8c1cf890 msgr2=0x7f6f8c09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.130+0000 7f6f91e24640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f8c1cf890 0x7f6f8c09db50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f78007330 tx=0x7f6f78007800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 1 -- 10.20.193.78:0/1041318821 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7800eb00 con 0x7f6f8c1cf890 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: dump: 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: _finish_auth 0 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 1 -- 10.20.193.78:0/1041318821 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f78018070 con 0x7f6f8c1cf890 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 1 -- 10.20.193.78:0/1041318821 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78013420 con 0x7f6f8c1cf890 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 10 monclient: dump: 2026-02-17T05:52:47.129 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f90e22640 4 set_mon_vals no callback set 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 5 monclient: authenticate success, global_id 74348 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 10 monclient: _renew_subs 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c1276c0 con 0x7f6f8c1cf890 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 10 monclient: _renew_subs 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c1273e0 con 0x7f6f8c1cf890 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 1 -- 10.20.193.78:0/1041318821 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6f780135c0 con 0x7f6f8c1cf890 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c09beb0 con 0x7f6f8c1cf890 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f5c03deb0 0x7f6f5c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.131+0000 7f6f76ffd640 1 -- 10.20.193.78:0/1041318821 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(258..258 src has 117..258) ==== 4795+0+0 (secure 0 0 0) 0x7f6f780528b0 con 0x7f6f8c1cf890 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.132+0000 7f6f92625640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f5c03deb0 0x7f6f5c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:47.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.132+0000 7f6f92625640 10 monclient: get_auth_request con 0x7f6f5c03deb0 auth_method 0 2026-02-17T05:52:47.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.134+0000 7f6f76ffd640 1 -- 10.20.193.78:0/1041318821 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6f780512a0 con 0x7f6f8c1cf890 2026-02-17T05:52:47.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.134+0000 7f6f76ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:47.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.134+0000 7f6f76ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:47.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.134+0000 7f6f92625640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f5c03deb0 0x7f6f5c040370 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f84003a80 tx=0x7f6f84005000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:47.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.252+0000 7f6f948b0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:47.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.252+0000 7f6f948b0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:47.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.252+0000 7f6f948b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:47.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.252+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6f8c134da0 con 0x7f6f8c1cf890 2026-02-17T05:52:47.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.252+0000 7f6f76ffd640 1 -- 10.20.193.78:0/1041318821 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f6f7801a310 con 0x7f6f8c1cf890 2026-02-17T05:52:47.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.252+0000 7f6f76ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:47.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.252+0000 7f6f76ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:47.251 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":74,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:47.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f5c03deb0 msgr2=0x7f6f5c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:47.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f5c03deb0 0x7f6f5c040370 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f84003a80 tx=0x7f6f84005000 comp rx=0 tx=0).stop 2026-02-17T05:52:47.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 10 monclient: shutdown 2026-02-17T05:52:47.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:47.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f8c1cf890 msgr2=0x7f6f8c09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f8c1cf890 0x7f6f8c09db50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f78007330 tx=0x7f6f78007800 comp rx=0 tx=0).stop 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 shutdown_connections 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f8c1cfcd0 0x7f6f8c09e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f8c1cf890 0x7f6f8c09db50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f5c03deb0 0x7f6f5c040370 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 --2- 10.20.193.78:0/1041318821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f7c003ab0 0x7f6f8c1cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 >> 10.20.193.78:0/1041318821 conn(0x7f6f8c0a6620 msgr2=0x7f6f8c072d30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 shutdown_connections 2026-02-17T05:52:47.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:47.253+0000 7f6f948b0640 1 -- 10.20.193.78:0/1041318821 wait complete. 2026-02-17T05:52:50.259 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient: get_monmap_and_config 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient: build_initial_monmap 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient: monmap: 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:50.330409+0000 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:50.330409+0000 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:50.328 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:50.329 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:50.329 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:50.329 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:50.329 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:50.329 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:50.329 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 1 Processor -- start 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 1 -- start start 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient: init 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient: _add_conns ranks=[3,0,4,2,1] 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2e5806c890 0x7f2e5806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient(hunting): picked mon.noname-a con 0x7f2e5806c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2e5809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2e5806d1b0 0x7f2e58135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient(hunting): picked mon.noname-c con 0x7f2e5806d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e5806daf0 con 0x7f2e5809c4a0 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f2e580a0880 con 0x7f2e5806c890 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2e5809cc80 con 0x7f2e5806d1b0 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e5e476640 10 monclient(hunting): authenticate will time out at 1015.883918s 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e57fff640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f2e5806c890 msgr2=0x7f2e5806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e577fe640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f2e5809c4a0 legacy=0x7f2e58134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.330+0000 7f2e57fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2e5806c890 0x7f2e5806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:50.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2e5806d1b0 0x7f2e58135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2e5806d1b0 0x7f2e58135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:47010/0 (socket says 10.20.193.78:47010) 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3235825507 learned_addr learned my addr 10.20.193.78:0/3235825507 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient(hunting): get_auth_request con 0x7f2e5806d1b0 auth_method 0 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient(hunting): handle_auth_done global_id 44521 payload 274 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3235825507 >> v2:10.20.193.78:3300/0 conn(0x7f2e5806c890 msgr2=0x7f2e5806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 1 --2- 10.20.193.78:0/3235825507 >> v2:10.20.193.78:3300/0 conn(0x7f2e5806c890 0x7f2e5806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3235825507 >> v1:10.20.193.78:6789/0 conn(0x7f2e5809c4a0 legacy=0x7f2e58134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient: _finish_hunting 0 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 1 monclient: found mon.noname-c 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:52:50.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.331+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3235825507 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5809cf60 con 0x7f2e5806d1b0 2026-02-17T05:52:50.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.333+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3235825507 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 msgr2=0x7f2e58135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:50.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.333+0000 7f2e5c9ec640 1 --2- 10.20.193.78:0/3235825507 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 0x7f2e58135500 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e58074de0 tx=0x7f2e4c02f200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be3ecef5221243cd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:50.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 1 -- 10.20.193.78:0/3235825507 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0373f0 con 0x7f2e5806d1b0 2026-02-17T05:52:50.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:50.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:50.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: dump: 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: _finish_auth 0 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 5 monclient: authenticate success, global_id 44521 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 1 -- 10.20.193.78:0/3235825507 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4c037590 con 0x7f2e5806d1b0 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 1 -- 10.20.193.78:0/3235825507 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c037960 con 0x7f2e5806d1b0 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e56ffd640 10 monclient: dump: 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:50.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 4 set_mon_vals no callback set 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 10 monclient: shutdown 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 1 -- 10.20.193.78:0/3235825507 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 msgr2=0x7f2e58135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3235825507 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 0x7f2e58135500 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e58074de0 tx=0x7f2e4c02f200 comp rx=0 tx=0).stop 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 1 -- 10.20.193.78:0/3235825507 shutdown_connections 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3235825507 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 0x7f2e58135500 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3235825507 >> v2:10.20.193.78:3300/0 conn(0x7f2e5806c890 0x7f2e5806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 1 -- 10.20.193.78:0/3235825507 >> 10.20.193.78:0/3235825507 conn(0x7f2e580a6620 msgr2=0x7f2e580a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 1 -- 10.20.193.78:0/3235825507 shutdown_connections 2026-02-17T05:52:50.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.334+0000 7f2e5e476640 1 -- 10.20.193.78:0/3235825507 wait complete. 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient: build_initial_monmap 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient: monmap: 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:50.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 1 Processor -- start 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 1 -- start start 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient: init 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e5806c890 0x7f2e581212c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient(hunting): picked mon.noname-a con 0x7f2e5806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 0x7f2e58121800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient(hunting): picked mon.noname-b con 0x7f2e5806d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e5809c4a0 0x7f2e58121d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient(hunting): picked mon.noname-c con 0x7f2e5809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e580a01a0 con 0x7f2e5806c890 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2e580a0020 con 0x7f2e5806d1b0 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5809be20 con 0x7f2e5809c4a0 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5e476640 10 monclient(hunting): authenticate will time out at 1015.888621s 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e57fff640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e5806c890 msgr2=0x7f2e581212c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e57fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e5806c890 0x7f2e581212c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e5809c4a0 0x7f2e58121d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e5809c4a0 0x7f2e58121d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56880/0 (socket says 10.20.193.78:56880) 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3637744083 learned_addr learned my addr 10.20.193.78:0/3637744083 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient(hunting): get_auth_request con 0x7f2e5809c4a0 auth_method 0 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:50.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient(hunting): handle_auth_done global_id 74351 payload 824 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 msgr2=0x7f2e58121800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 0x7f2e58121800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3637744083 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e5806c890 msgr2=0x7f2e581212c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e5806c890 0x7f2e581212c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient: _finish_hunting 0 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 monclient: found mon.noname-c 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:50.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.335+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3637744083 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e58125620 con 0x7f2e5809c4a0 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5c9ec640 1 -- 10.20.193.78:0/3637744083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e5809c4a0 msgr2=0x7f2e58121d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5c9ec640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e5809c4a0 0x7f2e58121d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c039040 tx=0x7f2e4c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 1 -- 10.20.193.78:0/3637744083 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c046020 con 0x7f2e5809c4a0 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: dump: 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: _finish_auth 0 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 1 -- 10.20.193.78:0/3637744083 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4c041870 con 0x7f2e5809c4a0 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 5 monclient: authenticate success, global_id 74351 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 10 monclient: _renew_subs 2026-02-17T05:52:50.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e58125200 con 0x7f2e5809c4a0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 1 -- 10.20.193.78:0/3637744083 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c041c60 con 0x7f2e5809c4a0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e567fc640 4 set_mon_vals no callback set 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 10 monclient: dump: 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 10 monclient: _renew_subs 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5806fd70 con 0x7f2e5809c4a0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 1 -- 10.20.193.78:0/3637744083 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2e4c042400 con 0x7f2e5809c4a0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5806cc70 con 0x7f2e5809c4a0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e2803de60 0x7f2e28040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.338+0000 7f2e54ff9640 1 -- 10.20.193.78:0/3637744083 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(258..258 src has 117..258) ==== 4795+0+0 (secure 0 0 0) 0x7f2e4c078920 con 0x7f2e5809c4a0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.339+0000 7f2e57fff640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e2803de60 0x7f2e28040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:50.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.339+0000 7f2e57fff640 10 monclient: get_auth_request con 0x7f2e2803de60 auth_method 0 2026-02-17T05:52:50.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.341+0000 7f2e54ff9640 1 -- 10.20.193.78:0/3637744083 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2e4c050030 con 0x7f2e5809c4a0 2026-02-17T05:52:50.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.341+0000 7f2e54ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:50.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.341+0000 7f2e54ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:50.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.341+0000 7f2e57fff640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e2803de60 0x7f2e28040320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e44003a80 tx=0x7f2e44009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:50.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.453+0000 7f2e5e476640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:50.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.453+0000 7f2e5e476640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.453+0000 7f2e5e476640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.453+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2e58122f10 con 0x7f2e5809c4a0 2026-02-17T05:52:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.453+0000 7f2e54ff9640 1 -- 10.20.193.78:0/3637744083 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f2e58125620 con 0x7f2e5809c4a0 2026-02-17T05:52:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.454+0000 7f2e54ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:50.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.454+0000 7f2e54ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:50.453 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":74,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e2803de60 msgr2=0x7f2e28040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e2803de60 0x7f2e28040320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e44003a80 tx=0x7f2e44009270 comp rx=0 tx=0).stop 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 10 monclient: shutdown 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e5809c4a0 msgr2=0x7f2e58121d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e5809c4a0 0x7f2e58121d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c039040 tx=0x7f2e4c004b40 comp rx=0 tx=0).stop 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 shutdown_connections 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e5809c4a0 0x7f2e58121d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e2803de60 0x7f2e28040320 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e5806d1b0 0x7f2e58121800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 --2- 10.20.193.78:0/3637744083 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e5806c890 0x7f2e581212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:50.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 >> 10.20.193.78:0/3637744083 conn(0x7f2e580a6620 msgr2=0x7f2e58072110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:50.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 shutdown_connections 2026-02-17T05:52:50.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:50.455+0000 7f2e5e476640 1 -- 10.20.193.78:0/3637744083 wait complete. 2026-02-17T05:52:53.462 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.531+0000 7fb4d5691640 10 monclient: get_monmap_and_config 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.531+0000 7fb4d5691640 10 monclient: build_initial_monmap 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.531+0000 7fb4d5691640 10 monclient: monmap: 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:53.531536+0000 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:53.531536+0000 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:53.529 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.531+0000 7fb4d5691640 1 Processor -- start 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.531+0000 7fb4d5691640 1 -- start start 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.531+0000 7fb4d5691640 10 monclient: init 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb4d006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb4d006d380 0x7fb4d00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient(hunting): picked mon.noname-c con 0x7fb4d006d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:53.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d013a240 0x7fb4d013c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient(hunting): picked mon.noname-b con 0x7fb4d013a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d0074150 con 0x7fb4d006c890 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d009ef70 con 0x7fb4d006d380 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d009f250 con 0x7fb4d013a240 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4d5691640 10 monclient(hunting): authenticate will time out at 1019.085131s 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d013a240 0x7fb4d013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d013a240 0x7fb4d013c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56894/0 (socket says 10.20.193.78:56894) 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 1 -- 10.20.193.78:0/1113249944 learned_addr learned my addr 10.20.193.78:0/1113249944 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient(hunting): get_auth_request con 0x7fb4d013a240 auth_method 0 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4ce7fc640 1 --2- 10.20.193.78:0/1113249944 >> v2:10.20.193.115:3300/0 conn(0x7fb4d006d380 0x7fb4d00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4ce7fc640 10 monclient(hunting): get_auth_request con 0x7fb4d006d380 auth_method 0 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient(hunting): handle_auth_done global_id 74354 payload 274 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 1 -- 10.20.193.78:0/1113249944 >> v2:10.20.193.115:3300/0 conn(0x7fb4d006d380 msgr2=0x7fb4d00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 1 --2- 10.20.193.78:0/1113249944 >> v2:10.20.193.115:3300/0 conn(0x7fb4d006d380 0x7fb4d00a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 1 -- 10.20.193.78:0/1113249944 >> v1:10.20.193.115:6789/0 conn(0x7fb4d006c890 legacy=0x7fb4d006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 1 monclient: found mon.noname-b 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:52:53.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.532+0000 7fb4cf7fe640 1 -- 10.20.193.78:0/1113249944 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d009df40 con 0x7fb4d013a240 2026-02-17T05:52:53.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.534+0000 7fb4cf7fe640 1 -- 10.20.193.78:0/1113249944 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d013a240 msgr2=0x7fb4d013c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.534+0000 7fb4cf7fe640 1 --2- 10.20.193.78:0/1113249944 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d013a240 0x7fb4d013c630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009e50 tx=0x7fb4c002f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34dfc02dc806543 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 1 -- 10.20.193.78:0/1113249944 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00373f0 con 0x7fb4d013a240 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: dump: 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: _finish_auth 0 2026-02-17T05:52:53.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: _check_auth_tickets 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 1 -- 10.20.193.78:0/1113249944 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c0037590 con 0x7fb4d013a240 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 5 monclient: authenticate success, global_id 74354 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 1 -- 10.20.193.78:0/1113249944 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0037960 con 0x7fb4d013a240 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 10 monclient: get_monmap_and_config success 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 4 set_mon_vals no callback set 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4cdffb640 10 monclient: dump: 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 10 monclient: shutdown 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 1 -- 10.20.193.78:0/1113249944 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d013a240 msgr2=0x7fb4d013c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 1 --2- 10.20.193.78:0/1113249944 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d013a240 0x7fb4d013c630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009e50 tx=0x7fb4c002f330 comp rx=0 tx=0).stop 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 1 -- 10.20.193.78:0/1113249944 shutdown_connections 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 1 --2- 10.20.193.78:0/1113249944 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d013a240 0x7fb4d013c630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 1 --2- 10.20.193.78:0/1113249944 >> v2:10.20.193.115:3300/0 conn(0x7fb4d006d380 0x7fb4d00a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 1 -- 10.20.193.78:0/1113249944 >> 10.20.193.78:0/1113249944 conn(0x7fb4d00a6620 msgr2=0x7fb4d00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 1 -- 10.20.193.78:0/1113249944 shutdown_connections 2026-02-17T05:52:53.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 1 -- 10.20.193.78:0/1113249944 wait complete. 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 10 monclient: build_initial_monmap 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.535+0000 7fb4d5691640 10 monclient: monmap: 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:52:53.535 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 1 Processor -- start 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 1 -- start start 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient: init 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4d006c890 0x7fb4d0138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient(hunting): picked mon.noname-b con 0x7fb4d006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d006d380 0x7fb4d01354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient(hunting): picked mon.noname-c con 0x7fb4d006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4d013a240 0x7fb4d01cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient(hunting): picked mon.noname-a con 0x7fb4d013a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d013ccf0 con 0x7fb4d013a240 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d013cb70 con 0x7fb4d006c890 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d013ce70 con 0x7fb4d006d380 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4d5691640 10 monclient(hunting): authenticate will time out at 1019.089329s 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4cf7fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4d013a240 msgr2=0x7fb4d01cef40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4cf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4d013a240 0x7fb4d01cef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d006d380 0x7fb4d01354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d006d380 0x7fb4d01354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56910/0 (socket says 10.20.193.78:56910) 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 -- 10.20.193.78:0/2236063459 learned_addr learned my addr 10.20.193.78:0/2236063459 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient(hunting): get_auth_request con 0x7fb4d006d380 auth_method 0 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ceffd640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4d006c890 0x7fb4d0138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ceffd640 10 monclient(hunting): get_auth_request con 0x7fb4d006c890 auth_method 0 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient(hunting): handle_auth_done global_id 74357 payload 824 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 -- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4d006c890 msgr2=0x7fb4d0138760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4d006c890 0x7fb4d0138760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 -- 10.20.193.78:0/2236063459 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4d013a240 msgr2=0x7fb4d01cef40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4d013a240 0x7fb4d01cef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 monclient: found mon.noname-c 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.536+0000 7fb4ce7fc640 1 -- 10.20.193.78:0/2236063459 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d00a1340 con 0x7fb4d006d380 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4ce7fc640 1 -- 10.20.193.78:0/2236063459 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d006d380 msgr2=0x7fb4d01354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4ce7fc640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d006d380 0x7fb4d01354f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800e7c0 tx=0x7fb4b800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 1 -- 10.20.193.78:0/2236063459 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8017070 con 0x7fb4d006d380 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: dump: 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: _finish_auth 0 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 1 -- 10.20.193.78:0/2236063459 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b800cae0 con 0x7fb4d006d380 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 1 -- 10.20.193.78:0/2236063459 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800f040 con 0x7fb4d006d380 2026-02-17T05:52:53.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4cd7fa640 4 set_mon_vals no callback set 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4af7fe640 10 monclient: dump: 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4d5691640 5 monclient: authenticate success, global_id 74357 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4d5691640 10 monclient: _renew_subs 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4d5691640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d011dd70 con 0x7fb4d006d380 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4d5691640 10 monclient: _renew_subs 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4d5691640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.538+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d011fe80 con 0x7fb4d006d380 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4af7fe640 1 -- 10.20.193.78:0/2236063459 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb4b8012050 con 0x7fb4d006d380 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4d5691640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4d5691640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4d5691640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d0074980 con 0x7fb4d006d380 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4af7fe640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb49c03de60 0x7fb49c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4af7fe640 1 -- 10.20.193.78:0/2236063459 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(258..258 src has 117..258) ==== 4795+0+0 (secure 0 0 0) 0x7fb4b8052640 con 0x7fb4d006d380 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4ceffd640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb49c03de60 0x7fb49c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:53.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.539+0000 7fb4ceffd640 10 monclient: get_auth_request con 0x7fb49c03de60 auth_method 0 2026-02-17T05:52:53.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.542+0000 7fb4af7fe640 1 -- 10.20.193.78:0/2236063459 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb4d00a1340 con 0x7fb4d006d380 2026-02-17T05:52:53.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.542+0000 7fb4af7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:52:53.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.542+0000 7fb4af7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:52:53.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.542+0000 7fb4ceffd640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb49c03de60 0x7fb49c040320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005020 tx=0x7fb4c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:52:53.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.656+0000 7fb4d5691640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:52:53.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.656+0000 7fb4d5691640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:53.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.656+0000 7fb4d5691640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:52:53.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.656+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb4d011e470 con 0x7fb4d006d380 2026-02-17T05:52:53.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.656+0000 7fb4af7fe640 1 -- 10.20.193.78:0/2236063459 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fb4b8051030 con 0x7fb4d006d380 2026-02-17T05:52:53.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.656+0000 7fb4af7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:52:53.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.656+0000 7fb4af7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:52:53.654 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":74,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb49c03de60 msgr2=0x7fb49c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb49c03de60 0x7fb49c040320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005020 tx=0x7fb4c4009040 comp rx=0 tx=0).stop 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 10 monclient: shutdown 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d006d380 msgr2=0x7fb4d01354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d006d380 0x7fb4d01354f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800e7c0 tx=0x7fb4b800ec90 comp rx=0 tx=0).stop 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 shutdown_connections 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4d013a240 0x7fb4d01cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb4d006d380 0x7fb4d01354f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb49c03de60 0x7fb49c040320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 --2- 10.20.193.78:0/2236063459 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4d006c890 0x7fb4d0138760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 >> 10.20.193.78:0/2236063459 conn(0x7fb4d00a6620 msgr2=0x7fb4d0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 shutdown_connections 2026-02-17T05:52:53.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:53.657+0000 7fb4d5691640 1 -- 10.20.193.78:0/2236063459 wait complete. 2026-02-17T05:52:56.663 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:52:56.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.733+0000 7f895b2ca640 10 monclient: get_monmap_and_config 2026-02-17T05:52:56.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.733+0000 7f895b2ca640 10 monclient: build_initial_monmap 2026-02-17T05:52:56.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.733+0000 7f895b2ca640 10 monclient: monmap: 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:52:56.734153+0000 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:52:56.734153+0000 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:52:56.735 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 1 Processor -- start 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 1 -- start start 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient: init 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient: _add_conns ranks=[1,3,4,0,2] 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f895406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 0x7f895413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient(hunting): picked mon.noname-a con 0x7f895409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:56.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient(hunting): picked mon.noname-c con 0x7f895406d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895406daf0 con 0x7f895406c890 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f89540a0880 con 0x7f895409c4a0 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f895409cc80 con 0x7f895406d380 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895b2ca640 10 monclient(hunting): authenticate will time out at 1022.287665s 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895883e640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 msgr2=0x7f895413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895883e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 0x7f895413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895903f640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f895406c890 0x7f895406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:47172/0 (socket says 10.20.193.78:47172) 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:56.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 learned_addr learned my addr 10.20.193.78:0/2465725035 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f895406d380 auth_method 0 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:56.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.734+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:56.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.737+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.115:6789/0 conn(0x7f895406c890 legacy=0x7f895406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:52:56.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.934+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 msgr2=0x7f895413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:56.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.934+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 0x7f895413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:56.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:56.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f895406d380 auth_method 0 2026-02-17T05:52:56.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:56.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:56.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:56.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:56.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:56.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:56.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:56.935+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:52:57.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.335+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 msgr2=0x7f895413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:57.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.335+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 0x7f895413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f895406d380 auth_method 0 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:57.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:57.336+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:52:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.136+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 msgr2=0x7f895413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:52:58.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.136+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 0x7f895413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:58.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f895406d380 auth_method 0 2026-02-17T05:52:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:52:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:58.137+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:52:59.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 10 monclient(hunting): tick 2026-02-17T05:52:59.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:52:59.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 monclient(hunting): continuing hunt 2026-02-17T05:52:59.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:52:59.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 msgr2=0x7f895413d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f895406d380 0x7f895413d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 msgr2=0x7f895413b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f895409c4a0 0x7f895413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.115:6789/0 conn(0x7f895406c890 legacy=0x7f895406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 10 monclient: _add_conns ranks=[2,0] 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f8938003810 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8938006370 addr v1:10.20.193.78:6789/0 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895409cf60 con 0x7f8938006370 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:52:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f895409cc80 con 0x7f8938003810 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f8942ffd640 10 monclient(hunting): _renew_subs 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938006370 legacy=0x7f8938008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 10 monclient(hunting): get_auth_request con 0x7f8938003810 auth_method 0 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.734+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:52:59.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938006370 legacy=0x7f8938008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:52:59.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:52:59.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 10 monclient(hunting): get_auth_request con 0x7f8938003810 auth_method 0 2026-02-17T05:52:59.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:52:59.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:52:59.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:52:59.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:52:59.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:52:59.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:52:59.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:52:59.935+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:53:00.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938006370 legacy=0x7f8938008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:53:00.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:00.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 10 monclient(hunting): get_auth_request con 0x7f8938003810 auth_method 0 2026-02-17T05:53:00.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:00.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:00.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:00.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:00.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:00.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:00.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:00.335+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:53:01.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938006370 legacy=0x7f8938008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:53:01.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:01.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 10 monclient(hunting): get_auth_request con 0x7f8938003810 auth_method 0 2026-02-17T05:53:01.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:01.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:01.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:01.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:53:01.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:01.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:01.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:01.136+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): tick 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 monclient(hunting): continuing hunt 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8938003810 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938006370 legacy=0x7f8938008830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 reap_dead start 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 0x7f8938005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f893800a0d0 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f893800a580 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f89380087e0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f895409cc80 con 0x7f893800a0d0 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f8938009520 con 0x7f893800a580 2026-02-17T05:53:02.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8938009720 con 0x7f89380087e0 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8942ffd640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f89380087e0 auth_method 0 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 10 monclient(hunting): get_auth_request con 0x7f893800a580 auth_method 0 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:02.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.734+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:02.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:53:02.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:53:02.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:02.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f89380087e0 auth_method 0 2026-02-17T05:53:02.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:02.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 10 monclient(hunting): get_auth_request con 0x7f893800a580 auth_method 0 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:02.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:02.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:02.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:02.935+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:53:03.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:53:03.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:53:03.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:03.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f89380087e0 auth_method 0 2026-02-17T05:53:03.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.335+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.336+0000 7f895883e640 10 monclient(hunting): get_auth_request con 0x7f893800a580 auth_method 0 2026-02-17T05:53:03.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.336+0000 7f895883e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:03.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.336+0000 7f895883e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:03.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.336+0000 7f895883e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:03.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.336+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:03.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.336+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:03.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.336+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:03.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:03.336+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:53:04.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:53:04.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:53:04.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:04.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f89380087e0 auth_method 0 2026-02-17T05:53:04.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:04.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:04.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:04.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:04.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:04.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:04.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.136+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:53:04.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:04.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 10 monclient(hunting): get_auth_request con 0x7f893800a580 auth_method 0 2026-02-17T05:53:04.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:04.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:04.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:04.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:04.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:04.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:04.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:04.137+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:53:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 10 monclient(hunting): tick 2026-02-17T05:53:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:53:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 monclient(hunting): continuing hunt 2026-02-17T05:53:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:53:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 msgr2=0x7f8938008fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f89380087e0 0x7f8938008fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 msgr2=0x7f893800aa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.115:3300/0 conn(0x7f893800a580 0x7f893800aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 msgr2=0x7f8938005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 --2- 10.20.193.78:0/2465725035 >> v2:10.20.193.78:3300/0 conn(0x7f893800a0d0 0x7f8938005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 10 monclient: _add_conns ranks=[0,1] 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8938009a40 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8938010d40 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893800b1d0 con 0x7f8938009a40 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 1 -- 10.20.193.78:0/2465725035 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895409cf60 con 0x7f8938010d40 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f8942ffd640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.734+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938009a40 legacy=0x7f893800f620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:53:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.736+0000 7f895883e640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.115:6789/0 conn(0x7f8938010d40 legacy=0x7f8938013130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:05.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:05.935+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938009a40 legacy=0x7f893800f620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:53:06.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.335+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938009a40 legacy=0x7f893800f620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:53:06.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.413+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2974861926 0 0) 0x7f895409cf60 con 0x7f8938010d40 2026-02-17T05:53:06.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.413+0000 7f8943fff640 10 monclient(hunting): _negotiate 2026-02-17T05:53:06.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.413+0000 7f8943fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.413+0000 7f8943fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.413+0000 7f8943fff640 10 monclient(hunting): my global_id is 44530 2026-02-17T05:53:06.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.413+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f893c003620 con 0x7f8938010d40 2026-02-17T05:53:06.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4193174029 0 0) 0x7f893c003620 con 0x7f8938010d40 2026-02-17T05:53:06.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.78:6789/0 conn(0x7f8938009a40 legacy=0x7f893800f620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:06.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: _finish_hunting 0 2026-02-17T05:53:06.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:53:06.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:06.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:53:06.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8938016870 con 0x7f8938010d40 2026-02-17T05:53:06.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: _finish_auth 0 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: _check_auth_tickets 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f8948008040 con 0x7f8938010d40 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: dump: 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:53:06.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 >> v1:10.20.193.115:6789/0 conn(0x7f8938010d40 legacy=0x7f8938013130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 reap_dead start 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c003ab0 0x7f893c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient(hunting): picked mon.c con 0x7f893c003ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 0x7f893c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient(hunting): picked mon.b con 0x7f893c004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c0054c0 0x7f893c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient(hunting): picked mon.a con 0x7f893c0054c0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f893c00c320 con 0x7f893c0054c0 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f893c00c4d0 con 0x7f893c004900 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f893c00c6d0 con 0x7f893c003ab0 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8943fff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8959840640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c0054c0 msgr2=0x7f893c00bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c003ab0 0x7f893c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient(hunting): get_auth_request con 0x7f893c003ab0 auth_method 0 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f8959840640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c0054c0 0x7f893c00bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895883e640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 0x7f893c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895883e640 10 monclient(hunting): get_auth_request con 0x7f893c004900 auth_method 0 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895883e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895883e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895883e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient(hunting): handle_auth_done global_id 44530 payload 293 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 msgr2=0x7f893c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 0x7f893c004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c0054c0 msgr2=0x7f893c00bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c0054c0 0x7f893c00bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient: _finish_hunting 0 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 1 monclient: found mon.c 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c003620 con 0x7f893c003ab0 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient: _finish_auth 0 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 10 monclient: _check_auth_tickets 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895903f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:06.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895b2ca640 5 monclient: authenticate success, global_id 44530 2026-02-17T05:53:06.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.415+0000 7f895b2ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:06.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895903f640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c003ab0 msgr2=0x7f893c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895903f640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c003ab0 0x7f893c003e90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8944007d00 tx=0x7f894402ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08f16ae01fafeb7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944036070 con 0x7f893c003ab0 2026-02-17T05:53:06.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:06.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 10 monclient: dump: 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8944030940 con 0x7f893c003ab0 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 1 -- 10.20.193.78:0/2465725035 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944005d00 con 0x7f893c003ab0 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f8943fff640 10 monclient: dump: 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 4 set_mon_vals no callback set 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 10 monclient: shutdown 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 -- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c003ab0 msgr2=0x7f893c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c003ab0 0x7f893c003e90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8944007d00 tx=0x7f894402ff00 comp rx=0 tx=0).stop 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 -- 10.20.193.78:0/2465725035 shutdown_connections 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c0054c0 0x7f893c00bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 0x7f893c004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2465725035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c003ab0 0x7f893c003e90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 -- 10.20.193.78:0/2465725035 >> 10.20.193.78:0/2465725035 conn(0x7f89540a6620 msgr2=0x7f89540a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 -- 10.20.193.78:0/2465725035 shutdown_connections 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.418+0000 7f895b2ca640 1 -- 10.20.193.78:0/2465725035 wait complete. 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient: build_initial_monmap 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient: monmap: 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 1 Processor -- start 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 1 -- start start 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient: init 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c003ab0 0x7f89541241c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient(hunting): picked mon.noname-a con 0x7f893c003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 0x7f8954124700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient(hunting): picked mon.noname-b con 0x7f893c004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c0054c0 0x7f8954124c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient(hunting): picked mon.noname-c con 0x7f893c0054c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f895409bfa0 con 0x7f893c003ab0 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f895409be20 con 0x7f893c004900 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f895406cc70 con 0x7f893c0054c0 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895b2ca640 10 monclient(hunting): authenticate will time out at 1031.973858s 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895903f640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c003ab0 msgr2=0x7f89541241c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895903f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c003ab0 0x7f89541241c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c0054c0 0x7f8954124c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c0054c0 0x7f8954124c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60506/0 (socket says 10.20.193.78:60506) 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 1 -- 10.20.193.78:0/2681602818 learned_addr learned my addr 10.20.193.78:0/2681602818 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 10 monclient(hunting): get_auth_request con 0x7f893c0054c0 auth_method 0 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f8959840640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.419+0000 7f895883e640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 0x7f8954124700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f895883e640 10 monclient(hunting): get_auth_request con 0x7f893c004900 auth_method 0 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f895883e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f895883e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f895883e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 10 monclient(hunting): handle_auth_done global_id 74360 payload 824 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 1 -- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 msgr2=0x7f8954124700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 0x7f8954124700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 1 -- 10.20.193.78:0/2681602818 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c003ab0 msgr2=0x7f89541241c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c003ab0 0x7f89541241c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 10 monclient: _finish_hunting 0 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 1 monclient: found mon.noname-c 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.420+0000 7f8959840640 1 -- 10.20.193.78:0/2681602818 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8954128570 con 0x7f893c0054c0 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8959840640 1 -- 10.20.193.78:0/2681602818 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c0054c0 msgr2=0x7f8954124c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8959840640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c0054c0 0x7f8954124c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f893800a070 tx=0x7f8950004780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 1 -- 10.20.193.78:0/2681602818 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950024480 con 0x7f893c0054c0 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: dump: 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: _finish_auth 0 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:06.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 1 -- 10.20.193.78:0/2681602818 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8950024640 con 0x7f893c0054c0 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f895b2ca640 5 monclient: authenticate success, global_id 74360 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f895b2ca640 10 monclient: _renew_subs 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f895b2ca640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8954126740 con 0x7f893c0054c0 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 1 -- 10.20.193.78:0/2681602818 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950024a30 con 0x7f893c0054c0 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 10 monclient: dump: 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f89437fe640 4 set_mon_vals no callback set 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f895b2ca640 10 monclient: _renew_subs 2026-02-17T05:53:06.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f895b2ca640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8954125e60 con 0x7f893c0054c0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.422+0000 7f8941ffb640 1 -- 10.20.193.78:0/2681602818 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8950019430 con 0x7f893c0054c0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.423+0000 7f895b2ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.423+0000 7f895b2ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.423+0000 7f895b2ca640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.423+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8954074980 con 0x7f893c0054c0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.423+0000 7f8941ffb640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f892403deb0 0x7f8924040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.423+0000 7f8941ffb640 1 -- 10.20.193.78:0/2681602818 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(258..258 src has 117..258) ==== 4795+0+0 (secure 0 0 0) 0x7f89500546d0 con 0x7f893c0054c0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.423+0000 7f895903f640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f892403deb0 0x7f8924040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.423+0000 7f895903f640 10 monclient: get_auth_request con 0x7f892403deb0 auth_method 0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.425+0000 7f8941ffb640 1 -- 10.20.193.78:0/2681602818 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8954128570 con 0x7f893c0054c0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.425+0000 7f8941ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.425+0000 7f8941ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:06.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.425+0000 7f895903f640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f892403deb0 0x7f8924040370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f893c00c8a0 tx=0x7f8944036000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.466 INFO:tasks.rados.rados.0.trial115.stdout:1215: done (2 left) 2026-02-17T05:53:06.466 INFO:tasks.rados.rados.0.trial115.stdout:1216: snap_create 2026-02-17T05:53:06.466 INFO:tasks.rados.rados.0.trial115.stdout:1213: finishing write tid 3 to trial11545420-31 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1213: finishing write tid 4 to trial11545420-31 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1213: finishing write tid 5 to trial11545420-31 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1213: oid 31 updating version 0 to 601 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1213: oid 31 version 601 is already newer than 600 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1213: oid 31 version 601 is already newer than 599 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1213: oid 31 version 601 is already newer than 598 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 601 (ObjNum 470 snap 121 seq_num 470) dirty exists 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1213: left oid 31 (ObjNum 470 snap 121 seq_num 470) 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1214: finishing copy_from racing read to trial11545420-6 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:1214: finishing copy_from to trial11545420-6 2026-02-17T05:53:06.467 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 425 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:53:06.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.539+0000 7f895b2ca640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:53:06.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.539+0000 7f895b2ca640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:06.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.539+0000 7f895b2ca640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.539+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f895406cfa0 con 0x7f893c0054c0 2026-02-17T05:53:06.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.540+0000 7f8941ffb640 1 -- 10.20.193.78:0/2681602818 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f8950010040 con 0x7f893c0054c0 2026-02-17T05:53:06.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.540+0000 7f8941ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:06.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.540+0000 7f8941ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:06.541 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f892403deb0 msgr2=0x7f8924040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f892403deb0 0x7f8924040370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f893c00c8a0 tx=0x7f8944036000 comp rx=0 tx=0).stop 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 10 monclient: shutdown 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c0054c0 msgr2=0x7f8954124c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c0054c0 0x7f8954124c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f893800a070 tx=0x7f8950004780 comp rx=0 tx=0).stop 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 shutdown_connections 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f893c0054c0 0x7f8954124c40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f892403deb0 0x7f8924040370 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f893c004900 0x7f8954124700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 --2- 10.20.193.78:0/2681602818 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f893c003ab0 0x7f89541241c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 >> 10.20.193.78:0/2681602818 conn(0x7f89540a6620 msgr2=0x7f89540a27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:06.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 shutdown_connections 2026-02-17T05:53:06.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.541+0000 7f895b2ca640 1 -- 10.20.193.78:0/2681602818 wait complete. 2026-02-17T05:53:06.549 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:53:06.549 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:53:06.550 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:53:06.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient: get_monmap_and_config 2026-02-17T05:53:06.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient: build_initial_monmap 2026-02-17T05:53:06.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient: monmap: 2026-02-17T05:53:06.622 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:06.622 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.622 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:06.620235+0000 2026-02-17T05:53:06.622 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:06.620235+0000 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 1 Processor -- start 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 1 -- start start 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient: init 2026-02-17T05:53:06.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f77ec06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f77ec09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f77ec06d380 0x7f77ec13ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient(hunting): picked mon.noname-c con 0x7f77ec06d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77ec06daf0 con 0x7f77ec06c890 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77ec09cc80 con 0x7f77ec09c4a0 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f77ec0a0880 con 0x7f77ec06d380 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:06.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77f1d14640 10 monclient(hunting): authenticate will time out at 1032.173796s 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77eb7fe640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f77ec06c890 legacy=0x7f77ec06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.619+0000 7f77eaffd640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f77ec09c4a0 0x7f77ec13b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:56264/0 (socket says 10.20.193.78:56264) 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77eaffd640 1 -- 10.20.193.78:0/294840610 learned_addr learned my addr 10.20.193.78:0/294840610 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 1 --2- 10.20.193.78:0/294840610 >> v2:10.20.193.115:3300/0 conn(0x7f77ec06d380 0x7f77ec13ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient(hunting): get_auth_request con 0x7f77ec06d380 auth_method 0 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient(hunting): handle_auth_done global_id 44536 payload 274 2026-02-17T05:53:06.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 1 -- 10.20.193.78:0/294840610 >> v1:10.20.193.115:6789/0 conn(0x7f77ec09c4a0 legacy=0x7f77ec13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:06.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 1 -- 10.20.193.78:0/294840610 >> v1:10.20.193.78:6789/0 conn(0x7f77ec06c890 legacy=0x7f77ec06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:06.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient: _finish_hunting 0 2026-02-17T05:53:06.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 1 monclient: found mon.noname-c 2026-02-17T05:53:06.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:06.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:53:06.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.620+0000 7f77ebfff640 1 -- 10.20.193.78:0/294840610 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec09cf60 con 0x7f77ec06d380 2026-02-17T05:53:06.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ebfff640 1 -- 10.20.193.78:0/294840610 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec06d380 msgr2=0x7f77ec13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ebfff640 1 --2- 10.20.193.78:0/294840610 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec06d380 0x7f77ec13ead0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77ec074de0 tx=0x7f77e002f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca3c53cb546f26 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 1 -- 10.20.193.78:0/294840610 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e003c070 con 0x7f77ec06d380 2026-02-17T05:53:06.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:06.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: dump: 2026-02-17T05:53:06.648 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.648 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.648 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.648 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.649 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.649 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.649 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.649 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.649 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.650 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: _finish_auth 0 2026-02-17T05:53:06.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:53:06.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:06.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 5 monclient: authenticate success, global_id 44536 2026-02-17T05:53:06.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:06.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 1 -- 10.20.193.78:0/294840610 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e0037640 con 0x7f77ec06d380 2026-02-17T05:53:06.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:06.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 1 -- 10.20.193.78:0/294840610 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0037a10 con 0x7f77ec06d380 2026-02-17T05:53:06.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:06.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77ea7fc640 10 monclient: dump: 2026-02-17T05:53:06.652 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.652 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.652 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.652 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.652 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.652 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.652 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.653 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.653 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.653 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:06.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 4 set_mon_vals no callback set 2026-02-17T05:53:06.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 10 monclient: shutdown 2026-02-17T05:53:06.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:06.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 1 -- 10.20.193.78:0/294840610 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec06d380 msgr2=0x7f77ec13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 1 --2- 10.20.193.78:0/294840610 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec06d380 0x7f77ec13ead0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77ec074de0 tx=0x7f77e002f130 comp rx=0 tx=0).stop 2026-02-17T05:53:06.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 1 -- 10.20.193.78:0/294840610 shutdown_connections 2026-02-17T05:53:06.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 1 --2- 10.20.193.78:0/294840610 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec06d380 0x7f77ec13ead0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 1 -- 10.20.193.78:0/294840610 >> 10.20.193.78:0/294840610 conn(0x7f77ec0a6620 msgr2=0x7f77ec0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:06.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 1 -- 10.20.193.78:0/294840610 shutdown_connections 2026-02-17T05:53:06.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 1 -- 10.20.193.78:0/294840610 wait complete. 2026-02-17T05:53:06.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.623+0000 7f77f1d14640 10 monclient: build_initial_monmap 2026-02-17T05:53:06.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient: monmap: 2026-02-17T05:53:06.655 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:06.655 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:06.655 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:06.655 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:06.656 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:06.656 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.656 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:06.656 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:06.656 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:06.656 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 1 Processor -- start 2026-02-17T05:53:06.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 1 -- start start 2026-02-17T05:53:06.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient: init 2026-02-17T05:53:06.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:06.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:53:06.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec06c890 0x7f77ec11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient(hunting): picked mon.noname-c con 0x7f77ec06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:06.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77ec06d380 0x7f77ec11d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient(hunting): picked mon.noname-a con 0x7f77ec06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:06.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec09c4a0 0x7f77ec11d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient(hunting): picked mon.noname-b con 0x7f77ec09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:06.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec09bfa0 con 0x7f77ec06d380 2026-02-17T05:53:06.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec09be20 con 0x7f77ec09c4a0 2026-02-17T05:53:06.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f77ec13f010 con 0x7f77ec06c890 2026-02-17T05:53:06.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:06.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77f1d14640 10 monclient(hunting): authenticate will time out at 1032.178477s 2026-02-17T05:53:06.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eaffd640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77ec06d380 msgr2=0x7f77ec11d1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:53:06.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eaffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77ec06d380 0x7f77ec11d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:06.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec06c890 0x7f77ec11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec06c890 0x7f77ec11cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60508/0 (socket says 10.20.193.78:60508) 2026-02-17T05:53:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 -- 10.20.193.78:0/3723417873 learned_addr learned my addr 10.20.193.78:0/3723417873 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient(hunting): get_auth_request con 0x7f77ec06c890 auth_method 0 2026-02-17T05:53:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77ebfff640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec09c4a0 0x7f77ec11d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77ebfff640 10 monclient(hunting): get_auth_request con 0x7f77ec09c4a0 auth_method 0 2026-02-17T05:53:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77ebfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77ebfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient(hunting): handle_auth_done global_id 74363 payload 824 2026-02-17T05:53:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec09c4a0 msgr2=0x7f77ec11d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77ec09c4a0 0x7f77ec11d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77ec06d380 msgr2=0x7f77ec11d1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77ec06d380 0x7f77ec11d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:53:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 monclient: found mon.noname-c 2026-02-17T05:53:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.624+0000 7f77eb7fe640 1 -- 10.20.193.78:0/3723417873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec09d810 con 0x7f77ec06c890 2026-02-17T05:53:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.626+0000 7f77eb7fe640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec06c890 msgr2=0x7f77ec11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.626+0000 7f77eb7fe640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec06c890 0x7f77ec11cc70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77d800b780 tx=0x7f77d800bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8005270 con 0x7f77ec06c890 2026-02-17T05:53:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: dump: 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.665 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: _finish_auth 0 2026-02-17T05:53:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:53:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d8010070 con 0x7f77ec06c890 2026-02-17T05:53:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 5 monclient: authenticate success, global_id 74363 2026-02-17T05:53:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8005650 con 0x7f77ec06c890 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77e9ffb640 4 set_mon_vals no callback set 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 10 monclient: dump: 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.666 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 10 monclient: _renew_subs 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec11f5c0 con 0x7f77ec06c890 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 10 monclient: _renew_subs 2026-02-17T05:53:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec13e410 con 0x7f77ec06c890 2026-02-17T05:53:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec11f800 0x7f77ec135a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f77ec06cc70 con 0x7f77ec11f800 2026-02-17T05:53:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec11f800 0x7f77ec135a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 10 monclient: get_auth_request con 0x7f77ec11f800 auth_method 0 2026-02-17T05:53:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f77d800c520 con 0x7f77ec06c890 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 10 monclient(hunting): handle_auth_done global_id 74366 payload 824 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77eaffd640 1 -- 10.20.193.78:0/3723417873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77ec11fc40 con 0x7f77ec11f800 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77bc03de60 0x7f77bc040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.629+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(259..259 src has 117..259) ==== 4811+0+0 (secure 0 0 0) 0x7f77d80520a0 con 0x7f77ec06c890 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.630+0000 7f77ebfff640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77bc03de60 0x7f77bc040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.630+0000 7f77ebfff640 10 monclient: get_auth_request con 0x7f77bc03de60 auth_method 0 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77eaffd640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec11f800 msgr2=0x7f77ec135a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77eaffd640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec11f800 0x7f77ec135a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77dc004910 tx=0x7f77dc00cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc017070 con 0x7f77ec11f800 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 10 monclient: dump: 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.670 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f77dc005d40 con 0x7f77ec11f800 2026-02-17T05:53:06.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec11f800 msgr2=0x7f77ec135a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec11f800 0x7f77ec135a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77dc004910 tx=0x7f77dc00cb20 comp rx=0 tx=0).stop 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec11f800 msgr2=0x7f77ec135a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77c7fff640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec11f800 0x7f77ec135a50 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77dc004910 tx=0x7f77dc00cb20 comp rx=0 tx=0).stop 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.632+0000 7f77ebfff640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77bc03de60 0x7f77bc040320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f77ec06a480 tx=0x7f77e0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77f1d14640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77f1d14640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec135f90 0x7f77ec1d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77f1d14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f77ec11e170 con 0x7f77ec135f90 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec135f90 0x7f77ec1d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 10 monclient: get_auth_request con 0x7f77ec135f90 auth_method 0 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 10 monclient(hunting): handle_auth_done global_id 74369 payload 824 2026-02-17T05:53:06.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.635+0000 7f77eaffd640 1 -- 10.20.193.78:0/3723417873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f77ec0744d0 con 0x7f77ec135f90 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.637+0000 7f77eaffd640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec135f90 msgr2=0x7f77ec1d84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.637+0000 7f77eaffd640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec135f90 0x7f77ec1d84b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77ec13dbb0 tx=0x7f77dc004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc01ae60 con 0x7f77ec135f90 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 10 monclient: dump: 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.674 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7f77dc007490 con 0x7f77ec135f90 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec135f90 msgr2=0x7f77ec1d84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec135f90 0x7f77ec1d84b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77ec13dbb0 tx=0x7f77dc004b10 comp rx=0 tx=0).stop 2026-02-17T05:53:06.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec135f90 msgr2=0x7f77ec1d84b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:06.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77c7fff640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec135f90 0x7f77ec1d84b0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77ec13dbb0 tx=0x7f77dc004b10 comp rx=0 tx=0).stop 2026-02-17T05:53:06.676 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:06.676 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:06.676 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:06.676 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:53:06.676 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 78, 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 158530, 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:06.677 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:06.678 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:06.679 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:06.680 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:06.681 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:06.682 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.683 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.684 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:06.685 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.686 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.687 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.688 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:06.689 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77bc03de60 msgr2=0x7f77bc040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77bc03de60 0x7f77bc040320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f77ec06a480 tx=0x7f77e0033000 comp rx=0 tx=0).stop 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 10 monclient: shutdown 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec06c890 msgr2=0x7f77ec11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec06c890 0x7f77ec11cc70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77d800b780 tx=0x7f77d800bc50 comp rx=0 tx=0).stop 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77ebfff640 1 -- 10.20.193.78:0/3723417873 reap_dead start 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 shutdown_connections 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 --2- 10.20.193.78:0/3723417873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77ec06c890 0x7f77ec11cc70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 >> 10.20.193.78:0/3723417873 conn(0x7f77ec0a6620 msgr2=0x7f77ec0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 shutdown_connections 2026-02-17T05:53:06.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.638+0000 7f77f1d14640 1 -- 10.20.193.78:0/3723417873 wait complete. 2026-02-17T05:53:06.692 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:53:06.692 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:53:06.692 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:53:06.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient: get_monmap_and_config 2026-02-17T05:53:06.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient: build_initial_monmap 2026-02-17T05:53:06.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient: monmap: 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:06.762392+0000 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:06.762392+0000 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:06.761 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 1 Processor -- start 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 1 -- start start 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient: init 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa64406c890 0x7fa64406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient(hunting): picked mon.noname-c con 0x7fa64406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa64409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa64406d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa644135c10 con 0x7fa64406d1b0 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64406daf0 con 0x7fa64409c4a0 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa6440a0880 con 0x7fa64406c890 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.761+0000 7fa6492fa640 10 monclient(hunting): authenticate will time out at 1032.315984s 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa6437fe640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7fa64406d1b0 legacy=0x7fa644135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa6427fc640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7fa64409c4a0 0x7fa644134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:56270/0 (socket says 10.20.193.78:56270) 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa6427fc640 1 -- 10.20.193.78:0/389398401 learned_addr learned my addr 10.20.193.78:0/389398401 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 1 --2- 10.20.193.78:0/389398401 >> v2:10.20.193.115:3300/0 conn(0x7fa64406c890 0x7fa64406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient(hunting): get_auth_request con 0x7fa64406c890 auth_method 0 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient(hunting): handle_auth_done global_id 44542 payload 274 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 1 -- 10.20.193.78:0/389398401 >> v1:10.20.193.115:6789/0 conn(0x7fa64409c4a0 legacy=0x7fa644134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 1 -- 10.20.193.78:0/389398401 >> v1:10.20.193.78:6789/0 conn(0x7fa64406d1b0 legacy=0x7fa644135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 1 monclient: found mon.noname-c 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:53:06.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.762+0000 7fa642ffd640 1 -- 10.20.193.78:0/389398401 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64409cc80 con 0x7fa64406c890 2026-02-17T05:53:06.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa642ffd640 1 -- 10.20.193.78:0/389398401 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 msgr2=0x7fa64406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa642ffd640 1 --2- 10.20.193.78:0/389398401 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 0x7fa64406cc70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098a0 tx=0x7fa63802ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a82b66e0d24693c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 1 -- 10.20.193.78:0/389398401 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63803c070 con 0x7fa64406c890 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: dump: 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: _finish_auth 0 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 5 monclient: authenticate success, global_id 44542 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 1 -- 10.20.193.78:0/389398401 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63802fce0 con 0x7fa64406c890 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 1 -- 10.20.193.78:0/389398401 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638037660 con 0x7fa64406c890 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa641ffb640 10 monclient: dump: 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.765 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 4 set_mon_vals no callback set 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 10 monclient: shutdown 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 1 -- 10.20.193.78:0/389398401 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 msgr2=0x7fa64406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 1 --2- 10.20.193.78:0/389398401 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 0x7fa64406cc70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098a0 tx=0x7fa63802ed80 comp rx=0 tx=0).stop 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 1 -- 10.20.193.78:0/389398401 shutdown_connections 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 1 --2- 10.20.193.78:0/389398401 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 0x7fa64406cc70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 1 -- 10.20.193.78:0/389398401 >> 10.20.193.78:0/389398401 conn(0x7fa6440a6620 msgr2=0x7fa6440a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 1 -- 10.20.193.78:0/389398401 shutdown_connections 2026-02-17T05:53:06.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.765+0000 7fa6492fa640 1 -- 10.20.193.78:0/389398401 wait complete. 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient: build_initial_monmap 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient: monmap: 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 1 Processor -- start 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 1 -- start start 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient: init 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:53:06.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 0x7fa644134db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient(hunting): picked mon.noname-b con 0x7fa64406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa64406d1b0 0x7fa6441352f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient(hunting): picked mon.noname-c con 0x7fa64406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa64409c4a0 0x7fa644122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient(hunting): picked mon.noname-a con 0x7fa64409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6440a0020 con 0x7fa64409c4a0 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa64406d640 con 0x7fa64406c890 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa6440a01a0 con 0x7fa64406d1b0 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6492fa640 10 monclient(hunting): authenticate will time out at 1032.320761s 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6437fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa64409c4a0 msgr2=0x7fa644122820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6437fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa64409c4a0 0x7fa644122820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa64406d1b0 0x7fa6441352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa64406d1b0 0x7fa6441352f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60540/0 (socket says 10.20.193.78:60540) 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 1 -- 10.20.193.78:0/3206292866 learned_addr learned my addr 10.20.193.78:0/3206292866 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 10 monclient(hunting): get_auth_request con 0x7fa64406d1b0 auth_method 0 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa6427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.766+0000 7fa642ffd640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 0x7fa644134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa642ffd640 10 monclient(hunting): get_auth_request con 0x7fa64406c890 auth_method 0 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa642ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa642ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 10 monclient(hunting): handle_auth_done global_id 74372 payload 824 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 msgr2=0x7fa644134db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa64406c890 0x7fa644134db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa64409c4a0 msgr2=0x7fa644122820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa64409c4a0 0x7fa644122820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 10 monclient: _finish_hunting 0 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 1 monclient: found mon.noname-c 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:06.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.767+0000 7fa6427fc640 1 -- 10.20.193.78:0/3206292866 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6441ce150 con 0x7fa64406d1b0 2026-02-17T05:53:06.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6427fc640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa64406d1b0 msgr2=0x7fa6441352f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6427fc640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa64406d1b0 0x7fa6441352f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c7b0 tx=0x7fa62c00cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c005de0 con 0x7fa64406d1b0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: dump: 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: _finish_auth 0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: _check_auth_tickets 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62c007070 con 0x7fa64406d1b0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 5 monclient: authenticate success, global_id 74372 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 10 monclient: _renew_subs 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6441264d0 con 0x7fa64406d1b0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c007440 con 0x7fa64406d1b0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6417fa640 4 set_mon_vals no callback set 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 10 monclient: dump: 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.770 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 10 monclient: _renew_subs 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6440727e0 con 0x7fa64406d1b0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441260c0 0x7fa644122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa6440a2800 con 0x7fa6441260c0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.769+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa62c0075e0 con 0x7fa64406d1b0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441260c0 0x7fa644122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 10 monclient: get_auth_request con 0x7fa6441260c0 auth_method 0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa6237fe640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa61003de60 0x7fa610040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(259..259 src has 117..259) ==== 4811+0+0 (secure 0 0 0) 0x7fa62c0526a0 con 0x7fa64406d1b0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa6437fe640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa61003de60 0x7fa610040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa6437fe640 10 monclient: get_auth_request con 0x7fa61003de60 auth_method 0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 10 monclient(hunting): handle_auth_done global_id 44548 payload 824 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.770+0000 7fa642ffd640 1 -- 10.20.193.78:0/3206292866 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa644074080 con 0x7fa6441260c0 2026-02-17T05:53:06.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.772+0000 7fa642ffd640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441260c0 msgr2=0x7fa644122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.772+0000 7fa642ffd640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441260c0 0x7fa644122d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa638039000 tx=0x7fa6380047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.772+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63803c040 con 0x7fa6441260c0 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.772+0000 7fa6237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.772+0000 7fa6237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.772+0000 7fa6237fe640 10 monclient: dump: 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.772+0000 7fa6437fe640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa61003de60 0x7fa610040320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa6441358f0 tx=0x7fa6340091f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.773+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa638041440 con 0x7fa6441260c0 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.773+0000 7fa6237fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.773+0000 7fa6237fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.773+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441260c0 msgr2=0x7fa644122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.773+0000 7fa6237fe640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441260c0 0x7fa644122d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa638039000 tx=0x7fa6380047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.773+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441260c0 msgr2=0x7fa644122d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:06.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.773+0000 7fa6237fe640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441260c0 0x7fa644122d60 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa638039000 tx=0x7fa6380047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.775+0000 7fa6492fa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.775+0000 7fa6492fa640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441cd4a0 0x7fa6441d8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.775+0000 7fa6492fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.775+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa64409bf60 con 0x7fa6441cd4a0 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441cd4a0 0x7fa6441d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 10 monclient: get_auth_request con 0x7fa6441cd4a0 auth_method 0 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 10 monclient(hunting): handle_auth_done global_id 44551 payload 824 2026-02-17T05:53:06.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.776+0000 7fa642ffd640 1 -- 10.20.193.78:0/3206292866 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa6440744d0 con 0x7fa6441cd4a0 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa642ffd640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441cd4a0 msgr2=0x7fa6441d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa642ffd640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441cd4a0 0x7fa6441d8580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa638002410 tx=0x7fa638008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63803c040 con 0x7fa6441cd4a0 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 10 monclient: dump: 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:06.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7fa638003bc0 con 0x7fa6441cd4a0 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441cd4a0 msgr2=0x7fa6441d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441cd4a0 0x7fa6441d8580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa638002410 tx=0x7fa638008030 comp rx=0 tx=0).stop 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441cd4a0 msgr2=0x7fa6441d8580 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6237fe640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6441cd4a0 0x7fa6441d8580 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa638002410 tx=0x7fa638008030 comp rx=0 tx=0).stop 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 78, 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 310427, 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:06.779 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:06.780 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:06.781 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.782 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:06.783 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa61003de60 msgr2=0x7fa610040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa61003de60 0x7fa610040320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa6441358f0 tx=0x7fa6340091f0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 10 monclient: shutdown 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa64406d1b0 msgr2=0x7fa6441352f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:06.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa64406d1b0 0x7fa6441352f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c7b0 tx=0x7fa62c00cc80 comp rx=0 tx=0).stop 2026-02-17T05:53:06.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6437fe640 1 -- 10.20.193.78:0/3206292866 reap_dead start 2026-02-17T05:53:06.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 shutdown_connections 2026-02-17T05:53:06.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 --2- 10.20.193.78:0/3206292866 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa64406d1b0 0x7fa6441352f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:06.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 >> 10.20.193.78:0/3206292866 conn(0x7fa6440a6620 msgr2=0x7fa644072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:06.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 shutdown_connections 2026-02-17T05:53:06.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:06.779+0000 7fa6492fa640 1 -- 10.20.193.78:0/3206292866 wait complete. 2026-02-17T05:53:06.786 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:53:06.786 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:53:07.531 INFO:tasks.rados.rados.0.trial115.stdout:1213: done (2 left) 2026-02-17T05:53:07.532 INFO:tasks.rados.rados.0.trial115.stdout:1214: done (1 left) 2026-02-17T05:53:07.532 INFO:tasks.rados.rados.0.trial115.stdout:1216: done (0 left) 2026-02-17T05:53:07.532 INFO:tasks.rados.rados.0.trial115.stdout:1217: copy_from oid 18 from oid 9 current snap is 122 2026-02-17T05:53:07.532 INFO:tasks.rados.rados.0.trial115.stdout:1218: rollback oid 19 current snap is 122 2026-02-17T05:53:07.532 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 19 to 110 2026-02-17T05:53:07.532 INFO:tasks.rados.rados.0.trial115.stdout:1219: write (excl) oid 44 current snap is 122 2026-02-17T05:53:07.532 INFO:tasks.rados.rados.0.trial115.stdout:1219: seq_num 471 ranges {716097=539812,1945607=795166,3535053=381044} 2026-02-17T05:53:07.535 INFO:tasks.rados.rados.0.trial115.stdout:1219: writing trial11545420-44 from 716097 to 1255909 tid 1 2026-02-17T05:53:07.541 INFO:tasks.rados.rados.0.trial115.stdout:1219: writing trial11545420-44 from 1945607 to 2740773 tid 2 2026-02-17T05:53:07.546 INFO:tasks.rados.rados.0.trial115.stdout:1219: writing trial11545420-44 from 3535053 to 3916097 tid 3 2026-02-17T05:53:07.546 INFO:tasks.rados.rados.0.trial115.stdout:1220: delete oid 13 current snap is 122 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:1218: finishing rollback tid 0 to trial11545420-19 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:1218: finishing rollback tid 1 to trial11545420-19 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:1218: finishing rollback tid 2 to trial11545420-19 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 626 (ObjNum 306 snap 73 seq_num 306) dirty exists 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:1219: finishing write tid 1 to trial11545420-44 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:1217: finishing copy_from racing read to trial11545420-18 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:1217: finishing copy_from to trial11545420-18 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 313 (ObjNum 377 snap 99 seq_num 377) dirty exists 2026-02-17T05:53:07.547 INFO:tasks.rados.rados.0.trial115.stdout:1219: finishing write tid 2 to trial11545420-44 2026-02-17T05:53:07.549 INFO:tasks.rados.rados.0.trial115.stdout:1219: finishing write tid 3 to trial11545420-44 2026-02-17T05:53:07.549 INFO:tasks.rados.rados.0.trial115.stdout:1219: finishing write tid 4 to trial11545420-44 2026-02-17T05:53:07.549 INFO:tasks.rados.rados.0.trial115.stdout:1217: done (3 left) 2026-02-17T05:53:07.549 INFO:tasks.rados.rados.0.trial115.stdout:1218: done (2 left) 2026-02-17T05:53:07.549 INFO:tasks.rados.rados.0.trial115.stdout:1220: done (1 left) 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1221: copy_from oid 26 from oid 17 current snap is 122 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1219: finishing write tid 5 to trial11545420-44 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1219: oid 44 updating version 0 to 321 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1219: oid 44 updating version 321 to 323 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1219: oid 44 updating version 323 to 324 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1219: oid 44 version 324 is already newer than 322 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 324 (ObjNum 471 snap 122 seq_num 471) dirty exists 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1219: left oid 44 (ObjNum 471 snap 122 seq_num 471) 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1219: done (1 left) 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1222: rollback oid 3 current snap is 122 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 3 to 94 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:1223: rollback oid 9 current snap is 122 2026-02-17T05:53:07.550 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 9 to 116 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1224: read oid 27 snap -1 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1224: expect deleted 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1225: rollback oid 24 current snap is 122 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 121 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1226: read oid 2 snap -1 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1226: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1227: read oid 18 snap -1 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1227: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1228: read oid 33 snap -1 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1228: expect (ObjNum 423 snap 107 seq_num 423) 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1229: read oid 49 snap -1 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1229: expect deleted 2026-02-17T05:53:07.551 INFO:tasks.rados.rados.0.trial115.stdout:1230: snap_remove snap 113 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:1230: done (9 left) 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:1231: copy_from oid 50 from oid 4 current snap is 122 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:1222: finishing rollback tid 0 to trial11545420-3 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:1223: finishing rollback tid 0 to trial11545420-9 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:1225: finishing rollback tid 1 to trial11545420-24 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 0 (ObjNum 129 snap 0 seq_num 1333734240) dirty dne 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:1223: finishing rollback tid 1 to trial11545420-9 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:1223: finishing rollback tid 2 to trial11545420-9 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 588 (ObjNum 377 snap 99 seq_num 377) dirty exists 2026-02-17T05:53:08.538 INFO:tasks.rados.rados.0.trial115.stdout:1226: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:53:08.590 INFO:tasks.rados.rados.0.trial115.stdout:1222: finishing rollback tid 1 to trial11545420-3 2026-02-17T05:53:08.590 INFO:tasks.rados.rados.0.trial115.stdout:1222: finishing rollback tid 2 to trial11545420-3 2026-02-17T05:53:08.590 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 615 (ObjNum 196 snap 37 seq_num 196) dirty exists 2026-02-17T05:53:08.590 INFO:tasks.rados.rados.0.trial115.stdout:1227: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:53:08.671 INFO:tasks.rados.rados.0.trial115.stdout:1221: finishing copy_from racing read to trial11545420-26 2026-02-17T05:53:08.671 INFO:tasks.rados.rados.0.trial115.stdout:1222: done (9 left) 2026-02-17T05:53:08.671 INFO:tasks.rados.rados.0.trial115.stdout:1223: done (8 left) 2026-02-17T05:53:08.671 INFO:tasks.rados.rados.0.trial115.stdout:1224: done (7 left) 2026-02-17T05:53:08.671 INFO:tasks.rados.rados.0.trial115.stdout:1225: done (6 left) 2026-02-17T05:53:08.671 INFO:tasks.rados.rados.0.trial115.stdout:1226: done (5 left) 2026-02-17T05:53:08.672 INFO:tasks.rados.rados.0.trial115.stdout:1227: done (4 left) 2026-02-17T05:53:08.672 INFO:tasks.rados.rados.0.trial115.stdout:1229: done (3 left) 2026-02-17T05:53:08.672 INFO:tasks.rados.rados.0.trial115.stdout:1232: write oid 27 current snap is 122 2026-02-17T05:53:08.672 INFO:tasks.rados.rados.0.trial115.stdout:1228: expect (ObjNum 423 snap 107 seq_num 423) 2026-02-17T05:53:08.747 INFO:tasks.rados.rados.0.trial115.stdout:1221: finishing copy_from to trial11545420-26 2026-02-17T05:53:08.747 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 191 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:53:08.747 INFO:tasks.rados.rados.0.trial115.stdout:1231: finishing copy_from racing read to trial11545420-50 2026-02-17T05:53:08.747 INFO:tasks.rados.rados.0.trial115.stdout:1231: finishing copy_from to trial11545420-50 2026-02-17T05:53:08.747 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 308 (ObjNum 469 snap 120 seq_num 469) dirty exists 2026-02-17T05:53:08.747 INFO:tasks.rados.rados.0.trial115.stdout:1232: seq_num 473 ranges {749711=406663,1778924=682482,3189899=716015} 2026-02-17T05:53:08.751 INFO:tasks.rados.rados.0.trial115.stdout:1232: writing trial11545420-27 from 749711 to 1156374 tid 1 2026-02-17T05:53:08.756 INFO:tasks.rados.rados.0.trial115.stdout:1232: writing trial11545420-27 from 1778924 to 2461406 tid 2 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1232: writing trial11545420-27 from 3189899 to 3905914 tid 3 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1221: done (3 left) 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1228: done (2 left) 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1231: done (1 left) 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1233: read oid 24 snap -1 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1233: expect deleted 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1234: rollback oid 20 current snap is 122 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 115 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1232: finishing write tid 1 to trial11545420-27 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1232: finishing write tid 2 to trial11545420-27 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1235: copy_from oid 33 from oid 7 current snap is 122 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1236: write (excl) oid 12 current snap is 122 2026-02-17T05:53:08.762 INFO:tasks.rados.rados.0.trial115.stdout:1236: seq_num 474 ranges {766518=481912,1837406=542493,2933675=527287,3876584=89934} 2026-02-17T05:53:08.767 INFO:tasks.rados.rados.0.trial115.stdout:1236: writing trial11545420-12 from 766518 to 1248430 tid 1 2026-02-17T05:53:08.771 INFO:tasks.rados.rados.0.trial115.stdout:1236: writing trial11545420-12 from 1837406 to 2379899 tid 2 2026-02-17T05:53:08.775 INFO:tasks.rados.rados.0.trial115.stdout:1236: writing trial11545420-12 from 2933675 to 3460962 tid 3 2026-02-17T05:53:08.780 INFO:tasks.rados.rados.0.trial115.stdout:1236: writing trial11545420-12 from 3876584 to 3966518 tid 4 2026-02-17T05:53:08.780 INFO:tasks.rados.rados.0.trial115.stdout:1237: read oid 46 snap 110 2026-02-17T05:53:08.780 INFO:tasks.rados.rados.0.trial115.stdout:1237: expect (ObjNum 397 snap 102 seq_num 397) 2026-02-17T05:53:08.780 INFO:tasks.rados.rados.0.trial115.stdout:1238: read oid 42 snap 108 2026-02-17T05:53:08.780 INFO:tasks.rados.rados.0.trial115.stdout:1238: expect (ObjNum 432 snap 108 seq_num 432) 2026-02-17T05:53:08.780 INFO:tasks.rados.rados.0.trial115.stdout:1239: read oid 11 snap -1 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1239: expect (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1240: snap_create 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1232: finishing write tid 3 to trial11545420-27 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1232: finishing write tid 4 to trial11545420-27 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1232: finishing write tid 5 to trial11545420-27 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1232: oid 27 updating version 0 to 541 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1232: oid 27 updating version 541 to 543 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1232: oid 27 version 543 is already newer than 542 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1232: oid 27 updating version 543 to 544 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 544 (ObjNum 473 snap 122 seq_num 473) dirty exists 2026-02-17T05:53:08.781 INFO:tasks.rados.rados.0.trial115.stdout:1232: left oid 27 (ObjNum 473 snap 122 seq_num 473) 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1234: finishing rollback tid 0 to trial11545420-20 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1234: finishing rollback tid 1 to trial11545420-20 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 472 (ObjNum 33 snap 0 seq_num 1334566784) dirty dne 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1236: finishing write tid 1 to trial11545420-12 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1235: finishing copy_from racing read to trial11545420-33 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1235: finishing copy_from to trial11545420-33 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 425 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1236: finishing write tid 2 to trial11545420-12 2026-02-17T05:53:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1236: finishing write tid 3 to trial11545420-12 2026-02-17T05:53:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1236: finishing write tid 4 to trial11545420-12 2026-02-17T05:53:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1236: finishing write tid 5 to trial11545420-12 2026-02-17T05:53:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1236: finishing write tid 6 to trial11545420-12 2026-02-17T05:53:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1236: oid 12 updating version 0 to 700 2026-02-17T05:53:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1236: oid 12 version 700 is already newer than 698 2026-02-17T05:53:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1236: oid 12 version 700 is already newer than 699 2026-02-17T05:53:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1236: oid 12 version 700 is already newer than 696 2026-02-17T05:53:08.784 INFO:tasks.rados.rados.0.trial115.stdout:1236: oid 12 version 700 is already newer than 697 2026-02-17T05:53:08.784 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 700 (ObjNum 474 snap 122 seq_num 474) dirty exists 2026-02-17T05:53:08.784 INFO:tasks.rados.rados.0.trial115.stdout:1236: left oid 12 (ObjNum 474 snap 122 seq_num 474) 2026-02-17T05:53:08.784 INFO:tasks.rados.rados.0.trial115.stdout:1237: expect (ObjNum 397 snap 102 seq_num 397) 2026-02-17T05:53:08.864 INFO:tasks.rados.rados.0.trial115.stdout:1238: expect (ObjNum 432 snap 108 seq_num 432) 2026-02-17T05:53:09.068 INFO:tasks.rados.rados.0.trial115.stdout:1239: expect (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:53:09.543 INFO:tasks.rados.rados.0.trial115.stdout:1232: done (8 left) 2026-02-17T05:53:09.544 INFO:tasks.rados.rados.0.trial115.stdout:1233: done (7 left) 2026-02-17T05:53:09.544 INFO:tasks.rados.rados.0.trial115.stdout:1234: done (6 left) 2026-02-17T05:53:09.544 INFO:tasks.rados.rados.0.trial115.stdout:1235: done (5 left) 2026-02-17T05:53:09.544 INFO:tasks.rados.rados.0.trial115.stdout:1236: done (4 left) 2026-02-17T05:53:09.545 INFO:tasks.rados.rados.0.trial115.stdout:1237: done (3 left) 2026-02-17T05:53:09.545 INFO:tasks.rados.rados.0.trial115.stdout:1238: done (2 left) 2026-02-17T05:53:09.545 INFO:tasks.rados.rados.0.trial115.stdout:1239: done (1 left) 2026-02-17T05:53:09.545 INFO:tasks.rados.rados.0.trial115.stdout:1240: done (0 left) 2026-02-17T05:53:09.545 INFO:tasks.rados.rados.0.trial115.stdout:1241: delete oid 24 current snap is 123 2026-02-17T05:53:09.546 INFO:tasks.rados.rados.0.trial115.stdout:1241: done (0 left) 2026-02-17T05:53:09.546 INFO:tasks.rados.rados.0.trial115.stdout:1242: snap_create 2026-02-17T05:53:10.544 INFO:tasks.rados.rados.0.trial115.stdout:1242: done (0 left) 2026-02-17T05:53:10.544 INFO:tasks.rados.rados.0.trial115.stdout:1243: snap_remove snap 119 2026-02-17T05:53:11.379 INFO:tasks.rados.rados.0.trial115.stdout:1243: done (0 left) 2026-02-17T05:53:11.379 INFO:tasks.rados.rados.0.trial115.stdout:1244: copy_from oid 12 from oid 20 current snap is 124 2026-02-17T05:53:11.379 INFO:tasks.rados.rados.0.trial115.stdout:1245: copy_from oid 13 from oid 30 current snap is 124 2026-02-17T05:53:11.379 INFO:tasks.rados.rados.0.trial115.stdout:1246: snap_remove snap 108 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:1246: done (2 left) 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:1247: rollback oid 40 current snap is 124 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 123 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:1244: finishing copy_from to trial11545420-12 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:1244: got expected ENOENT (src dne) 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:1244: finishing copy_from racing read to trial11545420-12 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:1245: finishing copy_from racing read to trial11545420-13 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:1245: finishing copy_from to trial11545420-13 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 525 (ObjNum 464 snap 117 seq_num 464) dirty exists 2026-02-17T05:53:12.380 INFO:tasks.rados.rados.0.trial115.stdout:1244: done (2 left) 2026-02-17T05:53:12.381 INFO:tasks.rados.rados.0.trial115.stdout:1245: done (1 left) 2026-02-17T05:53:12.381 INFO:tasks.rados.rados.0.trial115.stdout:1248: delete oid 28 current snap is 124 2026-02-17T05:53:12.381 INFO:tasks.rados.rados.0.trial115.stdout:1248: done (1 left) 2026-02-17T05:53:12.381 INFO:tasks.rados.rados.0.trial115.stdout:1249: write oid 28 current snap is 124 2026-02-17T05:53:12.381 INFO:tasks.rados.rados.0.trial115.stdout:1249: seq_num 477 ranges {416939=707659,1896499=120440} 2026-02-17T05:53:12.386 INFO:tasks.rados.rados.0.trial115.stdout:1249: writing trial11545420-28 from 416939 to 1124598 tid 1 2026-02-17T05:53:12.389 INFO:tasks.rados.rados.0.trial115.stdout:1249: writing trial11545420-28 from 1896499 to 2016939 tid 2 2026-02-17T05:53:12.390 INFO:tasks.rados.rados.0.trial115.stdout:1250: snap_remove snap 116 2026-02-17T05:53:13.412 INFO:tasks.rados.rados.0.trial115.stdout:1250: done (2 left) 2026-02-17T05:53:13.412 INFO:tasks.rados.rados.0.trial115.stdout:1251: read oid 23 snap 115 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:1251: expect deleted 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:1252: rollback oid 22 current snap is 124 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 22 to 111 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:1247: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:1247: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:1247: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 318 (ObjNum 451 snap 112 seq_num 451) dirty exists 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:1249: finishing write tid 1 to trial11545420-28 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:1249: finishing write tid 2 to trial11545420-28 2026-02-17T05:53:13.413 INFO:tasks.rados.rados.0.trial115.stdout:1249: finishing write tid 3 to trial11545420-28 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1249: finishing write tid 4 to trial11545420-28 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1249: oid 28 updating version 0 to 602 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1249: oid 28 version 602 is already newer than 601 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1249: oid 28 version 602 is already newer than 600 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 602 (ObjNum 477 snap 124 seq_num 477) dirty exists 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1249: left oid 28 (ObjNum 477 snap 124 seq_num 477) 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1247: done (3 left) 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1249: done (2 left) 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1253: write (excl) oid 45 current snap is 124 2026-02-17T05:53:13.414 INFO:tasks.rados.rados.0.trial115.stdout:1253: seq_num 478 ranges {433746=782908,1954981=78765} 2026-02-17T05:53:13.416 INFO:tasks.rados.rados.0.trial115.stdout:1253: writing trial11545420-45 from 433746 to 1216654 tid 1 2026-02-17T05:53:13.419 INFO:tasks.rados.rados.0.trial115.stdout:1253: writing trial11545420-45 from 1954981 to 2033746 tid 2 2026-02-17T05:53:13.420 INFO:tasks.rados.rados.0.trial115.stdout:1254: write oid 47 current snap is 124 2026-02-17T05:53:13.420 INFO:tasks.rados.rados.0.trial115.stdout:1254: seq_num 479 ranges {450553=574510,1613463=437090} 2026-02-17T05:53:13.423 INFO:tasks.rados.rados.0.trial115.stdout:1254: writing trial11545420-47 from 450553 to 1025063 tid 1 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1254: writing trial11545420-47 from 1613463 to 2050553 tid 2 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1255: copy_from oid 3 from oid 36 current snap is 124 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1252: finishing rollback tid 0 to trial11545420-22 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1252: finishing rollback tid 1 to trial11545420-22 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 549 (ObjNum 129 snap 0 seq_num 1333942936) dirty dne 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1253: finishing write tid 1 to trial11545420-45 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1253: finishing write tid 2 to trial11545420-45 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1253: finishing write tid 3 to trial11545420-45 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1253: finishing write tid 4 to trial11545420-45 2026-02-17T05:53:13.428 INFO:tasks.rados.rados.0.trial115.stdout:1253: oid 45 updating version 0 to 713 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1253: oid 45 version 713 is already newer than 712 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1253: oid 45 version 713 is already newer than 711 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 713 (ObjNum 478 snap 124 seq_num 478) dirty exists 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1253: left oid 45 (ObjNum 478 snap 124 seq_num 478) 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1254: finishing write tid 1 to trial11545420-47 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1251: done (4 left) 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1252: done (3 left) 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1253: done (2 left) 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1256: read oid 38 snap -1 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1256: expect deleted 2026-02-17T05:53:13.429 INFO:tasks.rados.rados.0.trial115.stdout:1257: snap_remove snap 114 2026-02-17T05:53:14.414 INFO:tasks.rados.rados.0.trial115.stdout:1257: done (3 left) 2026-02-17T05:53:14.414 INFO:tasks.rados.rados.0.trial115.stdout:1258: read oid 26 snap -1 2026-02-17T05:53:14.414 INFO:tasks.rados.rados.0.trial115.stdout:1258: expect (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:53:14.414 INFO:tasks.rados.rados.0.trial115.stdout:1259: snap_remove snap 121 2026-02-17T05:53:15.973 INFO:tasks.rados.rados.0.trial115.stdout:1259: done (4 left) 2026-02-17T05:53:15.973 INFO:tasks.rados.rados.0.trial115.stdout:1260: write (excl) oid 24 current snap is 124 2026-02-17T05:53:15.973 INFO:tasks.rados.rados.0.trial115.stdout:1260: seq_num 480 ranges {467360=649759,1671945=395415} 2026-02-17T05:53:15.977 INFO:tasks.rados.rados.0.trial115.stdout:1260: writing trial11545420-24 from 467360 to 1117119 tid 1 2026-02-17T05:53:15.981 INFO:tasks.rados.rados.0.trial115.stdout:1260: writing trial11545420-24 from 1671945 to 2067360 tid 2 2026-02-17T05:53:15.981 INFO:tasks.rados.rados.0.trial115.stdout:1261: write (excl) oid 21 current snap is 124 2026-02-17T05:53:15.981 INFO:tasks.rados.rados.0.trial115.stdout:1261: seq_num 481 ranges {484167=725008,1730427=353740} 2026-02-17T05:53:15.986 INFO:tasks.rados.rados.0.trial115.stdout:1261: writing trial11545420-21 from 484167 to 1209175 tid 1 2026-02-17T05:53:15.989 INFO:tasks.rados.rados.0.trial115.stdout:1261: writing trial11545420-21 from 1730427 to 2084167 tid 2 2026-02-17T05:53:15.989 INFO:tasks.rados.rados.0.trial115.stdout:1262: rollback oid 12 current snap is 124 2026-02-17T05:53:15.989 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 122 2026-02-17T05:53:15.989 INFO:tasks.rados.rados.0.trial115.stdout:1254: finishing write tid 2 to trial11545420-47 2026-02-17T05:53:15.989 INFO:tasks.rados.rados.0.trial115.stdout:1255: finishing copy_from to trial11545420-3 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1255: got expected ENOENT (src dne) 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1255: finishing copy_from racing read to trial11545420-3 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1254: finishing write tid 3 to trial11545420-47 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1255: done (6 left) 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1256: done (5 left) 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1263: snap_create 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1254: finishing write tid 4 to trial11545420-47 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1254: oid 47 updating version 0 to 636 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1254: oid 47 version 636 is already newer than 634 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1254: oid 47 version 636 is already newer than 635 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 636 (ObjNum 479 snap 124 seq_num 479) dirty exists 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1254: left oid 47 (ObjNum 479 snap 124 seq_num 479) 2026-02-17T05:53:15.990 INFO:tasks.rados.rados.0.trial115.stdout:1258: expect (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:53:16.143 INFO:tasks.rados.rados.0.trial115.stdout:1260: finishing write tid 1 to trial11545420-24 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1260: finishing write tid 2 to trial11545420-24 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1260: finishing write tid 3 to trial11545420-24 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1260: finishing write tid 4 to trial11545420-24 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1260: oid 24 updating version 0 to 722 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1260: oid 24 version 722 is already newer than 721 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1260: oid 24 version 722 is already newer than 720 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 722 (ObjNum 480 snap 124 seq_num 480) dirty exists 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1260: left oid 24 (ObjNum 480 snap 124 seq_num 480) 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1261: finishing write tid 1 to trial11545420-21 2026-02-17T05:53:16.144 INFO:tasks.rados.rados.0.trial115.stdout:1261: finishing write tid 2 to trial11545420-21 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1261: finishing write tid 3 to trial11545420-21 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1261: finishing write tid 4 to trial11545420-21 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1261: oid 21 updating version 0 to 541 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1261: oid 21 version 541 is already newer than 540 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1261: oid 21 version 541 is already newer than 539 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 541 (ObjNum 481 snap 124 seq_num 481) dirty exists 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1261: left oid 21 (ObjNum 481 snap 124 seq_num 481) 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1262: finishing rollback tid 0 to trial11545420-12 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1262: finishing rollback tid 1 to trial11545420-12 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:1262: finishing rollback tid 2 to trial11545420-12 2026-02-17T05:53:16.145 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 726 (ObjNum 474 snap 122 seq_num 474) dirty exists 2026-02-17T05:53:16.379 INFO:tasks.rados.rados.0.trial115.stdout:1254: done (5 left) 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1258: done (4 left) 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1260: done (3 left) 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1261: done (2 left) 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1262: done (1 left) 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1263: done (0 left) 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1264: rollback oid 29 current snap is 125 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 111 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1265: read oid 50 snap 124 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1265: expect (ObjNum 469 snap 120 seq_num 469) 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1266: read oid 36 snap 122 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1266: expect deleted 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1267: rollback oid 17 current snap is 125 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 110 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1268: write (excl) oid 16 current snap is 125 2026-02-17T05:53:16.380 INFO:tasks.rados.rados.0.trial115.stdout:1268: seq_num 482 ranges {500974=400257,1388909=557169} 2026-02-17T05:53:16.382 INFO:tasks.rados.rados.0.trial115.stdout:1268: writing trial11545420-16 from 500974 to 901231 tid 1 2026-02-17T05:53:16.386 INFO:tasks.rados.rados.0.trial115.stdout:1268: writing trial11545420-16 from 1388909 to 1946078 tid 2 2026-02-17T05:53:16.387 INFO:tasks.rados.rados.0.trial115.stdout:1269: snap_create 2026-02-17T05:53:16.387 INFO:tasks.rados.rados.0.trial115.stdout:1264: finishing rollback tid 0 to trial11545420-29 2026-02-17T05:53:16.387 INFO:tasks.rados.rados.0.trial115.stdout:1264: finishing rollback tid 1 to trial11545420-29 2026-02-17T05:53:16.387 INFO:tasks.rados.rados.0.trial115.stdout:1264: finishing rollback tid 2 to trial11545420-29 2026-02-17T05:53:16.387 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 528 (ObjNum 424 snap 107 seq_num 424) dirty exists 2026-02-17T05:53:16.387 INFO:tasks.rados.rados.0.trial115.stdout:1267: finishing rollback tid 0 to trial11545420-17 2026-02-17T05:53:16.387 INFO:tasks.rados.rados.0.trial115.stdout:1267: finishing rollback tid 1 to trial11545420-17 2026-02-17T05:53:16.387 INFO:tasks.rados.rados.0.trial115.stdout:1267: finishing rollback tid 2 to trial11545420-17 2026-02-17T05:53:16.388 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 240 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:53:16.389 INFO:tasks.rados.rados.0.trial115.stdout:1265: expect (ObjNum 469 snap 120 seq_num 469) 2026-02-17T05:53:16.589 INFO:tasks.rados.rados.0.trial115.stdout:1268: finishing write tid 1 to trial11545420-16 2026-02-17T05:53:16.589 INFO:tasks.rados.rados.0.trial115.stdout:1268: finishing write tid 2 to trial11545420-16 2026-02-17T05:53:16.589 INFO:tasks.rados.rados.0.trial115.stdout:1268: finishing write tid 3 to trial11545420-16 2026-02-17T05:53:16.590 INFO:tasks.rados.rados.0.trial115.stdout:1268: finishing write tid 4 to trial11545420-16 2026-02-17T05:53:16.590 INFO:tasks.rados.rados.0.trial115.stdout:1268: oid 16 updating version 0 to 642 2026-02-17T05:53:16.590 INFO:tasks.rados.rados.0.trial115.stdout:1268: oid 16 updating version 642 to 643 2026-02-17T05:53:16.590 INFO:tasks.rados.rados.0.trial115.stdout:1268: oid 16 updating version 643 to 644 2026-02-17T05:53:16.590 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 644 (ObjNum 482 snap 125 seq_num 482) dirty exists 2026-02-17T05:53:16.590 INFO:tasks.rados.rados.0.trial115.stdout:1268: left oid 16 (ObjNum 482 snap 125 seq_num 482) 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1264: done (5 left) 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1265: done (4 left) 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1266: done (3 left) 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1267: done (2 left) 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1268: done (1 left) 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1269: done (0 left) 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1270: read oid 34 snap -1 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1270: expect deleted 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1271: write oid 1 current snap is 126 2026-02-17T05:53:17.380 INFO:tasks.rados.rados.0.trial115.stdout:1271: seq_num 483 ranges {517781=475506,1731038=386743} 2026-02-17T05:53:17.383 INFO:tasks.rados.rados.0.trial115.stdout:1271: writing trial11545420-1 from 517781 to 993287 tid 1 2026-02-17T05:53:17.388 INFO:tasks.rados.rados.0.trial115.stdout:1271: writing trial11545420-1 from 1731038 to 2117781 tid 2 2026-02-17T05:53:17.388 INFO:tasks.rados.rados.0.trial115.stdout:1272: snap_create 2026-02-17T05:53:17.390 INFO:tasks.rados.rados.0.trial115.stdout:1271: finishing write tid 1 to trial11545420-1 2026-02-17T05:53:17.390 INFO:tasks.rados.rados.0.trial115.stdout:1271: finishing write tid 2 to trial11545420-1 2026-02-17T05:53:17.390 INFO:tasks.rados.rados.0.trial115.stdout:1271: finishing write tid 3 to trial11545420-1 2026-02-17T05:53:17.391 INFO:tasks.rados.rados.0.trial115.stdout:1271: finishing write tid 4 to trial11545420-1 2026-02-17T05:53:17.391 INFO:tasks.rados.rados.0.trial115.stdout:1271: oid 1 updating version 0 to 491 2026-02-17T05:53:17.391 INFO:tasks.rados.rados.0.trial115.stdout:1271: oid 1 updating version 491 to 492 2026-02-17T05:53:17.391 INFO:tasks.rados.rados.0.trial115.stdout:1271: oid 1 updating version 492 to 493 2026-02-17T05:53:17.391 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 493 (ObjNum 483 snap 126 seq_num 483) dirty exists 2026-02-17T05:53:17.391 INFO:tasks.rados.rados.0.trial115.stdout:1271: left oid 1 (ObjNum 483 snap 126 seq_num 483) 2026-02-17T05:53:18.382 INFO:tasks.rados.rados.0.trial115.stdout:1270: done (2 left) 2026-02-17T05:53:18.382 INFO:tasks.rados.rados.0.trial115.stdout:1271: done (1 left) 2026-02-17T05:53:18.382 INFO:tasks.rados.rados.0.trial115.stdout:1272: done (0 left) 2026-02-17T05:53:18.383 INFO:tasks.rados.rados.0.trial115.stdout:1273: copy_from oid 49 from oid 48 current snap is 127 2026-02-17T05:53:18.383 INFO:tasks.rados.rados.0.trial115.stdout:1274: snap_remove snap 110 2026-02-17T05:53:19.383 INFO:tasks.rados.rados.0.trial115.stdout:1274: done (1 left) 2026-02-17T05:53:19.383 INFO:tasks.rados.rados.0.trial115.stdout:1275: read oid 16 snap 111 2026-02-17T05:53:19.383 INFO:tasks.rados.rados.0.trial115.stdout:1275: expect (ObjNum 435 snap 109 seq_num 435) 2026-02-17T05:53:19.383 INFO:tasks.rados.rados.0.trial115.stdout:1276: snap_remove snap 124 2026-02-17T05:53:20.384 INFO:tasks.rados.rados.0.trial115.stdout:1276: done (2 left) 2026-02-17T05:53:20.384 INFO:tasks.rados.rados.0.trial115.stdout:1277: write (excl) oid 21 current snap is 127 2026-02-17T05:53:20.384 INFO:tasks.rados.rados.0.trial115.stdout:1277: seq_num 484 ranges {534588=550755,1789520=345068} 2026-02-17T05:53:20.388 INFO:tasks.rados.rados.0.trial115.stdout:1277: writing trial11545420-21 from 534588 to 1085343 tid 1 2026-02-17T05:53:20.393 INFO:tasks.rados.rados.0.trial115.stdout:1277: writing trial11545420-21 from 1789520 to 2134588 tid 2 2026-02-17T05:53:20.393 INFO:tasks.rados.rados.0.trial115.stdout:1278: write oid 36 current snap is 127 2026-02-17T05:53:20.393 INFO:tasks.rados.rados.0.trial115.stdout:1278: seq_num 485 ranges {551395=626004,1848002=303393} 2026-02-17T05:53:20.397 INFO:tasks.rados.rados.0.trial115.stdout:1278: writing trial11545420-36 from 551395 to 1177399 tid 1 2026-02-17T05:53:20.401 INFO:tasks.rados.rados.0.trial115.stdout:1278: writing trial11545420-36 from 1848002 to 2151395 tid 2 2026-02-17T05:53:20.401 INFO:tasks.rados.rados.0.trial115.stdout:1279: rollback oid 29 current snap is 127 2026-02-17T05:53:20.401 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 123 2026-02-17T05:53:20.401 INFO:tasks.rados.rados.0.trial115.stdout:1273: finishing copy_from racing read to trial11545420-49 2026-02-17T05:53:20.402 INFO:tasks.rados.rados.0.trial115.stdout:1273: finishing copy_from to trial11545420-49 2026-02-17T05:53:20.402 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 176 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:53:20.402 INFO:tasks.rados.rados.0.trial115.stdout:1275: expect (ObjNum 435 snap 109 seq_num 435) 2026-02-17T05:53:20.561 INFO:tasks.rados.rados.0.trial115.stdout:1277: finishing write tid 1 to trial11545420-21 2026-02-17T05:53:20.562 INFO:tasks.rados.rados.0.trial115.stdout:1277: finishing write tid 2 to trial11545420-21 2026-02-17T05:53:20.562 INFO:tasks.rados.rados.0.trial115.stdout:1277: finishing write tid 3 to trial11545420-21 2026-02-17T05:53:20.562 INFO:tasks.rados.rados.0.trial115.stdout:1277: finishing write tid 4 to trial11545420-21 2026-02-17T05:53:20.562 INFO:tasks.rados.rados.0.trial115.stdout:1277: oid 21 updating version 0 to 550 2026-02-17T05:53:20.562 INFO:tasks.rados.rados.0.trial115.stdout:1277: oid 21 updating version 550 to 551 2026-02-17T05:53:20.562 INFO:tasks.rados.rados.0.trial115.stdout:1277: oid 21 version 551 is already newer than 549 2026-02-17T05:53:20.562 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 551 (ObjNum 484 snap 127 seq_num 484) dirty exists 2026-02-17T05:53:20.562 INFO:tasks.rados.rados.0.trial115.stdout:1277: left oid 21 (ObjNum 484 snap 127 seq_num 484) 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1278: finishing write tid 1 to trial11545420-36 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1278: finishing write tid 2 to trial11545420-36 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1278: finishing write tid 3 to trial11545420-36 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1278: finishing write tid 4 to trial11545420-36 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1278: oid 36 updating version 0 to 504 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1278: oid 36 updating version 504 to 506 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1278: oid 36 version 506 is already newer than 505 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 506 (ObjNum 485 snap 127 seq_num 485) dirty exists 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1278: left oid 36 (ObjNum 485 snap 127 seq_num 485) 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1279: finishing rollback tid 0 to trial11545420-29 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1279: finishing rollback tid 1 to trial11545420-29 2026-02-17T05:53:20.563 INFO:tasks.rados.rados.0.trial115.stdout:1279: finishing rollback tid 2 to trial11545420-29 2026-02-17T05:53:20.564 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 531 (ObjNum 424 snap 107 seq_num 424) dirty exists 2026-02-17T05:53:20.564 INFO:tasks.rados.rados.0.trial115.stdout:1273: done (4 left) 2026-02-17T05:53:20.564 INFO:tasks.rados.rados.0.trial115.stdout:1275: done (3 left) 2026-02-17T05:53:20.564 INFO:tasks.rados.rados.0.trial115.stdout:1277: done (2 left) 2026-02-17T05:53:20.564 INFO:tasks.rados.rados.0.trial115.stdout:1278: done (1 left) 2026-02-17T05:53:20.564 INFO:tasks.rados.rados.0.trial115.stdout:1279: done (0 left) 2026-02-17T05:53:20.564 INFO:tasks.rados.rados.0.trial115.stdout:1280: delete oid 31 current snap is 127 2026-02-17T05:53:20.565 INFO:tasks.rados.rados.0.trial115.stdout:1280: done (0 left) 2026-02-17T05:53:20.565 INFO:tasks.rados.rados.0.trial115.stdout:1281: read oid 1 snap 111 2026-02-17T05:53:20.565 INFO:tasks.rados.rados.0.trial115.stdout:1281: expect (ObjNum 378 snap 99 seq_num 378) 2026-02-17T05:53:20.565 INFO:tasks.rados.rados.0.trial115.stdout:1282: write (excl) oid 6 current snap is 127 2026-02-17T05:53:20.565 INFO:tasks.rados.rados.0.trial115.stdout:1282: seq_num 487 ranges {585009=492855,1564966=620043} 2026-02-17T05:53:20.566 INFO:tasks.rados.rados.0.trial115.stdout:1282: writing trial11545420-6 from 585009 to 1077864 tid 1 2026-02-17T05:53:20.571 INFO:tasks.rados.rados.0.trial115.stdout:1282: writing trial11545420-6 from 1564966 to 2185009 tid 2 2026-02-17T05:53:20.571 INFO:tasks.rados.rados.0.trial115.stdout:1283: rollback oid 15 current snap is 127 2026-02-17T05:53:20.571 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 15 to 118 2026-02-17T05:53:20.571 INFO:tasks.rados.rados.0.trial115.stdout:1281: expect (ObjNum 378 snap 99 seq_num 378) 2026-02-17T05:53:20.627 INFO:tasks.rados.rados.0.trial115.stdout:1282: finishing write tid 1 to trial11545420-6 2026-02-17T05:53:20.627 INFO:tasks.rados.rados.0.trial115.stdout:1281: done (2 left) 2026-02-17T05:53:20.627 INFO:tasks.rados.rados.0.trial115.stdout:1284: write oid 2 current snap is 127 2026-02-17T05:53:20.628 INFO:tasks.rados.rados.0.trial115.stdout:1284: seq_num 488 ranges {601816=568104,1623448=451823} 2026-02-17T05:53:20.632 INFO:tasks.rados.rados.0.trial115.stdout:1284: writing trial11545420-2 from 601816 to 1169920 tid 1 2026-02-17T05:53:20.636 INFO:tasks.rados.rados.0.trial115.stdout:1284: writing trial11545420-2 from 1623448 to 2075271 tid 2 2026-02-17T05:53:20.636 INFO:tasks.rados.rados.0.trial115.stdout:1285: delete oid 34 current snap is 127 2026-02-17T05:53:20.636 INFO:tasks.rados.rados.0.trial115.stdout:1282: finishing write tid 2 to trial11545420-6 2026-02-17T05:53:20.636 INFO:tasks.rados.rados.0.trial115.stdout:1282: finishing write tid 3 to trial11545420-6 2026-02-17T05:53:20.636 INFO:tasks.rados.rados.0.trial115.stdout:1282: finishing write tid 4 to trial11545420-6 2026-02-17T05:53:20.636 INFO:tasks.rados.rados.0.trial115.stdout:1282: oid 6 updating version 0 to 427 2026-02-17T05:53:20.636 INFO:tasks.rados.rados.0.trial115.stdout:1282: oid 6 updating version 427 to 428 2026-02-17T05:53:20.636 INFO:tasks.rados.rados.0.trial115.stdout:1282: oid 6 version 428 is already newer than 426 2026-02-17T05:53:20.637 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 428 (ObjNum 487 snap 127 seq_num 487) dirty exists 2026-02-17T05:53:20.637 INFO:tasks.rados.rados.0.trial115.stdout:1282: left oid 6 (ObjNum 487 snap 127 seq_num 487) 2026-02-17T05:53:20.637 INFO:tasks.rados.rados.0.trial115.stdout:1283: finishing rollback tid 0 to trial11545420-15 2026-02-17T05:53:20.637 INFO:tasks.rados.rados.0.trial115.stdout:1283: finishing rollback tid 1 to trial11545420-15 2026-02-17T05:53:20.637 INFO:tasks.rados.rados.0.trial115.stdout:1283: finishing rollback tid 2 to trial11545420-15 2026-02-17T05:53:20.637 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 626 (ObjNum 450 snap 112 seq_num 450) dirty exists 2026-02-17T05:53:20.637 INFO:tasks.rados.rados.0.trial115.stdout:1284: finishing write tid 1 to trial11545420-2 2026-02-17T05:53:20.637 INFO:tasks.rados.rados.0.trial115.stdout:1282: done (3 left) 2026-02-17T05:53:20.638 INFO:tasks.rados.rados.0.trial115.stdout:1283: done (2 left) 2026-02-17T05:53:20.638 INFO:tasks.rados.rados.0.trial115.stdout:1285: done (1 left) 2026-02-17T05:53:20.638 INFO:tasks.rados.rados.0.trial115.stdout:1286: write (excl) oid 50 current snap is 127 2026-02-17T05:53:20.638 INFO:tasks.rados.rados.0.trial115.stdout:1286: seq_num 490 ranges {635430=718602,2140412=95018} 2026-02-17T05:53:20.641 INFO:tasks.rados.rados.0.trial115.stdout:1286: writing trial11545420-50 from 635430 to 1354032 tid 1 2026-02-17T05:53:20.644 INFO:tasks.rados.rados.0.trial115.stdout:1286: writing trial11545420-50 from 2140412 to 2235430 tid 2 2026-02-17T05:53:20.644 INFO:tasks.rados.rados.0.trial115.stdout:1287: write oid 6 current snap is 127 2026-02-17T05:53:20.644 INFO:tasks.rados.rados.0.trial115.stdout:1287: seq_num 491 ranges {652237=793851,2198894=53343} 2026-02-17T05:53:20.650 INFO:tasks.rados.rados.0.trial115.stdout:1287: writing trial11545420-6 from 652237 to 1446088 tid 1 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1287: writing trial11545420-6 from 2198894 to 2252237 tid 2 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1288: read oid 35 snap 122 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1288: expect (ObjNum 459 snap 116 seq_num 459) 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1289: write oid 36 current snap is 127 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1284: finishing write tid 2 to trial11545420-2 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1284: finishing write tid 3 to trial11545420-2 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1284: finishing write tid 4 to trial11545420-2 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1284: oid 2 updating version 0 to 651 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1284: oid 2 updating version 651 to 652 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1284: oid 2 version 652 is already newer than 650 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 652 (ObjNum 488 snap 127 seq_num 488) dirty exists 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1284: left oid 2 (ObjNum 488 snap 127 seq_num 488) 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1286: finishing write tid 1 to trial11545420-50 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1286: finishing write tid 2 to trial11545420-50 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1286: finishing write tid 3 to trial11545420-50 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1286: finishing write tid 4 to trial11545420-50 2026-02-17T05:53:20.653 INFO:tasks.rados.rados.0.trial115.stdout:1286: oid 50 updating version 0 to 561 2026-02-17T05:53:20.654 INFO:tasks.rados.rados.0.trial115.stdout:1286: oid 50 version 561 is already newer than 560 2026-02-17T05:53:20.654 INFO:tasks.rados.rados.0.trial115.stdout:1286: oid 50 version 561 is already newer than 559 2026-02-17T05:53:20.654 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 561 (ObjNum 490 snap 127 seq_num 490) dirty exists 2026-02-17T05:53:20.654 INFO:tasks.rados.rados.0.trial115.stdout:1286: left oid 50 (ObjNum 490 snap 127 seq_num 490) 2026-02-17T05:53:20.654 INFO:tasks.rados.rados.0.trial115.stdout:1287: finishing write tid 1 to trial11545420-6 2026-02-17T05:53:20.654 INFO:tasks.rados.rados.0.trial115.stdout:1289: seq_num 492 ranges {669044=469100,1741023=459161} 2026-02-17T05:53:20.657 INFO:tasks.rados.rados.0.trial115.stdout:1289: writing trial11545420-36 from 669044 to 1138144 tid 1 2026-02-17T05:53:20.661 INFO:tasks.rados.rados.0.trial115.stdout:1289: writing trial11545420-36 from 1741023 to 2200184 tid 2 2026-02-17T05:53:20.661 INFO:tasks.rados.rados.0.trial115.stdout:1284: done (4 left) 2026-02-17T05:53:20.661 INFO:tasks.rados.rados.0.trial115.stdout:1286: done (3 left) 2026-02-17T05:53:20.661 INFO:tasks.rados.rados.0.trial115.stdout:1290: snap_remove snap 125 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1290: done (3 left) 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1291: copy_from oid 21 from oid 4 current snap is 127 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1287: finishing write tid 2 to trial11545420-6 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1287: finishing write tid 3 to trial11545420-6 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1287: finishing write tid 4 to trial11545420-6 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1287: oid 6 updating version 0 to 430 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1287: oid 6 updating version 430 to 431 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1287: oid 6 version 431 is already newer than 429 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 431 (ObjNum 491 snap 127 seq_num 491) dirty exists 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1287: left oid 6 (ObjNum 491 snap 127 seq_num 491) 2026-02-17T05:53:21.382 INFO:tasks.rados.rados.0.trial115.stdout:1288: expect (ObjNum 459 snap 116 seq_num 459) 2026-02-17T05:53:21.495 INFO:tasks.rados.rados.0.trial115.stdout:1289: finishing write tid 1 to trial11545420-36 2026-02-17T05:53:21.496 INFO:tasks.rados.rados.0.trial115.stdout:1289: finishing write tid 2 to trial11545420-36 2026-02-17T05:53:21.496 INFO:tasks.rados.rados.0.trial115.stdout:1289: finishing write tid 3 to trial11545420-36 2026-02-17T05:53:21.496 INFO:tasks.rados.rados.0.trial115.stdout:1289: finishing write tid 4 to trial11545420-36 2026-02-17T05:53:21.496 INFO:tasks.rados.rados.0.trial115.stdout:1289: oid 36 updating version 0 to 508 2026-02-17T05:53:21.496 INFO:tasks.rados.rados.0.trial115.stdout:1289: oid 36 updating version 508 to 509 2026-02-17T05:53:21.496 INFO:tasks.rados.rados.0.trial115.stdout:1289: oid 36 version 509 is already newer than 507 2026-02-17T05:53:21.496 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 509 (ObjNum 492 snap 127 seq_num 492) dirty exists 2026-02-17T05:53:21.497 INFO:tasks.rados.rados.0.trial115.stdout:1289: left oid 36 (ObjNum 492 snap 127 seq_num 492) 2026-02-17T05:53:21.497 INFO:tasks.rados.rados.0.trial115.stdout:1291: finishing copy_from racing read to trial11545420-21 2026-02-17T05:53:21.497 INFO:tasks.rados.rados.0.trial115.stdout:1287: done (3 left) 2026-02-17T05:53:21.497 INFO:tasks.rados.rados.0.trial115.stdout:1288: done (2 left) 2026-02-17T05:53:21.497 INFO:tasks.rados.rados.0.trial115.stdout:1289: done (1 left) 2026-02-17T05:53:21.497 INFO:tasks.rados.rados.0.trial115.stdout:1292: write oid 3 current snap is 127 2026-02-17T05:53:21.497 INFO:tasks.rados.rados.0.trial115.stdout:1291: finishing copy_from to trial11545420-21 2026-02-17T05:53:21.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 308 (ObjNum 469 snap 120 seq_num 469) dirty exists 2026-02-17T05:53:21.498 INFO:tasks.rados.rados.0.trial115.stdout:1292: seq_num 493 ranges {685851=544349,1799505=486346} 2026-02-17T05:53:21.499 INFO:tasks.rados.rados.0.trial115.stdout:1292: writing trial11545420-3 from 685851 to 1230200 tid 1 2026-02-17T05:53:21.504 INFO:tasks.rados.rados.0.trial115.stdout:1292: writing trial11545420-3 from 1799505 to 2285851 tid 2 2026-02-17T05:53:21.504 INFO:tasks.rados.rados.0.trial115.stdout:1291: done (1 left) 2026-02-17T05:53:21.504 INFO:tasks.rados.rados.0.trial115.stdout:1293: read oid 8 snap -1 2026-02-17T05:53:21.504 INFO:tasks.rados.rados.0.trial115.stdout:1293: expect (ObjNum 458 snap 116 seq_num 458) 2026-02-17T05:53:21.504 INFO:tasks.rados.rados.0.trial115.stdout:1294: snap_remove snap 118 2026-02-17T05:53:23.328 INFO:tasks.rados.rados.0.trial115.stdout:1294: done (2 left) 2026-02-17T05:53:23.328 INFO:tasks.rados.rados.0.trial115.stdout:1295: read oid 14 snap -1 2026-02-17T05:53:23.328 INFO:tasks.rados.rados.0.trial115.stdout:1295: expect (ObjNum 460 snap 116 seq_num 460) 2026-02-17T05:53:23.328 INFO:tasks.rados.rados.0.trial115.stdout:1296: snap_create 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1292: finishing write tid 1 to trial11545420-3 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1292: finishing write tid 2 to trial11545420-3 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1292: finishing write tid 3 to trial11545420-3 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1292: finishing write tid 4 to trial11545420-3 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1292: oid 3 updating version 0 to 659 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1292: oid 3 updating version 659 to 660 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1292: oid 3 version 660 is already newer than 658 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 660 (ObjNum 493 snap 127 seq_num 493) dirty exists 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1292: left oid 3 (ObjNum 493 snap 127 seq_num 493) 2026-02-17T05:53:23.329 INFO:tasks.rados.rados.0.trial115.stdout:1293: expect (ObjNum 458 snap 116 seq_num 458) 2026-02-17T05:53:23.475 INFO:tasks.rados.rados.0.trial115.stdout:1295: expect (ObjNum 460 snap 116 seq_num 460) 2026-02-17T05:53:24.330 INFO:tasks.rados.rados.0.trial115.stdout:1292: done (3 left) 2026-02-17T05:53:24.330 INFO:tasks.rados.rados.0.trial115.stdout:1293: done (2 left) 2026-02-17T05:53:24.330 INFO:tasks.rados.rados.0.trial115.stdout:1295: done (1 left) 2026-02-17T05:53:24.330 INFO:tasks.rados.rados.0.trial115.stdout:1296: done (0 left) 2026-02-17T05:53:24.330 INFO:tasks.rados.rados.0.trial115.stdout:1297: read oid 39 snap -1 2026-02-17T05:53:24.330 INFO:tasks.rados.rados.0.trial115.stdout:1297: expect (ObjNum 428 snap 108 seq_num 428) 2026-02-17T05:53:24.330 INFO:tasks.rados.rados.0.trial115.stdout:1298: snap_create 2026-02-17T05:53:24.336 INFO:tasks.rados.rados.0.trial115.stdout:1297: expect (ObjNum 428 snap 108 seq_num 428) 2026-02-17T05:53:25.331 INFO:tasks.rados.rados.0.trial115.stdout:1297: done (1 left) 2026-02-17T05:53:25.331 INFO:tasks.rados.rados.0.trial115.stdout:1298: done (0 left) 2026-02-17T05:53:25.331 INFO:tasks.rados.rados.0.trial115.stdout:1299: snap_remove snap 111 2026-02-17T05:53:26.330 INFO:tasks.rados.rados.0.trial115.stdout:1299: done (0 left) 2026-02-17T05:53:26.330 INFO:tasks.rados.rados.0.trial115.stdout:1300: snap_remove snap 127 2026-02-17T05:53:26.381 INFO:tasks.rados.rados.0.trial115.stdout:1300: done (0 left) 2026-02-17T05:53:26.381 INFO:tasks.rados.rados.0.trial115.stdout:1301: delete oid 44 current snap is 129 2026-02-17T05:53:26.383 INFO:tasks.rados.rados.0.trial115.stdout:1301: done (0 left) 2026-02-17T05:53:26.383 INFO:tasks.rados.rados.0.trial115.stdout:1302: snap_create 2026-02-17T05:53:26.786 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:53:26.787 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T05:53:26.787 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:53:26.787 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:53:26.788 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:53:26.788 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:53:26.789 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:53:26.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient: get_monmap_and_config 2026-02-17T05:53:26.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient: build_initial_monmap 2026-02-17T05:53:26.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient: monmap: 2026-02-17T05:53:26.859 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:26.859 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:26.858439+0000 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:26.858439+0000 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:26.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 1 Processor -- start 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 1 -- start start 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient: init 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 0x7ff58006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient(hunting): picked mon.noname-a con 0x7ff58006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff58009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff58006d1b0 0x7ff580135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient(hunting): picked mon.noname-c con 0x7ff58006d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58006daf0 con 0x7ff58009c4a0 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:26.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff5800a0880 con 0x7ff58006c890 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff58009cc80 con 0x7ff58006d1b0 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.857+0000 7ff58681a640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff58681a640 10 monclient(hunting): authenticate will time out at 1052.412006s 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 0x7ff58006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 0x7ff58006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38258/0 (socket says 10.20.193.78:38258) 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 1 -- 10.20.193.78:0/880343069 learned_addr learned my addr 10.20.193.78:0/880343069 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 10 monclient(hunting): get_auth_request con 0x7ff58006c890 auth_method 0 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:26.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 1 -- 10.20.193.78:0/880343069 >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 msgr2=0x7ff58006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 1 -- 10.20.193.78:0/880343069 >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 msgr2=0x7ff58006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 1 --2- 10.20.193.78:0/880343069 >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 0x7ff58006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff57ffff640 1 --2- 10.20.193.78:0/880343069 >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 0x7ff58006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 1 --2- 10.20.193.78:0/880343069 >> v2:10.20.193.115:3300/0 conn(0x7ff58006d1b0 0x7ff580135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient(hunting): get_auth_request con 0x7ff58006d1b0 auth_method 0 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:26.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient(hunting): handle_auth_done global_id 44554 payload 274 2026-02-17T05:53:26.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 1 -- 10.20.193.78:0/880343069 >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 msgr2=0x7ff58006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:26.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 1 --2- 10.20.193.78:0/880343069 >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 0x7ff58006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 1 -- 10.20.193.78:0/880343069 >> v1:10.20.193.115:6789/0 conn(0x7ff58009c4a0 legacy=0x7ff580134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:26.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient: _finish_hunting 0 2026-02-17T05:53:26.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 1 monclient: found mon.noname-c 2026-02-17T05:53:26.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:26.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:53:26.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.858+0000 7ff584d90640 1 -- 10.20.193.78:0/880343069 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58009cf60 con 0x7ff58006d1b0 2026-02-17T05:53:26.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff584d90640 1 -- 10.20.193.78:0/880343069 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 msgr2=0x7ff580135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:26.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff584d90640 1 --2- 10.20.193.78:0/880343069 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 0x7ff580135500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff580074de0 tx=0x7ff57402f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bce71bcade1fc67 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 1 -- 10.20.193.78:0/880343069 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57403c070 con 0x7ff58006d1b0 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: dump: 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:26.872 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: _finish_auth 0 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: _check_auth_tickets 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 5 monclient: authenticate success, global_id 44554 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 1 -- 10.20.193.78:0/880343069 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff574037440 con 0x7ff58006d1b0 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 1 -- 10.20.193.78:0/880343069 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574037810 con 0x7ff58006d1b0 2026-02-17T05:53:26.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 4 set_mon_vals no callback set 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff57effd640 10 monclient: dump: 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:26.874 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 10 monclient: shutdown 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 1 -- 10.20.193.78:0/880343069 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 msgr2=0x7ff580135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 1 --2- 10.20.193.78:0/880343069 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 0x7ff580135500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff580074de0 tx=0x7ff57402f130 comp rx=0 tx=0).stop 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 1 -- 10.20.193.78:0/880343069 shutdown_connections 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 1 --2- 10.20.193.78:0/880343069 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 0x7ff580135500 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 1 --2- 10.20.193.78:0/880343069 >> v2:10.20.193.78:3300/0 conn(0x7ff58006c890 0x7ff58006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.861+0000 7ff58681a640 1 -- 10.20.193.78:0/880343069 >> 10.20.193.78:0/880343069 conn(0x7ff5800a6620 msgr2=0x7ff5800a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 -- 10.20.193.78:0/880343069 shutdown_connections 2026-02-17T05:53:26.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 -- 10.20.193.78:0/880343069 wait complete. 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient: build_initial_monmap 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient: monmap: 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:26.876 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 Processor -- start 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 -- start start 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient: init 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 0x7ff5801cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient(hunting): picked mon.noname-a con 0x7ff58006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 0x7ff5801cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient(hunting): picked mon.noname-b con 0x7ff58006d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 0x7ff580123530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient(hunting): picked mon.noname-c con 0x7ff58009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:26.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800a01a0 con 0x7ff58006c890 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800a0020 con 0x7ff58006d1b0 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff58009be20 con 0x7ff58009c4a0 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff58681a640 10 monclient(hunting): authenticate will time out at 1052.416878s 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff57ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 0x7ff5801cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff584d90640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 0x7ff580123530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff57ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 0x7ff5801cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38260/0 (socket says 10.20.193.78:38260) 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff57ffff640 1 -- 10.20.193.78:0/1011551097 learned_addr learned my addr 10.20.193.78:0/1011551097 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:26.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff57ffff640 10 monclient(hunting): get_auth_request con 0x7ff58006c890 auth_method 0 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff57ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff57ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff57ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff584d90640 10 monclient(hunting): get_auth_request con 0x7ff58009c4a0 auth_method 0 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff584d90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff584d90640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff584d90640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.862+0000 7ff57ffff640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 msgr2=0x7ff5801cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57ffff640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 msgr2=0x7ff5801cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff584d90640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57ffff640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 0x7ff5801cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:26.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff584d90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff584d90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57ffff640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 0x7ff5801cd330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff584d90640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 msgr2=0x7ff580123530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff584d90640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 msgr2=0x7ff580123530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff584d90640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 0x7ff580123530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 0x7ff5801cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient(hunting): get_auth_request con 0x7ff58006d1b0 auth_method 0 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff584d90640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 0x7ff580123530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:26.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient(hunting): handle_auth_done global_id 44557 payload 824 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 msgr2=0x7ff580123530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 0x7ff580123530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 msgr2=0x7ff5801cd330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 0x7ff5801cd330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 1 monclient: found mon.noname-b 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:53:26.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.863+0000 7ff57f7fe640 1 -- 10.20.193.78:0/1011551097 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff580127470 con 0x7ff58006d1b0 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57f7fe640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 msgr2=0x7ff5801cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57f7fe640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 0x7ff5801cd870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c450 tx=0x7ff57000c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 1 -- 10.20.193.78:0/1011551097 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570016020 con 0x7ff58006d1b0 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: dump: 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:26.882 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: _finish_auth 0 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: _check_auth_tickets 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 1 -- 10.20.193.78:0/1011551097 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff570012a40 con 0x7ff58006d1b0 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 1 -- 10.20.193.78:0/1011551097 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570011380 con 0x7ff58006d1b0 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:26.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57e7fc640 4 set_mon_vals no callback set 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff57cff9640 10 monclient: dump: 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:26.884 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff58681a640 5 monclient: authenticate success, global_id 44557 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff58681a640 10 monclient: _renew_subs 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff58681a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58006fd40 con 0x7ff58006d1b0 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff58681a640 10 monclient: _renew_subs 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff58681a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.865+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff580127190 con 0x7ff58006d1b0 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.866+0000 7ff58681a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.866+0000 7ff58681a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.866+0000 7ff58681a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.866+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff580074980 con 0x7ff58006d1b0 2026-02-17T05:53:26.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.866+0000 7ff57cff9640 1 -- 10.20.193.78:0/1011551097 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff5700115e0 con 0x7ff58006d1b0 2026-02-17T05:53:26.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.866+0000 7ff57cff9640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55003dee0 0x7ff5500403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:26.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.867+0000 7ff57cff9640 1 -- 10.20.193.78:0/1011551097 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(279..279 src has 117..279) ==== 4839+0+0 (secure 0 0 0) 0x7ff570052860 con 0x7ff58006d1b0 2026-02-17T05:53:26.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.867+0000 7ff57ffff640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55003dee0 0x7ff5500403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:26.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.867+0000 7ff57ffff640 10 monclient: get_auth_request con 0x7ff55003dee0 auth_method 0 2026-02-17T05:53:26.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.869+0000 7ff57ffff640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55003dee0 0x7ff5500403a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff580124d80 tx=0x7ff56c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:26.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.869+0000 7ff57cff9640 1 -- 10.20.193.78:0/1011551097 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff570020030 con 0x7ff58006d1b0 2026-02-17T05:53:26.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.869+0000 7ff57cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:26.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.869+0000 7ff57cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:26.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.985+0000 7ff58681a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:53:26.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.985+0000 7ff58681a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:26.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.985+0000 7ff58681a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:53:26.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.985+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff58006cc70 con 0x7ff58006d1b0 2026-02-17T05:53:26.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff57cff9640 1 -- 10.20.193.78:0/1011551097 <== mon.1 v2:10.20.193.115:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7ff570054cb0 con 0x7ff58006d1b0 2026-02-17T05:53:26.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff57cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:26.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff57cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:26.986 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55003dee0 msgr2=0x7ff5500403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55003dee0 0x7ff5500403a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff580124d80 tx=0x7ff56c008040 comp rx=0 tx=0).stop 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 10 monclient: shutdown 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 msgr2=0x7ff5801cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 0x7ff5801cd870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c450 tx=0x7ff57000c920 comp rx=0 tx=0).stop 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 shutdown_connections 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58009c4a0 0x7ff580123530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55003dee0 0x7ff5500403a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58006d1b0 0x7ff5801cd870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 --2- 10.20.193.78:0/1011551097 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58006c890 0x7ff5801cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:26.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 >> 10.20.193.78:0/1011551097 conn(0x7ff5800a6620 msgr2=0x7ff580072200 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:26.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 shutdown_connections 2026-02-17T05:53:26.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:26.987+0000 7ff58681a640 1 -- 10.20.193.78:0/1011551097 wait complete. 2026-02-17T05:53:27.385 INFO:tasks.rados.rados.0.trial115.stdout:1302: done (0 left) 2026-02-17T05:53:27.385 INFO:tasks.rados.rados.0.trial115.stdout:1303: snap_remove snap 129 2026-02-17T05:53:29.995 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:53:30.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.063+0000 7fd7ccec6640 10 monclient: get_monmap_and_config 2026-02-17T05:53:30.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.063+0000 7fd7ccec6640 10 monclient: build_initial_monmap 2026-02-17T05:53:30.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.063+0000 7fd7ccec6640 10 monclient: monmap: 2026-02-17T05:53:30.065 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:30.065 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:30.065 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:30.064997+0000 2026-02-17T05:53:30.065 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:30.064997+0000 2026-02-17T05:53:30.065 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 1 Processor -- start 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 1 -- start start 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient: init 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:30.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient(hunting): picked mon.noname-c con 0x7fd7c806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd7c809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd7c806d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c8135c10 con 0x7fd7c809c4a0 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c806daf0 con 0x7fd7c806d1b0 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c80a0880 con 0x7fd7c806c890 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7ccec6640 10 monclient(hunting): authenticate will time out at 1055.618507s 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c5d74640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd7c809c4a0 0x7fd7c8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56980/0 (socket says 10.20.193.78:56980) 2026-02-17T05:53:30.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c5d74640 1 -- 10.20.193.78:0/3147120809 learned_addr learned my addr 10.20.193.78:0/3147120809 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 10 monclient(hunting): get_auth_request con 0x7fd7c806c890 auth_method 0 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:30.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.064+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:30.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.066+0000 7fd7c5d74640 1 -- 10.20.193.78:0/3147120809 >> v1:10.20.193.78:6789/0 conn(0x7fd7c809c4a0 legacy=0x7fd7c8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:30.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.067+0000 7fd7c6d76640 1 -- 10.20.193.78:0/3147120809 >> v1:10.20.193.115:6789/0 conn(0x7fd7c806d1b0 legacy=0x7fd7c8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:30.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:30.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 10 monclient(hunting): get_auth_request con 0x7fd7c806c890 auth_method 0 2026-02-17T05:53:30.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:30.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:30.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:30.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:30.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:30.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:30.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.265+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:53:30.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.665+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:30.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.666+0000 7fd7c6575640 10 monclient(hunting): get_auth_request con 0x7fd7c806c890 auth_method 0 2026-02-17T05:53:30.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.666+0000 7fd7c6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:30.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.666+0000 7fd7c6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:30.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.666+0000 7fd7c6575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:30.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.666+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:30.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.666+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:30.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.666+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:30.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:30.666+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:53:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 10 monclient(hunting): get_auth_request con 0x7fd7c806c890 auth_method 0 2026-02-17T05:53:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:31.467+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:53:33.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient(hunting): tick 2026-02-17T05:53:33.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:53:33.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 monclient(hunting): continuing hunt 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 msgr2=0x7fd7c806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.115:3300/0 conn(0x7fd7c806c890 0x7fd7c806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 -- 10.20.193.78:0/3147120809 >> v1:10.20.193.115:6789/0 conn(0x7fd7c806d1b0 legacy=0x7fd7c8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 -- 10.20.193.78:0/3147120809 >> v1:10.20.193.78:6789/0 conn(0x7fd7c809c4a0 legacy=0x7fd7c8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient: _add_conns ranks=[3,2] 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.78:3300/0 conn(0x7fd7a40038c0 0x7fd7a4005d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient(hunting): picked mon.noname-a con 0x7fd7a40038c0 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 0x7fd7a40088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient(hunting): picked mon.noname-b con 0x7fd7a40063e0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 -- 10.20.193.78:0/3147120809 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a4008fa0 con 0x7fd7a40038c0 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 1 -- 10.20.193.78:0/3147120809 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a4009150 con 0x7fd7a40063e0 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7affff640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.78:3300/0 conn(0x7fd7a40038c0 0x7fd7a4005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 10 monclient(hunting): get_auth_request con 0x7fd7a40038c0 auth_method 0 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 0x7fd7a40088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.78:3300/0 conn(0x7fd7a40038c0 msgr2=0x7fd7a4005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:33.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> v2:10.20.193.78:3300/0 conn(0x7fd7a40038c0 msgr2=0x7fd7a4005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.78:3300/0 conn(0x7fd7a40038c0 0x7fd7a4005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 10 monclient(hunting): get_auth_request con 0x7fd7a40063e0 auth_method 0 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.78:3300/0 conn(0x7fd7a40038c0 0x7fd7a4005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 1 -- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 msgr2=0x7fd7a40088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 1 -- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 msgr2=0x7fd7a40088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 0x7fd7a40088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:33.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.065+0000 7fd7c5d74640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 0x7fd7a40088a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:33.139 INFO:tasks.rados.rados.0.trial115.stdout:1303: done (0 left) 2026-02-17T05:53:33.139 INFO:tasks.rados.rados.0.trial115.stdout:1304: write oid 38 current snap is 130 2026-02-17T05:53:33.139 INFO:tasks.rados.rados.0.trial115.stdout:1304: seq_num 495 ranges {719465=411200,1632822=686643} 2026-02-17T05:53:33.143 INFO:tasks.rados.rados.0.trial115.stdout:1304: writing trial11545420-38 from 719465 to 1130665 tid 1 2026-02-17T05:53:33.147 INFO:tasks.rados.rados.0.trial115.stdout:1304: writing trial11545420-38 from 1632822 to 2319465 tid 2 2026-02-17T05:53:33.148 INFO:tasks.rados.rados.0.trial115.stdout:1305: write (excl) oid 26 current snap is 130 2026-02-17T05:53:33.148 INFO:tasks.rados.rados.0.trial115.stdout:1305: seq_num 496 ranges {736272=486449,1974951=361321} 2026-02-17T05:53:33.152 INFO:tasks.rados.rados.0.trial115.stdout:1305: writing trial11545420-26 from 736272 to 1222721 tid 1 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1305: writing trial11545420-26 from 1974951 to 2336272 tid 2 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1306: write (excl) oid 29 current snap is 130 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1304: finishing write tid 1 to trial11545420-38 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1304: finishing write tid 2 to trial11545420-38 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1304: finishing write tid 3 to trial11545420-38 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1304: finishing write tid 4 to trial11545420-38 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1304: oid 38 updating version 0 to 664 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1304: oid 38 updating version 664 to 665 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1304: oid 38 version 665 is already newer than 663 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 665 (ObjNum 495 snap 130 seq_num 495) dirty exists 2026-02-17T05:53:33.157 INFO:tasks.rados.rados.0.trial115.stdout:1304: left oid 38 (ObjNum 495 snap 130 seq_num 495) 2026-02-17T05:53:33.158 INFO:tasks.rados.rados.0.trial115.stdout:1305: finishing write tid 1 to trial11545420-26 2026-02-17T05:53:33.158 INFO:tasks.rados.rados.0.trial115.stdout:1306: seq_num 497 ranges {753079=561698,2033433=319646} 2026-02-17T05:53:33.161 INFO:tasks.rados.rados.0.trial115.stdout:1306: writing trial11545420-29 from 753079 to 1314777 tid 1 2026-02-17T05:53:33.165 INFO:tasks.rados.rados.0.trial115.stdout:1306: writing trial11545420-29 from 2033433 to 2353079 tid 2 2026-02-17T05:53:33.165 INFO:tasks.rados.rados.0.trial115.stdout:1304: done (2 left) 2026-02-17T05:53:33.166 INFO:tasks.rados.rados.0.trial115.stdout:1307: write (excl) oid 37 current snap is 130 2026-02-17T05:53:33.166 INFO:tasks.rados.rados.0.trial115.stdout:1307: seq_num 498 ranges {769886=636947,2091915=277971} 2026-02-17T05:53:33.170 INFO:tasks.rados.rados.0.trial115.stdout:1307: writing trial11545420-37 from 769886 to 1406833 tid 1 2026-02-17T05:53:33.174 INFO:tasks.rados.rados.0.trial115.stdout:1307: writing trial11545420-37 from 2091915 to 2369886 tid 2 2026-02-17T05:53:33.174 INFO:tasks.rados.rados.0.trial115.stdout:1308: write oid 45 current snap is 130 2026-02-17T05:53:33.174 INFO:tasks.rados.rados.0.trial115.stdout:1308: seq_num 499 ranges {786693=712196,2150397=236296} 2026-02-17T05:53:33.179 INFO:tasks.rados.rados.0.trial115.stdout:1308: writing trial11545420-45 from 786693 to 1498889 tid 1 2026-02-17T05:53:33.183 INFO:tasks.rados.rados.0.trial115.stdout:1308: writing trial11545420-45 from 2150397 to 2386693 tid 2 2026-02-17T05:53:33.183 INFO:tasks.rados.rados.0.trial115.stdout:1309: read oid 42 snap -1 2026-02-17T05:53:33.183 INFO:tasks.rados.rados.0.trial115.stdout:1309: expect (ObjNum 438 snap 110 seq_num 438) 2026-02-17T05:53:33.183 INFO:tasks.rados.rados.0.trial115.stdout:1310: read oid 33 snap -1 2026-02-17T05:53:33.184 INFO:tasks.rados.rados.0.trial115.stdout:1310: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:53:33.184 INFO:tasks.rados.rados.0.trial115.stdout:1311: write oid 18 current snap is 130 2026-02-17T05:53:33.184 INFO:tasks.rados.rados.0.trial115.stdout:1311: seq_num 500 ranges {403500=787445,1692526=473837} 2026-02-17T05:53:33.189 INFO:tasks.rados.rados.0.trial115.stdout:1311: writing trial11545420-18 from 403500 to 1190945 tid 1 2026-02-17T05:53:33.192 INFO:tasks.rados.rados.0.trial115.stdout:1311: writing trial11545420-18 from 1692526 to 2166363 tid 2 2026-02-17T05:53:33.193 INFO:tasks.rados.rados.0.trial115.stdout:1312: snap_remove snap 115 2026-02-17T05:53:33.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> v2:10.20.193.78:3300/0 conn(0x7fd7a40038c0 0x7fd7a4005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient(hunting): get_auth_request con 0x7fd7a40038c0 auth_method 0 2026-02-17T05:53:33.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:33.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c5d74640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 0x7fd7a40088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c5d74640 10 monclient(hunting): get_auth_request con 0x7fd7a40063e0 auth_method 0 2026-02-17T05:53:33.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c5d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient(hunting): handle_auth_done global_id 104097 payload 274 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 msgr2=0x7fd7a40088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7a40063e0 0x7fd7a40088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 1 monclient: found mon.noname-a 2026-02-17T05:53:33.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.265+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c809cc80 con 0x7fd7a40038c0 2026-02-17T05:53:33.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c6575640 1 -- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7a40038c0 msgr2=0x7fd7a4005d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c6575640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7a40038c0 0x7fd7a4005d00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc006fb0 tx=0x7fd7bc0302e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e05d461c95f436e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c5573640 1 -- 10.20.193.78:0/3147120809 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc036070 con 0x7fd7a40038c0 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c5573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c5573640 10 monclient: dump: 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.276 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c5573640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c5573640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.267+0000 7fd7c5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 5 monclient: authenticate success, global_id 104097 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7c5573640 1 -- 10.20.193.78:0/3147120809 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc03a410 con 0x7fd7a40038c0 2026-02-17T05:53:33.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7c5573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7c5573640 1 -- 10.20.193.78:0/3147120809 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00f040 con 0x7fd7a40038c0 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7c5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7c5573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7c5573640 10 monclient: dump: 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.278 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 4 set_mon_vals no callback set 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 10 monclient: shutdown 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 1 -- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7a40038c0 msgr2=0x7fd7a4005d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7a40038c0 0x7fd7a4005d00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc006fb0 tx=0x7fd7bc0302e0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7c6d76640 1 -- 10.20.193.78:0/3147120809 reap_dead start 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 1 -- 10.20.193.78:0/3147120809 shutdown_connections 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 1 --2- 10.20.193.78:0/3147120809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7a40038c0 0x7fd7a4005d00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 1 -- 10.20.193.78:0/3147120809 >> 10.20.193.78:0/3147120809 conn(0x7fd7c80a6620 msgr2=0x7fd7c80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 1 -- 10.20.193.78:0/3147120809 shutdown_connections 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.268+0000 7fd7ccec6640 1 -- 10.20.193.78:0/3147120809 wait complete. 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient: monmap: 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:33.280 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 1 Processor -- start 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 1 -- start start 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient: init 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd7a40038c0 0x7fd7c81cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient(hunting): picked mon.noname-b con 0x7fd7a40038c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7c81cf890 0x7fd7c809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient(hunting): picked mon.noname-c con 0x7fd7c81cf890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 0x7fd7c809e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient(hunting): picked mon.noname-a con 0x7fd7c81cfc90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c80a0020 con 0x7fd7c81cfc90 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c806d640 con 0x7fd7a40038c0 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c80a01a0 con 0x7fd7c81cf890 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7ccec6640 10 monclient(hunting): authenticate will time out at 1058.823618s 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7c81cf890 0x7fd7c809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 0x7fd7c809e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7c81cf890 0x7fd7c809db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33350/0 (socket says 10.20.193.78:33350) 2026-02-17T05:53:33.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 -- 10.20.193.78:0/1201474340 learned_addr learned my addr 10.20.193.78:0/1201474340 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient(hunting): get_auth_request con 0x7fd7c81cf890 auth_method 0 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 10 monclient(hunting): get_auth_request con 0x7fd7c81cfc90 auth_method 0 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 1 -- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 msgr2=0x7fd7c809e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 1 -- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 msgr2=0x7fd7c809e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 0x7fd7c809e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c6d76640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 0x7fd7c809e090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient(hunting): handle_auth_done global_id 74381 payload 824 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 -- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd7a40038c0 msgr2=0x7fd7c81cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd7a40038c0 0x7fd7c81cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 -- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 msgr2=0x7fd7c809e090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 0x7fd7c809e090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 monclient: found mon.noname-c 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.269+0000 7fd7c5d74640 1 -- 10.20.193.78:0/1201474340 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c8125f80 con 0x7fd7c81cf890 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.271+0000 7fd7c5d74640 1 -- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7c81cf890 msgr2=0x7fd7c809db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.271+0000 7fd7c5d74640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7c81cf890 0x7fd7c809db50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0007130 tx=0x7fd7b0007600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 1 -- 10.20.193.78:0/1201474340 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b001a070 con 0x7fd7c81cf890 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: dump: 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.285 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 1 -- 10.20.193.78:0/1201474340 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b0004d00 con 0x7fd7c81cf890 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 5 monclient: authenticate success, global_id 74381 2026-02-17T05:53:33.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 10 monclient: _renew_subs 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c809d810 con 0x7fd7c81cf890 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 1 -- 10.20.193.78:0/1201474340 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0011a50 con 0x7fd7c81cf890 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7c4d72640 4 set_mon_vals no callback set 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 10 monclient: dump: 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.287 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 10 monclient: _renew_subs 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c8127710 con 0x7fd7c81cf890 2026-02-17T05:53:33.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c809beb0 con 0x7fd7c81cf890 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.272+0000 7fd7aeffd640 1 -- 10.20.193.78:0/1201474340 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd7b0011c70 con 0x7fd7c81cf890 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.273+0000 7fd7aeffd640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd79403deb0 0x7fd794040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.273+0000 7fd7aeffd640 1 -- 10.20.193.78:0/1201474340 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(281..281 src has 117..281) ==== 4811+0+0 (secure 0 0 0) 0x7fd7b00524b0 con 0x7fd7c81cf890 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.273+0000 7fd7c6575640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd79403deb0 0x7fd794040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.273+0000 7fd7c6575640 10 monclient: get_auth_request con 0x7fd79403deb0 auth_method 0 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.275+0000 7fd7aeffd640 1 -- 10.20.193.78:0/1201474340 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd7b001d360 con 0x7fd7c81cf890 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.275+0000 7fd7aeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.275+0000 7fd7aeffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:33.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.276+0000 7fd7c6575640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd79403deb0 0x7fd794040370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4003850 tx=0x7fd7bc032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.387+0000 7fd7ccec6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:53:33.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.387+0000 7fd7ccec6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:33.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.387+0000 7fd7ccec6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.387+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd7c8134d90 con 0x7fd7c81cf890 2026-02-17T05:53:33.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.387+0000 7fd7aeffd640 1 -- 10.20.193.78:0/1201474340 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fd7b001dc80 con 0x7fd7c81cf890 2026-02-17T05:53:33.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.387+0000 7fd7aeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:33.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.387+0000 7fd7aeffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:33.388 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:53:33.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd79403deb0 msgr2=0x7fd794040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd79403deb0 0x7fd794040370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4003850 tx=0x7fd7bc032000 comp rx=0 tx=0).stop 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 10 monclient: shutdown 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7c81cf890 msgr2=0x7fd7c809db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7c81cf890 0x7fd7c809db50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0007130 tx=0x7fd7b0007600 comp rx=0 tx=0).stop 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 shutdown_connections 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd7c81cfc90 0x7fd7c809e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd7c81cf890 0x7fd7c809db50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd79403deb0 0x7fd794040370 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 --2- 10.20.193.78:0/1201474340 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd7a40038c0 0x7fd7c81cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 >> 10.20.193.78:0/1201474340 conn(0x7fd7c80a6620 msgr2=0x7fd7c8135340 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 shutdown_connections 2026-02-17T05:53:33.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.388+0000 7fd7ccec6640 1 -- 10.20.193.78:0/1201474340 wait complete. 2026-02-17T05:53:33.396 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:53:33.396 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:53:33.396 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:53:33.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.465+0000 7fd070749640 10 monclient: get_monmap_and_config 2026-02-17T05:53:33.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.465+0000 7fd070749640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.465+0000 7fd070749640 10 monclient: monmap: 2026-02-17T05:53:33.465 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.465 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.465 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:33.466766+0000 2026-02-17T05:53:33.465 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:33.466766+0000 2026-02-17T05:53:33.466 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.466 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.466 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:33.466 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:33.466 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:33.466 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:33.466 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:33.466 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 1 Processor -- start 2026-02-17T05:53:33.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 1 -- start start 2026-02-17T05:53:33.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient: init 2026-02-17T05:53:33.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2026-02-17T05:53:33.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd06809ea90 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd06809f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient(hunting): picked mon.noname-b con 0x7fd06809ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd06813d640 0x7fd06813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient(hunting): picked mon.noname-c con 0x7fd06813d640 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0680a0a20 con 0x7fd06809ea90 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd0680a0d00 con 0x7fd06813d640 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0680a0460 con 0x7fd06809ee70 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd070749640 10 monclient(hunting): authenticate will time out at 1059.020460s 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd06809f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06e4be640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd06809ea90 0x7fd0680a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56994/0 (socket says 10.20.193.78:56994) 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06e4be640 1 -- 10.20.193.78:0/4216951549 learned_addr learned my addr 10.20.193.78:0/4216951549 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient(hunting): get_auth_request con 0x7fd06809ee70 auth_method 0 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient(hunting): handle_auth_done global_id 74384 payload 274 2026-02-17T05:53:33.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06ecbf640 1 --2- 10.20.193.78:0/4216951549 >> v2:10.20.193.115:3300/0 conn(0x7fd06813d640 0x7fd06813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 1 -- 10.20.193.78:0/4216951549 >> v2:10.20.193.115:3300/0 conn(0x7fd06813d640 msgr2=0x7fd06813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 1 --2- 10.20.193.78:0/4216951549 >> v2:10.20.193.115:3300/0 conn(0x7fd06813d640 0x7fd06813fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 1 -- 10.20.193.78:0/4216951549 >> v1:10.20.193.78:6789/0 conn(0x7fd06809ea90 legacy=0x7fd0680a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 1 monclient: found mon.noname-b 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06dcbd640 1 -- 10.20.193.78:0/4216951549 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0680a0740 con 0x7fd06809ee70 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06ecbf640 10 monclient: get_auth_request con 0x7fd06813d640 auth_method 0 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.466+0000 7fd06ecbf640 1 --2- 10.20.193.78:0/4216951549 >> v2:10.20.193.115:3300/0 conn(0x7fd06813d640 0x7fd06813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06dcbd640 1 -- 10.20.193.78:0/4216951549 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 msgr2=0x7fd06809f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06dcbd640 1 --2- 10.20.193.78:0/4216951549 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd06809f290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0098a0 tx=0x7fd05c02ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72ea240f0823d774 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 1 -- 10.20.193.78:0/4216951549 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c03c070 con 0x7fd06809ee70 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: dump: 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.481 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 1 -- 10.20.193.78:0/4216951549 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05c02fce0 con 0x7fd06809ee70 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 5 monclient: authenticate success, global_id 74384 2026-02-17T05:53:33.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 1 -- 10.20.193.78:0/4216951549 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c037660 con 0x7fd06809ee70 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 4 set_mon_vals no callback set 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd06d4bc640 10 monclient: dump: 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.483 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 10 monclient: shutdown 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 1 -- 10.20.193.78:0/4216951549 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 msgr2=0x7fd06809f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 1 --2- 10.20.193.78:0/4216951549 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd06809f290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0098a0 tx=0x7fd05c02ed80 comp rx=0 tx=0).stop 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 1 -- 10.20.193.78:0/4216951549 shutdown_connections 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 1 --2- 10.20.193.78:0/4216951549 >> v2:10.20.193.115:3300/0 conn(0x7fd06813d640 0x7fd06813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 1 --2- 10.20.193.78:0/4216951549 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd06809f290 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 1 -- 10.20.193.78:0/4216951549 >> 10.20.193.78:0/4216951549 conn(0x7fd0680a7520 msgr2=0x7fd0680a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 1 -- 10.20.193.78:0/4216951549 shutdown_connections 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.469+0000 7fd070749640 1 -- 10.20.193.78:0/4216951549 wait complete. 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient: monmap: 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.485 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 1 Processor -- start 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 1 -- start start 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient: init 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd06809ea90 0x7fd068073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient(hunting): picked mon.noname-b con 0x7fd06809ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd068073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient(hunting): picked mon.noname-c con 0x7fd06809ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd06813d640 0x7fd06813bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient(hunting): picked mon.noname-a con 0x7fd06813d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680a6820 con 0x7fd06813d640 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680a66a0 con 0x7fd06809ea90 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd06813ff70 con 0x7fd06809ee70 2026-02-17T05:53:33.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd070749640 10 monclient(hunting): authenticate will time out at 1059.024577s 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd068073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06ecbf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd06813d640 0x7fd06813bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd068073c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33378/0 (socket says 10.20.193.78:33378) 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 -- 10.20.193.78:0/3808141977 learned_addr learned my addr 10.20.193.78:0/3808141977 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient(hunting): get_auth_request con 0x7fd06809ee70 auth_method 0 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06ecbf640 10 monclient(hunting): get_auth_request con 0x7fd06813d640 auth_method 0 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06ecbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06ecbf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06ecbf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06ecbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06ecbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06ecbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06e4be640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd06809ea90 0x7fd068073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06e4be640 10 monclient(hunting): get_auth_request con 0x7fd06809ea90 auth_method 0 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06e4be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06e4be640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06e4be640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient(hunting): handle_auth_done global_id 74387 payload 824 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd06809ea90 msgr2=0x7fd068073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd06809ea90 0x7fd068073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd06813d640 msgr2=0x7fd06813bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd06813d640 0x7fd06813bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 monclient: found mon.noname-c 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.470+0000 7fd06dcbd640 1 -- 10.20.193.78:0/3808141977 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0680a9c30 con 0x7fd06809ee70 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.472+0000 7fd06dcbd640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 msgr2=0x7fd068073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.472+0000 7fd06dcbd640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd068073c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd05c02f290 tx=0x7fd05c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c03c040 con 0x7fd06809ee70 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: dump: 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.491 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05c040600 con 0x7fd06809ee70 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 5 monclient: authenticate success, global_id 74387 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 10 monclient: _renew_subs 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06813fcb0 con 0x7fd06809ee70 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0409d0 con 0x7fd06809ee70 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06ccbb640 4 set_mon_vals no callback set 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 10 monclient: dump: 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.493 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 10 monclient: _renew_subs 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd068070c00 con 0x7fd06809ee70 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd068070dd0 0x7fd06813c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681d0080 con 0x7fd068070dd0 2026-02-17T05:53:33.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd05c03f030 con 0x7fd06809ee70 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd068070dd0 0x7fd06813c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 10 monclient: get_auth_request con 0x7fd068070dd0 auth_method 0 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03803deb0 0x7fd038040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 10 monclient(hunting): handle_auth_done global_id 104103 payload 824 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd06e4be640 1 -- 10.20.193.78:0/3808141977 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd06809f290 con 0x7fd068070dd0 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.473+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(281..281 src has 117..281) ==== 4811+0+0 (secure 0 0 0) 0x7fd05c0777b0 con 0x7fd06809ee70 2026-02-17T05:53:33.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.474+0000 7fd06ecbf640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03803deb0 0x7fd038040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.474+0000 7fd06ecbf640 10 monclient: get_auth_request con 0x7fd03803deb0 auth_method 0 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd06e4be640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd068070dd0 msgr2=0x7fd06813c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd06e4be640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd068070dd0 0x7fd06813c000 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd05800e730 tx=0x7fd05800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058004d20 con 0x7fd068070dd0 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 10 monclient: dump: 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.496 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd058018070 con 0x7fd068070dd0 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd068070dd0 msgr2=0x7fd06813c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd068070dd0 0x7fd06813c000 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd05800e730 tx=0x7fd05800ec00 comp rx=0 tx=0).stop 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd068070dd0 msgr2=0x7fd06813c000 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:33.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.476+0000 7fd04effd640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd068070dd0 0x7fd06813c000 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd05800e730 tx=0x7fd05800ec00 comp rx=0 tx=0).stop 2026-02-17T05:53:33.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.477+0000 7fd06ecbf640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03803deb0 0x7fd038040370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd068135c60 tx=0x7fd0640077a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd070749640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd070749640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0681cf3b0 0x7fd0681da490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd070749640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680744d0 con 0x7fd0681cf3b0 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0681cf3b0 0x7fd0681da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 10 monclient: get_auth_request con 0x7fd0681cf3b0 auth_method 0 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 10 monclient(hunting): handle_auth_done global_id 104106 payload 824 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.479+0000 7fd06e4be640 1 -- 10.20.193.78:0/3808141977 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd06806fa30 con 0x7fd0681cf3b0 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd06e4be640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0681cf3b0 msgr2=0x7fd0681da490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd06e4be640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0681cf3b0 0x7fd0681da490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd058017000 tx=0x7fd058004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058007590 con 0x7fd0681cf3b0 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 10 monclient: dump: 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.501 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7fd05801ee50 con 0x7fd0681cf3b0 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:33.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0681cf3b0 msgr2=0x7fd0681da490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0681cf3b0 0x7fd0681da490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd058017000 tx=0x7fd058004a40 comp rx=0 tx=0).stop 2026-02-17T05:53:33.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.481+0000 7fd04effd640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0681cf3b0 msgr2=0x7fd0681da490 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:33.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd04effd640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0681cf3b0 0x7fd0681da490 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd058017000 tx=0x7fd058004a40 comp rx=0 tx=0).stop 2026-02-17T05:53:33.503 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:33.503 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:33.503 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:33.503 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:53:33.503 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 82, 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 6633, 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.504 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.505 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:33.506 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.507 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.508 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.509 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.510 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:33.511 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:33.512 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:33.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03803deb0 msgr2=0x7fd038040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03803deb0 0x7fd038040370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd068135c60 tx=0x7fd0640077a0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 10 monclient: shutdown 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 msgr2=0x7fd068073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd068073c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd05c02f290 tx=0x7fd05c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd06ecbf640 1 -- 10.20.193.78:0/3808141977 reap_dead start 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 shutdown_connections 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 --2- 10.20.193.78:0/3808141977 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd06809ee70 0x7fd068073c40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 >> 10.20.193.78:0/3808141977 conn(0x7fd0680a7520 msgr2=0x7fd068136dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 shutdown_connections 2026-02-17T05:53:33.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.482+0000 7fd070749640 1 -- 10.20.193.78:0/3808141977 wait complete. 2026-02-17T05:53:33.514 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:53:33.514 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:53:33.515 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:53:33.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.583+0000 7f5fadcd3640 10 monclient: get_monmap_and_config 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.583+0000 7f5fadcd3640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.583+0000 7f5fadcd3640 10 monclient: monmap: 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:33.584817+0000 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:33.584817+0000 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:33.583 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:33.584 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:33.584 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:33.584 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.583+0000 7f5fadcd3640 1 Processor -- start 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 1 -- start start 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient: init 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5fa806c890 0x7f5fa80a3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient(hunting): picked mon.noname-c con 0x7f5fa806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5fa806cc70 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5fa813d8e0 0x7f5fa813fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient(hunting): picked mon.noname-a con 0x7f5fa813d8e0 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa809c0c0 con 0x7f5fa806cc70 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa809e7a0 con 0x7f5fa813d8e0 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa809c3a0 con 0x7f5fa806c890 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fadcd3640 10 monclient(hunting): authenticate will time out at 1059.138343s 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa6ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f5fa806cc70 0x7f5fa806d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56996/0 (socket says 10.20.193.78:56996) 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa6ffd640 1 -- 10.20.193.78:0/596792023 learned_addr learned my addr 10.20.193.78:0/596792023 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 1 --2- 10.20.193.78:0/596792023 >> v2:10.20.193.78:3300/0 conn(0x7f5fa813d8e0 0x7f5fa813fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient(hunting): get_auth_request con 0x7f5fa813d8e0 auth_method 0 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa77fe640 1 --2- 10.20.193.78:0/596792023 >> v2:10.20.193.115:3300/0 conn(0x7f5fa806c890 0x7f5fa80a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa77fe640 10 monclient(hunting): get_auth_request con 0x7f5fa806c890 auth_method 0 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient(hunting): handle_auth_done global_id 104109 payload 274 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 1 -- 10.20.193.78:0/596792023 >> v2:10.20.193.115:3300/0 conn(0x7f5fa806c890 msgr2=0x7f5fa80a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 1 --2- 10.20.193.78:0/596792023 >> v2:10.20.193.115:3300/0 conn(0x7f5fa806c890 0x7f5fa80a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 1 -- 10.20.193.78:0/596792023 >> v1:10.20.193.78:6789/0 conn(0x7f5fa806cc70 legacy=0x7f5fa806d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 1 monclient: found mon.noname-a 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.584+0000 7f5fa7fff640 1 -- 10.20.193.78:0/596792023 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa809c680 con 0x7f5fa813d8e0 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa7fff640 1 -- 10.20.193.78:0/596792023 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 msgr2=0x7f5fa813fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa7fff640 1 --2- 10.20.193.78:0/596792023 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 0x7f5fa813fcd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8070040 tx=0x7f5f9c02f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556c36c1114df502 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 1 -- 10.20.193.78:0/596792023 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c03c070 con 0x7f5fa813d8e0 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: dump: 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.594 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 5 monclient: authenticate success, global_id 104109 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 1 -- 10.20.193.78:0/596792023 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9c037640 con 0x7f5fa813d8e0 2026-02-17T05:53:33.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 1 -- 10.20.193.78:0/596792023 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c037a10 con 0x7f5fa813d8e0 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 4 set_mon_vals no callback set 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fa67fc640 10 monclient: dump: 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.596 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 10 monclient: shutdown 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 1 -- 10.20.193.78:0/596792023 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 msgr2=0x7f5fa813fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 1 --2- 10.20.193.78:0/596792023 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 0x7f5fa813fcd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8070040 tx=0x7f5f9c02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 1 -- 10.20.193.78:0/596792023 shutdown_connections 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 1 --2- 10.20.193.78:0/596792023 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 0x7f5fa813fcd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 1 --2- 10.20.193.78:0/596792023 >> v2:10.20.193.115:3300/0 conn(0x7f5fa806c890 0x7f5fa80a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 1 -- 10.20.193.78:0/596792023 >> 10.20.193.78:0/596792023 conn(0x7f5fa80a4230 msgr2=0x7f5fa80a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 1 -- 10.20.193.78:0/596792023 shutdown_connections 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 1 -- 10.20.193.78:0/596792023 wait complete. 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.587+0000 7f5fadcd3640 10 monclient: monmap: 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.597 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 1 Processor -- start 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 1 -- start start 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient: init 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa806c890 0x7f5fa80999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient(hunting): picked mon.noname-c con 0x7f5fa806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5fa806cc70 0x7f5fa8099ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient(hunting): picked mon.noname-b con 0x7f5fa806cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 0x7f5fa80918c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient(hunting): picked mon.noname-a con 0x7f5fa813d8e0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa809cf90 con 0x7f5fa813d8e0 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa809ce10 con 0x7f5fa806cc70 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa8140210 con 0x7f5fa806c890 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fadcd3640 10 monclient(hunting): authenticate will time out at 1059.142441s 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 0x7f5fa80918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa806c890 0x7f5fa80999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 0x7f5fa80918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35304/0 (socket says 10.20.193.78:35304) 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 -- 10.20.193.78:0/2290796751 learned_addr learned my addr 10.20.193.78:0/2290796751 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient(hunting): get_auth_request con 0x7f5fa813d8e0 auth_method 0 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa77fe640 10 monclient(hunting): get_auth_request con 0x7f5fa806c890 auth_method 0 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient(hunting): handle_auth_done global_id 104112 payload 824 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa806c890 msgr2=0x7f5fa80999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa806c890 0x7f5fa80999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5fa806cc70 msgr2=0x7f5fa8099ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5fa806cc70 0x7f5fa8099ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 monclient: found mon.noname-a 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.588+0000 7f5fa7fff640 1 -- 10.20.193.78:0/2290796751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa8094e80 con 0x7f5fa813d8e0 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fa7fff640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 msgr2=0x7f5fa80918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fa7fff640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 0x7f5fa80918c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009d90 tx=0x7f5f9c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c047030 con 0x7f5fa813d8e0 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: dump: 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fadcd3640 5 monclient: authenticate success, global_id 104112 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9c042760 con 0x7f5fa813d8e0 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c03c040 con 0x7f5fa813d8e0 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fa5ffb640 4 set_mon_vals no callback set 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5f83fff640 10 monclient: dump: 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.602 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fadcd3640 10 monclient: _renew_subs 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fadcd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa8095160 con 0x7f5fa813d8e0 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fadcd3640 10 monclient: _renew_subs 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fadcd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.590+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa8095440 con 0x7f5fa813d8e0 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fadcd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fadcd3640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa8091e00 0x7f5fa80921e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fadcd3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa8092720 con 0x7f5fa8091e00 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5f9c042b50 con 0x7f5fa813d8e0 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa8091e00 0x7f5fa80921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 10 monclient: get_auth_request con 0x7f5fa8091e00 auth_method 0 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_done global_id 74393 payload 824 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa77fe640 1 -- 10.20.193.78:0/2290796751 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5fa8074980 con 0x7f5fa8091e00 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5f83fff640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f7803dee0 0x7f5f780403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(281..281 src has 117..281) ==== 4811+0+0 (secure 0 0 0) 0x7f5f9c078450 con 0x7f5fa813d8e0 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa6ffd640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f7803dee0 0x7f5f780403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.591+0000 7f5fa6ffd640 10 monclient: get_auth_request con 0x7f5f7803dee0 auth_method 0 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.593+0000 7f5fa77fe640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa8091e00 msgr2=0x7f5fa80921e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.593+0000 7f5fa77fe640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa8091e00 0x7f5fa80921e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009860 tx=0x7f5f94009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.593+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94018070 con 0x7f5fa8091e00 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.593+0000 7f5f83fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.593+0000 7f5f83fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.593+0000 7f5f83fff640 10 monclient: dump: 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.604 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.594+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f5f94010c60 con 0x7f5fa8091e00 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.594+0000 7f5f83fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.594+0000 7f5f83fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.594+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa8091e00 msgr2=0x7f5fa80921e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.594+0000 7f5f83fff640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa8091e00 0x7f5fa80921e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009860 tx=0x7f5f94009d30 comp rx=0 tx=0).stop 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.594+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa8091e00 msgr2=0x7f5fa80921e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.594+0000 7f5f83fff640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa8091e00 0x7f5fa80921e0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009860 tx=0x7f5f94009d30 comp rx=0 tx=0).stop 2026-02-17T05:53:33.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.594+0000 7f5fa6ffd640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f7803dee0 0x7f5f780403a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f98007fd0 tx=0x7f5f98007ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fadcd3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fadcd3640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa80700f0 0x7f5fa81da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fadcd3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa80a3480 con 0x7f5fa80700f0 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa80700f0 0x7f5fa81da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 10 monclient: get_auth_request con 0x7f5fa80700f0 auth_method 0 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 10 monclient(hunting): handle_auth_done global_id 74396 payload 824 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.597+0000 7f5fa77fe640 1 -- 10.20.193.78:0/2290796751 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5fa8093a30 con 0x7f5fa80700f0 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fa77fe640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa80700f0 msgr2=0x7f5fa81da0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fa77fe640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa80700f0 0x7f5fa81da0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009860 tx=0x7f5f94004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94014840 con 0x7f5fa80700f0 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 10 monclient: dump: 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.608 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f5f94014840 con 0x7f5fa80700f0 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa80700f0 msgr2=0x7f5fa81da0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa80700f0 0x7f5fa81da0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009860 tx=0x7f5f94004830 comp rx=0 tx=0).stop 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa80700f0 msgr2=0x7f5fa81da0c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5f83fff640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5fa80700f0 0x7f5fa81da0c0 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009860 tx=0x7f5f94004830 comp rx=0 tx=0).stop 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:33.609 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 82, 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 185492, 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.610 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.611 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:33.612 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.613 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.614 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:33.615 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:33.616 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:33.617 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.618 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:33.619 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f7803dee0 msgr2=0x7f5f780403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f7803dee0 0x7f5f780403a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f98007fd0 tx=0x7f5f98007ba0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 10 monclient: shutdown 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 msgr2=0x7f5fa80918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 1 --2- 10.20.193.78:0/2290796751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5fa813d8e0 0x7f5fa80918c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009d90 tx=0x7f5f9c004b40 comp rx=0 tx=0).stop 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fa7fff640 1 -- 10.20.193.78:0/2290796751 reap_dead start 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fa7fff640 1 -- 10.20.193.78:0/2290796751 reap_dead start 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 shutdown_connections 2026-02-17T05:53:33.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 >> 10.20.193.78:0/2290796751 conn(0x7f5fa80a4230 msgr2=0x7f5fa813e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 shutdown_connections 2026-02-17T05:53:33.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.600+0000 7f5fadcd3640 1 -- 10.20.193.78:0/2290796751 wait complete. 2026-02-17T05:53:33.621 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:53:33.621 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:53:33.621 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:53:33.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.689+0000 7fd313499640 10 monclient: get_monmap_and_config 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.689+0000 7fd313499640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.689+0000 7fd313499640 10 monclient: monmap: 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:33.691109+0000 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:33.691109+0000 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 1 Processor -- start 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 1 -- start start 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient: init 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 0x7fd30c0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient(hunting): picked mon.noname-c con 0x7fd30c09ea90 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd30c09ee70 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd30c13d640 0x7fd30c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient(hunting): picked mon.noname-a con 0x7fd30c13d640 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd30c0a0a20 con 0x7fd30c09ee70 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd30c0a0d00 con 0x7fd30c13d640 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd30c0a0460 con 0x7fd30c09ea90 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd313499640 10 monclient(hunting): authenticate will time out at 1059.244645s 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd30c13d640 0x7fd30c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd310a0d640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd30c09ee70 0x7fd30c09f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57002/0 (socket says 10.20.193.78:57002) 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd310a0d640 1 -- 10.20.193.78:0/144948282 learned_addr learned my addr 10.20.193.78:0/144948282 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient(hunting): get_auth_request con 0x7fd30c13d640 auth_method 0 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd31120e640 1 --2- 10.20.193.78:0/144948282 >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 0x7fd30c0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd31120e640 10 monclient(hunting): get_auth_request con 0x7fd30c09ea90 auth_method 0 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd31120e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd31120e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd31120e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient(hunting): handle_auth_done global_id 104115 payload 274 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 1 -- 10.20.193.78:0/144948282 >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 msgr2=0x7fd30c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 1 --2- 10.20.193.78:0/144948282 >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 0x7fd30c0a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 1 -- 10.20.193.78:0/144948282 >> v1:10.20.193.78:6789/0 conn(0x7fd30c09ee70 legacy=0x7fd30c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 1 monclient: found mon.noname-a 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.690+0000 7fd311a0f640 1 -- 10.20.193.78:0/144948282 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c0a0740 con 0x7fd30c13d640 2026-02-17T05:53:33.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd311a0f640 1 -- 10.20.193.78:0/144948282 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 msgr2=0x7fd30c13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd311a0f640 1 --2- 10.20.193.78:0/144948282 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c13fa30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06ecd0 tx=0x7fd30802f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcb2f1ec9dc4411 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 1 -- 10.20.193.78:0/144948282 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30803c070 con 0x7fd30c13d640 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: dump: 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.698 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd313499640 5 monclient: authenticate success, global_id 104115 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 1 -- 10.20.193.78:0/144948282 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd308037440 con 0x7fd30c13d640 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd313499640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 1 -- 10.20.193.78:0/144948282 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308037810 con 0x7fd30c13d640 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd313499640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd313499640 4 set_mon_vals no callback set 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd2fbfff640 10 monclient: dump: 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.699 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd313499640 10 monclient: shutdown 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd313499640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd313499640 1 -- 10.20.193.78:0/144948282 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 msgr2=0x7fd30c13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.693+0000 7fd313499640 1 --2- 10.20.193.78:0/144948282 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c13fa30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06ecd0 tx=0x7fd30802f130 comp rx=0 tx=0).stop 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 -- 10.20.193.78:0/144948282 shutdown_connections 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 --2- 10.20.193.78:0/144948282 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c13fa30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 --2- 10.20.193.78:0/144948282 >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 0x7fd30c0a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 -- 10.20.193.78:0/144948282 >> 10.20.193.78:0/144948282 conn(0x7fd30c0a7520 msgr2=0x7fd30c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 -- 10.20.193.78:0/144948282 shutdown_connections 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 -- 10.20.193.78:0/144948282 wait complete. 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient: monmap: 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.700 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 Processor -- start 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 -- start start 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient: init 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c09ea90 0x7fd30c1ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient(hunting): picked mon.noname-a con 0x7fd30c09ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ee70 0x7fd30c073450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient(hunting): picked mon.noname-c con 0x7fd30c09ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13d640 0x7fd30c073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient(hunting): picked mon.noname-b con 0x7fd30c13d640 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c0a6820 con 0x7fd30c09ea90 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c0a66a0 con 0x7fd30c13d640 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd30c13ff70 con 0x7fd30c09ee70 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd313499640 10 monclient(hunting): authenticate will time out at 1059.248932s 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd31120e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c09ea90 0x7fd30c1ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd31120e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c09ea90 0x7fd30c1ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35332/0 (socket says 10.20.193.78:35332) 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd31120e640 1 -- 10.20.193.78:0/3840977675 learned_addr learned my addr 10.20.193.78:0/3840977675 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd31120e640 10 monclient(hunting): get_auth_request con 0x7fd30c09ea90 auth_method 0 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.694+0000 7fd31120e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd310a0d640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ee70 0x7fd30c073450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd310a0d640 10 monclient(hunting): get_auth_request con 0x7fd30c09ee70 auth_method 0 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd310a0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd310a0d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd310a0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 10 monclient(hunting): handle_auth_done global_id 104118 payload 824 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ee70 msgr2=0x7fd30c073450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ee70 0x7fd30c073450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13d640 msgr2=0x7fd30c073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13d640 0x7fd30c073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 1 monclient: found mon.noname-a 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.695+0000 7fd31120e640 1 -- 10.20.193.78:0/3840977675 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c13fc40 con 0x7fd30c09ea90 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd31120e640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c09ea90 msgr2=0x7fd30c1ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd31120e640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c09ea90 0x7fd30c1ceec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00e7f0 tx=0x7fd2fc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc017070 con 0x7fd30c09ea90 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: dump: 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.703 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00ca60 con 0x7fd30c09ea90 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd313499640 5 monclient: authenticate success, global_id 104118 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd313499640 10 monclient: _renew_subs 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd313499640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30c0a1c10 con 0x7fd30c09ea90 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00f040 con 0x7fd30c09ea90 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2fb7fe640 4 set_mon_vals no callback set 2026-02-17T05:53:33.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd2f9ffb640 10 monclient: dump: 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd313499640 10 monclient: _renew_subs 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd313499640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.697+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30c070690 con 0x7fd30c09ea90 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd313499640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd313499640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13b8b0 0x7fd30c13bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd313499640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c0a13d0 con 0x7fd30c13b8b0 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd2fc012050 con 0x7fd30c09ea90 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13b8b0 0x7fd30c13bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 10 monclient: get_auth_request con 0x7fd30c13b8b0 auth_method 0 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd2f9ffb640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2dc03df00 0x7fd2dc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(281..281 src has 117..281) ==== 4811+0+0 (secure 0 0 0) 0x7fd2fc052580 con 0x7fd30c09ea90 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd311a0f640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2dc03df00 0x7fd2dc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 10 monclient(hunting): handle_auth_done global_id 44569 payload 824 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd310a0d640 1 -- 10.20.193.78:0/3840977675 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd30c0008d0 con 0x7fd30c13b8b0 2026-02-17T05:53:33.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.698+0000 7fd311a0f640 10 monclient: get_auth_request con 0x7fd2dc03df00 auth_method 0 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.700+0000 7fd310a0d640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13b8b0 msgr2=0x7fd30c13bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.700+0000 7fd310a0d640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13b8b0 0x7fd30c13bc90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a9a00 tx=0x7fd300005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300006050 con 0x7fd30c13b8b0 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 10 monclient: dump: 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.708 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd300018070 con 0x7fd30c13b8b0 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13b8b0 msgr2=0x7fd30c13bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13b8b0 0x7fd30c13bc90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a9a00 tx=0x7fd300005400 comp rx=0 tx=0).stop 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13b8b0 msgr2=0x7fd30c13bc90 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd2f9ffb640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13b8b0 0x7fd30c13bc90 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a9a00 tx=0x7fd300005400 comp rx=0 tx=0).stop 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.701+0000 7fd311a0f640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2dc03df00 0x7fd2dc0403c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06e270 tx=0x7fd308031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd313499640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd313499640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13c1d0 0x7fd30c1da250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd313499640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c13d360 con 0x7fd30c13c1d0 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd310a0d640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13c1d0 0x7fd30c1da250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd310a0d640 10 monclient: get_auth_request con 0x7fd30c13c1d0 auth_method 0 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd310a0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd310a0d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.704+0000 7fd310a0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.705+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.705+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.705+0000 7fd310a0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.705+0000 7fd310a0d640 10 monclient(hunting): handle_auth_done global_id 44572 payload 824 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.705+0000 7fd310a0d640 1 -- 10.20.193.78:0/3840977675 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd30c1cf400 con 0x7fd30c13c1d0 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd310a0d640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13c1d0 msgr2=0x7fd30c1da250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd310a0d640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13c1d0 0x7fd30c1da250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a9a00 tx=0x7fd300004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000067d0 con 0x7fd30c13c1d0 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 10 monclient: dump: 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7fd3000067d0 con 0x7fd30c13c1d0 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13c1d0 msgr2=0x7fd30c1da250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13c1d0 0x7fd30c1da250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a9a00 tx=0x7fd300004a40 comp rx=0 tx=0).stop 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13c1d0 msgr2=0x7fd30c1da250 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd2f9ffb640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c13c1d0 0x7fd30c1da250 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a9a00 tx=0x7fd300004a40 comp rx=0 tx=0).stop 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 82, 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 337355, 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:53:33.711 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:33.712 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:33.713 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:33.714 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.715 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:33.716 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.717 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:53:33.718 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:33.719 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:33.720 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2dc03df00 msgr2=0x7fd2dc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd313499640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2dc03df00 0x7fd2dc0403c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06e270 tx=0x7fd308031000 comp rx=0 tx=0).stop 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd313499640 10 monclient: shutdown 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd313499640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c09ea90 msgr2=0x7fd30c1ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd313499640 1 --2- 10.20.193.78:0/3840977675 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c09ea90 0x7fd30c1ceec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00e7f0 tx=0x7fd2fc00ecc0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.707+0000 7fd311a0f640 1 -- 10.20.193.78:0/3840977675 reap_dead start 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.708+0000 7fd311a0f640 1 -- 10.20.193.78:0/3840977675 reap_dead start 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.708+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 shutdown_connections 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.708+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 >> 10.20.193.78:0/3840977675 conn(0x7fd30c0a7520 msgr2=0x7fd30c136db0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.708+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 shutdown_connections 2026-02-17T05:53:33.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.708+0000 7fd313499640 1 -- 10.20.193.78:0/3840977675 wait complete. 2026-02-17T05:53:33.721 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:53:33.721 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:53:33.722 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.789+0000 7fd63c2f2640 10 monclient: get_monmap_and_config 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.789+0000 7fd63c2f2640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.789+0000 7fd63c2f2640 10 monclient: monmap: 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:33.790605+0000 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:33.790605+0000 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.789+0000 7fd63c2f2640 1 Processor -- start 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.789+0000 7fd63c2f2640 1 -- start start 2026-02-17T05:53:33.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.789+0000 7fd63c2f2640 10 monclient: init 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406c890 0x7fd6340a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient(hunting): picked mon.noname-b con 0x7fd63406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd63406cc70 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd63413a240 0x7fd63413c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient(hunting): picked mon.noname-a con 0x7fd63413a240 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd634074150 con 0x7fd63406cc70 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd63409ef70 con 0x7fd63413a240 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd63409f250 con 0x7fd63406c890 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63c2f2640 10 monclient(hunting): authenticate will time out at 1059.344149s 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406c890 0x7fd6340a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a868640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd63413a240 0x7fd63413c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd639866640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd63406cc70 0x7fd63406d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57012/0 (socket says 10.20.193.78:57012) 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd639866640 1 -- 10.20.193.78:0/1781471916 learned_addr learned my addr 10.20.193.78:0/1781471916 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient(hunting): get_auth_request con 0x7fd63406c890 auth_method 0 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a868640 10 monclient(hunting): get_auth_request con 0x7fd63413a240 auth_method 0 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a868640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a868640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a868640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a868640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a868640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient(hunting): handle_auth_done global_id 74402 payload 274 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 1 -- 10.20.193.78:0/1781471916 >> v2:10.20.193.78:3300/0 conn(0x7fd63413a240 msgr2=0x7fd63413c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 1 --2- 10.20.193.78:0/1781471916 >> v2:10.20.193.78:3300/0 conn(0x7fd63413a240 0x7fd63413c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 1 -- 10.20.193.78:0/1781471916 >> v1:10.20.193.78:6789/0 conn(0x7fd63406cc70 legacy=0x7fd63406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 1 monclient: found mon.noname-b 2026-02-17T05:53:33.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.790+0000 7fd63a067640 1 -- 10.20.193.78:0/1781471916 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63409df40 con 0x7fd63406c890 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.792+0000 7fd63a067640 1 -- 10.20.193.78:0/1781471916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406c890 msgr2=0x7fd6340a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.792+0000 7fd63a067640 1 --2- 10.20.193.78:0/1781471916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406c890 0x7fd6340a01f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd63406e270 tx=0x7fd62402eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfdd878212646958 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 1 -- 10.20.193.78:0/1781471916 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62403c070 con 0x7fd63406c890 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: dump: 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.795 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 5 monclient: authenticate success, global_id 74402 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 1 -- 10.20.193.78:0/1781471916 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62402fcf0 con 0x7fd63406c890 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 1 -- 10.20.193.78:0/1781471916 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624037660 con 0x7fd63406c890 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd639065640 10 monclient: dump: 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.796 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 4 set_mon_vals no callback set 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 10 monclient: shutdown 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 -- 10.20.193.78:0/1781471916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406c890 msgr2=0x7fd6340a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/1781471916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406c890 0x7fd6340a01f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd63406e270 tx=0x7fd62402eeb0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 -- 10.20.193.78:0/1781471916 shutdown_connections 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/1781471916 >> v2:10.20.193.78:3300/0 conn(0x7fd63413a240 0x7fd63413c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/1781471916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406c890 0x7fd6340a01f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 -- 10.20.193.78:0/1781471916 >> 10.20.193.78:0/1781471916 conn(0x7fd6340a6620 msgr2=0x7fd6340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 -- 10.20.193.78:0/1781471916 shutdown_connections 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 -- 10.20.193.78:0/1781471916 wait complete. 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 10 monclient: build_initial_monmap 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 10 monclient: monmap: 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:33.797 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 Processor -- start 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 1 -- start start 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.793+0000 7fd63c2f2640 10 monclient: init 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63406c890 0x7fd6341ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient(hunting): picked mon.noname-b con 0x7fd63406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406cc70 0x7fd634135320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient(hunting): picked mon.noname-c con 0x7fd63406cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63413a240 0x7fd634135860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient(hunting): picked mon.noname-a con 0x7fd63413a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd63413ccf0 con 0x7fd63413a240 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd63413cb70 con 0x7fd63406c890 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd63413ce70 con 0x7fd63406cc70 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:33.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63c2f2640 10 monclient(hunting): authenticate will time out at 1059.348235s 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406cc70 0x7fd634135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63413a240 0x7fd634135860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406cc70 0x7fd634135320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33440/0 (socket says 10.20.193.78:33440) 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 -- 10.20.193.78:0/3252428309 learned_addr learned my addr 10.20.193.78:0/3252428309 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient(hunting): get_auth_request con 0x7fd63406cc70 auth_method 0 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 10 monclient(hunting): get_auth_request con 0x7fd63413a240 auth_method 0 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient(hunting): handle_auth_done global_id 74405 payload 824 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 -- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63406c890 msgr2=0x7fd6341ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63406c890 0x7fd6341ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 -- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63413a240 msgr2=0x7fd634135860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63413a240 0x7fd634135860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient: _finish_hunting 0 2026-02-17T05:53:33.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 monclient: found mon.noname-c 2026-02-17T05:53:33.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:33.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd639866640 1 -- 10.20.193.78:0/3252428309 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63411f550 con 0x7fd63406cc70 2026-02-17T05:53:33.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.794+0000 7fd63a868640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63413a240 0x7fd634135860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd639866640 1 -- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406cc70 msgr2=0x7fd634135320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd639866640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406cc70 0x7fd634135320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c750 tx=0x7fd62800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 1 -- 10.20.193.78:0/3252428309 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628005de0 con 0x7fd63406cc70 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: dump: 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.801 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: _finish_auth 0 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: _check_auth_tickets 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 1 -- 10.20.193.78:0/3252428309 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd628007070 con 0x7fd63406cc70 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 1 -- 10.20.193.78:0/3252428309 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628007440 con 0x7fd63406cc70 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd638864640 4 set_mon_vals no callback set 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd61affd640 10 monclient: dump: 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:33.802 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd63c2f2640 5 monclient: authenticate success, global_id 74405 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd63c2f2640 10 monclient: _renew_subs 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd63c2f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd634139bc0 con 0x7fd63406cc70 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd63c2f2640 10 monclient: _renew_subs 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd63c2f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.796+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd634120ce0 con 0x7fd63406cc70 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.797+0000 7fd63c2f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.797+0000 7fd63c2f2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.797+0000 7fd63c2f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.797+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd634074980 con 0x7fd63406cc70 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.797+0000 7fd61affd640 1 -- 10.20.193.78:0/3252428309 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd628007660 con 0x7fd63406cc70 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.797+0000 7fd61affd640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.797+0000 7fd61affd640 1 -- 10.20.193.78:0/3252428309 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(281..281 src has 117..281) ==== 4811+0+0 (secure 0 0 0) 0x7fd628052600 con 0x7fd63406cc70 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.797+0000 7fd63a067640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.798+0000 7fd63a067640 10 monclient: get_auth_request con 0x7fd60403deb0 auth_method 0 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.800+0000 7fd61affd640 1 -- 10.20.193.78:0/3252428309 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd62800a8d0 con 0x7fd63406cc70 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.800+0000 7fd61affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:33.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.800+0000 7fd61affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:33.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.800+0000 7fd63a067640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd63409d790 tx=0x7fd624031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:33.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.914+0000 7fd63c2f2640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:53:33.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.914+0000 7fd63c2f2640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:53:33.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.914+0000 7fd63c2f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:33.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.914+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd6340a01f0 con 0x7fd63406cc70 2026-02-17T05:53:33.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.914+0000 7fd61affd640 1 -- 10.20.193.78:0/3252428309 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd62800a5e0 con 0x7fd63406cc70 2026-02-17T05:53:33.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.914+0000 7fd61affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:53:33.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.914+0000 7fd61affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:33.915 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.915 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=17,config=2,health=10,kv=12,logm=59} crc {auth=1876849263,config=692619927,health=85785212,kv=449989585,logm=4221863938}) 2026-02-17T05:53:33.915 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=1199038575,config=692619927,health=1149746517,kv=449989585,logm=1874610123}) 2026-02-17T05:53:33.915 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.915 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=17,config=2,health=10,kv=12,logm=59} crc {auth=1876849263,config=692619927,health=85785212,kv=449989585,logm=4221863938}) 2026-02-17T05:53:33.915 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=11,kv=12,logm=61} crc {auth=4106015875,config=692619927,health=2463728170,kv=449989585,logm=745174697}) 2026-02-17T05:53:33.916 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.916 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2140046325}) 2026-02-17T05:53:33.916 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4159168018}) 2026-02-17T05:53:33.916 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.916 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2140046325}) 2026-02-17T05:53:33.916 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.915+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2254658793}) 2026-02-17T05:53:33.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 msgr2=0x7fd604040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd63409d790 tx=0x7fd624031000 comp rx=0 tx=0).stop 2026-02-17T05:53:33.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 10 monclient: shutdown 2026-02-17T05:53:33.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:33.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406cc70 msgr2=0x7fd634135320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:33.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406cc70 0x7fd634135320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c750 tx=0x7fd62800cc20 comp rx=0 tx=0).stop 2026-02-17T05:53:33.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 shutdown_connections 2026-02-17T05:53:33.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63413a240 0x7fd634135860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63406cc70 0x7fd634135320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 --2- 10.20.193.78:0/3252428309 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63406c890 0x7fd6341ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:33.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 >> 10.20.193.78:0/3252428309 conn(0x7fd6340a6620 msgr2=0x7fd634138230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:33.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 shutdown_connections 2026-02-17T05:53:33.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:33.915+0000 7fd63c2f2640 1 -- 10.20.193.78:0/3252428309 wait complete. 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3925405784}) 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2644531939}) 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3925405784}) 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3621909224}) 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3584649856}) 2026-02-17T05:53:33.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949393982}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3584649856}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4105564772}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4223222865}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4003538732}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4223222865}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.916+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1945771909}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2459133921}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=291418868}) 2026-02-17T05:53:33.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2459133921}) 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3944765066}) 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1757171316}) 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3833306557}) 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1757171316}) 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2099216428}) 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743021096}) 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2947677250}) 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743021096}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.917+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=180641301}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3458665210}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1877070534}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3458665210}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2154617150}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1869035068}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1392546966}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1869035068}) 2026-02-17T05:53:33.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1750934502}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722782518}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2600481531}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722782518}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.918+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4106619987}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=256626536}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752117535}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=256626536}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3527426419}) 2026-02-17T05:53:33.931 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2745259374}) 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3360778984}) 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2745259374}) 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3834278188}) 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=691608867}) 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2805862189}) 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=691608867}) 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.919+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2529274361}) 2026-02-17T05:53:33.932 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2753017606}) 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1952455331}) 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2753017606}) 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776304703}) 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2492209213}) 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1217283326}) 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2492209213}) 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2126413754}) 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.933 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3144481521}) 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3300668295}) 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3144481521}) 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=936516214}) 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1038819537}) 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1792416471}) 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.934 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1038819537}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.920+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2334419178}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1639875370}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1708280103}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1639875370}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1861766769}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=50,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=1574272979,mdsmap=1160443688,mgr=3987721426,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2992507002,mon_config_key=4028116958,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2057857162,osd_snap=3862306161}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=48,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=6} crc {logm=1184739856,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3950449402,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2057857162,osd_snap=3881713549}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=50,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=1574272979,mdsmap=1160443688,mgr=3987721426,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2992507002,mon_config_key=4028116958,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2057857162,osd_snap=3862306161}) 2026-02-17T05:53:33.935 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.921+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=49,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=6} crc {logm=2778232484,mdsmap=1730306971,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2403998552,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=2057857162,osd_snap=3881713549}) 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=43,osdmap=57} crc {osd_snap=1480698300,osdmap=520067983}) 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=44,osdmap=56} crc {osd_snap=3289776494,osdmap=2819368814}) 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=43,osdmap=57} crc {osd_snap=1480698300,osdmap=520067983}) 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=2572562445,osdmap=680152603}) 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1836623627}) 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1566085664}) 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1836623627}) 2026-02-17T05:53:33.936 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.922+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=545141493}) 2026-02-17T05:53:33.938 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:53:33.938 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.939 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=92} crc {osdmap=2880537417}) 2026-02-17T05:53:33.939 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=93} crc {osdmap=2453055805}) 2026-02-17T05:53:33.939 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:53:33.939 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=92} crc {osdmap=2880537417}) 2026-02-17T05:53:33.939 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:53:33.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=91} crc {osdmap=3636434435}) 2026-02-17T05:53:34.144 INFO:tasks.rados.rados.0.trial115.stdout:1312: done (7 left) 2026-02-17T05:53:34.144 INFO:tasks.rados.rados.0.trial115.stdout:1313: delete oid 41 current snap is 130 2026-02-17T05:53:34.144 INFO:tasks.rados.rados.0.trial115.stdout:1305: finishing write tid 2 to trial11545420-26 2026-02-17T05:53:34.144 INFO:tasks.rados.rados.0.trial115.stdout:1305: finishing write tid 3 to trial11545420-26 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1305: finishing write tid 4 to trial11545420-26 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1305: oid 26 updating version 0 to 198 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1305: oid 26 version 198 is already newer than 197 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1305: oid 26 updating version 198 to 199 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 199 (ObjNum 496 snap 130 seq_num 496) dirty exists 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1305: left oid 26 (ObjNum 496 snap 130 seq_num 496) 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1306: finishing write tid 1 to trial11545420-29 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1306: finishing write tid 2 to trial11545420-29 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1306: finishing write tid 3 to trial11545420-29 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1306: finishing write tid 4 to trial11545420-29 2026-02-17T05:53:34.145 INFO:tasks.rados.rados.0.trial115.stdout:1306: oid 29 updating version 0 to 532 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:1306: oid 29 updating version 532 to 534 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:1306: oid 29 version 534 is already newer than 533 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 534 (ObjNum 497 snap 130 seq_num 497) dirty exists 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:1306: left oid 29 (ObjNum 497 snap 130 seq_num 497) 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:1307: finishing write tid 1 to trial11545420-37 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:1307: finishing write tid 2 to trial11545420-37 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:1307: finishing write tid 3 to trial11545420-37 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:1307: finishing write tid 4 to trial11545420-37 2026-02-17T05:53:34.146 INFO:tasks.rados.rados.0.trial115.stdout:1307: oid 37 updating version 0 to 481 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1307: oid 37 version 481 is already newer than 479 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1307: oid 37 version 481 is already newer than 480 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 481 (ObjNum 498 snap 130 seq_num 498) dirty exists 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1307: left oid 37 (ObjNum 498 snap 130 seq_num 498) 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1308: finishing write tid 1 to trial11545420-45 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1308: finishing write tid 2 to trial11545420-45 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1308: finishing write tid 3 to trial11545420-45 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1308: finishing write tid 4 to trial11545420-45 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1308: oid 45 updating version 0 to 744 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1308: oid 45 version 744 is already newer than 742 2026-02-17T05:53:34.147 INFO:tasks.rados.rados.0.trial115.stdout:1308: oid 45 version 744 is already newer than 743 2026-02-17T05:53:34.148 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 744 (ObjNum 499 snap 130 seq_num 499) dirty exists 2026-02-17T05:53:34.148 INFO:tasks.rados.rados.0.trial115.stdout:1308: left oid 45 (ObjNum 499 snap 130 seq_num 499) 2026-02-17T05:53:34.148 INFO:tasks.rados.rados.0.trial115.stdout:1309: expect (ObjNum 438 snap 110 seq_num 438) 2026-02-17T05:53:34.371 INFO:tasks.rados.rados.0.trial115.stdout:1310: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:53:34.515 INFO:tasks.rados.rados.0.trial115.stdout:1311: finishing write tid 1 to trial11545420-18 2026-02-17T05:53:34.515 INFO:tasks.rados.rados.0.trial115.stdout:1305: done (7 left) 2026-02-17T05:53:34.515 INFO:tasks.rados.rados.0.trial115.stdout:1306: done (6 left) 2026-02-17T05:53:34.515 INFO:tasks.rados.rados.0.trial115.stdout:1307: done (5 left) 2026-02-17T05:53:34.515 INFO:tasks.rados.rados.0.trial115.stdout:1308: done (4 left) 2026-02-17T05:53:34.515 INFO:tasks.rados.rados.0.trial115.stdout:1309: done (3 left) 2026-02-17T05:53:34.515 INFO:tasks.rados.rados.0.trial115.stdout:1310: done (2 left) 2026-02-17T05:53:34.515 INFO:tasks.rados.rados.0.trial115.stdout:1313: done (1 left) 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1314: write oid 3 current snap is 130 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1311: finishing write tid 2 to trial11545420-18 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1311: finishing write tid 3 to trial11545420-18 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1311: finishing write tid 4 to trial11545420-18 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1311: oid 18 updating version 0 to 569 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1311: oid 18 updating version 569 to 570 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1311: oid 18 version 570 is already newer than 568 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 570 (ObjNum 500 snap 130 seq_num 500) dirty exists 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1311: left oid 18 (ObjNum 500 snap 130 seq_num 500) 2026-02-17T05:53:34.516 INFO:tasks.rados.rados.0.trial115.stdout:1314: seq_num 502 ranges {437114=654296,1525843=477506} 2026-02-17T05:53:34.518 INFO:tasks.rados.rados.0.trial115.stdout:1314: writing trial11545420-3 from 437114 to 1091410 tid 1 2026-02-17T05:53:34.522 INFO:tasks.rados.rados.0.trial115.stdout:1314: writing trial11545420-3 from 1525843 to 2003349 tid 2 2026-02-17T05:53:34.522 INFO:tasks.rados.rados.0.trial115.stdout:1311: done (1 left) 2026-02-17T05:53:34.522 INFO:tasks.rados.rados.0.trial115.stdout:1315: copy_from oid 27 from oid 17 current snap is 130 2026-02-17T05:53:34.522 INFO:tasks.rados.rados.0.trial115.stdout:1316: write (excl) oid 47 current snap is 130 2026-02-17T05:53:34.522 INFO:tasks.rados.rados.0.trial115.stdout:1316: seq_num 503 ranges {453921=729545,1584325=621164} 2026-02-17T05:53:34.526 INFO:tasks.rados.rados.0.trial115.stdout:1316: writing trial11545420-47 from 453921 to 1183466 tid 1 2026-02-17T05:53:34.530 INFO:tasks.rados.rados.0.trial115.stdout:1316: writing trial11545420-47 from 1584325 to 2205489 tid 2 2026-02-17T05:53:34.530 INFO:tasks.rados.rados.0.trial115.stdout:1317: delete oid 14 current snap is 130 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1314: finishing write tid 1 to trial11545420-3 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1314: finishing write tid 2 to trial11545420-3 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1314: finishing write tid 3 to trial11545420-3 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1314: finishing write tid 4 to trial11545420-3 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1314: oid 3 updating version 0 to 680 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1314: oid 3 updating version 680 to 681 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1314: oid 3 updating version 681 to 682 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 682 (ObjNum 502 snap 130 seq_num 502) dirty exists 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1314: left oid 3 (ObjNum 502 snap 130 seq_num 502) 2026-02-17T05:53:34.531 INFO:tasks.rados.rados.0.trial115.stdout:1316: finishing write tid 1 to trial11545420-47 2026-02-17T05:53:34.532 INFO:tasks.rados.rados.0.trial115.stdout:1316: finishing write tid 2 to trial11545420-47 2026-02-17T05:53:34.532 INFO:tasks.rados.rados.0.trial115.stdout:1316: finishing write tid 3 to trial11545420-47 2026-02-17T05:53:34.532 INFO:tasks.rados.rados.0.trial115.stdout:1316: finishing write tid 4 to trial11545420-47 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:1316: oid 47 updating version 0 to 686 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:1316: oid 47 version 686 is already newer than 684 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:1316: oid 47 version 686 is already newer than 685 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 686 (ObjNum 503 snap 130 seq_num 503) dirty exists 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:1316: left oid 47 (ObjNum 503 snap 130 seq_num 503) 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:1314: done (3 left) 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:1316: done (2 left) 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:1317: done (1 left) 2026-02-17T05:53:34.533 INFO:tasks.rados.rados.0.trial115.stdout:1318: snap_remove snap 123 2026-02-17T05:53:34.939 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:53:34.939 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:53:35.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.008+0000 7fc90fd3e640 10 monclient: get_monmap_and_config 2026-02-17T05:53:35.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.008+0000 7fc90fd3e640 10 monclient: build_initial_monmap 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.008+0000 7fc90fd3e640 10 monclient: monmap: 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:35.009717+0000 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:35.009717+0000 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:35.011 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.008+0000 7fc90fd3e640 1 Processor -- start 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.008+0000 7fc90fd3e640 1 -- start start 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.008+0000 7fc90fd3e640 10 monclient: init 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc90806c890 0x7fc9080a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient(hunting): picked mon.noname-c con 0x7fc90806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc90806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient(hunting): picked mon.noname-b con 0x7fc90806cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc908134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc908074150 con 0x7fc908134df0 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc90809ef70 con 0x7fc90806c890 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc90809f250 con 0x7fc90806cc70 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90fd3e640 10 monclient(hunting): authenticate will time out at 1060.563253s 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc90806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc90806d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33456/0 (socket says 10.20.193.78:33456) 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 1 -- 10.20.193.78:0/1900130612 learned_addr learned my addr 10.20.193.78:0/1900130612 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient(hunting): get_auth_request con 0x7fc90806cc70 auth_method 0 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90dab3640 1 --2- 10.20.193.78:0/1900130612 >> v2:10.20.193.115:3300/0 conn(0x7fc90806c890 0x7fc9080a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90dab3640 10 monclient(hunting): get_auth_request con 0x7fc90806c890 auth_method 0 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90dab3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90dab3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90dab3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient(hunting): handle_auth_done global_id 74408 payload 274 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 1 -- 10.20.193.78:0/1900130612 >> v2:10.20.193.115:3300/0 conn(0x7fc90806c890 msgr2=0x7fc9080a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 1 --2- 10.20.193.78:0/1900130612 >> v2:10.20.193.115:3300/0 conn(0x7fc90806c890 0x7fc9080a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 1 -- 10.20.193.78:0/1900130612 >> v1:10.20.193.78:6789/0 conn(0x7fc908134df0 legacy=0x7fc908135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient: _finish_hunting 0 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 1 monclient: found mon.noname-b 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:53:35.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.009+0000 7fc90d2b2640 1 -- 10.20.193.78:0/1900130612 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90809df40 con 0x7fc90806cc70 2026-02-17T05:53:35.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.011+0000 7fc90d2b2640 1 -- 10.20.193.78:0/1900130612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 msgr2=0x7fc90806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.011+0000 7fc90d2b2640 1 --2- 10.20.193.78:0/1900130612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc90806d050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009970 tx=0x7fc8fc02efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc334f6fe891b2c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 1 -- 10.20.193.78:0/1900130612 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc03c070 con 0x7fc90806cc70 2026-02-17T05:53:35.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: dump: 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.023 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: _finish_auth 0 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: _check_auth_tickets 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 1 -- 10.20.193.78:0/1900130612 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc02fdf0 con 0x7fc90806cc70 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 1 -- 10.20.193.78:0/1900130612 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc037660 con 0x7fc90806cc70 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 5 monclient: authenticate success, global_id 74408 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90cab1640 10 monclient: dump: 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.024 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 4 set_mon_vals no callback set 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 10 monclient: shutdown 2026-02-17T05:53:35.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 1 -- 10.20.193.78:0/1900130612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 msgr2=0x7fc90806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/1900130612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc90806d050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009970 tx=0x7fc8fc02efb0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 1 -- 10.20.193.78:0/1900130612 shutdown_connections 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/1900130612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc90806d050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/1900130612 >> v2:10.20.193.115:3300/0 conn(0x7fc90806c890 0x7fc9080a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 1 -- 10.20.193.78:0/1900130612 >> 10.20.193.78:0/1900130612 conn(0x7fc9080a6620 msgr2=0x7fc9080a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 1 -- 10.20.193.78:0/1900130612 shutdown_connections 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 1 -- 10.20.193.78:0/1900130612 wait complete. 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 10 monclient: build_initial_monmap 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.012+0000 7fc90fd3e640 10 monclient: monmap: 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:35.026 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 1 Processor -- start 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 1 -- start start 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient: init 2026-02-17T05:53:35.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc90806c890 0x7fc9081cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient(hunting): picked mon.noname-a con 0x7fc90806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc9081248d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient(hunting): picked mon.noname-c con 0x7fc90806cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc908134df0 0x7fc908124e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient(hunting): picked mon.noname-b con 0x7fc908134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc90809d360 con 0x7fc90806c890 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc90809d1e0 con 0x7fc908134df0 2026-02-17T05:53:35.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc908135950 con 0x7fc90806cc70 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90fd3e640 10 monclient(hunting): authenticate will time out at 1060.567379s 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc90806c890 0x7fc9081cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc9081248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc90806c890 0x7fc9081cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35356/0 (socket says 10.20.193.78:35356) 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 -- 10.20.193.78:0/899351291 learned_addr learned my addr 10.20.193.78:0/899351291 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient(hunting): get_auth_request con 0x7fc90806c890 auth_method 0 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 10 monclient(hunting): get_auth_request con 0x7fc90806cc70 auth_method 0 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 1 -- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 msgr2=0x7fc9081248d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 1 -- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 msgr2=0x7fc9081248d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc9081248d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90d2b2640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc9081248d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90e2b4640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc908134df0 0x7fc908124e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90e2b4640 10 monclient(hunting): get_auth_request con 0x7fc908134df0 auth_method 0 2026-02-17T05:53:35.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90e2b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90e2b4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90e2b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient(hunting): handle_auth_done global_id 104127 payload 824 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 -- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 msgr2=0x7fc9081248d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc9081248d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 -- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc908134df0 msgr2=0x7fc908124e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc908134df0 0x7fc908124e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient: _finish_hunting 0 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 monclient: found mon.noname-a 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.013+0000 7fc90dab3640 1 -- 10.20.193.78:0/899351291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc908128570 con 0x7fc90806c890 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90dab3640 1 -- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc90806c890 msgr2=0x7fc9081cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90dab3640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc90806c890 0x7fc9081cf650 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800e7c0 tx=0x7fc8f800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 1 -- 10.20.193.78:0/899351291 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8017070 con 0x7fc90806c890 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: dump: 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.032 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: _finish_auth 0 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 1 -- 10.20.193.78:0/899351291 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f800cae0 con 0x7fc90806c890 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90fd3e640 5 monclient: authenticate success, global_id 104127 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 1 -- 10.20.193.78:0/899351291 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800f040 con 0x7fc90806c890 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f7fff640 4 set_mon_vals no callback set 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc8f67fc640 10 monclient: dump: 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90fd3e640 10 monclient: _renew_subs 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90fd3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc908128850 con 0x7fc90806c890 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90fd3e640 10 monclient: _renew_subs 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90fd3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.015+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90809d810 con 0x7fc90806c890 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.016+0000 7fc90fd3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.016+0000 7fc90fd3e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.016+0000 7fc90fd3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.016+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc908074980 con 0x7fc90806c890 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.016+0000 7fc8f67fc640 1 -- 10.20.193.78:0/899351291 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc8f8012050 con 0x7fc90806c890 2026-02-17T05:53:35.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.016+0000 7fc8f67fc640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc8d803deb0 0x7fc8d8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.016+0000 7fc8f67fc640 1 -- 10.20.193.78:0/899351291 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(282..282 src has 117..282) ==== 4827+0+0 (secure 0 0 0) 0x7fc8f80526a0 con 0x7fc90806c890 2026-02-17T05:53:35.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.017+0000 7fc90d2b2640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc8d803deb0 0x7fc8d8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.017+0000 7fc90d2b2640 10 monclient: get_auth_request con 0x7fc8d803deb0 auth_method 0 2026-02-17T05:53:35.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.019+0000 7fc8f67fc640 1 -- 10.20.193.78:0/899351291 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc8f8023d80 con 0x7fc90806c890 2026-02-17T05:53:35.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.019+0000 7fc8f67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.019+0000 7fc8f67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:35.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.019+0000 7fc90d2b2640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc8d803deb0 0x7fc8d8040370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc036000 tx=0x7fc8fc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.133+0000 7fc90fd3e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:53:35.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.133+0000 7fc90fd3e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:35.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.133+0000 7fc90fd3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.133+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc908126e40 con 0x7fc90806c890 2026-02-17T05:53:35.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.133+0000 7fc8f67fc640 1 -- 10.20.193.78:0/899351291 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fc8f8013c10 con 0x7fc90806c890 2026-02-17T05:53:35.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.133+0000 7fc8f67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:35.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.133+0000 7fc8f67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:35.134 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:53:35.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc8d803deb0 msgr2=0x7fc8d8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc8d803deb0 0x7fc8d8040370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc036000 tx=0x7fc8fc033000 comp rx=0 tx=0).stop 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 10 monclient: shutdown 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc90806c890 msgr2=0x7fc9081cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc90806c890 0x7fc9081cf650 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800e7c0 tx=0x7fc8f800ec90 comp rx=0 tx=0).stop 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 shutdown_connections 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc8d803deb0 0x7fc8d8040370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc908134df0 0x7fc908124e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc90806cc70 0x7fc9081248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 --2- 10.20.193.78:0/899351291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc90806c890 0x7fc9081cf650 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 >> 10.20.193.78:0/899351291 conn(0x7fc9080a6620 msgr2=0x7fc908073260 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 shutdown_connections 2026-02-17T05:53:35.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.134+0000 7fc90fd3e640 1 -- 10.20.193.78:0/899351291 wait complete. 2026-02-17T05:53:35.141 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:53:35.141 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:53:35.141 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:53:35.142 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:53:35.147 INFO:tasks.rados.rados.0.trial115.stdout:1318: done (1 left) 2026-02-17T05:53:35.147 INFO:tasks.rados.rados.0.trial115.stdout:1319: write oid 4 current snap is 130 2026-02-17T05:53:35.147 INFO:tasks.rados.rados.0.trial115.stdout:1319: seq_num 505 ranges {487535=480043,1584936=624833} 2026-02-17T05:53:35.151 INFO:tasks.rados.rados.0.trial115.stdout:1319: writing trial11545420-4 from 487535 to 967578 tid 1 2026-02-17T05:53:35.156 INFO:tasks.rados.rados.0.trial115.stdout:1319: writing trial11545420-4 from 1584936 to 2209769 tid 2 2026-02-17T05:53:35.156 INFO:tasks.rados.rados.0.trial115.stdout:1320: delete oid 2 current snap is 130 2026-02-17T05:53:35.156 INFO:tasks.rados.rados.0.trial115.stdout:1315: finishing copy_from racing read to trial11545420-27 2026-02-17T05:53:35.156 INFO:tasks.rados.rados.0.trial115.stdout:1315: finishing copy_from to trial11545420-27 2026-02-17T05:53:35.156 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 240 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:53:35.156 INFO:tasks.rados.rados.0.trial115.stdout:1319: finishing write tid 1 to trial11545420-4 2026-02-17T05:53:35.157 INFO:tasks.rados.rados.0.trial115.stdout:1315: done (2 left) 2026-02-17T05:53:35.157 INFO:tasks.rados.rados.0.trial115.stdout:1320: done (1 left) 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:1321: read oid 28 snap 122 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:1321: expect deleted 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:1322: read oid 10 snap -1 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:1322: expect (ObjNum 453 snap 112 seq_num 453) 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:1323: rollback oid 24 current snap is 130 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 126 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:1324: rollback oid 20 current snap is 130 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 122 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:1325: write oid 45 current snap is 130 2026-02-17T05:53:35.158 INFO:tasks.rados.rados.0.trial115.stdout:1325: seq_num 507 ranges {521149=630541,1701900=628502} 2026-02-17T05:53:35.162 INFO:tasks.rados.rados.0.trial115.stdout:1325: writing trial11545420-45 from 521149 to 1151690 tid 1 2026-02-17T05:53:35.167 INFO:tasks.rados.rados.0.trial115.stdout:1325: writing trial11545420-45 from 1701900 to 2330402 tid 2 2026-02-17T05:53:35.167 INFO:tasks.rados.rados.0.trial115.stdout:1326: delete oid 47 current snap is 130 2026-02-17T05:53:35.167 INFO:tasks.rados.rados.0.trial115.stdout:1319: finishing write tid 2 to trial11545420-4 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1324: finishing rollback tid 1 to trial11545420-20 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 472 (ObjNum 33 snap 0 seq_num 1330677968) dirty dne 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1319: finishing write tid 3 to trial11545420-4 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1319: finishing write tid 4 to trial11545420-4 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1319: oid 4 updating version 0 to 351 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1319: oid 4 version 351 is already newer than 350 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1319: oid 4 version 351 is already newer than 349 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 351 (ObjNum 505 snap 130 seq_num 505) dirty exists 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1319: left oid 4 (ObjNum 505 snap 130 seq_num 505) 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1323: finishing rollback tid 0 to trial11545420-24 2026-02-17T05:53:35.168 INFO:tasks.rados.rados.0.trial115.stdout:1323: finishing rollback tid 1 to trial11545420-24 2026-02-17T05:53:35.169 INFO:tasks.rados.rados.0.trial115.stdout:1323: finishing rollback tid 2 to trial11545420-24 2026-02-17T05:53:35.169 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 756 (ObjNum 480 snap 124 seq_num 480) dirty exists 2026-02-17T05:53:35.169 INFO:tasks.rados.rados.0.trial115.stdout:1322: expect (ObjNum 453 snap 112 seq_num 453) 2026-02-17T05:53:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient: get_monmap_and_config 2026-02-17T05:53:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient: build_initial_monmap 2026-02-17T05:53:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient: monmap: 2026-02-17T05:53:35.213 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:35.213 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.213 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:35.212314+0000 2026-02-17T05:53:35.213 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:35.212314+0000 2026-02-17T05:53:35.213 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 1 Processor -- start 2026-02-17T05:53:35.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 1 -- start start 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient: init 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006c890 0x7f41d006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient(hunting): picked mon.noname-b con 0x7f41d006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f41d009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f41d006d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d0135c10 con 0x7f41d009c4a0 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d006daf0 con 0x7f41d006d1b0 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f41d00a0880 con 0x7f41d006c890 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d7a7c640 10 monclient(hunting): authenticate will time out at 1060.765854s 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d57f1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006c890 0x7f41d006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d4ff0640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f41d009c4a0 0x7f41d0134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57028/0 (socket says 10.20.193.78:57028) 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d4ff0640 1 -- 10.20.193.78:0/2476835751 learned_addr learned my addr 10.20.193.78:0/2476835751 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d57f1640 10 monclient(hunting): get_auth_request con 0x7f41d006c890 auth_method 0 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d57f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d57f1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.211+0000 7f41d57f1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 10 monclient(hunting): handle_auth_done global_id 74411 payload 274 2026-02-17T05:53:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 1 -- 10.20.193.78:0/2476835751 >> v1:10.20.193.115:6789/0 conn(0x7f41d006d1b0 legacy=0x7f41d0135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 1 -- 10.20.193.78:0/2476835751 >> v1:10.20.193.78:6789/0 conn(0x7f41d009c4a0 legacy=0x7f41d0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 10 monclient: _finish_hunting 0 2026-02-17T05:53:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 1 monclient: found mon.noname-b 2026-02-17T05:53:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:53:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.212+0000 7f41d57f1640 1 -- 10.20.193.78:0/2476835751 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d009cc80 con 0x7f41d006c890 2026-02-17T05:53:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d57f1640 1 -- 10.20.193.78:0/2476835751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006c890 msgr2=0x7f41d006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d57f1640 1 --2- 10.20.193.78:0/2476835751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006c890 0x7f41d006cc70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41c00098d0 tx=0x7f41c002ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f140988bcd7d66 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 1 -- 10.20.193.78:0/2476835751 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c003c070 con 0x7f41d006c890 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: dump: 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.228 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: _finish_auth 0 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: _check_auth_tickets 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d7a7c640 5 monclient: authenticate success, global_id 74411 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 1 -- 10.20.193.78:0/2476835751 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c002fcc0 con 0x7f41d006c890 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d7a7c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d7a7c640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d7a7c640 4 set_mon_vals no callback set 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 1 -- 10.20.193.78:0/2476835751 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0037660 con 0x7f41d006c890 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41bffff640 10 monclient: dump: 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.230 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d7a7c640 10 monclient: shutdown 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d7a7c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d7a7c640 1 -- 10.20.193.78:0/2476835751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006c890 msgr2=0x7f41d006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.214+0000 7f41d7a7c640 1 --2- 10.20.193.78:0/2476835751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006c890 0x7f41d006cc70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41c00098d0 tx=0x7f41c002ed60 comp rx=0 tx=0).stop 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 -- 10.20.193.78:0/2476835751 shutdown_connections 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 --2- 10.20.193.78:0/2476835751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006c890 0x7f41d006cc70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 -- 10.20.193.78:0/2476835751 >> 10.20.193.78:0/2476835751 conn(0x7f41d00a6620 msgr2=0x7f41d00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 -- 10.20.193.78:0/2476835751 shutdown_connections 2026-02-17T05:53:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 -- 10.20.193.78:0/2476835751 wait complete. 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient: build_initial_monmap 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient: monmap: 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:35.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 Processor -- start 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 -- start start 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient: init 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d006c890 0x7f41d0133f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient(hunting): picked mon.noname-a con 0x7f41d006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006d1b0 0x7f41d01344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient(hunting): picked mon.noname-c con 0x7f41d006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f41d009c4a0 0x7f41d0121a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient(hunting): picked mon.noname-b con 0x7f41d009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41d00a0020 con 0x7f41d006c890 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f41d006d600 con 0x7f41d009c4a0 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f41d00a01a0 con 0x7f41d006d1b0 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.215+0000 7f41d7a7c640 10 monclient(hunting): authenticate will time out at 1060.769985s 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d006c890 0x7f41d0133f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d4ff0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006d1b0 0x7f41d01344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d006c890 0x7f41d0133f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35368/0 (socket says 10.20.193.78:35368) 2026-02-17T05:53:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 -- 10.20.193.78:0/841203979 learned_addr learned my addr 10.20.193.78:0/841203979 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient(hunting): get_auth_request con 0x7f41d006c890 auth_method 0 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d4ff0640 10 monclient(hunting): get_auth_request con 0x7f41d006d1b0 auth_method 0 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d4ff0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d4ff0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d4ff0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d5ff2640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f41d009c4a0 0x7f41d0121a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d5ff2640 10 monclient(hunting): get_auth_request con 0x7f41d009c4a0 auth_method 0 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d5ff2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d5ff2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d5ff2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient(hunting): handle_auth_done global_id 104130 payload 824 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006d1b0 msgr2=0x7f41d01344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f41d006d1b0 0x7f41d01344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f41d009c4a0 msgr2=0x7f41d0121a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f41d009c4a0 0x7f41d0121a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient: _finish_hunting 0 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 monclient: found mon.noname-a 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.216+0000 7f41d57f1640 1 -- 10.20.193.78:0/841203979 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d0134f10 con 0x7f41d006c890 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d57f1640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d006c890 msgr2=0x7f41d0133f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d57f1640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d006c890 0x7f41d0133f90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f41c00098d0 tx=0x7f41c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0046070 con 0x7f41d006c890 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: dump: 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.237 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: _finish_auth 0 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: _check_auth_tickets 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0041870 con 0x7f41d006c890 2026-02-17T05:53:35.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d7a7c640 5 monclient: authenticate success, global_id 104130 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d7a7c640 10 monclient: _renew_subs 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d7a7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d0124420 con 0x7f41d006c890 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c003c040 con 0x7f41d006c890 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bdffb640 10 monclient: dump: 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.239 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41bf7fe640 4 set_mon_vals no callback set 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d7a7c640 10 monclient: _renew_subs 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d7a7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d009d810 con 0x7f41d006c890 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.218+0000 7f41d7a7c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d7a7c640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d01cd3a0 0x7f41d0121f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d7a7c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0125af0 con 0x7f41d01cd3a0 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f41c0041c60 con 0x7f41d006c890 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d01cd3a0 0x7f41d0121f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 10 monclient: get_auth_request con 0x7f41d01cd3a0 auth_method 0 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_done global_id 104133 payload 824 2026-02-17T05:53:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d4ff0640 1 -- 10.20.193.78:0/841203979 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41d0068410 con 0x7f41d01cd3a0 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41bdffb640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f41a003deb0 0x7f41a0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(283..283 src has 117..283) ==== 4843+0+0 (secure 0 0 0) 0x7f41c0078100 con 0x7f41d006c890 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d5ff2640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f41a003deb0 0x7f41a0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.219+0000 7f41d5ff2640 10 monclient: get_auth_request con 0x7f41a003deb0 auth_method 0 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41d4ff0640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d01cd3a0 msgr2=0x7f41d0121f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41d4ff0640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d01cd3a0 0x7f41d0121f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41d01232a0 tx=0x7f41c400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c400cbf0 con 0x7f41d01cd3a0 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 10 monclient: dump: 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.242 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f41c4019070 con 0x7f41d01cd3a0 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d01cd3a0 msgr2=0x7f41d0121f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d01cd3a0 0x7f41d0121f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41d01232a0 tx=0x7f41c400ef10 comp rx=0 tx=0).stop 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d01cd3a0 msgr2=0x7f41d0121f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:35.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.221+0000 7f41bdffb640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d01cd3a0 0x7f41d0121f40 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41d01232a0 tx=0x7f41c400ef10 comp rx=0 tx=0).stop 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.222+0000 7f41d5ff2640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f41a003deb0 0x7f41a0040370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41d0073e30 tx=0x7f41cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d7a7c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d7a7c640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d0122480 0x7f41d01d8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d7a7c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0122890 con 0x7f41d0122480 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d0122480 0x7f41d01d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 10 monclient: get_auth_request con 0x7f41d0122480 auth_method 0 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 10 monclient(hunting): handle_auth_done global_id 104136 payload 824 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.224+0000 7f41d4ff0640 1 -- 10.20.193.78:0/841203979 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f41d006cc70 con 0x7f41d0122480 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d4ff0640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d0122480 msgr2=0x7f41d01d8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d4ff0640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d0122480 0x7f41d01d8480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f41d01232a0 tx=0x7f41c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c401e230 con 0x7f41d0122480 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 10 monclient: dump: 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.245 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f41c401e230 con 0x7f41d0122480 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d0122480 msgr2=0x7f41d01d8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d0122480 0x7f41d01d8480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f41d01232a0 tx=0x7f41c4004a40 comp rx=0 tx=0).stop 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d0122480 msgr2=0x7f41d01d8480 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41bdffb640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d0122480 0x7f41d01d8480 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f41d01232a0 tx=0x7f41c4004a40 comp rx=0 tx=0).stop 2026-02-17T05:53:35.246 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 82, 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 8379, 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:35.247 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.248 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:35.249 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.250 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:35.251 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:35.252 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.253 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.254 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.255 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.256 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.256 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:53:35.256 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.256 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.256 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.256 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:35.256 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f41a003deb0 msgr2=0x7f41a0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f41a003deb0 0x7f41a0040370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41d0073e30 tx=0x7f41cc009040 comp rx=0 tx=0).stop 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 10 monclient: shutdown 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d006c890 msgr2=0x7f41d0133f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d5ff2640 1 -- 10.20.193.78:0/841203979 reap_dead start 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d006c890 0x7f41d0133f90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f41c00098d0 tx=0x7f41c00047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 shutdown_connections 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 --2- 10.20.193.78:0/841203979 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f41d006c890 0x7f41d0133f90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 >> 10.20.193.78:0/841203979 conn(0x7f41d00a6620 msgr2=0x7f41d0135460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 shutdown_connections 2026-02-17T05:53:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.227+0000 7f41d7a7c640 1 -- 10.20.193.78:0/841203979 wait complete. 2026-02-17T05:53:35.258 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:53:35.258 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:53:35.258 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:53:35.317 INFO:tasks.rados.rados.0.trial115.stdout:1325: finishing write tid 1 to trial11545420-45 2026-02-17T05:53:35.317 INFO:tasks.rados.rados.0.trial115.stdout:1325: finishing write tid 2 to trial11545420-45 2026-02-17T05:53:35.317 INFO:tasks.rados.rados.0.trial115.stdout:1325: finishing write tid 3 to trial11545420-45 2026-02-17T05:53:35.317 INFO:tasks.rados.rados.0.trial115.stdout:1325: finishing write tid 4 to trial11545420-45 2026-02-17T05:53:35.317 INFO:tasks.rados.rados.0.trial115.stdout:1325: oid 45 updating version 0 to 758 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1325: oid 45 version 758 is already newer than 757 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1325: oid 45 updating version 758 to 759 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 759 (ObjNum 507 snap 130 seq_num 507) dirty exists 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1325: left oid 45 (ObjNum 507 snap 130 seq_num 507) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1319: done (6 left) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1321: done (5 left) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1322: done (4 left) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1323: done (3 left) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1324: done (2 left) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1325: done (1 left) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1326: done (0 left) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1327: read oid 13 snap -1 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1327: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:53:35.318 INFO:tasks.rados.rados.0.trial115.stdout:1328: read oid 16 snap 122 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1328: expect (ObjNum 435 snap 109 seq_num 435) 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1329: rollback oid 40 current snap is 130 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 128 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1330: rollback oid 34 current snap is 130 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 117 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1331: read oid 36 snap -1 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1331: expect (ObjNum 492 snap 127 seq_num 492) 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1332: delete oid 47 current snap is 130 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1330: finishing rollback tid 1 to trial11545420-34 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1330: done (5 left) 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1332: done (4 left) 2026-02-17T05:53:35.319 INFO:tasks.rados.rados.0.trial115.stdout:1333: delete oid 23 current snap is 130 2026-02-17T05:53:35.320 INFO:tasks.rados.rados.0.trial115.stdout:1329: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:53:35.320 INFO:tasks.rados.rados.0.trial115.stdout:1329: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:53:35.320 INFO:tasks.rados.rados.0.trial115.stdout:1329: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:53:35.320 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 355 (ObjNum 451 snap 112 seq_num 451) dirty exists 2026-02-17T05:53:35.320 INFO:tasks.rados.rados.0.trial115.stdout:1328: expect (ObjNum 435 snap 109 seq_num 435) 2026-02-17T05:53:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.327+0000 7f331dcf3640 10 monclient: get_monmap_and_config 2026-02-17T05:53:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.327+0000 7f331dcf3640 10 monclient: build_initial_monmap 2026-02-17T05:53:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.327+0000 7f331dcf3640 10 monclient: monmap: 2026-02-17T05:53:35.326 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:35.328582+0000 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:35.328582+0000 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:35.327 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.327+0000 7f331dcf3640 1 Processor -- start 2026-02-17T05:53:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.327+0000 7f331dcf3640 1 -- start start 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.327+0000 7f331dcf3640 10 monclient: init 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f331806c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f331806d380 0x7f33180a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient(hunting): picked mon.noname-a con 0x7f331806d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f331813a240 0x7f331813c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient(hunting): picked mon.noname-c con 0x7f331813a240 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3318074150 con 0x7f331806c890 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f331809ef70 con 0x7f331806d380 2026-02-17T05:53:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f331809f250 con 0x7f331813a240 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f331dcf3640 10 monclient(hunting): authenticate will time out at 1060.882102s 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f331806d380 0x7f33180a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f331806d380 0x7f33180a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35404/0 (socket says 10.20.193.78:35404) 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 1 -- 10.20.193.78:0/690869472 learned_addr learned my addr 10.20.193.78:0/690869472 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient(hunting): get_auth_request con 0x7f331806d380 auth_method 0 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3317fff640 1 --2- 10.20.193.78:0/690869472 >> v2:10.20.193.115:3300/0 conn(0x7f331813a240 0x7f331813c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3317fff640 10 monclient(hunting): get_auth_request con 0x7f331813a240 auth_method 0 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3317fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3317fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3317fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient(hunting): handle_auth_done global_id 104139 payload 274 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 1 -- 10.20.193.78:0/690869472 >> v2:10.20.193.115:3300/0 conn(0x7f331813a240 msgr2=0x7f331813c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 1 --2- 10.20.193.78:0/690869472 >> v2:10.20.193.115:3300/0 conn(0x7f331813a240 0x7f331813c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 1 -- 10.20.193.78:0/690869472 >> v1:10.20.193.115:6789/0 conn(0x7f331806c890 legacy=0x7f331806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 1 monclient: found mon.noname-a 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.328+0000 7f3316ffd640 1 -- 10.20.193.78:0/690869472 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331809df40 con 0x7f331806d380 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.330+0000 7f3316ffd640 1 -- 10.20.193.78:0/690869472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806d380 msgr2=0x7f33180a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.330+0000 7f3316ffd640 1 --2- 10.20.193.78:0/690869472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806d380 0x7f33180a01f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3318069d40 tx=0x7f330802eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46075823803af331 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.330+0000 7f33167fc640 1 -- 10.20.193.78:0/690869472 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330803c070 con 0x7f331806d380 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.330+0000 7f33167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 10 monclient: dump: 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 10 monclient: _finish_auth 0 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 10 monclient: _check_auth_tickets 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 5 monclient: authenticate success, global_id 104139 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 1 -- 10.20.193.78:0/690869472 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330802fd50 con 0x7f331806d380 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 1 -- 10.20.193.78:0/690869472 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308037660 con 0x7f331806d380 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f33167fc640 10 monclient: dump: 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:35.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 4 set_mon_vals no callback set 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 10 monclient: shutdown 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 -- 10.20.193.78:0/690869472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806d380 msgr2=0x7f33180a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 --2- 10.20.193.78:0/690869472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806d380 0x7f33180a01f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3318069d40 tx=0x7f330802eed0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 -- 10.20.193.78:0/690869472 shutdown_connections 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 --2- 10.20.193.78:0/690869472 >> v2:10.20.193.115:3300/0 conn(0x7f331813a240 0x7f331813c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 --2- 10.20.193.78:0/690869472 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806d380 0x7f33180a01f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 -- 10.20.193.78:0/690869472 >> 10.20.193.78:0/690869472 conn(0x7f33180a6620 msgr2=0x7f33180a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 -- 10.20.193.78:0/690869472 shutdown_connections 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 -- 10.20.193.78:0/690869472 wait complete. 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 10 monclient: build_initial_monmap 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 10 monclient: monmap: 2026-02-17T05:53:35.343 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 Processor -- start 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 1 -- start start 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.331+0000 7f331dcf3640 10 monclient: init 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:53:35.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806c890 0x7f33181cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient(hunting): picked mon.noname-a con 0x7f331806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331806d380 0x7f33181353c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient(hunting): picked mon.noname-c con 0x7f331806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f331813a240 0x7f3318135900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient(hunting): picked mon.noname-b con 0x7f331813a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f331813ccf0 con 0x7f331806c890 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f331813cb70 con 0x7f331813a240 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f331813ce70 con 0x7f331806d380 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f331dcf3640 10 monclient(hunting): authenticate will time out at 1060.886309s 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806c890 0x7f33181cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806c890 0x7f33181cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35406/0 (socket says 10.20.193.78:35406) 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 1 -- 10.20.193.78:0/3690357285 learned_addr learned my addr 10.20.193.78:0/3690357285 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 10 monclient(hunting): get_auth_request con 0x7f331806c890 auth_method 0 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331806d380 0x7f33181353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient(hunting): get_auth_request con 0x7f331806d380 auth_method 0 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3317fff640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f331813a240 0x7f3318135900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3317fff640 10 monclient(hunting): get_auth_request con 0x7f331813a240 auth_method 0 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3317fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3317fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3317fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3317fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3317fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3317fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient(hunting): handle_auth_done global_id 74417 payload 824 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f331813a240 msgr2=0x7f3318135900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f331813a240 0x7f3318135900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806c890 msgr2=0x7f33181cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806c890 0x7f33181cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 1 monclient: found mon.noname-c 2026-02-17T05:53:35.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f3316ffd640 1 -- 10.20.193.78:0/3690357285 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331809d810 con 0x7f331806d380 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.332+0000 7f33177fe640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f331806c890 0x7f33181cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3316ffd640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331806d380 msgr2=0x7f33181353c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3316ffd640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331806d380 0x7f33181353c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3318074320 tx=0x7f3308004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330803c040 con 0x7f331806d380 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: dump: 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.348 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: _finish_auth 0 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: _check_auth_tickets 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3308047070 con 0x7f331806d380 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f331dcf3640 5 monclient: authenticate success, global_id 74417 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f331dcf3640 10 monclient: _renew_subs 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f331dcf3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33181227c0 con 0x7f331806d380 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33080417e0 con 0x7f331806d380 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3315ffb640 4 set_mon_vals no callback set 2026-02-17T05:53:35.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.334+0000 7f3303fff640 10 monclient: dump: 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f331dcf3640 10 monclient: _renew_subs 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f331dcf3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f331813c840 con 0x7f331806d380 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f331dcf3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f331dcf3640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811edf0 0x7f331811f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f331dcf3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f33180a22b0 con 0x7f331811edf0 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811edf0 0x7f331811f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f3316ffd640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331806d380 0x7f33181353c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3318074320 tx=0x7f3308004830 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f3308077ac0 osd_map(283..283 src has 117..283) 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 10 monclient: get_auth_request con 0x7f331811edf0 auth_method 0 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f33080419c0 con 0x7f331806d380 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 10 monclient(hunting): handle_auth_done global_id 74420 payload 824 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f33177fe640 1 -- 10.20.193.78:0/3690357285 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3318074980 con 0x7f331811edf0 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.335+0000 7f3303fff640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32e403deb0 0x7f32e4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.336+0000 7f3317fff640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32e403deb0 0x7f32e4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.336+0000 7f3317fff640 10 monclient: get_auth_request con 0x7f32e403deb0 auth_method 0 2026-02-17T05:53:35.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.337+0000 7f33177fe640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811edf0 msgr2=0x7f331811f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.337+0000 7f33177fe640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811edf0 0x7f331811f1d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f330400fc30 tx=0x7f330400d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.337+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304018020 con 0x7f331811edf0 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.337+0000 7f3303fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.337+0000 7f3303fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.337+0000 7f3303fff640 10 monclient: dump: 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.354 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.338+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f330400fe70 con 0x7f331811edf0 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.338+0000 7f3303fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.338+0000 7f3303fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.338+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811edf0 msgr2=0x7f331811f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.338+0000 7f3303fff640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811edf0 0x7f331811f1d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f330400fc30 tx=0x7f330400d520 comp rx=0 tx=0).stop 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.338+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811edf0 msgr2=0x7f331811f1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.338+0000 7f3303fff640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811edf0 0x7f331811f1d0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f330400fc30 tx=0x7f330400d520 comp rx=0 tx=0).stop 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.338+0000 7f3317fff640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32e403deb0 0x7f32e4040370 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3318135e40 tx=0x7f330c009770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f331dcf3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f331dcf3640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811f710 0x7f33181da740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f331dcf3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f33180a01d0 con 0x7f331811f710 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811f710 0x7f33181da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 10 monclient: get_auth_request con 0x7f331811f710 auth_method 0 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 10 monclient(hunting): handle_auth_done global_id 74423 payload 824 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.341+0000 7f33177fe640 1 -- 10.20.193.78:0/3690357285 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3318121370 con 0x7f331811f710 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f33177fe640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811f710 msgr2=0x7f33181da740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f33177fe640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811f710 0x7f33181da740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f330400fc30 tx=0x7f3304003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304018070 con 0x7f331811f710 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 10 monclient: dump: 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f3304014ca0 con 0x7f331811f710 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811f710 msgr2=0x7f33181da740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811f710 0x7f33181da740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f330400fc30 tx=0x7f3304003860 comp rx=0 tx=0).stop 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811f710 msgr2=0x7f33181da740 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.343+0000 7f3303fff640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331811f710 0x7f33181da740 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 82, 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 187235, 2026-02-17T05:53:35.358 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.359 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:35.360 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.361 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.362 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:35.363 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.364 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.365 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.366 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.366 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:35.366 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32e403deb0 msgr2=0x7f32e4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32e403deb0 0x7f32e4040370 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3318135e40 tx=0x7f330c009770 comp rx=0 tx=0).stop 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 10 monclient: shutdown 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331806d380 msgr2=0x7f33181353c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 1 --2- 10.20.193.78:0/3690357285 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f331806d380 0x7f33181353c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3318074320 tx=0x7f3308004830 comp rx=0 tx=0).stop 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f3317fff640 1 -- 10.20.193.78:0/3690357285 reap_dead start 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f3303fff640 1 -- 10.20.193.78:0/3690357285 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(283..283 src has 117..283) ==== 4843+0+0 (unknown 0 0 0) 0x7f3308077ac0 con 0x7f331806d380 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f3317fff640 1 -- 10.20.193.78:0/3690357285 reap_dead start 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 shutdown_connections 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 >> 10.20.193.78:0/3690357285 conn(0x7f33180a6620 msgr2=0x7f331809be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 shutdown_connections 2026-02-17T05:53:35.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.344+0000 7f331dcf3640 1 -- 10.20.193.78:0/3690357285 wait complete. 2026-02-17T05:53:35.367 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:53:35.367 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:53:35.368 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:53:35.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.434+0000 7f9206c6f640 10 monclient: get_monmap_and_config 2026-02-17T05:53:35.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.434+0000 7f9206c6f640 10 monclient: build_initial_monmap 2026-02-17T05:53:35.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.434+0000 7f9206c6f640 10 monclient: monmap: 2026-02-17T05:53:35.435 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:35.435885+0000 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:35.435885+0000 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 1 Processor -- start 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 1 -- start start 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient: init 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2026-02-17T05:53:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f920006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f920009c4a0 0x7f920013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient(hunting): picked mon.noname-c con 0x7f920009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920006d380 0x7f920013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient(hunting): picked mon.noname-b con 0x7f920006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f920006daf0 con 0x7f920006c890 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f92000a0880 con 0x7f920009c4a0 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f920009cc80 con 0x7f920006d380 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f9206c6f640 10 monclient(hunting): authenticate will time out at 1060.989477s 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92049e4640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f920006c890 0x7f920006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57030/0 (socket says 10.20.193.78:57030) 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92049e4640 1 -- 10.20.193.78:0/1969041679 learned_addr learned my addr 10.20.193.78:0/1969041679 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 1 --2- 10.20.193.78:0/1969041679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920006d380 0x7f920013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient(hunting): get_auth_request con 0x7f920006d380 auth_method 0 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f91f7fff640 1 --2- 10.20.193.78:0/1969041679 >> v2:10.20.193.115:3300/0 conn(0x7f920009c4a0 0x7f920013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f91f7fff640 10 monclient(hunting): get_auth_request con 0x7f920009c4a0 auth_method 0 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f91f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f91f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f91f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient(hunting): handle_auth_done global_id 74426 payload 274 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 1 -- 10.20.193.78:0/1969041679 >> v2:10.20.193.115:3300/0 conn(0x7f920009c4a0 msgr2=0x7f920013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 1 --2- 10.20.193.78:0/1969041679 >> v2:10.20.193.115:3300/0 conn(0x7f920009c4a0 0x7f920013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 1 -- 10.20.193.78:0/1969041679 >> v1:10.20.193.78:6789/0 conn(0x7f920006c890 legacy=0x7f920006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient: _finish_hunting 0 2026-02-17T05:53:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 1 monclient: found mon.noname-b 2026-02-17T05:53:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:53:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.435+0000 7f92051e5640 1 -- 10.20.193.78:0/1969041679 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920009cf60 con 0x7f920006d380 2026-02-17T05:53:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f92051e5640 1 -- 10.20.193.78:0/1969041679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920006d380 msgr2=0x7f920013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f92051e5640 1 --2- 10.20.193.78:0/1969041679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920006d380 0x7f920013d8f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009d90 tx=0x7f91fc02f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63413be512bff3f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 1 -- 10.20.193.78:0/1969041679 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc03c070 con 0x7f920006d380 2026-02-17T05:53:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: dump: 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: _finish_auth 0 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 5 monclient: authenticate success, global_id 74426 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 1 -- 10.20.193.78:0/1969041679 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91fc037440 con 0x7f920006d380 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 1 -- 10.20.193.78:0/1969041679 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc037810 con 0x7f920006d380 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f91f77fe640 10 monclient: dump: 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:35.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 4 set_mon_vals no callback set 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 10 monclient: shutdown 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 1 -- 10.20.193.78:0/1969041679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920006d380 msgr2=0x7f920013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 1 --2- 10.20.193.78:0/1969041679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920006d380 0x7f920013d8f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009d90 tx=0x7f91fc02f190 comp rx=0 tx=0).stop 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 1 -- 10.20.193.78:0/1969041679 shutdown_connections 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 1 --2- 10.20.193.78:0/1969041679 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920006d380 0x7f920013d8f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 1 --2- 10.20.193.78:0/1969041679 >> v2:10.20.193.115:3300/0 conn(0x7f920009c4a0 0x7f920013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 1 -- 10.20.193.78:0/1969041679 >> 10.20.193.78:0/1969041679 conn(0x7f92000a6620 msgr2=0x7f92000a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 1 -- 10.20.193.78:0/1969041679 shutdown_connections 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.438+0000 7f9206c6f640 1 -- 10.20.193.78:0/1969041679 wait complete. 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient: build_initial_monmap 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient: monmap: 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.447 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 1 Processor -- start 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 1 -- start start 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient: init 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f920006c890 0x7f920013c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient(hunting): picked mon.noname-a con 0x7f920006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f920006d380 0x7f920013cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient(hunting): picked mon.noname-b con 0x7f920006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920009c4a0 0x7f9200137380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient(hunting): picked mon.noname-c con 0x7f920009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f920009bfa0 con 0x7f920006c890 2026-02-17T05:53:35.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f920009be20 con 0x7f920006d380 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f920013de30 con 0x7f920009c4a0 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f9206c6f640 10 monclient(hunting): authenticate will time out at 1060.993879s 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f920006c890 0x7f920013c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920009c4a0 0x7f9200137380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f920006c890 0x7f920013c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35422/0 (socket says 10.20.193.78:35422) 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 1 -- 10.20.193.78:0/3169248410 learned_addr learned my addr 10.20.193.78:0/3169248410 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 10 monclient(hunting): get_auth_request con 0x7f920006c890 auth_method 0 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 10 monclient(hunting): get_auth_request con 0x7f920009c4a0 auth_method 0 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920009c4a0 msgr2=0x7f9200137380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920009c4a0 msgr2=0x7f9200137380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920009c4a0 0x7f9200137380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92051e5640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920009c4a0 0x7f9200137380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.439+0000 7f92049e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f91f7fff640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f920006d380 0x7f920013cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f91f7fff640 10 monclient(hunting): get_auth_request con 0x7f920006d380 auth_method 0 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f91f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f91f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f91f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 10 monclient(hunting): handle_auth_done global_id 104145 payload 824 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920009c4a0 msgr2=0x7f9200137380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f920009c4a0 0x7f9200137380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f920006d380 msgr2=0x7f920013cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f920006d380 0x7f920013cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 10 monclient: _finish_hunting 0 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 1 monclient: found mon.noname-a 2026-02-17T05:53:35.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.440+0000 7f92049e4640 1 -- 10.20.193.78:0/3169248410 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920013afd0 con 0x7f920006c890 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f92049e4640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f920006c890 msgr2=0x7f920013c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f92049e4640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f920006c890 0x7f920013c7e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c750 tx=0x7f91f000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0005de0 con 0x7f920006c890 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: dump: 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.451 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: _finish_auth 0 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: _check_auth_tickets 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f0007070 con 0x7f920006c890 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f9206c6f640 5 monclient: authenticate success, global_id 104145 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0007440 con 0x7f920006c890 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f9206c6f640 10 monclient: _renew_subs 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f6ffd640 4 set_mon_vals no callback set 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f9206c6f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920013b2b0 con 0x7f920006c890 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f91f57fa640 10 monclient: dump: 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.452 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f9206c6f640 10 monclient: _renew_subs 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f9206c6f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.442+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920013da60 con 0x7f920006c890 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f9206c6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f9206c6f640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92001cd170 0x7f92001378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f9206c6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92000a3ed0 con 0x7f92001cd170 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f91f0007660 con 0x7f920006c890 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92001cd170 0x7f92001378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 10 monclient: get_auth_request con 0x7f92001cd170 auth_method 0 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f57fa640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(283..283 src has 117..283) ==== 4843+0+0 (secure 0 0 0) 0x7f91f00526c0 con 0x7f920006c890 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f92051e5640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f92051e5640 10 monclient: get_auth_request con 0x7f91d003deb0 auth_method 0 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 10 monclient(hunting): handle_auth_done global_id 44596 payload 824 2026-02-17T05:53:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.443+0000 7f91f7fff640 1 -- 10.20.193.78:0/3169248410 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9200074980 con 0x7f92001cd170 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f7fff640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92001cd170 msgr2=0x7f92001378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f7fff640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92001cd170 0x7f92001378c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9200138bc0 tx=0x7f91e8010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e801a030 con 0x7f92001cd170 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 10 monclient: dump: 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.456 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f91e800eb80 con 0x7f92001cd170 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92001cd170 msgr2=0x7f92001378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92001cd170 0x7f92001378c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9200138bc0 tx=0x7f91e8010ca0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92001cd170 msgr2=0x7f92001378c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f91f57fa640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92001cd170 0x7f92001378c0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9200138bc0 tx=0x7f91e8010ca0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.446+0000 7f92051e5640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f920006a480 tx=0x7f91fc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f9206c6f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f9206c6f640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9200137e00 0x7f92001d8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f9206c6f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9200138210 con 0x7f9200137e00 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f91f7fff640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9200137e00 0x7f92001d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f91f7fff640 10 monclient: get_auth_request con 0x7f9200137e00 auth_method 0 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f91f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f91f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.449+0000 7f91f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.450+0000 7f91f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.450+0000 7f91f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.450+0000 7f91f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.450+0000 7f91f7fff640 10 monclient(hunting): handle_auth_done global_id 44599 payload 824 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.450+0000 7f91f7fff640 1 -- 10.20.193.78:0/3169248410 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f920006cc70 con 0x7f9200137e00 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f7fff640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9200137e00 msgr2=0x7f92001d8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f7fff640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9200137e00 0x7f92001d8270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9200138bc0 tx=0x7f91e80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e800f040 con 0x7f9200137e00 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 10 monclient: dump: 2026-02-17T05:53:35.458 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f91e8004230 con 0x7f9200137e00 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9200137e00 msgr2=0x7f92001d8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9200137e00 0x7f92001d8270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9200138bc0 tx=0x7f91e80047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9200137e00 msgr2=0x7f92001d8270 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.452+0000 7f91f57fa640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9200137e00 0x7f92001d8270 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9200138bc0 tx=0x7f91e80047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 82, 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 339100, 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:35.460 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.461 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:35.462 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:35.463 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.464 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.465 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.466 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:35.467 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:53:35.468 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:35.469 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:35.469 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:35.469 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:35.469 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 msgr2=0x7f91d0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f920006a480 tx=0x7f91fc033000 comp rx=0 tx=0).stop 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 10 monclient: shutdown 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f920006c890 msgr2=0x7f920013c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 1 --2- 10.20.193.78:0/3169248410 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f920006c890 0x7f920013c7e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c750 tx=0x7f91f000cc20 comp rx=0 tx=0).stop 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f92051e5640 1 -- 10.20.193.78:0/3169248410 reap_dead start 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f92051e5640 1 -- 10.20.193.78:0/3169248410 reap_dead start 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 shutdown_connections 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 >> 10.20.193.78:0/3169248410 conn(0x7f92000a6620 msgr2=0x7f92000a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 shutdown_connections 2026-02-17T05:53:35.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:35.453+0000 7f9206c6f640 1 -- 10.20.193.78:0/3169248410 wait complete. 2026-02-17T05:53:35.471 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:53:35.471 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T05:53:35.471 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2026-02-17T05:53:35.471 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T05:53:35.471 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T05:53:35.471 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T05:53:35.471 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:53:35.481 INFO:tasks.rados.rados.0.trial115.stdout:1327: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:53:35.483 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T05:53:35.483 INFO:tasks.mon_thrash:Sending STOP to mon c 2026-02-17T05:53:35.483 INFO:tasks.ceph.mon.c:Sent signal 19 2026-02-17T05:53:35.483 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T05:53:35.606 INFO:tasks.rados.rados.0.trial115.stdout:1331: expect (ObjNum 492 snap 127 seq_num 492) 2026-02-17T05:53:35.684 INFO:tasks.rados.rados.0.trial115.stdout:1327: done (4 left) 2026-02-17T05:53:35.684 INFO:tasks.rados.rados.0.trial115.stdout:1328: done (3 left) 2026-02-17T05:53:35.684 INFO:tasks.rados.rados.0.trial115.stdout:1329: done (2 left) 2026-02-17T05:53:35.684 INFO:tasks.rados.rados.0.trial115.stdout:1331: done (1 left) 2026-02-17T05:53:35.684 INFO:tasks.rados.rados.0.trial115.stdout:1333: done (0 left) 2026-02-17T05:53:35.684 INFO:tasks.rados.rados.0.trial115.stdout:1334: rollback oid 50 current snap is 130 2026-02-17T05:53:35.684 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 50 to 122 2026-02-17T05:53:35.685 INFO:tasks.rados.rados.0.trial115.stdout:1335: snap_remove snap 94 2026-02-17T05:53:50.484 INFO:tasks.mon_thrash:Sending CONT to mon c 2026-02-17T05:53:50.484 INFO:tasks.ceph.mon.c:Sent signal 18 2026-02-17T05:53:50.484 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:53:50.485 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:53:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.554+0000 7f8128c4d640 10 monclient: get_monmap_and_config 2026-02-17T05:53:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.554+0000 7f8128c4d640 10 monclient: build_initial_monmap 2026-02-17T05:53:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.554+0000 7f8128c4d640 10 monclient: monmap: 2026-02-17T05:53:50.556 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:50.556 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:50.556 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:50.555529+0000 2026-02-17T05:53:50.556 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:50.555529+0000 2026-02-17T05:53:50.556 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:50.556 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:50.557 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:50.557 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:50.557 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:50.557 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:50.557 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.554+0000 7f8128c4d640 1 Processor -- start 2026-02-17T05:53:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.554+0000 7f8128c4d640 1 -- start start 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.554+0000 7f8128c4d640 10 monclient: init 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.554+0000 7f8128c4d640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.554+0000 7f8128c4d640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 10 monclient(hunting): picked mon.noname-a con 0x7f812406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 10 monclient(hunting): picked mon.noname-b con 0x7f812409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 0x7f8124135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 10 monclient(hunting): picked mon.noname-c con 0x7f8124134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f81240a0880 con 0x7f812406c890 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f812406d6f0 con 0x7f8124134df0 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f812406daf0 con 0x7f812409c4a0 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8128c4d640 10 monclient(hunting): authenticate will time out at 1076.109038s 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122d76640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 msgr2=0x7f8124135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122d76640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 0x7f8124135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51782/0 (socket says 10.20.193.78:51782) 2026-02-17T05:53:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58498/0 (socket says 10.20.193.78:58498) 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 learned_addr learned my addr 10.20.193.78:0/2267153678 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 10 monclient(hunting): get_auth_request con 0x7f812409c4a0 auth_method 0 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 10 monclient(hunting): get_auth_request con 0x7f812406c890 auth_method 0 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:53:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.555+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122d76640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 msgr2=0x7f8124135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122d76640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 0x7f8124135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 10 monclient(hunting): get_auth_request con 0x7f812409c4a0 auth_method 0 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 10 monclient(hunting): get_auth_request con 0x7f812406c890 auth_method 0 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:50.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:50.756+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122d76640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 msgr2=0x7f8124135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122d76640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 0x7f8124135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 10 monclient(hunting): get_auth_request con 0x7f812409c4a0 auth_method 0 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 10 monclient(hunting): get_auth_request con 0x7f812406c890 auth_method 0 2026-02-17T05:53:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:51.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.156+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:53:51.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122d76640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 msgr2=0x7f8124135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:51.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122d76640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 0x7f8124135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 10 monclient(hunting): get_auth_request con 0x7f812409c4a0 auth_method 0 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 10 monclient(hunting): get_auth_request con 0x7f812406c890 auth_method 0 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:51.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:51.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:53:51.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:51.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:51.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:53:51.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:53:51.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:53:51.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:53:51.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:51.957+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:53:53.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 10 monclient(hunting): tick 2026-02-17T05:53:53.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:53:53.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 monclient(hunting): continuing hunt 2026-02-17T05:53:53.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:53:53.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 msgr2=0x7f812406d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:53.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f812409c4a0 0x7f812406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:53.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 msgr2=0x7f8124135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:53.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.115:3300/0 conn(0x7f8124134df0 0x7f8124135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:53.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 -- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 msgr2=0x7f812406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:53.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 --2- 10.20.193.78:0/2267153678 >> v2:10.20.193.78:3300/0 conn(0x7f812406c890 0x7f812406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:53.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 10 monclient: _add_conns ranks=[0,1] 2026-02-17T05:53:53.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8100003790 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:53.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f81000074c0 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:53.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 -- 10.20.193.78:0/2267153678 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f812409cc80 con 0x7f8100003790 2026-02-17T05:53:53.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 1 -- 10.20.193.78:0/2267153678 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f810000c200 con 0x7f81000074c0 2026-02-17T05:53:53.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f810bfff640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:53.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.555+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> v1:10.20.193.115:6789/0 conn(0x7f81000074c0 legacy=0x7f8100009980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:53:53.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.557+0000 7f8122575640 1 -- 10.20.193.78:0/2267153678 >> v1:10.20.193.78:6789/0 conn(0x7f8100003790 legacy=0x7f8100005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:53.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:53.756+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> v1:10.20.193.115:6789/0 conn(0x7f81000074c0 legacy=0x7f8100009980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:53:54.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:54.156+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> v1:10.20.193.115:6789/0 conn(0x7f81000074c0 legacy=0x7f8100009980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:53:54.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:54.957+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> v1:10.20.193.115:6789/0 conn(0x7f81000074c0 legacy=0x7f8100009980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:53:55.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.492+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2910608165 0 0) 0x7f812409cc80 con 0x7f8100003790 2026-02-17T05:53:55.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.492+0000 7f8121573640 10 monclient(hunting): _negotiate 2026-02-17T05:53:55.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.492+0000 7f8121573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.492+0000 7f8121573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.492+0000 7f8121573640 10 monclient(hunting): my global_id is 104148 2026-02-17T05:53:55.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.492+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8110003620 con 0x7f8100003790 2026-02-17T05:53:55.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2643153122 0 0) 0x7f8110003620 con 0x7f8100003790 2026-02-17T05:53:55.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 >> v1:10.20.193.115:6789/0 conn(0x7f81000074c0 legacy=0x7f8100009980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: _finish_hunting 0 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810000d460 con 0x7f8100003790 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: _finish_auth 0 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: _check_auth_tickets 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f811800f040 con 0x7f8100003790 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: dump: 2026-02-17T05:53:55.502 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 >> v1:10.20.193.78:6789/0 conn(0x7f8100003790 legacy=0x7f8100005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122d76640 1 -- 10.20.193.78:0/2267153678 reap_dead start 2026-02-17T05:53:55.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8110003ae0 0x7f8110003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient(hunting): picked mon.c con 0x7f8110003ae0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f8110004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient(hunting): picked mon.a con 0x7f8110004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81100054c0 0x7f811000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient(hunting): picked mon.b con 0x7f81100054c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f811000c320 con 0x7f8110004900 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f811000c4d0 con 0x7f81100054c0 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f811000c6d0 con 0x7f8110003ae0 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121573640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:53:55.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122575640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8110003ae0 0x7f8110003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f8110004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122575640 10 monclient(hunting): get_auth_request con 0x7f8110003ae0 auth_method 0 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient(hunting): get_auth_request con 0x7f8110004900 auth_method 0 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122d76640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81100054c0 msgr2=0x7f811000bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122d76640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81100054c0 0x7f811000bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:53:55.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8122575640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient(hunting): handle_auth_done global_id 104148 payload 293 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8110003ae0 msgr2=0x7f8110003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8110003ae0 0x7f8110003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81100054c0 msgr2=0x7f811000bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81100054c0 0x7f811000bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient: _finish_hunting 0 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 1 monclient: found mon.a 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8110003620 con 0x7f8110004900 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient: _finish_auth 0 2026-02-17T05:53:55.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 10 monclient: _check_auth_tickets 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8121d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8128c4d640 5 monclient: authenticate success, global_id 104148 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.493+0000 7f8128c4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121d74640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 msgr2=0x7f8110004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121d74640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f8110004d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f810c006fd0 tx=0x7f810c02fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e21c1075b8b0001 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c036070 con 0x7f8110004900 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 10 monclient: dump: 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.507 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8128c4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c030650 con 0x7f8110004900 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 1 -- 10.20.193.78:0/2267153678 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c003750 con 0x7f8110004900 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8121573640 10 monclient: dump: 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.508 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8128c4d640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.496+0000 7f8128c4d640 4 set_mon_vals no callback set 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 10 monclient: shutdown 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 -- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 msgr2=0x7f8110004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f8110004d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f810c006fd0 tx=0x7f810c02fe30 comp rx=0 tx=0).stop 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 -- 10.20.193.78:0/2267153678 shutdown_connections 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81100054c0 0x7f811000bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f8110004d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 --2- 10.20.193.78:0/2267153678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8110003ae0 0x7f8110003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 -- 10.20.193.78:0/2267153678 >> 10.20.193.78:0/2267153678 conn(0x7f81240a6620 msgr2=0x7f81240a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 -- 10.20.193.78:0/2267153678 shutdown_connections 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 1 -- 10.20.193.78:0/2267153678 wait complete. 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 10 monclient: build_initial_monmap 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.497+0000 7f8128c4d640 10 monclient: monmap: 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:55.510 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 1 Processor -- start 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 1 -- start start 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient: init 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8110003ae0 0x7f812409ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient(hunting): picked mon.noname-b con 0x7f8110003ae0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f812409e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient(hunting): picked mon.noname-a con 0x7f8110004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81100054c0 0x7f812409e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient(hunting): picked mon.noname-c con 0x7f81100054c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81240a01a0 con 0x7f8110004900 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f81240a0020 con 0x7f8110003ae0 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f812409be20 con 0x7f81100054c0 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8128c4d640 10 monclient(hunting): authenticate will time out at 1081.052322s 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f812409e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122575640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8110003ae0 msgr2=0x7f812409ddc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81100054c0 0x7f812409e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122575640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8110003ae0 0x7f812409ddc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f812409e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58550/0 (socket says 10.20.193.78:58550) 2026-02-17T05:53:55.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 -- 10.20.193.78:0/3907634209 learned_addr learned my addr 10.20.193.78:0/3907634209 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient(hunting): get_auth_request con 0x7f8110004900 auth_method 0 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 10 monclient(hunting): get_auth_request con 0x7f81100054c0 auth_method 0 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient(hunting): handle_auth_done global_id 104151 payload 824 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 -- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81100054c0 msgr2=0x7f812409e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81100054c0 0x7f812409e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 -- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8110003ae0 msgr2=0x7f812409ddc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8110003ae0 0x7f812409ddc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient: _finish_hunting 0 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 monclient: found mon.noname-a 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8121d74640 1 -- 10.20.193.78:0/3907634209 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8124125170 con 0x7f8110004900 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.498+0000 7f8122d76640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81100054c0 0x7f812409e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8121d74640 1 -- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 msgr2=0x7f812409e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8121d74640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f812409e300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f810c02fe70 tx=0x7f810c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 1 -- 10.20.193.78:0/3907634209 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c030e20 con 0x7f8110004900 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: dump: 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.516 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: _finish_auth 0 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: _check_auth_tickets 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 1 -- 10.20.193.78:0/3907634209 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c049070 con 0x7f8110004900 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8128c4d640 5 monclient: authenticate success, global_id 104151 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8128c4d640 10 monclient: _renew_subs 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8128c4d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8124136cd0 con 0x7f8110004900 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 1 -- 10.20.193.78:0/3907634209 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c002870 con 0x7f8110004900 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8120d72640 4 set_mon_vals no callback set 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f810affd640 10 monclient: dump: 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8128c4d640 10 monclient: _renew_subs 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8128c4d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.500+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812409d810 con 0x7f8110004900 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.501+0000 7f810affd640 1 -- 10.20.193.78:0/3907634209 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f810c002a10 con 0x7f8110004900 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.501+0000 7f8128c4d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.501+0000 7f8128c4d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.501+0000 7f8128c4d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.501+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8124074980 con 0x7f8110004900 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.501+0000 7f810affd640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f80f003de90 0x7f80f0040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.501+0000 7f810affd640 1 -- 10.20.193.78:0/3907634209 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(283..283 src has 117..283) ==== 4843+0+0 (secure 0 0 0) 0x7f810c0781e0 con 0x7f8110004900 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.501+0000 7f8122575640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f80f003de90 0x7f80f0040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.502+0000 7f8122575640 10 monclient: get_auth_request con 0x7f80f003de90 auth_method 0 2026-02-17T05:53:55.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.504+0000 7f810affd640 1 -- 10.20.193.78:0/3907634209 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f810c0037f0 con 0x7f8110004900 2026-02-17T05:53:55.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.504+0000 7f810affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:55.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.504+0000 7f810affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:55.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.504+0000 7f8122575640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f80f003de90 0x7f80f0040350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f812409f350 tx=0x7f8118008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.624+0000 7f8128c4d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:53:55.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.624+0000 7f8128c4d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:55.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.624+0000 7f8128c4d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.624+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8124135240 con 0x7f8110004900 2026-02-17T05:53:55.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.624+0000 7f810affd640 1 -- 10.20.193.78:0/3907634209 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f810c076ba0 con 0x7f8110004900 2026-02-17T05:53:55.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.624+0000 7f810affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:53:55.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.624+0000 7f810affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:55.625 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":84,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:53:55.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f80f003de90 msgr2=0x7f80f0040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f80f003de90 0x7f80f0040350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f812409f350 tx=0x7f8118008040 comp rx=0 tx=0).stop 2026-02-17T05:53:55.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 10 monclient: shutdown 2026-02-17T05:53:55.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:55.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 msgr2=0x7f812409e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f812409e300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f810c02fe70 tx=0x7f810c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 shutdown_connections 2026-02-17T05:53:55.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81100054c0 0x7f812409e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8110004900 0x7f812409e300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f80f003de90 0x7f80f0040350 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 --2- 10.20.193.78:0/3907634209 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8110003ae0 0x7f812409ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 >> 10.20.193.78:0/3907634209 conn(0x7f81240a6620 msgr2=0x7f8124135450 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:55.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 shutdown_connections 2026-02-17T05:53:55.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.625+0000 7f8128c4d640 1 -- 10.20.193.78:0/3907634209 wait complete. 2026-02-17T05:53:55.633 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:53:55.633 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T05:53:55.633 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.701+0000 7f4914d11640 10 monclient: get_monmap_and_config 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.701+0000 7f4914d11640 10 monclient: build_initial_monmap 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.701+0000 7f4914d11640 10 monclient: monmap: 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:55.703064+0000 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:55.703064+0000 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:55.701 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 1 Processor -- start 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 1 -- start start 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient: init 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f491006c890 0x7f49100a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient(hunting): picked mon.noname-a con 0x7f491006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f491006cc70 0x7f491006d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient(hunting): picked mon.noname-c con 0x7f491006cc70 addr v2:10.20.193.115:3300/0 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4910134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4910074150 con 0x7f4910134df0 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f491009ef70 con 0x7f491006c890 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f491009f250 con 0x7f491006cc70 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f4914d11640 10 monclient(hunting): authenticate will time out at 1081.256603s 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490dd74640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f491006cc70 msgr2=0x7f491006d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490dd74640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f491006cc70 0x7f491006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f491006c890 0x7f49100a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490ed76640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f4910134df0 0x7f4910135200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:48774/0 (socket says 10.20.193.78:48774) 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490ed76640 1 -- 10.20.193.78:0/2496400683 learned_addr learned my addr 10.20.193.78:0/2496400683 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient(hunting): get_auth_request con 0x7f491006c890 auth_method 0 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient(hunting): handle_auth_done global_id 104154 payload 274 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 1 -- 10.20.193.78:0/2496400683 >> v2:10.20.193.115:3300/0 conn(0x7f491006cc70 msgr2=0x7f491006d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 1 --2- 10.20.193.78:0/2496400683 >> v2:10.20.193.115:3300/0 conn(0x7f491006cc70 0x7f491006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 1 -- 10.20.193.78:0/2496400683 >> v1:10.20.193.78:6789/0 conn(0x7f4910134df0 legacy=0x7f4910135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient: _finish_hunting 0 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 1 monclient: found mon.noname-a 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.702+0000 7f490e575640 1 -- 10.20.193.78:0/2496400683 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491009df40 con 0x7f491006c890 2026-02-17T05:53:55.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490e575640 1 -- 10.20.193.78:0/2496400683 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 msgr2=0x7f49100a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490e575640 1 --2- 10.20.193.78:0/2496400683 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49100a01f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4910074e00 tx=0x7f490402ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fc5f4ccbbb0b22 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 1 -- 10.20.193.78:0/2496400683 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490403c070 con 0x7f491006c890 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: dump: 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: _finish_auth 0 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: _check_auth_tickets 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 5 monclient: authenticate success, global_id 104154 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 1 -- 10.20.193.78:0/2496400683 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490402fd70 con 0x7f491006c890 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 1 -- 10.20.193.78:0/2496400683 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904037660 con 0x7f491006c890 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 4 set_mon_vals no callback set 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f490d573640 10 monclient: dump: 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.705 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 10 monclient: shutdown 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 1 -- 10.20.193.78:0/2496400683 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 msgr2=0x7f49100a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 1 --2- 10.20.193.78:0/2496400683 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49100a01f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4910074e00 tx=0x7f490402ee10 comp rx=0 tx=0).stop 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 1 -- 10.20.193.78:0/2496400683 shutdown_connections 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 1 --2- 10.20.193.78:0/2496400683 >> v2:10.20.193.115:3300/0 conn(0x7f491006cc70 0x7f491006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 1 --2- 10.20.193.78:0/2496400683 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49100a01f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 1 -- 10.20.193.78:0/2496400683 >> 10.20.193.78:0/2496400683 conn(0x7f49100a6620 msgr2=0x7f49100a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:55.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 1 -- 10.20.193.78:0/2496400683 shutdown_connections 2026-02-17T05:53:55.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.705+0000 7f4914d11640 1 -- 10.20.193.78:0/2496400683 wait complete. 2026-02-17T05:53:55.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient: build_initial_monmap 2026-02-17T05:53:55.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient: monmap: 2026-02-17T05:53:55.711 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:55.711 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 1 Processor -- start 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 1 -- start start 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient: init 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49101cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient(hunting): picked mon.noname-a con 0x7f491006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f491006cc70 0x7f49101cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient(hunting): picked mon.noname-b con 0x7f491006cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4910134df0 0x7f4910122740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient(hunting): picked mon.noname-c con 0x7f4910134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f491009d360 con 0x7f491006c890 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f491009d1e0 con 0x7f491006cc70 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4910135910 con 0x7f4910134df0 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f4914d11640 10 monclient(hunting): authenticate will time out at 1081.260617s 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49101cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49101cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58556/0 (socket says 10.20.193.78:58556) 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 -- 10.20.193.78:0/3972979593 learned_addr learned my addr 10.20.193.78:0/3972979593 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:55.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient(hunting): get_auth_request con 0x7f491006c890 auth_method 0 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490dd74640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f491006cc70 msgr2=0x7f49101cd720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490dd74640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f491006cc70 0x7f49101cd720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490ed76640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4910134df0 0x7f4910122740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490ed76640 10 monclient(hunting): get_auth_request con 0x7f4910134df0 auth_method 0 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient(hunting): handle_auth_done global_id 104157 payload 824 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4910134df0 msgr2=0x7f4910122740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4910134df0 0x7f4910122740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f491006cc70 msgr2=0x7f49101cd720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f491006cc70 0x7f49101cd720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient: _finish_hunting 0 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 monclient: found mon.noname-a 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.706+0000 7f490e575640 1 -- 10.20.193.78:0/3972979593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49100730a0 con 0x7f491006c890 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.708+0000 7f490e575640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 msgr2=0x7f49101cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.708+0000 7f490e575640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49101cd1e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49040099a0 tx=0x7f49040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904046070 con 0x7f491006c890 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: dump: 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.716 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4904041890 con 0x7f491006c890 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 5 monclient: authenticate success, global_id 104157 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 10 monclient: _renew_subs 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4910126110 con 0x7f491006c890 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490403c040 con 0x7f491006c890 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 10 monclient: dump: 2026-02-17T05:53:55.717 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490cd72640 4 set_mon_vals no callback set 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 10 monclient: _renew_subs 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:53:55.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4910124880 con 0x7f491006c890 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123440 0x7f491011e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49100a3af0 con 0x7f4910123440 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123440 0x7f491011e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 10 monclient: get_auth_request con 0x7f4910123440 auth_method 0 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4904041cc0 con 0x7f491006c890 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 10 monclient(hunting): handle_auth_done global_id 104160 payload 824 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f490dd74640 1 -- 10.20.193.78:0/3972979593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49100a01d0 con 0x7f4910123440 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48dc03de60 0x7f48dc040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.709+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(283..283 src has 117..283) ==== 4843+0+0 (secure 0 0 0) 0x7f4904077cd0 con 0x7f491006c890 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.710+0000 7f490ed76640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48dc03de60 0x7f48dc040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.710+0000 7f490ed76640 10 monclient: get_auth_request con 0x7f48dc03de60 auth_method 0 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.711+0000 7f490dd74640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123440 msgr2=0x7f491011e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.711+0000 7f490dd74640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123440 0x7f491011e390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4910123fe0 tx=0x7f48f800bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.711+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80050f0 con 0x7f4910123440 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.711+0000 7f48f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.711+0000 7f48f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.711+0000 7f48f6ffd640 10 monclient: dump: 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.720 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.712+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f48f80050f0 con 0x7f4910123440 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.712+0000 7f48f6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.712+0000 7f48f6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.712+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123440 msgr2=0x7f491011e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.712+0000 7f48f6ffd640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123440 0x7f491011e390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4910123fe0 tx=0x7f48f800bcc0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.712+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123440 msgr2=0x7f491011e390 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:55.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.712+0000 7f48f6ffd640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123440 0x7f491011e390 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4910123fe0 tx=0x7f48f800bcc0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.712+0000 7f490ed76640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48dc03de60 0x7f48dc040320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f49000055f0 tx=0x7f4900005580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f4914d11640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:55.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f4914d11640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123820 0x7f49101d83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f4914d11640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f491011fa50 con 0x7f4910123820 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123820 0x7f49101d83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 10 monclient: get_auth_request con 0x7f4910123820 auth_method 0 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 10 monclient(hunting): handle_auth_done global_id 104163 payload 824 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.715+0000 7f490dd74640 1 -- 10.20.193.78:0/3972979593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f49100744d0 con 0x7f4910123820 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f490dd74640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123820 msgr2=0x7f49101d83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f490dd74640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123820 0x7f49101d83f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4910123fe0 tx=0x7f48f8005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f801a730 con 0x7f4910123820 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 10 monclient: dump: 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7f48f801a730 con 0x7f4910123820 2026-02-17T05:53:55.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:55.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:55.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123820 msgr2=0x7f49101d83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123820 0x7f49101d83f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4910123fe0 tx=0x7f48f8005b20 comp rx=0 tx=0).stop 2026-02-17T05:53:55.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123820 msgr2=0x7f49101d83f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:55.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f48f6ffd640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4910123820 0x7f49101d83f0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4910123fe0 tx=0x7f48f8005b20 comp rx=0 tx=0).stop 2026-02-17T05:53:55.726 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 84, 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 28869, 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:55.727 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:55.728 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:55.729 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:55.730 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.731 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:55.732 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:55.733 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.734 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.735 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:55.736 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:55.736 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:53:55.736 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.736 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.736 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.736 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:55.736 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:55.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48dc03de60 msgr2=0x7f48dc040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48dc03de60 0x7f48dc040320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f49000055f0 tx=0x7f4900005580 comp rx=0 tx=0).stop 2026-02-17T05:53:55.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 10 monclient: shutdown 2026-02-17T05:53:55.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:55.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 msgr2=0x7f49101cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49101cd1e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49040099a0 tx=0x7f49040047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f490ed76640 1 -- 10.20.193.78:0/3972979593 reap_dead start 2026-02-17T05:53:55.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 shutdown_connections 2026-02-17T05:53:55.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 --2- 10.20.193.78:0/3972979593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f491006c890 0x7f49101cd1e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 >> 10.20.193.78:0/3972979593 conn(0x7f49100a6620 msgr2=0x7f491011cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:55.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 shutdown_connections 2026-02-17T05:53:55.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.717+0000 7f4914d11640 1 -- 10.20.193.78:0/3972979593 wait complete. 2026-02-17T05:53:55.738 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:53:55.738 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:53:55.738 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:53:55.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.806+0000 7f666d464640 10 monclient: get_monmap_and_config 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.806+0000 7f666d464640 10 monclient: build_initial_monmap 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.806+0000 7f666d464640 10 monclient: monmap: 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:53:55.807745+0000 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:53:55.807745+0000 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.806 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:53:55.807 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:53:55.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:53:55.807 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:53:55.807 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:53:55.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.806+0000 7f666d464640 1 Processor -- start 2026-02-17T05:53:55.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.806+0000 7f666d464640 1 -- start start 2026-02-17T05:53:55.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.806+0000 7f666d464640 10 monclient: init 2026-02-17T05:53:55.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:55.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 10 monclient: _add_conns ranks=[1,2,0,4,3] 2026-02-17T05:53:55.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f666806c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:53:55.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666809c4a0 0x7f666813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 10 monclient(hunting): picked mon.noname-b con 0x7f666809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:55.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f666806d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:53:55.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666806daf0 con 0x7f666806d380 2026-02-17T05:53:55.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666809cc80 con 0x7f666806c890 2026-02-17T05:53:55.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f66680a0880 con 0x7f666809c4a0 2026-02-17T05:53:55.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f666d464640 10 monclient(hunting): authenticate will time out at 1081.361454s 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f6666ffd640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f666806c890 legacy=0x7f666806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666809c4a0 0x7f666813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66677fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f666806d380 0x7f666813d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:48784/0 (socket says 10.20.193.78:48784) 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66677fe640 1 -- 10.20.193.78:0/805858673 learned_addr learned my addr 10.20.193.78:0/805858673 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 1 --2- 10.20.193.78:0/805858673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666809c4a0 0x7f666813b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51862/0 (socket says 10.20.193.78:51862) 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient(hunting): get_auth_request con 0x7f666809c4a0 auth_method 0 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient(hunting): handle_auth_done global_id 74435 payload 274 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 1 -- 10.20.193.78:0/805858673 >> v1:10.20.193.115:6789/0 conn(0x7f666806c890 legacy=0x7f666806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 1 -- 10.20.193.78:0/805858673 >> v1:10.20.193.78:6789/0 conn(0x7f666806d380 legacy=0x7f666813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient: _finish_hunting 0 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 1 monclient: found mon.noname-b 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.807+0000 7f66667fc640 1 -- 10.20.193.78:0/805858673 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666809cf60 con 0x7f666809c4a0 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f66667fc640 1 -- 10.20.193.78:0/805858673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666809c4a0 msgr2=0x7f666813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f66667fc640 1 --2- 10.20.193.78:0/805858673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666809c4a0 0x7f666813b3a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f66500098d0 tx=0x7f665002edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb3f94e1a9ba773 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 1 -- 10.20.193.78:0/805858673 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665003c070 con 0x7f666809c4a0 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: dump: 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.818 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: _finish_auth 0 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 1 -- 10.20.193.78:0/805858673 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665002fd20 con 0x7f666809c4a0 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 5 monclient: authenticate success, global_id 74435 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 10 monclient: get_monmap_and_config success 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 1 -- 10.20.193.78:0/805858673 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650037660 con 0x7f666809c4a0 2026-02-17T05:53:55.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 4 set_mon_vals no callback set 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f6665ffb640 10 monclient: dump: 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.820 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 10 monclient: shutdown 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 1 -- 10.20.193.78:0/805858673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666809c4a0 msgr2=0x7f666813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 1 --2- 10.20.193.78:0/805858673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666809c4a0 0x7f666813b3a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f66500098d0 tx=0x7f665002edc0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 1 -- 10.20.193.78:0/805858673 shutdown_connections 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 1 --2- 10.20.193.78:0/805858673 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666809c4a0 0x7f666813b3a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 1 -- 10.20.193.78:0/805858673 >> 10.20.193.78:0/805858673 conn(0x7f66680a6620 msgr2=0x7f66680a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 1 -- 10.20.193.78:0/805858673 shutdown_connections 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.810+0000 7f666d464640 1 -- 10.20.193.78:0/805858673 wait complete. 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient: build_initial_monmap 2026-02-17T05:53:55.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient: monmap: 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:53:55.822 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 1 Processor -- start 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 1 -- start start 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient: init 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient: _reopen_session rank -1 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666806c890 0x7f6668132110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient(hunting): picked mon.noname-c con 0x7f666806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f666806d380 0x7f6668132650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient(hunting): picked mon.noname-b con 0x7f666806d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f666809c4a0 0x7f6668139050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient(hunting): picked mon.noname-a con 0x7f666809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f666809bfa0 con 0x7f666809c4a0 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f666809be20 con 0x7f666806d380 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f666813f010 con 0x7f666806c890 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient(hunting): _renew_subs 2026-02-17T05:53:55.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f666d464640 10 monclient(hunting): authenticate will time out at 1081.365788s 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666806c890 0x7f6668132110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666806c890 0x7f6668132110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51878/0 (socket says 10.20.193.78:51878) 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 -- 10.20.193.78:0/3221701359 learned_addr learned my addr 10.20.193.78:0/3221701359 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient(hunting): get_auth_request con 0x7f666806c890 auth_method 0 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66667fc640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f666806d380 msgr2=0x7f6668132650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:53:55.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66667fc640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f666806d380 0x7f6668132650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66677fe640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f666809c4a0 0x7f6668139050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66677fe640 10 monclient(hunting): get_auth_request con 0x7f666809c4a0 auth_method 0 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66677fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f66677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient(hunting): handle_auth_done global_id 74438 payload 824 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f666806d380 msgr2=0x7f6668132650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f666806d380 0x7f6668132650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f666809c4a0 msgr2=0x7f6668139050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f666809c4a0 0x7f6668139050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 monclient: found mon.noname-c 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:53:55.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.811+0000 7f6666ffd640 1 -- 10.20.193.78:0/3221701359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666813cd00 con 0x7f666806c890 2026-02-17T05:53:55.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.814+0000 7f6666ffd640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666806c890 msgr2=0x7f6668132110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.814+0000 7f6666ffd640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666806c890 0x7f6668132110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f665c00b780 tx=0x7f665c00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0052d0 con 0x7f666806c890 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: dump: 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.833 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: _finish_auth 0 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: _check_auth_tickets 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c010070 con 0x7f666806c890 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 5 monclient: authenticate success, global_id 74438 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 10 monclient: _renew_subs 2026-02-17T05:53:55.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666813e510 con 0x7f666806c890 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0056d0 con 0x7f666806c890 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66657fa640 4 set_mon_vals no callback set 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 10 monclient: dump: 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.835 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 10 monclient: _renew_subs 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f666813ca20 con 0x7f666806c890 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668139590 0x7f6668139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f666806fd50 con 0x7f6668139590 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668139590 0x7f6668139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 10 monclient: get_auth_request con 0x7f6668139590 auth_method 0 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f665c01d030 con 0x7f666806c890 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 10 monclient(hunting): handle_auth_done global_id 74441 payload 824 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.816+0000 7f66667fc640 1 -- 10.20.193.78:0/3221701359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f666806cc70 con 0x7f6668139590 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.817+0000 7f66477fe640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f663403de60 0x7f6634040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.817+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(283..283 src has 117..283) ==== 4843+0+0 (secure 0 0 0) 0x7f665c051da0 con 0x7f666806c890 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.817+0000 7f66677fe640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f663403de60 0x7f6634040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.817+0000 7f66677fe640 10 monclient: get_auth_request con 0x7f663403de60 auth_method 0 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66667fc640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668139590 msgr2=0x7f6668139970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66667fc640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668139590 0x7f6668139970 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6650039000 tx=0x7f66500047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650047070 con 0x7f6668139590 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 10 monclient: dump: 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6650037c90 con 0x7f6668139590 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668139590 msgr2=0x7f6668139970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668139590 0x7f6668139970 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6650039000 tx=0x7f66500047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668139590 msgr2=0x7f6668139970 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66477fe640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668139590 0x7f6668139970 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6650039000 tx=0x7f66500047c0 comp rx=0 tx=0).stop 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.819+0000 7f66677fe640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f663403de60 0x7f6634040320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f66580055f0 tx=0x7f6658005580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f666d464640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f666d464640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668133ff0 0x7f66681d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f666d464640 10 monclient(hunting): start opening mon connection 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f666813b5d0 con 0x7f6668133ff0 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668133ff0 0x7f66681d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 10 monclient: get_auth_request con 0x7f6668133ff0 auth_method 0 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 10 monclient(hunting): handle_auth_done global_id 74444 payload 824 2026-02-17T05:53:55.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.822+0000 7f66667fc640 1 -- 10.20.193.78:0/3221701359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6668132b90 con 0x7f6668133ff0 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66667fc640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668133ff0 msgr2=0x7f66681d81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66667fc640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668133ff0 0x7f66681d81e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6650009820 tx=0x7f6650008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665003c040 con 0x7f6668133ff0 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 10 monclient: dump: 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:53:55.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7f6650003e30 con 0x7f6668133ff0 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668133ff0 msgr2=0x7f66681d81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668133ff0 0x7f66681d81e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6650009820 tx=0x7f6650008030 comp rx=0 tx=0).stop 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668133ff0 msgr2=0x7f66681d81e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.824+0000 7f66477fe640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6668133ff0 0x7f66681d81e0 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6650009820 tx=0x7f6650008030 comp rx=0 tx=0).stop 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 84, 2026-02-17T05:53:55.841 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 207716, 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:55.842 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.843 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:53:55.844 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:55.845 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:55.846 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.847 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:53:55.848 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:55.849 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:53:55.850 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f663403de60 msgr2=0x7f6634040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f663403de60 0x7f6634040320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f66580055f0 tx=0x7f6658005580 comp rx=0 tx=0).stop 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 10 monclient: shutdown 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666806c890 msgr2=0x7f6668132110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 1 --2- 10.20.193.78:0/3221701359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f666806c890 0x7f6668132110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f665c00b780 tx=0x7f665c00bc50 comp rx=0 tx=0).stop 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f66677fe640 1 -- 10.20.193.78:0/3221701359 reap_dead start 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f66677fe640 1 -- 10.20.193.78:0/3221701359 reap_dead start 2026-02-17T05:53:55.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 shutdown_connections 2026-02-17T05:53:55.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 >> 10.20.193.78:0/3221701359 conn(0x7f66680a6620 msgr2=0x7f6668070150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:53:55.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 shutdown_connections 2026-02-17T05:53:55.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:53:55.825+0000 7f666d464640 1 -- 10.20.193.78:0/3221701359 wait complete. 2026-02-17T05:53:55.852 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:53:55.852 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:53:56.496 INFO:tasks.rados.rados.0.trial115.stdout:1335: done (1 left) 2026-02-17T05:53:56.496 INFO:tasks.rados.rados.0.trial115.stdout:1336: read oid 41 snap 117 2026-02-17T05:53:56.496 INFO:tasks.rados.rados.0.trial115.stdout:1336: expect deleted 2026-02-17T05:53:56.496 INFO:tasks.rados.rados.0.trial115.stdout:1337: write (excl) oid 24 current snap is 130 2026-02-17T05:53:56.496 INFO:tasks.rados.rados.0.trial115.stdout:1334: finishing rollback tid 0 to trial11545420-50 2026-02-17T05:53:56.496 INFO:tasks.rados.rados.0.trial115.stdout:1337: seq_num 511 ranges {588377=647890,1935828=635840} 2026-02-17T05:53:56.500 INFO:tasks.rados.rados.0.trial115.stdout:1337: writing trial11545420-24 from 588377 to 1236267 tid 1 2026-02-17T05:53:56.505 INFO:tasks.rados.rados.0.trial115.stdout:1337: writing trial11545420-24 from 1935828 to 2571668 tid 2 2026-02-17T05:53:56.506 INFO:tasks.rados.rados.0.trial115.stdout:1338: snap_create 2026-02-17T05:53:56.506 INFO:tasks.rados.rados.0.trial115.stdout:1334: finishing rollback tid 1 to trial11545420-50 2026-02-17T05:53:56.506 INFO:tasks.rados.rados.0.trial115.stdout:1334: finishing rollback tid 2 to trial11545420-50 2026-02-17T05:53:56.506 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 581 (ObjNum 469 snap 120 seq_num 469) dirty exists 2026-02-17T05:53:56.506 INFO:tasks.rados.rados.0.trial115.stdout:1337: finishing write tid 1 to trial11545420-24 2026-02-17T05:53:56.507 INFO:tasks.rados.rados.0.trial115.stdout:1337: finishing write tid 2 to trial11545420-24 2026-02-17T05:53:56.507 INFO:tasks.rados.rados.0.trial115.stdout:1337: finishing write tid 3 to trial11545420-24 2026-02-17T05:53:56.507 INFO:tasks.rados.rados.0.trial115.stdout:1337: finishing write tid 4 to trial11545420-24 2026-02-17T05:53:56.507 INFO:tasks.rados.rados.0.trial115.stdout:1337: oid 24 updating version 0 to 769 2026-02-17T05:53:56.507 INFO:tasks.rados.rados.0.trial115.stdout:1337: oid 24 version 769 is already newer than 768 2026-02-17T05:53:56.507 INFO:tasks.rados.rados.0.trial115.stdout:1337: oid 24 updating version 769 to 770 2026-02-17T05:53:56.508 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 770 (ObjNum 511 snap 130 seq_num 511) dirty exists 2026-02-17T05:53:56.508 INFO:tasks.rados.rados.0.trial115.stdout:1337: left oid 24 (ObjNum 511 snap 130 seq_num 511) 2026-02-17T05:53:56.855 INFO:tasks.rados.rados.0.trial115.stdout:1334: done (3 left) 2026-02-17T05:53:56.856 INFO:tasks.rados.rados.0.trial115.stdout:1336: done (2 left) 2026-02-17T05:53:56.856 INFO:tasks.rados.rados.0.trial115.stdout:1337: done (1 left) 2026-02-17T05:53:56.856 INFO:tasks.rados.rados.0.trial115.stdout:1338: done (0 left) 2026-02-17T05:53:56.856 INFO:tasks.rados.rados.0.trial115.stdout:1339: read oid 28 snap -1 2026-02-17T05:53:56.856 INFO:tasks.rados.rados.0.trial115.stdout:1339: expect (ObjNum 477 snap 124 seq_num 477) 2026-02-17T05:53:56.856 INFO:tasks.rados.rados.0.trial115.stdout:1340: read oid 34 snap -1 2026-02-17T05:53:56.856 INFO:tasks.rados.rados.0.trial115.stdout:1340: expect deleted 2026-02-17T05:53:56.856 INFO:tasks.rados.rados.0.trial115.stdout:1341: write (excl) oid 23 current snap is 131 2026-02-17T05:53:56.857 INFO:tasks.rados.rados.0.trial115.stdout:1341: seq_num 512 ranges {605184=723139,1877957=727227} 2026-02-17T05:53:56.860 INFO:tasks.rados.rados.0.trial115.stdout:1341: writing trial11545420-23 from 605184 to 1328323 tid 1 2026-02-17T05:53:56.865 INFO:tasks.rados.rados.0.trial115.stdout:1341: writing trial11545420-23 from 1877957 to 2605184 tid 2 2026-02-17T05:53:56.866 INFO:tasks.rados.rados.0.trial115.stdout:1342: read oid 21 snap -1 2026-02-17T05:53:56.866 INFO:tasks.rados.rados.0.trial115.stdout:1342: expect (ObjNum 469 snap 120 seq_num 469) 2026-02-17T05:53:56.866 INFO:tasks.rados.rados.0.trial115.stdout:1343: delete oid 6 current snap is 131 2026-02-17T05:53:56.866 INFO:tasks.rados.rados.0.trial115.stdout:1339: expect (ObjNum 477 snap 124 seq_num 477) 2026-02-17T05:53:56.913 INFO:tasks.rados.rados.0.trial115.stdout:1341: finishing write tid 1 to trial11545420-23 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1341: finishing write tid 2 to trial11545420-23 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1339: done (4 left) 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1340: done (3 left) 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1343: done (2 left) 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1344: snap_create 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1341: finishing write tid 3 to trial11545420-23 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1341: finishing write tid 4 to trial11545420-23 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1341: oid 23 updating version 0 to 257 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1341: oid 23 updating version 257 to 258 2026-02-17T05:53:56.914 INFO:tasks.rados.rados.0.trial115.stdout:1341: oid 23 version 258 is already newer than 256 2026-02-17T05:53:56.915 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 258 (ObjNum 512 snap 131 seq_num 512) dirty exists 2026-02-17T05:53:56.915 INFO:tasks.rados.rados.0.trial115.stdout:1341: left oid 23 (ObjNum 512 snap 131 seq_num 512) 2026-02-17T05:53:56.915 INFO:tasks.rados.rados.0.trial115.stdout:1342: expect (ObjNum 469 snap 120 seq_num 469) 2026-02-17T05:53:57.857 INFO:tasks.rados.rados.0.trial115.stdout:1341: done (2 left) 2026-02-17T05:53:57.858 INFO:tasks.rados.rados.0.trial115.stdout:1342: done (1 left) 2026-02-17T05:53:57.858 INFO:tasks.rados.rados.0.trial115.stdout:1344: done (0 left) 2026-02-17T05:53:57.858 INFO:tasks.rados.rados.0.trial115.stdout:1345: write (excl) oid 7 current snap is 132 2026-02-17T05:53:57.858 INFO:tasks.rados.rados.0.trial115.stdout:1345: seq_num 514 ranges {638798=473637,1594921=783167} 2026-02-17T05:53:57.861 INFO:tasks.rados.rados.0.trial115.stdout:1345: writing trial11545420-7 from 638798 to 1112435 tid 1 2026-02-17T05:53:57.866 INFO:tasks.rados.rados.0.trial115.stdout:1345: writing trial11545420-7 from 1594921 to 2378088 tid 2 2026-02-17T05:53:57.867 INFO:tasks.rados.rados.0.trial115.stdout:1346: read oid 45 snap -1 2026-02-17T05:53:57.867 INFO:tasks.rados.rados.0.trial115.stdout:1346: expect (ObjNum 507 snap 130 seq_num 507) 2026-02-17T05:53:57.867 INFO:tasks.rados.rados.0.trial115.stdout:1347: write oid 40 current snap is 132 2026-02-17T05:53:57.867 INFO:tasks.rados.rados.0.trial115.stdout:1347: seq_num 515 ranges {655605=548886,1653403=643178} 2026-02-17T05:53:57.871 INFO:tasks.rados.rados.0.trial115.stdout:1347: writing trial11545420-40 from 655605 to 1204491 tid 1 2026-02-17T05:53:57.875 INFO:tasks.rados.rados.0.trial115.stdout:1347: writing trial11545420-40 from 1653403 to 2296581 tid 2 2026-02-17T05:53:57.875 INFO:tasks.rados.rados.0.trial115.stdout:1348: write oid 5 current snap is 132 2026-02-17T05:53:57.875 INFO:tasks.rados.rados.0.trial115.stdout:1348: seq_num 516 ranges {672412=624135,1995532=676880} 2026-02-17T05:53:57.880 INFO:tasks.rados.rados.0.trial115.stdout:1348: writing trial11545420-5 from 672412 to 1296547 tid 1 2026-02-17T05:53:57.886 INFO:tasks.rados.rados.0.trial115.stdout:1348: writing trial11545420-5 from 1995532 to 2672412 tid 2 2026-02-17T05:53:57.886 INFO:tasks.rados.rados.0.trial115.stdout:1349: delete oid 48 current snap is 132 2026-02-17T05:53:57.886 INFO:tasks.rados.rados.0.trial115.stdout:1345: finishing write tid 1 to trial11545420-7 2026-02-17T05:53:57.886 INFO:tasks.rados.rados.0.trial115.stdout:1345: finishing write tid 2 to trial11545420-7 2026-02-17T05:53:57.886 INFO:tasks.rados.rados.0.trial115.stdout:1345: finishing write tid 3 to trial11545420-7 2026-02-17T05:53:57.887 INFO:tasks.rados.rados.0.trial115.stdout:1345: finishing write tid 4 to trial11545420-7 2026-02-17T05:53:57.887 INFO:tasks.rados.rados.0.trial115.stdout:1345: oid 7 updating version 0 to 490 2026-02-17T05:53:57.887 INFO:tasks.rados.rados.0.trial115.stdout:1345: oid 7 updating version 490 to 491 2026-02-17T05:53:57.887 INFO:tasks.rados.rados.0.trial115.stdout:1345: oid 7 updating version 491 to 492 2026-02-17T05:53:57.887 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 492 (ObjNum 514 snap 132 seq_num 514) dirty exists 2026-02-17T05:53:57.887 INFO:tasks.rados.rados.0.trial115.stdout:1345: left oid 7 (ObjNum 514 snap 132 seq_num 514) 2026-02-17T05:53:57.887 INFO:tasks.rados.rados.0.trial115.stdout:1346: expect (ObjNum 507 snap 130 seq_num 507) 2026-02-17T05:53:57.975 INFO:tasks.rados.rados.0.trial115.stdout:1347: finishing write tid 1 to trial11545420-40 2026-02-17T05:53:57.975 INFO:tasks.rados.rados.0.trial115.stdout:1347: finishing write tid 2 to trial11545420-40 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1347: finishing write tid 3 to trial11545420-40 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1345: done (4 left) 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1346: done (3 left) 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1349: done (2 left) 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1350: snap_create 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1347: finishing write tid 4 to trial11545420-40 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1347: oid 40 updating version 0 to 364 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1347: oid 40 updating version 364 to 365 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1347: oid 40 version 365 is already newer than 363 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 365 (ObjNum 515 snap 132 seq_num 515) dirty exists 2026-02-17T05:53:57.976 INFO:tasks.rados.rados.0.trial115.stdout:1347: left oid 40 (ObjNum 515 snap 132 seq_num 515) 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:1348: finishing write tid 1 to trial11545420-5 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:1348: finishing write tid 2 to trial11545420-5 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:1348: finishing write tid 3 to trial11545420-5 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:1348: finishing write tid 4 to trial11545420-5 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:1348: oid 5 updating version 0 to 707 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:1348: oid 5 updating version 707 to 708 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:1348: oid 5 updating version 708 to 709 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 709 (ObjNum 516 snap 132 seq_num 516) dirty exists 2026-02-17T05:53:57.977 INFO:tasks.rados.rados.0.trial115.stdout:1348: left oid 5 (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:53:58.860 INFO:tasks.rados.rados.0.trial115.stdout:1347: done (2 left) 2026-02-17T05:53:58.860 INFO:tasks.rados.rados.0.trial115.stdout:1348: done (1 left) 2026-02-17T05:53:58.860 INFO:tasks.rados.rados.0.trial115.stdout:1350: done (0 left) 2026-02-17T05:53:58.860 INFO:tasks.rados.rados.0.trial115.stdout:1351: copy_from oid 21 from oid 30 current snap is 133 2026-02-17T05:53:58.860 INFO:tasks.rados.rados.0.trial115.stdout:1352: snap_remove snap 117 2026-02-17T05:53:59.863 INFO:tasks.rados.rados.0.trial115.stdout:1352: done (1 left) 2026-02-17T05:53:59.863 INFO:tasks.rados.rados.0.trial115.stdout:1353: read oid 1 snap -1 2026-02-17T05:53:59.863 INFO:tasks.rados.rados.0.trial115.stdout:1353: expect (ObjNum 483 snap 126 seq_num 483) 2026-02-17T05:53:59.864 INFO:tasks.rados.rados.0.trial115.stdout:1354: rollback oid 37 current snap is 133 2026-02-17T05:53:59.864 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 122 2026-02-17T05:53:59.864 INFO:tasks.rados.rados.0.trial115.stdout:1351: finishing copy_from racing read to trial11545420-21 2026-02-17T05:53:59.864 INFO:tasks.rados.rados.0.trial115.stdout:1351: finishing copy_from to trial11545420-21 2026-02-17T05:53:59.864 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 525 (ObjNum 464 snap 117 seq_num 464) dirty exists 2026-02-17T05:53:59.864 INFO:tasks.rados.rados.0.trial115.stdout:1351: done (2 left) 2026-02-17T05:53:59.864 INFO:tasks.rados.rados.0.trial115.stdout:1355: write (excl) oid 25 current snap is 133 2026-02-17T05:53:59.864 INFO:tasks.rados.rados.0.trial115.stdout:1355: seq_num 518 ranges {706026=490986,1828849=790505} 2026-02-17T05:53:59.867 INFO:tasks.rados.rados.0.trial115.stdout:1355: writing trial11545420-25 from 706026 to 1197012 tid 1 2026-02-17T05:53:59.873 INFO:tasks.rados.rados.0.trial115.stdout:1355: writing trial11545420-25 from 1828849 to 2619354 tid 2 2026-02-17T05:53:59.873 INFO:tasks.rados.rados.0.trial115.stdout:1356: write (excl) oid 36 current snap is 133 2026-02-17T05:53:59.873 INFO:tasks.rados.rados.0.trial115.stdout:1356: seq_num 519 ranges {722833=566235,1887331=650516} 2026-02-17T05:53:59.878 INFO:tasks.rados.rados.0.trial115.stdout:1356: writing trial11545420-36 from 722833 to 1289068 tid 1 2026-02-17T05:53:59.882 INFO:tasks.rados.rados.0.trial115.stdout:1356: writing trial11545420-36 from 1887331 to 2537847 tid 2 2026-02-17T05:53:59.883 INFO:tasks.rados.rados.0.trial115.stdout:1357: snap_create 2026-02-17T05:53:59.883 INFO:tasks.rados.rados.0.trial115.stdout:1354: finishing rollback tid 0 to trial11545420-37 2026-02-17T05:53:59.883 INFO:tasks.rados.rados.0.trial115.stdout:1354: finishing rollback tid 1 to trial11545420-37 2026-02-17T05:53:59.883 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 495 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-17T05:53:59.883 INFO:tasks.rados.rados.0.trial115.stdout:1353: expect (ObjNum 483 snap 126 seq_num 483) 2026-02-17T05:53:59.963 INFO:tasks.rados.rados.0.trial115.stdout:1355: finishing write tid 1 to trial11545420-25 2026-02-17T05:53:59.963 INFO:tasks.rados.rados.0.trial115.stdout:1355: finishing write tid 2 to trial11545420-25 2026-02-17T05:53:59.963 INFO:tasks.rados.rados.0.trial115.stdout:1355: finishing write tid 3 to trial11545420-25 2026-02-17T05:53:59.963 INFO:tasks.rados.rados.0.trial115.stdout:1355: finishing write tid 4 to trial11545420-25 2026-02-17T05:53:59.963 INFO:tasks.rados.rados.0.trial115.stdout:1355: oid 25 updating version 0 to 598 2026-02-17T05:53:59.963 INFO:tasks.rados.rados.0.trial115.stdout:1355: oid 25 version 598 is already newer than 591 2026-02-17T05:53:59.963 INFO:tasks.rados.rados.0.trial115.stdout:1355: oid 25 updating version 598 to 599 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 599 (ObjNum 518 snap 133 seq_num 518) dirty exists 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1355: left oid 25 (ObjNum 518 snap 133 seq_num 518) 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1356: finishing write tid 1 to trial11545420-36 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1356: finishing write tid 2 to trial11545420-36 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1356: finishing write tid 3 to trial11545420-36 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1356: finishing write tid 4 to trial11545420-36 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1356: oid 36 updating version 0 to 549 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1356: oid 36 version 549 is already newer than 548 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1356: oid 36 version 549 is already newer than 547 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 549 (ObjNum 519 snap 133 seq_num 519) dirty exists 2026-02-17T05:53:59.964 INFO:tasks.rados.rados.0.trial115.stdout:1356: left oid 36 (ObjNum 519 snap 133 seq_num 519) 2026-02-17T05:54:00.866 INFO:tasks.rados.rados.0.trial115.stdout:1353: done (4 left) 2026-02-17T05:54:00.866 INFO:tasks.rados.rados.0.trial115.stdout:1354: done (3 left) 2026-02-17T05:54:00.866 INFO:tasks.rados.rados.0.trial115.stdout:1355: done (2 left) 2026-02-17T05:54:00.867 INFO:tasks.rados.rados.0.trial115.stdout:1356: done (1 left) 2026-02-17T05:54:00.867 INFO:tasks.rados.rados.0.trial115.stdout:1357: done (0 left) 2026-02-17T05:54:00.867 INFO:tasks.rados.rados.0.trial115.stdout:1358: copy_from oid 19 from oid 33 current snap is 134 2026-02-17T05:54:00.867 INFO:tasks.rados.rados.0.trial115.stdout:1359: read oid 4 snap -1 2026-02-17T05:54:00.867 INFO:tasks.rados.rados.0.trial115.stdout:1359: expect (ObjNum 505 snap 130 seq_num 505) 2026-02-17T05:54:00.867 INFO:tasks.rados.rados.0.trial115.stdout:1360: read oid 37 snap -1 2026-02-17T05:54:00.867 INFO:tasks.rados.rados.0.trial115.stdout:1360: expect deleted 2026-02-17T05:54:00.867 INFO:tasks.rados.rados.0.trial115.stdout:1361: snap_create 2026-02-17T05:54:00.872 INFO:tasks.rados.rados.0.trial115.stdout:1359: expect (ObjNum 505 snap 130 seq_num 505) 2026-02-17T05:54:00.988 INFO:tasks.rados.rados.0.trial115.stdout:1358: finishing copy_from racing read to trial11545420-19 2026-02-17T05:54:00.989 INFO:tasks.rados.rados.0.trial115.stdout:1358: finishing copy_from to trial11545420-19 2026-02-17T05:54:00.989 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 425 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:54:01.855 INFO:tasks.rados.rados.0.trial115.stdout:1358: done (3 left) 2026-02-17T05:54:01.856 INFO:tasks.rados.rados.0.trial115.stdout:1359: done (2 left) 2026-02-17T05:54:01.856 INFO:tasks.rados.rados.0.trial115.stdout:1360: done (1 left) 2026-02-17T05:54:01.856 INFO:tasks.rados.rados.0.trial115.stdout:1361: done (0 left) 2026-02-17T05:54:01.856 INFO:tasks.rados.rados.0.trial115.stdout:1362: copy_from oid 13 from oid 10 current snap is 135 2026-02-17T05:54:01.856 INFO:tasks.rados.rados.0.trial115.stdout:1363: write (excl) oid 35 current snap is 135 2026-02-17T05:54:01.856 INFO:tasks.rados.rados.0.trial115.stdout:1363: seq_num 520 ranges {739640=641484,1829460=794174} 2026-02-17T05:54:01.861 INFO:tasks.rados.rados.0.trial115.stdout:1363: writing trial11545420-35 from 739640 to 1381124 tid 1 2026-02-17T05:54:01.867 INFO:tasks.rados.rados.0.trial115.stdout:1363: writing trial11545420-35 from 1829460 to 2623634 tid 2 2026-02-17T05:54:01.867 INFO:tasks.rados.rados.0.trial115.stdout:1364: read oid 27 snap -1 2026-02-17T05:54:01.867 INFO:tasks.rados.rados.0.trial115.stdout:1364: expect (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:54:01.867 INFO:tasks.rados.rados.0.trial115.stdout:1365: delete oid 16 current snap is 135 2026-02-17T05:54:01.867 INFO:tasks.rados.rados.0.trial115.stdout:1362: finishing copy_from racing read to trial11545420-13 2026-02-17T05:54:01.867 INFO:tasks.rados.rados.0.trial115.stdout:1363: finishing write tid 1 to trial11545420-35 2026-02-17T05:54:01.867 INFO:tasks.rados.rados.0.trial115.stdout:1362: finishing copy_from to trial11545420-13 2026-02-17T05:54:01.867 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 530 (ObjNum 453 snap 112 seq_num 453) dirty exists 2026-02-17T05:54:01.872 INFO:tasks.rados.rados.0.trial115.stdout:1363: finishing write tid 2 to trial11545420-35 2026-02-17T05:54:01.872 INFO:tasks.rados.rados.0.trial115.stdout:1363: finishing write tid 3 to trial11545420-35 2026-02-17T05:54:01.872 INFO:tasks.rados.rados.0.trial115.stdout:1363: finishing write tid 4 to trial11545420-35 2026-02-17T05:54:01.872 INFO:tasks.rados.rados.0.trial115.stdout:1363: oid 35 updating version 0 to 686 2026-02-17T05:54:01.872 INFO:tasks.rados.rados.0.trial115.stdout:1363: oid 35 version 686 is already newer than 685 2026-02-17T05:54:01.872 INFO:tasks.rados.rados.0.trial115.stdout:1363: oid 35 updating version 686 to 687 2026-02-17T05:54:01.872 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 687 (ObjNum 520 snap 135 seq_num 520) dirty exists 2026-02-17T05:54:01.872 INFO:tasks.rados.rados.0.trial115.stdout:1363: left oid 35 (ObjNum 520 snap 135 seq_num 520) 2026-02-17T05:54:01.873 INFO:tasks.rados.rados.0.trial115.stdout:1362: done (3 left) 2026-02-17T05:54:01.873 INFO:tasks.rados.rados.0.trial115.stdout:1363: done (2 left) 2026-02-17T05:54:01.873 INFO:tasks.rados.rados.0.trial115.stdout:1365: done (1 left) 2026-02-17T05:54:01.873 INFO:tasks.rados.rados.0.trial115.stdout:1366: copy_from oid 48 from oid 9 current snap is 135 2026-02-17T05:54:01.873 INFO:tasks.rados.rados.0.trial115.stdout:1367: write (excl) oid 50 current snap is 135 2026-02-17T05:54:01.873 INFO:tasks.rados.rados.0.trial115.stdout:1367: seq_num 522 ranges {773254=791982,2346424=426830} 2026-02-17T05:54:01.877 INFO:tasks.rados.rados.0.trial115.stdout:1367: writing trial11545420-50 from 773254 to 1565236 tid 1 2026-02-17T05:54:01.883 INFO:tasks.rados.rados.0.trial115.stdout:1367: writing trial11545420-50 from 2346424 to 2773254 tid 2 2026-02-17T05:54:01.883 INFO:tasks.rados.rados.0.trial115.stdout:1368: read oid 25 snap -1 2026-02-17T05:54:01.883 INFO:tasks.rados.rados.0.trial115.stdout:1368: expect (ObjNum 518 snap 133 seq_num 518) 2026-02-17T05:54:01.883 INFO:tasks.rados.rados.0.trial115.stdout:1369: read oid 2 snap -1 2026-02-17T05:54:01.883 INFO:tasks.rados.rados.0.trial115.stdout:1369: expect deleted 2026-02-17T05:54:01.884 INFO:tasks.rados.rados.0.trial115.stdout:1370: rollback oid 19 current snap is 135 2026-02-17T05:54:01.884 INFO:tasks.rados.rados.0.trial115.stdout:1364: expect (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:54:02.037 INFO:tasks.rados.rados.0.trial115.stdout:1366: finishing copy_from racing read to trial11545420-48 2026-02-17T05:54:02.037 INFO:tasks.rados.rados.0.trial115.stdout:1366: finishing copy_from to trial11545420-48 2026-02-17T05:54:02.037 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 588 (ObjNum 377 snap 99 seq_num 377) dirty exists 2026-02-17T05:54:02.037 INFO:tasks.rados.rados.0.trial115.stdout:1367: finishing write tid 1 to trial11545420-50 2026-02-17T05:54:02.037 INFO:tasks.rados.rados.0.trial115.stdout:1367: finishing write tid 2 to trial11545420-50 2026-02-17T05:54:02.038 INFO:tasks.rados.rados.0.trial115.stdout:1367: finishing write tid 3 to trial11545420-50 2026-02-17T05:54:02.038 INFO:tasks.rados.rados.0.trial115.stdout:1367: finishing write tid 4 to trial11545420-50 2026-02-17T05:54:02.038 INFO:tasks.rados.rados.0.trial115.stdout:1367: oid 50 updating version 0 to 594 2026-02-17T05:54:02.038 INFO:tasks.rados.rados.0.trial115.stdout:1367: oid 50 version 594 is already newer than 593 2026-02-17T05:54:02.038 INFO:tasks.rados.rados.0.trial115.stdout:1367: oid 50 updating version 594 to 595 2026-02-17T05:54:02.038 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 595 (ObjNum 522 snap 135 seq_num 522) dirty exists 2026-02-17T05:54:02.038 INFO:tasks.rados.rados.0.trial115.stdout:1367: left oid 50 (ObjNum 522 snap 135 seq_num 522) 2026-02-17T05:54:02.038 INFO:tasks.rados.rados.0.trial115.stdout:1368: expect (ObjNum 518 snap 133 seq_num 518) 2026-02-17T05:54:02.140 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 19 to 133 2026-02-17T05:54:02.140 INFO:tasks.rados.rados.0.trial115.stdout:1364: done (5 left) 2026-02-17T05:54:02.140 INFO:tasks.rados.rados.0.trial115.stdout:1366: done (4 left) 2026-02-17T05:54:02.140 INFO:tasks.rados.rados.0.trial115.stdout:1367: done (3 left) 2026-02-17T05:54:02.140 INFO:tasks.rados.rados.0.trial115.stdout:1368: done (2 left) 2026-02-17T05:54:02.140 INFO:tasks.rados.rados.0.trial115.stdout:1369: done (1 left) 2026-02-17T05:54:02.140 INFO:tasks.rados.rados.0.trial115.stdout:1371: write oid 7 current snap is 135 2026-02-17T05:54:02.140 INFO:tasks.rados.rados.0.trial115.stdout:1371: seq_num 523 ranges {790061=467231,2004906=541501} 2026-02-17T05:54:02.144 INFO:tasks.rados.rados.0.trial115.stdout:1371: writing trial11545420-7 from 790061 to 1257292 tid 1 2026-02-17T05:54:02.149 INFO:tasks.rados.rados.0.trial115.stdout:1371: writing trial11545420-7 from 2004906 to 2546407 tid 2 2026-02-17T05:54:02.149 INFO:tasks.rados.rados.0.trial115.stdout:1372: read oid 33 snap -1 2026-02-17T05:54:02.149 INFO:tasks.rados.rados.0.trial115.stdout:1372: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:54:02.149 INFO:tasks.rados.rados.0.trial115.stdout:1373: write (excl) oid 37 current snap is 135 2026-02-17T05:54:02.150 INFO:tasks.rados.rados.0.trial115.stdout:1373: seq_num 524 ranges {406868=542480,1547035=401512,2690354=116514} 2026-02-17T05:54:02.153 INFO:tasks.rados.rados.0.trial115.stdout:1373: writing trial11545420-37 from 406868 to 949348 tid 1 2026-02-17T05:54:02.156 INFO:tasks.rados.rados.0.trial115.stdout:1373: writing trial11545420-37 from 1547035 to 1948547 tid 2 2026-02-17T05:54:02.160 INFO:tasks.rados.rados.0.trial115.stdout:1373: writing trial11545420-37 from 2690354 to 2806868 tid 3 2026-02-17T05:54:02.160 INFO:tasks.rados.rados.0.trial115.stdout:1374: snap_remove snap 128 2026-02-17T05:54:02.870 INFO:tasks.rados.rados.0.trial115.stdout:1374: done (4 left) 2026-02-17T05:54:02.870 INFO:tasks.rados.rados.0.trial115.stdout:1375: delete oid 2 current snap is 135 2026-02-17T05:54:02.870 INFO:tasks.rados.rados.0.trial115.stdout:1370: finishing rollback tid 0 to trial11545420-19 2026-02-17T05:54:02.870 INFO:tasks.rados.rados.0.trial115.stdout:1370: finishing rollback tid 1 to trial11545420-19 2026-02-17T05:54:02.870 INFO:tasks.rados.rados.0.trial115.stdout:1370: finishing rollback tid 2 to trial11545420-19 2026-02-17T05:54:02.870 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 694 (ObjNum 306 snap 73 seq_num 306) dirty exists 2026-02-17T05:54:02.870 INFO:tasks.rados.rados.0.trial115.stdout:1371: finishing write tid 1 to trial11545420-7 2026-02-17T05:54:02.870 INFO:tasks.rados.rados.0.trial115.stdout:1371: finishing write tid 2 to trial11545420-7 2026-02-17T05:54:02.871 INFO:tasks.rados.rados.0.trial115.stdout:1372: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:54:03.014 INFO:tasks.rados.rados.0.trial115.stdout:1371: finishing write tid 3 to trial11545420-7 2026-02-17T05:54:03.014 INFO:tasks.rados.rados.0.trial115.stdout:1371: finishing write tid 4 to trial11545420-7 2026-02-17T05:54:03.014 INFO:tasks.rados.rados.0.trial115.stdout:1371: oid 7 updating version 0 to 500 2026-02-17T05:54:03.014 INFO:tasks.rados.rados.0.trial115.stdout:1371: oid 7 updating version 500 to 501 2026-02-17T05:54:03.014 INFO:tasks.rados.rados.0.trial115.stdout:1371: oid 7 version 501 is already newer than 499 2026-02-17T05:54:03.014 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 501 (ObjNum 523 snap 135 seq_num 523) dirty exists 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1371: left oid 7 (ObjNum 523 snap 135 seq_num 523) 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1373: finishing write tid 1 to trial11545420-37 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1370: done (4 left) 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1371: done (3 left) 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1372: done (2 left) 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1375: done (1 left) 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1376: snap_create 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1373: finishing write tid 2 to trial11545420-37 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1373: finishing write tid 3 to trial11545420-37 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1373: finishing write tid 4 to trial11545420-37 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1373: finishing write tid 5 to trial11545420-37 2026-02-17T05:54:03.015 INFO:tasks.rados.rados.0.trial115.stdout:1373: oid 37 updating version 0 to 504 2026-02-17T05:54:03.016 INFO:tasks.rados.rados.0.trial115.stdout:1373: oid 37 version 504 is already newer than 503 2026-02-17T05:54:03.016 INFO:tasks.rados.rados.0.trial115.stdout:1373: oid 37 updating version 504 to 505 2026-02-17T05:54:03.016 INFO:tasks.rados.rados.0.trial115.stdout:1373: oid 37 version 505 is already newer than 502 2026-02-17T05:54:03.016 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 505 (ObjNum 524 snap 135 seq_num 524) dirty exists 2026-02-17T05:54:03.016 INFO:tasks.rados.rados.0.trial115.stdout:1373: left oid 37 (ObjNum 524 snap 135 seq_num 524) 2026-02-17T05:54:03.871 INFO:tasks.rados.rados.0.trial115.stdout:1373: done (1 left) 2026-02-17T05:54:03.871 INFO:tasks.rados.rados.0.trial115.stdout:1376: done (0 left) 2026-02-17T05:54:03.871 INFO:tasks.rados.rados.0.trial115.stdout:1377: write (excl) oid 38 current snap is 136 2026-02-17T05:54:03.872 INFO:tasks.rados.rados.0.trial115.stdout:1377: seq_num 526 ranges {440482=409331,1380352=405181,2461553=378929} 2026-02-17T05:54:03.875 INFO:tasks.rados.rados.0.trial115.stdout:1377: writing trial11545420-38 from 440482 to 849813 tid 1 2026-02-17T05:54:03.878 INFO:tasks.rados.rados.0.trial115.stdout:1377: writing trial11545420-38 from 1380352 to 1785533 tid 2 2026-02-17T05:54:03.883 INFO:tasks.rados.rados.0.trial115.stdout:1377: writing trial11545420-38 from 2461553 to 2840482 tid 3 2026-02-17T05:54:03.883 INFO:tasks.rados.rados.0.trial115.stdout:1378: snap_remove snap 134 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1378: done (1 left) 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1379: delete oid 13 current snap is 136 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1377: finishing write tid 1 to trial11545420-38 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1377: finishing write tid 2 to trial11545420-38 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1377: finishing write tid 3 to trial11545420-38 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1377: finishing write tid 4 to trial11545420-38 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1377: finishing write tid 5 to trial11545420-38 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1377: oid 38 updating version 0 to 706 2026-02-17T05:54:04.874 INFO:tasks.rados.rados.0.trial115.stdout:1377: oid 38 version 706 is already newer than 704 2026-02-17T05:54:04.875 INFO:tasks.rados.rados.0.trial115.stdout:1377: oid 38 updating version 706 to 707 2026-02-17T05:54:04.875 INFO:tasks.rados.rados.0.trial115.stdout:1377: oid 38 version 707 is already newer than 705 2026-02-17T05:54:04.875 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 707 (ObjNum 526 snap 136 seq_num 526) dirty exists 2026-02-17T05:54:04.875 INFO:tasks.rados.rados.0.trial115.stdout:1377: left oid 38 (ObjNum 526 snap 136 seq_num 526) 2026-02-17T05:54:04.875 INFO:tasks.rados.rados.0.trial115.stdout:1377: done (1 left) 2026-02-17T05:54:04.875 INFO:tasks.rados.rados.0.trial115.stdout:1379: done (0 left) 2026-02-17T05:54:04.876 INFO:tasks.rados.rados.0.trial115.stdout:1380: write (excl) oid 12 current snap is 136 2026-02-17T05:54:04.876 INFO:tasks.rados.rados.0.trial115.stdout:1380: seq_num 528 ranges {474096=559829,1780963=408850,2800046=74050} 2026-02-17T05:54:04.880 INFO:tasks.rados.rados.0.trial115.stdout:1380: writing trial11545420-12 from 474096 to 1033925 tid 1 2026-02-17T05:54:04.884 INFO:tasks.rados.rados.0.trial115.stdout:1380: writing trial11545420-12 from 1780963 to 2189813 tid 2 2026-02-17T05:54:04.887 INFO:tasks.rados.rados.0.trial115.stdout:1380: writing trial11545420-12 from 2800046 to 2874096 tid 3 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1381: copy_from oid 36 from oid 28 current snap is 136 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1380: finishing write tid 1 to trial11545420-12 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1380: finishing write tid 2 to trial11545420-12 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1382: read oid 11 snap -1 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1382: expect (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1383: rollback oid 27 current snap is 136 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 131 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1384: rollback oid 45 current snap is 136 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 133 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1385: rollback oid 10 current snap is 136 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 10 to 133 2026-02-17T05:54:04.888 INFO:tasks.rados.rados.0.trial115.stdout:1386: write oid 7 current snap is 136 2026-02-17T05:54:04.889 INFO:tasks.rados.rados.0.trial115.stdout:1386: seq_num 529 ranges {490903=635078,1839445=552508} 2026-02-17T05:54:04.891 INFO:tasks.rados.rados.0.trial115.stdout:1386: writing trial11545420-7 from 490903 to 1125981 tid 1 2026-02-17T05:54:04.896 INFO:tasks.rados.rados.0.trial115.stdout:1386: writing trial11545420-7 from 1839445 to 2391953 tid 2 2026-02-17T05:54:04.896 INFO:tasks.rados.rados.0.trial115.stdout:1387: read oid 15 snap 132 2026-02-17T05:54:04.896 INFO:tasks.rados.rados.0.trial115.stdout:1387: expect (ObjNum 450 snap 112 seq_num 450) 2026-02-17T05:54:04.896 INFO:tasks.rados.rados.0.trial115.stdout:1388: delete oid 1 current snap is 136 2026-02-17T05:54:04.896 INFO:tasks.rados.rados.0.trial115.stdout:1382: expect (ObjNum 303 snap 72 seq_num 303) 2026-02-17T05:54:04.969 INFO:tasks.rados.rados.0.trial115.stdout:1380: finishing write tid 3 to trial11545420-12 2026-02-17T05:54:04.969 INFO:tasks.rados.rados.0.trial115.stdout:1380: finishing write tid 4 to trial11545420-12 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1382: done (8 left) 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1388: done (7 left) 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1389: delete oid 32 current snap is 136 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1380: finishing write tid 5 to trial11545420-12 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1380: oid 12 updating version 0 to 779 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1380: oid 12 updating version 779 to 781 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1380: oid 12 version 781 is already newer than 778 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1380: oid 12 version 781 is already newer than 780 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 781 (ObjNum 528 snap 136 seq_num 528) dirty exists 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1380: left oid 12 (ObjNum 528 snap 136 seq_num 528) 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1384: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:54:04.970 INFO:tasks.rados.rados.0.trial115.stdout:1384: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1385: finishing rollback tid 0 to trial11545420-10 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1385: finishing rollback tid 1 to trial11545420-10 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1384: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 785 (ObjNum 507 snap 130 seq_num 507) dirty exists 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1385: finishing rollback tid 2 to trial11545420-10 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 662 (ObjNum 453 snap 112 seq_num 453) dirty exists 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1383: finishing rollback tid 0 to trial11545420-27 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1383: finishing rollback tid 1 to trial11545420-27 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1383: finishing rollback tid 2 to trial11545420-27 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 603 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1386: finishing write tid 1 to trial11545420-7 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1381: finishing copy_from racing read to trial11545420-36 2026-02-17T05:54:04.971 INFO:tasks.rados.rados.0.trial115.stdout:1381: finishing copy_from to trial11545420-36 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 602 (ObjNum 477 snap 124 seq_num 477) dirty exists 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:1386: finishing write tid 2 to trial11545420-7 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:1386: finishing write tid 3 to trial11545420-7 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:1386: finishing write tid 4 to trial11545420-7 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:1386: oid 7 updating version 0 to 511 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:1386: oid 7 updating version 511 to 512 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:1386: oid 7 updating version 512 to 513 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 513 (ObjNum 529 snap 136 seq_num 529) dirty exists 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:1386: left oid 7 (ObjNum 529 snap 136 seq_num 529) 2026-02-17T05:54:04.972 INFO:tasks.rados.rados.0.trial115.stdout:1387: expect (ObjNum 450 snap 112 seq_num 450) 2026-02-17T05:54:05.084 INFO:tasks.rados.rados.0.trial115.stdout:1380: done (7 left) 2026-02-17T05:54:05.084 INFO:tasks.rados.rados.0.trial115.stdout:1381: done (6 left) 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1383: done (5 left) 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1384: done (4 left) 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1385: done (3 left) 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1386: done (2 left) 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1387: done (1 left) 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1389: done (0 left) 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1390: rollback oid 36 current snap is 136 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 36 to 126 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1391: copy_from oid 4 from oid 18 current snap is 136 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1392: copy_from oid 11 from oid 19 current snap is 136 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1393: write oid 23 current snap is 136 2026-02-17T05:54:05.085 INFO:tasks.rados.rados.0.trial115.stdout:1393: seq_num 532 ranges {541324=460825,1498538=416188,2393385=547939} 2026-02-17T05:54:05.087 INFO:tasks.rados.rados.0.trial115.stdout:1393: writing trial11545420-23 from 541324 to 1002149 tid 1 2026-02-17T05:54:05.091 INFO:tasks.rados.rados.0.trial115.stdout:1393: writing trial11545420-23 from 1498538 to 1914726 tid 2 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:1393: writing trial11545420-23 from 2393385 to 2941324 tid 3 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:1394: delete oid 3 current snap is 136 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:1390: finishing rollback tid 0 to trial11545420-36 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:1390: finishing rollback tid 1 to trial11545420-36 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 605 (ObjNum 641 snap 0 seq_num 1715449712) dirty dne 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:1391: finishing copy_from racing read to trial11545420-4 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:1391: finishing copy_from to trial11545420-4 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 570 (ObjNum 500 snap 130 seq_num 500) dirty exists 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:1392: finishing copy_from racing read to trial11545420-11 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:1392: finishing copy_from to trial11545420-11 2026-02-17T05:54:05.095 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 694 (ObjNum 306 snap 73 seq_num 306) dirty exists 2026-02-17T05:54:05.096 INFO:tasks.rados.rados.0.trial115.stdout:1390: done (4 left) 2026-02-17T05:54:05.096 INFO:tasks.rados.rados.0.trial115.stdout:1391: done (3 left) 2026-02-17T05:54:05.097 INFO:tasks.rados.rados.0.trial115.stdout:1392: done (2 left) 2026-02-17T05:54:05.097 INFO:tasks.rados.rados.0.trial115.stdout:1394: done (1 left) 2026-02-17T05:54:05.097 INFO:tasks.rados.rados.0.trial115.stdout:1395: read oid 30 snap -1 2026-02-17T05:54:05.097 INFO:tasks.rados.rados.0.trial115.stdout:1395: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:54:05.097 INFO:tasks.rados.rados.0.trial115.stdout:1396: snap_remove snap 131 2026-02-17T05:54:05.877 INFO:tasks.rados.rados.0.trial115.stdout:1396: done (2 left) 2026-02-17T05:54:05.877 INFO:tasks.rados.rados.0.trial115.stdout:1397: write (excl) oid 35 current snap is 136 2026-02-17T05:54:05.877 INFO:tasks.rados.rados.0.trial115.stdout:1397: seq_num 534 ranges {574938=727676,1731855=703504,2848231=126707} 2026-02-17T05:54:05.883 INFO:tasks.rados.rados.0.trial115.stdout:1397: writing trial11545420-35 from 574938 to 1302614 tid 1 2026-02-17T05:54:05.887 INFO:tasks.rados.rados.0.trial115.stdout:1397: writing trial11545420-35 from 1731855 to 2435359 tid 2 2026-02-17T05:54:05.890 INFO:tasks.rados.rados.0.trial115.stdout:1397: writing trial11545420-35 from 2848231 to 2974938 tid 3 2026-02-17T05:54:05.890 INFO:tasks.rados.rados.0.trial115.stdout:1398: rollback oid 14 current snap is 136 2026-02-17T05:54:05.890 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 130 2026-02-17T05:54:05.890 INFO:tasks.rados.rados.0.trial115.stdout:1393: finishing write tid 1 to trial11545420-23 2026-02-17T05:54:05.890 INFO:tasks.rados.rados.0.trial115.stdout:1399: rollback oid 4 current snap is 136 2026-02-17T05:54:05.890 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 135 2026-02-17T05:54:05.890 INFO:tasks.rados.rados.0.trial115.stdout:1395: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1393: finishing write tid 2 to trial11545420-23 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1393: finishing write tid 3 to trial11545420-23 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1395: done (4 left) 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1400: read oid 37 snap 132 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1400: expect (ObjNum 498 snap 130 seq_num 498) 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1401: read oid 41 snap -1 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1401: expect deleted 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1402: read oid 42 snap -1 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1402: expect (ObjNum 438 snap 110 seq_num 438) 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1403: read oid 31 snap 122 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1403: expect (ObjNum 470 snap 121 seq_num 470) 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1404: read oid 2 snap 126 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1404: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1405: write (excl) oid 32 current snap is 136 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1393: finishing write tid 4 to trial11545420-23 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1393: finishing write tid 5 to trial11545420-23 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1393: oid 23 updating version 0 to 265 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1393: oid 23 version 265 is already newer than 262 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1393: oid 23 version 265 is already newer than 263 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:1393: oid 23 version 265 is already newer than 264 2026-02-17T05:54:06.015 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 265 (ObjNum 532 snap 136 seq_num 532) dirty exists 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1393: left oid 23 (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: finishing write tid 1 to trial11545420-35 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: finishing write tid 2 to trial11545420-35 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1398: finishing rollback tid 1 to trial11545420-14 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 533 (ObjNum 81 snap 0 seq_num 2) dirty dne 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1399: finishing rollback tid 0 to trial11545420-4 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: finishing write tid 3 to trial11545420-35 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1399: finishing rollback tid 1 to trial11545420-4 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1399: finishing rollback tid 2 to trial11545420-4 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 573 (ObjNum 505 snap 130 seq_num 505) dirty exists 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: finishing write tid 4 to trial11545420-35 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: finishing write tid 5 to trial11545420-35 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: oid 35 updating version 0 to 729 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: oid 35 version 729 is already newer than 728 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: oid 35 updating version 729 to 730 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: oid 35 version 730 is already newer than 717 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 730 (ObjNum 534 snap 136 seq_num 534) dirty exists 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1397: left oid 35 (ObjNum 534 snap 136 seq_num 534) 2026-02-17T05:54:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1405: seq_num 535 ranges {591745=402925,1790337=563515} 2026-02-17T05:54:06.020 INFO:tasks.rados.rados.0.trial115.stdout:1405: writing trial11545420-32 from 591745 to 994670 tid 1 2026-02-17T05:54:06.025 INFO:tasks.rados.rados.0.trial115.stdout:1405: writing trial11545420-32 from 1790337 to 2353852 tid 2 2026-02-17T05:54:06.025 INFO:tasks.rados.rados.0.trial115.stdout:1393: done (9 left) 2026-02-17T05:54:06.025 INFO:tasks.rados.rados.0.trial115.stdout:1397: done (8 left) 2026-02-17T05:54:06.025 INFO:tasks.rados.rados.0.trial115.stdout:1398: done (7 left) 2026-02-17T05:54:06.025 INFO:tasks.rados.rados.0.trial115.stdout:1399: done (6 left) 2026-02-17T05:54:06.025 INFO:tasks.rados.rados.0.trial115.stdout:1406: snap_remove snap 130 2026-02-17T05:54:06.857 INFO:tasks.rados.rados.0.trial115.stdout:1406: done (6 left) 2026-02-17T05:54:06.857 INFO:tasks.rados.rados.0.trial115.stdout:1407: read oid 23 snap -1 2026-02-17T05:54:06.858 INFO:tasks.rados.rados.0.trial115.stdout:1407: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:06.858 INFO:tasks.rados.rados.0.trial115.stdout:1408: write oid 20 current snap is 136 2026-02-17T05:54:06.858 INFO:tasks.rados.rados.0.trial115.stdout:1408: seq_num 536 ranges {608552=478174,1848819=707173} 2026-02-17T05:54:06.861 INFO:tasks.rados.rados.0.trial115.stdout:1408: writing trial11545420-20 from 608552 to 1086726 tid 1 2026-02-17T05:54:06.867 INFO:tasks.rados.rados.0.trial115.stdout:1408: writing trial11545420-20 from 1848819 to 2555992 tid 2 2026-02-17T05:54:06.867 INFO:tasks.rados.rados.0.trial115.stdout:1409: snap_remove snap 135 2026-02-17T05:54:07.857 INFO:tasks.rados.rados.0.trial115.stdout:1409: done (8 left) 2026-02-17T05:54:07.857 INFO:tasks.rados.rados.0.trial115.stdout:1410: read oid 33 snap -1 2026-02-17T05:54:07.857 INFO:tasks.rados.rados.0.trial115.stdout:1410: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:54:07.857 INFO:tasks.rados.rados.0.trial115.stdout:1411: rollback oid 6 current snap is 136 2026-02-17T05:54:07.858 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 6 to 126 2026-02-17T05:54:07.858 INFO:tasks.rados.rados.0.trial115.stdout:1402: expect (ObjNum 438 snap 110 seq_num 438) 2026-02-17T05:54:08.083 INFO:tasks.rados.rados.0.trial115.stdout:1400: expect (ObjNum 498 snap 130 seq_num 498) 2026-02-17T05:54:08.139 INFO:tasks.rados.rados.0.trial115.stdout:1403: expect (ObjNum 470 snap 121 seq_num 470) 2026-02-17T05:54:08.329 INFO:tasks.rados.rados.0.trial115.stdout:1405: finishing write tid 1 to trial11545420-32 2026-02-17T05:54:08.329 INFO:tasks.rados.rados.0.trial115.stdout:1404: expect (ObjNum 248 snap 49 seq_num 248) 2026-02-17T05:54:08.380 INFO:tasks.rados.rados.0.trial115.stdout:1405: finishing write tid 2 to trial11545420-32 2026-02-17T05:54:08.380 INFO:tasks.rados.rados.0.trial115.stdout:1405: finishing write tid 3 to trial11545420-32 2026-02-17T05:54:08.380 INFO:tasks.rados.rados.0.trial115.stdout:1405: finishing write tid 4 to trial11545420-32 2026-02-17T05:54:08.380 INFO:tasks.rados.rados.0.trial115.stdout:1405: oid 32 updating version 0 to 327 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1405: oid 32 updating version 327 to 329 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1405: oid 32 version 329 is already newer than 328 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 329 (ObjNum 535 snap 136 seq_num 535) dirty exists 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1405: left oid 32 (ObjNum 535 snap 136 seq_num 535) 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1408: finishing write tid 1 to trial11545420-20 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1400: done (9 left) 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1401: done (8 left) 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1402: done (7 left) 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1403: done (6 left) 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1404: done (5 left) 2026-02-17T05:54:08.381 INFO:tasks.rados.rados.0.trial115.stdout:1405: done (4 left) 2026-02-17T05:54:08.382 INFO:tasks.rados.rados.0.trial115.stdout:1412: snap_create 2026-02-17T05:54:08.382 INFO:tasks.rados.rados.0.trial115.stdout:1407: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:08.466 INFO:tasks.rados.rados.0.trial115.stdout:1408: finishing write tid 2 to trial11545420-20 2026-02-17T05:54:08.466 INFO:tasks.rados.rados.0.trial115.stdout:1408: finishing write tid 3 to trial11545420-20 2026-02-17T05:54:08.466 INFO:tasks.rados.rados.0.trial115.stdout:1408: finishing write tid 4 to trial11545420-20 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:1408: oid 20 updating version 0 to 522 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:1408: oid 20 updating version 522 to 523 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:1408: oid 20 updating version 523 to 524 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 524 (ObjNum 536 snap 136 seq_num 536) dirty exists 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:1408: left oid 20 (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:1411: finishing rollback tid 1 to trial11545420-6 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:1411: finishing rollback tid 2 to trial11545420-6 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 434 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:54:08.467 INFO:tasks.rados.rados.0.trial115.stdout:1410: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:54:08.857 INFO:tasks.rados.rados.0.trial115.stdout:1407: done (4 left) 2026-02-17T05:54:08.858 INFO:tasks.rados.rados.0.trial115.stdout:1408: done (3 left) 2026-02-17T05:54:08.858 INFO:tasks.rados.rados.0.trial115.stdout:1410: done (2 left) 2026-02-17T05:54:08.858 INFO:tasks.rados.rados.0.trial115.stdout:1411: done (1 left) 2026-02-17T05:54:08.858 INFO:tasks.rados.rados.0.trial115.stdout:1412: done (0 left) 2026-02-17T05:54:08.858 INFO:tasks.rados.rados.0.trial115.stdout:1413: write (excl) oid 29 current snap is 137 2026-02-17T05:54:08.858 INFO:tasks.rados.rados.0.trial115.stdout:1413: seq_num 537 ranges {625359=553423,1790948=567184,2930500=94859} 2026-02-17T05:54:08.862 INFO:tasks.rados.rados.0.trial115.stdout:1413: writing trial11545420-29 from 625359 to 1178782 tid 1 2026-02-17T05:54:08.867 INFO:tasks.rados.rados.0.trial115.stdout:1413: writing trial11545420-29 from 1790948 to 2358132 tid 2 2026-02-17T05:54:08.869 INFO:tasks.rados.rados.0.trial115.stdout:1413: writing trial11545420-29 from 2930500 to 3025359 tid 3 2026-02-17T05:54:08.870 INFO:tasks.rados.rados.0.trial115.stdout:1414: read oid 44 snap -1 2026-02-17T05:54:08.870 INFO:tasks.rados.rados.0.trial115.stdout:1414: expect deleted 2026-02-17T05:54:08.870 INFO:tasks.rados.rados.0.trial115.stdout:1415: snap_remove snap 133 2026-02-17T05:54:08.870 INFO:tasks.rados.rados.0.trial115.stdout:1413: finishing write tid 1 to trial11545420-29 2026-02-17T05:54:08.870 INFO:tasks.rados.rados.0.trial115.stdout:1413: finishing write tid 2 to trial11545420-29 2026-02-17T05:54:09.860 INFO:tasks.rados.rados.0.trial115.stdout:1415: done (2 left) 2026-02-17T05:54:09.861 INFO:tasks.rados.rados.0.trial115.stdout:1416: write oid 19 current snap is 137 2026-02-17T05:54:09.861 INFO:tasks.rados.rados.0.trial115.stdout:1416: seq_num 538 ranges {642166=628672,1849430=710842} 2026-02-17T05:54:09.866 INFO:tasks.rados.rados.0.trial115.stdout:1416: writing trial11545420-19 from 642166 to 1270838 tid 1 2026-02-17T05:54:09.871 INFO:tasks.rados.rados.0.trial115.stdout:1416: writing trial11545420-19 from 1849430 to 2560272 tid 2 2026-02-17T05:54:09.871 INFO:tasks.rados.rados.0.trial115.stdout:1417: rollback oid 5 current snap is 137 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 5 to 132 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1413: finishing write tid 3 to trial11545420-29 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1413: finishing write tid 4 to trial11545420-29 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1413: finishing write tid 5 to trial11545420-29 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1413: oid 29 updating version 0 to 537 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1413: oid 29 updating version 537 to 538 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1413: oid 29 updating version 538 to 539 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1413: oid 29 version 539 is already newer than 536 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 539 (ObjNum 537 snap 137 seq_num 537) dirty exists 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1413: left oid 29 (ObjNum 537 snap 137 seq_num 537) 2026-02-17T05:54:09.872 INFO:tasks.rados.rados.0.trial115.stdout:1416: finishing write tid 1 to trial11545420-19 2026-02-17T05:54:09.873 INFO:tasks.rados.rados.0.trial115.stdout:1413: done (3 left) 2026-02-17T05:54:09.873 INFO:tasks.rados.rados.0.trial115.stdout:1414: done (2 left) 2026-02-17T05:54:09.873 INFO:tasks.rados.rados.0.trial115.stdout:1418: write (excl) oid 33 current snap is 137 2026-02-17T05:54:09.873 INFO:tasks.rados.rados.0.trial115.stdout:1418: seq_num 539 ranges {658973=703921,1907912=570853,2985346=73627} 2026-02-17T05:54:09.876 INFO:tasks.rados.rados.0.trial115.stdout:1418: writing trial11545420-33 from 658973 to 1362894 tid 1 2026-02-17T05:54:09.881 INFO:tasks.rados.rados.0.trial115.stdout:1418: writing trial11545420-33 from 1907912 to 2478765 tid 2 2026-02-17T05:54:09.884 INFO:tasks.rados.rados.0.trial115.stdout:1418: writing trial11545420-33 from 2985346 to 3058973 tid 3 2026-02-17T05:54:09.884 INFO:tasks.rados.rados.0.trial115.stdout:1419: copy_from oid 2 from oid 41 current snap is 137 2026-02-17T05:54:09.884 INFO:tasks.rados.rados.0.trial115.stdout:1416: finishing write tid 2 to trial11545420-19 2026-02-17T05:54:09.884 INFO:tasks.rados.rados.0.trial115.stdout:1416: finishing write tid 3 to trial11545420-19 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1416: finishing write tid 4 to trial11545420-19 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1416: oid 19 updating version 0 to 756 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1416: oid 19 version 756 is already newer than 755 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1416: oid 19 version 756 is already newer than 754 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 756 (ObjNum 538 snap 137 seq_num 538) dirty exists 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1416: left oid 19 (ObjNum 538 snap 137 seq_num 538) 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1417: finishing rollback tid 0 to trial11545420-5 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1417: finishing rollback tid 1 to trial11545420-5 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1417: finishing rollback tid 2 to trial11545420-5 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 734 (ObjNum 516 snap 132 seq_num 516) dirty exists 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1418: finishing write tid 1 to trial11545420-33 2026-02-17T05:54:09.885 INFO:tasks.rados.rados.0.trial115.stdout:1418: finishing write tid 2 to trial11545420-33 2026-02-17T05:54:09.886 INFO:tasks.rados.rados.0.trial115.stdout:1416: done (3 left) 2026-02-17T05:54:09.886 INFO:tasks.rados.rados.0.trial115.stdout:1417: done (2 left) 2026-02-17T05:54:09.886 INFO:tasks.rados.rados.0.trial115.stdout:1420: copy_from oid 45 from oid 17 current snap is 137 2026-02-17T05:54:09.886 INFO:tasks.rados.rados.0.trial115.stdout:1421: write oid 43 current snap is 137 2026-02-17T05:54:09.886 INFO:tasks.rados.rados.0.trial115.stdout:1421: seq_num 540 ranges {675780=495523,1682747=714511,3012769=63011} 2026-02-17T05:54:09.888 INFO:tasks.rados.rados.0.trial115.stdout:1421: writing trial11545420-43 from 675780 to 1171303 tid 1 2026-02-17T05:54:09.892 INFO:tasks.rados.rados.0.trial115.stdout:1421: writing trial11545420-43 from 1682747 to 2397258 tid 2 2026-02-17T05:54:09.896 INFO:tasks.rados.rados.0.trial115.stdout:1421: writing trial11545420-43 from 3012769 to 3075780 tid 3 2026-02-17T05:54:09.896 INFO:tasks.rados.rados.0.trial115.stdout:1422: rollback oid 7 current snap is 137 2026-02-17T05:54:09.896 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 126 2026-02-17T05:54:09.896 INFO:tasks.rados.rados.0.trial115.stdout:1419: finishing copy_from to trial11545420-2 2026-02-17T05:54:09.896 INFO:tasks.rados.rados.0.trial115.stdout:1419: got expected ENOENT (src dne) 2026-02-17T05:54:09.896 INFO:tasks.rados.rados.0.trial115.stdout:1419: finishing copy_from racing read to trial11545420-2 2026-02-17T05:54:09.896 INFO:tasks.rados.rados.0.trial115.stdout:1418: finishing write tid 3 to trial11545420-33 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1418: finishing write tid 4 to trial11545420-33 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1418: finishing write tid 5 to trial11545420-33 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1418: oid 33 updating version 0 to 607 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1418: oid 33 updating version 607 to 608 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1418: oid 33 updating version 608 to 609 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1418: oid 33 version 609 is already newer than 606 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 609 (ObjNum 539 snap 137 seq_num 539) dirty exists 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1418: left oid 33 (ObjNum 539 snap 137 seq_num 539) 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1420: finishing copy_from racing read to trial11545420-45 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1418: done (4 left) 2026-02-17T05:54:09.897 INFO:tasks.rados.rados.0.trial115.stdout:1419: done (3 left) 2026-02-17T05:54:09.898 INFO:tasks.rados.rados.0.trial115.stdout:1423: copy_from oid 37 from oid 23 current snap is 137 2026-02-17T05:54:09.898 INFO:tasks.rados.rados.0.trial115.stdout:1421: finishing write tid 1 to trial11545420-43 2026-02-17T05:54:09.898 INFO:tasks.rados.rados.0.trial115.stdout:1420: finishing copy_from to trial11545420-45 2026-02-17T05:54:09.898 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 240 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:54:09.898 INFO:tasks.rados.rados.0.trial115.stdout:1421: finishing write tid 2 to trial11545420-43 2026-02-17T05:54:09.898 INFO:tasks.rados.rados.0.trial115.stdout:1420: done (3 left) 2026-02-17T05:54:09.898 INFO:tasks.rados.rados.0.trial115.stdout:1424: snap_create 2026-02-17T05:54:09.899 INFO:tasks.rados.rados.0.trial115.stdout:1421: finishing write tid 3 to trial11545420-43 2026-02-17T05:54:09.899 INFO:tasks.rados.rados.0.trial115.stdout:1421: finishing write tid 4 to trial11545420-43 2026-02-17T05:54:09.899 INFO:tasks.rados.rados.0.trial115.stdout:1421: finishing write tid 5 to trial11545420-43 2026-02-17T05:54:09.899 INFO:tasks.rados.rados.0.trial115.stdout:1421: oid 43 updating version 0 to 530 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:1421: oid 43 version 530 is already newer than 529 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:1421: oid 43 updating version 530 to 531 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:1421: oid 43 updating version 531 to 532 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 532 (ObjNum 540 snap 137 seq_num 540) dirty exists 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:1421: left oid 43 (ObjNum 540 snap 137 seq_num 540) 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:1422: finishing rollback tid 0 to trial11545420-7 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:1422: finishing rollback tid 1 to trial11545420-7 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:1422: finishing rollback tid 2 to trial11545420-7 2026-02-17T05:54:09.900 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 536 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:54:09.902 INFO:tasks.rados.rados.0.trial115.stdout:1423: finishing copy_from racing read to trial11545420-37 2026-02-17T05:54:09.903 INFO:tasks.rados.rados.0.trial115.stdout:1423: finishing copy_from to trial11545420-37 2026-02-17T05:54:09.903 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 265 (ObjNum 532 snap 136 seq_num 532) dirty exists 2026-02-17T05:54:10.864 INFO:tasks.rados.rados.0.trial115.stdout:1421: done (3 left) 2026-02-17T05:54:10.865 INFO:tasks.rados.rados.0.trial115.stdout:1422: done (2 left) 2026-02-17T05:54:10.865 INFO:tasks.rados.rados.0.trial115.stdout:1423: done (1 left) 2026-02-17T05:54:10.865 INFO:tasks.rados.rados.0.trial115.stdout:1424: done (0 left) 2026-02-17T05:54:10.865 INFO:tasks.rados.rados.0.trial115.stdout:1425: snap_create 2026-02-17T05:54:11.856 INFO:tasks.rados.rados.0.trial115.stdout:1425: done (0 left) 2026-02-17T05:54:11.856 INFO:tasks.rados.rados.0.trial115.stdout:1426: copy_from oid 7 from oid 37 current snap is 139 2026-02-17T05:54:11.856 INFO:tasks.rados.rados.0.trial115.stdout:1427: copy_from oid 47 from oid 2 current snap is 139 2026-02-17T05:54:11.856 INFO:tasks.rados.rados.0.trial115.stdout:1428: read oid 22 snap -1 2026-02-17T05:54:11.856 INFO:tasks.rados.rados.0.trial115.stdout:1428: expect deleted 2026-02-17T05:54:11.856 INFO:tasks.rados.rados.0.trial115.stdout:1429: snap_create 2026-02-17T05:54:11.857 INFO:tasks.rados.rados.0.trial115.stdout:1427: finishing copy_from to trial11545420-47 2026-02-17T05:54:11.857 INFO:tasks.rados.rados.0.trial115.stdout:1427: got expected ENOENT (src dne) 2026-02-17T05:54:11.858 INFO:tasks.rados.rados.0.trial115.stdout:1427: finishing copy_from racing read to trial11545420-47 2026-02-17T05:54:11.866 INFO:tasks.rados.rados.0.trial115.stdout:1426: finishing copy_from racing read to trial11545420-7 2026-02-17T05:54:11.867 INFO:tasks.rados.rados.0.trial115.stdout:1426: finishing copy_from to trial11545420-7 2026-02-17T05:54:11.867 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 265 (ObjNum 532 snap 136 seq_num 532) dirty exists 2026-02-17T05:54:12.864 INFO:tasks.rados.rados.0.trial115.stdout:1426: done (3 left) 2026-02-17T05:54:12.864 INFO:tasks.rados.rados.0.trial115.stdout:1427: done (2 left) 2026-02-17T05:54:12.864 INFO:tasks.rados.rados.0.trial115.stdout:1428: done (1 left) 2026-02-17T05:54:12.864 INFO:tasks.rados.rados.0.trial115.stdout:1429: done (0 left) 2026-02-17T05:54:12.864 INFO:tasks.rados.rados.0.trial115.stdout:1430: snap_create 2026-02-17T05:54:14.013 INFO:tasks.rados.rados.0.trial115.stdout:1430: done (0 left) 2026-02-17T05:54:14.013 INFO:tasks.rados.rados.0.trial115.stdout:1431: read oid 41 snap 137 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:1431: expect deleted 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:1432: read oid 23 snap 139 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:1432: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:1433: read oid 19 snap -1 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:1433: expect (ObjNum 538 snap 137 seq_num 538) 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:1434: rollback oid 32 current snap is 141 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 139 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:1435: rollback oid 36 current snap is 141 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 36 to 138 2026-02-17T05:54:14.014 INFO:tasks.rados.rados.0.trial115.stdout:1436: write (excl) oid 18 current snap is 141 2026-02-17T05:54:14.015 INFO:tasks.rados.rados.0.trial115.stdout:1436: seq_num 541 ranges {692587=570772,2024876=574522,3040192=52395} 2026-02-17T05:54:14.018 INFO:tasks.rados.rados.0.trial115.stdout:1436: writing trial11545420-18 from 692587 to 1263359 tid 1 2026-02-17T05:54:14.023 INFO:tasks.rados.rados.0.trial115.stdout:1436: writing trial11545420-18 from 2024876 to 2599398 tid 2 2026-02-17T05:54:14.025 INFO:tasks.rados.rados.0.trial115.stdout:1436: writing trial11545420-18 from 3040192 to 3092587 tid 3 2026-02-17T05:54:14.026 INFO:tasks.rados.rados.0.trial115.stdout:1437: snap_remove snap 122 2026-02-17T05:54:14.966 INFO:tasks.rados.rados.0.trial115.stdout:1437: done (6 left) 2026-02-17T05:54:14.966 INFO:tasks.rados.rados.0.trial115.stdout:1438: read oid 31 snap -1 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:1438: expect deleted 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:1439: rollback oid 5 current snap is 141 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 5 to 136 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:1434: finishing rollback tid 0 to trial11545420-32 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:1434: finishing rollback tid 1 to trial11545420-32 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:1434: finishing rollback tid 2 to trial11545420-32 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 332 (ObjNum 535 snap 136 seq_num 535) dirty exists 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:1435: finishing rollback tid 1 to trial11545420-36 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 605 (ObjNum 49 snap 0 seq_num 1634300532) dirty dne 2026-02-17T05:54:14.967 INFO:tasks.rados.rados.0.trial115.stdout:1432: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:15.052 INFO:tasks.rados.rados.0.trial115.stdout:1433: expect (ObjNum 538 snap 137 seq_num 538) 2026-02-17T05:54:15.143 INFO:tasks.rados.rados.0.trial115.stdout:1436: finishing write tid 1 to trial11545420-18 2026-02-17T05:54:15.143 INFO:tasks.rados.rados.0.trial115.stdout:1431: done (7 left) 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1432: done (6 left) 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1433: done (5 left) 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1434: done (4 left) 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1435: done (3 left) 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1440: read oid 13 snap 140 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1440: expect deleted 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1441: delete oid 21 current snap is 141 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: finishing write tid 2 to trial11545420-18 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: finishing write tid 3 to trial11545420-18 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: finishing write tid 4 to trial11545420-18 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: finishing write tid 5 to trial11545420-18 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: oid 18 updating version 0 to 617 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: oid 18 version 617 is already newer than 615 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: oid 18 version 617 is already newer than 616 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: oid 18 updating version 617 to 618 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 618 (ObjNum 541 snap 141 seq_num 541) dirty exists 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1436: left oid 18 (ObjNum 541 snap 141 seq_num 541) 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1439: finishing rollback tid 0 to trial11545420-5 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1439: finishing rollback tid 1 to trial11545420-5 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:1439: finishing rollback tid 2 to trial11545420-5 2026-02-17T05:54:15.144 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 740 (ObjNum 516 snap 132 seq_num 516) dirty exists 2026-02-17T05:54:15.145 INFO:tasks.rados.rados.0.trial115.stdout:1436: done (4 left) 2026-02-17T05:54:15.145 INFO:tasks.rados.rados.0.trial115.stdout:1438: done (3 left) 2026-02-17T05:54:15.145 INFO:tasks.rados.rados.0.trial115.stdout:1439: done (2 left) 2026-02-17T05:54:15.145 INFO:tasks.rados.rados.0.trial115.stdout:1440: done (1 left) 2026-02-17T05:54:15.145 INFO:tasks.rados.rados.0.trial115.stdout:1441: done (0 left) 2026-02-17T05:54:15.145 INFO:tasks.rados.rados.0.trial115.stdout:1442: read oid 23 snap -1 2026-02-17T05:54:15.145 INFO:tasks.rados.rados.0.trial115.stdout:1442: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:15.145 INFO:tasks.rados.rados.0.trial115.stdout:1443: delete oid 17 current snap is 141 2026-02-17T05:54:15.149 INFO:tasks.rados.rados.0.trial115.stdout:1442: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:15.235 INFO:tasks.rados.rados.0.trial115.stdout:1442: done (1 left) 2026-02-17T05:54:15.235 INFO:tasks.rados.rados.0.trial115.stdout:1443: done (0 left) 2026-02-17T05:54:15.235 INFO:tasks.rados.rados.0.trial115.stdout:1444: write oid 23 current snap is 141 2026-02-17T05:54:15.235 INFO:tasks.rados.rados.0.trial115.stdout:1444: seq_num 544 ranges {743008=796519,2200322=721849} 2026-02-17T05:54:15.240 INFO:tasks.rados.rados.0.trial115.stdout:1444: writing trial11545420-23 from 743008 to 1539527 tid 1 2026-02-17T05:54:15.245 INFO:tasks.rados.rados.0.trial115.stdout:1444: writing trial11545420-23 from 2200322 to 2922171 tid 2 2026-02-17T05:54:15.245 INFO:tasks.rados.rados.0.trial115.stdout:1445: snap_remove snap 126 2026-02-17T05:54:15.246 INFO:tasks.rados.rados.0.trial115.stdout:1444: finishing write tid 1 to trial11545420-23 2026-02-17T05:54:15.853 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T05:54:15.853 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T05:54:15.853 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T05:54:15.853 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T05:54:15.857 INFO:tasks.ceph.mon.b:Started 2026-02-17T05:54:15.857 INFO:tasks.mon_thrash:Sending STOP to mon c 2026-02-17T05:54:15.857 INFO:tasks.ceph.mon.c:Sent signal 19 2026-02-17T05:54:15.857 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T05:54:30.858 INFO:tasks.mon_thrash:Sending CONT to mon c 2026-02-17T05:54:30.858 INFO:tasks.ceph.mon.c:Sent signal 18 2026-02-17T05:54:30.859 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:54:30.859 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:54:30.925 INFO:tasks.rados.rados.0.trial115.stdout:1445: done (1 left) 2026-02-17T05:54:30.925 INFO:tasks.rados.rados.0.trial115.stdout:1446: read oid 49 snap -1 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1446: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1447: copy_from oid 24 from oid 21 current snap is 141 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1444: finishing write tid 2 to trial11545420-23 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1444: finishing write tid 3 to trial11545420-23 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1444: finishing write tid 4 to trial11545420-23 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1444: oid 23 updating version 0 to 278 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1444: oid 23 version 278 is already newer than 277 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1444: oid 23 updating version 278 to 279 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 279 (ObjNum 544 snap 141 seq_num 544) dirty exists 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1444: left oid 23 (ObjNum 544 snap 141 seq_num 544) 2026-02-17T05:54:30.926 INFO:tasks.rados.rados.0.trial115.stdout:1444: done (2 left) 2026-02-17T05:54:30.927 INFO:tasks.rados.rados.0.trial115.stdout:1448: write (excl) oid 31 current snap is 141 2026-02-17T05:54:30.927 INFO:tasks.rados.rados.0.trial115.stdout:1448: seq_num 545 ranges {759815=471768,1742451=465507,2917178=242637} 2026-02-17T05:54:30.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.929+0000 7fb35f86c640 10 monclient: get_monmap_and_config 2026-02-17T05:54:30.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.929+0000 7fb35f86c640 10 monclient: build_initial_monmap 2026-02-17T05:54:30.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.929+0000 7fb35f86c640 10 monclient: monmap: 2026-02-17T05:54:30.929 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:30.929 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:30.929 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:30.931197+0000 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:30.931197+0000 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:30.930 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:30.931 INFO:tasks.rados.rados.0.trial115.stdout:1448: writing trial11545420-31 from 759815 to 1231583 tid 1 2026-02-17T05:54:30.936 INFO:tasks.rados.rados.0.trial115.stdout:1448: writing trial11545420-31 from 1742451 to 2207958 tid 2 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 1 Processor -- start 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 1 -- start start 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient: init 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806c890 0x7fb3580a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient(hunting): picked mon.noname-b con 0x7fb35806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb35806cc70 0x7fb35806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient(hunting): picked mon.noname-c con 0x7fb35806cc70 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb358134df0 0x7fb358135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient(hunting): picked mon.noname-a con 0x7fb358134df0 addr v2:10.20.193.78:3300/0 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:30.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb358074150 con 0x7fb358134df0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb35809ef70 con 0x7fb35806cc70 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb35809f250 con 0x7fb35806c890 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35f86c640 10 monclient(hunting): authenticate will time out at 1116.484900s 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.930+0000 7fb35d5e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806c890 0x7fb3580a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb358134df0 0x7fb358135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806c890 0x7fb3580a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44450/0 (socket says 10.20.193.78:44450) 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb358134df0 0x7fb358135240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39486/0 (socket says 10.20.193.78:39486) 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 1 -- 10.20.193.78:0/2963807593 learned_addr learned my addr 10.20.193.78:0/2963807593 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 10 monclient(hunting): get_auth_request con 0x7fb35806c890 auth_method 0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient(hunting): get_auth_request con 0x7fb358134df0 auth_method 0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35cde0640 1 --2- 10.20.193.78:0/2963807593 >> v2:10.20.193.115:3300/0 conn(0x7fb35806cc70 0x7fb35806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35cde0640 10 monclient(hunting): get_auth_request con 0x7fb35806cc70 auth_method 0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35cde0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35cde0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35cde0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient(hunting): handle_auth_done global_id 104169 payload 274 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 1 -- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806c890 msgr2=0x7fb3580a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 1 --2- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806c890 0x7fb3580a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 1 -- 10.20.193.78:0/2963807593 >> v2:10.20.193.115:3300/0 conn(0x7fb35806cc70 msgr2=0x7fb35806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 1 --2- 10.20.193.78:0/2963807593 >> v2:10.20.193.115:3300/0 conn(0x7fb35806cc70 0x7fb35806d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient: _finish_hunting 0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 1 monclient: found mon.noname-a 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35dde2640 1 -- 10.20.193.78:0/2963807593 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35809df40 con 0x7fb358134df0 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35d5e1640 1 --2- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806c890 0x7fb3580a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:54:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.931+0000 7fb35cde0640 1 --2- 10.20.193.78:0/2963807593 >> v2:10.20.193.115:3300/0 conn(0x7fb35806cc70 0x7fb35806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb35dde2640 1 -- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb358134df0 msgr2=0x7fb358135240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb35dde2640 1 --2- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb358134df0 0x7fb358135240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb35809d790 tx=0x7fb35402f250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8b5f747024a45f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 1 -- 10.20.193.78:0/2963807593 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35403c070 con 0x7fb358134df0 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: dump: 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: _finish_auth 0 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: _check_auth_tickets 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 1 -- 10.20.193.78:0/2963807593 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb354037640 con 0x7fb358134df0 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 1 -- 10.20.193.78:0/2963807593 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354037a10 con 0x7fb358134df0 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb35f86c640 5 monclient: authenticate success, global_id 104169 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb35f86c640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb35f86c640 4 set_mon_vals no callback set 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.933+0000 7fb347fff640 10 monclient: dump: 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 10 monclient: shutdown 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 -- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb358134df0 msgr2=0x7fb358135240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 --2- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb358134df0 0x7fb358135240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb35809d790 tx=0x7fb35402f250 comp rx=0 tx=0).stop 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 -- 10.20.193.78:0/2963807593 shutdown_connections 2026-02-17T05:54:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 --2- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb358134df0 0x7fb358135240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 --2- 10.20.193.78:0/2963807593 >> v2:10.20.193.115:3300/0 conn(0x7fb35806cc70 0x7fb35806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 --2- 10.20.193.78:0/2963807593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806c890 0x7fb3580a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 -- 10.20.193.78:0/2963807593 >> 10.20.193.78:0/2963807593 conn(0x7fb3580a6620 msgr2=0x7fb3580a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 -- 10.20.193.78:0/2963807593 shutdown_connections 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 -- 10.20.193.78:0/2963807593 wait complete. 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 10 monclient: build_initial_monmap 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 10 monclient: monmap: 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 Processor -- start 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 1 -- start start 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.934+0000 7fb35f86c640 10 monclient: init 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb35806c890 0x7fb358133640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient(hunting): picked mon.noname-a con 0x7fb35806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806cc70 0x7fb358133b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient(hunting): picked mon.noname-c con 0x7fb35806cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb358134df0 0x7fb358139a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient(hunting): picked mon.noname-b con 0x7fb358134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb35809d360 con 0x7fb35806c890 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb35809d1e0 con 0x7fb358134df0 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb358135780 con 0x7fb35806cc70 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35f86c640 10 monclient(hunting): authenticate will time out at 1116.489060s 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb35806c890 0x7fb358133640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb35806c890 0x7fb358133640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39500/0 (socket says 10.20.193.78:39500) 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 -- 10.20.193.78:0/886377869 learned_addr learned my addr 10.20.193.78:0/886377869 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35cde0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806cc70 0x7fb358133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient(hunting): get_auth_request con 0x7fb35806c890 auth_method 0 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35cde0640 10 monclient(hunting): get_auth_request con 0x7fb35806cc70 auth_method 0 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35cde0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35cde0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35cde0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35cde0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35cde0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35cde0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient(hunting): handle_auth_done global_id 104172 payload 824 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35dde2640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb358134df0 0x7fb358139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 -- 10.20.193.78:0/886377869 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806cc70 msgr2=0x7fb358133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806cc70 0x7fb358133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 -- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb358134df0 msgr2=0x7fb358139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb358134df0 0x7fb358139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient: _finish_hunting 0 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 monclient: found mon.noname-a 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35d5e1640 1 -- 10.20.193.78:0/886377869 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35811dfd0 con 0x7fb35806c890 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35dde2640 10 monclient: get_auth_request con 0x7fb358134df0 auth_method 0 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.935+0000 7fb35dde2640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb358134df0 0x7fb358139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35d5e1640 1 -- 10.20.193.78:0/886377869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb35806c890 msgr2=0x7fb358133640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35d5e1640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb35806c890 0x7fb358133640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb34800ebe0 tx=0x7fb34800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 1 -- 10.20.193.78:0/886377869 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348017070 con 0x7fb35806c890 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: dump: 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: _finish_auth 0 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 1 -- 10.20.193.78:0/886377869 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34800ce50 con 0x7fb35806c890 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35f86c640 5 monclient: authenticate success, global_id 104172 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35f86c640 10 monclient: _renew_subs 2026-02-17T05:54:30.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35f86c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35813bb50 con 0x7fb35806c890 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 1 -- 10.20.193.78:0/886377869 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348013690 con 0x7fb35806c890 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb345ffb640 10 monclient: dump: 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb3477fe640 4 set_mon_vals no callback set 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35f86c640 10 monclient: _renew_subs 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35f86c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb358072e20 con 0x7fb35806c890 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.937+0000 7fb35f86c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.938+0000 7fb35f86c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.938+0000 7fb35f86c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.938+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb358074980 con 0x7fb35806c890 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.938+0000 7fb345ffb640 1 -- 10.20.193.78:0/886377869 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb348024060 con 0x7fb35806c890 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.938+0000 7fb345ffb640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb32803deb0 0x7fb328040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.938+0000 7fb345ffb640 1 -- 10.20.193.78:0/886377869 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(306..306 src has 117..306) ==== 4779+0+0 (secure 0 0 0) 0x7fb3480528d0 con 0x7fb35806c890 2026-02-17T05:54:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.938+0000 7fb35cde0640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb32803deb0 0x7fb328040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.938+0000 7fb35cde0640 10 monclient: get_auth_request con 0x7fb32803deb0 auth_method 0 2026-02-17T05:54:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.941+0000 7fb35cde0640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb32803deb0 0x7fb328040370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb358134700 tx=0x7fb34c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.941+0000 7fb345ffb640 1 -- 10.20.193.78:0/886377869 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb348013d50 con 0x7fb35806c890 2026-02-17T05:54:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.941+0000 7fb345ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:30.941+0000 7fb345ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:30.943 INFO:tasks.rados.rados.0.trial115.stdout:1448: writing trial11545420-31 from 2917178 to 3159815 tid 3 2026-02-17T05:54:30.943 INFO:tasks.rados.rados.0.trial115.stdout:1449: write oid 32 current snap is 141 2026-02-17T05:54:30.943 INFO:tasks.rados.rados.0.trial115.stdout:1449: seq_num 546 ranges {776622=547017,1800933=725518,2944601=232021} 2026-02-17T05:54:30.943 INFO:tasks.rados.rados.0.trial115.stdout:1449: writing trial11545420-32 from 776622 to 1323639 tid 1 2026-02-17T05:54:30.946 INFO:tasks.rados.rados.0.trial115.stdout:1449: writing trial11545420-32 from 1800933 to 2526451 tid 2 2026-02-17T05:54:30.949 INFO:tasks.rados.rados.0.trial115.stdout:1449: writing trial11545420-32 from 2944601 to 3176622 tid 3 2026-02-17T05:54:30.949 INFO:tasks.rados.rados.0.trial115.stdout:1450: snap_create 2026-02-17T05:54:30.949 INFO:tasks.rados.rados.0.trial115.stdout:1447: finishing copy_from to trial11545420-24 2026-02-17T05:54:30.949 INFO:tasks.rados.rados.0.trial115.stdout:1447: got expected ENOENT (src dne) 2026-02-17T05:54:30.949 INFO:tasks.rados.rados.0.trial115.stdout:1447: finishing copy_from racing read to trial11545420-24 2026-02-17T05:54:30.949 INFO:tasks.rados.rados.0.trial115.stdout:1446: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:54:31.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.056+0000 7fb35f86c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:54:31.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.056+0000 7fb35f86c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:31.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.056+0000 7fb35f86c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.056+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb3580a3920 con 0x7fb35806c890 2026-02-17T05:54:31.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb345ffb640 1 -- 10.20.193.78:0/886377869 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fb348013830 con 0x7fb35806c890 2026-02-17T05:54:31.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb345ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:31.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb345ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:31.058 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb32803deb0 msgr2=0x7fb328040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb35f86c640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb32803deb0 0x7fb328040370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb358134700 tx=0x7fb34c008040 comp rx=0 tx=0).stop 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb35f86c640 10 monclient: shutdown 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb35f86c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb35806c890 msgr2=0x7fb358133640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.057+0000 7fb35f86c640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb35806c890 0x7fb358133640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb34800ebe0 tx=0x7fb34800c520 comp rx=0 tx=0).stop 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.058+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 shutdown_connections 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.058+0000 7fb35f86c640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb32803deb0 0x7fb328040370 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.058+0000 7fb35f86c640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb358134df0 0x7fb358139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.058+0000 7fb35f86c640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb35806cc70 0x7fb358133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.058+0000 7fb35f86c640 1 --2- 10.20.193.78:0/886377869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb35806c890 0x7fb358133640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.058+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 >> 10.20.193.78:0/886377869 conn(0x7fb3580a6620 msgr2=0x7fb358070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.058+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 shutdown_connections 2026-02-17T05:54:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.058+0000 7fb35f86c640 1 -- 10.20.193.78:0/886377869 wait complete. 2026-02-17T05:54:31.066 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:54:31.066 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:54:31.066 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:54:31.094 INFO:tasks.rados.rados.0.trial115.stdout:1448: finishing write tid 1 to trial11545420-31 2026-02-17T05:54:31.094 INFO:tasks.rados.rados.0.trial115.stdout:1448: finishing write tid 2 to trial11545420-31 2026-02-17T05:54:31.094 INFO:tasks.rados.rados.0.trial115.stdout:1448: finishing write tid 3 to trial11545420-31 2026-02-17T05:54:31.094 INFO:tasks.rados.rados.0.trial115.stdout:1448: finishing write tid 4 to trial11545420-31 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1448: finishing write tid 5 to trial11545420-31 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1448: oid 31 updating version 0 to 758 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1448: oid 31 version 758 is already newer than 751 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1448: oid 31 version 758 is already newer than 756 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1448: oid 31 version 758 is already newer than 757 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 758 (ObjNum 545 snap 141 seq_num 545) dirty exists 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1448: left oid 31 (ObjNum 545 snap 141 seq_num 545) 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1449: finishing write tid 1 to trial11545420-32 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1449: finishing write tid 2 to trial11545420-32 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1449: finishing write tid 3 to trial11545420-32 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1449: finishing write tid 4 to trial11545420-32 2026-02-17T05:54:31.095 INFO:tasks.rados.rados.0.trial115.stdout:1449: finishing write tid 5 to trial11545420-32 2026-02-17T05:54:31.096 INFO:tasks.rados.rados.0.trial115.stdout:1449: oid 32 updating version 0 to 335 2026-02-17T05:54:31.096 INFO:tasks.rados.rados.0.trial115.stdout:1449: oid 32 updating version 335 to 336 2026-02-17T05:54:31.096 INFO:tasks.rados.rados.0.trial115.stdout:1449: oid 32 version 336 is already newer than 333 2026-02-17T05:54:31.096 INFO:tasks.rados.rados.0.trial115.stdout:1449: oid 32 version 336 is already newer than 334 2026-02-17T05:54:31.096 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 336 (ObjNum 546 snap 141 seq_num 546) dirty exists 2026-02-17T05:54:31.096 INFO:tasks.rados.rados.0.trial115.stdout:1449: left oid 32 (ObjNum 546 snap 141 seq_num 546) 2026-02-17T05:54:31.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.136+0000 7fb9aea3f640 10 monclient: get_monmap_and_config 2026-02-17T05:54:31.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.136+0000 7fb9aea3f640 10 monclient: build_initial_monmap 2026-02-17T05:54:31.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.136+0000 7fb9aea3f640 10 monclient: monmap: 2026-02-17T05:54:31.139 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:31.139 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.139 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:31.137944+0000 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:31.137944+0000 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 1 Processor -- start 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 1 -- start start 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient: init 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a806c890 0x7fb9a806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient(hunting): picked mon.noname-b con 0x7fb9a806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:31.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb9a809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb9a806d1b0 0x7fb9a8135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient(hunting): picked mon.noname-a con 0x7fb9a806d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a806daf0 con 0x7fb9a809c4a0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a80a0880 con 0x7fb9a806d1b0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a809cc80 con 0x7fb9a806c890 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9aea3f640 10 monclient(hunting): authenticate will time out at 1116.691488s 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a806c890 0x7fb9a806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a806c890 0x7fb9a806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44460/0 (socket says 10.20.193.78:44460) 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb9a806d1b0 0x7fb9a8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 1 -- 10.20.193.78:0/1261200999 learned_addr learned my addr 10.20.193.78:0/1261200999 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient(hunting): get_auth_request con 0x7fb9a806c890 auth_method 0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 10 monclient(hunting): get_auth_request con 0x7fb9a806d1b0 auth_method 0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient(hunting): handle_auth_done global_id 74453 payload 274 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 1 -- 10.20.193.78:0/1261200999 >> v2:10.20.193.78:3300/0 conn(0x7fb9a806d1b0 msgr2=0x7fb9a8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 1 --2- 10.20.193.78:0/1261200999 >> v2:10.20.193.78:3300/0 conn(0x7fb9a806d1b0 0x7fb9a8135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 1 -- 10.20.193.78:0/1261200999 >> v1:10.20.193.115:6789/0 conn(0x7fb9a809c4a0 legacy=0x7fb9a8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 1 monclient: found mon.noname-b 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9a7fff640 1 -- 10.20.193.78:0/1261200999 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a809cf60 con 0x7fb9a806c890 2026-02-17T05:54:31.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.137+0000 7fb9acfb5640 1 --2- 10.20.193.78:0/1261200999 >> v2:10.20.193.78:3300/0 conn(0x7fb9a806d1b0 0x7fb9a8135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a7fff640 1 -- 10.20.193.78:0/1261200999 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a806c890 msgr2=0x7fb9a806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a7fff640 1 --2- 10.20.193.78:0/1261200999 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a806c890 0x7fb9a806cc70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb994009970 tx=0x7fb99402ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e1a49fcc7a8cfe server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 1 -- 10.20.193.78:0/1261200999 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99403c070 con 0x7fb9a806c890 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: dump: 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.143 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 1 -- 10.20.193.78:0/1261200999 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99402fcd0 con 0x7fb9a806c890 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 5 monclient: authenticate success, global_id 74453 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 1 -- 10.20.193.78:0/1261200999 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994037660 con 0x7fb9a806c890 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9a6ffd640 10 monclient: dump: 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 4 set_mon_vals no callback set 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 10 monclient: shutdown 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 1 -- 10.20.193.78:0/1261200999 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a806c890 msgr2=0x7fb9a806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 1 --2- 10.20.193.78:0/1261200999 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a806c890 0x7fb9a806cc70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb994009970 tx=0x7fb99402ee90 comp rx=0 tx=0).stop 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 1 -- 10.20.193.78:0/1261200999 shutdown_connections 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 1 --2- 10.20.193.78:0/1261200999 >> v2:10.20.193.78:3300/0 conn(0x7fb9a806d1b0 0x7fb9a8135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 1 --2- 10.20.193.78:0/1261200999 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a806c890 0x7fb9a806cc70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 1 -- 10.20.193.78:0/1261200999 >> 10.20.193.78:0/1261200999 conn(0x7fb9a80a6620 msgr2=0x7fb9a80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 1 -- 10.20.193.78:0/1261200999 shutdown_connections 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.140+0000 7fb9aea3f640 1 -- 10.20.193.78:0/1261200999 wait complete. 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient: build_initial_monmap 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient: monmap: 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:31.144 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 1 Processor -- start 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 1 -- start start 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient: init 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a806c890 0x7fb9a81cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient(hunting): picked mon.noname-a con 0x7fb9a806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb9a806d1b0 0x7fb9a81238c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient(hunting): picked mon.noname-b con 0x7fb9a806d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a809c4a0 0x7fb9a8123e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient(hunting): picked mon.noname-c con 0x7fb9a809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80a01a0 con 0x7fb9a806c890 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80a0020 con 0x7fb9a806d1b0 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a809be20 con 0x7fb9a809c4a0 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9aea3f640 10 monclient(hunting): authenticate will time out at 1116.695801s 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a806c890 0x7fb9a81cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a806c890 0x7fb9a81cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39530/0 (socket says 10.20.193.78:39530) 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 1 -- 10.20.193.78:0/707225071 learned_addr learned my addr 10.20.193.78:0/707225071 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9acfb5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a809c4a0 0x7fb9a8123e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 10 monclient(hunting): get_auth_request con 0x7fb9a806c890 auth_method 0 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9acfb5640 10 monclient(hunting): get_auth_request con 0x7fb9a809c4a0 auth_method 0 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9acfb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9acfb5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9acfb5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9acfb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9acfb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9acfb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 10 monclient(hunting): handle_auth_done global_id 104178 payload 824 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a809c4a0 msgr2=0x7fb9a8123e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9a809c4a0 0x7fb9a8123e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb9a806d1b0 msgr2=0x7fb9a81238c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.141+0000 7fb9a7fff640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb9a806d1b0 0x7fb9a81238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.142+0000 7fb9a7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.142+0000 7fb9a7fff640 1 monclient: found mon.noname-a 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.142+0000 7fb9a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.142+0000 7fb9a7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.142+0000 7fb9a7fff640 1 -- 10.20.193.78:0/707225071 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a8127570 con 0x7fb9a806c890 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a7fff640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a806c890 msgr2=0x7fb9a81cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a7fff640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a806c890 0x7fb9a81cd780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb994009970 tx=0x7fb994004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99403c040 con 0x7fb9a806c890 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: dump: 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb994047070 con 0x7fb9a806c890 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 5 monclient: authenticate success, global_id 104178 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 10 monclient: _renew_subs 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a8125a00 con 0x7fb9a806c890 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994042890 con 0x7fb9a806c890 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a67fc640 4 set_mon_vals no callback set 2026-02-17T05:54:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 10 monclient: dump: 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 10 monclient: _renew_subs 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a8136ea0 con 0x7fb9a806c890 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a81cdcc0 0x7fb9a8124340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a806f800 con 0x7fb9a81cdcc0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb994042a30 con 0x7fb9a806c890 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a77fe640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a81cdcc0 0x7fb9a8124340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a77fe640 10 monclient: get_auth_request con 0x7fb9a81cdcc0 auth_method 0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.144+0000 7fb9a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.145+0000 7fb9a77fe640 10 monclient(hunting): handle_auth_done global_id 104181 payload 824 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.145+0000 7fb9a77fe640 1 -- 10.20.193.78:0/707225071 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9a8074980 con 0x7fb9a81cdcc0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.145+0000 7fb9a4ff9640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97803deb0 0x7fb978040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.145+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(306..306 src has 117..306) ==== 4779+0+0 (secure 0 0 0) 0x7fb994077a60 con 0x7fb9a806c890 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.145+0000 7fb9acfb5640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97803deb0 0x7fb978040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.145+0000 7fb9acfb5640 10 monclient: get_auth_request con 0x7fb97803deb0 auth_method 0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a77fe640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a81cdcc0 msgr2=0x7fb9a8124340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a77fe640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a81cdcc0 0x7fb9a8124340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c360 tx=0x7fb99800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998011a80 con 0x7fb9a81cdcc0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 10 monclient: dump: 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fb998017070 con 0x7fb9a81cdcc0 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a81cdcc0 msgr2=0x7fb9a8124340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a81cdcc0 0x7fb9a8124340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c360 tx=0x7fb99800c830 comp rx=0 tx=0).stop 2026-02-17T05:54:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a81cdcc0 msgr2=0x7fb9a8124340 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9a4ff9640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a81cdcc0 0x7fb9a8124340 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c360 tx=0x7fb99800c830 comp rx=0 tx=0).stop 2026-02-17T05:54:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.147+0000 7fb9acfb5640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97803deb0 0x7fb978040370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8126990 tx=0x7fb99c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9aea3f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:54:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9aea3f640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a8124880 0x7fb9a81d8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9aea3f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a806cc70 con 0x7fb9a8124880 2026-02-17T05:54:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a8124880 0x7fb9a81d8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 10 monclient: get_auth_request con 0x7fb9a8124880 auth_method 0 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 10 monclient(hunting): handle_auth_done global_id 104184 payload 824 2026-02-17T05:54:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.150+0000 7fb9a77fe640 1 -- 10.20.193.78:0/707225071 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb9a8124c90 con 0x7fb9a8124880 2026-02-17T05:54:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.152+0000 7fb9a77fe640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a8124880 msgr2=0x7fb9a81d8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.152+0000 7fb9a77fe640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a8124880 0x7fb9a81d8410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c360 tx=0x7fb998003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.152+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99801cb00 con 0x7fb9a8124880 2026-02-17T05:54:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.152+0000 7fb9a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.152+0000 7fb9a4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.152+0000 7fb9a4ff9640 10 monclient: dump: 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fb9980068d0 con 0x7fb9a8124880 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9a4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9a4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a8124880 msgr2=0x7fb9a81d8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9a4ff9640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a8124880 0x7fb9a81d8410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c360 tx=0x7fb998003860 comp rx=0 tx=0).stop 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9a4ff9640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a8124880 msgr2=0x7fb9a81d8410 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9a4ff9640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a8124880 0x7fb9a81d8410 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c360 tx=0x7fb998003860 comp rx=0 tx=0).stop 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 88, 2026-02-17T05:54:31.154 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 64304, 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:54:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.159 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.160 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.161 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.162 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.163 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.164 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:54:31.165 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:54:31.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97803deb0 msgr2=0x7fb978040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97803deb0 0x7fb978040370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8126990 tx=0x7fb99c00b040 comp rx=0 tx=0).stop 2026-02-17T05:54:31.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 10 monclient: shutdown 2026-02-17T05:54:31.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a806c890 msgr2=0x7fb9a81cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 1 --2- 10.20.193.78:0/707225071 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9a806c890 0x7fb9a81cd780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb994009970 tx=0x7fb994004830 comp rx=0 tx=0).stop 2026-02-17T05:54:31.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9acfb5640 1 -- 10.20.193.78:0/707225071 reap_dead start 2026-02-17T05:54:31.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9acfb5640 1 -- 10.20.193.78:0/707225071 reap_dead start 2026-02-17T05:54:31.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 shutdown_connections 2026-02-17T05:54:31.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 >> 10.20.193.78:0/707225071 conn(0x7fb9a80a6620 msgr2=0x7fb9a8070320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 shutdown_connections 2026-02-17T05:54:31.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.153+0000 7fb9aea3f640 1 -- 10.20.193.78:0/707225071 wait complete. 2026-02-17T05:54:31.167 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:54:31.167 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:54:31.168 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:54:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.236+0000 7feed66f2640 10 monclient: get_monmap_and_config 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.236+0000 7feed66f2640 10 monclient: build_initial_monmap 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.236+0000 7feed66f2640 10 monclient: monmap: 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:31.237706+0000 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:31.237706+0000 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:31.236 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:31.237 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:31.237 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:31.237 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.236+0000 7feed66f2640 1 Processor -- start 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.236+0000 7feed66f2640 1 -- start start 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.236+0000 7feed66f2640 10 monclient: init 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7feed006c890 0x7feed006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient(hunting): picked mon.noname-c con 0x7feed006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feed009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient(hunting): picked mon.noname-b con 0x7feed006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed006daf0 con 0x7feed009c4a0 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7feed00a0880 con 0x7feed006c890 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7feed009cc80 con 0x7feed006d1b0 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed66f2640 10 monclient(hunting): authenticate will time out at 1116.791291s 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feecf7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7feed009c4a0 0x7feed0134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57226/0 (socket says 10.20.193.78:57226) 2026-02-17T05:54:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feecf7fe640 1 -- 10.20.193.78:0/175968448 learned_addr learned my addr 10.20.193.78:0/175968448 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient(hunting): get_auth_request con 0x7feed006d1b0 auth_method 0 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient(hunting): handle_auth_done global_id 74459 payload 274 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 1 -- 10.20.193.78:0/175968448 >> v2:10.20.193.115:3300/0 conn(0x7feed006c890 msgr2=0x7feed006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 1 --2- 10.20.193.78:0/175968448 >> v2:10.20.193.115:3300/0 conn(0x7feed006c890 0x7feed006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 1 -- 10.20.193.78:0/175968448 >> v1:10.20.193.78:6789/0 conn(0x7feed009c4a0 legacy=0x7feed0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient: _finish_hunting 0 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 1 monclient: found mon.noname-b 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.237+0000 7feed4c68640 1 -- 10.20.193.78:0/175968448 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed009cf60 con 0x7feed006d1b0 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed4c68640 1 -- 10.20.193.78:0/175968448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 msgr2=0x7feed0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed4c68640 1 --2- 10.20.193.78:0/175968448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0135500 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feed006dcc0 tx=0x7feec402f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1c556ffde92e98a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 1 -- 10.20.193.78:0/175968448 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec403c070 con 0x7feed006d1b0 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: dump: 2026-02-17T05:54:31.246 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: _finish_auth 0 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 1 -- 10.20.193.78:0/175968448 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4037640 con 0x7feed006d1b0 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 5 monclient: authenticate success, global_id 74459 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 1 -- 10.20.193.78:0/175968448 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4037a10 con 0x7feed006d1b0 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 4 set_mon_vals no callback set 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feeceffd640 10 monclient: dump: 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.247 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 10 monclient: shutdown 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 1 -- 10.20.193.78:0/175968448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 msgr2=0x7feed0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 1 --2- 10.20.193.78:0/175968448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0135500 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feed006dcc0 tx=0x7feec402f150 comp rx=0 tx=0).stop 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 1 -- 10.20.193.78:0/175968448 shutdown_connections 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 1 --2- 10.20.193.78:0/175968448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0135500 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 1 --2- 10.20.193.78:0/175968448 >> v2:10.20.193.115:3300/0 conn(0x7feed006c890 0x7feed006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 1 -- 10.20.193.78:0/175968448 >> 10.20.193.78:0/175968448 conn(0x7feed00a6620 msgr2=0x7feed00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 1 -- 10.20.193.78:0/175968448 shutdown_connections 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.240+0000 7feed66f2640 1 -- 10.20.193.78:0/175968448 wait complete. 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient: build_initial_monmap 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient: monmap: 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:31.248 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 1 Processor -- start 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 1 -- start start 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient: init 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feed006c890 0x7feed01cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient(hunting): picked mon.noname-b con 0x7feed006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient(hunting): picked mon.noname-c con 0x7feed006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feed009c4a0 0x7feed0134410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient(hunting): picked mon.noname-a con 0x7feed009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7feed00a01a0 con 0x7feed009c4a0 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7feed00a0020 con 0x7feed006c890 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7feed009be20 con 0x7feed006d1b0 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed66f2640 10 monclient(hunting): authenticate will time out at 1116.795696s 2026-02-17T05:54:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feed009c4a0 0x7feed0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0133ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44478/0 (socket says 10.20.193.78:44478) 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 1 -- 10.20.193.78:0/100399266 learned_addr learned my addr 10.20.193.78:0/100399266 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 10 monclient(hunting): get_auth_request con 0x7feed006d1b0 auth_method 0 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient(hunting): get_auth_request con 0x7feed009c4a0 auth_method 0 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecffff640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feed006c890 0x7feed01cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecffff640 10 monclient(hunting): get_auth_request con 0x7feed006c890 auth_method 0 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient(hunting): handle_auth_done global_id 104187 payload 824 2026-02-17T05:54:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 msgr2=0x7feed0133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed006d1b0 0x7feed0133ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feed006c890 msgr2=0x7feed01cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feed006c890 0x7feed01cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient: _finish_hunting 0 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 1 monclient: found mon.noname-a 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.241+0000 7feed4c68640 1 -- 10.20.193.78:0/100399266 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed009d810 con 0x7feed009c4a0 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed4c68640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feed009c4a0 msgr2=0x7feed0134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed4c68640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feed009c4a0 0x7feed0134410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feed0077b90 tx=0x7feec4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4047020 con 0x7feed009c4a0 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: dump: 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 5 monclient: authenticate success, global_id 104187 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4042880 con 0x7feed009c4a0 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec403c040 con 0x7feed009c4a0 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feece7fc640 4 set_mon_vals no callback set 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feeccff9640 10 monclient: dump: 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 10 monclient: _renew_subs 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed009f7f0 con 0x7feed009c4a0 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 10 monclient: _renew_subs 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed0126160 con 0x7feed009c4a0 2026-02-17T05:54:31.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e310 0x7feed011cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7feed00a22a0 con 0x7feed009e310 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feecffff640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e310 0x7feed011cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feecffff640 10 monclient: get_auth_request con 0x7feed009e310 auth_method 0 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feecffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feecffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.244+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.245+0000 7feecffff640 10 monclient(hunting): handle_auth_done global_id 74465 payload 824 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.245+0000 7feecffff640 1 -- 10.20.193.78:0/100399266 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feed0135ad0 con 0x7feed009e310 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.245+0000 7feed4c68640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feed009c4a0 0x7feed0134410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feed0077b90 tx=0x7feec4004b40 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7feec4078440 osd_map(306..306 src has 117..306) 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.245+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7feec4042cb0 con 0x7feed009c4a0 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.245+0000 7feeccff9640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feea003dee0 0x7feea00403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.245+0000 7feecf7fe640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feea003dee0 0x7feea00403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.245+0000 7feecf7fe640 10 monclient: get_auth_request con 0x7feea003dee0 auth_method 0 2026-02-17T05:54:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feecffff640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e310 msgr2=0x7feed011cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feecffff640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e310 0x7feed011cc70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feebc009630 tx=0x7feebc009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc016070 con 0x7feed009e310 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 10 monclient: dump: 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7feebc00ea30 con 0x7feed009e310 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e310 msgr2=0x7feed011cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e310 0x7feed011cc70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feebc009630 tx=0x7feebc009b00 comp rx=0 tx=0).stop 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e310 msgr2=0x7feed011cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.247+0000 7feeccff9640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e310 0x7feed011cc70 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feebc009630 tx=0x7feebc009b00 comp rx=0 tx=0).stop 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.248+0000 7feecf7fe640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feea003dee0 0x7feea00403a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feec0007e50 tx=0x7feec0007da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feed66f2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feed66f2640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e6f0 0x7feed01da510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feed66f2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7feed0134d90 con 0x7feed009e6f0 2026-02-17T05:54:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e6f0 0x7feed01da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 10 monclient: get_auth_request con 0x7feed009e6f0 auth_method 0 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 10 monclient(hunting): handle_auth_done global_id 74468 payload 824 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.250+0000 7feecffff640 1 -- 10.20.193.78:0/100399266 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feed011e330 con 0x7feed009e6f0 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feecffff640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e6f0 msgr2=0x7feed01da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feecffff640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e6f0 0x7feed01da510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feebc009630 tx=0x7feebc004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc016050 con 0x7feed009e6f0 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 10 monclient: dump: 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7feebc006030 con 0x7feed009e6f0 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e6f0 msgr2=0x7feed01da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e6f0 0x7feed01da510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feebc009630 tx=0x7feebc004830 comp rx=0 tx=0).stop 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e6f0 msgr2=0x7feed01da510 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feed009e6f0 0x7feed01da510 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 88, 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 243146, 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:54:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:54:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:54:31.262 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feea003dee0 msgr2=0x7feea00403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feea003dee0 0x7feea00403a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feec0007e50 tx=0x7feec0007da0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 10 monclient: shutdown 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feed009c4a0 msgr2=0x7feed0134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feed009c4a0 0x7feed0134410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feed0077b90 tx=0x7feec4004b40 comp rx=0 tx=0).stop 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed4c68640 1 -- 10.20.193.78:0/100399266 reap_dead start 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feeccff9640 1 -- 10.20.193.78:0/100399266 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(306..306 src has 117..306) ==== 4779+0+0 (unknown 0 0 0) 0x7feec4078440 con 0x7feed009c4a0 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 shutdown_connections 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 --2- 10.20.193.78:0/100399266 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feed009c4a0 0x7feed0134410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 >> 10.20.193.78:0/100399266 conn(0x7feed00a6620 msgr2=0x7feed00a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 shutdown_connections 2026-02-17T05:54:31.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.253+0000 7feed66f2640 1 -- 10.20.193.78:0/100399266 wait complete. 2026-02-17T05:54:31.264 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:54:31.264 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:54:31.264 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:54:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.332+0000 7f1a63d85640 10 monclient: get_monmap_and_config 2026-02-17T05:54:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.332+0000 7f1a63d85640 10 monclient: build_initial_monmap 2026-02-17T05:54:31.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.332+0000 7f1a63d85640 10 monclient: monmap: 2026-02-17T05:54:31.332 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:31.332 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.332 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:31.334004+0000 2026-02-17T05:54:31.332 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:31.334004+0000 2026-02-17T05:54:31.332 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:31.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.333 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:31.333 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:31.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:31.333 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:31.333 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:31.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 1 Processor -- start 2026-02-17T05:54:31.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 1 -- start start 2026-02-17T05:54:31.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient: init 2026-02-17T05:54:31.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:31.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2026-02-17T05:54:31.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1a5c09ea90 0x7f1a5c0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient(hunting): picked mon.noname-a con 0x7f1a5c09ea90 addr v2:10.20.193.78:3300/0 2026-02-17T05:54:31.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1a5c09ee70 addr v1:10.20.193.78:6789/0 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c13d640 0x7f1a5c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient(hunting): picked mon.noname-b con 0x7f1a5c13d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a5c0a0a20 con 0x7f1a5c09ee70 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f1a5c0a0d00 con 0x7f1a5c09ea90 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5c0a0460 con 0x7f1a5c13d640 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a63d85640 10 monclient(hunting): authenticate will time out at 1116.887677s 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1a5c09ea90 0x7f1a5c0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a612f9640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f1a5c09ee70 0x7f1a5c09f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57240/0 (socket says 10.20.193.78:57240) 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1a5c09ea90 0x7f1a5c0a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39574/0 (socket says 10.20.193.78:39574) 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a612f9640 1 -- 10.20.193.78:0/3033223247 learned_addr learned my addr 10.20.193.78:0/3033223247 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a622fb640 1 --2- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c13d640 0x7f1a5c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient(hunting): get_auth_request con 0x7f1a5c09ea90 auth_method 0 2026-02-17T05:54:31.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a622fb640 10 monclient(hunting): get_auth_request con 0x7f1a5c13d640 auth_method 0 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a622fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a622fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a622fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a622fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a622fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a622fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient(hunting): handle_auth_done global_id 104190 payload 274 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 1 -- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c13d640 msgr2=0x7f1a5c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 1 --2- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c13d640 0x7f1a5c13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 1 -- 10.20.193.78:0/3033223247 >> v1:10.20.193.78:6789/0 conn(0x7f1a5c09ee70 legacy=0x7f1a5c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient: _finish_hunting 0 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 1 monclient: found mon.noname-a 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.333+0000 7f1a61afa640 1 -- 10.20.193.78:0/3033223247 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c0a0740 con 0x7f1a5c09ea90 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a61afa640 1 -- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ea90 msgr2=0x7f1a5c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a61afa640 1 --2- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ea90 0x7f1a5c0a13d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098a0 tx=0x7f1a4c02ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb584230f177066 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 1 -- 10.20.193.78:0/3033223247 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c03c070 con 0x7f1a5c09ea90 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: dump: 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: _finish_auth 0 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: _check_auth_tickets 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 1 -- 10.20.193.78:0/3033223247 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a4c02fc00 con 0x7f1a5c09ea90 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 1 -- 10.20.193.78:0/3033223247 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c037440 con 0x7f1a5c09ea90 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 5 monclient: authenticate success, global_id 104190 2026-02-17T05:54:31.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 4 set_mon_vals no callback set 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a60af8640 10 monclient: dump: 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 10 monclient: shutdown 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 1 -- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ea90 msgr2=0x7f1a5c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 1 --2- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ea90 0x7f1a5c0a13d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098a0 tx=0x7f1a4c02ed80 comp rx=0 tx=0).stop 2026-02-17T05:54:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 1 -- 10.20.193.78:0/3033223247 shutdown_connections 2026-02-17T05:54:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 1 --2- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c13d640 0x7f1a5c13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 1 --2- 10.20.193.78:0/3033223247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ea90 0x7f1a5c0a13d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 1 -- 10.20.193.78:0/3033223247 >> 10.20.193.78:0/3033223247 conn(0x7f1a5c0a7520 msgr2=0x7f1a5c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 1 -- 10.20.193.78:0/3033223247 shutdown_connections 2026-02-17T05:54:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.336+0000 7f1a63d85640 1 -- 10.20.193.78:0/3033223247 wait complete. 2026-02-17T05:54:31.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient: build_initial_monmap 2026-02-17T05:54:31.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient: monmap: 2026-02-17T05:54:31.347 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 1 Processor -- start 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 1 -- start start 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient: init 2026-02-17T05:54:31.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c09ea90 0x7f1a5c1cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient(hunting): picked mon.noname-c con 0x7f1a5c09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ee70 0x7f1a5c0736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient(hunting): picked mon.noname-a con 0x7f1a5c09ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c13d640 0x7f1a5c073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient(hunting): picked mon.noname-b con 0x7f1a5c13d640 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c0a6820 con 0x7f1a5c09ee70 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c0a66a0 con 0x7f1a5c13d640 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5c13ff70 con 0x7f1a5c09ea90 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a63d85640 10 monclient(hunting): authenticate will time out at 1116.891957s 2026-02-17T05:54:31.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.337+0000 7f1a61afa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c09ea90 0x7f1a5c1cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ee70 0x7f1a5c0736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c09ea90 0x7f1a5c1cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44502/0 (socket says 10.20.193.78:44502) 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 1 -- 10.20.193.78:0/3756409973 learned_addr learned my addr 10.20.193.78:0/3756409973 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient(hunting): get_auth_request con 0x7f1a5c09ee70 auth_method 0 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 10 monclient(hunting): get_auth_request con 0x7f1a5c09ea90 auth_method 0 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient(hunting): handle_auth_done global_id 104193 payload 824 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c09ea90 msgr2=0x7f1a5c1cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1a5c09ea90 0x7f1a5c1cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c13d640 msgr2=0x7f1a5c073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c13d640 0x7f1a5c073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient: _finish_hunting 0 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 1 monclient: found mon.noname-a 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.338+0000 7f1a612f9640 1 -- 10.20.193.78:0/3756409973 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c06f6e0 con 0x7f1a5c09ee70 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a612f9640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ee70 msgr2=0x7f1a5c0736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a612f9640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ee70 0x7f1a5c0736d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c4a0 tx=0x7f1a5000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50007070 con 0x7f1a5c09ee70 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: dump: 2026-02-17T05:54:31.351 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a50007210 con 0x7f1a5c09ee70 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 5 monclient: authenticate success, global_id 104193 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 10 monclient: _renew_subs 2026-02-17T05:54:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5c06fe50 con 0x7f1a5c09ee70 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50007600 con 0x7f1a5c09ee70 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4bfff640 4 set_mon_vals no callback set 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a4a7fc640 10 monclient: dump: 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 10 monclient: _renew_subs 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5c06f990 con 0x7f1a5c09ee70 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3b90 0x7f1a5c139620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.340+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c070b30 con 0x7f1a5c0a3b90 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3b90 0x7f1a5c139620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 10 monclient: get_auth_request con 0x7f1a5c0a3b90 auth_method 0 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1a5001b400 con 0x7f1a5c09ee70 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a4a7fc640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a2c03deb0 0x7f1a2c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 10 monclient(hunting): handle_auth_done global_id 114106 payload 824 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a61afa640 1 -- 10.20.193.78:0/3756409973 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a5c0a1c10 con 0x7f1a5c0a3b90 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(306..306 src has 117..306) ==== 4779+0+0 (secure 0 0 0) 0x7f1a50052480 con 0x7f1a5c09ee70 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a622fb640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a2c03deb0 0x7f1a2c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.341+0000 7f1a622fb640 10 monclient: get_auth_request con 0x7f1a2c03deb0 auth_method 0 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.343+0000 7f1a61afa640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3b90 msgr2=0x7f1a5c139620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.343+0000 7f1a61afa640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3b90 0x7f1a5c139620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098a0 tx=0x7f1a4c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.343+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c048070 con 0x7f1a5c0a3b90 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.343+0000 7f1a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.343+0000 7f1a4a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.343+0000 7f1a4a7fc640 10 monclient: dump: 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.355 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.344+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f1a4c04d450 con 0x7f1a5c0a3b90 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.344+0000 7f1a4a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.344+0000 7f1a4a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.344+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3b90 msgr2=0x7f1a5c139620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.344+0000 7f1a4a7fc640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3b90 0x7f1a5c139620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098a0 tx=0x7f1a4c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.344+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3b90 msgr2=0x7f1a5c139620 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.344+0000 7f1a4a7fc640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3b90 0x7f1a5c139620 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098a0 tx=0x7f1a4c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.344+0000 7f1a622fb640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a2c03deb0 0x7f1a2c040370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0712d0 tx=0x7f1a5800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a63d85640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a63d85640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3f70 0x7f1a5c1da420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a63d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c13fcd0 con 0x7f1a5c0a3f70 2026-02-17T05:54:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3f70 0x7f1a5c1da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 10 monclient: get_auth_request con 0x7f1a5c0a3f70 auth_method 0 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 10 monclient(hunting): handle_auth_done global_id 114109 payload 824 2026-02-17T05:54:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.347+0000 7f1a61afa640 1 -- 10.20.193.78:0/3756409973 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1a5c070950 con 0x7f1a5c0a3f70 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a61afa640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3f70 msgr2=0x7f1a5c1da420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a61afa640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3f70 0x7f1a5c1da420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c06c650 tx=0x7f1a4c009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c03c040 con 0x7f1a5c0a3f70 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 10 monclient: dump: 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.358 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f1a4c041a60 con 0x7f1a5c0a3f70 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3f70 msgr2=0x7f1a5c1da420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3f70 0x7f1a5c1da420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c06c650 tx=0x7f1a4c009290 comp rx=0 tx=0).stop 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3f70 msgr2=0x7f1a5c1da420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a4a7fc640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1a5c0a3f70 0x7f1a5c1da420 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c06c650 tx=0x7f1a4c009290 comp rx=0 tx=0).stop 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 88, 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 15431, 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.360 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:54:31.361 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:54:31.362 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:31.363 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.364 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.365 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:31.366 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:54:31.367 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:54:31.368 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:54:31.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a2c03deb0 msgr2=0x7f1a2c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1a2c03deb0 0x7f1a2c040370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0712d0 tx=0x7f1a5800b040 comp rx=0 tx=0).stop 2026-02-17T05:54:31.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 10 monclient: shutdown 2026-02-17T05:54:31.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ee70 msgr2=0x7f1a5c0736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ee70 0x7f1a5c0736d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c4a0 tx=0x7f1a5000c970 comp rx=0 tx=0).stop 2026-02-17T05:54:31.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a622fb640 1 -- 10.20.193.78:0/3756409973 reap_dead start 2026-02-17T05:54:31.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 shutdown_connections 2026-02-17T05:54:31.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 --2- 10.20.193.78:0/3756409973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1a5c09ee70 0x7f1a5c0736d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 >> 10.20.193.78:0/3756409973 conn(0x7f1a5c0a7520 msgr2=0x7f1a5c13c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 shutdown_connections 2026-02-17T05:54:31.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.350+0000 7f1a63d85640 1 -- 10.20.193.78:0/3756409973 wait complete. 2026-02-17T05:54:31.370 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:54:31.370 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:54:31.370 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.437+0000 7f7c39c91640 10 monclient: get_monmap_and_config 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.437+0000 7f7c39c91640 10 monclient: build_initial_monmap 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.437+0000 7f7c39c91640 10 monclient: monmap: 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:31.438606+0000 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:31.438606+0000 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.436 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.437+0000 7f7c39c91640 1 Processor -- start 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.437+0000 7f7c39c91640 1 -- start start 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.437+0000 7f7c39c91640 10 monclient: init 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:31.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7c3409ea90 addr v1:10.20.193.78:6789/0 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3409ee70 0x7f7c3409f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 10 monclient(hunting): picked mon.noname-b con 0x7f7c3409ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7c3413d640 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c340a0a20 con 0x7f7c3409ea90 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c340a0460 con 0x7f7c3413d640 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7c340a0d00 con 0x7f7c3409ee70 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c39c91640 10 monclient(hunting): authenticate will time out at 1116.992285s 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c337fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f7c3409ea90 0x7f7c340a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57256/0 (socket says 10.20.193.78:57256) 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3409ee70 0x7f7c3409f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c337fe640 1 -- 10.20.193.78:0/2367617934 learned_addr learned my addr 10.20.193.78:0/2367617934 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient(hunting): get_auth_request con 0x7f7c3409ee70 auth_method 0 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient(hunting): handle_auth_done global_id 74477 payload 274 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 1 -- 10.20.193.78:0/2367617934 >> v1:10.20.193.115:6789/0 conn(0x7f7c3413d640 legacy=0x7f7c3413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 1 -- 10.20.193.78:0/2367617934 >> v1:10.20.193.78:6789/0 conn(0x7f7c3409ea90 legacy=0x7f7c340a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 1 monclient: found mon.noname-b 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:31.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.438+0000 7f7c32ffd640 1 -- 10.20.193.78:0/2367617934 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c340a0740 con 0x7f7c3409ee70 2026-02-17T05:54:31.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.440+0000 7f7c32ffd640 1 -- 10.20.193.78:0/2367617934 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3409ee70 msgr2=0x7f7c3409f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.440+0000 7f7c32ffd640 1 --2- 10.20.193.78:0/2367617934 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3409ee70 0x7f7c3409f290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7c240098d0 tx=0x7f7c2402edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b33a502aa60548d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 1 -- 10.20.193.78:0/2367617934 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2403c070 con 0x7f7c3409ee70 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: dump: 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 1 -- 10.20.193.78:0/2367617934 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2402fd20 con 0x7f7c3409ee70 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 5 monclient: authenticate success, global_id 74477 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 1 -- 10.20.193.78:0/2367617934 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24037660 con 0x7f7c3409ee70 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 4 set_mon_vals no callback set 2026-02-17T05:54:31.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c327fc640 10 monclient: dump: 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.441 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 10 monclient: shutdown 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 1 -- 10.20.193.78:0/2367617934 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3409ee70 msgr2=0x7f7c3409f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 1 --2- 10.20.193.78:0/2367617934 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3409ee70 0x7f7c3409f290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7c240098d0 tx=0x7f7c2402edc0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 1 -- 10.20.193.78:0/2367617934 shutdown_connections 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 1 --2- 10.20.193.78:0/2367617934 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3409ee70 0x7f7c3409f290 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 1 -- 10.20.193.78:0/2367617934 >> 10.20.193.78:0/2367617934 conn(0x7f7c340a7520 msgr2=0x7f7c340a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 1 -- 10.20.193.78:0/2367617934 shutdown_connections 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 1 -- 10.20.193.78:0/2367617934 wait complete. 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 10 monclient: build_initial_monmap 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.441+0000 7f7c39c91640 10 monclient: monmap: 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 1 Processor -- start 2026-02-17T05:54:31.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 1 -- start start 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient: init 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c3409ea90 0x7f7c341cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient(hunting): picked mon.noname-b con 0x7f7c3409ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c3409ee70 0x7f7c34073470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient(hunting): picked mon.noname-a con 0x7f7c3409ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3413d640 0x7f7c340739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient(hunting): picked mon.noname-c con 0x7f7c3413d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340a6820 con 0x7f7c3409ee70 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340a66a0 con 0x7f7c3409ea90 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7c34141150 con 0x7f7c3413d640 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c39c91640 10 monclient(hunting): authenticate will time out at 1116.996344s 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c3409ee70 0x7f7c34073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c3409ee70 0x7f7c34073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39578/0 (socket says 10.20.193.78:39578) 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c33fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3413d640 0x7f7c340739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 -- 10.20.193.78:0/1166406541 learned_addr learned my addr 10.20.193.78:0/1166406541 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient(hunting): get_auth_request con 0x7f7c3409ee70 auth_method 0 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c33fff640 10 monclient(hunting): get_auth_request con 0x7f7c3413d640 auth_method 0 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c33fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c337fe640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c3409ea90 0x7f7c341cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient(hunting): handle_auth_done global_id 104196 payload 824 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 -- 10.20.193.78:0/1166406541 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3413d640 msgr2=0x7f7c340739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3413d640 0x7f7c340739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 -- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c3409ea90 msgr2=0x7f7c341cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c3409ea90 0x7f7c341cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 monclient: found mon.noname-a 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c32ffd640 1 -- 10.20.193.78:0/1166406541 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3406ff20 con 0x7f7c3409ee70 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c337fe640 10 monclient: get_auth_request con 0x7f7c3409ea90 auth_method 0 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.442+0000 7f7c337fe640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c3409ea90 0x7f7c341cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c32ffd640 1 -- 10.20.193.78:0/1166406541 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c3409ee70 msgr2=0x7f7c34073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c32ffd640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c3409ee70 0x7f7c34073470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009820 tx=0x7f7c240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 1 -- 10.20.193.78:0/1166406541 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24046070 con 0x7f7c3409ee70 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: dump: 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.444 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: _finish_auth 0 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 1 -- 10.20.193.78:0/1166406541 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c24041870 con 0x7f7c3409ee70 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c39c91640 5 monclient: authenticate success, global_id 104196 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c39c91640 10 monclient: _renew_subs 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c39c91640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c340a1c10 con 0x7f7c3409ee70 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 1 -- 10.20.193.78:0/1166406541 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2403c040 con 0x7f7c3409ee70 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c0ffff640 10 monclient: dump: 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c31ffb640 4 set_mon_vals no callback set 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c39c91640 10 monclient: _renew_subs 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c39c91640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.444+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3406f680 con 0x7f7c3409ee70 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.445+0000 7f7c39c91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.445+0000 7f7c39c91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.445+0000 7f7c39c91640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.445+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3409f290 con 0x7f7c3409ee70 2026-02-17T05:54:31.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.445+0000 7f7c0ffff640 1 -- 10.20.193.78:0/1166406541 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7c24041c60 con 0x7f7c3409ee70 2026-02-17T05:54:31.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.445+0000 7f7c0ffff640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7c0403deb0 0x7f7c04040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:31.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.445+0000 7f7c0ffff640 1 -- 10.20.193.78:0/1166406541 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(306..306 src has 117..306) ==== 4779+0+0 (secure 0 0 0) 0x7f7c24078060 con 0x7f7c3409ee70 2026-02-17T05:54:31.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.446+0000 7f7c337fe640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7c0403deb0 0x7f7c04040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:31.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.446+0000 7f7c337fe640 10 monclient: get_auth_request con 0x7f7c0403deb0 auth_method 0 2026-02-17T05:54:31.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.448+0000 7f7c0ffff640 1 -- 10.20.193.78:0/1166406541 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7c24040c70 con 0x7f7c3409ee70 2026-02-17T05:54:31.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.448+0000 7f7c0ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:31.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.448+0000 7f7c0ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:31.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.448+0000 7f7c337fe640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7c0403deb0 0x7f7c04040370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c200052b0 tx=0x7f7c20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:31.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.562+0000 7f7c39c91640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:54:31.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.562+0000 7f7c39c91640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:54:31.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.562+0000 7f7c39c91640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:31.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.562+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f7c340744d0 con 0x7f7c3409ee70 2026-02-17T05:54:31.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.562+0000 7f7c0ffff640 1 -- 10.20.193.78:0/1166406541 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f7c24037e10 con 0x7f7c3409ee70 2026-02-17T05:54:31.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.562+0000 7f7c0ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:54:31.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.562+0000 7f7c0ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7c0403deb0 msgr2=0x7f7c04040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7c0403deb0 0x7f7c04040370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c200052b0 tx=0x7f7c20009040 comp rx=0 tx=0).stop 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 10 monclient: shutdown 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c3409ee70 msgr2=0x7f7c34073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c3409ee70 0x7f7c34073470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009820 tx=0x7f7c240047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 shutdown_connections 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c3413d640 0x7f7c340739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c3409ee70 0x7f7c34073470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7c0403deb0 0x7f7c04040370 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 --2- 10.20.193.78:0/1166406541 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c3409ea90 0x7f7c341cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:31.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 >> 10.20.193.78:0/1166406541 conn(0x7f7c340a7520 msgr2=0x7f7c340a3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:31.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 shutdown_connections 2026-02-17T05:54:31.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:31.563+0000 7f7c39c91640 1 -- 10.20.193.78:0/1166406541 wait complete. 2026-02-17T05:54:31.571 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:54:31.858 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.858+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.858 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.858+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=17,config=1,health=12,kv=12,logm=58} crc {auth=1695735992,config=1228700967,health=2677522725,kv=1748417377,logm=97147980}) 2026-02-17T05:54:31.858 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.858+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=18,config=2,health=11,kv=12,logm=57} crc {auth=2289712568,config=692619927,health=2975361555,kv=449989585,logm=44583436}) 2026-02-17T05:54:31.858 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.858+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.858 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.858+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=17,config=1,health=12,kv=12,logm=58} crc {auth=1695735992,config=1228700967,health=2677522725,kv=1748417377,logm=97147980}) 2026-02-17T05:54:31.859 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.858+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=19,config=2,health=11,kv=11,logm=57} crc {auth=2640902412,config=692619927,health=3428715403,kv=449989585,logm=2143717768}) 2026-02-17T05:54:31.860 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.861+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.860 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.861+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1990766000}) 2026-02-17T05:54:31.861 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.861+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=15848428}) 2026-02-17T05:54:31.861 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.861+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.861 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.861+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1990766000}) 2026-02-17T05:54:31.861 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.861+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2845905551}) 2026-02-17T05:54:31.862 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.863+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.862 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.863+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3335961245}) 2026-02-17T05:54:31.862 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.863+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3098338567}) 2026-02-17T05:54:31.863 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.863+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.863 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.863+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3335961245}) 2026-02-17T05:54:31.863 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.863+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1947216436}) 2026-02-17T05:54:31.863 INFO:tasks.rados.rados.0.trial115.stdout:1446: done (4 left) 2026-02-17T05:54:31.864 INFO:tasks.rados.rados.0.trial115.stdout:1447: done (3 left) 2026-02-17T05:54:31.864 INFO:tasks.rados.rados.0.trial115.stdout:1448: done (2 left) 2026-02-17T05:54:31.864 INFO:tasks.rados.rados.0.trial115.stdout:1449: done (1 left) 2026-02-17T05:54:31.864 INFO:tasks.rados.rados.0.trial115.stdout:1450: done (0 left) 2026-02-17T05:54:31.864 INFO:tasks.rados.rados.0.trial115.stdout:1451: read oid 10 snap -1 2026-02-17T05:54:31.864 INFO:tasks.rados.rados.0.trial115.stdout:1451: expect (ObjNum 453 snap 112 seq_num 453) 2026-02-17T05:54:31.864 INFO:tasks.rados.rados.0.trial115.stdout:1452: delete oid 27 current snap is 142 2026-02-17T05:54:31.865 INFO:tasks.rados.rados.0.trial115.stdout:1452: done (1 left) 2026-02-17T05:54:31.865 INFO:tasks.rados.rados.0.trial115.stdout:1453: write (excl) oid 11 current snap is 142 2026-02-17T05:54:31.865 INFO:tasks.rados.rados.0.trial115.stdout:1453: seq_num 548 ranges {410236=413868,1234250=729187,2715800=423063} 2026-02-17T05:54:31.869 INFO:tasks.rados.rados.0.trial115.stdout:1453: writing trial11545420-11 from 410236 to 824104 tid 1 2026-02-17T05:54:31.873 INFO:tasks.rados.rados.0.trial115.stdout:1453: writing trial11545420-11 from 1234250 to 1963437 tid 2 2026-02-17T05:54:31.878 INFO:tasks.rados.rados.0.trial115.stdout:1453: writing trial11545420-11 from 2715800 to 3138863 tid 3 2026-02-17T05:54:31.878 INFO:tasks.rados.rados.0.trial115.stdout:1454: delete oid 28 current snap is 142 2026-02-17T05:54:31.878 INFO:tasks.rados.rados.0.trial115.stdout:1451: expect (ObjNum 453 snap 112 seq_num 453) 2026-02-17T05:54:31.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.924+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.924+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167208917}) 2026-02-17T05:54:31.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.924+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=515372284}) 2026-02-17T05:54:31.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.924+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.924+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167208917}) 2026-02-17T05:54:31.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.924+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2629752624}) 2026-02-17T05:54:31.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2182922344}) 2026-02-17T05:54:31.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2512919976}) 2026-02-17T05:54:31.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2182922344}) 2026-02-17T05:54:31.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.925+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=637799389}) 2026-02-17T05:54:31.963 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.964+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.963 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.964+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=470140739}) 2026-02-17T05:54:31.963 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.964+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624696153}) 2026-02-17T05:54:31.964 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.964+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:31.964 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.964+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=470140739}) 2026-02-17T05:54:31.964 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:31.964+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3657086258}) 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: finishing write tid 1 to trial11545420-11 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1451: done (2 left) 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1454: done (1 left) 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1455: rollback oid 35 current snap is 142 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 35 to 136 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: finishing write tid 2 to trial11545420-11 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: finishing write tid 3 to trial11545420-11 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: finishing write tid 4 to trial11545420-11 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: finishing write tid 5 to trial11545420-11 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: oid 11 updating version 0 to 698 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: oid 11 version 698 is already newer than 696 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: oid 11 version 698 is already newer than 695 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: oid 11 version 698 is already newer than 697 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 698 (ObjNum 548 snap 142 seq_num 548) dirty exists 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: left oid 11 (ObjNum 548 snap 142 seq_num 548) 2026-02-17T05:54:32.028 INFO:tasks.rados.rados.0.trial115.stdout:1453: done (1 left) 2026-02-17T05:54:32.029 INFO:tasks.rados.rados.0.trial115.stdout:1456: write oid 26 current snap is 142 2026-02-17T05:54:32.029 INFO:tasks.rados.rados.0.trial115.stdout:1456: seq_num 550 ranges {443850=564366,1634861=732856,2770646=473204} 2026-02-17T05:54:32.030 INFO:tasks.rados.rados.0.trial115.stdout:1456: writing trial11545420-26 from 443850 to 1008216 tid 1 2026-02-17T05:54:32.035 INFO:tasks.rados.rados.0.trial115.stdout:1456: writing trial11545420-26 from 1634861 to 2367717 tid 2 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:1456: writing trial11545420-26 from 2770646 to 3243850 tid 3 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:1457: write oid 17 current snap is 142 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:1455: finishing rollback tid 0 to trial11545420-35 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:1455: finishing rollback tid 1 to trial11545420-35 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:1455: finishing rollback tid 2 to trial11545420-35 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 776 (ObjNum 534 snap 136 seq_num 534) dirty exists 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:1456: finishing write tid 1 to trial11545420-26 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:1456: finishing write tid 2 to trial11545420-26 2026-02-17T05:54:32.039 INFO:tasks.rados.rados.0.trial115.stdout:1457: seq_num 551 ranges {460657=639615,1693343=476514,2681716=578941} 2026-02-17T05:54:32.043 INFO:tasks.rados.rados.0.trial115.stdout:1457: writing trial11545420-17 from 460657 to 1100272 tid 1 2026-02-17T05:54:32.047 INFO:tasks.rados.rados.0.trial115.stdout:1457: writing trial11545420-17 from 1693343 to 2169857 tid 2 2026-02-17T05:54:32.051 INFO:tasks.rados.rados.0.trial115.stdout:1457: writing trial11545420-17 from 2681716 to 3260657 tid 3 2026-02-17T05:54:32.051 INFO:tasks.rados.rados.0.trial115.stdout:1455: done (2 left) 2026-02-17T05:54:32.051 INFO:tasks.rados.rados.0.trial115.stdout:1458: rollback oid 43 current snap is 142 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 137 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1456: finishing write tid 3 to trial11545420-26 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1456: finishing write tid 4 to trial11545420-26 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1456: finishing write tid 5 to trial11545420-26 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1456: oid 26 updating version 0 to 203 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1456: oid 26 version 203 is already newer than 200 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1456: oid 26 version 203 is already newer than 201 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1456: oid 26 version 203 is already newer than 202 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 203 (ObjNum 550 snap 142 seq_num 550) dirty exists 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1456: left oid 26 (ObjNum 550 snap 142 seq_num 550) 2026-02-17T05:54:32.052 INFO:tasks.rados.rados.0.trial115.stdout:1457: finishing write tid 1 to trial11545420-17 2026-02-17T05:54:32.053 INFO:tasks.rados.rados.0.trial115.stdout:1457: finishing write tid 2 to trial11545420-17 2026-02-17T05:54:32.053 INFO:tasks.rados.rados.0.trial115.stdout:1456: done (2 left) 2026-02-17T05:54:32.053 INFO:tasks.rados.rados.0.trial115.stdout:1459: write (excl) oid 31 current snap is 142 2026-02-17T05:54:32.053 INFO:tasks.rados.rados.0.trial115.stdout:1459: seq_num 552 ranges {477464=714864,1751825=736525,3225492=51972} 2026-02-17T05:54:32.055 INFO:tasks.rados.rados.0.trial115.stdout:1459: writing trial11545420-31 from 477464 to 1192328 tid 1 2026-02-17T05:54:32.060 INFO:tasks.rados.rados.0.trial115.stdout:1459: writing trial11545420-31 from 1751825 to 2488350 tid 2 2026-02-17T05:54:32.063 INFO:tasks.rados.rados.0.trial115.stdout:1459: writing trial11545420-31 from 3225492 to 3277464 tid 3 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1460: copy_from oid 42 from oid 20 current snap is 142 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1458: finishing rollback tid 0 to trial11545420-43 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1458: finishing rollback tid 1 to trial11545420-43 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1458: finishing rollback tid 2 to trial11545420-43 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 548 (ObjNum 540 snap 137 seq_num 540) dirty exists 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1457: finishing write tid 3 to trial11545420-17 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1457: finishing write tid 4 to trial11545420-17 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1457: finishing write tid 5 to trial11545420-17 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1457: oid 17 updating version 0 to 285 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1457: oid 17 version 285 is already newer than 282 2026-02-17T05:54:32.064 INFO:tasks.rados.rados.0.trial115.stdout:1457: oid 17 version 285 is already newer than 284 2026-02-17T05:54:32.065 INFO:tasks.rados.rados.0.trial115.stdout:1457: oid 17 version 285 is already newer than 283 2026-02-17T05:54:32.065 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 285 (ObjNum 551 snap 142 seq_num 551) dirty exists 2026-02-17T05:54:32.065 INFO:tasks.rados.rados.0.trial115.stdout:1457: left oid 17 (ObjNum 551 snap 142 seq_num 551) 2026-02-17T05:54:32.065 INFO:tasks.rados.rados.0.trial115.stdout:1457: done (3 left) 2026-02-17T05:54:32.065 INFO:tasks.rados.rados.0.trial115.stdout:1458: done (2 left) 2026-02-17T05:54:32.065 INFO:tasks.rados.rados.0.trial115.stdout:1461: write (excl) oid 6 current snap is 142 2026-02-17T05:54:32.065 INFO:tasks.rados.rados.0.trial115.stdout:1461: seq_num 553 ranges {494271=790113,1693954=480183,2620209=512129} 2026-02-17T05:54:32.068 INFO:tasks.rados.rados.0.trial115.stdout:1461: writing trial11545420-6 from 494271 to 1284384 tid 1 2026-02-17T05:54:32.071 INFO:tasks.rados.rados.0.trial115.stdout:1461: writing trial11545420-6 from 1693954 to 2174137 tid 2 2026-02-17T05:54:32.075 INFO:tasks.rados.rados.0.trial115.stdout:1461: writing trial11545420-6 from 2620209 to 3132338 tid 3 2026-02-17T05:54:32.075 INFO:tasks.rados.rados.0.trial115.stdout:1462: snap_create 2026-02-17T05:54:32.075 INFO:tasks.rados.rados.0.trial115.stdout:1459: finishing write tid 1 to trial11545420-31 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: finishing write tid 2 to trial11545420-31 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: finishing write tid 3 to trial11545420-31 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: finishing write tid 4 to trial11545420-31 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: finishing write tid 5 to trial11545420-31 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: oid 31 updating version 0 to 760 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: oid 31 updating version 760 to 763 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: oid 31 version 763 is already newer than 761 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: oid 31 version 763 is already newer than 762 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 763 (ObjNum 552 snap 142 seq_num 552) dirty exists 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1459: left oid 31 (ObjNum 552 snap 142 seq_num 552) 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1460: finishing copy_from racing read to trial11545420-42 2026-02-17T05:54:32.076 INFO:tasks.rados.rados.0.trial115.stdout:1461: finishing write tid 1 to trial11545420-6 2026-02-17T05:54:32.077 INFO:tasks.rados.rados.0.trial115.stdout:1460: finishing copy_from to trial11545420-42 2026-02-17T05:54:32.077 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 524 (ObjNum 536 snap 136 seq_num 536) dirty exists 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: finishing write tid 2 to trial11545420-6 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: finishing write tid 3 to trial11545420-6 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: finishing write tid 4 to trial11545420-6 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: finishing write tid 5 to trial11545420-6 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: oid 6 updating version 0 to 438 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: oid 6 version 438 is already newer than 436 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: oid 6 version 438 is already newer than 435 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: oid 6 version 438 is already newer than 437 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 438 (ObjNum 553 snap 142 seq_num 553) dirty exists 2026-02-17T05:54:32.078 INFO:tasks.rados.rados.0.trial115.stdout:1461: left oid 6 (ObjNum 553 snap 142 seq_num 553) 2026-02-17T05:54:32.309 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.310+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.309 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.310+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=652779222}) 2026-02-17T05:54:32.309 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.310+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=784736456}) 2026-02-17T05:54:32.309 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.310+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.309 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.310+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=652779222}) 2026-02-17T05:54:32.309 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.310+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2659909643}) 2026-02-17T05:54:32.488 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.489 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2988198882}) 2026-02-17T05:54:32.489 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351302945}) 2026-02-17T05:54:32.489 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.489 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2988198882}) 2026-02-17T05:54:32.489 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4217165151}) 2026-02-17T05:54:32.489 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.489 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792416048}) 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307925318}) 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792416048}) 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=907134045}) 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=755722333}) 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=369868954}) 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=755722333}) 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.479+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1765841223}) 2026-02-17T05:54:32.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907548017}) 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2632252917}) 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907548017}) 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1082872516}) 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2702571001}) 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1870355748}) 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2702571001}) 2026-02-17T05:54:32.491 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=252469450}) 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3790719853}) 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3666165506}) 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3790719853}) 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.480+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3710328998}) 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854721230}) 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=381626271}) 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.492 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1854721230}) 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=772656297}) 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3795473592}) 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2703225217}) 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3795473592}) 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4221879903}) 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4136557537}) 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2290858257}) 2026-02-17T05:54:32.493 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4136557537}) 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.481+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3236382122}) 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770138037}) 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2658907675}) 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770138037}) 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2815648408}) 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775925786}) 2026-02-17T05:54:32.494 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1411983920}) 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775925786}) 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3573850176}) 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2307698814}) 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2776828878}) 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2307698814}) 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.482+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2981337808}) 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.495 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1159479713}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2758816730}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1159479713}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=601615369}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3580570029}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1180201080}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3580570029}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3300273503}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32908489}) 2026-02-17T05:54:32.496 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1300394144}) 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32908489}) 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.483+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2043442326}) 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=58,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,osd_metadata=2} crc {logm=4071132774,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2645225359,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2773149886}) 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=56,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=6} crc {logm=2860201281,mdsmap=1160443688,mgr=3987721425,mgr_metadata=1258482571,mgrstat=2645225359,mon_config_key=4028116958,monmap=1784110042,osd_metadata=2997046373}) 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=58,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,osd_metadata=2} crc {logm=4071132774,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2645225359,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2773149886}) 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=5} crc {logm=3048910971,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2645225359,mon_config_key=2399717749,monmap=1784110042,osd_metadata=278982410}) 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=4,osd_pg_creating=1,osd_snap=49,osdmap=46} crc {osd_metadata=1498908973,osd_pg_creating=4047457720,osd_snap=380352526,osdmap=2322365042}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=4,osd_snap=53,osdmap=43} crc {osd_metadata=1498908973,osd_snap=4117617289,osdmap=308401162}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=4,osd_pg_creating=1,osd_snap=49,osdmap=46} crc {osd_metadata=1498908973,osd_pg_creating=4047457720,osd_snap=380352526,osdmap=2322365042}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.484+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=4,osd_pg_creating=1,osd_snap=52,osdmap=43} crc {osd_metadata=1498908973,osd_pg_creating=4047457720,osd_snap=1048418615,osdmap=1003292940}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.485+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.485+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1483357626}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.485+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=525596254}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.485+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.485+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1483357626}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.485+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3519018118}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.486+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.486+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4025879937}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.486+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=391747563}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.486+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.486+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4025879937}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.486+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2624394361}) 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.488+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.488+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=40} crc {osdmap=2414238922}) 2026-02-17T05:54:32.499 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.488+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=37} crc {osdmap=4280213647}) 2026-02-17T05:54:32.499 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.488+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:54:32.499 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.488+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=40} crc {osdmap=2414238922}) 2026-02-17T05:54:32.499 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:54:32.488+0000 7fdc70276640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=38} crc {osdmap=3483762768}) 2026-02-17T05:54:32.572 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:54:32.572 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:54:32.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.640+0000 7f38fdafb640 10 monclient: get_monmap_and_config 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.640+0000 7f38fdafb640 10 monclient: build_initial_monmap 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.640+0000 7f38fdafb640 10 monclient: monmap: 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:32.641683+0000 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:32.641683+0000 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.642 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.640+0000 7f38fdafb640 1 Processor -- start 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.640+0000 7f38fdafb640 1 -- start start 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.640+0000 7f38fdafb640 10 monclient: init 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T05:54:32.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f38f806c890 0x7f38f806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient(hunting): picked mon.noname-c con 0x7f38f806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:32.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f38f809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:54:32.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f8135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient(hunting): picked mon.noname-b con 0x7f38f806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:32.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f806daf0 con 0x7f38f809c4a0 2026-02-17T05:54:32.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f38f80a0880 con 0x7f38f806c890 2026-02-17T05:54:32.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f38f809cc80 con 0x7f38f806d1b0 2026-02-17T05:54:32.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:32.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38fdafb640 10 monclient(hunting): authenticate will time out at 1118.195257s 2026-02-17T05:54:32.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f6ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f38f809c4a0 0x7f38f8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57266/0 (socket says 10.20.193.78:57266) 2026-02-17T05:54:32.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f6ffd640 1 -- 10.20.193.78:0/255020893 learned_addr learned my addr 10.20.193.78:0/255020893 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:54:32.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient(hunting): get_auth_request con 0x7f38f806d1b0 auth_method 0 2026-02-17T05:54:32.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f77fe640 1 --2- 10.20.193.78:0/255020893 >> v2:10.20.193.115:3300/0 conn(0x7f38f806c890 0x7f38f806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient(hunting): handle_auth_done global_id 74483 payload 274 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 1 -- 10.20.193.78:0/255020893 >> v2:10.20.193.115:3300/0 conn(0x7f38f806c890 msgr2=0x7f38f806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 1 --2- 10.20.193.78:0/255020893 >> v2:10.20.193.115:3300/0 conn(0x7f38f806c890 0x7f38f806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 1 -- 10.20.193.78:0/255020893 >> v1:10.20.193.78:6789/0 conn(0x7f38f809c4a0 legacy=0x7f38f8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 1 monclient: found mon.noname-b 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f7fff640 1 -- 10.20.193.78:0/255020893 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f809cf60 con 0x7f38f806d1b0 2026-02-17T05:54:32.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f77fe640 10 monclient: get_auth_request con 0x7f38f806c890 auth_method 0 2026-02-17T05:54:32.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.641+0000 7f38f77fe640 1 --2- 10.20.193.78:0/255020893 >> v2:10.20.193.115:3300/0 conn(0x7f38f806c890 0x7f38f806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:54:32.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f7fff640 1 -- 10.20.193.78:0/255020893 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 msgr2=0x7f38f8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f7fff640 1 --2- 10.20.193.78:0/255020893 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f8135500 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38f806dcc0 tx=0x7f38ec02f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6867491d087d4be server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 1 -- 10.20.193.78:0/255020893 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec03c070 con 0x7f38f806d1b0 2026-02-17T05:54:32.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:32.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: dump: 2026-02-17T05:54:32.658 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.658 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.659 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.659 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.659 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.659 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.659 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.659 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.659 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.659 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:32.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:32.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:32.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 1 -- 10.20.193.78:0/255020893 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ec037640 con 0x7f38f806d1b0 2026-02-17T05:54:32.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:32.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 1 -- 10.20.193.78:0/255020893 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec037a10 con 0x7f38f806d1b0 2026-02-17T05:54:32.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:32.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38f67fc640 10 monclient: dump: 2026-02-17T05:54:32.661 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.661 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.661 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.661 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.661 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.661 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.661 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.662 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.662 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.662 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 5 monclient: authenticate success, global_id 74483 2026-02-17T05:54:32.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:32.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 4 set_mon_vals no callback set 2026-02-17T05:54:32.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 10 monclient: shutdown 2026-02-17T05:54:32.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:32.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 1 -- 10.20.193.78:0/255020893 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 msgr2=0x7f38f8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 1 --2- 10.20.193.78:0/255020893 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f8135500 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38f806dcc0 tx=0x7f38ec02f150 comp rx=0 tx=0).stop 2026-02-17T05:54:32.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 1 -- 10.20.193.78:0/255020893 shutdown_connections 2026-02-17T05:54:32.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 1 --2- 10.20.193.78:0/255020893 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f8135500 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 1 --2- 10.20.193.78:0/255020893 >> v2:10.20.193.115:3300/0 conn(0x7f38f806c890 0x7f38f806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 1 -- 10.20.193.78:0/255020893 >> 10.20.193.78:0/255020893 conn(0x7f38f80a6620 msgr2=0x7f38f80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:32.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 1 -- 10.20.193.78:0/255020893 shutdown_connections 2026-02-17T05:54:32.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.644+0000 7f38fdafb640 1 -- 10.20.193.78:0/255020893 wait complete. 2026-02-17T05:54:32.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient: build_initial_monmap 2026-02-17T05:54:32.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient: monmap: 2026-02-17T05:54:32.664 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:32.664 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:32.664 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:32.664 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:32.664 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:32.664 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.665 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:32.665 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:32.665 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:32.665 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 1 Processor -- start 2026-02-17T05:54:32.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 1 -- start start 2026-02-17T05:54:32.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient: init 2026-02-17T05:54:32.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:32.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:32.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38f806c890 0x7f38f81cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient(hunting): picked mon.noname-a con 0x7f38f806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:32.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f81cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient(hunting): picked mon.noname-c con 0x7f38f806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:32.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38f809c4a0 0x7f38f809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient(hunting): picked mon.noname-b con 0x7f38f809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:32.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38f80a01a0 con 0x7f38f806c890 2026-02-17T05:54:32.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38f80a0020 con 0x7f38f809c4a0 2026-02-17T05:54:32.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f38f809be20 con 0x7f38f806d1b0 2026-02-17T05:54:32.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:32.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38fdafb640 10 monclient(hunting): authenticate will time out at 1118.199522s 2026-02-17T05:54:32.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38f806c890 0x7f38f81cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f81cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38f806c890 0x7f38f81cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39580/0 (socket says 10.20.193.78:39580) 2026-02-17T05:54:32.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f81cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44550/0 (socket says 10.20.193.78:44550) 2026-02-17T05:54:32.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 1 -- 10.20.193.78:0/523349254 learned_addr learned my addr 10.20.193.78:0/523349254 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:32.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 10 monclient(hunting): get_auth_request con 0x7f38f806c890 auth_method 0 2026-02-17T05:54:32.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient(hunting): get_auth_request con 0x7f38f806d1b0 auth_method 0 2026-02-17T05:54:32.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient(hunting): handle_auth_done global_id 74486 payload 824 2026-02-17T05:54:32.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f7fff640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38f809c4a0 0x7f38f809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 1 -- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38f809c4a0 msgr2=0x7f38f809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38f809c4a0 0x7f38f809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 1 -- 10.20.193.78:0/523349254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38f806c890 msgr2=0x7f38f81cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38f806c890 0x7f38f81cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 1 monclient: found mon.noname-c 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f6ffd640 1 -- 10.20.193.78:0/523349254 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f809d810 con 0x7f38f806d1b0 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f7fff640 10 monclient: get_auth_request con 0x7f38f809c4a0 auth_method 0 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.645+0000 7f38f7fff640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38f809c4a0 0x7f38f809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38f6ffd640 1 -- 10.20.193.78:0/523349254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 msgr2=0x7f38f81cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38f6ffd640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f81cf890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c4a0 tx=0x7f38e800c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 1 -- 10.20.193.78:0/523349254 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8007070 con 0x7f38f806d1b0 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:32.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: dump: 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: _finish_auth 0 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:54:32.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 1 -- 10.20.193.78:0/523349254 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e8007210 con 0x7f38f806d1b0 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 1 -- 10.20.193.78:0/523349254 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8007640 con 0x7f38f806d1b0 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38fdafb640 5 monclient: authenticate success, global_id 74486 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38f5ffb640 4 set_mon_vals no callback set 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.647+0000 7f38d3fff640 10 monclient: dump: 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.673 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 10 monclient: _renew_subs 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f8125fe0 con 0x7f38f806d1b0 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 10 monclient: _renew_subs 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f8134c10 con 0x7f38f806d1b0 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:32.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f8074980 con 0x7f38f806d1b0 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.648+0000 7f38d3fff640 1 -- 10.20.193.78:0/523349254 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f38e801b400 con 0x7f38f806d1b0 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.649+0000 7f38d3fff640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38c803deb0 0x7f38c8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.649+0000 7f38d3fff640 1 -- 10.20.193.78:0/523349254 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(307..307 src has 117..307) ==== 4763+0+0 (secure 0 0 0) 0x7f38e8052510 con 0x7f38f806d1b0 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.649+0000 7f38f77fe640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38c803deb0 0x7f38c8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.649+0000 7f38f77fe640 10 monclient: get_auth_request con 0x7f38c803deb0 auth_method 0 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.651+0000 7f38d3fff640 1 -- 10.20.193.78:0/523349254 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f38e801a9c0 con 0x7f38f806d1b0 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.651+0000 7f38d3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.651+0000 7f38d3fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:32.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.651+0000 7f38f77fe640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38c803deb0 0x7f38c8040370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f38e400ac80 tx=0x7f38e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.769+0000 7f38fdafb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:54:32.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.769+0000 7f38fdafb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:32.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.769+0000 7f38fdafb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.769+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f38f8135ad0 con 0x7f38f806d1b0 2026-02-17T05:54:32.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.770+0000 7f38d3fff640 1 -- 10.20.193.78:0/523349254 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f38e801ac80 con 0x7f38f806d1b0 2026-02-17T05:54:32.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.770+0000 7f38d3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:32.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.770+0000 7f38d3fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:32.769 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38c803deb0 msgr2=0x7f38c8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38c803deb0 0x7f38c8040370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f38e400ac80 tx=0x7f38e4008040 comp rx=0 tx=0).stop 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 10 monclient: shutdown 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 msgr2=0x7f38f81cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f81cf890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c4a0 tx=0x7f38e800c970 comp rx=0 tx=0).stop 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 shutdown_connections 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38c803deb0 0x7f38c8040370 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38f809c4a0 0x7f38f809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38f806d1b0 0x7f38f81cf890 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 --2- 10.20.193.78:0/523349254 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38f806c890 0x7f38f81cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 >> 10.20.193.78:0/523349254 conn(0x7f38f80a6620 msgr2=0x7f38f8071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 shutdown_connections 2026-02-17T05:54:32.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.771+0000 7f38fdafb640 1 -- 10.20.193.78:0/523349254 wait complete. 2026-02-17T05:54:32.778 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:54:32.778 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:54:32.778 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:54:32.778 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:54:32.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.848+0000 7fadcad2c640 10 monclient: get_monmap_and_config 2026-02-17T05:54:32.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.848+0000 7fadcad2c640 10 monclient: build_initial_monmap 2026-02-17T05:54:32.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.848+0000 7fadcad2c640 10 monclient: monmap: 2026-02-17T05:54:32.851 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:32.851 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.851 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:32.849853+0000 2026-02-17T05:54:32.851 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:32.849853+0000 2026-02-17T05:54:32.851 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:32.851 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 1 Processor -- start 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 1 -- start start 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient: init 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2026-02-17T05:54:32.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc406c890 0x7fadc406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient(hunting): picked mon.noname-b con 0x7fadc406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fadc409c4a0 0x7fadc406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient(hunting): picked mon.noname-a con 0x7fadc409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fadc4134df0 0x7fadc4135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient(hunting): picked mon.noname-c con 0x7fadc4134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fadc40a0880 con 0x7fadc409c4a0 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fadc406d6f0 con 0x7fadc4134df0 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fadc406daf0 con 0x7fadc406c890 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadcad2c640 10 monclient(hunting): authenticate will time out at 1118.403427s 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc406c890 0x7fadc406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fadc409c4a0 0x7fadc406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc406c890 0x7fadc406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44566/0 (socket says 10.20.193.78:44566) 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 1 -- 10.20.193.78:0/2902411213 learned_addr learned my addr 10.20.193.78:0/2902411213 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 10 monclient(hunting): get_auth_request con 0x7fadc406c890 auth_method 0 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient(hunting): get_auth_request con 0x7fadc409c4a0 auth_method 0 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc92a2640 1 --2- 10.20.193.78:0/2902411213 >> v2:10.20.193.115:3300/0 conn(0x7fadc4134df0 0x7fadc4135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc92a2640 10 monclient(hunting): get_auth_request con 0x7fadc4134df0 auth_method 0 2026-02-17T05:54:32.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc92a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc92a2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadc92a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient(hunting): handle_auth_done global_id 104202 payload 274 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 1 -- 10.20.193.78:0/2902411213 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc406c890 msgr2=0x7fadc406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 1 --2- 10.20.193.78:0/2902411213 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc406c890 0x7fadc406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 1 -- 10.20.193.78:0/2902411213 >> v2:10.20.193.115:3300/0 conn(0x7fadc4134df0 msgr2=0x7fadc4135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 1 --2- 10.20.193.78:0/2902411213 >> v2:10.20.193.115:3300/0 conn(0x7fadc4134df0 0x7fadc4135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 1 monclient: found mon.noname-a 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:32.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.849+0000 7fadbbfff640 1 -- 10.20.193.78:0/2902411213 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc409cc80 con 0x7fadc409c4a0 2026-02-17T05:54:32.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbbfff640 1 -- 10.20.193.78:0/2902411213 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 msgr2=0x7fadc406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbbfff640 1 --2- 10.20.193.78:0/2902411213 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc406d1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fadc409c880 tx=0x7fadac02ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b5e39ea211a92e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 1 -- 10.20.193.78:0/2902411213 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac03c070 con 0x7fadc409c4a0 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: dump: 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: _finish_auth 0 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 1 -- 10.20.193.78:0/2902411213 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac02fbd0 con 0x7fadc409c4a0 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 5 monclient: authenticate success, global_id 104202 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 1 -- 10.20.193.78:0/2902411213 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac037440 con 0x7fadc409c4a0 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 4 set_mon_vals no callback set 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadbb7fe640 10 monclient: dump: 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.867 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 10 monclient: shutdown 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 -- 10.20.193.78:0/2902411213 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 msgr2=0x7fadc406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 --2- 10.20.193.78:0/2902411213 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc406d1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fadc409c880 tx=0x7fadac02ed50 comp rx=0 tx=0).stop 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 -- 10.20.193.78:0/2902411213 shutdown_connections 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 --2- 10.20.193.78:0/2902411213 >> v2:10.20.193.115:3300/0 conn(0x7fadc4134df0 0x7fadc4135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 --2- 10.20.193.78:0/2902411213 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc406d1b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 --2- 10.20.193.78:0/2902411213 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc406c890 0x7fadc406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 -- 10.20.193.78:0/2902411213 >> 10.20.193.78:0/2902411213 conn(0x7fadc40a6620 msgr2=0x7fadc40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 -- 10.20.193.78:0/2902411213 shutdown_connections 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.852+0000 7fadcad2c640 1 -- 10.20.193.78:0/2902411213 wait complete. 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient: build_initial_monmap 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient: monmap: 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 1 Processor -- start 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 1 -- start start 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient: init 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fadc406c890 0x7fadc41cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient(hunting): picked mon.noname-b con 0x7fadc406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc4133f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient(hunting): picked mon.noname-a con 0x7fadc409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc4134df0 0x7fadc41344a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient(hunting): picked mon.noname-c con 0x7fadc4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fadc40a01a0 con 0x7fadc409c4a0 2026-02-17T05:54:32.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fadc40a0020 con 0x7fadc406c890 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fadc409be20 con 0x7fadc4134df0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadcad2c640 10 monclient(hunting): authenticate will time out at 1118.407671s 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc4133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc92a2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc4134df0 0x7fadc41344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc4133f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39600/0 (socket says 10.20.193.78:39600) 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 -- 10.20.193.78:0/874589305 learned_addr learned my addr 10.20.193.78:0/874589305 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient(hunting): get_auth_request con 0x7fadc409c4a0 auth_method 0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc92a2640 10 monclient(hunting): get_auth_request con 0x7fadc4134df0 auth_method 0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc92a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc92a2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc92a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc92a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc92a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc92a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc8aa1640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fadc406c890 0x7fadc41cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc8aa1640 10 monclient(hunting): get_auth_request con 0x7fadc406c890 auth_method 0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc8aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc8aa1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadc8aa1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient(hunting): handle_auth_done global_id 104205 payload 824 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc4134df0 msgr2=0x7fadc41344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fadc4134df0 0x7fadc41344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fadc406c890 msgr2=0x7fadc41cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fadc406c890 0x7fadc41cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 monclient: found mon.noname-a 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:54:32.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.853+0000 7fadbbfff640 1 -- 10.20.193.78:0/874589305 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc40725d0 con 0x7fadc409c4a0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadbbfff640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 msgr2=0x7fadc4133f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadbbfff640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc4133f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadac0099a0 tx=0x7fadac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac03c040 con 0x7fadc409c4a0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: dump: 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: _finish_auth 0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: _check_auth_tickets 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac047070 con 0x7fadc409c4a0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 5 monclient: authenticate success, global_id 104205 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 10 monclient: _renew_subs 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc40a15b0 con 0x7fadc409c4a0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac040a60 con 0x7fadc409c4a0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadbaffd640 4 set_mon_vals no callback set 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 10 monclient: dump: 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.870 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 10 monclient: _renew_subs 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc409f7a0 con 0x7fadc409c4a0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc41261f0 0x7fadc409e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fadc406d510 con 0x7fadc41261f0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc41261f0 0x7fadc409e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 10 monclient: get_auth_request con 0x7fadc41261f0 auth_method 0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fadac03f030 con 0x7fadc409c4a0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_done global_id 104208 payload 824 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadc8aa1640 1 -- 10.20.193.78:0/874589305 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadc4074980 con 0x7fadc41261f0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad9403deb0 0x7fad94040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.856+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(307..307 src has 117..307) ==== 4763+0+0 (secure 0 0 0) 0x7fadac077ae0 con 0x7fadc409c4a0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.857+0000 7fadc92a2640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad9403deb0 0x7fad94040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.857+0000 7fadc92a2640 10 monclient: get_auth_request con 0x7fad9403deb0 auth_method 0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadc8aa1640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc41261f0 msgr2=0x7fadc409e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadc8aa1640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc41261f0 0x7fadc409e310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadb4011800 tx=0x7fadb4011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb400fe50 con 0x7fadc41261f0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 10 monclient: dump: 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fadb400fe50 con 0x7fadc41261f0 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc41261f0 msgr2=0x7fadc409e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc41261f0 0x7fadc409e310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadb4011800 tx=0x7fadb4011cd0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc41261f0 msgr2=0x7fadc409e310 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadb97fa640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc41261f0 0x7fadc409e310 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadb4011800 tx=0x7fadb4011cd0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.859+0000 7fadc92a2640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad9403deb0 0x7fad94040370 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fadc0003ba0 tx=0x7fadc0009770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadcad2c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadcad2c640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc411cc70 0x7fadc41da510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadcad2c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4135240 con 0x7fadc411cc70 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadc8aa1640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc411cc70 0x7fadc41da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadc8aa1640 10 monclient: get_auth_request con 0x7fadc411cc70 auth_method 0 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadc8aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadc8aa1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadc8aa1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.862+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.863+0000 7fadc8aa1640 10 monclient(hunting): handle_auth_done global_id 104211 payload 824 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.863+0000 7fadc8aa1640 1 -- 10.20.193.78:0/874589305 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fadc4127700 con 0x7fadc411cc70 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadc8aa1640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc411cc70 msgr2=0x7fadc41da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadc8aa1640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc411cc70 0x7fadc41da510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fadb4011800 tx=0x7fadb40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb402b020 con 0x7fadc411cc70 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 10 monclient: dump: 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fadb4015660 con 0x7fadc411cc70 2026-02-17T05:54:32.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc411cc70 msgr2=0x7fadc41da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc411cc70 0x7fadc41da510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fadb4011800 tx=0x7fadb40047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc411cc70 msgr2=0x7fadc41da510 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadb97fa640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc411cc70 0x7fadc41da510 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fadb4011800 tx=0x7fadb40047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 88, 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 66017, 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:32.873 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:32.874 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:32.875 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.876 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:54:32.877 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad9403deb0 msgr2=0x7fad94040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fad9403deb0 0x7fad94040370 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fadc0003ba0 tx=0x7fadc0009770 comp rx=0 tx=0).stop 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 10 monclient: shutdown 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 msgr2=0x7fadc4133f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc4133f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadac0099a0 tx=0x7fadac0047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadc92a2640 1 -- 10.20.193.78:0/874589305 reap_dead start 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 shutdown_connections 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 --2- 10.20.193.78:0/874589305 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fadc409c4a0 0x7fadc4133f60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 >> 10.20.193.78:0/874589305 conn(0x7fadc40a6620 msgr2=0x7fadc40a1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 shutdown_connections 2026-02-17T05:54:32.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.865+0000 7fadcad2c640 1 -- 10.20.193.78:0/874589305 wait complete. 2026-02-17T05:54:32.878 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:54:32.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:54:32.878 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:54:32.929 INFO:tasks.rados.rados.0.trial115.stdout:1459: done (3 left) 2026-02-17T05:54:32.929 INFO:tasks.rados.rados.0.trial115.stdout:1460: done (2 left) 2026-02-17T05:54:32.929 INFO:tasks.rados.rados.0.trial115.stdout:1461: done (1 left) 2026-02-17T05:54:32.929 INFO:tasks.rados.rados.0.trial115.stdout:1462: done (0 left) 2026-02-17T05:54:32.929 INFO:tasks.rados.rados.0.trial115.stdout:1463: read oid 28 snap -1 2026-02-17T05:54:32.930 INFO:tasks.rados.rados.0.trial115.stdout:1463: expect deleted 2026-02-17T05:54:32.930 INFO:tasks.rados.rados.0.trial115.stdout:1464: write oid 50 current snap is 143 2026-02-17T05:54:32.930 INFO:tasks.rados.rados.0.trial115.stdout:1464: seq_num 554 ranges {511078=465362,1752436=740194,3163985=147093} 2026-02-17T05:54:32.933 INFO:tasks.rados.rados.0.trial115.stdout:1464: writing trial11545420-50 from 511078 to 976440 tid 1 2026-02-17T05:54:32.939 INFO:tasks.rados.rados.0.trial115.stdout:1464: writing trial11545420-50 from 1752436 to 2492630 tid 2 2026-02-17T05:54:32.942 INFO:tasks.rados.rados.0.trial115.stdout:1464: writing trial11545420-50 from 3163985 to 3311078 tid 3 2026-02-17T05:54:32.942 INFO:tasks.rados.rados.0.trial115.stdout:1465: snap_create 2026-02-17T05:54:32.942 INFO:tasks.rados.rados.0.trial115.stdout:1464: finishing write tid 1 to trial11545420-50 2026-02-17T05:54:32.942 INFO:tasks.rados.rados.0.trial115.stdout:1464: finishing write tid 2 to trial11545420-50 2026-02-17T05:54:32.943 INFO:tasks.rados.rados.0.trial115.stdout:1464: finishing write tid 3 to trial11545420-50 2026-02-17T05:54:32.944 INFO:tasks.rados.rados.0.trial115.stdout:1464: finishing write tid 4 to trial11545420-50 2026-02-17T05:54:32.944 INFO:tasks.rados.rados.0.trial115.stdout:1464: finishing write tid 5 to trial11545420-50 2026-02-17T05:54:32.944 INFO:tasks.rados.rados.0.trial115.stdout:1464: oid 50 updating version 0 to 634 2026-02-17T05:54:32.944 INFO:tasks.rados.rados.0.trial115.stdout:1464: oid 50 version 634 is already newer than 633 2026-02-17T05:54:32.944 INFO:tasks.rados.rados.0.trial115.stdout:1464: oid 50 updating version 634 to 635 2026-02-17T05:54:32.945 INFO:tasks.rados.rados.0.trial115.stdout:1464: oid 50 updating version 635 to 636 2026-02-17T05:54:32.945 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 636 (ObjNum 554 snap 143 seq_num 554) dirty exists 2026-02-17T05:54:32.945 INFO:tasks.rados.rados.0.trial115.stdout:1464: left oid 50 (ObjNum 554 snap 143 seq_num 554) 2026-02-17T05:54:32.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.948+0000 7f3a6a5bc640 10 monclient: get_monmap_and_config 2026-02-17T05:54:32.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.948+0000 7f3a6a5bc640 10 monclient: build_initial_monmap 2026-02-17T05:54:32.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.948+0000 7f3a6a5bc640 10 monclient: monmap: 2026-02-17T05:54:32.967 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:32.967 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.967 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:32.949985+0000 2026-02-17T05:54:32.967 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:32.949985+0000 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 1 Processor -- start 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 1 -- start start 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient: init 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6406c890 0x7f3a6406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient(hunting): picked mon.noname-b con 0x7f3a6406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3a6409c4a0 0x7f3a6406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient(hunting): picked mon.noname-a con 0x7f3a6409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:54:32.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3a64134df0 0x7f3a64135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient(hunting): picked mon.noname-c con 0x7f3a64134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f3a640a0880 con 0x7f3a6409c4a0 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6406d6f0 con 0x7f3a64134df0 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6406daf0 con 0x7f3a6406c890 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a6a5bc640 10 monclient(hunting): authenticate will time out at 1118.503541s 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6406c890 0x7f3a6406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3a6409c4a0 0x7f3a6406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6406c890 0x7f3a6406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44584/0 (socket says 10.20.193.78:44584) 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 -- 10.20.193.78:0/3583873101 learned_addr learned my addr 10.20.193.78:0/3583873101 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient(hunting): get_auth_request con 0x7f3a6406c890 auth_method 0 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 10 monclient(hunting): get_auth_request con 0x7f3a6409c4a0 auth_method 0 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a68b32640 1 --2- 10.20.193.78:0/3583873101 >> v2:10.20.193.115:3300/0 conn(0x7f3a64134df0 0x7f3a64135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a68b32640 10 monclient(hunting): get_auth_request con 0x7f3a64134df0 auth_method 0 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a68b32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a68b32640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a68b32640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient(hunting): handle_auth_done global_id 74495 payload 274 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 -- 10.20.193.78:0/3583873101 >> v2:10.20.193.115:3300/0 conn(0x7f3a64134df0 msgr2=0x7f3a64135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 --2- 10.20.193.78:0/3583873101 >> v2:10.20.193.115:3300/0 conn(0x7f3a64134df0 0x7f3a64135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 -- 10.20.193.78:0/3583873101 >> v2:10.20.193.78:3300/0 conn(0x7f3a6409c4a0 msgr2=0x7f3a6406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 --2- 10.20.193.78:0/3583873101 >> v2:10.20.193.78:3300/0 conn(0x7f3a6409c4a0 0x7f3a6406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 monclient: found mon.noname-b 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a63fff640 1 -- 10.20.193.78:0/3583873101 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6409cc80 con 0x7f3a6406c890 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.949+0000 7f3a637fe640 1 --2- 10.20.193.78:0/3583873101 >> v2:10.20.193.78:3300/0 conn(0x7f3a6409c4a0 0x7f3a6406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a63fff640 1 -- 10.20.193.78:0/3583873101 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6406c890 msgr2=0x7f3a6406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a63fff640 1 --2- 10.20.193.78:0/3583873101 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6406c890 0x7f3a6406cc70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3a6409c880 tx=0x7f3a5002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31bfb00ef88147aa server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 1 -- 10.20.193.78:0/3583873101 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5003c070 con 0x7f3a6406c890 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: dump: 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.973 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: _finish_auth 0 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 1 -- 10.20.193.78:0/3583873101 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5002fca0 con 0x7f3a6406c890 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 5 monclient: authenticate success, global_id 74495 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 1 -- 10.20.193.78:0/3583873101 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50037660 con 0x7f3a6406c890 2026-02-17T05:54:32.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 4 set_mon_vals no callback set 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a62ffd640 10 monclient: dump: 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 10 monclient: shutdown 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:32.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3583873101 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6406c890 msgr2=0x7f3a6406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3583873101 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6406c890 0x7f3a6406cc70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3a6409c880 tx=0x7f3a5002ee60 comp rx=0 tx=0).stop 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3583873101 shutdown_connections 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3583873101 >> v2:10.20.193.115:3300/0 conn(0x7f3a64134df0 0x7f3a64135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3583873101 >> v2:10.20.193.78:3300/0 conn(0x7f3a6409c4a0 0x7f3a6406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3583873101 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6406c890 0x7f3a6406cc70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3583873101 >> 10.20.193.78:0/3583873101 conn(0x7f3a640a6620 msgr2=0x7f3a640a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3583873101 shutdown_connections 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.952+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3583873101 wait complete. 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient: build_initial_monmap 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient: monmap: 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.976 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 1 Processor -- start 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 1 -- start start 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient: init 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a6406c890 0x7f3a641cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient(hunting): picked mon.noname-a con 0x7f3a6406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6409c4a0 0x7f3a64122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient(hunting): picked mon.noname-c con 0x7f3a6409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a64134df0 0x7f3a64123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient(hunting): picked mon.noname-b con 0x7f3a64134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a640a01a0 con 0x7f3a6406c890 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a640a0020 con 0x7f3a64134df0 2026-02-17T05:54:32.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6409be20 con 0x7f3a6409c4a0 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a6a5bc640 10 monclient(hunting): authenticate will time out at 1118.507878s 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a6406c890 0x7f3a641cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a6406c890 0x7f3a641cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39638/0 (socket says 10.20.193.78:39638) 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a637fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6409c4a0 0x7f3a64122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 1 -- 10.20.193.78:0/3454750044 learned_addr learned my addr 10.20.193.78:0/3454750044 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 10 monclient(hunting): get_auth_request con 0x7f3a6406c890 auth_method 0 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a637fe640 10 monclient(hunting): get_auth_request con 0x7f3a6409c4a0 auth_method 0 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.953+0000 7f3a637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a68b32640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a64134df0 0x7f3a64123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a68b32640 10 monclient(hunting): get_auth_request con 0x7f3a64134df0 auth_method 0 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a68b32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a68b32640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a68b32640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 10 monclient(hunting): handle_auth_done global_id 74498 payload 824 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a64134df0 msgr2=0x7f3a64123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a64134df0 0x7f3a64123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a6406c890 msgr2=0x7f3a641cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a6406c890 0x7f3a641cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 1 monclient: found mon.noname-c 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a637fe640 1 -- 10.20.193.78:0/3454750044 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a64124830 con 0x7f3a6409c4a0 2026-02-17T05:54:32.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.954+0000 7f3a63fff640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a6406c890 0x7f3a641cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a637fe640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6409c4a0 msgr2=0x7f3a64122d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a637fe640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6409c4a0 0x7f3a64122d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400ec10 tx=0x7f3a5400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017070 con 0x7f3a6409c4a0 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: dump: 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: _finish_auth 0 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:32.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5400ce50 con 0x7f3a6409c4a0 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 5 monclient: authenticate success, global_id 74498 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 10 monclient: _renew_subs 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a640a1630 con 0x7f3a6409c4a0 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54013690 con 0x7f3a6409c4a0 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a627fc640 4 set_mon_vals no callback set 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 10 monclient: dump: 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 10 monclient: _renew_subs 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641268b0 con 0x7f3a6409c4a0 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a641cfbe0 0x7f3a641237a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3a64074980 con 0x7f3a641cfbe0 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a63fff640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a641cfbe0 0x7f3a641237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a63fff640 10 monclient: get_auth_request con 0x7f3a641cfbe0 auth_method 0 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a63fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a63fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.956+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3a54024060 con 0x7f3a6409c4a0 2026-02-17T05:54:32.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.957+0000 7f3a63fff640 10 monclient(hunting): handle_auth_done global_id 74501 payload 824 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.957+0000 7f3a63fff640 1 -- 10.20.193.78:0/3454750044 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a64135240 con 0x7f3a641cfbe0 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.957+0000 7f3a60ff9640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a3403deb0 0x7f3a34040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.957+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(308..308 src has 117..308) ==== 4735+0+0 (secure 0 0 0) 0x7f3a540527a0 con 0x7f3a6409c4a0 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.957+0000 7f3a68b32640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a3403deb0 0x7f3a34040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.957+0000 7f3a68b32640 10 monclient: get_auth_request con 0x7f3a3403deb0 auth_method 0 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a63fff640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a641cfbe0 msgr2=0x7f3a641237a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a63fff640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a641cfbe0 0x7f3a641237a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a5002eea0 tx=0x7f3a500047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50045070 con 0x7f3a641cfbe0 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 10 monclient: dump: 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.983 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f3a5002fe50 con 0x7f3a641cfbe0 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:32.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a641cfbe0 msgr2=0x7f3a641237a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a641cfbe0 0x7f3a641237a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a5002eea0 tx=0x7f3a500047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a641cfbe0 msgr2=0x7f3a641237a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a60ff9640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a641cfbe0 0x7f3a641237a0 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a5002eea0 tx=0x7f3a500047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.959+0000 7f3a68b32640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a3403deb0 0x7f3a34040370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3a64126020 tx=0x7f3a5800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a6a5bc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a64123ce0 0x7f3a641da350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a6a5bc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3a641240f0 con 0x7f3a64123ce0 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a64123ce0 0x7f3a641da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 10 monclient: get_auth_request con 0x7f3a64123ce0 auth_method 0 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:32.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 10 monclient(hunting): handle_auth_done global_id 74504 payload 824 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.962+0000 7f3a63fff640 1 -- 10.20.193.78:0/3454750044 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3a640744d0 con 0x7f3a64123ce0 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a63fff640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a64123ce0 msgr2=0x7f3a641da350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a63fff640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a64123ce0 0x7f3a641da350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a6406c650 tx=0x7f3a50009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50045040 con 0x7f3a64123ce0 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 10 monclient: dump: 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.986 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f3a5004aaf0 con 0x7f3a64123ce0 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a64123ce0 msgr2=0x7f3a641da350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a64123ce0 0x7f3a641da350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a6406c650 tx=0x7f3a50009340 comp rx=0 tx=0).stop 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a64123ce0 msgr2=0x7f3a641da350 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:32.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a60ff9640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a64123ce0 0x7f3a641da350 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a6406c650 tx=0x7f3a50009340 comp rx=0 tx=0).stop 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 88, 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 244858, 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:54:32.988 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:32.989 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.990 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:54:32.991 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:54:32.992 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.993 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.994 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:32.995 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:32.996 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.997 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:54:32.998 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a3403deb0 msgr2=0x7f3a34040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a3403deb0 0x7f3a34040370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3a64126020 tx=0x7f3a5800b040 comp rx=0 tx=0).stop 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 10 monclient: shutdown 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6409c4a0 msgr2=0x7f3a64122d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6409c4a0 0x7f3a64122d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400ec10 tx=0x7f3a5400c520 comp rx=0 tx=0).stop 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a68b32640 1 -- 10.20.193.78:0/3454750044 reap_dead start 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 shutdown_connections 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 --2- 10.20.193.78:0/3454750044 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a6409c4a0 0x7f3a64122d20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 >> 10.20.193.78:0/3454750044 conn(0x7f3a640a6620 msgr2=0x7f3a64135810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 shutdown_connections 2026-02-17T05:54:32.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:32.965+0000 7f3a6a5bc640 1 -- 10.20.193.78:0/3454750044 wait complete. 2026-02-17T05:54:32.999 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:54:32.999 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:54:33.000 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.069+0000 7f07cb6b0640 10 monclient: get_monmap_and_config 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.069+0000 7f07cb6b0640 10 monclient: build_initial_monmap 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.069+0000 7f07cb6b0640 10 monclient: monmap: 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:33.070568+0000 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:33.070568+0000 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:33.070 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.069+0000 7f07cb6b0640 1 Processor -- start 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.069+0000 7f07cb6b0640 1 -- start start 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.069+0000 7f07cb6b0640 10 monclient: init 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f07c406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f07c409c4a0 0x7f07c413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient(hunting): picked mon.noname-c con 0x7f07c409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient(hunting): picked mon.noname-b con 0x7f07c406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c406daf0 con 0x7f07c406c890 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f07c40a0880 con 0x7f07c409c4a0 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f07c409cc80 con 0x7f07c406d380 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07cb6b0640 10 monclient(hunting): authenticate will time out at 1118.624110s 2026-02-17T05:54:33.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44612/0 (socket says 10.20.193.78:44612) 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 1 -- 10.20.193.78:0/2811483341 learned_addr learned my addr 10.20.193.78:0/2811483341 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient(hunting): get_auth_request con 0x7f07c406d380 auth_method 0 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c8c24640 1 --2- 10.20.193.78:0/2811483341 >> v2:10.20.193.115:3300/0 conn(0x7f07c409c4a0 0x7f07c413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c8c24640 10 monclient(hunting): get_auth_request con 0x7f07c409c4a0 auth_method 0 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c8c24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c8c24640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c8c24640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient(hunting): handle_auth_done global_id 74507 payload 274 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 1 -- 10.20.193.78:0/2811483341 >> v2:10.20.193.115:3300/0 conn(0x7f07c409c4a0 msgr2=0x7f07c413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 1 --2- 10.20.193.78:0/2811483341 >> v2:10.20.193.115:3300/0 conn(0x7f07c409c4a0 0x7f07c413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 1 -- 10.20.193.78:0/2811483341 >> v1:10.20.193.115:6789/0 conn(0x7f07c406c890 legacy=0x7f07c406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient: _finish_hunting 0 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 1 monclient: found mon.noname-b 2026-02-17T05:54:33.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:33.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.070+0000 7f07c9c26640 1 -- 10.20.193.78:0/2811483341 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c409cf60 con 0x7f07c406d380 2026-02-17T05:54:33.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.072+0000 7f07c9c26640 1 -- 10.20.193.78:0/2811483341 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 msgr2=0x7f07c413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:33.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.072+0000 7f07c9c26640 1 --2- 10.20.193.78:0/2811483341 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c413d8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009e50 tx=0x7f07c002f260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b800e9fca802e411 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:33.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 1 -- 10.20.193.78:0/2811483341 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00373f0 con 0x7f07c406d380 2026-02-17T05:54:33.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: dump: 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: _finish_auth 0 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 5 monclient: authenticate success, global_id 74507 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 1 -- 10.20.193.78:0/2811483341 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c0037590 con 0x7f07c406d380 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:33.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 4 set_mon_vals no callback set 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 1 -- 10.20.193.78:0/2811483341 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0037960 con 0x7f07c406d380 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07b3fff640 10 monclient: dump: 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 10 monclient: shutdown 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 1 -- 10.20.193.78:0/2811483341 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 msgr2=0x7f07c413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 1 --2- 10.20.193.78:0/2811483341 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c413d8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009e50 tx=0x7f07c002f260 comp rx=0 tx=0).stop 2026-02-17T05:54:33.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 1 -- 10.20.193.78:0/2811483341 shutdown_connections 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 1 --2- 10.20.193.78:0/2811483341 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c413d8f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 1 --2- 10.20.193.78:0/2811483341 >> v2:10.20.193.115:3300/0 conn(0x7f07c409c4a0 0x7f07c413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 1 -- 10.20.193.78:0/2811483341 >> 10.20.193.78:0/2811483341 conn(0x7f07c40a6620 msgr2=0x7f07c40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 1 -- 10.20.193.78:0/2811483341 shutdown_connections 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 1 -- 10.20.193.78:0/2811483341 wait complete. 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 10 monclient: build_initial_monmap 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.073+0000 7f07cb6b0640 10 monclient: monmap: 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 1 Processor -- start 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 1 -- start start 2026-02-17T05:54:33.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient: init 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c406c890 0x7f07c413c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient(hunting): picked mon.noname-b con 0x7f07c406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient(hunting): picked mon.noname-c con 0x7f07c406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f07c409c4a0 0x7f07c409e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient(hunting): picked mon.noname-a con 0x7f07c409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07c409bfa0 con 0x7f07c409c4a0 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07c409be20 con 0x7f07c406c890 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f07c413de30 con 0x7f07c406d380 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07cb6b0640 10 monclient(hunting): authenticate will time out at 1118.628396s 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c409db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44614/0 (socket says 10.20.193.78:44614) 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9c26640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f07c409c4a0 0x7f07c409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 -- 10.20.193.78:0/1620203620 learned_addr learned my addr 10.20.193.78:0/1620203620 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient(hunting): get_auth_request con 0x7f07c406d380 auth_method 0 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9c26640 10 monclient(hunting): get_auth_request con 0x7f07c409c4a0 auth_method 0 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9c26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9c26640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9c26640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9c26640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9c26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9c26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9425640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c406c890 0x7f07c413c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9425640 10 monclient(hunting): get_auth_request con 0x7f07c406c890 auth_method 0 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9425640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9425640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c9425640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient(hunting): handle_auth_done global_id 74510 payload 824 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c406c890 msgr2=0x7f07c413c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c406c890 0x7f07c413c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f07c409c4a0 msgr2=0x7f07c409e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f07c409c4a0 0x7f07c409e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient: _finish_hunting 0 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 monclient: found mon.noname-c 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.074+0000 7f07c8c24640 1 -- 10.20.193.78:0/1620203620 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c413b9d0 con 0x7f07c406d380 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07c8c24640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 msgr2=0x7f07c409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07c8c24640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c409db50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e7f0 tx=0x7f07b800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8017070 con 0x7f07c406d380 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: dump: 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.084 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b800cae0 con 0x7f07c406d380 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b800f040 con 0x7f07c406d380 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b37fe640 4 set_mon_vals no callback set 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07b1ffb640 10 monclient: dump: 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.085 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07cb6b0640 5 monclient: authenticate success, global_id 74510 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07cb6b0640 10 monclient: _renew_subs 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07cb6b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c413bcb0 con 0x7f07c406d380 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07cb6b0640 10 monclient: _renew_subs 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07cb6b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.076+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c413d9f0 con 0x7f07c406d380 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07cb6b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07cb6b0640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c41338b0 0x7f07c409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07cb6b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4074980 con 0x7f07c41338b0 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f07b8012050 con 0x7f07c406d380 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9425640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c41338b0 0x7f07c409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9425640 10 monclient: get_auth_request con 0x7f07c41338b0 auth_method 0 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9425640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9425640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9425640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07b1ffb640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f079403df00 0x7f07940403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(308..308 src has 117..308) ==== 4735+0+0 (secure 0 0 0) 0x7f07b80525d0 con 0x7f07c406d380 2026-02-17T05:54:33.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9425640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:33.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9425640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:33.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9c26640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f079403df00 0x7f07940403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9425640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:33.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.077+0000 7f07c9c26640 10 monclient: get_auth_request con 0x7f079403df00 auth_method 0 2026-02-17T05:54:33.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.078+0000 7f07c9425640 10 monclient(hunting): handle_auth_done global_id 114130 payload 824 2026-02-17T05:54:33.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.078+0000 7f07c9425640 1 -- 10.20.193.78:0/1620203620 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07c406cc70 con 0x7f07c41338b0 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07c9c26640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f079403df00 0x7f07940403c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07c406a480 tx=0x7f07c0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07c9425640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c41338b0 msgr2=0x7f07c409e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07c9425640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c41338b0 0x7f07c409e5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e730 tx=0x7f07b400ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4006ae0 con 0x7f07c41338b0 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 10 monclient: dump: 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.089 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f07b4006ae0 con 0x7f07c41338b0 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c41338b0 msgr2=0x7f07c409e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c41338b0 0x7f07c409e5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e730 tx=0x7f07b400ec00 comp rx=0 tx=0).stop 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c41338b0 msgr2=0x7f07c409e5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.080+0000 7f07b1ffb640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c41338b0 0x7f07c409e5d0 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e730 tx=0x7f07b400ec00 comp rx=0 tx=0).stop 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07cb6b0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07cb6b0640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c40714b0 0x7f07c41d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07cb6b0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07c413bef0 con 0x7f07c40714b0 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c40714b0 0x7f07c41d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 10 monclient: get_auth_request con 0x7f07c40714b0 auth_method 0 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 10 monclient(hunting): handle_auth_done global_id 114133 payload 824 2026-02-17T05:54:33.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.084+0000 7f07c9425640 1 -- 10.20.193.78:0/1620203620 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f07c40744d0 con 0x7f07c40714b0 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.086+0000 7f07c9425640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c40714b0 msgr2=0x7f07c41d9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.086+0000 7f07c9425640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c40714b0 0x7f07c41d9d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e730 tx=0x7f07b4003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b400c930 con 0x7f07c40714b0 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 10 monclient: dump: 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f07b400c930 con 0x7f07c40714b0 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c40714b0 msgr2=0x7f07c41d9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c40714b0 0x7f07c41d9d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e730 tx=0x7f07b4003860 comp rx=0 tx=0).stop 2026-02-17T05:54:33.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c40714b0 msgr2=0x7f07c41d9d00 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07b1ffb640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f07c40714b0 0x7f07c41d9d00 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e730 tx=0x7f07b4003860 comp rx=0 tx=0).stop 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 88, 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 17167, 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:33.092 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:33.093 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:54:33.094 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:33.095 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:33.096 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.097 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:33.098 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:54:33.099 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:54:33.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f079403df00 msgr2=0x7f07940403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f079403df00 0x7f07940403c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07c406a480 tx=0x7f07c0033000 comp rx=0 tx=0).stop 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 10 monclient: shutdown 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 msgr2=0x7f07c409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 1 --2- 10.20.193.78:0/1620203620 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f07c406d380 0x7f07c409db50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e7f0 tx=0x7f07b800ecc0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07c9c26640 1 -- 10.20.193.78:0/1620203620 reap_dead start 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07c9c26640 1 -- 10.20.193.78:0/1620203620 reap_dead start 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 shutdown_connections 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 >> 10.20.193.78:0/1620203620 conn(0x7f07c40a6620 msgr2=0x7f07c40a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 shutdown_connections 2026-02-17T05:54:33.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.087+0000 7f07cb6b0640 1 -- 10.20.193.78:0/1620203620 wait complete. 2026-02-17T05:54:33.101 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:54:33.101 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:54:33.102 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:54:33.102 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:54:33.102 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:54:33.102 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:54:33.102 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:54:33.111 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:54:33.111 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:54:33.111 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.219+0000 7fb6d89fb640 10 monclient: get_monmap_and_config 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.219+0000 7fb6d89fb640 10 monclient: build_initial_monmap 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.219+0000 7fb6d89fb640 10 monclient: monmap: 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:33.220691+0000 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:33.220691+0000 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:33.221 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.219+0000 7fb6d89fb640 1 Processor -- start 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.219+0000 7fb6d89fb640 1 -- start start 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.219+0000 7fb6d89fb640 10 monclient: init 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2026-02-17T05:54:33.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d006c890 0x7fb6d006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient(hunting): picked mon.noname-b con 0x7fb6d006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6d009d430 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb6d009bf40 0x7fb6d006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient(hunting): picked mon.noname-c con 0x7fb6d009bf40 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d0073600 con 0x7fb6d009d430 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d009ca50 con 0x7fb6d009bf40 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d006d6f0 con 0x7fb6d006c890 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d89fb640 10 monclient(hunting): authenticate will time out at 1118.774281s 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d006c890 0x7fb6d006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d006c890 0x7fb6d006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44622/0 (socket says 10.20.193.78:44622) 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 1 -- 10.20.193.78:0/120953723 learned_addr learned my addr 10.20.193.78:0/120953723 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient(hunting): get_auth_request con 0x7fb6d006c890 auth_method 0 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6f71640 1 --2- 10.20.193.78:0/120953723 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009bf40 0x7fb6d006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient(hunting): handle_auth_done global_id 74513 payload 274 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 1 -- 10.20.193.78:0/120953723 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009bf40 msgr2=0x7fb6d006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 1 --2- 10.20.193.78:0/120953723 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009bf40 0x7fb6d006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 1 -- 10.20.193.78:0/120953723 >> v1:10.20.193.115:6789/0 conn(0x7fb6d009d430 legacy=0x7fb6d0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:33.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient: _finish_hunting 0 2026-02-17T05:54:33.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 1 monclient: found mon.noname-b 2026-02-17T05:54:33.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:33.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.220+0000 7fb6d6770640 1 -- 10.20.193.78:0/120953723 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d006dab0 con 0x7fb6d006c890 2026-02-17T05:54:33.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d6770640 1 -- 10.20.193.78:0/120953723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d006c890 msgr2=0x7fb6d006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:33.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d6770640 1 --2- 10.20.193.78:0/120953723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d006c890 0x7fb6d006cc70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb6d009c670 tx=0x7fb6c802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4d319cceb41d7a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:33.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 1 -- 10.20.193.78:0/120953723 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c803c070 con 0x7fb6d006c890 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: dump: 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: _finish_auth 0 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: _check_auth_tickets 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 1 -- 10.20.193.78:0/120953723 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c802fd80 con 0x7fb6d006c890 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 5 monclient: authenticate success, global_id 74513 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 1 -- 10.20.193.78:0/120953723 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8037660 con 0x7fb6d006c890 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 4 set_mon_vals no callback set 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d576e640 10 monclient: dump: 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 10 monclient: shutdown 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 1 -- 10.20.193.78:0/120953723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d006c890 msgr2=0x7fb6d006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/120953723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d006c890 0x7fb6d006cc70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb6d009c670 tx=0x7fb6c802ee60 comp rx=0 tx=0).stop 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 1 -- 10.20.193.78:0/120953723 shutdown_connections 2026-02-17T05:54:33.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/120953723 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009bf40 0x7fb6d006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/120953723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d006c890 0x7fb6d006cc70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 1 -- 10.20.193.78:0/120953723 >> 10.20.193.78:0/120953723 conn(0x7fb6d00a6620 msgr2=0x7fb6d00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 1 -- 10.20.193.78:0/120953723 shutdown_connections 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.223+0000 7fb6d89fb640 1 -- 10.20.193.78:0/120953723 wait complete. 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient: build_initial_monmap 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient: monmap: 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:33.235 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 1 Processor -- start 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 1 -- start start 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient: init 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d006c890 0x7fb6d01cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient(hunting): picked mon.noname-b con 0x7fb6d006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6d009bf40 0x7fb6d01cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient(hunting): picked mon.noname-a con 0x7fb6d009bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d009d430 0x7fb6d01215c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient(hunting): picked mon.noname-c con 0x7fb6d009d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d00733c0 con 0x7fb6d009bf40 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0073240 con 0x7fb6d006c890 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d0135500 con 0x7fb6d009d430 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d89fb640 10 monclient(hunting): authenticate will time out at 1118.778552s 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d5f6f640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6d009bf40 msgr2=0x7fb6d01cfd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d5f6f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6d009bf40 0x7fb6d01cfd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d009d430 0x7fb6d01215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d009d430 0x7fb6d01215c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44638/0 (socket says 10.20.193.78:44638) 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 -- 10.20.193.78:0/129014785 learned_addr learned my addr 10.20.193.78:0/129014785 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient(hunting): get_auth_request con 0x7fb6d009d430 auth_method 0 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6770640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d006c890 0x7fb6d01cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6770640 10 monclient(hunting): get_auth_request con 0x7fb6d006c890 auth_method 0 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6770640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6770640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6770640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6770640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6770640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6770640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient(hunting): handle_auth_done global_id 74516 payload 824 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 -- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d006c890 msgr2=0x7fb6d01cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d006c890 0x7fb6d01cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 -- 10.20.193.78:0/129014785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6d009bf40 msgr2=0x7fb6d01cfd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6d009bf40 0x7fb6d01cfd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient: _finish_hunting 0 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 monclient: found mon.noname-c 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.224+0000 7fb6d6f71640 1 -- 10.20.193.78:0/129014785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d0125f20 con 0x7fb6d009d430 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.226+0000 7fb6d6f71640 1 -- 10.20.193.78:0/129014785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d009d430 msgr2=0x7fb6d01215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.226+0000 7fb6d6f71640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d009d430 0x7fb6d01215c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400cce0 tx=0x7fb6c400a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 1 -- 10.20.193.78:0/129014785 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4003c90 con 0x7fb6d009d430 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: dump: 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.239 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: _finish_auth 0 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 1 -- 10.20.193.78:0/129014785 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c4003e50 con 0x7fb6d009d430 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 5 monclient: authenticate success, global_id 74516 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 10 monclient: _renew_subs 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d0125550 con 0x7fb6d009d430 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:33.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 1 -- 10.20.193.78:0/129014785 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400e840 con 0x7fb6d009d430 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 10 monclient: dump: 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d4f6d640 4 set_mon_vals no callback set 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 10 monclient: _renew_subs 2026-02-17T05:54:33.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d00a19d0 con 0x7fb6d009d430 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 1 -- 10.20.193.78:0/129014785 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb6c400ea60 con 0x7fb6d009d430 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d0074980 con 0x7fb6d009d430 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.227+0000 7fb6baffd640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.228+0000 7fb6baffd640 1 -- 10.20.193.78:0/129014785 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(308..308 src has 117..308) ==== 4735+0+0 (secure 0 0 0) 0x7fb6c4053210 con 0x7fb6d009d430 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.228+0000 7fb6d6770640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.228+0000 7fb6d6770640 10 monclient: get_auth_request con 0x7fb6a003deb0 auth_method 0 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.230+0000 7fb6d6770640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8036000 tx=0x7fb6c8033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.230+0000 7fb6baffd640 1 -- 10.20.193.78:0/129014785 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb6c4051c40 con 0x7fb6d009d430 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.230+0000 7fb6baffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:33.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.230+0000 7fb6baffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:33.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.347+0000 7fb6d89fb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:54:33.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.347+0000 7fb6d89fb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:33.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.347+0000 7fb6d89fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:33.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.347+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb6d009d810 con 0x7fb6d009d430 2026-02-17T05:54:33.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.347+0000 7fb6baffd640 1 -- 10.20.193.78:0/129014785 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fb6c40048b0 con 0x7fb6d009d430 2026-02-17T05:54:33.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.347+0000 7fb6baffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:33.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.347+0000 7fb6baffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:33.348 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:54:33.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 msgr2=0x7fb6a0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8036000 tx=0x7fb6c8033000 comp rx=0 tx=0).stop 2026-02-17T05:54:33.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 10 monclient: shutdown 2026-02-17T05:54:33.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:33.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d009d430 msgr2=0x7fb6d01215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d009d430 0x7fb6d01215c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400cce0 tx=0x7fb6c400a660 comp rx=0 tx=0).stop 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 shutdown_connections 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d009d430 0x7fb6d01215c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6d009bf40 0x7fb6d01cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 --2- 10.20.193.78:0/129014785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d006c890 0x7fb6d01cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 >> 10.20.193.78:0/129014785 conn(0x7fb6d00a6620 msgr2=0x7fb6d0137970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 shutdown_connections 2026-02-17T05:54:33.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:33.348+0000 7fb6d89fb640 1 -- 10.20.193.78:0/129014785 wait complete. 2026-02-17T05:54:36.356 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:54:36.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient: get_monmap_and_config 2026-02-17T05:54:36.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient: build_initial_monmap 2026-02-17T05:54:36.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient: monmap: 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:36.427366+0000 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:36.427366+0000 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:36.428 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 1 Processor -- start 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 1 -- start start 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient: init 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f74bc06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient(hunting): picked mon.noname-b con 0x7f74bc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f74bc06d380 0x7f74bc13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient(hunting): picked mon.noname-a con 0x7f74bc06d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:54:36.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc06daf0 con 0x7f74bc06c890 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f74bc0a0880 con 0x7f74bc06d380 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc09cc80 con 0x7f74bc09c4a0 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c326f640 10 monclient(hunting): authenticate will time out at 1121.980899s 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c17e5640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f74bc06d380 msgr2=0x7f74bc13d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74c17e5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f74bc06d380 0x7f74bc13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.426+0000 7f74b3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44640/0 (socket says 10.20.193.78:44640) 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 1 -- 10.20.193.78:0/1827131356 learned_addr learned my addr 10.20.193.78:0/1827131356 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient(hunting): get_auth_request con 0x7f74bc09c4a0 auth_method 0 2026-02-17T05:54:36.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient(hunting): handle_auth_done global_id 74519 payload 274 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 1 -- 10.20.193.78:0/1827131356 >> v2:10.20.193.78:3300/0 conn(0x7f74bc06d380 msgr2=0x7f74bc13d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 1 --2- 10.20.193.78:0/1827131356 >> v2:10.20.193.78:3300/0 conn(0x7f74bc06d380 0x7f74bc13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 1 -- 10.20.193.78:0/1827131356 >> v1:10.20.193.115:6789/0 conn(0x7f74bc06c890 legacy=0x7f74bc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 1 monclient: found mon.noname-b 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:36.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:36.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.427+0000 7f74b3fff640 1 -- 10.20.193.78:0/1827131356 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc09cf60 con 0x7f74bc09c4a0 2026-02-17T05:54:36.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b3fff640 1 -- 10.20.193.78:0/1827131356 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 msgr2=0x7f74bc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:36.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b3fff640 1 --2- 10.20.193.78:0/1827131356 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc13b3a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f74a400de90 tx=0x7f74a40332b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403440c1cc3f04e3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:36.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 1 -- 10.20.193.78:0/1827131356 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4033cd0 con 0x7f74bc09c4a0 2026-02-17T05:54:36.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:36.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:36.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: dump: 2026-02-17T05:54:36.438 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:36.438 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:36.438 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: _finish_auth 0 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 1 -- 10.20.193.78:0/1827131356 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a400b550 con 0x7f74bc09c4a0 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74c326f640 5 monclient: authenticate success, global_id 74519 2026-02-17T05:54:36.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74c326f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 1 -- 10.20.193.78:0/1827131356 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a400b900 con 0x7f74bc09c4a0 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74c326f640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74c326f640 4 set_mon_vals no callback set 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.429+0000 7f74b37fe640 10 monclient: dump: 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:36.440 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient: shutdown 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- 10.20.193.78:0/1827131356 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 msgr2=0x7f74bc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 --2- 10.20.193.78:0/1827131356 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc13b3a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f74a400de90 tx=0x7f74a40332b0 comp rx=0 tx=0).stop 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- 10.20.193.78:0/1827131356 shutdown_connections 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 --2- 10.20.193.78:0/1827131356 >> v2:10.20.193.78:3300/0 conn(0x7f74bc06d380 0x7f74bc13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 --2- 10.20.193.78:0/1827131356 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc13b3a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- 10.20.193.78:0/1827131356 >> 10.20.193.78:0/1827131356 conn(0x7f74bc0a6620 msgr2=0x7f74bc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- 10.20.193.78:0/1827131356 shutdown_connections 2026-02-17T05:54:36.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- 10.20.193.78:0/1827131356 wait complete. 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient: build_initial_monmap 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient: monmap: 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 Processor -- start 2026-02-17T05:54:36.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- start start 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient: init 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f74bc06c890 0x7f74bc1cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient(hunting): picked mon.noname-b con 0x7f74bc06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74bc06d380 0x7f74bc1cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient(hunting): picked mon.noname-a con 0x7f74bc06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc1214f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient(hunting): picked mon.noname-c con 0x7f74bc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc09bfa0 con 0x7f74bc06d380 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc09be20 con 0x7f74bc06c890 2026-02-17T05:54:36.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc09db20 con 0x7f74bc09c4a0 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c326f640 10 monclient(hunting): authenticate will time out at 1121.984934s 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74b3fff640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74bc06d380 msgr2=0x7f74bc1cb850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74b3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74bc06d380 0x7f74bc1cb850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.430+0000 7f74c17e5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc1214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc1214f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44646/0 (socket says 10.20.193.78:44646) 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 1 -- 10.20.193.78:0/3503339140 learned_addr learned my addr 10.20.193.78:0/3503339140 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient(hunting): get_auth_request con 0x7f74bc09c4a0 auth_method 0 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:36.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient(hunting): handle_auth_done global_id 74522 payload 824 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c0fe4640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f74bc06c890 0x7f74bc1cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 1 -- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f74bc06c890 msgr2=0x7f74bc1cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f74bc06c890 0x7f74bc1cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 1 -- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74bc06d380 msgr2=0x7f74bc1cb850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74bc06d380 0x7f74bc1cb850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient: _finish_hunting 0 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 1 monclient: found mon.noname-c 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c17e5640 1 -- 10.20.193.78:0/3503339140 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc124e80 con 0x7f74bc09c4a0 2026-02-17T05:54:36.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c0fe4640 10 monclient: get_auth_request con 0x7f74bc06c890 auth_method 0 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.431+0000 7f74c0fe4640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f74bc06c890 0x7f74bc1cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c17e5640 1 -- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 msgr2=0x7f74bc1214f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c17e5640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc1214f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74b800be60 tx=0x7f74b800bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c17e5640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc1214f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74b800be60 tx=0x7f74b800bf60 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f74b8010030 config(0 keys) 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 1 -- 10.20.193.78:0/3503339140 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8006e60 con 0x7f74bc09c4a0 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 10 monclient: dump: 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:36.446 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 10 monclient: _finish_auth 0 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 10 monclient: _check_auth_tickets 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 1 -- 10.20.193.78:0/3503339140 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b8010030 con 0x7f74bc09c4a0 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c326f640 5 monclient: authenticate success, global_id 74522 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c326f640 10 monclient: _renew_subs 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74b2ffd640 4 set_mon_vals no callback set 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c326f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:36.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc123cb0 con 0x7f74bc09c4a0 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c326f640 10 monclient: _renew_subs 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c326f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.433+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc0727a0 con 0x7f74bc09c4a0 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74c326f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74c326f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74c326f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc06cc70 con 0x7f74bc09c4a0 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74b17fa640 1 -- 10.20.193.78:0/3503339140 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80168b0 con 0x7f74bc09c4a0 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74b17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74b17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74b17fa640 10 monclient: dump: 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:36.448 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74b17fa640 1 -- 10.20.193.78:0/3503339140 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f74b8008070 con 0x7f74bc09c4a0 2026-02-17T05:54:36.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74b17fa640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f748c03de60 0x7f748c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:36.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74c0fe4640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f748c03de60 0x7f748c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:36.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.434+0000 7f74c0fe4640 10 monclient: get_auth_request con 0x7f748c03de60 auth_method 0 2026-02-17T05:54:36.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.436+0000 7f74b17fa640 1 -- 10.20.193.78:0/3503339140 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(308..308 src has 117..308) ==== 4735+0+0 (secure 0 0 0) 0x7f74b80527a0 con 0x7f74bc09c4a0 2026-02-17T05:54:36.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.436+0000 7f74c0fe4640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f748c03de60 0x7f748c040320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74ac004020 tx=0x7f74ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:36.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:36.436+0000 7f74b17fa640 1 -- 10.20.193.78:0/3503339140 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:54:37.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.057+0000 7f74b17fa640 1 -- 10.20.193.78:0/3503339140 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f74b80511d0 con 0x7f74bc09c4a0 2026-02-17T05:54:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.057+0000 7f74b17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.057+0000 7f74b17fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:37.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.168+0000 7f74c326f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:54:37.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.168+0000 7f74c326f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:37.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.168+0000 7f74c326f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:37.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.168+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f74bc1253a0 con 0x7f74bc09c4a0 2026-02-17T05:54:37.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.168+0000 7f74b17fa640 1 -- 10.20.193.78:0/3503339140 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f74b8012d50 con 0x7f74bc09c4a0 2026-02-17T05:54:37.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.168+0000 7f74b17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:37.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.168+0000 7f74b17fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:37.169 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:54:37.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f748c03de60 msgr2=0x7f748c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:37.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f748c03de60 0x7f748c040320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74ac004020 tx=0x7f74ac009040 comp rx=0 tx=0).stop 2026-02-17T05:54:37.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 10 monclient: shutdown 2026-02-17T05:54:37.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:37.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 msgr2=0x7f74bc1214f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc1214f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74b800be60 tx=0x7f74b800bf60 comp rx=0 tx=0).stop 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 shutdown_connections 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f74bc09c4a0 0x7f74bc1214f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f74bc06d380 0x7f74bc1cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f748c03de60 0x7f748c040320 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 --2- 10.20.193.78:0/3503339140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f74bc06c890 0x7f74bc1cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 >> 10.20.193.78:0/3503339140 conn(0x7f74bc0a6620 msgr2=0x7f74bc071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 shutdown_connections 2026-02-17T05:54:37.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:37.169+0000 7f74c326f640 1 -- 10.20.193.78:0/3503339140 wait complete. 2026-02-17T05:54:40.177 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:54:40.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient: get_monmap_and_config 2026-02-17T05:54:40.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient: build_initial_monmap 2026-02-17T05:54:40.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient: monmap: 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:40.248411+0000 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:40.248411+0000 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:40.247 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:40.248 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 1 Processor -- start 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 1 -- start start 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient: init 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fde4c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient(hunting): picked mon.noname-b con 0x7fde4c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:40.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fde4c06d380 0x7fde4c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient(hunting): picked mon.noname-c con 0x7fde4c06d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde4c06daf0 con 0x7fde4c06c890 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fde4c0a0880 con 0x7fde4c06d380 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c09cc80 con 0x7fde4c09c4a0 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.247+0000 7fde5135b640 10 monclient(hunting): authenticate will time out at 1125.801945s 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49220/0 (socket says 10.20.193.78:49220) 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 1 -- 10.20.193.78:0/4079619424 learned_addr learned my addr 10.20.193.78:0/4079619424 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:40.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient(hunting): get_auth_request con 0x7fde4c09c4a0 auth_method 0 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4b7fe640 1 --2- 10.20.193.78:0/4079619424 >> v2:10.20.193.115:3300/0 conn(0x7fde4c06d380 0x7fde4c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4b7fe640 10 monclient(hunting): get_auth_request con 0x7fde4c06d380 auth_method 0 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:40.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient(hunting): handle_auth_done global_id 74525 payload 274 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 1 -- 10.20.193.78:0/4079619424 >> v2:10.20.193.115:3300/0 conn(0x7fde4c06d380 msgr2=0x7fde4c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 1 --2- 10.20.193.78:0/4079619424 >> v2:10.20.193.115:3300/0 conn(0x7fde4c06d380 0x7fde4c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 1 -- 10.20.193.78:0/4079619424 >> v1:10.20.193.115:6789/0 conn(0x7fde4c06c890 legacy=0x7fde4c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 1 monclient: found mon.noname-b 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.248+0000 7fde4a7fc640 1 -- 10.20.193.78:0/4079619424 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c09cf60 con 0x7fde4c09c4a0 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde4a7fc640 1 -- 10.20.193.78:0/4079619424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 msgr2=0x7fde4c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde4a7fc640 1 --2- 10.20.193.78:0/4079619424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13b3a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde34009970 tx=0x7fde3402ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae61b2afe6b076f0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 1 -- 10.20.193.78:0/4079619424 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3403c070 con 0x7fde4c09c4a0 2026-02-17T05:54:40.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: dump: 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:40.261 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: _finish_auth 0 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 1 -- 10.20.193.78:0/4079619424 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3402fcd0 con 0x7fde4c09c4a0 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde5135b640 5 monclient: authenticate success, global_id 74525 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 1 -- 10.20.193.78:0/4079619424 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34037660 con 0x7fde4c09c4a0 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde49ffb640 10 monclient: dump: 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:40.262 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde5135b640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.250+0000 7fde5135b640 4 set_mon_vals no callback set 2026-02-17T05:54:40.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 10 monclient: shutdown 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 -- 10.20.193.78:0/4079619424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 msgr2=0x7fde4c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 --2- 10.20.193.78:0/4079619424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13b3a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde34009970 tx=0x7fde3402ee90 comp rx=0 tx=0).stop 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 -- 10.20.193.78:0/4079619424 shutdown_connections 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 --2- 10.20.193.78:0/4079619424 >> v2:10.20.193.115:3300/0 conn(0x7fde4c06d380 0x7fde4c13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 --2- 10.20.193.78:0/4079619424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13b3a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 -- 10.20.193.78:0/4079619424 >> 10.20.193.78:0/4079619424 conn(0x7fde4c0a6620 msgr2=0x7fde4c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 -- 10.20.193.78:0/4079619424 shutdown_connections 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 -- 10.20.193.78:0/4079619424 wait complete. 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 10 monclient: build_initial_monmap 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 10 monclient: monmap: 2026-02-17T05:54:40.264 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 Processor -- start 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 1 -- start start 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.251+0000 7fde5135b640 10 monclient: init 2026-02-17T05:54:40.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fde4c06c890 0x7fde4c1cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient(hunting): picked mon.noname-b con 0x7fde4c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fde4c06d380 0x7fde4c1331c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient(hunting): picked mon.noname-a con 0x7fde4c06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient(hunting): picked mon.noname-c con 0x7fde4c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c09bfa0 con 0x7fde4c06d380 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c09be20 con 0x7fde4c06c890 2026-02-17T05:54:40.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c13de30 con 0x7fde4c09c4a0 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde5135b640 10 monclient(hunting): authenticate will time out at 1125.806055s 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4a7fc640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fde4c06d380 msgr2=0x7fde4c1331c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4a7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fde4c06d380 0x7fde4c1331c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49228/0 (socket says 10.20.193.78:49228) 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 -- 10.20.193.78:0/2132402579 learned_addr learned my addr 10.20.193.78:0/2132402579 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient(hunting): get_auth_request con 0x7fde4c09c4a0 auth_method 0 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:40.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient(hunting): handle_auth_done global_id 74528 payload 824 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 -- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fde4c06c890 msgr2=0x7fde4c1cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4affd640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fde4c06c890 0x7fde4c1cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fde4c06c890 0x7fde4c1cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 -- 10.20.193.78:0/2132402579 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fde4c06d380 msgr2=0x7fde4c1331c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fde4c06d380 0x7fde4c1331c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 monclient: found mon.noname-c 2026-02-17T05:54:40.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4b7fe640 1 -- 10.20.193.78:0/2132402579 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c11db50 con 0x7fde4c09c4a0 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4affd640 10 monclient: get_auth_request con 0x7fde4c06c890 auth_method 0 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.252+0000 7fde4affd640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fde4c06c890 0x7fde4c1cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde4b7fe640 1 -- 10.20.193.78:0/2132402579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 msgr2=0x7fde4c13a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde4b7fe640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13a480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde3c007d90 tx=0x7fde3c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 1 -- 10.20.193.78:0/2132402579 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c013790 con 0x7fde4c09c4a0 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: dump: 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:40.269 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:40.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 1 -- 10.20.193.78:0/2132402579 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c019070 con 0x7fde4c09c4a0 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 1 -- 10.20.193.78:0/2132402579 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c013b90 con 0x7fde4c09c4a0 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde497fa640 4 set_mon_vals no callback set 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde2b7fe640 10 monclient: dump: 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:40.271 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde5135b640 5 monclient: authenticate success, global_id 74528 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde5135b640 10 monclient: _renew_subs 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde5135b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c13d9d0 con 0x7fde4c09c4a0 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde5135b640 10 monclient: _renew_subs 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde5135b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.254+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c11d590 con 0x7fde4c09c4a0 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde2b7fe640 1 -- 10.20.193.78:0/2132402579 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fde3c01e460 con 0x7fde4c09c4a0 2026-02-17T05:54:40.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde5135b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde5135b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde5135b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c13d7b0 con 0x7fde4c09c4a0 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde2b7fe640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde1803de60 0x7fde18040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde2b7fe640 1 -- 10.20.193.78:0/2132402579 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(308..308 src has 117..308) ==== 4735+0+0 (secure 0 0 0) 0x7fde3c053110 con 0x7fde4c09c4a0 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde4affd640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde1803de60 0x7fde18040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.255+0000 7fde4affd640 10 monclient: get_auth_request con 0x7fde1803de60 auth_method 0 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.257+0000 7fde2b7fe640 1 -- 10.20.193.78:0/2132402579 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fde4c13d7b0 con 0x7fde4c09c4a0 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.257+0000 7fde2b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:40.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.257+0000 7fde2b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:40.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.258+0000 7fde4affd640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde1803de60 0x7fde18040320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fde40004020 tx=0x7fde40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.378+0000 7fde5135b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:54:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.378+0000 7fde5135b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.378+0000 7fde5135b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.378+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fde4c074980 con 0x7fde4c09c4a0 2026-02-17T05:54:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.378+0000 7fde2b7fe640 1 -- 10.20.193.78:0/2132402579 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fde3c051b40 con 0x7fde4c09c4a0 2026-02-17T05:54:40.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.378+0000 7fde2b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:40.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.378+0000 7fde2b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:40.379 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:54:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde1803de60 msgr2=0x7fde18040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde1803de60 0x7fde18040320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fde40004020 tx=0x7fde40009040 comp rx=0 tx=0).stop 2026-02-17T05:54:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 10 monclient: shutdown 2026-02-17T05:54:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:40.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 msgr2=0x7fde4c13a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13a480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde3c007d90 tx=0x7fde3c00e4b0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 shutdown_connections 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fde4c09c4a0 0x7fde4c13a480 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fde4c06d380 0x7fde4c1331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde1803de60 0x7fde18040320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 --2- 10.20.193.78:0/2132402579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fde4c06c890 0x7fde4c1cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 >> 10.20.193.78:0/2132402579 conn(0x7fde4c0a6620 msgr2=0x7fde4c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 shutdown_connections 2026-02-17T05:54:40.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:40.379+0000 7fde5135b640 1 -- 10.20.193.78:0/2132402579 wait complete. 2026-02-17T05:54:43.387 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:54:43.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.457+0000 7f949006f640 10 monclient: get_monmap_and_config 2026-02-17T05:54:43.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.457+0000 7f949006f640 10 monclient: build_initial_monmap 2026-02-17T05:54:43.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.457+0000 7f949006f640 10 monclient: monmap: 2026-02-17T05:54:43.459 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:43.459 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:43.459 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:43.458997+0000 2026-02-17T05:54:43.459 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:43.458997+0000 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 1 Processor -- start 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 1 -- start start 2026-02-17T05:54:43.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient: init 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient: _add_conns ranks=[4,0,3,2,1] 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient(hunting): picked mon.noname-c con 0x7f948806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f948809d430 addr v1:10.20.193.78:6789/0 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 0x7f948806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient(hunting): picked mon.noname-a con 0x7f948809bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9488073600 con 0x7f948809d430 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f948809ca50 con 0x7f948809bf40 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:43.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f948806d6f0 con 0x7f948806c890 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f949006f640 10 monclient(hunting): authenticate will time out at 1129.012598s 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948d5e3640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f948809d430 legacy=0x7f9488134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948e5e5640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 msgr2=0x7f948806d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948e5e5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 0x7f948806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:44802/0 (socket says 10.20.193.78:44802) 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 learned_addr learned my addr 10.20.193.78:0/3795444577 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 10 monclient(hunting): get_auth_request con 0x7f948806c890 auth_method 0 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:43.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:43.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:43.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:43.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:43.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.458+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:43.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948e5e5640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 msgr2=0x7f948806d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:43.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948e5e5640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 0x7f948806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:43.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> v1:10.20.193.78:6789/0 conn(0x7f948809d430 legacy=0x7f9488134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:54:43.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:43.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 10 monclient(hunting): get_auth_request con 0x7f948806c890 auth_method 0 2026-02-17T05:54:43.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:43.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:43.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:43.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:43.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:43.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:43.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:43.659+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:44.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.059+0000 7f948e5e5640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 msgr2=0x7f948806d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:44.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.059+0000 7f948e5e5640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 0x7f948806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:44.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.059+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> v1:10.20.193.78:6789/0 conn(0x7f948809d430 legacy=0x7f9488134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:54:44.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:44.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 10 monclient(hunting): get_auth_request con 0x7f948806c890 auth_method 0 2026-02-17T05:54:44.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:44.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:44.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:44.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:44.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:44.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:44.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.060+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:44.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.860+0000 7f948e5e5640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 msgr2=0x7f948806d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:44.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.860+0000 7f948e5e5640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 0x7f948806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:44.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.860+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> v1:10.20.193.78:6789/0 conn(0x7f948809d430 legacy=0x7f9488134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 10 monclient(hunting): get_auth_request con 0x7f948806c890 auth_method 0 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:44.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:44.861+0000 7f948dde4640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:46.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 10 monclient(hunting): tick 2026-02-17T05:54:46.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 msgr2=0x7f948806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.115:3300/0 conn(0x7f948806c890 0x7f948806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 -- 10.20.193.78:0/3795444577 >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 msgr2=0x7f948806d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 --2- 10.20.193.78:0/3795444577 >> v2:10.20.193.78:3300/0 conn(0x7f948809bf40 0x7f948806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 -- 10.20.193.78:0/3795444577 >> v1:10.20.193.78:6789/0 conn(0x7f948809d430 legacy=0x7f9488134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 10 monclient: _add_conns ranks=[1,2] 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f946c003770 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:46.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 10 monclient(hunting): picked mon.noname-b con 0x7f946c007460 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 -- 10.20.193.78:0/3795444577 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948806dab0 con 0x7f946c003770 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 1 -- 10.20.193.78:0/3795444577 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f948809ca50 con 0x7f946c007460 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f94777fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 10 monclient(hunting): get_auth_request con 0x7f946c007460 auth_method 0 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:46.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:46.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.459+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:46.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.461+0000 7f948dde4640 1 -- 10.20.193.78:0/3795444577 >> v1:10.20.193.115:6789/0 conn(0x7f946c003770 legacy=0x7f946c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:46.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:46.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 10 monclient(hunting): get_auth_request con 0x7f946c007460 auth_method 0 2026-02-17T05:54:46.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:46.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:46.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:46.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:46.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:46.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:46.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:46.659+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 10 monclient(hunting): get_auth_request con 0x7f946c007460 auth_method 0 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:47.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.059+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:54:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 10 monclient(hunting): get_auth_request con 0x7f946c007460 auth_method 0 2026-02-17T05:54:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:54:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.861+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:54:47.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.942+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3836262978 0 0) 0x7f948806dab0 con 0x7f946c003770 2026-02-17T05:54:47.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.942+0000 7f948cde2640 10 monclient(hunting): _negotiate 2026-02-17T05:54:47.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.942+0000 7f948cde2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:47.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.942+0000 7f948cde2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.942+0000 7f948cde2640 10 monclient(hunting): my global_id is 114142 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.942+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9470003620 con 0x7f946c003770 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3507667500 0 0) 0x7f9470003620 con 0x7f946c003770 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 msgr2=0x7f946c009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f946c007460 0x7f946c009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: _finish_hunting 0 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:54:47.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9488073600 con 0x7f946c003770 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: _finish_auth 0 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: _check_auth_tickets 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f949006f640 5 monclient: authenticate success, global_id 114142 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f949006f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f9478009670 con 0x7f946c003770 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: dump: 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:47.954 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 >> v1:10.20.193.115:6789/0 conn(0x7f946c003770 legacy=0x7f946c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948e5e5640 1 -- 10.20.193.78:0/3795444577 reap_dead start 2026-02-17T05:54:47.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9470003ab0 0x7f9470003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient(hunting): picked mon.c con 0x7f9470003ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9470004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient(hunting): picked mon.b con 0x7f9470004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470005310 0x7f947000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient(hunting): picked mon.a con 0x7f9470005310 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f948809ca50 con 0x7f9470005310 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f947000c240 con 0x7f9470004900 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:47.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f947000c440 con 0x7f9470003ab0 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948cde2640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f949006f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948e5e5640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470005310 msgr2=0x7f947000bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948e5e5640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470005310 0x7f947000bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9470004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948d5e3640 10 monclient(hunting): get_auth_request con 0x7f9470004900 auth_method 0 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948d5e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948d5e3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.943+0000 7f948d5e3640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:47.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 10 monclient(hunting): handle_auth_done global_id 114142 payload 293 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9470003ab0 msgr2=0x7f9470003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9470003ab0 0x7f9470003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470005310 msgr2=0x7f947000bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470005310 0x7f947000bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 10 monclient: _finish_hunting 0 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 1 monclient: found mon.b 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9470003620 con 0x7f9470004900 2026-02-17T05:54:47.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 10 monclient: _finish_auth 0 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 10 monclient: _check_auth_tickets 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.944+0000 7f948d5e3640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948d5e3640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 msgr2=0x7f9470004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948d5e3640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9470004d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f947c003a80 tx=0x7f947c02ff20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52f17dc98d2dd46c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c036070 con 0x7f9470004900 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 10 monclient: dump: 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.959 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f949006f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f947c030950 con 0x7f9470004900 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 1 -- 10.20.193.78:0/3795444577 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c009ad0 con 0x7f9470004900 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f949006f640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:47.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f949006f640 4 set_mon_vals no callback set 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.946+0000 7f948cde2640 10 monclient: dump: 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:47.961 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 10 monclient: shutdown 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 -- 10.20.193.78:0/3795444577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 msgr2=0x7f9470004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9470004d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f947c003a80 tx=0x7f947c02ff20 comp rx=0 tx=0).stop 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 -- 10.20.193.78:0/3795444577 shutdown_connections 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470005310 0x7f947000bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9470004d80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 --2- 10.20.193.78:0/3795444577 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9470003ab0 0x7f9470003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 -- 10.20.193.78:0/3795444577 >> 10.20.193.78:0/3795444577 conn(0x7f94880a6620 msgr2=0x7f94880a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 -- 10.20.193.78:0/3795444577 shutdown_connections 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 1 -- 10.20.193.78:0/3795444577 wait complete. 2026-02-17T05:54:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 10 monclient: build_initial_monmap 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.947+0000 7f949006f640 10 monclient: monmap: 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 1 Processor -- start 2026-02-17T05:54:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 1 -- start start 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient: init 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470003ab0 0x7f94881cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient(hunting): picked mon.noname-a con 0x7f9470003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9488133010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient(hunting): picked mon.noname-b con 0x7f9470004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9470005310 0x7f9488133550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient(hunting): picked mon.noname-c con 0x7f9470005310 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f948809c7d0 con 0x7f9470003ab0 2026-02-17T05:54:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9488073410 con 0x7f9470004900 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f948806cc70 con 0x7f9470005310 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f949006f640 10 monclient(hunting): authenticate will time out at 1133.502315s 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948dde4640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470003ab0 msgr2=0x7f94881cea20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948dde4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470003ab0 0x7f94881cea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9488133010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9488133010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:57234/0 (socket says 10.20.193.78:57234) 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 -- 10.20.193.78:0/4222523988 learned_addr learned my addr 10.20.193.78:0/4222523988 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient(hunting): get_auth_request con 0x7f9470004900 auth_method 0 2026-02-17T05:54:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient(hunting): handle_auth_done global_id 114145 payload 824 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 -- 10.20.193.78:0/4222523988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9470005310 msgr2=0x7f9488133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9470005310 0x7f9488133550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 -- 10.20.193.78:0/4222523988 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470003ab0 msgr2=0x7f94881cea20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:54:47.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470003ab0 0x7f94881cea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient: _finish_hunting 0 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 monclient: found mon.noname-b 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.115:3300/0 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.948+0000 7f948d5e3640 1 -- 10.20.193.78:0/4222523988 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94881206f0 con 0x7f9470004900 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.950+0000 7f948d5e3640 1 -- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 msgr2=0x7f9488133010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.950+0000 7f948d5e3640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9488133010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f947c03c040 tx=0x7f947c005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 1 -- 10.20.193.78:0/4222523988 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c049070 con 0x7f9470004900 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: dump: 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:47.967 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:47.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 1 -- 10.20.193.78:0/4222523988 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f947c045860 con 0x7f9470004900 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 1 -- 10.20.193.78:0/4222523988 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c009ad0 con 0x7f9470004900 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f94767fc640 10 monclient: dump: 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:47.969 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f9477fff640 4 set_mon_vals no callback set 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 5 monclient: authenticate success, global_id 114145 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 10 monclient: _renew_subs 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94881cef60 con 0x7f9470004900 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 10 monclient: _renew_subs 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948809d810 con 0x7f9470004900 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:47.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:47.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:54:47.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.951+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948806cfa0 con 0x7f9470004900 2026-02-17T05:54:47.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.952+0000 7f94767fc640 1 -- 10.20.193.78:0/4222523988 <== mon.1 v2:10.20.193.115:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f947c045c90 con 0x7f9470004900 2026-02-17T05:54:47.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.952+0000 7f94767fc640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f945803deb0 0x7f9458040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:47.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.952+0000 7f94767fc640 1 -- 10.20.193.78:0/4222523988 <== mon.1 v2:10.20.193.115:3300/0 5 ==== osd_map(308..308 src has 117..308) ==== 4735+0+0 (secure 0 0 0) 0x7f947c078700 con 0x7f9470004900 2026-02-17T05:54:47.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.952+0000 7f948dde4640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f945803deb0 0x7f9458040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:47.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.952+0000 7f948dde4640 10 monclient: get_auth_request con 0x7f945803deb0 auth_method 0 2026-02-17T05:54:47.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.954+0000 7f948dde4640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f945803deb0 0x7f9458040370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94780097e0 tx=0x7f9478006b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:47.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.955+0000 7f94767fc640 1 -- 10.20.193.78:0/4222523988 <== mon.1 v2:10.20.193.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f947c043410 con 0x7f9470004900 2026-02-17T05:54:47.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.955+0000 7f94767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:47.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:47.955+0000 7f94767fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:48.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.069+0000 7f949006f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:54:48.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.069+0000 7f949006f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:48.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.069+0000 7f949006f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:54:48.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.069+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f94880744d0 con 0x7f9470004900 2026-02-17T05:54:48.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.069+0000 7f94767fc640 1 -- 10.20.193.78:0/4222523988 <== mon.1 v2:10.20.193.115:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f947c056350 con 0x7f9470004900 2026-02-17T05:54:48.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.069+0000 7f94767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:54:48.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.069+0000 7f94767fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:48.070 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":90,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:54:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f945803deb0 msgr2=0x7f9458040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f945803deb0 0x7f9458040370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94780097e0 tx=0x7f9478006b60 comp rx=0 tx=0).stop 2026-02-17T05:54:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 10 monclient: shutdown 2026-02-17T05:54:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 msgr2=0x7f9488133010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9488133010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f947c03c040 tx=0x7f947c005e00 comp rx=0 tx=0).stop 2026-02-17T05:54:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 shutdown_connections 2026-02-17T05:54:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9470005310 0x7f9488133550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f945803deb0 0x7f9458040370 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9470004900 0x7f9488133010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 --2- 10.20.193.78:0/4222523988 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9470003ab0 0x7f94881cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 >> 10.20.193.78:0/4222523988 conn(0x7f94880a6620 msgr2=0x7f9488137650 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 shutdown_connections 2026-02-17T05:54:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.070+0000 7f949006f640 1 -- 10.20.193.78:0/4222523988 wait complete. 2026-02-17T05:54:48.078 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:54:48.078 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:54:48.078 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:54:48.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient: get_monmap_and_config 2026-02-17T05:54:48.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient: build_initial_monmap 2026-02-17T05:54:48.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient: monmap: 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:48.148254+0000 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:48.148254+0000 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:48.147 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:48.148 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:48.148 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:48.148 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 1 Processor -- start 2026-02-17T05:54:48.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 1 -- start start 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient: init 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f427409ea90 0x7f42740a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient(hunting): picked mon.noname-a con 0x7f427409ea90 addr v2:10.20.193.78:3300/0 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient(hunting): picked mon.noname-b con 0x7f427409ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f427409d980 0x7f427409ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient(hunting): picked mon.noname-c con 0x7f427409d980 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f42740a0d00 con 0x7f427409ea90 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f42740a0a20 con 0x7f427409d980 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f42740a0460 con 0x7f427409ee70 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427c922640 10 monclient(hunting): authenticate will time out at 1133.701946s 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.147+0000 7f427a697640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f427409ea90 msgr2=0x7f42740a13d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f427a697640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f427409ea90 0x7f42740a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:48.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37542/0 (socket says 10.20.193.78:37542) 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 learned_addr learned my addr 10.20.193.78:0/4024662981 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient(hunting): get_auth_request con 0x7f427409ee70 auth_method 0 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f427ae98640 1 --2- 10.20.193.78:0/4024662981 >> v2:10.20.193.115:3300/0 conn(0x7f427409d980 0x7f427409ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:48.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f427ae98640 10 monclient(hunting): get_auth_request con 0x7f427409d980 auth_method 0 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f427ae98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f427ae98640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f427ae98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient(hunting): handle_auth_done global_id 74531 payload 274 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 >> v2:10.20.193.115:3300/0 conn(0x7f427409d980 msgr2=0x7f427409ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> v2:10.20.193.115:3300/0 conn(0x7f427409d980 0x7f427409ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 >> v2:10.20.193.78:3300/0 conn(0x7f427409ea90 msgr2=0x7f42740a13d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> v2:10.20.193.78:3300/0 conn(0x7f427409ea90 0x7f42740a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient: _finish_hunting 0 2026-02-17T05:54:48.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 monclient: found mon.noname-b 2026-02-17T05:54:48.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:48.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:54:48.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42740a0740 con 0x7f427409ee70 2026-02-17T05:54:48.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 msgr2=0x7f427409f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send send error: (32) Broken pipe 2026-02-17T05:54:48.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f42740a0160 tx=0x7f426002ee80 comp rx=0 tx=0).write compression request write failed r=-32 ((32) Broken pipe) 2026-02-17T05:54:48.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.148+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:48.212 INFO:tasks.rados.rados.0.trial115.stdout:1463: done (2 left) 2026-02-17T05:54:48.212 INFO:tasks.rados.rados.0.trial115.stdout:1464: done (1 left) 2026-02-17T05:54:48.212 INFO:tasks.rados.rados.0.trial115.stdout:1465: done (0 left) 2026-02-17T05:54:48.212 INFO:tasks.rados.rados.0.trial115.stdout:1466: write oid 24 current snap is 144 2026-02-17T05:54:48.212 INFO:tasks.rados.rados.0.trial115.stdout:1466: seq_num 555 ranges {527885=656964,1927271=483852,3191408=136477} 2026-02-17T05:54:48.216 INFO:tasks.rados.rados.0.trial115.stdout:1466: writing trial11545420-24 from 527885 to 1184849 tid 1 2026-02-17T05:54:48.221 INFO:tasks.rados.rados.0.trial115.stdout:1466: writing trial11545420-24 from 1927271 to 2411123 tid 2 2026-02-17T05:54:48.224 INFO:tasks.rados.rados.0.trial115.stdout:1466: writing trial11545420-24 from 3191408 to 3327885 tid 3 2026-02-17T05:54:48.224 INFO:tasks.rados.rados.0.trial115.stdout:1467: write (excl) oid 13 current snap is 144 2026-02-17T05:54:48.225 INFO:tasks.rados.rados.0.trial115.stdout:1467: seq_num 556 ranges {544692=732213,1985753=743863,3335184=9508} 2026-02-17T05:54:48.229 INFO:tasks.rados.rados.0.trial115.stdout:1467: writing trial11545420-13 from 544692 to 1276905 tid 1 2026-02-17T05:54:48.235 INFO:tasks.rados.rados.0.trial115.stdout:1467: writing trial11545420-13 from 1985753 to 2729616 tid 2 2026-02-17T05:54:48.237 INFO:tasks.rados.rados.0.trial115.stdout:1467: writing trial11545420-13 from 3335184 to 3344692 tid 3 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1468: snap_create 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: finishing write tid 1 to trial11545420-24 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: finishing write tid 2 to trial11545420-24 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: finishing write tid 3 to trial11545420-24 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: finishing write tid 4 to trial11545420-24 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: finishing write tid 5 to trial11545420-24 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: oid 24 updating version 0 to 819 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: oid 24 version 819 is already newer than 818 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: oid 24 updating version 819 to 820 2026-02-17T05:54:48.238 INFO:tasks.rados.rados.0.trial115.stdout:1466: oid 24 version 820 is already newer than 817 2026-02-17T05:54:48.239 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 820 (ObjNum 555 snap 144 seq_num 555) dirty exists 2026-02-17T05:54:48.239 INFO:tasks.rados.rados.0.trial115.stdout:1466: left oid 24 (ObjNum 555 snap 144 seq_num 555) 2026-02-17T05:54:48.239 INFO:tasks.rados.rados.0.trial115.stdout:1467: finishing write tid 1 to trial11545420-13 2026-02-17T05:54:48.239 INFO:tasks.rados.rados.0.trial115.stdout:1467: finishing write tid 2 to trial11545420-13 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:1467: finishing write tid 3 to trial11545420-13 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:1467: finishing write tid 4 to trial11545420-13 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:1467: finishing write tid 5 to trial11545420-13 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:1467: oid 13 updating version 0 to 541 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:1467: oid 13 updating version 541 to 542 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:1467: oid 13 updating version 542 to 543 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:1467: oid 13 version 543 is already newer than 540 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 543 (ObjNum 556 snap 144 seq_num 556) dirty exists 2026-02-17T05:54:48.240 INFO:tasks.rados.rados.0.trial115.stdout:1467: left oid 13 (ObjNum 556 snap 144 seq_num 556) 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279e96640 10 monclient: get_auth_request con 0x7f427409ee70 auth_method 0 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279e96640 0 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 10 monclient: ms_handle_reset current mon [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 msgr2=0x7f427409f290 unknown :-1 s=STATE_CLOSED l=0).mark_down 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427409ee70 0x7f427409f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 10 monclient: _add_conns ranks=[1,0] 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4264003600 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4264007590 addr v1:10.20.193.78:6789/0 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f426400b500 con 0x7f4264007590 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f426400c700 con 0x7f4264003600 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279695640 10 monclient(hunting): ms_handle_reset stray mon [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:48.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.348+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 >> v1:10.20.193.78:6789/0 conn(0x7f4264007590 legacy=0x7f4264009a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:54:48.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 1 -- 10.20.193.78:0/4024662981 >> v1:10.20.193.115:6789/0 conn(0x7f4264003600 legacy=0x7f4264005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590132989 0 0) 0x7f426400c700 con 0x7f4264003600 2026-02-17T05:54:48.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): _negotiate 2026-02-17T05:54:48.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:48.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 --> v1:10.20.193.115:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f426400c8d0 con 0x7f4264003600 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 3852731897 0 0) 0x7f426400c8d0 con 0x7f4264003600 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 >> v1:10.20.193.78:6789/0 conn(0x7f4264007590 legacy=0x7f4264009a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: _finish_hunting 0 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426400da10 con 0x7f4264003600 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: _finish_auth 0 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: _check_auth_tickets 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f426c004420 con 0x7f4264003600 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: dump: 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.351 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 >> v1:10.20.193.115:6789/0 conn(0x7f4264003600 legacy=0x7f4264005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427ae98640 1 -- 10.20.193.78:0/4024662981 reap_dead start 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 0x7f42640099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): picked mon.c con 0x7f426400d000 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 0x7f426400b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): picked mon.b con 0x7f426400d3e0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400bde0 0x7f426400a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): picked mon.a con 0x7f426400bde0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f426400a740 con 0x7f426400bde0 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f426400a8f0 con 0x7f426400d3e0 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f426400aaf0 con 0x7f426400d000 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279695640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427ae98640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400bde0 msgr2=0x7f426400a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 0x7f42640099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427ae98640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400bde0 0x7f426400a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 10 monclient(hunting): get_auth_request con 0x7f426400d000 auth_method 0 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 msgr2=0x7f42640099d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 msgr2=0x7f42640099d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 0x7f42640099d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f427a697640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 0x7f42640099d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 0x7f426400b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279e96640 10 monclient(hunting): get_auth_request con 0x7f426400d3e0 auth_method 0 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279e96640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.351+0000 7f4279e96640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 10 monclient(hunting): handle_auth_done global_id 74531 payload 293 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 msgr2=0x7f42640099d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 0x7f42640099d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400bde0 msgr2=0x7f426400a200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400bde0 0x7f426400a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 10 monclient: _finish_hunting 0 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 1 monclient: found mon.b 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426400c8d0 con 0x7f426400d3e0 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 10 monclient: _finish_auth 0 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 10 monclient: _check_auth_tickets 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f4279e96640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f427c922640 5 monclient: authenticate success, global_id 74531 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.352+0000 7f427c922640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279e96640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 msgr2=0x7f426400b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279e96640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 0x7f426400b6d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f42740a0160 tx=0x7f426400cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c959c59c2415521e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426003d020 con 0x7f426400d3e0 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 10 monclient: dump: 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f427c922640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4260004230 con 0x7f426400d3e0 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 1 -- 10.20.193.78:0/4024662981 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426002fc90 con 0x7f426400d3e0 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f4279695640 10 monclient: dump: 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f427c922640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:48.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.354+0000 7f427c922640 4 set_mon_vals no callback set 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 10 monclient: shutdown 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 -- 10.20.193.78:0/4024662981 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 msgr2=0x7f426400b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 0x7f426400b6d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f42740a0160 tx=0x7f426400cb10 comp rx=0 tx=0).stop 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 -- 10.20.193.78:0/4024662981 shutdown_connections 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400bde0 0x7f426400a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 0x7f426400b6d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 --2- 10.20.193.78:0/4024662981 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400d000 0x7f42640099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 -- 10.20.193.78:0/4024662981 >> 10.20.193.78:0/4024662981 conn(0x7f42740a7520 msgr2=0x7f42740a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 -- 10.20.193.78:0/4024662981 shutdown_connections 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.355+0000 7f427c922640 1 -- 10.20.193.78:0/4024662981 wait complete. 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient: build_initial_monmap 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient: monmap: 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:48.355 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 1 Processor -- start 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 1 -- start start 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient: init 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400bde0 0x7f4274072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient(hunting): picked mon.noname-c con 0x7f426400bde0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400d000 0x7f4274072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient(hunting): picked mon.noname-a con 0x7f426400d000 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 0x7f4274073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient(hunting): picked mon.noname-b con 0x7f426400d3e0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42740a1550 con 0x7f426400d000 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42740a13d0 con 0x7f426400d3e0 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f42740a16d0 con 0x7f426400bde0 2026-02-17T05:54:48.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427c922640 10 monclient(hunting): authenticate will time out at 1133.910494s 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f4279e96640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400d000 msgr2=0x7f4274072ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f4279e96640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400d000 0x7f4274072ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400bde0 0x7f4274072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400bde0 0x7f4274072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37576/0 (socket says 10.20.193.78:37576) 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 -- 10.20.193.78:0/2519576744 learned_addr learned my addr 10.20.193.78:0/2519576744 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient(hunting): get_auth_request con 0x7f426400bde0 auth_method 0 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427ae98640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 0x7f4274073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427ae98640 10 monclient(hunting): get_auth_request con 0x7f426400d3e0 auth_method 0 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427ae98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427ae98640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427ae98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient(hunting): handle_auth_done global_id 74534 payload 824 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 msgr2=0x7f4274073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f426400d3e0 0x7f4274073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400d000 msgr2=0x7f4274072ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f426400d000 0x7f4274072ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient: _finish_hunting 0 2026-02-17T05:54:48.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 monclient: found mon.noname-c 2026-02-17T05:54:48.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:48.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:54:48.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.356+0000 7f427a697640 1 -- 10.20.193.78:0/2519576744 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4274131d90 con 0x7f426400bde0 2026-02-17T05:54:48.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f427a697640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400bde0 msgr2=0x7f4274072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f427a697640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400bde0 0x7f4274072ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f426c003900 tx=0x7f426c00eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c015030 con 0x7f426400bde0 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 10 monclient: dump: 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 10 monclient: _finish_auth 0 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 10 monclient: _check_auth_tickets 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c00c9b0 con 0x7f426400bde0 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f427c922640 5 monclient: authenticate success, global_id 74534 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f427c922640 10 monclient: _renew_subs 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f427c922640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4274132070 con 0x7f426400bde0 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:48.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c00cda0 con 0x7f426400bde0 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f425effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f425effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f425effd640 10 monclient: dump: 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.358+0000 7f4278e94640 4 set_mon_vals no callback set 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f427c922640 10 monclient: _renew_subs 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f427c922640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42740a8e80 con 0x7f426400bde0 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f427c922640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f427c922640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406ef90 0x7f427406f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f427c922640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f42740a1c10 con 0x7f427406ef90 2026-02-17T05:54:48.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406ef90 0x7f427406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f426c011450 con 0x7f426400bde0 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 10 monclient: get_auth_request con 0x7f427406ef90 auth_method 0 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 10 monclient(hunting): handle_auth_done global_id 74537 payload 824 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f4279e96640 1 -- 10.20.193.78:0/2519576744 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42740abb10 con 0x7f427406ef90 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f425effd640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f424403de60 0x7f4244040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.359+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(309..309 src has 117..309) ==== 4735+0+0 (secure 0 0 0) 0x7f426c052bf0 con 0x7f426400bde0 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.360+0000 7f427ae98640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f424403de60 0x7f4244040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.360+0000 7f427ae98640 10 monclient: get_auth_request con 0x7f424403de60 auth_method 0 2026-02-17T05:54:48.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f4279e96640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406ef90 msgr2=0x7f427406f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f4279e96640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406ef90 0x7f427406f370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42740a05e0 tx=0x7f42600047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426003d050 con 0x7f427406ef90 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 10 monclient: dump: 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4260046070 con 0x7f427406ef90 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406ef90 msgr2=0x7f427406f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406ef90 0x7f427406f370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42740a05e0 tx=0x7f42600047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406ef90 msgr2=0x7f427406f370 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f425effd640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406ef90 0x7f427406f370 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42740a05e0 tx=0x7f42600047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.362+0000 7f427ae98640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f424403de60 0x7f4244040320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4274074810 tx=0x7f426800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f427c922640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f427c922640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406f8b0 0x7f42741d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f427c922640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f42740744d0 con 0x7f427406f8b0 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406f8b0 0x7f42741d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 10 monclient: get_auth_request con 0x7f427406f8b0 auth_method 0 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 10 monclient(hunting): handle_auth_done global_id 74540 payload 824 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.365+0000 7f4279e96640 1 -- 10.20.193.78:0/2519576744 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4274073a70 con 0x7f427406f8b0 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f4279e96640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406f8b0 msgr2=0x7f42741d93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f4279e96640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406f8b0 0x7f42741d93b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f42740a1d90 tx=0x7f4260008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260002aa0 con 0x7f427406f8b0 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 10 monclient: dump: 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.366 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7f4260002aa0 con 0x7f427406f8b0 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406f8b0 msgr2=0x7f42741d93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406f8b0 0x7f42741d93b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f42740a1d90 tx=0x7f4260008040 comp rx=0 tx=0).stop 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406f8b0 msgr2=0x7f42741d93b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.367+0000 7f425effd640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f427406f8b0 0x7f42741d93b0 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f42740a1d90 tx=0x7f4260008040 comp rx=0 tx=0).stop 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 90, 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 260260, 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:48.367 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:48.368 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.369 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:48.370 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f424403de60 msgr2=0x7f4244040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f424403de60 0x7f4244040320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4274074810 tx=0x7f426800b040 comp rx=0 tx=0).stop 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 10 monclient: shutdown 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400bde0 msgr2=0x7f4274072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400bde0 0x7f4274072ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f426c003900 tx=0x7f426c00eb60 comp rx=0 tx=0).stop 2026-02-17T05:54:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427ae98640 1 -- 10.20.193.78:0/2519576744 reap_dead start 2026-02-17T05:54:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 shutdown_connections 2026-02-17T05:54:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 --2- 10.20.193.78:0/2519576744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f426400bde0 0x7f4274072ab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 >> 10.20.193.78:0/2519576744 conn(0x7f42740a7520 msgr2=0x7f42740a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 shutdown_connections 2026-02-17T05:54:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.368+0000 7f427c922640 1 -- 10.20.193.78:0/2519576744 wait complete. 2026-02-17T05:54:48.373 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:54:48.373 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:54:48.373 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:54:48.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.440+0000 7f7157b79640 10 monclient: get_monmap_and_config 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.440+0000 7f7157b79640 10 monclient: build_initial_monmap 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.440+0000 7f7157b79640 10 monclient: monmap: 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:54:48.441866+0000 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:54:48.441866+0000 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:54:48.440 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 1 Processor -- start 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 1 -- start start 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient: init 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f715006c890 0x7f71500a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient(hunting): picked mon.noname-c con 0x7f715006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f715006cc70 addr v1:10.20.193.115:6789/0 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f715013a240 addr v1:10.20.193.78:6789/0 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7150074150 con 0x7f715013a240 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715009ef70 con 0x7f715006cc70 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f715009f250 con 0x7f715006c890 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f7157b79640 10 monclient(hunting): authenticate will time out at 1133.995595s 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71560ef640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f715013a240 legacy=0x7f715013c630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71558ee640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f715006c890 0x7f71500a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71550ed640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f715006cc70 0x7f715006d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:52572/0 (socket says 10.20.193.78:52572) 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71550ed640 1 -- 10.20.193.78:0/3717752579 learned_addr learned my addr 10.20.193.78:0/3717752579 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71558ee640 10 monclient(hunting): get_auth_request con 0x7f715006c890 auth_method 0 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71558ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71558ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71558ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.441+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.442+0000 7f71558ee640 10 monclient(hunting): handle_auth_done global_id 114154 payload 274 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.442+0000 7f71558ee640 1 -- 10.20.193.78:0/3717752579 >> v1:10.20.193.115:6789/0 conn(0x7f715006cc70 legacy=0x7f715006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.442+0000 7f71558ee640 1 -- 10.20.193.78:0/3717752579 >> v1:10.20.193.78:6789/0 conn(0x7f715013a240 legacy=0x7f715013c630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.442+0000 7f71558ee640 10 monclient: _finish_hunting 0 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.442+0000 7f71558ee640 1 monclient: found mon.noname-c 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.442+0000 7f71558ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:48.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.442+0000 7f71558ee640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:54:48.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.442+0000 7f71558ee640 1 -- 10.20.193.78:0/3717752579 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715009df40 con 0x7f715006c890 2026-02-17T05:54:48.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71558ee640 1 -- 10.20.193.78:0/3717752579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 msgr2=0x7f71500a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71558ee640 1 --2- 10.20.193.78:0/3717752579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 0x7f71500a01f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71400098a0 tx=0x7f714002ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f811e832dfe578 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 1 -- 10.20.193.78:0/3717752579 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714003c070 con 0x7f715006c890 2026-02-17T05:54:48.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:48.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: dump: 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: _finish_auth 0 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: _check_auth_tickets 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 1 -- 10.20.193.78:0/3717752579 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714002fd70 con 0x7f715006c890 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f7157b79640 5 monclient: authenticate success, global_id 114154 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f7157b79640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 1 -- 10.20.193.78:0/3717752579 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140037660 con 0x7f715006c890 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f71548ec640 10 monclient: dump: 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f7157b79640 10 monclient: get_monmap_and_config success 2026-02-17T05:54:48.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.444+0000 7f7157b79640 4 set_mon_vals no callback set 2026-02-17T05:54:48.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 10 monclient: shutdown 2026-02-17T05:54:48.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 -- 10.20.193.78:0/3717752579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 msgr2=0x7f71500a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 --2- 10.20.193.78:0/3717752579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 0x7f71500a01f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71400098a0 tx=0x7f714002ee10 comp rx=0 tx=0).stop 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 -- 10.20.193.78:0/3717752579 shutdown_connections 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 --2- 10.20.193.78:0/3717752579 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 0x7f71500a01f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 -- 10.20.193.78:0/3717752579 >> 10.20.193.78:0/3717752579 conn(0x7f71500a6620 msgr2=0x7f71500a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 -- 10.20.193.78:0/3717752579 shutdown_connections 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 -- 10.20.193.78:0/3717752579 wait complete. 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 10 monclient: build_initial_monmap 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 10 monclient: monmap: 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 Processor -- start 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 1 -- start start 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.445+0000 7f7157b79640 10 monclient: init 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient: _reopen_session rank -1 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 0x7f71501ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient(hunting): picked mon.noname-b con 0x7f715006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f715006cc70 0x7f7150131ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient(hunting): picked mon.noname-c con 0x7f715006cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f715013a240 0x7f715006f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient(hunting): picked mon.noname-a con 0x7f715013a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f715013ded0 con 0x7f715013a240 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f715013dd50 con 0x7f715006c890 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f715013e050 con 0x7f715006cc70 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient(hunting): _renew_subs 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f7157b79640 10 monclient(hunting): authenticate will time out at 1134.000323s 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71560ef640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f715013a240 msgr2=0x7f715006f9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:54:48.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71560ef640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f715013a240 0x7f715006f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f715006cc70 0x7f7150131ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f715006cc70 0x7f7150131ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37604/0 (socket says 10.20.193.78:37604) 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71558ee640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 0x7f71501ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 -- 10.20.193.78:0/2688366374 learned_addr learned my addr 10.20.193.78:0/2688366374 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient(hunting): get_auth_request con 0x7f715006cc70 auth_method 0 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71558ee640 10 monclient(hunting): get_auth_request con 0x7f715006c890 auth_method 0 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71558ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71558ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71558ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient(hunting): handle_auth_done global_id 74543 payload 824 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 msgr2=0x7f71501ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f715006c890 0x7f71501ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f715013a240 msgr2=0x7f715006f9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f715013a240 0x7f715006f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient: _finish_hunting 0 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 monclient: found mon.noname-c 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.446+0000 7f71550ed640 1 -- 10.20.193.78:0/2688366374 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7150073610 con 0x7f715006cc70 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f71550ed640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f715006cc70 msgr2=0x7f7150131ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f71550ed640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f715006cc70 0x7f7150131ec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f714400c780 tx=0x7f714400cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144005de0 con 0x7f715006cc70 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: dump: 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7144007070 con 0x7f715006cc70 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f7157b79640 5 monclient: authenticate success, global_id 74543 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f7157b79640 10 monclient: _renew_subs 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f7157b79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71500738f0 con 0x7f715006cc70 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144007440 con 0x7f715006cc70 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713ffff640 4 set_mon_vals no callback set 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f713e7fc640 10 monclient: dump: 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.448 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f7157b79640 10 monclient: _renew_subs 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f7157b79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.448+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71500a21d0 con 0x7f715006cc70 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f7157b79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f7157b79640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150132400 0x7f715006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f7157b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f715009d810 con 0x7f7150132400 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7144007620 con 0x7f715006cc70 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150132400 0x7f715006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 10 monclient: get_auth_request con 0x7f7150132400 auth_method 0 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f713e7fc640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f712003de60 0x7f7120040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(309..309 src has 117..309) ==== 4735+0+0 (secure 0 0 0) 0x7f7144052650 con 0x7f715006cc70 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71560ef640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f712003de60 0x7f7120040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71560ef640 10 monclient: get_auth_request con 0x7f712003de60 auth_method 0 2026-02-17T05:54:48.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 10 monclient(hunting): handle_auth_done global_id 114160 payload 824 2026-02-17T05:54:48.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.449+0000 7f71558ee640 1 -- 10.20.193.78:0/2688366374 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71500a01f0 con 0x7f7150132400 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f71558ee640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150132400 msgr2=0x7f715006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f71558ee640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150132400 0x7f715006fef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7140039000 tx=0x7f71400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f71560ef640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f712003de60 0x7f7120040320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f714c003e00 tx=0x7f714c0091f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714003c040 con 0x7f7150132400 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 10 monclient: dump: 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7140037cf0 con 0x7f7150132400 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:54:48.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:54:48.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150132400 msgr2=0x7f715006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150132400 0x7f715006fef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7140039000 tx=0x7f71400047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150132400 msgr2=0x7f715006fef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:48.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.452+0000 7f713e7fc640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150132400 0x7f715006fef0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7140039000 tx=0x7f71400047c0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f7157b79640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f7157b79640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150070430 0x7f71501d8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f7157b79640 10 monclient(hunting): start opening mon connection 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7150070840 con 0x7f7150070430 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150070430 0x7f71501d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 10 monclient: get_auth_request con 0x7f7150070430 auth_method 0 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:54:48.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:54:48.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:54:48.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 10 monclient(hunting): handle_auth_done global_id 114163 payload 824 2026-02-17T05:54:48.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.455+0000 7f71558ee640 1 -- 10.20.193.78:0/2688366374 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7150071ee0 con 0x7f7150070430 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f71558ee640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150070430 msgr2=0x7f71501d8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f71558ee640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150070430 0x7f71501d8260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7140002410 tx=0x7f7140008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714003c040 con 0x7f7150070430 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 10 monclient: dump: 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7f714004eb70 con 0x7f7150070430 2026-02-17T05:54:48.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150070430 msgr2=0x7f71501d8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150070430 0x7f71501d8260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7140002410 tx=0x7f7140008030 comp rx=0 tx=0).stop 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150070430 msgr2=0x7f71501d8260 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f713e7fc640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7150070430 0x7f71501d8260 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7140002410 tx=0x7f7140008030 comp rx=0 tx=0).stop 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 90, 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 32539, 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:54:48.458 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:48.459 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:54:48.460 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:54:48.461 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.462 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:48.463 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:54:48.464 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:54:48.465 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:54:48.466 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:54:48.466 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:54:48.466 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:54:48.466 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f712003de60 msgr2=0x7f7120040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f712003de60 0x7f7120040320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f714c003e00 tx=0x7f714c0091f0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 10 monclient: shutdown 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f715006cc70 msgr2=0x7f7150131ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f715006cc70 0x7f7150131ec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f714400c780 tx=0x7f714400cc50 comp rx=0 tx=0).stop 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f71560ef640 1 -- 10.20.193.78:0/2688366374 reap_dead start 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 shutdown_connections 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 --2- 10.20.193.78:0/2688366374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f715006cc70 0x7f7150131ec0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 >> 10.20.193.78:0/2688366374 conn(0x7f71500a6620 msgr2=0x7f715009be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 shutdown_connections 2026-02-17T05:54:48.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:54:48.458+0000 7f7157b79640 1 -- 10.20.193.78:0/2688366374 wait complete. 2026-02-17T05:54:48.467 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:54:48.467 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:54:49.213 INFO:tasks.rados.rados.0.trial115.stdout:1466: done (2 left) 2026-02-17T05:54:49.214 INFO:tasks.rados.rados.0.trial115.stdout:1467: done (1 left) 2026-02-17T05:54:49.214 INFO:tasks.rados.rados.0.trial115.stdout:1468: done (0 left) 2026-02-17T05:54:49.214 INFO:tasks.rados.rados.0.trial115.stdout:1469: snap_remove snap 132 2026-02-17T05:54:50.215 INFO:tasks.rados.rados.0.trial115.stdout:1469: done (0 left) 2026-02-17T05:54:50.216 INFO:tasks.rados.rados.0.trial115.stdout:1470: delete oid 21 current snap is 145 2026-02-17T05:54:50.219 INFO:tasks.rados.rados.0.trial115.stdout:1470: done (0 left) 2026-02-17T05:54:50.219 INFO:tasks.rados.rados.0.trial115.stdout:1471: rollback oid 33 current snap is 145 2026-02-17T05:54:50.219 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 144 2026-02-17T05:54:50.220 INFO:tasks.rados.rados.0.trial115.stdout:1472: rollback oid 30 current snap is 145 2026-02-17T05:54:50.220 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 144 2026-02-17T05:54:50.220 INFO:tasks.rados.rados.0.trial115.stdout:1473: delete oid 38 current snap is 145 2026-02-17T05:54:50.222 INFO:tasks.rados.rados.0.trial115.stdout:1473: done (2 left) 2026-02-17T05:54:50.222 INFO:tasks.rados.rados.0.trial115.stdout:1474: write (excl) oid 26 current snap is 145 2026-02-17T05:54:50.222 INFO:tasks.rados.rados.0.trial115.stdout:1474: seq_num 559 ranges {595113=557960,1644846=491190,2784747=610366} 2026-02-17T05:54:50.226 INFO:tasks.rados.rados.0.trial115.stdout:1474: writing trial11545420-26 from 595113 to 1153073 tid 1 2026-02-17T05:54:50.230 INFO:tasks.rados.rados.0.trial115.stdout:1474: writing trial11545420-26 from 1644846 to 2136036 tid 2 2026-02-17T05:54:50.235 INFO:tasks.rados.rados.0.trial115.stdout:1474: writing trial11545420-26 from 2784747 to 3395113 tid 3 2026-02-17T05:54:50.235 INFO:tasks.rados.rados.0.trial115.stdout:1475: read oid 49 snap -1 2026-02-17T05:54:50.235 INFO:tasks.rados.rados.0.trial115.stdout:1475: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1476: read oid 37 snap -1 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1476: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1477: read oid 48 snap -1 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1477: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1478: write oid 8 current snap is 145 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1471: finishing rollback tid 0 to trial11545420-33 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1472: finishing rollback tid 0 to trial11545420-30 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1471: finishing rollback tid 1 to trial11545420-33 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1471: finishing rollback tid 2 to trial11545420-33 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 612 (ObjNum 539 snap 137 seq_num 539) dirty exists 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1472: finishing rollback tid 1 to trial11545420-30 2026-02-17T05:54:50.236 INFO:tasks.rados.rados.0.trial115.stdout:1472: finishing rollback tid 2 to trial11545420-30 2026-02-17T05:54:50.237 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 644 (ObjNum 464 snap 117 seq_num 464) dirty exists 2026-02-17T05:54:50.237 INFO:tasks.rados.rados.0.trial115.stdout:1474: finishing write tid 1 to trial11545420-26 2026-02-17T05:54:50.237 INFO:tasks.rados.rados.0.trial115.stdout:1478: seq_num 560 ranges {611920=633209,1703328=634848,2812170=507386} 2026-02-17T05:54:50.239 INFO:tasks.rados.rados.0.trial115.stdout:1478: writing trial11545420-8 from 611920 to 1245129 tid 1 2026-02-17T05:54:50.243 INFO:tasks.rados.rados.0.trial115.stdout:1478: writing trial11545420-8 from 1703328 to 2338176 tid 2 2026-02-17T05:54:50.247 INFO:tasks.rados.rados.0.trial115.stdout:1478: writing trial11545420-8 from 2812170 to 3319556 tid 3 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1471: done (6 left) 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1472: done (5 left) 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1479: read oid 30 snap -1 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1479: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1480: write oid 4 current snap is 145 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1474: finishing write tid 2 to trial11545420-26 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1474: finishing write tid 3 to trial11545420-26 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1474: finishing write tid 4 to trial11545420-26 2026-02-17T05:54:50.248 INFO:tasks.rados.rados.0.trial115.stdout:1480: seq_num 561 ranges {628727=708458,2045457=494859,3123240=305487} 2026-02-17T05:54:50.252 INFO:tasks.rados.rados.0.trial115.stdout:1480: writing trial11545420-4 from 628727 to 1337185 tid 1 2026-02-17T05:54:50.257 INFO:tasks.rados.rados.0.trial115.stdout:1480: writing trial11545420-4 from 2045457 to 2540316 tid 2 2026-02-17T05:54:50.260 INFO:tasks.rados.rados.0.trial115.stdout:1480: writing trial11545420-4 from 3123240 to 3428727 tid 3 2026-02-17T05:54:50.260 INFO:tasks.rados.rados.0.trial115.stdout:1481: write oid 44 current snap is 145 2026-02-17T05:54:50.260 INFO:tasks.rados.rados.0.trial115.stdout:1481: seq_num 562 ranges {645534=783707,2103939=638517,3150663=294871} 2026-02-17T05:54:50.265 INFO:tasks.rados.rados.0.trial115.stdout:1481: writing trial11545420-44 from 645534 to 1429241 tid 1 2026-02-17T05:54:50.270 INFO:tasks.rados.rados.0.trial115.stdout:1481: writing trial11545420-44 from 2103939 to 2742456 tid 2 2026-02-17T05:54:50.273 INFO:tasks.rados.rados.0.trial115.stdout:1481: writing trial11545420-44 from 3150663 to 3445534 tid 3 2026-02-17T05:54:50.273 INFO:tasks.rados.rados.0.trial115.stdout:1482: snap_create 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:1474: finishing write tid 5 to trial11545420-26 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:1474: oid 26 updating version 0 to 207 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:1474: oid 26 version 207 is already newer than 206 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:1474: oid 26 version 207 is already newer than 204 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:1474: oid 26 version 207 is already newer than 205 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 207 (ObjNum 559 snap 145 seq_num 559) dirty exists 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:1474: left oid 26 (ObjNum 559 snap 145 seq_num 559) 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:1478: finishing write tid 1 to trial11545420-8 2026-02-17T05:54:50.274 INFO:tasks.rados.rados.0.trial115.stdout:1477: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:54:50.353 INFO:tasks.rados.rados.0.trial115.stdout:1475: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:54:50.496 INFO:tasks.rados.rados.0.trial115.stdout:1476: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:50.583 INFO:tasks.rados.rados.0.trial115.stdout:1478: finishing write tid 2 to trial11545420-8 2026-02-17T05:54:50.583 INFO:tasks.rados.rados.0.trial115.stdout:1478: finishing write tid 3 to trial11545420-8 2026-02-17T05:54:50.583 INFO:tasks.rados.rados.0.trial115.stdout:1478: finishing write tid 4 to trial11545420-8 2026-02-17T05:54:50.583 INFO:tasks.rados.rados.0.trial115.stdout:1478: finishing write tid 5 to trial11545420-8 2026-02-17T05:54:50.583 INFO:tasks.rados.rados.0.trial115.stdout:1478: oid 8 updating version 0 to 197 2026-02-17T05:54:50.583 INFO:tasks.rados.rados.0.trial115.stdout:1478: oid 8 version 197 is already newer than 195 2026-02-17T05:54:50.583 INFO:tasks.rados.rados.0.trial115.stdout:1478: oid 8 updating version 197 to 198 2026-02-17T05:54:50.583 INFO:tasks.rados.rados.0.trial115.stdout:1478: oid 8 version 198 is already newer than 196 2026-02-17T05:54:50.584 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 198 (ObjNum 560 snap 145 seq_num 560) dirty exists 2026-02-17T05:54:50.584 INFO:tasks.rados.rados.0.trial115.stdout:1478: left oid 8 (ObjNum 560 snap 145 seq_num 560) 2026-02-17T05:54:50.584 INFO:tasks.rados.rados.0.trial115.stdout:1479: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:54:50.709 INFO:tasks.rados.rados.0.trial115.stdout:1480: finishing write tid 1 to trial11545420-4 2026-02-17T05:54:50.709 INFO:tasks.rados.rados.0.trial115.stdout:1480: finishing write tid 2 to trial11545420-4 2026-02-17T05:54:50.709 INFO:tasks.rados.rados.0.trial115.stdout:1480: finishing write tid 3 to trial11545420-4 2026-02-17T05:54:50.709 INFO:tasks.rados.rados.0.trial115.stdout:1480: finishing write tid 4 to trial11545420-4 2026-02-17T05:54:50.709 INFO:tasks.rados.rados.0.trial115.stdout:1480: finishing write tid 5 to trial11545420-4 2026-02-17T05:54:50.709 INFO:tasks.rados.rados.0.trial115.stdout:1480: oid 4 updating version 0 to 576 2026-02-17T05:54:50.710 INFO:tasks.rados.rados.0.trial115.stdout:1480: oid 4 version 576 is already newer than 575 2026-02-17T05:54:50.710 INFO:tasks.rados.rados.0.trial115.stdout:1480: oid 4 updating version 576 to 577 2026-02-17T05:54:50.710 INFO:tasks.rados.rados.0.trial115.stdout:1480: oid 4 version 577 is already newer than 574 2026-02-17T05:54:50.710 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 577 (ObjNum 561 snap 145 seq_num 561) dirty exists 2026-02-17T05:54:50.710 INFO:tasks.rados.rados.0.trial115.stdout:1480: left oid 4 (ObjNum 561 snap 145 seq_num 561) 2026-02-17T05:54:50.710 INFO:tasks.rados.rados.0.trial115.stdout:1481: finishing write tid 1 to trial11545420-44 2026-02-17T05:54:50.710 INFO:tasks.rados.rados.0.trial115.stdout:1481: finishing write tid 2 to trial11545420-44 2026-02-17T05:54:50.710 INFO:tasks.rados.rados.0.trial115.stdout:1481: finishing write tid 3 to trial11545420-44 2026-02-17T05:54:50.711 INFO:tasks.rados.rados.0.trial115.stdout:1481: finishing write tid 4 to trial11545420-44 2026-02-17T05:54:50.711 INFO:tasks.rados.rados.0.trial115.stdout:1481: finishing write tid 5 to trial11545420-44 2026-02-17T05:54:50.711 INFO:tasks.rados.rados.0.trial115.stdout:1481: oid 44 updating version 0 to 338 2026-02-17T05:54:50.711 INFO:tasks.rados.rados.0.trial115.stdout:1481: oid 44 version 338 is already newer than 337 2026-02-17T05:54:50.711 INFO:tasks.rados.rados.0.trial115.stdout:1481: oid 44 updating version 338 to 340 2026-02-17T05:54:50.711 INFO:tasks.rados.rados.0.trial115.stdout:1481: oid 44 version 340 is already newer than 339 2026-02-17T05:54:50.711 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 340 (ObjNum 562 snap 145 seq_num 562) dirty exists 2026-02-17T05:54:50.711 INFO:tasks.rados.rados.0.trial115.stdout:1481: left oid 44 (ObjNum 562 snap 145 seq_num 562) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1474: done (8 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1475: done (7 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1476: done (6 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1477: done (5 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1478: done (4 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1479: done (3 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1480: done (2 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1481: done (1 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1482: done (0 left) 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1483: write (excl) oid 38 current snap is 146 2026-02-17T05:54:50.927 INFO:tasks.rados.rados.0.trial115.stdout:1483: seq_num 563 ranges {662341=575309,1878774=498528,2894439=457555} 2026-02-17T05:54:50.931 INFO:tasks.rados.rados.0.trial115.stdout:1483: writing trial11545420-38 from 662341 to 1237650 tid 1 2026-02-17T05:54:50.936 INFO:tasks.rados.rados.0.trial115.stdout:1483: writing trial11545420-38 from 1878774 to 2377302 tid 2 2026-02-17T05:54:50.940 INFO:tasks.rados.rados.0.trial115.stdout:1483: writing trial11545420-38 from 2894439 to 3351994 tid 3 2026-02-17T05:54:50.940 INFO:tasks.rados.rados.0.trial115.stdout:1484: copy_from oid 35 from oid 45 current snap is 146 2026-02-17T05:54:50.940 INFO:tasks.rados.rados.0.trial115.stdout:1483: finishing write tid 1 to trial11545420-38 2026-02-17T05:54:50.940 INFO:tasks.rados.rados.0.trial115.stdout:1483: finishing write tid 2 to trial11545420-38 2026-02-17T05:54:50.940 INFO:tasks.rados.rados.0.trial115.stdout:1485: read oid 50 snap -1 2026-02-17T05:54:50.940 INFO:tasks.rados.rados.0.trial115.stdout:1485: expect (ObjNum 554 snap 143 seq_num 554) 2026-02-17T05:54:50.940 INFO:tasks.rados.rados.0.trial115.stdout:1486: write oid 34 current snap is 146 2026-02-17T05:54:50.940 INFO:tasks.rados.rados.0.trial115.stdout:1486: seq_num 564 ranges {679148=650558,1937256=642186,3321862=157286} 2026-02-17T05:54:50.945 INFO:tasks.rados.rados.0.trial115.stdout:1486: writing trial11545420-34 from 679148 to 1329706 tid 1 2026-02-17T05:54:50.950 INFO:tasks.rados.rados.0.trial115.stdout:1486: writing trial11545420-34 from 1937256 to 2579442 tid 2 2026-02-17T05:54:50.954 INFO:tasks.rados.rados.0.trial115.stdout:1486: writing trial11545420-34 from 3321862 to 3479148 tid 3 2026-02-17T05:54:50.954 INFO:tasks.rados.rados.0.trial115.stdout:1487: delete oid 33 current snap is 146 2026-02-17T05:54:50.954 INFO:tasks.rados.rados.0.trial115.stdout:1483: finishing write tid 3 to trial11545420-38 2026-02-17T05:54:50.954 INFO:tasks.rados.rados.0.trial115.stdout:1483: finishing write tid 4 to trial11545420-38 2026-02-17T05:54:50.954 INFO:tasks.rados.rados.0.trial115.stdout:1483: finishing write tid 5 to trial11545420-38 2026-02-17T05:54:50.954 INFO:tasks.rados.rados.0.trial115.stdout:1483: oid 38 updating version 0 to 788 2026-02-17T05:54:50.954 INFO:tasks.rados.rados.0.trial115.stdout:1483: oid 38 updating version 788 to 789 2026-02-17T05:54:50.954 INFO:tasks.rados.rados.0.trial115.stdout:1483: oid 38 updating version 789 to 790 2026-02-17T05:54:50.955 INFO:tasks.rados.rados.0.trial115.stdout:1483: oid 38 version 790 is already newer than 787 2026-02-17T05:54:50.955 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 790 (ObjNum 563 snap 146 seq_num 563) dirty exists 2026-02-17T05:54:50.955 INFO:tasks.rados.rados.0.trial115.stdout:1483: left oid 38 (ObjNum 563 snap 146 seq_num 563) 2026-02-17T05:54:50.955 INFO:tasks.rados.rados.0.trial115.stdout:1485: expect (ObjNum 554 snap 143 seq_num 554) 2026-02-17T05:54:51.091 INFO:tasks.rados.rados.0.trial115.stdout:1486: finishing write tid 1 to trial11545420-34 2026-02-17T05:54:51.091 INFO:tasks.rados.rados.0.trial115.stdout:1486: finishing write tid 2 to trial11545420-34 2026-02-17T05:54:51.091 INFO:tasks.rados.rados.0.trial115.stdout:1484: finishing copy_from racing read to trial11545420-35 2026-02-17T05:54:51.091 INFO:tasks.rados.rados.0.trial115.stdout:1484: finishing copy_from to trial11545420-35 2026-02-17T05:54:51.091 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 240 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:54:51.091 INFO:tasks.rados.rados.0.trial115.stdout:1486: finishing write tid 3 to trial11545420-34 2026-02-17T05:54:51.091 INFO:tasks.rados.rados.0.trial115.stdout:1483: done (4 left) 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1484: done (3 left) 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1485: done (2 left) 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1487: done (1 left) 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1488: rollback oid 4 current snap is 146 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 140 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1486: finishing write tid 4 to trial11545420-34 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1486: finishing write tid 5 to trial11545420-34 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1486: oid 34 updating version 0 to 441 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1486: oid 34 updating version 441 to 442 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1486: oid 34 version 442 is already newer than 439 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:1486: oid 34 version 442 is already newer than 440 2026-02-17T05:54:51.092 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 442 (ObjNum 564 snap 146 seq_num 564) dirty exists 2026-02-17T05:54:51.093 INFO:tasks.rados.rados.0.trial115.stdout:1486: left oid 34 (ObjNum 564 snap 146 seq_num 564) 2026-02-17T05:54:51.093 INFO:tasks.rados.rados.0.trial115.stdout:1486: done (1 left) 2026-02-17T05:54:51.093 INFO:tasks.rados.rados.0.trial115.stdout:1489: write oid 50 current snap is 146 2026-02-17T05:54:51.093 INFO:tasks.rados.rados.0.trial115.stdout:1489: seq_num 566 ranges {712762=401056,1537867=645855,2860355=652407} 2026-02-17T05:54:51.094 INFO:tasks.rados.rados.0.trial115.stdout:1489: writing trial11545420-50 from 712762 to 1113818 tid 1 2026-02-17T05:54:51.098 INFO:tasks.rados.rados.0.trial115.stdout:1489: writing trial11545420-50 from 1537867 to 2183722 tid 2 2026-02-17T05:54:51.103 INFO:tasks.rados.rados.0.trial115.stdout:1489: writing trial11545420-50 from 2860355 to 3512762 tid 3 2026-02-17T05:54:51.103 INFO:tasks.rados.rados.0.trial115.stdout:1490: snap_create 2026-02-17T05:54:51.104 INFO:tasks.rados.rados.0.trial115.stdout:1488: finishing rollback tid 0 to trial11545420-4 2026-02-17T05:54:51.104 INFO:tasks.rados.rados.0.trial115.stdout:1488: finishing rollback tid 1 to trial11545420-4 2026-02-17T05:54:51.104 INFO:tasks.rados.rados.0.trial115.stdout:1488: finishing rollback tid 2 to trial11545420-4 2026-02-17T05:54:51.104 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 580 (ObjNum 505 snap 130 seq_num 505) dirty exists 2026-02-17T05:54:51.104 INFO:tasks.rados.rados.0.trial115.stdout:1489: finishing write tid 1 to trial11545420-50 2026-02-17T05:54:51.104 INFO:tasks.rados.rados.0.trial115.stdout:1489: finishing write tid 2 to trial11545420-50 2026-02-17T05:54:51.105 INFO:tasks.rados.rados.0.trial115.stdout:1489: finishing write tid 3 to trial11545420-50 2026-02-17T05:54:51.105 INFO:tasks.rados.rados.0.trial115.stdout:1489: finishing write tid 4 to trial11545420-50 2026-02-17T05:54:51.106 INFO:tasks.rados.rados.0.trial115.stdout:1489: finishing write tid 5 to trial11545420-50 2026-02-17T05:54:51.106 INFO:tasks.rados.rados.0.trial115.stdout:1489: oid 50 updating version 0 to 646 2026-02-17T05:54:51.106 INFO:tasks.rados.rados.0.trial115.stdout:1489: oid 50 updating version 646 to 648 2026-02-17T05:54:51.106 INFO:tasks.rados.rados.0.trial115.stdout:1489: oid 50 updating version 648 to 649 2026-02-17T05:54:51.106 INFO:tasks.rados.rados.0.trial115.stdout:1489: oid 50 version 649 is already newer than 647 2026-02-17T05:54:51.106 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 649 (ObjNum 566 snap 146 seq_num 566) dirty exists 2026-02-17T05:54:51.106 INFO:tasks.rados.rados.0.trial115.stdout:1489: left oid 50 (ObjNum 566 snap 146 seq_num 566) 2026-02-17T05:54:51.929 INFO:tasks.rados.rados.0.trial115.stdout:1488: done (2 left) 2026-02-17T05:54:51.929 INFO:tasks.rados.rados.0.trial115.stdout:1489: done (1 left) 2026-02-17T05:54:51.929 INFO:tasks.rados.rados.0.trial115.stdout:1490: done (0 left) 2026-02-17T05:54:51.929 INFO:tasks.rados.rados.0.trial115.stdout:1491: rollback oid 37 current snap is 147 2026-02-17T05:54:51.929 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 137 2026-02-17T05:54:51.929 INFO:tasks.rados.rados.0.trial115.stdout:1492: read oid 41 snap -1 2026-02-17T05:54:51.929 INFO:tasks.rados.rados.0.trial115.stdout:1492: expect deleted 2026-02-17T05:54:51.929 INFO:tasks.rados.rados.0.trial115.stdout:1493: delete oid 48 current snap is 147 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:1492: done (2 left) 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:1493: done (1 left) 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:1494: read oid 14 snap -1 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:1494: expect deleted 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:1495: rollback oid 34 current snap is 147 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 145 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:1496: copy_from oid 40 from oid 18 current snap is 147 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:1497: write oid 21 current snap is 147 2026-02-17T05:54:51.933 INFO:tasks.rados.rados.0.trial115.stdout:1497: seq_num 568 ranges {746376=551554,2054831=649524,3315201=231175} 2026-02-17T05:54:51.938 INFO:tasks.rados.rados.0.trial115.stdout:1497: writing trial11545420-21 from 746376 to 1297930 tid 1 2026-02-17T05:54:51.943 INFO:tasks.rados.rados.0.trial115.stdout:1497: writing trial11545420-21 from 2054831 to 2704355 tid 2 2026-02-17T05:54:51.946 INFO:tasks.rados.rados.0.trial115.stdout:1497: writing trial11545420-21 from 3315201 to 3546376 tid 3 2026-02-17T05:54:51.947 INFO:tasks.rados.rados.0.trial115.stdout:1498: write oid 24 current snap is 147 2026-02-17T05:54:51.947 INFO:tasks.rados.rados.0.trial115.stdout:1498: seq_num 569 ranges {763183=626803,1996960=793182,3226271=336912} 2026-02-17T05:54:51.952 INFO:tasks.rados.rados.0.trial115.stdout:1498: writing trial11545420-24 from 763183 to 1389986 tid 1 2026-02-17T05:54:51.957 INFO:tasks.rados.rados.0.trial115.stdout:1498: writing trial11545420-24 from 1996960 to 2790142 tid 2 2026-02-17T05:54:51.960 INFO:tasks.rados.rados.0.trial115.stdout:1498: writing trial11545420-24 from 3226271 to 3563183 tid 3 2026-02-17T05:54:51.960 INFO:tasks.rados.rados.0.trial115.stdout:1499: write (excl) oid 3 current snap is 147 2026-02-17T05:54:51.960 INFO:tasks.rados.rados.0.trial115.stdout:1499: seq_num 570 ranges {779990=702052,2055442=653193,3253694=326296} 2026-02-17T05:54:51.966 INFO:tasks.rados.rados.0.trial115.stdout:1499: writing trial11545420-3 from 779990 to 1482042 tid 1 2026-02-17T05:54:51.970 INFO:tasks.rados.rados.0.trial115.stdout:1499: writing trial11545420-3 from 2055442 to 2708635 tid 2 2026-02-17T05:54:51.974 INFO:tasks.rados.rados.0.trial115.stdout:1499: writing trial11545420-3 from 3253694 to 3579990 tid 3 2026-02-17T05:54:51.974 INFO:tasks.rados.rados.0.trial115.stdout:1500: snap_remove snap 136 2026-02-17T05:54:52.931 INFO:tasks.rados.rados.0.trial115.stdout:1500: done (7 left) 2026-02-17T05:54:52.931 INFO:tasks.rados.rados.0.trial115.stdout:1501: snap_remove snap 141 2026-02-17T05:54:53.937 INFO:tasks.rados.rados.0.trial115.stdout:1501: done (7 left) 2026-02-17T05:54:53.937 INFO:tasks.rados.rados.0.trial115.stdout:1502: delete oid 19 current snap is 147 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1491: finishing rollback tid 0 to trial11545420-37 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1491: finishing rollback tid 1 to trial11545420-37 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1491: finishing rollback tid 2 to trial11545420-37 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 556 (ObjNum 532 snap 136 seq_num 532) dirty exists 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1495: finishing rollback tid 0 to trial11545420-34 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1495: finishing rollback tid 1 to trial11545420-34 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 444 (ObjNum 49 snap 0 seq_num 1333699792) dirty dne 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1496: finishing copy_from racing read to trial11545420-40 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1497: finishing write tid 1 to trial11545420-21 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1496: finishing copy_from to trial11545420-40 2026-02-17T05:54:53.938 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 618 (ObjNum 541 snap 141 seq_num 541) dirty exists 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: finishing write tid 2 to trial11545420-21 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: finishing write tid 3 to trial11545420-21 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: finishing write tid 4 to trial11545420-21 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: finishing write tid 5 to trial11545420-21 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: oid 21 updating version 0 to 642 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: oid 21 version 642 is already newer than 641 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: oid 21 version 642 is already newer than 640 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: oid 21 updating version 642 to 643 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 643 (ObjNum 568 snap 147 seq_num 568) dirty exists 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1497: left oid 21 (ObjNum 568 snap 147 seq_num 568) 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1498: finishing write tid 1 to trial11545420-24 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1498: finishing write tid 2 to trial11545420-24 2026-02-17T05:54:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1498: finishing write tid 3 to trial11545420-24 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1498: finishing write tid 4 to trial11545420-24 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1498: finishing write tid 5 to trial11545420-24 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1498: oid 24 updating version 0 to 830 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1498: oid 24 updating version 830 to 832 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1498: oid 24 version 832 is already newer than 831 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1498: oid 24 updating version 832 to 833 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 833 (ObjNum 569 snap 147 seq_num 569) dirty exists 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1498: left oid 24 (ObjNum 569 snap 147 seq_num 569) 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1499: finishing write tid 1 to trial11545420-3 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1499: finishing write tid 2 to trial11545420-3 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1499: finishing write tid 3 to trial11545420-3 2026-02-17T05:54:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1499: finishing write tid 4 to trial11545420-3 2026-02-17T05:54:53.941 INFO:tasks.rados.rados.0.trial115.stdout:1499: finishing write tid 5 to trial11545420-3 2026-02-17T05:54:53.941 INFO:tasks.rados.rados.0.trial115.stdout:1499: oid 3 updating version 0 to 770 2026-02-17T05:54:53.941 INFO:tasks.rados.rados.0.trial115.stdout:1499: oid 3 version 770 is already newer than 769 2026-02-17T05:54:53.941 INFO:tasks.rados.rados.0.trial115.stdout:1499: oid 3 version 770 is already newer than 768 2026-02-17T05:54:53.941 INFO:tasks.rados.rados.0.trial115.stdout:1499: oid 3 updating version 770 to 771 2026-02-17T05:54:53.941 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 771 (ObjNum 570 snap 147 seq_num 570) dirty exists 2026-02-17T05:54:53.941 INFO:tasks.rados.rados.0.trial115.stdout:1499: left oid 3 (ObjNum 570 snap 147 seq_num 570) 2026-02-17T05:54:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1491: done (7 left) 2026-02-17T05:54:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1494: done (6 left) 2026-02-17T05:54:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1495: done (5 left) 2026-02-17T05:54:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1496: done (4 left) 2026-02-17T05:54:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1497: done (3 left) 2026-02-17T05:54:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1498: done (2 left) 2026-02-17T05:54:53.943 INFO:tasks.rados.rados.0.trial115.stdout:1499: done (1 left) 2026-02-17T05:54:53.943 INFO:tasks.rados.rados.0.trial115.stdout:1502: done (0 left) 2026-02-17T05:54:53.943 INFO:tasks.rados.rados.0.trial115.stdout:1503: write (excl) oid 32 current snap is 147 2026-02-17T05:54:53.943 INFO:tasks.rados.rados.0.trial115.stdout:1503: seq_num 572 ranges {413604=568903,1488759=656862,2624893=591709} 2026-02-17T05:54:53.943 INFO:tasks.rados.rados.0.trial115.stdout:1503: writing trial11545420-32 from 413604 to 982507 tid 1 2026-02-17T05:54:53.948 INFO:tasks.rados.rados.0.trial115.stdout:1503: writing trial11545420-32 from 1488759 to 2145621 tid 2 2026-02-17T05:54:53.952 INFO:tasks.rados.rados.0.trial115.stdout:1503: writing trial11545420-32 from 2624893 to 3216602 tid 3 2026-02-17T05:54:53.952 INFO:tasks.rados.rados.0.trial115.stdout:1504: read oid 20 snap -1 2026-02-17T05:54:53.952 INFO:tasks.rados.rados.0.trial115.stdout:1504: expect (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:54:53.952 INFO:tasks.rados.rados.0.trial115.stdout:1505: read oid 30 snap -1 2026-02-17T05:54:53.952 INFO:tasks.rados.rados.0.trial115.stdout:1505: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:54:53.953 INFO:tasks.rados.rados.0.trial115.stdout:1506: write oid 23 current snap is 147 2026-02-17T05:54:53.953 INFO:tasks.rados.rados.0.trial115.stdout:1503: finishing write tid 1 to trial11545420-32 2026-02-17T05:54:53.953 INFO:tasks.rados.rados.0.trial115.stdout:1503: finishing write tid 2 to trial11545420-32 2026-02-17T05:54:53.953 INFO:tasks.rados.rados.0.trial115.stdout:1506: seq_num 573 ranges {430411=644152,1830888=400520,2935963=402981} 2026-02-17T05:54:53.956 INFO:tasks.rados.rados.0.trial115.stdout:1506: writing trial11545420-23 from 430411 to 1074563 tid 1 2026-02-17T05:54:53.960 INFO:tasks.rados.rados.0.trial115.stdout:1506: writing trial11545420-23 from 1830888 to 2231408 tid 2 2026-02-17T05:54:53.964 INFO:tasks.rados.rados.0.trial115.stdout:1506: writing trial11545420-23 from 2935963 to 3338944 tid 3 2026-02-17T05:54:53.964 INFO:tasks.rados.rados.0.trial115.stdout:1507: copy_from oid 29 from oid 15 current snap is 147 2026-02-17T05:54:53.965 INFO:tasks.rados.rados.0.trial115.stdout:1505: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:54:54.091 INFO:tasks.rados.rados.0.trial115.stdout:1504: expect (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: finishing write tid 3 to trial11545420-32 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: finishing write tid 4 to trial11545420-32 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: finishing write tid 5 to trial11545420-32 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: oid 32 updating version 0 to 342 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: oid 32 updating version 342 to 344 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: oid 32 version 344 is already newer than 343 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: oid 32 version 344 is already newer than 341 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 344 (ObjNum 572 snap 147 seq_num 572) dirty exists 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: left oid 32 (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1506: finishing write tid 1 to trial11545420-23 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1506: finishing write tid 2 to trial11545420-23 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1503: done (4 left) 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1504: done (3 left) 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1505: done (2 left) 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1508: rollback oid 37 current snap is 147 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 145 2026-02-17T05:54:54.164 INFO:tasks.rados.rados.0.trial115.stdout:1506: finishing write tid 3 to trial11545420-23 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1506: finishing write tid 4 to trial11545420-23 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1506: finishing write tid 5 to trial11545420-23 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1506: oid 23 updating version 0 to 296 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1506: oid 23 version 296 is already newer than 295 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1506: oid 23 updating version 296 to 297 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1506: oid 23 updating version 297 to 298 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 298 (ObjNum 573 snap 147 seq_num 573) dirty exists 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1506: left oid 23 (ObjNum 573 snap 147 seq_num 573) 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1507: finishing copy_from racing read to trial11545420-29 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1507: finishing copy_from to trial11545420-29 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 626 (ObjNum 450 snap 112 seq_num 450) dirty exists 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1506: done (2 left) 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1507: done (1 left) 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1509: read oid 4 snap -1 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1509: expect (ObjNum 505 snap 130 seq_num 505) 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1510: read oid 9 snap -1 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1510: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1511: write oid 38 current snap is 147 2026-02-17T05:54:54.165 INFO:tasks.rados.rados.0.trial115.stdout:1511: seq_num 574 ranges {447218=719401,1889370=660531,2963386=614253} 2026-02-17T05:54:54.169 INFO:tasks.rados.rados.0.trial115.stdout:1511: writing trial11545420-38 from 447218 to 1166619 tid 1 2026-02-17T05:54:54.174 INFO:tasks.rados.rados.0.trial115.stdout:1511: writing trial11545420-38 from 1889370 to 2549901 tid 2 2026-02-17T05:54:54.178 INFO:tasks.rados.rados.0.trial115.stdout:1511: writing trial11545420-38 from 2963386 to 3577639 tid 3 2026-02-17T05:54:54.178 INFO:tasks.rados.rados.0.trial115.stdout:1512: rollback oid 27 current snap is 147 2026-02-17T05:54:54.178 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 145 2026-02-17T05:54:54.179 INFO:tasks.rados.rados.0.trial115.stdout:1508: finishing rollback tid 0 to trial11545420-37 2026-02-17T05:54:54.179 INFO:tasks.rados.rados.0.trial115.stdout:1508: finishing rollback tid 1 to trial11545420-37 2026-02-17T05:54:54.179 INFO:tasks.rados.rados.0.trial115.stdout:1508: finishing rollback tid 2 to trial11545420-37 2026-02-17T05:54:54.179 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 567 (ObjNum 532 snap 136 seq_num 532) dirty exists 2026-02-17T05:54:54.179 INFO:tasks.rados.rados.0.trial115.stdout:1510: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:54:54.258 INFO:tasks.rados.rados.0.trial115.stdout:1509: expect (ObjNum 505 snap 130 seq_num 505) 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: finishing write tid 1 to trial11545420-38 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1508: done (4 left) 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1509: done (3 left) 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1510: done (2 left) 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1513: snap_create 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: finishing write tid 2 to trial11545420-38 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1512: finishing rollback tid 1 to trial11545420-27 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 631 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: finishing write tid 3 to trial11545420-38 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: finishing write tid 4 to trial11545420-38 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: finishing write tid 5 to trial11545420-38 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: oid 38 updating version 0 to 814 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: oid 38 version 814 is already newer than 812 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: oid 38 version 814 is already newer than 813 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: oid 38 updating version 814 to 815 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 815 (ObjNum 574 snap 147 seq_num 574) dirty exists 2026-02-17T05:54:54.373 INFO:tasks.rados.rados.0.trial115.stdout:1511: left oid 38 (ObjNum 574 snap 147 seq_num 574) 2026-02-17T05:54:54.940 INFO:tasks.rados.rados.0.trial115.stdout:1511: done (2 left) 2026-02-17T05:54:54.940 INFO:tasks.rados.rados.0.trial115.stdout:1512: done (1 left) 2026-02-17T05:54:54.940 INFO:tasks.rados.rados.0.trial115.stdout:1513: done (0 left) 2026-02-17T05:54:54.940 INFO:tasks.rados.rados.0.trial115.stdout:1514: copy_from oid 38 from oid 22 current snap is 148 2026-02-17T05:54:54.940 INFO:tasks.rados.rados.0.trial115.stdout:1515: rollback oid 40 current snap is 148 2026-02-17T05:54:54.940 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 142 2026-02-17T05:54:54.940 INFO:tasks.rados.rados.0.trial115.stdout:1516: write (excl) oid 50 current snap is 148 2026-02-17T05:54:54.940 INFO:tasks.rados.rados.0.trial115.stdout:1516: seq_num 575 ranges {464025=794650,1947852=404189,2990809=425525} 2026-02-17T05:54:54.945 INFO:tasks.rados.rados.0.trial115.stdout:1516: writing trial11545420-50 from 464025 to 1258675 tid 1 2026-02-17T05:54:54.949 INFO:tasks.rados.rados.0.trial115.stdout:1516: writing trial11545420-50 from 1947852 to 2352041 tid 2 2026-02-17T05:54:54.954 INFO:tasks.rados.rados.0.trial115.stdout:1516: writing trial11545420-50 from 2990809 to 3416334 tid 3 2026-02-17T05:54:54.954 INFO:tasks.rados.rados.0.trial115.stdout:1517: delete oid 46 current snap is 148 2026-02-17T05:54:54.954 INFO:tasks.rados.rados.0.trial115.stdout:1514: finishing copy_from to trial11545420-38 2026-02-17T05:54:54.954 INFO:tasks.rados.rados.0.trial115.stdout:1514: got expected ENOENT (src dne) 2026-02-17T05:54:54.955 INFO:tasks.rados.rados.0.trial115.stdout:1514: finishing copy_from racing read to trial11545420-38 2026-02-17T05:54:54.955 INFO:tasks.rados.rados.0.trial115.stdout:1515: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:54:54.955 INFO:tasks.rados.rados.0.trial115.stdout:1515: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:54:54.955 INFO:tasks.rados.rados.0.trial115.stdout:1515: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:54:54.955 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 621 (ObjNum 515 snap 132 seq_num 515) dirty exists 2026-02-17T05:54:54.955 INFO:tasks.rados.rados.0.trial115.stdout:1516: finishing write tid 1 to trial11545420-50 2026-02-17T05:54:54.955 INFO:tasks.rados.rados.0.trial115.stdout:1516: finishing write tid 2 to trial11545420-50 2026-02-17T05:54:54.956 INFO:tasks.rados.rados.0.trial115.stdout:1516: finishing write tid 3 to trial11545420-50 2026-02-17T05:54:54.956 INFO:tasks.rados.rados.0.trial115.stdout:1516: finishing write tid 4 to trial11545420-50 2026-02-17T05:54:54.956 INFO:tasks.rados.rados.0.trial115.stdout:1516: finishing write tid 5 to trial11545420-50 2026-02-17T05:54:54.956 INFO:tasks.rados.rados.0.trial115.stdout:1516: oid 50 updating version 0 to 665 2026-02-17T05:54:54.956 INFO:tasks.rados.rados.0.trial115.stdout:1516: oid 50 version 665 is already newer than 664 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:1516: oid 50 updating version 665 to 667 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:1516: oid 50 version 667 is already newer than 666 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 667 (ObjNum 575 snap 148 seq_num 575) dirty exists 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:1516: left oid 50 (ObjNum 575 snap 148 seq_num 575) 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:1514: done (3 left) 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:1515: done (2 left) 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:1516: done (1 left) 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:1517: done (0 left) 2026-02-17T05:54:54.957 INFO:tasks.rados.rados.0.trial115.stdout:1518: snap_remove snap 146 2026-02-17T05:54:55.927 INFO:tasks.rados.rados.0.trial115.stdout:1518: done (0 left) 2026-02-17T05:54:55.927 INFO:tasks.rados.rados.0.trial115.stdout:1519: snap_create 2026-02-17T05:54:56.927 INFO:tasks.rados.rados.0.trial115.stdout:1519: done (0 left) 2026-02-17T05:54:56.927 INFO:tasks.rados.rados.0.trial115.stdout:1520: copy_from oid 13 from oid 6 current snap is 149 2026-02-17T05:54:56.927 INFO:tasks.rados.rados.0.trial115.stdout:1521: rollback oid 26 current snap is 149 2026-02-17T05:54:56.927 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 26 to 142 2026-02-17T05:54:56.927 INFO:tasks.rados.rados.0.trial115.stdout:1522: snap_create 2026-02-17T05:54:56.934 INFO:tasks.rados.rados.0.trial115.stdout:1521: finishing rollback tid 0 to trial11545420-26 2026-02-17T05:54:56.934 INFO:tasks.rados.rados.0.trial115.stdout:1521: finishing rollback tid 1 to trial11545420-26 2026-02-17T05:54:56.934 INFO:tasks.rados.rados.0.trial115.stdout:1521: finishing rollback tid 2 to trial11545420-26 2026-02-17T05:54:56.934 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 210 (ObjNum 550 snap 142 seq_num 550) dirty exists 2026-02-17T05:54:56.939 INFO:tasks.rados.rados.0.trial115.stdout:1520: finishing copy_from racing read to trial11545420-13 2026-02-17T05:54:56.942 INFO:tasks.rados.rados.0.trial115.stdout:1520: finishing copy_from to trial11545420-13 2026-02-17T05:54:56.942 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 438 (ObjNum 553 snap 142 seq_num 553) dirty exists 2026-02-17T05:54:57.928 INFO:tasks.rados.rados.0.trial115.stdout:1520: done (2 left) 2026-02-17T05:54:57.928 INFO:tasks.rados.rados.0.trial115.stdout:1521: done (1 left) 2026-02-17T05:54:57.928 INFO:tasks.rados.rados.0.trial115.stdout:1522: done (0 left) 2026-02-17T05:54:57.928 INFO:tasks.rados.rados.0.trial115.stdout:1523: read oid 2 snap 149 2026-02-17T05:54:57.928 INFO:tasks.rados.rados.0.trial115.stdout:1523: expect deleted 2026-02-17T05:54:57.928 INFO:tasks.rados.rados.0.trial115.stdout:1524: read oid 34 snap -1 2026-02-17T05:54:57.929 INFO:tasks.rados.rados.0.trial115.stdout:1524: expect deleted 2026-02-17T05:54:57.929 INFO:tasks.rados.rados.0.trial115.stdout:1525: write (excl) oid 40 current snap is 150 2026-02-17T05:54:57.929 INFO:tasks.rados.rados.0.trial115.stdout:1525: seq_num 577 ranges {497639=545148,1548463=407858,2529302=564422,3528143=169496} 2026-02-17T05:54:57.931 INFO:tasks.rados.rados.0.trial115.stdout:1525: writing trial11545420-40 from 497639 to 1042787 tid 1 2026-02-17T05:54:57.935 INFO:tasks.rados.rados.0.trial115.stdout:1525: writing trial11545420-40 from 1548463 to 1956321 tid 2 2026-02-17T05:54:57.939 INFO:tasks.rados.rados.0.trial115.stdout:1525: writing trial11545420-40 from 2529302 to 3093724 tid 3 2026-02-17T05:54:57.941 INFO:tasks.rados.rados.0.trial115.stdout:1525: writing trial11545420-40 from 3528143 to 3697639 tid 4 2026-02-17T05:54:57.941 INFO:tasks.rados.rados.0.trial115.stdout:1526: read oid 7 snap -1 2026-02-17T05:54:57.942 INFO:tasks.rados.rados.0.trial115.stdout:1526: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:57.942 INFO:tasks.rados.rados.0.trial115.stdout:1527: read oid 38 snap -1 2026-02-17T05:54:57.942 INFO:tasks.rados.rados.0.trial115.stdout:1527: expect (ObjNum 574 snap 147 seq_num 574) 2026-02-17T05:54:57.942 INFO:tasks.rados.rados.0.trial115.stdout:1528: snap_remove snap 144 2026-02-17T05:54:59.531 INFO:tasks.rados.rados.0.trial115.stdout:1528: done (5 left) 2026-02-17T05:54:59.532 INFO:tasks.rados.rados.0.trial115.stdout:1529: read oid 6 snap -1 2026-02-17T05:54:59.532 INFO:tasks.rados.rados.0.trial115.stdout:1529: expect (ObjNum 553 snap 142 seq_num 553) 2026-02-17T05:54:59.532 INFO:tasks.rados.rados.0.trial115.stdout:1530: copy_from oid 29 from oid 42 current snap is 150 2026-02-17T05:54:59.532 INFO:tasks.rados.rados.0.trial115.stdout:1525: finishing write tid 1 to trial11545420-40 2026-02-17T05:54:59.532 INFO:tasks.rados.rados.0.trial115.stdout:1523: done (6 left) 2026-02-17T05:54:59.533 INFO:tasks.rados.rados.0.trial115.stdout:1524: done (5 left) 2026-02-17T05:54:59.533 INFO:tasks.rados.rados.0.trial115.stdout:1531: write (excl) oid 45 current snap is 150 2026-02-17T05:54:59.533 INFO:tasks.rados.rados.0.trial115.stdout:1531: seq_num 578 ranges {514446=736750,1723298=667869,3073078=641368} 2026-02-17T05:54:59.535 INFO:tasks.rados.rados.0.trial115.stdout:1531: writing trial11545420-45 from 514446 to 1251196 tid 1 2026-02-17T05:54:59.540 INFO:tasks.rados.rados.0.trial115.stdout:1531: writing trial11545420-45 from 1723298 to 2391167 tid 2 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1531: writing trial11545420-45 from 3073078 to 3714446 tid 3 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1532: delete oid 27 current snap is 150 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: finishing write tid 2 to trial11545420-40 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: finishing write tid 3 to trial11545420-40 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: finishing write tid 4 to trial11545420-40 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: finishing write tid 5 to trial11545420-40 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: finishing write tid 6 to trial11545420-40 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: oid 40 updating version 0 to 622 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: oid 40 updating version 622 to 624 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: oid 40 updating version 624 to 625 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: oid 40 version 625 is already newer than 623 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: oid 40 updating version 625 to 626 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 626 (ObjNum 577 snap 150 seq_num 577) dirty exists 2026-02-17T05:54:59.545 INFO:tasks.rados.rados.0.trial115.stdout:1525: left oid 40 (ObjNum 577 snap 150 seq_num 577) 2026-02-17T05:54:59.546 INFO:tasks.rados.rados.0.trial115.stdout:1527: expect (ObjNum 574 snap 147 seq_num 574) 2026-02-17T05:54:59.665 INFO:tasks.rados.rados.0.trial115.stdout:1526: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:54:59.750 INFO:tasks.rados.rados.0.trial115.stdout:1529: expect (ObjNum 553 snap 142 seq_num 553) 2026-02-17T05:54:59.886 INFO:tasks.rados.rados.0.trial115.stdout:1530: finishing copy_from racing read to trial11545420-29 2026-02-17T05:54:59.887 INFO:tasks.rados.rados.0.trial115.stdout:1531: finishing write tid 1 to trial11545420-45 2026-02-17T05:54:59.887 INFO:tasks.rados.rados.0.trial115.stdout:1530: finishing copy_from to trial11545420-29 2026-02-17T05:54:59.887 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 524 (ObjNum 536 snap 136 seq_num 536) dirty exists 2026-02-17T05:54:59.887 INFO:tasks.rados.rados.0.trial115.stdout:1531: finishing write tid 2 to trial11545420-45 2026-02-17T05:54:59.887 INFO:tasks.rados.rados.0.trial115.stdout:1531: finishing write tid 3 to trial11545420-45 2026-02-17T05:54:59.887 INFO:tasks.rados.rados.0.trial115.stdout:1531: finishing write tid 4 to trial11545420-45 2026-02-17T05:54:59.887 INFO:tasks.rados.rados.0.trial115.stdout:1531: finishing write tid 5 to trial11545420-45 2026-02-17T05:54:59.887 INFO:tasks.rados.rados.0.trial115.stdout:1531: oid 45 updating version 0 to 851 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1531: oid 45 version 851 is already newer than 850 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1531: oid 45 version 851 is already newer than 849 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1531: oid 45 updating version 851 to 852 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 852 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1531: left oid 45 (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1525: done (6 left) 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1526: done (5 left) 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1527: done (4 left) 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1529: done (3 left) 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1530: done (2 left) 2026-02-17T05:54:59.888 INFO:tasks.rados.rados.0.trial115.stdout:1531: done (1 left) 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1532: done (0 left) 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1533: copy_from oid 44 from oid 21 current snap is 150 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1534: copy_from oid 42 from oid 23 current snap is 150 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1535: copy_from oid 3 from oid 47 current snap is 150 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1536: rollback oid 37 current snap is 150 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 138 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1537: delete oid 36 current snap is 150 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1535: finishing copy_from to trial11545420-3 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1535: got expected ENOENT (src dne) 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1535: finishing copy_from racing read to trial11545420-3 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1535: done (4 left) 2026-02-17T05:54:59.889 INFO:tasks.rados.rados.0.trial115.stdout:1537: done (3 left) 2026-02-17T05:54:59.890 INFO:tasks.rados.rados.0.trial115.stdout:1538: read oid 1 snap -1 2026-02-17T05:54:59.890 INFO:tasks.rados.rados.0.trial115.stdout:1538: expect deleted 2026-02-17T05:54:59.890 INFO:tasks.rados.rados.0.trial115.stdout:1539: snap_remove snap 148 2026-02-17T05:55:00.534 INFO:tasks.rados.rados.0.trial115.stdout:1539: done (4 left) 2026-02-17T05:55:00.534 INFO:tasks.rados.rados.0.trial115.stdout:1540: read oid 13 snap -1 2026-02-17T05:55:00.534 INFO:tasks.rados.rados.0.trial115.stdout:1540: expect (ObjNum 553 snap 142 seq_num 553) 2026-02-17T05:55:00.534 INFO:tasks.rados.rados.0.trial115.stdout:1541: rollback oid 47 current snap is 150 2026-02-17T05:55:00.534 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 47 to 143 2026-02-17T05:55:00.534 INFO:tasks.rados.rados.0.trial115.stdout:1536: finishing rollback tid 0 to trial11545420-37 2026-02-17T05:55:00.534 INFO:tasks.rados.rados.0.trial115.stdout:1536: finishing rollback tid 1 to trial11545420-37 2026-02-17T05:55:00.535 INFO:tasks.rados.rados.0.trial115.stdout:1538: done (5 left) 2026-02-17T05:55:00.535 INFO:tasks.rados.rados.0.trial115.stdout:1542: read oid 4 snap -1 2026-02-17T05:55:00.535 INFO:tasks.rados.rados.0.trial115.stdout:1542: expect (ObjNum 505 snap 130 seq_num 505) 2026-02-17T05:55:00.535 INFO:tasks.rados.rados.0.trial115.stdout:1543: snap_remove snap 137 2026-02-17T05:55:00.928 INFO:tasks.rados.rados.0.trial115.stdout:1543: done (6 left) 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:1544: snap_create 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:1536: finishing rollback tid 2 to trial11545420-37 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 575 (ObjNum 532 snap 136 seq_num 532) dirty exists 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:1533: finishing copy_from racing read to trial11545420-44 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:1533: finishing copy_from to trial11545420-44 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 643 (ObjNum 568 snap 147 seq_num 568) dirty exists 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:1534: finishing copy_from racing read to trial11545420-42 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:1534: finishing copy_from to trial11545420-42 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 298 (ObjNum 573 snap 147 seq_num 573) dirty exists 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:1541: finishing rollback tid 1 to trial11545420-47 2026-02-17T05:55:00.929 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:55:00.930 INFO:tasks.rados.rados.0.trial115.stdout:1542: expect (ObjNum 505 snap 130 seq_num 505) 2026-02-17T05:55:01.044 INFO:tasks.rados.rados.0.trial115.stdout:1540: expect (ObjNum 553 snap 142 seq_num 553) 2026-02-17T05:55:02.269 INFO:tasks.rados.rados.0.trial115.stdout:1533: done (6 left) 2026-02-17T05:55:02.269 INFO:tasks.rados.rados.0.trial115.stdout:1534: done (5 left) 2026-02-17T05:55:02.269 INFO:tasks.rados.rados.0.trial115.stdout:1536: done (4 left) 2026-02-17T05:55:02.269 INFO:tasks.rados.rados.0.trial115.stdout:1540: done (3 left) 2026-02-17T05:55:02.269 INFO:tasks.rados.rados.0.trial115.stdout:1541: done (2 left) 2026-02-17T05:55:02.269 INFO:tasks.rados.rados.0.trial115.stdout:1542: done (1 left) 2026-02-17T05:55:02.269 INFO:tasks.rados.rados.0.trial115.stdout:1544: done (0 left) 2026-02-17T05:55:02.269 INFO:tasks.rados.rados.0.trial115.stdout:1545: delete oid 10 current snap is 151 2026-02-17T05:55:02.272 INFO:tasks.rados.rados.0.trial115.stdout:1545: done (0 left) 2026-02-17T05:55:02.272 INFO:tasks.rados.rados.0.trial115.stdout:1546: snap_remove snap 147 2026-02-17T05:55:03.271 INFO:tasks.rados.rados.0.trial115.stdout:1546: done (0 left) 2026-02-17T05:55:03.271 INFO:tasks.rados.rados.0.trial115.stdout:1547: read oid 36 snap -1 2026-02-17T05:55:03.271 INFO:tasks.rados.rados.0.trial115.stdout:1547: expect deleted 2026-02-17T05:55:03.271 INFO:tasks.rados.rados.0.trial115.stdout:1548: write (excl) oid 3 current snap is 151 2026-02-17T05:55:03.271 INFO:tasks.rados.rados.0.trial115.stdout:1548: seq_num 582 ranges {581674=637746,1840873=558854,3066417=537135} 2026-02-17T05:55:03.275 INFO:tasks.rados.rados.0.trial115.stdout:1548: writing trial11545420-3 from 581674 to 1219420 tid 1 2026-02-17T05:55:03.280 INFO:tasks.rados.rados.0.trial115.stdout:1548: writing trial11545420-3 from 1840873 to 2399727 tid 2 2026-02-17T05:55:03.285 INFO:tasks.rados.rados.0.trial115.stdout:1548: writing trial11545420-3 from 3066417 to 3603552 tid 3 2026-02-17T05:55:03.286 INFO:tasks.rados.rados.0.trial115.stdout:1549: copy_from oid 40 from oid 11 current snap is 151 2026-02-17T05:55:03.286 INFO:tasks.rados.rados.0.trial115.stdout:1548: finishing write tid 1 to trial11545420-3 2026-02-17T05:55:03.286 INFO:tasks.rados.rados.0.trial115.stdout:1548: finishing write tid 2 to trial11545420-3 2026-02-17T05:55:03.286 INFO:tasks.rados.rados.0.trial115.stdout:1547: done (2 left) 2026-02-17T05:55:03.286 INFO:tasks.rados.rados.0.trial115.stdout:1550: write oid 23 current snap is 151 2026-02-17T05:55:03.286 INFO:tasks.rados.rados.0.trial115.stdout:1550: seq_num 583 ranges {598481=712995,1899355=418865,3093840=704641} 2026-02-17T05:55:03.291 INFO:tasks.rados.rados.0.trial115.stdout:1550: writing trial11545420-23 from 598481 to 1311476 tid 1 2026-02-17T05:55:03.295 INFO:tasks.rados.rados.0.trial115.stdout:1550: writing trial11545420-23 from 1899355 to 2318220 tid 2 2026-02-17T05:55:03.301 INFO:tasks.rados.rados.0.trial115.stdout:1550: writing trial11545420-23 from 3093840 to 3798481 tid 3 2026-02-17T05:55:03.301 INFO:tasks.rados.rados.0.trial115.stdout:1551: delete oid 30 current snap is 151 2026-02-17T05:55:03.301 INFO:tasks.rados.rados.0.trial115.stdout:1548: finishing write tid 3 to trial11545420-3 2026-02-17T05:55:03.301 INFO:tasks.rados.rados.0.trial115.stdout:1548: finishing write tid 4 to trial11545420-3 2026-02-17T05:55:03.301 INFO:tasks.rados.rados.0.trial115.stdout:1548: finishing write tid 5 to trial11545420-3 2026-02-17T05:55:03.301 INFO:tasks.rados.rados.0.trial115.stdout:1548: oid 3 updating version 0 to 782 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:1548: oid 3 version 782 is already newer than 779 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:1548: oid 3 version 782 is already newer than 780 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:1548: oid 3 version 782 is already newer than 781 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 782 (ObjNum 582 snap 151 seq_num 582) dirty exists 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:1548: left oid 3 (ObjNum 582 snap 151 seq_num 582) 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:1549: finishing copy_from racing read to trial11545420-40 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:1550: finishing write tid 1 to trial11545420-23 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:1549: finishing copy_from to trial11545420-40 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 698 (ObjNum 548 snap 142 seq_num 548) dirty exists 2026-02-17T05:55:03.302 INFO:tasks.rados.rados.0.trial115.stdout:1550: finishing write tid 2 to trial11545420-23 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1550: finishing write tid 3 to trial11545420-23 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1548: done (3 left) 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1549: done (2 left) 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1551: done (1 left) 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1552: delete oid 25 current snap is 151 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1550: finishing write tid 4 to trial11545420-23 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1550: finishing write tid 5 to trial11545420-23 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1550: oid 23 updating version 0 to 311 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1550: oid 23 version 311 is already newer than 308 2026-02-17T05:55:03.304 INFO:tasks.rados.rados.0.trial115.stdout:1550: oid 23 version 311 is already newer than 310 2026-02-17T05:55:03.305 INFO:tasks.rados.rados.0.trial115.stdout:1550: oid 23 version 311 is already newer than 309 2026-02-17T05:55:03.305 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 311 (ObjNum 583 snap 151 seq_num 583) dirty exists 2026-02-17T05:55:03.305 INFO:tasks.rados.rados.0.trial115.stdout:1550: left oid 23 (ObjNum 583 snap 151 seq_num 583) 2026-02-17T05:55:03.305 INFO:tasks.rados.rados.0.trial115.stdout:1550: done (1 left) 2026-02-17T05:55:03.305 INFO:tasks.rados.rados.0.trial115.stdout:1552: done (0 left) 2026-02-17T05:55:03.305 INFO:tasks.rados.rados.0.trial115.stdout:1553: delete oid 50 current snap is 151 2026-02-17T05:55:03.306 INFO:tasks.rados.rados.0.trial115.stdout:1553: done (0 left) 2026-02-17T05:55:03.306 INFO:tasks.rados.rados.0.trial115.stdout:1554: snap_remove snap 143 2026-02-17T05:55:04.274 INFO:tasks.rados.rados.0.trial115.stdout:1554: done (0 left) 2026-02-17T05:55:04.275 INFO:tasks.rados.rados.0.trial115.stdout:1555: snap_remove snap 142 2026-02-17T05:55:05.275 INFO:tasks.rados.rados.0.trial115.stdout:1555: done (0 left) 2026-02-17T05:55:05.275 INFO:tasks.rados.rados.0.trial115.stdout:1556: write (excl) oid 50 current snap is 151 2026-02-17T05:55:05.276 INFO:tasks.rados.rados.0.trial115.stdout:1556: seq_num 587 ranges {665709=730344,2133283=426203,3203532=509848} 2026-02-17T05:55:05.280 INFO:tasks.rados.rados.0.trial115.stdout:1556: writing trial11545420-50 from 665709 to 1396053 tid 1 2026-02-17T05:55:05.285 INFO:tasks.rados.rados.0.trial115.stdout:1556: writing trial11545420-50 from 2133283 to 2559486 tid 2 2026-02-17T05:55:05.290 INFO:tasks.rados.rados.0.trial115.stdout:1556: writing trial11545420-50 from 3203532 to 3713380 tid 3 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1557: read oid 1 snap -1 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1557: expect deleted 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1558: read oid 22 snap -1 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1558: expect deleted 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1559: read oid 7 snap -1 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1559: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1560: read oid 48 snap -1 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1560: expect deleted 2026-02-17T05:55:05.291 INFO:tasks.rados.rados.0.trial115.stdout:1561: snap_remove snap 145 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1561: done (5 left) 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1562: copy_from oid 39 from oid 49 current snap is 151 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1556: finishing write tid 1 to trial11545420-50 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1556: finishing write tid 2 to trial11545420-50 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1556: finishing write tid 3 to trial11545420-50 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1556: finishing write tid 4 to trial11545420-50 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1556: finishing write tid 5 to trial11545420-50 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1556: oid 50 updating version 0 to 698 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1556: oid 50 version 698 is already newer than 696 2026-02-17T05:55:05.929 INFO:tasks.rados.rados.0.trial115.stdout:1556: oid 50 version 698 is already newer than 695 2026-02-17T05:55:05.930 INFO:tasks.rados.rados.0.trial115.stdout:1556: oid 50 version 698 is already newer than 697 2026-02-17T05:55:05.930 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 698 (ObjNum 587 snap 151 seq_num 587) dirty exists 2026-02-17T05:55:05.930 INFO:tasks.rados.rados.0.trial115.stdout:1556: left oid 50 (ObjNum 587 snap 151 seq_num 587) 2026-02-17T05:55:05.930 INFO:tasks.rados.rados.0.trial115.stdout:1559: expect (ObjNum 532 snap 136 seq_num 532) 2026-02-17T05:55:06.016 INFO:tasks.rados.rados.0.trial115.stdout:1562: finishing copy_from racing read to trial11545420-39 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:1562: finishing copy_from to trial11545420-39 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 176 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:1556: done (5 left) 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:1557: done (4 left) 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:1558: done (3 left) 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:1559: done (2 left) 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:1560: done (1 left) 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:1562: done (0 left) 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:1563: rollback oid 3 current snap is 151 2026-02-17T05:55:06.017 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 3 to 140 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1564: copy_from oid 23 from oid 48 current snap is 151 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1565: rollback oid 13 current snap is 151 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 13 to 138 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1566: rollback oid 36 current snap is 151 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 36 to 150 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1567: read oid 47 snap 150 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1567: expect deleted 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1568: delete oid 21 current snap is 151 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1566: finishing rollback tid 1 to trial11545420-36 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 0 (ObjNum 177 snap 0 seq_num 1333699200) dirty dne 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1564: finishing copy_from to trial11545420-23 2026-02-17T05:55:06.018 INFO:tasks.rados.rados.0.trial115.stdout:1564: got expected ENOENT (src dne) 2026-02-17T05:55:06.019 INFO:tasks.rados.rados.0.trial115.stdout:1564: finishing copy_from racing read to trial11545420-23 2026-02-17T05:55:06.019 INFO:tasks.rados.rados.0.trial115.stdout:1563: finishing rollback tid 0 to trial11545420-3 2026-02-17T05:55:06.019 INFO:tasks.rados.rados.0.trial115.stdout:1563: finishing rollback tid 1 to trial11545420-3 2026-02-17T05:55:06.019 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 784 (ObjNum 49 snap 0 seq_num 1634300532) dirty dne 2026-02-17T05:55:06.019 INFO:tasks.rados.rados.0.trial115.stdout:1565: finishing rollback tid 0 to trial11545420-13 2026-02-17T05:55:06.020 INFO:tasks.rados.rados.0.trial115.stdout:1563: done (5 left) 2026-02-17T05:55:06.020 INFO:tasks.rados.rados.0.trial115.stdout:1564: done (4 left) 2026-02-17T05:55:06.020 INFO:tasks.rados.rados.0.trial115.stdout:1566: done (3 left) 2026-02-17T05:55:06.020 INFO:tasks.rados.rados.0.trial115.stdout:1567: done (2 left) 2026-02-17T05:55:06.020 INFO:tasks.rados.rados.0.trial115.stdout:1568: done (1 left) 2026-02-17T05:55:06.020 INFO:tasks.rados.rados.0.trial115.stdout:1569: snap_remove snap 150 2026-02-17T05:55:06.931 INFO:tasks.rados.rados.0.trial115.stdout:1569: done (1 left) 2026-02-17T05:55:06.931 INFO:tasks.rados.rados.0.trial115.stdout:1570: snap_remove snap 149 2026-02-17T05:55:07.931 INFO:tasks.rados.rados.0.trial115.stdout:1570: done (1 left) 2026-02-17T05:55:07.932 INFO:tasks.rados.rados.0.trial115.stdout:1571: write (excl) oid 34 current snap is 151 2026-02-17T05:55:07.932 INFO:tasks.rados.rados.0.trial115.stdout:1565: finishing rollback tid 1 to trial11545420-13 2026-02-17T05:55:07.932 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 628 (ObjNum 1041 snap 0 seq_num 153772960) dirty dne 2026-02-17T05:55:07.932 INFO:tasks.rados.rados.0.trial115.stdout:1571: seq_num 589 ranges {699323=480842,1850247=429872,2858378=532392} 2026-02-17T05:55:07.936 INFO:tasks.rados.rados.0.trial115.stdout:1571: writing trial11545420-34 from 699323 to 1180165 tid 1 2026-02-17T05:55:07.940 INFO:tasks.rados.rados.0.trial115.stdout:1571: writing trial11545420-34 from 1850247 to 2280119 tid 2 2026-02-17T05:55:07.944 INFO:tasks.rados.rados.0.trial115.stdout:1571: writing trial11545420-34 from 2858378 to 3390770 tid 3 2026-02-17T05:55:07.944 INFO:tasks.rados.rados.0.trial115.stdout:1565: done (1 left) 2026-02-17T05:55:07.944 INFO:tasks.rados.rados.0.trial115.stdout:1572: copy_from oid 27 from oid 35 current snap is 151 2026-02-17T05:55:07.944 INFO:tasks.rados.rados.0.trial115.stdout:1571: finishing write tid 1 to trial11545420-34 2026-02-17T05:55:07.944 INFO:tasks.rados.rados.0.trial115.stdout:1571: finishing write tid 2 to trial11545420-34 2026-02-17T05:55:07.944 INFO:tasks.rados.rados.0.trial115.stdout:1573: write (excl) oid 8 current snap is 151 2026-02-17T05:55:07.944 INFO:tasks.rados.rados.0.trial115.stdout:1573: seq_num 590 ranges {716130=556091,1792376=573530,2769448=460017,3821956=94174} 2026-02-17T05:55:07.949 INFO:tasks.rados.rados.0.trial115.stdout:1573: writing trial11545420-8 from 716130 to 1272221 tid 1 2026-02-17T05:55:07.953 INFO:tasks.rados.rados.0.trial115.stdout:1573: writing trial11545420-8 from 1792376 to 2365906 tid 2 2026-02-17T05:55:07.956 INFO:tasks.rados.rados.0.trial115.stdout:1573: writing trial11545420-8 from 2769448 to 3229465 tid 3 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1573: writing trial11545420-8 from 3821956 to 3916130 tid 4 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1574: snap_create 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1571: finishing write tid 3 to trial11545420-34 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1571: finishing write tid 4 to trial11545420-34 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1571: finishing write tid 5 to trial11545420-34 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1571: oid 34 updating version 0 to 446 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1571: oid 34 updating version 446 to 447 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1571: oid 34 version 447 is already newer than 445 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1571: oid 34 updating version 447 to 448 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 448 (ObjNum 589 snap 151 seq_num 589) dirty exists 2026-02-17T05:55:07.959 INFO:tasks.rados.rados.0.trial115.stdout:1571: left oid 34 (ObjNum 589 snap 151 seq_num 589) 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1572: finishing copy_from racing read to trial11545420-27 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: finishing write tid 1 to trial11545420-8 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1572: finishing copy_from to trial11545420-27 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 240 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: finishing write tid 2 to trial11545420-8 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: finishing write tid 3 to trial11545420-8 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: finishing write tid 4 to trial11545420-8 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: finishing write tid 5 to trial11545420-8 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: finishing write tid 6 to trial11545420-8 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: oid 8 updating version 0 to 203 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: oid 8 version 203 is already newer than 199 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: oid 8 version 203 is already newer than 200 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: oid 8 version 203 is already newer than 202 2026-02-17T05:55:07.960 INFO:tasks.rados.rados.0.trial115.stdout:1573: oid 8 version 203 is already newer than 201 2026-02-17T05:55:07.961 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 203 (ObjNum 590 snap 151 seq_num 590) dirty exists 2026-02-17T05:55:07.961 INFO:tasks.rados.rados.0.trial115.stdout:1573: left oid 8 (ObjNum 590 snap 151 seq_num 590) 2026-02-17T05:55:08.468 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:55:08.469 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T05:55:08.469 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:55:08.469 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:55:08.472 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:55:08.472 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:55:08.472 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.541+0000 7fcab0dde640 10 monclient: get_monmap_and_config 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.541+0000 7fcab0dde640 10 monclient: build_initial_monmap 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.541+0000 7fcab0dde640 10 monclient: monmap: 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:08.542852+0000 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:08.542852+0000 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:08.543 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.541+0000 7fcab0dde640 1 Processor -- start 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 1 -- start start 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient: init 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient(hunting): picked mon.noname-b con 0x7fcaac06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 0x7fcaac06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient(hunting): picked mon.noname-a con 0x7fcaac06cc70 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fcaac134df0 0x7fcaac135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient(hunting): picked mon.noname-c con 0x7fcaac134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fcaac074150 con 0x7fcaac06cc70 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fcaac09ef70 con 0x7fcaac134df0 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcaac09f250 con 0x7fcaac06c890 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcab0dde640 10 monclient(hunting): authenticate will time out at 1154.096433s 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 0x7fcaac06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38616/0 (socket says 10.20.193.78:38616) 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 0x7fcaac06d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58166/0 (socket says 10.20.193.78:58166) 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 -- 10.20.193.78:0/1669723676 learned_addr learned my addr 10.20.193.78:0/1669723676 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient(hunting): get_auth_request con 0x7fcaac06c890 auth_method 0 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 10 monclient(hunting): get_auth_request con 0x7fcaac06cc70 auth_method 0 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 1 -- 10.20.193.78:0/1669723676 >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 msgr2=0x7fcaac06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 1 -- 10.20.193.78:0/1669723676 >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 msgr2=0x7fcaac06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 1 --2- 10.20.193.78:0/1669723676 >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 0x7fcaac06d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaa9d74640 1 --2- 10.20.193.78:0/1669723676 >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 0x7fcaac06d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient(hunting): handle_auth_done global_id 74546 payload 274 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 -- 10.20.193.78:0/1669723676 >> v2:10.20.193.115:3300/0 conn(0x7fcaac134df0 msgr2=0x7fcaac135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 --2- 10.20.193.78:0/1669723676 >> v2:10.20.193.115:3300/0 conn(0x7fcaac134df0 0x7fcaac135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 -- 10.20.193.78:0/1669723676 >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 msgr2=0x7fcaac06d050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 --2- 10.20.193.78:0/1669723676 >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 0x7fcaac06d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient: _finish_hunting 0 2026-02-17T05:55:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 monclient: found mon.noname-b 2026-02-17T05:55:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:55:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.542+0000 7fcaaa575640 1 -- 10.20.193.78:0/1669723676 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaac09df40 con 0x7fcaac06c890 2026-02-17T05:55:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaaa575640 1 -- 10.20.193.78:0/1669723676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 msgr2=0x7fcaac0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaaa575640 1 --2- 10.20.193.78:0/1669723676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac0a01f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcaac06e270 tx=0x7fcaa002ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0070931dbd15ff server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 1 -- 10.20.193.78:0/1669723676 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa003c070 con 0x7fcaac06c890 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: dump: 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: _finish_auth 0 2026-02-17T05:55:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: _check_auth_tickets 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 5 monclient: authenticate success, global_id 74546 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 1 -- 10.20.193.78:0/1669723676 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa002fde0 con 0x7fcaac06c890 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 1 -- 10.20.193.78:0/1669723676 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0037660 con 0x7fcaac06c890 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 4 set_mon_vals no callback set 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcaa9573640 10 monclient: dump: 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:08.554 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 10 monclient: shutdown 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 -- 10.20.193.78:0/1669723676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 msgr2=0x7fcaac0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 --2- 10.20.193.78:0/1669723676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac0a01f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcaac06e270 tx=0x7fcaa002ef60 comp rx=0 tx=0).stop 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 -- 10.20.193.78:0/1669723676 shutdown_connections 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 --2- 10.20.193.78:0/1669723676 >> v2:10.20.193.115:3300/0 conn(0x7fcaac134df0 0x7fcaac135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 --2- 10.20.193.78:0/1669723676 >> v2:10.20.193.78:3300/0 conn(0x7fcaac06cc70 0x7fcaac06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 --2- 10.20.193.78:0/1669723676 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac0a01f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 -- 10.20.193.78:0/1669723676 >> 10.20.193.78:0/1669723676 conn(0x7fcaac0a6620 msgr2=0x7fcaac0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 -- 10.20.193.78:0/1669723676 shutdown_connections 2026-02-17T05:55:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.545+0000 7fcab0dde640 1 -- 10.20.193.78:0/1669723676 wait complete. 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient: build_initial_monmap 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient: monmap: 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 1 Processor -- start 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 1 -- start start 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient: init 2026-02-17T05:55:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient(hunting): picked mon.noname-c con 0x7fcaac06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcaac06cc70 0x7fcaac134510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient(hunting): picked mon.noname-b con 0x7fcaac06cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 0x7fcaac124a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient(hunting): picked mon.noname-a con 0x7fcaac134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac09d360 con 0x7fcaac134df0 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac09d1e0 con 0x7fcaac06cc70 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcaac135780 con 0x7fcaac06c890 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcab0dde640 10 monclient(hunting): authenticate will time out at 1154.100626s 2026-02-17T05:55:08.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac133fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38632/0 (socket says 10.20.193.78:38632) 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 -- 10.20.193.78:0/3488806344 learned_addr learned my addr 10.20.193.78:0/3488806344 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient(hunting): get_auth_request con 0x7fcaac06c890 auth_method 0 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 0x7fcaac124a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 10 monclient(hunting): get_auth_request con 0x7fcaac134df0 auth_method 0 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 1 -- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 msgr2=0x7fcaac124a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:55:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 1 -- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 msgr2=0x7fcaac124a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 0x7fcaac124a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaad76640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 0x7fcaac124a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient(hunting): handle_auth_done global_id 74549 payload 824 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaa9d74640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcaac06cc70 0x7fcaac134510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 -- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcaac06cc70 msgr2=0x7fcaac134510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcaac06cc70 0x7fcaac134510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 -- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 msgr2=0x7fcaac124a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 0x7fcaac124a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient: _finish_hunting 0 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 monclient: found mon.noname-c 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaaa575640 1 -- 10.20.193.78:0/3488806344 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaac128a00 con 0x7fcaac06c890 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaa9d74640 10 monclient: get_auth_request con 0x7fcaac06cc70 auth_method 0 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.546+0000 7fcaa9d74640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcaac06cc70 0x7fcaac134510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.548+0000 7fcaaa575640 1 -- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 msgr2=0x7fcaac133fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.548+0000 7fcaaa575640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac133fd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009a70 tx=0x7fcaa0031070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 1 -- 10.20.193.78:0/3488806344 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0047070 con 0x7fcaac06c890 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: dump: 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: _finish_auth 0 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:55:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 5 monclient: authenticate success, global_id 74549 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 1 -- 10.20.193.78:0/3488806344 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa003f590 con 0x7fcaac06c890 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 10 monclient: _renew_subs 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaac072ef0 con 0x7fcaac06c890 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 1 -- 10.20.193.78:0/3488806344 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa003c040 con 0x7fcaac06c890 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcaa8d72640 4 set_mon_vals no callback set 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 10 monclient: dump: 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:08.561 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 10 monclient: _renew_subs 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaac09d810 con 0x7fcaac06c890 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaac074980 con 0x7fcaac06c890 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 1 -- 10.20.193.78:0/3488806344 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fcaa003f9c0 con 0x7fcaac06c890 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fca7803deb0 0x7fca78040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.549+0000 7fca92ffd640 1 -- 10.20.193.78:0/3488806344 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(329..329 src has 117..329) ==== 4811+0+0 (secure 0 0 0) 0x7fcaa0077530 con 0x7fcaac06c890 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.550+0000 7fcaa9d74640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fca7803deb0 0x7fca78040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.550+0000 7fcaa9d74640 10 monclient: get_auth_request con 0x7fca7803deb0 auth_method 0 2026-02-17T05:55:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.552+0000 7fca92ffd640 1 -- 10.20.193.78:0/3488806344 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcaa003fd90 con 0x7fcaac06c890 2026-02-17T05:55:08.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.552+0000 7fca92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:08.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.552+0000 7fca92ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:08.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.552+0000 7fcaa9d74640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fca7803deb0 0x7fca78040370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcaac134dc0 tx=0x7fca94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:08.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.665+0000 7fcab0dde640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:08.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.665+0000 7fcab0dde640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:08.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.665+0000 7fcab0dde640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:08.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.665+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcaac135af0 con 0x7fcaac06c890 2026-02-17T05:55:08.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.665+0000 7fca92ffd640 1 -- 10.20.193.78:0/3488806344 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fcaa00413d0 con 0x7fcaac06c890 2026-02-17T05:55:08.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.665+0000 7fca92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:08.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.665+0000 7fca92ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:08.666 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":90,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fca7803deb0 msgr2=0x7fca78040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fca7803deb0 0x7fca78040370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcaac134dc0 tx=0x7fca94008040 comp rx=0 tx=0).stop 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 10 monclient: shutdown 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 msgr2=0x7fcaac133fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac133fd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009a70 tx=0x7fcaa0031070 comp rx=0 tx=0).stop 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 shutdown_connections 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcaac134df0 0x7fcaac124a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fca7803deb0 0x7fca78040370 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcaac06cc70 0x7fcaac134510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 --2- 10.20.193.78:0/3488806344 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcaac06c890 0x7fcaac133fd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:08.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 >> 10.20.193.78:0/3488806344 conn(0x7fcaac0a6620 msgr2=0x7fcaac070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:08.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 shutdown_connections 2026-02-17T05:55:08.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:08.666+0000 7fcab0dde640 1 -- 10.20.193.78:0/3488806344 wait complete. 2026-02-17T05:55:08.937 INFO:tasks.rados.rados.0.trial115.stdout:1571: done (3 left) 2026-02-17T05:55:08.937 INFO:tasks.rados.rados.0.trial115.stdout:1572: done (2 left) 2026-02-17T05:55:08.937 INFO:tasks.rados.rados.0.trial115.stdout:1573: done (1 left) 2026-02-17T05:55:08.937 INFO:tasks.rados.rados.0.trial115.stdout:1574: done (0 left) 2026-02-17T05:55:08.938 INFO:tasks.rados.rados.0.trial115.stdout:1575: rollback oid 6 current snap is 152 2026-02-17T05:55:08.938 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 6 to 140 2026-02-17T05:55:08.938 INFO:tasks.rados.rados.0.trial115.stdout:1576: write (excl) oid 19 current snap is 152 2026-02-17T05:55:08.938 INFO:tasks.rados.rados.0.trial115.stdout:1576: seq_num 591 ranges {732937=631340,1850858=433541,2796871=671289,3888195=44742} 2026-02-17T05:55:08.942 INFO:tasks.rados.rados.0.trial115.stdout:1576: writing trial11545420-19 from 732937 to 1364277 tid 1 2026-02-17T05:55:08.945 INFO:tasks.rados.rados.0.trial115.stdout:1576: writing trial11545420-19 from 1850858 to 2284399 tid 2 2026-02-17T05:55:08.950 INFO:tasks.rados.rados.0.trial115.stdout:1576: writing trial11545420-19 from 2796871 to 3468160 tid 3 2026-02-17T05:55:08.953 INFO:tasks.rados.rados.0.trial115.stdout:1576: writing trial11545420-19 from 3888195 to 3932937 tid 4 2026-02-17T05:55:08.953 INFO:tasks.rados.rados.0.trial115.stdout:1577: copy_from oid 30 from oid 20 current snap is 152 2026-02-17T05:55:08.953 INFO:tasks.rados.rados.0.trial115.stdout:1575: finishing rollback tid 0 to trial11545420-6 2026-02-17T05:55:08.953 INFO:tasks.rados.rados.0.trial115.stdout:1575: finishing rollback tid 1 to trial11545420-6 2026-02-17T05:55:08.953 INFO:tasks.rados.rados.0.trial115.stdout:1575: finishing rollback tid 2 to trial11545420-6 2026-02-17T05:55:08.953 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 451 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:55:08.953 INFO:tasks.rados.rados.0.trial115.stdout:1576: finishing write tid 1 to trial11545420-19 2026-02-17T05:55:08.953 INFO:tasks.rados.rados.0.trial115.stdout:1575: done (2 left) 2026-02-17T05:55:08.954 INFO:tasks.rados.rados.0.trial115.stdout:1578: write (excl) oid 15 current snap is 152 2026-02-17T05:55:08.954 INFO:tasks.rados.rados.0.trial115.stdout:1578: seq_num 592 ranges {749744=706589,1909340=577199,3224294=482561} 2026-02-17T05:55:08.957 INFO:tasks.rados.rados.0.trial115.stdout:1578: writing trial11545420-15 from 749744 to 1456333 tid 1 2026-02-17T05:55:08.961 INFO:tasks.rados.rados.0.trial115.stdout:1578: writing trial11545420-15 from 1909340 to 2486539 tid 2 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1578: writing trial11545420-15 from 3224294 to 3706855 tid 3 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1579: read oid 32 snap -1 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1579: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1580: rollback oid 13 current snap is 152 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1576: finishing write tid 2 to trial11545420-19 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1576: finishing write tid 3 to trial11545420-19 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1576: finishing write tid 4 to trial11545420-19 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1576: finishing write tid 5 to trial11545420-19 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1576: finishing write tid 6 to trial11545420-19 2026-02-17T05:55:08.967 INFO:tasks.rados.rados.0.trial115.stdout:1576: oid 19 updating version 0 to 864 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1576: oid 19 version 864 is already newer than 861 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1576: oid 19 version 864 is already newer than 863 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1576: oid 19 version 864 is already newer than 862 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1576: oid 19 version 864 is already newer than 860 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 864 (ObjNum 591 snap 152 seq_num 591) dirty exists 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1576: left oid 19 (ObjNum 591 snap 152 seq_num 591) 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1577: finishing copy_from racing read to trial11545420-30 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 13 to 151 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1577: finishing copy_from to trial11545420-30 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 524 (ObjNum 536 snap 136 seq_num 536) dirty exists 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1578: finishing write tid 1 to trial11545420-15 2026-02-17T05:55:08.968 INFO:tasks.rados.rados.0.trial115.stdout:1578: finishing write tid 2 to trial11545420-15 2026-02-17T05:55:08.969 INFO:tasks.rados.rados.0.trial115.stdout:1576: done (4 left) 2026-02-17T05:55:08.969 INFO:tasks.rados.rados.0.trial115.stdout:1577: done (3 left) 2026-02-17T05:55:08.969 INFO:tasks.rados.rados.0.trial115.stdout:1581: write oid 8 current snap is 152 2026-02-17T05:55:08.969 INFO:tasks.rados.rados.0.trial115.stdout:1581: seq_num 593 ranges {766551=498191,1684175=720857,2851717=693833} 2026-02-17T05:55:08.970 INFO:tasks.rados.rados.0.trial115.stdout:1581: writing trial11545420-8 from 766551 to 1264742 tid 1 2026-02-17T05:55:08.975 INFO:tasks.rados.rados.0.trial115.stdout:1581: writing trial11545420-8 from 1684175 to 2405032 tid 2 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1581: writing trial11545420-8 from 2851717 to 3545550 tid 3 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1582: copy_from oid 10 from oid 6 current snap is 152 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1580: finishing rollback tid 1 to trial11545420-13 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 0 (ObjNum 49 snap 0 seq_num 4) dirty dne 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1578: finishing write tid 3 to trial11545420-15 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1578: finishing write tid 4 to trial11545420-15 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1578: finishing write tid 5 to trial11545420-15 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1578: oid 15 updating version 0 to 752 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1578: oid 15 version 752 is already newer than 751 2026-02-17T05:55:08.980 INFO:tasks.rados.rados.0.trial115.stdout:1578: oid 15 version 752 is already newer than 749 2026-02-17T05:55:08.981 INFO:tasks.rados.rados.0.trial115.stdout:1578: oid 15 version 752 is already newer than 750 2026-02-17T05:55:08.981 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 752 (ObjNum 592 snap 152 seq_num 592) dirty exists 2026-02-17T05:55:08.981 INFO:tasks.rados.rados.0.trial115.stdout:1578: left oid 15 (ObjNum 592 snap 152 seq_num 592) 2026-02-17T05:55:08.981 INFO:tasks.rados.rados.0.trial115.stdout:1579: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: finishing write tid 1 to trial11545420-8 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: finishing write tid 2 to trial11545420-8 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: finishing write tid 3 to trial11545420-8 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: finishing write tid 4 to trial11545420-8 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: finishing write tid 5 to trial11545420-8 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: oid 8 updating version 0 to 206 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: oid 8 version 206 is already newer than 204 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: oid 8 updating version 206 to 207 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: oid 8 version 207 is already newer than 205 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 207 (ObjNum 593 snap 152 seq_num 593) dirty exists 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: left oid 8 (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1582: finishing copy_from racing read to trial11545420-10 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1582: finishing copy_from to trial11545420-10 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 451 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1578: done (4 left) 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1579: done (3 left) 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1580: done (2 left) 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1581: done (1 left) 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1582: done (0 left) 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1583: read oid 16 snap -1 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1583: expect deleted 2026-02-17T05:55:09.091 INFO:tasks.rados.rados.0.trial115.stdout:1584: snap_remove snap 138 2026-02-17T05:55:09.940 INFO:tasks.rados.rados.0.trial115.stdout:1584: done (1 left) 2026-02-17T05:55:09.940 INFO:tasks.rados.rados.0.trial115.stdout:1585: read oid 14 snap -1 2026-02-17T05:55:09.940 INFO:tasks.rados.rados.0.trial115.stdout:1585: expect deleted 2026-02-17T05:55:09.940 INFO:tasks.rados.rados.0.trial115.stdout:1586: copy_from oid 49 from oid 42 current snap is 152 2026-02-17T05:55:09.941 INFO:tasks.rados.rados.0.trial115.stdout:1583: done (2 left) 2026-02-17T05:55:09.941 INFO:tasks.rados.rados.0.trial115.stdout:1587: read oid 12 snap -1 2026-02-17T05:55:09.941 INFO:tasks.rados.rados.0.trial115.stdout:1587: expect (ObjNum 528 snap 136 seq_num 528) 2026-02-17T05:55:09.941 INFO:tasks.rados.rados.0.trial115.stdout:1588: write oid 11 current snap is 152 2026-02-17T05:55:09.941 INFO:tasks.rados.rados.0.trial115.stdout:1588: seq_num 594 ranges {783358=573440,2026304=580868,3279140=621458} 2026-02-17T05:55:09.945 INFO:tasks.rados.rados.0.trial115.stdout:1588: writing trial11545420-11 from 783358 to 1356798 tid 1 2026-02-17T05:55:09.950 INFO:tasks.rados.rados.0.trial115.stdout:1588: writing trial11545420-11 from 2026304 to 2607172 tid 2 2026-02-17T05:55:09.955 INFO:tasks.rados.rados.0.trial115.stdout:1588: writing trial11545420-11 from 3279140 to 3900598 tid 3 2026-02-17T05:55:09.955 INFO:tasks.rados.rados.0.trial115.stdout:1589: snap_create 2026-02-17T05:55:09.955 INFO:tasks.rados.rados.0.trial115.stdout:1587: expect (ObjNum 528 snap 136 seq_num 528) 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: finishing write tid 1 to trial11545420-11 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: finishing write tid 2 to trial11545420-11 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1586: finishing copy_from racing read to trial11545420-49 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1586: finishing copy_from to trial11545420-49 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 298 (ObjNum 573 snap 147 seq_num 573) dirty exists 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: finishing write tid 3 to trial11545420-11 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: finishing write tid 4 to trial11545420-11 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: finishing write tid 5 to trial11545420-11 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: oid 11 updating version 0 to 768 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: oid 11 version 768 is already newer than 766 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: oid 11 version 768 is already newer than 767 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: oid 11 version 768 is already newer than 765 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 768 (ObjNum 594 snap 152 seq_num 594) dirty exists 2026-02-17T05:55:10.047 INFO:tasks.rados.rados.0.trial115.stdout:1588: left oid 11 (ObjNum 594 snap 152 seq_num 594) 2026-02-17T05:55:10.942 INFO:tasks.rados.rados.0.trial115.stdout:1585: done (4 left) 2026-02-17T05:55:10.942 INFO:tasks.rados.rados.0.trial115.stdout:1586: done (3 left) 2026-02-17T05:55:10.943 INFO:tasks.rados.rados.0.trial115.stdout:1587: done (2 left) 2026-02-17T05:55:10.943 INFO:tasks.rados.rados.0.trial115.stdout:1588: done (1 left) 2026-02-17T05:55:10.943 INFO:tasks.rados.rados.0.trial115.stdout:1589: done (0 left) 2026-02-17T05:55:10.943 INFO:tasks.rados.rados.0.trial115.stdout:1590: snap_create 2026-02-17T05:55:11.674 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.744+0000 7f652aec7640 10 monclient: get_monmap_and_config 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.744+0000 7f652aec7640 10 monclient: build_initial_monmap 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient: monmap: 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:11.746209+0000 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:11.746209+0000 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:11.744 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:11.745 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:11.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 1 Processor -- start 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 1 -- start start 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient: init 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652406c890 0x7f652406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient(hunting): picked mon.noname-b con 0x7f652406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f652409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 0x7f6524135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient(hunting): picked mon.noname-a con 0x7f652406d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f652406daf0 con 0x7f652409c4a0 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f65240a0880 con 0x7f652406d1b0 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f652409cc80 con 0x7f652406c890 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652aec7640 10 monclient(hunting): authenticate will time out at 1157.299798s 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652406c890 0x7f652406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652406c890 0x7f652406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38672/0 (socket says 10.20.193.78:38672) 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 1 -- 10.20.193.78:0/3943136006 learned_addr learned my addr 10.20.193.78:0/3943136006 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:11.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 10 monclient(hunting): get_auth_request con 0x7f652406c890 auth_method 0 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652943d640 1 --2- 10.20.193.78:0/3943136006 >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 0x7f6524135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652943d640 10 monclient(hunting): get_auth_request con 0x7f652406d1b0 auth_method 0 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652943d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652943d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652943d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f6528c3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652943d640 1 -- 10.20.193.78:0/3943136006 >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 msgr2=0x7f6524135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652943d640 1 -- 10.20.193.78:0/3943136006 >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 msgr2=0x7f6524135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.745+0000 7f652943d640 1 --2- 10.20.193.78:0/3943136006 >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 0x7f6524135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f652943d640 1 --2- 10.20.193.78:0/3943136006 >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 0x7f6524135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 10 monclient(hunting): handle_auth_done global_id 74552 payload 274 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 1 -- 10.20.193.78:0/3943136006 >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 msgr2=0x7f6524135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 1 --2- 10.20.193.78:0/3943136006 >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 0x7f6524135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 1 -- 10.20.193.78:0/3943136006 >> v1:10.20.193.115:6789/0 conn(0x7f652409c4a0 legacy=0x7f6524134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 10 monclient: _finish_hunting 0 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 1 monclient: found mon.noname-b 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:55:11.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.746+0000 7f6528c3c640 1 -- 10.20.193.78:0/3943136006 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652409cf60 con 0x7f652406c890 2026-02-17T05:55:11.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f6528c3c640 1 -- 10.20.193.78:0/3943136006 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652406c890 msgr2=0x7f652406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:11.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f6528c3c640 1 --2- 10.20.193.78:0/3943136006 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652406c890 0x7f652406cc70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6514009970 tx=0x7f651402ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550c397756f220e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:11.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 1 -- 10.20.193.78:0/3943136006 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651403c070 con 0x7f652406c890 2026-02-17T05:55:11.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:11.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: dump: 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 1 -- 10.20.193.78:0/3943136006 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651402fcd0 con 0x7f652406c890 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f652aec7640 5 monclient: authenticate success, global_id 74552 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f652aec7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 1 -- 10.20.193.78:0/3943136006 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514037660 con 0x7f652406c890 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f652aec7640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f652aec7640 4 set_mon_vals no callback set 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.748+0000 7f651b7fe640 10 monclient: dump: 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:11.749 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 10 monclient: shutdown 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 -- 10.20.193.78:0/3943136006 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652406c890 msgr2=0x7f652406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 --2- 10.20.193.78:0/3943136006 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652406c890 0x7f652406cc70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6514009970 tx=0x7f651402ee90 comp rx=0 tx=0).stop 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 -- 10.20.193.78:0/3943136006 shutdown_connections 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 --2- 10.20.193.78:0/3943136006 >> v2:10.20.193.78:3300/0 conn(0x7f652406d1b0 0x7f6524135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 --2- 10.20.193.78:0/3943136006 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652406c890 0x7f652406cc70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 -- 10.20.193.78:0/3943136006 >> 10.20.193.78:0/3943136006 conn(0x7f65240a6620 msgr2=0x7f65240a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 -- 10.20.193.78:0/3943136006 shutdown_connections 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 -- 10.20.193.78:0/3943136006 wait complete. 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 10 monclient: build_initial_monmap 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 10 monclient: monmap: 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 Processor -- start 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 1 -- start start 2026-02-17T05:55:11.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.749+0000 7f652aec7640 10 monclient: init 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f652406c890 0x7f65241cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient(hunting): picked mon.noname-b con 0x7f652406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 0x7f65241cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient(hunting): picked mon.noname-a con 0x7f652406d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652409c4a0 0x7f6524124fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient(hunting): picked mon.noname-c con 0x7f652409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65240a01a0 con 0x7f652406d1b0 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f65240a0020 con 0x7f652406c890 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f652409be20 con 0x7f652409c4a0 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652aec7640 10 monclient(hunting): authenticate will time out at 1157.304248s 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652409c4a0 0x7f6524124fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 0x7f65241cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652409c4a0 0x7f6524124fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38684/0 (socket says 10.20.193.78:38684) 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 -- 10.20.193.78:0/4162653622 learned_addr learned my addr 10.20.193.78:0/4162653622 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient(hunting): get_auth_request con 0x7f652409c4a0 auth_method 0 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 10 monclient(hunting): get_auth_request con 0x7f652406d1b0 auth_method 0 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:11.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 1 -- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 msgr2=0x7f65241cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 1 -- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 msgr2=0x7f65241cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 0x7f65241cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f651bfff640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 0x7f65241cfc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f6528c3c640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f652406c890 0x7f65241cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient(hunting): handle_auth_done global_id 74555 payload 824 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 -- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f652406c890 msgr2=0x7f65241cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f652406c890 0x7f65241cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 -- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 msgr2=0x7f65241cfc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 0x7f65241cfc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient: _finish_hunting 0 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 monclient: found mon.noname-c 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f652943d640 1 -- 10.20.193.78:0/4162653622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65241297a0 con 0x7f652409c4a0 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f6528c3c640 10 monclient: get_auth_request con 0x7f652406c890 auth_method 0 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.750+0000 7f6528c3c640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f652406c890 0x7f65241cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652943d640 1 -- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652409c4a0 msgr2=0x7f6524124fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652943d640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652409c4a0 0x7f6524124fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6520009fd0 tx=0x7f652000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 1 -- 10.20.193.78:0/4162653622 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520014620 con 0x7f652409c4a0 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: dump: 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:11.754 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: _finish_auth 0 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: _check_auth_tickets 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 1 -- 10.20.193.78:0/4162653622 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6520019070 con 0x7f652409c4a0 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652aec7640 5 monclient: authenticate success, global_id 74555 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 1 -- 10.20.193.78:0/4162653622 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520014a00 con 0x7f652409c4a0 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f651affd640 4 set_mon_vals no callback set 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f65197fa640 10 monclient: dump: 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652aec7640 10 monclient: _renew_subs 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652aec7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6524128ae0 con 0x7f652409c4a0 2026-02-17T05:55:11.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652aec7640 10 monclient: _renew_subs 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652aec7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.752+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f652409d810 con 0x7f652409c4a0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.753+0000 7f652aec7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.753+0000 7f652aec7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.753+0000 7f652aec7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.753+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6524074980 con 0x7f652409c4a0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.753+0000 7f65197fa640 1 -- 10.20.193.78:0/4162653622 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f652001d410 con 0x7f652409c4a0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.753+0000 7f65197fa640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f64f403deb0 0x7f64f4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.753+0000 7f65197fa640 1 -- 10.20.193.78:0/4162653622 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(332..332 src has 117..332) ==== 4799+0+0 (secure 0 0 0) 0x7f6520052bd0 con 0x7f652409c4a0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.753+0000 7f6528c3c640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f64f403deb0 0x7f64f4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.754+0000 7f6528c3c640 10 monclient: get_auth_request con 0x7f64f403deb0 auth_method 0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.756+0000 7f65197fa640 1 -- 10.20.193.78:0/4162653622 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6520022020 con 0x7f652409c4a0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.756+0000 7f65197fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.756+0000 7f65197fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:11.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.756+0000 7f6528c3c640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f64f403deb0 0x7f64f4040370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6514036000 tx=0x7f6514039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:11.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.870+0000 7f652aec7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:11.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.870+0000 7f652aec7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:11.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.870+0000 7f652aec7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:11.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.870+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f652406cc70 con 0x7f652409c4a0 2026-02-17T05:55:11.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.870+0000 7f65197fa640 1 -- 10.20.193.78:0/4162653622 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f65241297a0 con 0x7f652409c4a0 2026-02-17T05:55:11.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.870+0000 7f65197fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:11.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.870+0000 7f65197fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":90,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f64f403deb0 msgr2=0x7f64f4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f64f403deb0 0x7f64f4040370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6514036000 tx=0x7f6514039040 comp rx=0 tx=0).stop 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 10 monclient: shutdown 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652409c4a0 msgr2=0x7f6524124fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652409c4a0 0x7f6524124fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6520009fd0 tx=0x7f652000e4b0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 shutdown_connections 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f652409c4a0 0x7f6524124fd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f652406d1b0 0x7f65241cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f64f403deb0 0x7f64f4040370 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 --2- 10.20.193.78:0/4162653622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f652406c890 0x7f65241cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 >> 10.20.193.78:0/4162653622 conn(0x7f65240a6620 msgr2=0x7f65240a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:11.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 shutdown_connections 2026-02-17T05:55:11.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:11.871+0000 7f652aec7640 1 -- 10.20.193.78:0/4162653622 wait complete. 2026-02-17T05:55:11.943 INFO:tasks.rados.rados.0.trial115.stdout:1590: done (0 left) 2026-02-17T05:55:11.944 INFO:tasks.rados.rados.0.trial115.stdout:1591: write (excl) oid 31 current snap is 154 2026-02-17T05:55:11.944 INFO:tasks.rados.rados.0.trial115.stdout:1591: seq_num 595 ranges {400165=648689,1684786=315379} 2026-02-17T05:55:11.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: writing trial11545420-31 from 400165 to 1048854 tid 1 2026-02-17T05:55:11.951 INFO:tasks.rados.rados.0.trial115.stdout:1591: writing trial11545420-31 from 1684786 to 2000165 tid 2 2026-02-17T05:55:11.951 INFO:tasks.rados.rados.0.trial115.stdout:1592: write oid 48 current snap is 154 2026-02-17T05:55:11.951 INFO:tasks.rados.rados.0.trial115.stdout:1592: seq_num 596 ranges {416972=723938,1743268=273704} 2026-02-17T05:55:11.955 INFO:tasks.rados.rados.0.trial115.stdout:1592: writing trial11545420-48 from 416972 to 1140910 tid 1 2026-02-17T05:55:11.958 INFO:tasks.rados.rados.0.trial115.stdout:1592: writing trial11545420-48 from 1743268 to 2016972 tid 2 2026-02-17T05:55:11.958 INFO:tasks.rados.rados.0.trial115.stdout:1593: snap_remove snap 153 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1593: done (2 left) 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1594: read oid 24 snap -1 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1594: expect (ObjNum 569 snap 147 seq_num 569) 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1595: delete oid 28 current snap is 154 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: finishing write tid 1 to trial11545420-31 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: finishing write tid 2 to trial11545420-31 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: finishing write tid 3 to trial11545420-31 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: finishing write tid 4 to trial11545420-31 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: oid 31 updating version 0 to 793 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: oid 31 version 793 is already newer than 792 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: oid 31 updating version 793 to 794 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 794 (ObjNum 595 snap 154 seq_num 595) dirty exists 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1591: left oid 31 (ObjNum 595 snap 154 seq_num 595) 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1592: finishing write tid 1 to trial11545420-48 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1592: finishing write tid 2 to trial11545420-48 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1592: finishing write tid 3 to trial11545420-48 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1592: finishing write tid 4 to trial11545420-48 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1592: oid 48 updating version 0 to 874 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1592: oid 48 version 874 is already newer than 873 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1592: oid 48 updating version 874 to 875 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 875 (ObjNum 596 snap 154 seq_num 596) dirty exists 2026-02-17T05:55:12.947 INFO:tasks.rados.rados.0.trial115.stdout:1592: left oid 48 (ObjNum 596 snap 154 seq_num 596) 2026-02-17T05:55:12.948 INFO:tasks.rados.rados.0.trial115.stdout:1591: done (3 left) 2026-02-17T05:55:12.948 INFO:tasks.rados.rados.0.trial115.stdout:1592: done (2 left) 2026-02-17T05:55:12.948 INFO:tasks.rados.rados.0.trial115.stdout:1595: done (1 left) 2026-02-17T05:55:12.948 INFO:tasks.rados.rados.0.trial115.stdout:1596: snap_create 2026-02-17T05:55:12.951 INFO:tasks.rados.rados.0.trial115.stdout:1594: expect (ObjNum 569 snap 147 seq_num 569) 2026-02-17T05:55:13.949 INFO:tasks.rados.rados.0.trial115.stdout:1594: done (1 left) 2026-02-17T05:55:13.949 INFO:tasks.rados.rados.0.trial115.stdout:1596: done (0 left) 2026-02-17T05:55:13.949 INFO:tasks.rados.rados.0.trial115.stdout:1597: rollback oid 48 current snap is 155 2026-02-17T05:55:13.949 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 48 to 152 2026-02-17T05:55:13.949 INFO:tasks.rados.rados.0.trial115.stdout:1598: read oid 40 snap -1 2026-02-17T05:55:13.949 INFO:tasks.rados.rados.0.trial115.stdout:1598: expect (ObjNum 548 snap 142 seq_num 548) 2026-02-17T05:55:13.950 INFO:tasks.rados.rados.0.trial115.stdout:1599: read oid 32 snap -1 2026-02-17T05:55:13.950 INFO:tasks.rados.rados.0.trial115.stdout:1599: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:55:13.950 INFO:tasks.rados.rados.0.trial115.stdout:1600: write oid 20 current snap is 155 2026-02-17T05:55:13.950 INFO:tasks.rados.rados.0.trial115.stdout:1600: seq_num 598 ranges {450586=474436,1343879=588206} 2026-02-17T05:55:13.953 INFO:tasks.rados.rados.0.trial115.stdout:1600: writing trial11545420-20 from 450586 to 925022 tid 1 2026-02-17T05:55:13.957 INFO:tasks.rados.rados.0.trial115.stdout:1600: writing trial11545420-20 from 1343879 to 1932085 tid 2 2026-02-17T05:55:13.957 INFO:tasks.rados.rados.0.trial115.stdout:1601: write oid 37 current snap is 155 2026-02-17T05:55:13.957 INFO:tasks.rados.rados.0.trial115.stdout:1597: finishing rollback tid 0 to trial11545420-48 2026-02-17T05:55:13.957 INFO:tasks.rados.rados.0.trial115.stdout:1597: finishing rollback tid 1 to trial11545420-48 2026-02-17T05:55:13.957 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 878 (ObjNum 129 snap 0 seq_num 1334143184) dirty dne 2026-02-17T05:55:13.957 INFO:tasks.rados.rados.0.trial115.stdout:1601: seq_num 599 ranges {467393=549685,1802361=265032} 2026-02-17T05:55:13.960 INFO:tasks.rados.rados.0.trial115.stdout:1601: writing trial11545420-37 from 467393 to 1017078 tid 1 2026-02-17T05:55:13.964 INFO:tasks.rados.rados.0.trial115.stdout:1601: writing trial11545420-37 from 1802361 to 2067393 tid 2 2026-02-17T05:55:13.964 INFO:tasks.rados.rados.0.trial115.stdout:1597: done (4 left) 2026-02-17T05:55:13.964 INFO:tasks.rados.rados.0.trial115.stdout:1602: read oid 30 snap -1 2026-02-17T05:55:13.964 INFO:tasks.rados.rados.0.trial115.stdout:1602: expect (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:55:13.965 INFO:tasks.rados.rados.0.trial115.stdout:1603: snap_create 2026-02-17T05:55:13.965 INFO:tasks.rados.rados.0.trial115.stdout:1599: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:55:14.074 INFO:tasks.rados.rados.0.trial115.stdout:1600: finishing write tid 1 to trial11545420-20 2026-02-17T05:55:14.075 INFO:tasks.rados.rados.0.trial115.stdout:1600: finishing write tid 2 to trial11545420-20 2026-02-17T05:55:14.075 INFO:tasks.rados.rados.0.trial115.stdout:1598: expect (ObjNum 548 snap 142 seq_num 548) 2026-02-17T05:55:14.173 INFO:tasks.rados.rados.0.trial115.stdout:1600: finishing write tid 3 to trial11545420-20 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1600: finishing write tid 4 to trial11545420-20 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1600: oid 20 updating version 0 to 603 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1600: oid 20 version 603 is already newer than 602 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1600: oid 20 version 603 is already newer than 601 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 603 (ObjNum 598 snap 155 seq_num 598) dirty exists 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1600: left oid 20 (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1601: finishing write tid 1 to trial11545420-37 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1601: finishing write tid 2 to trial11545420-37 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1601: finishing write tid 3 to trial11545420-37 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1601: finishing write tid 4 to trial11545420-37 2026-02-17T05:55:14.174 INFO:tasks.rados.rados.0.trial115.stdout:1601: oid 37 updating version 0 to 606 2026-02-17T05:55:14.175 INFO:tasks.rados.rados.0.trial115.stdout:1601: oid 37 updating version 606 to 607 2026-02-17T05:55:14.175 INFO:tasks.rados.rados.0.trial115.stdout:1601: oid 37 version 607 is already newer than 605 2026-02-17T05:55:14.175 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 607 (ObjNum 599 snap 155 seq_num 599) dirty exists 2026-02-17T05:55:14.175 INFO:tasks.rados.rados.0.trial115.stdout:1601: left oid 37 (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:55:14.175 INFO:tasks.rados.rados.0.trial115.stdout:1602: expect (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:55:14.879 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:14.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.947+0000 7fc6286e1640 10 monclient: get_monmap_and_config 2026-02-17T05:55:14.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.947+0000 7fc6286e1640 10 monclient: build_initial_monmap 2026-02-17T05:55:14.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.947+0000 7fc6286e1640 10 monclient: monmap: 2026-02-17T05:55:14.949 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:14.949 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:14.948564+0000 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:14.948564+0000 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:14.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.947+0000 7fc6286e1640 1 Processor -- start 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.947+0000 7fc6286e1640 1 -- start start 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.947+0000 7fc6286e1640 10 monclient: init 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient(hunting): picked mon.noname-b con 0x7fc62006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient(hunting): picked mon.noname-a con 0x7fc62009d430 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient(hunting): picked mon.noname-c con 0x7fc62009bf40 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:14.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc62009ca50 con 0x7fc62009d430 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc62006d6f0 con 0x7fc62009bf40 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc620073600 con 0x7fc62006c890 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc6286e1640 10 monclient(hunting): authenticate will time out at 1160.502153s 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38700/0 (socket says 10.20.193.78:38700) 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 learned_addr learned my addr 10.20.193.78:0/4037338289 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 10 monclient(hunting): get_auth_request con 0x7fc62006c890 auth_method 0 2026-02-17T05:55:14.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 10 monclient(hunting): get_auth_request con 0x7fc62009d430 auth_method 0 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:14.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 10 monclient(hunting): get_auth_request con 0x7fc62009bf40 auth_method 0 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:14.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:14.948+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 10 monclient(hunting): get_auth_request con 0x7fc62006c890 auth_method 0 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 10 monclient(hunting): get_auth_request con 0x7fc62009d430 auth_method 0 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 10 monclient(hunting): get_auth_request con 0x7fc62009bf40 auth_method 0 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:15.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:15.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:15.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:15.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:15.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.149+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:55:15.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:15.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 10 monclient(hunting): get_auth_request con 0x7fc62006c890 auth_method 0 2026-02-17T05:55:15.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:15.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:15.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:15.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:15.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 10 monclient(hunting): get_auth_request con 0x7fc62009d430 auth_method 0 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:15.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 10 monclient(hunting): get_auth_request con 0x7fc62009bf40 auth_method 0 2026-02-17T05:55:15.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:15.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:15.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:15.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:15.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:15.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:15.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:15.549+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 10 monclient(hunting): get_auth_request con 0x7fc62006c890 auth_method 0 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 10 monclient(hunting): get_auth_request con 0x7fc62009d430 auth_method 0 2026-02-17T05:55:16.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.350+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 10 monclient(hunting): get_auth_request con 0x7fc62009bf40 auth_method 0 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:16.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:16.351+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:55:17.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 10 monclient(hunting): tick 2026-02-17T05:55:17.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:55:17.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 monclient(hunting): continuing hunt 2026-02-17T05:55:17.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:55:17.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 msgr2=0x7fc62006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:17.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc62006c890 0x7fc62006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:17.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 msgr2=0x7fc620134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:17.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.115:3300/0 conn(0x7fc62009bf40 0x7fc620134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:17.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 -- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 msgr2=0x7fc62006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:17.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 --2- 10.20.193.78:0/4037338289 >> v2:10.20.193.78:3300/0 conn(0x7fc62009d430 0x7fc62006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:17.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 10 monclient: _add_conns ranks=[1,0] 2026-02-17T05:55:17.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc600003750 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:17.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc6000074a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:17.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 -- 10.20.193.78:0/4037338289 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62006dab0 con 0x7fc6000074a0 2026-02-17T05:55:17.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 1 -- 10.20.193.78:0/4037338289 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc60000c1e0 con 0x7fc600003750 2026-02-17T05:55:17.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.948+0000 7fc607fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:17.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.950+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v1:10.20.193.78:6789/0 conn(0x7fc6000074a0 legacy=0x7fc600009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:17.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:17.950+0000 7fc626456640 1 -- 10.20.193.78:0/4037338289 >> v1:10.20.193.115:6789/0 conn(0x7fc600003750 legacy=0x7fc600005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:18.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:18.536+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v1:10.20.193.78:6789/0 conn(0x7fc6000074a0 legacy=0x7fc600009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-17T05:55:18.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:18.536+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> v1:10.20.193.78:6789/0 conn(0x7fc6000074a0 legacy=0x7fc600009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:55:18.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:18.536+0000 7fc625c55640 1 --1- 10.20.193.78:0/4037338289 >> v1:10.20.193.78:6789/0 conn(0x7fc6000074a0 0x7fc600009960 :-1 s=OPENED pgs=24 cs=1 l=1).handle_message read tag failed 2026-02-17T05:55:18.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:18.536+0000 7fc625c55640 1 --1- 10.20.193.78:0/4037338289 >> v1:10.20.193.78:6789/0 conn(0x7fc6000074a0 0x7fc600009960 :-1 s=OPENED pgs=24 cs=1 l=1).fault on lossy channel, failing 2026-02-17T05:55:18.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:18.536+0000 7fc625454640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.78:6789/0 2026-02-17T05:55:19.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487597309 0 0) 0x7fc60000c1e0 con 0x7fc600003750 2026-02-17T05:55:19.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): _negotiate 2026-02-17T05:55:19.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:19.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:19.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): my global_id is 114166 2026-02-17T05:55:19.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc608003620 con 0x7fc600003750 2026-02-17T05:55:19.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2848910776 0 0) 0x7fc608003620 con 0x7fc600003750 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 >> v1:10.20.193.78:6789/0 conn(0x7fc6000074a0 legacy=0x7fc600009960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: _finish_hunting 0 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60000d4a0 con 0x7fc600003750 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: _finish_auth 0 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: _check_auth_tickets 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fc610009780 con 0x7fc600003750 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:19.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: dump: 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:19.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 >> v1:10.20.193.115:6789/0 conn(0x7fc600003750 legacy=0x7fc600005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626c57640 1 -- 10.20.193.78:0/4037338289 reap_dead start 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc608003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): picked mon.a con 0x7fc608003b50 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc608004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): picked mon.c con 0x7fc608004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 0x7fc60800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): picked mon.b con 0x7fc608009ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:19.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc60800c490 con 0x7fc608003b50 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc60800c610 con 0x7fc608009ab0 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc60800c810 con 0x7fc608004900 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625454640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626456640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc608003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc608004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626456640 10 monclient(hunting): get_auth_request con 0x7fc608003b50 auth_method 0 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626456640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:19.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625c55640 10 monclient(hunting): get_auth_request con 0x7fc608004900 auth_method 0 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625c55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625c55640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625c55640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625c55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc625c55640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626456640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626456640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.772+0000 7fc626456640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc626c57640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 0x7fc60800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:19.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 10 monclient(hunting): handle_auth_done global_id 114166 payload 293 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 msgr2=0x7fc60800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 0x7fc60800bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 msgr2=0x7fc608003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc608003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 10 monclient: _finish_hunting 0 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 1 monclient: found mon.c 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc608003620 con 0x7fc608004900 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 10 monclient: _finish_auth 0 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 10 monclient: _check_auth_tickets 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc625c55640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:19.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc6286e1640 5 monclient: authenticate success, global_id 114166 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.773+0000 7fc6286e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625c55640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 msgr2=0x7fc608004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625c55640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc608004d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc61400cc70 tx=0x7fc61402ffc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a67a901ad7bb3c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614036070 con 0x7fc608004900 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 10 monclient: dump: 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6140309e0 con 0x7fc608004900 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 1 -- 10.20.193.78:0/4037338289 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61400f040 con 0x7fc608004900 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 4 set_mon_vals no callback set 2026-02-17T05:55:19.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc625454640 10 monclient: dump: 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:19.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 10 monclient: shutdown 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 1 -- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 msgr2=0x7fc608004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc608004d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc61400cc70 tx=0x7fc61402ffc0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 1 -- 10.20.193.78:0/4037338289 shutdown_connections 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 0x7fc60800bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc608004d80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 1 --2- 10.20.193.78:0/4037338289 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc608003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.775+0000 7fc6286e1640 1 -- 10.20.193.78:0/4037338289 >> 10.20.193.78:0/4037338289 conn(0x7fc6200a6620 msgr2=0x7fc6200a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.776+0000 7fc6286e1640 1 -- 10.20.193.78:0/4037338289 shutdown_connections 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.776+0000 7fc6286e1640 1 -- 10.20.193.78:0/4037338289 wait complete. 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.776+0000 7fc6286e1640 10 monclient: build_initial_monmap 2026-02-17T05:55:19.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.776+0000 7fc6286e1640 10 monclient: monmap: 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.776+0000 7fc6286e1640 1 Processor -- start 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.776+0000 7fc6286e1640 1 -- start start 2026-02-17T05:55:19.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.776+0000 7fc6286e1640 10 monclient: init 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.776+0000 7fc6286e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc620122370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient(hunting): picked mon.noname-a con 0x7fc608003b50 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc6201228b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient(hunting): picked mon.noname-c con 0x7fc608004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 0x7fc620122df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient(hunting): picked mon.noname-b con 0x7fc608009ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc620073380 con 0x7fc608003b50 2026-02-17T05:55:19.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc620073200 con 0x7fc608009ab0 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc62006cc70 con 0x7fc608004900 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc6286e1640 10 monclient(hunting): authenticate will time out at 1165.331046s 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc6201228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc620122370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc6201228b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38978/0 (socket says 10.20.193.78:38978) 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 1 -- 10.20.193.78:0/2488887449 learned_addr learned my addr 10.20.193.78:0/2488887449 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 10 monclient(hunting): get_auth_request con 0x7fc608004900 auth_method 0 2026-02-17T05:55:19.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient(hunting): get_auth_request con 0x7fc608003b50 auth_method 0 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 1 -- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 msgr2=0x7fc6201228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 1 -- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 msgr2=0x7fc6201228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc6201228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:19.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc625c55640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc6201228b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient(hunting): handle_auth_done global_id 124098 payload 824 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 1 -- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 msgr2=0x7fc6201228b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc6201228b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 1 -- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 msgr2=0x7fc620122df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 0x7fc620122df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient: _finish_hunting 0 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 1 monclient: found mon.noname-a 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.777+0000 7fc626456640 1 -- 10.20.193.78:0/2488887449 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc620134da0 con 0x7fc608003b50 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc626456640 1 -- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 msgr2=0x7fc620122370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:19.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc626456640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc620122370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc610002410 tx=0x7fc610003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 1 -- 10.20.193.78:0/2488887449 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610015790 con 0x7fc608003b50 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: dump: 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:19.797 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: _finish_auth 0 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 1 -- 10.20.193.78:0/2488887449 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc610015930 con 0x7fc608003b50 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 1 -- 10.20.193.78:0/2488887449 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610015d20 con 0x7fc608003b50 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc624c53640 4 set_mon_vals no callback set 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc606ffd640 10 monclient: dump: 2026-02-17T05:55:19.798 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc6286e1640 5 monclient: authenticate success, global_id 124098 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc6286e1640 10 monclient: _renew_subs 2026-02-17T05:55:19.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc6286e1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc620125db0 con 0x7fc608003b50 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc6286e1640 10 monclient: _renew_subs 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc6286e1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.779+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6200a2330 con 0x7fc608003b50 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc606ffd640 1 -- 10.20.193.78:0/2488887449 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc610028440 con 0x7fc608003b50 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc6286e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc6286e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc6286e1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620074980 con 0x7fc608003b50 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc606ffd640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5f003deb0 0x7fc5f0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc606ffd640 1 -- 10.20.193.78:0/2488887449 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(335..335 src has 117..335) ==== 4815+0+0 (secure 0 0 0) 0x7fc610053870 con 0x7fc608003b50 2026-02-17T05:55:19.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc625c55640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5f003deb0 0x7fc5f0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:19.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.780+0000 7fc625c55640 10 monclient: get_auth_request con 0x7fc5f003deb0 auth_method 0 2026-02-17T05:55:19.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.782+0000 7fc606ffd640 1 -- 10.20.193.78:0/2488887449 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc610012e10 con 0x7fc608003b50 2026-02-17T05:55:19.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.782+0000 7fc606ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:19.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.782+0000 7fc606ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:19.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.782+0000 7fc625c55640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5f003deb0 0x7fc5f0040370 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc61400cc70 tx=0x7fc614036000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:19.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.898+0000 7fc6286e1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:19.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.898+0000 7fc6286e1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:19.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.898+0000 7fc6286e1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:19.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.898+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc62009d810 con 0x7fc608003b50 2026-02-17T05:55:19.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.898+0000 7fc606ffd640 1 -- 10.20.193.78:0/2488887449 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7fc610052250 con 0x7fc608003b50 2026-02-17T05:55:19.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.898+0000 7fc606ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:19.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.898+0000 7fc606ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:19.899 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":94,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:19.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5f003deb0 msgr2=0x7fc5f0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5f003deb0 0x7fc5f0040370 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc61400cc70 tx=0x7fc614036000 comp rx=0 tx=0).stop 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 10 monclient: shutdown 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 msgr2=0x7fc620122370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc620122370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc610002410 tx=0x7fc610003ae0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 shutdown_connections 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc5f003deb0 0x7fc5f0040370 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc608009ab0 0x7fc620122df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc608004900 0x7fc6201228b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 --2- 10.20.193.78:0/2488887449 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc608003b50 0x7fc620122370 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:19.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 >> 10.20.193.78:0/2488887449 conn(0x7fc6200a6620 msgr2=0x7fc6201375b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:19.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 shutdown_connections 2026-02-17T05:55:19.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:19.899+0000 7fc6286e1640 1 -- 10.20.193.78:0/2488887449 wait complete. 2026-02-17T05:55:22.908 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient: get_monmap_and_config 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient: build_initial_monmap 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient: monmap: 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:22.980266+0000 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:22.980266+0000 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:22.980 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 1 Processor -- start 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 1 -- start start 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient: init 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fefe806c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fefe809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:22.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient(hunting): picked mon.noname-c con 0x7fefe806d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe806daf0 con 0x7fefe806c890 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe809cc80 con 0x7fefe809c4a0 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fefe80a0880 con 0x7fefe806d380 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefefb96640 10 monclient(hunting): authenticate will time out at 1168.533804s 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefed90b640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fefe806c890 0x7fefe806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46068/0 (socket says 10.20.193.78:46068) 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.979+0000 7fefed90b640 1 -- 10.20.193.78:0/2971084303 learned_addr learned my addr 10.20.193.78:0/2971084303 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 10 monclient(hunting): get_auth_request con 0x7fefe806d380 auth_method 0 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:55:22.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:22.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:22.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.980+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:22.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.982+0000 7fefed90b640 1 -- 10.20.193.78:0/2971084303 >> v1:10.20.193.78:6789/0 conn(0x7fefe806c890 legacy=0x7fefe806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:22.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:22.982+0000 7fefed10a640 1 -- 10.20.193.78:0/2971084303 >> v1:10.20.193.115:6789/0 conn(0x7fefe809c4a0 legacy=0x7fefe813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:23.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:23.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 10 monclient(hunting): get_auth_request con 0x7fefe806d380 auth_method 0 2026-02-17T05:55:23.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:23.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:23.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:23.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:55:23.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:23.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:23.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.180+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:55:23.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:23.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 10 monclient(hunting): get_auth_request con 0x7fefe806d380 auth_method 0 2026-02-17T05:55:23.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:23.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:23.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:23.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:55:23.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:23.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:23.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:23.581+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:55:24.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:24.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 10 monclient(hunting): get_auth_request con 0x7fefe806d380 auth_method 0 2026-02-17T05:55:24.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:24.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:24.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:24.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:55:24.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:24.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:24.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:24.383+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:55:25.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282881249 0 0) 0x7fefe809cc80 con 0x7fefe809c4a0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 10 monclient(hunting): _negotiate 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 10 monclient(hunting): my global_id is 114169 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefd0003620 con 0x7fefe809c4a0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3802473521 0 0) 0x7fefd0003620 con 0x7fefe809c4a0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 msgr2=0x7fefe813ead0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.773+0000 7fefec909640 1 --2- 10.20.193.78:0/2971084303 >> v2:10.20.193.115:3300/0 conn(0x7fefe806d380 0x7fefe813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 >> v1:10.20.193.78:6789/0 conn(0x7fefe806c890 legacy=0x7fefe806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: _finish_hunting 0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe809cf60 con 0x7fefe809c4a0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: _finish_auth 0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: _check_auth_tickets 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fefdc0031f0 con 0x7fefe809c4a0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: dump: 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 >> v1:10.20.193.115:6789/0 conn(0x7fefe809c4a0 legacy=0x7fefe813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:55:25.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefd0003c20 0x7fefd0004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient(hunting): picked mon.a con 0x7fefd0003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefd0004900 0x7fefd0004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient(hunting): picked mon.b con 0x7fefd0004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0009ab0 0x7fefd000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient(hunting): picked mon.c con 0x7fefd0009ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fefd000c490 con 0x7fefd0003c20 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fefd000c640 con 0x7fefd0004900 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fefd000c840 con 0x7fefd0009ab0 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefec909640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefd0003c20 0x7fefd0004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 10 monclient(hunting): get_auth_request con 0x7fefd0003c20 auth_method 0 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0009ab0 0x7fefd000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 10 monclient(hunting): get_auth_request con 0x7fefd0009ab0 auth_method 0 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 1 -- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefd0003c20 msgr2=0x7fefd0004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 1 -- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefd0003c20 msgr2=0x7fefd0004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefd0003c20 0x7fefd0004040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed90b640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefd0003c20 0x7fefd0004040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefd0004900 0x7fefd0004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient(hunting): get_auth_request con 0x7fefd0004900 auth_method 0 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0009ab0 msgr2=0x7fefd000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0009ab0 msgr2=0x7fefd000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0009ab0 0x7fefd000bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:25.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0009ab0 0x7fefd000bea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient(hunting): handle_auth_done global_id 114169 payload 293 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 1 -- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0009ab0 msgr2=0x7fefd000bea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0009ab0 0x7fefd000bea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 1 -- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefd0003c20 msgr2=0x7fefd0004040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefd0003c20 0x7fefd0004040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient: _finish_hunting 0 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 1 monclient: found mon.b 2026-02-17T05:55:25.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefee10c640 1 -- 10.20.193.78:0/2971084303 reap_dead start 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 1 -- 10.20.193.78:0/2971084303 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd0003620 con 0x7fefd0004900 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient: _finish_auth 0 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 10 monclient: _check_auth_tickets 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefed10a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefefb96640 5 monclient: authenticate success, global_id 114169 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.774+0000 7fefefb96640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefed10a640 1 -- 10.20.193.78:0/2971084303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefd0004900 msgr2=0x7fefd0004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefed10a640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefd0004900 0x7fefd0004d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fefe806dcc0 tx=0x7fefdc02f5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2a38b1dd394423 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00c0e0 con 0x7fefd0004900 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 10 monclient: dump: 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:25.777 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefefb96640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefdc005ae0 con 0x7fefd0004900 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 1 -- 10.20.193.78:0/2971084303 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc007070 con 0x7fefd0004900 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefefb96640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefefb96640 4 set_mon_vals no callback set 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.777+0000 7fefec909640 10 monclient: dump: 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 10 monclient: shutdown 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 -- 10.20.193.78:0/2971084303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefd0004900 msgr2=0x7fefd0004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefd0004900 0x7fefd0004d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fefe806dcc0 tx=0x7fefdc02f5b0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 -- 10.20.193.78:0/2971084303 shutdown_connections 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 --2- 10.20.193.78:0/2971084303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefd0004900 0x7fefd0004d80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 -- 10.20.193.78:0/2971084303 >> 10.20.193.78:0/2971084303 conn(0x7fefe80a6620 msgr2=0x7fefe80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:25.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 -- 10.20.193.78:0/2971084303 shutdown_connections 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 -- 10.20.193.78:0/2971084303 wait complete. 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 10 monclient: build_initial_monmap 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 10 monclient: monmap: 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 Processor -- start 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 1 -- start start 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.778+0000 7fefefb96640 10 monclient: init 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 0x7fefe81cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient(hunting): picked mon.noname-c con 0x7fefd0004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefe81cf890 0x7fefe8124d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient(hunting): picked mon.noname-b con 0x7fefe81cf890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefe81cfc90 0x7fefe8125290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient(hunting): picked mon.noname-a con 0x7fefe81cfc90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fefe809bfa0 con 0x7fefe81cfc90 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fefe809be20 con 0x7fefe81cf890 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fefe806cc70 con 0x7fefd0004900 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:25.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefefb96640 10 monclient(hunting): authenticate will time out at 1171.333233s 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 0x7fefe81cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefe81cfc90 0x7fefe8125290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 0x7fefe81cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38998/0 (socket says 10.20.193.78:38998) 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 1 -- 10.20.193.78:0/1605638037 learned_addr learned my addr 10.20.193.78:0/1605638037 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 10 monclient(hunting): get_auth_request con 0x7fefd0004900 auth_method 0 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient(hunting): get_auth_request con 0x7fefe81cfc90 auth_method 0 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 1 -- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 msgr2=0x7fefe81cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 1 -- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 msgr2=0x7fefe81cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 0x7fefe81cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:25.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed90b640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 0x7fefe81cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed10a640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefe81cf890 0x7fefe8124d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed10a640 10 monclient(hunting): get_auth_request con 0x7fefe81cf890 auth_method 0 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed10a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed10a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefed10a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient(hunting): handle_auth_done global_id 124101 payload 824 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 1 -- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 msgr2=0x7fefe81cf350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 0x7fefe81cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 1 -- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefe81cf890 msgr2=0x7fefe8124d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefe81cf890 0x7fefe8124d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient: _finish_hunting 0 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 1 monclient: found mon.noname-a 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.779+0000 7fefee10c640 1 -- 10.20.193.78:0/1605638037 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe8136ea0 con 0x7fefe81cfc90 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefee10c640 1 -- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefe81cfc90 msgr2=0x7fefe8125290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:25.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefee10c640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefe81cfc90 0x7fefe8125290 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefe806d810 tx=0x7fefe4015330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 1 -- 10.20.193.78:0/1605638037 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe401b070 con 0x7fefe81cfc90 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: dump: 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 1 -- 10.20.193.78:0/1605638037 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe4020680 con 0x7fefe81cfc90 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefefb96640 5 monclient: authenticate success, global_id 124101 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefefb96640 10 monclient: _renew_subs 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefefb96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe8127500 con 0x7fefe81cfc90 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 1 -- 10.20.193.78:0/1605638037 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4020a70 con 0x7fefe81cfc90 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd7fff640 4 set_mon_vals no callback set 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefd67fc640 10 monclient: dump: 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:25.782 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefefb96640 10 monclient: _renew_subs 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefefb96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.781+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe81286d0 con 0x7fefe81cfc90 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefd67fc640 1 -- 10.20.193.78:0/1605638037 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fefe4017410 con 0x7fefe81cfc90 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefefb96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefefb96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefefb96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe806cfa0 con 0x7fefe81cfc90 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefd67fc640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fefb803deb0 0x7fefb8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefd67fc640 1 -- 10.20.193.78:0/1605638037 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(335..335 src has 117..335) ==== 4815+0+0 (secure 0 0 0) 0x7fefe4053d50 con 0x7fefe81cfc90 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefed90b640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fefb803deb0 0x7fefb8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.782+0000 7fefed90b640 10 monclient: get_auth_request con 0x7fefb803deb0 auth_method 0 2026-02-17T05:55:25.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.785+0000 7fefd67fc640 1 -- 10.20.193.78:0/1605638037 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fefe4025060 con 0x7fefe81cfc90 2026-02-17T05:55:25.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.785+0000 7fefd67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:25.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.785+0000 7fefd67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:25.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.785+0000 7fefed90b640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fefb803deb0 0x7fefb8040370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefd8007cd0 tx=0x7fefd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:25.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.898+0000 7fefefb96640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:25.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.898+0000 7fefefb96640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:25.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.898+0000 7fefefb96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:25.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.898+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fefe8127280 con 0x7fefe81cfc90 2026-02-17T05:55:25.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.898+0000 7fefd67fc640 1 -- 10.20.193.78:0/1605638037 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7fefe8136ea0 con 0x7fefe81cfc90 2026-02-17T05:55:25.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.898+0000 7fefd67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:25.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.898+0000 7fefd67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:25.898 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":98,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:25.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fefb803deb0 msgr2=0x7fefb8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fefb803deb0 0x7fefb8040370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefd8007cd0 tx=0x7fefd8009040 comp rx=0 tx=0).stop 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 10 monclient: shutdown 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefe81cfc90 msgr2=0x7fefe8125290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefe81cfc90 0x7fefe8125290 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefe806d810 tx=0x7fefe4015330 comp rx=0 tx=0).stop 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 shutdown_connections 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fefe81cfc90 0x7fefe8125290 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fefb803deb0 0x7fefb8040370 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fefe81cf890 0x7fefe8124d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 --2- 10.20.193.78:0/1605638037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fefd0004900 0x7fefe81cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 >> 10.20.193.78:0/1605638037 conn(0x7fefe80a6620 msgr2=0x7fefe80a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 shutdown_connections 2026-02-17T05:55:25.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:25.899+0000 7fefefb96640 1 -- 10.20.193.78:0/1605638037 wait complete. 2026-02-17T05:55:28.907 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:28.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient: get_monmap_and_config 2026-02-17T05:55:28.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient: build_initial_monmap 2026-02-17T05:55:28.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient: monmap: 2026-02-17T05:55:28.978 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:28.978 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:28.978473+0000 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:28.978473+0000 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 1 Processor -- start 2026-02-17T05:55:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 1 -- start start 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient: init 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6d006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient(hunting): picked mon.noname-c con 0x7fb6d009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6d006d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d006daf0 con 0x7fb6d006c890 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d009cc80 con 0x7fb6d006d380 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d00a0880 con 0x7fb6d009c4a0 2026-02-17T05:55:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.977+0000 7fb6d8134640 10 monclient(hunting): authenticate will time out at 1174.531976s 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d5ea9640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb6d006c890 0x7fb6d006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50678/0 (socket says 10.20.193.78:50678) 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/215320303 learned_addr learned my addr 10.20.193.78:0/215320303 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request con 0x7fb6d009c4a0 auth_method 0 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.978+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.980+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/215320303 >> v1:10.20.193.78:6789/0 conn(0x7fb6d006c890 legacy=0x7fb6d006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:28.980+0000 7fb6d66aa640 1 -- 10.20.193.78:0/215320303 >> v1:10.20.193.115:6789/0 conn(0x7fb6d006d380 legacy=0x7fb6d013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request con 0x7fb6d009c4a0 auth_method 0 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:29.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.179+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request con 0x7fb6d009c4a0 auth_method 0 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:29.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:29.580+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:55:30.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request con 0x7fb6d009c4a0 auth_method 0 2026-02-17T05:55:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:30.381+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:55:31.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084908664 0 0) 0x7fb6d006daf0 con 0x7fb6d006c890 2026-02-17T05:55:31.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): _negotiate 2026-02-17T05:55:31.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:31.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:31.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): my global_id is 124104 2026-02-17T05:55:31.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6b8003620 con 0x7fb6d006c890 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2147481342 0 0) 0x7fb6b8003620 con 0x7fb6d006c890 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 msgr2=0x7fb6d013b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 --2- 10.20.193.78:0/215320303 >> v2:10.20.193.115:3300/0 conn(0x7fb6d009c4a0 0x7fb6d013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 >> v1:10.20.193.115:6789/0 conn(0x7fb6d006d380 legacy=0x7fb6d013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: _finish_hunting 0 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d009cf60 con 0x7fb6d006c890 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: _finish_auth 0 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: _check_auth_tickets 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fb6c00031b0 con 0x7fb6d006c890 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d8134640 5 monclient: authenticate success, global_id 124104 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d8134640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: dump: 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.777 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 >> v1:10.20.193.78:6789/0 conn(0x7fb6d006c890 legacy=0x7fb6d006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6b8004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): picked mon.a con 0x7fb6b8003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6b8004900 0x7fb6b8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): picked mon.b con 0x7fb6b8004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6b8009ab0 0x7fb6b800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): picked mon.c con 0x7fb6b8009ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b800c490 con 0x7fb6b8003c20 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b800c640 con 0x7fb6b8004900 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b800c840 con 0x7fb6b8009ab0 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d4ea7640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d8134640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6b8004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient(hunting): get_auth_request con 0x7fb6b8003c20 auth_method 0 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6b8009ab0 0x7fb6b800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:31.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 10 monclient(hunting): get_auth_request con 0x7fb6b8009ab0 auth_method 0 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6b8004900 0x7fb6b8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient(hunting): handle_auth_done global_id 124104 payload 293 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6b8009ab0 msgr2=0x7fb6b800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6b8009ab0 0x7fb6b800bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/215320303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6b8004900 msgr2=0x7fb6b8004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6b8004900 0x7fb6b8004d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient: _finish_hunting 0 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 1 monclient: found mon.a 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/215320303 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b8003620 con 0x7fb6b8003c20 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient: _finish_auth 0 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 10 monclient: _check_auth_tickets 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d5ea9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:31.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.776+0000 7fb6d66aa640 1 -- 10.20.193.78:0/215320303 reap_dead start 2026-02-17T05:55:31.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 msgr2=0x7fb6b8004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:31.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d5ea9640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6b8004040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0002ce0 tx=0x7fb6c002f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0002e5cf7ce5d5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:31.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000c160 con 0x7fb6b8003c20 2026-02-17T05:55:31.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:31.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:31.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 10 monclient: dump: 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:31.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c0005bf0 con 0x7fb6b8003c20 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 1 -- 10.20.193.78:0/215320303 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0007070 con 0x7fb6b8003c20 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 4 set_mon_vals no callback set 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d4ea7640 10 monclient: dump: 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:31.784 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 10 monclient: shutdown 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 1 -- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 msgr2=0x7fb6b8004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6b8004040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0002ce0 tx=0x7fb6c002f6a0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 1 -- 10.20.193.78:0/215320303 shutdown_connections 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 1 --2- 10.20.193.78:0/215320303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6b8004040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 1 -- 10.20.193.78:0/215320303 >> 10.20.193.78:0/215320303 conn(0x7fb6d00a6620 msgr2=0x7fb6d00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 1 -- 10.20.193.78:0/215320303 shutdown_connections 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.779+0000 7fb6d8134640 1 -- 10.20.193.78:0/215320303 wait complete. 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient: build_initial_monmap 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient: monmap: 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:31.786 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 1 Processor -- start 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 1 -- start start 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient: init 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6d01cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient(hunting): picked mon.noname-a con 0x7fb6b8003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:31.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 0x7fb6d0124bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient(hunting): picked mon.noname-c con 0x7fb6d01cf890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d01cfc90 0x7fb6d0125130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient(hunting): picked mon.noname-b con 0x7fb6d01cfc90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d009bfa0 con 0x7fb6b8003c20 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d009be20 con 0x7fb6d01cfc90 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d006cc70 con 0x7fb6d01cf890 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d8134640 10 monclient(hunting): authenticate will time out at 1177.334838s 2026-02-17T05:55:31.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6d01cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 0x7fb6d0124bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 0x7fb6d0124bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38272/0 (socket says 10.20.193.78:38272) 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6d01cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51168/0 (socket says 10.20.193.78:51168) 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 1 -- 10.20.193.78:0/2562283171 learned_addr learned my addr 10.20.193.78:0/2562283171 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request con 0x7fb6d01cf890 auth_method 0 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 10 monclient(hunting): get_auth_request con 0x7fb6b8003c20 auth_method 0 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:31.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 1 -- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 msgr2=0x7fb6d0124bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 1 -- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 msgr2=0x7fb6d0124bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 0x7fb6d0124bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 0x7fb6d0124bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.780+0000 7fb6d5ea9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d66aa640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d01cfc90 0x7fb6d0125130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d66aa640 10 monclient(hunting): get_auth_request con 0x7fb6d01cfc90 auth_method 0 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d66aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:31.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d66aa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d66aa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 10 monclient(hunting): handle_auth_done global_id 124107 payload 824 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 msgr2=0x7fb6d0124bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 0x7fb6d0124bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d01cfc90 msgr2=0x7fb6d0125130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d01cfc90 0x7fb6d0125130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 10 monclient: _finish_hunting 0 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 1 monclient: found mon.noname-a 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:31.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.781+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/2562283171 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d0128820 con 0x7fb6b8003c20 2026-02-17T05:55:31.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d5ea9640 1 -- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 msgr2=0x7fb6d01cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:31.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d5ea9640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6d01cf350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0036000 tx=0x7fb6d009d130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:31.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 1 -- 10.20.193.78:0/2562283171 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0004d00 con 0x7fb6b8003c20 2026-02-17T05:55:31.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:31.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:31.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: dump: 2026-02-17T05:55:31.794 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:31.794 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:31.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 1 -- 10.20.193.78:0/2562283171 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c0007070 con 0x7fb6b8003c20 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 1 -- 10.20.193.78:0/2562283171 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0007460 con 0x7fb6b8003c20 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 10 monclient: dump: 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:31.796 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6bffff640 4 set_mon_vals no callback set 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 5 monclient: authenticate success, global_id 124107 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 10 monclient: _renew_subs 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d0128b00 con 0x7fb6b8003c20 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 10 monclient: _renew_subs 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d0136e10 con 0x7fb6b8003c20 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6be7fc640 1 -- 10.20.193.78:0/2562283171 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb6c0007600 con 0x7fb6b8003c20 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:31.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.783+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d006cfa0 con 0x7fb6b8003c20 2026-02-17T05:55:31.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.784+0000 7fb6be7fc640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:31.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.784+0000 7fb6be7fc640 1 -- 10.20.193.78:0/2562283171 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(335..335 src has 117..335) ==== 4815+0+0 (secure 0 0 0) 0x7fb6c00775b0 con 0x7fb6b8003c20 2026-02-17T05:55:31.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.784+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:31.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.784+0000 7fb6d56a8640 10 monclient: get_auth_request con 0x7fb6a003deb0 auth_method 0 2026-02-17T05:55:31.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.786+0000 7fb6be7fc640 1 -- 10.20.193.78:0/2562283171 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb6c0075f90 con 0x7fb6b8003c20 2026-02-17T05:55:31.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.786+0000 7fb6be7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:31.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.786+0000 7fb6be7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:31.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.787+0000 7fb6d56a8640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0073520 tx=0x7fb6c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:31.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.906+0000 7fb6d8134640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:31.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.906+0000 7fb6d8134640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:31.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.906+0000 7fb6d8134640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:31.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.906+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb6d00008d0 con 0x7fb6b8003c20 2026-02-17T05:55:31.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.906+0000 7fb6be7fc640 1 -- 10.20.193.78:0/2562283171 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fb6c00796d0 con 0x7fb6b8003c20 2026-02-17T05:55:31.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.906+0000 7fb6be7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:31.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.906+0000 7fb6be7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:31.907 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":102,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:31.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 msgr2=0x7fb6a0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:31.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0073520 tx=0x7fb6c4008040 comp rx=0 tx=0).stop 2026-02-17T05:55:31.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 10 monclient: shutdown 2026-02-17T05:55:31.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:31.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 msgr2=0x7fb6d01cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:31.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6d01cf350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0036000 tx=0x7fb6d009d130 comp rx=0 tx=0).stop 2026-02-17T05:55:31.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 shutdown_connections 2026-02-17T05:55:31.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6a003deb0 0x7fb6a0040370 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6d01cfc90 0x7fb6d0125130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6d01cf890 0x7fb6d0124bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 --2- 10.20.193.78:0/2562283171 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6b8003c20 0x7fb6d01cf350 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:31.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 >> 10.20.193.78:0/2562283171 conn(0x7fb6d00a6620 msgr2=0x7fb6d00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:31.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 shutdown_connections 2026-02-17T05:55:31.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:31.907+0000 7fb6d8134640 1 -- 10.20.193.78:0/2562283171 wait complete. 2026-02-17T05:55:32.274 INFO:tasks.rados.rados.0.trial115.stdout:1598: done (5 left) 2026-02-17T05:55:32.275 INFO:tasks.rados.rados.0.trial115.stdout:1599: done (4 left) 2026-02-17T05:55:32.275 INFO:tasks.rados.rados.0.trial115.stdout:1600: done (3 left) 2026-02-17T05:55:32.275 INFO:tasks.rados.rados.0.trial115.stdout:1601: done (2 left) 2026-02-17T05:55:32.275 INFO:tasks.rados.rados.0.trial115.stdout:1602: done (1 left) 2026-02-17T05:55:32.275 INFO:tasks.rados.rados.0.trial115.stdout:1603: done (0 left) 2026-02-17T05:55:32.275 INFO:tasks.rados.rados.0.trial115.stdout:1604: rollback oid 42 current snap is 156 2026-02-17T05:55:32.275 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 152 2026-02-17T05:55:32.275 INFO:tasks.rados.rados.0.trial115.stdout:1605: delete oid 11 current snap is 156 2026-02-17T05:55:32.279 INFO:tasks.rados.rados.0.trial115.stdout:1605: done (1 left) 2026-02-17T05:55:32.279 INFO:tasks.rados.rados.0.trial115.stdout:1606: read oid 8 snap -1 2026-02-17T05:55:32.279 INFO:tasks.rados.rados.0.trial115.stdout:1606: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:55:32.279 INFO:tasks.rados.rados.0.trial115.stdout:1607: delete oid 49 current snap is 156 2026-02-17T05:55:32.279 INFO:tasks.rados.rados.0.trial115.stdout:1604: finishing rollback tid 0 to trial11545420-42 2026-02-17T05:55:32.279 INFO:tasks.rados.rados.0.trial115.stdout:1604: finishing rollback tid 1 to trial11545420-42 2026-02-17T05:55:32.279 INFO:tasks.rados.rados.0.trial115.stdout:1604: finishing rollback tid 2 to trial11545420-42 2026-02-17T05:55:32.279 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 908 (ObjNum 573 snap 147 seq_num 573) dirty exists 2026-02-17T05:55:32.282 INFO:tasks.rados.rados.0.trial115.stdout:1604: done (2 left) 2026-02-17T05:55:32.282 INFO:tasks.rados.rados.0.trial115.stdout:1607: done (1 left) 2026-02-17T05:55:32.282 INFO:tasks.rados.rados.0.trial115.stdout:1608: snap_remove snap 155 2026-02-17T05:55:34.608 INFO:tasks.rados.rados.0.trial115.stdout:1608: done (1 left) 2026-02-17T05:55:34.609 INFO:tasks.rados.rados.0.trial115.stdout:1609: read oid 39 snap 152 2026-02-17T05:55:34.609 INFO:tasks.rados.rados.0.trial115.stdout:1609: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:55:34.609 INFO:tasks.rados.rados.0.trial115.stdout:1610: write oid 3 current snap is 156 2026-02-17T05:55:34.609 INFO:tasks.rados.rados.0.trial115.stdout:1610: seq_num 602 ranges {517814=491785,1577807=540007} 2026-02-17T05:55:34.613 INFO:tasks.rados.rados.0.trial115.stdout:1610: writing trial11545420-3 from 517814 to 1009599 tid 1 2026-02-17T05:55:34.618 INFO:tasks.rados.rados.0.trial115.stdout:1610: writing trial11545420-3 from 1577807 to 2117814 tid 2 2026-02-17T05:55:34.618 INFO:tasks.rados.rados.0.trial115.stdout:1611: snap_remove snap 140 2026-02-17T05:55:34.916 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient: get_monmap_and_config 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient: build_initial_monmap 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient: monmap: 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:34.987256+0000 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:34.987256+0000 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:34.985 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:34.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 1 Processor -- start 2026-02-17T05:55:34.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 1 -- start start 2026-02-17T05:55:34.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient: init 2026-02-17T05:55:34.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:34.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient: _add_conns ranks=[2,0,3,1,4] 2026-02-17T05:55:34.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f873006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient(hunting): picked mon.noname-b con 0x7f873006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f873009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f873006d1b0 0x7f8730135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient(hunting): picked mon.noname-a con 0x7f873006d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873006daf0 con 0x7f873009c4a0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f87300a0880 con 0x7f873006d1b0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f873009cc80 con 0x7f873006c890 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f8735f49640 10 monclient(hunting): authenticate will time out at 1180.540822s 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f873006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f873006d1b0 0x7f8730135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f873006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38290/0 (socket says 10.20.193.78:38290) 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872effd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f873009c4a0 0x7f8730134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50680/0 (socket says 10.20.193.78:50680) 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 1 -- 10.20.193.78:0/2330655352 learned_addr learned my addr 10.20.193.78:0/2330655352 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 10 monclient(hunting): get_auth_request con 0x7f873006c890 auth_method 0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872ffff640 10 monclient(hunting): get_auth_request con 0x7f873006d1b0 auth_method 0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.986+0000 7f872f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 10 monclient(hunting): handle_auth_done global_id 74558 payload 274 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 1 -- 10.20.193.78:0/2330655352 >> v2:10.20.193.78:3300/0 conn(0x7f873006d1b0 msgr2=0x7f8730135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 1 --2- 10.20.193.78:0/2330655352 >> v2:10.20.193.78:3300/0 conn(0x7f873006d1b0 0x7f8730135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 1 -- 10.20.193.78:0/2330655352 >> v1:10.20.193.78:6789/0 conn(0x7f873009c4a0 legacy=0x7f8730134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 1 monclient: found mon.noname-b 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872f7fe640 1 -- 10.20.193.78:0/2330655352 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873009cf60 con 0x7f873006c890 2026-02-17T05:55:34.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.987+0000 7f872ffff640 1 --2- 10.20.193.78:0/2330655352 >> v2:10.20.193.78:3300/0 conn(0x7f873006d1b0 0x7f8730135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:55:34.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872f7fe640 1 -- 10.20.193.78:0/2330655352 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 msgr2=0x7f873006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:34.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872f7fe640 1 --2- 10.20.193.78:0/2330655352 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f873006cc70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f871c009970 tx=0x7f871c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cf6ae725a93c66e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 1 -- 10.20.193.78:0/2330655352 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c03c070 con 0x7f873006c890 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: dump: 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 1 -- 10.20.193.78:0/2330655352 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c02fcd0 con 0x7f873006c890 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f8735f49640 5 monclient: authenticate success, global_id 74558 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f8735f49640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 1 -- 10.20.193.78:0/2330655352 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c037660 con 0x7f873006c890 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f8735f49640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f8735f49640 4 set_mon_vals no callback set 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.989+0000 7f872e7fc640 10 monclient: dump: 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:34.989 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient: shutdown 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- 10.20.193.78:0/2330655352 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 msgr2=0x7f873006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 --2- 10.20.193.78:0/2330655352 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f873006cc70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f871c009970 tx=0x7f871c02ee90 comp rx=0 tx=0).stop 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- 10.20.193.78:0/2330655352 shutdown_connections 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 --2- 10.20.193.78:0/2330655352 >> v2:10.20.193.78:3300/0 conn(0x7f873006d1b0 0x7f8730135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 --2- 10.20.193.78:0/2330655352 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f873006cc70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- 10.20.193.78:0/2330655352 >> 10.20.193.78:0/2330655352 conn(0x7f87300a6620 msgr2=0x7f87300a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- 10.20.193.78:0/2330655352 shutdown_connections 2026-02-17T05:55:34.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- 10.20.193.78:0/2330655352 wait complete. 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient: build_initial_monmap 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient: monmap: 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 Processor -- start 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- start start 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient: init 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f87301cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient(hunting): picked mon.noname-c con 0x7f873006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f873006d1b0 0x7f87301360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient(hunting): picked mon.noname-b con 0x7f873006d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f873009c4a0 0x7f87301365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient(hunting): picked mon.noname-a con 0x7f873009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:34.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87300a01a0 con 0x7f873009c4a0 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f87300a0020 con 0x7f873006d1b0 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f873009be20 con 0x7f873006c890 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.990+0000 7f8735f49640 10 monclient(hunting): authenticate will time out at 1180.544996s 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f87301cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f87301cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38292/0 (socket says 10.20.193.78:38292) 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 -- 10.20.193.78:0/1850873290 learned_addr learned my addr 10.20.193.78:0/1850873290 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient(hunting): get_auth_request con 0x7f873006c890 auth_method 0 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872ffff640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f873009c4a0 0x7f87301365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872ffff640 10 monclient(hunting): get_auth_request con 0x7f873009c4a0 auth_method 0 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient(hunting): handle_auth_done global_id 74561 payload 824 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 -- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f873006d1b0 msgr2=0x7f87301360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f873006d1b0 0x7f87301360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 -- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f873009c4a0 msgr2=0x7f87301365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f873009c4a0 0x7f87301365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 monclient: found mon.noname-c 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:34.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.991+0000 7f872f7fe640 1 -- 10.20.193.78:0/1850873290 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8730072fe0 con 0x7f873006c890 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f872f7fe640 1 -- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 msgr2=0x7f87301cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f872f7fe640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f87301cf360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f871c009aa0 tx=0x7f871c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 1 -- 10.20.193.78:0/1850873290 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c047070 con 0x7f873006c890 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: dump: 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: _finish_auth 0 2026-02-17T05:55:34.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 1 -- 10.20.193.78:0/1850873290 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c03f790 con 0x7f873006c890 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f8735f49640 5 monclient: authenticate success, global_id 74561 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f8735f49640 10 monclient: _renew_subs 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f8735f49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873009d810 con 0x7f873006c890 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 1 -- 10.20.193.78:0/1850873290 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c03c040 con 0x7f873006c890 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f870bfff640 10 monclient: dump: 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f872dffb640 4 set_mon_vals no callback set 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f8735f49640 10 monclient: _renew_subs 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f8735f49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.993+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8730129ad0 con 0x7f873006c890 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f870bfff640 1 -- 10.20.193.78:0/1850873290 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f871c03fb80 con 0x7f873006c890 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f8735f49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f8735f49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f8735f49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f873006f800 con 0x7f873006c890 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f870bfff640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f870003deb0 0x7f8700040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f870bfff640 1 -- 10.20.193.78:0/1850873290 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(337..337 src has 117..337) ==== 4859+0+0 (secure 0 0 0) 0x7f871c077540 con 0x7f873006c890 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f872effd640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f870003deb0 0x7f8700040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:34.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.994+0000 7f872effd640 10 monclient: get_auth_request con 0x7f870003deb0 auth_method 0 2026-02-17T05:55:34.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.996+0000 7f870bfff640 1 -- 10.20.193.78:0/1850873290 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f873006f800 con 0x7f873006c890 2026-02-17T05:55:34.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.996+0000 7f870bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:34.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.996+0000 7f870bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:34.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:34.997+0000 7f872effd640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f870003deb0 0x7f8700040370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8730128920 tx=0x7f8720007ae0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.111+0000 7f8735f49640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:35.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.111+0000 7f8735f49640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:35.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.111+0000 7f8735f49640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:35.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.111+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8730074980 con 0x7f873006c890 2026-02-17T05:55:35.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.111+0000 7f870bfff640 1 -- 10.20.193.78:0/1850873290 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f871c075ef0 con 0x7f873006c890 2026-02-17T05:55:35.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.111+0000 7f870bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:35.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.111+0000 7f870bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:35.112 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:35.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f870003deb0 msgr2=0x7f8700040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f870003deb0 0x7f8700040370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8730128920 tx=0x7f8720007ae0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 10 monclient: shutdown 2026-02-17T05:55:35.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 msgr2=0x7f87301cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f87301cf360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f871c009aa0 tx=0x7f871c004830 comp rx=0 tx=0).stop 2026-02-17T05:55:35.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 shutdown_connections 2026-02-17T05:55:35.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f873009c4a0 0x7f87301365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f870003deb0 0x7f8700040370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f873006d1b0 0x7f87301360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 --2- 10.20.193.78:0/1850873290 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f873006c890 0x7f87301cf360 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 >> 10.20.193.78:0/1850873290 conn(0x7f87300a6620 msgr2=0x7f8730070320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 shutdown_connections 2026-02-17T05:55:35.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.114+0000 7f8735f49640 1 -- 10.20.193.78:0/1850873290 wait complete. 2026-02-17T05:55:35.123 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:55:35.123 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:55:35.124 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:55:35.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient: get_monmap_and_config 2026-02-17T05:55:35.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient: build_initial_monmap 2026-02-17T05:55:35.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient: monmap: 2026-02-17T05:55:35.195 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:35.195 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.195 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:35.194388+0000 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:35.194388+0000 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 1 Processor -- start 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 1 -- start start 2026-02-17T05:55:35.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient: init 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f804806c890 0x7f804806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient(hunting): picked mon.noname-a con 0x7f804806c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f804809c4a0 0x7f804806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient(hunting): picked mon.noname-c con 0x7f804809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8048134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804806daf0 con 0x7f8048134df0 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f80480a0880 con 0x7f804806c890 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f804806d6f0 con 0x7f804809c4a0 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.193+0000 7f804e972640 10 monclient(hunting): authenticate will time out at 1180.747940s 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f804806c890 0x7f804806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f804cee8640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f8048134df0 0x7f8048135280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50692/0 (socket says 10.20.193.78:50692) 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f804cee8640 1 -- 10.20.193.78:0/4250014568 learned_addr learned my addr 10.20.193.78:0/4250014568 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient(hunting): get_auth_request con 0x7f804806c890 auth_method 0 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f80477fe640 1 --2- 10.20.193.78:0/4250014568 >> v2:10.20.193.115:3300/0 conn(0x7f804809c4a0 0x7f804806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f80477fe640 10 monclient(hunting): get_auth_request con 0x7f804809c4a0 auth_method 0 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f80477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f80477fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f80477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient(hunting): handle_auth_done global_id 124116 payload 274 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 1 -- 10.20.193.78:0/4250014568 >> v2:10.20.193.115:3300/0 conn(0x7f804809c4a0 msgr2=0x7f804806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 1 --2- 10.20.193.78:0/4250014568 >> v2:10.20.193.115:3300/0 conn(0x7f804809c4a0 0x7f804806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 1 -- 10.20.193.78:0/4250014568 >> v1:10.20.193.78:6789/0 conn(0x7f8048134df0 legacy=0x7f8048135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient: _finish_hunting 0 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 1 monclient: found mon.noname-a 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:35.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.194+0000 7f8047fff640 1 -- 10.20.193.78:0/4250014568 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804809cc80 con 0x7f804806c890 2026-02-17T05:55:35.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8047fff640 1 -- 10.20.193.78:0/4250014568 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f804806c890 msgr2=0x7f804806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8047fff640 1 --2- 10.20.193.78:0/4250014568 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f804806c890 0x7f804806cc70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f804809c980 tx=0x7f803402ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d1970526e100a0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 1 -- 10.20.193.78:0/4250014568 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803403c070 con 0x7f804806c890 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: dump: 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.210 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: _finish_auth 0 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 1 -- 10.20.193.78:0/4250014568 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803402fbd0 con 0x7f804806c890 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f804e972640 5 monclient: authenticate success, global_id 124116 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f804e972640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:35.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 1 -- 10.20.193.78:0/4250014568 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034037440 con 0x7f804806c890 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f804e972640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f804e972640 4 set_mon_vals no callback set 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.196+0000 7f8046ffd640 10 monclient: dump: 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.212 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 10 monclient: shutdown 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 -- 10.20.193.78:0/4250014568 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f804806c890 msgr2=0x7f804806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 --2- 10.20.193.78:0/4250014568 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f804806c890 0x7f804806cc70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f804809c980 tx=0x7f803402ed50 comp rx=0 tx=0).stop 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 -- 10.20.193.78:0/4250014568 shutdown_connections 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 --2- 10.20.193.78:0/4250014568 >> v2:10.20.193.115:3300/0 conn(0x7f804809c4a0 0x7f804806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 --2- 10.20.193.78:0/4250014568 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f804806c890 0x7f804806cc70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 -- 10.20.193.78:0/4250014568 >> 10.20.193.78:0/4250014568 conn(0x7f80480a6620 msgr2=0x7f80480a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 -- 10.20.193.78:0/4250014568 shutdown_connections 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 -- 10.20.193.78:0/4250014568 wait complete. 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 10 monclient: build_initial_monmap 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 10 monclient: monmap: 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:35.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 Processor -- start 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 1 -- start start 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.197+0000 7f804e972640 10 monclient: init 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f804806c890 0x7f80481cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient(hunting): picked mon.noname-c con 0x7f804806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f804809c4a0 0x7f80481cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient(hunting): picked mon.noname-b con 0x7f804809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 0x7f8048121620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient(hunting): picked mon.noname-a con 0x7f8048134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:35.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80480a01a0 con 0x7f8048134df0 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80480a0020 con 0x7f804809c4a0 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f804809be20 con 0x7f804806c890 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804e972640 10 monclient(hunting): authenticate will time out at 1180.752200s 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 0x7f8048121620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f804806c890 0x7f80481cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f804806c890 0x7f80481cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38308/0 (socket says 10.20.193.78:38308) 2026-02-17T05:55:35.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 1 -- 10.20.193.78:0/2023750751 learned_addr learned my addr 10.20.193.78:0/2023750751 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 0x7f8048121620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51230/0 (socket says 10.20.193.78:51230) 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 10 monclient(hunting): get_auth_request con 0x7f804806c890 auth_method 0 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient(hunting): get_auth_request con 0x7f8048134df0 auth_method 0 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f80477fe640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f804809c4a0 0x7f80481cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f80477fe640 10 monclient(hunting): get_auth_request con 0x7f804809c4a0 auth_method 0 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f80477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f80477fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f80477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient(hunting): handle_auth_done global_id 124119 payload 824 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f804806c890 msgr2=0x7f80481cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f804806c890 0x7f80481cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f804809c4a0 msgr2=0x7f80481cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f804809c4a0 0x7f80481cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient: _finish_hunting 0 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 1 monclient: found mon.noname-a 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.198+0000 7f804cee8640 1 -- 10.20.193.78:0/2023750751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8048125290 con 0x7f8048134df0 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804cee8640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 msgr2=0x7f8048121620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804cee8640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 0x7f8048121620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f803c00cb20 tx=0x7f803c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c00acf0 con 0x7f8048134df0 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: dump: 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.219 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: _finish_auth 0 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:35.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803c012400 con 0x7f8048134df0 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804e972640 5 monclient: authenticate success, global_id 124119 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c0127f0 con 0x7f8048134df0 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f80467fc640 4 set_mon_vals no callback set 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f8044ff9640 10 monclient: dump: 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.221 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804e972640 10 monclient: _renew_subs 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804e972640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8048123de0 con 0x7f8048134df0 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804e972640 10 monclient: _renew_subs 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804e972640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.200+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80480a1770 con 0x7f8048134df0 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f804e972640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f804e972640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048121b60 0x7f8048121f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f804e972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8048074980 con 0x7f8048121b60 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048121b60 0x7f8048121f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 10 monclient: get_auth_request con 0x7f8048121b60 auth_method 0 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f804cee8640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 0x7f8048121620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f803c00cb20 tx=0x7f803c005bc0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f803c052c60 osd_map(337..337 src has 117..337) 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f803c012990 con 0x7f8048134df0 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 10 monclient(hunting): handle_auth_done global_id 124122 payload 824 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8047fff640 1 -- 10.20.193.78:0/2023750751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80480726f0 con 0x7f8048121b60 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f8044ff9640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f801803deb0 0x7f8018040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f80477fe640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f801803deb0 0x7f8018040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.201+0000 7f80477fe640 10 monclient: get_auth_request con 0x7f801803deb0 auth_method 0 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.203+0000 7f8047fff640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048121b60 msgr2=0x7f8048121f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.203+0000 7f8047fff640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048121b60 0x7f8048121f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f803403a040 tx=0x7f80340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.203+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034048070 con 0x7f8048121b60 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.203+0000 7f8044ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.203+0000 7f8044ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.203+0000 7f8044ff9640 10 monclient: dump: 2026-02-17T05:55:35.224 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.204+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f803404d450 con 0x7f8048121b60 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.204+0000 7f8044ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.204+0000 7f8044ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.204+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048121b60 msgr2=0x7f8048121f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.204+0000 7f8044ff9640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048121b60 0x7f8048121f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f803403a040 tx=0x7f80340047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.204+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048121b60 msgr2=0x7f8048121f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.204+0000 7f8044ff9640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048121b60 0x7f8048121f40 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f803403a040 tx=0x7f80340047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.204+0000 7f80477fe640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f801803deb0 0x7f8018040370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8048122e80 tx=0x7f8038008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f804e972640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f804e972640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048122480 0x7f80481d8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f804e972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80480744d0 con 0x7f8048122480 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048122480 0x7f80481d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 10 monclient: get_auth_request con 0x7f8048122480 auth_method 0 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 10 monclient(hunting): handle_auth_done global_id 124125 payload 824 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.207+0000 7f8047fff640 1 -- 10.20.193.78:0/2023750751 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f804809d760 con 0x7f8048122480 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8047fff640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048122480 msgr2=0x7f80481d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8047fff640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048122480 0x7f80481d8540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8034047000 tx=0x7f8034009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803403c040 con 0x7f8048122480 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 10 monclient: dump: 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f8034041ce0 con 0x7f8048122480 2026-02-17T05:55:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:55:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048122480 msgr2=0x7f80481d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048122480 0x7f80481d8540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8034047000 tx=0x7f8034009290 comp rx=0 tx=0).stop 2026-02-17T05:55:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048122480 msgr2=0x7f80481d8540 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:35.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048122480 0x7f80481d8540 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8034047000 tx=0x7f8034009290 comp rx=0 tx=0).stop 2026-02-17T05:55:35.229 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:55:35.229 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 106, 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 26676, 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:55:35.230 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:55:35.231 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:55:35.232 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.233 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.234 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.235 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.236 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.237 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:35.238 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.239 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:55:35.240 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:55:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f801803deb0 msgr2=0x7f8018040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f801803deb0 0x7f8018040370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8048122e80 tx=0x7f8038008040 comp rx=0 tx=0).stop 2026-02-17T05:55:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 10 monclient: shutdown 2026-02-17T05:55:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 msgr2=0x7f8048121620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 0x7f8048121620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f803c00cb20 tx=0x7f803c005bc0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804cee8640 1 -- 10.20.193.78:0/2023750751 reap_dead start 2026-02-17T05:55:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f8044ff9640 1 -- 10.20.193.78:0/2023750751 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(337..337 src has 117..337) ==== 4859+0+0 (unknown 0 0 0) 0x7f803c052c60 con 0x7f8048134df0 2026-02-17T05:55:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 shutdown_connections 2026-02-17T05:55:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 1 --2- 10.20.193.78:0/2023750751 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8048134df0 0x7f8048121620 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.209+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 >> 10.20.193.78:0/2023750751 conn(0x7f80480a6620 msgr2=0x7f8048071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.210+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 shutdown_connections 2026-02-17T05:55:35.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.210+0000 7f804e972640 1 -- 10.20.193.78:0/2023750751 wait complete. 2026-02-17T05:55:35.242 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:55:35.242 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:55:35.242 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:55:35.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.312+0000 7f44275cc640 10 monclient: get_monmap_and_config 2026-02-17T05:55:35.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.312+0000 7f44275cc640 10 monclient: build_initial_monmap 2026-02-17T05:55:35.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.312+0000 7f44275cc640 10 monclient: monmap: 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:35.313390+0000 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:35.313390+0000 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:35.312 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.312+0000 7f44275cc640 1 Processor -- start 2026-02-17T05:55:35.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.312+0000 7f44275cc640 1 -- start start 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.312+0000 7f44275cc640 10 monclient: init 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.312+0000 7f44275cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f44200a6470 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6850 0x7f442013d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 10 monclient(hunting): picked mon.noname-b con 0x7f44200a6850 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f44200a6c30 0x7f442013f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 10 monclient(hunting): picked mon.noname-a con 0x7f44200a6c30 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44200a0730 con 0x7f44200a6470 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f442009fa00 con 0x7f44200a6c30 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f44200a0450 con 0x7f44200a6850 2026-02-17T05:55:35.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f44275cc640 10 monclient(hunting): authenticate will time out at 1180.867073s 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425341640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f44200a6470 0x7f44200a1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50708/0 (socket says 10.20.193.78:50708) 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f44200a6c30 0x7f442013f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425341640 1 -- 10.20.193.78:0/55232485 learned_addr learned my addr 10.20.193.78:0/55232485 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient(hunting): get_auth_request con 0x7f44200a6c30 auth_method 0 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4424b40640 1 --2- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6850 0x7f442013d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4424b40640 10 monclient(hunting): get_auth_request con 0x7f44200a6850 auth_method 0 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4424b40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4424b40640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4424b40640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4424b40640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4424b40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4424b40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient(hunting): handle_auth_done global_id 124128 payload 274 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 1 -- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6850 msgr2=0x7f442013d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 1 --2- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6850 0x7f442013d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 1 -- 10.20.193.78:0/55232485 >> v1:10.20.193.78:6789/0 conn(0x7f44200a6470 legacy=0x7f44200a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient: _finish_hunting 0 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 1 monclient: found mon.noname-a 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.313+0000 7f4425b42640 1 -- 10.20.193.78:0/55232485 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44200a5ea0 con 0x7f44200a6c30 2026-02-17T05:55:35.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.315+0000 7f4425b42640 1 -- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6c30 msgr2=0x7f442013f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.315+0000 7f4425b42640 1 --2- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6c30 0x7f442013f7c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f441c009d90 tx=0x7f441c02f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301054763ff2041 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.315+0000 7f440ffff640 1 -- 10.20.193.78:0/55232485 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c03c070 con 0x7f44200a6c30 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.315+0000 7f440ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 10 monclient: dump: 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.322 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 10 monclient: _finish_auth 0 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 1 -- 10.20.193.78:0/55232485 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f441c037440 con 0x7f44200a6c30 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 5 monclient: authenticate success, global_id 124128 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 1 -- 10.20.193.78:0/55232485 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c037810 con 0x7f44200a6c30 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f440ffff640 10 monclient: dump: 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.323 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 4 set_mon_vals no callback set 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 10 monclient: shutdown 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 1 -- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6c30 msgr2=0x7f442013f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 1 --2- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6c30 0x7f442013f7c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f441c009d90 tx=0x7f441c02f190 comp rx=0 tx=0).stop 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 1 -- 10.20.193.78:0/55232485 shutdown_connections 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 1 --2- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6c30 0x7f442013f7c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 1 --2- 10.20.193.78:0/55232485 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6850 0x7f442013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 1 -- 10.20.193.78:0/55232485 >> 10.20.193.78:0/55232485 conn(0x7f44200aae40 msgr2=0x7f44200ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 1 -- 10.20.193.78:0/55232485 shutdown_connections 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 1 -- 10.20.193.78:0/55232485 wait complete. 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 10 monclient: build_initial_monmap 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.316+0000 7f44275cc640 10 monclient: monmap: 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 1 Processor -- start 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 1 -- start start 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient: init 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6470 0x7f442013eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient(hunting): picked mon.noname-a con 0x7f44200a6470 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f44200a6850 0x7f442013f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient(hunting): picked mon.noname-b con 0x7f44200a6850 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6c30 0x7f4420094640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient(hunting): picked mon.noname-c con 0x7f44200a6c30 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4420140090 con 0x7f44200a6470 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f442013ff10 con 0x7f44200a6850 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4420140210 con 0x7f44200a6c30 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f44275cc640 10 monclient(hunting): authenticate will time out at 1180.871670s 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6470 0x7f442013eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6470 0x7f442013eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51270/0 (socket says 10.20.193.78:51270) 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 1 -- 10.20.193.78:0/242336443 learned_addr learned my addr 10.20.193.78:0/242336443 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 10 monclient(hunting): get_auth_request con 0x7f44200a6470 auth_method 0 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6c30 0x7f4420094640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient(hunting): get_auth_request con 0x7f44200a6c30 auth_method 0 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4424b40640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f44200a6850 0x7f442013f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4424b40640 10 monclient(hunting): get_auth_request con 0x7f44200a6850 auth_method 0 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4424b40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4424b40640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4424b40640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient(hunting): handle_auth_done global_id 74570 payload 824 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f44200a6850 msgr2=0x7f442013f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f44200a6850 0x7f442013f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6470 msgr2=0x7f442013eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6470 0x7f442013eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient: _finish_hunting 0 2026-02-17T05:55:35.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 1 monclient: found mon.noname-c 2026-02-17T05:55:35.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:35.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:35.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425b42640 1 -- 10.20.193.78:0/242336443 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4420096e00 con 0x7f44200a6c30 2026-02-17T05:55:35.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.317+0000 7f4425341640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f44200a6470 0x7f442013eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425b42640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6c30 msgr2=0x7f4420094640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425b42640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6c30 0x7f4420094640 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f441c036000 tx=0x7f441c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c03c050 con 0x7f44200a6c30 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: dump: 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: _finish_auth 0 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f441c048070 con 0x7f44200a6c30 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c041c60 con 0x7f44200a6c30 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:35.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440f7fe640 4 set_mon_vals no callback set 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 10 monclient: dump: 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 5 monclient: authenticate success, global_id 74570 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 10 monclient: _renew_subs 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44200982b0 con 0x7f44200a6c30 2026-02-17T05:55:35.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 10 monclient: _renew_subs 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44200a2a50 con 0x7f44200a6c30 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4420097d50 0x7f4420094b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4420098130 con 0x7f4420097d50 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4420097d50 0x7f4420094b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 10 monclient: get_auth_request con 0x7f4420097d50 auth_method 0 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f441c04d460 con 0x7f44200a6c30 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 10 monclient(hunting): handle_auth_done global_id 74573 payload 824 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.320+0000 7f4425341640 1 -- 10.20.193.78:0/242336443 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4420074980 con 0x7f4420097d50 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.321+0000 7f440dffb640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43f003deb0 0x7f43f0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.321+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(337..337 src has 117..337) ==== 4859+0+0 (secure 0 0 0) 0x7f441c078680 con 0x7f44200a6c30 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.321+0000 7f4424b40640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43f003deb0 0x7f43f0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.321+0000 7f4424b40640 10 monclient: get_auth_request con 0x7f43f003deb0 auth_method 0 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f4425341640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4420097d50 msgr2=0x7f4420094b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f4425341640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4420097d50 0x7f4420094b80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f441000ec30 tx=0x7f441000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410017070 con 0x7f4420097d50 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 10 monclient: dump: 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.337 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4410013470 con 0x7f4420097d50 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4420097d50 msgr2=0x7f4420094b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4420097d50 0x7f4420094b80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f441000ec30 tx=0x7f441000c520 comp rx=0 tx=0).stop 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4420097d50 msgr2=0x7f4420094b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f440dffb640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4420097d50 0x7f4420094b80 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f441000ec30 tx=0x7f441000c520 comp rx=0 tx=0).stop 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.323+0000 7f4424b40640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43f003deb0 0x7f43f0040370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4420095ea0 tx=0x7f4414008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f44275cc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:55:35.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f44275cc640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200950c0 0x7f44201da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f44275cc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f44200a1480 con 0x7f44200950c0 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200950c0 0x7f44201da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 10 monclient: get_auth_request con 0x7f44200950c0 auth_method 0 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 10 monclient(hunting): handle_auth_done global_id 74576 payload 824 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.326+0000 7f4425341640 1 -- 10.20.193.78:0/242336443 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4420096b80 con 0x7f44200950c0 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f4425341640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200950c0 msgr2=0x7f44201da4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f4425341640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200950c0 0x7f44201da4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4410019000 tx=0x7f44100059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410017070 con 0x7f44200950c0 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 10 monclient: dump: 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f4410006000 con 0x7f44200950c0 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:55:35.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200950c0 msgr2=0x7f44201da4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200950c0 0x7f44201da4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4410019000 tx=0x7f44100059d0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200950c0 msgr2=0x7f44201da4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:35.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.328+0000 7f440dffb640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200950c0 0x7f44201da4e0 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4410019000 tx=0x7f44100059d0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.341 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:55:35.341 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:35.341 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:35.341 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 106, 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 307221, 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.342 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.343 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:35.344 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.345 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.346 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.347 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.348 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.349 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.350 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:55:35.351 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.351 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.351 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.351 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:55:35.351 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43f003deb0 msgr2=0x7f43f0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43f003deb0 0x7f43f0040370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4420095ea0 tx=0x7f4414008040 comp rx=0 tx=0).stop 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 10 monclient: shutdown 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6c30 msgr2=0x7f4420094640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f4425b42640 1 -- 10.20.193.78:0/242336443 reap_dead start 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6c30 0x7f4420094640 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f441c036000 tx=0x7f441c004b40 comp rx=0 tx=0).stop 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 shutdown_connections 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 --2- 10.20.193.78:0/242336443 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f44200a6c30 0x7f4420094640 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 >> 10.20.193.78:0/242336443 conn(0x7f44200aae40 msgr2=0x7f44200a97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 shutdown_connections 2026-02-17T05:55:35.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.329+0000 7f44275cc640 1 -- 10.20.193.78:0/242336443 wait complete. 2026-02-17T05:55:35.353 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:55:35.353 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:55:35.353 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:55:35.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient: get_monmap_and_config 2026-02-17T05:55:35.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient: build_initial_monmap 2026-02-17T05:55:35.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient: monmap: 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:35.424253+0000 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:35.424253+0000 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 1 Processor -- start 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 1 -- start start 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient: init 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient(hunting): picked mon.noname-b con 0x7fc3c806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:35.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc3c809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc3c806d1b0 0x7fc3c8135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient(hunting): picked mon.noname-a con 0x7fc3c806d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c806daf0 con 0x7fc3c809c4a0 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c80a0880 con 0x7fc3c806d1b0 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c809cc80 con 0x7fc3c806c890 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3cd42c640 10 monclient(hunting): authenticate will time out at 1180.977843s 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c77fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc3c806d1b0 0x7fc3c8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c67fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc3c809c4a0 0x7fc3c8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50720/0 (socket says 10.20.193.78:50720) 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c6ffd640 10 monclient(hunting): get_auth_request con 0x7fc3c806c890 auth_method 0 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c67fc640 1 -- 10.20.193.78:0/1570262501 learned_addr learned my addr 10.20.193.78:0/1570262501 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c77fe640 10 monclient(hunting): get_auth_request con 0x7fc3c806d1b0 auth_method 0 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.423+0000 7fc3c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 10 monclient(hunting): handle_auth_done global_id 124134 payload 274 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 1 -- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 msgr2=0x7fc3c806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 1 --2- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 1 -- 10.20.193.78:0/1570262501 >> v1:10.20.193.78:6789/0 conn(0x7fc3c809c4a0 legacy=0x7fc3c8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 1 monclient: found mon.noname-a 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c77fe640 1 -- 10.20.193.78:0/1570262501 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c809cf60 con 0x7fc3c806d1b0 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.424+0000 7fc3c6ffd640 1 --2- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c77fe640 1 -- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 msgr2=0x7fc3c8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c77fe640 1 --2- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 0x7fc3c8135500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009d90 tx=0x7fc3b802f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e231e194e8ddec94 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 1 -- 10.20.193.78:0/1570262501 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b803c070 con 0x7fc3c806d1b0 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:35.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: dump: 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 1 -- 10.20.193.78:0/1570262501 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8037440 con 0x7fc3c806d1b0 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3cd42c640 5 monclient: authenticate success, global_id 124134 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3cd42c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 1 -- 10.20.193.78:0/1570262501 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8037810 con 0x7fc3c806d1b0 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3c5ffb640 10 monclient: dump: 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3cd42c640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3cd42c640 4 set_mon_vals no callback set 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.426+0000 7fc3cd42c640 10 monclient: shutdown 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 msgr2=0x7fc3c8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 --2- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 0x7fc3c8135500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009d90 tx=0x7fc3b802f150 comp rx=0 tx=0).stop 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- 10.20.193.78:0/1570262501 shutdown_connections 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 --2- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 0x7fc3c8135500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 --2- 10.20.193.78:0/1570262501 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- 10.20.193.78:0/1570262501 >> 10.20.193.78:0/1570262501 conn(0x7fc3c80a6620 msgr2=0x7fc3c80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- 10.20.193.78:0/1570262501 shutdown_connections 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- 10.20.193.78:0/1570262501 wait complete. 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient: build_initial_monmap 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient: monmap: 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 Processor -- start 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- start start 2026-02-17T05:55:35.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient: init 2026-02-17T05:55:35.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c81cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient(hunting): picked mon.noname-c con 0x7fc3c806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 0x7fc3c8125cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient(hunting): picked mon.noname-a con 0x7fc3c806d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c809c4a0 0x7fc3c8126200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient(hunting): picked mon.noname-b con 0x7fc3c809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80a01a0 con 0x7fc3c806d1b0 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80a0020 con 0x7fc3c809c4a0 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c809be20 con 0x7fc3c806c890 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3cd42c640 10 monclient(hunting): authenticate will time out at 1180.981930s 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3c67fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 0x7fc3c8125cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.427+0000 7fc3c6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c81cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c81cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38342/0 (socket says 10.20.193.78:38342) 2026-02-17T05:55:35.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 0x7fc3c8125cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51280/0 (socket says 10.20.193.78:51280) 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 1 -- 10.20.193.78:0/3671437753 learned_addr learned my addr 10.20.193.78:0/3671437753 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient(hunting): get_auth_request con 0x7fc3c806c890 auth_method 0 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 10 monclient(hunting): get_auth_request con 0x7fc3c806d1b0 auth_method 0 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient(hunting): handle_auth_done global_id 74582 payload 824 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c809c4a0 msgr2=0x7fc3c8126200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c809c4a0 0x7fc3c8126200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 msgr2=0x7fc3c8125cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 0x7fc3c8125cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 1 monclient: found mon.noname-c 2026-02-17T05:55:35.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c6ffd640 1 -- 10.20.193.78:0/3671437753 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c8072700 con 0x7fc3c806c890 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.428+0000 7fc3c67fc640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3c806d1b0 0x7fc3c8125cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3c6ffd640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 msgr2=0x7fc3c81cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3c6ffd640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c81cf710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00eb90 tx=0x7fc3bc00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc017070 con 0x7fc3c806c890 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: dump: 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc00ce50 con 0x7fc3c806c890 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 5 monclient: authenticate success, global_id 74582 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 10 monclient: _renew_subs 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c81283a0 con 0x7fc3c806c890 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc013690 con 0x7fc3c806c890 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3a77fe640 10 monclient: dump: 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.439 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3c57fa640 4 set_mon_vals no callback set 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 10 monclient: _renew_subs 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c8129570 con 0x7fc3c806c890 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c81cfc50 0x7fc3c8126740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c81278d0 con 0x7fc3c81cfc50 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3c67fc640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c81cfc50 0x7fc3c8126740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3c67fc640 10 monclient: get_auth_request con 0x7fc3c81cfc50 auth_method 0 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3c67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.430+0000 7fc3c67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc3bc024060 con 0x7fc3c806c890 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3a77fe640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403deb0 0x7fc394040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_done global_id 114187 payload 824 2026-02-17T05:55:35.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3c67fc640 1 -- 10.20.193.78:0/3671437753 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3c81293a0 con 0x7fc3c81cfc50 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(337..337 src has 117..337) ==== 4859+0+0 (secure 0 0 0) 0x7fc3bc052870 con 0x7fc3c806c890 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3c77fe640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403deb0 0x7fc394040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.431+0000 7fc3c77fe640 10 monclient: get_auth_request con 0x7fc39403deb0 auth_method 0 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3c67fc640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c81cfc50 msgr2=0x7fc3c8126740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3c67fc640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c81cfc50 0x7fc3c8126740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8136c50 tx=0x7fc3b000f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0017020 con 0x7fc3c81cfc50 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 10 monclient: dump: 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fc3b000fe90 con 0x7fc3c81cfc50 2026-02-17T05:55:35.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c81cfc50 msgr2=0x7fc3c8126740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c81cfc50 0x7fc3c8126740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8136c50 tx=0x7fc3b000f5a0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c81cfc50 msgr2=0x7fc3c8126740 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.433+0000 7fc3a77fe640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c81cfc50 0x7fc3c8126740 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8136c50 tx=0x7fc3b000f5a0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.434+0000 7fc3c77fe640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403deb0 0x7fc394040370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80a7600 tx=0x7fc3b8031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3cd42c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3cd42c640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c8126c80 0x7fc3c81da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3cd42c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c809e4e0 con 0x7fc3c8126c80 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c8126c80 0x7fc3c81da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 10 monclient: get_auth_request con 0x7fc3c8126c80 auth_method 0 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 10 monclient(hunting): handle_auth_done global_id 114190 payload 824 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.437+0000 7fc3c67fc640 1 -- 10.20.193.78:0/3671437753 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc3c8127090 con 0x7fc3c8126c80 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3c67fc640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c8126c80 msgr2=0x7fc3c81da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3c67fc640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c8126c80 0x7fc3c81da3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8136c50 tx=0x7fc3b0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0017030 con 0x7fc3c8126c80 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 10 monclient: dump: 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7fc3b00059b0 con 0x7fc3c8126c80 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c8126c80 msgr2=0x7fc3c81da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c8126c80 0x7fc3c81da3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8136c50 tx=0x7fc3b0004a40 comp rx=0 tx=0).stop 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c8126c80 msgr2=0x7fc3c81da3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3a77fe640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3c8126c80 0x7fc3c81da3c0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8136c50 tx=0x7fc3b0004a40 comp rx=0 tx=0).stop 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 106, 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 79521, 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:55:35.445 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:55:35.446 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:35.447 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:55:35.448 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:55:35.449 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:55:35.450 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:55:35.451 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.452 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:55:35.453 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403deb0 msgr2=0x7fc394040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc39403deb0 0x7fc394040370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80a7600 tx=0x7fc3b8031000 comp rx=0 tx=0).stop 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 10 monclient: shutdown 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 msgr2=0x7fc3c81cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c81cf710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00eb90 tx=0x7fc3bc00c520 comp rx=0 tx=0).stop 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3c77fe640 1 -- 10.20.193.78:0/3671437753 reap_dead start 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 shutdown_connections 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 --2- 10.20.193.78:0/3671437753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3c806c890 0x7fc3c81cf710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 >> 10.20.193.78:0/3671437753 conn(0x7fc3c80a6620 msgr2=0x7fc3c8071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 shutdown_connections 2026-02-17T05:55:35.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.440+0000 7fc3cd42c640 1 -- 10.20.193.78:0/3671437753 wait complete. 2026-02-17T05:55:35.455 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:55:35.455 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:55:35.455 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:55:35.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.522+0000 7f55051be640 10 monclient: get_monmap_and_config 2026-02-17T05:55:35.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.522+0000 7f55051be640 10 monclient: build_initial_monmap 2026-02-17T05:55:35.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.522+0000 7f55051be640 10 monclient: monmap: 2026-02-17T05:55:35.523 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:35.523 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.523 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:35.523756+0000 2026-02-17T05:55:35.523 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:35.523756+0000 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.522+0000 7f55051be640 1 Processor -- start 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.522+0000 7f55051be640 1 -- start start 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.522+0000 7f55051be640 10 monclient: init 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f550006c890 0x7f55000a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient(hunting): picked mon.noname-c con 0x7f550006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f550006cc70 0x7f550006d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient(hunting): picked mon.noname-a con 0x7f550006cc70 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:35.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5500134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5500074150 con 0x7f5500134df0 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f550009ef70 con 0x7f550006cc70 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f550009f250 con 0x7f550006c890 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f55051be640 10 monclient(hunting): authenticate will time out at 1181.077293s 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f550006cc70 0x7f550006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f550006cc70 0x7f550006d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51294/0 (socket says 10.20.193.78:51294) 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 1 -- 10.20.193.78:0/667094361 learned_addr learned my addr 10.20.193.78:0/667094361 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient(hunting): get_auth_request con 0x7f550006cc70 auth_method 0 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fed76640 1 --2- 10.20.193.78:0/667094361 >> v2:10.20.193.115:3300/0 conn(0x7f550006c890 0x7f55000a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fed76640 10 monclient(hunting): get_auth_request con 0x7f550006c890 auth_method 0 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient(hunting): handle_auth_done global_id 124140 payload 274 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 1 -- 10.20.193.78:0/667094361 >> v2:10.20.193.115:3300/0 conn(0x7f550006c890 msgr2=0x7f55000a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 1 --2- 10.20.193.78:0/667094361 >> v2:10.20.193.115:3300/0 conn(0x7f550006c890 0x7f55000a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 1 -- 10.20.193.78:0/667094361 >> v1:10.20.193.115:6789/0 conn(0x7f5500134df0 legacy=0x7f5500135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient: _finish_hunting 0 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 1 monclient: found mon.noname-a 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.523+0000 7f54fe575640 1 -- 10.20.193.78:0/667094361 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550009df40 con 0x7f550006cc70 2026-02-17T05:55:35.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.525+0000 7f54fe575640 1 -- 10.20.193.78:0/667094361 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006cc70 msgr2=0x7f550006d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.525+0000 7f54fe575640 1 --2- 10.20.193.78:0/667094361 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006cc70 0x7f550006d080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f550006a480 tx=0x7f54e802eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933c2f615ba6f941 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 1 -- 10.20.193.78:0/667094361 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e803c070 con 0x7f550006cc70 2026-02-17T05:55:35.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: dump: 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: _finish_auth 0 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: _check_auth_tickets 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 5 monclient: authenticate success, global_id 124140 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 1 -- 10.20.193.78:0/667094361 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e802fd50 con 0x7f550006cc70 2026-02-17T05:55:35.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 1 -- 10.20.193.78:0/667094361 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8037660 con 0x7f550006cc70 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 4 set_mon_vals no callback set 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f54fdd74640 10 monclient: dump: 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 10 monclient: shutdown 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 1 -- 10.20.193.78:0/667094361 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006cc70 msgr2=0x7f550006d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 1 --2- 10.20.193.78:0/667094361 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006cc70 0x7f550006d080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f550006a480 tx=0x7f54e802eed0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 1 -- 10.20.193.78:0/667094361 shutdown_connections 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 1 --2- 10.20.193.78:0/667094361 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006cc70 0x7f550006d080 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 1 --2- 10.20.193.78:0/667094361 >> v2:10.20.193.115:3300/0 conn(0x7f550006c890 0x7f55000a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 1 -- 10.20.193.78:0/667094361 >> 10.20.193.78:0/667094361 conn(0x7f55000a6620 msgr2=0x7f55000a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 1 -- 10.20.193.78:0/667094361 shutdown_connections 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 1 -- 10.20.193.78:0/667094361 wait complete. 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 10 monclient: build_initial_monmap 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.526+0000 7f55051be640 10 monclient: monmap: 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 1 Processor -- start 2026-02-17T05:55:35.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 1 -- start start 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient: init 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006c890 0x7f55001cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient(hunting): picked mon.noname-a con 0x7f550006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f550006cc70 0x7f5500134150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient(hunting): picked mon.noname-c con 0x7f550006cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5500134df0 0x7f5500134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient(hunting): picked mon.noname-b con 0x7f5500134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f550009d360 con 0x7f550006c890 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f550009d1e0 con 0x7f5500134df0 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5500135910 con 0x7f550006cc70 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f55051be640 10 monclient(hunting): authenticate will time out at 1181.081474s 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006c890 0x7f55001cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006c890 0x7f55001cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51306/0 (socket says 10.20.193.78:51306) 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 -- 10.20.193.78:0/1621405185 learned_addr learned my addr 10.20.193.78:0/1621405185 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient(hunting): get_auth_request con 0x7f550006c890 auth_method 0 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fe575640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f550006cc70 0x7f5500134150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fe575640 10 monclient(hunting): get_auth_request con 0x7f550006cc70 auth_method 0 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient(hunting): handle_auth_done global_id 124143 payload 824 2026-02-17T05:55:35.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 -- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f550006cc70 msgr2=0x7f5500134150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f550006cc70 0x7f5500134150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54ff577640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5500134df0 0x7f5500134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 -- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5500134df0 msgr2=0x7f5500134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5500134df0 0x7f5500134690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient: _finish_hunting 0 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 monclient: found mon.noname-a 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54fed76640 1 -- 10.20.193.78:0/1621405185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55001288a0 con 0x7f550006c890 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54ff577640 10 monclient: get_auth_request con 0x7f5500134df0 auth_method 0 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.527+0000 7f54ff577640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5500134df0 0x7f5500134690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54fed76640 1 -- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006c890 msgr2=0x7f55001cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54fed76640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006c890 0x7f55001cf280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54f400e7c0 tx=0x7f54f400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 1 -- 10.20.193.78:0/1621405185 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4017070 con 0x7f550006c890 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: dump: 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.536 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f55051be640 5 monclient: authenticate success, global_id 124143 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 1 -- 10.20.193.78:0/1621405185 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f400cae0 con 0x7f550006c890 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f55051be640 10 monclient: _renew_subs 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f55051be640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55001cf7c0 con 0x7f550006c890 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 1 -- 10.20.193.78:0/1621405185 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f400f040 con 0x7f550006c890 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54fd573640 4 set_mon_vals no callback set 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.529+0000 7f54df7fe640 10 monclient: dump: 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f55051be640 10 monclient: _renew_subs 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f55051be640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55000a3990 con 0x7f550006c890 2026-02-17T05:55:35.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f54df7fe640 1 -- 10.20.193.78:0/1621405185 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f54f4012050 con 0x7f550006c890 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f55051be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f55051be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f55051be640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5500068410 con 0x7f550006c890 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f54df7fe640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54cc03deb0 0x7f54cc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.530+0000 7f54df7fe640 1 -- 10.20.193.78:0/1621405185 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(337..337 src has 117..337) ==== 4859+0+0 (secure 0 0 0) 0x7f54f4052710 con 0x7f550006c890 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.531+0000 7f54fe575640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54cc03deb0 0x7f54cc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.531+0000 7f54fe575640 10 monclient: get_auth_request con 0x7f54cc03deb0 auth_method 0 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.533+0000 7f54df7fe640 1 -- 10.20.193.78:0/1621405185 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5500068410 con 0x7f550006c890 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.533+0000 7f54df7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:35.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.533+0000 7f54df7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:35.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.533+0000 7f54fe575640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54cc03deb0 0x7f54cc040370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f54e8036000 tx=0x7f54e8033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:35.611 INFO:tasks.rados.rados.0.trial115.stdout:1611: done (3 left) 2026-02-17T05:55:35.612 INFO:tasks.rados.rados.0.trial115.stdout:1612: snap_create 2026-02-17T05:55:35.612 INFO:tasks.rados.rados.0.trial115.stdout:1606: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:55:35.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.649+0000 7f55051be640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:55:35.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.649+0000 7f55051be640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:55:35.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.649+0000 7f55051be640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:35.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.649+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f55000a01d0 con 0x7f550006c890 2026-02-17T05:55:35.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.649+0000 7f54df7fe640 1 -- 10.20.193.78:0/1621405185 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f54f40510c0 con 0x7f550006c890 2026-02-17T05:55:35.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.649+0000 7f54df7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:55:35.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.649+0000 7f54df7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.649+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.649+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=2,health=12,kv=11,logm=56} crc {auth=2934678383,config=692619927,health=2677522725,kv=2049492640,logm=3639174154}) 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.649+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=2,health=12,kv=12,logm=55} crc {auth=3562308651,config=692619927,health=2677522725,kv=449989585,logm=3109368672}) 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.649+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.649+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=2,health=12,kv=11,logm=56} crc {auth=2934678383,config=692619927,health=2677522725,kv=2049492640,logm=3639174154}) 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.649+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=20,config=2,health=12,kv=11,logm=55} crc {auth=390803813,config=692619927,health=3012087796,kv=3037601728,logm=1747321010}) 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.650+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.650+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3080351961}) 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.650+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=531120615}) 2026-02-17T05:55:35.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.650+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.650+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3080351961}) 2026-02-17T05:55:35.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.650+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=642713520}) 2026-02-17T05:55:35.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.650+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164945217}) 2026-02-17T05:55:35.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1078785234}) 2026-02-17T05:55:35.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164945217}) 2026-02-17T05:55:35.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3444239326}) 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54cc03deb0 msgr2=0x7f54cc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54cc03deb0 0x7f54cc040370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f54e8036000 tx=0x7f54e8033000 comp rx=0 tx=0).stop 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 10 monclient: shutdown 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006c890 msgr2=0x7f55001cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006c890 0x7f55001cf280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54f400e7c0 tx=0x7f54f400ec90 comp rx=0 tx=0).stop 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 shutdown_connections 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54cc03deb0 0x7f54cc040370 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5500134df0 0x7f5500134690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f550006cc70 0x7f5500134150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 --2- 10.20.193.78:0/1621405185 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f550006c890 0x7f55001cf280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 >> 10.20.193.78:0/1621405185 conn(0x7f55000a6620 msgr2=0x7f550011ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 shutdown_connections 2026-02-17T05:55:35.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:35.651+0000 7f55051be640 1 -- 10.20.193.78:0/1621405185 wait complete. 2026-02-17T05:55:35.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222325836}) 2026-02-17T05:55:35.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1248834839}) 2026-02-17T05:55:35.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222325836}) 2026-02-17T05:55:35.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3299685920}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3412607123}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391915809}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3412607123}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.651+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=179938325}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=807944185}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1104662481}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=807944185}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3546268655}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2290529131}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=517143691}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2290529131}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.652+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2736253091}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570154275}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4262349939}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570154275}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2782536881}) 2026-02-17T05:55:35.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2413913348}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1592933050}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2413913348}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3160546470}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3190487939}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2034594400}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3190487939}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.653+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571855174}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3588860215}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1783829957}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3588860215}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4105008365}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1423704490}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=892448341}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1423704490}) 2026-02-17T05:55:35.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=458967355}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662883930}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328661594}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662883930}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.654+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3587964068}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2634620908}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2657441230}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2634620908}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1303401754}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891936940}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2585110565}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891936940}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.655+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=125013269}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341396663}) 2026-02-17T05:55:35.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1463140945}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341396663}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408681638}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3657168590}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1123367225}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3657168590}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2673679388}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1626429123}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2532644268}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1626429123}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.656+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=371624597}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2205343098}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=803083271}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2205343098}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2044810845}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=155559367}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2134678647}) 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=155559367}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=938362694}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3238609194}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2970409783}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3238609194}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.657+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3369850656}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1769147696}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1556160890}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1769147696}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4065739638}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1833155453}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1995121164}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1833155453}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1059337985}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4149767156}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3855464484}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4149767156}) 2026-02-17T05:55:35.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.658+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1753649502}) 2026-02-17T05:55:35.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=972861984}) 2026-02-17T05:55:35.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2755431277}) 2026-02-17T05:55:35.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=972861984}) 2026-02-17T05:55:35.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=337131994}) 2026-02-17T05:55:35.662 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=15} crc {logm=1074362851,mdsmap=1160443688,mgr=4261142878,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=499049966}) 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=77,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10} crc {logm=2579133724,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1891683720}) 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=15} crc {logm=1074362851,mdsmap=1160443688,mgr=4261142878,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=499049966}) 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.659+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=76,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12} crc {logm=547360694,mdsmap=1730306971,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1151828836}) 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=1,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=59,osdmap=17} crc {mgrstat=2321845214,mon_config_key=1912222953,monmap=1447252912,osd_metadata=2997046373,osd_pg_creating=3754449400,osd_snap=4054357488,osdmap=81130515}) 2026-02-17T05:55:35.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgrstat=1,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=62,osdmap=16} crc {mgrstat=2321845214,mon_config_key=1987855637,monmap=1784110042,osd_metadata=2240493509,osd_pg_creating=3754449400,osd_snap=1864987694,osdmap=51467529}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=1,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=59,osdmap=17} crc {mgrstat=2321845214,mon_config_key=1912222953,monmap=1447252912,osd_metadata=2997046373,osd_pg_creating=3754449400,osd_snap=4054357488,osdmap=81130515}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=1,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=60,osdmap=18} crc {mgrstat=2321845214,mon_config_key=89122857,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=3754449400,osd_snap=1062929390,osdmap=1316426660}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4166981222}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3997155876}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4166981222}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=749283728}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.660+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1118793099}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2836786431}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1118793099}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=999960427}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2124554519}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3718728925}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2124554519}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.661+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2490857991}) 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.663+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.663+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=14} crc {osdmap=23169585}) 2026-02-17T05:55:35.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.663+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=15} crc {osdmap=4104793154}) 2026-02-17T05:55:35.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.663+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:55:35.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.663+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=14} crc {osdmap=23169585}) 2026-02-17T05:55:35.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:55:35.663+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=16} crc {osdmap=1346863832}) 2026-02-17T05:55:35.836 INFO:tasks.rados.rados.0.trial115.stdout:1609: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:55:35.979 INFO:tasks.rados.rados.0.trial115.stdout:1610: finishing write tid 1 to trial11545420-3 2026-02-17T05:55:35.980 INFO:tasks.rados.rados.0.trial115.stdout:1610: finishing write tid 2 to trial11545420-3 2026-02-17T05:55:35.980 INFO:tasks.rados.rados.0.trial115.stdout:1610: finishing write tid 3 to trial11545420-3 2026-02-17T05:55:35.980 INFO:tasks.rados.rados.0.trial115.stdout:1610: finishing write tid 4 to trial11545420-3 2026-02-17T05:55:35.980 INFO:tasks.rados.rados.0.trial115.stdout:1610: oid 3 updating version 0 to 797 2026-02-17T05:55:35.980 INFO:tasks.rados.rados.0.trial115.stdout:1610: oid 3 updating version 797 to 799 2026-02-17T05:55:35.980 INFO:tasks.rados.rados.0.trial115.stdout:1610: oid 3 version 799 is already newer than 798 2026-02-17T05:55:35.980 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 799 (ObjNum 602 snap 156 seq_num 602) dirty exists 2026-02-17T05:55:35.980 INFO:tasks.rados.rados.0.trial115.stdout:1610: left oid 3 (ObjNum 602 snap 156 seq_num 602) 2026-02-17T05:55:36.619 INFO:tasks.rados.rados.0.trial115.stdout:1606: done (3 left) 2026-02-17T05:55:36.620 INFO:tasks.rados.rados.0.trial115.stdout:1609: done (2 left) 2026-02-17T05:55:36.620 INFO:tasks.rados.rados.0.trial115.stdout:1610: done (1 left) 2026-02-17T05:55:36.620 INFO:tasks.rados.rados.0.trial115.stdout:1612: done (0 left) 2026-02-17T05:55:36.620 INFO:tasks.rados.rados.0.trial115.stdout:1613: snap_create 2026-02-17T05:55:36.663 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:55:36.663 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:36.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.730+0000 7ff623f87640 10 monclient: get_monmap_and_config 2026-02-17T05:55:36.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.730+0000 7ff623f87640 10 monclient: build_initial_monmap 2026-02-17T05:55:36.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.730+0000 7ff623f87640 10 monclient: monmap: 2026-02-17T05:55:36.732 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:36.732 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.732 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:36.731631+0000 2026-02-17T05:55:36.732 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:36.731631+0000 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.730+0000 7ff623f87640 1 Processor -- start 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.730+0000 7ff623f87640 1 -- start start 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.730+0000 7ff623f87640 10 monclient: init 2026-02-17T05:55:36.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff61c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06d380 0x7ff61c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient(hunting): picked mon.noname-b con 0x7ff61c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff61c13a240 0x7ff61c13c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient(hunting): picked mon.noname-a con 0x7ff61c13a240 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff61c074150 con 0x7ff61c06c890 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff61c09ef70 con 0x7ff61c13a240 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff61c09f250 con 0x7ff61c06d380 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff623f87640 10 monclient(hunting): authenticate will time out at 1182.285166s 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06d380 0x7ff61c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6224fd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff61c13a240 0x7ff61c13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06d380 0x7ff61c0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38356/0 (socket says 10.20.193.78:38356) 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 1 -- 10.20.193.78:0/461958358 learned_addr learned my addr 10.20.193.78:0/461958358 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient(hunting): get_auth_request con 0x7ff61c06d380 auth_method 0 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6224fd640 10 monclient(hunting): get_auth_request con 0x7ff61c13a240 auth_method 0 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6224fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6224fd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6224fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6224fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6224fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6224fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient(hunting): handle_auth_done global_id 74588 payload 274 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 1 -- 10.20.193.78:0/461958358 >> v2:10.20.193.78:3300/0 conn(0x7ff61c13a240 msgr2=0x7ff61c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 1 --2- 10.20.193.78:0/461958358 >> v2:10.20.193.78:3300/0 conn(0x7ff61c13a240 0x7ff61c13c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 1 -- 10.20.193.78:0/461958358 >> v1:10.20.193.115:6789/0 conn(0x7ff61c06c890 legacy=0x7ff61c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient: _finish_hunting 0 2026-02-17T05:55:36.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 1 monclient: found mon.noname-b 2026-02-17T05:55:36.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:36.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:55:36.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.731+0000 7ff6214fb640 1 -- 10.20.193.78:0/461958358 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c09df40 con 0x7ff61c06d380 2026-02-17T05:55:36.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.733+0000 7ff6214fb640 1 -- 10.20.193.78:0/461958358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06d380 msgr2=0x7ff61c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:36.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.733+0000 7ff6214fb640 1 --2- 10.20.193.78:0/461958358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06d380 0x7ff61c0a01f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff61c06e270 tx=0x7ff61002ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7092a7c799c0cf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:36.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.733+0000 7ff620cfa640 1 -- 10.20.193.78:0/461958358 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61003c070 con 0x7ff61c06d380 2026-02-17T05:55:36.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.733+0000 7ff620cfa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.733+0000 7ff620cfa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.733+0000 7ff620cfa640 10 monclient: dump: 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 10 monclient: _finish_auth 0 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 10 monclient: _check_auth_tickets 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 5 monclient: authenticate success, global_id 74588 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 1 -- 10.20.193.78:0/461958358 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61002fde0 con 0x7ff61c06d380 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 1 -- 10.20.193.78:0/461958358 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610037660 con 0x7ff61c06d380 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff620cfa640 10 monclient: dump: 2026-02-17T05:55:36.745 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 4 set_mon_vals no callback set 2026-02-17T05:55:36.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 10 monclient: shutdown 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 -- 10.20.193.78:0/461958358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06d380 msgr2=0x7ff61c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 --2- 10.20.193.78:0/461958358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06d380 0x7ff61c0a01f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff61c06e270 tx=0x7ff61002ef60 comp rx=0 tx=0).stop 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 -- 10.20.193.78:0/461958358 shutdown_connections 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 --2- 10.20.193.78:0/461958358 >> v2:10.20.193.78:3300/0 conn(0x7ff61c13a240 0x7ff61c13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 --2- 10.20.193.78:0/461958358 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06d380 0x7ff61c0a01f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 -- 10.20.193.78:0/461958358 >> 10.20.193.78:0/461958358 conn(0x7ff61c0a6620 msgr2=0x7ff61c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 -- 10.20.193.78:0/461958358 shutdown_connections 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 -- 10.20.193.78:0/461958358 wait complete. 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 10 monclient: build_initial_monmap 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 10 monclient: monmap: 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:36.747 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 Processor -- start 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 1 -- start start 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.734+0000 7ff623f87640 10 monclient: init 2026-02-17T05:55:36.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06c890 0x7ff61c13db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient(hunting): picked mon.noname-c con 0x7ff61c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff61c06d380 0x7ff61c13e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient(hunting): picked mon.noname-b con 0x7ff61c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff61c13a240 0x7ff61c13e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient(hunting): picked mon.noname-a con 0x7ff61c13a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c13ccf0 con 0x7ff61c13a240 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c13cb70 con 0x7ff61c06d380 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff61c13ce70 con 0x7ff61c06c890 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff623f87640 10 monclient(hunting): authenticate will time out at 1182.289235s 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06c890 0x7ff61c13db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06c890 0x7ff61c13db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38366/0 (socket says 10.20.193.78:38366) 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 -- 10.20.193.78:0/390140203 learned_addr learned my addr 10.20.193.78:0/390140203 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient(hunting): get_auth_request con 0x7ff61c06c890 auth_method 0 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff6224fd640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff61c13a240 0x7ff61c13e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff6224fd640 10 monclient(hunting): get_auth_request con 0x7ff61c13a240 auth_method 0 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff6224fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff6224fd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff6224fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff6224fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff6224fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff6224fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient(hunting): handle_auth_done global_id 74591 payload 824 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 -- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff61c06d380 msgr2=0x7ff61c13e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff61c06d380 0x7ff61c13e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 -- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff61c13a240 msgr2=0x7ff61c13e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff61c13a240 0x7ff61c13e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient: _finish_hunting 0 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 monclient: found mon.noname-c 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.735+0000 7ff621cfc640 1 -- 10.20.193.78:0/390140203 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c06f210 con 0x7ff61c06c890 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff621cfc640 1 -- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06c890 msgr2=0x7ff61c13db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff621cfc640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06c890 0x7ff61c13db80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c600 tx=0x7ff60c00cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 1 -- 10.20.193.78:0/390140203 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c012a40 con 0x7ff61c06c890 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: dump: 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 1 -- 10.20.193.78:0/390140203 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c017070 con 0x7ff61c06c890 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 1 -- 10.20.193.78:0/390140203 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c012e20 con 0x7ff61c06c890 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:36.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60a7fc640 10 monclient: dump: 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff60bfff640 4 set_mon_vals no callback set 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff623f87640 5 monclient: authenticate success, global_id 74591 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff623f87640 10 monclient: _renew_subs 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff623f87640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c09d810 con 0x7ff61c06c890 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff623f87640 10 monclient: _renew_subs 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff623f87640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.737+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c06ef30 con 0x7ff61c06c890 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff60a7fc640 1 -- 10.20.193.78:0/390140203 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff60c01c460 con 0x7ff61c06c890 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff623f87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff623f87640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff623f87640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c073e20 con 0x7ff61c06c890 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff60a7fc640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff5ec03deb0 0x7ff5ec040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff60a7fc640 1 -- 10.20.193.78:0/390140203 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(339..339 src has 117..339) ==== 4847+0+0 (secure 0 0 0) 0x7ff60c052680 con 0x7ff61c06c890 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff6214fb640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff5ec03deb0 0x7ff5ec040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.738+0000 7ff6214fb640 10 monclient: get_auth_request con 0x7ff5ec03deb0 auth_method 0 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.741+0000 7ff60a7fc640 1 -- 10.20.193.78:0/390140203 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff61c073e20 con 0x7ff61c06c890 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.741+0000 7ff60a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.741+0000 7ff60a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:36.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.741+0000 7ff6214fb640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff5ec03deb0 0x7ff5ec040370 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff610036000 tx=0x7ff610039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:36.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.856+0000 7ff623f87640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:36.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.856+0000 7ff623f87640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:36.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.856+0000 7ff623f87640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:36.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.856+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff61c074980 con 0x7ff61c06c890 2026-02-17T05:55:36.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.856+0000 7ff60a7fc640 1 -- 10.20.193.78:0/390140203 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7ff60c01ac60 con 0x7ff61c06c890 2026-02-17T05:55:36.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.856+0000 7ff60a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:36.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.856+0000 7ff60a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:36.857 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:36.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff5ec03deb0 msgr2=0x7ff5ec040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff5ec03deb0 0x7ff5ec040370 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff610036000 tx=0x7ff610039040 comp rx=0 tx=0).stop 2026-02-17T05:55:36.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 10 monclient: shutdown 2026-02-17T05:55:36.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:36.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06c890 msgr2=0x7ff61c13db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06c890 0x7ff61c13db80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c600 tx=0x7ff60c00cad0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 shutdown_connections 2026-02-17T05:55:36.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff61c13a240 0x7ff61c13e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff5ec03deb0 0x7ff5ec040370 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff61c06d380 0x7ff61c13e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 --2- 10.20.193.78:0/390140203 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff61c06c890 0x7ff61c13db80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 >> 10.20.193.78:0/390140203 conn(0x7ff61c0a6620 msgr2=0x7ff61c1393e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:36.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 shutdown_connections 2026-02-17T05:55:36.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.857+0000 7ff623f87640 1 -- 10.20.193.78:0/390140203 wait complete. 2026-02-17T05:55:36.864 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:55:36.864 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:55:36.864 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:55:36.865 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:55:36.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient: get_monmap_and_config 2026-02-17T05:55:36.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient: build_initial_monmap 2026-02-17T05:55:36.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient: monmap: 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:36.934371+0000 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:36.934371+0000 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:36.935 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 1 Processor -- start 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 1 -- start start 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient: init 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2af406c890 0x7f2af406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient(hunting): picked mon.noname-a con 0x7f2af406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2af409c4a0 0x7f2af406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient(hunting): picked mon.noname-c con 0x7f2af409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2af4134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af406daf0 con 0x7f2af4134df0 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f2af40a0880 con 0x7f2af406c890 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2af406d6f0 con 0x7f2af409c4a0 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2afb71b640 10 monclient(hunting): authenticate will time out at 1182.487896s 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2af9490640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2af406c890 0x7f2af406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2af9490640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2af406c890 0x7f2af406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51326/0 (socket says 10.20.193.78:51326) 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2af9490640 1 -- 10.20.193.78:0/3239132246 learned_addr learned my addr 10.20.193.78:0/3239132246 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2af9490640 10 monclient(hunting): get_auth_request con 0x7f2af406c890 auth_method 0 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2af9490640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2af9490640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.933+0000 7f2af9490640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 10 monclient(hunting): handle_auth_done global_id 124152 payload 274 2026-02-17T05:55:36.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 1 -- 10.20.193.78:0/3239132246 >> v2:10.20.193.115:3300/0 conn(0x7f2af409c4a0 msgr2=0x7f2af406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:36.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 1 --2- 10.20.193.78:0/3239132246 >> v2:10.20.193.115:3300/0 conn(0x7f2af409c4a0 0x7f2af406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 1 -- 10.20.193.78:0/3239132246 >> v1:10.20.193.115:6789/0 conn(0x7f2af4134df0 legacy=0x7f2af4135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:36.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 10 monclient: _finish_hunting 0 2026-02-17T05:55:36.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 1 monclient: found mon.noname-a 2026-02-17T05:55:36.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:36.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:36.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.934+0000 7f2af9490640 1 -- 10.20.193.78:0/3239132246 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af409cc80 con 0x7f2af406c890 2026-02-17T05:55:36.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2af9490640 1 -- 10.20.193.78:0/3239132246 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406c890 msgr2=0x7f2af406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:36.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2af9490640 1 --2- 10.20.193.78:0/3239132246 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406c890 0x7f2af406cc70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2af409c980 tx=0x7f2ae402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb1bc2392aaa5cc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:36.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 1 -- 10.20.193.78:0/3239132246 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae403c070 con 0x7f2af406c890 2026-02-17T05:55:36.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:36.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: dump: 2026-02-17T05:55:36.946 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.946 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: _finish_auth 0 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2afb71b640 5 monclient: authenticate success, global_id 124152 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 1 -- 10.20.193.78:0/3239132246 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae402fd80 con 0x7f2af406c890 2026-02-17T05:55:36.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2afb71b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2afb71b640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 1 -- 10.20.193.78:0/3239132246 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4037660 con 0x7f2af406c890 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2afb71b640 4 set_mon_vals no callback set 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.936+0000 7f2ae3fff640 10 monclient: dump: 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.948 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient: shutdown 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- 10.20.193.78:0/3239132246 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406c890 msgr2=0x7f2af406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 --2- 10.20.193.78:0/3239132246 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406c890 0x7f2af406cc70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2af409c980 tx=0x7f2ae402ee60 comp rx=0 tx=0).stop 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- 10.20.193.78:0/3239132246 shutdown_connections 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 --2- 10.20.193.78:0/3239132246 >> v2:10.20.193.115:3300/0 conn(0x7f2af409c4a0 0x7f2af406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 --2- 10.20.193.78:0/3239132246 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406c890 0x7f2af406cc70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- 10.20.193.78:0/3239132246 >> 10.20.193.78:0/3239132246 conn(0x7f2af40a6620 msgr2=0x7f2af40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- 10.20.193.78:0/3239132246 shutdown_connections 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- 10.20.193.78:0/3239132246 wait complete. 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient: build_initial_monmap 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient: monmap: 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 Processor -- start 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- start start 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient: init 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2af406c890 0x7f2af41cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient(hunting): picked mon.noname-b con 0x7f2af406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af409c4a0 0x7f2af4133360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient(hunting): picked mon.noname-a con 0x7f2af409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2af4134df0 0x7f2af41338a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient(hunting): picked mon.noname-c con 0x7f2af4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2af40a01a0 con 0x7f2af409c4a0 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2af40a0020 con 0x7f2af406c890 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2af409be20 con 0x7f2af4134df0 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2afb71b640 10 monclient(hunting): authenticate will time out at 1182.491921s 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.937+0000 7f2af8c8f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af409c4a0 0x7f2af4133360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af409c4a0 0x7f2af4133360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51338/0 (socket says 10.20.193.78:51338) 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 1 -- 10.20.193.78:0/700743354 learned_addr learned my addr 10.20.193.78:0/700743354 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient(hunting): get_auth_request con 0x7f2af409c4a0 auth_method 0 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9c91640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2af4134df0 0x7f2af41338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9c91640 10 monclient(hunting): get_auth_request con 0x7f2af4134df0 auth_method 0 2026-02-17T05:55:36.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9c91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9c91640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9c91640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9490640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2af406c890 0x7f2af41cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9490640 10 monclient(hunting): get_auth_request con 0x7f2af406c890 auth_method 0 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9490640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9490640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9490640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9c91640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9c91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9c91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient(hunting): handle_auth_done global_id 124155 payload 824 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2af4134df0 msgr2=0x7f2af41338a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2af4134df0 0x7f2af41338a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2af406c890 msgr2=0x7f2af41cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2af406c890 0x7f2af41cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient: _finish_hunting 0 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 1 monclient: found mon.noname-a 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af8c8f640 1 -- 10.20.193.78:0/700743354 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af4072750 con 0x7f2af409c4a0 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.938+0000 7f2af9490640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2af406c890 0x7f2af41cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2af8c8f640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af409c4a0 msgr2=0x7f2af4133360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2af8c8f640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af409c4a0 0x7f2af4133360 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800b7b0 tx=0x7f2ae800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80026e0 con 0x7f2af409c4a0 2026-02-17T05:55:36.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: dump: 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:55:36.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae8010070 con 0x7f2af409c4a0 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 5 monclient: authenticate success, global_id 124155 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 10 monclient: _renew_subs 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af4126d80 con 0x7f2af409c4a0 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8002ac0 con 0x7f2af409c4a0 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae1ffb640 10 monclient: dump: 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.956 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2ae37fe640 4 set_mon_vals no callback set 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 10 monclient: _renew_subs 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af409d810 con 0x7f2af409c4a0 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af4126880 0x7f2af4123600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.940+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2af4135a20 con 0x7f2af4126880 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af4126880 0x7f2af4123600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 10 monclient: get_auth_request con 0x7f2af4126880 auth_method 0 2026-02-17T05:55:36.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2ae801d030 con 0x7f2af409c4a0 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 10 monclient(hunting): handle_auth_done global_id 124158 payload 824 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9490640 1 -- 10.20.193.78:0/700743354 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2af4125370 con 0x7f2af4126880 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2ae1ffb640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ac403df00 0x7f2ac40403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(339..339 src has 117..339) ==== 4847+0+0 (secure 0 0 0) 0x7f2ae8051d50 con 0x7f2af409c4a0 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9c91640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ac403df00 0x7f2ac40403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.941+0000 7f2af9c91640 10 monclient: get_auth_request con 0x7f2ac403df00 auth_method 0 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2af9490640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af4126880 msgr2=0x7f2af4123600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2af9490640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af4126880 0x7f2af4123600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2af4074de0 tx=0x7f2ae40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4047070 con 0x7f2af4126880 2026-02-17T05:55:36.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 10 monclient: dump: 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2ae4043460 con 0x7f2af4126880 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:36.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af4126880 msgr2=0x7f2af4123600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af4126880 0x7f2af4123600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2af4074de0 tx=0x7f2ae40047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af4126880 msgr2=0x7f2af4123600 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.943+0000 7f2ae1ffb640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af4126880 0x7f2af4123600 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2af4074de0 tx=0x7f2ae40047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.944+0000 7f2af9c91640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ac403df00 0x7f2ac40403c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2af40a1590 tx=0x7f2af0004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2afb71b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2afb71b640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406f410 0x7f2af41da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2afb71b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2af40744d0 con 0x7f2af406f410 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406f410 0x7f2af41da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 10 monclient: get_auth_request con 0x7f2af406f410 auth_method 0 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:36.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:36.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:36.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:36.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 10 monclient(hunting): handle_auth_done global_id 124161 payload 824 2026-02-17T05:55:36.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.946+0000 7f2af9490640 1 -- 10.20.193.78:0/700743354 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2af4123b40 con 0x7f2af406f410 2026-02-17T05:55:36.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2af9490640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406f410 msgr2=0x7f2af41da4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:36.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2af9490640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406f410 0x7f2af41da4a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2af41266f0 tx=0x7f2ae4009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:36.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae403c040 con 0x7f2af406f410 2026-02-17T05:55:36.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:36.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:36.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 10 monclient: dump: 2026-02-17T05:55:36.962 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f2ae4041430 con 0x7f2af406f410 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406f410 msgr2=0x7f2af41da4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406f410 0x7f2af41da4a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2af41266f0 tx=0x7f2ae4009340 comp rx=0 tx=0).stop 2026-02-17T05:55:36.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406f410 msgr2=0x7f2af41da4a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:36.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2ae1ffb640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af406f410 0x7f2af41da4a0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2af41266f0 tx=0x7f2ae4009340 comp rx=0 tx=0).stop 2026-02-17T05:55:36.964 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:55:36.964 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:36.964 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:36.964 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:55:36.964 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 106, 2026-02-17T05:55:36.964 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 28416, 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:36.965 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:36.966 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:36.967 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:36.968 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.969 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:36.970 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:36.971 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:55:36.972 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.973 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:55:36.974 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:36.975 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:36.975 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:36.975 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:55:36.975 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ac403df00 msgr2=0x7f2ac40403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ac403df00 0x7f2ac40403c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2af40a1590 tx=0x7f2af0004200 comp rx=0 tx=0).stop 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 10 monclient: shutdown 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af409c4a0 msgr2=0x7f2af4133360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2af9c91640 1 -- 10.20.193.78:0/700743354 reap_dead start 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af409c4a0 0x7f2af4133360 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800b7b0 tx=0x7f2ae800bc80 comp rx=0 tx=0).stop 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 shutdown_connections 2026-02-17T05:55:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 --2- 10.20.193.78:0/700743354 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2af409c4a0 0x7f2af4133360 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:36.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 >> 10.20.193.78:0/700743354 conn(0x7f2af40a6620 msgr2=0x7f2af4072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:36.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 shutdown_connections 2026-02-17T05:55:36.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:36.949+0000 7f2afb71b640 1 -- 10.20.193.78:0/700743354 wait complete. 2026-02-17T05:55:36.977 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:55:36.977 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:55:36.977 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:55:37.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.046+0000 7fd5eb0bf640 10 monclient: get_monmap_and_config 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.046+0000 7fd5eb0bf640 10 monclient: build_initial_monmap 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.046+0000 7fd5eb0bf640 10 monclient: monmap: 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:37.047749+0000 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:37.047749+0000 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.046 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:37.047 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:37.047 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:37.047 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:37.047 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:37.047 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.046+0000 7fd5eb0bf640 1 Processor -- start 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.046+0000 7fd5eb0bf640 1 -- start start 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.046+0000 7fd5eb0bf640 10 monclient: init 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd5e406c890 0x7fd5e406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient(hunting): picked mon.noname-c con 0x7fd5e406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient(hunting): picked mon.noname-b con 0x7fd5e409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd5e4134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e406daf0 con 0x7fd5e4134df0 2026-02-17T05:55:37.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e40a0880 con 0x7fd5e406c890 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e406d6f0 con 0x7fd5e409c4a0 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5eb0bf640 10 monclient(hunting): authenticate will time out at 1182.601321s 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5e9635640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd5e4134df0 0x7fd5e4135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50732/0 (socket says 10.20.193.78:50732) 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5e9635640 1 -- 10.20.193.78:0/2136816107 learned_addr learned my addr 10.20.193.78:0/2136816107 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient(hunting): get_auth_request con 0x7fd5e409c4a0 auth_method 0 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5e8e34640 1 --2- 10.20.193.78:0/2136816107 >> v2:10.20.193.115:3300/0 conn(0x7fd5e406c890 0x7fd5e406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5e8e34640 10 monclient(hunting): get_auth_request con 0x7fd5e406c890 auth_method 0 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5e8e34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5e8e34640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5e8e34640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient(hunting): handle_auth_done global_id 74597 payload 274 2026-02-17T05:55:37.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 1 -- 10.20.193.78:0/2136816107 >> v2:10.20.193.115:3300/0 conn(0x7fd5e406c890 msgr2=0x7fd5e406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 1 --2- 10.20.193.78:0/2136816107 >> v2:10.20.193.115:3300/0 conn(0x7fd5e406c890 0x7fd5e406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 1 -- 10.20.193.78:0/2136816107 >> v1:10.20.193.78:6789/0 conn(0x7fd5e4134df0 legacy=0x7fd5e4135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 1 monclient: found mon.noname-b 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.047+0000 7fd5dbfff640 1 -- 10.20.193.78:0/2136816107 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e409cc80 con 0x7fd5e409c4a0 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5dbfff640 1 -- 10.20.193.78:0/2136816107 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 msgr2=0x7fd5e406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5dbfff640 1 --2- 10.20.193.78:0/2136816107 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e406d1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd5e406d870 tx=0x7fd5cc02ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d087cb91c480e17 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 1 -- 10.20.193.78:0/2136816107 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc03c070 con 0x7fd5e409c4a0 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: dump: 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:37.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 1 -- 10.20.193.78:0/2136816107 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc02fbf0 con 0x7fd5e409c4a0 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 5 monclient: authenticate success, global_id 74597 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 1 -- 10.20.193.78:0/2136816107 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc037440 con 0x7fd5e409c4a0 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 4 set_mon_vals no callback set 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5db7fe640 10 monclient: dump: 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.061 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 10 monclient: shutdown 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/2136816107 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 msgr2=0x7fd5e406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 1 --2- 10.20.193.78:0/2136816107 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e406d1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd5e406d870 tx=0x7fd5cc02ed70 comp rx=0 tx=0).stop 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/2136816107 shutdown_connections 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 1 --2- 10.20.193.78:0/2136816107 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e406d1b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 1 --2- 10.20.193.78:0/2136816107 >> v2:10.20.193.115:3300/0 conn(0x7fd5e406c890 0x7fd5e406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/2136816107 >> 10.20.193.78:0/2136816107 conn(0x7fd5e40a6620 msgr2=0x7fd5e40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/2136816107 shutdown_connections 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/2136816107 wait complete. 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 10 monclient: build_initial_monmap 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.050+0000 7fd5eb0bf640 10 monclient: monmap: 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.063 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 1 Processor -- start 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 1 -- start start 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient: init 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e406c890 0x7fd5e41cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient(hunting): picked mon.noname-b con 0x7fd5e406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e4121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient(hunting): picked mon.noname-c con 0x7fd5e409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4134df0 0x7fd5e4121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient(hunting): picked mon.noname-a con 0x7fd5e4134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40a01a0 con 0x7fd5e4134df0 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40a0020 con 0x7fd5e406c890 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e409be20 con 0x7fd5e409c4a0 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5eb0bf640 10 monclient(hunting): authenticate will time out at 1182.605472s 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e4121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e4121760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38404/0 (socket says 10.20.193.78:38404) 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 -- 10.20.193.78:0/3580267054 learned_addr learned my addr 10.20.193.78:0/3580267054 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient(hunting): get_auth_request con 0x7fd5e409c4a0 auth_method 0 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e9635640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4134df0 0x7fd5e4121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e9635640 10 monclient(hunting): get_auth_request con 0x7fd5e4134df0 auth_method 0 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e9635640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e9635640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e9635640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e9635640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e9635640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e8e34640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e406c890 0x7fd5e41cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e9635640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient(hunting): handle_auth_done global_id 74600 payload 824 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e406c890 msgr2=0x7fd5e41cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e406c890 0x7fd5e41cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4134df0 msgr2=0x7fd5e4121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4134df0 0x7fd5e4121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient: _finish_hunting 0 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 monclient: found mon.noname-c 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:37.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5dbfff640 1 -- 10.20.193.78:0/3580267054 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e41260d0 con 0x7fd5e409c4a0 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e8e34640 10 monclient: get_auth_request con 0x7fd5e406c890 auth_method 0 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.051+0000 7fd5e8e34640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e406c890 0x7fd5e41cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5dbfff640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 msgr2=0x7fd5e4121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5dbfff640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e4121760 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0099a0 tx=0x7fd5cc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc03c040 con 0x7fd5e409c4a0 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: dump: 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.068 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: _finish_auth 0 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: _check_auth_tickets 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc047070 con 0x7fd5e409c4a0 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc040a60 con 0x7fd5e409c4a0 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5daffd640 4 set_mon_vals no callback set 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5d97fa640 10 monclient: dump: 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5eb0bf640 5 monclient: authenticate success, global_id 74600 2026-02-17T05:55:37.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5eb0bf640 10 monclient: _renew_subs 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5eb0bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.053+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e4125410 con 0x7fd5e409c4a0 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5eb0bf640 10 monclient: _renew_subs 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5eb0bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e4072760 con 0x7fd5e409c4a0 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5eb0bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5eb0bf640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e41cdbc0 0x7fd5e41221e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5eb0bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e4123f60 con 0x7fd5e41cdbc0 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd5cc03f030 con 0x7fd5e409c4a0 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e41cdbc0 0x7fd5e41221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 10 monclient: get_auth_request con 0x7fd5e41cdbc0 auth_method 0 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5d97fa640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b403deb0 0x7fd5b4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(339..339 src has 117..339) ==== 4847+0+0 (secure 0 0 0) 0x7fd5cc0775a0 con 0x7fd5e409c4a0 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 10 monclient(hunting): handle_auth_done global_id 74603 payload 824 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e8e34640 1 -- 10.20.193.78:0/3580267054 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5e4074980 con 0x7fd5e41cdbc0 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e9635640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b403deb0 0x7fd5b4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.054+0000 7fd5e9635640 10 monclient: get_auth_request con 0x7fd5b403deb0 auth_method 0 2026-02-17T05:55:37.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.056+0000 7fd5e8e34640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e41cdbc0 msgr2=0x7fd5e41221e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.056+0000 7fd5e8e34640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e41cdbc0 0x7fd5e41221e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400f580 tx=0x7fd5d400fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4018020 con 0x7fd5e41cdbc0 2026-02-17T05:55:37.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 10 monclient: dump: 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd5d4003710 con 0x7fd5e41cdbc0 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e41cdbc0 msgr2=0x7fd5e41221e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e41cdbc0 0x7fd5e41221e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400f580 tx=0x7fd5d400fa50 comp rx=0 tx=0).stop 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e41cdbc0 msgr2=0x7fd5e41221e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5d97fa640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e41cdbc0 0x7fd5e41221e0 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400f580 tx=0x7fd5d400fa50 comp rx=0 tx=0).stop 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.057+0000 7fd5e9635640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b403deb0 0x7fd5b4040370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00055f0 tx=0x7fd5e0005580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5eb0bf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5eb0bf640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e4122720 0x7fd5e41d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5eb0bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e4122b30 con 0x7fd5e4122720 2026-02-17T05:55:37.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e4122720 0x7fd5e41d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 10 monclient: get_auth_request con 0x7fd5e4122720 auth_method 0 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 10 monclient(hunting): handle_auth_done global_id 74606 payload 824 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.060+0000 7fd5e8e34640 1 -- 10.20.193.78:0/3580267054 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd5e40a17b0 con 0x7fd5e4122720 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5e8e34640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e4122720 msgr2=0x7fd5e41d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5e8e34640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e4122720 0x7fd5e41d8330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400f580 tx=0x7fd5d40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4018070 con 0x7fd5e4122720 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:37.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 10 monclient: dump: 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7fd5d4014480 con 0x7fd5e4122720 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e4122720 msgr2=0x7fd5e41d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e4122720 0x7fd5e41d8330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400f580 tx=0x7fd5d40047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e4122720 msgr2=0x7fd5e41d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.062+0000 7fd5d97fa640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e4122720 0x7fd5e41d8330 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400f580 tx=0x7fd5d40047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 106, 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 308955, 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:55:37.080 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:37.081 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.082 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:55:37.083 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.084 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:55:37.085 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:55:37.086 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.087 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:37.088 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:55:37.089 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:55:37.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b403deb0 msgr2=0x7fd5b4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b403deb0 0x7fd5b4040370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00055f0 tx=0x7fd5e0005580 comp rx=0 tx=0).stop 2026-02-17T05:55:37.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 10 monclient: shutdown 2026-02-17T05:55:37.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:37.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 msgr2=0x7fd5e4121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 1 --2- 10.20.193.78:0/3580267054 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409c4a0 0x7fd5e4121760 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0099a0 tx=0x7fd5cc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5e9635640 1 -- 10.20.193.78:0/3580267054 reap_dead start 2026-02-17T05:55:37.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5e9635640 1 -- 10.20.193.78:0/3580267054 reap_dead start 2026-02-17T05:55:37.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 shutdown_connections 2026-02-17T05:55:37.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 >> 10.20.193.78:0/3580267054 conn(0x7fd5e40a6620 msgr2=0x7fd5e4135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:37.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 shutdown_connections 2026-02-17T05:55:37.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.063+0000 7fd5eb0bf640 1 -- 10.20.193.78:0/3580267054 wait complete. 2026-02-17T05:55:37.091 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:55:37.091 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:55:37.091 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:55:37.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient: get_monmap_and_config 2026-02-17T05:55:37.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient: build_initial_monmap 2026-02-17T05:55:37.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient: monmap: 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:37.162283+0000 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:37.162283+0000 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:37.163 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 1 Processor -- start 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 1 -- start start 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient: init 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8c0c09ea90 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8c0c09ee70 0x7f8c0c09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient(hunting): picked mon.noname-c con 0x7f8c0c09ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8c0c13d640 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0c0a0a20 con 0x7f8c0c13d640 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0c0a0460 con 0x7f8c0c09ea90 2026-02-17T05:55:37.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c0a0d00 con 0x7f8c0c09ee70 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c13c2b640 10 monclient(hunting): authenticate will time out at 1182.715828s 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c121a1640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f8c0c13d640 0x7f8c0c13fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50738/0 (socket says 10.20.193.78:50738) 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.161+0000 7f8c121a1640 1 -- 10.20.193.78:0/1979414312 learned_addr learned my addr 10.20.193.78:0/1979414312 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 1 --2- 10.20.193.78:0/1979414312 >> v2:10.20.193.115:3300/0 conn(0x7f8c0c09ee70 0x7f8c0c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient(hunting): get_auth_request con 0x7f8c0c09ee70 auth_method 0 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient(hunting): handle_auth_done global_id 114202 payload 274 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 1 -- 10.20.193.78:0/1979414312 >> v1:10.20.193.115:6789/0 conn(0x7f8c0c09ea90 legacy=0x7f8c0c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 1 -- 10.20.193.78:0/1979414312 >> v1:10.20.193.78:6789/0 conn(0x7f8c0c13d640 legacy=0x7f8c0c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient: _finish_hunting 0 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 1 monclient: found mon.noname-c 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:55:37.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.162+0000 7f8c1119f640 1 -- 10.20.193.78:0/1979414312 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c0a0740 con 0x7f8c0c09ee70 2026-02-17T05:55:37.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.164+0000 7f8c1119f640 1 -- 10.20.193.78:0/1979414312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 msgr2=0x7f8c0c09f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.164+0000 7f8c1119f640 1 --2- 10.20.193.78:0/1979414312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 0x7f8c0c09f2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c000098a0 tx=0x7f8c0002ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8c3256524ca7be server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 1 -- 10.20.193.78:0/1979414312 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0003c070 con 0x7f8c0c09ee70 2026-02-17T05:55:37.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:37.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: dump: 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: _finish_auth 0 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: _check_auth_tickets 2026-02-17T05:55:37.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 5 monclient: authenticate success, global_id 114202 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 1 -- 10.20.193.78:0/1979414312 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0002fce0 con 0x7f8c0c09ee70 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 1 -- 10.20.193.78:0/1979414312 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00037660 con 0x7f8c0c09ee70 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 4 set_mon_vals no callback set 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c1099e640 10 monclient: dump: 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.176 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 10 monclient: shutdown 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 1 -- 10.20.193.78:0/1979414312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 msgr2=0x7f8c0c09f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 1 --2- 10.20.193.78:0/1979414312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 0x7f8c0c09f2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c000098a0 tx=0x7f8c0002ed80 comp rx=0 tx=0).stop 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 1 -- 10.20.193.78:0/1979414312 shutdown_connections 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 1 --2- 10.20.193.78:0/1979414312 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 0x7f8c0c09f2c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 1 -- 10.20.193.78:0/1979414312 >> 10.20.193.78:0/1979414312 conn(0x7f8c0c0a7520 msgr2=0x7f8c0c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 1 -- 10.20.193.78:0/1979414312 shutdown_connections 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 1 -- 10.20.193.78:0/1979414312 wait complete. 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 10 monclient: build_initial_monmap 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.165+0000 7f8c13c2b640 10 monclient: monmap: 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.178 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 1 Processor -- start 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 1 -- start start 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient: init 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8c0c09ea90 0x7f8c0c1cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient(hunting): picked mon.noname-c con 0x7f8c0c09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:37.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 0x7f8c0c073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient(hunting): picked mon.noname-b con 0x7f8c0c09ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8c0c13d640 0x7f8c0c073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient(hunting): picked mon.noname-a con 0x7f8c0c13d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c0a6820 con 0x7f8c0c13d640 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c0a66a0 con 0x7f8c0c09ee70 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c141150 con 0x7f8c0c09ea90 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c13c2b640 10 monclient(hunting): authenticate will time out at 1182.720533s 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8c0c09ea90 0x7f8c0c1cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c121a1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8c0c13d640 0x7f8c0c073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8c0c09ea90 0x7f8c0c1cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38442/0 (socket says 10.20.193.78:38442) 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 -- 10.20.193.78:0/2250725284 learned_addr learned my addr 10.20.193.78:0/2250725284 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient(hunting): get_auth_request con 0x7f8c0c09ea90 auth_method 0 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c121a1640 10 monclient(hunting): get_auth_request con 0x7f8c0c13d640 auth_method 0 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c121a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c121a1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c121a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c1119f640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 0x7f8c0c073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c121a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c121a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c121a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c1119f640 10 monclient(hunting): get_auth_request con 0x7f8c0c09ee70 auth_method 0 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c1119f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c1119f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c1119f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient(hunting): handle_auth_done global_id 74609 payload 824 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 msgr2=0x7f8c0c073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c09ee70 0x7f8c0c073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8c0c13d640 msgr2=0x7f8c0c073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8c0c13d640 0x7f8c0c073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient: _finish_hunting 0 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 monclient: found mon.noname-c 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.166+0000 7f8c119a0640 1 -- 10.20.193.78:0/2250725284 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c06f940 con 0x7f8c0c09ea90 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.168+0000 7f8c119a0640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8c0c09ea90 msgr2=0x7f8c0c1cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.168+0000 7f8c119a0640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8c0c09ea90 0x7f8c0c1cf4a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00c600 tx=0x7f8bfc00cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc012a40 con 0x7f8c0c09ea90 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: dump: 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.183 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 5 monclient: authenticate success, global_id 74609 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc017070 con 0x7f8c0c09ea90 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 10 monclient: _renew_subs 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c06fc20 con 0x7f8c0c09ea90 2026-02-17T05:55:37.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc012e20 con 0x7f8c0c09ea90 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfbfff640 4 set_mon_vals no callback set 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 10 monclient: dump: 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.185 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 10 monclient: _renew_subs 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c13ab50 con 0x7f8c0c09ea90 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13da20 0x7f8c0c13de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c09f2c0 con 0x7f8c0c13da20 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13da20 0x7f8c0c13de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8bfc01c460 con 0x7f8c0c09ea90 2026-02-17T05:55:37.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 10 monclient: get_auth_request con 0x7f8c0c13da20 auth_method 0 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8bfa7fc640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8bdc03deb0 0x7f8bdc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 10 monclient(hunting): handle_auth_done global_id 114208 payload 824 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.169+0000 7f8c1119f640 1 -- 10.20.193.78:0/2250725284 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c0c13bfc0 con 0x7f8c0c13da20 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.170+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(339..339 src has 117..339) ==== 4847+0+0 (secure 0 0 0) 0x7f8bfc052680 con 0x7f8c0c09ea90 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.170+0000 7f8c121a1640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8bdc03deb0 0x7f8bdc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.170+0000 7f8c121a1640 10 monclient: get_auth_request con 0x7f8bdc03deb0 auth_method 0 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8c1119f640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13da20 msgr2=0x7f8c0c13de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8c1119f640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13da20 0x7f8c0c13de00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c00039000 tx=0x7f8c000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0003c040 con 0x7f8c0c13da20 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 10 monclient: dump: 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f8c00041440 con 0x7f8c0c13da20 2026-02-17T05:55:37.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:37.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13da20 msgr2=0x7f8c0c13de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13da20 0x7f8c0c13de00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c00039000 tx=0x7f8c000047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13da20 msgr2=0x7f8c0c13de00 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:37.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8bfa7fc640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13da20 0x7f8c0c13de00 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c00039000 tx=0x7f8c000047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.172+0000 7f8c121a1640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8bdc03deb0 0x7f8bdc040370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c080056a0 tx=0x7f8c08005630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c13c2b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c13c2b640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13e340 0x7f8c0c1da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c13c2b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c13c7a0 con 0x7f8c0c13e340 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13e340 0x7f8c0c1da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 10 monclient: get_auth_request con 0x7f8c0c13e340 auth_method 0 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 10 monclient(hunting): handle_auth_done global_id 114211 payload 824 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.175+0000 7f8c1119f640 1 -- 10.20.193.78:0/2250725284 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8c0c06ffb0 con 0x7f8c0c13e340 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c1119f640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13e340 msgr2=0x7f8c0c1da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c1119f640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13e340 0x7f8c0c1da7f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c00002410 tx=0x7f8c00008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0003c040 con 0x7f8c0c13e340 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 10 monclient: dump: 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.192 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f8c00003bc0 con 0x7f8c0c13e340 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13e340 msgr2=0x7f8c0c1da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13e340 0x7f8c0c1da7f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c00002410 tx=0x7f8c00008030 comp rx=0 tx=0).stop 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13e340 msgr2=0x7f8c0c1da7f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:37.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8bfa7fc640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8c0c13e340 0x7f8c0c1da7f0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c00002410 tx=0x7f8c00008030 comp rx=0 tx=0).stop 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 106, 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 81259, 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:55:37.194 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:37.195 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:55:37.196 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:37.197 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:37.198 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:37.199 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.200 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:55:37.201 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:37.202 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:55:37.203 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8bdc03deb0 msgr2=0x7f8bdc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8bdc03deb0 0x7f8bdc040370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c080056a0 tx=0x7f8c08005630 comp rx=0 tx=0).stop 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 10 monclient: shutdown 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8c0c09ea90 msgr2=0x7f8c0c1cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 1 --2- 10.20.193.78:0/2250725284 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8c0c09ea90 0x7f8c0c1cf4a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00c600 tx=0x7f8bfc00cad0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c121a1640 1 -- 10.20.193.78:0/2250725284 reap_dead start 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c121a1640 1 -- 10.20.193.78:0/2250725284 reap_dead start 2026-02-17T05:55:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 shutdown_connections 2026-02-17T05:55:37.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 >> 10.20.193.78:0/2250725284 conn(0x7f8c0c0a7520 msgr2=0x7f8c0c0a3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:37.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 shutdown_connections 2026-02-17T05:55:37.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.178+0000 7f8c13c2b640 1 -- 10.20.193.78:0/2250725284 wait complete. 2026-02-17T05:55:37.205 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:55:37.205 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T05:55:37.205 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:55:37.205 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T05:55:37.205 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T05:55:37.205 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T05:55:37.205 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:55:37.216 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T05:55:37.216 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:55:37.216 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.283+0000 7fe0f6222640 10 monclient: get_monmap_and_config 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.283+0000 7fe0f6222640 10 monclient: build_initial_monmap 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.283+0000 7fe0f6222640 10 monclient: monmap: 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:37.284959+0000 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:37.284959+0000 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:37.283 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 1 Processor -- start 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 1 -- start start 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient: init 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe0f009ddd0 0x7fe0f009e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient(hunting): picked mon.noname-a con 0x7fe0f009ddd0 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f0137f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient(hunting): picked mon.noname-b con 0x7fe0f0073f60 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:37.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe0f0138490 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f00a2850 con 0x7fe0f0138490 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f00a2560 con 0x7fe0f009ddd0 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f00a3810 con 0x7fe0f0073f60 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0f6222640 10 monclient(hunting): authenticate will time out at 1182.838668s 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0effff640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fe0f0138490 legacy=0x7fe0f013f280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f0137f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0ef7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe0f009ddd0 0x7fe0f009e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f0137f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40980/0 (socket says 10.20.193.78:40980) 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 1 -- 10.20.193.78:0/1854425243 learned_addr learned my addr 10.20.193.78:0/1854425243 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient(hunting): get_auth_request con 0x7fe0f0073f60 auth_method 0 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0ef7fe640 10 monclient(hunting): get_auth_request con 0x7fe0f009ddd0 auth_method 0 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient(hunting): handle_auth_done global_id 74612 payload 274 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 1 -- 10.20.193.78:0/1854425243 >> v2:10.20.193.78:3300/0 conn(0x7fe0f009ddd0 msgr2=0x7fe0f009e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 1 --2- 10.20.193.78:0/1854425243 >> v2:10.20.193.78:3300/0 conn(0x7fe0f009ddd0 0x7fe0f009e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 1 -- 10.20.193.78:0/1854425243 >> v1:10.20.193.115:6789/0 conn(0x7fe0f0138490 legacy=0x7fe0f013f280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient: _finish_hunting 0 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 1 monclient: found mon.noname-b 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.284+0000 7fe0eeffd640 1 -- 10.20.193.78:0/1854425243 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f00a3af0 con 0x7fe0f0073f60 2026-02-17T05:55:37.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0eeffd640 1 -- 10.20.193.78:0/1854425243 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 msgr2=0x7fe0f0137f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0eeffd640 1 --2- 10.20.193.78:0/1854425243 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f0137f50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009970 tx=0x7fe0e002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e1765d28bf8e04 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 1 -- 10.20.193.78:0/1854425243 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e003c070 con 0x7fe0f0073f60 2026-02-17T05:55:37.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: dump: 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 1 -- 10.20.193.78:0/1854425243 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e002fcd0 con 0x7fe0f0073f60 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 1 -- 10.20.193.78:0/1854425243 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0037660 con 0x7fe0f0073f60 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 5 monclient: authenticate success, global_id 74612 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 4 set_mon_vals no callback set 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0ee7fc640 10 monclient: dump: 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 10 monclient: shutdown 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 1 -- 10.20.193.78:0/1854425243 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 msgr2=0x7fe0f0137f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 1 --2- 10.20.193.78:0/1854425243 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f0137f50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009970 tx=0x7fe0e002ee90 comp rx=0 tx=0).stop 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 1 -- 10.20.193.78:0/1854425243 shutdown_connections 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 1 --2- 10.20.193.78:0/1854425243 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f0137f50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 1 --2- 10.20.193.78:0/1854425243 >> v2:10.20.193.78:3300/0 conn(0x7fe0f009ddd0 0x7fe0f009e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 1 -- 10.20.193.78:0/1854425243 >> 10.20.193.78:0/1854425243 conn(0x7fe0f00aacd0 msgr2=0x7fe0f00ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 1 -- 10.20.193.78:0/1854425243 shutdown_connections 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.287+0000 7fe0f6222640 1 -- 10.20.193.78:0/1854425243 wait complete. 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient: build_initial_monmap 2026-02-17T05:55:37.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient: monmap: 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 1 Processor -- start 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 1 -- start start 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient: init 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f01cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient(hunting): picked mon.noname-c con 0x7fe0f0073f60 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe0f009ddd0 0x7fe0f01cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient(hunting): picked mon.noname-b con 0x7fe0f009ddd0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0f0138490 0x7fe0f006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient(hunting): picked mon.noname-a con 0x7fe0f0138490 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f009e6f0 con 0x7fe0f0138490 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f00a57e0 con 0x7fe0f009ddd0 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f00a4840 con 0x7fe0f0073f60 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0f6222640 10 monclient(hunting): authenticate will time out at 1182.842894s 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0ef7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f01cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0ef7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f01cf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40996/0 (socket says 10.20.193.78:40996) 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0ef7fe640 1 -- 10.20.193.78:0/3229594593 learned_addr learned my addr 10.20.193.78:0/3229594593 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0ef7fe640 10 monclient(hunting): get_auth_request con 0x7fe0f0073f60 auth_method 0 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0eeffd640 1 -- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe0f009ddd0 msgr2=0x7fe0f01cf8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0effff640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0f0138490 0x7fe0f006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0eeffd640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe0f009ddd0 0x7fe0f01cf8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0effff640 10 monclient(hunting): get_auth_request con 0x7fe0f0138490 auth_method 0 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.288+0000 7fe0ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:37.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 10 monclient(hunting): handle_auth_done global_id 74615 payload 824 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 1 -- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe0f009ddd0 msgr2=0x7fe0f01cf8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe0f009ddd0 0x7fe0f01cf8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 1 -- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0f0138490 msgr2=0x7fe0f006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0f0138490 0x7fe0f006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 1 monclient: found mon.noname-c 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.289+0000 7fe0ef7fe640 1 -- 10.20.193.78:0/3229594593 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f0070720 con 0x7fe0f0073f60 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0ef7fe640 1 -- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 msgr2=0x7fe0f01cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0ef7fe640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f01cf370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00ec10 tx=0x7fe0dc00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 1 -- 10.20.193.78:0/3229594593 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc017070 con 0x7fe0f0073f60 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: dump: 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.295 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: _finish_auth 0 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 1 -- 10.20.193.78:0/3229594593 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0dc00ce50 con 0x7fe0f0073f60 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 1 -- 10.20.193.78:0/3229594593 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0136b0 con 0x7fe0f0073f60 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 10 monclient: dump: 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 5 monclient: authenticate success, global_id 74615 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 10 monclient: _renew_subs 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0edffb640 4 set_mon_vals no callback set 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f00a99a0 con 0x7fe0f0073f60 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 10 monclient: _renew_subs 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f00724a0 con 0x7fe0f0073f60 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0cbfff640 1 -- 10.20.193.78:0/3229594593 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe0dc024060 con 0x7fe0f0073f60 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.291+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f0137f50 con 0x7fe0f0073f60 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.292+0000 7fe0cbfff640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe0c003de60 0x7fe0c0040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.292+0000 7fe0cbfff640 1 -- 10.20.193.78:0/3229594593 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(339..339 src has 117..339) ==== 4847+0+0 (secure 0 0 0) 0x7fe0dc052970 con 0x7fe0f0073f60 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.292+0000 7fe0eeffd640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe0c003de60 0x7fe0c0040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.292+0000 7fe0eeffd640 10 monclient: get_auth_request con 0x7fe0c003de60 auth_method 0 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.294+0000 7fe0cbfff640 1 -- 10.20.193.78:0/3229594593 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe0dc051330 con 0x7fe0f0073f60 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.294+0000 7fe0cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.294+0000 7fe0cbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:37.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.294+0000 7fe0eeffd640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe0c003de60 0x7fe0c0040320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0036000 tx=0x7fe0e0039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:37.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.407+0000 7fe0f6222640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:37.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.407+0000 7fe0f6222640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:37.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.407+0000 7fe0f6222640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:37.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.407+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe0f00744d0 con 0x7fe0f0073f60 2026-02-17T05:55:37.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.407+0000 7fe0cbfff640 1 -- 10.20.193.78:0/3229594593 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fe0dc013850 con 0x7fe0f0073f60 2026-02-17T05:55:37.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.407+0000 7fe0cbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:37.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.407+0000 7fe0cbfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:37.407 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:37.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe0c003de60 msgr2=0x7fe0c0040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe0c003de60 0x7fe0c0040320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0036000 tx=0x7fe0e0039040 comp rx=0 tx=0).stop 2026-02-17T05:55:37.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 10 monclient: shutdown 2026-02-17T05:55:37.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 msgr2=0x7fe0f01cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f01cf370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00ec10 tx=0x7fe0dc00c520 comp rx=0 tx=0).stop 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 shutdown_connections 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0f0138490 0x7fe0f006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe0c003de60 0x7fe0c0040320 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe0f009ddd0 0x7fe0f01cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 --2- 10.20.193.78:0/3229594593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe0f0073f60 0x7fe0f01cf370 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 >> 10.20.193.78:0/3229594593 conn(0x7fe0f00aacd0 msgr2=0x7fe0f01368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 shutdown_connections 2026-02-17T05:55:37.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:37.408+0000 7fe0f6222640 1 -- 10.20.193.78:0/3229594593 wait complete. 2026-02-17T05:55:40.416 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:40.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.488+0000 7f9af56e8640 10 monclient: get_monmap_and_config 2026-02-17T05:55:40.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.488+0000 7f9af56e8640 10 monclient: build_initial_monmap 2026-02-17T05:55:40.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.488+0000 7f9af56e8640 10 monclient: monmap: 2026-02-17T05:55:40.487 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:40.489700+0000 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:40.489700+0000 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:40.488 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.488+0000 7f9af56e8640 1 Processor -- start 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.488+0000 7f9af56e8640 1 -- start start 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.488+0000 7f9af56e8640 10 monclient: init 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9af006c890 0x7f9af00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient(hunting): picked mon.noname-c con 0x7f9af006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006cc70 0x7f9af006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient(hunting): picked mon.noname-b con 0x7f9af006cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9af0134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af0074150 con 0x7f9af0134df0 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f9af009ef70 con 0x7f9af006c890 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9af009f250 con 0x7f9af006cc70 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9af56e8640 10 monclient(hunting): authenticate will time out at 1186.043297s 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aeeffd640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f9af006c890 msgr2=0x7f9af00a01f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aeeffd640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9af006c890 0x7f9af00a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aef7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f9af0134df0 0x7f9af0135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:47626/0 (socket says 10.20.193.78:47626) 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006cc70 0x7f9af006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aef7fe640 1 -- 10.20.193.78:0/3164611966 learned_addr learned my addr 10.20.193.78:0/3164611966 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient(hunting): get_auth_request con 0x7f9af006cc70 auth_method 0 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient(hunting): handle_auth_done global_id 74618 payload 274 2026-02-17T05:55:40.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 1 -- 10.20.193.78:0/3164611966 >> v2:10.20.193.115:3300/0 conn(0x7f9af006c890 msgr2=0x7f9af00a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 1 --2- 10.20.193.78:0/3164611966 >> v2:10.20.193.115:3300/0 conn(0x7f9af006c890 0x7f9af00a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 1 -- 10.20.193.78:0/3164611966 >> v1:10.20.193.78:6789/0 conn(0x7f9af0134df0 legacy=0x7f9af0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:55:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 1 monclient: found mon.noname-b 2026-02-17T05:55:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:55:40.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.489+0000 7f9aee7fc640 1 -- 10.20.193.78:0/3164611966 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af009df40 con 0x7f9af006cc70 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aee7fc640 1 -- 10.20.193.78:0/3164611966 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006cc70 msgr2=0x7f9af006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aee7fc640 1 --2- 10.20.193.78:0/3164611966 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006cc70 0x7f9af006d050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098a0 tx=0x7f9ad802eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16d559d646a51494 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 1 -- 10.20.193.78:0/3164611966 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad803c070 con 0x7f9af006cc70 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: dump: 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: _finish_auth 0 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: _check_auth_tickets 2026-02-17T05:55:40.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 1 -- 10.20.193.78:0/3164611966 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad802fe30 con 0x7f9af006cc70 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 5 monclient: authenticate success, global_id 74618 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 1 -- 10.20.193.78:0/3164611966 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8037640 con 0x7f9af006cc70 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 4 set_mon_vals no callback set 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9aedffb640 10 monclient: dump: 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:40.492 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:40.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 10 monclient: shutdown 2026-02-17T05:55:40.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:40.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 1 -- 10.20.193.78:0/3164611966 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006cc70 msgr2=0x7f9af006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:40.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 1 --2- 10.20.193.78:0/3164611966 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006cc70 0x7f9af006d050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098a0 tx=0x7f9ad802eed0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 1 -- 10.20.193.78:0/3164611966 shutdown_connections 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 1 --2- 10.20.193.78:0/3164611966 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006cc70 0x7f9af006d050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 1 --2- 10.20.193.78:0/3164611966 >> v2:10.20.193.115:3300/0 conn(0x7f9af006c890 0x7f9af00a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 1 -- 10.20.193.78:0/3164611966 >> 10.20.193.78:0/3164611966 conn(0x7f9af00a6620 msgr2=0x7f9af00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 1 -- 10.20.193.78:0/3164611966 shutdown_connections 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.492+0000 7f9af56e8640 1 -- 10.20.193.78:0/3164611966 wait complete. 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient: build_initial_monmap 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient: monmap: 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 1 Processor -- start 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 1 -- start start 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient: init 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006c890 0x7f9af01cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient(hunting): picked mon.noname-c con 0x7f9af006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9af006cc70 0x7f9af01338d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient(hunting): picked mon.noname-a con 0x7f9af006cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9af0134df0 0x7f9af0133e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient(hunting): picked mon.noname-b con 0x7f9af0134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9af009d360 con 0x7f9af006cc70 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9af009d1e0 con 0x7f9af0134df0 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9af0135950 con 0x7f9af006c890 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9af56e8640 10 monclient(hunting): authenticate will time out at 1186.047644s 2026-02-17T05:55:40.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006c890 0x7f9af01cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9af006cc70 0x7f9af01338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006c890 0x7f9af01cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41024/0 (socket says 10.20.193.78:41024) 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 -- 10.20.193.78:0/4133630222 learned_addr learned my addr 10.20.193.78:0/4133630222 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient(hunting): get_auth_request con 0x7f9af006c890 auth_method 0 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 10 monclient(hunting): get_auth_request con 0x7f9af006cc70 auth_method 0 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aef7fe640 1 -- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9af0134df0 msgr2=0x7f9af0133e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aef7fe640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9af0134df0 0x7f9af0133e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient(hunting): handle_auth_done global_id 74621 payload 824 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 -- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9af0134df0 msgr2=0x7f9af0133e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9af0134df0 0x7f9af0133e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 -- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9af006cc70 msgr2=0x7f9af01338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9af006cc70 0x7f9af01338d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient: _finish_hunting 0 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 monclient: found mon.noname-c 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aeeffd640 1 -- 10.20.193.78:0/4133630222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af0126db0 con 0x7f9af006c890 2026-02-17T05:55:40.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.493+0000 7f9aee7fc640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9af006cc70 0x7f9af01338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9aeeffd640 1 -- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006c890 msgr2=0x7f9af01cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9aeeffd640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006c890 0x7f9af01cf540 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400b780 tx=0x7f9ae400bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 1 -- 10.20.193.78:0/4133630222 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40052d0 con 0x7f9af006c890 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: dump: 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:40.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 5 monclient: authenticate success, global_id 74621 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 1 -- 10.20.193.78:0/4133630222 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae4010070 con 0x7f9af006c890 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 10 monclient: _renew_subs 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 1 -- 10.20.193.78:0/4133630222 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40056d0 con 0x7f9af006c890 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9aed7fa640 4 set_mon_vals no callback set 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af0126b70 con 0x7f9af006c890 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 10 monclient: dump: 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 10 monclient: _renew_subs 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af0128300 con 0x7f9af006c890 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9acf7fe640 1 -- 10.20.193.78:0/4133630222 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9ae401d390 con 0x7f9af006c890 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.496+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af0068410 con 0x7f9af006c890 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.497+0000 7f9acf7fe640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9abc03dee0 0x7f9abc0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.497+0000 7f9acf7fe640 1 -- 10.20.193.78:0/4133630222 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(339..339 src has 117..339) ==== 4847+0+0 (secure 0 0 0) 0x7f9ae4052180 con 0x7f9af006c890 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.497+0000 7f9aee7fc640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9abc03dee0 0x7f9abc0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:40.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.497+0000 7f9aee7fc640 10 monclient: get_auth_request con 0x7f9abc03dee0 auth_method 0 2026-02-17T05:55:40.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.499+0000 7f9acf7fe640 1 -- 10.20.193.78:0/4133630222 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9af0068410 con 0x7f9af006c890 2026-02-17T05:55:40.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.499+0000 7f9acf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:40.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.499+0000 7f9acf7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:40.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.499+0000 7f9aee7fc640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9abc03dee0 0x7f9abc0403a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9af006e270 tx=0x7f9ad8033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:40.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.613+0000 7f9af56e8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.613+0000 7f9af56e8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.613+0000 7f9af56e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.613+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9af006d8c0 con 0x7f9af006c890 2026-02-17T05:55:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.614+0000 7f9acf7fe640 1 -- 10.20.193.78:0/4133630222 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f9af006d8c0 con 0x7f9af006c890 2026-02-17T05:55:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.614+0000 7f9acf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.614+0000 7f9acf7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:40.615 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:40.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9abc03dee0 msgr2=0x7f9abc0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:40.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9abc03dee0 0x7f9abc0403a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9af006e270 tx=0x7f9ad8033000 comp rx=0 tx=0).stop 2026-02-17T05:55:40.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 10 monclient: shutdown 2026-02-17T05:55:40.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:40.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006c890 msgr2=0x7f9af01cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006c890 0x7f9af01cf540 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400b780 tx=0x7f9ae400bc50 comp rx=0 tx=0).stop 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 shutdown_connections 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9abc03dee0 0x7f9abc0403a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9af0134df0 0x7f9af0133e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9af006cc70 0x7f9af01338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 --2- 10.20.193.78:0/4133630222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9af006c890 0x7f9af01cf540 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 >> 10.20.193.78:0/4133630222 conn(0x7f9af00a6620 msgr2=0x7f9af009be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 shutdown_connections 2026-02-17T05:55:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:40.615+0000 7f9af56e8640 1 -- 10.20.193.78:0/4133630222 wait complete. 2026-02-17T05:55:43.623 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:43.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.691+0000 7fb066043640 10 monclient: get_monmap_and_config 2026-02-17T05:55:43.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.691+0000 7fb066043640 10 monclient: build_initial_monmap 2026-02-17T05:55:43.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.691+0000 7fb066043640 10 monclient: monmap: 2026-02-17T05:55:43.693 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:43.693 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:43.693100+0000 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:43.693100+0000 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 1 Processor -- start 2026-02-17T05:55:43.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 1 -- start start 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient: init 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06006c890 0x7fb06006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient(hunting): picked mon.noname-b con 0x7fb06006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb06009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb06006d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb060135c10 con 0x7fb06006d1b0 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb06006daf0 con 0x7fb06009c4a0 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb0600a0880 con 0x7fb06006c890 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb066043640 10 monclient(hunting): authenticate will time out at 1189.246635s 2026-02-17T05:55:43.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05effd640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fb06009c4a0 legacy=0x7fb060134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06006c890 0x7fb06006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05ffff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb06006d1b0 0x7fb060135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:47638/0 (socket says 10.20.193.78:47638) 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05ffff640 1 -- 10.20.193.78:0/1204228217 learned_addr learned my addr 10.20.193.78:0/1204228217 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient(hunting): get_auth_request con 0x7fb06006c890 auth_method 0 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient(hunting): handle_auth_done global_id 74624 payload 274 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 1 -- 10.20.193.78:0/1204228217 >> v1:10.20.193.115:6789/0 conn(0x7fb06009c4a0 legacy=0x7fb060134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 1 -- 10.20.193.78:0/1204228217 >> v1:10.20.193.78:6789/0 conn(0x7fb06006d1b0 legacy=0x7fb060135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:43.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:43.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 1 monclient: found mon.noname-b 2026-02-17T05:55:43.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:43.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:55:43.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.692+0000 7fb05f7fe640 1 -- 10.20.193.78:0/1204228217 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06009cc80 con 0x7fb06006c890 2026-02-17T05:55:43.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05f7fe640 1 -- 10.20.193.78:0/1204228217 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06006c890 msgr2=0x7fb06006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05f7fe640 1 --2- 10.20.193.78:0/1204228217 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06006c890 0x7fb06006cc70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0098d0 tx=0x7fb04c02ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=423c936424f01172 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 1 -- 10.20.193.78:0/1204228217 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c03c070 con 0x7fb06006c890 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: dump: 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:43.703 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 1 -- 10.20.193.78:0/1204228217 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04c02fcc0 con 0x7fb06006c890 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 5 monclient: authenticate success, global_id 74624 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 1 -- 10.20.193.78:0/1204228217 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c037660 con 0x7fb06006c890 2026-02-17T05:55:43.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb05e7fc640 10 monclient: dump: 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:43.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 4 set_mon_vals no callback set 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 10 monclient: shutdown 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 1 -- 10.20.193.78:0/1204228217 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06006c890 msgr2=0x7fb06006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 1 --2- 10.20.193.78:0/1204228217 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06006c890 0x7fb06006cc70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0098d0 tx=0x7fb04c02ed60 comp rx=0 tx=0).stop 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 1 -- 10.20.193.78:0/1204228217 shutdown_connections 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 1 --2- 10.20.193.78:0/1204228217 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06006c890 0x7fb06006cc70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 1 -- 10.20.193.78:0/1204228217 >> 10.20.193.78:0/1204228217 conn(0x7fb0600a6620 msgr2=0x7fb0600a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 1 -- 10.20.193.78:0/1204228217 shutdown_connections 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.695+0000 7fb066043640 1 -- 10.20.193.78:0/1204228217 wait complete. 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.696+0000 7fb066043640 10 monclient: build_initial_monmap 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.696+0000 7fb066043640 10 monclient: monmap: 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.696+0000 7fb066043640 1 Processor -- start 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.696+0000 7fb066043640 1 -- start start 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.696+0000 7fb066043640 10 monclient: init 2026-02-17T05:55:43.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.696+0000 7fb066043640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb06006c890 0x7fb0601cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient(hunting): picked mon.noname-b con 0x7fb06006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb06006d1b0 0x7fb060134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient(hunting): picked mon.noname-a con 0x7fb06006d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06009c4a0 0x7fb0601346b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient(hunting): picked mon.noname-c con 0x7fb06009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0600a0020 con 0x7fb06006d1b0 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb06006d600 con 0x7fb06006c890 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb0600a01a0 con 0x7fb06009c4a0 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb066043640 10 monclient(hunting): authenticate will time out at 1189.251047s 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06009c4a0 0x7fb0601346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb06006d1b0 0x7fb060134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06009c4a0 0x7fb0601346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41040/0 (socket says 10.20.193.78:41040) 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 -- 10.20.193.78:0/744279550 learned_addr learned my addr 10.20.193.78:0/744279550 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient(hunting): get_auth_request con 0x7fb06009c4a0 auth_method 0 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05f7fe640 1 -- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb06006c890 msgr2=0x7fb0601cd620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05effd640 10 monclient(hunting): get_auth_request con 0x7fb06006d1b0 auth_method 0 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05f7fe640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb06006c890 0x7fb0601cd620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient(hunting): handle_auth_done global_id 74627 payload 824 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 -- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb06006c890 msgr2=0x7fb0601cd620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb06006c890 0x7fb0601cd620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 -- 10.20.193.78:0/744279550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb06006d1b0 msgr2=0x7fb060134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb06006d1b0 0x7fb060134170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 monclient: found mon.noname-c 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:43.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.697+0000 7fb05ffff640 1 -- 10.20.193.78:0/744279550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb060123880 con 0x7fb06009c4a0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb05ffff640 1 -- 10.20.193.78:0/744279550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06009c4a0 msgr2=0x7fb0601346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb05ffff640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06009c4a0 0x7fb0601346b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb05400ca40 tx=0x7fb05400cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 1 -- 10.20.193.78:0/744279550 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05400ad70 con 0x7fb06009c4a0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: dump: 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: _finish_auth 0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 1 -- 10.20.193.78:0/744279550 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb054012400 con 0x7fb06009c4a0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 1 -- 10.20.193.78:0/744279550 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540127f0 con 0x7fb06009c4a0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb05dffb640 4 set_mon_vals no callback set 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb03bfff640 10 monclient: dump: 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb066043640 5 monclient: authenticate success, global_id 74627 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb066043640 10 monclient: _renew_subs 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb066043640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0600a2620 con 0x7fb06009c4a0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb066043640 10 monclient: _renew_subs 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb066043640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.699+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb060125b70 con 0x7fb06009c4a0 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb066043640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:43.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb066043640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb066043640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06006f750 con 0x7fb06009c4a0 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb03bfff640 1 -- 10.20.193.78:0/744279550 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb054012a10 con 0x7fb06009c4a0 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb03bfff640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb03003deb0 0x7fb030040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb03bfff640 1 -- 10.20.193.78:0/744279550 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(339..339 src has 117..339) ==== 4847+0+0 (secure 0 0 0) 0x7fb054052cb0 con 0x7fb06009c4a0 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb05f7fe640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb03003deb0 0x7fb030040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.700+0000 7fb05f7fe640 10 monclient: get_auth_request con 0x7fb03003deb0 auth_method 0 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.702+0000 7fb03bfff640 1 -- 10.20.193.78:0/744279550 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb06006f750 con 0x7fb06009c4a0 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.702+0000 7fb03bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.702+0000 7fb03bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:43.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.703+0000 7fb05f7fe640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb03003deb0 0x7fb030040370 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb06009c880 tx=0x7fb04c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:43.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.821+0000 7fb066043640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:43.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.821+0000 7fb066043640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:43.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.821+0000 7fb066043640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:43.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.821+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb06006cc70 con 0x7fb06009c4a0 2026-02-17T05:55:43.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.821+0000 7fb03bfff640 1 -- 10.20.193.78:0/744279550 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fb0600a2620 con 0x7fb06009c4a0 2026-02-17T05:55:43.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.821+0000 7fb03bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:43.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.821+0000 7fb03bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:43.820 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb03003deb0 msgr2=0x7fb030040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb03003deb0 0x7fb030040370 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb06009c880 tx=0x7fb04c033000 comp rx=0 tx=0).stop 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 10 monclient: shutdown 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06009c4a0 msgr2=0x7fb0601346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06009c4a0 0x7fb0601346b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb05400ca40 tx=0x7fb05400cf10 comp rx=0 tx=0).stop 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 shutdown_connections 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb06009c4a0 0x7fb0601346b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb06006d1b0 0x7fb060134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb03003deb0 0x7fb030040370 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 --2- 10.20.193.78:0/744279550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb06006c890 0x7fb0601cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 >> 10.20.193.78:0/744279550 conn(0x7fb0600a6620 msgr2=0x7fb060071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 shutdown_connections 2026-02-17T05:55:43.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:43.822+0000 7fb066043640 1 -- 10.20.193.78:0/744279550 wait complete. 2026-02-17T05:55:46.828 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:55:46.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.899+0000 7f7c19546640 10 monclient: get_monmap_and_config 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.899+0000 7f7c19546640 10 monclient: build_initial_monmap 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.899+0000 7f7c19546640 10 monclient: monmap: 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:46.900927+0000 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:46.900927+0000 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:46.901 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 1 Processor -- start 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 1 -- start start 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient: init 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2026-02-17T05:55:46.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient(hunting): picked mon.noname-a con 0x7f7c1406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7c1409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 0x7f7c14135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient(hunting): picked mon.noname-c con 0x7f7c1406d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1406daf0 con 0x7f7c1409c4a0 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7c140a0880 con 0x7f7c1406c890 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1409cc80 con 0x7f7c1406d1b0 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c19546640 10 monclient(hunting): authenticate will time out at 1192.454518s 2026-02-17T05:55:46.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c127fc640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f7c1409c4a0 legacy=0x7f7c14134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c137fe640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 msgr2=0x7f7c14135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c137fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 0x7f7c14135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47370/0 (socket says 10.20.193.78:47370) 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 learned_addr learned my addr 10.20.193.78:0/3343160434 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 10 monclient(hunting): get_auth_request con 0x7f7c1406c890 auth_method 0 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:46.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:46.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:46.900+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:47.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> v1:10.20.193.115:6789/0 conn(0x7f7c1409c4a0 legacy=0x7f7c14134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:55:47.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c137fe640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 msgr2=0x7f7c14135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:47.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c137fe640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 0x7f7c14135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:55:47.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:47.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 10 monclient(hunting): get_auth_request con 0x7f7c1406c890 auth_method 0 2026-02-17T05:55:47.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:47.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:47.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:47.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:55:47.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:47.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:47.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.101+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:55:47.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> v1:10.20.193.115:6789/0 conn(0x7f7c1409c4a0 legacy=0x7f7c14134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:55:47.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c137fe640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 msgr2=0x7f7c14135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:47.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c137fe640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 0x7f7c14135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 10 monclient(hunting): get_auth_request con 0x7f7c1406c890 auth_method 0 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:47.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:47.501+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:55:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> v1:10.20.193.115:6789/0 conn(0x7f7c1409c4a0 legacy=0x7f7c14134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:55:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c137fe640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 msgr2=0x7f7c14135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c137fe640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 0x7f7c14135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 10 monclient(hunting): get_auth_request con 0x7f7c1406c890 auth_method 0 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:48.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:48.302+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:55:49.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 10 monclient(hunting): tick 2026-02-17T05:55:49.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:55:49.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 monclient(hunting): continuing hunt 2026-02-17T05:55:49.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:55:49.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 msgr2=0x7f7c14135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:49.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.115:3300/0 conn(0x7f7c1406d1b0 0x7f7c14135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 -- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 msgr2=0x7f7c1406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 --2- 10.20.193.78:0/3343160434 >> v2:10.20.193.78:3300/0 conn(0x7f7c1406c890 0x7f7c1406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 -- 10.20.193.78:0/3343160434 >> v1:10.20.193.115:6789/0 conn(0x7f7c1409c4a0 legacy=0x7f7c14134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 10 monclient: _add_conns ranks=[0,2] 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7bf8003790 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f7bf8007460 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 -- 10.20.193.78:0/3343160434 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1409cf60 con 0x7f7bf8003790 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 1 -- 10.20.193.78:0/3343160434 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1409cc80 con 0x7f7bf8007460 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c10ff9640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:49.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:49.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 10 monclient(hunting): get_auth_request con 0x7f7bf8007460 auth_method 0 2026-02-17T05:55:49.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:49.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:49.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:49.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:49.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:49.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:49.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.900+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:49.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:49.902+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> v1:10.20.193.78:6789/0 conn(0x7f7bf8003790 legacy=0x7f7bf8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:50.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:50.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 10 monclient(hunting): get_auth_request con 0x7f7bf8007460 auth_method 0 2026-02-17T05:55:50.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:50.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:50.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:50.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:50.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:50.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:50.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.101+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:55:50.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:50.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 10 monclient(hunting): get_auth_request con 0x7f7bf8007460 auth_method 0 2026-02-17T05:55:50.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:50.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:50.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:50.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:50.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:50.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:50.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:50.502+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:55:51.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:51.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 10 monclient(hunting): get_auth_request con 0x7f7bf8007460 auth_method 0 2026-02-17T05:55:51.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:51.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:51.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:51.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:55:51.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:51.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:51.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.303+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:55:51.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.627+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893708661 0 0) 0x7f7c1409cf60 con 0x7f7bf8003790 2026-02-17T05:55:51.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.627+0000 7f7c11ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:55:51.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.627+0000 7f7c11ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:51.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.627+0000 7f7c11ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:51.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.627+0000 7f7c11ffb640 10 monclient(hunting): my global_id is 124182 2026-02-17T05:55:51.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.627+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c00003620 con 0x7f7bf8003790 2026-02-17T05:55:51.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 72927543 0 0) 0x7f7c00003620 con 0x7f7bf8003790 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 msgr2=0x7f7bf8009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7bf8007460 0x7f7bf8009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: _finish_hunting 0 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1406daf0 con 0x7f7bf8003790 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: _finish_auth 0 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f7c08009670 con 0x7f7bf8003790 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: dump: 2026-02-17T05:55:51.634 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 >> v1:10.20.193.78:6789/0 conn(0x7f7bf8003790 legacy=0x7f7bf8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c137fe640 1 -- 10.20.193.78:0/3343160434 reap_dead start 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 0x7f7c00003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient(hunting): picked mon.b con 0x7f7c00003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c00004900 0x7f7c00004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient(hunting): picked mon.c con 0x7f7c00004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c000053e0 0x7f7c0000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient(hunting): picked mon.a con 0x7f7c000053e0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1409cc80 con 0x7f7c000053e0 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0000c290 con 0x7f7c00003ab0 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0000c490 con 0x7f7c00004900 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c11ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c00004900 0x7f7c00004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c137fe640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c000053e0 0x7f7c0000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c127fc640 10 monclient(hunting): get_auth_request con 0x7f7c00004900 auth_method 0 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c127fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:51.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c127fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c137fe640 10 monclient(hunting): get_auth_request con 0x7f7c000053e0 auth_method 0 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c137fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c12ffd640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 msgr2=0x7f7c00003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c137fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 0x7f7c00003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.628+0000 7f7c127fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c137fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 10 monclient(hunting): handle_auth_done global_id 124182 payload 293 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 msgr2=0x7f7c00003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 0x7f7c00003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c000053e0 msgr2=0x7f7c0000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:51.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c000053e0 0x7f7c0000bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 10 monclient: _finish_hunting 0 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 1 monclient: found mon.c 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c00003620 con 0x7f7c00004900 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c19546640 5 monclient: authenticate success, global_id 124182 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c19546640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.629+0000 7f7c137fe640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c000053e0 0x7f7c0000bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c127fc640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c00004900 msgr2=0x7f7c00004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c127fc640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c00004900 0x7f7c00004d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc003a30 tx=0x7f7bfc02ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce679cff2f4d42eb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc036070 con 0x7f7c00004900 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 10 monclient: dump: 2026-02-17T05:55:51.638 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bfc0309a0 con 0x7f7c00004900 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c19546640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 1 -- 10.20.193.78:0/3343160434 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc009b00 con 0x7f7c00004900 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c11ffb640 10 monclient: dump: 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:51.639 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c19546640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.631+0000 7f7c19546640 4 set_mon_vals no callback set 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 10 monclient: shutdown 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 -- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c00004900 msgr2=0x7f7c00004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c00004900 0x7f7c00004d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc003a30 tx=0x7f7bfc02ff70 comp rx=0 tx=0).stop 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 -- 10.20.193.78:0/3343160434 shutdown_connections 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c000053e0 0x7f7c0000bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c00004900 0x7f7c00004d80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 --2- 10.20.193.78:0/3343160434 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 0x7f7c00003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 -- 10.20.193.78:0/3343160434 >> 10.20.193.78:0/3343160434 conn(0x7f7c140a6620 msgr2=0x7f7c140a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 -- 10.20.193.78:0/3343160434 shutdown_connections 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 1 -- 10.20.193.78:0/3343160434 wait complete. 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 10 monclient: build_initial_monmap 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.632+0000 7f7c19546640 10 monclient: monmap: 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 1 Processor -- start 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 1 -- start start 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient: init 2026-02-17T05:55:51.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 0x7f7c141354b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient(hunting): picked mon.noname-b con 0x7f7c00003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c00004900 0x7f7c141359f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient(hunting): picked mon.noname-a con 0x7f7c00004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c000053e0 0x7f7c141cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient(hunting): picked mon.noname-c con 0x7f7c000053e0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140a01a0 con 0x7f7c00004900 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140a0020 con 0x7f7c00003ab0 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1409be20 con 0x7f7c000053e0 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c19546640 10 monclient(hunting): authenticate will time out at 1197.187399s 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c12ffd640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 msgr2=0x7f7c141354b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c000053e0 0x7f7c141cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:51.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c00004900 0x7f7c141359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c12ffd640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 0x7f7c141354b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c000053e0 0x7f7c141cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38064/0 (socket says 10.20.193.78:38064) 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c00004900 0x7f7c141359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45886/0 (socket says 10.20.193.78:45886) 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 -- 10.20.193.78:0/1735748797 learned_addr learned my addr 10.20.193.78:0/1735748797 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient(hunting): get_auth_request con 0x7f7c000053e0 auth_method 0 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 10 monclient(hunting): get_auth_request con 0x7f7c00004900 auth_method 0 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:51.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient(hunting): handle_auth_done global_id 74630 payload 824 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 -- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 msgr2=0x7f7c141354b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 0x7f7c141354b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 -- 10.20.193.78:0/1735748797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c00004900 msgr2=0x7f7c141359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c00004900 0x7f7c141359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 monclient: found mon.noname-c 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:51.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.633+0000 7f7c137fe640 1 -- 10.20.193.78:0/1735748797 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c14121b20 con 0x7f7c000053e0 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c137fe640 1 -- 10.20.193.78:0/1735748797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c000053e0 msgr2=0x7f7c141cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c137fe640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c000053e0 0x7f7c141cc830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7c1406dcf0 tx=0x7f7c0400fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 1 -- 10.20.193.78:0/1735748797 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04015070 con 0x7f7c000053e0 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: dump: 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:51.646 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 1 -- 10.20.193.78:0/1735748797 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0400de60 con 0x7f7c000053e0 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 1 -- 10.20.193.78:0/1735748797 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04011690 con 0x7f7c000053e0 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c117fa640 4 set_mon_vals no callback set 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7bf37fe640 10 monclient: dump: 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.647 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c19546640 5 monclient: authenticate success, global_id 74630 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c19546640 10 monclient: _renew_subs 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c19546640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1409d810 con 0x7f7c000053e0 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c19546640 10 monclient: _renew_subs 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c19546640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.635+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c14123e80 con 0x7f7c000053e0 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7bf37fe640 1 -- 10.20.193.78:0/1735748797 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7c0400e040 con 0x7f7c000053e0 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7c19546640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7c19546640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7c19546640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c14074980 con 0x7f7c000053e0 2026-02-17T05:55:51.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7bf37fe640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7be003deb0 0x7f7be0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7bf37fe640 1 -- 10.20.193.78:0/1735748797 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(340..340 src has 117..340) ==== 4847+0+0 (secure 0 0 0) 0x7f7c04053030 con 0x7f7c000053e0 2026-02-17T05:55:51.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7be003deb0 0x7f7be0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:51.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.636+0000 7f7c12ffd640 10 monclient: get_auth_request con 0x7f7be003deb0 auth_method 0 2026-02-17T05:55:51.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.639+0000 7f7bf37fe640 1 -- 10.20.193.78:0/1735748797 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7c040519f0 con 0x7f7c000053e0 2026-02-17T05:55:51.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.639+0000 7f7bf37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:51.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.639+0000 7f7bf37fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:51.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.639+0000 7f7c12ffd640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7be003deb0 0x7f7be0040370 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c080097e0 tx=0x7f7c08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:51.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.752+0000 7f7c19546640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:55:51.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.752+0000 7f7c19546640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:51.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.752+0000 7f7c19546640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:51.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.752+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7c1406cc70 con 0x7f7c000053e0 2026-02-17T05:55:51.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.752+0000 7f7bf37fe640 1 -- 10.20.193.78:0/1735748797 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f7c04011870 con 0x7f7c000053e0 2026-02-17T05:55:51.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.752+0000 7f7bf37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:55:51.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.752+0000 7f7bf37fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:51.753 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":110,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:55:51.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7be003deb0 msgr2=0x7f7be0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:51.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7be003deb0 0x7f7be0040370 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c080097e0 tx=0x7f7c08008040 comp rx=0 tx=0).stop 2026-02-17T05:55:51.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 10 monclient: shutdown 2026-02-17T05:55:51.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c000053e0 msgr2=0x7f7c141cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c000053e0 0x7f7c141cc830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7c1406dcf0 tx=0x7f7c0400fdb0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 shutdown_connections 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7c000053e0 0x7f7c141cc830 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7c00004900 0x7f7c141359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7be003deb0 0x7f7be0040370 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 --2- 10.20.193.78:0/1735748797 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7c00003ab0 0x7f7c141354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 >> 10.20.193.78:0/1735748797 conn(0x7f7c140a6620 msgr2=0x7f7c140a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 shutdown_connections 2026-02-17T05:55:51.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.753+0000 7f7c19546640 1 -- 10.20.193.78:0/1735748797 wait complete. 2026-02-17T05:55:51.760 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:55:51.761 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T05:55:51.761 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.831+0000 7f18e4196640 10 monclient: get_monmap_and_config 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.831+0000 7f18e4196640 10 monclient: build_initial_monmap 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.831+0000 7f18e4196640 10 monclient: monmap: 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:51.832936+0000 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:51.832936+0000 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:51.831 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 1 Processor -- start 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 1 -- start start 2026-02-17T05:55:51.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient: init 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient(hunting): picked mon.noname-a con 0x7f18dc0a1310 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18dc135d00 0x7f18dc0a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient(hunting): picked mon.noname-b con 0x7f18dc135d00 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f18dc1360e0 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18dc06c890 con 0x7f18dc1360e0 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f18dc06d5a0 con 0x7f18dc0a1310 2026-02-17T05:55:51.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f18dc0a1c30 con 0x7f18dc135d00 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e4196640 10 monclient(hunting): authenticate will time out at 1197.386467s 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45888/0 (socket says 10.20.193.78:45888) 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 learned_addr learned my addr 10.20.193.78:0/3579231687 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient(hunting): get_auth_request con 0x7f18dc0a1310 auth_method 0 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e270c640 1 -- 10.20.193.78:0/3579231687 >> v1:10.20.193.115:6789/0 conn(0x7f18dc1360e0 legacy=0x7f18dc0a3680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e170a640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18dc135d00 0x7f18dc0a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e170a640 10 monclient(hunting): get_auth_request con 0x7f18dc135d00 auth_method 0 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e170a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e170a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e170a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:51.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e170a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e170a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e170a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_done global_id 124188 payload 274 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18dc135d00 msgr2=0x7f18dc0a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18dc135d00 0x7f18dc0a3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 >> v1:10.20.193.115:6789/0 conn(0x7f18dc1360e0 legacy=0x7f18dc0a3680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient: _finish_hunting 0 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 1 monclient: found mon.noname-a 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.832+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc06d2c0 con 0x7f18dc0a1310 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.833+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 msgr2=0x7f18dc0a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.833+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 msgr2=0x7f18dc0a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.833+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f18cc009970 tx=0x7f18cc02ee90 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:51.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:51.833+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:52.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e1f0b640 10 monclient: get_auth_request con 0x7f18dc0a1310 auth_method 0 2026-02-17T05:55:52.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e1f0b640 0 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-17T05:55:52.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 10 monclient: ms_handle_reset current mon v2:10.20.193.78:3300/0 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 msgr2=0x7f18dc0a16f0 unknown :-1 s=STATE_CLOSED l=0).mark_down 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 1 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.78:3300/0 conn(0x7f18dc0a1310 0x7f18dc0a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 10 monclient: _add_conns ranks=[0,4] 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f18c4003600 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 1 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.115:3300/0 conn(0x7f18c4007590 0x7f18c4009a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 10 monclient(hunting): picked mon.noname-c con 0x7f18c4007590 addr v2:10.20.193.115:3300/0 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c400a320 con 0x7f18c4003600 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f18c400b520 con 0x7f18c4007590 2026-02-17T05:55:52.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:52.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e0f09640 10 monclient(hunting): ms_handle_reset stray mon v2:10.20.193.78:3300/0 2026-02-17T05:55:52.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e170a640 1 -- 10.20.193.78:0/3579231687 >> v2:10.20.193.115:3300/0 conn(0x7f18c4007590 msgr2=0x7f18c4009a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:52.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.033+0000 7f18e170a640 1 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.115:3300/0 conn(0x7f18c4007590 0x7f18c4009a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:52.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 >> v1:10.20.193.78:6789/0 conn(0x7f18c4003600 legacy=0x7f18c4005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4161954524 0 0) 0x7f18c400a320 con 0x7f18c4003600 2026-02-17T05:55:52.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 10 monclient(hunting): _negotiate 2026-02-17T05:55:52.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 --> v1:10.20.193.78:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f18c400a4f0 con 0x7f18c4003600 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 2802544679 0 0) 0x7f18c400a4f0 con 0x7f18c4003600 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 >> v2:10.20.193.115:3300/0 conn(0x7f18c4007590 msgr2=0x7f18c4009a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 1 --2- 10.20.193.78:0/3579231687 >> v2:10.20.193.115:3300/0 conn(0x7f18c4007590 0x7f18c4009a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 10 monclient: _finish_hunting 0 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.035+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c400b790 con 0x7f18c4003600 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient: _finish_auth 0 2026-02-17T05:55:52.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient: _check_auth_tickets 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f18cc032160 con 0x7f18c4003600 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e4196640 5 monclient: authenticate success, global_id 124188 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e4196640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient: dump: 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.050 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 >> v1:10.20.193.78:6789/0 conn(0x7f18c4003600 legacy=0x7f18c4005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 1 -- 10.20.193.78:0/3579231687 reap_dead start 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:55:52.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ac70 0x7f18c40099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient(hunting): picked mon.a con 0x7f18c400ac70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ba20 0x7f18c400de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient(hunting): picked mon.b con 0x7f18c400ba20 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18c4010740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient(hunting): picked mon.c con 0x7f18c400e350 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18c400b520 con 0x7f18c400ac70 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4010e70 con 0x7f18c400ba20 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f18c4011070 con 0x7f18c400e350 2026-02-17T05:55:52.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e0f09640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e4196640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ac70 0x7f18c40099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e170a640 1 -- 10.20.193.78:0/3579231687 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ba20 msgr2=0x7f18c400de10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient(hunting): get_auth_request con 0x7f18c400ac70 auth_method 0 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e170a640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ba20 0x7f18c400de10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18c4010740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 10 monclient(hunting): get_auth_request con 0x7f18c400e350 auth_method 0 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_done global_id 124188 payload 293 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 msgr2=0x7f18c4010740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18c4010740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ba20 msgr2=0x7f18c400de10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:52.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ba20 0x7f18c400de10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient: _finish_hunting 0 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 1 monclient: found mon.a 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c400a4f0 con 0x7f18c400ac70 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient: _finish_auth 0 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 10 monclient: _check_auth_tickets 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e1f0b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.036+0000 7f18e270c640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18c4010740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e1f0b640 1 -- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ac70 msgr2=0x7f18c40099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ac70 0x7f18c40099d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18dc06ffe0 tx=0x7f18c400a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f50e8e314902e73 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc03d020 con 0x7f18c400ac70 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 10 monclient: dump: 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.056 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e4196640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18cc003230 con 0x7f18c400ac70 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 1 -- 10.20.193.78:0/3579231687 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc004380 con 0x7f18c400ac70 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e0f09640 10 monclient: dump: 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.057 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e4196640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.038+0000 7f18e4196640 4 set_mon_vals no callback set 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 10 monclient: shutdown 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 -- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ac70 msgr2=0x7f18c40099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ac70 0x7f18c40099d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18dc06ffe0 tx=0x7f18c400a730 comp rx=0 tx=0).stop 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 -- 10.20.193.78:0/3579231687 shutdown_connections 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18c4010740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ba20 0x7f18c400de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 --2- 10.20.193.78:0/3579231687 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ac70 0x7f18c40099d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 -- 10.20.193.78:0/3579231687 >> 10.20.193.78:0/3579231687 conn(0x7f18dc0a4230 msgr2=0x7f18dc0a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 -- 10.20.193.78:0/3579231687 shutdown_connections 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.039+0000 7f18e4196640 1 -- 10.20.193.78:0/3579231687 wait complete. 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient: build_initial_monmap 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient: monmap: 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:52.059 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 1 Processor -- start 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 1 -- start start 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient: init 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:52.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ac70 0x7f18dc134f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient(hunting): picked mon.noname-b con 0x7f18c400ac70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ba20 0x7f18dc1cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient(hunting): picked mon.noname-a con 0x7f18c400ba20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18dc1cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient(hunting): picked mon.noname-c con 0x7f18c400e350 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc09e670 con 0x7f18c400ba20 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc09e4f0 con 0x7f18c400ac70 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f18dc0a16f0 con 0x7f18c400e350 2026-02-17T05:55:52.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e4196640 10 monclient(hunting): authenticate will time out at 1197.594663s 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e1f0b640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ac70 msgr2=0x7f18dc134f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18dc1cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ba20 0x7f18dc1cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e1f0b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ac70 0x7f18dc134f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18dc1cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38090/0 (socket says 10.20.193.78:38090) 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ba20 0x7f18dc1cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45914/0 (socket says 10.20.193.78:45914) 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 -- 10.20.193.78:0/1572446563 learned_addr learned my addr 10.20.193.78:0/1572446563 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient(hunting): get_auth_request con 0x7f18c400e350 auth_method 0 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 10 monclient(hunting): get_auth_request con 0x7f18c400ba20 auth_method 0 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e170a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient(hunting): handle_auth_done global_id 74636 payload 824 2026-02-17T05:55:52.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ac70 msgr2=0x7f18dc134f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18c400ac70 0x7f18dc134f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ba20 msgr2=0x7f18dc1cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18c400ba20 0x7f18dc1cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient: _finish_hunting 0 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 monclient: found mon.noname-c 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.040+0000 7f18e270c640 1 -- 10.20.193.78:0/1572446563 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc08e340 con 0x7f18c400e350 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.042+0000 7f18e270c640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 msgr2=0x7f18dc1cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.042+0000 7f18e270c640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18dc1cfa70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f18c40072c0 tx=0x7f18d800c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8014070 con 0x7f18c400e350 2026-02-17T05:55:52.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: dump: 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.065 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: _finish_auth 0 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d8010650 con 0x7f18c400e350 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8010a40 con 0x7f18c400e350 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 10 monclient: dump: 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.066 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.067 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.067 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.067 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.067 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.067 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.067 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18cbfff640 4 set_mon_vals no callback set 2026-02-17T05:55:52.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 5 monclient: authenticate success, global_id 74636 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 10 monclient: _renew_subs 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc0a3ff0 con 0x7f18c400e350 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 10 monclient: _renew_subs 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc09e920 con 0x7f18c400e350 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09b7a0 0x7f18dc09bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1cffb0 con 0x7f18dc09b7a0 2026-02-17T05:55:52.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09b7a0 0x7f18dc09bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 10 monclient: get_auth_request con 0x7f18dc09b7a0 auth_method 0 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f18d800d040 con 0x7f18c400e350 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_done global_id 124194 payload 824 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18e1f0b640 1 -- 10.20.193.78:0/1572446563 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18dc074980 con 0x7f18dc09b7a0 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18ac03de90 0x7f18ac040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.043+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(340..340 src has 117..340) ==== 4847+0+0 (secure 0 0 0) 0x7f18d8053000 con 0x7f18c400e350 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.044+0000 7f18e170a640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18ac03de90 0x7f18ac040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.044+0000 7f18e170a640 10 monclient: get_auth_request con 0x7f18ac03de90 auth_method 0 2026-02-17T05:55:52.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.045+0000 7f18e1f0b640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09b7a0 msgr2=0x7f18dc09bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.045+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09b7a0 0x7f18dc09bb80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009970 tx=0x7f18cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc02f3f0 con 0x7f18dc09b7a0 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 10 monclient: dump: 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.070 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f18cc02fad0 con 0x7f18dc09b7a0 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09b7a0 msgr2=0x7f18dc09bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09b7a0 0x7f18dc09bb80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009970 tx=0x7f18cc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09b7a0 msgr2=0x7f18dc09bb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.046+0000 7f18ca7fc640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09b7a0 0x7f18dc09bb80 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009970 tx=0x7f18cc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.047+0000 7f18e170a640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18ac03de90 0x7f18ac040350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f18dc135760 tx=0x7f18d0009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e4196640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:55:52.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e4196640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09c0c0 0x7f18dc1da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e4196640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc09d700 con 0x7f18dc09c0c0 2026-02-17T05:55:52.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09c0c0 0x7f18dc1da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e1f0b640 10 monclient: get_auth_request con 0x7f18dc09c0c0 auth_method 0 2026-02-17T05:55:52.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e1f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e1f0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e1f0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.049+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.050+0000 7f18e1f0b640 10 monclient(hunting): handle_auth_done global_id 124197 payload 824 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.050+0000 7f18e1f0b640 1 -- 10.20.193.78:0/1572446563 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f18dc0a5220 con 0x7f18dc09c0c0 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e1f0b640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09c0c0 msgr2=0x7f18dc1da3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e1f0b640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09c0c0 0x7f18dc1da3a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18dc075fe0 tx=0x7f18cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc032160 con 0x7f18dc09c0c0 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 10 monclient: dump: 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7f18cc032160 con 0x7f18dc09c0c0 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:55:52.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09c0c0 msgr2=0x7f18dc1da3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09c0c0 0x7f18dc1da3a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18dc075fe0 tx=0x7f18cc008040 comp rx=0 tx=0).stop 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09c0c0 msgr2=0x7f18dc1da3a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18ca7fc640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18dc09c0c0 0x7f18dc1da3a0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18dc075fe0 tx=0x7f18cc008040 comp rx=0 tx=0).stop 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:55:52.076 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 110, 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 43519, 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:52.077 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:52.078 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:52.079 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:52.080 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.081 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:52.082 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:52.083 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:52.084 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:52.085 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:55:52.086 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:55:52.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18ac03de90 msgr2=0x7f18ac040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f18ac03de90 0x7f18ac040350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f18dc135760 tx=0x7f18d0009290 comp rx=0 tx=0).stop 2026-02-17T05:55:52.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 10 monclient: shutdown 2026-02-17T05:55:52.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:52.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 msgr2=0x7f18dc1cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18dc1cfa70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f18c40072c0 tx=0x7f18d800c6c0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e270c640 1 -- 10.20.193.78:0/1572446563 reap_dead start 2026-02-17T05:55:52.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 shutdown_connections 2026-02-17T05:55:52.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 --2- 10.20.193.78:0/1572446563 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18c400e350 0x7f18dc1cfa70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 >> 10.20.193.78:0/1572446563 conn(0x7f18dc0a4230 msgr2=0x7f18dc072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:52.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 shutdown_connections 2026-02-17T05:55:52.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.052+0000 7f18e4196640 1 -- 10.20.193.78:0/1572446563 wait complete. 2026-02-17T05:55:52.088 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:55:52.088 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:55:52.088 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:55:52.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.156+0000 7f63254fb640 10 monclient: get_monmap_and_config 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.156+0000 7f63254fb640 10 monclient: build_initial_monmap 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.156+0000 7f63254fb640 10 monclient: monmap: 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:55:52.157649+0000 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:55:52.157649+0000 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:55:52.158 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.156+0000 7f63254fb640 1 Processor -- start 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.156+0000 7f63254fb640 1 -- start start 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.156+0000 7f63254fb640 10 monclient: init 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f632006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f632006d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f63200a01f0 0x7f632013d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 10 monclient(hunting): picked mon.noname-a con 0x7f63200a01f0 addr v2:10.20.193.78:3300/0 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63200a0600 con 0x7f632006c890 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6320074150 con 0x7f632006d380 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f632009ef70 con 0x7f63200a01f0 2026-02-17T05:55:52.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f63254fb640 10 monclient(hunting): authenticate will time out at 1197.711192s 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631effd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f632006c890 0x7f632006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50438/0 (socket says 10.20.193.78:50438) 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631effd640 1 -- 10.20.193.78:0/2851965953 learned_addr learned my addr 10.20.193.78:0/2851965953 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 1 --2- 10.20.193.78:0/2851965953 >> v2:10.20.193.78:3300/0 conn(0x7f63200a01f0 0x7f632013d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631e7fc640 1 -- 10.20.193.78:0/2851965953 >> v1:10.20.193.115:6789/0 conn(0x7f632006d380 legacy=0x7f632013a240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient(hunting): get_auth_request con 0x7f63200a01f0 auth_method 0 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient(hunting): handle_auth_done global_id 124200 payload 274 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 1 -- 10.20.193.78:0/2851965953 >> v1:10.20.193.115:6789/0 conn(0x7f632006d380 legacy=0x7f632013a240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:55:52.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 1 -- 10.20.193.78:0/2851965953 >> v1:10.20.193.78:6789/0 conn(0x7f632006c890 legacy=0x7f632006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:55:52.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:52.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 1 monclient: found mon.noname-a 2026-02-17T05:55:52.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:52.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:52.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.157+0000 7f631f7fe640 1 -- 10.20.193.78:0/2851965953 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632009f250 con 0x7f63200a01f0 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631f7fe640 1 -- 10.20.193.78:0/2851965953 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 msgr2=0x7f632013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631f7fe640 1 --2- 10.20.193.78:0/2851965953 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 0x7f632013d970 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6310009d90 tx=0x7f631002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c03ba2e615bf81b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 1 -- 10.20.193.78:0/2851965953 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631003c070 con 0x7f63200a01f0 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: dump: 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.167 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: _finish_auth 0 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 1 -- 10.20.193.78:0/2851965953 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6310037640 con 0x7f63200a01f0 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 5 monclient: authenticate success, global_id 124200 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 10 monclient: get_monmap_and_config success 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 4 set_mon_vals no callback set 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 1 -- 10.20.193.78:0/2851965953 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310037a10 con 0x7f63200a01f0 2026-02-17T05:55:52.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f631dffb640 10 monclient: dump: 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 10 monclient: shutdown 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 1 -- 10.20.193.78:0/2851965953 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 msgr2=0x7f632013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 1 --2- 10.20.193.78:0/2851965953 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 0x7f632013d970 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6310009d90 tx=0x7f631002f1e0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 1 -- 10.20.193.78:0/2851965953 shutdown_connections 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 1 --2- 10.20.193.78:0/2851965953 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 0x7f632013d970 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 1 -- 10.20.193.78:0/2851965953 >> 10.20.193.78:0/2851965953 conn(0x7f63200a6620 msgr2=0x7f63200a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 1 -- 10.20.193.78:0/2851965953 shutdown_connections 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.160+0000 7f63254fb640 1 -- 10.20.193.78:0/2851965953 wait complete. 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient: build_initial_monmap 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient: monmap: 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:55:52.170 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 1 Processor -- start 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 1 -- start start 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient: init 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient: _reopen_session rank -1 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f632006c890 0x7f63201cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient(hunting): picked mon.noname-b con 0x7f632006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f632006d380 0x7f632013e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient(hunting): picked mon.noname-c con 0x7f632006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 0x7f632013ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient(hunting): picked mon.noname-a con 0x7f63200a01f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f632009d360 con 0x7f63200a01f0 2026-02-17T05:55:52.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f632009d1e0 con 0x7f632006c890 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f632009be20 con 0x7f632006d380 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient(hunting): _renew_subs 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f63254fb640 10 monclient(hunting): authenticate will time out at 1197.715489s 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 0x7f632013ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631effd640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f632006c890 msgr2=0x7f63201cf4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 0x7f632013ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45962/0 (socket says 10.20.193.78:45962) 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f632006d380 0x7f632013e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 -- 10.20.193.78:0/1109374454 learned_addr learned my addr 10.20.193.78:0/1109374454 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient(hunting): get_auth_request con 0x7f63200a01f0 auth_method 0 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631effd640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f632006c890 0x7f63201cf4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 10 monclient(hunting): get_auth_request con 0x7f632006d380 auth_method 0 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f632006d380 msgr2=0x7f632013e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f632006d380 msgr2=0x7f632013e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f632006d380 0x7f632013e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631e7fc640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f632006d380 0x7f632013e900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:55:52.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient(hunting): handle_auth_done global_id 124203 payload 824 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f632006d380 msgr2=0x7f632013e900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f632006d380 0x7f632013e900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f632006c890 msgr2=0x7f63201cf4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f632006c890 0x7f63201cf4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 monclient: found mon.noname-a 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.161+0000 7f631f7fe640 1 -- 10.20.193.78:0/1109374454 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6320138250 con 0x7f63200a01f0 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.163+0000 7f631f7fe640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 msgr2=0x7f632013ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.163+0000 7f631f7fe640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 0x7f632013ee40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6310009d90 tx=0x7f6310004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310047020 con 0x7f63200a01f0 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: dump: 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: _finish_auth 0 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:55:52.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6310042680 con 0x7f63200a01f0 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631003c040 con 0x7f63200a01f0 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f62ff7fe640 10 monclient: dump: 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f631d7fa640 4 set_mon_vals no callback set 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f63254fb640 5 monclient: authenticate success, global_id 124203 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.165+0000 7f63254fb640 10 monclient: _renew_subs 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6320136840 con 0x7f63200a01f0 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 10 monclient: _renew_subs 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632009d810 con 0x7f63200a01f0 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6320137e70 0x7f63201340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f63201cfa20 con 0x7f6320137e70 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6320137e70 0x7f63201340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 10 monclient: get_auth_request con 0x7f6320137e70 auth_method 0 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6310042af0 con 0x7f63200a01f0 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 10 monclient(hunting): handle_auth_done global_id 74642 payload 824 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f631effd640 1 -- 10.20.193.78:0/1109374454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f632009ca50 con 0x7f6320137e70 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f62ff7fe640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03de90 0x7f62ec040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.166+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(340..340 src has 117..340) ==== 4847+0+0 (secure 0 0 0) 0x7f63100783f0 con 0x7f63200a01f0 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.167+0000 7f631e7fc640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03de90 0x7f62ec040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.167+0000 7f631e7fc640 10 monclient: get_auth_request con 0x7f62ec03de90 auth_method 0 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.168+0000 7f631effd640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6320137e70 msgr2=0x7f63201340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.168+0000 7f631effd640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6320137e70 0x7f63201340b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6314004970 tx=0x7f631400cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314017070 con 0x7f6320137e70 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 10 monclient: dump: 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.181 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6314005d40 con 0x7f6320137e70 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6320137e70 msgr2=0x7f63201340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6320137e70 0x7f63201340b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6314004970 tx=0x7f631400cb20 comp rx=0 tx=0).stop 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6320137e70 msgr2=0x7f63201340b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f62ff7fe640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6320137e70 0x7f63201340b0 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6314004970 tx=0x7f631400cb20 comp rx=0 tx=0).stop 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.169+0000 7f631e7fc640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03de90 0x7f62ec040350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6308009470 tx=0x7f6308009400 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f63254fb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:55:52.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f63254fb640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f63201345f0 0x7f63201da620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f63254fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6320137a30 con 0x7f63201345f0 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f63201345f0 0x7f63201da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 10 monclient: get_auth_request con 0x7f63201345f0 auth_method 0 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 10 monclient(hunting): handle_auth_done global_id 74645 payload 824 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.172+0000 7f631effd640 1 -- 10.20.193.78:0/1109374454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f632006cc70 con 0x7f63201345f0 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f631effd640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f63201345f0 msgr2=0x7f63201da620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:55:52.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f631effd640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f63201345f0 0x7f63201da620 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6314014000 tx=0x7f6314004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631401ae60 con 0x7f63201345f0 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 10 monclient: dump: 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:55:52.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7f6314007490 con 0x7f63201345f0 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f63201345f0 msgr2=0x7f63201da620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f63201345f0 0x7f63201da620 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6314014000 tx=0x7f6314004b10 comp rx=0 tx=0).stop 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f63201345f0 msgr2=0x7f63201da620 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.174+0000 7f62ff7fe640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f63201345f0 0x7f63201da620 secure :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6314014000 tx=0x7f6314004b10 comp rx=0 tx=0).stop 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:52.185 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 110, 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 324067, 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:52.186 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:52.187 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:55:52.188 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:55:52.189 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.190 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:52.191 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:55:52.192 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:55:52.193 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:55:52.194 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:52.195 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:55:52.196 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03de90 msgr2=0x7f62ec040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03de90 0x7f62ec040350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6308009470 tx=0x7f6308009400 comp rx=0 tx=0).stop 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 10 monclient: shutdown 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 msgr2=0x7f632013ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 1 --2- 10.20.193.78:0/1109374454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f63200a01f0 0x7f632013ee40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6310009d90 tx=0x7f6310004b40 comp rx=0 tx=0).stop 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f631f7fe640 1 -- 10.20.193.78:0/1109374454 reap_dead start 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f631f7fe640 1 -- 10.20.193.78:0/1109374454 reap_dead start 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 shutdown_connections 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 >> 10.20.193.78:0/1109374454 conn(0x7f63200a6620 msgr2=0x7f63201301d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 shutdown_connections 2026-02-17T05:55:52.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:55:52.175+0000 7f63254fb640 1 -- 10.20.193.78:0/1109374454 wait complete. 2026-02-17T05:55:52.199 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:55:52.199 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:55:52.629 INFO:tasks.rados.rados.0.trial115.stdout:1613: done (0 left) 2026-02-17T05:55:52.629 INFO:tasks.rados.rados.0.trial115.stdout:1614: read oid 8 snap -1 2026-02-17T05:55:52.629 INFO:tasks.rados.rados.0.trial115.stdout:1614: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:55:52.629 INFO:tasks.rados.rados.0.trial115.stdout:1615: copy_from oid 25 from oid 45 current snap is 158 2026-02-17T05:55:52.629 INFO:tasks.rados.rados.0.trial115.stdout:1616: rollback oid 40 current snap is 158 2026-02-17T05:55:52.629 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 156 2026-02-17T05:55:52.629 INFO:tasks.rados.rados.0.trial115.stdout:1617: rollback oid 14 current snap is 158 2026-02-17T05:55:52.630 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 152 2026-02-17T05:55:52.630 INFO:tasks.rados.rados.0.trial115.stdout:1618: snap_remove snap 151 2026-02-17T05:55:53.538 INFO:tasks.rados.rados.0.trial115.stdout:1618: done (4 left) 2026-02-17T05:55:53.538 INFO:tasks.rados.rados.0.trial115.stdout:1619: write oid 23 current snap is 158 2026-02-17T05:55:53.538 INFO:tasks.rados.rados.0.trial115.stdout:1617: finishing rollback tid 1 to trial11545420-14 2026-02-17T05:55:53.538 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 0 (ObjNum 33 snap 0 seq_num 1330673496) dirty dne 2026-02-17T05:55:53.538 INFO:tasks.rados.rados.0.trial115.stdout:1616: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:55:53.538 INFO:tasks.rados.rados.0.trial115.stdout:1616: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:55:53.538 INFO:tasks.rados.rados.0.trial115.stdout:1616: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:55:53.539 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 702 (ObjNum 548 snap 142 seq_num 548) dirty exists 2026-02-17T05:55:53.539 INFO:tasks.rados.rados.0.trial115.stdout:1614: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:55:53.762 INFO:tasks.rados.rados.0.trial115.stdout:1615: finishing copy_from racing read to trial11545420-25 2026-02-17T05:55:53.762 INFO:tasks.rados.rados.0.trial115.stdout:1615: finishing copy_from to trial11545420-25 2026-02-17T05:55:53.762 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 852 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:55:53.762 INFO:tasks.rados.rados.0.trial115.stdout:1619: seq_num 603 ranges {534621=567034,1636289=498332} 2026-02-17T05:55:53.766 INFO:tasks.rados.rados.0.trial115.stdout:1619: writing trial11545420-23 from 534621 to 1101655 tid 1 2026-02-17T05:55:53.770 INFO:tasks.rados.rados.0.trial115.stdout:1619: writing trial11545420-23 from 1636289 to 2134621 tid 2 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1614: done (4 left) 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1615: done (3 left) 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1616: done (2 left) 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1617: done (1 left) 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1620: rollback oid 4 current snap is 158 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 157 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1621: read oid 35 snap -1 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1621: expect (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1622: delete oid 11 current snap is 158 2026-02-17T05:55:53.771 INFO:tasks.rados.rados.0.trial115.stdout:1619: finishing write tid 1 to trial11545420-23 2026-02-17T05:55:53.772 INFO:tasks.rados.rados.0.trial115.stdout:1622: done (3 left) 2026-02-17T05:55:53.772 INFO:tasks.rados.rados.0.trial115.stdout:1623: rollback oid 29 current snap is 158 2026-02-17T05:55:53.772 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 154 2026-02-17T05:55:53.772 INFO:tasks.rados.rados.0.trial115.stdout:1624: rollback oid 40 current snap is 158 2026-02-17T05:55:53.772 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 152 2026-02-17T05:55:53.772 INFO:tasks.rados.rados.0.trial115.stdout:1625: copy_from oid 6 from oid 44 current snap is 158 2026-02-17T05:55:53.772 INFO:tasks.rados.rados.0.trial115.stdout:1626: delete oid 48 current snap is 158 2026-02-17T05:55:53.774 INFO:tasks.rados.rados.0.trial115.stdout:1623: finishing rollback tid 0 to trial11545420-29 2026-02-17T05:55:53.774 INFO:tasks.rados.rados.0.trial115.stdout:1623: finishing rollback tid 1 to trial11545420-29 2026-02-17T05:55:53.774 INFO:tasks.rados.rados.0.trial115.stdout:1623: finishing rollback tid 2 to trial11545420-29 2026-02-17T05:55:53.774 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 631 (ObjNum 536 snap 136 seq_num 536) dirty exists 2026-02-17T05:55:53.777 INFO:tasks.rados.rados.0.trial115.stdout:1621: expect (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1619: finishing write tid 2 to trial11545420-23 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1619: finishing write tid 3 to trial11545420-23 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1619: finishing write tid 4 to trial11545420-23 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1619: oid 23 updating version 0 to 331 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1619: oid 23 updating version 331 to 332 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1619: oid 23 updating version 332 to 333 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 333 (ObjNum 603 snap 158 seq_num 603) dirty exists 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1619: left oid 23 (ObjNum 603 snap 158 seq_num 603) 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1620: finishing rollback tid 0 to trial11545420-4 2026-02-17T05:55:53.935 INFO:tasks.rados.rados.0.trial115.stdout:1620: finishing rollback tid 1 to trial11545420-4 2026-02-17T05:55:53.936 INFO:tasks.rados.rados.0.trial115.stdout:1620: finishing rollback tid 2 to trial11545420-4 2026-02-17T05:55:53.936 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 705 (ObjNum 505 snap 130 seq_num 505) dirty exists 2026-02-17T05:55:53.936 INFO:tasks.rados.rados.0.trial115.stdout:1624: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:55:53.936 INFO:tasks.rados.rados.0.trial115.stdout:1624: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:55:53.936 INFO:tasks.rados.rados.0.trial115.stdout:1619: done (6 left) 2026-02-17T05:55:53.936 INFO:tasks.rados.rados.0.trial115.stdout:1620: done (5 left) 2026-02-17T05:55:53.937 INFO:tasks.rados.rados.0.trial115.stdout:1621: done (4 left) 2026-02-17T05:55:53.937 INFO:tasks.rados.rados.0.trial115.stdout:1623: done (3 left) 2026-02-17T05:55:53.937 INFO:tasks.rados.rados.0.trial115.stdout:1626: done (2 left) 2026-02-17T05:55:53.937 INFO:tasks.rados.rados.0.trial115.stdout:1627: rollback oid 9 current snap is 158 2026-02-17T05:55:53.937 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 9 to 152 2026-02-17T05:55:53.937 INFO:tasks.rados.rados.0.trial115.stdout:1624: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:55:53.938 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 708 (ObjNum 548 snap 142 seq_num 548) dirty exists 2026-02-17T05:55:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1625: finishing copy_from racing read to trial11545420-6 2026-02-17T05:55:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1625: finishing copy_from to trial11545420-6 2026-02-17T05:55:53.938 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 643 (ObjNum 568 snap 147 seq_num 568) dirty exists 2026-02-17T05:55:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1624: done (2 left) 2026-02-17T05:55:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1625: done (1 left) 2026-02-17T05:55:53.938 INFO:tasks.rados.rados.0.trial115.stdout:1628: read oid 41 snap -1 2026-02-17T05:55:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1628: expect deleted 2026-02-17T05:55:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1629: read oid 46 snap 157 2026-02-17T05:55:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1629: expect deleted 2026-02-17T05:55:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1630: read oid 23 snap -1 2026-02-17T05:55:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1630: expect (ObjNum 603 snap 158 seq_num 603) 2026-02-17T05:55:53.939 INFO:tasks.rados.rados.0.trial115.stdout:1631: delete oid 40 current snap is 158 2026-02-17T05:55:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1628: done (4 left) 2026-02-17T05:55:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1631: done (3 left) 2026-02-17T05:55:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1632: delete oid 43 current snap is 158 2026-02-17T05:55:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1627: finishing rollback tid 0 to trial11545420-9 2026-02-17T05:55:53.940 INFO:tasks.rados.rados.0.trial115.stdout:1627: finishing rollback tid 1 to trial11545420-9 2026-02-17T05:55:53.941 INFO:tasks.rados.rados.0.trial115.stdout:1627: finishing rollback tid 2 to trial11545420-9 2026-02-17T05:55:53.942 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 792 (ObjNum 377 snap 99 seq_num 377) dirty exists 2026-02-17T05:55:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1627: done (3 left) 2026-02-17T05:55:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1629: done (2 left) 2026-02-17T05:55:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1632: done (1 left) 2026-02-17T05:55:53.942 INFO:tasks.rados.rados.0.trial115.stdout:1633: delete oid 48 current snap is 158 2026-02-17T05:55:53.943 INFO:tasks.rados.rados.0.trial115.stdout:1630: expect (ObjNum 603 snap 158 seq_num 603) 2026-02-17T05:55:54.037 INFO:tasks.rados.rados.0.trial115.stdout:1630: done (1 left) 2026-02-17T05:55:54.037 INFO:tasks.rados.rados.0.trial115.stdout:1633: done (0 left) 2026-02-17T05:55:54.037 INFO:tasks.rados.rados.0.trial115.stdout:1634: delete oid 28 current snap is 158 2026-02-17T05:55:54.038 INFO:tasks.rados.rados.0.trial115.stdout:1634: done (0 left) 2026-02-17T05:55:54.038 INFO:tasks.rados.rados.0.trial115.stdout:1635: rollback oid 19 current snap is 158 2026-02-17T05:55:54.038 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 19 to 154 2026-02-17T05:55:54.038 INFO:tasks.rados.rados.0.trial115.stdout:1636: write oid 38 current snap is 158 2026-02-17T05:55:54.038 INFO:tasks.rados.rados.0.trial115.stdout:1636: seq_num 610 ranges {652270=410130,1529310=493867} 2026-02-17T05:55:54.041 INFO:tasks.rados.rados.0.trial115.stdout:1636: writing trial11545420-38 from 652270 to 1062400 tid 1 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1636: writing trial11545420-38 from 1529310 to 2023177 tid 2 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1637: read oid 9 snap -1 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1637: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1638: write (excl) oid 43 current snap is 158 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1635: finishing rollback tid 0 to trial11545420-19 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1635: finishing rollback tid 1 to trial11545420-19 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1635: finishing rollback tid 2 to trial11545420-19 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 892 (ObjNum 591 snap 152 seq_num 591) dirty exists 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1636: finishing write tid 1 to trial11545420-38 2026-02-17T05:55:54.046 INFO:tasks.rados.rados.0.trial115.stdout:1638: seq_num 611 ranges {669077=485379,1587792=681285} 2026-02-17T05:55:54.049 INFO:tasks.rados.rados.0.trial115.stdout:1638: writing trial11545420-43 from 669077 to 1154456 tid 1 2026-02-17T05:55:54.054 INFO:tasks.rados.rados.0.trial115.stdout:1638: writing trial11545420-43 from 1587792 to 2269077 tid 2 2026-02-17T05:55:54.054 INFO:tasks.rados.rados.0.trial115.stdout:1635: done (3 left) 2026-02-17T05:55:54.054 INFO:tasks.rados.rados.0.trial115.stdout:1639: rollback oid 11 current snap is 158 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 152 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:1636: finishing write tid 2 to trial11545420-38 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:1636: finishing write tid 3 to trial11545420-38 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:1636: finishing write tid 4 to trial11545420-38 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:1636: oid 38 updating version 0 to 896 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:1636: oid 38 version 896 is already newer than 895 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:1636: oid 38 version 896 is already newer than 894 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 896 (ObjNum 610 snap 158 seq_num 610) dirty exists 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:1636: left oid 38 (ObjNum 610 snap 158 seq_num 610) 2026-02-17T05:55:54.055 INFO:tasks.rados.rados.0.trial115.stdout:1637: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: finishing write tid 1 to trial11545420-43 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1636: done (3 left) 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1637: done (2 left) 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1640: read oid 5 snap -1 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1640: expect (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1641: copy_from oid 7 from oid 47 current snap is 158 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1639: finishing rollback tid 1 to trial11545420-11 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1639: finishing rollback tid 2 to trial11545420-11 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 795 (ObjNum 594 snap 152 seq_num 594) dirty exists 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: finishing write tid 2 to trial11545420-43 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: finishing write tid 3 to trial11545420-43 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: finishing write tid 4 to trial11545420-43 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: oid 43 updating version 0 to 621 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: oid 43 version 621 is already newer than 620 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: oid 43 updating version 621 to 622 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 622 (ObjNum 611 snap 158 seq_num 611) dirty exists 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: left oid 43 (ObjNum 611 snap 158 seq_num 611) 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1638: done (3 left) 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1639: done (2 left) 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:1642: rollback oid 17 current snap is 158 2026-02-17T05:55:54.134 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 152 2026-02-17T05:55:54.135 INFO:tasks.rados.rados.0.trial115.stdout:1643: rollback oid 11 current snap is 158 2026-02-17T05:55:54.135 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 152 2026-02-17T05:55:54.135 INFO:tasks.rados.rados.0.trial115.stdout:1644: copy_from oid 44 from oid 25 current snap is 158 2026-02-17T05:55:54.135 INFO:tasks.rados.rados.0.trial115.stdout:1645: copy_from oid 39 from oid 19 current snap is 158 2026-02-17T05:55:54.135 INFO:tasks.rados.rados.0.trial115.stdout:1646: write oid 18 current snap is 158 2026-02-17T05:55:54.135 INFO:tasks.rados.rados.0.trial115.stdout:1646: seq_num 612 ranges {685884=560628,2046274=239610} 2026-02-17T05:55:54.139 INFO:tasks.rados.rados.0.trial115.stdout:1646: writing trial11545420-18 from 685884 to 1246512 tid 1 2026-02-17T05:55:54.144 INFO:tasks.rados.rados.0.trial115.stdout:1646: writing trial11545420-18 from 2046274 to 2285884 tid 2 2026-02-17T05:55:54.144 INFO:tasks.rados.rados.0.trial115.stdout:1647: read oid 41 snap -1 2026-02-17T05:55:54.144 INFO:tasks.rados.rados.0.trial115.stdout:1647: expect deleted 2026-02-17T05:55:54.144 INFO:tasks.rados.rados.0.trial115.stdout:1648: read oid 24 snap -1 2026-02-17T05:55:54.144 INFO:tasks.rados.rados.0.trial115.stdout:1648: expect (ObjNum 569 snap 147 seq_num 569) 2026-02-17T05:55:54.144 INFO:tasks.rados.rados.0.trial115.stdout:1649: delete oid 4 current snap is 158 2026-02-17T05:55:54.144 INFO:tasks.rados.rados.0.trial115.stdout:1643: finishing rollback tid 0 to trial11545420-11 2026-02-17T05:55:54.144 INFO:tasks.rados.rados.0.trial115.stdout:1640: expect (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:55:54.207 INFO:tasks.rados.rados.0.trial115.stdout:1641: finishing copy_from to trial11545420-7 2026-02-17T05:55:54.207 INFO:tasks.rados.rados.0.trial115.stdout:1641: got expected ENOENT (src dne) 2026-02-17T05:55:54.208 INFO:tasks.rados.rados.0.trial115.stdout:1641: finishing copy_from racing read to trial11545420-7 2026-02-17T05:55:54.208 INFO:tasks.rados.rados.0.trial115.stdout:1643: finishing rollback tid 1 to trial11545420-11 2026-02-17T05:55:54.208 INFO:tasks.rados.rados.0.trial115.stdout:1640: done (9 left) 2026-02-17T05:55:54.208 INFO:tasks.rados.rados.0.trial115.stdout:1641: done (8 left) 2026-02-17T05:55:54.208 INFO:tasks.rados.rados.0.trial115.stdout:1649: done (7 left) 2026-02-17T05:55:54.208 INFO:tasks.rados.rados.0.trial115.stdout:1650: write oid 5 current snap is 158 2026-02-17T05:55:54.208 INFO:tasks.rados.rados.0.trial115.stdout:1650: seq_num 614 ranges {719498=711126,2046885=272613} 2026-02-17T05:55:54.212 INFO:tasks.rados.rados.0.trial115.stdout:1650: writing trial11545420-5 from 719498 to 1430624 tid 1 2026-02-17T05:55:54.216 INFO:tasks.rados.rados.0.trial115.stdout:1650: writing trial11545420-5 from 2046885 to 2319498 tid 2 2026-02-17T05:55:54.216 INFO:tasks.rados.rados.0.trial115.stdout:1651: snap_create 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1643: finishing rollback tid 2 to trial11545420-11 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 798 (ObjNum 594 snap 152 seq_num 594) dirty exists 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1645: finishing copy_from racing read to trial11545420-39 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1646: finishing write tid 1 to trial11545420-18 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1642: finishing rollback tid 0 to trial11545420-17 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1642: finishing rollback tid 1 to trial11545420-17 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1642: finishing rollback tid 2 to trial11545420-17 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 337 (ObjNum 551 snap 142 seq_num 551) dirty exists 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1646: finishing write tid 2 to trial11545420-18 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1644: finishing copy_from racing read to trial11545420-44 2026-02-17T05:55:54.217 INFO:tasks.rados.rados.0.trial115.stdout:1646: finishing write tid 3 to trial11545420-18 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:1646: finishing write tid 4 to trial11545420-18 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:1646: oid 18 updating version 0 to 854 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:1646: oid 18 updating version 854 to 855 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:1646: oid 18 version 855 is already newer than 853 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 855 (ObjNum 612 snap 158 seq_num 612) dirty exists 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:1646: left oid 18 (ObjNum 612 snap 158 seq_num 612) 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:1644: finishing copy_from to trial11545420-44 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 852 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:1645: finishing copy_from to trial11545420-39 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 892 (ObjNum 591 snap 152 seq_num 591) dirty exists 2026-02-17T05:55:54.218 INFO:tasks.rados.rados.0.trial115.stdout:1648: expect (ObjNum 569 snap 147 seq_num 569) 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:1650: finishing write tid 1 to trial11545420-5 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:1650: finishing write tid 2 to trial11545420-5 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:1650: finishing write tid 3 to trial11545420-5 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:1650: finishing write tid 4 to trial11545420-5 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:1650: oid 5 updating version 0 to 807 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:1650: oid 5 version 807 is already newer than 805 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:1650: oid 5 version 807 is already newer than 806 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 807 (ObjNum 614 snap 158 seq_num 614) dirty exists 2026-02-17T05:55:54.379 INFO:tasks.rados.rados.0.trial115.stdout:1650: left oid 5 (ObjNum 614 snap 158 seq_num 614) 2026-02-17T05:55:54.634 INFO:tasks.rados.rados.0.trial115.stdout:1642: done (8 left) 2026-02-17T05:55:54.634 INFO:tasks.rados.rados.0.trial115.stdout:1643: done (7 left) 2026-02-17T05:55:54.634 INFO:tasks.rados.rados.0.trial115.stdout:1644: done (6 left) 2026-02-17T05:55:54.634 INFO:tasks.rados.rados.0.trial115.stdout:1645: done (5 left) 2026-02-17T05:55:54.634 INFO:tasks.rados.rados.0.trial115.stdout:1646: done (4 left) 2026-02-17T05:55:54.634 INFO:tasks.rados.rados.0.trial115.stdout:1647: done (3 left) 2026-02-17T05:55:54.634 INFO:tasks.rados.rados.0.trial115.stdout:1648: done (2 left) 2026-02-17T05:55:54.634 INFO:tasks.rados.rados.0.trial115.stdout:1650: done (1 left) 2026-02-17T05:55:54.635 INFO:tasks.rados.rados.0.trial115.stdout:1651: done (0 left) 2026-02-17T05:55:54.635 INFO:tasks.rados.rados.0.trial115.stdout:1652: read oid 30 snap -1 2026-02-17T05:55:54.635 INFO:tasks.rados.rados.0.trial115.stdout:1652: expect (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:55:54.635 INFO:tasks.rados.rados.0.trial115.stdout:1653: snap_create 2026-02-17T05:55:54.638 INFO:tasks.rados.rados.0.trial115.stdout:1652: expect (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:55:55.634 INFO:tasks.rados.rados.0.trial115.stdout:1652: done (1 left) 2026-02-17T05:55:55.634 INFO:tasks.rados.rados.0.trial115.stdout:1653: done (0 left) 2026-02-17T05:55:55.634 INFO:tasks.rados.rados.0.trial115.stdout:1654: snap_create 2026-02-17T05:55:56.634 INFO:tasks.rados.rados.0.trial115.stdout:1654: done (0 left) 2026-02-17T05:55:56.634 INFO:tasks.rados.rados.0.trial115.stdout:1655: copy_from oid 49 from oid 5 current snap is 161 2026-02-17T05:55:56.634 INFO:tasks.rados.rados.0.trial115.stdout:1656: write oid 24 current snap is 161 2026-02-17T05:55:56.634 INFO:tasks.rados.rados.0.trial115.stdout:1656: seq_num 615 ranges {736305=786375,2105367=230938} 2026-02-17T05:55:56.640 INFO:tasks.rados.rados.0.trial115.stdout:1656: writing trial11545420-24 from 736305 to 1522680 tid 1 2026-02-17T05:55:56.643 INFO:tasks.rados.rados.0.trial115.stdout:1656: writing trial11545420-24 from 2105367 to 2336305 tid 2 2026-02-17T05:55:56.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: write (excl) oid 6 current snap is 161 2026-02-17T05:55:56.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: seq_num 616 ranges {753112=577977,1880202=472910} 2026-02-17T05:55:56.649 INFO:tasks.rados.rados.0.trial115.stdout:1657: writing trial11545420-6 from 753112 to 1331089 tid 1 2026-02-17T05:55:56.653 INFO:tasks.rados.rados.0.trial115.stdout:1657: writing trial11545420-6 from 1880202 to 2353112 tid 2 2026-02-17T05:55:56.653 INFO:tasks.rados.rados.0.trial115.stdout:1658: write oid 42 current snap is 161 2026-02-17T05:55:56.653 INFO:tasks.rados.rados.0.trial115.stdout:1655: finishing copy_from racing read to trial11545420-49 2026-02-17T05:55:56.653 INFO:tasks.rados.rados.0.trial115.stdout:1655: finishing copy_from to trial11545420-49 2026-02-17T05:55:56.653 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 807 (ObjNum 614 snap 158 seq_num 614) dirty exists 2026-02-17T05:55:56.653 INFO:tasks.rados.rados.0.trial115.stdout:1656: finishing write tid 1 to trial11545420-24 2026-02-17T05:55:56.653 INFO:tasks.rados.rados.0.trial115.stdout:1658: seq_num 617 ranges {769919=653226,1938684=431235} 2026-02-17T05:55:56.658 INFO:tasks.rados.rados.0.trial115.stdout:1658: writing trial11545420-42 from 769919 to 1423145 tid 1 2026-02-17T05:55:56.661 INFO:tasks.rados.rados.0.trial115.stdout:1658: writing trial11545420-42 from 1938684 to 2369919 tid 2 2026-02-17T05:55:56.662 INFO:tasks.rados.rados.0.trial115.stdout:1655: done (3 left) 2026-02-17T05:55:56.662 INFO:tasks.rados.rados.0.trial115.stdout:1659: read oid 4 snap -1 2026-02-17T05:55:56.662 INFO:tasks.rados.rados.0.trial115.stdout:1659: expect deleted 2026-02-17T05:55:56.662 INFO:tasks.rados.rados.0.trial115.stdout:1660: read oid 2 snap 154 2026-02-17T05:55:56.662 INFO:tasks.rados.rados.0.trial115.stdout:1660: expect deleted 2026-02-17T05:55:56.662 INFO:tasks.rados.rados.0.trial115.stdout:1661: snap_remove snap 160 2026-02-17T05:55:57.642 INFO:tasks.rados.rados.0.trial115.stdout:1661: done (5 left) 2026-02-17T05:55:57.642 INFO:tasks.rados.rados.0.trial115.stdout:1662: snap_create 2026-02-17T05:55:57.642 INFO:tasks.rados.rados.0.trial115.stdout:1656: finishing write tid 2 to trial11545420-24 2026-02-17T05:55:57.642 INFO:tasks.rados.rados.0.trial115.stdout:1656: finishing write tid 3 to trial11545420-24 2026-02-17T05:55:57.642 INFO:tasks.rados.rados.0.trial115.stdout:1656: finishing write tid 4 to trial11545420-24 2026-02-17T05:55:57.642 INFO:tasks.rados.rados.0.trial115.stdout:1656: oid 24 updating version 0 to 926 2026-02-17T05:55:57.642 INFO:tasks.rados.rados.0.trial115.stdout:1656: oid 24 version 926 is already newer than 925 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1656: oid 24 version 926 is already newer than 924 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 926 (ObjNum 615 snap 161 seq_num 615) dirty exists 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1656: left oid 24 (ObjNum 615 snap 161 seq_num 615) 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: finishing write tid 1 to trial11545420-6 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: finishing write tid 2 to trial11545420-6 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: finishing write tid 3 to trial11545420-6 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: finishing write tid 4 to trial11545420-6 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: oid 6 updating version 0 to 646 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: oid 6 version 646 is already newer than 644 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: oid 6 version 646 is already newer than 645 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 646 (ObjNum 616 snap 161 seq_num 616) dirty exists 2026-02-17T05:55:57.643 INFO:tasks.rados.rados.0.trial115.stdout:1657: left oid 6 (ObjNum 616 snap 161 seq_num 616) 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:1658: finishing write tid 1 to trial11545420-42 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:1658: finishing write tid 2 to trial11545420-42 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:1658: finishing write tid 3 to trial11545420-42 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:1658: finishing write tid 4 to trial11545420-42 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:1658: oid 42 updating version 0 to 929 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:1658: oid 42 updating version 929 to 930 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:1658: oid 42 version 930 is already newer than 928 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 930 (ObjNum 617 snap 161 seq_num 617) dirty exists 2026-02-17T05:55:57.644 INFO:tasks.rados.rados.0.trial115.stdout:1658: left oid 42 (ObjNum 617 snap 161 seq_num 617) 2026-02-17T05:55:58.971 INFO:tasks.rados.rados.0.trial115.stdout:1656: done (5 left) 2026-02-17T05:55:58.971 INFO:tasks.rados.rados.0.trial115.stdout:1657: done (4 left) 2026-02-17T05:55:58.971 INFO:tasks.rados.rados.0.trial115.stdout:1658: done (3 left) 2026-02-17T05:55:58.971 INFO:tasks.rados.rados.0.trial115.stdout:1659: done (2 left) 2026-02-17T05:55:58.971 INFO:tasks.rados.rados.0.trial115.stdout:1660: done (1 left) 2026-02-17T05:55:58.971 INFO:tasks.rados.rados.0.trial115.stdout:1662: done (0 left) 2026-02-17T05:55:58.971 INFO:tasks.rados.rados.0.trial115.stdout:1663: write (excl) oid 39 current snap is 162 2026-02-17T05:55:58.971 INFO:tasks.rados.rados.0.trial115.stdout:1663: seq_num 618 ranges {786726=728475,2280813=105913} 2026-02-17T05:55:58.976 INFO:tasks.rados.rados.0.trial115.stdout:1663: writing trial11545420-39 from 786726 to 1515201 tid 1 2026-02-17T05:55:58.979 INFO:tasks.rados.rados.0.trial115.stdout:1663: writing trial11545420-39 from 2280813 to 2386726 tid 2 2026-02-17T05:55:58.980 INFO:tasks.rados.rados.0.trial115.stdout:1664: copy_from oid 11 from oid 17 current snap is 162 2026-02-17T05:55:58.980 INFO:tasks.rados.rados.0.trial115.stdout:1665: read oid 50 snap -1 2026-02-17T05:55:58.980 INFO:tasks.rados.rados.0.trial115.stdout:1665: expect (ObjNum 587 snap 151 seq_num 587) 2026-02-17T05:55:58.980 INFO:tasks.rados.rados.0.trial115.stdout:1666: rollback oid 5 current snap is 162 2026-02-17T05:55:58.980 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 5 to 152 2026-02-17T05:55:58.980 INFO:tasks.rados.rados.0.trial115.stdout:1667: snap_remove snap 159 2026-02-17T05:55:59.972 INFO:tasks.rados.rados.0.trial115.stdout:1667: done (4 left) 2026-02-17T05:55:59.972 INFO:tasks.rados.rados.0.trial115.stdout:1668: rollback oid 31 current snap is 162 2026-02-17T05:55:59.972 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 31 to 139 2026-02-17T05:55:59.972 INFO:tasks.rados.rados.0.trial115.stdout:1663: finishing write tid 1 to trial11545420-39 2026-02-17T05:55:59.972 INFO:tasks.rados.rados.0.trial115.stdout:1666: finishing rollback tid 0 to trial11545420-5 2026-02-17T05:55:59.972 INFO:tasks.rados.rados.0.trial115.stdout:1663: finishing write tid 2 to trial11545420-39 2026-02-17T05:55:59.972 INFO:tasks.rados.rados.0.trial115.stdout:1663: finishing write tid 3 to trial11545420-39 2026-02-17T05:55:59.972 INFO:tasks.rados.rados.0.trial115.stdout:1663: finishing write tid 4 to trial11545420-39 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:1663: oid 39 updating version 0 to 937 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:1663: oid 39 updating version 937 to 938 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:1663: oid 39 version 938 is already newer than 932 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 938 (ObjNum 618 snap 162 seq_num 618) dirty exists 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:1663: left oid 39 (ObjNum 618 snap 162 seq_num 618) 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:1666: finishing rollback tid 1 to trial11545420-5 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:1666: finishing rollback tid 2 to trial11545420-5 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 811 (ObjNum 516 snap 132 seq_num 516) dirty exists 2026-02-17T05:55:59.973 INFO:tasks.rados.rados.0.trial115.stdout:1665: expect (ObjNum 587 snap 151 seq_num 587) 2026-02-17T05:56:00.066 INFO:tasks.rados.rados.0.trial115.stdout:1664: finishing copy_from racing read to trial11545420-11 2026-02-17T05:56:00.066 INFO:tasks.rados.rados.0.trial115.stdout:1664: finishing copy_from to trial11545420-11 2026-02-17T05:56:00.067 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 337 (ObjNum 551 snap 142 seq_num 551) dirty exists 2026-02-17T05:56:00.067 INFO:tasks.rados.rados.0.trial115.stdout:1668: finishing rollback tid 0 to trial11545420-31 2026-02-17T05:56:00.067 INFO:tasks.rados.rados.0.trial115.stdout:1668: finishing rollback tid 1 to trial11545420-31 2026-02-17T05:56:00.067 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 814 (ObjNum 113 snap 0 seq_num 1333803680) dirty dne 2026-02-17T05:56:00.067 INFO:tasks.rados.rados.0.trial115.stdout:1663: done (4 left) 2026-02-17T05:56:00.067 INFO:tasks.rados.rados.0.trial115.stdout:1664: done (3 left) 2026-02-17T05:56:00.067 INFO:tasks.rados.rados.0.trial115.stdout:1665: done (2 left) 2026-02-17T05:56:00.068 INFO:tasks.rados.rados.0.trial115.stdout:1666: done (1 left) 2026-02-17T05:56:00.068 INFO:tasks.rados.rados.0.trial115.stdout:1668: done (0 left) 2026-02-17T05:56:00.068 INFO:tasks.rados.rados.0.trial115.stdout:1669: snap_remove snap 154 2026-02-17T05:56:00.974 INFO:tasks.rados.rados.0.trial115.stdout:1669: done (0 left) 2026-02-17T05:56:00.975 INFO:tasks.rados.rados.0.trial115.stdout:1670: copy_from oid 20 from oid 4 current snap is 162 2026-02-17T05:56:00.975 INFO:tasks.rados.rados.0.trial115.stdout:1671: read oid 29 snap -1 2026-02-17T05:56:00.975 INFO:tasks.rados.rados.0.trial115.stdout:1671: expect (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:56:00.975 INFO:tasks.rados.rados.0.trial115.stdout:1672: copy_from oid 38 from oid 40 current snap is 162 2026-02-17T05:56:00.975 INFO:tasks.rados.rados.0.trial115.stdout:1673: write (excl) oid 17 current snap is 162 2026-02-17T05:56:00.975 INFO:tasks.rados.rados.0.trial115.stdout:1673: seq_num 619 ranges {403533=403724,1539295=652201} 2026-02-17T05:56:00.978 INFO:tasks.rados.rados.0.trial115.stdout:1673: writing trial11545420-17 from 403533 to 807257 tid 1 2026-02-17T05:56:00.983 INFO:tasks.rados.rados.0.trial115.stdout:1673: writing trial11545420-17 from 1539295 to 2191496 tid 2 2026-02-17T05:56:00.983 INFO:tasks.rados.rados.0.trial115.stdout:1674: write oid 44 current snap is 162 2026-02-17T05:56:00.983 INFO:tasks.rados.rados.0.trial115.stdout:1674: seq_num 620 ranges {420340=478973,1597777=512212} 2026-02-17T05:56:00.986 INFO:tasks.rados.rados.0.trial115.stdout:1674: writing trial11545420-44 from 420340 to 899313 tid 1 2026-02-17T05:56:00.991 INFO:tasks.rados.rados.0.trial115.stdout:1674: writing trial11545420-44 from 1597777 to 2109989 tid 2 2026-02-17T05:56:00.991 INFO:tasks.rados.rados.0.trial115.stdout:1675: write oid 35 current snap is 162 2026-02-17T05:56:00.991 INFO:tasks.rados.rados.0.trial115.stdout:1675: seq_num 621 ranges {437147=554222,1656259=655870} 2026-02-17T05:56:00.994 INFO:tasks.rados.rados.0.trial115.stdout:1675: writing trial11545420-35 from 437147 to 991369 tid 1 2026-02-17T05:56:01.000 INFO:tasks.rados.rados.0.trial115.stdout:1675: writing trial11545420-35 from 1656259 to 2312129 tid 2 2026-02-17T05:56:01.000 INFO:tasks.rados.rados.0.trial115.stdout:1676: read oid 8 snap -1 2026-02-17T05:56:01.000 INFO:tasks.rados.rados.0.trial115.stdout:1676: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:56:01.000 INFO:tasks.rados.rados.0.trial115.stdout:1677: write oid 46 current snap is 162 2026-02-17T05:56:01.000 INFO:tasks.rados.rados.0.trial115.stdout:1677: seq_num 622 ranges {453954=629471,1598388=515881} 2026-02-17T05:56:01.004 INFO:tasks.rados.rados.0.trial115.stdout:1677: writing trial11545420-46 from 453954 to 1083425 tid 1 2026-02-17T05:56:01.008 INFO:tasks.rados.rados.0.trial115.stdout:1677: writing trial11545420-46 from 1598388 to 2114269 tid 2 2026-02-17T05:56:01.008 INFO:tasks.rados.rados.0.trial115.stdout:1678: delete oid 23 current snap is 162 2026-02-17T05:56:01.009 INFO:tasks.rados.rados.0.trial115.stdout:1670: finishing copy_from to trial11545420-20 2026-02-17T05:56:01.009 INFO:tasks.rados.rados.0.trial115.stdout:1670: got expected ENOENT (src dne) 2026-02-17T05:56:01.009 INFO:tasks.rados.rados.0.trial115.stdout:1672: finishing copy_from to trial11545420-38 2026-02-17T05:56:01.009 INFO:tasks.rados.rados.0.trial115.stdout:1672: got expected ENOENT (src dne) 2026-02-17T05:56:01.009 INFO:tasks.rados.rados.0.trial115.stdout:1670: finishing copy_from racing read to trial11545420-20 2026-02-17T05:56:01.009 INFO:tasks.rados.rados.0.trial115.stdout:1672: finishing copy_from racing read to trial11545420-38 2026-02-17T05:56:01.009 INFO:tasks.rados.rados.0.trial115.stdout:1671: expect (ObjNum 536 snap 136 seq_num 536) 2026-02-17T05:56:01.081 INFO:tasks.rados.rados.0.trial115.stdout:1673: finishing write tid 1 to trial11545420-17 2026-02-17T05:56:01.081 INFO:tasks.rados.rados.0.trial115.stdout:1673: finishing write tid 2 to trial11545420-17 2026-02-17T05:56:01.081 INFO:tasks.rados.rados.0.trial115.stdout:1673: finishing write tid 3 to trial11545420-17 2026-02-17T05:56:01.081 INFO:tasks.rados.rados.0.trial115.stdout:1673: finishing write tid 4 to trial11545420-17 2026-02-17T05:56:01.081 INFO:tasks.rados.rados.0.trial115.stdout:1673: oid 17 updating version 0 to 343 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1673: oid 17 updating version 343 to 345 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1673: oid 17 version 345 is already newer than 344 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 345 (ObjNum 619 snap 162 seq_num 619) dirty exists 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1673: left oid 17 (ObjNum 619 snap 162 seq_num 619) 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1674: finishing write tid 1 to trial11545420-44 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1674: finishing write tid 2 to trial11545420-44 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1674: finishing write tid 3 to trial11545420-44 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1670: done (8 left) 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1671: done (7 left) 2026-02-17T05:56:01.082 INFO:tasks.rados.rados.0.trial115.stdout:1672: done (6 left) 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1673: done (5 left) 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1678: done (4 left) 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1679: rollback oid 36 current snap is 162 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 36 to 157 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1674: finishing write tid 4 to trial11545420-44 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1674: oid 44 updating version 0 to 855 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1674: oid 44 version 855 is already newer than 854 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1674: oid 44 version 855 is already newer than 853 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 855 (ObjNum 620 snap 162 seq_num 620) dirty exists 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1674: left oid 44 (ObjNum 620 snap 162 seq_num 620) 2026-02-17T05:56:01.083 INFO:tasks.rados.rados.0.trial115.stdout:1675: finishing write tid 1 to trial11545420-35 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:1675: finishing write tid 2 to trial11545420-35 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:1675: finishing write tid 3 to trial11545420-35 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:1675: finishing write tid 4 to trial11545420-35 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:1675: oid 35 updating version 0 to 905 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:1675: oid 35 version 905 is already newer than 904 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:1675: oid 35 updating version 905 to 906 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 906 (ObjNum 621 snap 162 seq_num 621) dirty exists 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:1675: left oid 35 (ObjNum 621 snap 162 seq_num 621) 2026-02-17T05:56:01.084 INFO:tasks.rados.rados.0.trial115.stdout:1676: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:56:01.305 INFO:tasks.rados.rados.0.trial115.stdout:1677: finishing write tid 1 to trial11545420-46 2026-02-17T05:56:01.306 INFO:tasks.rados.rados.0.trial115.stdout:1674: done (4 left) 2026-02-17T05:56:01.306 INFO:tasks.rados.rados.0.trial115.stdout:1675: done (3 left) 2026-02-17T05:56:01.306 INFO:tasks.rados.rados.0.trial115.stdout:1676: done (2 left) 2026-02-17T05:56:01.306 INFO:tasks.rados.rados.0.trial115.stdout:1680: read oid 37 snap -1 2026-02-17T05:56:01.306 INFO:tasks.rados.rados.0.trial115.stdout:1680: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:01.307 INFO:tasks.rados.rados.0.trial115.stdout:1681: rollback oid 39 current snap is 162 2026-02-17T05:56:01.307 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 39 to 152 2026-02-17T05:56:01.307 INFO:tasks.rados.rados.0.trial115.stdout:1677: finishing write tid 2 to trial11545420-46 2026-02-17T05:56:01.307 INFO:tasks.rados.rados.0.trial115.stdout:1677: finishing write tid 3 to trial11545420-46 2026-02-17T05:56:01.307 INFO:tasks.rados.rados.0.trial115.stdout:1677: finishing write tid 4 to trial11545420-46 2026-02-17T05:56:01.307 INFO:tasks.rados.rados.0.trial115.stdout:1677: oid 46 updating version 0 to 809 2026-02-17T05:56:01.308 INFO:tasks.rados.rados.0.trial115.stdout:1677: oid 46 version 809 is already newer than 808 2026-02-17T05:56:01.308 INFO:tasks.rados.rados.0.trial115.stdout:1677: oid 46 version 809 is already newer than 807 2026-02-17T05:56:01.308 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 809 (ObjNum 622 snap 162 seq_num 622) dirty exists 2026-02-17T05:56:01.308 INFO:tasks.rados.rados.0.trial115.stdout:1677: left oid 46 (ObjNum 622 snap 162 seq_num 622) 2026-02-17T05:56:01.308 INFO:tasks.rados.rados.0.trial115.stdout:1679: finishing rollback tid 1 to trial11545420-36 2026-02-17T05:56:01.308 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 0 (ObjNum 225 snap 0 seq_num 1333621792) dirty dne 2026-02-17T05:56:01.309 INFO:tasks.rados.rados.0.trial115.stdout:1677: done (3 left) 2026-02-17T05:56:01.309 INFO:tasks.rados.rados.0.trial115.stdout:1679: done (2 left) 2026-02-17T05:56:01.309 INFO:tasks.rados.rados.0.trial115.stdout:1682: write oid 23 current snap is 162 2026-02-17T05:56:01.309 INFO:tasks.rados.rados.0.trial115.stdout:1682: seq_num 624 ranges {487568=496322,1431705=519550} 2026-02-17T05:56:01.310 INFO:tasks.rados.rados.0.trial115.stdout:1682: writing trial11545420-23 from 487568 to 983890 tid 1 2026-02-17T05:56:01.312 INFO:tasks.rados.rados.0.trial115.stdout:1682: writing trial11545420-23 from 1431705 to 1951255 tid 2 2026-02-17T05:56:01.313 INFO:tasks.rados.rados.0.trial115.stdout:1683: read oid 38 snap -1 2026-02-17T05:56:01.313 INFO:tasks.rados.rados.0.trial115.stdout:1683: expect (ObjNum 610 snap 158 seq_num 610) 2026-02-17T05:56:01.313 INFO:tasks.rados.rados.0.trial115.stdout:1684: copy_from oid 22 from oid 26 current snap is 162 2026-02-17T05:56:01.313 INFO:tasks.rados.rados.0.trial115.stdout:1681: finishing rollback tid 0 to trial11545420-39 2026-02-17T05:56:01.313 INFO:tasks.rados.rados.0.trial115.stdout:1681: finishing rollback tid 1 to trial11545420-39 2026-02-17T05:56:01.313 INFO:tasks.rados.rados.0.trial115.stdout:1681: finishing rollback tid 2 to trial11545420-39 2026-02-17T05:56:01.314 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 953 (ObjNum 190 snap 33 seq_num 190) dirty exists 2026-02-17T05:56:01.314 INFO:tasks.rados.rados.0.trial115.stdout:1680: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:01.390 INFO:tasks.rados.rados.0.trial115.stdout:1682: finishing write tid 1 to trial11545420-23 2026-02-17T05:56:01.390 INFO:tasks.rados.rados.0.trial115.stdout:1682: finishing write tid 2 to trial11545420-23 2026-02-17T05:56:01.390 INFO:tasks.rados.rados.0.trial115.stdout:1680: done (4 left) 2026-02-17T05:56:01.390 INFO:tasks.rados.rados.0.trial115.stdout:1681: done (3 left) 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:1685: delete oid 34 current snap is 162 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:1682: finishing write tid 3 to trial11545420-23 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:1682: finishing write tid 4 to trial11545420-23 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:1682: oid 23 updating version 0 to 349 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:1682: oid 23 updating version 349 to 350 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:1682: oid 23 version 350 is already newer than 348 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 350 (ObjNum 624 snap 162 seq_num 624) dirty exists 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:1682: left oid 23 (ObjNum 624 snap 162 seq_num 624) 2026-02-17T05:56:01.391 INFO:tasks.rados.rados.0.trial115.stdout:1683: expect (ObjNum 610 snap 158 seq_num 610) 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:1684: finishing copy_from racing read to trial11545420-22 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:1684: finishing copy_from to trial11545420-22 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 210 (ObjNum 550 snap 142 seq_num 550) dirty exists 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:1682: done (3 left) 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:1683: done (2 left) 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:1684: done (1 left) 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:1685: done (0 left) 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:1686: copy_from oid 26 from oid 37 current snap is 162 2026-02-17T05:56:01.475 INFO:tasks.rados.rados.0.trial115.stdout:1687: snap_remove snap 152 2026-02-17T05:56:02.161 INFO:tasks.rados.rados.0.trial115.stdout:1687: done (1 left) 2026-02-17T05:56:02.161 INFO:tasks.rados.rados.0.trial115.stdout:1688: delete oid 1 current snap is 162 2026-02-17T05:56:02.161 INFO:tasks.rados.rados.0.trial115.stdout:1686: finishing copy_from racing read to trial11545420-26 2026-02-17T05:56:02.161 INFO:tasks.rados.rados.0.trial115.stdout:1686: finishing copy_from to trial11545420-26 2026-02-17T05:56:02.161 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 607 (ObjNum 599 snap 155 seq_num 599) dirty exists 2026-02-17T05:56:02.164 INFO:tasks.rados.rados.0.trial115.stdout:1686: done (1 left) 2026-02-17T05:56:02.165 INFO:tasks.rados.rados.0.trial115.stdout:1688: done (0 left) 2026-02-17T05:56:02.165 INFO:tasks.rados.rados.0.trial115.stdout:1689: read oid 19 snap 157 2026-02-17T05:56:02.165 INFO:tasks.rados.rados.0.trial115.stdout:1689: expect (ObjNum 591 snap 152 seq_num 591) 2026-02-17T05:56:02.165 INFO:tasks.rados.rados.0.trial115.stdout:1690: copy_from oid 41 from oid 21 current snap is 162 2026-02-17T05:56:02.165 INFO:tasks.rados.rados.0.trial115.stdout:1691: delete oid 17 current snap is 162 2026-02-17T05:56:02.165 INFO:tasks.rados.rados.0.trial115.stdout:1690: finishing copy_from to trial11545420-41 2026-02-17T05:56:02.166 INFO:tasks.rados.rados.0.trial115.stdout:1690: got expected ENOENT (src dne) 2026-02-17T05:56:02.166 INFO:tasks.rados.rados.0.trial115.stdout:1690: finishing copy_from racing read to trial11545420-41 2026-02-17T05:56:02.166 INFO:tasks.rados.rados.0.trial115.stdout:1690: done (2 left) 2026-02-17T05:56:02.166 INFO:tasks.rados.rados.0.trial115.stdout:1691: done (1 left) 2026-02-17T05:56:02.166 INFO:tasks.rados.rados.0.trial115.stdout:1692: read oid 4 snap -1 2026-02-17T05:56:02.166 INFO:tasks.rados.rados.0.trial115.stdout:1692: expect deleted 2026-02-17T05:56:02.166 INFO:tasks.rados.rados.0.trial115.stdout:1693: snap_create 2026-02-17T05:56:02.172 INFO:tasks.rados.rados.0.trial115.stdout:1689: expect (ObjNum 591 snap 152 seq_num 591) 2026-02-17T05:56:03.164 INFO:tasks.rados.rados.0.trial115.stdout:1689: done (2 left) 2026-02-17T05:56:03.164 INFO:tasks.rados.rados.0.trial115.stdout:1692: done (1 left) 2026-02-17T05:56:03.164 INFO:tasks.rados.rados.0.trial115.stdout:1693: done (0 left) 2026-02-17T05:56:03.164 INFO:tasks.rados.rados.0.trial115.stdout:1694: snap_create 2026-02-17T05:56:03.539 INFO:tasks.rados.rados.0.trial115.stdout:1694: done (0 left) 2026-02-17T05:56:03.540 INFO:tasks.rados.rados.0.trial115.stdout:1695: write (excl) oid 1 current snap is 164 2026-02-17T05:56:03.540 INFO:tasks.rados.rados.0.trial115.stdout:1695: seq_num 628 ranges {554796=797318,1949280=526888} 2026-02-17T05:56:03.544 INFO:tasks.rados.rados.0.trial115.stdout:1695: writing trial11545420-1 from 554796 to 1352114 tid 1 2026-02-17T05:56:03.549 INFO:tasks.rados.rados.0.trial115.stdout:1695: writing trial11545420-1 from 1949280 to 2476168 tid 2 2026-02-17T05:56:03.549 INFO:tasks.rados.rados.0.trial115.stdout:1696: read oid 28 snap -1 2026-02-17T05:56:03.549 INFO:tasks.rados.rados.0.trial115.stdout:1696: expect deleted 2026-02-17T05:56:03.549 INFO:tasks.rados.rados.0.trial115.stdout:1697: read oid 37 snap 158 2026-02-17T05:56:03.549 INFO:tasks.rados.rados.0.trial115.stdout:1697: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:03.550 INFO:tasks.rados.rados.0.trial115.stdout:1698: write oid 12 current snap is 164 2026-02-17T05:56:03.550 INFO:tasks.rados.rados.0.trial115.stdout:1698: seq_num 629 ranges {571603=472567,1607762=670546} 2026-02-17T05:56:03.552 INFO:tasks.rados.rados.0.trial115.stdout:1698: writing trial11545420-12 from 571603 to 1044170 tid 1 2026-02-17T05:56:03.558 INFO:tasks.rados.rados.0.trial115.stdout:1698: writing trial11545420-12 from 1607762 to 2278308 tid 2 2026-02-17T05:56:03.558 INFO:tasks.rados.rados.0.trial115.stdout:1699: write (excl) oid 15 current snap is 164 2026-02-17T05:56:03.558 INFO:tasks.rados.rados.0.trial115.stdout:1699: seq_num 630 ranges {588410=547816,1666244=414204,2484603=103807} 2026-02-17T05:56:03.561 INFO:tasks.rados.rados.0.trial115.stdout:1699: writing trial11545420-15 from 588410 to 1136226 tid 1 2026-02-17T05:56:03.565 INFO:tasks.rados.rados.0.trial115.stdout:1699: writing trial11545420-15 from 1666244 to 2080448 tid 2 2026-02-17T05:56:03.567 INFO:tasks.rados.rados.0.trial115.stdout:1699: writing trial11545420-15 from 2484603 to 2588410 tid 3 2026-02-17T05:56:03.567 INFO:tasks.rados.rados.0.trial115.stdout:1700: write oid 13 current snap is 164 2026-02-17T05:56:03.567 INFO:tasks.rados.rados.0.trial115.stdout:1700: seq_num 631 ranges {605217=739418,2124726=480491} 2026-02-17T05:56:03.572 INFO:tasks.rados.rados.0.trial115.stdout:1700: writing trial11545420-13 from 605217 to 1344635 tid 1 2026-02-17T05:56:03.577 INFO:tasks.rados.rados.0.trial115.stdout:1700: writing trial11545420-13 from 2124726 to 2605217 tid 2 2026-02-17T05:56:03.577 INFO:tasks.rados.rados.0.trial115.stdout:1701: read oid 50 snap 139 2026-02-17T05:56:03.577 INFO:tasks.rados.rados.0.trial115.stdout:1701: expect (ObjNum 522 snap 135 seq_num 522) 2026-02-17T05:56:03.577 INFO:tasks.rados.rados.0.trial115.stdout:1702: write (excl) oid 40 current snap is 164 2026-02-17T05:56:03.578 INFO:tasks.rados.rados.0.trial115.stdout:1702: seq_num 632 ranges {622024=414667,1783208=417873} 2026-02-17T05:56:03.580 INFO:tasks.rados.rados.0.trial115.stdout:1702: writing trial11545420-40 from 622024 to 1036691 tid 1 2026-02-17T05:56:03.585 INFO:tasks.rados.rados.0.trial115.stdout:1702: writing trial11545420-40 from 1783208 to 2201081 tid 2 2026-02-17T05:56:03.585 INFO:tasks.rados.rados.0.trial115.stdout:1703: write oid 39 current snap is 164 2026-02-17T05:56:03.586 INFO:tasks.rados.rados.0.trial115.stdout:1703: seq_num 633 ranges {638831=489916,1841690=677884} 2026-02-17T05:56:03.589 INFO:tasks.rados.rados.0.trial115.stdout:1703: writing trial11545420-39 from 638831 to 1128747 tid 1 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1703: writing trial11545420-39 from 1841690 to 2519574 tid 2 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1704: read oid 8 snap -1 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1704: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1705: snap_create 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1695: finishing write tid 1 to trial11545420-1 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1695: finishing write tid 2 to trial11545420-1 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1695: finishing write tid 3 to trial11545420-1 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1695: finishing write tid 4 to trial11545420-1 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1695: oid 1 updating version 0 to 630 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1695: oid 1 version 630 is already newer than 629 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1695: oid 1 version 630 is already newer than 628 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 630 (ObjNum 628 snap 164 seq_num 628) dirty exists 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1695: left oid 1 (ObjNum 628 snap 164 seq_num 628) 2026-02-17T05:56:03.594 INFO:tasks.rados.rados.0.trial115.stdout:1697: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1698: finishing write tid 1 to trial11545420-12 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1698: finishing write tid 2 to trial11545420-12 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1698: finishing write tid 3 to trial11545420-12 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1698: finishing write tid 4 to trial11545420-12 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1698: oid 12 updating version 0 to 962 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1698: oid 12 version 962 is already newer than 961 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1698: oid 12 updating version 962 to 963 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 963 (ObjNum 629 snap 164 seq_num 629) dirty exists 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1698: left oid 12 (ObjNum 629 snap 164 seq_num 629) 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: finishing write tid 1 to trial11545420-15 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: finishing write tid 2 to trial11545420-15 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: finishing write tid 3 to trial11545420-15 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: finishing write tid 4 to trial11545420-15 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: finishing write tid 5 to trial11545420-15 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: oid 15 updating version 0 to 816 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: oid 15 updating version 816 to 818 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: oid 15 version 818 is already newer than 815 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: oid 15 version 818 is already newer than 817 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 818 (ObjNum 630 snap 164 seq_num 630) dirty exists 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1699: left oid 15 (ObjNum 630 snap 164 seq_num 630) 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1700: finishing write tid 1 to trial11545420-13 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1700: finishing write tid 2 to trial11545420-13 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1700: finishing write tid 3 to trial11545420-13 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1700: finishing write tid 4 to trial11545420-13 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1700: oid 13 updating version 0 to 633 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1700: oid 13 updating version 633 to 634 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1700: oid 13 version 634 is already newer than 632 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 634 (ObjNum 631 snap 164 seq_num 631) dirty exists 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1700: left oid 13 (ObjNum 631 snap 164 seq_num 631) 2026-02-17T05:56:03.672 INFO:tasks.rados.rados.0.trial115.stdout:1701: expect (ObjNum 522 snap 135 seq_num 522) 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1702: finishing write tid 1 to trial11545420-40 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1702: finishing write tid 2 to trial11545420-40 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1702: finishing write tid 3 to trial11545420-40 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1702: finishing write tid 4 to trial11545420-40 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1702: oid 40 updating version 0 to 810 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1702: oid 40 version 810 is already newer than 809 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1702: oid 40 version 810 is already newer than 808 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 810 (ObjNum 632 snap 164 seq_num 632) dirty exists 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1702: left oid 40 (ObjNum 632 snap 164 seq_num 632) 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1703: finishing write tid 1 to trial11545420-39 2026-02-17T05:56:03.791 INFO:tasks.rados.rados.0.trial115.stdout:1704: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:56:04.017 INFO:tasks.rados.rados.0.trial115.stdout:1703: finishing write tid 2 to trial11545420-39 2026-02-17T05:56:04.017 INFO:tasks.rados.rados.0.trial115.stdout:1703: finishing write tid 3 to trial11545420-39 2026-02-17T05:56:04.017 INFO:tasks.rados.rados.0.trial115.stdout:1703: finishing write tid 4 to trial11545420-39 2026-02-17T05:56:04.018 INFO:tasks.rados.rados.0.trial115.stdout:1703: oid 39 updating version 0 to 966 2026-02-17T05:56:04.018 INFO:tasks.rados.rados.0.trial115.stdout:1703: oid 39 version 966 is already newer than 965 2026-02-17T05:56:04.018 INFO:tasks.rados.rados.0.trial115.stdout:1703: oid 39 updating version 966 to 967 2026-02-17T05:56:04.018 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 967 (ObjNum 633 snap 164 seq_num 633) dirty exists 2026-02-17T05:56:04.018 INFO:tasks.rados.rados.0.trial115.stdout:1703: left oid 39 (ObjNum 633 snap 164 seq_num 633) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1695: done (10 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1696: done (9 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1697: done (8 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1698: done (7 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1699: done (6 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1700: done (5 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1701: done (4 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1702: done (3 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1703: done (2 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1704: done (1 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1705: done (0 left) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1706: read oid 38 snap -1 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1706: expect (ObjNum 610 snap 158 seq_num 610) 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1707: write (excl) oid 11 current snap is 165 2026-02-17T05:56:04.543 INFO:tasks.rados.rados.0.trial115.stdout:1707: seq_num 634 ranges {655638=565165,1900172=421542} 2026-02-17T05:56:04.547 INFO:tasks.rados.rados.0.trial115.stdout:1707: writing trial11545420-11 from 655638 to 1220803 tid 1 2026-02-17T05:56:04.551 INFO:tasks.rados.rados.0.trial115.stdout:1707: writing trial11545420-11 from 1900172 to 2321714 tid 2 2026-02-17T05:56:04.551 INFO:tasks.rados.rados.0.trial115.stdout:1708: read oid 37 snap -1 2026-02-17T05:56:04.551 INFO:tasks.rados.rados.0.trial115.stdout:1708: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:04.552 INFO:tasks.rados.rados.0.trial115.stdout:1709: write oid 3 current snap is 165 2026-02-17T05:56:04.552 INFO:tasks.rados.rados.0.trial115.stdout:1709: seq_num 635 ranges {672445=640414,1842301=681553} 2026-02-17T05:56:04.556 INFO:tasks.rados.rados.0.trial115.stdout:1709: writing trial11545420-3 from 672445 to 1312859 tid 1 2026-02-17T05:56:04.561 INFO:tasks.rados.rados.0.trial115.stdout:1709: writing trial11545420-3 from 1842301 to 2523854 tid 2 2026-02-17T05:56:04.561 INFO:tasks.rados.rados.0.trial115.stdout:1710: snap_create 2026-02-17T05:56:04.561 INFO:tasks.rados.rados.0.trial115.stdout:1706: expect (ObjNum 610 snap 158 seq_num 610) 2026-02-17T05:56:04.647 INFO:tasks.rados.rados.0.trial115.stdout:1707: finishing write tid 1 to trial11545420-11 2026-02-17T05:56:04.647 INFO:tasks.rados.rados.0.trial115.stdout:1707: finishing write tid 2 to trial11545420-11 2026-02-17T05:56:04.648 INFO:tasks.rados.rados.0.trial115.stdout:1707: finishing write tid 3 to trial11545420-11 2026-02-17T05:56:04.648 INFO:tasks.rados.rados.0.trial115.stdout:1707: finishing write tid 4 to trial11545420-11 2026-02-17T05:56:04.648 INFO:tasks.rados.rados.0.trial115.stdout:1707: oid 11 updating version 0 to 822 2026-02-17T05:56:04.648 INFO:tasks.rados.rados.0.trial115.stdout:1707: oid 11 version 822 is already newer than 821 2026-02-17T05:56:04.648 INFO:tasks.rados.rados.0.trial115.stdout:1707: oid 11 version 822 is already newer than 820 2026-02-17T05:56:04.648 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 822 (ObjNum 634 snap 165 seq_num 634) dirty exists 2026-02-17T05:56:04.648 INFO:tasks.rados.rados.0.trial115.stdout:1707: left oid 11 (ObjNum 634 snap 165 seq_num 634) 2026-02-17T05:56:04.648 INFO:tasks.rados.rados.0.trial115.stdout:1708: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:04.725 INFO:tasks.rados.rados.0.trial115.stdout:1709: finishing write tid 1 to trial11545420-3 2026-02-17T05:56:04.725 INFO:tasks.rados.rados.0.trial115.stdout:1709: finishing write tid 2 to trial11545420-3 2026-02-17T05:56:04.725 INFO:tasks.rados.rados.0.trial115.stdout:1709: finishing write tid 3 to trial11545420-3 2026-02-17T05:56:04.725 INFO:tasks.rados.rados.0.trial115.stdout:1709: finishing write tid 4 to trial11545420-3 2026-02-17T05:56:04.725 INFO:tasks.rados.rados.0.trial115.stdout:1709: oid 3 updating version 0 to 826 2026-02-17T05:56:04.725 INFO:tasks.rados.rados.0.trial115.stdout:1709: oid 3 updating version 826 to 827 2026-02-17T05:56:04.725 INFO:tasks.rados.rados.0.trial115.stdout:1709: oid 3 updating version 827 to 828 2026-02-17T05:56:04.725 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 828 (ObjNum 635 snap 165 seq_num 635) dirty exists 2026-02-17T05:56:04.726 INFO:tasks.rados.rados.0.trial115.stdout:1709: left oid 3 (ObjNum 635 snap 165 seq_num 635) 2026-02-17T05:56:05.544 INFO:tasks.rados.rados.0.trial115.stdout:1706: done (4 left) 2026-02-17T05:56:05.545 INFO:tasks.rados.rados.0.trial115.stdout:1707: done (3 left) 2026-02-17T05:56:05.545 INFO:tasks.rados.rados.0.trial115.stdout:1708: done (2 left) 2026-02-17T05:56:05.545 INFO:tasks.rados.rados.0.trial115.stdout:1709: done (1 left) 2026-02-17T05:56:05.545 INFO:tasks.rados.rados.0.trial115.stdout:1710: done (0 left) 2026-02-17T05:56:05.545 INFO:tasks.rados.rados.0.trial115.stdout:1711: snap_create 2026-02-17T05:56:06.544 INFO:tasks.rados.rados.0.trial115.stdout:1711: done (0 left) 2026-02-17T05:56:06.545 INFO:tasks.rados.rados.0.trial115.stdout:1712: snap_create 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:1712: done (0 left) 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:1713: rollback oid 33 current snap is 168 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 158 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:1714: read oid 32 snap -1 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:1714: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:1715: read oid 26 snap -1 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:1715: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:1716: copy_from oid 37 from oid 13 current snap is 168 2026-02-17T05:56:07.545 INFO:tasks.rados.rados.0.trial115.stdout:1717: snap_remove snap 164 2026-02-17T05:56:08.540 INFO:tasks.rados.rados.0.trial115.stdout:1717: done (4 left) 2026-02-17T05:56:08.541 INFO:tasks.rados.rados.0.trial115.stdout:1718: delete oid 47 current snap is 168 2026-02-17T05:56:08.541 INFO:tasks.rados.rados.0.trial115.stdout:1713: finishing rollback tid 1 to trial11545420-33 2026-02-17T05:56:08.541 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 613 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:56:08.541 INFO:tasks.rados.rados.0.trial115.stdout:1715: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:08.618 INFO:tasks.rados.rados.0.trial115.stdout:1714: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:56:08.728 INFO:tasks.rados.rados.0.trial115.stdout:1716: finishing copy_from racing read to trial11545420-37 2026-02-17T05:56:08.729 INFO:tasks.rados.rados.0.trial115.stdout:1716: finishing copy_from to trial11545420-37 2026-02-17T05:56:08.729 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 634 (ObjNum 631 snap 164 seq_num 631) dirty exists 2026-02-17T05:56:08.729 INFO:tasks.rados.rados.0.trial115.stdout:1713: done (4 left) 2026-02-17T05:56:08.729 INFO:tasks.rados.rados.0.trial115.stdout:1714: done (3 left) 2026-02-17T05:56:08.729 INFO:tasks.rados.rados.0.trial115.stdout:1715: done (2 left) 2026-02-17T05:56:08.729 INFO:tasks.rados.rados.0.trial115.stdout:1716: done (1 left) 2026-02-17T05:56:08.729 INFO:tasks.rados.rados.0.trial115.stdout:1718: done (0 left) 2026-02-17T05:56:08.729 INFO:tasks.rados.rados.0.trial115.stdout:1719: rollback oid 38 current snap is 168 2026-02-17T05:56:08.730 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 38 to 163 2026-02-17T05:56:08.730 INFO:tasks.rados.rados.0.trial115.stdout:1720: write oid 35 current snap is 168 2026-02-17T05:56:08.730 INFO:tasks.rados.rados.0.trial115.stdout:1720: seq_num 637 ranges {706059=790912,1959265=685222} 2026-02-17T05:56:08.734 INFO:tasks.rados.rados.0.trial115.stdout:1720: writing trial11545420-35 from 706059 to 1496971 tid 1 2026-02-17T05:56:08.738 INFO:tasks.rados.rados.0.trial115.stdout:1720: writing trial11545420-35 from 1959265 to 2644487 tid 2 2026-02-17T05:56:08.738 INFO:tasks.rados.rados.0.trial115.stdout:1721: delete oid 44 current snap is 168 2026-02-17T05:56:08.739 INFO:tasks.rados.rados.0.trial115.stdout:1719: finishing rollback tid 0 to trial11545420-38 2026-02-17T05:56:08.739 INFO:tasks.rados.rados.0.trial115.stdout:1719: finishing rollback tid 1 to trial11545420-38 2026-02-17T05:56:08.739 INFO:tasks.rados.rados.0.trial115.stdout:1719: finishing rollback tid 2 to trial11545420-38 2026-02-17T05:56:08.739 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 916 (ObjNum 610 snap 158 seq_num 610) dirty exists 2026-02-17T05:56:08.739 INFO:tasks.rados.rados.0.trial115.stdout:1720: finishing write tid 1 to trial11545420-35 2026-02-17T05:56:08.741 INFO:tasks.rados.rados.0.trial115.stdout:1719: done (2 left) 2026-02-17T05:56:08.741 INFO:tasks.rados.rados.0.trial115.stdout:1721: done (1 left) 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1722: read oid 34 snap -1 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1722: expect deleted 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1723: read oid 20 snap -1 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1723: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1724: delete oid 22 current snap is 168 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1720: finishing write tid 2 to trial11545420-35 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1720: finishing write tid 3 to trial11545420-35 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1720: finishing write tid 4 to trial11545420-35 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1720: oid 35 updating version 0 to 919 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1720: oid 35 version 919 is already newer than 918 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:1720: oid 35 updating version 919 to 920 2026-02-17T05:56:08.742 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 920 (ObjNum 637 snap 168 seq_num 637) dirty exists 2026-02-17T05:56:08.743 INFO:tasks.rados.rados.0.trial115.stdout:1720: left oid 35 (ObjNum 637 snap 168 seq_num 637) 2026-02-17T05:56:08.743 INFO:tasks.rados.rados.0.trial115.stdout:1720: done (3 left) 2026-02-17T05:56:08.743 INFO:tasks.rados.rados.0.trial115.stdout:1722: done (2 left) 2026-02-17T05:56:08.743 INFO:tasks.rados.rados.0.trial115.stdout:1724: done (1 left) 2026-02-17T05:56:08.743 INFO:tasks.rados.rados.0.trial115.stdout:1725: snap_remove snap 158 2026-02-17T05:56:09.542 INFO:tasks.rados.rados.0.trial115.stdout:1725: done (1 left) 2026-02-17T05:56:09.542 INFO:tasks.rados.rados.0.trial115.stdout:1726: write (excl) oid 3 current snap is 168 2026-02-17T05:56:09.542 INFO:tasks.rados.rados.0.trial115.stdout:1726: seq_num 640 ranges {756480=733012,2134711=432549} 2026-02-17T05:56:09.549 INFO:tasks.rados.rados.0.trial115.stdout:1726: writing trial11545420-3 from 756480 to 1489492 tid 1 2026-02-17T05:56:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1726: writing trial11545420-3 from 2134711 to 2567260 tid 2 2026-02-17T05:56:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1727: copy_from oid 15 from oid 13 current snap is 168 2026-02-17T05:56:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1723: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:56:09.620 INFO:tasks.rados.rados.0.trial115.stdout:1726: finishing write tid 1 to trial11545420-3 2026-02-17T05:56:09.620 INFO:tasks.rados.rados.0.trial115.stdout:1726: finishing write tid 2 to trial11545420-3 2026-02-17T05:56:09.620 INFO:tasks.rados.rados.0.trial115.stdout:1726: finishing write tid 3 to trial11545420-3 2026-02-17T05:56:09.620 INFO:tasks.rados.rados.0.trial115.stdout:1723: done (2 left) 2026-02-17T05:56:09.620 INFO:tasks.rados.rados.0.trial115.stdout:1728: copy_from oid 14 from oid 22 current snap is 168 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1726: finishing write tid 4 to trial11545420-3 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1726: oid 3 updating version 0 to 839 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1726: oid 3 updating version 839 to 841 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1726: oid 3 version 841 is already newer than 840 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 841 (ObjNum 640 snap 168 seq_num 640) dirty exists 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1726: left oid 3 (ObjNum 640 snap 168 seq_num 640) 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1727: finishing copy_from racing read to trial11545420-15 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1727: finishing copy_from to trial11545420-15 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 634 (ObjNum 631 snap 164 seq_num 631) dirty exists 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1726: done (2 left) 2026-02-17T05:56:09.621 INFO:tasks.rados.rados.0.trial115.stdout:1727: done (1 left) 2026-02-17T05:56:09.622 INFO:tasks.rados.rados.0.trial115.stdout:1729: snap_create 2026-02-17T05:56:09.622 INFO:tasks.rados.rados.0.trial115.stdout:1728: finishing copy_from to trial11545420-14 2026-02-17T05:56:09.622 INFO:tasks.rados.rados.0.trial115.stdout:1728: got expected ENOENT (src dne) 2026-02-17T05:56:09.622 INFO:tasks.rados.rados.0.trial115.stdout:1728: finishing copy_from racing read to trial11545420-14 2026-02-17T05:56:10.545 INFO:tasks.rados.rados.0.trial115.stdout:1728: done (1 left) 2026-02-17T05:56:10.545 INFO:tasks.rados.rados.0.trial115.stdout:1729: done (0 left) 2026-02-17T05:56:10.545 INFO:tasks.rados.rados.0.trial115.stdout:1730: read oid 8 snap -1 2026-02-17T05:56:10.545 INFO:tasks.rados.rados.0.trial115.stdout:1730: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:56:10.545 INFO:tasks.rados.rados.0.trial115.stdout:1731: snap_remove snap 161 2026-02-17T05:56:11.546 INFO:tasks.rados.rados.0.trial115.stdout:1731: done (1 left) 2026-02-17T05:56:11.546 INFO:tasks.rados.rados.0.trial115.stdout:1732: read oid 49 snap -1 2026-02-17T05:56:11.546 INFO:tasks.rados.rados.0.trial115.stdout:1732: expect (ObjNum 614 snap 158 seq_num 614) 2026-02-17T05:56:11.546 INFO:tasks.rados.rados.0.trial115.stdout:1733: write (excl) oid 42 current snap is 169 2026-02-17T05:56:11.546 INFO:tasks.rados.rados.0.trial115.stdout:1733: seq_num 641 ranges {773287=408261,1793193=576207} 2026-02-17T05:56:11.549 INFO:tasks.rados.rados.0.trial115.stdout:1733: writing trial11545420-42 from 773287 to 1181548 tid 1 2026-02-17T05:56:11.554 INFO:tasks.rados.rados.0.trial115.stdout:1733: writing trial11545420-42 from 1793193 to 2369400 tid 2 2026-02-17T05:56:11.554 INFO:tasks.rados.rados.0.trial115.stdout:1734: snap_create 2026-02-17T05:56:11.555 INFO:tasks.rados.rados.0.trial115.stdout:1730: expect (ObjNum 593 snap 152 seq_num 593) 2026-02-17T05:56:11.777 INFO:tasks.rados.rados.0.trial115.stdout:1732: expect (ObjNum 614 snap 158 seq_num 614) 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:1733: finishing write tid 1 to trial11545420-42 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:1733: finishing write tid 2 to trial11545420-42 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:1733: finishing write tid 3 to trial11545420-42 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:1733: finishing write tid 4 to trial11545420-42 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:1733: oid 42 updating version 0 to 977 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:1733: oid 42 updating version 977 to 983 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:1733: oid 42 version 983 is already newer than 982 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 983 (ObjNum 641 snap 169 seq_num 641) dirty exists 2026-02-17T05:56:11.854 INFO:tasks.rados.rados.0.trial115.stdout:1733: left oid 42 (ObjNum 641 snap 169 seq_num 641) 2026-02-17T05:56:12.200 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T05:56:12.200 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T05:56:12.201 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T05:56:12.201 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T05:56:12.204 INFO:tasks.ceph.mon.b:Started 2026-02-17T05:56:12.204 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:56:12.204 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:12.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.274+0000 7fecca97e640 10 monclient: get_monmap_and_config 2026-02-17T05:56:12.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.274+0000 7fecca97e640 10 monclient: build_initial_monmap 2026-02-17T05:56:12.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.274+0000 7fecca97e640 10 monclient: monmap: 2026-02-17T05:56:12.273 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:12.275802+0000 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:12.275802+0000 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:12.274 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.274+0000 7fecca97e640 1 Processor -- start 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 1 -- start start 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient: init 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fecc406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fecc409c4a0 0x7fecc413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient(hunting): picked mon.noname-c con 0x7fecc409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fecc406d380 0x7fecc413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient(hunting): picked mon.noname-a con 0x7fecc406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc406daf0 con 0x7fecc406c890 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fecc40a0880 con 0x7fecc406d380 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fecc409cc80 con 0x7fecc409c4a0 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecca97e640 10 monclient(hunting): authenticate will time out at 1217.829334s 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc37fe640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fecc409c4a0 msgr2=0x7fecc413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc37fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fecc409c4a0 0x7fecc413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc3fff640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fecc406c890 legacy=0x7fecc406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fecc406d380 0x7fecc413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fecc406d380 0x7fecc413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:33806/0 (socket says 10.20.193.78:33806) 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 1 -- 10.20.193.78:0/3708315481 learned_addr learned my addr 10.20.193.78:0/3708315481 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient(hunting): get_auth_request con 0x7fecc406d380 auth_method 0 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:12.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient(hunting): handle_auth_done global_id 124206 payload 274 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 1 -- 10.20.193.78:0/3708315481 >> v2:10.20.193.115:3300/0 conn(0x7fecc409c4a0 msgr2=0x7fecc413b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 1 --2- 10.20.193.78:0/3708315481 >> v2:10.20.193.115:3300/0 conn(0x7fecc409c4a0 0x7fecc413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 1 -- 10.20.193.78:0/3708315481 >> v1:10.20.193.115:6789/0 conn(0x7fecc406c890 legacy=0x7fecc406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient: _finish_hunting 0 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 1 monclient: found mon.noname-a 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.275+0000 7fecc8ef4640 1 -- 10.20.193.78:0/3708315481 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc409cf60 con 0x7fecc406d380 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc8ef4640 1 -- 10.20.193.78:0/3708315481 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406d380 msgr2=0x7fecc413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:12.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc8ef4640 1 --2- 10.20.193.78:0/3708315481 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406d380 0x7fecc413d8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fecc406dcc0 tx=0x7fecb802f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4648f12ba481901 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 1 -- 10.20.193.78:0/3708315481 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80373f0 con 0x7fecc406d380 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: dump: 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:12.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 5 monclient: authenticate success, global_id 124206 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 1 -- 10.20.193.78:0/3708315481 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb8037590 con 0x7fecc406d380 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 1 -- 10.20.193.78:0/3708315481 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8037960 con 0x7fecc406d380 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecc2ffd640 10 monclient: dump: 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 4 set_mon_vals no callback set 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 10 monclient: shutdown 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 1 -- 10.20.193.78:0/3708315481 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406d380 msgr2=0x7fecc413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 1 --2- 10.20.193.78:0/3708315481 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406d380 0x7fecc413d8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fecc406dcc0 tx=0x7fecb802f200 comp rx=0 tx=0).stop 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 1 -- 10.20.193.78:0/3708315481 shutdown_connections 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 1 --2- 10.20.193.78:0/3708315481 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406d380 0x7fecc413d8f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 1 --2- 10.20.193.78:0/3708315481 >> v2:10.20.193.115:3300/0 conn(0x7fecc409c4a0 0x7fecc413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 1 -- 10.20.193.78:0/3708315481 >> 10.20.193.78:0/3708315481 conn(0x7fecc40a6620 msgr2=0x7fecc40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 1 -- 10.20.193.78:0/3708315481 shutdown_connections 2026-02-17T05:56:12.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.278+0000 7fecca97e640 1 -- 10.20.193.78:0/3708315481 wait complete. 2026-02-17T05:56:12.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient: build_initial_monmap 2026-02-17T05:56:12.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient: monmap: 2026-02-17T05:56:12.280 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:12.280 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 1 Processor -- start 2026-02-17T05:56:12.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 1 -- start start 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient: init 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406c890 0x7fecc4072790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient(hunting): picked mon.noname-a con 0x7fecc406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fecc406d380 0x7fecc41335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient(hunting): picked mon.noname-b con 0x7fecc406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fecc409c4a0 0x7fecc41cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient(hunting): picked mon.noname-c con 0x7fecc409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fecc409bfa0 con 0x7fecc406c890 2026-02-17T05:56:12.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fecc409be20 con 0x7fecc406d380 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fecc413de30 con 0x7fecc409c4a0 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecca97e640 10 monclient(hunting): authenticate will time out at 1217.833616s 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406c890 0x7fecc4072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406c890 0x7fecc4072790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:33814/0 (socket says 10.20.193.78:33814) 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 -- 10.20.193.78:0/3864210709 learned_addr learned my addr 10.20.193.78:0/3864210709 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient(hunting): get_auth_request con 0x7fecc406c890 auth_method 0 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:12.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc37fe640 1 -- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fecc406d380 msgr2=0x7fecc41335c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc37fe640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fecc406d380 0x7fecc41335c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc8ef4640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fecc409c4a0 0x7fecc41cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc8ef4640 10 monclient(hunting): get_auth_request con 0x7fecc409c4a0 auth_method 0 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc8ef4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc8ef4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc8ef4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc8ef4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:12.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc8ef4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc8ef4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient(hunting): handle_auth_done global_id 124209 payload 824 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 -- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fecc409c4a0 msgr2=0x7fecc41cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fecc409c4a0 0x7fecc41cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 -- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fecc406d380 msgr2=0x7fecc41335c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fecc406d380 0x7fecc41335c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 monclient: found mon.noname-a 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:12.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.279+0000 7fecc3fff640 1 -- 10.20.193.78:0/3864210709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc411ff70 con 0x7fecc406c890 2026-02-17T05:56:12.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.281+0000 7fecc3fff640 1 -- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406c890 msgr2=0x7fecc4072790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:12.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.281+0000 7fecc3fff640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406c890 0x7fecc4072790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b780 tx=0x7fecb000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:12.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 1 -- 10.20.193.78:0/3864210709 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0005270 con 0x7fecc406c890 2026-02-17T05:56:12.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: dump: 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:12.289 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 1 -- 10.20.193.78:0/3864210709 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb0010070 con 0x7fecc406c890 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 5 monclient: authenticate success, global_id 124209 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 10 monclient: _renew_subs 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc411e560 con 0x7fecc406c890 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 1 -- 10.20.193.78:0/3864210709 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0005690 con 0x7fecc406c890 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:12.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc27fc640 4 set_mon_vals no callback set 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 10 monclient: dump: 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:12.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 10 monclient: _renew_subs 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc411d180 con 0x7fecc406c890 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 1 -- 10.20.193.78:0/3864210709 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fecb000c520 con 0x7fecc406c890 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc406cc70 con 0x7fecc406c890 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec9403de60 0x7fec94040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.282+0000 7fecc0ff9640 1 -- 10.20.193.78:0/3864210709 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(360..360 src has 117..360) ==== 4811+0+0 (secure 0 0 0) 0x7fecb0052120 con 0x7fecc406c890 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.283+0000 7fecc37fe640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec9403de60 0x7fec94040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.283+0000 7fecc37fe640 10 monclient: get_auth_request con 0x7fec9403de60 auth_method 0 2026-02-17T05:56:12.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.285+0000 7fecc0ff9640 1 -- 10.20.193.78:0/3864210709 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fecb0013e10 con 0x7fecc406c890 2026-02-17T05:56:12.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.285+0000 7fecc0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:12.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.285+0000 7fecc0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:12.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.285+0000 7fecc37fe640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec9403de60 0x7fec94040320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fecc411f600 tx=0x7fecb4009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:12.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.399+0000 7fecca97e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:12.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.399+0000 7fecca97e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:12.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.399+0000 7fecca97e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:12.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.399+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fecc411fd40 con 0x7fecc406c890 2026-02-17T05:56:12.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.399+0000 7fecc0ff9640 1 -- 10.20.193.78:0/3864210709 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fecb0050b00 con 0x7fecc406c890 2026-02-17T05:56:12.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.399+0000 7fecc0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:12.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.399+0000 7fecc0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:12.399 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":110,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:12.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec9403de60 msgr2=0x7fec94040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:12.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec9403de60 0x7fec94040320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fecc411f600 tx=0x7fecb4009270 comp rx=0 tx=0).stop 2026-02-17T05:56:12.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 10 monclient: shutdown 2026-02-17T05:56:12.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:12.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406c890 msgr2=0x7fecc4072790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:12.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406c890 0x7fecc4072790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b780 tx=0x7fecb000bc50 comp rx=0 tx=0).stop 2026-02-17T05:56:12.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 shutdown_connections 2026-02-17T05:56:12.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fecc409c4a0 0x7fecc41cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec9403de60 0x7fec94040320 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fecc406d380 0x7fecc41335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 --2- 10.20.193.78:0/3864210709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fecc406c890 0x7fecc4072790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:12.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 >> 10.20.193.78:0/3864210709 conn(0x7fecc40a6620 msgr2=0x7fecc4134240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:12.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 shutdown_connections 2026-02-17T05:56:12.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:12.400+0000 7fecca97e640 1 -- 10.20.193.78:0/3864210709 wait complete. 2026-02-17T05:56:12.549 INFO:tasks.rados.rados.0.trial115.stdout:1730: done (3 left) 2026-02-17T05:56:12.549 INFO:tasks.rados.rados.0.trial115.stdout:1732: done (2 left) 2026-02-17T05:56:12.549 INFO:tasks.rados.rados.0.trial115.stdout:1733: done (1 left) 2026-02-17T05:56:12.549 INFO:tasks.rados.rados.0.trial115.stdout:1734: done (0 left) 2026-02-17T05:56:12.549 INFO:tasks.rados.rados.0.trial115.stdout:1735: snap_remove snap 167 2026-02-17T05:56:13.541 INFO:tasks.rados.rados.0.trial115.stdout:1735: done (0 left) 2026-02-17T05:56:13.541 INFO:tasks.rados.rados.0.trial115.stdout:1736: write (excl) oid 46 current snap is 170 2026-02-17T05:56:13.541 INFO:tasks.rados.rados.0.trial115.stdout:1736: seq_num 642 ranges {790094=483510,1851675=436218} 2026-02-17T05:56:13.547 INFO:tasks.rados.rados.0.trial115.stdout:1736: writing trial11545420-46 from 790094 to 1273604 tid 1 2026-02-17T05:56:13.551 INFO:tasks.rados.rados.0.trial115.stdout:1736: writing trial11545420-46 from 1851675 to 2287893 tid 2 2026-02-17T05:56:13.551 INFO:tasks.rados.rados.0.trial115.stdout:1737: snap_create 2026-02-17T05:56:13.551 INFO:tasks.rados.rados.0.trial115.stdout:1736: finishing write tid 1 to trial11545420-46 2026-02-17T05:56:13.553 INFO:tasks.rados.rados.0.trial115.stdout:1736: finishing write tid 2 to trial11545420-46 2026-02-17T05:56:13.553 INFO:tasks.rados.rados.0.trial115.stdout:1736: finishing write tid 3 to trial11545420-46 2026-02-17T05:56:13.553 INFO:tasks.rados.rados.0.trial115.stdout:1736: finishing write tid 4 to trial11545420-46 2026-02-17T05:56:13.553 INFO:tasks.rados.rados.0.trial115.stdout:1736: oid 46 updating version 0 to 840 2026-02-17T05:56:13.553 INFO:tasks.rados.rados.0.trial115.stdout:1736: oid 46 version 840 is already newer than 839 2026-02-17T05:56:13.553 INFO:tasks.rados.rados.0.trial115.stdout:1736: oid 46 version 840 is already newer than 836 2026-02-17T05:56:13.553 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 840 (ObjNum 642 snap 170 seq_num 642) dirty exists 2026-02-17T05:56:13.553 INFO:tasks.rados.rados.0.trial115.stdout:1736: left oid 46 (ObjNum 642 snap 170 seq_num 642) 2026-02-17T05:56:14.552 INFO:tasks.rados.rados.0.trial115.stdout:1736: done (1 left) 2026-02-17T05:56:14.552 INFO:tasks.rados.rados.0.trial115.stdout:1737: done (0 left) 2026-02-17T05:56:14.552 INFO:tasks.rados.rados.0.trial115.stdout:1738: snap_remove snap 169 2026-02-17T05:56:15.406 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:15.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 10 monclient: get_monmap_and_config 2026-02-17T05:56:15.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 10 monclient: build_initial_monmap 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 10 monclient: monmap: 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:15.478433+0000 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:15.478433+0000 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:15.479 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 1 Processor -- start 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 1 -- start start 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 10 monclient: init 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff64406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64409c4a0 0x7ff64413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.477+0000 7ff64b454640 10 monclient(hunting): picked mon.noname-b con 0x7ff64409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:15.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff64b454640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff64406d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff64b454640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64406daf0 con 0x7ff64406d380 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff64b454640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64409cc80 con 0x7ff64406c890 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff64b454640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff64b454640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff6440a0880 con 0x7ff64409c4a0 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff64b454640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff64b454640 10 monclient(hunting): authenticate will time out at 1221.032033s 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64409c4a0 0x7ff64413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6499ca640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff64406d380 0x7ff64413d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56502/0 (socket says 10.20.193.78:56502) 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64409c4a0 0x7ff64413b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43826/0 (socket says 10.20.193.78:43826) 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6499ca640 1 -- 10.20.193.78:0/731422470 learned_addr learned my addr 10.20.193.78:0/731422470 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient(hunting): get_auth_request con 0x7ff64409c4a0 auth_method 0 2026-02-17T05:56:15.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient(hunting): handle_auth_done global_id 74651 payload 274 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 1 -- 10.20.193.78:0/731422470 >> v1:10.20.193.115:6789/0 conn(0x7ff64406c890 legacy=0x7ff64406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 1 -- 10.20.193.78:0/731422470 >> v1:10.20.193.78:6789/0 conn(0x7ff64406d380 legacy=0x7ff64413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient: _finish_hunting 0 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 1 monclient: found mon.noname-b 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:56:15.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.478+0000 7ff6489c8640 1 -- 10.20.193.78:0/731422470 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64409cf60 con 0x7ff64409c4a0 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff6489c8640 1 -- 10.20.193.78:0/731422470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64409c4a0 msgr2=0x7ff64413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff6489c8640 1 --2- 10.20.193.78:0/731422470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64409c4a0 0x7ff64413b3a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098d0 tx=0x7ff63802edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e506b66c234dbc47 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 1 -- 10.20.193.78:0/731422470 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63803c070 con 0x7ff64409c4a0 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: dump: 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 1 -- 10.20.193.78:0/731422470 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff63802fd20 con 0x7ff64409c4a0 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff64b454640 5 monclient: authenticate success, global_id 74651 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 1 -- 10.20.193.78:0/731422470 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638037660 con 0x7ff64409c4a0 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff64b454640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff64b454640 4 set_mon_vals no callback set 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.480+0000 7ff633fff640 10 monclient: dump: 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:15.490 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 10 monclient: shutdown 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 -- 10.20.193.78:0/731422470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64409c4a0 msgr2=0x7ff64413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 --2- 10.20.193.78:0/731422470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64409c4a0 0x7ff64413b3a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098d0 tx=0x7ff63802edc0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 -- 10.20.193.78:0/731422470 shutdown_connections 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 --2- 10.20.193.78:0/731422470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64409c4a0 0x7ff64413b3a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 -- 10.20.193.78:0/731422470 >> 10.20.193.78:0/731422470 conn(0x7ff6440a6620 msgr2=0x7ff6440a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 -- 10.20.193.78:0/731422470 shutdown_connections 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 -- 10.20.193.78:0/731422470 wait complete. 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 10 monclient: build_initial_monmap 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 10 monclient: monmap: 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 Processor -- start 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 1 -- start start 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.481+0000 7ff64b454640 10 monclient: init 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff64406c890 0x7ff64413d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient(hunting): picked mon.noname-b con 0x7ff64406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64406d380 0x7ff644132120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient(hunting): picked mon.noname-c con 0x7ff64406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff64409c4a0 0x7ff6441cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient(hunting): picked mon.noname-a con 0x7ff64409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff64409bfa0 con 0x7ff64409c4a0 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff64409be20 con 0x7ff64406c890 2026-02-17T05:56:15.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff64413f010 con 0x7ff64406d380 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff64b454640 10 monclient(hunting): authenticate will time out at 1221.036241s 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64406d380 0x7ff644132120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64406d380 0x7ff644132120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43842/0 (socket says 10.20.193.78:43842) 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 -- 10.20.193.78:0/1800066030 learned_addr learned my addr 10.20.193.78:0/1800066030 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient(hunting): get_auth_request con 0x7ff64406d380 auth_method 0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6499ca640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff64409c4a0 0x7ff6441cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6499ca640 10 monclient(hunting): get_auth_request con 0x7ff64409c4a0 auth_method 0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6499ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6499ca640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6499ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6499ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6499ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6499ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient(hunting): handle_auth_done global_id 74654 payload 824 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6491c9640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff64406c890 0x7ff64413d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 -- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff64406c890 msgr2=0x7ff64413d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff64406c890 0x7ff64413d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 -- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff64409c4a0 msgr2=0x7ff6441cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff64409c4a0 0x7ff6441cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient: _finish_hunting 0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 monclient: found mon.noname-c 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6489c8640 1 -- 10.20.193.78:0/1800066030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64409d810 con 0x7ff64406d380 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6491c9640 10 monclient: get_auth_request con 0x7ff64406c890 auth_method 0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.482+0000 7ff6491c9640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff64406c890 0x7ff64413d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff6489c8640 1 -- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64406d380 msgr2=0x7ff644132120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff6489c8640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64406d380 0x7ff644132120 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff638009a00 tx=0x7ff6380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 1 -- 10.20.193.78:0/1800066030 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638045020 con 0x7ff64406d380 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: dump: 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:15.492 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 1 -- 10.20.193.78:0/1800066030 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff63803d040 con 0x7ff64406d380 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff64b454640 5 monclient: authenticate success, global_id 74654 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff64b454640 10 monclient: _renew_subs 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff64b454640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64406fc90 con 0x7ff64406d380 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 1 -- 10.20.193.78:0/1800066030 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63803c070 con 0x7ff64406d380 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff631ffb640 10 monclient: dump: 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff6337fe640 4 set_mon_vals no callback set 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff64b454640 10 monclient: _renew_subs 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff64b454640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.484+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64411d7e0 con 0x7ff64406d380 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff64b454640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff64b454640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff64b454640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64406cc70 con 0x7ff64406d380 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff631ffb640 1 -- 10.20.193.78:0/1800066030 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff638041c80 con 0x7ff64406d380 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff631ffb640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff61403deb0 0x7ff614040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff631ffb640 1 -- 10.20.193.78:0/1800066030 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(363..363 src has 117..363) ==== 4783+0+0 (secure 0 0 0) 0x7ff638077ae0 con 0x7ff64406d380 2026-02-17T05:56:15.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff6491c9640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff61403deb0 0x7ff614040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:15.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.485+0000 7ff6491c9640 10 monclient: get_auth_request con 0x7ff61403deb0 auth_method 0 2026-02-17T05:56:15.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.487+0000 7ff631ffb640 1 -- 10.20.193.78:0/1800066030 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff63804f030 con 0x7ff64406d380 2026-02-17T05:56:15.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.487+0000 7ff631ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:15.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.487+0000 7ff631ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:15.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.488+0000 7ff6491c9640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff61403deb0 0x7ff614040370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff634004020 tx=0x7ff634009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:15.554 INFO:tasks.rados.rados.0.trial115.stdout:1738: done (0 left) 2026-02-17T05:56:15.555 INFO:tasks.rados.rados.0.trial115.stdout:1739: read oid 32 snap -1 2026-02-17T05:56:15.555 INFO:tasks.rados.rados.0.trial115.stdout:1739: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:56:15.555 INFO:tasks.rados.rados.0.trial115.stdout:1740: delete oid 30 current snap is 171 2026-02-17T05:56:15.556 INFO:tasks.rados.rados.0.trial115.stdout:1740: done (1 left) 2026-02-17T05:56:15.556 INFO:tasks.rados.rados.0.trial115.stdout:1741: delete oid 4 current snap is 171 2026-02-17T05:56:15.557 INFO:tasks.rados.rados.0.trial115.stdout:1741: done (1 left) 2026-02-17T05:56:15.557 INFO:tasks.rados.rados.0.trial115.stdout:1742: snap_remove snap 162 2026-02-17T05:56:15.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.606+0000 7ff64b454640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:15.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.606+0000 7ff64b454640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:15.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.606+0000 7ff64b454640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:15.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.606+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff64406cc70 con 0x7ff64406d380 2026-02-17T05:56:15.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.607+0000 7ff631ffb640 1 -- 10.20.193.78:0/1800066030 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7ff63803fd80 con 0x7ff64406d380 2026-02-17T05:56:15.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.607+0000 7ff631ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:15.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.607+0000 7ff631ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:15.606 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":110,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff61403deb0 msgr2=0x7ff614040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff61403deb0 0x7ff614040370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff634004020 tx=0x7ff634009040 comp rx=0 tx=0).stop 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 10 monclient: shutdown 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64406d380 msgr2=0x7ff644132120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64406d380 0x7ff644132120 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff638009a00 tx=0x7ff6380047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 shutdown_connections 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff64409c4a0 0x7ff6441cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff64406d380 0x7ff644132120 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff61403deb0 0x7ff614040370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 --2- 10.20.193.78:0/1800066030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff64406c890 0x7ff64413d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 >> 10.20.193.78:0/1800066030 conn(0x7ff6440a6620 msgr2=0x7ff644132fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:15.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 shutdown_connections 2026-02-17T05:56:15.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:15.609+0000 7ff64b454640 1 -- 10.20.193.78:0/1800066030 wait complete. 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1742: done (1 left) 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1743: read oid 10 snap -1 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1743: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1744: read oid 50 snap 139 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1744: expect (ObjNum 522 snap 135 seq_num 522) 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1745: read oid 18 snap -1 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1745: expect (ObjNum 612 snap 158 seq_num 612) 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1746: copy_from oid 3 from oid 20 current snap is 171 2026-02-17T05:56:16.561 INFO:tasks.rados.rados.0.trial115.stdout:1739: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:56:16.673 INFO:tasks.rados.rados.0.trial115.stdout:1743: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:56:16.818 INFO:tasks.rados.rados.0.trial115.stdout:1744: expect (ObjNum 522 snap 135 seq_num 522) 2026-02-17T05:56:16.937 INFO:tasks.rados.rados.0.trial115.stdout:1745: expect (ObjNum 612 snap 158 seq_num 612) 2026-02-17T05:56:17.031 INFO:tasks.rados.rados.0.trial115.stdout:1746: finishing copy_from racing read to trial11545420-3 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1739: done (4 left) 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1743: done (3 left) 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1744: done (2 left) 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1745: done (1 left) 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1747: read oid 43 snap -1 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1747: expect (ObjNum 611 snap 158 seq_num 611) 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1748: rollback oid 41 current snap is 171 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 170 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1746: finishing copy_from to trial11545420-3 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 603 (ObjNum 598 snap 155 seq_num 598) dirty exists 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1746: done (2 left) 2026-02-17T05:56:17.032 INFO:tasks.rados.rados.0.trial115.stdout:1749: snap_create 2026-02-17T05:56:17.033 INFO:tasks.rados.rados.0.trial115.stdout:1748: finishing rollback tid 1 to trial11545420-41 2026-02-17T05:56:17.033 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 0 (ObjNum 129 snap 0 seq_num 1334133672) dirty dne 2026-02-17T05:56:17.039 INFO:tasks.rados.rados.0.trial115.stdout:1747: expect (ObjNum 611 snap 158 seq_num 611) 2026-02-17T05:56:17.563 INFO:tasks.rados.rados.0.trial115.stdout:1747: done (2 left) 2026-02-17T05:56:17.563 INFO:tasks.rados.rados.0.trial115.stdout:1748: done (1 left) 2026-02-17T05:56:17.563 INFO:tasks.rados.rados.0.trial115.stdout:1749: done (0 left) 2026-02-17T05:56:17.563 INFO:tasks.rados.rados.0.trial115.stdout:1750: read oid 10 snap -1 2026-02-17T05:56:17.563 INFO:tasks.rados.rados.0.trial115.stdout:1750: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:56:17.563 INFO:tasks.rados.rados.0.trial115.stdout:1751: snap_create 2026-02-17T05:56:17.568 INFO:tasks.rados.rados.0.trial115.stdout:1750: expect (ObjNum 190 snap 33 seq_num 190) 2026-02-17T05:56:18.617 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.686+0000 7fd142ab0640 10 monclient: get_monmap_and_config 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.686+0000 7fd142ab0640 10 monclient: build_initial_monmap 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.686+0000 7fd142ab0640 10 monclient: monmap: 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:18.687754+0000 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:18.687754+0000 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.686+0000 7fd142ab0640 1 Processor -- start 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.686+0000 7fd142ab0640 1 -- start start 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.686+0000 7fd142ab0640 10 monclient: init 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd13c06c890 0x7fd13c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient(hunting): picked mon.noname-a con 0x7fd13c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd13c09c4a0 0x7fd13c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient(hunting): picked mon.noname-c con 0x7fd13c09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c134df0 0x7fd13c135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient(hunting): picked mon.noname-b con 0x7fd13c134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd13c0a0880 con 0x7fd13c06c890 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd13c06d6f0 con 0x7fd13c09c4a0 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd13c06daf0 con 0x7fd13c134df0 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:18.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd142ab0640 10 monclient(hunting): authenticate will time out at 1224.241289s 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd13c06c890 0x7fd13c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c134df0 0x7fd13c135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd13c06c890 0x7fd13c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40956/0 (socket says 10.20.193.78:40956) 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 -- 10.20.193.78:0/1151675240 learned_addr learned my addr 10.20.193.78:0/1151675240 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient(hunting): get_auth_request con 0x7fd13c06c890 auth_method 0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 10 monclient(hunting): get_auth_request con 0x7fd13c134df0 auth_method 0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd133fff640 1 --2- 10.20.193.78:0/1151675240 >> v2:10.20.193.115:3300/0 conn(0x7fd13c09c4a0 0x7fd13c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd133fff640 10 monclient(hunting): get_auth_request con 0x7fd13c09c4a0 auth_method 0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd133fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd133fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd133fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient(hunting): handle_auth_done global_id 124215 payload 274 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 -- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c134df0 msgr2=0x7fd13c135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 --2- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c134df0 0x7fd13c135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 -- 10.20.193.78:0/1151675240 >> v2:10.20.193.115:3300/0 conn(0x7fd13c09c4a0 msgr2=0x7fd13c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 --2- 10.20.193.78:0/1151675240 >> v2:10.20.193.115:3300/0 conn(0x7fd13c09c4a0 0x7fd13c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient: _finish_hunting 0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 monclient: found mon.noname-a 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd140825640 1 -- 10.20.193.78:0/1151675240 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c09cc80 con 0x7fd13c06c890 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.687+0000 7fd141026640 1 --2- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c134df0 0x7fd13c135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd140825640 1 -- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c06c890 msgr2=0x7fd13c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd140825640 1 --2- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c06c890 0x7fd13c06cc70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd13c09c980 tx=0x7fd12c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44792012466dc809 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 1 -- 10.20.193.78:0/1151675240 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c03c070 con 0x7fd13c06c890 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: dump: 2026-02-17T05:56:18.697 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 5 monclient: authenticate success, global_id 124215 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 1 -- 10.20.193.78:0/1151675240 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c02fca0 con 0x7fd13c06c890 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 1 -- 10.20.193.78:0/1151675240 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c037660 con 0x7fd13c06c890 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 4 set_mon_vals no callback set 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd1337fe640 10 monclient: dump: 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 10 monclient: shutdown 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 -- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c06c890 msgr2=0x7fd13c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 --2- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c06c890 0x7fd13c06cc70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd13c09c980 tx=0x7fd12c02ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 -- 10.20.193.78:0/1151675240 shutdown_connections 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 --2- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c134df0 0x7fd13c135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 --2- 10.20.193.78:0/1151675240 >> v2:10.20.193.115:3300/0 conn(0x7fd13c09c4a0 0x7fd13c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 --2- 10.20.193.78:0/1151675240 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c06c890 0x7fd13c06cc70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 -- 10.20.193.78:0/1151675240 >> 10.20.193.78:0/1151675240 conn(0x7fd13c0a6620 msgr2=0x7fd13c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 -- 10.20.193.78:0/1151675240 shutdown_connections 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.690+0000 7fd142ab0640 1 -- 10.20.193.78:0/1151675240 wait complete. 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient: build_initial_monmap 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient: monmap: 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 1 Processor -- start 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 1 -- start start 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient: init 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c06c890 0x7fd13c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient(hunting): picked mon.noname-c con 0x7fd13c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c09c4a0 0x7fd13c1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient(hunting): picked mon.noname-a con 0x7fd13c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd13c134df0 0x7fd13c09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient(hunting): picked mon.noname-b con 0x7fd13c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c0a01a0 con 0x7fd13c09c4a0 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c0a0020 con 0x7fd13c134df0 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd13c09be20 con 0x7fd13c06c890 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd142ab0640 10 monclient(hunting): authenticate will time out at 1224.245694s 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c06c890 0x7fd13c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c06c890 0x7fd13c1cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33216/0 (socket says 10.20.193.78:33216) 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 -- 10.20.193.78:0/3895950978 learned_addr learned my addr 10.20.193.78:0/3895950978 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient(hunting): get_auth_request con 0x7fd13c06c890 auth_method 0 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd133fff640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c09c4a0 0x7fd13c1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd133fff640 10 monclient(hunting): get_auth_request con 0x7fd13c09c4a0 auth_method 0 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd133fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd133fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd133fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd133fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd133fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd133fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient(hunting): handle_auth_done global_id 74660 payload 824 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 -- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd13c134df0 msgr2=0x7fd13c09db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd13c134df0 0x7fd13c09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 -- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c09c4a0 msgr2=0x7fd13c1cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c09c4a0 0x7fd13c1cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient: _finish_hunting 0 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 monclient: found mon.noname-c 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.691+0000 7fd140825640 1 -- 10.20.193.78:0/3895950978 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c127730 con 0x7fd13c06c890 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd140825640 1 -- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c06c890 msgr2=0x7fd13c1cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd140825640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c06c890 0x7fd13c1cf350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009a70 tx=0x7fd12c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 1 -- 10.20.193.78:0/3895950978 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c047070 con 0x7fd13c06c890 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: dump: 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 1 -- 10.20.193.78:0/3895950978 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c02fe50 con 0x7fd13c06c890 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 5 monclient: authenticate success, global_id 74660 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 10 monclient: _renew_subs 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c0a1690 con 0x7fd13c06c890 2026-02-17T05:56:18.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 1 -- 10.20.193.78:0/3895950978 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c03c040 con 0x7fd13c06c890 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd132ffd640 4 set_mon_vals no callback set 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 10 monclient: dump: 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 10 monclient: _renew_subs 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c127450 con 0x7fd13c06c890 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 1 -- 10.20.193.78:0/3895950978 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd12c03f9f0 con 0x7fd13c06c890 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c074980 con 0x7fd13c06c890 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd10c03df00 0x7fd10c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.694+0000 7fd1317fa640 1 -- 10.20.193.78:0/3895950978 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(366..366 src has 117..366) ==== 4799+0+0 (secure 0 0 0) 0x7fd12c077500 con 0x7fd13c06c890 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.695+0000 7fd133fff640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd10c03df00 0x7fd10c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.695+0000 7fd133fff640 10 monclient: get_auth_request con 0x7fd10c03df00 auth_method 0 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.697+0000 7fd1317fa640 1 -- 10.20.193.78:0/3895950978 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd12c035340 con 0x7fd13c06c890 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.697+0000 7fd1317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.697+0000 7fd1317fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:18.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.698+0000 7fd133fff640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd10c03df00 0x7fd10c0403c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd13c09f2d0 tx=0x7fd124008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.814+0000 7fd142ab0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:18.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.814+0000 7fd142ab0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:18.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.814+0000 7fd142ab0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:18.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.814+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd13c135280 con 0x7fd13c06c890 2026-02-17T05:56:18.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.814+0000 7fd1317fa640 1 -- 10.20.193.78:0/3895950978 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd12c075ef0 con 0x7fd13c06c890 2026-02-17T05:56:18.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.814+0000 7fd1317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:18.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.814+0000 7fd1317fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:18.814 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd10c03df00 msgr2=0x7fd10c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd10c03df00 0x7fd10c0403c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd13c09f2d0 tx=0x7fd124008040 comp rx=0 tx=0).stop 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 10 monclient: shutdown 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c06c890 msgr2=0x7fd13c1cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c06c890 0x7fd13c1cf350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009a70 tx=0x7fd12c004830 comp rx=0 tx=0).stop 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 shutdown_connections 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd10c03df00 0x7fd10c0403c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd13c134df0 0x7fd13c09db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd13c09c4a0 0x7fd13c1cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 --2- 10.20.193.78:0/3895950978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd13c06c890 0x7fd13c1cf350 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 >> 10.20.193.78:0/3895950978 conn(0x7fd13c0a6620 msgr2=0x7fd13c072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 shutdown_connections 2026-02-17T05:56:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.815+0000 7fd142ab0640 1 -- 10.20.193.78:0/3895950978 wait complete. 2026-02-17T05:56:18.822 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:56:18.822 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:56:18.822 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:56:18.839 INFO:tasks.rados.rados.0.trial115.stdout:1750: done (1 left) 2026-02-17T05:56:18.839 INFO:tasks.rados.rados.0.trial115.stdout:1751: done (0 left) 2026-02-17T05:56:18.839 INFO:tasks.rados.rados.0.trial115.stdout:1752: delete oid 29 current snap is 173 2026-02-17T05:56:18.841 INFO:tasks.rados.rados.0.trial115.stdout:1752: done (0 left) 2026-02-17T05:56:18.841 INFO:tasks.rados.rados.0.trial115.stdout:1753: copy_from oid 37 from oid 16 current snap is 173 2026-02-17T05:56:18.842 INFO:tasks.rados.rados.0.trial115.stdout:1754: copy_from oid 25 from oid 6 current snap is 173 2026-02-17T05:56:18.842 INFO:tasks.rados.rados.0.trial115.stdout:1755: copy_from oid 30 from oid 28 current snap is 173 2026-02-17T05:56:18.842 INFO:tasks.rados.rados.0.trial115.stdout:1756: read oid 23 snap -1 2026-02-17T05:56:18.842 INFO:tasks.rados.rados.0.trial115.stdout:1756: expect (ObjNum 624 snap 162 seq_num 624) 2026-02-17T05:56:18.842 INFO:tasks.rados.rados.0.trial115.stdout:1757: read oid 20 snap 157 2026-02-17T05:56:18.842 INFO:tasks.rados.rados.0.trial115.stdout:1757: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:56:18.842 INFO:tasks.rados.rados.0.trial115.stdout:1758: write oid 9 current snap is 173 2026-02-17T05:56:18.842 INFO:tasks.rados.rados.0.trial115.stdout:1758: seq_num 646 ranges {457322=784506,1969250=443556} 2026-02-17T05:56:18.846 INFO:tasks.rados.rados.0.trial115.stdout:1758: writing trial11545420-9 from 457322 to 1241828 tid 1 2026-02-17T05:56:18.851 INFO:tasks.rados.rados.0.trial115.stdout:1758: writing trial11545420-9 from 1969250 to 2412806 tid 2 2026-02-17T05:56:18.851 INFO:tasks.rados.rados.0.trial115.stdout:1759: snap_remove snap 156 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.889+0000 7f5f9cf96640 10 monclient: get_monmap_and_config 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.889+0000 7f5f9cf96640 10 monclient: build_initial_monmap 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.889+0000 7f5f9cf96640 10 monclient: monmap: 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:18.891156+0000 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:18.891156+0000 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:18.889 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:18.890 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:18.890 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 1 Processor -- start 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 1 -- start start 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient: init 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5f9806c890 0x7f5f9806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient(hunting): picked mon.noname-c con 0x7f5f9806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5f9809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5f9806d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f98135c10 con 0x7f5f9809c4a0 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f9806daf0 con 0x7f5f9806d1b0 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5f980a0880 con 0x7f5f9806c890 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f9cf96640 10 monclient(hunting): authenticate will time out at 1224.444746s 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f95d74640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f5f9809c4a0 0x7f5f98134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:43854/0 (socket says 10.20.193.78:43854) 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f95d74640 1 -- 10.20.193.78:0/295295353 learned_addr learned my addr 10.20.193.78:0/295295353 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f96575640 1 --2- 10.20.193.78:0/295295353 >> v2:10.20.193.115:3300/0 conn(0x7f5f9806c890 0x7f5f9806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f96575640 10 monclient(hunting): get_auth_request con 0x7f5f9806c890 auth_method 0 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f96575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f96575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.890+0000 7f5f96575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.891+0000 7f5f96575640 1 -- 10.20.193.78:0/295295353 >> v2:10.20.193.115:3300/0 conn(0x7f5f9806c890 msgr2=0x7f5f9806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.891+0000 7f5f96575640 1 -- 10.20.193.78:0/295295353 >> v2:10.20.193.115:3300/0 conn(0x7f5f9806c890 msgr2=0x7f5f9806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.891+0000 7f5f96575640 1 --2- 10.20.193.78:0/295295353 >> v2:10.20.193.115:3300/0 conn(0x7f5f9806c890 0x7f5f9806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:18.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.891+0000 7f5f96575640 1 --2- 10.20.193.78:0/295295353 >> v2:10.20.193.115:3300/0 conn(0x7f5f9806c890 0x7f5f9806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:18.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.892+0000 7f5f95d74640 1 -- 10.20.193.78:0/295295353 >> v1:10.20.193.78:6789/0 conn(0x7f5f9809c4a0 legacy=0x7f5f98134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537786813 0 0) 0x7f5f98135c10 con 0x7f5f9809c4a0 2026-02-17T05:56:18.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): _negotiate 2026-02-17T05:56:18.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): my global_id is 124221 2026-02-17T05:56:18.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f84003620 con 0x7f5f9809c4a0 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96d76640 1 -- 10.20.193.78:0/295295353 >> v1:10.20.193.115:6789/0 conn(0x7f5f9806d1b0 legacy=0x7f5f98135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3076009385 0 0) 0x7f5f84003620 con 0x7f5f9809c4a0 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 >> v2:10.20.193.115:3300/0 conn(0x7f5f9806c890 msgr2=0x7f5f9806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 --2- 10.20.193.78:0/295295353 >> v2:10.20.193.115:3300/0 conn(0x7f5f9806c890 0x7f5f9806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 >> v1:10.20.193.115:6789/0 conn(0x7f5f9806d1b0 legacy=0x7f5f98135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: _finish_hunting 0 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9809cc80 con 0x7f5f9809c4a0 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: _finish_auth 0 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f5f80003270 con 0x7f5f9809c4a0 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: dump: 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.894 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 >> v1:10.20.193.78:6789/0 conn(0x7f5f9809c4a0 legacy=0x7f5f98134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f84003b50 0x7f5f84003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): picked mon.c con 0x7f5f84003b50 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f84004900 0x7f5f84004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): picked mon.a con 0x7f5f84004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f840054a0 0x7f5f8400bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): picked mon.b con 0x7f5f840054a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8400c3b0 con 0x7f5f84004900 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8400c560 con 0x7f5f840054a0 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8400c760 con 0x7f5f84003b50 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95573640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f84004900 0x7f5f84004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient(hunting): get_auth_request con 0x7f5f84004900 auth_method 0 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96575640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f84003b50 0x7f5f84003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:18.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96575640 10 monclient(hunting): get_auth_request con 0x7f5f84003b50 auth_method 0 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient(hunting): handle_auth_done global_id 124221 payload 293 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 1 -- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f84003b50 msgr2=0x7f5f84003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f84003b50 0x7f5f84003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 1 -- 10.20.193.78:0/295295353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f840054a0 msgr2=0x7f5f8400bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f840054a0 0x7f5f8400bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient: _finish_hunting 0 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 1 monclient: found mon.a 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 1 -- 10.20.193.78:0/295295353 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f84003620 con 0x7f5f84004900 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient: _finish_auth 0 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 10 monclient: _check_auth_tickets 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f96d76640 1 -- 10.20.193.78:0/295295353 reap_dead start 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f95d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f9cf96640 5 monclient: authenticate success, global_id 124221 2026-02-17T05:56:18.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.893+0000 7f5f9cf96640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.895+0000 7f5f95d74640 1 -- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f84004900 msgr2=0x7f5f84004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.895+0000 7f5f95d74640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f84004900 0x7f5f84004d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f80002ba0 tx=0x7f5f8002f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c76c59481728676 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000c200 con 0x7f5f84004900 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 10 monclient: dump: 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.898 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f80005cb0 con 0x7f5f84004900 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 1 -- 10.20.193.78:0/295295353 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80005e60 con 0x7f5f84004900 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 4 set_mon_vals no callback set 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f95573640 10 monclient: dump: 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 10 monclient: shutdown 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 1 -- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f84004900 msgr2=0x7f5f84004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f84004900 0x7f5f84004d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f80002ba0 tx=0x7f5f8002f7d0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 1 -- 10.20.193.78:0/295295353 shutdown_connections 2026-02-17T05:56:18.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 1 --2- 10.20.193.78:0/295295353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f84004900 0x7f5f84004d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 1 -- 10.20.193.78:0/295295353 >> 10.20.193.78:0/295295353 conn(0x7f5f980a6620 msgr2=0x7f5f980a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 1 -- 10.20.193.78:0/295295353 shutdown_connections 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.896+0000 7f5f9cf96640 1 -- 10.20.193.78:0/295295353 wait complete. 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient: build_initial_monmap 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient: monmap: 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 1 Processor -- start 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 1 -- start start 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient: init 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f84004900 0x7f5f981cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient(hunting): picked mon.noname-b con 0x7f5f84004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f981cf920 0x7f5f9809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient(hunting): picked mon.noname-c con 0x7f5f981cf920 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f981cfd00 0x7f5f9809e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient(hunting): picked mon.noname-a con 0x7f5f981cfd00 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:18.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f980a0020 con 0x7f5f981cfd00 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9806d640 con 0x7f5f84004900 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5f980a01a0 con 0x7f5f981cf920 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f9cf96640 10 monclient(hunting): authenticate will time out at 1224.451555s 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f981cf920 0x7f5f9809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f981cf920 0x7f5f9809db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33226/0 (socket says 10.20.193.78:33226) 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f981cfd00 0x7f5f9809e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 1 -- 10.20.193.78:0/1793257139 learned_addr learned my addr 10.20.193.78:0/1793257139 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 10 monclient(hunting): get_auth_request con 0x7f5f981cf920 auth_method 0 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient(hunting): get_auth_request con 0x7f5f981cfd00 auth_method 0 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f95d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96575640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f84004900 0x7f5f981cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96575640 10 monclient(hunting): get_auth_request con 0x7f5f84004900 auth_method 0 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient(hunting): handle_auth_done global_id 124224 payload 824 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f981cf920 msgr2=0x7f5f9809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f981cf920 0x7f5f9809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f84004900 msgr2=0x7f5f981cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f84004900 0x7f5f981cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:18.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 1 monclient: found mon.noname-a 2026-02-17T05:56:18.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:18.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:18.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.897+0000 7f5f96d76640 1 -- 10.20.193.78:0/1793257139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f981291f0 con 0x7f5f981cfd00 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.899+0000 7f5f96d76640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f981cfd00 msgr2=0x7f5f9809e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.899+0000 7f5f96d76640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f981cfd00 0x7f5f9809e090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f9809c880 tx=0x7f5f8800ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88017070 con 0x7f5f981cfd00 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: dump: 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.903 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: _finish_auth 0 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8800ac90 con 0x7f5f981cfd00 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f880070d0 con 0x7f5f981cfd00 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 5 monclient: authenticate success, global_id 124224 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 10 monclient: _renew_subs 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f94d72640 4 set_mon_vals no callback set 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f98072610 con 0x7f5f981cfd00 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 10 monclient: dump: 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 10 monclient: _renew_subs 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9809d810 con 0x7f5f981cfd00 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811e390 0x7f5f9811e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9809beb0 con 0x7f5f9811e390 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811e390 0x7f5f9811e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 10 monclient: get_auth_request con 0x7f5f9811e390 auth_method 0 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5f88007270 con 0x7f5f981cfd00 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 10 monclient(hunting): handle_auth_done global_id 124227 payload 824 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f96575640 1 -- 10.20.193.78:0/1793257139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f980008d0 con 0x7f5f9811e390 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f6403deb0 0x7f5f64040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.900+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(367..367 src has 117..367) ==== 4799+0+0 (secure 0 0 0) 0x7f5f88052440 con 0x7f5f981cfd00 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.901+0000 7f5f95d74640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f6403deb0 0x7f5f64040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.901+0000 7f5f95d74640 10 monclient: get_auth_request con 0x7f5f6403deb0 auth_method 0 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.902+0000 7f5f96575640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811e390 msgr2=0x7f5f9811e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.902+0000 7f5f96575640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811e390 0x7f5f9811e770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f980a73a0 tx=0x7f5f8c012d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c01b070 con 0x7f5f9811e390 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 10 monclient: dump: 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.905 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f5f8c010dc0 con 0x7f5f9811e390 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811e390 msgr2=0x7f5f9811e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811e390 0x7f5f9811e770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f980a73a0 tx=0x7f5f8c012d10 comp rx=0 tx=0).stop 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811e390 msgr2=0x7f5f9811e770 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f7effd640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811e390 0x7f5f9811e770 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f980a73a0 tx=0x7f5f8c012d10 comp rx=0 tx=0).stop 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.903+0000 7f5f95d74640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f6403deb0 0x7f5f64040370 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f8002f810 tx=0x7f5f80009070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f9cf96640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f9cf96640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811ecb0 0x7f5f9811f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f9cf96640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9809ed90 con 0x7f5f9811ecb0 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811ecb0 0x7f5f9811f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 10 monclient: get_auth_request con 0x7f5f9811ecb0 auth_method 0 2026-02-17T05:56:18.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 10 monclient(hunting): handle_auth_done global_id 124230 payload 824 2026-02-17T05:56:18.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.906+0000 7f5f96575640 1 -- 10.20.193.78:0/1793257139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5f9809ef40 con 0x7f5f9811ecb0 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f96575640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811ecb0 msgr2=0x7f5f9811f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f96575640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811ecb0 0x7f5f9811f0c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c008010 tx=0x7f5f8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00f040 con 0x7f5f9811ecb0 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 10 monclient: dump: 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f5f8c003230 con 0x7f5f9811ecb0 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811ecb0 msgr2=0x7f5f9811f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811ecb0 0x7f5f9811f0c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c008010 tx=0x7f5f8c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811ecb0 msgr2=0x7f5f9811f0c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:18.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f7effd640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f9811ecb0 0x7f5f9811f0c0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c008010 tx=0x7f5f8c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 112, 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 70374, 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:56:18.909 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:18.910 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:18.912 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:56:18.913 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f6403deb0 msgr2=0x7f5f64040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f9cf96640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f6403deb0 0x7f5f64040370 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f8002f810 tx=0x7f5f80009070 comp rx=0 tx=0).stop 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f9cf96640 10 monclient: shutdown 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f9cf96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f981cfd00 msgr2=0x7f5f9809e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f9cf96640 1 --2- 10.20.193.78:0/1793257139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f981cfd00 0x7f5f9809e090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f9809c880 tx=0x7f5f8800ce40 comp rx=0 tx=0).stop 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f96d76640 1 -- 10.20.193.78:0/1793257139 reap_dead start 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.908+0000 7f5f96d76640 1 -- 10.20.193.78:0/1793257139 reap_dead start 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.909+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 shutdown_connections 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.909+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 >> 10.20.193.78:0/1793257139 conn(0x7f5f980a6620 msgr2=0x7f5f9811cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.909+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 shutdown_connections 2026-02-17T05:56:18.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.909+0000 7f5f9cf96640 1 -- 10.20.193.78:0/1793257139 wait complete. 2026-02-17T05:56:18.915 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:56:18.915 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:56:18.915 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:56:18.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.983+0000 7fa6391b6640 10 monclient: get_monmap_and_config 2026-02-17T05:56:18.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.983+0000 7fa6391b6640 10 monclient: build_initial_monmap 2026-02-17T05:56:18.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.983+0000 7fa6391b6640 10 monclient: monmap: 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:18.985082+0000 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:18.985082+0000 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 1 Processor -- start 2026-02-17T05:56:18.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 1 -- start start 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient: init 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63406c890 0x7fa63406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient(hunting): picked mon.noname-b con 0x7fa63406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa63409c4a0 0x7fa63406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient(hunting): picked mon.noname-a con 0x7fa63409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa634134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa63406daf0 con 0x7fa634134df0 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa6340a0880 con 0x7fa63409c4a0 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa63406d6f0 con 0x7fa63406c890 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa6391b6640 10 monclient(hunting): authenticate will time out at 1224.538651s 2026-02-17T05:56:18.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63406c890 0x7fa63406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63406c890 0x7fa63406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33232/0 (socket says 10.20.193.78:33232) 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa63409c4a0 0x7fa63406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 1 -- 10.20.193.78:0/695776201 learned_addr learned my addr 10.20.193.78:0/695776201 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 10 monclient(hunting): get_auth_request con 0x7fa63406c890 auth_method 0 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient(hunting): get_auth_request con 0x7fa63409c4a0 auth_method 0 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:18.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient(hunting): handle_auth_done global_id 124233 payload 274 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 1 -- 10.20.193.78:0/695776201 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63406c890 msgr2=0x7fa63406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 1 --2- 10.20.193.78:0/695776201 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63406c890 0x7fa63406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 1 -- 10.20.193.78:0/695776201 >> v1:10.20.193.115:6789/0 conn(0x7fa634134df0 legacy=0x7fa634135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient: _finish_hunting 0 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 1 monclient: found mon.noname-a 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:18.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.984+0000 7fa632575640 1 -- 10.20.193.78:0/695776201 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63409cc80 con 0x7fa63409c4a0 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa632575640 1 -- 10.20.193.78:0/695776201 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 msgr2=0x7fa63406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa632575640 1 --2- 10.20.193.78:0/695776201 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 0x7fa63406d1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0098a0 tx=0x7fa61c02ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a6a8cfd491a233 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 1 -- 10.20.193.78:0/695776201 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c03c070 con 0x7fa63409c4a0 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: dump: 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.995 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: _finish_auth 0 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: _check_auth_tickets 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 1 -- 10.20.193.78:0/695776201 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c02fba0 con 0x7fa63409c4a0 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 1 -- 10.20.193.78:0/695776201 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c037440 con 0x7fa63409c4a0 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 5 monclient: authenticate success, global_id 124233 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 4 set_mon_vals no callback set 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:18.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa631d74640 10 monclient: dump: 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 10 monclient: shutdown 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 1 -- 10.20.193.78:0/695776201 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 msgr2=0x7fa63406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 1 --2- 10.20.193.78:0/695776201 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 0x7fa63406d1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0098a0 tx=0x7fa61c02ed20 comp rx=0 tx=0).stop 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 1 -- 10.20.193.78:0/695776201 shutdown_connections 2026-02-17T05:56:18.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 1 --2- 10.20.193.78:0/695776201 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 0x7fa63406d1b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 1 --2- 10.20.193.78:0/695776201 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63406c890 0x7fa63406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 1 -- 10.20.193.78:0/695776201 >> 10.20.193.78:0/695776201 conn(0x7fa6340a6620 msgr2=0x7fa6340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 1 -- 10.20.193.78:0/695776201 shutdown_connections 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.987+0000 7fa6391b6640 1 -- 10.20.193.78:0/695776201 wait complete. 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.988+0000 7fa6391b6640 10 monclient: build_initial_monmap 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.988+0000 7fa6391b6640 10 monclient: monmap: 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:18.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.988+0000 7fa6391b6640 1 Processor -- start 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.988+0000 7fa6391b6640 1 -- start start 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.988+0000 7fa6391b6640 10 monclient: init 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.988+0000 7fa6391b6640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.988+0000 7fa6391b6640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa63406c890 0x7fa6341cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 10 monclient(hunting): picked mon.noname-b con 0x7fa63406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 0x7fa63409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 10 monclient(hunting): picked mon.noname-a con 0x7fa63409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa634134df0 0x7fa63409e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 10 monclient(hunting): picked mon.noname-c con 0x7fa634134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6340a01a0 con 0x7fa63409c4a0 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa6340a0020 con 0x7fa63406c890 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa63409be20 con 0x7fa634134df0 2026-02-17T05:56:18.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa6391b6640 10 monclient(hunting): authenticate will time out at 1224.543044s 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 0x7fa63409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 0x7fa63409db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40990/0 (socket says 10.20.193.78:40990) 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 -- 10.20.193.78:0/3767147158 learned_addr learned my addr 10.20.193.78:0/3767147158 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient(hunting): get_auth_request con 0x7fa63409c4a0 auth_method 0 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa633577640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa634134df0 0x7fa63409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa633577640 10 monclient(hunting): get_auth_request con 0x7fa634134df0 auth_method 0 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa633577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa633577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa633577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa633577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa633577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa633577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient(hunting): handle_auth_done global_id 124236 payload 824 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa634134df0 msgr2=0x7fa63409e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa634134df0 0x7fa63409e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa63406c890 msgr2=0x7fa6341cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa63406c890 0x7fa6341cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient: _finish_hunting 0 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 monclient: found mon.noname-a 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.989+0000 7fa632575640 1 -- 10.20.193.78:0/3767147158 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63409d810 con 0x7fa63409c4a0 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa632575640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 msgr2=0x7fa63409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa632575640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 0x7fa63409db50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa61c002410 tx=0x7fa61c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c03c040 con 0x7fa63409c4a0 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: dump: 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.001 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c047070 con 0x7fa63409c4a0 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6391b6640 5 monclient: authenticate success, global_id 124236 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6391b6640 10 monclient: _renew_subs 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6391b6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63409f7f0 con 0x7fa63409c4a0 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c040a40 con 0x7fa63409c4a0 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:19.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa631573640 4 set_mon_vals no callback set 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6137fe640 10 monclient: dump: 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6391b6640 10 monclient: _renew_subs 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6391b6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.991+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa634126fb0 con 0x7fa63409c4a0 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa6391b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa6391b6640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6341cfd50 0x7fa63411cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa6391b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa6340a1810 con 0x7fa6341cfd50 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa61c03f030 con 0x7fa63409c4a0 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6341cfd50 0x7fa63411cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 10 monclient: get_auth_request con 0x7fa6341cfd50 auth_method 0 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa6137fe640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa60003df30 0x7fa6000403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 10 monclient(hunting): handle_auth_done global_id 74672 payload 824 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(367..367 src has 117..367) ==== 4799+0+0 (secure 0 0 0) 0x7fa61c077510 con 0x7fa63409c4a0 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa632d76640 1 -- 10.20.193.78:0/3767147158 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa634074980 con 0x7fa6341cfd50 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa633577640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa60003df30 0x7fa6000403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.992+0000 7fa633577640 10 monclient: get_auth_request con 0x7fa60003df30 auth_method 0 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa632d76640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6341cfd50 msgr2=0x7fa63411cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa632d76640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6341cfd50 0x7fa63411cc70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e800 tx=0x7fa62800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628017070 con 0x7fa6341cfd50 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 10 monclient: dump: 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.004 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa62800cb40 con 0x7fa6341cfd50 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6341cfd50 msgr2=0x7fa63411cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6341cfd50 0x7fa63411cc70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e800 tx=0x7fa62800ecd0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6341cfd50 msgr2=0x7fa63411cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa6137fe640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6341cfd50 0x7fa63411cc70 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e800 tx=0x7fa62800ecd0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.994+0000 7fa633577640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa60003df30 0x7fa6000403f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa62400f2e0 tx=0x7fa624004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.997+0000 7fa6391b6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.997+0000 7fa6391b6640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63411d1b0 0x7fa6341da430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.997+0000 7fa6391b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.997+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa634128250 con 0x7fa63411d1b0 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63411d1b0 0x7fa6341da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 10 monclient: get_auth_request con 0x7fa63411d1b0 auth_method 0 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 10 monclient(hunting): handle_auth_done global_id 74675 payload 824 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:18.998+0000 7fa632d76640 1 -- 10.20.193.78:0/3767147158 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa6340744d0 con 0x7fa63411d1b0 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa632d76640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63411d1b0 msgr2=0x7fa6341da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa632d76640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63411d1b0 0x7fa6341da430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa628018000 tx=0x7fa6280047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628017040 con 0x7fa63411d1b0 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 10 monclient: dump: 2026-02-17T05:56:19.007 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fa628003c70 con 0x7fa63411d1b0 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63411d1b0 msgr2=0x7fa6341da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63411d1b0 0x7fa6341da430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa628018000 tx=0x7fa6280047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63411d1b0 msgr2=0x7fa6341da430 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:19.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6137fe640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa63411d1b0 0x7fa6341da430 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa628018000 tx=0x7fa6280047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 112, 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 350892, 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:19.009 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:56:19.010 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:19.011 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:19.012 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.013 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.014 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:56:19.015 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:56:19.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa60003df30 msgr2=0x7fa6000403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa60003df30 0x7fa6000403f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa62400f2e0 tx=0x7fa624004200 comp rx=0 tx=0).stop 2026-02-17T05:56:19.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 10 monclient: shutdown 2026-02-17T05:56:19.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:19.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 msgr2=0x7fa63409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 1 --2- 10.20.193.78:0/3767147158 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa63409c4a0 0x7fa63409db50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa61c002410 tx=0x7fa61c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa633577640 1 -- 10.20.193.78:0/3767147158 reap_dead start 2026-02-17T05:56:19.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa633577640 1 -- 10.20.193.78:0/3767147158 reap_dead start 2026-02-17T05:56:19.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 shutdown_connections 2026-02-17T05:56:19.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 >> 10.20.193.78:0/3767147158 conn(0x7fa6340a6620 msgr2=0x7fa634072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:19.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 shutdown_connections 2026-02-17T05:56:19.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.000+0000 7fa6391b6640 1 -- 10.20.193.78:0/3767147158 wait complete. 2026-02-17T05:56:19.017 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:56:19.017 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:56:19.017 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:56:19.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient: get_monmap_and_config 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient: build_initial_monmap 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient: monmap: 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:19.085462+0000 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:19.085462+0000 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:19.084 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 1 Processor -- start 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 1 -- start start 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient: init 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac06c890 0x7f57ac06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient(hunting): picked mon.noname-b con 0x7f57ac06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f57ac09d430 0x7f57ac06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient(hunting): picked mon.noname-c con 0x7f57ac09d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f57ac09bf40 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57ac073600 con 0x7f57ac09bf40 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f57ac09ca50 con 0x7f57ac09d430 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f57ac06d6f0 con 0x7f57ac06c890 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.084+0000 7f57b3a76640 10 monclient(hunting): authenticate will time out at 1224.638968s 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b17eb640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac06c890 msgr2=0x7f57ac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b17eb640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac06c890 msgr2=0x7f57ac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b17eb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac06c890 0x7f57ac06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b1fec640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f57ac09bf40 0x7f57ac134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:43860/0 (socket says 10.20.193.78:43860) 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b1fec640 1 -- 10.20.193.78:0/3056616452 learned_addr learned my addr 10.20.193.78:0/3056616452 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b17eb640 1 --2- 10.20.193.78:0/3056616452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac06c890 0x7f57ac06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3056616452 >> v2:10.20.193.115:3300/0 conn(0x7f57ac09d430 0x7f57ac06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient(hunting): get_auth_request con 0x7f57ac09d430 auth_method 0 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient(hunting): handle_auth_done global_id 134101 payload 274 2026-02-17T05:56:19.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 1 -- 10.20.193.78:0/3056616452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac06c890 msgr2=0x7f57ac06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:19.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3056616452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac06c890 0x7f57ac06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 1 -- 10.20.193.78:0/3056616452 >> v1:10.20.193.78:6789/0 conn(0x7f57ac09bf40 legacy=0x7f57ac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:19.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient: _finish_hunting 0 2026-02-17T05:56:19.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 1 monclient: found mon.noname-c 2026-02-17T05:56:19.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:19.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:56:19.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.085+0000 7f57b0fea640 1 -- 10.20.193.78:0/3056616452 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac06dab0 con 0x7f57ac09d430 2026-02-17T05:56:19.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b0fea640 1 -- 10.20.193.78:0/3056616452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 msgr2=0x7f57ac06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3056616452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 0x7f57ac06d1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0a7600 tx=0x7f57a002ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1437727db79de8d0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 1 -- 10.20.193.78:0/3056616452 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a003c070 con 0x7f57ac09d430 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: dump: 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: _finish_auth 0 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 1 -- 10.20.193.78:0/3056616452 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a002fcd0 con 0x7f57ac09d430 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 5 monclient: authenticate success, global_id 134101 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 1 -- 10.20.193.78:0/3056616452 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0037660 con 0x7f57ac09d430 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 4 set_mon_vals no callback set 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f579bfff640 10 monclient: dump: 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.088 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 10 monclient: shutdown 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 1 -- 10.20.193.78:0/3056616452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 msgr2=0x7f57ac06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 1 --2- 10.20.193.78:0/3056616452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 0x7f57ac06d1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0a7600 tx=0x7f57a002ed70 comp rx=0 tx=0).stop 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 1 -- 10.20.193.78:0/3056616452 shutdown_connections 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 1 --2- 10.20.193.78:0/3056616452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 0x7f57ac06d1b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 1 --2- 10.20.193.78:0/3056616452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac06c890 0x7f57ac06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 1 -- 10.20.193.78:0/3056616452 >> 10.20.193.78:0/3056616452 conn(0x7f57ac0a6620 msgr2=0x7f57ac0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 1 -- 10.20.193.78:0/3056616452 shutdown_connections 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.088+0000 7f57b3a76640 1 -- 10.20.193.78:0/3056616452 wait complete. 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient: build_initial_monmap 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient: monmap: 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 1 Processor -- start 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 1 -- start start 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient: init 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f57ac06c890 0x7f57ac1cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient(hunting): picked mon.noname-a con 0x7f57ac06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac09bf40 0x7f57ac1cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient(hunting): picked mon.noname-c con 0x7f57ac09bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 0x7f57ac121c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient(hunting): picked mon.noname-b con 0x7f57ac09d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac0733c0 con 0x7f57ac06c890 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac073240 con 0x7f57ac09d430 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f57ac135500 con 0x7f57ac09bf40 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b3a76640 10 monclient(hunting): authenticate will time out at 1224.643728s 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f57ac06c890 0x7f57ac1cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b0fea640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac09bf40 0x7f57ac1cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f57ac06c890 0x7f57ac1cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40992/0 (socket says 10.20.193.78:40992) 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 -- 10.20.193.78:0/3711798297 learned_addr learned my addr 10.20.193.78:0/3711798297 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient(hunting): get_auth_request con 0x7f57ac06c890 auth_method 0 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b0fea640 10 monclient(hunting): get_auth_request con 0x7f57ac09bf40 auth_method 0 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b0fea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b0fea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b0fea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b1fec640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 0x7f57ac121c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b1fec640 10 monclient(hunting): get_auth_request con 0x7f57ac09d430 auth_method 0 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b1fec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b1fec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b1fec640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient(hunting): handle_auth_done global_id 124239 payload 824 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac09bf40 msgr2=0x7f57ac1cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f57ac09bf40 0x7f57ac1cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 msgr2=0x7f57ac121c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac09d430 0x7f57ac121c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient: _finish_hunting 0 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 monclient: found mon.noname-a 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.089+0000 7f57b17eb640 1 -- 10.20.193.78:0/3711798297 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac123d70 con 0x7f57ac06c890 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b17eb640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f57ac06c890 msgr2=0x7f57ac1cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b17eb640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f57ac06c890 0x7f57ac1cf5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f579c00b780 tx=0x7f579c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c0026e0 con 0x7f57ac06c890 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: dump: 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579c010070 con 0x7f57ac06c890 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 5 monclient: authenticate success, global_id 124239 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 10 monclient: _renew_subs 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ac1255d0 con 0x7f57ac06c890 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c002ae0 con 0x7f57ac06c890 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f579b7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 10 monclient: dump: 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 10 monclient: _renew_subs 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ac125b90 con 0x7f57ac06c890 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 0x7f57ac122560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac135930 con 0x7f57ac122180 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.092+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f579c01d030 con 0x7f57ac06c890 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 0x7f57ac122560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 10 monclient: get_auth_request con 0x7f57ac122180 auth_method 0 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f5799ffb640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f577c03deb0 0x7f577c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(367..367 src has 117..367) ==== 4799+0+0 (secure 0 0 0) 0x7f579c051d40 con 0x7f57ac06c890 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 10 monclient(hunting): handle_auth_done global_id 134107 payload 824 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 1 -- 10.20.193.78:0/3711798297 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57ac1253b0 con 0x7f57ac122180 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b1fec640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f577c03deb0 0x7f577c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 msgr2=0x7f57ac122560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 msgr2=0x7f57ac122560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 0x7f57ac122560 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f57ac09c670 tx=0x7f57a00047c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 0x7f57ac122560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:19.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.093+0000 7f57b1fec640 10 monclient: get_auth_request con 0x7f577c03deb0 auth_method 0 2026-02-17T05:56:19.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.096+0000 7f57b1fec640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f577c03deb0 0x7f577c040370 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57ac09c360 tx=0x7f57a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.294+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 0x7f57ac122560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.294+0000 7f57b0fea640 10 monclient: get_auth_request con 0x7f57ac122180 auth_method 0 2026-02-17T05:56:19.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.294+0000 7f57b0fea640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.294+0000 7f57b0fea640 10 monclient: _init_auth method 2 2026-02-17T05:56:19.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.294+0000 7f57b0fea640 10 monclient: _init_auth already have auth, reseting 2026-02-17T05:56:19.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.294+0000 7f57b0fea640 10 monclient: handle_auth_reply_more payload 9 2026-02-17T05:56:19.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.294+0000 7f57b0fea640 10 monclient: handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.294+0000 7f57b0fea640 10 monclient: handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:19.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.295+0000 7f57b0fea640 10 monclient: handle_auth_done global_id 134107 payload 293 2026-02-17T05:56:19.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f57b0fea640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 msgr2=0x7f57ac122560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 0x7f57ac122560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f57ac09c670 tx=0x7f57a0033000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0053070 con 0x7f57ac122180 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 10 monclient: dump: 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.297 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f57a003d040 con 0x7f57ac122180 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 msgr2=0x7f57ac122560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 0x7f57ac122560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f57ac09c670 tx=0x7f57a0033000 comp rx=0 tx=0).stop 2026-02-17T05:56:19.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 msgr2=0x7f57ac122560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:19.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.297+0000 7f5799ffb640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122180 0x7f57ac122560 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f57ac09c670 tx=0x7f57a0033000 comp rx=0 tx=0).stop 2026-02-17T05:56:19.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b3a76640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b3a76640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122aa0 0x7f57ac1da640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b3a76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac0744b0 con 0x7f57ac122aa0 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122aa0 0x7f57ac1da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 10 monclient: get_auth_request con 0x7f57ac122aa0 auth_method 0 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 10 monclient(hunting): handle_auth_done global_id 134110 payload 824 2026-02-17T05:56:19.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.301+0000 7f57b0fea640 1 -- 10.20.193.78:0/3711798297 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f57ac1d0050 con 0x7f57ac122aa0 2026-02-17T05:56:19.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b0fea640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122aa0 msgr2=0x7f57ac1da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b0fea640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122aa0 0x7f57ac1da640 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f57a0002410 tx=0x7f57a0008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0053070 con 0x7f57ac122aa0 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 10 monclient: dump: 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.304 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4707 (secure 0 0 0) 0x7f57a0054040 con 0x7f57ac122aa0 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:19.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122aa0 msgr2=0x7f57ac1da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122aa0 0x7f57ac1da640 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f57a0002410 tx=0x7f57a0008030 comp rx=0 tx=0).stop 2026-02-17T05:56:19.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122aa0 msgr2=0x7f57ac1da640 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:19.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f5799ffb640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f57ac122aa0 0x7f57ac1da640 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f57a0002410 tx=0x7f57a0008030 comp rx=0 tx=0).stop 2026-02-17T05:56:19.306 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:56:19.306 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:19.306 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 112, 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 7021, 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:56:19.307 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:19.308 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:19.309 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:19.310 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:19.311 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:56:19.312 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.313 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:19.314 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.315 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:19.316 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.317 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:19.318 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.319 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:56:19.320 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:56:19.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f577c03deb0 msgr2=0x7f577c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f577c03deb0 0x7f577c040370 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57ac09c360 tx=0x7f57a8009040 comp rx=0 tx=0).stop 2026-02-17T05:56:19.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 10 monclient: shutdown 2026-02-17T05:56:19.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:19.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f57ac06c890 msgr2=0x7f57ac1cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 1 --2- 10.20.193.78:0/3711798297 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f57ac06c890 0x7f57ac1cf5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f579c00b780 tx=0x7f579c00bc50 comp rx=0 tx=0).stop 2026-02-17T05:56:19.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b1fec640 1 -- 10.20.193.78:0/3711798297 reap_dead start 2026-02-17T05:56:19.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b1fec640 1 -- 10.20.193.78:0/3711798297 reap_dead start 2026-02-17T05:56:19.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 shutdown_connections 2026-02-17T05:56:19.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 >> 10.20.193.78:0/3711798297 conn(0x7f57ac0a6620 msgr2=0x7f57ac137580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:19.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 shutdown_connections 2026-02-17T05:56:19.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.304+0000 7f57b3a76640 1 -- 10.20.193.78:0/3711798297 wait complete. 2026-02-17T05:56:19.323 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:56:19.323 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:56:19.323 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient: get_monmap_and_config 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient: build_initial_monmap 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient: monmap: 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:19.392425+0000 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:19.392425+0000 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:19.391 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 1 Processor -- start 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 1 -- start start 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient: init 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient: _add_conns ranks=[2,0,3,1,4] 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient(hunting): picked mon.noname-b con 0x7f1cc006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1cc009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1cc006d1b0 0x7f1cc0135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient(hunting): picked mon.noname-a con 0x7f1cc006d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cc006daf0 con 0x7f1cc009c4a0 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f1cc00a0880 con 0x7f1cc006d1b0 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc009cc80 con 0x7f1cc006c890 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.392+0000 7f1cc4b14640 10 monclient(hunting): authenticate will time out at 1224.945962s 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1cc006d1b0 0x7f1cc0135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbdd74640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f1cc009c4a0 0x7f1cc0134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:43862/0 (socket says 10.20.193.78:43862) 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1cc006d1b0 0x7f1cc0135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:41004/0 (socket says 10.20.193.78:41004) 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33292/0 (socket says 10.20.193.78:33292) 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbdd74640 1 -- 10.20.193.78:0/1345646754 learned_addr learned my addr 10.20.193.78:0/1345646754 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient(hunting): get_auth_request con 0x7f1cc006d1b0 auth_method 0 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 10 monclient(hunting): get_auth_request con 0x7f1cc006c890 auth_method 0 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient(hunting): handle_auth_done global_id 124242 payload 274 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 1 -- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 msgr2=0x7f1cc006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 1 --2- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 1 -- 10.20.193.78:0/1345646754 >> v1:10.20.193.78:6789/0 conn(0x7f1cc009c4a0 legacy=0x7f1cc0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 1 monclient: found mon.noname-a 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbed76640 1 -- 10.20.193.78:0/1345646754 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc009cf60 con 0x7f1cc006d1b0 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.393+0000 7f1cbe575640 1 --2- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbed76640 1 -- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc006d1b0 msgr2=0x7f1cc0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbed76640 1 --2- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc006d1b0 0x7f1cc0135500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009d90 tx=0x7f1cb002f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e45e59fe4b69723 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 1 -- 10.20.193.78:0/1345646754 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb003c070 con 0x7f1cc006d1b0 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: dump: 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: _finish_auth 0 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 1 -- 10.20.193.78:0/1345646754 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb0037440 con 0x7f1cc006d1b0 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cc4b14640 5 monclient: authenticate success, global_id 124242 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 1 -- 10.20.193.78:0/1345646754 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0037810 con 0x7f1cc006d1b0 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cc4b14640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cc4b14640 4 set_mon_vals no callback set 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.395+0000 7f1cbd573640 10 monclient: dump: 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.395 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.396 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.396 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.396 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.396 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.396 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 10 monclient: shutdown 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 1 -- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc006d1b0 msgr2=0x7f1cc0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc006d1b0 0x7f1cc0135500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009d90 tx=0x7f1cb002f150 comp rx=0 tx=0).stop 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 1 -- 10.20.193.78:0/1345646754 shutdown_connections 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc006d1b0 0x7f1cc0135500 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/1345646754 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 1 -- 10.20.193.78:0/1345646754 >> 10.20.193.78:0/1345646754 conn(0x7f1cc00a6620 msgr2=0x7f1cc00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 1 -- 10.20.193.78:0/1345646754 shutdown_connections 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 1 -- 10.20.193.78:0/1345646754 wait complete. 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 10 monclient: build_initial_monmap 2026-02-17T05:56:19.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.396+0000 7f1cc4b14640 10 monclient: monmap: 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 1 Processor -- start 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 1 -- start start 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient: init 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc01cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient(hunting): picked mon.noname-c con 0x7f1cc006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1cc006d1b0 0x7f1cc0124c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient(hunting): picked mon.noname-b con 0x7f1cc006d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc009c4a0 0x7f1cc01251d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient(hunting): picked mon.noname-a con 0x7f1cc009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00a01a0 con 0x7f1cc009c4a0 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00a0020 con 0x7f1cc006d1b0 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc009be20 con 0x7f1cc006c890 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cc4b14640 10 monclient(hunting): authenticate will time out at 1224.950362s 2026-02-17T05:56:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc01cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc009c4a0 0x7f1cc01251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc01cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33304/0 (socket says 10.20.193.78:33304) 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 1 -- 10.20.193.78:0/2826996261 learned_addr learned my addr 10.20.193.78:0/2826996261 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc009c4a0 0x7f1cc01251d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:41008/0 (socket says 10.20.193.78:41008) 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 10 monclient(hunting): get_auth_request con 0x7f1cc006c890 auth_method 0 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient(hunting): get_auth_request con 0x7f1cc009c4a0 auth_method 0 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient(hunting): handle_auth_done global_id 124245 payload 824 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 1 -- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 msgr2=0x7f1cc01cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc01cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 1 -- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1cc006d1b0 msgr2=0x7f1cc0124c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1cc006d1b0 0x7f1cc0124c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 1 monclient: found mon.noname-a 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbed76640 1 -- 10.20.193.78:0/2826996261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc0128bd0 con 0x7f1cc009c4a0 2026-02-17T05:56:19.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.397+0000 7f1cbe575640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc01cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:56:19.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cbed76640 1 -- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc009c4a0 msgr2=0x7f1cc01251d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:19.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cbed76640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc009c4a0 0x7f1cc01251d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0036000 tx=0x7f1cb0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 1 -- 10.20.193.78:0/2826996261 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb003c050 con 0x7f1cc009c4a0 2026-02-17T05:56:19.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: dump: 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 1 -- 10.20.193.78:0/2826996261 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb0048070 con 0x7f1cc009c4a0 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 1 -- 10.20.193.78:0/2826996261 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0041c60 con 0x7f1cc009c4a0 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cbcd72640 4 set_mon_vals no callback set 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1ca6ffd640 10 monclient: dump: 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:19.401 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cc4b14640 5 monclient: authenticate success, global_id 124245 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cc4b14640 10 monclient: _renew_subs 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cc4b14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc0136e10 con 0x7f1cc009c4a0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cc4b14640 10 monclient: _renew_subs 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cc4b14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.399+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc0128610 con 0x7f1cc009c4a0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1cc4b14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1cc4b14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1cc4b14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc0074980 con 0x7f1cc009c4a0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1ca6ffd640 1 -- 10.20.193.78:0/2826996261 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1cb004d460 con 0x7f1cc009c4a0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1ca6ffd640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c8c03df00 0x7f1c8c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1ca6ffd640 1 -- 10.20.193.78:0/2826996261 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(367..367 src has 117..367) ==== 4799+0+0 (secure 0 0 0) 0x7f1cb0078560 con 0x7f1cc009c4a0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1cbe575640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c8c03df00 0x7f1c8c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.400+0000 7f1cbe575640 10 monclient: get_auth_request con 0x7f1c8c03df00 auth_method 0 2026-02-17T05:56:19.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.403+0000 7f1ca6ffd640 1 -- 10.20.193.78:0/2826996261 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1cb00423b0 con 0x7f1cc009c4a0 2026-02-17T05:56:19.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.403+0000 7f1ca6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:19.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.403+0000 7f1ca6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:19.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.403+0000 7f1cbe575640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c8c03df00 0x7f1c8c0403c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40114b0 tx=0x7f1cb4009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:19.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.523+0000 7f1cc4b14640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:56:19.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.523+0000 7f1cc4b14640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:56:19.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.523+0000 7f1cc4b14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:19.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.523+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f1cc006cc70 con 0x7f1cc009c4a0 2026-02-17T05:56:19.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.524+0000 7f1ca6ffd640 1 -- 10.20.193.78:0/2826996261 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f1cb0041e00 con 0x7f1cc009c4a0 2026-02-17T05:56:19.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.524+0000 7f1ca6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:56:19.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.524+0000 7f1ca6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:19.525 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.525 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=2,health=9,kv=11,logm=59} crc {auth=3045349933,config=692619927,health=411307251,kv=745643123,logm=3830796435}) 2026-02-17T05:56:19.525 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=11,kv=12,logm=55} crc {auth=934739242,config=692619927,health=2132485895,kv=449989585,logm=2931601812}) 2026-02-17T05:56:19.525 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.525 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=2,health=9,kv=11,logm=59} crc {auth=3045349933,config=692619927,health=411307251,kv=745643123,logm=3830796435}) 2026-02-17T05:56:19.526 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=19,config=2,health=10,kv=9,logm=60} crc {auth=3617298327,config=692619927,health=3714123964,kv=303214794,logm=4098393880}) 2026-02-17T05:56:19.526 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.526 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=25272342}) 2026-02-17T05:56:19.526 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=731860697}) 2026-02-17T05:56:19.526 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.524+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.526 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=25272342}) 2026-02-17T05:56:19.526 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137722550}) 2026-02-17T05:56:19.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c8c03df00 msgr2=0x7f1c8c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c8c03df00 0x7f1c8c0403c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40114b0 tx=0x7f1cb4009290 comp rx=0 tx=0).stop 2026-02-17T05:56:19.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 10 monclient: shutdown 2026-02-17T05:56:19.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:19.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc009c4a0 msgr2=0x7f1cc01251d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:19.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc009c4a0 0x7f1cc01251d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0036000 tx=0x7f1cb0004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:19.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 shutdown_connections 2026-02-17T05:56:19.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1cc009c4a0 0x7f1cc01251d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c8c03df00 0x7f1c8c0403c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1cc006d1b0 0x7f1cc0124c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 --2- 10.20.193.78:0/2826996261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1cc006c890 0x7f1cc01cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:19.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 >> 10.20.193.78:0/2826996261 conn(0x7f1cc00a6620 msgr2=0x7f1cc00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:19.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 shutdown_connections 2026-02-17T05:56:19.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:19.525+0000 7f1cc4b14640 1 -- 10.20.193.78:0/2826996261 wait complete. 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4207137418}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=651635805}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4207137418}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4058977247}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4127915283}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=697970568}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.525+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4127915283}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=108762079}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875718633}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=629447480}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875718633}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2911553507}) 2026-02-17T05:56:19.535 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161871656}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2057319569}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161871656}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.526+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2785898579}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3231106170}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=25128672}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3231106170}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236923028}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3247310929}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=990763450}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3247310929}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027644690}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=275823572}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1942542438}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=275823572}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.527+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1045851914}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4171819512}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2168983966}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4171819512}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2748525259}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1194902558}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3049817388}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1194902558}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=579103291}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49575709}) 2026-02-17T05:56:19.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4017128044}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49575709}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.528+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4125293320}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2162339300}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=852111351}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2162339300}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1203766949}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323701300}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1529332908}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323701300}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3177673400}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2265807373}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2677843731}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2265807373}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3544950693}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3329061308}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1597513415}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3329061308}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.529+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389188095}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3093649717}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=879629334}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3093649717}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2725224580}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4256742632}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054671771}) 2026-02-17T05:56:19.537 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4256742632}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2069689213}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2013875863}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=604545556}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2013875863}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.530+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2557304914}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212618323}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=290957538}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212618323}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1058821006}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=910200417}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1511365509}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=910200417}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4025708118}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=110245412}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2359937207}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=110245412}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.531+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=646643091}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3455273847}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2093973236}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3455273847}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2011455786}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2817105451}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3004182033}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2817105451}) 2026-02-17T05:56:19.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4169508726}) 2026-02-17T05:56:19.539 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3983517721}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3605074341}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3983517721}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2888555041}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1919790068}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932701575}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1919790068}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.532+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1565780277}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2428712804}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=692674693}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2428712804}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1671615009}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=516732859}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3005689251}) 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=516732859}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.533+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1204818627}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=50,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=8} crc {logm=3669664522,mdsmap=1160443688,mgr=69100650,mgr_command_descs=1238205436,mgr_metadata=1258482571,mgrstat=3898581683,mon_config_key=1987855637,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=294973011,osd_snap=2585644601}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=50,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=7} crc {logm=4166018707,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=622407678,mon_config_key=501681677,monmap=1784110042,osd_metadata=1501630441,osd_pg_creating=294973011,osd_snap=396442997}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=50,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=8} crc {logm=3669664522,mdsmap=1160443688,mgr=69100650,mgr_command_descs=1238205436,mgr_metadata=1258482571,mgrstat=3898581683,mon_config_key=1987855637,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=294973011,osd_snap=2585644601}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=50,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=7} crc {logm=2653464387,mdsmap=1730306971,mgr=787438293,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4071375962,mon_config_key=1912222953,monmap=1784110042,osd_metadata=1246192198,osd_pg_creating=294973011,osd_snap=785252622}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=53,osdmap=47} crc {osd_snap=724683859,osdmap=2525885196}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=54,osdmap=46} crc {osd_snap=3797853922,osdmap=286453025}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=53,osdmap=47} crc {osd_snap=724683859,osdmap=2525885196}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=56,osdmap=44} crc {osd_snap=2295781098,osdmap=980298827}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=61804252}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1871822800}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=61804252}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.534+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=165154378}) 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=398193213}) 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4046360870}) 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=398193213}) 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3383306415}) 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1118072910}) 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2882614050}) 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1118072910}) 2026-02-17T05:56:19.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.542+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2237855257}) 2026-02-17T05:56:19.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.545+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.545+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=34} crc {osdmap=2036686086}) 2026-02-17T05:56:19.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.545+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=35} crc {osdmap=189516790}) 2026-02-17T05:56:19.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.545+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:56:19.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.545+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=34} crc {osdmap=2036686086}) 2026-02-17T05:56:19.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:56:19.545+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=35} crc {osdmap=189516790}) 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1759: done (6 left) 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1760: rollback oid 7 current snap is 173 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 166 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1753: finishing copy_from to trial11545420-37 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1753: got expected ENOENT (src dne) 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1753: finishing copy_from racing read to trial11545420-37 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1755: finishing copy_from to trial11545420-30 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1755: got expected ENOENT (src dne) 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1755: finishing copy_from racing read to trial11545420-30 2026-02-17T05:56:19.849 INFO:tasks.rados.rados.0.trial115.stdout:1757: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:56:19.915 INFO:tasks.rados.rados.0.trial115.stdout:1756: expect (ObjNum 624 snap 162 seq_num 624) 2026-02-17T05:56:19.974 INFO:tasks.rados.rados.0.trial115.stdout:1758: finishing write tid 1 to trial11545420-9 2026-02-17T05:56:19.974 INFO:tasks.rados.rados.0.trial115.stdout:1754: finishing copy_from racing read to trial11545420-25 2026-02-17T05:56:19.974 INFO:tasks.rados.rados.0.trial115.stdout:1754: finishing copy_from to trial11545420-25 2026-02-17T05:56:19.974 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 646 (ObjNum 616 snap 161 seq_num 616) dirty exists 2026-02-17T05:56:19.974 INFO:tasks.rados.rados.0.trial115.stdout:1758: finishing write tid 2 to trial11545420-9 2026-02-17T05:56:19.974 INFO:tasks.rados.rados.0.trial115.stdout:1758: finishing write tid 3 to trial11545420-9 2026-02-17T05:56:19.974 INFO:tasks.rados.rados.0.trial115.stdout:1758: finishing write tid 4 to trial11545420-9 2026-02-17T05:56:19.974 INFO:tasks.rados.rados.0.trial115.stdout:1758: oid 9 updating version 0 to 852 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1758: oid 9 version 852 is already newer than 851 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1758: oid 9 version 852 is already newer than 850 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 852 (ObjNum 646 snap 173 seq_num 646) dirty exists 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1758: left oid 9 (ObjNum 646 snap 173 seq_num 646) 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1760: finishing rollback tid 0 to trial11545420-7 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1760: finishing rollback tid 1 to trial11545420-7 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1760: finishing rollback tid 2 to trial11545420-7 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 654 (ObjNum 532 snap 136 seq_num 532) dirty exists 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1753: done (6 left) 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1754: done (5 left) 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1755: done (4 left) 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1756: done (3 left) 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1757: done (2 left) 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1758: done (1 left) 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1760: done (0 left) 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1761: write oid 23 current snap is 173 2026-02-17T05:56:19.975 INFO:tasks.rados.rados.0.trial115.stdout:1761: seq_num 647 ranges {474129=576108,1627732=587214,2718088=156041} 2026-02-17T05:56:19.978 INFO:tasks.rados.rados.0.trial115.stdout:1761: writing trial11545420-23 from 474129 to 1050237 tid 1 2026-02-17T05:56:19.982 INFO:tasks.rados.rados.0.trial115.stdout:1761: writing trial11545420-23 from 1627732 to 2214946 tid 2 2026-02-17T05:56:19.985 INFO:tasks.rados.rados.0.trial115.stdout:1761: writing trial11545420-23 from 2718088 to 2874129 tid 3 2026-02-17T05:56:19.985 INFO:tasks.rados.rados.0.trial115.stdout:1762: write (excl) oid 13 current snap is 173 2026-02-17T05:56:19.985 INFO:tasks.rados.rados.0.trial115.stdout:1762: seq_num 648 ranges {490936=651357,1686214=447225,2861864=29072} 2026-02-17T05:56:19.989 INFO:tasks.rados.rados.0.trial115.stdout:1762: writing trial11545420-13 from 490936 to 1142293 tid 1 2026-02-17T05:56:19.993 INFO:tasks.rados.rados.0.trial115.stdout:1762: writing trial11545420-13 from 1686214 to 2133439 tid 2 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1762: writing trial11545420-13 from 2861864 to 2890936 tid 3 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1763: read oid 30 snap -1 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1763: expect deleted 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1761: finishing write tid 1 to trial11545420-23 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1764: read oid 32 snap -1 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1764: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1765: snap_create 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1761: finishing write tid 2 to trial11545420-23 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1761: finishing write tid 3 to trial11545420-23 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1761: finishing write tid 4 to trial11545420-23 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1761: finishing write tid 5 to trial11545420-23 2026-02-17T05:56:19.996 INFO:tasks.rados.rados.0.trial115.stdout:1761: oid 23 updating version 0 to 370 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1761: oid 23 updating version 370 to 371 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1761: oid 23 updating version 371 to 372 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1761: oid 23 version 372 is already newer than 369 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 372 (ObjNum 647 snap 173 seq_num 647) dirty exists 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1761: left oid 23 (ObjNum 647 snap 173 seq_num 647) 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: finishing write tid 1 to trial11545420-13 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: finishing write tid 2 to trial11545420-13 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: finishing write tid 3 to trial11545420-13 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: finishing write tid 4 to trial11545420-13 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: finishing write tid 5 to trial11545420-13 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: oid 13 updating version 0 to 636 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: oid 13 updating version 636 to 637 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: oid 13 updating version 637 to 639 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: oid 13 version 639 is already newer than 638 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 639 (ObjNum 648 snap 173 seq_num 648) dirty exists 2026-02-17T05:56:19.997 INFO:tasks.rados.rados.0.trial115.stdout:1762: left oid 13 (ObjNum 648 snap 173 seq_num 648) 2026-02-17T05:56:20.001 INFO:tasks.rados.rados.0.trial115.stdout:1764: expect (ObjNum 572 snap 147 seq_num 572) 2026-02-17T05:56:20.540 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:56:20.540 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:20.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.607+0000 7ff4a816e640 10 monclient: get_monmap_and_config 2026-02-17T05:56:20.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.607+0000 7ff4a816e640 10 monclient: build_initial_monmap 2026-02-17T05:56:20.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.607+0000 7ff4a816e640 10 monclient: monmap: 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:20.608596+0000 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:20.608596+0000 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:20.611 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.607+0000 7ff4a816e640 1 Processor -- start 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.607+0000 7ff4a816e640 1 -- start start 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.607+0000 7ff4a816e640 10 monclient: init 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient(hunting): picked mon.noname-b con 0x7ff4a006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff4a006cc70 0x7ff4a006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient(hunting): picked mon.noname-c con 0x7ff4a006cc70 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:20.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff4a0134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4a0074150 con 0x7ff4a0134df0 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff4a009ef70 con 0x7ff4a006cc70 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a009f250 con 0x7ff4a006c890 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a816e640 10 monclient(hunting): authenticate will time out at 1226.162157s 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a00a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33312/0 (socket says 10.20.193.78:33312) 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/1744030387 learned_addr learned my addr 10.20.193.78:0/1744030387 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient(hunting): get_auth_request con 0x7ff4a006c890 auth_method 0 2026-02-17T05:56:20.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a56e2640 1 --2- 10.20.193.78:0/1744030387 >> v2:10.20.193.115:3300/0 conn(0x7ff4a006cc70 0x7ff4a006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a56e2640 10 monclient(hunting): get_auth_request con 0x7ff4a006cc70 auth_method 0 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a56e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a56e2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a56e2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient(hunting): handle_auth_done global_id 74687 payload 274 2026-02-17T05:56:20.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/1744030387 >> v2:10.20.193.115:3300/0 conn(0x7ff4a006cc70 msgr2=0x7ff4a006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:20.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 1 --2- 10.20.193.78:0/1744030387 >> v2:10.20.193.115:3300/0 conn(0x7ff4a006cc70 0x7ff4a006d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/1744030387 >> v1:10.20.193.115:6789/0 conn(0x7ff4a0134df0 legacy=0x7ff4a0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:20.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient: _finish_hunting 0 2026-02-17T05:56:20.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 1 monclient: found mon.noname-b 2026-02-17T05:56:20.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:20.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/1744030387 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a009df40 con 0x7ff4a006c890 2026-02-17T05:56:20.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.608+0000 7ff4a56e2640 1 --2- 10.20.193.78:0/1744030387 >> v2:10.20.193.115:3300/0 conn(0x7ff4a006cc70 0x7ff4a006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:56:20.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/1744030387 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 msgr2=0x7ff4a00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a5ee3640 1 --2- 10.20.193.78:0/1744030387 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a00a01f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff490009970 tx=0x7ff49002efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b16ed79d027d9c5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 1 -- 10.20.193.78:0/1744030387 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49003c070 con 0x7ff4a006c890 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: dump: 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.623 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: _finish_auth 0 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: _check_auth_tickets 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 1 -- 10.20.193.78:0/1744030387 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff49002fdf0 con 0x7ff4a006c890 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 5 monclient: authenticate success, global_id 74687 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 1 -- 10.20.193.78:0/1744030387 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490037660 con 0x7ff4a006c890 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:20.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 4 set_mon_vals no callback set 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a4ee1640 10 monclient: dump: 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.625 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 10 monclient: shutdown 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 -- 10.20.193.78:0/1744030387 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 msgr2=0x7ff4a00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 --2- 10.20.193.78:0/1744030387 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a00a01f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff490009970 tx=0x7ff49002efb0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 -- 10.20.193.78:0/1744030387 shutdown_connections 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 --2- 10.20.193.78:0/1744030387 >> v2:10.20.193.115:3300/0 conn(0x7ff4a006cc70 0x7ff4a006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 --2- 10.20.193.78:0/1744030387 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a00a01f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 -- 10.20.193.78:0/1744030387 >> 10.20.193.78:0/1744030387 conn(0x7ff4a00a6620 msgr2=0x7ff4a00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 -- 10.20.193.78:0/1744030387 shutdown_connections 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 -- 10.20.193.78:0/1744030387 wait complete. 2026-02-17T05:56:20.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 10 monclient: build_initial_monmap 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 10 monclient: monmap: 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.611+0000 7ff4a816e640 1 Processor -- start 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 1 -- start start 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient: init 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a01339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient(hunting): picked mon.noname-c con 0x7ff4a006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4a006cc70 0x7ff4a0133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient(hunting): picked mon.noname-a con 0x7ff4a006cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4a0134df0 0x7ff4a011cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient(hunting): picked mon.noname-b con 0x7ff4a0134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a009d360 con 0x7ff4a006cc70 2026-02-17T05:56:20.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a009d1e0 con 0x7ff4a0134df0 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a0135950 con 0x7ff4a006c890 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a816e640 10 monclient(hunting): authenticate will time out at 1226.166385s 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a01339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a01339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33314/0 (socket says 10.20.193.78:33314) 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/2047478680 learned_addr learned my addr 10.20.193.78:0/2047478680 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient(hunting): get_auth_request con 0x7ff4a006c890 auth_method 0 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a56e2640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4a006cc70 0x7ff4a0133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a56e2640 10 monclient(hunting): get_auth_request con 0x7ff4a006cc70 auth_method 0 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a56e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a56e2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a56e2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a56e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a56e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a66e4640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4a0134df0 0x7ff4a011cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a56e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a66e4640 10 monclient(hunting): get_auth_request con 0x7ff4a0134df0 auth_method 0 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a66e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a66e4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a66e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient(hunting): handle_auth_done global_id 74690 payload 824 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4a0134df0 msgr2=0x7ff4a011cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4a0134df0 0x7ff4a011cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4a006cc70 msgr2=0x7ff4a0133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4a006cc70 0x7ff4a0133f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient: _finish_hunting 0 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 monclient: found mon.noname-c 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:20.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.612+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/2047478680 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a00a29c0 con 0x7ff4a006c890 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a5ee3640 1 -- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 msgr2=0x7ff4a01339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a5ee3640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a01339e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff490009970 tx=0x7ff490004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 1 -- 10.20.193.78:0/2047478680 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490046020 con 0x7ff4a006c890 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: dump: 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.632 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 1 -- 10.20.193.78:0/2047478680 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff490037e10 con 0x7ff4a006c890 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a816e640 5 monclient: authenticate success, global_id 74690 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 1 -- 10.20.193.78:0/2047478680 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49003c040 con 0x7ff4a006c890 2026-02-17T05:56:20.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48ffff640 4 set_mon_vals no callback set 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff48e7fc640 10 monclient: dump: 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.634 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a816e640 10 monclient: _renew_subs 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a816e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a009d810 con 0x7ff4a006c890 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a816e640 10 monclient: _renew_subs 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a816e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.614+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a011f2c0 con 0x7ff4a006c890 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff4a816e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff4a816e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff4a816e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a0074980 con 0x7ff4a006c890 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff48e7fc640 1 -- 10.20.193.78:0/2047478680 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff49003f030 con 0x7ff4a006c890 2026-02-17T05:56:20.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff48e7fc640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff47003deb0 0x7ff470040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff48e7fc640 1 -- 10.20.193.78:0/2047478680 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(368..368 src has 117..368) ==== 4843+0+0 (secure 0 0 0) 0x7ff490077950 con 0x7ff4a006c890 2026-02-17T05:56:20.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff4a56e2640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff47003deb0 0x7ff470040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.615+0000 7ff4a56e2640 10 monclient: get_auth_request con 0x7ff47003deb0 auth_method 0 2026-02-17T05:56:20.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.618+0000 7ff48e7fc640 1 -- 10.20.193.78:0/2047478680 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff490050030 con 0x7ff4a006c890 2026-02-17T05:56:20.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.618+0000 7ff48e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:20.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.618+0000 7ff48e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:20.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.618+0000 7ff4a56e2640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff47003deb0 0x7ff470040370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff4a013ed00 tx=0x7ff494008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.729+0000 7ff4a816e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.729+0000 7ff4a816e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.729+0000 7ff4a816e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.729+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff4a011f0c0 con 0x7ff4a006c890 2026-02-17T05:56:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.729+0000 7ff48e7fc640 1 -- 10.20.193.78:0/2047478680 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7ff490035dd0 con 0x7ff4a006c890 2026-02-17T05:56:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.729+0000 7ff48e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.729+0000 7ff48e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:20.732 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff47003deb0 msgr2=0x7ff470040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff47003deb0 0x7ff470040370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff4a013ed00 tx=0x7ff494008040 comp rx=0 tx=0).stop 2026-02-17T05:56:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 10 monclient: shutdown 2026-02-17T05:56:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 msgr2=0x7ff4a01339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a01339e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff490009970 tx=0x7ff490004830 comp rx=0 tx=0).stop 2026-02-17T05:56:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 shutdown_connections 2026-02-17T05:56:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff47003deb0 0x7ff470040370 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4a0134df0 0x7ff4a011cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4a006cc70 0x7ff4a0133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 --2- 10.20.193.78:0/2047478680 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4a006c890 0x7ff4a01339e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 >> 10.20.193.78:0/2047478680 conn(0x7ff4a00a6620 msgr2=0x7ff4a009be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 shutdown_connections 2026-02-17T05:56:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.730+0000 7ff4a816e640 1 -- 10.20.193.78:0/2047478680 wait complete. 2026-02-17T05:56:20.738 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:56:20.738 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:56:20.738 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:56:20.738 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:56:20.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.806+0000 7f92b761a640 10 monclient: get_monmap_and_config 2026-02-17T05:56:20.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.806+0000 7f92b761a640 10 monclient: build_initial_monmap 2026-02-17T05:56:20.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.806+0000 7f92b761a640 10 monclient: monmap: 2026-02-17T05:56:20.810 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:20.810 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.810 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:20.807878+0000 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:20.807878+0000 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 1 Processor -- start 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 1 -- start start 2026-02-17T05:56:20.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient: init 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient(hunting): picked mon.noname-b con 0x7f92b006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f92b009d430 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f92b009bf40 0x7f92b006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient(hunting): picked mon.noname-c con 0x7f92b009bf40 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b0073600 con 0x7f92b009d430 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f92b009ca50 con 0x7f92b009bf40 2026-02-17T05:56:20.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f92b006d6f0 con 0x7f92b006c890 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b761a640 10 monclient(hunting): authenticate will time out at 1226.361451s 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33318/0 (socket says 10.20.193.78:33318) 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 1 -- 10.20.193.78:0/1373675936 learned_addr learned my addr 10.20.193.78:0/1373675936 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient(hunting): get_auth_request con 0x7f92b006c890 auth_method 0 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b5b90640 1 --2- 10.20.193.78:0/1373675936 >> v2:10.20.193.115:3300/0 conn(0x7f92b009bf40 0x7f92b006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b5b90640 10 monclient(hunting): get_auth_request con 0x7f92b009bf40 auth_method 0 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b5b90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b5b90640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b5b90640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient(hunting): handle_auth_done global_id 74693 payload 274 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 1 -- 10.20.193.78:0/1373675936 >> v2:10.20.193.115:3300/0 conn(0x7f92b009bf40 msgr2=0x7f92b006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 1 --2- 10.20.193.78:0/1373675936 >> v2:10.20.193.115:3300/0 conn(0x7f92b009bf40 0x7f92b006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 1 -- 10.20.193.78:0/1373675936 >> v1:10.20.193.78:6789/0 conn(0x7f92b009d430 legacy=0x7f92b0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient: _finish_hunting 0 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 1 monclient: found mon.noname-b 2026-02-17T05:56:20.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:20.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.807+0000 7f92b538f640 1 -- 10.20.193.78:0/1373675936 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b006dab0 con 0x7f92b006c890 2026-02-17T05:56:20.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b538f640 1 -- 10.20.193.78:0/1373675936 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 msgr2=0x7f92b006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b538f640 1 --2- 10.20.193.78:0/1373675936 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b006cc70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f92b009c670 tx=0x7f92a002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4a4b593c0c6a6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 1 -- 10.20.193.78:0/1373675936 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a003c070 con 0x7f92b006c890 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: dump: 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.825 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: _finish_auth 0 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 5 monclient: authenticate success, global_id 74693 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 1 -- 10.20.193.78:0/1373675936 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a002fca0 con 0x7f92b006c890 2026-02-17T05:56:20.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 1 -- 10.20.193.78:0/1373675936 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0037660 con 0x7f92b006c890 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 4 set_mon_vals no callback set 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f929ffff640 10 monclient: dump: 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.827 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 10 monclient: shutdown 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 1 -- 10.20.193.78:0/1373675936 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 msgr2=0x7f92b006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 1 --2- 10.20.193.78:0/1373675936 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b006cc70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f92b009c670 tx=0x7f92a002ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 1 -- 10.20.193.78:0/1373675936 shutdown_connections 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 1 --2- 10.20.193.78:0/1373675936 >> v2:10.20.193.115:3300/0 conn(0x7f92b009bf40 0x7f92b006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 1 --2- 10.20.193.78:0/1373675936 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b006cc70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 1 -- 10.20.193.78:0/1373675936 >> 10.20.193.78:0/1373675936 conn(0x7f92b00a6620 msgr2=0x7f92b00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 1 -- 10.20.193.78:0/1373675936 shutdown_connections 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.810+0000 7f92b761a640 1 -- 10.20.193.78:0/1373675936 wait complete. 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient: build_initial_monmap 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient: monmap: 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:20.829 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 1 Processor -- start 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 1 -- start start 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient: init 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b01cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient(hunting): picked mon.noname-c con 0x7f92b006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b009bf40 0x7f92b01cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient(hunting): picked mon.noname-a con 0x7f92b009bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92b009d430 0x7f92b01215c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient(hunting): picked mon.noname-b con 0x7f92b009d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92b00733c0 con 0x7f92b009bf40 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0073240 con 0x7f92b009d430 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f92b0135500 con 0x7f92b006c890 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b761a640 10 monclient(hunting): authenticate will time out at 1226.365719s 2026-02-17T05:56:20.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b01cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b01cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33330/0 (socket says 10.20.193.78:33330) 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b4b8e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b009bf40 0x7f92b01cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 -- 10.20.193.78:0/4034614991 learned_addr learned my addr 10.20.193.78:0/4034614991 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient(hunting): get_auth_request con 0x7f92b006c890 auth_method 0 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b4b8e640 10 monclient(hunting): get_auth_request con 0x7f92b009bf40 auth_method 0 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b4b8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b4b8e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b4b8e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient(hunting): handle_auth_done global_id 74696 payload 824 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92b009d430 msgr2=0x7f92b01215c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92b009d430 0x7f92b01215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b009bf40 msgr2=0x7f92b01cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b009bf40 0x7f92b01cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient: _finish_hunting 0 2026-02-17T05:56:20.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 monclient: found mon.noname-c 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.811+0000 7f92b538f640 1 -- 10.20.193.78:0/4034614991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b0125f20 con 0x7f92b006c890 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b538f640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 msgr2=0x7f92b01cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b538f640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b01cf850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009a70 tx=0x7f92a0004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0047070 con 0x7f92b006c890 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: dump: 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.834 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a002fe50 con 0x7f92b006c890 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 5 monclient: authenticate success, global_id 74696 2026-02-17T05:56:20.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 10 monclient: _renew_subs 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b0125550 con 0x7f92b006c890 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a003c040 con 0x7f92b006c890 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 10 monclient: dump: 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.836 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929f7fe640 4 set_mon_vals no callback set 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 10 monclient: _renew_subs 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b00a19d0 con 0x7f92b006c890 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0124ef0 0x7f92b0121b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0123dc0 con 0x7f92b0124ef0 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0124ef0 0x7f92b0121b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 10 monclient: get_auth_request con 0x7f92b0124ef0 auth_method 0 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_done global_id 124254 payload 824 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f92b4b8e640 1 -- 10.20.193.78:0/4034614991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92b0074980 con 0x7f92b0124ef0 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.814+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f92a003fa10 con 0x7f92b006c890 2026-02-17T05:56:20.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.815+0000 7f929dffb640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f928003deb0 0x7f9280040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.815+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(368..368 src has 117..368) ==== 4843+0+0 (secure 0 0 0) 0x7f92a0077550 con 0x7f92b006c890 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.815+0000 7f92b5b90640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f928003deb0 0x7f9280040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.815+0000 7f92b5b90640 10 monclient: get_auth_request con 0x7f928003deb0 auth_method 0 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.816+0000 7f92b4b8e640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0124ef0 msgr2=0x7f92b0121b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f92b4b8e640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0124ef0 0x7f92b0121b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f92a400f770 tx=0x7f92a400fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4018030 con 0x7f92b0124ef0 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 10 monclient: dump: 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.839 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f92a400dbc0 con 0x7f92b0124ef0 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0124ef0 msgr2=0x7f92b0121b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0124ef0 0x7f92b0121b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f92a400f770 tx=0x7f92a400fc40 comp rx=0 tx=0).stop 2026-02-17T05:56:20.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0124ef0 msgr2=0x7f92b0121b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f929dffb640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0124ef0 0x7f92b0121b00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.817+0000 7f92b5b90640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f928003deb0 0x7f9280040370 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f92b00a15a0 tx=0x7f92ac0096a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b761a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b761a640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0122040 0x7f92b01da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b761a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0135930 con 0x7f92b0122040 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0122040 0x7f92b01da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 10 monclient: get_auth_request con 0x7f92b0122040 auth_method 0 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 10 monclient(hunting): handle_auth_done global_id 124257 payload 824 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.820+0000 7f92b4b8e640 1 -- 10.20.193.78:0/4034614991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f92b01252d0 con 0x7f92b0122040 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.822+0000 7f92b4b8e640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0122040 msgr2=0x7f92b01da8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.822+0000 7f92b4b8e640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0122040 0x7f92b01da8c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f92a400f770 tx=0x7f92a4003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.822+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4004230 con 0x7f92b0122040 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.822+0000 7f929dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.822+0000 7f929dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.822+0000 7f929dffb640 10 monclient: dump: 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.842 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f92a4004230 con 0x7f92b0122040 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f929dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f929dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:20.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0122040 msgr2=0x7f92b01da8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f929dffb640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0122040 0x7f92b01da8c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f92a400f770 tx=0x7f92a4003860 comp rx=0 tx=0).stop 2026-02-17T05:56:20.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f929dffb640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0122040 msgr2=0x7f92b01da8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:20.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f929dffb640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92b0122040 0x7f92b01da8c0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f92a400f770 tx=0x7f92a4003860 comp rx=0 tx=0).stop 2026-02-17T05:56:20.844 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:56:20.844 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:20.844 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:20.844 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:56:20.844 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 112, 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 72288, 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:20.845 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:20.846 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:20.847 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:20.848 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.849 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:20.850 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.851 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.852 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:20.853 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.854 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:56:20.855 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:56:20.856 INFO:tasks.rados.rados.0.trial115.stdout:1761: done (4 left) 2026-02-17T05:56:20.856 INFO:tasks.rados.rados.0.trial115.stdout:1762: done (3 left) 2026-02-17T05:56:20.856 INFO:tasks.rados.rados.0.trial115.stdout:1763: done (2 left) 2026-02-17T05:56:20.856 INFO:tasks.rados.rados.0.trial115.stdout:1764: done (1 left) 2026-02-17T05:56:20.856 INFO:tasks.rados.rados.0.trial115.stdout:1765: done (0 left) 2026-02-17T05:56:20.856 INFO:tasks.rados.rados.0.trial115.stdout:1766: write oid 44 current snap is 174 2026-02-17T05:56:20.856 INFO:tasks.rados.rados.0.trial115.stdout:1766: seq_num 649 ranges {507743=726606,1744696=590883,2772934=134809} 2026-02-17T05:56:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f928003deb0 msgr2=0x7f9280040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f928003deb0 0x7f9280040370 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f92b00a15a0 tx=0x7f92ac0096a0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 10 monclient: shutdown 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 msgr2=0x7f92b01cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b01cf850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009a70 tx=0x7f92a0004830 comp rx=0 tx=0).stop 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b5b90640 1 -- 10.20.193.78:0/4034614991 reap_dead start 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 shutdown_connections 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 --2- 10.20.193.78:0/4034614991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92b006c890 0x7f92b01cf850 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 >> 10.20.193.78:0/4034614991 conn(0x7f92b00a6620 msgr2=0x7f92b0137970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 shutdown_connections 2026-02-17T05:56:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.823+0000 7f92b761a640 1 -- 10.20.193.78:0/4034614991 wait complete. 2026-02-17T05:56:20.857 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:56:20.857 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:56:20.858 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:56:20.858 INFO:tasks.rados.rados.0.trial115.stdout:1766: writing trial11545420-44 from 507743 to 1234349 tid 1 2026-02-17T05:56:20.861 INFO:tasks.rados.rados.0.trial115.stdout:1766: writing trial11545420-44 from 1744696 to 2335579 tid 2 2026-02-17T05:56:20.863 INFO:tasks.rados.rados.0.trial115.stdout:1766: writing trial11545420-44 from 2772934 to 2907743 tid 3 2026-02-17T05:56:20.863 INFO:tasks.rados.rados.0.trial115.stdout:1767: rollback oid 25 current snap is 174 2026-02-17T05:56:20.864 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 171 2026-02-17T05:56:20.864 INFO:tasks.rados.rados.0.trial115.stdout:1766: finishing write tid 1 to trial11545420-44 2026-02-17T05:56:20.864 INFO:tasks.rados.rados.0.trial115.stdout:1766: finishing write tid 2 to trial11545420-44 2026-02-17T05:56:20.864 INFO:tasks.rados.rados.0.trial115.stdout:1768: read oid 22 snap 139 2026-02-17T05:56:20.864 INFO:tasks.rados.rados.0.trial115.stdout:1768: expect deleted 2026-02-17T05:56:20.864 INFO:tasks.rados.rados.0.trial115.stdout:1769: rollback oid 1 current snap is 174 2026-02-17T05:56:20.864 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 1 to 171 2026-02-17T05:56:20.864 INFO:tasks.rados.rados.0.trial115.stdout:1770: snap_remove snap 173 2026-02-17T05:56:20.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.926+0000 7f303c6e1640 10 monclient: get_monmap_and_config 2026-02-17T05:56:20.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.926+0000 7f303c6e1640 10 monclient: build_initial_monmap 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.926+0000 7f303c6e1640 10 monclient: monmap: 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:20.927959+0000 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:20.927959+0000 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 1 Processor -- start 2026-02-17T05:56:20.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 1 -- start start 2026-02-17T05:56:20.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient: init 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f303406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f303409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406d380 0x7f303413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient(hunting): picked mon.noname-b con 0x7f303406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303406daf0 con 0x7f303409c4a0 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303409cc80 con 0x7f303406c890 2026-02-17T05:56:20.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f30340a0880 con 0x7f303406d380 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303c6e1640 10 monclient(hunting): authenticate will time out at 1226.481537s 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f3039c55640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f303409c4a0 0x7f303413b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:43872/0 (socket says 10.20.193.78:43872) 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f3039c55640 1 -- 10.20.193.78:0/1767296086 learned_addr learned my addr 10.20.193.78:0/1767296086 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 1 --2- 10.20.193.78:0/1767296086 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406d380 0x7f303413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient(hunting): get_auth_request con 0x7f303406d380 auth_method 0 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient(hunting): handle_auth_done global_id 74699 payload 274 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 1 -- 10.20.193.78:0/1767296086 >> v1:10.20.193.115:6789/0 conn(0x7f303406c890 legacy=0x7f303406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:20.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 1 -- 10.20.193.78:0/1767296086 >> v1:10.20.193.78:6789/0 conn(0x7f303409c4a0 legacy=0x7f303413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:20.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient: _finish_hunting 0 2026-02-17T05:56:20.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 1 monclient: found mon.noname-b 2026-02-17T05:56:20.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:20.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:56:20.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.927+0000 7f303ac57640 1 -- 10.20.193.78:0/1767296086 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303409cf60 con 0x7f303406d380 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303ac57640 1 -- 10.20.193.78:0/1767296086 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406d380 msgr2=0x7f303413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303ac57640 1 --2- 10.20.193.78:0/1767296086 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406d380 0x7f303413ead0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3030009dc0 tx=0x7f303002f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674176d558bd0aba server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 1 -- 10.20.193.78:0/1767296086 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303003c070 con 0x7f303406d380 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: dump: 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.937 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: _finish_auth 0 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: _check_auth_tickets 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 1 -- 10.20.193.78:0/1767296086 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3030037640 con 0x7f303406d380 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 5 monclient: authenticate success, global_id 74699 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 1 -- 10.20.193.78:0/1767296086 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030037a10 con 0x7f303406d380 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 4 set_mon_vals no callback set 2026-02-17T05:56:20.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f3039454640 10 monclient: dump: 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 10 monclient: shutdown 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 1 -- 10.20.193.78:0/1767296086 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406d380 msgr2=0x7f303413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 1 --2- 10.20.193.78:0/1767296086 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406d380 0x7f303413ead0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3030009dc0 tx=0x7f303002f1c0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 1 -- 10.20.193.78:0/1767296086 shutdown_connections 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 1 --2- 10.20.193.78:0/1767296086 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406d380 0x7f303413ead0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 1 -- 10.20.193.78:0/1767296086 >> 10.20.193.78:0/1767296086 conn(0x7f30340a6620 msgr2=0x7f30340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 1 -- 10.20.193.78:0/1767296086 shutdown_connections 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.930+0000 7f303c6e1640 1 -- 10.20.193.78:0/1767296086 wait complete. 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient: build_initial_monmap 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient: monmap: 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 1 Processor -- start 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 1 -- start start 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient: init 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406c890 0x7f30341cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient(hunting): picked mon.noname-c con 0x7f303406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f303406d380 0x7f3034131290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient(hunting): picked mon.noname-b con 0x7f303406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f303409c4a0 0x7f303406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient(hunting): picked mon.noname-a con 0x7f303409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f303409bfa0 con 0x7f303409c4a0 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f303409be20 con 0x7f303406d380 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f303413f010 con 0x7f303406c890 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.931+0000 7f303c6e1640 10 monclient(hunting): authenticate will time out at 1226.485983s 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303a456640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406c890 0x7f30341cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f303409c4a0 0x7f303406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f303409c4a0 0x7f303406f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:41050/0 (socket says 10.20.193.78:41050) 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 -- 10.20.193.78:0/2596812791 learned_addr learned my addr 10.20.193.78:0/2596812791 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient(hunting): get_auth_request con 0x7f303409c4a0 auth_method 0 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303a456640 10 monclient(hunting): get_auth_request con 0x7f303406c890 auth_method 0 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303a456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303a456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303a456640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f3039c55640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f303406d380 0x7f3034131290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f3039c55640 10 monclient(hunting): get_auth_request con 0x7f303406d380 auth_method 0 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f3039c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f3039c55640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f3039c55640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient(hunting): handle_auth_done global_id 124260 payload 824 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406c890 msgr2=0x7f30341cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f303406c890 0x7f30341cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f303406d380 msgr2=0x7f3034131290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f303406d380 0x7f3034131290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient: _finish_hunting 0 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 monclient: found mon.noname-a 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.932+0000 7f303ac57640 1 -- 10.20.193.78:0/2596812791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3034071240 con 0x7f303409c4a0 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303ac57640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f303409c4a0 msgr2=0x7f303406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303ac57640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f303409c4a0 0x7f303406f6b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3030009ef0 tx=0x7f3030004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030047020 con 0x7f303409c4a0 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: dump: 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.944 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: _finish_auth 0 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3030042880 con 0x7f303409c4a0 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303003c040 con 0x7f303409c4a0 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f301affd640 10 monclient: dump: 2026-02-17T05:56:20.945 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f3038c53640 4 set_mon_vals no callback set 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 5 monclient: authenticate success, global_id 124260 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 10 monclient: _renew_subs 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3034072c70 con 0x7f303409c4a0 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 10 monclient: _renew_subs 2026-02-17T05:56:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3034071d80 con 0x7f303409c4a0 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3034073070 0x7f30341320f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.934+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f303406cc70 con 0x7f3034073070 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3034073070 0x7f30341320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 10 monclient: get_auth_request con 0x7f3034073070 auth_method 0 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 10 monclient(hunting): handle_auth_done global_id 74705 payload 824 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f303a456640 1 -- 10.20.193.78:0/2596812791 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3034071b00 con 0x7f3034073070 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3030042c70 con 0x7f303409c4a0 2026-02-17T05:56:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f301affd640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f300403df00 0x7f30040403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(369..369 src has 117..369) ==== 4815+0+0 (secure 0 0 0) 0x7f3030078300 con 0x7f303409c4a0 2026-02-17T05:56:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f3039c55640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f300403df00 0x7f30040403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.935+0000 7f3039c55640 10 monclient: get_auth_request con 0x7f300403df00 auth_method 0 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f303a456640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3034073070 msgr2=0x7f30341320f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f303a456640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3034073070 0x7f30341320f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f302400f7a0 tx=0x7f302400fc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024018030 con 0x7f3034073070 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 10 monclient: dump: 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.950 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f302400dbc0 con 0x7f3034073070 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3034073070 msgr2=0x7f30341320f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3034073070 0x7f30341320f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f302400f7a0 tx=0x7f302400fc70 comp rx=0 tx=0).stop 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3034073070 msgr2=0x7f30341320f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f301affd640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3034073070 0x7f30341320f0 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f302400f7a0 tx=0x7f302400fc70 comp rx=0 tx=0).stop 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.937+0000 7f3039c55640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f300403df00 0x7f30040403c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3034070e40 tx=0x7f3028009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303c6e1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303c6e1640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f30341317d0 0x7f30341da540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303c6e1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f30341337b0 con 0x7f30341317d0 2026-02-17T05:56:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f30341317d0 0x7f30341da540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 10 monclient: get_auth_request con 0x7f30341317d0 auth_method 0 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 10 monclient(hunting): handle_auth_done global_id 74708 payload 824 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.940+0000 7f303a456640 1 -- 10.20.193.78:0/2596812791 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f30340744d0 con 0x7f30341317d0 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.942+0000 7f303a456640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f30341317d0 msgr2=0x7f30341da540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.942+0000 7f303a456640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f30341317d0 0x7f30341da540 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3024018000 tx=0x7f3024003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.942+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024004230 con 0x7f30341317d0 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.942+0000 7f301affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.942+0000 7f301affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.942+0000 7f301affd640 10 monclient: dump: 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.952 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f3024004230 con 0x7f30341317d0 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f301affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f301affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f30341317d0 msgr2=0x7f30341da540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f301affd640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f30341317d0 0x7f30341da540 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3024018000 tx=0x7f3024003860 comp rx=0 tx=0).stop 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f301affd640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f30341317d0 msgr2=0x7f30341da540 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f301affd640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f30341317d0 0x7f30341da540 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3024018000 tx=0x7f3024003860 comp rx=0 tx=0).stop 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 112, 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 352835, 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:20.954 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:20.955 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:56:20.956 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:20.957 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:20.958 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:20.959 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.960 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:56:20.961 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:56:20.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f300403df00 msgr2=0x7f30040403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f300403df00 0x7f30040403c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3034070e40 tx=0x7f3028009040 comp rx=0 tx=0).stop 2026-02-17T05:56:20.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 10 monclient: shutdown 2026-02-17T05:56:20.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f303409c4a0 msgr2=0x7f303406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 1 --2- 10.20.193.78:0/2596812791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f303409c4a0 0x7f303406f6b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3030009ef0 tx=0x7f3030004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303ac57640 1 -- 10.20.193.78:0/2596812791 reap_dead start 2026-02-17T05:56:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303ac57640 1 -- 10.20.193.78:0/2596812791 reap_dead start 2026-02-17T05:56:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 shutdown_connections 2026-02-17T05:56:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 >> 10.20.193.78:0/2596812791 conn(0x7f30340a6620 msgr2=0x7f30340a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 shutdown_connections 2026-02-17T05:56:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:20.943+0000 7f303c6e1640 1 -- 10.20.193.78:0/2596812791 wait complete. 2026-02-17T05:56:20.963 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:56:20.963 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:56:20.963 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:56:21.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.029+0000 7f2fde10a640 10 monclient: get_monmap_and_config 2026-02-17T05:56:21.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.029+0000 7f2fde10a640 10 monclient: build_initial_monmap 2026-02-17T05:56:21.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.029+0000 7f2fde10a640 10 monclient: monmap: 2026-02-17T05:56:21.033 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:21.033 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.033 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:21.031212+0000 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:21.031212+0000 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 1 Processor -- start 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 1 -- start start 2026-02-17T05:56:21.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient: init 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd806c890 0x7f2fd806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient(hunting): picked mon.noname-b con 0x7f2fd806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2fd809d430 0x7f2fd806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient(hunting): picked mon.noname-c con 0x7f2fd809d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2fd809bf40 0x7f2fd8134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient(hunting): picked mon.noname-a con 0x7f2fd809bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd809ca50 con 0x7f2fd809bf40 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd806d6f0 con 0x7f2fd809d430 2026-02-17T05:56:21.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd8073600 con 0x7f2fd806c890 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fde10a640 10 monclient(hunting): authenticate will time out at 1226.584751s 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd806c890 0x7f2fd806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2fd809bf40 0x7f2fd8134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd806c890 0x7f2fd806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33384/0 (socket says 10.20.193.78:33384) 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2fd809bf40 0x7f2fd8134df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:41062/0 (socket says 10.20.193.78:41062) 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 1 -- 10.20.193.78:0/2172103932 learned_addr learned my addr 10.20.193.78:0/2172103932 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 10 monclient(hunting): get_auth_request con 0x7f2fd806c890 auth_method 0 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 10 monclient(hunting): get_auth_request con 0x7f2fd809bf40 auth_method 0 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd6ffd640 1 --2- 10.20.193.78:0/2172103932 >> v2:10.20.193.115:3300/0 conn(0x7f2fd809d430 0x7f2fd806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd6ffd640 10 monclient(hunting): get_auth_request con 0x7f2fd809d430 auth_method 0 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.030+0000 7f2fd6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_done global_id 124263 payload 274 2026-02-17T05:56:21.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 1 -- 10.20.193.78:0/2172103932 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd806c890 msgr2=0x7f2fd806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:21.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 1 --2- 10.20.193.78:0/2172103932 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd806c890 0x7f2fd806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 1 -- 10.20.193.78:0/2172103932 >> v2:10.20.193.115:3300/0 conn(0x7f2fd809d430 msgr2=0x7f2fd806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:21.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 1 --2- 10.20.193.78:0/2172103932 >> v2:10.20.193.115:3300/0 conn(0x7f2fd809d430 0x7f2fd806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:56:21.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 1 monclient: found mon.noname-a 2026-02-17T05:56:21.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:21.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:21.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.031+0000 7f2fd7fff640 1 -- 10.20.193.78:0/2172103932 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd806dab0 con 0x7f2fd809bf40 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd7fff640 1 -- 10.20.193.78:0/2172103932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809bf40 msgr2=0x7f2fd8134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd7fff640 1 --2- 10.20.193.78:0/2172103932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809bf40 0x7f2fd8134df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80a7600 tx=0x7f2fcc02f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f705451219e1fcb8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 1 -- 10.20.193.78:0/2172103932 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc03c070 con 0x7f2fd809bf40 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: dump: 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.043 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 1 -- 10.20.193.78:0/2172103932 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fcc037440 con 0x7f2fd809bf40 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fde10a640 5 monclient: authenticate success, global_id 124263 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fde10a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 1 -- 10.20.193.78:0/2172103932 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc037810 con 0x7f2fd809bf40 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fde10a640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fde10a640 4 set_mon_vals no callback set 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.033+0000 7f2fd67fc640 10 monclient: dump: 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.044 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 10 monclient: shutdown 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 -- 10.20.193.78:0/2172103932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809bf40 msgr2=0x7f2fd8134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 --2- 10.20.193.78:0/2172103932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809bf40 0x7f2fd8134df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80a7600 tx=0x7f2fcc02f130 comp rx=0 tx=0).stop 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 -- 10.20.193.78:0/2172103932 shutdown_connections 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 --2- 10.20.193.78:0/2172103932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809bf40 0x7f2fd8134df0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 --2- 10.20.193.78:0/2172103932 >> v2:10.20.193.115:3300/0 conn(0x7f2fd809d430 0x7f2fd806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 --2- 10.20.193.78:0/2172103932 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd806c890 0x7f2fd806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 -- 10.20.193.78:0/2172103932 >> 10.20.193.78:0/2172103932 conn(0x7f2fd80a6620 msgr2=0x7f2fd80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 -- 10.20.193.78:0/2172103932 shutdown_connections 2026-02-17T05:56:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 -- 10.20.193.78:0/2172103932 wait complete. 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 10 monclient: build_initial_monmap 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 10 monclient: monmap: 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 Processor -- start 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 1 -- start start 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.034+0000 7f2fde10a640 10 monclient: init 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd806c890 0x7f2fd8121750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient(hunting): picked mon.noname-b con 0x7f2fd806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd809bf40 0x7f2fd8121c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient(hunting): picked mon.noname-c con 0x7f2fd809bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809d430 0x7f2fd81221d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient(hunting): picked mon.noname-a con 0x7f2fd809d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8073380 con 0x7f2fd809d430 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8073200 con 0x7f2fd806c890 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd8135330 con 0x7f2fd809bf40 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fde10a640 10 monclient(hunting): authenticate will time out at 1226.589098s 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd809bf40 0x7f2fd8121c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd809bf40 0x7f2fd8121c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:33400/0 (socket says 10.20.193.78:33400) 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809d430 0x7f2fd81221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 -- 10.20.193.78:0/4013105561 learned_addr learned my addr 10.20.193.78:0/4013105561 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient(hunting): get_auth_request con 0x7f2fd809bf40 auth_method 0 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 10 monclient(hunting): get_auth_request con 0x7f2fd809d430 auth_method 0 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd77fe640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd806c890 0x7f2fd8121750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd77fe640 10 monclient(hunting): get_auth_request con 0x7f2fd806c890 auth_method 0 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient(hunting): handle_auth_done global_id 74714 payload 824 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd806c890 msgr2=0x7f2fd8121750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd806c890 0x7f2fd8121750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809d430 msgr2=0x7f2fd81221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809d430 0x7f2fd81221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 monclient: found mon.noname-c 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd6ffd640 1 -- 10.20.193.78:0/4013105561 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd8122fc0 con 0x7f2fd809bf40 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.035+0000 7f2fd7fff640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2fd809d430 0x7f2fd81221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fd6ffd640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd809bf40 msgr2=0x7f2fd8121c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fd6ffd640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd809bf40 0x7f2fd8121c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800e7c0 tx=0x7f2fc800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8017070 con 0x7f2fd809bf40 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: dump: 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.049 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc800cae0 con 0x7f2fd809bf40 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800f040 con 0x7f2fd809bf40 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fde10a640 5 monclient: authenticate success, global_id 74714 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fd5ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fde10a640 10 monclient: _renew_subs 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fde10a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd8125320 con 0x7f2fd809bf40 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fb3fff640 10 monclient: dump: 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.050 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fde10a640 10 monclient: _renew_subs 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fde10a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.037+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd8125040 con 0x7f2fd809bf40 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fde10a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fde10a640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8134b80 0x7f2fd811ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fde10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8124c10 con 0x7f2fd8134b80 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8134b80 0x7f2fd811ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 10 monclient: get_auth_request con 0x7f2fd8134b80 auth_method 0 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2fc8012050 con 0x7f2fd809bf40 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fb3fff640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2fa803deb0 0x7f2fa8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_done global_id 134131 payload 824 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd77fe640 1 -- 10.20.193.78:0/4013105561 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fd8074980 con 0x7f2fd8134b80 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(369..369 src has 117..369) ==== 4815+0+0 (secure 0 0 0) 0x7f2fc80526a0 con 0x7f2fd809bf40 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd7fff640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2fa803deb0 0x7f2fa8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.038+0000 7f2fd7fff640 10 monclient: get_auth_request con 0x7f2fa803deb0 auth_method 0 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.040+0000 7f2fd77fe640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8134b80 msgr2=0x7f2fd811ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.040+0000 7f2fd77fe640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8134b80 0x7f2fd811ddf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4011800 tx=0x7f2fc4011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.040+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400fe50 con 0x7f2fd8134b80 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.040+0000 7f2fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.040+0000 7f2fb3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.040+0000 7f2fb3fff640 10 monclient: dump: 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.052 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.041+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2fc400fe50 con 0x7f2fd8134b80 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.041+0000 7f2fb3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.041+0000 7f2fb3fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.041+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8134b80 msgr2=0x7f2fd811ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.041+0000 7f2fb3fff640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8134b80 0x7f2fd811ddf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4011800 tx=0x7f2fc4011cd0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.041+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8134b80 msgr2=0x7f2fd811ddf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.041+0000 7f2fb3fff640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8134b80 0x7f2fd811ddf0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4011800 tx=0x7f2fc4011cd0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.041+0000 7f2fd7fff640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2fa803deb0 0x7f2fa8040370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2fd809c670 tx=0x7f2fcc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:21.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fde10a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fde10a640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8063880 0x7f2fd8063c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fde10a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd811e330 con 0x7f2fd8063880 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8063880 0x7f2fd8063c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 10 monclient: get_auth_request con 0x7f2fd8063880 auth_method 0 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 10 monclient(hunting): handle_auth_done global_id 134134 payload 824 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.044+0000 7f2fd77fe640 1 -- 10.20.193.78:0/4013105561 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2fd80008d0 con 0x7f2fd8063880 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.046+0000 7f2fd77fe640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8063880 msgr2=0x7f2fd8063c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.046+0000 7f2fd77fe640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8063880 0x7f2fd8063c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4011800 tx=0x7f2fc40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.046+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4003b30 con 0x7f2fd8063880 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.046+0000 7f2fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.046+0000 7f2fb3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.046+0000 7f2fb3fff640 10 monclient: dump: 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4707 (secure 0 0 0) 0x7f2fc4003b30 con 0x7f2fd8063880 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fb3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fb3fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8063880 msgr2=0x7f2fd8063c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fb3fff640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8063880 0x7f2fd8063c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4011800 tx=0x7f2fc40047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fb3fff640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8063880 msgr2=0x7f2fd8063c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:21.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fb3fff640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2fd8063880 0x7f2fd8063c90 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4011800 tx=0x7f2fc40047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 112, 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 8766, 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:56:21.057 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:21.058 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:56:21.059 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:56:21.060 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:21.061 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:56:21.062 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:21.063 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:21.064 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:56:21.065 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2fa803deb0 msgr2=0x7f2fa8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2fa803deb0 0x7f2fa8040370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2fd809c670 tx=0x7f2fcc033000 comp rx=0 tx=0).stop 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 10 monclient: shutdown 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd809bf40 msgr2=0x7f2fd8121c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 1 --2- 10.20.193.78:0/4013105561 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2fd809bf40 0x7f2fd8121c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800e7c0 tx=0x7f2fc800ec90 comp rx=0 tx=0).stop 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fd7fff640 1 -- 10.20.193.78:0/4013105561 reap_dead start 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fd7fff640 1 -- 10.20.193.78:0/4013105561 reap_dead start 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 shutdown_connections 2026-02-17T05:56:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 >> 10.20.193.78:0/4013105561 conn(0x7f2fd80a6620 msgr2=0x7f2fd8136aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 shutdown_connections 2026-02-17T05:56:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.047+0000 7f2fde10a640 1 -- 10.20.193.78:0/4013105561 wait complete. 2026-02-17T05:56:21.067 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:56:21.067 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-17T05:56:21.067 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:56:21.067 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-17T05:56:21.067 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T05:56:21.067 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-17T05:56:21.067 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:56:21.079 INFO:tasks.ceph.mon.c:Stopped 2026-02-17T05:56:21.079 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:56:21.079 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.185+0000 7f0cdcd49640 10 monclient: get_monmap_and_config 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.185+0000 7f0cdcd49640 10 monclient: build_initial_monmap 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient: monmap: 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:21.187232+0000 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:21.187232+0000 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.187 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:21.188 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:21.188 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:21.188 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:21.188 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:21.188 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 1 Processor -- start 2026-02-17T05:56:21.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 1 -- start start 2026-02-17T05:56:21.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient: init 2026-02-17T05:56:21.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:21.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T05:56:21.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0cd806c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:21.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0cd809c4a0 0x7f0cd813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient(hunting): picked mon.noname-c con 0x7f0cd809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:21.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0cd806d380 0x7f0cd813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient(hunting): picked mon.noname-a con 0x7f0cd806d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd806daf0 con 0x7f0cd806c890 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd80a0880 con 0x7f0cd806d380 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd809cc80 con 0x7f0cd809c4a0 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cdcd49640 10 monclient(hunting): authenticate will time out at 1226.740778s 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f0cd806c890 0x7f0cd806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:43882/0 (socket says 10.20.193.78:43882) 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0cd806d380 0x7f0cd813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6575640 1 -- 10.20.193.78:0/70522547 learned_addr learned my addr 10.20.193.78:0/70522547 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:21.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6d76640 10 monclient(hunting): get_auth_request con 0x7f0cd806d380 auth_method 0 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.186+0000 7f0cd5d74640 1 --2- 10.20.193.78:0/70522547 >> v2:10.20.193.115:3300/0 conn(0x7f0cd809c4a0 0x7f0cd813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd5d74640 10 monclient(hunting): get_auth_request con 0x7f0cd809c4a0 auth_method 0 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 10 monclient(hunting): handle_auth_done global_id 124269 payload 274 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 1 -- 10.20.193.78:0/70522547 >> v2:10.20.193.115:3300/0 conn(0x7f0cd809c4a0 msgr2=0x7f0cd813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 1 --2- 10.20.193.78:0/70522547 >> v2:10.20.193.115:3300/0 conn(0x7f0cd809c4a0 0x7f0cd813b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 1 -- 10.20.193.78:0/70522547 >> v1:10.20.193.78:6789/0 conn(0x7f0cd806c890 legacy=0x7f0cd806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 1 monclient: found mon.noname-a 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.187+0000 7f0cd6d76640 1 -- 10.20.193.78:0/70522547 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd809cf60 con 0x7f0cd806d380 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd6d76640 1 -- 10.20.193.78:0/70522547 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd806d380 msgr2=0x7f0cd813d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd6d76640 1 --2- 10.20.193.78:0/70522547 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd806d380 0x7f0cd813d8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0cd806dcc0 tx=0x7f0cc802f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d3737f593c15f8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 1 -- 10.20.193.78:0/70522547 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc803c070 con 0x7f0cd806d380 2026-02-17T05:56:21.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: dump: 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.201 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: _finish_auth 0 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: _check_auth_tickets 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 1 -- 10.20.193.78:0/70522547 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc8037440 con 0x7f0cd806d380 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 1 -- 10.20.193.78:0/70522547 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8037810 con 0x7f0cd806d380 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cdcd49640 5 monclient: authenticate success, global_id 124269 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cdcd49640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cdcd49640 4 set_mon_vals no callback set 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:21.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.189+0000 7f0cd5573640 10 monclient: dump: 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient: shutdown 2026-02-17T05:56:21.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- 10.20.193.78:0/70522547 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd806d380 msgr2=0x7f0cd813d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/70522547 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd806d380 0x7f0cd813d8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0cd806dcc0 tx=0x7f0cc802f150 comp rx=0 tx=0).stop 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- 10.20.193.78:0/70522547 shutdown_connections 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/70522547 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd806d380 0x7f0cd813d8f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/70522547 >> v2:10.20.193.115:3300/0 conn(0x7f0cd809c4a0 0x7f0cd813b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- 10.20.193.78:0/70522547 >> 10.20.193.78:0/70522547 conn(0x7f0cd80a6620 msgr2=0x7f0cd80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- 10.20.193.78:0/70522547 shutdown_connections 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- 10.20.193.78:0/70522547 wait complete. 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient: build_initial_monmap 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient: monmap: 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:21.204 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 Processor -- start 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- start start 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient: init 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:21.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0cd806c890 0x7f0cd8072680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient(hunting): picked mon.noname-b con 0x7f0cd806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0cd806d380 0x7f0cd8133300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient(hunting): picked mon.noname-c con 0x7f0cd806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd809c4a0 0x7f0cd8133840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient(hunting): picked mon.noname-a con 0x7f0cd809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd809bfa0 con 0x7f0cd809c4a0 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd809be20 con 0x7f0cd806c890 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:21.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd813de30 con 0x7f0cd806d380 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cdcd49640 10 monclient(hunting): authenticate will time out at 1226.744895s 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd5d74640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0cd806d380 msgr2=0x7f0cd8133300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0cd806d380 0x7f0cd8133300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd6d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd809c4a0 0x7f0cd8133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd6d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd809c4a0 0x7f0cd8133840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:41086/0 (socket says 10.20.193.78:41086) 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd6d76640 1 -- 10.20.193.78:0/2822475223 learned_addr learned my addr 10.20.193.78:0/2822475223 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd6d76640 10 monclient(hunting): get_auth_request con 0x7f0cd809c4a0 auth_method 0 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.190+0000 7f0cd6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6575640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0cd806c890 0x7f0cd8072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6575640 10 monclient(hunting): get_auth_request con 0x7f0cd806c890 auth_method 0 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 10 monclient(hunting): handle_auth_done global_id 124272 payload 824 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 1 -- 10.20.193.78:0/2822475223 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0cd806d380 msgr2=0x7f0cd8133300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0cd806d380 0x7f0cd8133300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 1 -- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0cd806c890 msgr2=0x7f0cd8072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0cd806c890 0x7f0cd8072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 1 monclient: found mon.noname-a 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.191+0000 7f0cd6d76640 1 -- 10.20.193.78:0/2822475223 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd811d120 con 0x7f0cd809c4a0 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cd6d76640 1 -- 10.20.193.78:0/2822475223 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd809c4a0 msgr2=0x7f0cd8133840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cd6d76640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd809c4a0 0x7f0cd8133840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0cc803a040 tx=0x7f0cc8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 1 -- 10.20.193.78:0/2822475223 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc803c050 con 0x7f0cd809c4a0 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: dump: 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.209 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:21.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 1 -- 10.20.193.78:0/2822475223 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc8048070 con 0x7f0cd809c4a0 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 5 monclient: authenticate success, global_id 124272 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 10 monclient: _renew_subs 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd809e590 con 0x7f0cd809c4a0 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 1 -- 10.20.193.78:0/2822475223 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8041c60 con 0x7f0cd809c4a0 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 10 monclient: dump: 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:21.211 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cd4d72640 4 set_mon_vals no callback set 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 10 monclient: _renew_subs 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd809d810 con 0x7f0cd809c4a0 2026-02-17T05:56:21.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd8074980 con 0x7f0cd809c4a0 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.193+0000 7f0cbeffd640 1 -- 10.20.193.78:0/2822475223 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0cc8043450 con 0x7f0cd809c4a0 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.194+0000 7f0cbeffd640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ca403de60 0x7f0ca4040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.194+0000 7f0cbeffd640 1 -- 10.20.193.78:0/2822475223 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(369..369 src has 117..369) ==== 4815+0+0 (secure 0 0 0) 0x7f0cc8078240 con 0x7f0cd809c4a0 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.194+0000 7f0cd6575640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ca403de60 0x7f0ca4040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.194+0000 7f0cd6575640 10 monclient: get_auth_request con 0x7f0ca403de60 auth_method 0 2026-02-17T05:56:21.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.196+0000 7f0cbeffd640 1 -- 10.20.193.78:0/2822475223 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0cc8051020 con 0x7f0cd809c4a0 2026-02-17T05:56:21.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.196+0000 7f0cbeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:21.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.196+0000 7f0cbeffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:21.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.197+0000 7f0cd6575640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ca403de60 0x7f0ca4040320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc007340 tx=0x7f0ccc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:21.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.311+0000 7f0cdcd49640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:21.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.311+0000 7f0cdcd49640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:21.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.311+0000 7f0cdcd49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:21.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.311+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0cd806cc70 con 0x7f0cd809c4a0 2026-02-17T05:56:21.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.311+0000 7f0cbeffd640 1 -- 10.20.193.78:0/2822475223 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f0cc8035dc0 con 0x7f0cd809c4a0 2026-02-17T05:56:21.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.311+0000 7f0cbeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:21.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.311+0000 7f0cbeffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:21.314 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:21.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ca403de60 msgr2=0x7f0ca4040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ca403de60 0x7f0ca4040320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc007340 tx=0x7f0ccc009040 comp rx=0 tx=0).stop 2026-02-17T05:56:21.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 10 monclient: shutdown 2026-02-17T05:56:21.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:21.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd809c4a0 msgr2=0x7f0cd8133840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:21.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd809c4a0 0x7f0cd8133840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0cc803a040 tx=0x7f0cc8004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:21.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 shutdown_connections 2026-02-17T05:56:21.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0cd809c4a0 0x7f0cd8133840 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0cd806d380 0x7f0cd8133300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ca403de60 0x7f0ca4040320 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 --2- 10.20.193.78:0/2822475223 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0cd806c890 0x7f0cd8072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:21.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 >> 10.20.193.78:0/2822475223 conn(0x7f0cd80a6620 msgr2=0x7f0cd8134220 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:21.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 shutdown_connections 2026-02-17T05:56:21.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:21.312+0000 7f0cdcd49640 1 -- 10.20.193.78:0/2822475223 wait complete. 2026-02-17T05:56:24.322 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:24.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.389+0000 7f0cf03a9640 10 monclient: get_monmap_and_config 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.389+0000 7f0cf03a9640 10 monclient: build_initial_monmap 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.389+0000 7f0cf03a9640 10 monclient: monmap: 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:24.391039+0000 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:24.391039+0000 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:24.394 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 1 Processor -- start 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 1 -- start start 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient: init 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0ce806c890 0x7f0ce806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:24.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient(hunting): picked mon.noname-c con 0x7f0ce806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0ce809d430 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 0x7f0ce806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient(hunting): picked mon.noname-b con 0x7f0ce809bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ce8073600 con 0x7f0ce809d430 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0ce809ca50 con 0x7f0ce806c890 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce806d6f0 con 0x7f0ce809bf40 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cf03a9640 10 monclient(hunting): authenticate will time out at 1229.944579s 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee91f640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 msgr2=0x7f0ce806d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:24.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee91f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 0x7f0ce806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0ced91d640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f0ce809d430 0x7f0ce8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:43894/0 (socket says 10.20.193.78:43894) 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0ced91d640 1 -- 10.20.193.78:0/1201108388 learned_addr learned my addr 10.20.193.78:0/1201108388 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee11e640 1 --2- 10.20.193.78:0/1201108388 >> v2:10.20.193.115:3300/0 conn(0x7f0ce806c890 0x7f0ce806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee11e640 10 monclient(hunting): get_auth_request con 0x7f0ce806c890 auth_method 0 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee11e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee11e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee11e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee11e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee11e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.390+0000 7f0cee11e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:24.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 10 monclient(hunting): handle_auth_done global_id 134143 payload 274 2026-02-17T05:56:24.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 1 -- 10.20.193.78:0/1201108388 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 msgr2=0x7f0ce806d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:24.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 1 --2- 10.20.193.78:0/1201108388 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 0x7f0ce806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 1 -- 10.20.193.78:0/1201108388 >> v1:10.20.193.78:6789/0 conn(0x7f0ce809d430 legacy=0x7f0ce8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:24.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:24.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 1 monclient: found mon.noname-c 2026-02-17T05:56:24.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:24.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:56:24.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.391+0000 7f0cee11e640 1 -- 10.20.193.78:0/1201108388 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce806dab0 con 0x7f0ce806c890 2026-02-17T05:56:24.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0cee11e640 1 -- 10.20.193.78:0/1201108388 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce806c890 msgr2=0x7f0ce806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:24.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0cee11e640 1 --2- 10.20.193.78:0/1201108388 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce806c890 0x7f0ce806cc70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ce809cbd0 tx=0x7f0cd802eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81ac3a1f1d88fa84 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:24.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 1 -- 10.20.193.78:0/1201108388 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd803c070 con 0x7f0ce806c890 2026-02-17T05:56:24.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:24.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:24.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: dump: 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:24.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: _finish_auth 0 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: _check_auth_tickets 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0cf03a9640 5 monclient: authenticate success, global_id 134143 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0cf03a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 1 -- 10.20.193.78:0/1201108388 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd802fd00 con 0x7f0ce806c890 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0cf03a9640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 1 -- 10.20.193.78:0/1201108388 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8037660 con 0x7f0ce806c890 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0cf03a9640 4 set_mon_vals no callback set 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:24.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.393+0000 7f0ced11c640 10 monclient: dump: 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 10 monclient: shutdown 2026-02-17T05:56:24.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 -- 10.20.193.78:0/1201108388 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce806c890 msgr2=0x7f0ce806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/1201108388 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce806c890 0x7f0ce806cc70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ce809cbd0 tx=0x7f0cd802eda0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 -- 10.20.193.78:0/1201108388 shutdown_connections 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/1201108388 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 0x7f0ce806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/1201108388 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce806c890 0x7f0ce806cc70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 -- 10.20.193.78:0/1201108388 >> 10.20.193.78:0/1201108388 conn(0x7f0ce80a6620 msgr2=0x7f0ce80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 -- 10.20.193.78:0/1201108388 shutdown_connections 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 -- 10.20.193.78:0/1201108388 wait complete. 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 10 monclient: build_initial_monmap 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 10 monclient: monmap: 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:24.409 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 Processor -- start 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 1 -- start start 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 10 monclient: init 2026-02-17T05:56:24.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.394+0000 7f0cf03a9640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0ce806c890 0x7f0ce81cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient(hunting): picked mon.noname-a con 0x7f0ce806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 0x7f0ce81cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient(hunting): picked mon.noname-c con 0x7f0ce809bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce809d430 0x7f0ce8121a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient(hunting): picked mon.noname-b con 0x7f0ce809d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce809c7f0 con 0x7f0ce806c890 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8073410 con 0x7f0ce809d430 2026-02-17T05:56:24.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce8135500 con 0x7f0ce809bf40 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cf03a9640 10 monclient(hunting): authenticate will time out at 1229.949039s 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0ced91d640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 msgr2=0x7f0ce81cfaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0ced91d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 0x7f0ce81cfaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0ce806c890 0x7f0ce81cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0ce806c890 0x7f0ce81cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:41094/0 (socket says 10.20.193.78:41094) 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 -- 10.20.193.78:0/368974519 learned_addr learned my addr 10.20.193.78:0/368974519 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient(hunting): get_auth_request con 0x7f0ce806c890 auth_method 0 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:24.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient(hunting): handle_auth_done global_id 124275 payload 824 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 -- 10.20.193.78:0/368974519 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 msgr2=0x7f0ce81cfaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 0x7f0ce81cfaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 -- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce809d430 msgr2=0x7f0ce8121a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce809d430 0x7f0ce8121a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 monclient: found mon.noname-a 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:24.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.395+0000 7f0cee11e640 1 -- 10.20.193.78:0/368974519 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce80a2150 con 0x7f0ce806c890 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cee11e640 1 -- 10.20.193.78:0/368974519 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0ce806c890 msgr2=0x7f0ce81cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cee11e640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0ce806c890 0x7f0ce81cf560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80099a0 tx=0x7f0cd80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 1 -- 10.20.193.78:0/368974519 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8046070 con 0x7f0ce806c890 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: dump: 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:24.414 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: _finish_auth 0 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 1 -- 10.20.193.78:0/368974519 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd8041870 con 0x7f0ce806c890 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cf03a9640 5 monclient: authenticate success, global_id 124275 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cf03a9640 10 monclient: _renew_subs 2026-02-17T05:56:24.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cf03a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce8124210 con 0x7f0ce806c890 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 1 -- 10.20.193.78:0/368974519 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd803c040 con 0x7f0ce806c890 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cceffd640 10 monclient: dump: 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:24.416 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cec91b640 4 set_mon_vals no callback set 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cf03a9640 10 monclient: _renew_subs 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cf03a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.397+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce81256c0 con 0x7f0ce806c890 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0cceffd640 1 -- 10.20.193.78:0/368974519 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0cd8041c60 con 0x7f0ce806c890 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0cf03a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0cf03a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0cf03a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:24.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce8135910 con 0x7f0ce806c890 2026-02-17T05:56:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0cceffd640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0cb803de60 0x7f0cb8040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0cceffd640 1 -- 10.20.193.78:0/368974519 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(369..369 src has 117..369) ==== 4815+0+0 (secure 0 0 0) 0x7f0cd8078160 con 0x7f0ce806c890 2026-02-17T05:56:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0ced91d640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0cb803de60 0x7f0cb8040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.398+0000 7f0ced91d640 10 monclient: get_auth_request con 0x7f0cb803de60 auth_method 0 2026-02-17T05:56:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.400+0000 7f0cceffd640 1 -- 10.20.193.78:0/368974519 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0ce80a2150 con 0x7f0ce806c890 2026-02-17T05:56:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.400+0000 7f0cceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.400+0000 7f0cceffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.401+0000 7f0ced91d640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0cb803de60 0x7f0cb8040320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc004910 tx=0x7f0cdc007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.514+0000 7f0cf03a9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.514+0000 7f0cf03a9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.514+0000 7f0cf03a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.514+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0ce80744d0 con 0x7f0ce806c890 2026-02-17T05:56:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.514+0000 7f0cceffd640 1 -- 10.20.193.78:0/368974519 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f0cd8037e10 con 0x7f0ce806c890 2026-02-17T05:56:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.514+0000 7f0cceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.514+0000 7f0cceffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:24.517 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:24.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0cb803de60 msgr2=0x7f0cb8040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:24.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0cb803de60 0x7f0cb8040320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc004910 tx=0x7f0cdc007c20 comp rx=0 tx=0).stop 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 10 monclient: shutdown 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0ce806c890 msgr2=0x7f0ce81cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0ce806c890 0x7f0ce81cf560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80099a0 tx=0x7f0cd80047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 shutdown_connections 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0cb803de60 0x7f0cb8040320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ce809d430 0x7f0ce8121a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ce809bf40 0x7f0ce81cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 --2- 10.20.193.78:0/368974519 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0ce806c890 0x7f0ce81cf560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:24.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 >> 10.20.193.78:0/368974519 conn(0x7f0ce80a6620 msgr2=0x7f0ce8136930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:24.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 shutdown_connections 2026-02-17T05:56:24.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:24.515+0000 7f0cf03a9640 1 -- 10.20.193.78:0/368974519 wait complete. 2026-02-17T05:56:27.525 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:27.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.592+0000 7f37468a0640 10 monclient: get_monmap_and_config 2026-02-17T05:56:27.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.592+0000 7f37468a0640 10 monclient: build_initial_monmap 2026-02-17T05:56:27.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.592+0000 7f37468a0640 10 monclient: monmap: 2026-02-17T05:56:27.597 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:27.597 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:27.597 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:27.593939+0000 2026-02-17T05:56:27.597 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:27.593939+0000 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 1 Processor -- start 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 1 -- start start 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient: init 2026-02-17T05:56:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f374006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f374009c4a0 0x7f374013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient(hunting): picked mon.noname-c con 0x7f374009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f374006d380 0x7f374013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient(hunting): picked mon.noname-a con 0x7f374006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374006daf0 con 0x7f374006c890 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f37400a0880 con 0x7f374006d380 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f374009cc80 con 0x7f374009c4a0 2026-02-17T05:56:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f37468a0640 10 monclient(hunting): authenticate will time out at 1233.147532s 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f373ffff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f374006c890 0x7f374006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56970/0 (socket says 10.20.193.78:56970) 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f374006d380 0x7f374013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f373ffff640 1 -- 10.20.193.78:0/2489227466 learned_addr learned my addr 10.20.193.78:0/2489227466 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient(hunting): get_auth_request con 0x7f374006d380 auth_method 0 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f373f7fe640 1 --2- 10.20.193.78:0/2489227466 >> v2:10.20.193.115:3300/0 conn(0x7f374009c4a0 0x7f374013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f373f7fe640 10 monclient(hunting): get_auth_request con 0x7f374009c4a0 auth_method 0 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f373f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f373f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f373f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient(hunting): handle_auth_done global_id 124278 payload 274 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 1 -- 10.20.193.78:0/2489227466 >> v2:10.20.193.115:3300/0 conn(0x7f374009c4a0 msgr2=0x7f374013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 1 --2- 10.20.193.78:0/2489227466 >> v2:10.20.193.115:3300/0 conn(0x7f374009c4a0 0x7f374013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 1 -- 10.20.193.78:0/2489227466 >> v1:10.20.193.78:6789/0 conn(0x7f374006c890 legacy=0x7f374006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient: _finish_hunting 0 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 1 monclient: found mon.noname-a 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.593+0000 7f3744e16640 1 -- 10.20.193.78:0/2489227466 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374009cf60 con 0x7f374006d380 2026-02-17T05:56:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f3744e16640 1 -- 10.20.193.78:0/2489227466 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 msgr2=0x7f374013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f3744e16640 1 --2- 10.20.193.78:0/2489227466 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374013d8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f374006dcc0 tx=0x7f373402f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9f237e552bb5bb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 1 -- 10.20.193.78:0/2489227466 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373403c070 con 0x7f374006d380 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: dump: 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:27.609 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: _finish_auth 0 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 5 monclient: authenticate success, global_id 124278 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 1 -- 10.20.193.78:0/2489227466 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3734037440 con 0x7f374006d380 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 1 -- 10.20.193.78:0/2489227466 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734037810 con 0x7f374006d380 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 4 set_mon_vals no callback set 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f373effd640 10 monclient: dump: 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:27.611 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 10 monclient: shutdown 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 1 -- 10.20.193.78:0/2489227466 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 msgr2=0x7f374013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 1 --2- 10.20.193.78:0/2489227466 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374013d8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f374006dcc0 tx=0x7f373402f150 comp rx=0 tx=0).stop 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 1 -- 10.20.193.78:0/2489227466 shutdown_connections 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 1 --2- 10.20.193.78:0/2489227466 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374013d8f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 1 --2- 10.20.193.78:0/2489227466 >> v2:10.20.193.115:3300/0 conn(0x7f374009c4a0 0x7f374013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 1 -- 10.20.193.78:0/2489227466 >> 10.20.193.78:0/2489227466 conn(0x7f37400a6620 msgr2=0x7f37400a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 1 -- 10.20.193.78:0/2489227466 shutdown_connections 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.596+0000 7f37468a0640 1 -- 10.20.193.78:0/2489227466 wait complete. 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient: build_initial_monmap 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient: monmap: 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:27.613 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 1 Processor -- start 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 1 -- start start 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient: init 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f374006c890 0x7f374013c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient(hunting): picked mon.noname-c con 0x7f374006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:27.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient(hunting): picked mon.noname-a con 0x7f374006d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f374009c4a0 0x7f374009e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient(hunting): picked mon.noname-b con 0x7f374009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f374009bfa0 con 0x7f374006d380 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f374009be20 con 0x7f374009c4a0 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f374013de30 con 0x7f374006c890 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:27.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f37468a0640 10 monclient(hunting): authenticate will time out at 1233.151667s 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373ffff640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f374006c890 msgr2=0x7f374013c720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f374006c890 0x7f374013c720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374009db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51820/0 (socket says 10.20.193.78:51820) 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 -- 10.20.193.78:0/2570194659 learned_addr learned my addr 10.20.193.78:0/2570194659 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient(hunting): get_auth_request con 0x7f374006d380 auth_method 0 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:27.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f3744e16640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f374009c4a0 0x7f374009e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f3744e16640 10 monclient(hunting): get_auth_request con 0x7f374009c4a0 auth_method 0 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f3744e16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f3744e16640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f3744e16640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient(hunting): handle_auth_done global_id 124281 payload 824 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 -- 10.20.193.78:0/2570194659 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f374006c890 msgr2=0x7f374013c720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f374006c890 0x7f374013c720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 -- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f374009c4a0 msgr2=0x7f374009e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f374009c4a0 0x7f374009e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 monclient: found mon.noname-a 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.597+0000 7f373f7fe640 1 -- 10.20.193.78:0/2570194659 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374013b7f0 con 0x7f374006d380 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373f7fe640 1 -- 10.20.193.78:0/2570194659 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 msgr2=0x7f374009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373f7fe640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374009db50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f373000e7f0 tx=0x7f373000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 1 -- 10.20.193.78:0/2570194659 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730017070 con 0x7f374006d380 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: dump: 2026-02-17T05:56:27.617 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: _finish_auth 0 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: _check_auth_tickets 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 1 -- 10.20.193.78:0/2570194659 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373000cae0 con 0x7f374006d380 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 5 monclient: authenticate success, global_id 124281 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 10 monclient: _renew_subs 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374013bad0 con 0x7f374006d380 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 1 -- 10.20.193.78:0/2570194659 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373000f040 con 0x7f374006d380 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 10 monclient: dump: 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:27.618 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373e7fc640 4 set_mon_vals no callback set 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 10 monclient: _renew_subs 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374013cc60 con 0x7f374006d380 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3740074980 con 0x7f374006d380 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.600+0000 7f373cff9640 1 -- 10.20.193.78:0/2570194659 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3730012050 con 0x7f374006d380 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.601+0000 7f373cff9640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f371003deb0 0x7f3710040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.601+0000 7f373cff9640 1 -- 10.20.193.78:0/2570194659 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(369..369 src has 117..369) ==== 4815+0+0 (secure 0 0 0) 0x7f3730052600 con 0x7f374006d380 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.601+0000 7f373ffff640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f371003deb0 0x7f3710040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.601+0000 7f373ffff640 10 monclient: get_auth_request con 0x7f371003deb0 auth_method 0 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.603+0000 7f373cff9640 1 -- 10.20.193.78:0/2570194659 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3730023d80 con 0x7f374006d380 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.603+0000 7f373cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.603+0000 7f373cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:27.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.603+0000 7f373ffff640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f371003deb0 0x7f3710040370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f372c004910 tx=0x7f372c007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:27.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.716+0000 7f37468a0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:27.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.716+0000 7f37468a0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:27.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.716+0000 7f37468a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:27.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.716+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f374013db80 con 0x7f374006d380 2026-02-17T05:56:27.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.716+0000 7f373cff9640 1 -- 10.20.193.78:0/2570194659 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f37300546e0 con 0x7f374006d380 2026-02-17T05:56:27.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.716+0000 7f373cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:27.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.716+0000 7f373cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:27.717 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:27.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f371003deb0 msgr2=0x7f3710040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f371003deb0 0x7f3710040370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f372c004910 tx=0x7f372c007c20 comp rx=0 tx=0).stop 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 10 monclient: shutdown 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 msgr2=0x7f374009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374009db50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f373000e7f0 tx=0x7f373000ecc0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 shutdown_connections 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f371003deb0 0x7f3710040370 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f374009c4a0 0x7f374009e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f374006d380 0x7f374009db50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 --2- 10.20.193.78:0/2570194659 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f374006c890 0x7f374013c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:27.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 >> 10.20.193.78:0/2570194659 conn(0x7f37400a6620 msgr2=0x7f37400a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:27.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 shutdown_connections 2026-02-17T05:56:27.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:27.717+0000 7f37468a0640 1 -- 10.20.193.78:0/2570194659 wait complete. 2026-02-17T05:56:30.726 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:30.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient: get_monmap_and_config 2026-02-17T05:56:30.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient: build_initial_monmap 2026-02-17T05:56:30.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient: monmap: 2026-02-17T05:56:30.796 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:30.796 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:30.796 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:30.796294+0000 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:30.796294+0000 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 1 Processor -- start 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 1 -- start start 2026-02-17T05:56:30.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient: init 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 0x7f60fc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient(hunting): picked mon.noname-b con 0x7f60fc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f60fc09c4a0 0x7f60fc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient(hunting): picked mon.noname-c con 0x7f60fc09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f60fc134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc06daf0 con 0x7f60fc134df0 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:30.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc0a0880 con 0x7f60fc09c4a0 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc06d6f0 con 0x7f60fc06c890 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f610215e640 10 monclient(hunting): authenticate will time out at 1236.349856s 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f60fb7fe640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 msgr2=0x7f60fc06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f60fb7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 0x7f60fc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f60fbfff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f60fc134df0 0x7f60fc135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56980/0 (socket says 10.20.193.78:56980) 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.795+0000 7f60fbfff640 1 -- 10.20.193.78:0/2167455153 learned_addr learned my addr 10.20.193.78:0/2167455153 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 1 --2- 10.20.193.78:0/2167455153 >> v2:10.20.193.115:3300/0 conn(0x7f60fc09c4a0 0x7f60fc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient(hunting): get_auth_request con 0x7f60fc09c4a0 auth_method 0 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient(hunting): handle_auth_done global_id 134152 payload 274 2026-02-17T05:56:30.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 1 -- 10.20.193.78:0/2167455153 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 msgr2=0x7f60fc06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:30.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 1 --2- 10.20.193.78:0/2167455153 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 0x7f60fc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 1 -- 10.20.193.78:0/2167455153 >> v1:10.20.193.78:6789/0 conn(0x7f60fc134df0 legacy=0x7f60fc135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:30.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:30.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 1 monclient: found mon.noname-c 2026-02-17T05:56:30.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:30.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:56:30.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.796+0000 7f60faffd640 1 -- 10.20.193.78:0/2167455153 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc09cc80 con 0x7f60fc09c4a0 2026-02-17T05:56:30.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60faffd640 1 -- 10.20.193.78:0/2167455153 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc09c4a0 msgr2=0x7f60fc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:30.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60faffd640 1 --2- 10.20.193.78:0/2167455153 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc09c4a0 0x7f60fc06d1b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0098a0 tx=0x7f60ec02ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26add9de24be3f25 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:30.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 1 -- 10.20.193.78:0/2167455153 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec03c070 con 0x7f60fc09c4a0 2026-02-17T05:56:30.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:30.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:30.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: dump: 2026-02-17T05:56:30.805 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:30.805 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f610215e640 5 monclient: authenticate success, global_id 134152 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f610215e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 1 -- 10.20.193.78:0/2167455153 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec02fca0 con 0x7f60fc09c4a0 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 1 -- 10.20.193.78:0/2167455153 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec037660 con 0x7f60fc09c4a0 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f610215e640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f610215e640 4 set_mon_vals no callback set 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:30.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.799+0000 7f60fa7fc640 10 monclient: dump: 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient: shutdown 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- 10.20.193.78:0/2167455153 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc09c4a0 msgr2=0x7f60fc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 --2- 10.20.193.78:0/2167455153 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc09c4a0 0x7f60fc06d1b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0098a0 tx=0x7f60ec02ed40 comp rx=0 tx=0).stop 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- 10.20.193.78:0/2167455153 shutdown_connections 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 --2- 10.20.193.78:0/2167455153 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc09c4a0 0x7f60fc06d1b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 --2- 10.20.193.78:0/2167455153 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 0x7f60fc06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- 10.20.193.78:0/2167455153 >> 10.20.193.78:0/2167455153 conn(0x7f60fc0a6620 msgr2=0x7f60fc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- 10.20.193.78:0/2167455153 shutdown_connections 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- 10.20.193.78:0/2167455153 wait complete. 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient: build_initial_monmap 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient: monmap: 2026-02-17T05:56:30.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 Processor -- start 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- start start 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient: init 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 0x7f60fc133630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient(hunting): picked mon.noname-c con 0x7f60fc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60fc09c4a0 0x7f60fc133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient(hunting): picked mon.noname-a con 0x7f60fc09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc134df0 0x7f60fc123110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient(hunting): picked mon.noname-b con 0x7f60fc134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc0a01a0 con 0x7f60fc09c4a0 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc0a0020 con 0x7f60fc134df0 2026-02-17T05:56:30.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc09be20 con 0x7f60fc06c890 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f610215e640 10 monclient(hunting): authenticate will time out at 1236.354964s 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f60fb7fe640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 msgr2=0x7f60fc133630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.800+0000 7f60fb7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 0x7f60fc133630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60fc09c4a0 0x7f60fc133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60fc09c4a0 0x7f60fc133b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51824/0 (socket says 10.20.193.78:51824) 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 -- 10.20.193.78:0/1438377717 learned_addr learned my addr 10.20.193.78:0/1438377717 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient(hunting): get_auth_request con 0x7f60fc09c4a0 auth_method 0 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60fbfff640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc134df0 0x7f60fc123110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60fbfff640 10 monclient(hunting): get_auth_request con 0x7f60fc134df0 auth_method 0 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60fbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60fbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient(hunting): handle_auth_done global_id 124284 payload 824 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 -- 10.20.193.78:0/1438377717 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 msgr2=0x7f60fc133630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 0x7f60fc133630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 -- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc134df0 msgr2=0x7f60fc123110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc134df0 0x7f60fc123110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient: _finish_hunting 0 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 monclient: found mon.noname-a 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.801+0000 7f60faffd640 1 -- 10.20.193.78:0/1438377717 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc126a90 con 0x7f60fc09c4a0 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60faffd640 1 -- 10.20.193.78:0/1438377717 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60fc09c4a0 msgr2=0x7f60fc133b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60faffd640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60fc09c4a0 0x7f60fc133b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0098a0 tx=0x7f60ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 1 -- 10.20.193.78:0/1438377717 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec046070 con 0x7f60fc09c4a0 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: dump: 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:30.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 1 -- 10.20.193.78:0/1438377717 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec02fe50 con 0x7f60fc09c4a0 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f610215e640 5 monclient: authenticate success, global_id 124284 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f610215e640 10 monclient: _renew_subs 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f610215e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc1251c0 con 0x7f60fc09c4a0 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 1 -- 10.20.193.78:0/1438377717 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec03c040 con 0x7f60fc09c4a0 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60f9ffb640 4 set_mon_vals no callback set 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f60d7fff640 10 monclient: dump: 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f610215e640 10 monclient: _renew_subs 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f610215e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:30.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.803+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc0a17c0 con 0x7f60fc09c4a0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f60d7fff640 1 -- 10.20.193.78:0/1438377717 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f60ec041ab0 con 0x7f60fc09c4a0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f610215e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f610215e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f610215e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc1359e0 con 0x7f60fc09c4a0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f60d7fff640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f60cc03de90 0x7f60cc040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f60d7fff640 1 -- 10.20.193.78:0/1438377717 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(369..369 src has 117..369) ==== 4815+0+0 (secure 0 0 0) 0x7f60ec077c20 con 0x7f60fc09c4a0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f60fb7fe640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f60cc03de90 0x7f60cc040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.804+0000 7f60fb7fe640 10 monclient: get_auth_request con 0x7f60cc03de90 auth_method 0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.806+0000 7f60d7fff640 1 -- 10.20.193.78:0/1438377717 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f60ec050070 con 0x7f60fc09c4a0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.806+0000 7f60d7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.806+0000 7f60d7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:30.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.807+0000 7f60fb7fe640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f60cc03de90 0x7f60cc040350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f60e8003a80 tx=0x7f60e8009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:30.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.917+0000 7f610215e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:30.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.917+0000 7f610215e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:30.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.917+0000 7f610215e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:30.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.917+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f60fc0744d0 con 0x7f60fc09c4a0 2026-02-17T05:56:30.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f60d7fff640 1 -- 10.20.193.78:0/1438377717 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f60ec0363a0 con 0x7f60fc09c4a0 2026-02-17T05:56:30.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f60d7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:30.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f60d7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:30.920 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:30.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f60cc03de90 msgr2=0x7f60cc040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:30.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f610215e640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f60cc03de90 0x7f60cc040350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f60e8003a80 tx=0x7f60e8009270 comp rx=0 tx=0).stop 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f610215e640 10 monclient: shutdown 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f610215e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60fc09c4a0 msgr2=0x7f60fc133b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f610215e640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60fc09c4a0 0x7f60fc133b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0098a0 tx=0x7f60ec0047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.918+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 shutdown_connections 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.919+0000 7f610215e640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f60cc03de90 0x7f60cc040350 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.919+0000 7f610215e640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60fc134df0 0x7f60fc123110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.919+0000 7f610215e640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f60fc09c4a0 0x7f60fc133b70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.919+0000 7f610215e640 1 --2- 10.20.193.78:0/1438377717 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60fc06c890 0x7f60fc133630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.919+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 >> 10.20.193.78:0/1438377717 conn(0x7f60fc0a6620 msgr2=0x7f60fc0a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:30.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.919+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 shutdown_connections 2026-02-17T05:56:30.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:30.919+0000 7f610215e640 1 -- 10.20.193.78:0/1438377717 wait complete. 2026-02-17T05:56:33.928 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.995+0000 7f079fbc1640 10 monclient: get_monmap_and_config 2026-02-17T05:56:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.995+0000 7f079fbc1640 10 monclient: build_initial_monmap 2026-02-17T05:56:33.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.995+0000 7f079fbc1640 10 monclient: monmap: 2026-02-17T05:56:33.996 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:33.996 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:33.996 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:33.996730+0000 2026-02-17T05:56:33.996 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:33.996730+0000 2026-02-17T05:56:33.997 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:33.997 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:33.997 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:33.997 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:33.997 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:33.997 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:33.997 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:33.997 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.995+0000 7f079fbc1640 1 Processor -- start 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.995+0000 7f079fbc1640 1 -- start start 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.995+0000 7f079fbc1640 10 monclient: init 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 10 monclient: _add_conns ranks=[0,4,1,3,2] 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f079806c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 10 monclient(hunting): picked mon.noname-c con 0x7f079809d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f079809bf40 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0798073600 con 0x7f079806c890 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f079806dab0 con 0x7f079809bf40 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f079809ca50 con 0x7f079809d430 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079fbc1640 10 monclient(hunting): authenticate will time out at 1239.550353s 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079d936640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f079806c890 0x7f079806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56988/0 (socket says 10.20.193.78:56988) 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079d936640 1 -- 10.20.193.78:0/1197890948 learned_addr learned my addr 10.20.193.78:0/1197890948 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079d135640 10 monclient(hunting): get_auth_request con 0x7f079809d430 auth_method 0 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079d135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079d135640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:33.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.996+0000 7f079d135640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.998+0000 7f079d936640 1 -- 10.20.193.78:0/1197890948 >> v1:10.20.193.78:6789/0 conn(0x7f079806c890 legacy=0x7f079806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.998+0000 7f079e137640 1 -- 10.20.193.78:0/1197890948 >> v1:10.20.193.115:6789/0 conn(0x7f079809bf40 legacy=0x7f079813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.998+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:56:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.998+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:56:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.998+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:33.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:33.998+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:34.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:34.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 10 monclient(hunting): get_auth_request con 0x7f079809d430 auth_method 0 2026-02-17T05:56:34.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:34.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:34.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:34.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:56:34.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:56:34.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:34.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.199+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 10 monclient(hunting): get_auth_request con 0x7f079809d430 auth_method 0 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:34.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:34.600+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:56:35.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:35.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 10 monclient(hunting): get_auth_request con 0x7f079809d430 auth_method 0 2026-02-17T05:56:35.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:35.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:35.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:35.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:56:35.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:56:35.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:35.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:35.401+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:56:36.050 INFO:tasks.rados.rados.0.trial115.stdout:1770: done (4 left) 2026-02-17T05:56:36.050 INFO:tasks.rados.rados.0.trial115.stdout:1771: copy_from oid 38 from oid 3 current snap is 174 2026-02-17T05:56:36.050 INFO:tasks.rados.rados.0.trial115.stdout:1766: finishing write tid 3 to trial11545420-44 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1766: finishing write tid 4 to trial11545420-44 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1766: finishing write tid 5 to trial11545420-44 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1766: oid 44 updating version 0 to 859 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1766: oid 44 version 859 is already newer than 858 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1766: oid 44 updating version 859 to 860 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1766: oid 44 version 860 is already newer than 857 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 860 (ObjNum 649 snap 174 seq_num 649) dirty exists 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1766: left oid 44 (ObjNum 649 snap 174 seq_num 649) 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1769: finishing rollback tid 0 to trial11545420-1 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1769: finishing rollback tid 1 to trial11545420-1 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1767: finishing rollback tid 0 to trial11545420-25 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:1769: finishing rollback tid 2 to trial11545420-1 2026-02-17T05:56:36.051 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 635 (ObjNum 628 snap 164 seq_num 628) dirty exists 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1767: finishing rollback tid 1 to trial11545420-25 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1767: finishing rollback tid 2 to trial11545420-25 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 858 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1766: done (4 left) 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1767: done (3 left) 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1768: done (2 left) 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1769: done (1 left) 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1772: rollback oid 49 current snap is 174 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 49 to 172 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1773: snap_create 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1772: finishing rollback tid 0 to trial11545420-49 2026-02-17T05:56:36.052 INFO:tasks.rados.rados.0.trial115.stdout:1772: finishing rollback tid 1 to trial11545420-49 2026-02-17T05:56:36.053 INFO:tasks.rados.rados.0.trial115.stdout:1772: finishing rollback tid 2 to trial11545420-49 2026-02-17T05:56:36.053 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 813 (ObjNum 614 snap 158 seq_num 614) dirty exists 2026-02-17T05:56:36.054 INFO:tasks.rados.rados.0.trial115.stdout:1771: finishing copy_from racing read to trial11545420-38 2026-02-17T05:56:36.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1595451240 0 0) 0x7f079806dab0 con 0x7f079809bf40 2026-02-17T05:56:36.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): _negotiate 2026-02-17T05:56:36.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): my global_id is 134158 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0780003620 con 0x7f079809bf40 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2886375022 0 0) 0x7f0780003620 con 0x7f079809bf40 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 msgr2=0x7f079813d5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 --2- 10.20.193.78:0/1197890948 >> v2:10.20.193.115:3300/0 conn(0x7f079809d430 0x7f079813d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 >> v1:10.20.193.78:6789/0 conn(0x7f079806c890 legacy=0x7f079806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: _finish_hunting 0 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079809c770 con 0x7f079809bf40 2026-02-17T05:56:36.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: _finish_auth 0 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: _check_auth_tickets 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f0794003580 con 0x7f079809bf40 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079fbc1640 5 monclient: authenticate success, global_id 134158 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079fbc1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: dump: 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.059 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 >> v1:10.20.193.115:6789/0 conn(0x7f079809bf40 legacy=0x7f079813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0780003c20 0x7f0780004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): picked mon.b con 0x7f0780003c20 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780004900 0x7f0780004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): picked mon.c con 0x7f0780004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0780009ab0 0x7f078000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): picked mon.a con 0x7f0780009ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f079809ca50 con 0x7f0780009ab0 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f078000c4f0 con 0x7f0780003c20 2026-02-17T05:56:36.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f078000c6f0 con 0x7f0780004900 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079c934640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079fbc1640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079d135640 1 -- 10.20.193.78:0/1197890948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780004900 msgr2=0x7f0780004d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079d135640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780004900 0x7f0780004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079d936640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0780003c20 0x7f0780004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079d936640 10 monclient(hunting): get_auth_request con 0x7f0780003c20 auth_method 0 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079d936640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079d936640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.041+0000 7f079d936640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 10 monclient(hunting): handle_auth_done global_id 134158 payload 293 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 1 -- 10.20.193.78:0/1197890948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780004900 msgr2=0x7f0780004d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780004900 0x7f0780004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 1 -- 10.20.193.78:0/1197890948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0780009ab0 msgr2=0x7f078000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0780009ab0 0x7f078000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 10 monclient: _finish_hunting 0 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 1 monclient: found mon.b 2026-02-17T05:56:36.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.115:3300/0 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079e137640 1 -- 10.20.193.78:0/1197890948 reap_dead start 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 1 -- 10.20.193.78:0/1197890948 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0780003620 con 0x7f0780003c20 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 10 monclient: _finish_auth 0 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 10 monclient: _check_auth_tickets 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.042+0000 7f079d936640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.044+0000 7f079d936640 1 -- 10.20.193.78:0/1197890948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0780003c20 msgr2=0x7f0780004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.044+0000 7f079d936640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0780003c20 0x7f0780004060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0788003360 tx=0x7f078802f0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5200ad560d17b42 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788003060 con 0x7f0780003c20 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:36.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 10 monclient: dump: 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:36.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0788033030 con 0x7f0780003c20 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 1 -- 10.20.193.78:0/1197890948 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078802f8f0 con 0x7f0780003c20 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079c934640 10 monclient: dump: 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.066 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 4 set_mon_vals no callback set 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 10 monclient: shutdown 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 1 -- 10.20.193.78:0/1197890948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0780003c20 msgr2=0x7f0780004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0780003c20 0x7f0780004060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0788003360 tx=0x7f078802f0d0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 1 -- 10.20.193.78:0/1197890948 shutdown_connections 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 1 --2- 10.20.193.78:0/1197890948 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0780003c20 0x7f0780004060 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 1 -- 10.20.193.78:0/1197890948 >> 10.20.193.78:0/1197890948 conn(0x7f07980a6620 msgr2=0x7f07980a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 1 -- 10.20.193.78:0/1197890948 shutdown_connections 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.045+0000 7f079fbc1640 1 -- 10.20.193.78:0/1197890948 wait complete. 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient: build_initial_monmap 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient: monmap: 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:36.068 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 1 Processor -- start 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 1 -- start start 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient: init 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780003c20 0x7f07981cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient(hunting): picked mon.noname-c con 0x7f0780003c20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0798135400 0x7f0798135880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient(hunting): picked mon.noname-a con 0x7f0798135400 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0798122c40 0x7f07981230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient(hunting): picked mon.noname-b con 0x7f0798122c40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f079806cc70 con 0x7f0798135400 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0798073410 con 0x7f0798122c40 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f079806cdf0 con 0x7f0780003c20 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079fbc1640 10 monclient(hunting): authenticate will time out at 1241.600865s 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d936640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780003c20 msgr2=0x7f07981cf8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d936640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780003c20 0x7f07981cf8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d135640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0798135400 0x7f0798135880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d135640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0798135400 0x7f0798135880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51830/0 (socket says 10.20.193.78:51830) 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d135640 1 -- 10.20.193.78:0/1922394307 learned_addr learned my addr 10.20.193.78:0/1922394307 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d135640 10 monclient(hunting): get_auth_request con 0x7f0798135400 auth_method 0 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d135640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.046+0000 7f079d135640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079e137640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0798122c40 0x7f07981230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079e137640 10 monclient(hunting): get_auth_request con 0x7f0798122c40 auth_method 0 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079e137640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079e137640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079e137640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 10 monclient(hunting): handle_auth_done global_id 124290 payload 824 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 1 -- 10.20.193.78:0/1922394307 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780003c20 msgr2=0x7f07981cf8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780003c20 0x7f07981cf8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 1 -- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0798122c40 msgr2=0x7f07981230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0798122c40 0x7f07981230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 10 monclient: _finish_hunting 0 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 1 monclient: found mon.noname-a 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.047+0000 7f079d135640 1 -- 10.20.193.78:0/1922394307 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07981268f0 con 0x7f0798135400 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079d135640 1 -- 10.20.193.78:0/1922394307 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0798135400 msgr2=0x7f0798135880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079d135640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0798135400 0x7f0798135880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f078c014960 tx=0x7f078c014e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 1 -- 10.20.193.78:0/1922394307 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c01d070 con 0x7f0798135400 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: dump: 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.073 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: _finish_auth 0 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: _check_auth_tickets 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079fbc1640 5 monclient: authenticate success, global_id 124290 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079fbc1640 10 monclient: _renew_subs 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079fbc1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0798124d80 con 0x7f0798135400 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 1 -- 10.20.193.78:0/1922394307 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078c017470 con 0x7f0798135400 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:36.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f0787fff640 4 set_mon_vals no callback set 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079fbc1640 10 monclient: _renew_subs 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079fbc1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0798135dc0 con 0x7f0798135400 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 1 -- 10.20.193.78:0/1922394307 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c027060 con 0x7f0798135400 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.049+0000 7f07867fc640 10 monclient: dump: 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.075 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f07867fc640 1 -- 10.20.193.78:0/1922394307 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f078c015040 con 0x7f0798135400 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f079fbc1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f079fbc1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f079fbc1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f079806d170 con 0x7f0798135400 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f07867fc640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f076803de90 0x7f0768040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f07867fc640 1 -- 10.20.193.78:0/1922394307 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(370..370 src has 117..370) ==== 4859+0+0 (secure 0 0 0) 0x7f078c053260 con 0x7f0798135400 2026-02-17T05:56:36.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f079d936640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f076803de90 0x7f0768040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.050+0000 7f079d936640 10 monclient: get_auth_request con 0x7f076803de90 auth_method 0 2026-02-17T05:56:36.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.052+0000 7f079d936640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f076803de90 0x7f0768040350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0788003330 tx=0x7f0788007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.053+0000 7f07867fc640 1 -- 10.20.193.78:0/1922394307 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f078c051c10 con 0x7f0798135400 2026-02-17T05:56:36.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.053+0000 7f07867fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:36.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.053+0000 7f07867fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:36.078 INFO:tasks.rados.rados.0.trial115.stdout:1771: finishing copy_from to trial11545420-38 2026-02-17T05:56:36.078 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 603 (ObjNum 598 snap 155 seq_num 598) dirty exists 2026-02-17T05:56:36.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.169+0000 7f079fbc1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:36.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.169+0000 7f079fbc1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:36.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.169+0000 7f079fbc1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.169+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f079809d810 con 0x7f0798135400 2026-02-17T05:56:36.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.169+0000 7f07867fc640 1 -- 10.20.193.78:0/1922394307 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f078c02bcc0 con 0x7f0798135400 2026-02-17T05:56:36.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.169+0000 7f07867fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:36.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.169+0000 7f07867fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:36.172 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":116,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f076803de90 msgr2=0x7f0768040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f076803de90 0x7f0768040350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0788003330 tx=0x7f0788007000 comp rx=0 tx=0).stop 2026-02-17T05:56:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 10 monclient: shutdown 2026-02-17T05:56:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0798135400 msgr2=0x7f0798135880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0798135400 0x7f0798135880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f078c014960 tx=0x7f078c014e30 comp rx=0 tx=0).stop 2026-02-17T05:56:36.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 shutdown_connections 2026-02-17T05:56:36.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f076803de90 0x7f0768040350 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0798122c40 0x7f07981230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0798135400 0x7f0798135880 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 --2- 10.20.193.78:0/1922394307 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0780003c20 0x7f07981cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 >> 10.20.193.78:0/1922394307 conn(0x7f07980a6620 msgr2=0x7f07980a0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:36.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 shutdown_connections 2026-02-17T05:56:36.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.170+0000 7f079fbc1640 1 -- 10.20.193.78:0/1922394307 wait complete. 2026-02-17T05:56:36.180 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:56:36.180 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T05:56:36.180 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:56:36.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.246+0000 7ff1d6fc1640 10 monclient: get_monmap_and_config 2026-02-17T05:56:36.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.246+0000 7ff1d6fc1640 10 monclient: build_initial_monmap 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.246+0000 7ff1d6fc1640 10 monclient: monmap: 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:36.247943+0000 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:36.247943+0000 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:36.251 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:36.252 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:36.252 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:36.252 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 1 Processor -- start 2026-02-17T05:56:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 1 -- start start 2026-02-17T05:56:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient: init 2026-02-17T05:56:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff1d006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff1d009c4a0 0x7ff1d013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient(hunting): picked mon.noname-c con 0x7ff1d009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff1d006d380 0x7ff1d013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient(hunting): picked mon.noname-a con 0x7ff1d006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d006daf0 con 0x7ff1d006c890 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d00a0880 con 0x7ff1d006d380 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d009cc80 con 0x7ff1d009c4a0 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d6fc1640 10 monclient(hunting): authenticate will time out at 1241.801514s 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff1d006d380 0x7ff1d013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff1d006d380 0x7ff1d013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51846/0 (socket says 10.20.193.78:51846) 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 1 -- 10.20.193.78:0/1696349805 learned_addr learned my addr 10.20.193.78:0/1696349805 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient(hunting): get_auth_request con 0x7ff1d006d380 auth_method 0 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient(hunting): handle_auth_done global_id 124293 payload 274 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1c7fff640 1 --2- 10.20.193.78:0/1696349805 >> v2:10.20.193.115:3300/0 conn(0x7ff1d009c4a0 0x7ff1d013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 1 -- 10.20.193.78:0/1696349805 >> v2:10.20.193.115:3300/0 conn(0x7ff1d009c4a0 msgr2=0x7ff1d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 1 --2- 10.20.193.78:0/1696349805 >> v2:10.20.193.115:3300/0 conn(0x7ff1d009c4a0 0x7ff1d013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 1 -- 10.20.193.78:0/1696349805 >> v1:10.20.193.115:6789/0 conn(0x7ff1d006c890 legacy=0x7ff1d006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient: _finish_hunting 0 2026-02-17T05:56:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 1 monclient: found mon.noname-a 2026-02-17T05:56:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1d5537640 1 -- 10.20.193.78:0/1696349805 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d009cf60 con 0x7ff1d006d380 2026-02-17T05:56:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1c7fff640 10 monclient: get_auth_request con 0x7ff1d009c4a0 auth_method 0 2026-02-17T05:56:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.247+0000 7ff1c7fff640 1 --2- 10.20.193.78:0/1696349805 >> v2:10.20.193.115:3300/0 conn(0x7ff1d009c4a0 0x7ff1d013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:56:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d5537640 1 -- 10.20.193.78:0/1696349805 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d006d380 msgr2=0x7ff1d013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d5537640 1 --2- 10.20.193.78:0/1696349805 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d006d380 0x7ff1d013d8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1d006dcc0 tx=0x7ff1cc02f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afae162f5daf79a0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 1 -- 10.20.193.78:0/1696349805 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc02fe50 con 0x7ff1d006d380 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: dump: 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.267 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 1 -- 10.20.193.78:0/1696349805 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc0373f0 con 0x7ff1d006d380 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 1 -- 10.20.193.78:0/1696349805 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0377c0 con 0x7ff1d006d380 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 5 monclient: authenticate success, global_id 124293 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1c77fe640 10 monclient: dump: 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.269 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 4 set_mon_vals no callback set 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 10 monclient: shutdown 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/1696349805 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d006d380 msgr2=0x7ff1d013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 1 --2- 10.20.193.78:0/1696349805 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d006d380 0x7ff1d013d8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1d006dcc0 tx=0x7ff1cc02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/1696349805 shutdown_connections 2026-02-17T05:56:36.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 1 --2- 10.20.193.78:0/1696349805 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d006d380 0x7ff1d013d8f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 1 --2- 10.20.193.78:0/1696349805 >> v2:10.20.193.115:3300/0 conn(0x7ff1d009c4a0 0x7ff1d013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/1696349805 >> 10.20.193.78:0/1696349805 conn(0x7ff1d00a6620 msgr2=0x7ff1d00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/1696349805 shutdown_connections 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.250+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/1696349805 wait complete. 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient: build_initial_monmap 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient: monmap: 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:36.271 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 1 Processor -- start 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 1 -- start start 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient: init 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff1d006c890 0x7ff1d01cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient(hunting): picked mon.noname-c con 0x7ff1d006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:36.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff1d006d380 0x7ff1d01cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient(hunting): picked mon.noname-b con 0x7ff1d006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d009c4a0 0x7ff1d01cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient(hunting): picked mon.noname-a con 0x7ff1d009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d009bfa0 con 0x7ff1d009c4a0 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d009be20 con 0x7ff1d006d380 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d013de30 con 0x7ff1d006c890 2026-02-17T05:56:36.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d6fc1640 10 monclient(hunting): authenticate will time out at 1241.805516s 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d4d36640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff1d006c890 msgr2=0x7ff1d01cf020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d4d36640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff1d006c890 0x7ff1d01cf020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d009c4a0 0x7ff1d01cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d009c4a0 0x7ff1d01cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51860/0 (socket says 10.20.193.78:51860) 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 -- 10.20.193.78:0/3020336038 learned_addr learned my addr 10.20.193.78:0/3020336038 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient(hunting): get_auth_request con 0x7ff1d009c4a0 auth_method 0 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient(hunting): handle_auth_done global_id 124296 payload 824 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff1d006c890 msgr2=0x7ff1d01cf020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff1d006c890 0x7ff1d01cf020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff1d006d380 msgr2=0x7ff1d01cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1c7fff640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff1d006d380 0x7ff1d01cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff1d006d380 0x7ff1d01cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient: _finish_hunting 0 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 monclient: found mon.noname-a 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1d5537640 1 -- 10.20.193.78:0/3020336038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d011fac0 con 0x7ff1d009c4a0 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1c7fff640 10 monclient: get_auth_request con 0x7ff1d006d380 auth_method 0 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.251+0000 7ff1c7fff640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff1d006d380 0x7ff1d01cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.253+0000 7ff1d5537640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d009c4a0 msgr2=0x7ff1d01cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.253+0000 7ff1d5537640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d009c4a0 0x7ff1d01cfaa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc039040 tx=0x7ff1cc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc046020 con 0x7ff1d009c4a0 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: dump: 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: _finish_auth 0 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: _check_auth_tickets 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc041450 con 0x7ff1d009c4a0 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 5 monclient: authenticate success, global_id 124296 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc041840 con 0x7ff1d009c4a0 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 10 monclient: dump: 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.278 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c6ffd640 4 set_mon_vals no callback set 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 10 monclient: _renew_subs 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d009e2e0 con 0x7ff1d009c4a0 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 10 monclient: _renew_subs 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d0073030 con 0x7ff1d009c4a0 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d011e2c0 0x7ff1d0132af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d011f460 con 0x7ff1d011e2c0 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff1cc053060 con 0x7ff1d009c4a0 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d011e2c0 0x7ff1d0132af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 10 monclient: get_auth_request con 0x7ff1d011e2c0 auth_method 0 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff1a003de60 0x7ff1a0040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 10 monclient(hunting): handle_auth_done global_id 124299 payload 824 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1d4d36640 1 -- 10.20.193.78:0/3020336038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1d0074980 con 0x7ff1d011e2c0 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.254+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(370..370 src has 117..370) ==== 4859+0+0 (secure 0 0 0) 0x7ff1cc078210 con 0x7ff1d009c4a0 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.255+0000 7ff1c7fff640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff1a003de60 0x7ff1a0040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.255+0000 7ff1c7fff640 10 monclient: get_auth_request con 0x7ff1a003de60 auth_method 0 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.256+0000 7ff1d4d36640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d011e2c0 msgr2=0x7ff1d0132af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1d4d36640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d011e2c0 0x7ff1d0132af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c430 tx=0x7ff1c000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0017070 con 0x7ff1d011e2c0 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 10 monclient: dump: 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff1c0007070 con 0x7ff1d011e2c0 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d011e2c0 msgr2=0x7ff1d0132af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d011e2c0 0x7ff1d0132af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c430 tx=0x7ff1c000c900 comp rx=0 tx=0).stop 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d011e2c0 msgr2=0x7ff1d0132af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c57fa640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d011e2c0 0x7ff1d0132af0 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c430 tx=0x7ff1c000c900 comp rx=0 tx=0).stop 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.257+0000 7ff1c7fff640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff1a003de60 0x7ff1a0040320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8006fd0 tx=0x7ff1b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d6fc1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d6fc1640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d0133030 0x7ff1d01da580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d6fc1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d006cc70 con 0x7ff1d0133030 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d0133030 0x7ff1d01da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 10 monclient: get_auth_request con 0x7ff1d0133030 auth_method 0 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 10 monclient(hunting): handle_auth_done global_id 124302 payload 824 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.260+0000 7ff1d4d36640 1 -- 10.20.193.78:0/3020336038 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff1d00744d0 con 0x7ff1d0133030 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d4d36640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d0133030 msgr2=0x7ff1d01da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d4d36640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d0133030 0x7ff1d01da580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c430 tx=0x7ff1c00038e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0017030 con 0x7ff1d0133030 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 10 monclient: dump: 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.284 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4699 (secure 0 0 0) 0x7ff1c0019b60 con 0x7ff1d0133030 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d0133030 msgr2=0x7ff1d01da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d0133030 0x7ff1d01da580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c430 tx=0x7ff1c00038e0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d0133030 msgr2=0x7ff1d01da580 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:36.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1c57fa640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d0133030 0x7ff1d01da580 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c430 tx=0x7ff1c00038e0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.286 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:56:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:56:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 116, 2026-02-17T05:56:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 87730, 2026-02-17T05:56:36.286 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:36.287 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:36.288 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:56:36.289 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:56:36.290 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.291 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.292 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:36.293 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:56:36.294 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.295 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.296 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.296 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:56:36.296 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:56:36.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff1a003de60 msgr2=0x7ff1a0040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff1a003de60 0x7ff1a0040320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8006fd0 tx=0x7ff1b8008040 comp rx=0 tx=0).stop 2026-02-17T05:56:36.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 10 monclient: shutdown 2026-02-17T05:56:36.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:36.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d009c4a0 msgr2=0x7ff1d01cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 1 --2- 10.20.193.78:0/3020336038 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff1d009c4a0 0x7ff1d01cfaa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc039040 tx=0x7ff1cc004b40 comp rx=0 tx=0).stop 2026-02-17T05:56:36.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d5537640 1 -- 10.20.193.78:0/3020336038 reap_dead start 2026-02-17T05:56:36.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d5537640 1 -- 10.20.193.78:0/3020336038 reap_dead start 2026-02-17T05:56:36.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 shutdown_connections 2026-02-17T05:56:36.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 >> 10.20.193.78:0/3020336038 conn(0x7ff1d00a6620 msgr2=0x7ff1d0133ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:36.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 shutdown_connections 2026-02-17T05:56:36.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.263+0000 7ff1d6fc1640 1 -- 10.20.193.78:0/3020336038 wait complete. 2026-02-17T05:56:36.298 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:56:36.298 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:56:36.298 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:56:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.364+0000 7fc14c898640 10 monclient: get_monmap_and_config 2026-02-17T05:56:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.364+0000 7fc14c898640 10 monclient: build_initial_monmap 2026-02-17T05:56:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.364+0000 7fc14c898640 10 monclient: monmap: 2026-02-17T05:56:36.365 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:36.365 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.365 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:36.365992+0000 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:36.365992+0000 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 1 Processor -- start 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 1 -- start start 2026-02-17T05:56:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient: init 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc14409ddd0 0x7fc14409e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient(hunting): picked mon.noname-c con 0x7fc14409ddd0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc144073f60 addr v1:10.20.193.78:6789/0 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc144138660 addr v1:10.20.193.115:6789/0 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1440a2850 con 0x7fc144073f60 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1440a2560 con 0x7fc144138660 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc1440a3810 con 0x7fc14409ddd0 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14c898640 10 monclient(hunting): authenticate will time out at 1241.919759s 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc149e0c640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc144073f60 0x7fc144137f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:57002/0 (socket says 10.20.193.78:57002) 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc149e0c640 1 -- 10.20.193.78:0/1272686232 learned_addr learned my addr 10.20.193.78:0/1272686232 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14a60d640 1 --2- 10.20.193.78:0/1272686232 >> v2:10.20.193.115:3300/0 conn(0x7fc14409ddd0 0x7fc14409e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14a60d640 10 monclient(hunting): get_auth_request con 0x7fc14409ddd0 auth_method 0 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14a60d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14a60d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.365+0000 7fc14a60d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.366+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.366+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.366+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.367+0000 7fc149e0c640 1 -- 10.20.193.78:0/1272686232 >> v1:10.20.193.78:6789/0 conn(0x7fc144073f60 legacy=0x7fc144137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14ae0e640 1 -- 10.20.193.78:0/1272686232 >> v1:10.20.193.115:6789/0 conn(0x7fc144138660 legacy=0x7fc144140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14a60d640 10 monclient(hunting): handle_auth_done global_id 134164 payload 274 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14a60d640 1 -- 10.20.193.78:0/1272686232 >> v1:10.20.193.115:6789/0 conn(0x7fc144138660 legacy=0x7fc144140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14960b640 1 -- 10.20.193.78:0/1272686232 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553634092 0 0) 0x7fc1440a2850 con 0x7fc144073f60 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14a60d640 1 -- 10.20.193.78:0/1272686232 >> v1:10.20.193.78:6789/0 conn(0x7fc144073f60 legacy=0x7fc144137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14a60d640 10 monclient: _finish_hunting 0 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14a60d640 1 monclient: found mon.noname-c 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14a60d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14a60d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14a60d640 1 -- 10.20.193.78:0/1272686232 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1440a3af0 con 0x7fc14409ddd0 2026-02-17T05:56:36.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.368+0000 7fc14960b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T05:56:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.370+0000 7fc14a60d640 1 -- 10.20.193.78:0/1272686232 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 msgr2=0x7fc14409e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.370+0000 7fc14a60d640 1 --2- 10.20.193.78:0/1272686232 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 0x7fc14409e1b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0099a0 tx=0x7fc13c02eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391dba12b8d24da server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 1 -- 10.20.193.78:0/1272686232 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c03c070 con 0x7fc14409ddd0 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: dump: 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: _finish_auth 0 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: _check_auth_tickets 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 5 monclient: authenticate success, global_id 134164 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 1 -- 10.20.193.78:0/1272686232 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13c037440 con 0x7fc14409ddd0 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 1 -- 10.20.193.78:0/1272686232 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c037810 con 0x7fc14409ddd0 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 4 set_mon_vals no callback set 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14960b640 10 monclient: dump: 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.372 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 10 monclient: shutdown 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 1 -- 10.20.193.78:0/1272686232 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 msgr2=0x7fc14409e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 1 --2- 10.20.193.78:0/1272686232 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 0x7fc14409e1b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0099a0 tx=0x7fc13c02eee0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 1 -- 10.20.193.78:0/1272686232 shutdown_connections 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 1 --2- 10.20.193.78:0/1272686232 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 0x7fc14409e1b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 1 -- 10.20.193.78:0/1272686232 >> 10.20.193.78:0/1272686232 conn(0x7fc1440aacd0 msgr2=0x7fc1440ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 1 -- 10.20.193.78:0/1272686232 shutdown_connections 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 1 -- 10.20.193.78:0/1272686232 wait complete. 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 10 monclient: build_initial_monmap 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.371+0000 7fc14c898640 10 monclient: monmap: 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 1 Processor -- start 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 1 -- start start 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient: init 2026-02-17T05:56:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc144073f60 0x7fc1441cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient(hunting): picked mon.noname-c con 0x7fc144073f60 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 0x7fc1441cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient(hunting): picked mon.noname-b con 0x7fc14409ddd0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc144138660 0x7fc1441cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient(hunting): picked mon.noname-a con 0x7fc144138660 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc14409e6f0 con 0x7fc144138660 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440a57e0 con 0x7fc14409ddd0 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc1440a4840 con 0x7fc144073f60 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14c898640 10 monclient(hunting): authenticate will time out at 1241.926439s 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14a60d640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc144073f60 msgr2=0x7fc1441cefa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14a60d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc144073f60 0x7fc1441cefa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc144138660 0x7fc1441cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc144138660 0x7fc1441cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51886/0 (socket says 10.20.193.78:51886) 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 -- 10.20.193.78:0/3447819784 learned_addr learned my addr 10.20.193.78:0/3447819784 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient(hunting): get_auth_request con 0x7fc144138660 auth_method 0 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient(hunting): handle_auth_done global_id 124308 payload 824 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc149e0c640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 0x7fc1441cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc144073f60 msgr2=0x7fc1441cefa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc144073f60 0x7fc1441cefa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 msgr2=0x7fc1441cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 0x7fc1441cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient: _finish_hunting 0 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 monclient: found mon.noname-a 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc14ae0e640 1 -- 10.20.193.78:0/3447819784 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc144132290 con 0x7fc144138660 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc149e0c640 10 monclient: get_auth_request con 0x7fc14409ddd0 auth_method 0 2026-02-17T05:56:36.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.372+0000 7fc149e0c640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc14409ddd0 0x7fc1441cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:56:36.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.374+0000 7fc14ae0e640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc144138660 msgr2=0x7fc1441cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.374+0000 7fc14ae0e640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc144138660 0x7fc1441cfa20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc13800ce20 tx=0x7fc13800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138017070 con 0x7fc144138660 2026-02-17T05:56:36.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:36.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 10 monclient: dump: 2026-02-17T05:56:36.377 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.377 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.377 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 10 monclient: _finish_auth 0 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 10 monclient: _check_auth_tickets 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc14c898640 5 monclient: authenticate success, global_id 124308 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc138007270 con 0x7fc144138660 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc14c898640 10 monclient: _renew_subs 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc14c898640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc144132570 con 0x7fc144138660 2026-02-17T05:56:36.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.376+0000 7fc12effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138007620 con 0x7fc144138660 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc12effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc148e0a640 4 set_mon_vals no callback set 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc12effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc12effd640 10 monclient: dump: 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14c898640 10 monclient: _renew_subs 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14c898640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc144073500 con 0x7fc144138660 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14c898640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14c898640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130030 0x7fc144130410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14c898640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc144069d50 con 0x7fc144130030 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc1380077c0 con 0x7fc144138660 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14a60d640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130030 0x7fc144130410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14a60d640 10 monclient: get_auth_request con 0x7fc144130030 auth_method 0 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14a60d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14a60d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14a60d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc12effd640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc11403de60 0x7fc114040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.377+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(370..370 src has 117..370) ==== 4859+0+0 (secure 0 0 0) 0x7fc1380529b0 con 0x7fc144138660 2026-02-17T05:56:36.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.378+0000 7fc14a60d640 10 monclient(hunting): handle_auth_done global_id 134170 payload 824 2026-02-17T05:56:36.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.378+0000 7fc14a60d640 1 -- 10.20.193.78:0/3447819784 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc144138b10 con 0x7fc144130030 2026-02-17T05:56:36.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.378+0000 7fc149e0c640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc11403de60 0x7fc114040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.378+0000 7fc149e0c640 10 monclient: get_auth_request con 0x7fc11403de60 auth_method 0 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc14a60d640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130030 msgr2=0x7fc144130410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc14a60d640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130030 0x7fc144130410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc13c039000 tx=0x7fc13c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c03c040 con 0x7fc144130030 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 10 monclient: dump: 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc149e0c640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc11403de60 0x7fc114040320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc144073a00 tx=0x7fc130009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fc13c042670 con 0x7fc144130030 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:36.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130030 msgr2=0x7fc144130410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130030 0x7fc144130410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc13c039000 tx=0x7fc13c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130030 msgr2=0x7fc144130410 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:36.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.380+0000 7fc12effd640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130030 0x7fc144130410 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc13c039000 tx=0x7fc13c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:56:36.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14c898640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:56:36.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14c898640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130950 0x7fc1441da2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:36.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14c898640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:36.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440a1cf0 con 0x7fc144130950 2026-02-17T05:56:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130950 0x7fc1441da2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 10 monclient: get_auth_request con 0x7fc144130950 auth_method 0 2026-02-17T05:56:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:36.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:36.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 10 monclient(hunting): handle_auth_done global_id 134173 payload 824 2026-02-17T05:56:36.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.384+0000 7fc14a60d640 1 -- 10.20.193.78:0/3447819784 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc144073c30 con 0x7fc144130950 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.386+0000 7fc14a60d640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130950 msgr2=0x7fc1441da2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.386+0000 7fc14a60d640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130950 0x7fc1441da2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0099a0 tx=0x7fc13c009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.386+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c03c050 con 0x7fc144130950 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.386+0000 7fc12effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.386+0000 7fc12effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.386+0000 7fc12effd640 10 monclient: dump: 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:36.388 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7fc13c04db20 con 0x7fc144130950 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc12effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc12effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130950 msgr2=0x7fc1441da2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc12effd640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130950 0x7fc1441da2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0099a0 tx=0x7fc13c009290 comp rx=0 tx=0).stop 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc12effd640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130950 msgr2=0x7fc1441da2f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc12effd640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc144130950 0x7fc1441da2f0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0099a0 tx=0x7fc13c009290 comp rx=0 tx=0).stop 2026-02-17T05:56:36.389 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 116, 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 24106, 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:36.390 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:36.391 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:56:36.392 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:56:36.393 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.394 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:36.395 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:56:36.396 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.397 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:36.398 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:56:36.399 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc11403de60 msgr2=0x7fc114040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc11403de60 0x7fc114040320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc144073a00 tx=0x7fc130009040 comp rx=0 tx=0).stop 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 10 monclient: shutdown 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc144138660 msgr2=0x7fc1441cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 1 --2- 10.20.193.78:0/3447819784 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc144138660 0x7fc1441cfa20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc13800ce20 tx=0x7fc13800a430 comp rx=0 tx=0).stop 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14ae0e640 1 -- 10.20.193.78:0/3447819784 reap_dead start 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14ae0e640 1 -- 10.20.193.78:0/3447819784 reap_dead start 2026-02-17T05:56:36.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 shutdown_connections 2026-02-17T05:56:36.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 >> 10.20.193.78:0/3447819784 conn(0x7fc1440aacd0 msgr2=0x7fc1440a0910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:36.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 shutdown_connections 2026-02-17T05:56:36.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:36.387+0000 7fc14c898640 1 -- 10.20.193.78:0/3447819784 wait complete. 2026-02-17T05:56:36.401 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:56:36.401 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:56:37.046 INFO:tasks.rados.rados.0.trial115.stdout:1771: done (2 left) 2026-02-17T05:56:37.046 INFO:tasks.rados.rados.0.trial115.stdout:1772: done (1 left) 2026-02-17T05:56:37.046 INFO:tasks.rados.rados.0.trial115.stdout:1773: done (0 left) 2026-02-17T05:56:37.046 INFO:tasks.rados.rados.0.trial115.stdout:1774: read oid 6 snap -1 2026-02-17T05:56:37.046 INFO:tasks.rados.rados.0.trial115.stdout:1774: expect (ObjNum 616 snap 161 seq_num 616) 2026-02-17T05:56:37.046 INFO:tasks.rados.rados.0.trial115.stdout:1775: write oid 3 current snap is 175 2026-02-17T05:56:37.046 INFO:tasks.rados.rados.0.trial115.stdout:1775: seq_num 650 ranges {524550=401855,1403178=450894,2516710=407840} 2026-02-17T05:56:37.050 INFO:tasks.rados.rados.0.trial115.stdout:1775: writing trial11545420-3 from 524550 to 926405 tid 1 2026-02-17T05:56:37.053 INFO:tasks.rados.rados.0.trial115.stdout:1775: writing trial11545420-3 from 1403178 to 1854072 tid 2 2026-02-17T05:56:37.057 INFO:tasks.rados.rados.0.trial115.stdout:1775: writing trial11545420-3 from 2516710 to 2924550 tid 3 2026-02-17T05:56:37.057 INFO:tasks.rados.rados.0.trial115.stdout:1776: snap_create 2026-02-17T05:56:37.057 INFO:tasks.rados.rados.0.trial115.stdout:1774: expect (ObjNum 616 snap 161 seq_num 616) 2026-02-17T05:56:37.121 INFO:tasks.rados.rados.0.trial115.stdout:1775: finishing write tid 1 to trial11545420-3 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: finishing write tid 2 to trial11545420-3 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: finishing write tid 3 to trial11545420-3 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: finishing write tid 4 to trial11545420-3 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: finishing write tid 5 to trial11545420-3 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: oid 3 updating version 0 to 862 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: oid 3 updating version 862 to 863 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: oid 3 updating version 863 to 864 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: oid 3 version 864 is already newer than 861 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 864 (ObjNum 650 snap 175 seq_num 650) dirty exists 2026-02-17T05:56:37.122 INFO:tasks.rados.rados.0.trial115.stdout:1775: left oid 3 (ObjNum 650 snap 175 seq_num 650) 2026-02-17T05:56:38.049 INFO:tasks.rados.rados.0.trial115.stdout:1774: done (2 left) 2026-02-17T05:56:38.049 INFO:tasks.rados.rados.0.trial115.stdout:1775: done (1 left) 2026-02-17T05:56:38.049 INFO:tasks.rados.rados.0.trial115.stdout:1776: done (0 left) 2026-02-17T05:56:38.049 INFO:tasks.rados.rados.0.trial115.stdout:1777: read oid 44 snap -1 2026-02-17T05:56:38.050 INFO:tasks.rados.rados.0.trial115.stdout:1777: expect (ObjNum 649 snap 174 seq_num 649) 2026-02-17T05:56:38.050 INFO:tasks.rados.rados.0.trial115.stdout:1778: read oid 47 snap -1 2026-02-17T05:56:38.050 INFO:tasks.rados.rados.0.trial115.stdout:1778: expect deleted 2026-02-17T05:56:38.050 INFO:tasks.rados.rados.0.trial115.stdout:1779: read oid 3 snap -1 2026-02-17T05:56:38.050 INFO:tasks.rados.rados.0.trial115.stdout:1779: expect (ObjNum 650 snap 175 seq_num 650) 2026-02-17T05:56:38.050 INFO:tasks.rados.rados.0.trial115.stdout:1780: snap_remove snap 168 2026-02-17T05:56:38.546 INFO:tasks.rados.rados.0.trial115.stdout:1780: done (3 left) 2026-02-17T05:56:38.547 INFO:tasks.rados.rados.0.trial115.stdout:1781: read oid 27 snap -1 2026-02-17T05:56:38.547 INFO:tasks.rados.rados.0.trial115.stdout:1781: expect (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:56:38.547 INFO:tasks.rados.rados.0.trial115.stdout:1782: write oid 39 current snap is 176 2026-02-17T05:56:38.547 INFO:tasks.rados.rados.0.trial115.stdout:1782: seq_num 651 ranges {541357=477104,1745307=594552} 2026-02-17T05:56:38.551 INFO:tasks.rados.rados.0.trial115.stdout:1782: writing trial11545420-39 from 541357 to 1018461 tid 1 2026-02-17T05:56:38.556 INFO:tasks.rados.rados.0.trial115.stdout:1782: writing trial11545420-39 from 1745307 to 2339859 tid 2 2026-02-17T05:56:38.556 INFO:tasks.rados.rados.0.trial115.stdout:1783: rollback oid 32 current snap is 176 2026-02-17T05:56:38.556 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 171 2026-02-17T05:56:38.557 INFO:tasks.rados.rados.0.trial115.stdout:1777: expect (ObjNum 649 snap 174 seq_num 649) 2026-02-17T05:56:38.622 INFO:tasks.rados.rados.0.trial115.stdout:1779: expect (ObjNum 650 snap 175 seq_num 650) 2026-02-17T05:56:38.731 INFO:tasks.rados.rados.0.trial115.stdout:1781: expect (ObjNum 434 snap 109 seq_num 434) 2026-02-17T05:56:38.887 INFO:tasks.rados.rados.0.trial115.stdout:1782: finishing write tid 1 to trial11545420-39 2026-02-17T05:56:38.888 INFO:tasks.rados.rados.0.trial115.stdout:1777: done (5 left) 2026-02-17T05:56:38.888 INFO:tasks.rados.rados.0.trial115.stdout:1778: done (4 left) 2026-02-17T05:56:38.888 INFO:tasks.rados.rados.0.trial115.stdout:1779: done (3 left) 2026-02-17T05:56:38.888 INFO:tasks.rados.rados.0.trial115.stdout:1781: done (2 left) 2026-02-17T05:56:38.888 INFO:tasks.rados.rados.0.trial115.stdout:1784: snap_create 2026-02-17T05:56:38.888 INFO:tasks.rados.rados.0.trial115.stdout:1783: finishing rollback tid 0 to trial11545420-32 2026-02-17T05:56:38.888 INFO:tasks.rados.rados.0.trial115.stdout:1783: finishing rollback tid 1 to trial11545420-32 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:1782: finishing write tid 2 to trial11545420-39 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:1783: finishing rollback tid 2 to trial11545420-32 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 863 (ObjNum 572 snap 147 seq_num 572) dirty exists 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:1782: finishing write tid 3 to trial11545420-39 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:1782: finishing write tid 4 to trial11545420-39 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:1782: oid 39 updating version 0 to 1005 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:1782: oid 39 version 1005 is already newer than 1001 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:1782: oid 39 version 1005 is already newer than 1004 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1005 (ObjNum 651 snap 176 seq_num 651) dirty exists 2026-02-17T05:56:38.889 INFO:tasks.rados.rados.0.trial115.stdout:1782: left oid 39 (ObjNum 651 snap 176 seq_num 651) 2026-02-17T05:56:39.977 INFO:tasks.rados.rados.0.trial115.stdout:1782: done (2 left) 2026-02-17T05:56:39.977 INFO:tasks.rados.rados.0.trial115.stdout:1783: done (1 left) 2026-02-17T05:56:39.977 INFO:tasks.rados.rados.0.trial115.stdout:1784: done (0 left) 2026-02-17T05:56:39.977 INFO:tasks.rados.rados.0.trial115.stdout:1785: rollback oid 12 current snap is 177 2026-02-17T05:56:39.977 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 175 2026-02-17T05:56:39.978 INFO:tasks.rados.rados.0.trial115.stdout:1786: write oid 8 current snap is 177 2026-02-17T05:56:39.978 INFO:tasks.rados.rados.0.trial115.stdout:1786: seq_num 652 ranges {558164=552353,1803789=454563,2855203=102961} 2026-02-17T05:56:39.982 INFO:tasks.rados.rados.0.trial115.stdout:1786: writing trial11545420-8 from 558164 to 1110517 tid 1 2026-02-17T05:56:39.986 INFO:tasks.rados.rados.0.trial115.stdout:1786: writing trial11545420-8 from 1803789 to 2258352 tid 2 2026-02-17T05:56:39.989 INFO:tasks.rados.rados.0.trial115.stdout:1786: writing trial11545420-8 from 2855203 to 2958164 tid 3 2026-02-17T05:56:39.990 INFO:tasks.rados.rados.0.trial115.stdout:1787: delete oid 27 current snap is 177 2026-02-17T05:56:39.990 INFO:tasks.rados.rados.0.trial115.stdout:1785: finishing rollback tid 0 to trial11545420-12 2026-02-17T05:56:39.990 INFO:tasks.rados.rados.0.trial115.stdout:1785: finishing rollback tid 1 to trial11545420-12 2026-02-17T05:56:39.990 INFO:tasks.rados.rados.0.trial115.stdout:1785: finishing rollback tid 2 to trial11545420-12 2026-02-17T05:56:39.990 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1009 (ObjNum 629 snap 164 seq_num 629) dirty exists 2026-02-17T05:56:39.990 INFO:tasks.rados.rados.0.trial115.stdout:1786: finishing write tid 1 to trial11545420-8 2026-02-17T05:56:39.990 INFO:tasks.rados.rados.0.trial115.stdout:1786: finishing write tid 2 to trial11545420-8 2026-02-17T05:56:39.991 INFO:tasks.rados.rados.0.trial115.stdout:1786: finishing write tid 3 to trial11545420-8 2026-02-17T05:56:39.991 INFO:tasks.rados.rados.0.trial115.stdout:1786: finishing write tid 4 to trial11545420-8 2026-02-17T05:56:39.991 INFO:tasks.rados.rados.0.trial115.stdout:1786: finishing write tid 5 to trial11545420-8 2026-02-17T05:56:39.991 INFO:tasks.rados.rados.0.trial115.stdout:1786: oid 8 updating version 0 to 211 2026-02-17T05:56:39.991 INFO:tasks.rados.rados.0.trial115.stdout:1786: oid 8 version 211 is already newer than 209 2026-02-17T05:56:39.991 INFO:tasks.rados.rados.0.trial115.stdout:1786: oid 8 version 211 is already newer than 210 2026-02-17T05:56:39.992 INFO:tasks.rados.rados.0.trial115.stdout:1786: oid 8 version 211 is already newer than 208 2026-02-17T05:56:39.992 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 211 (ObjNum 652 snap 177 seq_num 652) dirty exists 2026-02-17T05:56:39.992 INFO:tasks.rados.rados.0.trial115.stdout:1786: left oid 8 (ObjNum 652 snap 177 seq_num 652) 2026-02-17T05:56:39.996 INFO:tasks.rados.rados.0.trial115.stdout:1785: done (2 left) 2026-02-17T05:56:39.996 INFO:tasks.rados.rados.0.trial115.stdout:1786: done (1 left) 2026-02-17T05:56:39.997 INFO:tasks.rados.rados.0.trial115.stdout:1787: done (0 left) 2026-02-17T05:56:39.997 INFO:tasks.rados.rados.0.trial115.stdout:1788: write oid 35 current snap is 177 2026-02-17T05:56:39.997 INFO:tasks.rados.rados.0.trial115.stdout:1788: seq_num 654 ranges {591778=419204,1637106=741879,2910049=81729} 2026-02-17T05:56:40.000 INFO:tasks.rados.rados.0.trial115.stdout:1788: writing trial11545420-35 from 591778 to 1010982 tid 1 2026-02-17T05:56:40.005 INFO:tasks.rados.rados.0.trial115.stdout:1788: writing trial11545420-35 from 1637106 to 2378985 tid 2 2026-02-17T05:56:40.008 INFO:tasks.rados.rados.0.trial115.stdout:1788: writing trial11545420-35 from 2910049 to 2991778 tid 3 2026-02-17T05:56:40.008 INFO:tasks.rados.rados.0.trial115.stdout:1789: delete oid 37 current snap is 177 2026-02-17T05:56:40.008 INFO:tasks.rados.rados.0.trial115.stdout:1788: finishing write tid 1 to trial11545420-35 2026-02-17T05:56:40.008 INFO:tasks.rados.rados.0.trial115.stdout:1788: finishing write tid 2 to trial11545420-35 2026-02-17T05:56:40.009 INFO:tasks.rados.rados.0.trial115.stdout:1788: finishing write tid 3 to trial11545420-35 2026-02-17T05:56:40.009 INFO:tasks.rados.rados.0.trial115.stdout:1788: finishing write tid 4 to trial11545420-35 2026-02-17T05:56:40.009 INFO:tasks.rados.rados.0.trial115.stdout:1788: finishing write tid 5 to trial11545420-35 2026-02-17T05:56:40.009 INFO:tasks.rados.rados.0.trial115.stdout:1788: oid 35 updating version 0 to 951 2026-02-17T05:56:40.010 INFO:tasks.rados.rados.0.trial115.stdout:1788: oid 35 version 951 is already newer than 948 2026-02-17T05:56:40.010 INFO:tasks.rados.rados.0.trial115.stdout:1788: oid 35 version 951 is already newer than 950 2026-02-17T05:56:40.010 INFO:tasks.rados.rados.0.trial115.stdout:1788: oid 35 version 951 is already newer than 949 2026-02-17T05:56:40.010 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 951 (ObjNum 654 snap 177 seq_num 654) dirty exists 2026-02-17T05:56:40.010 INFO:tasks.rados.rados.0.trial115.stdout:1788: left oid 35 (ObjNum 654 snap 177 seq_num 654) 2026-02-17T05:56:40.010 INFO:tasks.rados.rados.0.trial115.stdout:1788: done (1 left) 2026-02-17T05:56:40.010 INFO:tasks.rados.rados.0.trial115.stdout:1789: done (0 left) 2026-02-17T05:56:40.010 INFO:tasks.rados.rados.0.trial115.stdout:1790: snap_create 2026-02-17T05:56:40.552 INFO:tasks.rados.rados.0.trial115.stdout:1790: done (0 left) 2026-02-17T05:56:40.552 INFO:tasks.rados.rados.0.trial115.stdout:1791: read oid 9 snap 139 2026-02-17T05:56:40.553 INFO:tasks.rados.rados.0.trial115.stdout:1791: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:56:40.553 INFO:tasks.rados.rados.0.trial115.stdout:1792: read oid 34 snap 174 2026-02-17T05:56:40.553 INFO:tasks.rados.rados.0.trial115.stdout:1792: expect deleted 2026-02-17T05:56:40.553 INFO:tasks.rados.rados.0.trial115.stdout:1793: delete oid 3 current snap is 178 2026-02-17T05:56:40.555 INFO:tasks.rados.rados.0.trial115.stdout:1792: done (2 left) 2026-02-17T05:56:40.555 INFO:tasks.rados.rados.0.trial115.stdout:1793: done (1 left) 2026-02-17T05:56:40.555 INFO:tasks.rados.rados.0.trial115.stdout:1794: copy_from oid 42 from oid 29 current snap is 178 2026-02-17T05:56:40.555 INFO:tasks.rados.rados.0.trial115.stdout:1795: read oid 26 snap -1 2026-02-17T05:56:40.555 INFO:tasks.rados.rados.0.trial115.stdout:1795: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:40.556 INFO:tasks.rados.rados.0.trial115.stdout:1796: snap_remove snap 176 2026-02-17T05:56:41.554 INFO:tasks.rados.rados.0.trial115.stdout:1796: done (3 left) 2026-02-17T05:56:41.554 INFO:tasks.rados.rados.0.trial115.stdout:1797: read oid 23 snap -1 2026-02-17T05:56:41.554 INFO:tasks.rados.rados.0.trial115.stdout:1797: expect (ObjNum 647 snap 173 seq_num 647) 2026-02-17T05:56:41.554 INFO:tasks.rados.rados.0.trial115.stdout:1798: copy_from oid 32 from oid 46 current snap is 178 2026-02-17T05:56:41.554 INFO:tasks.rados.rados.0.trial115.stdout:1791: expect (ObjNum 377 snap 99 seq_num 377) 2026-02-17T05:56:41.634 INFO:tasks.rados.rados.0.trial115.stdout:1794: finishing copy_from to trial11545420-42 2026-02-17T05:56:41.635 INFO:tasks.rados.rados.0.trial115.stdout:1794: got expected ENOENT (src dne) 2026-02-17T05:56:41.635 INFO:tasks.rados.rados.0.trial115.stdout:1794: finishing copy_from racing read to trial11545420-42 2026-02-17T05:56:41.635 INFO:tasks.rados.rados.0.trial115.stdout:1795: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:41.711 INFO:tasks.rados.rados.0.trial115.stdout:1797: expect (ObjNum 647 snap 173 seq_num 647) 2026-02-17T05:56:41.805 INFO:tasks.rados.rados.0.trial115.stdout:1798: finishing copy_from racing read to trial11545420-32 2026-02-17T05:56:41.805 INFO:tasks.rados.rados.0.trial115.stdout:1798: finishing copy_from to trial11545420-32 2026-02-17T05:56:41.805 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 840 (ObjNum 642 snap 170 seq_num 642) dirty exists 2026-02-17T05:56:41.805 INFO:tasks.rados.rados.0.trial115.stdout:1791: done (4 left) 2026-02-17T05:56:41.805 INFO:tasks.rados.rados.0.trial115.stdout:1794: done (3 left) 2026-02-17T05:56:41.806 INFO:tasks.rados.rados.0.trial115.stdout:1795: done (2 left) 2026-02-17T05:56:41.806 INFO:tasks.rados.rados.0.trial115.stdout:1797: done (1 left) 2026-02-17T05:56:41.806 INFO:tasks.rados.rados.0.trial115.stdout:1798: done (0 left) 2026-02-17T05:56:41.806 INFO:tasks.rados.rados.0.trial115.stdout:1799: copy_from oid 47 from oid 38 current snap is 178 2026-02-17T05:56:41.806 INFO:tasks.rados.rados.0.trial115.stdout:1800: read oid 41 snap 163 2026-02-17T05:56:41.806 INFO:tasks.rados.rados.0.trial115.stdout:1800: expect deleted 2026-02-17T05:56:41.806 INFO:tasks.rados.rados.0.trial115.stdout:1801: write (excl) oid 6 current snap is 178 2026-02-17T05:56:41.806 INFO:tasks.rados.rados.0.trial115.stdout:1801: seq_num 657 ranges {642199=644951,1696199=605559,2992318=49881} 2026-02-17T05:56:41.808 INFO:tasks.rados.rados.0.trial115.stdout:1801: writing trial11545420-6 from 642199 to 1287150 tid 1 2026-02-17T05:56:41.812 INFO:tasks.rados.rados.0.trial115.stdout:1801: writing trial11545420-6 from 1696199 to 2301758 tid 2 2026-02-17T05:56:41.815 INFO:tasks.rados.rados.0.trial115.stdout:1801: writing trial11545420-6 from 2992318 to 3042199 tid 3 2026-02-17T05:56:41.815 INFO:tasks.rados.rados.0.trial115.stdout:1802: snap_create 2026-02-17T05:56:41.816 INFO:tasks.rados.rados.0.trial115.stdout:1799: finishing copy_from racing read to trial11545420-47 2026-02-17T05:56:41.816 INFO:tasks.rados.rados.0.trial115.stdout:1801: finishing write tid 1 to trial11545420-6 2026-02-17T05:56:41.816 INFO:tasks.rados.rados.0.trial115.stdout:1799: finishing copy_from to trial11545420-47 2026-02-17T05:56:41.816 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 603 (ObjNum 598 snap 155 seq_num 598) dirty exists 2026-02-17T05:56:41.816 INFO:tasks.rados.rados.0.trial115.stdout:1801: finishing write tid 2 to trial11545420-6 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:1801: finishing write tid 3 to trial11545420-6 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:1801: finishing write tid 4 to trial11545420-6 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:1801: finishing write tid 5 to trial11545420-6 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:1801: oid 6 updating version 0 to 650 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:1801: oid 6 version 650 is already newer than 649 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:1801: oid 6 version 650 is already newer than 648 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:1801: oid 6 updating version 650 to 651 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 651 (ObjNum 657 snap 178 seq_num 657) dirty exists 2026-02-17T05:56:41.817 INFO:tasks.rados.rados.0.trial115.stdout:1801: left oid 6 (ObjNum 657 snap 178 seq_num 657) 2026-02-17T05:56:42.559 INFO:tasks.rados.rados.0.trial115.stdout:1799: done (3 left) 2026-02-17T05:56:42.559 INFO:tasks.rados.rados.0.trial115.stdout:1800: done (2 left) 2026-02-17T05:56:42.560 INFO:tasks.rados.rados.0.trial115.stdout:1801: done (1 left) 2026-02-17T05:56:42.560 INFO:tasks.rados.rados.0.trial115.stdout:1802: done (0 left) 2026-02-17T05:56:42.560 INFO:tasks.rados.rados.0.trial115.stdout:1803: snap_create 2026-02-17T05:56:43.560 INFO:tasks.rados.rados.0.trial115.stdout:1803: done (0 left) 2026-02-17T05:56:43.560 INFO:tasks.rados.rados.0.trial115.stdout:1804: snap_create 2026-02-17T05:56:44.562 INFO:tasks.rados.rados.0.trial115.stdout:1804: done (0 left) 2026-02-17T05:56:44.562 INFO:tasks.rados.rados.0.trial115.stdout:1805: write (excl) oid 41 current snap is 181 2026-02-17T05:56:44.562 INFO:tasks.rados.rados.0.trial115.stdout:1805: seq_num 658 ranges {659006=720200,2154681=749217} 2026-02-17T05:56:44.568 INFO:tasks.rados.rados.0.trial115.stdout:1805: writing trial11545420-41 from 659006 to 1379206 tid 1 2026-02-17T05:56:44.574 INFO:tasks.rados.rados.0.trial115.stdout:1805: writing trial11545420-41 from 2154681 to 2903898 tid 2 2026-02-17T05:56:44.574 INFO:tasks.rados.rados.0.trial115.stdout:1806: rollback oid 2 current snap is 181 2026-02-17T05:56:44.574 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 2 to 179 2026-02-17T05:56:44.574 INFO:tasks.rados.rados.0.trial115.stdout:1805: finishing write tid 1 to trial11545420-41 2026-02-17T05:56:44.574 INFO:tasks.rados.rados.0.trial115.stdout:1807: copy_from oid 19 from oid 5 current snap is 181 2026-02-17T05:56:44.574 INFO:tasks.rados.rados.0.trial115.stdout:1808: snap_create 2026-02-17T05:56:44.575 INFO:tasks.rados.rados.0.trial115.stdout:1805: finishing write tid 2 to trial11545420-41 2026-02-17T05:56:44.576 INFO:tasks.rados.rados.0.trial115.stdout:1805: finishing write tid 3 to trial11545420-41 2026-02-17T05:56:44.576 INFO:tasks.rados.rados.0.trial115.stdout:1805: finishing write tid 4 to trial11545420-41 2026-02-17T05:56:44.576 INFO:tasks.rados.rados.0.trial115.stdout:1805: oid 41 updating version 0 to 860 2026-02-17T05:56:44.576 INFO:tasks.rados.rados.0.trial115.stdout:1805: oid 41 version 860 is already newer than 859 2026-02-17T05:56:44.576 INFO:tasks.rados.rados.0.trial115.stdout:1805: oid 41 updating version 860 to 861 2026-02-17T05:56:44.577 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 861 (ObjNum 658 snap 181 seq_num 658) dirty exists 2026-02-17T05:56:44.577 INFO:tasks.rados.rados.0.trial115.stdout:1805: left oid 41 (ObjNum 658 snap 181 seq_num 658) 2026-02-17T05:56:44.585 INFO:tasks.rados.rados.0.trial115.stdout:1806: finishing rollback tid 1 to trial11545420-2 2026-02-17T05:56:44.586 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 0 (ObjNum 33 snap 0 seq_num 1330673496) dirty dne 2026-02-17T05:56:44.592 INFO:tasks.rados.rados.0.trial115.stdout:1807: finishing copy_from racing read to trial11545420-19 2026-02-17T05:56:44.593 INFO:tasks.rados.rados.0.trial115.stdout:1807: finishing copy_from to trial11545420-19 2026-02-17T05:56:44.593 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 811 (ObjNum 516 snap 132 seq_num 516) dirty exists 2026-02-17T05:56:45.568 INFO:tasks.rados.rados.0.trial115.stdout:1805: done (3 left) 2026-02-17T05:56:45.568 INFO:tasks.rados.rados.0.trial115.stdout:1806: done (2 left) 2026-02-17T05:56:45.568 INFO:tasks.rados.rados.0.trial115.stdout:1807: done (1 left) 2026-02-17T05:56:45.568 INFO:tasks.rados.rados.0.trial115.stdout:1808: done (0 left) 2026-02-17T05:56:45.568 INFO:tasks.rados.rados.0.trial115.stdout:1809: read oid 38 snap -1 2026-02-17T05:56:45.568 INFO:tasks.rados.rados.0.trial115.stdout:1809: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:56:45.568 INFO:tasks.rados.rados.0.trial115.stdout:1810: write oid 19 current snap is 182 2026-02-17T05:56:45.569 INFO:tasks.rados.rados.0.trial115.stdout:1810: seq_num 659 ranges {675813=795449,2096810=609228} 2026-02-17T05:56:45.573 INFO:tasks.rados.rados.0.trial115.stdout:1810: writing trial11545420-19 from 675813 to 1471262 tid 1 2026-02-17T05:56:45.578 INFO:tasks.rados.rados.0.trial115.stdout:1810: writing trial11545420-19 from 2096810 to 2706038 tid 2 2026-02-17T05:56:45.578 INFO:tasks.rados.rados.0.trial115.stdout:1811: write oid 28 current snap is 182 2026-02-17T05:56:45.578 INFO:tasks.rados.rados.0.trial115.stdout:1811: seq_num 660 ranges {692620=470698,1755292=752886} 2026-02-17T05:56:45.582 INFO:tasks.rados.rados.0.trial115.stdout:1811: writing trial11545420-28 from 692620 to 1163318 tid 1 2026-02-17T05:56:45.588 INFO:tasks.rados.rados.0.trial115.stdout:1811: writing trial11545420-28 from 1755292 to 2508178 tid 2 2026-02-17T05:56:45.588 INFO:tasks.rados.rados.0.trial115.stdout:1812: rollback oid 45 current snap is 182 2026-02-17T05:56:45.588 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 139 2026-02-17T05:56:45.588 INFO:tasks.rados.rados.0.trial115.stdout:1809: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:56:45.655 INFO:tasks.rados.rados.0.trial115.stdout:1810: finishing write tid 1 to trial11545420-19 2026-02-17T05:56:45.655 INFO:tasks.rados.rados.0.trial115.stdout:1810: finishing write tid 2 to trial11545420-19 2026-02-17T05:56:45.655 INFO:tasks.rados.rados.0.trial115.stdout:1810: finishing write tid 3 to trial11545420-19 2026-02-17T05:56:45.655 INFO:tasks.rados.rados.0.trial115.stdout:1810: finishing write tid 4 to trial11545420-19 2026-02-17T05:56:45.655 INFO:tasks.rados.rados.0.trial115.stdout:1810: oid 19 updating version 0 to 959 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1810: oid 19 version 959 is already newer than 958 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1810: oid 19 version 959 is already newer than 957 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 959 (ObjNum 659 snap 182 seq_num 659) dirty exists 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1810: left oid 19 (ObjNum 659 snap 182 seq_num 659) 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1811: finishing write tid 1 to trial11545420-28 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1811: finishing write tid 2 to trial11545420-28 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1809: done (3 left) 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1810: done (2 left) 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1813: read oid 31 snap 170 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1813: expect deleted 2026-02-17T05:56:45.656 INFO:tasks.rados.rados.0.trial115.stdout:1814: snap_remove snap 171 2026-02-17T05:56:46.571 INFO:tasks.rados.rados.0.trial115.stdout:1814: done (3 left) 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:1815: rollback oid 14 current snap is 182 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 163 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:1811: finishing write tid 3 to trial11545420-28 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:1811: finishing write tid 4 to trial11545420-28 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:1811: oid 28 updating version 0 to 862 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:1811: oid 28 updating version 862 to 863 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:1811: oid 28 version 863 is already newer than 861 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 863 (ObjNum 660 snap 182 seq_num 660) dirty exists 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:1811: left oid 28 (ObjNum 660 snap 182 seq_num 660) 2026-02-17T05:56:46.572 INFO:tasks.rados.rados.0.trial115.stdout:1812: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:56:46.573 INFO:tasks.rados.rados.0.trial115.stdout:1812: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:56:46.573 INFO:tasks.rados.rados.0.trial115.stdout:1812: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:56:46.573 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1015 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:56:46.573 INFO:tasks.rados.rados.0.trial115.stdout:1811: done (3 left) 2026-02-17T05:56:46.573 INFO:tasks.rados.rados.0.trial115.stdout:1812: done (2 left) 2026-02-17T05:56:46.573 INFO:tasks.rados.rados.0.trial115.stdout:1813: done (1 left) 2026-02-17T05:56:46.573 INFO:tasks.rados.rados.0.trial115.stdout:1816: copy_from oid 39 from oid 9 current snap is 182 2026-02-17T05:56:46.573 INFO:tasks.rados.rados.0.trial115.stdout:1817: delete oid 31 current snap is 182 2026-02-17T05:56:46.574 INFO:tasks.rados.rados.0.trial115.stdout:1815: finishing rollback tid 1 to trial11545420-14 2026-02-17T05:56:46.574 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 0 (ObjNum 33 snap 0 seq_num 1331149504) dirty dne 2026-02-17T05:56:46.579 INFO:tasks.rados.rados.0.trial115.stdout:1816: finishing copy_from racing read to trial11545420-39 2026-02-17T05:56:46.579 INFO:tasks.rados.rados.0.trial115.stdout:1816: finishing copy_from to trial11545420-39 2026-02-17T05:56:46.579 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 852 (ObjNum 646 snap 173 seq_num 646) dirty exists 2026-02-17T05:56:46.580 INFO:tasks.rados.rados.0.trial115.stdout:1815: done (2 left) 2026-02-17T05:56:46.580 INFO:tasks.rados.rados.0.trial115.stdout:1816: done (1 left) 2026-02-17T05:56:46.580 INFO:tasks.rados.rados.0.trial115.stdout:1817: done (0 left) 2026-02-17T05:56:46.580 INFO:tasks.rados.rados.0.trial115.stdout:1818: write oid 40 current snap is 182 2026-02-17T05:56:46.580 INFO:tasks.rados.rados.0.trial115.stdout:1818: seq_num 662 ranges {726234=737549,1988609=756555} 2026-02-17T05:56:46.586 INFO:tasks.rados.rados.0.trial115.stdout:1818: writing trial11545420-40 from 726234 to 1463783 tid 1 2026-02-17T05:56:46.592 INFO:tasks.rados.rados.0.trial115.stdout:1818: writing trial11545420-40 from 1988609 to 2745164 tid 2 2026-02-17T05:56:46.592 INFO:tasks.rados.rados.0.trial115.stdout:1819: copy_from oid 49 from oid 36 current snap is 182 2026-02-17T05:56:46.592 INFO:tasks.rados.rados.0.trial115.stdout:1818: finishing write tid 1 to trial11545420-40 2026-02-17T05:56:46.592 INFO:tasks.rados.rados.0.trial115.stdout:1820: snap_create 2026-02-17T05:56:46.594 INFO:tasks.rados.rados.0.trial115.stdout:1819: finishing copy_from to trial11545420-49 2026-02-17T05:56:46.594 INFO:tasks.rados.rados.0.trial115.stdout:1819: got expected ENOENT (src dne) 2026-02-17T05:56:46.594 INFO:tasks.rados.rados.0.trial115.stdout:1819: finishing copy_from racing read to trial11545420-49 2026-02-17T05:56:46.595 INFO:tasks.rados.rados.0.trial115.stdout:1818: finishing write tid 2 to trial11545420-40 2026-02-17T05:56:46.595 INFO:tasks.rados.rados.0.trial115.stdout:1818: finishing write tid 3 to trial11545420-40 2026-02-17T05:56:46.595 INFO:tasks.rados.rados.0.trial115.stdout:1818: finishing write tid 4 to trial11545420-40 2026-02-17T05:56:46.595 INFO:tasks.rados.rados.0.trial115.stdout:1818: oid 40 updating version 0 to 814 2026-02-17T05:56:46.595 INFO:tasks.rados.rados.0.trial115.stdout:1818: oid 40 updating version 814 to 815 2026-02-17T05:56:46.595 INFO:tasks.rados.rados.0.trial115.stdout:1818: oid 40 updating version 815 to 816 2026-02-17T05:56:46.595 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 816 (ObjNum 662 snap 182 seq_num 662) dirty exists 2026-02-17T05:56:46.595 INFO:tasks.rados.rados.0.trial115.stdout:1818: left oid 40 (ObjNum 662 snap 182 seq_num 662) 2026-02-17T05:56:47.573 INFO:tasks.rados.rados.0.trial115.stdout:1818: done (2 left) 2026-02-17T05:56:47.574 INFO:tasks.rados.rados.0.trial115.stdout:1819: done (1 left) 2026-02-17T05:56:47.574 INFO:tasks.rados.rados.0.trial115.stdout:1820: done (0 left) 2026-02-17T05:56:47.574 INFO:tasks.rados.rados.0.trial115.stdout:1821: write (excl) oid 34 current snap is 183 2026-02-17T05:56:47.574 INFO:tasks.rados.rados.0.trial115.stdout:1821: seq_num 663 ranges {743041=412798,1930738=616566,3040503=102538} 2026-02-17T05:56:47.578 INFO:tasks.rados.rados.0.trial115.stdout:1821: writing trial11545420-34 from 743041 to 1155839 tid 1 2026-02-17T05:56:47.584 INFO:tasks.rados.rados.0.trial115.stdout:1821: writing trial11545420-34 from 1930738 to 2547304 tid 2 2026-02-17T05:56:47.587 INFO:tasks.rados.rados.0.trial115.stdout:1821: writing trial11545420-34 from 3040503 to 3143041 tid 3 2026-02-17T05:56:47.587 INFO:tasks.rados.rados.0.trial115.stdout:1822: read oid 42 snap -1 2026-02-17T05:56:47.587 INFO:tasks.rados.rados.0.trial115.stdout:1822: expect (ObjNum 641 snap 169 seq_num 641) 2026-02-17T05:56:47.587 INFO:tasks.rados.rados.0.trial115.stdout:1823: rollback oid 50 current snap is 183 2026-02-17T05:56:47.588 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 50 to 165 2026-02-17T05:56:47.588 INFO:tasks.rados.rados.0.trial115.stdout:1821: finishing write tid 1 to trial11545420-34 2026-02-17T05:56:47.588 INFO:tasks.rados.rados.0.trial115.stdout:1821: finishing write tid 2 to trial11545420-34 2026-02-17T05:56:47.588 INFO:tasks.rados.rados.0.trial115.stdout:1824: read oid 44 snap -1 2026-02-17T05:56:47.588 INFO:tasks.rados.rados.0.trial115.stdout:1824: expect (ObjNum 649 snap 174 seq_num 649) 2026-02-17T05:56:47.588 INFO:tasks.rados.rados.0.trial115.stdout:1825: write (excl) oid 1 current snap is 183 2026-02-17T05:56:47.588 INFO:tasks.rados.rados.0.trial115.stdout:1825: seq_num 664 ranges {759848=488047,1989220=760224} 2026-02-17T05:56:47.591 INFO:tasks.rados.rados.0.trial115.stdout:1825: writing trial11545420-1 from 759848 to 1247895 tid 1 2026-02-17T05:56:47.597 INFO:tasks.rados.rados.0.trial115.stdout:1825: writing trial11545420-1 from 1989220 to 2749444 tid 2 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1826: read oid 31 snap -1 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1826: expect deleted 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1827: snap_create 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1821: finishing write tid 3 to trial11545420-34 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1821: finishing write tid 4 to trial11545420-34 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1821: finishing write tid 5 to trial11545420-34 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1821: oid 34 updating version 0 to 653 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1821: oid 34 version 653 is already newer than 652 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1821: oid 34 updating version 653 to 654 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1821: oid 34 updating version 654 to 655 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 655 (ObjNum 663 snap 183 seq_num 663) dirty exists 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1821: left oid 34 (ObjNum 663 snap 183 seq_num 663) 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1823: finishing rollback tid 0 to trial11545420-50 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1823: finishing rollback tid 1 to trial11545420-50 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1823: finishing rollback tid 2 to trial11545420-50 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 749 (ObjNum 587 snap 151 seq_num 587) dirty exists 2026-02-17T05:56:47.598 INFO:tasks.rados.rados.0.trial115.stdout:1824: expect (ObjNum 649 snap 174 seq_num 649) 2026-02-17T05:56:47.664 INFO:tasks.rados.rados.0.trial115.stdout:1822: expect (ObjNum 641 snap 169 seq_num 641) 2026-02-17T05:56:47.793 INFO:tasks.rados.rados.0.trial115.stdout:1825: finishing write tid 1 to trial11545420-1 2026-02-17T05:56:47.793 INFO:tasks.rados.rados.0.trial115.stdout:1825: finishing write tid 2 to trial11545420-1 2026-02-17T05:56:47.794 INFO:tasks.rados.rados.0.trial115.stdout:1825: finishing write tid 3 to trial11545420-1 2026-02-17T05:56:47.794 INFO:tasks.rados.rados.0.trial115.stdout:1825: finishing write tid 4 to trial11545420-1 2026-02-17T05:56:47.794 INFO:tasks.rados.rados.0.trial115.stdout:1825: oid 1 updating version 0 to 644 2026-02-17T05:56:47.794 INFO:tasks.rados.rados.0.trial115.stdout:1825: oid 1 updating version 644 to 645 2026-02-17T05:56:47.794 INFO:tasks.rados.rados.0.trial115.stdout:1825: oid 1 updating version 645 to 646 2026-02-17T05:56:47.794 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 646 (ObjNum 664 snap 183 seq_num 664) dirty exists 2026-02-17T05:56:47.794 INFO:tasks.rados.rados.0.trial115.stdout:1825: left oid 1 (ObjNum 664 snap 183 seq_num 664) 2026-02-17T05:56:48.549 INFO:tasks.rados.rados.0.trial115.stdout:1821: done (6 left) 2026-02-17T05:56:48.549 INFO:tasks.rados.rados.0.trial115.stdout:1822: done (5 left) 2026-02-17T05:56:48.549 INFO:tasks.rados.rados.0.trial115.stdout:1823: done (4 left) 2026-02-17T05:56:48.549 INFO:tasks.rados.rados.0.trial115.stdout:1824: done (3 left) 2026-02-17T05:56:48.550 INFO:tasks.rados.rados.0.trial115.stdout:1825: done (2 left) 2026-02-17T05:56:48.550 INFO:tasks.rados.rados.0.trial115.stdout:1826: done (1 left) 2026-02-17T05:56:48.550 INFO:tasks.rados.rados.0.trial115.stdout:1827: done (0 left) 2026-02-17T05:56:48.550 INFO:tasks.rados.rados.0.trial115.stdout:1828: write oid 27 current snap is 184 2026-02-17T05:56:48.550 INFO:tasks.rados.rados.0.trial115.stdout:1828: seq_num 665 ranges {776655=563296,2047702=620235,3095349=81306} 2026-02-17T05:56:48.553 INFO:tasks.rados.rados.0.trial115.stdout:1828: writing trial11545420-27 from 776655 to 1339951 tid 1 2026-02-17T05:56:48.558 INFO:tasks.rados.rados.0.trial115.stdout:1828: writing trial11545420-27 from 2047702 to 2667937 tid 2 2026-02-17T05:56:48.560 INFO:tasks.rados.rados.0.trial115.stdout:1828: writing trial11545420-27 from 3095349 to 3176655 tid 3 2026-02-17T05:56:48.561 INFO:tasks.rados.rados.0.trial115.stdout:1829: rollback oid 46 current snap is 184 2026-02-17T05:56:48.561 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 46 to 183 2026-02-17T05:56:48.561 INFO:tasks.rados.rados.0.trial115.stdout:1828: finishing write tid 1 to trial11545420-27 2026-02-17T05:56:48.561 INFO:tasks.rados.rados.0.trial115.stdout:1830: write oid 19 current snap is 184 2026-02-17T05:56:48.561 INFO:tasks.rados.rados.0.trial115.stdout:1830: seq_num 666 ranges {793462=638545,2106184=763893} 2026-02-17T05:56:48.566 INFO:tasks.rados.rados.0.trial115.stdout:1830: writing trial11545420-19 from 793462 to 1432007 tid 1 2026-02-17T05:56:48.572 INFO:tasks.rados.rados.0.trial115.stdout:1830: writing trial11545420-19 from 2106184 to 2870077 tid 2 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1831: read oid 13 snap -1 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1831: expect (ObjNum 648 snap 173 seq_num 648) 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1832: snap_create 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1828: finishing write tid 2 to trial11545420-27 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1828: finishing write tid 3 to trial11545420-27 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1828: finishing write tid 4 to trial11545420-27 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1828: finishing write tid 5 to trial11545420-27 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1828: oid 27 updating version 0 to 751 2026-02-17T05:56:48.573 INFO:tasks.rados.rados.0.trial115.stdout:1828: oid 27 updating version 751 to 752 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:1828: oid 27 updating version 752 to 753 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:1828: oid 27 version 753 is already newer than 750 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 753 (ObjNum 665 snap 184 seq_num 665) dirty exists 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:1828: left oid 27 (ObjNum 665 snap 184 seq_num 665) 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:1829: finishing rollback tid 0 to trial11545420-46 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:1829: finishing rollback tid 1 to trial11545420-46 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:1829: finishing rollback tid 2 to trial11545420-46 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 869 (ObjNum 642 snap 170 seq_num 642) dirty exists 2026-02-17T05:56:48.574 INFO:tasks.rados.rados.0.trial115.stdout:1830: finishing write tid 1 to trial11545420-19 2026-02-17T05:56:48.575 INFO:tasks.rados.rados.0.trial115.stdout:1830: finishing write tid 2 to trial11545420-19 2026-02-17T05:56:48.575 INFO:tasks.rados.rados.0.trial115.stdout:1830: finishing write tid 3 to trial11545420-19 2026-02-17T05:56:48.575 INFO:tasks.rados.rados.0.trial115.stdout:1830: finishing write tid 4 to trial11545420-19 2026-02-17T05:56:48.575 INFO:tasks.rados.rados.0.trial115.stdout:1830: oid 19 updating version 0 to 967 2026-02-17T05:56:48.576 INFO:tasks.rados.rados.0.trial115.stdout:1830: oid 19 updating version 967 to 968 2026-02-17T05:56:48.576 INFO:tasks.rados.rados.0.trial115.stdout:1830: oid 19 updating version 968 to 969 2026-02-17T05:56:48.576 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 969 (ObjNum 666 snap 184 seq_num 666) dirty exists 2026-02-17T05:56:48.576 INFO:tasks.rados.rados.0.trial115.stdout:1830: left oid 19 (ObjNum 666 snap 184 seq_num 666) 2026-02-17T05:56:48.577 INFO:tasks.rados.rados.0.trial115.stdout:1831: expect (ObjNum 648 snap 173 seq_num 648) 2026-02-17T05:56:49.549 INFO:tasks.rados.rados.0.trial115.stdout:1828: done (4 left) 2026-02-17T05:56:49.549 INFO:tasks.rados.rados.0.trial115.stdout:1829: done (3 left) 2026-02-17T05:56:49.549 INFO:tasks.rados.rados.0.trial115.stdout:1830: done (2 left) 2026-02-17T05:56:49.550 INFO:tasks.rados.rados.0.trial115.stdout:1831: done (1 left) 2026-02-17T05:56:49.550 INFO:tasks.rados.rados.0.trial115.stdout:1832: done (0 left) 2026-02-17T05:56:49.550 INFO:tasks.rados.rados.0.trial115.stdout:1833: delete oid 44 current snap is 185 2026-02-17T05:56:49.552 INFO:tasks.rados.rados.0.trial115.stdout:1833: done (0 left) 2026-02-17T05:56:49.552 INFO:tasks.rados.rados.0.trial115.stdout:1834: snap_remove snap 157 2026-02-17T05:56:50.552 INFO:tasks.rados.rados.0.trial115.stdout:1834: done (0 left) 2026-02-17T05:56:50.552 INFO:tasks.rados.rados.0.trial115.stdout:1835: snap_remove snap 181 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1835: done (0 left) 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1836: read oid 46 snap -1 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1836: expect (ObjNum 642 snap 170 seq_num 642) 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1837: read oid 3 snap -1 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1837: expect deleted 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1838: read oid 12 snap -1 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1838: expect (ObjNum 629 snap 164 seq_num 629) 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1839: copy_from oid 44 from oid 11 current snap is 185 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:1840: rollback oid 27 current snap is 185 2026-02-17T05:56:51.553 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 163 2026-02-17T05:56:51.554 INFO:tasks.rados.rados.0.trial115.stdout:1841: read oid 26 snap -1 2026-02-17T05:56:51.554 INFO:tasks.rados.rados.0.trial115.stdout:1841: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:51.554 INFO:tasks.rados.rados.0.trial115.stdout:1842: delete oid 43 current snap is 185 2026-02-17T05:56:51.556 INFO:tasks.rados.rados.0.trial115.stdout:1840: finishing rollback tid 0 to trial11545420-27 2026-02-17T05:56:51.556 INFO:tasks.rados.rados.0.trial115.stdout:1837: done (6 left) 2026-02-17T05:56:51.556 INFO:tasks.rados.rados.0.trial115.stdout:1842: done (5 left) 2026-02-17T05:56:51.556 INFO:tasks.rados.rados.0.trial115.stdout:1843: read oid 48 snap 175 2026-02-17T05:56:51.556 INFO:tasks.rados.rados.0.trial115.stdout:1843: expect deleted 2026-02-17T05:56:51.556 INFO:tasks.rados.rados.0.trial115.stdout:1844: copy_from oid 2 from oid 9 current snap is 185 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1840: finishing rollback tid 1 to trial11545420-27 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1845: read oid 35 snap -1 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1845: expect (ObjNum 654 snap 177 seq_num 654) 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1846: read oid 20 snap 165 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1846: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1847: read oid 16 snap -1 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1847: expect deleted 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1848: read oid 24 snap -1 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1848: expect (ObjNum 615 snap 161 seq_num 615) 2026-02-17T05:56:51.557 INFO:tasks.rados.rados.0.trial115.stdout:1849: snap_remove snap 177 2026-02-17T05:56:52.884 INFO:tasks.rados.rados.0.trial115.stdout:1849: done (11 left) 2026-02-17T05:56:52.885 INFO:tasks.rados.rados.0.trial115.stdout:1850: write (excl) oid 8 current snap is 185 2026-02-17T05:56:52.885 INFO:tasks.rados.rados.0.trial115.stdout:1850: seq_num 669 ranges {443883=464292,1365277=511220,2572335=671548} 2026-02-17T05:56:52.889 INFO:tasks.rados.rados.0.trial115.stdout:1850: writing trial11545420-8 from 443883 to 908175 tid 1 2026-02-17T05:56:52.893 INFO:tasks.rados.rados.0.trial115.stdout:1850: writing trial11545420-8 from 1365277 to 1876497 tid 2 2026-02-17T05:56:52.898 INFO:tasks.rados.rados.0.trial115.stdout:1850: writing trial11545420-8 from 2572335 to 3243883 tid 3 2026-02-17T05:56:52.898 INFO:tasks.rados.rados.0.trial115.stdout:1851: snap_remove snap 170 2026-02-17T05:56:53.885 INFO:tasks.rados.rados.0.trial115.stdout:1851: done (12 left) 2026-02-17T05:56:53.886 INFO:tasks.rados.rados.0.trial115.stdout:1852: write (excl) oid 17 current snap is 185 2026-02-17T05:56:53.886 INFO:tasks.rados.rados.0.trial115.stdout:1852: seq_num 670 ranges {460690=655894,1540112=771231,2716111=539778} 2026-02-17T05:56:53.889 INFO:tasks.rados.rados.0.trial115.stdout:1852: writing trial11545420-17 from 460690 to 1116584 tid 1 2026-02-17T05:56:53.894 INFO:tasks.rados.rados.0.trial115.stdout:1852: writing trial11545420-17 from 1540112 to 2311343 tid 2 2026-02-17T05:56:53.898 INFO:tasks.rados.rados.0.trial115.stdout:1852: writing trial11545420-17 from 2716111 to 3255889 tid 3 2026-02-17T05:56:53.898 INFO:tasks.rados.rados.0.trial115.stdout:1853: snap_create 2026-02-17T05:56:53.898 INFO:tasks.rados.rados.0.trial115.stdout:1840: finishing rollback tid 2 to trial11545420-27 2026-02-17T05:56:53.898 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 763 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:56:53.898 INFO:tasks.rados.rados.0.trial115.stdout:1841: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:56:53.976 INFO:tasks.rados.rados.0.trial115.stdout:1846: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:56:54.042 INFO:tasks.rados.rados.0.trial115.stdout:1838: expect (ObjNum 629 snap 164 seq_num 629) 2026-02-17T05:56:54.132 INFO:tasks.rados.rados.0.trial115.stdout:1836: expect (ObjNum 642 snap 170 seq_num 642) 2026-02-17T05:56:54.222 INFO:tasks.rados.rados.0.trial115.stdout:1848: expect (ObjNum 615 snap 161 seq_num 615) 2026-02-17T05:56:54.336 INFO:tasks.rados.rados.0.trial115.stdout:1845: expect (ObjNum 654 snap 177 seq_num 654) 2026-02-17T05:56:54.475 INFO:tasks.rados.rados.0.trial115.stdout:1839: finishing copy_from racing read to trial11545420-44 2026-02-17T05:56:54.475 INFO:tasks.rados.rados.0.trial115.stdout:1844: finishing copy_from racing read to trial11545420-2 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1839: finishing copy_from to trial11545420-44 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 822 (ObjNum 634 snap 165 seq_num 634) dirty exists 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1844: finishing copy_from to trial11545420-2 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 852 (ObjNum 646 snap 173 seq_num 646) dirty exists 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: finishing write tid 1 to trial11545420-8 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: finishing write tid 2 to trial11545420-8 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: finishing write tid 3 to trial11545420-8 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: finishing write tid 4 to trial11545420-8 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: finishing write tid 5 to trial11545420-8 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: oid 8 updating version 0 to 212 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: oid 8 updating version 212 to 215 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: oid 8 version 215 is already newer than 213 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: oid 8 version 215 is already newer than 214 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 215 (ObjNum 669 snap 185 seq_num 669) dirty exists 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1850: left oid 8 (ObjNum 669 snap 185 seq_num 669) 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1852: finishing write tid 1 to trial11545420-17 2026-02-17T05:56:54.476 INFO:tasks.rados.rados.0.trial115.stdout:1852: finishing write tid 2 to trial11545420-17 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:1852: finishing write tid 3 to trial11545420-17 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:1852: finishing write tid 4 to trial11545420-17 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:1852: finishing write tid 5 to trial11545420-17 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:1852: oid 17 updating version 0 to 385 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:1852: oid 17 version 385 is already newer than 383 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:1852: oid 17 version 385 is already newer than 384 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:1852: oid 17 updating version 385 to 386 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 386 (ObjNum 670 snap 185 seq_num 670) dirty exists 2026-02-17T05:56:54.477 INFO:tasks.rados.rados.0.trial115.stdout:1852: left oid 17 (ObjNum 670 snap 185 seq_num 670) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1836: done (13 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1838: done (12 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1839: done (11 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1840: done (10 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1841: done (9 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1843: done (8 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1844: done (7 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1845: done (6 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1846: done (5 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1847: done (4 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1848: done (3 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1850: done (2 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1852: done (1 left) 2026-02-17T05:56:54.897 INFO:tasks.rados.rados.0.trial115.stdout:1853: done (0 left) 2026-02-17T05:56:54.898 INFO:tasks.rados.rados.0.trial115.stdout:1854: write oid 43 current snap is 186 2026-02-17T05:56:54.898 INFO:tasks.rados.rados.0.trial115.stdout:1854: seq_num 671 ranges {477497=731143,1998594=514889,3143534=133963} 2026-02-17T05:56:54.901 INFO:tasks.rados.rados.0.trial115.stdout:1854: writing trial11545420-43 from 477497 to 1208640 tid 1 2026-02-17T05:56:54.906 INFO:tasks.rados.rados.0.trial115.stdout:1854: writing trial11545420-43 from 1998594 to 2513483 tid 2 2026-02-17T05:56:54.909 INFO:tasks.rados.rados.0.trial115.stdout:1854: writing trial11545420-43 from 3143534 to 3277497 tid 3 2026-02-17T05:56:54.909 INFO:tasks.rados.rados.0.trial115.stdout:1855: write (excl) oid 12 current snap is 186 2026-02-17T05:56:54.909 INFO:tasks.rados.rados.0.trial115.stdout:1854: finishing write tid 1 to trial11545420-43 2026-02-17T05:56:54.909 INFO:tasks.rados.rados.0.trial115.stdout:1854: finishing write tid 2 to trial11545420-43 2026-02-17T05:56:54.909 INFO:tasks.rados.rados.0.trial115.stdout:1855: seq_num 672 ranges {494304=406392,1540723=774900,2770957=523347} 2026-02-17T05:56:54.912 INFO:tasks.rados.rados.0.trial115.stdout:1855: writing trial11545420-12 from 494304 to 900696 tid 1 2026-02-17T05:56:54.917 INFO:tasks.rados.rados.0.trial115.stdout:1855: writing trial11545420-12 from 1540723 to 2315623 tid 2 2026-02-17T05:56:54.921 INFO:tasks.rados.rados.0.trial115.stdout:1855: writing trial11545420-12 from 2770957 to 3294304 tid 3 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1856: copy_from oid 39 from oid 6 current snap is 186 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: finishing write tid 3 to trial11545420-43 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: finishing write tid 4 to trial11545420-43 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: finishing write tid 5 to trial11545420-43 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: oid 43 updating version 0 to 681 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: oid 43 updating version 681 to 683 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: oid 43 version 683 is already newer than 682 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: oid 43 updating version 683 to 684 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 684 (ObjNum 671 snap 186 seq_num 671) dirty exists 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: left oid 43 (ObjNum 671 snap 186 seq_num 671) 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1855: finishing write tid 1 to trial11545420-12 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1855: finishing write tid 2 to trial11545420-12 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1854: done (2 left) 2026-02-17T05:56:54.922 INFO:tasks.rados.rados.0.trial115.stdout:1857: delete oid 20 current snap is 186 2026-02-17T05:56:54.923 INFO:tasks.rados.rados.0.trial115.stdout:1857: done (2 left) 2026-02-17T05:56:54.923 INFO:tasks.rados.rados.0.trial115.stdout:1858: write (excl) oid 9 current snap is 186 2026-02-17T05:56:54.923 INFO:tasks.rados.rados.0.trial115.stdout:1858: seq_num 674 ranges {527918=556890,1657687=778569,3225803=102115} 2026-02-17T05:56:54.927 INFO:tasks.rados.rados.0.trial115.stdout:1858: writing trial11545420-9 from 527918 to 1084808 tid 1 2026-02-17T05:56:54.933 INFO:tasks.rados.rados.0.trial115.stdout:1858: writing trial11545420-9 from 1657687 to 2436256 tid 2 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1858: writing trial11545420-9 from 3225803 to 3327918 tid 3 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1859: write (excl) oid 22 current snap is 186 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1855: finishing write tid 3 to trial11545420-12 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1855: finishing write tid 4 to trial11545420-12 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1855: finishing write tid 5 to trial11545420-12 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1855: oid 12 updating version 0 to 1052 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1855: oid 12 version 1052 is already newer than 1050 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1855: oid 12 version 1052 is already newer than 1049 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:1855: oid 12 version 1052 is already newer than 1051 2026-02-17T05:56:54.937 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1052 (ObjNum 672 snap 186 seq_num 672) dirty exists 2026-02-17T05:56:54.938 INFO:tasks.rados.rados.0.trial115.stdout:1855: left oid 12 (ObjNum 672 snap 186 seq_num 672) 2026-02-17T05:56:54.938 INFO:tasks.rados.rados.0.trial115.stdout:1856: finishing copy_from racing read to trial11545420-39 2026-02-17T05:56:54.938 INFO:tasks.rados.rados.0.trial115.stdout:1858: finishing write tid 1 to trial11545420-9 2026-02-17T05:56:54.938 INFO:tasks.rados.rados.0.trial115.stdout:1859: seq_num 675 ranges {544725=632139,1716169=522227,2736873=512491} 2026-02-17T05:56:54.942 INFO:tasks.rados.rados.0.trial115.stdout:1859: writing trial11545420-22 from 544725 to 1176864 tid 1 2026-02-17T05:56:54.946 INFO:tasks.rados.rados.0.trial115.stdout:1859: writing trial11545420-22 from 1716169 to 2238396 tid 2 2026-02-17T05:56:54.950 INFO:tasks.rados.rados.0.trial115.stdout:1859: writing trial11545420-22 from 2736873 to 3249364 tid 3 2026-02-17T05:56:54.950 INFO:tasks.rados.rados.0.trial115.stdout:1855: done (3 left) 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1860: rollback oid 20 current snap is 186 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 184 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1856: finishing copy_from to trial11545420-39 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 651 (ObjNum 657 snap 178 seq_num 657) dirty exists 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: finishing write tid 2 to trial11545420-9 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: finishing write tid 3 to trial11545420-9 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1856: done (3 left) 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1861: read oid 5 snap -1 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1861: expect (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1862: delete oid 36 current snap is 186 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: finishing write tid 4 to trial11545420-9 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: finishing write tid 5 to trial11545420-9 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: oid 9 updating version 0 to 886 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: oid 9 version 886 is already newer than 884 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: oid 9 version 886 is already newer than 883 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: oid 9 version 886 is already newer than 885 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 886 (ObjNum 674 snap 186 seq_num 674) dirty exists 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1858: left oid 9 (ObjNum 674 snap 186 seq_num 674) 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1859: finishing write tid 1 to trial11545420-22 2026-02-17T05:56:54.951 INFO:tasks.rados.rados.0.trial115.stdout:1859: finishing write tid 2 to trial11545420-22 2026-02-17T05:56:54.952 INFO:tasks.rados.rados.0.trial115.stdout:1860: finishing rollback tid 1 to trial11545420-20 2026-02-17T05:56:54.952 INFO:tasks.rados.rados.0.trial115.stdout:1860: finishing rollback tid 2 to trial11545420-20 2026-02-17T05:56:54.952 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 688 (ObjNum 598 snap 155 seq_num 598) dirty exists 2026-02-17T05:56:54.952 INFO:tasks.rados.rados.0.trial115.stdout:1858: done (4 left) 2026-02-17T05:56:54.952 INFO:tasks.rados.rados.0.trial115.stdout:1860: done (3 left) 2026-02-17T05:56:54.952 INFO:tasks.rados.rados.0.trial115.stdout:1862: done (2 left) 2026-02-17T05:56:54.952 INFO:tasks.rados.rados.0.trial115.stdout:1863: write oid 39 current snap is 186 2026-02-17T05:56:54.952 INFO:tasks.rados.rados.0.trial115.stdout:1863: seq_num 677 ranges {578339=498990,1833133=525896,2791719=586620} 2026-02-17T05:56:54.956 INFO:tasks.rados.rados.0.trial115.stdout:1863: writing trial11545420-39 from 578339 to 1077329 tid 1 2026-02-17T05:56:54.960 INFO:tasks.rados.rados.0.trial115.stdout:1863: writing trial11545420-39 from 1833133 to 2359029 tid 2 2026-02-17T05:56:54.965 INFO:tasks.rados.rados.0.trial115.stdout:1863: writing trial11545420-39 from 2791719 to 3378339 tid 3 2026-02-17T05:56:54.965 INFO:tasks.rados.rados.0.trial115.stdout:1864: read oid 43 snap -1 2026-02-17T05:56:54.965 INFO:tasks.rados.rados.0.trial115.stdout:1864: expect (ObjNum 671 snap 186 seq_num 671) 2026-02-17T05:56:54.965 INFO:tasks.rados.rados.0.trial115.stdout:1865: read oid 6 snap 174 2026-02-17T05:56:54.965 INFO:tasks.rados.rados.0.trial115.stdout:1865: expect (ObjNum 616 snap 161 seq_num 616) 2026-02-17T05:56:54.965 INFO:tasks.rados.rados.0.trial115.stdout:1866: copy_from oid 7 from oid 47 current snap is 186 2026-02-17T05:56:54.965 INFO:tasks.rados.rados.0.trial115.stdout:1861: expect (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:56:55.026 INFO:tasks.rados.rados.0.trial115.stdout:1859: finishing write tid 3 to trial11545420-22 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1859: finishing write tid 4 to trial11545420-22 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1859: finishing write tid 5 to trial11545420-22 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1859: oid 22 updating version 0 to 774 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1859: oid 22 updating version 774 to 775 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1859: oid 22 updating version 775 to 776 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1859: oid 22 updating version 776 to 777 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 777 (ObjNum 675 snap 186 seq_num 675) dirty exists 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1859: left oid 22 (ObjNum 675 snap 186 seq_num 675) 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: finishing write tid 1 to trial11545420-39 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1859: done (5 left) 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1861: done (4 left) 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1867: delete oid 36 current snap is 186 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: finishing write tid 2 to trial11545420-39 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: finishing write tid 3 to trial11545420-39 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: finishing write tid 4 to trial11545420-39 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: finishing write tid 5 to trial11545420-39 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: oid 39 updating version 0 to 1058 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: oid 39 version 1058 is already newer than 1057 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: oid 39 version 1058 is already newer than 1055 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: oid 39 version 1058 is already newer than 1056 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1058 (ObjNum 677 snap 186 seq_num 677) dirty exists 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1863: left oid 39 (ObjNum 677 snap 186 seq_num 677) 2026-02-17T05:56:55.027 INFO:tasks.rados.rados.0.trial115.stdout:1865: expect (ObjNum 616 snap 161 seq_num 616) 2026-02-17T05:56:55.089 INFO:tasks.rados.rados.0.trial115.stdout:1864: expect (ObjNum 671 snap 186 seq_num 671) 2026-02-17T05:56:55.168 INFO:tasks.rados.rados.0.trial115.stdout:1866: finishing copy_from racing read to trial11545420-7 2026-02-17T05:56:55.168 INFO:tasks.rados.rados.0.trial115.stdout:1866: finishing copy_from to trial11545420-7 2026-02-17T05:56:55.168 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 603 (ObjNum 598 snap 155 seq_num 598) dirty exists 2026-02-17T05:56:55.168 INFO:tasks.rados.rados.0.trial115.stdout:1863: done (4 left) 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:1864: done (3 left) 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:1865: done (2 left) 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:1866: done (1 left) 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:1867: done (0 left) 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:1868: read oid 41 snap -1 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:1868: expect (ObjNum 658 snap 181 seq_num 658) 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:1869: rollback oid 17 current snap is 186 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 180 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:1870: rollback oid 40 current snap is 186 2026-02-17T05:56:55.169 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 179 2026-02-17T05:56:55.170 INFO:tasks.rados.rados.0.trial115.stdout:1871: write (excl) oid 23 current snap is 186 2026-02-17T05:56:55.170 INFO:tasks.rados.rados.0.trial115.stdout:1871: seq_num 679 ranges {611953=649488,1950097=529565,3246565=165388} 2026-02-17T05:56:55.172 INFO:tasks.rados.rados.0.trial115.stdout:1871: writing trial11545420-23 from 611953 to 1261441 tid 1 2026-02-17T05:56:55.176 INFO:tasks.rados.rados.0.trial115.stdout:1871: writing trial11545420-23 from 1950097 to 2479662 tid 2 2026-02-17T05:56:55.180 INFO:tasks.rados.rados.0.trial115.stdout:1871: writing trial11545420-23 from 3246565 to 3411953 tid 3 2026-02-17T05:56:55.180 INFO:tasks.rados.rados.0.trial115.stdout:1872: snap_remove snap 184 2026-02-17T05:56:55.893 INFO:tasks.rados.rados.0.trial115.stdout:1872: done (4 left) 2026-02-17T05:56:55.894 INFO:tasks.rados.rados.0.trial115.stdout:1873: write oid 31 current snap is 186 2026-02-17T05:56:55.894 INFO:tasks.rados.rados.0.trial115.stdout:1873: seq_num 680 ranges {628760=724737,1892226=673223,3157635=271125} 2026-02-17T05:56:55.898 INFO:tasks.rados.rados.0.trial115.stdout:1873: writing trial11545420-31 from 628760 to 1353497 tid 1 2026-02-17T05:56:55.902 INFO:tasks.rados.rados.0.trial115.stdout:1873: writing trial11545420-31 from 1892226 to 2565449 tid 2 2026-02-17T05:56:55.906 INFO:tasks.rados.rados.0.trial115.stdout:1873: writing trial11545420-31 from 3157635 to 3428760 tid 3 2026-02-17T05:56:55.907 INFO:tasks.rados.rados.0.trial115.stdout:1874: copy_from oid 20 from oid 28 current snap is 186 2026-02-17T05:56:55.907 INFO:tasks.rados.rados.0.trial115.stdout:1870: finishing rollback tid 0 to trial11545420-40 2026-02-17T05:56:55.907 INFO:tasks.rados.rados.0.trial115.stdout:1870: finishing rollback tid 1 to trial11545420-40 2026-02-17T05:56:55.907 INFO:tasks.rados.rados.0.trial115.stdout:1870: finishing rollback tid 2 to trial11545420-40 2026-02-17T05:56:55.907 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 819 (ObjNum 632 snap 164 seq_num 632) dirty exists 2026-02-17T05:56:55.907 INFO:tasks.rados.rados.0.trial115.stdout:1868: expect (ObjNum 658 snap 181 seq_num 658) 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:1869: finishing rollback tid 0 to trial11545420-17 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:1869: finishing rollback tid 1 to trial11545420-17 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 389 (ObjNum 65 snap 0 seq_num 0) dirty dne 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:1871: finishing write tid 1 to trial11545420-23 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:1871: finishing write tid 2 to trial11545420-23 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:1868: done (5 left) 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:1869: done (4 left) 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:1870: done (3 left) 2026-02-17T05:56:55.977 INFO:tasks.rados.rados.0.trial115.stdout:1875: snap_remove snap 166 2026-02-17T05:56:56.402 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T05:56:56.402 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-17T05:56:56.402 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-17T05:56:56.402 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-17T05:56:56.404 INFO:tasks.ceph.mon.c:Started 2026-02-17T05:56:56.404 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:56:56.405 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.471+0000 7f85f5076640 10 monclient: get_monmap_and_config 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.471+0000 7f85f5076640 10 monclient: build_initial_monmap 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.471+0000 7f85f5076640 10 monclient: monmap: 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:56.472029+0000 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:56.472029+0000 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:56.475 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 1 Processor -- start 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 1 -- start start 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient: init 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2026-02-17T05:56:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f85f006c890 0x7f85f006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient(hunting): picked mon.noname-a con 0x7f85f006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 0x7f85f006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient(hunting): picked mon.noname-b con 0x7f85f009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f85f0134df0 0x7f85f0135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient(hunting): picked mon.noname-c con 0x7f85f0134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f85f00a0880 con 0x7f85f006c890 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f85f006d6f0 con 0x7f85f0134df0 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f85f006daf0 con 0x7f85f009c4a0 2026-02-17T05:56:56.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85f5076640 10 monclient(hunting): authenticate will time out at 1262.025578s 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f85f006c890 0x7f85f006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 0x7f85f006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f85f006c890 0x7f85f006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55708/0 (socket says 10.20.193.78:55708) 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 -- 10.20.193.78:0/903051421 learned_addr learned my addr 10.20.193.78:0/903051421 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient(hunting): get_auth_request con 0x7f85f006c890 auth_method 0 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 10 monclient(hunting): get_auth_request con 0x7f85f009c4a0 auth_method 0 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 1 -- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 msgr2=0x7f85f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 1 -- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 msgr2=0x7f85f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 1 --2- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 0x7f85f006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85ee575640 1 --2- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 0x7f85f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient(hunting): handle_auth_done global_id 124311 payload 274 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 -- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 msgr2=0x7f85f006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 --2- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 0x7f85f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 -- 10.20.193.78:0/903051421 >> v2:10.20.193.115:3300/0 conn(0x7f85f0134df0 msgr2=0x7f85f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 --2- 10.20.193.78:0/903051421 >> v2:10.20.193.115:3300/0 conn(0x7f85f0134df0 0x7f85f0135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient: _finish_hunting 0 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 monclient: found mon.noname-a 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.472+0000 7f85eed76640 1 -- 10.20.193.78:0/903051421 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f009cc80 con 0x7f85f006c890 2026-02-17T05:56:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85eed76640 1 -- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f006c890 msgr2=0x7f85f006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85eed76640 1 --2- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f006c890 0x7f85f006cc70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85f00a7490 tx=0x7f85e402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dfccbc50760863 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 1 -- 10.20.193.78:0/903051421 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e403c070 con 0x7f85f006c890 2026-02-17T05:56:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: dump: 2026-02-17T05:56:56.488 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:56.488 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: _finish_auth 0 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: _check_auth_tickets 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 1 -- 10.20.193.78:0/903051421 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e402fca0 con 0x7f85f006c890 2026-02-17T05:56:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 1 -- 10.20.193.78:0/903051421 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4037660 con 0x7f85f006c890 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85edd74640 10 monclient: dump: 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:56.490 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 5 monclient: authenticate success, global_id 124311 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 4 set_mon_vals no callback set 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 10 monclient: shutdown 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 -- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f006c890 msgr2=0x7f85f006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 --2- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f006c890 0x7f85f006cc70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85f00a7490 tx=0x7f85e402ee60 comp rx=0 tx=0).stop 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 -- 10.20.193.78:0/903051421 shutdown_connections 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 --2- 10.20.193.78:0/903051421 >> v2:10.20.193.115:3300/0 conn(0x7f85f0134df0 0x7f85f0135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 --2- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f009c4a0 0x7f85f006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 --2- 10.20.193.78:0/903051421 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f006c890 0x7f85f006cc70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 -- 10.20.193.78:0/903051421 >> 10.20.193.78:0/903051421 conn(0x7f85f00a6620 msgr2=0x7f85f00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:56.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 -- 10.20.193.78:0/903051421 shutdown_connections 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.475+0000 7f85f5076640 1 -- 10.20.193.78:0/903051421 wait complete. 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient: build_initial_monmap 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient: monmap: 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:56.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 1 Processor -- start 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 1 -- start start 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient: init 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85f006c890 0x7f85f01cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient(hunting): picked mon.noname-b con 0x7f85f006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f009c4a0 0x7f85f01cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient(hunting): picked mon.noname-a con 0x7f85f009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 0x7f85f0071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient(hunting): picked mon.noname-c con 0x7f85f0134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85f00a01a0 con 0x7f85f009c4a0 2026-02-17T05:56:56.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85f00a0020 con 0x7f85f006c890 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f85f009be20 con 0x7f85f0134df0 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85f5076640 10 monclient(hunting): authenticate will time out at 1262.029723s 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f009c4a0 0x7f85f01cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 0x7f85f0071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f009c4a0 0x7f85f01cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55710/0 (socket says 10.20.193.78:55710) 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 -- 10.20.193.78:0/1754971093 learned_addr learned my addr 10.20.193.78:0/1754971093 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient(hunting): get_auth_request con 0x7f85f009c4a0 auth_method 0 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 10 monclient(hunting): get_auth_request con 0x7f85f0134df0 auth_method 0 2026-02-17T05:56:56.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 1 -- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 msgr2=0x7f85f0071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 1 -- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 msgr2=0x7f85f0071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 0x7f85f0071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ef577640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 0x7f85f0071590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85eed76640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85f006c890 0x7f85f01cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85eed76640 10 monclient(hunting): get_auth_request con 0x7f85f006c890 auth_method 0 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85eed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85eed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85eed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:56.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient(hunting): handle_auth_done global_id 124314 payload 824 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 -- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 msgr2=0x7f85f0071590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 0x7f85f0071590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 -- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85f006c890 msgr2=0x7f85f01cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85f006c890 0x7f85f01cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient: _finish_hunting 0 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 monclient: found mon.noname-a 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.476+0000 7f85ee575640 1 -- 10.20.193.78:0/1754971093 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f009d810 con 0x7f85f009c4a0 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85ee575640 1 -- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f009c4a0 msgr2=0x7f85f01cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85ee575640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f009c4a0 0x7f85f01cfd30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85d800e7c0 tx=0x7f85d800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 1 -- 10.20.193.78:0/1754971093 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8017070 con 0x7f85f009c4a0 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:56.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: dump: 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: _finish_auth 0 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 1 -- 10.20.193.78:0/1754971093 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d800ca80 con 0x7f85f009c4a0 2026-02-17T05:56:56.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 1 -- 10.20.193.78:0/1754971093 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d800f040 con 0x7f85f009c4a0 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85ed573640 4 set_mon_vals no callback set 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 10 monclient: dump: 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:56.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 5 monclient: authenticate success, global_id 124314 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 10 monclient: _renew_subs 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f00a16b0 con 0x7f85f009c4a0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 10 monclient: _renew_subs 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f0132670 con 0x7f85f009c4a0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85cf7fe640 1 -- 10.20.193.78:0/1754971093 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f85d8012050 con 0x7f85f009c4a0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.479+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f0130cc0 con 0x7f85f009c4a0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.480+0000 7f85cf7fe640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85bc03deb0 0x7f85bc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.480+0000 7f85cf7fe640 1 -- 10.20.193.78:0/1754971093 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(390..390 src has 335..390) ==== 4827+0+0 (secure 0 0 0) 0x7f85d8052650 con 0x7f85f009c4a0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.480+0000 7f85eed76640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85bc03deb0 0x7f85bc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:56.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.480+0000 7f85eed76640 10 monclient: get_auth_request con 0x7f85bc03deb0 auth_method 0 2026-02-17T05:56:56.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.482+0000 7f85cf7fe640 1 -- 10.20.193.78:0/1754971093 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f85f0130cc0 con 0x7f85f009c4a0 2026-02-17T05:56:56.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.482+0000 7f85cf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:56.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.482+0000 7f85cf7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:56.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.482+0000 7f85eed76640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85bc03deb0 0x7f85bc040370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f85e4036000 tx=0x7f85e4039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:56.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.603+0000 7f85f5076640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:56.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.603+0000 7f85f5076640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:56.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.603+0000 7f85f5076640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:56.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.603+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f85f0074980 con 0x7f85f009c4a0 2026-02-17T05:56:56.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.603+0000 7f85cf7fe640 1 -- 10.20.193.78:0/1754971093 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f85d8051020 con 0x7f85f009c4a0 2026-02-17T05:56:56.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.603+0000 7f85cf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:56.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.603+0000 7f85cf7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:56.604 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":116,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:56.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85bc03deb0 msgr2=0x7f85bc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:56.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85bc03deb0 0x7f85bc040370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f85e4036000 tx=0x7f85e4039040 comp rx=0 tx=0).stop 2026-02-17T05:56:56.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 10 monclient: shutdown 2026-02-17T05:56:56.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:56.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f009c4a0 msgr2=0x7f85f01cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:56.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f009c4a0 0x7f85f01cfd30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85d800e7c0 tx=0x7f85d800ec90 comp rx=0 tx=0).stop 2026-02-17T05:56:56.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 shutdown_connections 2026-02-17T05:56:56.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f85f0134df0 0x7f85f0071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f85f009c4a0 0x7f85f01cfd30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f85bc03deb0 0x7f85bc040370 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 --2- 10.20.193.78:0/1754971093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f85f006c890 0x7f85f01cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:56.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 >> 10.20.193.78:0/1754971093 conn(0x7f85f00a6620 msgr2=0x7f85f012d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:56.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 shutdown_connections 2026-02-17T05:56:56.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:56.604+0000 7f85f5076640 1 -- 10.20.193.78:0/1754971093 wait complete. 2026-02-17T05:56:56.898 INFO:tasks.rados.rados.0.trial115.stdout:1875: done (3 left) 2026-02-17T05:56:56.898 INFO:tasks.rados.rados.0.trial115.stdout:1876: rollback oid 45 current snap is 186 2026-02-17T05:56:56.899 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 174 2026-02-17T05:56:56.899 INFO:tasks.rados.rados.0.trial115.stdout:1871: finishing write tid 3 to trial11545420-23 2026-02-17T05:56:56.899 INFO:tasks.rados.rados.0.trial115.stdout:1871: finishing write tid 4 to trial11545420-23 2026-02-17T05:56:56.899 INFO:tasks.rados.rados.0.trial115.stdout:1871: finishing write tid 5 to trial11545420-23 2026-02-17T05:56:56.899 INFO:tasks.rados.rados.0.trial115.stdout:1871: oid 23 updating version 0 to 392 2026-02-17T05:56:56.899 INFO:tasks.rados.rados.0.trial115.stdout:1871: oid 23 updating version 392 to 394 2026-02-17T05:56:56.899 INFO:tasks.rados.rados.0.trial115.stdout:1871: oid 23 version 394 is already newer than 391 2026-02-17T05:56:56.899 INFO:tasks.rados.rados.0.trial115.stdout:1871: oid 23 version 394 is already newer than 393 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 394 (ObjNum 679 snap 186 seq_num 679) dirty exists 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1871: left oid 23 (ObjNum 679 snap 186 seq_num 679) 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: finishing write tid 1 to trial11545420-31 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: finishing write tid 2 to trial11545420-31 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: finishing write tid 3 to trial11545420-31 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: finishing write tid 4 to trial11545420-31 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: finishing write tid 5 to trial11545420-31 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: oid 31 updating version 0 to 887 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: oid 31 updating version 887 to 889 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: oid 31 version 889 is already newer than 886 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:1873: oid 31 version 889 is already newer than 888 2026-02-17T05:56:56.900 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 889 (ObjNum 680 snap 186 seq_num 680) dirty exists 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:1873: left oid 31 (ObjNum 680 snap 186 seq_num 680) 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:1874: finishing copy_from racing read to trial11545420-20 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:1874: finishing copy_from to trial11545420-20 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 863 (ObjNum 660 snap 182 seq_num 660) dirty exists 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:1871: done (3 left) 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:1873: done (2 left) 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:1874: done (1 left) 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:1877: rollback oid 30 current snap is 186 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 172 2026-02-17T05:56:56.901 INFO:tasks.rados.rados.0.trial115.stdout:1878: snap_remove snap 182 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:1878: done (2 left) 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:1879: write oid 34 current snap is 186 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:1877: finishing rollback tid 1 to trial11545420-30 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 731 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:1876: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:1876: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:1876: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1066 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:56:57.898 INFO:tasks.rados.rados.0.trial115.stdout:1879: seq_num 681 ranges {645567=799986,1950708=533234,3185058=260509} 2026-02-17T05:56:57.903 INFO:tasks.rados.rados.0.trial115.stdout:1879: writing trial11545420-34 from 645567 to 1445553 tid 1 2026-02-17T05:56:57.907 INFO:tasks.rados.rados.0.trial115.stdout:1879: writing trial11545420-34 from 1950708 to 2483942 tid 2 2026-02-17T05:56:57.911 INFO:tasks.rados.rados.0.trial115.stdout:1879: writing trial11545420-34 from 3185058 to 3445567 tid 3 2026-02-17T05:56:57.911 INFO:tasks.rados.rados.0.trial115.stdout:1876: done (2 left) 2026-02-17T05:56:57.911 INFO:tasks.rados.rados.0.trial115.stdout:1877: done (1 left) 2026-02-17T05:56:57.911 INFO:tasks.rados.rados.0.trial115.stdout:1880: rollback oid 33 current snap is 186 2026-02-17T05:56:57.911 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 175 2026-02-17T05:56:57.911 INFO:tasks.rados.rados.0.trial115.stdout:1879: finishing write tid 1 to trial11545420-34 2026-02-17T05:56:57.911 INFO:tasks.rados.rados.0.trial115.stdout:1879: finishing write tid 2 to trial11545420-34 2026-02-17T05:56:57.912 INFO:tasks.rados.rados.0.trial115.stdout:1881: write (excl) oid 3 current snap is 186 2026-02-17T05:56:57.912 INFO:tasks.rados.rados.0.trial115.stdout:1881: seq_num 682 ranges {662374=475235,1609190=676892,2812481=624101} 2026-02-17T05:56:57.915 INFO:tasks.rados.rados.0.trial115.stdout:1881: writing trial11545420-3 from 662374 to 1137609 tid 1 2026-02-17T05:56:57.920 INFO:tasks.rados.rados.0.trial115.stdout:1881: writing trial11545420-3 from 1609190 to 2286082 tid 2 2026-02-17T05:56:57.924 INFO:tasks.rados.rados.0.trial115.stdout:1881: writing trial11545420-3 from 2812481 to 3436582 tid 3 2026-02-17T05:56:57.924 INFO:tasks.rados.rados.0.trial115.stdout:1882: write oid 41 current snap is 186 2026-02-17T05:56:57.924 INFO:tasks.rados.rados.0.trial115.stdout:1882: seq_num 683 ranges {679181=550484,1667672=536903,2839904=435373} 2026-02-17T05:56:57.929 INFO:tasks.rados.rados.0.trial115.stdout:1882: writing trial11545420-41 from 679181 to 1229665 tid 1 2026-02-17T05:56:57.933 INFO:tasks.rados.rados.0.trial115.stdout:1882: writing trial11545420-41 from 1667672 to 2204575 tid 2 2026-02-17T05:56:57.938 INFO:tasks.rados.rados.0.trial115.stdout:1882: writing trial11545420-41 from 2839904 to 3275277 tid 3 2026-02-17T05:56:57.939 INFO:tasks.rados.rados.0.trial115.stdout:1883: snap_remove snap 183 2026-02-17T05:56:58.551 INFO:tasks.rados.rados.0.trial115.stdout:1883: done (4 left) 2026-02-17T05:56:58.552 INFO:tasks.rados.rados.0.trial115.stdout:1884: read oid 13 snap 172 2026-02-17T05:56:58.552 INFO:tasks.rados.rados.0.trial115.stdout:1884: expect (ObjNum 631 snap 164 seq_num 631) 2026-02-17T05:56:58.552 INFO:tasks.rados.rados.0.trial115.stdout:1885: snap_create 2026-02-17T05:56:58.552 INFO:tasks.rados.rados.0.trial115.stdout:1880: finishing rollback tid 1 to trial11545420-33 2026-02-17T05:56:58.553 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 613 (ObjNum 33 snap 0 seq_num 1331149504) dirty dne 2026-02-17T05:56:58.553 INFO:tasks.rados.rados.0.trial115.stdout:1879: finishing write tid 3 to trial11545420-34 2026-02-17T05:56:58.553 INFO:tasks.rados.rados.0.trial115.stdout:1879: finishing write tid 4 to trial11545420-34 2026-02-17T05:56:58.553 INFO:tasks.rados.rados.0.trial115.stdout:1879: finishing write tid 5 to trial11545420-34 2026-02-17T05:56:58.553 INFO:tasks.rados.rados.0.trial115.stdout:1879: oid 34 updating version 0 to 656 2026-02-17T05:56:58.553 INFO:tasks.rados.rados.0.trial115.stdout:1879: oid 34 updating version 656 to 659 2026-02-17T05:56:58.553 INFO:tasks.rados.rados.0.trial115.stdout:1879: oid 34 version 659 is already newer than 657 2026-02-17T05:56:58.554 INFO:tasks.rados.rados.0.trial115.stdout:1879: oid 34 version 659 is already newer than 658 2026-02-17T05:56:58.554 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 659 (ObjNum 681 snap 186 seq_num 681) dirty exists 2026-02-17T05:56:58.554 INFO:tasks.rados.rados.0.trial115.stdout:1879: left oid 34 (ObjNum 681 snap 186 seq_num 681) 2026-02-17T05:56:58.554 INFO:tasks.rados.rados.0.trial115.stdout:1881: finishing write tid 1 to trial11545420-3 2026-02-17T05:56:58.554 INFO:tasks.rados.rados.0.trial115.stdout:1881: finishing write tid 2 to trial11545420-3 2026-02-17T05:56:58.554 INFO:tasks.rados.rados.0.trial115.stdout:1881: finishing write tid 3 to trial11545420-3 2026-02-17T05:56:58.554 INFO:tasks.rados.rados.0.trial115.stdout:1881: finishing write tid 4 to trial11545420-3 2026-02-17T05:56:58.555 INFO:tasks.rados.rados.0.trial115.stdout:1881: finishing write tid 5 to trial11545420-3 2026-02-17T05:56:58.555 INFO:tasks.rados.rados.0.trial115.stdout:1881: oid 3 updating version 0 to 894 2026-02-17T05:56:58.555 INFO:tasks.rados.rados.0.trial115.stdout:1881: oid 3 updating version 894 to 895 2026-02-17T05:56:58.555 INFO:tasks.rados.rados.0.trial115.stdout:1881: oid 3 version 895 is already newer than 893 2026-02-17T05:56:58.555 INFO:tasks.rados.rados.0.trial115.stdout:1881: oid 3 version 895 is already newer than 892 2026-02-17T05:56:58.555 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 895 (ObjNum 682 snap 186 seq_num 682) dirty exists 2026-02-17T05:56:58.555 INFO:tasks.rados.rados.0.trial115.stdout:1881: left oid 3 (ObjNum 682 snap 186 seq_num 682) 2026-02-17T05:56:58.556 INFO:tasks.rados.rados.0.trial115.stdout:1882: finishing write tid 1 to trial11545420-41 2026-02-17T05:56:58.556 INFO:tasks.rados.rados.0.trial115.stdout:1882: finishing write tid 2 to trial11545420-41 2026-02-17T05:56:58.556 INFO:tasks.rados.rados.0.trial115.stdout:1882: finishing write tid 3 to trial11545420-41 2026-02-17T05:56:58.556 INFO:tasks.rados.rados.0.trial115.stdout:1882: finishing write tid 4 to trial11545420-41 2026-02-17T05:56:58.556 INFO:tasks.rados.rados.0.trial115.stdout:1882: finishing write tid 5 to trial11545420-41 2026-02-17T05:56:58.556 INFO:tasks.rados.rados.0.trial115.stdout:1882: oid 41 updating version 0 to 863 2026-02-17T05:56:58.556 INFO:tasks.rados.rados.0.trial115.stdout:1882: oid 41 version 863 is already newer than 862 2026-02-17T05:56:58.557 INFO:tasks.rados.rados.0.trial115.stdout:1882: oid 41 updating version 863 to 865 2026-02-17T05:56:58.557 INFO:tasks.rados.rados.0.trial115.stdout:1882: oid 41 version 865 is already newer than 864 2026-02-17T05:56:58.557 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 865 (ObjNum 683 snap 186 seq_num 683) dirty exists 2026-02-17T05:56:58.557 INFO:tasks.rados.rados.0.trial115.stdout:1882: left oid 41 (ObjNum 683 snap 186 seq_num 683) 2026-02-17T05:56:58.558 INFO:tasks.rados.rados.0.trial115.stdout:1884: expect (ObjNum 631 snap 164 seq_num 631) 2026-02-17T05:56:59.613 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:56:59.628 INFO:tasks.rados.rados.0.trial115.stdout:1879: done (5 left) 2026-02-17T05:56:59.628 INFO:tasks.rados.rados.0.trial115.stdout:1880: done (4 left) 2026-02-17T05:56:59.628 INFO:tasks.rados.rados.0.trial115.stdout:1881: done (3 left) 2026-02-17T05:56:59.628 INFO:tasks.rados.rados.0.trial115.stdout:1882: done (2 left) 2026-02-17T05:56:59.628 INFO:tasks.rados.rados.0.trial115.stdout:1884: done (1 left) 2026-02-17T05:56:59.628 INFO:tasks.rados.rados.0.trial115.stdout:1885: done (0 left) 2026-02-17T05:56:59.629 INFO:tasks.rados.rados.0.trial115.stdout:1886: read oid 24 snap -1 2026-02-17T05:56:59.629 INFO:tasks.rados.rados.0.trial115.stdout:1886: expect (ObjNum 615 snap 161 seq_num 615) 2026-02-17T05:56:59.629 INFO:tasks.rados.rados.0.trial115.stdout:1887: delete oid 1 current snap is 187 2026-02-17T05:56:59.631 INFO:tasks.rados.rados.0.trial115.stdout:1887: done (1 left) 2026-02-17T05:56:59.631 INFO:tasks.rados.rados.0.trial115.stdout:1888: snap_remove snap 179 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient: get_monmap_and_config 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient: build_initial_monmap 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient: monmap: 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:56:59.682425+0000 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:56:59.682425+0000 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:56:59.682 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:56:59.683 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 1 Processor -- start 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 1 -- start start 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient: init 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd0e406c890 0x7fd0e406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient(hunting): picked mon.noname-a con 0x7fd0e406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd0e409c4a0 0x7fd0e406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient(hunting): picked mon.noname-c con 0x7fd0e409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 0x7fd0e4135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient(hunting): picked mon.noname-b con 0x7fd0e4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:59.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e40a0880 con 0x7fd0e406c890 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e406d6f0 con 0x7fd0e409c4a0 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e406daf0 con 0x7fd0e4134df0 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.682+0000 7fd0eb666640 10 monclient(hunting): authenticate will time out at 1265.235947s 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd0e406c890 0x7fd0e406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 0x7fd0e4135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 0x7fd0e4135280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:35756/0 (socket says 10.20.193.78:35756) 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 1 -- 10.20.193.78:0/2061697012 learned_addr learned my addr 10.20.193.78:0/2061697012 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd0e406c890 0x7fd0e406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44526/0 (socket says 10.20.193.78:44526) 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 10 monclient(hunting): get_auth_request con 0x7fd0e4134df0 auth_method 0 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient(hunting): get_auth_request con 0x7fd0e406c890 auth_method 0 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e8bda640 1 --2- 10.20.193.78:0/2061697012 >> v2:10.20.193.115:3300/0 conn(0x7fd0e409c4a0 0x7fd0e406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e8bda640 10 monclient(hunting): get_auth_request con 0x7fd0e409c4a0 auth_method 0 2026-02-17T05:56:59.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e8bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e8bda640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e8bda640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 1 -- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 msgr2=0x7fd0e4135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 1 -- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 msgr2=0x7fd0e4135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 1 --2- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 0x7fd0e4135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e9bdc640 1 --2- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 0x7fd0e4135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient(hunting): handle_auth_done global_id 124317 payload 274 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 1 -- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 msgr2=0x7fd0e4135280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 1 --2- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 0x7fd0e4135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 1 -- 10.20.193.78:0/2061697012 >> v2:10.20.193.115:3300/0 conn(0x7fd0e409c4a0 msgr2=0x7fd0e406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 1 --2- 10.20.193.78:0/2061697012 >> v2:10.20.193.115:3300/0 conn(0x7fd0e409c4a0 0x7fd0e406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient: _finish_hunting 0 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 1 monclient: found mon.noname-a 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:59.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.683+0000 7fd0e93db640 1 -- 10.20.193.78:0/2061697012 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e409cc80 con 0x7fd0e406c890 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0e93db640 1 -- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e406c890 msgr2=0x7fd0e406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0e93db640 1 --2- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e406c890 0x7fd0e406cc70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd0e409c980 tx=0x7fd0d402ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc36d241b5aae04 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 1 -- 10.20.193.78:0/2061697012 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d403c070 con 0x7fd0e406c890 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: dump: 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: _finish_auth 0 2026-02-17T05:56:59.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0eb666640 5 monclient: authenticate success, global_id 124317 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 1 -- 10.20.193.78:0/2061697012 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d402fbd0 con 0x7fd0e406c890 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 1 -- 10.20.193.78:0/2061697012 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4037440 con 0x7fd0e406c890 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0eb666640 10 monclient: get_monmap_and_config success 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0eb666640 4 set_mon_vals no callback set 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.685+0000 7fd0d3fff640 10 monclient: dump: 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 10 monclient: shutdown 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 -- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e406c890 msgr2=0x7fd0e406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e406c890 0x7fd0e406cc70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd0e409c980 tx=0x7fd0d402ed50 comp rx=0 tx=0).stop 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 -- 10.20.193.78:0/2061697012 shutdown_connections 2026-02-17T05:56:59.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e4134df0 0x7fd0e4135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2061697012 >> v2:10.20.193.115:3300/0 conn(0x7fd0e409c4a0 0x7fd0e406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2061697012 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e406c890 0x7fd0e406cc70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 -- 10.20.193.78:0/2061697012 >> 10.20.193.78:0/2061697012 conn(0x7fd0e40a6620 msgr2=0x7fd0e40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 -- 10.20.193.78:0/2061697012 shutdown_connections 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 -- 10.20.193.78:0/2061697012 wait complete. 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 10 monclient: build_initial_monmap 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 10 monclient: monmap: 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 Processor -- start 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 1 -- start start 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.686+0000 7fd0eb666640 10 monclient: init 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient: _reopen_session rank -1 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 0x7fd0e41cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient(hunting): picked mon.noname-c con 0x7fd0e406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:56:59.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0e409c4a0 0x7fd0e41cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient(hunting): picked mon.noname-b con 0x7fd0e409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e4134df0 0x7fd0e409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient(hunting): picked mon.noname-a con 0x7fd0e4134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40a01a0 con 0x7fd0e4134df0 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40a0020 con 0x7fd0e409c4a0 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient(hunting): start opening mon connection 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e409be20 con 0x7fd0e406c890 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient(hunting): _renew_subs 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0eb666640 10 monclient(hunting): authenticate will time out at 1265.240091s 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 0x7fd0e41cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 0x7fd0e41cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:35766/0 (socket says 10.20.193.78:35766) 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 1 -- 10.20.193.78:0/2159128544 learned_addr learned my addr 10.20.193.78:0/2159128544 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 10 monclient(hunting): get_auth_request con 0x7fd0e406c890 auth_method 0 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e4134df0 0x7fd0e409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 1 -- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 msgr2=0x7fd0e41cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 1 -- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 msgr2=0x7fd0e41cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 0x7fd0e41cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient(hunting): get_auth_request con 0x7fd0e4134df0 auth_method 0 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:56:59.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e93db640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 0x7fd0e41cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient(hunting): handle_auth_done global_id 124320 payload 824 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e8bda640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0e409c4a0 0x7fd0e41cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 1 -- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 msgr2=0x7fd0e41cf6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 0x7fd0e41cf6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 1 -- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0e409c4a0 msgr2=0x7fd0e41cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0e409c4a0 0x7fd0e41cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient: _finish_hunting 0 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 1 monclient: found mon.noname-a 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e9bdc640 1 -- 10.20.193.78:0/2159128544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e40a1730 con 0x7fd0e4134df0 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e8bda640 10 monclient: get_auth_request con 0x7fd0e409c4a0 auth_method 0 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.687+0000 7fd0e8bda640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0e409c4a0 0x7fd0e41cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0e9bdc640 1 -- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e4134df0 msgr2=0x7fd0e409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:56:59.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0e9bdc640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e4134df0 0x7fd0e409db50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0007a60 tx=0x7fd0e0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:59.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 1 -- 10.20.193.78:0/2159128544 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e001a070 con 0x7fd0e4134df0 2026-02-17T05:56:59.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: dump: 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 1 -- 10.20.193.78:0/2159128544 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e000ddb0 con 0x7fd0e4134df0 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 1 -- 10.20.193.78:0/2159128544 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0015640 con 0x7fd0e4134df0 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d1ffb640 10 monclient: dump: 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:56:59.695 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0d37fe640 4 set_mon_vals no callback set 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0eb666640 5 monclient: authenticate success, global_id 124320 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0eb666640 10 monclient: _renew_subs 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0eb666640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e40726d0 con 0x7fd0e4134df0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0eb666640 10 monclient: _renew_subs 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0eb666640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.689+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e4127250 con 0x7fd0e4134df0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0eb666640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0eb666640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0eb666640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e4074980 con 0x7fd0e4134df0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0d1ffb640 1 -- 10.20.193.78:0/2159128544 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd0e00143a0 con 0x7fd0e4134df0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0d1ffb640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0b403deb0 0x7fd0b4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0d1ffb640 1 -- 10.20.193.78:0/2159128544 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(394..394 src has 335..394) ==== 4783+0+0 (secure 0 0 0) 0x7fd0e0052500 con 0x7fd0e4134df0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0e93db640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0b403deb0 0x7fd0b4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.690+0000 7fd0e93db640 10 monclient: get_auth_request con 0x7fd0b403deb0 auth_method 0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.692+0000 7fd0d1ffb640 1 -- 10.20.193.78:0/2159128544 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd0e40a1730 con 0x7fd0e4134df0 2026-02-17T05:56:59.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.693+0000 7fd0d1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:56:59.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.693+0000 7fd0d1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:56:59.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.693+0000 7fd0e93db640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0b403deb0 0x7fd0b4040370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009660 tx=0x7fd0d4033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:56:59.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.807+0000 7fd0eb666640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:56:59.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.807+0000 7fd0eb666640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:59.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.807+0000 7fd0eb666640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:56:59.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.807+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd0e4135280 con 0x7fd0e4134df0 2026-02-17T05:56:59.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.807+0000 7fd0d1ffb640 1 -- 10.20.193.78:0/2159128544 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fd0e0010070 con 0x7fd0e4134df0 2026-02-17T05:56:59.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.807+0000 7fd0d1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:56:59.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.807+0000 7fd0d1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":116,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0b403deb0 msgr2=0x7fd0b4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0b403deb0 0x7fd0b4040370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009660 tx=0x7fd0d4033000 comp rx=0 tx=0).stop 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 10 monclient: shutdown 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e4134df0 msgr2=0x7fd0e409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e4134df0 0x7fd0e409db50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0007a60 tx=0x7fd0e0007f30 comp rx=0 tx=0).stop 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 shutdown_connections 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0e4134df0 0x7fd0e409db50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0b403deb0 0x7fd0b4040370 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0e409c4a0 0x7fd0e41cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 --2- 10.20.193.78:0/2159128544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0e406c890 0x7fd0e41cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 >> 10.20.193.78:0/2159128544 conn(0x7fd0e40a6620 msgr2=0x7fd0e40a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 shutdown_connections 2026-02-17T05:56:59.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:56:59.808+0000 7fd0eb666640 1 -- 10.20.193.78:0/2159128544 wait complete. 2026-02-17T05:57:00.554 INFO:tasks.rados.rados.0.trial115.stdout:1888: done (1 left) 2026-02-17T05:57:00.554 INFO:tasks.rados.rados.0.trial115.stdout:1889: write oid 1 current snap is 187 2026-02-17T05:57:00.555 INFO:tasks.rados.rados.0.trial115.stdout:1889: seq_num 685 ranges {712795=417335,1784636=540572,2894750=457917} 2026-02-17T05:57:00.558 INFO:tasks.rados.rados.0.trial115.stdout:1889: writing trial11545420-1 from 712795 to 1130130 tid 1 2026-02-17T05:57:00.562 INFO:tasks.rados.rados.0.trial115.stdout:1889: writing trial11545420-1 from 1784636 to 2325208 tid 2 2026-02-17T05:57:00.566 INFO:tasks.rados.rados.0.trial115.stdout:1889: writing trial11545420-1 from 2894750 to 3352667 tid 3 2026-02-17T05:57:00.566 INFO:tasks.rados.rados.0.trial115.stdout:1890: snap_remove snap 175 2026-02-17T05:57:01.557 INFO:tasks.rados.rados.0.trial115.stdout:1890: done (2 left) 2026-02-17T05:57:01.557 INFO:tasks.rados.rados.0.trial115.stdout:1891: delete oid 31 current snap is 187 2026-02-17T05:57:01.557 INFO:tasks.rados.rados.0.trial115.stdout:1886: expect (ObjNum 615 snap 161 seq_num 615) 2026-02-17T05:57:01.673 INFO:tasks.rados.rados.0.trial115.stdout:1889: finishing write tid 1 to trial11545420-1 2026-02-17T05:57:01.673 INFO:tasks.rados.rados.0.trial115.stdout:1889: finishing write tid 2 to trial11545420-1 2026-02-17T05:57:01.673 INFO:tasks.rados.rados.0.trial115.stdout:1889: finishing write tid 3 to trial11545420-1 2026-02-17T05:57:01.673 INFO:tasks.rados.rados.0.trial115.stdout:1889: finishing write tid 4 to trial11545420-1 2026-02-17T05:57:01.673 INFO:tasks.rados.rados.0.trial115.stdout:1889: finishing write tid 5 to trial11545420-1 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1889: oid 1 updating version 0 to 667 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1889: oid 1 version 667 is already newer than 664 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1889: oid 1 version 667 is already newer than 666 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1889: oid 1 version 667 is already newer than 665 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 667 (ObjNum 685 snap 187 seq_num 685) dirty exists 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1889: left oid 1 (ObjNum 685 snap 187 seq_num 685) 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1886: done (2 left) 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1889: done (1 left) 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1891: done (0 left) 2026-02-17T05:57:01.674 INFO:tasks.rados.rados.0.trial115.stdout:1892: delete oid 47 current snap is 187 2026-02-17T05:57:01.675 INFO:tasks.rados.rados.0.trial115.stdout:1892: done (0 left) 2026-02-17T05:57:01.675 INFO:tasks.rados.rados.0.trial115.stdout:1893: write oid 19 current snap is 187 2026-02-17T05:57:01.675 INFO:tasks.rados.rados.0.trial115.stdout:1893: seq_num 688 ranges {763216=643082,1843729=687899,3260666=302550} 2026-02-17T05:57:01.679 INFO:tasks.rados.rados.0.trial115.stdout:1893: writing trial11545420-19 from 763216 to 1406298 tid 1 2026-02-17T05:57:01.683 INFO:tasks.rados.rados.0.trial115.stdout:1893: writing trial11545420-19 from 1843729 to 2531628 tid 2 2026-02-17T05:57:01.688 INFO:tasks.rados.rados.0.trial115.stdout:1893: writing trial11545420-19 from 3260666 to 3563216 tid 3 2026-02-17T05:57:01.688 INFO:tasks.rados.rados.0.trial115.stdout:1894: read oid 15 snap -1 2026-02-17T05:57:01.688 INFO:tasks.rados.rados.0.trial115.stdout:1894: expect (ObjNum 631 snap 164 seq_num 631) 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1895: rollback oid 14 current snap is 187 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 165 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1893: finishing write tid 1 to trial11545420-19 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1893: finishing write tid 2 to trial11545420-19 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1896: read oid 1 snap -1 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1896: expect (ObjNum 685 snap 187 seq_num 685) 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1897: rollback oid 33 current snap is 187 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 165 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1898: read oid 20 snap -1 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1898: expect (ObjNum 660 snap 182 seq_num 660) 2026-02-17T05:57:01.689 INFO:tasks.rados.rados.0.trial115.stdout:1899: write (excl) oid 21 current snap is 187 2026-02-17T05:57:01.690 INFO:tasks.rados.rados.0.trial115.stdout:1899: seq_num 689 ranges {780023=718331,1902211=547910,3004442=575581} 2026-02-17T05:57:01.693 INFO:tasks.rados.rados.0.trial115.stdout:1899: writing trial11545420-21 from 780023 to 1498354 tid 1 2026-02-17T05:57:01.697 INFO:tasks.rados.rados.0.trial115.stdout:1899: writing trial11545420-21 from 1902211 to 2450121 tid 2 2026-02-17T05:57:01.701 INFO:tasks.rados.rados.0.trial115.stdout:1899: writing trial11545420-21 from 3004442 to 3580023 tid 3 2026-02-17T05:57:01.701 INFO:tasks.rados.rados.0.trial115.stdout:1900: write (excl) oid 48 current snap is 187 2026-02-17T05:57:01.701 INFO:tasks.rados.rados.0.trial115.stdout:1900: seq_num 690 ranges {796830=793580,2360693=691568} 2026-02-17T05:57:01.707 INFO:tasks.rados.rados.0.trial115.stdout:1900: writing trial11545420-48 from 796830 to 1590410 tid 1 2026-02-17T05:57:01.713 INFO:tasks.rados.rados.0.trial115.stdout:1900: writing trial11545420-48 from 2360693 to 3052261 tid 2 2026-02-17T05:57:01.713 INFO:tasks.rados.rados.0.trial115.stdout:1901: copy_from oid 34 from oid 40 current snap is 187 2026-02-17T05:57:01.713 INFO:tasks.rados.rados.0.trial115.stdout:1893: finishing write tid 3 to trial11545420-19 2026-02-17T05:57:01.713 INFO:tasks.rados.rados.0.trial115.stdout:1893: finishing write tid 4 to trial11545420-19 2026-02-17T05:57:01.713 INFO:tasks.rados.rados.0.trial115.stdout:1893: finishing write tid 5 to trial11545420-19 2026-02-17T05:57:01.713 INFO:tasks.rados.rados.0.trial115.stdout:1893: oid 19 updating version 0 to 1003 2026-02-17T05:57:01.713 INFO:tasks.rados.rados.0.trial115.stdout:1893: oid 19 updating version 1003 to 1004 2026-02-17T05:57:01.714 INFO:tasks.rados.rados.0.trial115.stdout:1893: oid 19 updating version 1004 to 1005 2026-02-17T05:57:01.714 INFO:tasks.rados.rados.0.trial115.stdout:1893: oid 19 updating version 1005 to 1006 2026-02-17T05:57:01.714 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1006 (ObjNum 688 snap 187 seq_num 688) dirty exists 2026-02-17T05:57:01.714 INFO:tasks.rados.rados.0.trial115.stdout:1893: left oid 19 (ObjNum 688 snap 187 seq_num 688) 2026-02-17T05:57:01.714 INFO:tasks.rados.rados.0.trial115.stdout:1894: expect (ObjNum 631 snap 164 seq_num 631) 2026-02-17T05:57:01.773 INFO:tasks.rados.rados.0.trial115.stdout:1898: expect (ObjNum 660 snap 182 seq_num 660) 2026-02-17T05:57:01.847 INFO:tasks.rados.rados.0.trial115.stdout:1896: expect (ObjNum 685 snap 187 seq_num 685) 2026-02-17T05:57:01.932 INFO:tasks.rados.rados.0.trial115.stdout:1895: finishing rollback tid 1 to trial11545420-14 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 0 (ObjNum 513 snap 0 seq_num 1333599000) dirty dne 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1897: finishing rollback tid 1 to trial11545420-33 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 613 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1899: finishing write tid 1 to trial11545420-21 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1899: finishing write tid 2 to trial11545420-21 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1899: finishing write tid 3 to trial11545420-21 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1899: finishing write tid 4 to trial11545420-21 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1893: done (8 left) 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1894: done (7 left) 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1895: done (6 left) 2026-02-17T05:57:01.933 INFO:tasks.rados.rados.0.trial115.stdout:1896: done (5 left) 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1897: done (4 left) 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1898: done (3 left) 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1902: delete oid 18 current snap is 187 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1899: finishing write tid 5 to trial11545420-21 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1899: oid 21 updating version 0 to 868 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1899: oid 21 updating version 868 to 869 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1899: oid 21 version 869 is already newer than 867 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1899: oid 21 version 869 is already newer than 866 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 869 (ObjNum 689 snap 187 seq_num 689) dirty exists 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1899: left oid 21 (ObjNum 689 snap 187 seq_num 689) 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1900: finishing write tid 1 to trial11545420-48 2026-02-17T05:57:01.934 INFO:tasks.rados.rados.0.trial115.stdout:1900: finishing write tid 2 to trial11545420-48 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:1900: finishing write tid 3 to trial11545420-48 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:1900: finishing write tid 4 to trial11545420-48 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:1900: oid 48 updating version 0 to 1008 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:1900: oid 48 version 1008 is already newer than 1007 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:1900: oid 48 updating version 1008 to 1009 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1009 (ObjNum 690 snap 187 seq_num 690) dirty exists 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:1900: left oid 48 (ObjNum 690 snap 187 seq_num 690) 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:1901: finishing copy_from racing read to trial11545420-34 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:1901: finishing copy_from to trial11545420-34 2026-02-17T05:57:01.935 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 819 (ObjNum 632 snap 164 seq_num 632) dirty exists 2026-02-17T05:57:01.936 INFO:tasks.rados.rados.0.trial115.stdout:1899: done (3 left) 2026-02-17T05:57:01.936 INFO:tasks.rados.rados.0.trial115.stdout:1900: done (2 left) 2026-02-17T05:57:01.936 INFO:tasks.rados.rados.0.trial115.stdout:1901: done (1 left) 2026-02-17T05:57:01.936 INFO:tasks.rados.rados.0.trial115.stdout:1902: done (0 left) 2026-02-17T05:57:01.936 INFO:tasks.rados.rados.0.trial115.stdout:1903: rollback oid 30 current snap is 187 2026-02-17T05:57:01.936 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 180 2026-02-17T05:57:01.937 INFO:tasks.rados.rados.0.trial115.stdout:1904: snap_remove snap 174 2026-02-17T05:57:02.816 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient: get_monmap_and_config 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient: build_initial_monmap 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient: monmap: 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:02.883337+0000 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:02.883337+0000 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:02.883 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 1 Processor -- start 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 1 -- start start 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient: init 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faf6c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient(hunting): picked mon.noname-b con 0x7faf6c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient(hunting): picked mon.noname-a con 0x7faf6c06d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf6c06daf0 con 0x7faf6c06c890 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7faf6c0a0880 con 0x7faf6c06d380 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf6c09cc80 con 0x7faf6c09c4a0 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf73b72640 10 monclient(hunting): authenticate will time out at 1268.436838s 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf720e8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:02.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf720e8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44542/0 (socket says 10.20.193.78:44542) 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 learned_addr learned my addr 10.20.193.78:0/1533611055 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf720e8640 10 monclient(hunting): get_auth_request con 0x7faf6c06d380 auth_method 0 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf720e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf720e8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf720e8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf710e6640 10 monclient(hunting): get_auth_request con 0x7faf6c09c4a0 auth_method 0 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf710e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf710e6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf710e6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.883+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.884+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.884+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.884+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.884+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.884+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.884+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:02.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.884+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:02.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:02.886+0000 7faf718e7640 1 -- 10.20.193.78:0/1533611055 >> v1:10.20.193.115:6789/0 conn(0x7faf6c06c890 legacy=0x7faf6c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 10 monclient(hunting): get_auth_request con 0x7faf6c06d380 auth_method 0 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 10 monclient(hunting): get_auth_request con 0x7faf6c09c4a0 auth_method 0 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:03.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:03.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:03.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:03.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:03.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:57:03.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:03.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:03.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.084+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:03.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:03.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf720e8640 10 monclient(hunting): get_auth_request con 0x7faf6c06d380 auth_method 0 2026-02-17T05:57:03.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf720e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:03.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf720e8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:03.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf720e8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:03.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:03.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf710e6640 10 monclient(hunting): get_auth_request con 0x7faf6c09c4a0 auth_method 0 2026-02-17T05:57:03.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf710e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:03.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf710e6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf710e6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.484+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.485+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.485+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.485+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.485+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:03.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:03.485+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:04.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.285+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:04.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.285+0000 7faf720e8640 10 monclient(hunting): get_auth_request con 0x7faf6c06d380 auth_method 0 2026-02-17T05:57:04.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.285+0000 7faf720e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:04.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.285+0000 7faf720e8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:04.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.285+0000 7faf720e8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:04.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.285+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.285+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.285+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 10 monclient(hunting): get_auth_request con 0x7faf6c09c4a0 auth_method 0 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:57:04.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:04.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:04.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:04.286+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:57:05.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 10 monclient(hunting): tick 2026-02-17T05:57:05.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:57:05.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 monclient(hunting): continuing hunt 2026-02-17T05:57:05.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:57:05.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 msgr2=0x7faf6c13b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf6c09c4a0 0x7faf6c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 msgr2=0x7faf6c13d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.78:3300/0 conn(0x7faf6c06d380 0x7faf6c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 -- 10.20.193.78:0/1533611055 >> v1:10.20.193.115:6789/0 conn(0x7faf6c06c890 legacy=0x7faf6c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 10 monclient: _add_conns ranks=[0,4] 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faf50003820 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 10 monclient(hunting): picked mon.noname-c con 0x7faf50007550 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 -- 10.20.193.78:0/1533611055 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf6c09cf60 con 0x7faf50003820 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:05.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 1 -- 10.20.193.78:0/1533611055 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7faf6c09cc80 con 0x7faf50007550 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.883+0000 7faf5b7fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 10 monclient(hunting): get_auth_request con 0x7faf50007550 auth_method 0 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 msgr2=0x7faf50009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 msgr2=0x7faf50009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:05.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.884+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:05.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:05.886+0000 7faf718e7640 1 -- 10.20.193.78:0/1533611055 >> v1:10.20.193.78:6789/0 conn(0x7faf50003820 legacy=0x7faf50005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:06.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.084+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:06.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.084+0000 7faf710e6640 10 monclient(hunting): get_auth_request con 0x7faf50007550 auth_method 0 2026-02-17T05:57:06.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.084+0000 7faf710e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:06.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.084+0000 7faf710e6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:06.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.084+0000 7faf710e6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:06.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.084+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 msgr2=0x7faf50009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:06.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.084+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 msgr2=0x7faf50009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:06.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.085+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:06.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.085+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:06.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:06.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 10 monclient(hunting): get_auth_request con 0x7faf50007550 auth_method 0 2026-02-17T05:57:06.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:06.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:06.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:06.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 msgr2=0x7faf50009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:06.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 msgr2=0x7faf50009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:06.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:06.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.485+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:06.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.940+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3360473125 0 0) 0x7faf6c09cf60 con 0x7faf50003820 2026-02-17T05:57:06.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.940+0000 7faf708e5640 10 monclient(hunting): _negotiate 2026-02-17T05:57:06.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.940+0000 7faf708e5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:06.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.940+0000 7faf708e5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:06.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.940+0000 7faf708e5640 10 monclient(hunting): my global_id is 124323 2026-02-17T05:57:06.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.940+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf54003620 con 0x7faf50003820 2026-02-17T05:57:06.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2313385626 0 0) 0x7faf54003620 con 0x7faf50003820 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 msgr2=0x7faf50009a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 --2- 10.20.193.78:0/1533611055 >> v2:10.20.193.115:3300/0 conn(0x7faf50007550 0x7faf50009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: _finish_hunting 0 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5000b1a0 con 0x7faf50003820 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: _finish_auth 0 2026-02-17T05:57:06.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: _check_auth_tickets 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7faf5c003240 con 0x7faf50003820 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: dump: 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:06.951 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 >> v1:10.20.193.78:6789/0 conn(0x7faf50003820 legacy=0x7faf50005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 1 -- 10.20.193.78:0/1533611055 reap_dead start 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf54003ba0 0x7faf54003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient(hunting): picked mon.b con 0x7faf54003ba0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:06.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54004900 0x7faf54004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient(hunting): picked mon.c con 0x7faf54004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54005510 0x7faf5400bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient(hunting): picked mon.a con 0x7faf54005510 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c09cc80 con 0x7faf54005510 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf5400c420 con 0x7faf54003ba0 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf5400c620 con 0x7faf54004900 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf708e5640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:57:06.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54005510 0x7faf5400bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 10 monclient(hunting): get_auth_request con 0x7faf54005510 auth_method 0 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54004900 0x7faf54004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient(hunting): get_auth_request con 0x7faf54004900 auth_method 0 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:06.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient(hunting): handle_auth_done global_id 124323 payload 293 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf54003ba0 msgr2=0x7faf54003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf54003ba0 0x7faf54003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54005510 msgr2=0x7faf5400bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54005510 0x7faf5400bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient: _finish_hunting 0 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 1 monclient: found mon.c 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:06.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf54003620 con 0x7faf54004900 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient: _finish_auth 0 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 10 monclient: _check_auth_tickets 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf710e6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf720e8640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54005510 0x7faf5400bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf73b72640 5 monclient: authenticate success, global_id 124323 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.941+0000 7faf73b72640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.943+0000 7faf710e6640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54004900 msgr2=0x7faf54004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.943+0000 7faf710e6640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54004900 0x7faf54004d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf60018ab0 tx=0x7faf60030a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d09b6a6dd7c7af server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6003d020 con 0x7faf54004900 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:06.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 10 monclient: dump: 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:06.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf60004220 con 0x7faf54004900 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 1 -- 10.20.193.78:0/1533611055 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6000f040 con 0x7faf54004900 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf708e5640 10 monclient: dump: 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:06.958 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 4 set_mon_vals no callback set 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 10 monclient: shutdown 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 -- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54004900 msgr2=0x7faf54004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54004900 0x7faf54004d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf60018ab0 tx=0x7faf60030a00 comp rx=0 tx=0).stop 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 -- 10.20.193.78:0/1533611055 shutdown_connections 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54005510 0x7faf5400bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54004900 0x7faf54004d80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:06.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 --2- 10.20.193.78:0/1533611055 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf54003ba0 0x7faf54003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 -- 10.20.193.78:0/1533611055 >> 10.20.193.78:0/1533611055 conn(0x7faf6c0a6620 msgr2=0x7faf6c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 -- 10.20.193.78:0/1533611055 shutdown_connections 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.944+0000 7faf73b72640 1 -- 10.20.193.78:0/1533611055 wait complete. 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient: build_initial_monmap 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient: monmap: 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:06.960 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 1 Processor -- start 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 1 -- start start 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient: init 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54003ba0 0x7faf6c1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient(hunting): picked mon.noname-a con 0x7faf54003ba0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf54004900 0x7faf6c1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient(hunting): picked mon.noname-b con 0x7faf54004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:06.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 0x7faf6c121270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient(hunting): picked mon.noname-c con 0x7faf54005510 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c09bfa0 con 0x7faf54003ba0 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c09be20 con 0x7faf54004900 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf6c06cc70 con 0x7faf54005510 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf73b72640 10 monclient(hunting): authenticate will time out at 1272.498571s 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54003ba0 0x7faf6c1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 0x7faf6c121270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:06.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 0x7faf6c121270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:35822/0 (socket says 10.20.193.78:35822) 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 1 -- 10.20.193.78:0/3820559398 learned_addr learned my addr 10.20.193.78:0/3820559398 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient(hunting): get_auth_request con 0x7faf54003ba0 auth_method 0 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 10 monclient(hunting): get_auth_request con 0x7faf54005510 auth_method 0 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:06.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 1 -- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 msgr2=0x7faf6c121270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 1 -- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 msgr2=0x7faf6c121270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 0x7faf6c121270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf720e8640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 0x7faf6c121270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient(hunting): handle_auth_done global_id 124326 payload 824 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 1 -- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 msgr2=0x7faf6c121270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 0x7faf6c121270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 1 -- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf54004900 msgr2=0x7faf6c1cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf54004900 0x7faf6c1cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient: _finish_hunting 0 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 1 monclient: found mon.noname-a 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:06.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.945+0000 7faf718e7640 1 -- 10.20.193.78:0/3820559398 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c125620 con 0x7faf54003ba0 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.947+0000 7faf718e7640 1 -- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54003ba0 msgr2=0x7faf6c1cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.947+0000 7faf718e7640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54003ba0 0x7faf6c1cd330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf5c003460 tx=0x7faf5c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.947+0000 7faf5a7fc640 1 -- 10.20.193.78:0/3820559398 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c017070 con 0x7faf54003ba0 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.947+0000 7faf5a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.947+0000 7faf5a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.947+0000 7faf5a7fc640 10 monclient: dump: 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:06.965 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 1 -- 10.20.193.78:0/3820559398 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5c005cd0 con 0x7faf54003ba0 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 5 monclient: authenticate success, global_id 124326 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 10 monclient: _renew_subs 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:06.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6c123a70 con 0x7faf54003ba0 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 1 -- 10.20.193.78:0/3820559398 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c007270 con 0x7faf54003ba0 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5bfff640 4 set_mon_vals no callback set 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 10 monclient: dump: 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:06.967 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 10 monclient: _renew_subs 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf6c09d810 con 0x7faf54003ba0 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6c074980 con 0x7faf54003ba0 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 1 -- 10.20.193.78:0/3820559398 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7faf5c007450 con 0x7faf54003ba0 2026-02-17T05:57:06.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf3c03deb0 0x7faf3c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:06.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.948+0000 7faf5a7fc640 1 -- 10.20.193.78:0/3820559398 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(396..396 src has 335..396) ==== 4843+0+0 (secure 0 0 0) 0x7faf5c051f50 con 0x7faf54003ba0 2026-02-17T05:57:06.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.949+0000 7faf710e6640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf3c03deb0 0x7faf3c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:06.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.949+0000 7faf710e6640 10 monclient: get_auth_request con 0x7faf3c03deb0 auth_method 0 2026-02-17T05:57:06.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.951+0000 7faf5a7fc640 1 -- 10.20.193.78:0/3820559398 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7faf6c125620 con 0x7faf54003ba0 2026-02-17T05:57:06.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.951+0000 7faf5a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:06.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.951+0000 7faf5a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:06.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:06.951+0000 7faf710e6640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf3c03deb0 0x7faf3c040370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf60006fb0 tx=0x7faf60037000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:06.994 INFO:tasks.rados.rados.0.trial115.stdout:1904: done (1 left) 2026-02-17T05:57:06.994 INFO:tasks.rados.rados.0.trial115.stdout:1905: write (excl) oid 49 current snap is 187 2026-02-17T05:57:06.994 INFO:tasks.rados.rados.0.trial115.stdout:1903: finishing rollback tid 1 to trial11545420-30 2026-02-17T05:57:06.994 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 731 (ObjNum 49 snap 0 seq_num 1330675568) dirty dne 2026-02-17T05:57:06.994 INFO:tasks.rados.rados.0.trial115.stdout:1905: seq_num 692 ranges {430444=660431,1677657=695237,2970358=569527} 2026-02-17T05:57:06.998 INFO:tasks.rados.rados.0.trial115.stdout:1905: writing trial11545420-49 from 430444 to 1090875 tid 1 2026-02-17T05:57:07.003 INFO:tasks.rados.rados.0.trial115.stdout:1905: writing trial11545420-49 from 1677657 to 2372894 tid 2 2026-02-17T05:57:07.007 INFO:tasks.rados.rados.0.trial115.stdout:1905: writing trial11545420-49 from 2970358 to 3539885 tid 3 2026-02-17T05:57:07.007 INFO:tasks.rados.rados.0.trial115.stdout:1903: done (1 left) 2026-02-17T05:57:07.007 INFO:tasks.rados.rados.0.trial115.stdout:1906: delete oid 47 current snap is 187 2026-02-17T05:57:07.008 INFO:tasks.rados.rados.0.trial115.stdout:1905: finishing write tid 1 to trial11545420-49 2026-02-17T05:57:07.008 INFO:tasks.rados.rados.0.trial115.stdout:1905: finishing write tid 2 to trial11545420-49 2026-02-17T05:57:07.009 INFO:tasks.rados.rados.0.trial115.stdout:1905: finishing write tid 3 to trial11545420-49 2026-02-17T05:57:07.009 INFO:tasks.rados.rados.0.trial115.stdout:1905: finishing write tid 4 to trial11545420-49 2026-02-17T05:57:07.009 INFO:tasks.rados.rados.0.trial115.stdout:1905: finishing write tid 5 to trial11545420-49 2026-02-17T05:57:07.009 INFO:tasks.rados.rados.0.trial115.stdout:1905: oid 49 updating version 0 to 823 2026-02-17T05:57:07.010 INFO:tasks.rados.rados.0.trial115.stdout:1905: oid 49 version 823 is already newer than 822 2026-02-17T05:57:07.010 INFO:tasks.rados.rados.0.trial115.stdout:1905: oid 49 version 823 is already newer than 820 2026-02-17T05:57:07.010 INFO:tasks.rados.rados.0.trial115.stdout:1905: oid 49 version 823 is already newer than 821 2026-02-17T05:57:07.010 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 823 (ObjNum 692 snap 187 seq_num 692) dirty exists 2026-02-17T05:57:07.010 INFO:tasks.rados.rados.0.trial115.stdout:1905: left oid 49 (ObjNum 692 snap 187 seq_num 692) 2026-02-17T05:57:07.010 INFO:tasks.rados.rados.0.trial115.stdout:1905: done (1 left) 2026-02-17T05:57:07.010 INFO:tasks.rados.rados.0.trial115.stdout:1906: done (0 left) 2026-02-17T05:57:07.010 INFO:tasks.rados.rados.0.trial115.stdout:1907: snap_create 2026-02-17T05:57:07.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.065+0000 7faf73b72640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:07.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.065+0000 7faf73b72640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:07.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.065+0000 7faf73b72640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:07.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.065+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7faf6c06cfa0 con 0x7faf54003ba0 2026-02-17T05:57:07.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.065+0000 7faf5a7fc640 1 -- 10.20.193.78:0/3820559398 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7faf5c011de0 con 0x7faf54003ba0 2026-02-17T05:57:07.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.065+0000 7faf5a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:07.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.065+0000 7faf5a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:07.066 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":120,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:07.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf3c03deb0 msgr2=0x7faf3c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf3c03deb0 0x7faf3c040370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf60006fb0 tx=0x7faf60037000 comp rx=0 tx=0).stop 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 10 monclient: shutdown 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54003ba0 msgr2=0x7faf6c1cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54003ba0 0x7faf6c1cd330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf5c003460 tx=0x7faf5c00cba0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 shutdown_connections 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf54005510 0x7faf6c121270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf3c03deb0 0x7faf3c040370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf54004900 0x7faf6c1cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 --2- 10.20.193.78:0/3820559398 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf54003ba0 0x7faf6c1cd330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:07.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.066+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 >> 10.20.193.78:0/3820559398 conn(0x7faf6c0a6620 msgr2=0x7faf6c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:07.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.067+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 shutdown_connections 2026-02-17T05:57:07.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:07.067+0000 7faf73b72640 1 -- 10.20.193.78:0/3820559398 wait complete. 2026-02-17T05:57:07.995 INFO:tasks.rados.rados.0.trial115.stdout:1907: done (0 left) 2026-02-17T05:57:07.995 INFO:tasks.rados.rados.0.trial115.stdout:1908: delete oid 19 current snap is 188 2026-02-17T05:57:07.997 INFO:tasks.rados.rados.0.trial115.stdout:1908: done (0 left) 2026-02-17T05:57:07.997 INFO:tasks.rados.rados.0.trial115.stdout:1909: snap_create 2026-02-17T05:57:08.551 INFO:tasks.rados.rados.0.trial115.stdout:1909: done (0 left) 2026-02-17T05:57:08.551 INFO:tasks.rados.rados.0.trial115.stdout:1910: copy_from oid 23 from oid 49 current snap is 189 2026-02-17T05:57:08.551 INFO:tasks.rados.rados.0.trial115.stdout:1911: copy_from oid 38 from oid 22 current snap is 189 2026-02-17T05:57:08.551 INFO:tasks.rados.rados.0.trial115.stdout:1912: read oid 5 snap -1 2026-02-17T05:57:08.551 INFO:tasks.rados.rados.0.trial115.stdout:1912: expect (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:57:08.551 INFO:tasks.rados.rados.0.trial115.stdout:1913: read oid 42 snap -1 2026-02-17T05:57:08.551 INFO:tasks.rados.rados.0.trial115.stdout:1913: expect (ObjNum 641 snap 169 seq_num 641) 2026-02-17T05:57:08.552 INFO:tasks.rados.rados.0.trial115.stdout:1914: rollback oid 32 current snap is 189 2026-02-17T05:57:08.552 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 185 2026-02-17T05:57:08.552 INFO:tasks.rados.rados.0.trial115.stdout:1915: write oid 46 current snap is 189 2026-02-17T05:57:08.552 INFO:tasks.rados.rados.0.trial115.stdout:1915: seq_num 695 ranges {480865=486178,1453103=442564,2652627=519696} 2026-02-17T05:57:08.554 INFO:tasks.rados.rados.0.trial115.stdout:1915: writing trial11545420-46 from 480865 to 967043 tid 1 2026-02-17T05:57:08.558 INFO:tasks.rados.rados.0.trial115.stdout:1915: writing trial11545420-46 from 1453103 to 1895667 tid 2 2026-02-17T05:57:08.563 INFO:tasks.rados.rados.0.trial115.stdout:1915: writing trial11545420-46 from 2652627 to 3172323 tid 3 2026-02-17T05:57:08.564 INFO:tasks.rados.rados.0.trial115.stdout:1916: read oid 12 snap 186 2026-02-17T05:57:08.564 INFO:tasks.rados.rados.0.trial115.stdout:1916: expect (ObjNum 672 snap 186 seq_num 672) 2026-02-17T05:57:08.564 INFO:tasks.rados.rados.0.trial115.stdout:1917: write oid 41 current snap is 189 2026-02-17T05:57:08.564 INFO:tasks.rados.rados.0.trial115.stdout:1917: seq_num 696 ranges {497672=561427,1795232=702575,2963697=730968} 2026-02-17T05:57:08.567 INFO:tasks.rados.rados.0.trial115.stdout:1917: writing trial11545420-41 from 497672 to 1059099 tid 1 2026-02-17T05:57:08.572 INFO:tasks.rados.rados.0.trial115.stdout:1917: writing trial11545420-41 from 1795232 to 2497807 tid 2 2026-02-17T05:57:08.577 INFO:tasks.rados.rados.0.trial115.stdout:1917: writing trial11545420-41 from 2963697 to 3694665 tid 3 2026-02-17T05:57:08.577 INFO:tasks.rados.rados.0.trial115.stdout:1918: write (excl) oid 39 current snap is 189 2026-02-17T05:57:08.577 INFO:tasks.rados.rados.0.trial115.stdout:1918: seq_num 697 ranges {514479=636676,1853714=446233,2991120=542240} 2026-02-17T05:57:08.581 INFO:tasks.rados.rados.0.trial115.stdout:1918: writing trial11545420-39 from 514479 to 1151155 tid 1 2026-02-17T05:57:08.586 INFO:tasks.rados.rados.0.trial115.stdout:1918: writing trial11545420-39 from 1853714 to 2299947 tid 2 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:1918: writing trial11545420-39 from 2991120 to 3533360 tid 3 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:1919: read oid 4 snap -1 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:1919: expect deleted 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:1920: rollback oid 3 current snap is 189 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 3 to 172 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:1914: finishing rollback tid 0 to trial11545420-32 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:1914: finishing rollback tid 1 to trial11545420-32 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:1914: finishing rollback tid 2 to trial11545420-32 2026-02-17T05:57:08.591 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 867 (ObjNum 642 snap 170 seq_num 642) dirty exists 2026-02-17T05:57:08.592 INFO:tasks.rados.rados.0.trial115.stdout:1912: expect (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:57:08.652 INFO:tasks.rados.rados.0.trial115.stdout:1913: expect (ObjNum 641 snap 169 seq_num 641) 2026-02-17T05:57:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1915: finishing write tid 1 to trial11545420-46 2026-02-17T05:57:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1915: finishing write tid 2 to trial11545420-46 2026-02-17T05:57:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1911: finishing copy_from racing read to trial11545420-38 2026-02-17T05:57:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1912: done (10 left) 2026-02-17T05:57:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1913: done (9 left) 2026-02-17T05:57:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1914: done (8 left) 2026-02-17T05:57:08.782 INFO:tasks.rados.rados.0.trial115.stdout:1921: read oid 28 snap -1 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1921: expect (ObjNum 660 snap 182 seq_num 660) 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1922: read oid 26 snap -1 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1922: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1923: copy_from oid 17 from oid 5 current snap is 189 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1911: finishing copy_from to trial11545420-38 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 777 (ObjNum 675 snap 186 seq_num 675) dirty exists 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1910: finishing copy_from racing read to trial11545420-23 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1915: finishing write tid 3 to trial11545420-46 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1915: finishing write tid 4 to trial11545420-46 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1915: finishing write tid 5 to trial11545420-46 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1915: oid 46 updating version 0 to 917 2026-02-17T05:57:08.783 INFO:tasks.rados.rados.0.trial115.stdout:1915: oid 46 updating version 917 to 918 2026-02-17T05:57:08.784 INFO:tasks.rados.rados.0.trial115.stdout:1915: oid 46 version 918 is already newer than 916 2026-02-17T05:57:08.784 INFO:tasks.rados.rados.0.trial115.stdout:1915: oid 46 updating version 918 to 919 2026-02-17T05:57:08.784 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 919 (ObjNum 695 snap 189 seq_num 695) dirty exists 2026-02-17T05:57:08.784 INFO:tasks.rados.rados.0.trial115.stdout:1915: left oid 46 (ObjNum 695 snap 189 seq_num 695) 2026-02-17T05:57:08.784 INFO:tasks.rados.rados.0.trial115.stdout:1910: finishing copy_from to trial11545420-23 2026-02-17T05:57:08.784 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 823 (ObjNum 692 snap 187 seq_num 692) dirty exists 2026-02-17T05:57:08.784 INFO:tasks.rados.rados.0.trial115.stdout:1916: expect (ObjNum 672 snap 186 seq_num 672) 2026-02-17T05:57:08.911 INFO:tasks.rados.rados.0.trial115.stdout:1917: finishing write tid 1 to trial11545420-41 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1917: finishing write tid 2 to trial11545420-41 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1917: finishing write tid 3 to trial11545420-41 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1910: done (10 left) 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1911: done (9 left) 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1915: done (8 left) 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1916: done (7 left) 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1924: snap_create 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1917: finishing write tid 4 to trial11545420-41 2026-02-17T05:57:08.912 INFO:tasks.rados.rados.0.trial115.stdout:1917: finishing write tid 5 to trial11545420-41 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1917: oid 41 updating version 0 to 871 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1917: oid 41 updating version 871 to 872 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1917: oid 41 updating version 872 to 874 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1917: oid 41 version 874 is already newer than 873 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 874 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1917: left oid 41 (ObjNum 696 snap 189 seq_num 696) 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1918: finishing write tid 1 to trial11545420-39 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1918: finishing write tid 2 to trial11545420-39 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1918: finishing write tid 3 to trial11545420-39 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1918: finishing write tid 4 to trial11545420-39 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1918: finishing write tid 5 to trial11545420-39 2026-02-17T05:57:08.913 INFO:tasks.rados.rados.0.trial115.stdout:1918: oid 39 updating version 0 to 1109 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:1918: oid 39 version 1109 is already newer than 1106 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:1918: oid 39 version 1109 is already newer than 1107 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:1918: oid 39 version 1109 is already newer than 1108 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1109 (ObjNum 697 snap 189 seq_num 697) dirty exists 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:1918: left oid 39 (ObjNum 697 snap 189 seq_num 697) 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:1920: finishing rollback tid 0 to trial11545420-3 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:1920: finishing rollback tid 1 to trial11545420-3 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:1920: finishing rollback tid 2 to trial11545420-3 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 908 (ObjNum 598 snap 155 seq_num 598) dirty exists 2026-02-17T05:57:08.914 INFO:tasks.rados.rados.0.trial115.stdout:1922: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:57:08.988 INFO:tasks.rados.rados.0.trial115.stdout:1923: finishing copy_from racing read to trial11545420-17 2026-02-17T05:57:08.988 INFO:tasks.rados.rados.0.trial115.stdout:1923: finishing copy_from to trial11545420-17 2026-02-17T05:57:08.988 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 811 (ObjNum 516 snap 132 seq_num 516) dirty exists 2026-02-17T05:57:08.989 INFO:tasks.rados.rados.0.trial115.stdout:1921: expect (ObjNum 660 snap 182 seq_num 660) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1917: done (7 left) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1918: done (6 left) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1919: done (5 left) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1920: done (4 left) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1921: done (3 left) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1922: done (2 left) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1923: done (1 left) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1924: done (0 left) 2026-02-17T05:57:09.553 INFO:tasks.rados.rados.0.trial115.stdout:1925: write oid 16 current snap is 190 2026-02-17T05:57:09.554 INFO:tasks.rados.rados.0.trial115.stdout:1925: seq_num 698 ranges {531286=711925,1912196=706244,3018543=712743} 2026-02-17T05:57:09.557 INFO:tasks.rados.rados.0.trial115.stdout:1925: writing trial11545420-16 from 531286 to 1243211 tid 1 2026-02-17T05:57:09.562 INFO:tasks.rados.rados.0.trial115.stdout:1925: writing trial11545420-16 from 1912196 to 2618440 tid 2 2026-02-17T05:57:09.566 INFO:tasks.rados.rados.0.trial115.stdout:1925: writing trial11545420-16 from 3018543 to 3731286 tid 3 2026-02-17T05:57:09.566 INFO:tasks.rados.rados.0.trial115.stdout:1926: copy_from oid 40 from oid 9 current snap is 190 2026-02-17T05:57:09.566 INFO:tasks.rados.rados.0.trial115.stdout:1925: finishing write tid 1 to trial11545420-16 2026-02-17T05:57:09.566 INFO:tasks.rados.rados.0.trial115.stdout:1925: finishing write tid 2 to trial11545420-16 2026-02-17T05:57:09.567 INFO:tasks.rados.rados.0.trial115.stdout:1927: write oid 13 current snap is 190 2026-02-17T05:57:09.567 INFO:tasks.rados.rados.0.trial115.stdout:1927: seq_num 699 ranges {548093=787174,1970678=449902,3045966=681137} 2026-02-17T05:57:09.571 INFO:tasks.rados.rados.0.trial115.stdout:1927: writing trial11545420-13 from 548093 to 1335267 tid 1 2026-02-17T05:57:09.575 INFO:tasks.rados.rados.0.trial115.stdout:1927: writing trial11545420-13 from 1970678 to 2420580 tid 2 2026-02-17T05:57:09.580 INFO:tasks.rados.rados.0.trial115.stdout:1927: writing trial11545420-13 from 3045966 to 3727103 tid 3 2026-02-17T05:57:09.580 INFO:tasks.rados.rados.0.trial115.stdout:1928: snap_create 2026-02-17T05:57:09.580 INFO:tasks.rados.rados.0.trial115.stdout:1925: finishing write tid 3 to trial11545420-16 2026-02-17T05:57:09.580 INFO:tasks.rados.rados.0.trial115.stdout:1925: finishing write tid 4 to trial11545420-16 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1925: finishing write tid 5 to trial11545420-16 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1925: oid 16 updating version 0 to 1020 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1925: oid 16 updating version 1020 to 1021 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1925: oid 16 version 1021 is already newer than 1018 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1925: oid 16 version 1021 is already newer than 1019 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1021 (ObjNum 698 snap 190 seq_num 698) dirty exists 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1925: left oid 16 (ObjNum 698 snap 190 seq_num 698) 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1926: finishing copy_from racing read to trial11545420-40 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1926: finishing copy_from to trial11545420-40 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 886 (ObjNum 674 snap 186 seq_num 674) dirty exists 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1927: finishing write tid 1 to trial11545420-13 2026-02-17T05:57:09.581 INFO:tasks.rados.rados.0.trial115.stdout:1927: finishing write tid 2 to trial11545420-13 2026-02-17T05:57:09.582 INFO:tasks.rados.rados.0.trial115.stdout:1927: finishing write tid 3 to trial11545420-13 2026-02-17T05:57:09.582 INFO:tasks.rados.rados.0.trial115.stdout:1927: finishing write tid 4 to trial11545420-13 2026-02-17T05:57:09.583 INFO:tasks.rados.rados.0.trial115.stdout:1927: finishing write tid 5 to trial11545420-13 2026-02-17T05:57:09.583 INFO:tasks.rados.rados.0.trial115.stdout:1927: oid 13 updating version 0 to 642 2026-02-17T05:57:09.583 INFO:tasks.rados.rados.0.trial115.stdout:1927: oid 13 updating version 642 to 643 2026-02-17T05:57:09.583 INFO:tasks.rados.rados.0.trial115.stdout:1927: oid 13 version 643 is already newer than 641 2026-02-17T05:57:09.583 INFO:tasks.rados.rados.0.trial115.stdout:1927: oid 13 version 643 is already newer than 640 2026-02-17T05:57:09.583 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 643 (ObjNum 699 snap 190 seq_num 699) dirty exists 2026-02-17T05:57:09.583 INFO:tasks.rados.rados.0.trial115.stdout:1927: left oid 13 (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:57:10.076 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:10.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient: get_monmap_and_config 2026-02-17T05:57:10.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient: build_initial_monmap 2026-02-17T05:57:10.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient: monmap: 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:10.145452+0000 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:10.145452+0000 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:10.148 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 1 Processor -- start 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 1 -- start start 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient: init 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faeac06c890 0x7faeac0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient(hunting): picked mon.noname-a con 0x7faeac06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faeac06cc70 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faeac13a240 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac074150 con 0x7faeac06cc70 2026-02-17T05:57:10.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac09ef70 con 0x7faeac13a240 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7faeac09f250 con 0x7faeac06c890 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.145+0000 7faeb24ca640 10 monclient(hunting): authenticate will time out at 1275.698980s 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faeac06c890 0x7faeac0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeab7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7faeac06cc70 0x7faeac06d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37134/0 (socket says 10.20.193.78:37134) 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faeac06c890 0x7faeac0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48604/0 (socket says 10.20.193.78:48604) 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeab7fe640 1 -- 10.20.193.78:0/3186252667 learned_addr learned my addr 10.20.193.78:0/3186252667 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient(hunting): get_auth_request con 0x7faeac06c890 auth_method 0 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient(hunting): handle_auth_done global_id 124329 payload 274 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 1 -- 10.20.193.78:0/3186252667 >> v1:10.20.193.115:6789/0 conn(0x7faeac13a240 legacy=0x7faeac13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 1 -- 10.20.193.78:0/3186252667 >> v1:10.20.193.78:6789/0 conn(0x7faeac06cc70 legacy=0x7faeac06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 1 monclient: found mon.noname-a 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.146+0000 7faeabfff640 1 -- 10.20.193.78:0/3186252667 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac09df40 con 0x7faeac06c890 2026-02-17T05:57:10.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeabfff640 1 -- 10.20.193.78:0/3186252667 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 msgr2=0x7faeac0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:10.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeabfff640 1 --2- 10.20.193.78:0/3186252667 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac0a01f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae980098a0 tx=0x7fae9802ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c902441e4533b1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 1 -- 10.20.193.78:0/3186252667 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9803c070 con 0x7faeac06c890 2026-02-17T05:57:10.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:10.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: dump: 2026-02-17T05:57:10.158 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: _finish_auth 0 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:10.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 1 -- 10.20.193.78:0/3186252667 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae9802fd70 con 0x7faeac06c890 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeb24ca640 5 monclient: authenticate success, global_id 124329 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeb24ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeb24ca640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 1 -- 10.20.193.78:0/3186252667 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98037660 con 0x7faeac06c890 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeb24ca640 4 set_mon_vals no callback set 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.148+0000 7faeaaffd640 10 monclient: dump: 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.160 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 10 monclient: shutdown 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 -- 10.20.193.78:0/3186252667 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 msgr2=0x7faeac0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 --2- 10.20.193.78:0/3186252667 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac0a01f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae980098a0 tx=0x7fae9802ee10 comp rx=0 tx=0).stop 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 -- 10.20.193.78:0/3186252667 shutdown_connections 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 --2- 10.20.193.78:0/3186252667 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac0a01f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 -- 10.20.193.78:0/3186252667 >> 10.20.193.78:0/3186252667 conn(0x7faeac0a6620 msgr2=0x7faeac0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 -- 10.20.193.78:0/3186252667 shutdown_connections 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 -- 10.20.193.78:0/3186252667 wait complete. 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 10 monclient: build_initial_monmap 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 10 monclient: monmap: 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:10.162 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 Processor -- start 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 1 -- start start 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.149+0000 7faeb24ca640 10 monclient: init 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac135a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient(hunting): picked mon.noname-a con 0x7faeac06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faeac06cc70 0x7faeac13dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient(hunting): picked mon.noname-b con 0x7faeac06cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 0x7faeac13e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient(hunting): picked mon.noname-c con 0x7faeac13a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faeac09bfa0 con 0x7faeac06c890 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faeac09be20 con 0x7faeac06cc70 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faeac09c120 con 0x7faeac13a240 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb24ca640 10 monclient(hunting): authenticate will time out at 1275.703091s 2026-02-17T05:57:10.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac135a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac135a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48612/0 (socket says 10.20.193.78:48612) 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 -- 10.20.193.78:0/1142693819 learned_addr learned my addr 10.20.193.78:0/1142693819 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient(hunting): get_auth_request con 0x7faeac06c890 auth_method 0 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 0x7faeac13e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 10 monclient(hunting): get_auth_request con 0x7faeac13a240 auth_method 0 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient(hunting): handle_auth_done global_id 124332 payload 824 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 1 -- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 msgr2=0x7faeac13e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 1 -- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 msgr2=0x7faeac13e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 -- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 msgr2=0x7faeac13e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 0x7faeac13e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeab7fe640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faeac06cc70 0x7faeac13dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeb0a40640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 0x7faeac13e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 0x7faeac13e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 -- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faeac06cc70 msgr2=0x7faeac13dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faeac06cc70 0x7faeac13dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 monclient: found mon.noname-a 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeabfff640 1 -- 10.20.193.78:0/1142693819 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac1342c0 con 0x7faeac06c890 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeab7fe640 10 monclient: get_auth_request con 0x7faeac06cc70 auth_method 0 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.150+0000 7faeab7fe640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faeac06cc70 0x7faeac13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeabfff640 1 -- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 msgr2=0x7faeac135a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeabfff640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac135a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fae980098a0 tx=0x7fae980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 1 -- 10.20.193.78:0/1142693819 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98046070 con 0x7faeac06c890 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: dump: 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: _finish_auth 0 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 1 -- 10.20.193.78:0/1142693819 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae98041660 con 0x7faeac06c890 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeb24ca640 5 monclient: authenticate success, global_id 124332 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 1 -- 10.20.193.78:0/1142693819 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9803c040 con 0x7faeac06c890 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeb24ca640 10 monclient: _renew_subs 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeaa7fc640 4 set_mon_vals no callback set 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeb24ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac13f7d0 con 0x7faeac06c890 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faea8ff9640 10 monclient: dump: 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeb24ca640 10 monclient: _renew_subs 2026-02-17T05:57:10.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeb24ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.152+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac0a24f0 con 0x7faeac06c890 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faeb24ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faeb24ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faeb24ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac0a01d0 con 0x7faeac06c890 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faea8ff9640 1 -- 10.20.193.78:0/1142693819 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fae98041a50 con 0x7faeac06c890 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faea8ff9640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae7c03deb0 0x7fae7c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faea8ff9640 1 -- 10.20.193.78:0/1142693819 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(400..400 src has 335..400) ==== 4735+0+0 (secure 0 0 0) 0x7fae98078030 con 0x7faeac06c890 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faeab7fe640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae7c03deb0 0x7fae7c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.153+0000 7faeab7fe640 10 monclient: get_auth_request con 0x7fae7c03deb0 auth_method 0 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.155+0000 7faea8ff9640 1 -- 10.20.193.78:0/1142693819 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fae98040c70 con 0x7faeac06c890 2026-02-17T05:57:10.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.155+0000 7faea8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:10.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.155+0000 7faea8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:10.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.156+0000 7faeab7fe640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae7c03deb0 0x7fae7c040370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faeac1344d0 tx=0x7fae9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.273+0000 7faeb24ca640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:10.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.273+0000 7faeb24ca640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:10.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.273+0000 7faeb24ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.273+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7faeac13f550 con 0x7faeac06c890 2026-02-17T05:57:10.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faea8ff9640 1 -- 10.20.193.78:0/1142693819 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fae98041d70 con 0x7faeac06c890 2026-02-17T05:57:10.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faea8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:10.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faea8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:10.275 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:10.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae7c03deb0 msgr2=0x7fae7c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae7c03deb0 0x7fae7c040370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faeac1344d0 tx=0x7fae9c009040 comp rx=0 tx=0).stop 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 10 monclient: shutdown 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 msgr2=0x7faeac135a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac135a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fae980098a0 tx=0x7fae980047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 shutdown_connections 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faeac13a240 0x7faeac13e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae7c03deb0 0x7fae7c040370 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.274+0000 7faeb24ca640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faeac06cc70 0x7faeac13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.275+0000 7faeb24ca640 1 --2- 10.20.193.78:0/1142693819 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faeac06c890 0x7faeac135a60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.275+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 >> 10.20.193.78:0/1142693819 conn(0x7faeac0a6620 msgr2=0x7faeac0a0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:10.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.275+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 shutdown_connections 2026-02-17T05:57:10.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.275+0000 7faeb24ca640 1 -- 10.20.193.78:0/1142693819 wait complete. 2026-02-17T05:57:10.282 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:57:10.282 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:57:10.282 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:57:10.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.350+0000 7f348b334640 10 monclient: get_monmap_and_config 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.350+0000 7f348b334640 10 monclient: build_initial_monmap 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.350+0000 7f348b334640 10 monclient: monmap: 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:10.350793+0000 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:10.350793+0000 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:10.351 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:10.352 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:10.352 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:10.352 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 1 Processor -- start 2026-02-17T05:57:10.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 1 -- start start 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient: init 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient: _add_conns ranks=[4,0,3,2,1] 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f348409ea90 0x7f34840a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient(hunting): picked mon.noname-c con 0x7f348409ea90 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f348409ee70 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f348413d640 0x7f348413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient(hunting): picked mon.noname-a con 0x7f348413d640 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34840a0a20 con 0x7f348409ee70 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f34840a0d00 con 0x7f348413d640 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f34840a0460 con 0x7f348409ea90 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f348b334640 10 monclient(hunting): authenticate will time out at 1275.904557s 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f348413d640 0x7f348413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f348413d640 0x7f348413fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48618/0 (socket says 10.20.193.78:48618) 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34888a8640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f348409ee70 0x7f348409f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37142/0 (socket says 10.20.193.78:37142) 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 1 -- 10.20.193.78:0/652429116 learned_addr learned my addr 10.20.193.78:0/652429116 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient(hunting): get_auth_request con 0x7f348413d640 auth_method 0 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34890a9640 1 --2- 10.20.193.78:0/652429116 >> v2:10.20.193.115:3300/0 conn(0x7f348409ea90 0x7f34840a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34890a9640 10 monclient(hunting): get_auth_request con 0x7f348409ea90 auth_method 0 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34890a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34890a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34890a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient(hunting): handle_auth_done global_id 124335 payload 274 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 1 -- 10.20.193.78:0/652429116 >> v2:10.20.193.115:3300/0 conn(0x7f348409ea90 msgr2=0x7f34840a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 1 --2- 10.20.193.78:0/652429116 >> v2:10.20.193.115:3300/0 conn(0x7f348409ea90 0x7f34840a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 1 -- 10.20.193.78:0/652429116 >> v1:10.20.193.78:6789/0 conn(0x7f348409ee70 legacy=0x7f348409f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:10.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient: _finish_hunting 0 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 1 monclient: found mon.noname-a 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.351+0000 7f34898aa640 1 -- 10.20.193.78:0/652429116 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34840a0740 con 0x7f348413d640 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f34898aa640 1 -- 10.20.193.78:0/652429116 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348413d640 msgr2=0x7f348413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f34898aa640 1 --2- 10.20.193.78:0/652429116 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348413d640 0x7f348413fa30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f348406ecd0 tx=0x7f348002f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7eb710c8505780 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 1 -- 10.20.193.78:0/652429116 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348002fe60 con 0x7f348413d640 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: dump: 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.365 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: _finish_auth 0 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:10.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 1 -- 10.20.193.78:0/652429116 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34800373f0 con 0x7f348413d640 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 5 monclient: authenticate success, global_id 124335 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 4 set_mon_vals no callback set 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 1 -- 10.20.193.78:0/652429116 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800377c0 con 0x7f348413d640 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f3473fff640 10 monclient: dump: 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.367 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 10 monclient: shutdown 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 1 -- 10.20.193.78:0/652429116 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348413d640 msgr2=0x7f348413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 1 --2- 10.20.193.78:0/652429116 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348413d640 0x7f348413fa30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f348406ecd0 tx=0x7f348002f230 comp rx=0 tx=0).stop 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 1 -- 10.20.193.78:0/652429116 shutdown_connections 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 1 --2- 10.20.193.78:0/652429116 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348413d640 0x7f348413fa30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 1 --2- 10.20.193.78:0/652429116 >> v2:10.20.193.115:3300/0 conn(0x7f348409ea90 0x7f34840a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 1 -- 10.20.193.78:0/652429116 >> 10.20.193.78:0/652429116 conn(0x7f34840a7520 msgr2=0x7f34840a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 1 -- 10.20.193.78:0/652429116 shutdown_connections 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.354+0000 7f348b334640 1 -- 10.20.193.78:0/652429116 wait complete. 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient: build_initial_monmap 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient: monmap: 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:10.369 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 1 Processor -- start 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 1 -- start start 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient: init 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f348409ea90 0x7f348413b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient(hunting): picked mon.noname-b con 0x7f348409ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348409ee70 0x7f348413bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient(hunting): picked mon.noname-a con 0x7f348409ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:10.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f348413d640 0x7f348413c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient(hunting): picked mon.noname-c con 0x7f348413d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34840a6820 con 0x7f348409ee70 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f34840a66a0 con 0x7f348409ea90 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f348413ff70 con 0x7f348413d640 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f348b334640 10 monclient(hunting): authenticate will time out at 1275.908674s 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348409ee70 0x7f348413bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348409ee70 0x7f348413bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48620/0 (socket says 10.20.193.78:48620) 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 -- 10.20.193.78:0/4017874863 learned_addr learned my addr 10.20.193.78:0/4017874863 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:10.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient(hunting): get_auth_request con 0x7f348409ee70 auth_method 0 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f348413d640 0x7f348413c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 10 monclient(hunting): get_auth_request con 0x7f348413d640 auth_method 0 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f348413d640 msgr2=0x7f348413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f348413d640 msgr2=0x7f348413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f348413d640 0x7f348413c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient(hunting): handle_auth_done global_id 124338 payload 824 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f348413d640 msgr2=0x7f348413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34898aa640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f348413d640 0x7f348413c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f348413d640 0x7f348413c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f348409ea90 msgr2=0x7f348413b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f348409ea90 0x7f348413b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient: _finish_hunting 0 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 monclient: found mon.noname-a 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:10.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.355+0000 7f34888a8640 1 -- 10.20.193.78:0/4017874863 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3484070de0 con 0x7f348409ee70 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.357+0000 7f34888a8640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348409ee70 msgr2=0x7f348413bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.357+0000 7f34888a8640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348409ee70 0x7f348413bea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f347800c550 tx=0x7f347800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478017070 con 0x7f348409ee70 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: dump: 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.374 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3478007070 con 0x7f348409ee70 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 5 monclient: authenticate success, global_id 124338 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:10.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478007460 con 0x7f348409ee70 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34737fe640 4 set_mon_vals no callback set 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 10 monclient: dump: 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.376 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 10 monclient: _renew_subs 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34840a9b70 con 0x7f348409ee70 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 10 monclient: _renew_subs 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3484070b00 con 0x7f348409ee70 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348406ef90 0x7f34841cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3484070960 con 0x7f348406ef90 2026-02-17T05:57:10.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348406ef90 0x7f34841cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 10 monclient: get_auth_request con 0x7f348406ef90 auth_method 0 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 10 monclient(hunting): handle_auth_done global_id 124341 payload 824 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f34890a9640 1 -- 10.20.193.78:0/4017874863 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34840a13d0 con 0x7f348406ef90 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.358+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f347801b3b0 con 0x7f348409ee70 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.359+0000 7f3471ffb640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f345403deb0 0x7f3454040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.359+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(400..400 src has 335..400) ==== 4735+0+0 (secure 0 0 0) 0x7f3478051ff0 con 0x7f348409ee70 2026-02-17T05:57:10.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.359+0000 7f34898aa640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f345403deb0 0x7f3454040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.359+0000 7f34898aa640 10 monclient: get_auth_request con 0x7f345403deb0 auth_method 0 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.360+0000 7f34890a9640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348406ef90 msgr2=0x7f34841cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.360+0000 7f34890a9640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348406ef90 0x7f34841cf0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f347400e740 tx=0x7f347400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.360+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474017070 con 0x7f348406ef90 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.360+0000 7f3471ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.360+0000 7f3471ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.360+0000 7f3471ffb640 10 monclient: dump: 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.379 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.361+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f347400cb50 con 0x7f348406ef90 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.361+0000 7f3471ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.361+0000 7f3471ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.361+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348406ef90 msgr2=0x7f34841cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.361+0000 7f3471ffb640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348406ef90 0x7f34841cf0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f347400e740 tx=0x7f347400ec10 comp rx=0 tx=0).stop 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.361+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348406ef90 msgr2=0x7f34841cf0d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:10.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.361+0000 7f3471ffb640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348406ef90 0x7f34841cf0d0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f347400e740 tx=0x7f347400ec10 comp rx=0 tx=0).stop 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.362+0000 7f34898aa640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f345403deb0 0x7f3454040370 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f348406e270 tx=0x7f3480033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f348b334640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f348b334640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34841cf610 0x7f34841da420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f348b334640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34840744d0 con 0x7f34841cf610 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34841cf610 0x7f34841da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 10 monclient: get_auth_request con 0x7f34841cf610 auth_method 0 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 10 monclient(hunting): handle_auth_done global_id 124344 payload 824 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.364+0000 7f34890a9640 1 -- 10.20.193.78:0/4017874863 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f34841cfa20 con 0x7f34841cf610 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f34890a9640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34841cf610 msgr2=0x7f34841da420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f34890a9640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34841cf610 0x7f34841da420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3474018000 tx=0x7f34740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347400f040 con 0x7f34841cf610 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 10 monclient: dump: 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.384 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f3474003db0 con 0x7f34841cf610 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34841cf610 msgr2=0x7f34841da420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34841cf610 0x7f34841da420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3474018000 tx=0x7f34740047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34841cf610 msgr2=0x7f34841da420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:10.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f3471ffb640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f34841cf610 0x7f34841da420 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3474018000 tx=0x7f34740047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 122, 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 121832, 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:57:10.386 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:10.387 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:10.388 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:57:10.389 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:57:10.390 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:57:10.391 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:10.392 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:57:10.393 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:10.394 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:57:10.395 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:10.396 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:57:10.397 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:57:10.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f345403deb0 msgr2=0x7f3454040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f348b334640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f345403deb0 0x7f3454040370 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f348406e270 tx=0x7f3480033000 comp rx=0 tx=0).stop 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f348b334640 10 monclient: shutdown 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f348b334640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.366+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348409ee70 msgr2=0x7f348413bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.367+0000 7f348b334640 1 --2- 10.20.193.78:0/4017874863 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f348409ee70 0x7f348413bea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f347800c550 tx=0x7f347800ca20 comp rx=0 tx=0).stop 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.367+0000 7f34898aa640 1 -- 10.20.193.78:0/4017874863 reap_dead start 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.367+0000 7f34898aa640 1 -- 10.20.193.78:0/4017874863 reap_dead start 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.367+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 shutdown_connections 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.367+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 >> 10.20.193.78:0/4017874863 conn(0x7f34840a7520 msgr2=0x7f3484136dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.367+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 shutdown_connections 2026-02-17T05:57:10.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.367+0000 7f348b334640 1 -- 10.20.193.78:0/4017874863 wait complete. 2026-02-17T05:57:10.399 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:57:10.400 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:57:10.400 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:57:10.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.467+0000 7f5ae7779640 10 monclient: get_monmap_and_config 2026-02-17T05:57:10.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.467+0000 7f5ae7779640 10 monclient: build_initial_monmap 2026-02-17T05:57:10.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.467+0000 7f5ae7779640 10 monclient: monmap: 2026-02-17T05:57:10.469 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:10.469 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.469 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:10.467543+0000 2026-02-17T05:57:10.469 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:10.467543+0000 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.467+0000 7f5ae7779640 1 Processor -- start 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.467+0000 7f5ae7779640 1 -- start start 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.467+0000 7f5ae7779640 10 monclient: init 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5ae006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:10.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5ae009c4a0 0x7f5ae013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient(hunting): picked mon.noname-c con 0x7f5ae009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5ae006d380 0x7f5ae013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient(hunting): picked mon.noname-a con 0x7f5ae006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae006daf0 con 0x7f5ae006c890 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae00a0880 con 0x7f5ae006d380 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae009cc80 con 0x7f5ae009c4a0 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae7779640 10 monclient(hunting): authenticate will time out at 1276.021090s 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5ae006d380 0x7f5ae013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae54ee640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f5ae006c890 0x7f5ae006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37156/0 (socket says 10.20.193.78:37156) 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae54ee640 1 -- 10.20.193.78:0/1447844627 learned_addr learned my addr 10.20.193.78:0/1447844627 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient(hunting): get_auth_request con 0x7f5ae006d380 auth_method 0 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1447844627 >> v2:10.20.193.115:3300/0 conn(0x7f5ae009c4a0 0x7f5ae013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae4ced640 10 monclient(hunting): get_auth_request con 0x7f5ae009c4a0 auth_method 0 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae4ced640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae4ced640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae4ced640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient(hunting): handle_auth_done global_id 124347 payload 274 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 1 -- 10.20.193.78:0/1447844627 >> v2:10.20.193.115:3300/0 conn(0x7f5ae009c4a0 msgr2=0x7f5ae013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 1 --2- 10.20.193.78:0/1447844627 >> v2:10.20.193.115:3300/0 conn(0x7f5ae009c4a0 0x7f5ae013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 1 -- 10.20.193.78:0/1447844627 >> v1:10.20.193.78:6789/0 conn(0x7f5ae006c890 legacy=0x7f5ae006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient: _finish_hunting 0 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 1 monclient: found mon.noname-a 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.468+0000 7f5ae5cef640 1 -- 10.20.193.78:0/1447844627 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae009cf60 con 0x7f5ae006d380 2026-02-17T05:57:10.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5ae5cef640 1 -- 10.20.193.78:0/1447844627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006d380 msgr2=0x7f5ae013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5ae5cef640 1 --2- 10.20.193.78:0/1447844627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006d380 0x7f5ae013d8f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ae006dcc0 tx=0x7f5adc033660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74ae36b4e098f5f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 1 -- 10.20.193.78:0/1447844627 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc007950 con 0x7f5ae006d380 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: dump: 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: _finish_auth 0 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 1 -- 10.20.193.78:0/1447844627 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5adc007af0 con 0x7f5ae006d380 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5ae7779640 5 monclient: authenticate success, global_id 124347 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 1 -- 10.20.193.78:0/1447844627 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc03a400 con 0x7f5ae006d380 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5ae7779640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5ae7779640 4 set_mon_vals no callback set 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.470+0000 7f5acffff640 10 monclient: dump: 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.479 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 10 monclient: shutdown 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 -- 10.20.193.78:0/1447844627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006d380 msgr2=0x7f5ae013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 --2- 10.20.193.78:0/1447844627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006d380 0x7f5ae013d8f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ae006dcc0 tx=0x7f5adc033660 comp rx=0 tx=0).stop 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 -- 10.20.193.78:0/1447844627 shutdown_connections 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 --2- 10.20.193.78:0/1447844627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006d380 0x7f5ae013d8f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 --2- 10.20.193.78:0/1447844627 >> v2:10.20.193.115:3300/0 conn(0x7f5ae009c4a0 0x7f5ae013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 -- 10.20.193.78:0/1447844627 >> 10.20.193.78:0/1447844627 conn(0x7f5ae00a6620 msgr2=0x7f5ae00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 -- 10.20.193.78:0/1447844627 shutdown_connections 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 -- 10.20.193.78:0/1447844627 wait complete. 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 10 monclient: build_initial_monmap 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 10 monclient: monmap: 2026-02-17T05:57:10.480 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.471+0000 7f5ae7779640 1 Processor -- start 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 1 -- start start 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient: init 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006c890 0x7f5ae0133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient(hunting): picked mon.noname-a con 0x7f5ae006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:10.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ae006d380 0x7f5ae01342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient(hunting): picked mon.noname-b con 0x7f5ae006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae009c4a0 0x7f5ae011f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient(hunting): picked mon.noname-c con 0x7f5ae009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae009bfa0 con 0x7f5ae006c890 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae009be20 con 0x7f5ae006d380 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae009db20 con 0x7f5ae009c4a0 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae7779640 10 monclient(hunting): authenticate will time out at 1276.025272s 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006c890 0x7f5ae0133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006c890 0x7f5ae0133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48650/0 (socket says 10.20.193.78:48650) 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 -- 10.20.193.78:0/1280062078 learned_addr learned my addr 10.20.193.78:0/1280062078 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient(hunting): get_auth_request con 0x7f5ae006c890 auth_method 0 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae009c4a0 0x7f5ae011f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 10 monclient(hunting): get_auth_request con 0x7f5ae009c4a0 auth_method 0 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae009c4a0 msgr2=0x7f5ae011f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae009c4a0 msgr2=0x7f5ae011f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae009c4a0 0x7f5ae011f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae5cef640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae009c4a0 0x7f5ae011f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient(hunting): handle_auth_done global_id 124350 payload 824 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae009c4a0 msgr2=0x7f5ae011f7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae009c4a0 0x7f5ae011f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ae006d380 msgr2=0x7f5ae01342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:10.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5ae006d380 0x7f5ae01342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient: _finish_hunting 0 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 monclient: found mon.noname-a 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.472+0000 7f5ae54ee640 1 -- 10.20.193.78:0/1280062078 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae0121fc0 con 0x7f5ae006c890 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae54ee640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006c890 msgr2=0x7f5ae0133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae54ee640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006c890 0x7f5ae0133d70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000c750 tx=0x7f5ad000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0005de0 con 0x7f5ae006c890 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: dump: 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.484 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad0007070 con 0x7f5ae006c890 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae7779640 5 monclient: authenticate success, global_id 124350 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae7779640 10 monclient: _renew_subs 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae7779640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae00a1640 con 0x7f5ae006c890 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0007440 con 0x7f5ae006c890 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:10.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acf7fe640 4 set_mon_vals no callback set 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5acdffb640 10 monclient: dump: 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae7779640 10 monclient: _renew_subs 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae7779640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.474+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae0123190 con 0x7f5ae006c890 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae7779640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae7779640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae7779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae0074980 con 0x7f5ae011fd00 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 10 monclient: get_auth_request con 0x7f5ae011fd00 auth_method 0 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5ad0007660 con 0x7f5ae006c890 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5acdffb640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ab003deb0 0x7f5ab0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(400..400 src has 335..400) ==== 4735+0+0 (secure 0 0 0) 0x7f5ad00526a0 con 0x7f5ae006c890 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae5cef640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ab003deb0 0x7f5ab0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.475+0000 7f5ae5cef640 10 monclient: get_auth_request con 0x7f5ab003deb0 auth_method 0 2026-02-17T05:57:10.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.478+0000 7f5ae5cef640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ab003deb0 0x7f5ab0040370 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ae006a480 tx=0x7f5adc038000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:10.557 INFO:tasks.rados.rados.0.trial115.stdout:1925: done (3 left) 2026-02-17T05:57:10.557 INFO:tasks.rados.rados.0.trial115.stdout:1926: done (2 left) 2026-02-17T05:57:10.557 INFO:tasks.rados.rados.0.trial115.stdout:1927: done (1 left) 2026-02-17T05:57:10.557 INFO:tasks.rados.rados.0.trial115.stdout:1928: done (0 left) 2026-02-17T05:57:10.558 INFO:tasks.rados.rados.0.trial115.stdout:1929: copy_from oid 11 from oid 5 current snap is 191 2026-02-17T05:57:10.558 INFO:tasks.rados.rados.0.trial115.stdout:1930: rollback oid 29 current snap is 191 2026-02-17T05:57:10.558 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 172 2026-02-17T05:57:10.558 INFO:tasks.rados.rados.0.trial115.stdout:1931: read oid 42 snap -1 2026-02-17T05:57:10.558 INFO:tasks.rados.rados.0.trial115.stdout:1931: expect (ObjNum 641 snap 169 seq_num 641) 2026-02-17T05:57:10.558 INFO:tasks.rados.rados.0.trial115.stdout:1932: snap_create 2026-02-17T05:57:10.559 INFO:tasks.rados.rados.0.trial115.stdout:1930: finishing rollback tid 1 to trial11545420-29 2026-02-17T05:57:10.559 INFO:tasks.rados.rados.0.trial115.stdout:1930: finishing rollback tid 2 to trial11545420-29 2026-02-17T05:57:10.559 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 645 (ObjNum 536 snap 136 seq_num 536) dirty exists 2026-02-17T05:57:10.563 INFO:tasks.rados.rados.0.trial115.stdout:1931: expect (ObjNum 641 snap 169 seq_num 641) 2026-02-17T05:57:10.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:10.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 10 monclient: get_auth_request con 0x7f5ae011fd00 auth_method 0 2026-02-17T05:57:10.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:10.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:10.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:10.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:57:10.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:10.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:10.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:10.675+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:10.692 INFO:tasks.rados.rados.0.trial115.stdout:1929: finishing copy_from racing read to trial11545420-11 2026-02-17T05:57:10.692 INFO:tasks.rados.rados.0.trial115.stdout:1929: finishing copy_from to trial11545420-11 2026-02-17T05:57:10.692 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 811 (ObjNum 516 snap 132 seq_num 516) dirty exists 2026-02-17T05:57:11.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 10 monclient: get_auth_request con 0x7f5ae011fd00 auth_method 0 2026-02-17T05:57:11.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:11.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:57:11.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:11.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:11.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.076+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:1929: done (3 left) 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:1930: done (2 left) 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:1931: done (1 left) 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:1932: done (0 left) 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:1933: rollback oid 21 current snap is 192 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 21 to 139 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:1934: read oid 35 snap -1 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:1934: expect (ObjNum 654 snap 177 seq_num 654) 2026-02-17T05:57:11.557 INFO:tasks.rados.rados.0.trial115.stdout:1935: snap_create 2026-02-17T05:57:11.565 INFO:tasks.rados.rados.0.trial115.stdout:1934: expect (ObjNum 654 snap 177 seq_num 654) 2026-02-17T05:57:11.703 INFO:tasks.rados.rados.0.trial115.stdout:1933: finishing rollback tid 0 to trial11545420-21 2026-02-17T05:57:11.703 INFO:tasks.rados.rados.0.trial115.stdout:1933: finishing rollback tid 1 to trial11545420-21 2026-02-17T05:57:11.703 INFO:tasks.rados.rados.0.trial115.stdout:1933: finishing rollback tid 2 to trial11545420-21 2026-02-17T05:57:11.703 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 877 (ObjNum 464 snap 117 seq_num 464) dirty exists 2026-02-17T05:57:11.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 10 monclient: get_auth_request con 0x7f5ae011fd00 auth_method 0 2026-02-17T05:57:11.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:11.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:11.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:11.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:11.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 10 monclient(hunting): handle_auth_done global_id 144101 payload 824 2026-02-17T05:57:11.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.877+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ae006cc70 con 0x7f5ae011fd00 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0122de0 tx=0x7f5ad4014a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad401b030 con 0x7f5ae011fd00 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5acdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5acdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5acdffb640 10 monclient: dump: 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:11.880 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f5ad4009a70 con 0x7f5ae011fd00 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5acdffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.879+0000 7f5acdffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.880+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.880+0000 7f5acdffb640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0122de0 tx=0x7f5ad4014a90 comp rx=0 tx=0).stop 2026-02-17T05:57:11.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.880+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 msgr2=0x7f5ae01200e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:11.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.880+0000 7f5acdffb640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae011fd00 0x7f5ae01200e0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0122de0 tx=0x7f5ad4014a90 comp rx=0 tx=0).stop 2026-02-17T05:57:11.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.883+0000 7f5ae7779640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:57:11.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.883+0000 7f5ae7779640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae0120620 0x7f5ae01d6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.883+0000 7f5ae7779640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:11.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.883+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae006cf00 con 0x7f5ae0120620 2026-02-17T05:57:11.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae0120620 0x7f5ae01d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 10 monclient: get_auth_request con 0x7f5ae0120620 auth_method 0 2026-02-17T05:57:11.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:11.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:11.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:11.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:11.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 10 monclient(hunting): handle_auth_done global_id 144104 payload 824 2026-02-17T05:57:11.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.884+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5ae00a3f40 con 0x7f5ae0120620 2026-02-17T05:57:11.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae4ced640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae0120620 msgr2=0x7f5ae01d6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:11.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae4ced640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae0120620 0x7f5ae01d6310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40122e0 tx=0x7f5ad4006b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad402e020 con 0x7f5ae0120620 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 10 monclient: dump: 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:11.888 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f5ad40163d0 con 0x7f5ae0120620 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae0120620 msgr2=0x7f5ae01d6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae0120620 0x7f5ae01d6310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40122e0 tx=0x7f5ad4006b20 comp rx=0 tx=0).stop 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae0120620 msgr2=0x7f5ae01d6310 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:11.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5acdffb640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5ae0120620 0x7f5ae01d6310 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40122e0 tx=0x7f5ad4006b20 comp rx=0 tx=0).stop 2026-02-17T05:57:11.890 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:57:11.890 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:11.890 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:11.890 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:57:11.890 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 122, 2026-02-17T05:57:11.890 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 15421, 2026-02-17T05:57:11.890 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:11.891 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:11.892 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:11.893 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:57:11.894 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:11.895 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:11.896 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:11.897 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:11.898 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:11.899 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:11.900 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:57:11.901 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:11.902 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:57:11.903 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:57:11.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ab003deb0 msgr2=0x7f5ab0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ab003deb0 0x7f5ab0040370 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ae006a480 tx=0x7f5adc038000 comp rx=0 tx=0).stop 2026-02-17T05:57:11.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 10 monclient: shutdown 2026-02-17T05:57:11.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:11.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006c890 msgr2=0x7f5ae0133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 1 --2- 10.20.193.78:0/1280062078 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5ae006c890 0x7f5ae0133d70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000c750 tx=0x7f5ad000cc20 comp rx=0 tx=0).stop 2026-02-17T05:57:11.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae5cef640 1 -- 10.20.193.78:0/1280062078 reap_dead start 2026-02-17T05:57:11.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae5cef640 1 -- 10.20.193.78:0/1280062078 reap_dead start 2026-02-17T05:57:11.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 shutdown_connections 2026-02-17T05:57:11.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 >> 10.20.193.78:0/1280062078 conn(0x7f5ae00a6620 msgr2=0x7f5ae0136560 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:11.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 shutdown_connections 2026-02-17T05:57:11.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.886+0000 7f5ae7779640 1 -- 10.20.193.78:0/1280062078 wait complete. 2026-02-17T05:57:11.907 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:57:11.907 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:57:11.907 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:57:11.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.975+0000 7f2658146640 10 monclient: get_monmap_and_config 2026-02-17T05:57:11.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.975+0000 7f2658146640 10 monclient: build_initial_monmap 2026-02-17T05:57:11.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.975+0000 7f2658146640 10 monclient: monmap: 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:11.975673+0000 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:11.975673+0000 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:11.976 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:11.977 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.975+0000 7f2658146640 1 Processor -- start 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.975+0000 7f2658146640 1 -- start start 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.975+0000 7f2658146640 10 monclient: init 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f265006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f265006d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f26500a01f0 0x7f265013d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 10 monclient(hunting): picked mon.noname-c con 0x7f26500a01f0 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26500a0600 con 0x7f265006c890 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2650074150 con 0x7f265006d380 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f265009ef70 con 0x7f26500a01f0 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2658146640 10 monclient(hunting): authenticate will time out at 1277.529208s 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2655ebb640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f265006c890 0x7f265006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37162/0 (socket says 10.20.193.78:37162) 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f2655ebb640 1 -- 10.20.193.78:0/681543060 learned_addr learned my addr 10.20.193.78:0/681543060 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 1 --2- 10.20.193.78:0/681543060 >> v2:10.20.193.115:3300/0 conn(0x7f26500a01f0 0x7f265013d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient(hunting): get_auth_request con 0x7f26500a01f0 auth_method 0 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient(hunting): handle_auth_done global_id 134188 payload 274 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 1 -- 10.20.193.78:0/681543060 >> v1:10.20.193.115:6789/0 conn(0x7f265006d380 legacy=0x7f265013a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 1 -- 10.20.193.78:0/681543060 >> v1:10.20.193.78:6789/0 conn(0x7f265006c890 legacy=0x7f265006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient: _finish_hunting 0 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 1 monclient: found mon.noname-c 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:11.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.976+0000 7f26566bc640 1 -- 10.20.193.78:0/681543060 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265009f250 con 0x7f26500a01f0 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f26566bc640 1 -- 10.20.193.78:0/681543060 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26500a01f0 msgr2=0x7f265013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f26566bc640 1 --2- 10.20.193.78:0/681543060 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26500a01f0 0x7f265013d970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f264c009d90 tx=0x7f264c02f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5e8f5ae0b474dd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 1 -- 10.20.193.78:0/681543060 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c03c070 con 0x7f26500a01f0 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: dump: 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:11.990 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: _finish_auth 0 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: _check_auth_tickets 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 5 monclient: authenticate success, global_id 134188 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 1 -- 10.20.193.78:0/681543060 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c037640 con 0x7f26500a01f0 2026-02-17T05:57:11.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 1 -- 10.20.193.78:0/681543060 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c037a10 con 0x7f26500a01f0 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 4 set_mon_vals no callback set 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2654eb9640 10 monclient: dump: 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:11.992 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 10 monclient: shutdown 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 1 -- 10.20.193.78:0/681543060 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26500a01f0 msgr2=0x7f265013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 1 --2- 10.20.193.78:0/681543060 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26500a01f0 0x7f265013d970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f264c009d90 tx=0x7f264c02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 1 -- 10.20.193.78:0/681543060 shutdown_connections 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 1 --2- 10.20.193.78:0/681543060 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26500a01f0 0x7f265013d970 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 1 -- 10.20.193.78:0/681543060 >> 10.20.193.78:0/681543060 conn(0x7f26500a6620 msgr2=0x7f26500a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:11.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 1 -- 10.20.193.78:0/681543060 shutdown_connections 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.979+0000 7f2658146640 1 -- 10.20.193.78:0/681543060 wait complete. 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient: build_initial_monmap 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient: monmap: 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:11.994 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 1 Processor -- start 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 1 -- start start 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient: init 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f265006c890 0x7f26501327a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient(hunting): picked mon.noname-c con 0x7f265006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006d380 0x7f265013c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient(hunting): picked mon.noname-b con 0x7f265006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26500a01f0 0x7f265006eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:11.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient(hunting): picked mon.noname-a con 0x7f26500a01f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f265009d360 con 0x7f26500a01f0 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f265009d1e0 con 0x7f265006d380 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f265013deb0 con 0x7f265006c890 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2658146640 10 monclient(hunting): authenticate will time out at 1277.533690s 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26500a01f0 0x7f265006eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f265006c890 0x7f26501327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26500a01f0 0x7f265006eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48664/0 (socket says 10.20.193.78:48664) 2026-02-17T05:57:11.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f265006c890 0x7f26501327a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41478/0 (socket says 10.20.193.78:41478) 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 -- 10.20.193.78:0/2377283790 learned_addr learned my addr 10.20.193.78:0/2377283790 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient(hunting): get_auth_request con 0x7f26500a01f0 auth_method 0 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 10 monclient(hunting): get_auth_request con 0x7f265006c890 auth_method 0 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:11.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26556ba640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006d380 0x7f265013c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26556ba640 10 monclient(hunting): get_auth_request con 0x7f265006d380 auth_method 0 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26556ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26556ba640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26556ba640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient(hunting): handle_auth_done global_id 124353 payload 824 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f265006c890 msgr2=0x7f26501327a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f265006c890 0x7f26501327a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006d380 msgr2=0x7f265013c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006d380 0x7f265013c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient: _finish_hunting 0 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 monclient: found mon.noname-a 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f26566bc640 1 -- 10.20.193.78:0/2377283790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26500a27f0 con 0x7f26500a01f0 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.980+0000 7f2655ebb640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f265006c890 0x7f26501327a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f26566bc640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26500a01f0 msgr2=0x7f265006eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f26566bc640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26500a01f0 0x7f265006eae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f264c009d90 tx=0x7f264c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c047020 con 0x7f26500a01f0 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:11.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: dump: 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c042680 con 0x7f26500a01f0 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 5 monclient: authenticate success, global_id 124353 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 10 monclient: _renew_subs 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265009d810 con 0x7f26500a01f0 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c03c040 con 0x7f26500a01f0 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:12.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 10 monclient: dump: 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263ffff640 4 set_mon_vals no callback set 2026-02-17T05:57:12.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 10 monclient: _renew_subs 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265013ae30 con 0x7f26500a01f0 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006f020 0x7f265006f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f265006f940 con 0x7f265006f020 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2655ebb640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006f020 0x7f265006f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2655ebb640 10 monclient: get_auth_request con 0x7f265006f020 auth_method 0 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2655ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2655ebb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2655ebb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f264c042af0 con 0x7f26500a01f0 2026-02-17T05:57:12.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.983+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.984+0000 7f2655ebb640 10 monclient(hunting): handle_auth_done global_id 134194 payload 824 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.984+0000 7f2655ebb640 1 -- 10.20.193.78:0/2377283790 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f265006cc70 con 0x7f265006f020 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.984+0000 7f263e7fc640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f262003deb0 0x7f2620040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.984+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(402..402 src has 335..402) ==== 4735+0+0 (secure 0 0 0) 0x7f264c078610 con 0x7f26500a01f0 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.984+0000 7f26556ba640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f262003deb0 0x7f2620040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.984+0000 7f26556ba640 10 monclient: get_auth_request con 0x7f262003deb0 auth_method 0 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f2655ebb640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006f020 msgr2=0x7f265006f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f2655ebb640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006f020 0x7f265006f400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2640007400 tx=0x7f26400078d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264000fdc0 con 0x7f265006f020 2026-02-17T05:57:12.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 10 monclient: dump: 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:12.005 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f264000fdc0 con 0x7f265006f020 2026-02-17T05:57:12.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:12.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:12.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006f020 msgr2=0x7f265006f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006f020 0x7f265006f400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2640007400 tx=0x7f26400078d0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006f020 msgr2=0x7f265006f400 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:12.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f263e7fc640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f265006f020 0x7f265006f400 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2640007400 tx=0x7f26400078d0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.986+0000 7f26556ba640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f262003deb0 0x7f2620040370 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2650070260 tx=0x7f2644009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:12.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2658146640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:57:12.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2658146640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2650071bc0 0x7f26501d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2658146640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:12.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f265013d3b0 con 0x7f2650071bc0 2026-02-17T05:57:12.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2650071bc0 0x7f26501d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 10 monclient: get_auth_request con 0x7f2650071bc0 auth_method 0 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 10 monclient(hunting): handle_auth_done global_id 134197 payload 824 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.990+0000 7f2655ebb640 1 -- 10.20.193.78:0/2377283790 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f26500744d0 con 0x7f2650071bc0 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2655ebb640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2650071bc0 msgr2=0x7f26501d9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2655ebb640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2650071bc0 0x7f26501d9d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2640007400 tx=0x7f2640004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640006240 con 0x7f2650071bc0 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:12.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 10 monclient: dump: 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f2640006240 con 0x7f2650071bc0 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:12.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2650071bc0 msgr2=0x7f26501d9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2650071bc0 0x7f26501d9d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2640007400 tx=0x7f2640004a40 comp rx=0 tx=0).stop 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2650071bc0 msgr2=0x7f26501d9d80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f263e7fc640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2650071bc0 0x7f26501d9d80 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2640007400 tx=0x7f2640004a40 comp rx=0 tx=0).stop 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 122, 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 59711, 2026-02-17T05:57:12.011 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:12.012 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:12.013 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:57:12.014 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:12.015 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:57:12.016 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:57:12.017 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:12.018 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:12.019 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:12.020 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:57:12.021 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:57:12.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f262003deb0 msgr2=0x7f2620040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f262003deb0 0x7f2620040370 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2650070260 tx=0x7f2644009040 comp rx=0 tx=0).stop 2026-02-17T05:57:12.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 10 monclient: shutdown 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26500a01f0 msgr2=0x7f265006eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 1 --2- 10.20.193.78:0/2377283790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26500a01f0 0x7f265006eae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f264c009d90 tx=0x7f264c004b40 comp rx=0 tx=0).stop 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f26566bc640 1 -- 10.20.193.78:0/2377283790 reap_dead start 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f26566bc640 1 -- 10.20.193.78:0/2377283790 reap_dead start 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 shutdown_connections 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 >> 10.20.193.78:0/2377283790 conn(0x7f26500a6620 msgr2=0x7f265009be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 shutdown_connections 2026-02-17T05:57:12.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:11.993+0000 7f2658146640 1 -- 10.20.193.78:0/2377283790 wait complete. 2026-02-17T05:57:12.023 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:57:12.023 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:57:12.023 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:57:12.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.090+0000 7ffa69b03640 10 monclient: get_monmap_and_config 2026-02-17T05:57:12.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.090+0000 7ffa69b03640 10 monclient: build_initial_monmap 2026-02-17T05:57:12.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.090+0000 7ffa69b03640 10 monclient: monmap: 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:12.090966+0000 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:12.090966+0000 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:12.091 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 1 Processor -- start 2026-02-17T05:57:12.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 1 -- start start 2026-02-17T05:57:12.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient: init 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ffa6406c890 0x7ffa6406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient(hunting): picked mon.noname-a con 0x7ffa6406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffa6409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6406d1b0 0x7ffa64135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient(hunting): picked mon.noname-b con 0x7ffa6406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa6406daf0 con 0x7ffa6409c4a0 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ffa640a0880 con 0x7ffa6406c890 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6409cc80 con 0x7ffa6406d1b0 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa69b03640 10 monclient(hunting): authenticate will time out at 1277.644526s 2026-02-17T05:57:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ffa6406c890 0x7ffa6406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa62ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ffa6409c4a0 0x7ffa64134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37172/0 (socket says 10.20.193.78:37172) 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa62ffd640 1 -- 10.20.193.78:0/2820298288 learned_addr learned my addr 10.20.193.78:0/2820298288 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient(hunting): get_auth_request con 0x7ffa6406c890 auth_method 0 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 1 --2- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6406d1b0 0x7ffa64135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 10 monclient(hunting): get_auth_request con 0x7ffa6406d1b0 auth_method 0 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient(hunting): handle_auth_done global_id 124356 payload 274 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 1 -- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6406d1b0 msgr2=0x7ffa64135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 1 --2- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6406d1b0 0x7ffa64135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 1 -- 10.20.193.78:0/2820298288 >> v1:10.20.193.78:6789/0 conn(0x7ffa6409c4a0 legacy=0x7ffa64134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 1 monclient: found mon.noname-a 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa637fe640 1 -- 10.20.193.78:0/2820298288 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6409cf60 con 0x7ffa6406c890 2026-02-17T05:57:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.091+0000 7ffa63fff640 1 --2- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6406d1b0 0x7ffa64135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:57:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa637fe640 1 -- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406c890 msgr2=0x7ffa6406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa637fe640 1 --2- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406c890 0x7ffa6406cc70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa6406dcc0 tx=0x7ffa5002ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad138b8b082222c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 1 -- 10.20.193.78:0/2820298288 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5003c070 con 0x7ffa6406c890 2026-02-17T05:57:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: dump: 2026-02-17T05:57:12.101 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 5 monclient: authenticate success, global_id 124356 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 1 -- 10.20.193.78:0/2820298288 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5002fbf0 con 0x7ffa6406c890 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 1 -- 10.20.193.78:0/2820298288 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50037440 con 0x7ffa6406c890 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 4 set_mon_vals no callback set 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa627fc640 10 monclient: dump: 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:12.103 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 10 monclient: shutdown 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 1 -- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406c890 msgr2=0x7ffa6406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406c890 0x7ffa6406cc70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa6406dcc0 tx=0x7ffa5002ed70 comp rx=0 tx=0).stop 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 1 -- 10.20.193.78:0/2820298288 shutdown_connections 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6406d1b0 0x7ffa64135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2820298288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406c890 0x7ffa6406cc70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 1 -- 10.20.193.78:0/2820298288 >> 10.20.193.78:0/2820298288 conn(0x7ffa640a6620 msgr2=0x7ffa640a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 1 -- 10.20.193.78:0/2820298288 shutdown_connections 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.094+0000 7ffa69b03640 1 -- 10.20.193.78:0/2820298288 wait complete. 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient: build_initial_monmap 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient: monmap: 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:12.104 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 1 Processor -- start 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 1 -- start start 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient: init 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffa6406c890 0x7ffa64133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient(hunting): picked mon.noname-b con 0x7ffa6406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406d1b0 0x7ffa64134510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient(hunting): picked mon.noname-a con 0x7ffa6406d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6409c4a0 0x7ffa641c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient(hunting): picked mon.noname-c con 0x7ffa6409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640a01a0 con 0x7ffa6406d1b0 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640a0020 con 0x7ffa6406c890 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6409be20 con 0x7ffa6409c4a0 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa69b03640 10 monclient(hunting): authenticate will time out at 1277.648598s 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406d1b0 0x7ffa64134510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6409c4a0 0x7ffa641c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406d1b0 0x7ffa64134510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48676/0 (socket says 10.20.193.78:48676) 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 -- 10.20.193.78:0/2842247544 learned_addr learned my addr 10.20.193.78:0/2842247544 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient(hunting): get_auth_request con 0x7ffa6406d1b0 auth_method 0 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6409c4a0 0x7ffa641c0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41490/0 (socket says 10.20.193.78:41490) 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 10 monclient(hunting): get_auth_request con 0x7ffa6409c4a0 auth_method 0 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa637fe640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffa6406c890 0x7ffa64133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient(hunting): handle_auth_done global_id 124359 payload 824 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 -- 10.20.193.78:0/2842247544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6409c4a0 msgr2=0x7ffa641c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6409c4a0 0x7ffa641c0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 -- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffa6406c890 msgr2=0x7ffa64133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffa6406c890 0x7ffa64133fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 monclient: found mon.noname-a 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa62ffd640 1 -- 10.20.193.78:0/2842247544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa640a2610 con 0x7ffa6406d1b0 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa637fe640 10 monclient: get_auth_request con 0x7ffa6406c890 auth_method 0 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.095+0000 7ffa637fe640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffa6406c890 0x7ffa64133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.097+0000 7ffa62ffd640 1 -- 10.20.193.78:0/2842247544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406d1b0 msgr2=0x7ffa64134510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.097+0000 7ffa62ffd640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406d1b0 0x7ffa64134510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c750 tx=0x7ffa5400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 1 -- 10.20.193.78:0/2842247544 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54005de0 con 0x7ffa6406d1b0 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: dump: 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.108 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: _finish_auth 0 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 1 -- 10.20.193.78:0/2842247544 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa54007070 con 0x7ffa6406d1b0 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 1 -- 10.20.193.78:0/2842247544 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54007440 con 0x7ffa6406d1b0 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 5 monclient: authenticate success, global_id 124359 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 10 monclient: _renew_subs 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa61ffb640 4 set_mon_vals no callback set 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa64121180 con 0x7ffa6406d1b0 2026-02-17T05:57:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 10 monclient: dump: 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 10 monclient: _renew_subs 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa64121480 con 0x7ffa6406d1b0 2026-02-17T05:57:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa64074980 con 0x7ffa6406d1b0 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.098+0000 7ffa3ffff640 1 -- 10.20.193.78:0/2842247544 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ffa54007620 con 0x7ffa6406d1b0 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.099+0000 7ffa3ffff640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffa3403df00 0x7ffa340403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.099+0000 7ffa3ffff640 1 -- 10.20.193.78:0/2842247544 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(402..402 src has 335..402) ==== 4735+0+0 (secure 0 0 0) 0x7ffa54052570 con 0x7ffa6406d1b0 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.099+0000 7ffa637fe640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffa3403df00 0x7ffa340403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.099+0000 7ffa637fe640 10 monclient: get_auth_request con 0x7ffa3403df00 auth_method 0 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.101+0000 7ffa3ffff640 1 -- 10.20.193.78:0/2842247544 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ffa640a2610 con 0x7ffa6406d1b0 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.101+0000 7ffa3ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.101+0000 7ffa3ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.101+0000 7ffa637fe640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffa3403df00 0x7ffa340403c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa6409c980 tx=0x7ffa50033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:12.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.216+0000 7ffa69b03640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:57:12.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.216+0000 7ffa69b03640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:57:12.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.216+0000 7ffa69b03640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:12.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.216+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7ffa64135ad0 con 0x7ffa6406d1b0 2026-02-17T05:57:12.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.216+0000 7ffa3ffff640 1 -- 10.20.193.78:0/2842247544 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7ffa5400a8d0 con 0x7ffa6406d1b0 2026-02-17T05:57:12.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.217+0000 7ffa3ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:57:12.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.217+0000 7ffa3ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:12.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=12,kv=12,logm=52} crc {auth=3368352932,config=692619927,health=2822512867,kv=449989585,logm=2538746799}) 2026-02-17T05:57:12.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=11,logm=54} crc {auth=271328598,config=692619927,health=2822512867,kv=2510975679,logm=1470005151}) 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=12,kv=12,logm=52} crc {auth=3368352932,config=692619927,health=2822512867,kv=449989585,logm=2538746799}) 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=22,config=2,health=12,kv=12,logm=52} crc {auth=3368352932,config=692619927,health=2822512867,kv=449989585,logm=1591997869}) 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3372390116}) 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3206214073}) 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3372390116}) 2026-02-17T05:57:12.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.217+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4043992461}) 2026-02-17T05:57:12.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffa3403df00 msgr2=0x7ffa340403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffa3403df00 0x7ffa340403c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa6409c980 tx=0x7ffa50033000 comp rx=0 tx=0).stop 2026-02-17T05:57:12.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 10 monclient: shutdown 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406d1b0 msgr2=0x7ffa64134510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406d1b0 0x7ffa64134510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c750 tx=0x7ffa5400cc20 comp rx=0 tx=0).stop 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 shutdown_connections 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffa6409c4a0 0x7ffa641c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffa6406d1b0 0x7ffa64134510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffa3403df00 0x7ffa340403c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 --2- 10.20.193.78:0/2842247544 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffa6406c890 0x7ffa64133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 >> 10.20.193.78:0/2842247544 conn(0x7ffa640a6620 msgr2=0x7ffa640a1940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 shutdown_connections 2026-02-17T05:57:12.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:12.218+0000 7ffa69b03640 1 -- 10.20.193.78:0/2842247544 wait complete. 2026-02-17T05:57:12.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3035154464}) 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=556298567}) 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3035154464}) 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3598011463}) 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4247627276}) 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=510340990}) 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4247627276}) 2026-02-17T05:57:12.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.218+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1034640184}) 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2101530856}) 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693376458}) 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2101530856}) 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2357686636}) 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2026659312}) 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4231027939}) 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2026659312}) 2026-02-17T05:57:12.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.219+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280460824}) 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1654745929}) 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3663806731}) 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1654745929}) 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369568357}) 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4191515041}) 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=289769914}) 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4191515041}) 2026-02-17T05:57:12.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1407025962}) 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2337183540}) 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2589662743}) 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2337183540}) 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.220+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3303734714}) 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4007828904}) 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=455523564}) 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4007828904}) 2026-02-17T05:57:12.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3944296277}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=106648930}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451905330}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=106648930}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.221+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3141982857}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2871125940}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=33403201}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2871125940}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=467241477}) 2026-02-17T05:57:12.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2411450074}) 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3300616977}) 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2411450074}) 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.222+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3802050419}) 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692338198}) 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3750655222}) 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692338198}) 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3590358930}) 2026-02-17T05:57:12.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731351051}) 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=794091134}) 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731351051}) 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.223+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=233482589}) 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=640317988}) 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1426688348}) 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=640317988}) 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3346027481}) 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2601793343}) 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2331468512}) 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2601793343}) 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3964519951}) 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2295073301}) 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1974002362}) 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2295073301}) 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.224+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3769954837}) 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1012192254}) 2026-02-17T05:57:12.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1827971333}) 2026-02-17T05:57:12.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1012192254}) 2026-02-17T05:57:12.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2975611905}) 2026-02-17T05:57:12.237 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=704710516}) 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2566215522}) 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=704710516}) 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.225+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1174491077}) 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2339913469}) 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3350546181}) 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2339913469}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3868808221}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1408391403}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=67311064}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1408391403}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2291837819}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2553613881}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1437244652}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2553613881}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.226+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3431176434}) 2026-02-17T05:57:12.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3303757777}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1272217435}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3303757777}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=957871557}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3911622232}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2063083209}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3911622232}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.227+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4084151848}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3589486359}) 2026-02-17T05:57:12.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2663148648}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3589486359}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=837002217}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2328296188}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884888087}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2328296188}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2791704535}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3518942881}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=292464277}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3518942881}) 2026-02-17T05:57:12.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.228+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3004995735}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1279531079}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2826048650}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1279531079}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1282507667}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1532099210}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3551164759}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1532099210}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460299366}) 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82972985}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=321179813}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82972985}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.229+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2980160931}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029641463}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=378157988}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029641463}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3673578421}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=23} crc {logm=3866484449,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4054116887,mon_config_key=1987855637,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=1165943392,osd_snap=2434322421}) 2026-02-17T05:57:12.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=9,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=23} crc {logm=3679422607,mdsmap=1160443688,mgr=3987721426,mgr_metadata=1258482571,mgrstat=2012413123,mon_config_key=4028116958,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=1165943392,osd_snap=990352214}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=23} crc {logm=3866484449,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4054116887,mon_config_key=1987855637,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=1165943392,osd_snap=2434322421}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.230+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=33,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=24} crc {logm=3620194955,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1927188617,mon_config_key=1912222953,monmap=1784110042,osd_metadata=278982410,osd_pg_creating=1165943392,osd_snap=518946968}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.231+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.231+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=47,osdmap=53} crc {osd_snap=805549953,osdmap=3072140569}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.231+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=44,osdmap=56} crc {osd_snap=2600177098,osdmap=2994388165}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.231+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.231+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=47,osdmap=53} crc {osd_snap=805549953,osdmap=3072140569}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.231+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=45,osdmap=55} crc {osd_snap=3756925343,osdmap=52775249}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.234+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.234+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=2256101645}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.234+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=65} crc {osdmap=1152386284}) 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.234+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:57:12.251 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.234+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=2256101645}) 2026-02-17T05:57:12.252 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:57:12.234+0000 7f2c58147640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=59} crc {osdmap=4115239567}) 2026-02-17T05:57:12.566 INFO:tasks.rados.rados.0.trial115.stdout:1933: done (2 left) 2026-02-17T05:57:12.566 INFO:tasks.rados.rados.0.trial115.stdout:1934: done (1 left) 2026-02-17T05:57:12.566 INFO:tasks.rados.rados.0.trial115.stdout:1935: done (0 left) 2026-02-17T05:57:12.566 INFO:tasks.rados.rados.0.trial115.stdout:1936: copy_from oid 18 from oid 49 current snap is 193 2026-02-17T05:57:12.567 INFO:tasks.rados.rados.0.trial115.stdout:1937: write (excl) oid 46 current snap is 193 2026-02-17T05:57:12.567 INFO:tasks.rados.rados.0.trial115.stdout:1937: seq_num 700 ranges {564900=578776,1629160=709913,3073389=492409} 2026-02-17T05:57:12.570 INFO:tasks.rados.rados.0.trial115.stdout:1937: writing trial11545420-46 from 564900 to 1143676 tid 1 2026-02-17T05:57:12.574 INFO:tasks.rados.rados.0.trial115.stdout:1937: writing trial11545420-46 from 1629160 to 2339073 tid 2 2026-02-17T05:57:12.579 INFO:tasks.rados.rados.0.trial115.stdout:1937: writing trial11545420-46 from 3073389 to 3565798 tid 3 2026-02-17T05:57:12.579 INFO:tasks.rados.rados.0.trial115.stdout:1938: snap_remove snap 180 2026-02-17T05:57:13.238 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:57:13.238 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.307+0000 7f4e29e0c640 10 monclient: get_monmap_and_config 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.307+0000 7f4e29e0c640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.307+0000 7f4e29e0c640 10 monclient: monmap: 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:13.307604+0000 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:13.307604+0000 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:13.307 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.307+0000 7f4e29e0c640 1 Processor -- start 2026-02-17T05:57:13.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.307+0000 7f4e29e0c640 1 -- start start 2026-02-17T05:57:13.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.307+0000 7f4e29e0c640 10 monclient: init 2026-02-17T05:57:13.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2026-02-17T05:57:13.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4e2406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:13.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409d430 0x7f4e2413d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient(hunting): picked mon.noname-b con 0x7f4e2409d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4e2409bf40 0x7f4e2413fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient(hunting): picked mon.noname-a con 0x7f4e2409bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e24073600 con 0x7f4e2406c890 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f4e2409ca50 con 0x7f4e2409bf40 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2406dab0 con 0x7f4e2409d430 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e29e0c640 10 monclient(hunting): authenticate will time out at 1278.861286s 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e237fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f4e2406c890 0x7f4e2406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37188/0 (socket says 10.20.193.78:37188) 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e237fe640 1 -- 10.20.193.78:0/3849052382 learned_addr learned my addr 10.20.193.78:0/3849052382 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 1 --2- 10.20.193.78:0/3849052382 >> v2:10.20.193.78:3300/0 conn(0x7f4e2409bf40 0x7f4e2413fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient(hunting): get_auth_request con 0x7f4e2409bf40 auth_method 0 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e22ffd640 1 --2- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409d430 0x7f4e2413d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e22ffd640 10 monclient(hunting): get_auth_request con 0x7f4e2409d430 auth_method 0 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e22ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e22ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e22ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e22ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e22ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e22ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient(hunting): handle_auth_done global_id 124362 payload 274 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 1 -- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409d430 msgr2=0x7f4e2413d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 1 --2- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409d430 0x7f4e2413d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 1 -- 10.20.193.78:0/3849052382 >> v1:10.20.193.78:6789/0 conn(0x7f4e2406c890 legacy=0x7f4e2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 1 monclient: found mon.noname-a 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.308+0000 7f4e23fff640 1 -- 10.20.193.78:0/3849052382 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2409c770 con 0x7f4e2409bf40 2026-02-17T05:57:13.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.310+0000 7f4e23fff640 1 -- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2409bf40 msgr2=0x7f4e2413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.310+0000 7f4e23fff640 1 --2- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2409bf40 0x7f4e2413fb20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4e2409c670 tx=0x7f4e1802f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e91735b044c592 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 1 -- 10.20.193.78:0/3849052382 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1803c070 con 0x7f4e2409bf40 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: dump: 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 1 -- 10.20.193.78:0/3849052382 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e18037440 con 0x7f4e2409bf40 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 5 monclient: authenticate success, global_id 124362 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 1 -- 10.20.193.78:0/3849052382 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18037810 con 0x7f4e2409bf40 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 4 set_mon_vals no callback set 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e227fc640 10 monclient: dump: 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.311 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.312 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 10 monclient: shutdown 2026-02-17T05:57:13.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 1 -- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2409bf40 msgr2=0x7f4e2413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2409bf40 0x7f4e2413fb20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4e2409c670 tx=0x7f4e1802f130 comp rx=0 tx=0).stop 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 1 -- 10.20.193.78:0/3849052382 shutdown_connections 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2409bf40 0x7f4e2413fb20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/3849052382 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409d430 0x7f4e2413d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 1 -- 10.20.193.78:0/3849052382 >> 10.20.193.78:0/3849052382 conn(0x7f4e240a6620 msgr2=0x7f4e240a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 1 -- 10.20.193.78:0/3849052382 shutdown_connections 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 1 -- 10.20.193.78:0/3849052382 wait complete. 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.311+0000 7f4e29e0c640 10 monclient: monmap: 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 1 Processor -- start 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 1 -- start start 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient: init 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2406c890 0x7f4e241ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient(hunting): picked mon.noname-a con 0x7f4e2406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409bf40 0x7f4e24131ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient(hunting): picked mon.noname-c con 0x7f4e2409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e2409d430 0x7f4e2413b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient(hunting): picked mon.noname-b con 0x7f4e2409d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2406d500 con 0x7f4e2406c890 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2406d380 con 0x7f4e2409d430 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2406d680 con 0x7f4e2409bf40 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e29e0c640 10 monclient(hunting): authenticate will time out at 1278.865436s 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2406c890 0x7f4e241ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409bf40 0x7f4e24131ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2406c890 0x7f4e241ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48698/0 (socket says 10.20.193.78:48698) 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 1 -- 10.20.193.78:0/2484606278 learned_addr learned my addr 10.20.193.78:0/2484606278 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409bf40 0x7f4e24131ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41502/0 (socket says 10.20.193.78:41502) 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient(hunting): get_auth_request con 0x7f4e2409bf40 auth_method 0 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 10 monclient(hunting): get_auth_request con 0x7f4e2406c890 auth_method 0 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient(hunting): handle_auth_done global_id 144119 payload 824 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 1 -- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e2409d430 msgr2=0x7f4e2413b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e2409d430 0x7f4e2413b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 1 -- 10.20.193.78:0/2484606278 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2406c890 msgr2=0x7f4e241ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2406c890 0x7f4e241ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 1 monclient: found mon.noname-c 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.312+0000 7f4e22ffd640 1 -- 10.20.193.78:0/2484606278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2413f010 con 0x7f4e2409bf40 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e22ffd640 1 -- 10.20.193.78:0/2484606278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409bf40 msgr2=0x7f4e24131ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e22ffd640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409bf40 0x7f4e24131ec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400e960 tx=0x7f4e1400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 1 -- 10.20.193.78:0/2484606278 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14017070 con 0x7f4e2409bf40 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: dump: 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.315 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e29e0c640 5 monclient: authenticate success, global_id 144119 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 1 -- 10.20.193.78:0/2484606278 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1400cbb0 con 0x7f4e2409bf40 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e29e0c640 10 monclient: _renew_subs 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e29e0c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2413f2f0 con 0x7f4e2409bf40 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 1 -- 10.20.193.78:0/2484606278 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14013470 con 0x7f4e2409bf40 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4dfffff640 10 monclient: dump: 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e21ffb640 4 set_mon_vals no callback set 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e29e0c640 10 monclient: _renew_subs 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e29e0c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.314+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e240a2150 con 0x7f4e2409bf40 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4dfffff640 1 -- 10.20.193.78:0/2484606278 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4e14024060 con 0x7f4e2409bf40 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4e29e0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4e29e0c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4e29e0c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e24074980 con 0x7f4e2409bf40 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4dfffff640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4df403deb0 0x7f4df4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4dfffff640 1 -- 10.20.193.78:0/2484606278 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f4e14052800 con 0x7f4e2409bf40 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4e237fe640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4df403deb0 0x7f4df4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.315+0000 7f4e237fe640 10 monclient: get_auth_request con 0x7f4df403deb0 auth_method 0 2026-02-17T05:57:13.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.318+0000 7f4dfffff640 1 -- 10.20.193.78:0/2484606278 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4e14013e10 con 0x7f4e2409bf40 2026-02-17T05:57:13.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.318+0000 7f4dfffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.318+0000 7f4dfffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:13.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.318+0000 7f4e237fe640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4df403deb0 0x7f4df4040370 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4e10007e50 tx=0x7f4e10007da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.430+0000 7f4e29e0c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:13.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.430+0000 7f4e29e0c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:13.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.430+0000 7f4e29e0c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.430+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4e2413d8e0 con 0x7f4e2409bf40 2026-02-17T05:57:13.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.430+0000 7f4dfffff640 1 -- 10.20.193.78:0/2484606278 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f4e14051230 con 0x7f4e2409bf40 2026-02-17T05:57:13.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.430+0000 7f4dfffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:13.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.430+0000 7f4dfffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:13.430 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4df403deb0 msgr2=0x7f4df4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4df403deb0 0x7f4df4040370 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4e10007e50 tx=0x7f4e10007da0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 10 monclient: shutdown 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409bf40 msgr2=0x7f4e24131ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409bf40 0x7f4e24131ec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400e960 tx=0x7f4e1400ee30 comp rx=0 tx=0).stop 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 shutdown_connections 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4df403deb0 0x7f4df4040370 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e2409d430 0x7f4e2413b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e2409bf40 0x7f4e24131ec0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 --2- 10.20.193.78:0/2484606278 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e2406c890 0x7f4e241ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 >> 10.20.193.78:0/2484606278 conn(0x7f4e240a6620 msgr2=0x7f4e24137670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 shutdown_connections 2026-02-17T05:57:13.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.431+0000 7f4e29e0c640 1 -- 10.20.193.78:0/2484606278 wait complete. 2026-02-17T05:57:13.438 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:57:13.438 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:57:13.438 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:57:13.438 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:57:13.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient: get_monmap_and_config 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient: monmap: 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:13.507445+0000 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:13.507445+0000 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:13.510 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 1 Processor -- start 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 1 -- start start 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient: init 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f02e006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f02e009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:13.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f02e006d380 0x7f02e013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient(hunting): picked mon.noname-a con 0x7f02e006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e006daf0 con 0x7f02e009c4a0 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e009cc80 con 0x7f02e006c890 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f02e00a0880 con 0x7f02e006d380 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.507+0000 7f02e8987640 10 monclient(hunting): authenticate will time out at 1279.060996s 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f02e006d380 0x7f02e013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e5efb640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f02e009c4a0 0x7f02e013b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37204/0 (socket says 10.20.193.78:37204) 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e5efb640 1 -- 10.20.193.78:0/231625257 learned_addr learned my addr 10.20.193.78:0/231625257 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient(hunting): get_auth_request con 0x7f02e006d380 auth_method 0 2026-02-17T05:57:13.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient(hunting): handle_auth_done global_id 124368 payload 274 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 1 -- 10.20.193.78:0/231625257 >> v1:10.20.193.115:6789/0 conn(0x7f02e006c890 legacy=0x7f02e006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 1 -- 10.20.193.78:0/231625257 >> v1:10.20.193.78:6789/0 conn(0x7f02e009c4a0 legacy=0x7f02e013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 1 monclient: found mon.noname-a 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.508+0000 7f02e6efd640 1 -- 10.20.193.78:0/231625257 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e009cf60 con 0x7f02e006d380 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.510+0000 7f02e6efd640 1 -- 10.20.193.78:0/231625257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006d380 msgr2=0x7f02e013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.510+0000 7f02e6efd640 1 --2- 10.20.193.78:0/231625257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006d380 0x7f02e013ead0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02e0074de0 tx=0x7f02d402f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61645c05021ad497 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.510+0000 7f02e56fa640 1 -- 10.20.193.78:0/231625257 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d403c070 con 0x7f02e006d380 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: dump: 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.521 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 1 -- 10.20.193.78:0/231625257 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d4037640 con 0x7f02e006d380 2026-02-17T05:57:13.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 5 monclient: authenticate success, global_id 124368 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 1 -- 10.20.193.78:0/231625257 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4037a10 con 0x7f02e006d380 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e56fa640 10 monclient: dump: 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.523 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 4 set_mon_vals no callback set 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 10 monclient: shutdown 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 1 -- 10.20.193.78:0/231625257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006d380 msgr2=0x7f02e013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 1 --2- 10.20.193.78:0/231625257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006d380 0x7f02e013ead0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02e0074de0 tx=0x7f02d402f130 comp rx=0 tx=0).stop 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 1 -- 10.20.193.78:0/231625257 shutdown_connections 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 1 --2- 10.20.193.78:0/231625257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006d380 0x7f02e013ead0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 1 -- 10.20.193.78:0/231625257 >> 10.20.193.78:0/231625257 conn(0x7f02e00a6620 msgr2=0x7f02e00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 1 -- 10.20.193.78:0/231625257 shutdown_connections 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 1 -- 10.20.193.78:0/231625257 wait complete. 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 10 monclient: monmap: 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:13.525 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.511+0000 7f02e8987640 1 Processor -- start 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 1 -- start start 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient: init 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006c890 0x7f02e01cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient(hunting): picked mon.noname-a con 0x7f02e006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e006d380 0x7f02e0136300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient(hunting): picked mon.noname-b con 0x7f02e006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e009c4a0 0x7f02e011dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient(hunting): picked mon.noname-c con 0x7f02e009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02e009bfa0 con 0x7f02e006c890 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f02e009be20 con 0x7f02e006d380 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f02e013f010 con 0x7f02e009c4a0 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e8987640 10 monclient(hunting): authenticate will time out at 1279.065312s 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006c890 0x7f02e01cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006c890 0x7f02e01cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48724/0 (socket says 10.20.193.78:48724) 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 -- 10.20.193.78:0/2781560733 learned_addr learned my addr 10.20.193.78:0/2781560733 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:13.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient(hunting): get_auth_request con 0x7f02e006c890 auth_method 0 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e6efd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e009c4a0 0x7f02e011dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e6efd640 10 monclient(hunting): get_auth_request con 0x7f02e009c4a0 auth_method 0 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e6efd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e6efd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e6efd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient(hunting): handle_auth_done global_id 124371 payload 824 2026-02-17T05:57:13.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e009c4a0 msgr2=0x7f02e011dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e009c4a0 0x7f02e011dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e006d380 msgr2=0x7f02e0136300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e006d380 0x7f02e0136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 monclient: found mon.noname-a 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e66fc640 1 -- 10.20.193.78:0/2781560733 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e011f9b0 con 0x7f02e006c890 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.512+0000 7f02e6efd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e009c4a0 0x7f02e011dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.514+0000 7f02e66fc640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006c890 msgr2=0x7f02e01cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.514+0000 7f02e66fc640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006c890 0x7f02e01cf030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c5d0 tx=0x7f02d800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.514+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8016020 con 0x7f02e006c890 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: dump: 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.530 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d8012a60 con 0x7f02e006c890 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02e8987640 5 monclient: authenticate success, global_id 124371 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02e8987640 10 monclient: _renew_subs 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02e8987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e013e4f0 con 0x7f02e006c890 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8012e50 con 0x7f02e006c890 2026-02-17T05:57:13.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02e4ef9640 4 set_mon_vals no callback set 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02caffd640 10 monclient: dump: 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.532 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02e8987640 10 monclient: _renew_subs 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02e8987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.516+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e006fcc0 con 0x7f02e006c890 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e8987640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e8987640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e0136840 0x7f02e011e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e8987640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02e0121590 con 0x7f02e0136840 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e0136840 0x7f02e011e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 10 monclient: get_auth_request con 0x7f02e0136840 auth_method 0 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 10 monclient(hunting): handle_auth_done global_id 124374 payload 824 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f02d801b400 con 0x7f02e006c890 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e5efb640 1 -- 10.20.193.78:0/2781560733 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02e006cc70 con 0x7f02e0136840 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02caffd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b003e0d0 0x7f02b0040590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f02d80528c0 con 0x7f02e006c890 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.517+0000 7f02e6efd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b003e0d0 0x7f02b0040590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.518+0000 7f02e6efd640 10 monclient: get_auth_request con 0x7f02b003e0d0 auth_method 0 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.519+0000 7f02e5efb640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e0136840 msgr2=0x7f02e011e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.519+0000 7f02e5efb640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e0136840 0x7f02e011e2a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02e011f5b0 tx=0x7f02cc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.519+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc017070 con 0x7f02e0136840 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.519+0000 7f02caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.519+0000 7f02caffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.519+0000 7f02caffd640 10 monclient: dump: 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.520+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f02cc0072d0 con 0x7f02e0136840 2026-02-17T05:57:13.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.520+0000 7f02caffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.520+0000 7f02caffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:13.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.520+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e0136840 msgr2=0x7f02e011e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.520+0000 7f02caffd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e0136840 0x7f02e011e2a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02e011f5b0 tx=0x7f02cc00c840 comp rx=0 tx=0).stop 2026-02-17T05:57:13.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.520+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e0136840 msgr2=0x7f02e011e2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:13.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.520+0000 7f02caffd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e0136840 0x7f02e011e2a0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02e011f5b0 tx=0x7f02cc00c840 comp rx=0 tx=0).stop 2026-02-17T05:57:13.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.520+0000 7f02e6efd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b003e0d0 0x7f02b0040590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f02e009c980 tx=0x7f02d4033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e8987640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e8987640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e011e7e0 0x7f02e01da380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e8987640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02e011ebf0 con 0x7f02e011e7e0 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e011e7e0 0x7f02e01da380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 10 monclient: get_auth_request con 0x7f02e011e7e0 auth_method 0 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 10 monclient(hunting): handle_auth_done global_id 124377 payload 824 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.523+0000 7f02e5efb640 1 -- 10.20.193.78:0/2781560733 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f02e00744d0 con 0x7f02e011e7e0 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.525+0000 7f02e5efb640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e011e7e0 msgr2=0x7f02e01da380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.525+0000 7f02e5efb640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e011e7e0 0x7f02e01da380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02cc014000 tx=0x7f02cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.525+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc017030 con 0x7f02e011e7e0 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.525+0000 7f02caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.525+0000 7f02caffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.525+0000 7f02caffd640 10 monclient: dump: 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.539 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f02cc022410 con 0x7f02e011e7e0 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02caffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02caffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e011e7e0 msgr2=0x7f02e01da380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02caffd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e011e7e0 0x7f02e01da380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02cc014000 tx=0x7f02cc004a40 comp rx=0 tx=0).stop 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02caffd640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e011e7e0 msgr2=0x7f02e01da380 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:13.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02caffd640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e011e7e0 0x7f02e01da380 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02cc014000 tx=0x7f02cc004a40 comp rx=0 tx=0).stop 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 122, 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 124991, 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 5, 2026-02-17T05:57:13.541 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.542 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.543 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:57:13.544 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:13.545 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:57:13.546 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.547 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.548 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.549 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.550 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:57:13.551 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b003e0d0 msgr2=0x7f02b0040590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b003e0d0 0x7f02b0040590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f02e009c980 tx=0x7f02d4033000 comp rx=0 tx=0).stop 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 10 monclient: shutdown 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006c890 msgr2=0x7f02e01cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006c890 0x7f02e01cf030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c5d0 tx=0x7f02d800caa0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e6efd640 1 -- 10.20.193.78:0/2781560733 reap_dead start 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 shutdown_connections 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 --2- 10.20.193.78:0/2781560733 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e006c890 0x7f02e01cf030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 >> 10.20.193.78:0/2781560733 conn(0x7f02e00a6620 msgr2=0x7f02e00a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 shutdown_connections 2026-02-17T05:57:13.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.526+0000 7f02e8987640 1 -- 10.20.193.78:0/2781560733 wait complete. 2026-02-17T05:57:13.553 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:57:13.553 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:57:13.553 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient: get_monmap_and_config 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient: monmap: 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:13.619362+0000 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:13.619362+0000 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:13.619 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 1 Processor -- start 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 1 -- start start 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient: init 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f93d806c890 0x7f93d806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient(hunting): picked mon.noname-a con 0x7f93d806c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f93d809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f93d806d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d8135c10 con 0x7f93d806d1b0 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d806daf0 con 0x7f93d809c4a0 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f93d80a0880 con 0x7f93d806c890 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.619+0000 7f93dd349640 10 monclient(hunting): authenticate will time out at 1279.172936s 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f93d806c890 0x7f93d806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f93d806c890 0x7f93d806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48754/0 (socket says 10.20.193.78:48754) 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d77fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f93d806d1b0 0x7f93d8135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37210/0 (socket says 10.20.193.78:37210) 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 1 -- 10.20.193.78:0/1302740300 learned_addr learned my addr 10.20.193.78:0/1302740300 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request con 0x7f93d806c890 auth_method 0 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_done global_id 124380 payload 274 2026-02-17T05:57:13.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 1 -- 10.20.193.78:0/1302740300 >> v1:10.20.193.115:6789/0 conn(0x7f93d809c4a0 legacy=0x7f93d8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:13.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 1 -- 10.20.193.78:0/1302740300 >> v1:10.20.193.78:6789/0 conn(0x7f93d806d1b0 legacy=0x7f93d8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:13.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 1 monclient: found mon.noname-a 2026-02-17T05:57:13.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.620+0000 7f93d6ffd640 1 -- 10.20.193.78:0/1302740300 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d809cc80 con 0x7f93d806c890 2026-02-17T05:57:13.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d6ffd640 1 -- 10.20.193.78:0/1302740300 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d806c890 msgr2=0x7f93d806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d6ffd640 1 --2- 10.20.193.78:0/1302740300 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d806c890 0x7f93d806cc70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009970 tx=0x7f93cc02ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6340a998477b7bfd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 1 -- 10.20.193.78:0/1302740300 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc03c070 con 0x7f93d806c890 2026-02-17T05:57:13.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: dump: 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93dd349640 5 monclient: authenticate success, global_id 124380 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 1 -- 10.20.193.78:0/1302740300 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93cc02fdb0 con 0x7f93d806c890 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 1 -- 10.20.193.78:0/1302740300 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc037660 con 0x7f93d806c890 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93dd349640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93dd349640 4 set_mon_vals no callback set 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.622+0000 7f93d5ffb640 10 monclient: dump: 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 10 monclient: shutdown 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 -- 10.20.193.78:0/1302740300 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d806c890 msgr2=0x7f93d806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 --2- 10.20.193.78:0/1302740300 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d806c890 0x7f93d806cc70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009970 tx=0x7f93cc02ee90 comp rx=0 tx=0).stop 2026-02-17T05:57:13.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 -- 10.20.193.78:0/1302740300 shutdown_connections 2026-02-17T05:57:13.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 --2- 10.20.193.78:0/1302740300 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d806c890 0x7f93d806cc70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 -- 10.20.193.78:0/1302740300 >> 10.20.193.78:0/1302740300 conn(0x7f93d80a6620 msgr2=0x7f93d80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 -- 10.20.193.78:0/1302740300 shutdown_connections 2026-02-17T05:57:13.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 -- 10.20.193.78:0/1302740300 wait complete. 2026-02-17T05:57:13.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 10 monclient: monmap: 2026-02-17T05:57:13.624 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.624 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 Processor -- start 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 1 -- start start 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.623+0000 7f93dd349640 10 monclient: init 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d806c890 0x7f93d81cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient(hunting): picked mon.noname-c con 0x7f93d806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f93d806d1b0 0x7f93d81cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient(hunting): picked mon.noname-b con 0x7f93d806d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d809c4a0 0x7f93d8121600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient(hunting): picked mon.noname-a con 0x7f93d809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93d80a0020 con 0x7f93d809c4a0 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f93d806d640 con 0x7f93d806d1b0 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f93d80a01a0 con 0x7f93d806c890 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93dd349640 10 monclient(hunting): authenticate will time out at 1279.177220s 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d809c4a0 0x7f93d8121600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d809c4a0 0x7f93d8121600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48762/0 (socket says 10.20.193.78:48762) 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 -- 10.20.193.78:0/792052498 learned_addr learned my addr 10.20.193.78:0/792052498 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient(hunting): get_auth_request con 0x7f93d809c4a0 auth_method 0 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d6ffd640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d806c890 0x7f93d81cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request con 0x7f93d806c890 auth_method 0 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient(hunting): handle_auth_done global_id 124383 payload 824 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d806c890 msgr2=0x7f93d81cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d806c890 0x7f93d81cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f93d806d1b0 msgr2=0x7f93d81cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f93d806d1b0 0x7f93d81cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 monclient: found mon.noname-a 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.624+0000 7f93d77fe640 1 -- 10.20.193.78:0/792052498 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d81252b0 con 0x7f93d809c4a0 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93d77fe640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d809c4a0 msgr2=0x7f93d8121600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93d77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d809c4a0 0x7f93d8121600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f93c800caf0 tx=0x7f93c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8017070 con 0x7f93d809c4a0 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: dump: 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93dd349640 5 monclient: authenticate success, global_id 124383 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c8012400 con 0x7f93d809c4a0 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93dd349640 10 monclient: _renew_subs 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93dd349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d8123e00 con 0x7f93d809c4a0 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c80127f0 con 0x7f93d809c4a0 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93d57fa640 4 set_mon_vals no callback set 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93b77fe640 10 monclient: dump: 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93dd349640 10 monclient: _renew_subs 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93dd349640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.626+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d80a25d0 con 0x7f93d809c4a0 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93dd349640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93dd349640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d8063880 0x7f93d8063c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93dd349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f93d809d870 con 0x7f93d8063880 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d8063880 0x7f93d8063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 10 monclient: get_auth_request con 0x7f93d8063880 auth_method 0 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_done global_id 144128 payload 824 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d6ffd640 1 -- 10.20.193.78:0/792052498 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93d809bf40 con 0x7f93d8063880 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d809c4a0 0x7f93d8121600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f93c800caf0 tx=0x7f93c800a060 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f93c80528a0 osd_map(403..403 src has 335..403) 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f93c8007070 con 0x7f93d809c4a0 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93b77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f93a403deb0 0x7f93a4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d67fc640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f93a403deb0 0x7f93a4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.627+0000 7f93d67fc640 10 monclient: get_auth_request con 0x7f93a403deb0 auth_method 0 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93d6ffd640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d8063880 msgr2=0x7f93d8063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93d6ffd640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d8063880 0x7f93d8063c60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f93d8074de0 tx=0x7f93cc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc03c040 con 0x7f93d8063880 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 10 monclient: dump: 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f93cc003020 con 0x7f93d8063880 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d8063880 msgr2=0x7f93d8063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d8063880 0x7f93d8063c60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f93d8074de0 tx=0x7f93cc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d8063880 msgr2=0x7f93d8063c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:13.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.629+0000 7f93b77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d8063880 0x7f93d8063c60 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f93d8074de0 tx=0x7f93cc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.630+0000 7f93d67fc640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f93a403deb0 0x7f93a4040370 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f93c0003ec0 tx=0x7f93c0007ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93dd349640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93dd349640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d811cc70 0x7f93d81d8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93dd349640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f93d80744d0 con 0x7f93d811cc70 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d811cc70 0x7f93d81d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 10 monclient: get_auth_request con 0x7f93d811cc70 auth_method 0 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_done global_id 144131 payload 824 2026-02-17T05:57:13.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.632+0000 7f93d6ffd640 1 -- 10.20.193.78:0/792052498 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f93d81cddb0 con 0x7f93d811cc70 2026-02-17T05:57:13.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93d6ffd640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d811cc70 msgr2=0x7f93d81d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93d6ffd640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d811cc70 0x7f93d81d8540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f93cc02eed0 tx=0x7f93cc008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc03c040 con 0x7f93d811cc70 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 10 monclient: dump: 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f93cc004590 con 0x7f93d811cc70 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d811cc70 msgr2=0x7f93d81d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d811cc70 0x7f93d81d8540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f93cc02eed0 tx=0x7f93cc008030 comp rx=0 tx=0).stop 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d811cc70 msgr2=0x7f93d81d8540 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f93d811cc70 0x7f93d81d8540 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f93cc02eed0 tx=0x7f93cc008030 comp rx=0 tx=0).stop 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 122, 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 17170, 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:57:13.635 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 5, 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:57:13.636 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.637 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:57:13.638 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:57:13.639 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f93a403deb0 msgr2=0x7f93a4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f93a403deb0 0x7f93a4040370 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f93c0003ec0 tx=0x7f93c0007ab0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 10 monclient: shutdown 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d809c4a0 msgr2=0x7f93d8121600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 1 --2- 10.20.193.78:0/792052498 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f93d809c4a0 0x7f93d8121600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f93c800caf0 tx=0x7f93c800a060 comp rx=0 tx=0).stop 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93d77fe640 1 -- 10.20.193.78:0/792052498 reap_dead start 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93d77fe640 1 -- 10.20.193.78:0/792052498 reap_dead start 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93b77fe640 1 -- 10.20.193.78:0/792052498 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (unknown 0 0 0) 0x7f93c80528a0 con 0x7f93d809c4a0 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 shutdown_connections 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 >> 10.20.193.78:0/792052498 conn(0x7f93d80a6620 msgr2=0x7f93d81353d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 shutdown_connections 2026-02-17T05:57:13.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.635+0000 7f93dd349640 1 -- 10.20.193.78:0/792052498 wait complete. 2026-02-17T05:57:13.641 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:57:13.641 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:57:13.641 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:57:13.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.708+0000 7f898b2d3640 10 monclient: get_monmap_and_config 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient: monmap: 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:13.709231+0000 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:13.709231+0000 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:13.709 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 1 Processor -- start 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 1 -- start start 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient: init 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f898406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f898409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f898406d380 0x7f898413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient(hunting): picked mon.noname-a con 0x7f898406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f898406daf0 con 0x7f898409c4a0 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f898409cc80 con 0x7f898406c890 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f89840a0880 con 0x7f898406d380 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f898b2d3640 10 monclient(hunting): authenticate will time out at 1279.262810s 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8989849640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f898406d380 0x7f898413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8988847640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f898409c4a0 0x7f898413b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:37222/0 (socket says 10.20.193.78:37222) 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8988847640 1 -- 10.20.193.78:0/2260691468 learned_addr learned my addr 10.20.193.78:0/2260691468 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8989849640 10 monclient(hunting): get_auth_request con 0x7f898406d380 auth_method 0 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8989849640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8989849640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8989849640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8989849640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.709+0000 7f8989849640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 10 monclient(hunting): handle_auth_done global_id 124386 payload 274 2026-02-17T05:57:13.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 1 -- 10.20.193.78:0/2260691468 >> v1:10.20.193.115:6789/0 conn(0x7f898406c890 legacy=0x7f898406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:13.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 1 -- 10.20.193.78:0/2260691468 >> v1:10.20.193.78:6789/0 conn(0x7f898409c4a0 legacy=0x7f898413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:13.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 1 monclient: found mon.noname-a 2026-02-17T05:57:13.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.710+0000 7f8989849640 1 -- 10.20.193.78:0/2260691468 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898409cf60 con 0x7f898406d380 2026-02-17T05:57:13.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8989849640 1 -- 10.20.193.78:0/2260691468 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406d380 msgr2=0x7f898413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8989849640 1 --2- 10.20.193.78:0/2260691468 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406d380 0x7f898413ead0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8984074de0 tx=0x7f898002f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb881263d6c14a75 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 1 -- 10.20.193.78:0/2260691468 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898003c070 con 0x7f898406d380 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: dump: 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 1 -- 10.20.193.78:0/2260691468 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8980037640 con 0x7f898406d380 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f898b2d3640 5 monclient: authenticate success, global_id 124386 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 1 -- 10.20.193.78:0/2260691468 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980037a10 con 0x7f898406d380 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f898b2d3640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f898b2d3640 4 set_mon_vals no callback set 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.712+0000 7f8973fff640 10 monclient: dump: 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.719 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient: shutdown 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- 10.20.193.78:0/2260691468 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406d380 msgr2=0x7f898413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 --2- 10.20.193.78:0/2260691468 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406d380 0x7f898413ead0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8984074de0 tx=0x7f898002f130 comp rx=0 tx=0).stop 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- 10.20.193.78:0/2260691468 shutdown_connections 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 --2- 10.20.193.78:0/2260691468 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406d380 0x7f898413ead0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- 10.20.193.78:0/2260691468 >> 10.20.193.78:0/2260691468 conn(0x7f89840a6620 msgr2=0x7f89840a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- 10.20.193.78:0/2260691468 shutdown_connections 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- 10.20.193.78:0/2260691468 wait complete. 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient: monmap: 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:13.720 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 Processor -- start 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- start start 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient: init 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406c890 0x7f89841cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient(hunting): picked mon.noname-a con 0x7f898406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f898406d380 0x7f8984136300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient(hunting): picked mon.noname-c con 0x7f898406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898409c4a0 0x7f898411dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient(hunting): picked mon.noname-b con 0x7f898409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f898409bfa0 con 0x7f898406c890 2026-02-17T05:57:13.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f898409be20 con 0x7f898409c4a0 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f898413f010 con 0x7f898406d380 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.713+0000 7f898b2d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f898b2d3640 10 monclient(hunting): authenticate will time out at 1279.267001s 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406c890 0x7f89841cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406c890 0x7f89841cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48784/0 (socket says 10.20.193.78:48784) 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 -- 10.20.193.78:0/3758191103 learned_addr learned my addr 10.20.193.78:0/3758191103 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient(hunting): get_auth_request con 0x7f898406c890 auth_method 0 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8988847640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f898406d380 0x7f8984136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8988847640 10 monclient(hunting): get_auth_request con 0x7f898406d380 auth_method 0 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8988847640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8988847640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8988847640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient(hunting): handle_auth_done global_id 124389 payload 824 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989849640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898409c4a0 0x7f898411dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f898406d380 msgr2=0x7f8984136300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f898406d380 0x7f8984136300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898409c4a0 msgr2=0x7f898411dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898409c4a0 0x7f898411dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 monclient: found mon.noname-a 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989048640 1 -- 10.20.193.78:0/3758191103 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898411f9b0 con 0x7f898406c890 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989849640 10 monclient: get_auth_request con 0x7f898409c4a0 auth_method 0 2026-02-17T05:57:13.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.714+0000 7f8989849640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898409c4a0 0x7f898411dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8989048640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406c890 msgr2=0x7f89841cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8989048640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406c890 0x7f89841cf030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f897400c5d0 tx=0x7f897400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974016020 con 0x7f898406c890 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: dump: 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8974012a60 con 0x7f898406c890 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974012e50 con 0x7f898406c890 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f8971ffb640 10 monclient: dump: 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f89737fe640 4 set_mon_vals no callback set 2026-02-17T05:57:13.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f898b2d3640 5 monclient: authenticate success, global_id 124389 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f898b2d3640 10 monclient: _renew_subs 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f898b2d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898413e4f0 con 0x7f898406c890 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f898b2d3640 10 monclient: _renew_subs 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f898b2d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898406fcc0 con 0x7f898406c890 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.716+0000 7f898b2d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f898b2d3640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8984136840 0x7f898411e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f898b2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f898409d870 con 0x7f8984136840 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f897401b400 con 0x7f898406c890 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8984136840 0x7f898411e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 10 monclient: get_auth_request con 0x7f8984136840 auth_method 0 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8971ffb640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f895403e080 0x7f8954040540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f8974052810 con 0x7f898406c890 2026-02-17T05:57:13.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8989849640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f895403e080 0x7f8954040540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 10 monclient(hunting): handle_auth_done global_id 134200 payload 824 2026-02-17T05:57:13.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8988847640 1 -- 10.20.193.78:0/3758191103 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8984121590 con 0x7f8984136840 2026-02-17T05:57:13.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.717+0000 7f8989849640 10 monclient: get_auth_request con 0x7f895403e080 auth_method 0 2026-02-17T05:57:13.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8988847640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8984136840 msgr2=0x7f898411e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8988847640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8984136840 0x7f898411e2a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f898411f5b0 tx=0x7f897800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897800cbc0 con 0x7f8984136840 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 10 monclient: dump: 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8989849640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f895403e080 0x7f8954040540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f898409c980 tx=0x7f8980033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f8978019070 con 0x7f8984136840 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:13.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8984136840 msgr2=0x7f898411e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8984136840 0x7f898411e2a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f898411f5b0 tx=0x7f897800ef10 comp rx=0 tx=0).stop 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8984136840 msgr2=0x7f898411e2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.720+0000 7f8971ffb640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8984136840 0x7f898411e2a0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f898411f5b0 tx=0x7f897800ef10 comp rx=0 tx=0).stop 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f898b2d3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f898b2d3640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898411e7e0 0x7f89841da380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f898b2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f898411ebf0 con 0x7f898411e7e0 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898411e7e0 0x7f89841da380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 10 monclient: get_auth_request con 0x7f898411e7e0 auth_method 0 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 10 monclient(hunting): handle_auth_done global_id 134203 payload 824 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.723+0000 7f8988847640 1 -- 10.20.193.78:0/3758191103 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f898406cc70 con 0x7f898411e7e0 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8988847640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898411e7e0 msgr2=0x7f89841da380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8988847640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898411e7e0 0x7f89841da380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8978018000 tx=0x7f8978004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897801e220 con 0x7f898411e7e0 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 10 monclient: dump: 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f897801e220 con 0x7f898411e7e0 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898411e7e0 msgr2=0x7f89841da380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898411e7e0 0x7f89841da380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8978018000 tx=0x7f8978004a40 comp rx=0 tx=0).stop 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898411e7e0 msgr2=0x7f89841da380 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:13.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8971ffb640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f898411e7e0 0x7f89841da380 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8978018000 tx=0x7f8978004a40 comp rx=0 tx=0).stop 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 122, 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 61445, 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 5, 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.732 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.733 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:13.734 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:57:13.735 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.736 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.737 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:57:13.738 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:57:13.739 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f895403e080 msgr2=0x7f8954040540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f895403e080 0x7f8954040540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f898409c980 tx=0x7f8980033000 comp rx=0 tx=0).stop 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 10 monclient: shutdown 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406c890 msgr2=0x7f89841cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 1 --2- 10.20.193.78:0/3758191103 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f898406c890 0x7f89841cf030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f897400c5d0 tx=0x7f897400caa0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8989849640 1 -- 10.20.193.78:0/3758191103 reap_dead start 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f8989849640 1 -- 10.20.193.78:0/3758191103 reap_dead start 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 shutdown_connections 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 >> 10.20.193.78:0/3758191103 conn(0x7f89840a6620 msgr2=0x7f89840a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.726+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 shutdown_connections 2026-02-17T05:57:13.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.727+0000 7f898b2d3640 1 -- 10.20.193.78:0/3758191103 wait complete. 2026-02-17T05:57:13.741 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:57:13.741 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:57:13.741 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:57:13.741 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:57:13.741 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:57:13.741 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:57:13.741 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:57:13.751 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:57:13.751 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:57:13.751 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:13.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient: get_monmap_and_config 2026-02-17T05:57:13.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient: monmap: 2026-02-17T05:57:13.860 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:13.860414+0000 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:13.860414+0000 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 1 Processor -- start 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 1 -- start start 2026-02-17T05:57:13.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient: init 2026-02-17T05:57:13.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient: _add_conns ranks=[3,0,4,2,1] 2026-02-17T05:57:13.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3b7806c890 0x7f3b780a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient(hunting): picked mon.noname-a con 0x7f3b7806c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:13.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3b7806cc70 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:13.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3b7813a240 0x7f3b7813c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient(hunting): picked mon.noname-c con 0x7f3b7813a240 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b78074150 con 0x7f3b7806cc70 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7809ef70 con 0x7f3b7806c890 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7809f250 con 0x7f3b7813a240 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b7d161640 10 monclient(hunting): authenticate will time out at 1279.413962s 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b76d76640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f3b7806c890 msgr2=0x7f3b780a01f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b76575640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f3b7806cc70 legacy=0x7f3b7806d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.860+0000 7f3b76d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3b7806c890 0x7f3b780a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3b7813a240 0x7f3b7813c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3b7813a240 0x7f3b7813c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:40226/0 (socket says 10.20.193.78:40226) 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 1 -- 10.20.193.78:0/1857188468 learned_addr learned my addr 10.20.193.78:0/1857188468 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient(hunting): get_auth_request con 0x7f3b7813a240 auth_method 0 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient(hunting): handle_auth_done global_id 134206 payload 274 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 1 -- 10.20.193.78:0/1857188468 >> v2:10.20.193.78:3300/0 conn(0x7f3b7806c890 msgr2=0x7f3b780a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 1 --2- 10.20.193.78:0/1857188468 >> v2:10.20.193.78:3300/0 conn(0x7f3b7806c890 0x7f3b780a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 1 -- 10.20.193.78:0/1857188468 >> v1:10.20.193.78:6789/0 conn(0x7f3b7806cc70 legacy=0x7f3b7806d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 1 monclient: found mon.noname-c 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:57:13.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.861+0000 7f3b77577640 1 -- 10.20.193.78:0/1857188468 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7809df40 con 0x7f3b7813a240 2026-02-17T05:57:13.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b77577640 1 -- 10.20.193.78:0/1857188468 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 msgr2=0x7f3b7813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b77577640 1 --2- 10.20.193.78:0/1857188468 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 0x7f3b7813c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b78069d40 tx=0x7f3b6802f2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=739ae8c1afe6188 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 1 -- 10.20.193.78:0/1857188468 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b680373f0 con 0x7f3b7813a240 2026-02-17T05:57:13.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:13.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: dump: 2026-02-17T05:57:13.869 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.869 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 1 -- 10.20.193.78:0/1857188468 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b68037590 con 0x7f3b7813a240 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 1 -- 10.20.193.78:0/1857188468 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68037960 con 0x7f3b7813a240 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 5 monclient: authenticate success, global_id 134206 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 4 set_mon_vals no callback set 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:13.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b75d74640 10 monclient: dump: 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 10 monclient: shutdown 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 1 -- 10.20.193.78:0/1857188468 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 msgr2=0x7f3b7813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 1 --2- 10.20.193.78:0/1857188468 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 0x7f3b7813c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b78069d40 tx=0x7f3b6802f2a0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 1 -- 10.20.193.78:0/1857188468 shutdown_connections 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 1 --2- 10.20.193.78:0/1857188468 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 0x7f3b7813c630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 1 --2- 10.20.193.78:0/1857188468 >> v2:10.20.193.78:3300/0 conn(0x7f3b7806c890 0x7f3b780a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 1 -- 10.20.193.78:0/1857188468 >> 10.20.193.78:0/1857188468 conn(0x7f3b780a6620 msgr2=0x7f3b780a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 1 -- 10.20.193.78:0/1857188468 shutdown_connections 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.864+0000 7f3b7d161640 1 -- 10.20.193.78:0/1857188468 wait complete. 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient: build_initial_monmap 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient: monmap: 2026-02-17T05:57:13.871 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 1 Processor -- start 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 1 -- start start 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient: init 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b7806c890 0x7f3b781cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient(hunting): picked mon.noname-c con 0x7f3b7806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b7806cc70 0x7f3b78135390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient(hunting): picked mon.noname-a con 0x7f3b7806cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 0x7f3b781358d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient(hunting): picked mon.noname-b con 0x7f3b7813a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7813ccf0 con 0x7f3b7806cc70 2026-02-17T05:57:13.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7813cb70 con 0x7f3b7813a240 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7813ce70 con 0x7f3b7806c890 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b7d161640 10 monclient(hunting): authenticate will time out at 1279.418778s 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76575640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b7806cc70 msgr2=0x7f3b78135390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b7806cc70 0x7f3b78135390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b7806c890 0x7f3b781cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b7806c890 0x7f3b781cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41596/0 (socket says 10.20.193.78:41596) 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 1 -- 10.20.193.78:0/968091865 learned_addr learned my addr 10.20.193.78:0/968091865 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 10 monclient(hunting): get_auth_request con 0x7f3b7806c890 auth_method 0 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b76d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.865+0000 7f3b77577640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 0x7f3b781358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b77577640 10 monclient(hunting): get_auth_request con 0x7f3b7813a240 auth_method 0 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b77577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b77577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b77577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:13.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 10 monclient(hunting): handle_auth_done global_id 144137 payload 824 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 1 -- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 msgr2=0x7f3b781358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 0x7f3b781358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 1 -- 10.20.193.78:0/968091865 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b7806cc70 msgr2=0x7f3b78135390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b7806cc70 0x7f3b78135390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 10 monclient: _finish_hunting 0 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 1 monclient: found mon.noname-c 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.866+0000 7f3b76d76640 1 -- 10.20.193.78:0/968091865 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b780a3bb0 con 0x7f3b7806c890 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b76d76640 1 -- 10.20.193.78:0/968091865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b7806c890 msgr2=0x7f3b781cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b76d76640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b7806c890 0x7f3b781cf250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00b780 tx=0x7f3b6c00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 1 -- 10.20.193.78:0/968091865 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0052d0 con 0x7f3b7806c890 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: dump: 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.874 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 5 monclient: authenticate success, global_id 144137 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 10 monclient: _renew_subs 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b78122a90 con 0x7f3b7806c890 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 1 -- 10.20.193.78:0/968091865 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6c010070 con 0x7f3b7806c890 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 1 -- 10.20.193.78:0/968091865 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0056d0 con 0x7f3b7806c890 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b577fe640 10 monclient: dump: 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:13.875 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b75573640 4 set_mon_vals no callback set 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 10 monclient: _renew_subs 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781227b0 con 0x7f3b7806c890 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.868+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b781224d0 con 0x7f3b7806c890 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.869+0000 7f3b577fe640 1 -- 10.20.193.78:0/968091865 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3b6c01d030 con 0x7f3b7806c890 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.869+0000 7f3b577fe640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3b4403de60 0x7f3b44040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.869+0000 7f3b577fe640 1 -- 10.20.193.78:0/968091865 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f3b6c051dc0 con 0x7f3b7806c890 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.869+0000 7f3b76575640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3b4403de60 0x7f3b44040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.869+0000 7f3b76575640 10 monclient: get_auth_request con 0x7f3b4403de60 auth_method 0 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.871+0000 7f3b577fe640 1 -- 10.20.193.78:0/968091865 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3b780a3bb0 con 0x7f3b7806c890 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.871+0000 7f3b577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:13.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.871+0000 7f3b577fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:13.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.872+0000 7f3b76575640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3b4403de60 0x7f3b44040320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b78120720 tx=0x7f3b60009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:13.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.985+0000 7f3b7d161640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:13.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.985+0000 7f3b7d161640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:13.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.985+0000 7f3b7d161640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:13.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.985+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3b780a01f0 con 0x7f3b7806c890 2026-02-17T05:57:13.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.985+0000 7f3b577fe640 1 -- 10.20.193.78:0/968091865 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f3b6c0058f0 con 0x7f3b7806c890 2026-02-17T05:57:13.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.985+0000 7f3b577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:13.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.985+0000 7f3b577fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:13.986 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:13.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3b4403de60 msgr2=0x7f3b44040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3b4403de60 0x7f3b44040320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b78120720 tx=0x7f3b60009270 comp rx=0 tx=0).stop 2026-02-17T05:57:13.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 10 monclient: shutdown 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b7806c890 msgr2=0x7f3b781cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b7806c890 0x7f3b781cf250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00b780 tx=0x7f3b6c00bc50 comp rx=0 tx=0).stop 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 shutdown_connections 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3b4403de60 0x7f3b44040320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b7813a240 0x7f3b781358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b7806cc70 0x7f3b78135390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 --2- 10.20.193.78:0/968091865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b7806c890 0x7f3b781cf250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 >> 10.20.193.78:0/968091865 conn(0x7f3b780a6620 msgr2=0x7f3b781394f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 shutdown_connections 2026-02-17T05:57:13.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:13.986+0000 7f3b7d161640 1 -- 10.20.193.78:0/968091865 wait complete. 2026-02-17T05:57:16.995 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:17.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.063+0000 7f688178f640 10 monclient: get_monmap_and_config 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.063+0000 7f688178f640 10 monclient: build_initial_monmap 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.063+0000 7f688178f640 10 monclient: monmap: 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:17.063653+0000 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:17.063653+0000 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.063+0000 7f688178f640 1 Processor -- start 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.063+0000 7f688178f640 1 -- start start 2026-02-17T05:57:17.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.063+0000 7f688178f640 10 monclient: init 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f687c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f687c09d430 0x7f687c13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient(hunting): picked mon.noname-c con 0x7f687c09d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f687c09bf40 0x7f687c13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient(hunting): picked mon.noname-a con 0x7f687c09bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c073600 con 0x7f687c06c890 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f687c09ca50 con 0x7f687c09bf40 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f687c06dab0 con 0x7f687c09d430 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f688178f640 10 monclient(hunting): authenticate will time out at 1282.617200s 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687affd640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f687c06c890 legacy=0x7f687c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687b7fe640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f687c09bf40 msgr2=0x7f687c13fb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687b7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f687c09bf40 0x7f687c13fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f687c09d430 0x7f687c13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f687c09d430 0x7f687c13d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:40254/0 (socket says 10.20.193.78:40254) 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 1 -- 10.20.193.78:0/2887694452 learned_addr learned my addr 10.20.193.78:0/2887694452 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient(hunting): get_auth_request con 0x7f687c09d430 auth_method 0 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient(hunting): handle_auth_done global_id 134212 payload 274 2026-02-17T05:57:17.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 1 -- 10.20.193.78:0/2887694452 >> v2:10.20.193.78:3300/0 conn(0x7f687c09bf40 msgr2=0x7f687c13fb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:17.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 1 --2- 10.20.193.78:0/2887694452 >> v2:10.20.193.78:3300/0 conn(0x7f687c09bf40 0x7f687c13fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 1 -- 10.20.193.78:0/2887694452 >> v1:10.20.193.78:6789/0 conn(0x7f687c06c890 legacy=0x7f687c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:17.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:57:17.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 1 monclient: found mon.noname-c 2026-02-17T05:57:17.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:17.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:57:17.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.064+0000 7f687a7fc640 1 -- 10.20.193.78:0/2887694452 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c09c770 con 0x7f687c09d430 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f687a7fc640 1 -- 10.20.193.78:0/2887694452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09d430 msgr2=0x7f687c13d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f687a7fc640 1 --2- 10.20.193.78:0/2887694452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09d430 0x7f687c13d5d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f687c0a0ac0 tx=0x7f686402ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7417c337b801125 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 1 -- 10.20.193.78:0/2887694452 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686403c070 con 0x7f687c09d430 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: dump: 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:17.072 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 1 -- 10.20.193.78:0/2887694452 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686402fda0 con 0x7f687c09d430 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 5 monclient: authenticate success, global_id 134212 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:17.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 4 set_mon_vals no callback set 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 1 -- 10.20.193.78:0/2887694452 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864037660 con 0x7f687c09d430 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f6879ffb640 10 monclient: dump: 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 10 monclient: shutdown 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:17.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 1 -- 10.20.193.78:0/2887694452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09d430 msgr2=0x7f687c13d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 1 --2- 10.20.193.78:0/2887694452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09d430 0x7f687c13d5d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f687c0a0ac0 tx=0x7f686402ee80 comp rx=0 tx=0).stop 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 1 -- 10.20.193.78:0/2887694452 shutdown_connections 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 1 --2- 10.20.193.78:0/2887694452 >> v2:10.20.193.78:3300/0 conn(0x7f687c09bf40 0x7f687c13fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 1 --2- 10.20.193.78:0/2887694452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09d430 0x7f687c13d5d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 1 -- 10.20.193.78:0/2887694452 >> 10.20.193.78:0/2887694452 conn(0x7f687c0a6620 msgr2=0x7f687c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 1 -- 10.20.193.78:0/2887694452 shutdown_connections 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.067+0000 7f688178f640 1 -- 10.20.193.78:0/2887694452 wait complete. 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient: build_initial_monmap 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient: monmap: 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:17.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 1 Processor -- start 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 1 -- start start 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient: init 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06c890 0x7f687c1cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient(hunting): picked mon.noname-a con 0x7f687c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09bf40 0x7f687c13eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient(hunting): picked mon.noname-b con 0x7f687c09bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c09d430 0x7f687c13efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient(hunting): picked mon.noname-c con 0x7f687c09d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f687c06d380 con 0x7f687c06c890 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f687c073410 con 0x7f687c09bf40 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f687c06d500 con 0x7f687c09d430 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f688178f640 10 monclient(hunting): authenticate will time out at 1282.621756s 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687affd640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06c890 msgr2=0x7f687c1cf3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:17.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06c890 0x7f687c1cf3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c09d430 0x7f687c13efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c09d430 0x7f687c13efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41602/0 (socket says 10.20.193.78:41602) 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 1 -- 10.20.193.78:0/3301657222 learned_addr learned my addr 10.20.193.78:0/3301657222 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 10 monclient(hunting): get_auth_request con 0x7f687c09d430 auth_method 0 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687a7fc640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09bf40 0x7f687c13eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687a7fc640 10 monclient(hunting): get_auth_request con 0x7f687c09bf40 auth_method 0 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.068+0000 7f687a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:17.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 10 monclient(hunting): handle_auth_done global_id 144140 payload 824 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 1 -- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09bf40 msgr2=0x7f687c13eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09bf40 0x7f687c13eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 1 -- 10.20.193.78:0/3301657222 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06c890 msgr2=0x7f687c1cf3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06c890 0x7f687c1cf3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 1 monclient: found mon.noname-c 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.069+0000 7f687b7fe640 1 -- 10.20.193.78:0/3301657222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c0a2320 con 0x7f687c09d430 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f687b7fe640 1 -- 10.20.193.78:0/3301657222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c09d430 msgr2=0x7f687c13efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f687b7fe640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c09d430 0x7f687c13efe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f686c00be60 tx=0x7f686c00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 1 -- 10.20.193.78:0/3301657222 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c006e60 con 0x7f687c09d430 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: dump: 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:17.078 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 1 -- 10.20.193.78:0/3301657222 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c010030 con 0x7f687c09d430 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 5 monclient: authenticate success, global_id 144140 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 1 -- 10.20.193.78:0/3301657222 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0168b0 con 0x7f687c09d430 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 10 monclient: dump: 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:17.079 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f68797fa640 4 set_mon_vals no callback set 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 10 monclient: _renew_subs 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c13af90 con 0x7f687c09d430 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 10 monclient: _renew_subs 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c13b550 con 0x7f687c09d430 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c06cc70 con 0x7f687c09d430 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.071+0000 7f685b7fe640 1 -- 10.20.193.78:0/3301657222 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f686c008070 con 0x7f687c09d430 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.072+0000 7f685b7fe640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684803de60 0x7f6848040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.072+0000 7f685b7fe640 1 -- 10.20.193.78:0/3301657222 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f686c052670 con 0x7f687c09d430 2026-02-17T05:57:17.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.072+0000 7f687affd640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684803de60 0x7f6848040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:17.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.072+0000 7f687affd640 10 monclient: get_auth_request con 0x7f684803de60 auth_method 0 2026-02-17T05:57:17.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.074+0000 7f685b7fe640 1 -- 10.20.193.78:0/3301657222 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f687c0a2320 con 0x7f687c09d430 2026-02-17T05:57:17.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.074+0000 7f685b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:17.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.074+0000 7f685b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:17.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.074+0000 7f685b7fe640 1 -- 10.20.193.78:0/3301657222 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:57:17.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.074+0000 7f687affd640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684803de60 0x7f6848040320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6870003a80 tx=0x7f6870009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.187+0000 7f688178f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.187+0000 7f688178f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.187+0000 7f688178f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.187+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f687c0744d0 con 0x7f687c09d430 2026-02-17T05:57:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.187+0000 7f685b7fe640 1 -- 10.20.193.78:0/3301657222 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f686c012d50 con 0x7f687c09d430 2026-02-17T05:57:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.187+0000 7f685b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:17.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.187+0000 7f685b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:17.189 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684803de60 msgr2=0x7f6848040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684803de60 0x7f6848040320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6870003a80 tx=0x7f6870009270 comp rx=0 tx=0).stop 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 10 monclient: shutdown 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c09d430 msgr2=0x7f687c13efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c09d430 0x7f687c13efe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f686c00be60 tx=0x7f686c00bf60 comp rx=0 tx=0).stop 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 shutdown_connections 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c09d430 0x7f687c13efe0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684803de60 0x7f6848040320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c09bf40 0x7f687c13eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 --2- 10.20.193.78:0/3301657222 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06c890 0x7f687c1cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 >> 10.20.193.78:0/3301657222 conn(0x7f687c0a6620 msgr2=0x7f687c0a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:17.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 shutdown_connections 2026-02-17T05:57:17.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:17.188+0000 7f688178f640 1 -- 10.20.193.78:0/3301657222 wait complete. 2026-02-17T05:57:20.197 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:20.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.267+0000 7f05c69a8640 10 monclient: get_monmap_and_config 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.267+0000 7f05c69a8640 10 monclient: build_initial_monmap 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.267+0000 7f05c69a8640 10 monclient: monmap: 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:20.267579+0000 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:20.267579+0000 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:20.268 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:20.269 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:20.269 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:20.269 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:20.269 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:20.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.267+0000 7f05c69a8640 1 Processor -- start 2026-02-17T05:57:20.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.267+0000 7f05c69a8640 1 -- start start 2026-02-17T05:57:20.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.267+0000 7f05c69a8640 10 monclient: init 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 10 monclient: _add_conns ranks=[1,2,0,4,3] 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f05c006c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c009c4a0 0x7f05c013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 10 monclient(hunting): picked mon.noname-b con 0x7f05c009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05c006d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c006daf0 con 0x7f05c006d380 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c009cc80 con 0x7f05c006c890 2026-02-17T05:57:20.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f05c00a0880 con 0x7f05c009c4a0 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c69a8640 10 monclient(hunting): authenticate will time out at 1285.821107s 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05c4f1e640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f05c006d380 legacy=0x7f05c013d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c009c4a0 0x7f05c013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c009c4a0 0x7f05c013b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36570/0 (socket says 10.20.193.78:36570) 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 1 -- 10.20.193.78:0/2093117134 learned_addr learned my addr 10.20.193.78:0/2093117134 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient(hunting): get_auth_request con 0x7f05c009c4a0 auth_method 0 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:20.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient(hunting): handle_auth_done global_id 144143 payload 274 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 1 -- 10.20.193.78:0/2093117134 >> v1:10.20.193.115:6789/0 conn(0x7f05c006c890 legacy=0x7f05c006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 1 -- 10.20.193.78:0/2093117134 >> v1:10.20.193.78:6789/0 conn(0x7f05c006d380 legacy=0x7f05c013d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 1 monclient: found mon.noname-b 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.268+0000 7f05bf7fe640 1 -- 10.20.193.78:0/2093117134 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c009cf60 con 0x7f05c009c4a0 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.270+0000 7f05bf7fe640 1 -- 10.20.193.78:0/2093117134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c009c4a0 msgr2=0x7f05c013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.270+0000 7f05bf7fe640 1 --2- 10.20.193.78:0/2093117134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c009c4a0 0x7f05c013b3a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05b00099a0 tx=0x7f05b002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7468aee8b7c58a01 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:20.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.270+0000 7f05beffd640 1 -- 10.20.193.78:0/2093117134 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b003c070 con 0x7f05c009c4a0 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: dump: 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:20.281 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: _finish_auth 0 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 1 -- 10.20.193.78:0/2093117134 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b002fdf0 con 0x7f05c009c4a0 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 5 monclient: authenticate success, global_id 144143 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 1 -- 10.20.193.78:0/2093117134 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0037660 con 0x7f05c009c4a0 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:20.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05beffd640 10 monclient: dump: 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 4 set_mon_vals no callback set 2026-02-17T05:57:20.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 10 monclient: shutdown 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 1 -- 10.20.193.78:0/2093117134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c009c4a0 msgr2=0x7f05c013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 1 --2- 10.20.193.78:0/2093117134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c009c4a0 0x7f05c013b3a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05b00099a0 tx=0x7f05b002ee90 comp rx=0 tx=0).stop 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 1 -- 10.20.193.78:0/2093117134 shutdown_connections 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 1 --2- 10.20.193.78:0/2093117134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c009c4a0 0x7f05c013b3a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 1 -- 10.20.193.78:0/2093117134 >> 10.20.193.78:0/2093117134 conn(0x7f05c00a6620 msgr2=0x7f05c00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 1 -- 10.20.193.78:0/2093117134 shutdown_connections 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 1 -- 10.20.193.78:0/2093117134 wait complete. 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 10 monclient: build_initial_monmap 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 10 monclient: monmap: 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:20.284 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.271+0000 7f05c69a8640 1 Processor -- start 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 1 -- start start 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient: init 2026-02-17T05:57:20.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 0x7f05c01cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient(hunting): picked mon.noname-c con 0x7f05c006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f05c006d380 0x7f05c01cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient(hunting): picked mon.noname-b con 0x7f05c006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05c009c4a0 0x7f05c01cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient(hunting): picked mon.noname-a con 0x7f05c009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05c009bfa0 con 0x7f05c009c4a0 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f05c009be20 con 0x7f05c006d380 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:20.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f05c013f010 con 0x7f05c006c890 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c69a8640 10 monclient(hunting): authenticate will time out at 1285.825285s 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c4f1e640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05c009c4a0 msgr2=0x7f05c01cdad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05c4f1e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05c009c4a0 0x7f05c01cdad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 0x7f05c01cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 0x7f05c01cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36584/0 (socket says 10.20.193.78:36584) 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 -- 10.20.193.78:0/2445505606 learned_addr learned my addr 10.20.193.78:0/2445505606 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient(hunting): get_auth_request con 0x7f05c006c890 auth_method 0 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:20.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient(hunting): handle_auth_done global_id 144146 payload 824 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 -- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f05c006d380 msgr2=0x7f05c01cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f05c006d380 0x7f05c01cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 -- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05c009c4a0 msgr2=0x7f05c01cdad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05c009c4a0 0x7f05c01cdad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient: _finish_hunting 0 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 monclient: found mon.noname-c 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:57:20.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.272+0000 7f05bffff640 1 -- 10.20.193.78:0/2445505606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c011f9f0 con 0x7f05c006c890 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bffff640 1 -- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 msgr2=0x7f05c01cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bffff640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 0x7f05c01cd050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c5d0 tx=0x7f05ac00caa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 1 -- 10.20.193.78:0/2445505606 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac016020 con 0x7f05c006c890 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: dump: 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:20.289 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: _finish_auth 0 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: _check_auth_tickets 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 1 -- 10.20.193.78:0/2445505606 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ac012a60 con 0x7f05c006c890 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05c69a8640 5 monclient: authenticate success, global_id 144146 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 1 -- 10.20.193.78:0/2445505606 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac012e50 con 0x7f05c006c890 2026-02-17T05:57:20.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05be7fc640 4 set_mon_vals no callback set 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05bcff9640 10 monclient: dump: 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:20.291 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05c69a8640 10 monclient: _renew_subs 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05c69a8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c011fcd0 con 0x7f05c006c890 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05c69a8640 10 monclient: _renew_subs 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05c69a8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.274+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c006fe40 con 0x7f05c006c890 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05c69a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05c69a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05c69a8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c006cc70 con 0x7f05c006c890 2026-02-17T05:57:20.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05bcff9640 1 -- 10.20.193.78:0/2445505606 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f05ac01b400 con 0x7f05c006c890 2026-02-17T05:57:20.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05bcff9640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f059003e000 0x7f05900404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:20.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05bcff9640 1 -- 10.20.193.78:0/2445505606 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f05ac0529d0 con 0x7f05c006c890 2026-02-17T05:57:20.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05bf7fe640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f059003e000 0x7f05900404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:20.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.275+0000 7f05bf7fe640 10 monclient: get_auth_request con 0x7f059003e000 auth_method 0 2026-02-17T05:57:20.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.278+0000 7f05bcff9640 1 -- 10.20.193.78:0/2445505606 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f05ac020030 con 0x7f05c006c890 2026-02-17T05:57:20.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.278+0000 7f05bcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:20.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.278+0000 7f05bcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:20.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.278+0000 7f05bf7fe640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f059003e000 0x7f05900404c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05b0036000 tx=0x7f05b0031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:20.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.399+0000 7f05c69a8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:20.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.399+0000 7f05c69a8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:20.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.399+0000 7f05c69a8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:20.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.400+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f05c011e2c0 con 0x7f05c006c890 2026-02-17T05:57:20.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:20.400+0000 7f05bffff640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 0x7f05c01cd050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c5d0 tx=0x7f05ac00caa0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f05ac01b9b0 mon_command_ack([{"prefix": "quorum_status"}]=0 v0) 2026-02-17T05:57:21.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.228+0000 7f05bcff9640 1 -- 10.20.193.78:0/2445505606 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f05ac01b9b0 con 0x7f05c006c890 2026-02-17T05:57:21.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.228+0000 7f05bcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:21.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.228+0000 7f05bcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:21.228 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:21.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f059003e000 msgr2=0x7f05900404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:21.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f059003e000 0x7f05900404c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05b0036000 tx=0x7f05b0031000 comp rx=0 tx=0).stop 2026-02-17T05:57:21.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 10 monclient: shutdown 2026-02-17T05:57:21.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:21.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 msgr2=0x7f05c01cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 0x7f05c01cd050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c5d0 tx=0x7f05ac00caa0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 shutdown_connections 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05c009c4a0 0x7f05c01cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f059003e000 0x7f05900404c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f05c006d380 0x7f05c01cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 --2- 10.20.193.78:0/2445505606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05c006c890 0x7f05c01cd050 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.229+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 >> 10.20.193.78:0/2445505606 conn(0x7f05c00a6620 msgr2=0x7f05c0070250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.230+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 shutdown_connections 2026-02-17T05:57:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:21.230+0000 7f05c69a8640 1 -- 10.20.193.78:0/2445505606 wait complete. 2026-02-17T05:57:24.242 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:24.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.310+0000 7f31c15a7640 10 monclient: get_monmap_and_config 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.310+0000 7f31c15a7640 10 monclient: build_initial_monmap 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.310+0000 7f31c15a7640 10 monclient: monmap: 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:24.311177+0000 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:24.311177+0000 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:24.313 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 1 Processor -- start 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 1 -- start start 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient: init 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2026-02-17T05:57:24.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 0x7f31bc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient(hunting): picked mon.noname-a con 0x7f31bc06c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient(hunting): picked mon.noname-c con 0x7f31bc09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f31bc134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31bc06daf0 con 0x7f31bc134df0 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f31bc0a0880 con 0x7f31bc06c890 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f31bc06d6f0 con 0x7f31bc09c4a0 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31c15a7640 10 monclient(hunting): authenticate will time out at 1289.864727s 2026-02-17T05:57:24.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31baffd640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 msgr2=0x7f31bc06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31baffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 0x7f31bc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31bb7fe640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f31bc134df0 legacy=0x7f31bc135280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31ba7fc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31ba7fc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:52760/0 (socket says 10.20.193.78:52760) 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 learned_addr learned my addr 10.20.193.78:0/2652102460 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request con 0x7f31bc09c4a0 auth_method 0 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.311+0000 7f31ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.312+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.312+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:24.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.312+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:24.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.312+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.511+0000 7f31baffd640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 msgr2=0x7f31bc06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.511+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 0x7f31bc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31bb7fe640 1 -- 10.20.193.78:0/2652102460 >> v1:10.20.193.78:6789/0 conn(0x7f31bc134df0 legacy=0x7f31bc135280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request con 0x7f31bc09c4a0 auth_method 0 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.512+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:24.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.912+0000 7f31baffd640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 msgr2=0x7f31bc06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:24.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.912+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 0x7f31bc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:24.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.912+0000 7f31bb7fe640 1 -- 10.20.193.78:0/2652102460 >> v1:10.20.193.78:6789/0 conn(0x7f31bc134df0 legacy=0x7f31bc135280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:24.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:24.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request con 0x7f31bc09c4a0 auth_method 0 2026-02-17T05:57:24.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:24.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:24.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:24.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:57:24.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:24.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:24.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:24.913+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:25.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.713+0000 7f31baffd640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 msgr2=0x7f31bc06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:25.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.713+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 0x7f31bc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:57:25.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.713+0000 7f31bb7fe640 1 -- 10.20.193.78:0/2652102460 >> v1:10.20.193.78:6789/0 conn(0x7f31bc134df0 legacy=0x7f31bc135280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:25.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.714+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:25.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.714+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request con 0x7f31bc09c4a0 auth_method 0 2026-02-17T05:57:25.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.714+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:25.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.714+0000 7f31ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:25.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.714+0000 7f31ba7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:25.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.715+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:57:25.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.715+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:25.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.715+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:25.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:25.715+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:57:27.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 10 monclient(hunting): tick 2026-02-17T05:57:27.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:57:27.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 monclient(hunting): continuing hunt 2026-02-17T05:57:27.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 msgr2=0x7f31bc06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.115:3300/0 conn(0x7f31bc09c4a0 0x7f31bc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 -- 10.20.193.78:0/2652102460 >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 msgr2=0x7f31bc06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 --2- 10.20.193.78:0/2652102460 >> v2:10.20.193.78:3300/0 conn(0x7f31bc06c890 0x7f31bc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 -- 10.20.193.78:0/2652102460 >> v1:10.20.193.78:6789/0 conn(0x7f31bc134df0 legacy=0x7f31bc135280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 10 monclient: _add_conns ranks=[2,1] 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 0x7f31a0005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f31a0003770 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f31a0006250 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 -- 10.20.193.78:0/2652102460 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31bc09cc80 con 0x7f31a0006250 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 1 -- 10.20.193.78:0/2652102460 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f31bc06d6f0 con 0x7f31a0003770 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31b8ff9640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 0x7f31a0005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31baffd640 10 monclient(hunting): get_auth_request con 0x7f31a0003770 auth_method 0 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.311+0000 7f31baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.312+0000 7f31baffd640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 msgr2=0x7f31a0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.312+0000 7f31baffd640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 msgr2=0x7f31a0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.312+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 0x7f31a0005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.312+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 0x7f31a0005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:27.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.314+0000 7f31ba7fc640 1 -- 10.20.193.78:0/2652102460 >> v1:10.20.193.115:6789/0 conn(0x7f31a0006250 legacy=0x7f31a0008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 0x7f31a0005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 10 monclient(hunting): get_auth_request con 0x7f31a0003770 auth_method 0 2026-02-17T05:57:27.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:27.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 msgr2=0x7f31a0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:57:27.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 msgr2=0x7f31a0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:27.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 0x7f31a0005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:27.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.512+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 0x7f31a0005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:27.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.572+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258672527 0 0) 0x7f31bc09cc80 con 0x7f31a0006250 2026-02-17T05:57:27.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.572+0000 7f31b9ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:57:27.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.572+0000 7f31b9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.572+0000 7f31b9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.572+0000 7f31b9ffb640 10 monclient(hunting): my global_id is 134218 2026-02-17T05:57:27.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.572+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31a8003620 con 0x7f31a0006250 2026-02-17T05:57:27.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 239041209 0 0) 0x7f31a8003620 con 0x7f31a0006250 2026-02-17T05:57:27.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 msgr2=0x7f31a0005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:27.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a0003770 0x7f31a0005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:57:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:57:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc06daf0 con 0x7f31a0006250 2026-02-17T05:57:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f31a40096f0 con 0x7f31a0006250 2026-02-17T05:57:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:57:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: dump: 2026-02-17T05:57:27.587 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.587 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.588 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.588 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.588 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.588 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.588 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.588 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.588 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.589 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:57:27.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:27.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 >> v1:10.20.193.115:6789/0 conn(0x7f31a0006250 legacy=0x7f31a0008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 1 -- 10.20.193.78:0/2652102460 reap_dead start 2026-02-17T05:57:27.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:57:27.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8003ab0 0x7f31a8003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient(hunting): picked mon.a con 0x7f31a8003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:27.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8004900 0x7f31a8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient(hunting): picked mon.b con 0x7f31a8004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:27.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8005310 0x7f31a800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient(hunting): picked mon.c con 0x7f31a8005310 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:27.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc06d6f0 con 0x7f31a8003ab0 2026-02-17T05:57:27.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31baffd640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8003ab0 msgr2=0x7f31a8003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:27.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f31a800c240 con 0x7f31a8004900 2026-02-17T05:57:27.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f31a800c440 con 0x7f31a8005310 2026-02-17T05:57:27.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31baffd640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8003ab0 0x7f31a8003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:27.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31b9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:57:27.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8005310 0x7f31a800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient(hunting): get_auth_request con 0x7f31a8005310 auth_method 0 2026-02-17T05:57:27.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:27.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:27.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient(hunting): handle_auth_done global_id 134218 payload 293 2026-02-17T05:57:27.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8004900 0x7f31a8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8004900 msgr2=0x7f31a8004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:27.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8004900 0x7f31a8004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8003ab0 msgr2=0x7f31a8003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:27.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8003ab0 0x7f31a8003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 1 monclient: found mon.c 2026-02-17T05:57:27.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 1 -- 10.20.193.78:0/2652102460 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a8003620 con 0x7f31a8005310 2026-02-17T05:57:27.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31ba7fc640 10 monclient: get_auth_request con 0x7f31a8004900 auth_method 0 2026-02-17T05:57:27.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8004900 0x7f31a8004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:57:27.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31c15a7640 5 monclient: authenticate success, global_id 134218 2026-02-17T05:57:27.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.573+0000 7f31c15a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:27.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31bb7fe640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8005310 msgr2=0x7f31a800bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31bb7fe640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8005310 0x7f31a800bac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31bc06f280 tx=0x7f31ac02f480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21bf2950939c614 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac008040 con 0x7f31a8005310 2026-02-17T05:57:27.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:27.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 10 monclient: dump: 2026-02-17T05:57:27.597 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.597 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.597 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.597 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.597 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.597 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.597 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.598 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.598 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac033070 con 0x7f31a8005310 2026-02-17T05:57:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 1 -- 10.20.193.78:0/2652102460 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0386b0 con 0x7f31a8005310 2026-02-17T05:57:27.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:27.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31b9ffb640 10 monclient: dump: 2026-02-17T05:57:27.599 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.599 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.599 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.599 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.600 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.600 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.600 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.600 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.600 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 4 set_mon_vals no callback set 2026-02-17T05:57:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 10 monclient: shutdown 2026-02-17T05:57:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 -- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8005310 msgr2=0x7f31a800bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8005310 0x7f31a800bac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31bc06f280 tx=0x7f31ac02f480 comp rx=0 tx=0).stop 2026-02-17T05:57:27.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 -- 10.20.193.78:0/2652102460 shutdown_connections 2026-02-17T05:57:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8005310 0x7f31a800bac0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8004900 0x7f31a8004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 --2- 10.20.193.78:0/2652102460 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8003ab0 0x7f31a8003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 -- 10.20.193.78:0/2652102460 >> 10.20.193.78:0/2652102460 conn(0x7f31bc0a6620 msgr2=0x7f31bc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 -- 10.20.193.78:0/2652102460 shutdown_connections 2026-02-17T05:57:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.576+0000 7f31c15a7640 1 -- 10.20.193.78:0/2652102460 wait complete. 2026-02-17T05:57:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient: build_initial_monmap 2026-02-17T05:57:27.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient: monmap: 2026-02-17T05:57:27.603 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:27.603 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:27.603 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:27.603 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:27.603 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:27.603 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.604 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:27.604 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:27.604 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:27.604 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 1 Processor -- start 2026-02-17T05:57:27.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 1 -- start start 2026-02-17T05:57:27.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient: init 2026-02-17T05:57:27.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8003ab0 0x7f31bc1cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient(hunting): picked mon.noname-b con 0x7f31a8003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8004900 0x7f31bc1333f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient(hunting): picked mon.noname-c con 0x7f31a8004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8005310 0x7f31bc133930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient(hunting): picked mon.noname-a con 0x7f31a8005310 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc0a01a0 con 0x7f31a8005310 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc0a0020 con 0x7f31a8003ab0 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f31bc09be20 con 0x7f31a8004900 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.577+0000 7f31c15a7640 10 monclient(hunting): authenticate will time out at 1293.130997s 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31bb7fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8005310 msgr2=0x7f31bc133930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31bb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8005310 0x7f31bc133930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8004900 0x7f31bc1333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8004900 0x7f31bc1333f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49830/0 (socket says 10.20.193.78:49830) 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 -- 10.20.193.78:0/751439567 learned_addr learned my addr 10.20.193.78:0/751439567 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request con 0x7f31a8004900 auth_method 0 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient(hunting): handle_auth_done global_id 144149 payload 824 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 -- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8003ab0 msgr2=0x7f31bc1cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8003ab0 0x7f31bc1cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 -- 10.20.193.78:0/751439567 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8005310 msgr2=0x7f31bc133930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8005310 0x7f31bc133930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 monclient: found mon.noname-c 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.578+0000 7f31ba7fc640 1 -- 10.20.193.78:0/751439567 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc0a1020 con 0x7f31a8004900 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31ba7fc640 1 -- 10.20.193.78:0/751439567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8004900 msgr2=0x7f31bc1333f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31ba7fc640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8004900 0x7f31bc1333f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009300 tx=0x7f31a4009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 1 -- 10.20.193.78:0/751439567 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a401a030 con 0x7f31a8004900 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: dump: 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.609 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 1 -- 10.20.193.78:0/751439567 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a40158a0 con 0x7f31a8004900 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 1 -- 10.20.193.78:0/751439567 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4015a50 con 0x7f31a8004900 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31b97fa640 4 set_mon_vals no callback set 2026-02-17T05:57:27.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 10 monclient: dump: 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.611 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31c15a7640 5 monclient: authenticate success, global_id 144149 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31c15a7640 10 monclient: _renew_subs 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31c15a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc120ad0 con 0x7f31a8004900 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31c15a7640 10 monclient: _renew_subs 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31c15a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc1207f0 con 0x7f31a8004900 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.580+0000 7f319b7fe640 1 -- 10.20.193.78:0/751439567 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f31a4015bf0 con 0x7f31a8004900 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.581+0000 7f31c15a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.581+0000 7f31c15a7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.581+0000 7f31c15a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.581+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31bc074980 con 0x7f31a8004900 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.581+0000 7f319b7fe640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f318803de60 0x7f3188040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.581+0000 7f319b7fe640 1 -- 10.20.193.78:0/751439567 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f31a4053040 con 0x7f31a8004900 2026-02-17T05:57:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.581+0000 7f31baffd640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f318803de60 0x7f3188040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.581+0000 7f31baffd640 10 monclient: get_auth_request con 0x7f318803de60 auth_method 0 2026-02-17T05:57:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.583+0000 7f319b7fe640 1 -- 10.20.193.78:0/751439567 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f31a4024030 con 0x7f31a8004900 2026-02-17T05:57:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.583+0000 7f319b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.583+0000 7f319b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:27.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.584+0000 7f31baffd640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f318803de60 0x7f3188040320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f31b00026b0 tx=0x7f31b0009f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.696+0000 7f31c15a7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:27.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.696+0000 7f31c15a7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:27.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.696+0000 7f31c15a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.696+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f31bc1205f0 con 0x7f31a8004900 2026-02-17T05:57:27.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.696+0000 7f319b7fe640 1 -- 10.20.193.78:0/751439567 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f31bc0a1020 con 0x7f31a8004900 2026-02-17T05:57:27.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.696+0000 7f319b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:27.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.696+0000 7f319b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:27.698 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":124,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:27.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f318803de60 msgr2=0x7f3188040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f318803de60 0x7f3188040320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f31b00026b0 tx=0x7f31b0009f50 comp rx=0 tx=0).stop 2026-02-17T05:57:27.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 10 monclient: shutdown 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8004900 msgr2=0x7f31bc1333f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8004900 0x7f31bc1333f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009300 tx=0x7f31a4009a10 comp rx=0 tx=0).stop 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 shutdown_connections 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f31a8005310 0x7f31bc133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f31a8004900 0x7f31bc1333f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f318803de60 0x7f3188040320 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 --2- 10.20.193.78:0/751439567 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f31a8003ab0 0x7f31bc1cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 >> 10.20.193.78:0/751439567 conn(0x7f31bc0a6620 msgr2=0x7f31bc134050 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 shutdown_connections 2026-02-17T05:57:27.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.697+0000 7f31c15a7640 1 -- 10.20.193.78:0/751439567 wait complete. 2026-02-17T05:57:27.706 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:57:27.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:57:27.706 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:57:27.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.774+0000 7f78611de640 10 monclient: get_monmap_and_config 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.774+0000 7f78611de640 10 monclient: build_initial_monmap 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.774+0000 7f78611de640 10 monclient: monmap: 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:27.775097+0000 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:27.775097+0000 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.775 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:27.776 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:27.776 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:27.776 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:27.776 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:27.776 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 1 Processor -- start 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 1 -- start start 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient: init 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient: _add_conns ranks=[0,1,4,3,2] 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f785c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f785c06d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f785c0a01f0 0x7f785c13d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient(hunting): picked mon.noname-c con 0x7f785c0a01f0 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f785c0a0600 con 0x7f785c06c890 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f785c074150 con 0x7f785c06d380 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f785c09ef70 con 0x7f785c0a01f0 2026-02-17T05:57:27.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f785ad76640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f785c06c890 legacy=0x7f785c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f78611de640 10 monclient(hunting): authenticate will time out at 1293.328711s 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f785b577640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f785c0a01f0 0x7f785c13d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f785a575640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f785c06d380 0x7f785c13a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:36550/0 (socket says 10.20.193.78:36550) 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f785a575640 1 -- 10.20.193.78:0/3859118695 learned_addr learned my addr 10.20.193.78:0/3859118695 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f785b577640 10 monclient(hunting): get_auth_request con 0x7f785c0a01f0 auth_method 0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f785b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f785b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.775+0000 7f785b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 10 monclient(hunting): handle_auth_done global_id 134221 payload 274 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 1 -- 10.20.193.78:0/3859118695 >> v1:10.20.193.115:6789/0 conn(0x7f785c06d380 legacy=0x7f785c13a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 1 -- 10.20.193.78:0/3859118695 >> v1:10.20.193.78:6789/0 conn(0x7f785c06c890 legacy=0x7f785c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 1 monclient: found mon.noname-c 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.776+0000 7f785b577640 1 -- 10.20.193.78:0/3859118695 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c09f250 con 0x7f785c0a01f0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.778+0000 7f785b577640 1 -- 10.20.193.78:0/3859118695 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 msgr2=0x7f785c13d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.778+0000 7f785b577640 1 --2- 10.20.193.78:0/3859118695 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 0x7f785c13d970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f784c009d90 tx=0x7f784c02f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c981101dca912035 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 1 -- 10.20.193.78:0/3859118695 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c03c070 con 0x7f785c0a01f0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: dump: 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 5 monclient: authenticate success, global_id 134221 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 1 -- 10.20.193.78:0/3859118695 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784c037640 con 0x7f785c0a01f0 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 1 -- 10.20.193.78:0/3859118695 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c037a10 con 0x7f785c0a01f0 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f7859d74640 10 monclient: dump: 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 4 set_mon_vals no callback set 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 10 monclient: shutdown 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 1 -- 10.20.193.78:0/3859118695 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 msgr2=0x7f785c13d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 1 --2- 10.20.193.78:0/3859118695 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 0x7f785c13d970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f784c009d90 tx=0x7f784c02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 1 -- 10.20.193.78:0/3859118695 shutdown_connections 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 1 --2- 10.20.193.78:0/3859118695 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 0x7f785c13d970 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 1 -- 10.20.193.78:0/3859118695 >> 10.20.193.78:0/3859118695 conn(0x7f785c0a6620 msgr2=0x7f785c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 1 -- 10.20.193.78:0/3859118695 shutdown_connections 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 1 -- 10.20.193.78:0/3859118695 wait complete. 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 10 monclient: build_initial_monmap 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.779+0000 7f78611de640 10 monclient: monmap: 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 1 Processor -- start 2026-02-17T05:57:27.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 1 -- start start 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient: init 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c06c890 0x7f785c1cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient(hunting): picked mon.noname-c con 0x7f785c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c06d380 0x7f785c135250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient(hunting): picked mon.noname-a con 0x7f785c06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 0x7f785c135790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient(hunting): picked mon.noname-b con 0x7f785c0a01f0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f785c09d360 con 0x7f785c06d380 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f785c09d1e0 con 0x7f785c0a01f0 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f785c13deb0 con 0x7f785c06c890 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f78611de640 10 monclient(hunting): authenticate will time out at 1293.333420s 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785a575640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c06d380 msgr2=0x7f785c135250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785a575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c06d380 0x7f785c135250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c06c890 0x7f785c1cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c06c890 0x7f785c1cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49834/0 (socket says 10.20.193.78:49834) 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 -- 10.20.193.78:0/1699281683 learned_addr learned my addr 10.20.193.78:0/1699281683 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient(hunting): get_auth_request con 0x7f785c06c890 auth_method 0 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785b577640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 0x7f785c135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785b577640 10 monclient(hunting): get_auth_request con 0x7f785c0a01f0 auth_method 0 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient(hunting): handle_auth_done global_id 144152 payload 824 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 msgr2=0x7f785c135790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c0a01f0 0x7f785c135790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c06d380 msgr2=0x7f785c135250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c06d380 0x7f785c135250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 monclient: found mon.noname-c 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.780+0000 7f785ad76640 1 -- 10.20.193.78:0/1699281683 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c121920 con 0x7f785c06c890 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f785ad76640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c06c890 msgr2=0x7f785c1cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f785ad76640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c06c890 0x7f785c1cf0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f785000b780 tx=0x7f785000bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850005270 con 0x7f785c06c890 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: dump: 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7850010070 con 0x7f785c06c890 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f78611de640 5 monclient: authenticate success, global_id 144152 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f78611de640 10 monclient: _renew_subs 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f78611de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785c1cf5e0 con 0x7f785c06c890 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850005650 con 0x7f785c06c890 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f7859573640 4 set_mon_vals no callback set 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f783b7fe640 10 monclient: dump: 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f78611de640 10 monclient: _renew_subs 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f78611de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.782+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f785c06f310 con 0x7f785c06c890 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f78611de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f78611de640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11dc70 0x7f785c11e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f78611de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f785c09d810 con 0x7f785c11dc70 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f785001d030 con 0x7f785c06c890 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11dc70 0x7f785c11e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 10 monclient: get_auth_request con 0x7f785c11dc70 auth_method 0 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 10 monclient(hunting): handle_auth_done global_id 144155 payload 824 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785a575640 1 -- 10.20.193.78:0/1699281683 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f785c06cc70 con 0x7f785c11dc70 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803de60 0x7f7828040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f7850051d30 con 0x7f785c06c890 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785b577640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803de60 0x7f7828040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.783+0000 7f785b577640 10 monclient: get_auth_request con 0x7f782803de60 auth_method 0 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.785+0000 7f785a575640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11dc70 msgr2=0x7f785c11e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.785+0000 7f785a575640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11dc70 0x7f785c11e050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f785c11f510 tx=0x7f784400cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.785+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844017070 con 0x7f785c11dc70 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.785+0000 7f783b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.785+0000 7f783b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.785+0000 7f783b7fe640 10 monclient: dump: 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.786+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7844005d40 con 0x7f785c11dc70 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.786+0000 7f783b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.786+0000 7f783b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.786+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11dc70 msgr2=0x7f785c11e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.786+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11dc70 0x7f785c11e050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f785c11f510 tx=0x7f784400cb20 comp rx=0 tx=0).stop 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.786+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11dc70 msgr2=0x7f785c11e050 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.786+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11dc70 0x7f785c11e050 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f785c11f510 tx=0x7f784400cb20 comp rx=0 tx=0).stop 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.786+0000 7f785b577640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803de60 0x7f7828040320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f785c06a480 tx=0x7f784c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f78611de640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f78611de640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11e590 0x7f785c1da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f78611de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f785c1201f0 con 0x7f785c11e590 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11e590 0x7f785c1da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 10 monclient: get_auth_request con 0x7f785c11e590 auth_method 0 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 10 monclient(hunting): handle_auth_done global_id 144158 payload 824 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.789+0000 7f785a575640 1 -- 10.20.193.78:0/1699281683 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f785c0744d0 con 0x7f785c11e590 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f785a575640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11e590 msgr2=0x7f785c1da3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f785a575640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11e590 0x7f785c1da3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f785c11f5b0 tx=0x7f7844004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844017030 con 0x7f785c11e590 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 10 monclient: dump: 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7f7844007490 con 0x7f785c11e590 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11e590 msgr2=0x7f785c1da3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11e590 0x7f785c1da3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f785c11f5b0 tx=0x7f7844004b10 comp rx=0 tx=0).stop 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11e590 msgr2=0x7f785c1da3f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.791+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c11e590 0x7f785c1da3f0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f785c11f5b0 tx=0x7f7844004b10 comp rx=0 tx=0).stop 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 124, 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 31326, 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:57:27.793 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:57:27.794 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:27.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:27.796 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:57:27.797 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803de60 msgr2=0x7f7828040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803de60 0x7f7828040320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f785c06a480 tx=0x7f784c033000 comp rx=0 tx=0).stop 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 10 monclient: shutdown 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c06c890 msgr2=0x7f785c1cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f785b577640 1 -- 10.20.193.78:0/1699281683 reap_dead start 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c06c890 0x7f785c1cf0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f785000b780 tx=0x7f785000bc50 comp rx=0 tx=0).stop 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 shutdown_connections 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 --2- 10.20.193.78:0/1699281683 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c06c890 0x7f785c1cf0a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 >> 10.20.193.78:0/1699281683 conn(0x7f785c0a6620 msgr2=0x7f785c0a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 shutdown_connections 2026-02-17T05:57:27.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.792+0000 7f78611de640 1 -- 10.20.193.78:0/1699281683 wait complete. 2026-02-17T05:57:27.798 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:57:27.798 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:57:27.798 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.865+0000 7f6c5caf2640 10 monclient: get_monmap_and_config 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.865+0000 7f6c5caf2640 10 monclient: build_initial_monmap 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.865+0000 7f6c5caf2640 10 monclient: monmap: 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:27.865812+0000 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:27.865812+0000 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:27.865 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.865+0000 7f6c5caf2640 1 Processor -- start 2026-02-17T05:57:27.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 1 -- start start 2026-02-17T05:57:27.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient: init 2026-02-17T05:57:27.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:27.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2026-02-17T05:57:27.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6c5806c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:27.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5809c4a0 0x7f6c5813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient(hunting): picked mon.noname-b con 0x7f6c5809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6c5806d380 0x7f6c5813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient(hunting): picked mon.noname-c con 0x7f6c5806d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c5806daf0 con 0x7f6c5806c890 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6c580a0880 con 0x7f6c5806d380 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5809cc80 con 0x7f6c5809c4a0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c5caf2640 10 monclient(hunting): authenticate will time out at 1293.419352s 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c56575640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f6c5806c890 legacy=0x7f6c5806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5809c4a0 0x7f6c5813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5809c4a0 0x7f6c5813b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49860/0 (socket says 10.20.193.78:49860) 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 1 -- 10.20.193.78:0/1943812998 learned_addr learned my addr 10.20.193.78:0/1943812998 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient(hunting): get_auth_request con 0x7f6c5809c4a0 auth_method 0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c56d76640 1 --2- 10.20.193.78:0/1943812998 >> v2:10.20.193.115:3300/0 conn(0x7f6c5806d380 0x7f6c5813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c56d76640 10 monclient(hunting): get_auth_request con 0x7f6c5806d380 auth_method 0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c56d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c56d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c56d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient(hunting): handle_auth_done global_id 144161 payload 274 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 1 -- 10.20.193.78:0/1943812998 >> v2:10.20.193.115:3300/0 conn(0x7f6c5806d380 msgr2=0x7f6c5813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 1 --2- 10.20.193.78:0/1943812998 >> v2:10.20.193.115:3300/0 conn(0x7f6c5806d380 0x7f6c5813d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 1 -- 10.20.193.78:0/1943812998 >> v1:10.20.193.78:6789/0 conn(0x7f6c5806c890 legacy=0x7f6c5806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 1 monclient: found mon.noname-b 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c55d74640 1 -- 10.20.193.78:0/1943812998 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5809cf60 con 0x7f6c5809c4a0 2026-02-17T05:57:27.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.866+0000 7f6c56d76640 1 --2- 10.20.193.78:0/1943812998 >> v2:10.20.193.115:3300/0 conn(0x7f6c5806d380 0x7f6c5813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:57:27.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55d74640 1 -- 10.20.193.78:0/1943812998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5809c4a0 msgr2=0x7f6c5813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55d74640 1 --2- 10.20.193.78:0/1943812998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5809c4a0 0x7f6c5813b3a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009970 tx=0x7f6c4002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e41471c066a339 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 1 -- 10.20.193.78:0/1943812998 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4003c070 con 0x7f6c5809c4a0 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: dump: 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 1 -- 10.20.193.78:0/1943812998 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4002fcd0 con 0x7f6c5809c4a0 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 1 -- 10.20.193.78:0/1943812998 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40037660 con 0x7f6c5809c4a0 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c55573640 10 monclient: dump: 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 5 monclient: authenticate success, global_id 144161 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 4 set_mon_vals no callback set 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 10 monclient: shutdown 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 1 -- 10.20.193.78:0/1943812998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5809c4a0 msgr2=0x7f6c5813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 1 --2- 10.20.193.78:0/1943812998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5809c4a0 0x7f6c5813b3a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009970 tx=0x7f6c4002ee90 comp rx=0 tx=0).stop 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 1 -- 10.20.193.78:0/1943812998 shutdown_connections 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 1 --2- 10.20.193.78:0/1943812998 >> v2:10.20.193.115:3300/0 conn(0x7f6c5806d380 0x7f6c5813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 1 --2- 10.20.193.78:0/1943812998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5809c4a0 0x7f6c5813b3a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 1 -- 10.20.193.78:0/1943812998 >> 10.20.193.78:0/1943812998 conn(0x7f6c580a6620 msgr2=0x7f6c580a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 1 -- 10.20.193.78:0/1943812998 shutdown_connections 2026-02-17T05:57:27.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 1 -- 10.20.193.78:0/1943812998 wait complete. 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 10 monclient: build_initial_monmap 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.869+0000 7f6c5caf2640 10 monclient: monmap: 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 1 Processor -- start 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 1 -- start start 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient: init 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5806c890 0x7f6c581cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient(hunting): picked mon.noname-c con 0x7f6c5806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5806d380 0x7f6c58132f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient(hunting): picked mon.noname-b con 0x7f6c5806d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c5809c4a0 0x7f6c5813a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient(hunting): picked mon.noname-a con 0x7f6c5809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5809bfa0 con 0x7f6c5809c4a0 2026-02-17T05:57:27.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5809be20 con 0x7f6c5806d380 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5813de30 con 0x7f6c5806c890 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c5caf2640 10 monclient(hunting): authenticate will time out at 1293.423497s 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56d76640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c5809c4a0 msgr2=0x7f6c5813a100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c5809c4a0 0x7f6c5813a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5806c890 0x7f6c581cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5806c890 0x7f6c581cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49866/0 (socket says 10.20.193.78:49866) 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 -- 10.20.193.78:0/2281076398 learned_addr learned my addr 10.20.193.78:0/2281076398 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient(hunting): get_auth_request con 0x7f6c5806c890 auth_method 0 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c55d74640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5806d380 0x7f6c58132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c55d74640 10 monclient(hunting): get_auth_request con 0x7f6c5806d380 auth_method 0 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c55d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c55d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c55d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient(hunting): handle_auth_done global_id 144164 payload 824 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5806d380 msgr2=0x7f6c58132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5806d380 0x7f6c58132f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c5809c4a0 msgr2=0x7f6c5813a100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c5809c4a0 0x7f6c5813a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient: _finish_hunting 0 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 monclient: found mon.noname-c 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.870+0000 7f6c56575640 1 -- 10.20.193.78:0/2281076398 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5813c590 con 0x7f6c5806c890 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c56575640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5806c890 msgr2=0x7f6c581cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c56575640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5806c890 0x7f6c581cccf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b780 tx=0x7f6c4c00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c005270 con 0x7f6c5806c890 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: dump: 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: _finish_auth 0 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4c010070 con 0x7f6c5806c890 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c5caf2640 5 monclient: authenticate success, global_id 144164 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c5caf2640 10 monclient: _renew_subs 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c5caf2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5806f7a0 con 0x7f6c5806c890 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:27.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c005650 con 0x7f6c5806c890 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c3effd640 10 monclient: dump: 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c54d72640 4 set_mon_vals no callback set 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c5caf2640 10 monclient: _renew_subs 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c5caf2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.872+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5811d330 con 0x7f6c5806c890 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c5caf2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c5caf2640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5813a640 0x7f6c5813aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c5caf2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5813af60 con 0x7f6c5813a640 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6c4c01d030 con 0x7f6c5806c890 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5813a640 0x7f6c5813aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 10 monclient: get_auth_request con 0x7f6c5813a640 auth_method 0 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c3effd640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c2403de60 0x7f6c24040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 10 monclient(hunting): handle_auth_done global_id 134233 payload 824 2026-02-17T05:57:27.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(403..403 src has 335..403) ==== 4735+0+0 (secure 0 0 0) 0x7f6c4c051d40 con 0x7f6c5806c890 2026-02-17T05:57:27.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.873+0000 7f6c55d74640 1 -- 10.20.193.78:0/2281076398 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c58074980 con 0x7f6c5813a640 2026-02-17T05:57:27.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.874+0000 7f6c56d76640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c2403de60 0x7f6c24040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.874+0000 7f6c56d76640 10 monclient: get_auth_request con 0x7f6c2403de60 auth_method 0 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c55d74640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5813a640 msgr2=0x7f6c5813aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c55d74640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5813a640 0x7f6c5813aa20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c5806a480 tx=0x7f6c40004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4003c040 con 0x7f6c5813a640 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 10 monclient: dump: 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c56d76640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c2403de60 0x7f6c24040320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009780 tx=0x7f6c48009a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6c40042670 con 0x7f6c5813a640 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5813a640 msgr2=0x7f6c5813aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5813a640 0x7f6c5813aa20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c5806a480 tx=0x7f6c40004830 comp rx=0 tx=0).stop 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5813a640 msgr2=0x7f6c5813aa20 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:27.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.876+0000 7f6c3effd640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c5813a640 0x7f6c5813aa20 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c5806a480 tx=0x7f6c40004830 comp rx=0 tx=0).stop 2026-02-17T05:57:27.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c5caf2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:57:27.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c5caf2640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c58071670 0x7f6c581d8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:27.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c5caf2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:27.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5806cc70 con 0x7f6c58071670 2026-02-17T05:57:27.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c55d74640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c58071670 0x7f6c581d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:27.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c55d74640 10 monclient: get_auth_request con 0x7f6c58071670 auth_method 0 2026-02-17T05:57:27.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c55d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:27.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c55d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:27.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.879+0000 7f6c55d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:27.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.880+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:27.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.880+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:27.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.880+0000 7f6c55d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:27.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.880+0000 7f6c55d74640 10 monclient(hunting): handle_auth_done global_id 134236 payload 824 2026-02-17T05:57:27.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.880+0000 7f6c55d74640 1 -- 10.20.193.78:0/2281076398 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6c581cdb60 con 0x7f6c58071670 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c55d74640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c58071670 msgr2=0x7f6c581d8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c55d74640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c58071670 0x7f6c581d8040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c4002eed0 tx=0x7f6c400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4003c040 con 0x7f6c58071670 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 10 monclient: dump: 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7f6c4004de20 con 0x7f6c58071670 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c58071670 msgr2=0x7f6c581d8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c58071670 0x7f6c581d8040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c4002eed0 tx=0x7f6c400047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c58071670 msgr2=0x7f6c581d8040 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:57:27.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c3effd640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c58071670 0x7f6c581d8040 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c4002eed0 tx=0x7f6c400047c0 comp rx=0 tx=0).stop 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 124, 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 75601, 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:27.883 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.884 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.885 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:57:27.886 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c2403de60 msgr2=0x7f6c24040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6c2403de60 0x7f6c24040320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009780 tx=0x7f6c48009a20 comp rx=0 tx=0).stop 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 10 monclient: shutdown 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5806c890 msgr2=0x7f6c581cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 1 --2- 10.20.193.78:0/2281076398 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c5806c890 0x7f6c581cccf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b780 tx=0x7f6c4c00bc50 comp rx=0 tx=0).stop 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c56d76640 1 -- 10.20.193.78:0/2281076398 reap_dead start 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c56d76640 1 -- 10.20.193.78:0/2281076398 reap_dead start 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 shutdown_connections 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 >> 10.20.193.78:0/2281076398 conn(0x7f6c580a6620 msgr2=0x7f6c580a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 shutdown_connections 2026-02-17T05:57:27.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:27.882+0000 7f6c5caf2640 1 -- 10.20.193.78:0/2281076398 wait complete. 2026-02-17T05:57:27.888 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:57:27.888 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:57:28.573 INFO:tasks.rados.rados.0.trial115.stdout:1938: done (2 left) 2026-02-17T05:57:28.573 INFO:tasks.rados.rados.0.trial115.stdout:1939: read oid 2 snap -1 2026-02-17T05:57:28.573 INFO:tasks.rados.rados.0.trial115.stdout:1939: expect (ObjNum 646 snap 173 seq_num 646) 2026-02-17T05:57:28.573 INFO:tasks.rados.rados.0.trial115.stdout:1940: snap_remove snap 165 2026-02-17T05:57:29.576 INFO:tasks.rados.rados.0.trial115.stdout:1940: done (3 left) 2026-02-17T05:57:29.576 INFO:tasks.rados.rados.0.trial115.stdout:1941: delete oid 44 current snap is 193 2026-02-17T05:57:29.576 INFO:tasks.rados.rados.0.trial115.stdout:1936: finishing copy_from racing read to trial11545420-18 2026-02-17T05:57:29.576 INFO:tasks.rados.rados.0.trial115.stdout:1936: finishing copy_from to trial11545420-18 2026-02-17T05:57:29.576 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 823 (ObjNum 692 snap 187 seq_num 692) dirty exists 2026-02-17T05:57:29.576 INFO:tasks.rados.rados.0.trial115.stdout:1937: finishing write tid 1 to trial11545420-46 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: finishing write tid 2 to trial11545420-46 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: finishing write tid 3 to trial11545420-46 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: finishing write tid 4 to trial11545420-46 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: finishing write tid 5 to trial11545420-46 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: oid 46 updating version 0 to 924 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: oid 46 updating version 924 to 925 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: oid 46 updating version 925 to 926 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: oid 46 version 926 is already newer than 923 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 926 (ObjNum 700 snap 193 seq_num 700) dirty exists 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1937: left oid 46 (ObjNum 700 snap 193 seq_num 700) 2026-02-17T05:57:29.577 INFO:tasks.rados.rados.0.trial115.stdout:1939: expect (ObjNum 646 snap 173 seq_num 646) 2026-02-17T05:57:29.668 INFO:tasks.rados.rados.0.trial115.stdout:1936: done (3 left) 2026-02-17T05:57:29.668 INFO:tasks.rados.rados.0.trial115.stdout:1937: done (2 left) 2026-02-17T05:57:29.668 INFO:tasks.rados.rados.0.trial115.stdout:1939: done (1 left) 2026-02-17T05:57:29.669 INFO:tasks.rados.rados.0.trial115.stdout:1941: done (0 left) 2026-02-17T05:57:29.669 INFO:tasks.rados.rados.0.trial115.stdout:1942: copy_from oid 24 from oid 37 current snap is 193 2026-02-17T05:57:29.669 INFO:tasks.rados.rados.0.trial115.stdout:1943: write oid 23 current snap is 193 2026-02-17T05:57:29.670 INFO:tasks.rados.rados.0.trial115.stdout:1943: seq_num 702 ranges {598514=729274,1746124=597229,3128235=514953} 2026-02-17T05:57:29.674 INFO:tasks.rados.rados.0.trial115.stdout:1943: writing trial11545420-23 from 598514 to 1327788 tid 1 2026-02-17T05:57:29.676 INFO:tasks.rados.rados.0.trial115.stdout:1943: writing trial11545420-23 from 1746124 to 2343353 tid 2 2026-02-17T05:57:29.682 INFO:tasks.rados.rados.0.trial115.stdout:1943: writing trial11545420-23 from 3128235 to 3643188 tid 3 2026-02-17T05:57:29.682 INFO:tasks.rados.rados.0.trial115.stdout:1944: copy_from oid 19 from oid 5 current snap is 193 2026-02-17T05:57:29.683 INFO:tasks.rados.rados.0.trial115.stdout:1942: finishing copy_from to trial11545420-24 2026-02-17T05:57:29.683 INFO:tasks.rados.rados.0.trial115.stdout:1942: got expected ENOENT (src dne) 2026-02-17T05:57:29.683 INFO:tasks.rados.rados.0.trial115.stdout:1942: finishing copy_from racing read to trial11545420-24 2026-02-17T05:57:29.683 INFO:tasks.rados.rados.0.trial115.stdout:1943: finishing write tid 1 to trial11545420-23 2026-02-17T05:57:29.683 INFO:tasks.rados.rados.0.trial115.stdout:1943: finishing write tid 2 to trial11545420-23 2026-02-17T05:57:29.683 INFO:tasks.rados.rados.0.trial115.stdout:1942: done (2 left) 2026-02-17T05:57:29.683 INFO:tasks.rados.rados.0.trial115.stdout:1945: snap_remove snap 190 2026-02-17T05:57:30.579 INFO:tasks.rados.rados.0.trial115.stdout:1945: done (2 left) 2026-02-17T05:57:30.579 INFO:tasks.rados.rados.0.trial115.stdout:1946: read oid 29 snap 185 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1946: expect deleted 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1947: copy_from oid 50 from oid 2 current snap is 193 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1943: finishing write tid 3 to trial11545420-23 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1944: finishing copy_from to trial11545420-19 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 811 (ObjNum 516 snap 132 seq_num 516) dirty exists 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1944: finishing copy_from racing read to trial11545420-19 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1943: finishing write tid 4 to trial11545420-23 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1943: finishing write tid 5 to trial11545420-23 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1943: oid 23 updating version 0 to 824 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1943: oid 23 updating version 824 to 825 2026-02-17T05:57:30.580 INFO:tasks.rados.rados.0.trial115.stdout:1943: oid 23 updating version 825 to 827 2026-02-17T05:57:30.581 INFO:tasks.rados.rados.0.trial115.stdout:1943: oid 23 version 827 is already newer than 826 2026-02-17T05:57:30.581 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 827 (ObjNum 702 snap 193 seq_num 702) dirty exists 2026-02-17T05:57:30.581 INFO:tasks.rados.rados.0.trial115.stdout:1943: left oid 23 (ObjNum 702 snap 193 seq_num 702) 2026-02-17T05:57:30.581 INFO:tasks.rados.rados.0.trial115.stdout:1943: done (3 left) 2026-02-17T05:57:30.581 INFO:tasks.rados.rados.0.trial115.stdout:1944: done (2 left) 2026-02-17T05:57:30.581 INFO:tasks.rados.rados.0.trial115.stdout:1948: snap_create 2026-02-17T05:57:30.587 INFO:tasks.rados.rados.0.trial115.stdout:1947: finishing copy_from racing read to trial11545420-50 2026-02-17T05:57:30.588 INFO:tasks.rados.rados.0.trial115.stdout:1947: finishing copy_from to trial11545420-50 2026-02-17T05:57:30.588 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 852 (ObjNum 646 snap 173 seq_num 646) dirty exists 2026-02-17T05:57:31.580 INFO:tasks.rados.rados.0.trial115.stdout:1946: done (2 left) 2026-02-17T05:57:31.580 INFO:tasks.rados.rados.0.trial115.stdout:1947: done (1 left) 2026-02-17T05:57:31.580 INFO:tasks.rados.rados.0.trial115.stdout:1948: done (0 left) 2026-02-17T05:57:31.580 INFO:tasks.rados.rados.0.trial115.stdout:1949: write oid 35 current snap is 194 2026-02-17T05:57:31.580 INFO:tasks.rados.rados.0.trial115.stdout:1949: seq_num 703 ranges {615321=404523,1804606=457240,2755658=726225} 2026-02-17T05:57:31.584 INFO:tasks.rados.rados.0.trial115.stdout:1949: writing trial11545420-35 from 615321 to 1019844 tid 1 2026-02-17T05:57:31.588 INFO:tasks.rados.rados.0.trial115.stdout:1949: writing trial11545420-35 from 1804606 to 2261846 tid 2 2026-02-17T05:57:31.593 INFO:tasks.rados.rados.0.trial115.stdout:1949: writing trial11545420-35 from 2755658 to 3481883 tid 3 2026-02-17T05:57:31.593 INFO:tasks.rados.rados.0.trial115.stdout:1950: read oid 43 snap -1 2026-02-17T05:57:31.594 INFO:tasks.rados.rados.0.trial115.stdout:1950: expect (ObjNum 671 snap 186 seq_num 671) 2026-02-17T05:57:31.594 INFO:tasks.rados.rados.0.trial115.stdout:1951: write (excl) oid 34 current snap is 194 2026-02-17T05:57:31.594 INFO:tasks.rados.rados.0.trial115.stdout:1949: finishing write tid 1 to trial11545420-35 2026-02-17T05:57:31.594 INFO:tasks.rados.rados.0.trial115.stdout:1951: seq_num 704 ranges {632128=479772,1746735=600898,3066728=653850} 2026-02-17T05:57:31.597 INFO:tasks.rados.rados.0.trial115.stdout:1951: writing trial11545420-34 from 632128 to 1111900 tid 1 2026-02-17T05:57:31.602 INFO:tasks.rados.rados.0.trial115.stdout:1951: writing trial11545420-34 from 1746735 to 2347633 tid 2 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1951: writing trial11545420-34 from 3066728 to 3720578 tid 3 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1952: delete oid 14 current snap is 194 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: finishing write tid 2 to trial11545420-35 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: finishing write tid 3 to trial11545420-35 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: finishing write tid 4 to trial11545420-35 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: finishing write tid 5 to trial11545420-35 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: oid 35 updating version 0 to 1037 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: oid 35 version 1037 is already newer than 1034 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: oid 35 version 1037 is already newer than 1036 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: oid 35 version 1037 is already newer than 1035 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1037 (ObjNum 703 snap 194 seq_num 703) dirty exists 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1949: left oid 35 (ObjNum 703 snap 194 seq_num 703) 2026-02-17T05:57:31.608 INFO:tasks.rados.rados.0.trial115.stdout:1950: expect (ObjNum 671 snap 186 seq_num 671) 2026-02-17T05:57:31.686 INFO:tasks.rados.rados.0.trial115.stdout:1951: finishing write tid 1 to trial11545420-34 2026-02-17T05:57:31.686 INFO:tasks.rados.rados.0.trial115.stdout:1951: finishing write tid 2 to trial11545420-34 2026-02-17T05:57:31.686 INFO:tasks.rados.rados.0.trial115.stdout:1949: done (3 left) 2026-02-17T05:57:31.686 INFO:tasks.rados.rados.0.trial115.stdout:1950: done (2 left) 2026-02-17T05:57:31.686 INFO:tasks.rados.rados.0.trial115.stdout:1952: done (1 left) 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1953: delete oid 17 current snap is 194 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1951: finishing write tid 3 to trial11545420-34 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1951: finishing write tid 4 to trial11545420-34 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1951: finishing write tid 5 to trial11545420-34 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1951: oid 34 updating version 0 to 823 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1951: oid 34 version 823 is already newer than 821 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1951: oid 34 version 823 is already newer than 822 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1951: oid 34 version 823 is already newer than 820 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 823 (ObjNum 704 snap 194 seq_num 704) dirty exists 2026-02-17T05:57:31.687 INFO:tasks.rados.rados.0.trial115.stdout:1951: left oid 34 (ObjNum 704 snap 194 seq_num 704) 2026-02-17T05:57:31.688 INFO:tasks.rados.rados.0.trial115.stdout:1951: done (1 left) 2026-02-17T05:57:31.688 INFO:tasks.rados.rados.0.trial115.stdout:1953: done (0 left) 2026-02-17T05:57:31.688 INFO:tasks.rados.rados.0.trial115.stdout:1954: copy_from oid 23 from oid 35 current snap is 194 2026-02-17T05:57:31.688 INFO:tasks.rados.rados.0.trial115.stdout:1955: snap_remove snap 139 2026-02-17T05:57:32.290 INFO:tasks.rados.rados.0.trial115.stdout:1955: done (1 left) 2026-02-17T05:57:32.290 INFO:tasks.rados.rados.0.trial115.stdout:1956: write (excl) oid 47 current snap is 194 2026-02-17T05:57:32.290 INFO:tasks.rados.rados.0.trial115.stdout:1956: seq_num 707 ranges {682549=705519,1922181=464578,3148997=487666} 2026-02-17T05:57:32.295 INFO:tasks.rados.rados.0.trial115.stdout:1956: writing trial11545420-47 from 682549 to 1388068 tid 1 2026-02-17T05:57:32.299 INFO:tasks.rados.rados.0.trial115.stdout:1956: writing trial11545420-47 from 1922181 to 2386759 tid 2 2026-02-17T05:57:32.304 INFO:tasks.rados.rados.0.trial115.stdout:1956: writing trial11545420-47 from 3148997 to 3636663 tid 3 2026-02-17T05:57:32.304 INFO:tasks.rados.rados.0.trial115.stdout:1957: rollback oid 33 current snap is 194 2026-02-17T05:57:32.304 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 187 2026-02-17T05:57:32.304 INFO:tasks.rados.rados.0.trial115.stdout:1954: finishing copy_from racing read to trial11545420-23 2026-02-17T05:57:32.304 INFO:tasks.rados.rados.0.trial115.stdout:1954: finishing copy_from to trial11545420-23 2026-02-17T05:57:32.305 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1037 (ObjNum 703 snap 194 seq_num 703) dirty exists 2026-02-17T05:57:32.305 INFO:tasks.rados.rados.0.trial115.stdout:1956: finishing write tid 1 to trial11545420-47 2026-02-17T05:57:32.305 INFO:tasks.rados.rados.0.trial115.stdout:1956: finishing write tid 2 to trial11545420-47 2026-02-17T05:57:32.305 INFO:tasks.rados.rados.0.trial115.stdout:1954: done (2 left) 2026-02-17T05:57:32.305 INFO:tasks.rados.rados.0.trial115.stdout:1958: delete oid 38 current snap is 194 2026-02-17T05:57:32.305 INFO:tasks.rados.rados.0.trial115.stdout:1957: finishing rollback tid 1 to trial11545420-33 2026-02-17T05:57:32.305 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 0 (ObjNum 129 snap 0 seq_num 1333873136) dirty dne 2026-02-17T05:57:32.306 INFO:tasks.rados.rados.0.trial115.stdout:1956: finishing write tid 3 to trial11545420-47 2026-02-17T05:57:32.306 INFO:tasks.rados.rados.0.trial115.stdout:1957: done (2 left) 2026-02-17T05:57:32.306 INFO:tasks.rados.rados.0.trial115.stdout:1958: done (1 left) 2026-02-17T05:57:32.306 INFO:tasks.rados.rados.0.trial115.stdout:1959: write (excl) oid 19 current snap is 194 2026-02-17T05:57:32.306 INFO:tasks.rados.rados.0.trial115.stdout:1959: seq_num 709 ranges {716163=572370,2039145=468247,3203843=626563} 2026-02-17T05:57:32.310 INFO:tasks.rados.rados.0.trial115.stdout:1959: writing trial11545420-19 from 716163 to 1288533 tid 1 2026-02-17T05:57:32.314 INFO:tasks.rados.rados.0.trial115.stdout:1959: writing trial11545420-19 from 2039145 to 2507392 tid 2 2026-02-17T05:57:32.320 INFO:tasks.rados.rados.0.trial115.stdout:1959: writing trial11545420-19 from 3203843 to 3830406 tid 3 2026-02-17T05:57:32.320 INFO:tasks.rados.rados.0.trial115.stdout:1960: read oid 50 snap -1 2026-02-17T05:57:32.320 INFO:tasks.rados.rados.0.trial115.stdout:1960: expect (ObjNum 646 snap 173 seq_num 646) 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1961: read oid 3 snap -1 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1961: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1962: rollback oid 34 current snap is 194 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 188 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1956: finishing write tid 4 to trial11545420-47 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1956: finishing write tid 5 to trial11545420-47 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1956: oid 47 updating version 0 to 916 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1956: oid 47 version 916 is already newer than 915 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1956: oid 47 updating version 916 to 917 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1956: oid 47 updating version 917 to 918 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 918 (ObjNum 707 snap 194 seq_num 707) dirty exists 2026-02-17T05:57:32.321 INFO:tasks.rados.rados.0.trial115.stdout:1956: left oid 47 (ObjNum 707 snap 194 seq_num 707) 2026-02-17T05:57:32.322 INFO:tasks.rados.rados.0.trial115.stdout:1959: finishing write tid 1 to trial11545420-19 2026-02-17T05:57:32.322 INFO:tasks.rados.rados.0.trial115.stdout:1959: finishing write tid 2 to trial11545420-19 2026-02-17T05:57:32.322 INFO:tasks.rados.rados.0.trial115.stdout:1956: done (4 left) 2026-02-17T05:57:32.322 INFO:tasks.rados.rados.0.trial115.stdout:1963: delete oid 5 current snap is 194 2026-02-17T05:57:32.322 INFO:tasks.rados.rados.0.trial115.stdout:1962: finishing rollback tid 0 to trial11545420-34 2026-02-17T05:57:32.323 INFO:tasks.rados.rados.0.trial115.stdout:1962: finishing rollback tid 1 to trial11545420-34 2026-02-17T05:57:32.323 INFO:tasks.rados.rados.0.trial115.stdout:1962: finishing rollback tid 2 to trial11545420-34 2026-02-17T05:57:32.323 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 826 (ObjNum 632 snap 164 seq_num 632) dirty exists 2026-02-17T05:57:32.323 INFO:tasks.rados.rados.0.trial115.stdout:1959: finishing write tid 3 to trial11545420-19 2026-02-17T05:57:32.323 INFO:tasks.rados.rados.0.trial115.stdout:1961: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:57:32.389 INFO:tasks.rados.rados.0.trial115.stdout:1959: finishing write tid 4 to trial11545420-19 2026-02-17T05:57:32.389 INFO:tasks.rados.rados.0.trial115.stdout:1960: expect (ObjNum 646 snap 173 seq_num 646) 2026-02-17T05:57:32.481 INFO:tasks.rados.rados.0.trial115.stdout:1959: finishing write tid 5 to trial11545420-19 2026-02-17T05:57:32.481 INFO:tasks.rados.rados.0.trial115.stdout:1959: oid 19 updating version 0 to 1052 2026-02-17T05:57:32.481 INFO:tasks.rados.rados.0.trial115.stdout:1959: oid 19 version 1052 is already newer than 1050 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1959: oid 19 version 1052 is already newer than 1051 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1959: oid 19 version 1052 is already newer than 1049 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1052 (ObjNum 709 snap 194 seq_num 709) dirty exists 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1959: left oid 19 (ObjNum 709 snap 194 seq_num 709) 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1959: done (4 left) 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1960: done (3 left) 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1961: done (2 left) 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1962: done (1 left) 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1963: done (0 left) 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1964: write (excl) oid 4 current snap is 194 2026-02-17T05:57:32.482 INFO:tasks.rados.rados.0.trial115.stdout:1964: seq_num 711 ranges {749777=722868,2156109=471916,3258689=649107} 2026-02-17T05:57:32.486 INFO:tasks.rados.rados.0.trial115.stdout:1964: writing trial11545420-4 from 749777 to 1472645 tid 1 2026-02-17T05:57:32.490 INFO:tasks.rados.rados.0.trial115.stdout:1964: writing trial11545420-4 from 2156109 to 2628025 tid 2 2026-02-17T05:57:32.495 INFO:tasks.rados.rados.0.trial115.stdout:1964: writing trial11545420-4 from 3258689 to 3907796 tid 3 2026-02-17T05:57:32.495 INFO:tasks.rados.rados.0.trial115.stdout:1965: snap_create 2026-02-17T05:57:32.495 INFO:tasks.rados.rados.0.trial115.stdout:1964: finishing write tid 1 to trial11545420-4 2026-02-17T05:57:32.495 INFO:tasks.rados.rados.0.trial115.stdout:1964: finishing write tid 2 to trial11545420-4 2026-02-17T05:57:32.497 INFO:tasks.rados.rados.0.trial115.stdout:1964: finishing write tid 3 to trial11545420-4 2026-02-17T05:57:32.497 INFO:tasks.rados.rados.0.trial115.stdout:1964: finishing write tid 4 to trial11545420-4 2026-02-17T05:57:32.497 INFO:tasks.rados.rados.0.trial115.stdout:1964: finishing write tid 5 to trial11545420-4 2026-02-17T05:57:32.497 INFO:tasks.rados.rados.0.trial115.stdout:1964: oid 4 updating version 0 to 890 2026-02-17T05:57:32.497 INFO:tasks.rados.rados.0.trial115.stdout:1964: oid 4 version 890 is already newer than 887 2026-02-17T05:57:32.498 INFO:tasks.rados.rados.0.trial115.stdout:1964: oid 4 version 890 is already newer than 889 2026-02-17T05:57:32.498 INFO:tasks.rados.rados.0.trial115.stdout:1964: oid 4 version 890 is already newer than 888 2026-02-17T05:57:32.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 890 (ObjNum 711 snap 194 seq_num 711) dirty exists 2026-02-17T05:57:32.498 INFO:tasks.rados.rados.0.trial115.stdout:1964: left oid 4 (ObjNum 711 snap 194 seq_num 711) 2026-02-17T05:57:33.292 INFO:tasks.rados.rados.0.trial115.stdout:1964: done (1 left) 2026-02-17T05:57:33.293 INFO:tasks.rados.rados.0.trial115.stdout:1965: done (0 left) 2026-02-17T05:57:33.293 INFO:tasks.rados.rados.0.trial115.stdout:1966: delete oid 27 current snap is 195 2026-02-17T05:57:33.297 INFO:tasks.rados.rados.0.trial115.stdout:1966: done (0 left) 2026-02-17T05:57:33.297 INFO:tasks.rados.rados.0.trial115.stdout:1967: copy_from oid 37 from oid 2 current snap is 195 2026-02-17T05:57:33.297 INFO:tasks.rados.rados.0.trial115.stdout:1968: read oid 19 snap -1 2026-02-17T05:57:33.297 INFO:tasks.rados.rados.0.trial115.stdout:1968: expect (ObjNum 709 snap 194 seq_num 709) 2026-02-17T05:57:33.297 INFO:tasks.rados.rados.0.trial115.stdout:1969: write (excl) oid 26 current snap is 195 2026-02-17T05:57:33.297 INFO:tasks.rados.rados.0.trial115.stdout:1969: seq_num 713 ranges {783391=473366,1756720=475585,2797182=788004} 2026-02-17T05:57:33.302 INFO:tasks.rados.rados.0.trial115.stdout:1969: writing trial11545420-26 from 783391 to 1256757 tid 1 2026-02-17T05:57:33.305 INFO:tasks.rados.rados.0.trial115.stdout:1969: writing trial11545420-26 from 1756720 to 2232305 tid 2 2026-02-17T05:57:33.311 INFO:tasks.rados.rados.0.trial115.stdout:1969: writing trial11545420-26 from 2797182 to 3585186 tid 3 2026-02-17T05:57:33.311 INFO:tasks.rados.rados.0.trial115.stdout:1970: delete oid 40 current snap is 195 2026-02-17T05:57:33.311 INFO:tasks.rados.rados.0.trial115.stdout:1967: finishing copy_from racing read to trial11545420-37 2026-02-17T05:57:33.311 INFO:tasks.rados.rados.0.trial115.stdout:1968: expect (ObjNum 709 snap 194 seq_num 709) 2026-02-17T05:57:33.444 INFO:tasks.rados.rados.0.trial115.stdout:1969: finishing write tid 1 to trial11545420-26 2026-02-17T05:57:33.444 INFO:tasks.rados.rados.0.trial115.stdout:1967: finishing copy_from to trial11545420-37 2026-02-17T05:57:33.444 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 852 (ObjNum 646 snap 173 seq_num 646) dirty exists 2026-02-17T05:57:33.444 INFO:tasks.rados.rados.0.trial115.stdout:1969: finishing write tid 2 to trial11545420-26 2026-02-17T05:57:33.444 INFO:tasks.rados.rados.0.trial115.stdout:1969: finishing write tid 3 to trial11545420-26 2026-02-17T05:57:33.444 INFO:tasks.rados.rados.0.trial115.stdout:1969: finishing write tid 4 to trial11545420-26 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1967: done (3 left) 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1968: done (2 left) 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1970: done (1 left) 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1971: write (excl) oid 43 current snap is 195 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1969: finishing write tid 5 to trial11545420-26 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1969: oid 26 updating version 0 to 609 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1969: oid 26 updating version 609 to 611 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1969: oid 26 version 611 is already newer than 608 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1969: oid 26 version 611 is already newer than 610 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 611 (ObjNum 713 snap 195 seq_num 713) dirty exists 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1969: left oid 26 (ObjNum 713 snap 195 seq_num 713) 2026-02-17T05:57:33.445 INFO:tasks.rados.rados.0.trial115.stdout:1971: seq_num 715 ranges {417005=740217,1590037=426968} 2026-02-17T05:57:33.447 INFO:tasks.rados.rados.0.trial115.stdout:1971: writing trial11545420-43 from 417005 to 1157222 tid 1 2026-02-17T05:57:33.451 INFO:tasks.rados.rados.0.trial115.stdout:1971: writing trial11545420-43 from 1590037 to 2017005 tid 2 2026-02-17T05:57:33.451 INFO:tasks.rados.rados.0.trial115.stdout:1969: done (1 left) 2026-02-17T05:57:33.451 INFO:tasks.rados.rados.0.trial115.stdout:1972: rollback oid 25 current snap is 195 2026-02-17T05:57:33.451 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 191 2026-02-17T05:57:33.451 INFO:tasks.rados.rados.0.trial115.stdout:1973: copy_from oid 7 from oid 46 current snap is 195 2026-02-17T05:57:33.452 INFO:tasks.rados.rados.0.trial115.stdout:1974: read oid 24 snap -1 2026-02-17T05:57:33.452 INFO:tasks.rados.rados.0.trial115.stdout:1974: expect (ObjNum 615 snap 161 seq_num 615) 2026-02-17T05:57:33.452 INFO:tasks.rados.rados.0.trial115.stdout:1975: snap_create 2026-02-17T05:57:33.452 INFO:tasks.rados.rados.0.trial115.stdout:1971: finishing write tid 1 to trial11545420-43 2026-02-17T05:57:33.453 INFO:tasks.rados.rados.0.trial115.stdout:1974: expect (ObjNum 615 snap 161 seq_num 615) 2026-02-17T05:57:33.567 INFO:tasks.rados.rados.0.trial115.stdout:1971: finishing write tid 2 to trial11545420-43 2026-02-17T05:57:33.567 INFO:tasks.rados.rados.0.trial115.stdout:1971: finishing write tid 3 to trial11545420-43 2026-02-17T05:57:33.567 INFO:tasks.rados.rados.0.trial115.stdout:1971: finishing write tid 4 to trial11545420-43 2026-02-17T05:57:33.567 INFO:tasks.rados.rados.0.trial115.stdout:1971: oid 43 updating version 0 to 865 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:1971: oid 43 version 865 is already newer than 864 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:1971: oid 43 updating version 865 to 866 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 866 (ObjNum 715 snap 195 seq_num 715) dirty exists 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:1971: left oid 43 (ObjNum 715 snap 195 seq_num 715) 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:1972: finishing rollback tid 0 to trial11545420-25 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:1972: finishing rollback tid 1 to trial11545420-25 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:1972: finishing rollback tid 2 to trial11545420-25 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 880 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:1973: finishing copy_from racing read to trial11545420-7 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:1973: finishing copy_from to trial11545420-7 2026-02-17T05:57:33.568 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 926 (ObjNum 700 snap 193 seq_num 700) dirty exists 2026-02-17T05:57:34.297 INFO:tasks.rados.rados.0.trial115.stdout:1971: done (4 left) 2026-02-17T05:57:34.297 INFO:tasks.rados.rados.0.trial115.stdout:1972: done (3 left) 2026-02-17T05:57:34.297 INFO:tasks.rados.rados.0.trial115.stdout:1973: done (2 left) 2026-02-17T05:57:34.297 INFO:tasks.rados.rados.0.trial115.stdout:1974: done (1 left) 2026-02-17T05:57:34.297 INFO:tasks.rados.rados.0.trial115.stdout:1975: done (0 left) 2026-02-17T05:57:34.297 INFO:tasks.rados.rados.0.trial115.stdout:1976: snap_remove snap 187 2026-02-17T05:57:35.298 INFO:tasks.rados.rados.0.trial115.stdout:1976: done (0 left) 2026-02-17T05:57:35.298 INFO:tasks.rados.rados.0.trial115.stdout:1977: write oid 32 current snap is 196 2026-02-17T05:57:35.299 INFO:tasks.rados.rados.0.trial115.stdout:1977: seq_num 716 ranges {433812=415466,1648519=385293} 2026-02-17T05:57:35.302 INFO:tasks.rados.rados.0.trial115.stdout:1977: writing trial11545420-32 from 433812 to 849278 tid 1 2026-02-17T05:57:35.306 INFO:tasks.rados.rados.0.trial115.stdout:1977: writing trial11545420-32 from 1648519 to 2033812 tid 2 2026-02-17T05:57:35.306 INFO:tasks.rados.rados.0.trial115.stdout:1978: snap_create 2026-02-17T05:57:35.307 INFO:tasks.rados.rados.0.trial115.stdout:1977: finishing write tid 1 to trial11545420-32 2026-02-17T05:57:35.308 INFO:tasks.rados.rados.0.trial115.stdout:1977: finishing write tid 2 to trial11545420-32 2026-02-17T05:57:35.308 INFO:tasks.rados.rados.0.trial115.stdout:1977: finishing write tid 3 to trial11545420-32 2026-02-17T05:57:35.308 INFO:tasks.rados.rados.0.trial115.stdout:1977: finishing write tid 4 to trial11545420-32 2026-02-17T05:57:35.309 INFO:tasks.rados.rados.0.trial115.stdout:1977: oid 32 updating version 0 to 870 2026-02-17T05:57:35.309 INFO:tasks.rados.rados.0.trial115.stdout:1977: oid 32 updating version 870 to 871 2026-02-17T05:57:35.309 INFO:tasks.rados.rados.0.trial115.stdout:1977: oid 32 version 871 is already newer than 869 2026-02-17T05:57:35.309 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 871 (ObjNum 716 snap 196 seq_num 716) dirty exists 2026-02-17T05:57:35.309 INFO:tasks.rados.rados.0.trial115.stdout:1977: left oid 32 (ObjNum 716 snap 196 seq_num 716) 2026-02-17T05:57:36.301 INFO:tasks.rados.rados.0.trial115.stdout:1977: done (1 left) 2026-02-17T05:57:36.301 INFO:tasks.rados.rados.0.trial115.stdout:1978: done (0 left) 2026-02-17T05:57:36.301 INFO:tasks.rados.rados.0.trial115.stdout:1979: delete oid 4 current snap is 197 2026-02-17T05:57:36.303 INFO:tasks.rados.rados.0.trial115.stdout:1979: done (0 left) 2026-02-17T05:57:36.303 INFO:tasks.rados.rados.0.trial115.stdout:1980: write (excl) oid 38 current snap is 197 2026-02-17T05:57:36.304 INFO:tasks.rados.rados.0.trial115.stdout:1980: seq_num 718 ranges {467426=565964,1649130=418296} 2026-02-17T05:57:36.308 INFO:tasks.rados.rados.0.trial115.stdout:1980: writing trial11545420-38 from 467426 to 1033390 tid 1 2026-02-17T05:57:36.313 INFO:tasks.rados.rados.0.trial115.stdout:1980: writing trial11545420-38 from 1649130 to 2067426 tid 2 2026-02-17T05:57:36.313 INFO:tasks.rados.rados.0.trial115.stdout:1981: rollback oid 11 current snap is 197 2026-02-17T05:57:36.313 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 172 2026-02-17T05:57:36.313 INFO:tasks.rados.rados.0.trial115.stdout:1980: finishing write tid 1 to trial11545420-38 2026-02-17T05:57:36.313 INFO:tasks.rados.rados.0.trial115.stdout:1982: snap_create 2026-02-17T05:57:36.314 INFO:tasks.rados.rados.0.trial115.stdout:1980: finishing write tid 2 to trial11545420-38 2026-02-17T05:57:36.314 INFO:tasks.rados.rados.0.trial115.stdout:1980: finishing write tid 3 to trial11545420-38 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:1980: finishing write tid 4 to trial11545420-38 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:1980: oid 38 updating version 0 to 1059 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:1980: oid 38 updating version 1059 to 1060 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:1980: oid 38 updating version 1060 to 1061 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1061 (ObjNum 718 snap 197 seq_num 718) dirty exists 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:1980: left oid 38 (ObjNum 718 snap 197 seq_num 718) 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:1981: finishing rollback tid 0 to trial11545420-11 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:1981: finishing rollback tid 1 to trial11545420-11 2026-02-17T05:57:36.315 INFO:tasks.rados.rados.0.trial115.stdout:1981: finishing rollback tid 2 to trial11545420-11 2026-02-17T05:57:36.316 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 964 (ObjNum 634 snap 165 seq_num 634) dirty exists 2026-02-17T05:57:37.291 INFO:tasks.rados.rados.0.trial115.stdout:1980: done (2 left) 2026-02-17T05:57:37.291 INFO:tasks.rados.rados.0.trial115.stdout:1981: done (1 left) 2026-02-17T05:57:37.291 INFO:tasks.rados.rados.0.trial115.stdout:1982: done (0 left) 2026-02-17T05:57:37.291 INFO:tasks.rados.rados.0.trial115.stdout:1983: read oid 16 snap -1 2026-02-17T05:57:37.291 INFO:tasks.rados.rados.0.trial115.stdout:1983: expect (ObjNum 698 snap 190 seq_num 698) 2026-02-17T05:57:37.291 INFO:tasks.rados.rados.0.trial115.stdout:1984: snap_remove snap 188 2026-02-17T05:57:38.291 INFO:tasks.rados.rados.0.trial115.stdout:1984: done (1 left) 2026-02-17T05:57:38.291 INFO:tasks.rados.rados.0.trial115.stdout:1985: read oid 27 snap -1 2026-02-17T05:57:38.291 INFO:tasks.rados.rados.0.trial115.stdout:1985: expect deleted 2026-02-17T05:57:38.291 INFO:tasks.rados.rados.0.trial115.stdout:1986: copy_from oid 1 from oid 41 current snap is 198 2026-02-17T05:57:38.291 INFO:tasks.rados.rados.0.trial115.stdout:1983: expect (ObjNum 698 snap 190 seq_num 698) 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1986: finishing copy_from racing read to trial11545420-1 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1986: finishing copy_from to trial11545420-1 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 874 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1983: done (2 left) 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1985: done (1 left) 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1986: done (0 left) 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1987: read oid 11 snap -1 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1987: expect (ObjNum 634 snap 165 seq_num 634) 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1988: read oid 23 snap -1 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1988: expect (ObjNum 703 snap 194 seq_num 703) 2026-02-17T05:57:38.375 INFO:tasks.rados.rados.0.trial115.stdout:1989: delete oid 4 current snap is 198 2026-02-17T05:57:38.376 INFO:tasks.rados.rados.0.trial115.stdout:1989: done (2 left) 2026-02-17T05:57:38.376 INFO:tasks.rados.rados.0.trial115.stdout:1990: snap_remove snap 193 2026-02-17T05:57:39.304 INFO:tasks.rados.rados.0.trial115.stdout:1990: done (2 left) 2026-02-17T05:57:39.304 INFO:tasks.rados.rados.0.trial115.stdout:1991: rollback oid 6 current snap is 198 2026-02-17T05:57:39.304 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 6 to 189 2026-02-17T05:57:39.304 INFO:tasks.rados.rados.0.trial115.stdout:1987: expect (ObjNum 634 snap 165 seq_num 634) 2026-02-17T05:57:39.384 INFO:tasks.rados.rados.0.trial115.stdout:1988: expect (ObjNum 703 snap 194 seq_num 703) 2026-02-17T05:57:39.577 INFO:tasks.rados.rados.0.trial115.stdout:1991: finishing rollback tid 0 to trial11545420-6 2026-02-17T05:57:39.577 INFO:tasks.rados.rados.0.trial115.stdout:1991: finishing rollback tid 1 to trial11545420-6 2026-02-17T05:57:39.577 INFO:tasks.rados.rados.0.trial115.stdout:1987: done (2 left) 2026-02-17T05:57:39.577 INFO:tasks.rados.rados.0.trial115.stdout:1988: done (1 left) 2026-02-17T05:57:39.578 INFO:tasks.rados.rados.0.trial115.stdout:1992: rollback oid 29 current snap is 198 2026-02-17T05:57:39.578 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 186 2026-02-17T05:57:39.578 INFO:tasks.rados.rados.0.trial115.stdout:1991: finishing rollback tid 2 to trial11545420-6 2026-02-17T05:57:39.578 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 829 (ObjNum 657 snap 178 seq_num 657) dirty exists 2026-02-17T05:57:39.578 INFO:tasks.rados.rados.0.trial115.stdout:1991: done (1 left) 2026-02-17T05:57:39.578 INFO:tasks.rados.rados.0.trial115.stdout:1993: delete oid 20 current snap is 198 2026-02-17T05:57:39.579 INFO:tasks.rados.rados.0.trial115.stdout:1993: done (1 left) 2026-02-17T05:57:39.579 INFO:tasks.rados.rados.0.trial115.stdout:1994: delete oid 39 current snap is 198 2026-02-17T05:57:39.579 INFO:tasks.rados.rados.0.trial115.stdout:1992: finishing rollback tid 0 to trial11545420-29 2026-02-17T05:57:39.580 INFO:tasks.rados.rados.0.trial115.stdout:1992: finishing rollback tid 1 to trial11545420-29 2026-02-17T05:57:39.580 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 647 (ObjNum 49 snap 0 seq_num 1634300532) dirty dne 2026-02-17T05:57:39.582 INFO:tasks.rados.rados.0.trial115.stdout:1992: done (1 left) 2026-02-17T05:57:39.582 INFO:tasks.rados.rados.0.trial115.stdout:1994: done (0 left) 2026-02-17T05:57:39.582 INFO:tasks.rados.rados.0.trial115.stdout:1995: copy_from oid 25 from oid 19 current snap is 198 2026-02-17T05:57:39.582 INFO:tasks.rados.rados.0.trial115.stdout:1996: snap_create 2026-02-17T05:57:39.588 INFO:tasks.rados.rados.0.trial115.stdout:1995: finishing copy_from racing read to trial11545420-25 2026-02-17T05:57:39.589 INFO:tasks.rados.rados.0.trial115.stdout:1995: finishing copy_from to trial11545420-25 2026-02-17T05:57:39.589 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1052 (ObjNum 709 snap 194 seq_num 709) dirty exists 2026-02-17T05:57:41.263 INFO:tasks.rados.rados.0.trial115.stdout:1995: done (1 left) 2026-02-17T05:57:41.263 INFO:tasks.rados.rados.0.trial115.stdout:1996: done (0 left) 2026-02-17T05:57:41.263 INFO:tasks.rados.rados.0.trial115.stdout:1997: rollback oid 26 current snap is 199 2026-02-17T05:57:41.263 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 26 to 189 2026-02-17T05:57:41.263 INFO:tasks.rados.rados.0.trial115.stdout:1998: delete oid 18 current snap is 199 2026-02-17T05:57:41.266 INFO:tasks.rados.rados.0.trial115.stdout:1998: done (1 left) 2026-02-17T05:57:41.266 INFO:tasks.rados.rados.0.trial115.stdout:1999: copy_from oid 48 from oid 32 current snap is 199 2026-02-17T05:57:41.266 INFO:tasks.rados.rados.0.trial115.stdout:2000: copy_from oid 50 from oid 27 current snap is 199 2026-02-17T05:57:41.266 INFO:tasks.rados.rados.0.trial115.stdout:2001: write oid 19 current snap is 199 2026-02-17T05:57:41.266 INFO:tasks.rados.rados.0.trial115.stdout:2001: seq_num 723 ranges {551461=658562,1941540=209921} 2026-02-17T05:57:41.271 INFO:tasks.rados.rados.0.trial115.stdout:2001: writing trial11545420-19 from 551461 to 1210023 tid 1 2026-02-17T05:57:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2001: writing trial11545420-19 from 1941540 to 2151461 tid 2 2026-02-17T05:57:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2002: write (excl) oid 8 current snap is 199 2026-02-17T05:57:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2002: seq_num 724 ranges {568268=733811,2000022=168246} 2026-02-17T05:57:41.280 INFO:tasks.rados.rados.0.trial115.stdout:2002: writing trial11545420-8 from 568268 to 1302079 tid 1 2026-02-17T05:57:41.283 INFO:tasks.rados.rados.0.trial115.stdout:2002: writing trial11545420-8 from 2000022 to 2168268 tid 2 2026-02-17T05:57:41.283 INFO:tasks.rados.rados.0.trial115.stdout:2003: snap_remove snap 192 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:2003: done (5 left) 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:2004: delete oid 5 current snap is 199 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:1997: finishing rollback tid 0 to trial11545420-26 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:1997: finishing rollback tid 1 to trial11545420-26 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:1997: finishing rollback tid 2 to trial11545420-26 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 614 (ObjNum 599 snap 155 seq_num 599) dirty exists 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:2000: finishing copy_from to trial11545420-50 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:2000: got expected ENOENT (src dne) 2026-02-17T05:57:41.310 INFO:tasks.rados.rados.0.trial115.stdout:2000: finishing copy_from racing read to trial11545420-50 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:1999: finishing copy_from racing read to trial11545420-48 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:1999: finishing copy_from to trial11545420-48 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 871 (ObjNum 716 snap 196 seq_num 716) dirty exists 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2001: finishing write tid 1 to trial11545420-19 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2001: finishing write tid 2 to trial11545420-19 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2001: finishing write tid 3 to trial11545420-19 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2001: finishing write tid 4 to trial11545420-19 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2001: oid 19 updating version 0 to 1076 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2001: oid 19 version 1076 is already newer than 1075 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2001: oid 19 updating version 1076 to 1077 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1077 (ObjNum 723 snap 199 seq_num 723) dirty exists 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2001: left oid 19 (ObjNum 723 snap 199 seq_num 723) 2026-02-17T05:57:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2002: finishing write tid 1 to trial11545420-8 2026-02-17T05:57:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2002: finishing write tid 2 to trial11545420-8 2026-02-17T05:57:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2002: finishing write tid 3 to trial11545420-8 2026-02-17T05:57:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2002: finishing write tid 4 to trial11545420-8 2026-02-17T05:57:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2002: oid 8 updating version 0 to 224 2026-02-17T05:57:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2002: oid 8 updating version 224 to 225 2026-02-17T05:57:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2002: oid 8 updating version 225 to 226 2026-02-17T05:57:41.312 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 226 (ObjNum 724 snap 199 seq_num 724) dirty exists 2026-02-17T05:57:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2002: left oid 8 (ObjNum 724 snap 199 seq_num 724) 2026-02-17T05:57:41.313 INFO:tasks.rados.rados.0.trial115.stdout:1997: done (5 left) 2026-02-17T05:57:41.313 INFO:tasks.rados.rados.0.trial115.stdout:1999: done (4 left) 2026-02-17T05:57:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2000: done (3 left) 2026-02-17T05:57:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2001: done (2 left) 2026-02-17T05:57:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2002: done (1 left) 2026-02-17T05:57:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2004: done (0 left) 2026-02-17T05:57:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2005: snap_create 2026-02-17T05:57:42.291 INFO:tasks.rados.rados.0.trial115.stdout:2005: done (0 left) 2026-02-17T05:57:42.292 INFO:tasks.rados.rados.0.trial115.stdout:2006: snap_create 2026-02-17T05:57:43.292 INFO:tasks.rados.rados.0.trial115.stdout:2006: done (0 left) 2026-02-17T05:57:43.292 INFO:tasks.rados.rados.0.trial115.stdout:2007: snap_remove snap 194 2026-02-17T05:57:44.292 INFO:tasks.rados.rados.0.trial115.stdout:2007: done (0 left) 2026-02-17T05:57:44.292 INFO:tasks.rados.rados.0.trial115.stdout:2008: copy_from oid 38 from oid 19 current snap is 201 2026-02-17T05:57:44.293 INFO:tasks.rados.rados.0.trial115.stdout:2009: snap_remove snap 198 2026-02-17T05:57:45.294 INFO:tasks.rados.rados.0.trial115.stdout:2009: done (1 left) 2026-02-17T05:57:45.294 INFO:tasks.rados.rados.0.trial115.stdout:2010: copy_from oid 44 from oid 6 current snap is 201 2026-02-17T05:57:45.295 INFO:tasks.rados.rados.0.trial115.stdout:2008: finishing copy_from racing read to trial11545420-38 2026-02-17T05:57:45.295 INFO:tasks.rados.rados.0.trial115.stdout:2008: finishing copy_from to trial11545420-38 2026-02-17T05:57:45.295 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1077 (ObjNum 723 snap 199 seq_num 723) dirty exists 2026-02-17T05:57:45.295 INFO:tasks.rados.rados.0.trial115.stdout:2008: done (1 left) 2026-02-17T05:57:45.295 INFO:tasks.rados.rados.0.trial115.stdout:2011: write oid 37 current snap is 201 2026-02-17T05:57:45.295 INFO:tasks.rados.rados.0.trial115.stdout:2011: seq_num 726 ranges {601882=484309,1600633=524904} 2026-02-17T05:57:45.298 INFO:tasks.rados.rados.0.trial115.stdout:2011: writing trial11545420-37 from 601882 to 1086191 tid 1 2026-02-17T05:57:45.302 INFO:tasks.rados.rados.0.trial115.stdout:2011: writing trial11545420-37 from 1600633 to 2125537 tid 2 2026-02-17T05:57:45.302 INFO:tasks.rados.rados.0.trial115.stdout:2012: read oid 26 snap 189 2026-02-17T05:57:45.302 INFO:tasks.rados.rados.0.trial115.stdout:2012: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:57:45.303 INFO:tasks.rados.rados.0.trial115.stdout:2013: snap_remove snap 200 2026-02-17T05:57:46.299 INFO:tasks.rados.rados.0.trial115.stdout:2013: done (3 left) 2026-02-17T05:57:46.299 INFO:tasks.rados.rados.0.trial115.stdout:2014: rollback oid 41 current snap is 201 2026-02-17T05:57:46.299 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 191 2026-02-17T05:57:46.300 INFO:tasks.rados.rados.0.trial115.stdout:2011: finishing write tid 1 to trial11545420-37 2026-02-17T05:57:46.300 INFO:tasks.rados.rados.0.trial115.stdout:2010: finishing copy_from racing read to trial11545420-44 2026-02-17T05:57:46.300 INFO:tasks.rados.rados.0.trial115.stdout:2012: expect (ObjNum 599 snap 155 seq_num 599) 2026-02-17T05:57:46.377 INFO:tasks.rados.rados.0.trial115.stdout:2011: finishing write tid 2 to trial11545420-37 2026-02-17T05:57:46.377 INFO:tasks.rados.rados.0.trial115.stdout:2011: finishing write tid 3 to trial11545420-37 2026-02-17T05:57:46.377 INFO:tasks.rados.rados.0.trial115.stdout:2011: finishing write tid 4 to trial11545420-37 2026-02-17T05:57:46.377 INFO:tasks.rados.rados.0.trial115.stdout:2011: oid 37 updating version 0 to 928 2026-02-17T05:57:46.377 INFO:tasks.rados.rados.0.trial115.stdout:2011: oid 37 updating version 928 to 930 2026-02-17T05:57:46.377 INFO:tasks.rados.rados.0.trial115.stdout:2011: oid 37 version 930 is already newer than 929 2026-02-17T05:57:46.377 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 930 (ObjNum 726 snap 201 seq_num 726) dirty exists 2026-02-17T05:57:46.377 INFO:tasks.rados.rados.0.trial115.stdout:2011: left oid 37 (ObjNum 726 snap 201 seq_num 726) 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2010: finishing copy_from to trial11545420-44 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 829 (ObjNum 657 snap 178 seq_num 657) dirty exists 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2014: finishing rollback tid 0 to trial11545420-41 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2014: finishing rollback tid 1 to trial11545420-41 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2014: finishing rollback tid 2 to trial11545420-41 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1056 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2010: done (3 left) 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2011: done (2 left) 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2012: done (1 left) 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2014: done (0 left) 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2015: write oid 5 current snap is 201 2026-02-17T05:57:46.378 INFO:tasks.rados.rados.0.trial115.stdout:2015: seq_num 727 ranges {618689=559558,1659115=559574} 2026-02-17T05:57:46.381 INFO:tasks.rados.rados.0.trial115.stdout:2015: writing trial11545420-5 from 618689 to 1178247 tid 1 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2015: writing trial11545420-5 from 1659115 to 2218689 tid 2 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2015: finishing write tid 1 to trial11545420-5 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2016: read oid 6 snap -1 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2016: expect (ObjNum 657 snap 178 seq_num 657) 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2017: read oid 25 snap -1 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2017: expect (ObjNum 709 snap 194 seq_num 709) 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2018: copy_from oid 50 from oid 37 current snap is 201 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2019: rollback oid 4 current snap is 201 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 197 2026-02-17T05:57:46.386 INFO:tasks.rados.rados.0.trial115.stdout:2020: snap_create 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:2019: finishing rollback tid 1 to trial11545420-4 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 892 (ObjNum 177 snap 0 seq_num 1333877792) dirty dne 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:2015: finishing write tid 2 to trial11545420-5 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:2015: finishing write tid 3 to trial11545420-5 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:2015: finishing write tid 4 to trial11545420-5 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:2015: oid 5 updating version 0 to 934 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:2015: oid 5 updating version 934 to 935 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:2015: oid 5 updating version 935 to 936 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 936 (ObjNum 727 snap 201 seq_num 727) dirty exists 2026-02-17T05:57:46.388 INFO:tasks.rados.rados.0.trial115.stdout:2015: left oid 5 (ObjNum 727 snap 201 seq_num 727) 2026-02-17T05:57:46.389 INFO:tasks.rados.rados.0.trial115.stdout:2016: expect (ObjNum 657 snap 178 seq_num 657) 2026-02-17T05:57:46.486 INFO:tasks.rados.rados.0.trial115.stdout:2017: expect (ObjNum 709 snap 194 seq_num 709) 2026-02-17T05:57:46.618 INFO:tasks.rados.rados.0.trial115.stdout:2018: finishing copy_from racing read to trial11545420-50 2026-02-17T05:57:46.618 INFO:tasks.rados.rados.0.trial115.stdout:2018: finishing copy_from to trial11545420-50 2026-02-17T05:57:46.618 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 930 (ObjNum 726 snap 201 seq_num 726) dirty exists 2026-02-17T05:57:47.294 INFO:tasks.rados.rados.0.trial115.stdout:2015: done (5 left) 2026-02-17T05:57:47.294 INFO:tasks.rados.rados.0.trial115.stdout:2016: done (4 left) 2026-02-17T05:57:47.294 INFO:tasks.rados.rados.0.trial115.stdout:2017: done (3 left) 2026-02-17T05:57:47.294 INFO:tasks.rados.rados.0.trial115.stdout:2018: done (2 left) 2026-02-17T05:57:47.294 INFO:tasks.rados.rados.0.trial115.stdout:2019: done (1 left) 2026-02-17T05:57:47.294 INFO:tasks.rados.rados.0.trial115.stdout:2020: done (0 left) 2026-02-17T05:57:47.294 INFO:tasks.rados.rados.0.trial115.stdout:2021: delete oid 39 current snap is 202 2026-02-17T05:57:47.297 INFO:tasks.rados.rados.0.trial115.stdout:2021: done (0 left) 2026-02-17T05:57:47.297 INFO:tasks.rados.rados.0.trial115.stdout:2022: delete oid 24 current snap is 202 2026-02-17T05:57:47.298 INFO:tasks.rados.rados.0.trial115.stdout:2022: done (0 left) 2026-02-17T05:57:47.298 INFO:tasks.rados.rados.0.trial115.stdout:2023: read oid 14 snap -1 2026-02-17T05:57:47.298 INFO:tasks.rados.rados.0.trial115.stdout:2023: expect deleted 2026-02-17T05:57:47.299 INFO:tasks.rados.rados.0.trial115.stdout:2024: snap_remove snap 163 2026-02-17T05:57:47.888 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:57:47.888 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T05:57:47.888 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:57:47.889 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:57:47.892 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:57:47.892 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:57:47.892 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:47.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient: get_monmap_and_config 2026-02-17T05:57:47.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient: build_initial_monmap 2026-02-17T05:57:47.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient: monmap: 2026-02-17T05:57:47.961 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:47.961 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:47.961 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:47.958395+0000 2026-02-17T05:57:47.961 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:47.958395+0000 2026-02-17T05:57:47.961 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:47.961 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 1 Processor -- start 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 1 -- start start 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient: init 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2026-02-17T05:57:47.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f50a006c890 0x7f50a00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient(hunting): picked mon.noname-c con 0x7f50a006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient(hunting): picked mon.noname-b con 0x7f50a006cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f50a0134df0 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a0074150 con 0x7f50a0134df0 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f50a009ef70 con 0x7f50a006c890 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f50a009f250 con 0x7f50a006cc70 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:47.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.958+0000 7f50a50bf640 10 monclient(hunting): authenticate will time out at 1313.511970s 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a006d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37918/0 (socket says 10.20.193.78:37918) 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 1 -- 10.20.193.78:0/1009332005 learned_addr learned my addr 10.20.193.78:0/1009332005 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient(hunting): get_auth_request con 0x7f50a006cc70 auth_method 0 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509ed76640 1 --2- 10.20.193.78:0/1009332005 >> v2:10.20.193.115:3300/0 conn(0x7f50a006c890 0x7f50a00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509ed76640 10 monclient(hunting): get_auth_request con 0x7f50a006c890 auth_method 0 2026-02-17T05:57:47.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient(hunting): handle_auth_done global_id 144167 payload 274 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 1 -- 10.20.193.78:0/1009332005 >> v2:10.20.193.115:3300/0 conn(0x7f50a006c890 msgr2=0x7f50a00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 1 --2- 10.20.193.78:0/1009332005 >> v2:10.20.193.115:3300/0 conn(0x7f50a006c890 0x7f50a00a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 1 -- 10.20.193.78:0/1009332005 >> v1:10.20.193.78:6789/0 conn(0x7f50a0134df0 legacy=0x7f50a0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient: _finish_hunting 0 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 1 monclient: found mon.noname-b 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:57:47.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.959+0000 7f509e575640 1 -- 10.20.193.78:0/1009332005 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a009df40 con 0x7f50a006cc70 2026-02-17T05:57:47.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509e575640 1 -- 10.20.193.78:0/1009332005 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 msgr2=0x7f50a006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:47.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509e575640 1 --2- 10.20.193.78:0/1009332005 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a006d050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5088009970 tx=0x7f508802efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a30521dcbfe79c8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 1 -- 10.20.193.78:0/1009332005 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508803c070 con 0x7f50a006cc70 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: dump: 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:47.974 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: _finish_auth 0 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: _check_auth_tickets 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 1 -- 10.20.193.78:0/1009332005 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508802fdf0 con 0x7f50a006cc70 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f50a50bf640 5 monclient: authenticate success, global_id 144167 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 1 -- 10.20.193.78:0/1009332005 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088037660 con 0x7f50a006cc70 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f50a50bf640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:47.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f50a50bf640 4 set_mon_vals no callback set 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.961+0000 7f509dd74640 10 monclient: dump: 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:47.976 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient: shutdown 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- 10.20.193.78:0/1009332005 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 msgr2=0x7f50a006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 --2- 10.20.193.78:0/1009332005 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a006d050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5088009970 tx=0x7f508802efb0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- 10.20.193.78:0/1009332005 shutdown_connections 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 --2- 10.20.193.78:0/1009332005 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a006d050 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 --2- 10.20.193.78:0/1009332005 >> v2:10.20.193.115:3300/0 conn(0x7f50a006c890 0x7f50a00a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- 10.20.193.78:0/1009332005 >> 10.20.193.78:0/1009332005 conn(0x7f50a00a6620 msgr2=0x7f50a00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- 10.20.193.78:0/1009332005 shutdown_connections 2026-02-17T05:57:47.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- 10.20.193.78:0/1009332005 wait complete. 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient: build_initial_monmap 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient: monmap: 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:47.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 Processor -- start 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- start start 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient: init 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 0x7f50a0134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient(hunting): picked mon.noname-a con 0x7f50a006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a0134750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient(hunting): picked mon.noname-c con 0x7f50a006cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a0134df0 0x7f50a0124e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient(hunting): picked mon.noname-b con 0x7f50a0134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50a009d360 con 0x7f50a006c890 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50a009d1e0 con 0x7f50a0134df0 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f50a0135950 con 0x7f50a006cc70 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f50a50bf640 10 monclient(hunting): authenticate will time out at 1313.515939s 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f509e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a0134750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f509ed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 0x7f50a0134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 0x7f50a0134210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48082/0 (socket says 10.20.193.78:48082) 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 1 -- 10.20.193.78:0/3377922896 learned_addr learned my addr 10.20.193.78:0/3377922896 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:47.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 10 monclient(hunting): get_auth_request con 0x7f50a006c890 auth_method 0 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.962+0000 7f509e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a0134750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37920/0 (socket says 10.20.193.78:37920) 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient(hunting): get_auth_request con 0x7f50a006cc70 auth_method 0 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 1 -- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 msgr2=0x7f50a0134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 1 -- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 msgr2=0x7f50a0134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 0x7f50a0134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509ed76640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 0x7f50a0134210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509f577640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a0134df0 0x7f50a0124e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509f577640 10 monclient(hunting): get_auth_request con 0x7f50a0134df0 auth_method 0 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509f577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509f577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509f577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient(hunting): handle_auth_done global_id 144170 payload 824 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 1 -- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a0134df0 msgr2=0x7f50a0124e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a0134df0 0x7f50a0124e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 1 -- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 msgr2=0x7f50a0134210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 0x7f50a0134210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient: _finish_hunting 0 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 1 monclient: found mon.noname-c 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.963+0000 7f509e575640 1 -- 10.20.193.78:0/3377922896 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a0128780 con 0x7f50a006cc70 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f509e575640 1 -- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 msgr2=0x7f50a0134750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f509e575640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a0134750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50a009d580 tx=0x7f5088004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 1 -- 10.20.193.78:0/3377922896 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508803c040 con 0x7f50a006cc70 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: dump: 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:47.983 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 1 -- 10.20.193.78:0/3377922896 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508802fbf0 con 0x7f50a006cc70 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 1 -- 10.20.193.78:0/3377922896 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880408a0 con 0x7f50a006cc70 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:47.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f509d573640 4 set_mon_vals no callback set 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f507f7fe640 10 monclient: dump: 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f50a50bf640 5 monclient: authenticate success, global_id 144170 2026-02-17T05:57:47.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f50a50bf640 10 monclient: _renew_subs 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f50a50bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a0128a60 con 0x7f50a006cc70 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f50a50bf640 10 monclient: _renew_subs 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f50a50bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.965+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a0072c80 con 0x7f50a006cc70 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f507f7fe640 1 -- 10.20.193.78:0/3377922896 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f508803f070 con 0x7f50a006cc70 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f50a50bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f50a50bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f50a50bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a0068410 con 0x7f50a006cc70 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f507f7fe640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f506c03dee0 0x7f506c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f507f7fe640 1 -- 10.20.193.78:0/3377922896 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(423..423 src has 335..423) ==== 4811+0+0 (secure 0 0 0) 0x7f5088076e60 con 0x7f50a006cc70 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f509ed76640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f506c03dee0 0x7f506c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.966+0000 7f509ed76640 10 monclient: get_auth_request con 0x7f506c03dee0 auth_method 0 2026-02-17T05:57:47.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.969+0000 7f509ed76640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f506c03dee0 0x7f506c0403a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f50940092c0 tx=0x7f5094008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:47.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.969+0000 7f507f7fe640 1 -- 10.20.193.78:0/3377922896 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f50a0068410 con 0x7f50a006cc70 2026-02-17T05:57:47.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.969+0000 7f507f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:47.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:47.969+0000 7f507f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.089+0000 7f50a50bf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.089+0000 7f50a50bf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.089+0000 7f50a50bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:57:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.089+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f50a0126a50 con 0x7f50a006cc70 2026-02-17T05:57:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.089+0000 7f507f7fe640 1 -- 10.20.193.78:0/3377922896 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f508803e2f0 con 0x7f50a006cc70 2026-02-17T05:57:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.089+0000 7f507f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:48.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.089+0000 7f507f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:48.091 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":124,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:48.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f506c03dee0 msgr2=0x7f506c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:48.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f506c03dee0 0x7f506c0403a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f50940092c0 tx=0x7f5094008040 comp rx=0 tx=0).stop 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 10 monclient: shutdown 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 msgr2=0x7f50a0134750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a0134750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50a009d580 tx=0x7f5088004830 comp rx=0 tx=0).stop 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 shutdown_connections 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f506c03dee0 0x7f506c0403a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a0134df0 0x7f50a0124e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a006cc70 0x7f50a0134750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 --2- 10.20.193.78:0/3377922896 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a006c890 0x7f50a0134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 >> 10.20.193.78:0/3377922896 conn(0x7f50a00a6620 msgr2=0x7f50a009c720 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:48.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 shutdown_connections 2026-02-17T05:57:48.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:48.090+0000 7f50a50bf640 1 -- 10.20.193.78:0/3377922896 wait complete. 2026-02-17T05:57:48.302 INFO:tasks.rados.rados.0.trial115.stdout:2024: done (1 left) 2026-02-17T05:57:48.302 INFO:tasks.rados.rados.0.trial115.stdout:2025: write oid 32 current snap is 202 2026-02-17T05:57:48.302 INFO:tasks.rados.rados.0.trial115.stdout:2025: seq_num 730 ranges {669110=501658,1834561=434549} 2026-02-17T05:57:48.306 INFO:tasks.rados.rados.0.trial115.stdout:2025: writing trial11545420-32 from 669110 to 1170768 tid 1 2026-02-17T05:57:48.311 INFO:tasks.rados.rados.0.trial115.stdout:2025: writing trial11545420-32 from 1834561 to 2269110 tid 2 2026-02-17T05:57:48.311 INFO:tasks.rados.rados.0.trial115.stdout:2026: rollback oid 27 current snap is 202 2026-02-17T05:57:48.311 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 195 2026-02-17T05:57:48.311 INFO:tasks.rados.rados.0.trial115.stdout:2025: finishing write tid 1 to trial11545420-32 2026-02-17T05:57:48.311 INFO:tasks.rados.rados.0.trial115.stdout:2023: done (2 left) 2026-02-17T05:57:48.311 INFO:tasks.rados.rados.0.trial115.stdout:2027: delete oid 50 current snap is 202 2026-02-17T05:57:48.312 INFO:tasks.rados.rados.0.trial115.stdout:2026: finishing rollback tid 1 to trial11545420-27 2026-02-17T05:57:48.312 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 853 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:57:48.313 INFO:tasks.rados.rados.0.trial115.stdout:2026: done (2 left) 2026-02-17T05:57:48.313 INFO:tasks.rados.rados.0.trial115.stdout:2027: done (1 left) 2026-02-17T05:57:48.313 INFO:tasks.rados.rados.0.trial115.stdout:2028: rollback oid 47 current snap is 202 2026-02-17T05:57:48.313 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 47 to 186 2026-02-17T05:57:48.313 INFO:tasks.rados.rados.0.trial115.stdout:2029: copy_from oid 14 from oid 5 current snap is 202 2026-02-17T05:57:48.313 INFO:tasks.rados.rados.0.trial115.stdout:2030: snap_remove snap 195 2026-02-17T05:57:51.100 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:51.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.169+0000 7fb713c3a640 10 monclient: get_monmap_and_config 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.169+0000 7fb713c3a640 10 monclient: build_initial_monmap 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient: monmap: 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:51.170232+0000 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:51.170232+0000 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:51.170 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 1 Processor -- start 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 1 -- start start 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient: init 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb70c09ea90 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient(hunting): picked mon.noname-c con 0x7fb70c09ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient(hunting): picked mon.noname-a con 0x7fb70c13d640 addr v2:10.20.193.78:3300/0 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb70c0a0a20 con 0x7fb70c09ea90 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb70c0a0d00 con 0x7fb70c13d640 2026-02-17T05:57:51.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb70c0a0460 con 0x7fb70c09ee70 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb713c3a640 10 monclient(hunting): authenticate will time out at 1316.723763s 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7119af640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb70c09ea90 0x7fb70c0a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50036/0 (socket says 10.20.193.78:50036) 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7119af640 1 -- 10.20.193.78:0/27233109 learned_addr learned my addr 10.20.193.78:0/27233109 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 10 monclient(hunting): get_auth_request con 0x7fb70c13d640 auth_method 0 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7111ae640 10 monclient(hunting): get_auth_request con 0x7fb70c09ee70 auth_method 0 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7111ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7111ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.170+0000 7fb7111ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.171+0000 7fb7111ae640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.171+0000 7fb7111ae640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.171+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.171+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:51.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.172+0000 7fb7119af640 1 -- 10.20.193.78:0/27233109 >> v1:10.20.193.78:6789/0 conn(0x7fb70c09ea90 legacy=0x7fb70c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:51.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:51.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 10 monclient(hunting): get_auth_request con 0x7fb70c13d640 auth_method 0 2026-02-17T05:57:51.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:51.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 10 monclient(hunting): get_auth_request con 0x7fb70c09ee70 auth_method 0 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:51.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:57:51.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:51.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:51.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.371+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:51.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 10 monclient(hunting): get_auth_request con 0x7fb70c13d640 auth_method 0 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.771+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:51.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 10 monclient(hunting): get_auth_request con 0x7fb70c09ee70 auth_method 0 2026-02-17T05:57:51.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:51.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:51.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:51.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:51.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:51.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:51.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:51.772+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:52.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:52.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 10 monclient(hunting): get_auth_request con 0x7fb70c13d640 auth_method 0 2026-02-17T05:57:52.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:52.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:52.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:52.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:52.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:52.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:52.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.572+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 10 monclient(hunting): get_auth_request con 0x7fb70c09ee70 auth_method 0 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:52.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:52.573+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:57:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1714692731 0 0) 0x7fb70c0a0a20 con 0x7fb70c09ea90 2026-02-17T05:57:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient(hunting): _negotiate 2026-02-17T05:57:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient(hunting): my global_id is 154098 2026-02-17T05:57:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6f4003620 con 0x7fb70c09ea90 2026-02-17T05:57:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3840475625 0 0) 0x7fb6f4003620 con 0x7fb70c09ea90 2026-02-17T05:57:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 msgr2=0x7fb70c09f2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.115:3300/0 conn(0x7fb70c09ee70 0x7fb70c09f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 msgr2=0x7fb70c13fa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 --2- 10.20.193.78:0/27233109 >> v2:10.20.193.78:3300/0 conn(0x7fb70c13d640 0x7fb70c13fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient: _finish_hunting 0 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c0a0740 con 0x7fb70c09ea90 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient: _finish_auth 0 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient: _check_auth_tickets 2026-02-17T05:57:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fb6fc0031b0 con 0x7fb70c09ea90 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient: dump: 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:53.593 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.582+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 >> v1:10.20.193.78:6789/0 conn(0x7fb70c09ea90 legacy=0x7fb70c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f4003c20 0x7fb6f4004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient(hunting): picked mon.a con 0x7fb6f4003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4004900 0x7fb6f4004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient(hunting): picked mon.c con 0x7fb6f4004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:53.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f40054a0 0x7fb6f400bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient(hunting): picked mon.b con 0x7fb6f40054a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c0a0d00 con 0x7fb6f4003c20 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c0a0460 con 0x7fb6f40054a0 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f400c460 con 0x7fb6f4004900 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7109ad640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f4003c20 0x7fb6f4004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7111ae640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4004900 0x7fb6f4004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:53.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient(hunting): get_auth_request con 0x7fb6f4003c20 auth_method 0 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7111ae640 10 monclient(hunting): get_auth_request con 0x7fb6f4004900 auth_method 0 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7111ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7111ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7111ae640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7111ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7111ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:53.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7111ae640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7121b0640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f40054a0 0x7fb6f400bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7121b0640 10 monclient(hunting): get_auth_request con 0x7fb6f40054a0 auth_method 0 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7121b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7121b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7121b0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient(hunting): handle_auth_done global_id 154098 payload 293 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 1 -- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4004900 msgr2=0x7fb6f4004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4004900 0x7fb6f4004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 1 -- 10.20.193.78:0/27233109 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f40054a0 msgr2=0x7fb6f400bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f40054a0 0x7fb6f400bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient: _finish_hunting 0 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 1 monclient: found mon.a 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7121b0640 1 -- 10.20.193.78:0/27233109 reap_dead start 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 1 -- 10.20.193.78:0/27233109 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f4003620 con 0x7fb6f4003c20 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient: _finish_auth 0 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 10 monclient: _check_auth_tickets 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb7119af640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb713c3a640 5 monclient: authenticate success, global_id 154098 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.583+0000 7fb713c3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7119af640 1 -- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f4003c20 msgr2=0x7fb6f4004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:53.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7119af640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f4003c20 0x7fb6f4004040 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0033d0 tx=0x7fb6fc02f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ca95915f1b7c53 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00c200 con 0x7fb6f4003c20 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 10 monclient: dump: 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:53.599 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6fc005c90 con 0x7fb6f4003c20 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 1 -- 10.20.193.78:0/27233109 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc007310 con 0x7fb6f4003c20 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb713c3a640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb713c3a640 4 set_mon_vals no callback set 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:53.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.585+0000 7fb7109ad640 10 monclient: dump: 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:53.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 10 monclient: shutdown 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 1 -- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f4003c20 msgr2=0x7fb6f4004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f4003c20 0x7fb6f4004040 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0033d0 tx=0x7fb6fc02f7a0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 1 -- 10.20.193.78:0/27233109 shutdown_connections 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 1 --2- 10.20.193.78:0/27233109 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f4003c20 0x7fb6f4004040 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 1 -- 10.20.193.78:0/27233109 >> 10.20.193.78:0/27233109 conn(0x7fb70c0a7520 msgr2=0x7fb70c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 1 -- 10.20.193.78:0/27233109 shutdown_connections 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 1 -- 10.20.193.78:0/27233109 wait complete. 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 10 monclient: build_initial_monmap 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.586+0000 7fb713c3a640 10 monclient: monmap: 2026-02-17T05:57:53.602 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:53.603 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 1 Processor -- start 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 1 -- start start 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient: init 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 0x7fb70c1cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient(hunting): picked mon.noname-c con 0x7fb6f4003c20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb70c1cfed0 0x7fb70c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient(hunting): picked mon.noname-a con 0x7fb70c1cfed0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb70c1d0330 0x7fb70c070520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:53.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient(hunting): picked mon.noname-b con 0x7fb70c1d0330 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c0a1550 con 0x7fb70c1cfed0 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c0a13d0 con 0x7fb70c1d0330 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c0a16d0 con 0x7fb6f4003c20 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb713c3a640 10 monclient(hunting): authenticate will time out at 1319.140351s 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 0x7fb70c1cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb70c1cfed0 0x7fb70c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 0x7fb70c1cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37972/0 (socket says 10.20.193.78:37972) 2026-02-17T05:57:53.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb70c1cfed0 0x7fb70c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:48116/0 (socket says 10.20.193.78:48116) 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 1 -- 10.20.193.78:0/3412580910 learned_addr learned my addr 10.20.193.78:0/3412580910 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 10 monclient(hunting): get_auth_request con 0x7fb6f4003c20 auth_method 0 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient(hunting): get_auth_request con 0x7fb70c1cfed0 auth_method 0 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:53.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 1 -- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 msgr2=0x7fb70c1cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 1 -- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 msgr2=0x7fb70c1cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 0x7fb70c1cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7119af640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 0x7fb70c1cf990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient(hunting): handle_auth_done global_id 154101 payload 824 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 1 -- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 msgr2=0x7fb70c1cf990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 0x7fb70c1cf990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 1 -- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb70c1d0330 msgr2=0x7fb70c070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb70c1d0330 0x7fb70c070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient: _finish_hunting 0 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 1 monclient: found mon.noname-a 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:53.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.587+0000 7fb7111ae640 1 -- 10.20.193.78:0/3412580910 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c071700 con 0x7fb70c1cfed0 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb7111ae640 1 -- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb70c1cfed0 msgr2=0x7fb70c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb7111ae640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb70c1cfed0 0x7fb70c06ffe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb700003ea0 tx=0x7fb700012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 1 -- 10.20.193.78:0/3412580910 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000f040 con 0x7fb70c1cfed0 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: dump: 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.608 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: _finish_auth 0 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 1 -- 10.20.193.78:0/3412580910 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb700017990 con 0x7fb70c1cfed0 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:53.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 1 -- 10.20.193.78:0/3412580910 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700017d60 con 0x7fb70c1cfed0 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fa7fc640 10 monclient: dump: 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:53.610 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb6fbfff640 4 set_mon_vals no callback set 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb713c3a640 5 monclient: authenticate success, global_id 154101 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb713c3a640 10 monclient: _renew_subs 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb713c3a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c0a9ee0 con 0x7fb70c1cfed0 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb713c3a640 10 monclient: _renew_subs 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb713c3a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.589+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c073a60 con 0x7fb70c1cfed0 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb713c3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb713c3a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb713c3a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:53.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c0aa430 con 0x7fb70c1cfed0 2026-02-17T05:57:53.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb6fa7fc640 1 -- 10.20.193.78:0/3412580910 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb700015040 con 0x7fb70c1cfed0 2026-02-17T05:57:53.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb6fa7fc640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03df00 0x7fb6dc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:53.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb6fa7fc640 1 -- 10.20.193.78:0/3412580910 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(424..424 src has 335..424) ==== 4811+0+0 (secure 0 0 0) 0x7fb700053450 con 0x7fb70c1cfed0 2026-02-17T05:57:53.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb7119af640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03df00 0x7fb6dc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:53.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.590+0000 7fb7119af640 10 monclient: get_auth_request con 0x7fb6dc03df00 auth_method 0 2026-02-17T05:57:53.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.593+0000 7fb6fa7fc640 1 -- 10.20.193.78:0/3412580910 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb70c0aa430 con 0x7fb70c1cfed0 2026-02-17T05:57:53.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.593+0000 7fb6fa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:53.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.593+0000 7fb6fa7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:53.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.593+0000 7fb7119af640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03df00 0x7fb6dc0403c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc002cb0 tx=0x7fb6fc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:53.635 INFO:tasks.rados.rados.0.trial115.stdout:2030: done (3 left) 2026-02-17T05:57:53.635 INFO:tasks.rados.rados.0.trial115.stdout:2031: read oid 9 snap -1 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2031: expect (ObjNum 674 snap 186 seq_num 674) 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2032: read oid 23 snap -1 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2032: expect (ObjNum 703 snap 194 seq_num 703) 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2033: read oid 50 snap -1 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2033: expect deleted 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2034: delete oid 15 current snap is 202 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2025: finishing write tid 2 to trial11545420-32 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2025: finishing write tid 3 to trial11545420-32 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2025: finishing write tid 4 to trial11545420-32 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2025: oid 32 updating version 0 to 872 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2025: oid 32 updating version 872 to 873 2026-02-17T05:57:53.636 INFO:tasks.rados.rados.0.trial115.stdout:2025: oid 32 updating version 873 to 874 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 874 (ObjNum 730 snap 202 seq_num 730) dirty exists 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:2025: left oid 32 (ObjNum 730 snap 202 seq_num 730) 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:2028: finishing rollback tid 0 to trial11545420-47 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:2028: finishing rollback tid 1 to trial11545420-47 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:2028: finishing rollback tid 2 to trial11545420-47 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 944 (ObjNum 598 snap 155 seq_num 598) dirty exists 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:2029: finishing copy_from racing read to trial11545420-14 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:2029: finishing copy_from to trial11545420-14 2026-02-17T05:57:53.637 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 936 (ObjNum 727 snap 201 seq_num 727) dirty exists 2026-02-17T05:57:53.641 INFO:tasks.rados.rados.0.trial115.stdout:2031: expect (ObjNum 674 snap 186 seq_num 674) 2026-02-17T05:57:53.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.708+0000 7fb713c3a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:53.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.708+0000 7fb713c3a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:53.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.708+0000 7fb713c3a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:53.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.708+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb70c0a1c10 con 0x7fb70c1cfed0 2026-02-17T05:57:53.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.708+0000 7fb6fa7fc640 1 -- 10.20.193.78:0/3412580910 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fb70c0a9ee0 con 0x7fb70c1cfed0 2026-02-17T05:57:53.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.708+0000 7fb6fa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:53.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.708+0000 7fb6fa7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:53.710 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":128,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:53.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03df00 msgr2=0x7fb6dc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:53.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03df00 0x7fb6dc0403c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc002cb0 tx=0x7fb6fc009000 comp rx=0 tx=0).stop 2026-02-17T05:57:53.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 10 monclient: shutdown 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb70c1cfed0 msgr2=0x7fb70c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb70c1cfed0 0x7fb70c06ffe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb700003ea0 tx=0x7fb700012a00 comp rx=0 tx=0).stop 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 shutdown_connections 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03df00 0x7fb6dc0403c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb70c1d0330 0x7fb70c070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb70c1cfed0 0x7fb70c06ffe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 --2- 10.20.193.78:0/3412580910 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f4003c20 0x7fb70c1cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 >> 10.20.193.78:0/3412580910 conn(0x7fb70c0a7520 msgr2=0x7fb70c0a4790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 shutdown_connections 2026-02-17T05:57:53.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:53.709+0000 7fb713c3a640 1 -- 10.20.193.78:0/3412580910 wait complete. 2026-02-17T05:57:53.775 INFO:tasks.rados.rados.0.trial115.stdout:2032: expect (ObjNum 703 snap 194 seq_num 703) 2026-02-17T05:57:53.968 INFO:tasks.rados.rados.0.trial115.stdout:2025: done (6 left) 2026-02-17T05:57:53.968 INFO:tasks.rados.rados.0.trial115.stdout:2028: done (5 left) 2026-02-17T05:57:53.968 INFO:tasks.rados.rados.0.trial115.stdout:2029: done (4 left) 2026-02-17T05:57:53.968 INFO:tasks.rados.rados.0.trial115.stdout:2031: done (3 left) 2026-02-17T05:57:53.968 INFO:tasks.rados.rados.0.trial115.stdout:2032: done (2 left) 2026-02-17T05:57:53.969 INFO:tasks.rados.rados.0.trial115.stdout:2033: done (1 left) 2026-02-17T05:57:53.969 INFO:tasks.rados.rados.0.trial115.stdout:2034: done (0 left) 2026-02-17T05:57:53.969 INFO:tasks.rados.rados.0.trial115.stdout:2035: read oid 36 snap -1 2026-02-17T05:57:53.969 INFO:tasks.rados.rados.0.trial115.stdout:2035: expect deleted 2026-02-17T05:57:53.969 INFO:tasks.rados.rados.0.trial115.stdout:2036: write (excl) oid 3 current snap is 202 2026-02-17T05:57:53.969 INFO:tasks.rados.rados.0.trial115.stdout:2036: seq_num 733 ranges {719531=727405,1893654=425877} 2026-02-17T05:57:53.973 INFO:tasks.rados.rados.0.trial115.stdout:2036: writing trial11545420-3 from 719531 to 1446936 tid 1 2026-02-17T05:57:53.977 INFO:tasks.rados.rados.0.trial115.stdout:2036: writing trial11545420-3 from 1893654 to 2319531 tid 2 2026-02-17T05:57:53.977 INFO:tasks.rados.rados.0.trial115.stdout:2037: rollback oid 45 current snap is 202 2026-02-17T05:57:53.977 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 199 2026-02-17T05:57:53.978 INFO:tasks.rados.rados.0.trial115.stdout:2036: finishing write tid 1 to trial11545420-3 2026-02-17T05:57:53.978 INFO:tasks.rados.rados.0.trial115.stdout:2035: done (2 left) 2026-02-17T05:57:53.978 INFO:tasks.rados.rados.0.trial115.stdout:2038: read oid 25 snap 189 2026-02-17T05:57:53.978 INFO:tasks.rados.rados.0.trial115.stdout:2038: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:57:53.978 INFO:tasks.rados.rados.0.trial115.stdout:2039: rollback oid 20 current snap is 202 2026-02-17T05:57:53.978 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 201 2026-02-17T05:57:53.978 INFO:tasks.rados.rados.0.trial115.stdout:2040: write (excl) oid 40 current snap is 202 2026-02-17T05:57:53.978 INFO:tasks.rados.rados.0.trial115.stdout:2040: seq_num 734 ranges {736338=402654,1552136=539580} 2026-02-17T05:57:53.981 INFO:tasks.rados.rados.0.trial115.stdout:2040: writing trial11545420-40 from 736338 to 1138992 tid 1 2026-02-17T05:57:53.986 INFO:tasks.rados.rados.0.trial115.stdout:2040: writing trial11545420-40 from 1552136 to 2091716 tid 2 2026-02-17T05:57:53.986 INFO:tasks.rados.rados.0.trial115.stdout:2041: write (excl) oid 9 current snap is 202 2026-02-17T05:57:53.986 INFO:tasks.rados.rados.0.trial115.stdout:2041: seq_num 735 ranges {753145=477903,2010618=342527} 2026-02-17T05:57:53.990 INFO:tasks.rados.rados.0.trial115.stdout:2041: writing trial11545420-9 from 753145 to 1231048 tid 1 2026-02-17T05:57:53.995 INFO:tasks.rados.rados.0.trial115.stdout:2041: writing trial11545420-9 from 2010618 to 2353145 tid 2 2026-02-17T05:57:53.995 INFO:tasks.rados.rados.0.trial115.stdout:2042: snap_create 2026-02-17T05:57:53.995 INFO:tasks.rados.rados.0.trial115.stdout:2039: finishing rollback tid 1 to trial11545420-20 2026-02-17T05:57:53.995 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 927 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:57:53.995 INFO:tasks.rados.rados.0.trial115.stdout:2036: finishing write tid 2 to trial11545420-3 2026-02-17T05:57:53.995 INFO:tasks.rados.rados.0.trial115.stdout:2036: finishing write tid 3 to trial11545420-3 2026-02-17T05:57:53.995 INFO:tasks.rados.rados.0.trial115.stdout:2036: finishing write tid 4 to trial11545420-3 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2036: oid 3 updating version 0 to 948 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2036: oid 3 updating version 948 to 950 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2036: oid 3 version 950 is already newer than 949 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 950 (ObjNum 733 snap 202 seq_num 733) dirty exists 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2036: left oid 3 (ObjNum 733 snap 202 seq_num 733) 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2037: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2037: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2037: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1164 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2040: finishing write tid 1 to trial11545420-40 2026-02-17T05:57:53.996 INFO:tasks.rados.rados.0.trial115.stdout:2038: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2040: finishing write tid 2 to trial11545420-40 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2040: finishing write tid 3 to trial11545420-40 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2040: finishing write tid 4 to trial11545420-40 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2040: oid 40 updating version 0 to 893 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2040: oid 40 updating version 893 to 895 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2040: oid 40 version 895 is already newer than 894 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 895 (ObjNum 734 snap 202 seq_num 734) dirty exists 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2040: left oid 40 (ObjNum 734 snap 202 seq_num 734) 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2041: finishing write tid 1 to trial11545420-9 2026-02-17T05:57:54.189 INFO:tasks.rados.rados.0.trial115.stdout:2041: finishing write tid 2 to trial11545420-9 2026-02-17T05:57:54.190 INFO:tasks.rados.rados.0.trial115.stdout:2041: finishing write tid 3 to trial11545420-9 2026-02-17T05:57:54.190 INFO:tasks.rados.rados.0.trial115.stdout:2041: finishing write tid 4 to trial11545420-9 2026-02-17T05:57:54.190 INFO:tasks.rados.rados.0.trial115.stdout:2041: oid 9 updating version 0 to 992 2026-02-17T05:57:54.190 INFO:tasks.rados.rados.0.trial115.stdout:2041: oid 9 version 992 is already newer than 991 2026-02-17T05:57:54.190 INFO:tasks.rados.rados.0.trial115.stdout:2041: oid 9 version 992 is already newer than 990 2026-02-17T05:57:54.190 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 992 (ObjNum 735 snap 202 seq_num 735) dirty exists 2026-02-17T05:57:54.190 INFO:tasks.rados.rados.0.trial115.stdout:2041: left oid 9 (ObjNum 735 snap 202 seq_num 735) 2026-02-17T05:57:56.719 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:57:56.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.790+0000 7fd9015e4640 10 monclient: get_monmap_and_config 2026-02-17T05:57:56.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.790+0000 7fd9015e4640 10 monclient: build_initial_monmap 2026-02-17T05:57:56.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.790+0000 7fd9015e4640 10 monclient: monmap: 2026-02-17T05:57:56.793 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:56.793 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:56.793 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:57:56.791075+0000 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:57:56.791075+0000 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 1 Processor -- start 2026-02-17T05:57:56.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 1 -- start start 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient: init 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd8fc06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient(hunting): picked mon.noname-b con 0x7fd8fc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient(hunting): picked mon.noname-c con 0x7fd8fc06d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8fc06daf0 con 0x7fd8fc06c890 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:56.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd8fc0a0880 con 0x7fd8fc06d380 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc09cc80 con 0x7fd8fc09c4a0 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd9015e4640 10 monclient(hunting): authenticate will time out at 1322.344599s 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8faffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd8fc06c890 0x7fd8fc06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50046/0 (socket says 10.20.193.78:50046) 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8faffd640 1 -- 10.20.193.78:0/2098596616 learned_addr learned my addr 10.20.193.78:0/2098596616 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request con 0x7fd8fc09c4a0 auth_method 0 2026-02-17T05:57:56.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd8fc06d380 auth_method 0 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:56.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.791+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:56.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.792+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:57:56.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.792+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:56.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.792+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:56.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.792+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:56.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.793+0000 7fd8faffd640 1 -- 10.20.193.78:0/2098596616 >> v1:10.20.193.78:6789/0 conn(0x7fd8fc06c890 legacy=0x7fd8fc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request con 0x7fd8fc09c4a0 auth_method 0 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd8fc06d380 auth_method 0 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:56.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:56.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:56.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:56.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:56.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:56.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:56.992+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:57:57.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:57.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request con 0x7fd8fc09c4a0 auth_method 0 2026-02-17T05:57:57.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:57.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:57.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:57.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:57.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:57.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.392+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd8fc06d380 auth_method 0 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:57.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:57.393+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:57:58.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request con 0x7fd8fc09c4a0 auth_method 0 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.193+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd8fc06d380 auth_method 0 2026-02-17T05:57:58.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:58.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:58.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:58.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:57:58.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:57:58.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:58.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:58.194+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:57:59.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.583+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381855605 0 0) 0x7fd8fc06daf0 con 0x7fd8fc06c890 2026-02-17T05:57:59.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.583+0000 7fd8f9ffb640 10 monclient(hunting): _negotiate 2026-02-17T05:57:59.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.583+0000 7fd8f9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:59.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.583+0000 7fd8f9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:59.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.583+0000 7fd8f9ffb640 10 monclient(hunting): my global_id is 154104 2026-02-17T05:57:59.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.583+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8e8003620 con 0x7fd8fc06c890 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4124030870 0 0) 0x7fd8e8003620 con 0x7fd8fc06c890 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8fc13b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8fc13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 msgr2=0x7fd8fc13d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 --2- 10.20.193.78:0/2098596616 >> v2:10.20.193.115:3300/0 conn(0x7fd8fc06d380 0x7fd8fc13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: _finish_hunting 0 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:57:59.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc09cf60 con 0x7fd8fc06c890 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fd8f00031b0 con 0x7fd8fc06c890 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd9015e4640 5 monclient: authenticate success, global_id 154104 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd9015e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: dump: 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:59.596 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 >> v1:10.20.193.78:6789/0 conn(0x7fd8fc06c890 legacy=0x7fd8fc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8e8003bd0 0x7fd8e8003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient(hunting): picked mon.a con 0x7fd8e8003bd0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8e8004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient(hunting): picked mon.c con 0x7fd8fc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8004f60 0x7fd8e800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient(hunting): picked mon.b con 0x7fd8e8004f60 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc09cc80 con 0x7fd8e8003bd0 2026-02-17T05:57:59.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e8005750 con 0x7fd8e8004f60 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e800c000 con 0x7fd8fc09c4a0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd8f9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.584+0000 7fd9015e4640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8e8003bd0 0x7fd8e8003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient(hunting): get_auth_request con 0x7fd8e8003bd0 auth_method 0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8e8004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request con 0x7fd8fc09c4a0 auth_method 0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8fa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8fa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8fa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient(hunting): handle_auth_done global_id 154104 payload 293 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 msgr2=0x7fd8e8004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09c4a0 0x7fd8e8004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8004f60 msgr2=0x7fd8e800bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8004f60 0x7fd8e800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient: _finish_hunting 0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 1 monclient: found mon.a 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 1 -- 10.20.193.78:0/2098596616 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e8003620 con 0x7fd8e8003bd0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient: _finish_auth 0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 10 monclient: _check_auth_tickets 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.585+0000 7fd8faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.587+0000 7fd8faffd640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8e8003bd0 msgr2=0x7fd8e8003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.587+0000 7fd8faffd640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8e8003bd0 0x7fd8e8003ff0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c470 tx=0x7fd8f002f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ea242a04304d59 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:59.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000c1a0 con 0x7fd8e8003bd0 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 10 monclient: dump: 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f0005cb0 con 0x7fd8e8003bd0 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 1 -- 10.20.193.78:0/2098596616 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0005e60 con 0x7fd8e8003bd0 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8f9ffb640 10 monclient: dump: 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 10 monclient: get_monmap_and_config success 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 4 set_mon_vals no callback set 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 10 monclient: shutdown 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 1 -- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8e8003bd0 msgr2=0x7fd8e8003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 1 --2- 10.20.193.78:0/2098596616 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8e8003bd0 0x7fd8e8003ff0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c470 tx=0x7fd8f002f7d0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/2098596616 reap_dead start 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 1 -- 10.20.193.78:0/2098596616 shutdown_connections 2026-02-17T05:57:59.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 1 -- 10.20.193.78:0/2098596616 >> 10.20.193.78:0/2098596616 conn(0x7fd8fc0a6620 msgr2=0x7fd8fc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 1 -- 10.20.193.78:0/2098596616 shutdown_connections 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 1 -- 10.20.193.78:0/2098596616 wait complete. 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 10 monclient: build_initial_monmap 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.588+0000 7fd9015e4640 10 monclient: monmap: 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 1 Processor -- start 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 1 -- start start 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient: init 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient: _reopen_session rank -1 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8003bd0 0x7fd8fc09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient(hunting): picked mon.noname-b con 0x7fd8e8003bd0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 0x7fd8fc09e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient(hunting): picked mon.noname-c con 0x7fd8fc09e060 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8fc1d34a0 0x7fd8fc1d5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient(hunting): picked mon.noname-a con 0x7fd8fc1d34a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc09bfa0 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc09be20 con 0x7fd8e8003bd0 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient(hunting): start opening mon connection 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc139390 con 0x7fd8fc09e060 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient(hunting): _renew_subs 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd9015e4640 10 monclient(hunting): authenticate will time out at 1325.142463s 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8fc1d34a0 0x7fd8fc1d5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 0x7fd8fc09e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8fc1d34a0 0x7fd8fc1d5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:59184/0 (socket says 10.20.193.78:59184) 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/4082042505 learned_addr learned my addr 10.20.193.78:0/4082042505 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 0x7fd8fc09e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57500/0 (socket says 10.20.193.78:57500) 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd8fc1d34a0 auth_method 0 2026-02-17T05:57:59.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request con 0x7fd8fc09e060 auth_method 0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 msgr2=0x7fd8fc09e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 1 -- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 msgr2=0x7fd8fc09e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 0x7fd8fc09e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fa7fc640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 0x7fd8fc09e4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8faffd640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8003bd0 0x7fd8fc09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient(hunting): handle_auth_done global_id 154107 payload 824 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 msgr2=0x7fd8fc09e4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 0x7fd8fc09e4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8003bd0 msgr2=0x7fd8fc09db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8003bd0 0x7fd8fc09db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 monclient: found mon.noname-a 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/4082042505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc121ed0 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8faffd640 10 monclient: get_auth_request con 0x7fd8e8003bd0 auth_method 0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.589+0000 7fd8faffd640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8003bd0 0x7fd8fc09db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8fb7fe640 1 -- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8fc1d34a0 msgr2=0x7fd8fc1d5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8fb7fe640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8fc1d34a0 0x7fd8fc1d5890 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc06dcf0 tx=0x7fd8ec004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 1 -- 10.20.193.78:0/4082042505 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec01a020 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: dump: 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:59.601 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: _finish_auth 0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 1 -- 10.20.193.78:0/4082042505 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ec016460 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 1 -- 10.20.193.78:0/4082042505 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec016850 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8db7fe640 10 monclient: dump: 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd8f97fa640 4 set_mon_vals no callback set 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd9015e4640 5 monclient: authenticate success, global_id 154107 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd9015e4640 10 monclient: _renew_subs 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd9015e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.591+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc1221b0 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd9015e4640 10 monclient: _renew_subs 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd9015e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc137830 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd8db7fe640 1 -- 10.20.193.78:0/4082042505 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd8ec0169f0 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd9015e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd9015e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd9015e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8fc06cc70 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd8db7fe640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8c803deb0 0x7fd8c8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:57:59.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd8db7fe640 1 -- 10.20.193.78:0/4082042505 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(425..425 src has 335..425) ==== 4811+0+0 (secure 0 0 0) 0x7fd8ec053a20 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd8faffd640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8c803deb0 0x7fd8c8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:57:59.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.592+0000 7fd8faffd640 10 monclient: get_auth_request con 0x7fd8c803deb0 auth_method 0 2026-02-17T05:57:59.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.595+0000 7fd8db7fe640 1 -- 10.20.193.78:0/4082042505 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd8ec024020 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.595+0000 7fd8db7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:57:59.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.595+0000 7fd8db7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:57:59.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.595+0000 7fd8faffd640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8c803deb0 0x7fd8c8040370 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0002ce0 tx=0x7fd8f0009070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2036: done (6 left) 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2037: done (5 left) 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2038: done (4 left) 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2039: done (3 left) 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2040: done (2 left) 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2041: done (1 left) 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2042: done (0 left) 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2043: write (excl) oid 26 current snap is 203 2026-02-17T05:57:59.636 INFO:tasks.rados.rados.0.trial115.stdout:2043: seq_num 736 ranges {769952=553152,2069100=300852} 2026-02-17T05:57:59.641 INFO:tasks.rados.rados.0.trial115.stdout:2043: writing trial11545420-26 from 769952 to 1323104 tid 1 2026-02-17T05:57:59.645 INFO:tasks.rados.rados.0.trial115.stdout:2043: writing trial11545420-26 from 2069100 to 2369952 tid 2 2026-02-17T05:57:59.645 INFO:tasks.rados.rados.0.trial115.stdout:2044: rollback oid 37 current snap is 203 2026-02-17T05:57:59.645 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 189 2026-02-17T05:57:59.645 INFO:tasks.rados.rados.0.trial115.stdout:2045: write oid 2 current snap is 203 2026-02-17T05:57:59.645 INFO:tasks.rados.rados.0.trial115.stdout:2045: seq_num 737 ranges {786759=628401,2011229=375530} 2026-02-17T05:57:59.651 INFO:tasks.rados.rados.0.trial115.stdout:2045: writing trial11545420-2 from 786759 to 1415160 tid 1 2026-02-17T05:57:59.654 INFO:tasks.rados.rados.0.trial115.stdout:2045: writing trial11545420-2 from 2011229 to 2386759 tid 2 2026-02-17T05:57:59.654 INFO:tasks.rados.rados.0.trial115.stdout:2046: write (excl) oid 34 current snap is 203 2026-02-17T05:57:59.655 INFO:tasks.rados.rados.0.trial115.stdout:2046: seq_num 738 ranges {403566=420003,1386064=546918} 2026-02-17T05:57:59.658 INFO:tasks.rados.rados.0.trial115.stdout:2046: writing trial11545420-34 from 403566 to 823569 tid 1 2026-02-17T05:57:59.662 INFO:tasks.rados.rados.0.trial115.stdout:2046: writing trial11545420-34 from 1386064 to 1932982 tid 2 2026-02-17T05:57:59.662 INFO:tasks.rados.rados.0.trial115.stdout:2047: snap_remove snap 185 2026-02-17T05:57:59.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.710+0000 7fd9015e4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:57:59.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.710+0000 7fd9015e4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:59.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.710+0000 7fd9015e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:57:59.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.710+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd8fc09ef50 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.711+0000 7fd8db7fe640 1 -- 10.20.193.78:0/4082042505 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fd8ec010070 con 0x7fd8fc1d34a0 2026-02-17T05:57:59.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.711+0000 7fd8db7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:57:59.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.711+0000 7fd8db7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:57:59.712 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":132,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8c803deb0 msgr2=0x7fd8c8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8c803deb0 0x7fd8c8040370 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0002ce0 tx=0x7fd8f0009070 comp rx=0 tx=0).stop 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 10 monclient: shutdown 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8fc1d34a0 msgr2=0x7fd8fc1d5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8fc1d34a0 0x7fd8fc1d5890 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc06dcf0 tx=0x7fd8ec004760 comp rx=0 tx=0).stop 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 shutdown_connections 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8fc1d34a0 0x7fd8fc1d5890 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8fc09e060 0x7fd8fc09e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8c803deb0 0x7fd8c8040370 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 --2- 10.20.193.78:0/4082042505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8e8003bd0 0x7fd8fc09db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:57:59.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 >> 10.20.193.78:0/4082042505 conn(0x7fd8fc0a6620 msgr2=0x7fd8fc1363e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:57:59.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 shutdown_connections 2026-02-17T05:57:59.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:57:59.712+0000 7fd9015e4640 1 -- 10.20.193.78:0/4082042505 wait complete. 2026-02-17T05:58:00.638 INFO:tasks.rados.rados.0.trial115.stdout:2047: done (4 left) 2026-02-17T05:58:00.638 INFO:tasks.rados.rados.0.trial115.stdout:2048: write (excl) oid 42 current snap is 203 2026-02-17T05:58:00.638 INFO:tasks.rados.rados.0.trial115.stdout:2048: seq_num 739 ranges {420373=495252,1444546=690576} 2026-02-17T05:58:00.643 INFO:tasks.rados.rados.0.trial115.stdout:2048: writing trial11545420-42 from 420373 to 915625 tid 1 2026-02-17T05:58:00.650 INFO:tasks.rados.rados.0.trial115.stdout:2048: writing trial11545420-42 from 1444546 to 2135122 tid 2 2026-02-17T05:58:00.651 INFO:tasks.rados.rados.0.trial115.stdout:2049: read oid 13 snap -1 2026-02-17T05:58:00.651 INFO:tasks.rados.rados.0.trial115.stdout:2049: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:58:00.651 INFO:tasks.rados.rados.0.trial115.stdout:2050: read oid 9 snap -1 2026-02-17T05:58:00.651 INFO:tasks.rados.rados.0.trial115.stdout:2050: expect (ObjNum 735 snap 202 seq_num 735) 2026-02-17T05:58:00.651 INFO:tasks.rados.rados.0.trial115.stdout:2051: snap_create 2026-02-17T05:58:00.652 INFO:tasks.rados.rados.0.trial115.stdout:2043: finishing write tid 1 to trial11545420-26 2026-02-17T05:58:00.652 INFO:tasks.rados.rados.0.trial115.stdout:2043: finishing write tid 2 to trial11545420-26 2026-02-17T05:58:00.652 INFO:tasks.rados.rados.0.trial115.stdout:2043: finishing write tid 3 to trial11545420-26 2026-02-17T05:58:00.652 INFO:tasks.rados.rados.0.trial115.stdout:2043: finishing write tid 4 to trial11545420-26 2026-02-17T05:58:00.652 INFO:tasks.rados.rados.0.trial115.stdout:2043: oid 26 updating version 0 to 617 2026-02-17T05:58:00.652 INFO:tasks.rados.rados.0.trial115.stdout:2043: oid 26 version 617 is already newer than 616 2026-02-17T05:58:00.653 INFO:tasks.rados.rados.0.trial115.stdout:2043: oid 26 version 617 is already newer than 615 2026-02-17T05:58:00.653 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 617 (ObjNum 736 snap 203 seq_num 736) dirty exists 2026-02-17T05:58:00.653 INFO:tasks.rados.rados.0.trial115.stdout:2043: left oid 26 (ObjNum 736 snap 203 seq_num 736) 2026-02-17T05:58:00.653 INFO:tasks.rados.rados.0.trial115.stdout:2044: finishing rollback tid 0 to trial11545420-37 2026-02-17T05:58:00.653 INFO:tasks.rados.rados.0.trial115.stdout:2044: finishing rollback tid 1 to trial11545420-37 2026-02-17T05:58:00.653 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 932 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:58:00.653 INFO:tasks.rados.rados.0.trial115.stdout:2045: finishing write tid 1 to trial11545420-2 2026-02-17T05:58:00.654 INFO:tasks.rados.rados.0.trial115.stdout:2045: finishing write tid 2 to trial11545420-2 2026-02-17T05:58:00.654 INFO:tasks.rados.rados.0.trial115.stdout:2045: finishing write tid 3 to trial11545420-2 2026-02-17T05:58:00.654 INFO:tasks.rados.rados.0.trial115.stdout:2045: finishing write tid 4 to trial11545420-2 2026-02-17T05:58:00.654 INFO:tasks.rados.rados.0.trial115.stdout:2045: oid 2 updating version 0 to 952 2026-02-17T05:58:00.654 INFO:tasks.rados.rados.0.trial115.stdout:2045: oid 2 updating version 952 to 953 2026-02-17T05:58:00.654 INFO:tasks.rados.rados.0.trial115.stdout:2045: oid 2 updating version 953 to 954 2026-02-17T05:58:00.655 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 954 (ObjNum 737 snap 203 seq_num 737) dirty exists 2026-02-17T05:58:00.655 INFO:tasks.rados.rados.0.trial115.stdout:2045: left oid 2 (ObjNum 737 snap 203 seq_num 737) 2026-02-17T05:58:00.655 INFO:tasks.rados.rados.0.trial115.stdout:2046: finishing write tid 1 to trial11545420-34 2026-02-17T05:58:00.655 INFO:tasks.rados.rados.0.trial115.stdout:2046: finishing write tid 2 to trial11545420-34 2026-02-17T05:58:00.655 INFO:tasks.rados.rados.0.trial115.stdout:2046: finishing write tid 3 to trial11545420-34 2026-02-17T05:58:00.655 INFO:tasks.rados.rados.0.trial115.stdout:2046: finishing write tid 4 to trial11545420-34 2026-02-17T05:58:00.655 INFO:tasks.rados.rados.0.trial115.stdout:2046: oid 34 updating version 0 to 832 2026-02-17T05:58:00.656 INFO:tasks.rados.rados.0.trial115.stdout:2046: oid 34 version 832 is already newer than 830 2026-02-17T05:58:00.656 INFO:tasks.rados.rados.0.trial115.stdout:2046: oid 34 version 832 is already newer than 831 2026-02-17T05:58:00.656 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 832 (ObjNum 738 snap 203 seq_num 738) dirty exists 2026-02-17T05:58:00.656 INFO:tasks.rados.rados.0.trial115.stdout:2046: left oid 34 (ObjNum 738 snap 203 seq_num 738) 2026-02-17T05:58:00.656 INFO:tasks.rados.rados.0.trial115.stdout:2048: finishing write tid 1 to trial11545420-42 2026-02-17T05:58:00.657 INFO:tasks.rados.rados.0.trial115.stdout:2050: expect (ObjNum 735 snap 202 seq_num 735) 2026-02-17T05:58:00.738 INFO:tasks.rados.rados.0.trial115.stdout:2048: finishing write tid 2 to trial11545420-42 2026-02-17T05:58:00.739 INFO:tasks.rados.rados.0.trial115.stdout:2048: finishing write tid 3 to trial11545420-42 2026-02-17T05:58:00.739 INFO:tasks.rados.rados.0.trial115.stdout:2048: finishing write tid 4 to trial11545420-42 2026-02-17T05:58:00.739 INFO:tasks.rados.rados.0.trial115.stdout:2048: oid 42 updating version 0 to 1166 2026-02-17T05:58:00.739 INFO:tasks.rados.rados.0.trial115.stdout:2048: oid 42 updating version 1166 to 1175 2026-02-17T05:58:00.739 INFO:tasks.rados.rados.0.trial115.stdout:2048: oid 42 updating version 1175 to 1176 2026-02-17T05:58:00.739 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1176 (ObjNum 739 snap 203 seq_num 739) dirty exists 2026-02-17T05:58:00.739 INFO:tasks.rados.rados.0.trial115.stdout:2048: left oid 42 (ObjNum 739 snap 203 seq_num 739) 2026-02-17T05:58:00.739 INFO:tasks.rados.rados.0.trial115.stdout:2049: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:58:01.692 INFO:tasks.rados.rados.0.trial115.stdout:2043: done (7 left) 2026-02-17T05:58:01.693 INFO:tasks.rados.rados.0.trial115.stdout:2044: done (6 left) 2026-02-17T05:58:01.693 INFO:tasks.rados.rados.0.trial115.stdout:2045: done (5 left) 2026-02-17T05:58:01.693 INFO:tasks.rados.rados.0.trial115.stdout:2046: done (4 left) 2026-02-17T05:58:01.693 INFO:tasks.rados.rados.0.trial115.stdout:2048: done (3 left) 2026-02-17T05:58:01.693 INFO:tasks.rados.rados.0.trial115.stdout:2049: done (2 left) 2026-02-17T05:58:01.693 INFO:tasks.rados.rados.0.trial115.stdout:2050: done (1 left) 2026-02-17T05:58:01.693 INFO:tasks.rados.rados.0.trial115.stdout:2051: done (0 left) 2026-02-17T05:58:01.693 INFO:tasks.rados.rados.0.trial115.stdout:2052: snap_remove snap 172 2026-02-17T05:58:02.722 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:02.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.791+0000 7f46ec064640 10 monclient: get_monmap_and_config 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.791+0000 7f46ec064640 10 monclient: build_initial_monmap 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.791+0000 7f46ec064640 10 monclient: monmap: 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:02.792045+0000 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:02.792045+0000 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:02.794 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 1 Processor -- start 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 1 -- start start 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient: init 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient: _add_conns ranks=[0,1,3,2,4] 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f46e406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:02.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f46e406d380 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f46e40a01f0 0x7f46e413d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient(hunting): picked mon.noname-a con 0x7f46e40a01f0 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e40a0600 con 0x7f46e406c890 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e4074150 con 0x7f46e406d380 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f46e409ef70 con 0x7f46e40a01f0 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ec064640 10 monclient(hunting): authenticate will time out at 1328.345588s 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46e9dd9640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f46e406c890 0x7f46e406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:36464/0 (socket says 10.20.193.78:36464) 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46e9dd9640 1 -- 10.20.193.78:0/1966110414 learned_addr learned my addr 10.20.193.78:0/1966110414 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f46e40a01f0 0x7f46e413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:02.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient(hunting): get_auth_request con 0x7f46e40a01f0 auth_method 0 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient(hunting): handle_auth_done global_id 154110 payload 274 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 1 -- 10.20.193.78:0/1966110414 >> v1:10.20.193.115:6789/0 conn(0x7f46e406d380 legacy=0x7f46e413a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 1 -- 10.20.193.78:0/1966110414 >> v1:10.20.193.78:6789/0 conn(0x7f46e406c890 legacy=0x7f46e406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient: _finish_hunting 0 2026-02-17T05:58:02.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 1 monclient: found mon.noname-a 2026-02-17T05:58:02.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:02.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:02.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.792+0000 7f46ea5da640 1 -- 10.20.193.78:0/1966110414 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e409f250 con 0x7f46e40a01f0 2026-02-17T05:58:02.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ea5da640 1 -- 10.20.193.78:0/1966110414 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 msgr2=0x7f46e413d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:02.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ea5da640 1 --2- 10.20.193.78:0/1966110414 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e413d970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009d90 tx=0x7f46e002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15ce8142fa0047d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:02.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 1 -- 10.20.193.78:0/1966110414 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e003c070 con 0x7f46e40a01f0 2026-02-17T05:58:02.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:02.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:02.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: dump: 2026-02-17T05:58:02.804 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:02.804 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: _finish_auth 0 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: _check_auth_tickets 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:02.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 1 -- 10.20.193.78:0/1966110414 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e0037640 con 0x7f46e40a01f0 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 5 monclient: authenticate success, global_id 154110 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 1 -- 10.20.193.78:0/1966110414 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0037a10 con 0x7f46e40a01f0 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 4 set_mon_vals no callback set 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46e8dd7640 10 monclient: dump: 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:02.806 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 10 monclient: shutdown 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 1 -- 10.20.193.78:0/1966110414 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 msgr2=0x7f46e413d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 1 --2- 10.20.193.78:0/1966110414 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e413d970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009d90 tx=0x7f46e002f1e0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 1 -- 10.20.193.78:0/1966110414 shutdown_connections 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 1 --2- 10.20.193.78:0/1966110414 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e413d970 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 1 -- 10.20.193.78:0/1966110414 >> 10.20.193.78:0/1966110414 conn(0x7f46e40a6620 msgr2=0x7f46e40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 1 -- 10.20.193.78:0/1966110414 shutdown_connections 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.795+0000 7f46ec064640 1 -- 10.20.193.78:0/1966110414 wait complete. 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient: build_initial_monmap 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient: monmap: 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:02.808 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 1 Processor -- start 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 1 -- start start 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient: init 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f46e406c890 0x7f46e41332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient(hunting): picked mon.noname-c con 0x7f46e406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f46e406d380 0x7f46e41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:02.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient(hunting): picked mon.noname-b con 0x7f46e406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e41cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient(hunting): picked mon.noname-a con 0x7f46e40a01f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46e409d360 con 0x7f46e40a01f0 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46e409d1e0 con 0x7f46e406d380 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f46e413deb0 con 0x7f46e406c890 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ec064640 10 monclient(hunting): authenticate will time out at 1328.349658s 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e41cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46e9dd9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f46e406c890 0x7f46e41332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e41cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:59194/0 (socket says 10.20.193.78:59194) 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 -- 10.20.193.78:0/564800360 learned_addr learned my addr 10.20.193.78:0/564800360 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient(hunting): get_auth_request con 0x7f46e40a01f0 auth_method 0 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46e9dd9640 10 monclient(hunting): get_auth_request con 0x7f46e406c890 auth_method 0 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46e9dd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46e9dd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46e9dd9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46e9dd9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46e9dd9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46e9dd9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient(hunting): handle_auth_done global_id 154113 payload 824 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 -- 10.20.193.78:0/564800360 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f46e406c890 msgr2=0x7f46e41332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f46e406c890 0x7f46e41332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 -- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f46e406d380 msgr2=0x7f46e41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f46e406d380 0x7f46e41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient: _finish_hunting 0 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 monclient: found mon.noname-a 2026-02-17T05:58:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.796+0000 7f46ea5da640 1 -- 10.20.193.78:0/564800360 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e411fc60 con 0x7f46e40a01f0 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ea5da640 1 -- 10.20.193.78:0/564800360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 msgr2=0x7f46e41cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ea5da640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e41cf110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009d90 tx=0x7f46e0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 1 -- 10.20.193.78:0/564800360 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0047020 con 0x7f46e40a01f0 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: dump: 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:02.813 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 1 -- 10.20.193.78:0/564800360 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e0042680 con 0x7f46e40a01f0 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 1 -- 10.20.193.78:0/564800360 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e003c040 con 0x7f46e40a01f0 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:02.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d3fff640 4 set_mon_vals no callback set 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 10 monclient: dump: 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 5 monclient: authenticate success, global_id 154113 2026-02-17T05:58:02.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 10 monclient: _renew_subs 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e40a2800 con 0x7f46e40a01f0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 10 monclient: _renew_subs 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e411f980 con 0x7f46e40a01f0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e406cc70 con 0x7f46e40a01f0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.799+0000 7f46d27fc640 1 -- 10.20.193.78:0/564800360 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f46e0042a70 con 0x7f46e40a01f0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.800+0000 7f46d27fc640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f46b403deb0 0x7f46b4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.800+0000 7f46d27fc640 1 -- 10.20.193.78:0/564800360 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(428..428 src has 335..428) ==== 4799+0+0 (secure 0 0 0) 0x7f46e0078680 con 0x7f46e40a01f0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.800+0000 7f46e9dd9640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f46b403deb0 0x7f46b4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:02.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.800+0000 7f46e9dd9640 10 monclient: get_auth_request con 0x7f46b403deb0 auth_method 0 2026-02-17T05:58:02.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.802+0000 7f46d27fc640 1 -- 10.20.193.78:0/564800360 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f46e0051030 con 0x7f46e40a01f0 2026-02-17T05:58:02.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.802+0000 7f46d27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:02.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.802+0000 7f46d27fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:02.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.802+0000 7f46e9dd9640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f46b403deb0 0x7f46b4040370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f46d4007d90 tx=0x7f46d4007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:02.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.915+0000 7f46ec064640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:02.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.915+0000 7f46ec064640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:02.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.915+0000 7f46ec064640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:02.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.915+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f46e40744d0 con 0x7f46e40a01f0 2026-02-17T05:58:02.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.915+0000 7f46d27fc640 1 -- 10.20.193.78:0/564800360 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f46e0040370 con 0x7f46e40a01f0 2026-02-17T05:58:02.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.915+0000 7f46d27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:02.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.915+0000 7f46d27fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:02.917 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f46b403deb0 msgr2=0x7f46b4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f46b403deb0 0x7f46b4040370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f46d4007d90 tx=0x7f46d4007ce0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 10 monclient: shutdown 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 msgr2=0x7f46e41cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e41cf110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009d90 tx=0x7f46e0004b40 comp rx=0 tx=0).stop 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 shutdown_connections 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f46e40a01f0 0x7f46e41cf110 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f46b403deb0 0x7f46b4040370 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f46e406d380 0x7f46e41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 --2- 10.20.193.78:0/564800360 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f46e406c890 0x7f46e41332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:02.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 >> 10.20.193.78:0/564800360 conn(0x7f46e40a6620 msgr2=0x7f46e413d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:02.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 shutdown_connections 2026-02-17T05:58:02.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.916+0000 7f46ec064640 1 -- 10.20.193.78:0/564800360 wait complete. 2026-02-17T05:58:02.924 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:58:02.924 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:58:02.924 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:58:02.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.991+0000 7f54b9f72640 10 monclient: get_monmap_and_config 2026-02-17T05:58:02.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.991+0000 7f54b9f72640 10 monclient: build_initial_monmap 2026-02-17T05:58:02.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.991+0000 7f54b9f72640 10 monclient: monmap: 2026-02-17T05:58:02.994 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:02.994 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:02.992119+0000 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:02.992119+0000 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 1 Processor -- start 2026-02-17T05:58:02.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 1 -- start start 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient: init 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f54b406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f54b406d380 0x7f54b40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient(hunting): picked mon.noname-a con 0x7f54b406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f54b413a240 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b4074150 con 0x7f54b413a240 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b409ef70 con 0x7f54b406c890 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f54b409f250 con 0x7f54b406d380 2026-02-17T05:58:02.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b9f72640 10 monclient(hunting): authenticate will time out at 1328.545667s 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f54b406d380 0x7f54b40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b3fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f54b413a240 0x7f54b413c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:36480/0 (socket says 10.20.193.78:36480) 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b3fff640 1 -- 10.20.193.78:0/3759348686 learned_addr learned my addr 10.20.193.78:0/3759348686 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient(hunting): get_auth_request con 0x7f54b406d380 auth_method 0 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient(hunting): handle_auth_done global_id 154116 payload 274 2026-02-17T05:58:02.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 1 -- 10.20.193.78:0/3759348686 >> v1:10.20.193.115:6789/0 conn(0x7f54b406c890 legacy=0x7f54b406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:02.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 1 -- 10.20.193.78:0/3759348686 >> v1:10.20.193.78:6789/0 conn(0x7f54b413a240 legacy=0x7f54b413c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:02.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:58:02.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 1 monclient: found mon.noname-a 2026-02-17T05:58:02.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:02.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:02.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.992+0000 7f54b2ffd640 1 -- 10.20.193.78:0/3759348686 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b409df40 con 0x7f54b406d380 2026-02-17T05:58:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b2ffd640 1 -- 10.20.193.78:0/3759348686 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b406d380 msgr2=0x7f54b40a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b2ffd640 1 --2- 10.20.193.78:0/3759348686 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b406d380 0x7f54b40a01f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54a40098a0 tx=0x7f54a402ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77f49b32d85f865 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 1 -- 10.20.193.78:0/3759348686 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a403c070 con 0x7f54b406d380 2026-02-17T05:58:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: dump: 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 1 -- 10.20.193.78:0/3759348686 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a402fd70 con 0x7f54b406d380 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 5 monclient: authenticate success, global_id 154116 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 1 -- 10.20.193.78:0/3759348686 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4037660 con 0x7f54b406d380 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 4 set_mon_vals no callback set 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b27fc640 10 monclient: dump: 2026-02-17T05:58:03.009 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 10 monclient: shutdown 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 1 -- 10.20.193.78:0/3759348686 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b406d380 msgr2=0x7f54b40a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 1 --2- 10.20.193.78:0/3759348686 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b406d380 0x7f54b40a01f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54a40098a0 tx=0x7f54a402ee10 comp rx=0 tx=0).stop 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 1 -- 10.20.193.78:0/3759348686 shutdown_connections 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 1 --2- 10.20.193.78:0/3759348686 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b406d380 0x7f54b40a01f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 1 -- 10.20.193.78:0/3759348686 >> 10.20.193.78:0/3759348686 conn(0x7f54b40a6620 msgr2=0x7f54b40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 1 -- 10.20.193.78:0/3759348686 shutdown_connections 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.995+0000 7f54b9f72640 1 -- 10.20.193.78:0/3759348686 wait complete. 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient: monmap: 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:03.011 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 1 Processor -- start 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 1 -- start start 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient: init 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54b406c890 0x7f54b4135040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient(hunting): picked mon.noname-b con 0x7f54b406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:03.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54b406d380 0x7f54b4135580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient(hunting): picked mon.noname-c con 0x7f54b406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b413a240 0x7f54b411db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient(hunting): picked mon.noname-a con 0x7f54b413a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54b413ded0 con 0x7f54b413a240 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54b413dd50 con 0x7f54b406c890 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f54b413e050 con 0x7f54b406d380 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b9f72640 10 monclient(hunting): authenticate will time out at 1328.549860s 2026-02-17T05:58:03.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b413a240 0x7f54b411db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b2ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54b406d380 0x7f54b4135580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b413a240 0x7f54b411db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:59214/0 (socket says 10.20.193.78:59214) 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 1 -- 10.20.193.78:0/826808275 learned_addr learned my addr 10.20.193.78:0/826808275 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b2ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54b406d380 0x7f54b4135580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57536/0 (socket says 10.20.193.78:57536) 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 10 monclient(hunting): get_auth_request con 0x7f54b413a240 auth_method 0 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b2ffd640 10 monclient(hunting): get_auth_request con 0x7f54b406d380 auth_method 0 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b37fe640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54b406c890 0x7f54b4135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.996+0000 7f54b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b37fe640 10 monclient(hunting): get_auth_request con 0x7f54b406c890 auth_method 0 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 10 monclient(hunting): handle_auth_done global_id 154119 payload 824 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54b406d380 msgr2=0x7f54b4135580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54b406d380 0x7f54b4135580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54b406c890 msgr2=0x7f54b4135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54b406c890 0x7f54b4135040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 1 monclient: found mon.noname-a 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b3fff640 1 -- 10.20.193.78:0/826808275 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b4121810 con 0x7f54b413a240 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.997+0000 7f54b37fe640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54b406c890 0x7f54b4135040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b3fff640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b413a240 msgr2=0x7f54b411db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b3fff640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b413a240 0x7f54b411db60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f54a800ca40 tx=0x7f54a800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a800add0 con 0x7f54b413a240 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:03.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: dump: 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a8012400 con 0x7f54b413a240 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 5 monclient: authenticate success, global_id 154119 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a80127f0 con 0x7f54b413a240 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f548ffff640 10 monclient: dump: 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.018 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b1ffb640 4 set_mon_vals no callback set 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 10 monclient: _renew_subs 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b4121af0 con 0x7f54b413a240 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 10 monclient: _renew_subs 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b40a2090 con 0x7f54b413a240 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b41cf330 0x7f54b411e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54b40a01d0 con 0x7f54b41cf330 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:02.999+0000 7f54b37fe640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b41cf330 0x7f54b411e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 10 monclient: get_auth_request con 0x7f54b41cf330 auth_method 0 2026-02-17T05:58:03.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f54a80129d0 con 0x7f54b413a240 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 10 monclient(hunting): handle_auth_done global_id 154122 payload 824 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b37fe640 1 -- 10.20.193.78:0/826808275 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54b41200e0 con 0x7f54b41cf330 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f548ffff640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f548403deb0 0x7f5484040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(428..428 src has 335..428) ==== 4799+0+0 (secure 0 0 0) 0x7f54a8052cf0 con 0x7f54b413a240 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b2ffd640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f548403deb0 0x7f5484040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.000+0000 7f54b2ffd640 10 monclient: get_auth_request con 0x7f548403deb0 auth_method 0 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f54b37fe640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b41cf330 msgr2=0x7f54b411e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f54b37fe640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b41cf330 0x7f54b411e0a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e730 tx=0x7f54a000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a000c940 con 0x7f54b41cf330 2026-02-17T05:58:03.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 10 monclient: dump: 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f54a000c940 con 0x7f54b41cf330 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:03.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b41cf330 msgr2=0x7f54b411e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b41cf330 0x7f54b411e0a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e730 tx=0x7f54a000ec00 comp rx=0 tx=0).stop 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b41cf330 msgr2=0x7f54b411e0a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.002+0000 7f548ffff640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b41cf330 0x7f54b411e0a0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e730 tx=0x7f54a000ec00 comp rx=0 tx=0).stop 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.003+0000 7f54b2ffd640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f548403deb0 0x7f5484040370 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f54b4069d40 tx=0x7f54a4033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b9f72640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b9f72640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b411e5e0 0x7f54b41da470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b9f72640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54b411e9f0 con 0x7f54b411e5e0 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b411e5e0 0x7f54b41da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 10 monclient: get_auth_request con 0x7f54b411e5e0 auth_method 0 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 10 monclient(hunting): handle_auth_done global_id 154125 payload 824 2026-02-17T05:58:03.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.005+0000 7f54b37fe640 1 -- 10.20.193.78:0/826808275 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f54b40744d0 con 0x7f54b411e5e0 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b37fe640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b411e5e0 msgr2=0x7f54b41da470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b37fe640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b411e5e0 0x7f54b41da470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e730 tx=0x7f54a0003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a001ea80 con 0x7f54b411e5e0 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 10 monclient: dump: 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.024 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f54a001ea80 con 0x7f54b411e5e0 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b411e5e0 msgr2=0x7f54b41da470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b411e5e0 0x7f54b41da470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e730 tx=0x7f54a0003860 comp rx=0 tx=0).stop 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b411e5e0 msgr2=0x7f54b41da470 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f548ffff640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b411e5e0 0x7f54b41da470 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e730 tx=0x7f54a0003860 comp rx=0 tx=0).stop 2026-02-17T05:58:03.025 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 136, 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 15058, 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:58:03.026 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.027 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:03.028 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.029 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.030 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.031 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:03.032 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.033 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:58:03.034 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:58:03.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f548403deb0 msgr2=0x7f5484040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f548403deb0 0x7f5484040370 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f54b4069d40 tx=0x7f54a4033000 comp rx=0 tx=0).stop 2026-02-17T05:58:03.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 10 monclient: shutdown 2026-02-17T05:58:03.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b413a240 msgr2=0x7f54b411db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 1 --2- 10.20.193.78:0/826808275 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54b413a240 0x7f54b411db60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f54a800ca40 tx=0x7f54a800cf10 comp rx=0 tx=0).stop 2026-02-17T05:58:03.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b3fff640 1 -- 10.20.193.78:0/826808275 reap_dead start 2026-02-17T05:58:03.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b3fff640 1 -- 10.20.193.78:0/826808275 reap_dead start 2026-02-17T05:58:03.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 shutdown_connections 2026-02-17T05:58:03.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 >> 10.20.193.78:0/826808275 conn(0x7f54b40a6620 msgr2=0x7f54b4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 shutdown_connections 2026-02-17T05:58:03.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.008+0000 7f54b9f72640 1 -- 10.20.193.78:0/826808275 wait complete. 2026-02-17T05:58:03.036 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:58:03.036 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:58:03.036 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:58:03.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.103+0000 7fae7abf7640 10 monclient: get_monmap_and_config 2026-02-17T05:58:03.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.103+0000 7fae7abf7640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.103+0000 7fae7abf7640 10 monclient: monmap: 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:03.103923+0000 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:03.103923+0000 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:03.104 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 1 Processor -- start 2026-02-17T05:58:03.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 1 -- start start 2026-02-17T05:58:03.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient: init 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae7406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient(hunting): picked mon.noname-b con 0x7fae7406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fae7409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fae7406d1b0 0x7fae74135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient(hunting): picked mon.noname-c con 0x7fae7406d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7406daf0 con 0x7fae7409c4a0 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fae740a0880 con 0x7fae7406d1b0 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae7409cc80 con 0x7fae7406c890 2026-02-17T05:58:03.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7abf7640 10 monclient(hunting): authenticate will time out at 1328.657479s 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae7406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae6bfff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fae7409c4a0 0x7fae74134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:36490/0 (socket says 10.20.193.78:36490) 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae6bfff640 1 -- 10.20.193.78:0/1582741401 learned_addr learned my addr 10.20.193.78:0/1582741401 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient(hunting): get_auth_request con 0x7fae7406c890 auth_method 0 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7916d640 1 --2- 10.20.193.78:0/1582741401 >> v2:10.20.193.115:3300/0 conn(0x7fae7406d1b0 0x7fae74135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7916d640 10 monclient(hunting): get_auth_request con 0x7fae7406d1b0 auth_method 0 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7916d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7916d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7916d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient(hunting): handle_auth_done global_id 144179 payload 274 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 1 -- 10.20.193.78:0/1582741401 >> v2:10.20.193.115:3300/0 conn(0x7fae7406d1b0 msgr2=0x7fae74135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 1 --2- 10.20.193.78:0/1582741401 >> v2:10.20.193.115:3300/0 conn(0x7fae7406d1b0 0x7fae74135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 1 -- 10.20.193.78:0/1582741401 >> v1:10.20.193.78:6789/0 conn(0x7fae7409c4a0 legacy=0x7fae74134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 1 monclient: found mon.noname-b 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.104+0000 7fae7896c640 1 -- 10.20.193.78:0/1582741401 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7409cf60 con 0x7fae7406c890 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7896c640 1 -- 10.20.193.78:0/1582741401 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 msgr2=0x7fae7406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7896c640 1 --2- 10.20.193.78:0/1582741401 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae7406cc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae640098a0 tx=0x7fae6402ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563ebb2887e93245 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 1 -- 10.20.193.78:0/1582741401 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6403c070 con 0x7fae7406c890 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: dump: 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.114 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 1 -- 10.20.193.78:0/1582741401 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6402fc00 con 0x7fae7406c890 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 1 -- 10.20.193.78:0/1582741401 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64037440 con 0x7fae7406c890 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 5 monclient: authenticate success, global_id 144179 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 4 set_mon_vals no callback set 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:03.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae6b7fe640 10 monclient: dump: 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 10 monclient: shutdown 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 1 -- 10.20.193.78:0/1582741401 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 msgr2=0x7fae7406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 1 --2- 10.20.193.78:0/1582741401 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae7406cc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae640098a0 tx=0x7fae6402ed80 comp rx=0 tx=0).stop 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 1 -- 10.20.193.78:0/1582741401 shutdown_connections 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 1 --2- 10.20.193.78:0/1582741401 >> v2:10.20.193.115:3300/0 conn(0x7fae7406d1b0 0x7fae74135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 1 --2- 10.20.193.78:0/1582741401 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae7406cc70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 1 -- 10.20.193.78:0/1582741401 >> 10.20.193.78:0/1582741401 conn(0x7fae740a6620 msgr2=0x7fae740a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 1 -- 10.20.193.78:0/1582741401 shutdown_connections 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.107+0000 7fae7abf7640 1 -- 10.20.193.78:0/1582741401 wait complete. 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient: monmap: 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:03.117 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 1 Processor -- start 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 1 -- start start 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient: init 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae741cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient(hunting): picked mon.noname-c con 0x7fae7406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:03.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae7406d1b0 0x7fae741cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient(hunting): picked mon.noname-b con 0x7fae7406d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae7409c4a0 0x7fae74124d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient(hunting): picked mon.noname-a con 0x7fae7409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae740a01a0 con 0x7fae7409c4a0 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae740a0020 con 0x7fae7406d1b0 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae7409be20 con 0x7fae7406c890 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7abf7640 10 monclient(hunting): authenticate will time out at 1328.661838s 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae741cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae741cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57554/0 (socket says 10.20.193.78:57554) 2026-02-17T05:58:03.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 1 -- 10.20.193.78:0/2120011584 learned_addr learned my addr 10.20.193.78:0/2120011584 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 10 monclient(hunting): get_auth_request con 0x7fae7406c890 auth_method 0 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7916d640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae7409c4a0 0x7fae74124d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7916d640 10 monclient(hunting): get_auth_request con 0x7fae7409c4a0 auth_method 0 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7916d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7916d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7916d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7916d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7916d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7916d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 10 monclient(hunting): handle_auth_done global_id 144182 payload 824 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.108+0000 7fae7896c640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae7406d1b0 msgr2=0x7fae741cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.109+0000 7fae7896c640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae7406d1b0 0x7fae741cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.109+0000 7fae7896c640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae7409c4a0 msgr2=0x7fae74124d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.109+0000 7fae7896c640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae7409c4a0 0x7fae74124d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.109+0000 7fae7896c640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.109+0000 7fae7896c640 1 monclient: found mon.noname-c 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.109+0000 7fae7896c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.109+0000 7fae7896c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:03.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.109+0000 7fae7896c640 1 -- 10.20.193.78:0/2120011584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae74136ea0 con 0x7fae7406c890 2026-02-17T05:58:03.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7896c640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 msgr2=0x7fae741cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7896c640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae741cf350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae640099d0 tx=0x7fae640047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6403c040 con 0x7fae7406c890 2026-02-17T05:58:03.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:03.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: dump: 2026-02-17T05:58:03.126 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.126 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae64047070 con 0x7fae7406c890 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64040a40 con 0x7fae7406c890 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae697fa640 10 monclient: dump: 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.128 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6affd640 4 set_mon_vals no callback set 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 5 monclient: authenticate success, global_id 144182 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 10 monclient: _renew_subs 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7409d810 con 0x7fae7406c890 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 10 monclient: _renew_subs 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae74128820 con 0x7fae7406c890 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae741289f0 0x7fae741252d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae74074980 con 0x7fae741289f0 2026-02-17T05:58:03.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6bfff640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae741289f0 0x7fae741252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6bfff640 10 monclient: get_auth_request con 0x7fae741289f0 auth_method 0 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.111+0000 7fae6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.112+0000 7fae6bfff640 10 monclient(hunting): handle_auth_done global_id 144185 payload 824 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.112+0000 7fae6bfff640 1 -- 10.20.193.78:0/2120011584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae7406cc70 con 0x7fae741289f0 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.112+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fae6403f030 con 0x7fae7406c890 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.112+0000 7fae697fa640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae4403df00 0x7fae440403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.112+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(428..428 src has 335..428) ==== 4799+0+0 (secure 0 0 0) 0x7fae64077a80 con 0x7fae7406c890 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.112+0000 7fae7916d640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae4403df00 0x7fae440403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.112+0000 7fae7916d640 10 monclient: get_auth_request con 0x7fae4403df00 auth_method 0 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae6bfff640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae741289f0 msgr2=0x7fae741252d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae6bfff640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae741289f0 0x7fae741252d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae74126510 tx=0x7fae5c00c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c017070 con 0x7fae741289f0 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 10 monclient: dump: 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.131 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fae5c0072d0 con 0x7fae741289f0 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae741289f0 msgr2=0x7fae741252d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae741289f0 0x7fae741252d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae74126510 tx=0x7fae5c00c840 comp rx=0 tx=0).stop 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae741289f0 msgr2=0x7fae741252d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.114+0000 7fae697fa640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae741289f0 0x7fae741252d0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae74126510 tx=0x7fae5c00c840 comp rx=0 tx=0).stop 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.115+0000 7fae7916d640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae4403df00 0x7fae440403c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fae74136130 tx=0x7fae70009770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae7abf7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:58:03.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae7abf7640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae74125810 0x7fae741da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae7abf7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae741270f0 con 0x7fae74125810 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae74125810 0x7fae741da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 10 monclient: get_auth_request con 0x7fae74125810 auth_method 0 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 10 monclient(hunting): handle_auth_done global_id 144188 payload 824 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.117+0000 7fae6bfff640 1 -- 10.20.193.78:0/2120011584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fae74126890 con 0x7fae74125810 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae6bfff640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae74125810 msgr2=0x7fae741da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae6bfff640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae74125810 0x7fae741da560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae74075fe0 tx=0x7fae5c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c017040 con 0x7fae74125810 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 10 monclient: dump: 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7fae5c012400 con 0x7fae74125810 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae74125810 msgr2=0x7fae741da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae74125810 0x7fae741da560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae74075fe0 tx=0x7fae5c004a40 comp rx=0 tx=0).stop 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae74125810 msgr2=0x7fae741da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae697fa640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae74125810 0x7fae741da560 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae74075fe0 tx=0x7fae5c004a40 comp rx=0 tx=0).stop 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 136, 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 66655, 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:58:03.135 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.136 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.137 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:58:03.138 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.139 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.140 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.141 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.142 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:58:03.143 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.144 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:58:03.145 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:58:03.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae4403df00 msgr2=0x7fae440403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae4403df00 0x7fae440403c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fae74136130 tx=0x7fae70009770 comp rx=0 tx=0).stop 2026-02-17T05:58:03.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 10 monclient: shutdown 2026-02-17T05:58:03.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 msgr2=0x7fae741cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 1 --2- 10.20.193.78:0/2120011584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae7406c890 0x7fae741cf350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae640099d0 tx=0x7fae640047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7916d640 1 -- 10.20.193.78:0/2120011584 reap_dead start 2026-02-17T05:58:03.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7916d640 1 -- 10.20.193.78:0/2120011584 reap_dead start 2026-02-17T05:58:03.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 shutdown_connections 2026-02-17T05:58:03.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 >> 10.20.193.78:0/2120011584 conn(0x7fae740a6620 msgr2=0x7fae740a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 shutdown_connections 2026-02-17T05:58:03.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.120+0000 7fae7abf7640 1 -- 10.20.193.78:0/2120011584 wait complete. 2026-02-17T05:58:03.147 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:58:03.147 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:58:03.147 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:58:03.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.213+0000 7f18ac84d640 10 monclient: get_monmap_and_config 2026-02-17T05:58:03.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.213+0000 7f18ac84d640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.213+0000 7f18ac84d640 10 monclient: monmap: 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:03.214017+0000 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:03.214017+0000 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:03.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 1 Processor -- start 2026-02-17T05:58:03.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 1 -- start start 2026-02-17T05:58:03.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient: init 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406c890 0x7f18a406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient(hunting): picked mon.noname-b con 0x7f18a406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f18a409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f18a406d1b0 0x7f18a4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient(hunting): picked mon.noname-c con 0x7f18a406d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a406daf0 con 0x7f18a409c4a0 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f18a40a0880 con 0x7f18a406d1b0 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f18a409cc80 con 0x7f18a406c890 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18ac84d640 10 monclient(hunting): authenticate will time out at 1328.767534s 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406c890 0x7f18a406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406c890 0x7f18a406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57572/0 (socket says 10.20.193.78:57572) 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 1 -- 10.20.193.78:0/1809818581 learned_addr learned my addr 10.20.193.78:0/1809818581 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient(hunting): get_auth_request con 0x7f18a406c890 auth_method 0 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_done global_id 144191 payload 274 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 1 -- 10.20.193.78:0/1809818581 >> v2:10.20.193.115:3300/0 conn(0x7f18a406d1b0 msgr2=0x7f18a4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 1 --2- 10.20.193.78:0/1809818581 >> v2:10.20.193.115:3300/0 conn(0x7f18a406d1b0 0x7f18a4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 1 -- 10.20.193.78:0/1809818581 >> v1:10.20.193.78:6789/0 conn(0x7f18a409c4a0 legacy=0x7f18a4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 1 monclient: found mon.noname-b 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:03.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.214+0000 7f18aa5c2640 1 -- 10.20.193.78:0/1809818581 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a409cf60 con 0x7f18a406c890 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18aa5c2640 1 -- 10.20.193.78:0/1809818581 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406c890 msgr2=0x7f18a406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18aa5c2640 1 --2- 10.20.193.78:0/1809818581 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406c890 0x7f18a406cc70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f189c009970 tx=0x7f189c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c78378bbf2d1153 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 1 -- 10.20.193.78:0/1809818581 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c03c070 con 0x7f18a406c890 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: dump: 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.225 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 1 -- 10.20.193.78:0/1809818581 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c02fdb0 con 0x7f18a406c890 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 1 -- 10.20.193.78:0/1809818581 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c037660 con 0x7f18a406c890 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 5 monclient: authenticate success, global_id 144191 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 4 set_mon_vals no callback set 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18a95c0640 10 monclient: dump: 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.226 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 10 monclient: shutdown 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 1 -- 10.20.193.78:0/1809818581 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406c890 msgr2=0x7f18a406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 1 --2- 10.20.193.78:0/1809818581 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406c890 0x7f18a406cc70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f189c009970 tx=0x7f189c02ee90 comp rx=0 tx=0).stop 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 1 -- 10.20.193.78:0/1809818581 shutdown_connections 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 1 --2- 10.20.193.78:0/1809818581 >> v2:10.20.193.115:3300/0 conn(0x7f18a406d1b0 0x7f18a4135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 1 --2- 10.20.193.78:0/1809818581 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406c890 0x7f18a406cc70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 1 -- 10.20.193.78:0/1809818581 >> 10.20.193.78:0/1809818581 conn(0x7f18a40a6620 msgr2=0x7f18a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 1 -- 10.20.193.78:0/1809818581 shutdown_connections 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.217+0000 7f18ac84d640 1 -- 10.20.193.78:0/1809818581 wait complete. 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient: monmap: 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 1 Processor -- start 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 1 -- start start 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient: init 2026-02-17T05:58:03.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18a406c890 0x7f18a41cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient(hunting): picked mon.noname-a con 0x7f18a406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406d1b0 0x7f18a41cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient(hunting): picked mon.noname-c con 0x7f18a406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a409c4a0 0x7f18a4123570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient(hunting): picked mon.noname-b con 0x7f18a409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18a40a01a0 con 0x7f18a406c890 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18a40a0020 con 0x7f18a409c4a0 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f18a409be20 con 0x7f18a406d1b0 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18ac84d640 10 monclient(hunting): authenticate will time out at 1328.771569s 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18a406c890 0x7f18a41cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406d1b0 0x7f18a41cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18a406c890 0x7f18a41cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:59248/0 (socket says 10.20.193.78:59248) 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 1 -- 10.20.193.78:0/338960729 learned_addr learned my addr 10.20.193.78:0/338960729 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 10 monclient(hunting): get_auth_request con 0x7f18a406c890 auth_method 0 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient(hunting): get_auth_request con 0x7f18a406d1b0 auth_method 0 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aadc3640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a409c4a0 0x7f18a4123570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aadc3640 10 monclient(hunting): get_auth_request con 0x7f18a409c4a0 auth_method 0 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aadc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aadc3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18aadc3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient(hunting): handle_auth_done global_id 144194 payload 824 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a409c4a0 msgr2=0x7f18a4123570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a409c4a0 0x7f18a4123570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18a406c890 msgr2=0x7f18a41cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f18a406c890 0x7f18a41cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 1 monclient: found mon.noname-c 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.218+0000 7f18a9dc1640 1 -- 10.20.193.78:0/338960729 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a4126ef0 con 0x7f18a406d1b0 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.220+0000 7f18a9dc1640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406d1b0 msgr2=0x7f18a41cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.220+0000 7f18a9dc1640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406d1b0 0x7f18a41cd870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f189000c450 tx=0x7f189000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890016020 con 0x7f18a406d1b0 2026-02-17T05:58:03.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: dump: 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1890012a40 con 0x7f18a406d1b0 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 5 monclient: authenticate success, global_id 144194 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 10 monclient: _renew_subs 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a41271d0 con 0x7f18a406d1b0 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890011380 con 0x7f18a406d1b0 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 10 monclient: dump: 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.233 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18a8dbf640 4 set_mon_vals no callback set 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 10 monclient: _renew_subs 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a4136ea0 con 0x7f18a406d1b0 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a4063880 0x7f18a4063c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18a406cc70 con 0x7f18a4063880 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18aa5c2640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a4063880 0x7f18a4063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18aa5c2640 10 monclient: get_auth_request con 0x7f18a4063880 auth_method 0 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18aa5c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18aa5c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18aa5c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f18900115e0 con 0x7f18a406d1b0 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.221+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.222+0000 7f188effd640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f187403deb0 0x7f1874040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.222+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_done global_id 134254 payload 824 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.222+0000 7f18aa5c2640 1 -- 10.20.193.78:0/338960729 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18a40008d0 con 0x7f18a4063880 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.222+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(428..428 src has 335..428) ==== 4799+0+0 (secure 0 0 0) 0x7f1890052880 con 0x7f18a406d1b0 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.222+0000 7f18aadc3640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f187403deb0 0x7f1874040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.222+0000 7f18aadc3640 10 monclient: get_auth_request con 0x7f187403deb0 auth_method 0 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f18aa5c2640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a4063880 msgr2=0x7f18a4063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f18aa5c2640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a4063880 0x7f18a4063c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f189c002410 tx=0x7f189c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c03c040 con 0x7f18a4063880 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 10 monclient: dump: 2026-02-17T05:58:03.235 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f189c041410 con 0x7f18a4063880 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a4063880 msgr2=0x7f18a4063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a4063880 0x7f18a4063c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f189c002410 tx=0x7f189c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a4063880 msgr2=0x7f18a4063c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.224+0000 7f188effd640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a4063880 0x7f18a4063c60 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f189c002410 tx=0x7f189c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.225+0000 7f18aadc3640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f187403deb0 0x7f1874040370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18a4126ab0 tx=0x7f1898002a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18ac84d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:58:03.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18ac84d640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a411ec80 0x7f18a41d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18ac84d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4126c10 con 0x7f18a411ec80 2026-02-17T05:58:03.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a411ec80 0x7f18a41d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 10 monclient: get_auth_request con 0x7f18a411ec80 auth_method 0 2026-02-17T05:58:03.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 10 monclient(hunting): handle_auth_done global_id 134257 payload 824 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.228+0000 7f18aa5c2640 1 -- 10.20.193.78:0/338960729 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f18a41cddb0 con 0x7f18a411ec80 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.230+0000 7f18aa5c2640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a411ec80 msgr2=0x7f18a41d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.230+0000 7f18aa5c2640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a411ec80 0x7f18a41d8330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18a406cdf0 tx=0x7f189c008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c03c040 con 0x7f18a411ec80 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 10 monclient: dump: 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.239 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f189c051940 con 0x7f18a411ec80 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a411ec80 msgr2=0x7f18a41d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a411ec80 0x7f18a41d8330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18a406cdf0 tx=0x7f189c008030 comp rx=0 tx=0).stop 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a411ec80 msgr2=0x7f18a41d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:03.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f188effd640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f18a411ec80 0x7f18a41d8330 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18a406cdf0 tx=0x7f189c008030 comp rx=0 tx=0).stop 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 136, 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 110949, 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:58:03.241 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:58:03.242 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:58:03.243 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:03.244 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.245 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.246 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.247 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:58:03.248 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:03.249 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:03.250 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:03.251 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:03.251 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:03.251 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:03.251 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:58:03.251 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f187403deb0 msgr2=0x7f1874040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f187403deb0 0x7f1874040370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18a4126ab0 tx=0x7f1898002a60 comp rx=0 tx=0).stop 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 10 monclient: shutdown 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406d1b0 msgr2=0x7f18a41cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 1 --2- 10.20.193.78:0/338960729 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f18a406d1b0 0x7f18a41cd870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f189000c450 tx=0x7f189000c920 comp rx=0 tx=0).stop 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18aadc3640 1 -- 10.20.193.78:0/338960729 reap_dead start 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18aadc3640 1 -- 10.20.193.78:0/338960729 reap_dead start 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 shutdown_connections 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 >> 10.20.193.78:0/338960729 conn(0x7f18a40a6620 msgr2=0x7f18a40715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 shutdown_connections 2026-02-17T05:58:03.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.231+0000 7f18ac84d640 1 -- 10.20.193.78:0/338960729 wait complete. 2026-02-17T05:58:03.253 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:58:03.253 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:58:03.253 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient: get_monmap_and_config 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient: monmap: 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:03.321381+0000 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:03.321381+0000 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:03.321 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 1 Processor -- start 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 1 -- start start 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient: init 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd5e006c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd5e006d380 0x7fd5e00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient(hunting): picked mon.noname-a con 0x7fd5e006d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd5e013a240 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e0074150 con 0x7fd5e006c890 2026-02-17T05:58:03.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e009ef70 con 0x7fd5e013a240 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e009f250 con 0x7fd5e006d380 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5e6961640 10 monclient(hunting): authenticate will time out at 1328.874897s 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5dffff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd5e006c890 0x7fd5e006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:36494/0 (socket says 10.20.193.78:36494) 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5dffff640 1 -- 10.20.193.78:0/3892628330 learned_addr learned my addr 10.20.193.78:0/3892628330 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5df7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd5e006d380 0x7fd5e00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5df7fe640 10 monclient(hunting): get_auth_request con 0x7fd5e006d380 auth_method 0 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5df7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.321+0000 7fd5df7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 10 monclient(hunting): handle_auth_done global_id 154134 payload 274 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 1 -- 10.20.193.78:0/3892628330 >> v1:10.20.193.115:6789/0 conn(0x7fd5e013a240 legacy=0x7fd5e013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 1 -- 10.20.193.78:0/3892628330 >> v1:10.20.193.78:6789/0 conn(0x7fd5e006c890 legacy=0x7fd5e006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 1 monclient: found mon.noname-a 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.322+0000 7fd5df7fe640 1 -- 10.20.193.78:0/3892628330 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e009df40 con 0x7fd5e006d380 2026-02-17T05:58:03.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5df7fe640 1 -- 10.20.193.78:0/3892628330 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 msgr2=0x7fd5e00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5df7fe640 1 --2- 10.20.193.78:0/3892628330 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e00a01f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098a0 tx=0x7fd5d002ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5597a8f9594fe304 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 1 -- 10.20.193.78:0/3892628330 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d003c070 con 0x7fd5e006d380 2026-02-17T05:58:03.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:03.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: dump: 2026-02-17T05:58:03.327 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.327 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 1 -- 10.20.193.78:0/3892628330 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d002fd70 con 0x7fd5e006d380 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5e6961640 5 monclient: authenticate success, global_id 154134 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 1 -- 10.20.193.78:0/3892628330 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0037660 con 0x7fd5e006d380 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5deffd640 10 monclient: dump: 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5e6961640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:03.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5e6961640 4 set_mon_vals no callback set 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5e6961640 10 monclient: shutdown 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5e6961640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5e6961640 1 -- 10.20.193.78:0/3892628330 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 msgr2=0x7fd5e00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.324+0000 7fd5e6961640 1 --2- 10.20.193.78:0/3892628330 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e00a01f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098a0 tx=0x7fd5d002ee10 comp rx=0 tx=0).stop 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 -- 10.20.193.78:0/3892628330 shutdown_connections 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 --2- 10.20.193.78:0/3892628330 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e00a01f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 -- 10.20.193.78:0/3892628330 >> 10.20.193.78:0/3892628330 conn(0x7fd5e00a6620 msgr2=0x7fd5e00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 -- 10.20.193.78:0/3892628330 shutdown_connections 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 -- 10.20.193.78:0/3892628330 wait complete. 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient: build_initial_monmap 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient: monmap: 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:03.330 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 Processor -- start 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 -- start start 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient: init 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e006c890 0x7fd5e01cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient(hunting): picked mon.noname-c con 0x7fd5e006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e0134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient(hunting): picked mon.noname-a con 0x7fd5e006d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:03.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e013a240 0x7fd5e01346d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient(hunting): picked mon.noname-b con 0x7fd5e013a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e013ded0 con 0x7fd5e006d380 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e013dd50 con 0x7fd5e013a240 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e013e050 con 0x7fd5e006c890 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5e6961640 10 monclient(hunting): authenticate will time out at 1328.878871s 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e0134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e0134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:59274/0 (socket says 10.20.193.78:59274) 2026-02-17T05:58:03.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 1 -- 10.20.193.78:0/827809790 learned_addr learned my addr 10.20.193.78:0/827809790 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:03.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 10 monclient(hunting): get_auth_request con 0x7fd5e006d380 auth_method 0 2026-02-17T05:58:03.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5dffff640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e006c890 0x7fd5e01cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5dffff640 10 monclient(hunting): get_auth_request con 0x7fd5e006c890 auth_method 0 2026-02-17T05:58:03.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:03.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5dffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:03.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5dffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:03.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.325+0000 7fd5df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:03.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:03.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5e4ed7640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e013a240 0x7fd5e01346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 10 monclient(hunting): handle_auth_done global_id 154137 payload 824 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 1 -- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e006c890 msgr2=0x7fd5e01cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e006c890 0x7fd5e01cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 1 -- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e013a240 msgr2=0x7fd5e01346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e013a240 0x7fd5e01346d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 1 monclient: found mon.noname-a 2026-02-17T05:58:03.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:03.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.326+0000 7fd5df7fe640 1 -- 10.20.193.78:0/827809790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e0120860 con 0x7fd5e006d380 2026-02-17T05:58:03.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5df7fe640 1 -- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 msgr2=0x7fd5e0134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5df7fe640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e0134190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098a0 tx=0x7fd5d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 1 -- 10.20.193.78:0/827809790 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0046070 con 0x7fd5e006d380 2026-02-17T05:58:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: dump: 2026-02-17T05:58:03.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.341 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.341 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.341 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.341 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.341 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.341 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.341 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.342 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.342 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: _finish_auth 0 2026-02-17T05:58:03.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: _check_auth_tickets 2026-02-17T05:58:03.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:03.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 1 -- 10.20.193.78:0/827809790 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d0041660 con 0x7fd5e006d380 2026-02-17T05:58:03.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:03.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 1 -- 10.20.193.78:0/827809790 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d003c040 con 0x7fd5e006d380 2026-02-17T05:58:03.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5e6961640 5 monclient: authenticate success, global_id 154137 2026-02-17T05:58:03.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5e6961640 10 monclient: _renew_subs 2026-02-17T05:58:03.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5e6961640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5de7fc640 4 set_mon_vals no callback set 2026-02-17T05:58:03.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e011f340 con 0x7fd5e006d380 2026-02-17T05:58:03.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:03.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:03.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5dcff9640 10 monclient: dump: 2026-02-17T05:58:03.344 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:03.344 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:03.344 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.344 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:03.344 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:03.344 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:03.344 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:03.345 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:03.345 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:03.345 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:03.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5e6961640 10 monclient: _renew_subs 2026-02-17T05:58:03.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5e6961640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.328+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e009d810 con 0x7fd5e006d380 2026-02-17T05:58:03.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5e6961640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5e6961640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5e6961640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e011f8b0 con 0x7fd5e006d380 2026-02-17T05:58:03.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5dcff9640 1 -- 10.20.193.78:0/827809790 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd5d0041a50 con 0x7fd5e006d380 2026-02-17T05:58:03.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5dcff9640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003deb0 0x7fd5b0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:03.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5dcff9640 1 -- 10.20.193.78:0/827809790 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(428..428 src has 335..428) ==== 4799+0+0 (secure 0 0 0) 0x7fd5d0078050 con 0x7fd5e006d380 2026-02-17T05:58:03.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5dffff640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003deb0 0x7fd5b0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:03.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.329+0000 7fd5dffff640 10 monclient: get_auth_request con 0x7fd5b003deb0 auth_method 0 2026-02-17T05:58:03.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.331+0000 7fd5dcff9640 1 -- 10.20.193.78:0/827809790 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd5d0040c70 con 0x7fd5e006d380 2026-02-17T05:58:03.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.331+0000 7fd5dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:03.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.331+0000 7fd5dcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:03.347 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.331+0000 7fd5dffff640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003deb0 0x7fd5b0040370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc007d90 tx=0x7fd5cc007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:03.360 INFO:tasks.rados.rados.0.trial115.stdout:2052: done (0 left) 2026-02-17T05:58:03.360 INFO:tasks.rados.rados.0.trial115.stdout:2053: rollback oid 27 current snap is 204 2026-02-17T05:58:03.361 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 186 2026-02-17T05:58:03.361 INFO:tasks.rados.rados.0.trial115.stdout:2054: read oid 12 snap -1 2026-02-17T05:58:03.361 INFO:tasks.rados.rados.0.trial115.stdout:2054: expect (ObjNum 672 snap 186 seq_num 672) 2026-02-17T05:58:03.361 INFO:tasks.rados.rados.0.trial115.stdout:2055: write oid 44 current snap is 204 2026-02-17T05:58:03.361 INFO:tasks.rados.rados.0.trial115.stdout:2055: seq_num 740 ranges {437180=570501,1503028=550587} 2026-02-17T05:58:03.364 INFO:tasks.rados.rados.0.trial115.stdout:2055: writing trial11545420-44 from 437180 to 1007681 tid 1 2026-02-17T05:58:03.368 INFO:tasks.rados.rados.0.trial115.stdout:2055: writing trial11545420-44 from 1503028 to 2053615 tid 2 2026-02-17T05:58:03.368 INFO:tasks.rados.rados.0.trial115.stdout:2056: write (excl) oid 26 current snap is 204 2026-02-17T05:58:03.368 INFO:tasks.rados.rados.0.trial115.stdout:2056: seq_num 741 ranges {453987=645750,1845157=608830} 2026-02-17T05:58:03.372 INFO:tasks.rados.rados.0.trial115.stdout:2056: writing trial11545420-26 from 453987 to 1099737 tid 1 2026-02-17T05:58:03.377 INFO:tasks.rados.rados.0.trial115.stdout:2056: writing trial11545420-26 from 1845157 to 2453987 tid 2 2026-02-17T05:58:03.377 INFO:tasks.rados.rados.0.trial115.stdout:2057: write oid 40 current snap is 204 2026-02-17T05:58:03.377 INFO:tasks.rados.rados.0.trial115.stdout:2057: seq_num 742 ranges {470794=720999,1903639=554256} 2026-02-17T05:58:03.381 INFO:tasks.rados.rados.0.trial115.stdout:2057: writing trial11545420-40 from 470794 to 1191793 tid 1 2026-02-17T05:58:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2057: writing trial11545420-40 from 1903639 to 2457895 tid 2 2026-02-17T05:58:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2058: delete oid 14 current snap is 204 2026-02-17T05:58:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2053: finishing rollback tid 1 to trial11545420-27 2026-02-17T05:58:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2053: finishing rollback tid 2 to trial11545420-27 2026-02-17T05:58:03.387 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 933 (ObjNum 434 snap 109 seq_num 434) dirty exists 2026-02-17T05:58:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2054: expect (ObjNum 672 snap 186 seq_num 672) 2026-02-17T05:58:03.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.444+0000 7fd5e6961640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:58:03.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.444+0000 7fd5e6961640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:58:03.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.445+0000 7fd5e6961640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:03.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.445+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd5e011e7e0 con 0x7fd5e006d380 2026-02-17T05:58:03.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.445+0000 7fd5dcff9640 1 -- 10.20.193.78:0/827809790 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd5d0041d70 con 0x7fd5e006d380 2026-02-17T05:58:03.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.445+0000 7fd5dcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:58:03.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.445+0000 7fd5dcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:03.447 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.445+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.445+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=23,config=2,health=11,kv=12,logm=52} crc {auth=1764488358,config=692619927,health=1942745526,kv=449989585,logm=3172712327}) 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.445+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=2,health=11,kv=12,logm=56} crc {auth=685158839,config=692619927,health=4070015362,kv=449989585,logm=2981576313}) 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.445+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.445+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=23,config=2,health=11,kv=12,logm=52} crc {auth=1764488358,config=692619927,health=1942745526,kv=449989585,logm=3172712327}) 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.445+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=2,health=12,kv=11,logm=52} crc {auth=1764488358,config=692619927,health=3235959426,kv=449989585,logm=1440308299}) 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891500364}) 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3454467379}) 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.448 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891500364}) 2026-02-17T05:58:03.449 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2071148955}) 2026-02-17T05:58:03.449 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.449 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1176047671}) 2026-02-17T05:58:03.449 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082393176}) 2026-02-17T05:58:03.449 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.449 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1176047671}) 2026-02-17T05:58:03.449 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.446+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3824967017}) 2026-02-17T05:58:03.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.446+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003deb0 msgr2=0x7fd5b0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.446+0000 7fd5e6961640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003deb0 0x7fd5b0040370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc007d90 tx=0x7fd5cc007ce0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 10 monclient: shutdown 2026-02-17T05:58:03.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:03.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 msgr2=0x7fd5e0134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:03.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e0134190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098a0 tx=0x7fd5d00047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 shutdown_connections 2026-02-17T05:58:03.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003deb0 0x7fd5b0040370 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e013a240 0x7fd5e01346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e006d380 0x7fd5e0134190 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 --2- 10.20.193.78:0/827809790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e006c890 0x7fd5e01cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:03.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 >> 10.20.193.78:0/827809790 conn(0x7fd5e00a6620 msgr2=0x7fd5e0135000 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:03.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 shutdown_connections 2026-02-17T05:58:03.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:03.447+0000 7fd5e6961640 1 -- 10.20.193.78:0/827809790 wait complete. 2026-02-17T05:58:03.452 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.452 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1739625462}) 2026-02-17T05:58:03.452 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1422685183}) 2026-02-17T05:58:03.452 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.453 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1739625462}) 2026-02-17T05:58:03.453 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=736585768}) 2026-02-17T05:58:03.453 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.453 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1656523004}) 2026-02-17T05:58:03.453 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2182045871}) 2026-02-17T05:58:03.453 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.453 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1656523004}) 2026-02-17T05:58:03.453 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.447+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910919747}) 2026-02-17T05:58:03.454 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.455 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1943974074}) 2026-02-17T05:58:03.455 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3560615787}) 2026-02-17T05:58:03.455 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.455 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1943974074}) 2026-02-17T05:58:03.455 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3144951906}) 2026-02-17T05:58:03.456 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:58:03.464 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.464 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1827219508}) 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434022690}) 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1827219508}) 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.454+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=745842935}) 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2843579491}) 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1824612788}) 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2843579491}) 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2140363616}) 2026-02-17T05:58:03.465 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414975107}) 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3487513753}) 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414975107}) 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.455+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221929855}) 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2061723820}) 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2891870281}) 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2061723820}) 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1355316408}) 2026-02-17T05:58:03.466 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506124162}) 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1415650305}) 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506124162}) 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1517237855}) 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1420636601}) 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1955491882}) 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1420636601}) 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.456+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=263831390}) 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.467 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1159304645}) 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1319711884}) 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1159304645}) 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808196583}) 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4157283420}) 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112564016}) 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4157283420}) 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4108262935}) 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.468 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1588365731}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2402006659}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1588365731}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.457+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3108223670}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1741855230}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1180544553}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1741855230}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=564480664}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649775179}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2788243273}) 2026-02-17T05:58:03.469 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649775179}) 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4280650109}) 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142338922}) 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1323182999}) 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142338922}) 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.458+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=252077211}) 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668089968}) 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2471721177}) 2026-02-17T05:58:03.470 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668089968}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2574600976}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964032846}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3034534472}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964032846}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=722954988}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3583985634}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=17745458}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3583985634}) 2026-02-17T05:58:03.471 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.459+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=311920753}) 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=576864990}) 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2047049568}) 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=576864990}) 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2082239533}) 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3227785627}) 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466189785}) 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3227785627}) 2026-02-17T05:58:03.472 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=372369729}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014184339}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=928017795}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014184339}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.460+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2621401356}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1828263885}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2054876112}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1828263885}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558678490}) 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1335802889}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047664859}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1335802889}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142387490}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4064899582}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=456577074}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4064899582}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.461+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2032324878}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.462+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.462+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121963395}) 2026-02-17T05:58:03.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.462+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2058698721}) 2026-02-17T05:58:03.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.462+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.462+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121963395}) 2026-02-17T05:58:03.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.462+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=121053720}) 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.468+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.468+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130367116}) 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.468+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919677451}) 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.468+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.468+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130367116}) 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.468+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2911431615}) 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625590194}) 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609140097}) 2026-02-17T05:58:03.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625590194}) 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=65382772}) 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2532860666}) 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=524473839}) 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2532860666}) 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=432418887}) 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=744787409}) 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1352154937}) 2026-02-17T05:58:03.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=744787409}) 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.469+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4017800320}) 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1227816659}) 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519770123}) 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1227816659}) 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1464732201}) 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4231373811}) 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=645414727}) 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4231373811}) 2026-02-17T05:58:03.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2706983624}) 2026-02-17T05:58:03.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520950377}) 2026-02-17T05:58:03.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1076463126}) 2026-02-17T05:58:03.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520950377}) 2026-02-17T05:58:03.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.470+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3106401121}) 2026-02-17T05:58:03.517 INFO:tasks.rados.rados.0.trial115.stdout:2055: finishing write tid 1 to trial11545420-44 2026-02-17T05:58:03.517 INFO:tasks.rados.rados.0.trial115.stdout:2055: finishing write tid 2 to trial11545420-44 2026-02-17T05:58:03.517 INFO:tasks.rados.rados.0.trial115.stdout:2053: done (5 left) 2026-02-17T05:58:03.517 INFO:tasks.rados.rados.0.trial115.stdout:2054: done (4 left) 2026-02-17T05:58:03.517 INFO:tasks.rados.rados.0.trial115.stdout:2058: done (3 left) 2026-02-17T05:58:03.517 INFO:tasks.rados.rados.0.trial115.stdout:2059: write (excl) oid 5 current snap is 204 2026-02-17T05:58:03.518 INFO:tasks.rados.rados.0.trial115.stdout:2059: seq_num 744 ranges {504408=471497,1620603=557925} 2026-02-17T05:58:03.520 INFO:tasks.rados.rados.0.trial115.stdout:2059: writing trial11545420-5 from 504408 to 975905 tid 1 2026-02-17T05:58:03.524 INFO:tasks.rados.rados.0.trial115.stdout:2059: writing trial11545420-5 from 1620603 to 2178528 tid 2 2026-02-17T05:58:03.525 INFO:tasks.rados.rados.0.trial115.stdout:2060: write oid 35 current snap is 204 2026-02-17T05:58:03.525 INFO:tasks.rados.rados.0.trial115.stdout:2060: seq_num 745 ranges {521215=546746,1562732=701583} 2026-02-17T05:58:03.528 INFO:tasks.rados.rados.0.trial115.stdout:2060: writing trial11545420-35 from 521215 to 1067961 tid 1 2026-02-17T05:58:03.533 INFO:tasks.rados.rados.0.trial115.stdout:2060: writing trial11545420-35 from 1562732 to 2264315 tid 2 2026-02-17T05:58:03.533 INFO:tasks.rados.rados.0.trial115.stdout:2061: snap_remove snap 196 2026-02-17T05:58:03.588 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.587+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.589 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.587+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=67,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=5} crc {logm=551372580,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3242229648,mon_config_key=2407814413}) 2026-02-17T05:58:03.589 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.587+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=70,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=2} crc {logm=3586662594,mdsmap=1160443688,mgr=332162300,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3242229648,mon_config_key=3615360532}) 2026-02-17T05:58:03.589 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.587+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.589 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.587+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=67,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=5} crc {logm=551372580,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3242229648,mon_config_key=2407814413}) 2026-02-17T05:58:03.589 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.587+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=4} crc {logm=3695463527,mdsmap=1160443688,mgr=2723955629,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3242229648,mon_config_key=3379246293}) 2026-02-17T05:58:03.876 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.876 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=75,osdmap=8} crc {mon_config_key=3604644689,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=894306668,osd_snap=2578507547,osdmap=450704525}) 2026-02-17T05:58:03.876 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=4} crc {mon_config_key=4037059662,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=894306668,osd_snap=826396133,osdmap=3169036313}) 2026-02-17T05:58:03.876 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.876 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=75,osdmap=8} crc {mon_config_key=3604644689,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=894306668,osd_snap=2578507547,osdmap=450704525}) 2026-02-17T05:58:03.876 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=78,osdmap=5} crc {mon_config_key=3278643186,monmap=1784110042,osd_metadata=278982410,osd_pg_creating=894306668,osd_snap=1535383297,osdmap=3408388461}) 2026-02-17T05:58:03.876 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.876 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1765177799}) 2026-02-17T05:58:03.877 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2029739907}) 2026-02-17T05:58:03.877 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.877 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1765177799}) 2026-02-17T05:58:03.877 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.875+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3059455424}) 2026-02-17T05:58:03.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.879+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.879+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=43} crc {osdmap=3016872283}) 2026-02-17T05:58:03.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.879+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=42} crc {osdmap=1507020579}) 2026-02-17T05:58:03.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.879+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:58:03.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.879+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=43} crc {osdmap=3016872283}) 2026-02-17T05:58:03.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:58:03.879+0000 7ffaf2547640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=40} crc {osdmap=275686171}) 2026-02-17T05:58:04.369 INFO:tasks.rados.rados.0.trial115.stdout:2061: done (5 left) 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:2062: copy_from oid 11 from oid 6 current snap is 204 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:2055: finishing write tid 3 to trial11545420-44 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:2055: finishing write tid 4 to trial11545420-44 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:2055: oid 44 updating version 0 to 875 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:2055: oid 44 updating version 875 to 876 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:2055: oid 44 updating version 876 to 877 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 877 (ObjNum 740 snap 204 seq_num 740) dirty exists 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:2055: left oid 44 (ObjNum 740 snap 204 seq_num 740) 2026-02-17T05:58:04.370 INFO:tasks.rados.rados.0.trial115.stdout:2056: finishing write tid 1 to trial11545420-26 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2056: finishing write tid 2 to trial11545420-26 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2056: finishing write tid 3 to trial11545420-26 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2056: finishing write tid 4 to trial11545420-26 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2056: oid 26 updating version 0 to 619 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2056: oid 26 updating version 619 to 620 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2056: oid 26 version 620 is already newer than 618 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 620 (ObjNum 741 snap 204 seq_num 741) dirty exists 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2056: left oid 26 (ObjNum 741 snap 204 seq_num 741) 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2057: finishing write tid 1 to trial11545420-40 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2057: finishing write tid 2 to trial11545420-40 2026-02-17T05:58:04.371 INFO:tasks.rados.rados.0.trial115.stdout:2057: finishing write tid 3 to trial11545420-40 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2057: finishing write tid 4 to trial11545420-40 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2057: oid 40 updating version 0 to 896 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2057: oid 40 updating version 896 to 898 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2057: oid 40 version 898 is already newer than 897 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 898 (ObjNum 742 snap 204 seq_num 742) dirty exists 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2057: left oid 40 (ObjNum 742 snap 204 seq_num 742) 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2059: finishing write tid 1 to trial11545420-5 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2055: done (5 left) 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2056: done (4 left) 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2057: done (3 left) 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2063: read oid 7 snap 191 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2063: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:58:04.372 INFO:tasks.rados.rados.0.trial115.stdout:2064: delete oid 25 current snap is 204 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2059: finishing write tid 2 to trial11545420-5 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2059: finishing write tid 3 to trial11545420-5 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2059: finishing write tid 4 to trial11545420-5 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2059: oid 5 updating version 0 to 966 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2059: oid 5 updating version 966 to 967 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2059: oid 5 updating version 967 to 968 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 968 (ObjNum 744 snap 204 seq_num 744) dirty exists 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2059: left oid 5 (ObjNum 744 snap 204 seq_num 744) 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2060: finishing write tid 1 to trial11545420-35 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2060: finishing write tid 2 to trial11545420-35 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2060: finishing write tid 3 to trial11545420-35 2026-02-17T05:58:04.373 INFO:tasks.rados.rados.0.trial115.stdout:2060: finishing write tid 4 to trial11545420-35 2026-02-17T05:58:04.374 INFO:tasks.rados.rados.0.trial115.stdout:2060: oid 35 updating version 0 to 1122 2026-02-17T05:58:04.374 INFO:tasks.rados.rados.0.trial115.stdout:2060: oid 35 version 1122 is already newer than 1121 2026-02-17T05:58:04.374 INFO:tasks.rados.rados.0.trial115.stdout:2060: oid 35 updating version 1122 to 1123 2026-02-17T05:58:04.374 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1123 (ObjNum 745 snap 204 seq_num 745) dirty exists 2026-02-17T05:58:04.374 INFO:tasks.rados.rados.0.trial115.stdout:2060: left oid 35 (ObjNum 745 snap 204 seq_num 745) 2026-02-17T05:58:04.374 INFO:tasks.rados.rados.0.trial115.stdout:2059: done (4 left) 2026-02-17T05:58:04.375 INFO:tasks.rados.rados.0.trial115.stdout:2060: done (3 left) 2026-02-17T05:58:04.375 INFO:tasks.rados.rados.0.trial115.stdout:2064: done (2 left) 2026-02-17T05:58:04.375 INFO:tasks.rados.rados.0.trial115.stdout:2065: write (excl) oid 28 current snap is 204 2026-02-17T05:58:04.375 INFO:tasks.rados.rados.0.trial115.stdout:2065: seq_num 747 ranges {554829=413597,1396049=705252} 2026-02-17T05:58:04.375 INFO:tasks.rados.rados.0.trial115.stdout:2065: writing trial11545420-28 from 554829 to 968426 tid 1 2026-02-17T05:58:04.380 INFO:tasks.rados.rados.0.trial115.stdout:2065: writing trial11545420-28 from 1396049 to 2101301 tid 2 2026-02-17T05:58:04.380 INFO:tasks.rados.rados.0.trial115.stdout:2066: snap_create 2026-02-17T05:58:04.380 INFO:tasks.rados.rados.0.trial115.stdout:2063: expect (ObjNum 598 snap 155 seq_num 598) 2026-02-17T05:58:04.447 INFO:tasks.rados.rados.0.trial115.stdout:2062: finishing copy_from racing read to trial11545420-11 2026-02-17T05:58:04.447 INFO:tasks.rados.rados.0.trial115.stdout:2062: finishing copy_from to trial11545420-11 2026-02-17T05:58:04.447 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 829 (ObjNum 657 snap 178 seq_num 657) dirty exists 2026-02-17T05:58:04.447 INFO:tasks.rados.rados.0.trial115.stdout:2065: finishing write tid 1 to trial11545420-28 2026-02-17T05:58:04.447 INFO:tasks.rados.rados.0.trial115.stdout:2065: finishing write tid 2 to trial11545420-28 2026-02-17T05:58:04.447 INFO:tasks.rados.rados.0.trial115.stdout:2065: finishing write tid 3 to trial11545420-28 2026-02-17T05:58:04.447 INFO:tasks.rados.rados.0.trial115.stdout:2065: finishing write tid 4 to trial11545420-28 2026-02-17T05:58:04.447 INFO:tasks.rados.rados.0.trial115.stdout:2065: oid 28 updating version 0 to 1020 2026-02-17T05:58:04.448 INFO:tasks.rados.rados.0.trial115.stdout:2065: oid 28 version 1020 is already newer than 1018 2026-02-17T05:58:04.448 INFO:tasks.rados.rados.0.trial115.stdout:2065: oid 28 version 1020 is already newer than 1019 2026-02-17T05:58:04.448 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1020 (ObjNum 747 snap 204 seq_num 747) dirty exists 2026-02-17T05:58:04.448 INFO:tasks.rados.rados.0.trial115.stdout:2065: left oid 28 (ObjNum 747 snap 204 seq_num 747) 2026-02-17T05:58:04.457 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:58:04.457 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.522+0000 7fd10156e640 10 monclient: get_monmap_and_config 2026-02-17T05:58:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.522+0000 7fd10156e640 10 monclient: build_initial_monmap 2026-02-17T05:58:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.522+0000 7fd10156e640 10 monclient: monmap: 2026-02-17T05:58:04.526 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:04.526 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.526 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:04.523956+0000 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:04.523956+0000 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 1 Processor -- start 2026-02-17T05:58:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 1 -- start start 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient: init 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06c890 0x7fd0fc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient(hunting): picked mon.noname-b con 0x7fd0fc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd0fc09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd0fc06d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0fc135c10 con 0x7fd0fc09c4a0 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0fc06daf0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0fc0a0880 con 0x7fd0fc06c890 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd10156e640 10 monclient(hunting): authenticate will time out at 1330.077505s 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06c890 0x7fd0fc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0fa7fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd0fc09c4a0 0x7fd0fc134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:36498/0 (socket says 10.20.193.78:36498) 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0fa7fc640 1 -- 10.20.193.78:0/1622547069 learned_addr learned my addr 10.20.193.78:0/1622547069 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient(hunting): get_auth_request con 0x7fd0fc06c890 auth_method 0 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient(hunting): handle_auth_done global_id 144200 payload 274 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 1 -- 10.20.193.78:0/1622547069 >> v1:10.20.193.115:6789/0 conn(0x7fd0fc06d1b0 legacy=0x7fd0fc135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 1 -- 10.20.193.78:0/1622547069 >> v1:10.20.193.78:6789/0 conn(0x7fd0fc09c4a0 legacy=0x7fd0fc134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient: _finish_hunting 0 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 1 monclient: found mon.noname-b 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.523+0000 7fd0faffd640 1 -- 10.20.193.78:0/1622547069 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc09cc80 con 0x7fd0fc06c890 2026-02-17T05:58:04.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0faffd640 1 -- 10.20.193.78:0/1622547069 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06c890 msgr2=0x7fd0fc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0faffd640 1 --2- 10.20.193.78:0/1622547069 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06c890 0x7fd0fc06cc70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00098d0 tx=0x7fd0f002ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a21abb608c7226 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 1 -- 10.20.193.78:0/1622547069 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f003c070 con 0x7fd0fc06c890 2026-02-17T05:58:04.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:04.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: dump: 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 1 -- 10.20.193.78:0/1622547069 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f002fcc0 con 0x7fd0fc06c890 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 1 -- 10.20.193.78:0/1622547069 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0037660 con 0x7fd0fc06c890 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd0f9ffb640 10 monclient: dump: 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.540 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 5 monclient: authenticate success, global_id 144200 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 4 set_mon_vals no callback set 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 10 monclient: shutdown 2026-02-17T05:58:04.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 1 -- 10.20.193.78:0/1622547069 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06c890 msgr2=0x7fd0fc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 1 --2- 10.20.193.78:0/1622547069 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06c890 0x7fd0fc06cc70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00098d0 tx=0x7fd0f002ed60 comp rx=0 tx=0).stop 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 1 -- 10.20.193.78:0/1622547069 shutdown_connections 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 1 --2- 10.20.193.78:0/1622547069 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06c890 0x7fd0fc06cc70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 1 -- 10.20.193.78:0/1622547069 >> 10.20.193.78:0/1622547069 conn(0x7fd0fc0a6620 msgr2=0x7fd0fc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 1 -- 10.20.193.78:0/1622547069 shutdown_connections 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.526+0000 7fd10156e640 1 -- 10.20.193.78:0/1622547069 wait complete. 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient: build_initial_monmap 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient: monmap: 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:04.542 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 1 Processor -- start 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 1 -- start start 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient: init 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:04.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0fc06c890 0x7fd0fc1cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient(hunting): picked mon.noname-a con 0x7fd0fc06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06d1b0 0x7fd0fc1347b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient(hunting): picked mon.noname-c con 0x7fd0fc06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0fc09c4a0 0x7fd0fc134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient(hunting): picked mon.noname-b con 0x7fd0fc09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc0a0020 con 0x7fd0fc06c890 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc06d600 con 0x7fd0fc09c4a0 2026-02-17T05:58:04.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0fc0a01a0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd10156e640 10 monclient(hunting): authenticate will time out at 1330.081669s 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06d1b0 0x7fd0fc1347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0fc06c890 0x7fd0fc1cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06d1b0 0x7fd0fc1347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57622/0 (socket says 10.20.193.78:57622) 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 -- 10.20.193.78:0/627706438 learned_addr learned my addr 10.20.193.78:0/627706438 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient(hunting): get_auth_request con 0x7fd0fc06d1b0 auth_method 0 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 10 monclient(hunting): get_auth_request con 0x7fd0fc06c890 auth_method 0 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fb7fe640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0fc09c4a0 0x7fd0fc134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fb7fe640 10 monclient(hunting): get_auth_request con 0x7fd0fc09c4a0 auth_method 0 2026-02-17T05:58:04.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient(hunting): handle_auth_done global_id 144203 payload 824 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 -- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0fc09c4a0 msgr2=0x7fd0fc134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0fc09c4a0 0x7fd0fc134cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 -- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0fc06c890 msgr2=0x7fd0fc1cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0fc06c890 0x7fd0fc1cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 monclient: found mon.noname-c 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0fa7fc640 1 -- 10.20.193.78:0/627706438 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc125ee0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.527+0000 7fd0faffd640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0fc06c890 0x7fd0fc1cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0fa7fc640 1 -- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06d1b0 msgr2=0x7fd0fc1347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0fa7fc640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06d1b0 0x7fd0fc1347b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c750 tx=0x7fd0e400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 1 -- 10.20.193.78:0/627706438 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4005de0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: dump: 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.548 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 1 -- 10.20.193.78:0/627706438 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e4007070 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 1 -- 10.20.193.78:0/627706438 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4007440 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0f97fa640 4 set_mon_vals no callback set 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 10 monclient: dump: 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.550 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 5 monclient: authenticate success, global_id 144203 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 10 monclient: _renew_subs 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc126ad0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 10 monclient: _renew_subs 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc1cfcf0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0fc06cc70 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.530+0000 7fd0db7fe640 1 -- 10.20.193.78:0/627706438 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd0e4007660 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.531+0000 7fd0db7fe640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0c803deb0 0x7fd0c8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.531+0000 7fd0db7fe640 1 -- 10.20.193.78:0/627706438 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(430..430 src has 335..430) ==== 4843+0+0 (secure 0 0 0) 0x7fd0e40526c0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.531+0000 7fd0faffd640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0c803deb0 0x7fd0c8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.531+0000 7fd0faffd640 10 monclient: get_auth_request con 0x7fd0c803deb0 auth_method 0 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.533+0000 7fd0faffd640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0c803deb0 0x7fd0c8040370 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0a7600 tx=0x7fd0f0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.533+0000 7fd0db7fe640 1 -- 10.20.193.78:0/627706438 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd0e4011a00 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.533+0000 7fd0db7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.533+0000 7fd0db7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:04.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.651+0000 7fd10156e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:04.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.651+0000 7fd10156e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:04.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.651+0000 7fd10156e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.651+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd0fc1257d0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.651+0000 7fd0db7fe640 1 -- 10.20.193.78:0/627706438 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd0e400a8d0 con 0x7fd0fc06d1b0 2026-02-17T05:58:04.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.651+0000 7fd0db7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:04.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.651+0000 7fd0db7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:04.652 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:04.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0c803deb0 msgr2=0x7fd0c8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0c803deb0 0x7fd0c8040370 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0a7600 tx=0x7fd0f0033000 comp rx=0 tx=0).stop 2026-02-17T05:58:04.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 10 monclient: shutdown 2026-02-17T05:58:04.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06d1b0 msgr2=0x7fd0fc1347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06d1b0 0x7fd0fc1347b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c750 tx=0x7fd0e400cc20 comp rx=0 tx=0).stop 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 shutdown_connections 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0c803deb0 0x7fd0c8040370 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0fc09c4a0 0x7fd0fc134cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0fc06d1b0 0x7fd0fc1347b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 --2- 10.20.193.78:0/627706438 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0fc06c890 0x7fd0fc1cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 >> 10.20.193.78:0/627706438 conn(0x7fd0fc0a6620 msgr2=0x7fd0fc09e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 shutdown_connections 2026-02-17T05:58:04.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.652+0000 7fd10156e640 1 -- 10.20.193.78:0/627706438 wait complete. 2026-02-17T05:58:04.660 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:58:04.661 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:58:04.661 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:58:04.661 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:58:04.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.727+0000 7f21a9ade640 10 monclient: get_monmap_and_config 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.727+0000 7f21a9ade640 10 monclient: build_initial_monmap 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.727+0000 7f21a9ade640 10 monclient: monmap: 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:04.728824+0000 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:04.728824+0000 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.731 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.727+0000 7f21a9ade640 1 Processor -- start 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 1 -- start start 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient: init 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2026-02-17T05:58:04.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f21a406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f21a409c4a0 0x7f21a413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient(hunting): picked mon.noname-c con 0x7f21a409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406d380 0x7f21a413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient(hunting): picked mon.noname-b con 0x7f21a406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a406daf0 con 0x7f21a406c890 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f21a40a0880 con 0x7f21a409c4a0 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f21a409cc80 con 0x7f21a406d380 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a9ade640 10 monclient(hunting): authenticate will time out at 1330.282325s 2026-02-17T05:58:04.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406d380 0x7f21a413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406d380 0x7f21a413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57636/0 (socket says 10.20.193.78:57636) 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 1 -- 10.20.193.78:0/52851716 learned_addr learned my addr 10.20.193.78:0/52851716 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient(hunting): get_auth_request con 0x7f21a406d380 auth_method 0 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a2ffd640 1 --2- 10.20.193.78:0/52851716 >> v2:10.20.193.115:3300/0 conn(0x7f21a409c4a0 0x7f21a413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a2ffd640 10 monclient(hunting): get_auth_request con 0x7f21a409c4a0 auth_method 0 2026-02-17T05:58:04.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient(hunting): handle_auth_done global_id 144206 payload 274 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 1 -- 10.20.193.78:0/52851716 >> v2:10.20.193.115:3300/0 conn(0x7f21a409c4a0 msgr2=0x7f21a413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 1 --2- 10.20.193.78:0/52851716 >> v2:10.20.193.115:3300/0 conn(0x7f21a409c4a0 0x7f21a413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 1 -- 10.20.193.78:0/52851716 >> v1:10.20.193.115:6789/0 conn(0x7f21a406c890 legacy=0x7f21a406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 1 monclient: found mon.noname-b 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.728+0000 7f21a3fff640 1 -- 10.20.193.78:0/52851716 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a409cf60 con 0x7f21a406d380 2026-02-17T05:58:04.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.730+0000 7f21a3fff640 1 -- 10.20.193.78:0/52851716 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406d380 msgr2=0x7f21a413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.730+0000 7f21a3fff640 1 --2- 10.20.193.78:0/52851716 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406d380 0x7f21a413d8f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2198009e50 tx=0x7f219802f260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66ac2245c37945b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 1 -- 10.20.193.78:0/52851716 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21980373f0 con 0x7f21a406d380 2026-02-17T05:58:04.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:04.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: dump: 2026-02-17T05:58:04.744 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:04.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 1 -- 10.20.193.78:0/52851716 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2198037590 con 0x7f21a406d380 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 1 -- 10.20.193.78:0/52851716 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198037960 con 0x7f21a406d380 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 5 monclient: authenticate success, global_id 144206 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 4 set_mon_vals no callback set 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a27fc640 10 monclient: dump: 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.746 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 10 monclient: shutdown 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 1 -- 10.20.193.78:0/52851716 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406d380 msgr2=0x7f21a413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 1 --2- 10.20.193.78:0/52851716 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406d380 0x7f21a413d8f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2198009e50 tx=0x7f219802f260 comp rx=0 tx=0).stop 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 1 -- 10.20.193.78:0/52851716 shutdown_connections 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 1 --2- 10.20.193.78:0/52851716 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406d380 0x7f21a413d8f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 1 --2- 10.20.193.78:0/52851716 >> v2:10.20.193.115:3300/0 conn(0x7f21a409c4a0 0x7f21a413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 1 -- 10.20.193.78:0/52851716 >> 10.20.193.78:0/52851716 conn(0x7f21a40a6620 msgr2=0x7f21a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 1 -- 10.20.193.78:0/52851716 shutdown_connections 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 1 -- 10.20.193.78:0/52851716 wait complete. 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 10 monclient: build_initial_monmap 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.731+0000 7f21a9ade640 10 monclient: monmap: 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:04.748 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 1 Processor -- start 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 1 -- start start 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient: init 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:58:04.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406c890 0x7f21a413c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient(hunting): picked mon.noname-c con 0x7f21a406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a406d380 0x7f21a409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient(hunting): picked mon.noname-a con 0x7f21a406d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21a409c4a0 0x7f21a409e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient(hunting): picked mon.noname-b con 0x7f21a409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21a409bfa0 con 0x7f21a406d380 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f21a409be20 con 0x7f21a409c4a0 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f21a413de30 con 0x7f21a406c890 2026-02-17T05:58:04.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a9ade640 10 monclient(hunting): authenticate will time out at 1330.286345s 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406c890 0x7f21a413c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406c890 0x7f21a413c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57642/0 (socket says 10.20.193.78:57642) 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a406d380 0x7f21a409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 -- 10.20.193.78:0/303085261 learned_addr learned my addr 10.20.193.78:0/303085261 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient(hunting): get_auth_request con 0x7f21a406c890 auth_method 0 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 10 monclient(hunting): get_auth_request con 0x7f21a406d380 auth_method 0 2026-02-17T05:58:04.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a3fff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21a409c4a0 0x7f21a409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a3fff640 10 monclient(hunting): get_auth_request con 0x7f21a409c4a0 auth_method 0 2026-02-17T05:58:04.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient(hunting): handle_auth_done global_id 144209 payload 824 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21a409c4a0 msgr2=0x7f21a409e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21a409c4a0 0x7f21a409e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a406d380 msgr2=0x7f21a409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a406d380 0x7f21a409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 monclient: found mon.noname-c 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a37fe640 1 -- 10.20.193.78:0/303085261 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a413b9d0 con 0x7f21a406c890 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a3fff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21a409c4a0 0x7f21a409e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.732+0000 7f21a2ffd640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a406d380 0x7f21a409db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a37fe640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406c890 msgr2=0x7f21a413c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a37fe640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406c890 0x7f21a413c800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f219000c600 tx=0x7f219000cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190012a40 con 0x7f21a406c890 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: dump: 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.754 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: _finish_auth 0 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2190017070 con 0x7f21a406c890 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:04.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190012e20 con 0x7f21a406c890 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a1ffb640 4 set_mon_vals no callback set 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f217ffff640 10 monclient: dump: 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.756 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a9ade640 5 monclient: authenticate success, global_id 144209 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a9ade640 10 monclient: _renew_subs 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a9ade640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a413bcb0 con 0x7f21a406c890 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a9ade640 10 monclient: _renew_subs 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a9ade640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.734+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a413d9f0 con 0x7f21a406c890 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a9ade640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a9ade640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a41338b0 0x7f21a409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a9ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21a4074980 con 0x7f21a41338b0 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a41338b0 0x7f21a409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 10 monclient: get_auth_request con 0x7f21a41338b0 auth_method 0 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_done global_id 154146 payload 824 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f21a2ffd640 1 -- 10.20.193.78:0/303085261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21a406cc70 con 0x7f21a41338b0 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f219001c460 con 0x7f21a406c890 2026-02-17T05:58:04.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f217ffff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f217403deb0 0x7f2174040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.735+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(430..430 src has 335..430) ==== 4843+0+0 (secure 0 0 0) 0x7f2190052680 con 0x7f21a406c890 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.736+0000 7f21a3fff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f217403deb0 0x7f2174040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.736+0000 7f21a3fff640 10 monclient: get_auth_request con 0x7f217403deb0 auth_method 0 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.737+0000 7f21a2ffd640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a41338b0 msgr2=0x7f21a409e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.737+0000 7f21a2ffd640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a41338b0 0x7f21a409e5d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21a413af90 tx=0x7f2194011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.737+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194019070 con 0x7f21a41338b0 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.737+0000 7f217ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.737+0000 7f217ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.737+0000 7f217ffff640 10 monclient: dump: 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.759 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.738+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2194015660 con 0x7f21a41338b0 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.738+0000 7f217ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.738+0000 7f217ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.738+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a41338b0 msgr2=0x7f21a409e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.738+0000 7f217ffff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a41338b0 0x7f21a409e5d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21a413af90 tx=0x7f2194011f00 comp rx=0 tx=0).stop 2026-02-17T05:58:04.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.738+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a41338b0 msgr2=0x7f21a409e5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:04.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.738+0000 7f217ffff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a41338b0 0x7f21a409e5d0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21a413af90 tx=0x7f2194011f00 comp rx=0 tx=0).stop 2026-02-17T05:58:04.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.738+0000 7f21a3fff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f217403deb0 0x7f2174040370 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f21a406a480 tx=0x7f2198033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a9ade640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:58:04.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a9ade640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a40714b0 0x7f21a41d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a9ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21a413bef0 con 0x7f21a40714b0 2026-02-17T05:58:04.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a40714b0 0x7f21a41d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 10 monclient: get_auth_request con 0x7f21a40714b0 auth_method 0 2026-02-17T05:58:04.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 10 monclient(hunting): handle_auth_done global_id 154149 payload 824 2026-02-17T05:58:04.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.741+0000 7f21a2ffd640 1 -- 10.20.193.78:0/303085261 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f21a40744d0 con 0x7f21a40714b0 2026-02-17T05:58:04.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a2ffd640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a40714b0 msgr2=0x7f21a41d9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a2ffd640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a40714b0 0x7f21a41d9d00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f21a413af90 tx=0x7f2194004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219400f040 con 0x7f21a40714b0 2026-02-17T05:58:04.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:04.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 10 monclient: dump: 2026-02-17T05:58:04.766 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.766 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.766 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.766 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.766 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.766 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.767 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.767 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.767 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.767 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f21940143e0 con 0x7f21a40714b0 2026-02-17T05:58:04.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:58:04.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:04.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a40714b0 msgr2=0x7f21a41d9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a40714b0 0x7f21a41d9d00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f21a413af90 tx=0x7f2194004830 comp rx=0 tx=0).stop 2026-02-17T05:58:04.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a40714b0 msgr2=0x7f21a41d9d00 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:04.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f217ffff640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21a40714b0 0x7f21a41d9d00 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f21a413af90 tx=0x7f2194004830 comp rx=0 tx=0).stop 2026-02-17T05:58:04.768 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:58:04.769 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:04.769 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:04.769 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:58:04.769 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 136, 2026-02-17T05:58:04.769 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 16794, 2026-02-17T05:58:04.769 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:58:04.769 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:58:04.770 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:58:04.770 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:58:04.770 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.770 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T05:58:04.770 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:04.770 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:58:04.770 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:58:04.770 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.771 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.771 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.771 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.771 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.771 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.771 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.771 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.771 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.772 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.773 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:04.774 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:58:04.775 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.776 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:04.777 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.779 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.780 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.781 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.782 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.783 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:58:04.784 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f217403deb0 msgr2=0x7f2174040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f217403deb0 0x7f2174040370 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f21a406a480 tx=0x7f2198033000 comp rx=0 tx=0).stop 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 10 monclient: shutdown 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406c890 msgr2=0x7f21a413c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406c890 0x7f21a413c800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f219000c600 tx=0x7f219000cad0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a3fff640 1 -- 10.20.193.78:0/303085261 reap_dead start 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 shutdown_connections 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 --2- 10.20.193.78:0/303085261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21a406c890 0x7f21a413c800 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 >> 10.20.193.78:0/303085261 conn(0x7f21a40a6620 msgr2=0x7f21a40a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 shutdown_connections 2026-02-17T05:58:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.744+0000 7f21a9ade640 1 -- 10.20.193.78:0/303085261 wait complete. 2026-02-17T05:58:04.786 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:58:04.787 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:58:04.787 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:58:04.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.852+0000 7f3bfa091640 10 monclient: get_monmap_and_config 2026-02-17T05:58:04.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.852+0000 7f3bfa091640 10 monclient: build_initial_monmap 2026-02-17T05:58:04.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.852+0000 7f3bfa091640 10 monclient: monmap: 2026-02-17T05:58:04.853 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:04.853 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.853 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:04.853718+0000 2026-02-17T05:58:04.853 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:04.853718+0000 2026-02-17T05:58:04.854 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:04.854 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.854 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:04.854 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:04.854 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:04.854 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:04.854 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:04.854 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.852+0000 7f3bfa091640 1 Processor -- start 2026-02-17T05:58:04.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.852+0000 7f3bfa091640 1 -- start start 2026-02-17T05:58:04.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.852+0000 7f3bfa091640 10 monclient: init 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3bf406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3bf409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3bf406d380 0x7f3bf413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 10 monclient(hunting): picked mon.noname-c con 0x7f3bf406d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf406daf0 con 0x7f3bf409c4a0 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf409cc80 con 0x7f3bf406c890 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f3bf40a0880 con 0x7f3bf406d380 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bfa091640 10 monclient(hunting): authenticate will time out at 1330.407271s 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf2ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f3bf409c4a0 0x7f3bf413b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:36506/0 (socket says 10.20.193.78:36506) 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf2ffd640 1 -- 10.20.193.78:0/3755733666 learned_addr learned my addr 10.20.193.78:0/3755733666 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 1 --2- 10.20.193.78:0/3755733666 >> v2:10.20.193.115:3300/0 conn(0x7f3bf406d380 0x7f3bf413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient(hunting): get_auth_request con 0x7f3bf406d380 auth_method 0 2026-02-17T05:58:04.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient(hunting): handle_auth_done global_id 134269 payload 274 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 1 -- 10.20.193.78:0/3755733666 >> v1:10.20.193.115:6789/0 conn(0x7f3bf406c890 legacy=0x7f3bf406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 1 -- 10.20.193.78:0/3755733666 >> v1:10.20.193.78:6789/0 conn(0x7f3bf409c4a0 legacy=0x7f3bf413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 1 monclient: found mon.noname-c 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:58:04.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.853+0000 7f3bf3fff640 1 -- 10.20.193.78:0/3755733666 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf409cf60 con 0x7f3bf406d380 2026-02-17T05:58:04.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf3fff640 1 -- 10.20.193.78:0/3755733666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 msgr2=0x7f3bf413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf3fff640 1 --2- 10.20.193.78:0/3755733666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 0x7f3bf413ead0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4074de0 tx=0x7f3be802f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2af9dcabc8263b74 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 1 -- 10.20.193.78:0/3755733666 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be803c070 con 0x7f3bf406d380 2026-02-17T05:58:04.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:04.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: dump: 2026-02-17T05:58:04.863 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.863 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.863 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:04.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 1 -- 10.20.193.78:0/3755733666 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be8037640 con 0x7f3bf406d380 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 5 monclient: authenticate success, global_id 134269 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 1 -- 10.20.193.78:0/3755733666 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8037a10 con 0x7f3bf406d380 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 4 set_mon_vals no callback set 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:04.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bf27fc640 10 monclient: dump: 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.866 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 10 monclient: shutdown 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 1 -- 10.20.193.78:0/3755733666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 msgr2=0x7f3bf413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 1 --2- 10.20.193.78:0/3755733666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 0x7f3bf413ead0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4074de0 tx=0x7f3be802f130 comp rx=0 tx=0).stop 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 1 -- 10.20.193.78:0/3755733666 shutdown_connections 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 1 --2- 10.20.193.78:0/3755733666 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 0x7f3bf413ead0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 1 -- 10.20.193.78:0/3755733666 >> 10.20.193.78:0/3755733666 conn(0x7f3bf40a6620 msgr2=0x7f3bf40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 1 -- 10.20.193.78:0/3755733666 shutdown_connections 2026-02-17T05:58:04.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.856+0000 7f3bfa091640 1 -- 10.20.193.78:0/3755733666 wait complete. 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient: build_initial_monmap 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient: monmap: 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.868 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 1 Processor -- start 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 1 -- start start 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient: init 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:58:04.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf406c890 0x7f3bf41cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient(hunting): picked mon.noname-c con 0x7f3bf406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 0x7f3bf4136300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient(hunting): picked mon.noname-b con 0x7f3bf406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3bf409c4a0 0x7f3bf411dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient(hunting): picked mon.noname-a con 0x7f3bf409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf409bfa0 con 0x7f3bf409c4a0 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf409be20 con 0x7f3bf406d380 2026-02-17T05:58:04.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf413f010 con 0x7f3bf406c890 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bfa091640 10 monclient(hunting): authenticate will time out at 1330.411630s 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf406c890 0x7f3bf41cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3bf409c4a0 0x7f3bf411dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf406c890 0x7f3bf41cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57652/0 (socket says 10.20.193.78:57652) 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3bf409c4a0 0x7f3bf411dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:59330/0 (socket says 10.20.193.78:59330) 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 1 -- 10.20.193.78:0/1475274359 learned_addr learned my addr 10.20.193.78:0/1475274359 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:04.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 10 monclient(hunting): get_auth_request con 0x7f3bf406c890 auth_method 0 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient(hunting): get_auth_request con 0x7f3bf409c4a0 auth_method 0 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf2ffd640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 0x7f3bf4136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf2ffd640 10 monclient(hunting): get_auth_request con 0x7f3bf406d380 auth_method 0 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient(hunting): handle_auth_done global_id 154152 payload 824 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf406c890 msgr2=0x7f3bf41cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf406c890 0x7f3bf41cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 msgr2=0x7f3bf4136300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3bf406d380 0x7f3bf4136300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 1 monclient: found mon.noname-a 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:04.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.857+0000 7f3bf3fff640 1 -- 10.20.193.78:0/1475274359 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf411f9b0 con 0x7f3bf409c4a0 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf3fff640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3bf409c4a0 msgr2=0x7f3bf411dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf3fff640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3bf409c4a0 0x7f3bf411dd60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3be8036000 tx=0x7f3be8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8047020 con 0x7f3bf409c4a0 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: dump: 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.875 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: _finish_auth 0 2026-02-17T05:58:04.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 5 monclient: authenticate success, global_id 154152 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be8042880 con 0x7f3bf409c4a0 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 10 monclient: _renew_subs 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf413e4f0 con 0x7f3bf409c4a0 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be803c040 con 0x7f3bf409c4a0 2026-02-17T05:58:04.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 10 monclient: dump: 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.878 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf1ffb640 4 set_mon_vals no callback set 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 10 monclient: _renew_subs 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf406fcc0 con 0x7f3bf409c4a0 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf4136840 0x7f3bf411e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf409d870 con 0x7f3bf4136840 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3be8043420 con 0x7f3bf409c4a0 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf4136840 0x7f3bf411e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 10 monclient: get_auth_request con 0x7f3bf4136840 auth_method 0 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_done global_id 144215 payload 824 2026-02-17T05:58:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.860+0000 7f3bf37fe640 1 -- 10.20.193.78:0/1475274359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3bf406cc70 con 0x7f3bf4136840 2026-02-17T05:58:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.862+0000 7f3bcffff640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3bc403dee0 0x7f3bc40403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.862+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(430..430 src has 335..430) ==== 4843+0+0 (secure 0 0 0) 0x7f3be8078640 con 0x7f3bf409c4a0 2026-02-17T05:58:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.862+0000 7f3bf2ffd640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3bc403dee0 0x7f3bc40403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.862+0000 7f3bf2ffd640 10 monclient: get_auth_request con 0x7f3bc403dee0 auth_method 0 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bf37fe640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf4136840 msgr2=0x7f3bf411e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bf37fe640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf4136840 0x7f3bf411e2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3be000f780 tx=0x7f3be000fc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0018030 con 0x7f3bf4136840 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 10 monclient: dump: 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.884 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f3be000dbc0 con 0x7f3bf4136840 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf4136840 msgr2=0x7f3bf411e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf4136840 0x7f3bf411e2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3be000f780 tx=0x7f3be000fc50 comp rx=0 tx=0).stop 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf4136840 msgr2=0x7f3bf411e2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.863+0000 7f3bcffff640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf4136840 0x7f3bf411e2a0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3be000f780 tx=0x7f3be000fc50 comp rx=0 tx=0).stop 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.864+0000 7f3bf2ffd640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3bc403dee0 0x7f3bc40403a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3bf411f3d0 tx=0x7f3be4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bfa091640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bfa091640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf411e7e0 0x7f3bf41da380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bfa091640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf411ebf0 con 0x7f3bf411e7e0 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf411e7e0 0x7f3bf41da380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 10 monclient: get_auth_request con 0x7f3bf411e7e0 auth_method 0 2026-02-17T05:58:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 10 monclient(hunting): handle_auth_done global_id 144218 payload 824 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.866+0000 7f3bf37fe640 1 -- 10.20.193.78:0/1475274359 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3bf40744d0 con 0x7f3bf411e7e0 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bf37fe640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf411e7e0 msgr2=0x7f3bf41da380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bf37fe640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf411e7e0 0x7f3bf41da380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3be000f780 tx=0x7f3be0003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0014970 con 0x7f3bf411e7e0 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 10 monclient: dump: 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f3be0014970 con 0x7f3bf411e7e0 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf411e7e0 msgr2=0x7f3bf41da380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf411e7e0 0x7f3bf41da380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3be000f780 tx=0x7f3be0003860 comp rx=0 tx=0).stop 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf411e7e0 msgr2=0x7f3bf41da380 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.868+0000 7f3bcffff640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3bf411e7e0 0x7f3bf41da380 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3be000f780 tx=0x7f3be0003860 comp rx=0 tx=0).stop 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 136, 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 68403, 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:58:04.888 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.889 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:58:04.890 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.891 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:04.892 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.893 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.894 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:58:04.895 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:58:04.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3bc403dee0 msgr2=0x7f3bc40403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3bc403dee0 0x7f3bc40403a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3bf411f3d0 tx=0x7f3be4009040 comp rx=0 tx=0).stop 2026-02-17T05:58:04.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 10 monclient: shutdown 2026-02-17T05:58:04.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3bf409c4a0 msgr2=0x7f3bf411dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 1 --2- 10.20.193.78:0/1475274359 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3bf409c4a0 0x7f3bf411dd60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3be8036000 tx=0x7f3be8004b40 comp rx=0 tx=0).stop 2026-02-17T05:58:04.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bf3fff640 1 -- 10.20.193.78:0/1475274359 reap_dead start 2026-02-17T05:58:04.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bf3fff640 1 -- 10.20.193.78:0/1475274359 reap_dead start 2026-02-17T05:58:04.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 shutdown_connections 2026-02-17T05:58:04.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 >> 10.20.193.78:0/1475274359 conn(0x7f3bf40a6620 msgr2=0x7f3bf40a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 shutdown_connections 2026-02-17T05:58:04.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.870+0000 7f3bfa091640 1 -- 10.20.193.78:0/1475274359 wait complete. 2026-02-17T05:58:04.897 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:58:04.897 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:58:04.897 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.963+0000 7f5f7a691640 10 monclient: get_monmap_and_config 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.963+0000 7f5f7a691640 10 monclient: build_initial_monmap 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.963+0000 7f5f7a691640 10 monclient: monmap: 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:04.965024+0000 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:04.965024+0000 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:04.964 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 1 Processor -- start 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 1 -- start start 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient: init 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5f7406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5f7409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:04.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7406d380 0x7f5f7413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient(hunting): picked mon.noname-b con 0x7f5f7406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f7406daf0 con 0x7f5f7409c4a0 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f7409cc80 con 0x7f5f7406c890 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5f740a0880 con 0x7f5f7406d380 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f7a691640 10 monclient(hunting): authenticate will time out at 1330.518563s 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f737fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f5f7409c4a0 0x7f5f7413b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:36510/0 (socket says 10.20.193.78:36510) 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7406d380 0x7f5f7413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f737fe640 1 -- 10.20.193.78:0/911661580 learned_addr learned my addr 10.20.193.78:0/911661580 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient(hunting): get_auth_request con 0x7f5f7406d380 auth_method 0 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient(hunting): handle_auth_done global_id 144221 payload 274 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 1 -- 10.20.193.78:0/911661580 >> v1:10.20.193.115:6789/0 conn(0x7f5f7406c890 legacy=0x7f5f7406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 1 -- 10.20.193.78:0/911661580 >> v1:10.20.193.78:6789/0 conn(0x7f5f7409c4a0 legacy=0x7f5f7413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient: _finish_hunting 0 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 1 monclient: found mon.noname-b 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.964+0000 7f5f78c07640 1 -- 10.20.193.78:0/911661580 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7409cf60 con 0x7f5f7406d380 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f78c07640 1 -- 10.20.193.78:0/911661580 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7406d380 msgr2=0x7f5f7413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f78c07640 1 --2- 10.20.193.78:0/911661580 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7406d380 0x7f5f7413ead0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009dc0 tx=0x7f5f6802f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478d994ea462cc2f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 1 -- 10.20.193.78:0/911661580 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6803c070 con 0x7f5f7406d380 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: dump: 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: _finish_auth 0 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 1 -- 10.20.193.78:0/911661580 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f68037640 con 0x7f5f7406d380 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 5 monclient: authenticate success, global_id 144221 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 1 -- 10.20.193.78:0/911661580 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68037a10 con 0x7f5f7406d380 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 4 set_mon_vals no callback set 2026-02-17T05:58:04.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f72ffd640 10 monclient: dump: 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.969 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 10 monclient: shutdown 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 1 -- 10.20.193.78:0/911661580 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7406d380 msgr2=0x7f5f7413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 1 --2- 10.20.193.78:0/911661580 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7406d380 0x7f5f7413ead0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009dc0 tx=0x7f5f6802f1c0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 1 -- 10.20.193.78:0/911661580 shutdown_connections 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 1 --2- 10.20.193.78:0/911661580 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7406d380 0x7f5f7413ead0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 1 -- 10.20.193.78:0/911661580 >> 10.20.193.78:0/911661580 conn(0x7f5f740a6620 msgr2=0x7f5f740a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 1 -- 10.20.193.78:0/911661580 shutdown_connections 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.967+0000 7f5f7a691640 1 -- 10.20.193.78:0/911661580 wait complete. 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient: build_initial_monmap 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient: monmap: 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 1 Processor -- start 2026-02-17T05:58:04.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 1 -- start start 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient: init 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7406c890 0x7f5f741364d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient(hunting): picked mon.noname-b con 0x7f5f7406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f7406d380 0x7f5f74136a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient(hunting): picked mon.noname-a con 0x7f5f7406d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7409c4a0 0x7f5f7411df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient(hunting): picked mon.noname-c con 0x7f5f7409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7409bfa0 con 0x7f5f7406d380 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7409be20 con 0x7f5f7406c890 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7413f010 con 0x7f5f7409c4a0 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f7a691640 10 monclient(hunting): authenticate will time out at 1330.522704s 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f7406d380 0x7f5f74136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7409c4a0 0x7f5f7411df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7409c4a0 0x7f5f7411df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57684/0 (socket says 10.20.193.78:57684) 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 -- 10.20.193.78:0/2958206867 learned_addr learned my addr 10.20.193.78:0/2958206867 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f7406d380 0x7f5f74136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:59344/0 (socket says 10.20.193.78:59344) 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient(hunting): get_auth_request con 0x7f5f7409c4a0 auth_method 0 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 10 monclient(hunting): get_auth_request con 0x7f5f7406d380 auth_method 0 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f73fff640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7406c890 0x7f5f741364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f73fff640 10 monclient(hunting): get_auth_request con 0x7f5f7406c890 auth_method 0 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f73fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient(hunting): handle_auth_done global_id 144224 payload 824 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7406c890 msgr2=0x7f5f741364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7406c890 0x7f5f741364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f7406d380 msgr2=0x7f5f74136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f7406d380 0x7f5f74136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient: _finish_hunting 0 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 monclient: found mon.noname-c 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f78c07640 1 -- 10.20.193.78:0/2958206867 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f74121bb0 con 0x7f5f7409c4a0 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.968+0000 7f5f737fe640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f7406d380 0x7f5f74136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f78c07640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7409c4a0 msgr2=0x7f5f7411df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f78c07640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7409c4a0 0x7f5f7411df00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f6803a040 tx=0x7f5f68004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68047020 con 0x7f5f7409c4a0 2026-02-17T05:58:04.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: dump: 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: _finish_auth 0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f68042880 con 0x7f5f7409c4a0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6803c040 con 0x7f5f7409c4a0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f727fc640 4 set_mon_vals no callback set 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 10 monclient: dump: 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 5 monclient: authenticate success, global_id 144224 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 10 monclient: _renew_subs 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f74120480 con 0x7f5f7409c4a0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 10 monclient: _renew_subs 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f74072960 con 0x7f5f7409c4a0 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f741cf3d0 0x7f5f7411e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7411fc00 con 0x7f5f741cf3d0 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f741cf3d0 0x7f5f7411e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 10 monclient: get_auth_request con 0x7f5f741cf3d0 auth_method 0 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5f68043420 con 0x7f5f7409c4a0 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 10 monclient(hunting): handle_auth_done global_id 134278 payload 824 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f73fff640 1 -- 10.20.193.78:0/2958206867 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f741d00a0 con 0x7f5f741cf3d0 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f4403deb0 0x7f5f44040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(430..430 src has 335..430) ==== 4843+0+0 (secure 0 0 0) 0x7f5f680786e0 con 0x7f5f7409c4a0 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f737fe640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f4403deb0 0x7f5f44040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.972+0000 7f5f737fe640 10 monclient: get_auth_request con 0x7f5f4403deb0 auth_method 0 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f73fff640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f741cf3d0 msgr2=0x7f5f7411e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f73fff640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f741cf3d0 0x7f5f7411e440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e730 tx=0x7f5f6000ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60006ae0 con 0x7f5f741cf3d0 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 10 monclient: dump: 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f5f60006ae0 con 0x7f5f741cf3d0 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f741cf3d0 msgr2=0x7f5f7411e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f741cf3d0 0x7f5f7411e440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e730 tx=0x7f5f6000ec00 comp rx=0 tx=0).stop 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f741cf3d0 msgr2=0x7f5f7411e440 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f70ff9640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f741cf3d0 0x7f5f7411e440 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e730 tx=0x7f5f6000ec00 comp rx=0 tx=0).stop 2026-02-17T05:58:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.974+0000 7f5f737fe640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f4403deb0 0x7f5f44040370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5f64007d90 tx=0x7f5f64007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f7a691640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f7a691640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7411e980 0x7f5f741da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f7a691640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7411ed90 con 0x7f5f7411e980 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f73fff640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7411e980 0x7f5f741da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f73fff640 10 monclient: get_auth_request con 0x7f5f7411e980 auth_method 0 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f73fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.977+0000 7f5f73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.978+0000 7f5f73fff640 10 monclient(hunting): handle_auth_done global_id 134281 payload 824 2026-02-17T05:58:04.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.978+0000 7f5f73fff640 1 -- 10.20.193.78:0/2958206867 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5f7406fdd0 con 0x7f5f7411e980 2026-02-17T05:58:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f73fff640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7411e980 msgr2=0x7f5f741da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f73fff640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7411e980 0x7f5f741da4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e730 tx=0x7f5f60003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000c930 con 0x7f5f7411e980 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 10 monclient: dump: 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f5f6000c930 con 0x7f5f7411e980 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7411e980 msgr2=0x7f5f741da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7411e980 0x7f5f741da4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e730 tx=0x7f5f60003860 comp rx=0 tx=0).stop 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7411e980 msgr2=0x7f5f741da4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f70ff9640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f7411e980 0x7f5f741da4d0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e730 tx=0x7f5f60003860 comp rx=0 tx=0).stop 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:58:04.981 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 136, 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 112699, 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:58:04.982 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.983 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.984 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:58:04.985 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f4403deb0 msgr2=0x7f5f44040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5f4403deb0 0x7f5f44040370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5f64007d90 tx=0x7f5f64007ce0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 10 monclient: shutdown 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7409c4a0 msgr2=0x7f5f7411df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7409c4a0 0x7f5f7411df00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f6803a040 tx=0x7f5f68004b40 comp rx=0 tx=0).stop 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f78c07640 1 -- 10.20.193.78:0/2958206867 reap_dead start 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 shutdown_connections 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 --2- 10.20.193.78:0/2958206867 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f7409c4a0 0x7f5f7411df00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 >> 10.20.193.78:0/2958206867 conn(0x7f5f740a6620 msgr2=0x7f5f740a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 shutdown_connections 2026-02-17T05:58:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:04.980+0000 7f5f7a691640 1 -- 10.20.193.78:0/2958206867 wait complete. 2026-02-17T05:58:04.986 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:58:04.987 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:58:04.987 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:58:04.987 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:58:04.987 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:58:04.987 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:58:04.987 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:58:04.995 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:58:04.995 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:58:04.995 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient: get_monmap_and_config 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient: build_initial_monmap 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient: monmap: 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:05.102518+0000 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:05.102518+0000 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:05.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 1 Processor -- start 2026-02-17T05:58:05.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 1 -- start start 2026-02-17T05:58:05.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient: init 2026-02-17T05:58:05.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:05.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2026-02-17T05:58:05.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f583806c890 0x7f583806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:05.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient(hunting): picked mon.noname-c con 0x7f583806c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:05.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f583809c4a0 0x7f583806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:05.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient(hunting): picked mon.noname-a con 0x7f583809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f5838135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient(hunting): picked mon.noname-b con 0x7f5838134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f58380a0880 con 0x7f583809c4a0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f583806d6f0 con 0x7f583806c890 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f583806daf0 con 0x7f5838134df0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.101+0000 7f5840149640 10 monclient(hunting): authenticate will time out at 1330.655997s 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583d6bd640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f583809c4a0 msgr2=0x7f583806d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583d6bd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f583809c4a0 0x7f583806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f5838135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f5838135280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57688/0 (socket says 10.20.193.78:57688) 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 -- 10.20.193.78:0/2330104582 learned_addr learned my addr 10.20.193.78:0/2330104582 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient(hunting): get_auth_request con 0x7f5838134df0 auth_method 0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583debe640 1 --2- 10.20.193.78:0/2330104582 >> v2:10.20.193.115:3300/0 conn(0x7f583806c890 0x7f583806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583debe640 10 monclient(hunting): get_auth_request con 0x7f583806c890 auth_method 0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583debe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583debe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583debe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient(hunting): handle_auth_done global_id 144227 payload 274 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 -- 10.20.193.78:0/2330104582 >> v2:10.20.193.115:3300/0 conn(0x7f583806c890 msgr2=0x7f583806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 --2- 10.20.193.78:0/2330104582 >> v2:10.20.193.115:3300/0 conn(0x7f583806c890 0x7f583806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 -- 10.20.193.78:0/2330104582 >> v2:10.20.193.78:3300/0 conn(0x7f583809c4a0 msgr2=0x7f583806d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 --2- 10.20.193.78:0/2330104582 >> v2:10.20.193.78:3300/0 conn(0x7f583809c4a0 0x7f583806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient: _finish_hunting 0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 monclient: found mon.noname-b 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:05.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.102+0000 7f583e6bf640 1 -- 10.20.193.78:0/2330104582 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583809cc80 con 0x7f5838134df0 2026-02-17T05:58:05.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583e6bf640 1 -- 10.20.193.78:0/2330104582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 msgr2=0x7f5838135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:05.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583e6bf640 1 --2- 10.20.193.78:0/2330104582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f5838135280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f583806a480 tx=0x7f583402f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1d83a4e4e7d8fd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 1 -- 10.20.193.78:0/2330104582 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58340373f0 con 0x7f5838134df0 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: dump: 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: _finish_auth 0 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 1 -- 10.20.193.78:0/2330104582 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5834037590 con 0x7f5838134df0 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f5840149640 5 monclient: authenticate success, global_id 144227 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f5840149640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 1 -- 10.20.193.78:0/2330104582 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834037960 con 0x7f5838134df0 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f583cebc640 10 monclient: dump: 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:05.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f5840149640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.104+0000 7f5840149640 4 set_mon_vals no callback set 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient: shutdown 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- 10.20.193.78:0/2330104582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 msgr2=0x7f5838135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 --2- 10.20.193.78:0/2330104582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f5838135280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f583806a480 tx=0x7f583402f200 comp rx=0 tx=0).stop 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- 10.20.193.78:0/2330104582 shutdown_connections 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 --2- 10.20.193.78:0/2330104582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f5838135280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 --2- 10.20.193.78:0/2330104582 >> v2:10.20.193.78:3300/0 conn(0x7f583809c4a0 0x7f583806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 --2- 10.20.193.78:0/2330104582 >> v2:10.20.193.115:3300/0 conn(0x7f583806c890 0x7f583806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- 10.20.193.78:0/2330104582 >> 10.20.193.78:0/2330104582 conn(0x7f58380a6620 msgr2=0x7f58380a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- 10.20.193.78:0/2330104582 shutdown_connections 2026-02-17T05:58:05.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- 10.20.193.78:0/2330104582 wait complete. 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient: build_initial_monmap 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient: monmap: 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 Processor -- start 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- start start 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient: init 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f583806c890 0x7f58381cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient(hunting): picked mon.noname-a con 0x7f583806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f583809c4a0 0x7f58381cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient(hunting): picked mon.noname-b con 0x7f583809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f58381215a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient(hunting): picked mon.noname-c con 0x7f5838134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58380a01a0 con 0x7f583806c890 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f58380a0020 con 0x7f583809c4a0 2026-02-17T05:58:05.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f583809be20 con 0x7f5838134df0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f5840149640 10 monclient(hunting): authenticate will time out at 1330.659979s 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.105+0000 7f583debe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f583806c890 msgr2=0x7f58381cd3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583debe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f583806c890 0x7f58381cd3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f58381215a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f58381215a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57704/0 (socket says 10.20.193.78:57704) 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 -- 10.20.193.78:0/1234450915 learned_addr learned my addr 10.20.193.78:0/1234450915 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient(hunting): get_auth_request con 0x7f5838134df0 auth_method 0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient(hunting): handle_auth_done global_id 144230 payload 824 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 -- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f583809c4a0 msgr2=0x7f58381cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583d6bd640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f583809c4a0 0x7f58381cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f583809c4a0 0x7f58381cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 -- 10.20.193.78:0/1234450915 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f583806c890 msgr2=0x7f58381cd3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f583806c890 0x7f58381cd3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient: _finish_hunting 0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 monclient: found mon.noname-c 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583e6bf640 1 -- 10.20.193.78:0/1234450915 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5838125ec0 con 0x7f5838134df0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583d6bd640 10 monclient: get_auth_request con 0x7f583809c4a0 auth_method 0 2026-02-17T05:58:05.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.106+0000 7f583d6bd640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f583809c4a0 0x7f58381cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f583e6bf640 1 -- 10.20.193.78:0/1234450915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 msgr2=0x7f58381215a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f583e6bf640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f58381215a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5834009f50 tx=0x7f5834004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 1 -- 10.20.193.78:0/1234450915 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834046020 con 0x7f5838134df0 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: dump: 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 1 -- 10.20.193.78:0/1234450915 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5834041870 con 0x7f5838134df0 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f5840149640 5 monclient: authenticate success, global_id 144230 2026-02-17T05:58:05.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f5840149640 10 monclient: _renew_subs 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f5840149640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5838125200 con 0x7f5838134df0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 1 -- 10.20.193.78:0/1234450915 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834041c80 con 0x7f5838134df0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f58267fc640 10 monclient: dump: 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f5827fff640 4 set_mon_vals no callback set 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f5840149640 10 monclient: _renew_subs 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f5840149640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.108+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f58380724b0 con 0x7f5838134df0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f5840149640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f5840149640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f5840149640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5838074980 con 0x7f5838134df0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f58267fc640 1 -- 10.20.193.78:0/1234450915 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f583404b410 con 0x7f5838134df0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f58267fc640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f580803de90 0x7f5808040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f58267fc640 1 -- 10.20.193.78:0/1234450915 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(430..430 src has 335..430) ==== 4843+0+0 (secure 0 0 0) 0x7f5834078540 con 0x7f5838134df0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f583debe640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f580803de90 0x7f5808040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:05.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.109+0000 7f583debe640 10 monclient: get_auth_request con 0x7f580803de90 auth_method 0 2026-02-17T05:58:05.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.111+0000 7f58267fc640 1 -- 10.20.193.78:0/1234450915 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5834050070 con 0x7f5838134df0 2026-02-17T05:58:05.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.111+0000 7f58267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:05.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.111+0000 7f58267fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:05.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.112+0000 7f583debe640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f580803de90 0x7f5808040350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5828009180 tx=0x7f5828008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:05.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.230+0000 7f5840149640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:05.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.230+0000 7f5840149640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:05.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.230+0000 7f5840149640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:05.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.230+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f58380a14d0 con 0x7f5838134df0 2026-02-17T05:58:05.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.230+0000 7f58267fc640 1 -- 10.20.193.78:0/1234450915 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f5838125ec0 con 0x7f5838134df0 2026-02-17T05:58:05.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.230+0000 7f58267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:05.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.230+0000 7f58267fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:05.232 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:05.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f580803de90 msgr2=0x7f5808040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:05.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f580803de90 0x7f5808040350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5828009180 tx=0x7f5828008040 comp rx=0 tx=0).stop 2026-02-17T05:58:05.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 10 monclient: shutdown 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 msgr2=0x7f58381215a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f58381215a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5834009f50 tx=0x7f5834004b40 comp rx=0 tx=0).stop 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 shutdown_connections 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5838134df0 0x7f58381215a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f580803de90 0x7f5808040350 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f583809c4a0 0x7f58381cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 --2- 10.20.193.78:0/1234450915 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f583806c890 0x7f58381cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 >> 10.20.193.78:0/1234450915 conn(0x7f58380a6620 msgr2=0x7f58380a2860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 shutdown_connections 2026-02-17T05:58:05.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:05.231+0000 7f5840149640 1 -- 10.20.193.78:0/1234450915 wait complete. 2026-02-17T05:58:08.241 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:08.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.309+0000 7fbf7f73b640 10 monclient: get_monmap_and_config 2026-02-17T05:58:08.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.309+0000 7fbf7f73b640 10 monclient: build_initial_monmap 2026-02-17T05:58:08.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.309+0000 7fbf7f73b640 10 monclient: monmap: 2026-02-17T05:58:08.313 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:08.311107+0000 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:08.311107+0000 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:08.314 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 1 Processor -- start 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 1 -- start start 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient: init 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient: _add_conns ranks=[2,3,0,4,1] 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3140 0x7fbf7806eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient(hunting): picked mon.noname-b con 0x7fbf780a3140 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fbf780a3520 0x7fbf780a3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:08.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient(hunting): picked mon.noname-a con 0x7fbf780a3520 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbf7813c410 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf780a48a0 con 0x7fbf7813c410 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7806ffc0 con 0x7fbf780a3520 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7806f970 con 0x7fbf780a3140 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7f73b640 10 monclient(hunting): authenticate will time out at 1333.864693s 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7ccaf640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fbf780a3520 msgr2=0x7fbf780a3900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7dcb1640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7fbf7813c410 legacy=0x7fbf7813e800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:58:08.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7ccaf640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fbf780a3520 0x7fbf780a3900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3140 0x7fbf7806eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3140 0x7fbf7806eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45978/0 (socket says 10.20.193.78:45978) 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 1 -- 10.20.193.78:0/3110212661 learned_addr learned my addr 10.20.193.78:0/3110212661 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient(hunting): get_auth_request con 0x7fbf780a3140 auth_method 0 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:08.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient(hunting): handle_auth_done global_id 144233 payload 274 2026-02-17T05:58:08.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 1 -- 10.20.193.78:0/3110212661 >> v2:10.20.193.78:3300/0 conn(0x7fbf780a3520 msgr2=0x7fbf780a3900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:08.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 1 --2- 10.20.193.78:0/3110212661 >> v2:10.20.193.78:3300/0 conn(0x7fbf780a3520 0x7fbf780a3900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 1 -- 10.20.193.78:0/3110212661 >> v1:10.20.193.78:6789/0 conn(0x7fbf7813c410 legacy=0x7fbf7813e800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:08.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient: _finish_hunting 0 2026-02-17T05:58:08.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 1 monclient: found mon.noname-b 2026-02-17T05:58:08.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:08.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:08.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.310+0000 7fbf7d4b0640 1 -- 10.20.193.78:0/3110212661 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7806fc50 con 0x7fbf780a3140 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf7d4b0640 1 -- 10.20.193.78:0/3110212661 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3140 msgr2=0x7fbf7806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf7d4b0640 1 --2- 10.20.193.78:0/3110212661 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3140 0x7fbf7806eb10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf78070140 tx=0x7fbf6802eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490da9071b7659b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 1 -- 10.20.193.78:0/3110212661 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6803c070 con 0x7fbf780a3140 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: dump: 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:08.327 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: _finish_auth 0 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:08.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf7f73b640 5 monclient: authenticate success, global_id 144233 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 1 -- 10.20.193.78:0/3110212661 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6802fdd0 con 0x7fbf780a3140 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 1 -- 10.20.193.78:0/3110212661 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68037660 con 0x7fbf780a3140 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf67fff640 10 monclient: dump: 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:08.329 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf7f73b640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.313+0000 7fbf7f73b640 4 set_mon_vals no callback set 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 10 monclient: shutdown 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:08.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 -- 10.20.193.78:0/3110212661 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3140 msgr2=0x7fbf7806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/3110212661 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3140 0x7fbf7806eb10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf78070140 tx=0x7fbf6802eeb0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 -- 10.20.193.78:0/3110212661 shutdown_connections 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/3110212661 >> v2:10.20.193.78:3300/0 conn(0x7fbf780a3520 0x7fbf780a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/3110212661 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3140 0x7fbf7806eb10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 -- 10.20.193.78:0/3110212661 >> 10.20.193.78:0/3110212661 conn(0x7fbf7809c000 msgr2=0x7fbf780a8740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 -- 10.20.193.78:0/3110212661 shutdown_connections 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 -- 10.20.193.78:0/3110212661 wait complete. 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 10 monclient: build_initial_monmap 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 10 monclient: monmap: 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:08.331 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 Processor -- start 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 1 -- start start 2026-02-17T05:58:08.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.314+0000 7fbf7f73b640 10 monclient: init 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbf780a3140 0x7fbf7813e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient(hunting): picked mon.noname-b con 0x7fbf780a3140 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3520 0x7fbf78138720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient(hunting): picked mon.noname-c con 0x7fbf780a3520 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbf7813c410 0x7fbf781cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient(hunting): picked mon.noname-a con 0x7fbf7813c410 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:08.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf780a9300 con 0x7fbf7813c410 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf780a5070 con 0x7fbf780a3140 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbf780a9480 con 0x7fbf780a3520 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7f73b640 10 monclient(hunting): authenticate will time out at 1333.869148s 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7dcb1640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbf7813c410 msgr2=0x7fbf781cf0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7dcb1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbf7813c410 0x7fbf781cf0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3520 0x7fbf78138720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:08.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3520 0x7fbf78138720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45992/0 (socket says 10.20.193.78:45992) 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 -- 10.20.193.78:0/2080306424 learned_addr learned my addr 10.20.193.78:0/2080306424 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient(hunting): get_auth_request con 0x7fbf780a3520 auth_method 0 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7d4b0640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbf780a3140 0x7fbf7813e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7d4b0640 10 monclient(hunting): get_auth_request con 0x7fbf780a3140 auth_method 0 2026-02-17T05:58:08.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7d4b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7d4b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7d4b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient(hunting): handle_auth_done global_id 144236 payload 824 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 -- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbf780a3140 msgr2=0x7fbf7813e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbf780a3140 0x7fbf7813e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 -- 10.20.193.78:0/2080306424 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbf7813c410 msgr2=0x7fbf781cf0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbf7813c410 0x7fbf781cf0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient: _finish_hunting 0 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 monclient: found mon.noname-c 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:08.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.315+0000 7fbf7ccaf640 1 -- 10.20.193.78:0/2080306424 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7813f530 con 0x7fbf780a3520 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7ccaf640 1 -- 10.20.193.78:0/2080306424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3520 msgr2=0x7fbf78138720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7ccaf640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3520 0x7fbf78138720 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b780 tx=0x7fbf6c00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 1 -- 10.20.193.78:0/2080306424 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c005270 con 0x7fbf780a3520 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: dump: 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:08.337 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: _finish_auth 0 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7f73b640 5 monclient: authenticate success, global_id 144236 2026-02-17T05:58:08.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 1 -- 10.20.193.78:0/2080306424 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6c010070 con 0x7fbf780a3520 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 1 -- 10.20.193.78:0/2080306424 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c005650 con 0x7fbf780a3520 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf65ffb640 10 monclient: dump: 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:08.339 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf677fe640 4 set_mon_vals no callback set 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7f73b640 10 monclient: _renew_subs 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7f73b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf780a6aa0 con 0x7fbf780a3520 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7f73b640 10 monclient: _renew_subs 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7f73b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:08.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.317+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7809fe00 con 0x7fbf780a3520 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf7f73b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf7f73b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf7f73b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7806eb10 con 0x7fbf780a3520 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf65ffb640 1 -- 10.20.193.78:0/2080306424 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fbf6c00c520 con 0x7fbf780a3520 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf65ffb640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbf4803de60 0x7fbf48040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf65ffb640 1 -- 10.20.193.78:0/2080306424 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(430..430 src has 335..430) ==== 4843+0+0 (secure 0 0 0) 0x7fbf6c052070 con 0x7fbf780a3520 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf7d4b0640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbf4803de60 0x7fbf48040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.318+0000 7fbf7d4b0640 10 monclient: get_auth_request con 0x7fbf4803de60 auth_method 0 2026-02-17T05:58:08.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.321+0000 7fbf65ffb640 1 -- 10.20.193.78:0/2080306424 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbf6c00ce10 con 0x7fbf780a3520 2026-02-17T05:58:08.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.321+0000 7fbf65ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:08.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.321+0000 7fbf65ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:08.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.321+0000 7fbf7d4b0640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbf4803de60 0x7fbf48040320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf68036000 tx=0x7fbf68033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.436+0000 7fbf7f73b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.436+0000 7fbf7f73b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.436+0000 7fbf7f73b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.436+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbf780744d0 con 0x7fbf780a3520 2026-02-17T05:58:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.437+0000 7fbf65ffb640 1 -- 10.20.193.78:0/2080306424 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fbf6c0057f0 con 0x7fbf780a3520 2026-02-17T05:58:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.437+0000 7fbf65ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.437+0000 7fbf65ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:08.439 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbf4803de60 msgr2=0x7fbf48040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbf4803de60 0x7fbf48040320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf68036000 tx=0x7fbf68033000 comp rx=0 tx=0).stop 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 10 monclient: shutdown 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3520 msgr2=0x7fbf78138720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3520 0x7fbf78138720 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b780 tx=0x7fbf6c00bc50 comp rx=0 tx=0).stop 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 shutdown_connections 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbf7813c410 0x7fbf781cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbf780a3520 0x7fbf78138720 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbf4803de60 0x7fbf48040320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 --2- 10.20.193.78:0/2080306424 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbf780a3140 0x7fbf7813e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:08.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 >> 10.20.193.78:0/2080306424 conn(0x7fbf7809c000 msgr2=0x7fbf780a7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:08.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 shutdown_connections 2026-02-17T05:58:08.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:08.438+0000 7fbf7f73b640 1 -- 10.20.193.78:0/2080306424 wait complete. 2026-02-17T05:58:11.448 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:11.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient: get_monmap_and_config 2026-02-17T05:58:11.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient: build_initial_monmap 2026-02-17T05:58:11.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient: monmap: 2026-02-17T05:58:11.517 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:11.517 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:11.517 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:11.517394+0000 2026-02-17T05:58:11.517 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:11.517394+0000 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 1 Processor -- start 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 1 -- start start 2026-02-17T05:58:11.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient: init 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fec5406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient(hunting): picked mon.noname-b con 0x7fec5409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fec5406d380 0x7fec5413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient(hunting): picked mon.noname-c con 0x7fec5406d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec5406daf0 con 0x7fec5406c890 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fec540a0880 con 0x7fec5406d380 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fec5409cc80 con 0x7fec5409c4a0 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec5b8de640 10 monclient(hunting): authenticate will time out at 1337.070900s 2026-02-17T05:58:11.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.516+0000 7fec58e52640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45996/0 (socket says 10.20.193.78:45996) 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 1 -- 10.20.193.78:0/1942688422 learned_addr learned my addr 10.20.193.78:0/1942688422 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient(hunting): get_auth_request con 0x7fec5409c4a0 auth_method 0 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec59e54640 1 --2- 10.20.193.78:0/1942688422 >> v2:10.20.193.115:3300/0 conn(0x7fec5406d380 0x7fec5413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec59e54640 10 monclient(hunting): get_auth_request con 0x7fec5406d380 auth_method 0 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec59e54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec59e54640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec59e54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient(hunting): handle_auth_done global_id 144239 payload 274 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 1 -- 10.20.193.78:0/1942688422 >> v2:10.20.193.115:3300/0 conn(0x7fec5406d380 msgr2=0x7fec5413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 1 --2- 10.20.193.78:0/1942688422 >> v2:10.20.193.115:3300/0 conn(0x7fec5406d380 0x7fec5413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 1 -- 10.20.193.78:0/1942688422 >> v1:10.20.193.115:6789/0 conn(0x7fec5406c890 legacy=0x7fec5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:11.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient: _finish_hunting 0 2026-02-17T05:58:11.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 1 monclient: found mon.noname-b 2026-02-17T05:58:11.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:11.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:11.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.517+0000 7fec58e52640 1 -- 10.20.193.78:0/1942688422 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5409cf60 con 0x7fec5409c4a0 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec58e52640 1 -- 10.20.193.78:0/1942688422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 msgr2=0x7fec5413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec58e52640 1 --2- 10.20.193.78:0/1942688422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413b3a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec48009970 tx=0x7fec4802ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e4bf79ce1b1d53 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 1 -- 10.20.193.78:0/1942688422 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4803c070 con 0x7fec5409c4a0 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: dump: 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:11.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: _finish_auth 0 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 1 -- 10.20.193.78:0/1942688422 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4802fcd0 con 0x7fec5409c4a0 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 1 -- 10.20.193.78:0/1942688422 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48037660 con 0x7fec5409c4a0 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec5b8de640 5 monclient: authenticate success, global_id 144239 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec5b8de640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec5b8de640 4 set_mon_vals no callback set 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.519+0000 7fec43fff640 10 monclient: dump: 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:11.527 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 10 monclient: shutdown 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 -- 10.20.193.78:0/1942688422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 msgr2=0x7fec5413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 --2- 10.20.193.78:0/1942688422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413b3a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec48009970 tx=0x7fec4802ee90 comp rx=0 tx=0).stop 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 -- 10.20.193.78:0/1942688422 shutdown_connections 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 --2- 10.20.193.78:0/1942688422 >> v2:10.20.193.115:3300/0 conn(0x7fec5406d380 0x7fec5413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 --2- 10.20.193.78:0/1942688422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413b3a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 -- 10.20.193.78:0/1942688422 >> 10.20.193.78:0/1942688422 conn(0x7fec540a6620 msgr2=0x7fec540a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 -- 10.20.193.78:0/1942688422 shutdown_connections 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 -- 10.20.193.78:0/1942688422 wait complete. 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 10 monclient: build_initial_monmap 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 10 monclient: monmap: 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:11.528 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 Processor -- start 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 1 -- start start 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.520+0000 7fec5b8de640 10 monclient: init 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec5406c890 0x7fec541cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient(hunting): picked mon.noname-a con 0x7fec5406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec5406d380 0x7fec541331c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient(hunting): picked mon.noname-b con 0x7fec5406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient(hunting): picked mon.noname-c con 0x7fec5409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec5409bfa0 con 0x7fec5406c890 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec5409be20 con 0x7fec5406d380 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fec5413de30 con 0x7fec5409c4a0 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec5b8de640 10 monclient(hunting): authenticate will time out at 1337.075145s 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59653640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec5406c890 msgr2=0x7fec541cd000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59653640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec5406c890 0x7fec541cd000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:11.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46000/0 (socket says 10.20.193.78:46000) 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 -- 10.20.193.78:0/307615606 learned_addr learned my addr 10.20.193.78:0/307615606 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient(hunting): get_auth_request con 0x7fec5409c4a0 auth_method 0 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient(hunting): handle_auth_done global_id 144242 payload 824 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 -- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec5406d380 msgr2=0x7fec541331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec5406d380 0x7fec541331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 -- 10.20.193.78:0/307615606 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec5406c890 msgr2=0x7fec541cd000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec5406c890 0x7fec541cd000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient: _finish_hunting 0 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 monclient: found mon.noname-c 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.521+0000 7fec59e54640 1 -- 10.20.193.78:0/307615606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5411db50 con 0x7fec5409c4a0 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec59e54640 1 -- 10.20.193.78:0/307615606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 msgr2=0x7fec5413a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec59e54640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413a480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec50007d90 tx=0x7fec5000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 1 -- 10.20.193.78:0/307615606 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50013790 con 0x7fec5409c4a0 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:11.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: dump: 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: _finish_auth 0 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 1 -- 10.20.193.78:0/307615606 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec50019070 con 0x7fec5409c4a0 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec5b8de640 5 monclient: authenticate success, global_id 144242 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec5b8de640 10 monclient: _renew_subs 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec5b8de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5413d9d0 con 0x7fec5409c4a0 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 1 -- 10.20.193.78:0/307615606 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50013b90 con 0x7fec5409c4a0 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec41ffb640 10 monclient: dump: 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:11.531 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec437fe640 4 set_mon_vals no callback set 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec5b8de640 10 monclient: _renew_subs 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec5b8de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.523+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5411d590 con 0x7fec5409c4a0 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.524+0000 7fec5b8de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.524+0000 7fec5b8de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.524+0000 7fec5b8de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.524+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5413d7b0 con 0x7fec5409c4a0 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.524+0000 7fec41ffb640 1 -- 10.20.193.78:0/307615606 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fec5001e460 con 0x7fec5409c4a0 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.524+0000 7fec41ffb640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec2403de60 0x7fec24040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.524+0000 7fec41ffb640 1 -- 10.20.193.78:0/307615606 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(430..430 src has 335..430) ==== 4843+0+0 (secure 0 0 0) 0x7fec50053190 con 0x7fec5409c4a0 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.524+0000 7fec59653640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec2403de60 0x7fec24040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:11.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.525+0000 7fec59653640 10 monclient: get_auth_request con 0x7fec2403de60 auth_method 0 2026-02-17T05:58:11.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.527+0000 7fec41ffb640 1 -- 10.20.193.78:0/307615606 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fec5413d7b0 con 0x7fec5409c4a0 2026-02-17T05:58:11.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.527+0000 7fec41ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:11.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.527+0000 7fec41ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:11.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.527+0000 7fec59653640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec2403de60 0x7fec24040320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fec44003870 tx=0x7fec44007b10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:11.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.646+0000 7fec5b8de640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:11.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.646+0000 7fec5b8de640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:11.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.646+0000 7fec5b8de640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:11.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.646+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fec54074980 con 0x7fec5409c4a0 2026-02-17T05:58:11.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.646+0000 7fec41ffb640 1 -- 10.20.193.78:0/307615606 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fec50013d30 con 0x7fec5409c4a0 2026-02-17T05:58:11.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.646+0000 7fec41ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:11.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.646+0000 7fec41ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:11.649 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:11.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec2403de60 msgr2=0x7fec24040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:11.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec2403de60 0x7fec24040320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fec44003870 tx=0x7fec44007b10 comp rx=0 tx=0).stop 2026-02-17T05:58:11.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 10 monclient: shutdown 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 msgr2=0x7fec5413a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413a480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec50007d90 tx=0x7fec5000e4b0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 shutdown_connections 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec5409c4a0 0x7fec5413a480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fec2403de60 0x7fec24040320 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec5406d380 0x7fec541331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 --2- 10.20.193.78:0/307615606 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec5406c890 0x7fec541cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:11.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 >> 10.20.193.78:0/307615606 conn(0x7fec540a6620 msgr2=0x7fec540a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:11.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 shutdown_connections 2026-02-17T05:58:11.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:11.647+0000 7fec5b8de640 1 -- 10.20.193.78:0/307615606 wait complete. 2026-02-17T05:58:14.657 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:14.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient: get_monmap_and_config 2026-02-17T05:58:14.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient: build_initial_monmap 2026-02-17T05:58:14.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient: monmap: 2026-02-17T05:58:14.728 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:14.728 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:14.728 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:14.726381+0000 2026-02-17T05:58:14.728 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:14.726381+0000 2026-02-17T05:58:14.728 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:14.728 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 1 Processor -- start 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 1 -- start start 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient: init 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2026-02-17T05:58:14.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b7806c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5b7809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 0x7f5b7813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient(hunting): picked mon.noname-a con 0x7f5b7806d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7806daf0 con 0x7f5b7809c4a0 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7809cc80 con 0x7f5b7806c890 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5b780a0880 con 0x7f5b7806d380 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7f772640 10 monclient(hunting): authenticate will time out at 1340.279898s 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7cce6640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f5b7809c4a0 legacy=0x7f5b7813b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7dce8640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 msgr2=0x7f5b7813ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:14.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.725+0000 7f5b7dce8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 0x7f5b7813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:14.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.726+0000 7f5b7d4e7640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f5b7806c890 0x7f5b7806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:45076/0 (socket says 10.20.193.78:45076) 2026-02-17T05:58:14.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.726+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 learned_addr learned my addr 10.20.193.78:0/4103428701 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:14.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.728+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.115:6789/0 conn(0x7f5b7806c890 legacy=0x7f5b7806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.927+0000 7f5b7dce8640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 msgr2=0x7f5b7813ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.927+0000 7f5b7dce8640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 0x7f5b7813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:58:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:14.927+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.78:6789/0 conn(0x7f5b7809c4a0 legacy=0x7f5b7813b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:58:15.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:15.327+0000 7f5b7dce8640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 msgr2=0x7f5b7813ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:15.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:15.327+0000 7f5b7dce8640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 0x7f5b7813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:58:15.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:15.327+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.78:6789/0 conn(0x7f5b7809c4a0 legacy=0x7f5b7813b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:58:16.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:16.128+0000 7f5b7dce8640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 msgr2=0x7f5b7813ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:16.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:16.128+0000 7f5b7dce8640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 0x7f5b7813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:58:16.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:16.128+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.78:6789/0 conn(0x7f5b7809c4a0 legacy=0x7f5b7813b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:58:17.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient(hunting): tick 2026-02-17T05:58:17.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:58:17.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 monclient(hunting): continuing hunt 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 msgr2=0x7f5b7813ead0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b7806d380 0x7f5b7813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.115:6789/0 conn(0x7f5b7806c890 legacy=0x7f5b7806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.78:6789/0 conn(0x7f5b7809c4a0 legacy=0x7f5b7813b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient: _add_conns ranks=[2,4] 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f5b5c003840 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:17.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f5b5c006350 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c008ea0 con 0x7f5b5c006350 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c009050 con 0x7f5b5c003840 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b66ffd640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request con 0x7f5b5c003840 auth_method 0 2026-02-17T05:58:17.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request con 0x7f5b5c006350 auth_method 0 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:17.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:17.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:17.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:17.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:17.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:17.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.726+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:17.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request con 0x7f5b5c003840 auth_method 0 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request con 0x7f5b5c006350 auth_method 0 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:17.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:17.927+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:58:18.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:18.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request con 0x7f5b5c003840 auth_method 0 2026-02-17T05:58:18.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:18.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request con 0x7f5b5c006350 auth_method 0 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:18.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:18.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:18.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:18.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:18.328+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:58:19.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:19.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request con 0x7f5b5c003840 auth_method 0 2026-02-17T05:58:19.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request con 0x7f5b5c006350 auth_method 0 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:19.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:19.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:19.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:19.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:19.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:19.129+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:58:19.428 INFO:tasks.rados.rados.0.trial115.stdout:2062: done (3 left) 2026-02-17T05:58:19.428 INFO:tasks.rados.rados.0.trial115.stdout:2063: done (2 left) 2026-02-17T05:58:19.428 INFO:tasks.rados.rados.0.trial115.stdout:2065: done (1 left) 2026-02-17T05:58:19.428 INFO:tasks.rados.rados.0.trial115.stdout:2066: done (0 left) 2026-02-17T05:58:19.428 INFO:tasks.rados.rados.0.trial115.stdout:2067: read oid 22 snap -1 2026-02-17T05:58:19.428 INFO:tasks.rados.rados.0.trial115.stdout:2067: expect (ObjNum 675 snap 186 seq_num 675) 2026-02-17T05:58:19.428 INFO:tasks.rados.rados.0.trial115.stdout:2068: copy_from oid 48 from oid 21 current snap is 205 2026-02-17T05:58:19.429 INFO:tasks.rados.rados.0.trial115.stdout:2069: read oid 17 snap 189 2026-02-17T05:58:19.429 INFO:tasks.rados.rados.0.trial115.stdout:2069: expect (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:58:19.429 INFO:tasks.rados.rados.0.trial115.stdout:2070: rollback oid 45 current snap is 205 2026-02-17T05:58:19.429 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 203 2026-02-17T05:58:19.429 INFO:tasks.rados.rados.0.trial115.stdout:2071: write (excl) oid 2 current snap is 205 2026-02-17T05:58:19.429 INFO:tasks.rados.rados.0.trial115.stdout:2071: seq_num 748 ranges {571636=488846,1854531=565263} 2026-02-17T05:58:19.431 INFO:tasks.rados.rados.0.trial115.stdout:2071: writing trial11545420-2 from 571636 to 1060482 tid 1 2026-02-17T05:58:19.437 INFO:tasks.rados.rados.0.trial115.stdout:2071: writing trial11545420-2 from 1854531 to 2419794 tid 2 2026-02-17T05:58:19.437 INFO:tasks.rados.rados.0.trial115.stdout:2072: snap_remove snap 197 2026-02-17T05:58:20.432 INFO:tasks.rados.rados.0.trial115.stdout:2072: done (5 left) 2026-02-17T05:58:20.432 INFO:tasks.rados.rados.0.trial115.stdout:2073: snap_remove snap 202 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient(hunting): tick 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 monclient(hunting): continuing hunt 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 msgr2=0x7f5b5c005c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5c003840 0x7f5b5c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 msgr2=0x7f5b5c008810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.115:3300/0 conn(0x7f5b5c006350 0x7f5b5c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b7dce8640 1 -- 10.20.193.78:0/4103428701 reap_dead start 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b5c003c60 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5b5c007ea0 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:20.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b5c00c570 0x7f5b5c00e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f5b5c00c570 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7809cf60 con 0x7f5b5c007ea0 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7806daf0 con 0x7f5b5c003c60 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 1 -- 10.20.193.78:0/4103428701 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c008ea0 con 0x7f5b5c00c570 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b66ffd640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.78:6789/0 conn(0x7f5b5c007ea0 legacy=0x7f5b5c008300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b7dce8640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b5c00c570 msgr2=0x7f5b5c00e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:20.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.727+0000 7f5b7dce8640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b5c00c570 0x7f5b5c00e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:20.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.728+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.115:6789/0 conn(0x7f5b5c003c60 legacy=0x7f5b5c006780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:20.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1632979210 0 0) 0x7f5b7806daf0 con 0x7f5b5c003c60 2026-02-17T05:58:20.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): _negotiate 2026-02-17T05:58:20.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): my global_id is 134293 2026-02-17T05:58:20.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b60003620 con 0x7f5b5c003c60 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 669437190 0 0) 0x7f5b60003620 con 0x7f5b5c003c60 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b5c00c570 msgr2=0x7f5b5c00e960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 --2- 10.20.193.78:0/4103428701 >> v2:10.20.193.78:3300/0 conn(0x7f5b5c00c570 0x7f5b5c00e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.78:6789/0 conn(0x7f5b5c007ea0 legacy=0x7f5b5c008300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c010f60 con 0x7f5b5c003c60 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: _finish_auth 0 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f5b68003130 con 0x7f5b5c003c60 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7f772640 5 monclient: authenticate success, global_id 134293 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7f772640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: dump: 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.731 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 >> v1:10.20.193.115:6789/0 conn(0x7f5b5c003c60 legacy=0x7f5b5c006780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b60003ab0 0x7f5b60003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): picked mon.a con 0x7f5b60003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b60004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): picked mon.c con 0x7f5b60004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b600053e0 0x7f5b6000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): picked mon.b con 0x7f5b600053e0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c008ea0 con 0x7f5b60003ab0 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b60003ab0 msgr2=0x7f5b60003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000c330 con 0x7f5b600053e0 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b60003ab0 0x7f5b60003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6000c530 con 0x7f5b60004900 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b67fff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7f772640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b60004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request con 0x7f5b60004900 auth_method 0 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7dce8640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b600053e0 0x7f5b6000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7dce8640 10 monclient(hunting): get_auth_request con 0x7f5b600053e0 auth_method 0 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7dce8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7dce8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7dce8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient(hunting): handle_auth_done global_id 134293 payload 293 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b600053e0 msgr2=0x7f5b6000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:20.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b600053e0 0x7f5b6000bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b60003ab0 msgr2=0x7f5b60003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b60003ab0 0x7f5b60003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient: _finish_hunting 0 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 1 monclient: found mon.c 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7dce8640 1 -- 10.20.193.78:0/4103428701 reap_dead start 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b60003620 con 0x7f5b60004900 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient: _finish_auth 0 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 10 monclient: _check_auth_tickets 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.729+0000 7f5b7cce6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b7cce6640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 msgr2=0x7f5b60004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b60004d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c003c10 tx=0x7f5b6c02ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9621c56dadccc157 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c03d070 con 0x7f5b60004900 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 10 monclient: dump: 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.733 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c030b50 con 0x7f5b60004900 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b7f772640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 1 -- 10.20.193.78:0/4103428701 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c009a70 con 0x7f5b60004900 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b67fff640 10 monclient: dump: 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b7f772640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.732+0000 7f5b7f772640 4 set_mon_vals no callback set 2026-02-17T05:58:20.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 10 monclient: shutdown 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 1 -- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 msgr2=0x7f5b60004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b60004d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c003c10 tx=0x7f5b6c02ff00 comp rx=0 tx=0).stop 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 1 -- 10.20.193.78:0/4103428701 shutdown_connections 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 1 --2- 10.20.193.78:0/4103428701 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b60004d80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 1 -- 10.20.193.78:0/4103428701 >> 10.20.193.78:0/4103428701 conn(0x7f5b780a6620 msgr2=0x7f5b780a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 1 -- 10.20.193.78:0/4103428701 shutdown_connections 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 1 -- 10.20.193.78:0/4103428701 wait complete. 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 10 monclient: build_initial_monmap 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.733+0000 7f5b7f772640 10 monclient: monmap: 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 1 Processor -- start 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 1 -- start start 2026-02-17T05:58:20.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient: init 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b781ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient(hunting): picked mon.noname-c con 0x7f5b60004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78136360 0x7f5b781367a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient(hunting): picked mon.noname-b con 0x7f5b78136360 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b7809db50 0x7f5b7809dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient(hunting): picked mon.noname-a con 0x7f5b7809db50 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7809bfa0 con 0x7f5b7809db50 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7809be20 con 0x7f5b78136360 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7806cc70 con 0x7f5b60004900 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:20.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7f772640 10 monclient(hunting): authenticate will time out at 1346.288454s 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7dce8640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b7809db50 msgr2=0x7f5b7809dfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7dce8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b7809db50 0x7f5b7809dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b781ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b781ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37014/0 (socket says 10.20.193.78:37014) 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/3652934926 learned_addr learned my addr 10.20.193.78:0/3652934926 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request con 0x7f5b60004900 auth_method 0 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78136360 0x7f5b781367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request con 0x7f5b78136360 auth_method 0 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7cce6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7cce6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7cce6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient(hunting): handle_auth_done global_id 144245 payload 824 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78136360 msgr2=0x7f5b781367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78136360 0x7f5b781367a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/3652934926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b7809db50 msgr2=0x7f5b7809dfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b7809db50 0x7f5b7809dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient: _finish_hunting 0 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 monclient: found mon.noname-c 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.734+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/3652934926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b78125d40 con 0x7f5b60004900 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b7d4e7640 1 -- 10.20.193.78:0/3652934926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 msgr2=0x7f5b781ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b7d4e7640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b781ceb10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b68005dd0 tx=0x7f5b68007420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 1 -- 10.20.193.78:0/3652934926 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6801a020 con 0x7f5b60004900 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: dump: 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: _finish_auth 0 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 1 -- 10.20.193.78:0/3652934926 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68016870 con 0x7f5b60004900 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b7f772640 5 monclient: authenticate success, global_id 144245 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 1 -- 10.20.193.78:0/3652934926 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68016c40 con 0x7f5b60004900 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b677fe640 4 set_mon_vals no callback set 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.736+0000 7f5b65ffb640 10 monclient: dump: 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.738 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 10 monclient: _renew_subs 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b78136df0 con 0x7f5b60004900 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 10 monclient: _renew_subs 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b78125a60 con 0x7f5b60004900 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b65ffb640 1 -- 10.20.193.78:0/3652934926 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5b68007ce0 con 0x7f5b60004900 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7806cfa0 con 0x7f5b60004900 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b65ffb640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b4803de60 0x7f5b48040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.737+0000 7f5b65ffb640 1 -- 10.20.193.78:0/3652934926 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(432..432 src has 335..432) ==== 4827+0+0 (secure 0 0 0) 0x7f5b680533b0 con 0x7f5b60004900 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.738+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b4803de60 0x7f5b48040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.738+0000 7f5b7cce6640 10 monclient: get_auth_request con 0x7f5b4803de60 auth_method 0 2026-02-17T05:58:20.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.740+0000 7f5b65ffb640 1 -- 10.20.193.78:0/3652934926 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5b68024020 con 0x7f5b60004900 2026-02-17T05:58:20.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.740+0000 7f5b65ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:20.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.740+0000 7f5b65ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:20.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.740+0000 7f5b7cce6640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b4803de60 0x7f5b48040320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b600058e0 tx=0x7f5b6c034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.852+0000 7f5b7f772640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.852+0000 7f5b7f772640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.852+0000 7f5b7f772640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.852+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5b780744d0 con 0x7f5b60004900 2026-02-17T05:58:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.852+0000 7f5b65ffb640 1 -- 10.20.193.78:0/3652934926 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f5b68051d80 con 0x7f5b60004900 2026-02-17T05:58:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.852+0000 7f5b65ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:20.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.852+0000 7f5b65ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:20.855 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":138,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b4803de60 msgr2=0x7f5b48040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b4803de60 0x7f5b48040320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b600058e0 tx=0x7f5b6c034000 comp rx=0 tx=0).stop 2026-02-17T05:58:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 10 monclient: shutdown 2026-02-17T05:58:20.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 msgr2=0x7f5b781ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b781ceb10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b68005dd0 tx=0x7f5b68007420 comp rx=0 tx=0).stop 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 shutdown_connections 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b7809db50 0x7f5b7809dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b4803de60 0x7f5b48040320 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78136360 0x7f5b781367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 --2- 10.20.193.78:0/3652934926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b60004900 0x7f5b781ceb10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 >> 10.20.193.78:0/3652934926 conn(0x7f5b780a6620 msgr2=0x7f5b780a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 shutdown_connections 2026-02-17T05:58:20.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.853+0000 7f5b7f772640 1 -- 10.20.193.78:0/3652934926 wait complete. 2026-02-17T05:58:20.863 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:58:20.863 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:58:20.863 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:58:20.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.930+0000 7f4a3085f640 10 monclient: get_monmap_and_config 2026-02-17T05:58:20.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.930+0000 7f4a3085f640 10 monclient: build_initial_monmap 2026-02-17T05:58:20.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.930+0000 7f4a3085f640 10 monclient: monmap: 2026-02-17T05:58:20.932 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:20.932 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:20.931966+0000 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:20.931966+0000 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 1 Processor -- start 2026-02-17T05:58:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 1 -- start start 2026-02-17T05:58:20.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient: init 2026-02-17T05:58:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2026-02-17T05:58:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c09ddd0 0x7f4a2c09e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient(hunting): picked mon.noname-b con 0x7f4a2c09ddd0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f4a2c0a12d0 0x7f4a2c0a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient(hunting): picked mon.noname-c con 0x7f4a2c0a12d0 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4a2c0a6430 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a2c0a04c0 con 0x7f4a2c0a6430 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f4a2c09f560 con 0x7f4a2c0a12d0 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c09ee60 con 0x7f4a2c09ddd0 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a3085f640 10 monclient(hunting): authenticate will time out at 1346.485645s 2026-02-17T05:58:20.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c09ddd0 0x7f4a2c09e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c09ddd0 0x7f4a2c09e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37026/0 (socket says 10.20.193.78:37026) 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/883704030 learned_addr learned my addr 10.20.193.78:0/883704030 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient(hunting): get_auth_request con 0x7f4a2c09ddd0 auth_method 0 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_done global_id 144248 payload 274 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/883704030 >> v2:10.20.193.115:3300/0 conn(0x7f4a2c0a12d0 msgr2=0x7f4a2c0a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:20.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2affd640 1 --2- 10.20.193.78:0/883704030 >> v2:10.20.193.115:3300/0 conn(0x7f4a2c0a12d0 0x7f4a2c0a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 1 --2- 10.20.193.78:0/883704030 >> v2:10.20.193.115:3300/0 conn(0x7f4a2c0a12d0 0x7f4a2c0a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/883704030 >> v1:10.20.193.115:6789/0 conn(0x7f4a2c0a6430 legacy=0x7f4a2c0a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 1 monclient: found mon.noname-b 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.931+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/883704030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c09ff00 con 0x7f4a2c09ddd0 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/883704030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c09ddd0 msgr2=0x7f4a2c09e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2b7fe640 1 --2- 10.20.193.78:0/883704030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c09ddd0 0x7f4a2c09e1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009970 tx=0x7f4a1802ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f581fb9a7bef6f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 1 -- 10.20.193.78:0/883704030 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1803c070 con 0x7f4a2c09ddd0 2026-02-17T05:58:20.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: dump: 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 1 -- 10.20.193.78:0/883704030 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1802fdb0 con 0x7f4a2c09ddd0 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 5 monclient: authenticate success, global_id 144248 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 1 -- 10.20.193.78:0/883704030 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18037660 con 0x7f4a2c09ddd0 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 4 set_mon_vals no callback set 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a2a7fc640 10 monclient: dump: 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.949 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 10 monclient: shutdown 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 1 -- 10.20.193.78:0/883704030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c09ddd0 msgr2=0x7f4a2c09e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 1 --2- 10.20.193.78:0/883704030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c09ddd0 0x7f4a2c09e1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009970 tx=0x7f4a1802ee90 comp rx=0 tx=0).stop 2026-02-17T05:58:20.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 1 -- 10.20.193.78:0/883704030 shutdown_connections 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 1 --2- 10.20.193.78:0/883704030 >> v2:10.20.193.115:3300/0 conn(0x7f4a2c0a12d0 0x7f4a2c0a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 1 --2- 10.20.193.78:0/883704030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c09ddd0 0x7f4a2c09e1b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 1 -- 10.20.193.78:0/883704030 >> 10.20.193.78:0/883704030 conn(0x7f4a2c0a7520 msgr2=0x7f4a2c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 1 -- 10.20.193.78:0/883704030 shutdown_connections 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.934+0000 7f4a3085f640 1 -- 10.20.193.78:0/883704030 wait complete. 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient: build_initial_monmap 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient: monmap: 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.951 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 1 Processor -- start 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 1 -- start start 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient: init 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a2c09ddd0 0x7f4a2c131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient(hunting): picked mon.noname-a con 0x7f4a2c09ddd0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a2c0a12d0 0x7f4a2c1cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient(hunting): picked mon.noname-b con 0x7f4a2c0a12d0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:20.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0a6430 0x7f4a2c1cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient(hunting): picked mon.noname-c con 0x7f4a2c0a6430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c0a8430 con 0x7f4a2c09ddd0 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c0a82b0 con 0x7f4a2c0a12d0 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c0a6fd0 con 0x7f4a2c0a6430 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a3085f640 10 monclient(hunting): authenticate will time out at 1346.489743s 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2b7fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a2c09ddd0 msgr2=0x7f4a2c131820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a2c09ddd0 0x7f4a2c131820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0a6430 0x7f4a2c1cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0a6430 0x7f4a2c1cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37034/0 (socket says 10.20.193.78:37034) 2026-02-17T05:58:20.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 -- 10.20.193.78:0/1052975926 learned_addr learned my addr 10.20.193.78:0/1052975926 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient(hunting): get_auth_request con 0x7f4a2c0a6430 auth_method 0 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2affd640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a2c0a12d0 0x7f4a2c1cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2affd640 10 monclient(hunting): get_auth_request con 0x7f4a2c0a12d0 auth_method 0 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient(hunting): handle_auth_done global_id 144251 payload 824 2026-02-17T05:58:20.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a2c0a12d0 msgr2=0x7f4a2c1cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a2c0a12d0 0x7f4a2c1cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a2c09ddd0 msgr2=0x7f4a2c131820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a2c09ddd0 0x7f4a2c131820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 monclient: found mon.noname-c 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.935+0000 7f4a2bfff640 1 -- 10.20.193.78:0/1052975926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c133500 con 0x7f4a2c0a6430 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2bfff640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0a6430 msgr2=0x7f4a2c1cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2bfff640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0a6430 0x7f4a2c1cf500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000b9c0 tx=0x7f4a2000be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20006e60 con 0x7f4a2c0a6430 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: dump: 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: _finish_auth 0 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:20.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 5 monclient: authenticate success, global_id 144251 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a20010070 con 0x7f4a2c0a6430 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000cbd0 con 0x7f4a2c0a6430 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a29ffb640 4 set_mon_vals no callback set 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 10 monclient: dump: 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.957 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 10 monclient: _renew_subs 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2c133a50 con 0x7f4a2c0a6430 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 10 monclient: _renew_subs 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2c131d60 con 0x7f4a2c0a6430 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c1cfa40 0x7f4a2c06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c069d50 con 0x7f4a2c1cfa40 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c1cfa40 0x7f4a2c06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 10 monclient: get_auth_request con 0x7f4a2c1cfa40 auth_method 0 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4a20004070 con 0x7f4a2c0a6430 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_done global_id 144254 payload 824 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.938+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/1052975926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a2c1cfe20 con 0x7f4a2c1cfa40 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.939+0000 7f4a07fff640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f49fc03de60 0x7f49fc040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.939+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(432..432 src has 335..432) ==== 4827+0+0 (secure 0 0 0) 0x7f4a20052560 con 0x7f4a2c0a6430 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.939+0000 7f4a2affd640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f49fc03de60 0x7f49fc040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.939+0000 7f4a2affd640 10 monclient: get_auth_request con 0x7f49fc03de60 auth_method 0 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c1cfa40 msgr2=0x7f4a2c06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a2b7fe640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c1cfa40 0x7f4a2c06ff70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c074040 tx=0x7f4a18004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18048070 con 0x7f4a2c1cfa40 2026-02-17T05:58:20.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 10 monclient: dump: 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4a1803d040 con 0x7f4a2c1cfa40 2026-02-17T05:58:20.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:20.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:20.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c1cfa40 msgr2=0x7f4a2c06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c1cfa40 0x7f4a2c06ff70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c074040 tx=0x7f4a18004830 comp rx=0 tx=0).stop 2026-02-17T05:58:20.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c1cfa40 msgr2=0x7f4a2c06ff70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:20.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a07fff640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c1cfa40 0x7f4a2c06ff70 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c074040 tx=0x7f4a18004830 comp rx=0 tx=0).stop 2026-02-17T05:58:20.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.941+0000 7f4a2affd640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f49fc03de60 0x7f49fc040320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c1314a0 tx=0x7f4a1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a3085f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a3085f640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0704b0 0x7f4a2c1da310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a3085f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c09e1b0 con 0x7f4a2c0704b0 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0704b0 0x7f4a2c1da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 10 monclient: get_auth_request con 0x7f4a2c0704b0 auth_method 0 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 10 monclient(hunting): handle_auth_done global_id 144257 payload 824 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.944+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/1052975926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4a2c0744d0 con 0x7f4a2c0704b0 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a2b7fe640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0704b0 msgr2=0x7f4a2c1da310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a2b7fe640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0704b0 0x7f4a2c1da310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a18002410 tx=0x7f4a18009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1804e070 con 0x7f4a2c0704b0 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 10 monclient: dump: 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7f4a18003e20 con 0x7f4a2c0704b0 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0704b0 msgr2=0x7f4a2c1da310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0704b0 0x7f4a2c1da310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a18002410 tx=0x7f4a18009340 comp rx=0 tx=0).stop 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0704b0 msgr2=0x7f4a2c1da310 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a07fff640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0704b0 0x7f4a2c1da310 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a18002410 tx=0x7f4a18009340 comp rx=0 tx=0).stop 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:20.965 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 138, 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 84482, 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:20.966 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:20.967 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:58:20.968 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:20.969 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.970 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:20.971 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:20.972 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:20.973 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:20.974 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:20.974 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:58:20.974 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:20.974 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:20.974 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:20.974 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:58:20.974 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f49fc03de60 msgr2=0x7f49fc040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f49fc03de60 0x7f49fc040320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c1314a0 tx=0x7f4a1c008040 comp rx=0 tx=0).stop 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 10 monclient: shutdown 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0a6430 msgr2=0x7f4a2c1cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 1 --2- 10.20.193.78:0/1052975926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a2c0a6430 0x7f4a2c1cf500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000b9c0 tx=0x7f4a2000be90 comp rx=0 tx=0).stop 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a2bfff640 1 -- 10.20.193.78:0/1052975926 reap_dead start 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a2bfff640 1 -- 10.20.193.78:0/1052975926 reap_dead start 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 shutdown_connections 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 >> 10.20.193.78:0/1052975926 conn(0x7f4a2c0a7520 msgr2=0x7f4a2c0a3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 shutdown_connections 2026-02-17T05:58:20.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:20.947+0000 7f4a3085f640 1 -- 10.20.193.78:0/1052975926 wait complete. 2026-02-17T05:58:20.976 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:58:20.976 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:58:20.976 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:58:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.041+0000 7f3832ffb640 10 monclient: get_monmap_and_config 2026-02-17T05:58:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.041+0000 7f3832ffb640 10 monclient: build_initial_monmap 2026-02-17T05:58:21.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.041+0000 7f3832ffb640 10 monclient: monmap: 2026-02-17T05:58:21.045 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:21.045 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:21.045 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:21.043199+0000 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:21.043199+0000 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 1 Processor -- start 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 1 -- start start 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient: init 2026-02-17T05:58:21.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f382c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f382c09c4a0 0x7f382c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f382c09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f382c06d380 0x7f382c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f382c06d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f382c06daf0 con 0x7f382c06c890 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f382c0a0880 con 0x7f382c06d380 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f382c09cc80 con 0x7f382c09c4a0 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:21.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3832ffb640 10 monclient(hunting): authenticate will time out at 1346.596799s 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3831571640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f382c06d380 msgr2=0x7f382c13d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3831571640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f382c06d380 0x7f382c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3830d70640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f382c06c890 0x7f382c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:60522/0 (socket says 10.20.193.78:60522) 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3830d70640 1 -- 10.20.193.78:0/2710580435 learned_addr learned my addr 10.20.193.78:0/2710580435 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.042+0000 7f3823fff640 1 --2- 10.20.193.78:0/2710580435 >> v2:10.20.193.115:3300/0 conn(0x7f382c09c4a0 0x7f382c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient(hunting): get_auth_request con 0x7f382c09c4a0 auth_method 0 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:21.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient(hunting): handle_auth_done global_id 134302 payload 274 2026-02-17T05:58:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 1 -- 10.20.193.78:0/2710580435 >> v2:10.20.193.78:3300/0 conn(0x7f382c06d380 msgr2=0x7f382c13d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 1 --2- 10.20.193.78:0/2710580435 >> v2:10.20.193.78:3300/0 conn(0x7f382c06d380 0x7f382c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 1 -- 10.20.193.78:0/2710580435 >> v1:10.20.193.115:6789/0 conn(0x7f382c06c890 legacy=0x7f382c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 1 monclient: found mon.noname-c 2026-02-17T05:58:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:58:21.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.043+0000 7f3823fff640 1 -- 10.20.193.78:0/2710580435 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c09cf60 con 0x7f382c09c4a0 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3823fff640 1 -- 10.20.193.78:0/2710580435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c09c4a0 msgr2=0x7f382c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3823fff640 1 --2- 10.20.193.78:0/2710580435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c09c4a0 0x7f382c13b3a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f382c06dcc0 tx=0x7f381402ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f445cdab547002c6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 1 -- 10.20.193.78:0/2710580435 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381403c070 con 0x7f382c09c4a0 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: dump: 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.057 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 5 monclient: authenticate success, global_id 134302 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 1 -- 10.20.193.78:0/2710580435 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381402fcd0 con 0x7f382c09c4a0 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:21.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 1 -- 10.20.193.78:0/2710580435 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814037660 con 0x7f382c09c4a0 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f38237fe640 10 monclient: dump: 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:21.059 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 4 set_mon_vals no callback set 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 10 monclient: shutdown 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 1 -- 10.20.193.78:0/2710580435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c09c4a0 msgr2=0x7f382c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 1 --2- 10.20.193.78:0/2710580435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c09c4a0 0x7f382c13b3a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f382c06dcc0 tx=0x7f381402ed70 comp rx=0 tx=0).stop 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 1 -- 10.20.193.78:0/2710580435 shutdown_connections 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 1 --2- 10.20.193.78:0/2710580435 >> v2:10.20.193.78:3300/0 conn(0x7f382c06d380 0x7f382c13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 1 --2- 10.20.193.78:0/2710580435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c09c4a0 0x7f382c13b3a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 1 -- 10.20.193.78:0/2710580435 >> 10.20.193.78:0/2710580435 conn(0x7f382c0a6620 msgr2=0x7f382c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 1 -- 10.20.193.78:0/2710580435 shutdown_connections 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.046+0000 7f3832ffb640 1 -- 10.20.193.78:0/2710580435 wait complete. 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient: build_initial_monmap 2026-02-17T05:58:21.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient: monmap: 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:21.061 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 1 Processor -- start 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 1 -- start start 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient: init 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f382c06c890 0x7f382c11f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f382c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c06d380 0x7f382c11f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f382c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f382c09c4a0 0x7f382c11ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f382c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:21.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f382c09bfa0 con 0x7f382c09c4a0 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f382c09be20 con 0x7f382c06d380 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f382c13de30 con 0x7f382c06c890 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3832ffb640 10 monclient(hunting): authenticate will time out at 1346.601560s 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3831571640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f382c09c4a0 msgr2=0x7f382c11ff20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3831571640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f382c09c4a0 0x7f382c11ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f382c06c890 0x7f382c11f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f382c06c890 0x7f382c11f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37052/0 (socket says 10.20.193.78:37052) 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 -- 10.20.193.78:0/2937059921 learned_addr learned my addr 10.20.193.78:0/2937059921 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient(hunting): get_auth_request con 0x7f382c06c890 auth_method 0 2026-02-17T05:58:21.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3823fff640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c06d380 0x7f382c11f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3823fff640 10 monclient(hunting): get_auth_request con 0x7f382c06d380 auth_method 0 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3823fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3823fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient(hunting): handle_auth_done global_id 144260 payload 824 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c06d380 msgr2=0x7f382c11f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:21.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c06d380 0x7f382c11f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f382c09c4a0 msgr2=0x7f382c11ff20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f382c09c4a0 0x7f382c11ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient: _finish_hunting 0 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 monclient: found mon.noname-c 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.047+0000 7f3830d70640 1 -- 10.20.193.78:0/2937059921 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c1231c0 con 0x7f382c06c890 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3830d70640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f382c06c890 msgr2=0x7f382c11f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3830d70640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f382c06c890 0x7f382c11f4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f381c00b530 tx=0x7f381c00ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c00fa60 con 0x7f382c06c890 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: dump: 2026-02-17T05:58:21.065 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: _finish_auth 0 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: _check_auth_tickets 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:21.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c00fc00 con 0x7f382c06c890 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 5 monclient: authenticate success, global_id 144260 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c017400 con 0x7f382c06c890 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3822ffd640 4 set_mon_vals no callback set 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 10 monclient: dump: 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:21.067 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 10 monclient: _renew_subs 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382c072bd0 con 0x7f382c06c890 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 10 monclient: _renew_subs 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f382c122c00 con 0x7f382c06c890 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c13b890 0x7f382c13bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:21.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f382c121c50 con 0x7f382c13b890 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f381c01f060 con 0x7f382c06c890 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3823fff640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c13b890 0x7f382c13bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3823fff640 10 monclient: get_auth_request con 0x7f382c13b890 auth_method 0 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3823fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.050+0000 7f3823fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f38217fa640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37fc03de60 0x7f37fc040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(432..432 src has 335..432) ==== 4827+0+0 (secure 0 0 0) 0x7f381c051ee0 con 0x7f382c06c890 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:21.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f3831571640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37fc03de60 0x7f37fc040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f3823fff640 10 monclient(hunting): handle_auth_done global_id 134308 payload 824 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f3823fff640 1 -- 10.20.193.78:0/2937059921 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f382c0512a0 con 0x7f382c13b890 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.051+0000 7f3831571640 10 monclient: get_auth_request con 0x7f37fc03de60 auth_method 0 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.053+0000 7f3823fff640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c13b890 msgr2=0x7f382c13bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.053+0000 7f3823fff640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c13b890 0x7f382c13bc70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3814039000 tx=0x7f38140047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.053+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814048070 con 0x7f382c13b890 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.053+0000 7f38217fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.053+0000 7f38217fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.053+0000 7f38217fa640 10 monclient: dump: 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.070 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.054+0000 7f3831571640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37fc03de60 0x7f37fc040320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f382c122730 tx=0x7f38280091f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.054+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f3814037e30 con 0x7f382c13b890 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.054+0000 7f38217fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.054+0000 7f38217fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.054+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c13b890 msgr2=0x7f382c13bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.054+0000 7f38217fa640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c13b890 0x7f382c13bc70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3814039000 tx=0x7f38140047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.054+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c13b890 msgr2=0x7f382c13bc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:21.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.054+0000 7f38217fa640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c13b890 0x7f382c13bc70 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3814039000 tx=0x7f38140047c0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3832ffb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3832ffb640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c122dd0 0x7f382c13c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3832ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f382c122a60 con 0x7f382c122dd0 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c122dd0 0x7f382c13c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 10 monclient: get_auth_request con 0x7f382c122dd0 auth_method 0 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:21.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 10 monclient(hunting): handle_auth_done global_id 134311 payload 824 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.057+0000 7f3823fff640 1 -- 10.20.193.78:0/2937059921 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f382c06cc70 con 0x7f382c122dd0 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.059+0000 7f3823fff640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c122dd0 msgr2=0x7f382c13c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.059+0000 7f3823fff640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c122dd0 0x7f382c13c1b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f381402edb0 tx=0x7f3814009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814059020 con 0x7f382c122dd0 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 10 monclient: dump: 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:21.075 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7f3814041440 con 0x7f382c122dd0 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c122dd0 msgr2=0x7f382c13c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c122dd0 0x7f382c13c1b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f381402edb0 tx=0x7f3814009290 comp rx=0 tx=0).stop 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c122dd0 msgr2=0x7f382c13c1b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f38217fa640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f382c122dd0 0x7f382c13c1b0 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f381402edb0 tx=0x7f3814009290 comp rx=0 tx=0).stop 2026-02-17T05:58:21.076 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 138, 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 128779, 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:21.077 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:21.078 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:58:21.079 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:58:21.080 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:21.081 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:21.082 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.083 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:58:21.084 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:21.085 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:58:21.086 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:58:21.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37fc03de60 msgr2=0x7f37fc040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:21.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37fc03de60 0x7f37fc040320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f382c122730 tx=0x7f38280091f0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 10 monclient: shutdown 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f382c06c890 msgr2=0x7f382c11f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f382c06c890 0x7f382c11f4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f381c00b530 tx=0x7f381c00ba00 comp rx=0 tx=0).stop 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3831571640 1 -- 10.20.193.78:0/2937059921 reap_dead start 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 shutdown_connections 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 --2- 10.20.193.78:0/2937059921 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f382c06c890 0x7f382c11f4a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 >> 10.20.193.78:0/2937059921 conn(0x7f382c0a6620 msgr2=0x7f382c06f530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 shutdown_connections 2026-02-17T05:58:21.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:21.060+0000 7f3832ffb640 1 -- 10.20.193.78:0/2937059921 wait complete. 2026-02-17T05:58:21.088 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:58:21.088 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:58:21.433 INFO:tasks.rados.rados.0.trial115.stdout:2073: done (5 left) 2026-02-17T05:58:21.434 INFO:tasks.rados.rados.0.trial115.stdout:2074: snap_remove snap 186 2026-02-17T05:58:22.294 INFO:tasks.rados.rados.0.trial115.stdout:2074: done (5 left) 2026-02-17T05:58:22.295 INFO:tasks.rados.rados.0.trial115.stdout:2075: rollback oid 35 current snap is 205 2026-02-17T05:58:22.295 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 35 to 201 2026-02-17T05:58:22.295 INFO:tasks.rados.rados.0.trial115.stdout:2070: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:58:22.295 INFO:tasks.rados.rados.0.trial115.stdout:2067: expect (ObjNum 675 snap 186 seq_num 675) 2026-02-17T05:58:22.372 INFO:tasks.rados.rados.0.trial115.stdout:2070: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:58:22.372 INFO:tasks.rados.rados.0.trial115.stdout:2070: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:58:22.372 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1198 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:58:22.372 INFO:tasks.rados.rados.0.trial115.stdout:2069: expect (ObjNum 516 snap 132 seq_num 516) 2026-02-17T05:58:22.434 INFO:tasks.rados.rados.0.trial115.stdout:2071: finishing write tid 1 to trial11545420-2 2026-02-17T05:58:22.434 INFO:tasks.rados.rados.0.trial115.stdout:2068: finishing copy_from racing read to trial11545420-48 2026-02-17T05:58:22.434 INFO:tasks.rados.rados.0.trial115.stdout:2068: finishing copy_from to trial11545420-48 2026-02-17T05:58:22.434 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 877 (ObjNum 464 snap 117 seq_num 464) dirty exists 2026-02-17T05:58:22.434 INFO:tasks.rados.rados.0.trial115.stdout:2071: finishing write tid 2 to trial11545420-2 2026-02-17T05:58:22.434 INFO:tasks.rados.rados.0.trial115.stdout:2067: done (5 left) 2026-02-17T05:58:22.434 INFO:tasks.rados.rados.0.trial115.stdout:2068: done (4 left) 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2069: done (3 left) 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2070: done (2 left) 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2076: read oid 3 snap -1 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2076: expect (ObjNum 733 snap 202 seq_num 733) 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2077: read oid 44 snap -1 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2077: expect (ObjNum 740 snap 204 seq_num 740) 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2078: read oid 20 snap -1 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2078: expect deleted 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2079: write (excl) oid 38 current snap is 205 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2071: finishing write tid 3 to trial11545420-2 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2071: finishing write tid 4 to trial11545420-2 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2071: oid 2 updating version 0 to 976 2026-02-17T05:58:22.435 INFO:tasks.rados.rados.0.trial115.stdout:2071: oid 2 updating version 976 to 977 2026-02-17T05:58:22.436 INFO:tasks.rados.rados.0.trial115.stdout:2071: oid 2 updating version 977 to 978 2026-02-17T05:58:22.436 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 978 (ObjNum 748 snap 205 seq_num 748) dirty exists 2026-02-17T05:58:22.436 INFO:tasks.rados.rados.0.trial115.stdout:2071: left oid 2 (ObjNum 748 snap 205 seq_num 748) 2026-02-17T05:58:22.436 INFO:tasks.rados.rados.0.trial115.stdout:2079: seq_num 749 ranges {588443=564095,1796660=708921} 2026-02-17T05:58:22.438 INFO:tasks.rados.rados.0.trial115.stdout:2079: writing trial11545420-38 from 588443 to 1152538 tid 1 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:2079: writing trial11545420-38 from 1796660 to 2505581 tid 2 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:2071: done (5 left) 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:2080: rollback oid 11 current snap is 205 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 178 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:2075: finishing rollback tid 0 to trial11545420-35 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:2075: finishing rollback tid 1 to trial11545420-35 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:2075: finishing rollback tid 2 to trial11545420-35 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1147 (ObjNum 703 snap 194 seq_num 703) dirty exists 2026-02-17T05:58:22.444 INFO:tasks.rados.rados.0.trial115.stdout:2076: expect (ObjNum 733 snap 202 seq_num 733) 2026-02-17T05:58:22.510 INFO:tasks.rados.rados.0.trial115.stdout:2077: expect (ObjNum 740 snap 204 seq_num 740) 2026-02-17T05:58:22.611 INFO:tasks.rados.rados.0.trial115.stdout:2079: finishing write tid 1 to trial11545420-38 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2080: finishing rollback tid 0 to trial11545420-11 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2080: finishing rollback tid 1 to trial11545420-11 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2080: finishing rollback tid 2 to trial11545420-11 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1046 (ObjNum 634 snap 165 seq_num 634) dirty exists 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2079: finishing write tid 2 to trial11545420-38 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2079: finishing write tid 3 to trial11545420-38 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2079: finishing write tid 4 to trial11545420-38 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2079: oid 38 updating version 0 to 1156 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2079: oid 38 version 1156 is already newer than 1155 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:2079: oid 38 updating version 1156 to 1157 2026-02-17T05:58:22.612 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1157 (ObjNum 749 snap 205 seq_num 749) dirty exists 2026-02-17T05:58:22.613 INFO:tasks.rados.rados.0.trial115.stdout:2079: left oid 38 (ObjNum 749 snap 205 seq_num 749) 2026-02-17T05:58:22.613 INFO:tasks.rados.rados.0.trial115.stdout:2075: done (5 left) 2026-02-17T05:58:22.613 INFO:tasks.rados.rados.0.trial115.stdout:2076: done (4 left) 2026-02-17T05:58:22.613 INFO:tasks.rados.rados.0.trial115.stdout:2077: done (3 left) 2026-02-17T05:58:22.613 INFO:tasks.rados.rados.0.trial115.stdout:2078: done (2 left) 2026-02-17T05:58:22.613 INFO:tasks.rados.rados.0.trial115.stdout:2079: done (1 left) 2026-02-17T05:58:22.613 INFO:tasks.rados.rados.0.trial115.stdout:2080: done (0 left) 2026-02-17T05:58:22.613 INFO:tasks.rados.rados.0.trial115.stdout:2081: snap_create 2026-02-17T05:58:23.439 INFO:tasks.rados.rados.0.trial115.stdout:2081: done (0 left) 2026-02-17T05:58:23.440 INFO:tasks.rados.rados.0.trial115.stdout:2082: write (excl) oid 29 current snap is 206 2026-02-17T05:58:23.440 INFO:tasks.rados.rados.0.trial115.stdout:2082: seq_num 750 ranges {605250=639344,1855142=452579} 2026-02-17T05:58:23.444 INFO:tasks.rados.rados.0.trial115.stdout:2082: writing trial11545420-29 from 605250 to 1244594 tid 1 2026-02-17T05:58:23.449 INFO:tasks.rados.rados.0.trial115.stdout:2082: writing trial11545420-29 from 1855142 to 2307721 tid 2 2026-02-17T05:58:23.449 INFO:tasks.rados.rados.0.trial115.stdout:2083: snap_remove snap 199 2026-02-17T05:58:24.936 INFO:tasks.rados.rados.0.trial115.stdout:2083: done (1 left) 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:2084: copy_from oid 21 from oid 28 current snap is 206 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:2082: finishing write tid 1 to trial11545420-29 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:2082: finishing write tid 2 to trial11545420-29 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:2082: finishing write tid 3 to trial11545420-29 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:2082: finishing write tid 4 to trial11545420-29 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:2082: oid 29 updating version 0 to 649 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:2082: oid 29 version 649 is already newer than 648 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:2082: oid 29 updating version 649 to 650 2026-02-17T05:58:24.937 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 650 (ObjNum 750 snap 206 seq_num 750) dirty exists 2026-02-17T05:58:24.938 INFO:tasks.rados.rados.0.trial115.stdout:2082: left oid 29 (ObjNum 750 snap 206 seq_num 750) 2026-02-17T05:58:24.938 INFO:tasks.rados.rados.0.trial115.stdout:2082: done (1 left) 2026-02-17T05:58:24.938 INFO:tasks.rados.rados.0.trial115.stdout:2085: write oid 50 current snap is 206 2026-02-17T05:58:24.938 INFO:tasks.rados.rados.0.trial115.stdout:2085: seq_num 751 ranges {622057=714593,1913624=708433} 2026-02-17T05:58:24.941 INFO:tasks.rados.rados.0.trial115.stdout:2085: writing trial11545420-50 from 622057 to 1336650 tid 1 2026-02-17T05:58:24.946 INFO:tasks.rados.rados.0.trial115.stdout:2085: writing trial11545420-50 from 1913624 to 2622057 tid 2 2026-02-17T05:58:24.946 INFO:tasks.rados.rados.0.trial115.stdout:2086: snap_remove snap 189 2026-02-17T05:58:24.947 INFO:tasks.rados.rados.0.trial115.stdout:2085: finishing write tid 1 to trial11545420-50 2026-02-17T05:58:25.937 INFO:tasks.rados.rados.0.trial115.stdout:2086: done (2 left) 2026-02-17T05:58:25.937 INFO:tasks.rados.rados.0.trial115.stdout:2087: snap_remove snap 205 2026-02-17T05:58:26.938 INFO:tasks.rados.rados.0.trial115.stdout:2087: done (2 left) 2026-02-17T05:58:26.938 INFO:tasks.rados.rados.0.trial115.stdout:2088: delete oid 23 current snap is 206 2026-02-17T05:58:26.938 INFO:tasks.rados.rados.0.trial115.stdout:2084: finishing copy_from racing read to trial11545420-21 2026-02-17T05:58:26.938 INFO:tasks.rados.rados.0.trial115.stdout:2084: finishing copy_from to trial11545420-21 2026-02-17T05:58:26.938 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1020 (ObjNum 747 snap 204 seq_num 747) dirty exists 2026-02-17T05:58:26.939 INFO:tasks.rados.rados.0.trial115.stdout:2085: finishing write tid 2 to trial11545420-50 2026-02-17T05:58:26.939 INFO:tasks.rados.rados.0.trial115.stdout:2085: finishing write tid 3 to trial11545420-50 2026-02-17T05:58:26.939 INFO:tasks.rados.rados.0.trial115.stdout:2085: finishing write tid 4 to trial11545420-50 2026-02-17T05:58:26.939 INFO:tasks.rados.rados.0.trial115.stdout:2085: oid 50 updating version 0 to 935 2026-02-17T05:58:26.939 INFO:tasks.rados.rados.0.trial115.stdout:2085: oid 50 updating version 935 to 936 2026-02-17T05:58:26.939 INFO:tasks.rados.rados.0.trial115.stdout:2085: oid 50 version 936 is already newer than 934 2026-02-17T05:58:26.939 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 936 (ObjNum 751 snap 206 seq_num 751) dirty exists 2026-02-17T05:58:26.940 INFO:tasks.rados.rados.0.trial115.stdout:2085: left oid 50 (ObjNum 751 snap 206 seq_num 751) 2026-02-17T05:58:26.942 INFO:tasks.rados.rados.0.trial115.stdout:2084: done (2 left) 2026-02-17T05:58:26.942 INFO:tasks.rados.rados.0.trial115.stdout:2085: done (1 left) 2026-02-17T05:58:26.942 INFO:tasks.rados.rados.0.trial115.stdout:2088: done (0 left) 2026-02-17T05:58:26.942 INFO:tasks.rados.rados.0.trial115.stdout:2089: rollback oid 41 current snap is 206 2026-02-17T05:58:26.942 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 204 2026-02-17T05:58:26.943 INFO:tasks.rados.rados.0.trial115.stdout:2090: snap_remove snap 201 2026-02-17T05:58:27.295 INFO:tasks.rados.rados.0.trial115.stdout:2090: done (1 left) 2026-02-17T05:58:27.296 INFO:tasks.rados.rados.0.trial115.stdout:2091: copy_from oid 24 from oid 32 current snap is 206 2026-02-17T05:58:27.296 INFO:tasks.rados.rados.0.trial115.stdout:2089: finishing rollback tid 0 to trial11545420-41 2026-02-17T05:58:27.296 INFO:tasks.rados.rados.0.trial115.stdout:2089: finishing rollback tid 1 to trial11545420-41 2026-02-17T05:58:27.296 INFO:tasks.rados.rados.0.trial115.stdout:2089: finishing rollback tid 2 to trial11545420-41 2026-02-17T05:58:27.296 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1060 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:58:27.296 INFO:tasks.rados.rados.0.trial115.stdout:2089: done (1 left) 2026-02-17T05:58:27.296 INFO:tasks.rados.rados.0.trial115.stdout:2092: snap_remove snap 203 2026-02-17T05:58:28.299 INFO:tasks.rados.rados.0.trial115.stdout:2092: done (1 left) 2026-02-17T05:58:28.299 INFO:tasks.rados.rados.0.trial115.stdout:2093: write oid 30 current snap is 206 2026-02-17T05:58:28.299 INFO:tasks.rados.rados.0.trial115.stdout:2093: seq_num 753 ranges {655671=581444,2030588=625083} 2026-02-17T05:58:28.304 INFO:tasks.rados.rados.0.trial115.stdout:2093: writing trial11545420-30 from 655671 to 1237115 tid 1 2026-02-17T05:58:28.310 INFO:tasks.rados.rados.0.trial115.stdout:2093: writing trial11545420-30 from 2030588 to 2655671 tid 2 2026-02-17T05:58:28.310 INFO:tasks.rados.rados.0.trial115.stdout:2094: write (excl) oid 4 current snap is 206 2026-02-17T05:58:28.310 INFO:tasks.rados.rados.0.trial115.stdout:2094: seq_num 754 ranges {672478=656693,2089070=459917} 2026-02-17T05:58:28.315 INFO:tasks.rados.rados.0.trial115.stdout:2094: writing trial11545420-4 from 672478 to 1329171 tid 1 2026-02-17T05:58:28.320 INFO:tasks.rados.rados.0.trial115.stdout:2094: writing trial11545420-4 from 2089070 to 2548987 tid 2 2026-02-17T05:58:28.320 INFO:tasks.rados.rados.0.trial115.stdout:2095: write oid 11 current snap is 206 2026-02-17T05:58:28.320 INFO:tasks.rados.rados.0.trial115.stdout:2091: finishing copy_from racing read to trial11545420-24 2026-02-17T05:58:28.320 INFO:tasks.rados.rados.0.trial115.stdout:2095: seq_num 755 ranges {689285=731942,2147552=541733} 2026-02-17T05:58:28.325 INFO:tasks.rados.rados.0.trial115.stdout:2095: writing trial11545420-11 from 689285 to 1421227 tid 1 2026-02-17T05:58:28.330 INFO:tasks.rados.rados.0.trial115.stdout:2095: writing trial11545420-11 from 2147552 to 2689285 tid 2 2026-02-17T05:58:28.331 INFO:tasks.rados.rados.0.trial115.stdout:2096: snap_remove snap 191 2026-02-17T05:58:29.300 INFO:tasks.rados.rados.0.trial115.stdout:2096: done (4 left) 2026-02-17T05:58:29.300 INFO:tasks.rados.rados.0.trial115.stdout:2097: snap_remove snap 178 2026-02-17T05:58:30.303 INFO:tasks.rados.rados.0.trial115.stdout:2097: done (4 left) 2026-02-17T05:58:30.303 INFO:tasks.rados.rados.0.trial115.stdout:2098: snap_remove snap 204 2026-02-17T05:58:31.305 INFO:tasks.rados.rados.0.trial115.stdout:2098: done (4 left) 2026-02-17T05:58:31.305 INFO:tasks.rados.rados.0.trial115.stdout:2099: snap_create 2026-02-17T05:58:31.305 INFO:tasks.rados.rados.0.trial115.stdout:2091: finishing copy_from to trial11545420-24 2026-02-17T05:58:31.305 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 874 (ObjNum 730 snap 202 seq_num 730) dirty exists 2026-02-17T05:58:31.305 INFO:tasks.rados.rados.0.trial115.stdout:2093: finishing write tid 1 to trial11545420-30 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2093: finishing write tid 2 to trial11545420-30 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2093: finishing write tid 3 to trial11545420-30 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2093: finishing write tid 4 to trial11545420-30 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2093: oid 30 updating version 0 to 937 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2093: oid 30 updating version 937 to 938 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2093: oid 30 updating version 938 to 939 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 939 (ObjNum 753 snap 206 seq_num 753) dirty exists 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2093: left oid 30 (ObjNum 753 snap 206 seq_num 753) 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2094: finishing write tid 1 to trial11545420-4 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2094: finishing write tid 2 to trial11545420-4 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2094: finishing write tid 3 to trial11545420-4 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2094: finishing write tid 4 to trial11545420-4 2026-02-17T05:58:31.306 INFO:tasks.rados.rados.0.trial115.stdout:2094: oid 4 updating version 0 to 900 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2094: oid 4 updating version 900 to 901 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2094: oid 4 version 901 is already newer than 899 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 901 (ObjNum 754 snap 206 seq_num 754) dirty exists 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2094: left oid 4 (ObjNum 754 snap 206 seq_num 754) 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2095: finishing write tid 1 to trial11545420-11 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2095: finishing write tid 2 to trial11545420-11 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2095: finishing write tid 3 to trial11545420-11 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2095: finishing write tid 4 to trial11545420-11 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2095: oid 11 updating version 0 to 1079 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2095: oid 11 version 1079 is already newer than 1078 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:2095: oid 11 version 1079 is already newer than 1073 2026-02-17T05:58:31.307 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1079 (ObjNum 755 snap 206 seq_num 755) dirty exists 2026-02-17T05:58:31.308 INFO:tasks.rados.rados.0.trial115.stdout:2095: left oid 11 (ObjNum 755 snap 206 seq_num 755) 2026-02-17T05:58:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2091: done (4 left) 2026-02-17T05:58:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2093: done (3 left) 2026-02-17T05:58:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2094: done (2 left) 2026-02-17T05:58:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2095: done (1 left) 2026-02-17T05:58:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2099: done (0 left) 2026-02-17T05:58:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2100: write (excl) oid 10 current snap is 207 2026-02-17T05:58:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2100: seq_num 756 ranges {706092=407191,1806034=463586} 2026-02-17T05:58:32.300 INFO:tasks.rados.rados.0.trial115.stdout:2100: writing trial11545420-10 from 706092 to 1113283 tid 1 2026-02-17T05:58:32.305 INFO:tasks.rados.rados.0.trial115.stdout:2100: writing trial11545420-10 from 1806034 to 2269620 tid 2 2026-02-17T05:58:32.305 INFO:tasks.rados.rados.0.trial115.stdout:2101: copy_from oid 23 from oid 13 current snap is 207 2026-02-17T05:58:32.305 INFO:tasks.rados.rados.0.trial115.stdout:2100: finishing write tid 1 to trial11545420-10 2026-02-17T05:58:32.305 INFO:tasks.rados.rados.0.trial115.stdout:2102: write (excl) oid 16 current snap is 207 2026-02-17T05:58:32.305 INFO:tasks.rados.rados.0.trial115.stdout:2102: seq_num 757 ranges {722899=482440,1748163=723597} 2026-02-17T05:58:32.309 INFO:tasks.rados.rados.0.trial115.stdout:2102: writing trial11545420-16 from 722899 to 1205339 tid 1 2026-02-17T05:58:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2102: writing trial11545420-16 from 1748163 to 2471760 tid 2 2026-02-17T05:58:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2103: read oid 43 snap 206 2026-02-17T05:58:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2103: expect (ObjNum 715 snap 195 seq_num 715) 2026-02-17T05:58:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2104: write oid 15 current snap is 207 2026-02-17T05:58:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2104: seq_num 758 ranges {739706=557689,1806645=467255} 2026-02-17T05:58:32.319 INFO:tasks.rados.rados.0.trial115.stdout:2104: writing trial11545420-15 from 739706 to 1297395 tid 1 2026-02-17T05:58:32.323 INFO:tasks.rados.rados.0.trial115.stdout:2104: writing trial11545420-15 from 1806645 to 2273900 tid 2 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2105: rollback oid 42 current snap is 207 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 206 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2100: finishing write tid 2 to trial11545420-10 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2100: finishing write tid 3 to trial11545420-10 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2100: finishing write tid 4 to trial11545420-10 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2100: oid 10 updating version 0 to 1103 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2100: oid 10 version 1103 is already newer than 1102 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2100: oid 10 version 1103 is already newer than 1101 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1103 (ObjNum 756 snap 207 seq_num 756) dirty exists 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2100: left oid 10 (ObjNum 756 snap 207 seq_num 756) 2026-02-17T05:58:32.324 INFO:tasks.rados.rados.0.trial115.stdout:2102: finishing write tid 1 to trial11545420-16 2026-02-17T05:58:32.325 INFO:tasks.rados.rados.0.trial115.stdout:2101: finishing copy_from racing read to trial11545420-23 2026-02-17T05:58:32.325 INFO:tasks.rados.rados.0.trial115.stdout:2100: done (5 left) 2026-02-17T05:58:32.325 INFO:tasks.rados.rados.0.trial115.stdout:2106: write (excl) oid 9 current snap is 207 2026-02-17T05:58:32.325 INFO:tasks.rados.rados.0.trial115.stdout:2106: seq_num 759 ranges {756513=632938,1865127=727266} 2026-02-17T05:58:32.328 INFO:tasks.rados.rados.0.trial115.stdout:2106: writing trial11545420-9 from 756513 to 1389451 tid 1 2026-02-17T05:58:32.333 INFO:tasks.rados.rados.0.trial115.stdout:2106: writing trial11545420-9 from 1865127 to 2592393 tid 2 2026-02-17T05:58:32.333 INFO:tasks.rados.rados.0.trial115.stdout:2107: write oid 35 current snap is 207 2026-02-17T05:58:32.334 INFO:tasks.rados.rados.0.trial115.stdout:2107: seq_num 760 ranges {773320=708187,1923609=470924} 2026-02-17T05:58:32.339 INFO:tasks.rados.rados.0.trial115.stdout:2107: writing trial11545420-35 from 773320 to 1481507 tid 1 2026-02-17T05:58:32.343 INFO:tasks.rados.rados.0.trial115.stdout:2107: writing trial11545420-35 from 1923609 to 2394533 tid 2 2026-02-17T05:58:32.343 INFO:tasks.rados.rados.0.trial115.stdout:2108: write oid 27 current snap is 207 2026-02-17T05:58:32.343 INFO:tasks.rados.rados.0.trial115.stdout:2108: seq_num 761 ranges {790127=499789,1698444=730935} 2026-02-17T05:58:32.348 INFO:tasks.rados.rados.0.trial115.stdout:2108: writing trial11545420-27 from 790127 to 1289916 tid 1 2026-02-17T05:58:32.353 INFO:tasks.rados.rados.0.trial115.stdout:2108: writing trial11545420-27 from 1698444 to 2429379 tid 2 2026-02-17T05:58:32.353 INFO:tasks.rados.rados.0.trial115.stdout:2109: read oid 1 snap -1 2026-02-17T05:58:32.353 INFO:tasks.rados.rados.0.trial115.stdout:2109: expect (ObjNum 696 snap 189 seq_num 696) 2026-02-17T05:58:32.353 INFO:tasks.rados.rados.0.trial115.stdout:2110: rollback oid 33 current snap is 207 2026-02-17T05:58:32.353 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 206 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2101: finishing copy_from to trial11545420-23 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 643 (ObjNum 699 snap 190 seq_num 699) dirty exists 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2102: finishing write tid 2 to trial11545420-16 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2101: done (9 left) 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2111: read oid 45 snap 206 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2111: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2112: snap_create 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2102: finishing write tid 3 to trial11545420-16 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2102: finishing write tid 4 to trial11545420-16 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2102: oid 16 updating version 0 to 1201 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2102: oid 16 updating version 1201 to 1202 2026-02-17T05:58:32.354 INFO:tasks.rados.rados.0.trial115.stdout:2102: oid 16 updating version 1202 to 1203 2026-02-17T05:58:32.355 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1203 (ObjNum 757 snap 207 seq_num 757) dirty exists 2026-02-17T05:58:32.355 INFO:tasks.rados.rados.0.trial115.stdout:2102: left oid 16 (ObjNum 757 snap 207 seq_num 757) 2026-02-17T05:58:32.355 INFO:tasks.rados.rados.0.trial115.stdout:2103: expect (ObjNum 715 snap 195 seq_num 715) 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:2104: finishing write tid 1 to trial11545420-15 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:2104: finishing write tid 2 to trial11545420-15 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:2104: finishing write tid 3 to trial11545420-15 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:2104: finishing write tid 4 to trial11545420-15 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:2104: oid 15 updating version 0 to 1105 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:2104: oid 15 updating version 1105 to 1106 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:2104: oid 15 version 1106 is already newer than 1104 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1106 (ObjNum 758 snap 207 seq_num 758) dirty exists 2026-02-17T05:58:32.411 INFO:tasks.rados.rados.0.trial115.stdout:2104: left oid 15 (ObjNum 758 snap 207 seq_num 758) 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2105: finishing rollback tid 0 to trial11545420-42 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2105: finishing rollback tid 1 to trial11545420-42 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2105: finishing rollback tid 2 to trial11545420-42 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1265 (ObjNum 739 snap 203 seq_num 739) dirty exists 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2106: finishing write tid 1 to trial11545420-9 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2106: finishing write tid 2 to trial11545420-9 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2106: finishing write tid 3 to trial11545420-9 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2106: finishing write tid 4 to trial11545420-9 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2106: oid 9 updating version 0 to 1109 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2106: oid 9 version 1109 is already newer than 1108 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2106: oid 9 updating version 1109 to 1110 2026-02-17T05:58:32.412 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1110 (ObjNum 759 snap 207 seq_num 759) dirty exists 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2106: left oid 9 (ObjNum 759 snap 207 seq_num 759) 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2107: finishing write tid 1 to trial11545420-35 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2107: finishing write tid 2 to trial11545420-35 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2107: finishing write tid 3 to trial11545420-35 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2107: finishing write tid 4 to trial11545420-35 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2107: oid 35 updating version 0 to 1207 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2107: oid 35 version 1207 is already newer than 1206 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2107: oid 35 version 1207 is already newer than 1205 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1207 (ObjNum 760 snap 207 seq_num 760) dirty exists 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2107: left oid 35 (ObjNum 760 snap 207 seq_num 760) 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2108: finishing write tid 1 to trial11545420-27 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2108: finishing write tid 2 to trial11545420-27 2026-02-17T05:58:32.413 INFO:tasks.rados.rados.0.trial115.stdout:2108: finishing write tid 3 to trial11545420-27 2026-02-17T05:58:32.414 INFO:tasks.rados.rados.0.trial115.stdout:2108: finishing write tid 4 to trial11545420-27 2026-02-17T05:58:32.414 INFO:tasks.rados.rados.0.trial115.stdout:2108: oid 27 updating version 0 to 940 2026-02-17T05:58:32.414 INFO:tasks.rados.rados.0.trial115.stdout:2108: oid 27 updating version 940 to 941 2026-02-17T05:58:32.414 INFO:tasks.rados.rados.0.trial115.stdout:2108: oid 27 updating version 941 to 942 2026-02-17T05:58:32.414 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 942 (ObjNum 761 snap 207 seq_num 761) dirty exists 2026-02-17T05:58:32.414 INFO:tasks.rados.rados.0.trial115.stdout:2108: left oid 27 (ObjNum 761 snap 207 seq_num 761) 2026-02-17T05:58:32.414 INFO:tasks.rados.rados.0.trial115.stdout:2111: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:58:32.604 INFO:tasks.rados.rados.0.trial115.stdout:2109: expect (ObjNum 696 snap 189 seq_num 696) 2026-02-17T05:58:32.749 INFO:tasks.rados.rados.0.trial115.stdout:2110: finishing rollback tid 1 to trial11545420-33 2026-02-17T05:58:32.750 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2102: done (10 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2103: done (9 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2104: done (8 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2105: done (7 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2106: done (6 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2107: done (5 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2108: done (4 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2109: done (3 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2110: done (2 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2111: done (1 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2112: done (0 left) 2026-02-17T05:58:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2113: snap_create 2026-02-17T05:58:34.311 INFO:tasks.rados.rados.0.trial115.stdout:2113: done (0 left) 2026-02-17T05:58:34.311 INFO:tasks.rados.rados.0.trial115.stdout:2114: snap_remove snap 207 2026-02-17T05:58:35.312 INFO:tasks.rados.rados.0.trial115.stdout:2114: done (0 left) 2026-02-17T05:58:35.312 INFO:tasks.rados.rados.0.trial115.stdout:2115: read oid 26 snap -1 2026-02-17T05:58:35.312 INFO:tasks.rados.rados.0.trial115.stdout:2115: expect (ObjNum 741 snap 204 seq_num 741) 2026-02-17T05:58:35.312 INFO:tasks.rados.rados.0.trial115.stdout:2116: snap_remove snap 208 2026-02-17T05:58:36.314 INFO:tasks.rados.rados.0.trial115.stdout:2116: done (1 left) 2026-02-17T05:58:36.314 INFO:tasks.rados.rados.0.trial115.stdout:2117: read oid 28 snap 206 2026-02-17T05:58:36.314 INFO:tasks.rados.rados.0.trial115.stdout:2117: expect (ObjNum 747 snap 204 seq_num 747) 2026-02-17T05:58:36.314 INFO:tasks.rados.rados.0.trial115.stdout:2118: copy_from oid 32 from oid 25 current snap is 209 2026-02-17T05:58:36.314 INFO:tasks.rados.rados.0.trial115.stdout:2115: expect (ObjNum 741 snap 204 seq_num 741) 2026-02-17T05:58:36.435 INFO:tasks.rados.rados.0.trial115.stdout:2118: finishing copy_from to trial11545420-32 2026-02-17T05:58:36.435 INFO:tasks.rados.rados.0.trial115.stdout:2118: got expected ENOENT (src dne) 2026-02-17T05:58:36.435 INFO:tasks.rados.rados.0.trial115.stdout:2118: finishing copy_from racing read to trial11545420-32 2026-02-17T05:58:36.435 INFO:tasks.rados.rados.0.trial115.stdout:2117: expect (ObjNum 747 snap 204 seq_num 747) 2026-02-17T05:58:36.516 INFO:tasks.rados.rados.0.trial115.stdout:2115: done (2 left) 2026-02-17T05:58:36.517 INFO:tasks.rados.rados.0.trial115.stdout:2117: done (1 left) 2026-02-17T05:58:36.517 INFO:tasks.rados.rados.0.trial115.stdout:2118: done (0 left) 2026-02-17T05:58:36.517 INFO:tasks.rados.rados.0.trial115.stdout:2119: snap_create 2026-02-17T05:58:37.297 INFO:tasks.rados.rados.0.trial115.stdout:2119: done (0 left) 2026-02-17T05:58:37.297 INFO:tasks.rados.rados.0.trial115.stdout:2120: copy_from oid 31 from oid 49 current snap is 210 2026-02-17T05:58:37.297 INFO:tasks.rados.rados.0.trial115.stdout:2121: rollback oid 2 current snap is 210 2026-02-17T05:58:37.298 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 2 to 206 2026-02-17T05:58:37.298 INFO:tasks.rados.rados.0.trial115.stdout:2122: snap_remove snap 209 2026-02-17T05:58:38.298 INFO:tasks.rados.rados.0.trial115.stdout:2122: done (2 left) 2026-02-17T05:58:38.298 INFO:tasks.rados.rados.0.trial115.stdout:2123: copy_from oid 14 from oid 26 current snap is 210 2026-02-17T05:58:38.298 INFO:tasks.rados.rados.0.trial115.stdout:2121: finishing rollback tid 0 to trial11545420-2 2026-02-17T05:58:38.298 INFO:tasks.rados.rados.0.trial115.stdout:2121: finishing rollback tid 1 to trial11545420-2 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:2120: finishing copy_from racing read to trial11545420-31 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:2121: finishing rollback tid 2 to trial11545420-2 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1038 (ObjNum 748 snap 205 seq_num 748) dirty exists 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:2120: finishing copy_from to trial11545420-31 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 823 (ObjNum 692 snap 187 seq_num 692) dirty exists 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:2120: done (2 left) 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:2121: done (1 left) 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:2124: write oid 22 current snap is 210 2026-02-17T05:58:38.299 INFO:tasks.rados.rados.0.trial115.stdout:2124: seq_num 762 ranges {406934=575038,1640573=474593,2642791=164143} 2026-02-17T05:58:38.302 INFO:tasks.rados.rados.0.trial115.stdout:2124: writing trial11545420-22 from 406934 to 981972 tid 1 2026-02-17T05:58:38.306 INFO:tasks.rados.rados.0.trial115.stdout:2124: writing trial11545420-22 from 1640573 to 2115166 tid 2 2026-02-17T05:58:38.309 INFO:tasks.rados.rados.0.trial115.stdout:2124: writing trial11545420-22 from 2642791 to 2806934 tid 3 2026-02-17T05:58:38.309 INFO:tasks.rados.rados.0.trial115.stdout:2125: snap_remove snap 206 2026-02-17T05:58:39.300 INFO:tasks.rados.rados.0.trial115.stdout:2125: done (2 left) 2026-02-17T05:58:39.300 INFO:tasks.rados.rados.0.trial115.stdout:2126: write oid 8 current snap is 210 2026-02-17T05:58:39.300 INFO:tasks.rados.rados.0.trial115.stdout:2126: seq_num 763 ranges {423741=650287,1699055=618251} 2026-02-17T05:58:39.304 INFO:tasks.rados.rados.0.trial115.stdout:2126: writing trial11545420-8 from 423741 to 1074028 tid 1 2026-02-17T05:58:39.309 INFO:tasks.rados.rados.0.trial115.stdout:2126: writing trial11545420-8 from 1699055 to 2317306 tid 2 2026-02-17T05:58:39.309 INFO:tasks.rados.rados.0.trial115.stdout:2127: write (excl) oid 15 current snap is 210 2026-02-17T05:58:39.309 INFO:tasks.rados.rados.0.trial115.stdout:2123: finishing copy_from racing read to trial11545420-14 2026-02-17T05:58:39.309 INFO:tasks.rados.rados.0.trial115.stdout:2124: finishing write tid 1 to trial11545420-22 2026-02-17T05:58:39.309 INFO:tasks.rados.rados.0.trial115.stdout:2123: finishing copy_from to trial11545420-14 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 620 (ObjNum 741 snap 204 seq_num 741) dirty exists 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: finishing write tid 2 to trial11545420-22 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: finishing write tid 3 to trial11545420-22 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: finishing write tid 4 to trial11545420-22 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: finishing write tid 5 to trial11545420-22 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: oid 22 updating version 0 to 946 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: oid 22 version 946 is already newer than 944 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: oid 22 version 946 is already newer than 945 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: oid 22 version 946 is already newer than 943 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 946 (ObjNum 762 snap 210 seq_num 762) dirty exists 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2124: left oid 22 (ObjNum 762 snap 210 seq_num 762) 2026-02-17T05:58:39.310 INFO:tasks.rados.rados.0.trial115.stdout:2126: finishing write tid 1 to trial11545420-8 2026-02-17T05:58:39.311 INFO:tasks.rados.rados.0.trial115.stdout:2127: seq_num 764 ranges {440548=725536,1757537=478262,2813990=26558} 2026-02-17T05:58:39.313 INFO:tasks.rados.rados.0.trial115.stdout:2127: writing trial11545420-15 from 440548 to 1166084 tid 1 2026-02-17T05:58:39.317 INFO:tasks.rados.rados.0.trial115.stdout:2127: writing trial11545420-15 from 1757537 to 2235799 tid 2 2026-02-17T05:58:39.319 INFO:tasks.rados.rados.0.trial115.stdout:2127: writing trial11545420-15 from 2813990 to 2840548 tid 3 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2123: done (3 left) 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2124: done (2 left) 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2128: rollback oid 33 current snap is 210 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2128: done (2 left) 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2129: delete oid 43 current snap is 210 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2126: finishing write tid 2 to trial11545420-8 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2126: finishing write tid 3 to trial11545420-8 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2126: finishing write tid 4 to trial11545420-8 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2126: oid 8 updating version 0 to 253 2026-02-17T05:58:39.320 INFO:tasks.rados.rados.0.trial115.stdout:2126: oid 8 updating version 253 to 254 2026-02-17T05:58:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2126: oid 8 updating version 254 to 255 2026-02-17T05:58:39.321 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 255 (ObjNum 763 snap 210 seq_num 763) dirty exists 2026-02-17T05:58:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2126: left oid 8 (ObjNum 763 snap 210 seq_num 763) 2026-02-17T05:58:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2127: finishing write tid 1 to trial11545420-15 2026-02-17T05:58:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2127: finishing write tid 2 to trial11545420-15 2026-02-17T05:58:39.322 INFO:tasks.rados.rados.0.trial115.stdout:2127: finishing write tid 3 to trial11545420-15 2026-02-17T05:58:39.322 INFO:tasks.rados.rados.0.trial115.stdout:2127: finishing write tid 4 to trial11545420-15 2026-02-17T05:58:39.322 INFO:tasks.rados.rados.0.trial115.stdout:2127: finishing write tid 5 to trial11545420-15 2026-02-17T05:58:39.322 INFO:tasks.rados.rados.0.trial115.stdout:2127: oid 15 updating version 0 to 1118 2026-02-17T05:58:39.322 INFO:tasks.rados.rados.0.trial115.stdout:2127: oid 15 version 1118 is already newer than 1117 2026-02-17T05:58:39.322 INFO:tasks.rados.rados.0.trial115.stdout:2127: oid 15 version 1118 is already newer than 1115 2026-02-17T05:58:39.322 INFO:tasks.rados.rados.0.trial115.stdout:2127: oid 15 version 1118 is already newer than 1116 2026-02-17T05:58:39.323 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1118 (ObjNum 764 snap 210 seq_num 764) dirty exists 2026-02-17T05:58:39.323 INFO:tasks.rados.rados.0.trial115.stdout:2127: left oid 15 (ObjNum 764 snap 210 seq_num 764) 2026-02-17T05:58:39.323 INFO:tasks.rados.rados.0.trial115.stdout:2126: done (2 left) 2026-02-17T05:58:39.323 INFO:tasks.rados.rados.0.trial115.stdout:2127: done (1 left) 2026-02-17T05:58:39.323 INFO:tasks.rados.rados.0.trial115.stdout:2129: done (0 left) 2026-02-17T05:58:39.323 INFO:tasks.rados.rados.0.trial115.stdout:2130: snap_create 2026-02-17T05:58:40.302 INFO:tasks.rados.rados.0.trial115.stdout:2130: done (0 left) 2026-02-17T05:58:40.302 INFO:tasks.rados.rados.0.trial115.stdout:2131: read oid 38 snap 210 2026-02-17T05:58:40.302 INFO:tasks.rados.rados.0.trial115.stdout:2131: expect (ObjNum 749 snap 205 seq_num 749) 2026-02-17T05:58:40.303 INFO:tasks.rados.rados.0.trial115.stdout:2132: snap_create 2026-02-17T05:58:40.307 INFO:tasks.rados.rados.0.trial115.stdout:2131: expect (ObjNum 749 snap 205 seq_num 749) 2026-02-17T05:58:41.089 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:58:41.089 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T05:58:41.089 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:58:41.089 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:58:41.091 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:58:41.091 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:58:41.091 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.158+0000 7f7b86d85640 10 monclient: get_monmap_and_config 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.158+0000 7f7b86d85640 10 monclient: build_initial_monmap 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.158+0000 7f7b86d85640 10 monclient: monmap: 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:41.159298+0000 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:41.159298+0000 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:41.159 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.158+0000 7f7b86d85640 1 Processor -- start 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.158+0000 7f7b86d85640 1 -- start start 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.158+0000 7f7b86d85640 10 monclient: init 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.158+0000 7f7b86d85640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b80070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient(hunting): picked mon.noname-b con 0x7f7b801384d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 0x7f7b80138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient(hunting): picked mon.noname-a con 0x7f7b801388b0 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7b800a3220 0x7f7b800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient(hunting): picked mon.noname-c con 0x7f7b800a3220 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7b80073f70 con 0x7f7b801388b0 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7b800a4b20 con 0x7f7b800a3220 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7b800a4f70 con 0x7f7b801384d0 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b86d85640 10 monclient(hunting): authenticate will time out at 1366.713107s 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 0x7f7b80138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 0x7f7b80138cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49366/0 (socket says 10.20.193.78:49366) 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 1 -- 10.20.193.78:0/4200824011 learned_addr learned my addr 10.20.193.78:0/4200824011 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 10 monclient(hunting): get_auth_request con 0x7f7b801388b0 auth_method 0 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:41.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 1 --2- 10.20.193.78:0/4200824011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b80070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient(hunting): get_auth_request con 0x7f7b801384d0 auth_method 0 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 1 -- 10.20.193.78:0/4200824011 >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 msgr2=0x7f7b80138cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 1 -- 10.20.193.78:0/4200824011 >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 msgr2=0x7f7b80138cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 1 --2- 10.20.193.78:0/4200824011 >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 0x7f7b80138cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b77fff640 1 --2- 10.20.193.78:0/4200824011 >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 0x7f7b80138cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b852fb640 1 --2- 10.20.193.78:0/4200824011 >> v2:10.20.193.115:3300/0 conn(0x7f7b800a3220 0x7f7b800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b852fb640 10 monclient(hunting): get_auth_request con 0x7f7b800a3220 auth_method 0 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b852fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b852fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b852fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient(hunting): handle_auth_done global_id 144263 payload 274 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 1 -- 10.20.193.78:0/4200824011 >> v2:10.20.193.115:3300/0 conn(0x7f7b800a3220 msgr2=0x7f7b800a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 1 --2- 10.20.193.78:0/4200824011 >> v2:10.20.193.115:3300/0 conn(0x7f7b800a3220 0x7f7b800a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 1 -- 10.20.193.78:0/4200824011 >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 msgr2=0x7f7b80138cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 1 --2- 10.20.193.78:0/4200824011 >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 0x7f7b80138cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient: _finish_hunting 0 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 1 monclient: found mon.noname-b 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:58:41.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.159+0000 7f7b84afa640 1 -- 10.20.193.78:0/4200824011 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8006f870 con 0x7f7b801384d0 2026-02-17T05:58:41.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b84afa640 1 -- 10.20.193.78:0/4200824011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 msgr2=0x7f7b80070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:41.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b84afa640 1 --2- 10.20.193.78:0/4200824011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b80070530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b800a5240 tx=0x7f7b7002ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a24e44e8f7fdf50 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:41.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 1 -- 10.20.193.78:0/4200824011 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7003c070 con 0x7f7b801384d0 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: dump: 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b86d85640 5 monclient: authenticate success, global_id 144263 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 1 -- 10.20.193.78:0/4200824011 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7002fc60 con 0x7f7b801384d0 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b86d85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 1 -- 10.20.193.78:0/4200824011 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70037660 con 0x7f7b801384d0 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b86d85640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b86d85640 4 set_mon_vals no callback set 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.161+0000 7f7b777fe640 10 monclient: dump: 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:41.165 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 10 monclient: shutdown 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 -- 10.20.193.78:0/4200824011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 msgr2=0x7f7b80070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 --2- 10.20.193.78:0/4200824011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b80070530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b800a5240 tx=0x7f7b7002ede0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 -- 10.20.193.78:0/4200824011 shutdown_connections 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 --2- 10.20.193.78:0/4200824011 >> v2:10.20.193.115:3300/0 conn(0x7f7b800a3220 0x7f7b800a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 --2- 10.20.193.78:0/4200824011 >> v2:10.20.193.78:3300/0 conn(0x7f7b801388b0 0x7f7b80138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 --2- 10.20.193.78:0/4200824011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b80070530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 -- 10.20.193.78:0/4200824011 >> 10.20.193.78:0/4200824011 conn(0x7f7b8009c000 msgr2=0x7f7b800a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 -- 10.20.193.78:0/4200824011 shutdown_connections 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 -- 10.20.193.78:0/4200824011 wait complete. 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 10 monclient: build_initial_monmap 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 10 monclient: monmap: 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 Processor -- start 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 1 -- start start 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.162+0000 7f7b86d85640 10 monclient: init 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:58:41.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b800a3220 0x7f7b801cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient(hunting): picked mon.noname-b con 0x7f7b800a3220 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b801cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient(hunting): picked mon.noname-c con 0x7f7b801384d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 0x7f7b80124a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient(hunting): picked mon.noname-a con 0x7f7b801388b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800a94e0 con 0x7f7b801388b0 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7b80139210 con 0x7f7b800a3220 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7b800a9660 con 0x7f7b801384d0 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b86d85640 10 monclient(hunting): authenticate will time out at 1366.717078s 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b801cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b801cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46900/0 (socket says 10.20.193.78:46900) 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 -- 10.20.193.78:0/2634106992 learned_addr learned my addr 10.20.193.78:0/2634106992 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient(hunting): get_auth_request con 0x7f7b801384d0 auth_method 0 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 0x7f7b80124a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 10 monclient(hunting): get_auth_request con 0x7f7b801388b0 auth_method 0 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 1 -- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 msgr2=0x7f7b80124a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 1 -- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 msgr2=0x7f7b80124a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 0x7f7b80124a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b84afa640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b800a3220 0x7f7b801cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:41.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b84afa640 10 monclient(hunting): get_auth_request con 0x7f7b800a3220 auth_method 0 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b84afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b84afa640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b84afa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b852fb640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 0x7f7b80124a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient(hunting): handle_auth_done global_id 144266 payload 824 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 -- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b800a3220 msgr2=0x7f7b801cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b800a3220 0x7f7b801cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 -- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 msgr2=0x7f7b80124a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 0x7f7b80124a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 monclient: found mon.noname-c 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:41.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.163+0000 7f7b77fff640 1 -- 10.20.193.78:0/2634106992 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b80127200 con 0x7f7b801384d0 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b77fff640 1 -- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 msgr2=0x7f7b801cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b77fff640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b801cfbe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800d7e0 tx=0x7f7b6800dcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 1 -- 10.20.193.78:0/2634106992 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68018030 con 0x7f7b801384d0 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: dump: 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: _finish_auth 0 2026-02-17T05:58:41.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: _check_auth_tickets 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 1 -- 10.20.193.78:0/2634106992 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6800bae0 con 0x7f7b801384d0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b86d85640 5 monclient: authenticate success, global_id 144266 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 1 -- 10.20.193.78:0/2634106992 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800bcb0 con 0x7f7b801384d0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b76ffd640 4 set_mon_vals no callback set 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b757fa640 10 monclient: dump: 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b86d85640 10 monclient: _renew_subs 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b86d85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b80128990 con 0x7f7b801384d0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b86d85640 10 monclient: _renew_subs 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b86d85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.165+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b800a6620 con 0x7f7b801384d0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b757fa640 1 -- 10.20.193.78:0/2634106992 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7b68013050 con 0x7f7b801384d0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b86d85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b86d85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b86d85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8009fa10 con 0x7f7b801384d0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b757fa640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7b5003deb0 0x7f7b50040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b757fa640 1 -- 10.20.193.78:0/2634106992 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(452..452 src has 335..452) ==== 4751+0+0 (secure 0 0 0) 0x7f7b68052790 con 0x7f7b801384d0 2026-02-17T05:58:41.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b84afa640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7b5003deb0 0x7f7b50040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:41.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.166+0000 7f7b84afa640 10 monclient: get_auth_request con 0x7f7b5003deb0 auth_method 0 2026-02-17T05:58:41.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.168+0000 7f7b757fa640 1 -- 10.20.193.78:0/2634106992 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7b68022070 con 0x7f7b801384d0 2026-02-17T05:58:41.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.168+0000 7f7b757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:41.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.168+0000 7f7b757fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:41.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.169+0000 7f7b84afa640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7b5003deb0 0x7f7b50040370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7b800a98e0 tx=0x7f7b70033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:41.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.282+0000 7f7b86d85640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:41.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.282+0000 7f7b86d85640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:41.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.282+0000 7f7b86d85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:41.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.282+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7b80070530 con 0x7f7b801384d0 2026-02-17T05:58:41.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.282+0000 7f7b757fa640 1 -- 10.20.193.78:0/2634106992 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f7b680511b0 con 0x7f7b801384d0 2026-02-17T05:58:41.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.282+0000 7f7b757fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:41.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.282+0000 7f7b757fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:41.285 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":138,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:41.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7b5003deb0 msgr2=0x7f7b50040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:41.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7b5003deb0 0x7f7b50040370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7b800a98e0 tx=0x7f7b70033000 comp rx=0 tx=0).stop 2026-02-17T05:58:41.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 10 monclient: shutdown 2026-02-17T05:58:41.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:41.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 msgr2=0x7f7b801cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:41.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b801cfbe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800d7e0 tx=0x7f7b6800dcb0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 shutdown_connections 2026-02-17T05:58:41.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b801388b0 0x7f7b80124a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b801384d0 0x7f7b801cfbe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7b5003deb0 0x7f7b50040370 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 --2- 10.20.193.78:0/2634106992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b800a3220 0x7f7b801cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:41.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 >> 10.20.193.78:0/2634106992 conn(0x7f7b8009c000 msgr2=0x7f7b80071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:41.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 shutdown_connections 2026-02-17T05:58:41.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:41.283+0000 7f7b86d85640 1 -- 10.20.193.78:0/2634106992 wait complete. 2026-02-17T05:58:41.303 INFO:tasks.rados.rados.0.trial115.stdout:2131: done (1 left) 2026-02-17T05:58:41.304 INFO:tasks.rados.rados.0.trial115.stdout:2132: done (0 left) 2026-02-17T05:58:41.304 INFO:tasks.rados.rados.0.trial115.stdout:2133: snap_remove snap 211 2026-02-17T05:58:42.297 INFO:tasks.rados.rados.0.trial115.stdout:2133: done (0 left) 2026-02-17T05:58:42.297 INFO:tasks.rados.rados.0.trial115.stdout:2134: snap_remove snap 210 2026-02-17T05:58:43.298 INFO:tasks.rados.rados.0.trial115.stdout:2134: done (0 left) 2026-02-17T05:58:43.298 INFO:tasks.rados.rados.0.trial115.stdout:2135: write oid 4 current snap is 212 2026-02-17T05:58:43.298 INFO:tasks.rados.rados.0.trial115.stdout:2135: seq_num 766 ranges {474162=476034,1358148=481931,2352483=465303} 2026-02-17T05:58:43.302 INFO:tasks.rados.rados.0.trial115.stdout:2135: writing trial11545420-4 from 474162 to 950196 tid 1 2026-02-17T05:58:43.305 INFO:tasks.rados.rados.0.trial115.stdout:2135: writing trial11545420-4 from 1358148 to 1840079 tid 2 2026-02-17T05:58:43.309 INFO:tasks.rados.rados.0.trial115.stdout:2135: writing trial11545420-4 from 2352483 to 2817786 tid 3 2026-02-17T05:58:43.309 INFO:tasks.rados.rados.0.trial115.stdout:2136: snap_create 2026-02-17T05:58:43.309 INFO:tasks.rados.rados.0.trial115.stdout:2135: finishing write tid 1 to trial11545420-4 2026-02-17T05:58:43.309 INFO:tasks.rados.rados.0.trial115.stdout:2135: finishing write tid 2 to trial11545420-4 2026-02-17T05:58:43.311 INFO:tasks.rados.rados.0.trial115.stdout:2135: finishing write tid 3 to trial11545420-4 2026-02-17T05:58:43.312 INFO:tasks.rados.rados.0.trial115.stdout:2135: finishing write tid 4 to trial11545420-4 2026-02-17T05:58:43.312 INFO:tasks.rados.rados.0.trial115.stdout:2135: finishing write tid 5 to trial11545420-4 2026-02-17T05:58:43.312 INFO:tasks.rados.rados.0.trial115.stdout:2135: oid 4 updating version 0 to 903 2026-02-17T05:58:43.312 INFO:tasks.rados.rados.0.trial115.stdout:2135: oid 4 version 903 is already newer than 902 2026-02-17T05:58:43.312 INFO:tasks.rados.rados.0.trial115.stdout:2135: oid 4 updating version 903 to 905 2026-02-17T05:58:43.312 INFO:tasks.rados.rados.0.trial115.stdout:2135: oid 4 version 905 is already newer than 904 2026-02-17T05:58:43.312 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 905 (ObjNum 766 snap 212 seq_num 766) dirty exists 2026-02-17T05:58:43.312 INFO:tasks.rados.rados.0.trial115.stdout:2135: left oid 4 (ObjNum 766 snap 212 seq_num 766) 2026-02-17T05:58:44.293 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:44.310 INFO:tasks.rados.rados.0.trial115.stdout:2135: done (1 left) 2026-02-17T05:58:44.310 INFO:tasks.rados.rados.0.trial115.stdout:2136: done (0 left) 2026-02-17T05:58:44.310 INFO:tasks.rados.rados.0.trial115.stdout:2137: read oid 20 snap -1 2026-02-17T05:58:44.310 INFO:tasks.rados.rados.0.trial115.stdout:2137: expect deleted 2026-02-17T05:58:44.310 INFO:tasks.rados.rados.0.trial115.stdout:2138: copy_from oid 48 from oid 39 current snap is 213 2026-02-17T05:58:44.310 INFO:tasks.rados.rados.0.trial115.stdout:2139: copy_from oid 14 from oid 7 current snap is 213 2026-02-17T05:58:44.310 INFO:tasks.rados.rados.0.trial115.stdout:2140: snap_create 2026-02-17T05:58:44.313 INFO:tasks.rados.rados.0.trial115.stdout:2138: finishing copy_from to trial11545420-48 2026-02-17T05:58:44.313 INFO:tasks.rados.rados.0.trial115.stdout:2138: got expected ENOENT (src dne) 2026-02-17T05:58:44.313 INFO:tasks.rados.rados.0.trial115.stdout:2138: finishing copy_from racing read to trial11545420-48 2026-02-17T05:58:44.320 INFO:tasks.rados.rados.0.trial115.stdout:2139: finishing copy_from racing read to trial11545420-14 2026-02-17T05:58:44.322 INFO:tasks.rados.rados.0.trial115.stdout:2139: finishing copy_from to trial11545420-14 2026-02-17T05:58:44.322 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 926 (ObjNum 700 snap 193 seq_num 700) dirty exists 2026-02-17T05:58:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.361+0000 7f438b9a8640 10 monclient: get_monmap_and_config 2026-02-17T05:58:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.361+0000 7f438b9a8640 10 monclient: build_initial_monmap 2026-02-17T05:58:44.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.361+0000 7f438b9a8640 10 monclient: monmap: 2026-02-17T05:58:44.363 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:44.363 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:44.363109+0000 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:44.363109+0000 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 1 Processor -- start 2026-02-17T05:58:44.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 1 -- start start 2026-02-17T05:58:44.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient: init 2026-02-17T05:58:44.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:44.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2026-02-17T05:58:44.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f438406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:44.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 0x7f438413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:44.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient(hunting): picked mon.noname-a con 0x7f438409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:44.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f438406d380 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:44.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f438406daf0 con 0x7f438406d380 2026-02-17T05:58:44.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f438409cc80 con 0x7f438406c890 2026-02-17T05:58:44.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:44.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f43840a0880 con 0x7f438409c4a0 2026-02-17T05:58:44.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:44.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f438b9a8640 10 monclient(hunting): authenticate will time out at 1369.916711s 2026-02-17T05:58:44.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 0x7f438413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:44.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 0x7f438413b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49390/0 (socket says 10.20.193.78:49390) 2026-02-17T05:58:44.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 1 -- 10.20.193.78:0/4022442379 learned_addr learned my addr 10.20.193.78:0/4022442379 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:44.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 10 monclient(hunting): get_auth_request con 0x7f438409c4a0 auth_method 0 2026-02-17T05:58:44.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:44.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:44.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:44.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4389f1e640 1 --1- 10.20.193.78:0/4022442379 >> v1:10.20.193.78:6789/0 conn(0x7f438406d380 0x7f438413d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:32896/0 (socket says 10.20.193.78:32896) 2026-02-17T05:58:44.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 1 -- 10.20.193.78:0/4022442379 >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 msgr2=0x7f438413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:44.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 1 -- 10.20.193.78:0/4022442379 >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 msgr2=0x7f438413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:44.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 1 --2- 10.20.193.78:0/4022442379 >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 0x7f438413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:44.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.362+0000 7f4388f1c640 1 --2- 10.20.193.78:0/4022442379 >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 0x7f438413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:44.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.364+0000 7f4389f1e640 1 -- 10.20.193.78:0/4022442379 >> v1:10.20.193.78:6789/0 conn(0x7f438406d380 legacy=0x7f438413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:44.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 1 -- 10.20.193.78:0/4022442379 >> v1:10.20.193.115:6789/0 conn(0x7f438406c890 legacy=0x7f438406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:44.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261653707 0 0) 0x7f438409cc80 con 0x7f438406c890 2026-02-17T05:58:44.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): _negotiate 2026-02-17T05:58:44.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:44.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:44.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): my global_id is 134320 2026-02-17T05:58:44.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f436c003620 con 0x7f438406c890 2026-02-17T05:58:44.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1329089173 0 0) 0x7f436c003620 con 0x7f438406c890 2026-02-17T05:58:44.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 msgr2=0x7f438413b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:44.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 --2- 10.20.193.78:0/4022442379 >> v2:10.20.193.78:3300/0 conn(0x7f438409c4a0 0x7f438413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 >> v1:10.20.193.78:6789/0 conn(0x7f438406d380 legacy=0x7f438413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:44.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:44.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:58:44.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:44.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:58:44.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438409cf60 con 0x7f438406c890 2026-02-17T05:58:44.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: _finish_auth 0 2026-02-17T05:58:44.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:44.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:44.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f43740031f0 con 0x7f438406c890 2026-02-17T05:58:44.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:44.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:58:44.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: dump: 2026-02-17T05:58:44.386 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:44.386 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:44.386 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.386 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.386 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:44.387 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:44.387 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:44.387 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:44.387 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:44.387 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:44.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:58:44.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:44.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 >> v1:10.20.193.115:6789/0 conn(0x7f438406c890 legacy=0x7f438406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:44.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:58:44.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f436c003c20 0x7f436c004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:44.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): picked mon.c con 0x7f436c003c20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:44.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f436c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:44.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): picked mon.a con 0x7f436c004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:44.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f436c009ab0 0x7f436c00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:44.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): picked mon.b con 0x7f436c009ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:44.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:44.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f436c00c490 con 0x7f436c004900 2026-02-17T05:58:44.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:44.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f436c00c640 con 0x7f436c009ab0 2026-02-17T05:58:44.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:44.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f436c00c840 con 0x7f436c003c20 2026-02-17T05:58:44.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:44.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4373fff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:58:44.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f436c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:44.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient(hunting): get_auth_request con 0x7f436c004900 auth_method 0 2026-02-17T05:58:44.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:44.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:44.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:44.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f436c003c20 0x7f436c004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:44.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:44.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:44.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:44.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 10 monclient(hunting): get_auth_request con 0x7f436c003c20 auth_method 0 2026-02-17T05:58:44.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:44.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:44.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:44.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:44.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:44.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438971d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:44.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4389f1e640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f436c009ab0 0x7f436c00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:44.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient(hunting): handle_auth_done global_id 134320 payload 293 2026-02-17T05:58:44.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 1 -- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f436c003c20 msgr2=0x7f436c004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:44.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f436c003c20 0x7f436c004040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 1 -- 10.20.193.78:0/4022442379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f436c009ab0 msgr2=0x7f436c00bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:44.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f436c009ab0 0x7f436c00bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient: _finish_hunting 0 2026-02-17T05:58:44.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 1 monclient: found mon.a 2026-02-17T05:58:44.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:44.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:44.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 1 -- 10.20.193.78:0/4022442379 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c003620 con 0x7f436c004900 2026-02-17T05:58:44.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient: _finish_auth 0 2026-02-17T05:58:44.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 10 monclient: _check_auth_tickets 2026-02-17T05:58:44.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4388f1c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:44.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438b9a8640 5 monclient: authenticate success, global_id 134320 2026-02-17T05:58:44.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f438b9a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:44.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4389f1e640 10 monclient: get_auth_request con 0x7f436c009ab0 auth_method 0 2026-02-17T05:58:44.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4389f1e640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f436c009ab0 0x7f436c00bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:58:44.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.365+0000 7f4389f1e640 1 -- 10.20.193.78:0/4022442379 reap_dead start 2026-02-17T05:58:44.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4388f1c640 1 -- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 msgr2=0x7f436c004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:44.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4388f1c640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f436c004d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4378009b80 tx=0x7f437802f160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8478a44606f1a1a9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:44.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378035070 con 0x7f436c004900 2026-02-17T05:58:44.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:44.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:44.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 10 monclient: dump: 2026-02-17T05:58:44.396 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:44.396 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:44.396 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.396 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.396 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:44.396 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:44.396 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:44.396 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:44.397 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:44.397 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:44.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:44.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f437802fdb0 con 0x7f436c004900 2026-02-17T05:58:44.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:44.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 1 -- 10.20.193.78:0/4022442379 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378009780 con 0x7f436c004900 2026-02-17T05:58:44.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:44.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f4373fff640 10 monclient: dump: 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:44.398 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 4 set_mon_vals no callback set 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 10 monclient: shutdown 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 1 -- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 msgr2=0x7f436c004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f436c004d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4378009b80 tx=0x7f437802f160 comp rx=0 tx=0).stop 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 1 -- 10.20.193.78:0/4022442379 shutdown_connections 2026-02-17T05:58:44.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 1 --2- 10.20.193.78:0/4022442379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f436c004d80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 1 -- 10.20.193.78:0/4022442379 >> 10.20.193.78:0/4022442379 conn(0x7f43840a6620 msgr2=0x7f43840a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:44.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 1 -- 10.20.193.78:0/4022442379 shutdown_connections 2026-02-17T05:58:44.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.368+0000 7f438b9a8640 1 -- 10.20.193.78:0/4022442379 wait complete. 2026-02-17T05:58:44.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient: build_initial_monmap 2026-02-17T05:58:44.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient: monmap: 2026-02-17T05:58:44.400 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:44.400 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:44.400 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 1 Processor -- start 2026-02-17T05:58:44.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 1 -- start start 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient: init 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f43841cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient(hunting): picked mon.noname-a con 0x7f436c004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43841cf890 0x7f4384124c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient(hunting): picked mon.noname-b con 0x7f43841cf890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43841cfcb0 0x7f43841251c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:44.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient(hunting): picked mon.noname-c con 0x7f43841cfcb0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f438409bfa0 con 0x7f436c004900 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f438409be20 con 0x7f43841cf890 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f438406cc70 con 0x7f43841cfcb0 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438b9a8640 10 monclient(hunting): authenticate will time out at 1369.923781s 2026-02-17T05:58:44.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f43841cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f43841cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49418/0 (socket says 10.20.193.78:49418) 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 1 -- 10.20.193.78:0/465523874 learned_addr learned my addr 10.20.193.78:0/465523874 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 10 monclient(hunting): get_auth_request con 0x7f436c004900 auth_method 0 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43841cfcb0 0x7f43841251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 10 monclient(hunting): get_auth_request con 0x7f43841cfcb0 auth_method 0 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:44.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 1 -- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 msgr2=0x7f43841cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 1 -- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 msgr2=0x7f43841cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f43841cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f438971d640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f43841cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 10 monclient(hunting): handle_auth_done global_id 144269 payload 824 2026-02-17T05:58:44.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4388f1c640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43841cf890 0x7f4384124c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 1 -- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43841cf890 msgr2=0x7f4384124c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.369+0000 7f4389f1e640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43841cf890 0x7f4384124c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4389f1e640 1 -- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 msgr2=0x7f43841cf350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4389f1e640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f43841cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4389f1e640 10 monclient: _finish_hunting 0 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4389f1e640 1 monclient: found mon.noname-c 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4389f1e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4389f1e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4389f1e640 1 -- 10.20.193.78:0/465523874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4384128610 con 0x7f43841cfcb0 2026-02-17T05:58:44.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4388f1c640 10 monclient: get_auth_request con 0x7f43841cf890 auth_method 0 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.370+0000 7f4388f1c640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43841cf890 0x7f4384124c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4389f1e640 1 -- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43841cfcb0 msgr2=0x7f43841251c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4389f1e640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43841cfcb0 0x7f43841251c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43840a7460 tx=0x7f438000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 1 -- 10.20.193.78:0/465523874 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380019070 con 0x7f43841cfcb0 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: dump: 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.407 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: _finish_auth 0 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:44.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 5 monclient: authenticate success, global_id 144269 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 10 monclient: _renew_subs 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 1 -- 10.20.193.78:0/465523874 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438000bd80 con 0x7f43841cfcb0 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4384128bd0 con 0x7f43841cfcb0 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 1 -- 10.20.193.78:0/465523874 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380007310 con 0x7f43841cfcb0 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f43737fe640 4 set_mon_vals no callback set 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 10 monclient: dump: 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:44.409 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 10 monclient: _renew_subs 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4384128330 con 0x7f43841cfcb0 2026-02-17T05:58:44.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f438406cfa0 con 0x7f43841cfcb0 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.372+0000 7f4371ffb640 1 -- 10.20.193.78:0/465523874 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f43800074b0 con 0x7f43841cfcb0 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.373+0000 7f4371ffb640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f435403deb0 0x7f4354040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.373+0000 7f4371ffb640 1 -- 10.20.193.78:0/465523874 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(456..456 src has 335..456) ==== 4751+0+0 (secure 0 0 0) 0x7f4380052920 con 0x7f43841cfcb0 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.373+0000 7f438971d640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f435403deb0 0x7f4354040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.373+0000 7f438971d640 10 monclient: get_auth_request con 0x7f435403deb0 auth_method 0 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.375+0000 7f4371ffb640 1 -- 10.20.193.78:0/465523874 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f43800139b0 con 0x7f43841cfcb0 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.375+0000 7f4371ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.375+0000 7f4371ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:44.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.375+0000 7f438971d640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f435403deb0 0x7f4354040370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4374002920 tx=0x7f4374009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:44.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.489+0000 7f438b9a8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:44.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.489+0000 7f438b9a8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:44.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.489+0000 7f438b9a8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:44.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.489+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f43840744d0 con 0x7f43841cfcb0 2026-02-17T05:58:44.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f4371ffb640 1 -- 10.20.193.78:0/465523874 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f4380013470 con 0x7f43841cfcb0 2026-02-17T05:58:44.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f4371ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:44.492 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f4371ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:44.492 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":138,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:44.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f435403deb0 msgr2=0x7f4354040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:44.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f438b9a8640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f435403deb0 0x7f4354040370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4374002920 tx=0x7f4374009040 comp rx=0 tx=0).stop 2026-02-17T05:58:44.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f438b9a8640 10 monclient: shutdown 2026-02-17T05:58:44.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f438b9a8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43841cfcb0 msgr2=0x7f43841251c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.490+0000 7f438b9a8640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43841cfcb0 0x7f43841251c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43840a7460 tx=0x7f438000de10 comp rx=0 tx=0).stop 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.491+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 shutdown_connections 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.491+0000 7f438b9a8640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43841cfcb0 0x7f43841251c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.491+0000 7f438b9a8640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f435403deb0 0x7f4354040370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.491+0000 7f438b9a8640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43841cf890 0x7f4384124c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.491+0000 7f438b9a8640 1 --2- 10.20.193.78:0/465523874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f436c004900 0x7f43841cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.491+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 >> 10.20.193.78:0/465523874 conn(0x7f43840a6620 msgr2=0x7f43840a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.491+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 shutdown_connections 2026-02-17T05:58:44.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:44.491+0000 7f438b9a8640 1 -- 10.20.193.78:0/465523874 wait complete. 2026-02-17T05:58:45.312 INFO:tasks.rados.rados.0.trial115.stdout:2137: done (3 left) 2026-02-17T05:58:45.312 INFO:tasks.rados.rados.0.trial115.stdout:2138: done (2 left) 2026-02-17T05:58:45.312 INFO:tasks.rados.rados.0.trial115.stdout:2139: done (1 left) 2026-02-17T05:58:45.312 INFO:tasks.rados.rados.0.trial115.stdout:2140: done (0 left) 2026-02-17T05:58:45.313 INFO:tasks.rados.rados.0.trial115.stdout:2141: write (excl) oid 30 current snap is 214 2026-02-17T05:58:45.313 INFO:tasks.rados.rados.0.trial115.stdout:2141: seq_num 767 ranges {490969=551283,1816630=625589} 2026-02-17T05:58:45.316 INFO:tasks.rados.rados.0.trial115.stdout:2141: writing trial11545420-30 from 490969 to 1042252 tid 1 2026-02-17T05:58:45.322 INFO:tasks.rados.rados.0.trial115.stdout:2141: writing trial11545420-30 from 1816630 to 2442219 tid 2 2026-02-17T05:58:45.322 INFO:tasks.rados.rados.0.trial115.stdout:2142: rollback oid 38 current snap is 214 2026-02-17T05:58:45.322 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 38 to 212 2026-02-17T05:58:45.322 INFO:tasks.rados.rados.0.trial115.stdout:2141: finishing write tid 1 to trial11545420-30 2026-02-17T05:58:45.323 INFO:tasks.rados.rados.0.trial115.stdout:2143: snap_create 2026-02-17T05:58:45.324 INFO:tasks.rados.rados.0.trial115.stdout:2141: finishing write tid 2 to trial11545420-30 2026-02-17T05:58:45.325 INFO:tasks.rados.rados.0.trial115.stdout:2142: finishing rollback tid 0 to trial11545420-38 2026-02-17T05:58:45.325 INFO:tasks.rados.rados.0.trial115.stdout:2141: finishing write tid 3 to trial11545420-30 2026-02-17T05:58:45.325 INFO:tasks.rados.rados.0.trial115.stdout:2142: finishing rollback tid 1 to trial11545420-38 2026-02-17T05:58:45.326 INFO:tasks.rados.rados.0.trial115.stdout:2141: finishing write tid 4 to trial11545420-30 2026-02-17T05:58:45.326 INFO:tasks.rados.rados.0.trial115.stdout:2141: oid 30 updating version 0 to 949 2026-02-17T05:58:45.326 INFO:tasks.rados.rados.0.trial115.stdout:2141: oid 30 version 949 is already newer than 948 2026-02-17T05:58:45.326 INFO:tasks.rados.rados.0.trial115.stdout:2141: oid 30 version 949 is already newer than 947 2026-02-17T05:58:45.326 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 949 (ObjNum 767 snap 214 seq_num 767) dirty exists 2026-02-17T05:58:45.326 INFO:tasks.rados.rados.0.trial115.stdout:2141: left oid 30 (ObjNum 767 snap 214 seq_num 767) 2026-02-17T05:58:45.326 INFO:tasks.rados.rados.0.trial115.stdout:2142: finishing rollback tid 2 to trial11545420-38 2026-02-17T05:58:45.326 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1215 (ObjNum 749 snap 205 seq_num 749) dirty exists 2026-02-17T05:58:46.317 INFO:tasks.rados.rados.0.trial115.stdout:2141: done (2 left) 2026-02-17T05:58:46.317 INFO:tasks.rados.rados.0.trial115.stdout:2142: done (1 left) 2026-02-17T05:58:46.317 INFO:tasks.rados.rados.0.trial115.stdout:2143: done (0 left) 2026-02-17T05:58:46.317 INFO:tasks.rados.rados.0.trial115.stdout:2144: snap_create 2026-02-17T05:58:47.298 INFO:tasks.rados.rados.0.trial115.stdout:2144: done (0 left) 2026-02-17T05:58:47.298 INFO:tasks.rados.rados.0.trial115.stdout:2145: snap_remove snap 212 2026-02-17T05:58:47.500 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:47.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.568+0000 7f32961b6640 10 monclient: get_monmap_and_config 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.568+0000 7f32961b6640 10 monclient: build_initial_monmap 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.568+0000 7f32961b6640 10 monclient: monmap: 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:47.570046+0000 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:47.570046+0000 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:47.573 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 1 Processor -- start 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 1 -- start start 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient: init 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:47.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient(hunting): picked mon.noname-b con 0x7f329006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient(hunting): picked mon.noname-c con 0x7f329009d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient(hunting): picked mon.noname-a con 0x7f329009bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f329009ca50 con 0x7f329009bf40 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f329006d6f0 con 0x7f329009d430 2026-02-17T05:58:47.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3290073600 con 0x7f329006c890 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f32961b6640 10 monclient(hunting): authenticate will time out at 1373.123582s 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48000/0 (socket says 10.20.193.78:48000) 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 learned_addr learned my addr 10.20.193.78:0/2079807083 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 10 monclient(hunting): get_auth_request con 0x7f329006c890 auth_method 0 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 10 monclient(hunting): get_auth_request con 0x7f329009bf40 auth_method 0 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:47.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 10 monclient(hunting): get_auth_request con 0x7f329009d430 auth_method 0 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:47.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.569+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:47.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 10 monclient(hunting): get_auth_request con 0x7f329009bf40 auth_method 0 2026-02-17T05:58:47.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:47.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 10 monclient(hunting): get_auth_request con 0x7f329006c890 auth_method 0 2026-02-17T05:58:47.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:58:47.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:47.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 10 monclient(hunting): get_auth_request con 0x7f329009d430 auth_method 0 2026-02-17T05:58:47.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:47.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:47.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:47.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:58:47.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:47.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:47.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:47.770+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:58:48.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:48.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:48.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 10 monclient(hunting): get_auth_request con 0x7f329009bf40 auth_method 0 2026-02-17T05:58:48.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:48.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:48.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:48.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 10 monclient(hunting): get_auth_request con 0x7f329006c890 auth_method 0 2026-02-17T05:58:48.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:58:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 10 monclient(hunting): get_auth_request con 0x7f329009d430 auth_method 0 2026-02-17T05:58:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.171+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:58:48.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:48.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:48.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 10 monclient(hunting): get_auth_request con 0x7f329006c890 auth_method 0 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 10 monclient(hunting): get_auth_request con 0x7f329009bf40 auth_method 0 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:48.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 10 monclient(hunting): get_auth_request con 0x7f329009d430 auth_method 0 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:48.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:48.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:48.972+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:58:50.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 10 monclient(hunting): tick 2026-02-17T05:58:50.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:58:50.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 monclient(hunting): continuing hunt 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 msgr2=0x7f329006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f329006c890 0x7f329006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 msgr2=0x7f329006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.115:3300/0 conn(0x7f329009d430 0x7f329006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 -- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 msgr2=0x7f3290134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 --2- 10.20.193.78:0/2079807083 >> v2:10.20.193.78:3300/0 conn(0x7f329009bf40 0x7f3290134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 10 monclient: _add_conns ranks=[1,0] 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f326c003750 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:50.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f326c0074a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:58:50.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 -- 10.20.193.78:0/2079807083 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f329006dab0 con 0x7f326c0074a0 2026-02-17T05:58:50.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 1 -- 10.20.193.78:0/2079807083 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f326c00c1e0 con 0x7f326c003750 2026-02-17T05:58:50.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.570+0000 7f328d7fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:50.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.571+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v1:10.20.193.78:6789/0 conn(0x7f326c0074a0 legacy=0x7f326c009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:50.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:50.572+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> v1:10.20.193.115:6789/0 conn(0x7f326c003750 legacy=0x7f326c005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:51.154+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v1:10.20.193.78:6789/0 conn(0x7f326c0074a0 legacy=0x7f326c009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-17T05:58:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:51.154+0000 7f328effd640 1 -- 10.20.193.78:0/2079807083 >> v1:10.20.193.78:6789/0 conn(0x7f326c0074a0 legacy=0x7f326c009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:58:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:51.154+0000 7f328effd640 1 --1- 10.20.193.78:0/2079807083 >> v1:10.20.193.78:6789/0 conn(0x7f326c0074a0 0x7f326c009960 :-1 s=OPENED pgs=26 cs=1 l=1).handle_message read tag failed 2026-02-17T05:58:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:51.154+0000 7f328effd640 1 --1- 10.20.193.78:0/2079807083 >> v1:10.20.193.78:6789/0 conn(0x7f326c0074a0 0x7f326c009960 :-1 s=OPENED pgs=26 cs=1 l=1).fault on lossy channel, failing 2026-02-17T05:58:51.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:51.154+0000 7f328e7fc640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.78:6789/0 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3911611145 0 0) 0x7f326c00c1e0 con 0x7f326c003750 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient(hunting): _negotiate 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient(hunting): my global_id is 134323 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3274003620 con 0x7f326c003750 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3610376393 0 0) 0x7f3274003620 con 0x7f326c003750 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 >> v1:10.20.193.78:6789/0 conn(0x7f326c0074a0 legacy=0x7f326c009960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c00d4a0 con 0x7f326c003750 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient: _finish_auth 0 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:52.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f32961b6640 5 monclient: authenticate success, global_id 134323 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f32961b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f327c009780 con 0x7f326c003750 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.389+0000 7f328e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient: dump: 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:52.392 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:52.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:58:52.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:52.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 >> v1:10.20.193.115:6789/0 conn(0x7f326c003750 legacy=0x7f326c005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:52.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328ffff640 1 -- 10.20.193.78:0/2079807083 reap_dead start 2026-02-17T05:58:52.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274003b50 0x7f3274003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient(hunting): picked mon.c con 0x7f3274003b50 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3274004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient(hunting): picked mon.a con 0x7f3274004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274009ab0 0x7f327400bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient(hunting): picked mon.b con 0x7f3274009ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f327400c490 con 0x7f3274004900 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f327400c610 con 0x7f3274009ab0 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f327400c810 con 0x7f3274003b50 2026-02-17T05:58:52.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f32961b6640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274003b50 0x7f3274003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient(hunting): get_auth_request con 0x7f3274003b50 auth_method 0 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328effd640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3274004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328effd640 10 monclient(hunting): get_auth_request con 0x7f3274004900 auth_method 0 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:52.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient(hunting): handle_auth_done global_id 134323 payload 293 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274009ab0 0x7f327400bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274009ab0 msgr2=0x7f327400bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274009ab0 0x7f327400bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 msgr2=0x7f3274004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3274004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 1 monclient: found mon.c 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3274003620 con 0x7f3274003b50 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:52.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328ffff640 10 monclient: get_auth_request con 0x7f3274009ab0 auth_method 0 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.390+0000 7f328ffff640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274009ab0 0x7f327400bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328f7fe640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274003b50 msgr2=0x7f3274003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328f7fe640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274003b50 0x7f3274003f30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f327c004310 tx=0x7f327c030150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f55d4a4929fea1eb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c036070 con 0x7f3274003b50 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 10 monclient: dump: 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:52.403 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f32961b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327c030b70 con 0x7f3274003b50 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 1 -- 10.20.193.78:0/2079807083 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c004040 con 0x7f3274003b50 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f328e7fc640 10 monclient: dump: 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:52.404 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f32961b6640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.393+0000 7f32961b6640 4 set_mon_vals no callback set 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 10 monclient: shutdown 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 -- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274003b50 msgr2=0x7f3274003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274003b50 0x7f3274003f30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f327c004310 tx=0x7f327c030150 comp rx=0 tx=0).stop 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 -- 10.20.193.78:0/2079807083 shutdown_connections 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274009ab0 0x7f327400bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3274004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 --2- 10.20.193.78:0/2079807083 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274003b50 0x7f3274003f30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 -- 10.20.193.78:0/2079807083 >> 10.20.193.78:0/2079807083 conn(0x7f32900a6620 msgr2=0x7f32900a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 -- 10.20.193.78:0/2079807083 shutdown_connections 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 -- 10.20.193.78:0/2079807083 wait complete. 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 10 monclient: build_initial_monmap 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 10 monclient: monmap: 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:52.405 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.394+0000 7f32961b6640 1 Processor -- start 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 1 -- start start 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient: init 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274003b50 0x7f32901cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient(hunting): picked mon.noname-b con 0x7f3274003b50 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3290135400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient(hunting): picked mon.noname-a con 0x7f3274004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 0x7f32901ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient(hunting): picked mon.noname-c con 0x7f3274009ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:52.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3290073380 con 0x7f3274004900 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3290073200 con 0x7f3274003b50 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f329006cc70 con 0x7f3274009ab0 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f32961b6640 10 monclient(hunting): authenticate will time out at 1377.949382s 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3290135400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 0x7f32901ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3290135400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40566/0 (socket says 10.20.193.78:40566) 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 -- 10.20.193.78:0/1076814483 learned_addr learned my addr 10.20.193.78:0/1076814483 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient(hunting): get_auth_request con 0x7f3274004900 auth_method 0 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 10 monclient(hunting): get_auth_request con 0x7f3274009ab0 auth_method 0 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:52.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 1 -- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 msgr2=0x7f32901ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 1 -- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 msgr2=0x7f32901ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 0x7f32901ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328ffff640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 0x7f32901ce820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328f7fe640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274003b50 0x7f32901cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328f7fe640 10 monclient(hunting): get_auth_request con 0x7f3274003b50 auth_method 0 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient(hunting): handle_auth_done global_id 164097 payload 824 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 -- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 msgr2=0x7f32901ce820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 0x7f32901ce820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 -- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274003b50 msgr2=0x7f32901cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274003b50 0x7f32901cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient: _finish_hunting 0 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 monclient: found mon.noname-a 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:52.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.395+0000 7f328effd640 1 -- 10.20.193.78:0/1076814483 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3290123b00 con 0x7f3274004900 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f328effd640 1 -- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 msgr2=0x7f3290135400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f328effd640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3290135400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3280004590 tx=0x7f3280003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 1 -- 10.20.193.78:0/1076814483 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328001c070 con 0x7f3274004900 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: dump: 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: _finish_auth 0 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:58:52.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 1 -- 10.20.193.78:0/1076814483 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3280016c10 con 0x7f3274004900 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f32961b6640 5 monclient: authenticate success, global_id 164097 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f32961b6640 10 monclient: _renew_subs 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f32961b6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3290125df0 con 0x7f3274004900 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 1 -- 10.20.193.78:0/1076814483 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328000f040 con 0x7f3274004900 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f328dffb640 4 set_mon_vals no callback set 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.397+0000 7f326bfff640 10 monclient: dump: 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:52.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f32961b6640 10 monclient: _renew_subs 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f32961b6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3290125b10 con 0x7f3274004900 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f32961b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f326bfff640 1 -- 10.20.193.78:0/1076814483 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3280029030 con 0x7f3274004900 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f32961b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f32961b6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3290074980 con 0x7f3274004900 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f326bfff640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f326003df00 0x7f32600403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.398+0000 7f326bfff640 1 -- 10.20.193.78:0/1076814483 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(459..459 src has 335..459) ==== 4779+0+0 (secure 0 0 0) 0x7f3280053910 con 0x7f3274004900 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.399+0000 7f328f7fe640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f326003df00 0x7f32600403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.399+0000 7f328f7fe640 10 monclient: get_auth_request con 0x7f326003df00 auth_method 0 2026-02-17T05:58:52.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.401+0000 7f326bfff640 1 -- 10.20.193.78:0/1076814483 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3280021e10 con 0x7f3274004900 2026-02-17T05:58:52.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.401+0000 7f326bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:52.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.401+0000 7f326bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:52.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.401+0000 7f328f7fe640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f326003df00 0x7f32600403c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f327c006fd0 tx=0x7f327c036000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:52.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.516+0000 7f32961b6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:52.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.516+0000 7f32961b6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:52.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.516+0000 7f32961b6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:52.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.516+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f329006d050 con 0x7f3274004900 2026-02-17T05:58:52.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.516+0000 7f326bfff640 1 -- 10.20.193.78:0/1076814483 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f3280052310 con 0x7f3274004900 2026-02-17T05:58:52.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.516+0000 7f326bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:52.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.516+0000 7f326bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:52.519 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":142,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:52.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f326003df00 msgr2=0x7f32600403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:52.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f326003df00 0x7f32600403c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f327c006fd0 tx=0x7f327c036000 comp rx=0 tx=0).stop 2026-02-17T05:58:52.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 10 monclient: shutdown 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 msgr2=0x7f3290135400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3290135400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3280004590 tx=0x7f3280003ae0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 shutdown_connections 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3274009ab0 0x7f32901ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3274004900 0x7f3290135400 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f326003df00 0x7f32600403c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 --2- 10.20.193.78:0/1076814483 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3274003b50 0x7f32901cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 >> 10.20.193.78:0/1076814483 conn(0x7f32900a6620 msgr2=0x7f32901365d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 shutdown_connections 2026-02-17T05:58:52.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:52.517+0000 7f32961b6640 1 -- 10.20.193.78:0/1076814483 wait complete. 2026-02-17T05:58:52.737 INFO:tasks.rados.rados.0.trial115.stdout:2145: done (0 left) 2026-02-17T05:58:52.737 INFO:tasks.rados.rados.0.trial115.stdout:2146: read oid 4 snap 213 2026-02-17T05:58:52.737 INFO:tasks.rados.rados.0.trial115.stdout:2146: expect (ObjNum 766 snap 212 seq_num 766) 2026-02-17T05:58:52.737 INFO:tasks.rados.rados.0.trial115.stdout:2147: snap_create 2026-02-17T05:58:53.862 INFO:tasks.rados.rados.0.trial115.stdout:2146: expect (ObjNum 766 snap 212 seq_num 766) 2026-02-17T05:58:55.528 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:58:55.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.597+0000 7f5ef7243640 10 monclient: get_monmap_and_config 2026-02-17T05:58:55.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.597+0000 7f5ef7243640 10 monclient: build_initial_monmap 2026-02-17T05:58:55.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.597+0000 7f5ef7243640 10 monclient: monmap: 2026-02-17T05:58:55.603 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:55.604 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:55.604 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:58:55.598606+0000 2026-02-17T05:58:55.604 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:58:55.598606+0000 2026-02-17T05:58:55.604 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:55.604 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:55.604 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:58:55.605 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:58:55.605 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:58:55.605 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:58:55.605 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:58:55.605 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:55.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.597+0000 7f5ef7243640 1 Processor -- start 2026-02-17T05:58:55.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.597+0000 7f5ef7243640 1 -- start start 2026-02-17T05:58:55.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.597+0000 7f5ef7243640 10 monclient: init 2026-02-17T05:58:55.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:55.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2026-02-17T05:58:55.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:55.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient(hunting): picked mon.noname-c con 0x7f5ef006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:58:55.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:55.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient(hunting): picked mon.noname-a con 0x7f5ef006cc70 addr v2:10.20.193.78:3300/0 2026-02-17T05:58:55.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5ef0134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:58:55.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef0074150 con 0x7f5ef0134df0 2026-02-17T05:58:55.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:55.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef009ef70 con 0x7f5ef006cc70 2026-02-17T05:58:55.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:55.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef009f250 con 0x7f5ef006c890 2026-02-17T05:58:55.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:55.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef7243640 10 monclient(hunting): authenticate will time out at 1381.152139s 2026-02-17T05:58:55.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:55.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40582/0 (socket says 10.20.193.78:40582) 2026-02-17T05:58:55.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 learned_addr learned my addr 10.20.193.78:0/2843986065 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:55.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request con 0x7f5ef006cc70 auth_method 0 2026-02-17T05:58:55.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:55.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:55.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:55.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:55.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:55.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:55.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:55.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request con 0x7f5ef006c890 auth_method 0 2026-02-17T05:58:55.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:55.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:55.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:55.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:55.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:55.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:55.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:55.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.598+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:55.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.600+0000 7f5ef57b9640 1 -- 10.20.193.78:0/2843986065 >> v1:10.20.193.115:6789/0 conn(0x7f5ef0134df0 legacy=0x7f5ef0135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:55.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:55.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request con 0x7f5ef006cc70 auth_method 0 2026-02-17T05:58:55.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:55.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:55.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request con 0x7f5ef006c890 auth_method 0 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.798+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.799+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.799+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:55.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.799+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:55.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:55.799+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:58:56.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:56.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request con 0x7f5ef006cc70 auth_method 0 2026-02-17T05:58:56.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:56.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:56.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request con 0x7f5ef006c890 auth_method 0 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:56.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:56.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:56.199+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request con 0x7f5ef006cc70 auth_method 0 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request con 0x7f5ef006c890 auth_method 0 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:57.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.000+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.001+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.001+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.001+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:57.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:57.001+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.390+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916202384 0 0) 0x7f5ef0074150 con 0x7f5ef0134df0 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.390+0000 7f5ee77fe640 10 monclient(hunting): _negotiate 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.390+0000 7f5ee77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.390+0000 7f5ee77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.390+0000 7f5ee77fe640 10 monclient(hunting): my global_id is 134329 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.390+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5edc0058b0 con 0x7f5ef0134df0 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1241004986 0 0) 0x7f5edc0058b0 con 0x7f5ef0134df0 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 msgr2=0x7f5ef00a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.115:3300/0 conn(0x7f5ef006c890 0x7f5ef00a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 msgr2=0x7f5ef006d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> v2:10.20.193.78:3300/0 conn(0x7f5ef006cc70 0x7f5ef006d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef009df40 con 0x7f5ef0134df0 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: _finish_auth 0 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f5eec003580 con 0x7f5ef0134df0 2026-02-17T05:58:58.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef7243640 5 monclient: authenticate success, global_id 134329 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef7243640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: dump: 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 >> v1:10.20.193.115:6789/0 conn(0x7f5ef0134df0 legacy=0x7f5ef0135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc005eb0 0x7f5edc0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient(hunting): picked mon.b con 0x7f5edc005eb0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc006b90 0x7f5edc007010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient(hunting): picked mon.a con 0x7f5edc006b90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 0x7f5edc00dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient(hunting): picked mon.c con 0x7f5edc0076c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef009ef70 con 0x7f5edc006b90 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef009f250 con 0x7f5edc005eb0 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc00e660 con 0x7f5edc0076c0 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ee77fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef7243640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 0x7f5edc00dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient(hunting): get_auth_request con 0x7f5edc0076c0 auth_method 0 2026-02-17T05:58:58.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc005eb0 0x7f5edc0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request con 0x7f5edc005eb0 auth_method 0 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef4fb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef4fb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef4fb8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient(hunting): handle_auth_done global_id 134329 payload 293 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc005eb0 msgr2=0x7f5edc0062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc005eb0 0x7f5edc0062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc006b90 msgr2=0x7f5edc007010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc006b90 0x7f5edc007010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient: _finish_hunting 0 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 1 monclient: found mon.c 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 1 -- 10.20.193.78:0/2843986065 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc0058b0 con 0x7f5edc0076c0 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient: _finish_auth 0 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 10 monclient: _check_auth_tickets 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.391+0000 7f5ef57b9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:58.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.392+0000 7f5ef57b9640 1 -- 10.20.193.78:0/2843986065 reap_dead start 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 msgr2=0x7f5edc00dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 0x7f5edc00dd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0037a0 tx=0x7f5eec0300d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88b68163e8f9eaaf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00a870 con 0x7f5edc0076c0 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 msgr2=0x7f5edc00dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 msgr2=0x7f5edc00dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 0x7f5edc00dd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0037a0 tx=0x7f5eec0300d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 0x7f5edc00dd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0037a0 tx=0x7f5eec0300d0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient: dump: 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef7243640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:58.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient: ms_handle_reset current mon [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 msgr2=0x7f5edc00dd50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0076c0 0x7f5edc00dd50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0062d0 0x7f5edc006fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient(hunting): picked mon.c con 0x7f5edc0062d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc009ec0 0x7f5edc00a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient(hunting): picked mon.b con 0x7f5edc009ec0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc00aa30 0x7f5edc00aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient(hunting): picked mon.a con 0x7f5edc00aa30 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc00b720 con 0x7f5edc00aa30 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc00b8d0 con 0x7f5edc009ec0 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc00ba80 con 0x7f5edc0062d0 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ee77fe640 10 monclient(hunting): ms_handle_reset hunted mon [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0062d0 0x7f5edc006fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request con 0x7f5edc0062d0 auth_method 0 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc00aa30 0x7f5edc00aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 10 monclient(hunting): get_auth_request con 0x7f5edc00aa30 auth_method 0 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef57b9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient(hunting): handle_auth_done global_id 134329 payload 293 2026-02-17T05:58:58.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc009ec0 msgr2=0x7f5edc00a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc009ec0 0x7f5edc00a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc00aa30 msgr2=0x7f5edc00aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc00aa30 0x7f5edc00aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient: _finish_hunting 0 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 1 monclient: found mon.c 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=2+}) -- 0x7f5ef0074150 con 0x7f5edc0062d0 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient: _finish_auth 0 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 10 monclient: _check_auth_tickets 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.394+0000 7f5ef4fb8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef4fb8640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0062d0 msgr2=0x7f5edc006fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0062d0 0x7f5edc006fb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0004380 tx=0x7f5eec00a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c15d21508e70cc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:58.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee001c070 con 0x7f5edc0062d0 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ee77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ee77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ee77fe640 10 monclient: dump: 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ee77fe640 1 -- 10.20.193.78:0/2843986065 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee0012810 con 0x7f5edc0062d0 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ee77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 10 monclient: get_monmap_and_config success 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 4 set_mon_vals no callback set 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 10 monclient: shutdown 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 -- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0062d0 msgr2=0x7f5edc006fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0062d0 0x7f5edc006fb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0004380 tx=0x7f5eec00a360 comp rx=0 tx=0).stop 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 -- 10.20.193.78:0/2843986065 shutdown_connections 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc00aa30 0x7f5edc00aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc009ec0 0x7f5edc00a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 --2- 10.20.193.78:0/2843986065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc0062d0 0x7f5edc006fb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 -- 10.20.193.78:0/2843986065 >> 10.20.193.78:0/2843986065 conn(0x7f5ef00a6620 msgr2=0x7f5ef00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 -- 10.20.193.78:0/2843986065 shutdown_connections 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.397+0000 7f5ef7243640 1 -- 10.20.193.78:0/2843986065 wait complete. 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient: build_initial_monmap 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient: monmap: 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:58:58.399 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:58:58.400 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:58.400 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:58:58.400 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:58:58.400 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:58:58.400 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 1 Processor -- start 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 1 -- start start 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient: init 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient: _reopen_session rank -1 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc0062d0 0x7f5ef01cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient(hunting): picked mon.noname-b con 0x7f5edc0062d0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc009ec0 0x7f5ef01cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient(hunting): picked mon.noname-a con 0x7f5edc009ec0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 0x7f5ef0120700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient(hunting): picked mon.noname-c con 0x7f5edc00aa30 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef00a0370 con 0x7f5edc009ec0 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef00a01f0 con 0x7f5edc0062d0 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient(hunting): start opening mon connection 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef00a04f0 con 0x7f5edc00aa30 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient(hunting): _renew_subs 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef7243640 10 monclient(hunting): authenticate will time out at 1383.952668s 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc009ec0 0x7f5ef01cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 0x7f5ef0120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc009ec0 0x7f5ef01cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:56738/0 (socket says 10.20.193.78:56738) 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 -- 10.20.193.78:0/1445841593 learned_addr learned my addr 10.20.193.78:0/1445841593 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request con 0x7f5edc009ec0 auth_method 0 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:58.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 10 monclient(hunting): get_auth_request con 0x7f5edc00aa30 auth_method 0 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 1 -- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 msgr2=0x7f5ef0120700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 1 -- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 msgr2=0x7f5ef0120700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 0x7f5ef0120700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef57b9640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 0x7f5ef0120700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient(hunting): handle_auth_done global_id 164100 payload 824 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc0062d0 0x7f5ef01cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 -- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 msgr2=0x7f5ef0120700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 0x7f5ef0120700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 -- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc0062d0 msgr2=0x7f5ef01cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc0062d0 0x7f5ef01cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 monclient: found mon.noname-a 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ee7fff640 1 -- 10.20.193.78:0/1445841593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef0124010 con 0x7f5edc009ec0 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef4fb8640 10 monclient: get_auth_request con 0x7f5edc0062d0 auth_method 0 2026-02-17T05:58:58.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.398+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc0062d0 0x7f5ef01cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.400+0000 7f5ee7fff640 1 -- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc009ec0 msgr2=0x7f5ef01cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee7fff640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc009ec0 0x7f5ef01cb700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8012600 tx=0x7f5ed8012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 1 -- 10.20.193.78:0/1445841593 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8019070 con 0x7f5edc009ec0 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: dump: 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: _finish_auth 0 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: _check_auth_tickets 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 1 -- 10.20.193.78:0/1445841593 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed8009950 con 0x7f5edc009ec0 2026-02-17T05:58:58.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 1 -- 10.20.193.78:0/1445841593 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8009d40 con 0x7f5edc009ec0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 10 monclient: dump: 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee6ffd640 4 set_mon_vals no callback set 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 5 monclient: authenticate success, global_id 164100 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 10 monclient: _renew_subs 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef01245d0 con 0x7f5edc009ec0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 10 monclient: _renew_subs 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef0123d30 con 0x7f5edc009ec0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 1 -- 10.20.193.78:0/1445841593 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5ed8004030 con 0x7f5edc009ec0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef0074980 con 0x7f5edc009ec0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ec003deb0 0x7f5ec0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.401+0000 7f5ee57fa640 1 -- 10.20.193.78:0/1445841593 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(460..460 src has 335..460) ==== 4763+0+0 (secure 0 0 0) 0x7f5ed8052e80 con 0x7f5edc009ec0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.402+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ec003deb0 0x7f5ec0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:58:58.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.402+0000 7f5ef4fb8640 10 monclient: get_auth_request con 0x7f5ec003deb0 auth_method 0 2026-02-17T05:58:58.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.404+0000 7f5ee57fa640 1 -- 10.20.193.78:0/1445841593 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5ed8051890 con 0x7f5edc009ec0 2026-02-17T05:58:58.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.404+0000 7f5ee57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:58:58.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.404+0000 7f5ee57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:58:58.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.404+0000 7f5ef4fb8640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ec003deb0 0x7f5ec0040370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0002410 tx=0x7f5ee001b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:58:58.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.522+0000 7f5ef7243640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:58:58.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.522+0000 7f5ef7243640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.522+0000 7f5ef7243640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:58:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.522+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5ef006d080 con 0x7f5edc009ec0 2026-02-17T05:58:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.523+0000 7f5ee57fa640 1 -- 10.20.193.78:0/1445841593 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f5ed801dde0 con 0x7f5edc009ec0 2026-02-17T05:58:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.523+0000 7f5ee57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:58:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.523+0000 7f5ee57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:58:58.525 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":146,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ec003deb0 msgr2=0x7f5ec0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ec003deb0 0x7f5ec0040370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0002410 tx=0x7f5ee001b000 comp rx=0 tx=0).stop 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 10 monclient: shutdown 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc009ec0 msgr2=0x7f5ef01cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc009ec0 0x7f5ef01cb700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8012600 tx=0x7f5ed8012ad0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 shutdown_connections 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5edc00aa30 0x7f5ef0120700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5edc009ec0 0x7f5ef01cb700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ec003deb0 0x7f5ec0040370 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 --2- 10.20.193.78:0/1445841593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5edc0062d0 0x7f5ef01cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:58:58.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 >> 10.20.193.78:0/1445841593 conn(0x7f5ef00a6620 msgr2=0x7f5ef009be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:58:58.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 shutdown_connections 2026-02-17T05:58:58.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:58:58.524+0000 7f5ef7243640 1 -- 10.20.193.78:0/1445841593 wait complete. 2026-02-17T05:58:59.243 INFO:tasks.rados.rados.0.trial115.stdout:2146: done (1 left) 2026-02-17T05:58:59.243 INFO:tasks.rados.rados.0.trial115.stdout:2147: done (0 left) 2026-02-17T05:58:59.243 INFO:tasks.rados.rados.0.trial115.stdout:2148: write (excl) oid 5 current snap is 217 2026-02-17T05:58:59.243 INFO:tasks.rados.rados.0.trial115.stdout:2148: seq_num 768 ranges {507776=742885,1991465=485600} 2026-02-17T05:58:59.247 INFO:tasks.rados.rados.0.trial115.stdout:2148: writing trial11545420-5 from 507776 to 1250661 tid 1 2026-02-17T05:58:59.252 INFO:tasks.rados.rados.0.trial115.stdout:2148: writing trial11545420-5 from 1991465 to 2477065 tid 2 2026-02-17T05:58:59.252 INFO:tasks.rados.rados.0.trial115.stdout:2149: read oid 28 snap -1 2026-02-17T05:58:59.252 INFO:tasks.rados.rados.0.trial115.stdout:2149: expect (ObjNum 747 snap 204 seq_num 747) 2026-02-17T05:58:59.253 INFO:tasks.rados.rados.0.trial115.stdout:2150: rollback oid 24 current snap is 217 2026-02-17T05:58:59.253 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 215 2026-02-17T05:58:59.253 INFO:tasks.rados.rados.0.trial115.stdout:2148: finishing write tid 1 to trial11545420-5 2026-02-17T05:58:59.253 INFO:tasks.rados.rados.0.trial115.stdout:2151: read oid 6 snap 215 2026-02-17T05:58:59.253 INFO:tasks.rados.rados.0.trial115.stdout:2151: expect (ObjNum 657 snap 178 seq_num 657) 2026-02-17T05:58:59.253 INFO:tasks.rados.rados.0.trial115.stdout:2152: snap_remove snap 214 2026-02-17T05:59:01.535 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:01.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.601+0000 7f60658d3640 10 monclient: get_monmap_and_config 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.601+0000 7f60658d3640 10 monclient: build_initial_monmap 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.601+0000 7f60658d3640 10 monclient: monmap: 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:01.602961+0000 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:01.602961+0000 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:01.603 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:01.604 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:01.604 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:01.604 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:01.604 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:01.604 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:01.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 1 Processor -- start 2026-02-17T05:59:01.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 1 -- start start 2026-02-17T05:59:01.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient: init 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient(hunting): picked mon.noname-c con 0x7f606006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f606009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f606006d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6060135c10 con 0x7f606009c4a0 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606006daf0 con 0x7f606006d1b0 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f60600a0880 con 0x7f606006c890 2026-02-17T05:59:01.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f60658d3640 10 monclient(hunting): authenticate will time out at 1387.156700s 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f605e7fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f606009c4a0 0x7f6060134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52270/0 (socket says 10.20.193.78:52270) 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f605e7fc640 1 -- 10.20.193.78:0/3673090682 learned_addr learned my addr 10.20.193.78:0/3673090682 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f605effd640 10 monclient(hunting): get_auth_request con 0x7f606006c890 auth_method 0 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f605effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f605effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.602+0000 7f605effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.603+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.603+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.603+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.603+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:01.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.604+0000 7f605e7fc640 1 -- 10.20.193.78:0/3673090682 >> v1:10.20.193.78:6789/0 conn(0x7f606009c4a0 legacy=0x7f6060134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:01.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.605+0000 7f605f7fe640 1 -- 10.20.193.78:0/3673090682 >> v1:10.20.193.115:6789/0 conn(0x7f606006d1b0 legacy=0x7f6060135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 10 monclient(hunting): get_auth_request con 0x7f606006c890 auth_method 0 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:01.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:01.804+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:02.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:02.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 10 monclient(hunting): get_auth_request con 0x7f606006c890 auth_method 0 2026-02-17T05:59:02.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:02.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:02.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:02.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:02.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:02.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:02.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:02.204+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:59:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 10 monclient(hunting): get_auth_request con 0x7f606006c890 auth_method 0 2026-02-17T05:59:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:03.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:03.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:03.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:03.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:03.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:03.006+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:59:04.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 814873265 0 0) 0x7f606006daf0 con 0x7f606006d1b0 2026-02-17T05:59:04.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): _negotiate 2026-02-17T05:59:04.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:04.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:04.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): my global_id is 134332 2026-02-17T05:59:04.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f604c003620 con 0x7f606006d1b0 2026-02-17T05:59:04.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1365894442 0 0) 0x7f604c003620 con 0x7f606006d1b0 2026-02-17T05:59:04.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 msgr2=0x7f606006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 --2- 10.20.193.78:0/3673090682 >> v2:10.20.193.115:3300/0 conn(0x7f606006c890 0x7f606006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 >> v1:10.20.193.78:6789/0 conn(0x7f606009c4a0 legacy=0x7f6060134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: _finish_hunting 0 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606009cc80 con 0x7f606006d1b0 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f6050003580 con 0x7f606006d1b0 2026-02-17T05:59:04.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: dump: 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:04.405 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 >> v1:10.20.193.115:6789/0 conn(0x7f606006d1b0 legacy=0x7f6060135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003c20 0x7f604c004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): picked mon.c con 0x7f604c003c20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c004900 0x7f604c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): picked mon.a con 0x7f604c004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c009ab0 0x7f604c00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): picked mon.b con 0x7f604c009ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:04.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f604c00c490 con 0x7f604c004900 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f604c00c640 con 0x7f604c009ab0 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f604c00c840 con 0x7f604c003c20 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605dffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003c20 0x7f604c004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient(hunting): get_auth_request con 0x7f604c003c20 auth_method 0 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:04.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c009ab0 0x7f604c00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 10 monclient(hunting): get_auth_request con 0x7f604c009ab0 auth_method 0 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient(hunting): handle_auth_done global_id 134332 payload 293 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c009ab0 msgr2=0x7f604c00bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f604c009ab0 0x7f604c00bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c004900 msgr2=0x7f604c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c004900 0x7f604c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 1 monclient: found mon.c 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605f7fe640 1 -- 10.20.193.78:0/3673090682 reap_dead start 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c003620 con 0x7f604c003c20 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient: _finish_auth 0 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f605effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f60658d3640 5 monclient: authenticate success, global_id 134332 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.392+0000 7f60658d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605effd640 1 -- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003c20 msgr2=0x7f604c004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605effd640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003c20 0x7f604c004040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60540129d0 tx=0x7f605402fc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cea67ee8ef219df server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054039070 con 0x7f604c003c20 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 10 monclient: dump: 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.410 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f60658d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6054031670 con 0x7f604c003c20 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 1 -- 10.20.193.78:0/3673090682 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605400f040 con 0x7f604c003c20 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f60658d3640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f60658d3640 4 set_mon_vals no callback set 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f605dffb640 10 monclient: dump: 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f60658d3640 10 monclient: shutdown 2026-02-17T05:59:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f60658d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f60658d3640 1 -- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003c20 msgr2=0x7f604c004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.395+0000 7f60658d3640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003c20 0x7f604c004040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60540129d0 tx=0x7f605402fc20 comp rx=0 tx=0).stop 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 1 -- 10.20.193.78:0/3673090682 shutdown_connections 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 1 --2- 10.20.193.78:0/3673090682 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f604c003c20 0x7f604c004040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 1 -- 10.20.193.78:0/3673090682 >> 10.20.193.78:0/3673090682 conn(0x7f60600a6620 msgr2=0x7f60600a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 1 -- 10.20.193.78:0/3673090682 shutdown_connections 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 1 -- 10.20.193.78:0/3673090682 wait complete. 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 10 monclient: build_initial_monmap 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 10 monclient: monmap: 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:04.413 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 1 Processor -- start 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 1 -- start start 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.396+0000 7f60658d3640 10 monclient: init 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c003c20 0x7f60601cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient(hunting): picked mon.noname-a con 0x7f604c003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 0x7f606009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient(hunting): picked mon.noname-c con 0x7f60601cf890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60601cfcf0 0x7f606009e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient(hunting): picked mon.noname-b con 0x7f60601cfcf0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60600a0020 con 0x7f604c003c20 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f606006d640 con 0x7f60601cfcf0 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f60600a01a0 con 0x7f60601cf890 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f60658d3640 10 monclient(hunting): authenticate will time out at 1389.951106s 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c003c20 0x7f60601cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 0x7f606009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c003c20 0x7f60601cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:56782/0 (socket says 10.20.193.78:56782) 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 10 monclient(hunting): get_auth_request con 0x7f60601cf890 auth_method 0 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 -- 10.20.193.78:0/433282846 learned_addr learned my addr 10.20.193.78:0/433282846 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient(hunting): get_auth_request con 0x7f604c003c20 auth_method 0 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 1 -- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 msgr2=0x7f606009db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 1 -- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 msgr2=0x7f606009db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 0x7f606009db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605e7fc640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 0x7f606009db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient(hunting): handle_auth_done global_id 164103 payload 824 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 -- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 msgr2=0x7f606009db50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 0x7f606009db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 -- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60601cfcf0 msgr2=0x7f606009e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60601cfcf0 0x7f606009e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 monclient: found mon.noname-a 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.397+0000 7f605effd640 1 -- 10.20.193.78:0/433282846 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606009d810 con 0x7f604c003c20 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f605effd640 1 -- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c003c20 msgr2=0x7f60601cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f605effd640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c003c20 0x7f60601cf350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6054030130 tx=0x7f60540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 1 -- 10.20.193.78:0/433282846 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054039070 con 0x7f604c003c20 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: dump: 2026-02-17T05:59:04.418 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f60658d3640 5 monclient: authenticate success, global_id 164103 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 1 -- 10.20.193.78:0/433282846 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605403e860 con 0x7f604c003c20 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 1 -- 10.20.193.78:0/433282846 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605403ec10 con 0x7f604c003c20 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f6037fff640 10 monclient: dump: 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:04.420 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f605d7fa640 4 set_mon_vals no callback set 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f60658d3640 10 monclient: _renew_subs 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f60658d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606009fba0 con 0x7f604c003c20 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f60658d3640 10 monclient: _renew_subs 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f60658d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.399+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60601270e0 con 0x7f604c003c20 2026-02-17T05:59:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f6037fff640 1 -- 10.20.193.78:0/433282846 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6054045020 con 0x7f604c003c20 2026-02-17T05:59:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f60658d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f60658d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f60658d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606009beb0 con 0x7f604c003c20 2026-02-17T05:59:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f6037fff640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003deb0 0x7f6030040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f6037fff640 1 -- 10.20.193.78:0/433282846 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(461..461 src has 335..461) ==== 4735+0+0 (secure 0 0 0) 0x7f6054077c00 con 0x7f604c003c20 2026-02-17T05:59:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f605e7fc640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003deb0 0x7f6030040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.400+0000 7f605e7fc640 10 monclient: get_auth_request con 0x7f603003deb0 auth_method 0 2026-02-17T05:59:04.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.403+0000 7f6037fff640 1 -- 10.20.193.78:0/433282846 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6054076630 con 0x7f604c003c20 2026-02-17T05:59:04.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.403+0000 7f6037fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:04.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.403+0000 7f6037fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:04.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.403+0000 7f605e7fc640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003deb0 0x7f6030040370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6060051430 tx=0x7f6048009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:04.449 INFO:tasks.rados.rados.0.trial115.stdout:2152: done (4 left) 2026-02-17T05:59:04.449 INFO:tasks.rados.rados.0.trial115.stdout:2153: snap_create 2026-02-17T05:59:04.449 INFO:tasks.rados.rados.0.trial115.stdout:2149: expect (ObjNum 747 snap 204 seq_num 747) 2026-02-17T05:59:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.514+0000 7f60658d3640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.514+0000 7f60658d3640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.514+0000 7f60658d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.514+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f60600a20c0 con 0x7f604c003c20 2026-02-17T05:59:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.514+0000 7f6037fff640 1 -- 10.20.193.78:0/433282846 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f605403edb0 con 0x7f604c003c20 2026-02-17T05:59:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.514+0000 7f6037fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.514+0000 7f6037fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:04.517 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":150,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003deb0 msgr2=0x7f6030040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003deb0 0x7f6030040370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6060051430 tx=0x7f6048009040 comp rx=0 tx=0).stop 2026-02-17T05:59:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 10 monclient: shutdown 2026-02-17T05:59:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c003c20 msgr2=0x7f60601cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c003c20 0x7f60601cf350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6054030130 tx=0x7f60540047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 shutdown_connections 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003deb0 0x7f6030040370 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f60601cfcf0 0x7f606009e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f60601cf890 0x7f606009db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 --2- 10.20.193.78:0/433282846 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f604c003c20 0x7f60601cf350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 >> 10.20.193.78:0/433282846 conn(0x7f60600a6620 msgr2=0x7f6060072d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 shutdown_connections 2026-02-17T05:59:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:04.515+0000 7f60658d3640 1 -- 10.20.193.78:0/433282846 wait complete. 2026-02-17T05:59:04.530 INFO:tasks.rados.rados.0.trial115.stdout:2148: finishing write tid 2 to trial11545420-5 2026-02-17T05:59:04.531 INFO:tasks.rados.rados.0.trial115.stdout:2150: finishing rollback tid 0 to trial11545420-24 2026-02-17T05:59:04.531 INFO:tasks.rados.rados.0.trial115.stdout:2151: expect (ObjNum 657 snap 178 seq_num 657) 2026-02-17T05:59:04.628 INFO:tasks.rados.rados.0.trial115.stdout:2148: finishing write tid 3 to trial11545420-5 2026-02-17T05:59:04.628 INFO:tasks.rados.rados.0.trial115.stdout:2148: finishing write tid 4 to trial11545420-5 2026-02-17T05:59:04.628 INFO:tasks.rados.rados.0.trial115.stdout:2148: oid 5 updating version 0 to 1047 2026-02-17T05:59:04.628 INFO:tasks.rados.rados.0.trial115.stdout:2148: oid 5 version 1047 is already newer than 1046 2026-02-17T05:59:04.628 INFO:tasks.rados.rados.0.trial115.stdout:2148: oid 5 version 1047 is already newer than 1045 2026-02-17T05:59:04.628 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1047 (ObjNum 768 snap 217 seq_num 768) dirty exists 2026-02-17T05:59:04.628 INFO:tasks.rados.rados.0.trial115.stdout:2148: left oid 5 (ObjNum 768 snap 217 seq_num 768) 2026-02-17T05:59:04.628 INFO:tasks.rados.rados.0.trial115.stdout:2150: finishing rollback tid 1 to trial11545420-24 2026-02-17T05:59:04.629 INFO:tasks.rados.rados.0.trial115.stdout:2150: finishing rollback tid 2 to trial11545420-24 2026-02-17T05:59:04.629 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1271 (ObjNum 730 snap 202 seq_num 730) dirty exists 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2148: done (4 left) 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2149: done (3 left) 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2150: done (2 left) 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2151: done (1 left) 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2153: done (0 left) 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2154: copy_from oid 11 from oid 39 current snap is 218 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2155: read oid 37 snap -1 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2155: expect deleted 2026-02-17T05:59:06.646 INFO:tasks.rados.rados.0.trial115.stdout:2156: write (excl) oid 15 current snap is 218 2026-02-17T05:59:06.647 INFO:tasks.rados.rados.0.trial115.stdout:2156: seq_num 769 ranges {524583=418134,1649947=629258,2834752=89831} 2026-02-17T05:59:06.649 INFO:tasks.rados.rados.0.trial115.stdout:2156: writing trial11545420-15 from 524583 to 942717 tid 1 2026-02-17T05:59:06.654 INFO:tasks.rados.rados.0.trial115.stdout:2156: writing trial11545420-15 from 1649947 to 2279205 tid 2 2026-02-17T05:59:06.657 INFO:tasks.rados.rados.0.trial115.stdout:2156: writing trial11545420-15 from 2834752 to 2924583 tid 3 2026-02-17T05:59:06.657 INFO:tasks.rados.rados.0.trial115.stdout:2157: read oid 5 snap 216 2026-02-17T05:59:06.657 INFO:tasks.rados.rados.0.trial115.stdout:2157: expect (ObjNum 744 snap 204 seq_num 744) 2026-02-17T05:59:06.657 INFO:tasks.rados.rados.0.trial115.stdout:2158: write (excl) oid 33 current snap is 218 2026-02-17T05:59:06.657 INFO:tasks.rados.rados.0.trial115.stdout:2158: seq_num 770 ranges {541390=493383,1592076=489269,2862175=79215} 2026-02-17T05:59:06.660 INFO:tasks.rados.rados.0.trial115.stdout:2158: writing trial11545420-33 from 541390 to 1034773 tid 1 2026-02-17T05:59:06.664 INFO:tasks.rados.rados.0.trial115.stdout:2158: writing trial11545420-33 from 1592076 to 2081345 tid 2 2026-02-17T05:59:06.668 INFO:tasks.rados.rados.0.trial115.stdout:2158: writing trial11545420-33 from 2862175 to 2941390 tid 3 2026-02-17T05:59:06.668 INFO:tasks.rados.rados.0.trial115.stdout:2159: copy_from oid 41 from oid 22 current snap is 218 2026-02-17T05:59:06.668 INFO:tasks.rados.rados.0.trial115.stdout:2154: finishing copy_from to trial11545420-11 2026-02-17T05:59:06.668 INFO:tasks.rados.rados.0.trial115.stdout:2154: got expected ENOENT (src dne) 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2154: finishing copy_from racing read to trial11545420-11 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2156: finishing write tid 1 to trial11545420-15 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2154: done (5 left) 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2160: read oid 2 snap -1 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2160: expect (ObjNum 748 snap 205 seq_num 748) 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2161: read oid 48 snap -1 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2161: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2162: snap_create 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2156: finishing write tid 2 to trial11545420-15 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2156: finishing write tid 3 to trial11545420-15 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2156: finishing write tid 4 to trial11545420-15 2026-02-17T05:59:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2156: finishing write tid 5 to trial11545420-15 2026-02-17T05:59:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2156: oid 15 updating version 0 to 1122 2026-02-17T05:59:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2156: oid 15 updating version 1122 to 1123 2026-02-17T05:59:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2156: oid 15 version 1123 is already newer than 1120 2026-02-17T05:59:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2156: oid 15 version 1123 is already newer than 1121 2026-02-17T05:59:06.670 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1123 (ObjNum 769 snap 218 seq_num 769) dirty exists 2026-02-17T05:59:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2156: left oid 15 (ObjNum 769 snap 218 seq_num 769) 2026-02-17T05:59:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2157: expect (ObjNum 744 snap 204 seq_num 744) 2026-02-17T05:59:06.749 INFO:tasks.rados.rados.0.trial115.stdout:2158: finishing write tid 1 to trial11545420-33 2026-02-17T05:59:06.749 INFO:tasks.rados.rados.0.trial115.stdout:2158: finishing write tid 2 to trial11545420-33 2026-02-17T05:59:06.749 INFO:tasks.rados.rados.0.trial115.stdout:2158: finishing write tid 3 to trial11545420-33 2026-02-17T05:59:06.749 INFO:tasks.rados.rados.0.trial115.stdout:2158: finishing write tid 4 to trial11545420-33 2026-02-17T05:59:06.749 INFO:tasks.rados.rados.0.trial115.stdout:2158: finishing write tid 5 to trial11545420-33 2026-02-17T05:59:06.749 INFO:tasks.rados.rados.0.trial115.stdout:2158: oid 33 updating version 0 to 938 2026-02-17T05:59:06.749 INFO:tasks.rados.rados.0.trial115.stdout:2158: oid 33 updating version 938 to 941 2026-02-17T05:59:06.750 INFO:tasks.rados.rados.0.trial115.stdout:2158: oid 33 version 941 is already newer than 939 2026-02-17T05:59:06.750 INFO:tasks.rados.rados.0.trial115.stdout:2158: oid 33 version 941 is already newer than 940 2026-02-17T05:59:06.750 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 941 (ObjNum 770 snap 218 seq_num 770) dirty exists 2026-02-17T05:59:06.750 INFO:tasks.rados.rados.0.trial115.stdout:2158: left oid 33 (ObjNum 770 snap 218 seq_num 770) 2026-02-17T05:59:06.750 INFO:tasks.rados.rados.0.trial115.stdout:2161: expect (ObjNum 464 snap 117 seq_num 464) 2026-02-17T05:59:06.874 INFO:tasks.rados.rados.0.trial115.stdout:2160: expect (ObjNum 748 snap 205 seq_num 748) 2026-02-17T05:59:06.982 INFO:tasks.rados.rados.0.trial115.stdout:2159: finishing copy_from racing read to trial11545420-41 2026-02-17T05:59:06.982 INFO:tasks.rados.rados.0.trial115.stdout:2159: finishing copy_from to trial11545420-41 2026-02-17T05:59:06.982 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 946 (ObjNum 762 snap 210 seq_num 762) dirty exists 2026-02-17T05:59:07.526 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:07.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.594+0000 7f333c37e640 10 monclient: get_monmap_and_config 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.594+0000 7f333c37e640 10 monclient: build_initial_monmap 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.594+0000 7f333c37e640 10 monclient: monmap: 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:07.595757+0000 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:07.595757+0000 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:07.607 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.594+0000 7f333c37e640 1 Processor -- start 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.594+0000 7f333c37e640 1 -- start start 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.594+0000 7f333c37e640 10 monclient: init 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333406c890 0x7f333406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient(hunting): picked mon.noname-b con 0x7f333406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f333409d430 0x7f333406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient(hunting): picked mon.noname-c con 0x7f333409d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f333409bf40 0x7f3334134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient(hunting): picked mon.noname-a con 0x7f333409bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f333409ca50 con 0x7f333409bf40 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f333406d6f0 con 0x7f333409d430 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3334073600 con 0x7f333406c890 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333c37e640 10 monclient(hunting): authenticate will time out at 1393.149266s 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333406c890 0x7f333406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f333409bf40 0x7f3334134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333406c890 0x7f333406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58074/0 (socket says 10.20.193.78:58074) 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 1 -- 10.20.193.78:0/871687426 learned_addr learned my addr 10.20.193.78:0/871687426 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 10 monclient(hunting): get_auth_request con 0x7f333406c890 auth_method 0 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient(hunting): get_auth_request con 0x7f333409bf40 auth_method 0 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f33398f2640 1 --2- 10.20.193.78:0/871687426 >> v2:10.20.193.115:3300/0 conn(0x7f333409d430 0x7f333406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a0f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f33398f2640 10 monclient(hunting): get_auth_request con 0x7f333409d430 auth_method 0 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f33398f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f33398f2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f33398f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient(hunting): handle_auth_done global_id 164106 payload 274 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 1 -- 10.20.193.78:0/871687426 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333406c890 msgr2=0x7f333406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 1 --2- 10.20.193.78:0/871687426 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333406c890 0x7f333406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 1 -- 10.20.193.78:0/871687426 >> v2:10.20.193.115:3300/0 conn(0x7f333409d430 msgr2=0x7f333406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 1 --2- 10.20.193.78:0/871687426 >> v2:10.20.193.115:3300/0 conn(0x7f333409d430 0x7f333406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 1 monclient: found mon.noname-a 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.595+0000 7f333a8f4640 1 -- 10.20.193.78:0/871687426 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333406dab0 con 0x7f333409bf40 2026-02-17T05:59:07.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333a8f4640 1 -- 10.20.193.78:0/871687426 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333409bf40 msgr2=0x7f3334134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333a8f4640 1 --2- 10.20.193.78:0/871687426 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333409bf40 0x7f3334134df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f33340a7600 tx=0x7f333002f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3871cf4e8e337b2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 1 -- 10.20.193.78:0/871687426 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333003c070 con 0x7f333409bf40 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: dump: 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.613 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 5 monclient: authenticate success, global_id 164106 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 1 -- 10.20.193.78:0/871687426 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3330037440 con 0x7f333409bf40 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 1 -- 10.20.193.78:0/871687426 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330037810 con 0x7f333409bf40 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:07.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 4 set_mon_vals no callback set 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f33390f1640 10 monclient: dump: 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.615 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 10 monclient: shutdown 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 -- 10.20.193.78:0/871687426 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333409bf40 msgr2=0x7f3334134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 --2- 10.20.193.78:0/871687426 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333409bf40 0x7f3334134df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f33340a7600 tx=0x7f333002f130 comp rx=0 tx=0).stop 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 -- 10.20.193.78:0/871687426 shutdown_connections 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 --2- 10.20.193.78:0/871687426 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333409bf40 0x7f3334134df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 --2- 10.20.193.78:0/871687426 >> v2:10.20.193.115:3300/0 conn(0x7f333409d430 0x7f333406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 --2- 10.20.193.78:0/871687426 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333406c890 0x7f333406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 -- 10.20.193.78:0/871687426 >> 10.20.193.78:0/871687426 conn(0x7f33340a6620 msgr2=0x7f33340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 -- 10.20.193.78:0/871687426 shutdown_connections 2026-02-17T05:59:07.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 1 -- 10.20.193.78:0/871687426 wait complete. 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 10 monclient: build_initial_monmap 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.598+0000 7f333c37e640 10 monclient: monmap: 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:07.617 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 1 Processor -- start 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 1 -- start start 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient: init 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333406c890 0x7f3334121750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient(hunting): picked mon.noname-a con 0x7f333406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f333409bf40 0x7f3334121c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient(hunting): picked mon.noname-b con 0x7f333409bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333409d430 0x7f33341221d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient(hunting): picked mon.noname-c con 0x7f333409d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3334073380 con 0x7f333406c890 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3334073200 con 0x7f333409bf40 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3334135330 con 0x7f333409d430 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333c37e640 10 monclient(hunting): authenticate will time out at 1393.153415s 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333406c890 0x7f3334121750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333406c890 0x7f3334121750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45814/0 (socket says 10.20.193.78:45814) 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 -- 10.20.193.78:0/3383663903 learned_addr learned my addr 10.20.193.78:0/3383663903 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:07.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient(hunting): get_auth_request con 0x7f333406c890 auth_method 0 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a8f4640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333409d430 0x7f33341221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a8f4640 10 monclient(hunting): get_auth_request con 0x7f333409d430 auth_method 0 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a8f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a8f4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a8f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a8f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a8f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a8f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient(hunting): handle_auth_done global_id 164109 payload 824 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 -- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333409d430 msgr2=0x7f33341221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333409d430 0x7f33341221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 -- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f333409bf40 msgr2=0x7f3334121c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f333409bf40 0x7f3334121c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 monclient: found mon.noname-a 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.599+0000 7f333a0f3640 1 -- 10.20.193.78:0/3383663903 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3334122fc0 con 0x7f333406c890 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333a0f3640 1 -- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333406c890 msgr2=0x7f3334121750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333a0f3640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333406c890 0x7f3334121750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f332400e930 tx=0x7f332400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 1 -- 10.20.193.78:0/3383663903 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324017070 con 0x7f333406c890 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: dump: 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.622 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 1 -- 10.20.193.78:0/3383663903 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332400cbb0 con 0x7f333406c890 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333c37e640 5 monclient: authenticate success, global_id 164109 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333c37e640 10 monclient: _renew_subs 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333c37e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3334125320 con 0x7f333406c890 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 1 -- 10.20.193.78:0/3383663903 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324013470 con 0x7f333406c890 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f33388f0640 4 set_mon_vals no callback set 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f331affd640 10 monclient: dump: 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.624 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333c37e640 10 monclient: _renew_subs 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333c37e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.601+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3334125040 con 0x7f333406c890 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.602+0000 7f333c37e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.602+0000 7f333c37e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.602+0000 7f333c37e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.602+0000 7f331affd640 1 -- 10.20.193.78:0/3383663903 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3324024060 con 0x7f333406c890 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.602+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3334074980 con 0x7f333406c890 2026-02-17T05:59:07.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.602+0000 7f331affd640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f330403deb0 0x7f3304040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.602+0000 7f331affd640 1 -- 10.20.193.78:0/3383663903 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(463..463 src has 335..463) ==== 4751+0+0 (secure 0 0 0) 0x7f3324052870 con 0x7f333406c890 2026-02-17T05:59:07.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.603+0000 7f33398f2640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f330403deb0 0x7f3304040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.603+0000 7f33398f2640 10 monclient: get_auth_request con 0x7f330403deb0 auth_method 0 2026-02-17T05:59:07.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.605+0000 7f331affd640 1 -- 10.20.193.78:0/3383663903 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3324051290 con 0x7f333406c890 2026-02-17T05:59:07.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.605+0000 7f331affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.605+0000 7f331affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:07.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.605+0000 7f33398f2640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f330403deb0 0x7f3304040370 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3328006fd0 tx=0x7f3328008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.647 INFO:tasks.rados.rados.0.trial115.stdout:2155: done (7 left) 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2156: done (6 left) 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2157: done (5 left) 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2158: done (4 left) 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2159: done (3 left) 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2160: done (2 left) 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2161: done (1 left) 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2162: done (0 left) 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2163: write (excl) oid 48 current snap is 219 2026-02-17T05:59:07.648 INFO:tasks.rados.rados.0.trial115.stdout:2163: seq_num 771 ranges {558197=568632,1650558=632927,2773245=184952} 2026-02-17T05:59:07.652 INFO:tasks.rados.rados.0.trial115.stdout:2163: writing trial11545420-48 from 558197 to 1126829 tid 1 2026-02-17T05:59:07.656 INFO:tasks.rados.rados.0.trial115.stdout:2163: writing trial11545420-48 from 1650558 to 2283485 tid 2 2026-02-17T05:59:07.659 INFO:tasks.rados.rados.0.trial115.stdout:2163: writing trial11545420-48 from 2773245 to 2958197 tid 3 2026-02-17T05:59:07.659 INFO:tasks.rados.rados.0.trial115.stdout:2164: delete oid 17 current snap is 219 2026-02-17T05:59:07.659 INFO:tasks.rados.rados.0.trial115.stdout:2163: finishing write tid 1 to trial11545420-48 2026-02-17T05:59:07.659 INFO:tasks.rados.rados.0.trial115.stdout:2163: finishing write tid 2 to trial11545420-48 2026-02-17T05:59:07.660 INFO:tasks.rados.rados.0.trial115.stdout:2164: done (1 left) 2026-02-17T05:59:07.660 INFO:tasks.rados.rados.0.trial115.stdout:2165: copy_from oid 46 from oid 41 current snap is 219 2026-02-17T05:59:07.660 INFO:tasks.rados.rados.0.trial115.stdout:2166: snap_remove snap 218 2026-02-17T05:59:07.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.718+0000 7f333c37e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:07.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.718+0000 7f333c37e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:07.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.718+0000 7f333c37e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.718+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f33340008d0 con 0x7f333406c890 2026-02-17T05:59:07.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.718+0000 7f331affd640 1 -- 10.20.193.78:0/3383663903 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f332401b3b0 con 0x7f333406c890 2026-02-17T05:59:07.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.718+0000 7f331affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:07.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.718+0000 7f331affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:07.721 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":154,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f330403deb0 msgr2=0x7f3304040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f330403deb0 0x7f3304040370 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3328006fd0 tx=0x7f3328008040 comp rx=0 tx=0).stop 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 10 monclient: shutdown 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333406c890 msgr2=0x7f3334121750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333406c890 0x7f3334121750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f332400e930 tx=0x7f332400ee00 comp rx=0 tx=0).stop 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 shutdown_connections 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f333409d430 0x7f33341221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f330403deb0 0x7f3304040370 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f333409bf40 0x7f3334121c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 --2- 10.20.193.78:0/3383663903 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f333406c890 0x7f3334121750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 >> 10.20.193.78:0/3383663903 conn(0x7f33340a6620 msgr2=0x7f3334136aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 shutdown_connections 2026-02-17T05:59:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.719+0000 7f333c37e640 1 -- 10.20.193.78:0/3383663903 wait complete. 2026-02-17T05:59:07.729 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:59:07.729 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:59:07.729 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.796+0000 7f2f3dd5b640 10 monclient: get_monmap_and_config 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.796+0000 7f2f3dd5b640 10 monclient: build_initial_monmap 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.796+0000 7f2f3dd5b640 10 monclient: monmap: 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:07.797854+0000 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:07.797854+0000 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:07.797 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:07.798 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 1 Processor -- start 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 1 -- start start 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient: init 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient: _add_conns ranks=[0,2,4,3,1] 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2f3806c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3809c4a0 0x7f2f3813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient(hunting): picked mon.noname-b con 0x7f2f3809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2f3806d380 0x7f2f3813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient(hunting): picked mon.noname-c con 0x7f2f3806d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f3806daf0 con 0x7f2f3806c890 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2f380a0880 con 0x7f2f3806d380 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3809cc80 con 0x7f2f3809c4a0 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f3dd5b640 10 monclient(hunting): authenticate will time out at 1393.351433s 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f377fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f2f3806c890 0x7f2f3806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54560/0 (socket says 10.20.193.78:54560) 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f377fe640 1 -- 10.20.193.78:0/569408474 learned_addr learned my addr 10.20.193.78:0/569408474 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/569408474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3809c4a0 0x7f2f3813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f377fe640 1 -- 10.20.193.78:0/569408474 >> v1:10.20.193.78:6789/0 conn(0x7f2f3806c890 legacy=0x7f2f3806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f377fe640 1 -- 10.20.193.78:0/569408474 >> v1:10.20.193.78:6789/0 conn(0x7f2f3806c890 legacy=0x7f2f3806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f377fe640 1 --1- 10.20.193.78:0/569408474 >> v1:10.20.193.78:6789/0 conn(0x7f2f3806c890 0x7f2f3806cc70 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient(hunting): get_auth_request con 0x7f2f3809c4a0 auth_method 0 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f37fff640 1 --2- 10.20.193.78:0/569408474 >> v2:10.20.193.115:3300/0 conn(0x7f2f3806d380 0x7f2f3813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_done global_id 144278 payload 274 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 1 -- 10.20.193.78:0/569408474 >> v2:10.20.193.115:3300/0 conn(0x7f2f3806d380 msgr2=0x7f2f3813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/569408474 >> v2:10.20.193.115:3300/0 conn(0x7f2f3806d380 0x7f2f3813d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 1 -- 10.20.193.78:0/569408474 >> v1:10.20.193.78:6789/0 conn(0x7f2f3806c890 legacy=0x7f2f3806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 1 monclient: found mon.noname-b 2026-02-17T05:59:07.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:07.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.797+0000 7f2f36ffd640 1 -- 10.20.193.78:0/569408474 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3809cf60 con 0x7f2f3809c4a0 2026-02-17T05:59:07.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f36ffd640 1 -- 10.20.193.78:0/569408474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3809c4a0 msgr2=0x7f2f3813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/569408474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3809c4a0 0x7f2f3813b3a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f280098a0 tx=0x7f2f2802ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6bb4721278b7cd0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 1 -- 10.20.193.78:0/569408474 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2803c070 con 0x7f2f3809c4a0 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: dump: 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 5 monclient: authenticate success, global_id 144278 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 1 -- 10.20.193.78:0/569408474 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2802fce0 con 0x7f2f3809c4a0 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 1 -- 10.20.193.78:0/569408474 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28037660 con 0x7f2f3809c4a0 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f367fc640 10 monclient: dump: 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.801 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 4 set_mon_vals no callback set 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 10 monclient: shutdown 2026-02-17T05:59:07.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/569408474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3809c4a0 msgr2=0x7f2f3813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 1 --2- 10.20.193.78:0/569408474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3809c4a0 0x7f2f3813b3a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f280098a0 tx=0x7f2f2802ed80 comp rx=0 tx=0).stop 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/569408474 shutdown_connections 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 1 --2- 10.20.193.78:0/569408474 >> v2:10.20.193.115:3300/0 conn(0x7f2f3806d380 0x7f2f3813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 1 --2- 10.20.193.78:0/569408474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3809c4a0 0x7f2f3813b3a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/569408474 >> 10.20.193.78:0/569408474 conn(0x7f2f380a6620 msgr2=0x7f2f380a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/569408474 shutdown_connections 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.800+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/569408474 wait complete. 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient: build_initial_monmap 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient: monmap: 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:07.803 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 1 Processor -- start 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 1 -- start start 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient: init 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 0x7f2f38136080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient(hunting): picked mon.noname-a con 0x7f2f3806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 0x7f2f381365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient(hunting): picked mon.noname-c con 0x7f2f3806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 0x7f2f3811eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient(hunting): picked mon.noname-b con 0x7f2f3809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3809bfa0 con 0x7f2f3806c890 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3809be20 con 0x7f2f3809c4a0 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3813de30 con 0x7f2f3806d380 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f3dd5b640 10 monclient(hunting): authenticate will time out at 1393.355669s 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 0x7f2f381365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f377fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 0x7f2f38136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 0x7f2f381365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58090/0 (socket says 10.20.193.78:58090) 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 learned_addr learned my addr 10.20.193.78:0/1559255448 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient(hunting): get_auth_request con 0x7f2f3806d380 auth_method 0 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f377fe640 10 monclient(hunting): get_auth_request con 0x7f2f3806c890 auth_method 0 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f377fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_done global_id 144281 payload 824 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f37fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 0x7f2f3811eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 msgr2=0x7f2f3811eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 0x7f2f3811eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 msgr2=0x7f2f38136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 0x7f2f38136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 monclient: found mon.noname-c 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f38122490 con 0x7f2f3806d380 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f37fff640 10 monclient: get_auth_request con 0x7f2f3809c4a0 auth_method 0 2026-02-17T05:59:07.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.801+0000 7f2f37fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 0x7f2f3811eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.803+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 msgr2=0x7f2f381365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.803+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 0x7f2f381365c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f280099d0 tx=0x7f2f280047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28045020 con 0x7f2f3806d380 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: dump: 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2803d040 con 0x7f2f3806d380 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2803c070 con 0x7f2f3806d380 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: dump: 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f35ffb640 4 set_mon_vals no callback set 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 5 monclient: authenticate success, global_id 144281 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 10 monclient: _renew_subs 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f381212c0 con 0x7f2f3806d380 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 10 monclient: _renew_subs 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3806f7a0 con 0x7f2f3806d380 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 msgr2=0x7f2f381365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 msgr2=0x7f2f381365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 0x7f2f381365c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f280099d0 tx=0x7f2f280047c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 0x7f2f381365c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f280099d0 tx=0x7f2f280047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: ms_handle_reset current mon [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 msgr2=0x7f2f381365c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f3806d380 0x7f2f381365c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 0x7f2f08005970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient(hunting): picked mon.b con 0x7f2f3809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:07.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 0x7f2f08008320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient(hunting): picked mon.a con 0x7f2f3806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f08008d30 0x7f2f0800b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient(hunting): picked mon.c con 0x7f2f08008d30 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0800bc70 con 0x7f2f3806c890 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0800bdf0 con 0x7f2f3809c4a0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0800bfa0 con 0x7f2f08008d30 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f13fff640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 0x7f2f08008320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient(hunting): get_auth_request con 0x7f2f3806c890 auth_method 0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f37fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f08008d30 0x7f2f0800b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f37fff640 10 monclient(hunting): get_auth_request con 0x7f2f08008d30 auth_method 0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f37fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f37fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f377fe640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 0x7f2f08005970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f377fe640 10 monclient(hunting): get_auth_request con 0x7f2f3809c4a0 auth_method 0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f377fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f377fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient(hunting): handle_auth_done global_id 144281 payload 293 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f08008d30 msgr2=0x7f2f0800b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f08008d30 0x7f2f0800b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 msgr2=0x7f2f08005970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f3809c4a0 0x7f2f08005970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 monclient: found mon.a 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7f2f0800c2d0 con 0x7f2f3806c890 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f36ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f37fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f08008d30 0x7f2f0800b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381cf190 0x7f2f38136080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.804+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38120760 con 0x7f2f381cf190 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f36ffd640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 msgr2=0x7f2f08008320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f36ffd640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 0x7f2f08008320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f3809d8e0 tx=0x7f2f28033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f377fe640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381cf190 0x7f2f38136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f377fe640 10 monclient: get_auth_request con 0x7f2f381cf190 auth_method 0 2026-02-17T05:59:07.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f377fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28003040 con 0x7f2f3806c890 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f13fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f13fff640 10 monclient: dump: 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2802fce0 con 0x7f2f3806c890 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f13fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.806+0000 7f2f357fa640 4 set_mon_vals no callback set 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.807+0000 7f2f377fe640 10 monclient(hunting): handle_auth_done global_id 164115 payload 824 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.807+0000 7f2f377fe640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f38074980 con 0x7f2f381cf190 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.807+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2f280031e0 con 0x7f2f3806c890 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.807+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2f3806d380 0x7f2f080484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.807+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.0 v2:10.20.193.78:3300/0 4 ==== osd_map(464..464 src has 335..464) ==== 4751+0+0 (secure 0 0 0) 0x7f2f28079ae0 con 0x7f2f3806c890 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.807+0000 7f2f37fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2f3806d380 0x7f2f080484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.807+0000 7f2f37fff640 10 monclient: get_auth_request con 0x7f2f3806d380 auth_method 0 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f377fe640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381cf190 msgr2=0x7f2f38136080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f377fe640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381cf190 0x7f2f38136080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f24010780 tx=0x7f2f24010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2401a020 con 0x7f2f381cf190 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 10 monclient: dump: 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2f2400ecd0 con 0x7f2f381cf190 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381cf190 msgr2=0x7f2f38136080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381cf190 0x7f2f38136080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f24010780 tx=0x7f2f24010c50 comp rx=0 tx=0).stop 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381cf190 msgr2=0x7f2f38136080 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.809+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381cf190 0x7f2f38136080 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f24010780 tx=0x7f2f24010c50 comp rx=0 tx=0).stop 2026-02-17T05:59:07.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.810+0000 7f2f37fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2f3806d380 0x7f2f080484d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00f450 tx=0x7f2f2c0079f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f3dd5b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f3dd5b640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381365c0 0x7f2f381d4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f3dd5b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3806cc70 con 0x7f2f381365c0 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381365c0 0x7f2f381d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 10 monclient: get_auth_request con 0x7f2f381365c0 auth_method 0 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 10 monclient(hunting): handle_auth_done global_id 164118 payload 824 2026-02-17T05:59:07.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.812+0000 7f2f377fe640 1 -- 10.20.193.78:0/1559255448 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2f38121040 con 0x7f2f381365c0 2026-02-17T05:59:07.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f377fe640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381365c0 msgr2=0x7f2f381d4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f377fe640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381365c0 0x7f2f381d4240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f24010780 tx=0x7f2f24004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24027070 con 0x7f2f381365c0 2026-02-17T05:59:07.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 10 monclient: dump: 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f2f24003230 con 0x7f2f381365c0 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381365c0 msgr2=0x7f2f381d4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381365c0 0x7f2f381d4240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f24010780 tx=0x7f2f24004a70 comp rx=0 tx=0).stop 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381365c0 msgr2=0x7f2f381d4240 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f13fff640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f381365c0 0x7f2f381d4240 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f24010780 tx=0x7f2f24004a70 comp rx=0 tx=0).stop 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:59:07.816 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 154, 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 26665, 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.817 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.818 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:07.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2f3806d380 msgr2=0x7f2f080484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2f3806d380 0x7f2f080484d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00f450 tx=0x7f2f2c0079f0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 10 monclient: shutdown 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 msgr2=0x7f2f08008320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 0x7f2f08008320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f3809d8e0 tx=0x7f2f28033000 comp rx=0 tx=0).stop 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f37fff640 1 -- 10.20.193.78:0/1559255448 reap_dead start 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 shutdown_connections 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 --2- 10.20.193.78:0/1559255448 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f3806c890 0x7f2f08008320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 >> 10.20.193.78:0/1559255448 conn(0x7f2f380a6620 msgr2=0x7f2f3813b890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 shutdown_connections 2026-02-17T05:59:07.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.815+0000 7f2f3dd5b640 1 -- 10.20.193.78:0/1559255448 wait complete. 2026-02-17T05:59:07.823 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:07.824 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:59:07.824 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.890+0000 7efcbba38640 10 monclient: get_monmap_and_config 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.890+0000 7efcbba38640 10 monclient: build_initial_monmap 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.890+0000 7efcbba38640 10 monclient: monmap: 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:07.892132+0000 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:07.892132+0000 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:07.892 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 1 Processor -- start 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 1 -- start start 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient: init 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efcb406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efcb406d380 0x7efcb40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient(hunting): picked mon.noname-a con 0x7efcb406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efcb413a240 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:07.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb4074150 con 0x7efcb413a240 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb409ef70 con 0x7efcb406c890 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7efcb409f250 con 0x7efcb406d380 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcbba38640 10 monclient(hunting): authenticate will time out at 1393.445781s 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efcb406d380 0x7efcb40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efcb406d380 0x7efcb40a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45844/0 (socket says 10.20.193.78:45844) 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 1 -- 10.20.193.78:0/3785804869 learned_addr learned my addr 10.20.193.78:0/3785804869 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 10 monclient(hunting): get_auth_request con 0x7efcb406d380 auth_method 0 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.891+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.893+0000 7efcb9fae640 1 -- 10.20.193.78:0/3785804869 >> v1:10.20.193.78:6789/0 conn(0x7efcb413a240 legacy=0x7efcb413c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.894+0000 7efcb8fac640 10 monclient(hunting): handle_auth_done global_id 164121 payload 274 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.894+0000 7efcb8fac640 1 -- 10.20.193.78:0/3785804869 >> v1:10.20.193.115:6789/0 conn(0x7efcb406c890 legacy=0x7efcb406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.894+0000 7efcb8fac640 1 -- 10.20.193.78:0/3785804869 >> v1:10.20.193.78:6789/0 conn(0x7efcb413a240 legacy=0x7efcb413c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.894+0000 7efcb8fac640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.894+0000 7efcb8fac640 1 monclient: found mon.noname-a 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.894+0000 7efcb8fac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.894+0000 7efcb8fac640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.894+0000 7efcb8fac640 1 -- 10.20.193.78:0/3785804869 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb409df40 con 0x7efcb406d380 2026-02-17T05:59:07.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efcb8fac640 1 -- 10.20.193.78:0/3785804869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406d380 msgr2=0x7efcb40a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efcb8fac640 1 --2- 10.20.193.78:0/3785804869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406d380 0x7efcb40a01f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efca80099f0 tx=0x7efca802ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2aa293b58788fc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 1 -- 10.20.193.78:0/3785804869 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca803c070 con 0x7efcb406d380 2026-02-17T05:59:07.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:07.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: dump: 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 1 -- 10.20.193.78:0/3785804869 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca8037440 con 0x7efcb406d380 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efcbba38640 5 monclient: authenticate success, global_id 164121 2026-02-17T05:59:07.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efcbba38640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 1 -- 10.20.193.78:0/3785804869 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8037810 con 0x7efcb406d380 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efca3fff640 10 monclient: dump: 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efcbba38640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:07.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.896+0000 7efcbba38640 4 set_mon_vals no callback set 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 10 monclient: shutdown 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 -- 10.20.193.78:0/3785804869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406d380 msgr2=0x7efcb40a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 --2- 10.20.193.78:0/3785804869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406d380 0x7efcb40a01f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efca80099f0 tx=0x7efca802ef60 comp rx=0 tx=0).stop 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 -- 10.20.193.78:0/3785804869 shutdown_connections 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 --2- 10.20.193.78:0/3785804869 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406d380 0x7efcb40a01f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 -- 10.20.193.78:0/3785804869 >> 10.20.193.78:0/3785804869 conn(0x7efcb40a6620 msgr2=0x7efcb40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 -- 10.20.193.78:0/3785804869 shutdown_connections 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 -- 10.20.193.78:0/3785804869 wait complete. 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 10 monclient: build_initial_monmap 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 10 monclient: monmap: 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.901 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:07.902 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:07.902 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:07.902 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 Processor -- start 2026-02-17T05:59:07.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 1 -- start start 2026-02-17T05:59:07.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 10 monclient: init 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.897+0000 7efcbba38640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 0x7efcb41318c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient(hunting): picked mon.noname-a con 0x7efcb406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb406d380 0x7efcb4131e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient(hunting): picked mon.noname-c con 0x7efcb406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcb413a240 0x7efcb406f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient(hunting): picked mon.noname-b con 0x7efcb413a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcb413ded0 con 0x7efcb406c890 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7efcb413dd50 con 0x7efcb413a240 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efcb413e050 con 0x7efcb406d380 2026-02-17T05:59:07.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcbba38640 10 monclient(hunting): authenticate will time out at 1393.452048s 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 0x7efcb41318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 0x7efcb41318c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45854/0 (socket says 10.20.193.78:45854) 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 -- 10.20.193.78:0/1018316831 learned_addr learned my addr 10.20.193.78:0/1018316831 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient(hunting): get_auth_request con 0x7efcb406c890 auth_method 0 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb8fac640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb406d380 0x7efcb4131e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb8fac640 10 monclient(hunting): get_auth_request con 0x7efcb406d380 auth_method 0 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb8fac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb8fac640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb8fac640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient(hunting): handle_auth_done global_id 164124 payload 824 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb406d380 msgr2=0x7efcb4131e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb406d380 0x7efcb4131e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcb413a240 msgr2=0x7efcb406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb9fae640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcb413a240 0x7efcb406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcb413a240 0x7efcb406f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient: _finish_hunting 0 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 monclient: found mon.noname-a 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb97ad640 1 -- 10.20.193.78:0/1018316831 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb40734a0 con 0x7efcb406c890 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb9fae640 10 monclient: get_auth_request con 0x7efcb413a240 auth_method 0 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.898+0000 7efcb9fae640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcb413a240 0x7efcb406f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcb97ad640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 msgr2=0x7efcb41318c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcb97ad640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 0x7efcb41318c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efca400c5d0 tx=0x7efca400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4016020 con 0x7efcb406c890 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: dump: 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca4012a60 con 0x7efcb406c890 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 5 monclient: authenticate success, global_id 164124 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 10 monclient: _renew_subs 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb4073780 con 0x7efcb406c890 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4012e50 con 0x7efcb406c890 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca37fe640 4 set_mon_vals no callback set 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efca1ffb640 10 monclient: dump: 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.911 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 10 monclient: _renew_subs 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb40a2140 con 0x7efcb406c890 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb41362b0 0x7efcb4136690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.900+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efcb4073280 con 0x7efcb41362b0 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb41362b0 0x7efcb4136690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 10 monclient: get_auth_request con 0x7efcb41362b0 auth_method 0 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb97ad640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 0x7efcb41318c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efca400c5d0 tx=0x7efca400caa0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7efca401b400 mgrmap(e 5) 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 10 monclient(hunting): handle_auth_done global_id 144287 payload 824 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7efca401b400 con 0x7efcb406c890 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb8fac640 1 -- 10.20.193.78:0/1018316831 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcb40a01d0 con 0x7efcb41362b0 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efca1ffb640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc8403e050 0x7efc84040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb9fae640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc8403e050 0x7efc84040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.901+0000 7efcb9fae640 10 monclient: get_auth_request con 0x7efc8403e050 auth_method 0 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efcb8fac640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb41362b0 msgr2=0x7efcb4136690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efcb8fac640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb41362b0 0x7efcb4136690 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efcb4069d40 tx=0x7efca80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8045020 con 0x7efcb41362b0 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 10 monclient: dump: 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.913 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7efca803d040 con 0x7efcb41362b0 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb41362b0 msgr2=0x7efcb4136690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb41362b0 0x7efcb4136690 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efcb4069d40 tx=0x7efca80047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb41362b0 msgr2=0x7efcb4136690 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.903+0000 7efca1ffb640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb41362b0 0x7efcb4136690 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efcb4069d40 tx=0x7efca80047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.904+0000 7efcb9fae640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc8403e050 0x7efc84040510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efcb0003a40 tx=0x7efcb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcbba38640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcbba38640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb4136bd0 0x7efcb41d8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:07.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcbba38640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efcb40744d0 con 0x7efcb4136bd0 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb4136bd0 0x7efcb41d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 10 monclient: get_auth_request con 0x7efcb4136bd0 auth_method 0 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 10 monclient(hunting): handle_auth_done global_id 144290 payload 824 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.907+0000 7efcb8fac640 1 -- 10.20.193.78:0/1018316831 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efcb4071c80 con 0x7efcb4136bd0 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcb8fac640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb4136bd0 msgr2=0x7efcb41d8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcb8fac640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb4136bd0 0x7efcb41d8400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efca8002410 tx=0x7efca8009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8045070 con 0x7efcb4136bd0 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 10 monclient: dump: 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.916 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7efca8041810 con 0x7efcb4136bd0 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb4136bd0 msgr2=0x7efcb41d8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb4136bd0 0x7efcb41d8400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efca8002410 tx=0x7efca8009340 comp rx=0 tx=0).stop 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb4136bd0 msgr2=0x7efcb41d8400 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcb4136bd0 0x7efcb41d8400 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efca8002410 tx=0x7efca8009340 comp rx=0 tx=0).stop 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:07.917 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 154, 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 131444, 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:07.918 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:07.919 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:07.920 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:07.921 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:07.922 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.923 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:07.924 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc8403e050 msgr2=0x7efc84040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcbba38640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efc8403e050 0x7efc84040510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efcb0003a40 tx=0x7efcb0009040 comp rx=0 tx=0).stop 2026-02-17T05:59:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcbba38640 10 monclient: shutdown 2026-02-17T05:59:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcbba38640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 msgr2=0x7efcb41318c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcbba38640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 0x7efcb41318c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efca400c5d0 tx=0x7efca400caa0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efcb9fae640 1 -- 10.20.193.78:0/1018316831 reap_dead start 2026-02-17T05:59:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.909+0000 7efca1ffb640 1 -- 10.20.193.78:0/1018316831 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(464..464 src has 335..464) ==== 4751+0+0 (unknown 0 0 0) 0x7efca4052840 con 0x7efcb406c890 2026-02-17T05:59:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.910+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 shutdown_connections 2026-02-17T05:59:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.910+0000 7efcbba38640 1 --2- 10.20.193.78:0/1018316831 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcb406c890 0x7efcb41318c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.910+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 >> 10.20.193.78:0/1018316831 conn(0x7efcb40a6620 msgr2=0x7efcb413cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.910+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 shutdown_connections 2026-02-17T05:59:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.910+0000 7efcbba38640 1 -- 10.20.193.78:0/1018316831 wait complete. 2026-02-17T05:59:07.926 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:07.926 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:59:07.926 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.992+0000 7f927ec3b640 10 monclient: get_monmap_and_config 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.992+0000 7f927ec3b640 10 monclient: build_initial_monmap 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.992+0000 7f927ec3b640 10 monclient: monmap: 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:07.993864+0000 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:07.993864+0000 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:08.004 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 1 Processor -- start 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 1 -- start start 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient: init 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f927806c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f927809d430 0x7f927813d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient(hunting): picked mon.noname-a con 0x7f927809d430 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927809bf40 0x7f927813fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient(hunting): picked mon.noname-b con 0x7f927809bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9278073600 con 0x7f927806c890 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f927809ca50 con 0x7f927809d430 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f927806dab0 con 0x7f927809bf40 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927ec3b640 10 monclient(hunting): authenticate will time out at 1393.547434s 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927809bf40 0x7f927813fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f927809d430 0x7f927813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927809bf40 0x7f927813fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58130/0 (socket says 10.20.193.78:58130) 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 1 -- 10.20.193.78:0/4182616646 learned_addr learned my addr 10.20.193.78:0/4182616646 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f927809d430 0x7f927813d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45860/0 (socket says 10.20.193.78:45860) 2026-02-17T05:59:08.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 10 monclient(hunting): get_auth_request con 0x7f927809bf40 auth_method 0 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient(hunting): get_auth_request con 0x7f927809d430 auth_method 0 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f927d1b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient(hunting): handle_auth_done global_id 164127 payload 274 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 1 -- 10.20.193.78:0/4182616646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927809bf40 msgr2=0x7f927813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 1 --2- 10.20.193.78:0/4182616646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927809bf40 0x7f927813fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 1 -- 10.20.193.78:0/4182616646 >> v1:10.20.193.115:6789/0 conn(0x7f927806c890 legacy=0x7f927806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 1 monclient: found mon.noname-a 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.993+0000 7f926ffff640 1 -- 10.20.193.78:0/4182616646 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927809c770 con 0x7f927809d430 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926ffff640 1 -- 10.20.193.78:0/4182616646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809d430 msgr2=0x7f927813d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926ffff640 1 --2- 10.20.193.78:0/4182616646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809d430 0x7f927813d5d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f927809cbd0 tx=0x7f926002ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3825aad98431a9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 1 -- 10.20.193.78:0/4182616646 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926003c070 con 0x7f927809d430 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: dump: 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.006 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 1 -- 10.20.193.78:0/4182616646 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f926002fbf0 con 0x7f927809d430 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 5 monclient: authenticate success, global_id 164127 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 1 -- 10.20.193.78:0/4182616646 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260037440 con 0x7f927809d430 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 4 set_mon_vals no callback set 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f926f7fe640 10 monclient: dump: 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.007 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 10 monclient: shutdown 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 1 -- 10.20.193.78:0/4182616646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809d430 msgr2=0x7f927813d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 1 --2- 10.20.193.78:0/4182616646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809d430 0x7f927813d5d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f927809cbd0 tx=0x7f926002ed70 comp rx=0 tx=0).stop 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 1 -- 10.20.193.78:0/4182616646 shutdown_connections 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 1 --2- 10.20.193.78:0/4182616646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927809bf40 0x7f927813fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 1 --2- 10.20.193.78:0/4182616646 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809d430 0x7f927813d5d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 1 -- 10.20.193.78:0/4182616646 >> 10.20.193.78:0/4182616646 conn(0x7f92780a6620 msgr2=0x7f92780a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 1 -- 10.20.193.78:0/4182616646 shutdown_connections 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.996+0000 7f927ec3b640 1 -- 10.20.193.78:0/4182616646 wait complete. 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient: build_initial_monmap 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient: monmap: 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 1 Processor -- start 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 1 -- start start 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient: init 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927806c890 0x7f927811cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient(hunting): picked mon.noname-c con 0x7f927806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809bf40 0x7f927811d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient(hunting): picked mon.noname-a con 0x7f927809bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f927809d430 0x7f927811d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient(hunting): picked mon.noname-b con 0x7f927809d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f927806d380 con 0x7f927809bf40 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9278073410 con 0x7f927809d430 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f927806d500 con 0x7f927806c890 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927ec3b640 10 monclient(hunting): authenticate will time out at 1393.551667s 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809bf40 0x7f927811d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809bf40 0x7f927811d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45864/0 (socket says 10.20.193.78:45864) 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 -- 10.20.193.78:0/2999428960 learned_addr learned my addr 10.20.193.78:0/2999428960 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient(hunting): get_auth_request con 0x7f927809bf40 auth_method 0 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927c9b0640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927806c890 0x7f927811cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927c9b0640 10 monclient(hunting): get_auth_request con 0x7f927806c890 auth_method 0 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927c9b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927c9b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927c9b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient(hunting): handle_auth_done global_id 164130 payload 824 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927806c890 msgr2=0x7f927811cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f927806c890 0x7f927811cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f927809d430 msgr2=0x7f927811d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f927809d430 0x7f927811d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 monclient: found mon.noname-a 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.997+0000 7f926ffff640 1 -- 10.20.193.78:0/2999428960 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927811fd40 con 0x7f927809bf40 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:07.999+0000 7f926ffff640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809bf40 msgr2=0x7f927811d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926ffff640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809bf40 0x7f927811d1b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9260002410 tx=0x7f92600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926003c040 con 0x7f927809bf40 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: dump: 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9260047070 con 0x7f927809bf40 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 5 monclient: authenticate success, global_id 164130 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260040a60 con 0x7f927809bf40 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 10 monclient: _renew_subs 2026-02-17T05:59:08.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926effd640 4 set_mon_vals no callback set 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927811e890 con 0x7f927809bf40 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 10 monclient: dump: 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 10 monclient: _renew_subs 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92780a21f0 con 0x7f927809bf40 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278134f00 0x7f92781352e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9278074980 con 0x7f9278134f00 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f926003f030 con 0x7f927809bf40 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927c9b0640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278134f00 0x7f92781352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927c9b0640 10 monclient: get_auth_request con 0x7f9278134f00 auth_method 0 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927c9b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927c9b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f927c9b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f924803df00 0x7f92480403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.000+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(464..464 src has 335..464) ==== 4751+0+0 (secure 0 0 0) 0x7f9260077ad0 con 0x7f927809bf40 2026-02-17T05:59:08.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.001+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.001+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.001+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.001+0000 7f927d1b1640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f924803df00 0x7f92480403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.001+0000 7f927d1b1640 10 monclient: get_auth_request con 0x7f924803df00 auth_method 0 2026-02-17T05:59:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.001+0000 7f927c9b0640 10 monclient(hunting): handle_auth_done global_id 134338 payload 824 2026-02-17T05:59:08.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.001+0000 7f927c9b0640 1 -- 10.20.193.78:0/2999428960 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f927806cc70 con 0x7f9278134f00 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f927d1b1640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f924803df00 0x7f92480403c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f927811df20 tx=0x7f927400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f927c9b0640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278134f00 msgr2=0x7f92781352e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f927c9b0640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278134f00 0x7f92781352e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f926800f7a0 tx=0x7f926800fc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268018030 con 0x7f9278134f00 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 10 monclient: dump: 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f926800dbc0 con 0x7f9278134f00 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:08.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278134f00 msgr2=0x7f92781352e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.003+0000 7f926d7fa640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278134f00 0x7f92781352e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f926800f7a0 tx=0x7f926800fc70 comp rx=0 tx=0).stop 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.004+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278134f00 msgr2=0x7f92781352e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.004+0000 7f926d7fa640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278134f00 0x7f92781352e0 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f926800f7a0 tx=0x7f926800fc70 comp rx=0 tx=0).stop 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927ec3b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927ec3b640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278135820 0x7f92781da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927ec3b640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f927811e610 con 0x7f9278135820 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278135820 0x7f92781da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 10 monclient: get_auth_request con 0x7f9278135820 auth_method 0 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 10 monclient(hunting): handle_auth_done global_id 134341 payload 824 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.007+0000 7f927c9b0640 1 -- 10.20.193.78:0/2999428960 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f92780744d0 con 0x7f9278135820 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927c9b0640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278135820 msgr2=0x7f92781da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927c9b0640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278135820 0x7f92781da4d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9268018000 tx=0x7f9268003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268004230 con 0x7f9278135820 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 10 monclient: dump: 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f9268004230 con 0x7f9278135820 2026-02-17T05:59:08.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278135820 msgr2=0x7f92781da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278135820 0x7f92781da4d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9268018000 tx=0x7f9268003860 comp rx=0 tx=0).stop 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278135820 msgr2=0x7f92781da4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f926d7fa640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9278135820 0x7f92781da4d0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9268018000 tx=0x7f9268003860 comp rx=0 tx=0).stop 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 154, 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 175729, 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:08.015 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:08.016 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:08.018 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:59:08.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f924803df00 msgr2=0x7f92480403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f924803df00 0x7f92480403c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f927811df20 tx=0x7f927400b040 comp rx=0 tx=0).stop 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 10 monclient: shutdown 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809bf40 msgr2=0x7f927811d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809bf40 0x7f927811d1b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9260002410 tx=0x7f92600047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927d1b1640 1 -- 10.20.193.78:0/2999428960 reap_dead start 2026-02-17T05:59:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 shutdown_connections 2026-02-17T05:59:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 --2- 10.20.193.78:0/2999428960 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f927809bf40 0x7f927811d1b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 >> 10.20.193.78:0/2999428960 conn(0x7f92780a6620 msgr2=0x7f92780a1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 shutdown_connections 2026-02-17T05:59:08.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.010+0000 7f927ec3b640 1 -- 10.20.193.78:0/2999428960 wait complete. 2026-02-17T05:59:08.021 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:08.021 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:59:08.021 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:59:08.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.086+0000 7f984ec76640 10 monclient: get_monmap_and_config 2026-02-17T05:59:08.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.086+0000 7f984ec76640 10 monclient: build_initial_monmap 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.086+0000 7f984ec76640 10 monclient: monmap: 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:08.087741+0000 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:08.087741+0000 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:08.088 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 1 Processor -- start 2026-02-17T05:59:08.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 1 -- start start 2026-02-17T05:59:08.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient: init 2026-02-17T05:59:08.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:08.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f98480a6470 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient(hunting): picked mon.noname-b con 0x7f98480a6850 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f98480a6c30 0x7f984813f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient(hunting): picked mon.noname-a con 0x7f98480a6c30 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98480a0730 con 0x7f98480a6470 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f984809fa00 con 0x7f98480a6c30 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f98480a0450 con 0x7f98480a6850 2026-02-17T05:59:08.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984ec76640 10 monclient(hunting): authenticate will time out at 1393.641469s 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984c9eb640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f98480a6470 0x7f98480a1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54564/0 (socket says 10.20.193.78:54564) 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984c9eb640 1 -- 10.20.193.78:0/1757006916 learned_addr learned my addr 10.20.193.78:0/1757006916 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient(hunting): get_auth_request con 0x7f98480a6850 auth_method 0 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984d1ec640 1 --2- 10.20.193.78:0/1757006916 >> v2:10.20.193.78:3300/0 conn(0x7f98480a6c30 0x7f984813f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984d1ec640 10 monclient(hunting): get_auth_request con 0x7f98480a6c30 auth_method 0 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984d1ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984d1ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f984d1ec640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient(hunting): handle_auth_done global_id 144299 payload 274 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 1 -- 10.20.193.78:0/1757006916 >> v2:10.20.193.78:3300/0 conn(0x7f98480a6c30 msgr2=0x7f984813f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 1 --2- 10.20.193.78:0/1757006916 >> v2:10.20.193.78:3300/0 conn(0x7f98480a6c30 0x7f984813f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 1 -- 10.20.193.78:0/1757006916 >> v1:10.20.193.78:6789/0 conn(0x7f98480a6470 legacy=0x7f98480a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 1 monclient: found mon.noname-b 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:08.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.087+0000 7f983ffff640 1 -- 10.20.193.78:0/1757006916 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98480a5ea0 con 0x7f98480a6850 2026-02-17T05:59:08.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983ffff640 1 -- 10.20.193.78:0/1757006916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 msgr2=0x7f984813d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:08.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983ffff640 1 --2- 10.20.193.78:0/1757006916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813d270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f98300098a0 tx=0x7f983002ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1cc461aed26bfc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:08.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 1 -- 10.20.193.78:0/1757006916 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983003c070 con 0x7f98480a6850 2026-02-17T05:59:08.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: dump: 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 1 -- 10.20.193.78:0/1757006916 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983002fc00 con 0x7f98480a6850 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f984ec76640 5 monclient: authenticate success, global_id 144299 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f984ec76640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:08.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f984ec76640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 1 -- 10.20.193.78:0/1757006916 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830037440 con 0x7f98480a6850 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f984ec76640 4 set_mon_vals no callback set 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.090+0000 7f983f7fe640 10 monclient: dump: 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 10 monclient: shutdown 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 -- 10.20.193.78:0/1757006916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 msgr2=0x7f984813d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 --2- 10.20.193.78:0/1757006916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813d270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f98300098a0 tx=0x7f983002ed80 comp rx=0 tx=0).stop 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 -- 10.20.193.78:0/1757006916 shutdown_connections 2026-02-17T05:59:08.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 --2- 10.20.193.78:0/1757006916 >> v2:10.20.193.78:3300/0 conn(0x7f98480a6c30 0x7f984813f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 --2- 10.20.193.78:0/1757006916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813d270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 -- 10.20.193.78:0/1757006916 >> 10.20.193.78:0/1757006916 conn(0x7f98480aae40 msgr2=0x7f98480ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 -- 10.20.193.78:0/1757006916 shutdown_connections 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 -- 10.20.193.78:0/1757006916 wait complete. 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 10 monclient: build_initial_monmap 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 10 monclient: monmap: 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 Processor -- start 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 1 -- start start 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.091+0000 7f984ec76640 10 monclient: init 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:08.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98480a6470 0x7f984813eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient(hunting): picked mon.noname-a con 0x7f98480a6470 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient(hunting): picked mon.noname-c con 0x7f98480a6850 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f98480a6c30 0x7f98480946d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient(hunting): picked mon.noname-b con 0x7f98480a6c30 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9848140090 con 0x7f98480a6470 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f984813ff10 con 0x7f98480a6c30 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9848140210 con 0x7f98480a6850 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984ec76640 10 monclient(hunting): authenticate will time out at 1393.646090s 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984c9eb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98480a6470 0x7f984813eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58170/0 (socket says 10.20.193.78:58170) 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 -- 10.20.193.78:0/707487951 learned_addr learned my addr 10.20.193.78:0/707487951 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient(hunting): get_auth_request con 0x7f98480a6850 auth_method 0 2026-02-17T05:59:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984c9eb640 10 monclient(hunting): get_auth_request con 0x7f98480a6470 auth_method 0 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984c9eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984c9eb640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984c9eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984c9eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984c9eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984c9eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient(hunting): handle_auth_done global_id 144302 payload 824 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 -- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f98480a6c30 msgr2=0x7f98480946d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984d1ec640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f98480a6c30 0x7f98480946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f98480a6c30 0x7f98480946d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 -- 10.20.193.78:0/707487951 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98480a6470 msgr2=0x7f984813eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98480a6470 0x7f984813eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 monclient: found mon.noname-c 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:08.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f983ffff640 1 -- 10.20.193.78:0/707487951 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9848099020 con 0x7f98480a6850 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984d1ec640 10 monclient: get_auth_request con 0x7f98480a6c30 auth_method 0 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.092+0000 7f984d1ec640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f98480a6c30 0x7f98480946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983ffff640 1 -- 10.20.193.78:0/707487951 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 msgr2=0x7f984813f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983ffff640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813f410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98300099d0 tx=0x7f98300047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 1 -- 10.20.193.78:0/707487951 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983003c040 con 0x7f98480a6850 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: dump: 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.103 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: _finish_auth 0 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: _check_auth_tickets 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 1 -- 10.20.193.78:0/707487951 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9830040600 con 0x7f98480a6850 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 1 -- 10.20.193.78:0/707487951 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98300409d0 con 0x7f98480a6850 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983effd640 4 set_mon_vals no callback set 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f983d7fa640 10 monclient: dump: 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:08.104 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f984ec76640 5 monclient: authenticate success, global_id 144302 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f984ec76640 10 monclient: _renew_subs 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f984ec76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9848098660 con 0x7f98480a6850 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f984ec76640 10 monclient: _renew_subs 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f984ec76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.094+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98480a26e0 con 0x7f98480a6850 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f984ec76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f984ec76640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f984ec76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9848074980 con 0x7f98480a6850 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f983d7fa640 1 -- 10.20.193.78:0/707487951 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f983003f030 con 0x7f98480a6850 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f983d7fa640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f981803df00 0x7f98180403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f983d7fa640 1 -- 10.20.193.78:0/707487951 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(464..464 src has 335..464) ==== 4751+0+0 (secure 0 0 0) 0x7f98300776a0 con 0x7f98480a6850 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f984c9eb640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f981803df00 0x7f98180403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:08.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.095+0000 7f984c9eb640 10 monclient: get_auth_request con 0x7f981803df00 auth_method 0 2026-02-17T05:59:08.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.098+0000 7f983d7fa640 1 -- 10.20.193.78:0/707487951 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9848099020 con 0x7f98480a6850 2026-02-17T05:59:08.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.098+0000 7f983d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:08.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.098+0000 7f983d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:08.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.098+0000 7f984c9eb640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f981803df00 0x7f98180403c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f98380093b0 tx=0x7f9838009340 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:08.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.213+0000 7f984ec76640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:59:08.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.213+0000 7f984ec76640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:59:08.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.213+0000 7f984ec76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:08.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.213+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f98480a1480 con 0x7f98480a6850 2026-02-17T05:59:08.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.213+0000 7f983d7fa640 1 -- 10.20.193.78:0/707487951 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f98480a0730 con 0x7f98480a6850 2026-02-17T05:59:08.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.213+0000 7f983d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:59:08.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.213+0000 7f983d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:08.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f981803df00 msgr2=0x7f98180403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f981803df00 0x7f98180403c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f98380093b0 tx=0x7f9838009340 comp rx=0 tx=0).stop 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 10 monclient: shutdown 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 msgr2=0x7f984813f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813f410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98300099d0 tx=0x7f98300047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 shutdown_connections 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f981803df00 0x7f98180403c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f98480a6c30 0x7f98480946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f98480a6850 0x7f984813f410 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 --2- 10.20.193.78:0/707487951 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98480a6470 0x7f984813eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 >> 10.20.193.78:0/707487951 conn(0x7f98480aae40 msgr2=0x7f98480a97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 shutdown_connections 2026-02-17T05:59:08.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:08.214+0000 7f984ec76640 1 -- 10.20.193.78:0/707487951 wait complete. 2026-02-17T05:59:08.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.220+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.220+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3867382955,config=692619927,health=687770194,kv=449989585,logm=973606524}) 2026-02-17T05:59:08.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.220+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=24,config=2,health=9,kv=12,logm=53} crc {auth=3867382955,config=692619927,health=3965445838,kv=449989585,logm=724983056}) 2026-02-17T05:59:08.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.220+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.220+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3867382955,config=692619927,health=687770194,kv=449989585,logm=973606524}) 2026-02-17T05:59:08.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.220+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=2,health=12,kv=12,logm=51} crc {auth=1539849319,config=692619927,health=2096750356,kv=449989585,logm=2915000545}) 2026-02-17T05:59:08.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1274814860}) 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1408009056}) 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1274814860}) 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4107111580}) 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4232047717}) 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=611527446}) 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4232047717}) 2026-02-17T05:59:08.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.221+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2924050226}) 2026-02-17T05:59:08.226 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220309579}) 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1603808205}) 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4220309579}) 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2392519229}) 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1293541268}) 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=696752628}) 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1293541268}) 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3072339083}) 2026-02-17T05:59:08.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3694078644}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2044275620}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3694078644}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.222+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=981521743}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698196541}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4044289202}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698196541}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2970803144}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2829155312}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2497385337}) 2026-02-17T05:59:08.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2829155312}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3318246462}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3951015595}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1756042334}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3951015595}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323050850}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.223+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=825273508}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=87330473}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=825273508}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=413276119}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882393129}) 2026-02-17T05:59:08.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3863221992}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882393129}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2171245582}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2664769272}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=940575879}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2664769272}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1827761679}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1743044419}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2294986930}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1743044419}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.224+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2472540364}) 2026-02-17T05:59:08.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3180125946}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2782435833}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3180125946}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3321499071}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2978968295}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2018451634}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2978968295}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1360397461}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178293121}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1077660867}) 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178293121}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.225+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=494429401}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1975314273}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1364320107}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1975314273}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=238079984}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1247287288}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1496765769}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1247287288}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2357802828}) 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=400416075}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2398572234}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=400416075}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.226+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=272730793}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2901989659}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1592681659}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2901989659}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3910409541}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=817529221}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043079607}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=817529221}) 2026-02-17T05:59:08.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.227+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472687385}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1059359645}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1401459831}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1059359645}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=337714270}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=311252133}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=641352133}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=311252133}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2393713674}) 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2456679802}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3971935825}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2456679802}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.234+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=445207364}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=552059983}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2657159712}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=552059983}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2778089815}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2816613583}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404093888}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2816613583}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1558437916}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3413295680}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2532383718}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3413295680}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.235+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3501886269}) 2026-02-17T05:59:08.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263456408}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3087032876}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263456408}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=968244667}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1771411526}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1206581628}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1771411526}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1045945875}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153588347}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3735064561}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153588347}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.236+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1237830267}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696246942}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1324351616}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696246942}) 2026-02-17T05:59:08.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3715458224}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2008074235}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2987390020}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2008074235}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1846972978}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2991504177}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1688079591}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2991504177}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.237+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3775768748}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2192915058}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3678138451}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2192915058}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3509864372}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=404711966}) 2026-02-17T05:59:08.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2894197601}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=404711966}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082883799}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653773194}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2841027595}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653773194}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.238+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3238885830}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3050379797}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2889116408}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3050379797}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2647750720}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046705146}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457955788}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046705146}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.239+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2854780651}) 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.248 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=819754933}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3226217210}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=819754933}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3529884021}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15} crc {logm=2313523787,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1935919852}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11} crc {logm=3845579365,mdsmap=1160443688,mgr=1022463925,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1814168611}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15} crc {logm=2313523787,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1935919852}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.240+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=77,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10} crc {logm=1775684697,mdsmap=1160443688,mgr=1994034014,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=567880499}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=78} crc {mon_config_key=3847793152,monmap=1784110042,osd_metadata=3370527109,osd_pg_creating=465116460,osd_snap=429613662}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=77} crc {mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=465116460,osd_snap=1587719206}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=78} crc {mon_config_key=3847793152,monmap=1784110042,osd_metadata=3370527109,osd_pg_creating=465116460,osd_snap=429613662}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=78} crc {mon_config_key=1912222953,monmap=1784110042,osd_metadata=278982410,osd_pg_creating=465116460,osd_snap=2565766105}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=2,osdmap=98} crc {osd_snap=2846238367,osdmap=2230853223}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=2,osdmap=98} crc {osd_snap=2846238367,osdmap=1377687192}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=2,osdmap=98} crc {osd_snap=2846238367,osdmap=2230853223}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.241+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=2,osdmap=98} crc {osd_snap=2846238367,osdmap=2022870334}) 2026-02-17T05:59:08.249 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.242+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.242+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1160914641}) 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.242+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1415417690}) 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.242+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.242+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1160914641}) 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.242+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1135657015}) 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.244+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.244+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=8} crc {osdmap=65056786}) 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.244+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=8} crc {osdmap=1687974852}) 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.244+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.244+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=8} crc {osdmap=65056786}) 2026-02-17T05:59:08.250 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:08.244+0000 7fbbb4d69640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=7} crc {osdmap=2493066256}) 2026-02-17T05:59:08.655 INFO:tasks.rados.rados.0.trial115.stdout:2166: done (2 left) 2026-02-17T05:59:08.655 INFO:tasks.rados.rados.0.trial115.stdout:2167: write oid 43 current snap is 219 2026-02-17T05:59:08.655 INFO:tasks.rados.rados.0.trial115.stdout:2167: seq_num 773 ranges {591811=719130,1767522=636596,2828091=163720} 2026-02-17T05:59:08.659 INFO:tasks.rados.rados.0.trial115.stdout:2167: writing trial11545420-43 from 591811 to 1310941 tid 1 2026-02-17T05:59:08.664 INFO:tasks.rados.rados.0.trial115.stdout:2167: writing trial11545420-43 from 1767522 to 2404118 tid 2 2026-02-17T05:59:08.666 INFO:tasks.rados.rados.0.trial115.stdout:2167: writing trial11545420-43 from 2828091 to 2991811 tid 3 2026-02-17T05:59:08.667 INFO:tasks.rados.rados.0.trial115.stdout:2168: read oid 44 snap -1 2026-02-17T05:59:08.667 INFO:tasks.rados.rados.0.trial115.stdout:2168: expect (ObjNum 740 snap 204 seq_num 740) 2026-02-17T05:59:08.667 INFO:tasks.rados.rados.0.trial115.stdout:2169: read oid 37 snap -1 2026-02-17T05:59:08.667 INFO:tasks.rados.rados.0.trial115.stdout:2169: expect deleted 2026-02-17T05:59:08.667 INFO:tasks.rados.rados.0.trial115.stdout:2170: snap_remove snap 213 2026-02-17T05:59:09.226 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:59:09.227 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:09.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.294+0000 7fd5e93f5640 10 monclient: get_monmap_and_config 2026-02-17T05:59:09.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.294+0000 7fd5e93f5640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.294+0000 7fd5e93f5640 10 monclient: monmap: 2026-02-17T05:59:09.298 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:09.295942+0000 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:09.295942+0000 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:09.299 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 1 Processor -- start 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 1 -- start start 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient: init 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient: _add_conns ranks=[3,0,1,4,2] 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd5e409ddd0 0x7fd5e409e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient(hunting): picked mon.noname-a con 0x7fd5e409ddd0 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:09.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd5e4073f60 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd5e4138660 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e40a2850 con 0x7fd5e4073f60 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e40a2560 con 0x7fd5e4138660 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e40a3810 con 0x7fd5e409ddd0 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e93f5640 10 monclient(hunting): authenticate will time out at 1394.849598s 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd5e409ddd0 0x7fd5e409e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd5e409ddd0 0x7fd5e409e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45910/0 (socket says 10.20.193.78:45910) 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/3001068601 learned_addr learned my addr 10.20.193.78:0/3001068601 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e27fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd5e4073f60 0x7fd5e4137f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54572/0 (socket says 10.20.193.78:54572) 2026-02-17T05:59:09.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request con 0x7fd5e409ddd0 auth_method 0 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_done global_id 164139 payload 274 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/3001068601 >> v1:10.20.193.115:6789/0 conn(0x7fd5e4138660 legacy=0x7fd5e4140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/3001068601 >> v1:10.20.193.78:6789/0 conn(0x7fd5e4073f60 legacy=0x7fd5e4137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 1 monclient: found mon.noname-a 2026-02-17T05:59:09.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.295+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/3001068601 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40a3af0 con 0x7fd5e409ddd0 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/3001068601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e409ddd0 msgr2=0x7fd5e409e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e2ffd640 1 --2- 10.20.193.78:0/3001068601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e409ddd0 0x7fd5e409e1b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009970 tx=0x7fd5d802ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cb9568bc505de2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 1 -- 10.20.193.78:0/3001068601 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d803c070 con 0x7fd5e409ddd0 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: dump: 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 5 monclient: authenticate success, global_id 164139 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 1 -- 10.20.193.78:0/3001068601 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d802fdb0 con 0x7fd5e409ddd0 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 1 -- 10.20.193.78:0/3001068601 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8037660 con 0x7fd5e409ddd0 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 4 set_mon_vals no callback set 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e1ffb640 10 monclient: dump: 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.311 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 10 monclient: shutdown 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 1 -- 10.20.193.78:0/3001068601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e409ddd0 msgr2=0x7fd5e409e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 1 --2- 10.20.193.78:0/3001068601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e409ddd0 0x7fd5e409e1b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009970 tx=0x7fd5d802ee90 comp rx=0 tx=0).stop 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 1 -- 10.20.193.78:0/3001068601 shutdown_connections 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 1 --2- 10.20.193.78:0/3001068601 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e409ddd0 0x7fd5e409e1b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 1 -- 10.20.193.78:0/3001068601 >> 10.20.193.78:0/3001068601 conn(0x7fd5e40aacd0 msgr2=0x7fd5e40ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 1 -- 10.20.193.78:0/3001068601 shutdown_connections 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.298+0000 7fd5e93f5640 1 -- 10.20.193.78:0/3001068601 wait complete. 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient: monmap: 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 1 Processor -- start 2026-02-17T05:59:09.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 1 -- start start 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient: init 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4073f60 0x7fd5e41360c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient(hunting): picked mon.noname-a con 0x7fd5e4073f60 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409ddd0 0x7fd5e412fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient(hunting): picked mon.noname-c con 0x7fd5e409ddd0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e4138660 0x7fd5e4130320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient(hunting): picked mon.noname-b con 0x7fd5e4138660 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e409e6f0 con 0x7fd5e4073f60 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40a57e0 con 0x7fd5e4138660 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e40a4840 con 0x7fd5e409ddd0 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e93f5640 10 monclient(hunting): authenticate will time out at 1394.853685s 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4073f60 0x7fd5e41360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4073f60 0x7fd5e41360c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45916/0 (socket says 10.20.193.78:45916) 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/4125716916 learned_addr learned my addr 10.20.193.78:0/4125716916 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request con 0x7fd5e4073f60 auth_method 0 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e27fc640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409ddd0 0x7fd5e412fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e27fc640 10 monclient(hunting): get_auth_request con 0x7fd5e409ddd0 auth_method 0 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_done global_id 164142 payload 824 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409ddd0 msgr2=0x7fd5e412fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409ddd0 0x7fd5e412fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e37fe640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e4138660 0x7fd5e4130320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e4138660 msgr2=0x7fd5e4130320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e4138660 0x7fd5e4130320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 monclient: found mon.noname-a 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/4125716916 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e4133370 con 0x7fd5e4073f60 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e37fe640 10 monclient: get_auth_request con 0x7fd5e4138660 auth_method 0 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e27fc640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409ddd0 0x7fd5e412fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.299+0000 7fd5e37fe640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e4138660 0x7fd5e4130320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e2ffd640 1 -- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4073f60 msgr2=0x7fd5e41360c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e2ffd640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4073f60 0x7fd5e41360c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009aa0 tx=0x7fd5d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 1 -- 10.20.193.78:0/4125716916 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8046070 con 0x7fd5e4073f60 2026-02-17T05:59:09.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: dump: 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 1 -- 10.20.193.78:0/4125716916 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d8037e10 con 0x7fd5e4073f60 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 5 monclient: authenticate success, global_id 164142 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 1 -- 10.20.193.78:0/4125716916 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d803c040 con 0x7fd5e4073f60 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 10 monclient: dump: 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.315 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e17fa640 4 set_mon_vals no callback set 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 10 monclient: _renew_subs 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e40a1c80 con 0x7fd5e4073f60 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 10 monclient: _renew_subs 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e4133650 con 0x7fd5e4073f60 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e4138b10 con 0x7fd5e4073f60 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.302+0000 7fd5c37fe640 1 -- 10.20.193.78:0/4125716916 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd5d80418b0 con 0x7fd5e4073f60 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.303+0000 7fd5c37fe640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003df00 0x7fd5b00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.303+0000 7fd5c37fe640 1 -- 10.20.193.78:0/4125716916 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(465..465 src has 335..465) ==== 4795+0+0 (secure 0 0 0) 0x7fd5d8077870 con 0x7fd5e4073f60 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.303+0000 7fd5e27fc640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003df00 0x7fd5b00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.303+0000 7fd5e27fc640 10 monclient: get_auth_request con 0x7fd5b003df00 auth_method 0 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.305+0000 7fd5c37fe640 1 -- 10.20.193.78:0/4125716916 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd5d8050020 con 0x7fd5e4073f60 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.305+0000 7fd5c37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.305+0000 7fd5c37fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:09.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.306+0000 7fd5e27fc640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003df00 0x7fd5b00403c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4132640 tx=0x7fd5cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.417+0000 7fd5e93f5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.417+0000 7fd5e93f5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.417+0000 7fd5e93f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.417+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd5e4131680 con 0x7fd5e4073f60 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5c37fe640 1 -- 10.20.193.78:0/4125716916 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd5d80363a0 con 0x7fd5e4073f60 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5c37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5c37fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":154,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003df00 msgr2=0x7fd5b00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003df00 0x7fd5b00403c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4132640 tx=0x7fd5cc009040 comp rx=0 tx=0).stop 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 10 monclient: shutdown 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4073f60 msgr2=0x7fd5e41360c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4073f60 0x7fd5e41360c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009aa0 tx=0x7fd5d80047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 shutdown_connections 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd5b003df00 0x7fd5b00403c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd5e4138660 0x7fd5e4130320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd5e409ddd0 0x7fd5e412fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 --2- 10.20.193.78:0/4125716916 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd5e4073f60 0x7fd5e41360c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.418+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 >> 10.20.193.78:0/4125716916 conn(0x7fd5e40aacd0 msgr2=0x7fd5e4137360 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.419+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 shutdown_connections 2026-02-17T05:59:09.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.419+0000 7fd5e93f5640 1 -- 10.20.193.78:0/4125716916 wait complete. 2026-02-17T05:59:09.426 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:59:09.426 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:59:09.426 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:59:09.426 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:59:09.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.493+0000 7fb93d088640 10 monclient: get_monmap_and_config 2026-02-17T05:59:09.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.493+0000 7fb93d088640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.493+0000 7fb93d088640 10 monclient: monmap: 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:09.494746+0000 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:09.494746+0000 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:09.495 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.493+0000 7fb93d088640 1 Processor -- start 2026-02-17T05:59:09.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 1 -- start start 2026-02-17T05:59:09.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient: init 2026-02-17T05:59:09.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2026-02-17T05:59:09.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806c890 0x7fb93806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient(hunting): picked mon.noname-b con 0x7fb93806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb93809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb93806d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb938135c10 con 0x7fb93806d1b0 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93806daf0 con 0x7fb93809c4a0 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb9380a0880 con 0x7fb93806c890 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb93d088640 10 monclient(hunting): authenticate will time out at 1395.048451s 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806c890 0x7fb93806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb937577640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb93806d1b0 0x7fb938135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54574/0 (socket says 10.20.193.78:54574) 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb937577640 1 -- 10.20.193.78:0/1820955381 learned_addr learned my addr 10.20.193.78:0/1820955381 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient(hunting): get_auth_request con 0x7fb93806c890 auth_method 0 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient(hunting): handle_auth_done global_id 144308 payload 274 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 1 -- 10.20.193.78:0/1820955381 >> v1:10.20.193.115:6789/0 conn(0x7fb93809c4a0 legacy=0x7fb938134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 1 -- 10.20.193.78:0/1820955381 >> v1:10.20.193.78:6789/0 conn(0x7fb93806d1b0 legacy=0x7fb938135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 1 monclient: found mon.noname-b 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.494+0000 7fb936d76640 1 -- 10.20.193.78:0/1820955381 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93809cc80 con 0x7fb93806c890 2026-02-17T05:59:09.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb936d76640 1 -- 10.20.193.78:0/1820955381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806c890 msgr2=0x7fb93806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb936d76640 1 --2- 10.20.193.78:0/1820955381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806c890 0x7fb93806cc70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0098d0 tx=0x7fb92c02ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c2045e3056fa1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 1 -- 10.20.193.78:0/1820955381 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c03c070 con 0x7fb93806c890 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: dump: 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.506 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 5 monclient: authenticate success, global_id 144308 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 1 -- 10.20.193.78:0/1820955381 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c02fcc0 con 0x7fb93806c890 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 1 -- 10.20.193.78:0/1820955381 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c037660 con 0x7fb93806c890 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb935d74640 10 monclient: dump: 2026-02-17T05:59:09.507 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 4 set_mon_vals no callback set 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 10 monclient: shutdown 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 1 -- 10.20.193.78:0/1820955381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806c890 msgr2=0x7fb93806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 1 --2- 10.20.193.78:0/1820955381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806c890 0x7fb93806cc70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0098d0 tx=0x7fb92c02ed60 comp rx=0 tx=0).stop 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 1 -- 10.20.193.78:0/1820955381 shutdown_connections 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 1 --2- 10.20.193.78:0/1820955381 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806c890 0x7fb93806cc70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 1 -- 10.20.193.78:0/1820955381 >> 10.20.193.78:0/1820955381 conn(0x7fb9380a6620 msgr2=0x7fb9380a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 1 -- 10.20.193.78:0/1820955381 shutdown_connections 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 1 -- 10.20.193.78:0/1820955381 wait complete. 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.497+0000 7fb93d088640 10 monclient: monmap: 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:09.509 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 1 Processor -- start 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 1 -- start start 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient: init 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93806c890 0x7fb9381341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient(hunting): picked mon.noname-a con 0x7fb93806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806d1b0 0x7fb938134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient(hunting): picked mon.noname-c con 0x7fb93806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb93809c4a0 0x7fb93809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient(hunting): picked mon.noname-b con 0x7fb93809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9380a0020 con 0x7fb93806c890 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb93806d600 con 0x7fb93809c4a0 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb9380a01a0 con 0x7fb93806d1b0 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb93d088640 10 monclient(hunting): authenticate will time out at 1395.052405s 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93806c890 0x7fb9381341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806d1b0 0x7fb938134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93806c890 0x7fb9381341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45930/0 (socket says 10.20.193.78:45930) 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 1 -- 10.20.193.78:0/2210200170 learned_addr learned my addr 10.20.193.78:0/2210200170 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient(hunting): get_auth_request con 0x7fb93806d1b0 auth_method 0 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 10 monclient(hunting): get_auth_request con 0x7fb93806c890 auth_method 0 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb937577640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb93809c4a0 0x7fb93809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient(hunting): handle_auth_done global_id 144311 payload 824 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb93809c4a0 msgr2=0x7fb93809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb93809c4a0 0x7fb93809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93806c890 msgr2=0x7fb9381341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93806c890 0x7fb9381341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 1 monclient: found mon.noname-c 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb936575640 1 -- 10.20.193.78:0/2210200170 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93809d810 con 0x7fb93806d1b0 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb937577640 10 monclient: get_auth_request con 0x7fb93809c4a0 auth_method 0 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.498+0000 7fb937577640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb93809c4a0 0x7fb93809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb936575640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806d1b0 msgr2=0x7fb938134730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb936575640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806d1b0 0x7fb938134730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c450 tx=0x7fb92000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920016020 con 0x7fb93806d1b0 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: dump: 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.513 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb920012a40 con 0x7fb93806d1b0 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb93d088640 5 monclient: authenticate success, global_id 144311 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb93d088640 10 monclient: _renew_subs 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb93d088640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9380a26d0 con 0x7fb93806d1b0 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920011380 con 0x7fb93806d1b0 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb935573640 4 set_mon_vals no callback set 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb9177fe640 10 monclient: dump: 2026-02-17T05:59:09.514 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb93d088640 10 monclient: _renew_subs 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb93d088640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.500+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb938127930 con 0x7fb93806d1b0 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb93d088640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb93d088640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9381cf5d0 0x7fb93811cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb93d088640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381d02f0 con 0x7fb9381cf5d0 2026-02-17T05:59:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9381cf5d0 0x7fb93811cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 10 monclient: get_auth_request con 0x7fb9381cf5d0 auth_method 0 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb9200115e0 con 0x7fb93806d1b0 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 10 monclient(hunting): handle_auth_done global_id 164148 payload 824 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb936d76640 1 -- 10.20.193.78:0/2210200170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb938068410 con 0x7fb9381cf5d0 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb9177fe640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb90403deb0 0x7fb904040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(465..465 src has 335..465) ==== 4795+0+0 (secure 0 0 0) 0x7fb920052b00 con 0x7fb93806d1b0 2026-02-17T05:59:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb937577640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb90403deb0 0x7fb904040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.501+0000 7fb937577640 10 monclient: get_auth_request con 0x7fb90403deb0 auth_method 0 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.503+0000 7fb936d76640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9381cf5d0 msgr2=0x7fb93811cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.503+0000 7fb936d76640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9381cf5d0 0x7fb93811cc70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb92c03a040 tx=0x7fb92c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.503+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c03c040 con 0x7fb9381cf5d0 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.503+0000 7fb9177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.503+0000 7fb9177fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.503+0000 7fb9177fe640 10 monclient: dump: 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.504+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fb92c041410 con 0x7fb9381cf5d0 2026-02-17T05:59:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.504+0000 7fb9177fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.504+0000 7fb9177fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.504+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9381cf5d0 msgr2=0x7fb93811cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.504+0000 7fb9177fe640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9381cf5d0 0x7fb93811cc70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb92c03a040 tx=0x7fb92c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.504+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9381cf5d0 msgr2=0x7fb93811cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.504+0000 7fb9177fe640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9381cf5d0 0x7fb93811cc70 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb92c03a040 tx=0x7fb92c0047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.504+0000 7fb937577640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb90403deb0 0x7fb904040370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb9381272a0 tx=0x7fb928009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb93d088640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb93d088640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93811d1b0 0x7fb9381da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb93d088640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381277b0 con 0x7fb93811d1b0 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93811d1b0 0x7fb9381da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 10 monclient: get_auth_request con 0x7fb93811d1b0 auth_method 0 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 10 monclient(hunting): handle_auth_done global_id 164151 payload 824 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.507+0000 7fb936d76640 1 -- 10.20.193.78:0/2210200170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb93806cc70 con 0x7fb93811d1b0 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.509+0000 7fb936d76640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93811d1b0 msgr2=0x7fb9381da6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.509+0000 7fb936d76640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93811d1b0 0x7fb9381da6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb9380a41e0 tx=0x7fb92c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.509+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c050040 con 0x7fb93811d1b0 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.509+0000 7fb9177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.509+0000 7fb9177fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.509+0000 7fb9177fe640 10 monclient: dump: 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7fb92c003b90 con 0x7fb93811d1b0 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb9177fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb9177fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93811d1b0 msgr2=0x7fb9381da6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb9177fe640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93811d1b0 0x7fb9381da6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb9380a41e0 tx=0x7fb92c008030 comp rx=0 tx=0).stop 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb9177fe640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93811d1b0 msgr2=0x7fb9381da6d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb9177fe640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb93811d1b0 0x7fb9381da6d0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb9380a41e0 tx=0x7fb92c008030 comp rx=0 tx=0).stop 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 154, 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 28360, 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:09.522 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.523 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.524 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:09.525 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.526 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:09.527 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:09.528 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:09.529 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:09.530 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:09.532 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb90403deb0 msgr2=0x7fb904040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb90403deb0 0x7fb904040370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb9381272a0 tx=0x7fb928009040 comp rx=0 tx=0).stop 2026-02-17T05:59:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 10 monclient: shutdown 2026-02-17T05:59:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806d1b0 msgr2=0x7fb938134730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806d1b0 0x7fb938134730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c450 tx=0x7fb92000c920 comp rx=0 tx=0).stop 2026-02-17T05:59:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb937577640 1 -- 10.20.193.78:0/2210200170 reap_dead start 2026-02-17T05:59:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 shutdown_connections 2026-02-17T05:59:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 --2- 10.20.193.78:0/2210200170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb93806d1b0 0x7fb938134730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 >> 10.20.193.78:0/2210200170 conn(0x7fb9380a6620 msgr2=0x7fb938135460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 shutdown_connections 2026-02-17T05:59:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.510+0000 7fb93d088640 1 -- 10.20.193.78:0/2210200170 wait complete. 2026-02-17T05:59:09.534 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:09.534 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:59:09.535 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:59:09.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.599+0000 7f72a8ada640 10 monclient: get_monmap_and_config 2026-02-17T05:59:09.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.599+0000 7f72a8ada640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.599+0000 7f72a8ada640 10 monclient: monmap: 2026-02-17T05:59:09.602 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.602 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:09.600610+0000 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:09.600610+0000 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.599+0000 7f72a8ada640 1 Processor -- start 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.599+0000 7f72a8ada640 1 -- start start 2026-02-17T05:59:09.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.599+0000 7f72a8ada640 10 monclient: init 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f72a406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f72a409d430 0x7f72a413d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient(hunting): picked mon.noname-c con 0x7f72a409d430 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f72a409bf40 0x7f72a413fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient(hunting): picked mon.noname-a con 0x7f72a409bf40 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a4073600 con 0x7f72a406c890 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f72a409ca50 con 0x7f72a409bf40 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f72a406dab0 con 0x7f72a409d430 2026-02-17T05:59:09.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a8ada640 10 monclient(hunting): authenticate will time out at 1395.154121s 2026-02-17T05:59:09.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f72a409bf40 0x7f72a413fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f72a409bf40 0x7f72a413fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45956/0 (socket says 10.20.193.78:45956) 2026-02-17T05:59:09.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 1 -- 10.20.193.78:0/4190169627 learned_addr learned my addr 10.20.193.78:0/4190169627 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:09.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient(hunting): get_auth_request con 0x7f72a409bf40 auth_method 0 2026-02-17T05:59:09.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a1d74640 1 --2- 10.20.193.78:0/4190169627 >> v2:10.20.193.115:3300/0 conn(0x7f72a409d430 0x7f72a413d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient(hunting): handle_auth_done global_id 164154 payload 274 2026-02-17T05:59:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 1 -- 10.20.193.78:0/4190169627 >> v2:10.20.193.115:3300/0 conn(0x7f72a409d430 msgr2=0x7f72a413d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 1 --2- 10.20.193.78:0/4190169627 >> v2:10.20.193.115:3300/0 conn(0x7f72a409d430 0x7f72a413d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 1 -- 10.20.193.78:0/4190169627 >> v1:10.20.193.115:6789/0 conn(0x7f72a406c890 legacy=0x7f72a406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 1 monclient: found mon.noname-a 2026-02-17T05:59:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a2d76640 1 -- 10.20.193.78:0/4190169627 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a409c770 con 0x7f72a409bf40 2026-02-17T05:59:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a1d74640 10 monclient: get_auth_request con 0x7f72a409d430 auth_method 0 2026-02-17T05:59:09.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.600+0000 7f72a1d74640 1 --2- 10.20.193.78:0/4190169627 >> v2:10.20.193.115:3300/0 conn(0x7f72a409d430 0x7f72a413d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a2d76640 1 -- 10.20.193.78:0/4190169627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 msgr2=0x7f72a413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a2d76640 1 --2- 10.20.193.78:0/4190169627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 0x7f72a413fb20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f72a40a0ac0 tx=0x7f729402f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952ef965a9356c68 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 1 -- 10.20.193.78:0/4190169627 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729402fe50 con 0x7f72a409bf40 2026-02-17T05:59:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: dump: 2026-02-17T05:59:09.625 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.625 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.625 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.625 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.625 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.625 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.625 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.625 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a8ada640 5 monclient: authenticate success, global_id 164154 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 1 -- 10.20.193.78:0/4190169627 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72940373f0 con 0x7f72a409bf40 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a8ada640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:09.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a8ada640 4 set_mon_vals no callback set 2026-02-17T05:59:09.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 1 -- 10.20.193.78:0/4190169627 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940377c0 con 0x7f72a409bf40 2026-02-17T05:59:09.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:09.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.602+0000 7f72a1573640 10 monclient: dump: 2026-02-17T05:59:09.627 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.627 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.627 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 10 monclient: shutdown 2026-02-17T05:59:09.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 -- 10.20.193.78:0/4190169627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 msgr2=0x7f72a413fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 --2- 10.20.193.78:0/4190169627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 0x7f72a413fb20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f72a40a0ac0 tx=0x7f729402f1e0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 -- 10.20.193.78:0/4190169627 shutdown_connections 2026-02-17T05:59:09.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 --2- 10.20.193.78:0/4190169627 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 0x7f72a413fb20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 --2- 10.20.193.78:0/4190169627 >> v2:10.20.193.115:3300/0 conn(0x7f72a409d430 0x7f72a413d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 -- 10.20.193.78:0/4190169627 >> 10.20.193.78:0/4190169627 conn(0x7f72a40a6620 msgr2=0x7f72a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 -- 10.20.193.78:0/4190169627 shutdown_connections 2026-02-17T05:59:09.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 -- 10.20.193.78:0/4190169627 wait complete. 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 10 monclient: monmap: 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:09.630 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 Processor -- start 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 1 -- start start 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.603+0000 7f72a8ada640 10 monclient: init 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72a406c890 0x7f72a412f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient(hunting): picked mon.noname-b con 0x7f72a406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 0x7f72a412f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient(hunting): picked mon.noname-a con 0x7f72a409bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a409d430 0x7f72a41395b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient(hunting): picked mon.noname-c con 0x7f72a409d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72a406d380 con 0x7f72a409bf40 2026-02-17T05:59:09.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72a4073410 con 0x7f72a406c890 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f72a406d500 con 0x7f72a409d430 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a8ada640 10 monclient(hunting): authenticate will time out at 1395.158220s 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 0x7f72a412f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 0x7f72a412f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45966/0 (socket says 10.20.193.78:45966) 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 -- 10.20.193.78:0/2086263287 learned_addr learned my addr 10.20.193.78:0/2086263287 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient(hunting): get_auth_request con 0x7f72a409bf40 auth_method 0 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2d76640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a409d430 0x7f72a41395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2d76640 10 monclient(hunting): get_auth_request con 0x7f72a409d430 auth_method 0 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2575640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72a406c890 0x7f72a412f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient(hunting): handle_auth_done global_id 164157 payload 824 2026-02-17T05:59:09.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a409d430 msgr2=0x7f72a41395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a409d430 0x7f72a41395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72a406c890 msgr2=0x7f72a412f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72a406c890 0x7f72a412f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 monclient: found mon.noname-a 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a1d74640 1 -- 10.20.193.78:0/2086263287 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a413fa50 con 0x7f72a409bf40 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2575640 10 monclient: get_auth_request con 0x7f72a406c890 auth_method 0 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.604+0000 7f72a2575640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72a406c890 0x7f72a412f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a1d74640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 msgr2=0x7f72a412f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a1d74640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 0x7f72a412f620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c7d0 tx=0x7f728c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c015070 con 0x7f72a409bf40 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: dump: 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728c00acd0 con 0x7f72a409bf40 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a8ada640 5 monclient: authenticate success, global_id 164157 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a8ada640 10 monclient: _renew_subs 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a8ada640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a413aab0 con 0x7f72a409bf40 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c011630 con 0x7f72a409bf40 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f728affd640 10 monclient: dump: 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.636 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a0d72640 4 set_mon_vals no callback set 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a8ada640 10 monclient: _renew_subs 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a8ada640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.606+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a413ad30 con 0x7f72a409bf40 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a8ada640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a8ada640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a413bc20 0x7f72a412ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a8ada640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f72a413c330 con 0x7f72a413bc20 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a413bc20 0x7f72a412ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 10 monclient: get_auth_request con 0x7f72a413bc20 auth_method 0 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f728c0117d0 con 0x7f72a409bf40 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 10 monclient(hunting): handle_auth_done global_id 144317 payload 824 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2575640 1 -- 10.20.193.78:0/2086263287 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72a40a3dc0 con 0x7f72a413bc20 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f728affd640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f727003df30 0x7f72700403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(465..465 src has 335..465) ==== 4795+0+0 (secure 0 0 0) 0x7f728c052d10 con 0x7f72a409bf40 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2d76640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f727003df30 0x7f72700403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.607+0000 7f72a2d76640 10 monclient: get_auth_request con 0x7f727003df30 auth_method 0 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f72a2575640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a413bc20 msgr2=0x7f72a412ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f72a2575640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a413bc20 0x7f72a412ffb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f729800c720 tx=0x7f729800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298004480 con 0x7f72a413bc20 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 10 monclient: dump: 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.638 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7298016070 con 0x7f72a413bc20 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a413bc20 msgr2=0x7f72a412ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a413bc20 0x7f72a412ffb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f729800c720 tx=0x7f729800cbf0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a413bc20 msgr2=0x7f72a412ffb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.609+0000 7f728affd640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a413bc20 0x7f72a412ffb0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f729800c720 tx=0x7f729800cbf0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.610+0000 7f72a2d76640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f727003df30 0x7f72700403f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f72a406a480 tx=0x7f7294033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a8ada640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a8ada640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a41304f0 0x7f72a41da390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a8ada640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f72a413c000 con 0x7f72a41304f0 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a41304f0 0x7f72a41da390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 10 monclient: get_auth_request con 0x7f72a41304f0 auth_method 0 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 10 monclient(hunting): handle_auth_done global_id 144320 payload 824 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.613+0000 7f72a2575640 1 -- 10.20.193.78:0/2086263287 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f72a4074980 con 0x7f72a41304f0 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a2575640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a41304f0 msgr2=0x7f72a41da390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a2575640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a41304f0 0x7f72a41da390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f729800c720 tx=0x7f72980047f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298002b00 con 0x7f72a41304f0 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 10 monclient: dump: 2026-02-17T05:59:09.640 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f7298007690 con 0x7f72a41304f0 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a41304f0 msgr2=0x7f72a41da390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a41304f0 0x7f72a41da390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f729800c720 tx=0x7f72980047f0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a41304f0 msgr2=0x7f72a41da390 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:09.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f728affd640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72a41304f0 0x7f72a41da390 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f729800c720 tx=0x7f72980047f0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 154, 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 133150, 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:09.642 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.643 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:09.644 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:09.645 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.646 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.647 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.648 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.649 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.650 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.650 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:09.650 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f727003df30 msgr2=0x7f72700403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f727003df30 0x7f72700403f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f72a406a480 tx=0x7f7294033000 comp rx=0 tx=0).stop 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 10 monclient: shutdown 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 msgr2=0x7f72a412f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 1 --2- 10.20.193.78:0/2086263287 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72a409bf40 0x7f72a412f620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c7d0 tx=0x7f728c00cca0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a2d76640 1 -- 10.20.193.78:0/2086263287 reap_dead start 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a2d76640 1 -- 10.20.193.78:0/2086263287 reap_dead start 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 shutdown_connections 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 >> 10.20.193.78:0/2086263287 conn(0x7f72a40a6620 msgr2=0x7f72a40a26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 shutdown_connections 2026-02-17T05:59:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.615+0000 7f72a8ada640 1 -- 10.20.193.78:0/2086263287 wait complete. 2026-02-17T05:59:09.651 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:09.651 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:59:09.652 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2170: done (5 left) 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2171: delete oid 28 current snap is 219 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2163: finishing write tid 3 to trial11545420-48 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2163: finishing write tid 4 to trial11545420-48 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2163: finishing write tid 5 to trial11545420-48 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2163: oid 48 updating version 0 to 1220 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2163: oid 48 updating version 1220 to 1222 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2163: oid 48 version 1222 is already newer than 1219 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2163: oid 48 version 1222 is already newer than 1221 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1222 (ObjNum 771 snap 219 seq_num 771) dirty exists 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2163: left oid 48 (ObjNum 771 snap 219 seq_num 771) 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2165: finishing copy_from racing read to trial11545420-46 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2165: finishing copy_from to trial11545420-46 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 946 (ObjNum 762 snap 210 seq_num 762) dirty exists 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2167: finishing write tid 1 to trial11545420-43 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2167: finishing write tid 2 to trial11545420-43 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2167: finishing write tid 3 to trial11545420-43 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2167: finishing write tid 4 to trial11545420-43 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2167: finishing write tid 5 to trial11545420-43 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2167: oid 43 updating version 0 to 934 2026-02-17T05:59:09.658 INFO:tasks.rados.rados.0.trial115.stdout:2167: oid 43 updating version 934 to 937 2026-02-17T05:59:09.659 INFO:tasks.rados.rados.0.trial115.stdout:2167: oid 43 version 937 is already newer than 935 2026-02-17T05:59:09.659 INFO:tasks.rados.rados.0.trial115.stdout:2167: oid 43 version 937 is already newer than 936 2026-02-17T05:59:09.659 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 937 (ObjNum 773 snap 219 seq_num 773) dirty exists 2026-02-17T05:59:09.659 INFO:tasks.rados.rados.0.trial115.stdout:2167: left oid 43 (ObjNum 773 snap 219 seq_num 773) 2026-02-17T05:59:09.659 INFO:tasks.rados.rados.0.trial115.stdout:2168: expect (ObjNum 740 snap 204 seq_num 740) 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.720+0000 7f86aa5de640 10 monclient: get_monmap_and_config 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.720+0000 7f86aa5de640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.720+0000 7f86aa5de640 10 monclient: monmap: 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:09.721618+0000 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:09.721618+0000 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:09.721 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.720+0000 7f86aa5de640 1 Processor -- start 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.720+0000 7f86aa5de640 1 -- start start 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.720+0000 7f86aa5de640 10 monclient: init 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f86a406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f86a409c4a0 0x7f86a413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient(hunting): picked mon.noname-a con 0x7f86a409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f86a406d380 0x7f86a413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient(hunting): picked mon.noname-c con 0x7f86a406d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a406daf0 con 0x7f86a406c890 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f86a40a0880 con 0x7f86a409c4a0 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f86a409cc80 con 0x7f86a406d380 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86aa5de640 10 monclient(hunting): authenticate will time out at 1395.275152s 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a3fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f86a406c890 0x7f86a406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54584/0 (socket says 10.20.193.78:54584) 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a3fff640 1 -- 10.20.193.78:0/3918449715 learned_addr learned my addr 10.20.193.78:0/3918449715 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 1 --2- 10.20.193.78:0/3918449715 >> v2:10.20.193.78:3300/0 conn(0x7f86a409c4a0 0x7f86a413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient(hunting): get_auth_request con 0x7f86a409c4a0 auth_method 0 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a8b54640 1 --2- 10.20.193.78:0/3918449715 >> v2:10.20.193.115:3300/0 conn(0x7f86a406d380 0x7f86a413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a8b54640 10 monclient(hunting): get_auth_request con 0x7f86a406d380 auth_method 0 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a8b54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a8b54640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a8b54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient(hunting): handle_auth_done global_id 164160 payload 274 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 1 -- 10.20.193.78:0/3918449715 >> v2:10.20.193.115:3300/0 conn(0x7f86a406d380 msgr2=0x7f86a413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 1 --2- 10.20.193.78:0/3918449715 >> v2:10.20.193.115:3300/0 conn(0x7f86a406d380 0x7f86a413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 1 -- 10.20.193.78:0/3918449715 >> v1:10.20.193.78:6789/0 conn(0x7f86a406c890 legacy=0x7f86a406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 1 monclient: found mon.noname-a 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.721+0000 7f86a37fe640 1 -- 10.20.193.78:0/3918449715 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a409cf60 con 0x7f86a409c4a0 2026-02-17T05:59:09.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a37fe640 1 -- 10.20.193.78:0/3918449715 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 msgr2=0x7f86a413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a37fe640 1 --2- 10.20.193.78:0/3918449715 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 0x7f86a413b3a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f86a406dcc0 tx=0x7f869402ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb5de20fb3ddd30 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 1 -- 10.20.193.78:0/3918449715 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869403c070 con 0x7f86a409c4a0 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: dump: 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.726 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 1 -- 10.20.193.78:0/3918449715 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f869402fbd0 con 0x7f86a409c4a0 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 1 -- 10.20.193.78:0/3918449715 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694037440 con 0x7f86a409c4a0 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86a2ffd640 10 monclient: dump: 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.727 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86aa5de640 5 monclient: authenticate success, global_id 164160 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86aa5de640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.723+0000 7f86aa5de640 4 set_mon_vals no callback set 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 10 monclient: shutdown 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 -- 10.20.193.78:0/3918449715 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 msgr2=0x7f86a413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 --2- 10.20.193.78:0/3918449715 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 0x7f86a413b3a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f86a406dcc0 tx=0x7f869402ed50 comp rx=0 tx=0).stop 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 -- 10.20.193.78:0/3918449715 shutdown_connections 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 --2- 10.20.193.78:0/3918449715 >> v2:10.20.193.115:3300/0 conn(0x7f86a406d380 0x7f86a413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 --2- 10.20.193.78:0/3918449715 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 0x7f86a413b3a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 -- 10.20.193.78:0/3918449715 >> 10.20.193.78:0/3918449715 conn(0x7f86a40a6620 msgr2=0x7f86a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 -- 10.20.193.78:0/3918449715 shutdown_connections 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 -- 10.20.193.78:0/3918449715 wait complete. 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 10 monclient: monmap: 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:09.729 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.724+0000 7f86aa5de640 1 Processor -- start 2026-02-17T05:59:09.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 1 -- start start 2026-02-17T05:59:09.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient: init 2026-02-17T05:59:09.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86a406c890 0x7f86a41327c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient(hunting): picked mon.noname-c con 0x7f86a406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a406d380 0x7f86a409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient(hunting): picked mon.noname-b con 0x7f86a406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 0x7f86a409e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient(hunting): picked mon.noname-a con 0x7f86a409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86a409bfa0 con 0x7f86a409c4a0 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86a409be20 con 0x7f86a406d380 2026-02-17T05:59:09.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f86a413de30 con 0x7f86a406c890 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86aa5de640 10 monclient(hunting): authenticate will time out at 1395.279281s 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 0x7f86a409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 0x7f86a409e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:45982/0 (socket says 10.20.193.78:45982) 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86a406c890 0x7f86a41327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 -- 10.20.193.78:0/2353540249 learned_addr learned my addr 10.20.193.78:0/2353540249 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient(hunting): get_auth_request con 0x7f86a409c4a0 auth_method 0 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a3fff640 10 monclient(hunting): get_auth_request con 0x7f86a406c890 auth_method 0 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a37fe640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a406d380 0x7f86a409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a37fe640 10 monclient(hunting): get_auth_request con 0x7f86a406d380 auth_method 0 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient(hunting): handle_auth_done global_id 164163 payload 824 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86a406c890 msgr2=0x7f86a41327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f86a406c890 0x7f86a41327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a406d380 msgr2=0x7f86a409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a406d380 0x7f86a409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 monclient: found mon.noname-a 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.725+0000 7f86a8b54640 1 -- 10.20.193.78:0/2353540249 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a409f700 con 0x7f86a409c4a0 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a8b54640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 msgr2=0x7f86a409e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a8b54640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 0x7f86a409e090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8698007d90 tx=0x7f869800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698014620 con 0x7f86a409c4a0 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: dump: 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.739 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8698019070 con 0x7f86a409c4a0 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86aa5de640 5 monclient: authenticate success, global_id 164163 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698014a20 con 0x7f86a409c4a0 2026-02-17T05:59:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a27fc640 4 set_mon_vals no callback set 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86a0ff9640 10 monclient: dump: 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86aa5de640 10 monclient: _renew_subs 2026-02-17T05:59:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86aa5de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a413b5b0 con 0x7f86a409c4a0 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86aa5de640 10 monclient: _renew_subs 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86aa5de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.727+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a413daf0 con 0x7f86a409c4a0 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86aa5de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86aa5de640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a409e5d0 0x7f86a40717d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86aa5de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86a409e9b0 con 0x7f86a409e5d0 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8698014bc0 con 0x7f86a409c4a0 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a409e5d0 0x7f86a40717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 10 monclient: get_auth_request con 0x7f86a409e5d0 auth_method 0 2026-02-17T05:59:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a0ff9640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f867403deb0 0x7f8674040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(466..466 src has 335..466) ==== 4779+0+0 (secure 0 0 0) 0x7f8698052fe0 con 0x7f86a409c4a0 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 10 monclient(hunting): handle_auth_done global_id 134350 payload 824 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a3fff640 1 -- 10.20.193.78:0/2353540249 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86a4074980 con 0x7f86a409e5d0 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a37fe640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f867403deb0 0x7f8674040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.728+0000 7f86a37fe640 10 monclient: get_auth_request con 0x7f867403deb0 auth_method 0 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a3fff640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a409e5d0 msgr2=0x7f86a40717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a3fff640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a409e5d0 0x7f86a40717d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f869000eaa0 tx=0x7f869000ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869000cbf0 con 0x7f86a409e5d0 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 10 monclient: dump: 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f8690019070 con 0x7f86a409e5d0 2026-02-17T05:59:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a409e5d0 msgr2=0x7f86a40717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a409e5d0 0x7f86a40717d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f869000eaa0 tx=0x7f869000ef70 comp rx=0 tx=0).stop 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a409e5d0 msgr2=0x7f86a40717d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a0ff9640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a409e5d0 0x7f86a40717d0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f869000eaa0 tx=0x7f869000ef70 comp rx=0 tx=0).stop 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.731+0000 7f86a37fe640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f867403deb0 0x7f8674040370 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86a406a480 tx=0x7f8694033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86aa5de640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86aa5de640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a4071d10 0x7f86a4072120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86aa5de640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86a406cc70 con 0x7f86a4071d10 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a4071d10 0x7f86a4072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 10 monclient: get_auth_request con 0x7f86a4071d10 auth_method 0 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 10 monclient(hunting): handle_auth_done global_id 134353 payload 824 2026-02-17T05:59:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.734+0000 7f86a3fff640 1 -- 10.20.193.78:0/2353540249 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f86a4139990 con 0x7f86a4071d10 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a3fff640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a4071d10 msgr2=0x7f86a4072120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a3fff640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a4071d10 0x7f86a4072120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f869000eaa0 tx=0x7f8690004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690003230 con 0x7f86a4071d10 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 10 monclient: dump: 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.747 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f8690003230 con 0x7f86a4071d10 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a4071d10 msgr2=0x7f86a4072120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a4071d10 0x7f86a4072120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f869000eaa0 tx=0x7f8690004a40 comp rx=0 tx=0).stop 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a4071d10 msgr2=0x7f86a4072120 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a0ff9640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f86a4071d10 0x7f86a4072120 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f869000eaa0 tx=0x7f8690004a40 comp rx=0 tx=0).stop 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:09.748 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 154, 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 177456, 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.749 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.750 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:09.751 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:09.752 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.753 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.754 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:09.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.757 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:09.758 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f867403deb0 msgr2=0x7f8674040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f867403deb0 0x7f8674040370 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86a406a480 tx=0x7f8694033000 comp rx=0 tx=0).stop 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 10 monclient: shutdown 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 msgr2=0x7f86a409e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 1 --2- 10.20.193.78:0/2353540249 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f86a409c4a0 0x7f86a409e090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8698007d90 tx=0x7f869800e4b0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a8b54640 1 -- 10.20.193.78:0/2353540249 reap_dead start 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86a8b54640 1 -- 10.20.193.78:0/2353540249 reap_dead start 2026-02-17T05:59:09.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 shutdown_connections 2026-02-17T05:59:09.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 >> 10.20.193.78:0/2353540249 conn(0x7f86a40a6620 msgr2=0x7f86a4136190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 shutdown_connections 2026-02-17T05:59:09.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.737+0000 7f86aa5de640 1 -- 10.20.193.78:0/2353540249 wait complete. 2026-02-17T05:59:09.760 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:09.760 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:59:09.760 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:59:09.760 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:59:09.760 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:59:09.760 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:59:09.760 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:59:09.761 INFO:tasks.rados.rados.0.trial115.stdout:2163: done (5 left) 2026-02-17T05:59:09.761 INFO:tasks.rados.rados.0.trial115.stdout:2165: done (4 left) 2026-02-17T05:59:09.761 INFO:tasks.rados.rados.0.trial115.stdout:2167: done (3 left) 2026-02-17T05:59:09.761 INFO:tasks.rados.rados.0.trial115.stdout:2168: done (2 left) 2026-02-17T05:59:09.761 INFO:tasks.rados.rados.0.trial115.stdout:2169: done (1 left) 2026-02-17T05:59:09.761 INFO:tasks.rados.rados.0.trial115.stdout:2171: done (0 left) 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2172: rollback oid 38 current snap is 219 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 38 to 215 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2173: read oid 45 snap 215 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2173: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2174: read oid 26 snap -1 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2174: expect (ObjNum 741 snap 204 seq_num 741) 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2175: read oid 40 snap -1 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2175: expect (ObjNum 742 snap 204 seq_num 742) 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2176: write oid 46 current snap is 219 2026-02-17T05:59:09.762 INFO:tasks.rados.rados.0.trial115.stdout:2176: seq_num 775 ranges {625425=469628,1768133=640265} 2026-02-17T05:59:09.764 INFO:tasks.rados.rados.0.trial115.stdout:2176: writing trial11545420-46 from 625425 to 1095053 tid 1 2026-02-17T05:59:09.770 INFO:tasks.rados.rados.0.trial115.stdout:2176: writing trial11545420-46 from 1768133 to 2408398 tid 2 2026-02-17T05:59:09.770 INFO:tasks.rados.rados.0.trial115.stdout:2177: write (excl) oid 15 current snap is 219 2026-02-17T05:59:09.771 INFO:tasks.rados.rados.0.trial115.stdout:2177: seq_num 776 ranges {642232=661230,1942968=783923} 2026-02-17T05:59:09.771 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:59:09.771 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:59:09.771 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:09.775 INFO:tasks.rados.rados.0.trial115.stdout:2177: writing trial11545420-15 from 642232 to 1303462 tid 1 2026-02-17T05:59:09.780 INFO:tasks.rados.rados.0.trial115.stdout:2177: writing trial11545420-15 from 1942968 to 2726891 tid 2 2026-02-17T05:59:09.780 INFO:tasks.rados.rados.0.trial115.stdout:2178: write oid 5 current snap is 219 2026-02-17T05:59:09.780 INFO:tasks.rados.rados.0.trial115.stdout:2178: seq_num 777 ranges {659039=736479,2001450=643934} 2026-02-17T05:59:09.785 INFO:tasks.rados.rados.0.trial115.stdout:2178: writing trial11545420-5 from 659039 to 1395518 tid 1 2026-02-17T05:59:09.790 INFO:tasks.rados.rados.0.trial115.stdout:2178: writing trial11545420-5 from 2001450 to 2645384 tid 2 2026-02-17T05:59:09.790 INFO:tasks.rados.rados.0.trial115.stdout:2179: read oid 35 snap -1 2026-02-17T05:59:09.790 INFO:tasks.rados.rados.0.trial115.stdout:2179: expect (ObjNum 760 snap 207 seq_num 760) 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:2180: rollback oid 17 current snap is 219 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 217 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:2172: finishing rollback tid 0 to trial11545420-38 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:2172: finishing rollback tid 1 to trial11545420-38 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:2172: finishing rollback tid 2 to trial11545420-38 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1232 (ObjNum 749 snap 205 seq_num 749) dirty exists 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:2172: done (8 left) 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:2181: rollback oid 44 current snap is 219 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 44 to 216 2026-02-17T05:59:09.791 INFO:tasks.rados.rados.0.trial115.stdout:2173: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:59:09.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.877+0000 7fd0684a0640 10 monclient: get_monmap_and_config 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.877+0000 7fd0684a0640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.877+0000 7fd0684a0640 10 monclient: monmap: 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:09.879140+0000 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:09.879140+0000 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:09.879 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 1 Processor -- start 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 1 -- start start 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient: init 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd0600a3d00 0x7fd060134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient(hunting): picked mon.noname-c con 0x7fd0600a3d00 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0600a1cb0 0x7fd060072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient(hunting): picked mon.noname-b con 0x7fd0600a1cb0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd060137de0 0x7fd060138250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient(hunting): picked mon.noname-a con 0x7fd060137de0 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd0601348f0 con 0x7fd060137de0 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd0600a40e0 con 0x7fd0600a3d00 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0601353a0 con 0x7fd0600a1cb0 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd0684a0640 10 monclient(hunting): authenticate will time out at 1395.432740s 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd066a16640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fd060137de0 msgr2=0x7fd060138250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd066a16640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd060137de0 0x7fd060138250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0600a1cb0 0x7fd060072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0600a1cb0 0x7fd060072e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58248/0 (socket says 10.20.193.78:58248) 2026-02-17T05:59:09.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 1 -- 10.20.193.78:0/3532312526 learned_addr learned my addr 10.20.193.78:0/3532312526 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 10 monclient(hunting): get_auth_request con 0x7fd0600a1cb0 auth_method 0 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd066215640 1 --2- 10.20.193.78:0/3532312526 >> v2:10.20.193.115:3300/0 conn(0x7fd0600a3d00 0x7fd060134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd065a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd066215640 10 monclient(hunting): get_auth_request con 0x7fd0600a3d00 auth_method 0 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd066215640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd066215640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.878+0000 7fd066215640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 10 monclient(hunting): handle_auth_done global_id 144326 payload 274 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 1 -- 10.20.193.78:0/3532312526 >> v2:10.20.193.115:3300/0 conn(0x7fd0600a3d00 msgr2=0x7fd060134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 1 --2- 10.20.193.78:0/3532312526 >> v2:10.20.193.115:3300/0 conn(0x7fd0600a3d00 0x7fd060134ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 1 -- 10.20.193.78:0/3532312526 >> v2:10.20.193.78:3300/0 conn(0x7fd060137de0 msgr2=0x7fd060138250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 1 --2- 10.20.193.78:0/3532312526 >> v2:10.20.193.78:3300/0 conn(0x7fd060137de0 0x7fd060138250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 1 monclient: found mon.noname-b 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.879+0000 7fd065a14640 1 -- 10.20.193.78:0/3532312526 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0600a2d90 con 0x7fd0600a1cb0 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065a14640 1 -- 10.20.193.78:0/3532312526 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0600a1cb0 msgr2=0x7fd060072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065a14640 1 --2- 10.20.193.78:0/3532312526 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0600a1cb0 0x7fd060072e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0600a79a0 tx=0x7fd05402ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb818d51fed32e0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 1 -- 10.20.193.78:0/3532312526 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05403c070 con 0x7fd0600a1cb0 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: dump: 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.882 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 1 -- 10.20.193.78:0/3532312526 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05402fcc0 con 0x7fd0600a1cb0 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 5 monclient: authenticate success, global_id 144326 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 1 -- 10.20.193.78:0/3532312526 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054037660 con 0x7fd0600a1cb0 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd065213640 10 monclient: dump: 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 4 set_mon_vals no callback set 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 10 monclient: shutdown 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:09.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 -- 10.20.193.78:0/3532312526 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0600a1cb0 msgr2=0x7fd060072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 --2- 10.20.193.78:0/3532312526 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0600a1cb0 0x7fd060072e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0600a79a0 tx=0x7fd05402ee80 comp rx=0 tx=0).stop 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 -- 10.20.193.78:0/3532312526 shutdown_connections 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 --2- 10.20.193.78:0/3532312526 >> v2:10.20.193.78:3300/0 conn(0x7fd060137de0 0x7fd060138250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 --2- 10.20.193.78:0/3532312526 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0600a1cb0 0x7fd060072e60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 --2- 10.20.193.78:0/3532312526 >> v2:10.20.193.115:3300/0 conn(0x7fd0600a3d00 0x7fd060134ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 -- 10.20.193.78:0/3532312526 >> 10.20.193.78:0/3532312526 conn(0x7fd0600a7120 msgr2=0x7fd0600a35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 -- 10.20.193.78:0/3532312526 shutdown_connections 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.881+0000 7fd0684a0640 1 -- 10.20.193.78:0/3532312526 wait complete. 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient: build_initial_monmap 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient: monmap: 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 1 Processor -- start 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 1 -- start start 2026-02-17T05:59:09.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient: init 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0600a1cb0 0x7fd0601cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient(hunting): picked mon.noname-b con 0x7fd0600a1cb0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0600a3d00 0x7fd06013e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient(hunting): picked mon.noname-a con 0x7fd0600a3d00 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd060137de0 0x7fd06013e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient(hunting): picked mon.noname-c con 0x7fd060137de0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd060138b20 con 0x7fd0600a3d00 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601389a0 con 0x7fd0600a1cb0 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd060073480 con 0x7fd060137de0 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd0684a0640 10 monclient(hunting): authenticate will time out at 1395.436811s 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd065a14640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0600a3d00 msgr2=0x7fd06013e480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd065a14640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0600a3d00 0x7fd06013e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd060137de0 0x7fd06013e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd060137de0 0x7fd06013e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58254/0 (socket says 10.20.193.78:58254) 2026-02-17T05:59:09.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 1 -- 10.20.193.78:0/503623178 learned_addr learned my addr 10.20.193.78:0/503623178 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 10 monclient(hunting): get_auth_request con 0x7fd060137de0 auth_method 0 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.882+0000 7fd066a16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 10 monclient(hunting): handle_auth_done global_id 144329 payload 824 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 1 -- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0600a1cb0 msgr2=0x7fd0601cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0600a1cb0 0x7fd0601cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 1 -- 10.20.193.78:0/503623178 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0600a3d00 msgr2=0x7fd06013e480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0600a3d00 0x7fd06013e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 10 monclient: _finish_hunting 0 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 1 monclient: found mon.noname-c 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.883+0000 7fd066a16640 1 -- 10.20.193.78:0/503623178 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06013cc70 con 0x7fd060137de0 2026-02-17T05:59:09.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd066a16640 1 -- 10.20.193.78:0/503623178 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd060137de0 msgr2=0x7fd06013e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:09.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd066a16640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd060137de0 0x7fd06013e9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00be90 tx=0x7fd05c00bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 1 -- 10.20.193.78:0/503623178 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c006e60 con 0x7fd060137de0 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: dump: 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: _finish_auth 0 2026-02-17T05:59:09.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 1 -- 10.20.193.78:0/503623178 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05c010030 con 0x7fd060137de0 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 5 monclient: authenticate success, global_id 144329 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 1 -- 10.20.193.78:0/503623178 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c016870 con 0x7fd060137de0 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd064a12640 4 set_mon_vals no callback set 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd046ffd640 10 monclient: dump: 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:09.890 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 10 monclient: _renew_subs 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06013b750 con 0x7fd060137de0 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 10 monclient: _renew_subs 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0600a0a80 con 0x7fd060137de0 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.885+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06013c6b0 con 0x7fd060137de0 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.886+0000 7fd046ffd640 1 -- 10.20.193.78:0/503623178 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd05c008070 con 0x7fd060137de0 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.886+0000 7fd046ffd640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03003deb0 0x7fd030040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.886+0000 7fd046ffd640 1 -- 10.20.193.78:0/503623178 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(466..466 src has 335..466) ==== 4779+0+0 (secure 0 0 0) 0x7fd05c052ac0 con 0x7fd060137de0 2026-02-17T05:59:09.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.886+0000 7fd066215640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03003deb0 0x7fd030040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:09.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.886+0000 7fd066215640 10 monclient: get_auth_request con 0x7fd03003deb0 auth_method 0 2026-02-17T05:59:09.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.888+0000 7fd046ffd640 1 -- 10.20.193.78:0/503623178 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd05c01b020 con 0x7fd060137de0 2026-02-17T05:59:09.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.888+0000 7fd046ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:09.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.888+0000 7fd046ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:09.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:09.889+0000 7fd066215640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03003deb0 0x7fd030040370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd050006fd0 tx=0x7fd050008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:09.984 INFO:tasks.rados.rados.0.trial115.stdout:2174: expect (ObjNum 741 snap 204 seq_num 741) 2026-02-17T05:59:10.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.007+0000 7fd0684a0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:10.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.007+0000 7fd0684a0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:10.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.007+0000 7fd0684a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:10.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.007+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd060134ce0 con 0x7fd060137de0 2026-02-17T05:59:10.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd046ffd640 1 -- 10.20.193.78:0/503623178 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd05c01ed70 con 0x7fd060137de0 2026-02-17T05:59:10.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd046ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:10.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd046ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:10.010 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":154,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:10.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03003deb0 msgr2=0x7fd030040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03003deb0 0x7fd030040370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd050006fd0 tx=0x7fd050008040 comp rx=0 tx=0).stop 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 10 monclient: shutdown 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd060137de0 msgr2=0x7fd06013e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd060137de0 0x7fd06013e9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00be90 tx=0x7fd05c00bec0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 shutdown_connections 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd060137de0 0x7fd06013e9c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0600a3d00 0x7fd06013e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd03003deb0 0x7fd030040370 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 --2- 10.20.193.78:0/503623178 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0600a1cb0 0x7fd0601cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:10.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 >> 10.20.193.78:0/503623178 conn(0x7fd0600a7120 msgr2=0x7fd060132b90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:10.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 shutdown_connections 2026-02-17T05:59:10.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:10.008+0000 7fd0684a0640 1 -- 10.20.193.78:0/503623178 wait complete. 2026-02-17T05:59:10.103 INFO:tasks.rados.rados.0.trial115.stdout:2175: expect (ObjNum 742 snap 204 seq_num 742) 2026-02-17T05:59:10.183 INFO:tasks.rados.rados.0.trial115.stdout:2176: finishing write tid 1 to trial11545420-46 2026-02-17T05:59:10.183 INFO:tasks.rados.rados.0.trial115.stdout:2176: finishing write tid 2 to trial11545420-46 2026-02-17T05:59:10.183 INFO:tasks.rados.rados.0.trial115.stdout:2173: done (8 left) 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2174: done (7 left) 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2175: done (6 left) 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2182: copy_from oid 26 from oid 3 current snap is 219 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2176: finishing write tid 3 to trial11545420-46 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2176: finishing write tid 4 to trial11545420-46 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2176: oid 46 updating version 0 to 1135 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2176: oid 46 updating version 1135 to 1136 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2176: oid 46 version 1136 is already newer than 1134 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1136 (ObjNum 775 snap 219 seq_num 775) dirty exists 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2176: left oid 46 (ObjNum 775 snap 219 seq_num 775) 2026-02-17T05:59:10.184 INFO:tasks.rados.rados.0.trial115.stdout:2177: finishing write tid 1 to trial11545420-15 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2177: finishing write tid 2 to trial11545420-15 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2177: finishing write tid 3 to trial11545420-15 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2177: finishing write tid 4 to trial11545420-15 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2177: oid 15 updating version 0 to 1138 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2177: oid 15 updating version 1138 to 1139 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2177: oid 15 version 1139 is already newer than 1137 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1139 (ObjNum 776 snap 219 seq_num 776) dirty exists 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2177: left oid 15 (ObjNum 776 snap 219 seq_num 776) 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2178: finishing write tid 1 to trial11545420-5 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2180: finishing rollback tid 1 to trial11545420-17 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2178: finishing write tid 2 to trial11545420-5 2026-02-17T05:59:10.185 INFO:tasks.rados.rados.0.trial115.stdout:2178: finishing write tid 3 to trial11545420-5 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2178: finishing write tid 4 to trial11545420-5 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2178: oid 5 updating version 0 to 1055 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2178: oid 5 version 1055 is already newer than 1054 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2178: oid 5 version 1055 is already newer than 1053 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1055 (ObjNum 777 snap 219 seq_num 777) dirty exists 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2178: left oid 5 (ObjNum 777 snap 219 seq_num 777) 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2181: finishing rollback tid 0 to trial11545420-44 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2181: finishing rollback tid 1 to trial11545420-44 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2181: finishing rollback tid 2 to trial11545420-44 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 880 (ObjNum 740 snap 204 seq_num 740) dirty exists 2026-02-17T05:59:10.186 INFO:tasks.rados.rados.0.trial115.stdout:2179: expect (ObjNum 760 snap 207 seq_num 760) 2026-02-17T05:59:10.365 INFO:tasks.rados.rados.0.trial115.stdout:2182: finishing copy_from racing read to trial11545420-26 2026-02-17T05:59:10.365 INFO:tasks.rados.rados.0.trial115.stdout:2182: finishing copy_from to trial11545420-26 2026-02-17T05:59:10.365 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 950 (ObjNum 733 snap 202 seq_num 733) dirty exists 2026-02-17T05:59:10.365 INFO:tasks.rados.rados.0.trial115.stdout:2176: done (6 left) 2026-02-17T05:59:10.366 INFO:tasks.rados.rados.0.trial115.stdout:2177: done (5 left) 2026-02-17T05:59:10.366 INFO:tasks.rados.rados.0.trial115.stdout:2178: done (4 left) 2026-02-17T05:59:10.366 INFO:tasks.rados.rados.0.trial115.stdout:2179: done (3 left) 2026-02-17T05:59:10.366 INFO:tasks.rados.rados.0.trial115.stdout:2180: done (2 left) 2026-02-17T05:59:10.366 INFO:tasks.rados.rados.0.trial115.stdout:2181: done (1 left) 2026-02-17T05:59:10.366 INFO:tasks.rados.rados.0.trial115.stdout:2182: done (0 left) 2026-02-17T05:59:10.366 INFO:tasks.rados.rados.0.trial115.stdout:2183: copy_from oid 46 from oid 1 current snap is 219 2026-02-17T05:59:10.366 INFO:tasks.rados.rados.0.trial115.stdout:2184: delete oid 40 current snap is 219 2026-02-17T05:59:10.368 INFO:tasks.rados.rados.0.trial115.stdout:2184: done (1 left) 2026-02-17T05:59:10.368 INFO:tasks.rados.rados.0.trial115.stdout:2185: write (excl) oid 17 current snap is 219 2026-02-17T05:59:10.368 INFO:tasks.rados.rados.0.trial115.stdout:2185: seq_num 779 ranges {692653=486977,1602061=647603,2992629=100024} 2026-02-17T05:59:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2185: writing trial11545420-17 from 692653 to 1179630 tid 1 2026-02-17T05:59:10.376 INFO:tasks.rados.rados.0.trial115.stdout:2185: writing trial11545420-17 from 1602061 to 2249664 tid 2 2026-02-17T05:59:10.380 INFO:tasks.rados.rados.0.trial115.stdout:2185: writing trial11545420-17 from 2992629 to 3092653 tid 3 2026-02-17T05:59:10.380 INFO:tasks.rados.rados.0.trial115.stdout:2186: snap_remove snap 215 2026-02-17T05:59:13.018 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:13.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient: get_monmap_and_config 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient: build_initial_monmap 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient: monmap: 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:13.088334+0000 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:13.088334+0000 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:13.091 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 1 Processor -- start 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 1 -- start start 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient: init 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdea409ea90 0x7fdea40a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient(hunting): picked mon.noname-a con 0x7fdea409ea90 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:13.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdea409ee70 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fdea413d640 0x7fdea413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient(hunting): picked mon.noname-c con 0x7fdea413d640 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea40a0a20 con 0x7fdea409ee70 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fdea40a0d00 con 0x7fdea409ea90 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fdea40a0460 con 0x7fdea413d640 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdeab579640 10 monclient(hunting): authenticate will time out at 1398.641897s 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdea92ee640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fdea409ea90 msgr2=0x7fdea40a13d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:13.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.087+0000 7fdea92ee640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdea409ea90 0x7fdea40a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fdea413d640 0x7fdea413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea8aed640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7fdea409ee70 0x7fdea409f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:43996/0 (socket says 10.20.193.78:43996) 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea8aed640 1 -- 10.20.193.78:0/1246264258 learned_addr learned my addr 10.20.193.78:0/1246264258 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient(hunting): get_auth_request con 0x7fdea413d640 auth_method 0 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient(hunting): handle_auth_done global_id 134359 payload 274 2026-02-17T05:59:13.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 1 -- 10.20.193.78:0/1246264258 >> v2:10.20.193.78:3300/0 conn(0x7fdea409ea90 msgr2=0x7fdea40a13d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 1 --2- 10.20.193.78:0/1246264258 >> v2:10.20.193.78:3300/0 conn(0x7fdea409ea90 0x7fdea40a13d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 1 -- 10.20.193.78:0/1246264258 >> v1:10.20.193.115:6789/0 conn(0x7fdea409ee70 legacy=0x7fdea409f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient: _finish_hunting 0 2026-02-17T05:59:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 1 monclient: found mon.noname-c 2026-02-17T05:59:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:59:13.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.088+0000 7fdea9aef640 1 -- 10.20.193.78:0/1246264258 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea40a0740 con 0x7fdea413d640 2026-02-17T05:59:13.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdea9aef640 1 -- 10.20.193.78:0/1246264258 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea413d640 msgr2=0x7fdea413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdea9aef640 1 --2- 10.20.193.78:0/1246264258 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea413d640 0x7fdea413fa30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdea4135c40 tx=0x7fdea002f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671f0002abfa843 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 1 -- 10.20.193.78:0/1246264258 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea003c070 con 0x7fdea413d640 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: dump: 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:13.102 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 5 monclient: authenticate success, global_id 134359 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 1 -- 10.20.193.78:0/1246264258 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea0037640 con 0x7fdea413d640 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 1 -- 10.20.193.78:0/1246264258 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0037a10 con 0x7fdea413d640 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fde93fff640 10 monclient: dump: 2026-02-17T05:59:13.103 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 4 set_mon_vals no callback set 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 10 monclient: shutdown 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 1 -- 10.20.193.78:0/1246264258 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea413d640 msgr2=0x7fdea413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 1 --2- 10.20.193.78:0/1246264258 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea413d640 0x7fdea413fa30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdea4135c40 tx=0x7fdea002f150 comp rx=0 tx=0).stop 2026-02-17T05:59:13.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 1 -- 10.20.193.78:0/1246264258 shutdown_connections 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 1 --2- 10.20.193.78:0/1246264258 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea413d640 0x7fdea413fa30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 1 --2- 10.20.193.78:0/1246264258 >> v2:10.20.193.78:3300/0 conn(0x7fdea409ea90 0x7fdea40a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 1 -- 10.20.193.78:0/1246264258 >> 10.20.193.78:0/1246264258 conn(0x7fdea40a7520 msgr2=0x7fdea40a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 1 -- 10.20.193.78:0/1246264258 shutdown_connections 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.091+0000 7fdeab579640 1 -- 10.20.193.78:0/1246264258 wait complete. 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient: build_initial_monmap 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient: monmap: 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:13.105 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 1 Processor -- start 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 1 -- start start 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient: init 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea409ea90 0x7fdea41cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient(hunting): picked mon.noname-b con 0x7fdea409ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdea409ee70 0x7fdea4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient(hunting): picked mon.noname-a con 0x7fdea409ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 0x7fdea413b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient(hunting): picked mon.noname-c con 0x7fdea413d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdea40a6820 con 0x7fdea409ee70 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdea40a66a0 con 0x7fdea409ea90 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdea413ff70 con 0x7fdea413d640 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdeab579640 10 monclient(hunting): authenticate will time out at 1398.646864s 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea8aed640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdea409ee70 msgr2=0x7fdea4073590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea8aed640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdea409ee70 0x7fdea4073590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea9aef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 0x7fdea413b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea9aef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 0x7fdea413b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58258/0 (socket says 10.20.193.78:58258) 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea9aef640 1 -- 10.20.193.78:0/1444423456 learned_addr learned my addr 10.20.193.78:0/1444423456 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea9aef640 10 monclient(hunting): get_auth_request con 0x7fdea413d640 auth_method 0 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea9aef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea9aef640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.092+0000 7fdea9aef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:13.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea92ee640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea409ea90 0x7fdea41cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea92ee640 10 monclient(hunting): get_auth_request con 0x7fdea409ea90 auth_method 0 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea92ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea92ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea92ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 10 monclient(hunting): handle_auth_done global_id 144332 payload 824 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 1 -- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea409ea90 msgr2=0x7fdea41cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea409ea90 0x7fdea41cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 1 -- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdea409ee70 msgr2=0x7fdea4073590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdea409ee70 0x7fdea4073590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 10 monclient: _finish_hunting 0 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 1 monclient: found mon.noname-c 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.093+0000 7fdea9aef640 1 -- 10.20.193.78:0/1444423456 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea4070a90 con 0x7fdea413d640 2026-02-17T05:59:13.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdea9aef640 1 -- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 msgr2=0x7fdea413b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdea9aef640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 0x7fdea413b940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdea002f660 tx=0x7fdea0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdea9aef640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 0x7fdea413b940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdea002f660 tx=0x7fdea0004b40 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fdea0047020 mon_map magic: 0 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 1 -- 10.20.193.78:0/1444423456 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0047020 con 0x7fdea413d640 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 10 monclient: dump: 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:13.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 1 -- 10.20.193.78:0/1444423456 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea0037e10 con 0x7fdea413d640 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdeab579640 5 monclient: authenticate success, global_id 144332 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdeab579640 10 monclient: _renew_subs 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdeab579640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea41cf570 con 0x7fdea413d640 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde91ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fde937fe640 4 set_mon_vals no callback set 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdeab579640 10 monclient: _renew_subs 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdeab579640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.095+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea40707b0 con 0x7fdea413d640 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fdeab579640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fdeab579640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fdeab579640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea40a13b0 con 0x7fdea413d640 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fde91ffb640 1 -- 10.20.193.78:0/1444423456 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea003c040 con 0x7fdea413d640 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fde91ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:13.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fde91ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fde91ffb640 10 monclient: dump: 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fde91ffb640 1 -- 10.20.193.78:0/1444423456 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fdea0035ae0 con 0x7fdea413d640 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fde91ffb640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde7403de60 0x7fde74040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fdea92ee640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde7403de60 0x7fde74040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.096+0000 7fdea92ee640 10 monclient: get_auth_request con 0x7fde7403de60 auth_method 0 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.098+0000 7fdea9aef640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 0x7fdea413b940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdea002f660 tx=0x7fdea0004b40 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fdea007f080 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) 2026-02-17T05:59:13.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.099+0000 7fdea92ee640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde7403de60 0x7fde74040320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde94003a80 tx=0x7fde94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:13.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.719+0000 7fde91ffb640 1 -- 10.20.193.78:0/1444423456 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(466..466 src has 335..466) ==== 4779+0+0 (secure 0 0 0) 0x7fdea0078ca0 con 0x7fdea413d640 2026-02-17T05:59:13.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.925+0000 7fde91ffb640 1 -- 10.20.193.78:0/1444423456 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdea007f080 con 0x7fdea413d640 2026-02-17T05:59:13.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.925+0000 7fde91ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:13.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:13.925+0000 7fde91ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:14.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.037+0000 7fdeab579640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:14.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.037+0000 7fdeab579640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:14.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.037+0000 7fdeab579640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:14.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.037+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdea41cfeb0 con 0x7fdea413d640 2026-02-17T05:59:14.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fde91ffb640 1 -- 10.20.193.78:0/1444423456 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fdea0051020 con 0x7fdea413d640 2026-02-17T05:59:14.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fde91ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:14.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fde91ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:14.039 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":154,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde7403de60 msgr2=0x7fde74040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fdeab579640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde7403de60 0x7fde74040320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde94003a80 tx=0x7fde94008040 comp rx=0 tx=0).stop 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fdeab579640 10 monclient: shutdown 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fdeab579640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 msgr2=0x7fdea413b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.038+0000 7fdeab579640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 0x7fdea413b940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdea002f660 tx=0x7fdea0004b40 comp rx=0 tx=0).stop 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.039+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 shutdown_connections 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.039+0000 7fdeab579640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdea413d640 0x7fdea413b940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.039+0000 7fdeab579640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdea409ee70 0x7fdea4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.039+0000 7fdeab579640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde7403de60 0x7fde74040320 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:14.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.039+0000 7fdeab579640 1 --2- 10.20.193.78:0/1444423456 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdea409ea90 0x7fdea41cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:14.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.039+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 >> 10.20.193.78:0/1444423456 conn(0x7fdea40a7520 msgr2=0x7fdea40aa220 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:14.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.039+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 shutdown_connections 2026-02-17T05:59:14.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:14.039+0000 7fdeab579640 1 -- 10.20.193.78:0/1444423456 wait complete. 2026-02-17T05:59:17.048 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:17.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.116+0000 7f8594096640 10 monclient: get_monmap_and_config 2026-02-17T05:59:17.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.116+0000 7f8594096640 10 monclient: build_initial_monmap 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.116+0000 7f8594096640 10 monclient: monmap: 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:17.117799+0000 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:17.117799+0000 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:17.121 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.116+0000 7f8594096640 1 Processor -- start 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 1 -- start start 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient: init 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient(hunting): picked mon.noname-b con 0x7f858c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f858c09d430 0x7f858c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient(hunting): picked mon.noname-a con 0x7f858c09d430 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f858c09bf40 0x7f858c134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient(hunting): picked mon.noname-c con 0x7f858c09bf40 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f858c09ca50 con 0x7f858c09d430 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f858c06d6f0 con 0x7f858c09bf40 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f858c073600 con 0x7f858c06c890 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8594096640 10 monclient(hunting): authenticate will time out at 1402.671375s 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f859160a640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f858c09d430 msgr2=0x7f858c06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:17.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f859160a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f858c09d430 0x7f858c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58270/0 (socket says 10.20.193.78:58270) 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 -- 10.20.193.78:0/4111484852 learned_addr learned my addr 10.20.193.78:0/4111484852 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient(hunting): get_auth_request con 0x7f858c06c890 auth_method 0 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:17.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f859260c640 1 --2- 10.20.193.78:0/4111484852 >> v2:10.20.193.115:3300/0 conn(0x7f858c09bf40 0x7f858c134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f859260c640 10 monclient(hunting): get_auth_request con 0x7f858c09bf40 auth_method 0 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f859260c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f859260c640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f859260c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient(hunting): handle_auth_done global_id 144335 payload 274 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 -- 10.20.193.78:0/4111484852 >> v2:10.20.193.115:3300/0 conn(0x7f858c09bf40 msgr2=0x7f858c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 --2- 10.20.193.78:0/4111484852 >> v2:10.20.193.115:3300/0 conn(0x7f858c09bf40 0x7f858c134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 -- 10.20.193.78:0/4111484852 >> v2:10.20.193.78:3300/0 conn(0x7f858c09d430 msgr2=0x7f858c06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 --2- 10.20.193.78:0/4111484852 >> v2:10.20.193.78:3300/0 conn(0x7f858c09d430 0x7f858c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient: _finish_hunting 0 2026-02-17T05:59:17.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 monclient: found mon.noname-b 2026-02-17T05:59:17.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:17.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:17.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.117+0000 7f8591e0b640 1 -- 10.20.193.78:0/4111484852 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c06dab0 con 0x7f858c06c890 2026-02-17T05:59:17.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8591e0b640 1 -- 10.20.193.78:0/4111484852 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 msgr2=0x7f858c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:17.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8591e0b640 1 --2- 10.20.193.78:0/4111484852 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c06cc70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f858c09c670 tx=0x7f857c02ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496f371c235a9404 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 1 -- 10.20.193.78:0/4111484852 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c03c070 con 0x7f858c06c890 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: dump: 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: _finish_auth 0 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: _check_auth_tickets 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 1 -- 10.20.193.78:0/4111484852 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c02fcc0 con 0x7f858c06c890 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 5 monclient: authenticate success, global_id 144335 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 1 -- 10.20.193.78:0/4111484852 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c037660 con 0x7f858c06c890 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 4 set_mon_vals no callback set 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8590e09640 10 monclient: dump: 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:17.133 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 10 monclient: shutdown 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 -- 10.20.193.78:0/4111484852 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 msgr2=0x7f858c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 --2- 10.20.193.78:0/4111484852 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c06cc70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f858c09c670 tx=0x7f857c02ee80 comp rx=0 tx=0).stop 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 -- 10.20.193.78:0/4111484852 shutdown_connections 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 --2- 10.20.193.78:0/4111484852 >> v2:10.20.193.115:3300/0 conn(0x7f858c09bf40 0x7f858c134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 --2- 10.20.193.78:0/4111484852 >> v2:10.20.193.78:3300/0 conn(0x7f858c09d430 0x7f858c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 --2- 10.20.193.78:0/4111484852 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c06cc70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 -- 10.20.193.78:0/4111484852 >> 10.20.193.78:0/4111484852 conn(0x7f858c0a6620 msgr2=0x7f858c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 -- 10.20.193.78:0/4111484852 shutdown_connections 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.120+0000 7f8594096640 1 -- 10.20.193.78:0/4111484852 wait complete. 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient: build_initial_monmap 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient: monmap: 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 1 Processor -- start 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 1 -- start start 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient: init 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c1cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient(hunting): picked mon.noname-c con 0x7f858c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f858c09bf40 0x7f858c1cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient(hunting): picked mon.noname-b con 0x7f858c09bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f858c09d430 0x7f858c121430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient(hunting): picked mon.noname-a con 0x7f858c09d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f858c073380 con 0x7f858c09d430 2026-02-17T05:59:17.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f858c073200 con 0x7f858c09bf40 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f858c135330 con 0x7f858c06c890 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8594096640 10 monclient(hunting): authenticate will time out at 1402.675703s 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f859260c640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f858c09d430 msgr2=0x7f858c121430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c1cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f859260c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f858c09d430 0x7f858c121430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c1cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58286/0 (socket says 10.20.193.78:58286) 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 -- 10.20.193.78:0/2064544988 learned_addr learned my addr 10.20.193.78:0/2064544988 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient(hunting): get_auth_request con 0x7f858c06c890 auth_method 0 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient(hunting): handle_auth_done global_id 144338 payload 824 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 -- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f858c09bf40 msgr2=0x7f858c1cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f858c09bf40 0x7f858c1cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 -- 10.20.193.78:0/2064544988 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f858c09d430 msgr2=0x7f858c121430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f858c09d430 0x7f858c121430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient: _finish_hunting 0 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 monclient: found mon.noname-c 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.121+0000 7f8591e0b640 1 -- 10.20.193.78:0/2064544988 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c125370 con 0x7f858c06c890 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8591e0b640 1 -- 10.20.193.78:0/2064544988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 msgr2=0x7f858c1cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8591e0b640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c1cf560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f857c009a70 tx=0x7f857c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 1 -- 10.20.193.78:0/2064544988 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c047070 con 0x7f858c06c890 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: dump: 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:17.135 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 1 -- 10.20.193.78:0/2064544988 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c03f870 con 0x7f858c06c890 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 5 monclient: authenticate success, global_id 144338 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 10 monclient: _renew_subs 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858c0a1620 con 0x7f858c06c890 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 1 -- 10.20.193.78:0/2064544988 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c03c040 con 0x7f858c06c890 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 10 monclient: dump: 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857bfff640 4 set_mon_vals no callback set 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 10 monclient: _renew_subs 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f858c125090 con 0x7f858c06c890 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858c124c40 con 0x7f858c06c890 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.124+0000 7f857a7fc640 1 -- 10.20.193.78:0/2064544988 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f857c03fca0 con 0x7f858c06c890 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.125+0000 7f857a7fc640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f855c03deb0 0x7f855c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:17.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.125+0000 7f857a7fc640 1 -- 10.20.193.78:0/2064544988 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(466..466 src has 335..466) ==== 4779+0+0 (secure 0 0 0) 0x7f857c0777a0 con 0x7f858c06c890 2026-02-17T05:59:17.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.125+0000 7f859160a640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f855c03deb0 0x7f855c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:17.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.125+0000 7f859160a640 10 monclient: get_auth_request con 0x7f855c03deb0 auth_method 0 2026-02-17T05:59:17.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.127+0000 7f857a7fc640 1 -- 10.20.193.78:0/2064544988 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f858c124c40 con 0x7f858c06c890 2026-02-17T05:59:17.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.127+0000 7f857a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:17.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.127+0000 7f857a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:17.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.128+0000 7f859160a640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f855c03deb0 0x7f855c040370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f858c123db0 tx=0x7f8580006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:17.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.242+0000 7f8594096640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:17.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.242+0000 7f8594096640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:17.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.242+0000 7f8594096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:17.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.242+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f858c074980 con 0x7f858c06c890 2026-02-17T05:59:17.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.243+0000 7f857a7fc640 1 -- 10.20.193.78:0/2064544988 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f857c02fc60 con 0x7f858c06c890 2026-02-17T05:59:17.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.243+0000 7f857a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:17.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.243+0000 7f857a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:17.245 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":154,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:17.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f855c03deb0 msgr2=0x7f855c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f855c03deb0 0x7f855c040370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f858c123db0 tx=0x7f8580006d90 comp rx=0 tx=0).stop 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 10 monclient: shutdown 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 msgr2=0x7f858c1cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c1cf560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f857c009a70 tx=0x7f857c004830 comp rx=0 tx=0).stop 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 shutdown_connections 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f858c09d430 0x7f858c121430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f855c03deb0 0x7f855c040370 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f858c09bf40 0x7f858c1cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 --2- 10.20.193.78:0/2064544988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f858c06c890 0x7f858c1cf560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:17.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 >> 10.20.193.78:0/2064544988 conn(0x7f858c0a6620 msgr2=0x7f858c1389a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:17.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 shutdown_connections 2026-02-17T05:59:17.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:17.244+0000 7f8594096640 1 -- 10.20.193.78:0/2064544988 wait complete. 2026-02-17T05:59:20.254 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:20.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.322+0000 7f20378b7640 10 monclient: get_monmap_and_config 2026-02-17T05:59:20.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.322+0000 7f20378b7640 10 monclient: build_initial_monmap 2026-02-17T05:59:20.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.322+0000 7f20378b7640 10 monclient: monmap: 2026-02-17T05:59:20.323 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:20.323580+0000 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:20.323580+0000 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:20.324 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:20.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.322+0000 7f20378b7640 1 Processor -- start 2026-02-17T05:59:20.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.322+0000 7f20378b7640 1 -- start start 2026-02-17T05:59:20.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.322+0000 7f20378b7640 10 monclient: init 2026-02-17T05:59:20.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:20.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2026-02-17T05:59:20.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 0x7f203006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient(hunting): picked mon.noname-a con 0x7f203006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:20.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f203009d430 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient(hunting): picked mon.noname-b con 0x7f203009bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2030073600 con 0x7f203009d430 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f203009ca50 con 0x7f203006c890 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f203006d6f0 con 0x7f203009bf40 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f20378b7640 10 monclient(hunting): authenticate will time out at 1405.877137s 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f203562c640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 msgr2=0x7f203006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:20.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2034e2b640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f203009d430 legacy=0x7f2030134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f203562c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 0x7f203006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:52268/0 (socket says 10.20.193.78:52268) 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 learned_addr learned my addr 10.20.193.78:0/2415352630 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 10 monclient(hunting): get_auth_request con 0x7f203009bf40 auth_method 0 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:20.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.323+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f203562c640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 msgr2=0x7f203006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f203562c640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 0x7f203006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v1:10.20.193.78:6789/0 conn(0x7f203009d430 legacy=0x7f2030134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:59:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 10 monclient(hunting): get_auth_request con 0x7f203009bf40 auth_method 0 2026-02-17T05:59:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.524+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v1:10.20.193.78:6789/0 conn(0x7f203009d430 legacy=0x7f2030134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f203562c640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 msgr2=0x7f203006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f203562c640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 0x7f203006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 10 monclient(hunting): get_auth_request con 0x7f203009bf40 auth_method 0 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:20.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:20.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:20.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:20.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:20.924+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:59:21.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.724+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v1:10.20.193.78:6789/0 conn(0x7f203009d430 legacy=0x7f2030134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:59:21.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.724+0000 7f203562c640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 msgr2=0x7f203006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:21.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.724+0000 7f203562c640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 0x7f203006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 10 monclient(hunting): get_auth_request con 0x7f203009bf40 auth_method 0 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:21.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:21.725+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:59:23.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 10 monclient(hunting): tick 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 monclient(hunting): continuing hunt 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 msgr2=0x7f203006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f203009bf40 0x7f203006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 msgr2=0x7f203006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.78:3300/0 conn(0x7f203006c890 0x7f203006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 -- 10.20.193.78:0/2415352630 >> v1:10.20.193.78:6789/0 conn(0x7f203009d430 legacy=0x7f2030134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 10 monclient: _add_conns ranks=[1,4] 2026-02-17T05:59:23.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f20140037a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 10 monclient(hunting): picked mon.noname-c con 0x7f20140074f0 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 -- 10.20.193.78:0/2415352630 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f203006dab0 con 0x7f20140037a0 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 1 -- 10.20.193.78:0/2415352630 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f203009ca50 con 0x7f20140074f0 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f201effd640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 10 monclient(hunting): get_auth_request con 0x7f20140074f0 auth_method 0 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:23.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:23.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 msgr2=0x7f20140099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:23.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 msgr2=0x7f20140099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:23.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:23.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.323+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:23.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.325+0000 7f203562c640 1 -- 10.20.193.78:0/2415352630 >> v1:10.20.193.115:6789/0 conn(0x7f20140037a0 legacy=0x7f2014005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 10 monclient(hunting): get_auth_request con 0x7f20140074f0 auth_method 0 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 msgr2=0x7f20140099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 msgr2=0x7f20140099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:23.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.524+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 10 monclient(hunting): get_auth_request con 0x7f20140074f0 auth_method 0 2026-02-17T05:59:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 msgr2=0x7f20140099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 msgr2=0x7f20140099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:23.925+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:59:24.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.663+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 39197415 0 0) 0x7f203006dab0 con 0x7f20140037a0 2026-02-17T05:59:24.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.663+0000 7f201ffff640 10 monclient(hunting): _negotiate 2026-02-17T05:59:24.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.663+0000 7f201ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.663+0000 7f201ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:24.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.663+0000 7f201ffff640 10 monclient(hunting): my global_id is 134368 2026-02-17T05:59:24.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.663+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2018003620 con 0x7f20140037a0 2026-02-17T05:59:24.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.664+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3797222859 0 0) 0x7f2018003620 con 0x7f20140037a0 2026-02-17T05:59:24.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.664+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 msgr2=0x7f20140099b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.664+0000 7f201ffff640 1 --2- 10.20.193.78:0/2415352630 >> v2:10.20.193.115:3300/0 conn(0x7f20140074f0 0x7f20140099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.664+0000 7f201ffff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.664+0000 7f201ffff640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.664+0000 7f201ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.664+0000 7f201ffff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2030073600 con 0x7f20140037a0 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient: _finish_auth 0 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f20378b7640 5 monclient: authenticate success, global_id 134368 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f20378b7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f2020004420 con 0x7f20140037a0 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient: dump: 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 >> v1:10.20.193.115:6789/0 conn(0x7f20140037a0 legacy=0x7f2014005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2035e2d640 1 -- 10.20.193.78:0/2415352630 reap_dead start 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2018003ab0 0x7f2018003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient(hunting): picked mon.a con 0x7f2018003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2018004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient(hunting): picked mon.c con 0x7f2018004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20180053e0 0x7f201800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient(hunting): picked mon.b con 0x7f20180053e0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f203009ca50 con 0x7f2018003ab0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f201800c290 con 0x7f20180053e0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f201800c490 con 0x7f2018004900 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f203562c640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2018003ab0 msgr2=0x7f2018003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f201ffff640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f20378b7640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f203562c640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2018003ab0 0x7f2018003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2018004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient(hunting): get_auth_request con 0x7f2018004900 auth_method 0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2035e2d640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20180053e0 0x7f201800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2035e2d640 10 monclient(hunting): get_auth_request con 0x7f20180053e0 auth_method 0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2035e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2035e2d640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2035e2d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient(hunting): handle_auth_done global_id 134368 payload 293 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20180053e0 msgr2=0x7f201800bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20180053e0 0x7f201800bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2018003ab0 msgr2=0x7f2018003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2018003ab0 0x7f2018003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient: _finish_hunting 0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 1 monclient: found mon.c 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2018003620 con 0x7f2018004900 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient: _finish_auth 0 2026-02-17T05:59:24.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 10 monclient: _check_auth_tickets 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.665+0000 7f2034e2b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f2034e2b640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 msgr2=0x7f2018004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f2034e2b640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2018004d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2024003a80 tx=0x7f202402fd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62aedbec81174555 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024036070 con 0x7f2018004900 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 10 monclient: dump: 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f20378b7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20240309a0 con 0x7f2018004900 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 1 -- 10.20.193.78:0/2415352630 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024009ae0 con 0x7f2018004900 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f201ffff640 10 monclient: dump: 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f20378b7640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.667+0000 7f20378b7640 4 set_mon_vals no callback set 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 10 monclient: shutdown 2026-02-17T05:59:24.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 -- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 msgr2=0x7f2018004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2018004d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2024003a80 tx=0x7f202402fd50 comp rx=0 tx=0).stop 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 -- 10.20.193.78:0/2415352630 shutdown_connections 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20180053e0 0x7f201800bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2018004d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 --2- 10.20.193.78:0/2415352630 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2018003ab0 0x7f2018003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 -- 10.20.193.78:0/2415352630 >> 10.20.193.78:0/2415352630 conn(0x7f20300a6620 msgr2=0x7f20300a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 -- 10.20.193.78:0/2415352630 shutdown_connections 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 -- 10.20.193.78:0/2415352630 wait complete. 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 10 monclient: build_initial_monmap 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 10 monclient: monmap: 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.668+0000 7f20378b7640 1 Processor -- start 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 1 -- start start 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient: init 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2018003ab0 0x7f20301ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient(hunting): picked mon.noname-b con 0x7f2018003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2030137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient(hunting): picked mon.noname-c con 0x7f2018004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20180053e0 0x7f2030121930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient(hunting): picked mon.noname-a con 0x7f20180053e0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2030073380 con 0x7f20180053e0 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2030073200 con 0x7f2018003ab0 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f203006cc70 con 0x7f2018004900 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f20378b7640 10 monclient(hunting): authenticate will time out at 1410.223290s 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2035e2d640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20180053e0 msgr2=0x7f2030121930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2035e2d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20180053e0 0x7f2030121930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2030137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2030137f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:52300/0 (socket says 10.20.193.78:52300) 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 -- 10.20.193.78:0/1395258056 learned_addr learned my addr 10.20.193.78:0/1395258056 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient(hunting): get_auth_request con 0x7f2018004900 auth_method 0 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient(hunting): handle_auth_done global_id 144341 payload 824 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 -- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2018003ab0 msgr2=0x7f20301ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2018003ab0 0x7f20301ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 -- 10.20.193.78:0/1395258056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20180053e0 msgr2=0x7f2030121930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20180053e0 0x7f2030121930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient: _finish_hunting 0 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 monclient: found mon.noname-c 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.669+0000 7f2034e2b640 1 -- 10.20.193.78:0/1395258056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20301255e0 con 0x7f2018004900 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f2034e2b640 1 -- 10.20.193.78:0/1395258056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 msgr2=0x7f2030137f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f2034e2b640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2030137f00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2024030260 tx=0x7f2024005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 1 -- 10.20.193.78:0/1395258056 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024048020 con 0x7f2018004900 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: dump: 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.687 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 1 -- 10.20.193.78:0/1395258056 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2024009410 con 0x7f2018004900 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 1 -- 10.20.193.78:0/1395258056 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024009ae0 con 0x7f2018004900 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201dffb640 10 monclient: dump: 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f201f7fe640 4 set_mon_vals no callback set 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f20378b7640 5 monclient: authenticate success, global_id 144341 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f20378b7640 10 monclient: _renew_subs 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f20378b7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203009e070 con 0x7f2018004900 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f20378b7640 10 monclient: _renew_subs 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f20378b7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.671+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2030125300 con 0x7f2018004900 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f201dffb640 1 -- 10.20.193.78:0/1395258056 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2024043050 con 0x7f2018004900 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f20378b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f20378b7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f20378b7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2030074980 con 0x7f2018004900 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f201dffb640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f200003de60 0x7f2000040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f201dffb640 1 -- 10.20.193.78:0/1395258056 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(466..466 src has 335..466) ==== 4779+0+0 (secure 0 0 0) 0x7f2024078200 con 0x7f2018004900 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f203562c640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f200003de60 0x7f2000040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.672+0000 7f203562c640 10 monclient: get_auth_request con 0x7f200003de60 auth_method 0 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.674+0000 7f201dffb640 1 -- 10.20.193.78:0/1395258056 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2024052030 con 0x7f2018004900 2026-02-17T05:59:24.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.674+0000 7f201dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:24.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.674+0000 7f201dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:24.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.675+0000 7f203562c640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f200003de60 0x7f2000040320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f20200026b0 tx=0x7f2020003b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2186: done (2 left) 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2187: read oid 32 snap 216 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2187: expect (ObjNum 730 snap 202 seq_num 730) 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2188: rollback oid 30 current snap is 219 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 216 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2183: finishing copy_from racing read to trial11545420-46 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2183: finishing copy_from to trial11545420-46 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 874 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: finishing write tid 1 to trial11545420-17 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: finishing write tid 2 to trial11545420-17 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: finishing write tid 3 to trial11545420-17 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: finishing write tid 4 to trial11545420-17 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: finishing write tid 5 to trial11545420-17 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: oid 17 updating version 0 to 1042 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: oid 17 version 1042 is already newer than 1041 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: oid 17 version 1042 is already newer than 1040 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: oid 17 version 1042 is already newer than 1039 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1042 (ObjNum 779 snap 219 seq_num 779) dirty exists 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: left oid 17 (ObjNum 779 snap 219 seq_num 779) 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2183: done (3 left) 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2185: done (2 left) 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2189: read oid 18 snap -1 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2189: expect deleted 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2190: read oid 29 snap -1 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2190: expect (ObjNum 750 snap 206 seq_num 750) 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2191: read oid 44 snap -1 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2191: expect (ObjNum 740 snap 204 seq_num 740) 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2192: rollback oid 6 current snap is 219 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 6 to 216 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2193: rollback oid 19 current snap is 219 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 19 to 216 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2194: read oid 22 snap -1 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2194: expect (ObjNum 762 snap 210 seq_num 762) 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2195: read oid 36 snap 217 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2195: expect deleted 2026-02-17T05:59:24.719 INFO:tasks.rados.rados.0.trial115.stdout:2196: snap_create 2026-02-17T05:59:24.721 INFO:tasks.rados.rados.0.trial115.stdout:2192: finishing rollback tid 0 to trial11545420-6 2026-02-17T05:59:24.721 INFO:tasks.rados.rados.0.trial115.stdout:2192: finishing rollback tid 1 to trial11545420-6 2026-02-17T05:59:24.722 INFO:tasks.rados.rados.0.trial115.stdout:2192: finishing rollback tid 2 to trial11545420-6 2026-02-17T05:59:24.722 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 835 (ObjNum 657 snap 178 seq_num 657) dirty exists 2026-02-17T05:59:24.722 INFO:tasks.rados.rados.0.trial115.stdout:2188: finishing rollback tid 0 to trial11545420-30 2026-02-17T05:59:24.722 INFO:tasks.rados.rados.0.trial115.stdout:2188: finishing rollback tid 1 to trial11545420-30 2026-02-17T05:59:24.722 INFO:tasks.rados.rados.0.trial115.stdout:2193: finishing rollback tid 0 to trial11545420-19 2026-02-17T05:59:24.723 INFO:tasks.rados.rados.0.trial115.stdout:2193: finishing rollback tid 1 to trial11545420-19 2026-02-17T05:59:24.723 INFO:tasks.rados.rados.0.trial115.stdout:2193: finishing rollback tid 2 to trial11545420-19 2026-02-17T05:59:24.723 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1240 (ObjNum 723 snap 199 seq_num 723) dirty exists 2026-02-17T05:59:24.723 INFO:tasks.rados.rados.0.trial115.stdout:2190: expect (ObjNum 750 snap 206 seq_num 750) 2026-02-17T05:59:24.785 INFO:tasks.rados.rados.0.trial115.stdout:2187: expect (ObjNum 730 snap 202 seq_num 730) 2026-02-17T05:59:24.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.790+0000 7f20378b7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:24.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.790+0000 7f20378b7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:24.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.790+0000 7f20378b7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.790+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f203006cfa0 con 0x7f2018004900 2026-02-17T05:59:24.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.790+0000 7f201dffb640 1 -- 10.20.193.78:0/1395258056 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f2024054300 con 0x7f2018004900 2026-02-17T05:59:24.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.790+0000 7f201dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:24.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.790+0000 7f201dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:24.791 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":156,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:24.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f200003de60 msgr2=0x7f2000040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f200003de60 0x7f2000040320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f20200026b0 tx=0x7f2020003b60 comp rx=0 tx=0).stop 2026-02-17T05:59:24.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 10 monclient: shutdown 2026-02-17T05:59:24.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:24.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 msgr2=0x7f2030137f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2030137f00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2024030260 tx=0x7f2024005eb0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 shutdown_connections 2026-02-17T05:59:24.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20180053e0 0x7f2030121930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2018004900 0x7f2030137f00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f200003de60 0x7f2000040320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.791+0000 7f20378b7640 1 --2- 10.20.193.78:0/1395258056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2018003ab0 0x7f20301ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.792+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 >> 10.20.193.78:0/1395258056 conn(0x7f20300a6620 msgr2=0x7f203009e430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:24.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.792+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 shutdown_connections 2026-02-17T05:59:24.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.792+0000 7f20378b7640 1 -- 10.20.193.78:0/1395258056 wait complete. 2026-02-17T05:59:24.801 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T05:59:24.801 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T05:59:24.801 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:59:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient: get_monmap_and_config 2026-02-17T05:59:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient: build_initial_monmap 2026-02-17T05:59:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient: monmap: 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:24.870450+0000 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:24.870450+0000 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:24.873 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 1 Processor -- start 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 1 -- start start 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient: init 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f767406c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767409c4a0 0x7f767413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient(hunting): picked mon.noname-b con 0x7f767409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f767406d380 0x7f767413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 10 monclient(hunting): picked mon.noname-c con 0x7f767406d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:24.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.869+0000 7f767987c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f767406daf0 con 0x7f767406c890 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f767987c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f767987c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f76740a0880 con 0x7f767406d380 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f767987c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f767987c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f767409cc80 con 0x7f767409c4a0 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f767987c640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f767987c640 10 monclient(hunting): authenticate will time out at 1410.424018s 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f7672ffd640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f767406c890 legacy=0x7f767406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767409c4a0 0x7f767413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767409c4a0 0x7f767413b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:52306/0 (socket says 10.20.193.78:52306) 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 1 -- 10.20.193.78:0/175909406 learned_addr learned my addr 10.20.193.78:0/175909406 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient(hunting): get_auth_request con 0x7f767409c4a0 auth_method 0 2026-02-17T05:59:24.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76737fe640 1 --2- 10.20.193.78:0/175909406 >> v2:10.20.193.115:3300/0 conn(0x7f767406d380 0x7f767413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient(hunting): handle_auth_done global_id 144344 payload 274 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 1 -- 10.20.193.78:0/175909406 >> v2:10.20.193.115:3300/0 conn(0x7f767406d380 msgr2=0x7f767413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 1 --2- 10.20.193.78:0/175909406 >> v2:10.20.193.115:3300/0 conn(0x7f767406d380 0x7f767413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 1 -- 10.20.193.78:0/175909406 >> v1:10.20.193.78:6789/0 conn(0x7f767406c890 legacy=0x7f767406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient: _finish_hunting 0 2026-02-17T05:59:24.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 1 monclient: found mon.noname-b 2026-02-17T05:59:24.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:24.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76727fc640 1 -- 10.20.193.78:0/175909406 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767409cf60 con 0x7f767409c4a0 2026-02-17T05:59:24.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76737fe640 10 monclient: get_auth_request con 0x7f767406d380 auth_method 0 2026-02-17T05:59:24.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.870+0000 7f76737fe640 1 --2- 10.20.193.78:0/175909406 >> v2:10.20.193.115:3300/0 conn(0x7f767406d380 0x7f767413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:24.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f76727fc640 1 -- 10.20.193.78:0/175909406 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767409c4a0 msgr2=0x7f767413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:24.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f76727fc640 1 --2- 10.20.193.78:0/175909406 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767409c4a0 0x7f767413b3a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f765c009970 tx=0x7f765c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49559dd50284f064 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 1 -- 10.20.193.78:0/175909406 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c03c070 con 0x7f767409c4a0 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: dump: 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.886 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 1 -- 10.20.193.78:0/175909406 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765c02fdb0 con 0x7f767409c4a0 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f767987c640 5 monclient: authenticate success, global_id 144344 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 1 -- 10.20.193.78:0/175909406 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c037660 con 0x7f767409c4a0 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f767987c640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:24.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f767987c640 4 set_mon_vals no callback set 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.872+0000 7f7671ffb640 10 monclient: dump: 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.888 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 10 monclient: shutdown 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 -- 10.20.193.78:0/175909406 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767409c4a0 msgr2=0x7f767413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 --2- 10.20.193.78:0/175909406 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767409c4a0 0x7f767413b3a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f765c009970 tx=0x7f765c02ee90 comp rx=0 tx=0).stop 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 -- 10.20.193.78:0/175909406 shutdown_connections 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 --2- 10.20.193.78:0/175909406 >> v2:10.20.193.115:3300/0 conn(0x7f767406d380 0x7f767413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 --2- 10.20.193.78:0/175909406 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767409c4a0 0x7f767413b3a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 -- 10.20.193.78:0/175909406 >> 10.20.193.78:0/175909406 conn(0x7f76740a6620 msgr2=0x7f76740a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 -- 10.20.193.78:0/175909406 shutdown_connections 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 -- 10.20.193.78:0/175909406 wait complete. 2026-02-17T05:59:24.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 10 monclient: build_initial_monmap 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 10 monclient: monmap: 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 Processor -- start 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 1 -- start start 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.873+0000 7f767987c640 10 monclient: init 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f767406c890 0x7f76741cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient(hunting): picked mon.noname-b con 0x7f767406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767406d380 0x7f76741320d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient(hunting): picked mon.noname-c con 0x7f767406d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f767409c4a0 0x7f7674132610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient(hunting): picked mon.noname-a con 0x7f767409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f767409bfa0 con 0x7f767409c4a0 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f767409be20 con 0x7f767406c890 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f767413de30 con 0x7f767406d380 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f767987c640 10 monclient(hunting): authenticate will time out at 1410.428200s 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76737fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f767409c4a0 msgr2=0x7f7674132610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76737fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f767409c4a0 0x7f7674132610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767406d380 0x7f76741320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767406d380 0x7f76741320d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:52318/0 (socket says 10.20.193.78:52318) 2026-02-17T05:59:24.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 -- 10.20.193.78:0/4051348747 learned_addr learned my addr 10.20.193.78:0/4051348747 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient(hunting): get_auth_request con 0x7f767406d380 auth_method 0 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient(hunting): handle_auth_done global_id 144347 payload 824 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f767406c890 msgr2=0x7f76741cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f767406c890 0x7f76741cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f767409c4a0 msgr2=0x7f7674132610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f767409c4a0 0x7f7674132610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient: _finish_hunting 0 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 monclient: found mon.noname-c 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.874+0000 7f76727fc640 1 -- 10.20.193.78:0/4051348747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767411ea80 con 0x7f767406d380 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f76727fc640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767406d380 msgr2=0x7f76741320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f76727fc640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767406d380 0x7f76741320d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f765c0090b0 tx=0x7f765c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c03c050 con 0x7f767406d380 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: dump: 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.894 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: _finish_auth 0 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:24.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765c048070 con 0x7f767406d380 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c041a60 con 0x7f767406d380 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f767987c640 5 monclient: authenticate success, global_id 144347 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f767987c640 10 monclient: _renew_subs 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f76717fa640 4 set_mon_vals no callback set 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f767987c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767409e450 con 0x7f767406d380 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f764bfff640 10 monclient: dump: 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.896 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f767987c640 10 monclient: _renew_subs 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f767987c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.876+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7674073070 con 0x7f767406d380 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f767987c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:24.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f767987c640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674071cb0 0x7f7674134e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f767987c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7674074980 con 0x7f7674071cb0 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674071cb0 0x7f7674134e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 10 monclient: get_auth_request con 0x7f7674071cb0 auth_method 0 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f765c041c00 con 0x7f767406d380 2026-02-17T05:59:24.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 10 monclient(hunting): handle_auth_done global_id 144350 payload 824 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f7672ffd640 1 -- 10.20.193.78:0/4051348747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f767406cc70 con 0x7f7674071cb0 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f764bfff640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f764403de60 0x7f7644040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(467..467 src has 335..467) ==== 4779+0+0 (secure 0 0 0) 0x7f765c077a30 con 0x7f767406d380 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f76737fe640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f764403de60 0x7f7644040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.877+0000 7f76737fe640 10 monclient: get_auth_request con 0x7f764403de60 auth_method 0 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f7672ffd640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674071cb0 msgr2=0x7f7674134e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f7672ffd640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674071cb0 0x7f7674134e80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f766800b450 tx=0x7f766800b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668005270 con 0x7f7674071cb0 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 10 monclient: dump: 2026-02-17T05:59:24.899 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7668015070 con 0x7f7674071cb0 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:24.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674071cb0 msgr2=0x7f7674134e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674071cb0 0x7f7674134e80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f766800b450 tx=0x7f766800b920 comp rx=0 tx=0).stop 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674071cb0 msgr2=0x7f7674134e80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.879+0000 7f764bfff640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674071cb0 0x7f7674134e80 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f766800b450 tx=0x7f766800b920 comp rx=0 tx=0).stop 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.880+0000 7f76737fe640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f764403de60 0x7f7644040320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7674073e30 tx=0x7f7664008390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f767987c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f767987c640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674072090 0x7f76741da370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f767987c640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7674136540 con 0x7f7674072090 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674072090 0x7f76741da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 10 monclient: get_auth_request con 0x7f7674072090 auth_method 0 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:24.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:24.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:24.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:24.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:24.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 10 monclient(hunting): handle_auth_done global_id 144353 payload 824 2026-02-17T05:59:24.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.883+0000 7f7672ffd640 1 -- 10.20.193.78:0/4051348747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f767411e700 con 0x7f7674072090 2026-02-17T05:59:24.903 INFO:tasks.rados.rados.0.trial115.stdout:2194: expect (ObjNum 762 snap 210 seq_num 762) 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.885+0000 7f7672ffd640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674072090 msgr2=0x7f76741da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.885+0000 7f7672ffd640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674072090 0x7f76741da370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7668003fd0 tx=0x7f7668005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.885+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668005910 con 0x7f7674072090 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.885+0000 7f764bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.885+0000 7f764bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.885+0000 7f764bfff640 10 monclient: dump: 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.904 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7f7668005910 con 0x7f7674072090 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f764bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f764bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674072090 msgr2=0x7f76741da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f764bfff640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674072090 0x7f76741da370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7668003fd0 tx=0x7f7668005ad0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f764bfff640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674072090 msgr2=0x7f76741da370 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f764bfff640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7674072090 0x7f76741da370 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7668003fd0 tx=0x7f7668005ad0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.905 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 156, 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 148420, 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:24.906 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:24.907 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:24.908 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:24.909 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.910 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:24.911 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:24.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.913 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.914 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:24.915 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:24.916 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:59:24.916 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:24.916 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:24.916 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:24.916 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:24.916 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f764403de60 msgr2=0x7f7644040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f764403de60 0x7f7644040320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7674073e30 tx=0x7f7664008390 comp rx=0 tx=0).stop 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 10 monclient: shutdown 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767406d380 msgr2=0x7f76741320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 1 --2- 10.20.193.78:0/4051348747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f767406d380 0x7f76741320d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f765c0090b0 tx=0x7f765c004830 comp rx=0 tx=0).stop 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f76737fe640 1 -- 10.20.193.78:0/4051348747 reap_dead start 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f76737fe640 1 -- 10.20.193.78:0/4051348747 reap_dead start 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 shutdown_connections 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 >> 10.20.193.78:0/4051348747 conn(0x7f76740a6620 msgr2=0x7f7674134180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:24.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 shutdown_connections 2026-02-17T05:59:24.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.886+0000 7f767987c640 1 -- 10.20.193.78:0/4051348747 wait complete. 2026-02-17T05:59:24.918 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:59:24.918 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T05:59:24.918 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:59:24.973 INFO:tasks.rados.rados.0.trial115.stdout:2188: finishing rollback tid 2 to trial11545420-30 2026-02-17T05:59:24.973 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 952 (ObjNum 767 snap 214 seq_num 767) dirty exists 2026-02-17T05:59:24.973 INFO:tasks.rados.rados.0.trial115.stdout:2191: expect (ObjNum 740 snap 204 seq_num 740) 2026-02-17T05:59:24.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.985+0000 7ff4e337e640 10 monclient: get_monmap_and_config 2026-02-17T05:59:24.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.985+0000 7ff4e337e640 10 monclient: build_initial_monmap 2026-02-17T05:59:24.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.985+0000 7ff4e337e640 10 monclient: monmap: 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:24.987041+0000 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:24.987041+0000 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:24.987 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 1 Processor -- start 2026-02-17T05:59:24.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 1 -- start start 2026-02-17T05:59:24.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient: init 2026-02-17T05:59:24.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:24.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2026-02-17T05:59:24.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff4dc06c890 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:24.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc06d380 0x7ff4dc0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient(hunting): picked mon.noname-b con 0x7ff4dc06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff4dc13a240 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4dc074150 con 0x7ff4dc06c890 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e10f3640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7ff4dc06c890 legacy=0x7ff4dc06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4dc09ef70 con 0x7ff4dc13a240 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff4dc09f250 con 0x7ff4dc06d380 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e337e640 10 monclient(hunting): authenticate will time out at 1410.540814s 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc06d380 0x7ff4dc0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc06d380 0x7ff4dc0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:52348/0 (socket says 10.20.193.78:52348) 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 1 -- 10.20.193.78:0/3030559775 learned_addr learned my addr 10.20.193.78:0/3030559775 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 10 monclient(hunting): get_auth_request con 0x7ff4dc06d380 auth_method 0 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:24.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.986+0000 7ff4e08f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 10 monclient(hunting): handle_auth_done global_id 144356 payload 274 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 1 -- 10.20.193.78:0/3030559775 >> v1:10.20.193.115:6789/0 conn(0x7ff4dc13a240 legacy=0x7ff4dc13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 1 -- 10.20.193.78:0/3030559775 >> v1:10.20.193.78:6789/0 conn(0x7ff4dc06c890 legacy=0x7ff4dc06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 10 monclient: _finish_hunting 0 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 1 monclient: found mon.noname-b 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:24.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.987+0000 7ff4e08f2640 1 -- 10.20.193.78:0/3030559775 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc09df40 con 0x7ff4dc06d380 2026-02-17T05:59:24.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e08f2640 1 -- 10.20.193.78:0/3030559775 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc06d380 msgr2=0x7ff4dc0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:24.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e08f2640 1 --2- 10.20.193.78:0/3030559775 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc06d380 0x7ff4dc0a01f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc06e270 tx=0x7ff4d002ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a4167a08eef727a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:24.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 1 -- 10.20.193.78:0/3030559775 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d003c070 con 0x7ff4dc06d380 2026-02-17T05:59:24.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: dump: 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: _finish_auth 0 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 1 -- 10.20.193.78:0/3030559775 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d0037440 con 0x7ff4dc06d380 2026-02-17T05:59:24.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e337e640 5 monclient: authenticate success, global_id 144356 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 1 -- 10.20.193.78:0/3030559775 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0037810 con 0x7ff4dc06d380 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e337e640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e337e640 4 set_mon_vals no callback set 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4cbfff640 10 monclient: dump: 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:24.998 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e337e640 10 monclient: shutdown 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e337e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e337e640 1 -- 10.20.193.78:0/3030559775 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc06d380 msgr2=0x7ff4dc0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.989+0000 7ff4e337e640 1 --2- 10.20.193.78:0/3030559775 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc06d380 0x7ff4dc0a01f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc06e270 tx=0x7ff4d002ef90 comp rx=0 tx=0).stop 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 -- 10.20.193.78:0/3030559775 shutdown_connections 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 --2- 10.20.193.78:0/3030559775 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc06d380 0x7ff4dc0a01f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 -- 10.20.193.78:0/3030559775 >> 10.20.193.78:0/3030559775 conn(0x7ff4dc0a6620 msgr2=0x7ff4dc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 -- 10.20.193.78:0/3030559775 shutdown_connections 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 -- 10.20.193.78:0/3030559775 wait complete. 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient: build_initial_monmap 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient: monmap: 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:24.999 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 Processor -- start 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 -- start start 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient: init 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4dc06c890 0x7ff4dc132e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient(hunting): picked mon.noname-a con 0x7ff4dc06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc06d380 0x7ff4dc133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient(hunting): picked mon.noname-b con 0x7ff4dc06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc13a240 0x7ff4dc133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient(hunting): picked mon.noname-c con 0x7ff4dc13a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc09bfa0 con 0x7ff4dc06c890 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc09be20 con 0x7ff4dc06d380 2026-02-17T05:59:25.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff4dc09c120 con 0x7ff4dc13a240 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e337e640 10 monclient(hunting): authenticate will time out at 1410.544970s 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e10f3640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4dc06c890 msgr2=0x7ff4dc132e00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.990+0000 7ff4e10f3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4dc06c890 0x7ff4dc132e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc13a240 0x7ff4dc133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc13a240 0x7ff4dc133880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:52354/0 (socket says 10.20.193.78:52354) 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 -- 10.20.193.78:0/4050374438 learned_addr learned my addr 10.20.193.78:0/4050374438 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient(hunting): get_auth_request con 0x7ff4dc13a240 auth_method 0 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e08f2640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc06d380 0x7ff4dc133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:25.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e08f2640 10 monclient(hunting): get_auth_request con 0x7ff4dc06d380 auth_method 0 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e08f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e08f2640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e08f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient(hunting): handle_auth_done global_id 144359 payload 824 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc06d380 msgr2=0x7ff4dc133340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc06d380 0x7ff4dc133340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4dc06c890 msgr2=0x7ff4dc132e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4dc06c890 0x7ff4dc132e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient: _finish_hunting 0 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 monclient: found mon.noname-c 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e18f4640 1 -- 10.20.193.78:0/4050374438 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc06ef90 con 0x7ff4dc13a240 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.991+0000 7ff4e08f2640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc06d380 0x7ff4dc133340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e18f4640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc13a240 msgr2=0x7ff4dc133880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e18f4640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc13a240 0x7ff4dc133880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800cb20 tx=0x7ff4d800a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8016020 con 0x7ff4dc13a240 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:25.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: dump: 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d8012420 con 0x7ff4dc13a240 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 5 monclient: authenticate success, global_id 144359 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 10 monclient: _renew_subs 2026-02-17T05:59:25.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc0a2680 con 0x7ff4dc13a240 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8012810 con 0x7ff4dc13a240 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4cb7fe640 4 set_mon_vals no callback set 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4c9ffb640 10 monclient: dump: 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 10 monclient: _renew_subs 2026-02-17T05:59:25.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc09d810 con 0x7ff4dc13a240 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1ccf10 0x7ff4dc1cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.993+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc0008d0 con 0x7ff4dc1ccf10 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1ccf10 0x7ff4dc1cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 10 monclient: get_auth_request con 0x7ff4dc1ccf10 auth_method 0 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff4d8012a30 con 0x7ff4dc13a240 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4c9ffb640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff4ac03de60 0x7ff4ac040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(467..467 src has 335..467) ==== 4779+0+0 (secure 0 0 0) 0x7ff4d80532b0 con 0x7ff4dc13a240 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 10 monclient(hunting): handle_auth_done global_id 134374 payload 824 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e10f3640 1 -- 10.20.193.78:0/4050374438 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4dc135600 con 0x7ff4dc1ccf10 2026-02-17T05:59:25.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e08f2640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff4ac03de60 0x7ff4ac040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:25.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.994+0000 7ff4e08f2640 10 monclient: get_auth_request con 0x7ff4ac03de60 auth_method 0 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.996+0000 7ff4e10f3640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1ccf10 msgr2=0x7ff4dc1cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.996+0000 7ff4e10f3640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1ccf10 0x7ff4dc1cd2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bcc0 tx=0x7ff4cc00bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.996+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc005070 con 0x7ff4dc1ccf10 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.996+0000 7ff4c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.996+0000 7ff4c9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.996+0000 7ff4c9ffb640 10 monclient: dump: 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:25.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.997+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff4cc005070 con 0x7ff4dc1ccf10 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.997+0000 7ff4c9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.997+0000 7ff4c9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.997+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1ccf10 msgr2=0x7ff4dc1cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.997+0000 7ff4c9ffb640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1ccf10 0x7ff4dc1cd2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bcc0 tx=0x7ff4cc00bcf0 comp rx=0 tx=0).stop 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.997+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1ccf10 msgr2=0x7ff4dc1cd2f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.997+0000 7ff4c9ffb640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1ccf10 0x7ff4dc1cd2f0 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bcc0 tx=0x7ff4cc00bcf0 comp rx=0 tx=0).stop 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:24.997+0000 7ff4e08f2640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff4ac03de60 0x7ff4ac040320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0036000 tx=0x7ff4d0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e337e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e337e640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1cd830 0x7ff4dc1d8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e337e640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc134af0 con 0x7ff4dc1cd830 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1cd830 0x7ff4dc1d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 10 monclient: get_auth_request con 0x7ff4dc1cd830 auth_method 0 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:25.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 10 monclient(hunting): handle_auth_done global_id 134377 payload 824 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.000+0000 7ff4e10f3640 1 -- 10.20.193.78:0/4050374438 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff4dc0a01f0 con 0x7ff4dc1cd830 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.002+0000 7ff4e10f3640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1cd830 msgr2=0x7ff4dc1d8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.002+0000 7ff4e10f3640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1cd830 0x7ff4dc1d8260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bcc0 tx=0x7ff4cc005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc01a730 con 0x7ff4dc1cd830 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 10 monclient: dump: 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:25.010 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7ff4cc014430 con 0x7ff4dc1cd830 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1cd830 msgr2=0x7ff4dc1d8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1cd830 0x7ff4dc1d8260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bcc0 tx=0x7ff4cc005ad0 comp rx=0 tx=0).stop 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1cd830 msgr2=0x7ff4dc1d8260 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4c9ffb640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4dc1cd830 0x7ff4dc1d8260 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bcc0 tx=0x7ff4cc005ad0 comp rx=0 tx=0).stop 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:25.011 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 156, 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 192722, 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:25.012 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:25.013 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:25.014 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:25.015 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:25.016 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.017 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:25.018 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:25.019 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:25.020 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:25.020 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:25.020 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff4ac03de60 msgr2=0x7ff4ac040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff4ac03de60 0x7ff4ac040320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0036000 tx=0x7ff4d0033000 comp rx=0 tx=0).stop 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 10 monclient: shutdown 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc13a240 msgr2=0x7ff4dc133880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 1 --2- 10.20.193.78:0/4050374438 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4dc13a240 0x7ff4dc133880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800cb20 tx=0x7ff4d800a060 comp rx=0 tx=0).stop 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e18f4640 1 -- 10.20.193.78:0/4050374438 reap_dead start 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e18f4640 1 -- 10.20.193.78:0/4050374438 reap_dead start 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 shutdown_connections 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 >> 10.20.193.78:0/4050374438 conn(0x7ff4dc0a6620 msgr2=0x7ff4dc0a0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 shutdown_connections 2026-02-17T05:59:25.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:25.003+0000 7ff4e337e640 1 -- 10.20.193.78:0/4050374438 wait complete. 2026-02-17T05:59:25.021 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T05:59:25.021 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2187: done (9 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2188: done (8 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2189: done (7 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2190: done (6 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2191: done (5 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2192: done (4 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2193: done (3 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2194: done (2 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2195: done (1 left) 2026-02-17T05:59:25.720 INFO:tasks.rados.rados.0.trial115.stdout:2196: done (0 left) 2026-02-17T05:59:25.721 INFO:tasks.rados.rados.0.trial115.stdout:2197: delete oid 38 current snap is 220 2026-02-17T05:59:25.722 INFO:tasks.rados.rados.0.trial115.stdout:2197: done (0 left) 2026-02-17T05:59:25.722 INFO:tasks.rados.rados.0.trial115.stdout:2198: delete oid 18 current snap is 220 2026-02-17T05:59:25.725 INFO:tasks.rados.rados.0.trial115.stdout:2198: done (0 left) 2026-02-17T05:59:25.725 INFO:tasks.rados.rados.0.trial115.stdout:2199: copy_from oid 46 from oid 38 current snap is 220 2026-02-17T05:59:25.725 INFO:tasks.rados.rados.0.trial115.stdout:2200: snap_remove snap 219 2026-02-17T05:59:26.724 INFO:tasks.rados.rados.0.trial115.stdout:2200: done (1 left) 2026-02-17T05:59:26.724 INFO:tasks.rados.rados.0.trial115.stdout:2201: delete oid 12 current snap is 220 2026-02-17T05:59:26.724 INFO:tasks.rados.rados.0.trial115.stdout:2199: finishing copy_from to trial11545420-46 2026-02-17T05:59:26.725 INFO:tasks.rados.rados.0.trial115.stdout:2199: got expected ENOENT (src dne) 2026-02-17T05:59:26.725 INFO:tasks.rados.rados.0.trial115.stdout:2199: finishing copy_from racing read to trial11545420-46 2026-02-17T05:59:26.727 INFO:tasks.rados.rados.0.trial115.stdout:2199: done (1 left) 2026-02-17T05:59:26.728 INFO:tasks.rados.rados.0.trial115.stdout:2201: done (0 left) 2026-02-17T05:59:26.728 INFO:tasks.rados.rados.0.trial115.stdout:2202: rollback oid 44 current snap is 220 2026-02-17T05:59:26.728 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 44 to 217 2026-02-17T05:59:26.728 INFO:tasks.rados.rados.0.trial115.stdout:2203: rollback oid 28 current snap is 220 2026-02-17T05:59:26.728 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 28 to 216 2026-02-17T05:59:26.728 INFO:tasks.rados.rados.0.trial115.stdout:2204: write oid 38 current snap is 220 2026-02-17T05:59:26.728 INFO:tasks.rados.rados.0.trial115.stdout:2204: seq_num 783 ranges {759881=787973,2119636=654941} 2026-02-17T05:59:26.733 INFO:tasks.rados.rados.0.trial115.stdout:2204: writing trial11545420-38 from 759881 to 1547854 tid 1 2026-02-17T05:59:26.738 INFO:tasks.rados.rados.0.trial115.stdout:2204: writing trial11545420-38 from 2119636 to 2774577 tid 2 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:2205: read oid 19 snap 216 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:2205: expect (ObjNum 723 snap 199 seq_num 723) 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:2206: rollback oid 32 current snap is 220 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 217 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:2203: finishing rollback tid 1 to trial11545420-28 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:2203: finishing rollback tid 2 to trial11545420-28 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1148 (ObjNum 747 snap 204 seq_num 747) dirty exists 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:2202: finishing rollback tid 0 to trial11545420-44 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:2202: finishing rollback tid 1 to trial11545420-44 2026-02-17T05:59:26.739 INFO:tasks.rados.rados.0.trial115.stdout:2202: finishing rollback tid 2 to trial11545420-44 2026-02-17T05:59:26.740 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 883 (ObjNum 740 snap 204 seq_num 740) dirty exists 2026-02-17T05:59:26.740 INFO:tasks.rados.rados.0.trial115.stdout:2204: finishing write tid 1 to trial11545420-38 2026-02-17T05:59:26.740 INFO:tasks.rados.rados.0.trial115.stdout:2202: done (4 left) 2026-02-17T05:59:26.740 INFO:tasks.rados.rados.0.trial115.stdout:2203: done (3 left) 2026-02-17T05:59:26.740 INFO:tasks.rados.rados.0.trial115.stdout:2207: rollback oid 14 current snap is 220 2026-02-17T05:59:26.740 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 216 2026-02-17T05:59:26.740 INFO:tasks.rados.rados.0.trial115.stdout:2208: write oid 8 current snap is 220 2026-02-17T05:59:26.740 INFO:tasks.rados.rados.0.trial115.stdout:2208: seq_num 784 ranges {776688=579575,1894471=798599} 2026-02-17T05:59:26.743 INFO:tasks.rados.rados.0.trial115.stdout:2208: writing trial11545420-8 from 776688 to 1356263 tid 1 2026-02-17T05:59:26.748 INFO:tasks.rados.rados.0.trial115.stdout:2208: writing trial11545420-8 from 1894471 to 2693070 tid 2 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2209: copy_from oid 47 from oid 46 current snap is 220 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2206: finishing rollback tid 0 to trial11545420-32 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2206: finishing rollback tid 1 to trial11545420-32 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2206: finishing rollback tid 2 to trial11545420-32 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 886 (ObjNum 730 snap 202 seq_num 730) dirty exists 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2207: finishing rollback tid 0 to trial11545420-14 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2207: finishing rollback tid 1 to trial11545420-14 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2207: finishing rollback tid 2 to trial11545420-14 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 944 (ObjNum 700 snap 193 seq_num 700) dirty exists 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2204: finishing write tid 2 to trial11545420-38 2026-02-17T05:59:26.749 INFO:tasks.rados.rados.0.trial115.stdout:2204: finishing write tid 3 to trial11545420-38 2026-02-17T05:59:26.750 INFO:tasks.rados.rados.0.trial115.stdout:2204: finishing write tid 4 to trial11545420-38 2026-02-17T05:59:26.750 INFO:tasks.rados.rados.0.trial115.stdout:2204: oid 38 updating version 0 to 1246 2026-02-17T05:59:26.750 INFO:tasks.rados.rados.0.trial115.stdout:2204: oid 38 updating version 1246 to 1247 2026-02-17T05:59:26.750 INFO:tasks.rados.rados.0.trial115.stdout:2204: oid 38 version 1247 is already newer than 1245 2026-02-17T05:59:26.750 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1247 (ObjNum 783 snap 220 seq_num 783) dirty exists 2026-02-17T05:59:26.750 INFO:tasks.rados.rados.0.trial115.stdout:2204: left oid 38 (ObjNum 783 snap 220 seq_num 783) 2026-02-17T05:59:26.750 INFO:tasks.rados.rados.0.trial115.stdout:2205: expect (ObjNum 723 snap 199 seq_num 723) 2026-02-17T05:59:26.827 INFO:tasks.rados.rados.0.trial115.stdout:2208: finishing write tid 1 to trial11545420-8 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2208: finishing write tid 2 to trial11545420-8 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2208: finishing write tid 3 to trial11545420-8 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2204: done (5 left) 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2205: done (4 left) 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2206: done (3 left) 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2207: done (2 left) 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2210: copy_from oid 42 from oid 27 current snap is 220 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2208: finishing write tid 4 to trial11545420-8 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2208: oid 8 updating version 0 to 259 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2208: oid 8 version 259 is already newer than 258 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2208: oid 8 version 259 is already newer than 257 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 259 (ObjNum 784 snap 220 seq_num 784) dirty exists 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2208: left oid 8 (ObjNum 784 snap 220 seq_num 784) 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2209: finishing copy_from racing read to trial11545420-47 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2209: finishing copy_from to trial11545420-47 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 874 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2208: done (2 left) 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2209: done (1 left) 2026-02-17T05:59:26.828 INFO:tasks.rados.rados.0.trial115.stdout:2211: snap_create 2026-02-17T05:59:26.834 INFO:tasks.rados.rados.0.trial115.stdout:2210: finishing copy_from racing read to trial11545420-42 2026-02-17T05:59:26.836 INFO:tasks.rados.rados.0.trial115.stdout:2210: finishing copy_from to trial11545420-42 2026-02-17T05:59:26.836 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 942 (ObjNum 761 snap 207 seq_num 761) dirty exists 2026-02-17T05:59:27.302 INFO:tasks.rados.rados.0.trial115.stdout:2210: done (1 left) 2026-02-17T05:59:27.302 INFO:tasks.rados.rados.0.trial115.stdout:2211: done (0 left) 2026-02-17T05:59:27.302 INFO:tasks.rados.rados.0.trial115.stdout:2212: read oid 23 snap -1 2026-02-17T05:59:27.302 INFO:tasks.rados.rados.0.trial115.stdout:2212: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:59:27.303 INFO:tasks.rados.rados.0.trial115.stdout:2213: delete oid 11 current snap is 221 2026-02-17T05:59:27.310 INFO:tasks.rados.rados.0.trial115.stdout:2212: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:59:27.459 INFO:tasks.rados.rados.0.trial115.stdout:2212: done (1 left) 2026-02-17T05:59:27.459 INFO:tasks.rados.rados.0.trial115.stdout:2213: done (0 left) 2026-02-17T05:59:27.459 INFO:tasks.rados.rados.0.trial115.stdout:2214: write oid 17 current snap is 221 2026-02-17T05:59:27.459 INFO:tasks.rados.rados.0.trial115.stdout:2214: seq_num 786 ranges {410302=730073,1895082=402268,2951884=258418} 2026-02-17T05:59:27.464 INFO:tasks.rados.rados.0.trial115.stdout:2214: writing trial11545420-17 from 410302 to 1140375 tid 1 2026-02-17T05:59:27.468 INFO:tasks.rados.rados.0.trial115.stdout:2214: writing trial11545420-17 from 1895082 to 2297350 tid 2 2026-02-17T05:59:27.471 INFO:tasks.rados.rados.0.trial115.stdout:2214: writing trial11545420-17 from 2951884 to 3210302 tid 3 2026-02-17T05:59:27.472 INFO:tasks.rados.rados.0.trial115.stdout:2215: snap_remove snap 217 2026-02-17T05:59:27.472 INFO:tasks.rados.rados.0.trial115.stdout:2214: finishing write tid 1 to trial11545420-17 2026-02-17T05:59:27.472 INFO:tasks.rados.rados.0.trial115.stdout:2214: finishing write tid 2 to trial11545420-17 2026-02-17T05:59:28.304 INFO:tasks.rados.rados.0.trial115.stdout:2215: done (1 left) 2026-02-17T05:59:28.304 INFO:tasks.rados.rados.0.trial115.stdout:2216: write (excl) oid 26 current snap is 221 2026-02-17T05:59:28.305 INFO:tasks.rados.rados.0.trial115.stdout:2216: seq_num 787 ranges {427109=405322,1553564=545926,2579307=647802} 2026-02-17T05:59:28.307 INFO:tasks.rados.rados.0.trial115.stdout:2216: writing trial11545420-26 from 427109 to 832431 tid 1 2026-02-17T05:59:28.312 INFO:tasks.rados.rados.0.trial115.stdout:2216: writing trial11545420-26 from 1553564 to 2099490 tid 2 2026-02-17T05:59:28.316 INFO:tasks.rados.rados.0.trial115.stdout:2216: writing trial11545420-26 from 2579307 to 3227109 tid 3 2026-02-17T05:59:28.317 INFO:tasks.rados.rados.0.trial115.stdout:2217: read oid 13 snap -1 2026-02-17T05:59:28.317 INFO:tasks.rados.rados.0.trial115.stdout:2217: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:59:28.317 INFO:tasks.rados.rados.0.trial115.stdout:2218: snap_remove snap 216 2026-02-17T05:59:29.306 INFO:tasks.rados.rados.0.trial115.stdout:2218: done (3 left) 2026-02-17T05:59:29.306 INFO:tasks.rados.rados.0.trial115.stdout:2219: rollback oid 8 current snap is 221 2026-02-17T05:59:29.306 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 8 to 220 2026-02-17T05:59:29.306 INFO:tasks.rados.rados.0.trial115.stdout:2214: finishing write tid 3 to trial11545420-17 2026-02-17T05:59:29.306 INFO:tasks.rados.rados.0.trial115.stdout:2214: finishing write tid 4 to trial11545420-17 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2214: finishing write tid 5 to trial11545420-17 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2214: oid 17 updating version 0 to 1045 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2214: oid 17 updating version 1045 to 1046 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2214: oid 17 version 1046 is already newer than 1044 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2214: oid 17 version 1046 is already newer than 1043 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1046 (ObjNum 786 snap 221 seq_num 786) dirty exists 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2214: left oid 17 (ObjNum 786 snap 221 seq_num 786) 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2216: finishing write tid 1 to trial11545420-26 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2216: finishing write tid 2 to trial11545420-26 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2216: finishing write tid 3 to trial11545420-26 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2216: finishing write tid 4 to trial11545420-26 2026-02-17T05:59:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2216: finishing write tid 5 to trial11545420-26 2026-02-17T05:59:29.308 INFO:tasks.rados.rados.0.trial115.stdout:2216: oid 26 updating version 0 to 952 2026-02-17T05:59:29.308 INFO:tasks.rados.rados.0.trial115.stdout:2216: oid 26 version 952 is already newer than 951 2026-02-17T05:59:29.308 INFO:tasks.rados.rados.0.trial115.stdout:2216: oid 26 updating version 952 to 954 2026-02-17T05:59:29.308 INFO:tasks.rados.rados.0.trial115.stdout:2216: oid 26 version 954 is already newer than 953 2026-02-17T05:59:29.308 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 954 (ObjNum 787 snap 221 seq_num 787) dirty exists 2026-02-17T05:59:29.308 INFO:tasks.rados.rados.0.trial115.stdout:2216: left oid 26 (ObjNum 787 snap 221 seq_num 787) 2026-02-17T05:59:29.308 INFO:tasks.rados.rados.0.trial115.stdout:2217: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:59:29.455 INFO:tasks.rados.rados.0.trial115.stdout:2219: finishing rollback tid 0 to trial11545420-8 2026-02-17T05:59:29.455 INFO:tasks.rados.rados.0.trial115.stdout:2219: finishing rollback tid 1 to trial11545420-8 2026-02-17T05:59:29.455 INFO:tasks.rados.rados.0.trial115.stdout:2219: finishing rollback tid 2 to trial11545420-8 2026-02-17T05:59:29.455 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 265 (ObjNum 784 snap 220 seq_num 784) dirty exists 2026-02-17T05:59:29.455 INFO:tasks.rados.rados.0.trial115.stdout:2214: done (3 left) 2026-02-17T05:59:29.455 INFO:tasks.rados.rados.0.trial115.stdout:2216: done (2 left) 2026-02-17T05:59:29.455 INFO:tasks.rados.rados.0.trial115.stdout:2217: done (1 left) 2026-02-17T05:59:29.455 INFO:tasks.rados.rados.0.trial115.stdout:2219: done (0 left) 2026-02-17T05:59:29.456 INFO:tasks.rados.rados.0.trial115.stdout:2220: copy_from oid 17 from oid 1 current snap is 221 2026-02-17T05:59:29.456 INFO:tasks.rados.rados.0.trial115.stdout:2221: snap_create 2026-02-17T05:59:29.462 INFO:tasks.rados.rados.0.trial115.stdout:2220: finishing copy_from racing read to trial11545420-17 2026-02-17T05:59:29.464 INFO:tasks.rados.rados.0.trial115.stdout:2220: finishing copy_from to trial11545420-17 2026-02-17T05:59:29.464 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 874 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:59:30.311 INFO:tasks.rados.rados.0.trial115.stdout:2220: done (1 left) 2026-02-17T05:59:30.311 INFO:tasks.rados.rados.0.trial115.stdout:2221: done (0 left) 2026-02-17T05:59:30.311 INFO:tasks.rados.rados.0.trial115.stdout:2222: delete oid 48 current snap is 222 2026-02-17T05:59:30.312 INFO:tasks.rados.rados.0.trial115.stdout:2222: done (0 left) 2026-02-17T05:59:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2223: snap_remove snap 220 2026-02-17T05:59:31.312 INFO:tasks.rados.rados.0.trial115.stdout:2223: done (0 left) 2026-02-17T05:59:31.313 INFO:tasks.rados.rados.0.trial115.stdout:2224: snap_create 2026-02-17T05:59:32.300 INFO:tasks.rados.rados.0.trial115.stdout:2224: done (0 left) 2026-02-17T05:59:32.301 INFO:tasks.rados.rados.0.trial115.stdout:2225: snap_create 2026-02-17T05:59:33.301 INFO:tasks.rados.rados.0.trial115.stdout:2225: done (0 left) 2026-02-17T05:59:33.302 INFO:tasks.rados.rados.0.trial115.stdout:2226: write (excl) oid 44 current snap is 224 2026-02-17T05:59:33.302 INFO:tasks.rados.rados.0.trial115.stdout:2226: seq_num 789 ranges {460723=555820,1670528=549595,2634153=626570} 2026-02-17T05:59:33.306 INFO:tasks.rados.rados.0.trial115.stdout:2226: writing trial11545420-44 from 460723 to 1016543 tid 1 2026-02-17T05:59:33.310 INFO:tasks.rados.rados.0.trial115.stdout:2226: writing trial11545420-44 from 1670528 to 2220123 tid 2 2026-02-17T05:59:33.315 INFO:tasks.rados.rados.0.trial115.stdout:2226: writing trial11545420-44 from 2634153 to 3260723 tid 3 2026-02-17T05:59:33.315 INFO:tasks.rados.rados.0.trial115.stdout:2227: read oid 45 snap 222 2026-02-17T05:59:33.315 INFO:tasks.rados.rados.0.trial115.stdout:2227: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:59:33.315 INFO:tasks.rados.rados.0.trial115.stdout:2228: write oid 19 current snap is 224 2026-02-17T05:59:33.315 INFO:tasks.rados.rados.0.trial115.stdout:2226: finishing write tid 1 to trial11545420-44 2026-02-17T05:59:33.315 INFO:tasks.rados.rados.0.trial115.stdout:2226: finishing write tid 2 to trial11545420-44 2026-02-17T05:59:33.315 INFO:tasks.rados.rados.0.trial115.stdout:2228: seq_num 790 ranges {477530=631069,1612657=409606,2545223=517596} 2026-02-17T05:59:33.318 INFO:tasks.rados.rados.0.trial115.stdout:2228: writing trial11545420-19 from 477530 to 1108599 tid 1 2026-02-17T05:59:33.322 INFO:tasks.rados.rados.0.trial115.stdout:2228: writing trial11545420-19 from 1612657 to 2022263 tid 2 2026-02-17T05:59:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2228: writing trial11545420-19 from 2545223 to 3062819 tid 3 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2229: copy_from oid 6 from oid 30 current snap is 224 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2226: finishing write tid 3 to trial11545420-44 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2226: finishing write tid 4 to trial11545420-44 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2226: finishing write tid 5 to trial11545420-44 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2226: oid 44 updating version 0 to 887 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2226: oid 44 updating version 887 to 888 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2226: oid 44 updating version 888 to 890 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2226: oid 44 version 890 is already newer than 889 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 890 (ObjNum 789 snap 224 seq_num 789) dirty exists 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2226: left oid 44 (ObjNum 789 snap 224 seq_num 789) 2026-02-17T05:59:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2227: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:59:33.522 INFO:tasks.rados.rados.0.trial115.stdout:2228: finishing write tid 1 to trial11545420-19 2026-02-17T05:59:33.522 INFO:tasks.rados.rados.0.trial115.stdout:2226: done (3 left) 2026-02-17T05:59:33.522 INFO:tasks.rados.rados.0.trial115.stdout:2227: done (2 left) 2026-02-17T05:59:33.522 INFO:tasks.rados.rados.0.trial115.stdout:2230: rollback oid 14 current snap is 224 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 221 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: finishing write tid 2 to trial11545420-19 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: finishing write tid 3 to trial11545420-19 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: finishing write tid 4 to trial11545420-19 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: finishing write tid 5 to trial11545420-19 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: oid 19 updating version 0 to 1266 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: oid 19 updating version 1266 to 1267 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: oid 19 updating version 1267 to 1268 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: oid 19 version 1268 is already newer than 1265 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1268 (ObjNum 790 snap 224 seq_num 790) dirty exists 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2228: left oid 19 (ObjNum 790 snap 224 seq_num 790) 2026-02-17T05:59:33.523 INFO:tasks.rados.rados.0.trial115.stdout:2229: finishing copy_from racing read to trial11545420-6 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2229: finishing copy_from to trial11545420-6 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 952 (ObjNum 767 snap 214 seq_num 767) dirty exists 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2228: done (2 left) 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2229: done (1 left) 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2231: read oid 5 snap -1 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2231: expect (ObjNum 777 snap 219 seq_num 777) 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2232: read oid 10 snap -1 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2232: expect (ObjNum 756 snap 207 seq_num 756) 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2233: read oid 19 snap 222 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2233: expect (ObjNum 723 snap 199 seq_num 723) 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2234: read oid 39 snap -1 2026-02-17T05:59:33.524 INFO:tasks.rados.rados.0.trial115.stdout:2234: expect deleted 2026-02-17T05:59:33.525 INFO:tasks.rados.rados.0.trial115.stdout:2235: rollback oid 17 current snap is 224 2026-02-17T05:59:33.525 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 223 2026-02-17T05:59:33.525 INFO:tasks.rados.rados.0.trial115.stdout:2236: snap_create 2026-02-17T05:59:33.526 INFO:tasks.rados.rados.0.trial115.stdout:2230: finishing rollback tid 0 to trial11545420-14 2026-02-17T05:59:33.526 INFO:tasks.rados.rados.0.trial115.stdout:2235: finishing rollback tid 0 to trial11545420-17 2026-02-17T05:59:33.526 INFO:tasks.rados.rados.0.trial115.stdout:2230: finishing rollback tid 1 to trial11545420-14 2026-02-17T05:59:33.526 INFO:tasks.rados.rados.0.trial115.stdout:2235: finishing rollback tid 1 to trial11545420-17 2026-02-17T05:59:33.526 INFO:tasks.rados.rados.0.trial115.stdout:2231: expect (ObjNum 777 snap 219 seq_num 777) 2026-02-17T05:59:33.654 INFO:tasks.rados.rados.0.trial115.stdout:2233: expect (ObjNum 723 snap 199 seq_num 723) 2026-02-17T05:59:33.734 INFO:tasks.rados.rados.0.trial115.stdout:2232: expect (ObjNum 756 snap 207 seq_num 756) 2026-02-17T05:59:33.886 INFO:tasks.rados.rados.0.trial115.stdout:2230: finishing rollback tid 2 to trial11545420-14 2026-02-17T05:59:33.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 947 (ObjNum 700 snap 193 seq_num 700) dirty exists 2026-02-17T05:59:33.886 INFO:tasks.rados.rados.0.trial115.stdout:2235: finishing rollback tid 2 to trial11545420-17 2026-02-17T05:59:33.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1049 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:2230: done (6 left) 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:2231: done (5 left) 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:2232: done (4 left) 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:2233: done (3 left) 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:2234: done (2 left) 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:2235: done (1 left) 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:2236: done (0 left) 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:2237: rollback oid 7 current snap is 225 2026-02-17T05:59:34.320 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 223 2026-02-17T05:59:34.321 INFO:tasks.rados.rados.0.trial115.stdout:2238: write (excl) oid 8 current snap is 225 2026-02-17T05:59:34.321 INFO:tasks.rados.rados.0.trial115.stdout:2238: seq_num 791 ranges {494337=422671,1387492=553264,2688999=445221} 2026-02-17T05:59:34.322 INFO:tasks.rados.rados.0.trial115.stdout:2238: writing trial11545420-8 from 494337 to 917008 tid 1 2026-02-17T05:59:34.326 INFO:tasks.rados.rados.0.trial115.stdout:2238: writing trial11545420-8 from 1387492 to 1940756 tid 2 2026-02-17T05:59:34.331 INFO:tasks.rados.rados.0.trial115.stdout:2238: writing trial11545420-8 from 2688999 to 3134220 tid 3 2026-02-17T05:59:34.331 INFO:tasks.rados.rados.0.trial115.stdout:2239: snap_create 2026-02-17T05:59:34.331 INFO:tasks.rados.rados.0.trial115.stdout:2237: finishing rollback tid 0 to trial11545420-7 2026-02-17T05:59:34.331 INFO:tasks.rados.rados.0.trial115.stdout:2237: finishing rollback tid 1 to trial11545420-7 2026-02-17T05:59:34.331 INFO:tasks.rados.rados.0.trial115.stdout:2237: finishing rollback tid 2 to trial11545420-7 2026-02-17T05:59:34.331 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 940 (ObjNum 700 snap 193 seq_num 700) dirty exists 2026-02-17T05:59:34.334 INFO:tasks.rados.rados.0.trial115.stdout:2238: finishing write tid 1 to trial11545420-8 2026-02-17T05:59:34.334 INFO:tasks.rados.rados.0.trial115.stdout:2238: finishing write tid 2 to trial11545420-8 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:2238: finishing write tid 3 to trial11545420-8 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:2238: finishing write tid 4 to trial11545420-8 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:2238: finishing write tid 5 to trial11545420-8 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:2238: oid 8 updating version 0 to 273 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:2238: oid 8 version 273 is already newer than 272 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:2238: oid 8 updating version 273 to 274 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:2238: oid 8 version 274 is already newer than 271 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 274 (ObjNum 791 snap 225 seq_num 791) dirty exists 2026-02-17T05:59:34.335 INFO:tasks.rados.rados.0.trial115.stdout:2238: left oid 8 (ObjNum 791 snap 225 seq_num 791) 2026-02-17T05:59:35.321 INFO:tasks.rados.rados.0.trial115.stdout:2237: done (2 left) 2026-02-17T05:59:35.321 INFO:tasks.rados.rados.0.trial115.stdout:2238: done (1 left) 2026-02-17T05:59:35.321 INFO:tasks.rados.rados.0.trial115.stdout:2239: done (0 left) 2026-02-17T05:59:35.322 INFO:tasks.rados.rados.0.trial115.stdout:2240: snap_create 2026-02-17T05:59:36.324 INFO:tasks.rados.rados.0.trial115.stdout:2240: done (0 left) 2026-02-17T05:59:36.324 INFO:tasks.rados.rados.0.trial115.stdout:2241: read oid 41 snap -1 2026-02-17T05:59:36.324 INFO:tasks.rados.rados.0.trial115.stdout:2241: expect (ObjNum 762 snap 210 seq_num 762) 2026-02-17T05:59:36.324 INFO:tasks.rados.rados.0.trial115.stdout:2242: read oid 26 snap -1 2026-02-17T05:59:36.324 INFO:tasks.rados.rados.0.trial115.stdout:2242: expect (ObjNum 787 snap 221 seq_num 787) 2026-02-17T05:59:36.325 INFO:tasks.rados.rados.0.trial115.stdout:2243: rollback oid 1 current snap is 227 2026-02-17T05:59:36.325 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 1 to 224 2026-02-17T05:59:36.325 INFO:tasks.rados.rados.0.trial115.stdout:2244: snap_remove snap 226 2026-02-17T05:59:37.301 INFO:tasks.rados.rados.0.trial115.stdout:2244: done (3 left) 2026-02-17T05:59:37.302 INFO:tasks.rados.rados.0.trial115.stdout:2245: snap_create 2026-02-17T05:59:37.302 INFO:tasks.rados.rados.0.trial115.stdout:2241: expect (ObjNum 762 snap 210 seq_num 762) 2026-02-17T05:59:37.388 INFO:tasks.rados.rados.0.trial115.stdout:2243: finishing rollback tid 0 to trial11545420-1 2026-02-17T05:59:37.389 INFO:tasks.rados.rados.0.trial115.stdout:2243: finishing rollback tid 1 to trial11545420-1 2026-02-17T05:59:37.389 INFO:tasks.rados.rados.0.trial115.stdout:2243: finishing rollback tid 2 to trial11545420-1 2026-02-17T05:59:37.389 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 950 (ObjNum 696 snap 189 seq_num 696) dirty exists 2026-02-17T05:59:37.389 INFO:tasks.rados.rados.0.trial115.stdout:2242: expect (ObjNum 787 snap 221 seq_num 787) 2026-02-17T05:59:38.302 INFO:tasks.rados.rados.0.trial115.stdout:2241: done (3 left) 2026-02-17T05:59:38.303 INFO:tasks.rados.rados.0.trial115.stdout:2242: done (2 left) 2026-02-17T05:59:38.303 INFO:tasks.rados.rados.0.trial115.stdout:2243: done (1 left) 2026-02-17T05:59:38.303 INFO:tasks.rados.rados.0.trial115.stdout:2245: done (0 left) 2026-02-17T05:59:38.303 INFO:tasks.rados.rados.0.trial115.stdout:2246: snap_create 2026-02-17T05:59:39.304 INFO:tasks.rados.rados.0.trial115.stdout:2246: done (0 left) 2026-02-17T05:59:39.304 INFO:tasks.rados.rados.0.trial115.stdout:2247: snap_remove snap 227 2026-02-17T05:59:40.306 INFO:tasks.rados.rados.0.trial115.stdout:2247: done (0 left) 2026-02-17T05:59:40.306 INFO:tasks.rados.rados.0.trial115.stdout:2248: delete oid 2 current snap is 229 2026-02-17T05:59:40.310 INFO:tasks.rados.rados.0.trial115.stdout:2248: done (0 left) 2026-02-17T05:59:40.310 INFO:tasks.rados.rados.0.trial115.stdout:2249: write oid 47 current snap is 229 2026-02-17T05:59:40.310 INFO:tasks.rados.rados.0.trial115.stdout:2249: seq_num 793 ranges {527951=573169,1504456=556933,2743845=467765} 2026-02-17T05:59:40.314 INFO:tasks.rados.rados.0.trial115.stdout:2249: writing trial11545420-47 from 527951 to 1101120 tid 1 2026-02-17T05:59:40.318 INFO:tasks.rados.rados.0.trial115.stdout:2249: writing trial11545420-47 from 1504456 to 2061389 tid 2 2026-02-17T05:59:40.323 INFO:tasks.rados.rados.0.trial115.stdout:2249: writing trial11545420-47 from 2743845 to 3211610 tid 3 2026-02-17T05:59:40.323 INFO:tasks.rados.rados.0.trial115.stdout:2250: write (excl) oid 37 current snap is 229 2026-02-17T05:59:40.323 INFO:tasks.rados.rados.0.trial115.stdout:2250: seq_num 794 ranges {544758=648418,1846585=416944,2771268=573490} 2026-02-17T05:59:40.327 INFO:tasks.rados.rados.0.trial115.stdout:2250: writing trial11545420-37 from 544758 to 1193176 tid 1 2026-02-17T05:59:40.332 INFO:tasks.rados.rados.0.trial115.stdout:2250: writing trial11545420-37 from 1846585 to 2263529 tid 2 2026-02-17T05:59:40.336 INFO:tasks.rados.rados.0.trial115.stdout:2250: writing trial11545420-37 from 2771268 to 3344758 tid 3 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2251: delete oid 31 current snap is 229 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: finishing write tid 1 to trial11545420-47 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: finishing write tid 2 to trial11545420-47 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: finishing write tid 3 to trial11545420-47 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: finishing write tid 4 to trial11545420-47 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: finishing write tid 5 to trial11545420-47 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: oid 47 updating version 0 to 1072 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: oid 47 version 1072 is already newer than 1071 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: oid 47 updating version 1072 to 1074 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:2249: oid 47 version 1074 is already newer than 1073 2026-02-17T05:59:40.337 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1074 (ObjNum 793 snap 229 seq_num 793) dirty exists 2026-02-17T05:59:40.338 INFO:tasks.rados.rados.0.trial115.stdout:2249: left oid 47 (ObjNum 793 snap 229 seq_num 793) 2026-02-17T05:59:40.338 INFO:tasks.rados.rados.0.trial115.stdout:2250: finishing write tid 1 to trial11545420-37 2026-02-17T05:59:40.338 INFO:tasks.rados.rados.0.trial115.stdout:2250: finishing write tid 2 to trial11545420-37 2026-02-17T05:59:40.338 INFO:tasks.rados.rados.0.trial115.stdout:2249: done (2 left) 2026-02-17T05:59:40.338 INFO:tasks.rados.rados.0.trial115.stdout:2251: done (1 left) 2026-02-17T05:59:40.339 INFO:tasks.rados.rados.0.trial115.stdout:2252: write oid 10 current snap is 229 2026-02-17T05:59:40.339 INFO:tasks.rados.rados.0.trial115.stdout:2252: seq_num 796 ranges {578372=798916,1963549=420613,2826114=417934} 2026-02-17T05:59:40.343 INFO:tasks.rados.rados.0.trial115.stdout:2252: writing trial11545420-10 from 578372 to 1377288 tid 1 2026-02-17T05:59:40.347 INFO:tasks.rados.rados.0.trial115.stdout:2252: writing trial11545420-10 from 1963549 to 2384162 tid 2 2026-02-17T05:59:40.351 INFO:tasks.rados.rados.0.trial115.stdout:2252: writing trial11545420-10 from 2826114 to 3244048 tid 3 2026-02-17T05:59:40.351 INFO:tasks.rados.rados.0.trial115.stdout:2253: snap_remove snap 228 2026-02-17T05:59:41.310 INFO:tasks.rados.rados.0.trial115.stdout:2253: done (2 left) 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2254: copy_from oid 41 from oid 5 current snap is 229 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2250: finishing write tid 3 to trial11545420-37 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2250: finishing write tid 4 to trial11545420-37 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2250: finishing write tid 5 to trial11545420-37 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2250: oid 37 updating version 0 to 942 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2250: oid 37 updating version 942 to 944 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2250: oid 37 version 944 is already newer than 941 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:2250: oid 37 version 944 is already newer than 943 2026-02-17T05:59:41.311 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 944 (ObjNum 794 snap 229 seq_num 794) dirty exists 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2250: left oid 37 (ObjNum 794 snap 229 seq_num 794) 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: finishing write tid 1 to trial11545420-10 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: finishing write tid 2 to trial11545420-10 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: finishing write tid 3 to trial11545420-10 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: finishing write tid 4 to trial11545420-10 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: finishing write tid 5 to trial11545420-10 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: oid 10 updating version 0 to 1171 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: oid 10 version 1171 is already newer than 1170 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: oid 10 updating version 1171 to 1173 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: oid 10 version 1173 is already newer than 1172 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1173 (ObjNum 796 snap 229 seq_num 796) dirty exists 2026-02-17T05:59:41.312 INFO:tasks.rados.rados.0.trial115.stdout:2252: left oid 10 (ObjNum 796 snap 229 seq_num 796) 2026-02-17T05:59:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2250: done (2 left) 2026-02-17T05:59:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2252: done (1 left) 2026-02-17T05:59:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2255: read oid 9 snap 222 2026-02-17T05:59:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2255: expect (ObjNum 759 snap 207 seq_num 759) 2026-02-17T05:59:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2256: write (excl) oid 36 current snap is 229 2026-02-17T05:59:41.313 INFO:tasks.rados.rados.0.trial115.stdout:2256: seq_num 797 ranges {595179=474165,1622031=564271,2737184=629206} 2026-02-17T05:59:41.314 INFO:tasks.rados.rados.0.trial115.stdout:2256: writing trial11545420-36 from 595179 to 1069344 tid 1 2026-02-17T05:59:41.319 INFO:tasks.rados.rados.0.trial115.stdout:2256: writing trial11545420-36 from 1622031 to 2186302 tid 2 2026-02-17T05:59:41.324 INFO:tasks.rados.rados.0.trial115.stdout:2256: writing trial11545420-36 from 2737184 to 3366390 tid 3 2026-02-17T05:59:41.324 INFO:tasks.rados.rados.0.trial115.stdout:2257: snap_create 2026-02-17T05:59:41.324 INFO:tasks.rados.rados.0.trial115.stdout:2255: expect (ObjNum 759 snap 207 seq_num 759) 2026-02-17T05:59:41.437 INFO:tasks.rados.rados.0.trial115.stdout:2256: finishing write tid 1 to trial11545420-36 2026-02-17T05:59:41.437 INFO:tasks.rados.rados.0.trial115.stdout:2254: finishing copy_from racing read to trial11545420-41 2026-02-17T05:59:41.437 INFO:tasks.rados.rados.0.trial115.stdout:2254: finishing copy_from to trial11545420-41 2026-02-17T05:59:41.437 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1055 (ObjNum 777 snap 219 seq_num 777) dirty exists 2026-02-17T05:59:41.437 INFO:tasks.rados.rados.0.trial115.stdout:2256: finishing write tid 2 to trial11545420-36 2026-02-17T05:59:41.437 INFO:tasks.rados.rados.0.trial115.stdout:2256: finishing write tid 3 to trial11545420-36 2026-02-17T05:59:41.437 INFO:tasks.rados.rados.0.trial115.stdout:2256: finishing write tid 4 to trial11545420-36 2026-02-17T05:59:41.437 INFO:tasks.rados.rados.0.trial115.stdout:2256: finishing write tid 5 to trial11545420-36 2026-02-17T05:59:41.438 INFO:tasks.rados.rados.0.trial115.stdout:2256: oid 36 updating version 0 to 951 2026-02-17T05:59:41.438 INFO:tasks.rados.rados.0.trial115.stdout:2256: oid 36 updating version 951 to 953 2026-02-17T05:59:41.438 INFO:tasks.rados.rados.0.trial115.stdout:2256: oid 36 version 953 is already newer than 952 2026-02-17T05:59:41.438 INFO:tasks.rados.rados.0.trial115.stdout:2256: oid 36 updating version 953 to 954 2026-02-17T05:59:41.438 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 954 (ObjNum 797 snap 229 seq_num 797) dirty exists 2026-02-17T05:59:41.438 INFO:tasks.rados.rados.0.trial115.stdout:2256: left oid 36 (ObjNum 797 snap 229 seq_num 797) 2026-02-17T05:59:42.303 INFO:tasks.rados.rados.0.trial115.stdout:2254: done (3 left) 2026-02-17T05:59:42.303 INFO:tasks.rados.rados.0.trial115.stdout:2255: done (2 left) 2026-02-17T05:59:42.303 INFO:tasks.rados.rados.0.trial115.stdout:2256: done (1 left) 2026-02-17T05:59:42.303 INFO:tasks.rados.rados.0.trial115.stdout:2257: done (0 left) 2026-02-17T05:59:42.304 INFO:tasks.rados.rados.0.trial115.stdout:2258: read oid 4 snap -1 2026-02-17T05:59:42.304 INFO:tasks.rados.rados.0.trial115.stdout:2258: expect (ObjNum 766 snap 212 seq_num 766) 2026-02-17T05:59:42.304 INFO:tasks.rados.rados.0.trial115.stdout:2259: write (excl) oid 20 current snap is 230 2026-02-17T05:59:42.304 INFO:tasks.rados.rados.0.trial115.stdout:2259: seq_num 798 ranges {611986=549414,1564160=424282,2764607=440478} 2026-02-17T05:59:42.307 INFO:tasks.rados.rados.0.trial115.stdout:2259: writing trial11545420-20 from 611986 to 1161400 tid 1 2026-02-17T05:59:42.310 INFO:tasks.rados.rados.0.trial115.stdout:2259: writing trial11545420-20 from 1564160 to 1988442 tid 2 2026-02-17T05:59:42.315 INFO:tasks.rados.rados.0.trial115.stdout:2259: writing trial11545420-20 from 2764607 to 3205085 tid 3 2026-02-17T05:59:42.315 INFO:tasks.rados.rados.0.trial115.stdout:2260: read oid 10 snap -1 2026-02-17T05:59:42.315 INFO:tasks.rados.rados.0.trial115.stdout:2260: expect (ObjNum 796 snap 229 seq_num 796) 2026-02-17T05:59:42.316 INFO:tasks.rados.rados.0.trial115.stdout:2261: rollback oid 21 current snap is 230 2026-02-17T05:59:42.316 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 21 to 223 2026-02-17T05:59:42.316 INFO:tasks.rados.rados.0.trial115.stdout:2258: expect (ObjNum 766 snap 212 seq_num 766) 2026-02-17T05:59:42.405 INFO:tasks.rados.rados.0.trial115.stdout:2259: finishing write tid 1 to trial11545420-20 2026-02-17T05:59:42.405 INFO:tasks.rados.rados.0.trial115.stdout:2259: finishing write tid 2 to trial11545420-20 2026-02-17T05:59:42.406 INFO:tasks.rados.rados.0.trial115.stdout:2258: done (3 left) 2026-02-17T05:59:42.406 INFO:tasks.rados.rados.0.trial115.stdout:2262: write (excl) oid 11 current snap is 230 2026-02-17T05:59:42.406 INFO:tasks.rados.rados.0.trial115.stdout:2262: seq_num 799 ranges {628793=741016,2138995=567940,3192030=236763} 2026-02-17T05:59:42.410 INFO:tasks.rados.rados.0.trial115.stdout:2262: writing trial11545420-11 from 628793 to 1369809 tid 1 2026-02-17T05:59:42.415 INFO:tasks.rados.rados.0.trial115.stdout:2262: writing trial11545420-11 from 2138995 to 2706935 tid 2 2026-02-17T05:59:42.418 INFO:tasks.rados.rados.0.trial115.stdout:2262: writing trial11545420-11 from 3192030 to 3428793 tid 3 2026-02-17T05:59:42.418 INFO:tasks.rados.rados.0.trial115.stdout:2263: snap_remove snap 221 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2263: done (4 left) 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2264: rollback oid 45 current snap is 230 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 229 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2259: finishing write tid 3 to trial11545420-20 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2259: finishing write tid 4 to trial11545420-20 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2259: finishing write tid 5 to trial11545420-20 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2259: oid 20 updating version 0 to 946 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2259: oid 20 version 946 is already newer than 945 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2259: oid 20 updating version 946 to 947 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2259: oid 20 updating version 947 to 948 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 948 (ObjNum 798 snap 230 seq_num 798) dirty exists 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2259: left oid 20 (ObjNum 798 snap 230 seq_num 798) 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2261: finishing rollback tid 0 to trial11545420-21 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2261: finishing rollback tid 1 to trial11545420-21 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2261: finishing rollback tid 2 to trial11545420-21 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1063 (ObjNum 747 snap 204 seq_num 747) dirty exists 2026-02-17T05:59:43.314 INFO:tasks.rados.rados.0.trial115.stdout:2260: expect (ObjNum 796 snap 229 seq_num 796) 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: finishing write tid 1 to trial11545420-11 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: finishing write tid 2 to trial11545420-11 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: finishing write tid 3 to trial11545420-11 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: finishing write tid 4 to trial11545420-11 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: finishing write tid 5 to trial11545420-11 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: oid 11 updating version 0 to 1178 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: oid 11 version 1178 is already newer than 1177 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: oid 11 version 1178 is already newer than 1176 2026-02-17T05:59:43.515 INFO:tasks.rados.rados.0.trial115.stdout:2262: oid 11 updating version 1178 to 1179 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1179 (ObjNum 799 snap 230 seq_num 799) dirty exists 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2262: left oid 11 (ObjNum 799 snap 230 seq_num 799) 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2264: finishing rollback tid 0 to trial11545420-45 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2264: finishing rollback tid 1 to trial11545420-45 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2264: finishing rollback tid 2 to trial11545420-45 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1295 (ObjNum 578 snap 150 seq_num 578) dirty exists 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2259: done (4 left) 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2260: done (3 left) 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2261: done (2 left) 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2262: done (1 left) 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2264: done (0 left) 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2265: read oid 33 snap -1 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2265: expect (ObjNum 770 snap 218 seq_num 770) 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2266: write oid 3 current snap is 230 2026-02-17T05:59:43.516 INFO:tasks.rados.rados.0.trial115.stdout:2266: seq_num 800 ranges {645600=416265,1797477=711598,3219453=226147} 2026-02-17T05:59:43.519 INFO:tasks.rados.rados.0.trial115.stdout:2266: writing trial11545420-3 from 645600 to 1061865 tid 1 2026-02-17T05:59:43.525 INFO:tasks.rados.rados.0.trial115.stdout:2266: writing trial11545420-3 from 1797477 to 2509075 tid 2 2026-02-17T05:59:43.528 INFO:tasks.rados.rados.0.trial115.stdout:2266: writing trial11545420-3 from 3219453 to 3445600 tid 3 2026-02-17T05:59:43.528 INFO:tasks.rados.rados.0.trial115.stdout:2267: rollback oid 49 current snap is 230 2026-02-17T05:59:43.528 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 49 to 224 2026-02-17T05:59:43.528 INFO:tasks.rados.rados.0.trial115.stdout:2266: finishing write tid 1 to trial11545420-3 2026-02-17T05:59:43.528 INFO:tasks.rados.rados.0.trial115.stdout:2265: expect (ObjNum 770 snap 218 seq_num 770) 2026-02-17T05:59:43.601 INFO:tasks.rados.rados.0.trial115.stdout:2266: finishing write tid 2 to trial11545420-3 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2266: finishing write tid 3 to trial11545420-3 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2266: finishing write tid 4 to trial11545420-3 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2266: finishing write tid 5 to trial11545420-3 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2266: oid 3 updating version 0 to 1090 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2266: oid 3 updating version 1090 to 1091 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2266: oid 3 updating version 1091 to 1093 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2266: oid 3 version 1093 is already newer than 1092 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1093 (ObjNum 800 snap 230 seq_num 800) dirty exists 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2266: left oid 3 (ObjNum 800 snap 230 seq_num 800) 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2267: finishing rollback tid 0 to trial11545420-49 2026-02-17T05:59:43.602 INFO:tasks.rados.rados.0.trial115.stdout:2267: finishing rollback tid 1 to trial11545420-49 2026-02-17T05:59:43.603 INFO:tasks.rados.rados.0.trial115.stdout:2267: finishing rollback tid 2 to trial11545420-49 2026-02-17T05:59:43.603 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 909 (ObjNum 692 snap 187 seq_num 692) dirty exists 2026-02-17T05:59:43.603 INFO:tasks.rados.rados.0.trial115.stdout:2265: done (2 left) 2026-02-17T05:59:43.603 INFO:tasks.rados.rados.0.trial115.stdout:2266: done (1 left) 2026-02-17T05:59:43.603 INFO:tasks.rados.rados.0.trial115.stdout:2267: done (0 left) 2026-02-17T05:59:43.603 INFO:tasks.rados.rados.0.trial115.stdout:2268: snap_create 2026-02-17T05:59:44.317 INFO:tasks.rados.rados.0.trial115.stdout:2268: done (0 left) 2026-02-17T05:59:44.318 INFO:tasks.rados.rados.0.trial115.stdout:2269: write (excl) oid 26 current snap is 231 2026-02-17T05:59:44.318 INFO:tasks.rados.rados.0.trial115.stdout:2269: seq_num 801 ranges {662407=491514,1855959=571609,2846876=615531} 2026-02-17T05:59:44.321 INFO:tasks.rados.rados.0.trial115.stdout:2269: writing trial11545420-26 from 662407 to 1153921 tid 1 2026-02-17T05:59:44.326 INFO:tasks.rados.rados.0.trial115.stdout:2269: writing trial11545420-26 from 1855959 to 2427568 tid 2 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:2269: writing trial11545420-26 from 2846876 to 3462407 tid 3 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:2270: copy_from oid 22 from oid 9 current snap is 231 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:2269: finishing write tid 1 to trial11545420-26 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:2269: finishing write tid 2 to trial11545420-26 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:2271: rollback oid 16 current snap is 231 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 16 to 223 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:2272: rollback oid 42 current snap is 231 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 222 2026-02-17T05:59:44.331 INFO:tasks.rados.rados.0.trial115.stdout:2273: snap_remove snap 225 2026-02-17T05:59:45.022 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:59:45.023 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T05:59:45.023 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T05:59:45.023 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T05:59:45.026 INFO:tasks.ceph.mon.a:Started 2026-02-17T05:59:45.026 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:59:45.026 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:45.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.091+0000 7efcf6e9f640 10 monclient: get_monmap_and_config 2026-02-17T05:59:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.091+0000 7efcf6e9f640 10 monclient: build_initial_monmap 2026-02-17T05:59:45.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.091+0000 7efcf6e9f640 10 monclient: monmap: 2026-02-17T05:59:45.095 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:45.095 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:45.095 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:45.092508+0000 2026-02-17T05:59:45.095 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:45.092508+0000 2026-02-17T05:59:45.095 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:45.095 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.091+0000 7efcf6e9f640 1 Processor -- start 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.091+0000 7efcf6e9f640 1 -- start start 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.091+0000 7efcf6e9f640 10 monclient: init 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.091+0000 7efcf6e9f640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:45.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.091+0000 7efcf6e9f640 10 monclient: _add_conns ranks=[2,0,1,4,3] 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 10 monclient(hunting): picked mon.noname-b con 0x7efcf006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efcf009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efcf006d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf0135c10 con 0x7efcf009c4a0 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf006daf0 con 0x7efcf006d1b0 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efcf00a0880 con 0x7efcf006c890 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf6e9f640 10 monclient(hunting): authenticate will time out at 1430.646063s 2026-02-17T05:59:45.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efce7fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7efcf009c4a0 0x7efcf0134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:45952/0 (socket says 10.20.193.78:45952) 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efce7fff640 1 -- 10.20.193.78:0/3930239167 learned_addr learned my addr 10.20.193.78:0/3930239167 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient(hunting): get_auth_request con 0x7efcf006c890 auth_method 0 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:45.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient(hunting): handle_auth_done global_id 144362 payload 274 2026-02-17T05:59:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 1 -- 10.20.193.78:0/3930239167 >> v1:10.20.193.115:6789/0 conn(0x7efcf006d1b0 legacy=0x7efcf0135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 1 -- 10.20.193.78:0/3930239167 >> v1:10.20.193.78:6789/0 conn(0x7efcf009c4a0 legacy=0x7efcf0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient: _finish_hunting 0 2026-02-17T05:59:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 1 monclient: found mon.noname-b 2026-02-17T05:59:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:45.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.092+0000 7efcf4c14640 1 -- 10.20.193.78:0/3930239167 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf009cc80 con 0x7efcf006c890 2026-02-17T05:59:45.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.094+0000 7efcf4c14640 1 -- 10.20.193.78:0/3930239167 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 msgr2=0x7efcf006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:45.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.094+0000 7efcf4c14640 1 --2- 10.20.193.78:0/3930239167 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf006cc70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efce00098d0 tx=0x7efce002ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856cdf1c47a6267f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:45.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.094+0000 7efce77fe640 1 -- 10.20.193.78:0/3930239167 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce003c070 con 0x7efcf006c890 2026-02-17T05:59:45.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.094+0000 7efce77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:45.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:45.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 10 monclient: dump: 2026-02-17T05:59:45.110 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:45.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 1 -- 10.20.193.78:0/3930239167 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce002fcc0 con 0x7efcf006c890 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 5 monclient: authenticate success, global_id 144362 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 1 -- 10.20.193.78:0/3930239167 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0037660 con 0x7efcf006c890 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 4 set_mon_vals no callback set 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:45.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efce77fe640 10 monclient: dump: 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 10 monclient: shutdown 2026-02-17T05:59:45.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3930239167 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 msgr2=0x7efcf006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 --2- 10.20.193.78:0/3930239167 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf006cc70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efce00098d0 tx=0x7efce002ed60 comp rx=0 tx=0).stop 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3930239167 shutdown_connections 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 --2- 10.20.193.78:0/3930239167 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf006cc70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3930239167 >> 10.20.193.78:0/3930239167 conn(0x7efcf00a6620 msgr2=0x7efcf00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3930239167 shutdown_connections 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3930239167 wait complete. 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 10 monclient: build_initial_monmap 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 10 monclient: monmap: 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:45.114 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 Processor -- start 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.095+0000 7efcf6e9f640 1 -- start start 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient: init 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:45.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf0133f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient(hunting): picked mon.noname-c con 0x7efcf006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcf006d1b0 0x7efcf01344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient(hunting): picked mon.noname-b con 0x7efcf006d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 0x7efcf0121a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient(hunting): picked mon.noname-a con 0x7efcf009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcf00a0020 con 0x7efcf009c4a0 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7efcf006d600 con 0x7efcf006d1b0 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efcf00a01a0 con 0x7efcf006c890 2026-02-17T05:59:45.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf6e9f640 10 monclient(hunting): authenticate will time out at 1430.650279s 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf0133f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf0133f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51568/0 (socket says 10.20.193.78:51568) 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 -- 10.20.193.78:0/3330622738 learned_addr learned my addr 10.20.193.78:0/3330622738 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient(hunting): get_auth_request con 0x7efcf006c890 auth_method 0 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 0x7efcf0121a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:45.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 10 monclient(hunting): get_auth_request con 0x7efcf009c4a0 auth_method 0 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 1 -- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 msgr2=0x7efcf0121a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 1 -- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 msgr2=0x7efcf0121a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 0x7efcf0121a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf5415640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 0x7efcf0121a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efce7fff640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcf006d1b0 0x7efcf01344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efce7fff640 10 monclient(hunting): get_auth_request con 0x7efcf006d1b0 auth_method 0 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efce7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:45.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efce7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efce7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient(hunting): handle_auth_done global_id 144365 payload 824 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 -- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcf006d1b0 msgr2=0x7efcf01344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcf006d1b0 0x7efcf01344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 -- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 msgr2=0x7efcf0121a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 0x7efcf0121a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient: _finish_hunting 0 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 monclient: found mon.noname-c 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.096+0000 7efcf4c14640 1 -- 10.20.193.78:0/3330622738 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf0134f10 con 0x7efcf006c890 2026-02-17T05:59:45.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf4c14640 1 -- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 msgr2=0x7efcf0133f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf4c14640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf0133f90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efce00098d0 tx=0x7efce00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 1 -- 10.20.193.78:0/3330622738 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0046070 con 0x7efcf006c890 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: dump: 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:45.120 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: _finish_auth 0 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: _check_auth_tickets 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 1 -- 10.20.193.78:0/3330622738 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce0041870 con 0x7efcf006c890 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf6e9f640 5 monclient: authenticate success, global_id 144365 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf6e9f640 10 monclient: _renew_subs 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf6e9f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf0124420 con 0x7efcf006c890 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 1 -- 10.20.193.78:0/3330622738 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce003c040 con 0x7efcf006c890 2026-02-17T05:59:45.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce57fa640 10 monclient: dump: 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:45.122 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efce6ffd640 4 set_mon_vals no callback set 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf6e9f640 10 monclient: _renew_subs 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf6e9f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.098+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf009d810 con 0x7efcf006c890 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efcf6e9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efcf6e9f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efcf6e9f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf0068410 con 0x7efcf006c890 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efce57fa640 1 -- 10.20.193.78:0/3330622738 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7efce0041c60 con 0x7efcf006c890 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efce57fa640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efcc003deb0 0x7efcc0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efce57fa640 1 -- 10.20.193.78:0/3330622738 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(487..487 src has 335..487) ==== 4751+0+0 (secure 0 0 0) 0x7efce00780a0 con 0x7efcf006c890 2026-02-17T05:59:45.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efce7fff640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efcc003deb0 0x7efcc0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:45.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.099+0000 7efce7fff640 10 monclient: get_auth_request con 0x7efcc003deb0 auth_method 0 2026-02-17T05:59:45.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.102+0000 7efce57fa640 1 -- 10.20.193.78:0/3330622738 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efcf0068410 con 0x7efcf006c890 2026-02-17T05:59:45.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.102+0000 7efce57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:45.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.102+0000 7efce57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:45.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.102+0000 7efce7fff640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efcc003deb0 0x7efcc0040370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efcf0134ec0 tx=0x7efcd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:45.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.216+0000 7efcf6e9f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:45.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.216+0000 7efcf6e9f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:45.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.216+0000 7efcf6e9f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:45.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.216+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efcf006cc70 con 0x7efcf006c890 2026-02-17T05:59:45.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.216+0000 7efce57fa640 1 -- 10.20.193.78:0/3330622738 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7efcf0134f10 con 0x7efcf006c890 2026-02-17T05:59:45.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.216+0000 7efce57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:45.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.216+0000 7efce57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:45.219 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":156,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efcc003deb0 msgr2=0x7efcc0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efcc003deb0 0x7efcc0040370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efcf0134ec0 tx=0x7efcd8009040 comp rx=0 tx=0).stop 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 10 monclient: shutdown 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 msgr2=0x7efcf0133f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf0133f90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efce00098d0 tx=0x7efce00047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 shutdown_connections 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efcf009c4a0 0x7efcf0121a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:45.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efcc003deb0 0x7efcc0040370 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:45.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efcf006d1b0 0x7efcf01344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:45.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 --2- 10.20.193.78:0/3330622738 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efcf006c890 0x7efcf0133f90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:45.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 >> 10.20.193.78:0/3330622738 conn(0x7efcf00a6620 msgr2=0x7efcf0135460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:45.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 shutdown_connections 2026-02-17T05:59:45.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:45.217+0000 7efcf6e9f640 1 -- 10.20.193.78:0/3330622738 wait complete. 2026-02-17T05:59:45.318 INFO:tasks.rados.rados.0.trial115.stdout:2273: done (4 left) 2026-02-17T05:59:45.318 INFO:tasks.rados.rados.0.trial115.stdout:2274: write (excl) oid 7 current snap is 231 2026-02-17T05:59:45.318 INFO:tasks.rados.rados.0.trial115.stdout:2269: finishing write tid 3 to trial11545420-26 2026-02-17T05:59:45.318 INFO:tasks.rados.rados.0.trial115.stdout:2274: seq_num 802 ranges {679214=566763,1798088=715267,3157946=321268} 2026-02-17T05:59:45.323 INFO:tasks.rados.rados.0.trial115.stdout:2274: writing trial11545420-7 from 679214 to 1245977 tid 1 2026-02-17T05:59:45.328 INFO:tasks.rados.rados.0.trial115.stdout:2274: writing trial11545420-7 from 1798088 to 2513355 tid 2 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2274: writing trial11545420-7 from 3157946 to 3479214 tid 3 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2275: delete oid 14 current snap is 231 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2271: finishing rollback tid 0 to trial11545420-16 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2271: finishing rollback tid 1 to trial11545420-16 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2271: finishing rollback tid 2 to trial11545420-16 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1276 (ObjNum 757 snap 207 seq_num 757) dirty exists 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2269: finishing write tid 4 to trial11545420-26 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2269: finishing write tid 5 to trial11545420-26 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2269: oid 26 updating version 0 to 955 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2269: oid 26 updating version 955 to 956 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2269: oid 26 updating version 956 to 957 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2269: oid 26 updating version 957 to 958 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 958 (ObjNum 801 snap 231 seq_num 801) dirty exists 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2269: left oid 26 (ObjNum 801 snap 231 seq_num 801) 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2272: finishing rollback tid 0 to trial11545420-42 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2272: finishing rollback tid 1 to trial11545420-42 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2272: finishing rollback tid 2 to trial11545420-42 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1299 (ObjNum 761 snap 207 seq_num 761) dirty exists 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2270: finishing copy_from racing read to trial11545420-22 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2270: finishing copy_from to trial11545420-22 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1110 (ObjNum 759 snap 207 seq_num 759) dirty exists 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2274: finishing write tid 1 to trial11545420-7 2026-02-17T05:59:45.332 INFO:tasks.rados.rados.0.trial115.stdout:2274: finishing write tid 2 to trial11545420-7 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:2274: finishing write tid 3 to trial11545420-7 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:2274: finishing write tid 4 to trial11545420-7 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:2274: finishing write tid 5 to trial11545420-7 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:2274: oid 7 updating version 0 to 950 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:2274: oid 7 updating version 950 to 952 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:2274: oid 7 version 952 is already newer than 949 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:2274: oid 7 version 952 is already newer than 951 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 952 (ObjNum 802 snap 231 seq_num 802) dirty exists 2026-02-17T05:59:45.333 INFO:tasks.rados.rados.0.trial115.stdout:2274: left oid 7 (ObjNum 802 snap 231 seq_num 802) 2026-02-17T05:59:45.334 INFO:tasks.rados.rados.0.trial115.stdout:2269: done (5 left) 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:2270: done (4 left) 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:2271: done (3 left) 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:2272: done (2 left) 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:2274: done (1 left) 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:2275: done (0 left) 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:2276: rollback oid 6 current snap is 231 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 6 to 224 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:2277: write (excl) oid 31 current snap is 231 2026-02-17T05:59:45.335 INFO:tasks.rados.rados.0.trial115.stdout:2277: seq_num 804 ranges {712828=717261,1915052=718936,3212792=300036} 2026-02-17T05:59:45.340 INFO:tasks.rados.rados.0.trial115.stdout:2277: writing trial11545420-31 from 712828 to 1430089 tid 1 2026-02-17T05:59:45.345 INFO:tasks.rados.rados.0.trial115.stdout:2277: writing trial11545420-31 from 1915052 to 2633988 tid 2 2026-02-17T05:59:45.348 INFO:tasks.rados.rados.0.trial115.stdout:2277: writing trial11545420-31 from 3212792 to 3512828 tid 3 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:2278: rollback oid 30 current snap is 231 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 223 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:2276: finishing rollback tid 0 to trial11545420-6 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:2276: finishing rollback tid 1 to trial11545420-6 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:2276: finishing rollback tid 2 to trial11545420-6 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 955 (ObjNum 767 snap 214 seq_num 767) dirty exists 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:2277: finishing write tid 1 to trial11545420-31 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:2277: finishing write tid 2 to trial11545420-31 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:2276: done (2 left) 2026-02-17T05:59:45.349 INFO:tasks.rados.rados.0.trial115.stdout:2279: delete oid 8 current snap is 231 2026-02-17T05:59:45.351 INFO:tasks.rados.rados.0.trial115.stdout:2277: finishing write tid 3 to trial11545420-31 2026-02-17T05:59:45.351 INFO:tasks.rados.rados.0.trial115.stdout:2277: finishing write tid 4 to trial11545420-31 2026-02-17T05:59:45.351 INFO:tasks.rados.rados.0.trial115.stdout:2277: finishing write tid 5 to trial11545420-31 2026-02-17T05:59:45.351 INFO:tasks.rados.rados.0.trial115.stdout:2277: oid 31 updating version 0 to 1104 2026-02-17T05:59:45.351 INFO:tasks.rados.rados.0.trial115.stdout:2277: oid 31 version 1104 is already newer than 1103 2026-02-17T05:59:45.351 INFO:tasks.rados.rados.0.trial115.stdout:2277: oid 31 version 1104 is already newer than 1094 2026-02-17T05:59:45.351 INFO:tasks.rados.rados.0.trial115.stdout:2277: oid 31 updating version 1104 to 1105 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1105 (ObjNum 804 snap 231 seq_num 804) dirty exists 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2277: left oid 31 (ObjNum 804 snap 231 seq_num 804) 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2278: finishing rollback tid 0 to trial11545420-30 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2278: finishing rollback tid 1 to trial11545420-30 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2277: done (2 left) 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2279: done (1 left) 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2280: read oid 28 snap -1 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2280: expect (ObjNum 747 snap 204 seq_num 747) 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2281: snap_create 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:2278: finishing rollback tid 2 to trial11545420-30 2026-02-17T05:59:45.352 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1113 (ObjNum 767 snap 214 seq_num 767) dirty exists 2026-02-17T05:59:45.353 INFO:tasks.rados.rados.0.trial115.stdout:2280: expect (ObjNum 747 snap 204 seq_num 747) 2026-02-17T05:59:46.325 INFO:tasks.rados.rados.0.trial115.stdout:2278: done (2 left) 2026-02-17T05:59:46.325 INFO:tasks.rados.rados.0.trial115.stdout:2280: done (1 left) 2026-02-17T05:59:46.325 INFO:tasks.rados.rados.0.trial115.stdout:2281: done (0 left) 2026-02-17T05:59:46.325 INFO:tasks.rados.rados.0.trial115.stdout:2282: copy_from oid 32 from oid 7 current snap is 232 2026-02-17T05:59:46.325 INFO:tasks.rados.rados.0.trial115.stdout:2283: read oid 41 snap -1 2026-02-17T05:59:46.326 INFO:tasks.rados.rados.0.trial115.stdout:2283: expect (ObjNum 777 snap 219 seq_num 777) 2026-02-17T05:59:46.326 INFO:tasks.rados.rados.0.trial115.stdout:2284: write (excl) oid 47 current snap is 232 2026-02-17T05:59:46.326 INFO:tasks.rados.rados.0.trial115.stdout:2284: seq_num 806 ranges {746442=584112,1748369=722605,2983991=562451} 2026-02-17T05:59:46.329 INFO:tasks.rados.rados.0.trial115.stdout:2284: writing trial11545420-47 from 746442 to 1330554 tid 1 2026-02-17T05:59:46.334 INFO:tasks.rados.rados.0.trial115.stdout:2284: writing trial11545420-47 from 1748369 to 2470974 tid 2 2026-02-17T05:59:46.338 INFO:tasks.rados.rados.0.trial115.stdout:2284: writing trial11545420-47 from 2983991 to 3546442 tid 3 2026-02-17T05:59:46.338 INFO:tasks.rados.rados.0.trial115.stdout:2285: read oid 13 snap 223 2026-02-17T05:59:46.338 INFO:tasks.rados.rados.0.trial115.stdout:2285: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:59:46.338 INFO:tasks.rados.rados.0.trial115.stdout:2286: write (excl) oid 38 current snap is 232 2026-02-17T05:59:46.338 INFO:tasks.rados.rados.0.trial115.stdout:2286: seq_num 807 ranges {763249=659361,2090498=582616,3295061=268188} 2026-02-17T05:59:46.343 INFO:tasks.rados.rados.0.trial115.stdout:2286: writing trial11545420-38 from 763249 to 1422610 tid 1 2026-02-17T05:59:46.348 INFO:tasks.rados.rados.0.trial115.stdout:2286: writing trial11545420-38 from 2090498 to 2673114 tid 2 2026-02-17T05:59:46.351 INFO:tasks.rados.rados.0.trial115.stdout:2286: writing trial11545420-38 from 3295061 to 3563249 tid 3 2026-02-17T05:59:46.351 INFO:tasks.rados.rados.0.trial115.stdout:2287: read oid 45 snap 224 2026-02-17T05:59:46.351 INFO:tasks.rados.rados.0.trial115.stdout:2287: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:59:46.352 INFO:tasks.rados.rados.0.trial115.stdout:2288: delete oid 34 current snap is 232 2026-02-17T05:59:46.352 INFO:tasks.rados.rados.0.trial115.stdout:2283: expect (ObjNum 777 snap 219 seq_num 777) 2026-02-17T05:59:46.479 INFO:tasks.rados.rados.0.trial115.stdout:2284: finishing write tid 1 to trial11545420-47 2026-02-17T05:59:46.479 INFO:tasks.rados.rados.0.trial115.stdout:2283: done (6 left) 2026-02-17T05:59:46.479 INFO:tasks.rados.rados.0.trial115.stdout:2288: done (5 left) 2026-02-17T05:59:46.479 INFO:tasks.rados.rados.0.trial115.stdout:2289: write oid 30 current snap is 232 2026-02-17T05:59:46.479 INFO:tasks.rados.rados.0.trial115.stdout:2282: finishing copy_from racing read to trial11545420-32 2026-02-17T05:59:46.479 INFO:tasks.rados.rados.0.trial115.stdout:2284: finishing write tid 2 to trial11545420-47 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2282: finishing copy_from to trial11545420-32 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 952 (ObjNum 802 snap 231 seq_num 802) dirty exists 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2284: finishing write tid 3 to trial11545420-47 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2284: finishing write tid 4 to trial11545420-47 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2284: finishing write tid 5 to trial11545420-47 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2284: oid 47 updating version 0 to 1109 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2284: oid 47 version 1109 is already newer than 1107 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2284: oid 47 version 1109 is already newer than 1108 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2284: oid 47 updating version 1109 to 1110 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1110 (ObjNum 806 snap 232 seq_num 806) dirty exists 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2284: left oid 47 (ObjNum 806 snap 232 seq_num 806) 2026-02-17T05:59:46.480 INFO:tasks.rados.rados.0.trial115.stdout:2285: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:59:46.627 INFO:tasks.rados.rados.0.trial115.stdout:2286: finishing write tid 1 to trial11545420-38 2026-02-17T05:59:46.627 INFO:tasks.rados.rados.0.trial115.stdout:2289: seq_num 809 ranges {796863=409859,1807462=586285,3066260=530603} 2026-02-17T05:59:46.631 INFO:tasks.rados.rados.0.trial115.stdout:2289: writing trial11545420-30 from 796863 to 1206722 tid 1 2026-02-17T05:59:46.636 INFO:tasks.rados.rados.0.trial115.stdout:2289: writing trial11545420-30 from 1807462 to 2393747 tid 2 2026-02-17T05:59:46.641 INFO:tasks.rados.rados.0.trial115.stdout:2289: writing trial11545420-30 from 3066260 to 3596863 tid 3 2026-02-17T05:59:46.641 INFO:tasks.rados.rados.0.trial115.stdout:2282: done (5 left) 2026-02-17T05:59:46.641 INFO:tasks.rados.rados.0.trial115.stdout:2284: done (4 left) 2026-02-17T05:59:46.641 INFO:tasks.rados.rados.0.trial115.stdout:2285: done (3 left) 2026-02-17T05:59:46.641 INFO:tasks.rados.rados.0.trial115.stdout:2290: snap_create 2026-02-17T05:59:46.641 INFO:tasks.rados.rados.0.trial115.stdout:2286: finishing write tid 2 to trial11545420-38 2026-02-17T05:59:46.641 INFO:tasks.rados.rados.0.trial115.stdout:2286: finishing write tid 3 to trial11545420-38 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:2286: finishing write tid 4 to trial11545420-38 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:2286: finishing write tid 5 to trial11545420-38 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:2286: oid 38 updating version 0 to 1281 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:2286: oid 38 updating version 1281 to 1282 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:2286: oid 38 updating version 1282 to 1283 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:2286: oid 38 version 1283 is already newer than 1280 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1283 (ObjNum 807 snap 232 seq_num 807) dirty exists 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:2286: left oid 38 (ObjNum 807 snap 232 seq_num 807) 2026-02-17T05:59:46.642 INFO:tasks.rados.rados.0.trial115.stdout:2287: expect (ObjNum 578 snap 150 seq_num 578) 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: finishing write tid 1 to trial11545420-30 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: finishing write tid 2 to trial11545420-30 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: finishing write tid 3 to trial11545420-30 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: finishing write tid 4 to trial11545420-30 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: finishing write tid 5 to trial11545420-30 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: oid 30 updating version 0 to 1117 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: oid 30 version 1117 is already newer than 1115 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: oid 30 version 1117 is already newer than 1116 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: oid 30 version 1117 is already newer than 1114 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1117 (ObjNum 809 snap 232 seq_num 809) dirty exists 2026-02-17T05:59:46.836 INFO:tasks.rados.rados.0.trial115.stdout:2289: left oid 30 (ObjNum 809 snap 232 seq_num 809) 2026-02-17T05:59:47.303 INFO:tasks.rados.rados.0.trial115.stdout:2286: done (3 left) 2026-02-17T05:59:47.303 INFO:tasks.rados.rados.0.trial115.stdout:2287: done (2 left) 2026-02-17T05:59:47.303 INFO:tasks.rados.rados.0.trial115.stdout:2289: done (1 left) 2026-02-17T05:59:47.304 INFO:tasks.rados.rados.0.trial115.stdout:2290: done (0 left) 2026-02-17T05:59:47.304 INFO:tasks.rados.rados.0.trial115.stdout:2291: snap_create 2026-02-17T05:59:48.226 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:48.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.292+0000 7f2b055fb640 10 monclient: get_monmap_and_config 2026-02-17T05:59:48.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.292+0000 7f2b055fb640 10 monclient: build_initial_monmap 2026-02-17T05:59:48.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.292+0000 7f2b055fb640 10 monclient: monmap: 2026-02-17T05:59:48.297 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:48.297 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:48.297 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:48.294172+0000 2026-02-17T05:59:48.297 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:48.294172+0000 2026-02-17T05:59:48.297 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:48.297 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:48.297 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 1 Processor -- start 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 1 -- start start 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient: init 2026-02-17T05:59:48.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b000a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient(hunting): picked mon.noname-b con 0x7f2b0006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2b0006cc70 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2b0013a240 0x7f2b0013c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient(hunting): picked mon.noname-c con 0x7f2b0013a240 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b00074150 con 0x7f2b0006cc70 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2b0009ef70 con 0x7f2b0013a240 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:48.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0009f250 con 0x7f2b0006c890 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2b055fb640 10 monclient(hunting): authenticate will time out at 1433.847697s 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afe7fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f2b0006cc70 0x7f2b0006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:36906/0 (socket says 10.20.193.78:36906) 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afe7fc640 1 -- 10.20.193.78:0/3408058856 learned_addr learned my addr 10.20.193.78:0/3408058856 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afeffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b000a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afeffd640 10 monclient(hunting): get_auth_request con 0x7f2b0006c890 auth_method 0 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:48.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2aff7fe640 1 --2- 10.20.193.78:0/3408058856 >> v2:10.20.193.115:3300/0 conn(0x7f2b0013a240 0x7f2b0013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2afeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2aff7fe640 10 monclient(hunting): get_auth_request con 0x7f2b0013a240 auth_method 0 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2aff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2aff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.293+0000 7f2aff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 10 monclient(hunting): handle_auth_done global_id 144368 payload 274 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 1 -- 10.20.193.78:0/3408058856 >> v2:10.20.193.115:3300/0 conn(0x7f2b0013a240 msgr2=0x7f2b0013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 1 --2- 10.20.193.78:0/3408058856 >> v2:10.20.193.115:3300/0 conn(0x7f2b0013a240 0x7f2b0013c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 1 -- 10.20.193.78:0/3408058856 >> v1:10.20.193.78:6789/0 conn(0x7f2b0006cc70 legacy=0x7f2b0006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 1 monclient: found mon.noname-b 2026-02-17T05:59:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2afeffd640 1 -- 10.20.193.78:0/3408058856 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0009df40 con 0x7f2b0006c890 2026-02-17T05:59:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.294+0000 7f2aff7fe640 1 --2- 10.20.193.78:0/3408058856 >> v2:10.20.193.115:3300/0 conn(0x7f2b0013a240 0x7f2b0013c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:59:48.308 INFO:tasks.rados.rados.0.trial115.stdout:2291: done (0 left) 2026-02-17T05:59:48.308 INFO:tasks.rados.rados.0.trial115.stdout:2292: snap_remove snap 230 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afeffd640 1 -- 10.20.193.78:0/3408058856 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 msgr2=0x7f2b000a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afeffd640 1 --2- 10.20.193.78:0/3408058856 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b000a01f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b0006e270 tx=0x7f2af402eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8edcca76476f87d7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 1 -- 10.20.193.78:0/3408058856 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af403c070 con 0x7f2b0006c890 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: dump: 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:48.310 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2b055fb640 5 monclient: authenticate success, global_id 144368 2026-02-17T05:59:48.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 1 -- 10.20.193.78:0/3408058856 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af402fcf0 con 0x7f2b0006c890 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2b055fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 1 -- 10.20.193.78:0/3408058856 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4037660 con 0x7f2b0006c890 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2b055fb640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2b055fb640 4 set_mon_vals no callback set 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.296+0000 7f2afdffb640 10 monclient: dump: 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:48.312 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient: shutdown 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- 10.20.193.78:0/3408058856 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 msgr2=0x7f2b000a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 --2- 10.20.193.78:0/3408058856 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b000a01f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b0006e270 tx=0x7f2af402eeb0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- 10.20.193.78:0/3408058856 shutdown_connections 2026-02-17T05:59:48.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 --2- 10.20.193.78:0/3408058856 >> v2:10.20.193.115:3300/0 conn(0x7f2b0013a240 0x7f2b0013c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 --2- 10.20.193.78:0/3408058856 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b000a01f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- 10.20.193.78:0/3408058856 >> 10.20.193.78:0/3408058856 conn(0x7f2b000a6620 msgr2=0x7f2b000a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- 10.20.193.78:0/3408058856 shutdown_connections 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- 10.20.193.78:0/3408058856 wait complete. 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient: build_initial_monmap 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient: monmap: 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:48.314 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 Processor -- start 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- start start 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient: init 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:59:48.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b001cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient(hunting): picked mon.noname-c con 0x7f2b0006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0006cc70 0x7f2b001354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient(hunting): picked mon.noname-b con 0x7f2b0006cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 0x7f2b00135a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient(hunting): picked mon.noname-a con 0x7f2b0013a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0013ccf0 con 0x7f2b0013a240 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0013cb70 con 0x7f2b0006cc70 2026-02-17T05:59:48.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0013ce70 con 0x7f2b0006c890 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2b055fb640 10 monclient(hunting): authenticate will time out at 1433.851877s 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b001cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2aff7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 0x7f2b00135a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b001cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41126/0 (socket says 10.20.193.78:41126) 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 1 -- 10.20.193.78:0/792511326 learned_addr learned my addr 10.20.193.78:0/792511326 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 10 monclient(hunting): get_auth_request con 0x7f2b0006c890 auth_method 0 2026-02-17T05:59:48.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2aff7fe640 10 monclient(hunting): get_auth_request con 0x7f2b0013a240 auth_method 0 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2aff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2aff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2aff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.297+0000 7f2afeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:48.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2aff7fe640 1 -- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 msgr2=0x7f2b00135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2aff7fe640 1 -- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 msgr2=0x7f2b00135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2aff7fe640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 0x7f2b00135a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2aff7fe640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 0x7f2b00135a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afe7fc640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0006cc70 0x7f2b001354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afe7fc640 10 monclient(hunting): get_auth_request con 0x7f2b0006cc70 auth_method 0 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afe7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afe7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 10 monclient(hunting): handle_auth_done global_id 144371 payload 824 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 1 -- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0006cc70 msgr2=0x7f2b001354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:48.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0006cc70 0x7f2b001354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 1 -- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 msgr2=0x7f2b00135a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 0x7f2b00135a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 10 monclient: _finish_hunting 0 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 1 monclient: found mon.noname-c 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.298+0000 7f2afeffd640 1 -- 10.20.193.78:0/792511326 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b000a3c10 con 0x7f2b0006c890 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2afeffd640 1 -- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 msgr2=0x7f2b001cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2afeffd640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b001cf100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2af4002410 tx=0x7f2af40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 1 -- 10.20.193.78:0/792511326 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af403c040 con 0x7f2b0006c890 2026-02-17T05:59:48.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: dump: 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:48.321 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 1 -- 10.20.193.78:0/792511326 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af4047070 con 0x7f2b0006c890 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2b055fb640 5 monclient: authenticate success, global_id 144371 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2b055fb640 10 monclient: _renew_subs 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2b055fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0009d810 con 0x7f2b0006c890 2026-02-17T05:59:48.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 1 -- 10.20.193.78:0/792511326 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4040a60 con 0x7f2b0006c890 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2afd7fa640 4 set_mon_vals no callback set 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2adf7fe640 10 monclient: dump: 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:48.323 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2b055fb640 10 monclient: _renew_subs 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2b055fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.300+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b00122610 con 0x7f2b0006c890 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2adf7fe640 1 -- 10.20.193.78:0/792511326 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2af403f030 con 0x7f2b0006c890 2026-02-17T05:59:48.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2b055fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2b055fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2b055fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b00122fc0 con 0x7f2b0006c890 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2adf7fe640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2acc03deb0 0x7f2acc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2adf7fe640 1 -- 10.20.193.78:0/792511326 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(490..490 src has 335..490) ==== 4779+0+0 (secure 0 0 0) 0x7f2af4077b40 con 0x7f2b0006c890 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2afe7fc640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2acc03deb0 0x7f2acc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.301+0000 7f2afe7fc640 10 monclient: get_auth_request con 0x7f2acc03deb0 auth_method 0 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.303+0000 7f2adf7fe640 1 -- 10.20.193.78:0/792511326 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2af4076540 con 0x7f2b0006c890 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.303+0000 7f2adf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.303+0000 7f2adf7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:48.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.304+0000 7f2afe7fc640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2acc03deb0 0x7f2acc040370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b0013bdd0 tx=0x7f2ae8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:48.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.418+0000 7f2b055fb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:48.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.418+0000 7f2b055fb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:48.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.418+0000 7f2b055fb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:48.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.418+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2b00074980 con 0x7f2b0006c890 2026-02-17T05:59:48.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.419+0000 7f2adf7fe640 1 -- 10.20.193.78:0/792511326 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f2af4035d70 con 0x7f2b0006c890 2026-02-17T05:59:48.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.419+0000 7f2adf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:48.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.419+0000 7f2adf7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:48.421 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":156,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2acc03deb0 msgr2=0x7f2acc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2acc03deb0 0x7f2acc040370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b0013bdd0 tx=0x7f2ae8009040 comp rx=0 tx=0).stop 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 10 monclient: shutdown 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 msgr2=0x7f2b001cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b001cf100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2af4002410 tx=0x7f2af40047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 shutdown_connections 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0013a240 0x7f2b00135a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2acc03deb0 0x7f2acc040370 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0006cc70 0x7f2b001354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 --2- 10.20.193.78:0/792511326 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0006c890 0x7f2b001cf100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:48.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 >> 10.20.193.78:0/792511326 conn(0x7f2b000a6620 msgr2=0x7f2b00138230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:48.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 shutdown_connections 2026-02-17T05:59:48.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:48.420+0000 7f2b055fb640 1 -- 10.20.193.78:0/792511326 wait complete. 2026-02-17T05:59:49.304 INFO:tasks.rados.rados.0.trial115.stdout:2292: done (0 left) 2026-02-17T05:59:49.304 INFO:tasks.rados.rados.0.trial115.stdout:2293: write oid 47 current snap is 234 2026-02-17T05:59:49.304 INFO:tasks.rados.rados.0.trial115.stdout:2293: seq_num 810 ranges {413670=485108,1465944=729943,2977330=524801} 2026-02-17T05:59:49.308 INFO:tasks.rados.rados.0.trial115.stdout:2293: writing trial11545420-47 from 413670 to 898778 tid 1 2026-02-17T05:59:49.313 INFO:tasks.rados.rados.0.trial115.stdout:2293: writing trial11545420-47 from 1465944 to 2195887 tid 2 2026-02-17T05:59:49.318 INFO:tasks.rados.rados.0.trial115.stdout:2293: writing trial11545420-47 from 2977330 to 3502131 tid 3 2026-02-17T05:59:49.318 INFO:tasks.rados.rados.0.trial115.stdout:2294: write oid 20 current snap is 234 2026-02-17T05:59:49.318 INFO:tasks.rados.rados.0.trial115.stdout:2293: finishing write tid 1 to trial11545420-47 2026-02-17T05:59:49.318 INFO:tasks.rados.rados.0.trial115.stdout:2293: finishing write tid 2 to trial11545420-47 2026-02-17T05:59:49.318 INFO:tasks.rados.rados.0.trial115.stdout:2294: seq_num 811 ranges {430477=560357,1408073=473601,2488400=736073} 2026-02-17T05:59:49.321 INFO:tasks.rados.rados.0.trial115.stdout:2294: writing trial11545420-20 from 430477 to 990834 tid 1 2026-02-17T05:59:49.325 INFO:tasks.rados.rados.0.trial115.stdout:2294: writing trial11545420-20 from 1408073 to 1881674 tid 2 2026-02-17T05:59:49.331 INFO:tasks.rados.rados.0.trial115.stdout:2294: writing trial11545420-20 from 2488400 to 3224473 tid 3 2026-02-17T05:59:49.331 INFO:tasks.rados.rados.0.trial115.stdout:2295: read oid 27 snap -1 2026-02-17T05:59:49.331 INFO:tasks.rados.rados.0.trial115.stdout:2295: expect (ObjNum 761 snap 207 seq_num 761) 2026-02-17T05:59:49.331 INFO:tasks.rados.rados.0.trial115.stdout:2296: copy_from oid 23 from oid 21 current snap is 234 2026-02-17T05:59:49.331 INFO:tasks.rados.rados.0.trial115.stdout:2293: finishing write tid 3 to trial11545420-47 2026-02-17T05:59:49.331 INFO:tasks.rados.rados.0.trial115.stdout:2293: finishing write tid 4 to trial11545420-47 2026-02-17T05:59:49.331 INFO:tasks.rados.rados.0.trial115.stdout:2293: finishing write tid 5 to trial11545420-47 2026-02-17T05:59:49.331 INFO:tasks.rados.rados.0.trial115.stdout:2293: oid 47 updating version 0 to 1112 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2293: oid 47 updating version 1112 to 1115 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2293: oid 47 updating version 1115 to 1117 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2293: oid 47 version 1117 is already newer than 1116 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1117 (ObjNum 810 snap 234 seq_num 810) dirty exists 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2293: left oid 47 (ObjNum 810 snap 234 seq_num 810) 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2294: finishing write tid 1 to trial11545420-20 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2294: finishing write tid 2 to trial11545420-20 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2293: done (3 left) 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2297: rollback oid 7 current snap is 234 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 233 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2298: write oid 36 current snap is 234 2026-02-17T05:59:49.332 INFO:tasks.rados.rados.0.trial115.stdout:2298: seq_num 812 ranges {447284=635606,1866555=733612,3315823=331461} 2026-02-17T05:59:49.334 INFO:tasks.rados.rados.0.trial115.stdout:2298: writing trial11545420-36 from 447284 to 1082890 tid 1 2026-02-17T05:59:49.340 INFO:tasks.rados.rados.0.trial115.stdout:2298: writing trial11545420-36 from 1866555 to 2600167 tid 2 2026-02-17T05:59:49.345 INFO:tasks.rados.rados.0.trial115.stdout:2298: writing trial11545420-36 from 3315823 to 3647284 tid 3 2026-02-17T05:59:49.345 INFO:tasks.rados.rados.0.trial115.stdout:2299: read oid 39 snap -1 2026-02-17T05:59:49.345 INFO:tasks.rados.rados.0.trial115.stdout:2299: expect deleted 2026-02-17T05:59:49.345 INFO:tasks.rados.rados.0.trial115.stdout:2300: read oid 26 snap -1 2026-02-17T05:59:49.345 INFO:tasks.rados.rados.0.trial115.stdout:2300: expect (ObjNum 801 snap 231 seq_num 801) 2026-02-17T05:59:49.345 INFO:tasks.rados.rados.0.trial115.stdout:2301: write (excl) oid 37 current snap is 234 2026-02-17T05:59:49.346 INFO:tasks.rados.rados.0.trial115.stdout:2301: seq_num 813 ranges {464091=710855,1925037=477270,2943246=474970} 2026-02-17T05:59:49.349 INFO:tasks.rados.rados.0.trial115.stdout:2301: writing trial11545420-37 from 464091 to 1174946 tid 1 2026-02-17T05:59:49.353 INFO:tasks.rados.rados.0.trial115.stdout:2301: writing trial11545420-37 from 1925037 to 2402307 tid 2 2026-02-17T05:59:49.358 INFO:tasks.rados.rados.0.trial115.stdout:2301: writing trial11545420-37 from 2943246 to 3418216 tid 3 2026-02-17T05:59:49.358 INFO:tasks.rados.rados.0.trial115.stdout:2302: read oid 46 snap -1 2026-02-17T05:59:49.358 INFO:tasks.rados.rados.0.trial115.stdout:2302: expect (ObjNum 696 snap 189 seq_num 696) 2026-02-17T05:59:49.358 INFO:tasks.rados.rados.0.trial115.stdout:2303: snap_create 2026-02-17T05:59:49.358 INFO:tasks.rados.rados.0.trial115.stdout:2294: finishing write tid 3 to trial11545420-20 2026-02-17T05:59:49.358 INFO:tasks.rados.rados.0.trial115.stdout:2294: finishing write tid 4 to trial11545420-20 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2294: finishing write tid 5 to trial11545420-20 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2294: oid 20 updating version 0 to 956 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2294: oid 20 version 956 is already newer than 954 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2294: oid 20 version 956 is already newer than 955 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2294: oid 20 version 956 is already newer than 953 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 956 (ObjNum 811 snap 234 seq_num 811) dirty exists 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2294: left oid 20 (ObjNum 811 snap 234 seq_num 811) 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2297: finishing rollback tid 0 to trial11545420-7 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2297: finishing rollback tid 1 to trial11545420-7 2026-02-17T05:59:49.359 INFO:tasks.rados.rados.0.trial115.stdout:2295: expect (ObjNum 761 snap 207 seq_num 761) 2026-02-17T05:59:49.510 INFO:tasks.rados.rados.0.trial115.stdout:2297: finishing rollback tid 2 to trial11545420-7 2026-02-17T05:59:49.510 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 959 (ObjNum 802 snap 231 seq_num 802) dirty exists 2026-02-17T05:59:49.510 INFO:tasks.rados.rados.0.trial115.stdout:2296: finishing copy_from racing read to trial11545420-23 2026-02-17T05:59:49.510 INFO:tasks.rados.rados.0.trial115.stdout:2296: finishing copy_from to trial11545420-23 2026-02-17T05:59:49.510 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1063 (ObjNum 747 snap 204 seq_num 747) dirty exists 2026-02-17T05:59:49.510 INFO:tasks.rados.rados.0.trial115.stdout:2298: finishing write tid 1 to trial11545420-36 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: finishing write tid 2 to trial11545420-36 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: finishing write tid 3 to trial11545420-36 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: finishing write tid 4 to trial11545420-36 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: finishing write tid 5 to trial11545420-36 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: oid 36 updating version 0 to 958 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: oid 36 updating version 958 to 959 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: oid 36 version 959 is already newer than 957 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: oid 36 version 959 is already newer than 956 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 959 (ObjNum 812 snap 234 seq_num 812) dirty exists 2026-02-17T05:59:49.511 INFO:tasks.rados.rados.0.trial115.stdout:2298: left oid 36 (ObjNum 812 snap 234 seq_num 812) 2026-02-17T05:59:49.512 INFO:tasks.rados.rados.0.trial115.stdout:2300: expect (ObjNum 801 snap 231 seq_num 801) 2026-02-17T05:59:49.651 INFO:tasks.rados.rados.0.trial115.stdout:2301: finishing write tid 1 to trial11545420-37 2026-02-17T05:59:49.651 INFO:tasks.rados.rados.0.trial115.stdout:2301: finishing write tid 2 to trial11545420-37 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2301: finishing write tid 3 to trial11545420-37 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2301: finishing write tid 4 to trial11545420-37 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2301: finishing write tid 5 to trial11545420-37 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2301: oid 37 updating version 0 to 963 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2301: oid 37 version 963 is already newer than 960 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2301: oid 37 version 963 is already newer than 962 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2301: oid 37 version 963 is already newer than 961 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 963 (ObjNum 813 snap 234 seq_num 813) dirty exists 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2301: left oid 37 (ObjNum 813 snap 234 seq_num 813) 2026-02-17T05:59:49.652 INFO:tasks.rados.rados.0.trial115.stdout:2302: expect (ObjNum 696 snap 189 seq_num 696) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2294: done (9 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2295: done (8 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2296: done (7 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2297: done (6 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2298: done (5 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2299: done (4 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2300: done (3 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2301: done (2 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2302: done (1 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2303: done (0 left) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2304: copy_from oid 21 from oid 7 current snap is 235 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2305: read oid 24 snap -1 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2305: expect (ObjNum 730 snap 202 seq_num 730) 2026-02-17T05:59:50.311 INFO:tasks.rados.rados.0.trial115.stdout:2306: delete oid 34 current snap is 235 2026-02-17T05:59:50.312 INFO:tasks.rados.rados.0.trial115.stdout:2306: done (2 left) 2026-02-17T05:59:50.312 INFO:tasks.rados.rados.0.trial115.stdout:2307: write (excl) oid 32 current snap is 235 2026-02-17T05:59:50.312 INFO:tasks.rados.rados.0.trial115.stdout:2307: seq_num 815 ranges {497705=577706,1642001=480939,2598092=497514} 2026-02-17T05:59:50.316 INFO:tasks.rados.rados.0.trial115.stdout:2307: writing trial11545420-32 from 497705 to 1075411 tid 1 2026-02-17T05:59:50.320 INFO:tasks.rados.rados.0.trial115.stdout:2307: writing trial11545420-32 from 1642001 to 2122940 tid 2 2026-02-17T05:59:50.323 INFO:tasks.rados.rados.0.trial115.stdout:2307: writing trial11545420-32 from 2598092 to 3095606 tid 3 2026-02-17T05:59:50.323 INFO:tasks.rados.rados.0.trial115.stdout:2308: write (excl) oid 39 current snap is 235 2026-02-17T05:59:50.323 INFO:tasks.rados.rados.0.trial115.stdout:2308: seq_num 816 ranges {514512=652955,1700483=740950,3025515=688997} 2026-02-17T05:59:50.328 INFO:tasks.rados.rados.0.trial115.stdout:2308: writing trial11545420-39 from 514512 to 1167467 tid 1 2026-02-17T05:59:50.333 INFO:tasks.rados.rados.0.trial115.stdout:2308: writing trial11545420-39 from 1700483 to 2441433 tid 2 2026-02-17T05:59:50.338 INFO:tasks.rados.rados.0.trial115.stdout:2308: writing trial11545420-39 from 3025515 to 3714512 tid 3 2026-02-17T05:59:50.338 INFO:tasks.rados.rados.0.trial115.stdout:2309: write (excl) oid 45 current snap is 235 2026-02-17T05:59:50.338 INFO:tasks.rados.rados.0.trial115.stdout:2309: seq_num 817 ranges {531319=728204,1758965=484608,2652938=520058} 2026-02-17T05:59:50.342 INFO:tasks.rados.rados.0.trial115.stdout:2309: writing trial11545420-45 from 531319 to 1259523 tid 1 2026-02-17T05:59:50.346 INFO:tasks.rados.rados.0.trial115.stdout:2309: writing trial11545420-45 from 1758965 to 2243573 tid 2 2026-02-17T05:59:50.350 INFO:tasks.rados.rados.0.trial115.stdout:2309: writing trial11545420-45 from 2652938 to 3172996 tid 3 2026-02-17T05:59:50.350 INFO:tasks.rados.rados.0.trial115.stdout:2310: write (excl) oid 18 current snap is 235 2026-02-17T05:59:50.350 INFO:tasks.rados.rados.0.trial115.stdout:2310: seq_num 818 ranges {548126=403453,1417447=744619,2680361=447683,3680450=67676} 2026-02-17T05:59:50.353 INFO:tasks.rados.rados.0.trial115.stdout:2310: writing trial11545420-18 from 548126 to 951579 tid 1 2026-02-17T05:59:50.358 INFO:tasks.rados.rados.0.trial115.stdout:2310: writing trial11545420-18 from 1417447 to 2162066 tid 2 2026-02-17T05:59:50.362 INFO:tasks.rados.rados.0.trial115.stdout:2310: writing trial11545420-18 from 2680361 to 3128044 tid 3 2026-02-17T05:59:50.364 INFO:tasks.rados.rados.0.trial115.stdout:2310: writing trial11545420-18 from 3680450 to 3748126 tid 4 2026-02-17T05:59:50.365 INFO:tasks.rados.rados.0.trial115.stdout:2311: read oid 10 snap 234 2026-02-17T05:59:50.365 INFO:tasks.rados.rados.0.trial115.stdout:2311: expect (ObjNum 796 snap 229 seq_num 796) 2026-02-17T05:59:50.365 INFO:tasks.rados.rados.0.trial115.stdout:2312: write oid 46 current snap is 235 2026-02-17T05:59:50.365 INFO:tasks.rados.rados.0.trial115.stdout:2312: seq_num 819 ranges {564933=478702,1759576=488277,2991431=658955} 2026-02-17T05:59:50.368 INFO:tasks.rados.rados.0.trial115.stdout:2312: writing trial11545420-46 from 564933 to 1043635 tid 1 2026-02-17T05:59:50.373 INFO:tasks.rados.rados.0.trial115.stdout:2312: writing trial11545420-46 from 1759576 to 2247853 tid 2 2026-02-17T05:59:50.379 INFO:tasks.rados.rados.0.trial115.stdout:2312: writing trial11545420-46 from 2991431 to 3650386 tid 3 2026-02-17T05:59:50.379 INFO:tasks.rados.rados.0.trial115.stdout:2313: snap_remove snap 233 2026-02-17T05:59:50.379 INFO:tasks.rados.rados.0.trial115.stdout:2305: expect (ObjNum 730 snap 202 seq_num 730) 2026-02-17T05:59:50.479 INFO:tasks.rados.rados.0.trial115.stdout:2307: finishing write tid 1 to trial11545420-32 2026-02-17T05:59:50.480 INFO:tasks.rados.rados.0.trial115.stdout:2304: finishing copy_from racing read to trial11545420-21 2026-02-17T05:59:51.314 INFO:tasks.rados.rados.0.trial115.stdout:2305: done (8 left) 2026-02-17T05:59:51.315 INFO:tasks.rados.rados.0.trial115.stdout:2313: done (7 left) 2026-02-17T05:59:51.315 INFO:tasks.rados.rados.0.trial115.stdout:2314: snap_remove snap 231 2026-02-17T05:59:51.430 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:51.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 10 monclient: get_monmap_and_config 2026-02-17T05:59:51.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 10 monclient: build_initial_monmap 2026-02-17T05:59:51.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 10 monclient: monmap: 2026-02-17T05:59:51.503 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:51.503 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:51.503 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:51.500490+0000 2026-02-17T05:59:51.503 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:51.500490+0000 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 1 Processor -- start 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 1 -- start start 2026-02-17T05:59:51.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 10 monclient: init 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 10 monclient(hunting): picked mon.noname-b con 0x7f91f006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.499+0000 7f91f5ce2640 10 monclient(hunting): picked mon.noname-a con 0x7f91f009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 10 monclient(hunting): picked mon.noname-c con 0x7f91f0134df0 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f91f00a0880 con 0x7f91f009c4a0 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:51.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f91f006d6f0 con 0x7f91f0134df0 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f91f006daf0 con 0x7f91f006c890 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91f5ce2640 10 monclient(hunting): authenticate will time out at 1437.054025s 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41148/0 (socket says 10.20.193.78:41148) 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 learned_addr learned my addr 10.20.193.78:0/1897809293 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request con 0x7f91f006c890 auth_method 0 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:51.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91f009c4a0 auth_method 0 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:51.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 10 monclient(hunting): get_auth_request con 0x7f91f0134df0 auth_method 0 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:51.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.500+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:51.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request con 0x7f91f006c890 auth_method 0 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91f009c4a0 auth_method 0 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 10 monclient(hunting): get_auth_request con 0x7f91f0134df0 auth_method 0 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:51.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:51.700+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:52.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:52.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request con 0x7f91f006c890 auth_method 0 2026-02-17T05:59:52.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91f009c4a0 auth_method 0 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:52.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 10 monclient(hunting): get_auth_request con 0x7f91f0134df0 auth_method 0 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:52.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:52.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.101+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request con 0x7f91f006c890 auth_method 0 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91f009c4a0 auth_method 0 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:52.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 10 monclient(hunting): get_auth_request con 0x7f91f0134df0 auth_method 0 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:52.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:52.902+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T05:59:54.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 10 monclient(hunting): tick 2026-02-17T05:59:54.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:59:54.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 monclient(hunting): continuing hunt 2026-02-17T05:59:54.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:59:54.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 msgr2=0x7f91f006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:54.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91f006c890 0x7f91f006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:54.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 msgr2=0x7f91f0135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:54.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91f0134df0 0x7f91f0135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:54.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 msgr2=0x7f91f006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:54.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91f009c4a0 0x7f91f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:54.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 10 monclient: _add_conns ranks=[0,1] 2026-02-17T05:59:54.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f91cc003770 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:54.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f91cc0074a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:54.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f009cc80 con 0x7f91cc003770 2026-02-17T05:59:54.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc00c1e0 con 0x7f91cc0074a0 2026-02-17T05:59:54.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.500+0000 7f91ed7fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:54.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.502+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> v1:10.20.193.78:6789/0 conn(0x7f91cc003770 legacy=0x7f91cc005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:54.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:54.502+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v1:10.20.193.115:6789/0 conn(0x7f91cc0074a0 legacy=0x7f91cc009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:55.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:55.088+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> v1:10.20.193.78:6789/0 conn(0x7f91cc003770 legacy=0x7f91cc005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2026-02-17T05:59:55.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:55.088+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> v1:10.20.193.78:6789/0 conn(0x7f91cc003770 legacy=0x7f91cc005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T05:59:55.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:55.088+0000 7f91ef7fe640 1 --1- 10.20.193.78:0/1897809293 >> v1:10.20.193.78:6789/0 conn(0x7f91cc003770 0x7f91cc005bd0 :-1 s=OPENED pgs=26 cs=1 l=1).handle_message read tag failed 2026-02-17T05:59:55.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:55.088+0000 7f91ef7fe640 1 --1- 10.20.193.78:0/1897809293 >> v1:10.20.193.78:6789/0 conn(0x7f91cc003770 0x7f91cc005bd0 :-1 s=OPENED pgs=26 cs=1 l=1).fault on lossy channel, failing 2026-02-17T05:59:55.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:55.088+0000 7f91ee7fc640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.78:6789/0 2026-02-17T05:59:57.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): tick 2026-02-17T05:59:57.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): _check_auth_tickets 2026-02-17T05:59:57.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 1 monclient(hunting): continuing hunt 2026-02-17T05:59:57.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 >> v1:10.20.193.115:6789/0 conn(0x7f91cc0074a0 legacy=0x7f91cc009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 >> v1:10.20.193.78:6789/0 conn(0x7f91cc003770 legacy=0x7f91cc005bd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 reap_dead start 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f91cc00c4e0 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 0x7f91cc00cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): picked mon.noname-a con 0x7f91cc00c960 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 0x7f91cc00b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): picked mon.noname-c con 0x7f91cc005ce0 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc00d440 con 0x7f91cc00c4e0 2026-02-17T05:59:57.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f91f006d6f0 con 0x7f91cc00c960 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 1 -- 10.20.193.78:0/1897809293 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f91f006daf0 con 0x7f91cc005ce0 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91ed7fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 0x7f91cc00cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91cc00c960 auth_method 0 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 msgr2=0x7f91cc00cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 msgr2=0x7f91cc00cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 0x7f91cc00cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 0x7f91cc00cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 0x7f91cc00b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 10 monclient(hunting): get_auth_request con 0x7f91cc005ce0 auth_method 0 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 msgr2=0x7f91cc00b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T05:59:57.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 msgr2=0x7f91cc00b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:57.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 0x7f91cc00b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:57.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.500+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 0x7f91cc00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:57.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.502+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> v1:10.20.193.78:6789/0 conn(0x7f91cc00c4e0 legacy=0x7f91cc009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:57.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 0x7f91cc00cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91cc00c960 auth_method 0 2026-02-17T05:59:57.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:57.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 msgr2=0x7f91cc00cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:57.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 msgr2=0x7f91cc00cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:57.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 0x7f91cc00cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.700+0000 7f91eeffd640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 0x7f91cc00cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 0x7f91cc00b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 10 monclient(hunting): get_auth_request con 0x7f91cc005ce0 auth_method 0 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 msgr2=0x7f91cc00b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 msgr2=0x7f91cc00b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 0x7f91cc00b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T05:59:57.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.701+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 0x7f91cc00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T05:59:57.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.728+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751139238 0 0) 0x7f91cc00d440 con 0x7f91cc00c4e0 2026-02-17T05:59:57.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): _negotiate 2026-02-17T05:59:57.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): my global_id is 174099 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91d4003620 con 0x7f91cc00c4e0 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4244553102 0 0) 0x7f91d4003620 con 0x7f91cc00c4e0 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 msgr2=0x7f91cc00b1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.115:3300/0 conn(0x7f91cc005ce0 0x7f91cc00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 msgr2=0x7f91cc00cdd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 --2- 10.20.193.78:0/1897809293 >> v2:10.20.193.78:3300/0 conn(0x7f91cc00c960 0x7f91cc00cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:59:57.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 monclient: found mon.noname-a-legacy 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc00b8c0 con 0x7f91cc00c4e0 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f91dc0097b0 con 0x7f91cc00c4e0 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: dump: 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.738 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 >> v1:10.20.193.78:6789/0 conn(0x7f91cc00c4e0 legacy=0x7f91cc009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:57.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91d4003ab0 0x7f91d4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): picked mon.a con 0x7f91d4003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91d4004900 0x7f91d4004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): picked mon.b con 0x7f91d4004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4005330 0x7f91d400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): picked mon.c con 0x7f91d4005330 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91f006d6f0 con 0x7f91d4003ab0 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f91f006daf0 con 0x7f91d4004900 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f91d400c310 con 0x7f91d4005330 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:57.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ee7fc640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91d4003ab0 0x7f91d4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request con 0x7f91d4003ab0 auth_method 0 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4005330 0x7f91d400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 10 monclient(hunting): get_auth_request con 0x7f91d4005330 auth_method 0 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:59:57.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_done global_id 174099 payload 293 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4005330 msgr2=0x7f91d400bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4005330 0x7f91d400bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91d4004900 msgr2=0x7f91d4004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91d4004900 0x7f91d4004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 1 monclient: found mon.a 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4003620 con 0x7f91d4003ab0 2026-02-17T05:59:57.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91f5ce2640 5 monclient: authenticate success, global_id 174099 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91f5ce2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.729+0000 7f91effff640 1 -- 10.20.193.78:0/1897809293 reap_dead start 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ef7fe640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91d4003ab0 msgr2=0x7f91d4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91d4003ab0 0x7f91d4003ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0099d0 tx=0x7f91dc030140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52484560c4a1073 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc036070 con 0x7f91d4003ab0 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 10 monclient: dump: 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.743 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91dc030b60 con 0x7f91d4003ab0 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 1 -- 10.20.193.78:0/1897809293 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0040e0 con 0x7f91d4003ab0 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91ee7fc640 10 monclient: dump: 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:57.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 4 set_mon_vals no callback set 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 10 monclient: shutdown 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 1 -- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91d4003ab0 msgr2=0x7f91d4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91d4003ab0 0x7f91d4003ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0099d0 tx=0x7f91dc030140 comp rx=0 tx=0).stop 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 1 -- 10.20.193.78:0/1897809293 shutdown_connections 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 1 --2- 10.20.193.78:0/1897809293 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91d4003ab0 0x7f91d4003ed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 1 -- 10.20.193.78:0/1897809293 >> 10.20.193.78:0/1897809293 conn(0x7f91f00a6620 msgr2=0x7f91f00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 1 -- 10.20.193.78:0/1897809293 shutdown_connections 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.732+0000 7f91f5ce2640 1 -- 10.20.193.78:0/1897809293 wait complete. 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient: build_initial_monmap 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient: monmap: 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:57.746 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 1 Processor -- start 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 1 -- start start 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient: init 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4003ab0 0x7f91f01cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient(hunting): picked mon.noname-c con 0x7f91d4003ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:57.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91f01cf890 0x7f91f0121a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient(hunting): picked mon.noname-a con 0x7f91f01cf890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:57.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91f01cfc90 0x7f91f0121fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient(hunting): picked mon.noname-b con 0x7f91f01cfc90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:57.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91f00a01a0 con 0x7f91f01cf890 2026-02-17T05:59:57.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f91f00a0020 con 0x7f91f01cfc90 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f91f009be20 con 0x7f91d4003ab0 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91f5ce2640 10 monclient(hunting): authenticate will time out at 1443.287631s 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4003ab0 0x7f91f01cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91eeffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91f01cf890 0x7f91f0121a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4003ab0 0x7f91f01cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36496/0 (socket says 10.20.193.78:36496) 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 -- 10.20.193.78:0/376332468 learned_addr learned my addr 10.20.193.78:0/376332468 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request con 0x7f91d4003ab0 auth_method 0 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91f01cf890 auth_method 0 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91eeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_done global_id 144374 payload 824 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 -- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91f01cfc90 msgr2=0x7f91f0121fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91f01cfc90 0x7f91f0121fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 -- 10.20.193.78:0/376332468 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91f01cf890 msgr2=0x7f91f0121a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91f01cf890 0x7f91f0121a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 monclient: found mon.noname-c 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:57.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.733+0000 7f91ef7fe640 1 -- 10.20.193.78:0/376332468 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f0125410 con 0x7f91d4003ab0 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.735+0000 7f91ef7fe640 1 -- 10.20.193.78:0/376332468 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4003ab0 msgr2=0x7f91f01cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.735+0000 7f91ef7fe640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4003ab0 0x7f91f01cf350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91dc004480 tx=0x7f91dc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 1 -- 10.20.193.78:0/376332468 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0040e0 con 0x7f91d4003ab0 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: dump: 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.750 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: _finish_auth 0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 1 -- 10.20.193.78:0/376332468 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91dc049070 con 0x7f91d4003ab0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 1 -- 10.20.193.78:0/376332468 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc002aa0 con 0x7f91d4003ab0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 10 monclient: dump: 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91edffb640 4 set_mon_vals no callback set 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 5 monclient: authenticate success, global_id 144374 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 10 monclient: _renew_subs 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f01256f0 con 0x7f91d4003ab0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 10 monclient: _renew_subs 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f00a12d0 con 0x7f91d4003ab0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 1 -- 10.20.193.78:0/376332468 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f91dc002c40 con 0x7f91d4003ab0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f006cc70 con 0x7f91d4003ab0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91c003df00 0x7f91c00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.736+0000 7f91cbfff640 1 -- 10.20.193.78:0/376332468 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(494..494 src has 335..494) ==== 4811+0+0 (secure 0 0 0) 0x7f91dc078680 con 0x7f91d4003ab0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.737+0000 7f91eeffd640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91c003df00 0x7f91c00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.737+0000 7f91eeffd640 10 monclient: get_auth_request con 0x7f91c003df00 auth_method 0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.739+0000 7f91cbfff640 1 -- 10.20.193.78:0/376332468 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f91dc04ddf0 con 0x7f91d4003ab0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.739+0000 7f91cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.739+0000 7f91cbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:57.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.739+0000 7f91eeffd640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91c003df00 0x7f91c00403c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f91e0002410 tx=0x7f91e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2314: done (7 left) 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2315: copy_from oid 27 from oid 28 current snap is 235 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: finishing write tid 2 to trial11545420-32 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2304: finishing copy_from to trial11545420-21 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 959 (ObjNum 802 snap 231 seq_num 802) dirty exists 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: finishing write tid 3 to trial11545420-32 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: finishing write tid 4 to trial11545420-32 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: finishing write tid 5 to trial11545420-32 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: oid 32 updating version 0 to 956 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: oid 32 version 956 is already newer than 953 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: oid 32 version 956 is already newer than 955 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: oid 32 version 956 is already newer than 954 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 956 (ObjNum 815 snap 235 seq_num 815) dirty exists 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2307: left oid 32 (ObjNum 815 snap 235 seq_num 815) 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2308: finishing write tid 1 to trial11545420-39 2026-02-17T05:59:57.781 INFO:tasks.rados.rados.0.trial115.stdout:2308: finishing write tid 2 to trial11545420-39 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: finishing write tid 3 to trial11545420-39 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: finishing write tid 4 to trial11545420-39 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: finishing write tid 5 to trial11545420-39 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: oid 39 updating version 0 to 1306 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: oid 39 updating version 1306 to 1309 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: oid 39 version 1309 is already newer than 1308 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: oid 39 version 1309 is already newer than 1307 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1309 (ObjNum 816 snap 235 seq_num 816) dirty exists 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: left oid 39 (ObjNum 816 snap 235 seq_num 816) 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: finishing write tid 1 to trial11545420-45 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: finishing write tid 2 to trial11545420-45 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: finishing write tid 3 to trial11545420-45 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: finishing write tid 4 to trial11545420-45 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: finishing write tid 5 to trial11545420-45 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: oid 45 updating version 0 to 1313 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: oid 45 version 1313 is already newer than 1312 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: oid 45 updating version 1313 to 1314 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: oid 45 version 1314 is already newer than 1311 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1314 (ObjNum 817 snap 235 seq_num 817) dirty exists 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: left oid 45 (ObjNum 817 snap 235 seq_num 817) 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: finishing write tid 1 to trial11545420-18 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2304: done (7 left) 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2307: done (6 left) 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2308: done (5 left) 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2309: done (4 left) 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2316: delete oid 26 current snap is 235 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: finishing write tid 2 to trial11545420-18 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: finishing write tid 3 to trial11545420-18 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: finishing write tid 4 to trial11545420-18 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: finishing write tid 5 to trial11545420-18 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: finishing write tid 6 to trial11545420-18 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: oid 18 updating version 0 to 1066 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: oid 18 version 1066 is already newer than 1065 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: oid 18 version 1066 is already newer than 1064 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: oid 18 updating version 1066 to 1067 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: oid 18 updating version 1067 to 1068 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1068 (ObjNum 818 snap 235 seq_num 818) dirty exists 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2310: left oid 18 (ObjNum 818 snap 235 seq_num 818) 2026-02-17T05:59:57.782 INFO:tasks.rados.rados.0.trial115.stdout:2311: expect (ObjNum 796 snap 229 seq_num 796) 2026-02-17T05:59:57.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.854+0000 7f91f5ce2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:57.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.854+0000 7f91f5ce2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:57.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.854+0000 7f91f5ce2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:57.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.854+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f91f0123fc0 con 0x7f91d4003ab0 2026-02-17T05:59:57.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.855+0000 7f91cbfff640 1 -- 10.20.193.78:0/376332468 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f91dc077060 con 0x7f91d4003ab0 2026-02-17T05:59:57.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.855+0000 7f91cbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:57.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.855+0000 7f91cbfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:57.857 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":164,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:57.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91c003df00 msgr2=0x7f91c00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91c003df00 0x7f91c00403c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f91e0002410 tx=0x7f91e0008040 comp rx=0 tx=0).stop 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 10 monclient: shutdown 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4003ab0 msgr2=0x7f91f01cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4003ab0 0x7f91f01cf350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91dc004480 tx=0x7f91dc0047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 shutdown_connections 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91c003df00 0x7f91c00403c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91f01cfc90 0x7f91f0121fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f91f01cf890 0x7f91f0121a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 --2- 10.20.193.78:0/376332468 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91d4003ab0 0x7f91f01cf350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 >> 10.20.193.78:0/376332468 conn(0x7f91f00a6620 msgr2=0x7f91f00a25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:57.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 shutdown_connections 2026-02-17T05:59:57.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.856+0000 7f91f5ce2640 1 -- 10.20.193.78:0/376332468 wait complete. 2026-02-17T05:59:57.866 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:59:57.867 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:59:57.867 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.935+0000 7fd63a972640 10 monclient: get_monmap_and_config 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.935+0000 7fd63a972640 10 monclient: build_initial_monmap 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.935+0000 7fd63a972640 10 monclient: monmap: 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:57.936619+0000 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:57.936619+0000 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:57.936 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.935+0000 7fd63a972640 1 Processor -- start 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.935+0000 7fd63a972640 1 -- start start 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.935+0000 7fd63a972640 10 monclient: init 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd63406c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd63409c4a0 0x7fd63413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient(hunting): picked mon.noname-c con 0x7fd63409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd63406d380 0x7fd63413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient(hunting): picked mon.noname-a con 0x7fd63406d380 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63406daf0 con 0x7fd63406c890 2026-02-17T05:59:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd6340a0880 con 0x7fd63406d380 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd63409cc80 con 0x7fd63409c4a0 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd63a972640 10 monclient(hunting): authenticate will time out at 1443.490145s 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd63406d380 0x7fd63413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd63406d380 0x7fd63413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47218/0 (socket says 10.20.193.78:47218) 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 1 -- 10.20.193.78:0/1821163493 learned_addr learned my addr 10.20.193.78:0/1821163493 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient(hunting): get_auth_request con 0x7fd63406d380 auth_method 0 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd6337fe640 1 --2- 10.20.193.78:0/1821163493 >> v2:10.20.193.115:3300/0 conn(0x7fd63409c4a0 0x7fd63413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd6337fe640 10 monclient(hunting): get_auth_request con 0x7fd63409c4a0 auth_method 0 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd6337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd6337fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd6337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient(hunting): handle_auth_done global_id 174105 payload 274 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 1 -- 10.20.193.78:0/1821163493 >> v2:10.20.193.115:3300/0 conn(0x7fd63409c4a0 msgr2=0x7fd63413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 1 --2- 10.20.193.78:0/1821163493 >> v2:10.20.193.115:3300/0 conn(0x7fd63409c4a0 0x7fd63413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 1 -- 10.20.193.78:0/1821163493 >> v1:10.20.193.115:6789/0 conn(0x7fd63406c890 legacy=0x7fd63406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient: _finish_hunting 0 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 1 monclient: found mon.noname-a 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:57.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.936+0000 7fd638ee8640 1 -- 10.20.193.78:0/1821163493 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63409cf60 con 0x7fd63406d380 2026-02-17T05:59:57.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd638ee8640 1 -- 10.20.193.78:0/1821163493 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406d380 msgr2=0x7fd63413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:57.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd638ee8640 1 --2- 10.20.193.78:0/1821163493 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406d380 0x7fd63413d8f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd63406dcc0 tx=0x7fd62802f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241ee4463fbe7695 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 1 -- 10.20.193.78:0/1821163493 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62802fe50 con 0x7fd63406d380 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: dump: 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: _finish_auth 0 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 5 monclient: authenticate success, global_id 174105 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 1 -- 10.20.193.78:0/1821163493 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6280373f0 con 0x7fd63406d380 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 1 -- 10.20.193.78:0/1821163493 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6280377c0 con 0x7fd63406d380 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 4 set_mon_vals no callback set 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd632ffd640 10 monclient: dump: 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 10 monclient: shutdown 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 1 -- 10.20.193.78:0/1821163493 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406d380 msgr2=0x7fd63413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 1 --2- 10.20.193.78:0/1821163493 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406d380 0x7fd63413d8f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd63406dcc0 tx=0x7fd62802f1e0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 1 -- 10.20.193.78:0/1821163493 shutdown_connections 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 1 --2- 10.20.193.78:0/1821163493 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406d380 0x7fd63413d8f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 1 --2- 10.20.193.78:0/1821163493 >> v2:10.20.193.115:3300/0 conn(0x7fd63409c4a0 0x7fd63413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 1 -- 10.20.193.78:0/1821163493 >> 10.20.193.78:0/1821163493 conn(0x7fd6340a6620 msgr2=0x7fd6340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 1 -- 10.20.193.78:0/1821163493 shutdown_connections 2026-02-17T05:59:57.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 1 -- 10.20.193.78:0/1821163493 wait complete. 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 10 monclient: build_initial_monmap 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.939+0000 7fd63a972640 10 monclient: monmap: 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 1 Processor -- start 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 1 -- start start 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient: init 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:57.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406c890 0x7fd63413ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient(hunting): picked mon.noname-a con 0x7fd63406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63406d380 0x7fd63409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient(hunting): picked mon.noname-b con 0x7fd63406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63409c4a0 0x7fd63409e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient(hunting): picked mon.noname-c con 0x7fd63409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd63409bfa0 con 0x7fd63406c890 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd63409be20 con 0x7fd63406d380 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd63413de30 con 0x7fd63409c4a0 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd63a972640 10 monclient(hunting): authenticate will time out at 1443.494316s 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406c890 0x7fd63413ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406c890 0x7fd63413ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47222/0 (socket says 10.20.193.78:47222) 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 -- 10.20.193.78:0/3503149645 learned_addr learned my addr 10.20.193.78:0/3503149645 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient(hunting): get_auth_request con 0x7fd63406c890 auth_method 0 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd638ee8640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63409c4a0 0x7fd63409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd638ee8640 10 monclient(hunting): get_auth_request con 0x7fd63409c4a0 auth_method 0 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd638ee8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd638ee8640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd638ee8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd6337fe640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63406d380 0x7fd63409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd6337fe640 10 monclient(hunting): get_auth_request con 0x7fd63406d380 auth_method 0 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd6337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd6337fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd6337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd638ee8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd638ee8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd638ee8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient(hunting): handle_auth_done global_id 174108 payload 824 2026-02-17T05:59:57.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63409c4a0 msgr2=0x7fd63409e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd63409c4a0 0x7fd63409e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63406d380 msgr2=0x7fd63409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd63406d380 0x7fd63409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 monclient: found mon.noname-a 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.940+0000 7fd633fff640 1 -- 10.20.193.78:0/3503149645 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63409f7a0 con 0x7fd63406c890 2026-02-17T05:59:57.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd633fff640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406c890 msgr2=0x7fd63413ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd633fff640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406c890 0x7fd63413ca50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd62000c600 tx=0x7fd62000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620012a40 con 0x7fd63406c890 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: dump: 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: _finish_auth 0 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd620017070 con 0x7fd63406c890 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd63a972640 5 monclient: authenticate success, global_id 174108 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd63a972640 10 monclient: _renew_subs 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd63a972640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63413afd0 con 0x7fd63406c890 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620012e20 con 0x7fd63406c890 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd630ff9640 10 monclient: dump: 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.945 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd6327fc640 4 set_mon_vals no callback set 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd63a972640 10 monclient: _renew_subs 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd63a972640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.942+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63409d810 con 0x7fd63406c890 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd63a972640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd63a972640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63409e5d0 0x7fd6340716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd63a972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd63409e9b0 con 0x7fd63409e5d0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63409e5d0 0x7fd6340716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 10 monclient: get_auth_request con 0x7fd63409e5d0 auth_method 0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 10 monclient(hunting): handle_auth_done global_id 174111 payload 824 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd6337fe640 1 -- 10.20.193.78:0/3503149645 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd634074980 con 0x7fd63409e5d0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd62001c460 con 0x7fd63406c890 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd630ff9640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(495..495 src has 335..495) ==== 4811+0+0 (secure 0 0 0) 0x7fd620052690 con 0x7fd63406c890 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd638ee8640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.943+0000 7fd638ee8640 10 monclient: get_auth_request con 0x7fd60403deb0 auth_method 0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd6337fe640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63409e5d0 msgr2=0x7fd6340716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd6337fe640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63409e5d0 0x7fd6340716f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd63409f1c0 tx=0x7fd624010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62401a030 con 0x7fd63409e5d0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:57.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 10 monclient: dump: 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd62400eb50 con 0x7fd63409e5d0 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63409e5d0 msgr2=0x7fd6340716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63409e5d0 0x7fd6340716f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd63409f1c0 tx=0x7fd624010ca0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63409e5d0 msgr2=0x7fd6340716f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.945+0000 7fd630ff9640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63409e5d0 0x7fd6340716f0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd63409f1c0 tx=0x7fd624010ca0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.946+0000 7fd638ee8640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd63406a480 tx=0x7fd628033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd63a972640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd63a972640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd634071c30 0x7fd634072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd63a972640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6340744d0 con 0x7fd634071c30 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd634071c30 0x7fd634072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 10 monclient: get_auth_request con 0x7fd634071c30 auth_method 0 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 10 monclient(hunting): handle_auth_done global_id 174114 payload 824 2026-02-17T05:59:57.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.949+0000 7fd6337fe640 1 -- 10.20.193.78:0/3503149645 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd63413a7b0 con 0x7fd634071c30 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd6337fe640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd634071c30 msgr2=0x7fd634072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd6337fe640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd634071c30 0x7fd634072040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd63409f1c0 tx=0x7fd6240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62400f040 con 0x7fd634071c30 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 10 monclient: dump: 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:57.952 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7fd624020230 con 0x7fd634071c30 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd634071c30 msgr2=0x7fd634072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd634071c30 0x7fd634072040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd63409f1c0 tx=0x7fd6240047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd634071c30 msgr2=0x7fd634072040 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd630ff9640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd634071c30 0x7fd634072040 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd63409f1c0 tx=0x7fd6240047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 164, 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 12867, 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:57.953 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:57.954 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:57.955 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:57.956 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:57.957 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 msgr2=0x7fd604040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd63a972640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd60403deb0 0x7fd604040370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd63406a480 tx=0x7fd628033000 comp rx=0 tx=0).stop 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd63a972640 10 monclient: shutdown 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd63a972640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406c890 msgr2=0x7fd63413ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd63a972640 1 --2- 10.20.193.78:0/3503149645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd63406c890 0x7fd63413ca50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd62000c600 tx=0x7fd62000cad0 comp rx=0 tx=0).stop 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.951+0000 7fd638ee8640 1 -- 10.20.193.78:0/3503149645 reap_dead start 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.952+0000 7fd638ee8640 1 -- 10.20.193.78:0/3503149645 reap_dead start 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.952+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 shutdown_connections 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.952+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 >> 10.20.193.78:0/3503149645 conn(0x7fd6340a6620 msgr2=0x7fd634136190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.952+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 shutdown_connections 2026-02-17T05:59:57.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:57.952+0000 7fd63a972640 1 -- 10.20.193.78:0/3503149645 wait complete. 2026-02-17T05:59:57.959 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:57.959 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:59:57.959 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:59:57.980 INFO:tasks.rados.rados.0.trial115.stdout:2312: finishing write tid 1 to trial11545420-46 2026-02-17T05:59:57.980 INFO:tasks.rados.rados.0.trial115.stdout:2312: finishing write tid 2 to trial11545420-46 2026-02-17T05:59:57.980 INFO:tasks.rados.rados.0.trial115.stdout:2310: done (4 left) 2026-02-17T05:59:57.980 INFO:tasks.rados.rados.0.trial115.stdout:2311: done (3 left) 2026-02-17T05:59:57.980 INFO:tasks.rados.rados.0.trial115.stdout:2316: done (2 left) 2026-02-17T05:59:57.980 INFO:tasks.rados.rados.0.trial115.stdout:2317: write oid 35 current snap is 235 2026-02-17T05:59:57.981 INFO:tasks.rados.rados.0.trial115.stdout:2312: finishing write tid 3 to trial11545420-46 2026-02-17T05:59:57.981 INFO:tasks.rados.rados.0.trial115.stdout:2317: seq_num 821 ranges {598547=629200,1876540=491946,3046277=681499} 2026-02-17T05:59:57.985 INFO:tasks.rados.rados.0.trial115.stdout:2317: writing trial11545420-35 from 598547 to 1227747 tid 1 2026-02-17T05:59:57.989 INFO:tasks.rados.rados.0.trial115.stdout:2317: writing trial11545420-35 from 1876540 to 2368486 tid 2 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2317: writing trial11545420-35 from 3046277 to 3727776 tid 3 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2318: write (excl) oid 29 current snap is 235 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2312: finishing write tid 4 to trial11545420-46 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2312: finishing write tid 5 to trial11545420-46 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2312: oid 46 updating version 0 to 1185 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2312: oid 46 updating version 1185 to 1186 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2312: oid 46 updating version 1186 to 1188 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2312: oid 46 version 1188 is already newer than 1187 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1188 (ObjNum 819 snap 235 seq_num 819) dirty exists 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2312: left oid 46 (ObjNum 819 snap 235 seq_num 819) 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2315: finishing copy_from racing read to trial11545420-27 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2315: finishing copy_from to trial11545420-27 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1148 (ObjNum 747 snap 204 seq_num 747) dirty exists 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2317: finishing write tid 1 to trial11545420-35 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2317: finishing write tid 2 to trial11545420-35 2026-02-17T05:59:57.995 INFO:tasks.rados.rados.0.trial115.stdout:2318: seq_num 822 ranges {615354=420802,1651375=751957,3190053=492771} 2026-02-17T05:59:57.999 INFO:tasks.rados.rados.0.trial115.stdout:2318: writing trial11545420-29 from 615354 to 1036156 tid 1 2026-02-17T05:59:58.004 INFO:tasks.rados.rados.0.trial115.stdout:2318: writing trial11545420-29 from 1651375 to 2403332 tid 2 2026-02-17T05:59:58.009 INFO:tasks.rados.rados.0.trial115.stdout:2318: writing trial11545420-29 from 3190053 to 3682824 tid 3 2026-02-17T05:59:58.009 INFO:tasks.rados.rados.0.trial115.stdout:2312: done (3 left) 2026-02-17T05:59:58.009 INFO:tasks.rados.rados.0.trial115.stdout:2315: done (2 left) 2026-02-17T05:59:58.009 INFO:tasks.rados.rados.0.trial115.stdout:2319: snap_create 2026-02-17T05:59:58.009 INFO:tasks.rados.rados.0.trial115.stdout:2317: finishing write tid 3 to trial11545420-35 2026-02-17T05:59:58.009 INFO:tasks.rados.rados.0.trial115.stdout:2317: finishing write tid 4 to trial11545420-35 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:2317: finishing write tid 5 to trial11545420-35 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:2317: oid 35 updating version 0 to 1292 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:2317: oid 35 updating version 1292 to 1293 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:2317: oid 35 updating version 1293 to 1294 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:2317: oid 35 version 1294 is already newer than 1291 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1294 (ObjNum 821 snap 235 seq_num 821) dirty exists 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:2317: left oid 35 (ObjNum 821 snap 235 seq_num 821) 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:2318: finishing write tid 1 to trial11545420-29 2026-02-17T05:59:58.010 INFO:tasks.rados.rados.0.trial115.stdout:2318: finishing write tid 2 to trial11545420-29 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:2318: finishing write tid 3 to trial11545420-29 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:2318: finishing write tid 4 to trial11545420-29 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:2318: finishing write tid 5 to trial11545420-29 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:2318: oid 29 updating version 0 to 654 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:2318: oid 29 version 654 is already newer than 653 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:2318: oid 29 version 654 is already newer than 651 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:2318: oid 29 version 654 is already newer than 652 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 654 (ObjNum 822 snap 235 seq_num 822) dirty exists 2026-02-17T05:59:58.011 INFO:tasks.rados.rados.0.trial115.stdout:2318: left oid 29 (ObjNum 822 snap 235 seq_num 822) 2026-02-17T05:59:58.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient: get_monmap_and_config 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient: build_initial_monmap 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient: monmap: 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:58.026279+0000 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:58.026279+0000 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:58.026 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 1 Processor -- start 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 1 -- start start 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient: init 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf0406c890 0x7faf0406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient(hunting): picked mon.noname-a con 0x7faf0406c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7faf0409c4a0 0x7faf0406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient(hunting): picked mon.noname-c con 0x7faf0409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 0x7faf04135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient(hunting): picked mon.noname-b con 0x7faf04134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7faf040a0880 con 0x7faf0406c890 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7faf0406d6f0 con 0x7faf0409c4a0 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf0406daf0 con 0x7faf04134df0 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf08df7640 10 monclient(hunting): authenticate will time out at 1443.579796s 2026-02-17T05:59:58.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf0406c890 0x7faf0406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 0x7faf04135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf0406c890 0x7faf0406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47254/0 (socket says 10.20.193.78:47254) 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 1 -- 10.20.193.78:0/2787886871 learned_addr learned my addr 10.20.193.78:0/2787886871 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 10 monclient(hunting): get_auth_request con 0x7faf0406c890 auth_method 0 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02d76640 10 monclient(hunting): get_auth_request con 0x7faf04134df0 auth_method 0 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.025+0000 7faf02575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf01d74640 1 --2- 10.20.193.78:0/2787886871 >> v2:10.20.193.115:3300/0 conn(0x7faf0409c4a0 0x7faf0406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf01d74640 10 monclient(hunting): get_auth_request con 0x7faf0409c4a0 auth_method 0 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf01d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf01d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf01d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 10 monclient(hunting): handle_auth_done global_id 174117 payload 274 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 1 -- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 msgr2=0x7faf04135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 1 --2- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 0x7faf04135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 1 -- 10.20.193.78:0/2787886871 >> v2:10.20.193.115:3300/0 conn(0x7faf0409c4a0 msgr2=0x7faf0406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 1 --2- 10.20.193.78:0/2787886871 >> v2:10.20.193.115:3300/0 conn(0x7faf0409c4a0 0x7faf0406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 10 monclient: _finish_hunting 0 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 1 monclient: found mon.noname-a 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:58.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02575640 1 -- 10.20.193.78:0/2787886871 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0409cc80 con 0x7faf0406c890 2026-02-17T05:59:58.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.026+0000 7faf02d76640 1 --2- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 0x7faf04135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:59:58.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf02575640 1 -- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 msgr2=0x7faf0406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf02575640 1 --2- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf0406cc70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf0409c980 tx=0x7faef802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4feab3b43776eb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 1 -- 10.20.193.78:0/2787886871 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef803c070 con 0x7faf0406c890 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: dump: 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: _finish_auth 0 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: _check_auth_tickets 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 1 -- 10.20.193.78:0/2787886871 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef802fca0 con 0x7faf0406c890 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf08df7640 5 monclient: authenticate success, global_id 174117 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf08df7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 1 -- 10.20.193.78:0/2787886871 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8037660 con 0x7faf0406c890 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf08df7640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf08df7640 4 set_mon_vals no callback set 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.028+0000 7faf01573640 10 monclient: dump: 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.030 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient: shutdown 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 msgr2=0x7faf0406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 --2- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf0406cc70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf0409c980 tx=0x7faef802ee60 comp rx=0 tx=0).stop 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- 10.20.193.78:0/2787886871 shutdown_connections 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 --2- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 0x7faf04135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 --2- 10.20.193.78:0/2787886871 >> v2:10.20.193.115:3300/0 conn(0x7faf0409c4a0 0x7faf0406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 --2- 10.20.193.78:0/2787886871 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf0406cc70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- 10.20.193.78:0/2787886871 >> 10.20.193.78:0/2787886871 conn(0x7faf040a6620 msgr2=0x7faf040a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- 10.20.193.78:0/2787886871 shutdown_connections 2026-02-17T05:59:58.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- 10.20.193.78:0/2787886871 wait complete. 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient: build_initial_monmap 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient: monmap: 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 Processor -- start 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- start start 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient: init 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf041cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient(hunting): picked mon.noname-a con 0x7faf0406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:58.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0409c4a0 0x7faf041cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient(hunting): picked mon.noname-b con 0x7faf0409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 0x7faf0409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient(hunting): picked mon.noname-c con 0x7faf04134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf040a01a0 con 0x7faf0406c890 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf040a0020 con 0x7faf0409c4a0 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf0409be20 con 0x7faf04134df0 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.029+0000 7faf08df7640 10 monclient(hunting): authenticate will time out at 1443.583997s 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf041cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf041cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47264/0 (socket says 10.20.193.78:47264) 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 -- 10.20.193.78:0/1871802815 learned_addr learned my addr 10.20.193.78:0/1871802815 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient(hunting): get_auth_request con 0x7faf0406c890 auth_method 0 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02d76640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 0x7faf0409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02d76640 10 monclient(hunting): get_auth_request con 0x7faf04134df0 auth_method 0 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf01d74640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0409c4a0 0x7faf041cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient(hunting): handle_auth_done global_id 174120 payload 824 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 msgr2=0x7faf0409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf04134df0 0x7faf0409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0409c4a0 msgr2=0x7faf041cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf0409c4a0 0x7faf041cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient: _finish_hunting 0 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 monclient: found mon.noname-a 2026-02-17T05:59:58.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:58.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:58.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.030+0000 7faf02575640 1 -- 10.20.193.78:0/1871802815 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf04127730 con 0x7faf0406c890 2026-02-17T05:59:58.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf02575640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 msgr2=0x7faf041cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf02575640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf041cf350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faef8002410 tx=0x7faef8004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8047070 con 0x7faf0406c890 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: dump: 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: _finish_auth 0 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef802fe50 con 0x7faf0406c890 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf08df7640 5 monclient: authenticate success, global_id 174120 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf08df7640 10 monclient: _renew_subs 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf08df7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf040a1690 con 0x7faf0406c890 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef803c040 con 0x7faf0406c890 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf00d72640 4 set_mon_vals no callback set 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faeeaffd640 10 monclient: dump: 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.035 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf08df7640 10 monclient: _renew_subs 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf08df7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.032+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf04127450 con 0x7faf0406c890 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf08df7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf08df7640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409e850 0x7faf0411cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf08df7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf040726d0 con 0x7faf0409e850 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7faef8037ce0 con 0x7faf0406c890 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409e850 0x7faf0411cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 10 monclient: get_auth_request con 0x7faf0409e850 auth_method 0 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 10 monclient(hunting): handle_auth_done global_id 144386 payload 824 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf01d74640 1 -- 10.20.193.78:0/1871802815 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf04074980 con 0x7faf0409e850 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faeeaffd640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faed003df00 0x7faed00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(495..495 src has 335..495) ==== 4811+0+0 (secure 0 0 0) 0x7faef80771c0 con 0x7faf0406c890 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf02d76640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faed003df00 0x7faed00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.033+0000 7faf02d76640 10 monclient: get_auth_request con 0x7faed003df00 auth_method 0 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faf01d74640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409e850 msgr2=0x7faf0411cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faf01d74640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409e850 0x7faf0411cc70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faeec00f4a0 tx=0x7faeec00f970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec018070 con 0x7faf0409e850 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 10 monclient: dump: 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7faeec014b80 con 0x7faf0409e850 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409e850 msgr2=0x7faf0411cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409e850 0x7faf0411cc70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faeec00f4a0 tx=0x7faeec00f970 comp rx=0 tx=0).stop 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409e850 msgr2=0x7faf0411cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.035+0000 7faeeaffd640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409e850 0x7faf0411cc70 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faeec00f4a0 tx=0x7faeec00f970 comp rx=0 tx=0).stop 2026-02-17T05:59:58.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.036+0000 7faf02d76640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faed003df00 0x7faed00403c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faef4011420 tx=0x7faef400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf08df7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:58.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf08df7640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409ec30 0x7faf041da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf08df7640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf04127000 con 0x7faf0409ec30 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf01d74640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409ec30 0x7faf041da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf01d74640 10 monclient: get_auth_request con 0x7faf0409ec30 auth_method 0 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf01d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf01d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.038+0000 7faf01d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.039+0000 7faf01d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.039+0000 7faf01d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.039+0000 7faf01d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.039+0000 7faf01d74640 10 monclient(hunting): handle_auth_done global_id 144389 payload 824 2026-02-17T05:59:58.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.039+0000 7faf01d74640 1 -- 10.20.193.78:0/1871802815 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faf04135280 con 0x7faf0409ec30 2026-02-17T05:59:58.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf01d74640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409ec30 msgr2=0x7faf041da5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf01d74640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409ec30 0x7faf041da5d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faeec00f4a0 tx=0x7faeec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec003d30 con 0x7faf0409ec30 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 10 monclient: dump: 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7faeec003d30 con 0x7faf0409ec30 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409ec30 msgr2=0x7faf041da5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409ec30 0x7faf041da5d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faeec00f4a0 tx=0x7faeec0047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409ec30 msgr2=0x7faf041da5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faeeaffd640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf0409ec30 0x7faf041da5d0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faeec00f4a0 tx=0x7faeec0047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.042 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 164, 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 181576, 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:58.043 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.044 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.045 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.046 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faed003df00 msgr2=0x7faed00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faed003df00 0x7faed00403c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faef4011420 tx=0x7faef400b040 comp rx=0 tx=0).stop 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 10 monclient: shutdown 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 msgr2=0x7faf041cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf041cf350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faef8002410 tx=0x7faef8004830 comp rx=0 tx=0).stop 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf02d76640 1 -- 10.20.193.78:0/1871802815 reap_dead start 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 shutdown_connections 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 --2- 10.20.193.78:0/1871802815 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf0406c890 0x7faf041cf350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 >> 10.20.193.78:0/1871802815 conn(0x7faf040a6620 msgr2=0x7faf04072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:58.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 shutdown_connections 2026-02-17T05:59:58.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.041+0000 7faf08df7640 1 -- 10.20.193.78:0/1871802815 wait complete. 2026-02-17T05:59:58.049 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:58.049 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:59:58.049 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:59:58.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.116+0000 7fa7d7551640 10 monclient: get_monmap_and_config 2026-02-17T05:59:58.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.116+0000 7fa7d7551640 10 monclient: build_initial_monmap 2026-02-17T05:59:58.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.116+0000 7fa7d7551640 10 monclient: monmap: 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:58.117901+0000 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:58.117901+0000 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:58.118 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 1 Processor -- start 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 1 -- start start 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient: init 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa7d006c890 0x7fa7d006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient(hunting): picked mon.noname-c con 0x7fa7d006c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa7d009c4a0 0x7fa7d006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient(hunting): picked mon.noname-a con 0x7fa7d009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d0135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient(hunting): picked mon.noname-b con 0x7fa7d0134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa7d00a0880 con 0x7fa7d009c4a0 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa7d006d6f0 con 0x7fa7d006c890 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d006daf0 con 0x7fa7d0134df0 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d7551640 10 monclient(hunting): authenticate will time out at 1443.671423s 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d0135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa7d009c4a0 0x7fa7d006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d0135280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36564/0 (socket says 10.20.193.78:36564) 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/2783639694 learned_addr learned my addr 10.20.193.78:0/2783639694 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient(hunting): get_auth_request con 0x7fa7d0134df0 auth_method 0 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 10 monclient(hunting): get_auth_request con 0x7fa7d009c4a0 auth_method 0 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient(hunting): handle_auth_done global_id 144392 payload 274 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/2783639694 >> v2:10.20.193.115:3300/0 conn(0x7fa7d006c890 msgr2=0x7fa7d006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 --2- 10.20.193.78:0/2783639694 >> v2:10.20.193.115:3300/0 conn(0x7fa7d006c890 0x7fa7d006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/2783639694 >> v2:10.20.193.78:3300/0 conn(0x7fa7d009c4a0 msgr2=0x7fa7d006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 --2- 10.20.193.78:0/2783639694 >> v2:10.20.193.78:3300/0 conn(0x7fa7d009c4a0 0x7fa7d006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient: _finish_hunting 0 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 monclient: found mon.noname-b 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/2783639694 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d009cc80 con 0x7fa7d0134df0 2026-02-17T05:59:58.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.117+0000 7fa7d4ac5640 1 --2- 10.20.193.78:0/2783639694 >> v2:10.20.193.78:3300/0 conn(0x7fa7d009c4a0 0x7fa7d006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:59:58.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/2783639694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 msgr2=0x7fa7d0135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d5ac7640 1 --2- 10.20.193.78:0/2783639694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d0135280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7d006a480 tx=0x7fa7cc02f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220a73082fe2fe87 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 1 -- 10.20.193.78:0/2783639694 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc02fe50 con 0x7fa7d0134df0 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: dump: 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: _finish_auth 0 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 1 -- 10.20.193.78:0/2783639694 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7cc0373f0 con 0x7fa7d0134df0 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 5 monclient: authenticate success, global_id 144392 2026-02-17T05:59:58.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 4 set_mon_vals no callback set 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 1 -- 10.20.193.78:0/2783639694 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0377c0 con 0x7fa7d0134df0 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7bffff640 10 monclient: dump: 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 10 monclient: shutdown 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:58.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 -- 10.20.193.78:0/2783639694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 msgr2=0x7fa7d0135280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 --2- 10.20.193.78:0/2783639694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d0135280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7d006a480 tx=0x7fa7cc02f1e0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 -- 10.20.193.78:0/2783639694 shutdown_connections 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 --2- 10.20.193.78:0/2783639694 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d0135280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 --2- 10.20.193.78:0/2783639694 >> v2:10.20.193.78:3300/0 conn(0x7fa7d009c4a0 0x7fa7d006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 --2- 10.20.193.78:0/2783639694 >> v2:10.20.193.115:3300/0 conn(0x7fa7d006c890 0x7fa7d006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 -- 10.20.193.78:0/2783639694 >> 10.20.193.78:0/2783639694 conn(0x7fa7d00a6620 msgr2=0x7fa7d00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 -- 10.20.193.78:0/2783639694 shutdown_connections 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.120+0000 7fa7d7551640 1 -- 10.20.193.78:0/2783639694 wait complete. 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient: build_initial_monmap 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient: monmap: 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 1 Processor -- start 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 1 -- start start 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient: init 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7d006c890 0x7fa7d01cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient(hunting): picked mon.noname-a con 0x7fa7d006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009c4a0 0x7fa7d01cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient(hunting): picked mon.noname-b con 0x7fa7d009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient(hunting): picked mon.noname-c con 0x7fa7d0134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d00a01a0 con 0x7fa7d006c890 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d00a0020 con 0x7fa7d009c4a0 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d009be20 con 0x7fa7d0134df0 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d7551640 10 monclient(hunting): authenticate will time out at 1443.675735s 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7d006c890 0x7fa7d01cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7d006c890 0x7fa7d01cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47278/0 (socket says 10.20.193.78:47278) 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 1 -- 10.20.193.78:0/567998168 learned_addr learned my addr 10.20.193.78:0/567998168 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 10 monclient(hunting): get_auth_request con 0x7fa7d006c890 auth_method 0 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient(hunting): get_auth_request con 0x7fa7d0134df0 auth_method 0 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient(hunting): handle_auth_done global_id 144395 payload 824 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009c4a0 msgr2=0x7fa7d01cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009c4a0 0x7fa7d01cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7d006c890 msgr2=0x7fa7d01cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7d006c890 0x7fa7d01cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient: _finish_hunting 0 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 1 monclient: found mon.noname-c 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.121+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/567998168 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d01273f0 con 0x7fa7d0134df0 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 msgr2=0x7fa7d009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d5ac7640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d009db50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc035000 tx=0x7fa7cc004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc046030 con 0x7fa7d0134df0 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: dump: 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.132 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7cc041450 con 0x7fa7d0134df0 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc041640 con 0x7fa7d0134df0 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 5 monclient: authenticate success, global_id 144395 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bf7fe640 4 set_mon_vals no callback set 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 10 monclient: _renew_subs 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d00725f0 con 0x7fa7d0134df0 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 10 monclient: dump: 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 10 monclient: _renew_subs 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d0127110 con 0x7fa7d0134df0 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e090 0x7fa7d009e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d009f880 con 0x7fa7d009e090 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d52c6640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e090 0x7fa7d009e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d52c6640 10 monclient: get_auth_request con 0x7fa7d009e090 auth_method 0 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d52c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d52c6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d52c6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa7cc041860 con 0x7fa7d0134df0 2026-02-17T05:59:58.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.124+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.125+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_done global_id 134398 payload 824 2026-02-17T05:59:58.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.125+0000 7fa7d52c6640 1 -- 10.20.193.78:0/567998168 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7d0074980 con 0x7fa7d009e090 2026-02-17T05:59:58.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.125+0000 7fa7bdffb640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7a003dee0 0x7fa7a00403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.125+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(495..495 src has 335..495) ==== 4811+0+0 (secure 0 0 0) 0x7fa7cc077420 con 0x7fa7d0134df0 2026-02-17T05:59:58.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.125+0000 7fa7d4ac5640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7a003dee0 0x7fa7a00403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.125+0000 7fa7d4ac5640 10 monclient: get_auth_request con 0x7fa7a003dee0 auth_method 0 2026-02-17T05:59:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7d52c6640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e090 msgr2=0x7fa7d009e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7d52c6640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e090 0x7fa7d009e470 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009710 tx=0x7fa7c0009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0016070 con 0x7fa7d009e090 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 10 monclient: dump: 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa7c0009e00 con 0x7fa7d009e090 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e090 msgr2=0x7fa7d009e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e090 0x7fa7d009e470 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009710 tx=0x7fa7c0009be0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e090 msgr2=0x7fa7d009e470 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7bdffb640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e090 0x7fa7d009e470 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009710 tx=0x7fa7c0009be0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.127+0000 7fa7d4ac5640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7a003dee0 0x7fa7a00403a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4006f80 tx=0x7fa7c4006ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d7551640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d7551640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e9b0 0x7fa7d01da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d7551640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d0125f40 con 0x7fa7d009e9b0 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e9b0 0x7fa7d01da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 10 monclient: get_auth_request con 0x7fa7d009e9b0 auth_method 0 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 10 monclient(hunting): handle_auth_done global_id 134401 payload 824 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.130+0000 7fa7d52c6640 1 -- 10.20.193.78:0/567998168 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa7d00744d0 con 0x7fa7d009e9b0 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7d52c6640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e9b0 msgr2=0x7fa7d01da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7d52c6640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e9b0 0x7fa7d01da560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009710 tx=0x7fa7c00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0016040 con 0x7fa7d009e9b0 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 10 monclient: dump: 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7fa7c0025420 con 0x7fa7d009e9b0 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e9b0 msgr2=0x7fa7d01da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e9b0 0x7fa7d01da560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009710 tx=0x7fa7c00047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e9b0 msgr2=0x7fa7d01da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:58.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7bdffb640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7d009e9b0 0x7fa7d01da560 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009710 tx=0x7fa7c00047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 164, 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 225852, 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T05:59:58.140 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:58.141 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:58.142 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:58.143 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.144 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:58.145 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:58.146 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:58.147 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:58.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7a003dee0 msgr2=0x7fa7a00403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7d7551640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7a003dee0 0x7fa7a00403a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4006f80 tx=0x7fa7c4006ed0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7d7551640 10 monclient: shutdown 2026-02-17T05:59:58.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7d7551640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:58.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.133+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 msgr2=0x7fa7d009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.134+0000 7fa7d5ac7640 1 -- 10.20.193.78:0/567998168 reap_dead start 2026-02-17T05:59:58.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.134+0000 7fa7d7551640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d009db50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc035000 tx=0x7fa7cc004b40 comp rx=0 tx=0).stop 2026-02-17T05:59:58.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.134+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 shutdown_connections 2026-02-17T05:59:58.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.134+0000 7fa7d7551640 1 --2- 10.20.193.78:0/567998168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7d0134df0 0x7fa7d009db50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.134+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 >> 10.20.193.78:0/567998168 conn(0x7fa7d00a6620 msgr2=0x7fa7d0135850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:58.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.134+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 shutdown_connections 2026-02-17T05:59:58.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.134+0000 7fa7d7551640 1 -- 10.20.193.78:0/567998168 wait complete. 2026-02-17T05:59:58.149 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:58.149 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T05:59:58.149 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.224+0000 7ff9d3ade640 10 monclient: get_monmap_and_config 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.224+0000 7ff9d3ade640 10 monclient: build_initial_monmap 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.224+0000 7ff9d3ade640 10 monclient: monmap: 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:58.225605+0000 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:58.225605+0000 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:58.225 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.224+0000 7ff9d3ade640 1 Processor -- start 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.224+0000 7ff9d3ade640 1 -- start start 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.224+0000 7ff9d3ade640 10 monclient: init 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ea90 0x7ff9cc0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient(hunting): picked mon.noname-b con 0x7ff9cc09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff9cc09ee70 0x7ff9cc09f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient(hunting): picked mon.noname-c con 0x7ff9cc09ee70 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff9cc09d980 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9cc0a0a20 con 0x7ff9cc09d980 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff9cc0a0d00 con 0x7ff9cc09ee70 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc0a0460 con 0x7ff9cc09ea90 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d3ade640 10 monclient(hunting): authenticate will time out at 1443.779122s 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ea90 0x7ff9cc0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ea90 0x7ff9cc0a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36568/0 (socket says 10.20.193.78:36568) 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 1 -- 10.20.193.78:0/1028532678 learned_addr learned my addr 10.20.193.78:0/1028532678 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient(hunting): get_auth_request con 0x7ff9cc09ea90 auth_method 0 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1052640 1 --2- 10.20.193.78:0/1028532678 >> v2:10.20.193.115:3300/0 conn(0x7ff9cc09ee70 0x7ff9cc09f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1052640 10 monclient(hunting): get_auth_request con 0x7ff9cc09ee70 auth_method 0 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1052640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1052640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1052640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient(hunting): handle_auth_done global_id 144398 payload 274 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 1 -- 10.20.193.78:0/1028532678 >> v2:10.20.193.115:3300/0 conn(0x7ff9cc09ee70 msgr2=0x7ff9cc09f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 1 --2- 10.20.193.78:0/1028532678 >> v2:10.20.193.115:3300/0 conn(0x7ff9cc09ee70 0x7ff9cc09f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 1 -- 10.20.193.78:0/1028532678 >> v1:10.20.193.115:6789/0 conn(0x7ff9cc09d980 legacy=0x7ff9cc09ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient: _finish_hunting 0 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 1 monclient: found mon.noname-b 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.225+0000 7ff9d1853640 1 -- 10.20.193.78:0/1028532678 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc0a0740 con 0x7ff9cc09ea90 2026-02-17T05:59:58.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.227+0000 7ff9d1853640 1 -- 10.20.193.78:0/1028532678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ea90 msgr2=0x7ff9cc0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.227+0000 7ff9d1853640 1 --2- 10.20.193.78:0/1028532678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ea90 0x7ff9cc0a13d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009970 tx=0x7ff9bc02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faae15fb0b959ed3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.227+0000 7ff9d0851640 1 -- 10.20.193.78:0/1028532678 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc03c070 con 0x7ff9cc09ea90 2026-02-17T05:59:58.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.227+0000 7ff9d0851640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.227+0000 7ff9d0851640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:58.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.227+0000 7ff9d0851640 10 monclient: dump: 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 10 monclient: _finish_auth 0 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 10 monclient: _check_auth_tickets 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 1 -- 10.20.193.78:0/1028532678 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9bc02fcd0 con 0x7ff9cc09ea90 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 5 monclient: authenticate success, global_id 144398 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 1 -- 10.20.193.78:0/1028532678 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc037660 con 0x7ff9cc09ea90 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 4 set_mon_vals no callback set 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d0851640 10 monclient: dump: 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.229 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 10 monclient: shutdown 2026-02-17T05:59:58.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 -- 10.20.193.78:0/1028532678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ea90 msgr2=0x7ff9cc0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/1028532678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ea90 0x7ff9cc0a13d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009970 tx=0x7ff9bc02ee90 comp rx=0 tx=0).stop 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 -- 10.20.193.78:0/1028532678 shutdown_connections 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/1028532678 >> v2:10.20.193.115:3300/0 conn(0x7ff9cc09ee70 0x7ff9cc09f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/1028532678 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ea90 0x7ff9cc0a13d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 -- 10.20.193.78:0/1028532678 >> 10.20.193.78:0/1028532678 conn(0x7ff9cc0a7520 msgr2=0x7ff9cc0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 -- 10.20.193.78:0/1028532678 shutdown_connections 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 -- 10.20.193.78:0/1028532678 wait complete. 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 10 monclient: build_initial_monmap 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 10 monclient: monmap: 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 Processor -- start 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 1 -- start start 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.228+0000 7ff9d3ade640 10 monclient: init 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9cc09d980 0x7ff9cc134d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient(hunting): picked mon.noname-a con 0x7ff9cc09d980 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9cc09ea90 0x7ff9cc1352a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient(hunting): picked mon.noname-b con 0x7ff9cc09ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ee70 0x7ff9cc12fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient(hunting): picked mon.noname-c con 0x7ff9cc09ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0a66a0 con 0x7ff9cc09d980 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc09f7d0 con 0x7ff9cc09ea90 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc0a6820 con 0x7ff9cc09ee70 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:58.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d3ade640 10 monclient(hunting): authenticate will time out at 1443.783264s 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ee70 0x7ff9cc12fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9cc09d980 0x7ff9cc134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ee70 0x7ff9cc12fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36578/0 (socket says 10.20.193.78:36578) 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9cc09d980 0x7ff9cc134d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47282/0 (socket says 10.20.193.78:47282) 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 -- 10.20.193.78:0/508205926 learned_addr learned my addr 10.20.193.78:0/508205926 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient(hunting): get_auth_request con 0x7ff9cc09ee70 auth_method 0 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 10 monclient(hunting): get_auth_request con 0x7ff9cc09d980 auth_method 0 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1052640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9cc09ea90 0x7ff9cc1352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1052640 10 monclient(hunting): get_auth_request con 0x7ff9cc09ea90 auth_method 0 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1052640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1052640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1052640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient(hunting): handle_auth_done global_id 144401 payload 824 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 -- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9cc09ea90 msgr2=0x7ff9cc1352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9cc09ea90 0x7ff9cc1352a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 -- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9cc09d980 msgr2=0x7ff9cc134d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9cc09d980 0x7ff9cc134d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient: _finish_hunting 0 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 monclient: found mon.noname-c 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d2054640 1 -- 10.20.193.78:0/508205926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc132540 con 0x7ff9cc09ee70 2026-02-17T05:59:58.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.229+0000 7ff9d1853640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9cc09d980 0x7ff9cc134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d2054640 1 -- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ee70 msgr2=0x7ff9cc12fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d2054640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ee70 0x7ff9cc12fae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800cc10 tx=0x7ff9c800a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 1 -- 10.20.193.78:0/508205926 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8016020 con 0x7ff9cc09ee70 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: dump: 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 1 -- 10.20.193.78:0/508205926 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c8012620 con 0x7ff9cc09ee70 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 1 -- 10.20.193.78:0/508205926 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8012a10 con 0x7ff9cc09ee70 2026-02-17T05:59:58.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d3ade640 5 monclient: authenticate success, global_id 144401 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d3ade640 10 monclient: _renew_subs 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d3ade640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9bbfff640 4 set_mon_vals no callback set 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc131020 con 0x7ff9cc09ee70 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9ba7fc640 10 monclient: dump: 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d3ade640 10 monclient: _renew_subs 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d3ade640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.231+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1377f0 con 0x7ff9cc09ee70 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.232+0000 7ff9ba7fc640 1 -- 10.20.193.78:0/508205926 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff9c8012c30 con 0x7ff9cc09ee70 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.232+0000 7ff9d3ade640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.232+0000 7ff9d3ade640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.232+0000 7ff9d3ade640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.232+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9cc0a8ea0 con 0x7ff9cc09ee70 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.232+0000 7ff9ba7fc640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff99c03df00 0x7ff99c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.232+0000 7ff9ba7fc640 1 -- 10.20.193.78:0/508205926 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(495..495 src has 335..495) ==== 4811+0+0 (secure 0 0 0) 0x7ff9c80531f0 con 0x7ff9cc09ee70 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.232+0000 7ff9d1853640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff99c03df00 0x7ff99c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:58.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.233+0000 7ff9d1853640 10 monclient: get_auth_request con 0x7ff99c03df00 auth_method 0 2026-02-17T05:59:58.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.235+0000 7ff9ba7fc640 1 -- 10.20.193.78:0/508205926 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff9c8020020 con 0x7ff9cc09ee70 2026-02-17T05:59:58.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.235+0000 7ff9ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:58.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.235+0000 7ff9ba7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:58.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.235+0000 7ff9d1853640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff99c03df00 0x7ff99c0403c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc036000 tx=0x7ff9bc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:58.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.348+0000 7ff9d3ade640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T05:59:58.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.348+0000 7ff9d3ade640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T05:59:58.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.348+0000 7ff9d3ade640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:58.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.348+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7ff9cc073b70 con 0x7ff9cc09ee70 2026-02-17T05:59:58.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.348+0000 7ff9ba7fc640 1 -- 10.20.193.78:0/508205926 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7ff9c8051bd0 con 0x7ff9cc09ee70 2026-02-17T05:59:58.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.348+0000 7ff9ba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T05:59:58.349 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.348+0000 7ff9ba7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:58.350 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.348+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.350 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.348+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=23,config=2,health=12,kv=12,logm=51} crc {auth=3892087484,config=692619927,health=1984972710,kv=449989585,logm=419350310}) 2026-02-17T05:59:58.350 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.348+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=25,config=2,health=12,kv=11,logm=50} crc {auth=2961045615,config=692619927,health=1984972710,kv=3037601728,logm=976154574}) 2026-02-17T05:59:58.350 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.348+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.350 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.349+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=23,config=2,health=12,kv=12,logm=51} crc {auth=3892087484,config=692619927,health=1984972710,kv=449989585,logm=419350310}) 2026-02-17T05:59:58.350 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.349+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=484086191,config=692619927,health=1984972710,kv=449989585,logm=2488413972}) 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.349+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.349+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=428571371}) 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.349+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=683024954}) 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.349+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.349+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=428571371}) 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.349+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2564447137}) 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1035022717}) 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2062660108}) 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1035022717}) 2026-02-17T05:59:58.351 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3086655895}) 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff99c03df00 msgr2=0x7ff99c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff99c03df00 0x7ff99c0403c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc036000 tx=0x7ff9bc033000 comp rx=0 tx=0).stop 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 10 monclient: shutdown 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ee70 msgr2=0x7ff9cc12fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ee70 0x7ff9cc12fae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800cc10 tx=0x7ff9c800a430 comp rx=0 tx=0).stop 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 shutdown_connections 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff9cc09ee70 0x7ff9cc12fae0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff99c03df00 0x7ff99c0403c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff9cc09ea90 0x7ff9cc1352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 --2- 10.20.193.78:0/508205926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff9cc09d980 0x7ff9cc134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 >> 10.20.193.78:0/508205926 conn(0x7ff9cc0a7520 msgr2=0x7ff9cc138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 shutdown_connections 2026-02-17T05:59:58.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:58.350+0000 7ff9d3ade640 1 -- 10.20.193.78:0/508205926 wait complete. 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2968131570}) 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2399014139}) 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2968131570}) 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.350+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=427655858}) 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1786918412}) 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3398454556}) 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.355 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1786918412}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3140164694}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729510719}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2303683369}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729510719}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3846022647}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060819518}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615674232}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060819518}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.351+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1866278817}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293202228}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2841032647}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293202228}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=775816714}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=409575761}) 2026-02-17T05:59:58.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3683616046}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=409575761}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.352+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1250109970}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=739506663}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=528379698}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=739506663}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1734543408}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1546931599}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=644571493}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1546931599}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4042392613}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2510830428}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=734230486}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2510830428}) 2026-02-17T05:59:58.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.353+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2472602728}) 2026-02-17T05:59:58.359 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T05:59:58.473 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=5147347}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=908043372}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=5147347}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2342950902}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=949430277}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3529004353}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=949430277}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.473+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919835287}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=321592134}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4122887531}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=321592134}) 2026-02-17T05:59:58.474 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2586395337}) 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4081861085}) 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=692458423}) 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4081861085}) 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3356395564}) 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4003027377}) 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3199814245}) 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4003027377}) 2026-02-17T05:59:58.475 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.474+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3054398964}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=814939099}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=804080852}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=814939099}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3213294366}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3973232358}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3276973569}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3973232358}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.475+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1114814188}) 2026-02-17T05:59:58.476 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1595670613}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2617653676}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1595670613}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3254592929}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=824328785}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=162799877}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=824328785}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4236252651}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.476+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955322383}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2997452703}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955322383}) 2026-02-17T05:59:58.477 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1189812028}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=99614758}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1557098832}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=99614758}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1421722661}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1091896657}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=835460303}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1091896657}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.477+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1135060038}) 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.478 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=496676121}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2649928550}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=496676121}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3922275265}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874898838}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2245318179}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874898838}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.478+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913475467}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3228565271}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262966824}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3228565271}) 2026-02-17T05:59:58.479 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1097678156}) 2026-02-17T05:59:58.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.480 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3949060660}) 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539410784}) 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3949060660}) 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3100138958}) 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1158606590}) 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1979125422}) 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1158606590}) 2026-02-17T05:59:58.481 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.479+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3085741219}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3992000897}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3905574583}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3992000897}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=779092258}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2861312245}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2894484512}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2861312245}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1494223996}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3784560921}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=861080340}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3784560921}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.480+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783965209}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1331352130}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2416724638}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1331352130}) 2026-02-17T05:59:58.482 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1142542667}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2131581981}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1089110543}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2131581981}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2953502249}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2031655400}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3457765225}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2031655400}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.481+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=919696813}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1540546647}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2794962597}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1540546647}) 2026-02-17T05:59:58.483 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1979466079}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2926009883}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=97304428}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2926009883}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.482+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1587969604}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474676496}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3091110441}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474676496}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3423626503}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468945201}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76260468}) 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.484 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468945201}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2688034617}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1321614199}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3025299643}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1321614199}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.483+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1575662200}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1871251917}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3193178440}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1871251917}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2573713345}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698854018}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1692628706}) 2026-02-17T05:59:58.485 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698854018}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593027122}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1147884101}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1346800583}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1147884101}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.484+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2553136406}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=63,mdsmap=3,mgr=5,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=1} crc {logm=3475616734,mdsmap=1160443688,mgr=357862720,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2031637369,mon_config_key=1912222953,monmap=1784110042,osd_metadata=795255734}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=63,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=1} crc {logm=3454242209,mdsmap=1160443688,mgr=1039804757,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1200623470,mon_config_key=1912222953,monmap=3671747392}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=63,mdsmap=3,mgr=5,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=1} crc {logm=3475616734,mdsmap=1160443688,mgr=357862720,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2031637369,mon_config_key=1912222953,monmap=1784110042,osd_metadata=795255734}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=63,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=1} crc {logm=3699862249,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1200623470,mon_config_key=1987855637,monmap=3671747391}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=88,osdmap=6} crc {osd_metadata=1501630441,osd_pg_creating=32239340,osd_snap=3476221098,osdmap=2507129355}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=85,osdmap=9} crc {osd_metadata=1501630441,osd_pg_creating=32239340,osd_snap=130083623,osdmap=775534598}) 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.486 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=88,osdmap=6} crc {osd_metadata=1501630441,osd_pg_creating=32239340,osd_snap=3476221098,osdmap=2507129355}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.485+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=86,osdmap=8} crc {osd_metadata=1501630441,osd_pg_creating=32239340,osd_snap=2021295976,osdmap=450704525}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=422498854}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3378632429}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=422498854}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3357106168}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3590759126}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2637574872}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3590759126}) 2026-02-17T05:59:58.487 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.486+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3399777181}) 2026-02-17T05:59:58.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.489+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.489+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=35} crc {osdmap=1680578338}) 2026-02-17T05:59:58.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.489+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=32} crc {osdmap=3206254759}) 2026-02-17T05:59:58.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.489+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T05:59:58.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.489+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=35} crc {osdmap=1680578338}) 2026-02-17T05:59:58.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T05:59:58.489+0000 7fcae551b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=35} crc {osdmap=836836071}) 2026-02-17T05:59:58.789 INFO:tasks.rados.rados.0.trial115.stdout:2317: done (2 left) 2026-02-17T05:59:58.789 INFO:tasks.rados.rados.0.trial115.stdout:2318: done (1 left) 2026-02-17T05:59:58.790 INFO:tasks.rados.rados.0.trial115.stdout:2319: done (0 left) 2026-02-17T05:59:58.790 INFO:tasks.rados.rados.0.trial115.stdout:2320: write (excl) oid 38 current snap is 236 2026-02-17T05:59:58.790 INFO:tasks.rados.rados.0.trial115.stdout:2320: seq_num 823 ranges {632161=496051,1593504=495615,2701123=420396,3611645=220516} 2026-02-17T05:59:58.793 INFO:tasks.rados.rados.0.trial115.stdout:2320: writing trial11545420-38 from 632161 to 1128212 tid 1 2026-02-17T05:59:58.797 INFO:tasks.rados.rados.0.trial115.stdout:2320: writing trial11545420-38 from 1593504 to 2089119 tid 2 2026-02-17T05:59:58.801 INFO:tasks.rados.rados.0.trial115.stdout:2320: writing trial11545420-38 from 2701123 to 3121519 tid 3 2026-02-17T05:59:58.803 INFO:tasks.rados.rados.0.trial115.stdout:2320: writing trial11545420-38 from 3611645 to 3832161 tid 4 2026-02-17T05:59:58.803 INFO:tasks.rados.rados.0.trial115.stdout:2321: write (excl) oid 11 current snap is 236 2026-02-17T05:59:58.803 INFO:tasks.rados.rados.0.trial115.stdout:2320: finishing write tid 1 to trial11545420-38 2026-02-17T05:59:58.803 INFO:tasks.rados.rados.0.trial115.stdout:2320: finishing write tid 2 to trial11545420-38 2026-02-17T05:59:58.803 INFO:tasks.rados.rados.0.trial115.stdout:2320: finishing write tid 3 to trial11545420-38 2026-02-17T05:59:58.804 INFO:tasks.rados.rados.0.trial115.stdout:2321: seq_num 824 ranges {648968=571300,1651986=639273,2728546=631668} 2026-02-17T05:59:58.808 INFO:tasks.rados.rados.0.trial115.stdout:2321: writing trial11545420-11 from 648968 to 1220268 tid 1 2026-02-17T05:59:58.812 INFO:tasks.rados.rados.0.trial115.stdout:2321: writing trial11545420-11 from 1651986 to 2291259 tid 2 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2321: writing trial11545420-11 from 2728546 to 3360214 tid 3 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2322: read oid 13 snap -1 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2322: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2323: delete oid 31 current snap is 236 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2320: finishing write tid 4 to trial11545420-38 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2320: finishing write tid 5 to trial11545420-38 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2320: finishing write tid 6 to trial11545420-38 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2320: oid 38 updating version 0 to 1300 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2320: oid 38 version 1300 is already newer than 1297 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2320: oid 38 version 1300 is already newer than 1296 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2320: oid 38 version 1300 is already newer than 1299 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:2320: oid 38 version 1300 is already newer than 1298 2026-02-17T05:59:58.817 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1300 (ObjNum 823 snap 236 seq_num 823) dirty exists 2026-02-17T05:59:58.818 INFO:tasks.rados.rados.0.trial115.stdout:2320: left oid 38 (ObjNum 823 snap 236 seq_num 823) 2026-02-17T05:59:58.818 INFO:tasks.rados.rados.0.trial115.stdout:2321: finishing write tid 1 to trial11545420-11 2026-02-17T05:59:58.818 INFO:tasks.rados.rados.0.trial115.stdout:2321: finishing write tid 2 to trial11545420-11 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: finishing write tid 3 to trial11545420-11 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: finishing write tid 4 to trial11545420-11 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: finishing write tid 5 to trial11545420-11 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: oid 11 updating version 0 to 1195 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: oid 11 version 1195 is already newer than 1194 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: oid 11 updating version 1195 to 1196 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: oid 11 updating version 1196 to 1197 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1197 (ObjNum 824 snap 236 seq_num 824) dirty exists 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: left oid 11 (ObjNum 824 snap 236 seq_num 824) 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2320: done (3 left) 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2321: done (2 left) 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2323: done (1 left) 2026-02-17T05:59:58.819 INFO:tasks.rados.rados.0.trial115.stdout:2324: read oid 20 snap -1 2026-02-17T05:59:58.820 INFO:tasks.rados.rados.0.trial115.stdout:2324: expect (ObjNum 811 snap 234 seq_num 811) 2026-02-17T05:59:58.820 INFO:tasks.rados.rados.0.trial115.stdout:2325: delete oid 36 current snap is 236 2026-02-17T05:59:58.820 INFO:tasks.rados.rados.0.trial115.stdout:2322: expect (ObjNum 699 snap 190 seq_num 699) 2026-02-17T05:59:58.970 INFO:tasks.rados.rados.0.trial115.stdout:2324: expect (ObjNum 811 snap 234 seq_num 811) 2026-02-17T05:59:59.089 INFO:tasks.rados.rados.0.trial115.stdout:2322: done (2 left) 2026-02-17T05:59:59.089 INFO:tasks.rados.rados.0.trial115.stdout:2324: done (1 left) 2026-02-17T05:59:59.089 INFO:tasks.rados.rados.0.trial115.stdout:2325: done (0 left) 2026-02-17T05:59:59.089 INFO:tasks.rados.rados.0.trial115.stdout:2326: rollback oid 12 current snap is 236 2026-02-17T05:59:59.089 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 224 2026-02-17T05:59:59.089 INFO:tasks.rados.rados.0.trial115.stdout:2327: write (excl) oid 7 current snap is 236 2026-02-17T05:59:59.089 INFO:tasks.rados.rados.0.trial115.stdout:2327: seq_num 827 ranges {699389=797047,2111079=502953,3094462=581837} 2026-02-17T05:59:59.094 INFO:tasks.rados.rados.0.trial115.stdout:2327: writing trial11545420-7 from 699389 to 1496436 tid 1 2026-02-17T05:59:59.098 INFO:tasks.rados.rados.0.trial115.stdout:2327: writing trial11545420-7 from 2111079 to 2614032 tid 2 2026-02-17T05:59:59.103 INFO:tasks.rados.rados.0.trial115.stdout:2327: writing trial11545420-7 from 3094462 to 3676299 tid 3 2026-02-17T05:59:59.103 INFO:tasks.rados.rados.0.trial115.stdout:2328: copy_from oid 21 from oid 37 current snap is 236 2026-02-17T05:59:59.103 INFO:tasks.rados.rados.0.trial115.stdout:2326: finishing rollback tid 1 to trial11545420-12 2026-02-17T05:59:59.103 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 0 (ObjNum 129 snap 0 seq_num 1334162136) dirty dne 2026-02-17T05:59:59.103 INFO:tasks.rados.rados.0.trial115.stdout:2327: finishing write tid 1 to trial11545420-7 2026-02-17T05:59:59.103 INFO:tasks.rados.rados.0.trial115.stdout:2327: finishing write tid 2 to trial11545420-7 2026-02-17T05:59:59.103 INFO:tasks.rados.rados.0.trial115.stdout:2326: done (2 left) 2026-02-17T05:59:59.103 INFO:tasks.rados.rados.0.trial115.stdout:2329: copy_from oid 35 from oid 47 current snap is 236 2026-02-17T05:59:59.104 INFO:tasks.rados.rados.0.trial115.stdout:2330: snap_remove snap 224 2026-02-17T05:59:59.360 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T05:59:59.360 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:59.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.425+0000 7f5766ee1640 10 monclient: get_monmap_and_config 2026-02-17T05:59:59.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.425+0000 7f5766ee1640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.425+0000 7f5766ee1640 10 monclient: monmap: 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:59.426929+0000 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:59.426929+0000 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:59.430 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 1 Processor -- start 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 1 -- start start 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient: init 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576006c890 0x7f57600a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient(hunting): picked mon.noname-b con 0x7f576006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:59.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f576006cc70 0x7f576006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient(hunting): picked mon.noname-c con 0x7f576006cc70 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5760134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5760074150 con 0x7f5760134df0 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f576009ef70 con 0x7f576006cc70 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f576009f250 con 0x7f576006c890 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5766ee1640 10 monclient(hunting): authenticate will time out at 1444.980460s 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576006c890 0x7f57600a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576006c890 0x7f57600a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36584/0 (socket says 10.20.193.78:36584) 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 1 -- 10.20.193.78:0/1409774391 learned_addr learned my addr 10.20.193.78:0/1409774391 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient(hunting): get_auth_request con 0x7f576006c890 auth_method 0 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5757fff640 1 --2- 10.20.193.78:0/1409774391 >> v2:10.20.193.115:3300/0 conn(0x7f576006cc70 0x7f576006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient(hunting): handle_auth_done global_id 144404 payload 274 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 1 -- 10.20.193.78:0/1409774391 >> v2:10.20.193.115:3300/0 conn(0x7f576006cc70 msgr2=0x7f576006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 1 --2- 10.20.193.78:0/1409774391 >> v2:10.20.193.115:3300/0 conn(0x7f576006cc70 0x7f576006d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 1 -- 10.20.193.78:0/1409774391 >> v1:10.20.193.115:6789/0 conn(0x7f5760134df0 legacy=0x7f5760135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 1 monclient: found mon.noname-b 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5764c56640 1 -- 10.20.193.78:0/1409774391 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576009df40 con 0x7f576006c890 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5757fff640 10 monclient: get_auth_request con 0x7f576006cc70 auth_method 0 2026-02-17T05:59:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.426+0000 7f5757fff640 1 --2- 10.20.193.78:0/1409774391 >> v2:10.20.193.115:3300/0 conn(0x7f576006cc70 0x7f576006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:59.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5764c56640 1 -- 10.20.193.78:0/1409774391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576006c890 msgr2=0x7f57600a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5764c56640 1 --2- 10.20.193.78:0/1409774391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576006c890 0x7f57600a01f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5750009970 tx=0x7f575002efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce318864f35bfea server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 1 -- 10.20.193.78:0/1409774391 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575003c070 con 0x7f576006c890 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: dump: 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 5 monclient: authenticate success, global_id 144404 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 1 -- 10.20.193.78:0/1409774391 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5750037440 con 0x7f576006c890 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 1 -- 10.20.193.78:0/1409774391 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750037810 con 0x7f576006c890 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 4 set_mon_vals no callback set 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f57577fe640 10 monclient: dump: 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.442 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 10 monclient: shutdown 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 1 -- 10.20.193.78:0/1409774391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576006c890 msgr2=0x7f57600a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 1 --2- 10.20.193.78:0/1409774391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576006c890 0x7f57600a01f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5750009970 tx=0x7f575002efb0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 1 -- 10.20.193.78:0/1409774391 shutdown_connections 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 1 --2- 10.20.193.78:0/1409774391 >> v2:10.20.193.115:3300/0 conn(0x7f576006cc70 0x7f576006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 1 --2- 10.20.193.78:0/1409774391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f576006c890 0x7f57600a01f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 1 -- 10.20.193.78:0/1409774391 >> 10.20.193.78:0/1409774391 conn(0x7f57600a6620 msgr2=0x7f57600a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 1 -- 10.20.193.78:0/1409774391 shutdown_connections 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.429+0000 7f5766ee1640 1 -- 10.20.193.78:0/1409774391 wait complete. 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient: monmap: 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 1 Processor -- start 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 1 -- start start 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient: init 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576006c890 0x7f5760133bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient(hunting): picked mon.noname-a con 0x7f576006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576006cc70 0x7f5760134100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient(hunting): picked mon.noname-b con 0x7f576006cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5760134df0 0x7f576013e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient(hunting): picked mon.noname-c con 0x7f5760134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f576009d360 con 0x7f576006c890 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f576009d1e0 con 0x7f576006cc70 2026-02-17T05:59:59.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5760135950 con 0x7f5760134df0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5766ee1640 10 monclient(hunting): authenticate will time out at 1444.984585s 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5760134df0 0x7f576013e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576006c890 0x7f5760133bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5760134df0 0x7f576013e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36590/0 (socket says 10.20.193.78:36590) 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 -- 10.20.193.78:0/2958821296 learned_addr learned my addr 10.20.193.78:0/2958821296 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient(hunting): get_auth_request con 0x7f5760134df0 auth_method 0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 10 monclient(hunting): get_auth_request con 0x7f576006c890 auth_method 0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient(hunting): handle_auth_done global_id 144407 payload 824 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 -- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576006cc70 msgr2=0x7f5760134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576006cc70 0x7f5760134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 -- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576006c890 msgr2=0x7f5760133bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576006c890 0x7f5760133bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 monclient: found mon.noname-c 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5765457640 1 -- 10.20.193.78:0/2958821296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576011de10 con 0x7f5760134df0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.430+0000 7f5764c56640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576006c890 0x7f5760133bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.432+0000 7f5765457640 1 -- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5760134df0 msgr2=0x7f576013e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.432+0000 7f5765457640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5760134df0 0x7f576013e020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f575c00ba10 tx=0x7f575c00bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.432+0000 7f57557fa640 1 -- 10.20.193.78:0/2958821296 DispatchQueue::entry inject delay of 0.002000 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 1 -- 10.20.193.78:0/2958821296 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c006e60 con 0x7f5760134df0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: dump: 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 1 -- 10.20.193.78:0/2958821296 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575c010070 con 0x7f5760134df0 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 5 monclient: authenticate success, global_id 144407 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 10 monclient: _renew_subs 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576011e6b0 con 0x7f5760134df0 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 1 -- 10.20.193.78:0/2958821296 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c00cb90 con 0x7f5760134df0 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 10 monclient: dump: 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5756ffd640 4 set_mon_vals no callback set 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 10 monclient: _renew_subs 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f57600a6c90 con 0x7f5760134df0 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5760074980 con 0x7f5760134df0 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 1 -- 10.20.193.78:0/2958821296 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f575c004070 con 0x7f5760134df0 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.435+0000 7f57557fa640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573003df00 0x7f57300403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.436+0000 7f57557fa640 1 -- 10.20.193.78:0/2958821296 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(496..496 src has 335..496) ==== 4783+0+0 (secure 0 0 0) 0x7f575c052520 con 0x7f5760134df0 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.436+0000 7f5764c56640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573003df00 0x7f57300403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.436+0000 7f5764c56640 10 monclient: get_auth_request con 0x7f573003df00 auth_method 0 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.438+0000 7f57557fa640 1 -- 10.20.193.78:0/2958821296 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f575c0139d0 con 0x7f5760134df0 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.438+0000 7f57557fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.438+0000 7f57557fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:59.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.439+0000 7f5764c56640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573003df00 0x7f57300403c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5750036000 tx=0x7f5750033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.558+0000 7f5766ee1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T05:59:59.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.558+0000 7f5766ee1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:59.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.558+0000 7f5766ee1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.558+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f57600a01d0 con 0x7f5760134df0 2026-02-17T05:59:59.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.558+0000 7f57557fa640 1 -- 10.20.193.78:0/2958821296 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f575c004620 con 0x7f5760134df0 2026-02-17T05:59:59.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.558+0000 7f57557fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T05:59:59.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.558+0000 7f57557fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:59.561 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":164,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T05:59:59.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573003df00 msgr2=0x7f57300403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573003df00 0x7f57300403c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5750036000 tx=0x7f5750033000 comp rx=0 tx=0).stop 2026-02-17T05:59:59.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 10 monclient: shutdown 2026-02-17T05:59:59.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5760134df0 msgr2=0x7f576013e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5760134df0 0x7f576013e020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f575c00ba10 tx=0x7f575c00bee0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 shutdown_connections 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5760134df0 0x7f576013e020 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f573003df00 0x7f57300403c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f576006cc70 0x7f5760134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 --2- 10.20.193.78:0/2958821296 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f576006c890 0x7f5760133bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 >> 10.20.193.78:0/2958821296 conn(0x7f57600a6620 msgr2=0x7f57600a1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 shutdown_connections 2026-02-17T05:59:59.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.559+0000 7f5766ee1640 1 -- 10.20.193.78:0/2958821296 wait complete. 2026-02-17T05:59:59.568 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T05:59:59.568 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T05:59:59.568 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T05:59:59.568 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.634+0000 7fc00a276640 10 monclient: get_monmap_and_config 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.634+0000 7fc00a276640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.634+0000 7fc00a276640 10 monclient: monmap: 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:59.635846+0000 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:59.635846+0000 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.637 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.634+0000 7fc00a276640 1 Processor -- start 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 1 -- start start 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient: init 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc00406c890 0x7fc00406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient(hunting): picked mon.noname-c con 0x7fc00406c890 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc00409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc00406d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:59.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc004135c10 con 0x7fc00406d1b0 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00406daf0 con 0x7fc00409c4a0 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc0040a0880 con 0x7fc00406c890 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc00a276640 10 monclient(hunting): authenticate will time out at 1445.189367s 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc003fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc00406d1b0 0x7fc004135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:53966/0 (socket says 10.20.193.78:53966) 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc003fff640 1 -- 10.20.193.78:0/258666354 learned_addr learned my addr 10.20.193.78:0/258666354 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc0037fe640 1 --2- 10.20.193.78:0/258666354 >> v2:10.20.193.115:3300/0 conn(0x7fc00406c890 0x7fc00406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc0037fe640 10 monclient(hunting): get_auth_request con 0x7fc00406c890 auth_method 0 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc0037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc0037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc0037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.635+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc003fff640 1 -- 10.20.193.78:0/258666354 >> v1:10.20.193.78:6789/0 conn(0x7fc00406d1b0 legacy=0x7fc004135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0027fc640 1 -- 10.20.193.78:0/258666354 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1863638535 0 0) 0x7fc004135c10 con 0x7fc00406d1b0 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0027fc640 10 monclient(hunting): _negotiate 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0027fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0027fc640 10 monclient(hunting): my global_id is 174135 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0027fc640 1 -- 10.20.193.78:0/258666354 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfe8003620 con 0x7fc00406d1b0 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc002ffd640 1 -- 10.20.193.78:0/258666354 >> v1:10.20.193.115:6789/0 conn(0x7fc00409c4a0 legacy=0x7fc004134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0037fe640 10 monclient(hunting): handle_auth_done global_id 134410 payload 274 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0037fe640 1 -- 10.20.193.78:0/258666354 >> v1:10.20.193.115:6789/0 conn(0x7fc00409c4a0 legacy=0x7fc004134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0037fe640 1 -- 10.20.193.78:0/258666354 >> v1:10.20.193.78:6789/0 conn(0x7fc00406d1b0 legacy=0x7fc004135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0027fc640 1 -- 10.20.193.78:0/258666354 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1113392347 0 0) 0x7fbfe8003620 con 0x7fc00406d1b0 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0037fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0037fe640 1 monclient: found mon.noname-c 2026-02-17T05:59:59.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T05:59:59.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0037fe640 1 -- 10.20.193.78:0/258666354 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00409cc80 con 0x7fc00406c890 2026-02-17T05:59:59.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.637+0000 7fc0027fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0037fe640 1 -- 10.20.193.78:0/258666354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc00406c890 msgr2=0x7fc00406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0037fe640 1 --2- 10.20.193.78:0/258666354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc00406c890 0x7fc00406cc70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbff00099a0 tx=0x7fbff002eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa23fa9cb7c1fe5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 1 -- 10.20.193.78:0/258666354 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff003c070 con 0x7fc00406c890 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: dump: 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.647 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc00a276640 5 monclient: authenticate success, global_id 134410 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc00a276640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 1 -- 10.20.193.78:0/258666354 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff002fb10 con 0x7fc00406c890 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 1 -- 10.20.193.78:0/258666354 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8003530 con 0x7fc00406c890 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc0027fc640 10 monclient: dump: 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.648 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc00a276640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.640+0000 7fc00a276640 4 set_mon_vals no callback set 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 10 monclient: shutdown 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 -- 10.20.193.78:0/258666354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc00406c890 msgr2=0x7fc00406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 --2- 10.20.193.78:0/258666354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc00406c890 0x7fc00406cc70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbff00099a0 tx=0x7fbff002eeb0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 -- 10.20.193.78:0/258666354 shutdown_connections 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 --2- 10.20.193.78:0/258666354 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc00406c890 0x7fc00406cc70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 -- 10.20.193.78:0/258666354 >> 10.20.193.78:0/258666354 conn(0x7fc0040a6620 msgr2=0x7fc0040a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 -- 10.20.193.78:0/258666354 shutdown_connections 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 -- 10.20.193.78:0/258666354 wait complete. 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 10 monclient: monmap: 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:59.649 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 Processor -- start 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 1 -- start start 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.641+0000 7fc00a276640 10 monclient: init 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc00406c890 0x7fc0041cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient(hunting): picked mon.noname-a con 0x7fc00406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc00406d1b0 0x7fc0041cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient(hunting): picked mon.noname-c con 0x7fc00406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:59.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc00409c4a0 0x7fc004123d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient(hunting): picked mon.noname-b con 0x7fc00409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0040a0020 con 0x7fc00406c890 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc00406d640 con 0x7fc00409c4a0 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc0040a01a0 con 0x7fc00406d1b0 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc00a276640 10 monclient(hunting): authenticate will time out at 1445.196077s 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc00406c890 0x7fc0041cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc00406c890 0x7fc0041cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47300/0 (socket says 10.20.193.78:47300) 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 -- 10.20.193.78:0/379481868 learned_addr learned my addr 10.20.193.78:0/379481868 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient(hunting): get_auth_request con 0x7fc00406c890 auth_method 0 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc002ffd640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc00406d1b0 0x7fc0041cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc002ffd640 10 monclient(hunting): get_auth_request con 0x7fc00406d1b0 auth_method 0 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc002ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc002ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient(hunting): handle_auth_done global_id 174138 payload 824 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc00406d1b0 msgr2=0x7fc0041cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc00406d1b0 0x7fc0041cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc00409c4a0 msgr2=0x7fc004123d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc00409c4a0 0x7fc004123d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 monclient: found mon.noname-a 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.642+0000 7fc0037fe640 1 -- 10.20.193.78:0/379481868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc004127630 con 0x7fc00406c890 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc0037fe640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc00406c890 msgr2=0x7fc0041cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc0037fe640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc00406c890 0x7fc0041cf350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbff0002410 tx=0x7fbff00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0046070 con 0x7fc00406c890 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: dump: 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff0041660 con 0x7fc00406c890 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc00a276640 5 monclient: authenticate success, global_id 174138 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff003c040 con 0x7fc00406c890 2026-02-17T05:59:59.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fbfdffff640 10 monclient: dump: 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc001ffb640 4 set_mon_vals no callback set 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc00a276640 10 monclient: _renew_subs 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc00a276640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc004127910 con 0x7fc00406c890 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc00a276640 10 monclient: _renew_subs 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc00a276640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.644+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00409d810 con 0x7fc00406c890 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fbff0041a30 con 0x7fc00406c890 2026-02-17T05:59:59.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc00a276640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc00a276640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124a40 0x7fc004071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc00a276640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc004127d40 con 0x7fc004124a40 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124a40 0x7fc004071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 10 monclient: get_auth_request con 0x7fc004124a40 auth_method 0 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fbfdffff640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbfd403deb0 0x7fbfd4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(496..496 src has 335..496) ==== 4783+0+0 (secure 0 0 0) 0x7fbff00781b0 con 0x7fc00406c890 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 10 monclient(hunting): handle_auth_done global_id 174141 payload 824 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc002ffd640 1 -- 10.20.193.78:0/379481868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc00409bf40 con 0x7fc004124a40 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc003fff640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbfd403deb0 0x7fbfd4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.645+0000 7fc003fff640 10 monclient: get_auth_request con 0x7fbfd403deb0 auth_method 0 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fc002ffd640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124a40 msgr2=0x7fc004071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fc002ffd640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124a40 0x7fc004071420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbff400f770 tx=0x7fbff400fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff400fe40 con 0x7fc004124a40 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 10 monclient: dump: 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.657 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fbff4018070 con 0x7fc004124a40 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124a40 msgr2=0x7fc004071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124a40 0x7fc004071420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbff400f770 tx=0x7fbff400fc40 comp rx=0 tx=0).stop 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124a40 msgr2=0x7fc004071420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.647+0000 7fbfdffff640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124a40 0x7fc004071420 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbff400f770 tx=0x7fbff400fc40 comp rx=0 tx=0).stop 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.648+0000 7fc003fff640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbfd403deb0 0x7fbfd4040370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbff8003120 tx=0x7fbff8002eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc00a276640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc00a276640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124e20 0x7fc0041da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc00a276640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0041261e0 con 0x7fc004124e20 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124e20 0x7fc0041da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 10 monclient: get_auth_request con 0x7fc004124e20 auth_method 0 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 10 monclient(hunting): handle_auth_done global_id 174144 payload 824 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.651+0000 7fc002ffd640 1 -- 10.20.193.78:0/379481868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc004135270 con 0x7fc004124e20 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.653+0000 7fc002ffd640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124e20 msgr2=0x7fc0041da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.653+0000 7fc002ffd640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124e20 0x7fc0041da560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbff400f770 tx=0x7fbff40026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.653+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff400fe40 con 0x7fc004124e20 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.653+0000 7fbfdffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.653+0000 7fbfdffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.653+0000 7fbfdffff640 10 monclient: dump: 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7fbff400fe40 con 0x7fc004124e20 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fbfdffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fbfdffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:59.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124e20 msgr2=0x7fc0041da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fbfdffff640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124e20 0x7fc0041da560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbff400f770 tx=0x7fbff40026e0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fbfdffff640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124e20 msgr2=0x7fc0041da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fbfdffff640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc004124e20 0x7fc0041da560 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbff400f770 tx=0x7fbff40026e0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 164, 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 14570, 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:59.660 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.661 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.662 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:59.663 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:59.664 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:59.665 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:59.666 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbfd403deb0 msgr2=0x7fbfd4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbfd403deb0 0x7fbfd4040370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbff8003120 tx=0x7fbff8002eb0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 10 monclient: shutdown 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc00406c890 msgr2=0x7fc0041cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc00406c890 0x7fc0041cf350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbff0002410 tx=0x7fbff00047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc003fff640 1 -- 10.20.193.78:0/379481868 reap_dead start 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 shutdown_connections 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 --2- 10.20.193.78:0/379481868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc00406c890 0x7fc0041cf350 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 >> 10.20.193.78:0/379481868 conn(0x7fc0040a6620 msgr2=0x7fc0040a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 shutdown_connections 2026-02-17T05:59:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.654+0000 7fc00a276640 1 -- 10.20.193.78:0/379481868 wait complete. 2026-02-17T05:59:59.667 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:59.667 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T05:59:59.667 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T05:59:59.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.734+0000 7fe5ce8fa640 10 monclient: get_monmap_and_config 2026-02-17T05:59:59.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.734+0000 7fe5ce8fa640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.734+0000 7fe5ce8fa640 10 monclient: monmap: 2026-02-17T05:59:59.735 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:59.736029+0000 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:59.736029+0000 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:59.736 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 1 Processor -- start 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 1 -- start start 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient: init 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c806c890 0x7fe5c806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient(hunting): picked mon.noname-b con 0x7fe5c806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fe5c809c4a0 0x7fe5c806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient(hunting): picked mon.noname-c con 0x7fe5c809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe5c8134df0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c806daf0 con 0x7fe5c8134df0 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c80a0880 con 0x7fe5c809c4a0 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c806d6f0 con 0x7fe5c806c890 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5ce8fa640 10 monclient(hunting): authenticate will time out at 1445.289644s 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c806c890 0x7fe5c806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c806c890 0x7fe5c806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36606/0 (socket says 10.20.193.78:36606) 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 1 -- 10.20.193.78:0/2888755078 learned_addr learned my addr 10.20.193.78:0/2888755078 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient(hunting): get_auth_request con 0x7fe5c806c890 auth_method 0 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c77fe640 1 --2- 10.20.193.78:0/2888755078 >> v2:10.20.193.115:3300/0 conn(0x7fe5c809c4a0 0x7fe5c806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c77fe640 10 monclient(hunting): get_auth_request con 0x7fe5c809c4a0 auth_method 0 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_done global_id 144413 payload 274 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 1 -- 10.20.193.78:0/2888755078 >> v2:10.20.193.115:3300/0 conn(0x7fe5c809c4a0 msgr2=0x7fe5c806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 1 --2- 10.20.193.78:0/2888755078 >> v2:10.20.193.115:3300/0 conn(0x7fe5c809c4a0 0x7fe5c806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 1 -- 10.20.193.78:0/2888755078 >> v1:10.20.193.115:6789/0 conn(0x7fe5c8134df0 legacy=0x7fe5c8135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 1 monclient: found mon.noname-b 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.735+0000 7fe5c7fff640 1 -- 10.20.193.78:0/2888755078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c809cc80 con 0x7fe5c806c890 2026-02-17T05:59:59.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c7fff640 1 -- 10.20.193.78:0/2888755078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c806c890 msgr2=0x7fe5c806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c7fff640 1 --2- 10.20.193.78:0/2888755078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c806c890 0x7fe5c806cc70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009970 tx=0x7fe5b402ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b23407d4258a7c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 1 -- 10.20.193.78:0/2888755078 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b403c070 con 0x7fe5c806c890 2026-02-17T05:59:59.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T05:59:59.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: dump: 2026-02-17T05:59:59.739 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.739 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 1 -- 10.20.193.78:0/2888755078 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b402fcd0 con 0x7fe5c806c890 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5ce8fa640 5 monclient: authenticate success, global_id 144413 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5ce8fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 1 -- 10.20.193.78:0/2888755078 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4037660 con 0x7fe5c806c890 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5ce8fa640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5ce8fa640 4 set_mon_vals no callback set 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5c6ffd640 10 monclient: dump: 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5ce8fa640 10 monclient: shutdown 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5ce8fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/2888755078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c806c890 msgr2=0x7fe5c806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.738+0000 7fe5ce8fa640 1 --2- 10.20.193.78:0/2888755078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c806c890 0x7fe5c806cc70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009970 tx=0x7fe5b402ee90 comp rx=0 tx=0).stop 2026-02-17T05:59:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/2888755078 shutdown_connections 2026-02-17T05:59:59.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 --2- 10.20.193.78:0/2888755078 >> v2:10.20.193.115:3300/0 conn(0x7fe5c809c4a0 0x7fe5c806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 --2- 10.20.193.78:0/2888755078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c806c890 0x7fe5c806cc70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/2888755078 >> 10.20.193.78:0/2888755078 conn(0x7fe5c80a6620 msgr2=0x7fe5c80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/2888755078 shutdown_connections 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/2888755078 wait complete. 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 10 monclient: monmap: 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 Processor -- start 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 -- start start 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 10 monclient: init 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe5c806c890 0x7fe5c81cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 10 monclient(hunting): picked mon.noname-a con 0x7fe5c806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c809c4a0 0x7fe5c8121860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.739+0000 7fe5ce8fa640 10 monclient(hunting): picked mon.noname-c con 0x7fe5c809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe5c8134df0 0x7fe5c8121da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 10 monclient(hunting): picked mon.noname-b con 0x7fe5c8134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80a01a0 con 0x7fe5c806c890 2026-02-17T05:59:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80a0020 con 0x7fe5c8134df0 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c809be20 con 0x7fe5c809c4a0 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5ce8fa640 10 monclient(hunting): authenticate will time out at 1445.294039s 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c809c4a0 0x7fe5c8121860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c809c4a0 0x7fe5c8121860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36620/0 (socket says 10.20.193.78:36620) 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 -- 10.20.193.78:0/1881740778 learned_addr learned my addr 10.20.193.78:0/1881740778 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient(hunting): get_auth_request con 0x7fe5c809c4a0 auth_method 0 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c7fff640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe5c806c890 0x7fe5c81cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c7fff640 10 monclient(hunting): get_auth_request con 0x7fe5c806c890 auth_method 0 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient(hunting): handle_auth_done global_id 144416 payload 824 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe5c8134df0 msgr2=0x7fe5c8121da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe5c8134df0 0x7fe5c8121da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe5c806c890 msgr2=0x7fe5c81cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe5c806c890 0x7fe5c81cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 monclient: found mon.noname-c 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.740+0000 7fe5c77fe640 1 -- 10.20.193.78:0/1881740778 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c8126130 con 0x7fe5c809c4a0 2026-02-17T05:59:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c77fe640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c809c4a0 msgr2=0x7fe5c8121860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c77fe640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c809c4a0 0x7fe5c8121860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800e7f0 tx=0x7fe5b800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8017070 con 0x7fe5c809c4a0 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: dump: 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b800cae0 con 0x7fe5c809c4a0 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800f040 con 0x7fe5c809c4a0 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c4ff9640 10 monclient: dump: 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5c67fc640 4 set_mon_vals no callback set 2026-02-17T05:59:59.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 5 monclient: authenticate success, global_id 144416 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 10 monclient: _renew_subs 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c8123fc0 con 0x7fe5c809c4a0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 10 monclient: _renew_subs 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c8072660 con 0x7fe5c809c4a0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81250f0 0x7fe5c81222e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.742+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c80a16a0 con 0x7fe5c81250f0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81250f0 0x7fe5c81222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 10 monclient: get_auth_request con 0x7fe5c81250f0 auth_method 0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe5b8012050 con 0x7fe5c809c4a0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_done global_id 144419 payload 824 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c7fff640 1 -- 10.20.193.78:0/1881740778 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5c8074980 con 0x7fe5c81250f0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c4ff9640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe59803deb0 0x7fe598040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(496..496 src has 335..496) ==== 4783+0+0 (secure 0 0 0) 0x7fe5b8052660 con 0x7fe5c809c4a0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5cce70640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe59803deb0 0x7fe598040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.743+0000 7fe5cce70640 10 monclient: get_auth_request con 0x7fe59803deb0 auth_method 0 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c7fff640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81250f0 msgr2=0x7fe5c81222e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c7fff640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81250f0 0x7fe5c81222e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5c809c880 tx=0x7fe5b40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b403c050 con 0x7fe5c81250f0 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 10 monclient: dump: 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fe5b403d040 con 0x7fe5c81250f0 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81250f0 msgr2=0x7fe5c81222e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81250f0 0x7fe5c81222e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5c809c880 tx=0x7fe5b40047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81250f0 msgr2=0x7fe5c81222e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.745+0000 7fe5c4ff9640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81250f0 0x7fe5c81222e0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5c809c880 tx=0x7fe5b40047c0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.746+0000 7fe5cce70640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe59803deb0 0x7fe598040370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc004980 tx=0x7fe5bc003fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5ce8fa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:59.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5ce8fa640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81cdbc0 0x7fe5c81d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5ce8fa640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c806d4b0 con 0x7fe5c81cdbc0 2026-02-17T05:59:59.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81cdbc0 0x7fe5c81d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 10 monclient: get_auth_request con 0x7fe5c81cdbc0 auth_method 0 2026-02-17T05:59:59.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 10 monclient(hunting): handle_auth_done global_id 144422 payload 824 2026-02-17T05:59:59.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.748+0000 7fe5c7fff640 1 -- 10.20.193.78:0/1881740778 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe5c80744d0 con 0x7fe5c81cdbc0 2026-02-17T05:59:59.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c7fff640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81cdbc0 msgr2=0x7fe5c81d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c7fff640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81cdbc0 0x7fe5c81d8330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4002410 tx=0x7fe5b4009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40045a0 con 0x7fe5c81cdbc0 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 10 monclient: dump: 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fe5b40045a0 con 0x7fe5c81cdbc0 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81cdbc0 msgr2=0x7fe5c81d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81cdbc0 0x7fe5c81d8330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4002410 tx=0x7fe5b4009340 comp rx=0 tx=0).stop 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81cdbc0 msgr2=0x7fe5c81d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:59.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5c4ff9640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c81cdbc0 0x7fe5c81d8330 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4002410 tx=0x7fe5b4009340 comp rx=0 tx=0).stop 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 164, 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 183286, 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.753 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.754 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:59.755 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.756 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:59.757 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe59803deb0 msgr2=0x7fe598040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe59803deb0 0x7fe598040370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc004980 tx=0x7fe5bc003fc0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 10 monclient: shutdown 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c809c4a0 msgr2=0x7fe5c8121860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 1 --2- 10.20.193.78:0/1881740778 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe5c809c4a0 0x7fe5c8121860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800e7f0 tx=0x7fe5b800ecc0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5cce70640 1 -- 10.20.193.78:0/1881740778 reap_dead start 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5cce70640 1 -- 10.20.193.78:0/1881740778 reap_dead start 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 shutdown_connections 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 >> 10.20.193.78:0/1881740778 conn(0x7fe5c80a6620 msgr2=0x7fe5c8135bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 shutdown_connections 2026-02-17T05:59:59.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.751+0000 7fe5ce8fa640 1 -- 10.20.193.78:0/1881740778 wait complete. 2026-02-17T05:59:59.758 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:59.758 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T05:59:59.758 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T05:59:59.793 INFO:tasks.rados.rados.0.trial115.stdout:2330: done (3 left) 2026-02-17T05:59:59.793 INFO:tasks.rados.rados.0.trial115.stdout:2331: write oid 8 current snap is 236 2026-02-17T05:59:59.793 INFO:tasks.rados.rados.0.trial115.stdout:2331: seq_num 828 ranges {716196=472296,1769561=646611,3121885=793109} 2026-02-17T05:59:59.797 INFO:tasks.rados.rados.0.trial115.stdout:2331: writing trial11545420-8 from 716196 to 1188492 tid 1 2026-02-17T05:59:59.802 INFO:tasks.rados.rados.0.trial115.stdout:2331: writing trial11545420-8 from 1769561 to 2416172 tid 2 2026-02-17T05:59:59.807 INFO:tasks.rados.rados.0.trial115.stdout:2331: writing trial11545420-8 from 3121885 to 3914994 tid 3 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2332: read oid 43 snap -1 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2332: expect (ObjNum 773 snap 219 seq_num 773) 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2333: delete oid 13 current snap is 236 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2327: finishing write tid 3 to trial11545420-7 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2327: finishing write tid 4 to trial11545420-7 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2327: finishing write tid 5 to trial11545420-7 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2327: oid 7 updating version 0 to 966 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2327: oid 7 version 966 is already newer than 964 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2327: oid 7 updating version 966 to 967 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2327: oid 7 version 967 is already newer than 965 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 967 (ObjNum 827 snap 236 seq_num 827) dirty exists 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2327: left oid 7 (ObjNum 827 snap 236 seq_num 827) 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2328: finishing copy_from racing read to trial11545420-21 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2328: finishing copy_from to trial11545420-21 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 963 (ObjNum 813 snap 234 seq_num 813) dirty exists 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2329: finishing copy_from racing read to trial11545420-35 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2329: finishing copy_from to trial11545420-35 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1117 (ObjNum 810 snap 234 seq_num 810) dirty exists 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2331: finishing write tid 1 to trial11545420-8 2026-02-17T05:59:59.808 INFO:tasks.rados.rados.0.trial115.stdout:2331: finishing write tid 2 to trial11545420-8 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: finishing write tid 3 to trial11545420-8 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: finishing write tid 4 to trial11545420-8 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: finishing write tid 5 to trial11545420-8 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: oid 8 updating version 0 to 286 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: oid 8 version 286 is already newer than 285 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: oid 8 version 286 is already newer than 281 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: oid 8 version 286 is already newer than 284 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 286 (ObjNum 828 snap 236 seq_num 828) dirty exists 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: left oid 8 (ObjNum 828 snap 236 seq_num 828) 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2327: done (5 left) 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2328: done (4 left) 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2329: done (3 left) 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2331: done (2 left) 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2333: done (1 left) 2026-02-17T05:59:59.810 INFO:tasks.rados.rados.0.trial115.stdout:2334: snap_remove snap 232 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient: get_monmap_and_config 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient: monmap: 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:59.827404+0000 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:59.827404+0000 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:59.827 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 1 Processor -- start 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 1 -- start start 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient: init 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6ddc06c890 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6ddc09c4a0 0x7f6ddc13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient(hunting): picked mon.noname-a con 0x7f6ddc09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6ddc06d380 0x7f6ddc13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient(hunting): picked mon.noname-c con 0x7f6ddc06d380 addr v2:10.20.193.115:3300/0 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc06daf0 con 0x7f6ddc06c890 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc0a0880 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc09cc80 con 0x7f6ddc06d380 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.826+0000 7f6de1953640 10 monclient(hunting): authenticate will time out at 1445.380931s 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6ddc09c4a0 0x7f6ddc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6ddc09c4a0 0x7f6ddc13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47330/0 (socket says 10.20.193.78:47330) 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 1 -- 10.20.193.78:0/24203832 learned_addr learned my addr 10.20.193.78:0/24203832 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient(hunting): get_auth_request con 0x7f6ddc09c4a0 auth_method 0 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient(hunting): handle_auth_done global_id 174150 payload 274 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 1 -- 10.20.193.78:0/24203832 >> v2:10.20.193.115:3300/0 conn(0x7f6ddc06d380 msgr2=0x7f6ddc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 1 --2- 10.20.193.78:0/24203832 >> v2:10.20.193.115:3300/0 conn(0x7f6ddc06d380 0x7f6ddc13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 1 -- 10.20.193.78:0/24203832 >> v1:10.20.193.115:6789/0 conn(0x7f6ddc06c890 legacy=0x7f6ddc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 1 monclient: found mon.noname-a 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:59.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.827+0000 7f6dda7fc640 1 -- 10.20.193.78:0/24203832 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc09cf60 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dda7fc640 1 -- 10.20.193.78:0/24203832 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 msgr2=0x7f6ddc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dda7fc640 1 --2- 10.20.193.78:0/24203832 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc13b3a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc06dcc0 tx=0x7f6dc402ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6213d38cfefdfa9b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 1 -- 10.20.193.78:0/24203832 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc403c070 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: dump: 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.830 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 1 -- 10.20.193.78:0/24203832 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc402fd70 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6de1953640 5 monclient: authenticate success, global_id 174150 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 1 -- 10.20.193.78:0/24203832 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4037660 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6dd9ffb640 10 monclient: dump: 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6de1953640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:59.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.829+0000 7f6de1953640 4 set_mon_vals no callback set 2026-02-17T05:59:59.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 10 monclient: shutdown 2026-02-17T05:59:59.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 -- 10.20.193.78:0/24203832 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 msgr2=0x7f6ddc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 --2- 10.20.193.78:0/24203832 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc13b3a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc06dcc0 tx=0x7f6dc402ee10 comp rx=0 tx=0).stop 2026-02-17T05:59:59.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 -- 10.20.193.78:0/24203832 shutdown_connections 2026-02-17T05:59:59.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 --2- 10.20.193.78:0/24203832 >> v2:10.20.193.115:3300/0 conn(0x7f6ddc06d380 0x7f6ddc13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 --2- 10.20.193.78:0/24203832 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc13b3a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 -- 10.20.193.78:0/24203832 >> 10.20.193.78:0/24203832 conn(0x7f6ddc0a6620 msgr2=0x7f6ddc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 -- 10.20.193.78:0/24203832 shutdown_connections 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 -- 10.20.193.78:0/24203832 wait complete. 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 10 monclient: monmap: 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 Processor -- start 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 1 -- start start 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.830+0000 7f6de1953640 10 monclient: init 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ddc06c890 0x7f6ddc11f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient(hunting): picked mon.noname-c con 0x7f6ddc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc06d380 0x7f6ddc11f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient(hunting): picked mon.noname-b con 0x7f6ddc06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc11fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient(hunting): picked mon.noname-a con 0x7f6ddc09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc09bfa0 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc09be20 con 0x7f6ddc06d380 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc13de30 con 0x7f6ddc06c890 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6de1953640 10 monclient(hunting): authenticate will time out at 1445.385174s 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc11fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddaffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ddc06c890 0x7f6ddc11f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc11fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47342/0 (socket says 10.20.193.78:47342) 2026-02-17T05:59:59.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 -- 10.20.193.78:0/1499458874 learned_addr learned my addr 10.20.193.78:0/1499458874 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient(hunting): get_auth_request con 0x7f6ddc09c4a0 auth_method 0 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddaffd640 10 monclient(hunting): get_auth_request con 0x7f6ddc06c890 auth_method 0 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient(hunting): handle_auth_done global_id 174153 payload 824 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ddc06c890 msgr2=0x7f6ddc11f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ddc06c890 0x7f6ddc11f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6dda7fc640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc06d380 0x7f6ddc11f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc06d380 msgr2=0x7f6ddc11f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc06d380 0x7f6ddc11f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 monclient: found mon.noname-a 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6ddb7fe640 1 -- 10.20.193.78:0/1499458874 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc06f7a0 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6dda7fc640 10 monclient: get_auth_request con 0x7f6ddc06d380 auth_method 0 2026-02-17T05:59:59.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.831+0000 7f6dda7fc640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc06d380 0x7f6ddc11f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6ddb7fe640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 msgr2=0x7f6ddc11fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6ddb7fe640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc11fb70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00b9e0 tx=0x7f6dcc00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00c8b0 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: dump: 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6de1953640 5 monclient: authenticate success, global_id 174153 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc010070 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6de1953640 10 monclient: _renew_subs 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6de1953640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc13d9f0 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00ccb0 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6dd97fa640 4 set_mon_vals no callback set 2026-02-17T05:59:59.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6db3fff640 10 monclient: dump: 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6de1953640 10 monclient: _renew_subs 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6de1953640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.833+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc122ac0 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6de1953640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6de1953640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc1200b0 0x7f6ddc070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6de1953640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc120df0 con 0x7f6ddc1200b0 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc1200b0 0x7f6ddc070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 10 monclient: get_auth_request con 0x7f6ddc1200b0 auth_method 0 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6dcc004070 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_done global_id 134419 payload 824 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6ddaffd640 1 -- 10.20.193.78:0/1499458874 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ddc06cc70 con 0x7f6ddc1200b0 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6db3fff640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dac03dee0 0x7f6dac0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(497..497 src has 335..497) ==== 4811+0+0 (secure 0 0 0) 0x7f6dcc052600 con 0x7f6ddc09c4a0 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6dda7fc640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dac03dee0 0x7f6dac0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.834+0000 7f6dda7fc640 10 monclient: get_auth_request con 0x7f6dac03dee0 auth_method 0 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.836+0000 7f6ddaffd640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc1200b0 msgr2=0x7f6ddc070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.836+0000 7f6ddaffd640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc1200b0 0x7f6ddc070f10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000f770 tx=0x7f6dd000fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0018030 con 0x7f6ddc1200b0 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 10 monclient: dump: 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6dd000dbc0 con 0x7f6ddc1200b0 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc1200b0 msgr2=0x7f6ddc070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc1200b0 0x7f6ddc070f10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000f770 tx=0x7f6dd000fc40 comp rx=0 tx=0).stop 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc1200b0 msgr2=0x7f6ddc070f10 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6db3fff640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc1200b0 0x7f6ddc070f10 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000f770 tx=0x7f6dd000fc40 comp rx=0 tx=0).stop 2026-02-17T05:59:59.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.837+0000 7f6dda7fc640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dac03dee0 0x7f6dac0403a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4036000 tx=0x7f6dc4033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6de1953640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6de1953640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc071450 0x7f6ddc071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6de1953640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc120490 con 0x7f6ddc071450 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc071450 0x7f6ddc071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 10 monclient: get_auth_request con 0x7f6ddc071450 auth_method 0 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 10 monclient(hunting): handle_auth_done global_id 134422 payload 824 2026-02-17T05:59:59.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.840+0000 7f6ddaffd640 1 -- 10.20.193.78:0/1499458874 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6ddc121670 con 0x7f6ddc071450 2026-02-17T05:59:59.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6ddaffd640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc071450 msgr2=0x7f6ddc071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6ddaffd640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc071450 0x7f6ddc071860 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000f770 tx=0x7f6dd0003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0005260 con 0x7f6ddc071450 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 10 monclient: dump: 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f6dd0005260 con 0x7f6ddc071450 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc071450 msgr2=0x7f6ddc071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc071450 0x7f6ddc071860 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000f770 tx=0x7f6dd0003860 comp rx=0 tx=0).stop 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc071450 msgr2=0x7f6ddc071860 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6db3fff640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ddc071450 0x7f6ddc071860 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000f770 tx=0x7f6dd0003860 comp rx=0 tx=0).stop 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T05:59:59.844 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 164, 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 227562, 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T05:59:59.845 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.846 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T05:59:59.847 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T05:59:59.848 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dac03dee0 msgr2=0x7f6dac0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6dac03dee0 0x7f6dac0403a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4036000 tx=0x7f6dc4033000 comp rx=0 tx=0).stop 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 10 monclient: shutdown 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 msgr2=0x7f6ddc11fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc11fb70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00b9e0 tx=0x7f6dcc00beb0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6ddb7fe640 1 -- 10.20.193.78:0/1499458874 reap_dead start 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 shutdown_connections 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 --2- 10.20.193.78:0/1499458874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ddc09c4a0 0x7f6ddc11fb70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 >> 10.20.193.78:0/1499458874 conn(0x7f6ddc0a6620 msgr2=0x7f6ddc13c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 shutdown_connections 2026-02-17T05:59:59.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.843+0000 7f6de1953640 1 -- 10.20.193.78:0/1499458874 wait complete. 2026-02-17T05:59:59.850 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T05:59:59.850 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T05:59:59.850 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T05:59:59.850 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T05:59:59.850 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T05:59:59.850 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T05:59:59.851 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T05:59:59.860 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T05:59:59.860 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T05:59:59.860 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T05:59:59.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient: get_monmap_and_config 2026-02-17T05:59:59.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient: monmap: 2026-02-17T05:59:59.966 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.966 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.966 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:59:59.966276+0000 2026-02-17T05:59:59.966 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:59:59.966276+0000 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 1 Processor -- start 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 1 -- start start 2026-02-17T05:59:59.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient: init 2026-02-17T05:59:59.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2026-02-17T05:59:59.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f149006c890 0x7f149006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient(hunting): picked mon.noname-a con 0x7f149006c890 addr v2:10.20.193.78:3300/0 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f149009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f149006d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1490135c10 con 0x7f149006d1b0 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149006daf0 con 0x7f149009c4a0 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f14900a0880 con 0x7f149006c890 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f148ed76640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f149006c890 msgr2=0x7f149006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f149507d640 10 monclient(hunting): authenticate will time out at 1445.519791s 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f148ed76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f149006c890 0x7f149006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.965+0000 7f148f577640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f149006d1b0 legacy=0x7f1490135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.966+0000 7f148e575640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f149009c4a0 0x7f1490134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:58586/0 (socket says 10.20.193.78:58586) 2026-02-17T05:59:59.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.966+0000 7f148e575640 1 -- 10.20.193.78:0/190443338 learned_addr learned my addr 10.20.193.78:0/190443338 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T05:59:59.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148e575640 1 -- 10.20.193.78:0/190443338 >> v1:10.20.193.115:6789/0 conn(0x7f149009c4a0 legacy=0x7f1490134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268301231 0 0) 0x7f149006daf0 con 0x7f149009c4a0 2026-02-17T05:59:59.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): _negotiate 2026-02-17T05:59:59.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): my global_id is 134425 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f147c003620 con 0x7f149009c4a0 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3789391770 0 0) 0x7f147c003620 con 0x7f149009c4a0 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 >> v2:10.20.193.78:3300/0 conn(0x7f149006c890 msgr2=0x7f149006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 --2- 10.20.193.78:0/190443338 >> v2:10.20.193.78:3300/0 conn(0x7f149006c890 0x7f149006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 >> v1:10.20.193.78:6789/0 conn(0x7f149006d1b0 legacy=0x7f1490135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 monclient: found mon.noname-c-legacy 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149009cc80 con 0x7f149009c4a0 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f1478003270 con 0x7f149009c4a0 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: dump: 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T05:59:59.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 >> v1:10.20.193.115:6789/0 conn(0x7f149009c4a0 legacy=0x7f1490134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f147c003ab0 0x7f147c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): picked mon.b con 0x7f147c003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c004900 0x7f147c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): picked mon.a con 0x7f147c004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f147c005470 0x7f147c00bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): picked mon.c con 0x7f147c005470 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f147c00c390 con 0x7f147c004900 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f147c00c540 con 0x7f147c003ab0 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f147c00c740 con 0x7f147c005470 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148dd74640 10 monclient(hunting): _finish_auth 0 2026-02-17T05:59:59.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148e575640 1 -- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c004900 msgr2=0x7f147c004d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.968+0000 7f148e575640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c004900 0x7f147c004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f147c005470 0x7f147c00bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient(hunting): get_auth_request con 0x7f147c005470 auth_method 0 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148ed76640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f147c003ab0 0x7f147c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148ed76640 10 monclient(hunting): get_auth_request con 0x7f147c003ab0 auth_method 0 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148ed76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient(hunting): handle_auth_done global_id 134425 payload 293 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 1 -- 10.20.193.78:0/190443338 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f147c003ab0 msgr2=0x7f147c003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T05:59:59.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f147c003ab0 0x7f147c003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 1 -- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c004900 msgr2=0x7f147c004d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c004900 0x7f147c004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 1 monclient: found mon.c 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 1 -- 10.20.193.78:0/190443338 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147c003620 con 0x7f147c005470 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f149507d640 5 monclient: authenticate success, global_id 134425 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f149507d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148ed76640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f147c003ab0 0x7f147c003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.969+0000 7f148f577640 1 -- 10.20.193.78:0/190443338 reap_dead start 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148f577640 1 -- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f147c005470 msgr2=0x7f147c00bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148f577640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f147c005470 0x7f147c00bae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1480009e10 tx=0x7f148002f600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7260492c57dbb7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480008050 con 0x7f147c005470 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 10 monclient: dump: 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f149507d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T05:59:59.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1480037650 con 0x7f147c005470 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 1 -- 10.20.193.78:0/190443338 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480009a10 con 0x7f147c005470 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f148dd74640 10 monclient: dump: 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f149507d640 10 monclient: get_monmap_and_config success 2026-02-17T05:59:59.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.971+0000 7f149507d640 4 set_mon_vals no callback set 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 10 monclient: shutdown 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 -- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f147c005470 msgr2=0x7f147c00bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f147c005470 0x7f147c00bae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1480009e10 tx=0x7f148002f600 comp rx=0 tx=0).stop 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 -- 10.20.193.78:0/190443338 shutdown_connections 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 --2- 10.20.193.78:0/190443338 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f147c005470 0x7f147c00bae0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 -- 10.20.193.78:0/190443338 >> 10.20.193.78:0/190443338 conn(0x7f14900a6620 msgr2=0x7f14900a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 -- 10.20.193.78:0/190443338 shutdown_connections 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 -- 10.20.193.78:0/190443338 wait complete. 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 10 monclient: build_initial_monmap 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 10 monclient: monmap: 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.985 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 Processor -- start 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 1 -- start start 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.972+0000 7f149507d640 10 monclient: init 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient: _reopen_session rank -1 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c005470 0x7f14901cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient(hunting): picked mon.noname-a con 0x7f147c005470 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14901cd870 0x7f1490123480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient(hunting): picked mon.noname-c con 0x7f14901cd870 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14901cdcf0 0x7f14901239c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient(hunting): picked mon.noname-b con 0x7f14901cdcf0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14900a0020 con 0x7f147c005470 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f149006d640 con 0x7f14901cdcf0 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient(hunting): start opening mon connection 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f14900a01a0 con 0x7f14901cd870 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient(hunting): _renew_subs 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f149507d640 10 monclient(hunting): authenticate will time out at 1445.527228s 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148ed76640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c005470 msgr2=0x7f14901cd330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148ed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c005470 0x7f14901cd330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14901cd870 0x7f1490123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14901cd870 0x7f1490123480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36668/0 (socket says 10.20.193.78:36668) 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 -- 10.20.193.78:0/1312264265 learned_addr learned my addr 10.20.193.78:0/1312264265 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient(hunting): get_auth_request con 0x7f14901cd870 auth_method 0 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T05:59:59.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient(hunting): handle_auth_done global_id 144428 payload 824 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 -- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14901cdcf0 msgr2=0x7f14901239c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14901cdcf0 0x7f14901239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 -- 10.20.193.78:0/1312264265 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c005470 msgr2=0x7f14901cd330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c005470 0x7f14901cd330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient: _finish_hunting 0 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 monclient: found mon.noname-c 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.973+0000 7f148e575640 1 -- 10.20.193.78:0/1312264265 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1490127410 con 0x7f14901cd870 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f148e575640 1 -- 10.20.193.78:0/1312264265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14901cd870 msgr2=0x7f1490123480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f148e575640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14901cd870 0x7f1490123480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f147800c7f0 tx=0x7f147800ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 1 -- 10.20.193.78:0/1312264265 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478018070 con 0x7f14901cd870 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: dump: 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: _finish_auth 0 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: _check_auth_tickets 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 1 -- 10.20.193.78:0/1312264265 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14780070d0 con 0x7f14901cd870 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f149507d640 5 monclient: authenticate success, global_id 144428 2026-02-17T05:59:59.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f149507d640 10 monclient: _renew_subs 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f149507d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1490136ea0 con 0x7f14901cd870 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 1 -- 10.20.193.78:0/1312264265 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780074c0 con 0x7f14901cd870 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f146f7fe640 10 monclient: dump: 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T05:59:59.990 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f148d573640 4 set_mon_vals no callback set 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f149507d640 10 monclient: _renew_subs 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f149507d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.975+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1490126e50 con 0x7f14901cd870 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f146f7fe640 1 -- 10.20.193.78:0/1312264265 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1478007660 con 0x7f14901cd870 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f149507d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f149507d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f149507d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149009beb0 con 0x7f14901cd870 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f146f7fe640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f145c03deb0 0x7f145c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f146f7fe640 1 -- 10.20.193.78:0/1312264265 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(497..497 src has 335..497) ==== 4811+0+0 (secure 0 0 0) 0x7f1478052110 con 0x7f14901cd870 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f148ed76640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f145c03deb0 0x7f145c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T05:59:59.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.976+0000 7f148ed76640 10 monclient: get_auth_request con 0x7f145c03deb0 auth_method 0 2026-02-17T05:59:59.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.978+0000 7f146f7fe640 1 -- 10.20.193.78:0/1312264265 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1478050af0 con 0x7f14901cd870 2026-02-17T05:59:59.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.978+0000 7f146f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T05:59:59.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.978+0000 7f146f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T05:59:59.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T05:59:59.979+0000 7f148ed76640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f145c03deb0 0x7f145c040370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1484003a80 tx=0x7f14840092b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:00.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.091+0000 7f149507d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:00.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.091+0000 7f149507d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:00.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.091+0000 7f149507d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:00.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.091+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1490072530 con 0x7f14901cd870 2026-02-17T06:00:00.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f146f7fe640 1 -- 10.20.193.78:0/1312264265 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f147801c3b0 con 0x7f14901cd870 2026-02-17T06:00:00.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f146f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:00.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f146f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:00.094 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":164,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:00.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f145c03deb0 msgr2=0x7f145c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f149507d640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f145c03deb0 0x7f145c040370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1484003a80 tx=0x7f14840092b0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f149507d640 10 monclient: shutdown 2026-02-17T06:00:00.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f149507d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:00.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14901cd870 msgr2=0x7f1490123480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:00.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.092+0000 7f149507d640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14901cd870 0x7f1490123480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f147800c7f0 tx=0x7f147800ccc0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.093+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 shutdown_connections 2026-02-17T06:00:00.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.093+0000 7f149507d640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f145c03deb0 0x7f145c040370 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.093+0000 7f149507d640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14901cdcf0 0x7f14901239c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.093+0000 7f149507d640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14901cd870 0x7f1490123480 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.093+0000 7f149507d640 1 --2- 10.20.193.78:0/1312264265 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f147c005470 0x7f14901cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:00.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.093+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 >> 10.20.193.78:0/1312264265 conn(0x7f14900a6620 msgr2=0x7f1490070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:00.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.093+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 shutdown_connections 2026-02-17T06:00:00.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:00.093+0000 7f149507d640 1 -- 10.20.193.78:0/1312264265 wait complete. 2026-02-17T06:00:03.102 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:03.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient: get_monmap_and_config 2026-02-17T06:00:03.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient: build_initial_monmap 2026-02-17T06:00:03.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient: monmap: 2026-02-17T06:00:03.174 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:03.174 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:03.174 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:03.171345+0000 2026-02-17T06:00:03.174 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:03.171345+0000 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 1 Processor -- start 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 1 -- start start 2026-02-17T06:00:03.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient: init 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient: _add_conns ranks=[0,4,2,3,1] 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa4fc06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa4fc09c4a0 0x7fa4fc13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient(hunting): picked mon.noname-c con 0x7fa4fc09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06d380 0x7fa4fc13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient(hunting): picked mon.noname-b con 0x7fa4fc06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4fc06daf0 con 0x7fa4fc06c890 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa4fc0a0880 con 0x7fa4fc09c4a0 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa4fc09cc80 con 0x7fa4fc06d380 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa502ff8640 10 monclient(hunting): authenticate will time out at 1448.724898s 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa500d6d640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7fa4fc06c890 legacy=0x7fa4fc06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.170+0000 7fa50156e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06d380 0x7fa4fc13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06d380 0x7fa4fc13d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36672/0 (socket says 10.20.193.78:36672) 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 1 -- 10.20.193.78:0/1268928473 learned_addr learned my addr 10.20.193.78:0/1268928473 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient(hunting): get_auth_request con 0x7fa4fc06d380 auth_method 0 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa4f3fff640 1 --2- 10.20.193.78:0/1268928473 >> v2:10.20.193.115:3300/0 conn(0x7fa4fc09c4a0 0x7fa4fc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa4f3fff640 10 monclient(hunting): get_auth_request con 0x7fa4fc09c4a0 auth_method 0 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa4f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa4f3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa4f3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient(hunting): handle_auth_done global_id 144431 payload 274 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 1 -- 10.20.193.78:0/1268928473 >> v2:10.20.193.115:3300/0 conn(0x7fa4fc09c4a0 msgr2=0x7fa4fc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 1 --2- 10.20.193.78:0/1268928473 >> v2:10.20.193.115:3300/0 conn(0x7fa4fc09c4a0 0x7fa4fc13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 1 -- 10.20.193.78:0/1268928473 >> v1:10.20.193.78:6789/0 conn(0x7fa4fc06c890 legacy=0x7fa4fc06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient: _finish_hunting 0 2026-02-17T06:00:03.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 1 monclient: found mon.noname-b 2026-02-17T06:00:03.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:03.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:03.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.171+0000 7fa50156e640 1 -- 10.20.193.78:0/1268928473 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc09cf60 con 0x7fa4fc06d380 2026-02-17T06:00:03.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa50156e640 1 -- 10.20.193.78:0/1268928473 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06d380 msgr2=0x7fa4fc13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:03.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa50156e640 1 --2- 10.20.193.78:0/1268928473 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06d380 0x7fa4fc13d8f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009e50 tx=0x7fa4f802f260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a744f55ca37ffb01 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 1 -- 10.20.193.78:0/1268928473 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80373f0 con 0x7fa4fc06d380 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: dump: 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:03.187 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 1 -- 10.20.193.78:0/1268928473 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f8037590 con 0x7fa4fc06d380 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa502ff8640 5 monclient: authenticate success, global_id 144431 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 1 -- 10.20.193.78:0/1268928473 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8037960 con 0x7fa4fc06d380 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa502ff8640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa502ff8640 4 set_mon_vals no callback set 2026-02-17T06:00:03.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.173+0000 7fa4f37fe640 10 monclient: dump: 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:03.189 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 10 monclient: shutdown 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 -- 10.20.193.78:0/1268928473 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06d380 msgr2=0x7fa4fc13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 --2- 10.20.193.78:0/1268928473 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06d380 0x7fa4fc13d8f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009e50 tx=0x7fa4f802f260 comp rx=0 tx=0).stop 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 -- 10.20.193.78:0/1268928473 shutdown_connections 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 --2- 10.20.193.78:0/1268928473 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06d380 0x7fa4fc13d8f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 --2- 10.20.193.78:0/1268928473 >> v2:10.20.193.115:3300/0 conn(0x7fa4fc09c4a0 0x7fa4fc13b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 -- 10.20.193.78:0/1268928473 >> 10.20.193.78:0/1268928473 conn(0x7fa4fc0a6620 msgr2=0x7fa4fc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 -- 10.20.193.78:0/1268928473 shutdown_connections 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 -- 10.20.193.78:0/1268928473 wait complete. 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 10 monclient: build_initial_monmap 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 10 monclient: monmap: 2026-02-17T06:00:03.190 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 Processor -- start 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 1 -- start start 2026-02-17T06:00:03.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.174+0000 7fa502ff8640 10 monclient: init 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 0x7fa4fc1ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient(hunting): picked mon.noname-c con 0x7fa4fc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4fc06d380 0x7fa4fc1330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient(hunting): picked mon.noname-a con 0x7fa4fc06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4fc09c4a0 0x7fa4fc13a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient(hunting): picked mon.noname-b con 0x7fa4fc09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc09bfa0 con 0x7fa4fc06d380 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.192 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc09be20 con 0x7fa4fc09c4a0 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa4fc13de30 con 0x7fa4fc06c890 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa502ff8640 10 monclient(hunting): authenticate will time out at 1448.729189s 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa4f3fff640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4fc06d380 msgr2=0x7fa4fc1330b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa4f3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4fc06d380 0x7fa4fc1330b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 0x7fa4fc1ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 0x7fa4fc1ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36684/0 (socket says 10.20.193.78:36684) 2026-02-17T06:00:03.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 -- 10.20.193.78:0/225790758 learned_addr learned my addr 10.20.193.78:0/225790758 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient(hunting): get_auth_request con 0x7fa4fc06c890 auth_method 0 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa50156e640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4fc09c4a0 0x7fa4fc13a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa50156e640 10 monclient(hunting): get_auth_request con 0x7fa4fc09c4a0 auth_method 0 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa50156e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:03.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa50156e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa50156e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient(hunting): handle_auth_done global_id 144434 payload 824 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 -- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4fc09c4a0 msgr2=0x7fa4fc13a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4fc09c4a0 0x7fa4fc13a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 -- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4fc06d380 msgr2=0x7fa4fc1330b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4fc06d380 0x7fa4fc1330b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient: _finish_hunting 0 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 monclient: found mon.noname-c 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.175+0000 7fa500d6d640 1 -- 10.20.193.78:0/225790758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc1cdd10 con 0x7fa4fc06c890 2026-02-17T06:00:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa500d6d640 1 -- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 msgr2=0x7fa4fc1ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa500d6d640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 0x7fa4fc1ccdf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00b780 tx=0x7fa4ec00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 1 -- 10.20.193.78:0/225790758 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec005270 con 0x7fa4fc06c890 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: dump: 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:03.196 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: _finish_auth 0 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 1 -- 10.20.193.78:0/225790758 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec010070 con 0x7fa4fc06c890 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa502ff8640 5 monclient: authenticate success, global_id 144434 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa502ff8640 10 monclient: _renew_subs 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa502ff8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc13da60 con 0x7fa4fc06c890 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 1 -- 10.20.193.78:0/225790758 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec005650 con 0x7fa4fc06c890 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f2ffd640 4 set_mon_vals no callback set 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa4f17fa640 10 monclient: dump: 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:03.198 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa502ff8640 10 monclient: _renew_subs 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa502ff8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.177+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc09d810 con 0x7fa4fc06c890 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa502ff8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa502ff8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa502ff8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa4f17fa640 1 -- 10.20.193.78:0/225790758 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa4ec01d030 con 0x7fa4fc06c890 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4fc074980 con 0x7fa4fc06c890 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa4f17fa640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa4cc03de60 0x7fa4cc040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa4f17fa640 1 -- 10.20.193.78:0/225790758 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(497..497 src has 335..497) ==== 4811+0+0 (secure 0 0 0) 0x7fa4ec051d90 con 0x7fa4fc06c890 2026-02-17T06:00:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa4f3fff640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa4cc03de60 0x7fa4cc040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.178+0000 7fa4f3fff640 10 monclient: get_auth_request con 0x7fa4cc03de60 auth_method 0 2026-02-17T06:00:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.180+0000 7fa500d6d640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 0x7fa4fc1ccdf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00b780 tx=0x7fa4ec00bc50 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fa4ec056070 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) 2026-02-17T06:00:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:03.181+0000 7fa4f3fff640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa4cc03de60 0x7fa4cc040320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc13d960 tx=0x7fa4e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.009+0000 7fa4f17fa640 1 -- 10.20.193.78:0/225790758 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa4ec056070 con 0x7fa4fc06c890 2026-02-17T06:00:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.009+0000 7fa4f17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:04.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.009+0000 7fa4f17fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:04.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.120+0000 7fa502ff8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:04.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.120+0000 7fa502ff8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:04.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.120+0000 7fa502ff8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:04.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.120+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa4fc06cc70 con 0x7fa4fc06c890 2026-02-17T06:00:04.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa4f17fa640 1 -- 10.20.193.78:0/225790758 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa4fc13da60 con 0x7fa4fc06c890 2026-02-17T06:00:04.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa4f17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:04.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa4f17fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:04.123 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":164,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:04.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa4cc03de60 msgr2=0x7fa4cc040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:04.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa4cc03de60 0x7fa4cc040320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc13d960 tx=0x7fa4e4008040 comp rx=0 tx=0).stop 2026-02-17T06:00:04.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 10 monclient: shutdown 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 msgr2=0x7fa4fc1ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 0x7fa4fc1ccdf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00b780 tx=0x7fa4ec00bc50 comp rx=0 tx=0).stop 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 shutdown_connections 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa4cc03de60 0x7fa4cc040320 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4fc09c4a0 0x7fa4fc13a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4fc06d380 0x7fa4fc1330b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 --2- 10.20.193.78:0/225790758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4fc06c890 0x7fa4fc1ccdf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 >> 10.20.193.78:0/225790758 conn(0x7fa4fc0a6620 msgr2=0x7fa4fc0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 shutdown_connections 2026-02-17T06:00:04.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:04.121+0000 7fa502ff8640 1 -- 10.20.193.78:0/225790758 wait complete. 2026-02-17T06:00:07.132 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:07.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient: get_monmap_and_config 2026-02-17T06:00:07.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient: build_initial_monmap 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient: monmap: 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:07.200401+0000 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:07.200401+0000 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:07.203 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 1 Processor -- start 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 1 -- start start 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient: init 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa6c006c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa6c009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:07.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient(hunting): picked mon.noname-b con 0x7fa6c006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c006daf0 con 0x7fa6c006c890 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c009cc80 con 0x7fa6c009c4a0 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c00a0880 con 0x7fa6c006d380 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6c530a640 10 monclient(hunting): authenticate will time out at 1452.753934s 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.199+0000 7fa6beffd640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7fa6c006c890 legacy=0x7fa6c006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c013ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59162/0 (socket says 10.20.193.78:59162) 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 1 -- 10.20.193.78:0/4261195643 learned_addr learned my addr 10.20.193.78:0/4261195643 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient(hunting): get_auth_request con 0x7fa6c006d380 auth_method 0 2026-02-17T06:00:07.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient(hunting): handle_auth_done global_id 144437 payload 274 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 1 -- 10.20.193.78:0/4261195643 >> v1:10.20.193.115:6789/0 conn(0x7fa6c009c4a0 legacy=0x7fa6c013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 1 -- 10.20.193.78:0/4261195643 >> v1:10.20.193.78:6789/0 conn(0x7fa6c006c890 legacy=0x7fa6c006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 1 monclient: found mon.noname-b 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:07.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.200+0000 7fa6bf7fe640 1 -- 10.20.193.78:0/4261195643 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c009cf60 con 0x7fa6c006d380 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bf7fe640 1 -- 10.20.193.78:0/4261195643 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 msgr2=0x7fa6c013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bf7fe640 1 --2- 10.20.193.78:0/4261195643 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c013ead0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009e80 tx=0x7fa6b002f250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e97a2f6503a546 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 1 -- 10.20.193.78:0/4261195643 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00373f0 con 0x7fa6c006d380 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: dump: 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: _finish_auth 0 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: _check_auth_tickets 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 1 -- 10.20.193.78:0/4261195643 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b0037590 con 0x7fa6c006d380 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6c530a640 5 monclient: authenticate success, global_id 144437 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 1 -- 10.20.193.78:0/4261195643 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0037960 con 0x7fa6c006d380 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6bdffb640 10 monclient: dump: 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:07.213 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6c530a640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.202+0000 7fa6c530a640 4 set_mon_vals no callback set 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 10 monclient: shutdown 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 -- 10.20.193.78:0/4261195643 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 msgr2=0x7fa6c013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 --2- 10.20.193.78:0/4261195643 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c013ead0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009e80 tx=0x7fa6b002f250 comp rx=0 tx=0).stop 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 -- 10.20.193.78:0/4261195643 shutdown_connections 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 --2- 10.20.193.78:0/4261195643 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c013ead0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 -- 10.20.193.78:0/4261195643 >> 10.20.193.78:0/4261195643 conn(0x7fa6c00a6620 msgr2=0x7fa6c00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 -- 10.20.193.78:0/4261195643 shutdown_connections 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 -- 10.20.193.78:0/4261195643 wait complete. 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 10 monclient: build_initial_monmap 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 10 monclient: monmap: 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 Processor -- start 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 1 -- start start 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.203+0000 7fa6c530a640 10 monclient: init 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa6c006c890 0x7fa6c0135dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient(hunting): picked mon.noname-a con 0x7fa6c006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c0136310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient(hunting): picked mon.noname-c con 0x7fa6c006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6c009c4a0 0x7fa6c011cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient(hunting): picked mon.noname-b con 0x7fa6c009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:07.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c009bfa0 con 0x7fa6c006c890 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c009be20 con 0x7fa6c009c4a0 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c013f010 con 0x7fa6c006d380 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6c530a640 10 monclient(hunting): authenticate will time out at 1452.758078s 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6beffd640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa6c006c890 msgr2=0x7fa6c0135dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6beffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa6c006c890 0x7fa6c0135dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c0136310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c0136310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59176/0 (socket says 10.20.193.78:59176) 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 -- 10.20.193.78:0/3646046954 learned_addr learned my addr 10.20.193.78:0/3646046954 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient(hunting): get_auth_request con 0x7fa6c006d380 auth_method 0 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient(hunting): handle_auth_done global_id 144440 payload 824 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 -- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6c009c4a0 msgr2=0x7fa6c011cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6c009c4a0 0x7fa6c011cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 -- 10.20.193.78:0/3646046954 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa6c006c890 msgr2=0x7fa6c0135dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa6c006c890 0x7fa6c0135dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 monclient: found mon.noname-c 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.204+0000 7fa6be7fc640 1 -- 10.20.193.78:0/3646046954 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c011fa90 con 0x7fa6c006d380 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6be7fc640 1 -- 10.20.193.78:0/3646046954 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 msgr2=0x7fa6c0136310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6be7fc640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c0136310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800c580 tx=0x7fa6a800ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 1 -- 10.20.193.78:0/3646046954 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8012a40 con 0x7fa6c006d380 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: dump: 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:07.215 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 1 -- 10.20.193.78:0/3646046954 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a8017070 con 0x7fa6c006d380 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 1 -- 10.20.193.78:0/3646046954 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8012e20 con 0x7fa6c006d380 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa69f7fe640 10 monclient: dump: 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6bd7fa640 4 set_mon_vals no callback set 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6c530a640 5 monclient: authenticate success, global_id 144440 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6c530a640 10 monclient: _renew_subs 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6c530a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c006fc80 con 0x7fa6c006d380 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6c530a640 10 monclient: _renew_subs 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6c530a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.206+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c009d810 con 0x7fa6c006d380 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa6c530a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa6c530a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa6c530a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c006cc70 con 0x7fa6c006d380 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa69f7fe640 1 -- 10.20.193.78:0/3646046954 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa6a801c460 con 0x7fa6c006d380 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa69f7fe640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa68c03de60 0x7fa68c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa69f7fe640 1 -- 10.20.193.78:0/3646046954 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(497..497 src has 335..497) ==== 4811+0+0 (secure 0 0 0) 0x7fa6a8052740 con 0x7fa6c006d380 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa6beffd640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa68c03de60 0x7fa68c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.207+0000 7fa6beffd640 10 monclient: get_auth_request con 0x7fa68c03de60 auth_method 0 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.210+0000 7fa69f7fe640 1 -- 10.20.193.78:0/3646046954 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa6a8051120 con 0x7fa6c006d380 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.210+0000 7fa69f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.210+0000 7fa69f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:07.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.210+0000 7fa6beffd640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa68c03de60 0x7fa68c040320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa6c013e700 tx=0x7fa6b4009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:07.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.323+0000 7fa6c530a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:07.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.323+0000 7fa6c530a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:07.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.323+0000 7fa6c530a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:07.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.323+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa6c011e640 con 0x7fa6c006d380 2026-02-17T06:00:07.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.323+0000 7fa69f7fe640 1 -- 10.20.193.78:0/3646046954 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa6a8010d70 con 0x7fa6c006d380 2026-02-17T06:00:07.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.323+0000 7fa69f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:07.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.323+0000 7fa69f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:07.324 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":164,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:07.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa68c03de60 msgr2=0x7fa68c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:07.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa68c03de60 0x7fa68c040320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa6c013e700 tx=0x7fa6b4009270 comp rx=0 tx=0).stop 2026-02-17T06:00:07.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 10 monclient: shutdown 2026-02-17T06:00:07.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:07.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 msgr2=0x7fa6c0136310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:07.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c0136310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800c580 tx=0x7fa6a800ca50 comp rx=0 tx=0).stop 2026-02-17T06:00:07.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 shutdown_connections 2026-02-17T06:00:07.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa68c03de60 0x7fa68c040320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa6c009c4a0 0x7fa6c011cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa6c006d380 0x7fa6c0136310 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 --2- 10.20.193.78:0/3646046954 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa6c006c890 0x7fa6c0135dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:07.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 >> 10.20.193.78:0/3646046954 conn(0x7fa6c00a6620 msgr2=0x7fa6c0072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:07.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 shutdown_connections 2026-02-17T06:00:07.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:07.324+0000 7fa6c530a640 1 -- 10.20.193.78:0/3646046954 wait complete. 2026-02-17T06:00:10.334 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:10.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.402+0000 7f91ac832640 10 monclient: get_monmap_and_config 2026-02-17T06:00:10.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.402+0000 7f91ac832640 10 monclient: build_initial_monmap 2026-02-17T06:00:10.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.402+0000 7f91ac832640 10 monclient: monmap: 2026-02-17T06:00:10.406 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:10.406 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:10.406 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:10.403781+0000 2026-02-17T06:00:10.406 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:10.403781+0000 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.402+0000 7f91ac832640 1 Processor -- start 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.402+0000 7f91ac832640 1 -- start start 2026-02-17T06:00:10.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.402+0000 7f91ac832640 10 monclient: init 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient(hunting): picked mon.noname-b con 0x7f91a406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient(hunting): picked mon.noname-c con 0x7f91a406cc70 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 0x7f91a4135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient(hunting): picked mon.noname-a con 0x7f91a4134df0 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f91a4074150 con 0x7f91a4134df0 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:10.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f91a409ef70 con 0x7f91a406cc70 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f91a409f250 con 0x7f91a406c890 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91ac832640 10 monclient(hunting): authenticate will time out at 1455.957296s 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aada8640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 msgr2=0x7f91a4135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aada8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 0x7f91a4135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59178/0 (socket says 10.20.193.78:59178) 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 learned_addr learned my addr 10.20.193.78:0/3684834841 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 10 monclient(hunting): get_auth_request con 0x7f91a406c890 auth_method 0 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 10 monclient(hunting): get_auth_request con 0x7f91a406cc70 auth_method 0 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:00:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:10.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:10.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.403+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aada8640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 msgr2=0x7f91a4135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aada8640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 0x7f91a4135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 10 monclient(hunting): get_auth_request con 0x7f91a406c890 auth_method 0 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:10.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 10 monclient(hunting): get_auth_request con 0x7f91a406cc70 auth_method 0 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:10.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:10.604+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:00:11.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aada8640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 msgr2=0x7f91a4135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:11.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aada8640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 0x7f91a4135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:00:11.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:11.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 10 monclient(hunting): get_auth_request con 0x7f91a406c890 auth_method 0 2026-02-17T06:00:11.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91a9da6640 10 monclient(hunting): get_auth_request con 0x7f91a406cc70 auth_method 0 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91a9da6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91a9da6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:11.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.004+0000 7f91a9da6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:11.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.005+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:11.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.005+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:11.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.005+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:11.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.005+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:00:11.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aada8640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 msgr2=0x7f91a4135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:11.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aada8640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 0x7f91a4135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 10 monclient(hunting): get_auth_request con 0x7f91a406c890 auth_method 0 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.805+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:00:11.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 10 monclient(hunting): get_auth_request con 0x7f91a406cc70 auth_method 0 2026-02-17T06:00:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:11.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:11.806+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:00:13.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 10 monclient(hunting): tick 2026-02-17T06:00:13.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:00:13.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 monclient(hunting): continuing hunt 2026-02-17T06:00:13.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:00:13.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 msgr2=0x7f91a40a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:13.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f91a406c890 0x7f91a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 msgr2=0x7f91a406d050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.115:3300/0 conn(0x7f91a406cc70 0x7f91a406d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 -- 10.20.193.78:0/3684834841 >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 msgr2=0x7f91a4135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 --2- 10.20.193.78:0/3684834841 >> v2:10.20.193.78:3300/0 conn(0x7f91a4134df0 0x7f91a4135240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 10 monclient: _add_conns ranks=[0,1] 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9184003770 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9184007420 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 -- 10.20.193.78:0/3684834841 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a409df40 con 0x7f9184003770 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 1 -- 10.20.193.78:0/3684834841 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f918400c150 con 0x7f9184007420 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f918ffff640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:13.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.403+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> v1:10.20.193.78:6789/0 conn(0x7f9184003770 legacy=0x7f9184005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:00:13.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.405+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> v1:10.20.193.115:6789/0 conn(0x7f9184007420 legacy=0x7f91840098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:13.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:13.604+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> v1:10.20.193.78:6789/0 conn(0x7f9184003770 legacy=0x7f9184005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:00:14.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.004+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> v1:10.20.193.78:6789/0 conn(0x7f9184003770 legacy=0x7f9184005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:00:14.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.798+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 555003166 0 0) 0x7f918400c150 con 0x7f9184007420 2026-02-17T06:00:14.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.798+0000 7f91a95a5640 10 monclient(hunting): _negotiate 2026-02-17T06:00:14.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.798+0000 7f91a95a5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:14.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.798+0000 7f91a95a5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:14.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.798+0000 7f91a95a5640 10 monclient(hunting): my global_id is 134434 2026-02-17T06:00:14.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.798+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9194003620 con 0x7f9184007420 2026-02-17T06:00:14.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1758618357 0 0) 0x7f9194003620 con 0x7f9184007420 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 >> v1:10.20.193.78:6789/0 conn(0x7f9184003770 legacy=0x7f9184005bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: _finish_hunting 0 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918400d350 con 0x7f9184007420 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: _finish_auth 0 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: _check_auth_tickets 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91ac832640 5 monclient: authenticate success, global_id 134434 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91ac832640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f919000f040 con 0x7f9184007420 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: dump: 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T06:00:14.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 >> v1:10.20.193.115:6789/0 conn(0x7f9184007420 legacy=0x7f91840098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91aada8640 1 -- 10.20.193.78:0/3684834841 reap_dead start 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 0x7f9194003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient(hunting): picked mon.a con 0x7f9194003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f9194004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient(hunting): picked mon.c con 0x7f9194004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f919400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient(hunting): picked mon.b con 0x7f9194005390 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f919400c2b0 con 0x7f9194003ab0 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f919400c460 con 0x7f9194005390 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f919400c660 con 0x7f9194004900 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91a95a5640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91ac832640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91aa5a7640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 msgr2=0x7f9194003e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.800+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 0x7f9194003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f9194004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient(hunting): get_auth_request con 0x7f9194004900 auth_method 0 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91aada8640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f919400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:14.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91aada8640 10 monclient(hunting): get_auth_request con 0x7f9194005390 auth_method 0 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91aada8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91aada8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91aada8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient(hunting): handle_auth_done global_id 134434 payload 293 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 msgr2=0x7f919400bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f919400bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 msgr2=0x7f9194003e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 0x7f9194003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient: _finish_hunting 0 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 1 monclient: found mon.c 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9194003620 con 0x7f9194004900 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient: _finish_auth 0 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 10 monclient: _check_auth_tickets 2026-02-17T06:00:14.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.801+0000 7f91a9da6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a9da6640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 msgr2=0x7f9194004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a9da6640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f9194004d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f919000cdb0 tx=0x7f9190030910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e859c7c71cb8e4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919003f070 con 0x7f9194004900 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 10 monclient: dump: 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:14.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91ac832640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9190035070 con 0x7f9194004900 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 1 -- 10.20.193.78:0/3684834841 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190012690 con 0x7f9194004900 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91ac832640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91ac832640 4 set_mon_vals no callback set 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.803+0000 7f91a95a5640 10 monclient: dump: 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 10 monclient: shutdown 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 -- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 msgr2=0x7f9194004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f9194004d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f919000cdb0 tx=0x7f9190030910 comp rx=0 tx=0).stop 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 -- 10.20.193.78:0/3684834841 shutdown_connections 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f919400bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f9194004d80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 --2- 10.20.193.78:0/3684834841 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 0x7f9194003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 -- 10.20.193.78:0/3684834841 >> 10.20.193.78:0/3684834841 conn(0x7f91a40a6620 msgr2=0x7f91a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 -- 10.20.193.78:0/3684834841 shutdown_connections 2026-02-17T06:00:14.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 -- 10.20.193.78:0/3684834841 wait complete. 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 10 monclient: build_initial_monmap 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 10 monclient: monmap: 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 Processor -- start 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 1 -- start start 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.804+0000 7f91ac832640 10 monclient: init 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 0x7f91a41cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient(hunting): picked mon.noname-a con 0x7f9194003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f91a4133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient(hunting): picked mon.noname-c con 0x7f9194004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f91a4134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient(hunting): picked mon.noname-b con 0x7f9194005390 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91a40a0370 con 0x7f9194003ab0 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f91a40a01f0 con 0x7f9194005390 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f91a40a04f0 con 0x7f9194004900 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:14.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91ac832640 10 monclient(hunting): authenticate will time out at 1460.359062s 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91aa5a7640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 msgr2=0x7f91a41cc5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91aa5a7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 0x7f91a41cc5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f91a4133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f91a4133ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59234/0 (socket says 10.20.193.78:59234) 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 -- 10.20.193.78:0/2786450074 learned_addr learned my addr 10.20.193.78:0/2786450074 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient(hunting): get_auth_request con 0x7f9194004900 auth_method 0 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91aada8640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f91a4134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient(hunting): handle_auth_done global_id 144443 payload 824 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 -- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 msgr2=0x7f91a4134400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f91a4134400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 -- 10.20.193.78:0/2786450074 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 msgr2=0x7f91a41cc5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 0x7f91a41cc5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient: _finish_hunting 0 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 monclient: found mon.noname-c 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91a9da6640 1 -- 10.20.193.78:0/2786450074 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a41215a0 con 0x7f9194004900 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91aada8640 10 monclient: get_auth_request con 0x7f9194005390 auth_method 0 2026-02-17T06:00:14.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.805+0000 7f91aada8640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f91a4134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:00:14.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91a9da6640 1 -- 10.20.193.78:0/2786450074 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 msgr2=0x7f91a4133ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:14.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91a9da6640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f91a4133ec0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9190006fb0 tx=0x7f9190004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:14.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 1 -- 10.20.193.78:0/2786450074 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190049020 con 0x7f9194004900 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: dump: 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: _finish_auth 0 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 1 -- 10.20.193.78:0/2786450074 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9190045620 con 0x7f9194004900 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 1 -- 10.20.193.78:0/2786450074 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919003f040 con 0x7f9194004900 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f918effd640 10 monclient: dump: 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:14.812 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91a8da4640 4 set_mon_vals no callback set 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91ac832640 5 monclient: authenticate success, global_id 144443 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91ac832640 10 monclient: _renew_subs 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91ac832640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41ccaf0 con 0x7f9194004900 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91ac832640 10 monclient: _renew_subs 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91ac832640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.807+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a411fe70 con 0x7f9194004900 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f918effd640 1 -- 10.20.193.78:0/2786450074 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9190042030 con 0x7f9194004900 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f91ac832640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f91ac832640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f91ac832640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a41210e0 con 0x7f9194004900 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f918effd640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f917403deb0 0x7f9174040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f918effd640 1 -- 10.20.193.78:0/2786450074 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(497..497 src has 335..497) ==== 4811+0+0 (secure 0 0 0) 0x7f9190077fc0 con 0x7f9194004900 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f917403deb0 0x7f9174040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.808+0000 7f91aa5a7640 10 monclient: get_auth_request con 0x7f917403deb0 auth_method 0 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.810+0000 7f918effd640 1 -- 10.20.193.78:0/2786450074 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f91a41210e0 con 0x7f9194004900 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.810+0000 7f918effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.810+0000 7f918effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:14.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.810+0000 7f91aa5a7640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f917403deb0 0x7f9174040370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f919c006fd0 tx=0x7f919c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:14.853 INFO:tasks.rados.rados.0.trial115.stdout:2334: done (1 left) 2026-02-17T06:00:14.853 INFO:tasks.rados.rados.0.trial115.stdout:2335: write (excl) oid 25 current snap is 236 2026-02-17T06:00:14.854 INFO:tasks.rados.rados.0.trial115.stdout:2335: seq_num 830 ranges {749810=739147,2002878=650280,3293084=415653} 2026-02-17T06:00:14.859 INFO:tasks.rados.rados.0.trial115.stdout:2335: writing trial11545420-25 from 749810 to 1488957 tid 1 2026-02-17T06:00:14.863 INFO:tasks.rados.rados.0.trial115.stdout:2335: writing trial11545420-25 from 2002878 to 2653158 tid 2 2026-02-17T06:00:14.867 INFO:tasks.rados.rados.0.trial115.stdout:2335: writing trial11545420-25 from 3293084 to 3708737 tid 3 2026-02-17T06:00:14.867 INFO:tasks.rados.rados.0.trial115.stdout:2336: copy_from oid 16 from oid 11 current snap is 236 2026-02-17T06:00:14.868 INFO:tasks.rados.rados.0.trial115.stdout:2332: expect (ObjNum 773 snap 219 seq_num 773) 2026-02-17T06:00:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.925+0000 7f91ac832640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.925+0000 7f91ac832640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.925+0000 7f91ac832640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.925+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f91a4074980 con 0x7f9194004900 2026-02-17T06:00:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.925+0000 7f918effd640 1 -- 10.20.193.78:0/2786450074 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f9190053030 con 0x7f9194004900 2026-02-17T06:00:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.925+0000 7f918effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:14.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.925+0000 7f918effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:14.928 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":166,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:14.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.926+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f917403deb0 msgr2=0x7f9174040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:14.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.926+0000 7f91ac832640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f917403deb0 0x7f9174040370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f919c006fd0 tx=0x7f919c008040 comp rx=0 tx=0).stop 2026-02-17T06:00:14.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.926+0000 7f91ac832640 10 monclient: shutdown 2026-02-17T06:00:14.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.926+0000 7f91ac832640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:14.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.926+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 msgr2=0x7f91a4133ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:14.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.926+0000 7f91ac832640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f91a4133ec0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9190006fb0 tx=0x7f9190004830 comp rx=0 tx=0).stop 2026-02-17T06:00:14.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.926+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 shutdown_connections 2026-02-17T06:00:14.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.926+0000 7f91ac832640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f917403deb0 0x7f9174040370 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.927+0000 7f91ac832640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9194005390 0x7f91a4134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.927+0000 7f91ac832640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9194004900 0x7f91a4133ec0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.927+0000 7f91ac832640 1 --2- 10.20.193.78:0/2786450074 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9194003ab0 0x7f91a41cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:14.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.927+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 >> 10.20.193.78:0/2786450074 conn(0x7f91a40a6620 msgr2=0x7f91a4073000 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:14.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.927+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 shutdown_connections 2026-02-17T06:00:14.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:14.927+0000 7f91ac832640 1 -- 10.20.193.78:0/2786450074 wait complete. 2026-02-17T06:00:14.936 INFO:tasks.rados.rados.0.trial115.stdout:2335: finishing write tid 1 to trial11545420-25 2026-02-17T06:00:14.936 INFO:tasks.rados.rados.0.trial115.stdout:2335: finishing write tid 2 to trial11545420-25 2026-02-17T06:00:14.936 INFO:tasks.rados.rados.0.trial115.stdout:2335: finishing write tid 3 to trial11545420-25 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2335: finishing write tid 4 to trial11545420-25 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2335: finishing write tid 5 to trial11545420-25 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2335: oid 25 updating version 0 to 1069 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2335: oid 25 updating version 1069 to 1071 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2335: oid 25 version 1071 is already newer than 1070 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2335: oid 25 updating version 1071 to 1072 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1072 (ObjNum 830 snap 236 seq_num 830) dirty exists 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2335: left oid 25 (ObjNum 830 snap 236 seq_num 830) 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2332: done (2 left) 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2335: done (1 left) 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2337: copy_from oid 35 from oid 42 current snap is 236 2026-02-17T06:00:14.937 INFO:tasks.rados.rados.0.trial115.stdout:2338: write (excl) oid 38 current snap is 236 2026-02-17T06:00:14.938 INFO:tasks.rados.rados.0.trial115.stdout:2338: seq_num 831 ranges {766617=414396,1945007=510291,3204154=626925} 2026-02-17T06:00:14.939 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:00:14.939 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:00:14.939 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:00:14.942 INFO:tasks.rados.rados.0.trial115.stdout:2338: writing trial11545420-38 from 766617 to 1181013 tid 1 2026-02-17T06:00:14.947 INFO:tasks.rados.rados.0.trial115.stdout:2338: writing trial11545420-38 from 1945007 to 2455298 tid 2 2026-02-17T06:00:14.953 INFO:tasks.rados.rados.0.trial115.stdout:2338: writing trial11545420-38 from 3204154 to 3831079 tid 3 2026-02-17T06:00:14.953 INFO:tasks.rados.rados.0.trial115.stdout:2339: write (excl) oid 22 current snap is 236 2026-02-17T06:00:14.953 INFO:tasks.rados.rados.0.trial115.stdout:2336: finishing copy_from racing read to trial11545420-16 2026-02-17T06:00:14.953 INFO:tasks.rados.rados.0.trial115.stdout:2337: finishing copy_from racing read to trial11545420-35 2026-02-17T06:00:14.953 INFO:tasks.rados.rados.0.trial115.stdout:2337: finishing copy_from to trial11545420-35 2026-02-17T06:00:14.954 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1299 (ObjNum 761 snap 207 seq_num 761) dirty exists 2026-02-17T06:00:14.954 INFO:tasks.rados.rados.0.trial115.stdout:2336: finishing copy_from to trial11545420-16 2026-02-17T06:00:14.954 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1197 (ObjNum 824 snap 236 seq_num 824) dirty exists 2026-02-17T06:00:14.954 INFO:tasks.rados.rados.0.trial115.stdout:2338: finishing write tid 1 to trial11545420-38 2026-02-17T06:00:14.954 INFO:tasks.rados.rados.0.trial115.stdout:2339: seq_num 832 ranges {783424=489645,2003489=653949,3231577=554550} 2026-02-17T06:00:14.957 INFO:tasks.rados.rados.0.trial115.stdout:2339: writing trial11545420-22 from 783424 to 1273069 tid 1 2026-02-17T06:00:14.962 INFO:tasks.rados.rados.0.trial115.stdout:2339: writing trial11545420-22 from 2003489 to 2657438 tid 2 2026-02-17T06:00:14.967 INFO:tasks.rados.rados.0.trial115.stdout:2339: writing trial11545420-22 from 3231577 to 3786127 tid 3 2026-02-17T06:00:14.968 INFO:tasks.rados.rados.0.trial115.stdout:2336: done (3 left) 2026-02-17T06:00:14.968 INFO:tasks.rados.rados.0.trial115.stdout:2337: done (2 left) 2026-02-17T06:00:14.968 INFO:tasks.rados.rados.0.trial115.stdout:2340: read oid 12 snap 229 2026-02-17T06:00:14.968 INFO:tasks.rados.rados.0.trial115.stdout:2340: expect deleted 2026-02-17T06:00:14.968 INFO:tasks.rados.rados.0.trial115.stdout:2341: rollback oid 40 current snap is 236 2026-02-17T06:00:14.968 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 229 2026-02-17T06:00:14.968 INFO:tasks.rados.rados.0.trial115.stdout:2338: finishing write tid 2 to trial11545420-38 2026-02-17T06:00:14.968 INFO:tasks.rados.rados.0.trial115.stdout:2338: finishing write tid 3 to trial11545420-38 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2338: finishing write tid 4 to trial11545420-38 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2338: finishing write tid 5 to trial11545420-38 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2338: oid 38 updating version 0 to 1319 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2338: oid 38 version 1319 is already newer than 1317 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2338: oid 38 version 1319 is already newer than 1318 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2338: oid 38 version 1319 is already newer than 1316 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1319 (ObjNum 831 snap 236 seq_num 831) dirty exists 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2338: left oid 38 (ObjNum 831 snap 236 seq_num 831) 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2339: finishing write tid 1 to trial11545420-22 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2339: finishing write tid 2 to trial11545420-22 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2338: done (3 left) 2026-02-17T06:00:14.969 INFO:tasks.rados.rados.0.trial115.stdout:2342: delete oid 17 current snap is 236 2026-02-17T06:00:14.970 INFO:tasks.rados.rados.0.trial115.stdout:2341: finishing rollback tid 1 to trial11545420-40 2026-02-17T06:00:14.970 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 0 (ObjNum 33 snap 0 seq_num 1333649968) dirty dne 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: finishing write tid 3 to trial11545420-22 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: finishing write tid 4 to trial11545420-22 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: finishing write tid 5 to trial11545420-22 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: oid 22 updating version 0 to 1152 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: oid 22 version 1152 is already newer than 1151 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: oid 22 version 1152 is already newer than 1150 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: oid 22 version 1152 is already newer than 1149 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1152 (ObjNum 832 snap 236 seq_num 832) dirty exists 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: left oid 22 (ObjNum 832 snap 236 seq_num 832) 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2339: done (3 left) 2026-02-17T06:00:14.971 INFO:tasks.rados.rados.0.trial115.stdout:2340: done (2 left) 2026-02-17T06:00:14.972 INFO:tasks.rados.rados.0.trial115.stdout:2341: done (1 left) 2026-02-17T06:00:14.972 INFO:tasks.rados.rados.0.trial115.stdout:2342: done (0 left) 2026-02-17T06:00:14.972 INFO:tasks.rados.rados.0.trial115.stdout:2343: snap_create 2026-02-17T06:00:15.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.007+0000 7fa5be31b640 10 monclient: get_monmap_and_config 2026-02-17T06:00:15.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.007+0000 7fa5be31b640 10 monclient: build_initial_monmap 2026-02-17T06:00:15.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.007+0000 7fa5be31b640 10 monclient: monmap: 2026-02-17T06:00:15.011 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:15.011 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.011 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:15.008888+0000 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:15.008888+0000 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 1 Processor -- start 2026-02-17T06:00:15.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 1 -- start start 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient: init 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa5b806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b809c4a0 0x7fa5b813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient(hunting): picked mon.noname-b con 0x7fa5b809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa5b806d380 0x7fa5b813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient(hunting): picked mon.noname-a con 0x7fa5b806d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b806daf0 con 0x7fa5b806c890 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b80a0880 con 0x7fa5b806d380 2026-02-17T06:00:15.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b809cc80 con 0x7fa5b809c4a0 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5be31b640 10 monclient(hunting): authenticate will time out at 1460.562475s 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5bc891640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fa5b806d380 msgr2=0x7fa5b813d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5bc891640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa5b806d380 0x7fa5b813d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b809c4a0 0x7fa5b813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b809c4a0 0x7fa5b813b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59246/0 (socket says 10.20.193.78:59246) 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 1 -- 10.20.193.78:0/2787299372 learned_addr learned my addr 10.20.193.78:0/2787299372 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient(hunting): get_auth_request con 0x7fa5b809c4a0 auth_method 0 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_done global_id 144446 payload 274 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 1 -- 10.20.193.78:0/2787299372 >> v2:10.20.193.78:3300/0 conn(0x7fa5b806d380 msgr2=0x7fa5b813d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 1 --2- 10.20.193.78:0/2787299372 >> v2:10.20.193.78:3300/0 conn(0x7fa5b806d380 0x7fa5b813d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 1 -- 10.20.193.78:0/2787299372 >> v1:10.20.193.115:6789/0 conn(0x7fa5b806c890 legacy=0x7fa5b806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 1 monclient: found mon.noname-b 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:15.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.008+0000 7fa5b77fe640 1 -- 10.20.193.78:0/2787299372 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b809cf60 con 0x7fa5b809c4a0 2026-02-17T06:00:15.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b77fe640 1 -- 10.20.193.78:0/2787299372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b809c4a0 msgr2=0x7fa5b813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:15.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b77fe640 1 --2- 10.20.193.78:0/2787299372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b809c4a0 0x7fa5b813b3a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009970 tx=0x7fa5a802eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1ca231a548e735 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 1 -- 10.20.193.78:0/2787299372 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a803c070 con 0x7fa5b809c4a0 2026-02-17T06:00:15.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:15.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: dump: 2026-02-17T06:00:15.026 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:15.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 1 -- 10.20.193.78:0/2787299372 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a802fdd0 con 0x7fa5b809c4a0 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 1 -- 10.20.193.78:0/2787299372 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8037660 con 0x7fa5b809c4a0 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5b6ffd640 10 monclient: dump: 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.028 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 5 monclient: authenticate success, global_id 144446 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 4 set_mon_vals no callback set 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 10 monclient: shutdown 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 1 -- 10.20.193.78:0/2787299372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b809c4a0 msgr2=0x7fa5b813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 1 --2- 10.20.193.78:0/2787299372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b809c4a0 0x7fa5b813b3a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009970 tx=0x7fa5a802eeb0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 1 -- 10.20.193.78:0/2787299372 shutdown_connections 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 1 --2- 10.20.193.78:0/2787299372 >> v2:10.20.193.78:3300/0 conn(0x7fa5b806d380 0x7fa5b813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 1 --2- 10.20.193.78:0/2787299372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b809c4a0 0x7fa5b813b3a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 1 -- 10.20.193.78:0/2787299372 >> 10.20.193.78:0/2787299372 conn(0x7fa5b80a6620 msgr2=0x7fa5b80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 1 -- 10.20.193.78:0/2787299372 shutdown_connections 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.011+0000 7fa5be31b640 1 -- 10.20.193.78:0/2787299372 wait complete. 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient: build_initial_monmap 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient: monmap: 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:15.030 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 1 Processor -- start 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 1 -- start start 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient: init 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:15.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b806c890 0x7fa5b81cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient(hunting): picked mon.noname-c con 0x7fa5b806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5b806d380 0x7fa5b81364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient(hunting): picked mon.noname-a con 0x7fa5b806d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5b809c4a0 0x7fa5b81cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient(hunting): picked mon.noname-b con 0x7fa5b809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b809bfa0 con 0x7fa5b806d380 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b809be20 con 0x7fa5b809c4a0 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b813de30 con 0x7fa5b806c890 2026-02-17T06:00:15.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5be31b640 10 monclient(hunting): authenticate will time out at 1460.566618s 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b77fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5b806d380 msgr2=0x7fa5b81364c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5b806d380 0x7fa5b81364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b806c890 0x7fa5b81cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b806c890 0x7fa5b81cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59248/0 (socket says 10.20.193.78:59248) 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 -- 10.20.193.78:0/3698858558 learned_addr learned my addr 10.20.193.78:0/3698858558 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient(hunting): get_auth_request con 0x7fa5b806c890 auth_method 0 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:15.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient(hunting): handle_auth_done global_id 144449 payload 824 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5b809c4a0 msgr2=0x7fa5b81cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5b809c4a0 0x7fa5b81cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5b806d380 msgr2=0x7fa5b81364c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5b806d380 0x7fa5b81364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 monclient: found mon.noname-c 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.012+0000 7fa5b7fff640 1 -- 10.20.193.78:0/3698858558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b81203e0 con 0x7fa5b806c890 2026-02-17T06:00:15.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.014+0000 7fa5b7fff640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b806c890 msgr2=0x7fa5b81cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.014+0000 7fa5b7fff640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b806c890 0x7fa5b81cbd50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400c600 tx=0x7fa5a400cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4012a40 con 0x7fa5b806c890 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: dump: 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.035 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a4017070 con 0x7fa5b806c890 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 5 monclient: authenticate success, global_id 144449 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 10 monclient: _renew_subs 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b813d9d0 con 0x7fa5b806c890 2026-02-17T06:00:15.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4012e20 con 0x7fa5b806c890 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b67fc640 4 set_mon_vals no callback set 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 10 monclient: dump: 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.037 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 10 monclient: _renew_subs 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b8122e70 con 0x7fa5b806c890 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b8136a00 0x7fa5b81cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b806f830 con 0x7fa5b8136a00 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b8136a00 0x7fa5b81cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 10 monclient: get_auth_request con 0x7fa5b8136a00 auth_method 0 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa5a401c460 con 0x7fa5b806c890 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_done global_id 144452 payload 824 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b77fe640 1 -- 10.20.193.78:0/3698858558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5b8073b50 con 0x7fa5b8136a00 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa58803deb0 0x7fa588040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.015+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(498..498 src has 335..498) ==== 4795+0+0 (secure 0 0 0) 0x7fa5a4052730 con 0x7fa5b806c890 2026-02-17T06:00:15.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.016+0000 7fa5bc891640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa58803deb0 0x7fa588040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.016+0000 7fa5bc891640 10 monclient: get_auth_request con 0x7fa58803deb0 auth_method 0 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.017+0000 7fa5b77fe640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b8136a00 msgr2=0x7fa5b81cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.017+0000 7fa5b77fe640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b8136a00 0x7fa5b81cf7d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b806a480 tx=0x7fa5a80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.017+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8047020 con 0x7fa5b8136a00 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 10 monclient: dump: 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.040 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa5a8037d70 con 0x7fa5b8136a00 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b8136a00 msgr2=0x7fa5b81cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b8136a00 0x7fa5b81cf7d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b806a480 tx=0x7fa5a80047c0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b8136a00 msgr2=0x7fa5b81cf7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5b4ff9640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b8136a00 0x7fa5b81cf7d0 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b806a480 tx=0x7fa5a80047c0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.018+0000 7fa5bc891640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa58803deb0 0x7fa588040370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac003e00 tx=0x7fa5ac009230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5be31b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:00:15.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5be31b640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b81cfd10 0x7fa5b81da640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5be31b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b81d00f0 con 0x7fa5b81cfd10 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b81cfd10 0x7fa5b81da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 10 monclient: get_auth_request con 0x7fa5b81cfd10 auth_method 0 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 10 monclient(hunting): handle_auth_done global_id 144455 payload 824 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.021+0000 7fa5b77fe640 1 -- 10.20.193.78:0/3698858558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa5b806cc70 con 0x7fa5b81cfd10 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b77fe640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b81cfd10 msgr2=0x7fa5b81da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:15.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b77fe640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b81cfd10 0x7fa5b81da640 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009970 tx=0x7fa5a8009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8047030 con 0x7fa5b81cfd10 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 10 monclient: dump: 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7fa5a804eb80 con 0x7fa5b81cfd10 2026-02-17T06:00:15.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b81cfd10 msgr2=0x7fa5b81da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b81cfd10 0x7fa5b81da640 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009970 tx=0x7fa5a8009340 comp rx=0 tx=0).stop 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b81cfd10 msgr2=0x7fa5b81da640 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.023+0000 7fa5b4ff9640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b81cfd10 0x7fa5b81da640 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009970 tx=0x7fa5a8009340 comp rx=0 tx=0).stop 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:15.044 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 166, 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 198558, 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:15.045 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:15.046 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:00:15.047 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.048 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:00:15.049 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:15.050 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:00:15.051 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.052 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.053 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:00:15.054 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:00:15.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa58803deb0 msgr2=0x7fa588040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa58803deb0 0x7fa588040370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac003e00 tx=0x7fa5ac009230 comp rx=0 tx=0).stop 2026-02-17T06:00:15.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 10 monclient: shutdown 2026-02-17T06:00:15.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:15.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b806c890 msgr2=0x7fa5b81cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 1 --2- 10.20.193.78:0/3698858558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5b806c890 0x7fa5b81cbd50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400c600 tx=0x7fa5a400cad0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5bc891640 1 -- 10.20.193.78:0/3698858558 reap_dead start 2026-02-17T06:00:15.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5bc891640 1 -- 10.20.193.78:0/3698858558 reap_dead start 2026-02-17T06:00:15.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 shutdown_connections 2026-02-17T06:00:15.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 >> 10.20.193.78:0/3698858558 conn(0x7fa5b80a6620 msgr2=0x7fa5b80a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:15.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 shutdown_connections 2026-02-17T06:00:15.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.024+0000 7fa5be31b640 1 -- 10.20.193.78:0/3698858558 wait complete. 2026-02-17T06:00:15.056 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:00:15.056 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T06:00:15.056 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:00:15.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.123+0000 7f39cb01a640 10 monclient: get_monmap_and_config 2026-02-17T06:00:15.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.123+0000 7f39cb01a640 10 monclient: build_initial_monmap 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.123+0000 7f39cb01a640 10 monclient: monmap: 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:15.124700+0000 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:15.124700+0000 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.123+0000 7f39cb01a640 1 Processor -- start 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.123+0000 7f39cb01a640 1 -- start start 2026-02-17T06:00:15.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.123+0000 7f39cb01a640 10 monclient: init 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f39c406c890 0x7f39c406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient(hunting): picked mon.noname-a con 0x7f39c406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f39c409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient(hunting): picked mon.noname-b con 0x7f39c406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39c406daf0 con 0x7f39c409c4a0 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f39c40a0880 con 0x7f39c406c890 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f39c409cc80 con 0x7f39c406d1b0 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39cb01a640 10 monclient(hunting): authenticate will time out at 1460.678234s 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39bbfff640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f39c409c4a0 legacy=0x7f39c4134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c8d8f640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f39c406c890 msgr2=0x7f39c406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c8d8f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f39c406c890 0x7f39c406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:15.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c4135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59268/0 (socket says 10.20.193.78:59268) 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 1 -- 10.20.193.78:0/61394579 learned_addr learned my addr 10.20.193.78:0/61394579 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient(hunting): get_auth_request con 0x7f39c406d1b0 auth_method 0 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient(hunting): handle_auth_done global_id 144458 payload 274 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 1 -- 10.20.193.78:0/61394579 >> v2:10.20.193.78:3300/0 conn(0x7f39c406c890 msgr2=0x7f39c406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 1 --2- 10.20.193.78:0/61394579 >> v2:10.20.193.78:3300/0 conn(0x7f39c406c890 0x7f39c406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 1 -- 10.20.193.78:0/61394579 >> v1:10.20.193.78:6789/0 conn(0x7f39c409c4a0 legacy=0x7f39c4134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient: _finish_hunting 0 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 1 monclient: found mon.noname-b 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:15.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.124+0000 7f39c9590640 1 -- 10.20.193.78:0/61394579 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c409cf60 con 0x7f39c406d1b0 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39c9590640 1 -- 10.20.193.78:0/61394579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 msgr2=0x7f39c4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39c9590640 1 --2- 10.20.193.78:0/61394579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c4135500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f39c406dcc0 tx=0x7f39c002f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a9311a2d0b359b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 1 -- 10.20.193.78:0/61394579 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00373f0 con 0x7f39c406d1b0 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: dump: 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.131 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 1 -- 10.20.193.78:0/61394579 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c0037590 con 0x7f39c406d1b0 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 5 monclient: authenticate success, global_id 144458 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 1 -- 10.20.193.78:0/61394579 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0037960 con 0x7f39c406d1b0 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 4 set_mon_vals no callback set 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:15.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39bb7fe640 10 monclient: dump: 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 10 monclient: shutdown 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 1 -- 10.20.193.78:0/61394579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 msgr2=0x7f39c4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 1 --2- 10.20.193.78:0/61394579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c4135500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f39c406dcc0 tx=0x7f39c002f230 comp rx=0 tx=0).stop 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 1 -- 10.20.193.78:0/61394579 shutdown_connections 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 1 --2- 10.20.193.78:0/61394579 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c4135500 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 1 --2- 10.20.193.78:0/61394579 >> v2:10.20.193.78:3300/0 conn(0x7f39c406c890 0x7f39c406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 1 -- 10.20.193.78:0/61394579 >> 10.20.193.78:0/61394579 conn(0x7f39c40a6620 msgr2=0x7f39c40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 1 -- 10.20.193.78:0/61394579 shutdown_connections 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.127+0000 7f39cb01a640 1 -- 10.20.193.78:0/61394579 wait complete. 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient: build_initial_monmap 2026-02-17T06:00:15.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient: monmap: 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 1 Processor -- start 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 1 -- start start 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient: init 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f39c406c890 0x7f39c41cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient(hunting): picked mon.noname-a con 0x7f39c406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c41cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient(hunting): picked mon.noname-c con 0x7f39c406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c409c4a0 0x7f39c41235a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient(hunting): picked mon.noname-b con 0x7f39c409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39c40a01a0 con 0x7f39c406c890 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39c40a0020 con 0x7f39c409c4a0 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f39c409be20 con 0x7f39c406d1b0 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39c8d8f640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f39c406c890 msgr2=0x7f39c41cd330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39cb01a640 10 monclient(hunting): authenticate will time out at 1460.682732s 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39c8d8f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f39c406c890 0x7f39c41cd330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c41cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c41cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59270/0 (socket says 10.20.193.78:59270) 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 -- 10.20.193.78:0/1768354508 learned_addr learned my addr 10.20.193.78:0/1768354508 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient(hunting): get_auth_request con 0x7f39c406d1b0 auth_method 0 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39c9590640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c409c4a0 0x7f39c41235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39c9590640 10 monclient(hunting): get_auth_request con 0x7f39c409c4a0 auth_method 0 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39c9590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39c9590640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39c9590640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient(hunting): handle_auth_done global_id 144461 payload 824 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c409c4a0 msgr2=0x7f39c41235a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c409c4a0 0x7f39c41235a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f39c406c890 msgr2=0x7f39c41cd330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f39c406c890 0x7f39c41cd330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 monclient: found mon.noname-c 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:15.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.128+0000 7f39bbfff640 1 -- 10.20.193.78:0/1768354508 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c4125da0 con 0x7f39c406d1b0 2026-02-17T06:00:15.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39bbfff640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 msgr2=0x7f39c41cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:15.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39bbfff640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c41cd870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b780 tx=0x7f39ac00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0052d0 con 0x7f39c406d1b0 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39bbfff640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c41cd870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b780 tx=0x7f39ac00bc50 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f39ac0056d0 mon_map magic: 0 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: dump: 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: _finish_auth 0 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39ac010070 con 0x7f39c406d1b0 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 5 monclient: authenticate success, global_id 144461 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 10 monclient: _renew_subs 2026-02-17T06:00:15.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39baffd640 4 set_mon_vals no callback set 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c4127530 con 0x7f39c406d1b0 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 10 monclient: _renew_subs 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c409d810 con 0x7f39c406d1b0 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4126ea0 0x7f39c4123ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39c406cc70 con 0x7f39c4126ea0 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0056d0 con 0x7f39c406d1b0 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 10 monclient: dump: 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.140 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f39ac00c520 con 0x7f39c406d1b0 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39c8d8f640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4126ea0 0x7f39c4123ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39c8d8f640 10 monclient: get_auth_request con 0x7f39c4126ea0 auth_method 0 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39c8d8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39c8d8f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.131+0000 7f39c8d8f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.132+0000 7f39b97fa640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f399403de90 0x7f3994040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.132+0000 7f39c8d8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.132+0000 7f39c8d8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.132+0000 7f39c8d8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.132+0000 7f39c9590640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f399403de90 0x7f3994040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.132+0000 7f39c9590640 10 monclient: get_auth_request con 0x7f399403de90 auth_method 0 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.132+0000 7f39c8d8f640 10 monclient(hunting): handle_auth_done global_id 134440 payload 824 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.132+0000 7f39c8d8f640 1 -- 10.20.193.78:0/1768354508 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39c40008d0 con 0x7f39c4126ea0 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.134+0000 7f39c8d8f640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4126ea0 msgr2=0x7f39c4123ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.134+0000 7f39c8d8f640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4126ea0 0x7f39c4123ae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f39b4003a80 tx=0x7f39b400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.134+0000 7f39c9590640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f399403de90 0x7f3994040350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f39c409c980 tx=0x7f39c0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.134+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4005270 con 0x7f39c4126ea0 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.134+0000 7f39b97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.134+0000 7f39b97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.134+0000 7f39b97fa640 10 monclient: dump: 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.135+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f39b4015070 con 0x7f39c4126ea0 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.135+0000 7f39b97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.135+0000 7f39b97fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.135+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4126ea0 msgr2=0x7f39c4123ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.135+0000 7f39b97fa640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4126ea0 0x7f39c4123ae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f39b4003a80 tx=0x7f39b400b910 comp rx=0 tx=0).stop 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.135+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4126ea0 msgr2=0x7f39c4123ae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:15.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.135+0000 7f39b97fa640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4126ea0 0x7f39c4123ae0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f39b4003a80 tx=0x7f39b400b910 comp rx=0 tx=0).stop 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39cb01a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39cb01a640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4124020 0x7f39c41d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39cb01a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4127280 con 0x7f39c4124020 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39c8d8f640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4124020 0x7f39c41d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39c8d8f640 10 monclient: get_auth_request con 0x7f39c4124020 auth_method 0 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39c8d8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39c8d8f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39c8d8f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39c8d8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39c8d8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.138+0000 7f39c8d8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.139+0000 7f39c8d8f640 10 monclient(hunting): handle_auth_done global_id 134443 payload 824 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.139+0000 7f39c8d8f640 1 -- 10.20.193.78:0/1768354508 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f39c41cddb0 con 0x7f39c4124020 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39c8d8f640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4124020 msgr2=0x7f39c41d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39c8d8f640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4124020 0x7f39c41d8330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f39b4003a80 tx=0x7f39b4005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:15.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4005bb0 con 0x7f39c4124020 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 10 monclient: dump: 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7f39b4010480 con 0x7f39c4124020 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4124020 msgr2=0x7f39c41d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4124020 0x7f39c41d8330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f39b4003a80 tx=0x7f39b4005ad0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4124020 msgr2=0x7f39c41d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:15.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f39c4124020 0x7f39c41d8330 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f39b4003a80 tx=0x7f39b4005ad0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 166, 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 242860, 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:15.146 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.147 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:15.148 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:00:15.149 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:00:15.150 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:00:15.151 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:15.152 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:00:15.153 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:00:15.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f399403de90 msgr2=0x7f3994040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f399403de90 0x7f3994040350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f39c409c980 tx=0x7f39c0033000 comp rx=0 tx=0).stop 2026-02-17T06:00:15.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 10 monclient: shutdown 2026-02-17T06:00:15.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 msgr2=0x7f39c41cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c41cd870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b780 tx=0x7f39ac00bc50 comp rx=0 tx=0).stop 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39c9590640 1 -- 10.20.193.78:0/1768354508 reap_dead start 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39b97fa640 1 -- 10.20.193.78:0/1768354508 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(498..498 src has 335..498) ==== 4795+0+0 (unknown 0 0 0) 0x7f39ac052190 con 0x7f39c406d1b0 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 shutdown_connections 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 --2- 10.20.193.78:0/1768354508 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f39c406d1b0 0x7f39c41cd870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 >> 10.20.193.78:0/1768354508 conn(0x7f39c40a6620 msgr2=0x7f39c4072200 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 shutdown_connections 2026-02-17T06:00:15.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:15.141+0000 7f39cb01a640 1 -- 10.20.193.78:0/1768354508 wait complete. 2026-02-17T06:00:15.155 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:00:15.155 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:00:15.857 INFO:tasks.rados.rados.0.trial115.stdout:2343: done (0 left) 2026-02-17T06:00:15.858 INFO:tasks.rados.rados.0.trial115.stdout:2344: write oid 14 current snap is 237 2026-02-17T06:00:15.858 INFO:tasks.rados.rados.0.trial115.stdout:2344: seq_num 834 ranges {417038=640143,1720453=296585} 2026-02-17T06:00:15.861 INFO:tasks.rados.rados.0.trial115.stdout:2344: writing trial11545420-14 from 417038 to 1057181 tid 1 2026-02-17T06:00:15.866 INFO:tasks.rados.rados.0.trial115.stdout:2344: writing trial11545420-14 from 1720453 to 2017038 tid 2 2026-02-17T06:00:15.866 INFO:tasks.rados.rados.0.trial115.stdout:2345: delete oid 29 current snap is 237 2026-02-17T06:00:15.866 INFO:tasks.rados.rados.0.trial115.stdout:2344: finishing write tid 1 to trial11545420-14 2026-02-17T06:00:15.868 INFO:tasks.rados.rados.0.trial115.stdout:2345: done (1 left) 2026-02-17T06:00:15.868 INFO:tasks.rados.rados.0.trial115.stdout:2346: write (excl) oid 2 current snap is 237 2026-02-17T06:00:15.868 INFO:tasks.rados.rados.0.trial115.stdout:2346: seq_num 836 ranges {450652=790641,1721064=329588} 2026-02-17T06:00:15.873 INFO:tasks.rados.rados.0.trial115.stdout:2346: writing trial11545420-2 from 450652 to 1241293 tid 1 2026-02-17T06:00:15.877 INFO:tasks.rados.rados.0.trial115.stdout:2346: writing trial11545420-2 from 1721064 to 2050652 tid 2 2026-02-17T06:00:15.877 INFO:tasks.rados.rados.0.trial115.stdout:2347: delete oid 29 current snap is 237 2026-02-17T06:00:15.877 INFO:tasks.rados.rados.0.trial115.stdout:2344: finishing write tid 2 to trial11545420-14 2026-02-17T06:00:15.877 INFO:tasks.rados.rados.0.trial115.stdout:2344: finishing write tid 3 to trial11545420-14 2026-02-17T06:00:15.877 INFO:tasks.rados.rados.0.trial115.stdout:2344: finishing write tid 4 to trial11545420-14 2026-02-17T06:00:15.877 INFO:tasks.rados.rados.0.trial115.stdout:2344: oid 14 updating version 0 to 962 2026-02-17T06:00:15.877 INFO:tasks.rados.rados.0.trial115.stdout:2344: oid 14 version 962 is already newer than 961 2026-02-17T06:00:15.877 INFO:tasks.rados.rados.0.trial115.stdout:2344: oid 14 updating version 962 to 963 2026-02-17T06:00:15.878 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 963 (ObjNum 834 snap 237 seq_num 834) dirty exists 2026-02-17T06:00:15.878 INFO:tasks.rados.rados.0.trial115.stdout:2344: left oid 14 (ObjNum 834 snap 237 seq_num 834) 2026-02-17T06:00:15.878 INFO:tasks.rados.rados.0.trial115.stdout:2346: finishing write tid 1 to trial11545420-2 2026-02-17T06:00:15.878 INFO:tasks.rados.rados.0.trial115.stdout:2344: done (2 left) 2026-02-17T06:00:15.878 INFO:tasks.rados.rados.0.trial115.stdout:2347: done (1 left) 2026-02-17T06:00:15.879 INFO:tasks.rados.rados.0.trial115.stdout:2348: write oid 7 current snap is 237 2026-02-17T06:00:15.879 INFO:tasks.rados.rados.0.trial115.stdout:2348: seq_num 838 ranges {484266=657492,1554381=529885} 2026-02-17T06:00:15.881 INFO:tasks.rados.rados.0.trial115.stdout:2348: writing trial11545420-7 from 484266 to 1141758 tid 1 2026-02-17T06:00:15.885 INFO:tasks.rados.rados.0.trial115.stdout:2348: writing trial11545420-7 from 1554381 to 2084266 tid 2 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2349: rollback oid 37 current snap is 237 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 234 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2346: finishing write tid 2 to trial11545420-2 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2346: finishing write tid 3 to trial11545420-2 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2346: finishing write tid 4 to trial11545420-2 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2346: oid 2 updating version 0 to 1137 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2346: oid 2 updating version 1137 to 1138 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2346: oid 2 version 1138 is already newer than 1136 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2346: left oid 2 (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2346: done (2 left) 2026-02-17T06:00:15.886 INFO:tasks.rados.rados.0.trial115.stdout:2350: delete oid 10 current snap is 237 2026-02-17T06:00:15.887 INFO:tasks.rados.rados.0.trial115.stdout:2348: finishing write tid 1 to trial11545420-7 2026-02-17T06:00:15.888 INFO:tasks.rados.rados.0.trial115.stdout:2350: done (2 left) 2026-02-17T06:00:15.888 INFO:tasks.rados.rados.0.trial115.stdout:2351: read oid 32 snap -1 2026-02-17T06:00:15.888 INFO:tasks.rados.rados.0.trial115.stdout:2351: expect (ObjNum 815 snap 235 seq_num 815) 2026-02-17T06:00:15.888 INFO:tasks.rados.rados.0.trial115.stdout:2352: read oid 9 snap -1 2026-02-17T06:00:15.888 INFO:tasks.rados.rados.0.trial115.stdout:2352: expect (ObjNum 759 snap 207 seq_num 759) 2026-02-17T06:00:15.888 INFO:tasks.rados.rados.0.trial115.stdout:2353: write oid 20 current snap is 237 2026-02-17T06:00:15.888 INFO:tasks.rados.rados.0.trial115.stdout:2353: seq_num 840 ranges {517880=407990,1554992=562888} 2026-02-17T06:00:15.890 INFO:tasks.rados.rados.0.trial115.stdout:2353: writing trial11545420-20 from 517880 to 925870 tid 1 2026-02-17T06:00:15.895 INFO:tasks.rados.rados.0.trial115.stdout:2353: writing trial11545420-20 from 1554992 to 2117880 tid 2 2026-02-17T06:00:15.895 INFO:tasks.rados.rados.0.trial115.stdout:2354: snap_remove snap 236 2026-02-17T06:00:16.861 INFO:tasks.rados.rados.0.trial115.stdout:2354: done (5 left) 2026-02-17T06:00:16.861 INFO:tasks.rados.rados.0.trial115.stdout:2355: write oid 27 current snap is 237 2026-02-17T06:00:16.861 INFO:tasks.rados.rados.0.trial115.stdout:2355: seq_num 841 ranges {534687=483239,1613474=412283} 2026-02-17T06:00:16.865 INFO:tasks.rados.rados.0.trial115.stdout:2355: writing trial11545420-27 from 534687 to 1017926 tid 1 2026-02-17T06:00:16.869 INFO:tasks.rados.rados.0.trial115.stdout:2355: writing trial11545420-27 from 1613474 to 2025757 tid 2 2026-02-17T06:00:16.869 INFO:tasks.rados.rados.0.trial115.stdout:2356: snap_create 2026-02-17T06:00:16.869 INFO:tasks.rados.rados.0.trial115.stdout:2348: finishing write tid 2 to trial11545420-7 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2348: finishing write tid 3 to trial11545420-7 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2348: finishing write tid 4 to trial11545420-7 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2348: oid 7 updating version 0 to 968 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2348: oid 7 updating version 968 to 969 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2348: oid 7 updating version 969 to 970 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 970 (ObjNum 838 snap 237 seq_num 838) dirty exists 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2348: left oid 7 (ObjNum 838 snap 237 seq_num 838) 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2349: finishing rollback tid 0 to trial11545420-37 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2349: finishing rollback tid 1 to trial11545420-37 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2349: finishing rollback tid 2 to trial11545420-37 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 973 (ObjNum 813 snap 234 seq_num 813) dirty exists 2026-02-17T06:00:16.870 INFO:tasks.rados.rados.0.trial115.stdout:2352: expect (ObjNum 759 snap 207 seq_num 759) 2026-02-17T06:00:16.981 INFO:tasks.rados.rados.0.trial115.stdout:2351: expect (ObjNum 815 snap 235 seq_num 815) 2026-02-17T06:00:17.190 INFO:tasks.rados.rados.0.trial115.stdout:2353: finishing write tid 1 to trial11545420-20 2026-02-17T06:00:17.190 INFO:tasks.rados.rados.0.trial115.stdout:2353: finishing write tid 2 to trial11545420-20 2026-02-17T06:00:17.190 INFO:tasks.rados.rados.0.trial115.stdout:2353: finishing write tid 3 to trial11545420-20 2026-02-17T06:00:17.190 INFO:tasks.rados.rados.0.trial115.stdout:2353: finishing write tid 4 to trial11545420-20 2026-02-17T06:00:17.190 INFO:tasks.rados.rados.0.trial115.stdout:2353: oid 20 updating version 0 to 976 2026-02-17T06:00:17.190 INFO:tasks.rados.rados.0.trial115.stdout:2353: oid 20 version 976 is already newer than 975 2026-02-17T06:00:17.190 INFO:tasks.rados.rados.0.trial115.stdout:2353: oid 20 version 976 is already newer than 974 2026-02-17T06:00:17.190 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 976 (ObjNum 840 snap 237 seq_num 840) dirty exists 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2353: left oid 20 (ObjNum 840 snap 237 seq_num 840) 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2355: finishing write tid 1 to trial11545420-27 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2355: finishing write tid 2 to trial11545420-27 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2355: finishing write tid 3 to trial11545420-27 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2355: finishing write tid 4 to trial11545420-27 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2355: oid 27 updating version 0 to 1155 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2355: oid 27 version 1155 is already newer than 1153 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2355: oid 27 version 1155 is already newer than 1154 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1155 (ObjNum 841 snap 237 seq_num 841) dirty exists 2026-02-17T06:00:17.191 INFO:tasks.rados.rados.0.trial115.stdout:2355: left oid 27 (ObjNum 841 snap 237 seq_num 841) 2026-02-17T06:00:17.751 INFO:tasks.rados.rados.0.trial115.stdout:2348: done (6 left) 2026-02-17T06:00:17.752 INFO:tasks.rados.rados.0.trial115.stdout:2349: done (5 left) 2026-02-17T06:00:17.752 INFO:tasks.rados.rados.0.trial115.stdout:2351: done (4 left) 2026-02-17T06:00:17.752 INFO:tasks.rados.rados.0.trial115.stdout:2352: done (3 left) 2026-02-17T06:00:17.752 INFO:tasks.rados.rados.0.trial115.stdout:2353: done (2 left) 2026-02-17T06:00:17.752 INFO:tasks.rados.rados.0.trial115.stdout:2355: done (1 left) 2026-02-17T06:00:17.752 INFO:tasks.rados.rados.0.trial115.stdout:2356: done (0 left) 2026-02-17T06:00:17.753 INFO:tasks.rados.rados.0.trial115.stdout:2357: snap_create 2026-02-17T06:00:18.831 INFO:tasks.rados.rados.0.trial115.stdout:2357: done (0 left) 2026-02-17T06:00:18.831 INFO:tasks.rados.rados.0.trial115.stdout:2358: read oid 3 snap -1 2026-02-17T06:00:18.831 INFO:tasks.rados.rados.0.trial115.stdout:2358: expect (ObjNum 800 snap 230 seq_num 800) 2026-02-17T06:00:18.831 INFO:tasks.rados.rados.0.trial115.stdout:2359: rollback oid 21 current snap is 239 2026-02-17T06:00:18.831 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 21 to 237 2026-02-17T06:00:18.832 INFO:tasks.rados.rados.0.trial115.stdout:2360: copy_from oid 45 from oid 29 current snap is 239 2026-02-17T06:00:18.832 INFO:tasks.rados.rados.0.trial115.stdout:2361: copy_from oid 49 from oid 19 current snap is 239 2026-02-17T06:00:18.832 INFO:tasks.rados.rados.0.trial115.stdout:2362: copy_from oid 30 from oid 48 current snap is 239 2026-02-17T06:00:18.832 INFO:tasks.rados.rados.0.trial115.stdout:2363: snap_create 2026-02-17T06:00:18.832 INFO:tasks.rados.rados.0.trial115.stdout:2360: finishing copy_from to trial11545420-45 2026-02-17T06:00:18.833 INFO:tasks.rados.rados.0.trial115.stdout:2360: got expected ENOENT (src dne) 2026-02-17T06:00:18.833 INFO:tasks.rados.rados.0.trial115.stdout:2360: finishing copy_from racing read to trial11545420-45 2026-02-17T06:00:18.836 INFO:tasks.rados.rados.0.trial115.stdout:2359: finishing rollback tid 0 to trial11545420-21 2026-02-17T06:00:18.836 INFO:tasks.rados.rados.0.trial115.stdout:2359: finishing rollback tid 1 to trial11545420-21 2026-02-17T06:00:18.836 INFO:tasks.rados.rados.0.trial115.stdout:2359: finishing rollback tid 2 to trial11545420-21 2026-02-17T06:00:18.836 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1075 (ObjNum 813 snap 234 seq_num 813) dirty exists 2026-02-17T06:00:18.836 INFO:tasks.rados.rados.0.trial115.stdout:2362: finishing copy_from to trial11545420-30 2026-02-17T06:00:18.836 INFO:tasks.rados.rados.0.trial115.stdout:2362: got expected ENOENT (src dne) 2026-02-17T06:00:18.836 INFO:tasks.rados.rados.0.trial115.stdout:2362: finishing copy_from racing read to trial11545420-30 2026-02-17T06:00:18.837 INFO:tasks.rados.rados.0.trial115.stdout:2358: expect (ObjNum 800 snap 230 seq_num 800) 2026-02-17T06:00:18.956 INFO:tasks.rados.rados.0.trial115.stdout:2361: finishing copy_from racing read to trial11545420-49 2026-02-17T06:00:18.957 INFO:tasks.rados.rados.0.trial115.stdout:2361: finishing copy_from to trial11545420-49 2026-02-17T06:00:18.957 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1268 (ObjNum 790 snap 224 seq_num 790) dirty exists 2026-02-17T06:00:19.831 INFO:tasks.rados.rados.0.trial115.stdout:2358: done (5 left) 2026-02-17T06:00:19.832 INFO:tasks.rados.rados.0.trial115.stdout:2359: done (4 left) 2026-02-17T06:00:19.832 INFO:tasks.rados.rados.0.trial115.stdout:2360: done (3 left) 2026-02-17T06:00:19.832 INFO:tasks.rados.rados.0.trial115.stdout:2361: done (2 left) 2026-02-17T06:00:19.832 INFO:tasks.rados.rados.0.trial115.stdout:2362: done (1 left) 2026-02-17T06:00:19.832 INFO:tasks.rados.rados.0.trial115.stdout:2363: done (0 left) 2026-02-17T06:00:19.832 INFO:tasks.rados.rados.0.trial115.stdout:2364: delete oid 1 current snap is 240 2026-02-17T06:00:19.834 INFO:tasks.rados.rados.0.trial115.stdout:2364: done (0 left) 2026-02-17T06:00:19.834 INFO:tasks.rados.rados.0.trial115.stdout:2365: snap_create 2026-02-17T06:00:20.833 INFO:tasks.rados.rados.0.trial115.stdout:2365: done (0 left) 2026-02-17T06:00:20.833 INFO:tasks.rados.rados.0.trial115.stdout:2366: delete oid 15 current snap is 241 2026-02-17T06:00:20.836 INFO:tasks.rados.rados.0.trial115.stdout:2366: done (0 left) 2026-02-17T06:00:20.836 INFO:tasks.rados.rados.0.trial115.stdout:2367: read oid 4 snap -1 2026-02-17T06:00:20.837 INFO:tasks.rados.rados.0.trial115.stdout:2367: expect (ObjNum 766 snap 212 seq_num 766) 2026-02-17T06:00:20.837 INFO:tasks.rados.rados.0.trial115.stdout:2368: read oid 18 snap -1 2026-02-17T06:00:20.837 INFO:tasks.rados.rados.0.trial115.stdout:2368: expect (ObjNum 818 snap 235 seq_num 818) 2026-02-17T06:00:20.837 INFO:tasks.rados.rados.0.trial115.stdout:2369: write (excl) oid 41 current snap is 241 2026-02-17T06:00:20.837 INFO:tasks.rados.rados.0.trial115.stdout:2369: seq_num 844 ranges {585108=425339,1788920=396188} 2026-02-17T06:00:20.840 INFO:tasks.rados.rados.0.trial115.stdout:2369: writing trial11545420-41 from 585108 to 1010447 tid 1 2026-02-17T06:00:20.844 INFO:tasks.rados.rados.0.trial115.stdout:2369: writing trial11545420-41 from 1788920 to 2185108 tid 2 2026-02-17T06:00:20.845 INFO:tasks.rados.rados.0.trial115.stdout:2370: read oid 25 snap -1 2026-02-17T06:00:20.845 INFO:tasks.rados.rados.0.trial115.stdout:2370: expect (ObjNum 830 snap 236 seq_num 830) 2026-02-17T06:00:20.845 INFO:tasks.rados.rados.0.trial115.stdout:2371: copy_from oid 29 from oid 36 current snap is 241 2026-02-17T06:00:20.845 INFO:tasks.rados.rados.0.trial115.stdout:2368: expect (ObjNum 818 snap 235 seq_num 818) 2026-02-17T06:00:20.932 INFO:tasks.rados.rados.0.trial115.stdout:2367: expect (ObjNum 766 snap 212 seq_num 766) 2026-02-17T06:00:21.022 INFO:tasks.rados.rados.0.trial115.stdout:2369: finishing write tid 1 to trial11545420-41 2026-02-17T06:00:21.022 INFO:tasks.rados.rados.0.trial115.stdout:2367: done (4 left) 2026-02-17T06:00:21.022 INFO:tasks.rados.rados.0.trial115.stdout:2368: done (3 left) 2026-02-17T06:00:21.022 INFO:tasks.rados.rados.0.trial115.stdout:2372: write oid 28 current snap is 241 2026-02-17T06:00:21.022 INFO:tasks.rados.rados.0.trial115.stdout:2372: seq_num 845 ranges {601915=500588,1847402=354513} 2026-02-17T06:00:21.026 INFO:tasks.rados.rados.0.trial115.stdout:2372: writing trial11545420-28 from 601915 to 1102503 tid 1 2026-02-17T06:00:21.030 INFO:tasks.rados.rados.0.trial115.stdout:2372: writing trial11545420-28 from 1847402 to 2201915 tid 2 2026-02-17T06:00:21.030 INFO:tasks.rados.rados.0.trial115.stdout:2373: read oid 31 snap -1 2026-02-17T06:00:21.030 INFO:tasks.rados.rados.0.trial115.stdout:2373: expect deleted 2026-02-17T06:00:21.031 INFO:tasks.rados.rados.0.trial115.stdout:2374: snap_remove snap 240 2026-02-17T06:00:21.837 INFO:tasks.rados.rados.0.trial115.stdout:2374: done (5 left) 2026-02-17T06:00:21.838 INFO:tasks.rados.rados.0.trial115.stdout:2375: snap_create 2026-02-17T06:00:21.838 INFO:tasks.rados.rados.0.trial115.stdout:2371: finishing copy_from to trial11545420-29 2026-02-17T06:00:21.838 INFO:tasks.rados.rados.0.trial115.stdout:2371: got expected ENOENT (src dne) 2026-02-17T06:00:21.838 INFO:tasks.rados.rados.0.trial115.stdout:2371: finishing copy_from racing read to trial11545420-29 2026-02-17T06:00:21.838 INFO:tasks.rados.rados.0.trial115.stdout:2369: finishing write tid 2 to trial11545420-41 2026-02-17T06:00:21.838 INFO:tasks.rados.rados.0.trial115.stdout:2369: finishing write tid 3 to trial11545420-41 2026-02-17T06:00:21.838 INFO:tasks.rados.rados.0.trial115.stdout:2369: finishing write tid 4 to trial11545420-41 2026-02-17T06:00:21.839 INFO:tasks.rados.rados.0.trial115.stdout:2369: oid 41 updating version 0 to 1078 2026-02-17T06:00:21.839 INFO:tasks.rados.rados.0.trial115.stdout:2369: oid 41 version 1078 is already newer than 1077 2026-02-17T06:00:21.839 INFO:tasks.rados.rados.0.trial115.stdout:2369: oid 41 version 1078 is already newer than 1076 2026-02-17T06:00:21.839 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1078 (ObjNum 844 snap 241 seq_num 844) dirty exists 2026-02-17T06:00:21.839 INFO:tasks.rados.rados.0.trial115.stdout:2369: left oid 41 (ObjNum 844 snap 241 seq_num 844) 2026-02-17T06:00:21.839 INFO:tasks.rados.rados.0.trial115.stdout:2370: expect (ObjNum 830 snap 236 seq_num 830) 2026-02-17T06:00:21.929 INFO:tasks.rados.rados.0.trial115.stdout:2372: finishing write tid 1 to trial11545420-28 2026-02-17T06:00:21.929 INFO:tasks.rados.rados.0.trial115.stdout:2372: finishing write tid 2 to trial11545420-28 2026-02-17T06:00:21.929 INFO:tasks.rados.rados.0.trial115.stdout:2372: finishing write tid 3 to trial11545420-28 2026-02-17T06:00:21.929 INFO:tasks.rados.rados.0.trial115.stdout:2372: finishing write tid 4 to trial11545420-28 2026-02-17T06:00:21.930 INFO:tasks.rados.rados.0.trial115.stdout:2372: oid 28 updating version 0 to 1214 2026-02-17T06:00:21.930 INFO:tasks.rados.rados.0.trial115.stdout:2372: oid 28 updating version 1214 to 1215 2026-02-17T06:00:21.930 INFO:tasks.rados.rados.0.trial115.stdout:2372: oid 28 version 1215 is already newer than 1213 2026-02-17T06:00:21.930 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1215 (ObjNum 845 snap 241 seq_num 845) dirty exists 2026-02-17T06:00:21.930 INFO:tasks.rados.rados.0.trial115.stdout:2372: left oid 28 (ObjNum 845 snap 241 seq_num 845) 2026-02-17T06:00:23.156 INFO:tasks.rados.rados.0.trial115.stdout:2369: done (5 left) 2026-02-17T06:00:23.157 INFO:tasks.rados.rados.0.trial115.stdout:2370: done (4 left) 2026-02-17T06:00:23.157 INFO:tasks.rados.rados.0.trial115.stdout:2371: done (3 left) 2026-02-17T06:00:23.157 INFO:tasks.rados.rados.0.trial115.stdout:2372: done (2 left) 2026-02-17T06:00:23.157 INFO:tasks.rados.rados.0.trial115.stdout:2373: done (1 left) 2026-02-17T06:00:23.157 INFO:tasks.rados.rados.0.trial115.stdout:2375: done (0 left) 2026-02-17T06:00:23.157 INFO:tasks.rados.rados.0.trial115.stdout:2376: copy_from oid 35 from oid 47 current snap is 242 2026-02-17T06:00:23.157 INFO:tasks.rados.rados.0.trial115.stdout:2377: snap_remove snap 239 2026-02-17T06:00:23.840 INFO:tasks.rados.rados.0.trial115.stdout:2377: done (1 left) 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2378: read oid 29 snap -1 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2378: expect deleted 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2379: read oid 28 snap -1 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2379: expect (ObjNum 845 snap 241 seq_num 845) 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2380: read oid 37 snap -1 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2380: expect (ObjNum 813 snap 234 seq_num 813) 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2381: snap_create 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2376: finishing copy_from racing read to trial11545420-35 2026-02-17T06:00:23.841 INFO:tasks.rados.rados.0.trial115.stdout:2376: finishing copy_from to trial11545420-35 2026-02-17T06:00:23.842 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1117 (ObjNum 810 snap 234 seq_num 810) dirty exists 2026-02-17T06:00:23.847 INFO:tasks.rados.rados.0.trial115.stdout:2379: expect (ObjNum 845 snap 241 seq_num 845) 2026-02-17T06:00:23.928 INFO:tasks.rados.rados.0.trial115.stdout:2380: expect (ObjNum 813 snap 234 seq_num 813) 2026-02-17T06:00:24.843 INFO:tasks.rados.rados.0.trial115.stdout:2376: done (4 left) 2026-02-17T06:00:24.844 INFO:tasks.rados.rados.0.trial115.stdout:2378: done (3 left) 2026-02-17T06:00:24.844 INFO:tasks.rados.rados.0.trial115.stdout:2379: done (2 left) 2026-02-17T06:00:24.844 INFO:tasks.rados.rados.0.trial115.stdout:2380: done (1 left) 2026-02-17T06:00:24.844 INFO:tasks.rados.rados.0.trial115.stdout:2381: done (0 left) 2026-02-17T06:00:24.844 INFO:tasks.rados.rados.0.trial115.stdout:2382: write (excl) oid 17 current snap is 243 2026-02-17T06:00:24.844 INFO:tasks.rados.rados.0.trial115.stdout:2382: seq_num 846 ranges {618722=575837,1905884=312838} 2026-02-17T06:00:24.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: writing trial11545420-17 from 618722 to 1194559 tid 1 2026-02-17T06:00:24.851 INFO:tasks.rados.rados.0.trial115.stdout:2382: writing trial11545420-17 from 1905884 to 2218722 tid 2 2026-02-17T06:00:24.852 INFO:tasks.rados.rados.0.trial115.stdout:2383: snap_remove snap 234 2026-02-17T06:00:25.846 INFO:tasks.rados.rados.0.trial115.stdout:2383: done (1 left) 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2384: copy_from oid 33 from oid 25 current snap is 243 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: finishing write tid 1 to trial11545420-17 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: finishing write tid 2 to trial11545420-17 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: finishing write tid 3 to trial11545420-17 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: finishing write tid 4 to trial11545420-17 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: oid 17 updating version 0 to 1066 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: oid 17 updating version 1066 to 1067 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: oid 17 version 1067 is already newer than 1065 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1067 (ObjNum 846 snap 243 seq_num 846) dirty exists 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: left oid 17 (ObjNum 846 snap 243 seq_num 846) 2026-02-17T06:00:25.847 INFO:tasks.rados.rados.0.trial115.stdout:2382: done (1 left) 2026-02-17T06:00:25.848 INFO:tasks.rados.rados.0.trial115.stdout:2385: read oid 5 snap -1 2026-02-17T06:00:25.848 INFO:tasks.rados.rados.0.trial115.stdout:2385: expect (ObjNum 777 snap 219 seq_num 777) 2026-02-17T06:00:25.848 INFO:tasks.rados.rados.0.trial115.stdout:2386: snap_remove snap 242 2026-02-17T06:00:26.848 INFO:tasks.rados.rados.0.trial115.stdout:2386: done (2 left) 2026-02-17T06:00:26.849 INFO:tasks.rados.rados.0.trial115.stdout:2387: copy_from oid 49 from oid 2 current snap is 243 2026-02-17T06:00:26.849 INFO:tasks.rados.rados.0.trial115.stdout:2385: expect (ObjNum 777 snap 219 seq_num 777) 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:2384: finishing copy_from racing read to trial11545420-33 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:2384: finishing copy_from to trial11545420-33 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1072 (ObjNum 830 snap 236 seq_num 830) dirty exists 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:2387: finishing copy_from racing read to trial11545420-49 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:2387: finishing copy_from to trial11545420-49 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:2384: done (2 left) 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:2385: done (1 left) 2026-02-17T06:00:26.976 INFO:tasks.rados.rados.0.trial115.stdout:2387: done (0 left) 2026-02-17T06:00:26.977 INFO:tasks.rados.rados.0.trial115.stdout:2388: delete oid 14 current snap is 243 2026-02-17T06:00:26.977 INFO:tasks.rados.rados.0.trial115.stdout:2388: done (0 left) 2026-02-17T06:00:26.977 INFO:tasks.rados.rados.0.trial115.stdout:2389: copy_from oid 27 from oid 29 current snap is 243 2026-02-17T06:00:26.977 INFO:tasks.rados.rados.0.trial115.stdout:2390: snap_remove snap 238 2026-02-17T06:00:27.305 INFO:tasks.rados.rados.0.trial115.stdout:2390: done (1 left) 2026-02-17T06:00:27.306 INFO:tasks.rados.rados.0.trial115.stdout:2391: snap_create 2026-02-17T06:00:27.306 INFO:tasks.rados.rados.0.trial115.stdout:2389: finishing copy_from to trial11545420-27 2026-02-17T06:00:27.306 INFO:tasks.rados.rados.0.trial115.stdout:2389: got expected ENOENT (src dne) 2026-02-17T06:00:27.306 INFO:tasks.rados.rados.0.trial115.stdout:2389: finishing copy_from racing read to trial11545420-27 2026-02-17T06:00:28.306 INFO:tasks.rados.rados.0.trial115.stdout:2389: done (1 left) 2026-02-17T06:00:28.306 INFO:tasks.rados.rados.0.trial115.stdout:2391: done (0 left) 2026-02-17T06:00:28.307 INFO:tasks.rados.rados.0.trial115.stdout:2392: rollback oid 24 current snap is 244 2026-02-17T06:00:28.307 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 235 2026-02-17T06:00:28.307 INFO:tasks.rados.rados.0.trial115.stdout:2393: write oid 22 current snap is 244 2026-02-17T06:00:28.307 INFO:tasks.rados.rados.0.trial115.stdout:2393: seq_num 848 ranges {652336=726335,1906495=345841} 2026-02-17T06:00:28.311 INFO:tasks.rados.rados.0.trial115.stdout:2393: writing trial11545420-22 from 652336 to 1378671 tid 1 2026-02-17T06:00:28.315 INFO:tasks.rados.rados.0.trial115.stdout:2393: writing trial11545420-22 from 1906495 to 2252336 tid 2 2026-02-17T06:00:28.315 INFO:tasks.rados.rados.0.trial115.stdout:2394: snap_remove snap 223 2026-02-17T06:00:29.306 INFO:tasks.rados.rados.0.trial115.stdout:2394: done (2 left) 2026-02-17T06:00:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2395: write (excl) oid 38 current snap is 244 2026-02-17T06:00:29.307 INFO:tasks.rados.rados.0.trial115.stdout:2395: seq_num 849 ranges {669143=401584,1564977=426959} 2026-02-17T06:00:29.311 INFO:tasks.rados.rados.0.trial115.stdout:2395: writing trial11545420-38 from 669143 to 1070727 tid 1 2026-02-17T06:00:29.316 INFO:tasks.rados.rados.0.trial115.stdout:2395: writing trial11545420-38 from 1564977 to 1991936 tid 2 2026-02-17T06:00:29.316 INFO:tasks.rados.rados.0.trial115.stdout:2396: rollback oid 37 current snap is 244 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 229 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2392: finishing rollback tid 0 to trial11545420-24 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2392: finishing rollback tid 1 to trial11545420-24 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2392: finishing rollback tid 2 to trial11545420-24 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1330 (ObjNum 730 snap 202 seq_num 730) dirty exists 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2393: finishing write tid 1 to trial11545420-22 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2393: finishing write tid 2 to trial11545420-22 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2392: done (3 left) 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2397: rollback oid 1 current snap is 244 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 1 to 243 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2393: finishing write tid 3 to trial11545420-22 2026-02-17T06:00:29.317 INFO:tasks.rados.rados.0.trial115.stdout:2393: finishing write tid 4 to trial11545420-22 2026-02-17T06:00:29.318 INFO:tasks.rados.rados.0.trial115.stdout:2393: oid 22 updating version 0 to 1157 2026-02-17T06:00:29.318 INFO:tasks.rados.rados.0.trial115.stdout:2393: oid 22 version 1157 is already newer than 1156 2026-02-17T06:00:29.318 INFO:tasks.rados.rados.0.trial115.stdout:2393: oid 22 updating version 1157 to 1158 2026-02-17T06:00:29.318 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1158 (ObjNum 848 snap 244 seq_num 848) dirty exists 2026-02-17T06:00:29.318 INFO:tasks.rados.rados.0.trial115.stdout:2393: left oid 22 (ObjNum 848 snap 244 seq_num 848) 2026-02-17T06:00:29.318 INFO:tasks.rados.rados.0.trial115.stdout:2393: done (3 left) 2026-02-17T06:00:29.318 INFO:tasks.rados.rados.0.trial115.stdout:2398: write oid 8 current snap is 244 2026-02-17T06:00:29.318 INFO:tasks.rados.rados.0.trial115.stdout:2398: seq_num 850 ranges {685950=476833,1623459=570617} 2026-02-17T06:00:29.320 INFO:tasks.rados.rados.0.trial115.stdout:2398: writing trial11545420-8 from 685950 to 1162783 tid 1 2026-02-17T06:00:29.324 INFO:tasks.rados.rados.0.trial115.stdout:2398: writing trial11545420-8 from 1623459 to 2194076 tid 2 2026-02-17T06:00:29.324 INFO:tasks.rados.rados.0.trial115.stdout:2399: read oid 36 snap -1 2026-02-17T06:00:29.324 INFO:tasks.rados.rados.0.trial115.stdout:2399: expect deleted 2026-02-17T06:00:29.324 INFO:tasks.rados.rados.0.trial115.stdout:2400: snap_remove snap 222 2026-02-17T06:00:30.312 INFO:tasks.rados.rados.0.trial115.stdout:2400: done (5 left) 2026-02-17T06:00:30.312 INFO:tasks.rados.rados.0.trial115.stdout:2401: rollback oid 46 current snap is 244 2026-02-17T06:00:30.312 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 46 to 241 2026-02-17T06:00:30.312 INFO:tasks.rados.rados.0.trial115.stdout:2395: finishing write tid 1 to trial11545420-38 2026-02-17T06:00:30.312 INFO:tasks.rados.rados.0.trial115.stdout:2397: finishing rollback tid 1 to trial11545420-1 2026-02-17T06:00:30.312 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 964 (ObjNum 33 snap 0 seq_num 1333812144) dirty dne 2026-02-17T06:00:30.312 INFO:tasks.rados.rados.0.trial115.stdout:2396: finishing rollback tid 0 to trial11545420-37 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2396: finishing rollback tid 1 to trial11545420-37 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2396: finishing rollback tid 2 to trial11545420-37 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 979 (ObjNum 794 snap 229 seq_num 794) dirty exists 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2395: finishing write tid 2 to trial11545420-38 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2395: finishing write tid 3 to trial11545420-38 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2395: finishing write tid 4 to trial11545420-38 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2395: oid 38 updating version 0 to 1333 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2395: oid 38 updating version 1333 to 1342 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2395: oid 38 updating version 1342 to 1343 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1343 (ObjNum 849 snap 244 seq_num 849) dirty exists 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2395: left oid 38 (ObjNum 849 snap 244 seq_num 849) 2026-02-17T06:00:30.313 INFO:tasks.rados.rados.0.trial115.stdout:2398: finishing write tid 1 to trial11545420-8 2026-02-17T06:00:30.314 INFO:tasks.rados.rados.0.trial115.stdout:2398: finishing write tid 2 to trial11545420-8 2026-02-17T06:00:30.314 INFO:tasks.rados.rados.0.trial115.stdout:2395: done (5 left) 2026-02-17T06:00:30.314 INFO:tasks.rados.rados.0.trial115.stdout:2396: done (4 left) 2026-02-17T06:00:30.314 INFO:tasks.rados.rados.0.trial115.stdout:2397: done (3 left) 2026-02-17T06:00:30.314 INFO:tasks.rados.rados.0.trial115.stdout:2399: done (2 left) 2026-02-17T06:00:30.314 INFO:tasks.rados.rados.0.trial115.stdout:2402: snap_remove snap 237 2026-02-17T06:00:31.320 INFO:tasks.rados.rados.0.trial115.stdout:2402: done (2 left) 2026-02-17T06:00:31.320 INFO:tasks.rados.rados.0.trial115.stdout:2403: write oid 41 current snap is 244 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2398: finishing write tid 3 to trial11545420-8 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2398: finishing write tid 4 to trial11545420-8 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2398: oid 8 updating version 0 to 291 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2398: oid 8 updating version 291 to 292 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2398: oid 8 version 292 is already newer than 288 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 292 (ObjNum 850 snap 244 seq_num 850) dirty exists 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2398: left oid 8 (ObjNum 850 snap 244 seq_num 850) 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2401: finishing rollback tid 0 to trial11545420-46 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2401: finishing rollback tid 1 to trial11545420-46 2026-02-17T06:00:31.321 INFO:tasks.rados.rados.0.trial115.stdout:2401: finishing rollback tid 2 to trial11545420-46 2026-02-17T06:00:31.322 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1249 (ObjNum 819 snap 235 seq_num 819) dirty exists 2026-02-17T06:00:31.322 INFO:tasks.rados.rados.0.trial115.stdout:2403: seq_num 851 ranges {702757=552082,1681941=430628} 2026-02-17T06:00:31.325 INFO:tasks.rados.rados.0.trial115.stdout:2403: writing trial11545420-41 from 702757 to 1254839 tid 1 2026-02-17T06:00:31.328 INFO:tasks.rados.rados.0.trial115.stdout:2403: writing trial11545420-41 from 1681941 to 2112569 tid 2 2026-02-17T06:00:31.329 INFO:tasks.rados.rados.0.trial115.stdout:2398: done (2 left) 2026-02-17T06:00:31.329 INFO:tasks.rados.rados.0.trial115.stdout:2401: done (1 left) 2026-02-17T06:00:31.329 INFO:tasks.rados.rados.0.trial115.stdout:2404: write oid 21 current snap is 244 2026-02-17T06:00:31.329 INFO:tasks.rados.rados.0.trial115.stdout:2404: seq_num 852 ranges {719564=743684,2140423=179141} 2026-02-17T06:00:31.333 INFO:tasks.rados.rados.0.trial115.stdout:2404: writing trial11545420-21 from 719564 to 1463248 tid 1 2026-02-17T06:00:31.337 INFO:tasks.rados.rados.0.trial115.stdout:2404: writing trial11545420-21 from 2140423 to 2319564 tid 2 2026-02-17T06:00:31.337 INFO:tasks.rados.rados.0.trial115.stdout:2405: delete oid 12 current snap is 244 2026-02-17T06:00:31.337 INFO:tasks.rados.rados.0.trial115.stdout:2403: finishing write tid 1 to trial11545420-41 2026-02-17T06:00:31.337 INFO:tasks.rados.rados.0.trial115.stdout:2403: finishing write tid 2 to trial11545420-41 2026-02-17T06:00:31.337 INFO:tasks.rados.rados.0.trial115.stdout:2403: finishing write tid 3 to trial11545420-41 2026-02-17T06:00:31.337 INFO:tasks.rados.rados.0.trial115.stdout:2403: finishing write tid 4 to trial11545420-41 2026-02-17T06:00:31.338 INFO:tasks.rados.rados.0.trial115.stdout:2403: oid 41 updating version 0 to 1081 2026-02-17T06:00:31.338 INFO:tasks.rados.rados.0.trial115.stdout:2403: oid 41 version 1081 is already newer than 1079 2026-02-17T06:00:31.338 INFO:tasks.rados.rados.0.trial115.stdout:2403: oid 41 version 1081 is already newer than 1080 2026-02-17T06:00:31.338 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1081 (ObjNum 851 snap 244 seq_num 851) dirty exists 2026-02-17T06:00:31.338 INFO:tasks.rados.rados.0.trial115.stdout:2403: left oid 41 (ObjNum 851 snap 244 seq_num 851) 2026-02-17T06:00:31.338 INFO:tasks.rados.rados.0.trial115.stdout:2404: finishing write tid 1 to trial11545420-21 2026-02-17T06:00:31.339 INFO:tasks.rados.rados.0.trial115.stdout:2404: finishing write tid 2 to trial11545420-21 2026-02-17T06:00:31.339 INFO:tasks.rados.rados.0.trial115.stdout:2404: finishing write tid 3 to trial11545420-21 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2404: finishing write tid 4 to trial11545420-21 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2404: oid 21 updating version 0 to 1084 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2404: oid 21 version 1084 is already newer than 1082 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2404: oid 21 version 1084 is already newer than 1083 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1084 (ObjNum 852 snap 244 seq_num 852) dirty exists 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2404: left oid 21 (ObjNum 852 snap 244 seq_num 852) 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2403: done (2 left) 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2404: done (1 left) 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2405: done (0 left) 2026-02-17T06:00:31.340 INFO:tasks.rados.rados.0.trial115.stdout:2406: read oid 20 snap -1 2026-02-17T06:00:31.341 INFO:tasks.rados.rados.0.trial115.stdout:2406: expect (ObjNum 840 snap 237 seq_num 840) 2026-02-17T06:00:31.341 INFO:tasks.rados.rados.0.trial115.stdout:2407: write (excl) oid 24 current snap is 244 2026-02-17T06:00:31.341 INFO:tasks.rados.rados.0.trial115.stdout:2407: seq_num 854 ranges {753178=494182,1857387=495791} 2026-02-17T06:00:31.343 INFO:tasks.rados.rados.0.trial115.stdout:2407: writing trial11545420-24 from 753178 to 1247360 tid 1 2026-02-17T06:00:31.348 INFO:tasks.rados.rados.0.trial115.stdout:2407: writing trial11545420-24 from 1857387 to 2353178 tid 2 2026-02-17T06:00:31.348 INFO:tasks.rados.rados.0.trial115.stdout:2408: copy_from oid 8 from oid 3 current snap is 244 2026-02-17T06:00:31.348 INFO:tasks.rados.rados.0.trial115.stdout:2406: expect (ObjNum 840 snap 237 seq_num 840) 2026-02-17T06:00:31.423 INFO:tasks.rados.rados.0.trial115.stdout:2407: finishing write tid 1 to trial11545420-24 2026-02-17T06:00:31.423 INFO:tasks.rados.rados.0.trial115.stdout:2406: done (2 left) 2026-02-17T06:00:31.424 INFO:tasks.rados.rados.0.trial115.stdout:2409: snap_remove snap 243 2026-02-17T06:00:32.307 INFO:tasks.rados.rados.0.trial115.stdout:2409: done (2 left) 2026-02-17T06:00:32.307 INFO:tasks.rados.rados.0.trial115.stdout:2410: read oid 48 snap -1 2026-02-17T06:00:32.307 INFO:tasks.rados.rados.0.trial115.stdout:2410: expect deleted 2026-02-17T06:00:32.307 INFO:tasks.rados.rados.0.trial115.stdout:2411: write oid 40 current snap is 244 2026-02-17T06:00:32.307 INFO:tasks.rados.rados.0.trial115.stdout:2411: seq_num 855 ranges {769985=569431,1915869=437966} 2026-02-17T06:00:32.311 INFO:tasks.rados.rados.0.trial115.stdout:2411: writing trial11545420-40 from 769985 to 1339416 tid 1 2026-02-17T06:00:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2411: writing trial11545420-40 from 1915869 to 2353835 tid 2 2026-02-17T06:00:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2412: snap_remove snap 229 2026-02-17T06:00:33.307 INFO:tasks.rados.rados.0.trial115.stdout:2412: done (4 left) 2026-02-17T06:00:33.307 INFO:tasks.rados.rados.0.trial115.stdout:2413: rollback oid 17 current snap is 244 2026-02-17T06:00:33.307 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 241 2026-02-17T06:00:33.307 INFO:tasks.rados.rados.0.trial115.stdout:2407: finishing write tid 2 to trial11545420-24 2026-02-17T06:00:33.307 INFO:tasks.rados.rados.0.trial115.stdout:2407: finishing write tid 3 to trial11545420-24 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2407: finishing write tid 4 to trial11545420-24 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2407: oid 24 updating version 0 to 1346 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2407: oid 24 updating version 1346 to 1348 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2407: oid 24 version 1348 is already newer than 1347 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1348 (ObjNum 854 snap 244 seq_num 854) dirty exists 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2407: left oid 24 (ObjNum 854 snap 244 seq_num 854) 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2408: finishing copy_from racing read to trial11545420-8 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2408: finishing copy_from to trial11545420-8 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1093 (ObjNum 800 snap 230 seq_num 800) dirty exists 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2407: done (4 left) 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2408: done (3 left) 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2414: rollback oid 15 current snap is 244 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 15 to 235 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2411: finishing write tid 1 to trial11545420-40 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2411: finishing write tid 2 to trial11545420-40 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2411: finishing write tid 3 to trial11545420-40 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2411: finishing write tid 4 to trial11545420-40 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2411: oid 40 updating version 0 to 1271 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2411: oid 40 version 1271 is already newer than 1269 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2411: oid 40 version 1271 is already newer than 1270 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1271 (ObjNum 855 snap 244 seq_num 855) dirty exists 2026-02-17T06:00:33.308 INFO:tasks.rados.rados.0.trial115.stdout:2411: left oid 40 (ObjNum 855 snap 244 seq_num 855) 2026-02-17T06:00:33.309 INFO:tasks.rados.rados.0.trial115.stdout:2410: done (3 left) 2026-02-17T06:00:33.309 INFO:tasks.rados.rados.0.trial115.stdout:2411: done (2 left) 2026-02-17T06:00:33.309 INFO:tasks.rados.rados.0.trial115.stdout:2415: snap_create 2026-02-17T06:00:33.310 INFO:tasks.rados.rados.0.trial115.stdout:2414: finishing rollback tid 1 to trial11545420-15 2026-02-17T06:00:33.311 INFO:tasks.rados.rados.0.trial115.stdout:2414: finishing rollback tid 2 to trial11545420-15 2026-02-17T06:00:33.311 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1267 (ObjNum 776 snap 219 seq_num 776) dirty exists 2026-02-17T06:00:33.311 INFO:tasks.rados.rados.0.trial115.stdout:2413: finishing rollback tid 0 to trial11545420-17 2026-02-17T06:00:33.312 INFO:tasks.rados.rados.0.trial115.stdout:2413: finishing rollback tid 1 to trial11545420-17 2026-02-17T06:00:33.312 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1069 (ObjNum 129 snap 0 seq_num 1334072368) dirty dne 2026-02-17T06:00:34.308 INFO:tasks.rados.rados.0.trial115.stdout:2413: done (2 left) 2026-02-17T06:00:34.308 INFO:tasks.rados.rados.0.trial115.stdout:2414: done (1 left) 2026-02-17T06:00:34.308 INFO:tasks.rados.rados.0.trial115.stdout:2415: done (0 left) 2026-02-17T06:00:34.309 INFO:tasks.rados.rados.0.trial115.stdout:2416: delete oid 41 current snap is 245 2026-02-17T06:00:34.310 INFO:tasks.rados.rados.0.trial115.stdout:2416: done (0 left) 2026-02-17T06:00:34.310 INFO:tasks.rados.rados.0.trial115.stdout:2417: rollback oid 33 current snap is 245 2026-02-17T06:00:34.311 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 235 2026-02-17T06:00:34.311 INFO:tasks.rados.rados.0.trial115.stdout:2418: snap_create 2026-02-17T06:00:34.313 INFO:tasks.rados.rados.0.trial115.stdout:2417: finishing rollback tid 0 to trial11545420-33 2026-02-17T06:00:34.313 INFO:tasks.rados.rados.0.trial115.stdout:2417: finishing rollback tid 1 to trial11545420-33 2026-02-17T06:00:34.313 INFO:tasks.rados.rados.0.trial115.stdout:2417: finishing rollback tid 2 to trial11545420-33 2026-02-17T06:00:34.313 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1076 (ObjNum 770 snap 218 seq_num 770) dirty exists 2026-02-17T06:00:35.156 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T06:00:35.156 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T06:00:35.156 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T06:00:35.156 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T06:00:35.159 INFO:tasks.ceph.mon.a:Started 2026-02-17T06:00:35.159 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:00:35.160 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.228+0000 7f6217abf640 10 monclient: get_monmap_and_config 2026-02-17T06:00:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.228+0000 7f6217abf640 10 monclient: build_initial_monmap 2026-02-17T06:00:35.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.228+0000 7f6217abf640 10 monclient: monmap: 2026-02-17T06:00:35.231 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:35.231 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:35.231 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:35.229563+0000 2026-02-17T06:00:35.231 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:35.229563+0000 2026-02-17T06:00:35.231 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.228+0000 7f6217abf640 1 Processor -- start 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.228+0000 7f6217abf640 1 -- start start 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.228+0000 7f6217abf640 10 monclient: init 2026-02-17T06:00:35.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f621006c890 0x7f621006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 10 monclient(hunting): picked mon.noname-a con 0x7f621006c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f621009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f621006d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6210135c10 con 0x7f621006d1b0 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621006daf0 con 0x7f621009c4a0 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f62100a0880 con 0x7f621006c890 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6217abf640 10 monclient(hunting): authenticate will time out at 1480.783100s 2026-02-17T06:00:35.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f621006c890 0x7f621006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6216035640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f621006d1b0 0x7f6210135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:47242/0 (socket says 10.20.193.78:47242) 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6216035640 1 -- 10.20.193.78:0/1875283327 learned_addr learned my addr 10.20.193.78:0/1875283327 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 10 monclient(hunting): get_auth_request con 0x7f621006c890 auth_method 0 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 1 -- 10.20.193.78:0/1875283327 >> v2:10.20.193.78:3300/0 conn(0x7f621006c890 msgr2=0x7f621006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 1 -- 10.20.193.78:0/1875283327 >> v2:10.20.193.78:3300/0 conn(0x7f621006c890 msgr2=0x7f621006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 1 --2- 10.20.193.78:0/1875283327 >> v2:10.20.193.78:3300/0 conn(0x7f621006c890 0x7f621006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:35.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.229+0000 7f6215834640 1 --2- 10.20.193.78:0/1875283327 >> v2:10.20.193.78:3300/0 conn(0x7f621006c890 0x7f621006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.231+0000 7f6216035640 1 -- 10.20.193.78:0/1875283327 >> v1:10.20.193.78:6789/0 conn(0x7f621006d1b0 legacy=0x7f6210135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.231+0000 7f6215033640 1 -- 10.20.193.78:0/1875283327 >> v1:10.20.193.115:6789/0 conn(0x7f621009c4a0 legacy=0x7f6210134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.231+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930814115 0 0) 0x7f621006daf0 con 0x7f621009c4a0 2026-02-17T06:00:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.231+0000 7f6214832640 10 monclient(hunting): _negotiate 2026-02-17T06:00:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.231+0000 7f6214832640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.231+0000 7f6214832640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.231+0000 7f6214832640 10 monclient(hunting): my global_id is 134446 2026-02-17T06:00:35.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.231+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61f8003620 con 0x7f621009c4a0 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4018404762 0 0) 0x7f61f8003620 con 0x7f621009c4a0 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 >> v2:10.20.193.78:3300/0 conn(0x7f621006c890 msgr2=0x7f621006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 --2- 10.20.193.78:0/1875283327 >> v2:10.20.193.78:3300/0 conn(0x7f621006c890 0x7f621006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 >> v1:10.20.193.78:6789/0 conn(0x7f621006d1b0 legacy=0x7f6210135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: _finish_hunting 0 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621009cc80 con 0x7f621009c4a0 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: _finish_auth 0 2026-02-17T06:00:35.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: _check_auth_tickets 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f6204003200 con 0x7f621009c4a0 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: dump: 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:35.248 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 >> v1:10.20.193.115:6789/0 conn(0x7f621009c4a0 legacy=0x7f6210134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f61f8003c20 0x7f61f8004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient(hunting): picked mon.a con 0x7f61f8003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f61f8004900 0x7f61f8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient(hunting): picked mon.b con 0x7f61f8004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:35.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f61f8009ab0 0x7f61f800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient(hunting): picked mon.c con 0x7f61f8009ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61f800c490 con 0x7f61f8003c20 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61f800c640 con 0x7f61f8004900 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f61f800c840 con 0x7f61f8009ab0 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6214832640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f61f8003c20 0x7f61f8004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient(hunting): get_auth_request con 0x7f61f8003c20 auth_method 0 2026-02-17T06:00:35.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f61f8009ab0 0x7f61f800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 10 monclient(hunting): get_auth_request con 0x7f61f8009ab0 auth_method 0 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:35.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient(hunting): handle_auth_done global_id 134446 payload 293 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 1 -- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f61f8009ab0 msgr2=0x7f61f800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f61f8009ab0 0x7f61f800bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 1 -- 10.20.193.78:0/1875283327 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f61f8004900 msgr2=0x7f61f8004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f61f8004900 0x7f61f8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient: _finish_hunting 0 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 1 monclient: found mon.a 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6216035640 1 -- 10.20.193.78:0/1875283327 reap_dead start 2026-02-17T06:00:35.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 1 -- 10.20.193.78:0/1875283327 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f8003620 con 0x7f61f8003c20 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient: _finish_auth 0 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 10 monclient: _check_auth_tickets 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6215834640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6217abf640 5 monclient: authenticate success, global_id 134446 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.232+0000 7f6217abf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6215834640 1 -- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f61f8003c20 msgr2=0x7f61f8004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6215834640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f61f8003c20 0x7f61f8004040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62000098c0 tx=0x7f620002ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bf7a7b0f06b18a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620003c070 con 0x7f61f8003c20 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 10 monclient: dump: 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:35.253 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620002fb60 con 0x7f61f8003c20 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6217abf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:35.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 1 -- 10.20.193.78:0/1875283327 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000f040 con 0x7f61f8003c20 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6214832640 10 monclient: dump: 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.255 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6217abf640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.234+0000 7f6217abf640 4 set_mon_vals no callback set 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 10 monclient: shutdown 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 1 -- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f61f8003c20 msgr2=0x7f61f8004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f61f8003c20 0x7f61f8004040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62000098c0 tx=0x7f620002ef30 comp rx=0 tx=0).stop 2026-02-17T06:00:35.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 1 -- 10.20.193.78:0/1875283327 shutdown_connections 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 1 --2- 10.20.193.78:0/1875283327 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f61f8003c20 0x7f61f8004040 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 1 -- 10.20.193.78:0/1875283327 >> 10.20.193.78:0/1875283327 conn(0x7f62100a6620 msgr2=0x7f62100a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 1 -- 10.20.193.78:0/1875283327 shutdown_connections 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 1 -- 10.20.193.78:0/1875283327 wait complete. 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 10 monclient: build_initial_monmap 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.235+0000 7f6217abf640 10 monclient: monmap: 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:35.257 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 1 Processor -- start 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 1 -- start start 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient: init 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f61f8003c20 0x7f62101cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient(hunting): picked mon.noname-b con 0x7f61f8003c20 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f62101cfbe0 0x7f621009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:35.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient(hunting): picked mon.noname-c con 0x7f62101cfbe0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 0x7f621009e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient(hunting): picked mon.noname-a con 0x7f621009e090 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62100a0020 con 0x7f621009e090 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f621006d640 con 0x7f61f8003c20 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f62100a01a0 con 0x7f62101cfbe0 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6217abf640 10 monclient(hunting): authenticate will time out at 1480.790377s 2026-02-17T06:00:35.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f62101cfbe0 0x7f621009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 0x7f621009e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f62101cfbe0 0x7f621009db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54702/0 (socket says 10.20.193.78:54702) 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 -- 10.20.193.78:0/1989287854 learned_addr learned my addr 10.20.193.78:0/1989287854 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient(hunting): get_auth_request con 0x7f62101cfbe0 auth_method 0 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 10 monclient(hunting): get_auth_request con 0x7f621009e090 auth_method 0 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:35.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 1 -- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 msgr2=0x7f621009e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 1 -- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 msgr2=0x7f621009e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 0x7f621009e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215834640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f61f8003c20 0x7f62101cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6216035640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 0x7f621009e4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215834640 10 monclient(hunting): get_auth_request con 0x7f61f8003c20 auth_method 0 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215834640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215834640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215834640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:35.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient(hunting): handle_auth_done global_id 144464 payload 824 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 -- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f61f8003c20 msgr2=0x7f62101cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f61f8003c20 0x7f62101cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 -- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 msgr2=0x7f621009e4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 0x7f621009e4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient: _finish_hunting 0 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 monclient: found mon.noname-c 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.236+0000 7f6215033640 1 -- 10.20.193.78:0/1989287854 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621009f7f0 con 0x7f62101cfbe0 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6215033640 1 -- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f62101cfbe0 msgr2=0x7f621009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6215033640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f62101cfbe0 0x7f621009db50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f620400c730 tx=0x7f620400cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:35.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 1 -- 10.20.193.78:0/1989287854 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204017070 con 0x7f62101cfbe0 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: dump: 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:35.263 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: _finish_auth 0 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 1 -- 10.20.193.78:0/1989287854 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6204005ce0 con 0x7f62101cfbe0 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 1 -- 10.20.193.78:0/1989287854 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040072b0 con 0x7f62101cfbe0 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fffff640 4 set_mon_vals no callback set 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f61fe7fc640 10 monclient: dump: 2026-02-17T06:00:35.264 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6217abf640 5 monclient: authenticate success, global_id 144464 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6217abf640 10 monclient: _renew_subs 2026-02-17T06:00:35.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6217abf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62100a2610 con 0x7f62101cfbe0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6217abf640 10 monclient: _renew_subs 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6217abf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.238+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62100a20c0 con 0x7f62101cfbe0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f6217abf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f6217abf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f61fe7fc640 1 -- 10.20.193.78:0/1989287854 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6204007450 con 0x7f62101cfbe0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f6217abf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621009beb0 con 0x7f62101cfbe0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f61fe7fc640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f61e003dee0 0x7f61e00403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f61fe7fc640 1 -- 10.20.193.78:0/1989287854 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(518..518 src has 335..518) ==== 4783+0+0 (secure 0 0 0) 0x7f6204051f70 con 0x7f62101cfbe0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f6215834640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f61e003dee0 0x7f61e00403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.239+0000 7f6215834640 10 monclient: get_auth_request con 0x7f61e003dee0 auth_method 0 2026-02-17T06:00:35.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.242+0000 7f61fe7fc640 1 -- 10.20.193.78:0/1989287854 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6204050970 con 0x7f62101cfbe0 2026-02-17T06:00:35.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.242+0000 7f61fe7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:35.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.242+0000 7f61fe7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:35.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.242+0000 7f6215834640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f61e003dee0 0x7f61e00403a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6200036000 tx=0x7f6200033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:35.310 INFO:tasks.rados.rados.0.trial115.stdout:2417: done (1 left) 2026-02-17T06:00:35.311 INFO:tasks.rados.rados.0.trial115.stdout:2418: done (0 left) 2026-02-17T06:00:35.311 INFO:tasks.rados.rados.0.trial115.stdout:2419: read oid 16 snap -1 2026-02-17T06:00:35.311 INFO:tasks.rados.rados.0.trial115.stdout:2419: expect (ObjNum 824 snap 236 seq_num 824) 2026-02-17T06:00:35.311 INFO:tasks.rados.rados.0.trial115.stdout:2420: read oid 3 snap 245 2026-02-17T06:00:35.311 INFO:tasks.rados.rados.0.trial115.stdout:2420: expect (ObjNum 800 snap 230 seq_num 800) 2026-02-17T06:00:35.311 INFO:tasks.rados.rados.0.trial115.stdout:2421: write oid 47 current snap is 246 2026-02-17T06:00:35.311 INFO:tasks.rados.rados.0.trial115.stdout:2421: seq_num 857 ranges {403599=719929,1916480=441635} 2026-02-17T06:00:35.314 INFO:tasks.rados.rados.0.trial115.stdout:2421: writing trial11545420-47 from 403599 to 1123528 tid 1 2026-02-17T06:00:35.319 INFO:tasks.rados.rados.0.trial115.stdout:2421: writing trial11545420-47 from 1916480 to 2358115 tid 2 2026-02-17T06:00:35.320 INFO:tasks.rados.rados.0.trial115.stdout:2422: read oid 29 snap -1 2026-02-17T06:00:35.320 INFO:tasks.rados.rados.0.trial115.stdout:2422: expect deleted 2026-02-17T06:00:35.320 INFO:tasks.rados.rados.0.trial115.stdout:2423: write oid 12 current snap is 246 2026-02-17T06:00:35.320 INFO:tasks.rados.rados.0.trial115.stdout:2423: seq_num 858 ranges {420406=795178,1974962=445444} 2026-02-17T06:00:35.324 INFO:tasks.rados.rados.0.trial115.stdout:2423: writing trial11545420-12 from 420406 to 1215584 tid 1 2026-02-17T06:00:35.329 INFO:tasks.rados.rados.0.trial115.stdout:2423: writing trial11545420-12 from 1974962 to 2420406 tid 2 2026-02-17T06:00:35.330 INFO:tasks.rados.rados.0.trial115.stdout:2424: write oid 36 current snap is 246 2026-02-17T06:00:35.330 INFO:tasks.rados.rados.0.trial115.stdout:2424: seq_num 859 ranges {437213=470427,1633444=728951} 2026-02-17T06:00:35.333 INFO:tasks.rados.rados.0.trial115.stdout:2424: writing trial11545420-36 from 437213 to 907640 tid 1 2026-02-17T06:00:35.339 INFO:tasks.rados.rados.0.trial115.stdout:2424: writing trial11545420-36 from 1633444 to 2362395 tid 2 2026-02-17T06:00:35.339 INFO:tasks.rados.rados.0.trial115.stdout:2425: read oid 34 snap 244 2026-02-17T06:00:35.339 INFO:tasks.rados.rados.0.trial115.stdout:2425: expect deleted 2026-02-17T06:00:35.339 INFO:tasks.rados.rados.0.trial115.stdout:2426: write (excl) oid 41 current snap is 246 2026-02-17T06:00:35.339 INFO:tasks.rados.rados.0.trial115.stdout:2426: seq_num 860 ranges {454020=662029,1691926=588962} 2026-02-17T06:00:35.343 INFO:tasks.rados.rados.0.trial115.stdout:2426: writing trial11545420-41 from 454020 to 1116049 tid 1 2026-02-17T06:00:35.348 INFO:tasks.rados.rados.0.trial115.stdout:2426: writing trial11545420-41 from 1691926 to 2280888 tid 2 2026-02-17T06:00:35.348 INFO:tasks.rados.rados.0.trial115.stdout:2427: read oid 45 snap -1 2026-02-17T06:00:35.348 INFO:tasks.rados.rados.0.trial115.stdout:2427: expect (ObjNum 817 snap 235 seq_num 817) 2026-02-17T06:00:35.348 INFO:tasks.rados.rados.0.trial115.stdout:2428: rollback oid 14 current snap is 246 2026-02-17T06:00:35.348 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 235 2026-02-17T06:00:35.348 INFO:tasks.rados.rados.0.trial115.stdout:2420: expect (ObjNum 800 snap 230 seq_num 800) 2026-02-17T06:00:35.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.357+0000 7f6217abf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:35.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.357+0000 7f6217abf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:35.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.357+0000 7f6217abf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:35.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.357+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6210125fe0 con 0x7f62101cfbe0 2026-02-17T06:00:35.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.358+0000 7f61fe7fc640 1 -- 10.20.193.78:0/1989287854 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f620401ac80 con 0x7f62101cfbe0 2026-02-17T06:00:35.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.358+0000 7f61fe7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:35.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.358+0000 7f61fe7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:35.359 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":166,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f61e003dee0 msgr2=0x7f61e00403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f61e003dee0 0x7f61e00403a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6200036000 tx=0x7f6200033000 comp rx=0 tx=0).stop 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 10 monclient: shutdown 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f62101cfbe0 msgr2=0x7f621009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f62101cfbe0 0x7f621009db50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f620400c730 tx=0x7f620400cc00 comp rx=0 tx=0).stop 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 shutdown_connections 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f621009e090 0x7f621009e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f62101cfbe0 0x7f621009db50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f61e003dee0 0x7f61e00403a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 --2- 10.20.193.78:0/1989287854 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f61f8003c20 0x7f62101cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:35.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 >> 10.20.193.78:0/1989287854 conn(0x7f62100a6620 msgr2=0x7f6210072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:35.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 shutdown_connections 2026-02-17T06:00:35.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:35.359+0000 7f6217abf640 1 -- 10.20.193.78:0/1989287854 wait complete. 2026-02-17T06:00:35.468 INFO:tasks.rados.rados.0.trial115.stdout:2421: finishing write tid 1 to trial11545420-47 2026-02-17T06:00:35.469 INFO:tasks.rados.rados.0.trial115.stdout:2419: expect (ObjNum 824 snap 236 seq_num 824) 2026-02-17T06:00:35.590 INFO:tasks.rados.rados.0.trial115.stdout:2421: finishing write tid 2 to trial11545420-47 2026-02-17T06:00:35.590 INFO:tasks.rados.rados.0.trial115.stdout:2421: finishing write tid 3 to trial11545420-47 2026-02-17T06:00:35.590 INFO:tasks.rados.rados.0.trial115.stdout:2421: finishing write tid 4 to trial11545420-47 2026-02-17T06:00:35.590 INFO:tasks.rados.rados.0.trial115.stdout:2421: oid 47 updating version 0 to 1164 2026-02-17T06:00:35.590 INFO:tasks.rados.rados.0.trial115.stdout:2421: oid 47 version 1164 is already newer than 1163 2026-02-17T06:00:35.590 INFO:tasks.rados.rados.0.trial115.stdout:2421: oid 47 version 1164 is already newer than 1162 2026-02-17T06:00:35.590 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1164 (ObjNum 857 snap 246 seq_num 857) dirty exists 2026-02-17T06:00:35.590 INFO:tasks.rados.rados.0.trial115.stdout:2421: left oid 47 (ObjNum 857 snap 246 seq_num 857) 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2423: finishing write tid 1 to trial11545420-12 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2423: finishing write tid 2 to trial11545420-12 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2423: finishing write tid 3 to trial11545420-12 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2423: finishing write tid 4 to trial11545420-12 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2423: oid 12 updating version 0 to 1358 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2423: oid 12 version 1358 is already newer than 1357 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2423: oid 12 updating version 1358 to 1359 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1359 (ObjNum 858 snap 246 seq_num 858) dirty exists 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2423: left oid 12 (ObjNum 858 snap 246 seq_num 858) 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2424: finishing write tid 1 to trial11545420-36 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2424: finishing write tid 2 to trial11545420-36 2026-02-17T06:00:35.591 INFO:tasks.rados.rados.0.trial115.stdout:2424: finishing write tid 3 to trial11545420-36 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:2424: finishing write tid 4 to trial11545420-36 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:2424: oid 36 updating version 0 to 1078 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:2424: oid 36 updating version 1078 to 1079 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:2424: oid 36 version 1079 is already newer than 1077 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 1079 (ObjNum 859 snap 246 seq_num 859) dirty exists 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:2424: left oid 36 (ObjNum 859 snap 246 seq_num 859) 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:2426: finishing write tid 1 to trial11545420-41 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:2428: finishing rollback tid 1 to trial11545420-14 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1073 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-17T06:00:35.592 INFO:tasks.rados.rados.0.trial115.stdout:2426: finishing write tid 2 to trial11545420-41 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2426: finishing write tid 3 to trial11545420-41 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2419: done (9 left) 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2420: done (8 left) 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2421: done (7 left) 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2422: done (6 left) 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2423: done (5 left) 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2424: done (4 left) 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2425: done (3 left) 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2428: done (2 left) 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2429: rollback oid 44 current snap is 246 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 44 to 245 2026-02-17T06:00:35.593 INFO:tasks.rados.rados.0.trial115.stdout:2426: finishing write tid 4 to trial11545420-41 2026-02-17T06:00:35.594 INFO:tasks.rados.rados.0.trial115.stdout:2426: oid 41 updating version 0 to 1086 2026-02-17T06:00:35.594 INFO:tasks.rados.rados.0.trial115.stdout:2426: oid 41 updating version 1086 to 1087 2026-02-17T06:00:35.594 INFO:tasks.rados.rados.0.trial115.stdout:2426: oid 41 updating version 1087 to 1088 2026-02-17T06:00:35.594 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1088 (ObjNum 860 snap 246 seq_num 860) dirty exists 2026-02-17T06:00:35.594 INFO:tasks.rados.rados.0.trial115.stdout:2426: left oid 41 (ObjNum 860 snap 246 seq_num 860) 2026-02-17T06:00:35.594 INFO:tasks.rados.rados.0.trial115.stdout:2427: expect (ObjNum 817 snap 235 seq_num 817) 2026-02-17T06:00:35.775 INFO:tasks.rados.rados.0.trial115.stdout:2429: finishing rollback tid 0 to trial11545420-44 2026-02-17T06:00:35.776 INFO:tasks.rados.rados.0.trial115.stdout:2429: finishing rollback tid 1 to trial11545420-44 2026-02-17T06:00:35.776 INFO:tasks.rados.rados.0.trial115.stdout:2429: finishing rollback tid 2 to trial11545420-44 2026-02-17T06:00:35.776 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 959 (ObjNum 789 snap 224 seq_num 789) dirty exists 2026-02-17T06:00:35.776 INFO:tasks.rados.rados.0.trial115.stdout:2426: done (2 left) 2026-02-17T06:00:35.776 INFO:tasks.rados.rados.0.trial115.stdout:2427: done (1 left) 2026-02-17T06:00:35.776 INFO:tasks.rados.rados.0.trial115.stdout:2429: done (0 left) 2026-02-17T06:00:35.776 INFO:tasks.rados.rados.0.trial115.stdout:2430: snap_remove snap 245 2026-02-17T06:00:36.314 INFO:tasks.rados.rados.0.trial115.stdout:2430: done (0 left) 2026-02-17T06:00:36.315 INFO:tasks.rados.rados.0.trial115.stdout:2431: delete oid 43 current snap is 246 2026-02-17T06:00:36.318 INFO:tasks.rados.rados.0.trial115.stdout:2431: done (0 left) 2026-02-17T06:00:36.318 INFO:tasks.rados.rados.0.trial115.stdout:2432: snap_remove snap 244 2026-02-17T06:00:37.306 INFO:tasks.rados.rados.0.trial115.stdout:2432: done (0 left) 2026-02-17T06:00:37.306 INFO:tasks.rados.rados.0.trial115.stdout:2433: read oid 39 snap 235 2026-02-17T06:00:37.306 INFO:tasks.rados.rados.0.trial115.stdout:2433: expect (ObjNum 816 snap 235 seq_num 816) 2026-02-17T06:00:37.306 INFO:tasks.rados.rados.0.trial115.stdout:2434: write (excl) oid 44 current snap is 246 2026-02-17T06:00:37.307 INFO:tasks.rados.rados.0.trial115.stdout:2434: seq_num 862 ranges {487634=412527,1408890=592631} 2026-02-17T06:00:37.310 INFO:tasks.rados.rados.0.trial115.stdout:2434: writing trial11545420-44 from 487634 to 900161 tid 1 2026-02-17T06:00:37.314 INFO:tasks.rados.rados.0.trial115.stdout:2434: writing trial11545420-44 from 1408890 to 2001521 tid 2 2026-02-17T06:00:37.314 INFO:tasks.rados.rados.0.trial115.stdout:2435: read oid 21 snap 235 2026-02-17T06:00:37.315 INFO:tasks.rados.rados.0.trial115.stdout:2435: expect (ObjNum 802 snap 231 seq_num 802) 2026-02-17T06:00:37.315 INFO:tasks.rados.rados.0.trial115.stdout:2436: snap_remove snap 241 2026-02-17T06:00:38.317 INFO:tasks.rados.rados.0.trial115.stdout:2436: done (3 left) 2026-02-17T06:00:38.317 INFO:tasks.rados.rados.0.trial115.stdout:2437: delete oid 16 current snap is 246 2026-02-17T06:00:38.317 INFO:tasks.rados.rados.0.trial115.stdout:2433: expect (ObjNum 816 snap 235 seq_num 816) 2026-02-17T06:00:38.369 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:38.399 INFO:tasks.rados.rados.0.trial115.stdout:2434: finishing write tid 1 to trial11545420-44 2026-02-17T06:00:38.399 INFO:tasks.rados.rados.0.trial115.stdout:2434: finishing write tid 2 to trial11545420-44 2026-02-17T06:00:38.399 INFO:tasks.rados.rados.0.trial115.stdout:2434: finishing write tid 3 to trial11545420-44 2026-02-17T06:00:38.399 INFO:tasks.rados.rados.0.trial115.stdout:2434: finishing write tid 4 to trial11545420-44 2026-02-17T06:00:38.399 INFO:tasks.rados.rados.0.trial115.stdout:2434: oid 44 updating version 0 to 962 2026-02-17T06:00:38.399 INFO:tasks.rados.rados.0.trial115.stdout:2434: oid 44 version 962 is already newer than 961 2026-02-17T06:00:38.399 INFO:tasks.rados.rados.0.trial115.stdout:2434: oid 44 version 962 is already newer than 960 2026-02-17T06:00:38.400 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 962 (ObjNum 862 snap 246 seq_num 862) dirty exists 2026-02-17T06:00:38.400 INFO:tasks.rados.rados.0.trial115.stdout:2434: left oid 44 (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:00:38.400 INFO:tasks.rados.rados.0.trial115.stdout:2435: expect (ObjNum 802 snap 231 seq_num 802) 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.436+0000 7f73c6fa0640 10 monclient: get_monmap_and_config 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.436+0000 7f73c6fa0640 10 monclient: build_initial_monmap 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.436+0000 7f73c6fa0640 10 monclient: monmap: 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:38.438098+0000 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:38.438098+0000 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:38.438 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:38.439 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:38.439 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:38.439 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:38.439 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:38.439 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:38.439 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:38.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 1 Processor -- start 2026-02-17T06:00:38.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 1 -- start start 2026-02-17T06:00:38.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient: init 2026-02-17T06:00:38.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:38.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c006d140 0x7f73c009be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient(hunting): picked mon.noname-b con 0x7f73c006d140 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f73c009f4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f73c0139e20 0x7f73c013c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient(hunting): picked mon.noname-c con 0x7f73c0139e20 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73c009d340 con 0x7f73c009f4a0 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f73c009f060 con 0x7f73c0139e20 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f73c006d520 con 0x7f73c006d140 2026-02-17T06:00:38.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c6fa0640 10 monclient(hunting): authenticate will time out at 1483.991768s 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c006d140 0x7f73c009be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c006d140 0x7f73c009be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43128/0 (socket says 10.20.193.78:43128) 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 1 -- 10.20.193.78:0/1944898164 learned_addr learned my addr 10.20.193.78:0/1944898164 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient(hunting): get_auth_request con 0x7f73c006d140 auth_method 0 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:38.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c5516640 1 --2- 10.20.193.78:0/1944898164 >> v2:10.20.193.115:3300/0 conn(0x7f73c0139e20 0x7f73c013c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient(hunting): handle_auth_done global_id 144467 payload 274 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 1 -- 10.20.193.78:0/1944898164 >> v2:10.20.193.115:3300/0 conn(0x7f73c0139e20 msgr2=0x7f73c013c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 1 --2- 10.20.193.78:0/1944898164 >> v2:10.20.193.115:3300/0 conn(0x7f73c0139e20 0x7f73c013c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 1 -- 10.20.193.78:0/1944898164 >> v1:10.20.193.115:6789/0 conn(0x7f73c009f4a0 legacy=0x7f73c009c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 10 monclient: _finish_hunting 0 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.437+0000 7f73c4d15640 1 monclient: found mon.noname-b 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.438+0000 7f73c4d15640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.438+0000 7f73c4d15640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.438+0000 7f73c4d15640 1 -- 10.20.193.78:0/1944898164 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c009e750 con 0x7f73c006d140 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.438+0000 7f73c5516640 10 monclient: get_auth_request con 0x7f73c0139e20 auth_method 0 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.438+0000 7f73c5516640 1 --2- 10.20.193.78:0/1944898164 >> v2:10.20.193.115:3300/0 conn(0x7f73c0139e20 0x7f73c013c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:00:38.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c4d15640 1 -- 10.20.193.78:0/1944898164 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c006d140 msgr2=0x7f73c009be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c4d15640 1 --2- 10.20.193.78:0/1944898164 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c006d140 0x7f73c009be20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009970 tx=0x7f73b002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a0fcf60dfc5d7d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 1 -- 10.20.193.78:0/1944898164 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b003c070 con 0x7f73c006d140 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: dump: 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:38.453 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 1 -- 10.20.193.78:0/1944898164 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b002fdb0 con 0x7f73c006d140 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 5 monclient: authenticate success, global_id 144467 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 1 -- 10.20.193.78:0/1944898164 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0037660 con 0x7f73c006d140 2026-02-17T06:00:38.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 4 set_mon_vals no callback set 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73b77fe640 10 monclient: dump: 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:38.455 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 10 monclient: shutdown 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 1 -- 10.20.193.78:0/1944898164 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c006d140 msgr2=0x7f73c009be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/1944898164 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c006d140 0x7f73c009be20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009970 tx=0x7f73b002ee90 comp rx=0 tx=0).stop 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 1 -- 10.20.193.78:0/1944898164 shutdown_connections 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/1944898164 >> v2:10.20.193.115:3300/0 conn(0x7f73c0139e20 0x7f73c013c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/1944898164 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c006d140 0x7f73c009be20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 1 -- 10.20.193.78:0/1944898164 >> 10.20.193.78:0/1944898164 conn(0x7f73c00a6620 msgr2=0x7f73c00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 1 -- 10.20.193.78:0/1944898164 shutdown_connections 2026-02-17T06:00:38.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.440+0000 7f73c6fa0640 1 -- 10.20.193.78:0/1944898164 wait complete. 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient: build_initial_monmap 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient: monmap: 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:38.457 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 1 Processor -- start 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 1 -- start start 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient: init 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 0x7f73c01348e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient(hunting): picked mon.noname-a con 0x7f73c006d140 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73c009f4a0 0x7f73c0134e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient(hunting): picked mon.noname-b con 0x7f73c009f4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c0139e20 0x7f73c0135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient(hunting): picked mon.noname-c con 0x7f73c0139e20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:38.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73c00a0ae0 con 0x7f73c006d140 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f73c00a0960 con 0x7f73c009f4a0 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f73c009ca70 con 0x7f73c0139e20 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c6fa0640 10 monclient(hunting): authenticate will time out at 1483.995662s 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c0139e20 0x7f73c0135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 0x7f73c01348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c0139e20 0x7f73c0135360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43140/0 (socket says 10.20.193.78:43140) 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 -- 10.20.193.78:0/2757322439 learned_addr learned my addr 10.20.193.78:0/2757322439 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient(hunting): get_auth_request con 0x7f73c0139e20 auth_method 0 2026-02-17T06:00:38.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 10 monclient(hunting): get_auth_request con 0x7f73c006d140 auth_method 0 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 1 -- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 msgr2=0x7f73c01348e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 1 -- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 msgr2=0x7f73c01348e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:00:38.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 0x7f73c01348e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c4d15640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 0x7f73c01348e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient(hunting): handle_auth_done global_id 144470 payload 824 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 -- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73c009f4a0 msgr2=0x7f73c0134e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73b7fff640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73c009f4a0 0x7f73c0134e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73c009f4a0 0x7f73c0134e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 -- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 msgr2=0x7f73c01348e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 0x7f73c01348e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient: _finish_hunting 0 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 monclient: found mon.noname-c 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:38.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73c5516640 1 -- 10.20.193.78:0/2757322439 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c00a2530 con 0x7f73c0139e20 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73b7fff640 10 monclient: get_auth_request con 0x7f73c009f4a0 auth_method 0 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.441+0000 7f73b7fff640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73c009f4a0 0x7f73c0134e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c5516640 1 -- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c0139e20 msgr2=0x7f73c0135360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c5516640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c0139e20 0x7f73c0135360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00ce40 tx=0x7f73bc00a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 1 -- 10.20.193.78:0/2757322439 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc01a070 con 0x7f73c0139e20 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: dump: 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:38.462 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: _finish_auth 0 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 1 -- 10.20.193.78:0/2757322439 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73bc00da50 con 0x7f73c0139e20 2026-02-17T06:00:38.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 5 monclient: authenticate success, global_id 144470 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 1 -- 10.20.193.78:0/2757322439 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00de40 con 0x7f73c0139e20 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b6ffd640 4 set_mon_vals no callback set 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 10 monclient: dump: 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:38.464 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 10 monclient: _renew_subs 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c0138260 con 0x7f73c0139e20 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 10 monclient: _renew_subs 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c0137f80 con 0x7f73c0139e20 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:38.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c006c890 con 0x7f73c0139e20 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.444+0000 7f73b57fa640 1 -- 10.20.193.78:0/2757322439 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f73bc01e3b0 con 0x7f73c0139e20 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.445+0000 7f73b57fa640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f739003deb0 0x7f7390040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.445+0000 7f73b57fa640 1 -- 10.20.193.78:0/2757322439 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(522..522 src has 335..522) ==== 4795+0+0 (secure 0 0 0) 0x7f73bc053320 con 0x7f73c0139e20 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.445+0000 7f73c4d15640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f739003deb0 0x7f7390040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.445+0000 7f73c4d15640 10 monclient: get_auth_request con 0x7f739003deb0 auth_method 0 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.447+0000 7f73b57fa640 1 -- 10.20.193.78:0/2757322439 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f73c00a2530 con 0x7f73c0139e20 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.447+0000 7f73b57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.447+0000 7f73b57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:38.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.447+0000 7f73c4d15640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f739003deb0 0x7f7390040370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f73b0036000 tx=0x7f73b0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:38.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.561+0000 7f73c6fa0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:38.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.561+0000 7f73c6fa0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:38.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.561+0000 7f73c6fa0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:38.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.561+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f73c0136e10 con 0x7f73c0139e20 2026-02-17T06:00:38.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.561+0000 7f73b57fa640 1 -- 10.20.193.78:0/2757322439 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f73bc053080 con 0x7f73c0139e20 2026-02-17T06:00:38.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.561+0000 7f73b57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:38.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.561+0000 7f73b57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:38.563 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":166,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f739003deb0 msgr2=0x7f7390040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f739003deb0 0x7f7390040370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f73b0036000 tx=0x7f73b0033000 comp rx=0 tx=0).stop 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 10 monclient: shutdown 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c0139e20 msgr2=0x7f73c0135360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c0139e20 0x7f73c0135360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00ce40 tx=0x7f73bc00a710 comp rx=0 tx=0).stop 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 shutdown_connections 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73c0139e20 0x7f73c0135360 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f739003deb0 0x7f7390040370 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73c009f4a0 0x7f73c0134e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 --2- 10.20.193.78:0/2757322439 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73c006d140 0x7f73c01348e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:38.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 >> 10.20.193.78:0/2757322439 conn(0x7f73c00a6620 msgr2=0x7f73c01394e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:38.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 shutdown_connections 2026-02-17T06:00:38.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:38.562+0000 7f73c6fa0640 1 -- 10.20.193.78:0/2757322439 wait complete. 2026-02-17T06:00:38.578 INFO:tasks.rados.rados.0.trial115.stdout:2433: done (3 left) 2026-02-17T06:00:38.578 INFO:tasks.rados.rados.0.trial115.stdout:2434: done (2 left) 2026-02-17T06:00:38.578 INFO:tasks.rados.rados.0.trial115.stdout:2435: done (1 left) 2026-02-17T06:00:38.579 INFO:tasks.rados.rados.0.trial115.stdout:2437: done (0 left) 2026-02-17T06:00:38.579 INFO:tasks.rados.rados.0.trial115.stdout:2438: read oid 29 snap -1 2026-02-17T06:00:38.579 INFO:tasks.rados.rados.0.trial115.stdout:2438: expect deleted 2026-02-17T06:00:38.579 INFO:tasks.rados.rados.0.trial115.stdout:2439: rollback oid 31 current snap is 246 2026-02-17T06:00:38.579 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 31 to 235 2026-02-17T06:00:38.579 INFO:tasks.rados.rados.0.trial115.stdout:2440: copy_from oid 42 from oid 44 current snap is 246 2026-02-17T06:00:38.579 INFO:tasks.rados.rados.0.trial115.stdout:2441: delete oid 22 current snap is 246 2026-02-17T06:00:38.580 INFO:tasks.rados.rados.0.trial115.stdout:2438: done (3 left) 2026-02-17T06:00:38.580 INFO:tasks.rados.rados.0.trial115.stdout:2441: done (2 left) 2026-02-17T06:00:38.580 INFO:tasks.rados.rados.0.trial115.stdout:2442: delete oid 11 current snap is 246 2026-02-17T06:00:38.581 INFO:tasks.rados.rados.0.trial115.stdout:2439: finishing rollback tid 1 to trial11545420-31 2026-02-17T06:00:38.581 INFO:tasks.rados.rados.0.trial115.stdout:2439: finishing rollback tid 2 to trial11545420-31 2026-02-17T06:00:38.581 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1170 (ObjNum 804 snap 231 seq_num 804) dirty exists 2026-02-17T06:00:38.582 INFO:tasks.rados.rados.0.trial115.stdout:2439: done (2 left) 2026-02-17T06:00:38.582 INFO:tasks.rados.rados.0.trial115.stdout:2442: done (1 left) 2026-02-17T06:00:38.582 INFO:tasks.rados.rados.0.trial115.stdout:2443: copy_from oid 39 from oid 11 current snap is 246 2026-02-17T06:00:38.582 INFO:tasks.rados.rados.0.trial115.stdout:2444: write (excl) oid 7 current snap is 246 2026-02-17T06:00:38.582 INFO:tasks.rados.rados.0.trial115.stdout:2444: seq_num 866 ranges {554862=713523,1926465=599969} 2026-02-17T06:00:38.586 INFO:tasks.rados.rados.0.trial115.stdout:2444: writing trial11545420-7 from 554862 to 1268385 tid 1 2026-02-17T06:00:38.592 INFO:tasks.rados.rados.0.trial115.stdout:2444: writing trial11545420-7 from 1926465 to 2526434 tid 2 2026-02-17T06:00:38.592 INFO:tasks.rados.rados.0.trial115.stdout:2445: read oid 36 snap -1 2026-02-17T06:00:38.592 INFO:tasks.rados.rados.0.trial115.stdout:2445: expect (ObjNum 859 snap 246 seq_num 859) 2026-02-17T06:00:38.592 INFO:tasks.rados.rados.0.trial115.stdout:2446: snap_create 2026-02-17T06:00:38.592 INFO:tasks.rados.rados.0.trial115.stdout:2443: finishing copy_from to trial11545420-39 2026-02-17T06:00:38.592 INFO:tasks.rados.rados.0.trial115.stdout:2443: got expected ENOENT (src dne) 2026-02-17T06:00:38.593 INFO:tasks.rados.rados.0.trial115.stdout:2443: finishing copy_from racing read to trial11545420-39 2026-02-17T06:00:38.593 INFO:tasks.rados.rados.0.trial115.stdout:2440: finishing copy_from racing read to trial11545420-42 2026-02-17T06:00:38.593 INFO:tasks.rados.rados.0.trial115.stdout:2440: finishing copy_from to trial11545420-42 2026-02-17T06:00:38.593 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 962 (ObjNum 862 snap 246 seq_num 862) dirty exists 2026-02-17T06:00:38.593 INFO:tasks.rados.rados.0.trial115.stdout:2444: finishing write tid 1 to trial11545420-7 2026-02-17T06:00:38.594 INFO:tasks.rados.rados.0.trial115.stdout:2444: finishing write tid 2 to trial11545420-7 2026-02-17T06:00:38.594 INFO:tasks.rados.rados.0.trial115.stdout:2444: finishing write tid 3 to trial11545420-7 2026-02-17T06:00:38.594 INFO:tasks.rados.rados.0.trial115.stdout:2444: finishing write tid 4 to trial11545420-7 2026-02-17T06:00:38.594 INFO:tasks.rados.rados.0.trial115.stdout:2444: oid 7 updating version 0 to 983 2026-02-17T06:00:38.594 INFO:tasks.rados.rados.0.trial115.stdout:2444: oid 7 version 983 is already newer than 982 2026-02-17T06:00:38.595 INFO:tasks.rados.rados.0.trial115.stdout:2444: oid 7 version 983 is already newer than 981 2026-02-17T06:00:38.595 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 983 (ObjNum 866 snap 246 seq_num 866) dirty exists 2026-02-17T06:00:38.595 INFO:tasks.rados.rados.0.trial115.stdout:2444: left oid 7 (ObjNum 866 snap 246 seq_num 866) 2026-02-17T06:00:38.597 INFO:tasks.rados.rados.0.trial115.stdout:2445: expect (ObjNum 859 snap 246 seq_num 859) 2026-02-17T06:00:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2440: done (4 left) 2026-02-17T06:00:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2443: done (3 left) 2026-02-17T06:00:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2444: done (2 left) 2026-02-17T06:00:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2445: done (1 left) 2026-02-17T06:00:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2446: done (0 left) 2026-02-17T06:00:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2447: rollback oid 40 current snap is 247 2026-02-17T06:00:39.321 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 246 2026-02-17T06:00:39.321 INFO:tasks.rados.rados.0.trial115.stdout:2448: snap_remove snap 235 2026-02-17T06:00:40.323 INFO:tasks.rados.rados.0.trial115.stdout:2448: done (1 left) 2026-02-17T06:00:40.324 INFO:tasks.rados.rados.0.trial115.stdout:2449: write (excl) oid 20 current snap is 247 2026-02-17T06:00:40.324 INFO:tasks.rados.rados.0.trial115.stdout:2449: seq_num 867 ranges {571669=505125,1701300=743627} 2026-02-17T06:00:40.328 INFO:tasks.rados.rados.0.trial115.stdout:2449: writing trial11545420-20 from 571669 to 1076794 tid 1 2026-02-17T06:00:40.333 INFO:tasks.rados.rados.0.trial115.stdout:2449: writing trial11545420-20 from 1701300 to 2444927 tid 2 2026-02-17T06:00:40.333 INFO:tasks.rados.rados.0.trial115.stdout:2450: snap_create 2026-02-17T06:00:40.334 INFO:tasks.rados.rados.0.trial115.stdout:2447: finishing rollback tid 0 to trial11545420-40 2026-02-17T06:00:40.334 INFO:tasks.rados.rados.0.trial115.stdout:2447: finishing rollback tid 1 to trial11545420-40 2026-02-17T06:00:40.334 INFO:tasks.rados.rados.0.trial115.stdout:2447: finishing rollback tid 2 to trial11545420-40 2026-02-17T06:00:40.334 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1274 (ObjNum 855 snap 244 seq_num 855) dirty exists 2026-02-17T06:00:40.334 INFO:tasks.rados.rados.0.trial115.stdout:2449: finishing write tid 1 to trial11545420-20 2026-02-17T06:00:40.336 INFO:tasks.rados.rados.0.trial115.stdout:2449: finishing write tid 2 to trial11545420-20 2026-02-17T06:00:40.336 INFO:tasks.rados.rados.0.trial115.stdout:2449: finishing write tid 3 to trial11545420-20 2026-02-17T06:00:40.336 INFO:tasks.rados.rados.0.trial115.stdout:2449: finishing write tid 4 to trial11545420-20 2026-02-17T06:00:40.336 INFO:tasks.rados.rados.0.trial115.stdout:2449: oid 20 updating version 0 to 990 2026-02-17T06:00:40.336 INFO:tasks.rados.rados.0.trial115.stdout:2449: oid 20 version 990 is already newer than 984 2026-02-17T06:00:40.336 INFO:tasks.rados.rados.0.trial115.stdout:2449: oid 20 version 990 is already newer than 989 2026-02-17T06:00:40.336 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 990 (ObjNum 867 snap 247 seq_num 867) dirty exists 2026-02-17T06:00:40.336 INFO:tasks.rados.rados.0.trial115.stdout:2449: left oid 20 (ObjNum 867 snap 247 seq_num 867) 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2447: done (2 left) 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2449: done (1 left) 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2450: done (0 left) 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2451: read oid 31 snap -1 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2451: expect (ObjNum 804 snap 231 seq_num 804) 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2452: read oid 45 snap 246 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2452: expect (ObjNum 817 snap 235 seq_num 817) 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2453: write oid 16 current snap is 248 2026-02-17T06:00:41.325 INFO:tasks.rados.rados.0.trial115.stdout:2453: seq_num 868 ranges {588476=580374,1643429=603638} 2026-02-17T06:00:41.330 INFO:tasks.rados.rados.0.trial115.stdout:2453: writing trial11545420-16 from 588476 to 1168850 tid 1 2026-02-17T06:00:41.334 INFO:tasks.rados.rados.0.trial115.stdout:2453: writing trial11545420-16 from 1643429 to 2247067 tid 2 2026-02-17T06:00:41.334 INFO:tasks.rados.rados.0.trial115.stdout:2454: rollback oid 30 current snap is 248 2026-02-17T06:00:41.334 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 247 2026-02-17T06:00:41.334 INFO:tasks.rados.rados.0.trial115.stdout:2451: expect (ObjNum 804 snap 231 seq_num 804) 2026-02-17T06:00:41.415 INFO:tasks.rados.rados.0.trial115.stdout:2452: expect (ObjNum 817 snap 235 seq_num 817) 2026-02-17T06:00:41.571 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: finishing write tid 1 to trial11545420-16 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: finishing write tid 2 to trial11545420-16 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: finishing write tid 3 to trial11545420-16 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: finishing write tid 4 to trial11545420-16 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: oid 16 updating version 0 to 1377 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: oid 16 version 1377 is already newer than 1375 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: oid 16 version 1377 is already newer than 1376 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1377 (ObjNum 868 snap 248 seq_num 868) dirty exists 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: left oid 16 (ObjNum 868 snap 248 seq_num 868) 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2454: finishing rollback tid 0 to trial11545420-30 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2454: finishing rollback tid 1 to trial11545420-30 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2454: finishing rollback tid 2 to trial11545420-30 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1162 (ObjNum 809 snap 232 seq_num 809) dirty exists 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2451: done (3 left) 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2452: done (2 left) 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2453: done (1 left) 2026-02-17T06:00:41.603 INFO:tasks.rados.rados.0.trial115.stdout:2454: done (0 left) 2026-02-17T06:00:41.604 INFO:tasks.rados.rados.0.trial115.stdout:2455: read oid 28 snap -1 2026-02-17T06:00:41.604 INFO:tasks.rados.rados.0.trial115.stdout:2455: expect (ObjNum 845 snap 241 seq_num 845) 2026-02-17T06:00:41.604 INFO:tasks.rados.rados.0.trial115.stdout:2456: read oid 43 snap -1 2026-02-17T06:00:41.604 INFO:tasks.rados.rados.0.trial115.stdout:2456: expect deleted 2026-02-17T06:00:41.604 INFO:tasks.rados.rados.0.trial115.stdout:2457: copy_from oid 48 from oid 5 current snap is 248 2026-02-17T06:00:41.604 INFO:tasks.rados.rados.0.trial115.stdout:2458: snap_create 2026-02-17T06:00:41.605 INFO:tasks.rados.rados.0.trial115.stdout:2455: expect (ObjNum 845 snap 241 seq_num 845) 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.638+0000 7f4a7c03b640 10 monclient: get_monmap_and_config 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.638+0000 7f4a7c03b640 10 monclient: build_initial_monmap 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.638+0000 7f4a7c03b640 10 monclient: monmap: 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:41.639856+0000 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:41.639856+0000 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:41.639 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:41.640 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:41.640 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 1 Processor -- start 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 1 -- start start 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient: init 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a7406c890 0x7f4a740a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient(hunting): picked mon.noname-b con 0x7f4a7406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 0x7f4a7406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient(hunting): picked mon.noname-a con 0x7f4a7406cc70 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f4a74134df0 0x7f4a74135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient(hunting): picked mon.noname-c con 0x7f4a74134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f4a74074150 con 0x7f4a7406cc70 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7409ef70 con 0x7f4a74134df0 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7409f250 con 0x7f4a7406c890 2026-02-17T06:00:41.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7c03b640 10 monclient(hunting): authenticate will time out at 1487.193438s 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a7406c890 0x7f4a740a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 0x7f4a7406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a7406c890 0x7f4a740a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43170/0 (socket says 10.20.193.78:43170) 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 0x7f4a7406d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40908/0 (socket says 10.20.193.78:40908) 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 -- 10.20.193.78:0/2340157191 learned_addr learned my addr 10.20.193.78:0/2340157191 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient(hunting): get_auth_request con 0x7f4a7406c890 auth_method 0 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 10 monclient(hunting): get_auth_request con 0x7f4a7406cc70 auth_method 0 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 1 -- 10.20.193.78:0/2340157191 >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 msgr2=0x7f4a7406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 1 -- 10.20.193.78:0/2340157191 >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 msgr2=0x7f4a7406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 1 --2- 10.20.193.78:0/2340157191 >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 0x7f4a7406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a795af640 1 --2- 10.20.193.78:0/2340157191 >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 0x7f4a7406d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a7a5b1640 1 --2- 10.20.193.78:0/2340157191 >> v2:10.20.193.115:3300/0 conn(0x7f4a74134df0 0x7f4a74135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient(hunting): handle_auth_done global_id 144473 payload 274 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 -- 10.20.193.78:0/2340157191 >> v2:10.20.193.115:3300/0 conn(0x7f4a74134df0 msgr2=0x7f4a74135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 --2- 10.20.193.78:0/2340157191 >> v2:10.20.193.115:3300/0 conn(0x7f4a74134df0 0x7f4a74135240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 -- 10.20.193.78:0/2340157191 >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 msgr2=0x7f4a7406d050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 --2- 10.20.193.78:0/2340157191 >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 0x7f4a7406d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient: _finish_hunting 0 2026-02-17T06:00:41.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 monclient: found mon.noname-b 2026-02-17T06:00:41.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:41.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:41.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.639+0000 7f4a79db0640 1 -- 10.20.193.78:0/2340157191 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7409df40 con 0x7f4a7406c890 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a79db0640 1 -- 10.20.193.78:0/2340157191 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a7406c890 msgr2=0x7f4a740a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a79db0640 1 --2- 10.20.193.78:0/2340157191 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a7406c890 0x7f4a740a01f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a7406e270 tx=0x7f4a6402ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b832a83f2ac239e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 1 -- 10.20.193.78:0/2340157191 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6403c070 con 0x7f4a7406c890 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: dump: 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: _finish_auth 0 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: _check_auth_tickets 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 5 monclient: authenticate success, global_id 144473 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 1 -- 10.20.193.78:0/2340157191 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6402fde0 con 0x7f4a7406c890 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 1 -- 10.20.193.78:0/2340157191 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64037660 con 0x7f4a7406c890 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:41.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 4 set_mon_vals no callback set 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a78dae640 10 monclient: dump: 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 10 monclient: shutdown 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2340157191 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a7406c890 msgr2=0x7f4a740a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2340157191 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a7406c890 0x7f4a740a01f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a7406e270 tx=0x7f4a6402ef60 comp rx=0 tx=0).stop 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2340157191 shutdown_connections 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2340157191 >> v2:10.20.193.115:3300/0 conn(0x7f4a74134df0 0x7f4a74135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2340157191 >> v2:10.20.193.78:3300/0 conn(0x7f4a7406cc70 0x7f4a7406d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2340157191 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a7406c890 0x7f4a740a01f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2340157191 >> 10.20.193.78:0/2340157191 conn(0x7f4a740a6620 msgr2=0x7f4a740a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2340157191 shutdown_connections 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.642+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2340157191 wait complete. 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient: build_initial_monmap 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient: monmap: 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:41.645 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 1 Processor -- start 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 1 -- start start 2026-02-17T06:00:41.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient: init 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 0x7f4a74139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient(hunting): picked mon.noname-a con 0x7f4a7406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a7406cc70 0x7f4a74139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient(hunting): picked mon.noname-b con 0x7f4a7406cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a74134df0 0x7f4a7413a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient(hunting): picked mon.noname-c con 0x7f4a74134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7409d360 con 0x7f4a7406c890 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7409d1e0 con 0x7f4a7406cc70 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4a74135780 con 0x7f4a74134df0 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7c03b640 10 monclient(hunting): authenticate will time out at 1487.197558s 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 0x7f4a74139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 0x7f4a74139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40912/0 (socket says 10.20.193.78:40912) 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a74134df0 0x7f4a7413a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 1 -- 10.20.193.78:0/2154659804 learned_addr learned my addr 10.20.193.78:0/2154659804 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 10 monclient(hunting): get_auth_request con 0x7f4a7406c890 auth_method 0 2026-02-17T06:00:41.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient(hunting): get_auth_request con 0x7f4a74134df0 auth_method 0 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 1 -- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 msgr2=0x7f4a74139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 1 -- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 msgr2=0x7f4a74139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 0x7f4a74139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a79db0640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 0x7f4a74139940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a795af640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a7406cc70 0x7f4a74139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a795af640 10 monclient(hunting): get_auth_request con 0x7f4a7406cc70 auth_method 0 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a795af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a795af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a795af640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient(hunting): handle_auth_done global_id 144476 payload 824 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 1 -- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a7406cc70 msgr2=0x7f4a74139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a7406cc70 0x7f4a74139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 1 -- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 msgr2=0x7f4a74139940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 0x7f4a74139940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient: _finish_hunting 0 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 1 monclient: found mon.noname-c 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.643+0000 7f4a7a5b1640 1 -- 10.20.193.78:0/2154659804 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7413d310 con 0x7f4a74134df0 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.645+0000 7f4a7a5b1640 1 -- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a74134df0 msgr2=0x7f4a7413a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:41.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.645+0000 7f4a7a5b1640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a74134df0 0x7f4a7413a3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000cd20 tx=0x7f4a7000a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 1 -- 10.20.193.78:0/2154659804 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70003c90 con 0x7f4a74134df0 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: dump: 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: _finish_auth 0 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: _check_auth_tickets 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 1 -- 10.20.193.78:0/2154659804 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a70003e30 con 0x7f4a74134df0 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 5 monclient: authenticate success, global_id 144476 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 10 monclient: _renew_subs 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a74072cf0 con 0x7f4a74134df0 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 1 -- 10.20.193.78:0/2154659804 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000e800 con 0x7f4a74134df0 2026-02-17T06:00:41.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:41.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:41.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 10 monclient: dump: 2026-02-17T06:00:41.651 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:41.651 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:41.651 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:41.651 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:41.651 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:41.651 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a63fff640 4 set_mon_vals no callback set 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 10 monclient: _renew_subs 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7413d030 con 0x7f4a74134df0 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:41.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:41.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a74074980 con 0x7f4a74134df0 2026-02-17T06:00:41.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 1 -- 10.20.193.78:0/2154659804 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4a7000e9a0 con 0x7f4a74134df0 2026-02-17T06:00:41.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a4403deb0 0x7f4a44040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:41.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.646+0000 7f4a627fc640 1 -- 10.20.193.78:0/2154659804 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(525..525 src has 335..525) ==== 4783+0+0 (secure 0 0 0) 0x7f4a700532a0 con 0x7f4a74134df0 2026-02-17T06:00:41.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.647+0000 7f4a79db0640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a4403deb0 0x7f4a44040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:41.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.647+0000 7f4a79db0640 10 monclient: get_auth_request con 0x7f4a4403deb0 auth_method 0 2026-02-17T06:00:41.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.649+0000 7f4a627fc640 1 -- 10.20.193.78:0/2154659804 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4a70014db0 con 0x7f4a74134df0 2026-02-17T06:00:41.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.649+0000 7f4a627fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:41.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.649+0000 7f4a627fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:41.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.649+0000 7f4a79db0640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a4403deb0 0x7f4a44040370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4a6403a040 tx=0x7f4a64033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:41.685 INFO:tasks.rados.rados.0.trial115.stdout:2457: finishing copy_from racing read to trial11545420-48 2026-02-17T06:00:41.685 INFO:tasks.rados.rados.0.trial115.stdout:2457: finishing copy_from to trial11545420-48 2026-02-17T06:00:41.685 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1055 (ObjNum 777 snap 219 seq_num 777) dirty exists 2026-02-17T06:00:41.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.762+0000 7f4a7c03b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:41.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.762+0000 7f4a7c03b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:41.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.762+0000 7f4a7c03b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:41.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.762+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4a74135bb0 con 0x7f4a74134df0 2026-02-17T06:00:41.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.762+0000 7f4a627fc640 1 -- 10.20.193.78:0/2154659804 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f4a70053080 con 0x7f4a74134df0 2026-02-17T06:00:41.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.762+0000 7f4a627fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:41.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.762+0000 7f4a627fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:41.765 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":166,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":26,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:41.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a4403deb0 msgr2=0x7f4a44040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:41.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a4403deb0 0x7f4a44040370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4a6403a040 tx=0x7f4a64033000 comp rx=0 tx=0).stop 2026-02-17T06:00:41.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 10 monclient: shutdown 2026-02-17T06:00:41.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:41.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a74134df0 msgr2=0x7f4a7413a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:41.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a74134df0 0x7f4a7413a3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000cd20 tx=0x7f4a7000a660 comp rx=0 tx=0).stop 2026-02-17T06:00:41.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 shutdown_connections 2026-02-17T06:00:41.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4a74134df0 0x7f4a7413a3c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4a4403deb0 0x7f4a44040370 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4a7406cc70 0x7f4a74139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 --2- 10.20.193.78:0/2154659804 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4a7406c890 0x7f4a74139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:41.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 >> 10.20.193.78:0/2154659804 conn(0x7f4a740a6620 msgr2=0x7f4a74070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:41.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 shutdown_connections 2026-02-17T06:00:41.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:41.763+0000 7f4a7c03b640 1 -- 10.20.193.78:0/2154659804 wait complete. 2026-02-17T06:00:44.773 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:44.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.841+0000 7f14fe840640 10 monclient: get_monmap_and_config 2026-02-17T06:00:44.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.841+0000 7f14fe840640 10 monclient: build_initial_monmap 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.841+0000 7f14fe840640 10 monclient: monmap: 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:44.842676+0000 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:44.842676+0000 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:44.843 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:44.844 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:44.844 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:44.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.841+0000 7f14fe840640 1 Processor -- start 2026-02-17T06:00:44.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.841+0000 7f14fe840640 1 -- start start 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.841+0000 7f14fe840640 10 monclient: init 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient(hunting): picked mon.noname-c con 0x7f14f80a6470 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f14f80a6850 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient(hunting): picked mon.noname-b con 0x7f14f80a6c60 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f80a0730 con 0x7f14f80a6850 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:44.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f14f809fa00 con 0x7f14f80a6470 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80a19c0 con 0x7f14f80a6c60 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fe840640 10 monclient(hunting): authenticate will time out at 1490.396249s 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f77fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f14f80a6850 0x7f14f8137f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:59698/0 (socket says 10.20.193.78:59698) 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f77fe640 1 -- 10.20.193.78:0/2103267140 learned_addr learned my addr 10.20.193.78:0/2103267140 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request con 0x7f14f80a6c60 auth_method 0 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:44.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 10 monclient(hunting): get_auth_request con 0x7f14f80a6470 auth_method 0 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:44.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.842+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:44.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:44.844+0000 7f14f77fe640 1 -- 10.20.193.78:0/2103267140 >> v1:10.20.193.78:6789/0 conn(0x7f14f80a6850 legacy=0x7f14f8137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:45.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:45.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request con 0x7f14f80a6c60 auth_method 0 2026-02-17T06:00:45.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:45.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 10 monclient(hunting): get_auth_request con 0x7f14f80a6470 auth_method 0 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:45.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:45.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:45.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:45.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.043+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:00:45.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.223+0000 7f14f77fe640 1 -- 10.20.193.78:0/2103267140 >> v1:10.20.193.78:6789/0 conn(0x7f14f80a6850 legacy=0x7f14f8137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-17T06:00:45.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.223+0000 7f14f77fe640 1 -- 10.20.193.78:0/2103267140 >> v1:10.20.193.78:6789/0 conn(0x7f14f80a6850 legacy=0x7f14f8137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:00:45.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.223+0000 7f14f77fe640 1 --1- 10.20.193.78:0/2103267140 >> v1:10.20.193.78:6789/0 conn(0x7f14f80a6850 0x7f14f8137f50 :-1 s=OPENED pgs=26 cs=1 l=1).handle_message read tag failed 2026-02-17T06:00:45.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.223+0000 7f14f77fe640 1 --1- 10.20.193.78:0/2103267140 >> v1:10.20.193.78:6789/0 conn(0x7f14f80a6850 0x7f14f8137f50 :-1 s=OPENED pgs=26 cs=1 l=1).fault on lossy channel, failing 2026-02-17T06:00:45.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.223+0000 7f14f6ffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.78:6789/0 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request con 0x7f14f80a6c60 auth_method 0 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14f7fff640 10 monclient(hunting): get_auth_request con 0x7f14f80a6470 auth_method 0 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14f7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.443+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.444+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.444+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:45.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:45.444+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:00:46.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:46.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request con 0x7f14f80a6c60 auth_method 0 2026-02-17T06:00:46.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.244+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 10 monclient(hunting): get_auth_request con 0x7f14f80a6470 auth_method 0 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:46.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:46.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:46.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:46.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:46.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:46.245+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:00:47.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 10 monclient(hunting): tick 2026-02-17T06:00:47.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:00:47.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 monclient(hunting): continuing hunt 2026-02-17T06:00:47.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:00:47.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 msgr2=0x7f14f8138660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:47.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14f80a6c60 0x7f14f8138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 msgr2=0x7f14f80a1480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.115:3300/0 conn(0x7f14f80a6470 0x7f14f80a1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 -- 10.20.193.78:0/2103267140 >> v1:10.20.193.78:6789/0 conn(0x7f14f80a6850 legacy=0x7f14f8137f50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 10 monclient: _add_conns ranks=[1,3] 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f14d40036d0 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.78:3300/0 conn(0x7f14d4007420 0x7f14d40098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f14d4007420 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 -- 10.20.193.78:0/2103267140 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f80a0450 con 0x7f14d40036d0 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:47.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 1 -- 10.20.193.78:0/2103267140 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f14f809fa00 con 0x7f14d4007420 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f5ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.78:3300/0 conn(0x7f14d4007420 0x7f14d40098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 10 monclient(hunting): get_auth_request con 0x7f14d4007420 auth_method 0 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.78:3300/0 conn(0x7f14d4007420 msgr2=0x7f14d40098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.78:3300/0 conn(0x7f14d4007420 msgr2=0x7f14d40098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.78:3300/0 conn(0x7f14d4007420 0x7f14d40098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:47.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.842+0000 7f14f77fe640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.78:3300/0 conn(0x7f14d4007420 0x7f14d40098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:47.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.845+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> v1:10.20.193.115:6789/0 conn(0x7f14d40036d0 legacy=0x7f14d4005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3615312778 0 0) 0x7f14f80a0450 con 0x7f14d40036d0 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): _negotiate 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): my global_id is 134455 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14dc003620 con 0x7f14d40036d0 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 1 --1- 10.20.193.78:0/2103267140 >> v1:10.20.193.115:6789/0 conn(0x7f14d40036d0 0x7f14d4005b50 :-1 s=OPENED pgs=181 cs=1 l=1).queue_received will delay after 1.77131e+09 on 0x7f14e400f040 mon_map magic: 0 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2504285337 0 0) 0x7f14dc003620 con 0x7f14d40036d0 2026-02-17T06:00:47.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 >> v2:10.20.193.78:3300/0 conn(0x7f14d4007420 msgr2=0x7f14d40098e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:47.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 --2- 10.20.193.78:0/2103267140 >> v2:10.20.193.78:3300/0 conn(0x7f14d4007420 0x7f14d40098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:00:47.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:00:47.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:47.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T06:00:47.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d400b070 con 0x7f14d40036d0 2026-02-17T06:00:47.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fe840640 5 monclient: authenticate success, global_id 134455 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fe840640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f14e400f040 con 0x7f14d40036d0 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: dump: 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 >> v1:10.20.193.115:6789/0 conn(0x7f14d40036d0 legacy=0x7f14d4005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fcdb6640 1 -- 10.20.193.78:0/2103267140 reap_dead start 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc003b00 0x7f14dc003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): picked mon.c con 0x7f14dc003b00 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc004900 0x7f14dc004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): picked mon.a con 0x7f14dc004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 0x7f14dc00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): picked mon.b con 0x7f14dc005410 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14f809fa00 con 0x7f14dc004900 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc00c350 con 0x7f14dc005410 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc00c550 con 0x7f14dc003b00 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fe840640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 <== mon.1 v1:10.20.193.115:6789/0 5 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f14e400ccf0 con 0x7f14d40036d0 2026-02-17T06:00:47.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f6ffd640 10 monclient(hunting): discarding stray monitor message mon_map magic: 0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc003b00 0x7f14dc003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 10 monclient(hunting): get_auth_request con 0x7f14dc003b00 auth_method 0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f77fe640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc004900 0x7f14dc004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f77fe640 10 monclient(hunting): get_auth_request con 0x7f14dc004900 auth_method 0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 0x7f14dc00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request con 0x7f14dc005410 auth_method 0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fcdb6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14fcdb6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 10 monclient(hunting): handle_auth_done global_id 134455 payload 293 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 msgr2=0x7f14dc00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 0x7f14dc00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc004900 msgr2=0x7f14dc004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.855+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc004900 0x7f14dc004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.856+0000 7f14f7fff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.856+0000 7f14f7fff640 1 monclient: found mon.c 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.856+0000 7f14f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.856+0000 7f14f7fff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.856+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc003620 con 0x7f14dc003b00 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.856+0000 7f14f7fff640 10 monclient: _finish_auth 0 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.856+0000 7f14f7fff640 10 monclient: _check_auth_tickets 2026-02-17T06:00:47.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.856+0000 7f14f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f7fff640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc003b00 msgr2=0x7f14dc003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f7fff640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc003b00 0x7f14dc003ee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f14e400cf10 tx=0x7f14e4030cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603f8711547c68e2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4012930 con 0x7f14dc003b00 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 10 monclient: dump: 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e4032700 con 0x7f14dc003b00 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14fe840640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 1 -- 10.20.193.78:0/2103267140 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4032af0 con 0x7f14dc003b00 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14fe840640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14fe840640 4 set_mon_vals no callback set 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:47.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.858+0000 7f14f6ffd640 10 monclient: dump: 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 10 monclient: shutdown 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 -- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc003b00 msgr2=0x7f14dc003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc003b00 0x7f14dc003ee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f14e400cf10 tx=0x7f14e4030cd0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 -- 10.20.193.78:0/2103267140 shutdown_connections 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 0x7f14dc00bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc004900 0x7f14dc004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 --2- 10.20.193.78:0/2103267140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc003b00 0x7f14dc003ee0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 -- 10.20.193.78:0/2103267140 >> 10.20.193.78:0/2103267140 conn(0x7f14f80aae40 msgr2=0x7f14f80ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 -- 10.20.193.78:0/2103267140 shutdown_connections 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 -- 10.20.193.78:0/2103267140 wait complete. 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 10 monclient: build_initial_monmap 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 10 monclient: monmap: 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:47.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 Processor -- start 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 1 -- start start 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.859+0000 7f14fe840640 10 monclient: init 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 0x7f14f8130b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient(hunting): picked mon.noname-a con 0x7f14dc003b00 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc004900 0x7f14f8131070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient(hunting): picked mon.noname-c con 0x7f14dc004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 0x7f14f81315b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient(hunting): picked mon.noname-b con 0x7f14dc005410 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80a1480 con 0x7f14dc003b00 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:47.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80a1600 con 0x7f14dc005410 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80a1780 con 0x7f14dc004900 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fe840640 10 monclient(hunting): authenticate will time out at 1493.414096s 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 0x7f14f8130b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc004900 0x7f14f8131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 0x7f14f8130b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49692/0 (socket says 10.20.193.78:49692) 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc004900 0x7f14f8131070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54458/0 (socket says 10.20.193.78:54458) 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 1 -- 10.20.193.78:0/3158337303 learned_addr learned my addr 10.20.193.78:0/3158337303 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 10 monclient(hunting): get_auth_request con 0x7f14dc003b00 auth_method 0 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient(hunting): get_auth_request con 0x7f14dc004900 auth_method 0 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 1 -- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 msgr2=0x7f14f8130b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 1 -- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 msgr2=0x7f14f8130b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 0x7f14f8130b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f7fff640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 0x7f14f8130b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fcdb6640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 0x7f14f81315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request con 0x7f14dc005410 auth_method 0 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fcdb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fcdb6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14fcdb6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient(hunting): handle_auth_done global_id 144479 payload 824 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 1 -- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 msgr2=0x7f14f81315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 0x7f14f81315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 1 -- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 msgr2=0x7f14f8130b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 0x7f14f8130b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 1 monclient: found mon.noname-c 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:47.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.860+0000 7f14f77fe640 1 -- 10.20.193.78:0/3158337303 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f81326e0 con 0x7f14dc004900 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f77fe640 1 -- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc004900 msgr2=0x7f14f8131070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f77fe640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc004900 0x7f14f8131070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f14e8007d80 tx=0x7f14e8007780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 1 -- 10.20.193.78:0/3158337303 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8019020 con 0x7f14dc004900 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: dump: 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 1 -- 10.20.193.78:0/3158337303 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e80078c0 con 0x7f14dc004900 2026-02-17T06:00:47.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 1 -- 10.20.193.78:0/3158337303 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011e90 con 0x7f14dc004900 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f67fc640 4 set_mon_vals no callback set 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14f4ff9640 10 monclient: dump: 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14fe840640 5 monclient: authenticate success, global_id 144479 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14fe840640 10 monclient: _renew_subs 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14fe840640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f81342c0 con 0x7f14dc004900 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14fe840640 10 monclient: _renew_subs 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14fe840640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.862+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f8133fe0 con 0x7f14dc004900 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14f4ff9640 1 -- 10.20.193.78:0/3158337303 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f14e8015470 con 0x7f14dc004900 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14fe840640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14fe840640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14fe840640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f80a4350 con 0x7f14dc004900 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14f4ff9640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f14c803e050 0x7f14c8040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14f4ff9640 1 -- 10.20.193.78:0/3158337303 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(525..525 src has 335..525) ==== 4783+0+0 (secure 0 0 0) 0x7f14e8052ec0 con 0x7f14dc004900 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14f7fff640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f14c803e050 0x7f14c8040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:47.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.863+0000 7f14f7fff640 10 monclient: get_auth_request con 0x7f14c803e050 auth_method 0 2026-02-17T06:00:47.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.865+0000 7f14f4ff9640 1 -- 10.20.193.78:0/3158337303 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f14f80a4350 con 0x7f14dc004900 2026-02-17T06:00:47.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.865+0000 7f14f4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:47.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.865+0000 7f14f4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:47.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.866+0000 7f14f7fff640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f14c803e050 0x7f14c8040510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f14e40036c0 tx=0x7f14e4035000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.979+0000 7f14fe840640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.979+0000 7f14fe840640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.979+0000 7f14fe840640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.979+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f14f8074980 con 0x7f14dc004900 2026-02-17T06:00:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.979+0000 7f14f4ff9640 1 -- 10.20.193.78:0/3158337303 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f14e8023030 con 0x7f14dc004900 2026-02-17T06:00:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.979+0000 7f14f4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:47.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.979+0000 7f14f4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":174,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f14c803e050 msgr2=0x7f14c8040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f14c803e050 0x7f14c8040510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f14e40036c0 tx=0x7f14e4035000 comp rx=0 tx=0).stop 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 10 monclient: shutdown 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc004900 msgr2=0x7f14f8131070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc004900 0x7f14f8131070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f14e8007d80 tx=0x7f14e8007780 comp rx=0 tx=0).stop 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 shutdown_connections 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f14c803e050 0x7f14c8040510 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f14dc005410 0x7f14f81315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f14dc004900 0x7f14f8131070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 --2- 10.20.193.78:0/3158337303 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f14dc003b00 0x7f14f8130b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 >> 10.20.193.78:0/3158337303 conn(0x7f14f80aae40 msgr2=0x7f14f80a9740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:47.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 shutdown_connections 2026-02-17T06:00:47.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:47.980+0000 7f14fe840640 1 -- 10.20.193.78:0/3158337303 wait complete. 2026-02-17T06:00:47.990 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:00:47.990 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:00:47.991 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.058+0000 7f4872ebf640 10 monclient: get_monmap_and_config 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.058+0000 7f4872ebf640 10 monclient: build_initial_monmap 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.058+0000 7f4872ebf640 10 monclient: monmap: 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:48.060200+0000 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:48.060200+0000 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 1 Processor -- start 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 1 -- start start 2026-02-17T06:00:48.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient: init 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient(hunting): picked mon.noname-b con 0x7f486c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f486c09c4a0 0x7f486c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient(hunting): picked mon.noname-a con 0x7f486c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f486c134df0 0x7f486c135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient(hunting): picked mon.noname-c con 0x7f486c134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f486c0a0880 con 0x7f486c09c4a0 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f486c06d6f0 con 0x7f486c134df0 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f486c06daf0 con 0x7f486c06c890 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4872ebf640 10 monclient(hunting): authenticate will time out at 1493.613783s 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54464/0 (socket says 10.20.193.78:54464) 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 1 -- 10.20.193.78:0/923460723 learned_addr learned my addr 10.20.193.78:0/923460723 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 10 monclient(hunting): get_auth_request con 0x7f486c06c890 auth_method 0 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4863fff640 1 --2- 10.20.193.78:0/923460723 >> v2:10.20.193.78:3300/0 conn(0x7f486c09c4a0 0x7f486c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4863fff640 10 monclient(hunting): get_auth_request con 0x7f486c09c4a0 auth_method 0 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4863fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4863fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4863fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4870c34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.059+0000 7f4871435640 1 --2- 10.20.193.78:0/923460723 >> v2:10.20.193.115:3300/0 conn(0x7f486c134df0 0x7f486c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 10 monclient(hunting): handle_auth_done global_id 144482 payload 274 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 1 -- 10.20.193.78:0/923460723 >> v2:10.20.193.115:3300/0 conn(0x7f486c134df0 msgr2=0x7f486c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 1 --2- 10.20.193.78:0/923460723 >> v2:10.20.193.115:3300/0 conn(0x7f486c134df0 0x7f486c135240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 1 -- 10.20.193.78:0/923460723 >> v2:10.20.193.78:3300/0 conn(0x7f486c09c4a0 msgr2=0x7f486c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 1 --2- 10.20.193.78:0/923460723 >> v2:10.20.193.78:3300/0 conn(0x7f486c09c4a0 0x7f486c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 10 monclient: _finish_hunting 0 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 1 monclient: found mon.noname-b 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.060+0000 7f4870c34640 1 -- 10.20.193.78:0/923460723 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c09cc80 con 0x7f486c06c890 2026-02-17T06:00:48.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4870c34640 1 -- 10.20.193.78:0/923460723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 msgr2=0x7f486c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4870c34640 1 --2- 10.20.193.78:0/923460723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c06cc70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f486c09c880 tx=0x7f485c02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b7b26c2f68ddf6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 1 -- 10.20.193.78:0/923460723 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c03c070 con 0x7f486c06c890 2026-02-17T06:00:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:48.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: dump: 2026-02-17T06:00:48.071 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.071 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.071 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 1 -- 10.20.193.78:0/923460723 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c02fca0 con 0x7f486c06c890 2026-02-17T06:00:48.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4872ebf640 5 monclient: authenticate success, global_id 144482 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4872ebf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 1 -- 10.20.193.78:0/923460723 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c037660 con 0x7f486c06c890 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4872ebf640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4872ebf640 4 set_mon_vals no callback set 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f48637fe640 10 monclient: dump: 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.073 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4872ebf640 10 monclient: shutdown 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4872ebf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4872ebf640 1 -- 10.20.193.78:0/923460723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 msgr2=0x7f486c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.062+0000 7f4872ebf640 1 --2- 10.20.193.78:0/923460723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c06cc70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f486c09c880 tx=0x7f485c02ee60 comp rx=0 tx=0).stop 2026-02-17T06:00:48.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 -- 10.20.193.78:0/923460723 shutdown_connections 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 --2- 10.20.193.78:0/923460723 >> v2:10.20.193.115:3300/0 conn(0x7f486c134df0 0x7f486c135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 --2- 10.20.193.78:0/923460723 >> v2:10.20.193.78:3300/0 conn(0x7f486c09c4a0 0x7f486c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 --2- 10.20.193.78:0/923460723 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c06cc70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 -- 10.20.193.78:0/923460723 >> 10.20.193.78:0/923460723 conn(0x7f486c0a6620 msgr2=0x7f486c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 -- 10.20.193.78:0/923460723 shutdown_connections 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 -- 10.20.193.78:0/923460723 wait complete. 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 10 monclient: build_initial_monmap 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 10 monclient: monmap: 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:48.075 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 Processor -- start 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 1 -- start start 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.063+0000 7f4872ebf640 10 monclient: init 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:48.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient(hunting): picked mon.noname-c con 0x7f486c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c09c4a0 0x7f486c09e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient(hunting): picked mon.noname-a con 0x7f486c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f486c134df0 0x7f486c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient(hunting): picked mon.noname-b con 0x7f486c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0a01a0 con 0x7f486c09c4a0 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0a0020 con 0x7f486c134df0 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f486c09be20 con 0x7f486c06c890 2026-02-17T06:00:48.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4872ebf640 10 monclient(hunting): authenticate will time out at 1493.618127s 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c09db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54472/0 (socket says 10.20.193.78:54472) 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 -- 10.20.193.78:0/1295451048 learned_addr learned my addr 10.20.193.78:0/1295451048 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient(hunting): get_auth_request con 0x7f486c06c890 auth_method 0 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c09c4a0 0x7f486c09e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 10 monclient(hunting): get_auth_request con 0x7f486c09c4a0 auth_method 0 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient(hunting): handle_auth_done global_id 144485 payload 824 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f486c134df0 msgr2=0x7f486c09e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4871435640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f486c134df0 0x7f486c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f486c134df0 0x7f486c09e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c09c4a0 msgr2=0x7f486c09e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c09c4a0 0x7f486c09e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient: _finish_hunting 0 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 monclient: found mon.noname-c 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4870c34640 1 -- 10.20.193.78:0/1295451048 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c127570 con 0x7f486c06c890 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4863fff640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c09c4a0 0x7f486c09e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4871435640 10 monclient: get_auth_request con 0x7f486c134df0 auth_method 0 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.064+0000 7f4871435640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f486c134df0 0x7f486c09e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4870c34640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 msgr2=0x7f486c09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4870c34640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c09db50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f485c002410 tx=0x7f485c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c047070 con 0x7f486c06c890 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: dump: 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.081 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: _finish_auth 0 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c02fe50 con 0x7f486c06c890 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4872ebf640 5 monclient: authenticate success, global_id 144485 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4872ebf640 10 monclient: _renew_subs 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4872ebf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c1260c0 con 0x7f486c06c890 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:48.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c03c040 con 0x7f486c06c890 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4862ffd640 4 set_mon_vals no callback set 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f48617fa640 10 monclient: dump: 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.083 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4872ebf640 10 monclient: _renew_subs 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4872ebf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.066+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c0a1750 con 0x7f486c06c890 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4872ebf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4872ebf640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cf150 0x7f486c1cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4872ebf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0a11d0 con 0x7f486c1cf150 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f485c03f9f0 con 0x7f486c06c890 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cf150 0x7f486c1cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 10 monclient: get_auth_request con 0x7f486c1cf150 auth_method 0 2026-02-17T06:00:48.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 10 monclient(hunting): handle_auth_done global_id 184104 payload 824 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4863fff640 1 -- 10.20.193.78:0/1295451048 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f486c074980 con 0x7f486c1cf150 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f48617fa640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f483c03deb0 0x7f483c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(525..525 src has 335..525) ==== 4783+0+0 (secure 0 0 0) 0x7f485c0773d0 con 0x7f486c06c890 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4871435640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f483c03deb0 0x7f483c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.067+0000 7f4871435640 10 monclient: get_auth_request con 0x7f483c03deb0 auth_method 0 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f4863fff640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cf150 msgr2=0x7f486c1cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f4863fff640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cf150 0x7f486c1cf530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f486c09f350 tx=0x7f485400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854019070 con 0x7f486c1cf150 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 10 monclient: dump: 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.086 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4854004450 con 0x7f486c1cf150 2026-02-17T06:00:48.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:48.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cf150 msgr2=0x7f486c1cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cf150 0x7f486c1cf530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f486c09f350 tx=0x7f485400f610 comp rx=0 tx=0).stop 2026-02-17T06:00:48.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cf150 msgr2=0x7f486c1cf530 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:48.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.069+0000 7f48617fa640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cf150 0x7f486c1cf530 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f486c09f350 tx=0x7f485400f610 comp rx=0 tx=0).stop 2026-02-17T06:00:48.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.070+0000 7f4871435640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f483c03deb0 0x7f483c040370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f486c0732c0 tx=0x7f4868007430 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4872ebf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:00:48.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4872ebf640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cfa70 0x7f486c1da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4872ebf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f486c135240 con 0x7f486c1cfa70 2026-02-17T06:00:48.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cfa70 0x7f486c1da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 10 monclient: get_auth_request con 0x7f486c1cfa70 auth_method 0 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 10 monclient(hunting): handle_auth_done global_id 184107 payload 824 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.073+0000 7f4863fff640 1 -- 10.20.193.78:0/1295451048 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f486c0744d0 con 0x7f486c1cfa70 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f4863fff640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cfa70 msgr2=0x7f486c1da4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f4863fff640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cfa70 0x7f486c1da4a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4854009fd0 tx=0x7f48540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854019040 con 0x7f486c1cfa70 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:48.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 10 monclient: dump: 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f4854003e20 con 0x7f486c1cfa70 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cfa70 msgr2=0x7f486c1da4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cfa70 0x7f486c1da4a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4854009fd0 tx=0x7f48540047c0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cfa70 msgr2=0x7f486c1da4a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.075+0000 7f48617fa640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f486c1cfa70 0x7f486c1da4a0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4854009fd0 tx=0x7f48540047c0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 174, 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 12856, 2026-02-17T06:00:48.092 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.093 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.094 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:00:48.095 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.096 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.097 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.098 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.099 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.100 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:00:48.101 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:00:48.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f483c03deb0 msgr2=0x7f483c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f483c03deb0 0x7f483c040370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f486c0732c0 tx=0x7f4868007430 comp rx=0 tx=0).stop 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 10 monclient: shutdown 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 msgr2=0x7f486c09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c09db50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f485c002410 tx=0x7f485c004830 comp rx=0 tx=0).stop 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4871435640 1 -- 10.20.193.78:0/1295451048 reap_dead start 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 shutdown_connections 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 --2- 10.20.193.78:0/1295451048 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f486c06c890 0x7f486c09db50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 >> 10.20.193.78:0/1295451048 conn(0x7f486c0a6620 msgr2=0x7f486c071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 shutdown_connections 2026-02-17T06:00:48.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.076+0000 7f4872ebf640 1 -- 10.20.193.78:0/1295451048 wait complete. 2026-02-17T06:00:48.103 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:00:48.103 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:00:48.104 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:00:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient: get_monmap_and_config 2026-02-17T06:00:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient: build_initial_monmap 2026-02-17T06:00:48.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient: monmap: 2026-02-17T06:00:48.174 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:48.174 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:48.172499+0000 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:48.172499+0000 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 1 Processor -- start 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 1 -- start start 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient: init 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fe4b406c890 0x7fe4b40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient(hunting): picked mon.noname-c con 0x7fe4b406c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe4b406cc70 0x7fe4b406d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient(hunting): picked mon.noname-a con 0x7fe4b406cc70 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b4134df0 0x7fe4b4135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.171+0000 7fe4bc829640 10 monclient(hunting): picked mon.noname-b con 0x7fe4b4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bc829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bc829640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b4074150 con 0x7fe4b406cc70 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bc829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bc829640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b409ef70 con 0x7fe4b406c890 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bc829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bc829640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b409f250 con 0x7fe4b4134df0 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bc829640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bc829640 10 monclient(hunting): authenticate will time out at 1493.726025s 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe4b406cc70 0x7fe4b406d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b4134df0 0x7fe4b4135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe4b406cc70 0x7fe4b406d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49728/0 (socket says 10.20.193.78:49728) 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 1 -- 10.20.193.78:0/3248063193 learned_addr learned my addr 10.20.193.78:0/3248063193 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient(hunting): get_auth_request con 0x7fe4b4134df0 auth_method 0 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 10 monclient(hunting): get_auth_request con 0x7fe4b406cc70 auth_method 0 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4b9d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient(hunting): handle_auth_done global_id 144488 payload 274 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 1 -- 10.20.193.78:0/3248063193 >> v2:10.20.193.115:3300/0 conn(0x7fe4b406c890 msgr2=0x7fe4b40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 1 --2- 10.20.193.78:0/3248063193 >> v2:10.20.193.115:3300/0 conn(0x7fe4b406c890 0x7fe4b40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 1 -- 10.20.193.78:0/3248063193 >> v2:10.20.193.78:3300/0 conn(0x7fe4b406cc70 msgr2=0x7fe4b406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 1 --2- 10.20.193.78:0/3248063193 >> v2:10.20.193.78:3300/0 conn(0x7fe4b406cc70 0x7fe4b406d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient: _finish_hunting 0 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 1 monclient: found mon.noname-b 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.172+0000 7fe4bad9f640 1 -- 10.20.193.78:0/3248063193 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b409df40 con 0x7fe4b4134df0 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.174+0000 7fe4bad9f640 1 -- 10.20.193.78:0/3248063193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b4134df0 msgr2=0x7fe4b4135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.174+0000 7fe4bad9f640 1 --2- 10.20.193.78:0/3248063193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b4134df0 0x7fe4b4135220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4b406a480 tx=0x7fe4a802f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c407a41094e16c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 1 -- 10.20.193.78:0/3248063193 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a803c070 con 0x7fe4b4134df0 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: dump: 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: _finish_auth 0 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: _check_auth_tickets 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 1 -- 10.20.193.78:0/3248063193 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a8037640 con 0x7fe4b4134df0 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 1 -- 10.20.193.78:0/3248063193 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8037a10 con 0x7fe4b4134df0 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 5 monclient: authenticate success, global_id 144488 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 4 set_mon_vals no callback set 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4b959c640 10 monclient: dump: 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.178 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 10 monclient: shutdown 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 -- 10.20.193.78:0/3248063193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b4134df0 msgr2=0x7fe4b4135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 --2- 10.20.193.78:0/3248063193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b4134df0 0x7fe4b4135220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4b406a480 tx=0x7fe4a802f270 comp rx=0 tx=0).stop 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 -- 10.20.193.78:0/3248063193 shutdown_connections 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 --2- 10.20.193.78:0/3248063193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b4134df0 0x7fe4b4135220 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 --2- 10.20.193.78:0/3248063193 >> v2:10.20.193.78:3300/0 conn(0x7fe4b406cc70 0x7fe4b406d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 --2- 10.20.193.78:0/3248063193 >> v2:10.20.193.115:3300/0 conn(0x7fe4b406c890 0x7fe4b40a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 -- 10.20.193.78:0/3248063193 >> 10.20.193.78:0/3248063193 conn(0x7fe4b40a6620 msgr2=0x7fe4b40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 -- 10.20.193.78:0/3248063193 shutdown_connections 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 1 -- 10.20.193.78:0/3248063193 wait complete. 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 10 monclient: build_initial_monmap 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.175+0000 7fe4bc829640 10 monclient: monmap: 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 1 Processor -- start 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 1 -- start start 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient: init 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b406c890 0x7fe4b41cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient(hunting): picked mon.noname-c con 0x7fe4b406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4b406cc70 0x7fe4b4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient(hunting): picked mon.noname-a con 0x7fe4b406cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe4b4134df0 0x7fe4b4134470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient(hunting): picked mon.noname-b con 0x7fe4b4134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b409d360 con 0x7fe4b406cc70 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b409d1e0 con 0x7fe4b4134df0 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b4135760 con 0x7fe4b406c890 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4bc829640 10 monclient(hunting): authenticate will time out at 1493.730566s 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b406c890 0x7fe4b41cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4b406cc70 0x7fe4b4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b406c890 0x7fe4b41cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54490/0 (socket says 10.20.193.78:54490) 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 1 -- 10.20.193.78:0/3079861144 learned_addr learned my addr 10.20.193.78:0/3079861144 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 10 monclient(hunting): get_auth_request con 0x7fe4b406c890 auth_method 0 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient(hunting): get_auth_request con 0x7fe4b406cc70 auth_method 0 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient(hunting): handle_auth_done global_id 184113 payload 824 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b406c890 msgr2=0x7fe4b41cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b406c890 0x7fe4b41cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe4b4134df0 msgr2=0x7fe4b4134470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe4b4134df0 0x7fe4b4134470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient: _finish_hunting 0 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 1 monclient: found mon.noname-a 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4b9d9d640 1 -- 10.20.193.78:0/3079861144 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b40a3800 con 0x7fe4b406cc70 2026-02-17T06:00:48.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.176+0000 7fe4ba59e640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b406c890 0x7fe4b41cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.178+0000 7fe4b9d9d640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4b406cc70 msgr2=0x7fe4b4133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.178+0000 7fe4b9d9d640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4b406cc70 0x7fe4b4133f30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0006dc0 tx=0x7fe4a000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.178+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0017070 con 0x7fe4b406cc70 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.178+0000 7fe49effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 10 monclient: dump: 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 10 monclient: _finish_auth 0 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 5 monclient: authenticate success, global_id 184113 2026-02-17T06:00:48.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a000caa0 con 0x7fe4b406cc70 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a000ce50 con 0x7fe4b406cc70 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 10 monclient: _renew_subs 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4b8d9b640 4 set_mon_vals no callback set 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b409d810 con 0x7fe4b406cc70 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 10 monclient: dump: 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 10 monclient: _renew_subs 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b41208e0 con 0x7fe4b406cc70 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d430 0x7fe4b4139950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b40a2670 con 0x7fe4b411d430 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d430 0x7fe4b4139950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 10 monclient: get_auth_request con 0x7fe4b411d430 auth_method 0 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe4a0024060 con 0x7fe4b406cc70 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_done global_id 144494 payload 824 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe4ba59e640 1 -- 10.20.193.78:0/3079861144 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4b409ccb0 con 0x7fe4b411d430 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe48403e050 0x7fe484040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.179+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(525..525 src has 335..525) ==== 4783+0+0 (secure 0 0 0) 0x7fe4a00527a0 con 0x7fe4b406cc70 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.180+0000 7fe4bad9f640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe48403e050 0x7fe484040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.180+0000 7fe4bad9f640 10 monclient: get_auth_request con 0x7fe48403e050 auth_method 0 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.181+0000 7fe4ba59e640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d430 msgr2=0x7fe4b4139950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.181+0000 7fe4ba59e640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d430 0x7fe4b4139950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac002a80 tx=0x7fe4ac002f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac016070 con 0x7fe4b411d430 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 10 monclient: dump: 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fe4ac01a400 con 0x7fe4b411d430 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d430 msgr2=0x7fe4b4139950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d430 0x7fe4b4139950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac002a80 tx=0x7fe4ac002f50 comp rx=0 tx=0).stop 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d430 msgr2=0x7fe4b4139950 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe49effd640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d430 0x7fe4b4139950 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac002a80 tx=0x7fe4ac002f50 comp rx=0 tx=0).stop 2026-02-17T06:00:48.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.182+0000 7fe4bad9f640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe48403e050 0x7fe484040510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4074e00 tx=0x7fe4a8031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4bc829640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4bc829640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d810 0x7fe4b41da600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4bc829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b4074980 con 0x7fe4b411d810 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d810 0x7fe4b41da600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 10 monclient: get_auth_request con 0x7fe4b411d810 auth_method 0 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 10 monclient(hunting): handle_auth_done global_id 144497 payload 824 2026-02-17T06:00:48.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.185+0000 7fe4ba59e640 1 -- 10.20.193.78:0/3079861144 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe4b4135b90 con 0x7fe4b411d810 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.187+0000 7fe4ba59e640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d810 msgr2=0x7fe4b41da600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.187+0000 7fe4ba59e640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d810 0x7fe4b41da600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac017000 tx=0x7fe4ac0048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.187+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac016070 con 0x7fe4b411d810 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.187+0000 7fe49effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.187+0000 7fe49effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.187+0000 7fe49effd640 10 monclient: dump: 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.189 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fe4ac0128f0 con 0x7fe4b411d810 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe49effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe49effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d810 msgr2=0x7fe4b41da600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe49effd640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d810 0x7fe4b41da600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac017000 tx=0x7fe4ac0048d0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe49effd640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d810 msgr2=0x7fe4b41da600 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:48.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe49effd640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4b411d810 0x7fe4b41da600 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac017000 tx=0x7fe4ac0048d0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 174, 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 231722, 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:00:48.191 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.192 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.193 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:00:48.194 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.195 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.196 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.197 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.198 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:00:48.199 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.200 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:00:48.201 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe48403e050 msgr2=0x7fe484040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe48403e050 0x7fe484040510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4074e00 tx=0x7fe4a8031000 comp rx=0 tx=0).stop 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 10 monclient: shutdown 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4b406cc70 msgr2=0x7fe4b4133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 1 --2- 10.20.193.78:0/3079861144 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4b406cc70 0x7fe4b4133f30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0006dc0 tx=0x7fe4a000ed70 comp rx=0 tx=0).stop 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bad9f640 1 -- 10.20.193.78:0/3079861144 reap_dead start 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bad9f640 1 -- 10.20.193.78:0/3079861144 reap_dead start 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 shutdown_connections 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 >> 10.20.193.78:0/3079861144 conn(0x7fe4b40a6620 msgr2=0x7fe4b4138230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 shutdown_connections 2026-02-17T06:00:48.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.188+0000 7fe4bc829640 1 -- 10.20.193.78:0/3079861144 wait complete. 2026-02-17T06:00:48.203 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:00:48.203 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:00:48.203 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2455: done (3 left) 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2456: done (2 left) 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2457: done (1 left) 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2458: done (0 left) 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2459: copy_from oid 40 from oid 17 current snap is 249 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2460: write oid 48 current snap is 249 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2460: seq_num 869 ranges {605283=655623,1701911=747296} 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2460: writing trial11545420-48 from 605283 to 1260906 tid 1 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2460: writing trial11545420-48 from 1701911 to 2449207 tid 2 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2461: snap_remove snap 247 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2459: finishing copy_from to trial11545420-40 2026-02-17T06:00:48.204 INFO:tasks.rados.rados.0.trial115.stdout:2459: got expected ENOENT (src dne) 2026-02-17T06:00:48.205 INFO:tasks.rados.rados.0.trial115.stdout:2459: finishing copy_from racing read to trial11545420-40 2026-02-17T06:00:48.205 INFO:tasks.rados.rados.0.trial115.stdout:2460: finishing write tid 1 to trial11545420-48 2026-02-17T06:00:48.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.271+0000 7fae12ee8640 10 monclient: get_monmap_and_config 2026-02-17T06:00:48.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.271+0000 7fae12ee8640 10 monclient: build_initial_monmap 2026-02-17T06:00:48.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.271+0000 7fae12ee8640 10 monclient: monmap: 2026-02-17T06:00:48.274 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:48.274 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:48.272979+0000 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:48.272979+0000 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 1 Processor -- start 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 1 -- start start 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient: init 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae0c06c890 0x7fae0c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient(hunting): picked mon.noname-a con 0x7fae0c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fae0c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:48.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06d1b0 0x7fae0c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient(hunting): picked mon.noname-b con 0x7fae0c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0c06daf0 con 0x7fae0c09c4a0 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fae0c0a0880 con 0x7fae0c06c890 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae0c09cc80 con 0x7fae0c06d1b0 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae12ee8640 10 monclient(hunting): authenticate will time out at 1493.826524s 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae0c06c890 0x7fae0c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae0c06c890 0x7fae0c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49750/0 (socket says 10.20.193.78:49750) 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 1 -- 10.20.193.78:0/3982688058 learned_addr learned my addr 10.20.193.78:0/3982688058 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae1145e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06d1b0 0x7fae0c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient(hunting): get_auth_request con 0x7fae0c06c890 auth_method 0 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae1145e640 10 monclient(hunting): get_auth_request con 0x7fae0c06d1b0 auth_method 0 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae1145e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae1145e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae1145e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae1145e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae1145e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae1145e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient(hunting): handle_auth_done global_id 184116 payload 274 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 1 -- 10.20.193.78:0/3982688058 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06d1b0 msgr2=0x7fae0c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 1 --2- 10.20.193.78:0/3982688058 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06d1b0 0x7fae0c135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 1 -- 10.20.193.78:0/3982688058 >> v1:10.20.193.115:6789/0 conn(0x7fae0c09c4a0 legacy=0x7fae0c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient: _finish_hunting 0 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 1 monclient: found mon.noname-a 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:00:48.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.272+0000 7fae10c5d640 1 -- 10.20.193.78:0/3982688058 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c09cf60 con 0x7fae0c06c890 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae10c5d640 1 -- 10.20.193.78:0/3982688058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06c890 msgr2=0x7fae0c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae10c5d640 1 --2- 10.20.193.78:0/3982688058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06c890 0x7fae0c06cc70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fae0c06dcc0 tx=0x7fadfc02ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197d6a280b93835c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 1 -- 10.20.193.78:0/3982688058 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc03c070 con 0x7fae0c06c890 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: dump: 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: _finish_auth 0 2026-02-17T06:00:48.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: _check_auth_tickets 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 1 -- 10.20.193.78:0/3982688058 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc02fcc0 con 0x7fae0c06c890 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 5 monclient: authenticate success, global_id 184116 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 1 -- 10.20.193.78:0/3982688058 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc037660 con 0x7fae0c06c890 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 4 set_mon_vals no callback set 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae037fe640 10 monclient: dump: 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 10 monclient: shutdown 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 1 -- 10.20.193.78:0/3982688058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06c890 msgr2=0x7fae0c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 1 --2- 10.20.193.78:0/3982688058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06c890 0x7fae0c06cc70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fae0c06dcc0 tx=0x7fadfc02ee80 comp rx=0 tx=0).stop 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 1 -- 10.20.193.78:0/3982688058 shutdown_connections 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 1 --2- 10.20.193.78:0/3982688058 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06d1b0 0x7fae0c135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 1 --2- 10.20.193.78:0/3982688058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06c890 0x7fae0c06cc70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 1 -- 10.20.193.78:0/3982688058 >> 10.20.193.78:0/3982688058 conn(0x7fae0c0a6620 msgr2=0x7fae0c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 1 -- 10.20.193.78:0/3982688058 shutdown_connections 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.275+0000 7fae12ee8640 1 -- 10.20.193.78:0/3982688058 wait complete. 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient: build_initial_monmap 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient: monmap: 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.282 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 1 Processor -- start 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 1 -- start start 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient: init 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06c890 0x7fae0c1cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient(hunting): picked mon.noname-c con 0x7fae0c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06d1b0 0x7fae0c1cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient(hunting): picked mon.noname-a con 0x7fae0c06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c09c4a0 0x7fae0c1238a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient(hunting): picked mon.noname-b con 0x7fae0c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c0a01a0 con 0x7fae0c06d1b0 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c0a0020 con 0x7fae0c09c4a0 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae0c09be20 con 0x7fae0c06c890 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae12ee8640 10 monclient(hunting): authenticate will time out at 1493.830803s 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06c890 0x7fae0c1cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06c890 0x7fae0c1cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54530/0 (socket says 10.20.193.78:54530) 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 -- 10.20.193.78:0/1256740 learned_addr learned my addr 10.20.193.78:0/1256740 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient(hunting): get_auth_request con 0x7fae0c06c890 auth_method 0 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae03fff640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06d1b0 0x7fae0c1cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae03fff640 10 monclient(hunting): get_auth_request con 0x7fae0c06d1b0 auth_method 0 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient(hunting): handle_auth_done global_id 144503 payload 824 2026-02-17T06:00:48.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c09c4a0 msgr2=0x7fae0c1238a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c09c4a0 0x7fae0c1238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06d1b0 msgr2=0x7fae0c1cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0c06d1b0 0x7fae0c1cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient: _finish_hunting 0 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 monclient: found mon.noname-c 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.276+0000 7fae10c5d640 1 -- 10.20.193.78:0/1256740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c09d810 con 0x7fae0c06c890 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae10c5d640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06c890 msgr2=0x7fae0c1cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae10c5d640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06c890 0x7fae0c1cf730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fadfc002410 tx=0x7fadfc004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc03c040 con 0x7fae0c06c890 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: dump: 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: _finish_auth 0 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc047070 con 0x7fae0c06c890 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 5 monclient: authenticate success, global_id 144503 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 10 monclient: _renew_subs 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c0a21c0 con 0x7fae0c06c890 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc042690 con 0x7fae0c06c890 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae02ffd640 4 set_mon_vals no callback set 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae017fa640 10 monclient: dump: 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.284 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 10 monclient: _renew_subs 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c127830 con 0x7fae0c06c890 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c1245a0 0x7fae0c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.279+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c074980 con 0x7fae0c1245a0 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fadfc042870 con 0x7fae0c06c890 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c1245a0 0x7fae0c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 10 monclient: get_auth_request con 0x7fae0c1245a0 auth_method 0 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae017fa640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faddc03deb0 0x7faddc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(526..526 src has 335..526) ==== 4783+0+0 (secure 0 0 0) 0x7fadfc0779b0 con 0x7fae0c06c890 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae1145e640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faddc03deb0 0x7faddc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae1145e640 10 monclient: get_auth_request con 0x7faddc03deb0 auth_method 0 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 10 monclient(hunting): handle_auth_done global_id 134461 payload 824 2026-02-17T06:00:48.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.280+0000 7fae03fff640 1 -- 10.20.193.78:0/1256740 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae0c135ad0 con 0x7fae0c1245a0 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae03fff640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c1245a0 msgr2=0x7fae0c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae03fff640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c1245a0 0x7fae0c0709e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae0c125140 tx=0x7fadf400fc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4018030 con 0x7fae0c1245a0 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 10 monclient: dump: 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae1145e640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faddc03deb0 0x7faddc040370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fae08009fd0 tx=0x7fae0800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fadf400dbc0 con 0x7fae0c1245a0 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c1245a0 msgr2=0x7fae0c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c1245a0 0x7fae0c0709e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae0c125140 tx=0x7fadf400fc10 comp rx=0 tx=0).stop 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c1245a0 msgr2=0x7fae0c0709e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:48.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.283+0000 7fae017fa640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c1245a0 0x7fae0c0709e0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae0c125140 tx=0x7fadf400fc10 comp rx=0 tx=0).stop 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.286+0000 7fae12ee8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.286+0000 7fae12ee8640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c124980 0x7fae0c1da430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.286+0000 7fae12ee8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.286+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c125540 con 0x7fae0c124980 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c124980 0x7fae0c1da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 10 monclient: get_auth_request con 0x7fae0c124980 auth_method 0 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 10 monclient(hunting): handle_auth_done global_id 134464 payload 824 2026-02-17T06:00:48.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.287+0000 7fae03fff640 1 -- 10.20.193.78:0/1256740 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fae0c125e20 con 0x7fae0c124980 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae03fff640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c124980 msgr2=0x7fae0c1da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae03fff640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c124980 0x7fae0c1da430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fadf4018000 tx=0x7fadf4003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4004000 con 0x7fae0c124980 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 10 monclient: dump: 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.291 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7fadf4004000 con 0x7fae0c124980 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c124980 msgr2=0x7fae0c1da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c124980 0x7fae0c1da430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fadf4018000 tx=0x7fadf4003860 comp rx=0 tx=0).stop 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c124980 msgr2=0x7fae0c1da430 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.289+0000 7fae017fa640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0c124980 0x7fae0c1da430 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fadf4018000 tx=0x7fadf4003860 comp rx=0 tx=0).stop 2026-02-17T06:00:48.292 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 174, 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 276009, 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.293 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.294 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:48.295 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.296 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.297 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:48.298 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.299 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:00:48.300 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:48.301 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:48.301 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:48.301 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:48.301 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:48.301 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:48.301 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:48.301 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:00:48.301 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faddc03deb0 msgr2=0x7faddc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faddc03deb0 0x7faddc040370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fae08009fd0 tx=0x7fae0800b040 comp rx=0 tx=0).stop 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 10 monclient: shutdown 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06c890 msgr2=0x7fae0c1cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 1 --2- 10.20.193.78:0/1256740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0c06c890 0x7fae0c1cf730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fadfc002410 tx=0x7fadfc004830 comp rx=0 tx=0).stop 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae1145e640 1 -- 10.20.193.78:0/1256740 reap_dead start 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae1145e640 1 -- 10.20.193.78:0/1256740 reap_dead start 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 shutdown_connections 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 >> 10.20.193.78:0/1256740 conn(0x7fae0c0a6620 msgr2=0x7fae0c072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 shutdown_connections 2026-02-17T06:00:48.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.290+0000 7fae12ee8640 1 -- 10.20.193.78:0/1256740 wait complete. 2026-02-17T06:00:48.303 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:00:48.303 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:00:48.303 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.368+0000 7f2e782bf640 10 monclient: get_monmap_and_config 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.368+0000 7f2e782bf640 10 monclient: build_initial_monmap 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.368+0000 7f2e782bf640 10 monclient: monmap: 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:48.369811+0000 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:48.369811+0000 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.368+0000 7f2e782bf640 1 Processor -- start 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 1 -- start start 2026-02-17T06:00:48.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient: init 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2e7006c890 0x7f2e7006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient(hunting): picked mon.noname-a con 0x7f2e7006c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2e7009c4a0 0x7f2e7006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient(hunting): picked mon.noname-c con 0x7f2e7009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2e70134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e7006daf0 con 0x7f2e70134df0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f2e700a0880 con 0x7f2e7006c890 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2e7006d6f0 con 0x7f2e7009c4a0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e782bf640 10 monclient(hunting): authenticate will time out at 1493.923338s 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2e7006c890 0x7f2e7006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2e7006c890 0x7f2e7006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49774/0 (socket says 10.20.193.78:49774) 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 1 -- 10.20.193.78:0/1826180970 learned_addr learned my addr 10.20.193.78:0/1826180970 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient(hunting): get_auth_request con 0x7f2e7006c890 auth_method 0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e75833640 1 --2- 10.20.193.78:0/1826180970 >> v2:10.20.193.115:3300/0 conn(0x7f2e7009c4a0 0x7f2e7006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e75833640 10 monclient(hunting): get_auth_request con 0x7f2e7009c4a0 auth_method 0 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e75833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e75833640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e75833640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient(hunting): handle_auth_done global_id 184122 payload 274 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 1 -- 10.20.193.78:0/1826180970 >> v2:10.20.193.115:3300/0 conn(0x7f2e7009c4a0 msgr2=0x7f2e7006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 1 --2- 10.20.193.78:0/1826180970 >> v2:10.20.193.115:3300/0 conn(0x7f2e7009c4a0 0x7f2e7006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 1 -- 10.20.193.78:0/1826180970 >> v1:10.20.193.115:6789/0 conn(0x7f2e70134df0 legacy=0x7f2e70135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient: _finish_hunting 0 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 1 monclient: found mon.noname-a 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.369+0000 7f2e76034640 1 -- 10.20.193.78:0/1826180970 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7009cc80 con 0x7f2e7006c890 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.371+0000 7f2e76034640 1 -- 10.20.193.78:0/1826180970 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e7006c890 msgr2=0x7f2e7006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.371+0000 7f2e76034640 1 --2- 10.20.193.78:0/1826180970 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e7006c890 0x7f2e7006cc70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2e7009c980 tx=0x7f2e6002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e9950d14ce1427 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 1 -- 10.20.193.78:0/1826180970 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6003c070 con 0x7f2e7006c890 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: dump: 2026-02-17T06:00:48.373 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: _finish_auth 0 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: _check_auth_tickets 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 5 monclient: authenticate success, global_id 184122 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 1 -- 10.20.193.78:0/1826180970 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6002fca0 con 0x7f2e7006c890 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 1 -- 10.20.193.78:0/1826180970 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60037660 con 0x7f2e7006c890 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e75032640 10 monclient: dump: 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 4 set_mon_vals no callback set 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 10 monclient: shutdown 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 -- 10.20.193.78:0/1826180970 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e7006c890 msgr2=0x7f2e7006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1826180970 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e7006c890 0x7f2e7006cc70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2e7009c980 tx=0x7f2e6002ee60 comp rx=0 tx=0).stop 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 -- 10.20.193.78:0/1826180970 shutdown_connections 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1826180970 >> v2:10.20.193.115:3300/0 conn(0x7f2e7009c4a0 0x7f2e7006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1826180970 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e7006c890 0x7f2e7006cc70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 -- 10.20.193.78:0/1826180970 >> 10.20.193.78:0/1826180970 conn(0x7f2e700a6620 msgr2=0x7f2e700a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 -- 10.20.193.78:0/1826180970 shutdown_connections 2026-02-17T06:00:48.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 -- 10.20.193.78:0/1826180970 wait complete. 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 10 monclient: build_initial_monmap 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 10 monclient: monmap: 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.372+0000 7f2e782bf640 1 Processor -- start 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 1 -- start start 2026-02-17T06:00:48.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient: init 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e7006c890 0x7f2e701cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient(hunting): picked mon.noname-c con 0x7f2e7006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e7009c4a0 0x7f2e701cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient(hunting): picked mon.noname-b con 0x7f2e7009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e70134df0 0x7f2e70121620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient(hunting): picked mon.noname-a con 0x7f2e70134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e700a01a0 con 0x7f2e70134df0 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2e700a0020 con 0x7f2e7009c4a0 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7009be20 con 0x7f2e7006c890 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e782bf640 10 monclient(hunting): authenticate will time out at 1493.927273s 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e7006c890 0x7f2e701cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e70134df0 0x7f2e70121620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e7006c890 0x7f2e701cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54544/0 (socket says 10.20.193.78:54544) 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e70134df0 0x7f2e70121620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49784/0 (socket says 10.20.193.78:49784) 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 -- 10.20.193.78:0/1481025798 learned_addr learned my addr 10.20.193.78:0/1481025798 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient(hunting): get_auth_request con 0x7f2e7006c890 auth_method 0 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 10 monclient(hunting): get_auth_request con 0x7f2e70134df0 auth_method 0 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:48.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient(hunting): handle_auth_done global_id 144506 payload 824 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 -- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e7009c4a0 msgr2=0x7f2e701cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e75833640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e7009c4a0 0x7f2e701cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e7009c4a0 0x7f2e701cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 -- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e70134df0 msgr2=0x7f2e70121620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e70134df0 0x7f2e70121620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient: _finish_hunting 0 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 monclient: found mon.noname-c 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76034640 1 -- 10.20.193.78:0/1481025798 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e70125290 con 0x7f2e7006c890 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e76835640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e70134df0 0x7f2e70121620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e75833640 10 monclient: get_auth_request con 0x7f2e7009c4a0 auth_method 0 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.373+0000 7f2e75833640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e7009c4a0 0x7f2e701cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:00:48.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e76034640 1 -- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e7006c890 msgr2=0x7f2e701cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e76034640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e7006c890 0x7f2e701cd330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009a70 tx=0x7f2e60004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 1 -- 10.20.193.78:0/1481025798 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60047070 con 0x7f2e7006c890 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: dump: 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 1 -- 10.20.193.78:0/1481025798 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6002fe50 con 0x7f2e7006c890 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e782bf640 5 monclient: authenticate success, global_id 144506 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 1 -- 10.20.193.78:0/1481025798 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6003c040 con 0x7f2e7006c890 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e56ffd640 10 monclient: dump: 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e74831640 4 set_mon_vals no callback set 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e782bf640 10 monclient: _renew_subs 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e782bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e70123de0 con 0x7f2e7006c890 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e782bf640 10 monclient: _renew_subs 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e782bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.375+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e700a1770 con 0x7f2e7006c890 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.376+0000 7f2e782bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.376+0000 7f2e782bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.376+0000 7f2e782bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.376+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e70074980 con 0x7f2e7006c890 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.376+0000 7f2e56ffd640 1 -- 10.20.193.78:0/1481025798 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2e6003f7d0 con 0x7f2e7006c890 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.376+0000 7f2e56ffd640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e4003df00 0x7f2e400403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.376+0000 7f2e56ffd640 1 -- 10.20.193.78:0/1481025798 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(526..526 src has 335..526) ==== 4783+0+0 (secure 0 0 0) 0x7f2e60076f30 con 0x7f2e7006c890 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.377+0000 7f2e75833640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e4003df00 0x7f2e400403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:48.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.377+0000 7f2e75833640 10 monclient: get_auth_request con 0x7f2e4003df00 auth_method 0 2026-02-17T06:00:48.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.379+0000 7f2e56ffd640 1 -- 10.20.193.78:0/1481025798 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2e60035e10 con 0x7f2e7006c890 2026-02-17T06:00:48.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.379+0000 7f2e56ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:48.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.379+0000 7f2e56ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:48.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.380+0000 7f2e75833640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e4003df00 0x7f2e400403c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2e70122e80 tx=0x7f2e64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:48.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.494+0000 7f2e782bf640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:00:48.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.494+0000 7f2e782bf640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:00:48.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.494+0000 7f2e782bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:48.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.494+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f2e70135a20 con 0x7f2e7006c890 2026-02-17T06:00:48.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.495+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.495+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=12,kv=12,logm=49} crc {auth=1619277854,config=692619927,health=2906667990,kv=449989585,logm=2630634496}) 2026-02-17T06:00:48.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.495+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=26,config=2,health=12,kv=12,logm=48} crc {auth=945975066,config=692619927,health=2906667990,kv=449989585,logm=3880952191}) 2026-02-17T06:00:48.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.495+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.495+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=12,kv=12,logm=49} crc {auth=1619277854,config=692619927,health=2906667990,kv=449989585,logm=2630634496}) 2026-02-17T06:00:48.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.495+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=24,config=2,health=12,kv=11,logm=51} crc {auth=2201864541,config=692619927,health=2906667990,kv=2589905713,logm=986103871}) 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.495+0000 7f2e56ffd640 1 -- 10.20.193.78:0/1481025798 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f2e6004ac10 con 0x7f2e7006c890 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.495+0000 7f2e56ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.495+0000 7f2e56ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e4003df00 msgr2=0x7f2e400403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e4003df00 0x7f2e400403c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2e70122e80 tx=0x7f2e64008040 comp rx=0 tx=0).stop 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 10 monclient: shutdown 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e7006c890 msgr2=0x7f2e701cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e7006c890 0x7f2e701cd330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009a70 tx=0x7f2e60004830 comp rx=0 tx=0).stop 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 shutdown_connections 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2e70134df0 0x7f2e70121620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2e4003df00 0x7f2e400403c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2e7009c4a0 0x7f2e701cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 --2- 10.20.193.78:0/1481025798 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2e7006c890 0x7f2e701cd330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 >> 10.20.193.78:0/1481025798 conn(0x7f2e700a6620 msgr2=0x7f2e70071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 shutdown_connections 2026-02-17T06:00:48.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:48.497+0000 7f2e782bf640 1 -- 10.20.193.78:0/1481025798 wait complete. 2026-02-17T06:00:48.506 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:00:48.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3618974654}) 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2739702025}) 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3618974654}) 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1760851033}) 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=874351116}) 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2864220853}) 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=874351116}) 2026-02-17T06:00:48.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2387862102}) 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151625360}) 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3308320884}) 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151625360}) 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155740991}) 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1094847910}) 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=24525178}) 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1094847910}) 2026-02-17T06:00:48.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=480483140}) 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4044275939}) 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067751692}) 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4044275939}) 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1347575763}) 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1911061631}) 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=796941290}) 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1911061631}) 2026-02-17T06:00:48.557 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4107468650}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3055432626}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=152157823}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3055432626}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770288198}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1117256119}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2002713958}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1117256119}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3046637176}) 2026-02-17T06:00:48.558 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2354938184}) 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134898511}) 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2354938184}) 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035259918}) 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137117814}) 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2895475517}) 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137117814}) 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4245295358}) 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.559 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=24922210}) 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3886976}) 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=24922210}) 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1736951618}) 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2071567775}) 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1227056664}) 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2071567775}) 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3461438003}) 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.560 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2999910874}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3335024922}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2999910874}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2734932995}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=803789161}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986867433}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=803789161}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=871145379}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1812354288}) 2026-02-17T06:00:48.561 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=281745375}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1812354288}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1512689752}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4172838583}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1523916282}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4172838583}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2092167547}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2593314108}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1124633807}) 2026-02-17T06:00:48.562 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2593314108}) 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.545+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2110457495}) 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2250164772}) 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2610652366}) 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2250164772}) 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2955273945}) 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=443850497}) 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1042428803}) 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.563 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=443850497}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2293190204}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3518101962}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3968422270}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3518101962}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.546+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=552441370}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2265752708}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3557173125}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2265752708}) 2026-02-17T06:00:48.564 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2896289612}) 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1250995694}) 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3228884114}) 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1250995694}) 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435298495}) 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2609047510}) 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=897456286}) 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2609047510}) 2026-02-17T06:00:48.565 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.547+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4013347409}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=499567622}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2648881327}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=499567622}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441717139}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1782723808}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1670273215}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1782723808}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1587783837}) 2026-02-17T06:00:48.566 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=700465411}) 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3152253826}) 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=700465411}) 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=770111857}) 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763922789}) 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3383177646}) 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763922789}) 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.548+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=367993476}) 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.567 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2911725896}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3002693437}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2911725896}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=323908839}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2836207930}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3949106030}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2836207930}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2051908515}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=450842346}) 2026-02-17T06:00:48.568 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=759198858}) 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=450842346}) 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.549+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=598846582}) 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=980315692}) 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4251351367}) 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=980315692}) 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3867855320}) 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652552464}) 2026-02-17T06:00:48.569 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1512166340}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652552464}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3809769197}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=389774081}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3559750177}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=389774081}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1623531977}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=433227631}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4090350702}) 2026-02-17T06:00:48.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=433227631}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.550+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3082361778}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692521591}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3523329993}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692521591}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4022846286}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2448689990}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80867609}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2448689990}) 2026-02-17T06:00:48.571 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.551+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3181268787}) 2026-02-17T06:00:48.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191601989}) 2026-02-17T06:00:48.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=122079984}) 2026-02-17T06:00:48.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191601989}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=338616501}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1730272927}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2452814007}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1730272927}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4106666846}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2263032421}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1867263511}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2263032421}) 2026-02-17T06:00:48.585 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.581+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1862749657}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=399935487}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4198815948}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=399935487}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=616520985}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3733623100}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4189579629}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3733623100}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=885336721}) 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681742499}) 2026-02-17T06:00:48.587 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3862119252}) 2026-02-17T06:00:48.587 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.587 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681742499}) 2026-02-17T06:00:48.587 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.582+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=475547451}) 2026-02-17T06:00:48.595 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.589+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.595 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.589+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2217765988}) 2026-02-17T06:00:48.595 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.589+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3964595225}) 2026-02-17T06:00:48.595 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.589+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.595 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.589+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2217765988}) 2026-02-17T06:00:48.595 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.589+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=164542461}) 2026-02-17T06:00:48.595 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.595 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3835415765}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4123752488}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3835415765}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3200988167}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1287411099}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1599378020}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1287411099}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.590+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391868824}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2392511828}) 2026-02-17T06:00:48.596 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002604940}) 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2392511828}) 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2831655351}) 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=87,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=1} crc {logm=4292102235,mdsmap=1160443688,mgr=787438293,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4133068992}) 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=85,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=4} crc {logm=1259125384,mdsmap=1160443688,mgr=1115775221,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4040869149}) 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=87,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=1} crc {logm=4292102235,mdsmap=1160443688,mgr=787438293,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4133068992}) 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=87,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1} crc {logm=2919295529,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571}) 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.597 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=10,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=69} crc {mgrstat=345827803,mon_config_key=2934532541,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=1583479185,osd_snap=3008392893}) 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=67} crc {mgrstat=345827803,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=1583479185,osd_snap=4144149760}) 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=10,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=69} crc {mgrstat=345827803,mon_config_key=2934532541,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=1583479185,osd_snap=3008392893}) 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.591+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=69} crc {mgrstat=3497847635,mon_config_key=1912222953,monmap=1784110042,osd_metadata=1246192198,osd_pg_creating=1583479185,osd_snap=2889915259}) 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=25,osdmap=75} crc {osd_snap=2311415521,osdmap=3144424685}) 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=24,osdmap=76} crc {osd_snap=1099950501,osdmap=4285101421}) 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=25,osdmap=75} crc {osd_snap=2311415521,osdmap=3144424685}) 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=24,osdmap=76} crc {osd_snap=1041040741,osdmap=1096683174}) 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1845065346}) 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2875819066}) 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1845065346}) 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.592+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=388402203}) 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.593+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.593+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3710254981}) 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.593+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2379357080}) 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.593+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.593+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3710254981}) 2026-02-17T06:00:48.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.593+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=549018702}) 2026-02-17T06:00:48.600 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.596+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.600 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.596+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=13} crc {osdmap=2475863164}) 2026-02-17T06:00:48.600 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.596+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=12} crc {osdmap=4044938471}) 2026-02-17T06:00:48.600 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.596+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:00:48.600 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.596+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=13} crc {osdmap=2475863164}) 2026-02-17T06:00:48.601 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:00:48.596+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=11} crc {osdmap=1221345677}) 2026-02-17T06:00:49.199 INFO:tasks.rados.rados.0.trial115.stdout:2459: done (2 left) 2026-02-17T06:00:49.199 INFO:tasks.rados.rados.0.trial115.stdout:2461: done (1 left) 2026-02-17T06:00:49.199 INFO:tasks.rados.rados.0.trial115.stdout:2462: snap_remove snap 246 2026-02-17T06:00:49.507 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:00:49.507 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:49.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.573+0000 7ff58e4de640 10 monclient: get_monmap_and_config 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.573+0000 7ff58e4de640 10 monclient: build_initial_monmap 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.573+0000 7ff58e4de640 10 monclient: monmap: 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:49.575162+0000 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:49.575162+0000 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.575 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:49.576 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:49.576 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:49.576 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:49.576 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:49.576 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 1 Processor -- start 2026-02-17T06:00:49.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 1 -- start start 2026-02-17T06:00:49.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient: init 2026-02-17T06:00:49.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff58806c890 0x7ff58806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient(hunting): picked mon.noname-a con 0x7ff58806c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809d430 0x7ff58806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient(hunting): picked mon.noname-b con 0x7ff58809d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff58809bf40 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff588073600 con 0x7ff58809bf40 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff58809ca50 con 0x7ff58806c890 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff58806d6f0 con 0x7ff58809d430 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58e4de640 10 monclient(hunting): authenticate will time out at 1495.128742s 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff58806c890 0x7ff58806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff5877fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809d430 0x7ff58806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58ca54640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff58809bf40 0x7ff588134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:59334/0 (socket says 10.20.193.78:59334) 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff58ca54640 1 -- 10.20.193.78:0/1052092418 learned_addr learned my addr 10.20.193.78:0/1052092418 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 10 monclient(hunting): get_auth_request con 0x7ff58806c890 auth_method 0 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff5877fe640 10 monclient(hunting): get_auth_request con 0x7ff58809d430 auth_method 0 2026-02-17T06:00:49.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff5877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff5877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff5877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 10 monclient(hunting): handle_auth_done global_id 184128 payload 274 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 1 -- 10.20.193.78:0/1052092418 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809d430 msgr2=0x7ff58806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.574+0000 7ff587fff640 1 --2- 10.20.193.78:0/1052092418 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809d430 0x7ff58806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.575+0000 7ff587fff640 1 -- 10.20.193.78:0/1052092418 >> v1:10.20.193.78:6789/0 conn(0x7ff58809bf40 legacy=0x7ff588134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:49.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.575+0000 7ff587fff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.575+0000 7ff587fff640 1 monclient: found mon.noname-a 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.575+0000 7ff587fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.575+0000 7ff587fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.575+0000 7ff587fff640 1 -- 10.20.193.78:0/1052092418 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58806dab0 con 0x7ff58806c890 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff587fff640 1 -- 10.20.193.78:0/1052092418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58806c890 msgr2=0x7ff58806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff587fff640 1 --2- 10.20.193.78:0/1052092418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58806c890 0x7ff58806cc70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff58806a480 tx=0x7ff57402ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea3c5886987e54 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 1 -- 10.20.193.78:0/1052092418 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57403c070 con 0x7ff58806c890 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: dump: 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.588 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: _finish_auth 0 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:49.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 1 -- 10.20.193.78:0/1052092418 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57402fbf0 con 0x7ff58806c890 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 5 monclient: authenticate success, global_id 184128 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 1 -- 10.20.193.78:0/1052092418 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574037440 con 0x7ff58806c890 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 4 set_mon_vals no callback set 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff586ffd640 10 monclient: dump: 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.590 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 10 monclient: shutdown 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 1 -- 10.20.193.78:0/1052092418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58806c890 msgr2=0x7ff58806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 1 --2- 10.20.193.78:0/1052092418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58806c890 0x7ff58806cc70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff58806a480 tx=0x7ff57402ed70 comp rx=0 tx=0).stop 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 1 -- 10.20.193.78:0/1052092418 shutdown_connections 2026-02-17T06:00:49.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 1 --2- 10.20.193.78:0/1052092418 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809d430 0x7ff58806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 1 --2- 10.20.193.78:0/1052092418 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58806c890 0x7ff58806cc70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 1 -- 10.20.193.78:0/1052092418 >> 10.20.193.78:0/1052092418 conn(0x7ff5880a6620 msgr2=0x7ff5880a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 1 -- 10.20.193.78:0/1052092418 shutdown_connections 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.577+0000 7ff58e4de640 1 -- 10.20.193.78:0/1052092418 wait complete. 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient: build_initial_monmap 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient: monmap: 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.592 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 1 Processor -- start 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 1 -- start start 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient: init 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58806c890 0x7ff588133e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient(hunting): picked mon.noname-b con 0x7ff58806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809bf40 0x7ff5881343c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient(hunting): picked mon.noname-c con 0x7ff58809bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58809d430 0x7ff5881218f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient(hunting): picked mon.noname-a con 0x7ff58809d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5880733c0 con 0x7ff58809d430 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff588073240 con 0x7ff58806c890 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff588135500 con 0x7ff58809bf40 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.578+0000 7ff58e4de640 10 monclient(hunting): authenticate will time out at 1495.132963s 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58809d430 0x7ff5881218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff5877fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809bf40 0x7ff5881343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58809d430 0x7ff5881218f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49812/0 (socket says 10.20.193.78:49812) 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 -- 10.20.193.78:0/3658914786 learned_addr learned my addr 10.20.193.78:0/3658914786 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient(hunting): get_auth_request con 0x7ff58809d430 auth_method 0 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff5877fe640 10 monclient(hunting): get_auth_request con 0x7ff58809bf40 auth_method 0 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff5877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff5877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff5877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient(hunting): handle_auth_done global_id 184131 payload 824 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 -- 10.20.193.78:0/3658914786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809bf40 msgr2=0x7ff5881343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809bf40 0x7ff5881343c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 -- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58806c890 msgr2=0x7ff588133e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58806c890 0x7ff588133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient: _finish_hunting 0 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 monclient: found mon.noname-a 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.579+0000 7ff58ca54640 1 -- 10.20.193.78:0/3658914786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff588124310 con 0x7ff58809d430 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58ca54640 1 -- 10.20.193.78:0/3658914786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58809d430 msgr2=0x7ff5881218f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58ca54640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58809d430 0x7ff5881218f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00caf0 tx=0x7ff57c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 1 -- 10.20.193.78:0/3658914786 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c017070 con 0x7ff58809d430 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: dump: 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.597 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 1 -- 10.20.193.78:0/3658914786 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c00adf0 con 0x7ff58809d430 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 5 monclient: authenticate success, global_id 184131 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 1 -- 10.20.193.78:0/3658914786 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c012640 con 0x7ff58809d430 2026-02-17T06:00:49.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff584ff9640 10 monclient: dump: 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff5867fc640 4 set_mon_vals no callback set 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 10 monclient: _renew_subs 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5881255a0 con 0x7ff58809d430 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 10 monclient: _renew_subs 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5880a1a50 con 0x7ff58809d430 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.581+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58806d510 con 0x7ff58809d430 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.582+0000 7ff584ff9640 1 -- 10.20.193.78:0/3658914786 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff57c01b3b0 con 0x7ff58809d430 2026-02-17T06:00:49.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.582+0000 7ff584ff9640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55803deb0 0x7ff558040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.582+0000 7ff584ff9640 1 -- 10.20.193.78:0/3658914786 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(527..527 src has 335..527) ==== 4827+0+0 (secure 0 0 0) 0x7ff57c052750 con 0x7ff58809d430 2026-02-17T06:00:49.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.582+0000 7ff587fff640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55803deb0 0x7ff558040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.582+0000 7ff587fff640 10 monclient: get_auth_request con 0x7ff55803deb0 auth_method 0 2026-02-17T06:00:49.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.585+0000 7ff584ff9640 1 -- 10.20.193.78:0/3658914786 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff58806d510 con 0x7ff58809d430 2026-02-17T06:00:49.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.585+0000 7ff584ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.585+0000 7ff584ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:49.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.585+0000 7ff587fff640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55803deb0 0x7ff558040370 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff58809c670 tx=0x7ff574031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.698+0000 7ff58e4de640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:49.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.698+0000 7ff58e4de640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:49.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.698+0000 7ff58e4de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.698+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff588135930 con 0x7ff58809d430 2026-02-17T06:00:49.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.698+0000 7ff584ff9640 1 -- 10.20.193.78:0/3658914786 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7ff57c01b8b0 con 0x7ff58809d430 2026-02-17T06:00:49.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.698+0000 7ff584ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:49.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.698+0000 7ff584ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:49.700 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":174,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:49.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55803deb0 msgr2=0x7ff558040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55803deb0 0x7ff558040370 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff58809c670 tx=0x7ff574031000 comp rx=0 tx=0).stop 2026-02-17T06:00:49.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 10 monclient: shutdown 2026-02-17T06:00:49.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58809d430 msgr2=0x7ff5881218f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58809d430 0x7ff5881218f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00caf0 tx=0x7ff57c005c50 comp rx=0 tx=0).stop 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 shutdown_connections 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff58809d430 0x7ff5881218f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff58809bf40 0x7ff5881343c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff55803deb0 0x7ff558040370 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 --2- 10.20.193.78:0/3658914786 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff58806c890 0x7ff588133e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 >> 10.20.193.78:0/3658914786 conn(0x7ff5880a6620 msgr2=0x7ff588136910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 shutdown_connections 2026-02-17T06:00:49.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.699+0000 7ff58e4de640 1 -- 10.20.193.78:0/3658914786 wait complete. 2026-02-17T06:00:49.707 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:00:49.708 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:00:49.708 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:00:49.708 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:00:49.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.775+0000 7f7787723640 10 monclient: get_monmap_and_config 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.775+0000 7f7787723640 10 monclient: build_initial_monmap 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.775+0000 7f7787723640 10 monclient: monmap: 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:49.777130+0000 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:49.777130+0000 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.777 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:49.778 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:49.778 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:49.778 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:49.778 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:49.778 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 1 Processor -- start 2026-02-17T06:00:49.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 1 -- start start 2026-02-17T06:00:49.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient: init 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f778006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f778009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778006d380 0x7f778013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient(hunting): picked mon.noname-b con 0x7f778006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778006daf0 con 0x7f778009c4a0 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778009cc80 con 0x7f778006c890 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f77800a0880 con 0x7f778006d380 2026-02-17T06:00:49.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7787723640 10 monclient(hunting): authenticate will time out at 1495.330660s 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778006d380 0x7f778013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7784c97640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f778009c4a0 0x7f778013b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:59346/0 (socket says 10.20.193.78:59346) 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7784c97640 1 -- 10.20.193.78:0/1799298582 learned_addr learned my addr 10.20.193.78:0/1799298582 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient(hunting): get_auth_request con 0x7f778006d380 auth_method 0 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient(hunting): handle_auth_done global_id 144515 payload 274 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 1 -- 10.20.193.78:0/1799298582 >> v1:10.20.193.115:6789/0 conn(0x7f778006c890 legacy=0x7f778006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 1 -- 10.20.193.78:0/1799298582 >> v1:10.20.193.78:6789/0 conn(0x7f778009c4a0 legacy=0x7f778013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient: _finish_hunting 0 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 1 monclient: found mon.noname-b 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.776+0000 7f7785c99640 1 -- 10.20.193.78:0/1799298582 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778009cf60 con 0x7f778006d380 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7785c99640 1 -- 10.20.193.78:0/1799298582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778006d380 msgr2=0x7f778013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7785c99640 1 --2- 10.20.193.78:0/1799298582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778006d380 0x7f778013ead0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f777c009dc0 tx=0x7f777c02f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affe5c3f88bdea92 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 1 -- 10.20.193.78:0/1799298582 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c03c070 con 0x7f778006d380 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: dump: 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: _finish_auth 0 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 5 monclient: authenticate success, global_id 144515 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 1 -- 10.20.193.78:0/1799298582 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f777c037640 con 0x7f778006d380 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 1 -- 10.20.193.78:0/1799298582 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c037a10 con 0x7f778006d380 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f776ffff640 10 monclient: dump: 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.791 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 4 set_mon_vals no callback set 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 10 monclient: shutdown 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:49.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 1 -- 10.20.193.78:0/1799298582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778006d380 msgr2=0x7f778013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 1 --2- 10.20.193.78:0/1799298582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778006d380 0x7f778013ead0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f777c009dc0 tx=0x7f777c02f1c0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 1 -- 10.20.193.78:0/1799298582 shutdown_connections 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 1 --2- 10.20.193.78:0/1799298582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778006d380 0x7f778013ead0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 1 -- 10.20.193.78:0/1799298582 >> 10.20.193.78:0/1799298582 conn(0x7f77800a6620 msgr2=0x7f77800a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 1 -- 10.20.193.78:0/1799298582 shutdown_connections 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.779+0000 7f7787723640 1 -- 10.20.193.78:0/1799298582 wait complete. 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient: build_initial_monmap 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient: monmap: 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:49.793 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 1 Processor -- start 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 1 -- start start 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient: init 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:00:49.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f778006c890 0x7f77801ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient(hunting): picked mon.noname-b con 0x7f778006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f778006d380 0x7f77801cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient(hunting): picked mon.noname-a con 0x7f778006d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778009c4a0 0x7f77801cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient(hunting): picked mon.noname-c con 0x7f778009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f778009bfa0 con 0x7f778006d380 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f778009be20 con 0x7f778006c890 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f778013f010 con 0x7f778009c4a0 2026-02-17T06:00:49.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7787723640 10 monclient(hunting): authenticate will time out at 1495.334907s 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7784c97640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f778006d380 0x7f77801cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7785c99640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778009c4a0 0x7f77801cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7784c97640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f778006d380 0x7f77801cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49814/0 (socket says 10.20.193.78:49814) 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7784c97640 1 -- 10.20.193.78:0/599161242 learned_addr learned my addr 10.20.193.78:0/599161242 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7784c97640 10 monclient(hunting): get_auth_request con 0x7f778006d380 auth_method 0 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7784c97640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7784c97640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7784c97640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7785c99640 10 monclient(hunting): get_auth_request con 0x7f778009c4a0 auth_method 0 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7785c99640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7785c99640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.780+0000 7f7785c99640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7784c97640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7784c97640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7784c97640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 10 monclient(hunting): handle_auth_done global_id 144518 payload 824 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f778006c890 msgr2=0x7f77801ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785498640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f778006c890 0x7f77801ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f778006c890 0x7f77801ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f778006d380 msgr2=0x7f77801cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f778006d380 0x7f77801cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 10 monclient: _finish_hunting 0 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 1 monclient: found mon.noname-c 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785c99640 1 -- 10.20.193.78:0/599161242 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778006fdd0 con 0x7f778009c4a0 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785498640 10 monclient: get_auth_request con 0x7f778006c890 auth_method 0 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7785498640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f778006c890 0x7f77801ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.781+0000 7f7784c97640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f778006d380 0x7f77801cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7785c99640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778009c4a0 msgr2=0x7f77801cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7785c99640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778009c4a0 0x7f77801cda40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f777c036000 tx=0x7f777c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c047020 con 0x7f778009c4a0 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: dump: 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.799 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: _finish_auth 0 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f777c042880 con 0x7f778009c4a0 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c03c040 con 0x7f778009c4a0 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 5 monclient: authenticate success, global_id 144518 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776f7fe640 4 set_mon_vals no callback set 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:49.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f776dffb640 10 monclient: dump: 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 10 monclient: _renew_subs 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:49.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778011f980 con 0x7f778009c4a0 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 10 monclient: _renew_subs 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7780072960 con 0x7f778009c4a0 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7780136200 0x7f7780139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.783+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f778006cc70 con 0x7f7780136200 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7780136200 0x7f7780139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 10 monclient: get_auth_request con 0x7f7780136200 auth_method 0 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f777c042cb0 con 0x7f778009c4a0 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 10 monclient(hunting): handle_auth_done global_id 184137 payload 824 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7785498640 1 -- 10.20.193.78:0/599161242 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f778011e6f0 con 0x7f7780136200 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f776dffb640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f775003deb0 0x7f7750040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(527..527 src has 335..527) ==== 4827+0+0 (secure 0 0 0) 0x7f777c078610 con 0x7f778009c4a0 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7784c97640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f775003deb0 0x7f7750040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.784+0000 7f7784c97640 10 monclient: get_auth_request con 0x7f775003deb0 auth_method 0 2026-02-17T06:00:49.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f7785498640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7780136200 msgr2=0x7f7780139390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f7785498640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7780136200 0x7f7780139390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f777000d6f0 tx=0x7f777000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770016020 con 0x7f7780136200 2026-02-17T06:00:49.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 10 monclient: dump: 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7770005460 con 0x7f7780136200 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7780136200 msgr2=0x7f7780139390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7780136200 0x7f7780139390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f777000d6f0 tx=0x7f777000dbc0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7780136200 msgr2=0x7f7780139390 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.786+0000 7f776dffb640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7780136200 0x7f7780139390 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f777000d6f0 tx=0x7f777000dbc0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.787+0000 7f7784c97640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f775003deb0 0x7f7750040370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7774007e50 tx=0x7f7774007da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7787723640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7787723640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77801365e0 0x7f77801d8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7787723640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f778013aa50 con 0x7f77801365e0 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77801365e0 0x7f77801d8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 10 monclient: get_auth_request con 0x7f77801365e0 auth_method 0 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 10 monclient(hunting): handle_auth_done global_id 184140 payload 824 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.789+0000 7f7785498640 1 -- 10.20.193.78:0/599161242 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f77800744d0 con 0x7f77801365e0 2026-02-17T06:00:49.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7785498640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77801365e0 msgr2=0x7f77801d8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7785498640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77801365e0 0x7f77801d8520 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f777000d6f0 tx=0x7f7770003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770016030 con 0x7f77801365e0 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 10 monclient: dump: 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.810 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f777000b720 con 0x7f77801365e0 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77801365e0 msgr2=0x7f77801d8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77801365e0 0x7f77801d8520 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f777000d6f0 tx=0x7f7770003860 comp rx=0 tx=0).stop 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77801365e0 msgr2=0x7f77801d8520 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f776dffb640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77801365e0 0x7f77801d8520 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f777000d6f0 tx=0x7f7770003860 comp rx=0 tx=0).stop 2026-02-17T06:00:49.811 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 174, 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 14573, 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:00:49.812 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.813 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.814 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:00:49.815 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:00:49.816 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.817 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.818 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:49.819 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:00:49.820 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:49.821 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:49.822 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:49.822 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.822 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.822 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.822 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:00:49.822 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:00:49.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f775003deb0 msgr2=0x7f7750040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f775003deb0 0x7f7750040370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7774007e50 tx=0x7f7774007da0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 10 monclient: shutdown 2026-02-17T06:00:49.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:49.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778009c4a0 msgr2=0x7f77801cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 1 --2- 10.20.193.78:0/599161242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f778009c4a0 0x7f77801cda40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f777c036000 tx=0x7f777c004b40 comp rx=0 tx=0).stop 2026-02-17T06:00:49.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7785c99640 1 -- 10.20.193.78:0/599161242 reap_dead start 2026-02-17T06:00:49.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7785c99640 1 -- 10.20.193.78:0/599161242 reap_dead start 2026-02-17T06:00:49.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 shutdown_connections 2026-02-17T06:00:49.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 >> 10.20.193.78:0/599161242 conn(0x7f77800a6620 msgr2=0x7f77800714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:49.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 shutdown_connections 2026-02-17T06:00:49.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.792+0000 7f7787723640 1 -- 10.20.193.78:0/599161242 wait complete. 2026-02-17T06:00:49.824 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:00:49.824 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:00:49.824 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:00:49.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.892+0000 7fa8a2643640 10 monclient: get_monmap_and_config 2026-02-17T06:00:49.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.892+0000 7fa8a2643640 10 monclient: build_initial_monmap 2026-02-17T06:00:49.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.892+0000 7fa8a2643640 10 monclient: monmap: 2026-02-17T06:00:49.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:49.893 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.893 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:49.893614+0000 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:49.893614+0000 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:49.894 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.892+0000 7fa8a2643640 1 Processor -- start 2026-02-17T06:00:49.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.892+0000 7fa8a2643640 1 -- start start 2026-02-17T06:00:49.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.892+0000 7fa8a2643640 10 monclient: init 2026-02-17T06:00:49.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:49.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa89c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa89c09c4a0 0x7fa89c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient(hunting): picked mon.noname-c con 0x7fa89c09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06d380 0x7fa89c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient(hunting): picked mon.noname-b con 0x7fa89c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89c06daf0 con 0x7fa89c06c890 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa89c0a0880 con 0x7fa89c09c4a0 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c09cc80 con 0x7fa89c06d380 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a2643640 10 monclient(hunting): authenticate will time out at 1495.447177s 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa89bfff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fa89c06c890 0x7fa89c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:59356/0 (socket says 10.20.193.78:59356) 2026-02-17T06:00:49.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa89bfff640 1 -- 10.20.193.78:0/2507147155 learned_addr learned my addr 10.20.193.78:0/2507147155 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 1 --2- 10.20.193.78:0/2507147155 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06d380 0x7fa89c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient(hunting): get_auth_request con 0x7fa89c06d380 auth_method 0 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa89b7fe640 1 --2- 10.20.193.78:0/2507147155 >> v2:10.20.193.115:3300/0 conn(0x7fa89c09c4a0 0x7fa89c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa89b7fe640 10 monclient(hunting): get_auth_request con 0x7fa89c09c4a0 auth_method 0 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa89b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa89b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa89b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient(hunting): handle_auth_done global_id 144521 payload 274 2026-02-17T06:00:49.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/2507147155 >> v2:10.20.193.115:3300/0 conn(0x7fa89c09c4a0 msgr2=0x7fa89c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 1 --2- 10.20.193.78:0/2507147155 >> v2:10.20.193.115:3300/0 conn(0x7fa89c09c4a0 0x7fa89c13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/2507147155 >> v1:10.20.193.78:6789/0 conn(0x7fa89c06c890 legacy=0x7fa89c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient: _finish_hunting 0 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 1 monclient: found mon.noname-b 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.893+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/2507147155 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c09cf60 con 0x7fa89c06d380 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.895+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/2507147155 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06d380 msgr2=0x7fa89c13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.895+0000 7fa8a0bb9640 1 --2- 10.20.193.78:0/2507147155 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06d380 0x7fa89c13d8f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa890009d90 tx=0x7fa89002f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f14a8b7564ed3a1b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.895+0000 7fa89affd640 1 -- 10.20.193.78:0/2507147155 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89003c070 con 0x7fa89c06d380 2026-02-17T06:00:49.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.895+0000 7fa89affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:49.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 10 monclient: dump: 2026-02-17T06:00:49.906 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.906 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 10 monclient: _finish_auth 0 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 10 monclient: _check_auth_tickets 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 1 -- 10.20.193.78:0/2507147155 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa890037440 con 0x7fa89c06d380 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 5 monclient: authenticate success, global_id 144521 2026-02-17T06:00:49.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 1 -- 10.20.193.78:0/2507147155 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890037810 con 0x7fa89c06d380 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 4 set_mon_vals no callback set 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa89affd640 10 monclient: dump: 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.908 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 10 monclient: shutdown 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 -- 10.20.193.78:0/2507147155 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06d380 msgr2=0x7fa89c13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 --2- 10.20.193.78:0/2507147155 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06d380 0x7fa89c13d8f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa890009d90 tx=0x7fa89002f190 comp rx=0 tx=0).stop 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 -- 10.20.193.78:0/2507147155 shutdown_connections 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 --2- 10.20.193.78:0/2507147155 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06d380 0x7fa89c13d8f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 --2- 10.20.193.78:0/2507147155 >> v2:10.20.193.115:3300/0 conn(0x7fa89c09c4a0 0x7fa89c13b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 -- 10.20.193.78:0/2507147155 >> 10.20.193.78:0/2507147155 conn(0x7fa89c0a6620 msgr2=0x7fa89c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 -- 10.20.193.78:0/2507147155 shutdown_connections 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 -- 10.20.193.78:0/2507147155 wait complete. 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 10 monclient: build_initial_monmap 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 10 monclient: monmap: 2026-02-17T06:00:49.909 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 Processor -- start 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 1 -- start start 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.896+0000 7fa8a2643640 10 monclient: init 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06c890 0x7fa89c1cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient(hunting): picked mon.noname-c con 0x7fa89c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa89c06d380 0x7fa89c130360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient(hunting): picked mon.noname-b con 0x7fa89c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa89c09c4a0 0x7fa89c13b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient(hunting): picked mon.noname-a con 0x7fa89c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c09bfa0 con 0x7fa89c09c4a0 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c09be20 con 0x7fa89c06d380 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c13de30 con 0x7fa89c06c890 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a2643640 10 monclient(hunting): authenticate will time out at 1495.451219s 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06c890 0x7fa89c1cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa89c09c4a0 0x7fa89c13b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa89c09c4a0 0x7fa89c13b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49842/0 (socket says 10.20.193.78:49842) 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/618652874 learned_addr learned my addr 10.20.193.78:0/618652874 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient(hunting): get_auth_request con 0x7fa89c09c4a0 auth_method 0 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 10 monclient(hunting): get_auth_request con 0x7fa89c06c890 auth_method 0 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89b7fe640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa89c06d380 0x7fa89c130360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89b7fe640 10 monclient(hunting): get_auth_request con 0x7fa89c06d380 auth_method 0 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient(hunting): handle_auth_done global_id 184143 payload 824 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06c890 msgr2=0x7fa89c1cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06c890 0x7fa89c1cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa89c06d380 msgr2=0x7fa89c130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa89c06d380 0x7fa89c130360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient: _finish_hunting 0 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 monclient: found mon.noname-a 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/618652874 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c09d810 con 0x7fa89c09c4a0 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.897+0000 7fa89bfff640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c06c890 0x7fa89c1cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa89c09c4a0 msgr2=0x7fa89c13b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a0bb9640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa89c09c4a0 0x7fa89c13b890 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa890009d90 tx=0x7fa890004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89003c050 con 0x7fa89c09c4a0 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: dump: 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.914 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: _finish_auth 0 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa890048070 con 0x7fa89c09c4a0 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a2643640 5 monclient: authenticate success, global_id 184143 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a2643640 10 monclient: _renew_subs 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a2643640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c13da60 con 0x7fa89c09c4a0 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890041c80 con 0x7fa89c09c4a0 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:49.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa89a7fc640 4 set_mon_vals no callback set 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa898ff9640 10 monclient: dump: 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a2643640 10 monclient: _renew_subs 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a2643640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.899+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c09fb20 con 0x7fa89c09c4a0 2026-02-17T06:00:49.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa8a2643640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa8a2643640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c09f440 0x7fa89c13bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa8a2643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c0a3ed0 con 0x7fa89c09f440 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c09f440 0x7fa89c13bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa89004d460 con 0x7fa89c09c4a0 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 10 monclient: get_auth_request con 0x7fa89c09f440 auth_method 0 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 10 monclient(hunting): handle_auth_done global_id 144527 payload 824 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89bfff640 1 -- 10.20.193.78:0/618652874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa89c074980 con 0x7fa89c09f440 2026-02-17T06:00:49.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa898ff9640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa86c03deb0 0x7fa86c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(527..527 src has 335..527) ==== 4827+0+0 (secure 0 0 0) 0x7fa8900785b0 con 0x7fa89c09c4a0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89b7fe640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa86c03deb0 0x7fa86c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.900+0000 7fa89b7fe640 10 monclient: get_auth_request con 0x7fa86c03deb0 auth_method 0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.902+0000 7fa89bfff640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c09f440 msgr2=0x7fa89c13bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.902+0000 7fa89bfff640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c09f440 0x7fa89c13bdd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa888007400 tx=0x7fa8880078d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.902+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88800fdc0 con 0x7fa89c09f440 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.902+0000 7fa898ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.902+0000 7fa898ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.902+0000 7fa898ff9640 10 monclient: dump: 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.918 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.903+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa88800fdc0 con 0x7fa89c09f440 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.903+0000 7fa898ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.903+0000 7fa898ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.903+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c09f440 msgr2=0x7fa89c13bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.903+0000 7fa898ff9640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c09f440 0x7fa89c13bdd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa888007400 tx=0x7fa8880078d0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.903+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c09f440 msgr2=0x7fa89c13bdd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.903+0000 7fa898ff9640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c09f440 0x7fa89c13bdd0 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa888007400 tx=0x7fa8880078d0 comp rx=0 tx=0).stop 2026-02-17T06:00:49.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.903+0000 7fa89b7fe640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa86c03deb0 0x7fa86c040370 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa89c13d0d0 tx=0x7fa88c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.905+0000 7fa8a2643640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.905+0000 7fa8a2643640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c1308a0 0x7fa89c1da500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.905+0000 7fa8a2643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.905+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c1d0000 con 0x7fa89c1308a0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c1308a0 0x7fa89c1da500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 10 monclient: get_auth_request con 0x7fa89c1308a0 auth_method 0 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:49.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 10 monclient(hunting): handle_auth_done global_id 144530 payload 824 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.906+0000 7fa89bfff640 1 -- 10.20.193.78:0/618652874 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa89c0744d0 con 0x7fa89c1308a0 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa89bfff640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c1308a0 msgr2=0x7fa89c1da500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa89bfff640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c1308a0 0x7fa89c1da500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa888017000 tx=0x7fa888004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888006240 con 0x7fa89c1308a0 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 10 monclient: dump: 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.922 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fa888006240 con 0x7fa89c1308a0 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c1308a0 msgr2=0x7fa89c1da500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c1308a0 0x7fa89c1da500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa888017000 tx=0x7fa888004a40 comp rx=0 tx=0).stop 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c1308a0 msgr2=0x7fa89c1da500 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:49.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa898ff9640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa89c1308a0 0x7fa89c1da500 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa888017000 tx=0x7fa888004a40 comp rx=0 tx=0).stop 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 174, 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 233443, 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:49.924 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:49.925 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:00:49.926 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:49.927 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.928 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.929 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:49.930 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:49.931 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:49.932 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:00:49.933 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa86c03deb0 msgr2=0x7fa86c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa86c03deb0 0x7fa86c040370 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa89c13d0d0 tx=0x7fa88c008040 comp rx=0 tx=0).stop 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 10 monclient: shutdown 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa89c09c4a0 msgr2=0x7fa89c13b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 1 --2- 10.20.193.78:0/618652874 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa89c09c4a0 0x7fa89c13b890 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa890009d90 tx=0x7fa890004b40 comp rx=0 tx=0).stop 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/618652874 reap_dead start 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a0bb9640 1 -- 10.20.193.78:0/618652874 reap_dead start 2026-02-17T06:00:49.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 shutdown_connections 2026-02-17T06:00:49.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 >> 10.20.193.78:0/618652874 conn(0x7fa89c0a6620 msgr2=0x7fa89c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:49.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 shutdown_connections 2026-02-17T06:00:49.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:49.908+0000 7fa8a2643640 1 -- 10.20.193.78:0/618652874 wait complete. 2026-02-17T06:00:49.935 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:00:49.935 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:00:49.935 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:00:50.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.007+0000 7f6894287640 10 monclient: get_monmap_and_config 2026-02-17T06:00:50.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.007+0000 7f6894287640 10 monclient: build_initial_monmap 2026-02-17T06:00:50.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.007+0000 7f6894287640 10 monclient: monmap: 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:50.009113+0000 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:50.009113+0000 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 1 Processor -- start 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 1 -- start start 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient: init 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f688c06c890 0x7f688c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient(hunting): picked mon.noname-c con 0x7f688c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c09c4a0 0x7f688c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient(hunting): picked mon.noname-b con 0x7f688c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f688c134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f688c06daf0 con 0x7f688c134df0 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f688c0a0880 con 0x7f688c06c890 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f688c06d6f0 con 0x7f688c09c4a0 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6894287640 10 monclient(hunting): authenticate will time out at 1495.562642s 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c09c4a0 0x7f688c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c09c4a0 0x7f688c06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54632/0 (socket says 10.20.193.78:54632) 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 1 -- 10.20.193.78:0/2624233917 learned_addr learned my addr 10.20.193.78:0/2624233917 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient(hunting): get_auth_request con 0x7f688c09c4a0 auth_method 0 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6891ffc640 1 --2- 10.20.193.78:0/2624233917 >> v2:10.20.193.115:3300/0 conn(0x7f688c06c890 0x7f688c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:50.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6891ffc640 10 monclient(hunting): get_auth_request con 0x7f688c06c890 auth_method 0 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6891ffc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6891ffc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f6891ffc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient(hunting): handle_auth_done global_id 144533 payload 274 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 1 -- 10.20.193.78:0/2624233917 >> v2:10.20.193.115:3300/0 conn(0x7f688c06c890 msgr2=0x7f688c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 1 --2- 10.20.193.78:0/2624233917 >> v2:10.20.193.115:3300/0 conn(0x7f688c06c890 0x7f688c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 1 -- 10.20.193.78:0/2624233917 >> v1:10.20.193.115:6789/0 conn(0x7f688c134df0 legacy=0x7f688c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient: _finish_hunting 0 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 1 monclient: found mon.noname-b 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:50.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.008+0000 7f68917fb640 1 -- 10.20.193.78:0/2624233917 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c09cc80 con 0x7f688c09c4a0 2026-02-17T06:00:50.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f68917fb640 1 -- 10.20.193.78:0/2624233917 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c09c4a0 msgr2=0x7f688c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:50.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f68917fb640 1 --2- 10.20.193.78:0/2624233917 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c09c4a0 0x7f688c06d1b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f688c06d870 tx=0x7f688002ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9f167a4e802ee server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 1 -- 10.20.193.78:0/2624233917 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688003c070 con 0x7f688c09c4a0 2026-02-17T06:00:50.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:50.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: dump: 2026-02-17T06:00:50.021 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: _finish_auth 0 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: _check_auth_tickets 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 1 -- 10.20.193.78:0/2624233917 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688002fcc0 con 0x7f688c09c4a0 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 5 monclient: authenticate success, global_id 144533 2026-02-17T06:00:50.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 1 -- 10.20.193.78:0/2624233917 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880037660 con 0x7f688c09c4a0 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 4 set_mon_vals no callback set 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6890ffa640 10 monclient: dump: 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.023 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 10 monclient: shutdown 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 1 -- 10.20.193.78:0/2624233917 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c09c4a0 msgr2=0x7f688c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 1 --2- 10.20.193.78:0/2624233917 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c09c4a0 0x7f688c06d1b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f688c06d870 tx=0x7f688002ee80 comp rx=0 tx=0).stop 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 1 -- 10.20.193.78:0/2624233917 shutdown_connections 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 1 --2- 10.20.193.78:0/2624233917 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c09c4a0 0x7f688c06d1b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 1 --2- 10.20.193.78:0/2624233917 >> v2:10.20.193.115:3300/0 conn(0x7f688c06c890 0x7f688c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.011+0000 7f6894287640 1 -- 10.20.193.78:0/2624233917 >> 10.20.193.78:0/2624233917 conn(0x7f688c0a6620 msgr2=0x7f688c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 -- 10.20.193.78:0/2624233917 shutdown_connections 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 -- 10.20.193.78:0/2624233917 wait complete. 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient: build_initial_monmap 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient: monmap: 2026-02-17T06:00:50.024 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 Processor -- start 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 -- start start 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient: init 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c06c890 0x7f688c1cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient(hunting): picked mon.noname-c con 0x7f688c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c09c4a0 0x7f688c1333c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient(hunting): picked mon.noname-b con 0x7f688c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f688c134df0 0x7f688c133900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient(hunting): picked mon.noname-a con 0x7f688c134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f688c0a01a0 con 0x7f688c134df0 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f688c0a0020 con 0x7f688c09c4a0 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f688c09be20 con 0x7f688c06c890 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.012+0000 7f6894287640 10 monclient(hunting): authenticate will time out at 1495.566967s 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c06c890 0x7f688c1cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c06c890 0x7f688c1cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54636/0 (socket says 10.20.193.78:54636) 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 -- 10.20.193.78:0/601797785 learned_addr learned my addr 10.20.193.78:0/601797785 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient(hunting): get_auth_request con 0x7f688c06c890 auth_method 0 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68927fd640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f688c134df0 0x7f688c133900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68927fd640 10 monclient(hunting): get_auth_request con 0x7f688c134df0 auth_method 0 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68927fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68927fd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68927fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68917fb640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c09c4a0 0x7f688c1333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68927fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:50.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68927fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68927fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68917fb640 10 monclient(hunting): get_auth_request con 0x7f688c09c4a0 auth_method 0 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68917fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68917fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68917fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient(hunting): handle_auth_done global_id 144536 payload 824 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c09c4a0 msgr2=0x7f688c1333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c09c4a0 0x7f688c1333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f688c134df0 msgr2=0x7f688c133900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f688c134df0 0x7f688c133900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient: _finish_hunting 0 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 monclient: found mon.noname-c 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f6891ffc640 1 -- 10.20.193.78:0/601797785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c0a17b0 con 0x7f688c06c890 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.013+0000 7f68917fb640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c09c4a0 0x7f688c1333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T06:00:50.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6891ffc640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c06c890 msgr2=0x7f688c1cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6891ffc640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c06c890 0x7f688c1cf1c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f687c00e810 tx=0x7f687c00ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c017070 con 0x7f688c06c890 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: dump: 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:00:50.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c00cae0 con 0x7f688c06c890 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00f040 con 0x7f688c06c890 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687a7fc640 10 monclient: dump: 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f687bfff640 4 set_mon_vals no callback set 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6894287640 5 monclient: authenticate success, global_id 144536 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6894287640 10 monclient: _renew_subs 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6894287640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c126b00 con 0x7f688c06c890 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6894287640 10 monclient: _renew_subs 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6894287640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c124af0 con 0x7f688c06c890 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.015+0000 7f6894287640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f6894287640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c122ea0 0x7f688c123280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f6894287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f688c074980 con 0x7f688c122ea0 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f687c012050 con 0x7f688c06c890 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c122ea0 0x7f688c123280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 10 monclient: get_auth_request con 0x7f688c122ea0 auth_method 0 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f687a7fc640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f685c03dee0 0x7f685c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(527..527 src has 335..527) ==== 4827+0+0 (secure 0 0 0) 0x7f687c0525b0 con 0x7f688c06c890 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68927fd640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f685c03dee0 0x7f685c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68927fd640 10 monclient: get_auth_request con 0x7f685c03dee0 auth_method 0 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 10 monclient(hunting): handle_auth_done global_id 134482 payload 824 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.016+0000 7f68917fb640 1 -- 10.20.193.78:0/601797785 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f688c135a70 con 0x7f688c122ea0 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f68917fb640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c122ea0 msgr2=0x7f688c123280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f68917fb640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c122ea0 0x7f688c123280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6880036000 tx=0x7f68800047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f68927fd640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f685c03dee0 0x7f685c0403a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f688c126fa0 tx=0x7f6888004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688003c040 con 0x7f688c122ea0 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 10 monclient: dump: 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.032 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6880037700 con 0x7f688c122ea0 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c122ea0 msgr2=0x7f688c123280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c122ea0 0x7f688c123280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6880036000 tx=0x7f68800047c0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c122ea0 msgr2=0x7f688c123280 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:50.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.019+0000 7f687a7fc640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c122ea0 0x7f688c123280 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6880036000 tx=0x7f68800047c0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f6894287640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f6894287640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c1237c0 0x7f688c1da510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f6894287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f688c125870 con 0x7f688c1237c0 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f68917fb640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c1237c0 0x7f688c1da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f68917fb640 10 monclient: get_auth_request con 0x7f688c1237c0 auth_method 0 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f68917fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f68917fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f68917fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.022+0000 7f68917fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:50.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.023+0000 7f68917fb640 10 monclient(hunting): handle_auth_done global_id 134485 payload 824 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.023+0000 7f68917fb640 1 -- 10.20.193.78:0/601797785 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f688c0744d0 con 0x7f688c1237c0 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f68917fb640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c1237c0 msgr2=0x7f688c1da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f68917fb640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c1237c0 0x7f688c1da510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f688c124510 tx=0x7f6880008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688003c040 con 0x7f688c1237c0 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 10 monclient: dump: 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f6880043860 con 0x7f688c1237c0 2026-02-17T06:00:50.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c1237c0 msgr2=0x7f688c1da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c1237c0 0x7f688c1da510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f688c124510 tx=0x7f6880008030 comp rx=0 tx=0).stop 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c1237c0 msgr2=0x7f688c1da510 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.025+0000 7f687a7fc640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f688c1237c0 0x7f688c1da510 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f688c124510 tx=0x7f6880008030 comp rx=0 tx=0).stop 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 174, 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 277744, 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:00:50.037 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:00:50.038 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:50.039 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:00:50.040 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:00:50.041 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:00:50.042 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:00:50.043 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:50.044 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:00:50.045 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f685c03dee0 msgr2=0x7f685c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f685c03dee0 0x7f685c0403a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f688c126fa0 tx=0x7f6888004200 comp rx=0 tx=0).stop 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 10 monclient: shutdown 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c06c890 msgr2=0x7f688c1cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 1 --2- 10.20.193.78:0/601797785 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f688c06c890 0x7f688c1cf1c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f687c00e810 tx=0x7f687c00ece0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f68927fd640 1 -- 10.20.193.78:0/601797785 reap_dead start 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f68927fd640 1 -- 10.20.193.78:0/601797785 reap_dead start 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 shutdown_connections 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 >> 10.20.193.78:0/601797785 conn(0x7f688c0a6620 msgr2=0x7f688c135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 shutdown_connections 2026-02-17T06:00:50.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.026+0000 7f6894287640 1 -- 10.20.193.78:0/601797785 wait complete. 2026-02-17T06:00:50.047 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:00:50.047 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T06:00:50.047 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:00:50.047 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T06:00:50.047 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:00:50.047 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T06:00:50.047 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:00:50.058 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T06:00:50.058 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:00:50.058 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:50.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.124+0000 7f6c2f3a0640 10 monclient: get_monmap_and_config 2026-02-17T06:00:50.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.124+0000 7f6c2f3a0640 10 monclient: build_initial_monmap 2026-02-17T06:00:50.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.124+0000 7f6c2f3a0640 10 monclient: monmap: 2026-02-17T06:00:50.125 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:50.126025+0000 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:50.126025+0000 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:50.126 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 1 Processor -- start 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 1 -- start start 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient: init 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6c2806d140 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6c2809f4a0 0x7f6c2809c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient(hunting): picked mon.noname-a con 0x7f6c2809f4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6c28139e20 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2809d340 con 0x7f6c28139e20 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2809e750 con 0x7f6c2806d140 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2809f060 con 0x7f6c2809f4a0 2026-02-17T06:00:50.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2f3a0640 10 monclient(hunting): authenticate will time out at 1495.679639s 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6c2809f4a0 0x7f6c2809c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2d115640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f6c2806d140 legacy=0x7f6c2809be20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6c2809f4a0 0x7f6c2809c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49860/0 (socket says 10.20.193.78:49860) 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 1 -- 10.20.193.78:0/2643766671 learned_addr learned my addr 10.20.193.78:0/2643766671 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 10 monclient(hunting): get_auth_request con 0x7f6c2809f4a0 auth_method 0 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:50.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.125+0000 7f6c2c914640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2d916640 1 -- 10.20.193.78:0/2643766671 >> v1:10.20.193.78:6789/0 conn(0x7f6c28139e20 legacy=0x7f6c2813c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2c914640 10 monclient(hunting): handle_auth_done global_id 184149 payload 274 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2c914640 1 -- 10.20.193.78:0/2643766671 >> v1:10.20.193.115:6789/0 conn(0x7f6c2806d140 legacy=0x7f6c2809be20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2c914640 1 -- 10.20.193.78:0/2643766671 >> v1:10.20.193.78:6789/0 conn(0x7f6c28139e20 legacy=0x7f6c2813c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2c914640 10 monclient: _finish_hunting 0 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2c914640 1 monclient: found mon.noname-a 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2c914640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2c914640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:00:50.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.127+0000 7f6c2c914640 1 -- 10.20.193.78:0/2643766671 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2809ea30 con 0x7f6c2809f4a0 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c2c914640 1 -- 10.20.193.78:0/2643766671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 msgr2=0x7f6c2809c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c2c914640 1 --2- 10.20.193.78:0/2643766671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c2809c530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00bca0 tx=0x7f6c1c031070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e7a73e378c6c88 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 1 -- 10.20.193.78:0/2643766671 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c03c680 con 0x7f6c2809f4a0 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: dump: 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.132 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: _finish_auth 0 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: _check_auth_tickets 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 1 -- 10.20.193.78:0/2643766671 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c03c820 con 0x7f6c2809f4a0 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 1 -- 10.20.193.78:0/2643766671 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c03cbf0 con 0x7f6c2809f4a0 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c2f3a0640 5 monclient: authenticate success, global_id 184149 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c17fff640 10 monclient: dump: 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.133 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c2f3a0640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.130+0000 7f6c2f3a0640 4 set_mon_vals no callback set 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 10 monclient: shutdown 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/2643766671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 msgr2=0x7f6c2809c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 --2- 10.20.193.78:0/2643766671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c2809c530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00bca0 tx=0x7f6c1c031070 comp rx=0 tx=0).stop 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/2643766671 shutdown_connections 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 --2- 10.20.193.78:0/2643766671 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c2809c530 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/2643766671 >> 10.20.193.78:0/2643766671 conn(0x7f6c280a6620 msgr2=0x7f6c280a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:50.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/2643766671 shutdown_connections 2026-02-17T06:00:50.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/2643766671 wait complete. 2026-02-17T06:00:50.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 10 monclient: build_initial_monmap 2026-02-17T06:00:50.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 10 monclient: monmap: 2026-02-17T06:00:50.138 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:50.138 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:50.138 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:50.138 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:50.138 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 Processor -- start 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 1 -- start start 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.131+0000 7f6c2f3a0640 10 monclient: init 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2806d140 0x7f6c281387c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient(hunting): picked mon.noname-c con 0x7f6c2806d140 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c28138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient(hunting): picked mon.noname-a con 0x7f6c2809f4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c28139e20 0x7f6c281216c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient(hunting): picked mon.noname-b con 0x7f6c28139e20 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280a0ae0 con 0x7f6c2809f4a0 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280a0960 con 0x7f6c28139e20 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2809ca70 con 0x7f6c2806d140 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2f3a0640 10 monclient(hunting): authenticate will time out at 1495.686157s 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c28138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d115640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2806d140 0x7f6c281387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c28138d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49868/0 (socket says 10.20.193.78:49868) 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 -- 10.20.193.78:0/737370513 learned_addr learned my addr 10.20.193.78:0/737370513 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient(hunting): get_auth_request con 0x7f6c2809f4a0 auth_method 0 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d115640 10 monclient(hunting): get_auth_request con 0x7f6c2806d140 auth_method 0 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d115640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d115640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d115640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d916640 1 -- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c28139e20 msgr2=0x7f6c281216c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d916640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c28139e20 0x7f6c281216c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d115640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d115640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2d115640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient(hunting): handle_auth_done global_id 184155 payload 824 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 -- 10.20.193.78:0/737370513 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2806d140 msgr2=0x7f6c281387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2806d140 0x7f6c281387c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 -- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c28139e20 msgr2=0x7f6c281216c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c28139e20 0x7f6c281216c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient: _finish_hunting 0 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 monclient: found mon.noname-a 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.132+0000 7f6c2c914640 1 -- 10.20.193.78:0/737370513 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c28123e70 con 0x7f6c2809f4a0 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2c914640 1 -- 10.20.193.78:0/737370513 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 msgr2=0x7f6c28138d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2c914640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c28138d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00bdd0 tx=0x7f6c1c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 1 -- 10.20.193.78:0/737370513 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c04c070 con 0x7f6c2809f4a0 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: dump: 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.142 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: _finish_auth 0 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 1 -- 10.20.193.78:0/737370513 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c00b2d0 con 0x7f6c2809f4a0 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 1 -- 10.20.193.78:0/737370513 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c03cbf0 con 0x7f6c2809f4a0 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c177fe640 4 set_mon_vals no callback set 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c15ffb640 10 monclient: dump: 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:50.143 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2f3a0640 5 monclient: authenticate success, global_id 184155 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2f3a0640 10 monclient: _renew_subs 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2f3a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c28125040 con 0x7f6c2809f4a0 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2f3a0640 10 monclient: _renew_subs 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2f3a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.134+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c28072f70 con 0x7f6c2809f4a0 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c2f3a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c2f3a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:50.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c2f3a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c280a2920 con 0x7f6c2809f4a0 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c15ffb640 1 -- 10.20.193.78:0/737370513 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6c1c04f070 con 0x7f6c2809f4a0 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c15ffb640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf803de60 0x7f6bf8040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c15ffb640 1 -- 10.20.193.78:0/737370513 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(527..527 src has 335..527) ==== 4827+0+0 (secure 0 0 0) 0x7f6c1c079120 con 0x7f6c2809f4a0 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c2d115640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf803de60 0x7f6bf8040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.135+0000 7f6c2d115640 10 monclient: get_auth_request con 0x7f6bf803de60 auth_method 0 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.137+0000 7f6c15ffb640 1 -- 10.20.193.78:0/737370513 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6c1c039d60 con 0x7f6c2809f4a0 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.137+0000 7f6c15ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:50.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.137+0000 7f6c15ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:50.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.138+0000 7f6c2d115640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf803de60 0x7f6bf8040320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c28122f10 tx=0x7f6c18003c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:50.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.251+0000 7f6c2f3a0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:50.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.251+0000 7f6c2f3a0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:50.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.251+0000 7f6c2f3a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:00:50.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.251+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6c2806c890 con 0x7f6c2809f4a0 2026-02-17T06:00:50.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.251+0000 7f6c15ffb640 1 -- 10.20.193.78:0/737370513 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f6c1c07b120 con 0x7f6c2809f4a0 2026-02-17T06:00:50.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c15ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:50.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c15ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:50.254 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":174,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf803de60 msgr2=0x7f6bf8040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf803de60 0x7f6bf8040320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c28122f10 tx=0x7f6c18003c90 comp rx=0 tx=0).stop 2026-02-17T06:00:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 10 monclient: shutdown 2026-02-17T06:00:50.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 msgr2=0x7f6c28138d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c28138d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00bdd0 tx=0x7f6c1c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 shutdown_connections 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf803de60 0x7f6bf8040320 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c28139e20 0x7f6c281216c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c2809f4a0 0x7f6c28138d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 --2- 10.20.193.78:0/737370513 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2806d140 0x7f6c281387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 >> 10.20.193.78:0/737370513 conn(0x7f6c280a6620 msgr2=0x7f6c280a1920 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 shutdown_connections 2026-02-17T06:00:50.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:50.252+0000 7f6c2f3a0640 1 -- 10.20.193.78:0/737370513 wait complete. 2026-02-17T06:00:53.263 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.331+0000 7f06f6099640 10 monclient: get_monmap_and_config 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.331+0000 7f06f6099640 10 monclient: build_initial_monmap 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.331+0000 7f06f6099640 10 monclient: monmap: 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:53.332607+0000 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:53.332607+0000 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:53.332 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.331+0000 7f06f6099640 1 Processor -- start 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.331+0000 7f06f6099640 1 -- start start 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.331+0000 7f06f6099640 10 monclient: init 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f006c890 0x7f06f006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient(hunting): picked mon.noname-b con 0x7f06f006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f06f009c4a0 0x7f06f006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient(hunting): picked mon.noname-c con 0x7f06f009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f06f0134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f006daf0 con 0x7f06f0134df0 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f06f00a0880 con 0x7f06f009c4a0 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f06f006d6f0 con 0x7f06f006c890 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06f6099640 10 monclient(hunting): authenticate will time out at 1498.886180s 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f006c890 0x7f06f006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06eeffd640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f06f009c4a0 msgr2=0x7f06f006d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06eeffd640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f06f009c4a0 0x7f06f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f006c890 0x7f06f006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54664/0 (socket says 10.20.193.78:54664) 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 1 -- 10.20.193.78:0/2259082060 learned_addr learned my addr 10.20.193.78:0/2259082060 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient(hunting): get_auth_request con 0x7f06f006c890 auth_method 0 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06effff640 1 --1- 10.20.193.78:0/2259082060 >> v1:10.20.193.78:6789/0 conn(0x7f06f0134df0 0x7f06f0135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:59370/0 (socket says 10.20.193.78:59370) 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient(hunting): handle_auth_done global_id 144542 payload 274 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 1 -- 10.20.193.78:0/2259082060 >> v2:10.20.193.115:3300/0 conn(0x7f06f009c4a0 msgr2=0x7f06f006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 1 --2- 10.20.193.78:0/2259082060 >> v2:10.20.193.115:3300/0 conn(0x7f06f009c4a0 0x7f06f006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 1 -- 10.20.193.78:0/2259082060 >> v1:10.20.193.78:6789/0 conn(0x7f06f0134df0 legacy=0x7f06f0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 1 monclient: found mon.noname-b 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:53.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.332+0000 7f06ef7fe640 1 -- 10.20.193.78:0/2259082060 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f009cc80 con 0x7f06f006c890 2026-02-17T06:00:53.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.334+0000 7f06ef7fe640 1 -- 10.20.193.78:0/2259082060 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f006c890 msgr2=0x7f06f006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:53.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.334+0000 7f06ef7fe640 1 --2- 10.20.193.78:0/2259082060 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f006c890 0x7f06f006cc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009970 tx=0x7f06dc02eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e1e3050c7990a7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 1 -- 10.20.193.78:0/2259082060 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc03c070 con 0x7f06f006c890 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: dump: 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: _finish_auth 0 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 5 monclient: authenticate success, global_id 144542 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 1 -- 10.20.193.78:0/2259082060 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc02fdd0 con 0x7f06f006c890 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 1 -- 10.20.193.78:0/2259082060 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc037660 con 0x7f06f006c890 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 4 set_mon_vals no callback set 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06ee7fc640 10 monclient: dump: 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:53.336 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 10 monclient: shutdown 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 1 -- 10.20.193.78:0/2259082060 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f006c890 msgr2=0x7f06f006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 1 --2- 10.20.193.78:0/2259082060 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f006c890 0x7f06f006cc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009970 tx=0x7f06dc02eeb0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 1 -- 10.20.193.78:0/2259082060 shutdown_connections 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 1 --2- 10.20.193.78:0/2259082060 >> v2:10.20.193.115:3300/0 conn(0x7f06f009c4a0 0x7f06f006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 1 --2- 10.20.193.78:0/2259082060 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f006c890 0x7f06f006cc70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 1 -- 10.20.193.78:0/2259082060 >> 10.20.193.78:0/2259082060 conn(0x7f06f00a6620 msgr2=0x7f06f00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 1 -- 10.20.193.78:0/2259082060 shutdown_connections 2026-02-17T06:00:53.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 1 -- 10.20.193.78:0/2259082060 wait complete. 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 10 monclient: build_initial_monmap 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.335+0000 7f06f6099640 10 monclient: monmap: 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 1 Processor -- start 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 1 -- start start 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient: init 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f06f006c890 0x7f06f01cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient(hunting): picked mon.noname-a con 0x7f06f006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f06f009c4a0 0x7f06f01cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient(hunting): picked mon.noname-b con 0x7f06f009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f0134df0 0x7f06f01215e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient(hunting): picked mon.noname-c con 0x7f06f0134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06f00a01a0 con 0x7f06f006c890 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f06f00a0020 con 0x7f06f009c4a0 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f06f009be20 con 0x7f06f0134df0 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06f6099640 10 monclient(hunting): authenticate will time out at 1498.890408s 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f06f006c890 0x7f06f01cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f0134df0 0x7f06f01215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06eeffd640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f06f009c4a0 msgr2=0x7f06f01cd870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f06f006c890 0x7f06f01cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49874/0 (socket says 10.20.193.78:49874) 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f0134df0 0x7f06f01215e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54676/0 (socket says 10.20.193.78:54676) 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 1 -- 10.20.193.78:0/548179912 learned_addr learned my addr 10.20.193.78:0/548179912 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 10 monclient(hunting): get_auth_request con 0x7f06f006c890 auth_method 0 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:53.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06eeffd640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f06f009c4a0 0x7f06f01cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient(hunting): get_auth_request con 0x7f06f0134df0 auth_method 0 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient(hunting): handle_auth_done global_id 144545 payload 824 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 1 -- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f06f009c4a0 msgr2=0x7f06f01cd870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f06f009c4a0 0x7f06f01cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 1 -- 10.20.193.78:0/548179912 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f06f006c890 msgr2=0x7f06f01cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f06f006c890 0x7f06f01cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient: _finish_hunting 0 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 1 monclient: found mon.noname-c 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.336+0000 7f06effff640 1 -- 10.20.193.78:0/548179912 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f0123de0 con 0x7f06f0134df0 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06effff640 1 -- 10.20.193.78:0/548179912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f0134df0 msgr2=0x7f06f01215e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06effff640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f0134df0 0x7f06f01215e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06e400caf0 tx=0x7f06e4005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:53.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 1 -- 10.20.193.78:0/548179912 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4017070 con 0x7f06f0134df0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: dump: 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: _finish_auth 0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: _check_auth_tickets 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 1 -- 10.20.193.78:0/548179912 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e4012400 con 0x7f06f0134df0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06f6099640 5 monclient: authenticate success, global_id 144545 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06f6099640 10 monclient: _renew_subs 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06f6099640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f0125570 con 0x7f06f0134df0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 1 -- 10.20.193.78:0/548179912 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40127f0 con 0x7f06f0134df0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06edffb640 4 set_mon_vals no callback set 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06cbfff640 10 monclient: dump: 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06f6099640 10 monclient: _renew_subs 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06f6099640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:53.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.338+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f009d810 con 0x7f06f0134df0 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.339+0000 7f06cbfff640 1 -- 10.20.193.78:0/548179912 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f06e4007070 con 0x7f06f0134df0 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.339+0000 7f06f6099640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.339+0000 7f06f6099640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.339+0000 7f06f6099640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.339+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f01359e0 con 0x7f06f0134df0 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.339+0000 7f06cbfff640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f06c003de60 0x7f06c0040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.339+0000 7f06cbfff640 1 -- 10.20.193.78:0/548179912 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(527..527 src has 335..527) ==== 4827+0+0 (secure 0 0 0) 0x7f06e4052850 con 0x7f06f0134df0 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.340+0000 7f06ef7fe640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f06c003de60 0x7f06c0040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:53.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.340+0000 7f06ef7fe640 10 monclient: get_auth_request con 0x7f06c003de60 auth_method 0 2026-02-17T06:00:53.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.342+0000 7f06cbfff640 1 -- 10.20.193.78:0/548179912 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f06e4051220 con 0x7f06f0134df0 2026-02-17T06:00:53.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.342+0000 7f06cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:53.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.342+0000 7f06cbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:53.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.342+0000 7f06ef7fe640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f06c003de60 0x7f06c0040320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f06dc02eef0 tx=0x7f06dc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:53.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.458+0000 7f06f6099640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:53.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.458+0000 7f06f6099640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:53.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.458+0000 7f06f6099640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:53.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.458+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f06f00744d0 con 0x7f06f0134df0 2026-02-17T06:00:53.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.458+0000 7f06cbfff640 1 -- 10.20.193.78:0/548179912 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f06e4007810 con 0x7f06f0134df0 2026-02-17T06:00:53.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.458+0000 7f06cbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:53.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.458+0000 7f06cbfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:53.459 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":174,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f06c003de60 msgr2=0x7f06c0040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f06c003de60 0x7f06c0040320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f06dc02eef0 tx=0x7f06dc033000 comp rx=0 tx=0).stop 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 10 monclient: shutdown 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f0134df0 msgr2=0x7f06f01215e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f0134df0 0x7f06f01215e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06e400caf0 tx=0x7f06e4005bb0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 shutdown_connections 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f06f0134df0 0x7f06f01215e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f06c003de60 0x7f06c0040320 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f06f009c4a0 0x7f06f01cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 --2- 10.20.193.78:0/548179912 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f06f006c890 0x7f06f01cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 >> 10.20.193.78:0/548179912 conn(0x7f06f00a6620 msgr2=0x7f06f00a1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 shutdown_connections 2026-02-17T06:00:53.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:53.459+0000 7f06f6099640 1 -- 10.20.193.78:0/548179912 wait complete. 2026-02-17T06:00:56.469 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.534+0000 7f925331b640 10 monclient: get_monmap_and_config 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.534+0000 7f925331b640 10 monclient: build_initial_monmap 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.534+0000 7f925331b640 10 monclient: monmap: 2026-02-17T06:00:56.538 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:56.536124+0000 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:56.536124+0000 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:56.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 1 Processor -- start 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 1 -- start start 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient: init 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f924c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c09c4a0 0x7f924c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient(hunting): picked mon.noname-b con 0x7f924c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f924c06d380 0x7f924c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient(hunting): picked mon.noname-c con 0x7f924c06d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924c06daf0 con 0x7f924c06c890 2026-02-17T06:00:56.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f924c0a0880 con 0x7f924c06d380 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f924c09cc80 con 0x7f924c09c4a0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925331b640 10 monclient(hunting): authenticate will time out at 1502.089675s 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f9251090640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f924c06c890 0x7f924c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:59384/0 (socket says 10.20.193.78:59384) 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f9251090640 1 -- 10.20.193.78:0/548551663 learned_addr learned my addr 10.20.193.78:0/548551663 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 1 --2- 10.20.193.78:0/548551663 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c09c4a0 0x7f924c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f9251891640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f924c06d380 msgr2=0x7f924c13d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient(hunting): get_auth_request con 0x7f924c09c4a0 auth_method 0 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:56.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f9251891640 1 --2- 10.20.193.78:0/548551663 >> v2:10.20.193.115:3300/0 conn(0x7f924c06d380 0x7f924c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient(hunting): handle_auth_done global_id 144548 payload 274 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 1 -- 10.20.193.78:0/548551663 >> v2:10.20.193.115:3300/0 conn(0x7f924c06d380 msgr2=0x7f924c13d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 1 --2- 10.20.193.78:0/548551663 >> v2:10.20.193.115:3300/0 conn(0x7f924c06d380 0x7f924c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 1 -- 10.20.193.78:0/548551663 >> v1:10.20.193.78:6789/0 conn(0x7f924c06c890 legacy=0x7f924c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient: _finish_hunting 0 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 1 monclient: found mon.noname-b 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:56.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:00:56.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.535+0000 7f925088f640 1 -- 10.20.193.78:0/548551663 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c09cf60 con 0x7f924c09c4a0 2026-02-17T06:00:56.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925088f640 1 -- 10.20.193.78:0/548551663 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c09c4a0 msgr2=0x7f924c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:56.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925088f640 1 --2- 10.20.193.78:0/548551663 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c09c4a0 0x7f924c13b3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92400098a0 tx=0x7f924002eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3609a92290584633 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:56.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 1 -- 10.20.193.78:0/548551663 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924003c070 con 0x7f924c09c4a0 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: dump: 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:56.550 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: _finish_auth 0 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 1 -- 10.20.193.78:0/548551663 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924002fd00 con 0x7f924c09c4a0 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 5 monclient: authenticate success, global_id 144548 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 1 -- 10.20.193.78:0/548551663 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240037660 con 0x7f924c09c4a0 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 10 monclient: get_monmap_and_config success 2026-02-17T06:00:56.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 4 set_mon_vals no callback set 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f923bfff640 10 monclient: dump: 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:56.552 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 10 monclient: shutdown 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 1 -- 10.20.193.78:0/548551663 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c09c4a0 msgr2=0x7f924c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 1 --2- 10.20.193.78:0/548551663 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c09c4a0 0x7f924c13b3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92400098a0 tx=0x7f924002eda0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 1 -- 10.20.193.78:0/548551663 shutdown_connections 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 1 --2- 10.20.193.78:0/548551663 >> v2:10.20.193.115:3300/0 conn(0x7f924c06d380 0x7f924c13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 1 --2- 10.20.193.78:0/548551663 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c09c4a0 0x7f924c13b3a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 1 -- 10.20.193.78:0/548551663 >> 10.20.193.78:0/548551663 conn(0x7f924c0a6620 msgr2=0x7f924c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 1 -- 10.20.193.78:0/548551663 shutdown_connections 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.538+0000 7f925331b640 1 -- 10.20.193.78:0/548551663 wait complete. 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient: build_initial_monmap 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient: monmap: 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:00:56.554 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 1 Processor -- start 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 1 -- start start 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient: init 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c06c890 0x7f924c1cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient(hunting): picked mon.noname-c con 0x7f924c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f924c06d380 0x7f924c1364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient(hunting): picked mon.noname-b con 0x7f924c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:00:56.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f924c09c4a0 0x7f924c1cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient(hunting): picked mon.noname-a con 0x7f924c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f924c09bfa0 con 0x7f924c09c4a0 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f924c09be20 con 0x7f924c06d380 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f924c13de30 con 0x7f924c06c890 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925331b640 10 monclient(hunting): authenticate will time out at 1502.093788s 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c06c890 0x7f924c1cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251891640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f924c09c4a0 0x7f924c1cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c06c890 0x7f924c1cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54686/0 (socket says 10.20.193.78:54686) 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 1 -- 10.20.193.78:0/512963193 learned_addr learned my addr 10.20.193.78:0/512963193 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925088f640 1 -- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f924c06d380 msgr2=0x7f924c1364c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient(hunting): get_auth_request con 0x7f924c06c890 auth_method 0 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f925088f640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f924c06d380 0x7f924c1364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251891640 10 monclient(hunting): get_auth_request con 0x7f924c09c4a0 auth_method 0 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251891640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251891640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251891640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:56.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251891640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251891640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251891640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient(hunting): handle_auth_done global_id 144551 payload 824 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 1 -- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f924c06d380 msgr2=0x7f924c1364c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f924c06d380 0x7f924c1364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 1 -- 10.20.193.78:0/512963193 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f924c09c4a0 msgr2=0x7f924c1cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f924c09c4a0 0x7f924c1cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 10 monclient: _finish_hunting 0 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.539+0000 7f9251090640 1 monclient: found mon.noname-c 2026-02-17T06:00:56.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.540+0000 7f9251090640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.540+0000 7f9251090640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.540+0000 7f9251090640 1 -- 10.20.193.78:0/512963193 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c1203e0 con 0x7f924c06c890 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9251090640 1 -- 10.20.193.78:0/512963193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c06c890 msgr2=0x7f924c1cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9251090640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c06c890 0x7f924c1cbd50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f923c00c780 tx=0x7f923c00cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 1 -- 10.20.193.78:0/512963193 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c005de0 con 0x7f924c06c890 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: dump: 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:56.559 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: _finish_auth 0 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 5 monclient: authenticate success, global_id 144551 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 1 -- 10.20.193.78:0/512963193 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923c007070 con 0x7f924c06c890 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 10 monclient: _renew_subs 2026-02-17T06:00:56.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c13d9d0 con 0x7f924c06c890 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 1 -- 10.20.193.78:0/512963193 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c007440 con 0x7f924c06c890 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f923b7fe640 4 set_mon_vals no callback set 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f9239ffb640 10 monclient: dump: 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:00:56.561 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 10 monclient: _renew_subs 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c122e70 con 0x7f924c06c890 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.542+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924c073b50 con 0x7f924c06c890 2026-02-17T06:00:56.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.543+0000 7f9239ffb640 1 -- 10.20.193.78:0/512963193 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f923c0075e0 con 0x7f924c06c890 2026-02-17T06:00:56.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.543+0000 7f9239ffb640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f921c03deb0 0x7f921c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:56.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.543+0000 7f9239ffb640 1 -- 10.20.193.78:0/512963193 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(527..527 src has 335..527) ==== 4827+0+0 (secure 0 0 0) 0x7f923c052540 con 0x7f924c06c890 2026-02-17T06:00:56.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.543+0000 7f925088f640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f921c03deb0 0x7f921c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:56.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.543+0000 7f925088f640 10 monclient: get_auth_request con 0x7f921c03deb0 auth_method 0 2026-02-17T06:00:56.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.545+0000 7f9239ffb640 1 -- 10.20.193.78:0/512963193 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f923c00a8d0 con 0x7f924c06c890 2026-02-17T06:00:56.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.545+0000 7f9239ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:00:56.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.545+0000 7f9239ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:00:56.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.545+0000 7f925088f640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f921c03deb0 0x7f921c040370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f924c06a480 tx=0x7f9240033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:00:56.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.657+0000 7f925331b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:00:56.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.657+0000 7f925331b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:56.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.657+0000 7f925331b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:00:56.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.657+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f924c06cc70 con 0x7f924c06c890 2026-02-17T06:00:56.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f9239ffb640 1 -- 10.20.193.78:0/512963193 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f923c00a5e0 con 0x7f924c06c890 2026-02-17T06:00:56.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f9239ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:00:56.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f9239ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:00:56.660 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":174,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:00:56.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f921c03deb0 msgr2=0x7f921c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f925331b640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f921c03deb0 0x7f921c040370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f924c06a480 tx=0x7f9240033000 comp rx=0 tx=0).stop 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f925331b640 10 monclient: shutdown 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f925331b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c06c890 msgr2=0x7f924c1cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.658+0000 7f925331b640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c06c890 0x7f924c1cbd50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f923c00c780 tx=0x7f923c00cc50 comp rx=0 tx=0).stop 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.659+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 shutdown_connections 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.659+0000 7f925331b640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f924c09c4a0 0x7f924c1cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.659+0000 7f925331b640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f921c03deb0 0x7f921c040370 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.659+0000 7f925331b640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f924c06d380 0x7f924c1364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.659+0000 7f925331b640 1 --2- 10.20.193.78:0/512963193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f924c06c890 0x7f924c1cbd50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.659+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 >> 10.20.193.78:0/512963193 conn(0x7f924c0a6620 msgr2=0x7f924c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:00:56.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.659+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 shutdown_connections 2026-02-17T06:00:56.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:56.659+0000 7f925331b640 1 -- 10.20.193.78:0/512963193 wait complete. 2026-02-17T06:00:59.667 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:00:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.736+0000 7fb71564c640 10 monclient: get_monmap_and_config 2026-02-17T06:00:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.736+0000 7fb71564c640 10 monclient: build_initial_monmap 2026-02-17T06:00:59.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.736+0000 7fb71564c640 10 monclient: monmap: 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:00:59.738185+0000 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:00:59.738185+0000 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:00:59.741 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 1 Processor -- start 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 1 -- start start 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient: init 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient: _reopen_session rank -1 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 0x7fb710070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient(hunting): picked mon.noname-c con 0x7fb7101384d0 addr v2:10.20.193.115:3300/0 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient(hunting): picked mon.noname-a con 0x7fb7101388b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:00:59.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb7100a3220 addr v1:10.20.193.115:6789/0 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb710073f70 con 0x7fb7100a3220 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb7100a4b20 con 0x7fb7101388b0 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb7100a4f70 con 0x7fb7101384d0 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient(hunting): _renew_subs 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb71564c640 10 monclient(hunting): authenticate will time out at 1505.291881s 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70effd640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 msgr2=0x7fb710070530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70e7fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70effd640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 0x7fb710070530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:59.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70f7fe640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fb7100a3220 legacy=0x7fb7100a3630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70e7fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58344/0 (socket says 10.20.193.78:58344) 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 learned_addr learned my addr 10.20.193.78:0/1428120072 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb7101388b0 auth_method 0 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.737+0000 7fb70e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.738+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.738+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.738+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:59.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.738+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:00:59.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70f7fe640 1 -- 10.20.193.78:0/1428120072 >> v1:10.20.193.115:6789/0 conn(0x7fb7100a3220 legacy=0x7fb7100a3630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:00:59.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70effd640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 msgr2=0x7fb710070530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:00:59.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70effd640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 0x7fb710070530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:00:59.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:00:59.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb7101388b0 auth_method 0 2026-02-17T06:00:59.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:00:59.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:00:59.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:00:59.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:00:59.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:00:59.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:00:59.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:00:59.938+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:01:00.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70effd640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 msgr2=0x7fb710070530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:00.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70effd640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 0x7fb710070530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70f7fe640 1 -- 10.20.193.78:0/1428120072 >> v1:10.20.193.115:6789/0 conn(0x7fb7100a3220 legacy=0x7fb7100a3630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb7101388b0 auth_method 0 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:00.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:00.339+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:01:01.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.139+0000 7fb70f7fe640 1 -- 10.20.193.78:0/1428120072 >> v1:10.20.193.115:6789/0 conn(0x7fb7100a3220 legacy=0x7fb7100a3630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:01.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.139+0000 7fb70effd640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 msgr2=0x7fb710070530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:01.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.139+0000 7fb70effd640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 0x7fb710070530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:01:01.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.139+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:01.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.139+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb7101388b0 auth_method 0 2026-02-17T06:01:01.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.139+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:01.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.139+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:01.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.139+0000 7fb70e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:01.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.140+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:01:01.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.140+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:01.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.140+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:01.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:01.140+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:01:02.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 10 monclient(hunting): tick 2026-02-17T06:01:02.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:01:02.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 monclient(hunting): continuing hunt 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 msgr2=0x7fb710070530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.115:3300/0 conn(0x7fb7101384d0 0x7fb710070530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 -- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 msgr2=0x7fb710138d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 --2- 10.20.193.78:0/1428120072 >> v2:10.20.193.78:3300/0 conn(0x7fb7101388b0 0x7fb710138d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 -- 10.20.193.78:0/1428120072 >> v1:10.20.193.115:6789/0 conn(0x7fb7100a3220 legacy=0x7fb7100a3630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 10 monclient: _add_conns ranks=[0,2] 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6f4003790 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 10 monclient(hunting): picked mon.noname-b con 0x7fb6f40073b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:02.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 -- 10.20.193.78:0/1428120072 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71006f870 con 0x7fb6f4003790 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 1 -- 10.20.193.78:0/1428120072 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb7100a4b20 con 0x7fb6f40073b0 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.737+0000 7fb70cff9640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb6f40073b0 auth_method 0 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:02.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.738+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:02.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.740+0000 7fb70effd640 1 -- 10.20.193.78:0/1428120072 >> v1:10.20.193.78:6789/0 conn(0x7fb6f4003790 legacy=0x7fb6f4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:02.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:02.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb6f40073b0 auth_method 0 2026-02-17T06:01:02.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:02.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:02.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:02.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:01:02.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:02.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:02.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:02.938+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb6f40073b0 auth_method 0 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:03.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:03.339+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:01:04.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.139+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.139+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb6f40073b0 auth_method 0 2026-02-17T06:01:04.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.139+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.139+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.139+0000 7fb70e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:04.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.140+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:01:04.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.140+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:04.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.140+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:04.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.140+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:01:04.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.203+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 629554117 0 0) 0x7fb71006f870 con 0x7fb6f4003790 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.203+0000 7fb70dffb640 10 monclient(hunting): _negotiate 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.203+0000 7fb70dffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.203+0000 7fb70dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.203+0000 7fb70dffb640 10 monclient(hunting): my global_id is 184164 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.203+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6fc003620 con 0x7fb6f4003790 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 754317064 0 0) 0x7fb6fc003620 con 0x7fb6f4003790 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 msgr2=0x7fb6f4009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f40073b0 0x7fb6f4009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: _finish_hunting 0 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb710073f70 con 0x7fb6f4003790 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fb7040044a0 con 0x7fb6f4003790 2026-02-17T06:01:04.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: dump: 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.209 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 >> v1:10.20.193.78:6789/0 conn(0x7fb6f4003790 legacy=0x7fb6f4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 1 -- 10.20.193.78:0/1428120072 reap_dead start 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc003ab0 0x7fb6fc003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient(hunting): picked mon.b con 0x7fb6fc003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc004900 0x7fb6fc004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient(hunting): picked mon.a con 0x7fb6fc004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 0x7fb6fc00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient(hunting): picked mon.c con 0x7fb6fc0053e0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100a4b20 con 0x7fb6fc004900 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc00c290 con 0x7fb6fc003ab0 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb6fc00c490 con 0x7fb6fc0053e0 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70dffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70effd640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc003ab0 msgr2=0x7fb6fc003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc004900 0x7fb6fc004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70effd640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc003ab0 0x7fb6fc003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:04.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request con 0x7fb6fc004900 auth_method 0 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 0x7fb6fc00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 10 monclient(hunting): get_auth_request con 0x7fb6fc0053e0 auth_method 0 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:04.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient(hunting): handle_auth_done global_id 184164 payload 293 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 msgr2=0x7fb6fc00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 0x7fb6fc00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc003ab0 msgr2=0x7fb6fc003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc003ab0 0x7fb6fc003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 1 monclient: found mon.a 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc003620 con 0x7fb6fc004900 2026-02-17T06:01:04.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:04.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:04.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb70e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:04.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb71564c640 5 monclient: authenticate success, global_id 184164 2026-02-17T06:01:04.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.204+0000 7fb71564c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:04.215 INFO:tasks.rados.rados.0.trial115.stdout:2462: done (1 left) 2026-02-17T06:01:04.215 INFO:tasks.rados.rados.0.trial115.stdout:2463: snap_remove snap 248 2026-02-17T06:01:04.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.206+0000 7fb70e7fc640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc004900 msgr2=0x7fb6fc004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.206+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc004900 0x7fb6fc004d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8006fd0 tx=0x7fb6f8030fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f770962ba367bcab server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8037070 con 0x7fb6fc004900 2026-02-17T06:01:04.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:04.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 10 monclient: dump: 2026-02-17T06:01:04.222 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f8031800 con 0x7fb6fc004900 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:04.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 1 -- 10.20.193.78:0/1428120072 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8003710 con 0x7fb6fc004900 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 4 set_mon_vals no callback set 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb70dffb640 10 monclient: dump: 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.224 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 10 monclient: shutdown 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 -- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc004900 msgr2=0x7fb6fc004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc004900 0x7fb6fc004d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8006fd0 tx=0x7fb6f8030fe0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 -- 10.20.193.78:0/1428120072 shutdown_connections 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 0x7fb6fc00bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc004900 0x7fb6fc004d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 --2- 10.20.193.78:0/1428120072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc003ab0 0x7fb6fc003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 -- 10.20.193.78:0/1428120072 >> 10.20.193.78:0/1428120072 conn(0x7fb71009c000 msgr2=0x7fb7100a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 -- 10.20.193.78:0/1428120072 shutdown_connections 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.207+0000 7fb71564c640 1 -- 10.20.193.78:0/1428120072 wait complete. 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient: build_initial_monmap 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient: monmap: 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:04.226 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 1 Processor -- start 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 1 -- start start 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient: init 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc003ab0 0x7fb7101343b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient(hunting): picked mon.noname-a con 0x7fb6fc003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc004900 0x7fb71009d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient(hunting): picked mon.noname-b con 0x7fb6fc004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 0x7fb71009d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient(hunting): picked mon.noname-c con 0x7fb6fc0053e0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100a9660 con 0x7fb6fc003ab0 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100a94e0 con 0x7fb6fc004900 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb710070530 con 0x7fb6fc0053e0 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb71564c640 10 monclient(hunting): authenticate will time out at 1509.762655s 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc003ab0 0x7fb7101343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70e7fc640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc004900 msgr2=0x7fb71009d490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc003ab0 0x7fb7101343b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58386/0 (socket says 10.20.193.78:58386) 2026-02-17T06:01:04.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 -- 10.20.193.78:0/2821546269 learned_addr learned my addr 10.20.193.78:0/2821546269 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient(hunting): get_auth_request con 0x7fb6fc003ab0 auth_method 0 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc004900 0x7fb71009d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70f7fe640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 0x7fb71009d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70f7fe640 10 monclient(hunting): get_auth_request con 0x7fb6fc0053e0 auth_method 0 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient(hunting): handle_auth_done global_id 184167 payload 824 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 -- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 msgr2=0x7fb71009d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 0x7fb71009d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 -- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc004900 msgr2=0x7fb71009d490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc004900 0x7fb71009d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient: _finish_hunting 0 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 monclient: found mon.noname-a 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:04.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.208+0000 7fb70effd640 1 -- 10.20.193.78:0/2821546269 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb710120af0 con 0x7fb6fc003ab0 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.210+0000 7fb70effd640 1 -- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc003ab0 msgr2=0x7fb7101343b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb70effd640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc003ab0 0x7fb7101343b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb704015570 tx=0x7fb704015a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/2821546269 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70401c070 con 0x7fb6fc003ab0 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: dump: 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.231 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/2821546269 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb704011040 con 0x7fb6fc003ab0 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 5 monclient: authenticate success, global_id 184167 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 10 monclient: _renew_subs 2026-02-17T06:01:04.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb710072a40 con 0x7fb6fc003ab0 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/2821546269 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704018a50 con 0x7fb6fc003ab0 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb70d7fa640 4 set_mon_vals no callback set 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 10 monclient: dump: 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.233 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 10 monclient: _renew_subs 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71009fa10 con 0x7fb6fc003ab0 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/2821546269 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb704018cb0 con 0x7fb6fc003ab0 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.211+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7101203c0 con 0x7fb6fc003ab0 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.212+0000 7fb6ef7fe640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03de60 0x7fb6dc040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.212+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/2821546269 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(528..528 src has 335..528) ==== 4811+0+0 (secure 0 0 0) 0x7fb70405bad0 con 0x7fb6fc003ab0 2026-02-17T06:01:04.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.212+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03de60 0x7fb6dc040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.212+0000 7fb70e7fc640 10 monclient: get_auth_request con 0x7fb6dc03de60 auth_method 0 2026-02-17T06:01:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.214+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/2821546269 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb7101203c0 con 0x7fb6fc003ab0 2026-02-17T06:01:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.214+0000 7fb6ef7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.214+0000 7fb6ef7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:04.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.215+0000 7fb70e7fc640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03de60 0x7fb6dc040320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8039000 tx=0x7fb6f8037000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.329+0000 7fb71564c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:01:04.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.329+0000 7fb71564c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:04.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.329+0000 7fb71564c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.329+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb710138d00 con 0x7fb6fc003ab0 2026-02-17T06:01:04.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.329+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/2821546269 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fb70405a4b0 con 0x7fb6fc003ab0 2026-02-17T06:01:04.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.329+0000 7fb6ef7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:04.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.329+0000 7fb6ef7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:04.332 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":178,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:01:04.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03de60 msgr2=0x7fb6dc040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03de60 0x7fb6dc040320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8039000 tx=0x7fb6f8037000 comp rx=0 tx=0).stop 2026-02-17T06:01:04.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 10 monclient: shutdown 2026-02-17T06:01:04.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:04.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc003ab0 msgr2=0x7fb7101343b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc003ab0 0x7fb7101343b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb704015570 tx=0x7fb704015a40 comp rx=0 tx=0).stop 2026-02-17T06:01:04.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 shutdown_connections 2026-02-17T06:01:04.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6fc0053e0 0x7fb71009d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6dc03de60 0x7fb6dc040320 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6fc004900 0x7fb71009d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 --2- 10.20.193.78:0/2821546269 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6fc003ab0 0x7fb7101343b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 >> 10.20.193.78:0/2821546269 conn(0x7fb71009c000 msgr2=0x7fb710135370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:04.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 shutdown_connections 2026-02-17T06:01:04.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.330+0000 7fb71564c640 1 -- 10.20.193.78:0/2821546269 wait complete. 2026-02-17T06:01:04.340 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:01:04.340 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:01:04.340 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.407+0000 7f69869c1640 10 monclient: get_monmap_and_config 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.407+0000 7f69869c1640 10 monclient: build_initial_monmap 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.407+0000 7f69869c1640 10 monclient: monmap: 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:04.408849+0000 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:04.408849+0000 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:04.408 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:04.409 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:04.409 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:04.409 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:04.409 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 1 Processor -- start 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 1 -- start start 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient: init 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f698006c890 0x7f698006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient(hunting): picked mon.noname-c con 0x7f698006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f698009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006d1b0 0x7f6980135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient(hunting): picked mon.noname-b con 0x7f698006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698006daf0 con 0x7f698009c4a0 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f69800a0880 con 0x7f698006c890 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f698009cc80 con 0x7f698006d1b0 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f69869c1640 10 monclient(hunting): authenticate will time out at 1509.962459s 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f697ffff640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f698006c890 msgr2=0x7f698006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f697f7fe640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f698009c4a0 legacy=0x7f6980134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f697ffff640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f698006c890 0x7f698006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006d1b0 0x7f6980135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006d1b0 0x7f6980135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36920/0 (socket says 10.20.193.78:36920) 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 1 -- 10.20.193.78:0/925603363 learned_addr learned my addr 10.20.193.78:0/925603363 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient(hunting): get_auth_request con 0x7f698006d1b0 auth_method 0 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient(hunting): handle_auth_done global_id 144557 payload 274 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 1 -- 10.20.193.78:0/925603363 >> v2:10.20.193.115:3300/0 conn(0x7f698006c890 msgr2=0x7f698006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:04.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 1 --2- 10.20.193.78:0/925603363 >> v2:10.20.193.115:3300/0 conn(0x7f698006c890 0x7f698006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 1 -- 10.20.193.78:0/925603363 >> v1:10.20.193.115:6789/0 conn(0x7f698009c4a0 legacy=0x7f6980134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient: _finish_hunting 0 2026-02-17T06:01:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 1 monclient: found mon.noname-b 2026-02-17T06:01:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:01:04.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.408+0000 7f6984f37640 1 -- 10.20.193.78:0/925603363 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698009cf60 con 0x7f698006d1b0 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f6984f37640 1 -- 10.20.193.78:0/925603363 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006d1b0 msgr2=0x7f6980135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f6984f37640 1 --2- 10.20.193.78:0/925603363 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006d1b0 0x7f6980135500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f698006dcc0 tx=0x7f697402f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a8b8caa756635e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 1 -- 10.20.193.78:0/925603363 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740373f0 con 0x7f698006d1b0 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: dump: 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: _finish_auth 0 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 1 -- 10.20.193.78:0/925603363 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6974037590 con 0x7f698006d1b0 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 5 monclient: authenticate success, global_id 144557 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 1 -- 10.20.193.78:0/925603363 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974037960 con 0x7f698006d1b0 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 4 set_mon_vals no callback set 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f697effd640 10 monclient: dump: 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.413 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 10 monclient: shutdown 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 1 -- 10.20.193.78:0/925603363 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006d1b0 msgr2=0x7f6980135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 1 --2- 10.20.193.78:0/925603363 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006d1b0 0x7f6980135500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f698006dcc0 tx=0x7f697402f230 comp rx=0 tx=0).stop 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 1 -- 10.20.193.78:0/925603363 shutdown_connections 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 1 --2- 10.20.193.78:0/925603363 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006d1b0 0x7f6980135500 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 1 --2- 10.20.193.78:0/925603363 >> v2:10.20.193.115:3300/0 conn(0x7f698006c890 0x7f698006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 1 -- 10.20.193.78:0/925603363 >> 10.20.193.78:0/925603363 conn(0x7f69800a6620 msgr2=0x7f69800a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 1 -- 10.20.193.78:0/925603363 shutdown_connections 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.411+0000 7f69869c1640 1 -- 10.20.193.78:0/925603363 wait complete. 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient: build_initial_monmap 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient: monmap: 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:04.414 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 1 Processor -- start 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 1 -- start start 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient: init 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 0x7f69801cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient(hunting): picked mon.noname-c con 0x7f698006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698006d1b0 0x7f69801cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient(hunting): picked mon.noname-a con 0x7f698006d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f698009c4a0 0x7f698009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient(hunting): picked mon.noname-b con 0x7f698009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69800a01a0 con 0x7f698006d1b0 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f69800a0020 con 0x7f698009c4a0 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f698009be20 con 0x7f698006c890 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f69869c1640 10 monclient(hunting): authenticate will time out at 1509.966660s 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 0x7f69801cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 0x7f69801cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36934/0 (socket says 10.20.193.78:36934) 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 -- 10.20.193.78:0/3845690890 learned_addr learned my addr 10.20.193.78:0/3845690890 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient(hunting): get_auth_request con 0x7f698006c890 auth_method 0 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f6984f37640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f698009c4a0 msgr2=0x7f698009db50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f6984f37640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f698009c4a0 0x7f698009db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697f7fe640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698006d1b0 0x7f69801cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697f7fe640 10 monclient(hunting): get_auth_request con 0x7f698006d1b0 auth_method 0 2026-02-17T06:01:04.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient(hunting): handle_auth_done global_id 144560 payload 824 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f698009c4a0 msgr2=0x7f698009db50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f698009c4a0 0x7f698009db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698006d1b0 msgr2=0x7f69801cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698006d1b0 0x7f69801cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 monclient: found mon.noname-c 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:04.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.412+0000 7f697ffff640 1 -- 10.20.193.78:0/3845690890 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801294d0 con 0x7f698006c890 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697ffff640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 msgr2=0x7f69801cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697ffff640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 0x7f69801cf6a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f696c00b7b0 tx=0x7f696c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697ffff640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 0x7f69801cf6a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f696c00b7b0 tx=0x7f696c00bc80 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f696c010070 config(0 keys) 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c0052d0 con 0x7f698006c890 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: dump: 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.417 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: _finish_auth 0 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c010070 con 0x7f698006c890 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 5 monclient: authenticate success, global_id 144560 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 10 monclient: _renew_subs 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697e7fc640 4 set_mon_vals no callback set 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698006fd20 con 0x7f698006c890 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 10 monclient: _renew_subs 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f698009f7f0 con 0x7f698006c890 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e090 0x7f698009e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f698006cc70 con 0x7f698009e090 2026-02-17T06:01:04.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e090 0x7f698009e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 10 monclient: get_auth_request con 0x7f698009e090 auth_method 0 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 10 monclient(hunting): handle_auth_done global_id 184173 payload 824 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697f7fe640 1 -- 10.20.193.78:0/3845690890 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69800008d0 con 0x7f698009e090 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c0056d0 con 0x7f698006c890 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 10 monclient: dump: 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.415+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f696c01d030 con 0x7f698006c890 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.416+0000 7f697cff9640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f695003de60 0x7f6950040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.416+0000 7f6984f37640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f695003de60 0x7f6950040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.416+0000 7f6984f37640 10 monclient: get_auth_request con 0x7f695003de60 auth_method 0 2026-02-17T06:01:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697f7fe640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e090 msgr2=0x7f698009e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697f7fe640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e090 0x7f698009e470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f698009f2d0 tx=0x7f6970009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970016070 con 0x7f698009e090 2026-02-17T06:01:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:04.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 10 monclient: dump: 2026-02-17T06:01:04.420 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.420 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f697000e9c0 con 0x7f698009e090 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e090 msgr2=0x7f698009e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e090 0x7f698009e470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f698009f2d0 tx=0x7f6970009ae0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e090 msgr2=0x7f698009e470 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.418+0000 7f697cff9640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e090 0x7f698009e470 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f698009f2d0 tx=0x7f6970009ae0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.419+0000 7f6984f37640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f695003de60 0x7f6950040320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f698009c980 tx=0x7f6974033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f69869c1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f69869c1640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e9b0 0x7f69801da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f69869c1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6980128020 con 0x7f698009e9b0 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e9b0 0x7f69801da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 10 monclient: get_auth_request con 0x7f698009e9b0 auth_method 0 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 10 monclient(hunting): handle_auth_done global_id 184176 payload 824 2026-02-17T06:01:04.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.421+0000 7f697f7fe640 1 -- 10.20.193.78:0/3845690890 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f698009edc0 con 0x7f698009e9b0 2026-02-17T06:01:04.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697f7fe640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e9b0 msgr2=0x7f69801da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697f7fe640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e9b0 0x7f69801da3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f698009f2d0 tx=0x7f6970005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970016040 con 0x7f698009e9b0 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 10 monclient: dump: 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4699 (secure 0 0 0) 0x7f697001b660 con 0x7f698009e9b0 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e9b0 msgr2=0x7f69801da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e9b0 0x7f69801da3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f698009f2d0 tx=0x7f6970005a80 comp rx=0 tx=0).stop 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e9b0 msgr2=0x7f69801da3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.423+0000 7f697cff9640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f698009e9b0 0x7f69801da3c0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f698009f2d0 tx=0x7f6970005a80 comp rx=0 tx=0).stop 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:04.425 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 178, 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 29204, 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:04.426 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:04.427 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.428 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.429 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:04.430 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:04.431 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f695003de60 msgr2=0x7f6950040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f695003de60 0x7f6950040320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f698009c980 tx=0x7f6974033000 comp rx=0 tx=0).stop 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 10 monclient: shutdown 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 msgr2=0x7f69801cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 0x7f69801cf6a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f696c00b7b0 tx=0x7f696c00bc80 comp rx=0 tx=0).stop 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f6984f37640 1 -- 10.20.193.78:0/3845690890 reap_dead start 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f697cff9640 1 -- 10.20.193.78:0/3845690890 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(528..528 src has 335..528) ==== 4811+0+0 (unknown 0 0 0) 0x7f696c051dd0 con 0x7f698006c890 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 shutdown_connections 2026-02-17T06:01:04.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 --2- 10.20.193.78:0/3845690890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f698006c890 0x7f69801cf6a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 >> 10.20.193.78:0/3845690890 conn(0x7f69800a6620 msgr2=0x7f6980072200 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:04.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 shutdown_connections 2026-02-17T06:01:04.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.424+0000 7f69869c1640 1 -- 10.20.193.78:0/3845690890 wait complete. 2026-02-17T06:01:04.433 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:01:04.433 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:01:04.433 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:01:04.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.501+0000 7f3ca3fff640 10 monclient: get_monmap_and_config 2026-02-17T06:01:04.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.501+0000 7f3ca3fff640 10 monclient: build_initial_monmap 2026-02-17T06:01:04.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.501+0000 7f3ca3fff640 10 monclient: monmap: 2026-02-17T06:01:04.502 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:04.502647+0000 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:04.502647+0000 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:04.503 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.501+0000 7f3ca3fff640 1 Processor -- start 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.501+0000 7f3ca3fff640 1 -- start start 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.501+0000 7f3ca3fff640 10 monclient: init 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3ca4072f40 0x7f3ca41381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f3ca4072f40 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3ca4073320 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3ca40a2e20 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca40a0a40 con 0x7f3ca4073320 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca406ef20 con 0x7f3ca40a2e20 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca40a4720 con 0x7f3ca4072f40 2026-02-17T06:01:04.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:04.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca3fff640 10 monclient(hunting): authenticate will time out at 1510.056416s 2026-02-17T06:01:04.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca2ffd640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f3ca4072f40 msgr2=0x7f3ca41381c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:04.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca2ffd640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3ca4072f40 0x7f3ca41381c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:04.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca37fe640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f3ca40a2e20 legacy=0x7f3ca413fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:04.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca27fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f3ca4073320 0x7f3ca40a2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:34616/0 (socket says 10.20.193.78:34616) 2026-02-17T06:01:04.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.502+0000 7f3ca27fc640 1 -- 10.20.193.78:0/666208973 learned_addr learned my addr 10.20.193.78:0/666208973 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:01:04.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca27fc640 1 -- 10.20.193.78:0/666208973 >> v1:10.20.193.78:6789/0 conn(0x7f3ca4073320 legacy=0x7f3ca40a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 806515107 0 0) 0x7f3ca40a0a40 con 0x7f3ca4073320 2026-02-17T06:01:04.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): _negotiate 2026-02-17T06:01:04.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): my global_id is 184179 2026-02-17T06:01:04.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c90003620 con 0x7f3ca4073320 2026-02-17T06:01:04.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1795090620 0 0) 0x7f3c90003620 con 0x7f3ca4073320 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 >> v2:10.20.193.115:3300/0 conn(0x7f3ca4072f40 msgr2=0x7f3ca41381c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 --2- 10.20.193.78:0/666208973 >> v2:10.20.193.115:3300/0 conn(0x7f3ca4072f40 0x7f3ca41381c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 >> v1:10.20.193.115:6789/0 conn(0x7f3ca40a2e20 legacy=0x7f3ca413fff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: _finish_hunting 0 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca406f200 con 0x7f3ca4073320 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f3c8c0031b0 con 0x7f3ca4073320 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca3fff640 5 monclient: authenticate success, global_id 184179 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca3fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: dump: 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.512 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 >> v1:10.20.193.78:6789/0 conn(0x7f3ca4073320 legacy=0x7f3ca40a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3c90003ab0 0x7f3c90003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): picked mon.c con 0x7f3c90003ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90004900 0x7f3c90004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): picked mon.a con 0x7f3c90004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3c900054c0 0x7f3c9000bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): picked mon.b con 0x7f3c900054c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca40a4720 con 0x7f3c90004900 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9000c400 con 0x7f3c900054c0 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9000c600 con 0x7f3c90003ab0 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.504+0000 7f3ca1ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca1ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca3fff640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3c90003ab0 0x7f3c90003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca37fe640 1 -- 10.20.193.78:0/666208973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3c900054c0 msgr2=0x7f3c9000bb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90004900 0x7f3c90004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca37fe640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3c900054c0 0x7f3c9000bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient(hunting): get_auth_request con 0x7f3c90003ab0 auth_method 0 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 10 monclient(hunting): get_auth_request con 0x7f3c90004900 auth_method 0 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient(hunting): handle_auth_done global_id 184179 payload 293 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/666208973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3c900054c0 msgr2=0x7f3c9000bb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3c900054c0 0x7f3c9000bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90004900 msgr2=0x7f3c90004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90004900 0x7f3c90004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 1 monclient: found mon.c 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca37fe640 1 -- 10.20.193.78:0/666208973 reap_dead start 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/666208973 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c90003620 con 0x7f3c90003ab0 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.505+0000 7f3ca27fc640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90004900 0x7f3c90004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3c90003ab0 msgr2=0x7f3c90003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3c90003ab0 0x7f3c90003eb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009e30 tx=0x7f3c9802f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97378ef6a43a1f0e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98009a30 con 0x7f3c90003ab0 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca1ffb640 10 monclient: dump: 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca3fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3c90003ab0 0x7f3c90003eb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009e30 tx=0x7f3c9802f660 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f3c98003e60 mon_map magic: 0 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c98003cb0 con 0x7f3c90003ab0 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca3fff640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.507+0000 7f3ca3fff640 4 set_mon_vals no callback set 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 10 monclient: shutdown 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 -- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3c90003ab0 msgr2=0x7f3c90003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3c90003ab0 0x7f3c90003eb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009e30 tx=0x7f3c9802f660 comp rx=0 tx=0).stop 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca1ffb640 1 -- 10.20.193.78:0/666208973 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7f3c98003e60 con 0x7f3c90003ab0 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca1ffb640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 -- 10.20.193.78:0/666208973 shutdown_connections 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 --2- 10.20.193.78:0/666208973 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3c90003ab0 0x7f3c90003eb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 -- 10.20.193.78:0/666208973 >> 10.20.193.78:0/666208973 conn(0x7f3ca40a7330 msgr2=0x7f3ca40a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 -- 10.20.193.78:0/666208973 shutdown_connections 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 -- 10.20.193.78:0/666208973 wait complete. 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 10 monclient: build_initial_monmap 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 10 monclient: monmap: 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:04.517 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 Processor -- start 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 1 -- start start 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.508+0000 7f3ca3fff640 10 monclient: init 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90003ab0 0x7f3ca41345a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient(hunting): picked mon.noname-a con 0x7f3c90003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3ca41ced50 0x7f3ca4134ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient(hunting): picked mon.noname-b con 0x7f3ca41ced50 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4122010 0x7f3ca4122490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f3ca4122010 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4068950 con 0x7f3c90003ab0 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca406f5a0 con 0x7f3ca41ced50 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca4068ad0 con 0x7f3ca4122010 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca3fff640 10 monclient(hunting): authenticate will time out at 1510.063280s 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90003ab0 0x7f3ca41345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90003ab0 0x7f3ca41345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58408/0 (socket says 10.20.193.78:58408) 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/914980991 learned_addr learned my addr 10.20.193.78:0/914980991 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient(hunting): get_auth_request con 0x7f3c90003ab0 auth_method 0 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca27fc640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3ca41ced50 msgr2=0x7f3ca4134ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca37fe640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4122010 0x7f3ca4122490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca27fc640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3ca41ced50 0x7f3ca4134ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca37fe640 10 monclient(hunting): get_auth_request con 0x7f3ca4122010 auth_method 0 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient(hunting): handle_auth_done global_id 184182 payload 824 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4122010 msgr2=0x7f3ca4122490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4122010 0x7f3ca4122490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3ca41ced50 msgr2=0x7f3ca4134ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3ca41ced50 0x7f3ca4134ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 monclient: found mon.noname-a 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.509+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/914980991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca4125cc0 con 0x7f3c90003ab0 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca2ffd640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90003ab0 msgr2=0x7f3ca41345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90003ab0 0x7f3ca41345a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009f60 tx=0x7f3c98005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca2ffd640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90003ab0 0x7f3ca41345a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009f60 tx=0x7f3c98005b10 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f3c98046020 mon_map magic: 0 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98046020 con 0x7f3c90003ab0 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 10 monclient: dump: 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.524 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 10 monclient: _finish_auth 0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9803d040 con 0x7f3c90003ab0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3c7bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca3fff640 5 monclient: authenticate success, global_id 184182 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca3fff640 10 monclient: _renew_subs 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca3fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca4137690 con 0x7f3c90003ab0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca17fa640 4 set_mon_vals no callback set 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca3fff640 10 monclient: _renew_subs 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca3fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.511+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca41259e0 con 0x7f3c90003ab0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca3fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca3fff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca41229d0 0x7f3ca411e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca3fff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca40a3260 con 0x7f3ca41229d0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca41229d0 0x7f3ca411e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 10 monclient: get_auth_request con 0x7f3ca41229d0 auth_method 0 2026-02-17T06:01:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_done global_id 144566 payload 824 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3ca27fc640 1 -- 10.20.193.78:0/914980991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ca4124100 con 0x7f3ca41229d0 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c980092d0 con 0x7f3c90003ab0 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3c7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3c7bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3c7bfff640 10 monclient: dump: 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3c98037430 con 0x7f3c90003ab0 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.512+0000 7f3c7bfff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3c7403de90 0x7f3c74040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.513+0000 7f3ca37fe640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3c7403de90 0x7f3c74040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.513+0000 7f3ca37fe640 10 monclient: get_auth_request con 0x7f3c7403de90 auth_method 0 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3ca27fc640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca41229d0 msgr2=0x7f3ca411e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3ca27fc640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca41229d0 0x7f3ca411e4d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4125ed0 tx=0x7f3c8c007730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00c180 con 0x7f3ca41229d0 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 10 monclient: dump: 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f3c8c00c150 con 0x7f3ca41229d0 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca41229d0 msgr2=0x7f3ca411e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca41229d0 0x7f3ca411e4d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4125ed0 tx=0x7f3c8c007730 comp rx=0 tx=0).stop 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca41229d0 msgr2=0x7f3ca411e4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.514+0000 7f3c7bfff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca41229d0 0x7f3ca411e4d0 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4125ed0 tx=0x7f3c8c007730 comp rx=0 tx=0).stop 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.515+0000 7f3ca37fe640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3c7403de90 0x7f3c74040350 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3ca406f8e0 tx=0x7f3c94009550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca3fff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca3fff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4063880 0x7f3ca4063c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca3fff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca4122db0 con 0x7f3ca4063880 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca27fc640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4063880 0x7f3ca4063c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca27fc640 10 monclient: get_auth_request con 0x7f3ca4063880 auth_method 0 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:04.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.517+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.518+0000 7f3ca27fc640 10 monclient(hunting): handle_auth_done global_id 144569 payload 824 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.518+0000 7f3ca27fc640 1 -- 10.20.193.78:0/914980991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3ca411ea10 con 0x7f3ca4063880 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca27fc640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4063880 msgr2=0x7f3ca4063c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca27fc640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4063880 0x7f3ca4063c90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c014000 tx=0x7f3ca406f0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c013810 con 0x7f3ca4063880 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 10 monclient: dump: 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:04.529 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7f3c8c013810 con 0x7f3ca4063880 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4063880 msgr2=0x7f3ca4063c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4063880 0x7f3ca4063c90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c014000 tx=0x7f3ca406f0f0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4063880 msgr2=0x7f3ca4063c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3ca4063880 0x7f3ca4063c90 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c014000 tx=0x7f3ca406f0f0 comp rx=0 tx=0).stop 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 178, 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 248055, 2026-02-17T06:01:04.530 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:04.531 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:04.532 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:04.533 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.534 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.535 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.536 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.537 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:04.538 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3c7403de90 msgr2=0x7f3c74040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3c7403de90 0x7f3c74040350 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3ca406f8e0 tx=0x7f3c94009550 comp rx=0 tx=0).stop 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 10 monclient: shutdown 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90003ab0 msgr2=0x7f3ca41345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 1 --2- 10.20.193.78:0/914980991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3c90003ab0 0x7f3ca41345a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009f60 tx=0x7f3c98005b10 comp rx=0 tx=0).stop 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca37fe640 1 -- 10.20.193.78:0/914980991 reap_dead start 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3c7bfff640 1 -- 10.20.193.78:0/914980991 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(528..528 src has 335..528) ==== 4811+0+0 (unknown 0 0 0) 0x7f3c980784c0 con 0x7f3c90003ab0 2026-02-17T06:01:04.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca37fe640 1 -- 10.20.193.78:0/914980991 reap_dead start 2026-02-17T06:01:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 shutdown_connections 2026-02-17T06:01:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 >> 10.20.193.78:0/914980991 conn(0x7f3ca40a7330 msgr2=0x7f3ca4072370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 shutdown_connections 2026-02-17T06:01:04.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:04.520+0000 7f3ca3fff640 1 -- 10.20.193.78:0/914980991 wait complete. 2026-02-17T06:01:04.540 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:01:04.540 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:01:05.208 INFO:tasks.rados.rados.0.trial115.stdout:2463: done (1 left) 2026-02-17T06:01:05.209 INFO:tasks.rados.rados.0.trial115.stdout:2464: write oid 41 current snap is 249 2026-02-17T06:01:05.209 INFO:tasks.rados.rados.0.trial115.stdout:2464: seq_num 870 ranges {622090=730872,1760393=607307} 2026-02-17T06:01:05.213 INFO:tasks.rados.rados.0.trial115.stdout:2464: writing trial11545420-41 from 622090 to 1352962 tid 1 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2464: writing trial11545420-41 from 1760393 to 2367700 tid 2 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2465: rollback oid 24 current snap is 249 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2460: finishing write tid 2 to trial11545420-48 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2465: done (2 left) 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2466: snap_create 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2460: finishing write tid 3 to trial11545420-48 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2460: finishing write tid 4 to trial11545420-48 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2460: oid 48 updating version 0 to 1382 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2460: oid 48 version 1382 is already newer than 1380 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2460: oid 48 version 1382 is already newer than 1381 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1382 (ObjNum 869 snap 249 seq_num 869) dirty exists 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2460: left oid 48 (ObjNum 869 snap 249 seq_num 869) 2026-02-17T06:01:05.217 INFO:tasks.rados.rados.0.trial115.stdout:2464: finishing write tid 1 to trial11545420-41 2026-02-17T06:01:05.219 INFO:tasks.rados.rados.0.trial115.stdout:2464: finishing write tid 2 to trial11545420-41 2026-02-17T06:01:05.219 INFO:tasks.rados.rados.0.trial115.stdout:2464: finishing write tid 3 to trial11545420-41 2026-02-17T06:01:05.219 INFO:tasks.rados.rados.0.trial115.stdout:2464: finishing write tid 4 to trial11545420-41 2026-02-17T06:01:05.220 INFO:tasks.rados.rados.0.trial115.stdout:2464: oid 41 updating version 0 to 1090 2026-02-17T06:01:05.220 INFO:tasks.rados.rados.0.trial115.stdout:2464: oid 41 updating version 1090 to 1091 2026-02-17T06:01:05.220 INFO:tasks.rados.rados.0.trial115.stdout:2464: oid 41 version 1091 is already newer than 1089 2026-02-17T06:01:05.220 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1091 (ObjNum 870 snap 249 seq_num 870) dirty exists 2026-02-17T06:01:05.220 INFO:tasks.rados.rados.0.trial115.stdout:2464: left oid 41 (ObjNum 870 snap 249 seq_num 870) 2026-02-17T06:01:05.226 INFO:tasks.rados.rados.0.trial115.stdout:2460: done (2 left) 2026-02-17T06:01:05.226 INFO:tasks.rados.rados.0.trial115.stdout:2464: done (1 left) 2026-02-17T06:01:05.226 INFO:tasks.rados.rados.0.trial115.stdout:2466: done (0 left) 2026-02-17T06:01:05.227 INFO:tasks.rados.rados.0.trial115.stdout:2467: snap_create 2026-02-17T06:01:06.454 INFO:tasks.rados.rados.0.trial115.stdout:2467: done (0 left) 2026-02-17T06:01:06.454 INFO:tasks.rados.rados.0.trial115.stdout:2468: snap_remove snap 249 2026-02-17T06:01:07.456 INFO:tasks.rados.rados.0.trial115.stdout:2468: done (0 left) 2026-02-17T06:01:07.456 INFO:tasks.rados.rados.0.trial115.stdout:2469: delete oid 40 current snap is 251 2026-02-17T06:01:07.460 INFO:tasks.rados.rados.0.trial115.stdout:2469: done (0 left) 2026-02-17T06:01:07.460 INFO:tasks.rados.rados.0.trial115.stdout:2470: rollback oid 40 current snap is 251 2026-02-17T06:01:07.461 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 250 2026-02-17T06:01:07.461 INFO:tasks.rados.rados.0.trial115.stdout:2471: copy_from oid 38 from oid 10 current snap is 251 2026-02-17T06:01:07.461 INFO:tasks.rados.rados.0.trial115.stdout:2472: rollback oid 11 current snap is 251 2026-02-17T06:01:07.461 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 250 2026-02-17T06:01:07.461 INFO:tasks.rados.rados.0.trial115.stdout:2473: read oid 39 snap -1 2026-02-17T06:01:07.461 INFO:tasks.rados.rados.0.trial115.stdout:2473: expect (ObjNum 816 snap 235 seq_num 816) 2026-02-17T06:01:07.461 INFO:tasks.rados.rados.0.trial115.stdout:2474: delete oid 45 current snap is 251 2026-02-17T06:01:07.461 INFO:tasks.rados.rados.0.trial115.stdout:2470: finishing rollback tid 1 to trial11545420-40 2026-02-17T06:01:07.462 INFO:tasks.rados.rados.0.trial115.stdout:2470: finishing rollback tid 2 to trial11545420-40 2026-02-17T06:01:07.462 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1277 (ObjNum 855 snap 244 seq_num 855) dirty exists 2026-02-17T06:01:07.462 INFO:tasks.rados.rados.0.trial115.stdout:2472: finishing rollback tid 1 to trial11545420-11 2026-02-17T06:01:07.462 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 0 (ObjNum 49 snap 0 seq_num 1330673232) dirty dne 2026-02-17T06:01:07.462 INFO:tasks.rados.rados.0.trial115.stdout:2471: finishing copy_from to trial11545420-38 2026-02-17T06:01:07.462 INFO:tasks.rados.rados.0.trial115.stdout:2471: got expected ENOENT (src dne) 2026-02-17T06:01:07.462 INFO:tasks.rados.rados.0.trial115.stdout:2471: finishing copy_from racing read to trial11545420-38 2026-02-17T06:01:07.464 INFO:tasks.rados.rados.0.trial115.stdout:2473: expect (ObjNum 816 snap 235 seq_num 816) 2026-02-17T06:01:07.547 INFO:tasks.rados.rados.0.trial115.stdout:2470: done (4 left) 2026-02-17T06:01:07.547 INFO:tasks.rados.rados.0.trial115.stdout:2471: done (3 left) 2026-02-17T06:01:07.547 INFO:tasks.rados.rados.0.trial115.stdout:2472: done (2 left) 2026-02-17T06:01:07.547 INFO:tasks.rados.rados.0.trial115.stdout:2473: done (1 left) 2026-02-17T06:01:07.547 INFO:tasks.rados.rados.0.trial115.stdout:2474: done (0 left) 2026-02-17T06:01:07.547 INFO:tasks.rados.rados.0.trial115.stdout:2475: snap_remove snap 250 2026-02-17T06:01:08.460 INFO:tasks.rados.rados.0.trial115.stdout:2475: done (0 left) 2026-02-17T06:01:08.460 INFO:tasks.rados.rados.0.trial115.stdout:2476: read oid 36 snap -1 2026-02-17T06:01:08.460 INFO:tasks.rados.rados.0.trial115.stdout:2476: expect (ObjNum 859 snap 246 seq_num 859) 2026-02-17T06:01:08.460 INFO:tasks.rados.rados.0.trial115.stdout:2477: read oid 31 snap -1 2026-02-17T06:01:08.460 INFO:tasks.rados.rados.0.trial115.stdout:2477: expect (ObjNum 804 snap 231 seq_num 804) 2026-02-17T06:01:08.460 INFO:tasks.rados.rados.0.trial115.stdout:2478: write oid 6 current snap is 251 2026-02-17T06:01:08.461 INFO:tasks.rados.rados.0.trial115.stdout:2478: seq_num 873 ranges {672511=556619,1819486=754634} 2026-02-17T06:01:08.465 INFO:tasks.rados.rados.0.trial115.stdout:2478: writing trial11545420-6 from 672511 to 1229130 tid 1 2026-02-17T06:01:08.470 INFO:tasks.rados.rados.0.trial115.stdout:2478: writing trial11545420-6 from 1819486 to 2574120 tid 2 2026-02-17T06:01:08.470 INFO:tasks.rados.rados.0.trial115.stdout:2479: write oid 38 current snap is 251 2026-02-17T06:01:08.471 INFO:tasks.rados.rados.0.trial115.stdout:2479: seq_num 874 ranges {689318=631868,1877968=498292} 2026-02-17T06:01:08.475 INFO:tasks.rados.rados.0.trial115.stdout:2479: writing trial11545420-38 from 689318 to 1321186 tid 1 2026-02-17T06:01:08.480 INFO:tasks.rados.rados.0.trial115.stdout:2479: writing trial11545420-38 from 1877968 to 2376260 tid 2 2026-02-17T06:01:08.480 INFO:tasks.rados.rados.0.trial115.stdout:2480: copy_from oid 4 from oid 13 current snap is 251 2026-02-17T06:01:08.480 INFO:tasks.rados.rados.0.trial115.stdout:2476: expect (ObjNum 859 snap 246 seq_num 859) 2026-02-17T06:01:08.536 INFO:tasks.rados.rados.0.trial115.stdout:2477: expect (ObjNum 804 snap 231 seq_num 804) 2026-02-17T06:01:08.617 INFO:tasks.rados.rados.0.trial115.stdout:2478: finishing write tid 1 to trial11545420-6 2026-02-17T06:01:08.617 INFO:tasks.rados.rados.0.trial115.stdout:2476: done (4 left) 2026-02-17T06:01:08.617 INFO:tasks.rados.rados.0.trial115.stdout:2477: done (3 left) 2026-02-17T06:01:08.617 INFO:tasks.rados.rados.0.trial115.stdout:2481: copy_from oid 11 from oid 42 current snap is 251 2026-02-17T06:01:08.617 INFO:tasks.rados.rados.0.trial115.stdout:2478: finishing write tid 2 to trial11545420-6 2026-02-17T06:01:08.617 INFO:tasks.rados.rados.0.trial115.stdout:2478: finishing write tid 3 to trial11545420-6 2026-02-17T06:01:08.617 INFO:tasks.rados.rados.0.trial115.stdout:2478: finishing write tid 4 to trial11545420-6 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2478: oid 6 updating version 0 to 957 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2478: oid 6 updating version 957 to 958 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2478: oid 6 updating version 958 to 959 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 959 (ObjNum 873 snap 251 seq_num 873) dirty exists 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2478: left oid 6 (ObjNum 873 snap 251 seq_num 873) 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2479: finishing write tid 1 to trial11545420-38 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2479: finishing write tid 2 to trial11545420-38 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2479: finishing write tid 3 to trial11545420-38 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2479: finishing write tid 4 to trial11545420-38 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2479: oid 38 updating version 0 to 1386 2026-02-17T06:01:08.618 INFO:tasks.rados.rados.0.trial115.stdout:2479: oid 38 version 1386 is already newer than 1385 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2479: oid 38 updating version 1386 to 1387 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1387 (ObjNum 874 snap 251 seq_num 874) dirty exists 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2479: left oid 38 (ObjNum 874 snap 251 seq_num 874) 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2478: done (3 left) 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2479: done (2 left) 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2482: read oid 20 snap -1 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2482: expect (ObjNum 867 snap 247 seq_num 867) 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2483: snap_create 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2480: finishing copy_from to trial11545420-4 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2480: got expected ENOENT (src dne) 2026-02-17T06:01:08.619 INFO:tasks.rados.rados.0.trial115.stdout:2480: finishing copy_from racing read to trial11545420-4 2026-02-17T06:01:08.621 INFO:tasks.rados.rados.0.trial115.stdout:2482: expect (ObjNum 867 snap 247 seq_num 867) 2026-02-17T06:01:08.732 INFO:tasks.rados.rados.0.trial115.stdout:2481: finishing copy_from racing read to trial11545420-11 2026-02-17T06:01:08.733 INFO:tasks.rados.rados.0.trial115.stdout:2481: finishing copy_from to trial11545420-11 2026-02-17T06:01:08.733 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 962 (ObjNum 862 snap 246 seq_num 862) dirty exists 2026-02-17T06:01:09.465 INFO:tasks.rados.rados.0.trial115.stdout:2480: done (3 left) 2026-02-17T06:01:09.465 INFO:tasks.rados.rados.0.trial115.stdout:2481: done (2 left) 2026-02-17T06:01:09.465 INFO:tasks.rados.rados.0.trial115.stdout:2482: done (1 left) 2026-02-17T06:01:09.465 INFO:tasks.rados.rados.0.trial115.stdout:2483: done (0 left) 2026-02-17T06:01:09.465 INFO:tasks.rados.rados.0.trial115.stdout:2484: copy_from oid 42 from oid 8 current snap is 252 2026-02-17T06:01:09.466 INFO:tasks.rados.rados.0.trial115.stdout:2485: snap_remove snap 251 2026-02-17T06:01:10.229 INFO:tasks.rados.rados.0.trial115.stdout:2485: done (1 left) 2026-02-17T06:01:10.229 INFO:tasks.rados.rados.0.trial115.stdout:2486: write oid 5 current snap is 252 2026-02-17T06:01:10.229 INFO:tasks.rados.rados.0.trial115.stdout:2486: seq_num 875 ranges {706125=423470,1652803=758303} 2026-02-17T06:01:10.233 INFO:tasks.rados.rados.0.trial115.stdout:2486: writing trial11545420-5 from 706125 to 1129595 tid 1 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:2486: writing trial11545420-5 from 1652803 to 2411106 tid 2 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:2487: read oid 24 snap -1 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:2487: expect (ObjNum 854 snap 244 seq_num 854) 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:2488: write oid 32 current snap is 252 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:2484: finishing copy_from racing read to trial11545420-42 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:2484: finishing copy_from to trial11545420-42 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1093 (ObjNum 800 snap 230 seq_num 800) dirty exists 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:2486: finishing write tid 1 to trial11545420-5 2026-02-17T06:01:10.239 INFO:tasks.rados.rados.0.trial115.stdout:2488: seq_num 876 ranges {722932=498719,1994932=501961} 2026-02-17T06:01:10.243 INFO:tasks.rados.rados.0.trial115.stdout:2488: writing trial11545420-32 from 722932 to 1221651 tid 1 2026-02-17T06:01:10.249 INFO:tasks.rados.rados.0.trial115.stdout:2488: writing trial11545420-32 from 1994932 to 2496893 tid 2 2026-02-17T06:01:10.249 INFO:tasks.rados.rados.0.trial115.stdout:2484: done (3 left) 2026-02-17T06:01:10.249 INFO:tasks.rados.rados.0.trial115.stdout:2489: delete oid 47 current snap is 252 2026-02-17T06:01:10.249 INFO:tasks.rados.rados.0.trial115.stdout:2486: finishing write tid 2 to trial11545420-5 2026-02-17T06:01:10.250 INFO:tasks.rados.rados.0.trial115.stdout:2486: finishing write tid 3 to trial11545420-5 2026-02-17T06:01:10.250 INFO:tasks.rados.rados.0.trial115.stdout:2486: finishing write tid 4 to trial11545420-5 2026-02-17T06:01:10.250 INFO:tasks.rados.rados.0.trial115.stdout:2486: oid 5 updating version 0 to 1178 2026-02-17T06:01:10.250 INFO:tasks.rados.rados.0.trial115.stdout:2486: oid 5 updating version 1178 to 1179 2026-02-17T06:01:10.250 INFO:tasks.rados.rados.0.trial115.stdout:2486: oid 5 version 1179 is already newer than 1177 2026-02-17T06:01:10.250 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1179 (ObjNum 875 snap 252 seq_num 875) dirty exists 2026-02-17T06:01:10.250 INFO:tasks.rados.rados.0.trial115.stdout:2486: left oid 5 (ObjNum 875 snap 252 seq_num 875) 2026-02-17T06:01:10.250 INFO:tasks.rados.rados.0.trial115.stdout:2487: expect (ObjNum 854 snap 244 seq_num 854) 2026-02-17T06:01:10.371 INFO:tasks.rados.rados.0.trial115.stdout:2488: finishing write tid 1 to trial11545420-32 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2486: done (3 left) 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2487: done (2 left) 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2489: done (1 left) 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2490: copy_from oid 7 from oid 40 current snap is 252 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2488: finishing write tid 2 to trial11545420-32 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2488: finishing write tid 3 to trial11545420-32 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2488: finishing write tid 4 to trial11545420-32 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2488: oid 32 updating version 0 to 963 2026-02-17T06:01:10.372 INFO:tasks.rados.rados.0.trial115.stdout:2488: oid 32 updating version 963 to 964 2026-02-17T06:01:10.373 INFO:tasks.rados.rados.0.trial115.stdout:2488: oid 32 updating version 964 to 965 2026-02-17T06:01:10.373 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 965 (ObjNum 876 snap 252 seq_num 876) dirty exists 2026-02-17T06:01:10.373 INFO:tasks.rados.rados.0.trial115.stdout:2488: left oid 32 (ObjNum 876 snap 252 seq_num 876) 2026-02-17T06:01:10.373 INFO:tasks.rados.rados.0.trial115.stdout:2488: done (1 left) 2026-02-17T06:01:10.373 INFO:tasks.rados.rados.0.trial115.stdout:2491: delete oid 15 current snap is 252 2026-02-17T06:01:10.373 INFO:tasks.rados.rados.0.trial115.stdout:2491: done (1 left) 2026-02-17T06:01:10.373 INFO:tasks.rados.rados.0.trial115.stdout:2492: snap_create 2026-02-17T06:01:10.378 INFO:tasks.rados.rados.0.trial115.stdout:2490: finishing copy_from racing read to trial11545420-7 2026-02-17T06:01:10.380 INFO:tasks.rados.rados.0.trial115.stdout:2490: finishing copy_from to trial11545420-7 2026-02-17T06:01:10.380 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1277 (ObjNum 855 snap 244 seq_num 855) dirty exists 2026-02-17T06:01:11.228 INFO:tasks.rados.rados.0.trial115.stdout:2490: done (1 left) 2026-02-17T06:01:11.229 INFO:tasks.rados.rados.0.trial115.stdout:2492: done (0 left) 2026-02-17T06:01:11.229 INFO:tasks.rados.rados.0.trial115.stdout:2493: copy_from oid 40 from oid 26 current snap is 253 2026-02-17T06:01:11.229 INFO:tasks.rados.rados.0.trial115.stdout:2494: write oid 24 current snap is 253 2026-02-17T06:01:11.229 INFO:tasks.rados.rados.0.trial115.stdout:2494: seq_num 879 ranges {773353=724466,2170378=602975} 2026-02-17T06:01:11.234 INFO:tasks.rados.rados.0.trial115.stdout:2494: writing trial11545420-24 from 773353 to 1497819 tid 1 2026-02-17T06:01:11.240 INFO:tasks.rados.rados.0.trial115.stdout:2494: writing trial11545420-24 from 2170378 to 2773353 tid 2 2026-02-17T06:01:11.240 INFO:tasks.rados.rados.0.trial115.stdout:2495: copy_from oid 8 from oid 2 current snap is 253 2026-02-17T06:01:11.240 INFO:tasks.rados.rados.0.trial115.stdout:2493: finishing copy_from to trial11545420-40 2026-02-17T06:01:11.240 INFO:tasks.rados.rados.0.trial115.stdout:2493: got expected ENOENT (src dne) 2026-02-17T06:01:11.240 INFO:tasks.rados.rados.0.trial115.stdout:2493: finishing copy_from racing read to trial11545420-40 2026-02-17T06:01:11.240 INFO:tasks.rados.rados.0.trial115.stdout:2494: finishing write tid 1 to trial11545420-24 2026-02-17T06:01:11.240 INFO:tasks.rados.rados.0.trial115.stdout:2493: done (2 left) 2026-02-17T06:01:11.240 INFO:tasks.rados.rados.0.trial115.stdout:2496: snap_remove snap 252 2026-02-17T06:01:12.231 INFO:tasks.rados.rados.0.trial115.stdout:2496: done (2 left) 2026-02-17T06:01:12.231 INFO:tasks.rados.rados.0.trial115.stdout:2497: read oid 48 snap -1 2026-02-17T06:01:12.231 INFO:tasks.rados.rados.0.trial115.stdout:2497: expect (ObjNum 869 snap 249 seq_num 869) 2026-02-17T06:01:12.231 INFO:tasks.rados.rados.0.trial115.stdout:2498: write (excl) oid 39 current snap is 253 2026-02-17T06:01:12.231 INFO:tasks.rados.rados.0.trial115.stdout:2498: seq_num 880 ranges {790160=799715,2112507=509299} 2026-02-17T06:01:12.237 INFO:tasks.rados.rados.0.trial115.stdout:2498: writing trial11545420-39 from 790160 to 1589875 tid 1 2026-02-17T06:01:12.241 INFO:tasks.rados.rados.0.trial115.stdout:2498: writing trial11545420-39 from 2112507 to 2621806 tid 2 2026-02-17T06:01:12.241 INFO:tasks.rados.rados.0.trial115.stdout:2499: snap_create 2026-02-17T06:01:12.241 INFO:tasks.rados.rados.0.trial115.stdout:2494: finishing write tid 2 to trial11545420-24 2026-02-17T06:01:12.241 INFO:tasks.rados.rados.0.trial115.stdout:2494: finishing write tid 3 to trial11545420-24 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:2494: finishing write tid 4 to trial11545420-24 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:2494: oid 24 updating version 0 to 1377 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:2494: oid 24 updating version 1377 to 1379 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:2494: oid 24 version 1379 is already newer than 1378 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1379 (ObjNum 879 snap 253 seq_num 879) dirty exists 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:2494: left oid 24 (ObjNum 879 snap 253 seq_num 879) 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:2495: finishing copy_from racing read to trial11545420-8 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:2495: finishing copy_from to trial11545420-8 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:01:12.242 INFO:tasks.rados.rados.0.trial115.stdout:2497: expect (ObjNum 869 snap 249 seq_num 869) 2026-02-17T06:01:12.349 INFO:tasks.rados.rados.0.trial115.stdout:2498: finishing write tid 1 to trial11545420-39 2026-02-17T06:01:12.349 INFO:tasks.rados.rados.0.trial115.stdout:2498: finishing write tid 2 to trial11545420-39 2026-02-17T06:01:12.349 INFO:tasks.rados.rados.0.trial115.stdout:2498: finishing write tid 3 to trial11545420-39 2026-02-17T06:01:12.349 INFO:tasks.rados.rados.0.trial115.stdout:2498: finishing write tid 4 to trial11545420-39 2026-02-17T06:01:12.349 INFO:tasks.rados.rados.0.trial115.stdout:2498: oid 39 updating version 0 to 1384 2026-02-17T06:01:12.350 INFO:tasks.rados.rados.0.trial115.stdout:2498: oid 39 version 1384 is already newer than 1380 2026-02-17T06:01:12.350 INFO:tasks.rados.rados.0.trial115.stdout:2498: oid 39 version 1384 is already newer than 1383 2026-02-17T06:01:12.350 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1384 (ObjNum 880 snap 253 seq_num 880) dirty exists 2026-02-17T06:01:12.350 INFO:tasks.rados.rados.0.trial115.stdout:2498: left oid 39 (ObjNum 880 snap 253 seq_num 880) 2026-02-17T06:01:13.232 INFO:tasks.rados.rados.0.trial115.stdout:2494: done (4 left) 2026-02-17T06:01:13.232 INFO:tasks.rados.rados.0.trial115.stdout:2495: done (3 left) 2026-02-17T06:01:13.232 INFO:tasks.rados.rados.0.trial115.stdout:2497: done (2 left) 2026-02-17T06:01:13.232 INFO:tasks.rados.rados.0.trial115.stdout:2498: done (1 left) 2026-02-17T06:01:13.232 INFO:tasks.rados.rados.0.trial115.stdout:2499: done (0 left) 2026-02-17T06:01:13.232 INFO:tasks.rados.rados.0.trial115.stdout:2500: write oid 48 current snap is 254 2026-02-17T06:01:13.233 INFO:tasks.rados.rados.0.trial115.stdout:2500: seq_num 881 ranges {406967=474964,1370989=769310,2677186=129781} 2026-02-17T06:01:13.235 INFO:tasks.rados.rados.0.trial115.stdout:2500: writing trial11545420-48 from 406967 to 881931 tid 1 2026-02-17T06:01:13.240 INFO:tasks.rados.rados.0.trial115.stdout:2500: writing trial11545420-48 from 1370989 to 2140299 tid 2 2026-02-17T06:01:13.243 INFO:tasks.rados.rados.0.trial115.stdout:2500: writing trial11545420-48 from 2677186 to 2806967 tid 3 2026-02-17T06:01:13.243 INFO:tasks.rados.rados.0.trial115.stdout:2501: read oid 23 snap -1 2026-02-17T06:01:13.243 INFO:tasks.rados.rados.0.trial115.stdout:2501: expect (ObjNum 747 snap 204 seq_num 747) 2026-02-17T06:01:13.244 INFO:tasks.rados.rados.0.trial115.stdout:2502: write (excl) oid 11 current snap is 254 2026-02-17T06:01:13.244 INFO:tasks.rados.rados.0.trial115.stdout:2502: seq_num 882 ranges {423774=666566,1545824=512968,2704609=119165} 2026-02-17T06:01:13.247 INFO:tasks.rados.rados.0.trial115.stdout:2502: writing trial11545420-11 from 423774 to 1090340 tid 1 2026-02-17T06:01:13.251 INFO:tasks.rados.rados.0.trial115.stdout:2502: writing trial11545420-11 from 1545824 to 2058792 tid 2 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2502: writing trial11545420-11 from 2704609 to 2823774 tid 3 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2503: copy_from oid 40 from oid 14 current snap is 254 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2500: finishing write tid 1 to trial11545420-48 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2500: finishing write tid 2 to trial11545420-48 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2500: finishing write tid 3 to trial11545420-48 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2500: finishing write tid 4 to trial11545420-48 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2500: finishing write tid 5 to trial11545420-48 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2500: oid 48 updating version 0 to 1390 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2500: oid 48 version 1390 is already newer than 1389 2026-02-17T06:01:13.255 INFO:tasks.rados.rados.0.trial115.stdout:2500: oid 48 updating version 1390 to 1392 2026-02-17T06:01:13.256 INFO:tasks.rados.rados.0.trial115.stdout:2500: oid 48 version 1392 is already newer than 1391 2026-02-17T06:01:13.256 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1392 (ObjNum 881 snap 254 seq_num 881) dirty exists 2026-02-17T06:01:13.256 INFO:tasks.rados.rados.0.trial115.stdout:2500: left oid 48 (ObjNum 881 snap 254 seq_num 881) 2026-02-17T06:01:13.256 INFO:tasks.rados.rados.0.trial115.stdout:2501: expect (ObjNum 747 snap 204 seq_num 747) 2026-02-17T06:01:13.336 INFO:tasks.rados.rados.0.trial115.stdout:2502: finishing write tid 1 to trial11545420-11 2026-02-17T06:01:13.336 INFO:tasks.rados.rados.0.trial115.stdout:2502: finishing write tid 2 to trial11545420-11 2026-02-17T06:01:13.336 INFO:tasks.rados.rados.0.trial115.stdout:2503: finishing copy_from to trial11545420-40 2026-02-17T06:01:13.336 INFO:tasks.rados.rados.0.trial115.stdout:2503: got expected ENOENT (src dne) 2026-02-17T06:01:13.336 INFO:tasks.rados.rados.0.trial115.stdout:2503: finishing copy_from racing read to trial11545420-40 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2502: finishing write tid 3 to trial11545420-11 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2500: done (3 left) 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2501: done (2 left) 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2503: done (1 left) 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2504: copy_from oid 14 from oid 25 current snap is 254 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2502: finishing write tid 4 to trial11545420-11 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2502: finishing write tid 5 to trial11545420-11 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2502: oid 11 updating version 0 to 1295 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2502: oid 11 version 1295 is already newer than 1294 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2502: oid 11 version 1295 is already newer than 1292 2026-02-17T06:01:13.337 INFO:tasks.rados.rados.0.trial115.stdout:2502: oid 11 version 1295 is already newer than 1293 2026-02-17T06:01:13.338 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1295 (ObjNum 882 snap 254 seq_num 882) dirty exists 2026-02-17T06:01:13.338 INFO:tasks.rados.rados.0.trial115.stdout:2502: left oid 11 (ObjNum 882 snap 254 seq_num 882) 2026-02-17T06:01:13.338 INFO:tasks.rados.rados.0.trial115.stdout:2502: done (1 left) 2026-02-17T06:01:13.338 INFO:tasks.rados.rados.0.trial115.stdout:2505: read oid 26 snap -1 2026-02-17T06:01:13.338 INFO:tasks.rados.rados.0.trial115.stdout:2505: expect deleted 2026-02-17T06:01:13.338 INFO:tasks.rados.rados.0.trial115.stdout:2506: read oid 35 snap -1 2026-02-17T06:01:13.338 INFO:tasks.rados.rados.0.trial115.stdout:2506: expect (ObjNum 810 snap 234 seq_num 810) 2026-02-17T06:01:13.338 INFO:tasks.rados.rados.0.trial115.stdout:2507: snap_remove snap 253 2026-02-17T06:01:14.237 INFO:tasks.rados.rados.0.trial115.stdout:2507: done (3 left) 2026-02-17T06:01:14.237 INFO:tasks.rados.rados.0.trial115.stdout:2508: read oid 31 snap -1 2026-02-17T06:01:14.237 INFO:tasks.rados.rados.0.trial115.stdout:2508: expect (ObjNum 804 snap 231 seq_num 804) 2026-02-17T06:01:14.237 INFO:tasks.rados.rados.0.trial115.stdout:2506: expect (ObjNum 810 snap 234 seq_num 810) 2026-02-17T06:01:14.436 INFO:tasks.rados.rados.0.trial115.stdout:2504: finishing copy_from racing read to trial11545420-14 2026-02-17T06:01:14.436 INFO:tasks.rados.rados.0.trial115.stdout:2504: finishing copy_from to trial11545420-14 2026-02-17T06:01:14.436 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1072 (ObjNum 830 snap 236 seq_num 830) dirty exists 2026-02-17T06:01:14.437 INFO:tasks.rados.rados.0.trial115.stdout:2504: done (3 left) 2026-02-17T06:01:14.437 INFO:tasks.rados.rados.0.trial115.stdout:2505: done (2 left) 2026-02-17T06:01:14.437 INFO:tasks.rados.rados.0.trial115.stdout:2506: done (1 left) 2026-02-17T06:01:14.437 INFO:tasks.rados.rados.0.trial115.stdout:2509: write oid 4 current snap is 254 2026-02-17T06:01:14.437 INFO:tasks.rados.rados.0.trial115.stdout:2509: seq_num 883 ranges {440581=741815,1604306=656626,2732032=108549} 2026-02-17T06:01:14.440 INFO:tasks.rados.rados.0.trial115.stdout:2509: writing trial11545420-4 from 440581 to 1182396 tid 1 2026-02-17T06:01:14.444 INFO:tasks.rados.rados.0.trial115.stdout:2509: writing trial11545420-4 from 1604306 to 2260932 tid 2 2026-02-17T06:01:14.447 INFO:tasks.rados.rados.0.trial115.stdout:2509: writing trial11545420-4 from 2732032 to 2840581 tid 3 2026-02-17T06:01:14.447 INFO:tasks.rados.rados.0.trial115.stdout:2510: write oid 13 current snap is 254 2026-02-17T06:01:14.447 INFO:tasks.rados.rados.0.trial115.stdout:2510: seq_num 884 ranges {457388=417064,1546435=516637,2643102=214286} 2026-02-17T06:01:14.450 INFO:tasks.rados.rados.0.trial115.stdout:2510: writing trial11545420-13 from 457388 to 874452 tid 1 2026-02-17T06:01:14.454 INFO:tasks.rados.rados.0.trial115.stdout:2510: writing trial11545420-13 from 1546435 to 2063072 tid 2 2026-02-17T06:01:14.457 INFO:tasks.rados.rados.0.trial115.stdout:2510: writing trial11545420-13 from 2643102 to 2857388 tid 3 2026-02-17T06:01:14.457 INFO:tasks.rados.rados.0.trial115.stdout:2511: write (excl) oid 11 current snap is 254 2026-02-17T06:01:14.458 INFO:tasks.rados.rados.0.trial115.stdout:2511: seq_num 885 ranges {474195=492313,1604917=660295,2670525=203670} 2026-02-17T06:01:14.461 INFO:tasks.rados.rados.0.trial115.stdout:2511: writing trial11545420-11 from 474195 to 966508 tid 1 2026-02-17T06:01:14.466 INFO:tasks.rados.rados.0.trial115.stdout:2511: writing trial11545420-11 from 1604917 to 2265212 tid 2 2026-02-17T06:01:14.469 INFO:tasks.rados.rados.0.trial115.stdout:2511: writing trial11545420-11 from 2670525 to 2874195 tid 3 2026-02-17T06:01:14.469 INFO:tasks.rados.rados.0.trial115.stdout:2512: read oid 33 snap -1 2026-02-17T06:01:14.469 INFO:tasks.rados.rados.0.trial115.stdout:2512: expect (ObjNum 770 snap 218 seq_num 770) 2026-02-17T06:01:14.469 INFO:tasks.rados.rados.0.trial115.stdout:2513: copy_from oid 14 from oid 44 current snap is 254 2026-02-17T06:01:14.469 INFO:tasks.rados.rados.0.trial115.stdout:2508: expect (ObjNum 804 snap 231 seq_num 804) 2026-02-17T06:01:14.549 INFO:tasks.rados.rados.0.trial115.stdout:2509: finishing write tid 1 to trial11545420-4 2026-02-17T06:01:14.549 INFO:tasks.rados.rados.0.trial115.stdout:2508: done (5 left) 2026-02-17T06:01:14.549 INFO:tasks.rados.rados.0.trial115.stdout:2514: write oid 35 current snap is 254 2026-02-17T06:01:14.549 INFO:tasks.rados.rados.0.trial115.stdout:2509: finishing write tid 2 to trial11545420-4 2026-02-17T06:01:14.550 INFO:tasks.rados.rados.0.trial115.stdout:2509: finishing write tid 3 to trial11545420-4 2026-02-17T06:01:14.550 INFO:tasks.rados.rados.0.trial115.stdout:2514: seq_num 886 ranges {491002=567562,1663399=520306,2814301=76701} 2026-02-17T06:01:14.552 INFO:tasks.rados.rados.0.trial115.stdout:2514: writing trial11545420-35 from 491002 to 1058564 tid 1 2026-02-17T06:01:14.557 INFO:tasks.rados.rados.0.trial115.stdout:2514: writing trial11545420-35 from 1663399 to 2183705 tid 2 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:2514: writing trial11545420-35 from 2814301 to 2891002 tid 3 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:2515: write oid 31 current snap is 254 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:2509: finishing write tid 4 to trial11545420-4 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:2509: finishing write tid 5 to trial11545420-4 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:2509: oid 4 updating version 0 to 1279 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:2509: oid 4 updating version 1279 to 1281 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:2509: oid 4 version 1281 is already newer than 1280 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:2509: oid 4 version 1281 is already newer than 1278 2026-02-17T06:01:14.560 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1281 (ObjNum 883 snap 254 seq_num 883) dirty exists 2026-02-17T06:01:14.561 INFO:tasks.rados.rados.0.trial115.stdout:2509: left oid 4 (ObjNum 883 snap 254 seq_num 883) 2026-02-17T06:01:14.561 INFO:tasks.rados.rados.0.trial115.stdout:2510: finishing write tid 1 to trial11545420-13 2026-02-17T06:01:14.561 INFO:tasks.rados.rados.0.trial115.stdout:2515: seq_num 887 ranges {507809=642811,1721881=663964} 2026-02-17T06:01:14.564 INFO:tasks.rados.rados.0.trial115.stdout:2515: writing trial11545420-31 from 507809 to 1150620 tid 1 2026-02-17T06:01:14.569 INFO:tasks.rados.rados.0.trial115.stdout:2515: writing trial11545420-31 from 1721881 to 2385845 tid 2 2026-02-17T06:01:14.569 INFO:tasks.rados.rados.0.trial115.stdout:2509: done (6 left) 2026-02-17T06:01:14.569 INFO:tasks.rados.rados.0.trial115.stdout:2516: rollback oid 16 current snap is 254 2026-02-17T06:01:14.569 INFO:tasks.rados.rados.0.trial115.stdout:2516: done (6 left) 2026-02-17T06:01:14.569 INFO:tasks.rados.rados.0.trial115.stdout:2517: copy_from oid 41 from oid 16 current snap is 254 2026-02-17T06:01:14.569 INFO:tasks.rados.rados.0.trial115.stdout:2510: finishing write tid 2 to trial11545420-13 2026-02-17T06:01:14.569 INFO:tasks.rados.rados.0.trial115.stdout:2510: finishing write tid 3 to trial11545420-13 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2510: finishing write tid 4 to trial11545420-13 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2510: finishing write tid 5 to trial11545420-13 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2510: oid 13 updating version 0 to 660 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2510: oid 13 version 660 is already newer than 659 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2510: oid 13 version 660 is already newer than 658 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2510: oid 13 version 660 is already newer than 657 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 660 (ObjNum 884 snap 254 seq_num 884) dirty exists 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2510: left oid 13 (ObjNum 884 snap 254 seq_num 884) 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2511: finishing write tid 1 to trial11545420-11 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2510: done (6 left) 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2518: read oid 49 snap -1 2026-02-17T06:01:14.570 INFO:tasks.rados.rados.0.trial115.stdout:2518: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:14.571 INFO:tasks.rados.rados.0.trial115.stdout:2519: write (excl) oid 13 current snap is 254 2026-02-17T06:01:14.571 INFO:tasks.rados.rados.0.trial115.stdout:2519: seq_num 888 ranges {524616=718060,1664010=523975,2752794=171822} 2026-02-17T06:01:14.573 INFO:tasks.rados.rados.0.trial115.stdout:2519: writing trial11545420-13 from 524616 to 1242676 tid 1 2026-02-17T06:01:14.576 INFO:tasks.rados.rados.0.trial115.stdout:2519: writing trial11545420-13 from 1664010 to 2187985 tid 2 2026-02-17T06:01:14.580 INFO:tasks.rados.rados.0.trial115.stdout:2519: writing trial11545420-13 from 2752794 to 2924616 tid 3 2026-02-17T06:01:14.580 INFO:tasks.rados.rados.0.trial115.stdout:2520: read oid 40 snap -1 2026-02-17T06:01:14.580 INFO:tasks.rados.rados.0.trial115.stdout:2520: expect (ObjNum 855 snap 244 seq_num 855) 2026-02-17T06:01:14.580 INFO:tasks.rados.rados.0.trial115.stdout:2521: write (excl) oid 46 current snap is 254 2026-02-17T06:01:14.580 INFO:tasks.rados.rados.0.trial115.stdout:2511: finishing write tid 2 to trial11545420-11 2026-02-17T06:01:14.580 INFO:tasks.rados.rados.0.trial115.stdout:2511: finishing write tid 3 to trial11545420-11 2026-02-17T06:01:14.580 INFO:tasks.rados.rados.0.trial115.stdout:2521: seq_num 889 ranges {541423=793309,2122492=667633} 2026-02-17T06:01:14.584 INFO:tasks.rados.rados.0.trial115.stdout:2521: writing trial11545420-46 from 541423 to 1334732 tid 1 2026-02-17T06:01:14.590 INFO:tasks.rados.rados.0.trial115.stdout:2521: writing trial11545420-46 from 2122492 to 2790125 tid 2 2026-02-17T06:01:14.590 INFO:tasks.rados.rados.0.trial115.stdout:2522: delete oid 4 current snap is 254 2026-02-17T06:01:14.590 INFO:tasks.rados.rados.0.trial115.stdout:2511: finishing write tid 4 to trial11545420-11 2026-02-17T06:01:14.590 INFO:tasks.rados.rados.0.trial115.stdout:2511: finishing write tid 5 to trial11545420-11 2026-02-17T06:01:14.591 INFO:tasks.rados.rados.0.trial115.stdout:2511: oid 11 updating version 0 to 1299 2026-02-17T06:01:14.591 INFO:tasks.rados.rados.0.trial115.stdout:2511: oid 11 updating version 1299 to 1300 2026-02-17T06:01:14.591 INFO:tasks.rados.rados.0.trial115.stdout:2511: oid 11 version 1300 is already newer than 1298 2026-02-17T06:01:14.591 INFO:tasks.rados.rados.0.trial115.stdout:2511: oid 11 updating version 1300 to 1301 2026-02-17T06:01:14.591 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1301 (ObjNum 885 snap 254 seq_num 885) dirty exists 2026-02-17T06:01:14.591 INFO:tasks.rados.rados.0.trial115.stdout:2511: left oid 11 (ObjNum 885 snap 254 seq_num 885) 2026-02-17T06:01:14.591 INFO:tasks.rados.rados.0.trial115.stdout:2512: expect (ObjNum 770 snap 218 seq_num 770) 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:2513: finishing copy_from racing read to trial11545420-14 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:2513: finishing copy_from to trial11545420-14 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 962 (ObjNum 862 snap 246 seq_num 862) dirty exists 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:2511: done (10 left) 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:2512: done (9 left) 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:2513: done (8 left) 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:2522: done (7 left) 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:2523: read oid 38 snap -1 2026-02-17T06:01:14.662 INFO:tasks.rados.rados.0.trial115.stdout:2523: expect (ObjNum 874 snap 251 seq_num 874) 2026-02-17T06:01:14.663 INFO:tasks.rados.rados.0.trial115.stdout:2524: write oid 6 current snap is 254 2026-02-17T06:01:14.663 INFO:tasks.rados.rados.0.trial115.stdout:2524: seq_num 891 ranges {575037=660160,1955809=671302} 2026-02-17T06:01:14.666 INFO:tasks.rados.rados.0.trial115.stdout:2524: writing trial11545420-6 from 575037 to 1235197 tid 1 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2524: writing trial11545420-6 from 1955809 to 2627111 tid 2 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2525: snap_create 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2514: finishing write tid 1 to trial11545420-35 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2514: finishing write tid 2 to trial11545420-35 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2514: finishing write tid 3 to trial11545420-35 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2514: finishing write tid 4 to trial11545420-35 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2514: finishing write tid 5 to trial11545420-35 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2514: oid 35 updating version 0 to 1397 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2514: oid 35 updating version 1397 to 1398 2026-02-17T06:01:14.672 INFO:tasks.rados.rados.0.trial115.stdout:2514: oid 35 version 1398 is already newer than 1396 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2514: oid 35 version 1398 is already newer than 1395 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1398 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2514: left oid 35 (ObjNum 886 snap 254 seq_num 886) 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2515: finishing write tid 1 to trial11545420-31 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2515: finishing write tid 2 to trial11545420-31 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2515: finishing write tid 3 to trial11545420-31 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2515: finishing write tid 4 to trial11545420-31 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2515: oid 31 updating version 0 to 1183 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2515: oid 31 version 1183 is already newer than 1182 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2515: oid 31 version 1183 is already newer than 1181 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1183 (ObjNum 887 snap 254 seq_num 887) dirty exists 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2515: left oid 31 (ObjNum 887 snap 254 seq_num 887) 2026-02-17T06:01:14.673 INFO:tasks.rados.rados.0.trial115.stdout:2517: finishing copy_from racing read to trial11545420-41 2026-02-17T06:01:14.674 INFO:tasks.rados.rados.0.trial115.stdout:2518: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:14.717 INFO:tasks.rados.rados.0.trial115.stdout:2517: finishing copy_from to trial11545420-41 2026-02-17T06:01:14.717 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1377 (ObjNum 868 snap 248 seq_num 868) dirty exists 2026-02-17T06:01:14.717 INFO:tasks.rados.rados.0.trial115.stdout:2519: finishing write tid 1 to trial11545420-13 2026-02-17T06:01:14.717 INFO:tasks.rados.rados.0.trial115.stdout:2519: finishing write tid 2 to trial11545420-13 2026-02-17T06:01:14.717 INFO:tasks.rados.rados.0.trial115.stdout:2519: finishing write tid 3 to trial11545420-13 2026-02-17T06:01:14.717 INFO:tasks.rados.rados.0.trial115.stdout:2519: finishing write tid 4 to trial11545420-13 2026-02-17T06:01:14.718 INFO:tasks.rados.rados.0.trial115.stdout:2519: finishing write tid 5 to trial11545420-13 2026-02-17T06:01:14.718 INFO:tasks.rados.rados.0.trial115.stdout:2519: oid 13 updating version 0 to 663 2026-02-17T06:01:14.718 INFO:tasks.rados.rados.0.trial115.stdout:2519: oid 13 updating version 663 to 664 2026-02-17T06:01:14.718 INFO:tasks.rados.rados.0.trial115.stdout:2519: oid 13 version 664 is already newer than 661 2026-02-17T06:01:14.718 INFO:tasks.rados.rados.0.trial115.stdout:2519: oid 13 version 664 is already newer than 662 2026-02-17T06:01:14.718 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 664 (ObjNum 888 snap 254 seq_num 888) dirty exists 2026-02-17T06:01:14.718 INFO:tasks.rados.rados.0.trial115.stdout:2519: left oid 13 (ObjNum 888 snap 254 seq_num 888) 2026-02-17T06:01:14.718 INFO:tasks.rados.rados.0.trial115.stdout:2520: expect (ObjNum 855 snap 244 seq_num 855) 2026-02-17T06:01:14.773 INFO:tasks.rados.rados.0.trial115.stdout:2521: finishing write tid 1 to trial11545420-46 2026-02-17T06:01:14.773 INFO:tasks.rados.rados.0.trial115.stdout:2521: finishing write tid 2 to trial11545420-46 2026-02-17T06:01:14.774 INFO:tasks.rados.rados.0.trial115.stdout:2521: finishing write tid 3 to trial11545420-46 2026-02-17T06:01:14.774 INFO:tasks.rados.rados.0.trial115.stdout:2521: finishing write tid 4 to trial11545420-46 2026-02-17T06:01:14.774 INFO:tasks.rados.rados.0.trial115.stdout:2521: oid 46 updating version 0 to 1303 2026-02-17T06:01:14.774 INFO:tasks.rados.rados.0.trial115.stdout:2521: oid 46 version 1303 is already newer than 1302 2026-02-17T06:01:14.774 INFO:tasks.rados.rados.0.trial115.stdout:2521: oid 46 updating version 1303 to 1304 2026-02-17T06:01:14.774 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1304 (ObjNum 889 snap 254 seq_num 889) dirty exists 2026-02-17T06:01:14.774 INFO:tasks.rados.rados.0.trial115.stdout:2521: left oid 46 (ObjNum 889 snap 254 seq_num 889) 2026-02-17T06:01:14.774 INFO:tasks.rados.rados.0.trial115.stdout:2523: expect (ObjNum 874 snap 251 seq_num 874) 2026-02-17T06:01:14.898 INFO:tasks.rados.rados.0.trial115.stdout:2524: finishing write tid 1 to trial11545420-6 2026-02-17T06:01:14.898 INFO:tasks.rados.rados.0.trial115.stdout:2524: finishing write tid 2 to trial11545420-6 2026-02-17T06:01:14.898 INFO:tasks.rados.rados.0.trial115.stdout:2524: finishing write tid 3 to trial11545420-6 2026-02-17T06:01:14.898 INFO:tasks.rados.rados.0.trial115.stdout:2524: finishing write tid 4 to trial11545420-6 2026-02-17T06:01:14.898 INFO:tasks.rados.rados.0.trial115.stdout:2524: oid 6 updating version 0 to 960 2026-02-17T06:01:14.899 INFO:tasks.rados.rados.0.trial115.stdout:2524: oid 6 updating version 960 to 962 2026-02-17T06:01:14.899 INFO:tasks.rados.rados.0.trial115.stdout:2524: oid 6 version 962 is already newer than 961 2026-02-17T06:01:14.899 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 962 (ObjNum 891 snap 254 seq_num 891) dirty exists 2026-02-17T06:01:14.899 INFO:tasks.rados.rados.0.trial115.stdout:2524: left oid 6 (ObjNum 891 snap 254 seq_num 891) 2026-02-17T06:01:15.230 INFO:tasks.rados.rados.0.trial115.stdout:2514: done (9 left) 2026-02-17T06:01:15.230 INFO:tasks.rados.rados.0.trial115.stdout:2515: done (8 left) 2026-02-17T06:01:15.230 INFO:tasks.rados.rados.0.trial115.stdout:2517: done (7 left) 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2518: done (6 left) 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2519: done (5 left) 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2520: done (4 left) 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2521: done (3 left) 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2523: done (2 left) 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2524: done (1 left) 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2525: done (0 left) 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2526: write (excl) oid 48 current snap is 255 2026-02-17T06:01:15.231 INFO:tasks.rados.rados.0.trial115.stdout:2526: seq_num 892 ranges {591844=735409,1897938=531313,2862486=129358} 2026-02-17T06:01:15.235 INFO:tasks.rados.rados.0.trial115.stdout:2526: writing trial11545420-48 from 591844 to 1327253 tid 1 2026-02-17T06:01:15.239 INFO:tasks.rados.rados.0.trial115.stdout:2526: writing trial11545420-48 from 1897938 to 2429251 tid 2 2026-02-17T06:01:15.242 INFO:tasks.rados.rados.0.trial115.stdout:2526: writing trial11545420-48 from 2862486 to 2991844 tid 3 2026-02-17T06:01:15.242 INFO:tasks.rados.rados.0.trial115.stdout:2527: rollback oid 9 current snap is 255 2026-02-17T06:01:15.242 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 9 to 254 2026-02-17T06:01:15.242 INFO:tasks.rados.rados.0.trial115.stdout:2528: rollback oid 41 current snap is 255 2026-02-17T06:01:15.242 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 254 2026-02-17T06:01:15.243 INFO:tasks.rados.rados.0.trial115.stdout:2529: delete oid 26 current snap is 255 2026-02-17T06:01:15.244 INFO:tasks.rados.rados.0.trial115.stdout:2529: done (3 left) 2026-02-17T06:01:15.244 INFO:tasks.rados.rados.0.trial115.stdout:2530: snap_create 2026-02-17T06:01:15.244 INFO:tasks.rados.rados.0.trial115.stdout:2526: finishing write tid 1 to trial11545420-48 2026-02-17T06:01:15.244 INFO:tasks.rados.rados.0.trial115.stdout:2526: finishing write tid 2 to trial11545420-48 2026-02-17T06:01:15.244 INFO:tasks.rados.rados.0.trial115.stdout:2526: finishing write tid 3 to trial11545420-48 2026-02-17T06:01:15.244 INFO:tasks.rados.rados.0.trial115.stdout:2526: finishing write tid 4 to trial11545420-48 2026-02-17T06:01:15.244 INFO:tasks.rados.rados.0.trial115.stdout:2526: finishing write tid 5 to trial11545420-48 2026-02-17T06:01:15.245 INFO:tasks.rados.rados.0.trial115.stdout:2526: oid 48 updating version 0 to 1403 2026-02-17T06:01:15.245 INFO:tasks.rados.rados.0.trial115.stdout:2526: oid 48 version 1403 is already newer than 1402 2026-02-17T06:01:15.245 INFO:tasks.rados.rados.0.trial115.stdout:2526: oid 48 version 1403 is already newer than 1401 2026-02-17T06:01:15.245 INFO:tasks.rados.rados.0.trial115.stdout:2526: oid 48 version 1403 is already newer than 1400 2026-02-17T06:01:15.245 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1403 (ObjNum 892 snap 255 seq_num 892) dirty exists 2026-02-17T06:01:15.245 INFO:tasks.rados.rados.0.trial115.stdout:2526: left oid 48 (ObjNum 892 snap 255 seq_num 892) 2026-02-17T06:01:15.246 INFO:tasks.rados.rados.0.trial115.stdout:2528: finishing rollback tid 0 to trial11545420-41 2026-02-17T06:01:15.246 INFO:tasks.rados.rados.0.trial115.stdout:2528: finishing rollback tid 1 to trial11545420-41 2026-02-17T06:01:15.246 INFO:tasks.rados.rados.0.trial115.stdout:2528: finishing rollback tid 2 to trial11545420-41 2026-02-17T06:01:15.246 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1380 (ObjNum 868 snap 248 seq_num 868) dirty exists 2026-02-17T06:01:15.246 INFO:tasks.rados.rados.0.trial115.stdout:2527: finishing rollback tid 0 to trial11545420-9 2026-02-17T06:01:15.246 INFO:tasks.rados.rados.0.trial115.stdout:2527: finishing rollback tid 1 to trial11545420-9 2026-02-17T06:01:15.246 INFO:tasks.rados.rados.0.trial115.stdout:2527: finishing rollback tid 2 to trial11545420-9 2026-02-17T06:01:15.247 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1308 (ObjNum 759 snap 207 seq_num 759) dirty exists 2026-02-17T06:01:16.241 INFO:tasks.rados.rados.0.trial115.stdout:2526: done (3 left) 2026-02-17T06:01:16.241 INFO:tasks.rados.rados.0.trial115.stdout:2527: done (2 left) 2026-02-17T06:01:16.242 INFO:tasks.rados.rados.0.trial115.stdout:2528: done (1 left) 2026-02-17T06:01:16.242 INFO:tasks.rados.rados.0.trial115.stdout:2530: done (0 left) 2026-02-17T06:01:16.242 INFO:tasks.rados.rados.0.trial115.stdout:2531: snap_create 2026-02-17T06:01:17.242 INFO:tasks.rados.rados.0.trial115.stdout:2531: done (0 left) 2026-02-17T06:01:17.243 INFO:tasks.rados.rados.0.trial115.stdout:2532: read oid 6 snap -1 2026-02-17T06:01:17.243 INFO:tasks.rados.rados.0.trial115.stdout:2532: expect (ObjNum 891 snap 254 seq_num 891) 2026-02-17T06:01:17.243 INFO:tasks.rados.rados.0.trial115.stdout:2533: copy_from oid 5 from oid 16 current snap is 257 2026-02-17T06:01:17.243 INFO:tasks.rados.rados.0.trial115.stdout:2534: read oid 8 snap -1 2026-02-17T06:01:17.243 INFO:tasks.rados.rados.0.trial115.stdout:2534: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:17.243 INFO:tasks.rados.rados.0.trial115.stdout:2535: read oid 49 snap 254 2026-02-17T06:01:17.243 INFO:tasks.rados.rados.0.trial115.stdout:2535: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:17.243 INFO:tasks.rados.rados.0.trial115.stdout:2536: snap_remove snap 255 2026-02-17T06:01:18.245 INFO:tasks.rados.rados.0.trial115.stdout:2536: done (4 left) 2026-02-17T06:01:18.245 INFO:tasks.rados.rados.0.trial115.stdout:2537: write (excl) oid 15 current snap is 257 2026-02-17T06:01:18.246 INFO:tasks.rados.rados.0.trial115.stdout:2537: seq_num 894 ranges {625458=485907,1614902=534982,2917332=108126} 2026-02-17T06:01:18.249 INFO:tasks.rados.rados.0.trial115.stdout:2537: writing trial11545420-15 from 625458 to 1111365 tid 1 2026-02-17T06:01:18.254 INFO:tasks.rados.rados.0.trial115.stdout:2537: writing trial11545420-15 from 1614902 to 2149884 tid 2 2026-02-17T06:01:18.258 INFO:tasks.rados.rados.0.trial115.stdout:2537: writing trial11545420-15 from 2917332 to 3025458 tid 3 2026-02-17T06:01:18.258 INFO:tasks.rados.rados.0.trial115.stdout:2538: snap_remove snap 256 2026-02-17T06:01:19.247 INFO:tasks.rados.rados.0.trial115.stdout:2538: done (5 left) 2026-02-17T06:01:19.247 INFO:tasks.rados.rados.0.trial115.stdout:2539: read oid 28 snap -1 2026-02-17T06:01:19.247 INFO:tasks.rados.rados.0.trial115.stdout:2539: expect (ObjNum 845 snap 241 seq_num 845) 2026-02-17T06:01:19.247 INFO:tasks.rados.rados.0.trial115.stdout:2540: copy_from oid 27 from oid 17 current snap is 257 2026-02-17T06:01:19.247 INFO:tasks.rados.rados.0.trial115.stdout:2535: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:19.295 INFO:tasks.rados.rados.0.trial115.stdout:2534: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:19.341 INFO:tasks.rados.rados.0.trial115.stdout:2532: expect (ObjNum 891 snap 254 seq_num 891) 2026-02-17T06:01:19.487 INFO:tasks.rados.rados.0.trial115.stdout:2533: finishing copy_from racing read to trial11545420-5 2026-02-17T06:01:19.487 INFO:tasks.rados.rados.0.trial115.stdout:2533: finishing copy_from to trial11545420-5 2026-02-17T06:01:19.487 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1377 (ObjNum 868 snap 248 seq_num 868) dirty exists 2026-02-17T06:01:19.487 INFO:tasks.rados.rados.0.trial115.stdout:2537: finishing write tid 1 to trial11545420-15 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2537: finishing write tid 2 to trial11545420-15 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2532: done (6 left) 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2533: done (5 left) 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2534: done (4 left) 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2535: done (3 left) 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2541: copy_from oid 12 from oid 33 current snap is 257 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2537: finishing write tid 3 to trial11545420-15 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2537: finishing write tid 4 to trial11545420-15 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2537: finishing write tid 5 to trial11545420-15 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2537: oid 15 updating version 0 to 1309 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2537: oid 15 updating version 1309 to 1311 2026-02-17T06:01:19.488 INFO:tasks.rados.rados.0.trial115.stdout:2537: oid 15 version 1311 is already newer than 1310 2026-02-17T06:01:19.489 INFO:tasks.rados.rados.0.trial115.stdout:2537: oid 15 updating version 1311 to 1312 2026-02-17T06:01:19.489 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1312 (ObjNum 894 snap 257 seq_num 894) dirty exists 2026-02-17T06:01:19.489 INFO:tasks.rados.rados.0.trial115.stdout:2537: left oid 15 (ObjNum 894 snap 257 seq_num 894) 2026-02-17T06:01:19.489 INFO:tasks.rados.rados.0.trial115.stdout:2539: expect (ObjNum 845 snap 241 seq_num 845) 2026-02-17T06:01:19.566 INFO:tasks.rados.rados.0.trial115.stdout:2540: finishing copy_from to trial11545420-27 2026-02-17T06:01:19.566 INFO:tasks.rados.rados.0.trial115.stdout:2540: got expected ENOENT (src dne) 2026-02-17T06:01:19.566 INFO:tasks.rados.rados.0.trial115.stdout:2540: finishing copy_from racing read to trial11545420-27 2026-02-17T06:01:19.566 INFO:tasks.rados.rados.0.trial115.stdout:2541: finishing copy_from racing read to trial11545420-12 2026-02-17T06:01:19.566 INFO:tasks.rados.rados.0.trial115.stdout:2541: finishing copy_from to trial11545420-12 2026-02-17T06:01:19.567 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1076 (ObjNum 770 snap 218 seq_num 770) dirty exists 2026-02-17T06:01:19.567 INFO:tasks.rados.rados.0.trial115.stdout:2537: done (3 left) 2026-02-17T06:01:19.567 INFO:tasks.rados.rados.0.trial115.stdout:2539: done (2 left) 2026-02-17T06:01:19.567 INFO:tasks.rados.rados.0.trial115.stdout:2540: done (1 left) 2026-02-17T06:01:19.567 INFO:tasks.rados.rados.0.trial115.stdout:2541: done (0 left) 2026-02-17T06:01:19.567 INFO:tasks.rados.rados.0.trial115.stdout:2542: snap_remove snap 254 2026-02-17T06:01:20.229 INFO:tasks.rados.rados.0.trial115.stdout:2542: done (0 left) 2026-02-17T06:01:20.229 INFO:tasks.rados.rados.0.trial115.stdout:2543: snap_create 2026-02-17T06:01:21.230 INFO:tasks.rados.rados.0.trial115.stdout:2543: done (0 left) 2026-02-17T06:01:21.231 INFO:tasks.rados.rados.0.trial115.stdout:2544: snap_remove snap 257 2026-02-17T06:01:22.230 INFO:tasks.rados.rados.0.trial115.stdout:2544: done (0 left) 2026-02-17T06:01:22.231 INFO:tasks.rados.rados.0.trial115.stdout:2545: snap_create 2026-02-17T06:01:23.231 INFO:tasks.rados.rados.0.trial115.stdout:2545: done (0 left) 2026-02-17T06:01:23.232 INFO:tasks.rados.rados.0.trial115.stdout:2546: rollback oid 19 current snap is 259 2026-02-17T06:01:23.232 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 19 to 258 2026-02-17T06:01:23.232 INFO:tasks.rados.rados.0.trial115.stdout:2547: read oid 49 snap -1 2026-02-17T06:01:23.232 INFO:tasks.rados.rados.0.trial115.stdout:2547: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:23.232 INFO:tasks.rados.rados.0.trial115.stdout:2548: write oid 44 current snap is 259 2026-02-17T06:01:23.232 INFO:tasks.rados.rados.0.trial115.stdout:2548: seq_num 895 ranges {642265=561156,1673384=678640,2828402=213863} 2026-02-17T06:01:23.236 INFO:tasks.rados.rados.0.trial115.stdout:2548: writing trial11545420-44 from 642265 to 1203421 tid 1 2026-02-17T06:01:23.240 INFO:tasks.rados.rados.0.trial115.stdout:2548: writing trial11545420-44 from 1673384 to 2352024 tid 2 2026-02-17T06:01:23.243 INFO:tasks.rados.rados.0.trial115.stdout:2548: writing trial11545420-44 from 2828402 to 3042265 tid 3 2026-02-17T06:01:23.243 INFO:tasks.rados.rados.0.trial115.stdout:2549: copy_from oid 21 from oid 20 current snap is 259 2026-02-17T06:01:23.244 INFO:tasks.rados.rados.0.trial115.stdout:2546: finishing rollback tid 0 to trial11545420-19 2026-02-17T06:01:23.244 INFO:tasks.rados.rados.0.trial115.stdout:2546: finishing rollback tid 1 to trial11545420-19 2026-02-17T06:01:23.244 INFO:tasks.rados.rados.0.trial115.stdout:2546: finishing rollback tid 2 to trial11545420-19 2026-02-17T06:01:23.244 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1409 (ObjNum 790 snap 224 seq_num 790) dirty exists 2026-02-17T06:01:23.244 INFO:tasks.rados.rados.0.trial115.stdout:2547: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:23.290 INFO:tasks.rados.rados.0.trial115.stdout:2548: finishing write tid 1 to trial11545420-44 2026-02-17T06:01:23.290 INFO:tasks.rados.rados.0.trial115.stdout:2548: finishing write tid 2 to trial11545420-44 2026-02-17T06:01:23.290 INFO:tasks.rados.rados.0.trial115.stdout:2548: finishing write tid 3 to trial11545420-44 2026-02-17T06:01:23.290 INFO:tasks.rados.rados.0.trial115.stdout:2548: finishing write tid 4 to trial11545420-44 2026-02-17T06:01:23.290 INFO:tasks.rados.rados.0.trial115.stdout:2548: finishing write tid 5 to trial11545420-44 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2548: oid 44 updating version 0 to 969 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2548: oid 44 version 969 is already newer than 966 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2548: oid 44 version 969 is already newer than 968 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2548: oid 44 version 969 is already newer than 967 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 969 (ObjNum 895 snap 259 seq_num 895) dirty exists 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2548: left oid 44 (ObjNum 895 snap 259 seq_num 895) 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2549: finishing copy_from racing read to trial11545420-21 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2546: done (3 left) 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2547: done (2 left) 2026-02-17T06:01:23.291 INFO:tasks.rados.rados.0.trial115.stdout:2548: done (1 left) 2026-02-17T06:01:23.292 INFO:tasks.rados.rados.0.trial115.stdout:2550: snap_remove snap 258 2026-02-17T06:01:24.236 INFO:tasks.rados.rados.0.trial115.stdout:2550: done (1 left) 2026-02-17T06:01:24.236 INFO:tasks.rados.rados.0.trial115.stdout:2551: snap_create 2026-02-17T06:01:24.236 INFO:tasks.rados.rados.0.trial115.stdout:2549: finishing copy_from to trial11545420-21 2026-02-17T06:01:24.236 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 990 (ObjNum 867 snap 247 seq_num 867) dirty exists 2026-02-17T06:01:24.541 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:01:24.541 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T06:01:24.542 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T06:01:24.542 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T06:01:24.545 INFO:tasks.ceph.mon.b:Started 2026-02-17T06:01:24.545 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:01:24.545 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:01:24.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.612+0000 7f70d3d90640 10 monclient: get_monmap_and_config 2026-02-17T06:01:24.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.612+0000 7f70d3d90640 10 monclient: build_initial_monmap 2026-02-17T06:01:24.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.612+0000 7f70d3d90640 10 monclient: monmap: 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:24.613736+0000 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:24.613736+0000 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:24.614 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:24.615 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.612+0000 7f70d3d90640 1 Processor -- start 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.612+0000 7f70d3d90640 1 -- start start 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.612+0000 7f70d3d90640 10 monclient: init 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f70cc06c890 0x7f70cc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient(hunting): picked mon.noname-a con 0x7f70cc06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f70cc09c4a0 0x7f70cc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient(hunting): picked mon.noname-c con 0x7f70cc09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f70cc134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70cc06daf0 con 0x7f70cc134df0 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f70cc0a0880 con 0x7f70cc06c890 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f70cc06d6f0 con 0x7f70cc09c4a0 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d3d90640 10 monclient(hunting): authenticate will time out at 1530.167287s 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f70cc06c890 0x7f70cc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f70cc06c890 0x7f70cc06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:60182/0 (socket says 10.20.193.78:60182) 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 1 -- 10.20.193.78:0/1564160442 learned_addr learned my addr 10.20.193.78:0/1564160442 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient(hunting): get_auth_request con 0x7f70cc06c890 auth_method 0 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1304640 1 --2- 10.20.193.78:0/1564160442 >> v2:10.20.193.115:3300/0 conn(0x7f70cc09c4a0 0x7f70cc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient(hunting): handle_auth_done global_id 184185 payload 274 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 1 -- 10.20.193.78:0/1564160442 >> v2:10.20.193.115:3300/0 conn(0x7f70cc09c4a0 msgr2=0x7f70cc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 1 --2- 10.20.193.78:0/1564160442 >> v2:10.20.193.115:3300/0 conn(0x7f70cc09c4a0 0x7f70cc06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 1 -- 10.20.193.78:0/1564160442 >> v1:10.20.193.78:6789/0 conn(0x7f70cc134df0 legacy=0x7f70cc135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient: _finish_hunting 0 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 1 monclient: found mon.noname-a 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.613+0000 7f70d1b05640 1 -- 10.20.193.78:0/1564160442 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc09cc80 con 0x7f70cc06c890 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d1b05640 1 -- 10.20.193.78:0/1564160442 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc06c890 msgr2=0x7f70cc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d1b05640 1 --2- 10.20.193.78:0/1564160442 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc06c890 0x7f70cc06cc70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70cc09c980 tx=0x7f70bc02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb461a4a01ba44ad server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 1 -- 10.20.193.78:0/1564160442 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc03c070 con 0x7f70cc06c890 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: dump: 2026-02-17T06:01:24.627 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: _finish_auth 0 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: _check_auth_tickets 2026-02-17T06:01:24.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 1 -- 10.20.193.78:0/1564160442 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70bc02fd80 con 0x7f70cc06c890 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 5 monclient: authenticate success, global_id 184185 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 1 -- 10.20.193.78:0/1564160442 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc037660 con 0x7f70cc06c890 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d0b03640 10 monclient: dump: 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:24.629 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 4 set_mon_vals no callback set 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 10 monclient: shutdown 2026-02-17T06:01:24.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 1 -- 10.20.193.78:0/1564160442 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc06c890 msgr2=0x7f70cc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 1 --2- 10.20.193.78:0/1564160442 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc06c890 0x7f70cc06cc70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70cc09c980 tx=0x7f70bc02ee60 comp rx=0 tx=0).stop 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 1 -- 10.20.193.78:0/1564160442 shutdown_connections 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 1 --2- 10.20.193.78:0/1564160442 >> v2:10.20.193.115:3300/0 conn(0x7f70cc09c4a0 0x7f70cc06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 1 --2- 10.20.193.78:0/1564160442 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc06c890 0x7f70cc06cc70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 1 -- 10.20.193.78:0/1564160442 >> 10.20.193.78:0/1564160442 conn(0x7f70cc0a6620 msgr2=0x7f70cc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 1 -- 10.20.193.78:0/1564160442 shutdown_connections 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 1 -- 10.20.193.78:0/1564160442 wait complete. 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 10 monclient: build_initial_monmap 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.616+0000 7f70d3d90640 10 monclient: monmap: 2026-02-17T06:01:24.631 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 1 Processor -- start 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 1 -- start start 2026-02-17T06:01:24.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient: init 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f70cc06c890 0x7f70cc1cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient(hunting): picked mon.noname-b con 0x7f70cc06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f70cc09c4a0 0x7f70cc1cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient(hunting): picked mon.noname-c con 0x7f70cc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc134df0 0x7f70cc09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient(hunting): picked mon.noname-a con 0x7f70cc134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc0a01a0 con 0x7f70cc134df0 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc0a0020 con 0x7f70cc06c890 2026-02-17T06:01:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f70cc09be20 con 0x7f70cc09c4a0 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d3d90640 10 monclient(hunting): authenticate will time out at 1530.171454s 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f70cc09c4a0 0x7f70cc1cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f70cc09c4a0 0x7f70cc1cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46424/0 (socket says 10.20.193.78:46424) 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 -- 10.20.193.78:0/4121593204 learned_addr learned my addr 10.20.193.78:0/4121593204 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient(hunting): get_auth_request con 0x7f70cc09c4a0 auth_method 0 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc134df0 0x7f70cc09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 10 monclient(hunting): get_auth_request con 0x7f70cc134df0 auth_method 0 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1b05640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f70cc06c890 0x7f70cc1cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1b05640 10 monclient(hunting): get_auth_request con 0x7f70cc06c890 auth_method 0 2026-02-17T06:01:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1b05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1b05640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1b05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient(hunting): handle_auth_done global_id 144572 payload 824 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 -- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f70cc06c890 msgr2=0x7f70cc1cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f70cc06c890 0x7f70cc1cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 -- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc134df0 msgr2=0x7f70cc09db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc134df0 0x7f70cc09db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient: _finish_hunting 0 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 monclient: found mon.noname-c 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d1304640 1 -- 10.20.193.78:0/4121593204 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc09d810 con 0x7f70cc09c4a0 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.617+0000 7f70d2306640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc134df0 0x7f70cc09db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70d1304640 1 -- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f70cc09c4a0 msgr2=0x7f70cc1cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70d1304640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f70cc09c4a0 0x7f70cc1cfc70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c7f0 tx=0x7f70c000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 1 -- 10.20.193.78:0/4121593204 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0007ca0 con 0x7f70cc09c4a0 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: dump: 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:24.637 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 1 -- 10.20.193.78:0/4121593204 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c0004020 con 0x7f70cc09c4a0 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70d3d90640 5 monclient: authenticate success, global_id 144572 2026-02-17T06:01:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70d3d90640 10 monclient: _renew_subs 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70d3d90640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70cc09f7b0 con 0x7f70cc09c4a0 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 1 -- 10.20.193.78:0/4121593204 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0007e60 con 0x7f70cc09c4a0 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70bbfff640 4 set_mon_vals no callback set 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70ba7fc640 10 monclient: dump: 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:24.639 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.619+0000 7f70d3d90640 10 monclient: _renew_subs 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70d3d90640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f70cc126060 con 0x7f70cc09c4a0 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70d3d90640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70d3d90640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70d3d90640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70cc074980 con 0x7f70cc09c4a0 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70ba7fc640 1 -- 10.20.193.78:0/4121593204 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f70c0004410 con 0x7f70cc09c4a0 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70ba7fc640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f709c03dee0 0x7f709c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.620+0000 7f70ba7fc640 1 -- 10.20.193.78:0/4121593204 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(549..549 src has 335..549) ==== 4763+0+0 (secure 0 0 0) 0x7f70c0052eb0 con 0x7f70cc09c4a0 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.621+0000 7f70d1b05640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f709c03dee0 0x7f709c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.621+0000 7f70d1b05640 10 monclient: get_auth_request con 0x7f709c03dee0 auth_method 0 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.623+0000 7f70ba7fc640 1 -- 10.20.193.78:0/4121593204 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f70c001e020 con 0x7f70cc09c4a0 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.623+0000 7f70ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.623+0000 7f70ba7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:24.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.623+0000 7f70d1b05640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f709c03dee0 0x7f709c0403a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f70bc036000 tx=0x7f70bc039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.735+0000 7f70d3d90640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:01:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.735+0000 7f70d3d90640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.735+0000 7f70d3d90640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.735+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f70cc135a20 con 0x7f70cc09c4a0 2026-02-17T06:01:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.735+0000 7f70ba7fc640 1 -- 10.20.193.78:0/4121593204 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f70c0015030 con 0x7f70cc09c4a0 2026-02-17T06:01:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.735+0000 7f70ba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.735+0000 7f70ba7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:24.736 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":178,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f709c03dee0 msgr2=0x7f709c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f709c03dee0 0x7f709c0403a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f70bc036000 tx=0x7f70bc039040 comp rx=0 tx=0).stop 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 10 monclient: shutdown 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f70cc09c4a0 msgr2=0x7f70cc1cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f70cc09c4a0 0x7f70cc1cfc70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c7f0 tx=0x7f70c000ccc0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 shutdown_connections 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f70cc134df0 0x7f70cc09db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f70cc09c4a0 0x7f70cc1cfc70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f709c03dee0 0x7f709c0403a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 --2- 10.20.193.78:0/4121593204 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f70cc06c890 0x7f70cc1cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 >> 10.20.193.78:0/4121593204 conn(0x7f70cc0a6620 msgr2=0x7f70cc071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 shutdown_connections 2026-02-17T06:01:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:24.736+0000 7f70d3d90640 1 -- 10.20.193.78:0/4121593204 wait complete. 2026-02-17T06:01:25.230 INFO:tasks.rados.rados.0.trial115.stdout:2549: done (1 left) 2026-02-17T06:01:25.231 INFO:tasks.rados.rados.0.trial115.stdout:2551: done (0 left) 2026-02-17T06:01:25.231 INFO:tasks.rados.rados.0.trial115.stdout:2552: delete oid 41 current snap is 260 2026-02-17T06:01:25.233 INFO:tasks.rados.rados.0.trial115.stdout:2552: done (0 left) 2026-02-17T06:01:25.234 INFO:tasks.rados.rados.0.trial115.stdout:2553: snap_create 2026-02-17T06:01:26.236 INFO:tasks.rados.rados.0.trial115.stdout:2553: done (0 left) 2026-02-17T06:01:26.236 INFO:tasks.rados.rados.0.trial115.stdout:2554: snap_create 2026-02-17T06:01:27.238 INFO:tasks.rados.rados.0.trial115.stdout:2554: done (0 left) 2026-02-17T06:01:27.238 INFO:tasks.rados.rados.0.trial115.stdout:2555: write oid 26 current snap is 262 2026-02-17T06:01:27.238 INFO:tasks.rados.rados.0.trial115.stdout:2555: seq_num 897 ranges {675879=711654,2073995=682309} 2026-02-17T06:01:27.243 INFO:tasks.rados.rados.0.trial115.stdout:2555: writing trial11545420-26 from 675879 to 1387533 tid 1 2026-02-17T06:01:27.249 INFO:tasks.rados.rados.0.trial115.stdout:2555: writing trial11545420-26 from 2073995 to 2756304 tid 2 2026-02-17T06:01:27.249 INFO:tasks.rados.rados.0.trial115.stdout:2556: delete oid 10 current snap is 262 2026-02-17T06:01:27.249 INFO:tasks.rados.rados.0.trial115.stdout:2555: finishing write tid 1 to trial11545420-26 2026-02-17T06:01:27.250 INFO:tasks.rados.rados.0.trial115.stdout:2556: done (1 left) 2026-02-17T06:01:27.250 INFO:tasks.rados.rados.0.trial115.stdout:2557: write oid 48 current snap is 262 2026-02-17T06:01:27.250 INFO:tasks.rados.rados.0.trial115.stdout:2557: seq_num 899 ranges {709493=578505,1907312=685978} 2026-02-17T06:01:27.255 INFO:tasks.rados.rados.0.trial115.stdout:2557: writing trial11545420-48 from 709493 to 1287998 tid 1 2026-02-17T06:01:27.260 INFO:tasks.rados.rados.0.trial115.stdout:2557: writing trial11545420-48 from 1907312 to 2593290 tid 2 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2558: copy_from oid 25 from oid 14 current snap is 262 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2555: finishing write tid 2 to trial11545420-26 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2555: finishing write tid 3 to trial11545420-26 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2555: finishing write tid 4 to trial11545420-26 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2555: oid 26 updating version 0 to 960 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2555: oid 26 updating version 960 to 961 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2555: oid 26 updating version 961 to 962 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 962 (ObjNum 897 snap 262 seq_num 897) dirty exists 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2555: left oid 26 (ObjNum 897 snap 262 seq_num 897) 2026-02-17T06:01:27.261 INFO:tasks.rados.rados.0.trial115.stdout:2555: done (2 left) 2026-02-17T06:01:27.262 INFO:tasks.rados.rados.0.trial115.stdout:2559: copy_from oid 23 from oid 2 current snap is 262 2026-02-17T06:01:27.262 INFO:tasks.rados.rados.0.trial115.stdout:2560: write oid 7 current snap is 262 2026-02-17T06:01:27.262 INFO:tasks.rados.rados.0.trial115.stdout:2560: seq_num 900 ranges {726300=653754,1965794=429636,2965517=160783} 2026-02-17T06:01:27.265 INFO:tasks.rados.rados.0.trial115.stdout:2560: writing trial11545420-7 from 726300 to 1380054 tid 1 2026-02-17T06:01:27.269 INFO:tasks.rados.rados.0.trial115.stdout:2560: writing trial11545420-7 from 1965794 to 2395430 tid 2 2026-02-17T06:01:27.272 INFO:tasks.rados.rados.0.trial115.stdout:2560: writing trial11545420-7 from 2965517 to 3126300 tid 3 2026-02-17T06:01:27.272 INFO:tasks.rados.rados.0.trial115.stdout:2561: snap_remove snap 259 2026-02-17T06:01:27.272 INFO:tasks.rados.rados.0.trial115.stdout:2559: finishing copy_from racing read to trial11545420-23 2026-02-17T06:01:27.272 INFO:tasks.rados.rados.0.trial115.stdout:2559: finishing copy_from to trial11545420-23 2026-02-17T06:01:27.272 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:01:27.272 INFO:tasks.rados.rados.0.trial115.stdout:2557: finishing write tid 1 to trial11545420-48 2026-02-17T06:01:27.745 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:01:27.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient: get_monmap_and_config 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient: build_initial_monmap 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient: monmap: 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:27.814428+0000 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:27.814428+0000 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:27.817 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 1 Processor -- start 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 1 -- start start 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient: init 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f806c890 0x7f72f806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:27.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient(hunting): picked mon.noname-b con 0x7f72f806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f72f809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f72f806d1b0 0x7f72f8135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient(hunting): picked mon.noname-a con 0x7f72f806d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72f806daf0 con 0x7f72f809c4a0 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f72f80a0880 con 0x7f72f806d1b0 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f72f809cc80 con 0x7f72f806c890 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.813+0000 7f72fd111640 10 monclient(hunting): authenticate will time out at 1533.367957s 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f806c890 0x7f72f806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:27.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f72f806d1b0 0x7f72f8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f806c890 0x7f72f806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56252/0 (socket says 10.20.193.78:56252) 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 1 -- 10.20.193.78:0/2533656907 learned_addr learned my addr 10.20.193.78:0/2533656907 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient(hunting): get_auth_request con 0x7f72f806d1b0 auth_method 0 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 10 monclient(hunting): get_auth_request con 0x7f72f806c890 auth_method 0 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:27.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient(hunting): handle_auth_done global_id 184191 payload 274 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 1 -- 10.20.193.78:0/2533656907 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f806c890 msgr2=0x7f72f806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 1 --2- 10.20.193.78:0/2533656907 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f806c890 0x7f72f806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 1 -- 10.20.193.78:0/2533656907 >> v1:10.20.193.78:6789/0 conn(0x7f72f809c4a0 legacy=0x7f72f8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient: _finish_hunting 0 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 1 monclient: found mon.noname-a 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:27.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.814+0000 7f72f7577640 1 -- 10.20.193.78:0/2533656907 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f809cf60 con 0x7f72f806d1b0 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f7577640 1 -- 10.20.193.78:0/2533656907 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806d1b0 msgr2=0x7f72f8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f7577640 1 --2- 10.20.193.78:0/2533656907 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806d1b0 0x7f72f8135500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009d90 tx=0x7f72e802f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f21a9a399c3f98 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 1 -- 10.20.193.78:0/2533656907 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e803c070 con 0x7f72f806d1b0 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: dump: 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:27.828 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: _finish_auth 0 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: _check_auth_tickets 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 1 -- 10.20.193.78:0/2533656907 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e8037440 con 0x7f72f806d1b0 2026-02-17T06:01:27.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72fd111640 5 monclient: authenticate success, global_id 184191 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 1 -- 10.20.193.78:0/2533656907 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8037810 con 0x7f72f806d1b0 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72f5d74640 10 monclient: dump: 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:27.830 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72fd111640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.816+0000 7f72fd111640 4 set_mon_vals no callback set 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 10 monclient: shutdown 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 -- 10.20.193.78:0/2533656907 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806d1b0 msgr2=0x7f72f8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 --2- 10.20.193.78:0/2533656907 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806d1b0 0x7f72f8135500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009d90 tx=0x7f72e802f190 comp rx=0 tx=0).stop 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 -- 10.20.193.78:0/2533656907 shutdown_connections 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 --2- 10.20.193.78:0/2533656907 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806d1b0 0x7f72f8135500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 --2- 10.20.193.78:0/2533656907 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f806c890 0x7f72f806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 -- 10.20.193.78:0/2533656907 >> 10.20.193.78:0/2533656907 conn(0x7f72f80a6620 msgr2=0x7f72f80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 -- 10.20.193.78:0/2533656907 shutdown_connections 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 -- 10.20.193.78:0/2533656907 wait complete. 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 10 monclient: build_initial_monmap 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 10 monclient: monmap: 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:27.832 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 Processor -- start 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 1 -- start start 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.817+0000 7f72fd111640 10 monclient: init 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806c890 0x7f72f81cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient(hunting): picked mon.noname-a con 0x7f72f806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72f806d1b0 0x7f72f81cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient(hunting): picked mon.noname-b con 0x7f72f806d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:27.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f809c4a0 0x7f72f8123660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient(hunting): picked mon.noname-c con 0x7f72f809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72f80a01a0 con 0x7f72f806c890 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72f80a0020 con 0x7f72f806d1b0 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f72f809be20 con 0x7f72f809c4a0 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72fd111640 10 monclient(hunting): authenticate will time out at 1533.372161s 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806c890 0x7f72f81cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f809c4a0 0x7f72f8123660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:27.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f809c4a0 0x7f72f8123660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56264/0 (socket says 10.20.193.78:56264) 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806c890 0x7f72f81cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49684/0 (socket says 10.20.193.78:49684) 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 -- 10.20.193.78:0/2470405803 learned_addr learned my addr 10.20.193.78:0/2470405803 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient(hunting): get_auth_request con 0x7f72f809c4a0 auth_method 0 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 10 monclient(hunting): get_auth_request con 0x7f72f806c890 auth_method 0 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:27.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient(hunting): handle_auth_done global_id 144578 payload 824 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 -- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72f806d1b0 msgr2=0x7f72f81cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6575640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72f806d1b0 0x7f72f81cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72f806d1b0 0x7f72f81cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 -- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806c890 msgr2=0x7f72f81cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806c890 0x7f72f81cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient: _finish_hunting 0 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 monclient: found mon.noname-c 2026-02-17T06:01:27.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f7577640 1 -- 10.20.193.78:0/2470405803 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f8125e60 con 0x7f72f809c4a0 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6d76640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806c890 0x7f72f81cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6575640 10 monclient: get_auth_request con 0x7f72f806d1b0 auth_method 0 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.818+0000 7f72f6575640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72f806d1b0 0x7f72f81cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72f7577640 1 -- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f809c4a0 msgr2=0x7f72f8123660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72f7577640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f809c4a0 0x7f72f8123660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f72e803a040 tx=0x7f72e8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 1 -- 10.20.193.78:0/2470405803 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e803c050 con 0x7f72f809c4a0 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: dump: 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:27.837 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:27.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 1 -- 10.20.193.78:0/2470405803 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e8048070 con 0x7f72f809c4a0 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 1 -- 10.20.193.78:0/2470405803 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8041a70 con 0x7f72f809c4a0 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72f5573640 4 set_mon_vals no callback set 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72d77fe640 10 monclient: dump: 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:27.839 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72fd111640 5 monclient: authenticate success, global_id 144578 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72fd111640 10 monclient: _renew_subs 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72fd111640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f81275f0 con 0x7f72f809c4a0 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72fd111640 10 monclient: _renew_subs 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72fd111640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.820+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f8136e10 con 0x7f72f809c4a0 2026-02-17T06:01:27.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72fd111640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72fd111640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72fd111640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f8074980 con 0x7f72f809c4a0 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72d77fe640 1 -- 10.20.193.78:0/2470405803 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f72e8041c10 con 0x7f72f809c4a0 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72d77fe640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f72c403deb0 0x7f72c4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72d77fe640 1 -- 10.20.193.78:0/2470405803 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(552..552 src has 335..552) ==== 4735+0+0 (secure 0 0 0) 0x7f72e80784f0 con 0x7f72f809c4a0 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72f6d76640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f72c403deb0 0x7f72c4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.821+0000 7f72f6d76640 10 monclient: get_auth_request con 0x7f72c403deb0 auth_method 0 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.823+0000 7f72d77fe640 1 -- 10.20.193.78:0/2470405803 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f72e8052020 con 0x7f72f809c4a0 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.823+0000 7f72d77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.823+0000 7f72d77fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:27.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.824+0000 7f72f6d76640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f72c403deb0 0x7f72c4040370 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f72ec006dc0 tx=0x7f72ec009340 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:27.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.938+0000 7f72fd111640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:01:27.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.938+0000 7f72fd111640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:27.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.938+0000 7f72fd111640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:27.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.938+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f72f806cc70 con 0x7f72f809c4a0 2026-02-17T06:01:27.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.938+0000 7f72d77fe640 1 -- 10.20.193.78:0/2470405803 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f72e8035dc0 con 0x7f72f809c4a0 2026-02-17T06:01:27.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.938+0000 7f72d77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:27.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.938+0000 7f72d77fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:27.940 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":178,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:01:27.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f72c403deb0 msgr2=0x7f72c4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f72c403deb0 0x7f72c4040370 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f72ec006dc0 tx=0x7f72ec009340 comp rx=0 tx=0).stop 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 10 monclient: shutdown 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f809c4a0 msgr2=0x7f72f8123660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f809c4a0 0x7f72f8123660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f72e803a040 tx=0x7f72e8004b40 comp rx=0 tx=0).stop 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 shutdown_connections 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f72f809c4a0 0x7f72f8123660 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f72c403deb0 0x7f72c4040370 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f72f806d1b0 0x7f72f81cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 --2- 10.20.193.78:0/2470405803 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f72f806c890 0x7f72f81cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:27.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 >> 10.20.193.78:0/2470405803 conn(0x7f72f80a6620 msgr2=0x7f72f8071130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:27.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 shutdown_connections 2026-02-17T06:01:27.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:27.939+0000 7f72fd111640 1 -- 10.20.193.78:0/2470405803 wait complete. 2026-02-17T06:01:28.240 INFO:tasks.rados.rados.0.trial115.stdout:2559: done (4 left) 2026-02-17T06:01:28.240 INFO:tasks.rados.rados.0.trial115.stdout:2561: done (3 left) 2026-02-17T06:01:28.241 INFO:tasks.rados.rados.0.trial115.stdout:2562: snap_remove snap 260 2026-02-17T06:01:29.243 INFO:tasks.rados.rados.0.trial115.stdout:2562: done (3 left) 2026-02-17T06:01:29.243 INFO:tasks.rados.rados.0.trial115.stdout:2563: read oid 47 snap -1 2026-02-17T06:01:29.243 INFO:tasks.rados.rados.0.trial115.stdout:2563: expect deleted 2026-02-17T06:01:29.243 INFO:tasks.rados.rados.0.trial115.stdout:2564: write (excl) oid 37 current snap is 262 2026-02-17T06:01:29.243 INFO:tasks.rados.rados.0.trial115.stdout:2557: finishing write tid 2 to trial11545420-48 2026-02-17T06:01:29.243 INFO:tasks.rados.rados.0.trial115.stdout:2557: finishing write tid 3 to trial11545420-48 2026-02-17T06:01:29.243 INFO:tasks.rados.rados.0.trial115.stdout:2564: seq_num 901 ranges {743107=729003,1907923=689647} 2026-02-17T06:01:29.248 INFO:tasks.rados.rados.0.trial115.stdout:2564: writing trial11545420-37 from 743107 to 1472110 tid 1 2026-02-17T06:01:29.253 INFO:tasks.rados.rados.0.trial115.stdout:2564: writing trial11545420-37 from 1907923 to 2597570 tid 2 2026-02-17T06:01:29.253 INFO:tasks.rados.rados.0.trial115.stdout:2565: copy_from oid 36 from oid 50 current snap is 262 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2557: finishing write tid 4 to trial11545420-48 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2557: oid 48 updating version 0 to 1414 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2557: oid 48 updating version 1414 to 1415 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2557: oid 48 version 1415 is already newer than 1413 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1415 (ObjNum 899 snap 262 seq_num 899) dirty exists 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2557: left oid 48 (ObjNum 899 snap 262 seq_num 899) 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2558: finishing copy_from racing read to trial11545420-25 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2560: finishing write tid 1 to trial11545420-7 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2558: finishing copy_from to trial11545420-25 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 962 (ObjNum 862 snap 246 seq_num 862) dirty exists 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2560: finishing write tid 2 to trial11545420-7 2026-02-17T06:01:29.254 INFO:tasks.rados.rados.0.trial115.stdout:2560: finishing write tid 3 to trial11545420-7 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2560: finishing write tid 4 to trial11545420-7 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2560: finishing write tid 5 to trial11545420-7 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2560: oid 7 updating version 0 to 1280 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2560: oid 7 updating version 1280 to 1281 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2560: oid 7 version 1281 is already newer than 1279 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2560: oid 7 version 1281 is already newer than 1278 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1281 (ObjNum 900 snap 262 seq_num 900) dirty exists 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2560: left oid 7 (ObjNum 900 snap 262 seq_num 900) 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2557: done (5 left) 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2558: done (4 left) 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2560: done (3 left) 2026-02-17T06:01:29.255 INFO:tasks.rados.rados.0.trial115.stdout:2563: done (2 left) 2026-02-17T06:01:29.256 INFO:tasks.rados.rados.0.trial115.stdout:2566: write (excl) oid 40 current snap is 262 2026-02-17T06:01:29.256 INFO:tasks.rados.rados.0.trial115.stdout:2566: seq_num 902 ranges {759914=404252,1566405=433305,2504010=655904} 2026-02-17T06:01:29.259 INFO:tasks.rados.rados.0.trial115.stdout:2566: writing trial11545420-40 from 759914 to 1164166 tid 1 2026-02-17T06:01:29.261 INFO:tasks.rados.rados.0.trial115.stdout:2566: writing trial11545420-40 from 1566405 to 1999710 tid 2 2026-02-17T06:01:29.266 INFO:tasks.rados.rados.0.trial115.stdout:2566: writing trial11545420-40 from 2504010 to 3159914 tid 3 2026-02-17T06:01:29.266 INFO:tasks.rados.rados.0.trial115.stdout:2567: write (excl) oid 16 current snap is 262 2026-02-17T06:01:29.266 INFO:tasks.rados.rados.0.trial115.stdout:2564: finishing write tid 1 to trial11545420-37 2026-02-17T06:01:29.266 INFO:tasks.rados.rados.0.trial115.stdout:2564: finishing write tid 2 to trial11545420-37 2026-02-17T06:01:29.266 INFO:tasks.rados.rados.0.trial115.stdout:2564: finishing write tid 3 to trial11545420-37 2026-02-17T06:01:29.267 INFO:tasks.rados.rados.0.trial115.stdout:2567: seq_num 903 ranges {776721=479501,2024887=693316} 2026-02-17T06:01:29.270 INFO:tasks.rados.rados.0.trial115.stdout:2567: writing trial11545420-16 from 776721 to 1256222 tid 1 2026-02-17T06:01:29.277 INFO:tasks.rados.rados.0.trial115.stdout:2567: writing trial11545420-16 from 2024887 to 2718203 tid 2 2026-02-17T06:01:29.277 INFO:tasks.rados.rados.0.trial115.stdout:2568: copy_from oid 44 from oid 43 current snap is 262 2026-02-17T06:01:29.277 INFO:tasks.rados.rados.0.trial115.stdout:2564: finishing write tid 4 to trial11545420-37 2026-02-17T06:01:29.277 INFO:tasks.rados.rados.0.trial115.stdout:2564: oid 37 updating version 0 to 1282 2026-02-17T06:01:29.277 INFO:tasks.rados.rados.0.trial115.stdout:2564: oid 37 updating version 1282 to 1283 2026-02-17T06:01:29.277 INFO:tasks.rados.rados.0.trial115.stdout:2564: oid 37 updating version 1283 to 1284 2026-02-17T06:01:29.277 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1284 (ObjNum 901 snap 262 seq_num 901) dirty exists 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2564: left oid 37 (ObjNum 901 snap 262 seq_num 901) 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2565: finishing copy_from racing read to trial11545420-36 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2565: finishing copy_from to trial11545420-36 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 936 (ObjNum 751 snap 206 seq_num 751) dirty exists 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2566: finishing write tid 1 to trial11545420-40 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2566: finishing write tid 2 to trial11545420-40 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2566: finishing write tid 3 to trial11545420-40 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2566: finishing write tid 4 to trial11545420-40 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2566: finishing write tid 5 to trial11545420-40 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2566: oid 40 updating version 0 to 1286 2026-02-17T06:01:29.278 INFO:tasks.rados.rados.0.trial115.stdout:2566: oid 40 version 1286 is already newer than 1284 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2566: oid 40 version 1286 is already newer than 1285 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2566: oid 40 version 1286 is already newer than 1283 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1286 (ObjNum 902 snap 262 seq_num 902) dirty exists 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2566: left oid 40 (ObjNum 902 snap 262 seq_num 902) 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2567: finishing write tid 1 to trial11545420-16 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2564: done (4 left) 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2565: done (3 left) 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2566: done (2 left) 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2569: write oid 30 current snap is 262 2026-02-17T06:01:29.279 INFO:tasks.rados.rados.0.trial115.stdout:2569: seq_num 904 ranges {793528=554750,2083369=436974,2958856=234672} 2026-02-17T06:01:29.281 INFO:tasks.rados.rados.0.trial115.stdout:2569: writing trial11545420-30 from 793528 to 1348278 tid 1 2026-02-17T06:01:29.285 INFO:tasks.rados.rados.0.trial115.stdout:2569: writing trial11545420-30 from 2083369 to 2520343 tid 2 2026-02-17T06:01:29.289 INFO:tasks.rados.rados.0.trial115.stdout:2569: writing trial11545420-30 from 2958856 to 3193528 tid 3 2026-02-17T06:01:29.289 INFO:tasks.rados.rados.0.trial115.stdout:2570: snap_create 2026-02-17T06:01:29.289 INFO:tasks.rados.rados.0.trial115.stdout:2568: finishing copy_from to trial11545420-44 2026-02-17T06:01:29.289 INFO:tasks.rados.rados.0.trial115.stdout:2568: got expected ENOENT (src dne) 2026-02-17T06:01:29.289 INFO:tasks.rados.rados.0.trial115.stdout:2568: finishing copy_from racing read to trial11545420-44 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2567: finishing write tid 2 to trial11545420-16 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2567: finishing write tid 3 to trial11545420-16 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2567: finishing write tid 4 to trial11545420-16 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2567: oid 16 updating version 0 to 1423 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2567: oid 16 version 1423 is already newer than 1421 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2567: oid 16 version 1423 is already newer than 1422 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1423 (ObjNum 903 snap 262 seq_num 903) dirty exists 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2567: left oid 16 (ObjNum 903 snap 262 seq_num 903) 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2569: finishing write tid 1 to trial11545420-30 2026-02-17T06:01:29.290 INFO:tasks.rados.rados.0.trial115.stdout:2569: finishing write tid 2 to trial11545420-30 2026-02-17T06:01:29.291 INFO:tasks.rados.rados.0.trial115.stdout:2569: finishing write tid 3 to trial11545420-30 2026-02-17T06:01:29.291 INFO:tasks.rados.rados.0.trial115.stdout:2569: finishing write tid 4 to trial11545420-30 2026-02-17T06:01:29.292 INFO:tasks.rados.rados.0.trial115.stdout:2569: finishing write tid 5 to trial11545420-30 2026-02-17T06:01:29.292 INFO:tasks.rados.rados.0.trial115.stdout:2569: oid 30 updating version 0 to 1164 2026-02-17T06:01:29.292 INFO:tasks.rados.rados.0.trial115.stdout:2569: oid 30 updating version 1164 to 1165 2026-02-17T06:01:29.292 INFO:tasks.rados.rados.0.trial115.stdout:2569: oid 30 updating version 1165 to 1166 2026-02-17T06:01:29.292 INFO:tasks.rados.rados.0.trial115.stdout:2569: oid 30 version 1166 is already newer than 1163 2026-02-17T06:01:29.292 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1166 (ObjNum 904 snap 262 seq_num 904) dirty exists 2026-02-17T06:01:29.292 INFO:tasks.rados.rados.0.trial115.stdout:2569: left oid 30 (ObjNum 904 snap 262 seq_num 904) 2026-02-17T06:01:30.231 INFO:tasks.rados.rados.0.trial115.stdout:2567: done (3 left) 2026-02-17T06:01:30.232 INFO:tasks.rados.rados.0.trial115.stdout:2568: done (2 left) 2026-02-17T06:01:30.232 INFO:tasks.rados.rados.0.trial115.stdout:2569: done (1 left) 2026-02-17T06:01:30.232 INFO:tasks.rados.rados.0.trial115.stdout:2570: done (0 left) 2026-02-17T06:01:30.232 INFO:tasks.rados.rados.0.trial115.stdout:2571: write (excl) oid 5 current snap is 263 2026-02-17T06:01:30.232 INFO:tasks.rados.rados.0.trial115.stdout:2571: seq_num 905 ranges {410335=746352,1741851=696985,3102632=107703} 2026-02-17T06:01:30.237 INFO:tasks.rados.rados.0.trial115.stdout:2571: writing trial11545420-5 from 410335 to 1156687 tid 1 2026-02-17T06:01:30.242 INFO:tasks.rados.rados.0.trial115.stdout:2571: writing trial11545420-5 from 1741851 to 2438836 tid 2 2026-02-17T06:01:30.245 INFO:tasks.rados.rados.0.trial115.stdout:2571: writing trial11545420-5 from 3102632 to 3210335 tid 3 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2572: read oid 25 snap -1 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2572: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2573: copy_from oid 29 from oid 48 current snap is 263 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2571: finishing write tid 1 to trial11545420-5 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2571: finishing write tid 2 to trial11545420-5 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2574: read oid 27 snap -1 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2574: expect (ObjNum 841 snap 237 seq_num 841) 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2575: copy_from oid 22 from oid 2 current snap is 263 2026-02-17T06:01:30.246 INFO:tasks.rados.rados.0.trial115.stdout:2576: snap_create 2026-02-17T06:01:30.247 INFO:tasks.rados.rados.0.trial115.stdout:2571: finishing write tid 3 to trial11545420-5 2026-02-17T06:01:30.247 INFO:tasks.rados.rados.0.trial115.stdout:2571: finishing write tid 4 to trial11545420-5 2026-02-17T06:01:30.247 INFO:tasks.rados.rados.0.trial115.stdout:2571: finishing write tid 5 to trial11545420-5 2026-02-17T06:01:30.248 INFO:tasks.rados.rados.0.trial115.stdout:2571: oid 5 updating version 0 to 1379 2026-02-17T06:01:30.248 INFO:tasks.rados.rados.0.trial115.stdout:2571: oid 5 updating version 1379 to 1381 2026-02-17T06:01:30.248 INFO:tasks.rados.rados.0.trial115.stdout:2571: oid 5 version 1381 is already newer than 1380 2026-02-17T06:01:30.248 INFO:tasks.rados.rados.0.trial115.stdout:2571: oid 5 version 1381 is already newer than 1378 2026-02-17T06:01:30.248 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1381 (ObjNum 905 snap 263 seq_num 905) dirty exists 2026-02-17T06:01:30.248 INFO:tasks.rados.rados.0.trial115.stdout:2571: left oid 5 (ObjNum 905 snap 263 seq_num 905) 2026-02-17T06:01:30.252 INFO:tasks.rados.rados.0.trial115.stdout:2574: expect (ObjNum 841 snap 237 seq_num 841) 2026-02-17T06:01:30.345 INFO:tasks.rados.rados.0.trial115.stdout:2572: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:01:30.498 INFO:tasks.rados.rados.0.trial115.stdout:2575: finishing copy_from racing read to trial11545420-22 2026-02-17T06:01:30.499 INFO:tasks.rados.rados.0.trial115.stdout:2575: finishing copy_from to trial11545420-22 2026-02-17T06:01:30.499 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:01:30.499 INFO:tasks.rados.rados.0.trial115.stdout:2573: finishing copy_from racing read to trial11545420-29 2026-02-17T06:01:30.499 INFO:tasks.rados.rados.0.trial115.stdout:2573: finishing copy_from to trial11545420-29 2026-02-17T06:01:30.499 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1415 (ObjNum 899 snap 262 seq_num 899) dirty exists 2026-02-17T06:01:30.948 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:01:31.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient: get_monmap_and_config 2026-02-17T06:01:31.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient: build_initial_monmap 2026-02-17T06:01:31.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient: monmap: 2026-02-17T06:01:31.018 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:31.018 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:31.018427+0000 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:31.018427+0000 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:31.019 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 1 Processor -- start 2026-02-17T06:01:31.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 1 -- start start 2026-02-17T06:01:31.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient: init 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7a3406c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7a3409c4a0 0x7f7a3413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient(hunting): picked mon.noname-c con 0x7f7a3409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7a3406d380 0x7f7a3413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient(hunting): picked mon.noname-a con 0x7f7a3406d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a3406daf0 con 0x7f7a3406c890 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7a340a0880 con 0x7f7a3406d380 2026-02-17T06:01:31.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7a3409cc80 con 0x7f7a3409c4a0 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.017+0000 7f7a3b1e7640 10 monclient(hunting): authenticate will time out at 1536.571955s 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7a3406d380 0x7f7a3413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7a3406d380 0x7f7a3413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49686/0 (socket says 10.20.193.78:49686) 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 1 -- 10.20.193.78:0/619450764 learned_addr learned my addr 10.20.193.78:0/619450764 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient(hunting): get_auth_request con 0x7f7a3406d380 auth_method 0 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient(hunting): handle_auth_done global_id 184197 payload 274 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 1 -- 10.20.193.78:0/619450764 >> v2:10.20.193.115:3300/0 conn(0x7f7a3409c4a0 msgr2=0x7f7a3413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 1 --2- 10.20.193.78:0/619450764 >> v2:10.20.193.115:3300/0 conn(0x7f7a3409c4a0 0x7f7a3413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 1 -- 10.20.193.78:0/619450764 >> v1:10.20.193.78:6789/0 conn(0x7f7a3406c890 legacy=0x7f7a3406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 1 monclient: found mon.noname-a 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.018+0000 7f7a3975d640 1 -- 10.20.193.78:0/619450764 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3409cf60 con 0x7f7a3406d380 2026-02-17T06:01:31.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a3975d640 1 -- 10.20.193.78:0/619450764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406d380 msgr2=0x7f7a3413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a3975d640 1 --2- 10.20.193.78:0/619450764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406d380 0x7f7a3413d8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a3406dcc0 tx=0x7f7a3002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8640e8338597cea4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 1 -- 10.20.193.78:0/619450764 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3002fe50 con 0x7f7a3406d380 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: dump: 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.031 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a3b1e7640 5 monclient: authenticate success, global_id 184197 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a3b1e7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 1 -- 10.20.193.78:0/619450764 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a300373f0 con 0x7f7a3406d380 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 1 -- 10.20.193.78:0/619450764 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300377c0 con 0x7f7a3406d380 2026-02-17T06:01:31.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a2b7fe640 10 monclient: dump: 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.033 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a3b1e7640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.020+0000 7f7a3b1e7640 4 set_mon_vals no callback set 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 10 monclient: shutdown 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/619450764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406d380 msgr2=0x7f7a3413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/619450764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406d380 0x7f7a3413d8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a3406dcc0 tx=0x7f7a3002f1e0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/619450764 shutdown_connections 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/619450764 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406d380 0x7f7a3413d8f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/619450764 >> v2:10.20.193.115:3300/0 conn(0x7f7a3409c4a0 0x7f7a3413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/619450764 >> 10.20.193.78:0/619450764 conn(0x7f7a340a6620 msgr2=0x7f7a340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/619450764 shutdown_connections 2026-02-17T06:01:31.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/619450764 wait complete. 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 10 monclient: build_initial_monmap 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 10 monclient: monmap: 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:31.035 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 Processor -- start 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 1 -- start start 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.021+0000 7f7a3b1e7640 10 monclient: init 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406c890 0x7f7a341cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient(hunting): picked mon.noname-a con 0x7f7a3406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7a3406d380 0x7f7a3413c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient(hunting): picked mon.noname-b con 0x7f7a3406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7a3409c4a0 0x7f7a3413cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient(hunting): picked mon.noname-c con 0x7f7a3409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:31.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3409bfa0 con 0x7f7a3406c890 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3409be20 con 0x7f7a3406d380 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7a3413de30 con 0x7f7a3409c4a0 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3b1e7640 10 monclient(hunting): authenticate will time out at 1536.576158s 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406c890 0x7f7a341cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406c890 0x7f7a341cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49694/0 (socket says 10.20.193.78:49694) 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 -- 10.20.193.78:0/3971477935 learned_addr learned my addr 10.20.193.78:0/3971477935 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient(hunting): get_auth_request con 0x7f7a3406c890 auth_method 0 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3975d640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7a3409c4a0 0x7f7a3413cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3975d640 10 monclient(hunting): get_auth_request con 0x7f7a3409c4a0 auth_method 0 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3975d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3975d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3975d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3975d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3975d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a3975d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient(hunting): handle_auth_done global_id 184200 payload 824 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 -- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7a3409c4a0 msgr2=0x7f7a3413cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7a3409c4a0 0x7f7a3413cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 -- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7a3406d380 msgr2=0x7f7a3413c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7a3406d380 0x7f7a3413c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 monclient: found mon.noname-a 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.022+0000 7f7a38f5c640 1 -- 10.20.193.78:0/3971477935 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3406f7c0 con 0x7f7a3406c890 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a38f5c640 1 -- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406c890 msgr2=0x7f7a341cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a38f5c640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406c890 0x7f7a341cd060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400c5d0 tx=0x7f7a2400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 1 -- 10.20.193.78:0/3971477935 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24016020 con 0x7f7a3406c890 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: dump: 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.040 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 1 -- 10.20.193.78:0/3971477935 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a24012a60 con 0x7f7a3406c890 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 1 -- 10.20.193.78:0/3971477935 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24012e50 con 0x7f7a3406c890 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a297fa640 10 monclient: dump: 2026-02-17T06:01:31.041 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a2affd640 4 set_mon_vals no callback set 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a3b1e7640 5 monclient: authenticate success, global_id 184200 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a3b1e7640 10 monclient: _renew_subs 2026-02-17T06:01:31.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a3b1e7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3413b110 con 0x7f7a3406c890 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a3b1e7640 10 monclient: _renew_subs 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a3b1e7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.024+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a34138e20 con 0x7f7a3406c890 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a3b1e7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a3b1e7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a3b1e7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a297fa640 1 -- 10.20.193.78:0/3971477935 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7a2401b400 con 0x7f7a3406c890 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3406cc70 con 0x7f7a3406c890 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a297fa640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7a0403e0a0 0x7f7a04040560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a297fa640 1 -- 10.20.193.78:0/3971477935 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(555..555 src has 335..555) ==== 4763+0+0 (secure 0 0 0) 0x7f7a240528b0 con 0x7f7a3406c890 2026-02-17T06:01:31.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a2bfff640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7a0403e0a0 0x7f7a04040560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.025+0000 7f7a2bfff640 10 monclient: get_auth_request con 0x7f7a0403e0a0 auth_method 0 2026-02-17T06:01:31.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.027+0000 7f7a297fa640 1 -- 10.20.193.78:0/3971477935 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7a24020030 con 0x7f7a3406c890 2026-02-17T06:01:31.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.027+0000 7f7a297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:31.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.027+0000 7f7a297fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:31.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.028+0000 7f7a2bfff640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7a0403e0a0 0x7f7a04040560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7a3413b320 tx=0x7f7a1c006d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.142+0000 7f7a3b1e7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:01:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.142+0000 7f7a3b1e7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.142+0000 7f7a3b1e7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.142+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7a340744d0 con 0x7f7a3406c890 2026-02-17T06:01:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.142+0000 7f7a297fa640 1 -- 10.20.193.78:0/3971477935 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f7a3406f7c0 con 0x7f7a3406c890 2026-02-17T06:01:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.142+0000 7f7a297fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:31.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.142+0000 7f7a297fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:31.145 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":180,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:01:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.143+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7a0403e0a0 msgr2=0x7f7a04040560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.143+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7a0403e0a0 0x7f7a04040560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7a3413b320 tx=0x7f7a1c006d90 comp rx=0 tx=0).stop 2026-02-17T06:01:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.143+0000 7f7a3b1e7640 10 monclient: shutdown 2026-02-17T06:01:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406c890 msgr2=0x7f7a341cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406c890 0x7f7a341cd060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400c5d0 tx=0x7f7a2400caa0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 shutdown_connections 2026-02-17T06:01:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7a3409c4a0 0x7f7a3413cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7a0403e0a0 0x7f7a04040560 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7a3406d380 0x7f7a3413c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 --2- 10.20.193.78:0/3971477935 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7a3406c890 0x7f7a341cd060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 >> 10.20.193.78:0/3971477935 conn(0x7f7a340a6620 msgr2=0x7f7a340a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 shutdown_connections 2026-02-17T06:01:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.144+0000 7f7a3b1e7640 1 -- 10.20.193.78:0/3971477935 wait complete. 2026-02-17T06:01:31.152 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:01:31.153 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:01:31.153 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:01:31.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.218+0000 7fd427bce640 10 monclient: get_monmap_and_config 2026-02-17T06:01:31.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.218+0000 7fd427bce640 10 monclient: build_initial_monmap 2026-02-17T06:01:31.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.218+0000 7fd427bce640 10 monclient: monmap: 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:31.219866+0000 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:31.219866+0000 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:31.220 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:31.221 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 1 Processor -- start 2026-02-17T06:01:31.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 1 -- start start 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient: init 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd42006c890 0x7fd4200a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient(hunting): picked mon.noname-a con 0x7fd42006c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd42006cc70 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42013a240 0x7fd42013c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient(hunting): picked mon.noname-b con 0x7fd42013a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd420074150 con 0x7fd42006cc70 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd42009ef70 con 0x7fd42006c890 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd42009f250 con 0x7fd42013a240 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd427bce640 10 monclient(hunting): authenticate will time out at 1536.773377s 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd42006c890 0x7fd4200a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd42006c890 0x7fd4200a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49710/0 (socket says 10.20.193.78:49710) 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425142640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd42006cc70 0x7fd42006d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:34710/0 (socket says 10.20.193.78:34710) 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 1 -- 10.20.193.78:0/521199422 learned_addr learned my addr 10.20.193.78:0/521199422 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient(hunting): get_auth_request con 0x7fd42006c890 auth_method 0 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd426144640 1 --2- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42013a240 0x7fd42013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd426144640 10 monclient(hunting): get_auth_request con 0x7fd42013a240 auth_method 0 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd426144640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd426144640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd426144640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd426144640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd426144640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd426144640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient(hunting): handle_auth_done global_id 184203 payload 274 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 1 -- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42013a240 msgr2=0x7fd42013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 1 --2- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42013a240 0x7fd42013c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 1 -- 10.20.193.78:0/521199422 >> v1:10.20.193.78:6789/0 conn(0x7fd42006cc70 legacy=0x7fd42006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 1 monclient: found mon.noname-a 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.219+0000 7fd425943640 1 -- 10.20.193.78:0/521199422 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42009df40 con 0x7fd42006c890 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd425943640 1 -- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 msgr2=0x7fd4200a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd425943640 1 --2- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd4200a01f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd410009970 tx=0x7fd41002efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7730fd2e05b97bcd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 1 -- 10.20.193.78:0/521199422 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41003c070 con 0x7fd42006c890 2026-02-17T06:01:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: dump: 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 1 -- 10.20.193.78:0/521199422 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41002fdf0 con 0x7fd42006c890 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 5 monclient: authenticate success, global_id 184203 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 1 -- 10.20.193.78:0/521199422 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410037660 con 0x7fd42006c890 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 4 set_mon_vals no callback set 2026-02-17T06:01:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd424941640 10 monclient: dump: 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 10 monclient: shutdown 2026-02-17T06:01:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 1 -- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 msgr2=0x7fd4200a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 1 --2- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd4200a01f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd410009970 tx=0x7fd41002efb0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 1 -- 10.20.193.78:0/521199422 shutdown_connections 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 1 --2- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42013a240 0x7fd42013c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 1 --2- 10.20.193.78:0/521199422 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd4200a01f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 1 -- 10.20.193.78:0/521199422 >> 10.20.193.78:0/521199422 conn(0x7fd4200a6620 msgr2=0x7fd4200a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 1 -- 10.20.193.78:0/521199422 shutdown_connections 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.222+0000 7fd427bce640 1 -- 10.20.193.78:0/521199422 wait complete. 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient: build_initial_monmap 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient: monmap: 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:31.238 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 1 Processor -- start 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 1 -- start start 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient: init 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd42011cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient(hunting): picked mon.noname-a con 0x7fd42006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42006cc70 0x7fd42011d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient(hunting): picked mon.noname-c con 0x7fd42006cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd42013a240 0x7fd42011d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient(hunting): picked mon.noname-b con 0x7fd42013a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd42009bfa0 con 0x7fd42006c890 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd42009be20 con 0x7fd42013a240 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd42009c120 con 0x7fd42006cc70 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd427bce640 10 monclient(hunting): authenticate will time out at 1536.777621s 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd42011cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd42011cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49714/0 (socket says 10.20.193.78:49714) 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 -- 10.20.193.78:0/1067158291 learned_addr learned my addr 10.20.193.78:0/1067158291 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient(hunting): get_auth_request con 0x7fd42006c890 auth_method 0 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425142640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42006cc70 0x7fd42011d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425142640 10 monclient(hunting): get_auth_request con 0x7fd42006cc70 auth_method 0 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425142640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425142640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425142640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient(hunting): handle_auth_done global_id 184206 payload 824 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42006cc70 msgr2=0x7fd42011d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd42006cc70 0x7fd42011d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd42013a240 msgr2=0x7fd42011d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd42013a240 0x7fd42011d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 monclient: found mon.noname-a 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.223+0000 7fd425943640 1 -- 10.20.193.78:0/1067158291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42011f5a0 con 0x7fd42006c890 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.225+0000 7fd425943640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 msgr2=0x7fd42011cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.225+0000 7fd425943640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd42011cc70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4100090b0 tx=0x7fd410004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410046030 con 0x7fd42006c890 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: dump: 2026-02-17T06:01:31.243 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 5 monclient: authenticate success, global_id 184206 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41002fbf0 con 0x7fd42006c890 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41003c040 con 0x7fd42006c890 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40ffff640 4 set_mon_vals no callback set 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 10 monclient: dump: 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.245 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 10 monclient: _renew_subs 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42012fe30 con 0x7fd42006c890 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 10 monclient: _renew_subs 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42009d810 con 0x7fd42006c890 2026-02-17T06:01:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd410053060 con 0x7fd42006c890 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42011f1b0 0x7fd42012ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd42011de10 con 0x7fd42011f1b0 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42011f1b0 0x7fd42012ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 10 monclient: get_auth_request con 0x7fd42011f1b0 auth_method 0 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3f003deb0 0x7fd3f0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 10 monclient(hunting): handle_auth_done global_id 184209 payload 824 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd425142640 1 -- 10.20.193.78:0/1067158291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd420074980 con 0x7fd42011f1b0 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.226+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(555..555 src has 335..555) ==== 4763+0+0 (secure 0 0 0) 0x7fd410077770 con 0x7fd42006c890 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.227+0000 7fd426144640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3f003deb0 0x7fd3f0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.227+0000 7fd426144640 10 monclient: get_auth_request con 0x7fd3f003deb0 auth_method 0 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd425142640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42011f1b0 msgr2=0x7fd42012ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd425142640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42011f1b0 0x7fd42012ece0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd414007fd0 tx=0x7fd41400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41400cbf0 con 0x7fd42011f1b0 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 10 monclient: dump: 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd414019070 con 0x7fd42011f1b0 2026-02-17T06:01:31.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42011f1b0 msgr2=0x7fd42012ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42011f1b0 0x7fd42012ece0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd414007fd0 tx=0x7fd41400ef10 comp rx=0 tx=0).stop 2026-02-17T06:01:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42011f1b0 msgr2=0x7fd42012ece0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd40e7fc640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42011f1b0 0x7fd42012ece0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd414007fd0 tx=0x7fd41400ef10 comp rx=0 tx=0).stop 2026-02-17T06:01:31.250 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.229+0000 7fd426144640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3f003deb0 0x7fd3f0040370 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd42011fa90 tx=0x7fd41c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.251 INFO:tasks.rados.rados.0.trial115.stdout:2571: done (5 left) 2026-02-17T06:01:31.252 INFO:tasks.rados.rados.0.trial115.stdout:2572: done (4 left) 2026-02-17T06:01:31.252 INFO:tasks.rados.rados.0.trial115.stdout:2573: done (3 left) 2026-02-17T06:01:31.252 INFO:tasks.rados.rados.0.trial115.stdout:2574: done (2 left) 2026-02-17T06:01:31.252 INFO:tasks.rados.rados.0.trial115.stdout:2575: done (1 left) 2026-02-17T06:01:31.252 INFO:tasks.rados.rados.0.trial115.stdout:2576: done (0 left) 2026-02-17T06:01:31.252 INFO:tasks.rados.rados.0.trial115.stdout:2577: write oid 20 current snap is 264 2026-02-17T06:01:31.252 INFO:tasks.rados.rados.0.trial115.stdout:2577: seq_num 906 ranges {427142=421601,1400333=440643,2613702=450326} 2026-02-17T06:01:31.252 INFO:tasks.rados.rados.0.trial115.stdout:2577: writing trial11545420-20 from 427142 to 848743 tid 1 2026-02-17T06:01:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd427bce640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:31.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd427bce640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd4201333b0 0x7fd4201da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd427bce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd420073390 con 0x7fd4201333b0 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd4201333b0 0x7fd4201da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 10 monclient: get_auth_request con 0x7fd4201333b0 auth_method 0 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 10 monclient(hunting): handle_auth_done global_id 184212 payload 824 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.232+0000 7fd425142640 1 -- 10.20.193.78:0/1067158291 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd4200a01d0 con 0x7fd4201333b0 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd425142640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd4201333b0 msgr2=0x7fd4201da7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd425142640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd4201333b0 0x7fd4201da7d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd414007fd0 tx=0x7fd414004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41401e220 con 0x7fd4201333b0 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 10 monclient: dump: 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7fd41401e220 con 0x7fd4201333b0 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd4201333b0 msgr2=0x7fd4201da7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd4201333b0 0x7fd4201da7d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd414007fd0 tx=0x7fd414004a30 comp rx=0 tx=0).stop 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd4201333b0 msgr2=0x7fd4201da7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd40e7fc640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd4201333b0 0x7fd4201da7d0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd414007fd0 tx=0x7fd414004a30 comp rx=0 tx=0).stop 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:31.256 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 180, 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 56016, 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:31.257 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:31.258 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:31.259 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:31.260 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:31.261 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:31.262 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.263 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:31.264 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.265 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.266 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:31.267 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:31.267 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:01:31.267 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.267 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.267 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.267 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:31.267 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:31.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3f003deb0 msgr2=0x7fd3f0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3f003deb0 0x7fd3f0040370 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd42011fa90 tx=0x7fd41c00b040 comp rx=0 tx=0).stop 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 10 monclient: shutdown 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 msgr2=0x7fd42011cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd42011cc70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4100090b0 tx=0x7fd410004830 comp rx=0 tx=0).stop 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd426144640 1 -- 10.20.193.78:0/1067158291 reap_dead start 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 shutdown_connections 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 --2- 10.20.193.78:0/1067158291 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd42006c890 0x7fd42011cc70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 >> 10.20.193.78:0/1067158291 conn(0x7fd4200a6620 msgr2=0x7fd420132c50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 shutdown_connections 2026-02-17T06:01:31.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.235+0000 7fd427bce640 1 -- 10.20.193.78:0/1067158291 wait complete. 2026-02-17T06:01:31.270 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:01:31.270 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:01:31.270 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:01:31.270 INFO:tasks.rados.rados.0.trial115.stdout:2577: writing trial11545420-20 from 1400333 to 1840976 tid 2 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2577: writing trial11545420-20 from 2613702 to 3064028 tid 3 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2578: rollback oid 13 current snap is 264 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 13 to 263 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2577: finishing write tid 1 to trial11545420-20 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2577: finishing write tid 2 to trial11545420-20 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2579: write oid 26 current snap is 264 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2579: seq_num 907 ranges {443949=496850,1458815=584301,2641125=602824} 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2579: writing trial11545420-26 from 443949 to 940799 tid 1 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2579: writing trial11545420-26 from 1458815 to 2043116 tid 2 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2579: writing trial11545420-26 from 2641125 to 3243949 tid 3 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2580: write (excl) oid 27 current snap is 264 2026-02-17T06:01:31.271 INFO:tasks.rados.rados.0.trial115.stdout:2580: seq_num 908 ranges {460756=572099,1517297=444312,2668548=472870} 2026-02-17T06:01:31.273 INFO:tasks.rados.rados.0.trial115.stdout:2580: writing trial11545420-27 from 460756 to 1032855 tid 1 2026-02-17T06:01:31.276 INFO:tasks.rados.rados.0.trial115.stdout:2580: writing trial11545420-27 from 1517297 to 1961609 tid 2 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2580: writing trial11545420-27 from 2668548 to 3141418 tid 3 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2581: snap_remove snap 261 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2577: finishing write tid 3 to trial11545420-20 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2577: finishing write tid 4 to trial11545420-20 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2577: finishing write tid 5 to trial11545420-20 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2577: oid 20 updating version 0 to 1287 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2577: oid 20 updating version 1287 to 1288 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2577: oid 20 version 1288 is already newer than 1285 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2577: oid 20 version 1288 is already newer than 1286 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1288 (ObjNum 906 snap 264 seq_num 906) dirty exists 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2577: left oid 20 (ObjNum 906 snap 264 seq_num 906) 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2578: finishing rollback tid 0 to trial11545420-13 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2578: finishing rollback tid 1 to trial11545420-13 2026-02-17T06:01:31.281 INFO:tasks.rados.rados.0.trial115.stdout:2578: finishing rollback tid 2 to trial11545420-13 2026-02-17T06:01:31.282 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1418 (ObjNum 888 snap 254 seq_num 888) dirty exists 2026-02-17T06:01:31.282 INFO:tasks.rados.rados.0.trial115.stdout:2579: finishing write tid 1 to trial11545420-26 2026-02-17T06:01:31.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient: get_monmap_and_config 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient: build_initial_monmap 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient: monmap: 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:31.339360+0000 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:31.339360+0000 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:31.339 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 1 Processor -- start 2026-02-17T06:01:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 1 -- start start 2026-02-17T06:01:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient: init 2026-02-17T06:01:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2026-02-17T06:01:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9d2c09ea90 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:31.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f9d2c09ee70 0x7f9d2c09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient(hunting): picked mon.noname-a con 0x7f9d2c09ee70 addr v2:10.20.193.78:3300/0 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13d640 0x7f9d2c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient(hunting): picked mon.noname-b con 0x7f9d2c13d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d2c0a0a20 con 0x7f9d2c09ea90 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f9d2c0a0d00 con 0x7f9d2c09ee70 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c0a0460 con 0x7f9d2c13d640 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.338+0000 7f9d322da640 10 monclient(hunting): authenticate will time out at 1536.892919s 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2bfff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f9d2c09ea90 0x7f9d2c0a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:34712/0 (socket says 10.20.193.78:34712) 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2bfff640 1 -- 10.20.193.78:0/3593415470 learned_addr learned my addr 10.20.193.78:0/3593415470 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/3593415470 >> v2:10.20.193.78:3300/0 conn(0x7f9d2c09ee70 0x7f9d2c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d30850640 1 --2- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13d640 0x7f9d2c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient(hunting): get_auth_request con 0x7f9d2c09ee70 auth_method 0 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d30850640 10 monclient(hunting): get_auth_request con 0x7f9d2c13d640 auth_method 0 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d30850640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d30850640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d30850640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d30850640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d30850640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d30850640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_done global_id 184215 payload 274 2026-02-17T06:01:31.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13d640 msgr2=0x7f9d2c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13d640 0x7f9d2c13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/3593415470 >> v1:10.20.193.78:6789/0 conn(0x7f9d2c09ea90 legacy=0x7f9d2c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 1 monclient: found mon.noname-a 2026-02-17T06:01:31.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.339+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/3593415470 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c0a0740 con 0x7f9d2c09ee70 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 msgr2=0x7f9d2c09f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c09f2c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c06e270 tx=0x7f9d1c02ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6ffbbdda32c8dc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 1 -- 10.20.193.78:0/3593415470 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c03c070 con 0x7f9d2c09ee70 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: dump: 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.350 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 1 -- 10.20.193.78:0/3593415470 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1c02fbd0 con 0x7f9d2c09ee70 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 1 -- 10.20.193.78:0/3593415470 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c037440 con 0x7f9d2c09ee70 2026-02-17T06:01:31.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d322da640 5 monclient: authenticate success, global_id 184215 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d322da640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d322da640 4 set_mon_vals no callback set 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.341+0000 7f9d2affd640 10 monclient: dump: 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.352 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 10 monclient: shutdown 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 -- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 msgr2=0x7f9d2c09f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 --2- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c09f2c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c06e270 tx=0x7f9d1c02ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 -- 10.20.193.78:0/3593415470 shutdown_connections 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 --2- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13d640 0x7f9d2c13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 --2- 10.20.193.78:0/3593415470 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c09f2c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 -- 10.20.193.78:0/3593415470 >> 10.20.193.78:0/3593415470 conn(0x7f9d2c0a7520 msgr2=0x7f9d2c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 -- 10.20.193.78:0/3593415470 shutdown_connections 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 -- 10.20.193.78:0/3593415470 wait complete. 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 10 monclient: build_initial_monmap 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 10 monclient: monmap: 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:31.353 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 Processor -- start 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 1 -- start start 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.342+0000 7f9d322da640 10 monclient: init 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c1cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient(hunting): picked mon.noname-c con 0x7f9d2c09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:31.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient(hunting): picked mon.noname-a con 0x7f9d2c09ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c13d640 0x7f9d2c073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient(hunting): picked mon.noname-b con 0x7f9d2c13d640 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c0a6820 con 0x7f9d2c09ee70 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c0a66a0 con 0x7f9d2c13d640 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c13ff70 con 0x7f9d2c09ea90 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d322da640 10 monclient(hunting): authenticate will time out at 1536.897107s 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c1cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49748/0 (socket says 10.20.193.78:49748) 2026-02-17T06:01:31.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/2073636063 learned_addr learned my addr 10.20.193.78:0/2073636063 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient(hunting): get_auth_request con 0x7f9d2c09ee70 auth_method 0 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2bfff640 10 monclient(hunting): get_auth_request con 0x7f9d2c09ea90 auth_method 0 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient(hunting): handle_auth_done global_id 184218 payload 824 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d30850640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c13d640 0x7f9d2c073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 msgr2=0x7f9d2c1cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c09ea90 0x7f9d2c1cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c13d640 msgr2=0x7f9d2c073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c13d640 0x7f9d2c073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 monclient: found mon.noname-a 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/2073636063 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c070e60 con 0x7f9d2c09ee70 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d30850640 10 monclient: get_auth_request con 0x7f9d2c13d640 auth_method 0 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.343+0000 7f9d30850640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9d2c13d640 0x7f9d2c073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d2b7fe640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 msgr2=0x7f9d2c073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d2b7fe640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c073680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c002410 tx=0x7f9d1c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c03c040 con 0x7f9d2c09ee70 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: dump: 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1c047070 con 0x7f9d2c09ee70 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d322da640 5 monclient: authenticate success, global_id 184218 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c040a60 con 0x7f9d2c09ee70 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d2a7fc640 4 set_mon_vals no callback set 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d28ff9640 10 monclient: dump: 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.359 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d322da640 10 monclient: _renew_subs 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d322da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2c0a9ce0 con 0x7f9d2c09ee70 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d322da640 10 monclient: _renew_subs 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d322da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.345+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2c070b80 con 0x7f9d2c09ee70 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d322da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d322da640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c0a0 0x7f9d2c0a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d322da640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c06fa30 con 0x7f9d2c13c0a0 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c0a0 0x7f9d2c0a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 10 monclient: get_auth_request con 0x7f9d2c13c0a0 auth_method 0 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_done global_id 144596 payload 824 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d2bfff640 1 -- 10.20.193.78:0/2073636063 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d2c136ac0 con 0x7f9d2c13c0a0 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9d1c03f030 con 0x7f9d2c09ee70 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d28ff9640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cfc03deb0 0x7f9cfc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(556..556 src has 335..556) ==== 4735+0+0 (secure 0 0 0) 0x7f9d1c077a40 con 0x7f9d2c09ee70 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d30850640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cfc03deb0 0x7f9cfc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.346+0000 7f9d30850640 10 monclient: get_auth_request con 0x7f9cfc03deb0 auth_method 0 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d2bfff640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c0a0 msgr2=0x7f9d2c0a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d2bfff640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c0a0 0x7f9d2c0a1e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800eaa0 tx=0x7f9d1800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800cbf0 con 0x7f9d2c13c0a0 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 10 monclient: dump: 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f9d18019070 con 0x7f9d2c13c0a0 2026-02-17T06:01:31.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:31.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:31.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c0a0 msgr2=0x7f9d2c0a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c0a0 0x7f9d2c0a1e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800eaa0 tx=0x7f9d1800ef70 comp rx=0 tx=0).stop 2026-02-17T06:01:31.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c0a0 msgr2=0x7f9d2c0a1e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:31.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.348+0000 7f9d28ff9640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c0a0 0x7f9d2c0a1e20 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800eaa0 tx=0x7f9d1800ef70 comp rx=0 tx=0).stop 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.349+0000 7f9d30850640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cfc03deb0 0x7f9cfc040370 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c13d4f0 tx=0x7f9d2000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d322da640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d322da640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c480 0x7f9d2c1da420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d322da640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c09f2c0 con 0x7f9d2c13c480 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c480 0x7f9d2c1da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 10 monclient: get_auth_request con 0x7f9d2c13c480 auth_method 0 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 10 monclient(hunting): handle_auth_done global_id 144599 payload 824 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.352+0000 7f9d2bfff640 1 -- 10.20.193.78:0/2073636063 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9d2c06f730 con 0x7f9d2c13c480 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d2bfff640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c480 msgr2=0x7f9d2c1da420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d2bfff640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c480 0x7f9d2c1da420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800eaa0 tx=0x7f9d18004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1801e250 con 0x7f9d2c13c480 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 10 monclient: dump: 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.366 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f9d1801e250 con 0x7f9d2c13c480 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c480 msgr2=0x7f9d2c1da420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c480 0x7f9d2c1da420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800eaa0 tx=0x7f9d18004a40 comp rx=0 tx=0).stop 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c480 msgr2=0x7f9d2c1da420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d28ff9640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9d2c13c480 0x7f9d2c1da420 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800eaa0 tx=0x7f9d18004a40 comp rx=0 tx=0).stop 2026-02-17T06:01:31.367 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 180, 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 274889, 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:31.368 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:31.369 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:31.370 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:31.371 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.372 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.373 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.374 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:31.375 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:01:31.376 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:31.377 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:31.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cfc03deb0 msgr2=0x7f9cfc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cfc03deb0 0x7f9cfc040370 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c13d4f0 tx=0x7f9d2000b040 comp rx=0 tx=0).stop 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 10 monclient: shutdown 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 msgr2=0x7f9d2c073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c073680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c002410 tx=0x7f9d1c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d30850640 1 -- 10.20.193.78:0/2073636063 reap_dead start 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 shutdown_connections 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 --2- 10.20.193.78:0/2073636063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9d2c09ee70 0x7f9d2c073680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 >> 10.20.193.78:0/2073636063 conn(0x7f9d2c0a7520 msgr2=0x7f9d2c0aa230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 shutdown_connections 2026-02-17T06:01:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.354+0000 7f9d322da640 1 -- 10.20.193.78:0/2073636063 wait complete. 2026-02-17T06:01:31.379 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:01:31.379 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:01:31.380 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.446+0000 7f2f1abb6640 10 monclient: get_monmap_and_config 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.446+0000 7f2f1abb6640 10 monclient: build_initial_monmap 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.446+0000 7f2f1abb6640 10 monclient: monmap: 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:31.447803+0000 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:31.447803+0000 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.447 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:31.448 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:31.448 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:31.448 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:31.448 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:31.448 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.446+0000 7f2f1abb6640 1 Processor -- start 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 1 -- start start 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient: init 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2f1406c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2f1409c4a0 0x7f2f1413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient(hunting): picked mon.noname-c con 0x7f2f1409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2f1406d380 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f1406daf0 con 0x7f2f1406d380 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f1409cc80 con 0x7f2f1406c890 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2f140a0880 con 0x7f2f1409c4a0 2026-02-17T06:01:31.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1abb6640 10 monclient(hunting): authenticate will time out at 1537.001351s 2026-02-17T06:01:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1912c640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f2f1406d380 0x7f2f1413d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:34720/0 (socket says 10.20.193.78:34720) 2026-02-17T06:01:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f1912c640 1 -- 10.20.193.78:0/1874640890 learned_addr learned my addr 10.20.193.78:0/1874640890 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:01:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1874640890 >> v2:10.20.193.115:3300/0 conn(0x7f2f1409c4a0 0x7f2f1413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request con 0x7f2f1409c4a0 auth_method 0 2026-02-17T06:01:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.447+0000 7f2f0bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f1912c640 1 -- 10.20.193.78:0/1874640890 >> v1:10.20.193.78:6789/0 conn(0x7f2f1406d380 legacy=0x7f2f1413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935246707 0 0) 0x7f2f1406daf0 con 0x7f2f1406d380 2026-02-17T06:01:31.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): _negotiate 2026-02-17T06:01:31.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): my global_id is 184221 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f00003620 con 0x7f2f1406d380 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 12565915 0 0) 0x7f2f00003620 con 0x7f2f1406d380 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 >> v2:10.20.193.115:3300/0 conn(0x7f2f1409c4a0 msgr2=0x7f2f1413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 --2- 10.20.193.78:0/1874640890 >> v2:10.20.193.115:3300/0 conn(0x7f2f1409c4a0 0x7f2f1413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 >> v1:10.20.193.115:6789/0 conn(0x7f2f1406c890 legacy=0x7f2f1406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1409cf60 con 0x7f2f1406d380 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f2f10003530 con 0x7f2f1406d380 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: dump: 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 >> v1:10.20.193.78:6789/0 conn(0x7f2f1406d380 legacy=0x7f2f1413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f00003bd0 0x7f2f00003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): picked mon.c con 0x7f2f00003bd0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:31.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f00004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): picked mon.a con 0x7f2f00004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f00009ab0 0x7f2f0000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): picked mon.b con 0x7f2f00009ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0000c490 con 0x7f2f00004900 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0000c610 con 0x7f2f00009ab0 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0000c810 con 0x7f2f00003bd0 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.449+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f00004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request con 0x7f2f00004900 auth_method 0 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1892b640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f00003bd0 0x7f2f00003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1892b640 10 monclient(hunting): get_auth_request con 0x7f2f00003bd0 auth_method 0 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1892b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1892b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1892b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1892b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1892b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1892b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_done global_id 184221 payload 293 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f00003bd0 msgr2=0x7f2f00003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f00003bd0 0x7f2f00003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1874640890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f00009ab0 msgr2=0x7f2f0000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f00009ab0 0x7f2f0000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 1 monclient: found mon.a 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1874640890 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f00003620 con 0x7f2f00004900 2026-02-17T06:01:31.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f0bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1912c640 1 -- 10.20.193.78:0/1874640890 reap_dead start 2026-02-17T06:01:31.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1abb6640 5 monclient: authenticate success, global_id 184221 2026-02-17T06:01:31.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.450+0000 7f2f1abb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:31.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 msgr2=0x7f2f00004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f00004d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009900 tx=0x7f2efc02efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b630ae85936e0ec server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc03c070 con 0x7f2f00004900 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 10 monclient: dump: 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2efc02fbd0 con 0x7f2f00004900 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f1abb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:31.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 1 -- 10.20.193.78:0/1874640890 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00f040 con 0x7f2f00004900 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f1abb6640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f1abb6640 4 set_mon_vals no callback set 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.452+0000 7f2f0b7fe640 10 monclient: dump: 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 10 monclient: shutdown 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:31.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 msgr2=0x7f2f00004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f00004d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009900 tx=0x7f2efc02efa0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1874640890 shutdown_connections 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 --2- 10.20.193.78:0/1874640890 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f00004d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1874640890 >> 10.20.193.78:0/1874640890 conn(0x7f2f140a6620 msgr2=0x7f2f140a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1874640890 shutdown_connections 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1874640890 wait complete. 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 10 monclient: build_initial_monmap 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 10 monclient: monmap: 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 Processor -- start 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 1 -- start start 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.453+0000 7f2f1abb6640 10 monclient: init 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f141cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient(hunting): picked mon.noname-a con 0x7f2f00004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141cd870 0x7f2f14122620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient(hunting): picked mon.noname-b con 0x7f2f141cd870 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f141cdc50 0x7f2f14122b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient(hunting): picked mon.noname-c con 0x7f2f141cdc50 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1409bfa0 con 0x7f2f00004900 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1409be20 con 0x7f2f141cd870 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1406cc70 con 0x7f2f141cdc50 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:31.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1abb6640 10 monclient(hunting): authenticate will time out at 1537.008147s 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f141cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1912c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f141cdc50 0x7f2f14122b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f141cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49764/0 (socket says 10.20.193.78:49764) 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 -- 10.20.193.78:0/1669706837 learned_addr learned my addr 10.20.193.78:0/1669706837 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient(hunting): get_auth_request con 0x7f2f00004900 auth_method 0 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1912c640 10 monclient(hunting): get_auth_request con 0x7f2f141cdc50 auth_method 0 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1912c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1912c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1912c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1912c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:31.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1912c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1912c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141cd870 0x7f2f14122620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request con 0x7f2f141cd870 auth_method 0 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f0bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient(hunting): handle_auth_done global_id 184224 payload 824 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f141cdc50 msgr2=0x7f2f14122b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2f141cdc50 0x7f2f14122b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141cd870 msgr2=0x7f2f14122620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141cd870 0x7f2f14122620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient: _finish_hunting 0 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 monclient: found mon.noname-a 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.454+0000 7f2f1892b640 1 -- 10.20.193.78:0/1669706837 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f14125f70 con 0x7f2f00004900 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1892b640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 msgr2=0x7f2f141cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1892b640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f141cd330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400f8d0 tx=0x7f2f0400fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04018020 con 0x7f2f00004900 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: dump: 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0400daf0 con 0x7f2f00004900 2026-02-17T06:01:31.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 5 monclient: authenticate success, global_id 184224 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 10 monclient: _renew_subs 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f141246f0 con 0x7f2f00004900 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04014470 con 0x7f2f00004900 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f097fa640 10 monclient: dump: 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:31.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f0affd640 4 set_mon_vals no callback set 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 10 monclient: _renew_subs 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f14136ea0 con 0x7f2f00004900 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.456+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14126590 con 0x7f2f141230a0 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2f04014690 con 0x7f2f00004900 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 10 monclient: get_auth_request con 0x7f2f141230a0 auth_method 0 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f097fa640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ee403deb0 0x7f2ee4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(556..556 src has 335..556) ==== 4735+0+0 (secure 0 0 0) 0x7f2f04052f70 con 0x7f2f00004900 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:01:31.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:31.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:31.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f1912c640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ee403deb0 0x7f2ee4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.457+0000 7f2f1912c640 10 monclient: get_auth_request con 0x7f2ee403deb0 auth_method 0 2026-02-17T06:01:31.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.459+0000 7f2f1912c640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ee403deb0 0x7f2ee4040370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2f140a7460 tx=0x7f2f1409d130 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:31.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:31.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 10 monclient: get_auth_request con 0x7f2f141230a0 auth_method 0 2026-02-17T06:01:31.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:31.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:31.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:31.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:01:31.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:01:31.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:31.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:31.658+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:01:32.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:32.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 10 monclient: get_auth_request con 0x7f2f141230a0 auth_method 0 2026-02-17T06:01:32.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:32.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:32.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:32.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:01:32.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:01:32.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:32.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.058+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:01:32.252 INFO:tasks.rados.rados.0.trial115.stdout:2577: done (4 left) 2026-02-17T06:01:32.252 INFO:tasks.rados.rados.0.trial115.stdout:2578: done (3 left) 2026-02-17T06:01:32.252 INFO:tasks.rados.rados.0.trial115.stdout:2581: done (2 left) 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2582: read oid 22 snap -1 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2582: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2583: delete oid 5 current snap is 264 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2579: finishing write tid 2 to trial11545420-26 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2579: finishing write tid 3 to trial11545420-26 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2579: finishing write tid 4 to trial11545420-26 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2579: finishing write tid 5 to trial11545420-26 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2579: oid 26 updating version 0 to 966 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2579: oid 26 version 966 is already newer than 964 2026-02-17T06:01:32.253 INFO:tasks.rados.rados.0.trial115.stdout:2579: oid 26 version 966 is already newer than 963 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2579: oid 26 version 966 is already newer than 965 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 966 (ObjNum 907 snap 264 seq_num 907) dirty exists 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2579: left oid 26 (ObjNum 907 snap 264 seq_num 907) 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2580: finishing write tid 1 to trial11545420-27 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2580: finishing write tid 2 to trial11545420-27 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2580: finishing write tid 3 to trial11545420-27 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2580: finishing write tid 4 to trial11545420-27 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2580: finishing write tid 5 to trial11545420-27 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2580: oid 27 updating version 0 to 1167 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2580: oid 27 updating version 1167 to 1169 2026-02-17T06:01:32.254 INFO:tasks.rados.rados.0.trial115.stdout:2580: oid 27 version 1169 is already newer than 1168 2026-02-17T06:01:32.255 INFO:tasks.rados.rados.0.trial115.stdout:2580: oid 27 updating version 1169 to 1170 2026-02-17T06:01:32.255 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1170 (ObjNum 908 snap 264 seq_num 908) dirty exists 2026-02-17T06:01:32.255 INFO:tasks.rados.rados.0.trial115.stdout:2580: left oid 27 (ObjNum 908 snap 264 seq_num 908) 2026-02-17T06:01:32.255 INFO:tasks.rados.rados.0.trial115.stdout:2579: done (3 left) 2026-02-17T06:01:32.256 INFO:tasks.rados.rados.0.trial115.stdout:2580: done (2 left) 2026-02-17T06:01:32.256 INFO:tasks.rados.rados.0.trial115.stdout:2583: done (1 left) 2026-02-17T06:01:32.256 INFO:tasks.rados.rados.0.trial115.stdout:2584: write oid 39 current snap is 264 2026-02-17T06:01:32.256 INFO:tasks.rados.rados.0.trial115.stdout:2584: seq_num 910 ranges {494370=722597,1917908=447981,3007041=287329} 2026-02-17T06:01:32.259 INFO:tasks.rados.rados.0.trial115.stdout:2584: writing trial11545420-39 from 494370 to 1216967 tid 1 2026-02-17T06:01:32.263 INFO:tasks.rados.rados.0.trial115.stdout:2584: writing trial11545420-39 from 1917908 to 2365889 tid 2 2026-02-17T06:01:32.267 INFO:tasks.rados.rados.0.trial115.stdout:2584: writing trial11545420-39 from 3007041 to 3294370 tid 3 2026-02-17T06:01:32.267 INFO:tasks.rados.rados.0.trial115.stdout:2585: copy_from oid 7 from oid 6 current snap is 264 2026-02-17T06:01:32.267 INFO:tasks.rados.rados.0.trial115.stdout:2582: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2584: finishing write tid 1 to trial11545420-39 2026-02-17T06:01:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2584: finishing write tid 2 to trial11545420-39 2026-02-17T06:01:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2582: done (2 left) 2026-02-17T06:01:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2586: read oid 13 snap 263 2026-02-17T06:01:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2586: expect (ObjNum 888 snap 254 seq_num 888) 2026-02-17T06:01:32.315 INFO:tasks.rados.rados.0.trial115.stdout:2587: rollback oid 48 current snap is 264 2026-02-17T06:01:32.315 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 48 to 263 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2584: finishing write tid 3 to trial11545420-39 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2584: finishing write tid 4 to trial11545420-39 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2584: finishing write tid 5 to trial11545420-39 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2584: oid 39 updating version 0 to 1391 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2584: oid 39 updating version 1391 to 1392 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2584: oid 39 updating version 1392 to 1393 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2584: oid 39 version 1393 is already newer than 1390 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1393 (ObjNum 910 snap 264 seq_num 910) dirty exists 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2584: left oid 39 (ObjNum 910 snap 264 seq_num 910) 2026-02-17T06:01:32.316 INFO:tasks.rados.rados.0.trial115.stdout:2585: finishing copy_from racing read to trial11545420-7 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:2585: finishing copy_from to trial11545420-7 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 962 (ObjNum 891 snap 254 seq_num 891) dirty exists 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:2584: done (3 left) 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:2585: done (2 left) 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:2588: read oid 33 snap -1 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:2588: expect (ObjNum 770 snap 218 seq_num 770) 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:2589: rollback oid 25 current snap is 264 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 262 2026-02-17T06:01:32.317 INFO:tasks.rados.rados.0.trial115.stdout:2590: snap_create 2026-02-17T06:01:32.318 INFO:tasks.rados.rados.0.trial115.stdout:2586: expect (ObjNum 888 snap 254 seq_num 888) 2026-02-17T06:01:32.412 INFO:tasks.rados.rados.0.trial115.stdout:2588: expect (ObjNum 770 snap 218 seq_num 770) 2026-02-17T06:01:32.483 INFO:tasks.rados.rados.0.trial115.stdout:2589: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:01:32.483 INFO:tasks.rados.rados.0.trial115.stdout:2589: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:01:32.483 INFO:tasks.rados.rados.0.trial115.stdout:2589: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:01:32.483 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1384 (ObjNum 862 snap 246 seq_num 862) dirty exists 2026-02-17T06:01:32.484 INFO:tasks.rados.rados.0.trial115.stdout:2587: finishing rollback tid 0 to trial11545420-48 2026-02-17T06:01:32.484 INFO:tasks.rados.rados.0.trial115.stdout:2587: finishing rollback tid 1 to trial11545420-48 2026-02-17T06:01:32.484 INFO:tasks.rados.rados.0.trial115.stdout:2587: finishing rollback tid 2 to trial11545420-48 2026-02-17T06:01:32.484 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1431 (ObjNum 899 snap 262 seq_num 899) dirty exists 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 10 monclient: get_auth_request con 0x7f2f141230a0 auth_method 0 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:32.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:32.860+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:01:33.253 INFO:tasks.rados.rados.0.trial115.stdout:2586: done (4 left) 2026-02-17T06:01:33.253 INFO:tasks.rados.rados.0.trial115.stdout:2587: done (3 left) 2026-02-17T06:01:33.253 INFO:tasks.rados.rados.0.trial115.stdout:2588: done (2 left) 2026-02-17T06:01:33.253 INFO:tasks.rados.rados.0.trial115.stdout:2589: done (1 left) 2026-02-17T06:01:33.253 INFO:tasks.rados.rados.0.trial115.stdout:2590: done (0 left) 2026-02-17T06:01:33.253 INFO:tasks.rados.rados.0.trial115.stdout:2591: read oid 47 snap -1 2026-02-17T06:01:33.254 INFO:tasks.rados.rados.0.trial115.stdout:2591: expect deleted 2026-02-17T06:01:33.254 INFO:tasks.rados.rados.0.trial115.stdout:2592: rollback oid 29 current snap is 265 2026-02-17T06:01:33.254 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 264 2026-02-17T06:01:33.254 INFO:tasks.rados.rados.0.trial115.stdout:2593: read oid 23 snap -1 2026-02-17T06:01:33.254 INFO:tasks.rados.rados.0.trial115.stdout:2593: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:33.254 INFO:tasks.rados.rados.0.trial115.stdout:2594: write (excl) oid 18 current snap is 265 2026-02-17T06:01:33.254 INFO:tasks.rados.rados.0.trial115.stdout:2594: seq_num 911 ranges {511177=797846,1976390=591639,3034464=276713} 2026-02-17T06:01:33.257 INFO:tasks.rados.rados.0.trial115.stdout:2594: writing trial11545420-18 from 511177 to 1309023 tid 1 2026-02-17T06:01:33.262 INFO:tasks.rados.rados.0.trial115.stdout:2594: writing trial11545420-18 from 1976390 to 2568029 tid 2 2026-02-17T06:01:33.265 INFO:tasks.rados.rados.0.trial115.stdout:2594: writing trial11545420-18 from 3034464 to 3311177 tid 3 2026-02-17T06:01:33.265 INFO:tasks.rados.rados.0.trial115.stdout:2595: write (excl) oid 6 current snap is 265 2026-02-17T06:01:33.265 INFO:tasks.rados.rados.0.trial115.stdout:2595: seq_num 912 ranges {527984=473095,1634872=451650,2661887=634311} 2026-02-17T06:01:33.268 INFO:tasks.rados.rados.0.trial115.stdout:2595: writing trial11545420-6 from 527984 to 1001079 tid 1 2026-02-17T06:01:33.272 INFO:tasks.rados.rados.0.trial115.stdout:2595: writing trial11545420-6 from 1634872 to 2086522 tid 2 2026-02-17T06:01:33.277 INFO:tasks.rados.rados.0.trial115.stdout:2595: writing trial11545420-6 from 2661887 to 3296198 tid 3 2026-02-17T06:01:33.277 INFO:tasks.rados.rados.0.trial115.stdout:2596: delete oid 9 current snap is 265 2026-02-17T06:01:33.277 INFO:tasks.rados.rados.0.trial115.stdout:2592: finishing rollback tid 0 to trial11545420-29 2026-02-17T06:01:33.278 INFO:tasks.rados.rados.0.trial115.stdout:2592: finishing rollback tid 1 to trial11545420-29 2026-02-17T06:01:33.278 INFO:tasks.rados.rados.0.trial115.stdout:2592: finishing rollback tid 2 to trial11545420-29 2026-02-17T06:01:33.278 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1421 (ObjNum 899 snap 262 seq_num 899) dirty exists 2026-02-17T06:01:33.278 INFO:tasks.rados.rados.0.trial115.stdout:2593: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:01:33.324 INFO:tasks.rados.rados.0.trial115.stdout:2594: finishing write tid 1 to trial11545420-18 2026-02-17T06:01:33.324 INFO:tasks.rados.rados.0.trial115.stdout:2594: finishing write tid 2 to trial11545420-18 2026-02-17T06:01:33.324 INFO:tasks.rados.rados.0.trial115.stdout:2591: done (5 left) 2026-02-17T06:01:33.324 INFO:tasks.rados.rados.0.trial115.stdout:2592: done (4 left) 2026-02-17T06:01:33.324 INFO:tasks.rados.rados.0.trial115.stdout:2593: done (3 left) 2026-02-17T06:01:33.324 INFO:tasks.rados.rados.0.trial115.stdout:2596: done (2 left) 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2597: delete oid 36 current snap is 265 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2594: finishing write tid 3 to trial11545420-18 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2594: finishing write tid 4 to trial11545420-18 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2594: finishing write tid 5 to trial11545420-18 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2594: oid 18 updating version 0 to 1388 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2594: oid 18 version 1388 is already newer than 1385 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2594: oid 18 version 1388 is already newer than 1387 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2594: oid 18 version 1388 is already newer than 1386 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1388 (ObjNum 911 snap 265 seq_num 911) dirty exists 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2594: left oid 18 (ObjNum 911 snap 265 seq_num 911) 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2595: finishing write tid 1 to trial11545420-6 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2595: finishing write tid 2 to trial11545420-6 2026-02-17T06:01:33.325 INFO:tasks.rados.rados.0.trial115.stdout:2595: finishing write tid 3 to trial11545420-6 2026-02-17T06:01:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2595: finishing write tid 4 to trial11545420-6 2026-02-17T06:01:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2595: finishing write tid 5 to trial11545420-6 2026-02-17T06:01:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2595: oid 6 updating version 0 to 965 2026-02-17T06:01:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2595: oid 6 version 965 is already newer than 964 2026-02-17T06:01:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2595: oid 6 version 965 is already newer than 963 2026-02-17T06:01:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2595: oid 6 updating version 965 to 966 2026-02-17T06:01:33.326 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 966 (ObjNum 912 snap 265 seq_num 912) dirty exists 2026-02-17T06:01:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2595: left oid 6 (ObjNum 912 snap 265 seq_num 912) 2026-02-17T06:01:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2594: done (2 left) 2026-02-17T06:01:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2595: done (1 left) 2026-02-17T06:01:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2597: done (0 left) 2026-02-17T06:01:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2598: write (excl) oid 10 current snap is 265 2026-02-17T06:01:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2598: seq_num 915 ranges {578405=415195,1410318=598977,2744156=584480} 2026-02-17T06:01:33.329 INFO:tasks.rados.rados.0.trial115.stdout:2598: writing trial11545420-10 from 578405 to 993600 tid 1 2026-02-17T06:01:33.333 INFO:tasks.rados.rados.0.trial115.stdout:2598: writing trial11545420-10 from 1410318 to 2009295 tid 2 2026-02-17T06:01:33.338 INFO:tasks.rados.rados.0.trial115.stdout:2598: writing trial11545420-10 from 2744156 to 3328636 tid 3 2026-02-17T06:01:33.338 INFO:tasks.rados.rados.0.trial115.stdout:2599: read oid 25 snap -1 2026-02-17T06:01:33.338 INFO:tasks.rados.rados.0.trial115.stdout:2599: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:01:33.338 INFO:tasks.rados.rados.0.trial115.stdout:2600: copy_from oid 12 from oid 39 current snap is 265 2026-02-17T06:01:33.338 INFO:tasks.rados.rados.0.trial115.stdout:2598: finishing write tid 1 to trial11545420-10 2026-02-17T06:01:33.339 INFO:tasks.rados.rados.0.trial115.stdout:2598: finishing write tid 2 to trial11545420-10 2026-02-17T06:01:33.339 INFO:tasks.rados.rados.0.trial115.stdout:2601: read oid 32 snap -1 2026-02-17T06:01:33.339 INFO:tasks.rados.rados.0.trial115.stdout:2601: expect (ObjNum 876 snap 252 seq_num 876) 2026-02-17T06:01:33.339 INFO:tasks.rados.rados.0.trial115.stdout:2602: delete oid 34 current snap is 265 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:2598: finishing write tid 3 to trial11545420-10 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:2598: finishing write tid 4 to trial11545420-10 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:2598: finishing write tid 5 to trial11545420-10 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:2598: oid 10 updating version 0 to 1320 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:2598: oid 10 version 1320 is already newer than 1318 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:2598: oid 10 version 1320 is already newer than 1319 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:2598: oid 10 updating version 1320 to 1321 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1321 (ObjNum 915 snap 265 seq_num 915) dirty exists 2026-02-17T06:01:33.340 INFO:tasks.rados.rados.0.trial115.stdout:2598: left oid 10 (ObjNum 915 snap 265 seq_num 915) 2026-02-17T06:01:33.341 INFO:tasks.rados.rados.0.trial115.stdout:2598: done (4 left) 2026-02-17T06:01:33.341 INFO:tasks.rados.rados.0.trial115.stdout:2602: done (3 left) 2026-02-17T06:01:33.341 INFO:tasks.rados.rados.0.trial115.stdout:2603: write (excl) oid 41 current snap is 265 2026-02-17T06:01:33.341 INFO:tasks.rados.rados.0.trial115.stdout:2603: seq_num 917 ranges {612019=565693,1810929=602646,3082649=329370} 2026-02-17T06:01:33.345 INFO:tasks.rados.rados.0.trial115.stdout:2603: writing trial11545420-41 from 612019 to 1177712 tid 1 2026-02-17T06:01:33.349 INFO:tasks.rados.rados.0.trial115.stdout:2603: writing trial11545420-41 from 1810929 to 2413575 tid 2 2026-02-17T06:01:33.354 INFO:tasks.rados.rados.0.trial115.stdout:2603: writing trial11545420-41 from 3082649 to 3412019 tid 3 2026-02-17T06:01:33.354 INFO:tasks.rados.rados.0.trial115.stdout:2604: copy_from oid 7 from oid 13 current snap is 265 2026-02-17T06:01:33.355 INFO:tasks.rados.rados.0.trial115.stdout:2599: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:01:33.507 INFO:tasks.rados.rados.0.trial115.stdout:2601: expect (ObjNum 876 snap 252 seq_num 876) 2026-02-17T06:01:33.698 INFO:tasks.rados.rados.0.trial115.stdout:2600: finishing copy_from racing read to trial11545420-12 2026-02-17T06:01:33.698 INFO:tasks.rados.rados.0.trial115.stdout:2603: finishing write tid 1 to trial11545420-41 2026-02-17T06:01:33.698 INFO:tasks.rados.rados.0.trial115.stdout:2599: done (4 left) 2026-02-17T06:01:33.698 INFO:tasks.rados.rados.0.trial115.stdout:2601: done (3 left) 2026-02-17T06:01:33.698 INFO:tasks.rados.rados.0.trial115.stdout:2605: rollback oid 18 current snap is 265 2026-02-17T06:01:33.698 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 263 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2600: finishing copy_from to trial11545420-12 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1393 (ObjNum 910 snap 264 seq_num 910) dirty exists 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: finishing write tid 2 to trial11545420-41 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: finishing write tid 3 to trial11545420-41 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: finishing write tid 4 to trial11545420-41 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: finishing write tid 5 to trial11545420-41 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: oid 41 updating version 0 to 1392 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: oid 41 version 1392 is already newer than 1390 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: oid 41 version 1392 is already newer than 1391 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: oid 41 version 1392 is already newer than 1389 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1392 (ObjNum 917 snap 265 seq_num 917) dirty exists 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: left oid 41 (ObjNum 917 snap 265 seq_num 917) 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2604: finishing copy_from racing read to trial11545420-7 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2604: finishing copy_from to trial11545420-7 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1418 (ObjNum 888 snap 254 seq_num 888) dirty exists 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2600: done (3 left) 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2603: done (2 left) 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2604: done (1 left) 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:2606: rollback oid 33 current snap is 265 2026-02-17T06:01:33.699 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 263 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:2607: copy_from oid 47 from oid 35 current snap is 265 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:2608: snap_create 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:2605: finishing rollback tid 0 to trial11545420-18 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:2605: finishing rollback tid 1 to trial11545420-18 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:2605: finishing rollback tid 2 to trial11545420-18 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1395 (ObjNum 818 snap 235 seq_num 818) dirty exists 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:2606: finishing rollback tid 0 to trial11545420-33 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:2606: finishing rollback tid 1 to trial11545420-33 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:2606: finishing rollback tid 2 to trial11545420-33 2026-02-17T06:01:33.700 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1083 (ObjNum 770 snap 218 seq_num 770) dirty exists 2026-02-17T06:01:33.703 INFO:tasks.rados.rados.0.trial115.stdout:2607: finishing copy_from racing read to trial11545420-47 2026-02-17T06:01:33.704 INFO:tasks.rados.rados.0.trial115.stdout:2607: finishing copy_from to trial11545420-47 2026-02-17T06:01:33.704 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1398 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:01:34.259 INFO:tasks.rados.rados.0.trial115.stdout:2605: done (3 left) 2026-02-17T06:01:34.259 INFO:tasks.rados.rados.0.trial115.stdout:2606: done (2 left) 2026-02-17T06:01:34.260 INFO:tasks.rados.rados.0.trial115.stdout:2607: done (1 left) 2026-02-17T06:01:34.260 INFO:tasks.rados.rados.0.trial115.stdout:2608: done (0 left) 2026-02-17T06:01:34.260 INFO:tasks.rados.rados.0.trial115.stdout:2609: snap_create 2026-02-17T06:01:34.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.454+0000 7f2ee3fff640 10 monclient: tick 2026-02-17T06:01:34.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.454+0000 7f2ee3fff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:34.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.454+0000 7f2ee3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:34.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.454+0000 7f2ee3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 10 monclient: get_auth_request con 0x7f2f141230a0 auth_method 0 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:34.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:34.462+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2026-02-17T06:01:35.232 INFO:tasks.rados.rados.0.trial115.stdout:2609: done (0 left) 2026-02-17T06:01:35.232 INFO:tasks.rados.rados.0.trial115.stdout:2610: copy_from oid 35 from oid 39 current snap is 267 2026-02-17T06:01:35.232 INFO:tasks.rados.rados.0.trial115.stdout:2611: copy_from oid 3 from oid 44 current snap is 267 2026-02-17T06:01:35.232 INFO:tasks.rados.rados.0.trial115.stdout:2612: copy_from oid 24 from oid 8 current snap is 267 2026-02-17T06:01:35.232 INFO:tasks.rados.rados.0.trial115.stdout:2613: snap_create 2026-02-17T06:01:35.240 INFO:tasks.rados.rados.0.trial115.stdout:2612: finishing copy_from racing read to trial11545420-24 2026-02-17T06:01:35.241 INFO:tasks.rados.rados.0.trial115.stdout:2612: finishing copy_from to trial11545420-24 2026-02-17T06:01:35.241 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:01:35.244 INFO:tasks.rados.rados.0.trial115.stdout:2610: finishing copy_from racing read to trial11545420-35 2026-02-17T06:01:35.245 INFO:tasks.rados.rados.0.trial115.stdout:2610: finishing copy_from to trial11545420-35 2026-02-17T06:01:35.245 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1393 (ObjNum 910 snap 264 seq_num 910) dirty exists 2026-02-17T06:01:35.246 INFO:tasks.rados.rados.0.trial115.stdout:2611: finishing copy_from racing read to trial11545420-3 2026-02-17T06:01:35.249 INFO:tasks.rados.rados.0.trial115.stdout:2611: finishing copy_from to trial11545420-3 2026-02-17T06:01:35.250 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 969 (ObjNum 895 snap 259 seq_num 895) dirty exists 2026-02-17T06:01:35.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 10 monclient: tick 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 5 monclient: _check_tell_commands timeout tell command 1 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2edc003640 0x7f2edc005ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 msgr2=0x7f2f14123480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f141230a0 0x7f2f14123480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2ee3fff640 1 -- 10.20.193.78:0/1669706837 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14126590 con 0x7f2edc003640 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2edc003640 0x7f2edc005ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2f0bfff640 10 monclient: get_auth_request con 0x7f2edc003640 auth_method 0 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2f0bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.454+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.455+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_done global_id 194098 payload 824 2026-02-17T06:01:35.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.455+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2edc0061b0 con 0x7f2edc003640 2026-02-17T06:01:35.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2edc003640 msgr2=0x7f2edc005ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2edc003640 0x7f2edc005ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2efc039000 tx=0x7f2efc004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc03c040 con 0x7f2edc003640 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 10 monclient: dump: 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.460 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2efc004540 con 0x7f2edc003640 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2edc003640 msgr2=0x7f2edc005ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2edc003640 0x7f2edc005ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2efc039000 tx=0x7f2efc004830 comp rx=0 tx=0).stop 2026-02-17T06:01:35.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2edc003640 msgr2=0x7f2edc005ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:35.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.457+0000 7f2f097fa640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2edc003640 0x7f2edc005ad0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2efc039000 tx=0x7f2efc004830 comp rx=0 tx=0).stop 2026-02-17T06:01:35.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f1abb6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:35.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f1abb6640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f1411ec80 0x7f2f141d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:35.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f1abb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:35.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14126370 con 0x7f2f1411ec80 2026-02-17T06:01:35.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f1411ec80 0x7f2f141d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 10 monclient: get_auth_request con 0x7f2f1411ec80 auth_method 0 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 10 monclient(hunting): handle_auth_done global_id 194101 payload 824 2026-02-17T06:01:35.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.461+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2f1406cff0 con 0x7f2f1411ec80 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.463+0000 7f2f0bfff640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f1411ec80 msgr2=0x7f2f141d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.463+0000 7f2f0bfff640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f1411ec80 0x7f2f141d8330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009900 tx=0x7f2efc008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc03c040 con 0x7f2f1411ec80 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 10 monclient: dump: 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.466 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f2efc055070 con 0x7f2f1411ec80 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f1411ec80 msgr2=0x7f2f141d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f1411ec80 0x7f2f141d8330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009900 tx=0x7f2efc008040 comp rx=0 tx=0).stop 2026-02-17T06:01:35.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f1411ec80 msgr2=0x7f2f141d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:35.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f097fa640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2f1411ec80 0x7f2f141d8330 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009900 tx=0x7f2efc008040 comp rx=0 tx=0).stop 2026-02-17T06:01:35.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1912c640 1 -- 10.20.193.78:0/1669706837 reap_dead start 2026-02-17T06:01:35.468 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:35.468 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 180, 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 10859, 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:35.469 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:35.470 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:35.471 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:35.472 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:35.473 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:35.474 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:35.475 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:35.476 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:35.477 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:35.478 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:35.479 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:35.480 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:35.481 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:35.482 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:35.483 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:35.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ee403deb0 msgr2=0x7f2ee4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ee403deb0 0x7f2ee4040370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2f140a7460 tx=0x7f2f1409d130 comp rx=0 tx=0).stop 2026-02-17T06:01:35.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 10 monclient: shutdown 2026-02-17T06:01:35.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:35.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 msgr2=0x7f2f141cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f141cd330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400f8d0 tx=0x7f2f0400fda0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 shutdown_connections 2026-02-17T06:01:35.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ee403deb0 0x7f2ee4040370 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 --2- 10.20.193.78:0/1669706837 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2f00004900 0x7f2f141cd330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 >> 10.20.193.78:0/1669706837 conn(0x7f2f140a6620 msgr2=0x7f2f141366f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:35.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 shutdown_connections 2026-02-17T06:01:35.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.464+0000 7f2f1abb6640 1 -- 10.20.193.78:0/1669706837 wait complete. 2026-02-17T06:01:35.486 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:01:35.486 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:01:35.486 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:01:35.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.553+0000 7fe1c86bf640 10 monclient: get_monmap_and_config 2026-02-17T06:01:35.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.553+0000 7fe1c86bf640 10 monclient: build_initial_monmap 2026-02-17T06:01:35.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.553+0000 7fe1c86bf640 10 monclient: monmap: 2026-02-17T06:01:35.557 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:35.557 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:35.554800+0000 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:35.554800+0000 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.553+0000 7fe1c86bf640 1 Processor -- start 2026-02-17T06:01:35.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 1 -- start start 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient: init 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fe1c006c890 0x7fe1c006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient(hunting): picked mon.noname-c con 0x7fe1c006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe1c009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006d1b0 0x7fe1c0135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient(hunting): picked mon.noname-b con 0x7fe1c006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c006daf0 con 0x7fe1c009c4a0 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c00a0880 con 0x7fe1c006c890 2026-02-17T06:01:35.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c009cc80 con 0x7fe1c006d1b0 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c86bf640 10 monclient(hunting): authenticate will time out at 1541.108407s 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006d1b0 0x7fe1c0135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006d1b0 0x7fe1c0135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56364/0 (socket says 10.20.193.78:56364) 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 1 -- 10.20.193.78:0/875177988 learned_addr learned my addr 10.20.193.78:0/875177988 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient(hunting): get_auth_request con 0x7fe1c006d1b0 auth_method 0 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:35.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6434640 1 --2- 10.20.193.78:0/875177988 >> v2:10.20.193.115:3300/0 conn(0x7fe1c006c890 0x7fe1c006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6434640 10 monclient(hunting): get_auth_request con 0x7fe1c006c890 auth_method 0 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6434640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient(hunting): handle_auth_done global_id 144605 payload 274 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 1 -- 10.20.193.78:0/875177988 >> v2:10.20.193.115:3300/0 conn(0x7fe1c006c890 msgr2=0x7fe1c006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 1 --2- 10.20.193.78:0/875177988 >> v2:10.20.193.115:3300/0 conn(0x7fe1c006c890 0x7fe1c006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 1 -- 10.20.193.78:0/875177988 >> v1:10.20.193.115:6789/0 conn(0x7fe1c009c4a0 legacy=0x7fe1c0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient: _finish_hunting 0 2026-02-17T06:01:35.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 1 monclient: found mon.noname-b 2026-02-17T06:01:35.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:35.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:01:35.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.554+0000 7fe1c6c35640 1 -- 10.20.193.78:0/875177988 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c009cf60 con 0x7fe1c006d1b0 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c6c35640 1 -- 10.20.193.78:0/875177988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006d1b0 msgr2=0x7fe1c0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c6c35640 1 --2- 10.20.193.78:0/875177988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006d1b0 0x7fe1c0135500 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1c006dcc0 tx=0x7fe1bc02f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62fb67b142bd39f0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 1 -- 10.20.193.78:0/875177988 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0373f0 con 0x7fe1c006d1b0 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: dump: 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:35.569 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: _finish_auth 0 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: _check_auth_tickets 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 1 -- 10.20.193.78:0/875177988 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc037590 con 0x7fe1c006d1b0 2026-02-17T06:01:35.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 1 -- 10.20.193.78:0/875177988 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc037960 con 0x7fe1c006d1b0 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c5432640 10 monclient: dump: 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:35.571 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 5 monclient: authenticate success, global_id 144605 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 4 set_mon_vals no callback set 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 10 monclient: shutdown 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 1 -- 10.20.193.78:0/875177988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006d1b0 msgr2=0x7fe1c0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/875177988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006d1b0 0x7fe1c0135500 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1c006dcc0 tx=0x7fe1bc02f200 comp rx=0 tx=0).stop 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 1 -- 10.20.193.78:0/875177988 shutdown_connections 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/875177988 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006d1b0 0x7fe1c0135500 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/875177988 >> v2:10.20.193.115:3300/0 conn(0x7fe1c006c890 0x7fe1c006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 1 -- 10.20.193.78:0/875177988 >> 10.20.193.78:0/875177988 conn(0x7fe1c00a6620 msgr2=0x7fe1c00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 1 -- 10.20.193.78:0/875177988 shutdown_connections 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.557+0000 7fe1c86bf640 1 -- 10.20.193.78:0/875177988 wait complete. 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient: build_initial_monmap 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient: monmap: 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:35.573 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 1 Processor -- start 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 1 -- start start 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient: init 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006c890 0x7fe1c01cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient(hunting): picked mon.noname-c con 0x7fe1c006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe1c006d1b0 0x7fe1c01cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient(hunting): picked mon.noname-b con 0x7fe1c006d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:35.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe1c009c4a0 0x7fe1c009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient(hunting): picked mon.noname-a con 0x7fe1c009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c00a01a0 con 0x7fe1c009c4a0 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c00a0020 con 0x7fe1c006d1b0 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c009be20 con 0x7fe1c006c890 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c86bf640 10 monclient(hunting): authenticate will time out at 1541.112593s 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006c890 0x7fe1c01cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006c890 0x7fe1c01cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56372/0 (socket says 10.20.193.78:56372) 2026-02-17T06:01:35.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 -- 10.20.193.78:0/192264812 learned_addr learned my addr 10.20.193.78:0/192264812 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient(hunting): get_auth_request con 0x7fe1c006c890 auth_method 0 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe1c009c4a0 0x7fe1c009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 10 monclient(hunting): get_auth_request con 0x7fe1c009c4a0 auth_method 0 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:35.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient(hunting): handle_auth_done global_id 144608 payload 824 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 -- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe1c006d1b0 msgr2=0x7fe1c01cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c5c33640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe1c006d1b0 0x7fe1c01cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe1c006d1b0 0x7fe1c01cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 -- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe1c009c4a0 msgr2=0x7fe1c009db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe1c009c4a0 0x7fe1c009db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient: _finish_hunting 0 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 monclient: found mon.noname-c 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:35.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6434640 1 -- 10.20.193.78:0/192264812 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c0127220 con 0x7fe1c006c890 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c5c33640 10 monclient: get_auth_request con 0x7fe1c006d1b0 auth_method 0 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c5c33640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe1c006d1b0 0x7fe1c01cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.558+0000 7fe1c6c35640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe1c009c4a0 0x7fe1c009db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.560+0000 7fe1c6434640 1 -- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006c890 msgr2=0x7fe1c01cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.560+0000 7fe1c6434640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006c890 0x7fe1c01cf350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000e810 tx=0x7fe1b000ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 1 -- 10.20.193.78:0/192264812 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0017070 con 0x7fe1c006c890 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: dump: 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:35.578 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:35.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 1 -- 10.20.193.78:0/192264812 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b000cae0 con 0x7fe1c006c890 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 5 monclient: authenticate success, global_id 144608 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 10 monclient: _renew_subs 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c0127500 con 0x7fe1c006c890 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 1 -- 10.20.193.78:0/192264812 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000f040 con 0x7fe1c006c890 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c4c31640 4 set_mon_vals no callback set 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:35.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 10 monclient: dump: 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 10 monclient: _renew_subs 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:35.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c00a2030 con 0x7fe1c006c890 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1a6ffd640 1 -- 10.20.193.78:0/192264812 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe1b0012050 con 0x7fe1c006c890 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.561+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c0074980 con 0x7fe1c006c890 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.562+0000 7fe1a6ffd640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe19003df30 0x7fe1900403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.562+0000 7fe1a6ffd640 1 -- 10.20.193.78:0/192264812 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(560..560 src has 335..560) ==== 4763+0+0 (secure 0 0 0) 0x7fe1b0052510 con 0x7fe1c006c890 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.562+0000 7fe1c5c33640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe19003df30 0x7fe1900403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.562+0000 7fe1c5c33640 10 monclient: get_auth_request con 0x7fe19003df30 auth_method 0 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.564+0000 7fe1a6ffd640 1 -- 10.20.193.78:0/192264812 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe1b0050f20 con 0x7fe1c006c890 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.564+0000 7fe1a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:35.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.564+0000 7fe1a6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:35.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.565+0000 7fe1c5c33640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe19003df30 0x7fe1900403f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe1c009f2d0 tx=0x7fe1b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.677+0000 7fe1c86bf640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:01:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.677+0000 7fe1c86bf640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:01:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.677+0000 7fe1c86bf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:35.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.677+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fe1c0126050 con 0x7fe1c006c890 2026-02-17T06:01:35.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.677+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.677+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=12,logm=47} crc {auth=546595944,config=692619927,health=828959108,kv=449989585,logm=3886605230}) 2026-02-17T06:01:35.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.677+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=26,config=2,health=11,kv=12,logm=49} crc {auth=2087684699,config=692619927,health=828959108,kv=449989585,logm=3777215156}) 2026-02-17T06:01:35.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.677+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.677+0000 7fe1a6ffd640 1 -- 10.20.193.78:0/192264812 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fe1b0052030 con 0x7fe1c006c890 2026-02-17T06:01:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.677+0000 7fe1a6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:01:35.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.677+0000 7fe1a6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:35.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.677+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=12,logm=47} crc {auth=546595944,config=692619927,health=828959108,kv=449989585,logm=3886605230}) 2026-02-17T06:01:35.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.677+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=24,config=2,health=11,kv=12,logm=51} crc {auth=2980876782,config=692619927,health=4286341386,kv=449989585,logm=2172184543}) 2026-02-17T06:01:35.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1274436248}) 2026-02-17T06:01:35.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1559890367}) 2026-02-17T06:01:35.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1274436248}) 2026-02-17T06:01:35.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1277504148}) 2026-02-17T06:01:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe19003df30 msgr2=0x7fe1900403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe19003df30 0x7fe1900403f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe1c009f2d0 tx=0x7fe1b4009040 comp rx=0 tx=0).stop 2026-02-17T06:01:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 10 monclient: shutdown 2026-02-17T06:01:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006c890 msgr2=0x7fe1c01cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:35.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006c890 0x7fe1c01cf350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000e810 tx=0x7fe1b000ece0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2179349895}) 2026-02-17T06:01:35.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1511026443}) 2026-02-17T06:01:35.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 shutdown_connections 2026-02-17T06:01:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe1c009c4a0 0x7fe1c009db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe19003df30 0x7fe1900403f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2179349895}) 2026-02-17T06:01:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe1c006d1b0 0x7fe1c01cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 --2- 10.20.193.78:0/192264812 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe1c006c890 0x7fe1c01cf350 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:35.686 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.678+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1343479854}) 2026-02-17T06:01:35.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 >> 10.20.193.78:0/192264812 conn(0x7fe1c00a6620 msgr2=0x7fe1c0071920 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:35.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 shutdown_connections 2026-02-17T06:01:35.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:35.678+0000 7fe1c86bf640 1 -- 10.20.193.78:0/192264812 wait complete. 2026-02-17T06:01:35.695 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.695 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1381883029}) 2026-02-17T06:01:35.695 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2208691212}) 2026-02-17T06:01:35.695 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.695 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1381883029}) 2026-02-17T06:01:35.695 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4177016926}) 2026-02-17T06:01:35.695 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493050219}) 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3928017224}) 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493050219}) 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.679+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3934534385}) 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3619863890}) 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1343377331}) 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3619863890}) 2026-02-17T06:01:35.696 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=577345567}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2783768348}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=824780969}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2783768348}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023520610}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932632801}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079613949}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932632801}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.680+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3830815998}) 2026-02-17T06:01:35.697 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2953714312}) 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1835658936}) 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2953714312}) 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4211013482}) 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236632776}) 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=701383210}) 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236632776}) 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2350351640}) 2026-02-17T06:01:35.698 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1381950649}) 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2187372808}) 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1381950649}) 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.681+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2305451803}) 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866353837}) 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3577509827}) 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866353837}) 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2820620294}) 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.699 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3904223966}) 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1392920473}) 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3904223966}) 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442600960}) 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863126751}) 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4073913252}) 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863126751}) 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3591032787}) 2026-02-17T06:01:35.700 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3411339326}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1400358576}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3411339326}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.682+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2080436627}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3685225061}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172161709}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3685225061}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1471749123}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4043641419}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489228615}) 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.701 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4043641419}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1882721937}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3248961488}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3627561130}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3248961488}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.683+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855093235}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3445223104}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4118651958}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3445223104}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=189385587}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1182374119}) 2026-02-17T06:01:35.702 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1812362752}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1182374119}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3736928807}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3702542775}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2874017978}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3702542775}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.684+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3099998423}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2506517510}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4189515984}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2506517510}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454200155}) 2026-02-17T06:01:35.703 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3395138632}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2914037817}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3395138632}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1592600637}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1601197369}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131638721}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1601197369}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.685+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1672140266}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2348146717}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3694755521}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2348146717}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3273410119}) 2026-02-17T06:01:35.704 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.705 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=259630927}) 2026-02-17T06:01:35.705 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=616614320}) 2026-02-17T06:01:35.705 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:35.705 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=259630927}) 2026-02-17T06:01:35.705 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:35.686+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1534150748}) 2026-02-17T06:01:35.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:01:36.007 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.006+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.008 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.006+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1552459317}) 2026-02-17T06:01:36.008 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.006+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3150415061}) 2026-02-17T06:01:36.008 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.006+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.008 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.006+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1552459317}) 2026-02-17T06:01:36.008 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.006+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1954414379}) 2026-02-17T06:01:36.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.231+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.231+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=915775448}) 2026-02-17T06:01:36.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.231+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=833593389}) 2026-02-17T06:01:36.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.231+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.231+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=915775448}) 2026-02-17T06:01:36.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.231+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=5943249}) 2026-02-17T06:01:36.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.234+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.234+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4097979376}) 2026-02-17T06:01:36.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.234+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=261260976}) 2026-02-17T06:01:36.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.234+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.234+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4097979376}) 2026-02-17T06:01:36.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.234+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2594848719}) 2026-02-17T06:01:36.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.235+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.235+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4149018197}) 2026-02-17T06:01:36.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.235+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1530709006}) 2026-02-17T06:01:36.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.235+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.235+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4149018197}) 2026-02-17T06:01:36.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.235+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2303773027}) 2026-02-17T06:01:36.237 INFO:tasks.rados.rados.0.trial115.stdout:2610: done (3 left) 2026-02-17T06:01:36.238 INFO:tasks.rados.rados.0.trial115.stdout:2611: done (2 left) 2026-02-17T06:01:36.238 INFO:tasks.rados.rados.0.trial115.stdout:2612: done (1 left) 2026-02-17T06:01:36.238 INFO:tasks.rados.rados.0.trial115.stdout:2613: done (0 left) 2026-02-17T06:01:36.238 INFO:tasks.rados.rados.0.trial115.stdout:2614: snap_create 2026-02-17T06:01:36.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.258+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.258+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3135797345}) 2026-02-17T06:01:36.259 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.258+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4181361101}) 2026-02-17T06:01:36.260 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.258+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.260 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.258+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3135797345}) 2026-02-17T06:01:36.260 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.258+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3203668401}) 2026-02-17T06:01:36.262 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.261+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.262 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.261+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137377634}) 2026-02-17T06:01:36.263 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.261+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2754962731}) 2026-02-17T06:01:36.263 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.261+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.263 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.261+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137377634}) 2026-02-17T06:01:36.263 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.261+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3436871137}) 2026-02-17T06:01:36.266 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.265+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.266 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.265+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2371155037}) 2026-02-17T06:01:36.266 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.265+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2235214596}) 2026-02-17T06:01:36.267 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.265+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.267 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.265+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2371155037}) 2026-02-17T06:01:36.267 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.265+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=979773880}) 2026-02-17T06:01:36.268 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.266+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.268 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.266+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1234311867}) 2026-02-17T06:01:36.268 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.266+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2806777059}) 2026-02-17T06:01:36.268 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.266+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.268 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.266+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1234311867}) 2026-02-17T06:01:36.268 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.266+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3974572165}) 2026-02-17T06:01:36.340 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.339+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.341 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.339+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2387771863}) 2026-02-17T06:01:36.341 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.339+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2255685828}) 2026-02-17T06:01:36.341 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.339+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.341 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.339+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2387771863}) 2026-02-17T06:01:36.341 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.339+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330960317}) 2026-02-17T06:01:36.455 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.454+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.455 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.454+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3551959850}) 2026-02-17T06:01:36.456 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.454+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=204525647}) 2026-02-17T06:01:36.456 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.454+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.456 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.454+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3551959850}) 2026-02-17T06:01:36.456 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.454+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1651801960}) 2026-02-17T06:01:36.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.643+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.643+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797074644}) 2026-02-17T06:01:36.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.643+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3336667900}) 2026-02-17T06:01:36.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.643+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:36.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.643+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797074644}) 2026-02-17T06:01:36.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:36.643+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3251016431}) 2026-02-17T06:01:36.706 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:01:36.706 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:01:36.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.772+0000 7f1b030f3640 10 monclient: get_monmap_and_config 2026-02-17T06:01:36.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.772+0000 7f1b030f3640 10 monclient: build_initial_monmap 2026-02-17T06:01:36.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.772+0000 7f1b030f3640 10 monclient: monmap: 2026-02-17T06:01:36.773 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:36.773 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.773 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:36.773581+0000 2026-02-17T06:01:36.773 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:36.773581+0000 2026-02-17T06:01:36.774 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:36.774 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.774 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:36.774 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:36.774 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:36.774 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:36.774 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:36.774 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.772+0000 7f1b030f3640 1 Processor -- start 2026-02-17T06:01:36.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.772+0000 7f1b030f3640 1 -- start start 2026-02-17T06:01:36.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.772+0000 7f1b030f3640 10 monclient: init 2026-02-17T06:01:36.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:36.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1afc06c890 0x7f1afc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient(hunting): picked mon.noname-a con 0x7f1afc06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1afc09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient(hunting): picked mon.noname-b con 0x7f1afc06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1afc06daf0 con 0x7f1afc09c4a0 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f1afc0a0880 con 0x7f1afc06c890 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1afc09cc80 con 0x7f1afc06d1b0 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b030f3640 10 monclient(hunting): authenticate will time out at 1542.327111s 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1afc06c890 0x7f1afc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1afc06c890 0x7f1afc06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49778/0 (socket says 10.20.193.78:49778) 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 1 -- 10.20.193.78:0/829650877 learned_addr learned my addr 10.20.193.78:0/829650877 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient(hunting): get_auth_request con 0x7f1afc06c890 auth_method 0 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 10 monclient(hunting): get_auth_request con 0x7f1afc06d1b0 auth_method 0 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient(hunting): handle_auth_done global_id 184230 payload 274 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 1 -- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 msgr2=0x7f1afc135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 1 --2- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 1 -- 10.20.193.78:0/829650877 >> v1:10.20.193.115:6789/0 conn(0x7f1afc09c4a0 legacy=0x7f1afc134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient: _finish_hunting 0 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 1 monclient: found mon.noname-a 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b00e68640 1 -- 10.20.193.78:0/829650877 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc09cf60 con 0x7f1afc06c890 2026-02-17T06:01:36.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.773+0000 7f1b01669640 1 --2- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.775+0000 7f1b00e68640 1 -- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc06c890 msgr2=0x7f1afc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.775+0000 7f1b00e68640 1 --2- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc06c890 0x7f1afc06cc70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1afc06dcc0 tx=0x7f1aec02ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80763979a1cadf6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 1 -- 10.20.193.78:0/829650877 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec03c070 con 0x7f1afc06c890 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: dump: 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.785 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 1 -- 10.20.193.78:0/829650877 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aec02fcc0 con 0x7f1afc06c890 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 1 -- 10.20.193.78:0/829650877 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec037660 con 0x7f1afc06c890 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 5 monclient: authenticate success, global_id 184230 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 4 set_mon_vals no callback set 2026-02-17T06:01:36.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1af37fe640 10 monclient: dump: 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 10 monclient: shutdown 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:36.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 1 -- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc06c890 msgr2=0x7f1afc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 1 --2- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc06c890 0x7f1afc06cc70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1afc06dcc0 tx=0x7f1aec02ee80 comp rx=0 tx=0).stop 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 1 -- 10.20.193.78:0/829650877 shutdown_connections 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 1 --2- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 1 --2- 10.20.193.78:0/829650877 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc06c890 0x7f1afc06cc70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 1 -- 10.20.193.78:0/829650877 >> 10.20.193.78:0/829650877 conn(0x7f1afc0a6620 msgr2=0x7f1afc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 1 -- 10.20.193.78:0/829650877 shutdown_connections 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 1 -- 10.20.193.78:0/829650877 wait complete. 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 10 monclient: build_initial_monmap 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.776+0000 7f1b030f3640 10 monclient: monmap: 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:36.788 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 1 Processor -- start 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 1 -- start start 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient: init 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1afc06c890 0x7f1afc1cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient(hunting): picked mon.noname-b con 0x7f1afc06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc1344a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient(hunting): picked mon.noname-c con 0x7f1afc06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:36.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc09c4a0 0x7f1afc1cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient(hunting): picked mon.noname-a con 0x7f1afc09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc0a01a0 con 0x7f1afc09c4a0 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc0a0020 con 0x7f1afc06c890 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1afc09be20 con 0x7f1afc06d1b0 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b030f3640 10 monclient(hunting): authenticate will time out at 1542.331463s 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc09c4a0 0x7f1afc1cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1af3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc1344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc09c4a0 0x7f1afc1cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49788/0 (socket says 10.20.193.78:49788) 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 -- 10.20.193.78:0/479116858 learned_addr learned my addr 10.20.193.78:0/479116858 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient(hunting): get_auth_request con 0x7f1afc09c4a0 auth_method 0 2026-02-17T06:01:36.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1af3fff640 10 monclient(hunting): get_auth_request con 0x7f1afc06d1b0 auth_method 0 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b00e68640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1afc06c890 0x7f1afc1cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b00e68640 10 monclient(hunting): get_auth_request con 0x7f1afc06c890 auth_method 0 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b00e68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b00e68640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b00e68640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient(hunting): handle_auth_done global_id 184233 payload 824 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 -- 10.20.193.78:0/479116858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 msgr2=0x7f1afc1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc1344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 -- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1afc06c890 msgr2=0x7f1afc1cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1afc06c890 0x7f1afc1cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient: _finish_hunting 0 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 monclient: found mon.noname-a 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.777+0000 7f1b01669640 1 -- 10.20.193.78:0/479116858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc127be0 con 0x7f1afc09c4a0 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1b01669640 1 -- 10.20.193.78:0/479116858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc09c4a0 msgr2=0x7f1afc1cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:36.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1b01669640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc09c4a0 0x7f1afc1cc120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1af800efc0 tx=0x7f1af800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 1 -- 10.20.193.78:0/479116858 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8018030 con 0x7f1afc09c4a0 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: dump: 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:36.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 1 -- 10.20.193.78:0/479116858 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af800ce00 con 0x7f1afc09c4a0 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 1 -- 10.20.193.78:0/479116858 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8003cb0 con 0x7f1afc09c4a0 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af2ffd640 4 set_mon_vals no callback set 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1af17fa640 10 monclient: dump: 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:36.794 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1b030f3640 5 monclient: authenticate success, global_id 184233 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1b030f3640 10 monclient: _renew_subs 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1b030f3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.779+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc126070 con 0x7f1afc09c4a0 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1b030f3640 10 monclient: _renew_subs 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1b030f3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc0a2680 con 0x7f1afc09c4a0 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1b030f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1b030f3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1b030f3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc068410 con 0x7f1afc09c4a0 2026-02-17T06:01:36.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1af17fa640 1 -- 10.20.193.78:0/479116858 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1af8014690 con 0x7f1afc09c4a0 2026-02-17T06:01:36.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1af17fa640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1acc03df00 0x7f1acc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.780+0000 7f1af17fa640 1 -- 10.20.193.78:0/479116858 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(561..561 src has 335..561) ==== 4735+0+0 (secure 0 0 0) 0x7f1af80533c0 con 0x7f1afc09c4a0 2026-02-17T06:01:36.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.781+0000 7f1b00e68640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1acc03df00 0x7f1acc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.781+0000 7f1b00e68640 10 monclient: get_auth_request con 0x7f1acc03df00 auth_method 0 2026-02-17T06:01:36.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.783+0000 7f1af17fa640 1 -- 10.20.193.78:0/479116858 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1afc068410 con 0x7f1afc09c4a0 2026-02-17T06:01:36.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.783+0000 7f1af17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:36.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.783+0000 7f1af17fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:36.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.783+0000 7f1b00e68640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1acc03df00 0x7f1acc0403c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1afc06f340 tx=0x7f1aec033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:36.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.896+0000 7f1b030f3640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:01:36.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.896+0000 7f1b030f3640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:36.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.896+0000 7f1b030f3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:36.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.896+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1afc074980 con 0x7f1afc09c4a0 2026-02-17T06:01:36.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.896+0000 7f1af17fa640 1 -- 10.20.193.78:0/479116858 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f1af8022070 con 0x7f1afc09c4a0 2026-02-17T06:01:36.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.896+0000 7f1af17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:36.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.896+0000 7f1af17fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:36.899 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":180,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:01:36.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1acc03df00 msgr2=0x7f1acc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1acc03df00 0x7f1acc0403c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1afc06f340 tx=0x7f1aec033000 comp rx=0 tx=0).stop 2026-02-17T06:01:36.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 10 monclient: shutdown 2026-02-17T06:01:36.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:36.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc09c4a0 msgr2=0x7f1afc1cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc09c4a0 0x7f1afc1cc120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1af800efc0 tx=0x7f1af800c490 comp rx=0 tx=0).stop 2026-02-17T06:01:36.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 shutdown_connections 2026-02-17T06:01:36.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1afc09c4a0 0x7f1afc1cc120 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1afc06d1b0 0x7f1afc1344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1acc03df00 0x7f1acc0403c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 --2- 10.20.193.78:0/479116858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1afc06c890 0x7f1afc1cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 >> 10.20.193.78:0/479116858 conn(0x7f1afc0a6620 msgr2=0x7f1afc06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:36.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 shutdown_connections 2026-02-17T06:01:36.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.897+0000 7f1b030f3640 1 -- 10.20.193.78:0/479116858 wait complete. 2026-02-17T06:01:36.906 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:01:36.907 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:01:36.907 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:01:36.907 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:01:36.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.973+0000 7f2b0d491640 10 monclient: get_monmap_and_config 2026-02-17T06:01:36.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.973+0000 7f2b0d491640 10 monclient: build_initial_monmap 2026-02-17T06:01:36.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.973+0000 7f2b0d491640 10 monclient: monmap: 2026-02-17T06:01:36.975 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:36.975 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.975 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:36.975125+0000 2026-02-17T06:01:36.975 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:36.975125+0000 2026-02-17T06:01:36.975 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:36.975 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.976 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:36.976 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:36.976 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:36.976 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:36.976 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:36.976 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 1 Processor -- start 2026-02-17T06:01:36.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 1 -- start start 2026-02-17T06:01:36.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient: init 2026-02-17T06:01:36.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:36.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient: _add_conns ranks=[1,0,3,4,2] 2026-02-17T06:01:36.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2b0806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:36.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2b0809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:36.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2b0806d380 0x7f2b0813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient(hunting): picked mon.noname-a con 0x7f2b0806d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b0806daf0 con 0x7f2b0809c4a0 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b0809cc80 con 0x7f2b0806c890 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f2b080a0880 con 0x7f2b0806d380 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b0d491640 10 monclient(hunting): authenticate will time out at 1542.528741s 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2b0806d380 0x7f2b0813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f2b0806d380 0x7f2b0813ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49802/0 (socket says 10.20.193.78:49802) 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b067fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f2b0809c4a0 0x7f2b0813b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:34722/0 (socket says 10.20.193.78:34722) 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 1 -- 10.20.193.78:0/1891431279 learned_addr learned my addr 10.20.193.78:0/1891431279 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:36.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 10 monclient(hunting): get_auth_request con 0x7f2b0806d380 auth_method 0 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.974+0000 7f2b077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.975+0000 7f2b077fe640 10 monclient(hunting): handle_auth_done global_id 184236 payload 274 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.975+0000 7f2b077fe640 1 -- 10.20.193.78:0/1891431279 >> v1:10.20.193.115:6789/0 conn(0x7f2b0806c890 legacy=0x7f2b0806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.975+0000 7f2b077fe640 1 -- 10.20.193.78:0/1891431279 >> v1:10.20.193.78:6789/0 conn(0x7f2b0809c4a0 legacy=0x7f2b0813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.975+0000 7f2b077fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.975+0000 7f2b077fe640 1 monclient: found mon.noname-a 2026-02-17T06:01:36.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.975+0000 7f2b077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.975+0000 7f2b077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.975+0000 7f2b077fe640 1 -- 10.20.193.78:0/1891431279 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0809cf60 con 0x7f2b0806d380 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b077fe640 1 -- 10.20.193.78:0/1891431279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0806d380 msgr2=0x7f2b0813ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b077fe640 1 --2- 10.20.193.78:0/1891431279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0806d380 0x7f2b0813ead0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2b08074de0 tx=0x7f2af802f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fbcd548ade0cc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 1 -- 10.20.193.78:0/1891431279 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af802fe60 con 0x7f2b0806d380 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: dump: 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.988 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 1 -- 10.20.193.78:0/1891431279 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af80373f0 con 0x7f2b0806d380 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b0d491640 5 monclient: authenticate success, global_id 184236 2026-02-17T06:01:36.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b0d491640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 1 -- 10.20.193.78:0/1891431279 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af80377c0 con 0x7f2b0806d380 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b0d491640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b0d491640 4 set_mon_vals no callback set 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.977+0000 7f2b05ffb640 10 monclient: dump: 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.990 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient: shutdown 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- 10.20.193.78:0/1891431279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0806d380 msgr2=0x7f2b0813ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 --2- 10.20.193.78:0/1891431279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0806d380 0x7f2b0813ead0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2b08074de0 tx=0x7f2af802f230 comp rx=0 tx=0).stop 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- 10.20.193.78:0/1891431279 shutdown_connections 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 --2- 10.20.193.78:0/1891431279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0806d380 0x7f2b0813ead0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- 10.20.193.78:0/1891431279 >> 10.20.193.78:0/1891431279 conn(0x7f2b080a6620 msgr2=0x7f2b080a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- 10.20.193.78:0/1891431279 shutdown_connections 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- 10.20.193.78:0/1891431279 wait complete. 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient: build_initial_monmap 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient: monmap: 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:36.992 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 Processor -- start 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- start start 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient: init 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0806c890 0x7f2b08131dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient(hunting): picked mon.noname-b con 0x7f2b0806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0806d380 0x7f2b08132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient(hunting): picked mon.noname-c con 0x7f2b0806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0809c4a0 0x7f2b08132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:36.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient(hunting): picked mon.noname-a con 0x7f2b0809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0809bfa0 con 0x7f2b0809c4a0 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0809be20 con 0x7f2b0806c890 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0813f010 con 0x7f2b0806d380 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.978+0000 7f2b0d491640 10 monclient(hunting): authenticate will time out at 1542.532958s 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0809c4a0 0x7f2b08132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0809c4a0 0x7f2b08132840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49810/0 (socket says 10.20.193.78:49810) 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 -- 10.20.193.78:0/3102956791 learned_addr learned my addr 10.20.193.78:0/3102956791 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:36.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient(hunting): get_auth_request con 0x7f2b0809c4a0 auth_method 0 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b067fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0806d380 0x7f2b08132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b067fc640 10 monclient(hunting): get_auth_request con 0x7f2b0806d380 auth_method 0 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b067fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b067fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b067fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b067fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:36.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b067fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b067fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient(hunting): handle_auth_done global_id 184239 payload 824 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0806d380 msgr2=0x7f2b08132300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2b0806d380 0x7f2b08132300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b06ffd640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0806c890 0x7f2b08131dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0806c890 msgr2=0x7f2b08131dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0806c890 0x7f2b08131dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 monclient: found mon.noname-a 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:01:36.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b077fe640 1 -- 10.20.193.78:0/3102956791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0813cef0 con 0x7f2b0809c4a0 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b06ffd640 10 monclient: get_auth_request con 0x7f2b0806c890 auth_method 0 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.979+0000 7f2b06ffd640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2b0806c890 0x7f2b08131dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b077fe640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0809c4a0 msgr2=0x7f2b08132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b077fe640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0809c4a0 0x7f2b08132840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2af80093e0 tx=0x7f2af8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8046020 con 0x7f2b0809c4a0 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: dump: 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.997 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af8041870 con 0x7f2b0809c4a0 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:36.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8041c40 con 0x7f2b0809c4a0 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b057fa640 4 set_mon_vals no callback set 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2ae77fe640 10 monclient: dump: 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:36.999 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b0d491640 5 monclient: authenticate success, global_id 184239 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b0d491640 10 monclient: _renew_subs 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b0d491640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0806fc90 con 0x7f2b0809c4a0 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b0d491640 10 monclient: _renew_subs 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b0d491640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.981+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0813c930 con 0x7f2b0809c4a0 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b0d491640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b0d491640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08134ff0 0x7f2b081353d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b0d491640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0809d8a0 con 0x7f2b08134ff0 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2af8042400 con 0x7f2b0809c4a0 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08134ff0 0x7f2b081353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 10 monclient: get_auth_request con 0x7f2b08134ff0 auth_method 0 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 10 monclient(hunting): handle_auth_done global_id 184242 payload 824 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2ae77fe640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ad403deb0 0x7f2ad4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b06ffd640 1 -- 10.20.193.78:0/3102956791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b0806cc70 con 0x7f2b08134ff0 2026-02-17T06:01:37.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(561..561 src has 335..561) ==== 4735+0+0 (secure 0 0 0) 0x7f2af8078550 con 0x7f2b0809c4a0 2026-02-17T06:01:37.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b067fc640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ad403deb0 0x7f2ad4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.982+0000 7f2b067fc640 10 monclient: get_auth_request con 0x7f2ad403deb0 auth_method 0 2026-02-17T06:01:37.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2b06ffd640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08134ff0 msgr2=0x7f2b081353d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2b06ffd640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08134ff0 0x7f2b081353d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d720 tx=0x7f2afc00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00bb30 con 0x7f2b08134ff0 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 10 monclient: dump: 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.005 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2afc00bb30 con 0x7f2b08134ff0 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08134ff0 msgr2=0x7f2b081353d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08134ff0 0x7f2b081353d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d720 tx=0x7f2afc00dbf0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08134ff0 msgr2=0x7f2b081353d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.984+0000 7f2ae77fe640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08134ff0 0x7f2b081353d0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d720 tx=0x7f2afc00dbf0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.985+0000 7f2b067fc640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ad403deb0 0x7f2ad4040370 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2b081cdf10 tx=0x7f2af0007f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.987+0000 7f2b0d491640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.987+0000 7f2b0d491640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08135910 0x7f2b081d8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.987+0000 7f2b0d491640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.987+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0813b4e0 con 0x7f2b08135910 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08135910 0x7f2b081d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 10 monclient: get_auth_request con 0x7f2b08135910 auth_method 0 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 10 monclient(hunting): handle_auth_done global_id 184245 payload 824 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.988+0000 7f2b06ffd640 1 -- 10.20.193.78:0/3102956791 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2b080744d0 con 0x7f2b08135910 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b06ffd640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08135910 msgr2=0x7f2b081d8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b06ffd640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08135910 0x7f2b081d8420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d720 tx=0x7f2afc003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc005490 con 0x7f2b08135910 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 10 monclient: dump: 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f2afc005490 con 0x7f2b08135910 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08135910 msgr2=0x7f2b081d8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08135910 0x7f2b081d8420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d720 tx=0x7f2afc003860 comp rx=0 tx=0).stop 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08135910 msgr2=0x7f2b081d8420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2ae77fe640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b08135910 0x7f2b081d8420 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d720 tx=0x7f2afc003860 comp rx=0 tx=0).stop 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:37.009 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 180, 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 61771, 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.010 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.011 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:37.012 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.013 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:37.014 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:37.015 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:37.016 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:37.017 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:37.018 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:01:37.019 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.020 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.020 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.020 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:37.020 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ad403deb0 msgr2=0x7f2ad4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2ad403deb0 0x7f2ad4040370 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2b081cdf10 tx=0x7f2af0007f60 comp rx=0 tx=0).stop 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 10 monclient: shutdown 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0809c4a0 msgr2=0x7f2b08132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 1 --2- 10.20.193.78:0/3102956791 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2b0809c4a0 0x7f2b08132840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2af80093e0 tx=0x7f2af8004b40 comp rx=0 tx=0).stop 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b077fe640 1 -- 10.20.193.78:0/3102956791 reap_dead start 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b077fe640 1 -- 10.20.193.78:0/3102956791 reap_dead start 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 shutdown_connections 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 >> 10.20.193.78:0/3102956791 conn(0x7f2b080a6620 msgr2=0x7f2b080a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 shutdown_connections 2026-02-17T06:01:37.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:36.990+0000 7f2b0d491640 1 -- 10.20.193.78:0/3102956791 wait complete. 2026-02-17T06:01:37.022 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:01:37.022 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:01:37.022 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:01:37.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.089+0000 7f0ee38e8640 10 monclient: get_monmap_and_config 2026-02-17T06:01:37.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.089+0000 7f0ee38e8640 10 monclient: build_initial_monmap 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.089+0000 7f0ee38e8640 10 monclient: monmap: 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:37.091105+0000 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:37.091105+0000 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:37.091 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 1 Processor -- start 2026-02-17T06:01:37.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 1 -- start start 2026-02-17T06:01:37.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient: init 2026-02-17T06:01:37.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:37.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2026-02-17T06:01:37.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient(hunting): picked mon.noname-b con 0x7f0edc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0edc09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0edc06d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0edc135c10 con 0x7f0edc06d1b0 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0edc06daf0 con 0x7f0edc09c4a0 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc0a0880 con 0x7f0edc06c890 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee38e8640 10 monclient(hunting): authenticate will time out at 1542.644715s 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee1e5e640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f0edc06d1b0 0x7f0edc135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:34724/0 (socket says 10.20.193.78:34724) 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee1e5e640 1 -- 10.20.193.78:0/190460622 learned_addr learned my addr 10.20.193.78:0/190460622 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 10 monclient(hunting): get_auth_request con 0x7f0edc06c890 auth_method 0 2026-02-17T06:01:37.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 10 monclient(hunting): handle_auth_done global_id 144620 payload 274 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.090+0000 7f0ee165d640 1 -- 10.20.193.78:0/190460622 >> v1:10.20.193.115:6789/0 conn(0x7f0edc09c4a0 legacy=0x7f0edc134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.091+0000 7f0ee165d640 1 -- 10.20.193.78:0/190460622 >> v1:10.20.193.78:6789/0 conn(0x7f0edc06d1b0 legacy=0x7f0edc135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.091+0000 7f0ee165d640 10 monclient: _finish_hunting 0 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.091+0000 7f0ee165d640 1 monclient: found mon.noname-b 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.091+0000 7f0ee165d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.091+0000 7f0ee165d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:01:37.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.091+0000 7f0ee165d640 1 -- 10.20.193.78:0/190460622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc09cc80 con 0x7f0edc06c890 2026-02-17T06:01:37.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee165d640 1 -- 10.20.193.78:0/190460622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 msgr2=0x7f0edc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee165d640 1 --2- 10.20.193.78:0/190460622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc06cc70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0098d0 tx=0x7f0ecc02ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9c9bfcbf1a09b5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 1 -- 10.20.193.78:0/190460622 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc03c070 con 0x7f0edc06c890 2026-02-17T06:01:37.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: dump: 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: _finish_auth 0 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:37.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 5 monclient: authenticate success, global_id 144620 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 1 -- 10.20.193.78:0/190460622 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ecc02fcc0 con 0x7f0edc06c890 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 1 -- 10.20.193.78:0/190460622 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc037660 con 0x7f0edc06c890 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 4 set_mon_vals no callback set 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ecbfff640 10 monclient: dump: 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.103 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 10 monclient: shutdown 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 1 -- 10.20.193.78:0/190460622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 msgr2=0x7f0edc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 1 --2- 10.20.193.78:0/190460622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc06cc70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0098d0 tx=0x7f0ecc02ed60 comp rx=0 tx=0).stop 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 1 -- 10.20.193.78:0/190460622 shutdown_connections 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 1 --2- 10.20.193.78:0/190460622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc06cc70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 1 -- 10.20.193.78:0/190460622 >> 10.20.193.78:0/190460622 conn(0x7f0edc0a6620 msgr2=0x7f0edc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 1 -- 10.20.193.78:0/190460622 shutdown_connections 2026-02-17T06:01:37.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.093+0000 7f0ee38e8640 1 -- 10.20.193.78:0/190460622 wait complete. 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient: build_initial_monmap 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient: monmap: 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 1 Processor -- start 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 1 -- start start 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient: init 2026-02-17T06:01:37.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc1341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient(hunting): picked mon.noname-c con 0x7f0edc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0edc06d1b0 0x7f0edc134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient(hunting): picked mon.noname-b con 0x7f0edc06d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0edc09c4a0 0x7f0edc09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient(hunting): picked mon.noname-a con 0x7f0edc09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc0a0020 con 0x7f0edc09c4a0 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc06d600 con 0x7f0edc06d1b0 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc0a01a0 con 0x7f0edc06c890 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:37.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee38e8640 10 monclient(hunting): authenticate will time out at 1542.648788s 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc1341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee1e5e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0edc09c4a0 0x7f0edc09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc1341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:56414/0 (socket says 10.20.193.78:56414) 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 1 -- 10.20.193.78:0/4023847106 learned_addr learned my addr 10.20.193.78:0/4023847106 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 10 monclient(hunting): get_auth_request con 0x7f0edc06c890 auth_method 0 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee1e5e640 10 monclient(hunting): get_auth_request con 0x7f0edc09c4a0 auth_method 0 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee1e5e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee1e5e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee1e5e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee1e5e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee1e5e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee1e5e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 10 monclient(hunting): handle_auth_done global_id 144623 payload 824 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee0e5c640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0edc06d1b0 0x7f0edc134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.094+0000 7f0ee165d640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0edc06d1b0 msgr2=0x7f0edc134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee165d640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0edc06d1b0 0x7f0edc134730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee165d640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0edc09c4a0 msgr2=0x7f0edc09db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee165d640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0edc09c4a0 0x7f0edc09db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee165d640 10 monclient: _finish_hunting 0 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee165d640 1 monclient: found mon.noname-c 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee165d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee165d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee165d640 1 -- 10.20.193.78:0/4023847106 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc09d810 con 0x7f0edc06c890 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee0e5c640 10 monclient: get_auth_request con 0x7f0edc06d1b0 auth_method 0 2026-02-17T06:01:37.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.095+0000 7f0ee0e5c640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0edc06d1b0 0x7f0edc134730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee165d640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 msgr2=0x7f0edc1341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee165d640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc1341f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009820 tx=0x7f0ecc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc046070 con 0x7f0edc06c890 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: dump: 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.109 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ecc041870 con 0x7f0edc06c890 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee38e8640 5 monclient: authenticate success, global_id 144623 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee38e8640 10 monclient: _renew_subs 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee38e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc0a26d0 con 0x7f0edc06c890 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc03c040 con 0x7f0edc06c890 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ecb7fe640 4 set_mon_vals no callback set 2026-02-17T06:01:37.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 10 monclient: dump: 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee38e8640 10 monclient: _renew_subs 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee38e8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc127930 con 0x7f0edc06c890 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0ecc041c60 con 0x7f0edc06c890 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.097+0000 7f0ee38e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:37.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee38e8640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc1cf5d0 0x7f0edc11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee38e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc1d02f0 con 0x7f0edc1cf5d0 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc1cf5d0 0x7f0edc11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 10 monclient: get_auth_request con 0x7f0edc1cf5d0 auth_method 0 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ec9ffb640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0eac03df00 0x7f0eac0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 10 monclient(hunting): handle_auth_done global_id 144626 payload 824 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee0e5c640 1 -- 10.20.193.78:0/4023847106 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0edc127750 con 0x7f0edc1cf5d0 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(561..561 src has 335..561) ==== 4735+0+0 (secure 0 0 0) 0x7f0ecc0783e0 con 0x7f0edc06c890 2026-02-17T06:01:37.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee1e5e640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0eac03df00 0x7f0eac0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.098+0000 7f0ee1e5e640 10 monclient: get_auth_request con 0x7f0eac03df00 auth_method 0 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ee0e5c640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc1cf5d0 msgr2=0x7f0edc11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ee0e5c640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc1cf5d0 0x7f0edc11cc70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000d6f0 tx=0x7f0ed000dbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000bb30 con 0x7f0edc1cf5d0 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 10 monclient: dump: 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.113 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.114 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f0ed000bb30 con 0x7f0edc1cf5d0 2026-02-17T06:01:37.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:37.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:37.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc1cf5d0 msgr2=0x7f0edc11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc1cf5d0 0x7f0edc11cc70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000d6f0 tx=0x7f0ed000dbc0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc1cf5d0 msgr2=0x7f0edc11cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:37.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.100+0000 7f0ec9ffb640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc1cf5d0 0x7f0edc11cc70 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000d6f0 tx=0x7f0ed000dbc0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.101+0000 7f0ee1e5e640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0eac03df00 0x7f0eac0403c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80055f0 tx=0x7f0ed8005580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee38e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee38e8640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc11d1b0 0x7f0edc1da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee38e8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc068410 con 0x7f0edc11d1b0 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc11d1b0 0x7f0edc1da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 10 monclient: get_auth_request con 0x7f0edc11d1b0 auth_method 0 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 10 monclient(hunting): handle_auth_done global_id 144629 payload 824 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.103+0000 7f0ee0e5c640 1 -- 10.20.193.78:0/4023847106 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0edc06cc70 con 0x7f0edc11d1b0 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee0e5c640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc11d1b0 msgr2=0x7f0edc1da6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee0e5c640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc11d1b0 0x7f0edc1da6d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000d6f0 tx=0x7f0ed0003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0005070 con 0x7f0edc11d1b0 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 10 monclient: dump: 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.117 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f0ed0005070 con 0x7f0edc11d1b0 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc11d1b0 msgr2=0x7f0edc1da6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc11d1b0 0x7f0edc1da6d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000d6f0 tx=0x7f0ed0003860 comp rx=0 tx=0).stop 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc11d1b0 msgr2=0x7f0edc1da6d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:37.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ec9ffb640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc11d1b0 0x7f0edc1da6d0 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000d6f0 tx=0x7f0ed0003860 comp rx=0 tx=0).stop 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 180, 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 280641, 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T06:01:37.119 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.120 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:37.121 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.122 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.123 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.124 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:37.125 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.126 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.127 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.128 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:37.129 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:37.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0eac03df00 msgr2=0x7f0eac0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0eac03df00 0x7f0eac0403c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80055f0 tx=0x7f0ed8005580 comp rx=0 tx=0).stop 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 10 monclient: shutdown 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 msgr2=0x7f0edc1341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc1341f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009820 tx=0x7f0ecc0047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee1e5e640 1 -- 10.20.193.78:0/4023847106 reap_dead start 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 shutdown_connections 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 --2- 10.20.193.78:0/4023847106 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0edc06c890 0x7f0edc1341f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 >> 10.20.193.78:0/4023847106 conn(0x7f0edc0a6620 msgr2=0x7f0edc135460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 shutdown_connections 2026-02-17T06:01:37.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.106+0000 7f0ee38e8640 1 -- 10.20.193.78:0/4023847106 wait complete. 2026-02-17T06:01:37.130 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:01:37.131 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:01:37.131 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.156+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.156+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1640052292}) 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.156+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1515796056}) 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.156+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.156+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1640052292}) 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.156+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1914504293}) 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120027073}) 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1881566004}) 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.162 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120027073}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=611673225}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145593387}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=849151242}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145593387}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.157+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077943582}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259749720}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2149119047}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259749720}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2552383473}) 2026-02-17T06:01:37.163 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2212248358}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2557549591}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2212248358}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1063461207}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3445174453}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=703664380}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3445174453}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.158+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=162295013}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=746839233}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2394486690}) 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.164 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=746839233}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3293884269}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2460697498}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2203379925}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2460697498}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2819961935}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=359106556}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2990714746}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=359106556}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.159+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3823997486}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1879735463}) 2026-02-17T06:01:37.165 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4093614221}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1879735463}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4222178336}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=571503405}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1720942337}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=571503405}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3938454910}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464434277}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3546175422}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464434277}) 2026-02-17T06:01:37.166 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.160+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1502591163}) 2026-02-17T06:01:37.167 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.167 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2458496442}) 2026-02-17T06:01:37.167 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3010856873}) 2026-02-17T06:01:37.167 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.167 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2458496442}) 2026-02-17T06:01:37.167 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4073107780}) 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2871751160}) 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2221052223}) 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2871751160}) 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1159290749}) 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555611937}) 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1958188171}) 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.170 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555611937}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.161+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1135784834}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884797483}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1875895195}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884797483}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2517227814}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=75,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4} crc {logm=1869814851,mdsmap=1160443688,mgr=1039804757,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4131631302,mon_config_key=3379246293}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=3} crc {logm=3282277832,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4131631302,mon_config_key=3785818138}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=75,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4} crc {logm=1869814851,mdsmap=1160443688,mgr=1039804757,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4131631302,mon_config_key=3379246293}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=3} crc {logm=18666719,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4131631302,mon_config_key=3785818138}) 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.171 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=8,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=83} crc {mon_config_key=3278643186,monmap=3140514009,osd_metadata=2997046373,osd_pg_creating=777065117,osd_snap=2879081577}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=85} crc {mon_config_key=1936424560,monmap=1784110042,osd_metadata=1501630441,osd_pg_creating=777065117,osd_snap=2073546676}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=8,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=83} crc {mon_config_key=3278643186,monmap=3140514009,osd_metadata=2997046373,osd_pg_creating=777065117,osd_snap=2879081577}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.162+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=82} crc {mon_config_key=3848187981,monmap=3140514009,osd_metadata=2997046373,osd_pg_creating=777065117,osd_snap=782896071}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=13,osdmap=87} crc {osd_snap=1931674119,osdmap=4085734765}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=12,osdmap=88} crc {osd_snap=289867420,osdmap=1499000394}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=13,osdmap=87} crc {osd_snap=1931674119,osdmap=4085734765}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=13,osdmap=87} crc {osd_snap=1931674119,osdmap=1808818618}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2532218631}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1395696716}) 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.172 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2532218631}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.163+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=634075404}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.164+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.164+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1716324048}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.164+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3495636454}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.164+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.164+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1716324048}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.164+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4008271652}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.166+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.166+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=61} crc {osdmap=1846711938}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.166+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=61} crc {osdmap=48630089}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.166+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.166+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=61} crc {osdmap=1846711938}) 2026-02-17T06:01:37.173 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:01:37.166+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=56} crc {osdmap=1751853064}) 2026-02-17T06:01:37.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.197+0000 7fb540c4d640 10 monclient: get_monmap_and_config 2026-02-17T06:01:37.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.197+0000 7fb540c4d640 10 monclient: build_initial_monmap 2026-02-17T06:01:37.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.197+0000 7fb540c4d640 10 monclient: monmap: 2026-02-17T06:01:37.198 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:37.198 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:37.198767+0000 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:37.198767+0000 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:37.199 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.197+0000 7fb540c4d640 1 Processor -- start 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.197+0000 7fb540c4d640 1 -- start start 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.197+0000 7fb540c4d640 10 monclient: init 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 10 monclient: _add_conns ranks=[1,4,0,2,3] 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb53c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb53c09c4a0 0x7fb53c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 10 monclient(hunting): picked mon.noname-c con 0x7fb53c09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb53c06d380 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb53c06daf0 con 0x7fb53c06d380 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb53c09cc80 con 0x7fb53c06c890 2026-02-17T06:01:37.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb53c0a0880 con 0x7fb53c09c4a0 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb540c4d640 10 monclient(hunting): authenticate will time out at 1542.752527s 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb53ad76640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb53c06d380 0x7fb53c13d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:53088/0 (socket says 10.20.193.78:53088) 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb53ad76640 1 -- 10.20.193.78:0/2910852656 learned_addr learned my addr 10.20.193.78:0/2910852656 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb539d74640 1 --2- 10.20.193.78:0/2910852656 >> v2:10.20.193.115:3300/0 conn(0x7fb53c09c4a0 0x7fb53c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb539d74640 10 monclient(hunting): get_auth_request con 0x7fb53c09c4a0 auth_method 0 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb539d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb539d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.198+0000 7fb539d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 10 monclient(hunting): handle_auth_done global_id 194110 payload 274 2026-02-17T06:01:37.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 1 -- 10.20.193.78:0/2910852656 >> v1:10.20.193.115:6789/0 conn(0x7fb53c06c890 legacy=0x7fb53c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 1 -- 10.20.193.78:0/2910852656 >> v1:10.20.193.78:6789/0 conn(0x7fb53c06d380 legacy=0x7fb53c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 10 monclient: _finish_hunting 0 2026-02-17T06:01:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 1 monclient: found mon.noname-c 2026-02-17T06:01:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:01:37.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.199+0000 7fb539d74640 1 -- 10.20.193.78:0/2910852656 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c09cf60 con 0x7fb53c09c4a0 2026-02-17T06:01:37.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.201+0000 7fb539d74640 1 -- 10.20.193.78:0/2910852656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 msgr2=0x7fb53c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.201+0000 7fb539d74640 1 --2- 10.20.193.78:0/2910852656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 0x7fb53c13b3a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb53c09c980 tx=0x7fb52402ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e75df600760f1d1b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 1 -- 10.20.193.78:0/2910852656 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52403c070 con 0x7fb53c09c4a0 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: dump: 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.210 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: _finish_auth 0 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: _check_auth_tickets 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 5 monclient: authenticate success, global_id 194110 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 1 -- 10.20.193.78:0/2910852656 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52402fcb0 con 0x7fb53c09c4a0 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 1 -- 10.20.193.78:0/2910852656 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402fe60 con 0x7fb53c09c4a0 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 4 set_mon_vals no callback set 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:37.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb539573640 10 monclient: dump: 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 10 monclient: shutdown 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 1 -- 10.20.193.78:0/2910852656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 msgr2=0x7fb53c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 1 --2- 10.20.193.78:0/2910852656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 0x7fb53c13b3a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb53c09c980 tx=0x7fb52402ed50 comp rx=0 tx=0).stop 2026-02-17T06:01:37.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 1 -- 10.20.193.78:0/2910852656 shutdown_connections 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 1 --2- 10.20.193.78:0/2910852656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 0x7fb53c13b3a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 1 -- 10.20.193.78:0/2910852656 >> 10.20.193.78:0/2910852656 conn(0x7fb53c0a6620 msgr2=0x7fb53c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 1 -- 10.20.193.78:0/2910852656 shutdown_connections 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 1 -- 10.20.193.78:0/2910852656 wait complete. 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 10 monclient: build_initial_monmap 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.202+0000 7fb540c4d640 10 monclient: monmap: 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:37.213 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 1 Processor -- start 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 1 -- start start 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient: init 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb53c06c890 0x7fb53c1cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient(hunting): picked mon.noname-c con 0x7fb53c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb53c06d380 0x7fb53c1cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient(hunting): picked mon.noname-a con 0x7fb53c06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 0x7fb53c1cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient(hunting): picked mon.noname-b con 0x7fb53c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c09bfa0 con 0x7fb53c06d380 2026-02-17T06:01:37.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c09be20 con 0x7fb53c09c4a0 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb53c13f010 con 0x7fb53c06c890 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb540c4d640 10 monclient(hunting): authenticate will time out at 1542.757347s 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb53c06c890 0x7fb53c1cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb53c06d380 0x7fb53c1cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb53c06c890 0x7fb53c1cf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53544/0 (socket says 10.20.193.78:53544) 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 -- 10.20.193.78:0/903389912 learned_addr learned my addr 10.20.193.78:0/903389912 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb53c06d380 0x7fb53c1cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:41300/0 (socket says 10.20.193.78:41300) 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient(hunting): get_auth_request con 0x7fb53c06c890 auth_method 0 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 10 monclient(hunting): get_auth_request con 0x7fb53c06d380 auth_method 0 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53ad76640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 0x7fb53c1cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53ad76640 10 monclient(hunting): get_auth_request con 0x7fb53c09c4a0 auth_method 0 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient(hunting): handle_auth_done global_id 144632 payload 824 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 msgr2=0x7fb53c1cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c09c4a0 0x7fb53c1cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb53c06d380 msgr2=0x7fb53c1cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb53c06d380 0x7fb53c1cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient: _finish_hunting 0 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 monclient: found mon.noname-c 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb53a575640 1 -- 10.20.193.78:0/903389912 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c121ee0 con 0x7fb53c06c890 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.203+0000 7fb539d74640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb53c06d380 0x7fb53c1cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb53a575640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb53c06c890 msgr2=0x7fb53c1cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb53a575640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb53c06c890 0x7fb53c1cf240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c600 tx=0x7fb53000cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530012a40 con 0x7fb53c06c890 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: dump: 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb530017070 con 0x7fb53c06c890 2026-02-17T06:01:37.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb540c4d640 5 monclient: authenticate success, global_id 144632 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb540c4d640 10 monclient: _renew_subs 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb540c4d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53c09d810 con 0x7fb53c06c890 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530012e20 con 0x7fb53c06c890 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb522ffd640 10 monclient: dump: 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.219 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb538d72640 4 set_mon_vals no callback set 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb540c4d640 10 monclient: _renew_subs 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb540c4d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.205+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53c121210 con 0x7fb53c06c890 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb540c4d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb540c4d640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c135d10 0x7fb53c1360f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb540c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c06cc70 con 0x7fb53c135d10 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb53001c460 con 0x7fb53c06c890 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb539d74640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c135d10 0x7fb53c1360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb539d74640 10 monclient: get_auth_request con 0x7fb53c135d10 auth_method 0 2026-02-17T06:01:37.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb539d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb539d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb539d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb522ffd640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb50803deb0 0x7fb508040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(561..561 src has 335..561) ==== 4735+0+0 (secure 0 0 0) 0x7fb5300525b0 con 0x7fb53c06c890 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.206+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.207+0000 7fb539d74640 10 monclient(hunting): handle_auth_done global_id 194116 payload 824 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.207+0000 7fb53ad76640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb50803deb0 0x7fb508040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.207+0000 7fb539d74640 1 -- 10.20.193.78:0/903389912 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb53c121740 con 0x7fb53c135d10 2026-02-17T06:01:37.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.207+0000 7fb53ad76640 10 monclient: get_auth_request con 0x7fb50803deb0 auth_method 0 2026-02-17T06:01:37.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb539d74640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c135d10 msgr2=0x7fb53c1360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb539d74640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c135d10 0x7fb53c1360f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb524039000 tx=0x7fb5240047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52403c040 con 0x7fb53c135d10 2026-02-17T06:01:37.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:37.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 10 monclient: dump: 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb53ad76640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb50803deb0 0x7fb508040370 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb52c007c90 tx=0x7fb52c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fb524037e30 con 0x7fb53c135d10 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c135d10 msgr2=0x7fb53c1360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c135d10 0x7fb53c1360f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb524039000 tx=0x7fb5240047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c135d10 msgr2=0x7fb53c1360f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.209+0000 7fb522ffd640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c135d10 0x7fb53c1360f0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb524039000 tx=0x7fb5240047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.212+0000 7fb540c4d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb540c4d640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c136630 0x7fb53c1da800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb540c4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c0744d0 con 0x7fb53c136630 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c136630 0x7fb53c1da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 10 monclient: get_auth_request con 0x7fb53c136630 auth_method 0 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 10 monclient(hunting): handle_auth_done global_id 194119 payload 824 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.213+0000 7fb539d74640 1 -- 10.20.193.78:0/903389912 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb53c136a40 con 0x7fb53c136630 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.215+0000 7fb539d74640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c136630 msgr2=0x7fb53c1da800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.215+0000 7fb539d74640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c136630 0x7fb53c1da800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb52402ed90 tx=0x7fb524008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:37.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524055070 con 0x7fb53c136630 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 10 monclient: dump: 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7fb5240513f0 con 0x7fb53c136630 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c136630 msgr2=0x7fb53c1da800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c136630 0x7fb53c1da800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb52402ed90 tx=0x7fb524008030 comp rx=0 tx=0).stop 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c136630 msgr2=0x7fb53c1da800 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb522ffd640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb53c136630 0x7fb53c1da800 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb52402ed90 tx=0x7fb524008030 comp rx=0 tx=0).stop 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 180, 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 12611, 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 6, 2026-02-17T06:01:37.227 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.228 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:37.229 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.230 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:37.231 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:37.232 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.233 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:37.234 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:37.235 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2614: done (0 left) 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2615: read oid 14 snap 267 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2615: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2616: read oid 21 snap 262 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2616: expect (ObjNum 867 snap 247 seq_num 867) 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2617: read oid 6 snap 267 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2617: expect (ObjNum 912 snap 265 seq_num 912) 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2618: copy_from oid 42 from oid 30 current snap is 269 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2619: read oid 1 snap -1 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2619: expect deleted 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2620: write (excl) oid 28 current snap is 269 2026-02-17T06:01:37.236 INFO:tasks.rados.rados.0.trial115.stdout:2620: seq_num 918 ranges {628826=640942,1869411=462657,2826425=534649} 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb50803deb0 msgr2=0x7fb508040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb50803deb0 0x7fb508040370 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb52c007c90 tx=0x7fb52c009040 comp rx=0 tx=0).stop 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 10 monclient: shutdown 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb53c06c890 msgr2=0x7fb53c1cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 1 --2- 10.20.193.78:0/903389912 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb53c06c890 0x7fb53c1cf240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c600 tx=0x7fb53000cad0 comp rx=0 tx=0).stop 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb53ad76640 1 -- 10.20.193.78:0/903389912 reap_dead start 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb53ad76640 1 -- 10.20.193.78:0/903389912 reap_dead start 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 shutdown_connections 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 >> 10.20.193.78:0/903389912 conn(0x7fb53c0a6620 msgr2=0x7fb53c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 shutdown_connections 2026-02-17T06:01:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:37.216+0000 7fb540c4d640 1 -- 10.20.193.78:0/903389912 wait complete. 2026-02-17T06:01:37.237 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:01:37.238 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T06:01:37.238 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2026-02-17T06:01:37.238 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T06:01:37.238 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:01:37.238 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T06:01:37.238 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:01:37.240 INFO:tasks.rados.rados.0.trial115.stdout:2620: writing trial11545420-28 from 628826 to 1269768 tid 1 2026-02-17T06:01:37.244 INFO:tasks.rados.rados.0.trial115.stdout:2620: writing trial11545420-28 from 1869411 to 2332068 tid 2 2026-02-17T06:01:37.247 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T06:01:37.247 INFO:tasks.mon_thrash:Sending STOP to mon a 2026-02-17T06:01:37.247 INFO:tasks.ceph.mon.a:Sent signal 19 2026-02-17T06:01:37.247 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T06:01:37.248 INFO:tasks.rados.rados.0.trial115.stdout:2620: writing trial11545420-28 from 2826425 to 3361074 tid 3 2026-02-17T06:01:37.248 INFO:tasks.rados.rados.0.trial115.stdout:2621: write oid 12 current snap is 269 2026-02-17T06:01:37.248 INFO:tasks.rados.rados.0.trial115.stdout:2621: seq_num 919 ranges {645633=716191,1927893=606315,3137495=308138} 2026-02-17T06:01:37.253 INFO:tasks.rados.rados.0.trial115.stdout:2621: writing trial11545420-12 from 645633 to 1361824 tid 1 2026-02-17T06:01:37.258 INFO:tasks.rados.rados.0.trial115.stdout:2621: writing trial11545420-12 from 1927893 to 2534208 tid 2 2026-02-17T06:01:37.261 INFO:tasks.rados.rados.0.trial115.stdout:2621: writing trial11545420-12 from 3137495 to 3445633 tid 3 2026-02-17T06:01:37.262 INFO:tasks.rados.rados.0.trial115.stdout:2622: read oid 33 snap -1 2026-02-17T06:01:37.262 INFO:tasks.rados.rados.0.trial115.stdout:2622: expect (ObjNum 770 snap 218 seq_num 770) 2026-02-17T06:01:37.262 INFO:tasks.rados.rados.0.trial115.stdout:2623: snap_create 2026-02-17T06:01:37.262 INFO:tasks.rados.rados.0.trial115.stdout:2617: expect (ObjNum 912 snap 265 seq_num 912) 2026-02-17T06:01:37.401 INFO:tasks.rados.rados.0.trial115.stdout:2615: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:01:37.554 INFO:tasks.rados.rados.0.trial115.stdout:2616: expect (ObjNum 867 snap 247 seq_num 867) 2026-02-17T06:01:37.664 INFO:tasks.rados.rados.0.trial115.stdout:2620: finishing write tid 1 to trial11545420-28 2026-02-17T06:01:37.665 INFO:tasks.rados.rados.0.trial115.stdout:2620: finishing write tid 2 to trial11545420-28 2026-02-17T06:01:37.665 INFO:tasks.rados.rados.0.trial115.stdout:2618: finishing copy_from racing read to trial11545420-42 2026-02-17T06:01:37.665 INFO:tasks.rados.rados.0.trial115.stdout:2618: finishing copy_from to trial11545420-42 2026-02-17T06:01:37.665 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1166 (ObjNum 904 snap 262 seq_num 904) dirty exists 2026-02-17T06:01:37.665 INFO:tasks.rados.rados.0.trial115.stdout:2620: finishing write tid 3 to trial11545420-28 2026-02-17T06:01:37.665 INFO:tasks.rados.rados.0.trial115.stdout:2620: finishing write tid 4 to trial11545420-28 2026-02-17T06:01:37.665 INFO:tasks.rados.rados.0.trial115.stdout:2620: finishing write tid 5 to trial11545420-28 2026-02-17T06:01:37.665 INFO:tasks.rados.rados.0.trial115.stdout:2620: oid 28 updating version 0 to 1325 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2620: oid 28 version 1325 is already newer than 1324 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2620: oid 28 version 1325 is already newer than 1323 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2620: oid 28 updating version 1325 to 1326 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1326 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2620: left oid 28 (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2621: finishing write tid 1 to trial11545420-12 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2621: finishing write tid 2 to trial11545420-12 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2621: finishing write tid 3 to trial11545420-12 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2621: finishing write tid 4 to trial11545420-12 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2621: finishing write tid 5 to trial11545420-12 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2621: oid 12 updating version 0 to 1401 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2621: oid 12 updating version 1401 to 1404 2026-02-17T06:01:37.666 INFO:tasks.rados.rados.0.trial115.stdout:2621: oid 12 version 1404 is already newer than 1403 2026-02-17T06:01:37.667 INFO:tasks.rados.rados.0.trial115.stdout:2621: oid 12 version 1404 is already newer than 1402 2026-02-17T06:01:37.667 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1404 (ObjNum 919 snap 269 seq_num 919) dirty exists 2026-02-17T06:01:37.667 INFO:tasks.rados.rados.0.trial115.stdout:2621: left oid 12 (ObjNum 919 snap 269 seq_num 919) 2026-02-17T06:01:37.667 INFO:tasks.rados.rados.0.trial115.stdout:2622: expect (ObjNum 770 snap 218 seq_num 770) 2026-02-17T06:01:52.248 INFO:tasks.mon_thrash:Sending CONT to mon a 2026-02-17T06:01:52.249 INFO:tasks.ceph.mon.a:Sent signal 18 2026-02-17T06:01:52.249 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:01:52.249 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:01:52.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.317+0000 7fb38ecb0640 10 monclient: get_monmap_and_config 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.317+0000 7fb38ecb0640 10 monclient: build_initial_monmap 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.317+0000 7fb38ecb0640 10 monclient: monmap: 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:52.318826+0000 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:52.318826+0000 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:52.320 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 1 Processor -- start 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 1 -- start start 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient: init 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2026-02-17T06:01:52.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 0x7fb38806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient(hunting): picked mon.noname-c con 0x7fb38806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb38809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb38806d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb388135c10 con 0x7fb38809c4a0 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb38806daf0 con 0x7fb38806d1b0 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb3880a0880 con 0x7fb38806c890 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ecb0640 10 monclient(hunting): authenticate will time out at 1557.872381s 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ca25640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 msgr2=0x7fb38806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38ca25640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 0x7fb38806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:52.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb38d226640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fb38806d1b0 legacy=0x7fb388135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:52.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb37ffff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb38809c4a0 0x7fb388134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:48278/0 (socket says 10.20.193.78:48278) 2026-02-17T06:01:52.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.318+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 learned_addr learned my addr 10.20.193.78:0/1675030651 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:01:52.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.320+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.78:6789/0 conn(0x7fb38809c4a0 legacy=0x7fb388134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:52.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.518+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 msgr2=0x7fb38806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:52.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.518+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 0x7fb38806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:01:52.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.518+0000 7fb38d226640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.115:6789/0 conn(0x7fb38806d1b0 legacy=0x7fb388135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:52.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.919+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 msgr2=0x7fb38806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:52.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.919+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 0x7fb38806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:01:52.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:52.919+0000 7fb38d226640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.115:6789/0 conn(0x7fb38806d1b0 legacy=0x7fb388135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:53.720+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 msgr2=0x7fb38806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:53.720+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 0x7fb38806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:01:53.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:53.720+0000 7fb38d226640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.115:6789/0 conn(0x7fb38806d1b0 legacy=0x7fb388135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient(hunting): tick 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 monclient(hunting): continuing hunt 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 msgr2=0x7fb38806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb38806c890 0x7fb38806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.115:6789/0 conn(0x7fb38806d1b0 legacy=0x7fb388135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.78:6789/0 conn(0x7fb38809c4a0 legacy=0x7fb388134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient: _add_conns ranks=[2,3] 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient(hunting): picked mon.noname-b con 0x7fb36c003840 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient(hunting): picked mon.noname-a con 0x7fb36c006350 addr v2:10.20.193.78:3300/0 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb36c008ea0 con 0x7fb36c006350 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb36c009050 con 0x7fb36c003840 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37e7fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 10 monclient(hunting): get_auth_request con 0x7fb36c006350 auth_method 0 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 10 monclient(hunting): get_auth_request con 0x7fb36c003840 auth_method 0 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:55.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.318+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 10 monclient(hunting): get_auth_request con 0x7fb36c003840 auth_method 0 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 10 monclient(hunting): get_auth_request con 0x7fb36c006350 auth_method 0 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:55.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:01:55.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:55.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:55.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:01:55.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:55.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:01:55.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:55.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.519+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:01:55.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 10 monclient(hunting): get_auth_request con 0x7fb36c003840 auth_method 0 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:55.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 10 monclient(hunting): get_auth_request con 0x7fb36c006350 auth_method 0 2026-02-17T06:01:55.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:55.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:01:55.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:55.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:55.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:01:55.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:55.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:55.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:55.920+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:01:56.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 10 monclient(hunting): get_auth_request con 0x7fb36c003840 auth_method 0 2026-02-17T06:01:56.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 10 monclient(hunting): get_auth_request con 0x7fb36c006350 auth_method 0 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:56.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:56.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:01:56.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:01:56.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:56.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:56.720+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:01:58.257 INFO:tasks.rados.rados.0.trial115.stdout:2615: done (8 left) 2026-02-17T06:01:58.257 INFO:tasks.rados.rados.0.trial115.stdout:2616: done (7 left) 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2617: done (6 left) 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2618: done (5 left) 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2619: done (4 left) 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2620: done (3 left) 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2621: done (2 left) 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2622: done (1 left) 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2623: done (0 left) 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2624: copy_from oid 44 from oid 22 current snap is 270 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2625: copy_from oid 12 from oid 13 current snap is 270 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2626: write (excl) oid 20 current snap is 270 2026-02-17T06:01:58.258 INFO:tasks.rados.rados.0.trial115.stdout:2626: seq_num 920 ranges {662440=507793,1702728=749973,2881271=557193} 2026-02-17T06:01:58.262 INFO:tasks.rados.rados.0.trial115.stdout:2626: writing trial11545420-20 from 662440 to 1170233 tid 1 2026-02-17T06:01:58.267 INFO:tasks.rados.rados.0.trial115.stdout:2626: writing trial11545420-20 from 1702728 to 2452701 tid 2 2026-02-17T06:01:58.271 INFO:tasks.rados.rados.0.trial115.stdout:2626: writing trial11545420-20 from 2881271 to 3438464 tid 3 2026-02-17T06:01:58.271 INFO:tasks.rados.rados.0.trial115.stdout:2627: read oid 11 snap -1 2026-02-17T06:01:58.271 INFO:tasks.rados.rados.0.trial115.stdout:2627: expect (ObjNum 885 snap 254 seq_num 885) 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:2628: snap_create 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:2624: finishing copy_from racing read to trial11545420-44 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:2625: finishing copy_from racing read to trial11545420-12 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:2624: finishing copy_from to trial11545420-44 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:2626: finishing write tid 1 to trial11545420-20 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:2625: finishing copy_from to trial11545420-12 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1418 (ObjNum 888 snap 254 seq_num 888) dirty exists 2026-02-17T06:01:58.272 INFO:tasks.rados.rados.0.trial115.stdout:2626: finishing write tid 2 to trial11545420-20 2026-02-17T06:01:58.273 INFO:tasks.rados.rados.0.trial115.stdout:2626: finishing write tid 3 to trial11545420-20 2026-02-17T06:01:58.273 INFO:tasks.rados.rados.0.trial115.stdout:2626: finishing write tid 4 to trial11545420-20 2026-02-17T06:01:58.274 INFO:tasks.rados.rados.0.trial115.stdout:2626: finishing write tid 5 to trial11545420-20 2026-02-17T06:01:58.274 INFO:tasks.rados.rados.0.trial115.stdout:2626: oid 20 updating version 0 to 1421 2026-02-17T06:01:58.274 INFO:tasks.rados.rados.0.trial115.stdout:2626: oid 20 version 1421 is already newer than 1419 2026-02-17T06:01:58.274 INFO:tasks.rados.rados.0.trial115.stdout:2626: oid 20 version 1421 is already newer than 1420 2026-02-17T06:01:58.274 INFO:tasks.rados.rados.0.trial115.stdout:2626: oid 20 updating version 1421 to 1422 2026-02-17T06:01:58.274 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1422 (ObjNum 920 snap 270 seq_num 920) dirty exists 2026-02-17T06:01:58.274 INFO:tasks.rados.rados.0.trial115.stdout:2626: left oid 20 (ObjNum 920 snap 270 seq_num 920) 2026-02-17T06:01:58.274 INFO:tasks.rados.rados.0.trial115.stdout:2627: expect (ObjNum 885 snap 254 seq_num 885) 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient(hunting): tick 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 monclient(hunting): continuing hunt 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 msgr2=0x7fb36c005c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb36c003840 0x7fb36c005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 msgr2=0x7fb36c008810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.78:3300/0 conn(0x7fb36c006350 0x7fb36c008810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb38d226640 1 -- 10.20.193.78:0/1675030651 reap_dead start 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb36c003c60 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb36c007ea0 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb36c00c510 0x7fb36c00e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient(hunting): picked mon.noname-c con 0x7fb36c00c510 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb38809cc80 con 0x7fb36c007ea0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb38806daf0 con 0x7fb36c003c60 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 1 -- 10.20.193.78:0/1675030651 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb36c008ea0 con 0x7fb36c00c510 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb37e7fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.115:6789/0 conn(0x7fb36c003c60 legacy=0x7fb36c006780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb38d226640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb36c00c510 msgr2=0x7fb36c00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.318+0000 7fb38d226640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb36c00c510 0x7fb36c00e900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.320+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.78:6789/0 conn(0x7fb36c007ea0 legacy=0x7fb36c008300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.320+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2282803970 0 0) 0x7fb38809cc80 con 0x7fb36c007ea0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.320+0000 7fb37f7fe640 10 monclient(hunting): _negotiate 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.320+0000 7fb37f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.320+0000 7fb37f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.320+0000 7fb37f7fe640 10 monclient(hunting): my global_id is 184254 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb374003620 con 0x7fb36c007ea0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 36432178 0 0) 0x7fb374003620 con 0x7fb36c007ea0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb36c00c510 msgr2=0x7fb36c00e900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 --2- 10.20.193.78:0/1675030651 >> v2:10.20.193.115:3300/0 conn(0x7fb36c00c510 0x7fb36c00e900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.115:6789/0 conn(0x7fb36c003c60 legacy=0x7fb36c006780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c010f00 con 0x7fb36c007ea0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fb3700030f0 con 0x7fb36c007ea0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: dump: 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.339 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 >> v1:10.20.193.78:6789/0 conn(0x7fb36c007ea0 legacy=0x7fb36c008300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb374003ab0 0x7fb374003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient(hunting): picked mon.a con 0x7fb374003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb374004900 0x7fb374004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient(hunting): picked mon.b con 0x7fb374004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb3740053e0 0x7fb37400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient(hunting): picked mon.c con 0x7fb3740053e0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c008ea0 con 0x7fb374003ab0 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb37400c330 con 0x7fb374004900 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb37400c530 con 0x7fb3740053e0 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb374003ab0 0x7fb374003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient(hunting): get_auth_request con 0x7fb374003ab0 auth_method 0 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37ffff640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb374004900 msgr2=0x7fb374004d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb3740053e0 0x7fb37400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb37ffff640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb374004900 0x7fb374004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 10 monclient(hunting): get_auth_request con 0x7fb3740053e0 auth_method 0 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient(hunting): handle_auth_done global_id 184254 payload 293 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb3740053e0 msgr2=0x7fb37400bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb3740053e0 0x7fb37400bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb374004900 msgr2=0x7fb374004d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb374004900 0x7fb374004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 1 monclient: found mon.a 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb374003620 con 0x7fb374003ab0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ca25640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ecb0640 5 monclient: authenticate success, global_id 184254 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb3740053e0 0x7fb37400bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38ecb0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.321+0000 7fb38d226640 1 -- 10.20.193.78:0/1675030651 reap_dead start 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ca25640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb374003ab0 msgr2=0x7fb374003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ca25640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb374003ab0 0x7fb374003eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb378003a30 tx=0x7fb37802feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657b406b67991044 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37803d070 con 0x7fb374003ab0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 10 monclient: dump: 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb378030ae0 con 0x7fb374003ab0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 1 -- 10.20.193.78:0/1675030651 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378009a20 con 0x7fb374003ab0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 4 set_mon_vals no callback set 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb37f7fe640 10 monclient: dump: 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.341 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 10 monclient: shutdown 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 1 -- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb374003ab0 msgr2=0x7fb374003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb374003ab0 0x7fb374003eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb378003a30 tx=0x7fb37802feb0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 1 -- 10.20.193.78:0/1675030651 shutdown_connections 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 1 --2- 10.20.193.78:0/1675030651 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb374003ab0 0x7fb374003eb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 1 -- 10.20.193.78:0/1675030651 >> 10.20.193.78:0/1675030651 conn(0x7fb3880a6620 msgr2=0x7fb3880a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 1 -- 10.20.193.78:0/1675030651 shutdown_connections 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.324+0000 7fb38ecb0640 1 -- 10.20.193.78:0/1675030651 wait complete. 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient: monmap: 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 1 Processor -- start 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 1 -- start start 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient: init 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb374003ab0 0x7fb3881cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient(hunting): picked mon.noname-c con 0x7fb374003ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb3881cf890 0x7fb388123ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient(hunting): picked mon.noname-a con 0x7fb3881cf890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb3881cfcb0 0x7fb388124020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient(hunting): picked mon.noname-b con 0x7fb3881cfcb0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3880a0020 con 0x7fb3881cf890 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb38806d640 con 0x7fb3881cfcb0 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb3880a01a0 con 0x7fb374003ab0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ecb0640 10 monclient(hunting): authenticate will time out at 1563.879571s 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb374003ab0 0x7fb3881cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb374003ab0 0x7fb3881cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44862/0 (socket says 10.20.193.78:44862) 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb37ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb3881cf890 0x7fb388123ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 -- 10.20.193.78:0/2484502742 learned_addr learned my addr 10.20.193.78:0/2484502742 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient(hunting): get_auth_request con 0x7fb374003ab0 auth_method 0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb37ffff640 10 monclient(hunting): get_auth_request con 0x7fb3881cf890 auth_method 0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb37ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb37ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38d226640 1 -- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb3881cfcb0 msgr2=0x7fb388124020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb37ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38d226640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb3881cfcb0 0x7fb388124020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb37ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb37ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb37ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient(hunting): handle_auth_done global_id 144635 payload 824 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 -- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb3881cfcb0 msgr2=0x7fb388124020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb3881cfcb0 0x7fb388124020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 -- 10.20.193.78:0/2484502742 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb3881cf890 msgr2=0x7fb388123ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb3881cf890 0x7fb388123ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 monclient: found mon.noname-c 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.325+0000 7fb38ca25640 1 -- 10.20.193.78:0/2484502742 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb388127790 con 0x7fb374003ab0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.327+0000 7fb38ca25640 1 -- 10.20.193.78:0/2484502742 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb374003ab0 msgr2=0x7fb3881cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.327+0000 7fb38ca25640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb374003ab0 0x7fb3881cf350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb378009f20 tx=0x7fb378005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.327+0000 7fb37d7fa640 1 -- 10.20.193.78:0/2484502742 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378048020 con 0x7fb374003ab0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.327+0000 7fb37d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.327+0000 7fb37d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.327+0000 7fb37d7fa640 10 monclient: dump: 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.343 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 1 -- 10.20.193.78:0/2484502742 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb378043470 con 0x7fb374003ab0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 1 -- 10.20.193.78:0/2484502742 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37803d040 con 0x7fb374003ab0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37effd640 4 set_mon_vals no callback set 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 10 monclient: dump: 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 5 monclient: authenticate success, global_id 144635 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 10 monclient: _renew_subs 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb388127a70 con 0x7fb374003ab0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 10 monclient: _renew_subs 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38809f630 con 0x7fb374003ab0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 1 -- 10.20.193.78:0/2484502742 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb378043860 con 0x7fb374003ab0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3880a0810 con 0x7fb374003ab0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb35803de60 0x7fb358040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.328+0000 7fb37d7fa640 1 -- 10.20.193.78:0/2484502742 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(563..563 src has 335..563) ==== 4735+0+0 (secure 0 0 0) 0x7fb378078ee0 con 0x7fb374003ab0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.329+0000 7fb37ffff640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb35803de60 0x7fb358040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.329+0000 7fb37ffff640 10 monclient: get_auth_request con 0x7fb35803de60 auth_method 0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.331+0000 7fb37d7fa640 1 -- 10.20.193.78:0/2484502742 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb3880a0810 con 0x7fb374003ab0 2026-02-17T06:01:58.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.331+0000 7fb37d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.331+0000 7fb37d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:58.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.331+0000 7fb37ffff640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb35803de60 0x7fb358040320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb3700036e0 tx=0x7fb370009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.447+0000 7fb38ecb0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:01:58.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.447+0000 7fb38ecb0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:58.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.447+0000 7fb38ecb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.447+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb38809beb0 con 0x7fb374003ab0 2026-02-17T06:01:58.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.447+0000 7fb37d7fa640 1 -- 10.20.193.78:0/2484502742 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fb378052020 con 0x7fb374003ab0 2026-02-17T06:01:58.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.447+0000 7fb37d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:01:58.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.447+0000 7fb37d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:58.448 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":182,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb35803de60 msgr2=0x7fb358040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb35803de60 0x7fb358040320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb3700036e0 tx=0x7fb370009040 comp rx=0 tx=0).stop 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 10 monclient: shutdown 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb374003ab0 msgr2=0x7fb3881cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb374003ab0 0x7fb3881cf350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb378009f20 tx=0x7fb378005d70 comp rx=0 tx=0).stop 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 shutdown_connections 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb35803de60 0x7fb358040320 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb3881cfcb0 0x7fb388124020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb3881cf890 0x7fb388123ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 --2- 10.20.193.78:0/2484502742 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb374003ab0 0x7fb3881cf350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 >> 10.20.193.78:0/2484502742 conn(0x7fb3880a6620 msgr2=0x7fb3881354d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 shutdown_connections 2026-02-17T06:01:58.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.448+0000 7fb38ecb0640 1 -- 10.20.193.78:0/2484502742 wait complete. 2026-02-17T06:01:58.457 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:01:58.457 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:01:58.457 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.524+0000 7f2d95c29640 10 monclient: get_monmap_and_config 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.524+0000 7f2d95c29640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.524+0000 7f2d95c29640 10 monclient: monmap: 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:58.525944+0000 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:58.525944+0000 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:58.525 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 1 Processor -- start 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 1 -- start start 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient: init 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2d9009ea90 addr v1:10.20.193.78:6789/0 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2d9009ee70 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9013e650 0x7f2d90140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient(hunting): picked mon.noname-b con 0x7f2d9013e650 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d900a0a20 con 0x7f2d9009ea90 2026-02-17T06:01:58.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d900a0460 con 0x7f2d9009ee70 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2d900a0d00 con 0x7f2d9013e650 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d95c29640 10 monclient(hunting): authenticate will time out at 1564.079675s 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8effd640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f2d9009ee70 legacy=0x7f2d9009f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8f7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f2d9009ea90 0x7f2d900a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56862/0 (socket says 10.20.193.78:56862) 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8f7fe640 1 -- 10.20.193.78:0/4228279576 learned_addr learned my addr 10.20.193.78:0/4228279576 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 1 --2- 10.20.193.78:0/4228279576 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9013e650 0x7f2d90140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient(hunting): get_auth_request con 0x7f2d9013e650 auth_method 0 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_done global_id 144638 payload 274 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 1 -- 10.20.193.78:0/4228279576 >> v1:10.20.193.115:6789/0 conn(0x7f2d9009ee70 legacy=0x7f2d9009f2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 1 -- 10.20.193.78:0/4228279576 >> v1:10.20.193.78:6789/0 conn(0x7f2d9009ea90 legacy=0x7f2d900a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 1 monclient: found mon.noname-b 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.525+0000 7f2d8ffff640 1 -- 10.20.193.78:0/4228279576 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d900a0740 con 0x7f2d9013e650 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8ffff640 1 -- 10.20.193.78:0/4228279576 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9013e650 msgr2=0x7f2d90140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8ffff640 1 --2- 10.20.193.78:0/4228279576 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9013e650 0x7f2d90140a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009dc0 tx=0x7f2d8402f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe459da11ff3837 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 1 -- 10.20.193.78:0/4228279576 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8403c070 con 0x7f2d9013e650 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: dump: 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 1 -- 10.20.193.78:0/4228279576 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d84037640 con 0x7f2d9013e650 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 5 monclient: authenticate success, global_id 144638 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 1 -- 10.20.193.78:0/4228279576 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84037a10 con 0x7f2d9013e650 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d8e7fc640 10 monclient: dump: 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 4 set_mon_vals no callback set 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 10 monclient: shutdown 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 1 -- 10.20.193.78:0/4228279576 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9013e650 msgr2=0x7f2d90140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 1 --2- 10.20.193.78:0/4228279576 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9013e650 0x7f2d90140a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009dc0 tx=0x7f2d8402f1c0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 1 -- 10.20.193.78:0/4228279576 shutdown_connections 2026-02-17T06:01:58.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 1 --2- 10.20.193.78:0/4228279576 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9013e650 0x7f2d90140a40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 1 -- 10.20.193.78:0/4228279576 >> 10.20.193.78:0/4228279576 conn(0x7f2d900a7520 msgr2=0x7f2d900a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 1 -- 10.20.193.78:0/4228279576 shutdown_connections 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.528+0000 7f2d95c29640 1 -- 10.20.193.78:0/4228279576 wait complete. 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient: monmap: 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 1 Processor -- start 2026-02-17T06:01:58.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 1 -- start start 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient: init 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9009ea90 0x7f2d9013b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient(hunting): picked mon.noname-c con 0x7f2d9009ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2d9009ee70 0x7f2d9013bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient(hunting): picked mon.noname-b con 0x7f2d9009ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d9013e650 0x7f2d9013c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient(hunting): picked mon.noname-a con 0x7f2d9013e650 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90141100 con 0x7f2d9013e650 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90140f80 con 0x7f2d9009ee70 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2d90141280 con 0x7f2d9009ea90 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d95c29640 10 monclient(hunting): authenticate will time out at 1564.083752s 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9009ea90 0x7f2d9013b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9009ea90 0x7f2d9013b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44876/0 (socket says 10.20.193.78:44876) 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 -- 10.20.193.78:0/187566636 learned_addr learned my addr 10.20.193.78:0/187566636 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d9013e650 0x7f2d9013c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient(hunting): get_auth_request con 0x7f2d9009ea90 auth_method 0 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8effd640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2d9009ee70 msgr2=0x7f2d9013bb50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 10 monclient(hunting): get_auth_request con 0x7f2d9013e650 auth_method 0 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8effd640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2d9009ee70 0x7f2d9013bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient(hunting): handle_auth_done global_id 144641 payload 824 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2d9009ee70 msgr2=0x7f2d9013bb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2d9009ee70 0x7f2d9013bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d9013e650 msgr2=0x7f2d9013c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d9013e650 0x7f2d9013c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 monclient: found mon.noname-c 2026-02-17T06:01:58.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8f7fe640 1 -- 10.20.193.78:0/187566636 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9013d900 con 0x7f2d9009ea90 2026-02-17T06:01:58.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.529+0000 7f2d8ffff640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d9013e650 0x7f2d9013c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:01:58.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8f7fe640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9009ea90 msgr2=0x7f2d9013b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8f7fe640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9009ea90 0x7f2d9013b610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c750 tx=0x7f2d7c00cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c005de0 con 0x7f2d9009ea90 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: dump: 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7c007070 con 0x7f2d9009ea90 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 5 monclient: authenticate success, global_id 144641 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 10 monclient: _renew_subs 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9006f890 con 0x7f2d9009ea90 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c007440 con 0x7f2d9009ea90 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8dffb640 4 set_mon_vals no callback set 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 10 monclient: dump: 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 10 monclient: _renew_subs 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9013d220 con 0x7f2d9009ea90 2026-02-17T06:01:58.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2d7c007660 con 0x7f2d9009ea90 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cf1c0 0x7f2d901cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d900a5d20 con 0x7f2d901cf1c0 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8effd640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cf1c0 0x7f2d901cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8effd640 10 monclient: get_auth_request con 0x7f2d901cf1c0 auth_method 0 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.532+0000 7f2d8effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.533+0000 7f2d8effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.533+0000 7f2d6bfff640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2d6003de90 0x7f2d60040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.533+0000 7f2d8effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.533+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(563..563 src has 335..563) ==== 4735+0+0 (secure 0 0 0) 0x7f2d7c052550 con 0x7f2d9009ea90 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.533+0000 7f2d8effd640 10 monclient(hunting): handle_auth_done global_id 184263 payload 824 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.533+0000 7f2d8effd640 1 -- 10.20.193.78:0/187566636 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d900a13b0 con 0x7f2d901cf1c0 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.533+0000 7f2d8ffff640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2d6003de90 0x7f2d60040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.533+0000 7f2d8ffff640 10 monclient: get_auth_request con 0x7f2d6003de90 auth_method 0 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d8effd640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cf1c0 msgr2=0x7f2d901cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d8effd640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cf1c0 0x7f2d901cf5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2d9013ce20 tx=0x7f2d8000bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d800050f0 con 0x7f2d901cf1c0 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 10 monclient: dump: 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2d800050f0 con 0x7f2d901cf1c0 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cf1c0 msgr2=0x7f2d901cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cf1c0 0x7f2d901cf5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2d9013ce20 tx=0x7f2d8000bcc0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cf1c0 msgr2=0x7f2d901cf5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d6bfff640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cf1c0 0x7f2d901cf5a0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2d9013ce20 tx=0x7f2d8000bcc0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.535+0000 7f2d8ffff640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2d6003de90 0x7f2d60040350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006ecd0 tx=0x7f2d84033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d95c29640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d95c29640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cfae0 0x7f2d901da510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d95c29640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d900744d0 con 0x7f2d901cfae0 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cfae0 0x7f2d901da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 10 monclient: get_auth_request con 0x7f2d901cfae0 auth_method 0 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 10 monclient(hunting): handle_auth_done global_id 184266 payload 824 2026-02-17T06:01:58.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.538+0000 7f2d8effd640 1 -- 10.20.193.78:0/187566636 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2d9006f5b0 con 0x7f2d901cfae0 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d8effd640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cfae0 msgr2=0x7f2d901da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d8effd640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cfae0 0x7f2d901da510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2d80014000 tx=0x7f2d80005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8001a940 con 0x7f2d901cfae0 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 10 monclient: dump: 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.541 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4699 (secure 0 0 0) 0x7f2d8001a940 con 0x7f2d901cfae0 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cfae0 msgr2=0x7f2d901da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cfae0 0x7f2d901da510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2d80014000 tx=0x7f2d80005ad0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cfae0 msgr2=0x7f2d901da510 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.540+0000 7f2d6bfff640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2d901cfae0 0x7f2d901da510 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2d80014000 tx=0x7f2d80005ad0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 182, 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 83321, 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:58.542 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:58.543 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.544 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.545 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2d6003de90 msgr2=0x7f2d60040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2d6003de90 0x7f2d60040350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006ecd0 tx=0x7f2d84033000 comp rx=0 tx=0).stop 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 10 monclient: shutdown 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9009ea90 msgr2=0x7f2d9013b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 1 --2- 10.20.193.78:0/187566636 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2d9009ea90 0x7f2d9013b610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c750 tx=0x7f2d7c00cc20 comp rx=0 tx=0).stop 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d8ffff640 1 -- 10.20.193.78:0/187566636 reap_dead start 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d8ffff640 1 -- 10.20.193.78:0/187566636 reap_dead start 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 shutdown_connections 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 >> 10.20.193.78:0/187566636 conn(0x7f2d900a7520 msgr2=0x7f2d900a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 shutdown_connections 2026-02-17T06:01:58.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.541+0000 7f2d95c29640 1 -- 10.20.193.78:0/187566636 wait complete. 2026-02-17T06:01:58.547 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:01:58.547 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:01:58.547 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:01:58.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient: get_monmap_and_config 2026-02-17T06:01:58.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient: monmap: 2026-02-17T06:01:58.617 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:58.617 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.617 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:01:58.614410+0000 2026-02-17T06:01:58.617 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:01:58.614410+0000 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 1 Processor -- start 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 1 -- start start 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient: init 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc6f406c890 0x7fc6f406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient(hunting): picked mon.noname-c con 0x7fc6f406c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:01:58.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f409c4a0 0x7fc6f406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient(hunting): picked mon.noname-b con 0x7fc6f409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc6f4134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f406daf0 con 0x7fc6f4134df0 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f40a0880 con 0x7fc6f406c890 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f406d6f0 con 0x7fc6f409c4a0 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.613+0000 7fc6fc83b640 10 monclient(hunting): authenticate will time out at 1564.167956s 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6fadb1640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fc6f4134df0 legacy=0x7fc6f4135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6fa5b0640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fc6f406c890 msgr2=0x7fc6f406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f409c4a0 0x7fc6f406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6fa5b0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc6f406c890 0x7fc6f406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f409c4a0 0x7fc6f406d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44882/0 (socket says 10.20.193.78:44882) 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 1 -- 10.20.193.78:0/771391545 learned_addr learned my addr 10.20.193.78:0/771391545 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient(hunting): get_auth_request con 0x7fc6f409c4a0 auth_method 0 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_done global_id 144644 payload 274 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 1 -- 10.20.193.78:0/771391545 >> v2:10.20.193.115:3300/0 conn(0x7fc6f406c890 msgr2=0x7fc6f406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 1 --2- 10.20.193.78:0/771391545 >> v2:10.20.193.115:3300/0 conn(0x7fc6f406c890 0x7fc6f406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 1 -- 10.20.193.78:0/771391545 >> v1:10.20.193.115:6789/0 conn(0x7fc6f4134df0 legacy=0x7fc6f4135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 1 monclient: found mon.noname-b 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.614+0000 7fc6f9daf640 1 -- 10.20.193.78:0/771391545 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f409cc80 con 0x7fc6f409c4a0 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f9daf640 1 -- 10.20.193.78:0/771391545 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f409c4a0 msgr2=0x7fc6f406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f9daf640 1 --2- 10.20.193.78:0/771391545 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f409c4a0 0x7fc6f406d1b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc6f406d870 tx=0x7fc6e002eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0e50abed6b79ada server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 1 -- 10.20.193.78:0/771391545 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e003c070 con 0x7fc6f409c4a0 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: dump: 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.625 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 1 -- 10.20.193.78:0/771391545 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e002fdd0 con 0x7fc6f409c4a0 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 1 -- 10.20.193.78:0/771391545 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0037660 con 0x7fc6f409c4a0 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6f95ae640 10 monclient: dump: 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.626 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6fc83b640 5 monclient: authenticate success, global_id 144644 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6fc83b640 10 monclient: get_monmap_and_config success 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.616+0000 7fc6fc83b640 4 set_mon_vals no callback set 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 10 monclient: shutdown 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 -- 10.20.193.78:0/771391545 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f409c4a0 msgr2=0x7fc6f406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 --2- 10.20.193.78:0/771391545 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f409c4a0 0x7fc6f406d1b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc6f406d870 tx=0x7fc6e002eeb0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 -- 10.20.193.78:0/771391545 shutdown_connections 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 --2- 10.20.193.78:0/771391545 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f409c4a0 0x7fc6f406d1b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 --2- 10.20.193.78:0/771391545 >> v2:10.20.193.115:3300/0 conn(0x7fc6f406c890 0x7fc6f406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 -- 10.20.193.78:0/771391545 >> 10.20.193.78:0/771391545 conn(0x7fc6f40a6620 msgr2=0x7fc6f40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 -- 10.20.193.78:0/771391545 shutdown_connections 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 -- 10.20.193.78:0/771391545 wait complete. 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 10 monclient: build_initial_monmap 2026-02-17T06:01:58.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 10 monclient: monmap: 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 Processor -- start 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 1 -- start start 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.617+0000 7fc6fc83b640 10 monclient: init 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient: _reopen_session rank -1 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f406c890 0x7fc6f41cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient(hunting): picked mon.noname-c con 0x7fc6f406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6f409c4a0 0x7fc6f41cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient(hunting): picked mon.noname-b con 0x7fc6f409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6f4134df0 0x7fc6f409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient(hunting): picked mon.noname-a con 0x7fc6f4134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f40a01a0 con 0x7fc6f4134df0 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f40a0020 con 0x7fc6f409c4a0 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f409be20 con 0x7fc6f406c890 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient(hunting): _renew_subs 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fc83b640 10 monclient(hunting): authenticate will time out at 1564.172262s 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f406c890 0x7fc6f41cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f406c890 0x7fc6f41cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44898/0 (socket says 10.20.193.78:44898) 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 -- 10.20.193.78:0/783010209 learned_addr learned my addr 10.20.193.78:0/783010209 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6f9daf640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6f409c4a0 msgr2=0x7fc6f41cf890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient(hunting): get_auth_request con 0x7fc6f406c890 auth_method 0 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fadb1640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6f4134df0 0x7fc6f409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6f9daf640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6f409c4a0 0x7fc6f41cf890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fadb1640 10 monclient(hunting): get_auth_request con 0x7fc6f4134df0 auth_method 0 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fadb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fadb1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fadb1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fadb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fadb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fadb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient(hunting): handle_auth_done global_id 144647 payload 824 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6f409c4a0 msgr2=0x7fc6f41cf890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc6f409c4a0 0x7fc6f41cf890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6f4134df0 msgr2=0x7fc6f409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc6f4134df0 0x7fc6f409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient: _finish_hunting 0 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 monclient: found mon.noname-c 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:01:58.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.618+0000 7fc6fa5b0640 1 -- 10.20.193.78:0/783010209 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f4127090 con 0x7fc6f406c890 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fa5b0640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f406c890 msgr2=0x7fc6f41cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fa5b0640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f406c890 0x7fc6f41cf350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00b780 tx=0x7fc6ec00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0052d0 con 0x7fc6f406c890 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: dump: 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: _finish_auth 0 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: _check_auth_tickets 2026-02-17T06:01:58.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6ec010070 con 0x7fc6f406c890 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fc83b640 5 monclient: authenticate success, global_id 144647 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0056d0 con 0x7fc6f406c890 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6f8dad640 4 set_mon_vals no callback set 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6deffd640 10 monclient: dump: 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fc83b640 10 monclient: _renew_subs 2026-02-17T06:01:58.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fc83b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f4127370 con 0x7fc6f406c890 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fc83b640 10 monclient: _renew_subs 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fc83b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.620+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f4127650 con 0x7fc6f406c890 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6fc83b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6fc83b640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f4063880 0x7fc6f4063c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6fc83b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f409d870 con 0x7fc6f4063880 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f4063880 0x7fc6f4063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 10 monclient: get_auth_request con 0x7fc6f4063880 auth_method 0 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_done global_id 144650 payload 824 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6f9daf640 1 -- 10.20.193.78:0/783010209 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6f4135ac0 con 0x7fc6f4063880 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc6ec00c520 con 0x7fc6f406c890 2026-02-17T06:01:58.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6deffd640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6c403de60 0x7fc6c4040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.621+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(563..563 src has 335..563) ==== 4735+0+0 (secure 0 0 0) 0x7fc6ec052050 con 0x7fc6f406c890 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.622+0000 7fc6fadb1640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6c403de60 0x7fc6c4040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.622+0000 7fc6fadb1640 10 monclient: get_auth_request con 0x7fc6c403de60 auth_method 0 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.623+0000 7fc6f9daf640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f4063880 msgr2=0x7fc6f4063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.623+0000 7fc6f9daf640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f4063880 0x7fc6f4063c60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40a7490 tx=0x7fc6e00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.623+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e003c040 con 0x7fc6f4063880 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.623+0000 7fc6deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.623+0000 7fc6deffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.623+0000 7fc6deffd640 10 monclient: dump: 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fc6e0037d70 con 0x7fc6f4063880 2026-02-17T06:01:58.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6deffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:01:58.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6deffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:01:58.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f4063880 msgr2=0x7fc6f4063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6deffd640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f4063880 0x7fc6f4063c60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40a7490 tx=0x7fc6e00047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f4063880 msgr2=0x7fc6f4063c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:58.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6deffd640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f4063880 0x7fc6f4063c60 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40a7490 tx=0x7fc6e00047c0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6fadb1640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6c403de60 msgr2=0x7fc6c4040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:01:58.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6fadb1640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6c403de60 msgr2=0x7fc6c4040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:01:58.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6fadb1640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6c403de60 0x7fc6c4040320 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7fc6f41365c0 tx=0x7fc6e800d040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:01:58.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.624+0000 7fc6fadb1640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6c403de60 0x7fc6c4040320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6fc83b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6fc83b640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f411cc70 0x7fc6f41da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6fc83b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f4074980 con 0x7fc6f411cc70 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f411cc70 0x7fc6f41da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 10 monclient: get_auth_request con 0x7fc6f411cc70 auth_method 0 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 10 monclient(hunting): handle_auth_done global_id 144653 payload 824 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.627+0000 7fc6f9daf640 1 -- 10.20.193.78:0/783010209 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc6f4125c40 con 0x7fc6f411cc70 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6f9daf640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f411cc70 msgr2=0x7fc6f41da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6f9daf640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f411cc70 0x7fc6f41da560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6f406c6a0 tx=0x7fc6e0009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e003c040 con 0x7fc6f411cc70 2026-02-17T06:01:58.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 10 monclient: dump: 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7fc6e0005c90 con 0x7fc6f411cc70 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f411cc70 msgr2=0x7fc6f41da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f411cc70 0x7fc6f41da560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6f406c6a0 tx=0x7fc6e0009340 comp rx=0 tx=0).stop 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f411cc70 msgr2=0x7fc6f41da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:01:58.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6deffd640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f411cc70 0x7fc6f41da560 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6f406c6a0 tx=0x7fc6e0009340 comp rx=0 tx=0).stop 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 182, 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 302164, 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:58.639 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:58.640 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:01:58.641 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.642 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:58.643 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:01:58.644 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.645 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:58.646 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:01:58.647 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:01:58.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6c403de60 msgr2=0x7fc6c4040320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:01:58.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6fc83b640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6c403de60 0x7fc6c4040320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6fc83b640 10 monclient: shutdown 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6fc83b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f406c890 msgr2=0x7fc6f41cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6fadb1640 1 -- 10.20.193.78:0/783010209 reap_dead start 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.629+0000 7fc6fc83b640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f406c890 0x7fc6f41cf350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00b780 tx=0x7fc6ec00bc50 comp rx=0 tx=0).stop 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.630+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 shutdown_connections 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.630+0000 7fc6fc83b640 1 --2- 10.20.193.78:0/783010209 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc6f406c890 0x7fc6f41cf350 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.630+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 >> 10.20.193.78:0/783010209 conn(0x7fc6f40a6620 msgr2=0x7fc6f40a1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.630+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 shutdown_connections 2026-02-17T06:01:58.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:01:58.630+0000 7fc6fc83b640 1 -- 10.20.193.78:0/783010209 wait complete. 2026-02-17T06:01:58.649 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:01:58.649 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:01:59.262 INFO:tasks.rados.rados.0.trial115.stdout:2624: done (4 left) 2026-02-17T06:01:59.262 INFO:tasks.rados.rados.0.trial115.stdout:2625: done (3 left) 2026-02-17T06:01:59.262 INFO:tasks.rados.rados.0.trial115.stdout:2626: done (2 left) 2026-02-17T06:01:59.262 INFO:tasks.rados.rados.0.trial115.stdout:2627: done (1 left) 2026-02-17T06:01:59.262 INFO:tasks.rados.rados.0.trial115.stdout:2628: done (0 left) 2026-02-17T06:01:59.262 INFO:tasks.rados.rados.0.trial115.stdout:2629: snap_remove snap 263 2026-02-17T06:02:00.264 INFO:tasks.rados.rados.0.trial115.stdout:2629: done (0 left) 2026-02-17T06:02:00.264 INFO:tasks.rados.rados.0.trial115.stdout:2630: copy_from oid 50 from oid 9 current snap is 271 2026-02-17T06:02:00.264 INFO:tasks.rados.rados.0.trial115.stdout:2631: read oid 14 snap -1 2026-02-17T06:02:00.264 INFO:tasks.rados.rados.0.trial115.stdout:2631: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:02:00.264 INFO:tasks.rados.rados.0.trial115.stdout:2632: read oid 27 snap -1 2026-02-17T06:02:00.264 INFO:tasks.rados.rados.0.trial115.stdout:2632: expect (ObjNum 908 snap 264 seq_num 908) 2026-02-17T06:02:00.264 INFO:tasks.rados.rados.0.trial115.stdout:2633: write (excl) oid 19 current snap is 271 2026-02-17T06:02:00.264 INFO:tasks.rados.rados.0.trial115.stdout:2633: seq_num 921 ranges {679247=583042,2044857=609984,3192341=286906} 2026-02-17T06:02:00.269 INFO:tasks.rados.rados.0.trial115.stdout:2633: writing trial11545420-19 from 679247 to 1262289 tid 1 2026-02-17T06:02:00.274 INFO:tasks.rados.rados.0.trial115.stdout:2633: writing trial11545420-19 from 2044857 to 2654841 tid 2 2026-02-17T06:02:00.277 INFO:tasks.rados.rados.0.trial115.stdout:2633: writing trial11545420-19 from 3192341 to 3479247 tid 3 2026-02-17T06:02:00.278 INFO:tasks.rados.rados.0.trial115.stdout:2634: read oid 28 snap -1 2026-02-17T06:02:00.278 INFO:tasks.rados.rados.0.trial115.stdout:2634: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:02:00.278 INFO:tasks.rados.rados.0.trial115.stdout:2635: write oid 29 current snap is 271 2026-02-17T06:02:00.278 INFO:tasks.rados.rados.0.trial115.stdout:2635: seq_num 922 ranges {696054=658291,2103339=753642} 2026-02-17T06:02:00.283 INFO:tasks.rados.rados.0.trial115.stdout:2635: writing trial11545420-29 from 696054 to 1354345 tid 1 2026-02-17T06:02:00.288 INFO:tasks.rados.rados.0.trial115.stdout:2635: writing trial11545420-29 from 2103339 to 2856981 tid 2 2026-02-17T06:02:00.288 INFO:tasks.rados.rados.0.trial115.stdout:2636: snap_remove snap 264 2026-02-17T06:02:01.266 INFO:tasks.rados.rados.0.trial115.stdout:2636: done (6 left) 2026-02-17T06:02:01.266 INFO:tasks.rados.rados.0.trial115.stdout:2637: rollback oid 12 current snap is 271 2026-02-17T06:02:01.266 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 266 2026-02-17T06:02:01.266 INFO:tasks.rados.rados.0.trial115.stdout:2631: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:02:01.421 INFO:tasks.rados.rados.0.trial115.stdout:2630: finishing copy_from to trial11545420-50 2026-02-17T06:02:01.421 INFO:tasks.rados.rados.0.trial115.stdout:2630: got expected ENOENT (src dne) 2026-02-17T06:02:01.421 INFO:tasks.rados.rados.0.trial115.stdout:2630: finishing copy_from racing read to trial11545420-50 2026-02-17T06:02:01.422 INFO:tasks.rados.rados.0.trial115.stdout:2632: expect (ObjNum 908 snap 264 seq_num 908) 2026-02-17T06:02:01.575 INFO:tasks.rados.rados.0.trial115.stdout:2633: finishing write tid 1 to trial11545420-19 2026-02-17T06:02:01.576 INFO:tasks.rados.rados.0.trial115.stdout:2630: done (6 left) 2026-02-17T06:02:01.576 INFO:tasks.rados.rados.0.trial115.stdout:2631: done (5 left) 2026-02-17T06:02:01.576 INFO:tasks.rados.rados.0.trial115.stdout:2632: done (4 left) 2026-02-17T06:02:01.576 INFO:tasks.rados.rados.0.trial115.stdout:2638: write (excl) oid 24 current snap is 271 2026-02-17T06:02:01.576 INFO:tasks.rados.rados.0.trial115.stdout:2633: finishing write tid 2 to trial11545420-19 2026-02-17T06:02:01.576 INFO:tasks.rados.rados.0.trial115.stdout:2638: seq_num 923 ranges {712861=733540,2161821=613653,3247187=265674} 2026-02-17T06:02:01.580 INFO:tasks.rados.rados.0.trial115.stdout:2638: writing trial11545420-24 from 712861 to 1446401 tid 1 2026-02-17T06:02:01.585 INFO:tasks.rados.rados.0.trial115.stdout:2638: writing trial11545420-24 from 2161821 to 2775474 tid 2 2026-02-17T06:02:01.588 INFO:tasks.rados.rados.0.trial115.stdout:2638: writing trial11545420-24 from 3247187 to 3512861 tid 3 2026-02-17T06:02:01.588 INFO:tasks.rados.rados.0.trial115.stdout:2639: read oid 1 snap -1 2026-02-17T06:02:01.588 INFO:tasks.rados.rados.0.trial115.stdout:2639: expect deleted 2026-02-17T06:02:01.588 INFO:tasks.rados.rados.0.trial115.stdout:2640: write (excl) oid 5 current snap is 271 2026-02-17T06:02:01.588 INFO:tasks.rados.rados.0.trial115.stdout:2640: seq_num 924 ranges {729668=408789,1820303=757311,3274610=255058} 2026-02-17T06:02:01.592 INFO:tasks.rados.rados.0.trial115.stdout:2640: writing trial11545420-5 from 729668 to 1138457 tid 1 2026-02-17T06:02:01.598 INFO:tasks.rados.rados.0.trial115.stdout:2640: writing trial11545420-5 from 1820303 to 2577614 tid 2 2026-02-17T06:02:01.601 INFO:tasks.rados.rados.0.trial115.stdout:2640: writing trial11545420-5 from 3274610 to 3529668 tid 3 2026-02-17T06:02:01.601 INFO:tasks.rados.rados.0.trial115.stdout:2641: snap_remove snap 262 2026-02-17T06:02:02.253 INFO:tasks.rados.rados.0.trial115.stdout:2641: done (7 left) 2026-02-17T06:02:02.254 INFO:tasks.rados.rados.0.trial115.stdout:2642: read oid 9 snap -1 2026-02-17T06:02:02.254 INFO:tasks.rados.rados.0.trial115.stdout:2642: expect deleted 2026-02-17T06:02:02.254 INFO:tasks.rados.rados.0.trial115.stdout:2643: read oid 34 snap -1 2026-02-17T06:02:02.254 INFO:tasks.rados.rados.0.trial115.stdout:2643: expect deleted 2026-02-17T06:02:02.254 INFO:tasks.rados.rados.0.trial115.stdout:2644: snap_remove snap 267 2026-02-17T06:02:03.254 INFO:tasks.rados.rados.0.trial115.stdout:2644: done (9 left) 2026-02-17T06:02:03.254 INFO:tasks.rados.rados.0.trial115.stdout:2645: read oid 25 snap -1 2026-02-17T06:02:03.254 INFO:tasks.rados.rados.0.trial115.stdout:2645: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2646: read oid 49 snap -1 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2646: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: finishing write tid 3 to trial11545420-19 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: finishing write tid 4 to trial11545420-19 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2647: snap_create 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: finishing write tid 5 to trial11545420-19 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: oid 19 updating version 0 to 1439 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: oid 19 updating version 1439 to 1441 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: oid 19 updating version 1441 to 1442 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: oid 19 version 1442 is already newer than 1440 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1442 (ObjNum 921 snap 271 seq_num 921) dirty exists 2026-02-17T06:02:03.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: left oid 19 (ObjNum 921 snap 271 seq_num 921) 2026-02-17T06:02:03.256 INFO:tasks.rados.rados.0.trial115.stdout:2634: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:02:03.385 INFO:tasks.rados.rados.0.trial115.stdout:2635: finishing write tid 1 to trial11545420-29 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2635: finishing write tid 2 to trial11545420-29 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2635: finishing write tid 3 to trial11545420-29 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2635: finishing write tid 4 to trial11545420-29 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2635: oid 29 updating version 0 to 1423 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2635: oid 29 updating version 1423 to 1424 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2635: oid 29 version 1424 is already newer than 1422 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1424 (ObjNum 922 snap 271 seq_num 922) dirty exists 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2635: left oid 29 (ObjNum 922 snap 271 seq_num 922) 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2637: finishing rollback tid 0 to trial11545420-12 2026-02-17T06:02:03.386 INFO:tasks.rados.rados.0.trial115.stdout:2637: finishing rollback tid 1 to trial11545420-12 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2637: finishing rollback tid 2 to trial11545420-12 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1421 (ObjNum 910 snap 264 seq_num 910) dirty exists 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: finishing write tid 1 to trial11545420-24 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: finishing write tid 2 to trial11545420-24 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: finishing write tid 3 to trial11545420-24 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: finishing write tid 4 to trial11545420-24 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: finishing write tid 5 to trial11545420-24 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: oid 24 updating version 0 to 1426 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: oid 24 updating version 1426 to 1429 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: oid 24 version 1429 is already newer than 1428 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:2638: oid 24 version 1429 is already newer than 1427 2026-02-17T06:02:03.387 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1429 (ObjNum 923 snap 271 seq_num 923) dirty exists 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2638: left oid 24 (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: finishing write tid 1 to trial11545420-5 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: finishing write tid 2 to trial11545420-5 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: finishing write tid 3 to trial11545420-5 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: finishing write tid 4 to trial11545420-5 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: finishing write tid 5 to trial11545420-5 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: oid 5 updating version 0 to 1402 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: oid 5 version 1402 is already newer than 1401 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: oid 5 version 1402 is already newer than 1400 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:2640: oid 5 version 1402 is already newer than 1399 2026-02-17T06:02:03.388 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1402 (ObjNum 924 snap 271 seq_num 924) dirty exists 2026-02-17T06:02:03.389 INFO:tasks.rados.rados.0.trial115.stdout:2640: left oid 5 (ObjNum 924 snap 271 seq_num 924) 2026-02-17T06:02:03.389 INFO:tasks.rados.rados.0.trial115.stdout:2646: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:03.430 INFO:tasks.rados.rados.0.trial115.stdout:2645: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:02:04.255 INFO:tasks.rados.rados.0.trial115.stdout:2633: done (11 left) 2026-02-17T06:02:04.255 INFO:tasks.rados.rados.0.trial115.stdout:2634: done (10 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2635: done (9 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2637: done (8 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2638: done (7 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2639: done (6 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2640: done (5 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2642: done (4 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2643: done (3 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2645: done (2 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2646: done (1 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2647: done (0 left) 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2648: read oid 34 snap -1 2026-02-17T06:02:04.256 INFO:tasks.rados.rados.0.trial115.stdout:2648: expect deleted 2026-02-17T06:02:04.257 INFO:tasks.rados.rados.0.trial115.stdout:2649: read oid 38 snap -1 2026-02-17T06:02:04.257 INFO:tasks.rados.rados.0.trial115.stdout:2649: expect (ObjNum 874 snap 251 seq_num 874) 2026-02-17T06:02:04.257 INFO:tasks.rados.rados.0.trial115.stdout:2650: write (excl) oid 36 current snap is 272 2026-02-17T06:02:04.257 INFO:tasks.rados.rados.0.trial115.stdout:2650: seq_num 925 ranges {746475=484038,1762432=617322,2785680=529906} 2026-02-17T06:02:04.259 INFO:tasks.rados.rados.0.trial115.stdout:2650: writing trial11545420-36 from 746475 to 1230513 tid 1 2026-02-17T06:02:04.263 INFO:tasks.rados.rados.0.trial115.stdout:2650: writing trial11545420-36 from 1762432 to 2379754 tid 2 2026-02-17T06:02:04.267 INFO:tasks.rados.rados.0.trial115.stdout:2650: writing trial11545420-36 from 2785680 to 3315586 tid 3 2026-02-17T06:02:04.267 INFO:tasks.rados.rados.0.trial115.stdout:2651: write oid 5 current snap is 272 2026-02-17T06:02:04.267 INFO:tasks.rados.rados.0.trial115.stdout:2651: seq_num 926 ranges {763282=559287,1820914=760980,3213103=350179} 2026-02-17T06:02:04.272 INFO:tasks.rados.rados.0.trial115.stdout:2651: writing trial11545420-5 from 763282 to 1322569 tid 1 2026-02-17T06:02:04.276 INFO:tasks.rados.rados.0.trial115.stdout:2651: writing trial11545420-5 from 1820914 to 2581894 tid 2 2026-02-17T06:02:04.281 INFO:tasks.rados.rados.0.trial115.stdout:2651: writing trial11545420-5 from 3213103 to 3563282 tid 3 2026-02-17T06:02:04.281 INFO:tasks.rados.rados.0.trial115.stdout:2652: snap_remove snap 269 2026-02-17T06:02:05.270 INFO:tasks.rados.rados.0.trial115.stdout:2652: done (4 left) 2026-02-17T06:02:05.270 INFO:tasks.rados.rados.0.trial115.stdout:2653: snap_create 2026-02-17T06:02:05.270 INFO:tasks.rados.rados.0.trial115.stdout:2649: expect (ObjNum 874 snap 251 seq_num 874) 2026-02-17T06:02:05.394 INFO:tasks.rados.rados.0.trial115.stdout:2650: finishing write tid 1 to trial11545420-36 2026-02-17T06:02:05.394 INFO:tasks.rados.rados.0.trial115.stdout:2650: finishing write tid 2 to trial11545420-36 2026-02-17T06:02:05.394 INFO:tasks.rados.rados.0.trial115.stdout:2650: finishing write tid 3 to trial11545420-36 2026-02-17T06:02:05.394 INFO:tasks.rados.rados.0.trial115.stdout:2650: finishing write tid 4 to trial11545420-36 2026-02-17T06:02:05.394 INFO:tasks.rados.rados.0.trial115.stdout:2650: finishing write tid 5 to trial11545420-36 2026-02-17T06:02:05.394 INFO:tasks.rados.rados.0.trial115.stdout:2650: oid 36 updating version 0 to 1084 2026-02-17T06:02:05.394 INFO:tasks.rados.rados.0.trial115.stdout:2650: oid 36 updating version 1084 to 1087 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2650: oid 36 version 1087 is already newer than 1086 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2650: oid 36 version 1087 is already newer than 1085 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 1087 (ObjNum 925 snap 272 seq_num 925) dirty exists 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2650: left oid 36 (ObjNum 925 snap 272 seq_num 925) 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: finishing write tid 1 to trial11545420-5 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: finishing write tid 2 to trial11545420-5 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: finishing write tid 3 to trial11545420-5 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: finishing write tid 4 to trial11545420-5 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: finishing write tid 5 to trial11545420-5 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: oid 5 updating version 0 to 1404 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: oid 5 updating version 1404 to 1406 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: oid 5 version 1406 is already newer than 1405 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: oid 5 version 1406 is already newer than 1403 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1406 (ObjNum 926 snap 272 seq_num 926) dirty exists 2026-02-17T06:02:05.395 INFO:tasks.rados.rados.0.trial115.stdout:2651: left oid 5 (ObjNum 926 snap 272 seq_num 926) 2026-02-17T06:02:06.272 INFO:tasks.rados.rados.0.trial115.stdout:2648: done (4 left) 2026-02-17T06:02:06.272 INFO:tasks.rados.rados.0.trial115.stdout:2649: done (3 left) 2026-02-17T06:02:06.273 INFO:tasks.rados.rados.0.trial115.stdout:2650: done (2 left) 2026-02-17T06:02:06.273 INFO:tasks.rados.rados.0.trial115.stdout:2651: done (1 left) 2026-02-17T06:02:06.273 INFO:tasks.rados.rados.0.trial115.stdout:2653: done (0 left) 2026-02-17T06:02:06.273 INFO:tasks.rados.rados.0.trial115.stdout:2654: read oid 19 snap -1 2026-02-17T06:02:06.273 INFO:tasks.rados.rados.0.trial115.stdout:2654: expect (ObjNum 921 snap 271 seq_num 921) 2026-02-17T06:02:06.273 INFO:tasks.rados.rados.0.trial115.stdout:2655: write oid 17 current snap is 273 2026-02-17T06:02:06.273 INFO:tasks.rados.rados.0.trial115.stdout:2655: seq_num 927 ranges {780089=634536,1879396=620991,3240526=339563} 2026-02-17T06:02:06.277 INFO:tasks.rados.rados.0.trial115.stdout:2655: writing trial11545420-17 from 780089 to 1414625 tid 1 2026-02-17T06:02:06.281 INFO:tasks.rados.rados.0.trial115.stdout:2655: writing trial11545420-17 from 1879396 to 2500387 tid 2 2026-02-17T06:02:06.285 INFO:tasks.rados.rados.0.trial115.stdout:2655: writing trial11545420-17 from 3240526 to 3580089 tid 3 2026-02-17T06:02:06.285 INFO:tasks.rados.rados.0.trial115.stdout:2656: delete oid 2 current snap is 273 2026-02-17T06:02:06.285 INFO:tasks.rados.rados.0.trial115.stdout:2654: expect (ObjNum 921 snap 271 seq_num 921) 2026-02-17T06:02:06.452 INFO:tasks.rados.rados.0.trial115.stdout:2655: finishing write tid 1 to trial11545420-17 2026-02-17T06:02:06.452 INFO:tasks.rados.rados.0.trial115.stdout:2655: finishing write tid 2 to trial11545420-17 2026-02-17T06:02:06.452 INFO:tasks.rados.rados.0.trial115.stdout:2654: done (2 left) 2026-02-17T06:02:06.452 INFO:tasks.rados.rados.0.trial115.stdout:2656: done (1 left) 2026-02-17T06:02:06.452 INFO:tasks.rados.rados.0.trial115.stdout:2657: delete oid 40 current snap is 273 2026-02-17T06:02:06.452 INFO:tasks.rados.rados.0.trial115.stdout:2655: finishing write tid 3 to trial11545420-17 2026-02-17T06:02:06.452 INFO:tasks.rados.rados.0.trial115.stdout:2655: finishing write tid 4 to trial11545420-17 2026-02-17T06:02:06.453 INFO:tasks.rados.rados.0.trial115.stdout:2655: finishing write tid 5 to trial11545420-17 2026-02-17T06:02:06.453 INFO:tasks.rados.rados.0.trial115.stdout:2655: oid 17 updating version 0 to 1139 2026-02-17T06:02:06.453 INFO:tasks.rados.rados.0.trial115.stdout:2655: oid 17 updating version 1139 to 1140 2026-02-17T06:02:06.453 INFO:tasks.rados.rados.0.trial115.stdout:2655: oid 17 updating version 1140 to 1141 2026-02-17T06:02:06.453 INFO:tasks.rados.rados.0.trial115.stdout:2655: oid 17 updating version 1141 to 1142 2026-02-17T06:02:06.453 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1142 (ObjNum 927 snap 273 seq_num 927) dirty exists 2026-02-17T06:02:06.453 INFO:tasks.rados.rados.0.trial115.stdout:2655: left oid 17 (ObjNum 927 snap 273 seq_num 927) 2026-02-17T06:02:06.454 INFO:tasks.rados.rados.0.trial115.stdout:2655: done (1 left) 2026-02-17T06:02:06.454 INFO:tasks.rados.rados.0.trial115.stdout:2657: done (0 left) 2026-02-17T06:02:06.454 INFO:tasks.rados.rados.0.trial115.stdout:2658: delete oid 20 current snap is 273 2026-02-17T06:02:06.456 INFO:tasks.rados.rados.0.trial115.stdout:2658: done (0 left) 2026-02-17T06:02:06.456 INFO:tasks.rados.rados.0.trial115.stdout:2659: rollback oid 50 current snap is 273 2026-02-17T06:02:06.457 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 50 to 271 2026-02-17T06:02:06.457 INFO:tasks.rados.rados.0.trial115.stdout:2660: write oid 30 current snap is 273 2026-02-17T06:02:06.457 INFO:tasks.rados.rados.0.trial115.stdout:2660: seq_num 931 ranges {447317=651885,1713324=628329,3066571=580746} 2026-02-17T06:02:06.460 INFO:tasks.rados.rados.0.trial115.stdout:2660: writing trial11545420-30 from 447317 to 1099202 tid 1 2026-02-17T06:02:06.465 INFO:tasks.rados.rados.0.trial115.stdout:2660: writing trial11545420-30 from 1713324 to 2341653 tid 2 2026-02-17T06:02:06.470 INFO:tasks.rados.rados.0.trial115.stdout:2660: writing trial11545420-30 from 3066571 to 3647317 tid 3 2026-02-17T06:02:06.470 INFO:tasks.rados.rados.0.trial115.stdout:2661: rollback oid 43 current snap is 273 2026-02-17T06:02:06.470 INFO:tasks.rados.rados.0.trial115.stdout:2659: finishing rollback tid 0 to trial11545420-50 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:2659: finishing rollback tid 1 to trial11545420-50 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:2659: finishing rollback tid 2 to trial11545420-50 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1173 (ObjNum 751 snap 206 seq_num 751) dirty exists 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:2660: finishing write tid 1 to trial11545420-30 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 270 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:2660: finishing write tid 2 to trial11545420-30 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:2659: done (2 left) 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:2662: write (excl) oid 15 current snap is 273 2026-02-17T06:02:06.471 INFO:tasks.rados.rados.0.trial115.stdout:2662: seq_num 932 ranges {464124=727134,1771806=771987,2977641=641516} 2026-02-17T06:02:06.475 INFO:tasks.rados.rados.0.trial115.stdout:2662: writing trial11545420-15 from 464124 to 1191258 tid 1 2026-02-17T06:02:06.479 INFO:tasks.rados.rados.0.trial115.stdout:2662: writing trial11545420-15 from 1771806 to 2543793 tid 2 2026-02-17T06:02:06.484 INFO:tasks.rados.rados.0.trial115.stdout:2662: writing trial11545420-15 from 2977641 to 3619157 tid 3 2026-02-17T06:02:06.484 INFO:tasks.rados.rados.0.trial115.stdout:2663: copy_from oid 2 from oid 41 current snap is 273 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:2661: finishing rollback tid 1 to trial11545420-43 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 0 (ObjNum 129 snap 0 seq_num 1334570432) dirty dne 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:2660: finishing write tid 3 to trial11545420-30 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:2660: finishing write tid 4 to trial11545420-30 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:2660: finishing write tid 5 to trial11545420-30 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:2660: oid 30 updating version 0 to 1176 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:2660: oid 30 version 1176 is already newer than 1175 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:2660: oid 30 updating version 1176 to 1177 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:2660: oid 30 version 1177 is already newer than 1174 2026-02-17T06:02:06.485 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1177 (ObjNum 931 snap 273 seq_num 931) dirty exists 2026-02-17T06:02:06.486 INFO:tasks.rados.rados.0.trial115.stdout:2660: left oid 30 (ObjNum 931 snap 273 seq_num 931) 2026-02-17T06:02:06.486 INFO:tasks.rados.rados.0.trial115.stdout:2662: finishing write tid 1 to trial11545420-15 2026-02-17T06:02:06.486 INFO:tasks.rados.rados.0.trial115.stdout:2660: done (3 left) 2026-02-17T06:02:06.486 INFO:tasks.rados.rados.0.trial115.stdout:2661: done (2 left) 2026-02-17T06:02:06.486 INFO:tasks.rados.rados.0.trial115.stdout:2664: copy_from oid 26 from oid 14 current snap is 273 2026-02-17T06:02:06.486 INFO:tasks.rados.rados.0.trial115.stdout:2662: finishing write tid 2 to trial11545420-15 2026-02-17T06:02:06.487 INFO:tasks.rados.rados.0.trial115.stdout:2665: write oid 32 current snap is 273 2026-02-17T06:02:06.487 INFO:tasks.rados.rados.0.trial115.stdout:2665: seq_num 933 ranges {480931=402383,1313935=515645,2488711=452788,3443230=237701} 2026-02-17T06:02:06.487 INFO:tasks.rados.rados.0.trial115.stdout:2665: writing trial11545420-32 from 480931 to 883314 tid 1 2026-02-17T06:02:06.491 INFO:tasks.rados.rados.0.trial115.stdout:2665: writing trial11545420-32 from 1313935 to 1829580 tid 2 2026-02-17T06:02:06.495 INFO:tasks.rados.rados.0.trial115.stdout:2665: writing trial11545420-32 from 2488711 to 2941499 tid 3 2026-02-17T06:02:06.500 INFO:tasks.rados.rados.0.trial115.stdout:2665: writing trial11545420-32 from 3443230 to 3680931 tid 4 2026-02-17T06:02:06.500 INFO:tasks.rados.rados.0.trial115.stdout:2666: snap_create 2026-02-17T06:02:06.500 INFO:tasks.rados.rados.0.trial115.stdout:2662: finishing write tid 3 to trial11545420-15 2026-02-17T06:02:06.500 INFO:tasks.rados.rados.0.trial115.stdout:2662: finishing write tid 4 to trial11545420-15 2026-02-17T06:02:06.501 INFO:tasks.rados.rados.0.trial115.stdout:2662: finishing write tid 5 to trial11545420-15 2026-02-17T06:02:06.501 INFO:tasks.rados.rados.0.trial115.stdout:2662: oid 15 updating version 0 to 1344 2026-02-17T06:02:06.501 INFO:tasks.rados.rados.0.trial115.stdout:2662: oid 15 version 1344 is already newer than 1342 2026-02-17T06:02:06.501 INFO:tasks.rados.rados.0.trial115.stdout:2662: oid 15 updating version 1344 to 1345 2026-02-17T06:02:06.501 INFO:tasks.rados.rados.0.trial115.stdout:2662: oid 15 version 1345 is already newer than 1343 2026-02-17T06:02:06.501 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1345 (ObjNum 932 snap 273 seq_num 932) dirty exists 2026-02-17T06:02:06.502 INFO:tasks.rados.rados.0.trial115.stdout:2662: left oid 15 (ObjNum 932 snap 273 seq_num 932) 2026-02-17T06:02:06.502 INFO:tasks.rados.rados.0.trial115.stdout:2665: finishing write tid 1 to trial11545420-32 2026-02-17T06:02:06.502 INFO:tasks.rados.rados.0.trial115.stdout:2663: finishing copy_from racing read to trial11545420-2 2026-02-17T06:02:06.502 INFO:tasks.rados.rados.0.trial115.stdout:2664: finishing copy_from racing read to trial11545420-26 2026-02-17T06:02:06.502 INFO:tasks.rados.rados.0.trial115.stdout:2663: finishing copy_from to trial11545420-2 2026-02-17T06:02:06.502 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1392 (ObjNum 917 snap 265 seq_num 917) dirty exists 2026-02-17T06:02:06.503 INFO:tasks.rados.rados.0.trial115.stdout:2665: finishing write tid 2 to trial11545420-32 2026-02-17T06:02:06.503 INFO:tasks.rados.rados.0.trial115.stdout:2664: finishing copy_from to trial11545420-26 2026-02-17T06:02:06.503 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 962 (ObjNum 862 snap 246 seq_num 862) dirty exists 2026-02-17T06:02:06.503 INFO:tasks.rados.rados.0.trial115.stdout:2665: finishing write tid 3 to trial11545420-32 2026-02-17T06:02:06.503 INFO:tasks.rados.rados.0.trial115.stdout:2665: finishing write tid 4 to trial11545420-32 2026-02-17T06:02:06.503 INFO:tasks.rados.rados.0.trial115.stdout:2665: finishing write tid 5 to trial11545420-32 2026-02-17T06:02:06.504 INFO:tasks.rados.rados.0.trial115.stdout:2665: finishing write tid 6 to trial11545420-32 2026-02-17T06:02:06.504 INFO:tasks.rados.rados.0.trial115.stdout:2665: oid 32 updating version 0 to 1140 2026-02-17T06:02:06.504 INFO:tasks.rados.rados.0.trial115.stdout:2665: oid 32 version 1140 is already newer than 1139 2026-02-17T06:02:06.504 INFO:tasks.rados.rados.0.trial115.stdout:2665: oid 32 updating version 1140 to 1141 2026-02-17T06:02:06.504 INFO:tasks.rados.rados.0.trial115.stdout:2665: oid 32 updating version 1141 to 1143 2026-02-17T06:02:06.504 INFO:tasks.rados.rados.0.trial115.stdout:2665: oid 32 version 1143 is already newer than 1142 2026-02-17T06:02:06.505 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1143 (ObjNum 933 snap 273 seq_num 933) dirty exists 2026-02-17T06:02:06.505 INFO:tasks.rados.rados.0.trial115.stdout:2665: left oid 32 (ObjNum 933 snap 273 seq_num 933) 2026-02-17T06:02:07.255 INFO:tasks.rados.rados.0.trial115.stdout:2662: done (4 left) 2026-02-17T06:02:07.256 INFO:tasks.rados.rados.0.trial115.stdout:2663: done (3 left) 2026-02-17T06:02:07.256 INFO:tasks.rados.rados.0.trial115.stdout:2664: done (2 left) 2026-02-17T06:02:07.256 INFO:tasks.rados.rados.0.trial115.stdout:2665: done (1 left) 2026-02-17T06:02:07.256 INFO:tasks.rados.rados.0.trial115.stdout:2666: done (0 left) 2026-02-17T06:02:07.256 INFO:tasks.rados.rados.0.trial115.stdout:2667: read oid 41 snap 270 2026-02-17T06:02:07.256 INFO:tasks.rados.rados.0.trial115.stdout:2667: expect (ObjNum 917 snap 265 seq_num 917) 2026-02-17T06:02:07.256 INFO:tasks.rados.rados.0.trial115.stdout:2668: write (excl) oid 4 current snap is 274 2026-02-17T06:02:07.256 INFO:tasks.rados.rados.0.trial115.stdout:2668: seq_num 934 ranges {497738=477632,1772417=775656,3316134=381604} 2026-02-17T06:02:07.259 INFO:tasks.rados.rados.0.trial115.stdout:2668: writing trial11545420-4 from 497738 to 975370 tid 1 2026-02-17T06:02:07.264 INFO:tasks.rados.rados.0.trial115.stdout:2668: writing trial11545420-4 from 1772417 to 2548073 tid 2 2026-02-17T06:02:07.269 INFO:tasks.rados.rados.0.trial115.stdout:2668: writing trial11545420-4 from 3316134 to 3697738 tid 3 2026-02-17T06:02:07.269 INFO:tasks.rados.rados.0.trial115.stdout:2669: snap_remove snap 271 2026-02-17T06:02:08.258 INFO:tasks.rados.rados.0.trial115.stdout:2669: done (2 left) 2026-02-17T06:02:08.258 INFO:tasks.rados.rados.0.trial115.stdout:2670: write oid 6 current snap is 274 2026-02-17T06:02:08.258 INFO:tasks.rados.rados.0.trial115.stdout:2670: seq_num 935 ranges {514545=552881,1830899=519314,2943557=475332} 2026-02-17T06:02:08.261 INFO:tasks.rados.rados.0.trial115.stdout:2670: writing trial11545420-6 from 514545 to 1067426 tid 1 2026-02-17T06:02:08.266 INFO:tasks.rados.rados.0.trial115.stdout:2670: writing trial11545420-6 from 1830899 to 2350213 tid 2 2026-02-17T06:02:08.270 INFO:tasks.rados.rados.0.trial115.stdout:2670: writing trial11545420-6 from 2943557 to 3418889 tid 3 2026-02-17T06:02:08.270 INFO:tasks.rados.rados.0.trial115.stdout:2671: read oid 26 snap 273 2026-02-17T06:02:08.270 INFO:tasks.rados.rados.0.trial115.stdout:2671: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:02:08.270 INFO:tasks.rados.rados.0.trial115.stdout:2672: read oid 46 snap 265 2026-02-17T06:02:08.270 INFO:tasks.rados.rados.0.trial115.stdout:2672: expect (ObjNum 889 snap 254 seq_num 889) 2026-02-17T06:02:08.271 INFO:tasks.rados.rados.0.trial115.stdout:2673: copy_from oid 43 from oid 22 current snap is 274 2026-02-17T06:02:08.271 INFO:tasks.rados.rados.0.trial115.stdout:2667: expect (ObjNum 917 snap 265 seq_num 917) 2026-02-17T06:02:08.352 INFO:tasks.rados.rados.0.trial115.stdout:2668: finishing write tid 1 to trial11545420-4 2026-02-17T06:02:08.352 INFO:tasks.rados.rados.0.trial115.stdout:2668: finishing write tid 2 to trial11545420-4 2026-02-17T06:02:08.352 INFO:tasks.rados.rados.0.trial115.stdout:2667: done (5 left) 2026-02-17T06:02:08.353 INFO:tasks.rados.rados.0.trial115.stdout:2674: snap_remove snap 268 2026-02-17T06:02:09.260 INFO:tasks.rados.rados.0.trial115.stdout:2674: done (5 left) 2026-02-17T06:02:09.260 INFO:tasks.rados.rados.0.trial115.stdout:2675: delete oid 33 current snap is 274 2026-02-17T06:02:09.260 INFO:tasks.rados.rados.0.trial115.stdout:2668: finishing write tid 3 to trial11545420-4 2026-02-17T06:02:09.260 INFO:tasks.rados.rados.0.trial115.stdout:2668: finishing write tid 4 to trial11545420-4 2026-02-17T06:02:09.260 INFO:tasks.rados.rados.0.trial115.stdout:2668: finishing write tid 5 to trial11545420-4 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2668: oid 4 updating version 0 to 1289 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2668: oid 4 updating version 1289 to 1290 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2668: oid 4 version 1290 is already newer than 1288 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2668: oid 4 updating version 1290 to 1291 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1291 (ObjNum 934 snap 274 seq_num 934) dirty exists 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2668: left oid 4 (ObjNum 934 snap 274 seq_num 934) 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2670: finishing write tid 1 to trial11545420-6 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2670: finishing write tid 2 to trial11545420-6 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2670: finishing write tid 3 to trial11545420-6 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2670: finishing write tid 4 to trial11545420-6 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2670: finishing write tid 5 to trial11545420-6 2026-02-17T06:02:09.261 INFO:tasks.rados.rados.0.trial115.stdout:2670: oid 6 updating version 0 to 970 2026-02-17T06:02:09.262 INFO:tasks.rados.rados.0.trial115.stdout:2670: oid 6 version 970 is already newer than 968 2026-02-17T06:02:09.262 INFO:tasks.rados.rados.0.trial115.stdout:2670: oid 6 version 970 is already newer than 969 2026-02-17T06:02:09.262 INFO:tasks.rados.rados.0.trial115.stdout:2670: oid 6 version 970 is already newer than 967 2026-02-17T06:02:09.262 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 970 (ObjNum 935 snap 274 seq_num 935) dirty exists 2026-02-17T06:02:09.262 INFO:tasks.rados.rados.0.trial115.stdout:2670: left oid 6 (ObjNum 935 snap 274 seq_num 935) 2026-02-17T06:02:09.262 INFO:tasks.rados.rados.0.trial115.stdout:2673: finishing copy_from racing read to trial11545420-43 2026-02-17T06:02:09.262 INFO:tasks.rados.rados.0.trial115.stdout:2671: expect (ObjNum 862 snap 246 seq_num 862) 2026-02-17T06:02:09.413 INFO:tasks.rados.rados.0.trial115.stdout:2673: finishing copy_from to trial11545420-43 2026-02-17T06:02:09.413 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:02:09.413 INFO:tasks.rados.rados.0.trial115.stdout:2672: expect (ObjNum 889 snap 254 seq_num 889) 2026-02-17T06:02:09.566 INFO:tasks.rados.rados.0.trial115.stdout:2668: done (5 left) 2026-02-17T06:02:09.566 INFO:tasks.rados.rados.0.trial115.stdout:2670: done (4 left) 2026-02-17T06:02:09.566 INFO:tasks.rados.rados.0.trial115.stdout:2671: done (3 left) 2026-02-17T06:02:09.566 INFO:tasks.rados.rados.0.trial115.stdout:2672: done (2 left) 2026-02-17T06:02:09.566 INFO:tasks.rados.rados.0.trial115.stdout:2673: done (1 left) 2026-02-17T06:02:09.567 INFO:tasks.rados.rados.0.trial115.stdout:2675: done (0 left) 2026-02-17T06:02:09.567 INFO:tasks.rados.rados.0.trial115.stdout:2676: write oid 31 current snap is 274 2026-02-17T06:02:09.567 INFO:tasks.rados.rados.0.trial115.stdout:2676: seq_num 937 ranges {548159=419732,1664216=522983,2714756=614229} 2026-02-17T06:02:09.569 INFO:tasks.rados.rados.0.trial115.stdout:2676: writing trial11545420-31 from 548159 to 967891 tid 1 2026-02-17T06:02:09.574 INFO:tasks.rados.rados.0.trial115.stdout:2676: writing trial11545420-31 from 1664216 to 2187199 tid 2 2026-02-17T06:02:09.578 INFO:tasks.rados.rados.0.trial115.stdout:2676: writing trial11545420-31 from 2714756 to 3328985 tid 3 2026-02-17T06:02:09.579 INFO:tasks.rados.rados.0.trial115.stdout:2677: read oid 46 snap -1 2026-02-17T06:02:09.579 INFO:tasks.rados.rados.0.trial115.stdout:2677: expect (ObjNum 889 snap 254 seq_num 889) 2026-02-17T06:02:09.579 INFO:tasks.rados.rados.0.trial115.stdout:2678: read oid 50 snap 270 2026-02-17T06:02:09.579 INFO:tasks.rados.rados.0.trial115.stdout:2678: expect (ObjNum 751 snap 206 seq_num 751) 2026-02-17T06:02:09.579 INFO:tasks.rados.rados.0.trial115.stdout:2679: snap_remove snap 266 2026-02-17T06:02:10.262 INFO:tasks.rados.rados.0.trial115.stdout:2679: done (3 left) 2026-02-17T06:02:10.262 INFO:tasks.rados.rados.0.trial115.stdout:2680: read oid 7 snap 273 2026-02-17T06:02:10.262 INFO:tasks.rados.rados.0.trial115.stdout:2680: expect (ObjNum 888 snap 254 seq_num 888) 2026-02-17T06:02:10.263 INFO:tasks.rados.rados.0.trial115.stdout:2681: rollback oid 12 current snap is 274 2026-02-17T06:02:10.263 INFO:tasks.rados.rados.0.trial115.stdout:2676: finishing write tid 1 to trial11545420-31 2026-02-17T06:02:10.263 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 273 2026-02-17T06:02:10.263 INFO:tasks.rados.rados.0.trial115.stdout:2676: finishing write tid 2 to trial11545420-31 2026-02-17T06:02:10.263 INFO:tasks.rados.rados.0.trial115.stdout:2677: expect (ObjNum 889 snap 254 seq_num 889) 2026-02-17T06:02:10.416 INFO:tasks.rados.rados.0.trial115.stdout:2676: finishing write tid 3 to trial11545420-31 2026-02-17T06:02:10.416 INFO:tasks.rados.rados.0.trial115.stdout:2676: finishing write tid 4 to trial11545420-31 2026-02-17T06:02:10.417 INFO:tasks.rados.rados.0.trial115.stdout:2676: finishing write tid 5 to trial11545420-31 2026-02-17T06:02:10.417 INFO:tasks.rados.rados.0.trial115.stdout:2676: oid 31 updating version 0 to 1411 2026-02-17T06:02:10.417 INFO:tasks.rados.rados.0.trial115.stdout:2676: oid 31 version 1411 is already newer than 1409 2026-02-17T06:02:10.417 INFO:tasks.rados.rados.0.trial115.stdout:2676: oid 31 version 1411 is already newer than 1410 2026-02-17T06:02:10.417 INFO:tasks.rados.rados.0.trial115.stdout:2676: oid 31 version 1411 is already newer than 1408 2026-02-17T06:02:10.417 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1411 (ObjNum 937 snap 274 seq_num 937) dirty exists 2026-02-17T06:02:10.417 INFO:tasks.rados.rados.0.trial115.stdout:2676: left oid 31 (ObjNum 937 snap 274 seq_num 937) 2026-02-17T06:02:10.417 INFO:tasks.rados.rados.0.trial115.stdout:2678: expect (ObjNum 751 snap 206 seq_num 751) 2026-02-17T06:02:10.475 INFO:tasks.rados.rados.0.trial115.stdout:2681: finishing rollback tid 0 to trial11545420-12 2026-02-17T06:02:10.475 INFO:tasks.rados.rados.0.trial115.stdout:2681: finishing rollback tid 1 to trial11545420-12 2026-02-17T06:02:10.475 INFO:tasks.rados.rados.0.trial115.stdout:2681: finishing rollback tid 2 to trial11545420-12 2026-02-17T06:02:10.475 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1457 (ObjNum 910 snap 264 seq_num 910) dirty exists 2026-02-17T06:02:10.475 INFO:tasks.rados.rados.0.trial115.stdout:2680: expect (ObjNum 888 snap 254 seq_num 888) 2026-02-17T06:02:10.568 INFO:tasks.rados.rados.0.trial115.stdout:2676: done (4 left) 2026-02-17T06:02:10.568 INFO:tasks.rados.rados.0.trial115.stdout:2677: done (3 left) 2026-02-17T06:02:10.569 INFO:tasks.rados.rados.0.trial115.stdout:2678: done (2 left) 2026-02-17T06:02:10.569 INFO:tasks.rados.rados.0.trial115.stdout:2680: done (1 left) 2026-02-17T06:02:10.569 INFO:tasks.rados.rados.0.trial115.stdout:2681: done (0 left) 2026-02-17T06:02:10.569 INFO:tasks.rados.rados.0.trial115.stdout:2682: delete oid 14 current snap is 274 2026-02-17T06:02:10.570 INFO:tasks.rados.rados.0.trial115.stdout:2682: done (0 left) 2026-02-17T06:02:10.570 INFO:tasks.rados.rados.0.trial115.stdout:2683: snap_remove snap 270 2026-02-17T06:02:11.265 INFO:tasks.rados.rados.0.trial115.stdout:2683: done (0 left) 2026-02-17T06:02:11.265 INFO:tasks.rados.rados.0.trial115.stdout:2684: write oid 3 current snap is 274 2026-02-17T06:02:11.265 INFO:tasks.rados.rados.0.trial115.stdout:2684: seq_num 939 ranges {581773=570230,1664827=526652,2653249=636773} 2026-02-17T06:02:11.269 INFO:tasks.rados.rados.0.trial115.stdout:2684: writing trial11545420-3 from 581773 to 1152003 tid 1 2026-02-17T06:02:11.273 INFO:tasks.rados.rados.0.trial115.stdout:2684: writing trial11545420-3 from 1664827 to 2191479 tid 2 2026-02-17T06:02:11.277 INFO:tasks.rados.rados.0.trial115.stdout:2684: writing trial11545420-3 from 2653249 to 3290022 tid 3 2026-02-17T06:02:11.277 INFO:tasks.rados.rados.0.trial115.stdout:2685: copy_from oid 38 from oid 41 current snap is 274 2026-02-17T06:02:11.277 INFO:tasks.rados.rados.0.trial115.stdout:2684: finishing write tid 1 to trial11545420-3 2026-02-17T06:02:11.277 INFO:tasks.rados.rados.0.trial115.stdout:2684: finishing write tid 2 to trial11545420-3 2026-02-17T06:02:11.277 INFO:tasks.rados.rados.0.trial115.stdout:2686: delete oid 37 current snap is 274 2026-02-17T06:02:11.279 INFO:tasks.rados.rados.0.trial115.stdout:2684: finishing write tid 3 to trial11545420-3 2026-02-17T06:02:11.279 INFO:tasks.rados.rados.0.trial115.stdout:2684: finishing write tid 4 to trial11545420-3 2026-02-17T06:02:11.280 INFO:tasks.rados.rados.0.trial115.stdout:2686: done (2 left) 2026-02-17T06:02:11.280 INFO:tasks.rados.rados.0.trial115.stdout:2687: write oid 39 current snap is 274 2026-02-17T06:02:11.280 INFO:tasks.rados.rados.0.trial115.stdout:2687: seq_num 941 ranges {615387=720728,1781791=530321,3108095=707292} 2026-02-17T06:02:11.284 INFO:tasks.rados.rados.0.trial115.stdout:2687: writing trial11545420-39 from 615387 to 1336115 tid 1 2026-02-17T06:02:11.288 INFO:tasks.rados.rados.0.trial115.stdout:2687: writing trial11545420-39 from 1781791 to 2312112 tid 2 2026-02-17T06:02:11.295 INFO:tasks.rados.rados.0.trial115.stdout:2687: writing trial11545420-39 from 3108095 to 3815387 tid 3 2026-02-17T06:02:11.295 INFO:tasks.rados.rados.0.trial115.stdout:2688: copy_from oid 49 from oid 1 current snap is 274 2026-02-17T06:02:11.295 INFO:tasks.rados.rados.0.trial115.stdout:2684: finishing write tid 5 to trial11545420-3 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2684: oid 3 updating version 0 to 1415 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2684: oid 3 version 1415 is already newer than 1414 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2684: oid 3 version 1415 is already newer than 1412 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2684: oid 3 version 1415 is already newer than 1413 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1415 (ObjNum 939 snap 274 seq_num 939) dirty exists 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2684: left oid 3 (ObjNum 939 snap 274 seq_num 939) 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2685: finishing copy_from racing read to trial11545420-38 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2685: finishing copy_from to trial11545420-38 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1392 (ObjNum 917 snap 265 seq_num 917) dirty exists 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2687: finishing write tid 1 to trial11545420-39 2026-02-17T06:02:11.296 INFO:tasks.rados.rados.0.trial115.stdout:2687: finishing write tid 2 to trial11545420-39 2026-02-17T06:02:11.297 INFO:tasks.rados.rados.0.trial115.stdout:2684: done (3 left) 2026-02-17T06:02:11.297 INFO:tasks.rados.rados.0.trial115.stdout:2685: done (2 left) 2026-02-17T06:02:11.297 INFO:tasks.rados.rados.0.trial115.stdout:2689: snap_remove snap 265 2026-02-17T06:02:12.255 INFO:tasks.rados.rados.0.trial115.stdout:2689: done (2 left) 2026-02-17T06:02:12.256 INFO:tasks.rados.rados.0.trial115.stdout:2690: write oid 32 current snap is 274 2026-02-17T06:02:12.256 INFO:tasks.rados.rados.0.trial115.stdout:2690: seq_num 942 ranges {632194=795977,2123920=790332,3419165=413029} 2026-02-17T06:02:12.261 INFO:tasks.rados.rados.0.trial115.stdout:2690: writing trial11545420-32 from 632194 to 1428171 tid 1 2026-02-17T06:02:12.266 INFO:tasks.rados.rados.0.trial115.stdout:2690: writing trial11545420-32 from 2123920 to 2914252 tid 2 2026-02-17T06:02:12.270 INFO:tasks.rados.rados.0.trial115.stdout:2690: writing trial11545420-32 from 3419165 to 3832194 tid 3 2026-02-17T06:02:12.270 INFO:tasks.rados.rados.0.trial115.stdout:2691: write (excl) oid 17 current snap is 274 2026-02-17T06:02:12.270 INFO:tasks.rados.rados.0.trial115.stdout:2691: seq_num 943 ranges {649001=587579,1898755=533990,3162941=686060} 2026-02-17T06:02:12.274 INFO:tasks.rados.rados.0.trial115.stdout:2691: writing trial11545420-17 from 649001 to 1236580 tid 1 2026-02-17T06:02:12.279 INFO:tasks.rados.rados.0.trial115.stdout:2691: writing trial11545420-17 from 1898755 to 2432745 tid 2 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2691: writing trial11545420-17 from 3162941 to 3849001 tid 3 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2692: read oid 5 snap -1 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2692: expect (ObjNum 926 snap 272 seq_num 926) 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2693: read oid 44 snap 273 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2693: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2694: delete oid 36 current snap is 274 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2688: finishing copy_from to trial11545420-49 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2688: got expected ENOENT (src dne) 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2688: finishing copy_from racing read to trial11545420-49 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2687: finishing write tid 3 to trial11545420-39 2026-02-17T06:02:12.284 INFO:tasks.rados.rados.0.trial115.stdout:2687: finishing write tid 4 to trial11545420-39 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2687: finishing write tid 5 to trial11545420-39 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2687: oid 39 updating version 0 to 1471 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2687: oid 39 version 1471 is already newer than 1469 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2687: oid 39 version 1471 is already newer than 1470 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2687: oid 39 updating version 1471 to 1472 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1472 (ObjNum 941 snap 274 seq_num 941) dirty exists 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2687: left oid 39 (ObjNum 941 snap 274 seq_num 941) 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: finishing write tid 1 to trial11545420-32 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: finishing write tid 2 to trial11545420-32 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: finishing write tid 3 to trial11545420-32 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: finishing write tid 4 to trial11545420-32 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: finishing write tid 5 to trial11545420-32 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: oid 32 updating version 0 to 1144 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: oid 32 updating version 1144 to 1146 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: oid 32 version 1146 is already newer than 1145 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: oid 32 updating version 1146 to 1147 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1147 (ObjNum 942 snap 274 seq_num 942) dirty exists 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2690: left oid 32 (ObjNum 942 snap 274 seq_num 942) 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2691: finishing write tid 1 to trial11545420-17 2026-02-17T06:02:12.285 INFO:tasks.rados.rados.0.trial115.stdout:2691: finishing write tid 2 to trial11545420-17 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2691: finishing write tid 3 to trial11545420-17 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2687: done (6 left) 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2688: done (5 left) 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2690: done (4 left) 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2694: done (3 left) 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2695: delete oid 37 current snap is 274 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2691: finishing write tid 4 to trial11545420-17 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2691: finishing write tid 5 to trial11545420-17 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2691: oid 17 updating version 0 to 1146 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2691: oid 17 version 1146 is already newer than 1143 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2691: oid 17 version 1146 is already newer than 1145 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2691: oid 17 version 1146 is already newer than 1144 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1146 (ObjNum 943 snap 274 seq_num 943) dirty exists 2026-02-17T06:02:12.287 INFO:tasks.rados.rados.0.trial115.stdout:2691: left oid 17 (ObjNum 943 snap 274 seq_num 943) 2026-02-17T06:02:12.289 INFO:tasks.rados.rados.0.trial115.stdout:2692: expect (ObjNum 926 snap 272 seq_num 926) 2026-02-17T06:02:12.391 INFO:tasks.rados.rados.0.trial115.stdout:2693: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:12.437 INFO:tasks.rados.rados.0.trial115.stdout:2691: done (3 left) 2026-02-17T06:02:12.437 INFO:tasks.rados.rados.0.trial115.stdout:2692: done (2 left) 2026-02-17T06:02:12.437 INFO:tasks.rados.rados.0.trial115.stdout:2693: done (1 left) 2026-02-17T06:02:12.437 INFO:tasks.rados.rados.0.trial115.stdout:2695: done (0 left) 2026-02-17T06:02:12.437 INFO:tasks.rados.rados.0.trial115.stdout:2696: snap_remove snap 272 2026-02-17T06:02:13.269 INFO:tasks.rados.rados.0.trial115.stdout:2696: done (0 left) 2026-02-17T06:02:13.269 INFO:tasks.rados.rados.0.trial115.stdout:2697: rollback oid 31 current snap is 274 2026-02-17T06:02:13.269 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 31 to 273 2026-02-17T06:02:13.269 INFO:tasks.rados.rados.0.trial115.stdout:2698: snap_create 2026-02-17T06:02:13.274 INFO:tasks.rados.rados.0.trial115.stdout:2697: finishing rollback tid 0 to trial11545420-31 2026-02-17T06:02:13.274 INFO:tasks.rados.rados.0.trial115.stdout:2697: finishing rollback tid 1 to trial11545420-31 2026-02-17T06:02:13.274 INFO:tasks.rados.rados.0.trial115.stdout:2697: finishing rollback tid 2 to trial11545420-31 2026-02-17T06:02:13.274 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1418 (ObjNum 887 snap 254 seq_num 887) dirty exists 2026-02-17T06:02:14.272 INFO:tasks.rados.rados.0.trial115.stdout:2697: done (1 left) 2026-02-17T06:02:14.273 INFO:tasks.rados.rados.0.trial115.stdout:2698: done (0 left) 2026-02-17T06:02:14.273 INFO:tasks.rados.rados.0.trial115.stdout:2699: copy_from oid 46 from oid 11 current snap is 275 2026-02-17T06:02:14.273 INFO:tasks.rados.rados.0.trial115.stdout:2700: write (excl) oid 9 current snap is 275 2026-02-17T06:02:14.273 INFO:tasks.rados.rados.0.trial115.stdout:2700: seq_num 946 ranges {699422=413326,1557848=681317,3012504=748383} 2026-02-17T06:02:14.278 INFO:tasks.rados.rados.0.trial115.stdout:2700: writing trial11545420-9 from 699422 to 1112748 tid 1 2026-02-17T06:02:14.282 INFO:tasks.rados.rados.0.trial115.stdout:2700: writing trial11545420-9 from 1557848 to 2239165 tid 2 2026-02-17T06:02:14.288 INFO:tasks.rados.rados.0.trial115.stdout:2700: writing trial11545420-9 from 3012504 to 3760887 tid 3 2026-02-17T06:02:14.288 INFO:tasks.rados.rados.0.trial115.stdout:2701: snap_create 2026-02-17T06:02:14.288 INFO:tasks.rados.rados.0.trial115.stdout:2699: finishing copy_from racing read to trial11545420-46 2026-02-17T06:02:14.288 INFO:tasks.rados.rados.0.trial115.stdout:2699: finishing copy_from to trial11545420-46 2026-02-17T06:02:14.289 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1301 (ObjNum 885 snap 254 seq_num 885) dirty exists 2026-02-17T06:02:14.289 INFO:tasks.rados.rados.0.trial115.stdout:2700: finishing write tid 1 to trial11545420-9 2026-02-17T06:02:14.289 INFO:tasks.rados.rados.0.trial115.stdout:2700: finishing write tid 2 to trial11545420-9 2026-02-17T06:02:14.290 INFO:tasks.rados.rados.0.trial115.stdout:2700: finishing write tid 3 to trial11545420-9 2026-02-17T06:02:14.290 INFO:tasks.rados.rados.0.trial115.stdout:2700: finishing write tid 4 to trial11545420-9 2026-02-17T06:02:14.290 INFO:tasks.rados.rados.0.trial115.stdout:2700: finishing write tid 5 to trial11545420-9 2026-02-17T06:02:14.290 INFO:tasks.rados.rados.0.trial115.stdout:2700: oid 9 updating version 0 to 1367 2026-02-17T06:02:14.291 INFO:tasks.rados.rados.0.trial115.stdout:2700: oid 9 version 1367 is already newer than 1366 2026-02-17T06:02:14.291 INFO:tasks.rados.rados.0.trial115.stdout:2700: oid 9 updating version 1367 to 1368 2026-02-17T06:02:14.291 INFO:tasks.rados.rados.0.trial115.stdout:2700: oid 9 updating version 1368 to 1369 2026-02-17T06:02:14.291 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1369 (ObjNum 946 snap 275 seq_num 946) dirty exists 2026-02-17T06:02:14.291 INFO:tasks.rados.rados.0.trial115.stdout:2700: left oid 9 (ObjNum 946 snap 275 seq_num 946) 2026-02-17T06:02:15.273 INFO:tasks.rados.rados.0.trial115.stdout:2699: done (2 left) 2026-02-17T06:02:15.273 INFO:tasks.rados.rados.0.trial115.stdout:2700: done (1 left) 2026-02-17T06:02:15.273 INFO:tasks.rados.rados.0.trial115.stdout:2701: done (0 left) 2026-02-17T06:02:15.273 INFO:tasks.rados.rados.0.trial115.stdout:2702: write (excl) oid 49 current snap is 276 2026-02-17T06:02:15.273 INFO:tasks.rados.rados.0.trial115.stdout:2702: seq_num 947 ranges {716229=488575,1616330=541328,2756280=559655} 2026-02-17T06:02:15.278 INFO:tasks.rados.rados.0.trial115.stdout:2702: writing trial11545420-49 from 716229 to 1204804 tid 1 2026-02-17T06:02:15.282 INFO:tasks.rados.rados.0.trial115.stdout:2702: writing trial11545420-49 from 1616330 to 2157658 tid 2 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2702: writing trial11545420-49 from 2756280 to 3315935 tid 3 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2703: read oid 16 snap -1 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2703: expect (ObjNum 903 snap 262 seq_num 903) 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2704: read oid 44 snap 273 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2704: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2705: read oid 29 snap 274 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2705: expect (ObjNum 922 snap 271 seq_num 922) 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2706: rollback oid 32 current snap is 276 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2702: finishing write tid 1 to trial11545420-49 2026-02-17T06:02:15.287 INFO:tasks.rados.rados.0.trial115.stdout:2702: finishing write tid 2 to trial11545420-49 2026-02-17T06:02:15.288 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 273 2026-02-17T06:02:15.288 INFO:tasks.rados.rados.0.trial115.stdout:2707: write (excl) oid 35 current snap is 276 2026-02-17T06:02:15.288 INFO:tasks.rados.rados.0.trial115.stdout:2707: seq_num 948 ranges {733036=563824,2074812=684986,3183703=749333} 2026-02-17T06:02:15.292 INFO:tasks.rados.rados.0.trial115.stdout:2707: writing trial11545420-35 from 733036 to 1296860 tid 1 2026-02-17T06:02:15.297 INFO:tasks.rados.rados.0.trial115.stdout:2707: writing trial11545420-35 from 2074812 to 2759798 tid 2 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2707: writing trial11545420-35 from 3183703 to 3933036 tid 3 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2708: delete oid 46 current snap is 276 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2702: finishing write tid 3 to trial11545420-49 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2702: finishing write tid 4 to trial11545420-49 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2702: finishing write tid 5 to trial11545420-49 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2702: oid 49 updating version 0 to 1294 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2702: oid 49 version 1294 is already newer than 1293 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2702: oid 49 updating version 1294 to 1295 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2702: oid 49 version 1295 is already newer than 1292 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1295 (ObjNum 947 snap 276 seq_num 947) dirty exists 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2702: left oid 49 (ObjNum 947 snap 276 seq_num 947) 2026-02-17T06:02:15.302 INFO:tasks.rados.rados.0.trial115.stdout:2703: expect (ObjNum 903 snap 262 seq_num 903) 2026-02-17T06:02:15.403 INFO:tasks.rados.rados.0.trial115.stdout:2704: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:15.449 INFO:tasks.rados.rados.0.trial115.stdout:2706: finishing rollback tid 0 to trial11545420-32 2026-02-17T06:02:15.449 INFO:tasks.rados.rados.0.trial115.stdout:2706: finishing rollback tid 1 to trial11545420-32 2026-02-17T06:02:15.449 INFO:tasks.rados.rados.0.trial115.stdout:2702: done (6 left) 2026-02-17T06:02:15.449 INFO:tasks.rados.rados.0.trial115.stdout:2703: done (5 left) 2026-02-17T06:02:15.449 INFO:tasks.rados.rados.0.trial115.stdout:2704: done (4 left) 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2708: done (3 left) 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2709: read oid 48 snap -1 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2709: expect (ObjNum 899 snap 262 seq_num 899) 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2710: read oid 6 snap -1 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2710: expect (ObjNum 935 snap 274 seq_num 935) 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2711: read oid 12 snap 274 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2711: expect (ObjNum 910 snap 264 seq_num 910) 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2712: copy_from oid 19 from oid 20 current snap is 276 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2706: finishing rollback tid 2 to trial11545420-32 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1150 (ObjNum 933 snap 273 seq_num 933) dirty exists 2026-02-17T06:02:15.450 INFO:tasks.rados.rados.0.trial115.stdout:2705: expect (ObjNum 922 snap 271 seq_num 922) 2026-02-17T06:02:15.634 INFO:tasks.rados.rados.0.trial115.stdout:2707: finishing write tid 1 to trial11545420-35 2026-02-17T06:02:15.634 INFO:tasks.rados.rados.0.trial115.stdout:2707: finishing write tid 2 to trial11545420-35 2026-02-17T06:02:15.634 INFO:tasks.rados.rados.0.trial115.stdout:2705: done (6 left) 2026-02-17T06:02:15.634 INFO:tasks.rados.rados.0.trial115.stdout:2706: done (5 left) 2026-02-17T06:02:15.635 INFO:tasks.rados.rados.0.trial115.stdout:2713: snap_remove snap 273 2026-02-17T06:02:16.279 INFO:tasks.rados.rados.0.trial115.stdout:2713: done (5 left) 2026-02-17T06:02:16.279 INFO:tasks.rados.rados.0.trial115.stdout:2714: copy_from oid 36 from oid 29 current snap is 276 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2707: finishing write tid 3 to trial11545420-35 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2707: finishing write tid 4 to trial11545420-35 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2707: finishing write tid 5 to trial11545420-35 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2707: oid 35 updating version 0 to 1478 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2707: oid 35 version 1478 is already newer than 1477 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2707: oid 35 version 1478 is already newer than 1476 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2707: oid 35 updating version 1478 to 1479 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1479 (ObjNum 948 snap 276 seq_num 948) dirty exists 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2707: left oid 35 (ObjNum 948 snap 276 seq_num 948) 2026-02-17T06:02:16.280 INFO:tasks.rados.rados.0.trial115.stdout:2711: expect (ObjNum 910 snap 264 seq_num 910) 2026-02-17T06:02:16.399 INFO:tasks.rados.rados.0.trial115.stdout:2709: expect (ObjNum 899 snap 262 seq_num 899) 2026-02-17T06:02:16.582 INFO:tasks.rados.rados.0.trial115.stdout:2712: finishing copy_from to trial11545420-19 2026-02-17T06:02:16.582 INFO:tasks.rados.rados.0.trial115.stdout:2712: got expected ENOENT (src dne) 2026-02-17T06:02:16.582 INFO:tasks.rados.rados.0.trial115.stdout:2712: finishing copy_from racing read to trial11545420-19 2026-02-17T06:02:16.583 INFO:tasks.rados.rados.0.trial115.stdout:2710: expect (ObjNum 935 snap 274 seq_num 935) 2026-02-17T06:02:16.764 INFO:tasks.rados.rados.0.trial115.stdout:2714: finishing copy_from racing read to trial11545420-36 2026-02-17T06:02:16.764 INFO:tasks.rados.rados.0.trial115.stdout:2714: finishing copy_from to trial11545420-36 2026-02-17T06:02:16.764 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 1424 (ObjNum 922 snap 271 seq_num 922) dirty exists 2026-02-17T06:02:16.764 INFO:tasks.rados.rados.0.trial115.stdout:2707: done (5 left) 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2709: done (4 left) 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2710: done (3 left) 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2711: done (2 left) 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2712: done (1 left) 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2714: done (0 left) 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2715: copy_from oid 33 from oid 43 current snap is 276 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2716: read oid 29 snap -1 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2716: expect (ObjNum 922 snap 271 seq_num 922) 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2717: rollback oid 5 current snap is 276 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 5 to 275 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2718: read oid 15 snap -1 2026-02-17T06:02:16.765 INFO:tasks.rados.rados.0.trial115.stdout:2718: expect (ObjNum 932 snap 273 seq_num 932) 2026-02-17T06:02:16.766 INFO:tasks.rados.rados.0.trial115.stdout:2719: delete oid 4 current snap is 276 2026-02-17T06:02:16.766 INFO:tasks.rados.rados.0.trial115.stdout:2719: done (4 left) 2026-02-17T06:02:16.766 INFO:tasks.rados.rados.0.trial115.stdout:2720: read oid 16 snap -1 2026-02-17T06:02:16.767 INFO:tasks.rados.rados.0.trial115.stdout:2720: expect (ObjNum 903 snap 262 seq_num 903) 2026-02-17T06:02:16.767 INFO:tasks.rados.rados.0.trial115.stdout:2721: snap_create 2026-02-17T06:02:16.767 INFO:tasks.rados.rados.0.trial115.stdout:2716: expect (ObjNum 922 snap 271 seq_num 922) 2026-02-17T06:02:16.950 INFO:tasks.rados.rados.0.trial115.stdout:2717: finishing rollback tid 0 to trial11545420-5 2026-02-17T06:02:16.951 INFO:tasks.rados.rados.0.trial115.stdout:2717: finishing rollback tid 1 to trial11545420-5 2026-02-17T06:02:16.951 INFO:tasks.rados.rados.0.trial115.stdout:2717: finishing rollback tid 2 to trial11545420-5 2026-02-17T06:02:16.951 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1421 (ObjNum 926 snap 272 seq_num 926) dirty exists 2026-02-17T06:02:16.951 INFO:tasks.rados.rados.0.trial115.stdout:2718: expect (ObjNum 932 snap 273 seq_num 932) 2026-02-17T06:02:17.068 INFO:tasks.rados.rados.0.trial115.stdout:2720: expect (ObjNum 903 snap 262 seq_num 903) 2026-02-17T06:02:17.168 INFO:tasks.rados.rados.0.trial115.stdout:2715: finishing copy_from racing read to trial11545420-33 2026-02-17T06:02:17.169 INFO:tasks.rados.rados.0.trial115.stdout:2715: finishing copy_from to trial11545420-33 2026-02-17T06:02:17.169 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:02:17.257 INFO:tasks.rados.rados.0.trial115.stdout:2715: done (5 left) 2026-02-17T06:02:17.257 INFO:tasks.rados.rados.0.trial115.stdout:2716: done (4 left) 2026-02-17T06:02:17.258 INFO:tasks.rados.rados.0.trial115.stdout:2717: done (3 left) 2026-02-17T06:02:17.258 INFO:tasks.rados.rados.0.trial115.stdout:2718: done (2 left) 2026-02-17T06:02:17.258 INFO:tasks.rados.rados.0.trial115.stdout:2720: done (1 left) 2026-02-17T06:02:17.258 INFO:tasks.rados.rados.0.trial115.stdout:2721: done (0 left) 2026-02-17T06:02:17.258 INFO:tasks.rados.rados.0.trial115.stdout:2722: read oid 50 snap 275 2026-02-17T06:02:17.258 INFO:tasks.rados.rados.0.trial115.stdout:2722: expect (ObjNum 751 snap 206 seq_num 751) 2026-02-17T06:02:17.258 INFO:tasks.rados.rados.0.trial115.stdout:2723: delete oid 37 current snap is 277 2026-02-17T06:02:17.260 INFO:tasks.rados.rados.0.trial115.stdout:2723: done (1 left) 2026-02-17T06:02:17.260 INFO:tasks.rados.rados.0.trial115.stdout:2724: snap_remove snap 274 2026-02-17T06:02:18.259 INFO:tasks.rados.rados.0.trial115.stdout:2724: done (1 left) 2026-02-17T06:02:18.259 INFO:tasks.rados.rados.0.trial115.stdout:2725: read oid 10 snap 275 2026-02-17T06:02:18.259 INFO:tasks.rados.rados.0.trial115.stdout:2725: expect (ObjNum 915 snap 265 seq_num 915) 2026-02-17T06:02:18.259 INFO:tasks.rados.rados.0.trial115.stdout:2726: snap_create 2026-02-17T06:02:18.260 INFO:tasks.rados.rados.0.trial115.stdout:2722: expect (ObjNum 751 snap 206 seq_num 751) 2026-02-17T06:02:18.318 INFO:tasks.rados.rados.0.trial115.stdout:2725: expect (ObjNum 915 snap 265 seq_num 915) 2026-02-17T06:02:18.650 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:02:18.650 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T06:02:18.650 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T06:02:18.650 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T06:02:18.653 INFO:tasks.ceph.mon.b:Started 2026-02-17T06:02:18.653 INFO:tasks.mon_thrash:Sending STOP to mon a 2026-02-17T06:02:18.653 INFO:tasks.ceph.mon.a:Sent signal 19 2026-02-17T06:02:18.653 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T06:02:24.347 INFO:tasks.rados.rados.0.trial115.stdout:2722: done (2 left) 2026-02-17T06:02:24.347 INFO:tasks.rados.rados.0.trial115.stdout:2725: done (1 left) 2026-02-17T06:02:24.347 INFO:tasks.rados.rados.0.trial115.stdout:2726: done (0 left) 2026-02-17T06:02:24.347 INFO:tasks.rados.rados.0.trial115.stdout:2727: write (excl) oid 4 current snap is 278 2026-02-17T06:02:24.347 INFO:tasks.rados.rados.0.trial115.stdout:2727: seq_num 952 ranges {400264=581173,1508740=491524} 2026-02-17T06:02:24.351 INFO:tasks.rados.rados.0.trial115.stdout:2727: writing trial11545420-4 from 400264 to 981437 tid 1 2026-02-17T06:02:24.354 INFO:tasks.rados.rados.0.trial115.stdout:2727: writing trial11545420-4 from 1508740 to 2000264 tid 2 2026-02-17T06:02:24.355 INFO:tasks.rados.rados.0.trial115.stdout:2728: write (excl) oid 48 current snap is 278 2026-02-17T06:02:24.355 INFO:tasks.rados.rados.0.trial115.stdout:2728: seq_num 953 ranges {417071=656422,1567222=449849} 2026-02-17T06:02:24.358 INFO:tasks.rados.rados.0.trial115.stdout:2728: writing trial11545420-48 from 417071 to 1073493 tid 1 2026-02-17T06:02:24.362 INFO:tasks.rados.rados.0.trial115.stdout:2728: writing trial11545420-48 from 1567222 to 2017071 tid 2 2026-02-17T06:02:24.362 INFO:tasks.rados.rados.0.trial115.stdout:2729: delete oid 26 current snap is 278 2026-02-17T06:02:24.362 INFO:tasks.rados.rados.0.trial115.stdout:2727: finishing write tid 1 to trial11545420-4 2026-02-17T06:02:24.362 INFO:tasks.rados.rados.0.trial115.stdout:2727: finishing write tid 2 to trial11545420-4 2026-02-17T06:02:24.362 INFO:tasks.rados.rados.0.trial115.stdout:2727: finishing write tid 3 to trial11545420-4 2026-02-17T06:02:24.362 INFO:tasks.rados.rados.0.trial115.stdout:2727: finishing write tid 4 to trial11545420-4 2026-02-17T06:02:24.362 INFO:tasks.rados.rados.0.trial115.stdout:2727: oid 4 updating version 0 to 1297 2026-02-17T06:02:24.362 INFO:tasks.rados.rados.0.trial115.stdout:2727: oid 4 updating version 1297 to 1298 2026-02-17T06:02:24.363 INFO:tasks.rados.rados.0.trial115.stdout:2727: oid 4 updating version 1298 to 1299 2026-02-17T06:02:24.363 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1299 (ObjNum 952 snap 278 seq_num 952) dirty exists 2026-02-17T06:02:24.363 INFO:tasks.rados.rados.0.trial115.stdout:2727: left oid 4 (ObjNum 952 snap 278 seq_num 952) 2026-02-17T06:02:24.363 INFO:tasks.rados.rados.0.trial115.stdout:2728: finishing write tid 1 to trial11545420-48 2026-02-17T06:02:24.364 INFO:tasks.rados.rados.0.trial115.stdout:2728: finishing write tid 2 to trial11545420-48 2026-02-17T06:02:24.364 INFO:tasks.rados.rados.0.trial115.stdout:2728: finishing write tid 3 to trial11545420-48 2026-02-17T06:02:24.364 INFO:tasks.rados.rados.0.trial115.stdout:2728: finishing write tid 4 to trial11545420-48 2026-02-17T06:02:24.364 INFO:tasks.rados.rados.0.trial115.stdout:2728: oid 48 updating version 0 to 1489 2026-02-17T06:02:24.364 INFO:tasks.rados.rados.0.trial115.stdout:2728: oid 48 version 1489 is already newer than 1487 2026-02-17T06:02:24.364 INFO:tasks.rados.rados.0.trial115.stdout:2728: oid 48 version 1489 is already newer than 1488 2026-02-17T06:02:24.364 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1489 (ObjNum 953 snap 278 seq_num 953) dirty exists 2026-02-17T06:02:24.364 INFO:tasks.rados.rados.0.trial115.stdout:2728: left oid 48 (ObjNum 953 snap 278 seq_num 953) 2026-02-17T06:02:25.178 INFO:tasks.rados.rados.0.trial115.stdout:2727: done (2 left) 2026-02-17T06:02:25.178 INFO:tasks.rados.rados.0.trial115.stdout:2728: done (1 left) 2026-02-17T06:02:25.178 INFO:tasks.rados.rados.0.trial115.stdout:2729: done (0 left) 2026-02-17T06:02:25.178 INFO:tasks.rados.rados.0.trial115.stdout:2730: delete oid 41 current snap is 278 2026-02-17T06:02:25.179 INFO:tasks.rados.rados.0.trial115.stdout:2730: done (0 left) 2026-02-17T06:02:25.179 INFO:tasks.rados.rados.0.trial115.stdout:2731: delete oid 49 current snap is 278 2026-02-17T06:02:25.180 INFO:tasks.rados.rados.0.trial115.stdout:2731: done (0 left) 2026-02-17T06:02:25.180 INFO:tasks.rados.rados.0.trial115.stdout:2732: snap_remove snap 275 2026-02-17T06:02:26.016 INFO:tasks.rados.rados.0.trial115.stdout:2732: done (0 left) 2026-02-17T06:02:26.016 INFO:tasks.rados.rados.0.trial115.stdout:2733: read oid 40 snap -1 2026-02-17T06:02:26.016 INFO:tasks.rados.rados.0.trial115.stdout:2733: expect deleted 2026-02-17T06:02:26.016 INFO:tasks.rados.rados.0.trial115.stdout:2734: write (excl) oid 19 current snap is 278 2026-02-17T06:02:26.017 INFO:tasks.rados.rados.0.trial115.stdout:2734: seq_num 957 ranges {484299=557418,1684797=399502} 2026-02-17T06:02:26.019 INFO:tasks.rados.rados.0.trial115.stdout:2734: writing trial11545420-19 from 484299 to 1041717 tid 1 2026-02-17T06:02:26.023 INFO:tasks.rados.rados.0.trial115.stdout:2734: writing trial11545420-19 from 1684797 to 2084299 tid 2 2026-02-17T06:02:26.024 INFO:tasks.rados.rados.0.trial115.stdout:2735: snap_remove snap 276 2026-02-17T06:02:27.017 INFO:tasks.rados.rados.0.trial115.stdout:2735: done (2 left) 2026-02-17T06:02:27.017 INFO:tasks.rados.rados.0.trial115.stdout:2736: write oid 23 current snap is 278 2026-02-17T06:02:27.017 INFO:tasks.rados.rados.0.trial115.stdout:2736: seq_num 958 ranges {501106=749020,1743279=357827} 2026-02-17T06:02:27.021 INFO:tasks.rados.rados.0.trial115.stdout:2736: writing trial11545420-23 from 501106 to 1250126 tid 1 2026-02-17T06:02:27.025 INFO:tasks.rados.rados.0.trial115.stdout:2736: writing trial11545420-23 from 1743279 to 2101106 tid 2 2026-02-17T06:02:27.025 INFO:tasks.rados.rados.0.trial115.stdout:2737: write (excl) oid 41 current snap is 278 2026-02-17T06:02:27.025 INFO:tasks.rados.rados.0.trial115.stdout:2737: seq_num 959 ranges {517913=424269,1401761=446989} 2026-02-17T06:02:27.028 INFO:tasks.rados.rados.0.trial115.stdout:2737: writing trial11545420-41 from 517913 to 942182 tid 1 2026-02-17T06:02:27.031 INFO:tasks.rados.rados.0.trial115.stdout:2737: writing trial11545420-41 from 1401761 to 1848750 tid 2 2026-02-17T06:02:27.031 INFO:tasks.rados.rados.0.trial115.stdout:2738: copy_from oid 4 from oid 45 current snap is 278 2026-02-17T06:02:27.031 INFO:tasks.rados.rados.0.trial115.stdout:2734: finishing write tid 1 to trial11545420-19 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2734: finishing write tid 2 to trial11545420-19 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2734: finishing write tid 3 to trial11545420-19 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2734: finishing write tid 4 to trial11545420-19 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2734: oid 19 updating version 0 to 1496 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2734: oid 19 updating version 1496 to 1497 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2734: oid 19 version 1497 is already newer than 1495 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1497 (ObjNum 957 snap 278 seq_num 957) dirty exists 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2734: left oid 19 (ObjNum 957 snap 278 seq_num 957) 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2736: finishing write tid 1 to trial11545420-23 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2733: done (4 left) 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2734: done (3 left) 2026-02-17T06:02:27.032 INFO:tasks.rados.rados.0.trial115.stdout:2739: snap_remove snap 277 2026-02-17T06:02:28.021 INFO:tasks.rados.rados.0.trial115.stdout:2739: done (3 left) 2026-02-17T06:02:28.021 INFO:tasks.rados.rados.0.trial115.stdout:2740: rollback oid 24 current snap is 278 2026-02-17T06:02:28.021 INFO:tasks.rados.rados.0.trial115.stdout:2740: done (3 left) 2026-02-17T06:02:28.022 INFO:tasks.rados.rados.0.trial115.stdout:2741: read oid 16 snap -1 2026-02-17T06:02:28.022 INFO:tasks.rados.rados.0.trial115.stdout:2741: expect (ObjNum 903 snap 262 seq_num 903) 2026-02-17T06:02:28.022 INFO:tasks.rados.rados.0.trial115.stdout:2742: write oid 46 current snap is 278 2026-02-17T06:02:28.022 INFO:tasks.rados.rados.0.trial115.stdout:2736: finishing write tid 2 to trial11545420-23 2026-02-17T06:02:28.022 INFO:tasks.rados.rados.0.trial115.stdout:2742: seq_num 960 ranges {534720=499518,1460243=674477} 2026-02-17T06:02:28.024 INFO:tasks.rados.rados.0.trial115.stdout:2742: writing trial11545420-46 from 534720 to 1034238 tid 1 2026-02-17T06:02:28.029 INFO:tasks.rados.rados.0.trial115.stdout:2742: writing trial11545420-46 from 1460243 to 2134720 tid 2 2026-02-17T06:02:28.029 INFO:tasks.rados.rados.0.trial115.stdout:2743: write oid 25 current snap is 278 2026-02-17T06:02:28.029 INFO:tasks.rados.rados.0.trial115.stdout:2743: seq_num 961 ranges {551527=574767,1918725=232802} 2026-02-17T06:02:28.033 INFO:tasks.rados.rados.0.trial115.stdout:2743: writing trial11545420-25 from 551527 to 1126294 tid 1 2026-02-17T06:02:28.037 INFO:tasks.rados.rados.0.trial115.stdout:2743: writing trial11545420-25 from 1918725 to 2151527 tid 2 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:2744: delete oid 36 current snap is 278 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:2736: finishing write tid 3 to trial11545420-23 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:2736: finishing write tid 4 to trial11545420-23 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:2736: oid 23 updating version 0 to 1147 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:2736: oid 23 updating version 1147 to 1148 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:2736: oid 23 updating version 1148 to 1149 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1149 (ObjNum 958 snap 278 seq_num 958) dirty exists 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:2736: left oid 23 (ObjNum 958 snap 278 seq_num 958) 2026-02-17T06:02:28.038 INFO:tasks.rados.rados.0.trial115.stdout:2737: finishing write tid 1 to trial11545420-41 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2738: finishing copy_from to trial11545420-4 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2738: got expected ENOENT (src dne) 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2738: finishing copy_from racing read to trial11545420-4 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2737: finishing write tid 2 to trial11545420-41 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2737: finishing write tid 3 to trial11545420-41 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2737: finishing write tid 4 to trial11545420-41 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2737: oid 41 updating version 0 to 1399 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2737: oid 41 version 1399 is already newer than 1397 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2737: oid 41 version 1399 is already newer than 1398 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1399 (ObjNum 959 snap 278 seq_num 959) dirty exists 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2737: left oid 41 (ObjNum 959 snap 278 seq_num 959) 2026-02-17T06:02:28.039 INFO:tasks.rados.rados.0.trial115.stdout:2741: expect (ObjNum 903 snap 262 seq_num 903) 2026-02-17T06:02:28.139 INFO:tasks.rados.rados.0.trial115.stdout:2742: finishing write tid 1 to trial11545420-46 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2742: finishing write tid 2 to trial11545420-46 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2742: finishing write tid 3 to trial11545420-46 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2742: finishing write tid 4 to trial11545420-46 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2742: oid 46 updating version 0 to 1380 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2742: oid 46 version 1380 is already newer than 1379 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2742: oid 46 version 1380 is already newer than 1378 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1380 (ObjNum 960 snap 278 seq_num 960) dirty exists 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2742: left oid 46 (ObjNum 960 snap 278 seq_num 960) 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2743: finishing write tid 1 to trial11545420-25 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2743: finishing write tid 2 to trial11545420-25 2026-02-17T06:02:28.140 INFO:tasks.rados.rados.0.trial115.stdout:2736: done (6 left) 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2737: done (5 left) 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2738: done (4 left) 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2741: done (3 left) 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2742: done (2 left) 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2744: done (1 left) 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2745: delete oid 1 current snap is 278 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2743: finishing write tid 3 to trial11545420-25 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2743: finishing write tid 4 to trial11545420-25 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2743: oid 25 updating version 0 to 1402 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2743: oid 25 version 1402 is already newer than 1400 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:2743: oid 25 version 1402 is already newer than 1401 2026-02-17T06:02:28.141 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1402 (ObjNum 961 snap 278 seq_num 961) dirty exists 2026-02-17T06:02:28.142 INFO:tasks.rados.rados.0.trial115.stdout:2743: left oid 25 (ObjNum 961 snap 278 seq_num 961) 2026-02-17T06:02:28.142 INFO:tasks.rados.rados.0.trial115.stdout:2743: done (1 left) 2026-02-17T06:02:28.142 INFO:tasks.rados.rados.0.trial115.stdout:2745: done (0 left) 2026-02-17T06:02:28.142 INFO:tasks.rados.rados.0.trial115.stdout:2746: delete oid 26 current snap is 278 2026-02-17T06:02:28.142 INFO:tasks.rados.rados.0.trial115.stdout:2746: done (0 left) 2026-02-17T06:02:28.142 INFO:tasks.rados.rados.0.trial115.stdout:2747: snap_create 2026-02-17T06:02:28.720 INFO:tasks.rados.rados.0.trial115.stdout:2747: done (0 left) 2026-02-17T06:02:28.721 INFO:tasks.rados.rados.0.trial115.stdout:2748: write (excl) oid 33 current snap is 279 2026-02-17T06:02:28.721 INFO:tasks.rados.rados.0.trial115.stdout:2748: seq_num 965 ranges {618755=475763,1636300=457996} 2026-02-17T06:02:28.724 INFO:tasks.rados.rados.0.trial115.stdout:2748: writing trial11545420-33 from 618755 to 1094518 tid 1 2026-02-17T06:02:28.729 INFO:tasks.rados.rados.0.trial115.stdout:2748: writing trial11545420-33 from 1636300 to 2094296 tid 2 2026-02-17T06:02:28.729 INFO:tasks.rados.rados.0.trial115.stdout:2749: write oid 27 current snap is 279 2026-02-17T06:02:28.729 INFO:tasks.rados.rados.0.trial115.stdout:2749: seq_num 966 ranges {635562=667365,2094782=140780} 2026-02-17T06:02:28.734 INFO:tasks.rados.rados.0.trial115.stdout:2749: writing trial11545420-27 from 635562 to 1302927 tid 1 2026-02-17T06:02:28.737 INFO:tasks.rados.rados.0.trial115.stdout:2749: writing trial11545420-27 from 2094782 to 2235562 tid 2 2026-02-17T06:02:28.737 INFO:tasks.rados.rados.0.trial115.stdout:2750: snap_remove snap 278 2026-02-17T06:02:29.723 INFO:tasks.rados.rados.0.trial115.stdout:2750: done (2 left) 2026-02-17T06:02:29.723 INFO:tasks.rados.rados.0.trial115.stdout:2751: write oid 20 current snap is 279 2026-02-17T06:02:29.724 INFO:tasks.rados.rados.0.trial115.stdout:2751: seq_num 967 ranges {652369=742614,2153264=99105} 2026-02-17T06:02:29.728 INFO:tasks.rados.rados.0.trial115.stdout:2751: writing trial11545420-20 from 652369 to 1394983 tid 1 2026-02-17T06:02:29.732 INFO:tasks.rados.rados.0.trial115.stdout:2751: writing trial11545420-20 from 2153264 to 2252369 tid 2 2026-02-17T06:02:29.732 INFO:tasks.rados.rados.0.trial115.stdout:2752: snap_create 2026-02-17T06:02:29.732 INFO:tasks.rados.rados.0.trial115.stdout:2748: finishing write tid 1 to trial11545420-33 2026-02-17T06:02:29.732 INFO:tasks.rados.rados.0.trial115.stdout:2748: finishing write tid 2 to trial11545420-33 2026-02-17T06:02:29.732 INFO:tasks.rados.rados.0.trial115.stdout:2748: finishing write tid 3 to trial11545420-33 2026-02-17T06:02:29.732 INFO:tasks.rados.rados.0.trial115.stdout:2748: finishing write tid 4 to trial11545420-33 2026-02-17T06:02:29.732 INFO:tasks.rados.rados.0.trial115.stdout:2748: oid 33 updating version 0 to 1428 2026-02-17T06:02:29.732 INFO:tasks.rados.rados.0.trial115.stdout:2748: oid 33 version 1428 is already newer than 1427 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2748: oid 33 version 1428 is already newer than 1426 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1428 (ObjNum 965 snap 279 seq_num 965) dirty exists 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2748: left oid 33 (ObjNum 965 snap 279 seq_num 965) 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2749: finishing write tid 1 to trial11545420-27 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2749: finishing write tid 2 to trial11545420-27 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2749: finishing write tid 3 to trial11545420-27 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2749: finishing write tid 4 to trial11545420-27 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2749: oid 27 updating version 0 to 1180 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2749: oid 27 version 1180 is already newer than 1179 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2749: oid 27 version 1180 is already newer than 1178 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1180 (ObjNum 966 snap 279 seq_num 966) dirty exists 2026-02-17T06:02:29.733 INFO:tasks.rados.rados.0.trial115.stdout:2749: left oid 27 (ObjNum 966 snap 279 seq_num 966) 2026-02-17T06:02:29.734 INFO:tasks.rados.rados.0.trial115.stdout:2751: finishing write tid 1 to trial11545420-20 2026-02-17T06:02:29.735 INFO:tasks.rados.rados.0.trial115.stdout:2751: finishing write tid 2 to trial11545420-20 2026-02-17T06:02:29.735 INFO:tasks.rados.rados.0.trial115.stdout:2751: finishing write tid 3 to trial11545420-20 2026-02-17T06:02:29.735 INFO:tasks.rados.rados.0.trial115.stdout:2751: finishing write tid 4 to trial11545420-20 2026-02-17T06:02:29.735 INFO:tasks.rados.rados.0.trial115.stdout:2751: oid 20 updating version 0 to 1426 2026-02-17T06:02:29.735 INFO:tasks.rados.rados.0.trial115.stdout:2751: oid 20 updating version 1426 to 1427 2026-02-17T06:02:29.735 INFO:tasks.rados.rados.0.trial115.stdout:2751: oid 20 version 1427 is already newer than 1425 2026-02-17T06:02:29.735 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1427 (ObjNum 967 snap 279 seq_num 967) dirty exists 2026-02-17T06:02:29.735 INFO:tasks.rados.rados.0.trial115.stdout:2751: left oid 20 (ObjNum 967 snap 279 seq_num 967) 2026-02-17T06:02:30.725 INFO:tasks.rados.rados.0.trial115.stdout:2748: done (3 left) 2026-02-17T06:02:30.725 INFO:tasks.rados.rados.0.trial115.stdout:2749: done (2 left) 2026-02-17T06:02:30.725 INFO:tasks.rados.rados.0.trial115.stdout:2751: done (1 left) 2026-02-17T06:02:30.725 INFO:tasks.rados.rados.0.trial115.stdout:2752: done (0 left) 2026-02-17T06:02:30.725 INFO:tasks.rados.rados.0.trial115.stdout:2753: copy_from oid 40 from oid 18 current snap is 280 2026-02-17T06:02:30.725 INFO:tasks.rados.rados.0.trial115.stdout:2754: delete oid 19 current snap is 280 2026-02-17T06:02:30.727 INFO:tasks.rados.rados.0.trial115.stdout:2754: done (1 left) 2026-02-17T06:02:30.727 INFO:tasks.rados.rados.0.trial115.stdout:2755: read oid 17 snap -1 2026-02-17T06:02:30.727 INFO:tasks.rados.rados.0.trial115.stdout:2755: expect (ObjNum 943 snap 274 seq_num 943) 2026-02-17T06:02:30.727 INFO:tasks.rados.rados.0.trial115.stdout:2756: snap_remove snap 279 2026-02-17T06:02:31.734 INFO:tasks.rados.rados.0.trial115.stdout:2756: done (2 left) 2026-02-17T06:02:31.734 INFO:tasks.rados.rados.0.trial115.stdout:2757: write oid 21 current snap is 280 2026-02-17T06:02:31.734 INFO:tasks.rados.rados.0.trial115.stdout:2757: seq_num 969 ranges {685983=493112,1870228=415755} 2026-02-17T06:02:31.738 INFO:tasks.rados.rados.0.trial115.stdout:2757: writing trial11545420-21 from 685983 to 1179095 tid 1 2026-02-17T06:02:31.743 INFO:tasks.rados.rados.0.trial115.stdout:2757: writing trial11545420-21 from 1870228 to 2285983 tid 2 2026-02-17T06:02:31.743 INFO:tasks.rados.rados.0.trial115.stdout:2758: write oid 48 current snap is 280 2026-02-17T06:02:31.743 INFO:tasks.rados.rados.0.trial115.stdout:2758: seq_num 970 ranges {702790=568361,1812357=490433} 2026-02-17T06:02:31.748 INFO:tasks.rados.rados.0.trial115.stdout:2758: writing trial11545420-48 from 702790 to 1271151 tid 1 2026-02-17T06:02:31.752 INFO:tasks.rados.rados.0.trial115.stdout:2758: writing trial11545420-48 from 1812357 to 2302790 tid 2 2026-02-17T06:02:31.752 INFO:tasks.rados.rados.0.trial115.stdout:2759: delete oid 20 current snap is 280 2026-02-17T06:02:31.752 INFO:tasks.rados.rados.0.trial115.stdout:2755: expect (ObjNum 943 snap 274 seq_num 943) 2026-02-17T06:02:31.866 INFO:tasks.rados.rados.0.trial115.stdout:2753: finishing copy_from racing read to trial11545420-40 2026-02-17T06:02:31.866 INFO:tasks.rados.rados.0.trial115.stdout:2753: finishing copy_from to trial11545420-40 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1395 (ObjNum 818 snap 235 seq_num 818) dirty exists 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:2757: finishing write tid 1 to trial11545420-21 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:2757: finishing write tid 2 to trial11545420-21 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:2757: finishing write tid 3 to trial11545420-21 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:2757: finishing write tid 4 to trial11545420-21 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:2757: oid 21 updating version 0 to 1404 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:2757: oid 21 updating version 1404 to 1405 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:2757: oid 21 version 1405 is already newer than 1403 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1405 (ObjNum 969 snap 280 seq_num 969) dirty exists 2026-02-17T06:02:31.867 INFO:tasks.rados.rados.0.trial115.stdout:2757: left oid 21 (ObjNum 969 snap 280 seq_num 969) 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2758: finishing write tid 1 to trial11545420-48 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2758: finishing write tid 2 to trial11545420-48 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2758: finishing write tid 3 to trial11545420-48 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2758: finishing write tid 4 to trial11545420-48 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2758: oid 48 updating version 0 to 1512 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2758: oid 48 version 1512 is already newer than 1510 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2758: oid 48 version 1512 is already newer than 1511 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1512 (ObjNum 970 snap 280 seq_num 970) dirty exists 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2758: left oid 48 (ObjNum 970 snap 280 seq_num 970) 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2753: done (4 left) 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2755: done (3 left) 2026-02-17T06:02:31.868 INFO:tasks.rados.rados.0.trial115.stdout:2757: done (2 left) 2026-02-17T06:02:31.869 INFO:tasks.rados.rados.0.trial115.stdout:2758: done (1 left) 2026-02-17T06:02:31.869 INFO:tasks.rados.rados.0.trial115.stdout:2759: done (0 left) 2026-02-17T06:02:31.869 INFO:tasks.rados.rados.0.trial115.stdout:2760: read oid 8 snap -1 2026-02-17T06:02:31.869 INFO:tasks.rados.rados.0.trial115.stdout:2760: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:31.869 INFO:tasks.rados.rados.0.trial115.stdout:2761: write oid 25 current snap is 280 2026-02-17T06:02:31.869 INFO:tasks.rados.rados.0.trial115.stdout:2761: seq_num 972 ranges {736404=718859,1929321=407083} 2026-02-17T06:02:31.871 INFO:tasks.rados.rados.0.trial115.stdout:2761: writing trial11545420-25 from 736404 to 1455263 tid 1 2026-02-17T06:02:31.875 INFO:tasks.rados.rados.0.trial115.stdout:2761: writing trial11545420-25 from 1929321 to 2336404 tid 2 2026-02-17T06:02:31.875 INFO:tasks.rados.rados.0.trial115.stdout:2762: write oid 27 current snap is 280 2026-02-17T06:02:31.875 INFO:tasks.rados.rados.0.trial115.stdout:2762: seq_num 973 ranges {753211=794108,1987803=365408} 2026-02-17T06:02:31.880 INFO:tasks.rados.rados.0.trial115.stdout:2762: writing trial11545420-27 from 753211 to 1547319 tid 1 2026-02-17T06:02:31.884 INFO:tasks.rados.rados.0.trial115.stdout:2762: writing trial11545420-27 from 1987803 to 2353211 tid 2 2026-02-17T06:02:31.884 INFO:tasks.rados.rados.0.trial115.stdout:2763: write oid 13 current snap is 280 2026-02-17T06:02:31.884 INFO:tasks.rados.rados.0.trial115.stdout:2763: seq_num 974 ranges {770018=585710,2046285=323733} 2026-02-17T06:02:31.889 INFO:tasks.rados.rados.0.trial115.stdout:2763: writing trial11545420-13 from 770018 to 1355728 tid 1 2026-02-17T06:02:31.893 INFO:tasks.rados.rados.0.trial115.stdout:2763: writing trial11545420-13 from 2046285 to 2370018 tid 2 2026-02-17T06:02:31.894 INFO:tasks.rados.rados.0.trial115.stdout:2764: read oid 23 snap -1 2026-02-17T06:02:31.894 INFO:tasks.rados.rados.0.trial115.stdout:2764: expect (ObjNum 958 snap 278 seq_num 958) 2026-02-17T06:02:31.894 INFO:tasks.rados.rados.0.trial115.stdout:2765: delete oid 1 current snap is 280 2026-02-17T06:02:31.894 INFO:tasks.rados.rados.0.trial115.stdout:2760: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:31.941 INFO:tasks.rados.rados.0.trial115.stdout:2761: finishing write tid 1 to trial11545420-25 2026-02-17T06:02:31.941 INFO:tasks.rados.rados.0.trial115.stdout:2760: done (5 left) 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2765: done (4 left) 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2766: rollback oid 38 current snap is 280 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2766: done (4 left) 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2767: read oid 36 snap -1 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2767: expect deleted 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2768: copy_from oid 29 from oid 38 current snap is 280 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2761: finishing write tid 2 to trial11545420-25 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2761: finishing write tid 3 to trial11545420-25 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2761: finishing write tid 4 to trial11545420-25 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2761: oid 25 updating version 0 to 1407 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2761: oid 25 updating version 1407 to 1408 2026-02-17T06:02:31.942 INFO:tasks.rados.rados.0.trial115.stdout:2761: oid 25 version 1408 is already newer than 1406 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1408 (ObjNum 972 snap 280 seq_num 972) dirty exists 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2761: left oid 25 (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2762: finishing write tid 1 to trial11545420-27 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2762: finishing write tid 2 to trial11545420-27 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2762: finishing write tid 3 to trial11545420-27 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2762: finishing write tid 4 to trial11545420-27 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2762: oid 27 updating version 0 to 1181 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2762: oid 27 updating version 1181 to 1182 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2762: oid 27 updating version 1182 to 1183 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1183 (ObjNum 973 snap 280 seq_num 973) dirty exists 2026-02-17T06:02:31.943 INFO:tasks.rados.rados.0.trial115.stdout:2762: left oid 27 (ObjNum 973 snap 280 seq_num 973) 2026-02-17T06:02:31.944 INFO:tasks.rados.rados.0.trial115.stdout:2764: expect (ObjNum 958 snap 278 seq_num 958) 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2761: done (5 left) 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2762: done (4 left) 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2764: done (3 left) 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2767: done (2 left) 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2769: read oid 23 snap -1 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2769: expect (ObjNum 958 snap 278 seq_num 958) 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2770: read oid 5 snap -1 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2770: expect (ObjNum 926 snap 272 seq_num 926) 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2771: copy_from oid 10 from oid 3 current snap is 280 2026-02-17T06:02:32.006 INFO:tasks.rados.rados.0.trial115.stdout:2772: write (excl) oid 46 current snap is 280 2026-02-17T06:02:32.007 INFO:tasks.rados.rados.0.trial115.stdout:2772: seq_num 976 ranges {403632=736208,1763249=619999} 2026-02-17T06:02:32.011 INFO:tasks.rados.rados.0.trial115.stdout:2772: writing trial11545420-46 from 403632 to 1139840 tid 1 2026-02-17T06:02:32.016 INFO:tasks.rados.rados.0.trial115.stdout:2772: writing trial11545420-46 from 1763249 to 2383248 tid 2 2026-02-17T06:02:32.016 INFO:tasks.rados.rados.0.trial115.stdout:2773: rollback oid 33 current snap is 280 2026-02-17T06:02:32.016 INFO:tasks.rados.rados.0.trial115.stdout:2773: done (6 left) 2026-02-17T06:02:32.016 INFO:tasks.rados.rados.0.trial115.stdout:2774: delete oid 15 current snap is 280 2026-02-17T06:02:32.016 INFO:tasks.rados.rados.0.trial115.stdout:2769: expect (ObjNum 958 snap 278 seq_num 958) 2026-02-17T06:02:32.082 INFO:tasks.rados.rados.0.trial115.stdout:2770: expect (ObjNum 926 snap 272 seq_num 926) 2026-02-17T06:02:32.185 INFO:tasks.rados.rados.0.trial115.stdout:2772: finishing write tid 1 to trial11545420-46 2026-02-17T06:02:32.185 INFO:tasks.rados.rados.0.trial115.stdout:2769: done (6 left) 2026-02-17T06:02:32.185 INFO:tasks.rados.rados.0.trial115.stdout:2770: done (5 left) 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2774: done (4 left) 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2775: copy_from oid 23 from oid 47 current snap is 280 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2771: finishing copy_from racing read to trial11545420-10 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2771: finishing copy_from to trial11545420-10 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1415 (ObjNum 939 snap 274 seq_num 939) dirty exists 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2772: finishing write tid 2 to trial11545420-46 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2772: finishing write tid 3 to trial11545420-46 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2772: finishing write tid 4 to trial11545420-46 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2772: oid 46 updating version 0 to 1381 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2772: oid 46 updating version 1381 to 1417 2026-02-17T06:02:32.186 INFO:tasks.rados.rados.0.trial115.stdout:2772: oid 46 version 1417 is already newer than 1416 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1417 (ObjNum 976 snap 280 seq_num 976) dirty exists 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2772: left oid 46 (ObjNum 976 snap 280 seq_num 976) 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2771: done (4 left) 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2772: done (3 left) 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2776: read oid 21 snap -1 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2776: expect (ObjNum 969 snap 280 seq_num 969) 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2777: read oid 45 snap -1 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2777: expect deleted 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2778: write oid 2 current snap is 280 2026-02-17T06:02:32.187 INFO:tasks.rados.rados.0.trial115.stdout:2778: seq_num 978 ranges {437246=486706,1363860=623668} 2026-02-17T06:02:32.188 INFO:tasks.rados.rados.0.trial115.stdout:2778: writing trial11545420-2 from 437246 to 923952 tid 1 2026-02-17T06:02:32.192 INFO:tasks.rados.rados.0.trial115.stdout:2778: writing trial11545420-2 from 1363860 to 1987528 tid 2 2026-02-17T06:02:32.192 INFO:tasks.rados.rados.0.trial115.stdout:2779: snap_create 2026-02-17T06:02:32.192 INFO:tasks.rados.rados.0.trial115.stdout:2776: expect (ObjNum 969 snap 280 seq_num 969) 2026-02-17T06:02:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2778: finishing write tid 1 to trial11545420-2 2026-02-17T06:02:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2775: finishing copy_from racing read to trial11545420-23 2026-02-17T06:02:32.296 INFO:tasks.rados.rados.0.trial115.stdout:2775: finishing copy_from to trial11545420-23 2026-02-17T06:02:32.296 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1398 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:02:32.297 INFO:tasks.rados.rados.0.trial115.stdout:2778: finishing write tid 2 to trial11545420-2 2026-02-17T06:02:32.297 INFO:tasks.rados.rados.0.trial115.stdout:2778: finishing write tid 3 to trial11545420-2 2026-02-17T06:02:32.297 INFO:tasks.rados.rados.0.trial115.stdout:2778: finishing write tid 4 to trial11545420-2 2026-02-17T06:02:32.297 INFO:tasks.rados.rados.0.trial115.stdout:2778: oid 2 updating version 0 to 1423 2026-02-17T06:02:32.297 INFO:tasks.rados.rados.0.trial115.stdout:2778: oid 2 updating version 1423 to 1424 2026-02-17T06:02:32.297 INFO:tasks.rados.rados.0.trial115.stdout:2778: oid 2 version 1424 is already newer than 1422 2026-02-17T06:02:32.297 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1424 (ObjNum 978 snap 280 seq_num 978) dirty exists 2026-02-17T06:02:32.297 INFO:tasks.rados.rados.0.trial115.stdout:2778: left oid 2 (ObjNum 978 snap 280 seq_num 978) 2026-02-17T06:02:32.436 INFO:tasks.rados.rados.0.trial115.stdout:2763: finishing write tid 1 to trial11545420-13 2026-02-17T06:02:32.436 INFO:tasks.rados.rados.0.trial115.stdout:2763: finishing write tid 2 to trial11545420-13 2026-02-17T06:02:32.437 INFO:tasks.rados.rados.0.trial115.stdout:2763: finishing write tid 3 to trial11545420-13 2026-02-17T06:02:32.437 INFO:tasks.rados.rados.0.trial115.stdout:2763: finishing write tid 4 to trial11545420-13 2026-02-17T06:02:32.437 INFO:tasks.rados.rados.0.trial115.stdout:2763: oid 13 updating version 0 to 1425 2026-02-17T06:02:32.437 INFO:tasks.rados.rados.0.trial115.stdout:2763: oid 13 updating version 1425 to 1427 2026-02-17T06:02:32.437 INFO:tasks.rados.rados.0.trial115.stdout:2763: oid 13 version 1427 is already newer than 1426 2026-02-17T06:02:32.437 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1427 (ObjNum 974 snap 280 seq_num 974) dirty exists 2026-02-17T06:02:32.437 INFO:tasks.rados.rados.0.trial115.stdout:2763: left oid 13 (ObjNum 974 snap 280 seq_num 974) 2026-02-17T06:02:32.440 INFO:tasks.rados.rados.0.trial115.stdout:2768: finishing copy_from racing read to trial11545420-29 2026-02-17T06:02:32.442 INFO:tasks.rados.rados.0.trial115.stdout:2768: finishing copy_from to trial11545420-29 2026-02-17T06:02:32.442 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1392 (ObjNum 917 snap 265 seq_num 917) dirty exists 2026-02-17T06:02:33.132 INFO:tasks.rados.rados.0.trial115.stdout:2763: done (6 left) 2026-02-17T06:02:33.132 INFO:tasks.rados.rados.0.trial115.stdout:2768: done (5 left) 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:2775: done (4 left) 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:2776: done (3 left) 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:2777: done (2 left) 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:2778: done (1 left) 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:2779: done (0 left) 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:2780: rollback oid 40 current snap is 281 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 280 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:2781: read oid 35 snap -1 2026-02-17T06:02:33.133 INFO:tasks.rados.rados.0.trial115.stdout:2781: expect (ObjNum 948 snap 276 seq_num 948) 2026-02-17T06:02:33.134 INFO:tasks.rados.rados.0.trial115.stdout:2782: read oid 36 snap -1 2026-02-17T06:02:33.134 INFO:tasks.rados.rados.0.trial115.stdout:2782: expect deleted 2026-02-17T06:02:33.134 INFO:tasks.rados.rados.0.trial115.stdout:2783: write oid 1 current snap is 281 2026-02-17T06:02:33.134 INFO:tasks.rados.rados.0.trial115.stdout:2783: seq_num 979 ranges {454053=561955,1422342=483679,2368404=85649} 2026-02-17T06:02:33.135 INFO:tasks.rados.rados.0.trial115.stdout:2783: writing trial11545420-1 from 454053 to 1016008 tid 1 2026-02-17T06:02:33.139 INFO:tasks.rados.rados.0.trial115.stdout:2783: writing trial11545420-1 from 1422342 to 1906021 tid 2 2026-02-17T06:02:33.141 INFO:tasks.rados.rados.0.trial115.stdout:2783: writing trial11545420-1 from 2368404 to 2454053 tid 3 2026-02-17T06:02:33.141 INFO:tasks.rados.rados.0.trial115.stdout:2784: rollback oid 47 current snap is 281 2026-02-17T06:02:33.141 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 47 to 280 2026-02-17T06:02:33.141 INFO:tasks.rados.rados.0.trial115.stdout:2780: finishing rollback tid 0 to trial11545420-40 2026-02-17T06:02:33.142 INFO:tasks.rados.rados.0.trial115.stdout:2780: finishing rollback tid 1 to trial11545420-40 2026-02-17T06:02:33.142 INFO:tasks.rados.rados.0.trial115.stdout:2780: finishing rollback tid 2 to trial11545420-40 2026-02-17T06:02:33.142 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1398 (ObjNum 818 snap 235 seq_num 818) dirty exists 2026-02-17T06:02:33.142 INFO:tasks.rados.rados.0.trial115.stdout:2780: done (4 left) 2026-02-17T06:02:33.142 INFO:tasks.rados.rados.0.trial115.stdout:2782: done (3 left) 2026-02-17T06:02:33.142 INFO:tasks.rados.rados.0.trial115.stdout:2785: write oid 4 current snap is 281 2026-02-17T06:02:33.142 INFO:tasks.rados.rados.0.trial115.stdout:2785: seq_num 980 ranges {470860=637204,1880824=590036} 2026-02-17T06:02:33.145 INFO:tasks.rados.rados.0.trial115.stdout:2785: writing trial11545420-4 from 470860 to 1108064 tid 1 2026-02-17T06:02:33.150 INFO:tasks.rados.rados.0.trial115.stdout:2785: writing trial11545420-4 from 1880824 to 2470860 tid 2 2026-02-17T06:02:33.150 INFO:tasks.rados.rados.0.trial115.stdout:2786: write oid 3 current snap is 281 2026-02-17T06:02:33.150 INFO:tasks.rados.rados.0.trial115.stdout:2786: seq_num 981 ranges {487667=428806,1655659=770995} 2026-02-17T06:02:33.153 INFO:tasks.rados.rados.0.trial115.stdout:2786: writing trial11545420-3 from 487667 to 916473 tid 1 2026-02-17T06:02:33.159 INFO:tasks.rados.rados.0.trial115.stdout:2786: writing trial11545420-3 from 1655659 to 2426654 tid 2 2026-02-17T06:02:33.159 INFO:tasks.rados.rados.0.trial115.stdout:2787: rollback oid 45 current snap is 281 2026-02-17T06:02:33.159 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 280 2026-02-17T06:02:33.159 INFO:tasks.rados.rados.0.trial115.stdout:2781: expect (ObjNum 948 snap 276 seq_num 948) 2026-02-17T06:02:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2783: finishing write tid 1 to trial11545420-1 2026-02-17T06:02:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2783: finishing write tid 2 to trial11545420-1 2026-02-17T06:02:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2783: finishing write tid 3 to trial11545420-1 2026-02-17T06:02:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2783: finishing write tid 4 to trial11545420-1 2026-02-17T06:02:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2783: finishing write tid 5 to trial11545420-1 2026-02-17T06:02:33.326 INFO:tasks.rados.rados.0.trial115.stdout:2783: oid 1 updating version 0 to 1430 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2783: oid 1 version 1430 is already newer than 1429 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2783: oid 1 updating version 1430 to 1431 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2783: oid 1 updating version 1431 to 1432 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 1432 (ObjNum 979 snap 281 seq_num 979) dirty exists 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2783: left oid 1 (ObjNum 979 snap 281 seq_num 979) 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2784: finishing rollback tid 0 to trial11545420-47 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2784: finishing rollback tid 1 to trial11545420-47 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2781: done (5 left) 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2783: done (4 left) 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2788: snap_create 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:2784: finishing rollback tid 2 to trial11545420-47 2026-02-17T06:02:33.327 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1427 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2785: finishing write tid 1 to trial11545420-4 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2785: finishing write tid 2 to trial11545420-4 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2785: finishing write tid 3 to trial11545420-4 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2785: finishing write tid 4 to trial11545420-4 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2785: oid 4 updating version 0 to 1399 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2785: oid 4 updating version 1399 to 1400 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2785: oid 4 updating version 1400 to 1401 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1401 (ObjNum 980 snap 281 seq_num 980) dirty exists 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2785: left oid 4 (ObjNum 980 snap 281 seq_num 980) 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2786: finishing write tid 1 to trial11545420-3 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2787: finishing rollback tid 1 to trial11545420-45 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 0 (ObjNum 145 snap 0 seq_num 1715449216) dirty dne 2026-02-17T06:02:33.328 INFO:tasks.rados.rados.0.trial115.stdout:2786: finishing write tid 2 to trial11545420-3 2026-02-17T06:02:33.329 INFO:tasks.rados.rados.0.trial115.stdout:2786: finishing write tid 3 to trial11545420-3 2026-02-17T06:02:33.329 INFO:tasks.rados.rados.0.trial115.stdout:2786: finishing write tid 4 to trial11545420-3 2026-02-17T06:02:33.329 INFO:tasks.rados.rados.0.trial115.stdout:2786: oid 3 updating version 0 to 1430 2026-02-17T06:02:33.329 INFO:tasks.rados.rados.0.trial115.stdout:2786: oid 3 version 1430 is already newer than 1428 2026-02-17T06:02:33.329 INFO:tasks.rados.rados.0.trial115.stdout:2786: oid 3 version 1430 is already newer than 1429 2026-02-17T06:02:33.329 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1430 (ObjNum 981 snap 281 seq_num 981) dirty exists 2026-02-17T06:02:33.329 INFO:tasks.rados.rados.0.trial115.stdout:2786: left oid 3 (ObjNum 981 snap 281 seq_num 981) 2026-02-17T06:02:33.653 INFO:tasks.mon_thrash:Sending CONT to mon a 2026-02-17T06:02:33.654 INFO:tasks.ceph.mon.a:Sent signal 18 2026-02-17T06:02:33.654 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:02:33.654 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:02:33.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.721+0000 7f9c33e3b640 10 monclient: get_monmap_and_config 2026-02-17T06:02:33.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.721+0000 7f9c33e3b640 10 monclient: build_initial_monmap 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.721+0000 7f9c33e3b640 10 monclient: monmap: 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:33.722847+0000 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:33.722847+0000 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.721+0000 7f9c33e3b640 1 Processor -- start 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 1 -- start start 2026-02-17T06:02:33.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient: init 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient(hunting): picked mon.noname-c con 0x7f9c2c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9c2c09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient(hunting): picked mon.noname-b con 0x7f9c2c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c2c06daf0 con 0x7f9c2c09c4a0 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f9c2c0a0880 con 0x7f9c2c06c890 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2c09cc80 con 0x7f9c2c06d1b0 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c33e3b640 10 monclient(hunting): authenticate will time out at 1599.276372s 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c313af640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f9c2c09c4a0 0x7f9c2c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:45658/0 (socket says 10.20.193.78:45658) 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 learned_addr learned my addr 10.20.193.78:0/3807453840 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 10 monclient(hunting): get_auth_request con 0x7f9c2c06d1b0 auth_method 0 2026-02-17T06:02:33.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request con 0x7f9c2c06c890 auth_method 0 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.722+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:33.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.724+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v1:10.20.193.78:6789/0 conn(0x7f9c2c09c4a0 legacy=0x7f9c2c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 10 monclient(hunting): get_auth_request con 0x7f9c2c06d1b0 auth_method 0 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:33.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request con 0x7f9c2c06c890 auth_method 0 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:33.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:33.923+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:02:34.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:34.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 10 monclient(hunting): get_auth_request con 0x7f9c2c06d1b0 auth_method 0 2026-02-17T06:02:34.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:34.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:34.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:34.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:02:34.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:34.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request con 0x7f9c2c06c890 auth_method 0 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c31bb0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.323+0000 7f9c31bb0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.324+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.324+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.324+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:34.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:34.324+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 10 monclient(hunting): get_auth_request con 0x7f9c2c06d1b0 auth_method 0 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:35.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.124+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:02:35.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:35.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request con 0x7f9c2c06c890 auth_method 0 2026-02-17T06:02:35.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:35.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:35.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:35.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:02:35.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:35.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:35.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:35.125+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:02:36.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 10 monclient(hunting): tick 2026-02-17T06:02:36.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:02:36.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 monclient(hunting): continuing hunt 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 msgr2=0x7f9c2c135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c2c06d1b0 0x7f9c2c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 msgr2=0x7f9c2c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.115:3300/0 conn(0x7f9c2c06c890 0x7f9c2c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 -- 10.20.193.78:0/3807453840 >> v1:10.20.193.78:6789/0 conn(0x7f9c2c09c4a0 legacy=0x7f9c2c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 10 monclient: _add_conns ranks=[1,3] 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9c10003820 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f9c10007550 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 -- 10.20.193.78:0/3807453840 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c2c09cf60 con 0x7f9c10003820 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 1 -- 10.20.193.78:0/3807453840 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f9c2c09cc80 con 0x7f9c10007550 2026-02-17T06:02:36.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c1b7fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 10 monclient(hunting): get_auth_request con 0x7f9c10007550 auth_method 0 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.722+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:36.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.725+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3807453840 >> v1:10.20.193.115:6789/0 conn(0x7f9c10003820 legacy=0x7f9c10005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 10 monclient(hunting): get_auth_request con 0x7f9c10007550 auth_method 0 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:36.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:36.923+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:02:37.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:37.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 10 monclient(hunting): get_auth_request con 0x7f9c10007550 auth_method 0 2026-02-17T06:02:37.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:37.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:37.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:37.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:02:37.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:37.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:37.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:37.323+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:02:38.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 10 monclient(hunting): get_auth_request con 0x7f9c10007550 auth_method 0 2026-02-17T06:02:38.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:38.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:02:38.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:38.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:38.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.124+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1082457526 0 0) 0x7f9c2c09cf60 con 0x7f9c10003820 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): _negotiate 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): my global_id is 204100 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c14003620 con 0x7f9c10003820 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3301542239 0 0) 0x7f9c14003620 con 0x7f9c10003820 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 msgr2=0x7f9c10009a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 --2- 10.20.193.78:0/3807453840 >> v2:10.20.193.78:3300/0 conn(0x7f9c10007550 0x7f9c10009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: _finish_hunting 0 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1000b190 con 0x7f9c10003820 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: _finish_auth 0 2026-02-17T06:02:38.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: _check_auth_tickets 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f9c1c00f040 con 0x7f9c10003820 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: dump: 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 >> v1:10.20.193.115:6789/0 conn(0x7f9c10003820 legacy=0x7f9c10005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 1 -- 10.20.193.78:0/3807453840 reap_dead start 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14003ba0 0x7f9c14003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): picked mon.b con 0x7f9c14003ba0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c14004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): picked mon.a con 0x7f9c14004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14005510 0x7f9c1400bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): picked mon.c con 0x7f9c14005510 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c09cc80 con 0x7f9c14004900 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1400c420 con 0x7f9c14003ba0 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1400c620 con 0x7f9c14005510 2026-02-17T06:02:38.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c30bae640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c14004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14005510 0x7f9c1400bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient(hunting): get_auth_request con 0x7f9c14004900 auth_method 0 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 10 monclient(hunting): get_auth_request con 0x7f9c14005510 auth_method 0 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c323b1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14003ba0 0x7f9c14003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request con 0x7f9c14003ba0 auth_method 0 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c31bb0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c31bb0640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient(hunting): handle_auth_done global_id 204100 payload 293 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14005510 msgr2=0x7f9c1400bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14005510 0x7f9c1400bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14003ba0 msgr2=0x7f9c14003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14003ba0 0x7f9c14003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient: _finish_hunting 0 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 1 monclient: found mon.a 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:38.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:38.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c14003620 con 0x7f9c14004900 2026-02-17T06:02:38.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient: _finish_auth 0 2026-02-17T06:02:38.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 10 monclient: _check_auth_tickets 2026-02-17T06:02:38.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c313af640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:38.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c33e3b640 5 monclient: authenticate success, global_id 204100 2026-02-17T06:02:38.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.662+0000 7f9c33e3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:38.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c313af640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 msgr2=0x7f9c14004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c313af640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c14004d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9c20002ee0 tx=0x7f9c2002fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5846ad9f2f482fc1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20036070 con 0x7f9c14004900 2026-02-17T06:02:38.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 10 monclient: dump: 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c20030910 con 0x7f9c14004900 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 1 -- 10.20.193.78:0/3807453840 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20005d00 con 0x7f9c14004900 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 4 set_mon_vals no callback set 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c30bae640 10 monclient: dump: 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 10 monclient: shutdown 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 msgr2=0x7f9c14004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c14004d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9c20002ee0 tx=0x7f9c2002fef0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3807453840 shutdown_connections 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14005510 0x7f9c1400bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c14004d80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3807453840 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14003ba0 0x7f9c14003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3807453840 >> 10.20.193.78:0/3807453840 conn(0x7f9c2c0a6620 msgr2=0x7f9c2c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3807453840 shutdown_connections 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.665+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3807453840 wait complete. 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.666+0000 7f9c33e3b640 10 monclient: build_initial_monmap 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.666+0000 7f9c33e3b640 10 monclient: monmap: 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.666+0000 7f9c33e3b640 1 Processor -- start 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.666+0000 7f9c33e3b640 1 -- start start 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.666+0000 7f9c33e3b640 10 monclient: init 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14003ba0 0x7f9c2c136160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient(hunting): picked mon.noname-c con 0x7f9c14003ba0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c2c1366a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient(hunting): picked mon.noname-a con 0x7f9c14004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:38.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14005510 0x7f9c2c11fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient(hunting): picked mon.noname-b con 0x7f9c14005510 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c0a01a0 con 0x7f9c14004900 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c0a0020 con 0x7f9c14005510 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2c09be20 con 0x7f9c14003ba0 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c33e3b640 10 monclient(hunting): authenticate will time out at 1604.221261s 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14003ba0 0x7f9c2c136160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c2c1366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14003ba0 0x7f9c2c136160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51212/0 (socket says 10.20.193.78:51212) 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c2c1366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36576/0 (socket says 10.20.193.78:36576) 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3110957833 learned_addr learned my addr 10.20.193.78:0/3110957833 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request con 0x7f9c14003ba0 auth_method 0 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 10 monclient(hunting): get_auth_request con 0x7f9c14004900 auth_method 0 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient(hunting): handle_auth_done global_id 144656 payload 824 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14005510 msgr2=0x7f9c2c11fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14005510 0x7f9c2c11fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 msgr2=0x7f9c2c1366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c2c1366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient: _finish_hunting 0 2026-02-17T06:02:38.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 monclient: found mon.noname-c 2026-02-17T06:02:38.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:38.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:02:38.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3110957833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2c123b40 con 0x7f9c14003ba0 2026-02-17T06:02:38.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.667+0000 7f9c313af640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c2c1366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c31bb0640 1 -- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14003ba0 msgr2=0x7f9c2c136160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c31bb0640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14003ba0 0x7f9c2c136160 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0158b0 tx=0x7f9c1c015d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 1 -- 10.20.193.78:0/3110957833 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c020480 con 0x7f9c14003ba0 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: dump: 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c33e3b640 5 monclient: authenticate success, global_id 144656 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 1 -- 10.20.193.78:0/3110957833 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1c020620 con 0x7f9c14003ba0 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 1 -- 10.20.193.78:0/3110957833 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0209f0 con 0x7f9c14003ba0 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1bfff640 4 set_mon_vals no callback set 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c1a7fc640 10 monclient: dump: 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.673 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c33e3b640 10 monclient: _renew_subs 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c33e3b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2c136ea0 con 0x7f9c14003ba0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c33e3b640 10 monclient: _renew_subs 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c33e3b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.669+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2c09d810 con 0x7f9c14003ba0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c1a7fc640 1 -- 10.20.193.78:0/3110957833 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9c1c017440 con 0x7f9c14003ba0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c33e3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c33e3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c33e3b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2c074980 con 0x7f9c14003ba0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c1a7fc640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9bfc03deb0 0x7f9bfc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c1a7fc640 1 -- 10.20.193.78:0/3110957833 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(592..592 src has 335..592) ==== 4735+0+0 (secure 0 0 0) 0x7f9c1c054080 con 0x7f9c14003ba0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c313af640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9bfc03deb0 0x7f9bfc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.670+0000 7f9c313af640 10 monclient: get_auth_request con 0x7f9bfc03deb0 auth_method 0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.673+0000 7f9c1a7fc640 1 -- 10.20.193.78:0/3110957833 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9c1c052ab0 con 0x7f9c14003ba0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.673+0000 7f9c1a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.673+0000 7f9c1a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:38.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.673+0000 7f9c313af640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9bfc03deb0 0x7f9bfc040370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c20038000 tx=0x7f9c20036000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.716 INFO:tasks.rados.rados.0.trial115.stdout:2784: done (4 left) 2026-02-17T06:02:38.716 INFO:tasks.rados.rados.0.trial115.stdout:2785: done (3 left) 2026-02-17T06:02:38.716 INFO:tasks.rados.rados.0.trial115.stdout:2786: done (2 left) 2026-02-17T06:02:38.716 INFO:tasks.rados.rados.0.trial115.stdout:2787: done (1 left) 2026-02-17T06:02:38.716 INFO:tasks.rados.rados.0.trial115.stdout:2788: done (0 left) 2026-02-17T06:02:38.716 INFO:tasks.rados.rados.0.trial115.stdout:2789: read oid 8 snap -1 2026-02-17T06:02:38.716 INFO:tasks.rados.rados.0.trial115.stdout:2789: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:38.716 INFO:tasks.rados.rados.0.trial115.stdout:2790: snap_remove snap 280 2026-02-17T06:02:38.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.786+0000 7f9c33e3b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:02:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.786+0000 7f9c33e3b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.786+0000 7f9c33e3b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.786+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9c2c122130 con 0x7f9c14003ba0 2026-02-17T06:02:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.786+0000 7f9c1a7fc640 1 -- 10.20.193.78:0/3110957833 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f9c1c01e9c0 con 0x7f9c14003ba0 2026-02-17T06:02:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.786+0000 7f9c1a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:38.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.786+0000 7f9c1a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:38.787 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9bfc03deb0 msgr2=0x7f9bfc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9bfc03deb0 0x7f9bfc040370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c20038000 tx=0x7f9c20036000 comp rx=0 tx=0).stop 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 10 monclient: shutdown 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14003ba0 msgr2=0x7f9c2c136160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14003ba0 0x7f9c2c136160 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0158b0 tx=0x7f9c1c015d80 comp rx=0 tx=0).stop 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 shutdown_connections 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9bfc03deb0 0x7f9bfc040370 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9c14005510 0x7f9c2c11fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9c14004900 0x7f9c2c1366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 --2- 10.20.193.78:0/3110957833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9c14003ba0 0x7f9c2c136160 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 >> 10.20.193.78:0/3110957833 conn(0x7f9c2c0a6620 msgr2=0x7f9c2c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 shutdown_connections 2026-02-17T06:02:38.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.787+0000 7f9c33e3b640 1 -- 10.20.193.78:0/3110957833 wait complete. 2026-02-17T06:02:38.796 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:02:38.796 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:02:38.796 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:02:38.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient: get_monmap_and_config 2026-02-17T06:02:38.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient: build_initial_monmap 2026-02-17T06:02:38.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient: monmap: 2026-02-17T06:02:38.864 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:38.864 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:38.864367+0000 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:38.864367+0000 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:38.865 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 1 Processor -- start 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 1 -- start start 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient: init 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa7c409ea90 0x7fa7c40a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient(hunting): picked mon.noname-c con 0x7fa7c409ea90 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa7c409ee70 addr v1:10.20.193.78:6789/0 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.863+0000 7fa7c9af2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa7c413d640 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c9af2640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c40a0a20 con 0x7fa7c409ee70 2026-02-17T06:02:38.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c9af2640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c40a0460 con 0x7fa7c413d640 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c9af2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c9af2640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c40a0d00 con 0x7fa7c409ea90 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c9af2640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c9af2640 10 monclient(hunting): authenticate will time out at 1604.418023s 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c2ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fa7c409ee70 0x7fa7c409f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50596/0 (socket says 10.20.193.78:50596) 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c2ffd640 1 -- 10.20.193.78:0/3099217967 learned_addr learned my addr 10.20.193.78:0/3099217967 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c37fe640 1 --2- 10.20.193.78:0/3099217967 >> v2:10.20.193.115:3300/0 conn(0x7fa7c409ea90 0x7fa7c40a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c37fe640 10 monclient(hunting): get_auth_request con 0x7fa7c409ea90 auth_method 0 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.864+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c2ffd640 1 -- 10.20.193.78:0/3099217967 >> v1:10.20.193.78:6789/0 conn(0x7fa7c409ee70 legacy=0x7fa7c409f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c27fc640 1 -- 10.20.193.78:0/3099217967 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386149221 0 0) 0x7fa7c40a0a20 con 0x7fa7c409ee70 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c27fc640 10 monclient(hunting): _negotiate 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c27fc640 10 monclient(hunting): my global_id is 184275 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c27fc640 1 -- 10.20.193.78:0/3099217967 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7a8003620 con 0x7fa7c409ee70 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c3fff640 1 -- 10.20.193.78:0/3099217967 >> v1:10.20.193.115:6789/0 conn(0x7fa7c413d640 legacy=0x7fa7c413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_done global_id 204103 payload 274 2026-02-17T06:02:38.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c37fe640 1 -- 10.20.193.78:0/3099217967 >> v1:10.20.193.115:6789/0 conn(0x7fa7c413d640 legacy=0x7fa7c413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c37fe640 1 -- 10.20.193.78:0/3099217967 >> v1:10.20.193.78:6789/0 conn(0x7fa7c409ee70 legacy=0x7fa7c409f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c37fe640 10 monclient: _finish_hunting 0 2026-02-17T06:02:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c37fe640 1 monclient: found mon.noname-c 2026-02-17T06:02:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:02:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c37fe640 1 -- 10.20.193.78:0/3099217967 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c40a0740 con 0x7fa7c409ea90 2026-02-17T06:02:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c27fc640 1 -- 10.20.193.78:0/3099217967 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fa7b40031b0 con 0x7fa7c409ee70 2026-02-17T06:02:38.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.866+0000 7fa7c27fc640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-17T06:02:38.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.868+0000 7fa7c37fe640 1 -- 10.20.193.78:0/3099217967 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ea90 msgr2=0x7fa7c40a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c37fe640 1 --2- 10.20.193.78:0/3099217967 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ea90 0x7fa7c40a13d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00099a0 tx=0x7fa7b002eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2807058ea9e0330c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 1 -- 10.20.193.78:0/3099217967 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b003c070 con 0x7fa7c409ea90 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: dump: 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 1 -- 10.20.193.78:0/3099217967 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b002fd10 con 0x7fa7c409ea90 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 5 monclient: authenticate success, global_id 204103 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:38.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 1 -- 10.20.193.78:0/3099217967 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0037660 con 0x7fa7c409ea90 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c27fc640 10 monclient: dump: 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.884 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 4 set_mon_vals no callback set 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 10 monclient: shutdown 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3099217967 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ea90 msgr2=0x7fa7c40a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 1 --2- 10.20.193.78:0/3099217967 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ea90 0x7fa7c40a13d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00099a0 tx=0x7fa7b002eeb0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3099217967 shutdown_connections 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 1 --2- 10.20.193.78:0/3099217967 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ea90 0x7fa7c40a13d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3099217967 >> 10.20.193.78:0/3099217967 conn(0x7fa7c40a7520 msgr2=0x7fa7c40a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3099217967 shutdown_connections 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3099217967 wait complete. 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 10 monclient: build_initial_monmap 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.869+0000 7fa7c9af2640 10 monclient: monmap: 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:38.886 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 1 Processor -- start 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 1 -- start start 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient: init 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7c409ea90 0x7fa7c41cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient(hunting): picked mon.noname-c con 0x7fa7c409ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ee70 0x7fa7c41cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient(hunting): picked mon.noname-b con 0x7fa7c409ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413d640 0x7fa7c413a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient(hunting): picked mon.noname-a con 0x7fa7c413d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c40a6820 con 0x7fa7c413d640 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c40a66a0 con 0x7fa7c409ee70 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c4141150 con 0x7fa7c409ea90 2026-02-17T06:02:38.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c9af2640 10 monclient(hunting): authenticate will time out at 1604.424423s 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413d640 0x7fa7c413a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413d640 0x7fa7c413a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36592/0 (socket says 10.20.193.78:36592) 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 -- 10.20.193.78:0/3045101114 learned_addr learned my addr 10.20.193.78:0/3045101114 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient(hunting): get_auth_request con 0x7fa7c413d640 auth_method 0 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c37fe640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7c409ea90 0x7fa7c41cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c37fe640 10 monclient(hunting): get_auth_request con 0x7fa7c409ea90 auth_method 0 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient(hunting): handle_auth_done global_id 184278 payload 824 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c2ffd640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ee70 0x7fa7c41cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7c409ea90 msgr2=0x7fa7c41cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7c409ea90 0x7fa7c41cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ee70 msgr2=0x7fa7c41cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ee70 0x7fa7c41cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 monclient: found mon.noname-a 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c3fff640 1 -- 10.20.193.78:0/3045101114 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c406ef90 con 0x7fa7c413d640 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c2ffd640 10 monclient: get_auth_request con 0x7fa7c409ee70 auth_method 0 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.870+0000 7fa7c2ffd640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7c409ee70 0x7fa7c41cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa7c3fff640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413d640 msgr2=0x7fa7c413a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa7c3fff640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413d640 0x7fa7c413a670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800cdf0 tx=0x7fa7b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8017070 con 0x7fa7c413d640 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: dump: 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: _finish_auth 0 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b8007270 con 0x7fa7c413d640 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8007640 con 0x7fa7c413d640 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa79ffff640 10 monclient: dump: 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.893 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa7c1ffb640 4 set_mon_vals no callback set 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa7c9af2640 5 monclient: authenticate success, global_id 184278 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa7c9af2640 10 monclient: _renew_subs 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa7c9af2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.872+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c406faf0 con 0x7fa7c413d640 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c9af2640 10 monclient: _renew_subs 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c9af2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c41cfda0 con 0x7fa7c413d640 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c9af2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c9af2640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413abb0 0x7fa7c413af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c9af2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c406f250 con 0x7fa7c413abb0 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413abb0 0x7fa7c413af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 10 monclient: get_auth_request con 0x7fa7c413abb0 auth_method 0 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_done global_id 184281 payload 824 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa7c37fe640 1 -- 10.20.193.78:0/3045101114 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7c40a1c10 con 0x7fa7c413abb0 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa7b8007820 con 0x7fa7c413d640 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa79ffff640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa79403deb0 0x7fa794040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.873+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(593..593 src has 335..593) ==== 4735+0+0 (secure 0 0 0) 0x7fa7b80528d0 con 0x7fa7c413d640 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.874+0000 7fa7c2ffd640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa79403deb0 0x7fa794040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.874+0000 7fa7c2ffd640 10 monclient: get_auth_request con 0x7fa79403deb0 auth_method 0 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.875+0000 7fa7c37fe640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413abb0 msgr2=0x7fa7c413af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.875+0000 7fa7c37fe640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413abb0 0x7fa7c413af90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0039000 tx=0x7fa7b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.875+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b003c040 con 0x7fa7c413abb0 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.875+0000 7fa79ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.875+0000 7fa79ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.875+0000 7fa79ffff640 10 monclient: dump: 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.876+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa7b0037e00 con 0x7fa7c413abb0 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.876+0000 7fa79ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.876+0000 7fa79ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.876+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413abb0 msgr2=0x7fa7c413af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.876+0000 7fa79ffff640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413abb0 0x7fa7c413af90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0039000 tx=0x7fa7b00047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.876+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413abb0 msgr2=0x7fa7c413af90 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.876+0000 7fa79ffff640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413abb0 0x7fa7c413af90 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0039000 tx=0x7fa7b00047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.876+0000 7fa7c2ffd640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa79403deb0 0x7fa794040370 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4003550 tx=0x7fa7b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c9af2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:02:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c9af2640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413b4d0 0x7fa7c41da390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c9af2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c40a13b0 con 0x7fa7c413b4d0 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413b4d0 0x7fa7c41da390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 10 monclient: get_auth_request con 0x7fa7c413b4d0 auth_method 0 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 10 monclient(hunting): handle_auth_done global_id 184284 payload 824 2026-02-17T06:02:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.879+0000 7fa7c37fe640 1 -- 10.20.193.78:0/3045101114 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa7c40744d0 con 0x7fa7c413b4d0 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa7c37fe640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413b4d0 msgr2=0x7fa7c41da390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa7c37fe640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413b4d0 0x7fa7c41da390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00099a0 tx=0x7fa7b0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0055070 con 0x7fa7c413b4d0 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 10 monclient: dump: 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.901 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7fa7b00513f0 con 0x7fa7c413b4d0 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413b4d0 msgr2=0x7fa7c41da390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413b4d0 0x7fa7c41da390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00099a0 tx=0x7fa7b0008030 comp rx=0 tx=0).stop 2026-02-17T06:02:38.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413b4d0 msgr2=0x7fa7c41da390 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:38.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.881+0000 7fa79ffff640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413b4d0 0x7fa7c41da390 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00099a0 tx=0x7fa7b0008030 comp rx=0 tx=0).stop 2026-02-17T06:02:38.903 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:02:38.903 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:38.903 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:38.903 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:02:38.903 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 194, 2026-02-17T06:02:38.903 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 123662, 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:38.904 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:38.905 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:02:38.906 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:38.907 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:38.908 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:38.909 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:02:38.910 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:38.911 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:38.912 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:02:38.913 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:38.914 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:38.914 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:38.914 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:02:38.914 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:02:38.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa79403deb0 msgr2=0x7fa794040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa79403deb0 0x7fa794040370 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4003550 tx=0x7fa7b4009040 comp rx=0 tx=0).stop 2026-02-17T06:02:38.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 10 monclient: shutdown 2026-02-17T06:02:38.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:38.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413d640 msgr2=0x7fa7c413a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 1 --2- 10.20.193.78:0/3045101114 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7c413d640 0x7fa7c413a670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800cdf0 tx=0x7fa7b800a430 comp rx=0 tx=0).stop 2026-02-17T06:02:38.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c3fff640 1 -- 10.20.193.78:0/3045101114 reap_dead start 2026-02-17T06:02:38.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c3fff640 1 -- 10.20.193.78:0/3045101114 reap_dead start 2026-02-17T06:02:38.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 shutdown_connections 2026-02-17T06:02:38.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 >> 10.20.193.78:0/3045101114 conn(0x7fa7c40a7520 msgr2=0x7fa7c40a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:38.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 shutdown_connections 2026-02-17T06:02:38.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.882+0000 7fa7c9af2640 1 -- 10.20.193.78:0/3045101114 wait complete. 2026-02-17T06:02:38.916 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:02:38.916 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:02:38.916 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:02:38.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.981+0000 7fd8c139e640 10 monclient: get_monmap_and_config 2026-02-17T06:02:38.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.981+0000 7fd8c139e640 10 monclient: build_initial_monmap 2026-02-17T06:02:38.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.981+0000 7fd8c139e640 10 monclient: monmap: 2026-02-17T06:02:38.984 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:38.983134+0000 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:38.983134+0000 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 1 Processor -- start 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 1 -- start start 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient: init 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd8bc06c890 0x7fd8bc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient(hunting): picked mon.noname-a con 0x7fd8bc06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd8bc09c4a0 0x7fd8bc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient(hunting): picked mon.noname-c con 0x7fd8bc09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd8bc134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8bc06daf0 con 0x7fd8bc134df0 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd8bc0a0880 con 0x7fd8bc06c890 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd8bc06d6f0 con 0x7fd8bc09c4a0 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8c139e640 10 monclient(hunting): authenticate will time out at 1604.536692s 2026-02-17T06:02:38.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd8bc06c890 0x7fd8bc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd8bc06c890 0x7fd8bc06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36620/0 (socket says 10.20.193.78:36620) 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 1 -- 10.20.193.78:0/3601471941 learned_addr learned my addr 10.20.193.78:0/3601471941 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 10 monclient(hunting): get_auth_request con 0x7fd8bc06c890 auth_method 0 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8ba7fc640 1 --2- 10.20.193.78:0/3601471941 >> v2:10.20.193.115:3300/0 conn(0x7fd8bc09c4a0 0x7fd8bc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8ba7fc640 10 monclient(hunting): get_auth_request con 0x7fd8bc09c4a0 auth_method 0 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.982+0000 7fd8ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 10 monclient(hunting): handle_auth_done global_id 184287 payload 274 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 1 -- 10.20.193.78:0/3601471941 >> v2:10.20.193.115:3300/0 conn(0x7fd8bc09c4a0 msgr2=0x7fd8bc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 1 --2- 10.20.193.78:0/3601471941 >> v2:10.20.193.115:3300/0 conn(0x7fd8bc09c4a0 0x7fd8bc06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 1 -- 10.20.193.78:0/3601471941 >> v1:10.20.193.115:6789/0 conn(0x7fd8bc134df0 legacy=0x7fd8bc135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:38.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 10 monclient: _finish_hunting 0 2026-02-17T06:02:38.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 1 monclient: found mon.noname-a 2026-02-17T06:02:38.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:38.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:38.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.983+0000 7fd8baffd640 1 -- 10.20.193.78:0/3601471941 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8bc09cc80 con 0x7fd8bc06c890 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8baffd640 1 -- 10.20.193.78:0/3601471941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc06c890 msgr2=0x7fd8bc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8baffd640 1 --2- 10.20.193.78:0/3601471941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc06c890 0x7fd8bc06cc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc09c980 tx=0x7fd8b002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa77bf17ad6a57fe server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 1 -- 10.20.193.78:0/3601471941 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b003c070 con 0x7fd8bc06c890 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: dump: 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.992 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 1 -- 10.20.193.78:0/3601471941 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b002fca0 con 0x7fd8bc06c890 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 5 monclient: authenticate success, global_id 184287 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 1 -- 10.20.193.78:0/3601471941 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0037660 con 0x7fd8bc06c890 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 4 set_mon_vals no callback set 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8b9ffb640 10 monclient: dump: 2026-02-17T06:02:38.993 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 10 monclient: shutdown 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 1 -- 10.20.193.78:0/3601471941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc06c890 msgr2=0x7fd8bc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 1 --2- 10.20.193.78:0/3601471941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc06c890 0x7fd8bc06cc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc09c980 tx=0x7fd8b002ee60 comp rx=0 tx=0).stop 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 1 -- 10.20.193.78:0/3601471941 shutdown_connections 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 1 --2- 10.20.193.78:0/3601471941 >> v2:10.20.193.115:3300/0 conn(0x7fd8bc09c4a0 0x7fd8bc06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 1 --2- 10.20.193.78:0/3601471941 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc06c890 0x7fd8bc06cc70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.985+0000 7fd8c139e640 1 -- 10.20.193.78:0/3601471941 >> 10.20.193.78:0/3601471941 conn(0x7fd8bc0a6620 msgr2=0x7fd8bc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 -- 10.20.193.78:0/3601471941 shutdown_connections 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 -- 10.20.193.78:0/3601471941 wait complete. 2026-02-17T06:02:38.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient: build_initial_monmap 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient: monmap: 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 Processor -- start 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 -- start start 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient: init 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8bc06c890 0x7fd8bc1cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient(hunting): picked mon.noname-b con 0x7fd8bc06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09c4a0 0x7fd8bc1cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient(hunting): picked mon.noname-c con 0x7fd8bc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:38.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc134df0 0x7fd8bc09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient(hunting): picked mon.noname-a con 0x7fd8bc134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc0a01a0 con 0x7fd8bc134df0 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc0a0020 con 0x7fd8bc06c890 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd8bc09be20 con 0x7fd8bc09c4a0 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8c139e640 10 monclient(hunting): authenticate will time out at 1604.540893s 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8bb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc134df0 0x7fd8bc09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8ba7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09c4a0 0x7fd8bc1cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8bb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc134df0 0x7fd8bc09db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36626/0 (socket says 10.20.193.78:36626) 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8bb7fe640 1 -- 10.20.193.78:0/94528278 learned_addr learned my addr 10.20.193.78:0/94528278 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8bb7fe640 10 monclient(hunting): get_auth_request con 0x7fd8bc134df0 auth_method 0 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8ba7fc640 10 monclient(hunting): get_auth_request con 0x7fd8bc09c4a0 auth_method 0 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.986+0000 7fd8ba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8baffd640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8bc06c890 0x7fd8bc1cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8baffd640 10 monclient(hunting): get_auth_request con 0x7fd8bc06c890 auth_method 0 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 10 monclient(hunting): handle_auth_done global_id 184290 payload 824 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09c4a0 msgr2=0x7fd8bc1cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09c4a0 0x7fd8bc1cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8bc06c890 msgr2=0x7fd8bc1cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd8bc06c890 0x7fd8bc1cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 1 monclient: found mon.noname-a 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:38.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.987+0000 7fd8bb7fe640 1 -- 10.20.193.78:0/94528278 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8bc09d810 con 0x7fd8bc134df0 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8bb7fe640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc134df0 msgr2=0x7fd8bc09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8bb7fe640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc134df0 0x7fd8bc09db50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00cb60 tx=0x7fd8ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac016020 con 0x7fd8bc134df0 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: dump: 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:02:38.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ac012420 con 0x7fd8bc134df0 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac012810 con 0x7fd8bc134df0 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8b97fa640 4 set_mon_vals no callback set 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd89b7fe640 10 monclient: dump: 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 5 monclient: authenticate success, global_id 184290 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 10 monclient: _renew_subs 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:38.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8bc09f7b0 con 0x7fd8bc134df0 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 10 monclient: _renew_subs 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8bc126060 con 0x7fd8bc134df0 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e090 0x7fd8bc09e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.989+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd8bc127880 con 0x7fd8bc09e090 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e090 0x7fd8bc09e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 10 monclient: get_auth_request con 0x7fd8bc09e090 auth_method 0 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd8ac0129f0 con 0x7fd8bc134df0 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 10 monclient(hunting): handle_auth_done global_id 144665 payload 824 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8baffd640 1 -- 10.20.193.78:0/94528278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8bc074980 con 0x7fd8bc09e090 2026-02-17T06:02:39.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd89b7fe640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd88803df30 0x7fd8880403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(593..593 src has 335..593) ==== 4735+0+0 (secure 0 0 0) 0x7fd8ac0530e0 con 0x7fd8bc134df0 2026-02-17T06:02:39.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8ba7fc640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd88803df30 0x7fd8880403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.990+0000 7fd8ba7fc640 10 monclient: get_auth_request con 0x7fd88803df30 auth_method 0 2026-02-17T06:02:39.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd8baffd640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e090 msgr2=0x7fd8bc09e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:39.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd8baffd640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e090 0x7fd8bc09e470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc074de0 tx=0x7fd8b0004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b003c040 con 0x7fd8bc09e090 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 10 monclient: dump: 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd8b002fe50 con 0x7fd8bc09e090 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:39.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e090 msgr2=0x7fd8bc09e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e090 0x7fd8bc09e470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc074de0 tx=0x7fd8b0004830 comp rx=0 tx=0).stop 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e090 msgr2=0x7fd8bc09e470 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.992+0000 7fd89b7fe640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e090 0x7fd8bc09e470 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc074de0 tx=0x7fd8b0004830 comp rx=0 tx=0).stop 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.993+0000 7fd8ba7fc640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd88803df30 0x7fd8880403f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc09f290 tx=0x7fd8a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8c139e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8c139e640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e9b0 0x7fd8bc1da430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8c139e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd8bc135a20 con 0x7fd8bc09e9b0 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e9b0 0x7fd8bc1da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 10 monclient: get_auth_request con 0x7fd8bc09e9b0 auth_method 0 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 10 monclient(hunting): handle_auth_done global_id 144668 payload 824 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.995+0000 7fd8baffd640 1 -- 10.20.193.78:0/94528278 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd8bc0744d0 con 0x7fd8bc09e9b0 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.997+0000 7fd8baffd640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e9b0 msgr2=0x7fd8bc1da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.997+0000 7fd8baffd640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e9b0 0x7fd8bc1da430 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc06c6a0 tx=0x7fd8b00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.997+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b003c050 con 0x7fd8bc09e9b0 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.997+0000 7fd89b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.997+0000 7fd89b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.997+0000 7fd89b7fe640 10 monclient: dump: 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7fd8b0041660 con 0x7fd8bc09e9b0 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd89b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd89b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e9b0 msgr2=0x7fd8bc1da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd89b7fe640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e9b0 0x7fd8bc1da430 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc06c6a0 tx=0x7fd8b00047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd89b7fe640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e9b0 msgr2=0x7fd8bc1da430 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd89b7fe640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd8bc09e9b0 0x7fd8bc1da430 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc06c6a0 tx=0x7fd8b00047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 194, 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 342532, 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:02:39.006 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:39.007 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:02:39.008 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:39.009 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.010 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:02:39.011 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.012 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:39.013 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.014 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.015 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:02:39.015 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd88803df30 msgr2=0x7fd8880403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd88803df30 0x7fd8880403f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc09f290 tx=0x7fd8a4008040 comp rx=0 tx=0).stop 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 10 monclient: shutdown 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc134df0 msgr2=0x7fd8bc09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 1 --2- 10.20.193.78:0/94528278 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd8bc134df0 0x7fd8bc09db50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00cb60 tx=0x7fd8ac00a430 comp rx=0 tx=0).stop 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8bb7fe640 1 -- 10.20.193.78:0/94528278 reap_dead start 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8bb7fe640 1 -- 10.20.193.78:0/94528278 reap_dead start 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 shutdown_connections 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 >> 10.20.193.78:0/94528278 conn(0x7fd8bc0a6620 msgr2=0x7fd8bc071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 shutdown_connections 2026-02-17T06:02:39.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:38.998+0000 7fd8c139e640 1 -- 10.20.193.78:0/94528278 wait complete. 2026-02-17T06:02:39.016 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:02:39.016 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:02:39.017 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:02:39.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient: get_monmap_and_config 2026-02-17T06:02:39.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient: build_initial_monmap 2026-02-17T06:02:39.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient: monmap: 2026-02-17T06:02:39.083 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:39.083 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.083 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:39.083412+0000 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:39.083412+0000 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 1 Processor -- start 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 1 -- start start 2026-02-17T06:02:39.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient: init 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0320135f20 0x7f0320136300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient(hunting): picked mon.noname-a con 0x7f0320135f20 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0320136840 0x7f032006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient(hunting): picked mon.noname-c con 0x7f0320136840 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f032006cdd0 addr v1:10.20.193.78:6789/0 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032009d140 con 0x7f032006cdd0 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f032009cb10 con 0x7f0320135f20 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0320135c60 con 0x7f0320136840 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.082+0000 7f031f577640 10 monclient(hunting): authenticate will time out at 1604.636973s 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0320135f20 0x7f0320136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0320135f20 0x7f0320136300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36638/0 (socket says 10.20.193.78:36638) 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 1 -- 10.20.193.78:0/2706464320 learned_addr learned my addr 10.20.193.78:0/2706464320 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 10 monclient(hunting): get_auth_request con 0x7f0320135f20 auth_method 0 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 1 --2- 10.20.193.78:0/2706464320 >> v2:10.20.193.115:3300/0 conn(0x7f0320136840 0x7f032006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient(hunting): get_auth_request con 0x7f0320136840 auth_method 0 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient(hunting): handle_auth_done global_id 204115 payload 274 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 1 -- 10.20.193.78:0/2706464320 >> v2:10.20.193.78:3300/0 conn(0x7f0320135f20 msgr2=0x7f0320136300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 1 --2- 10.20.193.78:0/2706464320 >> v2:10.20.193.78:3300/0 conn(0x7f0320135f20 0x7f0320136300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 1 -- 10.20.193.78:0/2706464320 >> v1:10.20.193.78:6789/0 conn(0x7f032006cdd0 legacy=0x7f032013cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient: _finish_hunting 0 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 1 monclient: found mon.noname-c 2026-02-17T06:02:39.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:39.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:02:39.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.083+0000 7f031dd74640 1 -- 10.20.193.78:0/2706464320 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032009c330 con 0x7f0320136840 2026-02-17T06:02:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.085+0000 7f031dd74640 1 -- 10.20.193.78:0/2706464320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320136840 msgr2=0x7f032006c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.085+0000 7f031dd74640 1 --2- 10.20.193.78:0/2706464320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320136840 0x7f032006c890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f032009d5d0 tx=0x7f030802ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceaecec33258f1dc server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 1 -- 10.20.193.78:0/2706464320 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030803c070 con 0x7f0320136840 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: dump: 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: _finish_auth 0 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: _check_auth_tickets 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 1 -- 10.20.193.78:0/2706464320 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f030802fbb0 con 0x7f0320136840 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 1 -- 10.20.193.78:0/2706464320 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308037440 con 0x7f0320136840 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 5 monclient: authenticate success, global_id 204115 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 4 set_mon_vals no callback set 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031d573640 10 monclient: dump: 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.096 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 10 monclient: shutdown 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 1 -- 10.20.193.78:0/2706464320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320136840 msgr2=0x7f032006c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 1 --2- 10.20.193.78:0/2706464320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320136840 0x7f032006c890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f032009d5d0 tx=0x7f030802ed30 comp rx=0 tx=0).stop 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 1 -- 10.20.193.78:0/2706464320 shutdown_connections 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 1 --2- 10.20.193.78:0/2706464320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320136840 0x7f032006c890 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 1 --2- 10.20.193.78:0/2706464320 >> v2:10.20.193.78:3300/0 conn(0x7f0320135f20 0x7f0320136300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 1 -- 10.20.193.78:0/2706464320 >> 10.20.193.78:0/2706464320 conn(0x7f03200a65f0 msgr2=0x7f03200a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 1 -- 10.20.193.78:0/2706464320 shutdown_connections 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 1 -- 10.20.193.78:0/2706464320 wait complete. 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 10 monclient: build_initial_monmap 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.086+0000 7f031f577640 10 monclient: monmap: 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 1 Processor -- start 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 1 -- start start 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient: init 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f032006cdd0 0x7f03201ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient(hunting): picked mon.noname-b con 0x7f032006cdd0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0320135f20 0x7f0320134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient(hunting): picked mon.noname-a con 0x7f0320135f20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0320136840 0x7f0320135330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient(hunting): picked mon.noname-c con 0x7f0320136840 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f032009ff90 con 0x7f0320135f20 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f032009fe10 con 0x7f032006cdd0 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f03200a02f0 con 0x7f0320136840 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031f577640 10 monclient(hunting): authenticate will time out at 1604.641514s 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0320135f20 0x7f0320134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0320135f20 0x7f0320134df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36654/0 (socket says 10.20.193.78:36654) 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 -- 10.20.193.78:0/3683169368 learned_addr learned my addr 10.20.193.78:0/3683169368 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient(hunting): get_auth_request con 0x7f0320135f20 auth_method 0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031ed76640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0320136840 0x7f0320135330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031ed76640 10 monclient(hunting): get_auth_request con 0x7f0320136840 auth_method 0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031e575640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f032006cdd0 0x7f03201ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031e575640 10 monclient(hunting): get_auth_request con 0x7f032006cdd0 auth_method 0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient(hunting): handle_auth_done global_id 184296 payload 824 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0320136840 msgr2=0x7f0320135330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0320136840 0x7f0320135330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f032006cdd0 msgr2=0x7f03201ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f032006cdd0 0x7f03201ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient: _finish_hunting 0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 monclient: found mon.noname-a 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.087+0000 7f031dd74640 1 -- 10.20.193.78:0/3683169368 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0320126510 con 0x7f0320135f20 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f031dd74640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0320135f20 msgr2=0x7f0320134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f031dd74640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0320135f20 0x7f0320134df0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0308002410 tx=0x7f03080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030803c040 con 0x7f0320135f20 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f0306ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f0306ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f0306ffd640 10 monclient: dump: 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.098 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f0306ffd640 10 monclient: _finish_auth 0 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f0306ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f0306ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0308047070 con 0x7f0320135f20 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f031f577640 5 monclient: authenticate success, global_id 184296 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f031f577640 10 monclient: _renew_subs 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f031f577640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.089+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03201cd180 con 0x7f0320135f20 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f0306ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308040a60 con 0x7f0320135f20 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f0306ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031cd72640 4 set_mon_vals no callback set 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f0306ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f0306ffd640 10 monclient: dump: 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031f577640 10 monclient: _renew_subs 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031f577640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03200a36c0 con 0x7f0320135f20 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031f577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031f577640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320125f50 0x7f0320122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031f577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0320126330 con 0x7f0320125f50 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f030803f030 con 0x7f0320135f20 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031e575640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320125f50 0x7f0320122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031e575640 10 monclient: get_auth_request con 0x7f0320125f50 auth_method 0 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f0306ffd640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02ec03deb0 0x7f02ec040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(593..593 src has 335..593) ==== 4735+0+0 (secure 0 0 0) 0x7f0308077b60 con 0x7f0320135f20 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031ed76640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02ec03deb0 0x7f02ec040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.090+0000 7f031ed76640 10 monclient: get_auth_request con 0x7f02ec03deb0 auth_method 0 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.091+0000 7f031e575640 10 monclient(hunting): handle_auth_done global_id 204121 payload 824 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.091+0000 7f031e575640 1 -- 10.20.193.78:0/3683169368 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03200729a0 con 0x7f0320125f50 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f031ed76640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02ec03deb0 0x7f02ec040370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f03201255c0 tx=0x7f03100042b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f031e575640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320125f50 msgr2=0x7f0320122860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f031e575640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320125f50 0x7f0320122860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0314011800 tx=0x7f0314011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031400fe50 con 0x7f0320125f50 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 10 monclient: dump: 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f031400fe50 con 0x7f0320125f50 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320125f50 msgr2=0x7f0320122860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320125f50 0x7f0320122860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0314011800 tx=0x7f0314011cd0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320125f50 msgr2=0x7f0320122860 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.093+0000 7f0306ffd640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320125f50 0x7f0320122860 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0314011800 tx=0x7f0314011cd0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031f577640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031f577640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320122da0 0x7f03201d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031f577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0320136300 con 0x7f0320122da0 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320122da0 0x7f03201d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 10 monclient: get_auth_request con 0x7f0320122da0 auth_method 0 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 10 monclient(hunting): handle_auth_done global_id 204124 payload 824 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.097+0000 7f031e575640 1 -- 10.20.193.78:0/3683169368 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f03200744d0 con 0x7f0320122da0 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031e575640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320122da0 msgr2=0x7f03201d7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031e575640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320122da0 0x7f03201d7f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0314011800 tx=0x7f03140047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031402b020 con 0x7f0320122da0 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 10 monclient: dump: 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f0314015660 con 0x7f0320122da0 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320122da0 msgr2=0x7f03201d7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320122da0 0x7f03201d7f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0314011800 tx=0x7f03140047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320122da0 msgr2=0x7f03201d7f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f0306ffd640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0320122da0 0x7f03201d7f90 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0314011800 tx=0x7f03140047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 194, 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 20387, 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:02:39.102 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:02:39.103 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.104 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:02:39.105 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:02:39.106 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02ec03deb0 msgr2=0x7f02ec040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031f577640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02ec03deb0 0x7f02ec040370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f03201255c0 tx=0x7f03100042b0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031f577640 10 monclient: shutdown 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031f577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0320135f20 msgr2=0x7f0320134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031ed76640 1 -- 10.20.193.78:0/3683169368 reap_dead start 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.100+0000 7f031f577640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0320135f20 0x7f0320134df0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0308002410 tx=0x7f03080047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.101+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 shutdown_connections 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.101+0000 7f031f577640 1 --2- 10.20.193.78:0/3683169368 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0320135f20 0x7f0320134df0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.101+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 >> 10.20.193.78:0/3683169368 conn(0x7f03200a65f0 msgr2=0x7f032006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.101+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 shutdown_connections 2026-02-17T06:02:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.101+0000 7f031f577640 1 -- 10.20.193.78:0/3683169368 wait complete. 2026-02-17T06:02:39.107 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:02:39.107 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:02:39.108 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:02:39.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.173+0000 7fcc82551640 10 monclient: get_monmap_and_config 2026-02-17T06:02:39.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.173+0000 7fcc82551640 10 monclient: build_initial_monmap 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.173+0000 7fcc82551640 10 monclient: monmap: 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:39.174623+0000 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:39.174623+0000 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.173+0000 7fcc82551640 1 Processor -- start 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.173+0000 7fcc82551640 1 -- start start 2026-02-17T06:02:39.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.173+0000 7fcc82551640 10 monclient: init 2026-02-17T06:02:39.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcc7c137fc0 addr v1:10.20.193.78:6789/0 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcc7c1383a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c13e570 0x7fcc7c140960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 10 monclient(hunting): picked mon.noname-b con 0x7fcc7c13e570 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c09ef70 con 0x7fcc7c137fc0 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c0a0470 con 0x7fcc7c1383a0 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c0a0750 con 0x7fcc7c13e570 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc82551640 10 monclient(hunting): authenticate will time out at 1604.728328s 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc7bfff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fcc7c137fc0 0x7fcc7c0a14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50608/0 (socket says 10.20.193.78:50608) 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc7bfff640 1 -- 10.20.193.78:0/455878642 learned_addr learned my addr 10.20.193.78:0/455878642 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 1 --2- 10.20.193.78:0/455878642 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c13e570 0x7fcc7c140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient(hunting): get_auth_request con 0x7fcc7c13e570 auth_method 0 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient(hunting): handle_auth_done global_id 144674 payload 274 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 1 -- 10.20.193.78:0/455878642 >> v1:10.20.193.115:6789/0 conn(0x7fcc7c1383a0 legacy=0x7fcc7c1387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 1 -- 10.20.193.78:0/455878642 >> v1:10.20.193.78:6789/0 conn(0x7fcc7c137fc0 legacy=0x7fcc7c0a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient: _finish_hunting 0 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 1 monclient: found mon.noname-b 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:02:39.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.174+0000 7fcc80ac7640 1 -- 10.20.193.78:0/455878642 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c0740e0 con 0x7fcc7c13e570 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc80ac7640 1 -- 10.20.193.78:0/455878642 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c13e570 msgr2=0x7fcc7c140960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc80ac7640 1 --2- 10.20.193.78:0/455878642 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c13e570 0x7fcc7c140960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009dc0 tx=0x7fcc7002f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ca145084cf2b392 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 1 -- 10.20.193.78:0/455878642 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7003c070 con 0x7fcc7c13e570 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: dump: 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: _finish_auth 0 2026-02-17T06:02:39.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 1 -- 10.20.193.78:0/455878642 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc70037640 con 0x7fcc7c13e570 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 5 monclient: authenticate success, global_id 144674 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 1 -- 10.20.193.78:0/455878642 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70037a10 con 0x7fcc7c13e570 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc7affd640 10 monclient: dump: 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:39.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 4 set_mon_vals no callback set 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 10 monclient: shutdown 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 1 -- 10.20.193.78:0/455878642 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c13e570 msgr2=0x7fcc7c140960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 1 --2- 10.20.193.78:0/455878642 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c13e570 0x7fcc7c140960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009dc0 tx=0x7fcc7002f1c0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 1 -- 10.20.193.78:0/455878642 shutdown_connections 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 1 --2- 10.20.193.78:0/455878642 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c13e570 0x7fcc7c140960 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 1 -- 10.20.193.78:0/455878642 >> 10.20.193.78:0/455878642 conn(0x7fcc7c0aae40 msgr2=0x7fcc7c0ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 1 -- 10.20.193.78:0/455878642 shutdown_connections 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.177+0000 7fcc82551640 1 -- 10.20.193.78:0/455878642 wait complete. 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient: build_initial_monmap 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient: monmap: 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:39.183 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 1 Processor -- start 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 1 -- start start 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient: init 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcc7c137fc0 0x7fcc7c09dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient(hunting): picked mon.noname-b con 0x7fcc7c137fc0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c1383a0 0x7fcc7c09e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient(hunting): picked mon.noname-c con 0x7fcc7c1383a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcc7c13e570 0x7fcc7c13a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient(hunting): picked mon.noname-a con 0x7fcc7c13e570 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c141230 con 0x7fcc7c13e570 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c1410b0 con 0x7fcc7c137fc0 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c1413b0 con 0x7fcc7c1383a0 2026-02-17T06:02:39.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc82551640 10 monclient(hunting): authenticate will time out at 1604.732641s 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c1383a0 0x7fcc7c09e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcc7c13e570 0x7fcc7c13a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcc7c13e570 0x7fcc7c13a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36662/0 (socket says 10.20.193.78:36662) 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c1383a0 0x7fcc7c09e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51286/0 (socket says 10.20.193.78:51286) 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 1 -- 10.20.193.78:0/842280875 learned_addr learned my addr 10.20.193.78:0/842280875 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 10 monclient(hunting): get_auth_request con 0x7fcc7c13e570 auth_method 0 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient(hunting): get_auth_request con 0x7fcc7c1383a0 auth_method 0 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc80ac7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7bfff640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcc7c137fc0 0x7fcc7c09dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient(hunting): handle_auth_done global_id 144677 payload 824 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 1 -- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcc7c137fc0 msgr2=0x7fcc7c09dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcc7c137fc0 0x7fcc7c09dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 1 -- 10.20.193.78:0/842280875 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcc7c13e570 msgr2=0x7fcc7c13a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcc7c13e570 0x7fcc7c13a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 1 monclient: found mon.noname-c 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7b7fe640 1 -- 10.20.193.78:0/842280875 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c13c430 con 0x7fcc7c1383a0 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7bfff640 10 monclient: get_auth_request con 0x7fcc7c137fc0 auth_method 0 2026-02-17T06:02:39.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.178+0000 7fcc7bfff640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcc7c137fc0 0x7fcc7c09dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc7b7fe640 1 -- 10.20.193.78:0/842280875 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c1383a0 msgr2=0x7fcc7c09e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc7b7fe640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c1383a0 0x7fcc7c09e130 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c480 tx=0x7fcc6c00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 1 -- 10.20.193.78:0/842280875 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c012a40 con 0x7fcc7c1383a0 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: dump: 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.188 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: _finish_auth 0 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 1 -- 10.20.193.78:0/842280875 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c017070 con 0x7fcc7c1383a0 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 5 monclient: authenticate success, global_id 144677 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 10 monclient: _renew_subs 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c06f4e0 con 0x7fcc7c1383a0 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 1 -- 10.20.193.78:0/842280875 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c012e40 con 0x7fcc7c1383a0 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc7a7fc640 4 set_mon_vals no callback set 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 10 monclient: dump: 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.189 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 10 monclient: _renew_subs 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c13bf60 con 0x7fcc7c1383a0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c0a14a0 con 0x7fcc7c1383a0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.181+0000 7fcc78ff9640 1 -- 10.20.193.78:0/842280875 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fcc6c01c460 con 0x7fcc7c1383a0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.182+0000 7fcc78ff9640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc4c03df00 0x7fcc4c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.182+0000 7fcc78ff9640 1 -- 10.20.193.78:0/842280875 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(593..593 src has 335..593) ==== 4735+0+0 (secure 0 0 0) 0x7fcc6c052520 con 0x7fcc7c1383a0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.182+0000 7fcc7bfff640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc4c03df00 0x7fcc4c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.182+0000 7fcc7bfff640 10 monclient: get_auth_request con 0x7fcc4c03df00 auth_method 0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.184+0000 7fcc78ff9640 1 -- 10.20.193.78:0/842280875 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcc6c01ac60 con 0x7fcc7c1383a0 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.184+0000 7fcc78ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:39.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.184+0000 7fcc78ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:39.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.185+0000 7fcc7bfff640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc4c03df00 0x7fcc4c0403c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcc680052b0 tx=0x7fcc68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:39.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.299+0000 7fcc82551640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:02:39.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.299+0000 7fcc82551640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:02:39.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.299+0000 7fcc82551640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:39.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.299+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fcc7c13c640 con 0x7fcc7c1383a0 2026-02-17T06:02:39.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.300+0000 7fcc78ff9640 1 -- 10.20.193.78:0/842280875 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fcc6c019d00 con 0x7fcc7c1383a0 2026-02-17T06:02:39.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.300+0000 7fcc78ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:02:39.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.300+0000 7fcc78ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=30,config=2,health=11,kv=11,logm=46} crc {auth=70627968,config=692619927,health=2887653597,kv=3215367626,logm=1918465157}) 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=30,config=2,health=11,kv=12,logm=45} crc {auth=70627968,config=692619927,health=2887653597,kv=1893472683,logm=4062574279}) 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=30,config=2,health=11,kv=11,logm=46} crc {auth=70627968,config=692619927,health=2887653597,kv=3215367626,logm=1918465157}) 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=28,config=1,health=12,kv=12,logm=47} crc {auth=1951861589,config=3678701417,health=2923988628,kv=449989585,logm=3017503760}) 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=81033622}) 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1298604985}) 2026-02-17T06:02:39.303 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.304 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=81033622}) 2026-02-17T06:02:39.304 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.300+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=819676696}) 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc4c03df00 msgr2=0x7fcc4c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc4c03df00 0x7fcc4c0403c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcc680052b0 tx=0x7fcc68009040 comp rx=0 tx=0).stop 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 10 monclient: shutdown 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c1383a0 msgr2=0x7fcc7c09e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c1383a0 0x7fcc7c09e130 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c480 tx=0x7fcc6c00c950 comp rx=0 tx=0).stop 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 shutdown_connections 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcc7c13e570 0x7fcc7c13a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcc7c1383a0 0x7fcc7c09e130 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc4c03df00 0x7fcc4c0403c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 --2- 10.20.193.78:0/842280875 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcc7c137fc0 0x7fcc7c09dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:39.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 >> 10.20.193.78:0/842280875 conn(0x7fcc7c0aae40 msgr2=0x7fcc7c13d790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:39.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 shutdown_connections 2026-02-17T06:02:39.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:39.301+0000 7fcc82551640 1 -- 10.20.193.78:0/842280875 wait complete. 2026-02-17T06:02:39.313 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.301+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.313 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.301+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=74319020}) 2026-02-17T06:02:39.313 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.301+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4240747177}) 2026-02-17T06:02:39.313 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.301+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.313 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.301+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=74319020}) 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.301+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3390901947}) 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=941967806}) 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2707219615}) 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=941967806}) 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=723840933}) 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=572706922}) 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=367140704}) 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.314 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=572706922}) 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.302+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=987382875}) 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2926042426}) 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2906351247}) 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2926042426}) 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2541029904}) 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.315 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308580756}) 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1895395850}) 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308580756}) 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=429983501}) 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866124940}) 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1566075772}) 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866124940}) 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.303+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1154478172}) 2026-02-17T06:02:39.316 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=96763381}) 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1401761691}) 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=96763381}) 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=674101267}) 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2847205881}) 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2586280351}) 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2847205881}) 2026-02-17T06:02:39.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.304+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2710966944}) 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761759471}) 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4022756201}) 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761759471}) 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1164973244}) 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=276023208}) 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1327084171}) 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.318 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=276023208}) 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1037500944}) 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2622944541}) 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3150284698}) 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2622944541}) 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.305+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=126259883}) 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2818611}) 2026-02-17T06:02:39.319 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3077714413}) 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2818611}) 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3589050635}) 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1971270991}) 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564381447}) 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1971270991}) 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2011481583}) 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3469175413}) 2026-02-17T06:02:39.320 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=922286033}) 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3469175413}) 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.306+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3837424178}) 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1574834149}) 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=189810685}) 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1574834149}) 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3080746384}) 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.321 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3234377460}) 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39558279}) 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3234377460}) 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2542290100}) 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3765194078}) 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=819109324}) 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3765194078}) 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.307+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1611862462}) 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178948556}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3574129162}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178948556}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4116361895}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2767733069}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=435436768}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2767733069}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3365406445}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3958779274}) 2026-02-17T06:02:39.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2647576709}) 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3958779274}) 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.308+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3579044598}) 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.309+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.309+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087763623}) 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.309+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1345058201}) 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.309+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.309+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087763623}) 2026-02-17T06:02:39.324 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.309+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1085610834}) 2026-02-17T06:02:39.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:02:39.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.665+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.665+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3204731731}) 2026-02-17T06:02:39.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.665+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2746944134}) 2026-02-17T06:02:39.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.665+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.665+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3204731731}) 2026-02-17T06:02:39.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.665+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2387079450}) 2026-02-17T06:02:39.718 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.717+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.719 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.717+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2540038995}) 2026-02-17T06:02:39.719 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.717+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4137426688}) 2026-02-17T06:02:39.719 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.717+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.719 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.717+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2540038995}) 2026-02-17T06:02:39.719 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.717+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3666126470}) 2026-02-17T06:02:39.723 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.722+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.723 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.722+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466512693}) 2026-02-17T06:02:39.723 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.722+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3532849028}) 2026-02-17T06:02:39.723 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.722+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.723 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.722+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466512693}) 2026-02-17T06:02:39.724 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.722+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=257170773}) 2026-02-17T06:02:39.725 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.723+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.725 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.723+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1124906202}) 2026-02-17T06:02:39.725 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.723+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=606773989}) 2026-02-17T06:02:39.725 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.723+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.725 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.723+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1124906202}) 2026-02-17T06:02:39.725 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.723+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2975926927}) 2026-02-17T06:02:39.726 INFO:tasks.rados.rados.0.trial115.stdout:2790: done (1 left) 2026-02-17T06:02:39.726 INFO:tasks.rados.rados.0.trial115.stdout:2791: snap_remove snap 281 2026-02-17T06:02:39.829 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.829 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1480847613}) 2026-02-17T06:02:39.829 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1044849951}) 2026-02-17T06:02:39.829 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.829 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1480847613}) 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3978438433}) 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1015396011}) 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2246967162}) 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1015396011}) 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3266920626}) 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1584363304}) 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2259691838}) 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.830 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1584363304}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.817+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3672740783}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1643045653}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1131379704}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1643045653}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2159142573}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280560627}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2084078460}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280560627}) 2026-02-17T06:02:39.831 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4009042929}) 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=940168542}) 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=735470653}) 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=940168542}) 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.818+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=869592816}) 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230704788}) 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=41698263}) 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.832 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230704788}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2362330077}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1561863791}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2775259450}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1561863791}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531565086}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=571476253}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3959486618}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=571476253}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.819+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2031614372}) 2026-02-17T06:02:39.833 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759127654}) 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1386631754}) 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759127654}) 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2774333959}) 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2174974248}) 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=340394169}) 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2174974248}) 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3684747757}) 2026-02-17T06:02:39.834 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=388242444}) 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2592981230}) 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=388242444}) 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.820+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=279658986}) 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2984555260}) 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=789864360}) 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.835 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2984555260}) 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=79482879}) 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918291314}) 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3088723276}) 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918291314}) 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2737803354}) 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1188545440}) 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1143013143}) 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.836 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1188545440}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.821+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3369311461}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3039514795}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523471287}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3039514795}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2071468443}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335810075}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=694052532}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335810075}) 2026-02-17T06:02:39.837 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=736986533}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1998364000}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=247761512}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1998364000}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.822+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3171138952}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1080177035}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2659581267}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1080177035}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1754720944}) 2026-02-17T06:02:39.838 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1805559568}) 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262361010}) 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1805559568}) 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998747554}) 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3174241368}) 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2127484179}) 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3174241368}) 2026-02-17T06:02:39.839 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=252572750}) 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=847630071}) 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2835511464}) 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=847630071}) 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3659530865}) 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=407371000}) 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3494137510}) 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=407371000}) 2026-02-17T06:02:39.840 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3976775862}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3384042937}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3396625601}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3384042937}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.824+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3912359192}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557108245}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1218121308}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557108245}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2022198488}) 2026-02-17T06:02:39.841 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2781010175}) 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4234127788}) 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2781010175}) 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856045707}) 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825492920}) 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1057937553}) 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825492920}) 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.825+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=201071147}) 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.842 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2819896302}) 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642245763}) 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2819896302}) 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272954033}) 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360001952}) 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375603687}) 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360001952}) 2026-02-17T06:02:39.843 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:39.826+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4184141939}) 2026-02-17T06:02:40.079 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.079 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933932052}) 2026-02-17T06:02:40.079 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1346296757}) 2026-02-17T06:02:40.079 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.079 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933932052}) 2026-02-17T06:02:40.079 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3738921966}) 2026-02-17T06:02:40.079 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.079 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3501556827}) 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3614971119}) 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3501556827}) 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.077+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3991181994}) 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=38,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=17} crc {logm=4235679292,mdsmap=3140514009,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1095157140,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=3200932474,osd_snap=4040696886}) 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=15} crc {logm=1164455820,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1024424577,mon_config_key=4028116958,monmap=1784110042,osd_metadata=1528513801,osd_pg_creating=3200932474,osd_snap=1671278854}) 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=38,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=17} crc {logm=4235679292,mdsmap=3140514009,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1095157140,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=3200932474,osd_snap=4040696886}) 2026-02-17T06:02:40.080 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=37,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=18} crc {logm=366977131,mdsmap=1160443688,mgr=4262968943,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4030053510,mon_config_key=1912222953,monmap=1784110042,osd_metadata=1611097156,osd_pg_creating=3200932474,osd_snap=2297364386}) 2026-02-17T06:02:40.082 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.082 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=84,osdmap=16} crc {osd_snap=1916564551,osdmap=616102006}) 2026-02-17T06:02:40.082 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=2769933757,osdmap=1347073862}) 2026-02-17T06:02:40.082 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.082 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=84,osdmap=16} crc {osd_snap=1916564551,osdmap=616102006}) 2026-02-17T06:02:40.082 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=84,osdmap=16} crc {osd_snap=2430643558,osdmap=3619283497}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3226998188}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=655751979}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3226998188}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.078+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1598994735}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.079+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.079+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=924871174}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.079+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3541015104}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.079+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.079+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=924871174}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.079+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1940549728}) 2026-02-17T06:02:40.083 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.080+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.084 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.080+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2916799358}) 2026-02-17T06:02:40.084 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.080+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4118509799}) 2026-02-17T06:02:40.084 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.080+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.084 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.080+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2916799358}) 2026-02-17T06:02:40.084 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.080+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1394487163}) 2026-02-17T06:02:40.085 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.083+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.085 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.083+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=73} crc {osdmap=1900989900}) 2026-02-17T06:02:40.085 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.083+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=359128693}) 2026-02-17T06:02:40.085 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.083+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:02:40.085 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.083+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=73} crc {osdmap=1900989900}) 2026-02-17T06:02:40.085 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:02:40.083+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=1242889274}) 2026-02-17T06:02:40.326 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:02:40.326 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:02:40.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.394+0000 7fa693e85640 10 monclient: get_monmap_and_config 2026-02-17T06:02:40.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.394+0000 7fa693e85640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.394+0000 7fa693e85640 10 monclient: monmap: 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:40.396066+0000 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:40.396066+0000 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:40.399 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 1 Processor -- start 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 1 -- start start 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient: init 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c09ddd0 0x7fa68c09e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient(hunting): picked mon.noname-b con 0x7fa68c09ddd0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa68c073f60 0x7fa68c137f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient(hunting): picked mon.noname-c con 0x7fa68c073f60 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa68c138490 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68c0a2850 con 0x7fa68c138490 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa68c0a2560 con 0x7fa68c073f60 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa68c0a3810 con 0x7fa68c09ddd0 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa693e85640 10 monclient(hunting): authenticate will time out at 1605.949624s 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c09ddd0 0x7fa68c09e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c09ddd0 0x7fa68c09e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51294/0 (socket says 10.20.193.78:51294) 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 1 -- 10.20.193.78:0/4061338646 learned_addr learned my addr 10.20.193.78:0/4061338646 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient(hunting): get_auth_request con 0x7fa68c09ddd0 auth_method 0 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient(hunting): handle_auth_done global_id 144680 payload 274 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 1 -- 10.20.193.78:0/4061338646 >> v2:10.20.193.115:3300/0 conn(0x7fa68c073f60 msgr2=0x7fa68c137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 1 --2- 10.20.193.78:0/4061338646 >> v2:10.20.193.115:3300/0 conn(0x7fa68c073f60 0x7fa68c137f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 1 -- 10.20.193.78:0/4061338646 >> v1:10.20.193.115:6789/0 conn(0x7fa68c138490 legacy=0x7fa68c13f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:40.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 1 monclient: found mon.noname-b 2026-02-17T06:02:40.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:02:40.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.395+0000 7fa691bfa640 1 -- 10.20.193.78:0/4061338646 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c0a3af0 con 0x7fa68c09ddd0 2026-02-17T06:02:40.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa691bfa640 1 -- 10.20.193.78:0/4061338646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c09ddd0 msgr2=0x7fa68c09e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa691bfa640 1 --2- 10.20.193.78:0/4061338646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c09ddd0 0x7fa68c09e1b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009970 tx=0x7fa67c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfdd6a294356be7e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 1 -- 10.20.193.78:0/4061338646 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c03c070 con 0x7fa68c09ddd0 2026-02-17T06:02:40.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:40.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: dump: 2026-02-17T06:02:40.410 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 5 monclient: authenticate success, global_id 144680 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 1 -- 10.20.193.78:0/4061338646 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67c02fdb0 con 0x7fa68c09ddd0 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 1 -- 10.20.193.78:0/4061338646 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c037660 con 0x7fa68c09ddd0 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 4 set_mon_vals no callback set 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa690bf8640 10 monclient: dump: 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 10 monclient: shutdown 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 1 -- 10.20.193.78:0/4061338646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c09ddd0 msgr2=0x7fa68c09e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 1 --2- 10.20.193.78:0/4061338646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c09ddd0 0x7fa68c09e1b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009970 tx=0x7fa67c02ee90 comp rx=0 tx=0).stop 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 1 -- 10.20.193.78:0/4061338646 shutdown_connections 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 1 --2- 10.20.193.78:0/4061338646 >> v2:10.20.193.115:3300/0 conn(0x7fa68c073f60 0x7fa68c137f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 1 --2- 10.20.193.78:0/4061338646 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c09ddd0 0x7fa68c09e1b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 1 -- 10.20.193.78:0/4061338646 >> 10.20.193.78:0/4061338646 conn(0x7fa68c0aacd0 msgr2=0x7fa68c0ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.398+0000 7fa693e85640 1 -- 10.20.193.78:0/4061338646 shutdown_connections 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 -- 10.20.193.78:0/4061338646 wait complete. 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient: monmap: 2026-02-17T06:02:40.414 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 Processor -- start 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 -- start start 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient: init 2026-02-17T06:02:40.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c073f60 0x7fa68c12fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient(hunting): picked mon.noname-c con 0x7fa68c073f60 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa68c09ddd0 0x7fa68c130090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient(hunting): picked mon.noname-a con 0x7fa68c09ddd0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa68c138490 0x7fa68c1305d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient(hunting): picked mon.noname-b con 0x7fa68c138490 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c09e6f0 con 0x7fa68c09ddd0 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c0a57e0 con 0x7fa68c138490 2026-02-17T06:02:40.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa68c0a4840 con 0x7fa68c073f60 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa693e85640 10 monclient(hunting): authenticate will time out at 1605.953938s 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c073f60 0x7fa68c12fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c073f60 0x7fa68c12fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51306/0 (socket says 10.20.193.78:51306) 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 1 -- 10.20.193.78:0/1332165849 learned_addr learned my addr 10.20.193.78:0/1332165849 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 10 monclient(hunting): get_auth_request con 0x7fa68c073f60 auth_method 0 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa6913f9640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa68c09ddd0 0x7fa68c130090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.399+0000 7fa691bfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa6913f9640 10 monclient(hunting): get_auth_request con 0x7fa68c09ddd0 auth_method 0 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa6913f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa6913f9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa6913f9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa6913f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa6913f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa6913f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 10 monclient(hunting): handle_auth_done global_id 144683 payload 824 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 1 -- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa68c138490 msgr2=0x7fa68c1305d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa68c138490 0x7fa68c1305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 1 -- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa68c09ddd0 msgr2=0x7fa68c130090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa68c09ddd0 0x7fa68c130090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 1 monclient: found mon.noname-c 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.400+0000 7fa691bfa640 1 -- 10.20.193.78:0/1332165849 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c0aa4a0 con 0x7fa68c073f60 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa691bfa640 1 -- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c073f60 msgr2=0x7fa68c12fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa691bfa640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c073f60 0x7fa68c12fb50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009aa0 tx=0x7fa67c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 1 -- 10.20.193.78:0/1332165849 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c046070 con 0x7fa68c073f60 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:40.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: dump: 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 1 -- 10.20.193.78:0/1332165849 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67c037e10 con 0x7fa68c073f60 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 1 -- 10.20.193.78:0/1332165849 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c03c040 con 0x7fa68c073f60 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67bfff640 4 set_mon_vals no callback set 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa67a7fc640 10 monclient: dump: 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.421 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa693e85640 5 monclient: authenticate success, global_id 144683 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa693e85640 10 monclient: _renew_subs 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa693e85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68c131af0 con 0x7fa68c073f60 2026-02-17T06:02:40.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa693e85640 10 monclient: _renew_subs 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa693e85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.402+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68c1330d0 con 0x7fa68c073f60 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa693e85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa693e85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa693e85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68c137f50 con 0x7fa68c073f60 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa67a7fc640 1 -- 10.20.193.78:0/1332165849 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa67c0418b0 con 0x7fa68c073f60 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa67a7fc640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa65c03df00 0x7fa65c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa67a7fc640 1 -- 10.20.193.78:0/1332165849 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(594..594 src has 335..594) ==== 4763+0+0 (secure 0 0 0) 0x7fa67c077f30 con 0x7fa68c073f60 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa6913f9640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa65c03df00 0x7fa65c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.403+0000 7fa6913f9640 10 monclient: get_auth_request con 0x7fa65c03df00 auth_method 0 2026-02-17T06:02:40.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.405+0000 7fa67a7fc640 1 -- 10.20.193.78:0/1332165849 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa67c049c80 con 0x7fa68c073f60 2026-02-17T06:02:40.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.405+0000 7fa67a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.405+0000 7fa67a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:40.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.406+0000 7fa6913f9640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa65c03df00 0x7fa65c0403c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa68c073880 tx=0x7fa680009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.520+0000 7fa693e85640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:02:40.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.520+0000 7fa693e85640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:40.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.520+0000 7fa693e85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:40.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.520+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa68c0744d0 con 0x7fa68c073f60 2026-02-17T06:02:40.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.520+0000 7fa67a7fc640 1 -- 10.20.193.78:0/1332165849 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa67c041bd0 con 0x7fa68c073f60 2026-02-17T06:02:40.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.520+0000 7fa67a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:40.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.520+0000 7fa67a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:40.523 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:02:40.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa65c03df00 msgr2=0x7fa65c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa65c03df00 0x7fa65c0403c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa68c073880 tx=0x7fa680009210 comp rx=0 tx=0).stop 2026-02-17T06:02:40.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 10 monclient: shutdown 2026-02-17T06:02:40.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c073f60 msgr2=0x7fa68c12fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c073f60 0x7fa68c12fb50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009aa0 tx=0x7fa67c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 shutdown_connections 2026-02-17T06:02:40.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa65c03df00 0x7fa65c0403c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa68c138490 0x7fa68c1305d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa68c09ddd0 0x7fa68c130090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 --2- 10.20.193.78:0/1332165849 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa68c073f60 0x7fa68c12fb50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 >> 10.20.193.78:0/1332165849 conn(0x7fa68c0aacd0 msgr2=0x7fa68c0a1480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 shutdown_connections 2026-02-17T06:02:40.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.521+0000 7fa693e85640 1 -- 10.20.193.78:0/1332165849 wait complete. 2026-02-17T06:02:40.531 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:02:40.531 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:02:40.531 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:02:40.531 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:02:40.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.597+0000 7f95debcc640 10 monclient: get_monmap_and_config 2026-02-17T06:02:40.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.597+0000 7f95debcc640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.597+0000 7f95debcc640 10 monclient: monmap: 2026-02-17T06:02:40.601 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:40.599107+0000 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:40.599107+0000 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:40.602 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 1 Processor -- start 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 1 -- start start 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient: init 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f95d806c890 0x7f95d806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient(hunting): picked mon.noname-c con 0x7f95d806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f95d809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f95d806d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d8135c10 con 0x7f95d806d1b0 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d806daf0 con 0x7f95d809c4a0 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f95d80a0880 con 0x7f95d806c890 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95debcc640 10 monclient(hunting): authenticate will time out at 1606.152653s 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dd142640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f95d806d1b0 0x7f95d8135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50614/0 (socket says 10.20.193.78:50614) 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dd142640 1 -- 10.20.193.78:0/904110819 learned_addr learned my addr 10.20.193.78:0/904110819 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dc941640 1 --2- 10.20.193.78:0/904110819 >> v2:10.20.193.115:3300/0 conn(0x7f95d806c890 0x7f95d806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dc941640 10 monclient(hunting): get_auth_request con 0x7f95d806c890 auth_method 0 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dc941640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dc941640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dc941640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dc941640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dc941640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.598+0000 7f95dc941640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.599+0000 7f95dc941640 10 monclient(hunting): handle_auth_done global_id 204127 payload 274 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.599+0000 7f95dc941640 1 -- 10.20.193.78:0/904110819 >> v1:10.20.193.115:6789/0 conn(0x7f95d809c4a0 legacy=0x7f95d8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.599+0000 7f95dc941640 1 -- 10.20.193.78:0/904110819 >> v1:10.20.193.78:6789/0 conn(0x7f95d806d1b0 legacy=0x7f95d8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:40.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.599+0000 7f95dc941640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.599+0000 7f95dc941640 1 monclient: found mon.noname-c 2026-02-17T06:02:40.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.599+0000 7f95dc941640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.599+0000 7f95dc941640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:02:40.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.599+0000 7f95dc941640 1 -- 10.20.193.78:0/904110819 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d809cc80 con 0x7f95d806c890 2026-02-17T06:02:40.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95dc941640 1 -- 10.20.193.78:0/904110819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d806c890 msgr2=0x7f95d806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95dc941640 1 --2- 10.20.193.78:0/904110819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d806c890 0x7f95d806cc70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098a0 tx=0x7f95c802ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ad34af52096c41 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 1 -- 10.20.193.78:0/904110819 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c803c070 con 0x7f95d806c890 2026-02-17T06:02:40.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: dump: 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 5 monclient: authenticate success, global_id 204127 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 1 -- 10.20.193.78:0/904110819 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c802fce0 con 0x7f95d806c890 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 1 -- 10.20.193.78:0/904110819 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8037660 con 0x7f95d806c890 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 4 set_mon_vals no callback set 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95cf7fe640 10 monclient: dump: 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 10 monclient: shutdown 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 1 -- 10.20.193.78:0/904110819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d806c890 msgr2=0x7f95d806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 1 --2- 10.20.193.78:0/904110819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d806c890 0x7f95d806cc70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098a0 tx=0x7f95c802ed80 comp rx=0 tx=0).stop 2026-02-17T06:02:40.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 1 -- 10.20.193.78:0/904110819 shutdown_connections 2026-02-17T06:02:40.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 1 --2- 10.20.193.78:0/904110819 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d806c890 0x7f95d806cc70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 1 -- 10.20.193.78:0/904110819 >> 10.20.193.78:0/904110819 conn(0x7f95d80a6620 msgr2=0x7f95d80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 1 -- 10.20.193.78:0/904110819 shutdown_connections 2026-02-17T06:02:40.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.603+0000 7f95debcc640 1 -- 10.20.193.78:0/904110819 wait complete. 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient: monmap: 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 1 Processor -- start 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 1 -- start start 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient: init 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d806c890 0x7f95d81cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient(hunting): picked mon.noname-a con 0x7f95d806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95d806d1b0 0x7f95d8123930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient(hunting): picked mon.noname-c con 0x7f95d806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d809c4a0 0x7f95d8123e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient(hunting): picked mon.noname-b con 0x7f95d809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95d80a0020 con 0x7f95d806c890 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95d806d640 con 0x7f95d809c4a0 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f95d80a01a0 con 0x7f95d806d1b0 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95debcc640 10 monclient(hunting): authenticate will time out at 1606.158803s 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d806c890 0x7f95d81cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d806c890 0x7f95d81cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36694/0 (socket says 10.20.193.78:36694) 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 1 -- 10.20.193.78:0/765428761 learned_addr learned my addr 10.20.193.78:0/765428761 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 10 monclient(hunting): get_auth_request con 0x7f95d806c890 auth_method 0 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95cffff640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95d806d1b0 0x7f95d8123930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95cffff640 10 monclient(hunting): get_auth_request con 0x7f95d806d1b0 auth_method 0 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95dc941640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.604+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dd142640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d809c4a0 0x7f95d8123e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dd142640 10 monclient(hunting): get_auth_request con 0x7f95d809c4a0 auth_method 0 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dd142640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dd142640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dd142640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 10 monclient(hunting): handle_auth_done global_id 184305 payload 824 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95d806d1b0 msgr2=0x7f95d8123930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f95d806d1b0 0x7f95d8123930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d809c4a0 msgr2=0x7f95d8123e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f95d809c4a0 0x7f95d8123e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 1 monclient: found mon.noname-a 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.605+0000 7f95dc941640 1 -- 10.20.193.78:0/765428761 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d8128250 con 0x7f95d806c890 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95dc941640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d806c890 msgr2=0x7f95d81cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95dc941640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d806c890 0x7f95d81cf7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098a0 tx=0x7f95c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8046070 con 0x7f95d806c890 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: dump: 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.609 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95debcc640 5 monclient: authenticate success, global_id 184305 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95debcc640 10 monclient: _renew_subs 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95debcc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d81272b0 con 0x7f95d806c890 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c8041870 con 0x7f95d806c890 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c803c040 con 0x7f95d806c890 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95ceffd640 4 set_mon_vals no callback set 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95cd7fa640 10 monclient: dump: 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95debcc640 10 monclient: _renew_subs 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95debcc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.607+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d8134e80 con 0x7f95d806c890 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95debcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95debcc640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81cfce0 0x7f95d81243b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95debcc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95d809bf40 con 0x7f95d81cfce0 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f95c8041c60 con 0x7f95d806c890 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81cfce0 0x7f95d81243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 10 monclient: get_auth_request con 0x7f95d81cfce0 auth_method 0 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 10 monclient(hunting): handle_auth_done global_id 184308 payload 824 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cffff640 1 -- 10.20.193.78:0/765428761 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95d8125e60 con 0x7f95d81cfce0 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cd7fa640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f95a803deb0 0x7f95a8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(594..594 src has 335..594) ==== 4763+0+0 (secure 0 0 0) 0x7f95c8078090 con 0x7f95d806c890 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95dd142640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f95a803deb0 0x7f95a8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.608+0000 7f95dd142640 10 monclient: get_auth_request con 0x7f95a803deb0 auth_method 0 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cffff640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81cfce0 msgr2=0x7f95d81243b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cffff640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81cfce0 0x7f95d81243b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f95d8126d00 tx=0x7f95c000fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0018030 con 0x7f95d81cfce0 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 10 monclient: dump: 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f95c000db90 con 0x7f95d81cfce0 2026-02-17T06:02:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81cfce0 msgr2=0x7f95d81243b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81cfce0 0x7f95d81243b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f95d8126d00 tx=0x7f95c000fc10 comp rx=0 tx=0).stop 2026-02-17T06:02:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81cfce0 msgr2=0x7f95d81243b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.610+0000 7f95cd7fa640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81cfce0 0x7f95d81243b0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f95d8126d00 tx=0x7f95c000fc10 comp rx=0 tx=0).stop 2026-02-17T06:02:40.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.611+0000 7f95dd142640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f95a803deb0 0x7f95a8040370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f95d80a2380 tx=0x7f95d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.613+0000 7f95debcc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:02:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.613+0000 7f95debcc640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81248f0 0x7f95d81da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95debcc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8124cd0 con 0x7f95d81248f0 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81248f0 0x7f95d81da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 10 monclient: get_auth_request con 0x7f95d81248f0 auth_method 0 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 10 monclient(hunting): handle_auth_done global_id 184311 payload 824 2026-02-17T06:02:40.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.614+0000 7f95cffff640 1 -- 10.20.193.78:0/765428761 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f95d80744d0 con 0x7f95d81248f0 2026-02-17T06:02:40.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cffff640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81248f0 msgr2=0x7f95d81da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cffff640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81248f0 0x7f95d81da3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f95c0018000 tx=0x7f95c0003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0004230 con 0x7f95d81248f0 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 10 monclient: dump: 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f95c0004230 con 0x7f95d81248f0 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81248f0 msgr2=0x7f95d81da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81248f0 0x7f95d81da3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f95c0018000 tx=0x7f95c0003860 comp rx=0 tx=0).stop 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81248f0 msgr2=0x7f95d81da3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95cd7fa640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d81248f0 0x7f95d81da3c0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f95c0018000 tx=0x7f95c0003860 comp rx=0 tx=0).stop 2026-02-17T06:02:40.617 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 194, 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 125397, 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.618 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:40.619 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.620 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.621 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f95a803deb0 msgr2=0x7f95a8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f95a803deb0 0x7f95a8040370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f95d80a2380 tx=0x7f95d4009040 comp rx=0 tx=0).stop 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 10 monclient: shutdown 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d806c890 msgr2=0x7f95d81cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d806c890 0x7f95d81cf7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098a0 tx=0x7f95c80047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95dd142640 1 -- 10.20.193.78:0/765428761 reap_dead start 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 shutdown_connections 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 --2- 10.20.193.78:0/765428761 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f95d806c890 0x7f95d81cf7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 >> 10.20.193.78:0/765428761 conn(0x7f95d80a6620 msgr2=0x7f95d8072dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 shutdown_connections 2026-02-17T06:02:40.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.616+0000 7f95debcc640 1 -- 10.20.193.78:0/765428761 wait complete. 2026-02-17T06:02:40.623 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:02:40.624 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:02:40.624 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.691+0000 7eff31f7e640 10 monclient: get_monmap_and_config 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.691+0000 7eff31f7e640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.691+0000 7eff31f7e640 10 monclient: monmap: 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:40.692530+0000 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:40.692530+0000 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.694 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.691+0000 7eff31f7e640 1 Processor -- start 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.691+0000 7eff31f7e640 1 -- start start 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.691+0000 7eff31f7e640 10 monclient: init 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1384d0 0x7eff2c070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient(hunting): picked mon.noname-b con 0x7eff2c1384d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7eff2c1388b0 0x7eff2c138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient(hunting): picked mon.noname-a con 0x7eff2c1388b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7eff2c0a3220 0x7eff2c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient(hunting): picked mon.noname-c con 0x7eff2c0a3220 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:40.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7eff2c073f70 con 0x7eff2c1388b0 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7eff2c0a4b20 con 0x7eff2c0a3220 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c0a4f70 con 0x7eff2c1384d0 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff31f7e640 10 monclient(hunting): authenticate will time out at 1606.246088s 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7eff2c1388b0 0x7eff2c138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1384d0 0x7eff2c070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7eff2c1388b0 0x7eff2c138cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36706/0 (socket says 10.20.193.78:36706) 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1384d0 0x7eff2c070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51314/0 (socket says 10.20.193.78:51314) 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 -- 10.20.193.78:0/4138684134 learned_addr learned my addr 10.20.193.78:0/4138684134 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient(hunting): get_auth_request con 0x7eff2c1388b0 auth_method 0 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 10 monclient(hunting): get_auth_request con 0x7eff2c1384d0 auth_method 0 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2bfff640 1 --2- 10.20.193.78:0/4138684134 >> v2:10.20.193.115:3300/0 conn(0x7eff2c0a3220 0x7eff2c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2bfff640 10 monclient(hunting): get_auth_request con 0x7eff2c0a3220 auth_method 0 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient(hunting): handle_auth_done global_id 184314 payload 274 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 -- 10.20.193.78:0/4138684134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1384d0 msgr2=0x7eff2c070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:40.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 --2- 10.20.193.78:0/4138684134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1384d0 0x7eff2c070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 -- 10.20.193.78:0/4138684134 >> v2:10.20.193.115:3300/0 conn(0x7eff2c0a3220 msgr2=0x7eff2c0a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:40.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 --2- 10.20.193.78:0/4138684134 >> v2:10.20.193.115:3300/0 conn(0x7eff2c0a3220 0x7eff2c0a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 monclient: found mon.noname-a 2026-02-17T06:02:40.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.692+0000 7eff2affd640 1 -- 10.20.193.78:0/4138684134 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c06f870 con 0x7eff2c1388b0 2026-02-17T06:02:40.703 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2affd640 1 -- 10.20.193.78:0/4138684134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c1388b0 msgr2=0x7eff2c138cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2affd640 1 --2- 10.20.193.78:0/4138684134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c1388b0 0x7eff2c138cd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0a5240 tx=0x7eff1c02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadf01458220b1a5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 1 -- 10.20.193.78:0/4138684134 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c03c070 con 0x7eff2c1388b0 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: dump: 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 1 -- 10.20.193.78:0/4138684134 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff1c02fd50 con 0x7eff2c1388b0 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff31f7e640 5 monclient: authenticate success, global_id 184314 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff31f7e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 1 -- 10.20.193.78:0/4138684134 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c037660 con 0x7eff2c1388b0 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff31f7e640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff31f7e640 4 set_mon_vals no callback set 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.694+0000 7eff2a7fc640 10 monclient: dump: 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.705 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 10 monclient: shutdown 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 -- 10.20.193.78:0/4138684134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c1388b0 msgr2=0x7eff2c138cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 --2- 10.20.193.78:0/4138684134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c1388b0 0x7eff2c138cd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0a5240 tx=0x7eff1c02eed0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 -- 10.20.193.78:0/4138684134 shutdown_connections 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 --2- 10.20.193.78:0/4138684134 >> v2:10.20.193.115:3300/0 conn(0x7eff2c0a3220 0x7eff2c0a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 --2- 10.20.193.78:0/4138684134 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c1388b0 0x7eff2c138cd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 --2- 10.20.193.78:0/4138684134 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1384d0 0x7eff2c070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 -- 10.20.193.78:0/4138684134 >> 10.20.193.78:0/4138684134 conn(0x7eff2c09c000 msgr2=0x7eff2c0a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 -- 10.20.193.78:0/4138684134 shutdown_connections 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 -- 10.20.193.78:0/4138684134 wait complete. 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 10 monclient: monmap: 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.706 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 Processor -- start 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 1 -- start start 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.695+0000 7eff31f7e640 10 monclient: init 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c0a3220 0x7eff2c1cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient(hunting): picked mon.noname-a con 0x7eff2c0a3220 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7eff2c1384d0 0x7eff2c1cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient(hunting): picked mon.noname-b con 0x7eff2c1384d0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1388b0 0x7eff2c124a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient(hunting): picked mon.noname-c con 0x7eff2c1388b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c0a94e0 con 0x7eff2c0a3220 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c139210 con 0x7eff2c1384d0 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c0a9660 con 0x7eff2c1388b0 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff31f7e640 10 monclient(hunting): authenticate will time out at 1606.250225s 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c0a3220 0x7eff2c1cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1388b0 0x7eff2c124a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c0a3220 0x7eff2c1cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36716/0 (socket says 10.20.193.78:36716) 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1388b0 0x7eff2c124a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51324/0 (socket says 10.20.193.78:51324) 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 -- 10.20.193.78:0/2577973037 learned_addr learned my addr 10.20.193.78:0/2577973037 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient(hunting): get_auth_request con 0x7eff2c0a3220 auth_method 0 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 10 monclient(hunting): get_auth_request con 0x7eff2c1388b0 auth_method 0 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2affd640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7eff2c1384d0 0x7eff2c1cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2affd640 10 monclient(hunting): get_auth_request con 0x7eff2c1384d0 auth_method 0 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient(hunting): handle_auth_done global_id 184317 payload 824 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1388b0 msgr2=0x7eff2c124a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c1388b0 0x7eff2c124a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7eff2c1384d0 msgr2=0x7eff2c1cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7eff2c1384d0 0x7eff2c1cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 monclient: found mon.noname-a 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.696+0000 7eff2b7fe640 1 -- 10.20.193.78:0/2577973037 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c1289a0 con 0x7eff2c0a3220 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff2b7fe640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c0a3220 msgr2=0x7eff2c1cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff2b7fe640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c0a3220 0x7eff2c1cf6a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff1800ebe0 tx=0x7eff1800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18017070 con 0x7eff2c0a3220 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: dump: 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff1800ce50 con 0x7eff2c0a3220 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180136b0 con 0x7eff2c0a3220 2026-02-17T06:02:40.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff07fff640 10 monclient: dump: 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff29ffb640 4 set_mon_vals no callback set 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff31f7e640 5 monclient: authenticate success, global_id 184317 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff31f7e640 10 monclient: _renew_subs 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff31f7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c0a5ec0 con 0x7eff2c0a3220 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff31f7e640 10 monclient: _renew_subs 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff31f7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.698+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c09fa10 con 0x7eff2c0a3220 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff31f7e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff31f7e640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c128240 0x7eff2c125710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff31f7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c069d50 con 0x7eff2c128240 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c128240 0x7eff2c125710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 10 monclient: get_auth_request con 0x7eff2c128240 auth_method 0 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 10 monclient(hunting): handle_auth_done global_id 144695 payload 824 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2affd640 1 -- 10.20.193.78:0/2577973037 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff2c127210 con 0x7eff2c128240 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7eff18024060 con 0x7eff2c0a3220 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff07fff640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efefc03deb0 0x7efefc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(594..594 src has 335..594) ==== 4763+0+0 (secure 0 0 0) 0x7eff180528e0 con 0x7eff2c0a3220 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2bfff640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efefc03deb0 0x7efefc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.699+0000 7eff2bfff640 10 monclient: get_auth_request con 0x7efefc03deb0 auth_method 0 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff2affd640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c128240 msgr2=0x7eff2c125710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff2affd640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c128240 0x7eff2c125710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0a98e0 tx=0x7eff1c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c047020 con 0x7eff2c128240 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 10 monclient: dump: 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7eff1c037c70 con 0x7eff2c128240 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c128240 msgr2=0x7eff2c125710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c128240 0x7eff2c125710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0a98e0 tx=0x7eff1c004830 comp rx=0 tx=0).stop 2026-02-17T06:02:40.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c128240 msgr2=0x7eff2c125710 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:40.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.701+0000 7eff07fff640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c128240 0x7eff2c125710 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0a98e0 tx=0x7eff1c004830 comp rx=0 tx=0).stop 2026-02-17T06:02:40.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.702+0000 7eff2bfff640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efefc03deb0 0x7efefc040370 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff20009b60 tx=0x7eff20009780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff31f7e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff31f7e640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c09c6c0 0x7eff2c1da6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff31f7e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c0744d0 con 0x7eff2c09c6c0 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c09c6c0 0x7eff2c1da6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 10 monclient: get_auth_request con 0x7eff2c09c6c0 auth_method 0 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 10 monclient(hunting): handle_auth_done global_id 144698 payload 824 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.704+0000 7eff2affd640 1 -- 10.20.193.78:0/2577973037 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7eff2c1d0120 con 0x7eff2c09c6c0 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.706+0000 7eff2affd640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c09c6c0 msgr2=0x7eff2c1da6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.706+0000 7eff2affd640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c09c6c0 0x7eff2c1da6a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7eff1c02ef10 tx=0x7eff1c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.706+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c047040 con 0x7eff2c09c6c0 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 10 monclient: dump: 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.716 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7eff1c042670 con 0x7eff2c09c6c0 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c09c6c0 msgr2=0x7eff2c1da6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c09c6c0 0x7eff2c1da6a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7eff1c02ef10 tx=0x7eff1c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c09c6c0 msgr2=0x7eff2c1da6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff07fff640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7eff2c09c6c0 0x7eff2c1da6a0 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7eff1c02ef10 tx=0x7eff1c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:40.717 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 194, 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 344242, 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.718 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.719 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:02:40.720 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.721 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.722 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:02:40.723 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:02:40.724 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.725 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.726 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:40.727 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.727 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.727 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.727 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:02:40.727 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efefc03deb0 msgr2=0x7efefc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efefc03deb0 0x7efefc040370 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff20009b60 tx=0x7eff20009780 comp rx=0 tx=0).stop 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 10 monclient: shutdown 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c0a3220 msgr2=0x7eff2c1cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c0a3220 0x7eff2c1cf6a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff1800ebe0 tx=0x7eff1800c520 comp rx=0 tx=0).stop 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff2bfff640 1 -- 10.20.193.78:0/2577973037 reap_dead start 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 shutdown_connections 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 --2- 10.20.193.78:0/2577973037 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7eff2c0a3220 0x7eff2c1cf6a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 >> 10.20.193.78:0/2577973037 conn(0x7eff2c09c000 msgr2=0x7eff2c0728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 shutdown_connections 2026-02-17T06:02:40.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.707+0000 7eff31f7e640 1 -- 10.20.193.78:0/2577973037 wait complete. 2026-02-17T06:02:40.728 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:02:40.728 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:02:40.729 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:02:40.729 INFO:tasks.rados.rados.0.trial115.stdout:2791: done (1 left) 2026-02-17T06:02:40.729 INFO:tasks.rados.rados.0.trial115.stdout:2792: rollback oid 12 current snap is 282 2026-02-17T06:02:40.729 INFO:tasks.rados.rados.0.trial115.stdout:2792: done (1 left) 2026-02-17T06:02:40.729 INFO:tasks.rados.rados.0.trial115.stdout:2793: write oid 9 current snap is 282 2026-02-17T06:02:40.729 INFO:tasks.rados.rados.0.trial115.stdout:2793: seq_num 982 ranges {504474=504055,1714141=631006} 2026-02-17T06:02:40.733 INFO:tasks.rados.rados.0.trial115.stdout:2793: writing trial11545420-9 from 504474 to 1008529 tid 1 2026-02-17T06:02:40.739 INFO:tasks.rados.rados.0.trial115.stdout:2793: writing trial11545420-9 from 1714141 to 2345147 tid 2 2026-02-17T06:02:40.739 INFO:tasks.rados.rados.0.trial115.stdout:2794: write (excl) oid 30 current snap is 282 2026-02-17T06:02:40.740 INFO:tasks.rados.rados.0.trial115.stdout:2794: seq_num 983 ranges {521281=579304,1656270=774664} 2026-02-17T06:02:40.742 INFO:tasks.rados.rados.0.trial115.stdout:2794: writing trial11545420-30 from 521281 to 1100585 tid 1 2026-02-17T06:02:40.748 INFO:tasks.rados.rados.0.trial115.stdout:2794: writing trial11545420-30 from 1656270 to 2430934 tid 2 2026-02-17T06:02:40.748 INFO:tasks.rados.rados.0.trial115.stdout:2795: delete oid 47 current snap is 282 2026-02-17T06:02:40.748 INFO:tasks.rados.rados.0.trial115.stdout:2789: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:02:40.794 INFO:tasks.rados.rados.0.trial115.stdout:2793: finishing write tid 1 to trial11545420-9 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2793: finishing write tid 2 to trial11545420-9 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2793: finishing write tid 3 to trial11545420-9 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2793: finishing write tid 4 to trial11545420-9 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2793: oid 9 updating version 0 to 1420 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2793: oid 9 updating version 1420 to 1421 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2793: oid 9 version 1421 is already newer than 1419 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1421 (ObjNum 982 snap 282 seq_num 982) dirty exists 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2793: left oid 9 (ObjNum 982 snap 282 seq_num 982) 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2794: finishing write tid 1 to trial11545420-30 2026-02-17T06:02:40.795 INFO:tasks.rados.rados.0.trial115.stdout:2794: finishing write tid 2 to trial11545420-30 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2794: finishing write tid 3 to trial11545420-30 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2794: finishing write tid 4 to trial11545420-30 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2794: oid 30 updating version 0 to 1185 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2794: oid 30 updating version 1185 to 1186 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2794: oid 30 version 1186 is already newer than 1184 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1186 (ObjNum 983 snap 282 seq_num 983) dirty exists 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2794: left oid 30 (ObjNum 983 snap 282 seq_num 983) 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2789: done (3 left) 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2793: done (2 left) 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2794: done (1 left) 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2795: done (0 left) 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2796: copy_from oid 13 from oid 12 current snap is 282 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2797: write (excl) oid 40 current snap is 282 2026-02-17T06:02:40.796 INFO:tasks.rados.rados.0.trial115.stdout:2797: seq_num 985 ranges {554895=729802,1773234=778333} 2026-02-17T06:02:40.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.797+0000 7fb6f564d640 10 monclient: get_monmap_and_config 2026-02-17T06:02:40.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.797+0000 7fb6f564d640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.797+0000 7fb6f564d640 10 monclient: monmap: 2026-02-17T06:02:40.798 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.798 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:40.798345+0000 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:40.798345+0000 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:40.799 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:40.800 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.800 INFO:tasks.rados.rados.0.trial115.stdout:2797: writing trial11545420-40 from 554895 to 1284697 tid 1 2026-02-17T06:02:40.806 INFO:tasks.rados.rados.0.trial115.stdout:2797: writing trial11545420-40 from 1773234 to 2551567 tid 2 2026-02-17T06:02:40.806 INFO:tasks.rados.rados.0.trial115.stdout:2798: write (excl) oid 33 current snap is 282 2026-02-17T06:02:40.806 INFO:tasks.rados.rados.0.trial115.stdout:2796: finishing copy_from racing read to trial11545420-13 2026-02-17T06:02:40.806 INFO:tasks.rados.rados.0.trial115.stdout:2796: finishing copy_from to trial11545420-13 2026-02-17T06:02:40.807 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1457 (ObjNum 910 snap 264 seq_num 910) dirty exists 2026-02-17T06:02:40.807 INFO:tasks.rados.rados.0.trial115.stdout:2797: finishing write tid 1 to trial11545420-40 2026-02-17T06:02:40.807 INFO:tasks.rados.rados.0.trial115.stdout:2798: seq_num 986 ranges {571702=405051,1431716=638344,2560365=11337} 2026-02-17T06:02:40.811 INFO:tasks.rados.rados.0.trial115.stdout:2798: writing trial11545420-33 from 571702 to 976753 tid 1 2026-02-17T06:02:40.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.797+0000 7fb6f564d640 1 Processor -- start 2026-02-17T06:02:40.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.797+0000 7fb6f564d640 1 -- start start 2026-02-17T06:02:40.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.797+0000 7fb6f564d640 10 monclient: init 2026-02-17T06:02:40.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2026-02-17T06:02:40.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6f006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:40.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb6f009c4a0 0x7fb6f013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient(hunting): picked mon.noname-c con 0x7fb6f009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb6f006d380 0x7fb6f013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient(hunting): picked mon.noname-a con 0x7fb6f006d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f006daf0 con 0x7fb6f006c890 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f00a0880 con 0x7fb6f006d380 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f009cc80 con 0x7fb6f009c4a0 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6f564d640 10 monclient(hunting): authenticate will time out at 1606.352358s 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb6f006d380 0x7fb6f013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb6f006d380 0x7fb6f013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36720/0 (socket says 10.20.193.78:36720) 2026-02-17T06:02:40.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/848235850 learned_addr learned my addr 10.20.193.78:0/848235850 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient(hunting): get_auth_request con 0x7fb6f006d380 auth_method 0 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ee7fc640 1 --2- 10.20.193.78:0/848235850 >> v2:10.20.193.115:3300/0 conn(0x7fb6f009c4a0 0x7fb6f013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ee7fc640 10 monclient(hunting): get_auth_request con 0x7fb6f009c4a0 auth_method 0 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient(hunting): handle_auth_done global_id 184320 payload 274 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/848235850 >> v2:10.20.193.115:3300/0 conn(0x7fb6f009c4a0 msgr2=0x7fb6f013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 1 --2- 10.20.193.78:0/848235850 >> v2:10.20.193.115:3300/0 conn(0x7fb6f009c4a0 0x7fb6f013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/848235850 >> v1:10.20.193.115:6789/0 conn(0x7fb6f006c890 legacy=0x7fb6f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 1 monclient: found mon.noname-a 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.798+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/848235850 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f009cf60 con 0x7fb6f006d380 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/848235850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006d380 msgr2=0x7fb6f013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6ef7fe640 1 --2- 10.20.193.78:0/848235850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006d380 0x7fb6f013d8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6f006dcc0 tx=0x7fb6e002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4577c4425ff50b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 1 -- 10.20.193.78:0/848235850 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e002fe50 con 0x7fb6f006d380 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: dump: 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 1 -- 10.20.193.78:0/848235850 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e00373f0 con 0x7fb6f006d380 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 5 monclient: authenticate success, global_id 184320 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 1 -- 10.20.193.78:0/848235850 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00377c0 con 0x7fb6f006d380 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 4 set_mon_vals no callback set 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6edffb640 10 monclient: dump: 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.817 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 10 monclient: shutdown 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 1 -- 10.20.193.78:0/848235850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006d380 msgr2=0x7fb6f013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 1 --2- 10.20.193.78:0/848235850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006d380 0x7fb6f013d8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6f006dcc0 tx=0x7fb6e002f1e0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 1 -- 10.20.193.78:0/848235850 shutdown_connections 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 1 --2- 10.20.193.78:0/848235850 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006d380 0x7fb6f013d8f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 1 --2- 10.20.193.78:0/848235850 >> v2:10.20.193.115:3300/0 conn(0x7fb6f009c4a0 0x7fb6f013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 1 -- 10.20.193.78:0/848235850 >> 10.20.193.78:0/848235850 conn(0x7fb6f00a6620 msgr2=0x7fb6f00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 1 -- 10.20.193.78:0/848235850 shutdown_connections 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.801+0000 7fb6f564d640 1 -- 10.20.193.78:0/848235850 wait complete. 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient: monmap: 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:40.819 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 1 Processor -- start 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 1 -- start start 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient: init 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006c890 0x7fb6f013ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient(hunting): picked mon.noname-a con 0x7fb6f006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:40.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f006d380 0x7fb6f009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient(hunting): picked mon.noname-c con 0x7fb6f006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009c4a0 0x7fb6f009e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient(hunting): picked mon.noname-b con 0x7fb6f009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f009bfa0 con 0x7fb6f006c890 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f009be20 con 0x7fb6f009c4a0 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f013de30 con 0x7fb6f006d380 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6f564d640 10 monclient(hunting): authenticate will time out at 1606.356837s 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006c890 0x7fb6f013ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006c890 0x7fb6f013ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36736/0 (socket says 10.20.193.78:36736) 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 1 -- 10.20.193.78:0/1962281851 learned_addr learned my addr 10.20.193.78:0/1962281851 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 10 monclient(hunting): get_auth_request con 0x7fb6f006c890 auth_method 0 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6ee7fc640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f006d380 0x7fb6f009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6ee7fc640 10 monclient(hunting): get_auth_request con 0x7fb6f006d380 auth_method 0 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.802+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 10 monclient(hunting): handle_auth_done global_id 184323 payload 824 2026-02-17T06:02:40.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f006d380 msgr2=0x7fb6f009db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb6f006d380 0x7fb6f009db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6ef7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009c4a0 0x7fb6f009e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009c4a0 msgr2=0x7fb6f009e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009c4a0 0x7fb6f009e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 1 monclient: found mon.noname-a 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6eeffd640 1 -- 10.20.193.78:0/1962281851 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f009f7a0 con 0x7fb6f006c890 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6ef7fe640 10 monclient: get_auth_request con 0x7fb6f009c4a0 auth_method 0 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.803+0000 7fb6ef7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009c4a0 0x7fb6f009e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6eeffd640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006c890 msgr2=0x7fb6f013ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6eeffd640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006c890 0x7fb6f013ca50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400c600 tx=0x7fb6e400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4012a40 con 0x7fb6f006c890 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: dump: 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 5 monclient: authenticate success, global_id 184323 2026-02-17T06:02:40.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e4017070 con 0x7fb6f006c890 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 10 monclient: _renew_subs 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f013afd0 con 0x7fb6f006c890 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4012e20 con 0x7fb6f006c890 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6ed7fa640 4 set_mon_vals no callback set 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6cf7fe640 10 monclient: dump: 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.825 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 10 monclient: _renew_subs 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f009d810 con 0x7fb6f006c890 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009e5d0 0x7fb6f00716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.805+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f009e9b0 con 0x7fb6f009e5d0 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009e5d0 0x7fb6f00716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 10 monclient: get_auth_request con 0x7fb6f009e5d0 auth_method 0 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb6e401c460 con 0x7fb6f006c890 2026-02-17T06:02:40.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6cf7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6bc03deb0 0x7fb6bc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(595..595 src has 335..595) ==== 4763+0+0 (secure 0 0 0) 0x7fb6e4052710 con 0x7fb6f006c890 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_done global_id 204142 payload 824 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ee7fc640 1 -- 10.20.193.78:0/1962281851 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6f0074980 con 0x7fb6f009e5d0 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ef7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6bc03deb0 0x7fb6bc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.806+0000 7fb6ef7fe640 10 monclient: get_auth_request con 0x7fb6bc03deb0 auth_method 0 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6ee7fc640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009e5d0 msgr2=0x7fb6f00716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6ee7fc640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009e5d0 0x7fb6f00716f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb6f009f1c0 tx=0x7fb6d8011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8019070 con 0x7fb6f009e5d0 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 10 monclient: dump: 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.827 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fb6d800fe20 con 0x7fb6f009e5d0 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009e5d0 msgr2=0x7fb6f00716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009e5d0 0x7fb6f00716f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb6f009f1c0 tx=0x7fb6d8011ca0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009e5d0 msgr2=0x7fb6f00716f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6cf7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f009e5d0 0x7fb6f00716f0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb6f009f1c0 tx=0x7fb6d8011ca0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.809+0000 7fb6ef7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6bc03deb0 0x7fb6bc040370 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb6f006a480 tx=0x7fb6e0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6f564d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6f564d640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f0071c30 0x7fb6f0072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6f564d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00744d0 con 0x7fb6f0071c30 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f0071c30 0x7fb6f0072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 10 monclient: get_auth_request con 0x7fb6f0071c30 auth_method 0 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 10 monclient(hunting): handle_auth_done global_id 204145 payload 824 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.812+0000 7fb6ee7fc640 1 -- 10.20.193.78:0/1962281851 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb6f013a7b0 con 0x7fb6f0071c30 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6ee7fc640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f0071c30 msgr2=0x7fb6f0072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6ee7fc640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f0071c30 0x7fb6f0072040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb6f009f1c0 tx=0x7fb6d8004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800f040 con 0x7fb6f0071c30 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 10 monclient: dump: 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7fb6d8015e50 con 0x7fb6f0071c30 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f0071c30 msgr2=0x7fb6f0072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f0071c30 0x7fb6f0072040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb6f009f1c0 tx=0x7fb6d8004830 comp rx=0 tx=0).stop 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f0071c30 msgr2=0x7fb6f0072040 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6cf7fe640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb6f0071c30 0x7fb6f0072040 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb6f009f1c0 tx=0x7fb6d8004830 comp rx=0 tx=0).stop 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 194, 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 22102, 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:02:40.834 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:02:40.835 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:40.836 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.837 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.838 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:02:40.839 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.840 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:40.841 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:02:40.842 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:02:40.842 INFO:tasks.rados.rados.0.trial115.stdout:2798: writing trial11545420-33 from 1431716 to 2070060 tid 2 2026-02-17T06:02:40.842 INFO:tasks.rados.rados.0.trial115.stdout:2798: writing trial11545420-33 from 2560365 to 2571702 tid 3 2026-02-17T06:02:40.842 INFO:tasks.rados.rados.0.trial115.stdout:2796: done (2 left) 2026-02-17T06:02:40.842 INFO:tasks.rados.rados.0.trial115.stdout:2799: delete oid 48 current snap is 282 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2797: finishing write tid 2 to trial11545420-40 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2797: finishing write tid 3 to trial11545420-40 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2797: finishing write tid 4 to trial11545420-40 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2797: oid 40 updating version 0 to 1404 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2797: oid 40 version 1404 is already newer than 1402 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2797: oid 40 version 1404 is already newer than 1403 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1404 (ObjNum 985 snap 282 seq_num 985) dirty exists 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2797: left oid 40 (ObjNum 985 snap 282 seq_num 985) 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: finishing write tid 1 to trial11545420-33 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: finishing write tid 2 to trial11545420-33 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: finishing write tid 3 to trial11545420-33 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: finishing write tid 4 to trial11545420-33 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: finishing write tid 5 to trial11545420-33 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: oid 33 updating version 0 to 1436 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: oid 33 version 1436 is already newer than 1434 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: oid 33 version 1436 is already newer than 1435 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: oid 33 version 1436 is already newer than 1433 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1436 (ObjNum 986 snap 282 seq_num 986) dirty exists 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: left oid 33 (ObjNum 986 snap 282 seq_num 986) 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2797: done (2 left) 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2798: done (1 left) 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2799: done (0 left) 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2800: rollback oid 23 current snap is 282 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2800: done (0 left) 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2801: copy_from oid 10 from oid 23 current snap is 282 2026-02-17T06:02:40.843 INFO:tasks.rados.rados.0.trial115.stdout:2802: delete oid 32 current snap is 282 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2802: done (1 left) 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2803: read oid 19 snap -1 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2803: expect deleted 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: write oid 48 current snap is 282 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: seq_num 989 ranges {622123=747151,2007162=614961} 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: writing trial11545420-48 from 622123 to 1369274 tid 1 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: writing trial11545420-48 from 2007162 to 2622123 tid 2 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2805: snap_create 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2801: finishing copy_from racing read to trial11545420-10 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: finishing write tid 1 to trial11545420-48 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2801: finishing copy_from to trial11545420-10 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1398 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: finishing write tid 2 to trial11545420-48 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: finishing write tid 3 to trial11545420-48 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: finishing write tid 4 to trial11545420-48 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: oid 48 updating version 0 to 1515 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: oid 48 updating version 1515 to 1516 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: oid 48 version 1516 is already newer than 1514 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1516 (ObjNum 989 snap 282 seq_num 989) dirty exists 2026-02-17T06:02:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2804: left oid 48 (ObjNum 989 snap 282 seq_num 989) 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6bc03deb0 msgr2=0x7fb6bc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb6bc03deb0 0x7fb6bc040370 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb6f006a480 tx=0x7fb6e0033000 comp rx=0 tx=0).stop 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 10 monclient: shutdown 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006c890 msgr2=0x7fb6f013ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 1 --2- 10.20.193.78:0/1962281851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb6f006c890 0x7fb6f013ca50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400c600 tx=0x7fb6e400cad0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/1962281851 reap_dead start 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6ef7fe640 1 -- 10.20.193.78:0/1962281851 reap_dead start 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 shutdown_connections 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 >> 10.20.193.78:0/1962281851 conn(0x7fb6f00a6620 msgr2=0x7fb6f0136190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 shutdown_connections 2026-02-17T06:02:40.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.815+0000 7fb6f564d640 1 -- 10.20.193.78:0/1962281851 wait complete. 2026-02-17T06:02:40.845 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:02:40.846 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T06:02:40.846 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:02:40.846 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T06:02:40.846 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:02:40.846 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T06:02:40.846 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:02:40.854 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T06:02:40.855 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:02:40.855 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:02:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.919+0000 7fc35c264640 10 monclient: get_monmap_and_config 2026-02-17T06:02:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.919+0000 7fc35c264640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.919+0000 7fc35c264640 10 monclient: monmap: 2026-02-17T06:02:40.922 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.922 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.922 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:40.920679+0000 2026-02-17T06:02:40.922 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:40.920679+0000 2026-02-17T06:02:40.922 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.919+0000 7fc35c264640 1 Processor -- start 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.919+0000 7fc35c264640 1 -- start start 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.919+0000 7fc35c264640 10 monclient: init 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc35406c890 0x7fc35406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient(hunting): picked mon.noname-a con 0x7fc35406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc35409d430 0x7fc35406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient(hunting): picked mon.noname-c con 0x7fc35409d430 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc35409bf40 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc354073600 con 0x7fc35409bf40 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc35409ca50 con 0x7fc35406c890 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc35406d6f0 con 0x7fc35409d430 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35c264640 10 monclient(hunting): authenticate will time out at 1606.474194s 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc3597d8640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fc35409d430 msgr2=0x7fc35406d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc35a7da640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fc35409bf40 legacy=0x7fc354134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc35406c890 0x7fc35406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc3597d8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc35409d430 0x7fc35406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc35406c890 0x7fc35406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36746/0 (socket says 10.20.193.78:36746) 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 1 -- 10.20.193.78:0/2252858929 learned_addr learned my addr 10.20.193.78:0/2252858929 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient(hunting): get_auth_request con 0x7fc35406c890 auth_method 0 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient(hunting): handle_auth_done global_id 184326 payload 274 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 1 -- 10.20.193.78:0/2252858929 >> v2:10.20.193.115:3300/0 conn(0x7fc35409d430 msgr2=0x7fc35406d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 1 --2- 10.20.193.78:0/2252858929 >> v2:10.20.193.115:3300/0 conn(0x7fc35409d430 0x7fc35406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 1 -- 10.20.193.78:0/2252858929 >> v1:10.20.193.115:6789/0 conn(0x7fc35409bf40 legacy=0x7fc354134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 1 monclient: found mon.noname-a 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.920+0000 7fc359fd9640 1 -- 10.20.193.78:0/2252858929 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35406dab0 con 0x7fc35406c890 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.922+0000 7fc359fd9640 1 -- 10.20.193.78:0/2252858929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 msgr2=0x7fc35406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc359fd9640 1 --2- 10.20.193.78:0/2252858929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc35406cc70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc3540a0ac0 tx=0x7fc34402ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad7504fb05abf9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 1 -- 10.20.193.78:0/2252858929 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34403c070 con 0x7fc35406c890 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: dump: 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.929 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 5 monclient: authenticate success, global_id 184326 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 1 -- 10.20.193.78:0/2252858929 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34402fda0 con 0x7fc35406c890 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 1 -- 10.20.193.78:0/2252858929 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344037660 con 0x7fc35406c890 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc358fd7640 10 monclient: dump: 2026-02-17T06:02:40.930 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 4 set_mon_vals no callback set 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 10 monclient: shutdown 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 -- 10.20.193.78:0/2252858929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 msgr2=0x7fc35406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 --2- 10.20.193.78:0/2252858929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc35406cc70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc3540a0ac0 tx=0x7fc34402ee80 comp rx=0 tx=0).stop 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 -- 10.20.193.78:0/2252858929 shutdown_connections 2026-02-17T06:02:40.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 --2- 10.20.193.78:0/2252858929 >> v2:10.20.193.115:3300/0 conn(0x7fc35409d430 0x7fc35406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 --2- 10.20.193.78:0/2252858929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc35406cc70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 -- 10.20.193.78:0/2252858929 >> 10.20.193.78:0/2252858929 conn(0x7fc3540a6620 msgr2=0x7fc3540a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 -- 10.20.193.78:0/2252858929 shutdown_connections 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 -- 10.20.193.78:0/2252858929 wait complete. 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 10 monclient: build_initial_monmap 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 10 monclient: monmap: 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.923+0000 7fc35c264640 1 Processor -- start 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 1 -- start start 2026-02-17T06:02:40.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient: init 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc3541cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient(hunting): picked mon.noname-a con 0x7fc35406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc35409bf40 0x7fc3541cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient(hunting): picked mon.noname-c con 0x7fc35409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc35409d430 0x7fc354121ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient(hunting): picked mon.noname-b con 0x7fc35409d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc35409c800 con 0x7fc35406c890 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc354073410 con 0x7fc35409d430 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc354135500 con 0x7fc35409bf40 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35c264640 10 monclient(hunting): authenticate will time out at 1606.478306s 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc3541cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc3541cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36756/0 (socket says 10.20.193.78:36756) 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 -- 10.20.193.78:0/801807170 learned_addr learned my addr 10.20.193.78:0/801807170 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient(hunting): get_auth_request con 0x7fc35406c890 auth_method 0 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35a7da640 1 -- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc35409d430 msgr2=0x7fc354121ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc35a7da640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc35409d430 0x7fc354121ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc3597d8640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc35409bf40 0x7fc3541cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc3597d8640 10 monclient(hunting): get_auth_request con 0x7fc35409bf40 auth_method 0 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc3597d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc3597d8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc3597d8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient(hunting): handle_auth_done global_id 184329 payload 824 2026-02-17T06:02:40.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 -- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc35409bf40 msgr2=0x7fc3541cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc35409bf40 0x7fc3541cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 -- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc35409d430 msgr2=0x7fc354121ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc35409d430 0x7fc354121ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient: _finish_hunting 0 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 monclient: found mon.noname-a 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.924+0000 7fc359fd9640 1 -- 10.20.193.78:0/801807170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc354125480 con 0x7fc35406c890 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc359fd9640 1 -- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 msgr2=0x7fc3541cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc359fd9640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc3541cf560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc344002410 tx=0x7fc3440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 1 -- 10.20.193.78:0/801807170 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344046020 con 0x7fc35406c890 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: dump: 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.935 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc35c264640 5 monclient: authenticate success, global_id 184329 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 1 -- 10.20.193.78:0/801807170 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc344042640 con 0x7fc35406c890 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc35c264640 10 monclient: _renew_subs 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc35c264640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc354125760 con 0x7fc35406c890 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 1 -- 10.20.193.78:0/801807170 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34403c040 con 0x7fc35406c890 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc343fff640 4 set_mon_vals no callback set 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc3427fc640 10 monclient: dump: 2026-02-17T06:02:40.936 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc35c264640 10 monclient: _renew_subs 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc35c264640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.926+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3540a1ab0 con 0x7fc35406c890 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.927+0000 7fc35c264640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.927+0000 7fc35c264640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.927+0000 7fc35c264640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.927+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3541358f0 con 0x7fc35406c890 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.927+0000 7fc3427fc640 1 -- 10.20.193.78:0/801807170 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc34403f030 con 0x7fc35406c890 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.927+0000 7fc3427fc640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc32403deb0 0x7fc324040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.928+0000 7fc3427fc640 1 -- 10.20.193.78:0/801807170 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(595..595 src has 335..595) ==== 4763+0+0 (secure 0 0 0) 0x7fc3440777d0 con 0x7fc35406c890 2026-02-17T06:02:40.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.928+0000 7fc3597d8640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc32403deb0 0x7fc324040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:40.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.928+0000 7fc3597d8640 10 monclient: get_auth_request con 0x7fc32403deb0 auth_method 0 2026-02-17T06:02:40.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.930+0000 7fc3427fc640 1 -- 10.20.193.78:0/801807170 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc344050070 con 0x7fc35406c890 2026-02-17T06:02:40.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.930+0000 7fc3427fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:40.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.930+0000 7fc3427fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:40.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:40.930+0000 7fc3597d8640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc32403deb0 0x7fc324040370 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc354125690 tx=0x7fc348003ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:41.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.043+0000 7fc35c264640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:02:41.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.043+0000 7fc35c264640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:41.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.043+0000 7fc35c264640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:41.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.043+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc3540744d0 con 0x7fc35406c890 2026-02-17T06:02:41.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.043+0000 7fc3427fc640 1 -- 10.20.193.78:0/801807170 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fc34403f350 con 0x7fc35406c890 2026-02-17T06:02:41.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.043+0000 7fc3427fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:41.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.043+0000 7fc3427fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:41.046 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:02:41.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc32403deb0 msgr2=0x7fc324040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:41.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc32403deb0 0x7fc324040370 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc354125690 tx=0x7fc348003ab0 comp rx=0 tx=0).stop 2026-02-17T06:02:41.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 10 monclient: shutdown 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 msgr2=0x7fc3541cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc3541cf560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc344002410 tx=0x7fc3440047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 shutdown_connections 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc32403deb0 0x7fc324040370 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc35409d430 0x7fc354121ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc35409bf40 0x7fc3541cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 --2- 10.20.193.78:0/801807170 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc35406c890 0x7fc3541cf560 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 >> 10.20.193.78:0/801807170 conn(0x7fc3540a6620 msgr2=0x7fc354136900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 shutdown_connections 2026-02-17T06:02:41.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:41.044+0000 7fc35c264640 1 -- 10.20.193.78:0/801807170 wait complete. 2026-02-17T06:02:44.054 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.122+0000 7f5be713e640 10 monclient: get_monmap_and_config 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.122+0000 7f5be713e640 10 monclient: build_initial_monmap 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.122+0000 7f5be713e640 10 monclient: monmap: 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:44.123859+0000 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:44.123859+0000 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:44.123 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:44.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 1 Processor -- start 2026-02-17T06:02:44.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 1 -- start start 2026-02-17T06:02:44.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient: init 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5be00a3220 0x7f5be00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient(hunting): picked mon.noname-c con 0x7f5be00a3220 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00704c0 0x7f5be00a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient(hunting): picked mon.noname-b con 0x7f5be00704c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5be013c4f0 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be0074000 con 0x7f5be013c4f0 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5be0070130 con 0x7f5be00a3220 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5be00a4080 con 0x7f5be00704c0 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be713e640 10 monclient(hunting): authenticate will time out at 1609.677448s 2026-02-17T06:02:44.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be4eb3640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f5be00a3220 msgr2=0x7f5be00a3600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be56b4640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f5be013c4f0 legacy=0x7f5be013e8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00704c0 0x7f5be00a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5be4eb3640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5be00a3220 0x7f5be00a3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00704c0 0x7f5be00a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51358/0 (socket says 10.20.193.78:51358) 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 1 -- 10.20.193.78:0/18220036 learned_addr learned my addr 10.20.193.78:0/18220036 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient(hunting): get_auth_request con 0x7f5be00704c0 auth_method 0 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient(hunting): handle_auth_done global_id 144707 payload 274 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 1 -- 10.20.193.78:0/18220036 >> v2:10.20.193.115:3300/0 conn(0x7f5be00a3220 msgr2=0x7f5be00a3600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 1 --2- 10.20.193.78:0/18220036 >> v2:10.20.193.115:3300/0 conn(0x7f5be00a3220 0x7f5be00a3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 1 -- 10.20.193.78:0/18220036 >> v1:10.20.193.115:6789/0 conn(0x7f5be013c4f0 legacy=0x7f5be013e8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 1 monclient: found mon.noname-b 2026-02-17T06:02:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:02:44.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.123+0000 7f5bd7fff640 1 -- 10.20.193.78:0/18220036 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be00a4af0 con 0x7f5be00704c0 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd7fff640 1 -- 10.20.193.78:0/18220036 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00704c0 msgr2=0x7f5be00a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd7fff640 1 --2- 10.20.193.78:0/18220036 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00704c0 0x7f5be00a3b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80099a0 tx=0x7f5bc802eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dac6bc126fb345bf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 1 -- 10.20.193.78:0/18220036 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc803c070 con 0x7f5be00704c0 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: dump: 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:44.132 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: _finish_auth 0 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 1 -- 10.20.193.78:0/18220036 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc802fdd0 con 0x7f5be00704c0 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 5 monclient: authenticate success, global_id 144707 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 1 -- 10.20.193.78:0/18220036 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8037660 con 0x7f5be00704c0 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 4 set_mon_vals no callback set 2026-02-17T06:02:44.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5bd77fe640 10 monclient: dump: 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 10 monclient: shutdown 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 1 -- 10.20.193.78:0/18220036 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00704c0 msgr2=0x7f5be00a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 1 --2- 10.20.193.78:0/18220036 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00704c0 0x7f5be00a3b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80099a0 tx=0x7f5bc802eeb0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 1 -- 10.20.193.78:0/18220036 shutdown_connections 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 1 --2- 10.20.193.78:0/18220036 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00704c0 0x7f5be00a3b40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 1 --2- 10.20.193.78:0/18220036 >> v2:10.20.193.115:3300/0 conn(0x7f5be00a3220 0x7f5be00a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 1 -- 10.20.193.78:0/18220036 >> 10.20.193.78:0/18220036 conn(0x7f5be009c000 msgr2=0x7f5be00a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 1 -- 10.20.193.78:0/18220036 shutdown_connections 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.126+0000 7f5be713e640 1 -- 10.20.193.78:0/18220036 wait complete. 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient: build_initial_monmap 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient: monmap: 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 1 Processor -- start 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 1 -- start start 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient: init 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:02:44.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5be00704c0 0x7f5be00a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient(hunting): picked mon.noname-b con 0x7f5be00704c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00a3220 0x7f5be013b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient(hunting): picked mon.noname-c con 0x7f5be00a3220 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5be013c4f0 0x7f5be01cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient(hunting): picked mon.noname-a con 0x7f5be013c4f0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5be00a9680 con 0x7f5be013c4f0 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5be00a9500 con 0x7f5be00704c0 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5be00a4f70 con 0x7f5be00a3220 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be713e640 10 monclient(hunting): authenticate will time out at 1609.681688s 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be4eb3640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5be00704c0 msgr2=0x7f5be00a1cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00a3220 0x7f5be013b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be4eb3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5be00704c0 0x7f5be00a1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5be013c4f0 0x7f5be01cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00a3220 0x7f5be013b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51368/0 (socket says 10.20.193.78:51368) 2026-02-17T06:02:44.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 -- 10.20.193.78:0/846448929 learned_addr learned my addr 10.20.193.78:0/846448929 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient(hunting): get_auth_request con 0x7f5be00a3220 auth_method 0 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 10 monclient(hunting): get_auth_request con 0x7f5be013c4f0 auth_method 0 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient(hunting): handle_auth_done global_id 144710 payload 824 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 -- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5be00704c0 msgr2=0x7f5be00a1cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5be00704c0 0x7f5be00a1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 -- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5be013c4f0 msgr2=0x7f5be01cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5be013c4f0 0x7f5be01cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 monclient: found mon.noname-c 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5bd7fff640 1 -- 10.20.193.78:0/846448929 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be00a28a0 con 0x7f5be00a3220 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.127+0000 7f5be56b4640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5be013c4f0 0x7f5be01cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd7fff640 1 -- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00a3220 msgr2=0x7f5be013b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd7fff640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00a3220 0x7f5be013b7d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80099a0 tx=0x7f5bc80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 1 -- 10.20.193.78:0/846448929 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8046020 con 0x7f5be00a3220 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: dump: 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:44.138 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: _finish_auth 0 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: _check_auth_tickets 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 1 -- 10.20.193.78:0/846448929 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc8042620 con 0x7f5be00a3220 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 5 monclient: authenticate success, global_id 144710 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 10 monclient: _renew_subs 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be0123840 con 0x7f5be00a3220 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 1 -- 10.20.193.78:0/846448929 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc803c040 con 0x7f5be00a3220 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd6ffd640 4 set_mon_vals no callback set 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 10 monclient: dump: 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:44.139 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 10 monclient: _renew_subs 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be01258c0 con 0x7f5be00a3220 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5bd57fa640 1 -- 10.20.193.78:0/846448929 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5bc803f030 con 0x7f5be00a3220 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.130+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be013f1a0 con 0x7f5be00a3220 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.131+0000 7f5bd57fa640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bb003de90 0x7f5bb0040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.131+0000 7f5bd57fa640 1 -- 10.20.193.78:0/846448929 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(595..595 src has 335..595) ==== 4763+0+0 (secure 0 0 0) 0x7f5bc80775d0 con 0x7f5be00a3220 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.131+0000 7f5be4eb3640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bb003de90 0x7f5bb0040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.131+0000 7f5be4eb3640 10 monclient: get_auth_request con 0x7f5bb003de90 auth_method 0 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.133+0000 7f5bd57fa640 1 -- 10.20.193.78:0/846448929 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5be013f1a0 con 0x7f5be00a3220 2026-02-17T06:02:44.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.133+0000 7f5bd57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:44.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.133+0000 7f5bd57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:44.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.134+0000 7f5be4eb3640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bb003de90 0x7f5bb0040350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5be00a2540 tx=0x7f5bd0009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:44.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.254+0000 7f5be713e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:02:44.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.254+0000 7f5be713e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:44.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.254+0000 7f5be713e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.254+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5be00a3600 con 0x7f5be00a3220 2026-02-17T06:02:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.254+0000 7f5bd57fa640 1 -- 10.20.193.78:0/846448929 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f5bc8050070 con 0x7f5be00a3220 2026-02-17T06:02:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.254+0000 7f5bd57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:44.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.254+0000 7f5bd57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:44.256 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:02:44.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bb003de90 msgr2=0x7f5bb0040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:44.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bb003de90 0x7f5bb0040350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5be00a2540 tx=0x7f5bd0009270 comp rx=0 tx=0).stop 2026-02-17T06:02:44.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 10 monclient: shutdown 2026-02-17T06:02:44.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00a3220 msgr2=0x7f5be013b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00a3220 0x7f5be013b7d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80099a0 tx=0x7f5bc80047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 shutdown_connections 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5be013c4f0 0x7f5be01cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5be00a3220 0x7f5be013b7d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5bb003de90 0x7f5bb0040350 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 --2- 10.20.193.78:0/846448929 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5be00704c0 0x7f5be00a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 >> 10.20.193.78:0/846448929 conn(0x7f5be009c000 msgr2=0x7f5be013ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 shutdown_connections 2026-02-17T06:02:44.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:44.255+0000 7f5be713e640 1 -- 10.20.193.78:0/846448929 wait complete. 2026-02-17T06:02:47.265 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.332+0000 7f273cae8640 10 monclient: get_monmap_and_config 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.332+0000 7f273cae8640 10 monclient: build_initial_monmap 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.332+0000 7f273cae8640 10 monclient: monmap: 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:47.334107+0000 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:47.334107+0000 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:47.333 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:47.334 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:47.334 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:47.334 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:47.334 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:47.334 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 1 Processor -- start 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 1 -- start start 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient: init 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f273806c890 0x7f27380a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient(hunting): picked mon.noname-a con 0x7f273806c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f273806cc70 0x7f273806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient(hunting): picked mon.noname-b con 0x7f273806cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2738134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2738074150 con 0x7f2738134df0 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f273809ef70 con 0x7f273806c890 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f273809f250 con 0x7f273806cc70 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f273cae8640 10 monclient(hunting): authenticate will time out at 1612.887851s 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f273806cc70 0x7f273806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2736575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f273806c890 0x7f27380a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f273806cc70 0x7f273806d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44088/0 (socket says 10.20.193.78:44088) 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 1 -- 10.20.193.78:0/4080733088 learned_addr learned my addr 10.20.193.78:0/4080733088 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 10 monclient(hunting): get_auth_request con 0x7f273806cc70 auth_method 0 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2736575640 10 monclient(hunting): get_auth_request con 0x7f273806c890 auth_method 0 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2736575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2736575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2736575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.333+0000 7f2735d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2736575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2736575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2736575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 10 monclient(hunting): handle_auth_done global_id 144713 payload 274 2026-02-17T06:02:47.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 1 -- 10.20.193.78:0/4080733088 >> v2:10.20.193.78:3300/0 conn(0x7f273806c890 msgr2=0x7f27380a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:47.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 1 --2- 10.20.193.78:0/4080733088 >> v2:10.20.193.78:3300/0 conn(0x7f273806c890 0x7f27380a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 1 -- 10.20.193.78:0/4080733088 >> v1:10.20.193.78:6789/0 conn(0x7f2738134df0 legacy=0x7f2738135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:47.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 10 monclient: _finish_hunting 0 2026-02-17T06:02:47.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 1 monclient: found mon.noname-b 2026-02-17T06:02:47.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:47.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:02:47.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.334+0000 7f2735d74640 1 -- 10.20.193.78:0/4080733088 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273809df40 con 0x7f273806cc70 2026-02-17T06:02:47.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735d74640 1 -- 10.20.193.78:0/4080733088 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f273806cc70 msgr2=0x7f273806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:47.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735d74640 1 --2- 10.20.193.78:0/4080733088 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f273806cc70 0x7f273806d050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2720009970 tx=0x7f272002efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ae9fe1604aa1836 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:47.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 1 -- 10.20.193.78:0/4080733088 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272003c070 con 0x7f273806cc70 2026-02-17T06:02:47.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:47.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:47.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: dump: 2026-02-17T06:02:47.337 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:47.337 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: _finish_auth 0 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: _check_auth_tickets 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 1 -- 10.20.193.78:0/4080733088 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272002fdf0 con 0x7f273806cc70 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 1 -- 10.20.193.78:0/4080733088 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720037660 con 0x7f273806cc70 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f273cae8640 5 monclient: authenticate success, global_id 144713 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f2735573640 10 monclient: dump: 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f273cae8640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.336+0000 7f273cae8640 4 set_mon_vals no callback set 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 10 monclient: shutdown 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 -- 10.20.193.78:0/4080733088 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f273806cc70 msgr2=0x7f273806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 --2- 10.20.193.78:0/4080733088 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f273806cc70 0x7f273806d050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2720009970 tx=0x7f272002efb0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 -- 10.20.193.78:0/4080733088 shutdown_connections 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 --2- 10.20.193.78:0/4080733088 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f273806cc70 0x7f273806d050 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 --2- 10.20.193.78:0/4080733088 >> v2:10.20.193.78:3300/0 conn(0x7f273806c890 0x7f27380a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 -- 10.20.193.78:0/4080733088 >> 10.20.193.78:0/4080733088 conn(0x7f27380a6620 msgr2=0x7f27380a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:47.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 -- 10.20.193.78:0/4080733088 shutdown_connections 2026-02-17T06:02:47.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 -- 10.20.193.78:0/4080733088 wait complete. 2026-02-17T06:02:47.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 10 monclient: build_initial_monmap 2026-02-17T06:02:47.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 10 monclient: monmap: 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 Processor -- start 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 1 -- start start 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.337+0000 7f273cae8640 10 monclient: init 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f273806c890 0x7f27381cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient(hunting): picked mon.noname-b con 0x7f273806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f273806cc70 0x7f2738134200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient(hunting): picked mon.noname-a con 0x7f273806cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2738134df0 0x7f2738134740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient(hunting): picked mon.noname-c con 0x7f2738134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f273809d360 con 0x7f273806cc70 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f273809d1e0 con 0x7f273806c890 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2738135950 con 0x7f2738134df0 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f273cae8640 10 monclient(hunting): authenticate will time out at 1612.892209s 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f273806cc70 0x7f2738134200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736575640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f273806c890 msgr2=0x7f27381cf380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2738134df0 0x7f2738134740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f273806cc70 0x7f2738134200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:60456/0 (socket says 10.20.193.78:60456) 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 -- 10.20.193.78:0/3662168952 learned_addr learned my addr 10.20.193.78:0/3662168952 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient(hunting): get_auth_request con 0x7f273806cc70 auth_method 0 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736575640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f273806c890 0x7f27381cf380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:47.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 10 monclient(hunting): get_auth_request con 0x7f2738134df0 auth_method 0 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient(hunting): handle_auth_done global_id 184338 payload 824 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 -- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2738134df0 msgr2=0x7f2738134740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2738134df0 0x7f2738134740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 -- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f273806c890 msgr2=0x7f27381cf380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f273806c890 0x7f27381cf380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient: _finish_hunting 0 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 monclient: found mon.noname-a 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2735d74640 1 -- 10.20.193.78:0/3662168952 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2738128950 con 0x7f273806cc70 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.338+0000 7f2736d76640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2738134df0 0x7f2738134740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:02:47.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f2735d74640 1 -- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f273806cc70 msgr2=0x7f2738134200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f2735d74640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f273806cc70 0x7f2738134200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27200090b0 tx=0x7f2720004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 1 -- 10.20.193.78:0/3662168952 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720046030 con 0x7f273806cc70 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: dump: 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: _finish_auth 0 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 1 -- 10.20.193.78:0/3662168952 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272002fbf0 con 0x7f273806cc70 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f273cae8640 5 monclient: authenticate success, global_id 184338 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f273cae8640 10 monclient: _renew_subs 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f273cae8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2738128c30 con 0x7f273806cc70 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 1 -- 10.20.193.78:0/3662168952 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272003c040 con 0x7f273806cc70 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f271effd640 10 monclient: dump: 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:47.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f2734d72640 4 set_mon_vals no callback set 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f273cae8640 10 monclient: _renew_subs 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f273cae8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.340+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2738070780 con 0x7f273806cc70 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f273cae8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f273cae8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f273cae8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2738068410 con 0x7f273806cc70 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f271effd640 1 -- 10.20.193.78:0/3662168952 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2720053060 con 0x7f273806cc70 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f271effd640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f270403de60 0x7f2704040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f271effd640 1 -- 10.20.193.78:0/3662168952 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(595..595 src has 335..595) ==== 4763+0+0 (secure 0 0 0) 0x7f2720077750 con 0x7f273806cc70 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f2736575640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f270403de60 0x7f2704040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:47.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.341+0000 7f2736575640 10 monclient: get_auth_request con 0x7f270403de60 auth_method 0 2026-02-17T06:02:47.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.344+0000 7f271effd640 1 -- 10.20.193.78:0/3662168952 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2738068410 con 0x7f273806cc70 2026-02-17T06:02:47.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.344+0000 7f271effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:47.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.344+0000 7f271effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:47.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.344+0000 7f2736575640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f270403de60 0x7f2704040320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f272c0095a0 tx=0x7f272c009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:47.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.456+0000 7f273cae8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:02:47.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.456+0000 7f273cae8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:47.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.456+0000 7f273cae8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:47.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.456+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f273809e1f0 con 0x7f273806cc70 2026-02-17T06:02:47.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.456+0000 7f271effd640 1 -- 10.20.193.78:0/3662168952 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f273809e1f0 con 0x7f273806cc70 2026-02-17T06:02:47.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.456+0000 7f271effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:47.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.456+0000 7f271effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:47.457 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:02:47.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f270403de60 msgr2=0x7f2704040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:47.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f270403de60 0x7f2704040320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f272c0095a0 tx=0x7f272c009290 comp rx=0 tx=0).stop 2026-02-17T06:02:47.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 10 monclient: shutdown 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f273806cc70 msgr2=0x7f2738134200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f273806cc70 0x7f2738134200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27200090b0 tx=0x7f2720004830 comp rx=0 tx=0).stop 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 shutdown_connections 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2738134df0 0x7f2738134740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f273806cc70 0x7f2738134200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f270403de60 0x7f2704040320 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 --2- 10.20.193.78:0/3662168952 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f273806c890 0x7f27381cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 >> 10.20.193.78:0/3662168952 conn(0x7f27380a6620 msgr2=0x7f273811cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 shutdown_connections 2026-02-17T06:02:47.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:47.457+0000 7f273cae8640 1 -- 10.20.193.78:0/3662168952 wait complete. 2026-02-17T06:02:50.467 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:02:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.533+0000 7f02f045b640 10 monclient: get_monmap_and_config 2026-02-17T06:02:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.533+0000 7f02f045b640 10 monclient: build_initial_monmap 2026-02-17T06:02:50.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.533+0000 7f02f045b640 10 monclient: monmap: 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:50.534787+0000 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:50.534787+0000 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:50.538 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.533+0000 7f02f045b640 1 Processor -- start 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 1 -- start start 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient: init 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient: _add_conns ranks=[0,4,3,1,2] 2026-02-17T06:02:50.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f02e806c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f02e806d380 0x7f02e80a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient(hunting): picked mon.noname-c con 0x7f02e806d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f02e813a240 0x7f02e813c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient(hunting): picked mon.noname-a con 0x7f02e813a240 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e8074150 con 0x7f02e806c890 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f02e809ef70 con 0x7f02e813a240 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f02e809f250 con 0x7f02e806d380 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02f045b640 10 monclient(hunting): authenticate will time out at 1616.088324s 2026-02-17T06:02:50.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ed9cf640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f02e806d380 msgr2=0x7f02e80a01f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee1d0640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f02e806c890 0x7f02e806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:59388/0 (socket says 10.20.193.78:59388) 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee1d0640 1 -- 10.20.193.78:0/2806690257 learned_addr learned my addr 10.20.193.78:0/2806690257 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ed9cf640 1 --2- 10.20.193.78:0/2806690257 >> v2:10.20.193.115:3300/0 conn(0x7f02e806d380 0x7f02e80a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 1 --2- 10.20.193.78:0/2806690257 >> v2:10.20.193.78:3300/0 conn(0x7f02e813a240 0x7f02e813c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient(hunting): get_auth_request con 0x7f02e813a240 auth_method 0 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:50.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient(hunting): handle_auth_done global_id 184341 payload 274 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 1 -- 10.20.193.78:0/2806690257 >> v2:10.20.193.115:3300/0 conn(0x7f02e806d380 msgr2=0x7f02e80a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 1 --2- 10.20.193.78:0/2806690257 >> v2:10.20.193.115:3300/0 conn(0x7f02e806d380 0x7f02e80a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 1 -- 10.20.193.78:0/2806690257 >> v1:10.20.193.78:6789/0 conn(0x7f02e806c890 legacy=0x7f02e806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient: _finish_hunting 0 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 1 monclient: found mon.noname-a 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:50.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.534+0000 7f02ee9d1640 1 -- 10.20.193.78:0/2806690257 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e809df40 con 0x7f02e813a240 2026-02-17T06:02:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ee9d1640 1 -- 10.20.193.78:0/2806690257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e813a240 msgr2=0x7f02e813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:50.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ee9d1640 1 --2- 10.20.193.78:0/2806690257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e813a240 0x7f02e813c630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02e8074e00 tx=0x7f02e402f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe4700bbd620e07 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 1 -- 10.20.193.78:0/2806690257 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e403c070 con 0x7f02e813a240 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: dump: 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:50.551 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: _finish_auth 0 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: _check_auth_tickets 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 1 -- 10.20.193.78:0/2806690257 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e4037640 con 0x7f02e813a240 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 5 monclient: authenticate success, global_id 184341 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:50.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 1 -- 10.20.193.78:0/2806690257 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4037a10 con 0x7f02e813a240 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02ed1ce640 10 monclient: dump: 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:50.553 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 4 set_mon_vals no callback set 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 10 monclient: shutdown 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 1 -- 10.20.193.78:0/2806690257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e813a240 msgr2=0x7f02e813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 1 --2- 10.20.193.78:0/2806690257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e813a240 0x7f02e813c630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02e8074e00 tx=0x7f02e402f1e0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 1 -- 10.20.193.78:0/2806690257 shutdown_connections 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 1 --2- 10.20.193.78:0/2806690257 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e813a240 0x7f02e813c630 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 1 --2- 10.20.193.78:0/2806690257 >> v2:10.20.193.115:3300/0 conn(0x7f02e806d380 0x7f02e80a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 1 -- 10.20.193.78:0/2806690257 >> 10.20.193.78:0/2806690257 conn(0x7f02e80a6620 msgr2=0x7f02e80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 1 -- 10.20.193.78:0/2806690257 shutdown_connections 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.537+0000 7f02f045b640 1 -- 10.20.193.78:0/2806690257 wait complete. 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient: build_initial_monmap 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient: monmap: 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:50.555 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 1 Processor -- start 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 1 -- start start 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient: init 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e806c890 0x7f02e8135790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient(hunting): picked mon.noname-a con 0x7f02e806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e806d380 0x7f02e813db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient(hunting): picked mon.noname-c con 0x7f02e806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e813a240 0x7f02e813e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient(hunting): picked mon.noname-b con 0x7f02e813a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02e813ccf0 con 0x7f02e806c890 2026-02-17T06:02:50.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f02e813cb70 con 0x7f02e813a240 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f02e813ce70 con 0x7f02e806d380 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02f045b640 10 monclient(hunting): authenticate will time out at 1616.092505s 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e806c890 0x7f02e8135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e806c890 0x7f02e8135790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:60466/0 (socket says 10.20.193.78:60466) 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 -- 10.20.193.78:0/1353960670 learned_addr learned my addr 10.20.193.78:0/1353960670 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient(hunting): get_auth_request con 0x7f02e806c890 auth_method 0 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:50.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee9d1640 1 -- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e813a240 msgr2=0x7f02e813e0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ed9cf640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e806d380 0x7f02e813db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee9d1640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e813a240 0x7f02e813e0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ed9cf640 10 monclient(hunting): get_auth_request con 0x7f02e806d380 auth_method 0 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ed9cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ed9cf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ed9cf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ed9cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ed9cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ed9cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient(hunting): handle_auth_done global_id 184344 payload 824 2026-02-17T06:02:50.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 -- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e806d380 msgr2=0x7f02e813db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e806d380 0x7f02e813db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 -- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e813a240 msgr2=0x7f02e813e0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e813a240 0x7f02e813e0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient: _finish_hunting 0 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 monclient: found mon.noname-a 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.538+0000 7f02ee1d0640 1 -- 10.20.193.78:0/1353960670 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e806ee30 con 0x7f02e806c890 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.540+0000 7f02ee1d0640 1 -- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e806c890 msgr2=0x7f02e8135790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.540+0000 7f02ee1d0640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e806c890 0x7f02e8135790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c750 tx=0x7f02d800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.540+0000 7f02ceffd640 1 -- 10.20.193.78:0/1353960670 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8005de0 con 0x7f02e806c890 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.540+0000 7f02ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:50.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.540+0000 7f02ceffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.540+0000 7f02ceffd640 10 monclient: dump: 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:50.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 10 monclient: _finish_auth 0 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 1 -- 10.20.193.78:0/1353960670 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d8007070 con 0x7f02e806c890 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 5 monclient: authenticate success, global_id 184344 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 10 monclient: _renew_subs 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e80a1600 con 0x7f02e806c890 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 1 -- 10.20.193.78:0/1353960670 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8007440 con 0x7f02e806c890 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ec9cd640 4 set_mon_vals no callback set 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 10 monclient: dump: 2026-02-17T06:02:50.561 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 10 monclient: _renew_subs 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e809d810 con 0x7f02e806c890 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:50.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02ceffd640 1 -- 10.20.193.78:0/1353960670 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f02d8007660 con 0x7f02e806c890 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.541+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e80a01d0 con 0x7f02e806c890 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.542+0000 7f02ceffd640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b803deb0 0x7f02b8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.542+0000 7f02ceffd640 1 -- 10.20.193.78:0/1353960670 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(595..595 src has 335..595) ==== 4763+0+0 (secure 0 0 0) 0x7f02d8052520 con 0x7f02e806c890 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.542+0000 7f02ed9cf640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b803deb0 0x7f02b8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.542+0000 7f02ed9cf640 10 monclient: get_auth_request con 0x7f02b803deb0 auth_method 0 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.544+0000 7f02ceffd640 1 -- 10.20.193.78:0/1353960670 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f02d800a8d0 con 0x7f02e806c890 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.544+0000 7f02ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.544+0000 7f02ceffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:50.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.545+0000 7f02ed9cf640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b803deb0 0x7f02b8040370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0094d0 tx=0x7f02dc009460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:50.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.658+0000 7f02f045b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:02:50.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.658+0000 7f02f045b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:50.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.658+0000 7f02f045b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:02:50.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.658+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f02e813ef80 con 0x7f02e806c890 2026-02-17T06:02:50.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.658+0000 7f02ceffd640 1 -- 10.20.193.78:0/1353960670 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f02d800a5e0 con 0x7f02e806c890 2026-02-17T06:02:50.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.658+0000 7f02ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:50.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.658+0000 7f02ceffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:50.661 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:02:50.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b803deb0 msgr2=0x7f02b8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:50.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b803deb0 0x7f02b8040370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0094d0 tx=0x7f02dc009460 comp rx=0 tx=0).stop 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 10 monclient: shutdown 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e806c890 msgr2=0x7f02e8135790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e806c890 0x7f02e8135790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c750 tx=0x7f02d800cc20 comp rx=0 tx=0).stop 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 shutdown_connections 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f02b803deb0 0x7f02b8040370 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f02e813a240 0x7f02e813e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f02e806d380 0x7f02e813db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 --2- 10.20.193.78:0/1353960670 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f02e806c890 0x7f02e8135790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.659+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 >> 10.20.193.78:0/1353960670 conn(0x7f02e80a6620 msgr2=0x7f02e81398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:50.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.660+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 shutdown_connections 2026-02-17T06:02:50.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:50.660+0000 7f02f045b640 1 -- 10.20.193.78:0/1353960670 wait complete. 2026-02-17T06:02:53.669 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient: get_monmap_and_config 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient: build_initial_monmap 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient: monmap: 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:53.737407+0000 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:53.737407+0000 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:53.737 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 1 Processor -- start 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 1 -- start start 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient: init 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f36a006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 0x7f36a013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient(hunting): picked mon.noname-c con 0x7f36a009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient(hunting): picked mon.noname-a con 0x7f36a006d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a006daf0 con 0x7f36a006c890 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f36a00a0880 con 0x7f36a006d380 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f36a009cc80 con 0x7f36a009c4a0 2026-02-17T06:02:53.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a692c640 10 monclient(hunting): authenticate will time out at 1619.290914s 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f369ffff640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f36a006c890 legacy=0x7f36a006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f369f7fe640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 msgr2=0x7f36a013b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f369f7fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 0x7f36a013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.736+0000 7f36a4ea2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:60482/0 (socket says 10.20.193.78:60482) 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 learned_addr learned my addr 10.20.193.78:0/950203380 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 10 monclient(hunting): get_auth_request con 0x7f36a006d380 auth_method 0 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:53.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.737+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:53.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f369ffff640 1 -- 10.20.193.78:0/950203380 >> v1:10.20.193.115:6789/0 conn(0x7f36a006c890 legacy=0x7f36a006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:02:53.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:53.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 10 monclient(hunting): get_auth_request con 0x7f36a006d380 auth_method 0 2026-02-17T06:02:53.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:53.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:53.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:53.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f369f7fe640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 msgr2=0x7f36a013b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:53.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f369f7fe640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 0x7f36a013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:02:53.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:02:53.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:53.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:53.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:53.937+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:02:54.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f369ffff640 1 -- 10.20.193.78:0/950203380 >> v1:10.20.193.115:6789/0 conn(0x7f36a006c890 legacy=0x7f36a006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 10 monclient(hunting): get_auth_request con 0x7f36a006d380 auth_method 0 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f369f7fe640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 msgr2=0x7f36a013b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:02:54.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:54.337+0000 7f369f7fe640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 0x7f36a013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:02:55.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:55.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 10 monclient(hunting): get_auth_request con 0x7f36a006d380 auth_method 0 2026-02-17T06:02:55.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:55.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:55.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:02:55.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f369ffff640 1 -- 10.20.193.78:0/950203380 >> v1:10.20.193.115:6789/0 conn(0x7f36a006c890 legacy=0x7f36a006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:02:55.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:02:55.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:02:55.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:02:55.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:02:55.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f369f7fe640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 msgr2=0x7f36a013b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:55.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:55.138+0000 7f369f7fe640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 0x7f36a013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:02:56.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 10 monclient(hunting): tick 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 1 monclient(hunting): continuing hunt 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 msgr2=0x7f36a013b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.115:3300/0 conn(0x7f36a009c4a0 0x7f36a013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 1 -- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 msgr2=0x7f36a013d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 1 --2- 10.20.193.78:0/950203380 >> v2:10.20.193.78:3300/0 conn(0x7f36a006d380 0x7f36a013d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.736+0000 7f369dffb640 1 -- 10.20.193.78:0/950203380 >> v1:10.20.193.115:6789/0 conn(0x7f36a006c890 legacy=0x7f36a006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:56.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369dffb640 10 monclient: _add_conns ranks=[0,2] 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369dffb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f367c003770 addr v1:10.20.193.78:6789/0 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369dffb640 1 --2- 10.20.193.78:0/950203380 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f367c009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369dffb640 10 monclient(hunting): picked mon.noname-b con 0x7f367c007460 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369dffb640 1 -- 10.20.193.78:0/950203380 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a009cf60 con 0x7f367c003770 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369dffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369dffb640 1 -- 10.20.193.78:0/950203380 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f36a009cc80 con 0x7f367c007460 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369dffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 1 --2- 10.20.193.78:0/950203380 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f367c009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient(hunting): get_auth_request con 0x7f367c007460 auth_method 0 2026-02-17T06:02:56.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient(hunting): handle_auth_done global_id 144722 payload 274 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 1 -- 10.20.193.78:0/950203380 >> v1:10.20.193.78:6789/0 conn(0x7f367c003770 legacy=0x7f367c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 1 monclient: found mon.noname-b 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:56.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.737+0000 7f369f7fe640 1 -- 10.20.193.78:0/950203380 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a006daf0 con 0x7f367c007460 2026-02-17T06:02:56.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.739+0000 7f369f7fe640 1 -- 10.20.193.78:0/950203380 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 msgr2=0x7f367c009920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:56.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.739+0000 7f369f7fe640 1 --2- 10.20.193.78:0/950203380 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f367c009920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3690009d30 tx=0x7f369002f4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8512e17720d76571 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:56.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 1 -- 10.20.193.78:0/950203380 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690005480 con 0x7f367c007460 2026-02-17T06:02:56.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:56.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: dump: 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: _finish_auth 0 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: _check_auth_tickets 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 1 -- 10.20.193.78:0/950203380 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3690033070 con 0x7f367c007460 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 1 -- 10.20.193.78:0/950203380 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690005880 con 0x7f367c007460 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 5 monclient: authenticate success, global_id 144722 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 4 set_mon_vals no callback set 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f369effd640 10 monclient: dump: 2026-02-17T06:02:56.752 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 10 monclient: shutdown 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:56.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 1 -- 10.20.193.78:0/950203380 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 msgr2=0x7f367c009920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 1 --2- 10.20.193.78:0/950203380 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f367c009920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3690009d30 tx=0x7f369002f4e0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a4ea2640 1 -- 10.20.193.78:0/950203380 reap_dead start 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 1 -- 10.20.193.78:0/950203380 shutdown_connections 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 1 -- 10.20.193.78:0/950203380 >> 10.20.193.78:0/950203380 conn(0x7f36a00a6620 msgr2=0x7f36a00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 1 -- 10.20.193.78:0/950203380 shutdown_connections 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 1 -- 10.20.193.78:0/950203380 wait complete. 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 10 monclient: build_initial_monmap 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.740+0000 7f36a692c640 10 monclient: monmap: 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:56.754 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 1 Processor -- start 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 1 -- start start 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient: init 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:02:56.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f36a01cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient(hunting): picked mon.noname-c con 0x7f367c007460 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f36a01346c0 0x7f36a01cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient(hunting): picked mon.noname-a con 0x7f36a01346c0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f36a0134b40 0x7f36a01cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient(hunting): picked mon.noname-b con 0x7f36a0134b40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36a009bfa0 con 0x7f36a01346c0 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f36a009be20 con 0x7f36a0134b40 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f36a013de30 con 0x7f367c007460 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:56.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a692c640 10 monclient(hunting): authenticate will time out at 1622.295412s 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f36a01cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f36a01346c0 0x7f36a01cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f36a01cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44136/0 (socket says 10.20.193.78:44136) 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 -- 10.20.193.78:0/1731886168 learned_addr learned my addr 10.20.193.78:0/1731886168 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient(hunting): get_auth_request con 0x7f367c007460 auth_method 0 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369f7fe640 10 monclient(hunting): get_auth_request con 0x7f36a01346c0 auth_method 0 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:56.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a4ea2640 1 -- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f36a0134b40 msgr2=0x7f36a01cc320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f36a4ea2640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f36a0134b40 0x7f36a01cc320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient(hunting): handle_auth_done global_id 144725 payload 824 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 -- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f36a0134b40 msgr2=0x7f36a01cc320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:02:56.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f36a0134b40 0x7f36a01cc320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 -- 10.20.193.78:0/1731886168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f36a01346c0 msgr2=0x7f36a01cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f36a01346c0 0x7f36a01cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 monclient: found mon.noname-c 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.741+0000 7f369ffff640 1 -- 10.20.193.78:0/1731886168 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a011e590 con 0x7f367c007460 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369ffff640 1 -- 10.20.193.78:0/1731886168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 msgr2=0x7f36a01cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369ffff640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f36a01cf2e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f368c00b910 tx=0x7f368c00bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 1 -- 10.20.193.78:0/1731886168 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c016430 con 0x7f367c007460 2026-02-17T06:02:56.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: dump: 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.760 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: _finish_auth 0 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: _check_auth_tickets 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 1 -- 10.20.193.78:0/1731886168 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c010030 con 0x7f367c007460 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f36a692c640 5 monclient: authenticate success, global_id 144725 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 1 -- 10.20.193.78:0/1731886168 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c016810 con 0x7f367c007460 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369e7fc640 4 set_mon_vals no callback set 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f369cff9640 10 monclient: dump: 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.762 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f36a692c640 10 monclient: _renew_subs 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f36a692c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a0121070 con 0x7f367c007460 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f36a692c640 10 monclient: _renew_subs 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f36a692c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.743+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a009db90 con 0x7f367c007460 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f369cff9640 1 -- 10.20.193.78:0/1731886168 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f368c0153e0 con 0x7f367c007460 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f36a692c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:56.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f36a692c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f36a692c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a011ed80 con 0x7f367c007460 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f369cff9640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f367003deb0 0x7f3670040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f369cff9640 1 -- 10.20.193.78:0/1731886168 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(596..596 src has 335..596) ==== 4735+0+0 (secure 0 0 0) 0x7f368c052680 con 0x7f367c007460 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f369f7fe640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f367003deb0 0x7f3670040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.744+0000 7f369f7fe640 10 monclient: get_auth_request con 0x7f367003deb0 auth_method 0 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.747+0000 7f369cff9640 1 -- 10.20.193.78:0/1731886168 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f368c01b030 con 0x7f367c007460 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.747+0000 7f369cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.747+0000 7f369cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:56.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.747+0000 7f369f7fe640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f367003deb0 0x7f3670040370 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f36a009c980 tx=0x7f3690005000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:56.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.862+0000 7f36a692c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:02:56.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.862+0000 7f36a692c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:56.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.862+0000 7f36a692c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.862+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f36a00744d0 con 0x7f367c007460 2026-02-17T06:02:56.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.862+0000 7f369cff9640 1 -- 10.20.193.78:0/1731886168 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f368c0510b0 con 0x7f367c007460 2026-02-17T06:02:56.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.862+0000 7f369cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:02:56.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.862+0000 7f369cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:56.865 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":198,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:02:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f367003deb0 msgr2=0x7f3670040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f367003deb0 0x7f3670040370 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f36a009c980 tx=0x7f3690005000 comp rx=0 tx=0).stop 2026-02-17T06:02:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 10 monclient: shutdown 2026-02-17T06:02:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 msgr2=0x7f36a01cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f36a01cf2e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f368c00b910 tx=0x7f368c00bde0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 shutdown_connections 2026-02-17T06:02:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f367003deb0 0x7f3670040370 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f36a0134b40 0x7f36a01cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f36a01346c0 0x7f36a01cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 --2- 10.20.193.78:0/1731886168 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f367c007460 0x7f36a01cf2e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 >> 10.20.193.78:0/1731886168 conn(0x7f36a00a6620 msgr2=0x7f36a00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 shutdown_connections 2026-02-17T06:02:56.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.863+0000 7f36a692c640 1 -- 10.20.193.78:0/1731886168 wait complete. 2026-02-17T06:02:56.868 INFO:tasks.rados.rados.0.trial115.stdout:2801: done (3 left) 2026-02-17T06:02:56.868 INFO:tasks.rados.rados.0.trial115.stdout:2803: done (2 left) 2026-02-17T06:02:56.868 INFO:tasks.rados.rados.0.trial115.stdout:2804: done (1 left) 2026-02-17T06:02:56.868 INFO:tasks.rados.rados.0.trial115.stdout:2805: done (0 left) 2026-02-17T06:02:56.868 INFO:tasks.rados.rados.0.trial115.stdout:2806: snap_create 2026-02-17T06:02:56.872 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:02:56.872 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:02:56.872 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:02:56.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.938+0000 7fec31d79640 10 monclient: get_monmap_and_config 2026-02-17T06:02:56.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.938+0000 7fec31d79640 10 monclient: build_initial_monmap 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.938+0000 7fec31d79640 10 monclient: monmap: 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:56.939783+0000 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:56.939783+0000 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:56.943 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.938+0000 7fec31d79640 1 Processor -- start 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 1 -- start start 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient: init 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fec2c06c890 0x7fec2c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient(hunting): picked mon.noname-c con 0x7fec2c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fec2c06cc70 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c13a240 0x7fec2c13c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient(hunting): picked mon.noname-b con 0x7fec2c13a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec2c074150 con 0x7fec2c06cc70 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fec2c09ef70 con 0x7fec2c06c890 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c09f250 con 0x7fec2c13a240 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec31d79640 10 monclient(hunting): authenticate will time out at 1622.493341s 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2b7fe640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fec2c06c890 msgr2=0x7fec2c0a01f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:56.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2b7fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fec2c06c890 0x7fec2c0a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c13a240 0x7fec2c13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c13a240 0x7fec2c13c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44148/0 (socket says 10.20.193.78:44148) 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 1 -- 10.20.193.78:0/3432691135 learned_addr learned my addr 10.20.193.78:0/3432691135 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient(hunting): get_auth_request con 0x7fec2c13a240 auth_method 0 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2affd640 1 -- 10.20.193.78:0/3432691135 >> v1:10.20.193.115:6789/0 conn(0x7fec2c06cc70 legacy=0x7fec2c06d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:02:56.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient(hunting): handle_auth_done global_id 144728 payload 274 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 1 -- 10.20.193.78:0/3432691135 >> v2:10.20.193.115:3300/0 conn(0x7fec2c06c890 msgr2=0x7fec2c0a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 1 --2- 10.20.193.78:0/3432691135 >> v2:10.20.193.115:3300/0 conn(0x7fec2c06c890 0x7fec2c0a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 1 -- 10.20.193.78:0/3432691135 >> v1:10.20.193.115:6789/0 conn(0x7fec2c06cc70 legacy=0x7fec2c06d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 1 monclient: found mon.noname-b 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.939+0000 7fec2bfff640 1 -- 10.20.193.78:0/3432691135 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c09df40 con 0x7fec2c13a240 2026-02-17T06:02:56.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2bfff640 1 -- 10.20.193.78:0/3432691135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c13a240 msgr2=0x7fec2c13c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:56.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2bfff640 1 --2- 10.20.193.78:0/3432691135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c13a240 0x7fec2c13c630 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fec20009e50 tx=0x7fec2002f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8aeb0c026fd4d1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:56.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 1 -- 10.20.193.78:0/3432691135 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec200373f0 con 0x7fec2c13a240 2026-02-17T06:02:56.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: dump: 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.959 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 5 monclient: authenticate success, global_id 144728 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 1 -- 10.20.193.78:0/3432691135 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec20037590 con 0x7fec2c13a240 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:56.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 1 -- 10.20.193.78:0/3432691135 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20037960 con 0x7fec2c13a240 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec2a7fc640 10 monclient: dump: 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.961 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 4 set_mon_vals no callback set 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 10 monclient: shutdown 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 1 -- 10.20.193.78:0/3432691135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c13a240 msgr2=0x7fec2c13c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 1 --2- 10.20.193.78:0/3432691135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c13a240 0x7fec2c13c630 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fec20009e50 tx=0x7fec2002f360 comp rx=0 tx=0).stop 2026-02-17T06:02:56.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 1 -- 10.20.193.78:0/3432691135 shutdown_connections 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 1 --2- 10.20.193.78:0/3432691135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c13a240 0x7fec2c13c630 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 1 --2- 10.20.193.78:0/3432691135 >> v2:10.20.193.115:3300/0 conn(0x7fec2c06c890 0x7fec2c0a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 1 -- 10.20.193.78:0/3432691135 >> 10.20.193.78:0/3432691135 conn(0x7fec2c0a6620 msgr2=0x7fec2c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 1 -- 10.20.193.78:0/3432691135 shutdown_connections 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.942+0000 7fec31d79640 1 -- 10.20.193.78:0/3432691135 wait complete. 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient: build_initial_monmap 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient: monmap: 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:56.963 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 1 Processor -- start 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 1 -- start start 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient: init 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec2c06c890 0x7fec2c1cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient(hunting): picked mon.noname-b con 0x7fec2c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c06cc70 0x7fec2c134010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient(hunting): picked mon.noname-c con 0x7fec2c06cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c13a240 0x7fec2c11da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient(hunting): picked mon.noname-a con 0x7fec2c13a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c13ccf0 con 0x7fec2c13a240 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c13cb70 con 0x7fec2c06c890 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c13ce70 con 0x7fec2c06cc70 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec31d79640 10 monclient(hunting): authenticate will time out at 1622.497650s 2026-02-17T06:02:56.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c06cc70 0x7fec2c134010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2b7fe640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec2c06c890 msgr2=0x7fec2c1cf070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c06cc70 0x7fec2c134010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44152/0 (socket says 10.20.193.78:44152) 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 -- 10.20.193.78:0/1039292709 learned_addr learned my addr 10.20.193.78:0/1039292709 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient(hunting): get_auth_request con 0x7fec2c06cc70 auth_method 0 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2b7fe640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec2c06c890 0x7fec2c1cf070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:56.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2bfff640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c13a240 0x7fec2c11da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2bfff640 10 monclient(hunting): get_auth_request con 0x7fec2c13a240 auth_method 0 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient(hunting): handle_auth_done global_id 144731 payload 824 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec2c06c890 msgr2=0x7fec2c1cf070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fec2c06c890 0x7fec2c1cf070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c13a240 msgr2=0x7fec2c11da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c13a240 0x7fec2c11da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient: _finish_hunting 0 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 monclient: found mon.noname-c 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.943+0000 7fec2affd640 1 -- 10.20.193.78:0/1039292709 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c073170 con 0x7fec2c06cc70 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.945+0000 7fec2affd640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c06cc70 msgr2=0x7fec2c134010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.945+0000 7fec2affd640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c06cc70 0x7fec2c134010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00b7b0 tx=0x7fec1c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c005270 con 0x7fec2c06cc70 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: dump: 2026-02-17T06:02:56.968 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: _finish_auth 0 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: _check_auth_tickets 2026-02-17T06:02:56.969 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1c010070 con 0x7fec2c06cc70 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c005670 con 0x7fec2c06cc70 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 10 monclient: dump: 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.970 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec29ffb640 4 set_mon_vals no callback set 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 5 monclient: authenticate success, global_id 144731 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 10 monclient: _renew_subs 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c1cff90 con 0x7fec2c06cc70 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 10 monclient: _renew_subs 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c1219d0 con 0x7fec2c06cc70 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:56.971 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c134550 0x7fec2c11dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c120240 con 0x7fec2c134550 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c134550 0x7fec2c11dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 10 monclient: get_auth_request con 0x7fec2c134550 auth_method 0 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:56.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 10 monclient(hunting): handle_auth_done global_id 184353 payload 824 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec2b7fe640 1 -- 10.20.193.78:0/1039292709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec2c09d810 con 0x7fec2c134550 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.946+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fec1c00c520 con 0x7fec2c06cc70 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.947+0000 7fec07fff640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7febfc03de60 0x7febfc040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.947+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(596..596 src has 335..596) ==== 4735+0+0 (secure 0 0 0) 0x7fec1c052130 con 0x7fec2c06cc70 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.947+0000 7fec2bfff640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7febfc03de60 0x7febfc040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.947+0000 7fec2bfff640 10 monclient: get_auth_request con 0x7febfc03de60 auth_method 0 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.948+0000 7fec2b7fe640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c134550 msgr2=0x7fec2c11dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.948+0000 7fec2b7fe640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c134550 0x7fec2c11dfc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec18003a30 tx=0x7fec1800bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:56.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18005070 con 0x7fec2c134550 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 10 monclient: dump: 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.974 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fec18005070 con 0x7fec2c134550 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c134550 msgr2=0x7fec2c11dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c134550 0x7fec2c11dfc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec18003a30 tx=0x7fec1800bcc0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c134550 msgr2=0x7fec2c11dfc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec07fff640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c134550 0x7fec2c11dfc0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec18003a30 tx=0x7fec1800bcc0 comp rx=0 tx=0).stop 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.949+0000 7fec2bfff640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7febfc03de60 0x7febfc040320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fec2c074e00 tx=0x7fec20033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec31d79640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec31d79640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c11e500 0x7fec2c1da400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:56.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec31d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c11e910 con 0x7fec2c11e500 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c11e500 0x7fec2c1da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 10 monclient: get_auth_request con 0x7fec2c11e500 auth_method 0 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 10 monclient(hunting): handle_auth_done global_id 184356 payload 824 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.952+0000 7fec2b7fe640 1 -- 10.20.193.78:0/1039292709 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fec2c13c7d0 con 0x7fec2c11e500 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.954+0000 7fec2b7fe640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c11e500 msgr2=0x7fec2c1da400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.954+0000 7fec2b7fe640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c11e500 0x7fec2c1da400 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec18003a30 tx=0x7fec18005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:56.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1801ae30 con 0x7fec2c11e500 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 10 monclient: dump: 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:56.977 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4700 (secure 0 0 0) 0x7fec1801ae30 con 0x7fec2c11e500 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c11e500 msgr2=0x7fec2c1da400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c11e500 0x7fec2c1da400 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec18003a30 tx=0x7fec18005b20 comp rx=0 tx=0).stop 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c11e500 msgr2=0x7fec2c1da400 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec07fff640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fec2c11e500 0x7fec2c1da400 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec18003a30 tx=0x7fec18005b20 comp rx=0 tx=0).stop 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:02:56.978 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 198, 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 141736, 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:56.979 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:56.980 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:02:56.981 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:56.982 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:56.983 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:02:56.984 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:02:56.985 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:02:56.986 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.987 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:56.988 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:56.989 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:02:56.989 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7febfc03de60 msgr2=0x7febfc040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7febfc03de60 0x7febfc040320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fec2c074e00 tx=0x7fec20033000 comp rx=0 tx=0).stop 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 10 monclient: shutdown 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c06cc70 msgr2=0x7fec2c134010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 1 --2- 10.20.193.78:0/1039292709 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fec2c06cc70 0x7fec2c134010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00b7b0 tx=0x7fec1c00bc80 comp rx=0 tx=0).stop 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec2bfff640 1 -- 10.20.193.78:0/1039292709 reap_dead start 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec2bfff640 1 -- 10.20.193.78:0/1039292709 reap_dead start 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 shutdown_connections 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 >> 10.20.193.78:0/1039292709 conn(0x7fec2c0a6620 msgr2=0x7fec2c139660 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 shutdown_connections 2026-02-17T06:02:56.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:56.955+0000 7fec31d79640 1 -- 10.20.193.78:0/1039292709 wait complete. 2026-02-17T06:02:56.991 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:02:56.991 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:02:56.991 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:02:57.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient: get_monmap_and_config 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient: build_initial_monmap 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient: monmap: 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:02:57.059295+0000 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:02:57.059295+0000 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:57.062 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 1 Processor -- start 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 1 -- start start 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient: init 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:57.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f6006c890 0x7f6f6006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient(hunting): picked mon.noname-b con 0x7f6f6006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6f6009c4a0 0x7f6f6006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient(hunting): picked mon.noname-c con 0x7f6f6009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6f60134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6006daf0 con 0x7f6f60134df0 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6f600a0880 con 0x7f6f6009c4a0 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:57.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6006d6f0 con 0x7f6f6006c890 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f66204640 10 monclient(hunting): authenticate will time out at 1622.612843s 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f6006c890 0x7f6f6006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5effd640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f6f6009c4a0 msgr2=0x7f6f6006d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5ffff640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f6f60134df0 legacy=0x7f6f60135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5effd640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6f6009c4a0 0x7f6f6006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f6006c890 0x7f6f6006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44164/0 (socket says 10.20.193.78:44164) 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/3726655176 learned_addr learned my addr 10.20.193.78:0/3726655176 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5f7fe640 10 monclient(hunting): get_auth_request con 0x7f6f6006c890 auth_method 0 2026-02-17T06:02:57.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.058+0000 7f6f5f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_done global_id 144734 payload 274 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/3726655176 >> v2:10.20.193.115:3300/0 conn(0x7f6f6009c4a0 msgr2=0x7f6f6006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 1 --2- 10.20.193.78:0/3726655176 >> v2:10.20.193.115:3300/0 conn(0x7f6f6009c4a0 0x7f6f6006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/3726655176 >> v1:10.20.193.115:6789/0 conn(0x7f6f60134df0 legacy=0x7f6f60135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:02:57.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:02:57.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 1 monclient: found mon.noname-b 2026-02-17T06:02:57.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:57.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:02:57.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.059+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/3726655176 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6009cc80 con 0x7f6f6006c890 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/3726655176 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f6006c890 msgr2=0x7f6f6006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5f7fe640 1 --2- 10.20.193.78:0/3726655176 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f6006c890 0x7f6f6006cc70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009970 tx=0x7f6f4c02eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f683da42bad12f5e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 1 -- 10.20.193.78:0/3726655176 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c03c070 con 0x7f6f6006c890 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: dump: 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 1 -- 10.20.193.78:0/3726655176 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f4c02fdd0 con 0x7f6f6006c890 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f66204640 5 monclient: authenticate success, global_id 144734 2026-02-17T06:02:57.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f66204640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 1 -- 10.20.193.78:0/3726655176 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c037660 con 0x7f6f6006c890 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f66204640 10 monclient: get_monmap_and_config success 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f66204640 4 set_mon_vals no callback set 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.061+0000 7f6f5e7fc640 10 monclient: dump: 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 10 monclient: shutdown 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 -- 10.20.193.78:0/3726655176 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f6006c890 msgr2=0x7f6f6006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 --2- 10.20.193.78:0/3726655176 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f6006c890 0x7f6f6006cc70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009970 tx=0x7f6f4c02eeb0 comp rx=0 tx=0).stop 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 -- 10.20.193.78:0/3726655176 shutdown_connections 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 --2- 10.20.193.78:0/3726655176 >> v2:10.20.193.115:3300/0 conn(0x7f6f6009c4a0 0x7f6f6006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 --2- 10.20.193.78:0/3726655176 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f6006c890 0x7f6f6006cc70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 -- 10.20.193.78:0/3726655176 >> 10.20.193.78:0/3726655176 conn(0x7f6f600a6620 msgr2=0x7f6f600a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 -- 10.20.193.78:0/3726655176 shutdown_connections 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 -- 10.20.193.78:0/3726655176 wait complete. 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 10 monclient: build_initial_monmap 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 10 monclient: monmap: 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:02:57.077 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 Processor -- start 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 1 -- start start 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.062+0000 7f6f66204640 10 monclient: init 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient: _reopen_session rank -1 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f6006c890 0x7f6f601cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient(hunting): picked mon.noname-b con 0x7f6f6006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f6009c4a0 0x7f6f601cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient(hunting): picked mon.noname-a con 0x7f6f6009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60134df0 0x7f6f60121700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient(hunting): picked mon.noname-c con 0x7f6f60134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f600a01a0 con 0x7f6f6009c4a0 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f600a0020 con 0x7f6f6006c890 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6009be20 con 0x7f6f60134df0 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient(hunting): _renew_subs 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f66204640 10 monclient(hunting): authenticate will time out at 1622.617069s 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60134df0 0x7f6f60121700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5f7fe640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f6006c890 msgr2=0x7f6f601cd330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60134df0 0x7f6f60121700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44172/0 (socket says 10.20.193.78:44172) 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 -- 10.20.193.78:0/4109795602 learned_addr learned my addr 10.20.193.78:0/4109795602 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient(hunting): get_auth_request con 0x7f6f60134df0 auth_method 0 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5effd640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f6009c4a0 0x7f6f601cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5f7fe640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f6006c890 0x7f6f601cd330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5effd640 10 monclient(hunting): get_auth_request con 0x7f6f6009c4a0 auth_method 0 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient(hunting): handle_auth_done global_id 144737 payload 824 2026-02-17T06:02:57.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f6006c890 msgr2=0x7f6f601cd330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f6006c890 0x7f6f601cd330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f6009c4a0 msgr2=0x7f6f601cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f6009c4a0 0x7f6f601cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 monclient: found mon.noname-c 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.063+0000 7f6f5ffff640 1 -- 10.20.193.78:0/4109795602 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f601250d0 con 0x7f6f60134df0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f5ffff640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60134df0 msgr2=0x7f6f60121700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f5ffff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60134df0 0x7f6f60121700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400be90 tx=0x7f6f5400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f5ffff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60134df0 0x7f6f60121700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400be90 tx=0x7f6f5400bf90 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f6f54010070 config(0 keys) 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54006e60 con 0x7f6f60134df0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 10 monclient: dump: 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 10 monclient: _finish_auth 0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f66204640 5 monclient: authenticate success, global_id 144737 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f54010070 con 0x7f6f60134df0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f66204640 10 monclient: _renew_subs 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f66204640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f60123f00 con 0x7f6f60134df0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f3bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f5dffb640 4 set_mon_vals no callback set 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f66204640 10 monclient: _renew_subs 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f66204640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.065+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f60125690 con 0x7f6f60134df0 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f66204640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f66204640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60121c40 0x7f6f60122020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:57.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f66204640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6f601359e0 con 0x7f6f60121c40 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f540158b0 con 0x7f6f60134df0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f3bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f3bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f3bfff640 10 monclient: dump: 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6f54008030 con 0x7f6f60134df0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60121c40 0x7f6f60122020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 10 monclient: get_auth_request con 0x7f6f60121c40 auth_method 0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_done global_id 144740 payload 824 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/4109795602 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f600008d0 con 0x7f6f60121c40 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f3bfff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f3003de60 0x7f6f30040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5effd640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f3003de60 0x7f6f30040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.066+0000 7f6f5effd640 10 monclient: get_auth_request con 0x7f6f3003de60 auth_method 0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60121c40 msgr2=0x7f6f60122020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f5f7fe640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60121c40 0x7f6f60122020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6f6009c880 tx=0x7f6f4c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c047070 con 0x7f6f60121c40 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 10 monclient: dump: 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:57.080 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6f4c037d70 con 0x7f6f60121c40 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60121c40 msgr2=0x7f6f60122020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60121c40 0x7f6f60122020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6f6009c880 tx=0x7f6f4c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60121c40 msgr2=0x7f6f60122020 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.068+0000 7f6f3bfff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60121c40 0x7f6f60122020 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6f6009c880 tx=0x7f6f4c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.069+0000 7f6f5effd640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f3003de60 0x7f6f30040320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6f500094d0 tx=0x7f6f50009460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f66204640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f66204640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60122560 0x7f6f601d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f66204640 10 monclient(hunting): start opening mon connection 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6f60124df0 con 0x7f6f60122560 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60122560 0x7f6f601d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 10 monclient: get_auth_request con 0x7f6f60122560 auth_method 0 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 10 monclient(hunting): handle_auth_done global_id 144743 payload 824 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.072+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/4109795602 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6f601cddb0 con 0x7f6f60122560 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f5f7fe640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60122560 msgr2=0x7f6f601d85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f5f7fe640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60122560 0x7f6f601d85b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009970 tx=0x7f6f4c009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c052070 con 0x7f6f60122560 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 10 monclient: dump: 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:02:57.081 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7f6f4c004590 con 0x7f6f60122560 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60122560 msgr2=0x7f6f601d85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60122560 0x7f6f601d85b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009970 tx=0x7f6f4c009340 comp rx=0 tx=0).stop 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60122560 msgr2=0x7f6f601d85b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60122560 0x7f6f601d85b0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009970 tx=0x7f6f4c009340 comp rx=0 tx=0).stop 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 198, 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 360609, 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:57.082 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:02:57.083 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.084 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:57.085 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:02:57.086 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f3003de60 msgr2=0x7f6f30040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f3003de60 0x7f6f30040320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6f500094d0 tx=0x7f6f50009460 comp rx=0 tx=0).stop 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 10 monclient: shutdown 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60134df0 msgr2=0x7f6f60121700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 1 --2- 10.20.193.78:0/4109795602 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f60134df0 0x7f6f60121700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400be90 tx=0x7f6f5400bf90 comp rx=0 tx=0).stop 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f5ffff640 1 -- 10.20.193.78:0/4109795602 reap_dead start 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f5ffff640 1 -- 10.20.193.78:0/4109795602 reap_dead start 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f3bfff640 1 -- 10.20.193.78:0/4109795602 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(596..596 src has 335..596) ==== 4735+0+0 (unknown 0 0 0) 0x7f6f54052720 con 0x7f6f60134df0 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 shutdown_connections 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 >> 10.20.193.78:0/4109795602 conn(0x7f6f600a6620 msgr2=0x7f6f60135bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 shutdown_connections 2026-02-17T06:02:57.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:02:57.074+0000 7f6f66204640 1 -- 10.20.193.78:0/4109795602 wait complete. 2026-02-17T06:02:57.087 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:02:57.087 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:02:57.738 INFO:tasks.rados.rados.0.trial115.stdout:2806: done (0 left) 2026-02-17T06:02:57.738 INFO:tasks.rados.rados.0.trial115.stdout:2807: snap_remove snap 283 2026-02-17T06:02:58.659 INFO:tasks.rados.rados.0.trial115.stdout:2807: done (0 left) 2026-02-17T06:02:58.659 INFO:tasks.rados.rados.0.trial115.stdout:2808: read oid 25 snap -1 2026-02-17T06:02:58.660 INFO:tasks.rados.rados.0.trial115.stdout:2808: expect (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:02:58.660 INFO:tasks.rados.rados.0.trial115.stdout:2809: copy_from oid 26 from oid 37 current snap is 284 2026-02-17T06:02:58.660 INFO:tasks.rados.rados.0.trial115.stdout:2810: read oid 35 snap -1 2026-02-17T06:02:58.660 INFO:tasks.rados.rados.0.trial115.stdout:2810: expect (ObjNum 948 snap 276 seq_num 948) 2026-02-17T06:02:58.660 INFO:tasks.rados.rados.0.trial115.stdout:2811: write (excl) oid 43 current snap is 284 2026-02-17T06:02:58.660 INFO:tasks.rados.rados.0.trial115.stdout:2811: seq_num 990 ranges {638930=422400,1665644=645682} 2026-02-17T06:02:58.663 INFO:tasks.rados.rados.0.trial115.stdout:2811: writing trial11545420-43 from 638930 to 1061330 tid 1 2026-02-17T06:02:58.669 INFO:tasks.rados.rados.0.trial115.stdout:2811: writing trial11545420-43 from 1665644 to 2311326 tid 2 2026-02-17T06:02:58.669 INFO:tasks.rados.rados.0.trial115.stdout:2812: read oid 9 snap 282 2026-02-17T06:02:58.669 INFO:tasks.rados.rados.0.trial115.stdout:2812: expect (ObjNum 982 snap 282 seq_num 982) 2026-02-17T06:02:58.669 INFO:tasks.rados.rados.0.trial115.stdout:2813: read oid 4 snap 282 2026-02-17T06:02:58.669 INFO:tasks.rados.rados.0.trial115.stdout:2813: expect (ObjNum 980 snap 281 seq_num 980) 2026-02-17T06:02:58.669 INFO:tasks.rados.rados.0.trial115.stdout:2814: write oid 18 current snap is 284 2026-02-17T06:02:58.669 INFO:tasks.rados.rados.0.trial115.stdout:2814: seq_num 991 ranges {655737=497649,1607773=789340} 2026-02-17T06:02:58.673 INFO:tasks.rados.rados.0.trial115.stdout:2814: writing trial11545420-18 from 655737 to 1153386 tid 1 2026-02-17T06:02:58.678 INFO:tasks.rados.rados.0.trial115.stdout:2814: writing trial11545420-18 from 1607773 to 2397113 tid 2 2026-02-17T06:02:58.678 INFO:tasks.rados.rados.0.trial115.stdout:2815: delete oid 1 current snap is 284 2026-02-17T06:02:58.678 INFO:tasks.rados.rados.0.trial115.stdout:2809: finishing copy_from to trial11545420-26 2026-02-17T06:02:58.678 INFO:tasks.rados.rados.0.trial115.stdout:2809: got expected ENOENT (src dne) 2026-02-17T06:02:58.678 INFO:tasks.rados.rados.0.trial115.stdout:2809: finishing copy_from racing read to trial11545420-26 2026-02-17T06:02:58.679 INFO:tasks.rados.rados.0.trial115.stdout:2810: expect (ObjNum 948 snap 276 seq_num 948) 2026-02-17T06:02:58.843 INFO:tasks.rados.rados.0.trial115.stdout:2808: expect (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2811: finishing write tid 1 to trial11545420-43 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2808: done (7 left) 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2809: done (6 left) 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2810: done (5 left) 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2815: done (4 left) 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2816: read oid 50 snap 282 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2816: expect (ObjNum 751 snap 206 seq_num 751) 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2817: snap_create 2026-02-17T06:02:58.941 INFO:tasks.rados.rados.0.trial115.stdout:2812: expect (ObjNum 982 snap 282 seq_num 982) 2026-02-17T06:02:59.015 INFO:tasks.rados.rados.0.trial115.stdout:2813: expect (ObjNum 980 snap 281 seq_num 980) 2026-02-17T06:02:59.087 INFO:tasks.rados.rados.0.trial115.stdout:2814: finishing write tid 1 to trial11545420-18 2026-02-17T06:02:59.087 INFO:tasks.rados.rados.0.trial115.stdout:2811: finishing write tid 2 to trial11545420-43 2026-02-17T06:02:59.087 INFO:tasks.rados.rados.0.trial115.stdout:2811: finishing write tid 3 to trial11545420-43 2026-02-17T06:02:59.087 INFO:tasks.rados.rados.0.trial115.stdout:2811: finishing write tid 4 to trial11545420-43 2026-02-17T06:02:59.087 INFO:tasks.rados.rados.0.trial115.stdout:2811: oid 43 updating version 0 to 1430 2026-02-17T06:02:59.087 INFO:tasks.rados.rados.0.trial115.stdout:2811: oid 43 version 1430 is already newer than 1429 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2811: oid 43 updating version 1430 to 1431 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1431 (ObjNum 990 snap 284 seq_num 990) dirty exists 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2811: left oid 43 (ObjNum 990 snap 284 seq_num 990) 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2814: finishing write tid 2 to trial11545420-18 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2814: finishing write tid 3 to trial11545420-18 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2814: finishing write tid 4 to trial11545420-18 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2814: oid 18 updating version 0 to 1410 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2814: oid 18 updating version 1410 to 1411 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2814: oid 18 version 1411 is already newer than 1409 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1411 (ObjNum 991 snap 284 seq_num 991) dirty exists 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2814: left oid 18 (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:02:59.088 INFO:tasks.rados.rados.0.trial115.stdout:2816: expect (ObjNum 751 snap 206 seq_num 751) 2026-02-17T06:02:59.741 INFO:tasks.rados.rados.0.trial115.stdout:2811: done (5 left) 2026-02-17T06:02:59.741 INFO:tasks.rados.rados.0.trial115.stdout:2812: done (4 left) 2026-02-17T06:02:59.741 INFO:tasks.rados.rados.0.trial115.stdout:2813: done (3 left) 2026-02-17T06:02:59.741 INFO:tasks.rados.rados.0.trial115.stdout:2814: done (2 left) 2026-02-17T06:02:59.741 INFO:tasks.rados.rados.0.trial115.stdout:2816: done (1 left) 2026-02-17T06:02:59.741 INFO:tasks.rados.rados.0.trial115.stdout:2817: done (0 left) 2026-02-17T06:02:59.741 INFO:tasks.rados.rados.0.trial115.stdout:2818: snap_create 2026-02-17T06:03:00.742 INFO:tasks.rados.rados.0.trial115.stdout:2818: done (0 left) 2026-02-17T06:03:00.742 INFO:tasks.rados.rados.0.trial115.stdout:2819: read oid 5 snap 282 2026-02-17T06:03:00.742 INFO:tasks.rados.rados.0.trial115.stdout:2819: expect (ObjNum 926 snap 272 seq_num 926) 2026-02-17T06:03:00.742 INFO:tasks.rados.rados.0.trial115.stdout:2820: read oid 27 snap -1 2026-02-17T06:03:00.742 INFO:tasks.rados.rados.0.trial115.stdout:2820: expect (ObjNum 973 snap 280 seq_num 973) 2026-02-17T06:03:00.742 INFO:tasks.rados.rados.0.trial115.stdout:2821: read oid 26 snap -1 2026-02-17T06:03:00.742 INFO:tasks.rados.rados.0.trial115.stdout:2821: expect deleted 2026-02-17T06:03:00.742 INFO:tasks.rados.rados.0.trial115.stdout:2822: snap_create 2026-02-17T06:03:00.746 INFO:tasks.rados.rados.0.trial115.stdout:2819: expect (ObjNum 926 snap 272 seq_num 926) 2026-02-17T06:03:00.848 INFO:tasks.rados.rados.0.trial115.stdout:2820: expect (ObjNum 973 snap 280 seq_num 973) 2026-02-17T06:03:01.745 INFO:tasks.rados.rados.0.trial115.stdout:2819: done (3 left) 2026-02-17T06:03:01.745 INFO:tasks.rados.rados.0.trial115.stdout:2820: done (2 left) 2026-02-17T06:03:01.745 INFO:tasks.rados.rados.0.trial115.stdout:2821: done (1 left) 2026-02-17T06:03:01.745 INFO:tasks.rados.rados.0.trial115.stdout:2822: done (0 left) 2026-02-17T06:03:01.745 INFO:tasks.rados.rados.0.trial115.stdout:2823: snap_create 2026-02-17T06:03:02.748 INFO:tasks.rados.rados.0.trial115.stdout:2823: done (0 left) 2026-02-17T06:03:02.749 INFO:tasks.rados.rados.0.trial115.stdout:2824: copy_from oid 16 from oid 28 current snap is 288 2026-02-17T06:03:02.749 INFO:tasks.rados.rados.0.trial115.stdout:2825: snap_remove snap 286 2026-02-17T06:03:03.662 INFO:tasks.rados.rados.0.trial115.stdout:2825: done (1 left) 2026-02-17T06:03:03.662 INFO:tasks.rados.rados.0.trial115.stdout:2826: rollback oid 46 current snap is 288 2026-02-17T06:03:03.662 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 46 to 287 2026-02-17T06:03:03.663 INFO:tasks.rados.rados.0.trial115.stdout:2824: finishing copy_from racing read to trial11545420-16 2026-02-17T06:03:03.663 INFO:tasks.rados.rados.0.trial115.stdout:2824: finishing copy_from to trial11545420-16 2026-02-17T06:03:03.663 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1326 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:03:03.663 INFO:tasks.rados.rados.0.trial115.stdout:2824: done (1 left) 2026-02-17T06:03:03.663 INFO:tasks.rados.rados.0.trial115.stdout:2827: write (excl) oid 32 current snap is 288 2026-02-17T06:03:03.663 INFO:tasks.rados.rados.0.trial115.stdout:2827: seq_num 993 ranges {689351=648147,2124737=564614} 2026-02-17T06:03:03.666 INFO:tasks.rados.rados.0.trial115.stdout:2827: writing trial11545420-32 from 689351 to 1337498 tid 1 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:2827: writing trial11545420-32 from 2124737 to 2689351 tid 2 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:2828: copy_from oid 28 from oid 47 current snap is 288 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:2826: finishing rollback tid 0 to trial11545420-46 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:2826: finishing rollback tid 1 to trial11545420-46 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:2826: finishing rollback tid 2 to trial11545420-46 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1424 (ObjNum 976 snap 280 seq_num 976) dirty exists 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:2827: finishing write tid 1 to trial11545420-32 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:2826: done (2 left) 2026-02-17T06:03:03.672 INFO:tasks.rados.rados.0.trial115.stdout:2829: snap_create 2026-02-17T06:03:03.673 INFO:tasks.rados.rados.0.trial115.stdout:2828: finishing copy_from to trial11545420-28 2026-02-17T06:03:03.673 INFO:tasks.rados.rados.0.trial115.stdout:2828: got expected ENOENT (src dne) 2026-02-17T06:03:03.673 INFO:tasks.rados.rados.0.trial115.stdout:2828: finishing copy_from racing read to trial11545420-28 2026-02-17T06:03:03.673 INFO:tasks.rados.rados.0.trial115.stdout:2827: finishing write tid 2 to trial11545420-32 2026-02-17T06:03:03.674 INFO:tasks.rados.rados.0.trial115.stdout:2827: finishing write tid 3 to trial11545420-32 2026-02-17T06:03:03.674 INFO:tasks.rados.rados.0.trial115.stdout:2827: finishing write tid 4 to trial11545420-32 2026-02-17T06:03:03.674 INFO:tasks.rados.rados.0.trial115.stdout:2827: oid 32 updating version 0 to 1152 2026-02-17T06:03:03.674 INFO:tasks.rados.rados.0.trial115.stdout:2827: oid 32 updating version 1152 to 1153 2026-02-17T06:03:03.674 INFO:tasks.rados.rados.0.trial115.stdout:2827: oid 32 updating version 1153 to 1154 2026-02-17T06:03:03.674 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1154 (ObjNum 993 snap 288 seq_num 993) dirty exists 2026-02-17T06:03:03.674 INFO:tasks.rados.rados.0.trial115.stdout:2827: left oid 32 (ObjNum 993 snap 288 seq_num 993) 2026-02-17T06:03:04.661 INFO:tasks.rados.rados.0.trial115.stdout:2827: done (2 left) 2026-02-17T06:03:04.662 INFO:tasks.rados.rados.0.trial115.stdout:2828: done (1 left) 2026-02-17T06:03:04.662 INFO:tasks.rados.rados.0.trial115.stdout:2829: done (0 left) 2026-02-17T06:03:04.662 INFO:tasks.rados.rados.0.trial115.stdout:2830: copy_from oid 50 from oid 8 current snap is 289 2026-02-17T06:03:04.662 INFO:tasks.rados.rados.0.trial115.stdout:2831: rollback oid 15 current snap is 289 2026-02-17T06:03:04.662 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 15 to 287 2026-02-17T06:03:04.663 INFO:tasks.rados.rados.0.trial115.stdout:2832: snap_remove snap 282 2026-02-17T06:03:05.661 INFO:tasks.rados.rados.0.trial115.stdout:2832: done (2 left) 2026-02-17T06:03:05.662 INFO:tasks.rados.rados.0.trial115.stdout:2833: write oid 13 current snap is 289 2026-02-17T06:03:05.662 INFO:tasks.rados.rados.0.trial115.stdout:2831: finishing rollback tid 1 to trial11545420-15 2026-02-17T06:03:05.662 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:03:05.662 INFO:tasks.rados.rados.0.trial115.stdout:2833: seq_num 994 ranges {706158=723396,2183219=522939} 2026-02-17T06:03:05.666 INFO:tasks.rados.rados.0.trial115.stdout:2833: writing trial11545420-13 from 706158 to 1429554 tid 1 2026-02-17T06:03:05.672 INFO:tasks.rados.rados.0.trial115.stdout:2833: writing trial11545420-13 from 2183219 to 2706158 tid 2 2026-02-17T06:03:05.673 INFO:tasks.rados.rados.0.trial115.stdout:2831: done (2 left) 2026-02-17T06:03:05.673 INFO:tasks.rados.rados.0.trial115.stdout:2834: snap_create 2026-02-17T06:03:05.673 INFO:tasks.rados.rados.0.trial115.stdout:2830: finishing copy_from racing read to trial11545420-50 2026-02-17T06:03:05.673 INFO:tasks.rados.rados.0.trial115.stdout:2830: finishing copy_from to trial11545420-50 2026-02-17T06:03:05.673 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:03:05.673 INFO:tasks.rados.rados.0.trial115.stdout:2833: finishing write tid 1 to trial11545420-13 2026-02-17T06:03:05.674 INFO:tasks.rados.rados.0.trial115.stdout:2833: finishing write tid 2 to trial11545420-13 2026-02-17T06:03:05.674 INFO:tasks.rados.rados.0.trial115.stdout:2833: finishing write tid 3 to trial11545420-13 2026-02-17T06:03:05.675 INFO:tasks.rados.rados.0.trial115.stdout:2833: finishing write tid 4 to trial11545420-13 2026-02-17T06:03:05.675 INFO:tasks.rados.rados.0.trial115.stdout:2833: oid 13 updating version 0 to 1460 2026-02-17T06:03:05.675 INFO:tasks.rados.rados.0.trial115.stdout:2833: oid 13 version 1460 is already newer than 1458 2026-02-17T06:03:05.675 INFO:tasks.rados.rados.0.trial115.stdout:2833: oid 13 version 1460 is already newer than 1459 2026-02-17T06:03:05.675 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1460 (ObjNum 994 snap 289 seq_num 994) dirty exists 2026-02-17T06:03:05.675 INFO:tasks.rados.rados.0.trial115.stdout:2833: left oid 13 (ObjNum 994 snap 289 seq_num 994) 2026-02-17T06:03:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2830: done (2 left) 2026-02-17T06:03:06.669 INFO:tasks.rados.rados.0.trial115.stdout:2833: done (1 left) 2026-02-17T06:03:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2834: done (0 left) 2026-02-17T06:03:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2835: write (excl) oid 40 current snap is 290 2026-02-17T06:03:06.670 INFO:tasks.rados.rados.0.trial115.stdout:2835: seq_num 995 ranges {722965=798645,2125348=597617} 2026-02-17T06:03:06.675 INFO:tasks.rados.rados.0.trial115.stdout:2835: writing trial11545420-40 from 722965 to 1521610 tid 1 2026-02-17T06:03:06.679 INFO:tasks.rados.rados.0.trial115.stdout:2835: writing trial11545420-40 from 2125348 to 2722965 tid 2 2026-02-17T06:03:06.679 INFO:tasks.rados.rados.0.trial115.stdout:2836: delete oid 17 current snap is 290 2026-02-17T06:03:06.680 INFO:tasks.rados.rados.0.trial115.stdout:2835: finishing write tid 1 to trial11545420-40 2026-02-17T06:03:06.681 INFO:tasks.rados.rados.0.trial115.stdout:2836: done (1 left) 2026-02-17T06:03:06.681 INFO:tasks.rados.rados.0.trial115.stdout:2837: delete oid 23 current snap is 290 2026-02-17T06:03:06.682 INFO:tasks.rados.rados.0.trial115.stdout:2835: finishing write tid 2 to trial11545420-40 2026-02-17T06:03:06.682 INFO:tasks.rados.rados.0.trial115.stdout:2835: finishing write tid 3 to trial11545420-40 2026-02-17T06:03:06.682 INFO:tasks.rados.rados.0.trial115.stdout:2835: finishing write tid 4 to trial11545420-40 2026-02-17T06:03:06.682 INFO:tasks.rados.rados.0.trial115.stdout:2835: oid 40 updating version 0 to 1406 2026-02-17T06:03:06.682 INFO:tasks.rados.rados.0.trial115.stdout:2835: oid 40 version 1406 is already newer than 1405 2026-02-17T06:03:06.682 INFO:tasks.rados.rados.0.trial115.stdout:2835: oid 40 updating version 1406 to 1407 2026-02-17T06:03:06.682 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1407 (ObjNum 995 snap 290 seq_num 995) dirty exists 2026-02-17T06:03:06.682 INFO:tasks.rados.rados.0.trial115.stdout:2835: left oid 40 (ObjNum 995 snap 290 seq_num 995) 2026-02-17T06:03:06.683 INFO:tasks.rados.rados.0.trial115.stdout:2835: done (1 left) 2026-02-17T06:03:06.683 INFO:tasks.rados.rados.0.trial115.stdout:2837: done (0 left) 2026-02-17T06:03:06.683 INFO:tasks.rados.rados.0.trial115.stdout:2838: write oid 20 current snap is 290 2026-02-17T06:03:06.683 INFO:tasks.rados.rados.0.trial115.stdout:2838: seq_num 998 ranges {773386=740745,2017147=544005} 2026-02-17T06:03:06.689 INFO:tasks.rados.rados.0.trial115.stdout:2838: writing trial11545420-20 from 773386 to 1514131 tid 1 2026-02-17T06:03:06.693 INFO:tasks.rados.rados.0.trial115.stdout:2838: writing trial11545420-20 from 2017147 to 2561152 tid 2 2026-02-17T06:03:06.693 INFO:tasks.rados.rados.0.trial115.stdout:2839: delete oid 13 current snap is 290 2026-02-17T06:03:06.693 INFO:tasks.rados.rados.0.trial115.stdout:2838: finishing write tid 1 to trial11545420-20 2026-02-17T06:03:06.694 INFO:tasks.rados.rados.0.trial115.stdout:2838: finishing write tid 2 to trial11545420-20 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2838: finishing write tid 3 to trial11545420-20 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2838: finishing write tid 4 to trial11545420-20 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2838: oid 20 updating version 0 to 1433 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2838: oid 20 version 1433 is already newer than 1432 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2838: oid 20 updating version 1433 to 1434 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1434 (ObjNum 998 snap 290 seq_num 998) dirty exists 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2838: left oid 20 (ObjNum 998 snap 290 seq_num 998) 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2838: done (1 left) 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2839: done (0 left) 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2840: read oid 33 snap -1 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2840: expect (ObjNum 986 snap 282 seq_num 986) 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2841: copy_from oid 41 from oid 49 current snap is 290 2026-02-17T06:03:06.695 INFO:tasks.rados.rados.0.trial115.stdout:2842: delete oid 21 current snap is 290 2026-02-17T06:03:06.696 INFO:tasks.rados.rados.0.trial115.stdout:2841: finishing copy_from to trial11545420-41 2026-02-17T06:03:06.696 INFO:tasks.rados.rados.0.trial115.stdout:2841: got expected ENOENT (src dne) 2026-02-17T06:03:06.697 INFO:tasks.rados.rados.0.trial115.stdout:2841: finishing copy_from racing read to trial11545420-41 2026-02-17T06:03:06.697 INFO:tasks.rados.rados.0.trial115.stdout:2841: done (2 left) 2026-02-17T06:03:06.697 INFO:tasks.rados.rados.0.trial115.stdout:2842: done (1 left) 2026-02-17T06:03:06.697 INFO:tasks.rados.rados.0.trial115.stdout:2843: copy_from oid 37 from oid 24 current snap is 290 2026-02-17T06:03:06.697 INFO:tasks.rados.rados.0.trial115.stdout:2844: write (excl) oid 8 current snap is 290 2026-02-17T06:03:06.697 INFO:tasks.rados.rados.0.trial115.stdout:2844: seq_num 1001 ranges {423807=566492,1676240=407685,2739004=84803} 2026-02-17T06:03:06.700 INFO:tasks.rados.rados.0.trial115.stdout:2844: writing trial11545420-8 from 423807 to 990299 tid 1 2026-02-17T06:03:06.704 INFO:tasks.rados.rados.0.trial115.stdout:2844: writing trial11545420-8 from 1676240 to 2083925 tid 2 2026-02-17T06:03:06.707 INFO:tasks.rados.rados.0.trial115.stdout:2844: writing trial11545420-8 from 2739004 to 2823807 tid 3 2026-02-17T06:03:06.708 INFO:tasks.rados.rados.0.trial115.stdout:2845: snap_create 2026-02-17T06:03:06.708 INFO:tasks.rados.rados.0.trial115.stdout:2840: expect (ObjNum 986 snap 282 seq_num 986) 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: finishing write tid 1 to trial11545420-8 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2843: finishing copy_from racing read to trial11545420-37 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: finishing write tid 2 to trial11545420-8 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2843: finishing copy_from to trial11545420-37 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1429 (ObjNum 923 snap 271 seq_num 923) dirty exists 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: finishing write tid 3 to trial11545420-8 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: finishing write tid 4 to trial11545420-8 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: finishing write tid 5 to trial11545420-8 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: oid 8 updating version 0 to 1141 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: oid 8 updating version 1141 to 1142 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: oid 8 version 1142 is already newer than 1140 2026-02-17T06:03:06.814 INFO:tasks.rados.rados.0.trial115.stdout:2844: oid 8 version 1142 is already newer than 1139 2026-02-17T06:03:06.815 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1142 (ObjNum 1001 snap 290 seq_num 1001) dirty exists 2026-02-17T06:03:06.815 INFO:tasks.rados.rados.0.trial115.stdout:2844: left oid 8 (ObjNum 1001 snap 290 seq_num 1001) 2026-02-17T06:03:07.672 INFO:tasks.rados.rados.0.trial115.stdout:2840: done (3 left) 2026-02-17T06:03:07.673 INFO:tasks.rados.rados.0.trial115.stdout:2843: done (2 left) 2026-02-17T06:03:07.673 INFO:tasks.rados.rados.0.trial115.stdout:2844: done (1 left) 2026-02-17T06:03:07.673 INFO:tasks.rados.rados.0.trial115.stdout:2845: done (0 left) 2026-02-17T06:03:07.673 INFO:tasks.rados.rados.0.trial115.stdout:2846: snap_create 2026-02-17T06:03:08.660 INFO:tasks.rados.rados.0.trial115.stdout:2846: done (0 left) 2026-02-17T06:03:08.660 INFO:tasks.rados.rados.0.trial115.stdout:2847: delete oid 48 current snap is 292 2026-02-17T06:03:08.663 INFO:tasks.rados.rados.0.trial115.stdout:2847: done (0 left) 2026-02-17T06:03:08.663 INFO:tasks.rados.rados.0.trial115.stdout:2848: delete oid 21 current snap is 292 2026-02-17T06:03:08.665 INFO:tasks.rados.rados.0.trial115.stdout:2848: done (0 left) 2026-02-17T06:03:08.665 INFO:tasks.rados.rados.0.trial115.stdout:2849: delete oid 34 current snap is 292 2026-02-17T06:03:08.667 INFO:tasks.rados.rados.0.trial115.stdout:2849: done (0 left) 2026-02-17T06:03:08.667 INFO:tasks.rados.rados.0.trial115.stdout:2850: snap_create 2026-02-17T06:03:09.663 INFO:tasks.rados.rados.0.trial115.stdout:2850: done (0 left) 2026-02-17T06:03:09.663 INFO:tasks.rados.rados.0.trial115.stdout:2851: snap_create 2026-02-17T06:03:11.326 INFO:tasks.rados.rados.0.trial115.stdout:2851: done (0 left) 2026-02-17T06:03:11.327 INFO:tasks.rados.rados.0.trial115.stdout:2852: write oid 38 current snap is 294 2026-02-17T06:03:11.327 INFO:tasks.rados.rados.0.trial115.stdout:2852: seq_num 1005 ranges {491035=583841,1510168=698670,2732343=158692} 2026-02-17T06:03:11.331 INFO:tasks.rados.rados.0.trial115.stdout:2852: writing trial11545420-38 from 491035 to 1074876 tid 1 2026-02-17T06:03:11.335 INFO:tasks.rados.rados.0.trial115.stdout:2852: writing trial11545420-38 from 1510168 to 2208838 tid 2 2026-02-17T06:03:11.338 INFO:tasks.rados.rados.0.trial115.stdout:2852: writing trial11545420-38 from 2732343 to 2891035 tid 3 2026-02-17T06:03:11.338 INFO:tasks.rados.rados.0.trial115.stdout:2853: snap_remove snap 285 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2853: done (1 left) 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2854: copy_from oid 11 from oid 18 current snap is 294 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2852: finishing write tid 1 to trial11545420-38 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2852: finishing write tid 2 to trial11545420-38 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2852: finishing write tid 3 to trial11545420-38 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2852: finishing write tid 4 to trial11545420-38 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2852: finishing write tid 5 to trial11545420-38 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2852: oid 38 updating version 0 to 1527 2026-02-17T06:03:11.669 INFO:tasks.rados.rados.0.trial115.stdout:2852: oid 38 updating version 1527 to 1528 2026-02-17T06:03:11.670 INFO:tasks.rados.rados.0.trial115.stdout:2852: oid 38 updating version 1528 to 1529 2026-02-17T06:03:11.670 INFO:tasks.rados.rados.0.trial115.stdout:2852: oid 38 version 1529 is already newer than 1526 2026-02-17T06:03:11.670 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1529 (ObjNum 1005 snap 294 seq_num 1005) dirty exists 2026-02-17T06:03:11.670 INFO:tasks.rados.rados.0.trial115.stdout:2852: left oid 38 (ObjNum 1005 snap 294 seq_num 1005) 2026-02-17T06:03:11.670 INFO:tasks.rados.rados.0.trial115.stdout:2852: done (1 left) 2026-02-17T06:03:11.670 INFO:tasks.rados.rados.0.trial115.stdout:2855: write oid 50 current snap is 294 2026-02-17T06:03:11.670 INFO:tasks.rados.rados.0.trial115.stdout:2855: seq_num 1006 ranges {507842=659090,1568650=558681,2759766=148076} 2026-02-17T06:03:11.673 INFO:tasks.rados.rados.0.trial115.stdout:2855: writing trial11545420-50 from 507842 to 1166932 tid 1 2026-02-17T06:03:11.676 INFO:tasks.rados.rados.0.trial115.stdout:2855: writing trial11545420-50 from 1568650 to 2127331 tid 2 2026-02-17T06:03:11.680 INFO:tasks.rados.rados.0.trial115.stdout:2855: writing trial11545420-50 from 2759766 to 2907842 tid 3 2026-02-17T06:03:11.680 INFO:tasks.rados.rados.0.trial115.stdout:2856: read oid 21 snap 284 2026-02-17T06:03:11.680 INFO:tasks.rados.rados.0.trial115.stdout:2856: expect (ObjNum 969 snap 280 seq_num 969) 2026-02-17T06:03:11.680 INFO:tasks.rados.rados.0.trial115.stdout:2857: copy_from oid 36 from oid 9 current snap is 294 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:2855: finishing write tid 1 to trial11545420-50 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:2854: finishing copy_from racing read to trial11545420-11 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:2855: finishing write tid 2 to trial11545420-50 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:2854: finishing copy_from to trial11545420-11 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1411 (ObjNum 991 snap 284 seq_num 991) dirty exists 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:2854: done (3 left) 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:2858: rollback oid 26 current snap is 294 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 26 to 290 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:2859: write oid 12 current snap is 294 2026-02-17T06:03:11.681 INFO:tasks.rados.rados.0.trial115.stdout:2859: seq_num 1007 ranges {524649=734339,1910779=702339} 2026-02-17T06:03:11.685 INFO:tasks.rados.rados.0.trial115.stdout:2859: writing trial11545420-12 from 524649 to 1258988 tid 1 2026-02-17T06:03:11.691 INFO:tasks.rados.rados.0.trial115.stdout:2859: writing trial11545420-12 from 1910779 to 2613118 tid 2 2026-02-17T06:03:11.691 INFO:tasks.rados.rados.0.trial115.stdout:2860: copy_from oid 32 from oid 44 current snap is 294 2026-02-17T06:03:11.691 INFO:tasks.rados.rados.0.trial115.stdout:2858: finishing rollback tid 1 to trial11545420-26 2026-02-17T06:03:11.692 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 0 (ObjNum 129 snap 0 seq_num 1334259152) dirty dne 2026-02-17T06:03:11.692 INFO:tasks.rados.rados.0.trial115.stdout:2855: finishing write tid 3 to trial11545420-50 2026-02-17T06:03:11.692 INFO:tasks.rados.rados.0.trial115.stdout:2855: finishing write tid 4 to trial11545420-50 2026-02-17T06:03:11.692 INFO:tasks.rados.rados.0.trial115.stdout:2858: done (5 left) 2026-02-17T06:03:11.692 INFO:tasks.rados.rados.0.trial115.stdout:2861: snap_remove snap 288 2026-02-17T06:03:12.672 INFO:tasks.rados.rados.0.trial115.stdout:2861: done (5 left) 2026-02-17T06:03:12.672 INFO:tasks.rados.rados.0.trial115.stdout:2862: copy_from oid 34 from oid 22 current snap is 294 2026-02-17T06:03:12.672 INFO:tasks.rados.rados.0.trial115.stdout:2855: finishing write tid 5 to trial11545420-50 2026-02-17T06:03:12.672 INFO:tasks.rados.rados.0.trial115.stdout:2855: oid 50 updating version 0 to 1190 2026-02-17T06:03:12.673 INFO:tasks.rados.rados.0.trial115.stdout:2855: oid 50 version 1190 is already newer than 1189 2026-02-17T06:03:12.673 INFO:tasks.rados.rados.0.trial115.stdout:2855: oid 50 version 1190 is already newer than 1187 2026-02-17T06:03:12.673 INFO:tasks.rados.rados.0.trial115.stdout:2855: oid 50 version 1190 is already newer than 1188 2026-02-17T06:03:12.673 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1190 (ObjNum 1006 snap 294 seq_num 1006) dirty exists 2026-02-17T06:03:12.673 INFO:tasks.rados.rados.0.trial115.stdout:2855: left oid 50 (ObjNum 1006 snap 294 seq_num 1006) 2026-02-17T06:03:12.673 INFO:tasks.rados.rados.0.trial115.stdout:2856: expect (ObjNum 969 snap 280 seq_num 969) 2026-02-17T06:03:12.779 INFO:tasks.rados.rados.0.trial115.stdout:2857: finishing copy_from racing read to trial11545420-36 2026-02-17T06:03:12.779 INFO:tasks.rados.rados.0.trial115.stdout:2857: finishing copy_from to trial11545420-36 2026-02-17T06:03:12.779 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 1421 (ObjNum 982 snap 282 seq_num 982) dirty exists 2026-02-17T06:03:12.779 INFO:tasks.rados.rados.0.trial115.stdout:2859: finishing write tid 1 to trial11545420-12 2026-02-17T06:03:12.780 INFO:tasks.rados.rados.0.trial115.stdout:2859: finishing write tid 2 to trial11545420-12 2026-02-17T06:03:12.780 INFO:tasks.rados.rados.0.trial115.stdout:2859: finishing write tid 3 to trial11545420-12 2026-02-17T06:03:12.780 INFO:tasks.rados.rados.0.trial115.stdout:2859: finishing write tid 4 to trial11545420-12 2026-02-17T06:03:12.780 INFO:tasks.rados.rados.0.trial115.stdout:2859: oid 12 updating version 0 to 1492 2026-02-17T06:03:12.780 INFO:tasks.rados.rados.0.trial115.stdout:2859: oid 12 version 1492 is already newer than 1491 2026-02-17T06:03:12.780 INFO:tasks.rados.rados.0.trial115.stdout:2859: oid 12 version 1492 is already newer than 1490 2026-02-17T06:03:12.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1492 (ObjNum 1007 snap 294 seq_num 1007) dirty exists 2026-02-17T06:03:12.781 INFO:tasks.rados.rados.0.trial115.stdout:2859: left oid 12 (ObjNum 1007 snap 294 seq_num 1007) 2026-02-17T06:03:12.781 INFO:tasks.rados.rados.0.trial115.stdout:2860: finishing copy_from racing read to trial11545420-32 2026-02-17T06:03:12.781 INFO:tasks.rados.rados.0.trial115.stdout:2860: finishing copy_from to trial11545420-32 2026-02-17T06:03:12.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:03:12.781 INFO:tasks.rados.rados.0.trial115.stdout:2862: finishing copy_from racing read to trial11545420-34 2026-02-17T06:03:12.782 INFO:tasks.rados.rados.0.trial115.stdout:2862: finishing copy_from to trial11545420-34 2026-02-17T06:03:12.782 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1138 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:03:12.782 INFO:tasks.rados.rados.0.trial115.stdout:2855: done (5 left) 2026-02-17T06:03:12.782 INFO:tasks.rados.rados.0.trial115.stdout:2856: done (4 left) 2026-02-17T06:03:12.782 INFO:tasks.rados.rados.0.trial115.stdout:2857: done (3 left) 2026-02-17T06:03:12.782 INFO:tasks.rados.rados.0.trial115.stdout:2859: done (2 left) 2026-02-17T06:03:12.783 INFO:tasks.rados.rados.0.trial115.stdout:2860: done (1 left) 2026-02-17T06:03:12.783 INFO:tasks.rados.rados.0.trial115.stdout:2862: done (0 left) 2026-02-17T06:03:12.783 INFO:tasks.rados.rados.0.trial115.stdout:2863: rollback oid 46 current snap is 294 2026-02-17T06:03:12.783 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 46 to 292 2026-02-17T06:03:12.783 INFO:tasks.rados.rados.0.trial115.stdout:2864: read oid 1 snap 289 2026-02-17T06:03:12.783 INFO:tasks.rados.rados.0.trial115.stdout:2864: expect deleted 2026-02-17T06:03:12.784 INFO:tasks.rados.rados.0.trial115.stdout:2865: read oid 2 snap -1 2026-02-17T06:03:12.784 INFO:tasks.rados.rados.0.trial115.stdout:2865: expect (ObjNum 978 snap 280 seq_num 978) 2026-02-17T06:03:12.784 INFO:tasks.rados.rados.0.trial115.stdout:2866: write (excl) oid 5 current snap is 294 2026-02-17T06:03:12.784 INFO:tasks.rados.rados.0.trial115.stdout:2866: seq_num 1008 ranges {541456=409588,1569261=562350,2814612=126844} 2026-02-17T06:03:12.785 INFO:tasks.rados.rados.0.trial115.stdout:2866: writing trial11545420-5 from 541456 to 951044 tid 1 2026-02-17T06:03:12.786 INFO:tasks.rados.rados.0.trial115.stdout:2866: writing trial11545420-5 from 1569261 to 2131611 tid 2 2026-02-17T06:03:12.788 INFO:tasks.rados.rados.0.trial115.stdout:2866: writing trial11545420-5 from 2814612 to 2941456 tid 3 2026-02-17T06:03:12.789 INFO:tasks.rados.rados.0.trial115.stdout:2867: copy_from oid 30 from oid 13 current snap is 294 2026-02-17T06:03:12.789 INFO:tasks.rados.rados.0.trial115.stdout:2863: finishing rollback tid 0 to trial11545420-46 2026-02-17T06:03:12.789 INFO:tasks.rados.rados.0.trial115.stdout:2865: expect (ObjNum 978 snap 280 seq_num 978) 2026-02-17T06:03:12.865 INFO:tasks.rados.rados.0.trial115.stdout:2863: finishing rollback tid 1 to trial11545420-46 2026-02-17T06:03:12.865 INFO:tasks.rados.rados.0.trial115.stdout:2863: finishing rollback tid 2 to trial11545420-46 2026-02-17T06:03:12.865 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1427 (ObjNum 976 snap 280 seq_num 976) dirty exists 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2866: finishing write tid 1 to trial11545420-5 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2866: finishing write tid 2 to trial11545420-5 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2863: done (4 left) 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2864: done (3 left) 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2865: done (2 left) 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2868: copy_from oid 10 from oid 7 current snap is 294 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2867: finishing copy_from to trial11545420-30 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2867: got expected ENOENT (src dne) 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2867: finishing copy_from racing read to trial11545420-30 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2866: finishing write tid 3 to trial11545420-5 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2866: finishing write tid 4 to trial11545420-5 2026-02-17T06:03:12.866 INFO:tasks.rados.rados.0.trial115.stdout:2866: finishing write tid 5 to trial11545420-5 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2866: oid 5 updating version 0 to 1433 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2866: oid 5 updating version 1433 to 1435 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2866: oid 5 version 1435 is already newer than 1432 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2866: oid 5 version 1435 is already newer than 1434 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1435 (ObjNum 1008 snap 294 seq_num 1008) dirty exists 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2866: left oid 5 (ObjNum 1008 snap 294 seq_num 1008) 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2866: done (2 left) 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2867: done (1 left) 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2869: rollback oid 25 current snap is 294 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 287 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2870: read oid 17 snap 293 2026-02-17T06:03:12.867 INFO:tasks.rados.rados.0.trial115.stdout:2870: expect deleted 2026-02-17T06:03:12.868 INFO:tasks.rados.rados.0.trial115.stdout:2871: write (excl) oid 15 current snap is 294 2026-02-17T06:03:12.868 INFO:tasks.rados.rados.0.trial115.stdout:2871: seq_num 1009 ranges {558263=484837,1627743=706008} 2026-02-17T06:03:12.869 INFO:tasks.rados.rados.0.trial115.stdout:2871: writing trial11545420-15 from 558263 to 1043100 tid 1 2026-02-17T06:03:12.874 INFO:tasks.rados.rados.0.trial115.stdout:2871: writing trial11545420-15 from 1627743 to 2333751 tid 2 2026-02-17T06:03:12.874 INFO:tasks.rados.rados.0.trial115.stdout:2872: write oid 11 current snap is 294 2026-02-17T06:03:12.874 INFO:tasks.rados.rados.0.trial115.stdout:2872: seq_num 1010 ranges {575070=560086,1686225=566019,2869458=105612} 2026-02-17T06:03:12.878 INFO:tasks.rados.rados.0.trial115.stdout:2872: writing trial11545420-11 from 575070 to 1135156 tid 1 2026-02-17T06:03:12.882 INFO:tasks.rados.rados.0.trial115.stdout:2872: writing trial11545420-11 from 1686225 to 2252244 tid 2 2026-02-17T06:03:12.885 INFO:tasks.rados.rados.0.trial115.stdout:2872: writing trial11545420-11 from 2869458 to 2975070 tid 3 2026-02-17T06:03:12.885 INFO:tasks.rados.rados.0.trial115.stdout:2873: write oid 34 current snap is 294 2026-02-17T06:03:12.885 INFO:tasks.rados.rados.0.trial115.stdout:2873: seq_num 1011 ranges {591877=635335,1628354=709677} 2026-02-17T06:03:12.889 INFO:tasks.rados.rados.0.trial115.stdout:2873: writing trial11545420-34 from 591877 to 1227212 tid 1 2026-02-17T06:03:12.894 INFO:tasks.rados.rados.0.trial115.stdout:2873: writing trial11545420-34 from 1628354 to 2338031 tid 2 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2874: copy_from oid 20 from oid 6 current snap is 294 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2869: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2869: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2869: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1415 (ObjNum 972 snap 280 seq_num 972) dirty exists 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2868: finishing copy_from racing read to trial11545420-10 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2868: finishing copy_from to trial11545420-10 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1418 (ObjNum 888 snap 254 seq_num 888) dirty exists 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2871: finishing write tid 1 to trial11545420-15 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2871: finishing write tid 2 to trial11545420-15 2026-02-17T06:03:12.895 INFO:tasks.rados.rados.0.trial115.stdout:2871: finishing write tid 3 to trial11545420-15 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2871: finishing write tid 4 to trial11545420-15 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2871: oid 15 updating version 0 to 1429 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2871: oid 15 version 1429 is already newer than 1428 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2871: oid 15 updating version 1429 to 1430 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1430 (ObjNum 1009 snap 294 seq_num 1009) dirty exists 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2871: left oid 15 (ObjNum 1009 snap 294 seq_num 1009) 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2872: finishing write tid 1 to trial11545420-11 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2872: finishing write tid 2 to trial11545420-11 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2872: finishing write tid 3 to trial11545420-11 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2872: finishing write tid 4 to trial11545420-11 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2872: finishing write tid 5 to trial11545420-11 2026-02-17T06:03:12.896 INFO:tasks.rados.rados.0.trial115.stdout:2872: oid 11 updating version 0 to 1434 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2872: oid 11 version 1434 is already newer than 1431 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2872: oid 11 version 1434 is already newer than 1432 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2872: oid 11 version 1434 is already newer than 1433 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1434 (ObjNum 1010 snap 294 seq_num 1010) dirty exists 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2872: left oid 11 (ObjNum 1010 snap 294 seq_num 1010) 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2873: finishing write tid 1 to trial11545420-34 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2868: done (6 left) 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2869: done (5 left) 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2870: done (4 left) 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2871: done (3 left) 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2872: done (2 left) 2026-02-17T06:03:12.897 INFO:tasks.rados.rados.0.trial115.stdout:2875: snap_remove snap 291 2026-02-17T06:03:13.662 INFO:tasks.rados.rados.0.trial115.stdout:2875: done (2 left) 2026-02-17T06:03:13.662 INFO:tasks.rados.rados.0.trial115.stdout:2876: write (excl) oid 30 current snap is 294 2026-02-17T06:03:13.662 INFO:tasks.rados.rados.0.trial115.stdout:2873: finishing write tid 2 to trial11545420-34 2026-02-17T06:03:13.662 INFO:tasks.rados.rados.0.trial115.stdout:2876: seq_num 1012 ranges {608684=426937,1803189=569688,2924304=84380} 2026-02-17T06:03:13.667 INFO:tasks.rados.rados.0.trial115.stdout:2876: writing trial11545420-30 from 608684 to 1035621 tid 1 2026-02-17T06:03:13.671 INFO:tasks.rados.rados.0.trial115.stdout:2876: writing trial11545420-30 from 1803189 to 2372877 tid 2 2026-02-17T06:03:13.674 INFO:tasks.rados.rados.0.trial115.stdout:2876: writing trial11545420-30 from 2924304 to 3008684 tid 3 2026-02-17T06:03:13.674 INFO:tasks.rados.rados.0.trial115.stdout:2877: snap_remove snap 284 2026-02-17T06:03:14.998 INFO:tasks.rados.rados.0.trial115.stdout:2877: done (3 left) 2026-02-17T06:03:14.998 INFO:tasks.rados.rados.0.trial115.stdout:2878: write (excl) oid 26 current snap is 294 2026-02-17T06:03:14.998 INFO:tasks.rados.rados.0.trial115.stdout:2873: finishing write tid 3 to trial11545420-34 2026-02-17T06:03:14.998 INFO:tasks.rados.rados.0.trial115.stdout:2873: finishing write tid 4 to trial11545420-34 2026-02-17T06:03:14.999 INFO:tasks.rados.rados.0.trial115.stdout:2873: oid 34 updating version 0 to 1139 2026-02-17T06:03:14.999 INFO:tasks.rados.rados.0.trial115.stdout:2873: oid 34 updating version 1139 to 1141 2026-02-17T06:03:14.999 INFO:tasks.rados.rados.0.trial115.stdout:2873: oid 34 version 1141 is already newer than 1140 2026-02-17T06:03:14.999 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1141 (ObjNum 1011 snap 294 seq_num 1011) dirty exists 2026-02-17T06:03:14.999 INFO:tasks.rados.rados.0.trial115.stdout:2873: left oid 34 (ObjNum 1011 snap 294 seq_num 1011) 2026-02-17T06:03:14.999 INFO:tasks.rados.rados.0.trial115.stdout:2874: finishing copy_from racing read to trial11545420-20 2026-02-17T06:03:14.999 INFO:tasks.rados.rados.0.trial115.stdout:2874: finishing copy_from to trial11545420-20 2026-02-17T06:03:14.999 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 970 (ObjNum 935 snap 274 seq_num 935) dirty exists 2026-02-17T06:03:15.000 INFO:tasks.rados.rados.0.trial115.stdout:2876: finishing write tid 1 to trial11545420-30 2026-02-17T06:03:15.000 INFO:tasks.rados.rados.0.trial115.stdout:2876: finishing write tid 2 to trial11545420-30 2026-02-17T06:03:15.000 INFO:tasks.rados.rados.0.trial115.stdout:2876: finishing write tid 3 to trial11545420-30 2026-02-17T06:03:15.000 INFO:tasks.rados.rados.0.trial115.stdout:2878: seq_num 1013 ranges {625491=502186,1861671=713346} 2026-02-17T06:03:15.002 INFO:tasks.rados.rados.0.trial115.stdout:2878: writing trial11545420-26 from 625491 to 1127677 tid 1 2026-02-17T06:03:15.008 INFO:tasks.rados.rados.0.trial115.stdout:2878: writing trial11545420-26 from 1861671 to 2575017 tid 2 2026-02-17T06:03:15.008 INFO:tasks.rados.rados.0.trial115.stdout:2873: done (3 left) 2026-02-17T06:03:15.008 INFO:tasks.rados.rados.0.trial115.stdout:2874: done (2 left) 2026-02-17T06:03:15.008 INFO:tasks.rados.rados.0.trial115.stdout:2879: read oid 47 snap -1 2026-02-17T06:03:15.008 INFO:tasks.rados.rados.0.trial115.stdout:2879: expect deleted 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2880: read oid 40 snap -1 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2880: expect (ObjNum 995 snap 290 seq_num 995) 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2881: write (excl) oid 16 current snap is 294 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2876: finishing write tid 4 to trial11545420-30 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2876: finishing write tid 5 to trial11545420-30 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2876: oid 30 updating version 0 to 1194 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2876: oid 30 version 1194 is already newer than 1191 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2876: oid 30 version 1194 is already newer than 1193 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2876: oid 30 version 1194 is already newer than 1192 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1194 (ObjNum 1012 snap 294 seq_num 1012) dirty exists 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2876: left oid 30 (ObjNum 1012 snap 294 seq_num 1012) 2026-02-17T06:03:15.009 INFO:tasks.rados.rados.0.trial115.stdout:2878: finishing write tid 1 to trial11545420-26 2026-02-17T06:03:15.010 INFO:tasks.rados.rados.0.trial115.stdout:2881: seq_num 1014 ranges {642298=577435,1920153=573357,2979150=63148} 2026-02-17T06:03:15.012 INFO:tasks.rados.rados.0.trial115.stdout:2881: writing trial11545420-16 from 642298 to 1219733 tid 1 2026-02-17T06:03:15.017 INFO:tasks.rados.rados.0.trial115.stdout:2881: writing trial11545420-16 from 1920153 to 2493510 tid 2 2026-02-17T06:03:15.020 INFO:tasks.rados.rados.0.trial115.stdout:2881: writing trial11545420-16 from 2979150 to 3042298 tid 3 2026-02-17T06:03:15.020 INFO:tasks.rados.rados.0.trial115.stdout:2876: done (4 left) 2026-02-17T06:03:15.020 INFO:tasks.rados.rados.0.trial115.stdout:2882: copy_from oid 22 from oid 28 current snap is 294 2026-02-17T06:03:15.020 INFO:tasks.rados.rados.0.trial115.stdout:2878: finishing write tid 2 to trial11545420-26 2026-02-17T06:03:15.020 INFO:tasks.rados.rados.0.trial115.stdout:2878: finishing write tid 3 to trial11545420-26 2026-02-17T06:03:15.020 INFO:tasks.rados.rados.0.trial115.stdout:2879: done (4 left) 2026-02-17T06:03:15.021 INFO:tasks.rados.rados.0.trial115.stdout:2883: read oid 25 snap -1 2026-02-17T06:03:15.021 INFO:tasks.rados.rados.0.trial115.stdout:2883: expect (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:03:15.021 INFO:tasks.rados.rados.0.trial115.stdout:2884: snap_remove snap 292 2026-02-17T06:03:16.003 INFO:tasks.rados.rados.0.trial115.stdout:2884: done (5 left) 2026-02-17T06:03:16.003 INFO:tasks.rados.rados.0.trial115.stdout:2885: read oid 42 snap -1 2026-02-17T06:03:16.003 INFO:tasks.rados.rados.0.trial115.stdout:2885: expect (ObjNum 904 snap 262 seq_num 904) 2026-02-17T06:03:16.003 INFO:tasks.rados.rados.0.trial115.stdout:2886: snap_create 2026-02-17T06:03:16.003 INFO:tasks.rados.rados.0.trial115.stdout:2878: finishing write tid 4 to trial11545420-26 2026-02-17T06:03:16.004 INFO:tasks.rados.rados.0.trial115.stdout:2878: oid 26 updating version 0 to 968 2026-02-17T06:03:16.004 INFO:tasks.rados.rados.0.trial115.stdout:2878: oid 26 updating version 968 to 970 2026-02-17T06:03:16.004 INFO:tasks.rados.rados.0.trial115.stdout:2878: oid 26 version 970 is already newer than 969 2026-02-17T06:03:16.004 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 970 (ObjNum 1013 snap 294 seq_num 1013) dirty exists 2026-02-17T06:03:16.004 INFO:tasks.rados.rados.0.trial115.stdout:2878: left oid 26 (ObjNum 1013 snap 294 seq_num 1013) 2026-02-17T06:03:16.004 INFO:tasks.rados.rados.0.trial115.stdout:2880: expect (ObjNum 995 snap 290 seq_num 995) 2026-02-17T06:03:16.099 INFO:tasks.rados.rados.0.trial115.stdout:2881: finishing write tid 1 to trial11545420-16 2026-02-17T06:03:16.099 INFO:tasks.rados.rados.0.trial115.stdout:2881: finishing write tid 2 to trial11545420-16 2026-02-17T06:03:16.099 INFO:tasks.rados.rados.0.trial115.stdout:2881: finishing write tid 3 to trial11545420-16 2026-02-17T06:03:16.099 INFO:tasks.rados.rados.0.trial115.stdout:2881: finishing write tid 4 to trial11545420-16 2026-02-17T06:03:16.099 INFO:tasks.rados.rados.0.trial115.stdout:2881: finishing write tid 5 to trial11545420-16 2026-02-17T06:03:16.100 INFO:tasks.rados.rados.0.trial115.stdout:2881: oid 16 updating version 0 to 1552 2026-02-17T06:03:16.100 INFO:tasks.rados.rados.0.trial115.stdout:2881: oid 16 updating version 1552 to 1554 2026-02-17T06:03:16.100 INFO:tasks.rados.rados.0.trial115.stdout:2881: oid 16 version 1554 is already newer than 1545 2026-02-17T06:03:16.100 INFO:tasks.rados.rados.0.trial115.stdout:2881: oid 16 version 1554 is already newer than 1553 2026-02-17T06:03:16.100 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1554 (ObjNum 1014 snap 294 seq_num 1014) dirty exists 2026-02-17T06:03:16.100 INFO:tasks.rados.rados.0.trial115.stdout:2881: left oid 16 (ObjNum 1014 snap 294 seq_num 1014) 2026-02-17T06:03:16.100 INFO:tasks.rados.rados.0.trial115.stdout:2883: expect (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:03:16.197 INFO:tasks.rados.rados.0.trial115.stdout:2882: finishing copy_from racing read to trial11545420-22 2026-02-17T06:03:16.197 INFO:tasks.rados.rados.0.trial115.stdout:2882: finishing copy_from to trial11545420-22 2026-02-17T06:03:16.198 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1326 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:03:16.198 INFO:tasks.rados.rados.0.trial115.stdout:2885: expect (ObjNum 904 snap 262 seq_num 904) 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2878: done (6 left) 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2880: done (5 left) 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2881: done (4 left) 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2882: done (3 left) 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2883: done (2 left) 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2885: done (1 left) 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2886: done (0 left) 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2887: copy_from oid 32 from oid 18 current snap is 295 2026-02-17T06:03:17.004 INFO:tasks.rados.rados.0.trial115.stdout:2888: snap_create 2026-02-17T06:03:17.011 INFO:tasks.rados.rados.0.trial115.stdout:2887: finishing copy_from racing read to trial11545420-32 2026-02-17T06:03:17.011 INFO:tasks.rados.rados.0.trial115.stdout:2887: finishing copy_from to trial11545420-32 2026-02-17T06:03:17.011 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1411 (ObjNum 991 snap 284 seq_num 991) dirty exists 2026-02-17T06:03:17.088 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:03:17.088 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T06:03:17.088 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T06:03:17.088 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T06:03:17.091 INFO:tasks.ceph.mon.b:Started 2026-02-17T06:03:17.092 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:03:17.092 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:03:17.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.158+0000 7f66aa287640 10 monclient: get_monmap_and_config 2026-02-17T06:03:17.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.158+0000 7f66aa287640 10 monclient: build_initial_monmap 2026-02-17T06:03:17.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.158+0000 7f66aa287640 10 monclient: monmap: 2026-02-17T06:03:17.162 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:17.162 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:17.162 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:17.160090+0000 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:17.160090+0000 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 1 Processor -- start 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 1 -- start start 2026-02-17T06:03:17.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient: init 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f66a409ea90 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a409ee70 0x7f66a409f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient(hunting): picked mon.noname-b con 0x7f66a409ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f66a413d640 0x7f66a413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient(hunting): picked mon.noname-c con 0x7f66a413d640 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a40a0a20 con 0x7f66a409ea90 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f66a40a0d00 con 0x7f66a413d640 2026-02-17T06:03:17.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f66a40a0460 con 0x7f66a409ee70 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66aa287640 10 monclient(hunting): authenticate will time out at 1642.713650s 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a409ee70 0x7f66a409f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a409ee70 0x7f66a409f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36968/0 (socket says 10.20.193.78:36968) 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 1 -- 10.20.193.78:0/4213789474 learned_addr learned my addr 10.20.193.78:0/4213789474 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient(hunting): get_auth_request con 0x7f66a409ee70 auth_method 0 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:17.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a3fff640 1 --2- 10.20.193.78:0/4213789474 >> v2:10.20.193.115:3300/0 conn(0x7f66a413d640 0x7f66a413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a3fff640 10 monclient(hunting): get_auth_request con 0x7f66a413d640 auth_method 0 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient(hunting): handle_auth_done global_id 144746 payload 274 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 1 -- 10.20.193.78:0/4213789474 >> v2:10.20.193.115:3300/0 conn(0x7f66a413d640 msgr2=0x7f66a413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 1 --2- 10.20.193.78:0/4213789474 >> v2:10.20.193.115:3300/0 conn(0x7f66a413d640 0x7f66a413fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 1 -- 10.20.193.78:0/4213789474 >> v1:10.20.193.115:6789/0 conn(0x7f66a409ea90 legacy=0x7f66a40a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:03:17.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 1 monclient: found mon.noname-b 2026-02-17T06:03:17.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:17.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:03:17.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.159+0000 7f66a2ffd640 1 -- 10.20.193.78:0/4213789474 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a40a0740 con 0x7f66a409ee70 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a2ffd640 1 -- 10.20.193.78:0/4213789474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a409ee70 msgr2=0x7f66a409f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a2ffd640 1 --2- 10.20.193.78:0/4213789474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a409ee70 0x7f66a409f290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6694009970 tx=0x7f669402ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b10c1bf05fcee7d1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 1 -- 10.20.193.78:0/4213789474 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669403c070 con 0x7f66a409ee70 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: dump: 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:17.174 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: _finish_auth 0 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: _check_auth_tickets 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 1 -- 10.20.193.78:0/4213789474 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f669402fcd0 con 0x7f66a409ee70 2026-02-17T06:03:17.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66aa287640 5 monclient: authenticate success, global_id 144746 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66aa287640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 1 -- 10.20.193.78:0/4213789474 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694037660 con 0x7f66a409ee70 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66aa287640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66aa287640 4 set_mon_vals no callback set 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66a27fc640 10 monclient: dump: 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:17.176 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66aa287640 10 monclient: shutdown 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66aa287640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66aa287640 1 -- 10.20.193.78:0/4213789474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a409ee70 msgr2=0x7f66a409f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.162+0000 7f66aa287640 1 --2- 10.20.193.78:0/4213789474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a409ee70 0x7f66a409f290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6694009970 tx=0x7f669402ee90 comp rx=0 tx=0).stop 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 -- 10.20.193.78:0/4213789474 shutdown_connections 2026-02-17T06:03:17.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 --2- 10.20.193.78:0/4213789474 >> v2:10.20.193.115:3300/0 conn(0x7f66a413d640 0x7f66a413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 --2- 10.20.193.78:0/4213789474 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a409ee70 0x7f66a409f290 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 -- 10.20.193.78:0/4213789474 >> 10.20.193.78:0/4213789474 conn(0x7f66a40a7520 msgr2=0x7f66a40a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 -- 10.20.193.78:0/4213789474 shutdown_connections 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 -- 10.20.193.78:0/4213789474 wait complete. 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient: build_initial_monmap 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient: monmap: 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:17.178 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 Processor -- start 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 -- start start 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient: init 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f66a409ea90 0x7f66a40a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient(hunting): picked mon.noname-a con 0x7f66a409ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f66a409ee70 0x7f66a40a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:17.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient(hunting): picked mon.noname-b con 0x7f66a409ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a413d640 0x7f66a40a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient(hunting): picked mon.noname-c con 0x7f66a413d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66a40a6820 con 0x7f66a409ea90 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66a40a66a0 con 0x7f66a409ee70 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f66a413ff70 con 0x7f66a413d640 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66aa287640 10 monclient(hunting): authenticate will time out at 1642.717858s 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f66a409ea90 0x7f66a40a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:17.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a413d640 0x7f66a40a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f66a409ea90 0x7f66a40a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58434/0 (socket says 10.20.193.78:58434) 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 1 -- 10.20.193.78:0/2379302584 learned_addr learned my addr 10.20.193.78:0/2379302584 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a3fff640 10 monclient(hunting): get_auth_request con 0x7f66a413d640 auth_method 0 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 10 monclient(hunting): get_auth_request con 0x7f66a409ea90 auth_method 0 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:17.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.163+0000 7f66a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 10 monclient(hunting): handle_auth_done global_id 144749 payload 824 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 1 -- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f66a409ee70 msgr2=0x7f66a40a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a2ffd640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f66a409ee70 0x7f66a40a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f66a409ee70 0x7f66a40a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 1 -- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f66a409ea90 msgr2=0x7f66a40a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f66a409ea90 0x7f66a40a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 10 monclient: _finish_hunting 0 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 1 monclient: found mon.noname-c 2026-02-17T06:03:17.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a3fff640 1 -- 10.20.193.78:0/2379302584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a40711e0 con 0x7f66a413d640 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a2ffd640 10 monclient: get_auth_request con 0x7f66a409ee70 auth_method 0 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a2ffd640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f66a409ee70 0x7f66a40a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.164+0000 7f66a37fe640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f66a409ea90 0x7f66a40a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66a3fff640 1 -- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a413d640 msgr2=0x7f66a40a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66a3fff640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a413d640 0x7f66a40a3e90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6698007d60 tx=0x7f669800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 1 -- 10.20.193.78:0/2379302584 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698003a90 con 0x7f66a413d640 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: dump: 2026-02-17T06:03:17.183 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: _finish_auth 0 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:03:17.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 1 -- 10.20.193.78:0/2379302584 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6698003c50 con 0x7f66a413d640 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 1 -- 10.20.193.78:0/2379302584 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698004020 con 0x7f66a413d640 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66a1ffb640 4 set_mon_vals no callback set 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f667ffff640 10 monclient: dump: 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:17.185 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 5 monclient: authenticate success, global_id 144749 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 10 monclient: _renew_subs 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a40a1c10 con 0x7f66a413d640 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 10 monclient: _renew_subs 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a4070940 con 0x7f66a413d640 2026-02-17T06:03:17.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.166+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a409f290 con 0x7f66a413d640 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.167+0000 7f667ffff640 1 -- 10.20.193.78:0/2379302584 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6698004450 con 0x7f66a413d640 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.167+0000 7f667ffff640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f667403deb0 0x7f6674040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.167+0000 7f667ffff640 1 -- 10.20.193.78:0/2379302584 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(616..616 src has 335..616) ==== 4767+0+0 (secure 0 0 0) 0x7f6698052e50 con 0x7f66a413d640 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.167+0000 7f66a37fe640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f667403deb0 0x7f6674040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.167+0000 7f66a37fe640 10 monclient: get_auth_request con 0x7f667403deb0 auth_method 0 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.169+0000 7f667ffff640 1 -- 10.20.193.78:0/2379302584 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f669801c370 con 0x7f66a413d640 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.169+0000 7f667ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.169+0000 7f667ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:17.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.169+0000 7f66a37fe640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f667403deb0 0x7f6674040370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6690004670 tx=0x7f6690003960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:17.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.282+0000 7f66aa287640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:03:17.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.282+0000 7f66aa287640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:17.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.282+0000 7f66aa287640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:17.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.282+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f66a40744d0 con 0x7f66a413d640 2026-02-17T06:03:17.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.282+0000 7f667ffff640 1 -- 10.20.193.78:0/2379302584 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f6698014420 con 0x7f66a413d640 2026-02-17T06:03:17.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.282+0000 7f667ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:17.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.282+0000 7f667ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:17.284 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":198,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:03:17.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f667403deb0 msgr2=0x7f6674040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:17.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f667403deb0 0x7f6674040370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6690004670 tx=0x7f6690003960 comp rx=0 tx=0).stop 2026-02-17T06:03:17.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 10 monclient: shutdown 2026-02-17T06:03:17.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a413d640 msgr2=0x7f66a40a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a413d640 0x7f66a40a3e90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6698007d60 tx=0x7f669800e4b0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 shutdown_connections 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f66a413d640 0x7f66a40a3e90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f667403deb0 0x7f6674040370 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f66a409ee70 0x7f66a40a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 --2- 10.20.193.78:0/2379302584 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f66a409ea90 0x7f66a40a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 >> 10.20.193.78:0/2379302584 conn(0x7f66a40a7520 msgr2=0x7f66a413c860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 shutdown_connections 2026-02-17T06:03:17.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:17.283+0000 7f66aa287640 1 -- 10.20.193.78:0/2379302584 wait complete. 2026-02-17T06:03:20.292 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:03:20.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient: get_monmap_and_config 2026-02-17T06:03:20.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient: build_initial_monmap 2026-02-17T06:03:20.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient: monmap: 2026-02-17T06:03:20.362 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:20.362 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:20.362 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:20.360327+0000 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:20.360327+0000 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 1 Processor -- start 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 1 -- start start 2026-02-17T06:03:20.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient: init 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient(hunting): picked mon.noname-b con 0x7f504806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient(hunting): picked mon.noname-a con 0x7f504809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5048134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f504806daf0 con 0x7f5048134df0 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f50480a0880 con 0x7f504809c4a0 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:20.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f504806d6f0 con 0x7f504806c890 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504f0b0640 10 monclient(hunting): authenticate will time out at 1645.913885s 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f503ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504ce25640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f503ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42384/0 (socket says 10.20.193.78:42384) 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 learned_addr learned my addr 10.20.193.78:0/3018331802 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504ce25640 10 monclient(hunting): get_auth_request con 0x7f504806c890 auth_method 0 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504ce25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504ce25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.360+0000 7f504ce25640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f503ffff640 10 monclient(hunting): get_auth_request con 0x7f504809c4a0 auth_method 0 2026-02-17T06:03:20.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f503ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f503ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f503ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:20.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.361+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:20.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.363+0000 7f504d626640 1 -- 10.20.193.78:0/3018331802 >> v1:10.20.193.78:6789/0 conn(0x7f5048134df0 legacy=0x7f5048135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:20.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 10 monclient(hunting): get_auth_request con 0x7f504809c4a0 auth_method 0 2026-02-17T06:03:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 10 monclient(hunting): get_auth_request con 0x7f504806c890 auth_method 0 2026-02-17T06:03:20.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:20.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:20.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:20.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:20.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:20.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:20.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:03:20.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:20.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:20.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:03:20.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.561+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:03:20.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 10 monclient(hunting): get_auth_request con 0x7f504806c890 auth_method 0 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 10 monclient(hunting): get_auth_request con 0x7f504809c4a0 auth_method 0 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:20.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:03:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:03:20.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:20.962+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:03:21.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:21.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 10 monclient(hunting): get_auth_request con 0x7f504809c4a0 auth_method 0 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.762+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 10 monclient(hunting): get_auth_request con 0x7f504806c890 auth_method 0 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:21.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:21.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:21.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:21.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:21.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:21.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:21.763+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:03:22.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.786+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2267971940 0 0) 0x7f504806daf0 con 0x7f5048134df0 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.786+0000 7f503f7fe640 10 monclient(hunting): _negotiate 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.786+0000 7f503f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.786+0000 7f503f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.786+0000 7f503f7fe640 10 monclient(hunting): my global_id is 184365 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.786+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5034003620 con 0x7f5048134df0 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3355586569 0 0) 0x7f5034003620 con 0x7f5048134df0 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f504806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f504806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 msgr2=0x7f504806d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:03:22.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 --2- 10.20.193.78:0/3018331802 >> v2:10.20.193.78:3300/0 conn(0x7f504809c4a0 0x7f504806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504809cc80 con 0x7f5048134df0 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f5044003580 con 0x7f5048134df0 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:22.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: dump: 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:22.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 >> v1:10.20.193.78:6789/0 conn(0x7f5048134df0 legacy=0x7f5048135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f5034003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient(hunting): picked mon.c con 0x7f504806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50340042a0 0x7f5034004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient(hunting): picked mon.a con 0x7f50340042a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5034004f60 0x7f503400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient(hunting): picked mon.b con 0x7f5034004f60 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f504806d6f0 con 0x7f50340042a0 2026-02-17T06:03:22.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5034005750 con 0x7f5034004f60 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f503400c050 con 0x7f504806c890 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f5034003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50340042a0 0x7f5034004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient(hunting): get_auth_request con 0x7f504806c890 auth_method 0 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:22.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 10 monclient(hunting): get_auth_request con 0x7f50340042a0 auth_method 0 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient(hunting): handle_auth_done global_id 184365 payload 293 2026-02-17T06:03:22.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5034004f60 msgr2=0x7f503400bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504d626640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5034004f60 0x7f503400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5034004f60 0x7f503400bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50340042a0 msgr2=0x7f5034004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50340042a0 0x7f5034004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient: _finish_hunting 0 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 1 monclient: found mon.c 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5034003620 con 0x7f504806c890 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient: _finish_auth 0 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 10 monclient: _check_auth_tickets 2026-02-17T06:03:22.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504ce25640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f503ffff640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50340042a0 0x7f5034004900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504f0b0640 5 monclient: authenticate success, global_id 184365 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504f0b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504d626640 10 monclient: get_auth_request con 0x7f5034004f60 auth_method 0 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.787+0000 7f504d626640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5034004f60 0x7f503400bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504ce25640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f5034003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504ce25640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f5034003c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5038012a10 tx=0x7f503802fca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bced285df85d819 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503800f040 con 0x7f504806c890 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 10 monclient: dump: 2026-02-17T06:03:22.804 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:22.805 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50380316d0 con 0x7f504806c890 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 1 -- 10.20.193.78:0/3018331802 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038031ac0 con 0x7f504806c890 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 4 set_mon_vals no callback set 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f503f7fe640 10 monclient: dump: 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:22.806 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 10 monclient: shutdown 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:22.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 1 -- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 msgr2=0x7f5034003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 1 --2- 10.20.193.78:0/3018331802 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f504806c890 0x7f5034003c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5038012a10 tx=0x7f503802fca0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504d626640 1 -- 10.20.193.78:0/3018331802 reap_dead start 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 1 -- 10.20.193.78:0/3018331802 shutdown_connections 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 1 -- 10.20.193.78:0/3018331802 >> 10.20.193.78:0/3018331802 conn(0x7f50480a6620 msgr2=0x7f50480a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 1 -- 10.20.193.78:0/3018331802 shutdown_connections 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.790+0000 7f504f0b0640 1 -- 10.20.193.78:0/3018331802 wait complete. 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient: build_initial_monmap 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient: monmap: 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:22.808 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 1 Processor -- start 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 1 -- start start 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient: init 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:03:22.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f504806c890 0x7f50481cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient(hunting): picked mon.noname-b con 0x7f504806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50481cd870 0x7f504811f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient(hunting): picked mon.noname-c con 0x7f50481cd870 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50481cdcf0 0x7f504811f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient(hunting): picked mon.noname-a con 0x7f50481cdcf0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50480a01a0 con 0x7f50481cdcf0 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50480a0020 con 0x7f504806c890 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:22.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.791+0000 7f504f0b0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f504809be20 con 0x7f50481cd870 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504f0b0640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504f0b0640 10 monclient(hunting): authenticate will time out at 1648.345002s 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50481cd870 0x7f504811f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50481cdcf0 0x7f504811f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50481cd870 0x7f504811f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37936/0 (socket says 10.20.193.78:37936) 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50481cdcf0 0x7f504811f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42422/0 (socket says 10.20.193.78:42422) 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 -- 10.20.193.78:0/446876974 learned_addr learned my addr 10.20.193.78:0/446876974 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient(hunting): get_auth_request con 0x7f50481cd870 auth_method 0 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:22.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 10 monclient(hunting): get_auth_request con 0x7f50481cdcf0 auth_method 0 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504ce25640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f504806c890 0x7f50481cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504ce25640 10 monclient(hunting): get_auth_request con 0x7f504806c890 auth_method 0 2026-02-17T06:03:22.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504ce25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504ce25640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504ce25640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient(hunting): handle_auth_done global_id 144752 payload 824 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 -- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f504806c890 msgr2=0x7f50481cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f504806c890 0x7f50481cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 -- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50481cdcf0 msgr2=0x7f504811f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50481cdcf0 0x7f504811f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 monclient: found mon.noname-c 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:03:22.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f503ffff640 1 -- 10.20.193.78:0/446876974 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5048133de0 con 0x7f50481cd870 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.792+0000 7f504d626640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50481cdcf0 0x7f504811f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503ffff640 1 -- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50481cd870 msgr2=0x7f504811f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503ffff640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50481cd870 0x7f504811f3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5030015a60 tx=0x7f5030015f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 1 -- 10.20.193.78:0/446876974 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030013c60 con 0x7f50481cd870 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: dump: 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.814 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: _finish_auth 0 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:22.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 1 -- 10.20.193.78:0/446876974 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5030013e00 con 0x7f50481cd870 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 1 -- 10.20.193.78:0/446876974 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503001f650 con 0x7f50481cd870 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f504f0b0640 5 monclient: authenticate success, global_id 144752 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503effd640 4 set_mon_vals no callback set 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f504f0b0640 10 monclient: _renew_subs 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f504f0b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5048122d70 con 0x7f50481cd870 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f503d7fa640 10 monclient: dump: 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:22.816 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f504f0b0640 10 monclient: _renew_subs 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f504f0b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.794+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f50480a1340 con 0x7f50481cd870 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f503d7fa640 1 -- 10.20.193.78:0/446876974 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5030028430 con 0x7f50481cd870 2026-02-17T06:03:22.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f504f0b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:22.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f504f0b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:22.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f504f0b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:22.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50481359e0 con 0x7f50481cd870 2026-02-17T06:03:22.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f503d7fa640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f501803deb0 0x7f5018040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f503d7fa640 1 -- 10.20.193.78:0/446876974 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(616..616 src has 335..616) ==== 4767+0+0 (secure 0 0 0) 0x7f5030053840 con 0x7f50481cd870 2026-02-17T06:03:22.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f504ce25640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f501803deb0 0x7f5018040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:22.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.795+0000 7f504ce25640 10 monclient: get_auth_request con 0x7f501803deb0 auth_method 0 2026-02-17T06:03:22.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.797+0000 7f503d7fa640 1 -- 10.20.193.78:0/446876974 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5048133de0 con 0x7f50481cd870 2026-02-17T06:03:22.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.797+0000 7f503d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:22.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.797+0000 7f503d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:22.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.798+0000 7f504ce25640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f501803deb0 0x7f5018040370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5038038000 tx=0x7f5038034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:22.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.910+0000 7f504f0b0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:03:22.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.910+0000 7f504f0b0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:22.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.910+0000 7f504f0b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:03:22.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.910+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f50480744d0 con 0x7f50481cd870 2026-02-17T06:03:22.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.910+0000 7f503d7fa640 1 -- 10.20.193.78:0/446876974 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f5030052250 con 0x7f50481cd870 2026-02-17T06:03:22.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.910+0000 7f503d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:22.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.910+0000 7f503d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:22.912 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":206,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:03:22.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f501803deb0 msgr2=0x7f5018040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f501803deb0 0x7f5018040370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5038038000 tx=0x7f5038034000 comp rx=0 tx=0).stop 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 10 monclient: shutdown 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50481cd870 msgr2=0x7f504811f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50481cd870 0x7f504811f3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5030015a60 tx=0x7f5030015f30 comp rx=0 tx=0).stop 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 shutdown_connections 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50481cdcf0 0x7f504811f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50481cd870 0x7f504811f3a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f501803deb0 0x7f5018040370 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 --2- 10.20.193.78:0/446876974 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f504806c890 0x7f50481cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 >> 10.20.193.78:0/446876974 conn(0x7f50480a6620 msgr2=0x7f50481336c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:22.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 shutdown_connections 2026-02-17T06:03:22.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.912+0000 7f504f0b0640 1 -- 10.20.193.78:0/446876974 wait complete. 2026-02-17T06:03:22.920 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:03:22.921 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:03:22.921 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:03:22.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.990+0000 7f2612a0e640 10 monclient: get_monmap_and_config 2026-02-17T06:03:22.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.990+0000 7f2612a0e640 10 monclient: build_initial_monmap 2026-02-17T06:03:22.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.990+0000 7f2612a0e640 10 monclient: monmap: 2026-02-17T06:03:22.993 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:22.993 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:22.993 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:22.990672+0000 2026-02-17T06:03:22.993 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:22.990672+0000 2026-02-17T06:03:22.993 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:22.993 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.990+0000 7f2612a0e640 1 Processor -- start 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.990+0000 7f2612a0e640 1 -- start start 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.990+0000 7f2612a0e640 10 monclient: init 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:22.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f260c06c890 0x7f260c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient(hunting): picked mon.noname-c con 0x7f260c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f260c09d170 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09b200 0x7f260c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient(hunting): picked mon.noname-b con 0x7f260c09b200 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260c09bf60 con 0x7f260c09d170 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f260c06d6f0 con 0x7f260c06c890 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f260c09c520 con 0x7f260c09b200 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:22.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2612a0e640 10 monclient(hunting): authenticate will time out at 1648.544240s 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f260b7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f260c09d170 0x7f260c1388b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:60990/0 (socket says 10.20.193.78:60990) 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09b200 0x7f260c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f260b7fe640 1 -- 10.20.193.78:0/3250638232 learned_addr learned my addr 10.20.193.78:0/3250638232 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient(hunting): get_auth_request con 0x7f260c09b200 auth_method 0 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:22.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient(hunting): handle_auth_done global_id 144755 payload 274 2026-02-17T06:03:22.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 1 -- 10.20.193.78:0/3250638232 >> v2:10.20.193.115:3300/0 conn(0x7f260c06c890 msgr2=0x7f260c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:22.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 1 --2- 10.20.193.78:0/3250638232 >> v2:10.20.193.115:3300/0 conn(0x7f260c06c890 0x7f260c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:22.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 1 -- 10.20.193.78:0/3250638232 >> v1:10.20.193.78:6789/0 conn(0x7f260c09d170 legacy=0x7f260c1388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:22.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient: _finish_hunting 0 2026-02-17T06:03:22.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 1 monclient: found mon.noname-b 2026-02-17T06:03:22.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:22.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:03:22.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.991+0000 7f2610f84640 1 -- 10.20.193.78:0/3250638232 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c09c800 con 0x7f260c09b200 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2610f84640 1 -- 10.20.193.78:0/3250638232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09b200 msgr2=0x7f260c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2610f84640 1 --2- 10.20.193.78:0/3250638232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09b200 0x7f260c06d1b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2600009d90 tx=0x7f260002f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11879d8976d139e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 1 -- 10.20.193.78:0/3250638232 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260003c070 con 0x7f260c09b200 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: dump: 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: _finish_auth 0 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: _check_auth_tickets 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 5 monclient: authenticate success, global_id 144755 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 1 -- 10.20.193.78:0/3250638232 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2600037640 con 0x7f260c09b200 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 1 -- 10.20.193.78:0/3250638232 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600037a10 con 0x7f260c09b200 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f260affd640 10 monclient: dump: 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 4 set_mon_vals no callback set 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 10 monclient: shutdown 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 1 -- 10.20.193.78:0/3250638232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09b200 msgr2=0x7f260c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 1 --2- 10.20.193.78:0/3250638232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09b200 0x7f260c06d1b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2600009d90 tx=0x7f260002f190 comp rx=0 tx=0).stop 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 1 -- 10.20.193.78:0/3250638232 shutdown_connections 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 1 --2- 10.20.193.78:0/3250638232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09b200 0x7f260c06d1b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 1 --2- 10.20.193.78:0/3250638232 >> v2:10.20.193.115:3300/0 conn(0x7f260c06c890 0x7f260c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 1 -- 10.20.193.78:0/3250638232 >> 10.20.193.78:0/3250638232 conn(0x7f260c0a4230 msgr2=0x7f260c0a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 1 -- 10.20.193.78:0/3250638232 shutdown_connections 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 1 -- 10.20.193.78:0/3250638232 wait complete. 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 10 monclient: build_initial_monmap 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.994+0000 7f2612a0e640 10 monclient: monmap: 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:23.010 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 1 Processor -- start 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 1 -- start start 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient: init 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f260c06c890 0x7f260c1cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient(hunting): picked mon.noname-b con 0x7f260c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c09b200 0x7f260c1cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient(hunting): picked mon.noname-a con 0x7f260c09b200 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09d170 0x7f260c0932a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient(hunting): picked mon.noname-c con 0x7f260c09d170 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f260c0a3530 con 0x7f260c09b200 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f260c0a33b0 con 0x7f260c06c890 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f260c138fc0 con 0x7f260c09d170 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2612a0e640 10 monclient(hunting): authenticate will time out at 1648.548332s 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c09b200 0x7f260c1cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c09b200 0x7f260c1cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42432/0 (socket says 10.20.193.78:42432) 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 -- 10.20.193.78:0/2878950758 learned_addr learned my addr 10.20.193.78:0/2878950758 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:23.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient(hunting): get_auth_request con 0x7f260c09b200 auth_method 0 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09d170 0x7f260c0932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 10 monclient(hunting): get_auth_request con 0x7f260c09d170 auth_method 0 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260bfff640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f260c06c890 0x7f260c1cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260bfff640 10 monclient(hunting): get_auth_request con 0x7f260c06c890 auth_method 0 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient(hunting): handle_auth_done global_id 184371 payload 824 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09d170 msgr2=0x7f260c0932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09d170 0x7f260c0932a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f260c06c890 msgr2=0x7f260c1cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f260c06c890 0x7f260c1cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 monclient: found mon.noname-a 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f260b7fe640 1 -- 10.20.193.78:0/2878950758 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c096b50 con 0x7f260c09b200 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.995+0000 7f2610f84640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f260c09d170 0x7f260c0932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f260b7fe640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c09b200 msgr2=0x7f260c1cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f260b7fe640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c09b200 0x7f260c1cffc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c530 tx=0x7f25fc00ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc007070 con 0x7f260c09b200 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: dump: 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.015 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: _finish_auth 0 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25fc007230 con 0x7f260c09b200 2026-02-17T06:03:23.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc007620 con 0x7f260c09b200 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f260a7fc640 4 set_mon_vals no callback set 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2608ff9640 10 monclient: dump: 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.017 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2612a0e640 5 monclient: authenticate success, global_id 184371 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2612a0e640 10 monclient: _renew_subs 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2612a0e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c096e30 con 0x7f260c09b200 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2612a0e640 10 monclient: _renew_subs 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2612a0e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.997+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c138940 con 0x7f260c09b200 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f2612a0e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f2612a0e640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c0937e0 0x7f260c093bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f2612a0e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f260c074980 con 0x7f260c0937e0 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c0937e0 0x7f260c093bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 10 monclient: get_auth_request con 0x7f260c0937e0 auth_method 0 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f25fc01b400 con 0x7f260c09b200 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 10 monclient(hunting): handle_auth_done global_id 184374 payload 824 2026-02-17T06:03:23.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f260bfff640 1 -- 10.20.193.78:0/2878950758 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f260c139370 con 0x7f260c0937e0 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f2608ff9640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25dc03deb0 0x7f25dc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(616..616 src has 335..616) ==== 4767+0+0 (secure 0 0 0) 0x7f25fc0525b0 con 0x7f260c09b200 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.998+0000 7f2610f84640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25dc03deb0 0x7f25dc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:22.999+0000 7f2610f84640 10 monclient: get_auth_request con 0x7f25dc03deb0 auth_method 0 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.000+0000 7f260bfff640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c0937e0 msgr2=0x7f260c093bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.000+0000 7f260bfff640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c0937e0 0x7f260c093bc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25f800d730 tx=0x7f25f800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.000+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80042e0 con 0x7f260c0937e0 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.000+0000 7f2608ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.000+0000 7f2608ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.000+0000 7f2608ff9640 10 monclient: dump: 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.020 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.001+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f25f8017070 con 0x7f260c0937e0 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.001+0000 7f2608ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.001+0000 7f2608ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.001+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c0937e0 msgr2=0x7f260c093bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.001+0000 7f2608ff9640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c0937e0 0x7f260c093bc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25f800d730 tx=0x7f25f800dc00 comp rx=0 tx=0).stop 2026-02-17T06:03:23.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.001+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c0937e0 msgr2=0x7f260c093bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.001+0000 7f2608ff9640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c0937e0 0x7f260c093bc0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25f800d730 tx=0x7f25f800dc00 comp rx=0 tx=0).stop 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.001+0000 7f2610f84640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25dc03deb0 0x7f25dc040370 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f260c068c30 tx=0x7f2600033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f2612a0e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f2612a0e640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c094100 0x7f260c1daa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f2612a0e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f260c0a3870 con 0x7f260c094100 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c094100 0x7f260c1daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 10 monclient: get_auth_request con 0x7f260c094100 auth_method 0 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 10 monclient(hunting): handle_auth_done global_id 184377 payload 824 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.004+0000 7f260bfff640 1 -- 10.20.193.78:0/2878950758 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f260c0744d0 con 0x7f260c094100 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f260bfff640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c094100 msgr2=0x7f260c1daa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f260bfff640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c094100 0x7f260c1daa80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25f8016000 tx=0x7f25f80048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f801cde0 con 0x7f260c094100 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 10 monclient: dump: 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.023 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7f25f800b730 con 0x7f260c094100 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c094100 msgr2=0x7f260c1daa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c094100 0x7f260c1daa80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25f8016000 tx=0x7f25f80048d0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c094100 msgr2=0x7f260c1daa80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:23.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.006+0000 7f2608ff9640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c094100 0x7f260c1daa80 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25f8016000 tx=0x7f25f80048d0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 206, 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 167786, 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:03:23.025 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.026 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.027 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:03:23.028 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:03:23.029 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.030 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:03:23.031 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.032 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:23.033 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.034 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.035 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:03:23.035 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25dc03deb0 msgr2=0x7f25dc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f25dc03deb0 0x7f25dc040370 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f260c068c30 tx=0x7f2600033000 comp rx=0 tx=0).stop 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 10 monclient: shutdown 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c09b200 msgr2=0x7f260c1cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 1 --2- 10.20.193.78:0/2878950758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f260c09b200 0x7f260c1cffc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c530 tx=0x7f25fc00ca00 comp rx=0 tx=0).stop 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2610f84640 1 -- 10.20.193.78:0/2878950758 reap_dead start 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2610f84640 1 -- 10.20.193.78:0/2878950758 reap_dead start 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 shutdown_connections 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 >> 10.20.193.78:0/2878950758 conn(0x7f260c0a4230 msgr2=0x7f260c0a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 shutdown_connections 2026-02-17T06:03:23.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.007+0000 7f2612a0e640 1 -- 10.20.193.78:0/2878950758 wait complete. 2026-02-17T06:03:23.036 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:03:23.037 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:03:23.037 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.105+0000 7f6b75a0c640 10 monclient: get_monmap_and_config 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.105+0000 7f6b75a0c640 10 monclient: build_initial_monmap 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.105+0000 7f6b75a0c640 10 monclient: monmap: 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:23.105742+0000 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:23.105742+0000 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.105 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:23.106 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:23.106 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:23.106 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:23.106 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:23.106 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.105+0000 7f6b75a0c640 1 Processor -- start 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.105+0000 7f6b75a0c640 1 -- start start 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.105+0000 7f6b75a0c640 10 monclient: init 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6b7006c890 0x7f6b7006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 10 monclient(hunting): picked mon.noname-c con 0x7f6b7006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6b7009d430 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6b7009bf40 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b70135c10 con 0x7f6b7009d430 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b70073600 con 0x7f6b7009bf40 2026-02-17T06:03:23.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6b7009ca50 con 0x7f6b7006c890 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b75a0c640 10 monclient(hunting): authenticate will time out at 1648.659276s 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6e7fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f6b7009d430 0x7f6b70134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:60998/0 (socket says 10.20.193.78:60998) 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6e7fc640 1 -- 10.20.193.78:0/2141824574 learned_addr learned my addr 10.20.193.78:0/2141824574 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 1 --2- 10.20.193.78:0/2141824574 >> v2:10.20.193.115:3300/0 conn(0x7f6b7006c890 0x7f6b7006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient(hunting): get_auth_request con 0x7f6b7006c890 auth_method 0 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient(hunting): handle_auth_done global_id 214105 payload 274 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 1 -- 10.20.193.78:0/2141824574 >> v1:10.20.193.115:6789/0 conn(0x7f6b7009bf40 legacy=0x7f6b70135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 1 -- 10.20.193.78:0/2141824574 >> v1:10.20.193.78:6789/0 conn(0x7f6b7009d430 legacy=0x7f6b70134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient: _finish_hunting 0 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 1 monclient: found mon.noname-c 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:03:23.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.106+0000 7f6b6effd640 1 -- 10.20.193.78:0/2141824574 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7006dab0 con 0x7f6b7006c890 2026-02-17T06:03:23.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6effd640 1 -- 10.20.193.78:0/2141824574 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7006c890 msgr2=0x7f6b7006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6effd640 1 --2- 10.20.193.78:0/2141824574 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7006c890 0x7f6b7006cc70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b640098a0 tx=0x7f6b6402ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec43654ac8542b84 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 1 -- 10.20.193.78:0/2141824574 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6403c070 con 0x7f6b7006c890 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: dump: 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: _finish_auth 0 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b75a0c640 5 monclient: authenticate success, global_id 214105 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b75a0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 1 -- 10.20.193.78:0/2141824574 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6402fce0 con 0x7f6b7006c890 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:23.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 1 -- 10.20.193.78:0/2141824574 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64037660 con 0x7f6b7006c890 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b75a0c640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b75a0c640 4 set_mon_vals no callback set 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.109+0000 7f6b6dffb640 10 monclient: dump: 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient: shutdown 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- 10.20.193.78:0/2141824574 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7006c890 msgr2=0x7f6b7006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 --2- 10.20.193.78:0/2141824574 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7006c890 0x7f6b7006cc70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b640098a0 tx=0x7f6b6402ed80 comp rx=0 tx=0).stop 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- 10.20.193.78:0/2141824574 shutdown_connections 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 --2- 10.20.193.78:0/2141824574 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7006c890 0x7f6b7006cc70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- 10.20.193.78:0/2141824574 >> 10.20.193.78:0/2141824574 conn(0x7f6b700a6620 msgr2=0x7f6b700a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- 10.20.193.78:0/2141824574 shutdown_connections 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- 10.20.193.78:0/2141824574 wait complete. 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient: build_initial_monmap 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient: monmap: 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 Processor -- start 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- start start 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient: init 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b7006c890 0x7f6b701cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient(hunting): picked mon.noname-c con 0x7f6b7006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:23.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7009bf40 0x7f6b701cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient(hunting): picked mon.noname-b con 0x7f6b7009bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b7009d430 0x7f6b70122450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient(hunting): picked mon.noname-a con 0x7f6b7009d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7009c800 con 0x7f6b7009d430 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70073410 con 0x7f6b7009bf40 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7006d1b0 con 0x7f6b7006c890 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.110+0000 7f6b75a0c640 10 monclient(hunting): authenticate will time out at 1648.663928s 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b7006c890 0x7f6b701cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b7009d430 0x7f6b70122450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b7006c890 0x7f6b701cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37966/0 (socket says 10.20.193.78:37966) 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 1 -- 10.20.193.78:0/23302868 learned_addr learned my addr 10.20.193.78:0/23302868 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 10 monclient(hunting): get_auth_request con 0x7f6b7006c890 auth_method 0 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b7009d430 0x7f6b70122450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42456/0 (socket says 10.20.193.78:42456) 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient(hunting): get_auth_request con 0x7f6b7009d430 auth_method 0 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6e7fc640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7009bf40 0x7f6b701cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6e7fc640 10 monclient(hunting): get_auth_request con 0x7f6b7009bf40 auth_method 0 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient(hunting): handle_auth_done global_id 184380 payload 824 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b7006c890 msgr2=0x7f6b701cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b7006c890 0x7f6b701cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7009bf40 msgr2=0x7f6b701cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6b7009bf40 0x7f6b701cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 1 monclient: found mon.noname-a 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6f7fe640 1 -- 10.20.193.78:0/23302868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b70137430 con 0x7f6b7009d430 2026-02-17T06:03:23.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.111+0000 7f6b6effd640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b7006c890 0x7f6b701cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b6f7fe640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b7009d430 msgr2=0x7f6b70122450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b6f7fe640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b7009d430 0x7f6b70122450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000caf0 tx=0x7f6b60004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60016030 con 0x7f6b7009d430 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: dump: 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.117 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: _finish_auth 0 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: _check_auth_tickets 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6000ac50 con 0x7f6b7009d430 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60004a80 con 0x7f6b7009d430 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b6d7fa640 4 set_mon_vals no callback set 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b47fff640 10 monclient: dump: 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b75a0c640 5 monclient: authenticate success, global_id 184380 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b75a0c640 10 monclient: _renew_subs 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b75a0c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b70126100 con 0x7f6b7009d430 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b75a0c640 10 monclient: _renew_subs 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b75a0c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.113+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b700a2750 con 0x7f6b7009d430 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b75a0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b75a0c640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b70122990 0x7f6b70122d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b75a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6b70124cc0 con 0x7f6b70122990 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b70122990 0x7f6b70122d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6b60023060 con 0x7f6b7009d430 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 10 monclient: get_auth_request con 0x7f6b70122990 auth_method 0 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 10 monclient(hunting): handle_auth_done global_id 144764 payload 824 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6effd640 1 -- 10.20.193.78:0/23302868 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b7009d810 con 0x7f6b70122990 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b47fff640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b4003deb0 0x7f6b40040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(616..616 src has 335..616) ==== 4767+0+0 (secure 0 0 0) 0x7f6b60052950 con 0x7f6b7009d430 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6e7fc640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b4003deb0 0x7f6b40040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.114+0000 7f6b6e7fc640 10 monclient: get_auth_request con 0x7f6b4003deb0 auth_method 0 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b6effd640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b70122990 msgr2=0x7f6b70122d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b6effd640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b70122990 0x7f6b70122d70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b64002410 tx=0x7f6b640047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6403c040 con 0x7f6b70122990 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 10 monclient: dump: 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6b64037e50 con 0x7f6b70122990 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b70122990 msgr2=0x7f6b70122d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b70122990 0x7f6b70122d70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b64002410 tx=0x7f6b640047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b70122990 msgr2=0x7f6b70122d70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.116+0000 7f6b47fff640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b70122990 0x7f6b70122d70 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b64002410 tx=0x7f6b640047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.117+0000 7f6b6e7fc640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b4003deb0 0x7f6b40040370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6b70126310 tx=0x7f6b58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b75a0c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b75a0c640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b701232b0 0x7f6b701da770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b75a0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6b70124150 con 0x7f6b701232b0 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b701232b0 0x7f6b701da770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 10 monclient: get_auth_request con 0x7f6b701232b0 auth_method 0 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 10 monclient(hunting): handle_auth_done global_id 144767 payload 824 2026-02-17T06:03:23.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.120+0000 7f6b6effd640 1 -- 10.20.193.78:0/23302868 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6b7006d5b0 con 0x7f6b701232b0 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b6effd640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b701232b0 msgr2=0x7f6b701da770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b6effd640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b701232b0 0x7f6b701da770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6b7006c6a0 tx=0x7f6b64008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6403c040 con 0x7f6b701232b0 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 10 monclient: dump: 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.122 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f6b64051940 con 0x7f6b701232b0 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b701232b0 msgr2=0x7f6b701da770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b701232b0 0x7f6b701da770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6b7006c6a0 tx=0x7f6b64008030 comp rx=0 tx=0).stop 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b701232b0 msgr2=0x7f6b701da770 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.122+0000 7f6b47fff640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6b701232b0 0x7f6b701da770 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6b7006c6a0 tx=0x7f6b64008030 comp rx=0 tx=0).stop 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 206, 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 386656, 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.123 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:03:23.124 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.125 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.126 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b4003deb0 msgr2=0x7f6b40040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b4003deb0 0x7f6b40040370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6b70126310 tx=0x7f6b58009040 comp rx=0 tx=0).stop 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 10 monclient: shutdown 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b7009d430 msgr2=0x7f6b70122450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 1 --2- 10.20.193.78:0/23302868 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6b7009d430 0x7f6b70122450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000caf0 tx=0x7f6b60004a10 comp rx=0 tx=0).stop 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b6f7fe640 1 -- 10.20.193.78:0/23302868 reap_dead start 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b6f7fe640 1 -- 10.20.193.78:0/23302868 reap_dead start 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 shutdown_connections 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 >> 10.20.193.78:0/23302868 conn(0x7f6b700a6620 msgr2=0x7f6b7009da20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 shutdown_connections 2026-02-17T06:03:23.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.123+0000 7f6b75a0c640 1 -- 10.20.193.78:0/23302868 wait complete. 2026-02-17T06:03:23.128 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:03:23.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:03:23.128 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:03:23.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient: get_monmap_and_config 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient: build_initial_monmap 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient: monmap: 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:23.196281+0000 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:23.196281+0000 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:23.196 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 1 Processor -- start 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 1 -- start start 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient: init 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f78a00730c0 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a00734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient(hunting): picked mon.noname-b con 0x7f78a00a2530 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f78a00a0a40 0x7f78a01403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient(hunting): picked mon.noname-a con 0x7f78a00a0a40 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78a00a2c20 con 0x7f78a00730c0 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f78a0138fd0 con 0x7f78a00a0a40 2026-02-17T06:03:23.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f78a0073b50 con 0x7f78a00a2530 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.196+0000 7f78a7643640 10 monclient(hunting): authenticate will time out at 1648.749918s 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a00734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a53b8640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f78a00730c0 0x7f78a00a0330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:32772/0 (socket says 10.20.193.78:32772) 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a53b8640 1 -- 10.20.193.78:0/2445906255 learned_addr learned my addr 10.20.193.78:0/2445906255 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient(hunting): get_auth_request con 0x7f78a00a2530 auth_method 0 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a5bb9640 1 --2- 10.20.193.78:0/2445906255 >> v2:10.20.193.78:3300/0 conn(0x7f78a00a0a40 0x7f78a01403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a5bb9640 10 monclient(hunting): get_auth_request con 0x7f78a00a0a40 auth_method 0 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a5bb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a5bb9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a5bb9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a5bb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a5bb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a5bb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient(hunting): handle_auth_done global_id 144770 payload 274 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 1 -- 10.20.193.78:0/2445906255 >> v2:10.20.193.78:3300/0 conn(0x7f78a00a0a40 msgr2=0x7f78a01403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 1 --2- 10.20.193.78:0/2445906255 >> v2:10.20.193.78:3300/0 conn(0x7f78a00a0a40 0x7f78a01403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 1 -- 10.20.193.78:0/2445906255 >> v1:10.20.193.78:6789/0 conn(0x7f78a00730c0 legacy=0x7f78a00a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient: _finish_hunting 0 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 1 monclient: found mon.noname-b 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:03:23.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.197+0000 7f78a4bb7640 1 -- 10.20.193.78:0/2445906255 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a00a35d0 con 0x7f78a00a2530 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f78a4bb7640 1 -- 10.20.193.78:0/2445906255 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 msgr2=0x7f78a00734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f78a4bb7640 1 --2- 10.20.193.78:0/2445906255 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a00734a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78940098a0 tx=0x7f789402ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae75284c6195db3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 1 -- 10.20.193.78:0/2445906255 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789403c070 con 0x7f78a00a2530 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: dump: 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.199 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: _finish_auth 0 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 1 -- 10.20.193.78:0/2445906255 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789402fc00 con 0x7f78a00a2530 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f78a7643640 5 monclient: authenticate success, global_id 144770 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 1 -- 10.20.193.78:0/2445906255 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894037440 con 0x7f78a00a2530 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f78a7643640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f78a7643640 4 set_mon_vals no callback set 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.199+0000 7f788ffff640 10 monclient: dump: 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.201 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 10 monclient: shutdown 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 -- 10.20.193.78:0/2445906255 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 msgr2=0x7f78a00734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 --2- 10.20.193.78:0/2445906255 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a00734a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78940098a0 tx=0x7f789402ed80 comp rx=0 tx=0).stop 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 -- 10.20.193.78:0/2445906255 shutdown_connections 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 --2- 10.20.193.78:0/2445906255 >> v2:10.20.193.78:3300/0 conn(0x7f78a00a0a40 0x7f78a01403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 --2- 10.20.193.78:0/2445906255 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a00734a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 -- 10.20.193.78:0/2445906255 >> 10.20.193.78:0/2445906255 conn(0x7f78a00a7370 msgr2=0x7f78a00a7780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 -- 10.20.193.78:0/2445906255 shutdown_connections 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 -- 10.20.193.78:0/2445906255 wait complete. 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 10 monclient: build_initial_monmap 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 10 monclient: monmap: 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 Processor -- start 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 1 -- start start 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.200+0000 7f78a7643640 10 monclient: init 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:23.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a00730c0 0x7f78a01cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient(hunting): picked mon.noname-b con 0x7f78a00730c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f78a00a0a40 0x7f78a0131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient(hunting): picked mon.noname-a con 0x7f78a00a0a40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a01320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient(hunting): picked mon.noname-c con 0x7f78a00a2530 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0068950 con 0x7f78a00a0a40 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f78a00a2260 con 0x7f78a00730c0 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f78a0068ad0 con 0x7f78a00a2530 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a7643640 10 monclient(hunting): authenticate will time out at 1648.754103s 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f78a00a0a40 0x7f78a0131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f78a00a0a40 0x7f78a0131b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42470/0 (socket says 10.20.193.78:42470) 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a01320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 -- 10.20.193.78:0/1243145744 learned_addr learned my addr 10.20.193.78:0/1243145744 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient(hunting): get_auth_request con 0x7f78a00a0a40 auth_method 0 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 10 monclient(hunting): get_auth_request con 0x7f78a00a2530 auth_method 0 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a53b8640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a00730c0 0x7f78a01cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient(hunting): handle_auth_done global_id 184386 payload 824 2026-02-17T06:03:23.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 msgr2=0x7f78a01320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a01320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a00730c0 msgr2=0x7f78a01cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a00730c0 0x7f78a01cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient: _finish_hunting 0 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 monclient: found mon.noname-a 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a4bb7640 1 -- 10.20.193.78:0/1243145744 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a0121270 con 0x7f78a00a0a40 2026-02-17T06:03:23.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.201+0000 7f78a5bb9640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f78a00a2530 0x7f78a01320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a4bb7640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f78a00a0a40 msgr2=0x7f78a0131b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a4bb7640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f78a00a0a40 0x7f78a0131b60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7894002410 tx=0x7f78940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789403c040 con 0x7f78a00a0a40 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: dump: 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: _finish_auth 0 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7894047070 con 0x7f78a00a0a40 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a7643640 5 monclient: authenticate success, global_id 184386 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a7643640 10 monclient: _renew_subs 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a7643640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a0137930 con 0x7f78a00a0a40 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894040a60 con 0x7f78a00a0a40 2026-02-17T06:03:23.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788f7fe640 4 set_mon_vals no callback set 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f788dffb640 10 monclient: dump: 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a7643640 10 monclient: _renew_subs 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a7643640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.203+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a00740c0 con 0x7f78a00a0a40 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a7643640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a7643640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a01231e0 0x7f78a011f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a7643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0069d50 con 0x7f78a01231e0 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f789403f030 con 0x7f78a00a0a40 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a01231e0 0x7f78a011f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 10 monclient: get_auth_request con 0x7f78a01231e0 auth_method 0 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f788dffb640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f787003deb0 0x7f7870040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(616..616 src has 335..616) ==== 4767+0+0 (secure 0 0 0) 0x7f7894077ae0 con 0x7f78a00a0a40 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a5bb9640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f787003deb0 0x7f7870040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a5bb9640 10 monclient: get_auth_request con 0x7f787003deb0 auth_method 0 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 10 monclient(hunting): handle_auth_done global_id 214111 payload 824 2026-02-17T06:03:23.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.204+0000 7f78a53b8640 1 -- 10.20.193.78:0/1243145744 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78a0136090 con 0x7f78a01231e0 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f78a53b8640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a01231e0 msgr2=0x7f78a011f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f78a53b8640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a01231e0 0x7f78a011f5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7890005980 tx=0x7f789000d960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890007070 con 0x7f78a01231e0 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 10 monclient: dump: 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f78a5bb9640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f787003deb0 0x7f7870040370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f78a0121ab0 tx=0x7f789c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7890017070 con 0x7f78a01231e0 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a01231e0 msgr2=0x7f78a011f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a01231e0 0x7f78a011f5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7890005980 tx=0x7f789000d960 comp rx=0 tx=0).stop 2026-02-17T06:03:23.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a01231e0 msgr2=0x7f78a011f5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:23.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.207+0000 7f788dffb640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a01231e0 0x7f78a011f5d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a7643640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:03:23.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a7643640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a011fb10 0x7f78a01da570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a7643640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f78a01325e0 con 0x7f78a011fb10 2026-02-17T06:03:23.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a53b8640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a011fb10 0x7f78a01da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a53b8640 10 monclient: get_auth_request con 0x7f78a011fb10 auth_method 0 2026-02-17T06:03:23.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a53b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a53b8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.210+0000 7f78a53b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.211+0000 7f78a53b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.211+0000 7f78a53b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.211+0000 7f78a53b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.211+0000 7f78a53b8640 10 monclient(hunting): handle_auth_done global_id 214114 payload 824 2026-02-17T06:03:23.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.211+0000 7f78a53b8640 1 -- 10.20.193.78:0/1243145744 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f78a00a0f60 con 0x7f78a011fb10 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a53b8640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a011fb10 msgr2=0x7f78a01da570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a53b8640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a011fb10 0x7f78a01da570 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7890005980 tx=0x7f7890004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890012820 con 0x7f78a011fb10 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 10 monclient: dump: 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f7890012820 con 0x7f78a011fb10 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:23.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a011fb10 msgr2=0x7f78a01da570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a011fb10 0x7f78a01da570 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7890005980 tx=0x7f7890004a40 comp rx=0 tx=0).stop 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a011fb10 msgr2=0x7f78a01da570 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f788dffb640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f78a011fb10 0x7f78a01da570 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7890005980 tx=0x7f7890004a40 comp rx=0 tx=0).stop 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 206, 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 6059, 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.214 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:03:23.215 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.216 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:03:23.217 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:03:23.218 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f787003deb0 msgr2=0x7f7870040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f787003deb0 0x7f7870040370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f78a0121ab0 tx=0x7f789c00b040 comp rx=0 tx=0).stop 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 10 monclient: shutdown 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f78a00a0a40 msgr2=0x7f78a0131b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f78a00a0a40 0x7f78a0131b60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7894002410 tx=0x7f78940047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a5bb9640 1 -- 10.20.193.78:0/1243145744 reap_dead start 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 shutdown_connections 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 1 --2- 10.20.193.78:0/1243145744 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f78a00a0a40 0x7f78a0131b60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 >> 10.20.193.78:0/1243145744 conn(0x7f78a00a7370 msgr2=0x7f78a0072310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.213+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 shutdown_connections 2026-02-17T06:03:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.214+0000 7f78a7643640 1 -- 10.20.193.78:0/1243145744 wait complete. 2026-02-17T06:03:23.219 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:03:23.219 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:03:23.220 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:03:23.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.286+0000 7fd73eb6a640 10 monclient: get_monmap_and_config 2026-02-17T06:03:23.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.286+0000 7fd73eb6a640 10 monclient: build_initial_monmap 2026-02-17T06:03:23.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.286+0000 7fd73eb6a640 10 monclient: monmap: 2026-02-17T06:03:23.286 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:23.286 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:23.286540+0000 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:23.286540+0000 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.286+0000 7fd73eb6a640 1 Processor -- start 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.286+0000 7fd73eb6a640 1 -- start start 2026-02-17T06:03:23.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.286+0000 7fd73eb6a640 10 monclient: init 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd73806c890 0x7fd73806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient(hunting): picked mon.noname-a con 0x7fd73806c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd73809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd73806d1b0 0x7fd738135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient(hunting): picked mon.noname-c con 0x7fd73806d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73806daf0 con 0x7fd73809c4a0 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd7380a0880 con 0x7fd73806c890 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd73809cc80 con 0x7fd73806d1b0 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73eb6a640 10 monclient(hunting): authenticate will time out at 1648.840073s 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd73806c890 0x7fd73806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd73806c890 0x7fd73806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42480/0 (socket says 10.20.193.78:42480) 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 1 -- 10.20.193.78:0/1337753673 learned_addr learned my addr 10.20.193.78:0/1337753673 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:23.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient(hunting): get_auth_request con 0x7fd73806c890 auth_method 0 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73d0e0640 1 --2- 10.20.193.78:0/1337753673 >> v2:10.20.193.115:3300/0 conn(0x7fd73806d1b0 0x7fd738135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73d0e0640 10 monclient(hunting): get_auth_request con 0x7fd73806d1b0 auth_method 0 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73d0e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73d0e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73d0e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient(hunting): handle_auth_done global_id 184389 payload 274 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 1 -- 10.20.193.78:0/1337753673 >> v2:10.20.193.115:3300/0 conn(0x7fd73806d1b0 msgr2=0x7fd738135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 1 --2- 10.20.193.78:0/1337753673 >> v2:10.20.193.115:3300/0 conn(0x7fd73806d1b0 0x7fd738135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 1 -- 10.20.193.78:0/1337753673 >> v1:10.20.193.115:6789/0 conn(0x7fd73809c4a0 legacy=0x7fd738134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient: _finish_hunting 0 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 1 monclient: found mon.noname-a 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73c8df640 1 -- 10.20.193.78:0/1337753673 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73809cf60 con 0x7fd73806c890 2026-02-17T06:03:23.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.287+0000 7fd73d0e0640 1 --2- 10.20.193.78:0/1337753673 >> v2:10.20.193.115:3300/0 conn(0x7fd73806d1b0 0x7fd738135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.289+0000 7fd73c8df640 1 -- 10.20.193.78:0/1337753673 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 msgr2=0x7fd73806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.289+0000 7fd73c8df640 1 --2- 10.20.193.78:0/1337753673 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd73806cc70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd738074de0 tx=0x7fd72802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ff8e9a77d9f801 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 1 -- 10.20.193.78:0/1337753673 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72803c070 con 0x7fd73806c890 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: dump: 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.294 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 1 -- 10.20.193.78:0/1337753673 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72802fca0 con 0x7fd73806c890 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 5 monclient: authenticate success, global_id 184389 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 1 -- 10.20.193.78:0/1337753673 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728037660 con 0x7fd73806c890 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 4 set_mon_vals no callback set 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd72f7fe640 10 monclient: dump: 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.295 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 10 monclient: shutdown 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 -- 10.20.193.78:0/1337753673 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 msgr2=0x7fd73806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/1337753673 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd73806cc70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd738074de0 tx=0x7fd72802ee60 comp rx=0 tx=0).stop 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 -- 10.20.193.78:0/1337753673 shutdown_connections 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/1337753673 >> v2:10.20.193.115:3300/0 conn(0x7fd73806d1b0 0x7fd738135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/1337753673 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd73806cc70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 -- 10.20.193.78:0/1337753673 >> 10.20.193.78:0/1337753673 conn(0x7fd7380a6620 msgr2=0x7fd7380a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 -- 10.20.193.78:0/1337753673 shutdown_connections 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 -- 10.20.193.78:0/1337753673 wait complete. 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 10 monclient: build_initial_monmap 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 10 monclient: monmap: 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.296 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.290+0000 7fd73eb6a640 1 Processor -- start 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 1 -- start start 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient: init 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd7381cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient(hunting): picked mon.noname-a con 0x7fd73806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd73806d1b0 0x7fd7381cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient(hunting): picked mon.noname-c con 0x7fd73806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd73809c4a0 0x7fd738125be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient(hunting): picked mon.noname-b con 0x7fd73809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7380a01a0 con 0x7fd73806c890 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd7380a0020 con 0x7fd73809c4a0 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd73809be20 con 0x7fd73806d1b0 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73eb6a640 10 monclient(hunting): authenticate will time out at 1648.844282s 2026-02-17T06:03:23.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd7381cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd7381cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42494/0 (socket says 10.20.193.78:42494) 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 -- 10.20.193.78:0/3898792797 learned_addr learned my addr 10.20.193.78:0/3898792797 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient(hunting): get_auth_request con 0x7fd73806c890 auth_method 0 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd72ffff640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd73806d1b0 0x7fd7381cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd72ffff640 10 monclient(hunting): get_auth_request con 0x7fd73806d1b0 auth_method 0 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd72ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd72ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd72ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd72ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd72ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd72ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73d0e0640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd73809c4a0 0x7fd738125be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73d0e0640 10 monclient(hunting): get_auth_request con 0x7fd73809c4a0 auth_method 0 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73d0e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:23.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73d0e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73d0e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient(hunting): handle_auth_done global_id 184392 payload 824 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 -- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd73806d1b0 msgr2=0x7fd7381cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd73806d1b0 0x7fd7381cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 -- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd73809c4a0 msgr2=0x7fd738125be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd73809c4a0 0x7fd738125be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient: _finish_hunting 0 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 monclient: found mon.noname-a 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.291+0000 7fd73c8df640 1 -- 10.20.193.78:0/3898792797 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd738072890 con 0x7fd73806c890 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73c8df640 1 -- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 msgr2=0x7fd7381cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73c8df640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd7381cf6a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd728002410 tx=0x7fd728004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 1 -- 10.20.193.78:0/3898792797 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72803c040 con 0x7fd73806c890 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: dump: 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.299 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: _finish_auth 0 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73eb6a640 5 monclient: authenticate success, global_id 184392 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 1 -- 10.20.193.78:0/3898792797 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd728047070 con 0x7fd73806c890 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73eb6a640 10 monclient: _renew_subs 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73eb6a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7381283a0 con 0x7fd73806c890 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 1 -- 10.20.193.78:0/3898792797 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728042890 con 0x7fd73806c890 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72d7fa640 10 monclient: dump: 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.300 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd72effd640 4 set_mon_vals no callback set 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73eb6a640 10 monclient: _renew_subs 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73eb6a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.293+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd738129570 con 0x7fd73806c890 2026-02-17T06:03:23.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.294+0000 7fd72d7fa640 1 -- 10.20.193.78:0/3898792797 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd728042a30 con 0x7fd73806c890 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.294+0000 7fd73eb6a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.294+0000 7fd73eb6a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.294+0000 7fd73eb6a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.294+0000 7fd72d7fa640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd70803deb0 0x7fd708040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.294+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7381293a0 con 0x7fd73806c890 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.294+0000 7fd72d7fa640 1 -- 10.20.193.78:0/3898792797 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(616..616 src has 335..616) ==== 4767+0+0 (secure 0 0 0) 0x7fd728077b80 con 0x7fd73806c890 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.295+0000 7fd72ffff640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd70803deb0 0x7fd708040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.295+0000 7fd72ffff640 10 monclient: get_auth_request con 0x7fd70803deb0 auth_method 0 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.297+0000 7fd72d7fa640 1 -- 10.20.193.78:0/3898792797 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd738072890 con 0x7fd73806c890 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.297+0000 7fd72d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.297+0000 7fd72d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:23.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.297+0000 7fd72ffff640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd70803deb0 0x7fd708040370 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd738136c50 tx=0x7fd7200091c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.411+0000 7fd73eb6a640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:03:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.411+0000 7fd73eb6a640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:03:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.411+0000 7fd73eb6a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.411+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd738074980 con 0x7fd73806c890 2026-02-17T06:03:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.412+0000 7fd72d7fa640 1 -- 10.20.193.78:0/3898792797 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd72802fc60 con 0x7fd73806c890 2026-02-17T06:03:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.412+0000 7fd72d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:03:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.412+0000 7fd72d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:23.415 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.412+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.415 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.412+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=30,config=2,health=12,kv=10,logm=46} crc {auth=1219767419,config=692619927,health=1968765669,kv=2589905713,logm=3237219572}) 2026-02-17T06:03:23.415 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.412+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=28,config=2,health=12,kv=11,logm=47} crc {auth=1346580200,config=692619927,health=416335068,kv=2589905713,logm=1367243228}) 2026-02-17T06:03:23.415 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.412+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.415 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.412+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=30,config=2,health=12,kv=10,logm=46} crc {auth=1219767419,config=692619927,health=1968765669,kv=2589905713,logm=3237219572}) 2026-02-17T06:03:23.415 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.412+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=31,config=2,health=12,kv=11,logm=44} crc {auth=98571820,config=692619927,health=1968765668,kv=2049492640,logm=455802408}) 2026-02-17T06:03:23.415 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.416 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1595545666}) 2026-02-17T06:03:23.416 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1063469419}) 2026-02-17T06:03:23.416 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.416 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1595545666}) 2026-02-17T06:03:23.416 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1631328927}) 2026-02-17T06:03:23.416 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.416 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1619361496}) 2026-02-17T06:03:23.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd70803deb0 msgr2=0x7fd708040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd70803deb0 0x7fd708040370 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd738136c50 tx=0x7fd7200091c0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 10 monclient: shutdown 2026-02-17T06:03:23.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:23.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 msgr2=0x7fd7381cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:23.418 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1943520997}) 2026-02-17T06:03:23.418 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd7381cf6a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd728002410 tx=0x7fd728004830 comp rx=0 tx=0).stop 2026-02-17T06:03:23.419 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1619361496}) 2026-02-17T06:03:23.419 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.413+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=467881773}) 2026-02-17T06:03:23.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 shutdown_connections 2026-02-17T06:03:23.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd70803deb0 0x7fd708040370 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd73809c4a0 0x7fd738125be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd73806d1b0 0x7fd7381cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 --2- 10.20.193.78:0/3898792797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd73806c890 0x7fd7381cf6a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:23.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 >> 10.20.193.78:0/3898792797 conn(0x7fd7380a6620 msgr2=0x7fd738072dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:23.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 shutdown_connections 2026-02-17T06:03:23.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:23.413+0000 7fd73eb6a640 1 -- 10.20.193.78:0/3898792797 wait complete. 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2707963905}) 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4263983918}) 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2707963905}) 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=199464317}) 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2005740709}) 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2490450951}) 2026-02-17T06:03:23.429 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2005740709}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773149061}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=810398741}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3959274874}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=810398741}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.414+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2869293040}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3427317558}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2668824069}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3427317558}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3529021181}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738924818}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=385119744}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738924818}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.415+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2723652576}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2567915078}) 2026-02-17T06:03:23.430 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2284781136}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2567915078}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998807550}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4216872026}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3158297947}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4216872026}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1914729001}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648826401}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=500019357}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1648826401}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.416+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1057593354}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3309726685}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2934857476}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3309726685}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2530964846}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362489425}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=233973725}) 2026-02-17T06:03:23.431 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362489425}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2181984073}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2081146546}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=787968122}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2081146546}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.417+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=86407703}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2140865337}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=319469422}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2140865337}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1210048673}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823330366}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1632972360}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823330366}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4038857484}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1599225508}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3934509649}) 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.432 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1599225508}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.418+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=720278371}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2262525563}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=536455670}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2262525563}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1378327682}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=789083671}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2000011764}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=789083671}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=331505033}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244228216}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3165645727}) 2026-02-17T06:03:23.433 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.419+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244228216}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1945009998}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3771214371}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3825605874}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3771214371}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=743254486}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2432080401}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3898119435}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2432080401}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.420+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1402462836}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2387240899}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2290548477}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2387240899}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=839014282}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=894105071}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3667266356}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=894105071}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522740194}) 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.434 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3368905387}) 2026-02-17T06:03:23.435 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1850667570}) 2026-02-17T06:03:23.435 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.435 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3368905387}) 2026-02-17T06:03:23.435 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3071214325}) 2026-02-17T06:03:23.435 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:03:23.436 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2840410382}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=753428544}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.421+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2840410382}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3506350000}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465259355}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2144915438}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465259355}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=667244355}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1365283601}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2613493660}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1365283601}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.422+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=344883887}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=91823257}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1397913885}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=91823257}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=615112521}) 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.437 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863394905}) 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1132318663}) 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863394905}) 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1195327412}) 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637070977}) 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3206076846}) 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637070977}) 2026-02-17T06:03:23.438 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.423+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=50286315}) 2026-02-17T06:03:23.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.661+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.661+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4115821434}) 2026-02-17T06:03:23.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.661+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883001427}) 2026-02-17T06:03:23.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.661+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.661+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4115821434}) 2026-02-17T06:03:23.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.661+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3368798030}) 2026-02-17T06:03:23.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.663+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.663+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2217950018}) 2026-02-17T06:03:23.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.663+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4124835540}) 2026-02-17T06:03:23.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.663+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.663+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2217950018}) 2026-02-17T06:03:23.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.663+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3040322390}) 2026-02-17T06:03:23.663 INFO:tasks.rados.rados.0.trial115.stdout:2887: done (1 left) 2026-02-17T06:03:23.663 INFO:tasks.rados.rados.0.trial115.stdout:2888: done (0 left) 2026-02-17T06:03:23.663 INFO:tasks.rados.rados.0.trial115.stdout:2889: snap_remove snap 293 2026-02-17T06:03:23.779 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.779+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.779 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.779+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901842351}) 2026-02-17T06:03:23.780 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.780+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=236500604}) 2026-02-17T06:03:23.780 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.780+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.780 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.780+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901842351}) 2026-02-17T06:03:23.780 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.780+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=899641827}) 2026-02-17T06:03:23.785 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.785+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.785 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.785+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4166615771}) 2026-02-17T06:03:23.785 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.785+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2140036867}) 2026-02-17T06:03:23.785 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.785+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.785 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.785+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4166615771}) 2026-02-17T06:03:23.785 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.785+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3286389536}) 2026-02-17T06:03:23.786 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.787+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.786 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.787+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1460220604}) 2026-02-17T06:03:23.786 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.787+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2774908213}) 2026-02-17T06:03:23.786 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.787+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.786 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.787+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1460220604}) 2026-02-17T06:03:23.787 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.787+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2105269662}) 2026-02-17T06:03:23.823 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.823 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1820550501}) 2026-02-17T06:03:23.823 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=841910307}) 2026-02-17T06:03:23.823 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.823 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1820550501}) 2026-02-17T06:03:23.823 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.823+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3181641606}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798085232}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=676602200}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798085232}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1268566288}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1040778186}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2836200361}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1040778186}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.958+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3168223480}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3706494237}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=425233078}) 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.958 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3706494237}) 2026-02-17T06:03:23.959 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2112145581}) 2026-02-17T06:03:23.959 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.959 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3210781225}) 2026-02-17T06:03:23.959 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=926094565}) 2026-02-17T06:03:23.959 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:23.959 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3210781225}) 2026-02-17T06:03:23.959 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:23.959+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3597758585}) 2026-02-17T06:03:24.436 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:03:24.437 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.505+0000 7f81b9b75640 10 monclient: get_monmap_and_config 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.505+0000 7f81b9b75640 10 monclient: build_initial_monmap 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.505+0000 7f81b9b75640 10 monclient: monmap: 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:24.505612+0000 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:24.505612+0000 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:24.505 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:24.506 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:24.506 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:24.506 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.505+0000 7f81b9b75640 1 Processor -- start 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.505+0000 7f81b9b75640 1 -- start start 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.505+0000 7f81b9b75640 10 monclient: init 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient(hunting): picked mon.noname-b con 0x7f81b406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f81b409d430 0x7f81b406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient(hunting): picked mon.noname-a con 0x7f81b409d430 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f81b409bf40 0x7f81b4134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient(hunting): picked mon.noname-c con 0x7f81b409bf40 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f81b409ca50 con 0x7f81b409d430 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f81b406d6f0 con 0x7f81b409bf40 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f81b4073600 con 0x7f81b406c890 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b9b75640 10 monclient(hunting): authenticate will time out at 1650.059140s 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38032/0 (socket says 10.20.193.78:38032) 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 -- 10.20.193.78:0/1363436822 learned_addr learned my addr 10.20.193.78:0/1363436822 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient(hunting): get_auth_request con 0x7f81b406c890 auth_method 0 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 1 --2- 10.20.193.78:0/1363436822 >> v2:10.20.193.78:3300/0 conn(0x7f81b409d430 0x7f81b406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 10 monclient(hunting): get_auth_request con 0x7f81b409d430 auth_method 0 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b3fff640 1 --2- 10.20.193.78:0/1363436822 >> v2:10.20.193.115:3300/0 conn(0x7f81b409bf40 0x7f81b4134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient(hunting): handle_auth_done global_id 144779 payload 274 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 -- 10.20.193.78:0/1363436822 >> v2:10.20.193.115:3300/0 conn(0x7f81b409bf40 msgr2=0x7f81b4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 --2- 10.20.193.78:0/1363436822 >> v2:10.20.193.115:3300/0 conn(0x7f81b409bf40 0x7f81b4134df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 -- 10.20.193.78:0/1363436822 >> v2:10.20.193.78:3300/0 conn(0x7f81b409d430 msgr2=0x7f81b406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:24.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 --2- 10.20.193.78:0/1363436822 >> v2:10.20.193.78:3300/0 conn(0x7f81b409d430 0x7f81b406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:24.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 monclient: found mon.noname-b 2026-02-17T06:03:24.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:24.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:03:24.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b37fe640 1 -- 10.20.193.78:0/1363436822 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b406dab0 con 0x7f81b406c890 2026-02-17T06:03:24.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.506+0000 7f81b2ffd640 1 --2- 10.20.193.78:0/1363436822 >> v2:10.20.193.78:3300/0 conn(0x7f81b409d430 0x7f81b406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.508+0000 7f81b37fe640 1 -- 10.20.193.78:0/1363436822 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 msgr2=0x7f81b406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b37fe640 1 --2- 10.20.193.78:0/1363436822 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b406cc70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f81b409c670 tx=0x7f81a002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee48ea48313303b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 1 -- 10.20.193.78:0/1363436822 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a003c070 con 0x7f81b406c890 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: dump: 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: _finish_auth 0 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: _check_auth_tickets 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 5 monclient: authenticate success, global_id 144779 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 1 -- 10.20.193.78:0/1363436822 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a002fca0 con 0x7f81b406c890 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:24.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 1 -- 10.20.193.78:0/1363436822 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0037660 con 0x7f81b406c890 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b27fc640 10 monclient: dump: 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 4 set_mon_vals no callback set 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 10 monclient: shutdown 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 -- 10.20.193.78:0/1363436822 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 msgr2=0x7f81b406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1363436822 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b406cc70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f81b409c670 tx=0x7f81a002ee60 comp rx=0 tx=0).stop 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 -- 10.20.193.78:0/1363436822 shutdown_connections 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1363436822 >> v2:10.20.193.115:3300/0 conn(0x7f81b409bf40 0x7f81b4134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1363436822 >> v2:10.20.193.78:3300/0 conn(0x7f81b409d430 0x7f81b406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1363436822 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b406cc70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 -- 10.20.193.78:0/1363436822 >> 10.20.193.78:0/1363436822 conn(0x7f81b40a6620 msgr2=0x7f81b40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 -- 10.20.193.78:0/1363436822 shutdown_connections 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.509+0000 7f81b9b75640 1 -- 10.20.193.78:0/1363436822 wait complete. 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient: build_initial_monmap 2026-02-17T06:03:24.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient: monmap: 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 1 Processor -- start 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 1 -- start start 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient: init 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b41cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient(hunting): picked mon.noname-c con 0x7f81b406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f81b409bf40 0x7f81b41cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient(hunting): picked mon.noname-a con 0x7f81b409bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81b409d430 0x7f81b4121580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient(hunting): picked mon.noname-b con 0x7f81b409d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4073380 con 0x7f81b409bf40 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4073200 con 0x7f81b409d430 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f81b4135330 con 0x7f81b406c890 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:24.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b9b75640 10 monclient(hunting): authenticate will time out at 1650.063478s 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b41cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f81b409bf40 0x7f81b41cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b41cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38036/0 (socket says 10.20.193.78:38036) 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f81b409bf40 0x7f81b41cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42506/0 (socket says 10.20.193.78:42506) 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 1 -- 10.20.193.78:0/1387811824 learned_addr learned my addr 10.20.193.78:0/1387811824 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 10 monclient(hunting): get_auth_request con 0x7f81b406c890 auth_method 0 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient(hunting): get_auth_request con 0x7f81b409bf40 auth_method 0 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient(hunting): handle_auth_done global_id 184398 payload 824 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 1 -- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 msgr2=0x7f81b41cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b41cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 1 -- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81b409d430 msgr2=0x7f81b4121580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81b409d430 0x7f81b4121580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 1 monclient: found mon.noname-a 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b2ffd640 1 -- 10.20.193.78:0/1387811824 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b4123d80 con 0x7f81b409bf40 2026-02-17T06:03:24.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.510+0000 7f81b37fe640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b41cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:24.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b2ffd640 1 -- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f81b409bf40 msgr2=0x7f81b41cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b2ffd640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f81b409bf40 0x7f81b41cfbf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f81a400ec10 tx=0x7f81a400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 1 -- 10.20.193.78:0/1387811824 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4017070 con 0x7f81b409bf40 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: dump: 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: _finish_auth 0 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 1 -- 10.20.193.78:0/1387811824 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a400ce50 con 0x7f81b409bf40 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 1 -- 10.20.193.78:0/1387811824 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4013690 con 0x7f81b409bf40 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b9b75640 5 monclient: authenticate success, global_id 184398 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b9b75640 10 monclient: _renew_subs 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b1ffb640 4 set_mon_vals no callback set 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b9b75640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b4125510 con 0x7f81b409bf40 2026-02-17T06:03:24.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f818ffff640 10 monclient: dump: 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b9b75640 10 monclient: _renew_subs 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b9b75640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.512+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b40a1540 con 0x7f81b409bf40 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.513+0000 7f81b9b75640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.513+0000 7f81b9b75640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.513+0000 7f81b9b75640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.513+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b4074980 con 0x7f81b409bf40 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.513+0000 7f818ffff640 1 -- 10.20.193.78:0/1387811824 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f81a4024060 con 0x7f81b409bf40 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.513+0000 7f818ffff640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f818403deb0 0x7f8184040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.513+0000 7f818ffff640 1 -- 10.20.193.78:0/1387811824 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(617..617 src has 335..617) ==== 4767+0+0 (secure 0 0 0) 0x7f81a4052720 con 0x7f81b409bf40 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.514+0000 7f81b37fe640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f818403deb0 0x7f8184040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.514+0000 7f81b37fe640 10 monclient: get_auth_request con 0x7f818403deb0 auth_method 0 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.516+0000 7f818ffff640 1 -- 10.20.193.78:0/1387811824 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f81a40093e0 con 0x7f81b409bf40 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.516+0000 7f818ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:24.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.516+0000 7f818ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:24.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.516+0000 7f81b37fe640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f818403deb0 0x7f8184040370 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009730 tx=0x7f81a0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3601278122}) 2026-02-17T06:03:24.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1818842014}) 2026-02-17T06:03:24.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.598 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3601278122}) 2026-02-17T06:03:24.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3767415864}) 2026-02-17T06:03:24.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1107489300}) 2026-02-17T06:03:24.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1783610262}) 2026-02-17T06:03:24.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1107489300}) 2026-02-17T06:03:24.599 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.597+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3223223601}) 2026-02-17T06:03:24.615 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.615 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1319430193}) 2026-02-17T06:03:24.615 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1001686309}) 2026-02-17T06:03:24.615 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1319430193}) 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=505482491}) 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777293503}) 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2993928655}) 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777293503}) 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.604+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=831071955}) 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.616 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=501384550}) 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2707717816}) 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=501384550}) 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3522696030}) 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1010993316}) 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2929177309}) 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1010993316}) 2026-02-17T06:03:24.617 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.605+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4002533404}) 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1297976708}) 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=512152104}) 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1297976708}) 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066966023}) 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193000620}) 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2179211685}) 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193000620}) 2026-02-17T06:03:24.618 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.606+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=840421231}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200030569}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2625362724}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200030569}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2039782169}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=895827699}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2713923325}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=895827699}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3007235977}) 2026-02-17T06:03:24.619 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461110786}) 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956360878}) 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461110786}) 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.607+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=523571708}) 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=433955399}) 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3388715867}) 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=433955399}) 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321473929}) 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.620 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2048242691}) 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2743697813}) 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2048242691}) 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035553862}) 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=783741019}) 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=275205777}) 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.621 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=783741019}) 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.608+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=549910521}) 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3450967473}) 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1011998947}) 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3450967473}) 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=614348421}) 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=805373560}) 2026-02-17T06:03:24.622 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1334730723}) 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=805373560}) 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=637913142}) 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1453202519}) 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585086396}) 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1453202519}) 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.609+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2803899969}) 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.623 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1558450318}) 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3983874353}) 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1558450318}) 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3931942301}) 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224049752}) 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=199561488}) 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224049752}) 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1406131280}) 2026-02-17T06:03:24.624 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4195740237}) 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=305180131}) 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4195740237}) 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.610+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4114587185}) 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208999577}) 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3556711435}) 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.625 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208999577}) 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3189750611}) 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=649287022}) 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1553998384}) 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=649287022}) 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3043032164}) 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1124569747}) 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3116778691}) 2026-02-17T06:03:24.626 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1124569747}) 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=308924802}) 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=398784549}) 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1533381850}) 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=398784549}) 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.611+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494912791}) 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.627 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=8,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=49} crc {logm=3250790204,mdsmap=1160443688,mgr=1202061829,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1772320324,mon_config_key=1912222953,monmap=1784110042,osd_metadata=1501630441,osd_pg_creating=3083763401,osd_snap=1727536183}) 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=46} crc {logm=508374124,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1772320324,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=3083763401,osd_snap=2764295238}) 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=8,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=49} crc {logm=3250790204,mdsmap=1160443688,mgr=1202061829,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1772320324,mon_config_key=1912222953,monmap=1784110042,osd_metadata=1501630441,osd_pg_creating=3083763401,osd_snap=1727536183}) 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=8,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=12,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=2897795933,mdsmap=1160443688,mgr=3779401690,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1772320325,mon_config_key=1253202242,monmap=3140514009,osd_metadata=2997046373,osd_pg_creating=3083763401,osd_snap=582012820}) 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=58,osdmap=42} crc {osd_snap=2607228896,osdmap=17457395}) 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=52,osdmap=48} crc {osd_snap=2285669217,osdmap=3798268489}) 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=58,osdmap=42} crc {osd_snap=2607228896,osdmap=17457395}) 2026-02-17T06:03:24.628 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=57,osdmap=43} crc {osd_snap=3032917070,osdmap=2184492688}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2535620905}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1890332434}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2535620905}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.612+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=793428795}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3682169839}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3498306398}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3682169839}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=239081420}) 2026-02-17T06:03:24.629 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.630 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4212787477}) 2026-02-17T06:03:24.630 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2042411666}) 2026-02-17T06:03:24.630 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.630 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4212787477}) 2026-02-17T06:03:24.630 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.613+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=277795833}) 2026-02-17T06:03:24.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.616+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.616+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=81} crc {osdmap=3776288961}) 2026-02-17T06:03:24.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.616+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=79} crc {osdmap=3021118066}) 2026-02-17T06:03:24.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.616+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:03:24.632 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.616+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=81} crc {osdmap=3776288961}) 2026-02-17T06:03:24.632 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:03:24.616+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=1699010199}) 2026-02-17T06:03:24.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.631+0000 7f81b9b75640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:03:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.631+0000 7f81b9b75640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.631+0000 7f81b9b75640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.631+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f81b4134df0 con 0x7f81b409bf40 2026-02-17T06:03:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.631+0000 7f818ffff640 1 -- 10.20.193.78:0/1387811824 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f81a4013d40 con 0x7f81b409bf40 2026-02-17T06:03:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.631+0000 7f818ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:24.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.631+0000 7f818ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:24.633 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":206,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:03:24.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f818403deb0 msgr2=0x7f8184040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f818403deb0 0x7f8184040370 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009730 tx=0x7f81a0033000 comp rx=0 tx=0).stop 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 10 monclient: shutdown 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f81b409bf40 msgr2=0x7f81b41cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f81b409bf40 0x7f81b41cfbf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f81a400ec10 tx=0x7f81a400c520 comp rx=0 tx=0).stop 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 shutdown_connections 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f818403deb0 0x7f8184040370 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f81b409d430 0x7f81b4121580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f81b409bf40 0x7f81b41cfbf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 --2- 10.20.193.78:0/1387811824 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81b406c890 0x7f81b41cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 >> 10.20.193.78:0/1387811824 conn(0x7f81b40a6620 msgr2=0x7f81b4135e00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 shutdown_connections 2026-02-17T06:03:24.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.633+0000 7f81b9b75640 1 -- 10.20.193.78:0/1387811824 wait complete. 2026-02-17T06:03:24.641 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:03:24.641 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:03:24.641 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:03:24.642 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:03:24.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient: get_monmap_and_config 2026-02-17T06:03:24.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient: build_initial_monmap 2026-02-17T06:03:24.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient: monmap: 2026-02-17T06:03:24.713 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:24.713 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.713 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:24.710359+0000 2026-02-17T06:03:24.713 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:24.710359+0000 2026-02-17T06:03:24.713 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:24.713 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 1 Processor -- start 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 1 -- start start 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient: init 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:24.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6038135f20 0x7f6038136300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient(hunting): picked mon.noname-a con 0x7f6038135f20 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 0x7f603806c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient(hunting): picked mon.noname-b con 0x7f6038136840 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f603806cdd0 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603809d140 con 0x7f603806cdd0 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f603809cb10 con 0x7f6038135f20 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6038135c60 con 0x7f6038136840 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.710+0000 7f603f6d8640 10 monclient(hunting): authenticate will time out at 1650.263962s 2026-02-17T06:03:24.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6038135f20 0x7f6038136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6038135f20 0x7f6038136300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42524/0 (socket says 10.20.193.78:42524) 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 1 -- 10.20.193.78:0/591903279 learned_addr learned my addr 10.20.193.78:0/591903279 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient(hunting): get_auth_request con 0x7f6038135f20 auth_method 0 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603ded5640 1 --2- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 0x7f603806c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603ded5640 10 monclient(hunting): get_auth_request con 0x7f6038136840 auth_method 0 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603ded5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603ded5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603ded5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient(hunting): handle_auth_done global_id 184401 payload 274 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 1 -- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 msgr2=0x7f603806c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 1 --2- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 0x7f603806c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 1 -- 10.20.193.78:0/591903279 >> v1:10.20.193.115:6789/0 conn(0x7f603806cdd0 legacy=0x7f603813cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient: _finish_hunting 0 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 1 monclient: found mon.noname-a 2026-02-17T06:03:24.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:24.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.711+0000 7f603e6d6640 1 -- 10.20.193.78:0/591903279 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603809c330 con 0x7f6038135f20 2026-02-17T06:03:24.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603e6d6640 1 -- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038135f20 msgr2=0x7f6038136300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603e6d6640 1 --2- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038135f20 0x7f6038136300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6030009970 tx=0x7f603002ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f8660e18090c58 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 1 -- 10.20.193.78:0/591903279 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603003c070 con 0x7f6038135f20 2026-02-17T06:03:24.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:24.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: dump: 2026-02-17T06:03:24.728 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.728 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: _finish_auth 0 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: _check_auth_tickets 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:24.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 1 -- 10.20.193.78:0/591903279 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603002fcd0 con 0x7f6038135f20 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603f6d8640 5 monclient: authenticate success, global_id 184401 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 1 -- 10.20.193.78:0/591903279 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030037660 con 0x7f6038135f20 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603d6d4640 10 monclient: dump: 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.730 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603f6d8640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.713+0000 7f603f6d8640 4 set_mon_vals no callback set 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 10 monclient: shutdown 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 -- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038135f20 msgr2=0x7f6038136300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 --2- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038135f20 0x7f6038136300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6030009970 tx=0x7f603002ee90 comp rx=0 tx=0).stop 2026-02-17T06:03:24.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 -- 10.20.193.78:0/591903279 shutdown_connections 2026-02-17T06:03:24.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 --2- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 0x7f603806c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 --2- 10.20.193.78:0/591903279 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038135f20 0x7f6038136300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 -- 10.20.193.78:0/591903279 >> 10.20.193.78:0/591903279 conn(0x7f60380a65f0 msgr2=0x7f60380a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:24.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 -- 10.20.193.78:0/591903279 shutdown_connections 2026-02-17T06:03:24.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 -- 10.20.193.78:0/591903279 wait complete. 2026-02-17T06:03:24.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 10 monclient: build_initial_monmap 2026-02-17T06:03:24.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 10 monclient: monmap: 2026-02-17T06:03:24.732 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 Processor -- start 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 1 -- start start 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.714+0000 7f603f6d8640 10 monclient: init 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f603806cdd0 0x7f60381cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient(hunting): picked mon.noname-a con 0x7f603806cdd0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6038135f20 0x7f60381345b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient(hunting): picked mon.noname-b con 0x7f6038135f20 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 0x7f6038134af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient(hunting): picked mon.noname-c con 0x7f6038136840 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f603809ff90 con 0x7f603806cdd0 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f603809fe10 con 0x7f6038135f20 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f60380a02f0 con 0x7f6038136840 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603f6d8640 10 monclient(hunting): authenticate will time out at 1650.268121s 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f603806cdd0 0x7f60381cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f603806cdd0 0x7f60381cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42528/0 (socket says 10.20.193.78:42528) 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 -- 10.20.193.78:0/603525430 learned_addr learned my addr 10.20.193.78:0/603525430 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient(hunting): get_auth_request con 0x7f603806cdd0 auth_method 0 2026-02-17T06:03:24.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603eed7640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 0x7f6038134af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603eed7640 10 monclient(hunting): get_auth_request con 0x7f6038136840 auth_method 0 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603eed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603eed7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603eed7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603eed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603eed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603eed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient(hunting): handle_auth_done global_id 184404 payload 824 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 msgr2=0x7f6038134af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6038136840 0x7f6038134af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6038135f20 msgr2=0x7f60381345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6038135f20 0x7f60381345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient: _finish_hunting 0 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 monclient: found mon.noname-a 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.715+0000 7f603e6d6640 1 -- 10.20.193.78:0/603525430 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60381cf370 con 0x7f603806cdd0 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603e6d6640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f603806cdd0 msgr2=0x7f60381cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603e6d6640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f603806cdd0 0x7f60381cee30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6030009970 tx=0x7f6030004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603003c040 con 0x7f603806cdd0 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: dump: 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.738 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: _finish_auth 0 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030047070 con 0x7f603806cdd0 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603f6d8640 5 monclient: authenticate success, global_id 184404 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030042890 con 0x7f603806cdd0 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603ced3640 4 set_mon_vals no callback set 2026-02-17T06:03:24.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f6022ffd640 10 monclient: dump: 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603f6d8640 10 monclient: _renew_subs 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603f6d8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6038126180 con 0x7f603806cdd0 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603f6d8640 10 monclient: _renew_subs 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603f6d8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.717+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603809f100 con 0x7f603806cdd0 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603f6d8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603f6d8640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038127970 0x7f6038124040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603f6d8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6038135030 con 0x7f6038127970 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6030042a30 con 0x7f603806cdd0 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038127970 0x7f6038124040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 10 monclient: get_auth_request con 0x7f6038127970 auth_method 0 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f6022ffd640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f600803dee0 0x7f60080403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 10 monclient(hunting): handle_auth_done global_id 184407 payload 824 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603ded5640 1 -- 10.20.193.78:0/603525430 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6038136300 con 0x7f6038127970 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(617..617 src has 335..617) ==== 4767+0+0 (secure 0 0 0) 0x7f6030077370 con 0x7f603806cdd0 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603eed7640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f600803dee0 0x7f60080403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.718+0000 7f603eed7640 10 monclient: get_auth_request con 0x7f600803dee0 auth_method 0 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f603ded5640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038127970 msgr2=0x7f6038124040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f603ded5640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038127970 0x7f6038124040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f602400e8b0 tx=0x7f602400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024017070 con 0x7f6038127970 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 10 monclient: dump: 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f602400cb40 con 0x7f6038127970 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038127970 msgr2=0x7f6038124040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038127970 0x7f6038124040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f602400e8b0 tx=0x7f602400ed80 comp rx=0 tx=0).stop 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038127970 msgr2=0x7f6038124040 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.720+0000 7f6022ffd640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038127970 0x7f6038124040 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f602400e8b0 tx=0x7f602400ed80 comp rx=0 tx=0).stop 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.721+0000 7f603eed7640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f600803dee0 0x7f60080403a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f602c00f2e0 tx=0x7f602c0042b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.723+0000 7f603f6d8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.723+0000 7f603f6d8640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038124580 0x7f60381da180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.723+0000 7f603f6d8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.723+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6038127d50 con 0x7f6038124580 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038124580 0x7f60381da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 10 monclient: get_auth_request con 0x7f6038124580 auth_method 0 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 10 monclient(hunting): handle_auth_done global_id 184410 payload 824 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.724+0000 7f603ded5640 1 -- 10.20.193.78:0/603525430 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f60380744d0 con 0x7f6038124580 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603ded5640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038124580 msgr2=0x7f60381da180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603ded5640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038124580 0x7f60381da180 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6024018000 tx=0x7f60240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024017030 con 0x7f6038124580 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 10 monclient: dump: 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.745 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7f6024005e40 con 0x7f6038124580 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038124580 msgr2=0x7f60381da180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038124580 0x7f60381da180 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6024018000 tx=0x7f60240047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038124580 msgr2=0x7f60381da180 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f6022ffd640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6038124580 0x7f60381da180 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6024018000 tx=0x7f60240047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.746 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 206, 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 169506, 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:03:24.747 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:24.748 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:03:24.749 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:24.750 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:03:24.751 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:24.752 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.753 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:24.754 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:24.755 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:03:24.756 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:03:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f600803dee0 msgr2=0x7f60080403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f600803dee0 0x7f60080403a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f602c00f2e0 tx=0x7f602c0042b0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 10 monclient: shutdown 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f603806cdd0 msgr2=0x7f60381cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 1 --2- 10.20.193.78:0/603525430 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f603806cdd0 0x7f60381cee30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6030009970 tx=0x7f6030004830 comp rx=0 tx=0).stop 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603eed7640 1 -- 10.20.193.78:0/603525430 reap_dead start 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603eed7640 1 -- 10.20.193.78:0/603525430 reap_dead start 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 shutdown_connections 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 >> 10.20.193.78:0/603525430 conn(0x7f60380a65f0 msgr2=0x7f60380a23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 shutdown_connections 2026-02-17T06:03:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.726+0000 7f603f6d8640 1 -- 10.20.193.78:0/603525430 wait complete. 2026-02-17T06:03:24.758 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:03:24.758 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:03:24.759 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:03:24.780 INFO:tasks.rados.rados.0.trial115.stdout:2889: done (0 left) 2026-02-17T06:03:24.781 INFO:tasks.rados.rados.0.trial115.stdout:2890: delete oid 15 current snap is 296 2026-02-17T06:03:24.784 INFO:tasks.rados.rados.0.trial115.stdout:2890: done (0 left) 2026-02-17T06:03:24.784 INFO:tasks.rados.rados.0.trial115.stdout:2891: snap_remove snap 294 2026-02-17T06:03:24.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.828+0000 7fb3410bf640 10 monclient: get_monmap_and_config 2026-02-17T06:03:24.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.828+0000 7fb3410bf640 10 monclient: build_initial_monmap 2026-02-17T06:03:24.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.828+0000 7fb3410bf640 10 monclient: monmap: 2026-02-17T06:03:24.831 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:24.831 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.832 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:24.828723+0000 2026-02-17T06:03:24.832 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:24.828723+0000 2026-02-17T06:03:24.832 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:24.832 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.832 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:24.832 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:24.832 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:24.833 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:24.833 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:24.833 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.828+0000 7fb3410bf640 1 Processor -- start 2026-02-17T06:03:24.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.828+0000 7fb3410bf640 1 -- start start 2026-02-17T06:03:24.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.828+0000 7fb3410bf640 10 monclient: init 2026-02-17T06:03:24.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:24.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2026-02-17T06:03:24.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb33c06c890 0x7fb33c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient(hunting): picked mon.noname-a con 0x7fb33c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:24.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb33c09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:24.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb33c06d1b0 0x7fb33c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient(hunting): picked mon.noname-c con 0x7fb33c06d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:24.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb33c06daf0 con 0x7fb33c09c4a0 2026-02-17T06:03:24.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb33c0a0880 con 0x7fb33c06c890 2026-02-17T06:03:24.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb33c09cc80 con 0x7fb33c06d1b0 2026-02-17T06:03:24.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:24.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb3410bf640 10 monclient(hunting): authenticate will time out at 1650.382261s 2026-02-17T06:03:24.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb33c06c890 0x7fb33c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33a575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb33c09c4a0 0x7fb33c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:32776/0 (socket says 10.20.193.78:32776) 2026-02-17T06:03:24.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33a575640 1 -- 10.20.193.78:0/3678593367 learned_addr learned my addr 10.20.193.78:0/3678593367 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:03:24.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient(hunting): get_auth_request con 0x7fb33c06c890 auth_method 0 2026-02-17T06:03:24.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33b577640 1 --2- 10.20.193.78:0/3678593367 >> v2:10.20.193.115:3300/0 conn(0x7fb33c06d1b0 0x7fb33c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33b577640 10 monclient(hunting): get_auth_request con 0x7fb33c06d1b0 auth_method 0 2026-02-17T06:03:24.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient(hunting): handle_auth_done global_id 184413 payload 274 2026-02-17T06:03:24.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 1 -- 10.20.193.78:0/3678593367 >> v2:10.20.193.115:3300/0 conn(0x7fb33c06d1b0 msgr2=0x7fb33c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:24.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 1 --2- 10.20.193.78:0/3678593367 >> v2:10.20.193.115:3300/0 conn(0x7fb33c06d1b0 0x7fb33c135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 1 -- 10.20.193.78:0/3678593367 >> v1:10.20.193.78:6789/0 conn(0x7fb33c09c4a0 legacy=0x7fb33c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:24.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:03:24.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 1 monclient: found mon.noname-a 2026-02-17T06:03:24.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:24.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.829+0000 7fb33ad76640 1 -- 10.20.193.78:0/3678593367 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c09cf60 con 0x7fb33c06c890 2026-02-17T06:03:24.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb33ad76640 1 -- 10.20.193.78:0/3678593367 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c06c890 msgr2=0x7fb33c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb33ad76640 1 --2- 10.20.193.78:0/3678593367 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c06c890 0x7fb33c06cc70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb33c074de0 tx=0x7fb33002ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a01b99c467cd432 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 1 -- 10.20.193.78:0/3678593367 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33003c070 con 0x7fb33c06c890 2026-02-17T06:03:24.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:24.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: dump: 2026-02-17T06:03:24.855 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.855 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.855 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.855 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.855 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.855 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.855 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.856 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.856 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: _finish_auth 0 2026-02-17T06:03:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: _check_auth_tickets 2026-02-17T06:03:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 1 -- 10.20.193.78:0/3678593367 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33002fbd0 con 0x7fb33c06c890 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 5 monclient: authenticate success, global_id 184413 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 1 -- 10.20.193.78:0/3678593367 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330037440 con 0x7fb33c06c890 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 4 set_mon_vals no callback set 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb339d74640 10 monclient: dump: 2026-02-17T06:03:24.857 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 10 monclient: shutdown 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 1 -- 10.20.193.78:0/3678593367 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c06c890 msgr2=0x7fb33c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 1 --2- 10.20.193.78:0/3678593367 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c06c890 0x7fb33c06cc70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb33c074de0 tx=0x7fb33002ed50 comp rx=0 tx=0).stop 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 1 -- 10.20.193.78:0/3678593367 shutdown_connections 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 1 --2- 10.20.193.78:0/3678593367 >> v2:10.20.193.115:3300/0 conn(0x7fb33c06d1b0 0x7fb33c135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 1 --2- 10.20.193.78:0/3678593367 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c06c890 0x7fb33c06cc70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 1 -- 10.20.193.78:0/3678593367 >> 10.20.193.78:0/3678593367 conn(0x7fb33c0a6620 msgr2=0x7fb33c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 1 -- 10.20.193.78:0/3678593367 shutdown_connections 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 1 -- 10.20.193.78:0/3678593367 wait complete. 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 10 monclient: build_initial_monmap 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.832+0000 7fb3410bf640 10 monclient: monmap: 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:24.859 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 1 Processor -- start 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 1 -- start start 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient: init 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c06c890 0x7fb33c1360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient(hunting): picked mon.noname-c con 0x7fb33c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb33c06d1b0 0x7fb33c1365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient(hunting): picked mon.noname-b con 0x7fb33c06d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c09c4a0 0x7fb33c123b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient(hunting): picked mon.noname-a con 0x7fb33c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c0a01a0 con 0x7fb33c09c4a0 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c0a0020 con 0x7fb33c06d1b0 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb33c09be20 con 0x7fb33c06c890 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb3410bf640 10 monclient(hunting): authenticate will time out at 1650.386584s 2026-02-17T06:03:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c06c890 0x7fb33c1360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c09c4a0 0x7fb33c123b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c06c890 0x7fb33c1360b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38066/0 (socket says 10.20.193.78:38066) 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 1 -- 10.20.193.78:0/63847149 learned_addr learned my addr 10.20.193.78:0/63847149 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 10 monclient(hunting): get_auth_request con 0x7fb33c06c890 auth_method 0 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient(hunting): get_auth_request con 0x7fb33c09c4a0 auth_method 0 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33a575640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb33c06d1b0 0x7fb33c1365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33a575640 10 monclient(hunting): get_auth_request con 0x7fb33c06d1b0 auth_method 0 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient(hunting): handle_auth_done global_id 184416 payload 824 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c06c890 msgr2=0x7fb33c1360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c06c890 0x7fb33c1360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb33c06d1b0 msgr2=0x7fb33c1365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb33c06d1b0 0x7fb33c1365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient: _finish_hunting 0 2026-02-17T06:03:24.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 1 monclient: found mon.noname-a 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33b577640 1 -- 10.20.193.78:0/63847149 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c1274f0 con 0x7fb33c09c4a0 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.833+0000 7fb33ad76640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c06c890 0x7fb33c1360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.835+0000 7fb33b577640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c09c4a0 msgr2=0x7fb33c123b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.835+0000 7fb33b577640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c09c4a0 0x7fb33c123b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb32c007d40 tx=0x7fb32c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c003a90 con 0x7fb33c09c4a0 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: dump: 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.864 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32c003c50 con 0x7fb33c09c4a0 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 5 monclient: authenticate success, global_id 184416 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 10 monclient: _renew_subs 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c125c20 con 0x7fb33c09c4a0 2026-02-17T06:03:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c004020 con 0x7fb33c09c4a0 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb339573640 4 set_mon_vals no callback set 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 10 monclient: dump: 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.866 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 10 monclient: _renew_subs 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c073000 con 0x7fb33c09c4a0 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c1cd5b0 0x7fb33c127730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb33c068410 con 0x7fb33c1cd5b0 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c1cd5b0 0x7fb33c127730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 10 monclient: get_auth_request con 0x7fb33c1cd5b0 auth_method 0 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb32c004240 con 0x7fb33c09c4a0 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 10 monclient(hunting): handle_auth_done global_id 144794 payload 824 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb33ad76640 1 -- 10.20.193.78:0/63847149 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb33c074980 con 0x7fb33c1cd5b0 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb30803df30 0x7fb3080403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.836+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(618..618 src has 335..618) ==== 4795+0+0 (secure 0 0 0) 0x7fb32c052b30 con 0x7fb33c09c4a0 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.837+0000 7fb33a575640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb30803df30 0x7fb3080403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.837+0000 7fb33a575640 10 monclient: get_auth_request con 0x7fb30803df30 auth_method 0 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.838+0000 7fb33ad76640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c1cd5b0 msgr2=0x7fb33c127730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.838+0000 7fb33ad76640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c1cd5b0 0x7fb33c127730 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb33003a040 tx=0x7fb3300047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330048070 con 0x7fb33c1cd5b0 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 10 monclient: dump: 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:24.868 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fb330041850 con 0x7fb33c1cd5b0 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c1cd5b0 msgr2=0x7fb33c127730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c1cd5b0 0x7fb33c127730 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb33003a040 tx=0x7fb3300047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c1cd5b0 msgr2=0x7fb33c127730 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb31b7fe640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c1cd5b0 0x7fb33c127730 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb33003a040 tx=0x7fb3300047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.839+0000 7fb33a575640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb30803df30 0x7fb3080403f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb33c071e90 tx=0x7fb324009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb3410bf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb3410bf640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 0x7fb33c1d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb3410bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb33c06cc70 con 0x7fb33c124060 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 0x7fb33c1d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 10 monclient: get_auth_request con 0x7fb33c124060 auth_method 0 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 10 monclient(hunting): handle_auth_done global_id 144797 payload 824 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 1 -- 10.20.193.78:0/63847149 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb33c0744d0 con 0x7fb33c124060 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 msgr2=0x7fb33c1d86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 msgr2=0x7fb33c1d86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:03:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 0x7fb33c1d86d0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7fb330047000 tx=0x7fb330009290 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:24.842+0000 7fb33ad76640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 0x7fb33c1d86d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:25.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.042+0000 7fb33ad76640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 0x7fb33c1d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.042+0000 7fb33ad76640 10 monclient: get_auth_request con 0x7fb33c124060 auth_method 0 2026-02-17T06:03:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.042+0000 7fb33ad76640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.042+0000 7fb33ad76640 10 monclient: _init_auth method 2 2026-02-17T06:03:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.042+0000 7fb33ad76640 10 monclient: _init_auth already have auth, reseting 2026-02-17T06:03:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.042+0000 7fb33ad76640 10 monclient: handle_auth_reply_more payload 9 2026-02-17T06:03:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.042+0000 7fb33ad76640 10 monclient: handle_auth_reply_more payload_len 9 2026-02-17T06:03:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.042+0000 7fb33ad76640 10 monclient: handle_auth_reply_more responding with 132 bytes 2026-02-17T06:03:25.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.043+0000 7fb33ad76640 10 monclient: handle_auth_done global_id 144797 payload 293 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb33ad76640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 msgr2=0x7fb33c1d86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb33ad76640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 0x7fb33c1d86d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb33003a040 tx=0x7fb33004f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33003c040 con 0x7fb33c124060 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 10 monclient: dump: 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.046 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fb330058640 con 0x7fb33c124060 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:25.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 msgr2=0x7fb33c1d86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 0x7fb33c1d86d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb33003a040 tx=0x7fb33004f040 comp rx=0 tx=0).stop 2026-02-17T06:03:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 msgr2=0x7fb33c1d86d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:25.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb31b7fe640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb33c124060 0x7fb33c1d86d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.048 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:03:25.048 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:25.048 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 206, 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 388579, 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:25.049 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:25.050 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:25.051 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:03:25.052 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:03:25.053 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.054 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.055 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:25.056 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.057 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:25.058 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.059 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.060 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.061 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.062 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.063 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:03:25.063 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:03:25.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb30803df30 msgr2=0x7fb3080403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb30803df30 0x7fb3080403f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb33c071e90 tx=0x7fb324009040 comp rx=0 tx=0).stop 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 10 monclient: shutdown 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c09c4a0 msgr2=0x7fb33c123b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c09c4a0 0x7fb33c123b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb32c007d40 tx=0x7fb32c00e4b0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb33b577640 1 -- 10.20.193.78:0/63847149 reap_dead start 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 shutdown_connections 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 1 --2- 10.20.193.78:0/63847149 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb33c09c4a0 0x7fb33c123b20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 >> 10.20.193.78:0/63847149 conn(0x7fb33c0a6620 msgr2=0x7fb33c0722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.045+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 shutdown_connections 2026-02-17T06:03:25.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.046+0000 7fb3410bf640 1 -- 10.20.193.78:0/63847149 wait complete. 2026-02-17T06:03:25.066 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:03:25.066 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:03:25.066 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:03:25.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.134+0000 7f20b856a640 10 monclient: get_monmap_and_config 2026-02-17T06:03:25.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.134+0000 7f20b856a640 10 monclient: build_initial_monmap 2026-02-17T06:03:25.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.134+0000 7f20b856a640 10 monclient: monmap: 2026-02-17T06:03:25.137 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:25.137 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.137 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:25.134780+0000 2026-02-17T06:03:25.137 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:25.134780+0000 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.134+0000 7f20b856a640 1 Processor -- start 2026-02-17T06:03:25.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.134+0000 7f20b856a640 1 -- start start 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.134+0000 7f20b856a640 10 monclient: init 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient: _add_conns ranks=[1,3,4,0,2] 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f20b006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f20b006d380 0x7f20b00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient(hunting): picked mon.noname-a con 0x7f20b006d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f20b013a240 0x7f20b013c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient(hunting): picked mon.noname-c con 0x7f20b013a240 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b0074150 con 0x7f20b006c890 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f20b009ef70 con 0x7f20b006d380 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f20b009f250 con 0x7f20b013a240 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b856a640 10 monclient(hunting): authenticate will time out at 1650.688306s 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f20b006d380 0x7f20b00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f20b006d380 0x7f20b00a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42580/0 (socket says 10.20.193.78:42580) 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 1 -- 10.20.193.78:0/889959190 learned_addr learned my addr 10.20.193.78:0/889959190 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient(hunting): get_auth_request con 0x7f20b006d380 auth_method 0 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b6ae0640 1 --2- 10.20.193.78:0/889959190 >> v2:10.20.193.115:3300/0 conn(0x7f20b013a240 0x7f20b013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b6ae0640 10 monclient(hunting): get_auth_request con 0x7f20b013a240 auth_method 0 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b6ae0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b6ae0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b6ae0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient(hunting): handle_auth_done global_id 184419 payload 274 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 1 -- 10.20.193.78:0/889959190 >> v2:10.20.193.115:3300/0 conn(0x7f20b013a240 msgr2=0x7f20b013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:25.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 1 --2- 10.20.193.78:0/889959190 >> v2:10.20.193.115:3300/0 conn(0x7f20b013a240 0x7f20b013c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 1 -- 10.20.193.78:0/889959190 >> v1:10.20.193.115:6789/0 conn(0x7f20b006c890 legacy=0x7f20b006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:25.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient: _finish_hunting 0 2026-02-17T06:03:25.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 1 monclient: found mon.noname-a 2026-02-17T06:03:25.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:25.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.135+0000 7f20b5ade640 1 -- 10.20.193.78:0/889959190 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b009df40 con 0x7f20b006d380 2026-02-17T06:03:25.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.137+0000 7f20b5ade640 1 -- 10.20.193.78:0/889959190 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 msgr2=0x7f20b00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:25.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.137+0000 7f20b5ade640 1 --2- 10.20.193.78:0/889959190 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00a01f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20b0069d40 tx=0x7f20a402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8b9a8f9c72ba82 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 1 -- 10.20.193.78:0/889959190 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a403c070 con 0x7f20b006d380 2026-02-17T06:03:25.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: dump: 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: _finish_auth 0 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: _check_auth_tickets 2026-02-17T06:03:25.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 1 -- 10.20.193.78:0/889959190 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a402fd50 con 0x7f20b006d380 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 5 monclient: authenticate success, global_id 184419 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 1 -- 10.20.193.78:0/889959190 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4037660 con 0x7f20b006d380 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b52dd640 10 monclient: dump: 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.150 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 4 set_mon_vals no callback set 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 10 monclient: shutdown 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 1 -- 10.20.193.78:0/889959190 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 msgr2=0x7f20b00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 1 --2- 10.20.193.78:0/889959190 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00a01f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20b0069d40 tx=0x7f20a402eed0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 1 -- 10.20.193.78:0/889959190 shutdown_connections 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 1 --2- 10.20.193.78:0/889959190 >> v2:10.20.193.115:3300/0 conn(0x7f20b013a240 0x7f20b013c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 1 --2- 10.20.193.78:0/889959190 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00a01f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 1 -- 10.20.193.78:0/889959190 >> 10.20.193.78:0/889959190 conn(0x7f20b00a6620 msgr2=0x7f20b00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 1 -- 10.20.193.78:0/889959190 shutdown_connections 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 1 -- 10.20.193.78:0/889959190 wait complete. 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 10 monclient: build_initial_monmap 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.138+0000 7f20b856a640 10 monclient: monmap: 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:25.151 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 1 Processor -- start 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 1 -- start start 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient: init 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b006c890 0x7f20b01ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient(hunting): picked mon.noname-b con 0x7f20b006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:25.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00732f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient(hunting): picked mon.noname-a con 0x7f20b006d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f20b013a240 0x7f20b0132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient(hunting): picked mon.noname-c con 0x7f20b013a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20b013ccf0 con 0x7f20b006d380 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b013cb70 con 0x7f20b006c890 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f20b013ce70 con 0x7f20b013a240 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b856a640 10 monclient(hunting): authenticate will time out at 1650.692458s 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00732f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42592/0 (socket says 10.20.193.78:42592) 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 -- 10.20.193.78:0/4175411674 learned_addr learned my addr 10.20.193.78:0/4175411674 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient(hunting): get_auth_request con 0x7f20b006d380 auth_method 0 2026-02-17T06:03:25.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b6ae0640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f20b013a240 0x7f20b0132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b6ae0640 10 monclient(hunting): get_auth_request con 0x7f20b013a240 auth_method 0 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b6ae0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b6ae0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b6ae0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b6ae0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b6ae0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b6ae0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient(hunting): handle_auth_done global_id 184422 payload 824 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f20b013a240 msgr2=0x7f20b0132300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f20b013a240 0x7f20b0132300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b006c890 msgr2=0x7f20b01ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b62df640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b006c890 0x7f20b01ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b006c890 0x7f20b01ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient: _finish_hunting 0 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 monclient: found mon.noname-a 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b5ade640 1 -- 10.20.193.78:0/4175411674 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b01cdeb0 con 0x7f20b006d380 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b62df640 10 monclient: get_auth_request con 0x7f20b006c890 auth_method 0 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.139+0000 7f20b62df640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b006c890 0x7f20b01ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f20b5ade640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 msgr2=0x7f20b00732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f20b5ade640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00732f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20a4002410 tx=0x7f20a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4046020 con 0x7f20b006d380 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:25.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: dump: 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: _finish_auth 0 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a4042420 con 0x7f20b006d380 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a403c040 con 0x7f20b006d380 2026-02-17T06:03:25.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f20b4adc640 4 set_mon_vals no callback set 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f2096ffd640 10 monclient: dump: 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f20b856a640 5 monclient: authenticate success, global_id 184422 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f20b856a640 10 monclient: _renew_subs 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f20b856a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.141+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b0133fa0 con 0x7f20b006d380 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b856a640 10 monclient: _renew_subs 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b856a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b0135cd0 con 0x7f20b006d380 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b856a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b856a640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0134b00 0x7f20b0132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b856a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0074980 con 0x7f20b0134b00 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f20a403f030 con 0x7f20b006d380 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b62df640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0134b00 0x7f20b0132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b62df640 10 monclient: get_auth_request con 0x7f20b0134b00 auth_method 0 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b62df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b62df640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.142+0000 7f20b62df640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f2096ffd640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f208003deb0 0x7f2080040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(618..618 src has 335..618) ==== 4795+0+0 (secure 0 0 0) 0x7f20a4077a30 con 0x7f20b006d380 2026-02-17T06:03:25.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f20b62df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f20b62df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f20b62df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f20b6ae0640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f208003deb0 0x7f2080040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f20b6ae0640 10 monclient: get_auth_request con 0x7f208003deb0 auth_method 0 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f20b62df640 10 monclient(hunting): handle_auth_done global_id 214132 payload 824 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.143+0000 7f20b62df640 1 -- 10.20.193.78:0/4175411674 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20b006eb90 con 0x7f20b0134b00 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.145+0000 7f20b6ae0640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f208003deb0 0x7f2080040370 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f20ac003ba0 tx=0x7f20ac009770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.145+0000 7f20b62df640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0134b00 msgr2=0x7f20b0132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.145+0000 7f20b62df640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0134b00 0x7f20b0132840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d720 tx=0x7f20a000dbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.145+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a000bb30 con 0x7f20b0134b00 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.145+0000 7f2096ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.145+0000 7f2096ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.145+0000 7f2096ffd640 10 monclient: dump: 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.159 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.146+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f20a000bb30 con 0x7f20b0134b00 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.146+0000 7f2096ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.146+0000 7f2096ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.146+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0134b00 msgr2=0x7f20b0132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.146+0000 7f2096ffd640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0134b00 0x7f20b0132840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d720 tx=0x7f20a000dbf0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.146+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0134b00 msgr2=0x7f20b0132840 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:25.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.146+0000 7f2096ffd640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0134b00 0x7f20b0132840 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d720 tx=0x7f20a000dbf0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b856a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b856a640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0073830 0x7f20b01d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b856a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00744d0 con 0x7f20b0073830 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0073830 0x7f20b01d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 10 monclient: get_auth_request con 0x7f20b0073830 auth_method 0 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 10 monclient(hunting): handle_auth_done global_id 214135 payload 824 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.149+0000 7f20b62df640 1 -- 10.20.193.78:0/4175411674 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f20b01cd4d0 con 0x7f20b0073830 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.151+0000 7f20b62df640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0073830 msgr2=0x7f20b01d82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.151+0000 7f20b62df640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0073830 0x7f20b01d82e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d720 tx=0x7f20a0003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.151+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0005490 con 0x7f20b0073830 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.151+0000 7f2096ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.151+0000 7f2096ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.151+0000 7f2096ffd640 10 monclient: dump: 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.163 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f20a0005490 con 0x7f20b0073830 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f2096ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f2096ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0073830 msgr2=0x7f20b01d82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f2096ffd640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0073830 0x7f20b01d82e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d720 tx=0x7f20a0003860 comp rx=0 tx=0).stop 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f2096ffd640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0073830 msgr2=0x7f20b01d82e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f2096ffd640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f20b0073830 0x7f20b01d82e0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d720 tx=0x7f20a0003860 comp rx=0 tx=0).stop 2026-02-17T06:03:25.164 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 206, 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 7998, 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:03:25.165 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:25.166 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:03:25.167 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:25.168 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.169 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:25.170 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.171 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:25.172 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:25.173 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:03:25.174 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:03:25.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f208003deb0 msgr2=0x7f2080040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f208003deb0 0x7f2080040370 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f20ac003ba0 tx=0x7f20ac009770 comp rx=0 tx=0).stop 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 10 monclient: shutdown 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 msgr2=0x7f20b00732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00732f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20a4002410 tx=0x7f20a40047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b6ae0640 1 -- 10.20.193.78:0/4175411674 reap_dead start 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 shutdown_connections 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 --2- 10.20.193.78:0/4175411674 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f20b006d380 0x7f20b00732f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 >> 10.20.193.78:0/4175411674 conn(0x7f20b00a6620 msgr2=0x7f20b01394e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 shutdown_connections 2026-02-17T06:03:25.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.152+0000 7f20b856a640 1 -- 10.20.193.78:0/4175411674 wait complete. 2026-02-17T06:03:25.175 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:03:25.176 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-17T06:03:25.176 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:03:25.176 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-17T06:03:25.176 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T06:03:25.176 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-17T06:03:25.176 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:03:25.189 INFO:tasks.ceph.mon.c:Stopped 2026-02-17T06:03:25.189 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:03:25.189 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:03:25.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.296+0000 7f60219a8640 10 monclient: get_monmap_and_config 2026-02-17T06:03:25.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.296+0000 7f60219a8640 10 monclient: build_initial_monmap 2026-02-17T06:03:25.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.296+0000 7f60219a8640 10 monclient: monmap: 2026-02-17T06:03:25.296 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:25.296 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.296 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:25.296760+0000 2026-02-17T06:03:25.296 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:25.296760+0000 2026-02-17T06:03:25.296 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:25.297 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.297 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:25.297 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:25.297 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:25.297 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:25.297 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:25.297 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.296+0000 7f60219a8640 1 Processor -- start 2026-02-17T06:03:25.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 1 -- start start 2026-02-17T06:03:25.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient: init 2026-02-17T06:03:25.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:25.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2026-02-17T06:03:25.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 0x7f601c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient(hunting): picked mon.noname-b con 0x7f601c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f601c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f601c06d1b0 0x7f601c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient(hunting): picked mon.noname-c con 0x7f601c06d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f601c06daf0 con 0x7f601c09c4a0 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f601c0a0880 con 0x7f601c06d1b0 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f601c09cc80 con 0x7f601c06c890 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f60219a8640 10 monclient(hunting): authenticate will time out at 1650.850345s 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601affd640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 msgr2=0x7f601c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 0x7f601c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601a7fc640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f601c09c4a0 0x7f601c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:60068/0 (socket says 10.20.193.78:60068) 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f601c06d1b0 0x7f601c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601a7fc640 1 -- 10.20.193.78:0/2712071347 learned_addr learned my addr 10.20.193.78:0/2712071347 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient(hunting): get_auth_request con 0x7f601c06d1b0 auth_method 0 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient(hunting): handle_auth_done global_id 214138 payload 274 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 1 -- 10.20.193.78:0/2712071347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 msgr2=0x7f601c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 1 --2- 10.20.193.78:0/2712071347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 0x7f601c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 1 -- 10.20.193.78:0/2712071347 >> v1:10.20.193.115:6789/0 conn(0x7f601c09c4a0 legacy=0x7f601c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.297+0000 7f601b7fe640 1 monclient: found mon.noname-c 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.298+0000 7f601b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:25.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.298+0000 7f601b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:03:25.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.298+0000 7f601b7fe640 1 -- 10.20.193.78:0/2712071347 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c09cf60 con 0x7f601c06d1b0 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f601b7fe640 1 -- 10.20.193.78:0/2712071347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 msgr2=0x7f601c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f601b7fe640 1 --2- 10.20.193.78:0/2712071347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 0x7f601c135500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f600c009d90 tx=0x7f600c02f170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d75be32e460149 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 1 -- 10.20.193.78:0/2712071347 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c03c070 con 0x7f601c06d1b0 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 10 monclient: dump: 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.306 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 10 monclient: _finish_auth 0 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f60219a8640 5 monclient: authenticate success, global_id 214138 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f60219a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 1 -- 10.20.193.78:0/2712071347 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600c037640 con 0x7f601c06d1b0 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.300+0000 7f6019ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f6019ffb640 1 -- 10.20.193.78:0/2712071347 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c037a10 con 0x7f601c06d1b0 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 4 set_mon_vals no callback set 2026-02-17T06:03:25.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f6019ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f6019ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f6019ffb640 10 monclient: dump: 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 10 monclient: shutdown 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 -- 10.20.193.78:0/2712071347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 msgr2=0x7f601c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 --2- 10.20.193.78:0/2712071347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 0x7f601c135500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f600c009d90 tx=0x7f600c02f170 comp rx=0 tx=0).stop 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 -- 10.20.193.78:0/2712071347 shutdown_connections 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 --2- 10.20.193.78:0/2712071347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 0x7f601c135500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 --2- 10.20.193.78:0/2712071347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 0x7f601c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 -- 10.20.193.78:0/2712071347 >> 10.20.193.78:0/2712071347 conn(0x7f601c0a6620 msgr2=0x7f601c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 -- 10.20.193.78:0/2712071347 shutdown_connections 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 -- 10.20.193.78:0/2712071347 wait complete. 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 10 monclient: build_initial_monmap 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 10 monclient: monmap: 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 Processor -- start 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 1 -- start start 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.301+0000 7f60219a8640 10 monclient: init 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 0x7f601c1cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient(hunting): picked mon.noname-c con 0x7f601c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 0x7f601c1cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient(hunting): picked mon.noname-b con 0x7f601c06d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f601c09c4a0 0x7f601c123620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient(hunting): picked mon.noname-a con 0x7f601c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f601c0a01a0 con 0x7f601c09c4a0 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f601c0a0020 con 0x7f601c06d1b0 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f601c09be20 con 0x7f601c06c890 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f60219a8640 10 monclient(hunting): authenticate will time out at 1650.855108s 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601affd640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 msgr2=0x7f601c1cd410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 0x7f601c1cd410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f601c09c4a0 0x7f601c123620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f601c09c4a0 0x7f601c123620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42630/0 (socket says 10.20.193.78:42630) 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 -- 10.20.193.78:0/831888465 learned_addr learned my addr 10.20.193.78:0/831888465 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient(hunting): get_auth_request con 0x7f601c09c4a0 auth_method 0 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601a7fc640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 0x7f601c1cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601a7fc640 10 monclient(hunting): get_auth_request con 0x7f601c06d1b0 auth_method 0 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient(hunting): handle_auth_done global_id 184425 payload 824 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 -- 10.20.193.78:0/831888465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 msgr2=0x7f601c1cd410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 0x7f601c1cd410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 -- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 msgr2=0x7f601c1cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 0x7f601c1cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 monclient: found mon.noname-a 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.302+0000 7f601b7fe640 1 -- 10.20.193.78:0/831888465 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c127200 con 0x7f601c09c4a0 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f601b7fe640 1 -- 10.20.193.78:0/831888465 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f601c09c4a0 msgr2=0x7f601c123620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f601b7fe640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f601c09c4a0 0x7f601c123620 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f600c009ec0 tx=0x7f600c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 1 -- 10.20.193.78:0/831888465 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c047020 con 0x7f601c09c4a0 2026-02-17T06:03:25.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: dump: 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: _finish_auth 0 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: _check_auth_tickets 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 1 -- 10.20.193.78:0/831888465 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600c03d040 con 0x7f601c09c4a0 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f60219a8640 5 monclient: authenticate success, global_id 184425 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 1 -- 10.20.193.78:0/831888465 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c03c070 con 0x7f601c09c4a0 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:25.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f5ff3fff640 10 monclient: dump: 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f60197fa640 4 set_mon_vals no callback set 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f60219a8640 10 monclient: _renew_subs 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f60219a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c136ea0 con 0x7f601c09c4a0 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f60219a8640 10 monclient: _renew_subs 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f60219a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.304+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c126c40 con 0x7f601c09c4a0 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.305+0000 7f60219a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.305+0000 7f60219a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.305+0000 7f60219a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.305+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601c06cc70 con 0x7f601c09c4a0 2026-02-17T06:03:25.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.305+0000 7f5ff3fff640 1 -- 10.20.193.78:0/831888465 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f600c0423e0 con 0x7f601c09c4a0 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.305+0000 7f5ff3fff640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5fec03de60 0x7f5fec040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.305+0000 7f5ff3fff640 1 -- 10.20.193.78:0/831888465 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(618..618 src has 335..618) ==== 4795+0+0 (secure 0 0 0) 0x7f600c078760 con 0x7f601c09c4a0 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.306+0000 7f601affd640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5fec03de60 0x7f5fec040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.306+0000 7f601affd640 10 monclient: get_auth_request con 0x7f5fec03de60 auth_method 0 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.308+0000 7f5ff3fff640 1 -- 10.20.193.78:0/831888465 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f600c051030 con 0x7f601c09c4a0 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.308+0000 7f5ff3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.308+0000 7f5ff3fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.308+0000 7f5ff3fff640 1 -- 10.20.193.78:0/831888465 DispatchQueue::entry inject delay of 0.002000 2026-02-17T06:03:25.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.308+0000 7f601affd640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5fec03de60 0x7f5fec040320 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6010003a80 tx=0x7f6010009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:25.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.423+0000 7f60219a8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:03:25.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.423+0000 7f60219a8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:25.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.423+0000 7f60219a8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:25.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.423+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f601c127440 con 0x7f601c09c4a0 2026-02-17T06:03:25.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.423+0000 7f5ff3fff640 1 -- 10.20.193.78:0/831888465 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f600c005e10 con 0x7f601c09c4a0 2026-02-17T06:03:25.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.423+0000 7f5ff3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:25.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.423+0000 7f5ff3fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:25.423 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":206,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:03:25.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5fec03de60 msgr2=0x7f5fec040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5fec03de60 0x7f5fec040320 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6010003a80 tx=0x7f6010009270 comp rx=0 tx=0).stop 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 10 monclient: shutdown 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f601c09c4a0 msgr2=0x7f601c123620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f601c09c4a0 0x7f601c123620 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f600c009ec0 tx=0x7f600c004b40 comp rx=0 tx=0).stop 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 shutdown_connections 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f601c09c4a0 0x7f601c123620 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5fec03de60 0x7f5fec040320 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f601c06d1b0 0x7f601c1cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 --2- 10.20.193.78:0/831888465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f601c06c890 0x7f601c1cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 >> 10.20.193.78:0/831888465 conn(0x7f601c0a6620 msgr2=0x7f601c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:25.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 shutdown_connections 2026-02-17T06:03:25.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:25.424+0000 7f60219a8640 1 -- 10.20.193.78:0/831888465 wait complete. 2026-02-17T06:03:28.433 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:03:28.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.501+0000 7fd27f13e640 10 monclient: get_monmap_and_config 2026-02-17T06:03:28.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.501+0000 7fd27f13e640 10 monclient: build_initial_monmap 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.501+0000 7fd27f13e640 10 monclient: monmap: 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:28.502107+0000 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:28.502107+0000 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 1 Processor -- start 2026-02-17T06:03:28.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 1 -- start start 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient: init 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 0x7fd27806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient(hunting): picked mon.noname-b con 0x7fd27806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd27809d430 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd27809bf40 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd278135c10 con 0x7fd27809bf40 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd278073600 con 0x7fd27809d430 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd27809ca50 con 0x7fd27806c890 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27f13e640 10 monclient(hunting): authenticate will time out at 1654.055665s 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27ceb3640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 msgr2=0x7fd27806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27ceb3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 0x7fd27806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27d6b4640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd27809bf40 0x7fd278135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:42650/0 (socket says 10.20.193.78:42650) 2026-02-17T06:03:28.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.502+0000 7fd27d6b4640 1 -- 10.20.193.78:0/2032223779 learned_addr learned my addr 10.20.193.78:0/2032223779 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:03:28.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.504+0000 7fd27d6b4640 1 -- 10.20.193.78:0/2032223779 >> v1:10.20.193.78:6789/0 conn(0x7fd27809bf40 legacy=0x7fd278135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:28.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.504+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1634946287 0 0) 0x7fd278135c10 con 0x7fd27809bf40 2026-02-17T06:03:28.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): _negotiate 2026-02-17T06:03:28.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:28.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:28.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): my global_id is 184428 2026-02-17T06:03:28.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd264003620 con 0x7fd27809bf40 2026-02-17T06:03:28.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26ffff640 1 -- 10.20.193.78:0/2032223779 >> v1:10.20.193.115:6789/0 conn(0x7fd27809d430 legacy=0x7fd278134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:28.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1450790829 0 0) 0x7fd264003620 con 0x7fd27809bf40 2026-02-17T06:03:28.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 msgr2=0x7fd27806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 0x7fd27806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 >> v1:10.20.193.115:6789/0 conn(0x7fd27809d430 legacy=0x7fd278134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27806dab0 con 0x7fd27809bf40 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fd274003530 con 0x7fd27809bf40 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27f13e640 5 monclient: authenticate success, global_id 184428 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27f13e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: dump: 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 >> v1:10.20.193.78:6789/0 conn(0x7fd27809bf40 legacy=0x7fd278135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd264003b30 0x7fd264003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): picked mon.a con 0x7fd264003b30 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264004900 0x7fd264004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): picked mon.b con 0x7fd264004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 0x7fd26400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): picked mon.c con 0x7fd27806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd27809ca50 con 0x7fd264003b30 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:28.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd26400c000 con 0x7fd264004900 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd26400c180 con 0x7fd27806c890 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27f13e640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27d6b4640 1 -- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 msgr2=0x7fd26400bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27d6b4640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 0x7fd26400bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd264003b30 0x7fd264003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient(hunting): get_auth_request con 0x7fd264003b30 auth_method 0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26ffff640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264004900 0x7fd264004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26ffff640 10 monclient(hunting): get_auth_request con 0x7fd264004900 auth_method 0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd26ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient(hunting): handle_auth_done global_id 184428 payload 293 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 1 -- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 msgr2=0x7fd26400bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd27806c890 0x7fd26400bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 1 -- 10.20.193.78:0/2032223779 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264004900 msgr2=0x7fd264004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264004900 0x7fd264004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient: _finish_hunting 0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 1 monclient: found mon.a 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 1 -- 10.20.193.78:0/2032223779 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd264003620 con 0x7fd264003b30 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient: _finish_auth 0 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 10 monclient: _check_auth_tickets 2026-02-17T06:03:28.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.505+0000 7fd27ceb3640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.507+0000 7fd27ceb3640 1 -- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd264003b30 msgr2=0x7fd264003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.507+0000 7fd27ceb3640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd264003b30 0x7fd264003f50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd268009e30 tx=0x7fd26802f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c54ed32c6280d7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268009a30 con 0x7fd264003b30 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 10 monclient: dump: 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd268003cb0 con 0x7fd264003b30 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 1 -- 10.20.193.78:0/2032223779 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268003e60 con 0x7fd264003b30 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 4 set_mon_vals no callback set 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd26f7fe640 10 monclient: dump: 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.508 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 10 monclient: shutdown 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 1 -- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd264003b30 msgr2=0x7fd264003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 1 --2- 10.20.193.78:0/2032223779 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd264003b30 0x7fd264003f50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd268009e30 tx=0x7fd26802f660 comp rx=0 tx=0).stop 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27d6b4640 1 -- 10.20.193.78:0/2032223779 reap_dead start 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 1 -- 10.20.193.78:0/2032223779 shutdown_connections 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 1 -- 10.20.193.78:0/2032223779 >> 10.20.193.78:0/2032223779 conn(0x7fd2780a6620 msgr2=0x7fd2780a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 1 -- 10.20.193.78:0/2032223779 shutdown_connections 2026-02-17T06:03:28.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 1 -- 10.20.193.78:0/2032223779 wait complete. 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 10 monclient: build_initial_monmap 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.508+0000 7fd27f13e640 10 monclient: monmap: 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 1 Processor -- start 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 1 -- start start 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient: init 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264003b30 0x7fd2781cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient(hunting): picked mon.noname-b con 0x7fd264003b30 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd2781cfaa0 0x7fd2781223e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient(hunting): picked mon.noname-a con 0x7fd2781cfaa0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd2781cff20 0x7fd278122920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient(hunting): picked mon.noname-c con 0x7fd2781cff20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2780733c0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd278073240 con 0x7fd264003b30 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd27806cc70 con 0x7fd2781cff20 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27f13e640 10 monclient(hunting): authenticate will time out at 1654.062403s 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27d6b4640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd2781cff20 msgr2=0x7fd278122920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27d6b4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd2781cff20 0x7fd278122920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd2781cfaa0 0x7fd2781223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd2781cfaa0 0x7fd2781223e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57090/0 (socket says 10.20.193.78:57090) 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 -- 10.20.193.78:0/79214306 learned_addr learned my addr 10.20.193.78:0/79214306 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient(hunting): get_auth_request con 0x7fd2781cfaa0 auth_method 0 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient(hunting): handle_auth_done global_id 184431 payload 824 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 -- 10.20.193.78:0/79214306 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd2781cff20 msgr2=0x7fd278122920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27ceb3640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264003b30 0x7fd2781cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd2781cff20 0x7fd278122920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 -- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264003b30 msgr2=0x7fd2781cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264003b30 0x7fd2781cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 monclient: found mon.noname-a 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd26ffff640 1 -- 10.20.193.78:0/79214306 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd278124be0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27ceb3640 10 monclient: get_auth_request con 0x7fd264003b30 auth_method 0 2026-02-17T06:03:28.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.509+0000 7fd27ceb3640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264003b30 0x7fd2781cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26ffff640 1 -- 10.20.193.78:0/79214306 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd2781cfaa0 msgr2=0x7fd2781223e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26ffff640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd2781cfaa0 0x7fd2781223e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e780 tx=0x7fd26000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 1 -- 10.20.193.78:0/79214306 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260005ad0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: dump: 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: _finish_auth 0 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 1 -- 10.20.193.78:0/79214306 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd260018070 con 0x7fd2781cfaa0 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 1 -- 10.20.193.78:0/79214306 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2600070d0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26effd640 4 set_mon_vals no callback set 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd26d7fa640 10 monclient: dump: 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:28.512 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd27f13e640 5 monclient: authenticate success, global_id 184431 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd27f13e640 10 monclient: _renew_subs 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd27f13e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd278126090 con 0x7fd2781cfaa0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd27f13e640 10 monclient: _renew_subs 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd27f13e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.511+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd278125db0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.512+0000 7fd27f13e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.512+0000 7fd27f13e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.512+0000 7fd27f13e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.512+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27806d0a0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.512+0000 7fd26d7fa640 1 -- 10.20.193.78:0/79214306 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd260007270 con 0x7fd2781cfaa0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.512+0000 7fd26d7fa640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd24803de60 0x7fd248040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.512+0000 7fd26d7fa640 1 -- 10.20.193.78:0/79214306 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(618..618 src has 335..618) ==== 4795+0+0 (secure 0 0 0) 0x7fd260052640 con 0x7fd2781cfaa0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.513+0000 7fd27ceb3640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd24803de60 0x7fd248040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:28.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.513+0000 7fd27ceb3640 10 monclient: get_auth_request con 0x7fd24803de60 auth_method 0 2026-02-17T06:03:28.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.515+0000 7fd26d7fa640 1 -- 10.20.193.78:0/79214306 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd26001bdc0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.515+0000 7fd26d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:28.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.515+0000 7fd26d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:28.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.515+0000 7fd27ceb3640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd24803de60 0x7fd248040320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd268034000 tx=0x7fd268005000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:28.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.627+0000 7fd27f13e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:03:28.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.627+0000 7fd27f13e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:28.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.627+0000 7fd27f13e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:28.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.627+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd2781244d0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd26d7fa640 1 -- 10.20.193.78:0/79214306 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd26000c6c0 con 0x7fd2781cfaa0 2026-02-17T06:03:28.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd26d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:28.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd26d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:28.630 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":206,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:03:28.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd24803de60 msgr2=0x7fd248040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:28.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd24803de60 0x7fd248040320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd268034000 tx=0x7fd268005000 comp rx=0 tx=0).stop 2026-02-17T06:03:28.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 10 monclient: shutdown 2026-02-17T06:03:28.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:28.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd2781cfaa0 msgr2=0x7fd2781223e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:28.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd2781cfaa0 0x7fd2781223e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e780 tx=0x7fd26000eb80 comp rx=0 tx=0).stop 2026-02-17T06:03:28.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 shutdown_connections 2026-02-17T06:03:28.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd2781cff20 0x7fd278122920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd2781cfaa0 0x7fd2781223e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd24803de60 0x7fd248040320 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 --2- 10.20.193.78:0/79214306 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd264003b30 0x7fd2781cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:28.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.628+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 >> 10.20.193.78:0/79214306 conn(0x7fd2780a6620 msgr2=0x7fd2781377f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:28.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.629+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 shutdown_connections 2026-02-17T06:03:28.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:28.629+0000 7fd27f13e640 1 -- 10.20.193.78:0/79214306 wait complete. 2026-02-17T06:03:31.637 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:03:31.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.706+0000 7fdbbaf49640 10 monclient: get_monmap_and_config 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.706+0000 7fdbbaf49640 10 monclient: build_initial_monmap 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.706+0000 7fdbbaf49640 10 monclient: monmap: 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:31.706692+0000 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:31.706692+0000 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:31.707 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:31.708 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:31.708 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:31.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.706+0000 7fdbbaf49640 1 Processor -- start 2026-02-17T06:03:31.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.706+0000 7fdbbaf49640 1 -- start start 2026-02-17T06:03:31.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.706+0000 7fdbbaf49640 10 monclient: init 2026-02-17T06:03:31.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:31.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2026-02-17T06:03:31.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdbb406c890 0x7fdbb40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient(hunting): picked mon.noname-a con 0x7fdbb406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fdbb406cc70 0x7fdbb406d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient(hunting): picked mon.noname-c con 0x7fdbb406cc70 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdbb4134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb4074150 con 0x7fdbb4134df0 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb409ef70 con 0x7fdbb406c890 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb409f250 con 0x7fdbb406cc70 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:31.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbbaf49640 10 monclient(hunting): authenticate will time out at 1657.260259s 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdbb406c890 0x7fdbb40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdbb406c890 0x7fdbb40a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57092/0 (socket says 10.20.193.78:57092) 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 1 -- 10.20.193.78:0/268125033 learned_addr learned my addr 10.20.193.78:0/268125033 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient(hunting): get_auth_request con 0x7fdbb406c890 auth_method 0 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbabfff640 1 --2- 10.20.193.78:0/268125033 >> v2:10.20.193.115:3300/0 conn(0x7fdbb406cc70 0x7fdbb406d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:31.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbabfff640 10 monclient(hunting): get_auth_request con 0x7fdbb406cc70 auth_method 0 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbabfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbabfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient(hunting): handle_auth_done global_id 184434 payload 274 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 1 -- 10.20.193.78:0/268125033 >> v2:10.20.193.115:3300/0 conn(0x7fdbb406cc70 msgr2=0x7fdbb406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 1 --2- 10.20.193.78:0/268125033 >> v2:10.20.193.115:3300/0 conn(0x7fdbb406cc70 0x7fdbb406d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 1 -- 10.20.193.78:0/268125033 >> v1:10.20.193.115:6789/0 conn(0x7fdbb4134df0 legacy=0x7fdbb4135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 1 monclient: found mon.noname-a 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:31.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.707+0000 7fdbb8cbe640 1 -- 10.20.193.78:0/268125033 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb409df40 con 0x7fdbb406c890 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbb8cbe640 1 -- 10.20.193.78:0/268125033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406c890 msgr2=0x7fdbb40a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbb8cbe640 1 --2- 10.20.193.78:0/268125033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406c890 0x7fdbb40a01f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4074e00 tx=0x7fdba402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce032fe72ea5bb5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 1 -- 10.20.193.78:0/268125033 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba403c070 con 0x7fdbb406c890 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: dump: 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:31.721 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: _finish_auth 0 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 1 -- 10.20.193.78:0/268125033 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba402fd50 con 0x7fdbb406c890 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 5 monclient: authenticate success, global_id 184434 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 1 -- 10.20.193.78:0/268125033 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4037660 con 0x7fdbb406c890 2026-02-17T06:03:31.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 4 set_mon_vals no callback set 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbab7fe640 10 monclient: dump: 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:31.723 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 10 monclient: shutdown 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 1 -- 10.20.193.78:0/268125033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406c890 msgr2=0x7fdbb40a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/268125033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406c890 0x7fdbb40a01f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4074e00 tx=0x7fdba402eed0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 1 -- 10.20.193.78:0/268125033 shutdown_connections 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/268125033 >> v2:10.20.193.115:3300/0 conn(0x7fdbb406cc70 0x7fdbb406d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/268125033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406c890 0x7fdbb40a01f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 1 -- 10.20.193.78:0/268125033 >> 10.20.193.78:0/268125033 conn(0x7fdbb40a6620 msgr2=0x7fdbb40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 1 -- 10.20.193.78:0/268125033 shutdown_connections 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.710+0000 7fdbbaf49640 1 -- 10.20.193.78:0/268125033 wait complete. 2026-02-17T06:03:31.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient: build_initial_monmap 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient: monmap: 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:31.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 1 Processor -- start 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 1 -- start start 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient: init 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdbb406c890 0x7fdbb406eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient(hunting): picked mon.noname-b con 0x7fdbb406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406cc70 0x7fdbb406f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient(hunting): picked mon.noname-a con 0x7fdbb406cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdbb4134df0 0x7fdbb406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient(hunting): picked mon.noname-c con 0x7fdbb4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:31.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb409d360 con 0x7fdbb406cc70 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb409d1e0 con 0x7fdbb406c890 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb4135910 con 0x7fdbb4134df0 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbbaf49640 10 monclient(hunting): authenticate will time out at 1657.264501s 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbb94bf640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdbb4134df0 msgr2=0x7fdbb406f560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406cc70 0x7fdbb406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbb94bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdbb4134df0 0x7fdbb406f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406cc70 0x7fdbb406f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57106/0 (socket says 10.20.193.78:57106) 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 -- 10.20.193.78:0/794664311 learned_addr learned my addr 10.20.193.78:0/794664311 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient(hunting): get_auth_request con 0x7fdbb406cc70 auth_method 0 2026-02-17T06:03:31.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbb8cbe640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdbb406c890 0x7fdbb406eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbb8cbe640 10 monclient(hunting): get_auth_request con 0x7fdbb406c890 auth_method 0 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbb8cbe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbb8cbe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbb8cbe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient(hunting): handle_auth_done global_id 184437 payload 824 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 -- 10.20.193.78:0/794664311 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdbb4134df0 msgr2=0x7fdbb406f560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:03:31.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdbb4134df0 0x7fdbb406f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 -- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdbb406c890 msgr2=0x7fdbb406eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdbb406c890 0x7fdbb406eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient: _finish_hunting 0 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 monclient: found mon.noname-a 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.711+0000 7fdbabfff640 1 -- 10.20.193.78:0/794664311 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb4128900 con 0x7fdbb406cc70 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdbabfff640 1 -- 10.20.193.78:0/794664311 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406cc70 msgr2=0x7fdbb406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdbabfff640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406cc70 0x7fdbb406f020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00e7c0 tx=0x7fdb9c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 1 -- 10.20.193.78:0/794664311 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c017070 con 0x7fdbb406cc70 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:31.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 10 monclient: dump: 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:31.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 10 monclient: _finish_auth 0 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 10 monclient: _check_auth_tickets 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 1 -- 10.20.193.78:0/794664311 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c00cae0 con 0x7fdbb406cc70 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.713+0000 7fdba97fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdba97fa640 1 -- 10.20.193.78:0/794664311 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c00f040 con 0x7fdbb406cc70 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdba97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdba97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbaaffd640 4 set_mon_vals no callback set 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdba97fa640 10 monclient: dump: 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:31.731 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 5 monclient: authenticate success, global_id 184437 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 10 monclient: _renew_subs 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb409d810 con 0x7fdbb406cc70 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 10 monclient: _renew_subs 2026-02-17T06:03:31.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb4128620 con 0x7fdbb406cc70 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb4074980 con 0x7fdbb406cc70 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdba97fa640 1 -- 10.20.193.78:0/794664311 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fdb9c012050 con 0x7fdbb406cc70 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.714+0000 7fdba97fa640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb8403deb0 0x7fdb84040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.715+0000 7fdba97fa640 1 -- 10.20.193.78:0/794664311 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(618..618 src has 335..618) ==== 4795+0+0 (secure 0 0 0) 0x7fdb9c0525a0 con 0x7fdbb406cc70 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.715+0000 7fdbb8cbe640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb8403deb0 0x7fdb84040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.715+0000 7fdbb8cbe640 10 monclient: get_auth_request con 0x7fdb8403deb0 auth_method 0 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.717+0000 7fdba97fa640 1 -- 10.20.193.78:0/794664311 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdb9c023d80 con 0x7fdbb406cc70 2026-02-17T06:03:31.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.717+0000 7fdba97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:31.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.717+0000 7fdba97fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:31.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.717+0000 7fdbb8cbe640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb8403deb0 0x7fdb84040370 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdba4036000 tx=0x7fdba4039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:31.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.832+0000 7fdbbaf49640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:03:31.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.832+0000 7fdbbaf49640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:31.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.832+0000 7fdbbaf49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:31.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.832+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdbb4134840 con 0x7fdbb406cc70 2026-02-17T06:03:31.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.832+0000 7fdba97fa640 1 -- 10.20.193.78:0/794664311 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fdb9c013c10 con 0x7fdbb406cc70 2026-02-17T06:03:31.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.832+0000 7fdba97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:31.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.832+0000 7fdba97fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:31.833 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":206,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:03:31.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb8403deb0 msgr2=0x7fdb84040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:31.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb8403deb0 0x7fdb84040370 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdba4036000 tx=0x7fdba4039040 comp rx=0 tx=0).stop 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 10 monclient: shutdown 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406cc70 msgr2=0x7fdbb406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406cc70 0x7fdbb406f020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00e7c0 tx=0x7fdb9c00ec90 comp rx=0 tx=0).stop 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 shutdown_connections 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdbb4134df0 0x7fdbb406f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdbb406cc70 0x7fdbb406f020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb8403deb0 0x7fdb84040370 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 --2- 10.20.193.78:0/794664311 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdbb406c890 0x7fdbb406eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:31.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 >> 10.20.193.78:0/794664311 conn(0x7fdbb40a6620 msgr2=0x7fdbb40735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:31.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 shutdown_connections 2026-02-17T06:03:31.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:31.833+0000 7fdbbaf49640 1 -- 10.20.193.78:0/794664311 wait complete. 2026-02-17T06:03:34.842 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:03:34.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient: get_monmap_and_config 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient: build_initial_monmap 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient: monmap: 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:34.910411+0000 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:34.910411+0000 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:34.912 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 1 Processor -- start 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 1 -- start start 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient: init 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f505806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:34.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f505809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 0x7f505813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 10 monclient(hunting): picked mon.noname-b con 0x7f505806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.910+0000 7f505f1f5640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f505806daf0 con 0x7f505809c4a0 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f505f1f5640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f505809cc80 con 0x7f505806c890 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f505f1f5640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f505f1f5640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f50580a0880 con 0x7f505806d380 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f505f1f5640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f505f1f5640 10 monclient(hunting): authenticate will time out at 1660.464020s 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f505d76b640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 msgr2=0x7f505813ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f505d76b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 0x7f505813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:34.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f504ffff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f505809c4a0 0x7f505813b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:42656/0 (socket says 10.20.193.78:42656) 2026-02-17T06:03:34.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.911+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 learned_addr learned my addr 10.20.193.78:0/4102631571 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:03:34.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.913+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v1:10.20.193.78:6789/0 conn(0x7f505809c4a0 legacy=0x7f505813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:34.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:34.913+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v1:10.20.193.115:6789/0 conn(0x7f505806c890 legacy=0x7f505806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:35.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:35.111+0000 7f505d76b640 1 -- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 msgr2=0x7f505813ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:35.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:35.111+0000 7f505d76b640 1 --2- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 0x7f505813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:03:35.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:35.511+0000 7f505d76b640 1 -- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 msgr2=0x7f505813ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:35.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:35.511+0000 7f505d76b640 1 --2- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 0x7f505813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:03:36.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:36.312+0000 7f505d76b640 1 -- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 msgr2=0x7f505813ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:36.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:36.312+0000 7f505d76b640 1 --2- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 0x7f505813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:03:37.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient(hunting): tick 2026-02-17T06:03:37.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:03:37.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 monclient(hunting): continuing hunt 2026-02-17T06:03:37.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:03:37.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 msgr2=0x7f505813ead0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 --2- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f505806d380 0x7f505813ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 >> v1:10.20.193.115:6789/0 conn(0x7f505806c890 legacy=0x7f505806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 >> v1:10.20.193.78:6789/0 conn(0x7f505809c4a0 legacy=0x7f505813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient: _add_conns ranks=[4,3] 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient(hunting): picked mon.noname-c con 0x7f503c003910 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient(hunting): picked mon.noname-a con 0x7f503c006440 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f503c008f90 con 0x7f503c006440 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f503c009140 con 0x7f503c003910 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504e7fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 10 monclient(hunting): get_auth_request con 0x7f503c006440 auth_method 0 2026-02-17T06:03:37.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 10 monclient(hunting): get_auth_request con 0x7f503c003910 auth_method 0 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:37.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:37.911+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:38.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:38.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 10 monclient(hunting): get_auth_request con 0x7f503c006440 auth_method 0 2026-02-17T06:03:38.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:38.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:38.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:38.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:38.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.111+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 10 monclient(hunting): get_auth_request con 0x7f503c003910 auth_method 0 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:38.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:38.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.112+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 10 monclient(hunting): get_auth_request con 0x7f503c006440 auth_method 0 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:38.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f505cf6a640 10 monclient(hunting): get_auth_request con 0x7f503c003910 auth_method 0 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.512+0000 7f505cf6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.513+0000 7f505cf6a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.513+0000 7f505cf6a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.513+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.513+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.513+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:38.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:38.513+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 10 monclient(hunting): get_auth_request con 0x7f503c006440 auth_method 0 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:39.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.313+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 10 monclient(hunting): get_auth_request con 0x7f503c003910 auth_method 0 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:03:39.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:39.314+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:03:39.806 INFO:tasks.rados.rados.0.trial115.stdout:2891: done (0 left) 2026-02-17T06:03:39.807 INFO:tasks.rados.rados.0.trial115.stdout:2892: rollback oid 30 current snap is 296 2026-02-17T06:03:39.807 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 295 2026-02-17T06:03:39.807 INFO:tasks.rados.rados.0.trial115.stdout:2893: read oid 49 snap -1 2026-02-17T06:03:39.807 INFO:tasks.rados.rados.0.trial115.stdout:2893: expect deleted 2026-02-17T06:03:39.807 INFO:tasks.rados.rados.0.trial115.stdout:2894: copy_from oid 25 from oid 27 current snap is 296 2026-02-17T06:03:39.807 INFO:tasks.rados.rados.0.trial115.stdout:2895: delete oid 43 current snap is 296 2026-02-17T06:03:39.810 INFO:tasks.rados.rados.0.trial115.stdout:2893: done (3 left) 2026-02-17T06:03:39.810 INFO:tasks.rados.rados.0.trial115.stdout:2895: done (2 left) 2026-02-17T06:03:39.811 INFO:tasks.rados.rados.0.trial115.stdout:2896: copy_from oid 20 from oid 18 current snap is 296 2026-02-17T06:03:39.811 INFO:tasks.rados.rados.0.trial115.stdout:2897: copy_from oid 41 from oid 19 current snap is 296 2026-02-17T06:03:39.811 INFO:tasks.rados.rados.0.trial115.stdout:2898: rollback oid 26 current snap is 296 2026-02-17T06:03:39.811 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 26 to 290 2026-02-17T06:03:39.811 INFO:tasks.rados.rados.0.trial115.stdout:2899: copy_from oid 2 from oid 22 current snap is 296 2026-02-17T06:03:39.811 INFO:tasks.rados.rados.0.trial115.stdout:2900: write oid 12 current snap is 296 2026-02-17T06:03:39.811 INFO:tasks.rados.rados.0.trial115.stdout:2900: seq_num 1017 ranges {692719=403182,1579246=720684,2828713=264006} 2026-02-17T06:03:39.814 INFO:tasks.rados.rados.0.trial115.stdout:2900: writing trial11545420-12 from 692719 to 1095901 tid 1 2026-02-17T06:03:39.819 INFO:tasks.rados.rados.0.trial115.stdout:2900: writing trial11545420-12 from 1579246 to 2299930 tid 2 2026-02-17T06:03:39.822 INFO:tasks.rados.rados.0.trial115.stdout:2900: writing trial11545420-12 from 2828713 to 3092719 tid 3 2026-02-17T06:03:39.822 INFO:tasks.rados.rados.0.trial115.stdout:2901: rollback oid 7 current snap is 296 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 287 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2897: finishing copy_from to trial11545420-41 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2897: got expected ENOENT (src dne) 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2897: finishing copy_from racing read to trial11545420-41 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2898: finishing rollback tid 0 to trial11545420-26 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2898: finishing rollback tid 1 to trial11545420-26 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 972 (ObjNum 129 snap 0 seq_num 1334264536) dirty dne 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2892: finishing rollback tid 0 to trial11545420-30 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2892: finishing rollback tid 1 to trial11545420-30 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2892: finishing rollback tid 2 to trial11545420-30 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1329 (ObjNum 1012 snap 294 seq_num 1012) dirty exists 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2896: finishing copy_from racing read to trial11545420-20 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2896: finishing copy_from to trial11545420-20 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1411 (ObjNum 991 snap 284 seq_num 991) dirty exists 2026-02-17T06:03:39.823 INFO:tasks.rados.rados.0.trial115.stdout:2894: finishing copy_from racing read to trial11545420-25 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2900: finishing write tid 1 to trial11545420-12 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2894: finishing copy_from to trial11545420-25 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1183 (ObjNum 973 snap 280 seq_num 973) dirty exists 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2900: finishing write tid 2 to trial11545420-12 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2892: done (7 left) 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2894: done (6 left) 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2896: done (5 left) 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2897: done (4 left) 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2898: done (3 left) 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2902: rollback oid 25 current snap is 296 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 287 2026-02-17T06:03:39.824 INFO:tasks.rados.rados.0.trial115.stdout:2903: snap_create 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2900: finishing write tid 3 to trial11545420-12 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2900: finishing write tid 4 to trial11545420-12 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2900: finishing write tid 5 to trial11545420-12 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2900: oid 12 updating version 0 to 1504 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2900: oid 12 updating version 1504 to 1505 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2900: oid 12 updating version 1505 to 1507 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2900: oid 12 version 1507 is already newer than 1506 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1507 (ObjNum 1017 snap 296 seq_num 1017) dirty exists 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2900: left oid 12 (ObjNum 1017 snap 296 seq_num 1017) 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2899: finishing copy_from racing read to trial11545420-2 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2902: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2902: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:03:39.825 INFO:tasks.rados.rados.0.trial115.stdout:2902: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:03:39.826 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1418 (ObjNum 972 snap 280 seq_num 972) dirty exists 2026-02-17T06:03:39.826 INFO:tasks.rados.rados.0.trial115.stdout:2901: finishing rollback tid 0 to trial11545420-7 2026-02-17T06:03:39.826 INFO:tasks.rados.rados.0.trial115.stdout:2901: finishing rollback tid 1 to trial11545420-7 2026-02-17T06:03:39.826 INFO:tasks.rados.rados.0.trial115.stdout:2899: finishing copy_from to trial11545420-2 2026-02-17T06:03:39.826 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1326 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:03:39.826 INFO:tasks.rados.rados.0.trial115.stdout:2901: finishing rollback tid 2 to trial11545420-7 2026-02-17T06:03:39.826 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1438 (ObjNum 888 snap 254 seq_num 888) dirty exists 2026-02-17T06:03:40.808 INFO:tasks.rados.rados.0.trial115.stdout:2899: done (4 left) 2026-02-17T06:03:40.808 INFO:tasks.rados.rados.0.trial115.stdout:2900: done (3 left) 2026-02-17T06:03:40.808 INFO:tasks.rados.rados.0.trial115.stdout:2901: done (2 left) 2026-02-17T06:03:40.808 INFO:tasks.rados.rados.0.trial115.stdout:2902: done (1 left) 2026-02-17T06:03:40.808 INFO:tasks.rados.rados.0.trial115.stdout:2903: done (0 left) 2026-02-17T06:03:40.808 INFO:tasks.rados.rados.0.trial115.stdout:2904: copy_from oid 38 from oid 13 current snap is 297 2026-02-17T06:03:40.808 INFO:tasks.rados.rados.0.trial115.stdout:2905: write (excl) oid 10 current snap is 297 2026-02-17T06:03:40.808 INFO:tasks.rados.rados.0.trial115.stdout:2905: seq_num 1018 ranges {709526=478431,1637728=464342,2856136=253390} 2026-02-17T06:03:40.812 INFO:tasks.rados.rados.0.trial115.stdout:2905: writing trial11545420-10 from 709526 to 1187957 tid 1 2026-02-17T06:03:40.816 INFO:tasks.rados.rados.0.trial115.stdout:2905: writing trial11545420-10 from 1637728 to 2102070 tid 2 2026-02-17T06:03:40.820 INFO:tasks.rados.rados.0.trial115.stdout:2905: writing trial11545420-10 from 2856136 to 3109526 tid 3 2026-02-17T06:03:40.820 INFO:tasks.rados.rados.0.trial115.stdout:2906: read oid 25 snap -1 2026-02-17T06:03:40.820 INFO:tasks.rados.rados.0.trial115.stdout:2906: expect (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:03:40.820 INFO:tasks.rados.rados.0.trial115.stdout:2907: write (excl) oid 7 current snap is 297 2026-02-17T06:03:40.820 INFO:tasks.rados.rados.0.trial115.stdout:2907: seq_num 1019 ranges {726333=670033,2096210=724353} 2026-02-17T06:03:40.825 INFO:tasks.rados.rados.0.trial115.stdout:2907: writing trial11545420-7 from 726333 to 1396366 tid 1 2026-02-17T06:03:40.830 INFO:tasks.rados.rados.0.trial115.stdout:2907: writing trial11545420-7 from 2096210 to 2820563 tid 2 2026-02-17T06:03:40.830 INFO:tasks.rados.rados.0.trial115.stdout:2908: write (excl) oid 26 current snap is 297 2026-02-17T06:03:40.830 INFO:tasks.rados.rados.0.trial115.stdout:2908: seq_num 1020 ranges {743140=745282,2154692=468011} 2026-02-17T06:03:40.836 INFO:tasks.rados.rados.0.trial115.stdout:2908: writing trial11545420-26 from 743140 to 1488422 tid 1 2026-02-17T06:03:40.840 INFO:tasks.rados.rados.0.trial115.stdout:2908: writing trial11545420-26 from 2154692 to 2622703 tid 2 2026-02-17T06:03:40.840 INFO:tasks.rados.rados.0.trial115.stdout:2909: read oid 32 snap 296 2026-02-17T06:03:40.840 INFO:tasks.rados.rados.0.trial115.stdout:2909: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:03:40.840 INFO:tasks.rados.rados.0.trial115.stdout:2910: write (excl) oid 5 current snap is 297 2026-02-17T06:03:40.840 INFO:tasks.rados.rados.0.trial115.stdout:2904: finishing copy_from to trial11545420-38 2026-02-17T06:03:40.840 INFO:tasks.rados.rados.0.trial115.stdout:2904: got expected ENOENT (src dne) 2026-02-17T06:03:40.840 INFO:tasks.rados.rados.0.trial115.stdout:2910: seq_num 1021 ranges {759947=420531,1813174=728022} 2026-02-17T06:03:40.844 INFO:tasks.rados.rados.0.trial115.stdout:2910: writing trial11545420-5 from 759947 to 1180478 tid 1 2026-02-17T06:03:40.849 INFO:tasks.rados.rados.0.trial115.stdout:2910: writing trial11545420-5 from 1813174 to 2541196 tid 2 2026-02-17T06:03:40.849 INFO:tasks.rados.rados.0.trial115.stdout:2911: write (excl) oid 34 current snap is 297 2026-02-17T06:03:40.849 INFO:tasks.rados.rados.0.trial115.stdout:2911: seq_num 1022 ranges {776754=495780,1871656=471680,2965828=210926} 2026-02-17T06:03:40.854 INFO:tasks.rados.rados.0.trial115.stdout:2911: writing trial11545420-34 from 776754 to 1272534 tid 1 2026-02-17T06:03:40.858 INFO:tasks.rados.rados.0.trial115.stdout:2911: writing trial11545420-34 from 1871656 to 2343336 tid 2 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2911: writing trial11545420-34 from 2965828 to 3176754 tid 3 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2912: rollback oid 40 current snap is 297 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 290 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2904: finishing copy_from racing read to trial11545420-38 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2905: finishing write tid 1 to trial11545420-10 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2905: finishing write tid 2 to trial11545420-10 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2905: finishing write tid 3 to trial11545420-10 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2905: finishing write tid 4 to trial11545420-10 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2905: finishing write tid 5 to trial11545420-10 2026-02-17T06:03:40.863 INFO:tasks.rados.rados.0.trial115.stdout:2905: oid 10 updating version 0 to 1447 2026-02-17T06:03:40.864 INFO:tasks.rados.rados.0.trial115.stdout:2905: oid 10 updating version 1447 to 1448 2026-02-17T06:03:40.864 INFO:tasks.rados.rados.0.trial115.stdout:2905: oid 10 version 1448 is already newer than 1445 2026-02-17T06:03:40.864 INFO:tasks.rados.rados.0.trial115.stdout:2905: oid 10 version 1448 is already newer than 1446 2026-02-17T06:03:40.864 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1448 (ObjNum 1018 snap 297 seq_num 1018) dirty exists 2026-02-17T06:03:40.864 INFO:tasks.rados.rados.0.trial115.stdout:2905: left oid 10 (ObjNum 1018 snap 297 seq_num 1018) 2026-02-17T06:03:40.864 INFO:tasks.rados.rados.0.trial115.stdout:2906: expect (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:03:40.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient(hunting): tick 2026-02-17T06:03:40.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:03:40.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 monclient(hunting): continuing hunt 2026-02-17T06:03:40.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:03:40.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 msgr2=0x7f503c005d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:03:40.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.115:3300/0 conn(0x7f503c003910 0x7f503c005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 msgr2=0x7f503c008900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c006440 0x7f503c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f505d76b640 1 -- 10.20.193.78:0/4102631571 reap_dead start 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient: _add_conns ranks=[1,3,0,4,2] 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f503c003d60 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c008010 0x7f503c008430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient(hunting): picked mon.noname-a con 0x7f503c008010 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f503c008970 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f505809cf60 con 0x7f503c008970 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f503c00fe00 con 0x7f503c003d60 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 1 -- 10.20.193.78:0/4102631571 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f503c008f90 con 0x7f503c008010 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504e7fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> v2:10.20.193.78:3300/0 conn(0x7f503c008010 0x7f503c008430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient(hunting): get_auth_request con 0x7f503c008010 auth_method 0 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient(hunting): handle_auth_done global_id 184440 payload 274 2026-02-17T06:03:40.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v1:10.20.193.115:6789/0 conn(0x7f503c003d60 legacy=0x7f503c0068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:40.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> v1:10.20.193.78:6789/0 conn(0x7f503c008970 legacy=0x7f503c00d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:40.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:03:40.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 1 monclient: found mon.noname-a 2026-02-17T06:03:40.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:40.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:40.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.911+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c0110f0 con 0x7f503c008010 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504ffff640 1 -- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c008010 msgr2=0x7f503c008430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504ffff640 1 --2- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c008010 0x7f503c008430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5040007d00 tx=0x7f504002fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4fb24617471ec4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 1 -- 10.20.193.78:0/4102631571 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040036070 con 0x7f503c008010 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: dump: 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 1 -- 10.20.193.78:0/4102631571 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5040030850 con 0x7f503c008010 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:40.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 1 -- 10.20.193.78:0/4102631571 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040030c40 con 0x7f503c008010 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f504f7fe640 10 monclient: dump: 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 5 monclient: authenticate success, global_id 184440 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 4 set_mon_vals no callback set 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 10 monclient: shutdown 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 1 -- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c008010 msgr2=0x7f503c008430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 1 --2- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c008010 0x7f503c008430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5040007d00 tx=0x7f504002fd50 comp rx=0 tx=0).stop 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 1 -- 10.20.193.78:0/4102631571 shutdown_connections 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 1 --2- 10.20.193.78:0/4102631571 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c008010 0x7f503c008430 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 1 -- 10.20.193.78:0/4102631571 >> 10.20.193.78:0/4102631571 conn(0x7f50580a6620 msgr2=0x7f50580a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 1 -- 10.20.193.78:0/4102631571 shutdown_connections 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.914+0000 7f505f1f5640 1 -- 10.20.193.78:0/4102631571 wait complete. 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient: build_initial_monmap 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient: monmap: 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:40.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:40.917 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:40.917 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:40.917 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:40.917 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:40.917 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:40.917 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:40.917 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 1 Processor -- start 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 1 -- start start 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient: init 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c003d60 0x7f50581cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient(hunting): picked mon.noname-a con 0x7f503c003d60 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f503c008010 0x7f5058123810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient(hunting): picked mon.noname-c con 0x7f503c008010 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f503c008970 0x7f5058123d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient(hunting): picked mon.noname-b con 0x7f503c008970 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f505809bfa0 con 0x7f503c003d60 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f505809be20 con 0x7f503c008970 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f505806cc70 con 0x7f503c008010 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505f1f5640 10 monclient(hunting): authenticate will time out at 1666.468947s 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f504ffff640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f503c008010 msgr2=0x7f5058123810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505cf6a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c003d60 0x7f50581cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f504ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f503c008010 0x7f5058123810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.915+0000 7f505cf6a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c003d60 0x7f50581cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:37276/0 (socket says 10.20.193.78:37276) 2026-02-17T06:03:40.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 1 -- 10.20.193.78:0/4264272186 learned_addr learned my addr 10.20.193.78:0/4264272186 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient(hunting): get_auth_request con 0x7f503c003d60 auth_method 0 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505d76b640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f503c008970 0x7f5058123d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient(hunting): handle_auth_done global_id 184443 payload 824 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 1 -- 10.20.193.78:0/4264272186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f503c008010 msgr2=0x7f5058123810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f503c008010 0x7f5058123810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 1 -- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f503c008970 msgr2=0x7f5058123d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f503c008970 0x7f5058123d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient: _finish_hunting 0 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 1 monclient: found mon.noname-a 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505cf6a640 1 -- 10.20.193.78:0/4264272186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5058128180 con 0x7f503c003d60 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505d76b640 10 monclient: get_auth_request con 0x7f503c008970 auth_method 0 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.916+0000 7f505d76b640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f503c008970 0x7f5058123d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505cf6a640 1 -- 10.20.193.78:0/4264272186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c003d60 msgr2=0x7f50581cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505cf6a640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c003d60 0x7f50581cd710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5048007620 tx=0x7f50480031d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 1 -- 10.20.193.78:0/4264272186 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504801a020 con 0x7f503c003d60 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: dump: 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:40.920 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: _finish_auth 0 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 1 -- 10.20.193.78:0/4264272186 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5048015890 con 0x7f503c003d60 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 1 -- 10.20.193.78:0/4264272186 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048015c80 con 0x7f503c003d60 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505f1f5640 5 monclient: authenticate success, global_id 184443 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505f1f5640 10 monclient: _renew_subs 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504effd640 4 set_mon_vals no callback set 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505f1f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50581271e0 con 0x7f503c003d60 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 10 monclient: dump: 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:40.921 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505f1f5640 10 monclient: _renew_subs 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505f1f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5058136ea0 con 0x7f503c003d60 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.918+0000 7f504d7fa640 1 -- 10.20.193.78:0/4264272186 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5048016450 con 0x7f503c003d60 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.919+0000 7f505f1f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.919+0000 7f505f1f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.919+0000 7f505f1f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.919+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5058068410 con 0x7f503c003d60 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.919+0000 7f504d7fa640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803deb0 0x7f5028040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.919+0000 7f504d7fa640 1 -- 10.20.193.78:0/4264272186 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(620..620 src has 335..620) ==== 4767+0+0 (secure 0 0 0) 0x7f5048053480 con 0x7f503c003d60 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.919+0000 7f504ffff640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803deb0 0x7f5028040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.919+0000 7f504ffff640 10 monclient: get_auth_request con 0x7f502803deb0 auth_method 0 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.921+0000 7f504d7fa640 1 -- 10.20.193.78:0/4264272186 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5058068410 con 0x7f503c003d60 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.921+0000 7f504d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:40.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.921+0000 7f504d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:40.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:40.921+0000 7f504ffff640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803deb0 0x7f5028040370 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5040038000 tx=0x7f5040010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2907: finishing write tid 1 to trial11545420-7 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2907: finishing write tid 2 to trial11545420-7 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2907: finishing write tid 3 to trial11545420-7 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2907: finishing write tid 4 to trial11545420-7 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2907: oid 7 updating version 0 to 1440 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2907: oid 7 updating version 1440 to 1441 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2907: oid 7 version 1441 is already newer than 1439 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1441 (ObjNum 1019 snap 297 seq_num 1019) dirty exists 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2907: left oid 7 (ObjNum 1019 snap 297 seq_num 1019) 2026-02-17T06:03:40.961 INFO:tasks.rados.rados.0.trial115.stdout:2908: finishing write tid 1 to trial11545420-26 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2908: finishing write tid 2 to trial11545420-26 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2908: finishing write tid 3 to trial11545420-26 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2904: done (8 left) 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2905: done (7 left) 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2906: done (6 left) 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2907: done (5 left) 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2913: copy_from oid 25 from oid 38 current snap is 297 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2908: finishing write tid 4 to trial11545420-26 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2908: oid 26 updating version 0 to 973 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2908: oid 26 updating version 973 to 974 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2908: oid 26 updating version 974 to 975 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 975 (ObjNum 1020 snap 297 seq_num 1020) dirty exists 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2908: left oid 26 (ObjNum 1020 snap 297 seq_num 1020) 2026-02-17T06:03:40.962 INFO:tasks.rados.rados.0.trial115.stdout:2909: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:03:41.036 INFO:tasks.rados.rados.0.trial115.stdout:2910: finishing write tid 1 to trial11545420-5 2026-02-17T06:03:41.036 INFO:tasks.rados.rados.0.trial115.stdout:2910: finishing write tid 2 to trial11545420-5 2026-02-17T06:03:41.036 INFO:tasks.rados.rados.0.trial115.stdout:2910: finishing write tid 3 to trial11545420-5 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2910: finishing write tid 4 to trial11545420-5 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2910: oid 5 updating version 0 to 1438 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2910: oid 5 version 1438 is already newer than 1437 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2910: oid 5 version 1438 is already newer than 1436 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1438 (ObjNum 1021 snap 297 seq_num 1021) dirty exists 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2910: left oid 5 (ObjNum 1021 snap 297 seq_num 1021) 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: finishing write tid 1 to trial11545420-34 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: finishing write tid 2 to trial11545420-34 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: finishing write tid 3 to trial11545420-34 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: finishing write tid 4 to trial11545420-34 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: finishing write tid 5 to trial11545420-34 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: oid 34 updating version 0 to 1144 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: oid 34 version 1144 is already newer than 1142 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: oid 34 version 1144 is already newer than 1143 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: oid 34 updating version 1144 to 1145 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1145 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2911: left oid 34 (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2912: finishing rollback tid 0 to trial11545420-40 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2912: finishing rollback tid 1 to trial11545420-40 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:2912: finishing rollback tid 2 to trial11545420-40 2026-02-17T06:03:41.037 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1410 (ObjNum 995 snap 290 seq_num 995) dirty exists 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2913: finishing copy_from racing read to trial11545420-25 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2913: finishing copy_from to trial11545420-25 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1529 (ObjNum 1005 snap 294 seq_num 1005) dirty exists 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2908: done (5 left) 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2909: done (4 left) 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2910: done (3 left) 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2911: done (2 left) 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2912: done (1 left) 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2913: done (0 left) 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2914: read oid 18 snap -1 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2914: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2915: rollback oid 25 current snap is 297 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 296 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2916: copy_from oid 13 from oid 7 current snap is 297 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2917: read oid 47 snap -1 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2917: expect deleted 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2918: write (excl) oid 33 current snap is 297 2026-02-17T06:03:41.038 INFO:tasks.rados.rados.0.trial115.stdout:2918: seq_num 1023 ranges {793561=571029,1813785=731691,2993251=200310} 2026-02-17T06:03:41.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.040+0000 7f505f1f5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:03:41.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.040+0000 7f505f1f5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:41.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.040+0000 7f505f1f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:41.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.040+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f505806fe00 con 0x7f503c003d60 2026-02-17T06:03:41.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.040+0000 7f504d7fa640 1 -- 10.20.193.78:0/4264272186 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f5048024020 con 0x7f503c003d60 2026-02-17T06:03:41.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.040+0000 7f504d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:03:41.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.040+0000 7f504d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:41.040 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":210,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:03:41.041 INFO:tasks.rados.rados.0.trial115.stdout:2918: writing trial11545420-33 from 793561 to 1364590 tid 1 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803deb0 msgr2=0x7f5028040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803deb0 0x7f5028040370 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5040038000 tx=0x7f5040010040 comp rx=0 tx=0).stop 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 10 monclient: shutdown 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c003d60 msgr2=0x7f50581cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c003d60 0x7f50581cd710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5048007620 tx=0x7f50480031d0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 shutdown_connections 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f502803deb0 0x7f5028040370 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f503c008970 0x7f5058123d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f503c008010 0x7f5058123810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 --2- 10.20.193.78:0/4264272186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f503c003d60 0x7f50581cd710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 >> 10.20.193.78:0/4264272186 conn(0x7f50580a6620 msgr2=0x7f5058072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 shutdown_connections 2026-02-17T06:03:41.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.042+0000 7f505f1f5640 1 -- 10.20.193.78:0/4264272186 wait complete. 2026-02-17T06:03:41.046 INFO:tasks.rados.rados.0.trial115.stdout:2918: writing trial11545420-33 from 1813785 to 2545476 tid 2 2026-02-17T06:03:41.048 INFO:tasks.rados.rados.0.trial115.stdout:2918: writing trial11545420-33 from 2993251 to 3193561 tid 3 2026-02-17T06:03:41.048 INFO:tasks.rados.rados.0.trial115.stdout:2919: copy_from oid 50 from oid 14 current snap is 297 2026-02-17T06:03:41.048 INFO:tasks.rados.rados.0.trial115.stdout:2914: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:03:41.051 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:03:41.051 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:03:41.051 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.117+0000 7f1e1d63b640 10 monclient: get_monmap_and_config 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.117+0000 7f1e1d63b640 10 monclient: build_initial_monmap 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.117+0000 7f1e1d63b640 10 monclient: monmap: 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:41.117611+0000 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:41.117611+0000 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:41.117 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.117+0000 7f1e1d63b640 1 Processor -- start 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.117+0000 7f1e1d63b640 1 -- start start 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.117+0000 7f1e1d63b640 10 monclient: init 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient: _add_conns ranks=[1,3,4,0,2] 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1e180a3140 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1e180a3c30 0x7f1e1806eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient(hunting): picked mon.noname-a con 0x7f1e180a3c30 addr v2:10.20.193.78:3300/0 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f1e1813d420 0x7f1e1813f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient(hunting): picked mon.noname-c con 0x7f1e1813d420 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:41.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e180a48a0 con 0x7f1e180a3140 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1806ffc0 con 0x7f1e180a3c30 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1806f970 con 0x7f1e1813d420 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e1d63b640 10 monclient(hunting): authenticate will time out at 1666.671219s 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1e180a3c30 0x7f1e1806eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f1e180a3c30 0x7f1e1806eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:37286/0 (socket says 10.20.193.78:37286) 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 1 -- 10.20.193.78:0/293851299 learned_addr learned my addr 10.20.193.78:0/293851299 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient(hunting): get_auth_request con 0x7f1e180a3c30 auth_method 0 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e177fe640 1 --2- 10.20.193.78:0/293851299 >> v2:10.20.193.115:3300/0 conn(0x7f1e1813d420 0x7f1e1813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e177fe640 10 monclient(hunting): get_auth_request con 0x7f1e1813d420 auth_method 0 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient(hunting): handle_auth_done global_id 184446 payload 274 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 1 -- 10.20.193.78:0/293851299 >> v2:10.20.193.115:3300/0 conn(0x7f1e1813d420 msgr2=0x7f1e1813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 1 --2- 10.20.193.78:0/293851299 >> v2:10.20.193.115:3300/0 conn(0x7f1e1813d420 0x7f1e1813f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 1 -- 10.20.193.78:0/293851299 >> v1:10.20.193.115:6789/0 conn(0x7f1e180a3140 legacy=0x7f1e180a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient: _finish_hunting 0 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 1 monclient: found mon.noname-a 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:41.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.118+0000 7f1e167fc640 1 -- 10.20.193.78:0/293851299 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1806fc50 con 0x7f1e180a3c30 2026-02-17T06:03:41.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.120+0000 7f1e167fc640 1 -- 10.20.193.78:0/293851299 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e180a3c30 msgr2=0x7f1e1806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:41.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.120+0000 7f1e167fc640 1 --2- 10.20.193.78:0/293851299 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e180a3c30 0x7f1e1806eb10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e18070d40 tx=0x7f1e0002edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3b136c300533f5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 1 -- 10.20.193.78:0/293851299 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0003c070 con 0x7f1e180a3c30 2026-02-17T06:03:41.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:41.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: dump: 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: _finish_auth 0 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 1 -- 10.20.193.78:0/293851299 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0002fc70 con 0x7f1e180a3c30 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 1 -- 10.20.193.78:0/293851299 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00037660 con 0x7f1e180a3c30 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e15ffb640 10 monclient: dump: 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 5 monclient: authenticate success, global_id 184446 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:41.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 4 set_mon_vals no callback set 2026-02-17T06:03:41.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 10 monclient: shutdown 2026-02-17T06:03:41.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:41.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 1 -- 10.20.193.78:0/293851299 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e180a3c30 msgr2=0x7f1e1806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 1 --2- 10.20.193.78:0/293851299 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e180a3c30 0x7f1e1806eb10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e18070d40 tx=0x7f1e0002edf0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 1 -- 10.20.193.78:0/293851299 shutdown_connections 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 1 --2- 10.20.193.78:0/293851299 >> v2:10.20.193.115:3300/0 conn(0x7f1e1813d420 0x7f1e1813f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 1 --2- 10.20.193.78:0/293851299 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e180a3c30 0x7f1e1806eb10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 1 -- 10.20.193.78:0/293851299 >> 10.20.193.78:0/293851299 conn(0x7f1e1809c000 msgr2=0x7f1e180a8740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 1 -- 10.20.193.78:0/293851299 shutdown_connections 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 1 -- 10.20.193.78:0/293851299 wait complete. 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 10 monclient: build_initial_monmap 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.121+0000 7f1e1d63b640 10 monclient: monmap: 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 1 Processor -- start 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 1 -- start start 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient: init 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1e180a3140 0x7f1e181cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient(hunting): picked mon.noname-c con 0x7f1e180a3140 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1e180a3c30 0x7f1e18139700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient(hunting): picked mon.noname-b con 0x7f1e180a3c30 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813d420 0x7f1e1809c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient(hunting): picked mon.noname-a con 0x7f1e1813d420 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e180a9480 con 0x7f1e1813d420 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e180a9300 con 0x7f1e180a3c30 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1813fd50 con 0x7f1e180a3140 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e1d63b640 10 monclient(hunting): authenticate will time out at 1666.675452s 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e16ffd640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1e180a3140 msgr2=0x7f1e181cef40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e16ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1e180a3140 0x7f1e181cef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813d420 0x7f1e1809c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813d420 0x7f1e1809c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:37300/0 (socket says 10.20.193.78:37300) 2026-02-17T06:03:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 -- 10.20.193.78:0/3656560628 learned_addr learned my addr 10.20.193.78:0/3656560628 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient(hunting): get_auth_request con 0x7f1e1813d420 auth_method 0 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e167fc640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1e180a3c30 0x7f1e18139700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e167fc640 10 monclient(hunting): get_auth_request con 0x7f1e180a3c30 auth_method 0 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e167fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e167fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient(hunting): handle_auth_done global_id 184449 payload 824 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1e180a3140 msgr2=0x7f1e181cef40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1e180a3140 0x7f1e181cef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1e180a3c30 msgr2=0x7f1e18139700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1e180a3c30 0x7f1e18139700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient: _finish_hunting 0 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 monclient: found mon.noname-a 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.122+0000 7f1e177fe640 1 -- 10.20.193.78:0/3656560628 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e18123dd0 con 0x7f1e1813d420 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2915: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2915: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2915: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1532 (ObjNum 972 snap 280 seq_num 972) dirty exists 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2916: finishing copy_from racing read to trial11545420-13 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2916: finishing copy_from to trial11545420-13 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1441 (ObjNum 1019 snap 297 seq_num 1019) dirty exists 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2919: finishing copy_from to trial11545420-50 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2919: got expected ENOENT (src dne) 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2919: finishing copy_from racing read to trial11545420-50 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: finishing write tid 1 to trial11545420-33 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: finishing write tid 2 to trial11545420-33 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: finishing write tid 3 to trial11545420-33 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: finishing write tid 4 to trial11545420-33 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: finishing write tid 5 to trial11545420-33 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: oid 33 updating version 0 to 1438 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: oid 33 updating version 1438 to 1439 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: oid 33 updating version 1439 to 1441 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: oid 33 version 1441 is already newer than 1440 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1441 (ObjNum 1023 snap 297 seq_num 1023) dirty exists 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: left oid 33 (ObjNum 1023 snap 297 seq_num 1023) 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2914: done (5 left) 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2915: done (4 left) 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2916: done (3 left) 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2917: done (2 left) 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2918: done (1 left) 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2919: done (0 left) 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2920: copy_from oid 45 from oid 34 current snap is 297 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2921: write (excl) oid 14 current snap is 297 2026-02-17T06:03:41.125 INFO:tasks.rados.rados.0.trial115.stdout:2921: seq_num 1024 ranges {410368=646278,1472267=475349,2504321=405600} 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1e177fe640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813d420 msgr2=0x7f1e1809c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1e177fe640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813d420 0x7f1e1809c750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e08007e40 tx=0x7f1e0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08013790 con 0x7f1e1813d420 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1df77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1df77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1df77fe640 10 monclient: dump: 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1df77fe640 10 monclient: _finish_auth 0 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1df77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1df77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e08019070 con 0x7f1e1813d420 2026-02-17T06:03:41.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.124+0000 7f1e1d63b640 5 monclient: authenticate success, global_id 184449 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1df77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08013b90 con 0x7f1e1813d420 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 10 monclient: _renew_subs 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e157fa640 4 set_mon_vals no callback set 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1809ef50 con 0x7f1e1813d420 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1df77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1df77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1df77fe640 10 monclient: dump: 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 10 monclient: _renew_subs 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e180a6840 con 0x7f1e1813d420 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813db20 0x7f1e1813df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1813fab0 con 0x7f1e1813db20 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813db20 0x7f1e1813df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 10 monclient: get_auth_request con 0x7f1e1813db20 auth_method 0 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1e0801e460 con 0x7f1e1813d420 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 10 monclient(hunting): handle_auth_done global_id 184452 payload 824 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1e16ffd640 1 -- 10.20.193.78:0/3656560628 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e1806eb10 con 0x7f1e1813db20 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1df77fe640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1de403dee0 0x7f1de40403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.125+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(620..620 src has 335..620) ==== 4767+0+0 (secure 0 0 0) 0x7f1e08052fd0 con 0x7f1e1813d420 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.126+0000 7f1e167fc640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1de403dee0 0x7f1de40403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.126+0000 7f1e167fc640 10 monclient: get_auth_request con 0x7f1de403dee0 auth_method 0 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.127+0000 7f1e16ffd640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813db20 msgr2=0x7f1e1813df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.127+0000 7f1e16ffd640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813db20 0x7f1e1813df00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c450 tx=0x7f1e0c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c017070 con 0x7f1e1813db20 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 10 monclient: dump: 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f1e0c007070 con 0x7f1e1813db20 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813db20 msgr2=0x7f1e1813df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813db20 0x7f1e1813df00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c450 tx=0x7f1e0c00c920 comp rx=0 tx=0).stop 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813db20 msgr2=0x7f1e1813df00 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:41.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1df77fe640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813db20 0x7f1e1813df00 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c450 tx=0x7f1e0c00c920 comp rx=0 tx=0).stop 2026-02-17T06:03:41.129 INFO:tasks.rados.rados.0.trial115.stdout:2921: writing trial11545420-14 from 410368 to 1056646 tid 1 2026-02-17T06:03:41.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.128+0000 7f1e167fc640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1de403dee0 0x7f1de40403a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a97a0 tx=0x7f1e00033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e1d63b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e1d63b640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813e440 0x7f1e1813e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e1d63b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e180744d0 con 0x7f1e1813e440 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813e440 0x7f1e1813e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 10 monclient: get_auth_request con 0x7f1e1813e440 auth_method 0 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 10 monclient(hunting): handle_auth_done global_id 184455 payload 824 2026-02-17T06:03:41.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.131+0000 7f1e16ffd640 1 -- 10.20.193.78:0/3656560628 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1e1809d450 con 0x7f1e1813e440 2026-02-17T06:03:41.132 INFO:tasks.rados.rados.0.trial115.stdout:2921: writing trial11545420-14 from 1472267 to 1947616 tid 2 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e16ffd640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813e440 msgr2=0x7f1e1813e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e16ffd640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813e440 0x7f1e1813e850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c450 tx=0x7f1e0c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c017030 con 0x7f1e1813e440 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 10 monclient: dump: 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7f1e0c019b60 con 0x7f1e1813e440 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:03:41.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813e440 msgr2=0x7f1e1813e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813e440 0x7f1e1813e850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c450 tx=0x7f1e0c0038b0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813e440 msgr2=0x7f1e1813e850 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1df77fe640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813e440 0x7f1e1813e850 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c450 tx=0x7f1e0c0038b0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 210, 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 185914, 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:41.135 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:03:41.136 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:41.137 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:41.138 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:41.139 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:41.140 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.141 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:03:41.142 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:03:41.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1de403dee0 msgr2=0x7f1de40403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1de403dee0 0x7f1de40403a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a97a0 tx=0x7f1e00033000 comp rx=0 tx=0).stop 2026-02-17T06:03:41.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 10 monclient: shutdown 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813d420 msgr2=0x7f1e1809c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 1 --2- 10.20.193.78:0/3656560628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1e1813d420 0x7f1e1809c750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e08007e40 tx=0x7f1e0800e4b0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e177fe640 1 -- 10.20.193.78:0/3656560628 reap_dead start 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e177fe640 1 -- 10.20.193.78:0/3656560628 reap_dead start 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 shutdown_connections 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 >> 10.20.193.78:0/3656560628 conn(0x7f1e1809c000 msgr2=0x7f1e1813c630 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 shutdown_connections 2026-02-17T06:03:41.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.134+0000 7f1e1d63b640 1 -- 10.20.193.78:0/3656560628 wait complete. 2026-02-17T06:03:41.143 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:03:41.143 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T06:03:41.143 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2921: writing trial11545420-14 from 2504321 to 2909921 tid 3 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2922: rollback oid 43 current snap is 297 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 287 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2921: finishing write tid 1 to trial11545420-14 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2920: finishing copy_from racing read to trial11545420-45 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2923: read oid 37 snap 295 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2923: expect (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2924: write (excl) oid 49 current snap is 297 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2920: finishing copy_from to trial11545420-45 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1145 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2921: finishing write tid 2 to trial11545420-14 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2924: seq_num 1025 ranges {427175=721527,1930749=735360} 2026-02-17T06:03:41.144 INFO:tasks.rados.rados.0.trial115.stdout:2924: writing trial11545420-49 from 427175 to 1148702 tid 1 2026-02-17T06:03:41.145 INFO:tasks.rados.rados.0.trial115.stdout:2924: writing trial11545420-49 from 1930749 to 2666109 tid 2 2026-02-17T06:03:41.145 INFO:tasks.rados.rados.0.trial115.stdout:2920: done (4 left) 2026-02-17T06:03:41.145 INFO:tasks.rados.rados.0.trial115.stdout:2925: write oid 40 current snap is 297 2026-02-17T06:03:41.145 INFO:tasks.rados.rados.0.trial115.stdout:2925: seq_num 1026 ranges {443982=796776,1989231=479018,2959167=284815} 2026-02-17T06:03:41.150 INFO:tasks.rados.rados.0.trial115.stdout:2925: writing trial11545420-40 from 443982 to 1240758 tid 1 2026-02-17T06:03:41.154 INFO:tasks.rados.rados.0.trial115.stdout:2925: writing trial11545420-40 from 1989231 to 2468249 tid 2 2026-02-17T06:03:41.157 INFO:tasks.rados.rados.0.trial115.stdout:2925: writing trial11545420-40 from 2959167 to 3243982 tid 3 2026-02-17T06:03:41.157 INFO:tasks.rados.rados.0.trial115.stdout:2926: read oid 36 snap 295 2026-02-17T06:03:41.157 INFO:tasks.rados.rados.0.trial115.stdout:2926: expect (ObjNum 982 snap 282 seq_num 982) 2026-02-17T06:03:41.157 INFO:tasks.rados.rados.0.trial115.stdout:2927: write (excl) oid 11 current snap is 297 2026-02-17T06:03:41.158 INFO:tasks.rados.rados.0.trial115.stdout:2927: seq_num 1027 ranges {460789=588378,1764066=739029,3219296=41493} 2026-02-17T06:03:41.161 INFO:tasks.rados.rados.0.trial115.stdout:2927: writing trial11545420-11 from 460789 to 1049167 tid 1 2026-02-17T06:03:41.167 INFO:tasks.rados.rados.0.trial115.stdout:2927: writing trial11545420-11 from 1764066 to 2503095 tid 2 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2927: writing trial11545420-11 from 3219296 to 3260789 tid 3 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2928: delete oid 3 current snap is 297 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2921: finishing write tid 3 to trial11545420-14 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2921: finishing write tid 4 to trial11545420-14 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2921: finishing write tid 5 to trial11545420-14 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2921: oid 14 updating version 0 to 1445 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2921: oid 14 version 1445 is already newer than 1442 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2921: oid 14 version 1445 is already newer than 1444 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2921: oid 14 version 1445 is already newer than 1443 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1445 (ObjNum 1024 snap 297 seq_num 1024) dirty exists 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2921: left oid 14 (ObjNum 1024 snap 297 seq_num 1024) 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2922: finishing rollback tid 1 to trial11545420-43 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2922: finishing rollback tid 2 to trial11545420-43 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1443 (ObjNum 990 snap 284 seq_num 990) dirty exists 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2924: finishing write tid 1 to trial11545420-49 2026-02-17T06:03:41.170 INFO:tasks.rados.rados.0.trial115.stdout:2923: expect (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:03:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient: get_monmap_and_config 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient: build_initial_monmap 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient: monmap: 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:03:41.212340+0000 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:03:41.212340+0000 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:03:41.212 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 1 Processor -- start 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 1 -- start start 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient: init 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f166c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f166c09c4a0 0x7f166c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient(hunting): picked mon.noname-c con 0x7f166c09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f166c06d380 addr v1:10.20.193.115:6789/0 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c06daf0 con 0x7f166c06c890 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c09cc80 con 0x7f166c06d380 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f166c0a0880 con 0x7f166c09c4a0 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f1671a4d640 10 monclient(hunting): authenticate will time out at 1666.765869s 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f166affd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f166c06c890 0x7f166c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:43722/0 (socket says 10.20.193.78:43722) 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.212+0000 7f166affd640 1 -- 10.20.193.78:0/905369758 learned_addr learned my addr 10.20.193.78:0/905369758 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 1 --2- 10.20.193.78:0/905369758 >> v2:10.20.193.115:3300/0 conn(0x7f166c09c4a0 0x7f166c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient(hunting): get_auth_request con 0x7f166c09c4a0 auth_method 0 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient(hunting): handle_auth_done global_id 214156 payload 274 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 1 -- 10.20.193.78:0/905369758 >> v1:10.20.193.115:6789/0 conn(0x7f166c06d380 legacy=0x7f166c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 1 -- 10.20.193.78:0/905369758 >> v1:10.20.193.78:6789/0 conn(0x7f166c06c890 legacy=0x7f166c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 1 monclient: found mon.noname-c 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:03:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.213+0000 7f166a7fc640 1 -- 10.20.193.78:0/905369758 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c09cf60 con 0x7f166c09c4a0 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f166a7fc640 1 -- 10.20.193.78:0/905369758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 msgr2=0x7f166c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f166a7fc640 1 --2- 10.20.193.78:0/905369758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 0x7f166c13b3a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f166c09c980 tx=0x7f165402ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f01e8e010b8a444 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 1 -- 10.20.193.78:0/905369758 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165403c070 con 0x7f166c09c4a0 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: dump: 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.216 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: _finish_auth 0 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 5 monclient: authenticate success, global_id 214156 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 1 -- 10.20.193.78:0/905369758 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165402fcb0 con 0x7f166c09c4a0 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 1 -- 10.20.193.78:0/905369758 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165402fe60 con 0x7f166c09c4a0 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1669ffb640 10 monclient: dump: 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 10 monclient: get_monmap_and_config success 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 4 set_mon_vals no callback set 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 10 monclient: shutdown 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 1 -- 10.20.193.78:0/905369758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 msgr2=0x7f166c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 1 --2- 10.20.193.78:0/905369758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 0x7f166c13b3a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f166c09c980 tx=0x7f165402ed50 comp rx=0 tx=0).stop 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 1 -- 10.20.193.78:0/905369758 shutdown_connections 2026-02-17T06:03:41.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 1 --2- 10.20.193.78:0/905369758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 0x7f166c13b3a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 1 -- 10.20.193.78:0/905369758 >> 10.20.193.78:0/905369758 conn(0x7f166c0a6620 msgr2=0x7f166c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:41.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 1 -- 10.20.193.78:0/905369758 shutdown_connections 2026-02-17T06:03:41.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.216+0000 7f1671a4d640 1 -- 10.20.193.78:0/905369758 wait complete. 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient: build_initial_monmap 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient: monmap: 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 1 Processor -- start 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 1 -- start start 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient: init 2026-02-17T06:03:41.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient: _reopen_session rank -1 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f166c06c890 0x7f166c13d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient(hunting): picked mon.noname-c con 0x7f166c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f166c06d380 0x7f166c1cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient(hunting): picked mon.noname-a con 0x7f166c06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 0x7f166c1cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient(hunting): picked mon.noname-b con 0x7f166c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f166c09bfa0 con 0x7f166c06d380 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f166c09be20 con 0x7f166c09c4a0 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f166c13f010 con 0x7f166c06c890 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient(hunting): _renew_subs 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f1671a4d640 10 monclient(hunting): authenticate will time out at 1666.770571s 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166affd640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f166c06c890 msgr2=0x7f166c13d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f166c06c890 0x7f166c13d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f166c06d380 0x7f166c1cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f166c06d380 0x7f166c1cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:37340/0 (socket says 10.20.193.78:37340) 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 -- 10.20.193.78:0/2026416994 learned_addr learned my addr 10.20.193.78:0/2026416994 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient(hunting): get_auth_request con 0x7f166c06d380 auth_method 0 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166b7fe640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 0x7f166c1cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166b7fe640 10 monclient(hunting): get_auth_request con 0x7f166c09c4a0 auth_method 0 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient(hunting): handle_auth_done global_id 184458 payload 824 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f166c06c890 msgr2=0x7f166c13d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f166c06c890 0x7f166c13d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 msgr2=0x7f166c1cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c09c4a0 0x7f166c1cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 monclient: found mon.noname-a 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:03:41.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.217+0000 7f166a7fc640 1 -- 10.20.193.78:0/2026416994 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c121990 con 0x7f166c06d380 2026-02-17T06:03:41.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f166a7fc640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f166c06d380 msgr2=0x7f166c1cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f166a7fc640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f166c06d380 0x7f166c1cf2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f16540099a0 tx=0x7f16540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654046070 con 0x7f166c06d380 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: dump: 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: _finish_auth 0 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: _check_auth_tickets 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 5 monclient: authenticate success, global_id 184458 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 10 monclient: _renew_subs 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c121c70 con 0x7f166c06d380 2026-02-17T06:03:41.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1654041890 con 0x7f166c06d380 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165403c040 con 0x7f166c06d380 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 10 monclient: dump: 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f16697fa640 4 set_mon_vals no callback set 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 10 monclient: _renew_subs 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c09f100 con 0x7f166c06d380 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c1cfd30 0x7f166c1363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1d0110 con 0x7f166c1cfd30 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1654041cc0 con 0x7f166c06d380 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.220+0000 7f166affd640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c1cfd30 0x7f166c1363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 10 monclient: get_auth_request con 0x7f166c1cfd30 auth_method 0 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f1643fff640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f163c03de60 0x7f163c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(620..620 src has 335..620) ==== 4767+0+0 (secure 0 0 0) 0x7f1654077b40 con 0x7f166c06d380 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 10 monclient(hunting): handle_auth_done global_id 214162 payload 824 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166affd640 1 -- 10.20.193.78:0/2026416994 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f166c121210 con 0x7f166c1cfd30 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166b7fe640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f163c03de60 0x7f163c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.221+0000 7f166b7fe640 10 monclient: get_auth_request con 0x7f163c03de60 auth_method 0 2026-02-17T06:03:41.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f166affd640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c1cfd30 msgr2=0x7f166c1363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f166affd640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c1cfd30 0x7f166c1363d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1660004970 tx=0x7f166000cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660017070 con 0x7f166c1cfd30 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 10 monclient: dump: 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f166b7fe640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f163c03de60 0x7f163c040320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f166c070440 tx=0x7f165c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f1660005d40 con 0x7f166c1cfd30 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c1cfd30 msgr2=0x7f166c1363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c1cfd30 0x7f166c1363d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1660004970 tx=0x7f166000cb20 comp rx=0 tx=0).stop 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c1cfd30 msgr2=0x7f166c1363d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:41.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.223+0000 7f1643fff640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c1cfd30 0x7f166c1363d0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1660004970 tx=0x7f166000cb20 comp rx=0 tx=0).stop 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f1671a4d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f1671a4d640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c138390 0x7f166c1da810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f1671a4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f166c13e370 con 0x7f166c138390 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c138390 0x7f166c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 10 monclient: get_auth_request con 0x7f166c138390 auth_method 0 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:03:41.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:03:41.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:03:41.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:03:41.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 10 monclient(hunting): handle_auth_done global_id 214165 payload 824 2026-02-17T06:03:41.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.227+0000 7f166affd640 1 -- 10.20.193.78:0/2026416994 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f166c06cc70 con 0x7f166c138390 2026-02-17T06:03:41.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f166affd640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c138390 msgr2=0x7f166c1da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:03:41.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f166affd640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c138390 0x7f166c1da810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1660014000 tx=0x7f1660004b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660017070 con 0x7f166c138390 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 10 monclient: dump: 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:03:41.230 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7f1660007490 con 0x7f166c138390 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c138390 msgr2=0x7f166c1da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c138390 0x7f166c1da810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1660014000 tx=0x7f1660004b10 comp rx=0 tx=0).stop 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c138390 msgr2=0x7f166c1da810 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1643fff640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f166c138390 0x7f166c1da810 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1660014000 tx=0x7f1660004b10 comp rx=0 tx=0).stop 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 210, 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 24077, 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.231 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:41.232 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:03:41.233 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.234 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.235 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:03:41.236 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:03:41.237 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:03:41.238 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:03:41.238 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:03:41.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f163c03de60 msgr2=0x7f163c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f163c03de60 0x7f163c040320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f166c070440 tx=0x7f165c009040 comp rx=0 tx=0).stop 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 10 monclient: shutdown 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f166c06d380 msgr2=0x7f166c1cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 1 --2- 10.20.193.78:0/2026416994 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f166c06d380 0x7f166c1cf2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f16540099a0 tx=0x7f16540047c0 comp rx=0 tx=0).stop 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f166b7fe640 1 -- 10.20.193.78:0/2026416994 reap_dead start 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f166b7fe640 1 -- 10.20.193.78:0/2026416994 reap_dead start 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 shutdown_connections 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 >> 10.20.193.78:0/2026416994 conn(0x7f166c0a6620 msgr2=0x7f166c0a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 shutdown_connections 2026-02-17T06:03:41.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:03:41.230+0000 7f1671a4d640 1 -- 10.20.193.78:0/2026416994 wait complete. 2026-02-17T06:03:41.239 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:03:41.239 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:03:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2924: finishing write tid 2 to trial11545420-49 2026-02-17T06:03:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2924: finishing write tid 3 to trial11545420-49 2026-02-17T06:03:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2924: finishing write tid 4 to trial11545420-49 2026-02-17T06:03:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2924: oid 49 updating version 0 to 1412 2026-02-17T06:03:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2924: oid 49 updating version 1412 to 1413 2026-02-17T06:03:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2924: oid 49 version 1413 is already newer than 1411 2026-02-17T06:03:41.275 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1413 (ObjNum 1025 snap 297 seq_num 1025) dirty exists 2026-02-17T06:03:41.275 INFO:tasks.rados.rados.0.trial115.stdout:2924: left oid 49 (ObjNum 1025 snap 297 seq_num 1025) 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: finishing write tid 1 to trial11545420-40 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: finishing write tid 2 to trial11545420-40 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: finishing write tid 3 to trial11545420-40 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: finishing write tid 4 to trial11545420-40 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: finishing write tid 5 to trial11545420-40 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: oid 40 updating version 0 to 1415 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: oid 40 version 1415 is already newer than 1414 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: oid 40 updating version 1415 to 1417 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: oid 40 version 1417 is already newer than 1416 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1417 (ObjNum 1026 snap 297 seq_num 1026) dirty exists 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2925: left oid 40 (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:03:41.276 INFO:tasks.rados.rados.0.trial115.stdout:2926: expect (ObjNum 982 snap 282 seq_num 982) 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2927: finishing write tid 1 to trial11545420-11 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2921: done (7 left) 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2922: done (6 left) 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2923: done (5 left) 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2924: done (4 left) 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2925: done (3 left) 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2926: done (2 left) 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2928: done (1 left) 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2929: copy_from oid 35 from oid 17 current snap is 297 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2927: finishing write tid 2 to trial11545420-11 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2927: finishing write tid 3 to trial11545420-11 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2927: finishing write tid 4 to trial11545420-11 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2927: finishing write tid 5 to trial11545420-11 2026-02-17T06:03:41.350 INFO:tasks.rados.rados.0.trial115.stdout:2927: oid 11 updating version 0 to 1453 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2927: oid 11 version 1453 is already newer than 1452 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2927: oid 11 version 1453 is already newer than 1451 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2927: oid 11 version 1453 is already newer than 1450 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1453 (ObjNum 1027 snap 297 seq_num 1027) dirty exists 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2927: left oid 11 (ObjNum 1027 snap 297 seq_num 1027) 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2927: done (1 left) 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2930: read oid 25 snap 296 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2930: expect (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2931: copy_from oid 46 from oid 19 current snap is 297 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2932: snap_create 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2931: finishing copy_from to trial11545420-46 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2931: got expected ENOENT (src dne) 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2929: finishing copy_from to trial11545420-35 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2929: got expected ENOENT (src dne) 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2931: finishing copy_from racing read to trial11545420-46 2026-02-17T06:03:41.351 INFO:tasks.rados.rados.0.trial115.stdout:2929: finishing copy_from racing read to trial11545420-35 2026-02-17T06:03:41.356 INFO:tasks.rados.rados.0.trial115.stdout:2930: expect (ObjNum 972 snap 280 seq_num 972) 2026-02-17T06:03:41.812 INFO:tasks.rados.rados.0.trial115.stdout:2929: done (3 left) 2026-02-17T06:03:41.812 INFO:tasks.rados.rados.0.trial115.stdout:2930: done (2 left) 2026-02-17T06:03:41.812 INFO:tasks.rados.rados.0.trial115.stdout:2931: done (1 left) 2026-02-17T06:03:41.812 INFO:tasks.rados.rados.0.trial115.stdout:2932: done (0 left) 2026-02-17T06:03:41.812 INFO:tasks.rados.rados.0.trial115.stdout:2933: copy_from oid 48 from oid 36 current snap is 298 2026-02-17T06:03:41.812 INFO:tasks.rados.rados.0.trial115.stdout:2934: delete oid 50 current snap is 298 2026-02-17T06:03:41.814 INFO:tasks.rados.rados.0.trial115.stdout:2934: done (1 left) 2026-02-17T06:03:41.815 INFO:tasks.rados.rados.0.trial115.stdout:2935: snap_create 2026-02-17T06:03:41.817 INFO:tasks.rados.rados.0.trial115.stdout:2933: finishing copy_from racing read to trial11545420-48 2026-02-17T06:03:41.818 INFO:tasks.rados.rados.0.trial115.stdout:2933: finishing copy_from to trial11545420-48 2026-02-17T06:03:41.818 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1421 (ObjNum 982 snap 282 seq_num 982) dirty exists 2026-02-17T06:03:42.814 INFO:tasks.rados.rados.0.trial115.stdout:2933: done (1 left) 2026-02-17T06:03:42.815 INFO:tasks.rados.rados.0.trial115.stdout:2935: done (0 left) 2026-02-17T06:03:42.815 INFO:tasks.rados.rados.0.trial115.stdout:2936: rollback oid 44 current snap is 299 2026-02-17T06:03:42.815 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 44 to 296 2026-02-17T06:03:42.815 INFO:tasks.rados.rados.0.trial115.stdout:2937: snap_remove snap 290 2026-02-17T06:03:43.663 INFO:tasks.rados.rados.0.trial115.stdout:2937: done (1 left) 2026-02-17T06:03:43.664 INFO:tasks.rados.rados.0.trial115.stdout:2938: delete oid 42 current snap is 299 2026-02-17T06:03:43.664 INFO:tasks.rados.rados.0.trial115.stdout:2936: finishing rollback tid 0 to trial11545420-44 2026-02-17T06:03:43.664 INFO:tasks.rados.rados.0.trial115.stdout:2936: finishing rollback tid 1 to trial11545420-44 2026-02-17T06:03:43.664 INFO:tasks.rados.rados.0.trial115.stdout:2936: finishing rollback tid 2 to trial11545420-44 2026-02-17T06:03:43.664 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1414 (ObjNum 836 snap 237 seq_num 836) dirty exists 2026-02-17T06:03:43.667 INFO:tasks.rados.rados.0.trial115.stdout:2936: done (1 left) 2026-02-17T06:03:43.668 INFO:tasks.rados.rados.0.trial115.stdout:2938: done (0 left) 2026-02-17T06:03:43.668 INFO:tasks.rados.rados.0.trial115.stdout:2939: delete oid 36 current snap is 299 2026-02-17T06:03:43.670 INFO:tasks.rados.rados.0.trial115.stdout:2939: done (0 left) 2026-02-17T06:03:43.670 INFO:tasks.rados.rados.0.trial115.stdout:2940: rollback oid 9 current snap is 299 2026-02-17T06:03:43.670 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 9 to 297 2026-02-17T06:03:43.670 INFO:tasks.rados.rados.0.trial115.stdout:2941: write oid 5 current snap is 299 2026-02-17T06:03:43.670 INFO:tasks.rados.rados.0.trial115.stdout:2941: seq_num 1032 ranges {544824=564623,1823770=490025,3007352=337472} 2026-02-17T06:03:43.674 INFO:tasks.rados.rados.0.trial115.stdout:2941: writing trial11545420-5 from 544824 to 1109447 tid 1 2026-02-17T06:03:43.679 INFO:tasks.rados.rados.0.trial115.stdout:2941: writing trial11545420-5 from 1823770 to 2313795 tid 2 2026-02-17T06:03:43.684 INFO:tasks.rados.rados.0.trial115.stdout:2941: writing trial11545420-5 from 3007352 to 3344824 tid 3 2026-02-17T06:03:43.684 INFO:tasks.rados.rados.0.trial115.stdout:2942: delete oid 33 current snap is 299 2026-02-17T06:03:43.684 INFO:tasks.rados.rados.0.trial115.stdout:2940: finishing rollback tid 0 to trial11545420-9 2026-02-17T06:03:43.684 INFO:tasks.rados.rados.0.trial115.stdout:2940: finishing rollback tid 1 to trial11545420-9 2026-02-17T06:03:43.684 INFO:tasks.rados.rados.0.trial115.stdout:2940: finishing rollback tid 2 to trial11545420-9 2026-02-17T06:03:43.684 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1463 (ObjNum 982 snap 282 seq_num 982) dirty exists 2026-02-17T06:03:43.684 INFO:tasks.rados.rados.0.trial115.stdout:2941: finishing write tid 1 to trial11545420-5 2026-02-17T06:03:43.684 INFO:tasks.rados.rados.0.trial115.stdout:2941: finishing write tid 2 to trial11545420-5 2026-02-17T06:03:43.685 INFO:tasks.rados.rados.0.trial115.stdout:2941: finishing write tid 3 to trial11545420-5 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2941: finishing write tid 4 to trial11545420-5 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2941: finishing write tid 5 to trial11545420-5 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2941: oid 5 updating version 0 to 1443 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2941: oid 5 version 1443 is already newer than 1440 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2941: oid 5 version 1443 is already newer than 1441 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2941: oid 5 version 1443 is already newer than 1442 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1443 (ObjNum 1032 snap 299 seq_num 1032) dirty exists 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2941: left oid 5 (ObjNum 1032 snap 299 seq_num 1032) 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2940: done (2 left) 2026-02-17T06:03:43.686 INFO:tasks.rados.rados.0.trial115.stdout:2941: done (1 left) 2026-02-17T06:03:43.687 INFO:tasks.rados.rados.0.trial115.stdout:2942: done (0 left) 2026-02-17T06:03:43.687 INFO:tasks.rados.rados.0.trial115.stdout:2943: write (excl) oid 10 current snap is 299 2026-02-17T06:03:43.687 INFO:tasks.rados.rados.0.trial115.stdout:2943: seq_num 1034 ranges {578438=431474,1657087=493694,2778551=599887} 2026-02-17T06:03:43.689 INFO:tasks.rados.rados.0.trial115.stdout:2943: writing trial11545420-10 from 578438 to 1009912 tid 1 2026-02-17T06:03:43.694 INFO:tasks.rados.rados.0.trial115.stdout:2943: writing trial11545420-10 from 1657087 to 2150781 tid 2 2026-02-17T06:03:43.698 INFO:tasks.rados.rados.0.trial115.stdout:2943: writing trial11545420-10 from 2778551 to 3378438 tid 3 2026-02-17T06:03:43.698 INFO:tasks.rados.rados.0.trial115.stdout:2944: write oid 9 current snap is 299 2026-02-17T06:03:43.698 INFO:tasks.rados.rados.0.trial115.stdout:2944: seq_num 1035 ranges {595245=506723,1715569=637352,2805974=562298} 2026-02-17T06:03:43.702 INFO:tasks.rados.rados.0.trial115.stdout:2944: writing trial11545420-9 from 595245 to 1101968 tid 1 2026-02-17T06:03:43.707 INFO:tasks.rados.rados.0.trial115.stdout:2944: writing trial11545420-9 from 1715569 to 2352921 tid 2 2026-02-17T06:03:43.711 INFO:tasks.rados.rados.0.trial115.stdout:2944: writing trial11545420-9 from 2805974 to 3368272 tid 3 2026-02-17T06:03:43.711 INFO:tasks.rados.rados.0.trial115.stdout:2945: read oid 17 snap -1 2026-02-17T06:03:43.711 INFO:tasks.rados.rados.0.trial115.stdout:2945: expect deleted 2026-02-17T06:03:43.711 INFO:tasks.rados.rados.0.trial115.stdout:2946: write oid 50 current snap is 299 2026-02-17T06:03:43.711 INFO:tasks.rados.rados.0.trial115.stdout:2946: seq_num 1036 ranges {612052=581972,1657698=497363,2717044=695008} 2026-02-17T06:03:43.715 INFO:tasks.rados.rados.0.trial115.stdout:2946: writing trial11545420-50 from 612052 to 1194024 tid 1 2026-02-17T06:03:43.719 INFO:tasks.rados.rados.0.trial115.stdout:2946: writing trial11545420-50 from 1657698 to 2155061 tid 2 2026-02-17T06:03:43.723 INFO:tasks.rados.rados.0.trial115.stdout:2946: writing trial11545420-50 from 2717044 to 3412052 tid 3 2026-02-17T06:03:43.724 INFO:tasks.rados.rados.0.trial115.stdout:2947: write (excl) oid 20 current snap is 299 2026-02-17T06:03:43.724 INFO:tasks.rados.rados.0.trial115.stdout:2947: seq_num 1037 ranges {628859=657221,1716180=641021,3144467=284392} 2026-02-17T06:03:43.728 INFO:tasks.rados.rados.0.trial115.stdout:2947: writing trial11545420-20 from 628859 to 1286080 tid 1 2026-02-17T06:03:43.732 INFO:tasks.rados.rados.0.trial115.stdout:2947: writing trial11545420-20 from 1716180 to 2357201 tid 2 2026-02-17T06:03:43.736 INFO:tasks.rados.rados.0.trial115.stdout:2947: writing trial11545420-20 from 3144467 to 3428859 tid 3 2026-02-17T06:03:43.736 INFO:tasks.rados.rados.0.trial115.stdout:2948: snap_remove snap 295 2026-02-17T06:03:44.665 INFO:tasks.rados.rados.0.trial115.stdout:2948: done (5 left) 2026-02-17T06:03:44.665 INFO:tasks.rados.rados.0.trial115.stdout:2949: snap_create 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: finishing write tid 1 to trial11545420-10 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: finishing write tid 2 to trial11545420-10 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: finishing write tid 3 to trial11545420-10 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: finishing write tid 4 to trial11545420-10 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: finishing write tid 5 to trial11545420-10 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: oid 10 updating version 0 to 1466 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: oid 10 updating version 1466 to 1467 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: oid 10 version 1467 is already newer than 1465 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: oid 10 updating version 1467 to 1468 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1468 (ObjNum 1034 snap 299 seq_num 1034) dirty exists 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: left oid 10 (ObjNum 1034 snap 299 seq_num 1034) 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: finishing write tid 1 to trial11545420-9 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: finishing write tid 2 to trial11545420-9 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: finishing write tid 3 to trial11545420-9 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: finishing write tid 4 to trial11545420-9 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: finishing write tid 5 to trial11545420-9 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: oid 9 updating version 0 to 1470 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: oid 9 updating version 1470 to 1471 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: oid 9 version 1471 is already newer than 1469 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: oid 9 updating version 1471 to 1472 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1472 (ObjNum 1035 snap 299 seq_num 1035) dirty exists 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: left oid 9 (ObjNum 1035 snap 299 seq_num 1035) 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: finishing write tid 1 to trial11545420-50 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: finishing write tid 2 to trial11545420-50 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: finishing write tid 3 to trial11545420-50 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: finishing write tid 4 to trial11545420-50 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: finishing write tid 5 to trial11545420-50 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: oid 50 updating version 0 to 1334 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: oid 50 version 1334 is already newer than 1332 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: oid 50 version 1334 is already newer than 1333 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: oid 50 version 1334 is already newer than 1331 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1334 (ObjNum 1036 snap 299 seq_num 1036) dirty exists 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: left oid 50 (ObjNum 1036 snap 299 seq_num 1036) 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2947: finishing write tid 1 to trial11545420-20 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2947: finishing write tid 2 to trial11545420-20 2026-02-17T06:03:44.666 INFO:tasks.rados.rados.0.trial115.stdout:2947: finishing write tid 3 to trial11545420-20 2026-02-17T06:03:44.667 INFO:tasks.rados.rados.0.trial115.stdout:2947: finishing write tid 4 to trial11545420-20 2026-02-17T06:03:44.667 INFO:tasks.rados.rados.0.trial115.stdout:2947: finishing write tid 5 to trial11545420-20 2026-02-17T06:03:44.667 INFO:tasks.rados.rados.0.trial115.stdout:2947: oid 20 updating version 0 to 1446 2026-02-17T06:03:44.667 INFO:tasks.rados.rados.0.trial115.stdout:2947: oid 20 version 1446 is already newer than 1444 2026-02-17T06:03:44.667 INFO:tasks.rados.rados.0.trial115.stdout:2947: oid 20 updating version 1446 to 1447 2026-02-17T06:03:44.667 INFO:tasks.rados.rados.0.trial115.stdout:2947: oid 20 version 1447 is already newer than 1445 2026-02-17T06:03:44.667 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1447 (ObjNum 1037 snap 299 seq_num 1037) dirty exists 2026-02-17T06:03:44.667 INFO:tasks.rados.rados.0.trial115.stdout:2947: left oid 20 (ObjNum 1037 snap 299 seq_num 1037) 2026-02-17T06:03:45.666 INFO:tasks.rados.rados.0.trial115.stdout:2943: done (5 left) 2026-02-17T06:03:45.666 INFO:tasks.rados.rados.0.trial115.stdout:2944: done (4 left) 2026-02-17T06:03:45.666 INFO:tasks.rados.rados.0.trial115.stdout:2945: done (3 left) 2026-02-17T06:03:45.666 INFO:tasks.rados.rados.0.trial115.stdout:2946: done (2 left) 2026-02-17T06:03:45.666 INFO:tasks.rados.rados.0.trial115.stdout:2947: done (1 left) 2026-02-17T06:03:45.666 INFO:tasks.rados.rados.0.trial115.stdout:2949: done (0 left) 2026-02-17T06:03:45.666 INFO:tasks.rados.rados.0.trial115.stdout:2950: snap_create 2026-02-17T06:03:46.667 INFO:tasks.rados.rados.0.trial115.stdout:2950: done (0 left) 2026-02-17T06:03:46.667 INFO:tasks.rados.rados.0.trial115.stdout:2951: snap_remove snap 298 2026-02-17T06:03:47.668 INFO:tasks.rados.rados.0.trial115.stdout:2951: done (0 left) 2026-02-17T06:03:47.668 INFO:tasks.rados.rados.0.trial115.stdout:2952: rollback oid 37 current snap is 301 2026-02-17T06:03:47.668 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 296 2026-02-17T06:03:47.669 INFO:tasks.rados.rados.0.trial115.stdout:2953: delete oid 36 current snap is 301 2026-02-17T06:03:47.673 INFO:tasks.rados.rados.0.trial115.stdout:2952: finishing rollback tid 0 to trial11545420-37 2026-02-17T06:03:47.674 INFO:tasks.rados.rados.0.trial115.stdout:2952: finishing rollback tid 1 to trial11545420-37 2026-02-17T06:03:47.674 INFO:tasks.rados.rados.0.trial115.stdout:2952: finishing rollback tid 2 to trial11545420-37 2026-02-17T06:03:47.674 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1450 (ObjNum 923 snap 271 seq_num 923) dirty exists 2026-02-17T06:03:47.678 INFO:tasks.rados.rados.0.trial115.stdout:2952: done (1 left) 2026-02-17T06:03:47.678 INFO:tasks.rados.rados.0.trial115.stdout:2953: done (0 left) 2026-02-17T06:03:47.678 INFO:tasks.rados.rados.0.trial115.stdout:2954: snap_create 2026-02-17T06:03:48.672 INFO:tasks.rados.rados.0.trial115.stdout:2954: done (0 left) 2026-02-17T06:03:48.672 INFO:tasks.rados.rados.0.trial115.stdout:2955: copy_from oid 9 from oid 24 current snap is 302 2026-02-17T06:03:48.672 INFO:tasks.rados.rados.0.trial115.stdout:2956: rollback oid 49 current snap is 302 2026-02-17T06:03:48.672 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 49 to 300 2026-02-17T06:03:48.672 INFO:tasks.rados.rados.0.trial115.stdout:2957: read oid 39 snap -1 2026-02-17T06:03:48.672 INFO:tasks.rados.rados.0.trial115.stdout:2957: expect (ObjNum 941 snap 274 seq_num 941) 2026-02-17T06:03:48.672 INFO:tasks.rados.rados.0.trial115.stdout:2958: copy_from oid 41 from oid 43 current snap is 302 2026-02-17T06:03:48.672 INFO:tasks.rados.rados.0.trial115.stdout:2959: delete oid 44 current snap is 302 2026-02-17T06:03:48.674 INFO:tasks.rados.rados.0.trial115.stdout:2959: done (4 left) 2026-02-17T06:03:48.674 INFO:tasks.rados.rados.0.trial115.stdout:2960: read oid 8 snap 287 2026-02-17T06:03:48.674 INFO:tasks.rados.rados.0.trial115.stdout:2960: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:03:48.674 INFO:tasks.rados.rados.0.trial115.stdout:2961: read oid 18 snap -1 2026-02-17T06:03:48.674 INFO:tasks.rados.rados.0.trial115.stdout:2961: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:03:48.674 INFO:tasks.rados.rados.0.trial115.stdout:2962: rollback oid 4 current snap is 302 2026-02-17T06:03:48.675 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 297 2026-02-17T06:03:48.675 INFO:tasks.rados.rados.0.trial115.stdout:2963: snap_create 2026-02-17T06:03:48.675 INFO:tasks.rados.rados.0.trial115.stdout:2956: finishing rollback tid 0 to trial11545420-49 2026-02-17T06:03:48.677 INFO:tasks.rados.rados.0.trial115.stdout:2956: finishing rollback tid 1 to trial11545420-49 2026-02-17T06:03:48.677 INFO:tasks.rados.rados.0.trial115.stdout:2956: finishing rollback tid 2 to trial11545420-49 2026-02-17T06:03:48.677 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1420 (ObjNum 1025 snap 297 seq_num 1025) dirty exists 2026-02-17T06:03:48.677 INFO:tasks.rados.rados.0.trial115.stdout:2962: finishing rollback tid 0 to trial11545420-4 2026-02-17T06:03:48.677 INFO:tasks.rados.rados.0.trial115.stdout:2962: finishing rollback tid 1 to trial11545420-4 2026-02-17T06:03:48.677 INFO:tasks.rados.rados.0.trial115.stdout:2957: expect (ObjNum 941 snap 274 seq_num 941) 2026-02-17T06:03:48.839 INFO:tasks.rados.rados.0.trial115.stdout:2962: finishing rollback tid 2 to trial11545420-4 2026-02-17T06:03:48.839 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1423 (ObjNum 980 snap 281 seq_num 980) dirty exists 2026-02-17T06:03:48.839 INFO:tasks.rados.rados.0.trial115.stdout:2961: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:03:48.914 INFO:tasks.rados.rados.0.trial115.stdout:2960: expect (ObjNum 836 snap 237 seq_num 836) 2026-02-17T06:03:48.959 INFO:tasks.rados.rados.0.trial115.stdout:2958: finishing copy_from racing read to trial11545420-41 2026-02-17T06:03:48.960 INFO:tasks.rados.rados.0.trial115.stdout:2958: finishing copy_from to trial11545420-41 2026-02-17T06:03:48.960 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1443 (ObjNum 990 snap 284 seq_num 990) dirty exists 2026-02-17T06:03:48.960 INFO:tasks.rados.rados.0.trial115.stdout:2955: finishing copy_from racing read to trial11545420-9 2026-02-17T06:03:48.960 INFO:tasks.rados.rados.0.trial115.stdout:2955: finishing copy_from to trial11545420-9 2026-02-17T06:03:48.960 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1429 (ObjNum 923 snap 271 seq_num 923) dirty exists 2026-02-17T06:03:49.674 INFO:tasks.rados.rados.0.trial115.stdout:2955: done (7 left) 2026-02-17T06:03:49.674 INFO:tasks.rados.rados.0.trial115.stdout:2956: done (6 left) 2026-02-17T06:03:49.674 INFO:tasks.rados.rados.0.trial115.stdout:2957: done (5 left) 2026-02-17T06:03:49.674 INFO:tasks.rados.rados.0.trial115.stdout:2958: done (4 left) 2026-02-17T06:03:49.674 INFO:tasks.rados.rados.0.trial115.stdout:2960: done (3 left) 2026-02-17T06:03:49.674 INFO:tasks.rados.rados.0.trial115.stdout:2961: done (2 left) 2026-02-17T06:03:49.674 INFO:tasks.rados.rados.0.trial115.stdout:2962: done (1 left) 2026-02-17T06:03:49.674 INFO:tasks.rados.rados.0.trial115.stdout:2963: done (0 left) 2026-02-17T06:03:49.675 INFO:tasks.rados.rados.0.trial115.stdout:2964: delete oid 13 current snap is 303 2026-02-17T06:03:49.677 INFO:tasks.rados.rados.0.trial115.stdout:2964: done (0 left) 2026-02-17T06:03:49.677 INFO:tasks.rados.rados.0.trial115.stdout:2965: rollback oid 27 current snap is 303 2026-02-17T06:03:49.677 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 296 2026-02-17T06:03:49.677 INFO:tasks.rados.rados.0.trial115.stdout:2966: rollback oid 28 current snap is 303 2026-02-17T06:03:49.677 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 28 to 300 2026-02-17T06:03:49.678 INFO:tasks.rados.rados.0.trial115.stdout:2967: snap_create 2026-02-17T06:03:49.680 INFO:tasks.rados.rados.0.trial115.stdout:2966: finishing rollback tid 0 to trial11545420-28 2026-02-17T06:03:49.680 INFO:tasks.rados.rados.0.trial115.stdout:2966: finishing rollback tid 1 to trial11545420-28 2026-02-17T06:03:49.680 INFO:tasks.rados.rados.0.trial115.stdout:2966: finishing rollback tid 2 to trial11545420-28 2026-02-17T06:03:49.680 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1490 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:03:49.681 INFO:tasks.rados.rados.0.trial115.stdout:2965: finishing rollback tid 0 to trial11545420-27 2026-02-17T06:03:49.681 INFO:tasks.rados.rados.0.trial115.stdout:2965: finishing rollback tid 1 to trial11545420-27 2026-02-17T06:03:49.681 INFO:tasks.rados.rados.0.trial115.stdout:2965: finishing rollback tid 2 to trial11545420-27 2026-02-17T06:03:49.681 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1337 (ObjNum 973 snap 280 seq_num 973) dirty exists 2026-02-17T06:03:51.467 INFO:tasks.rados.rados.0.trial115.stdout:2965: done (2 left) 2026-02-17T06:03:51.468 INFO:tasks.rados.rados.0.trial115.stdout:2966: done (1 left) 2026-02-17T06:03:51.468 INFO:tasks.rados.rados.0.trial115.stdout:2967: done (0 left) 2026-02-17T06:03:51.468 INFO:tasks.rados.rados.0.trial115.stdout:2968: read oid 50 snap -1 2026-02-17T06:03:51.468 INFO:tasks.rados.rados.0.trial115.stdout:2968: expect (ObjNum 1036 snap 299 seq_num 1036) 2026-02-17T06:03:51.468 INFO:tasks.rados.rados.0.trial115.stdout:2969: snap_create 2026-02-17T06:03:51.478 INFO:tasks.rados.rados.0.trial115.stdout:2968: expect (ObjNum 1036 snap 299 seq_num 1036) 2026-02-17T06:03:52.468 INFO:tasks.rados.rados.0.trial115.stdout:2968: done (1 left) 2026-02-17T06:03:52.468 INFO:tasks.rados.rados.0.trial115.stdout:2969: done (0 left) 2026-02-17T06:03:52.468 INFO:tasks.rados.rados.0.trial115.stdout:2970: snap_remove snap 296 2026-02-17T06:03:53.469 INFO:tasks.rados.rados.0.trial115.stdout:2970: done (0 left) 2026-02-17T06:03:53.469 INFO:tasks.rados.rados.0.trial115.stdout:2971: rollback oid 41 current snap is 305 2026-02-17T06:03:53.469 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 297 2026-02-17T06:03:53.469 INFO:tasks.rados.rados.0.trial115.stdout:2972: rollback oid 25 current snap is 305 2026-02-17T06:03:53.469 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 303 2026-02-17T06:03:53.469 INFO:tasks.rados.rados.0.trial115.stdout:2973: rollback oid 43 current snap is 305 2026-02-17T06:03:53.469 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 304 2026-02-17T06:03:53.470 INFO:tasks.rados.rados.0.trial115.stdout:2974: snap_remove snap 300 2026-02-17T06:03:53.666 INFO:tasks.rados.rados.0.trial115.stdout:2974: done (3 left) 2026-02-17T06:03:53.666 INFO:tasks.rados.rados.0.trial115.stdout:2975: write oid 8 current snap is 305 2026-02-17T06:03:53.666 INFO:tasks.rados.rados.0.trial115.stdout:2975: seq_num 1041 ranges {696087=558217,1833755=648359,3137806=358281} 2026-02-17T06:03:53.670 INFO:tasks.rados.rados.0.trial115.stdout:2975: writing trial11545420-8 from 696087 to 1254304 tid 1 2026-02-17T06:03:53.675 INFO:tasks.rados.rados.0.trial115.stdout:2975: writing trial11545420-8 from 1833755 to 2482114 tid 2 2026-02-17T06:03:53.679 INFO:tasks.rados.rados.0.trial115.stdout:2975: writing trial11545420-8 from 3137806 to 3496087 tid 3 2026-02-17T06:03:53.679 INFO:tasks.rados.rados.0.trial115.stdout:2976: write oid 23 current snap is 305 2026-02-17T06:03:53.679 INFO:tasks.rados.rados.0.trial115.stdout:2976: seq_num 1042 ranges {712894=749819,2008590=792017,3281582=231312} 2026-02-17T06:03:53.684 INFO:tasks.rados.rados.0.trial115.stdout:2976: writing trial11545420-23 from 712894 to 1462713 tid 1 2026-02-17T06:03:53.689 INFO:tasks.rados.rados.0.trial115.stdout:2976: writing trial11545420-23 from 2008590 to 2800607 tid 2 2026-02-17T06:03:53.692 INFO:tasks.rados.rados.0.trial115.stdout:2976: writing trial11545420-23 from 3281582 to 3512894 tid 3 2026-02-17T06:03:53.692 INFO:tasks.rados.rados.0.trial115.stdout:2977: rollback oid 12 current snap is 305 2026-02-17T06:03:53.692 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 289 2026-02-17T06:03:53.692 INFO:tasks.rados.rados.0.trial115.stdout:2971: finishing rollback tid 0 to trial11545420-41 2026-02-17T06:03:53.692 INFO:tasks.rados.rados.0.trial115.stdout:2971: finishing rollback tid 1 to trial11545420-41 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2971: finishing rollback tid 2 to trial11545420-41 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1535 (ObjNum 959 snap 278 seq_num 959) dirty exists 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2973: finishing rollback tid 0 to trial11545420-43 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2973: finishing rollback tid 1 to trial11545420-43 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2973: finishing rollback tid 2 to trial11545420-43 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1453 (ObjNum 990 snap 284 seq_num 990) dirty exists 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2972: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2972: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2972: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1538 (ObjNum 972 snap 280 seq_num 972) dirty exists 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2975: finishing write tid 1 to trial11545420-8 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2971: done (5 left) 2026-02-17T06:03:53.693 INFO:tasks.rados.rados.0.trial115.stdout:2972: done (4 left) 2026-02-17T06:03:53.694 INFO:tasks.rados.rados.0.trial115.stdout:2973: done (3 left) 2026-02-17T06:03:53.694 INFO:tasks.rados.rados.0.trial115.stdout:2978: snap_remove snap 297 2026-02-17T06:03:55.327 INFO:tasks.rados.rados.0.trial115.stdout:2978: done (3 left) 2026-02-17T06:03:55.327 INFO:tasks.rados.rados.0.trial115.stdout:2979: delete oid 39 current snap is 305 2026-02-17T06:03:55.327 INFO:tasks.rados.rados.0.trial115.stdout:2975: finishing write tid 2 to trial11545420-8 2026-02-17T06:03:55.327 INFO:tasks.rados.rados.0.trial115.stdout:2975: finishing write tid 3 to trial11545420-8 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2975: finishing write tid 4 to trial11545420-8 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2975: finishing write tid 5 to trial11545420-8 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2975: oid 8 updating version 0 to 1143 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2975: oid 8 updating version 1143 to 1144 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2975: oid 8 updating version 1144 to 1146 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2975: oid 8 version 1146 is already newer than 1145 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1146 (ObjNum 1041 snap 305 seq_num 1041) dirty exists 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2975: left oid 8 (ObjNum 1041 snap 305 seq_num 1041) 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2976: finishing write tid 1 to trial11545420-23 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2976: finishing write tid 2 to trial11545420-23 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2976: finishing write tid 3 to trial11545420-23 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2976: finishing write tid 4 to trial11545420-23 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2976: finishing write tid 5 to trial11545420-23 2026-02-17T06:03:55.328 INFO:tasks.rados.rados.0.trial115.stdout:2976: oid 23 updating version 0 to 1402 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:2976: oid 23 version 1402 is already newer than 1401 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:2976: oid 23 updating version 1402 to 1404 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:2976: oid 23 version 1404 is already newer than 1403 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1404 (ObjNum 1042 snap 305 seq_num 1042) dirty exists 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:2976: left oid 23 (ObjNum 1042 snap 305 seq_num 1042) 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:2977: finishing rollback tid 0 to trial11545420-12 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:2977: finishing rollback tid 1 to trial11545420-12 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:2977: finishing rollback tid 2 to trial11545420-12 2026-02-17T06:03:55.329 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1524 (ObjNum 910 snap 264 seq_num 910) dirty exists 2026-02-17T06:03:55.332 INFO:tasks.rados.rados.0.trial115.stdout:2975: done (3 left) 2026-02-17T06:03:55.332 INFO:tasks.rados.rados.0.trial115.stdout:2976: done (2 left) 2026-02-17T06:03:55.332 INFO:tasks.rados.rados.0.trial115.stdout:2977: done (1 left) 2026-02-17T06:03:55.332 INFO:tasks.rados.rados.0.trial115.stdout:2979: done (0 left) 2026-02-17T06:03:55.333 INFO:tasks.rados.rados.0.trial115.stdout:2980: write oid 22 current snap is 305 2026-02-17T06:03:55.333 INFO:tasks.rados.rados.0.trial115.stdout:2980: seq_num 1044 ranges {746508=500317,2009201=795686,3220075=326433} 2026-02-17T06:03:55.337 INFO:tasks.rados.rados.0.trial115.stdout:2980: writing trial11545420-22 from 746508 to 1246825 tid 1 2026-02-17T06:03:55.342 INFO:tasks.rados.rados.0.trial115.stdout:2980: writing trial11545420-22 from 2009201 to 2804887 tid 2 2026-02-17T06:03:55.346 INFO:tasks.rados.rados.0.trial115.stdout:2980: writing trial11545420-22 from 3220075 to 3546508 tid 3 2026-02-17T06:03:55.346 INFO:tasks.rados.rados.0.trial115.stdout:2981: copy_from oid 25 from oid 33 current snap is 305 2026-02-17T06:03:55.346 INFO:tasks.rados.rados.0.trial115.stdout:2980: finishing write tid 1 to trial11545420-22 2026-02-17T06:03:55.346 INFO:tasks.rados.rados.0.trial115.stdout:2980: finishing write tid 2 to trial11545420-22 2026-02-17T06:03:55.346 INFO:tasks.rados.rados.0.trial115.stdout:2982: rollback oid 17 current snap is 305 2026-02-17T06:03:55.346 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 301 2026-02-17T06:03:55.346 INFO:tasks.rados.rados.0.trial115.stdout:2983: delete oid 8 current snap is 305 2026-02-17T06:03:55.347 INFO:tasks.rados.rados.0.trial115.stdout:2981: finishing copy_from to trial11545420-25 2026-02-17T06:03:55.347 INFO:tasks.rados.rados.0.trial115.stdout:2981: got expected ENOENT (src dne) 2026-02-17T06:03:55.347 INFO:tasks.rados.rados.0.trial115.stdout:2981: finishing copy_from racing read to trial11545420-25 2026-02-17T06:03:55.347 INFO:tasks.rados.rados.0.trial115.stdout:2981: done (3 left) 2026-02-17T06:03:55.347 INFO:tasks.rados.rados.0.trial115.stdout:2983: done (2 left) 2026-02-17T06:03:55.347 INFO:tasks.rados.rados.0.trial115.stdout:2984: snap_create 2026-02-17T06:03:55.352 INFO:tasks.rados.rados.0.trial115.stdout:2982: finishing rollback tid 1 to trial11545420-17 2026-02-17T06:03:55.352 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1399 (ObjNum 81 snap 0 seq_num 1) dirty dne 2026-02-17T06:03:55.353 INFO:tasks.rados.rados.0.trial115.stdout:2980: finishing write tid 3 to trial11545420-22 2026-02-17T06:03:55.353 INFO:tasks.rados.rados.0.trial115.stdout:2980: finishing write tid 4 to trial11545420-22 2026-02-17T06:03:55.353 INFO:tasks.rados.rados.0.trial115.stdout:2980: finishing write tid 5 to trial11545420-22 2026-02-17T06:03:55.353 INFO:tasks.rados.rados.0.trial115.stdout:2980: oid 22 updating version 0 to 1338 2026-02-17T06:03:55.354 INFO:tasks.rados.rados.0.trial115.stdout:2980: oid 22 updating version 1338 to 1339 2026-02-17T06:03:55.354 INFO:tasks.rados.rados.0.trial115.stdout:2980: oid 22 updating version 1339 to 1341 2026-02-17T06:03:55.354 INFO:tasks.rados.rados.0.trial115.stdout:2980: oid 22 version 1341 is already newer than 1340 2026-02-17T06:03:55.354 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1341 (ObjNum 1044 snap 305 seq_num 1044) dirty exists 2026-02-17T06:03:55.354 INFO:tasks.rados.rados.0.trial115.stdout:2980: left oid 22 (ObjNum 1044 snap 305 seq_num 1044) 2026-02-17T06:03:56.330 INFO:tasks.rados.rados.0.trial115.stdout:2980: done (2 left) 2026-02-17T06:03:56.330 INFO:tasks.rados.rados.0.trial115.stdout:2982: done (1 left) 2026-02-17T06:03:56.330 INFO:tasks.rados.rados.0.trial115.stdout:2984: done (0 left) 2026-02-17T06:03:56.330 INFO:tasks.rados.rados.0.trial115.stdout:2985: delete oid 23 current snap is 306 2026-02-17T06:03:56.334 INFO:tasks.rados.rados.0.trial115.stdout:2985: done (0 left) 2026-02-17T06:03:56.334 INFO:tasks.rados.rados.0.trial115.stdout:2986: rollback oid 4 current snap is 306 2026-02-17T06:03:56.334 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 289 2026-02-17T06:03:56.334 INFO:tasks.rados.rados.0.trial115.stdout:2987: copy_from oid 23 from oid 8 current snap is 306 2026-02-17T06:03:56.334 INFO:tasks.rados.rados.0.trial115.stdout:2988: copy_from oid 44 from oid 31 current snap is 306 2026-02-17T06:03:56.334 INFO:tasks.rados.rados.0.trial115.stdout:2989: copy_from oid 33 from oid 36 current snap is 306 2026-02-17T06:03:56.334 INFO:tasks.rados.rados.0.trial115.stdout:2990: snap_create 2026-02-17T06:03:56.334 INFO:tasks.rados.rados.0.trial115.stdout:2989: finishing copy_from to trial11545420-33 2026-02-17T06:03:56.335 INFO:tasks.rados.rados.0.trial115.stdout:2989: got expected ENOENT (src dne) 2026-02-17T06:03:56.335 INFO:tasks.rados.rados.0.trial115.stdout:2987: finishing copy_from to trial11545420-23 2026-02-17T06:03:56.335 INFO:tasks.rados.rados.0.trial115.stdout:2987: got expected ENOENT (src dne) 2026-02-17T06:03:56.335 INFO:tasks.rados.rados.0.trial115.stdout:2989: finishing copy_from racing read to trial11545420-33 2026-02-17T06:03:56.335 INFO:tasks.rados.rados.0.trial115.stdout:2987: finishing copy_from racing read to trial11545420-23 2026-02-17T06:03:56.336 INFO:tasks.rados.rados.0.trial115.stdout:2986: finishing rollback tid 0 to trial11545420-4 2026-02-17T06:03:56.336 INFO:tasks.rados.rados.0.trial115.stdout:2986: finishing rollback tid 1 to trial11545420-4 2026-02-17T06:03:56.336 INFO:tasks.rados.rados.0.trial115.stdout:2986: finishing rollback tid 2 to trial11545420-4 2026-02-17T06:03:56.336 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1426 (ObjNum 980 snap 281 seq_num 980) dirty exists 2026-02-17T06:03:56.341 INFO:tasks.rados.rados.0.trial115.stdout:2988: finishing copy_from racing read to trial11545420-44 2026-02-17T06:03:56.342 INFO:tasks.rados.rados.0.trial115.stdout:2988: finishing copy_from to trial11545420-44 2026-02-17T06:03:56.342 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1418 (ObjNum 887 snap 254 seq_num 887) dirty exists 2026-02-17T06:03:57.331 INFO:tasks.rados.rados.0.trial115.stdout:2986: done (4 left) 2026-02-17T06:03:57.331 INFO:tasks.rados.rados.0.trial115.stdout:2987: done (3 left) 2026-02-17T06:03:57.331 INFO:tasks.rados.rados.0.trial115.stdout:2988: done (2 left) 2026-02-17T06:03:57.331 INFO:tasks.rados.rados.0.trial115.stdout:2989: done (1 left) 2026-02-17T06:03:57.331 INFO:tasks.rados.rados.0.trial115.stdout:2990: done (0 left) 2026-02-17T06:03:57.331 INFO:tasks.rados.rados.0.trial115.stdout:2991: snap_create 2026-02-17T06:03:58.331 INFO:tasks.rados.rados.0.trial115.stdout:2991: done (0 left) 2026-02-17T06:03:58.331 INFO:tasks.rados.rados.0.trial115.stdout:2992: delete oid 31 current snap is 308 2026-02-17T06:03:58.334 INFO:tasks.rados.rados.0.trial115.stdout:2992: done (0 left) 2026-02-17T06:03:58.335 INFO:tasks.rados.rados.0.trial115.stdout:2993: snap_create 2026-02-17T06:03:58.666 INFO:tasks.rados.rados.0.trial115.stdout:2993: done (0 left) 2026-02-17T06:03:58.667 INFO:tasks.rados.rados.0.trial115.stdout:2994: snap_create 2026-02-17T06:03:59.668 INFO:tasks.rados.rados.0.trial115.stdout:2994: done (0 left) 2026-02-17T06:03:59.668 INFO:tasks.rados.rados.0.trial115.stdout:2995: rollback oid 20 current snap is 310 2026-02-17T06:03:59.668 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 20 to 299 2026-02-17T06:03:59.668 INFO:tasks.rados.rados.0.trial115.stdout:2996: snap_remove snap 308 2026-02-17T06:04:00.670 INFO:tasks.rados.rados.0.trial115.stdout:2996: done (1 left) 2026-02-17T06:04:00.670 INFO:tasks.rados.rados.0.trial115.stdout:2997: delete oid 10 current snap is 310 2026-02-17T06:04:00.670 INFO:tasks.rados.rados.0.trial115.stdout:2995: finishing rollback tid 0 to trial11545420-20 2026-02-17T06:04:00.670 INFO:tasks.rados.rados.0.trial115.stdout:2995: finishing rollback tid 1 to trial11545420-20 2026-02-17T06:04:00.670 INFO:tasks.rados.rados.0.trial115.stdout:2995: finishing rollback tid 2 to trial11545420-20 2026-02-17T06:04:00.670 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1456 (ObjNum 1037 snap 299 seq_num 1037) dirty exists 2026-02-17T06:04:00.673 INFO:tasks.rados.rados.0.trial115.stdout:2995: done (1 left) 2026-02-17T06:04:00.673 INFO:tasks.rados.rados.0.trial115.stdout:2997: done (0 left) 2026-02-17T06:04:00.673 INFO:tasks.rados.rados.0.trial115.stdout:2998: copy_from oid 37 from oid 1 current snap is 310 2026-02-17T06:04:00.673 INFO:tasks.rados.rados.0.trial115.stdout:2999: snap_create 2026-02-17T06:04:00.676 INFO:tasks.rados.rados.0.trial115.stdout:2998: finishing copy_from to trial11545420-37 2026-02-17T06:04:00.676 INFO:tasks.rados.rados.0.trial115.stdout:2998: got expected ENOENT (src dne) 2026-02-17T06:04:00.676 INFO:tasks.rados.rados.0.trial115.stdout:2998: finishing copy_from racing read to trial11545420-37 2026-02-17T06:04:01.240 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T06:04:01.240 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-17T06:04:01.240 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-17T06:04:01.240 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-17T06:04:01.242 INFO:tasks.ceph.mon.c:Started 2026-02-17T06:04:01.243 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:04:01.243 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:04:01.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.310+0000 7f424003b640 10 monclient: get_monmap_and_config 2026-02-17T06:04:01.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.310+0000 7f424003b640 10 monclient: build_initial_monmap 2026-02-17T06:04:01.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.310+0000 7f424003b640 10 monclient: monmap: 2026-02-17T06:04:01.313 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:01.311053+0000 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:01.311053+0000 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:01.314 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 1 Processor -- start 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 1 -- start start 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient: init 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f423809ea90 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f423809ee70 0x7f423809f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient(hunting): picked mon.noname-c con 0x7f423809ee70 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f423813d640 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42380a0a20 con 0x7f423809ea90 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42380a0460 con 0x7f423813d640 2026-02-17T06:04:01.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f42380a0d00 con 0x7f423809ee70 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f424003b640 10 monclient(hunting): authenticate will time out at 1686.864618s 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423ddb0640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f423809ea90 0x7f42380a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:39346/0 (socket says 10.20.193.78:39346) 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423ddb0640 1 -- 10.20.193.78:0/2309733018 learned_addr learned my addr 10.20.193.78:0/2309733018 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 1 --2- 10.20.193.78:0/2309733018 >> v2:10.20.193.115:3300/0 conn(0x7f423809ee70 0x7f423809f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient(hunting): get_auth_request con 0x7f423809ee70 auth_method 0 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:01.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient(hunting): handle_auth_done global_id 214168 payload 274 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 1 -- 10.20.193.78:0/2309733018 >> v1:10.20.193.115:6789/0 conn(0x7f423813d640 legacy=0x7f423813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 1 -- 10.20.193.78:0/2309733018 >> v1:10.20.193.78:6789/0 conn(0x7f423809ea90 legacy=0x7f42380a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient: _finish_hunting 0 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 1 monclient: found mon.noname-c 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:04:01.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.311+0000 7f423d5af640 1 -- 10.20.193.78:0/2309733018 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42380a0740 con 0x7f423809ee70 2026-02-17T06:04:01.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423d5af640 1 -- 10.20.193.78:0/2309733018 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423809ee70 msgr2=0x7f423809f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:01.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423d5af640 1 --2- 10.20.193.78:0/2309733018 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423809ee70 0x7f423809f2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f422c0098a0 tx=0x7f422c02ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28bb488b9c44d38 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:01.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 1 -- 10.20.193.78:0/2309733018 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c03c070 con 0x7f423809ee70 2026-02-17T06:04:01.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:01.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:04:01.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: dump: 2026-02-17T06:04:01.325 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:01.325 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:01.326 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:01.326 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:01.326 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:01.326 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:01.326 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:01.326 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:01.326 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:01.327 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:01.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: _finish_auth 0 2026-02-17T06:04:01.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: _check_auth_tickets 2026-02-17T06:04:01.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:01.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 1 -- 10.20.193.78:0/2309733018 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f422c02fce0 con 0x7f423809ee70 2026-02-17T06:04:01.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:01.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 1 -- 10.20.193.78:0/2309733018 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c037660 con 0x7f423809ee70 2026-02-17T06:04:01.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:01.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:04:01.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f423cdae640 10 monclient: dump: 2026-02-17T06:04:01.328 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:01.328 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:01.328 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:01.328 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:01.328 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:01.328 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:01.329 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:01.329 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:01.329 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:01.329 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:01.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f424003b640 5 monclient: authenticate success, global_id 214168 2026-02-17T06:04:01.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f424003b640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:01.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f424003b640 4 set_mon_vals no callback set 2026-02-17T06:04:01.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f424003b640 10 monclient: shutdown 2026-02-17T06:04:01.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f424003b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:01.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f424003b640 1 -- 10.20.193.78:0/2309733018 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423809ee70 msgr2=0x7f423809f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:01.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.314+0000 7f424003b640 1 --2- 10.20.193.78:0/2309733018 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423809ee70 0x7f423809f2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f422c0098a0 tx=0x7f422c02ed80 comp rx=0 tx=0).stop 2026-02-17T06:04:01.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 -- 10.20.193.78:0/2309733018 shutdown_connections 2026-02-17T06:04:01.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 --2- 10.20.193.78:0/2309733018 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423809ee70 0x7f423809f2c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:01.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 -- 10.20.193.78:0/2309733018 >> 10.20.193.78:0/2309733018 conn(0x7f42380a7520 msgr2=0x7f42380a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:01.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 -- 10.20.193.78:0/2309733018 shutdown_connections 2026-02-17T06:04:01.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 -- 10.20.193.78:0/2309733018 wait complete. 2026-02-17T06:04:01.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient: build_initial_monmap 2026-02-17T06:04:01.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient: monmap: 2026-02-17T06:04:01.331 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:01.331 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:01.331 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:01.331 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:01.331 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:01.331 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:01.332 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:01.332 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:01.332 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:01.332 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:01.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 Processor -- start 2026-02-17T06:04:01.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 -- start start 2026-02-17T06:04:01.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient: init 2026-02-17T06:04:01.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:01.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:04:01.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f423809ea90 0x7f42381cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:01.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient(hunting): picked mon.noname-a con 0x7f423809ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:01.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 0x7f42380736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:01.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient(hunting): picked mon.noname-c con 0x7f423809ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:01.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423813d640 0x7f4238073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:01.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient(hunting): picked mon.noname-b con 0x7f423813d640 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:01.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42380a6820 con 0x7f423809ea90 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42380a66a0 con 0x7f423813d640 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4238141150 con 0x7f423809ee70 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f424003b640 10 monclient(hunting): authenticate will time out at 1686.868903s 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423ddb0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f423809ea90 0x7f42381cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:01.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423d5af640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 0x7f42380736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:01.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423ddb0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f423809ea90 0x7f42381cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:34570/0 (socket says 10.20.193.78:34570) 2026-02-17T06:04:01.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423d5af640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 0x7f42380736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59038/0 (socket says 10.20.193.78:59038) 2026-02-17T06:04:01.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423ddb0640 1 -- 10.20.193.78:0/3590425100 learned_addr learned my addr 10.20.193.78:0/3590425100 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:01.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423ddb0640 10 monclient(hunting): get_auth_request con 0x7f423809ea90 auth_method 0 2026-02-17T06:04:01.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423ddb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:01.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423ddb0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:01.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423ddb0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:01.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423d5af640 10 monclient(hunting): get_auth_request con 0x7f423809ee70 auth_method 0 2026-02-17T06:04:01.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423d5af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:01.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423d5af640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:01.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.315+0000 7f423d5af640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:01.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423d5af640 1 -- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 msgr2=0x7f42380736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:04:01.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:01.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:01.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423d5af640 1 -- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 msgr2=0x7f42380736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423d5af640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 0x7f42380736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423d5af640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 0x7f42380736c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423e5b1640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423813d640 0x7f4238073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423e5b1640 10 monclient(hunting): get_auth_request con 0x7f423813d640 auth_method 0 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423e5b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423e5b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423e5b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:01.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 10 monclient(hunting): handle_auth_done global_id 184461 payload 824 2026-02-17T06:04:01.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 1 -- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 msgr2=0x7f42380736c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:04:01.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 0x7f42380736c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:01.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 1 -- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423813d640 msgr2=0x7f4238073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:01.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423813d640 0x7f4238073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:01.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 10 monclient: _finish_hunting 0 2026-02-17T06:04:01.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 1 monclient: found mon.noname-a 2026-02-17T06:04:01.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:01.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:01.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423ddb0640 1 -- 10.20.193.78:0/3590425100 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42380a1c10 con 0x7f423809ea90 2026-02-17T06:04:01.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.316+0000 7f423e5b1640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423813d640 0x7f4238073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T06:04:01.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f423ddb0640 1 -- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f423809ea90 msgr2=0x7f42381cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:01.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f423ddb0640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f423809ea90 0x7f42381cf3a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f422800c750 tx=0x7f422800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:01.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 1 -- 10.20.193.78:0/3590425100 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228005de0 con 0x7f423809ea90 2026-02-17T06:04:01.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:01.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:01.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: dump: 2026-02-17T06:04:01.340 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:01.340 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:01.340 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:01.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:01.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:01.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:01.340 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:01.340 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:01.341 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:01.341 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:01.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: _finish_auth 0 2026-02-17T06:04:01.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: _check_auth_tickets 2026-02-17T06:04:01.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:01.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 1 -- 10.20.193.78:0/3590425100 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4228007070 con 0x7f423809ea90 2026-02-17T06:04:01.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f424003b640 5 monclient: authenticate success, global_id 184461 2026-02-17T06:04:01.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:01.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 1 -- 10.20.193.78:0/3590425100 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228007440 con 0x7f423809ea90 2026-02-17T06:04:01.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:01.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:01.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f42267fc640 10 monclient: dump: 2026-02-17T06:04:01.342 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:01.342 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:01.342 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:01.342 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f4227fff640 4 set_mon_vals no callback set 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f424003b640 10 monclient: _renew_subs 2026-02-17T06:04:01.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f424003b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423806f8d0 con 0x7f423809ea90 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f424003b640 10 monclient: _renew_subs 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f424003b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42381cf8e0 con 0x7f423809ea90 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.318+0000 7f424003b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.319+0000 7f424003b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.319+0000 7f424003b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.319+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42380a7cb0 con 0x7f423809ea90 2026-02-17T06:04:01.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.319+0000 7f42267fc640 1 -- 10.20.193.78:0/3590425100 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4228007660 con 0x7f423809ea90 2026-02-17T06:04:01.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.319+0000 7f42267fc640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f420803deb0 0x7f4208040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:01.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.319+0000 7f42267fc640 1 -- 10.20.193.78:0/3590425100 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(640..640 src has 525..640) ==== 4795+0+0 (secure 0 0 0) 0x7f42280526c0 con 0x7f423809ea90 2026-02-17T06:04:01.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.319+0000 7f423d5af640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f420803deb0 0x7f4208040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:01.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.319+0000 7f423d5af640 10 monclient: get_auth_request con 0x7f420803deb0 auth_method 0 2026-02-17T06:04:01.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.321+0000 7f42267fc640 1 -- 10.20.193.78:0/3590425100 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f42380a7cb0 con 0x7f423809ea90 2026-02-17T06:04:01.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.321+0000 7f42267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:01.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.321+0000 7f42267fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:01.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.322+0000 7f423d5af640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f420803deb0 0x7f4208040370 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f422c036000 tx=0x7f422c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:01.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.435+0000 7f424003b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:04:01.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.435+0000 7f424003b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:01.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.435+0000 7f424003b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:01.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.435+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f423809f2c0 con 0x7f423809ea90 2026-02-17T06:04:01.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.435+0000 7f42267fc640 1 -- 10.20.193.78:0/3590425100 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f422800a8d0 con 0x7f423809ea90 2026-02-17T06:04:01.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.435+0000 7f42267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:01.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.435+0000 7f42267fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:01.436 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":210,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:04:01.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f420803deb0 msgr2=0x7f4208040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:01.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f420803deb0 0x7f4208040370 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f422c036000 tx=0x7f422c033000 comp rx=0 tx=0).stop 2026-02-17T06:04:01.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 10 monclient: shutdown 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f423809ea90 msgr2=0x7f42381cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f423809ea90 0x7f42381cf3a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f422800c750 tx=0x7f422800cc20 comp rx=0 tx=0).stop 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 shutdown_connections 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f420803deb0 0x7f4208040370 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f423813d640 0x7f4238073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f423809ee70 0x7f42380736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 --2- 10.20.193.78:0/3590425100 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f423809ea90 0x7f42381cf3a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 >> 10.20.193.78:0/3590425100 conn(0x7f42380a7520 msgr2=0x7f423813fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 shutdown_connections 2026-02-17T06:04:01.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:01.436+0000 7f424003b640 1 -- 10.20.193.78:0/3590425100 wait complete. 2026-02-17T06:04:01.672 INFO:tasks.rados.rados.0.trial115.stdout:2998: done (1 left) 2026-02-17T06:04:01.672 INFO:tasks.rados.rados.0.trial115.stdout:2999: done (0 left) 2026-02-17T06:04:01.672 INFO:tasks.rados.rados.0.trial115.stdout:3000: read oid 39 snap 309 2026-02-17T06:04:01.672 INFO:tasks.rados.rados.0.trial115.stdout:3000: expect deleted 2026-02-17T06:04:01.673 INFO:tasks.rados.rados.0.trial115.stdout:3001: snap_remove snap 306 2026-02-17T06:04:02.671 INFO:tasks.rados.rados.0.trial115.stdout:3001: done (1 left) 2026-02-17T06:04:02.671 INFO:tasks.rados.rados.0.trial115.stdout:3002: snap_create 2026-02-17T06:04:03.667 INFO:tasks.rados.rados.0.trial115.stdout:3000: done (1 left) 2026-02-17T06:04:03.667 INFO:tasks.rados.rados.0.trial115.stdout:3002: done (0 left) 2026-02-17T06:04:03.667 INFO:tasks.rados.rados.0.trial115.stdout:3003: write (excl) oid 14 current snap is 312 2026-02-17T06:04:03.667 INFO:tasks.rados.rados.0.trial115.stdout:3003: seq_num 1049 ranges {430543=476562,1385258=663035,2840837=669165} 2026-02-17T06:04:03.671 INFO:tasks.rados.rados.0.trial115.stdout:3003: writing trial11545420-14 from 430543 to 907105 tid 1 2026-02-17T06:04:03.675 INFO:tasks.rados.rados.0.trial115.stdout:3003: writing trial11545420-14 from 1385258 to 2048293 tid 2 2026-02-17T06:04:03.681 INFO:tasks.rados.rados.0.trial115.stdout:3003: writing trial11545420-14 from 2840837 to 3510002 tid 3 2026-02-17T06:04:03.681 INFO:tasks.rados.rados.0.trial115.stdout:3004: copy_from oid 25 from oid 7 current snap is 312 2026-02-17T06:04:03.681 INFO:tasks.rados.rados.0.trial115.stdout:3003: finishing write tid 1 to trial11545420-14 2026-02-17T06:04:03.681 INFO:tasks.rados.rados.0.trial115.stdout:3003: finishing write tid 2 to trial11545420-14 2026-02-17T06:04:03.681 INFO:tasks.rados.rados.0.trial115.stdout:3005: write oid 22 current snap is 312 2026-02-17T06:04:03.681 INFO:tasks.rados.rados.0.trial115.stdout:3005: seq_num 1050 ranges {447350=668164,1560093=406693,2584613=480437} 2026-02-17T06:04:03.685 INFO:tasks.rados.rados.0.trial115.stdout:3005: writing trial11545420-22 from 447350 to 1115514 tid 1 2026-02-17T06:04:03.688 INFO:tasks.rados.rados.0.trial115.stdout:3005: writing trial11545420-22 from 1560093 to 1966786 tid 2 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3005: writing trial11545420-22 from 2584613 to 3065050 tid 3 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3006: snap_create 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3003: finishing write tid 3 to trial11545420-14 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3003: finishing write tid 4 to trial11545420-14 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3003: finishing write tid 5 to trial11545420-14 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3003: oid 14 updating version 0 to 1450 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3003: oid 14 version 1450 is already newer than 1448 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3003: oid 14 version 1450 is already newer than 1449 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3003: oid 14 updating version 1450 to 1451 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1451 (ObjNum 1049 snap 312 seq_num 1049) dirty exists 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3003: left oid 14 (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3004: finishing copy_from racing read to trial11545420-25 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:3004: finishing copy_from to trial11545420-25 2026-02-17T06:04:03.692 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1441 (ObjNum 1019 snap 297 seq_num 1019) dirty exists 2026-02-17T06:04:03.693 INFO:tasks.rados.rados.0.trial115.stdout:3005: finishing write tid 1 to trial11545420-22 2026-02-17T06:04:03.695 INFO:tasks.rados.rados.0.trial115.stdout:3005: finishing write tid 2 to trial11545420-22 2026-02-17T06:04:03.695 INFO:tasks.rados.rados.0.trial115.stdout:3005: finishing write tid 3 to trial11545420-22 2026-02-17T06:04:03.695 INFO:tasks.rados.rados.0.trial115.stdout:3005: finishing write tid 4 to trial11545420-22 2026-02-17T06:04:03.695 INFO:tasks.rados.rados.0.trial115.stdout:3005: finishing write tid 5 to trial11545420-22 2026-02-17T06:04:03.695 INFO:tasks.rados.rados.0.trial115.stdout:3005: oid 22 updating version 0 to 1342 2026-02-17T06:04:03.695 INFO:tasks.rados.rados.0.trial115.stdout:3005: oid 22 updating version 1342 to 1343 2026-02-17T06:04:03.695 INFO:tasks.rados.rados.0.trial115.stdout:3005: oid 22 updating version 1343 to 1344 2026-02-17T06:04:03.696 INFO:tasks.rados.rados.0.trial115.stdout:3005: oid 22 updating version 1344 to 1345 2026-02-17T06:04:03.696 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1345 (ObjNum 1050 snap 312 seq_num 1050) dirty exists 2026-02-17T06:04:03.696 INFO:tasks.rados.rados.0.trial115.stdout:3005: left oid 22 (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:04:04.445 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:04:04.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient: get_monmap_and_config 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient: build_initial_monmap 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient: monmap: 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:04.514261+0000 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:04.514261+0000 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 1 Processor -- start 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 1 -- start start 2026-02-17T06:04:04.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient: init 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd0d406c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd0d409d430 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 0x7fd0d4140d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient(hunting): picked mon.noname-b con 0x7fd0d409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d4073600 con 0x7fd0d409d430 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d406dab0 con 0x7fd0d406c890 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d409ca50 con 0x7fd0d409bf40 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0dbc0c640 10 monclient(hunting): authenticate will time out at 1690.067790s 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0d9180640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd0d409d430 0x7fd0d413d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:39360/0 (socket says 10.20.193.78:39360) 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 0x7fd0d4140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0d9180640 1 -- 10.20.193.78:0/1702442128 learned_addr learned my addr 10.20.193.78:0/1702442128 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 10 monclient(hunting): get_auth_request con 0x7fd0d409bf40 auth_method 0 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 1 -- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 msgr2=0x7fd0d4140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 1 -- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 msgr2=0x7fd0d4140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 0x7fd0d4140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:04.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.514+0000 7fd0da182640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 0x7fd0d4140d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:04.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.516+0000 7fd0d9180640 1 -- 10.20.193.78:0/1702442128 >> v1:10.20.193.78:6789/0 conn(0x7fd0d409d430 legacy=0x7fd0d413d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762814936 0 0) 0x7fd0d4073600 con 0x7fd0d409d430 2026-02-17T06:04:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): _negotiate 2026-02-17T06:04:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): my global_id is 184464 2026-02-17T06:04:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0bc003620 con 0x7fd0d409d430 2026-02-17T06:04:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 1 -- 10.20.193.78:0/1702442128 >> v1:10.20.193.115:6789/0 conn(0x7fd0d406c890 legacy=0x7fd0d406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:04.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2518331746 0 0) 0x7fd0bc003620 con 0x7fd0d409d430 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 msgr2=0x7fd0d4140d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 0x7fd0d4140d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 >> v1:10.20.193.115:6789/0 conn(0x7fd0d406c890 legacy=0x7fd0d406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: _finish_hunting 0 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d409c770 con 0x7fd0d409d430 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: _finish_auth 0 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: _check_auth_tickets 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fd0c80031b0 con 0x7fd0d409d430 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0dbc0c640 5 monclient: authenticate success, global_id 184464 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0dbc0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: dump: 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 >> v1:10.20.193.78:6789/0 conn(0x7fd0d409d430 legacy=0x7fd0d413d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0bc003c00 0x7fd0bc004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): picked mon.a con 0x7fd0bc003c00 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 0x7fd0bc004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): picked mon.c con 0x7fd0d409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0bc004ff0 0x7fd0bc00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): picked mon.b con 0x7fd0bc004ff0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d409ca50 con 0x7fd0bc003c00 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:04.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc00c000 con 0x7fd0bc004ff0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0bc00c180 con 0x7fd0d409bf40 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d897f640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0dbc0c640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9180640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 0x7fd0bc004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9180640 10 monclient(hunting): get_auth_request con 0x7fd0d409bf40 auth_method 0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9180640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9180640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9180640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0bc003c00 0x7fd0bc004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient(hunting): get_auth_request con 0x7fd0bc003c00 auth_method 0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9180640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9180640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9180640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient(hunting): handle_auth_done global_id 184464 payload 293 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 1 -- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 msgr2=0x7fd0bc004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d409bf40 0x7fd0bc004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 1 -- 10.20.193.78:0/1702442128 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0bc004ff0 msgr2=0x7fd0bc00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0bc004ff0 0x7fd0bc00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient: _finish_hunting 0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 1 monclient: found mon.a 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 1 -- 10.20.193.78:0/1702442128 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc003620 con 0x7fd0bc003c00 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient: _finish_auth 0 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 10 monclient: _check_auth_tickets 2026-02-17T06:04:04.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.517+0000 7fd0d9981640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d9981640 1 -- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0bc003c00 msgr2=0x7fd0bc004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d9981640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0bc003c00 0x7fd0bc004020 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40032a0 tx=0x7fd0c402efe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8405e7ed2f969f6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4002fa0 con 0x7fd0bc003c00 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 10 monclient: dump: 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c4033030 con 0x7fd0bc003c00 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 1 -- 10.20.193.78:0/1702442128 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4005d20 con 0x7fd0bc003c00 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 4 set_mon_vals no callback set 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0d897f640 10 monclient: dump: 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.520 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 10 monclient: shutdown 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0bc003c00 msgr2=0x7fd0bc004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 1 --2- 10.20.193.78:0/1702442128 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0bc003c00 0x7fd0bc004020 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40032a0 tx=0x7fd0c402efe0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0da182640 1 -- 10.20.193.78:0/1702442128 reap_dead start 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/1702442128 shutdown_connections 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/1702442128 >> 10.20.193.78:0/1702442128 conn(0x7fd0d40a6620 msgr2=0x7fd0d40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/1702442128 shutdown_connections 2026-02-17T06:04:04.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/1702442128 wait complete. 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 10 monclient: build_initial_monmap 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.520+0000 7fd0dbc0c640 10 monclient: monmap: 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 1 Processor -- start 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 1 -- start start 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient: init 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0d41cbba0 0x7fd0d4135a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient(hunting): picked mon.noname-b con 0x7fd0d41cbba0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 0x7fd0d41cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient(hunting): picked mon.noname-c con 0x7fd0d41cf0a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0d41cfa60 0x7fd0d41d6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient(hunting): picked mon.noname-a con 0x7fd0d41cfa60 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d406d380 con 0x7fd0d41cfa60 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d4073410 con 0x7fd0d41cbba0 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d406d500 con 0x7fd0d41cf0a0 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0dbc0c640 10 monclient(hunting): authenticate will time out at 1690.074536s 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 0x7fd0d41cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0d41cfa60 0x7fd0d41d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 0x7fd0d41cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59080/0 (socket says 10.20.193.78:59080) 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 1 -- 10.20.193.78:0/765794270 learned_addr learned my addr 10.20.193.78:0/765794270 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 10 monclient(hunting): get_auth_request con 0x7fd0d41cf0a0 auth_method 0 2026-02-17T06:04:04.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient(hunting): get_auth_request con 0x7fd0d41cfa60 auth_method 0 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 1 -- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 msgr2=0x7fd0d41cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 1 -- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 msgr2=0x7fd0d41cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 0x7fd0d41cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0d9180640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 0x7fd0d41cf520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient(hunting): handle_auth_done global_id 184467 payload 824 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 1 -- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 msgr2=0x7fd0d41cf520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 0x7fd0d41cf520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 1 -- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0d41cbba0 msgr2=0x7fd0d4135a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0d41cbba0 0x7fd0d4135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient: _finish_hunting 0 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 1 monclient: found mon.noname-a 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.521+0000 7fd0da182640 1 -- 10.20.193.78:0/765794270 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d41cfee0 con 0x7fd0d41cfa60 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.523+0000 7fd0da182640 1 -- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0d41cfa60 msgr2=0x7fd0d41d6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.523+0000 7fd0da182640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0d41cfa60 0x7fd0d41d6410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0d409c670 tx=0x7fd0d0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 1 -- 10.20.193.78:0/765794270 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000ed00 con 0x7fd0d41cfa60 2026-02-17T06:04:04.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: dump: 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: _finish_auth 0 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: _check_auth_tickets 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 1 -- 10.20.193.78:0/765794270 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d0019070 con 0x7fd0d41cfa60 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 1 -- 10.20.193.78:0/765794270 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000eea0 con 0x7fd0d41cfa60 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c3fff640 4 set_mon_vals no callback set 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 10 monclient: dump: 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 5 monclient: authenticate success, global_id 184467 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 10 monclient: _renew_subs 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d41213c0 con 0x7fd0d41cfa60 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 10 monclient: _renew_subs 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:04.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d409d810 con 0x7fd0d41cfa60 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d406cc70 con 0x7fd0d41cfa60 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 1 -- 10.20.193.78:0/765794270 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd0d0014720 con 0x7fd0d41cfa60 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0a403df00 0x7fd0a40403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.524+0000 7fd0c27fc640 1 -- 10.20.193.78:0/765794270 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(643..643 src has 525..643) ==== 4795+0+0 (secure 0 0 0) 0x7fd0d00529f0 con 0x7fd0d41cfa60 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.525+0000 7fd0d9981640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0a403df00 0x7fd0a40403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:04.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.525+0000 7fd0d9981640 10 monclient: get_auth_request con 0x7fd0a403df00 auth_method 0 2026-02-17T06:04:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.527+0000 7fd0c27fc640 1 -- 10.20.193.78:0/765794270 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd0d0024250 con 0x7fd0d41cfa60 2026-02-17T06:04:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.527+0000 7fd0c27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:04.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.527+0000 7fd0c27fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:04.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.528+0000 7fd0d9981640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0a403df00 0x7fd0a40403c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4033000 tx=0x7fd0c4031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:04.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.640+0000 7fd0dbc0c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:04:04.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.640+0000 7fd0dbc0c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:04.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.640+0000 7fd0dbc0c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:04.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.640+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd0d411f0d0 con 0x7fd0d41cfa60 2026-02-17T06:04:04.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.641+0000 7fd0c27fc640 1 -- 10.20.193.78:0/765794270 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fd0d001c360 con 0x7fd0d41cfa60 2026-02-17T06:04:04.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.641+0000 7fd0c27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:04.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.641+0000 7fd0c27fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:04.643 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":210,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:04:04.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0a403df00 msgr2=0x7fd0a40403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:04.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0a403df00 0x7fd0a40403c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4033000 tx=0x7fd0c4031000 comp rx=0 tx=0).stop 2026-02-17T06:04:04.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 10 monclient: shutdown 2026-02-17T06:04:04.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:04.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0d41cfa60 msgr2=0x7fd0d41d6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:04.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0d41cfa60 0x7fd0d41d6410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0d409c670 tx=0x7fd0d0007f30 comp rx=0 tx=0).stop 2026-02-17T06:04:04.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 shutdown_connections 2026-02-17T06:04:04.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd0d41cfa60 0x7fd0d41d6410 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd0d41cf0a0 0x7fd0d41cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd0a403df00 0x7fd0a40403c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 --2- 10.20.193.78:0/765794270 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd0d41cbba0 0x7fd0d4135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:04.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 >> 10.20.193.78:0/765794270 conn(0x7fd0d40a6620 msgr2=0x7fd0d409f320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:04.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 shutdown_connections 2026-02-17T06:04:04.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:04.642+0000 7fd0dbc0c640 1 -- 10.20.193.78:0/765794270 wait complete. 2026-02-17T06:04:04.671 INFO:tasks.rados.rados.0.trial115.stdout:3003: done (3 left) 2026-02-17T06:04:04.671 INFO:tasks.rados.rados.0.trial115.stdout:3004: done (2 left) 2026-02-17T06:04:04.671 INFO:tasks.rados.rados.0.trial115.stdout:3005: done (1 left) 2026-02-17T06:04:04.671 INFO:tasks.rados.rados.0.trial115.stdout:3006: done (0 left) 2026-02-17T06:04:04.671 INFO:tasks.rados.rados.0.trial115.stdout:3007: delete oid 5 current snap is 313 2026-02-17T06:04:04.674 INFO:tasks.rados.rados.0.trial115.stdout:3007: done (0 left) 2026-02-17T06:04:04.674 INFO:tasks.rados.rados.0.trial115.stdout:3008: read oid 1 snap -1 2026-02-17T06:04:04.674 INFO:tasks.rados.rados.0.trial115.stdout:3008: expect deleted 2026-02-17T06:04:04.674 INFO:tasks.rados.rados.0.trial115.stdout:3009: rollback oid 34 current snap is 313 2026-02-17T06:04:04.674 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 301 2026-02-17T06:04:04.674 INFO:tasks.rados.rados.0.trial115.stdout:3010: write (excl) oid 31 current snap is 313 2026-02-17T06:04:04.674 INFO:tasks.rados.rados.0.trial115.stdout:3010: seq_num 1052 ranges {480964=418662,1560704=410362,2523106=619334} 2026-02-17T06:04:04.677 INFO:tasks.rados.rados.0.trial115.stdout:3010: writing trial11545420-31 from 480964 to 899626 tid 1 2026-02-17T06:04:04.681 INFO:tasks.rados.rados.0.trial115.stdout:3010: writing trial11545420-31 from 1560704 to 1971066 tid 2 2026-02-17T06:04:04.685 INFO:tasks.rados.rados.0.trial115.stdout:3010: writing trial11545420-31 from 2523106 to 3142440 tid 3 2026-02-17T06:04:04.685 INFO:tasks.rados.rados.0.trial115.stdout:3011: snap_create 2026-02-17T06:04:04.685 INFO:tasks.rados.rados.0.trial115.stdout:3009: finishing rollback tid 0 to trial11545420-34 2026-02-17T06:04:04.686 INFO:tasks.rados.rados.0.trial115.stdout:3009: finishing rollback tid 1 to trial11545420-34 2026-02-17T06:04:04.686 INFO:tasks.rados.rados.0.trial115.stdout:3009: finishing rollback tid 2 to trial11545420-34 2026-02-17T06:04:04.686 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1148 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:04:04.686 INFO:tasks.rados.rados.0.trial115.stdout:3010: finishing write tid 1 to trial11545420-31 2026-02-17T06:04:04.686 INFO:tasks.rados.rados.0.trial115.stdout:3010: finishing write tid 2 to trial11545420-31 2026-02-17T06:04:04.687 INFO:tasks.rados.rados.0.trial115.stdout:3010: finishing write tid 3 to trial11545420-31 2026-02-17T06:04:04.687 INFO:tasks.rados.rados.0.trial115.stdout:3010: finishing write tid 4 to trial11545420-31 2026-02-17T06:04:04.687 INFO:tasks.rados.rados.0.trial115.stdout:3010: finishing write tid 5 to trial11545420-31 2026-02-17T06:04:04.688 INFO:tasks.rados.rados.0.trial115.stdout:3010: oid 31 updating version 0 to 1447 2026-02-17T06:04:04.688 INFO:tasks.rados.rados.0.trial115.stdout:3010: oid 31 updating version 1447 to 1449 2026-02-17T06:04:04.688 INFO:tasks.rados.rados.0.trial115.stdout:3010: oid 31 version 1449 is already newer than 1446 2026-02-17T06:04:04.688 INFO:tasks.rados.rados.0.trial115.stdout:3010: oid 31 version 1449 is already newer than 1448 2026-02-17T06:04:04.688 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1449 (ObjNum 1052 snap 313 seq_num 1052) dirty exists 2026-02-17T06:04:04.688 INFO:tasks.rados.rados.0.trial115.stdout:3010: left oid 31 (ObjNum 1052 snap 313 seq_num 1052) 2026-02-17T06:04:05.672 INFO:tasks.rados.rados.0.trial115.stdout:3008: done (3 left) 2026-02-17T06:04:05.672 INFO:tasks.rados.rados.0.trial115.stdout:3009: done (2 left) 2026-02-17T06:04:05.672 INFO:tasks.rados.rados.0.trial115.stdout:3010: done (1 left) 2026-02-17T06:04:05.672 INFO:tasks.rados.rados.0.trial115.stdout:3011: done (0 left) 2026-02-17T06:04:05.673 INFO:tasks.rados.rados.0.trial115.stdout:3012: delete oid 49 current snap is 314 2026-02-17T06:04:05.675 INFO:tasks.rados.rados.0.trial115.stdout:3012: done (0 left) 2026-02-17T06:04:05.675 INFO:tasks.rados.rados.0.trial115.stdout:3013: copy_from oid 12 from oid 11 current snap is 314 2026-02-17T06:04:05.675 INFO:tasks.rados.rados.0.trial115.stdout:3014: delete oid 30 current snap is 314 2026-02-17T06:04:05.676 INFO:tasks.rados.rados.0.trial115.stdout:3014: done (1 left) 2026-02-17T06:04:05.677 INFO:tasks.rados.rados.0.trial115.stdout:3015: delete oid 27 current snap is 314 2026-02-17T06:04:05.677 INFO:tasks.rados.rados.0.trial115.stdout:3015: done (1 left) 2026-02-17T06:04:05.678 INFO:tasks.rados.rados.0.trial115.stdout:3016: snap_remove snap 304 2026-02-17T06:04:06.924 INFO:tasks.rados.rados.0.trial115.stdout:3016: done (1 left) 2026-02-17T06:04:06.925 INFO:tasks.rados.rados.0.trial115.stdout:3017: read oid 8 snap 310 2026-02-17T06:04:06.925 INFO:tasks.rados.rados.0.trial115.stdout:3017: expect deleted 2026-02-17T06:04:06.925 INFO:tasks.rados.rados.0.trial115.stdout:3018: write oid 37 current snap is 314 2026-02-17T06:04:06.925 INFO:tasks.rados.rados.0.trial115.stdout:3013: finishing copy_from racing read to trial11545420-12 2026-02-17T06:04:06.925 INFO:tasks.rados.rados.0.trial115.stdout:3013: finishing copy_from to trial11545420-12 2026-02-17T06:04:06.925 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1453 (ObjNum 1027 snap 297 seq_num 1027) dirty exists 2026-02-17T06:04:06.925 INFO:tasks.rados.rados.0.trial115.stdout:3018: seq_num 1056 ranges {548192=719658,1678279=417700,2516445=780775} 2026-02-17T06:04:06.930 INFO:tasks.rados.rados.0.trial115.stdout:3018: writing trial11545420-37 from 548192 to 1267850 tid 1 2026-02-17T06:04:06.933 INFO:tasks.rados.rados.0.trial115.stdout:3018: writing trial11545420-37 from 1678279 to 2095979 tid 2 2026-02-17T06:04:06.938 INFO:tasks.rados.rados.0.trial115.stdout:3018: writing trial11545420-37 from 2516445 to 3297220 tid 3 2026-02-17T06:04:06.938 INFO:tasks.rados.rados.0.trial115.stdout:3013: done (2 left) 2026-02-17T06:04:06.938 INFO:tasks.rados.rados.0.trial115.stdout:3019: snap_remove snap 289 2026-02-17T06:04:07.651 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:04:07.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient: get_monmap_and_config 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient: build_initial_monmap 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient: monmap: 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:07.718448+0000 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:07.718448+0000 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.721 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 1 Processor -- start 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 1 -- start start 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient: init 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7efdac06c890 0x7efdac06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient(hunting): picked mon.noname-c con 0x7efdac06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efdac09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient(hunting): picked mon.noname-b con 0x7efdac06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdac06daf0 con 0x7efdac09c4a0 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7efdac0a0880 con 0x7efdac06c890 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efdac09cc80 con 0x7efdac06d1b0 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.718+0000 7efdb0ec7640 10 monclient(hunting): authenticate will time out at 1693.271979s 2026-02-17T06:04:07.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59348/0 (socket says 10.20.193.78:59348) 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 1 -- 10.20.193.78:0/3937412447 learned_addr learned my addr 10.20.193.78:0/3937412447 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient(hunting): get_auth_request con 0x7efdac06d1b0 auth_method 0 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaa575640 1 --2- 10.20.193.78:0/3937412447 >> v2:10.20.193.115:3300/0 conn(0x7efdac06c890 0x7efdac06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaa575640 10 monclient(hunting): get_auth_request con 0x7efdac06c890 auth_method 0 2026-02-17T06:04:07.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaa575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaa575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaa575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient(hunting): handle_auth_done global_id 224099 payload 274 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 1 -- 10.20.193.78:0/3937412447 >> v2:10.20.193.115:3300/0 conn(0x7efdac06c890 msgr2=0x7efdac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 1 --2- 10.20.193.78:0/3937412447 >> v2:10.20.193.115:3300/0 conn(0x7efdac06c890 0x7efdac06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 1 -- 10.20.193.78:0/3937412447 >> v1:10.20.193.115:6789/0 conn(0x7efdac09c4a0 legacy=0x7efdac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient: _finish_hunting 0 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 1 monclient: found mon.noname-b 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:04:07.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.719+0000 7efdaad76640 1 -- 10.20.193.78:0/3937412447 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac09cf60 con 0x7efdac06d1b0 2026-02-17T06:04:07.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efdaad76640 1 -- 10.20.193.78:0/3937412447 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 msgr2=0x7efdac135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:07.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efdaad76640 1 --2- 10.20.193.78:0/3937412447 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac135500 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efdac06dcc0 tx=0x7efd9c02f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2d093584bcd3b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:07.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 1 -- 10.20.193.78:0/3937412447 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0373f0 con 0x7efdac06d1b0 2026-02-17T06:04:07.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 10 monclient: dump: 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.733 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 10 monclient: _finish_auth 0 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 10 monclient: _check_auth_tickets 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 1 -- 10.20.193.78:0/3937412447 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9c037590 con 0x7efdac06d1b0 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efdb0ec7640 5 monclient: authenticate success, global_id 224099 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efdb0ec7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.721+0000 7efda9573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efda9573640 1 -- 10.20.193.78:0/3937412447 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c037960 con 0x7efdac06d1b0 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efda9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 4 set_mon_vals no callback set 2026-02-17T06:04:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efda9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efda9573640 10 monclient: dump: 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 10 monclient: shutdown 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 -- 10.20.193.78:0/3937412447 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 msgr2=0x7efdac135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/3937412447 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac135500 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efdac06dcc0 tx=0x7efd9c02f200 comp rx=0 tx=0).stop 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 -- 10.20.193.78:0/3937412447 shutdown_connections 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/3937412447 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac135500 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/3937412447 >> v2:10.20.193.115:3300/0 conn(0x7efdac06c890 0x7efdac06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 -- 10.20.193.78:0/3937412447 >> 10.20.193.78:0/3937412447 conn(0x7efdac0a6620 msgr2=0x7efdac0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 -- 10.20.193.78:0/3937412447 shutdown_connections 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 -- 10.20.193.78:0/3937412447 wait complete. 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 10 monclient: build_initial_monmap 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 10 monclient: monmap: 2026-02-17T06:04:07.736 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 Processor -- start 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 1 -- start start 2026-02-17T06:04:07.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.722+0000 7efdb0ec7640 10 monclient: init 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdac06c890 0x7efdac1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient(hunting): picked mon.noname-b con 0x7efdac06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient(hunting): picked mon.noname-c con 0x7efdac06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdac09c4a0 0x7efdac09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient(hunting): picked mon.noname-a con 0x7efdac09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7efdac0a01a0 con 0x7efdac09c4a0 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7efdac0a0020 con 0x7efdac06c890 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efdac09be20 con 0x7efdac06d1b0 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdb0ec7640 10 monclient(hunting): authenticate will time out at 1693.276196s 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac1cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59358/0 (socket says 10.20.193.78:59358) 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 -- 10.20.193.78:0/732145765 learned_addr learned my addr 10.20.193.78:0/732145765 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient(hunting): get_auth_request con 0x7efdac06d1b0 auth_method 0 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaad76640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdac09c4a0 0x7efdac09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaad76640 10 monclient(hunting): get_auth_request con 0x7efdac09c4a0 auth_method 0 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaa575640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdac06c890 0x7efdac1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaa575640 10 monclient(hunting): get_auth_request con 0x7efdac06c890 auth_method 0 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaa575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaa575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efdaa575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient(hunting): handle_auth_done global_id 224102 payload 824 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 -- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdac06c890 msgr2=0x7efdac1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdac06c890 0x7efdac1cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 -- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdac09c4a0 msgr2=0x7efdac09db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdac09c4a0 0x7efdac09db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient: _finish_hunting 0 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 monclient: found mon.noname-c 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:07.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.723+0000 7efda9d74640 1 -- 10.20.193.78:0/732145765 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac06fd20 con 0x7efdac06d1b0 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efda9d74640 1 -- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 msgr2=0x7efdac1cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efda9d74640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac1cf890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c630 tx=0x7efd9400cb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 1 -- 10.20.193.78:0/732145765 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94016020 con 0x7efdac06d1b0 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: dump: 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.742 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: _finish_auth 0 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 1 -- 10.20.193.78:0/732145765 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9400aa70 con 0x7efdac06d1b0 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 1 -- 10.20.193.78:0/732145765 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9400ae60 con 0x7efdac06d1b0 2026-02-17T06:04:07.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efdb0ec7640 5 monclient: authenticate success, global_id 224102 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efda8d72640 4 set_mon_vals no callback set 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efdb0ec7640 10 monclient: _renew_subs 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efdb0ec7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac128eb0 con 0x7efdac06d1b0 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efd92ffd640 10 monclient: dump: 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efdb0ec7640 10 monclient: _renew_subs 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efdb0ec7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.725+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac127fc0 con 0x7efdac06d1b0 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efdb0ec7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efdb0ec7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efdb0ec7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:04:07.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdac09d760 con 0x7efdac06d1b0 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efd92ffd640 1 -- 10.20.193.78:0/732145765 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7efd94012420 con 0x7efdac06d1b0 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efd92ffd640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd7803df00 0x7efd780403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efd92ffd640 1 -- 10.20.193.78:0/732145765 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(646..646 src has 525..646) ==== 4795+0+0 (secure 0 0 0) 0x7efd94052b60 con 0x7efdac06d1b0 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efdaa575640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd7803df00 0x7efd780403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.726+0000 7efdaa575640 10 monclient: get_auth_request con 0x7efd7803df00 auth_method 0 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.729+0000 7efdaa575640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd7803df00 0x7efd780403c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efda00099b0 tx=0x7efda0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.729+0000 7efd92ffd640 1 -- 10.20.193.78:0/732145765 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efd94020020 con 0x7efdac06d1b0 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.729+0000 7efd92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:07.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.729+0000 7efd92ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:07.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.841+0000 7efdb0ec7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:04:07.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.841+0000 7efdb0ec7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:07.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.841+0000 7efdb0ec7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:04:07.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.841+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efdac06cc70 con 0x7efdac06d1b0 2026-02-17T06:04:07.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efd92ffd640 1 -- 10.20.193.78:0/732145765 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7efd940119e0 con 0x7efdac06d1b0 2026-02-17T06:04:07.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efd92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:07.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efd92ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:07.843 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":212,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:04:07.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd7803df00 msgr2=0x7efd780403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd7803df00 0x7efd780403c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efda00099b0 tx=0x7efda0008040 comp rx=0 tx=0).stop 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 10 monclient: shutdown 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 msgr2=0x7efdac1cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac1cf890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c630 tx=0x7efd9400cb00 comp rx=0 tx=0).stop 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 shutdown_connections 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdac09c4a0 0x7efdac09db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdac06d1b0 0x7efdac1cf890 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd7803df00 0x7efd780403c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 --2- 10.20.193.78:0/732145765 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdac06c890 0x7efdac1cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 >> 10.20.193.78:0/732145765 conn(0x7efdac0a6620 msgr2=0x7efdac072130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:07.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 shutdown_connections 2026-02-17T06:04:07.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.842+0000 7efdb0ec7640 1 -- 10.20.193.78:0/732145765 wait complete. 2026-02-17T06:04:07.850 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:04:07.850 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:04:07.851 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:04:07.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.919+0000 7f626f123640 10 monclient: get_monmap_and_config 2026-02-17T06:04:07.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.919+0000 7f626f123640 10 monclient: build_initial_monmap 2026-02-17T06:04:07.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.919+0000 7f626f123640 10 monclient: monmap: 2026-02-17T06:04:07.922 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:07.922 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.922 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:07.919662+0000 2026-02-17T06:04:07.922 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:07.919662+0000 2026-02-17T06:04:07.922 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:07.922 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.922 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:07.922 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.919+0000 7f626f123640 1 Processor -- start 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.919+0000 7f626f123640 1 -- start start 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.919+0000 7f626f123640 10 monclient: init 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:07.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f626806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f626809c4a0 0x7f626813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient(hunting): picked mon.noname-c con 0x7f626809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f626806d380 0x7f626813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient(hunting): picked mon.noname-a con 0x7f626806d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f626806daf0 con 0x7f626806c890 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f62680a0880 con 0x7f626806d380 2026-02-17T06:04:07.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f626809cc80 con 0x7f626809c4a0 2026-02-17T06:04:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626f123640 10 monclient(hunting): authenticate will time out at 1693.473260s 2026-02-17T06:04:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f626806d380 0x7f626813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f626806d380 0x7f626813d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39466/0 (socket says 10.20.193.78:39466) 2026-02-17T06:04:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 1 -- 10.20.193.78:0/3521542990 learned_addr learned my addr 10.20.193.78:0/3521542990 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient(hunting): get_auth_request con 0x7f626806d380 auth_method 0 2026-02-17T06:04:07.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f625ffff640 1 --2- 10.20.193.78:0/3521542990 >> v2:10.20.193.115:3300/0 conn(0x7f626809c4a0 0x7f626813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f625ffff640 10 monclient(hunting): get_auth_request con 0x7f626809c4a0 auth_method 0 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f625ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f625ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f625ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient(hunting): handle_auth_done global_id 184473 payload 274 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 1 -- 10.20.193.78:0/3521542990 >> v2:10.20.193.115:3300/0 conn(0x7f626809c4a0 msgr2=0x7f626813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 1 --2- 10.20.193.78:0/3521542990 >> v2:10.20.193.115:3300/0 conn(0x7f626809c4a0 0x7f626813b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 1 -- 10.20.193.78:0/3521542990 >> v1:10.20.193.115:6789/0 conn(0x7f626806c890 legacy=0x7f626806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient: _finish_hunting 0 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 1 monclient: found mon.noname-a 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:07.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:07.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.920+0000 7f626d699640 1 -- 10.20.193.78:0/3521542990 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626809cf60 con 0x7f626806d380 2026-02-17T06:04:07.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.922+0000 7f626d699640 1 -- 10.20.193.78:0/3521542990 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626806d380 msgr2=0x7f626813d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:07.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.922+0000 7f626d699640 1 --2- 10.20.193.78:0/3521542990 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626806d380 0x7f626813d8f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f626806dcc0 tx=0x7f626402f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5888436751e2d3d4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:07.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.922+0000 7f625f7fe640 1 -- 10.20.193.78:0/3521542990 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626402fe50 con 0x7f626806d380 2026-02-17T06:04:07.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: dump: 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.939 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 5 monclient: authenticate success, global_id 184473 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 1 -- 10.20.193.78:0/3521542990 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62640373f0 con 0x7f626806d380 2026-02-17T06:04:07.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 1 -- 10.20.193.78:0/3521542990 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62640377c0 con 0x7f626806d380 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 4 set_mon_vals no callback set 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f625f7fe640 10 monclient: dump: 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.941 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 10 monclient: shutdown 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 1 -- 10.20.193.78:0/3521542990 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626806d380 msgr2=0x7f626813d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 1 --2- 10.20.193.78:0/3521542990 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626806d380 0x7f626813d8f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f626806dcc0 tx=0x7f626402f1e0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 1 -- 10.20.193.78:0/3521542990 shutdown_connections 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 1 --2- 10.20.193.78:0/3521542990 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626806d380 0x7f626813d8f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 1 --2- 10.20.193.78:0/3521542990 >> v2:10.20.193.115:3300/0 conn(0x7f626809c4a0 0x7f626813b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 1 -- 10.20.193.78:0/3521542990 >> 10.20.193.78:0/3521542990 conn(0x7f62680a6620 msgr2=0x7f62680a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 1 -- 10.20.193.78:0/3521542990 shutdown_connections 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 1 -- 10.20.193.78:0/3521542990 wait complete. 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 10 monclient: build_initial_monmap 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.923+0000 7f626f123640 10 monclient: monmap: 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:07.943 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 1 Processor -- start 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 1 -- start start 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient: init 2026-02-17T06:04:07.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f626806c890 0x7f62681ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient(hunting): picked mon.noname-c con 0x7f626806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f626806d380 0x7f6268132000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient(hunting): picked mon.noname-b con 0x7f626806d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626809c4a0 0x7f62681392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient(hunting): picked mon.noname-a con 0x7f626809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f626809bfa0 con 0x7f626809c4a0 2026-02-17T06:04:07.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f626809be20 con 0x7f626806d380 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f626813de30 con 0x7f626806c890 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626f123640 10 monclient(hunting): authenticate will time out at 1693.477430s 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626809c4a0 0x7f62681392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626ce98640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f626806c890 0x7f62681ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626809c4a0 0x7f62681392d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39480/0 (socket says 10.20.193.78:39480) 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 -- 10.20.193.78:0/2729348918 learned_addr learned my addr 10.20.193.78:0/2729348918 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient(hunting): get_auth_request con 0x7f626809c4a0 auth_method 0 2026-02-17T06:04:07.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626ce98640 10 monclient(hunting): get_auth_request con 0x7f626806c890 auth_method 0 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626ce98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626ce98640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626ce98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:07.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient(hunting): handle_auth_done global_id 184476 payload 824 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f626806c890 msgr2=0x7f62681ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f626806c890 0x7f62681ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f626806d380 msgr2=0x7f6268132000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f626806d380 0x7f6268132000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient: _finish_hunting 0 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 monclient: found mon.noname-a 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.924+0000 7f626d699640 1 -- 10.20.193.78:0/2729348918 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626809d810 con 0x7f626809c4a0 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f626d699640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626809c4a0 msgr2=0x7f62681392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:07.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f626d699640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626809c4a0 0x7f62681392d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6264039040 tx=0x7f6264004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264046020 con 0x7f626809c4a0 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: dump: 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.949 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: _finish_auth 0 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6264041450 con 0x7f626809c4a0 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f626f123640 5 monclient: authenticate success, global_id 184476 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f626f123640 10 monclient: _renew_subs 2026-02-17T06:04:07.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f626f123640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626806f830 con 0x7f626809c4a0 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264041840 con 0x7f626809c4a0 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625d7fa640 10 monclient: dump: 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.951 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.926+0000 7f625effd640 4 set_mon_vals no callback set 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f626f123640 10 monclient: _renew_subs 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f626f123640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626813bdc0 con 0x7f626809c4a0 2026-02-17T06:04:07.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f626f123640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f626f123640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6268132540 0x7f626813cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f626f123640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f626813ba80 con 0x7f6268132540 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f62640419e0 con 0x7f626809c4a0 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f625d7fa640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f623803df30 0x7f62380403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(646..646 src has 525..646) ==== 4795+0+0 (secure 0 0 0) 0x7f6264078290 con 0x7f626809c4a0 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f625ffff640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f623803df30 0x7f62380403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.927+0000 7f625ffff640 10 monclient: get_auth_request con 0x7f623803df30 auth_method 0 2026-02-17T06:04:07.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f625ffff640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f623803df30 msgr2=0x7f62380403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f625ffff640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f623803df30 msgr2=0x7f62380403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f625ffff640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f623803df30 0x7f62380403f0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f6250006fd0 tx=0x7f6250008040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6268132540 0x7f626813cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f625ffff640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f623803df30 0x7f62380403f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 10 monclient: get_auth_request con 0x7f6268132540 auth_method 0 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:07.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 10 monclient(hunting): handle_auth_done global_id 184479 payload 824 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.928+0000 7f626ce98640 1 -- 10.20.193.78:0/2729348918 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6268074980 con 0x7f6268132540 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f626ce98640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6268132540 msgr2=0x7f626813cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f626ce98640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6268132540 0x7f626813cbb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f625800f7a0 tx=0x7f625800fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258018030 con 0x7f6268132540 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 10 monclient: dump: 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.955 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f625800dbc0 con 0x7f6268132540 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6268132540 msgr2=0x7f626813cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6268132540 0x7f626813cbb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f625800f7a0 tx=0x7f625800fc70 comp rx=0 tx=0).stop 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6268132540 msgr2=0x7f626813cbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.930+0000 7f625d7fa640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6268132540 0x7f626813cbb0 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f625800f7a0 tx=0x7f625800fc70 comp rx=0 tx=0).stop 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626f123640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626f123640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626813d0f0 0x7f62681da640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:07.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626f123640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f626806cc70 con 0x7f626813d0f0 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626813d0f0 0x7f62681da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 10 monclient: get_auth_request con 0x7f626813d0f0 auth_method 0 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 10 monclient(hunting): handle_auth_done global_id 184482 payload 824 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.934+0000 7f626ce98640 1 -- 10.20.193.78:0/2729348918 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f626809fb30 con 0x7f626813d0f0 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f626ce98640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626813d0f0 msgr2=0x7f62681da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f626ce98640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626813d0f0 0x7f62681da640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6258018000 tx=0x7f6258003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258014470 con 0x7f626813d0f0 2026-02-17T06:04:07.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 10 monclient: dump: 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7f6258014470 con 0x7f626813d0f0 2026-02-17T06:04:07.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:04:07.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:07.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626813d0f0 msgr2=0x7f62681da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626813d0f0 0x7f62681da640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6258018000 tx=0x7f6258003860 comp rx=0 tx=0).stop 2026-02-17T06:04:07.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626813d0f0 msgr2=0x7f62681da640 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:07.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.936+0000 7f625d7fa640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626813d0f0 0x7f62681da640 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6258018000 tx=0x7f6258003860 comp rx=0 tx=0).stop 2026-02-17T06:04:07.959 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:04:07.959 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 212, 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 212716, 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:07.960 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:07.961 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:07.962 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:04:07.963 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:07.964 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.965 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:07.966 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.967 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:04:07.968 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:07.969 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:04:07.970 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f623803df30 msgr2=0x7f62380403f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f623803df30 0x7f62380403f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 10 monclient: shutdown 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626809c4a0 msgr2=0x7f62681392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 1 --2- 10.20.193.78:0/2729348918 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f626809c4a0 0x7f62681392d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6264039040 tx=0x7f6264004b40 comp rx=0 tx=0).stop 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626d699640 1 -- 10.20.193.78:0/2729348918 reap_dead start 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626d699640 1 -- 10.20.193.78:0/2729348918 reap_dead start 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 shutdown_connections 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 >> 10.20.193.78:0/2729348918 conn(0x7f62680a6620 msgr2=0x7f6268136190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 shutdown_connections 2026-02-17T06:04:07.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:07.937+0000 7f626f123640 1 -- 10.20.193.78:0/2729348918 wait complete. 2026-02-17T06:04:07.973 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:04:07.973 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:04:07.973 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3019: done (2 left) 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3020: copy_from oid 12 from oid 14 current snap is 314 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: finishing write tid 1 to trial11545420-37 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: finishing write tid 2 to trial11545420-37 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: finishing write tid 3 to trial11545420-37 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: finishing write tid 4 to trial11545420-37 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: finishing write tid 5 to trial11545420-37 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: oid 37 updating version 0 to 1460 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: oid 37 version 1460 is already newer than 1459 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: oid 37 version 1460 is already newer than 1458 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: oid 37 version 1460 is already newer than 1457 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1460 (ObjNum 1056 snap 314 seq_num 1056) dirty exists 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: left oid 37 (ObjNum 1056 snap 314 seq_num 1056) 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3017: done (2 left) 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3018: done (1 left) 2026-02-17T06:04:07.974 INFO:tasks.rados.rados.0.trial115.stdout:3021: copy_from oid 6 from oid 35 current snap is 314 2026-02-17T06:04:07.975 INFO:tasks.rados.rados.0.trial115.stdout:3022: read oid 34 snap -1 2026-02-17T06:04:07.975 INFO:tasks.rados.rados.0.trial115.stdout:3022: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:04:07.975 INFO:tasks.rados.rados.0.trial115.stdout:3023: snap_create 2026-02-17T06:04:07.975 INFO:tasks.rados.rados.0.trial115.stdout:3022: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:04:08.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.043+0000 7f251d522640 10 monclient: get_monmap_and_config 2026-02-17T06:04:08.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.043+0000 7f251d522640 10 monclient: build_initial_monmap 2026-02-17T06:04:08.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.043+0000 7f251d522640 10 monclient: monmap: 2026-02-17T06:04:08.046 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:08.046 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.046 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:08.043788+0000 2026-02-17T06:04:08.046 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:08.043788+0000 2026-02-17T06:04:08.046 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:08.046 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.043+0000 7f251d522640 1 Processor -- start 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.043+0000 7f251d522640 1 -- start start 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.043+0000 7f251d522640 10 monclient: init 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 0x7f251806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient(hunting): picked mon.noname-b con 0x7f251806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f251809c4a0 0x7f251806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient(hunting): picked mon.noname-a con 0x7f251809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2518134df0 0x7f2518135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient(hunting): picked mon.noname-c con 0x7f2518134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f25180a0880 con 0x7f251809c4a0 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f251806d6f0 con 0x7f2518134df0 2026-02-17T06:04:08.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f251806daf0 con 0x7f251806c890 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f251d522640 10 monclient(hunting): authenticate will time out at 1693.597311s 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f251809c4a0 0x7f251806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f251809c4a0 0x7f251806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39506/0 (socket says 10.20.193.78:39506) 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 0x7f251806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 -- 10.20.193.78:0/4039951797 learned_addr learned my addr 10.20.193.78:0/4039951797 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient(hunting): get_auth_request con 0x7f251809c4a0 auth_method 0 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 10 monclient(hunting): get_auth_request con 0x7f251806c890 auth_method 0 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 1 -- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 msgr2=0x7f251806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 1 -- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 msgr2=0x7f251806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:08.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 1 --2- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 0x7f251806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f2516ffd640 1 --2- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 0x7f251806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient(hunting): handle_auth_done global_id 184485 payload 274 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 -- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 msgr2=0x7f251806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 --2- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 0x7f251806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 -- 10.20.193.78:0/4039951797 >> v2:10.20.193.115:3300/0 conn(0x7f2518134df0 msgr2=0x7f2518135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25177fe640 1 --2- 10.20.193.78:0/4039951797 >> v2:10.20.193.115:3300/0 conn(0x7f2518134df0 0x7f2518135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 --2- 10.20.193.78:0/4039951797 >> v2:10.20.193.115:3300/0 conn(0x7f2518134df0 0x7f2518135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient: _finish_hunting 0 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 monclient: found mon.noname-a 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25167fc640 1 -- 10.20.193.78:0/4039951797 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251809cc80 con 0x7f251809c4a0 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25177fe640 10 monclient: get_auth_request con 0x7f2518134df0 auth_method 0 2026-02-17T06:04:08.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.044+0000 7f25177fe640 1 --2- 10.20.193.78:0/4039951797 >> v2:10.20.193.115:3300/0 conn(0x7f2518134df0 0x7f2518135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:04:08.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.046+0000 7f25167fc640 1 -- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 msgr2=0x7f251806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f25167fc640 1 --2- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f251806d1b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f251809c880 tx=0x7f250002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb9c5a833d8cab6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 1 -- 10.20.193.78:0/4039951797 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250003c070 con 0x7f251809c4a0 2026-02-17T06:04:08.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:08.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: dump: 2026-02-17T06:04:08.052 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.052 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: _finish_auth 0 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 1 -- 10.20.193.78:0/4039951797 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250002fca0 con 0x7f251809c4a0 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 5 monclient: authenticate success, global_id 184485 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 1 -- 10.20.193.78:0/4039951797 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500037660 con 0x7f251809c4a0 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f2515ffb640 10 monclient: dump: 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.053 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 4 set_mon_vals no callback set 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 10 monclient: shutdown 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 -- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 msgr2=0x7f251806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 --2- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f251806d1b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f251809c880 tx=0x7f250002ee60 comp rx=0 tx=0).stop 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 -- 10.20.193.78:0/4039951797 shutdown_connections 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 --2- 10.20.193.78:0/4039951797 >> v2:10.20.193.115:3300/0 conn(0x7f2518134df0 0x7f2518135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 --2- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f251806d1b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 --2- 10.20.193.78:0/4039951797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251806c890 0x7f251806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 -- 10.20.193.78:0/4039951797 >> 10.20.193.78:0/4039951797 conn(0x7f25180a6620 msgr2=0x7f25180a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 -- 10.20.193.78:0/4039951797 shutdown_connections 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.047+0000 7f251d522640 1 -- 10.20.193.78:0/4039951797 wait complete. 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient: build_initial_monmap 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient: monmap: 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:08.054 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 1 Processor -- start 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 1 -- start start 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient: init 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f251806c890 0x7f25181cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient(hunting): picked mon.noname-b con 0x7f251806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f25181cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient(hunting): picked mon.noname-a con 0x7f251809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2518134df0 0x7f251809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient(hunting): picked mon.noname-c con 0x7f2518134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f25180a01a0 con 0x7f251809c4a0 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f25180a0020 con 0x7f251806c890 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f251809be20 con 0x7f2518134df0 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f251d522640 10 monclient(hunting): authenticate will time out at 1693.601686s 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f25181cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f25181cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39516/0 (socket says 10.20.193.78:39516) 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 -- 10.20.193.78:0/4156135093 learned_addr learned my addr 10.20.193.78:0/4156135093 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient(hunting): get_auth_request con 0x7f251809c4a0 auth_method 0 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25177fe640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2518134df0 0x7f251809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25177fe640 10 monclient(hunting): get_auth_request con 0x7f2518134df0 auth_method 0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f2516ffd640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f251806c890 0x7f25181cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient(hunting): handle_auth_done global_id 184488 payload 824 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2518134df0 msgr2=0x7f251809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2518134df0 0x7f251809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f251806c890 msgr2=0x7f25181cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f251806c890 0x7f25181cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient: _finish_hunting 0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 monclient: found mon.noname-a 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f25167fc640 1 -- 10.20.193.78:0/4156135093 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25181335d0 con 0x7f251809c4a0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f2516ffd640 10 monclient: get_auth_request con 0x7f251806c890 auth_method 0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.048+0000 7f2516ffd640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f251806c890 0x7f25181cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f25167fc640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 msgr2=0x7f25181cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f25167fc640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f25181cf890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2500002410 tx=0x7f2500004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500047070 con 0x7f251809c4a0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: dump: 2026-02-17T06:04:08.056 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: _finish_auth 0 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250002fe50 con 0x7f251809c4a0 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250003c040 con 0x7f251809c4a0 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 10 monclient: dump: 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f25157fa640 4 set_mon_vals no callback set 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 5 monclient: authenticate success, global_id 184488 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 10 monclient: _renew_subs 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2518126370 con 0x7f251809c4a0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 10 monclient: _renew_subs 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25180725b0 con 0x7f251809c4a0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e090 0x7f251809e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2518074980 con 0x7f251809e090 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e090 0x7f251809e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 10 monclient: get_auth_request con 0x7f251809e090 auth_method 0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 10 monclient(hunting): handle_auth_done global_id 224114 payload 824 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f2516ffd640 1 -- 10.20.193.78:0/4156135093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25180a1610 con 0x7f251809e090 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.051+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f250003f9f0 con 0x7f251809c4a0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.052+0000 7f24f77fe640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f24e403deb0 0x7f24e4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.052+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(647..647 src has 525..647) ==== 4811+0+0 (secure 0 0 0) 0x7f2500077510 con 0x7f251809c4a0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.052+0000 7f25177fe640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f24e403deb0 0x7f24e4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.052+0000 7f25177fe640 10 monclient: get_auth_request con 0x7f24e403deb0 auth_method 0 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f2516ffd640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e090 msgr2=0x7f251809e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f2516ffd640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e090 0x7f251809e470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f250c0108d0 tx=0x7f250c010da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c00f040 con 0x7f251809e090 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 10 monclient: dump: 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f250c004020 con 0x7f251809e090 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e090 msgr2=0x7f251809e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e090 0x7f251809e470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f250c0108d0 tx=0x7f250c010da0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e090 msgr2=0x7f251809e470 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.054+0000 7f24f77fe640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e090 0x7f251809e470 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f250c0108d0 tx=0x7f250c010da0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.055+0000 7f25177fe640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f24e403deb0 0x7f24e4040370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f251809fbb0 tx=0x7f25080076d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f251d522640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f251d522640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e9b0 0x7f25181da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f251d522640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f25180744d0 con 0x7f251809e9b0 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e9b0 0x7f25181da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 10 monclient: get_auth_request con 0x7f251809e9b0 auth_method 0 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 10 monclient(hunting): handle_auth_done global_id 224117 payload 824 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.057+0000 7f2516ffd640 1 -- 10.20.193.78:0/4156135093 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f251809d760 con 0x7f251809e9b0 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f2516ffd640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e9b0 msgr2=0x7f25181da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f2516ffd640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e9b0 0x7f25181da560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f250c0108d0 tx=0x7f250c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c005e40 con 0x7f251809e9b0 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 10 monclient: dump: 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.061 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f250c005e40 con 0x7f251809e9b0 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e9b0 msgr2=0x7f25181da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e9b0 0x7f25181da560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f250c0108d0 tx=0x7f250c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e9b0 msgr2=0x7f25181da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f24f77fe640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f251809e9b0 0x7f25181da560 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f250c0108d0 tx=0x7f250c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 212, 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 6758, 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.062 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:08.063 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:08.064 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:08.065 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.066 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.067 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:08.068 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:08.068 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:08.068 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.068 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.068 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.068 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:04:08.068 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f24e403deb0 msgr2=0x7f24e4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f24e403deb0 0x7f24e4040370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f251809fbb0 tx=0x7f25080076d0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 10 monclient: shutdown 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 msgr2=0x7f25181cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f25177fe640 1 -- 10.20.193.78:0/4156135093 reap_dead start 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f25181cf890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2500002410 tx=0x7f2500004830 comp rx=0 tx=0).stop 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 shutdown_connections 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 --2- 10.20.193.78:0/4156135093 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f251809c4a0 0x7f25181cf890 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 >> 10.20.193.78:0/4156135093 conn(0x7f25180a6620 msgr2=0x7f2518136920 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 shutdown_connections 2026-02-17T06:04:08.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.060+0000 7f251d522640 1 -- 10.20.193.78:0/4156135093 wait complete. 2026-02-17T06:04:08.069 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:04:08.069 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:04:08.070 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:04:08.070 INFO:tasks.rados.rados.0.trial115.stdout:3020: finishing copy_from racing read to trial11545420-12 2026-02-17T06:04:08.070 INFO:tasks.rados.rados.0.trial115.stdout:3020: finishing copy_from to trial11545420-12 2026-02-17T06:04:08.070 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1451 (ObjNum 1049 snap 312 seq_num 1049) dirty exists 2026-02-17T06:04:08.070 INFO:tasks.rados.rados.0.trial115.stdout:3021: finishing copy_from racing read to trial11545420-6 2026-02-17T06:04:08.070 INFO:tasks.rados.rados.0.trial115.stdout:3021: finishing copy_from to trial11545420-6 2026-02-17T06:04:08.070 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 1479 (ObjNum 948 snap 276 seq_num 948) dirty exists 2026-02-17T06:04:08.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.137+0000 7fdb8130a640 10 monclient: get_monmap_and_config 2026-02-17T06:04:08.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.137+0000 7fdb8130a640 10 monclient: build_initial_monmap 2026-02-17T06:04:08.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.137+0000 7fdb8130a640 10 monclient: monmap: 2026-02-17T06:04:08.139 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:08.139 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.139 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:08.137787+0000 2026-02-17T06:04:08.139 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:08.137787+0000 2026-02-17T06:04:08.139 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.137+0000 7fdb8130a640 1 Processor -- start 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.137+0000 7fdb8130a640 1 -- start start 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.137+0000 7fdb8130a640 10 monclient: init 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdb7c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fdb7c06d380 0x7fdb7c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 10 monclient(hunting): picked mon.noname-c con 0x7fdb7c06d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdb7c13a240 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb7c074150 con 0x7fdb7c13a240 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb7c09ef70 con 0x7fdb7c06c890 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7c09f250 con 0x7fdb7c06d380 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb8130a640 10 monclient(hunting): authenticate will time out at 1693.691403s 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7b7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fdb7c13a240 0x7fdb7c13c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38802/0 (socket says 10.20.193.78:38802) 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7b7fe640 1 -- 10.20.193.78:0/750236858 learned_addr learned my addr 10.20.193.78:0/750236858 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/750236858 >> v2:10.20.193.115:3300/0 conn(0x7fdb7c06d380 0x7fdb7c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7a7fc640 10 monclient(hunting): get_auth_request con 0x7fdb7c06d380 auth_method 0 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.138+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb7b7fe640 1 -- 10.20.193.78:0/750236858 >> v1:10.20.193.78:6789/0 conn(0x7fdb7c13a240 legacy=0x7fdb7c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077328128 0 0) 0x7fdb7c074150 con 0x7fdb7c13a240 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): _negotiate 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): my global_id is 184491 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb68003620 con 0x7fdb7c13a240 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb7affd640 1 -- 10.20.193.78:0/750236858 >> v1:10.20.193.115:6789/0 conn(0x7fdb7c06c890 legacy=0x7fdb7c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 443253673 0 0) 0x7fdb68003620 con 0x7fdb7c13a240 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 >> v2:10.20.193.115:3300/0 conn(0x7fdb7c06d380 msgr2=0x7fdb7c0a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 --2- 10.20.193.78:0/750236858 >> v2:10.20.193.115:3300/0 conn(0x7fdb7c06d380 0x7fdb7c0a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 >> v1:10.20.193.115:6789/0 conn(0x7fdb7c06c890 legacy=0x7fdb7c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: _finish_hunting 0 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c09df40 con 0x7fdb7c13a240 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: _finish_auth 0 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fdb6c003530 con 0x7fdb7c13a240 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: dump: 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.142 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 >> v1:10.20.193.78:6789/0 conn(0x7fdb7c13a240 legacy=0x7fdb7c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb68004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): picked mon.a con 0x7fdb68003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb68004900 0x7fdb68004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): picked mon.b con 0x7fdb68004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb680054c0 0x7fdb6800dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): picked mon.c con 0x7fdb680054c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c09f250 con 0x7fdb68003c20 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6800e5f0 con 0x7fdb68004900 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdb6800e7f0 con 0x7fdb680054c0 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.140+0000 7fdb79ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb68004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient(hunting): get_auth_request con 0x7fdb68003c20 auth_method 0 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb680054c0 0x7fdb6800dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 10 monclient(hunting): get_auth_request con 0x7fdb680054c0 auth_method 0 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb68004900 0x7fdb68004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7a7fc640 10 monclient(hunting): get_auth_request con 0x7fdb68004900 auth_method 0 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:08.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient(hunting): handle_auth_done global_id 184491 payload 293 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 1 -- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb680054c0 msgr2=0x7fdb6800dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb680054c0 0x7fdb6800dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 1 -- 10.20.193.78:0/750236858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb68004900 msgr2=0x7fdb68004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb68004900 0x7fdb68004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient: _finish_hunting 0 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 1 monclient: found mon.a 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 1 -- 10.20.193.78:0/750236858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb68003620 con 0x7fdb68003c20 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient: _finish_auth 0 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 10 monclient: _check_auth_tickets 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb8130a640 5 monclient: authenticate success, global_id 184491 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb8130a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb680054c0 0x7fdb6800dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:04:08.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.141+0000 7fdb7b7fe640 1 -- 10.20.193.78:0/750236858 reap_dead start 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb7affd640 1 -- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 msgr2=0x7fdb68004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb7affd640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb68004040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdb700032a0 tx=0x7fdb7002efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fc4963947cd21e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70002fa0 con 0x7fdb68003c20 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 10 monclient: dump: 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.145 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb8130a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb70033030 con 0x7fdb68003c20 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb8130a640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 1 -- 10.20.193.78:0/750236858 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70005d20 con 0x7fdb68003c20 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb8130a640 4 set_mon_vals no callback set 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.143+0000 7fdb79ffb640 10 monclient: dump: 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 10 monclient: shutdown 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 -- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 msgr2=0x7fdb68004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb68004040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdb700032a0 tx=0x7fdb7002efd0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 -- 10.20.193.78:0/750236858 shutdown_connections 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 --2- 10.20.193.78:0/750236858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb68004040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 -- 10.20.193.78:0/750236858 >> 10.20.193.78:0/750236858 conn(0x7fdb7c0a6620 msgr2=0x7fdb7c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 -- 10.20.193.78:0/750236858 shutdown_connections 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 -- 10.20.193.78:0/750236858 wait complete. 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 10 monclient: build_initial_monmap 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 10 monclient: monmap: 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:08.146 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 Processor -- start 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 1 -- start start 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.144+0000 7fdb8130a640 10 monclient: init 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb7c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient(hunting): picked mon.noname-a con 0x7fdb68003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c06eff0 0x7fdb7c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient(hunting): picked mon.noname-b con 0x7fdb7c06eff0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb7c1cf330 0x7fdb7c1d1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient(hunting): picked mon.noname-c con 0x7fdb7c1cf330 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c0a0370 con 0x7fdb68003c20 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c0a01f0 con 0x7fdb7c06eff0 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7c0a04f0 con 0x7fdb7c1cf330 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb8130a640 10 monclient(hunting): authenticate will time out at 1693.698183s 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb7c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb7c06eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39534/0 (socket says 10.20.193.78:39534) 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 -- 10.20.193.78:0/2792735379 learned_addr learned my addr 10.20.193.78:0/2792735379 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient(hunting): get_auth_request con 0x7fdb68003c20 auth_method 0 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb7c1cf330 0x7fdb7c1d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 10 monclient(hunting): get_auth_request con 0x7fdb7c1cf330 auth_method 0 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c06eff0 0x7fdb7c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb7c1cf330 msgr2=0x7fdb7c1d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:04:08.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb7c1cf330 msgr2=0x7fdb7c1d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb7c1cf330 0x7fdb7c1d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb7c1cf330 0x7fdb7c1d1720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient(hunting): handle_auth_done global_id 184494 payload 824 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb7c1cf330 msgr2=0x7fdb7c1d1720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdb7c1cf330 0x7fdb7c1d1720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c06eff0 msgr2=0x7fdb7c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c06eff0 0x7fdb7c06f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient: _finish_hunting 0 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 monclient: found mon.noname-a 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7affd640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c0a2250 con 0x7fdb68003c20 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7a7fc640 10 monclient: get_auth_request con 0x7fdb7c06eff0 auth_method 0 2026-02-17T06:04:08.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.145+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c06eff0 0x7fdb7c06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb7affd640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 msgr2=0x7fdb7c06eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb7affd640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb7c06eab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb70002410 tx=0x7fdb70002cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70002fa0 con 0x7fdb68003c20 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: dump: 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7002fc10 con 0x7fdb68003c20 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70007070 con 0x7fdb68003c20 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb797fa640 4 set_mon_vals no callback set 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb5b7fe640 10 monclient: dump: 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb8130a640 5 monclient: authenticate success, global_id 184494 2026-02-17T06:04:08.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb8130a640 10 monclient: _renew_subs 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb8130a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c09d810 con 0x7fdb68003c20 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb8130a640 10 monclient: _renew_subs 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb8130a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.147+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c072130 con 0x7fdb68003c20 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb8130a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb8130a640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 0x7fdb7c063880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb8130a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c072aa0 con 0x7fdb7c138970 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fdb7004b030 con 0x7fdb68003c20 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 0x7fdb7c063880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 10 monclient: get_auth_request con 0x7fdb7c138970 auth_method 0 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb5b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb4803df00 0x7fdb480403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(647..647 src has 525..647) ==== 4811+0+0 (secure 0 0 0) 0x7fdb70076520 con 0x7fdb68003c20 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_done global_id 214189 payload 824 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7a7fc640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb7c06cc70 con 0x7fdb7c138970 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb4803df00 0x7fdb480403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.148+0000 7fdb7b7fe640 10 monclient: get_auth_request con 0x7fdb4803df00 auth_method 0 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 msgr2=0x7fdb7c063880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 0x7fdb7c063880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb64012870 tx=0x7fdb64012d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 msgr2=0x7fdb7c063880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:04:08.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 msgr2=0x7fdb7c063880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:04:08.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 0x7fdb7c063880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb64012870 tx=0x7fdb64012d40 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:08.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 0x7fdb7c063880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb64012870 tx=0x7fdb64012d40 comp rx=0 tx=0).stop 2026-02-17T06:04:08.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 msgr2=0x7fdb7c063880 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:08.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb5b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 0x7fdb7c063880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb5b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb48043f90 0x7fdb48046380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 msgr2=0x7fdb7c063880 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb5b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138970 0x7fdb7c063880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb5b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb480469c0 con 0x7fdb48043f90 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb48043f90 0x7fdb48046380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 10 monclient: get_auth_request con 0x7fdb48043f90 auth_method 0 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb4803df00 0x7fdb480403c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0a0760 tx=0x7fdb6c00a310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_done global_id 214192 payload 824 2026-02-17T06:04:08.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.151+0000 7fdb7a7fc640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb70008e60 con 0x7fdb48043f90 2026-02-17T06:04:08.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb7a7fc640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb48043f90 msgr2=0x7fdb48046380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb48043f90 0x7fdb48046380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdb64012d80 tx=0x7fdb64004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6402c020 con 0x7fdb48043f90 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 10 monclient: dump: 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.154 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fdb64028a50 con 0x7fdb48043f90 2026-02-17T06:04:08.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:08.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:08.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb48043f90 msgr2=0x7fdb48046380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb48043f90 0x7fdb48046380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdb64012d80 tx=0x7fdb64004830 comp rx=0 tx=0).stop 2026-02-17T06:04:08.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb48043f90 msgr2=0x7fdb48046380 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:08.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.154+0000 7fdb5b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb48043f90 0x7fdb48046380 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdb64012d80 tx=0x7fdb64004830 comp rx=0 tx=0).stop 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.157+0000 7fdb8130a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.157+0000 7fdb8130a640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138d50 0x7fdb7c1d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.157+0000 7fdb8130a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.157+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c070720 con 0x7fdb7c138d50 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138d50 0x7fdb7c1d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 10 monclient: get_auth_request con 0x7fdb7c138d50 auth_method 0 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 10 monclient(hunting): handle_auth_done global_id 214195 payload 824 2026-02-17T06:04:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.158+0000 7fdb7a7fc640 1 -- 10.20.193.78:0/2792735379 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdb7c0744d0 con 0x7fdb7c138d50 2026-02-17T06:04:08.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb7a7fc640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138d50 msgr2=0x7fdb7c1d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb7a7fc640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138d50 0x7fdb7c1d5c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb48046b40 tx=0x7fdb64019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64039070 con 0x7fdb7c138d50 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 10 monclient: dump: 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7fdb64037070 con 0x7fdb7c138d50 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138d50 msgr2=0x7fdb7c1d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138d50 0x7fdb7c1d5c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb48046b40 tx=0x7fdb64019000 comp rx=0 tx=0).stop 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138d50 msgr2=0x7fdb7c1d5c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb5b7fe640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdb7c138d50 0x7fdb7c1d5c80 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb48046b40 tx=0x7fdb64019000 comp rx=0 tx=0).stop 2026-02-17T06:04:08.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb7b7fe640 1 -- 10.20.193.78:0/2792735379 reap_dead start 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 212, 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 51007, 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:08.162 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:08.163 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:04:08.164 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:08.165 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.166 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:08.167 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:08.168 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.169 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.170 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:04:08.171 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb4803df00 msgr2=0x7fdb480403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb8130a640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb4803df00 0x7fdb480403c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0a0760 tx=0x7fdb6c00a310 comp rx=0 tx=0).stop 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb8130a640 10 monclient: shutdown 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb8130a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 msgr2=0x7fdb7c06eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.160+0000 7fdb8130a640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb7c06eab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb70002410 tx=0x7fdb70002cc0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.161+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 shutdown_connections 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.161+0000 7fdb8130a640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdb4803df00 0x7fdb480403c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.161+0000 7fdb8130a640 1 --2- 10.20.193.78:0/2792735379 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdb68003c20 0x7fdb7c06eab0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.161+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 >> 10.20.193.78:0/2792735379 conn(0x7fdb7c0a6620 msgr2=0x7fdb7c1352c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.161+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 shutdown_connections 2026-02-17T06:04:08.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.161+0000 7fdb8130a640 1 -- 10.20.193.78:0/2792735379 wait complete. 2026-02-17T06:04:08.174 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:04:08.174 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:04:08.174 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:04:08.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient: get_monmap_and_config 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient: build_initial_monmap 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient: monmap: 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:08.243318+0000 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:08.243318+0000 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:08.243 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 1 Processor -- start 2026-02-17T06:04:08.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 1 -- start start 2026-02-17T06:04:08.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient: init 2026-02-17T06:04:08.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc806c890 0x7f1bc806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient(hunting): picked mon.noname-b con 0x7f1bc806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1bc809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1bc806d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc8135c10 con 0x7f1bc809c4a0 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc806daf0 con 0x7f1bc806d1b0 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc80a0880 con 0x7f1bc806c890 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bcd888640 10 monclient(hunting): authenticate will time out at 1693.796920s 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.243+0000 7f1bc6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc806c890 0x7f1bc806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc67fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f1bc809c4a0 0x7f1bc8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38806/0 (socket says 10.20.193.78:38806) 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc67fc640 1 -- 10.20.193.78:0/3131490030 learned_addr learned my addr 10.20.193.78:0/3131490030 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient(hunting): get_auth_request con 0x7f1bc806c890 auth_method 0 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient(hunting): handle_auth_done global_id 224123 payload 274 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 1 -- 10.20.193.78:0/3131490030 >> v1:10.20.193.115:6789/0 conn(0x7f1bc806d1b0 legacy=0x7f1bc8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 1 -- 10.20.193.78:0/3131490030 >> v1:10.20.193.78:6789/0 conn(0x7f1bc809c4a0 legacy=0x7f1bc8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:08.245 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:04:08.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 1 monclient: found mon.noname-b 2026-02-17T06:04:08.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:08.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:04:08.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.244+0000 7f1bc6ffd640 1 -- 10.20.193.78:0/3131490030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc809cc80 con 0x7f1bc806c890 2026-02-17T06:04:08.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc6ffd640 1 -- 10.20.193.78:0/3131490030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc806c890 msgr2=0x7f1bc806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc6ffd640 1 --2- 10.20.193.78:0/3131490030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc806c890 0x7f1bc806cc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0098d0 tx=0x7f1bbc02ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72f1b018e725c06 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 1 -- 10.20.193.78:0/3131490030 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc03c070 con 0x7f1bc806c890 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: dump: 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: _finish_auth 0 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 1 -- 10.20.193.78:0/3131490030 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc02fcc0 con 0x7f1bc806c890 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bcd888640 5 monclient: authenticate success, global_id 224123 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bcd888640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 1 -- 10.20.193.78:0/3131490030 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc037660 con 0x7f1bc806c890 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bcd888640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:08.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bcd888640 4 set_mon_vals no callback set 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.246+0000 7f1bc5ffb640 10 monclient: dump: 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient: shutdown 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- 10.20.193.78:0/3131490030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc806c890 msgr2=0x7f1bc806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 --2- 10.20.193.78:0/3131490030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc806c890 0x7f1bc806cc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0098d0 tx=0x7f1bbc02ed60 comp rx=0 tx=0).stop 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- 10.20.193.78:0/3131490030 shutdown_connections 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 --2- 10.20.193.78:0/3131490030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc806c890 0x7f1bc806cc70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- 10.20.193.78:0/3131490030 >> 10.20.193.78:0/3131490030 conn(0x7f1bc80a6620 msgr2=0x7f1bc80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- 10.20.193.78:0/3131490030 shutdown_connections 2026-02-17T06:04:08.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- 10.20.193.78:0/3131490030 wait complete. 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient: build_initial_monmap 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient: monmap: 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 Processor -- start 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- start start 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient: init 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1bc806c890 0x7f1bc81341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient(hunting): picked mon.noname-b con 0x7f1bc806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1bc806d1b0 0x7f1bc8134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient(hunting): picked mon.noname-a con 0x7f1bc806d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc809c4a0 0x7f1bc809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient(hunting): picked mon.noname-c con 0x7f1bc809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc80a0020 con 0x7f1bc806d1b0 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc806d600 con 0x7f1bc806c890 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc80a01a0 con 0x7f1bc809c4a0 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bcd888640 10 monclient(hunting): authenticate will time out at 1693.800914s 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bc67fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1bc806d1b0 0x7f1bc8134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bc77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc809c4a0 0x7f1bc809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bc67fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1bc806d1b0 0x7f1bc8134730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39550/0 (socket says 10.20.193.78:39550) 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bc67fc640 1 -- 10.20.193.78:0/3821333365 learned_addr learned my addr 10.20.193.78:0/3821333365 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bc67fc640 10 monclient(hunting): get_auth_request con 0x7f1bc806d1b0 auth_method 0 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.247+0000 7f1bc67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc77fe640 10 monclient(hunting): get_auth_request con 0x7f1bc809c4a0 auth_method 0 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc6ffd640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1bc806c890 0x7f1bc81341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc6ffd640 10 monclient(hunting): get_auth_request con 0x7f1bc806c890 auth_method 0 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 10 monclient(hunting): handle_auth_done global_id 184497 payload 824 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 1 -- 10.20.193.78:0/3821333365 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc809c4a0 msgr2=0x7f1bc809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc809c4a0 0x7f1bc809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 1 -- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1bc806c890 msgr2=0x7f1bc81341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1bc806c890 0x7f1bc81341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 10 monclient: _finish_hunting 0 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 1 monclient: found mon.noname-a 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.248+0000 7f1bc67fc640 1 -- 10.20.193.78:0/3821333365 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc809d810 con 0x7f1bc806d1b0 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bc67fc640 1 -- 10.20.193.78:0/3821333365 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1bc806d1b0 msgr2=0x7f1bc8134730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bc67fc640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1bc806d1b0 0x7f1bc8134730 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000c750 tx=0x7f1bb000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 1 -- 10.20.193.78:0/3821333365 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0005de0 con 0x7f1bc806d1b0 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:08.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: dump: 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: _finish_auth 0 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 1 -- 10.20.193.78:0/3821333365 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb0007070 con 0x7f1bc806d1b0 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 1 -- 10.20.193.78:0/3821333365 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0007440 con 0x7f1bc806d1b0 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bcd888640 5 monclient: authenticate success, global_id 184497 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bcd888640 10 monclient: _renew_subs 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bcd888640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc80a26d0 con 0x7f1bc806d1b0 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bc57fa640 4 set_mon_vals no callback set 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1b9ffff640 10 monclient: dump: 2026-02-17T06:04:08.254 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bcd888640 10 monclient: _renew_subs 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bcd888640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.250+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc8127930 con 0x7f1bc806d1b0 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1bcd888640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1bcd888640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1bcd888640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1b9ffff640 1 -- 10.20.193.78:0/3821333365 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1bb0007660 con 0x7f1bc806d1b0 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc8068410 con 0x7f1bc806d1b0 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1b9ffff640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1b9803deb0 0x7f1b98040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1b9ffff640 1 -- 10.20.193.78:0/3821333365 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(647..647 src has 525..647) ==== 4811+0+0 (secure 0 0 0) 0x7f1bb00525d0 con 0x7f1bc806d1b0 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1bc6ffd640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1b9803deb0 0x7f1b98040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:08.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.251+0000 7f1bc6ffd640 10 monclient: get_auth_request con 0x7f1b9803deb0 auth_method 0 2026-02-17T06:04:08.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.253+0000 7f1b9ffff640 1 -- 10.20.193.78:0/3821333365 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1bc8068410 con 0x7f1bc806d1b0 2026-02-17T06:04:08.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.253+0000 7f1b9ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:08.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.253+0000 7f1b9ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:08.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.253+0000 7f1bc6ffd640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1b9803deb0 0x7f1b98040370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80a7600 tx=0x7f1bbc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.368+0000 7f1bcd888640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:04:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.368+0000 7f1bcd888640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:04:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.368+0000 7f1bcd888640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.368+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f1bc806cc70 con 0x7f1bc806d1b0 2026-02-17T06:04:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.368+0000 7f1b9ffff640 1 -- 10.20.193.78:0/3821333365 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f1bc809d810 con 0x7f1bc806d1b0 2026-02-17T06:04:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.368+0000 7f1b9ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:04:08.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.368+0000 7f1b9ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=11,logm=47} crc {auth=1752140563,config=692619927,health=3015364183,kv=2785245193,logm=2348347115}) 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=31,config=2,health=10,kv=11,logm=46} crc {auth=1286426633,config=692619927,health=1700149591,kv=2510975679,logm=2756504464}) 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=11,logm=47} crc {auth=1752140563,config=692619927,health=3015364183,kv=2785245193,logm=2348347115}) 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=28,config=2,health=11,kv=11,logm=48} crc {auth=2344858262,config=692619927,health=151810877,kv=745643123,logm=4132219969}) 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3131746522}) 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420254643}) 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3131746522}) 2026-02-17T06:04:08.369 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.369+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=785580838}) 2026-02-17T06:04:08.369 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1b9803deb0 msgr2=0x7f1b98040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1b9803deb0 0x7f1b98040370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80a7600 tx=0x7f1bbc033000 comp rx=0 tx=0).stop 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 10 monclient: shutdown 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1bc806d1b0 msgr2=0x7f1bc8134730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1bc806d1b0 0x7f1bc8134730 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000c750 tx=0x7f1bb000cc20 comp rx=0 tx=0).stop 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 shutdown_connections 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1bc809c4a0 0x7f1bc809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1bc806d1b0 0x7f1bc8134730 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1b9803deb0 0x7f1b98040370 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 --2- 10.20.193.78:0/3821333365 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1bc806c890 0x7f1bc81341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 >> 10.20.193.78:0/3821333365 conn(0x7f1bc80a6620 msgr2=0x7f1bc8135460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 shutdown_connections 2026-02-17T06:04:08.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:08.370+0000 7f1bcd888640 1 -- 10.20.193.78:0/3821333365 wait complete. 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2642380248}) 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3425953519}) 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2642380248}) 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3337280909}) 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931464173}) 2026-02-17T06:04:08.370 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284369050}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931464173}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.370+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2906768802}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1716571440}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4090400555}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1716571440}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2960707944}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660586510}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=271309037}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3660586510}) 2026-02-17T06:04:08.371 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.371+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3184342230}) 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1149341416}) 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3577296712}) 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1149341416}) 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3688444478}) 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4206390914}) 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099515295}) 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4206390914}) 2026-02-17T06:04:08.372 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.372+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=684845126}) 2026-02-17T06:04:08.378 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1177236532}) 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1348861576}) 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1177236532}) 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2272031607}) 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2044978146}) 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3510315707}) 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2044978146}) 2026-02-17T06:04:08.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.382+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=347163471}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2112584412}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=442428567}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2112584412}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1384713598}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2391304853}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1555045095}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2391304853}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3276266007}) 2026-02-17T06:04:08.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=514855854}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158077276}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=514855854}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.383+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3501750927}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2688259203}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3138769267}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2688259203}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2047376003}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1218128156}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1424245024}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1218128156}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.384+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=96975520}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1204005880}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1473014062}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1204005880}) 2026-02-17T06:04:08.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1032739143}) 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3319556083}) 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2048372720}) 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3319556083}) 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3288009196}) 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1828382051}) 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1824662216}) 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1828382051}) 2026-02-17T06:04:08.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.385+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2129253463}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111211733}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2990357738}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111211733}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=805434269}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705482182}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1898470562}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=705482182}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1998994854}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803123736}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2926055816}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803123736}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.386+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4011013748}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662202593}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3236741185}) 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662202593}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2343664073}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=740634630}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1589513632}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=740634630}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2795774901}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=812342558}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2795149850}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=812342558}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=706747703}) 2026-02-17T06:04:08.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1072509149}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2441891576}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1072509149}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.387+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=607335017}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=47589402}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1462306507}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=47589402}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=197045628}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131789655}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3438898852}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131789655}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2440427615}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927577550}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3225920834}) 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.388 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927577550}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.388+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=194086624}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3807229413}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=542339217}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3807229413}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=281398784}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1783608741}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3794296399}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1783608741}) 2026-02-17T06:04:08.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.389+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=722317992}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=43673672}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=79595391}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=43673672}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3299020629}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033331417}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1370557813}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033331417}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3230423272}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=935859728}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4044568666}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=935859728}) 2026-02-17T06:04:08.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.390+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2305503185}) 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523351124}) 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2545093707}) 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523351124}) 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873545805}) 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172303302}) 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2549308553}) 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172303302}) 2026-02-17T06:04:08.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1012076930}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478553889}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.391+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2944725792}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478553889}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853059369}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464664943}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1845741759}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464664943}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1690659705}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4229349384}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131074476}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4229349384}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2905460900}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.392+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2971801239}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029328125}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2971801239}) 2026-02-17T06:04:08.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1807930017}) 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021970590}) 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424115853}) 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021970590}) 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=867121944}) 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=917056744}) 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=46665233}) 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=917056744}) 2026-02-17T06:04:08.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.393+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558160049}) 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117327284}) 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2794779324}) 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117327284}) 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1060169319}) 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=332935164}) 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3382775964}) 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=332935164}) 2026-02-17T06:04:08.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.394+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2958787801}) 2026-02-17T06:04:08.580 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.579+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.580 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.580+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2392734345}) 2026-02-17T06:04:08.580 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.580+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=403110129}) 2026-02-17T06:04:08.580 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.580+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.581 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.580+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2392734345}) 2026-02-17T06:04:08.581 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.580+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2288858489}) 2026-02-17T06:04:08.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.666+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.666+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707079388}) 2026-02-17T06:04:08.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.666+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1302288010}) 2026-02-17T06:04:08.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.666+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.666+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707079388}) 2026-02-17T06:04:08.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.666+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1644017087}) 2026-02-17T06:04:08.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.669+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.669+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2889517661}) 2026-02-17T06:04:08.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.669+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=871747788}) 2026-02-17T06:04:08.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.669+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.669+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2889517661}) 2026-02-17T06:04:08.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.669+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4039671580}) 2026-02-17T06:04:08.783 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.783+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.784 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.783+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1525924006}) 2026-02-17T06:04:08.784 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.783+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=49621252}) 2026-02-17T06:04:08.784 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.783+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.784 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.783+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1525924006}) 2026-02-17T06:04:08.784 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.783+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2582889735}) 2026-02-17T06:04:08.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3443360461}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4028263815}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3443360461}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3943166331}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=514736897}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3573443501}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=514736897}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191851908}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3218712700}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3937974322}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3218712700}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.887+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3681774338}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4062046743}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4192297772}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4062046743}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1597025720}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1233081417}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3795125257}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1233081417}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2773154916}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1175181589}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538744113}) 2026-02-17T06:04:08.897 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1175181589}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.888+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1849149219}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3322818282}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=756391516}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3322818282}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1993917281}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2209092894}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3867020045}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2209092894}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2618981080}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1955242966}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=454405613}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1955242966}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317510242}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2023425805}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4166351503}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2023425805}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.889+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2954131378}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1750899083}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2236620639}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1750899083}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2295860544}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4156644103}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1688114916}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4156644103}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=245939982}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1665128928}) 2026-02-17T06:04:08.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83737986}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1665128928}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.890+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729107358}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396136476}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=428137649}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396136476}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2827080856}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4239391891}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2095301121}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4239391891}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=682082122}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269689606}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2656650353}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269689606}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.891+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3473865651}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1405323429}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907888503}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1405323429}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1410481376}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1363349836}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270780268}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1363349836}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3612281566}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3654903459}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2468565270}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3654903459}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2638776490}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3992553751}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=413854925}) 2026-02-17T06:04:08.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3992553751}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.892+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1803899690}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2487948147}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3935782642}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2487948147}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620376876}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=28} crc {logm=152785227,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3454339573,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2240493509,osd_pg_creating=3755683804,osd_snap=3648688058}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=28} crc {logm=4256509528,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1067198419,mon_config_key=1253202242,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=3755683804,osd_snap=2914814039}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=28,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=28} crc {logm=152785227,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3454339573,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2240493509,osd_pg_creating=3755683804,osd_snap=3648688058}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.893+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=29,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=25} crc {logm=388205906,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=2746852068,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2240493509,osd_pg_creating=3755683804,osd_snap=362353736}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=918023202,osdmap=2919882698}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=77,osdmap=23} crc {osd_snap=2208886095,osdmap=2973211984}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=918023202,osdmap=2919882698}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=82,osdmap=18} crc {osd_snap=721534649,osdmap=3326454446}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=727528020}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1837418330}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=727528020}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.894+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1340805481}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.897+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.897+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=75} crc {osdmap=3292460364}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.897+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=74} crc {osdmap=2508894365}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.897+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.897+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=75} crc {osdmap=3292460364}) 2026-02-17T06:04:08.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:04:08.897+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=75} crc {osdmap=4224154610}) 2026-02-17T06:04:08.939 INFO:tasks.rados.rados.0.trial115.stdout:3020: done (3 left) 2026-02-17T06:04:08.939 INFO:tasks.rados.rados.0.trial115.stdout:3021: done (2 left) 2026-02-17T06:04:08.940 INFO:tasks.rados.rados.0.trial115.stdout:3022: done (1 left) 2026-02-17T06:04:08.940 INFO:tasks.rados.rados.0.trial115.stdout:3023: done (0 left) 2026-02-17T06:04:08.940 INFO:tasks.rados.rados.0.trial115.stdout:3024: snap_remove snap 313 2026-02-17T06:04:09.379 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:04:09.379 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.448+0000 7f5217e7d640 10 monclient: get_monmap_and_config 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.448+0000 7f5217e7d640 10 monclient: build_initial_monmap 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.448+0000 7f5217e7d640 10 monclient: monmap: 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:09.448963+0000 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:09.448963+0000 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:09.451 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 1 Processor -- start 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 1 -- start start 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient: init 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f521006c890 0x7f521006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient(hunting): picked mon.noname-a con 0x7f521006c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f521006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient(hunting): picked mon.noname-b con 0x7f521009d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f521009bf40 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5210073600 con 0x7f521009bf40 2026-02-17T06:04:09.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f521009ca50 con 0x7f521006c890 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f521006d6f0 con 0x7f521009d430 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5217e7d640 10 monclient(hunting): authenticate will time out at 1695.002512s 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f521006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f521006c890 0x7f521006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f521006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59444/0 (socket says 10.20.193.78:59444) 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 1 -- 10.20.193.78:0/2587289232 learned_addr learned my addr 10.20.193.78:0/2587289232 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient(hunting): get_auth_request con 0x7f521009d430 auth_method 0 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 10 monclient(hunting): get_auth_request con 0x7f521006c890 auth_method 0 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient(hunting): handle_auth_done global_id 224129 payload 274 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 1 -- 10.20.193.78:0/2587289232 >> v2:10.20.193.78:3300/0 conn(0x7f521006c890 msgr2=0x7f521006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 1 --2- 10.20.193.78:0/2587289232 >> v2:10.20.193.78:3300/0 conn(0x7f521006c890 0x7f521006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 1 -- 10.20.193.78:0/2587289232 >> v1:10.20.193.115:6789/0 conn(0x7f521009bf40 legacy=0x7f5210134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient: _finish_hunting 0 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 1 monclient: found mon.noname-b 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f52153f1640 1 -- 10.20.193.78:0/2587289232 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521006dab0 con 0x7f521009d430 2026-02-17T06:04:09.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.449+0000 7f5215bf2640 1 --2- 10.20.193.78:0/2587289232 >> v2:10.20.193.78:3300/0 conn(0x7f521006c890 0x7f521006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:04:09.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f52153f1640 1 -- 10.20.193.78:0/2587289232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 msgr2=0x7f521006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f52153f1640 1 --2- 10.20.193.78:0/2587289232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f521006d1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f521006a480 tx=0x7f520402ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a4d4c7c287d34b7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 1 -- 10.20.193.78:0/2587289232 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520403c070 con 0x7f521009d430 2026-02-17T06:04:09.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: dump: 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: _finish_auth 0 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: _check_auth_tickets 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 5 monclient: authenticate success, global_id 224129 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 1 -- 10.20.193.78:0/2587289232 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520402fcd0 con 0x7f521009d430 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 1 -- 10.20.193.78:0/2587289232 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204037660 con 0x7f521009d430 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 4 set_mon_vals no callback set 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5214bf0640 10 monclient: dump: 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.460 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 10 monclient: shutdown 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 1 -- 10.20.193.78:0/2587289232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 msgr2=0x7f521006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 1 --2- 10.20.193.78:0/2587289232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f521006d1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f521006a480 tx=0x7f520402ee90 comp rx=0 tx=0).stop 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 1 -- 10.20.193.78:0/2587289232 shutdown_connections 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 1 --2- 10.20.193.78:0/2587289232 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f521006d1b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 1 --2- 10.20.193.78:0/2587289232 >> v2:10.20.193.78:3300/0 conn(0x7f521006c890 0x7f521006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 1 -- 10.20.193.78:0/2587289232 >> 10.20.193.78:0/2587289232 conn(0x7f52100a6620 msgr2=0x7f52100a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 1 -- 10.20.193.78:0/2587289232 shutdown_connections 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.452+0000 7f5217e7d640 1 -- 10.20.193.78:0/2587289232 wait complete. 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient: build_initial_monmap 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient: monmap: 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:09.461 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 1 Processor -- start 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 1 -- start start 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient: init 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f521006c890 0x7f52101cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient(hunting): picked mon.noname-b con 0x7f521006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f521009bf40 0x7f52101cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient(hunting): picked mon.noname-a con 0x7f521009bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f5210121590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient(hunting): picked mon.noname-c con 0x7f521009d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52100733c0 con 0x7f521009bf40 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5210073240 con 0x7f521006c890 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5210135500 con 0x7f521009d430 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f5217e7d640 10 monclient(hunting): authenticate will time out at 1695.006605s 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f521009bf40 0x7f52101cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f521009bf40 0x7f52101cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39558/0 (socket says 10.20.193.78:39558) 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 -- 10.20.193.78:0/3104062056 learned_addr learned my addr 10.20.193.78:0/3104062056 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient(hunting): get_auth_request con 0x7f521009bf40 auth_method 0 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52163f3640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f5210121590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52163f3640 10 monclient(hunting): get_auth_request con 0x7f521009d430 auth_method 0 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52163f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52163f3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52163f3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52163f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52163f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52163f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient(hunting): handle_auth_done global_id 184503 payload 824 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 -- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 msgr2=0x7f5210121590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f5210121590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 -- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f521006c890 msgr2=0x7f52101cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f521006c890 0x7f52101cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient: _finish_hunting 0 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 monclient: found mon.noname-a 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.453+0000 7f52153f1640 1 -- 10.20.193.78:0/3104062056 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52101254d0 con 0x7f521009bf40 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.455+0000 7f52153f1640 1 -- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f521009bf40 msgr2=0x7f52101cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.455+0000 7f52153f1640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f521009bf40 0x7f52101cfcf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5204009a70 tx=0x7f52040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 1 -- 10.20.193.78:0/3104062056 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204044070 con 0x7f521009bf40 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:09.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: dump: 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: _finish_auth 0 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 1 -- 10.20.193.78:0/3104062056 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5204040610 con 0x7f521009bf40 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 5 monclient: authenticate success, global_id 184503 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 10 monclient: _renew_subs 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5210134e80 con 0x7f521009bf40 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 1 -- 10.20.193.78:0/3104062056 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520403c040 con 0x7f521009bf40 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fffff640 4 set_mon_vals no callback set 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 10 monclient: dump: 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.464 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 10 monclient: _renew_subs 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521009d810 con 0x7f521009bf40 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 1 -- 10.20.193.78:0/3104062056 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f520403f050 con 0x7f521009bf40 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5210074980 con 0x7f521009bf40 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51e003deb0 0x7f51e0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.456+0000 7f51fe7fc640 1 -- 10.20.193.78:0/3104062056 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(648..648 src has 525..648) ==== 4783+0+0 (secure 0 0 0) 0x7f5204077490 con 0x7f521009bf40 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.457+0000 7f5215bf2640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51e003deb0 0x7f51e0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.457+0000 7f5215bf2640 10 monclient: get_auth_request con 0x7f51e003deb0 auth_method 0 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.459+0000 7f51fe7fc640 1 -- 10.20.193.78:0/3104062056 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f520402fc60 con 0x7f521009bf40 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.459+0000 7f51fe7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.459+0000 7f51fe7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:09.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.459+0000 7f5215bf2640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51e003deb0 0x7f51e0040370 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5200006f80 tx=0x7f5200006ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.571+0000 7f5217e7d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:04:09.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.571+0000 7f5217e7d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:09.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.571+0000 7f5217e7d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.571+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5210135930 con 0x7f521009bf40 2026-02-17T06:04:09.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.571+0000 7f51fe7fc640 1 -- 10.20.193.78:0/3104062056 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f5204075e90 con 0x7f521009bf40 2026-02-17T06:04:09.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.571+0000 7f51fe7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:09.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.571+0000 7f51fe7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:09.571 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":212,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51e003deb0 msgr2=0x7f51e0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51e003deb0 0x7f51e0040370 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5200006f80 tx=0x7f5200006ed0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 10 monclient: shutdown 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f521009bf40 msgr2=0x7f52101cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f521009bf40 0x7f52101cfcf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5204009a70 tx=0x7f52040047c0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 shutdown_connections 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f521009d430 0x7f5210121590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f521009bf40 0x7f52101cfcf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51e003deb0 0x7f51e0040370 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 --2- 10.20.193.78:0/3104062056 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f521006c890 0x7f52101cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 >> 10.20.193.78:0/3104062056 conn(0x7f52100a6620 msgr2=0x7f5210136d00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 shutdown_connections 2026-02-17T06:04:09.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.572+0000 7f5217e7d640 1 -- 10.20.193.78:0/3104062056 wait complete. 2026-02-17T06:04:09.579 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:04:09.580 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:04:09.580 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:04:09.580 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:04:09.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.647+0000 7f45a9fa9640 10 monclient: get_monmap_and_config 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.647+0000 7f45a9fa9640 10 monclient: build_initial_monmap 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.647+0000 7f45a9fa9640 10 monclient: monmap: 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:09.648162+0000 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:09.648162+0000 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:09.648 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 1 Processor -- start 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 1 -- start start 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient: init 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f45a4135f20 0x7f45a4136300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient(hunting): picked mon.noname-a con 0x7f45a4135f20 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f45a4136840 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a406cfa0 0x7f45a413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient(hunting): picked mon.noname-b con 0x7f45a406cfa0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45a409d140 con 0x7f45a4136840 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f45a409cb10 con 0x7f45a4135f20 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f45a4135c60 con 0x7f45a406cfa0 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a9fa9640 10 monclient(hunting): authenticate will time out at 1695.201744s 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f45a4135f20 0x7f45a4136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f45a4135f20 0x7f45a4136300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39568/0 (socket says 10.20.193.78:39568) 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a97a8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a406cfa0 0x7f45a413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 1 -- 10.20.193.78:0/1861330121 learned_addr learned my addr 10.20.193.78:0/1861330121 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 10 monclient(hunting): get_auth_request con 0x7f45a4135f20 auth_method 0 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a97a8640 10 monclient(hunting): get_auth_request con 0x7f45a406cfa0 auth_method 0 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a97a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a97a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a97a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a97a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a97a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.648+0000 7f45a97a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_done global_id 184506 payload 274 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 1 -- 10.20.193.78:0/1861330121 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a406cfa0 msgr2=0x7f45a413dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 1 --2- 10.20.193.78:0/1861330121 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a406cfa0 0x7f45a413dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 1 -- 10.20.193.78:0/1861330121 >> v1:10.20.193.115:6789/0 conn(0x7f45a4136840 legacy=0x7f45a406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 10 monclient: _finish_hunting 0 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 1 monclient: found mon.noname-a 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.649+0000 7f45a8fa7640 1 -- 10.20.193.78:0/1861330121 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a409c330 con 0x7f45a4135f20 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f45a8fa7640 1 -- 10.20.193.78:0/1861330121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a4135f20 msgr2=0x7f45a4136300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f45a8fa7640 1 --2- 10.20.193.78:0/1861330121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a4135f20 0x7f45a4136300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4594009970 tx=0x7f459402ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c46934e29fa0ff server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 1 -- 10.20.193.78:0/1861330121 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459403c070 con 0x7f45a4135f20 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: dump: 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.652 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 1 -- 10.20.193.78:0/1861330121 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f459402fcd0 con 0x7f45a4135f20 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f45a9fa9640 5 monclient: authenticate success, global_id 184506 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f45a9fa9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 1 -- 10.20.193.78:0/1861330121 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594037660 con 0x7f45a4135f20 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f45a9fa9640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f45a9fa9640 4 set_mon_vals no callback set 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.651+0000 7f459b7fe640 10 monclient: dump: 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient: shutdown 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- 10.20.193.78:0/1861330121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a4135f20 msgr2=0x7f45a4136300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 --2- 10.20.193.78:0/1861330121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a4135f20 0x7f45a4136300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4594009970 tx=0x7f459402ee90 comp rx=0 tx=0).stop 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- 10.20.193.78:0/1861330121 shutdown_connections 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 --2- 10.20.193.78:0/1861330121 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a406cfa0 0x7f45a413dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 --2- 10.20.193.78:0/1861330121 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a4135f20 0x7f45a4136300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- 10.20.193.78:0/1861330121 >> 10.20.193.78:0/1861330121 conn(0x7f45a40a65f0 msgr2=0x7f45a40a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- 10.20.193.78:0/1861330121 shutdown_connections 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- 10.20.193.78:0/1861330121 wait complete. 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient: build_initial_monmap 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient: monmap: 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:09.653 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 Processor -- start 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- start start 2026-02-17T06:04:09.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient: init 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a406cfa0 0x7f45a41cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient(hunting): picked mon.noname-a con 0x7f45a406cfa0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a4135f20 0x7f45a4135000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient(hunting): picked mon.noname-c con 0x7f45a4135f20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f45a4136840 0x7f45a4135540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient(hunting): picked mon.noname-b con 0x7f45a4136840 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f45a409ff90 con 0x7f45a406cfa0 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f45a409fe10 con 0x7f45a4136840 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f45a40a02f0 con 0x7f45a4135f20 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.652+0000 7f45a9fa9640 10 monclient(hunting): authenticate will time out at 1695.205977s 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a4135f20 0x7f45a4135000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f45a8fa7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a406cfa0 0x7f45a41cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a4135f20 0x7f45a4135000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59460/0 (socket says 10.20.193.78:59460) 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 -- 10.20.193.78:0/3173694454 learned_addr learned my addr 10.20.193.78:0/3173694454 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient(hunting): get_auth_request con 0x7f45a4135f20 auth_method 0 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f45a8fa7640 10 monclient(hunting): get_auth_request con 0x7f45a406cfa0 auth_method 0 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f45a8fa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f45a8fa7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f45a8fa7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient(hunting): handle_auth_done global_id 224138 payload 824 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f45a4136840 msgr2=0x7f45a4135540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f45a4136840 0x7f45a4135540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a406cfa0 msgr2=0x7f45a41cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a406cfa0 0x7f45a41cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 monclient: found mon.noname-c 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.653+0000 7f459bfff640 1 -- 10.20.193.78:0/3173694454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a41249e0 con 0x7f45a4135f20 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f459bfff640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a4135f20 msgr2=0x7f45a4135000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f459bfff640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a4135f20 0x7f45a4135000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f458c00c600 tx=0x7f458c00cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c012a40 con 0x7f45a4135f20 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: dump: 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: _finish_auth 0 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: _check_auth_tickets 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458c017070 con 0x7f45a4135f20 2026-02-17T06:04:09.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c012e20 con 0x7f45a4135f20 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f459affd640 4 set_mon_vals no callback set 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45997fa640 10 monclient: dump: 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45a9fa9640 5 monclient: authenticate success, global_id 224138 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45a9fa9640 10 monclient: _renew_subs 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45a9fa9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a409f5a0 con 0x7f45a4135f20 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45a9fa9640 10 monclient: _renew_subs 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45a9fa9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:04:09.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a4124420 con 0x7f45a4135f20 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.655+0000 7f45a9fa9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a9fa9640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409d940 0x7f45a409dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a9fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f45a41cf5f0 con 0x7f45a409d940 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409d940 0x7f45a409dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 10 monclient: get_auth_request con 0x7f45a409d940 auth_method 0 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f458c01c460 con 0x7f45a4135f20 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_done global_id 184512 payload 824 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a8fa7640 1 -- 10.20.193.78:0/3173694454 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45a40728b0 con 0x7f45a409d940 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45997fa640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f457403deb0 0x7f4574040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(648..648 src has 525..648) ==== 4783+0+0 (secure 0 0 0) 0x7f458c052690 con 0x7f45a4135f20 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a97a8640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f457403deb0 0x7f4574040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.656+0000 7f45a97a8640 10 monclient: get_auth_request con 0x7f457403deb0 auth_method 0 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.658+0000 7f45a8fa7640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409d940 msgr2=0x7f45a409dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.658+0000 7f45a8fa7640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409d940 0x7f45a409dd20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f45a40697e0 tx=0x7f45940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.659+0000 7f45a97a8640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f457403deb0 0x7f4574040370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f45a4123f50 tx=0x7f45a000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594037e60 con 0x7f45a409d940 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 10 monclient: dump: 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f459403d040 con 0x7f45a409d940 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409d940 msgr2=0x7f45a409dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409d940 0x7f45a409dd20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f45a40697e0 tx=0x7f45940047c0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409d940 msgr2=0x7f45a409dd20 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:09.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.660+0000 7f45997fa640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409d940 0x7f45a409dd20 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f45a40697e0 tx=0x7f45940047c0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a9fa9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a9fa9640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409e260 0x7f45a41da400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a9fa9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f45a40a6dd0 con 0x7f45a409e260 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409e260 0x7f45a41da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 10 monclient: get_auth_request con 0x7f45a409e260 auth_method 0 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 10 monclient(hunting): handle_auth_done global_id 184515 payload 824 2026-02-17T06:04:09.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.663+0000 7f45a8fa7640 1 -- 10.20.193.78:0/3173694454 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f45a406d430 con 0x7f45a409e260 2026-02-17T06:04:09.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.665+0000 7f45a8fa7640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409e260 msgr2=0x7f45a41da400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.665+0000 7f45a8fa7640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409e260 0x7f45a41da400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4594002410 tx=0x7f4594009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594037e60 con 0x7f45a409e260 2026-02-17T06:04:09.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 10 monclient: dump: 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7f45940045a0 con 0x7f45a409e260 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409e260 msgr2=0x7f45a41da400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409e260 0x7f45a41da400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4594002410 tx=0x7f4594009340 comp rx=0 tx=0).stop 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409e260 msgr2=0x7f45a41da400 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45997fa640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f45a409e260 0x7f45a41da400 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4594002410 tx=0x7f4594009340 comp rx=0 tx=0).stop 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:09.666 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 212, 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 214446, 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.667 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.668 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.669 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.670 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:04:09.671 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f457403deb0 msgr2=0x7f4574040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f457403deb0 0x7f4574040370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f45a4123f50 tx=0x7f45a000b040 comp rx=0 tx=0).stop 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 10 monclient: shutdown 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a4135f20 msgr2=0x7f45a4135000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a4135f20 0x7f45a4135000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f458c00c600 tx=0x7f458c00cad0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a97a8640 1 -- 10.20.193.78:0/3173694454 reap_dead start 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 shutdown_connections 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 --2- 10.20.193.78:0/3173694454 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f45a4135f20 0x7f45a4135000 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 >> 10.20.193.78:0/3173694454 conn(0x7f45a40a65f0 msgr2=0x7f45a40a23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 shutdown_connections 2026-02-17T06:04:09.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.666+0000 7f45a9fa9640 1 -- 10.20.193.78:0/3173694454 wait complete. 2026-02-17T06:04:09.673 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:04:09.673 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:04:09.673 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.739+0000 7f3301ffb640 10 monclient: get_monmap_and_config 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.739+0000 7f3301ffb640 10 monclient: build_initial_monmap 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.739+0000 7f3301ffb640 10 monclient: monmap: 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:09.740192+0000 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:09.740192+0000 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 1 Processor -- start 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 1 -- start start 2026-02-17T06:04:09.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient: init 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f32fc06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f32fc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f32fc06d380 0x7f32fc13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f32fc06d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32fc06daf0 con 0x7f32fc06c890 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f32fc0a0880 con 0x7f32fc06d380 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f32fc09cc80 con 0x7f32fc09c4a0 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f3301ffb640 10 monclient(hunting): authenticate will time out at 1695.293720s 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f32fc06d380 0x7f32fc13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f32fc06d380 0x7f32fc13d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39598/0 (socket says 10.20.193.78:39598) 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59468/0 (socket says 10.20.193.78:59468) 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 1 -- 10.20.193.78:0/471202894 learned_addr learned my addr 10.20.193.78:0/471202894 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fb7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f32fc06c890 0x7f32fc06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38820/0 (socket says 10.20.193.78:38820) 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 10 monclient(hunting): get_auth_request con 0x7f32fc09c4a0 auth_method 0 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 10 monclient(hunting): get_auth_request con 0x7f32fc06d380 auth_method 0 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 10 monclient(hunting): handle_auth_done global_id 224141 payload 274 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 1 -- 10.20.193.78:0/471202894 >> v2:10.20.193.78:3300/0 conn(0x7f32fc06d380 msgr2=0x7f32fc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.740+0000 7f32faffd640 1 --2- 10.20.193.78:0/471202894 >> v2:10.20.193.78:3300/0 conn(0x7f32fc06d380 0x7f32fc13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.741+0000 7f32faffd640 1 -- 10.20.193.78:0/471202894 >> v1:10.20.193.78:6789/0 conn(0x7f32fc06c890 legacy=0x7f32fc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.741+0000 7f32faffd640 10 monclient: _finish_hunting 0 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.741+0000 7f32faffd640 1 monclient: found mon.noname-b 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.741+0000 7f32faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.741+0000 7f32faffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.741+0000 7f32faffd640 1 -- 10.20.193.78:0/471202894 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc09cf60 con 0x7f32fc09c4a0 2026-02-17T06:04:09.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32faffd640 1 -- 10.20.193.78:0/471202894 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 msgr2=0x7f32fc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32faffd640 1 --2- 10.20.193.78:0/471202894 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc13b3a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009970 tx=0x7f32ec02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a547f8769af52c7b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 1 -- 10.20.193.78:0/471202894 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec03c070 con 0x7f32fc09c4a0 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: dump: 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: _finish_auth 0 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:09.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 5 monclient: authenticate success, global_id 224141 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 1 -- 10.20.193.78:0/471202894 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ec02fcd0 con 0x7f32fc09c4a0 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 1 -- 10.20.193.78:0/471202894 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec037660 con 0x7f32fc09c4a0 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 4 set_mon_vals no callback set 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f32fa7fc640 10 monclient: dump: 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 10 monclient: shutdown 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 1 -- 10.20.193.78:0/471202894 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 msgr2=0x7f32fc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 1 --2- 10.20.193.78:0/471202894 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc13b3a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009970 tx=0x7f32ec02ee90 comp rx=0 tx=0).stop 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 1 -- 10.20.193.78:0/471202894 shutdown_connections 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 1 --2- 10.20.193.78:0/471202894 >> v2:10.20.193.78:3300/0 conn(0x7f32fc06d380 0x7f32fc13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 1 --2- 10.20.193.78:0/471202894 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc13b3a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 1 -- 10.20.193.78:0/471202894 >> 10.20.193.78:0/471202894 conn(0x7f32fc0a6620 msgr2=0x7f32fc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 1 -- 10.20.193.78:0/471202894 shutdown_connections 2026-02-17T06:04:09.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.743+0000 7f3301ffb640 1 -- 10.20.193.78:0/471202894 wait complete. 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient: build_initial_monmap 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient: monmap: 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 1 Processor -- start 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 1 -- start start 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient: init 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f32fc06c890 0x7f32fc072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f32fc06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32fc06d380 0x7f32fc133270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f32fc06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:09.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc1337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f32fc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc09bfa0 con 0x7f32fc06d380 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc09be20 con 0x7f32fc06c890 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f32fc13de30 con 0x7f32fc09c4a0 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f3301ffb640 10 monclient(hunting): authenticate will time out at 1695.297848s 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc1337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32faffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32fc06d380 0x7f32fc133270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc1337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59482/0 (socket says 10.20.193.78:59482) 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 1 -- 10.20.193.78:0/3832177641 learned_addr learned my addr 10.20.193.78:0/3832177641 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 10 monclient(hunting): get_auth_request con 0x7f32fc09c4a0 auth_method 0 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32faffd640 10 monclient(hunting): get_auth_request con 0x7f32fc06d380 auth_method 0 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32faffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.744+0000 7f32faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 10 monclient(hunting): handle_auth_done global_id 224144 payload 824 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f32fc06c890 msgr2=0x7f32fc072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f32fc06c890 0x7f32fc072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32fc06d380 msgr2=0x7f32fc133270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f32fc06d380 0x7f32fc133270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 10 monclient: _finish_hunting 0 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 1 monclient: found mon.noname-c 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:04:09.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.745+0000 7f32fbfff640 1 -- 10.20.193.78:0/3832177641 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc11f660 con 0x7f32fc09c4a0 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fbfff640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 msgr2=0x7f32fc1337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fbfff640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc1337b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f32f000efc0 tx=0x7f32f000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0018020 con 0x7f32fc09c4a0 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: dump: 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: _finish_auth 0 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: _check_auth_tickets 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f0014680 con 0x7f32fc09c4a0 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0014a70 con 0x7f32fc09c4a0 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32f9ffb640 4 set_mon_vals no callback set 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 10 monclient: dump: 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.747 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 5 monclient: authenticate success, global_id 224144 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 10 monclient: _renew_subs 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc09e4e0 con 0x7f32fc09c4a0 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 10 monclient: _renew_subs 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc09d810 con 0x7f32fc09c4a0 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d430 0x7f32fc134ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f32fc074980 con 0x7f32fc11d430 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fb7fe640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d430 0x7f32fc134ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fb7fe640 10 monclient: get_auth_request con 0x7f32fc11d430 auth_method 0 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f32f0014c10 con 0x7f32fc09c4a0 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.747+0000 7f32fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.748+0000 7f32fb7fe640 10 monclient(hunting): handle_auth_done global_id 224147 payload 824 2026-02-17T06:04:09.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.748+0000 7f32fb7fe640 1 -- 10.20.193.78:0/3832177641 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32fc06cc70 con 0x7f32fc11d430 2026-02-17T06:04:09.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.748+0000 7f32d7fff640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32cc03dee0 0x7f32cc0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.748+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(648..648 src has 525..648) ==== 4783+0+0 (secure 0 0 0) 0x7f32f0053460 con 0x7f32fc09c4a0 2026-02-17T06:04:09.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.748+0000 7f32faffd640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32cc03dee0 0x7f32cc0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.748+0000 7f32faffd640 10 monclient: get_auth_request con 0x7f32cc03dee0 auth_method 0 2026-02-17T06:04:09.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32fb7fe640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d430 msgr2=0x7f32fc134ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32fb7fe640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d430 0x7f32fc134ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c3d0 tx=0x7f32e800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8017070 con 0x7f32fc11d430 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 10 monclient: dump: 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f32e8007070 con 0x7f32fc11d430 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d430 msgr2=0x7f32fc134ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d430 0x7f32fc134ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c3d0 tx=0x7f32e800c8a0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d430 msgr2=0x7f32fc134ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32d7fff640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d430 0x7f32fc134ff0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c3d0 tx=0x7f32e800c8a0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.750+0000 7f32faffd640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32cc03dee0 0x7f32cc0403a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f32ec036000 tx=0x7f32ec039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f3301ffb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f3301ffb640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d810 0x7f32fc1da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f3301ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f32fc073260 con 0x7f32fc11d810 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d810 0x7f32fc1da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 10 monclient: get_auth_request con 0x7f32fc11d810 auth_method 0 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 10 monclient(hunting): handle_auth_done global_id 224150 payload 824 2026-02-17T06:04:09.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.753+0000 7f32fb7fe640 1 -- 10.20.193.78:0/3832177641 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f32fc11fce0 con 0x7f32fc11d810 2026-02-17T06:04:09.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32fb7fe640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d810 msgr2=0x7f32fc1da2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32fb7fe640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d810 0x7f32fc1da2d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32e8014000 tx=0x7f32e8004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8017040 con 0x7f32fc11d810 2026-02-17T06:04:09.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 10 monclient: dump: 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f32e8012400 con 0x7f32fc11d810 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d810 msgr2=0x7f32fc1da2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d810 0x7f32fc1da2d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32e8014000 tx=0x7f32e8004a30 comp rx=0 tx=0).stop 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d810 msgr2=0x7f32fc1da2d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32d7fff640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc11d810 0x7f32fc1da2d0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32e8014000 tx=0x7f32e8004a30 comp rx=0 tx=0).stop 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 212, 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 8454, 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:04:09.756 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:04:09.757 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:04:09.758 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.759 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:04:09.760 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32cc03dee0 msgr2=0x7f32cc0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f32cc03dee0 0x7f32cc0403a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f32ec036000 tx=0x7f32ec039040 comp rx=0 tx=0).stop 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 10 monclient: shutdown 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 msgr2=0x7f32fc1337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 1 --2- 10.20.193.78:0/3832177641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f32fc09c4a0 0x7f32fc1337b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f32f000efc0 tx=0x7f32f000c490 comp rx=0 tx=0).stop 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32fbfff640 1 -- 10.20.193.78:0/3832177641 reap_dead start 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f32fbfff640 1 -- 10.20.193.78:0/3832177641 reap_dead start 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 shutdown_connections 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 >> 10.20.193.78:0/3832177641 conn(0x7f32fc0a6620 msgr2=0x7f32fc134170 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 shutdown_connections 2026-02-17T06:04:09.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.756+0000 7f3301ffb640 1 -- 10.20.193.78:0/3832177641 wait complete. 2026-02-17T06:04:09.762 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:04:09.763 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:04:09.763 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.830+0000 7f3dcf2d3640 10 monclient: get_monmap_and_config 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.830+0000 7f3dcf2d3640 10 monclient: build_initial_monmap 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.830+0000 7f3dcf2d3640 10 monclient: monmap: 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:09.830994+0000 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:09.830994+0000 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:09.832 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 1 Processor -- start 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 1 -- start start 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient: init 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3dc806c890 0x7f3dc806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient(hunting): picked mon.noname-a con 0x7f3dc806c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3dc809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 0x7f3dc8135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient(hunting): picked mon.noname-b con 0x7f3dc806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc806daf0 con 0x7f3dc809c4a0 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f3dc80a0880 con 0x7f3dc806c890 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc809cc80 con 0x7f3dc806d1b0 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcf2d3640 10 monclient(hunting): authenticate will time out at 1695.384526s 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3dc806c890 0x7f3dc806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcc847640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f3dc809c4a0 0x7f3dc8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38836/0 (socket says 10.20.193.78:38836) 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3dc806c890 0x7f3dc806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39624/0 (socket says 10.20.193.78:39624) 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcc847640 1 -- 10.20.193.78:0/1542964504 learned_addr learned my addr 10.20.193.78:0/1542964504 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd849640 1 --2- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 0x7f3dc8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient(hunting): get_auth_request con 0x7f3dc806c890 auth_method 0 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd849640 10 monclient(hunting): get_auth_request con 0x7f3dc806d1b0 auth_method 0 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd849640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd849640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd849640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd849640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd849640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd849640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient(hunting): handle_auth_done global_id 184524 payload 274 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 1 -- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 msgr2=0x7f3dc8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 1 --2- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 0x7f3dc8135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 1 -- 10.20.193.78:0/1542964504 >> v1:10.20.193.78:6789/0 conn(0x7f3dc809c4a0 legacy=0x7f3dc8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient: _finish_hunting 0 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 1 monclient: found mon.noname-a 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.831+0000 7f3dcd048640 1 -- 10.20.193.78:0/1542964504 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc809cf60 con 0x7f3dc806c890 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcd048640 1 -- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 msgr2=0x7f3dc806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcd048640 1 --2- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 0x7f3dc806cc70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3dc806dcc0 tx=0x7f3db802ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab13739367c69fcf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 1 -- 10.20.193.78:0/1542964504 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db803c070 con 0x7f3dc806c890 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: dump: 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: _finish_auth 0 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: _check_auth_tickets 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:09.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 1 -- 10.20.193.78:0/1542964504 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db802fbf0 con 0x7f3dc806c890 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 5 monclient: authenticate success, global_id 184524 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 4 set_mon_vals no callback set 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 1 -- 10.20.193.78:0/1542964504 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8037440 con 0x7f3dc806c890 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3db7fff640 10 monclient: dump: 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 10 monclient: shutdown 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 msgr2=0x7f3dc806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 1 --2- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 0x7f3dc806cc70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3dc806dcc0 tx=0x7f3db802ed70 comp rx=0 tx=0).stop 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/1542964504 shutdown_connections 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 1 --2- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 0x7f3dc8135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 1 --2- 10.20.193.78:0/1542964504 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 0x7f3dc806cc70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/1542964504 >> 10.20.193.78:0/1542964504 conn(0x7f3dc80a6620 msgr2=0x7f3dc80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/1542964504 shutdown_connections 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.834+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/1542964504 wait complete. 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient: build_initial_monmap 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient: monmap: 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 1 Processor -- start 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 1 -- start start 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient: init 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 0x7f3dc8134050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient(hunting): picked mon.noname-a con 0x7f3dc806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 0x7f3dc8134590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient(hunting): picked mon.noname-c con 0x7f3dc806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc809c4a0 0x7f3dc8123b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient(hunting): picked mon.noname-b con 0x7f3dc809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc80a01a0 con 0x7f3dc806c890 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc80a0020 con 0x7f3dc809c4a0 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc809be20 con 0x7f3dc806d1b0 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcf2d3640 10 monclient(hunting): authenticate will time out at 1695.388796s 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 0x7f3dc8134050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 0x7f3dc8134050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39640/0 (socket says 10.20.193.78:39640) 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 -- 10.20.193.78:0/2963466749 learned_addr learned my addr 10.20.193.78:0/2963466749 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient(hunting): get_auth_request con 0x7f3dc806c890 auth_method 0 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcc847640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 0x7f3dc8134590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcc847640 10 monclient(hunting): get_auth_request con 0x7f3dc806d1b0 auth_method 0 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcc847640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcc847640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcc847640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient(hunting): handle_auth_done global_id 184527 payload 824 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 msgr2=0x7f3dc8134590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3dc806d1b0 0x7f3dc8134590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc809c4a0 msgr2=0x7f3dc8123b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc809c4a0 0x7f3dc8123b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient: _finish_hunting 0 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 monclient: found mon.noname-a 2026-02-17T06:04:09.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.835+0000 7f3dcd048640 1 -- 10.20.193.78:0/2963466749 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81277b0 con 0x7f3dc806c890 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcd048640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 msgr2=0x7f3dc8134050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcd048640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 0x7f3dc8134050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3db8002410 tx=0x7f3db80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db803c040 con 0x7f3dc806c890 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: dump: 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.840 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: _finish_auth 0 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db8047070 con 0x7f3dc806c890 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 5 monclient: authenticate success, global_id 184527 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 10 monclient: _renew_subs 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc8127a90 con 0x7f3dc806c890 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8040a60 con 0x7f3dc806c890 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 10 monclient: dump: 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db77fe640 4 set_mon_vals no callback set 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 10 monclient: _renew_subs 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc80a21e0 con 0x7f3dc806c890 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc81cf530 0x7f3dc80709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8069a00 con 0x7f3dc81cf530 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcc847640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc81cf530 0x7f3dc80709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcc847640 10 monclient: get_auth_request con 0x7f3dc81cf530 auth_method 0 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3db803f030 con 0x7f3dc806c890 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcc847640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcc847640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.838+0000 7f3dcc847640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3db5ffb640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3d9803dee0 0x7f3d980403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(648..648 src has 525..648) ==== 4783+0+0 (secure 0 0 0) 0x7f3db8077420 con 0x7f3dc806c890 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3dcd849640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3d9803dee0 0x7f3d980403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3dcc847640 10 monclient(hunting): handle_auth_done global_id 214201 payload 824 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3dcc847640 1 -- 10.20.193.78:0/2963466749 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3dc8074980 con 0x7f3dc81cf530 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.839+0000 7f3dcd849640 10 monclient: get_auth_request con 0x7f3d9803dee0 auth_method 0 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.841+0000 7f3dcc847640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc81cf530 msgr2=0x7f3dc80709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.841+0000 7f3dcc847640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc81cf530 0x7f3dc80709c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8126290 tx=0x7f3dbc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.841+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00cbc0 con 0x7f3dc81cf530 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.841+0000 7f3db5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.841+0000 7f3db5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.841+0000 7f3db5ffb640 10 monclient: dump: 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.842 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.842+0000 7f3dcd849640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3d9803dee0 0x7f3d980403a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009fd0 tx=0x7f3dc400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.842+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f3dbc019070 con 0x7f3dc81cf530 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.842+0000 7f3db5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.842+0000 7f3db5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.842+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc81cf530 msgr2=0x7f3dc80709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.842+0000 7f3db5ffb640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc81cf530 0x7f3dc80709c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8126290 tx=0x7f3dbc00ef10 comp rx=0 tx=0).stop 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.842+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc81cf530 msgr2=0x7f3dc80709c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:09.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.842+0000 7f3db5ffb640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc81cf530 0x7f3dc80709c0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8126290 tx=0x7f3dbc00ef10 comp rx=0 tx=0).stop 2026-02-17T06:04:09.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcf2d3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:04:09.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcf2d3640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc8070f00 0x7f3dc81da650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:09.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcf2d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:09.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8135ad0 con 0x7f3dc8070f00 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc8070f00 0x7f3dc81da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 10 monclient: get_auth_request con 0x7f3dc8070f00 auth_method 0 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 10 monclient(hunting): handle_auth_done global_id 214204 payload 824 2026-02-17T06:04:09.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.845+0000 7f3dcc847640 1 -- 10.20.193.78:0/2963466749 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3dc8125c00 con 0x7f3dc8070f00 2026-02-17T06:04:09.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcc847640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc8070f00 msgr2=0x7f3dc81da650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:09.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcc847640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc8070f00 0x7f3dc81da650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8126290 tx=0x7f3dbc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc01e230 con 0x7f3dc8070f00 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 10 monclient: dump: 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f3dbc01e230 con 0x7f3dc8070f00 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc8070f00 msgr2=0x7f3dc81da650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc8070f00 0x7f3dc81da650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8126290 tx=0x7f3dbc004a40 comp rx=0 tx=0).stop 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc8070f00 msgr2=0x7f3dc81da650 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3db5ffb640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3dc8070f00 0x7f3dc81da650 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8126290 tx=0x7f3dbc004a40 comp rx=0 tx=0).stop 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 212, 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 52695, 2026-02-17T06:04:09.848 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:04:09.849 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.850 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.851 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:04:09.852 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3d9803dee0 msgr2=0x7f3d980403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3d9803dee0 0x7f3d980403a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009fd0 tx=0x7f3dc400b040 comp rx=0 tx=0).stop 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 10 monclient: shutdown 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 msgr2=0x7f3dc8134050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 1 --2- 10.20.193.78:0/2963466749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3dc806c890 0x7f3dc8134050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3db8002410 tx=0x7f3db80047c0 comp rx=0 tx=0).stop 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcd849640 1 -- 10.20.193.78:0/2963466749 reap_dead start 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcd849640 1 -- 10.20.193.78:0/2963466749 reap_dead start 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 shutdown_connections 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 >> 10.20.193.78:0/2963466749 conn(0x7f3dc80a6620 msgr2=0x7f3dc8072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 shutdown_connections 2026-02-17T06:04:09.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:09.848+0000 7f3dcf2d3640 1 -- 10.20.193.78:0/2963466749 wait complete. 2026-02-17T06:04:09.854 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:04:09.854 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-17T06:04:09.854 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2026-02-17T06:04:09.854 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-17T06:04:09.854 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T06:04:09.854 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-17T06:04:09.854 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:04:09.863 INFO:tasks.ceph.mon.c:Stopped 2026-02-17T06:04:09.863 INFO:tasks.mon_thrash:Sending STOP to mon b 2026-02-17T06:04:09.863 INFO:tasks.ceph.mon.b:Sent signal 19 2026-02-17T06:04:09.863 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T06:04:24.863 INFO:tasks.mon_thrash:Sending CONT to mon b 2026-02-17T06:04:24.864 INFO:tasks.ceph.mon.b:Sent signal 18 2026-02-17T06:04:24.864 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:04:24.865 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:04:24.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.933+0000 7fc4eca98640 10 monclient: get_monmap_and_config 2026-02-17T06:04:24.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.933+0000 7fc4eca98640 10 monclient: build_initial_monmap 2026-02-17T06:04:24.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.933+0000 7fc4eca98640 10 monclient: monmap: 2026-02-17T06:04:24.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:24.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:24.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:24.934015+0000 2026-02-17T06:04:24.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:24.934015+0000 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 1 Processor -- start 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 1 -- start start 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient: init 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:24.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 0x7fc4e806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient(hunting): picked mon.noname-b con 0x7fc4e806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc4e809c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc4e806d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e8135c10 con 0x7fc4e809c4a0 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e806daf0 con 0x7fc4e806d1b0 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e80a0880 con 0x7fc4e806c890 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4eca98640 10 monclient(hunting): authenticate will time out at 1710.487544s 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4e6575640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 msgr2=0x7fc4e806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4e6575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 0x7fc4e806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:24.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4e5d74640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc4e809c4a0 0x7fc4e8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:45176/0 (socket says 10.20.193.78:45176) 2026-02-17T06:04:24.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.934+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 learned_addr learned my addr 10.20.193.78:0/1978267558 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:04:24.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.936+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v1:10.20.193.78:6789/0 conn(0x7fc4e809c4a0 legacy=0x7fc4e8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:24.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:24.936+0000 7fc4e6d76640 1 -- 10.20.193.78:0/1978267558 >> v1:10.20.193.115:6789/0 conn(0x7fc4e806d1b0 legacy=0x7fc4e8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:25.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:25.134+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 msgr2=0x7fc4e806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:25.135 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:25.134+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 0x7fc4e806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:04:25.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:25.535+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 msgr2=0x7fc4e806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:25.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:25.535+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 0x7fc4e806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:04:26.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:26.336+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 msgr2=0x7fc4e806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:26.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:26.336+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 0x7fc4e806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:04:27.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient(hunting): tick 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 monclient(hunting): continuing hunt 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 msgr2=0x7fc4e806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e806c890 0x7fc4e806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 >> v1:10.20.193.115:6789/0 conn(0x7fc4e806d1b0 legacy=0x7fc4e8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 >> v1:10.20.193.78:6789/0 conn(0x7fc4e809c4a0 legacy=0x7fc4e8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient: _add_conns ranks=[4,3] 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient(hunting): picked mon.noname-c con 0x7fc4c4003910 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient(hunting): picked mon.noname-a con 0x7fc4c4006440 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c4008f90 con 0x7fc4c4006440 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c4009140 con 0x7fc4c4003910 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4cffff640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:27.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request con 0x7fc4c4006440 auth_method 0 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e6575640 10 monclient(hunting): get_auth_request con 0x7fc4c4003910 auth_method 0 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.934+0000 7fc4e6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.935+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.935+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.935+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:27.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:27.935+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:28.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:28.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request con 0x7fc4c4006440 auth_method 0 2026-02-17T06:04:28.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 10 monclient(hunting): get_auth_request con 0x7fc4c4003910 auth_method 0 2026-02-17T06:04:28.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:28.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:28.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:28.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:04:28.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:28.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:28.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.135+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:04:28.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:28.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request con 0x7fc4c4006440 auth_method 0 2026-02-17T06:04:28.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:28.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:28.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:28.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:04:28.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:28.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:28.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.535+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 10 monclient(hunting): get_auth_request con 0x7fc4c4003910 auth_method 0 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:28.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:28.536+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request con 0x7fc4c4006440 auth_method 0 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:29.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.336+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 10 monclient(hunting): get_auth_request con 0x7fc4c4003910 auth_method 0 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:04:29.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:29.337+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:04:29.875 INFO:tasks.rados.rados.0.trial115.stdout:3024: done (0 left) 2026-02-17T06:04:29.875 INFO:tasks.rados.rados.0.trial115.stdout:3025: delete oid 29 current snap is 315 2026-02-17T06:04:29.876 INFO:tasks.rados.rados.0.trial115.stdout:3025: done (0 left) 2026-02-17T06:04:29.877 INFO:tasks.rados.rados.0.trial115.stdout:3026: snap_create 2026-02-17T06:04:30.876 INFO:tasks.rados.rados.0.trial115.stdout:3026: done (0 left) 2026-02-17T06:04:30.876 INFO:tasks.rados.rados.0.trial115.stdout:3027: write oid 18 current snap is 316 2026-02-17T06:04:30.876 INFO:tasks.rados.rados.0.trial115.stdout:3027: seq_num 1058 ranges {581806=586509,1911596=421369,3087644=403319} 2026-02-17T06:04:30.880 INFO:tasks.rados.rados.0.trial115.stdout:3027: writing trial11545420-18 from 581806 to 1168315 tid 1 2026-02-17T06:04:30.885 INFO:tasks.rados.rados.0.trial115.stdout:3027: writing trial11545420-18 from 1911596 to 2332965 tid 2 2026-02-17T06:04:30.889 INFO:tasks.rados.rados.0.trial115.stdout:3027: writing trial11545420-18 from 3087644 to 3490963 tid 3 2026-02-17T06:04:30.889 INFO:tasks.rados.rados.0.trial115.stdout:3028: read oid 41 snap 312 2026-02-17T06:04:30.890 INFO:tasks.rados.rados.0.trial115.stdout:3028: expect (ObjNum 959 snap 278 seq_num 959) 2026-02-17T06:04:30.890 INFO:tasks.rados.rados.0.trial115.stdout:3029: snap_create 2026-02-17T06:04:30.890 INFO:tasks.rados.rados.0.trial115.stdout:3027: finishing write tid 1 to trial11545420-18 2026-02-17T06:04:30.890 INFO:tasks.rados.rados.0.trial115.stdout:3027: finishing write tid 2 to trial11545420-18 2026-02-17T06:04:30.893 INFO:tasks.rados.rados.0.trial115.stdout:3027: finishing write tid 3 to trial11545420-18 2026-02-17T06:04:30.894 INFO:tasks.rados.rados.0.trial115.stdout:3028: expect (ObjNum 959 snap 278 seq_num 959) 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient(hunting): tick 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 monclient(hunting): continuing hunt 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 msgr2=0x7fc4c4005d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4003910 0x7fc4c4005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 msgr2=0x7fc4c4008900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.78:3300/0 conn(0x7fc4c4006440 0x7fc4c4008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4e6d76640 1 -- 10.20.193.78:0/1978267558 reap_dead start 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient: _add_conns ranks=[0,4,1,3,2] 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc4c4003d60 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4007f90 0x7fc4c40083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient(hunting): picked mon.noname-c con 0x7fc4c4007f90 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc4c4008930 addr v1:10.20.193.115:6789/0 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e809cc80 con 0x7fc4c4003d60 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c400fe10 con 0x7fc4c4008930 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 1 -- 10.20.193.78:0/1978267558 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c4008f90 con 0x7fc4c4007f90 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.934+0000 7fc4cffff640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.935+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4007f90 0x7fc4c40083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.935+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request con 0x7fc4c4007f90 auth_method 0 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.935+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.935+0000 7fc4e5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:30.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.935+0000 7fc4e5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:30.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.936+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> v1:10.20.193.78:6789/0 conn(0x7fc4c4003d60 legacy=0x7fc4c4006870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2144123781 0 0) 0x7fc4e809cc80 con 0x7fc4c4003d60 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): _negotiate 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): my global_id is 184530 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4d4003620 con 0x7fc4c4003d60 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6d76640 1 -- 10.20.193.78:0/1978267558 >> v1:10.20.193.115:6789/0 conn(0x7fc4c4008930 legacy=0x7fc4c400d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2565813317 0 0) 0x7fc4d4003620 con 0x7fc4c4003d60 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4007f90 msgr2=0x7fc4c40083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 --2- 10.20.193.78:0/1978267558 >> v2:10.20.193.115:3300/0 conn(0x7fc4c4007f90 0x7fc4c40083f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 >> v1:10.20.193.115:6789/0 conn(0x7fc4c4008930 legacy=0x7fc4c400d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: _finish_hunting 0 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4011100 con 0x7fc4c4003d60 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: _finish_auth 0 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: _check_auth_tickets 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fc4dc009ba0 con 0x7fc4c4003d60 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:04:30.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: dump: 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 >> v1:10.20.193.78:6789/0 conn(0x7fc4c4003d60 legacy=0x7fc4c4006870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4d4003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): picked mon.a con 0x7fc4d4003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4d4004900 0x7fc4d4004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): picked mon.c con 0x7fc4d4004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4d4005310 0x7fc4d400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): picked mon.b con 0x7fc4d4005310 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4008f90 con 0x7fc4d4003ab0 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d400c240 con 0x7fc4d4005310 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d400c440 con 0x7fc4d4004900 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5573640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5d74640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4d4004900 msgr2=0x7fc4d4004d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:30.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4d4004900 0x7fc4d4004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4d4003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient(hunting): get_auth_request con 0x7fc4d4003ab0 auth_method 0 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6d76640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4d4005310 0x7fc4d400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6d76640 10 monclient(hunting): get_auth_request con 0x7fc4d4005310 auth_method 0 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient(hunting): handle_auth_done global_id 184530 payload 293 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4d4004900 msgr2=0x7fc4d4004d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4d4004900 0x7fc4d4004d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4d4005310 msgr2=0x7fc4d400bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4d4005310 0x7fc4d400bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient: _finish_hunting 0 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 1 monclient: found mon.a 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4003620 con 0x7fc4d4003ab0 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient: _finish_auth 0 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 10 monclient: _check_auth_tickets 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6575640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4e6d76640 1 -- 10.20.193.78:0/1978267558 reap_dead start 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4eca98640 5 monclient: authenticate success, global_id 184530 2026-02-17T06:04:30.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.937+0000 7fc4eca98640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e6575640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 msgr2=0x7fc4d4003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e6575640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4d4003e90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009d80 tx=0x7fc4dc0304b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195bb7f6c3a6ee10 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc03d070 con 0x7fc4d4003ab0 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 10 monclient: dump: 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:30.942 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc038440 con 0x7fc4d4003ab0 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 1 -- 10.20.193.78:0/1978267558 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00b850 con 0x7fc4d4003ab0 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 4 set_mon_vals no callback set 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4e5573640 10 monclient: dump: 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 10 monclient: shutdown 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 1 -- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 msgr2=0x7fc4d4003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4d4003e90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009d80 tx=0x7fc4dc0304b0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 1 -- 10.20.193.78:0/1978267558 shutdown_connections 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 1 --2- 10.20.193.78:0/1978267558 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4d4003e90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 1 -- 10.20.193.78:0/1978267558 >> 10.20.193.78:0/1978267558 conn(0x7fc4e80a6620 msgr2=0x7fc4e80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 1 -- 10.20.193.78:0/1978267558 shutdown_connections 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.940+0000 7fc4eca98640 1 -- 10.20.193.78:0/1978267558 wait complete. 2026-02-17T06:04:30.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient: build_initial_monmap 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient: monmap: 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 1 Processor -- start 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 1 -- start start 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient: init 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4e81ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient(hunting): picked mon.noname-a con 0x7fc4d4003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4e81343c0 0x7fc4e8134840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient(hunting): picked mon.noname-b con 0x7fc4e81343c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e809db50 0x7fc4e809dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient(hunting): picked mon.noname-c con 0x7fc4e809db50 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e80a0020 con 0x7fc4d4003ab0 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e806d600 con 0x7fc4e81343c0 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e80a01a0 con 0x7fc4e809db50 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4eca98640 10 monclient(hunting): authenticate will time out at 1716.494632s 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6d76640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e809db50 msgr2=0x7fc4e809dfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:30.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e809db50 0x7fc4e809dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4e81ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4e81ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36820/0 (socket says 10.20.193.78:36820) 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 -- 10.20.193.78:0/2612396140 learned_addr learned my addr 10.20.193.78:0/2612396140 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient(hunting): get_auth_request con 0x7fc4d4003ab0 auth_method 0 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4e81343c0 0x7fc4e8134840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request con 0x7fc4e81343c0 auth_method 0 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient(hunting): handle_auth_done global_id 184533 payload 824 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 -- 10.20.193.78:0/2612396140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e809db50 msgr2=0x7fc4e809dfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e809db50 0x7fc4e809dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 -- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4e81343c0 msgr2=0x7fc4e8134840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4e81343c0 0x7fc4e8134840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient: _finish_hunting 0 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 monclient: found mon.noname-a 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:30.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.941+0000 7fc4e6575640 1 -- 10.20.193.78:0/2612396140 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e809f7f0 con 0x7fc4d4003ab0 2026-02-17T06:04:30.946 INFO:tasks.rados.rados.0.trial115.stdout:3027: finishing write tid 4 to trial11545420-18 2026-02-17T06:04:30.946 INFO:tasks.rados.rados.0.trial115.stdout:3027: finishing write tid 5 to trial11545420-18 2026-02-17T06:04:30.946 INFO:tasks.rados.rados.0.trial115.stdout:3027: oid 18 updating version 0 to 1541 2026-02-17T06:04:30.946 INFO:tasks.rados.rados.0.trial115.stdout:3027: oid 18 version 1541 is already newer than 1540 2026-02-17T06:04:30.946 INFO:tasks.rados.rados.0.trial115.stdout:3027: oid 18 updating version 1541 to 1542 2026-02-17T06:04:30.946 INFO:tasks.rados.rados.0.trial115.stdout:3027: oid 18 version 1542 is already newer than 1539 2026-02-17T06:04:30.946 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1542 (ObjNum 1058 snap 316 seq_num 1058) dirty exists 2026-02-17T06:04:30.946 INFO:tasks.rados.rados.0.trial115.stdout:3027: left oid 18 (ObjNum 1058 snap 316 seq_num 1058) 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.943+0000 7fc4e6575640 1 -- 10.20.193.78:0/2612396140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 msgr2=0x7fc4e81ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.943+0000 7fc4e6575640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4e81ced30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00bbf0 tx=0x7fc4dc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 1 -- 10.20.193.78:0/2612396140 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc03d040 con 0x7fc4d4003ab0 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: dump: 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:30.947 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: _finish_auth 0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: _check_auth_tickets 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 1 -- 10.20.193.78:0/2612396140 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc038d50 con 0x7fc4d4003ab0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 5 monclient: authenticate success, global_id 184533 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 10 monclient: _renew_subs 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e8134d80 con 0x7fc4d4003ab0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 1 -- 10.20.193.78:0/2612396140 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00b850 con 0x7fc4d4003ab0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4e4d72640 4 set_mon_vals no callback set 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 10 monclient: dump: 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 10 monclient: _renew_subs 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e8123da0 con 0x7fc4d4003ab0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 1 -- 10.20.193.78:0/2612396140 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc4dc055060 con 0x7fc4d4003ab0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:30.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e809be20 con 0x7fc4d4003ab0 2026-02-17T06:04:30.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.944+0000 7fc4ceffd640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4b403de60 0x7fc4b4040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:30.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.945+0000 7fc4ceffd640 1 -- 10.20.193.78:0/2612396140 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(650..650 src has 525..650) ==== 4799+0+0 (secure 0 0 0) 0x7fc4dc078c60 con 0x7fc4d4003ab0 2026-02-17T06:04:30.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.945+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4b403de60 0x7fc4b4040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:30.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.945+0000 7fc4e5d74640 10 monclient: get_auth_request con 0x7fc4b403de60 auth_method 0 2026-02-17T06:04:30.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.947+0000 7fc4ceffd640 1 -- 10.20.193.78:0/2612396140 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc4dc077650 con 0x7fc4d4003ab0 2026-02-17T06:04:30.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.947+0000 7fc4ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:30.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.947+0000 7fc4ceffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:30.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:30.947+0000 7fc4e5d74640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4b403de60 0x7fc4b4040320 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0007fd0 tx=0x7fc4d0006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.059 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.059+0000 7fc4eca98640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:04:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.059+0000 7fc4eca98640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.059+0000 7fc4eca98640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.059+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc4e80a3f30 con 0x7fc4d4003ab0 2026-02-17T06:04:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.060+0000 7fc4ceffd640 1 -- 10.20.193.78:0/2612396140 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fc4dc042cb0 con 0x7fc4d4003ab0 2026-02-17T06:04:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.060+0000 7fc4ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:04:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.060+0000 7fc4ceffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:31.060 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":214,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:04:31.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4b403de60 msgr2=0x7fc4b4040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4b403de60 0x7fc4b4040320 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0007fd0 tx=0x7fc4d0006040 comp rx=0 tx=0).stop 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 10 monclient: shutdown 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 msgr2=0x7fc4e81ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4e81ced30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00bbf0 tx=0x7fc4dc005e40 comp rx=0 tx=0).stop 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 shutdown_connections 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc4e809db50 0x7fc4e809dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc4b403de60 0x7fc4b4040320 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc4e81343c0 0x7fc4e8134840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 --2- 10.20.193.78:0/2612396140 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc4d4003ab0 0x7fc4e81ced30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 >> 10.20.193.78:0/2612396140 conn(0x7fc4e80a6620 msgr2=0x7fc4e81c0370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 shutdown_connections 2026-02-17T06:04:31.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.061+0000 7fc4eca98640 1 -- 10.20.193.78:0/2612396140 wait complete. 2026-02-17T06:04:31.069 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:04:31.069 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:04:31.069 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:04:31.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.136+0000 7fe05c33e640 10 monclient: get_monmap_and_config 2026-02-17T06:04:31.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.136+0000 7fe05c33e640 10 monclient: build_initial_monmap 2026-02-17T06:04:31.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.136+0000 7fe05c33e640 10 monclient: monmap: 2026-02-17T06:04:31.136 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:31.136562+0000 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:31.136562+0000 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:31.137 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.136+0000 7fe05c33e640 1 Processor -- start 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.136+0000 7fe05c33e640 1 -- start start 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.136+0000 7fe05c33e640 10 monclient: init 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient: _add_conns ranks=[3,2,4,1,0] 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe05406c890 0x7fe05406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient(hunting): picked mon.noname-a con 0x7fe05406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe05409c4a0 0x7fe05406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient(hunting): picked mon.noname-b con 0x7fe05409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fe054134df0 0x7fe054135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient(hunting): picked mon.noname-c con 0x7fe054134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fe0540a0880 con 0x7fe05406c890 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fe05406d6f0 con 0x7fe054134df0 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe05406daf0 con 0x7fe05409c4a0 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05c33e640 10 monclient(hunting): authenticate will time out at 1716.690142s 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe0598b2640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe05409c4a0 msgr2=0x7fe05406d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe0598b2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe05409c4a0 0x7fe05406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe05406c890 0x7fe05406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe05406c890 0x7fe05406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36824/0 (socket says 10.20.193.78:36824) 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 -- 10.20.193.78:0/4106638312 learned_addr learned my addr 10.20.193.78:0/4106638312 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient(hunting): get_auth_request con 0x7fe05406c890 auth_method 0 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:31.147 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a8b4640 1 --2- 10.20.193.78:0/4106638312 >> v2:10.20.193.115:3300/0 conn(0x7fe054134df0 0x7fe054135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a8b4640 10 monclient(hunting): get_auth_request con 0x7fe054134df0 auth_method 0 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a8b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a8b4640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a8b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient(hunting): handle_auth_done global_id 184536 payload 274 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 -- 10.20.193.78:0/4106638312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe05409c4a0 msgr2=0x7fe05406d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 --2- 10.20.193.78:0/4106638312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe05409c4a0 0x7fe05406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 -- 10.20.193.78:0/4106638312 >> v2:10.20.193.115:3300/0 conn(0x7fe054134df0 msgr2=0x7fe054135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 --2- 10.20.193.78:0/4106638312 >> v2:10.20.193.115:3300/0 conn(0x7fe054134df0 0x7fe054135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient: _finish_hunting 0 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 monclient: found mon.noname-a 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.137+0000 7fe05a0b3640 1 -- 10.20.193.78:0/4106638312 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05409cc80 con 0x7fe05406c890 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.139+0000 7fe05a0b3640 1 -- 10.20.193.78:0/4106638312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 msgr2=0x7fe05406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.139+0000 7fe05a0b3640 1 --2- 10.20.193.78:0/4106638312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 0x7fe05406cc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0540a7490 tx=0x7fe04402ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fca32f188d4e5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 1 -- 10.20.193.78:0/4106638312 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04403c070 con 0x7fe05406c890 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: dump: 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: _finish_auth 0 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: _check_auth_tickets 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 1 -- 10.20.193.78:0/4106638312 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04402fcc0 con 0x7fe05406c890 2026-02-17T06:04:31.148 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 5 monclient: authenticate success, global_id 184536 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 1 -- 10.20.193.78:0/4106638312 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044037660 con 0x7fe05406c890 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 4 set_mon_vals no callback set 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe0590b1640 10 monclient: dump: 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 10 monclient: shutdown 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 -- 10.20.193.78:0/4106638312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 msgr2=0x7fe05406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 --2- 10.20.193.78:0/4106638312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 0x7fe05406cc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0540a7490 tx=0x7fe04402ee80 comp rx=0 tx=0).stop 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 -- 10.20.193.78:0/4106638312 shutdown_connections 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 --2- 10.20.193.78:0/4106638312 >> v2:10.20.193.115:3300/0 conn(0x7fe054134df0 0x7fe054135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 --2- 10.20.193.78:0/4106638312 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe05409c4a0 0x7fe05406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 --2- 10.20.193.78:0/4106638312 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 0x7fe05406cc70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 -- 10.20.193.78:0/4106638312 >> 10.20.193.78:0/4106638312 conn(0x7fe0540a6620 msgr2=0x7fe0540a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 -- 10.20.193.78:0/4106638312 shutdown_connections 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.140+0000 7fe05c33e640 1 -- 10.20.193.78:0/4106638312 wait complete. 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient: build_initial_monmap 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient: monmap: 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:31.149 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 1 Processor -- start 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 1 -- start start 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient: init 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 0x7fe0541cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient(hunting): picked mon.noname-a con 0x7fe05406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe05409c4a0 0x7fe0541217d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient(hunting): picked mon.noname-b con 0x7fe05409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe054134df0 0x7fe054121d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient(hunting): picked mon.noname-c con 0x7fe054134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0540a01a0 con 0x7fe05406c890 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe0540a0020 con 0x7fe05409c4a0 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe05409be20 con 0x7fe054134df0 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05c33e640 10 monclient(hunting): authenticate will time out at 1716.694596s 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a8b4640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe054134df0 msgr2=0x7fe054121d10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a8b4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe054134df0 0x7fe054121d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 0x7fe0541cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 0x7fe0541cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36826/0 (socket says 10.20.193.78:36826) 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 -- 10.20.193.78:0/1787532780 learned_addr learned my addr 10.20.193.78:0/1787532780 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient(hunting): get_auth_request con 0x7fe05406c890 auth_method 0 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient(hunting): handle_auth_done global_id 184539 payload 824 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe054134df0 msgr2=0x7fe054121d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe054134df0 0x7fe054121d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe05409c4a0 msgr2=0x7fe0541217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.150 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe05409c4a0 0x7fe0541217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient: _finish_hunting 0 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 monclient: found mon.noname-a 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.141+0000 7fe05a0b3640 1 -- 10.20.193.78:0/1787532780 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0541260a0 con 0x7fe05406c890 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.143+0000 7fe05a0b3640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 msgr2=0x7fe0541cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.143+0000 7fe05a0b3640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 0x7fe0541cd680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe044002410 tx=0x7fe044004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04403c040 con 0x7fe05406c890 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: dump: 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: _finish_auth 0 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: _check_auth_tickets 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe044047070 con 0x7fe05406c890 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 5 monclient: authenticate success, global_id 184539 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 10 monclient: _renew_subs 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0541256c0 con 0x7fe05406c890 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044041800 con 0x7fe05406c890 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0588b0640 4 set_mon_vals no callback set 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 10 monclient: dump: 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.151 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 10 monclient: _renew_subs 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0540a16d0 con 0x7fe05406c890 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0541cdbc0 0x7fe054125010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe054123f30 con 0x7fe0541cdbc0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0541cdbc0 0x7fe054125010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 10 monclient: get_auth_request con 0x7fe0541cdbc0 auth_method 0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 10 monclient(hunting): handle_auth_done global_id 184542 payload 824 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe0440419a0 con 0x7fe05406c890 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.144+0000 7fe0598b2640 1 -- 10.20.193.78:0/1787532780 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe054074980 con 0x7fe0541cdbc0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.145+0000 7fe03affd640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe02403deb0 0x7fe024040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.145+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(650..650 src has 525..650) ==== 4799+0+0 (secure 0 0 0) 0x7fe044077620 con 0x7fe05406c890 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.145+0000 7fe05a8b4640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe02403deb0 0x7fe024040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.145+0000 7fe05a8b4640 10 monclient: get_auth_request con 0x7fe02403deb0 auth_method 0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe0598b2640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0541cdbc0 msgr2=0x7fe054125010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe0598b2640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0541cdbc0 0x7fe054125010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe054134800 tx=0x7fe04800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0480052f0 con 0x7fe0541cdbc0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 10 monclient: dump: 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.152 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fe048015070 con 0x7fe0541cdbc0 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0541cdbc0 msgr2=0x7fe054125010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0541cdbc0 0x7fe054125010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe054134800 tx=0x7fe04800b8f0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0541cdbc0 msgr2=0x7fe054125010 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe03affd640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe0541cdbc0 0x7fe054125010 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.147+0000 7fe05a8b4640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe02403deb0 0x7fe024040370 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe050009700 tx=0x7fe050009950 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe05c33e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe05c33e640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05411cc70 0x7fe0541d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe05c33e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe054135240 con 0x7fe05411cc70 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05411cc70 0x7fe0541d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 10 monclient: get_auth_request con 0x7fe05411cc70 auth_method 0 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 10 monclient(hunting): handle_auth_done global_id 184545 payload 824 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.150+0000 7fe0598b2640 1 -- 10.20.193.78:0/1787532780 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe0540744d0 con 0x7fe05411cc70 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.152+0000 7fe0598b2640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05411cc70 msgr2=0x7fe0541d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.152+0000 7fe0598b2640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05411cc70 0x7fe0541d8330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe054073e30 tx=0x7fe048005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048010610 con 0x7fe05411cc70 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 10 monclient: dump: 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.154 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7fe048010610 con 0x7fe05411cc70 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05411cc70 msgr2=0x7fe0541d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05411cc70 0x7fe0541d8330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe054073e30 tx=0x7fe048005ad0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05411cc70 msgr2=0x7fe0541d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe03affd640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05411cc70 0x7fe0541d8330 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe054073e30 tx=0x7fe048005ad0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 214, 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 235933, 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:31.155 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:31.156 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:31.157 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:31.158 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:04:31.159 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe02403deb0 msgr2=0x7fe024040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe02403deb0 0x7fe024040370 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe050009700 tx=0x7fe050009950 comp rx=0 tx=0).stop 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 10 monclient: shutdown 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 msgr2=0x7fe0541cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 1 --2- 10.20.193.78:0/1787532780 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe05406c890 0x7fe0541cd680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe044002410 tx=0x7fe044004830 comp rx=0 tx=0).stop 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05a8b4640 1 -- 10.20.193.78:0/1787532780 reap_dead start 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05a8b4640 1 -- 10.20.193.78:0/1787532780 reap_dead start 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 shutdown_connections 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 >> 10.20.193.78:0/1787532780 conn(0x7fe0540a6620 msgr2=0x7fe0540a1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 shutdown_connections 2026-02-17T06:04:31.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.153+0000 7fe05c33e640 1 -- 10.20.193.78:0/1787532780 wait complete. 2026-02-17T06:04:31.160 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:04:31.161 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T06:04:31.161 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient: get_monmap_and_config 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient: build_initial_monmap 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient: monmap: 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:04:31.227362+0000 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:04:31.227362+0000 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:04:31.227 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 1 Processor -- start 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 1 -- start start 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient: init 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient: _add_conns ranks=[0,3,4,2,1] 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb2006c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ffb2009c4a0 0x7ffb2013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient(hunting): picked mon.noname-a con 0x7ffb2009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ffb2006d380 0x7ffb2013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient(hunting): picked mon.noname-c con 0x7ffb2006d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb2006daf0 con 0x7ffb2006c890 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ffb200a0880 con 0x7ffb2009c4a0 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ffb2009cc80 con 0x7ffb2006d380 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb24d49640 10 monclient(hunting): authenticate will time out at 1716.780883s 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb1dd74640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ffb2009c4a0 0x7ffb2013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb1e575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ffb2006c890 0x7ffb2006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54934/0 (socket says 10.20.193.78:54934) 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb1e575640 1 -- 10.20.193.78:0/3452642204 learned_addr learned my addr 10.20.193.78:0/3452642204 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb1dd74640 10 monclient(hunting): get_auth_request con 0x7ffb2009c4a0 auth_method 0 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb1dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb1dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.227+0000 7ffb1dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:31.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1ed76640 1 --2- 10.20.193.78:0/3452642204 >> v2:10.20.193.115:3300/0 conn(0x7ffb2006d380 0x7ffb2013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 10 monclient(hunting): handle_auth_done global_id 184548 payload 274 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 1 -- 10.20.193.78:0/3452642204 >> v2:10.20.193.115:3300/0 conn(0x7ffb2006d380 msgr2=0x7ffb2013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 1 --2- 10.20.193.78:0/3452642204 >> v2:10.20.193.115:3300/0 conn(0x7ffb2006d380 0x7ffb2013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 1 -- 10.20.193.78:0/3452642204 >> v1:10.20.193.78:6789/0 conn(0x7ffb2006c890 legacy=0x7ffb2006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 10 monclient: _finish_hunting 0 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 1 monclient: found mon.noname-a 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1dd74640 1 -- 10.20.193.78:0/3452642204 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2009cf60 con 0x7ffb2009c4a0 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1ed76640 10 monclient: get_auth_request con 0x7ffb2006d380 auth_method 0 2026-02-17T06:04:31.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.228+0000 7ffb1ed76640 1 --2- 10.20.193.78:0/3452642204 >> v2:10.20.193.115:3300/0 conn(0x7ffb2006d380 0x7ffb2013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:04:31.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.230+0000 7ffb1dd74640 1 -- 10.20.193.78:0/3452642204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 msgr2=0x7ffb2013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:31.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.230+0000 7ffb1dd74640 1 --2- 10.20.193.78:0/3452642204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb2013b3a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb2006dcc0 tx=0x7ffb0802ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2867adb8bc0277 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 1 -- 10.20.193.78:0/3452642204 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0803c070 con 0x7ffb2009c4a0 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: dump: 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: _finish_auth 0 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: _check_auth_tickets 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 1 -- 10.20.193.78:0/3452642204 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb0802fcb0 con 0x7ffb2009c4a0 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb24d49640 5 monclient: authenticate success, global_id 184548 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 1 -- 10.20.193.78:0/3452642204 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0802fe60 con 0x7ffb2009c4a0 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb24d49640 10 monclient: get_monmap_and_config success 2026-02-17T06:04:31.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb24d49640 4 set_mon_vals no callback set 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb1d573640 10 monclient: dump: 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.233 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb24d49640 10 monclient: shutdown 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb24d49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb24d49640 1 -- 10.20.193.78:0/3452642204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 msgr2=0x7ffb2013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.232+0000 7ffb24d49640 1 --2- 10.20.193.78:0/3452642204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb2013b3a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb2006dcc0 tx=0x7ffb0802ed50 comp rx=0 tx=0).stop 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 1 -- 10.20.193.78:0/3452642204 shutdown_connections 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 1 --2- 10.20.193.78:0/3452642204 >> v2:10.20.193.115:3300/0 conn(0x7ffb2006d380 0x7ffb2013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 1 --2- 10.20.193.78:0/3452642204 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb2013b3a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 1 -- 10.20.193.78:0/3452642204 >> 10.20.193.78:0/3452642204 conn(0x7ffb200a6620 msgr2=0x7ffb200a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 1 -- 10.20.193.78:0/3452642204 shutdown_connections 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 1 -- 10.20.193.78:0/3452642204 wait complete. 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 10 monclient: build_initial_monmap 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 10 monclient: monmap: 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 1 Processor -- start 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 1 -- start start 2026-02-17T06:04:31.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.233+0000 7ffb24d49640 10 monclient: init 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient: _reopen_session rank -1 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb2006c890 0x7ffb201ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient(hunting): picked mon.noname-c con 0x7ffb2006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb2006d380 0x7ffb20131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient(hunting): picked mon.noname-b con 0x7ffb2006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb201392f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient(hunting): picked mon.noname-a con 0x7ffb2009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2009bfa0 con 0x7ffb2009c4a0 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2009be20 con 0x7ffb2006d380 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2013de30 con 0x7ffb2006c890 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient(hunting): _renew_subs 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb24d49640 10 monclient(hunting): authenticate will time out at 1716.787073s 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1e575640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb2006c890 msgr2=0x7ffb201ccdf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb2006c890 0x7ffb201ccdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb201392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb201392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36844/0 (socket says 10.20.193.78:36844) 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 -- 10.20.193.78:0/3625709139 learned_addr learned my addr 10.20.193.78:0/3625709139 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient(hunting): get_auth_request con 0x7ffb2009c4a0 auth_method 0 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1dd74640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb2006d380 0x7ffb20131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1dd74640 10 monclient(hunting): get_auth_request con 0x7ffb2006d380 auth_method 0 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient(hunting): handle_auth_done global_id 184551 payload 824 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb2006c890 msgr2=0x7ffb201ccdf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ffb2006c890 0x7ffb201ccdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb2006d380 msgr2=0x7ffb20131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb2006d380 0x7ffb20131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient: _finish_hunting 0 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 monclient: found mon.noname-a 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:04:31.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.234+0000 7ffb1ed76640 1 -- 10.20.193.78:0/3625709139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2013cf50 con 0x7ffb2009c4a0 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb1ed76640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 msgr2=0x7ffb201392f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb1ed76640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb201392f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000ce40 tx=0x7ffb1000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1001a070 con 0x7ffb2009c4a0 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: dump: 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: _finish_auth 0 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:04:31.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1000da50 con 0x7ffb2009c4a0 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb24d49640 5 monclient: authenticate success, global_id 184551 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000de40 con 0x7ffb2009c4a0 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb06ffd640 10 monclient: dump: 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb1cd72640 4 set_mon_vals no callback set 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb24d49640 10 monclient: _renew_subs 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb24d49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2013d230 con 0x7ffb2009c4a0 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb24d49640 10 monclient: _renew_subs 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb24d49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:04:31.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.236+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2013daf0 con 0x7ffb2009c4a0 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb24d49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb24d49640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20132500 0x7ffb20139830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb24d49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20074980 con 0x7ffb20132500 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ffb1001e3b0 con 0x7ffb2009c4a0 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20132500 0x7ffb20139830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 10 monclient: get_auth_request con 0x7ffb20132500 auth_method 0 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb06ffd640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffaec03de60 0x7ffaec040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(650..650 src has 525..650) ==== 4799+0+0 (secure 0 0 0) 0x7ffb10053280 con 0x7ffb2009c4a0 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 10 monclient(hunting): handle_auth_done global_id 214222 payload 824 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1e575640 1 -- 10.20.193.78:0/3625709139 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb2006cc70 con 0x7ffb20132500 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1dd74640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffaec03de60 0x7ffaec040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.237+0000 7ffb1dd74640 10 monclient: get_auth_request con 0x7ffaec03de60 auth_method 0 2026-02-17T06:04:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb1e575640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20132500 msgr2=0x7ffb20139830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb1e575640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20132500 0x7ffb20139830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb14004970 tx=0x7ffb1400cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14017070 con 0x7ffb20132500 2026-02-17T06:04:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 10 monclient: dump: 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb1dd74640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffaec03de60 0x7ffaec040320 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb2006a480 tx=0x7ffb08033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ffb14005d40 con 0x7ffb20132500 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20132500 msgr2=0x7ffb20139830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20132500 0x7ffb20139830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb14004970 tx=0x7ffb1400cb20 comp rx=0 tx=0).stop 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20132500 msgr2=0x7ffb20139830 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:31.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.240+0000 7ffb06ffd640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20132500 0x7ffb20139830 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb14004970 tx=0x7ffb1400cb20 comp rx=0 tx=0).stop 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb24d49640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb24d49640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20139d70 0x7ffb201d8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb24d49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2013a180 con 0x7ffb20139d70 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb1e575640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20139d70 0x7ffb201d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb1e575640 10 monclient: get_auth_request con 0x7ffb20139d70 auth_method 0 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb1e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb1e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.243+0000 7ffb1e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.244+0000 7ffb1e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.244+0000 7ffb1e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.244+0000 7ffb1e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.244+0000 7ffb1e575640 10 monclient(hunting): handle_auth_done global_id 214225 payload 824 2026-02-17T06:04:31.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.244+0000 7ffb1e575640 1 -- 10.20.193.78:0/3625709139 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ffb2013bde0 con 0x7ffb20139d70 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb1e575640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20139d70 msgr2=0x7ffb201d8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb1e575640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20139d70 0x7ffb201d8160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb14014000 tx=0x7ffb14004b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1401ae60 con 0x7ffb20139d70 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 10 monclient: dump: 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:04:31.246 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7ffb14007490 con 0x7ffb20139d70 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20139d70 msgr2=0x7ffb201d8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.246+0000 7ffb06ffd640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20139d70 0x7ffb201d8160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb14014000 tx=0x7ffb14004b10 comp rx=0 tx=0).stop 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb06ffd640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20139d70 msgr2=0x7ffb201d8160 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb06ffd640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ffb20139d70 0x7ffb201d8160 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb14014000 tx=0x7ffb14004b10 comp rx=0 tx=0).stop 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:31.247 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 214, 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 74092, 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:31.248 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:04:31.249 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:04:31.250 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:04:31.251 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:04:31.252 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:04:31.253 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:04:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffaec03de60 msgr2=0x7ffaec040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ffaec03de60 0x7ffaec040320 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb2006a480 tx=0x7ffb08033000 comp rx=0 tx=0).stop 2026-02-17T06:04:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 10 monclient: shutdown 2026-02-17T06:04:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:04:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 msgr2=0x7ffb201392f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:04:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb201392f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000ce40 tx=0x7ffb1000a710 comp rx=0 tx=0).stop 2026-02-17T06:04:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb1ed76640 1 -- 10.20.193.78:0/3625709139 reap_dead start 2026-02-17T06:04:31.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 shutdown_connections 2026-02-17T06:04:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 --2- 10.20.193.78:0/3625709139 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ffb2009c4a0 0x7ffb201392f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:04:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 >> 10.20.193.78:0/3625709139 conn(0x7ffb200a6620 msgr2=0x7ffb20136190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:04:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 shutdown_connections 2026-02-17T06:04:31.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:04:31.247+0000 7ffb24d49640 1 -- 10.20.193.78:0/3625709139 wait complete. 2026-02-17T06:04:31.255 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:04:31.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:04:31.880 INFO:tasks.rados.rados.0.trial115.stdout:3027: done (2 left) 2026-02-17T06:04:31.881 INFO:tasks.rados.rados.0.trial115.stdout:3028: done (1 left) 2026-02-17T06:04:31.881 INFO:tasks.rados.rados.0.trial115.stdout:3029: done (0 left) 2026-02-17T06:04:31.881 INFO:tasks.rados.rados.0.trial115.stdout:3030: delete oid 23 current snap is 317 2026-02-17T06:04:31.882 INFO:tasks.rados.rados.0.trial115.stdout:3030: done (0 left) 2026-02-17T06:04:31.882 INFO:tasks.rados.rados.0.trial115.stdout:3031: snap_create 2026-02-17T06:04:32.883 INFO:tasks.rados.rados.0.trial115.stdout:3031: done (0 left) 2026-02-17T06:04:32.883 INFO:tasks.rados.rados.0.trial115.stdout:3032: read oid 39 snap 309 2026-02-17T06:04:32.883 INFO:tasks.rados.rados.0.trial115.stdout:3032: expect deleted 2026-02-17T06:04:32.884 INFO:tasks.rados.rados.0.trial115.stdout:3033: read oid 3 snap 301 2026-02-17T06:04:32.884 INFO:tasks.rados.rados.0.trial115.stdout:3033: expect deleted 2026-02-17T06:04:32.884 INFO:tasks.rados.rados.0.trial115.stdout:3034: write oid 44 current snap is 318 2026-02-17T06:04:32.884 INFO:tasks.rados.rados.0.trial115.stdout:3034: seq_num 1060 ranges {615420=737007,1912207=425038,3026137=542216} 2026-02-17T06:04:32.888 INFO:tasks.rados.rados.0.trial115.stdout:3034: writing trial11545420-44 from 615420 to 1352427 tid 1 2026-02-17T06:04:32.891 INFO:tasks.rados.rados.0.trial115.stdout:3034: writing trial11545420-44 from 1912207 to 2337245 tid 2 2026-02-17T06:04:32.896 INFO:tasks.rados.rados.0.trial115.stdout:3034: writing trial11545420-44 from 3026137 to 3568353 tid 3 2026-02-17T06:04:32.896 INFO:tasks.rados.rados.0.trial115.stdout:3035: copy_from oid 26 from oid 31 current snap is 318 2026-02-17T06:04:32.897 INFO:tasks.rados.rados.0.trial115.stdout:3034: finishing write tid 1 to trial11545420-44 2026-02-17T06:04:32.897 INFO:tasks.rados.rados.0.trial115.stdout:3034: finishing write tid 2 to trial11545420-44 2026-02-17T06:04:32.897 INFO:tasks.rados.rados.0.trial115.stdout:3032: done (3 left) 2026-02-17T06:04:32.897 INFO:tasks.rados.rados.0.trial115.stdout:3033: done (2 left) 2026-02-17T06:04:32.897 INFO:tasks.rados.rados.0.trial115.stdout:3036: snap_remove snap 303 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3036: done (2 left) 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3037: read oid 41 snap 316 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3037: expect (ObjNum 959 snap 278 seq_num 959) 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3038: snap_create 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: finishing write tid 3 to trial11545420-44 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: finishing write tid 4 to trial11545420-44 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: finishing write tid 5 to trial11545420-44 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: oid 44 updating version 0 to 1420 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: oid 44 version 1420 is already newer than 1419 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: oid 44 updating version 1420 to 1422 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: oid 44 version 1422 is already newer than 1421 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1422 (ObjNum 1060 snap 318 seq_num 1060) dirty exists 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: left oid 44 (ObjNum 1060 snap 318 seq_num 1060) 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3035: finishing copy_from racing read to trial11545420-26 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:3035: finishing copy_from to trial11545420-26 2026-02-17T06:04:33.670 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1449 (ObjNum 1052 snap 313 seq_num 1052) dirty exists 2026-02-17T06:04:33.676 INFO:tasks.rados.rados.0.trial115.stdout:3037: expect (ObjNum 959 snap 278 seq_num 959) 2026-02-17T06:04:34.670 INFO:tasks.rados.rados.0.trial115.stdout:3034: done (3 left) 2026-02-17T06:04:34.670 INFO:tasks.rados.rados.0.trial115.stdout:3035: done (2 left) 2026-02-17T06:04:34.670 INFO:tasks.rados.rados.0.trial115.stdout:3037: done (1 left) 2026-02-17T06:04:34.670 INFO:tasks.rados.rados.0.trial115.stdout:3038: done (0 left) 2026-02-17T06:04:34.671 INFO:tasks.rados.rados.0.trial115.stdout:3039: read oid 29 snap 315 2026-02-17T06:04:34.671 INFO:tasks.rados.rados.0.trial115.stdout:3039: expect deleted 2026-02-17T06:04:34.671 INFO:tasks.rados.rados.0.trial115.stdout:3040: read oid 11 snap -1 2026-02-17T06:04:34.671 INFO:tasks.rados.rados.0.trial115.stdout:3040: expect (ObjNum 1027 snap 297 seq_num 1027) 2026-02-17T06:04:34.671 INFO:tasks.rados.rados.0.trial115.stdout:3041: snap_remove snap 311 2026-02-17T06:04:35.671 INFO:tasks.rados.rados.0.trial115.stdout:3041: done (2 left) 2026-02-17T06:04:35.671 INFO:tasks.rados.rados.0.trial115.stdout:3042: write (excl) oid 23 current snap is 319 2026-02-17T06:04:35.671 INFO:tasks.rados.rados.0.trial115.stdout:3042: seq_num 1061 ranges {632227=412256,1570689=568696,2653560=753488} 2026-02-17T06:04:35.674 INFO:tasks.rados.rados.0.trial115.stdout:3042: writing trial11545420-23 from 632227 to 1044483 tid 1 2026-02-17T06:04:35.679 INFO:tasks.rados.rados.0.trial115.stdout:3042: writing trial11545420-23 from 1570689 to 2139385 tid 2 2026-02-17T06:04:35.684 INFO:tasks.rados.rados.0.trial115.stdout:3042: writing trial11545420-23 from 2653560 to 3407048 tid 3 2026-02-17T06:04:35.685 INFO:tasks.rados.rados.0.trial115.stdout:3043: read oid 18 snap -1 2026-02-17T06:04:35.685 INFO:tasks.rados.rados.0.trial115.stdout:3043: expect (ObjNum 1058 snap 316 seq_num 1058) 2026-02-17T06:04:35.685 INFO:tasks.rados.rados.0.trial115.stdout:3044: read oid 21 snap -1 2026-02-17T06:04:35.685 INFO:tasks.rados.rados.0.trial115.stdout:3044: expect deleted 2026-02-17T06:04:35.685 INFO:tasks.rados.rados.0.trial115.stdout:3045: delete oid 41 current snap is 319 2026-02-17T06:04:35.685 INFO:tasks.rados.rados.0.trial115.stdout:3040: expect (ObjNum 1027 snap 297 seq_num 1027) 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: finishing write tid 1 to trial11545420-23 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3039: done (5 left) 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3040: done (4 left) 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3045: done (3 left) 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3046: read oid 24 snap -1 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3046: expect (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3047: read oid 32 snap -1 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3047: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3048: copy_from oid 48 from oid 22 current snap is 319 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: finishing write tid 2 to trial11545420-23 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: finishing write tid 3 to trial11545420-23 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: finishing write tid 4 to trial11545420-23 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: finishing write tid 5 to trial11545420-23 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: oid 23 updating version 0 to 1406 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: oid 23 updating version 1406 to 1409 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: oid 23 version 1409 is already newer than 1407 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: oid 23 version 1409 is already newer than 1408 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1409 (ObjNum 1061 snap 319 seq_num 1061) dirty exists 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3042: left oid 23 (ObjNum 1061 snap 319 seq_num 1061) 2026-02-17T06:04:35.835 INFO:tasks.rados.rados.0.trial115.stdout:3043: expect (ObjNum 1058 snap 316 seq_num 1058) 2026-02-17T06:04:35.977 INFO:tasks.rados.rados.0.trial115.stdout:3042: done (5 left) 2026-02-17T06:04:35.977 INFO:tasks.rados.rados.0.trial115.stdout:3043: done (4 left) 2026-02-17T06:04:35.977 INFO:tasks.rados.rados.0.trial115.stdout:3044: done (3 left) 2026-02-17T06:04:35.977 INFO:tasks.rados.rados.0.trial115.stdout:3049: read oid 13 snap -1 2026-02-17T06:04:35.977 INFO:tasks.rados.rados.0.trial115.stdout:3049: expect deleted 2026-02-17T06:04:35.978 INFO:tasks.rados.rados.0.trial115.stdout:3050: read oid 44 snap -1 2026-02-17T06:04:35.978 INFO:tasks.rados.rados.0.trial115.stdout:3050: expect (ObjNum 1060 snap 318 seq_num 1060) 2026-02-17T06:04:35.978 INFO:tasks.rados.rados.0.trial115.stdout:3051: write oid 19 current snap is 319 2026-02-17T06:04:35.978 INFO:tasks.rados.rados.0.trial115.stdout:3051: seq_num 1063 ranges {665841=562754,1687653=572365,2708406=776032} 2026-02-17T06:04:35.980 INFO:tasks.rados.rados.0.trial115.stdout:3051: writing trial11545420-19 from 665841 to 1228595 tid 1 2026-02-17T06:04:35.984 INFO:tasks.rados.rados.0.trial115.stdout:3051: writing trial11545420-19 from 1687653 to 2260018 tid 2 2026-02-17T06:04:35.989 INFO:tasks.rados.rados.0.trial115.stdout:3051: writing trial11545420-19 from 2708406 to 3484438 tid 3 2026-02-17T06:04:35.989 INFO:tasks.rados.rados.0.trial115.stdout:3052: read oid 10 snap -1 2026-02-17T06:04:35.990 INFO:tasks.rados.rados.0.trial115.stdout:3052: expect deleted 2026-02-17T06:04:35.990 INFO:tasks.rados.rados.0.trial115.stdout:3053: write oid 50 current snap is 319 2026-02-17T06:04:35.990 INFO:tasks.rados.rados.0.trial115.stdout:3053: seq_num 1064 ranges {682648=638003,2029782=432376,3019476=587304} 2026-02-17T06:04:35.994 INFO:tasks.rados.rados.0.trial115.stdout:3053: writing trial11545420-50 from 682648 to 1320651 tid 1 2026-02-17T06:04:35.998 INFO:tasks.rados.rados.0.trial115.stdout:3053: writing trial11545420-50 from 2029782 to 2462158 tid 2 2026-02-17T06:04:36.003 INFO:tasks.rados.rados.0.trial115.stdout:3053: writing trial11545420-50 from 3019476 to 3606780 tid 3 2026-02-17T06:04:36.003 INFO:tasks.rados.rados.0.trial115.stdout:3054: write oid 26 current snap is 319 2026-02-17T06:04:36.003 INFO:tasks.rados.rados.0.trial115.stdout:3046: expect (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:04:36.108 INFO:tasks.rados.rados.0.trial115.stdout:3047: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:04:36.184 INFO:tasks.rados.rados.0.trial115.stdout:3048: finishing copy_from racing read to trial11545420-48 2026-02-17T06:04:36.185 INFO:tasks.rados.rados.0.trial115.stdout:3048: finishing copy_from to trial11545420-48 2026-02-17T06:04:36.185 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1345 (ObjNum 1050 snap 312 seq_num 1050) dirty exists 2026-02-17T06:04:36.185 INFO:tasks.rados.rados.0.trial115.stdout:3054: seq_num 1065 ranges {699455=429605,1804617=576034,3163252=514929} 2026-02-17T06:04:36.188 INFO:tasks.rados.rados.0.trial115.stdout:3054: writing trial11545420-26 from 699455 to 1129060 tid 1 2026-02-17T06:04:36.193 INFO:tasks.rados.rados.0.trial115.stdout:3054: writing trial11545420-26 from 1804617 to 2380651 tid 2 2026-02-17T06:04:36.198 INFO:tasks.rados.rados.0.trial115.stdout:3054: writing trial11545420-26 from 3163252 to 3678181 tid 3 2026-02-17T06:04:36.198 INFO:tasks.rados.rados.0.trial115.stdout:3046: done (8 left) 2026-02-17T06:04:36.198 INFO:tasks.rados.rados.0.trial115.stdout:3047: done (7 left) 2026-02-17T06:04:36.198 INFO:tasks.rados.rados.0.trial115.stdout:3048: done (6 left) 2026-02-17T06:04:36.198 INFO:tasks.rados.rados.0.trial115.stdout:3055: delete oid 17 current snap is 319 2026-02-17T06:04:36.198 INFO:tasks.rados.rados.0.trial115.stdout:3050: expect (ObjNum 1060 snap 318 seq_num 1060) 2026-02-17T06:04:36.349 INFO:tasks.rados.rados.0.trial115.stdout:3051: finishing write tid 1 to trial11545420-19 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:3051: finishing write tid 2 to trial11545420-19 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:3051: finishing write tid 3 to trial11545420-19 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:3051: finishing write tid 4 to trial11545420-19 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:3051: finishing write tid 5 to trial11545420-19 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:3051: oid 19 updating version 0 to 1580 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:3051: oid 19 updating version 1580 to 1581 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:3051: oid 19 version 1581 is already newer than 1579 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:3051: oid 19 version 1581 is already newer than 1578 2026-02-17T06:04:36.350 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1581 (ObjNum 1063 snap 319 seq_num 1063) dirty exists 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3051: left oid 19 (ObjNum 1063 snap 319 seq_num 1063) 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3053: finishing write tid 1 to trial11545420-50 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3049: done (6 left) 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3050: done (5 left) 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3051: done (4 left) 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3052: done (3 left) 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3055: done (2 left) 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3056: copy_from oid 39 from oid 12 current snap is 319 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3053: finishing write tid 2 to trial11545420-50 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3053: finishing write tid 3 to trial11545420-50 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3053: finishing write tid 4 to trial11545420-50 2026-02-17T06:04:36.351 INFO:tasks.rados.rados.0.trial115.stdout:3053: finishing write tid 5 to trial11545420-50 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3053: oid 50 updating version 0 to 1351 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3053: oid 50 version 1351 is already newer than 1349 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3053: oid 50 version 1351 is already newer than 1348 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3053: oid 50 version 1351 is already newer than 1350 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1351 (ObjNum 1064 snap 319 seq_num 1064) dirty exists 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3053: left oid 50 (ObjNum 1064 snap 319 seq_num 1064) 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3054: finishing write tid 1 to trial11545420-26 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3054: finishing write tid 2 to trial11545420-26 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3054: finishing write tid 3 to trial11545420-26 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3054: finishing write tid 4 to trial11545420-26 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3054: finishing write tid 5 to trial11545420-26 2026-02-17T06:04:36.352 INFO:tasks.rados.rados.0.trial115.stdout:3054: oid 26 updating version 0 to 1451 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3054: oid 26 updating version 1451 to 1452 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3054: oid 26 updating version 1452 to 1453 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3054: oid 26 version 1453 is already newer than 1450 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1453 (ObjNum 1065 snap 319 seq_num 1065) dirty exists 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3054: left oid 26 (ObjNum 1065 snap 319 seq_num 1065) 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3053: done (2 left) 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3054: done (1 left) 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3057: read oid 32 snap 318 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3057: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3058: read oid 45 snap -1 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3058: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:04:36.353 INFO:tasks.rados.rados.0.trial115.stdout:3059: snap_remove snap 302 2026-02-17T06:04:36.674 INFO:tasks.rados.rados.0.trial115.stdout:3059: done (3 left) 2026-02-17T06:04:36.674 INFO:tasks.rados.rados.0.trial115.stdout:3060: snap_create 2026-02-17T06:04:36.674 INFO:tasks.rados.rados.0.trial115.stdout:3058: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:04:36.793 INFO:tasks.rados.rados.0.trial115.stdout:3057: expect (ObjNum 991 snap 284 seq_num 991) 2026-02-17T06:04:36.868 INFO:tasks.rados.rados.0.trial115.stdout:3056: finishing copy_from racing read to trial11545420-39 2026-02-17T06:04:36.868 INFO:tasks.rados.rados.0.trial115.stdout:3056: finishing copy_from to trial11545420-39 2026-02-17T06:04:36.868 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1451 (ObjNum 1049 snap 312 seq_num 1049) dirty exists 2026-02-17T06:04:37.675 INFO:tasks.rados.rados.0.trial115.stdout:3056: done (3 left) 2026-02-17T06:04:37.675 INFO:tasks.rados.rados.0.trial115.stdout:3057: done (2 left) 2026-02-17T06:04:37.675 INFO:tasks.rados.rados.0.trial115.stdout:3058: done (1 left) 2026-02-17T06:04:37.675 INFO:tasks.rados.rados.0.trial115.stdout:3060: done (0 left) 2026-02-17T06:04:37.675 INFO:tasks.rados.rados.0.trial115.stdout:3061: read oid 7 snap -1 2026-02-17T06:04:37.675 INFO:tasks.rados.rados.0.trial115.stdout:3061: expect (ObjNum 1019 snap 297 seq_num 1019) 2026-02-17T06:04:37.675 INFO:tasks.rados.rados.0.trial115.stdout:3062: snap_create 2026-02-17T06:04:37.678 INFO:tasks.rados.rados.0.trial115.stdout:3061: expect (ObjNum 1019 snap 297 seq_num 1019) 2026-02-17T06:04:38.676 INFO:tasks.rados.rados.0.trial115.stdout:3061: done (1 left) 2026-02-17T06:04:38.676 INFO:tasks.rados.rados.0.trial115.stdout:3062: done (0 left) 2026-02-17T06:04:38.676 INFO:tasks.rados.rados.0.trial115.stdout:3063: snap_remove snap 310 2026-02-17T06:04:39.677 INFO:tasks.rados.rados.0.trial115.stdout:3063: done (0 left) 2026-02-17T06:04:39.677 INFO:tasks.rados.rados.0.trial115.stdout:3064: snap_create 2026-02-17T06:04:40.678 INFO:tasks.rados.rados.0.trial115.stdout:3064: done (0 left) 2026-02-17T06:04:40.678 INFO:tasks.rados.rados.0.trial115.stdout:3065: write oid 45 current snap is 322 2026-02-17T06:04:40.678 INFO:tasks.rados.rados.0.trial115.stdout:3065: seq_num 1067 ranges {733069=580103,1921581=579703,3218098=537473} 2026-02-17T06:04:40.684 INFO:tasks.rados.rados.0.trial115.stdout:3065: writing trial11545420-45 from 733069 to 1313172 tid 1 2026-02-17T06:04:40.689 INFO:tasks.rados.rados.0.trial115.stdout:3065: writing trial11545420-45 from 1921581 to 2501284 tid 2 2026-02-17T06:04:40.694 INFO:tasks.rados.rados.0.trial115.stdout:3065: writing trial11545420-45 from 3218098 to 3755571 tid 3 2026-02-17T06:04:40.694 INFO:tasks.rados.rados.0.trial115.stdout:3066: write (excl) oid 14 current snap is 322 2026-02-17T06:04:40.694 INFO:tasks.rados.rados.0.trial115.stdout:3065: finishing write tid 1 to trial11545420-45 2026-02-17T06:04:40.694 INFO:tasks.rados.rados.0.trial115.stdout:3065: finishing write tid 2 to trial11545420-45 2026-02-17T06:04:40.694 INFO:tasks.rados.rados.0.trial115.stdout:3066: seq_num 1068 ranges {749876=655352,1863710=723361,3012815=748745} 2026-02-17T06:04:40.699 INFO:tasks.rados.rados.0.trial115.stdout:3066: writing trial11545420-14 from 749876 to 1405228 tid 1 2026-02-17T06:04:40.704 INFO:tasks.rados.rados.0.trial115.stdout:3066: writing trial11545420-14 from 1863710 to 2587071 tid 2 2026-02-17T06:04:40.709 INFO:tasks.rados.rados.0.trial115.stdout:3066: writing trial11545420-14 from 3012815 to 3761560 tid 3 2026-02-17T06:04:40.709 INFO:tasks.rados.rados.0.trial115.stdout:3067: write oid 25 current snap is 322 2026-02-17T06:04:40.709 INFO:tasks.rados.rados.0.trial115.stdout:3065: finishing write tid 3 to trial11545420-45 2026-02-17T06:04:40.709 INFO:tasks.rados.rados.0.trial115.stdout:3065: finishing write tid 4 to trial11545420-45 2026-02-17T06:04:40.709 INFO:tasks.rados.rados.0.trial115.stdout:3065: finishing write tid 5 to trial11545420-45 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:3065: oid 45 updating version 0 to 1560 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:3065: oid 45 updating version 1560 to 1561 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:3065: oid 45 updating version 1561 to 1562 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:3065: oid 45 updating version 1562 to 1563 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1563 (ObjNum 1067 snap 322 seq_num 1067) dirty exists 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:3065: left oid 45 (ObjNum 1067 snap 322 seq_num 1067) 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:3066: finishing write tid 1 to trial11545420-14 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:3066: finishing write tid 2 to trial11545420-14 2026-02-17T06:04:40.710 INFO:tasks.rados.rados.0.trial115.stdout:3067: seq_num 1069 ranges {766683=730601,1922192=583372,3156591=676370} 2026-02-17T06:04:40.714 INFO:tasks.rados.rados.0.trial115.stdout:3067: writing trial11545420-25 from 766683 to 1497284 tid 1 2026-02-17T06:04:40.718 INFO:tasks.rados.rados.0.trial115.stdout:3067: writing trial11545420-25 from 1922192 to 2505564 tid 2 2026-02-17T06:04:40.724 INFO:tasks.rados.rados.0.trial115.stdout:3067: writing trial11545420-25 from 3156591 to 3832961 tid 3 2026-02-17T06:04:40.724 INFO:tasks.rados.rados.0.trial115.stdout:3065: done (2 left) 2026-02-17T06:04:40.724 INFO:tasks.rados.rados.0.trial115.stdout:3068: write oid 28 current snap is 322 2026-02-17T06:04:40.724 INFO:tasks.rados.rados.0.trial115.stdout:3066: finishing write tid 3 to trial11545420-14 2026-02-17T06:04:40.724 INFO:tasks.rados.rados.0.trial115.stdout:3066: finishing write tid 4 to trial11545420-14 2026-02-17T06:04:40.724 INFO:tasks.rados.rados.0.trial115.stdout:3068: seq_num 1070 ranges {783490=405850,1980674=727030,3184014=487642} 2026-02-17T06:04:40.727 INFO:tasks.rados.rados.0.trial115.stdout:3068: writing trial11545420-28 from 783490 to 1189340 tid 1 2026-02-17T06:04:40.733 INFO:tasks.rados.rados.0.trial115.stdout:3068: writing trial11545420-28 from 1980674 to 2707704 tid 2 2026-02-17T06:04:40.737 INFO:tasks.rados.rados.0.trial115.stdout:3068: writing trial11545420-28 from 3184014 to 3671656 tid 3 2026-02-17T06:04:40.737 INFO:tasks.rados.rados.0.trial115.stdout:3069: snap_remove snap 316 2026-02-17T06:04:41.683 INFO:tasks.rados.rados.0.trial115.stdout:3069: done (3 left) 2026-02-17T06:04:41.684 INFO:tasks.rados.rados.0.trial115.stdout:3070: write (excl) oid 12 current snap is 322 2026-02-17T06:04:41.684 INFO:tasks.rados.rados.0.trial115.stdout:3070: seq_num 1071 ranges {400297=481099,1639156=361141} 2026-02-17T06:04:41.687 INFO:tasks.rados.rados.0.trial115.stdout:3070: writing trial11545420-12 from 400297 to 881396 tid 1 2026-02-17T06:04:41.692 INFO:tasks.rados.rados.0.trial115.stdout:3070: writing trial11545420-12 from 1639156 to 2000297 tid 2 2026-02-17T06:04:41.692 INFO:tasks.rados.rados.0.trial115.stdout:3071: snap_create 2026-02-17T06:04:41.692 INFO:tasks.rados.rados.0.trial115.stdout:3066: finishing write tid 5 to trial11545420-14 2026-02-17T06:04:41.692 INFO:tasks.rados.rados.0.trial115.stdout:3066: oid 14 updating version 0 to 1453 2026-02-17T06:04:41.692 INFO:tasks.rados.rados.0.trial115.stdout:3066: oid 14 version 1453 is already newer than 1452 2026-02-17T06:04:41.692 INFO:tasks.rados.rados.0.trial115.stdout:3066: oid 14 updating version 1453 to 1455 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3066: oid 14 version 1455 is already newer than 1454 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1455 (ObjNum 1068 snap 322 seq_num 1068) dirty exists 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3066: left oid 14 (ObjNum 1068 snap 322 seq_num 1068) 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3067: finishing write tid 1 to trial11545420-25 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3067: finishing write tid 2 to trial11545420-25 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3067: finishing write tid 3 to trial11545420-25 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3067: finishing write tid 4 to trial11545420-25 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3067: finishing write tid 5 to trial11545420-25 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3067: oid 25 updating version 0 to 1544 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3067: oid 25 updating version 1544 to 1545 2026-02-17T06:04:41.693 INFO:tasks.rados.rados.0.trial115.stdout:3067: oid 25 updating version 1545 to 1547 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3067: oid 25 version 1547 is already newer than 1546 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1547 (ObjNum 1069 snap 322 seq_num 1069) dirty exists 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3067: left oid 25 (ObjNum 1069 snap 322 seq_num 1069) 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: finishing write tid 1 to trial11545420-28 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: finishing write tid 2 to trial11545420-28 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: finishing write tid 3 to trial11545420-28 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: finishing write tid 4 to trial11545420-28 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: finishing write tid 5 to trial11545420-28 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: oid 28 updating version 0 to 1533 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: oid 28 updating version 1533 to 1535 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: oid 28 version 1535 is already newer than 1534 2026-02-17T06:04:41.694 INFO:tasks.rados.rados.0.trial115.stdout:3068: oid 28 version 1535 is already newer than 1532 2026-02-17T06:04:41.695 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1535 (ObjNum 1070 snap 322 seq_num 1070) dirty exists 2026-02-17T06:04:41.695 INFO:tasks.rados.rados.0.trial115.stdout:3068: left oid 28 (ObjNum 1070 snap 322 seq_num 1070) 2026-02-17T06:04:41.695 INFO:tasks.rados.rados.0.trial115.stdout:3070: finishing write tid 1 to trial11545420-12 2026-02-17T06:04:41.696 INFO:tasks.rados.rados.0.trial115.stdout:3070: finishing write tid 2 to trial11545420-12 2026-02-17T06:04:41.696 INFO:tasks.rados.rados.0.trial115.stdout:3070: finishing write tid 3 to trial11545420-12 2026-02-17T06:04:41.696 INFO:tasks.rados.rados.0.trial115.stdout:3070: finishing write tid 4 to trial11545420-12 2026-02-17T06:04:41.696 INFO:tasks.rados.rados.0.trial115.stdout:3070: oid 12 updating version 0 to 1567 2026-02-17T06:04:41.696 INFO:tasks.rados.rados.0.trial115.stdout:3070: oid 12 updating version 1567 to 1568 2026-02-17T06:04:41.696 INFO:tasks.rados.rados.0.trial115.stdout:3070: oid 12 updating version 1568 to 1569 2026-02-17T06:04:41.696 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1569 (ObjNum 1071 snap 322 seq_num 1071) dirty exists 2026-02-17T06:04:41.696 INFO:tasks.rados.rados.0.trial115.stdout:3070: left oid 12 (ObjNum 1071 snap 322 seq_num 1071) 2026-02-17T06:04:42.686 INFO:tasks.rados.rados.0.trial115.stdout:3066: done (4 left) 2026-02-17T06:04:42.686 INFO:tasks.rados.rados.0.trial115.stdout:3067: done (3 left) 2026-02-17T06:04:42.686 INFO:tasks.rados.rados.0.trial115.stdout:3068: done (2 left) 2026-02-17T06:04:42.686 INFO:tasks.rados.rados.0.trial115.stdout:3070: done (1 left) 2026-02-17T06:04:42.686 INFO:tasks.rados.rados.0.trial115.stdout:3071: done (0 left) 2026-02-17T06:04:42.686 INFO:tasks.rados.rados.0.trial115.stdout:3072: snap_create 2026-02-17T06:04:43.670 INFO:tasks.rados.rados.0.trial115.stdout:3072: done (0 left) 2026-02-17T06:04:43.670 INFO:tasks.rados.rados.0.trial115.stdout:3073: read oid 14 snap 312 2026-02-17T06:04:43.670 INFO:tasks.rados.rados.0.trial115.stdout:3073: expect (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:04:43.670 INFO:tasks.rados.rados.0.trial115.stdout:3074: read oid 47 snap -1 2026-02-17T06:04:43.670 INFO:tasks.rados.rados.0.trial115.stdout:3074: expect deleted 2026-02-17T06:04:43.671 INFO:tasks.rados.rados.0.trial115.stdout:3075: delete oid 26 current snap is 324 2026-02-17T06:04:43.673 INFO:tasks.rados.rados.0.trial115.stdout:3074: done (2 left) 2026-02-17T06:04:43.674 INFO:tasks.rados.rados.0.trial115.stdout:3075: done (1 left) 2026-02-17T06:04:43.674 INFO:tasks.rados.rados.0.trial115.stdout:3076: read oid 49 snap -1 2026-02-17T06:04:43.674 INFO:tasks.rados.rados.0.trial115.stdout:3076: expect deleted 2026-02-17T06:04:43.674 INFO:tasks.rados.rados.0.trial115.stdout:3077: write oid 33 current snap is 324 2026-02-17T06:04:43.674 INFO:tasks.rados.rados.0.trial115.stdout:3077: seq_num 1073 ranges {433911=747950,1756120=277791} 2026-02-17T06:04:43.678 INFO:tasks.rados.rados.0.trial115.stdout:3077: writing trial11545420-33 from 433911 to 1181861 tid 1 2026-02-17T06:04:43.682 INFO:tasks.rados.rados.0.trial115.stdout:3077: writing trial11545420-33 from 1756120 to 2033911 tid 2 2026-02-17T06:04:43.682 INFO:tasks.rados.rados.0.trial115.stdout:3078: delete oid 20 current snap is 324 2026-02-17T06:04:43.682 INFO:tasks.rados.rados.0.trial115.stdout:3073: expect (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:04:43.799 INFO:tasks.rados.rados.0.trial115.stdout:3077: finishing write tid 1 to trial11545420-33 2026-02-17T06:04:43.800 INFO:tasks.rados.rados.0.trial115.stdout:3077: finishing write tid 2 to trial11545420-33 2026-02-17T06:04:43.800 INFO:tasks.rados.rados.0.trial115.stdout:3077: finishing write tid 3 to trial11545420-33 2026-02-17T06:04:43.800 INFO:tasks.rados.rados.0.trial115.stdout:3073: done (3 left) 2026-02-17T06:04:43.800 INFO:tasks.rados.rados.0.trial115.stdout:3076: done (2 left) 2026-02-17T06:04:43.800 INFO:tasks.rados.rados.0.trial115.stdout:3078: done (1 left) 2026-02-17T06:04:43.800 INFO:tasks.rados.rados.0.trial115.stdout:3079: snap_remove snap 320 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3079: done (1 left) 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3080: rollback oid 32 current snap is 324 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 315 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3077: finishing write tid 4 to trial11545420-33 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3077: oid 33 updating version 0 to 1457 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3077: oid 33 updating version 1457 to 1458 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3077: oid 33 version 1458 is already newer than 1456 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1458 (ObjNum 1073 snap 324 seq_num 1073) dirty exists 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3077: left oid 33 (ObjNum 1073 snap 324 seq_num 1073) 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3077: done (1 left) 2026-02-17T06:04:44.688 INFO:tasks.rados.rados.0.trial115.stdout:3081: rollback oid 29 current snap is 324 2026-02-17T06:04:44.689 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 299 2026-02-17T06:04:44.689 INFO:tasks.rados.rados.0.trial115.stdout:3082: delete oid 47 current snap is 324 2026-02-17T06:04:44.689 INFO:tasks.rados.rados.0.trial115.stdout:3082: done (2 left) 2026-02-17T06:04:44.689 INFO:tasks.rados.rados.0.trial115.stdout:3083: rollback oid 35 current snap is 324 2026-02-17T06:04:44.690 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 35 to 319 2026-02-17T06:04:44.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: write (excl) oid 17 current snap is 324 2026-02-17T06:04:44.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: seq_num 1076 ranges {484332=573697,1531566=552766} 2026-02-17T06:04:44.693 INFO:tasks.rados.rados.0.trial115.stdout:3084: writing trial11545420-17 from 484332 to 1058029 tid 1 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3084: writing trial11545420-17 from 1531566 to 2084332 tid 2 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3085: copy_from oid 43 from oid 7 current snap is 324 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3081: finishing rollback tid 1 to trial11545420-29 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3081: finishing rollback tid 2 to trial11545420-29 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1465 (ObjNum 917 snap 265 seq_num 917) dirty exists 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3080: finishing rollback tid 0 to trial11545420-32 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3080: finishing rollback tid 1 to trial11545420-32 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3080: finishing rollback tid 2 to trial11545420-32 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1425 (ObjNum 991 snap 284 seq_num 991) dirty exists 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3083: finishing rollback tid 0 to trial11545420-35 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3083: finishing rollback tid 1 to trial11545420-35 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3083: finishing rollback tid 2 to trial11545420-35 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1591 (ObjNum 948 snap 276 seq_num 948) dirty exists 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3084: finishing write tid 1 to trial11545420-17 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3080: done (4 left) 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3081: done (3 left) 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3083: done (2 left) 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3086: rollback oid 18 current snap is 324 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 307 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3087: read oid 41 snap 312 2026-02-17T06:04:44.698 INFO:tasks.rados.rados.0.trial115.stdout:3087: expect (ObjNum 959 snap 278 seq_num 959) 2026-02-17T06:04:44.699 INFO:tasks.rados.rados.0.trial115.stdout:3088: read oid 36 snap -1 2026-02-17T06:04:44.699 INFO:tasks.rados.rados.0.trial115.stdout:3088: expect deleted 2026-02-17T06:04:44.699 INFO:tasks.rados.rados.0.trial115.stdout:3089: snap_remove snap 322 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3089: done (5 left) 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3090: delete oid 46 current snap is 324 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: finishing write tid 2 to trial11545420-17 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: finishing write tid 3 to trial11545420-17 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: finishing write tid 4 to trial11545420-17 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: oid 17 updating version 0 to 1412 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: oid 17 version 1412 is already newer than 1410 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: oid 17 version 1412 is already newer than 1411 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1412 (ObjNum 1076 snap 324 seq_num 1076) dirty exists 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3084: left oid 17 (ObjNum 1076 snap 324 seq_num 1076) 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3086: finishing rollback tid 0 to trial11545420-18 2026-02-17T06:04:45.690 INFO:tasks.rados.rados.0.trial115.stdout:3086: finishing rollback tid 1 to trial11545420-18 2026-02-17T06:04:45.691 INFO:tasks.rados.rados.0.trial115.stdout:3086: finishing rollback tid 2 to trial11545420-18 2026-02-17T06:04:45.691 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1550 (ObjNum 991 snap 284 seq_num 991) dirty exists 2026-02-17T06:04:45.691 INFO:tasks.rados.rados.0.trial115.stdout:3085: finishing copy_from racing read to trial11545420-43 2026-02-17T06:04:45.691 INFO:tasks.rados.rados.0.trial115.stdout:3085: finishing copy_from to trial11545420-43 2026-02-17T06:04:45.691 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1441 (ObjNum 1019 snap 297 seq_num 1019) dirty exists 2026-02-17T06:04:45.691 INFO:tasks.rados.rados.0.trial115.stdout:3087: expect (ObjNum 959 snap 278 seq_num 959) 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3084: done (5 left) 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3085: done (4 left) 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3086: done (3 left) 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3087: done (2 left) 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3088: done (1 left) 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3090: done (0 left) 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3091: read oid 35 snap -1 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3091: expect (ObjNum 948 snap 276 seq_num 948) 2026-02-17T06:04:45.741 INFO:tasks.rados.rados.0.trial115.stdout:3092: delete oid 23 current snap is 324 2026-02-17T06:04:45.742 INFO:tasks.rados.rados.0.trial115.stdout:3092: done (1 left) 2026-02-17T06:04:45.743 INFO:tasks.rados.rados.0.trial115.stdout:3093: copy_from oid 23 from oid 21 current snap is 324 2026-02-17T06:04:45.743 INFO:tasks.rados.rados.0.trial115.stdout:3094: snap_create 2026-02-17T06:04:45.743 INFO:tasks.rados.rados.0.trial115.stdout:3093: finishing copy_from to trial11545420-23 2026-02-17T06:04:45.743 INFO:tasks.rados.rados.0.trial115.stdout:3093: got expected ENOENT (src dne) 2026-02-17T06:04:45.743 INFO:tasks.rados.rados.0.trial115.stdout:3093: finishing copy_from racing read to trial11545420-23 2026-02-17T06:04:45.749 INFO:tasks.rados.rados.0.trial115.stdout:3091: expect (ObjNum 948 snap 276 seq_num 948) 2026-02-17T06:04:46.693 INFO:tasks.rados.rados.0.trial115.stdout:3091: done (2 left) 2026-02-17T06:04:46.694 INFO:tasks.rados.rados.0.trial115.stdout:3093: done (1 left) 2026-02-17T06:04:46.694 INFO:tasks.rados.rados.0.trial115.stdout:3094: done (0 left) 2026-02-17T06:04:46.694 INFO:tasks.rados.rados.0.trial115.stdout:3095: delete oid 37 current snap is 325 2026-02-17T06:04:46.697 INFO:tasks.rados.rados.0.trial115.stdout:3095: done (0 left) 2026-02-17T06:04:46.697 INFO:tasks.rados.rados.0.trial115.stdout:3096: rollback oid 2 current snap is 325 2026-02-17T06:04:46.697 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 2 to 315 2026-02-17T06:04:46.697 INFO:tasks.rados.rados.0.trial115.stdout:3097: read oid 23 snap -1 2026-02-17T06:04:46.697 INFO:tasks.rados.rados.0.trial115.stdout:3097: expect deleted 2026-02-17T06:04:46.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: write oid 41 current snap is 325 2026-02-17T06:04:46.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: seq_num 1080 ranges {551560=591046,1765494=386066} 2026-02-17T06:04:46.701 INFO:tasks.rados.rados.0.trial115.stdout:3098: writing trial11545420-41 from 551560 to 1142606 tid 1 2026-02-17T06:04:46.705 INFO:tasks.rados.rados.0.trial115.stdout:3098: writing trial11545420-41 from 1765494 to 2151560 tid 2 2026-02-17T06:04:46.705 INFO:tasks.rados.rados.0.trial115.stdout:3099: read oid 13 snap 323 2026-02-17T06:04:46.705 INFO:tasks.rados.rados.0.trial115.stdout:3099: expect deleted 2026-02-17T06:04:46.705 INFO:tasks.rados.rados.0.trial115.stdout:3096: finishing rollback tid 0 to trial11545420-2 2026-02-17T06:04:46.705 INFO:tasks.rados.rados.0.trial115.stdout:3096: finishing rollback tid 1 to trial11545420-2 2026-02-17T06:04:46.705 INFO:tasks.rados.rados.0.trial115.stdout:3096: finishing rollback tid 2 to trial11545420-2 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1452 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:3098: finishing write tid 1 to trial11545420-41 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:3096: done (3 left) 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:3097: done (2 left) 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:3100: copy_from oid 49 from oid 29 current snap is 325 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:3101: read oid 1 snap -1 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:3101: expect deleted 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:3102: write (excl) oid 32 current snap is 325 2026-02-17T06:04:46.706 INFO:tasks.rados.rados.0.trial115.stdout:3102: seq_num 1081 ranges {568367=666295,1707623=460744} 2026-02-17T06:04:46.709 INFO:tasks.rados.rados.0.trial115.stdout:3102: writing trial11545420-32 from 568367 to 1234662 tid 1 2026-02-17T06:04:46.713 INFO:tasks.rados.rados.0.trial115.stdout:3102: writing trial11545420-32 from 1707623 to 2168367 tid 2 2026-02-17T06:04:46.713 INFO:tasks.rados.rados.0.trial115.stdout:3103: write (excl) oid 50 current snap is 325 2026-02-17T06:04:46.713 INFO:tasks.rados.rados.0.trial115.stdout:3103: seq_num 1082 ranges {585174=741544,1766105=419069} 2026-02-17T06:04:46.718 INFO:tasks.rados.rados.0.trial115.stdout:3103: writing trial11545420-50 from 585174 to 1326718 tid 1 2026-02-17T06:04:46.722 INFO:tasks.rados.rados.0.trial115.stdout:3103: writing trial11545420-50 from 1766105 to 2185174 tid 2 2026-02-17T06:04:46.722 INFO:tasks.rados.rados.0.trial115.stdout:3104: read oid 34 snap 299 2026-02-17T06:04:46.722 INFO:tasks.rados.rados.0.trial115.stdout:3104: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:04:46.722 INFO:tasks.rados.rados.0.trial115.stdout:3105: write (excl) oid 14 current snap is 325 2026-02-17T06:04:46.722 INFO:tasks.rados.rados.0.trial115.stdout:3105: seq_num 1083 ranges {601981=416793,1424587=492702} 2026-02-17T06:04:46.725 INFO:tasks.rados.rados.0.trial115.stdout:3105: writing trial11545420-14 from 601981 to 1018774 tid 1 2026-02-17T06:04:46.729 INFO:tasks.rados.rados.0.trial115.stdout:3105: writing trial11545420-14 from 1424587 to 1917289 tid 2 2026-02-17T06:04:46.729 INFO:tasks.rados.rados.0.trial115.stdout:3106: snap_remove snap 319 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3106: done (8 left) 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3107: delete oid 35 current snap is 325 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: finishing write tid 2 to trial11545420-41 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: finishing write tid 3 to trial11545420-41 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: finishing write tid 4 to trial11545420-41 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: oid 41 updating version 0 to 1551 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: oid 41 updating version 1551 to 1553 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: oid 41 version 1553 is already newer than 1552 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1553 (ObjNum 1080 snap 325 seq_num 1080) dirty exists 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3098: left oid 41 (ObjNum 1080 snap 325 seq_num 1080) 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3102: finishing write tid 1 to trial11545420-32 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3100: finishing copy_from racing read to trial11545420-49 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3100: finishing copy_from to trial11545420-49 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1465 (ObjNum 917 snap 265 seq_num 917) dirty exists 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3102: finishing write tid 2 to trial11545420-32 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3102: finishing write tid 3 to trial11545420-32 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3102: finishing write tid 4 to trial11545420-32 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3102: oid 32 updating version 0 to 1428 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3102: oid 32 version 1428 is already newer than 1427 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:3102: oid 32 version 1428 is already newer than 1426 2026-02-17T06:04:47.697 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1428 (ObjNum 1081 snap 325 seq_num 1081) dirty exists 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3102: left oid 32 (ObjNum 1081 snap 325 seq_num 1081) 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3103: finishing write tid 1 to trial11545420-50 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3103: finishing write tid 2 to trial11545420-50 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3103: finishing write tid 3 to trial11545420-50 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3103: finishing write tid 4 to trial11545420-50 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3103: oid 50 updating version 0 to 1353 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3103: oid 50 version 1353 is already newer than 1352 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3103: oid 50 updating version 1353 to 1354 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1354 (ObjNum 1082 snap 325 seq_num 1082) dirty exists 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3103: left oid 50 (ObjNum 1082 snap 325 seq_num 1082) 2026-02-17T06:04:47.698 INFO:tasks.rados.rados.0.trial115.stdout:3104: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:04:47.816 INFO:tasks.rados.rados.0.trial115.stdout:3105: finishing write tid 1 to trial11545420-14 2026-02-17T06:04:47.816 INFO:tasks.rados.rados.0.trial115.stdout:3105: finishing write tid 2 to trial11545420-14 2026-02-17T06:04:47.816 INFO:tasks.rados.rados.0.trial115.stdout:3105: finishing write tid 3 to trial11545420-14 2026-02-17T06:04:47.816 INFO:tasks.rados.rados.0.trial115.stdout:3105: finishing write tid 4 to trial11545420-14 2026-02-17T06:04:47.816 INFO:tasks.rados.rados.0.trial115.stdout:3105: oid 14 updating version 0 to 1460 2026-02-17T06:04:47.816 INFO:tasks.rados.rados.0.trial115.stdout:3105: oid 14 version 1460 is already newer than 1459 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3105: oid 14 updating version 1460 to 1461 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1461 (ObjNum 1083 snap 325 seq_num 1083) dirty exists 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3105: left oid 14 (ObjNum 1083 snap 325 seq_num 1083) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3098: done (8 left) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3099: done (7 left) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3100: done (6 left) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3101: done (5 left) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3102: done (4 left) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3103: done (3 left) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3104: done (2 left) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3105: done (1 left) 2026-02-17T06:04:47.817 INFO:tasks.rados.rados.0.trial115.stdout:3107: done (0 left) 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:3108: rollback oid 24 current snap is 325 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 301 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:3109: read oid 29 snap -1 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:3109: expect (ObjNum 917 snap 265 seq_num 917) 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:3110: read oid 39 snap 314 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:3110: expect deleted 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:3111: read oid 2 snap -1 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:3111: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:04:47.818 INFO:tasks.rados.rados.0.trial115.stdout:3112: snap_create 2026-02-17T06:04:47.819 INFO:tasks.rados.rados.0.trial115.stdout:3108: finishing rollback tid 0 to trial11545420-24 2026-02-17T06:04:47.819 INFO:tasks.rados.rados.0.trial115.stdout:3108: finishing rollback tid 1 to trial11545420-24 2026-02-17T06:04:47.819 INFO:tasks.rados.rados.0.trial115.stdout:3108: finishing rollback tid 2 to trial11545420-24 2026-02-17T06:04:47.819 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1581 (ObjNum 923 snap 271 seq_num 923) dirty exists 2026-02-17T06:04:47.819 INFO:tasks.rados.rados.0.trial115.stdout:3109: expect (ObjNum 917 snap 265 seq_num 917) 2026-02-17T06:04:47.899 INFO:tasks.rados.rados.0.trial115.stdout:3111: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:04:48.699 INFO:tasks.rados.rados.0.trial115.stdout:3108: done (4 left) 2026-02-17T06:04:48.699 INFO:tasks.rados.rados.0.trial115.stdout:3109: done (3 left) 2026-02-17T06:04:48.699 INFO:tasks.rados.rados.0.trial115.stdout:3110: done (2 left) 2026-02-17T06:04:48.699 INFO:tasks.rados.rados.0.trial115.stdout:3111: done (1 left) 2026-02-17T06:04:48.699 INFO:tasks.rados.rados.0.trial115.stdout:3112: done (0 left) 2026-02-17T06:04:48.699 INFO:tasks.rados.rados.0.trial115.stdout:3113: write (excl) oid 44 current snap is 326 2026-02-17T06:04:48.699 INFO:tasks.rados.rados.0.trial115.stdout:3113: seq_num 1085 ranges {635595=567291,1825198=410397} 2026-02-17T06:04:48.704 INFO:tasks.rados.rados.0.trial115.stdout:3113: writing trial11545420-44 from 635595 to 1202886 tid 1 2026-02-17T06:04:48.708 INFO:tasks.rados.rados.0.trial115.stdout:3113: writing trial11545420-44 from 1825198 to 2235595 tid 2 2026-02-17T06:04:48.708 INFO:tasks.rados.rados.0.trial115.stdout:3114: rollback oid 6 current snap is 326 2026-02-17T06:04:48.708 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 6 to 318 2026-02-17T06:04:48.708 INFO:tasks.rados.rados.0.trial115.stdout:3113: finishing write tid 1 to trial11545420-44 2026-02-17T06:04:48.708 INFO:tasks.rados.rados.0.trial115.stdout:3115: snap_create 2026-02-17T06:04:48.710 INFO:tasks.rados.rados.0.trial115.stdout:3113: finishing write tid 2 to trial11545420-44 2026-02-17T06:04:48.710 INFO:tasks.rados.rados.0.trial115.stdout:3113: finishing write tid 3 to trial11545420-44 2026-02-17T06:04:48.711 INFO:tasks.rados.rados.0.trial115.stdout:3113: finishing write tid 4 to trial11545420-44 2026-02-17T06:04:48.711 INFO:tasks.rados.rados.0.trial115.stdout:3113: oid 44 updating version 0 to 1429 2026-02-17T06:04:48.711 INFO:tasks.rados.rados.0.trial115.stdout:3113: oid 44 updating version 1429 to 1431 2026-02-17T06:04:48.711 INFO:tasks.rados.rados.0.trial115.stdout:3113: oid 44 version 1431 is already newer than 1430 2026-02-17T06:04:48.711 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1431 (ObjNum 1085 snap 326 seq_num 1085) dirty exists 2026-02-17T06:04:48.711 INFO:tasks.rados.rados.0.trial115.stdout:3113: left oid 44 (ObjNum 1085 snap 326 seq_num 1085) 2026-02-17T06:04:48.712 INFO:tasks.rados.rados.0.trial115.stdout:3114: finishing rollback tid 0 to trial11545420-6 2026-02-17T06:04:48.712 INFO:tasks.rados.rados.0.trial115.stdout:3114: finishing rollback tid 1 to trial11545420-6 2026-02-17T06:04:48.712 INFO:tasks.rados.rados.0.trial115.stdout:3114: finishing rollback tid 2 to trial11545420-6 2026-02-17T06:04:48.712 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 1482 (ObjNum 948 snap 276 seq_num 948) dirty exists 2026-02-17T06:04:49.700 INFO:tasks.rados.rados.0.trial115.stdout:3113: done (2 left) 2026-02-17T06:04:49.700 INFO:tasks.rados.rados.0.trial115.stdout:3114: done (1 left) 2026-02-17T06:04:49.700 INFO:tasks.rados.rados.0.trial115.stdout:3115: done (0 left) 2026-02-17T06:04:49.700 INFO:tasks.rados.rados.0.trial115.stdout:3116: delete oid 12 current snap is 327 2026-02-17T06:04:49.701 INFO:tasks.rados.rados.0.trial115.stdout:3116: done (0 left) 2026-02-17T06:04:49.701 INFO:tasks.rados.rados.0.trial115.stdout:3117: snap_remove snap 312 2026-02-17T06:04:50.703 INFO:tasks.rados.rados.0.trial115.stdout:3117: done (0 left) 2026-02-17T06:04:50.704 INFO:tasks.rados.rados.0.trial115.stdout:3118: rollback oid 38 current snap is 327 2026-02-17T06:04:50.704 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 38 to 324 2026-02-17T06:04:50.704 INFO:tasks.rados.rados.0.trial115.stdout:3119: snap_remove snap 315 2026-02-17T06:04:51.255 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T06:04:51.255 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-17T06:04:51.256 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-17T06:04:51.256 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-17T06:04:51.259 INFO:tasks.ceph.mon.c:Started 2026-02-17T06:04:51.259 INFO:tasks.mon_thrash:Sending STOP to mon b 2026-02-17T06:04:51.260 INFO:tasks.ceph.mon.b:Sent signal 19 2026-02-17T06:04:51.260 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T06:05:06.260 INFO:tasks.mon_thrash:Sending CONT to mon b 2026-02-17T06:05:06.261 INFO:tasks.ceph.mon.b:Sent signal 18 2026-02-17T06:05:06.261 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:05:06.262 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:06.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.330+0000 7f6c2b133640 10 monclient: get_monmap_and_config 2026-02-17T06:05:06.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.330+0000 7f6c2b133640 10 monclient: build_initial_monmap 2026-02-17T06:05:06.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.330+0000 7f6c2b133640 10 monclient: monmap: 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:06.331605+0000 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:06.331605+0000 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:06.334 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.330+0000 7f6c2b133640 1 Processor -- start 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.330+0000 7f6c2b133640 1 -- start start 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.330+0000 7f6c2b133640 10 monclient: init 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient: _add_conns ranks=[3,1,2,0,4] 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient(hunting): picked mon.noname-a con 0x7f6c2406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6c2409d430 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient(hunting): picked mon.noname-b con 0x7f6c2409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:06.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c24073600 con 0x7f6c2409d430 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2409ca50 con 0x7f6c2406c890 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2406d6f0 con 0x7f6c2409bf40 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c2b133640 10 monclient(hunting): authenticate will time out at 1751.885164s 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45522/0 (socket says 10.20.193.78:45522) 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 learned_addr learned my addr 10.20.193.78:0/1133198011 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:06.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:34252/0 (socket says 10.20.193.78:34252) 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 10 monclient(hunting): get_auth_request con 0x7f6c2409bf40 auth_method 0 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request con 0x7f6c2406c890 auth_method 0 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:06.337 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:06.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:06.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:06.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:06.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.331+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:06.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.333+0000 7f6c1bfff640 1 -- 10.20.193.78:0/1133198011 >> v1:10.20.193.115:6789/0 conn(0x7f6c2409d430 legacy=0x7f6c24134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:06.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:06.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:06.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 10 monclient(hunting): get_auth_request con 0x7f6c2409bf40 auth_method 0 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request con 0x7f6c2406c890 auth_method 0 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:06.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:05:06.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:06.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:06.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:06.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.532+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:06.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:06.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 10 monclient(hunting): get_auth_request con 0x7f6c2409bf40 auth_method 0 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request con 0x7f6c2406c890 auth_method 0 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:06.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:06.932+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:05:07.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 10 monclient(hunting): get_auth_request con 0x7f6c2409bf40 auth_method 0 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request con 0x7f6c2406c890 auth_method 0 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c296a9640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:07.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:07.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:07.733+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:05:09.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 10 monclient(hunting): tick 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 monclient(hunting): continuing hunt 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 msgr2=0x7f6c2406d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c2409bf40 0x7f6c2406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 msgr2=0x7f6c2406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.78:3300/0 conn(0x7f6c2406c890 0x7f6c2406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 -- 10.20.193.78:0/1133198011 >> v1:10.20.193.115:6789/0 conn(0x7f6c2409d430 legacy=0x7f6c24134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 10 monclient: _add_conns ranks=[0,4] 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6c08003820 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:09.334 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 10 monclient(hunting): picked mon.noname-c con 0x7f6c08007550 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 -- 10.20.193.78:0/1133198011 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2406dab0 con 0x7f6c08003820 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 1 -- 10.20.193.78:0/1133198011 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2409ca50 con 0x7f6c08007550 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1a7fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request con 0x7f6c08007550 auth_method 0 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:09.335 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 msgr2=0x7f6c08009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:05:09.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 msgr2=0x7f6c08009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:09.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:09.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.332+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:09.336 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.333+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> v1:10.20.193.78:6789/0 conn(0x7f6c08003820 legacy=0x7f6c08005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request con 0x7f6c08007550 auth_method 0 2026-02-17T06:05:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 msgr2=0x7f6c08009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:05:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 msgr2=0x7f6c08009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:09.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.532+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request con 0x7f6c08007550 auth_method 0 2026-02-17T06:05:09.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 msgr2=0x7f6c08009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:05:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 msgr2=0x7f6c08009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:09.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:09.933+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:05:10.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.708+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3019550778 0 0) 0x7f6c2406dab0 con 0x7f6c08003820 2026-02-17T06:05:10.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.708+0000 7f6c1b7fe640 10 monclient(hunting): _negotiate 2026-02-17T06:05:10.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.708+0000 7f6c1b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:10.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.708+0000 7f6c1b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:10.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.708+0000 7f6c1b7fe640 10 monclient(hunting): my global_id is 184554 2026-02-17T06:05:10.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.708+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c10003620 con 0x7f6c08003820 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1479700585 0 0) 0x7f6c10003620 con 0x7f6c08003820 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 msgr2=0x7f6c08009a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 --2- 10.20.193.78:0/1133198011 >> v2:10.20.193.115:3300/0 conn(0x7f6c08007550 0x7f6c08009a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0800b130 con 0x7f6c08003820 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:10.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f6c1400f040 con 0x7f6c08003820 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: dump: 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 >> v1:10.20.193.78:6789/0 conn(0x7f6c08003820 legacy=0x7f6c08005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c296a9640 1 -- 10.20.193.78:0/1133198011 reap_dead start 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10003ba0 0x7f6c10003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:10.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient(hunting): picked mon.a con 0x7f6c10003ba0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10004900 0x7f6c10004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient(hunting): picked mon.c con 0x7f6c10004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c10005510 0x7f6c1000bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient(hunting): picked mon.b con 0x7f6c10005510 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2409ca50 con 0x7f6c10003ba0 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1000c420 con 0x7f6c10005510 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1000c620 con 0x7f6c10004900 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10003ba0 0x7f6c10003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request con 0x7f6c10003ba0 auth_method 0 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10004900 0x7f6c10004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request con 0x7f6c10004900 auth_method 0 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:10.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient(hunting): handle_auth_done global_id 184554 payload 293 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10004900 msgr2=0x7f6c10004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10004900 0x7f6c10004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c10005510 msgr2=0x7f6c1000bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c10005510 0x7f6c1000bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient: _finish_hunting 0 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 1 monclient: found mon.a 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c10003620 con 0x7f6c10003ba0 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient: _finish_auth 0 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 10 monclient: _check_auth_tickets 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c28ea8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c2b133640 5 monclient: authenticate success, global_id 184554 2026-02-17T06:05:10.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.709+0000 7f6c2b133640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c28ea8640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10003ba0 msgr2=0x7f6c10003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10003ba0 0x7f6c10003fa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6c14006fb0 tx=0x7f6c14030480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cca223ba20a7fd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14036070 con 0x7f6c10003ba0 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 10 monclient: dump: 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1403a400 con 0x7f6c10003ba0 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 1 -- 10.20.193.78:0/1133198011 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c140142f0 con 0x7f6c10003ba0 2026-02-17T06:05:10.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 4 set_mon_vals no callback set 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c1b7fe640 10 monclient: dump: 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 10 monclient: shutdown 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 -- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10003ba0 msgr2=0x7f6c10003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10003ba0 0x7f6c10003fa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6c14006fb0 tx=0x7f6c14030480 comp rx=0 tx=0).stop 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 -- 10.20.193.78:0/1133198011 shutdown_connections 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c10005510 0x7f6c1000bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10004900 0x7f6c10004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 --2- 10.20.193.78:0/1133198011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10003ba0 0x7f6c10003fa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 -- 10.20.193.78:0/1133198011 >> 10.20.193.78:0/1133198011 conn(0x7f6c240a6620 msgr2=0x7f6c240a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 -- 10.20.193.78:0/1133198011 shutdown_connections 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.712+0000 7f6c2b133640 1 -- 10.20.193.78:0/1133198011 wait complete. 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient: build_initial_monmap 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient: monmap: 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 1 Processor -- start 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 1 -- start start 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient: init 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:05:10.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c10003ba0 0x7f6c241203f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient(hunting): picked mon.noname-b con 0x7f6c10003ba0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10004900 0x7f6c24120930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient(hunting): picked mon.noname-a con 0x7f6c10004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 0x7f6c24120e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient(hunting): picked mon.noname-c con 0x7f6c10005510 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24073380 con 0x7f6c10004900 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24073200 con 0x7f6c10003ba0 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2406cc70 con 0x7f6c10005510 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c2b133640 10 monclient(hunting): authenticate will time out at 1756.267792s 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10004900 0x7f6c24120930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10004900 0x7f6c24120930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:56974/0 (socket says 10.20.193.78:56974) 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 -- 10.20.193.78:0/981014531 learned_addr learned my addr 10.20.193.78:0/981014531 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request con 0x7f6c10004900 auth_method 0 2026-02-17T06:05:10.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 0x7f6c24120e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 10 monclient(hunting): get_auth_request con 0x7f6c10005510 auth_method 0 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient(hunting): handle_auth_done global_id 184557 payload 824 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 1 -- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 msgr2=0x7f6c24120e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 1 -- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 msgr2=0x7f6c24120e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 0x7f6c24120e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 -- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 msgr2=0x7f6c24120e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c296a9640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 0x7f6c24120e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 0x7f6c24120e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 -- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c10003ba0 msgr2=0x7f6c241203f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c10003ba0 0x7f6c241203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 monclient: found mon.noname-a 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.713+0000 7f6c1bfff640 1 -- 10.20.193.78:0/981014531 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c240a2030 con 0x7f6c10004900 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c1bfff640 1 -- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10004900 msgr2=0x7f6c24120930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c1bfff640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10004900 0x7f6c24120930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c014990 tx=0x7f6c0c014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 1 -- 10.20.193.78:0/981014531 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c01a070 con 0x7f6c10004900 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:10.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: dump: 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: _finish_auth 0 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: _check_auth_tickets 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 5 monclient: authenticate success, global_id 184557 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 10 monclient: _renew_subs 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c24124110 con 0x7f6c10004900 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 1 -- 10.20.193.78:0/981014531 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0c005ce0 con 0x7f6c10004900 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 1 -- 10.20.193.78:0/981014531 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0129e0 con 0x7f6c10004900 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c1affd640 4 set_mon_vals no callback set 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 10 monclient: dump: 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 10 monclient: _renew_subs 2026-02-17T06:05:10.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2409d810 con 0x7f6c10004900 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c24074980 con 0x7f6c10004900 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.716+0000 7f6c197fa640 1 -- 10.20.193.78:0/981014531 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6c0c012b80 con 0x7f6c10004900 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.717+0000 7f6c197fa640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf403deb0 0x7f6bf4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.717+0000 7f6c197fa640 1 -- 10.20.193.78:0/981014531 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(670..670 src has 525..670) ==== 4827+0+0 (secure 0 0 0) 0x7f6c0c053130 con 0x7f6c10004900 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.717+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf403deb0 0x7f6bf4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.717+0000 7f6c28ea8640 10 monclient: get_auth_request con 0x7f6bf403deb0 auth_method 0 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.719+0000 7f6c197fa640 1 -- 10.20.193.78:0/981014531 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6c0c01ede0 con 0x7f6c10004900 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.719+0000 7f6c197fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.719+0000 7f6c197fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:10.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.720+0000 7f6c28ea8640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf403deb0 0x7f6bf4040370 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6c14038000 tx=0x7f6c14032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:10.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.833+0000 7f6c2b133640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:05:10.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.833+0000 7f6c2b133640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:10.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.833+0000 7f6c2b133640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:10.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.833+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6c2406cff0 con 0x7f6c10004900 2026-02-17T06:05:10.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c197fa640 1 -- 10.20.193.78:0/981014531 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f6c0c007070 con 0x7f6c10004900 2026-02-17T06:05:10.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c197fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:10.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c197fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:10.836 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":216,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:05:10.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf403deb0 msgr2=0x7f6bf4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:10.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c2b133640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf403deb0 0x7f6bf4040370 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6c14038000 tx=0x7f6c14032000 comp rx=0 tx=0).stop 2026-02-17T06:05:10.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c2b133640 10 monclient: shutdown 2026-02-17T06:05:10.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c2b133640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10004900 msgr2=0x7f6c24120930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.834+0000 7f6c2b133640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10004900 0x7f6c24120930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c014990 tx=0x7f6c0c014e60 comp rx=0 tx=0).stop 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.835+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 shutdown_connections 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.835+0000 7f6c2b133640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6c10005510 0x7f6c24120e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.835+0000 7f6c2b133640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6c10004900 0x7f6c24120930 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.835+0000 7f6c2b133640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6bf403deb0 0x7f6bf4040370 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.835+0000 7f6c2b133640 1 --2- 10.20.193.78:0/981014531 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6c10003ba0 0x7f6c241203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.835+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 >> 10.20.193.78:0/981014531 conn(0x7f6c240a6620 msgr2=0x7f6c241c0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.835+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 shutdown_connections 2026-02-17T06:05:10.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:10.835+0000 7f6c2b133640 1 -- 10.20.193.78:0/981014531 wait complete. 2026-02-17T06:05:11.715 INFO:tasks.rados.rados.0.trial115.stdout:3119: done (1 left) 2026-02-17T06:05:11.715 INFO:tasks.rados.rados.0.trial115.stdout:3120: read oid 37 snap -1 2026-02-17T06:05:11.715 INFO:tasks.rados.rados.0.trial115.stdout:3120: expect deleted 2026-02-17T06:05:11.715 INFO:tasks.rados.rados.0.trial115.stdout:3121: delete oid 4 current snap is 327 2026-02-17T06:05:11.715 INFO:tasks.rados.rados.0.trial115.stdout:3118: finishing rollback tid 0 to trial11545420-38 2026-02-17T06:05:11.716 INFO:tasks.rados.rados.0.trial115.stdout:3118: finishing rollback tid 1 to trial11545420-38 2026-02-17T06:05:11.716 INFO:tasks.rados.rados.0.trial115.stdout:3118: finishing rollback tid 2 to trial11545420-38 2026-02-17T06:05:11.716 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1601 (ObjNum 1005 snap 294 seq_num 1005) dirty exists 2026-02-17T06:05:11.718 INFO:tasks.rados.rados.0.trial115.stdout:3118: done (2 left) 2026-02-17T06:05:11.718 INFO:tasks.rados.rados.0.trial115.stdout:3120: done (1 left) 2026-02-17T06:05:11.718 INFO:tasks.rados.rados.0.trial115.stdout:3121: done (0 left) 2026-02-17T06:05:11.718 INFO:tasks.rados.rados.0.trial115.stdout:3122: snap_create 2026-02-17T06:05:12.724 INFO:tasks.rados.rados.0.trial115.stdout:3122: done (0 left) 2026-02-17T06:05:12.724 INFO:tasks.rados.rados.0.trial115.stdout:3123: read oid 5 snap -1 2026-02-17T06:05:12.724 INFO:tasks.rados.rados.0.trial115.stdout:3123: expect deleted 2026-02-17T06:05:12.724 INFO:tasks.rados.rados.0.trial115.stdout:3124: read oid 27 snap 326 2026-02-17T06:05:12.724 INFO:tasks.rados.rados.0.trial115.stdout:3124: expect deleted 2026-02-17T06:05:12.724 INFO:tasks.rados.rados.0.trial115.stdout:3125: copy_from oid 47 from oid 46 current snap is 328 2026-02-17T06:05:12.724 INFO:tasks.rados.rados.0.trial115.stdout:3126: snap_remove snap 321 2026-02-17T06:05:13.673 INFO:tasks.rados.rados.0.trial115.stdout:3126: done (3 left) 2026-02-17T06:05:13.673 INFO:tasks.rados.rados.0.trial115.stdout:3127: rollback oid 21 current snap is 328 2026-02-17T06:05:13.673 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 21 to 307 2026-02-17T06:05:13.673 INFO:tasks.rados.rados.0.trial115.stdout:3125: finishing copy_from to trial11545420-47 2026-02-17T06:05:13.673 INFO:tasks.rados.rados.0.trial115.stdout:3125: got expected ENOENT (src dne) 2026-02-17T06:05:13.673 INFO:tasks.rados.rados.0.trial115.stdout:3125: finishing copy_from racing read to trial11545420-47 2026-02-17T06:05:13.674 INFO:tasks.rados.rados.0.trial115.stdout:3123: done (3 left) 2026-02-17T06:05:13.674 INFO:tasks.rados.rados.0.trial115.stdout:3125: done (2 left) 2026-02-17T06:05:13.674 INFO:tasks.rados.rados.0.trial115.stdout:3128: write (excl) oid 14 current snap is 328 2026-02-17T06:05:13.674 INFO:tasks.rados.rados.0.trial115.stdout:3128: seq_num 1088 ranges {686016=509391,1716997=569019} 2026-02-17T06:05:13.677 INFO:tasks.rados.rados.0.trial115.stdout:3128: writing trial11545420-14 from 686016 to 1195407 tid 1 2026-02-17T06:05:13.682 INFO:tasks.rados.rados.0.trial115.stdout:3128: writing trial11545420-14 from 1716997 to 2286016 tid 2 2026-02-17T06:05:13.683 INFO:tasks.rados.rados.0.trial115.stdout:3129: copy_from oid 32 from oid 2 current snap is 328 2026-02-17T06:05:13.683 INFO:tasks.rados.rados.0.trial115.stdout:3127: finishing rollback tid 1 to trial11545420-21 2026-02-17T06:05:13.683 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1412 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:05:13.683 INFO:tasks.rados.rados.0.trial115.stdout:3124: done (3 left) 2026-02-17T06:05:13.683 INFO:tasks.rados.rados.0.trial115.stdout:3127: done (2 left) 2026-02-17T06:05:13.683 INFO:tasks.rados.rados.0.trial115.stdout:3130: write (excl) oid 31 current snap is 328 2026-02-17T06:05:13.683 INFO:tasks.rados.rados.0.trial115.stdout:3130: seq_num 1089 ranges {702823=584640,2059126=243697} 2026-02-17T06:05:13.687 INFO:tasks.rados.rados.0.trial115.stdout:3130: writing trial11545420-31 from 702823 to 1287463 tid 1 2026-02-17T06:05:13.691 INFO:tasks.rados.rados.0.trial115.stdout:3130: writing trial11545420-31 from 2059126 to 2302823 tid 2 2026-02-17T06:05:13.692 INFO:tasks.rados.rados.0.trial115.stdout:3131: write (excl) oid 20 current snap is 328 2026-02-17T06:05:13.692 INFO:tasks.rados.rados.0.trial115.stdout:3131: seq_num 1090 ranges {719630=659889,2117608=202022} 2026-02-17T06:05:13.697 INFO:tasks.rados.rados.0.trial115.stdout:3131: writing trial11545420-20 from 719630 to 1379519 tid 1 2026-02-17T06:05:13.701 INFO:tasks.rados.rados.0.trial115.stdout:3131: writing trial11545420-20 from 2117608 to 2319630 tid 2 2026-02-17T06:05:13.701 INFO:tasks.rados.rados.0.trial115.stdout:3132: write oid 33 current snap is 328 2026-02-17T06:05:13.701 INFO:tasks.rados.rados.0.trial115.stdout:3132: seq_num 1091 ranges {736437=735138,2176090=160347} 2026-02-17T06:05:13.707 INFO:tasks.rados.rados.0.trial115.stdout:3132: writing trial11545420-33 from 736437 to 1471575 tid 1 2026-02-17T06:05:13.710 INFO:tasks.rados.rados.0.trial115.stdout:3132: writing trial11545420-33 from 2176090 to 2336437 tid 2 2026-02-17T06:05:13.710 INFO:tasks.rados.rados.0.trial115.stdout:3133: read oid 47 snap 314 2026-02-17T06:05:13.711 INFO:tasks.rados.rados.0.trial115.stdout:3133: expect deleted 2026-02-17T06:05:13.711 INFO:tasks.rados.rados.0.trial115.stdout:3134: read oid 36 snap -1 2026-02-17T06:05:13.711 INFO:tasks.rados.rados.0.trial115.stdout:3134: expect deleted 2026-02-17T06:05:13.711 INFO:tasks.rados.rados.0.trial115.stdout:3135: write oid 43 current snap is 328 2026-02-17T06:05:13.711 INFO:tasks.rados.rados.0.trial115.stdout:3128: finishing write tid 1 to trial11545420-14 2026-02-17T06:05:13.711 INFO:tasks.rados.rados.0.trial115.stdout:3135: seq_num 1092 ranges {753244=410387,1834572=518672} 2026-02-17T06:05:13.714 INFO:tasks.rados.rados.0.trial115.stdout:3135: writing trial11545420-43 from 753244 to 1163631 tid 1 2026-02-17T06:05:13.720 INFO:tasks.rados.rados.0.trial115.stdout:3135: writing trial11545420-43 from 1834572 to 2353244 tid 2 2026-02-17T06:05:13.720 INFO:tasks.rados.rados.0.trial115.stdout:3136: snap_create 2026-02-17T06:05:13.720 INFO:tasks.rados.rados.0.trial115.stdout:3128: finishing write tid 2 to trial11545420-14 2026-02-17T06:05:13.720 INFO:tasks.rados.rados.0.trial115.stdout:3128: finishing write tid 3 to trial11545420-14 2026-02-17T06:05:13.720 INFO:tasks.rados.rados.0.trial115.stdout:3128: finishing write tid 4 to trial11545420-14 2026-02-17T06:05:13.720 INFO:tasks.rados.rados.0.trial115.stdout:3128: oid 14 updating version 0 to 1463 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3128: oid 14 updating version 1463 to 1464 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3128: oid 14 version 1464 is already newer than 1462 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1464 (ObjNum 1088 snap 328 seq_num 1088) dirty exists 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3128: left oid 14 (ObjNum 1088 snap 328 seq_num 1088) 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3129: finishing copy_from racing read to trial11545420-32 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3129: finishing copy_from to trial11545420-32 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1452 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3130: finishing write tid 1 to trial11545420-31 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3130: finishing write tid 2 to trial11545420-31 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3130: finishing write tid 3 to trial11545420-31 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3130: finishing write tid 4 to trial11545420-31 2026-02-17T06:05:13.721 INFO:tasks.rados.rados.0.trial115.stdout:3130: oid 31 updating version 0 to 1453 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3130: oid 31 updating version 1453 to 1454 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3130: oid 31 updating version 1454 to 1455 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1455 (ObjNum 1089 snap 328 seq_num 1089) dirty exists 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3130: left oid 31 (ObjNum 1089 snap 328 seq_num 1089) 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3131: finishing write tid 1 to trial11545420-20 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3131: finishing write tid 2 to trial11545420-20 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3131: finishing write tid 3 to trial11545420-20 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3131: finishing write tid 4 to trial11545420-20 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3131: oid 20 updating version 0 to 1463 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3131: oid 20 updating version 1463 to 1464 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:3131: oid 20 updating version 1464 to 1465 2026-02-17T06:05:13.722 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1465 (ObjNum 1090 snap 328 seq_num 1090) dirty exists 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3131: left oid 20 (ObjNum 1090 snap 328 seq_num 1090) 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3132: finishing write tid 1 to trial11545420-33 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3132: finishing write tid 2 to trial11545420-33 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3132: finishing write tid 3 to trial11545420-33 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3132: finishing write tid 4 to trial11545420-33 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3132: oid 33 updating version 0 to 1465 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3132: oid 33 updating version 1465 to 1467 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3132: oid 33 version 1467 is already newer than 1466 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1467 (ObjNum 1091 snap 328 seq_num 1091) dirty exists 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3132: left oid 33 (ObjNum 1091 snap 328 seq_num 1091) 2026-02-17T06:05:13.723 INFO:tasks.rados.rados.0.trial115.stdout:3135: finishing write tid 1 to trial11545420-43 2026-02-17T06:05:13.724 INFO:tasks.rados.rados.0.trial115.stdout:3135: finishing write tid 2 to trial11545420-43 2026-02-17T06:05:13.724 INFO:tasks.rados.rados.0.trial115.stdout:3135: finishing write tid 3 to trial11545420-43 2026-02-17T06:05:13.725 INFO:tasks.rados.rados.0.trial115.stdout:3135: finishing write tid 4 to trial11545420-43 2026-02-17T06:05:13.725 INFO:tasks.rados.rados.0.trial115.stdout:3135: oid 43 updating version 0 to 1467 2026-02-17T06:05:13.725 INFO:tasks.rados.rados.0.trial115.stdout:3135: oid 43 version 1467 is already newer than 1466 2026-02-17T06:05:13.725 INFO:tasks.rados.rados.0.trial115.stdout:3135: oid 43 updating version 1467 to 1468 2026-02-17T06:05:13.725 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1468 (ObjNum 1092 snap 328 seq_num 1092) dirty exists 2026-02-17T06:05:13.725 INFO:tasks.rados.rados.0.trial115.stdout:3135: left oid 43 (ObjNum 1092 snap 328 seq_num 1092) 2026-02-17T06:05:13.845 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.913+0000 7f6bd0eb8640 10 monclient: get_monmap_and_config 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.913+0000 7f6bd0eb8640 10 monclient: build_initial_monmap 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.913+0000 7f6bd0eb8640 10 monclient: monmap: 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:13.914894+0000 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:13.914894+0000 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:13.914 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:13.915 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:13.915 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:13.915 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:13.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 1 Processor -- start 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 1 -- start start 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient: init 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6bcc09ea90 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc09f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient(hunting): picked mon.noname-b con 0x7f6bcc09ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6bcc13d640 0x7f6bcc13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient(hunting): picked mon.noname-a con 0x7f6bcc13d640 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bcc0a0a20 con 0x7f6bcc09ea90 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc0a0d00 con 0x7f6bcc13d640 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc0a0460 con 0x7f6bcc09ee70 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bd0eb8640 10 monclient(hunting): authenticate will time out at 1759.468448s 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bca575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f6bcc09ea90 0x7f6bcc0a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:51744/0 (socket says 10.20.193.78:51744) 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bca575640 1 -- 10.20.193.78:0/3751376163 learned_addr learned my addr 10.20.193.78:0/3751376163 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bcad76640 1 --2- 10.20.193.78:0/3751376163 >> v2:10.20.193.78:3300/0 conn(0x7f6bcc13d640 0x7f6bcc13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 1 --2- 10.20.193.78:0/3751376163 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc09f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bcad76640 10 monclient(hunting): get_auth_request con 0x7f6bcc13d640 auth_method 0 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bcad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bcad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bcad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient(hunting): get_auth_request con 0x7f6bcc09ee70 auth_method 0 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bcad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bcad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bcad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:13.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient(hunting): handle_auth_done global_id 234101 payload 274 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 1 -- 10.20.193.78:0/3751376163 >> v2:10.20.193.78:3300/0 conn(0x7f6bcc13d640 msgr2=0x7f6bcc13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 1 --2- 10.20.193.78:0/3751376163 >> v2:10.20.193.78:3300/0 conn(0x7f6bcc13d640 0x7f6bcc13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 1 -- 10.20.193.78:0/3751376163 >> v1:10.20.193.78:6789/0 conn(0x7f6bcc09ea90 legacy=0x7f6bcc0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient: _finish_hunting 0 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 1 monclient: found mon.noname-b 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.914+0000 7f6bc9d74640 1 -- 10.20.193.78:0/3751376163 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc0a0740 con 0x7f6bcc09ee70 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9d74640 1 -- 10.20.193.78:0/3751376163 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 msgr2=0x7f6bcc09f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:13.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9d74640 1 --2- 10.20.193.78:0/3751376163 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc09f290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40098a0 tx=0x7f6bb402ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9960df1ce855773 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 1 -- 10.20.193.78:0/3751376163 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb403c070 con 0x7f6bcc09ee70 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: dump: 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: _finish_auth 0 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: _check_auth_tickets 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 1 -- 10.20.193.78:0/3751376163 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb402fc00 con 0x7f6bcc09ee70 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 1 -- 10.20.193.78:0/3751376163 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4037440 con 0x7f6bcc09ee70 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 5 monclient: authenticate success, global_id 234101 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 4 set_mon_vals no callback set 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:13.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bc9573640 10 monclient: dump: 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:13.919 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 10 monclient: shutdown 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/3751376163 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 msgr2=0x7f6bcc09f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/3751376163 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc09f290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40098a0 tx=0x7f6bb402ed80 comp rx=0 tx=0).stop 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/3751376163 shutdown_connections 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/3751376163 >> v2:10.20.193.78:3300/0 conn(0x7f6bcc13d640 0x7f6bcc13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/3751376163 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc09f290 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/3751376163 >> 10.20.193.78:0/3751376163 conn(0x7f6bcc0a7520 msgr2=0x7f6bcc0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/3751376163 shutdown_connections 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.917+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/3751376163 wait complete. 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient: build_initial_monmap 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient: monmap: 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:13.920 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 1 Processor -- start 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 1 -- start start 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient: init 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bcc09ea90 0x7f6bcc0a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient(hunting): picked mon.noname-a con 0x7f6bcc09ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc0a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient(hunting): picked mon.noname-c con 0x7f6bcc09ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bcc13d640 0x7f6bcc0a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient(hunting): picked mon.noname-b con 0x7f6bcc13d640 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0a6820 con 0x7f6bcc09ea90 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0a66a0 con 0x7f6bcc13d640 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc13ff70 con 0x7f6bcc09ee70 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bd0eb8640 10 monclient(hunting): authenticate will time out at 1759.472784s 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc0a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bca575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bcc09ea90 0x7f6bcc0a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc0a3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48460/0 (socket says 10.20.193.78:48460) 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 1 -- 10.20.193.78:0/2210072554 learned_addr learned my addr 10.20.193.78:0/2210072554 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient(hunting): get_auth_request con 0x7f6bcc09ee70 auth_method 0 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bca575640 10 monclient(hunting): get_auth_request con 0x7f6bcc09ea90 auth_method 0 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bca575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bca575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bca575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:13.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bca575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bca575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bca575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient(hunting): handle_auth_done global_id 234104 payload 824 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 1 -- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bcc13d640 msgr2=0x7f6bcc0a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bcc13d640 0x7f6bcc0a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 1 -- 10.20.193.78:0/2210072554 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bcc09ea90 msgr2=0x7f6bcc0a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bcc09ea90 0x7f6bcc0a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 10 monclient: _finish_hunting 0 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.918+0000 7f6bc9d74640 1 monclient: found mon.noname-c 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.919+0000 7f6bc9d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.919+0000 7f6bc9d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:13.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.919+0000 7f6bc9d74640 1 -- 10.20.193.78:0/2210072554 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc0711e0 con 0x7f6bcc09ee70 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bc9d74640 1 -- 10.20.193.78:0/2210072554 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 msgr2=0x7f6bcc0a3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bc9d74640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc0a3950 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6bb403a040 tx=0x7f6bb40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 1 -- 10.20.193.78:0/2210072554 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb403c040 con 0x7f6bcc09ee70 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: dump: 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 1 -- 10.20.193.78:0/2210072554 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb4040600 con 0x7f6bcc09ee70 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 1 -- 10.20.193.78:0/2210072554 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4040a10 con 0x7f6bcc09ee70 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 10 monclient: dump: 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:13.923 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bc8d72640 4 set_mon_vals no callback set 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 5 monclient: authenticate success, global_id 234104 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 10 monclient: _renew_subs 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc0a1c10 con 0x7f6bcc09ee70 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 10 monclient: _renew_subs 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc070940 con 0x7f6bcc09ee70 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bb2ffd640 1 -- 10.20.193.78:0/2210072554 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6bb403f030 con 0x7f6bcc09ee70 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.921+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc09f290 con 0x7f6bcc09ee70 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.922+0000 7f6bb2ffd640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b9803deb0 0x7f6b98040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.922+0000 7f6bb2ffd640 1 -- 10.20.193.78:0/2210072554 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(673..673 src has 525..673) ==== 4871+0+0 (secure 0 0 0) 0x7f6bb40777c0 con 0x7f6bcc09ee70 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.922+0000 7f6bca575640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b9803deb0 0x7f6b98040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:13.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.922+0000 7f6bca575640 10 monclient: get_auth_request con 0x7f6b9803deb0 auth_method 0 2026-02-17T06:05:13.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.924+0000 7f6bb2ffd640 1 -- 10.20.193.78:0/2210072554 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6bb4076160 con 0x7f6bcc09ee70 2026-02-17T06:05:13.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.924+0000 7f6bb2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:13.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.924+0000 7f6bb2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:13.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:13.925+0000 7f6bca575640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b9803deb0 0x7f6b98040370 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0007d90 tx=0x7f6bc0007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.038+0000 7f6bd0eb8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:05:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.038+0000 7f6bd0eb8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.038+0000 7f6bd0eb8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.038+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6bcc0008d0 con 0x7f6bcc09ee70 2026-02-17T06:05:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.038+0000 7f6bb2ffd640 1 -- 10.20.193.78:0/2210072554 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f6bb403fdc0 con 0x7f6bcc09ee70 2026-02-17T06:05:14.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.038+0000 7f6bb2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:14.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.038+0000 7f6bb2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:14.041 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":220,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:05:14.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b9803deb0 msgr2=0x7f6b98040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b9803deb0 0x7f6b98040370 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0007d90 tx=0x7f6bc0007ce0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 10 monclient: shutdown 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 msgr2=0x7f6bcc0a3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc0a3950 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6bb403a040 tx=0x7f6bb40047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 shutdown_connections 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b9803deb0 0x7f6b98040370 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bcc13d640 0x7f6bcc0a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bcc09ee70 0x7f6bcc0a3950 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.039+0000 7f6bd0eb8640 1 --2- 10.20.193.78:0/2210072554 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bcc09ea90 0x7f6bcc0a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.040+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 >> 10.20.193.78:0/2210072554 conn(0x7f6bcc0a7520 msgr2=0x7f6bcc13c860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.040+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 shutdown_connections 2026-02-17T06:05:14.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.040+0000 7f6bd0eb8640 1 -- 10.20.193.78:0/2210072554 wait complete. 2026-02-17T06:05:14.049 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:05:14.049 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:05:14.049 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:05:14.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.116+0000 7faf23054640 10 monclient: get_monmap_and_config 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.116+0000 7faf23054640 10 monclient: build_initial_monmap 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.116+0000 7faf23054640 10 monclient: monmap: 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:14.117912+0000 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:14.117912+0000 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.118 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:14.119 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:14.119 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:14.119 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:14.119 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:14.119 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 1 Processor -- start 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 1 -- start start 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient: init 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf1c06c890 0x7faf1c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient(hunting): picked mon.noname-a con 0x7faf1c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faf1c06cc70 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faf1c13a240 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1c074150 con 0x7faf1c13a240 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1c09ef70 con 0x7faf1c06cc70 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7faf1c09f250 con 0x7faf1c06c890 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf23054640 10 monclient(hunting): authenticate will time out at 1759.671694s 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf1c06c890 0x7faf1c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7faf1c06c890 0x7faf1c0a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:56992/0 (socket says 10.20.193.78:56992) 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 1 -- 10.20.193.78:0/3113308432 learned_addr learned my addr 10.20.193.78:0/3113308432 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient(hunting): get_auth_request con 0x7faf1c06c890 auth_method 0 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient(hunting): handle_auth_done global_id 184566 payload 274 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 1 -- 10.20.193.78:0/3113308432 >> v1:10.20.193.115:6789/0 conn(0x7faf1c06cc70 legacy=0x7faf1c06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 1 -- 10.20.193.78:0/3113308432 >> v1:10.20.193.78:6789/0 conn(0x7faf1c13a240 legacy=0x7faf1c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient: _finish_hunting 0 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 1 monclient: found mon.noname-a 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.117+0000 7faf20dc9640 1 -- 10.20.193.78:0/3113308432 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c09df40 con 0x7faf1c06c890 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf20dc9640 1 -- 10.20.193.78:0/3113308432 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 msgr2=0x7faf1c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf20dc9640 1 --2- 10.20.193.78:0/3113308432 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 0x7faf1c0a01f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009970 tx=0x7faf0c02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e503217acd0d1a0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 1 -- 10.20.193.78:0/3113308432 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c03c070 con 0x7faf1c06c890 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:14.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: dump: 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: _finish_auth 0 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: _check_auth_tickets 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 1 -- 10.20.193.78:0/3113308432 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c02fe50 con 0x7faf1c06c890 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 5 monclient: authenticate success, global_id 184566 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 1 -- 10.20.193.78:0/3113308432 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c037660 con 0x7faf1c06c890 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 4 set_mon_vals no callback set 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf137fe640 10 monclient: dump: 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.129 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 10 monclient: shutdown 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 1 -- 10.20.193.78:0/3113308432 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 msgr2=0x7faf1c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 1 --2- 10.20.193.78:0/3113308432 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 0x7faf1c0a01f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009970 tx=0x7faf0c02eef0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 1 -- 10.20.193.78:0/3113308432 shutdown_connections 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 1 --2- 10.20.193.78:0/3113308432 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 0x7faf1c0a01f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.120+0000 7faf23054640 1 -- 10.20.193.78:0/3113308432 >> 10.20.193.78:0/3113308432 conn(0x7faf1c0a6620 msgr2=0x7faf1c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 -- 10.20.193.78:0/3113308432 shutdown_connections 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 -- 10.20.193.78:0/3113308432 wait complete. 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient: build_initial_monmap 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient: monmap: 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 Processor -- start 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 -- start start 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient: init 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 0x7faf1c073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient(hunting): picked mon.noname-a con 0x7faf1c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf1c06cc70 0x7faf1c13a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient(hunting): picked mon.noname-c con 0x7faf1c06cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf1c13a240 0x7faf1c13ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient(hunting): picked mon.noname-b con 0x7faf1c13a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c13ded0 con 0x7faf1c06c890 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c13dd50 con 0x7faf1c13a240 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c13e050 con 0x7faf1c06cc70 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf23054640 10 monclient(hunting): authenticate will time out at 1759.675854s 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 0x7faf1c073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 0x7faf1c073300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57006/0 (socket says 10.20.193.78:57006) 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 1 -- 10.20.193.78:0/3360031174 learned_addr learned my addr 10.20.193.78:0/3360031174 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 10 monclient(hunting): get_auth_request con 0x7faf1c06c890 auth_method 0 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf13fff640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf1c06cc70 0x7faf1c13a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf20dc9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf13fff640 10 monclient(hunting): get_auth_request con 0x7faf1c06cc70 auth_method 0 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf13fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf13fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.121+0000 7faf13fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 10 monclient(hunting): handle_auth_done global_id 184569 payload 824 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf1c06cc70 msgr2=0x7faf1c13a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faf1c06cc70 0x7faf1c13a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf1c13a240 msgr2=0x7faf1c13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faf1c13a240 0x7faf1c13ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 10 monclient: _finish_hunting 0 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 1 monclient: found mon.noname-a 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.122+0000 7faf20dc9640 1 -- 10.20.193.78:0/3360031174 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c0715c0 con 0x7faf1c06c890 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf20dc9640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 msgr2=0x7faf1c073300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf20dc9640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 0x7faf1c073300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009970 tx=0x7faf0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c046070 con 0x7faf1c06c890 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: dump: 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.132 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: _finish_auth 0 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: _check_auth_tickets 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c037e10 con 0x7faf1c06c890 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 5 monclient: authenticate success, global_id 184569 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 10 monclient: _renew_subs 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c13d600 con 0x7faf1c06c890 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c03c040 con 0x7faf1c06c890 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf12ffd640 4 set_mon_vals no callback set 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf117fa640 10 monclient: dump: 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.133 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 10 monclient: _renew_subs 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c0712e0 con 0x7faf1c06c890 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c13b0a0 0x7faf1c136220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c13b480 con 0x7faf1c13b0a0 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf13fff640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c13b0a0 0x7faf1c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf13fff640 10 monclient: get_auth_request con 0x7faf1c13b0a0 auth_method 0 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf13fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf13fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf13fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.124+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.125+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7faf0c0418b0 con 0x7faf1c06c890 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.125+0000 7faf13fff640 10 monclient(hunting): handle_auth_done global_id 184572 payload 824 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.125+0000 7faf13fff640 1 -- 10.20.193.78:0/3360031174 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf1c071800 con 0x7faf1c13b0a0 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.125+0000 7faf117fa640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faeec03deb0 0x7faeec040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.125+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(673..673 src has 525..673) ==== 4871+0+0 (secure 0 0 0) 0x7faf0c077fc0 con 0x7faf1c06c890 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.125+0000 7faf215ca640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faeec03deb0 0x7faeec040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.125+0000 7faf215ca640 10 monclient: get_auth_request con 0x7faeec03deb0 auth_method 0 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf13fff640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c13b0a0 msgr2=0x7faf1c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf13fff640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c13b0a0 0x7faf1c136220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faf0400f770 tx=0x7faf0400fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04018030 con 0x7faf1c13b0a0 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 10 monclient: dump: 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7faf0400dbc0 con 0x7faf1c13b0a0 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c13b0a0 msgr2=0x7faf1c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c13b0a0 0x7faf1c136220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faf0400f770 tx=0x7faf0400fc40 comp rx=0 tx=0).stop 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c13b0a0 msgr2=0x7faf1c136220 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf117fa640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c13b0a0 0x7faf1c136220 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faf0400f770 tx=0x7faf0400fc40 comp rx=0 tx=0).stop 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.127+0000 7faf215ca640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faeec03deb0 0x7faeec040370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faf1c13bc70 tx=0x7faf18003fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf23054640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf23054640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c136760 0x7faf1c136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf23054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0a01d0 con 0x7faf1c136760 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf13fff640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c136760 0x7faf1c136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf13fff640 10 monclient: get_auth_request con 0x7faf1c136760 auth_method 0 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf13fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf13fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.130+0000 7faf13fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.131+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.131+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.131+0000 7faf13fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.131+0000 7faf13fff640 10 monclient(hunting): handle_auth_done global_id 184575 payload 824 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.131+0000 7faf13fff640 1 -- 10.20.193.78:0/3360031174 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faf1c0744d0 con 0x7faf1c136760 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf13fff640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c136760 msgr2=0x7faf1c136b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf13fff640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c136760 0x7faf1c136b70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faf0400f770 tx=0x7faf04003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04004230 con 0x7faf1c136760 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 10 monclient: dump: 2026-02-17T06:05:14.137 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7faf04004230 con 0x7faf1c136760 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c136760 msgr2=0x7faf1c136b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c136760 0x7faf1c136b70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faf0400f770 tx=0x7faf04003860 comp rx=0 tx=0).stop 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c136760 msgr2=0x7faf1c136b70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf117fa640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c136760 0x7faf1c136b70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 220, 2026-02-17T06:05:14.138 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 278913, 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.139 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:05:14.140 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:05:14.141 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:05:14.142 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.143 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:05:14.144 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faeec03deb0 msgr2=0x7faeec040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faeec03deb0 0x7faeec040370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faf1c13bc70 tx=0x7faf18003fc0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 10 monclient: shutdown 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 msgr2=0x7faf1c073300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 1 --2- 10.20.193.78:0/3360031174 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faf1c06c890 0x7faf1c073300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009970 tx=0x7faf0c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf215ca640 1 -- 10.20.193.78:0/3360031174 reap_dead start 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf215ca640 1 -- 10.20.193.78:0/3360031174 reap_dead start 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 shutdown_connections 2026-02-17T06:05:14.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 >> 10.20.193.78:0/3360031174 conn(0x7faf1c0a6620 msgr2=0x7faf1c09be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 shutdown_connections 2026-02-17T06:05:14.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.133+0000 7faf23054640 1 -- 10.20.193.78:0/3360031174 wait complete. 2026-02-17T06:05:14.146 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:05:14.146 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:05:14.146 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:05:14.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.212+0000 7f7381e4d640 10 monclient: get_monmap_and_config 2026-02-17T06:05:14.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.212+0000 7f7381e4d640 10 monclient: build_initial_monmap 2026-02-17T06:05:14.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.212+0000 7f7381e4d640 10 monclient: monmap: 2026-02-17T06:05:14.213 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:14.213772+0000 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:14.213772+0000 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.212+0000 7f7381e4d640 1 Processor -- start 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.212+0000 7f7381e4d640 1 -- start start 2026-02-17T06:05:14.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.212+0000 7f7381e4d640 10 monclient: init 2026-02-17T06:05:14.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient: _add_conns ranks=[1,2,3,0,4] 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f737c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 0x7f737c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient(hunting): picked mon.noname-b con 0x7f737c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f737c06d380 0x7f737c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient(hunting): picked mon.noname-a con 0x7f737c06d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f737c06daf0 con 0x7f737c06c890 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f737c0a0880 con 0x7f737c06d380 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f737c09cc80 con 0x7f737c09c4a0 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f7381e4d640 10 monclient(hunting): authenticate will time out at 1759.767291s 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f737c06d380 0x7f737c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f737c06d380 0x7f737c13d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57026/0 (socket says 10.20.193.78:57026) 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 1 -- 10.20.193.78:0/1471503822 learned_addr learned my addr 10.20.193.78:0/1471503822 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient(hunting): get_auth_request con 0x7f737c06d380 auth_method 0 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 0x7f737c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737affd640 10 monclient(hunting): get_auth_request con 0x7f737c09c4a0 auth_method 0 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient(hunting): handle_auth_done global_id 184578 payload 274 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 1 -- 10.20.193.78:0/1471503822 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 msgr2=0x7f737c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 1 --2- 10.20.193.78:0/1471503822 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 0x7f737c13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 1 -- 10.20.193.78:0/1471503822 >> v1:10.20.193.115:6789/0 conn(0x7f737c06c890 legacy=0x7f737c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 1 monclient: found mon.noname-a 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.213+0000 7f737bfff640 1 -- 10.20.193.78:0/1471503822 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c09cf60 con 0x7f737c06d380 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737bfff640 1 -- 10.20.193.78:0/1471503822 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06d380 msgr2=0x7f737c13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737bfff640 1 --2- 10.20.193.78:0/1471503822 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06d380 0x7f737c13d8f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7370009e50 tx=0x7f737002f260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3e2f68703e8b28 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 1 -- 10.20.193.78:0/1471503822 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73700373f0 con 0x7f737c06d380 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: dump: 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 1 -- 10.20.193.78:0/1471503822 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7370037590 con 0x7f737c06d380 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 5 monclient: authenticate success, global_id 184578 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 1 -- 10.20.193.78:0/1471503822 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370037960 con 0x7f737c06d380 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 4 set_mon_vals no callback set 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f737a7fc640 10 monclient: dump: 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.221 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 10 monclient: shutdown 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 1 -- 10.20.193.78:0/1471503822 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06d380 msgr2=0x7f737c13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 1 --2- 10.20.193.78:0/1471503822 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06d380 0x7f737c13d8f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7370009e50 tx=0x7f737002f260 comp rx=0 tx=0).stop 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 1 -- 10.20.193.78:0/1471503822 shutdown_connections 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 1 --2- 10.20.193.78:0/1471503822 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06d380 0x7f737c13d8f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 1 --2- 10.20.193.78:0/1471503822 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 0x7f737c13b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 1 -- 10.20.193.78:0/1471503822 >> 10.20.193.78:0/1471503822 conn(0x7f737c0a6620 msgr2=0x7f737c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 1 -- 10.20.193.78:0/1471503822 shutdown_connections 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 1 -- 10.20.193.78:0/1471503822 wait complete. 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 10 monclient: build_initial_monmap 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.216+0000 7f7381e4d640 10 monclient: monmap: 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:14.222 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 1 Processor -- start 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 1 -- start start 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient: init 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06c890 0x7f737c13c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient(hunting): picked mon.noname-a con 0x7f737c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f737c06d380 0x7f737c09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient(hunting): picked mon.noname-b con 0x7f737c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 0x7f737c09e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient(hunting): picked mon.noname-c con 0x7f737c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f737c09bfa0 con 0x7f737c06c890 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f737c09be20 con 0x7f737c06d380 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f737c13de30 con 0x7f737c09c4a0 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f7381e4d640 10 monclient(hunting): authenticate will time out at 1759.771399s 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06c890 0x7f737c13c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 0x7f737c09e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06c890 0x7f737c13c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57028/0 (socket says 10.20.193.78:57028) 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 -- 10.20.193.78:0/4198455766 learned_addr learned my addr 10.20.193.78:0/4198455766 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient(hunting): get_auth_request con 0x7f737c06c890 auth_method 0 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737bfff640 10 monclient(hunting): get_auth_request con 0x7f737c09c4a0 auth_method 0 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737affd640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f737c06d380 0x7f737c09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737affd640 10 monclient(hunting): get_auth_request con 0x7f737c06d380 auth_method 0 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient(hunting): handle_auth_done global_id 184581 payload 824 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 msgr2=0x7f737c09e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09c4a0 0x7f737c09e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f737c06d380 msgr2=0x7f737c09db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f737c06d380 0x7f737c09db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:05:14.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 monclient: found mon.noname-a 2026-02-17T06:05:14.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:14.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.217+0000 7f737b7fe640 1 -- 10.20.193.78:0/4198455766 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c13ba00 con 0x7f737c06c890 2026-02-17T06:05:14.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f737b7fe640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06c890 msgr2=0x7f737c13c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f737b7fe640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06c890 0x7f737c13c4a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f736800c5d0 tx=0x7f736800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368016020 con 0x7f737c06c890 2026-02-17T06:05:14.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:14.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: dump: 2026-02-17T06:05:14.227 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: _finish_auth 0 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: _check_auth_tickets 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7368012a60 con 0x7f737c06c890 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7381e4d640 5 monclient: authenticate success, global_id 184581 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7381e4d640 10 monclient: _renew_subs 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7381e4d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c13da00 con 0x7f737c06c890 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368012e50 con 0x7f737c06c890 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7379ffb640 4 set_mon_vals no callback set 2026-02-17T06:05:14.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.219+0000 7f7357fff640 10 monclient: dump: 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7381e4d640 10 monclient: _renew_subs 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7381e4d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c13b440 con 0x7f737c06c890 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7381e4d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7381e4d640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09e5a0 0x7f737c0716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7381e4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f737c06f650 con 0x7f737c09e5a0 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09e5a0 0x7f737c0716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 10 monclient: get_auth_request con 0x7f737c09e5a0 auth_method 0 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f736801b420 con 0x7f737c06c890 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 10 monclient(hunting): handle_auth_done global_id 234116 payload 824 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f737affd640 1 -- 10.20.193.78:0/4198455766 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f737c074980 con 0x7f737c09e5a0 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7357fff640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f734c03e0a0 0x7f734c040560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.220+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(673..673 src has 525..673) ==== 4871+0+0 (secure 0 0 0) 0x7f73680528e0 con 0x7f737c06c890 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.221+0000 7f737bfff640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f734c03e0a0 0x7f734c040560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.221+0000 7f737bfff640 10 monclient: get_auth_request con 0x7f734c03e0a0 auth_method 0 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.222+0000 7f737affd640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09e5a0 msgr2=0x7f737c0716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.222+0000 7f737affd640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09e5a0 0x7f737c0716c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f737c09f310 tx=0x7f736c011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.222+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c00fe20 con 0x7f737c09e5a0 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.222+0000 7f7357fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.222+0000 7f7357fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.222+0000 7f7357fff640 10 monclient: dump: 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.230 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.223+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f736c00fe20 con 0x7f737c09e5a0 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.223+0000 7f7357fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.223+0000 7f7357fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.223+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09e5a0 msgr2=0x7f737c0716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.223+0000 7f7357fff640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09e5a0 0x7f737c0716c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f737c09f310 tx=0x7f736c011ca0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.223+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09e5a0 msgr2=0x7f737c0716c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.223+0000 7f7357fff640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c09e5a0 0x7f737c0716c0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f737c09f310 tx=0x7f736c011ca0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.223+0000 7f737bfff640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f734c03e0a0 0x7f734c040560 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f737c06a480 tx=0x7f7370033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f7381e4d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f7381e4d640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c071c00 0x7f737c072010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f7381e4d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f737c06cc70 con 0x7f737c071c00 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c071c00 0x7f737c072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 10 monclient: get_auth_request con 0x7f737c071c00 auth_method 0 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 10 monclient(hunting): handle_auth_done global_id 234119 payload 824 2026-02-17T06:05:14.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.226+0000 7f737affd640 1 -- 10.20.193.78:0/4198455766 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f737c09e980 con 0x7f737c071c00 2026-02-17T06:05:14.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f737affd640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c071c00 msgr2=0x7f737c072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f737affd640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c071c00 0x7f737c072010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f737c09f310 tx=0x7f736c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c022b60 con 0x7f737c071c00 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 10 monclient: dump: 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f736c015460 con 0x7f737c071c00 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:14.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c071c00 msgr2=0x7f737c072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c071c00 0x7f737c072010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f737c09f310 tx=0x7f736c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c071c00 msgr2=0x7f737c072010 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7357fff640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f737c071c00 0x7f737c072010 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f737c09f310 tx=0x7f736c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 220, 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 22911, 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:05:14.234 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:05:14.235 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:14.236 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.237 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.238 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.239 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:05:14.240 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.241 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:05:14.242 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.228+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f734c03e0a0 msgr2=0x7f734c040560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f734c03e0a0 0x7f734c040560 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f737c06a480 tx=0x7f7370033000 comp rx=0 tx=0).stop 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 10 monclient: shutdown 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06c890 msgr2=0x7f737c13c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06c890 0x7f737c13c4a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f736800c5d0 tx=0x7f736800caa0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f737bfff640 1 -- 10.20.193.78:0/4198455766 reap_dead start 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 shutdown_connections 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 1 --2- 10.20.193.78:0/4198455766 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f737c06c890 0x7f737c13c4a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 >> 10.20.193.78:0/4198455766 conn(0x7f737c0a6620 msgr2=0x7f737c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 shutdown_connections 2026-02-17T06:05:14.244 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.229+0000 7f7381e4d640 1 -- 10.20.193.78:0/4198455766 wait complete. 2026-02-17T06:05:14.244 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:05:14.244 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:05:14.244 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.311+0000 7f6aeeb8f640 10 monclient: get_monmap_and_config 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.311+0000 7f6aeeb8f640 10 monclient: build_initial_monmap 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.311+0000 7f6aeeb8f640 10 monclient: monmap: 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:14.312601+0000 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:14.312601+0000 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:14.312 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.311+0000 7f6aeeb8f640 1 Processor -- start 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.311+0000 7f6aeeb8f640 1 -- start start 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.311+0000 7f6aeeb8f640 10 monclient: init 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6ae806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6ae806d380 0x7f6ae80a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 10 monclient(hunting): picked mon.noname-c con 0x7f6ae806d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6ae813a240 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae8074150 con 0x7f6ae813a240 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae809ef70 con 0x7f6ae806c890 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae809f250 con 0x7f6ae806d380 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aeeb8f640 10 monclient(hunting): authenticate will time out at 1759.866138s 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aed105640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f6ae813a240 0x7f6ae813c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:51760/0 (socket says 10.20.193.78:51760) 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6aed105640 1 -- 10.20.193.78:0/3018534389 learned_addr learned my addr 10.20.193.78:0/3018534389 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 1 --2- 10.20.193.78:0/3018534389 >> v2:10.20.193.115:3300/0 conn(0x7f6ae806d380 0x7f6ae80a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient(hunting): get_auth_request con 0x7f6ae806d380 auth_method 0 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient(hunting): handle_auth_done global_id 214231 payload 274 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 1 -- 10.20.193.78:0/3018534389 >> v1:10.20.193.115:6789/0 conn(0x7f6ae806c890 legacy=0x7f6ae806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 1 -- 10.20.193.78:0/3018534389 >> v1:10.20.193.78:6789/0 conn(0x7f6ae813a240 legacy=0x7f6ae813c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient: _finish_hunting 0 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 1 monclient: found mon.noname-c 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.312+0000 7f6adffff640 1 -- 10.20.193.78:0/3018534389 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae809df40 con 0x7f6ae806d380 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.315+0000 7f6adffff640 1 -- 10.20.193.78:0/3018534389 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806d380 msgr2=0x7f6ae80a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.315+0000 7f6adffff640 1 --2- 10.20.193.78:0/3018534389 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806d380 0x7f6ae80a01f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00098a0 tx=0x7f6ad00310a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e3a77dd083f1a51 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 1 -- 10.20.193.78:0/3018534389 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad003e070 con 0x7f6ae806d380 2026-02-17T06:05:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:14.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: dump: 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: _finish_auth 0 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 1 -- 10.20.193.78:0/3018534389 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad0039440 con 0x7f6ae806d380 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 1 -- 10.20.193.78:0/3018534389 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0039810 con 0x7f6ae806d380 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6aeeb8f640 5 monclient: authenticate success, global_id 214231 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6aeeb8f640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6aeeb8f640 4 set_mon_vals no callback set 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.316+0000 7f6adf7fe640 10 monclient: dump: 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.318 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 10 monclient: shutdown 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3018534389 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806d380 msgr2=0x7f6ae80a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 --2- 10.20.193.78:0/3018534389 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806d380 0x7f6ae80a01f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00098a0 tx=0x7f6ad00310a0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3018534389 shutdown_connections 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 --2- 10.20.193.78:0/3018534389 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806d380 0x7f6ae80a01f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3018534389 >> 10.20.193.78:0/3018534389 conn(0x7f6ae80a6620 msgr2=0x7f6ae80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3018534389 shutdown_connections 2026-02-17T06:05:14.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3018534389 wait complete. 2026-02-17T06:05:14.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 10 monclient: build_initial_monmap 2026-02-17T06:05:14.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 10 monclient: monmap: 2026-02-17T06:05:14.320 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 Processor -- start 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 1 -- start start 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.317+0000 7f6aeeb8f640 10 monclient: init 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ae806c890 0x7f6ae8134b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient(hunting): picked mon.noname-c con 0x7f6ae806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ae806d380 0x7f6ae8135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient(hunting): picked mon.noname-a con 0x7f6ae806d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae813a240 0x7f6ae8121970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient(hunting): picked mon.noname-b con 0x7f6ae813a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae809bfa0 con 0x7f6ae806d380 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae809be20 con 0x7f6ae813a240 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae809c120 con 0x7f6ae806c890 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aeeb8f640 10 monclient(hunting): authenticate will time out at 1759.872145s 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ae806c890 0x7f6ae8134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6adffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ae806d380 0x7f6ae8135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ae806c890 0x7f6ae8134b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48532/0 (socket says 10.20.193.78:48532) 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 -- 10.20.193.78:0/3160170452 learned_addr learned my addr 10.20.193.78:0/3160170452 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient(hunting): get_auth_request con 0x7f6ae806c890 auth_method 0 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6adffff640 10 monclient(hunting): get_auth_request con 0x7f6ae806d380 auth_method 0 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6adffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6adffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6adffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient(hunting): handle_auth_done global_id 234122 payload 824 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aed105640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae813a240 0x7f6ae8121970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae813a240 msgr2=0x7f6ae8121970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae813a240 0x7f6ae8121970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ae806d380 msgr2=0x7f6ae8135080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6ae806d380 0x7f6ae8135080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient: _finish_hunting 0 2026-02-17T06:05:14.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 monclient: found mon.noname-c 2026-02-17T06:05:14.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:14.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:14.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aec904640 1 -- 10.20.193.78:0/3160170452 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae8124130 con 0x7f6ae806c890 2026-02-17T06:05:14.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aed105640 10 monclient: get_auth_request con 0x7f6ae813a240 auth_method 0 2026-02-17T06:05:14.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.318+0000 7f6aed105640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae813a240 0x7f6ae8121970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aec904640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ae806c890 msgr2=0x7f6ae8134b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aec904640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ae806c890 0x7f6ae8134b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800c600 tx=0x7f6ad800cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8012a40 con 0x7f6ae806c890 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: dump: 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.324 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: _finish_auth 0 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad8017070 con 0x7f6ae806c890 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aeeb8f640 5 monclient: authenticate success, global_id 234122 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8012e20 con 0x7f6ae806c890 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6adeffd640 4 set_mon_vals no callback set 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6add7fa640 10 monclient: dump: 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aeeb8f640 10 monclient: _renew_subs 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aeeb8f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae81255e0 con 0x7f6ae806c890 2026-02-17T06:05:14.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aeeb8f640 10 monclient: _renew_subs 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aeeb8f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.320+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae80a2640 con 0x7f6ae806c890 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6aeeb8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6aeeb8f640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae8122670 0x7f6ae8122a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6aeeb8f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae80a01d0 con 0x7f6ae8122670 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6ad801c460 con 0x7f6ae806c890 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae8122670 0x7f6ae8122a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 10 monclient: get_auth_request con 0x7f6ae8122670 auth_method 0 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6add7fa640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ab803deb0 0x7f6ab8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(673..673 src has 525..673) ==== 4871+0+0 (secure 0 0 0) 0x7f6ad80526a0 con 0x7f6ae806c890 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6aed105640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ab803deb0 0x7f6ab8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6aed105640 10 monclient: get_auth_request con 0x7f6ab803deb0 auth_method 0 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 10 monclient(hunting): handle_auth_done global_id 214234 payload 824 2026-02-17T06:05:14.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.321+0000 7f6adffff640 1 -- 10.20.193.78:0/3160170452 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ae80008d0 con 0x7f6ae8122670 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6adffff640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae8122670 msgr2=0x7f6ae8122a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6adffff640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae8122670 0x7f6ae8122a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8069d40 tx=0x7f6ad00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6aed105640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ab803deb0 0x7f6ab8040370 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80a23f0 tx=0x7f6ae4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0049070 con 0x7f6ae8122670 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 10 monclient: dump: 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6ad0039e20 con 0x7f6ae8122670 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae8122670 msgr2=0x7f6ae8122a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae8122670 0x7f6ae8122a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8069d40 tx=0x7f6ad00047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae8122670 msgr2=0x7f6ae8122a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.324+0000 7f6add7fa640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae8122670 0x7f6ae8122a50 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8069d40 tx=0x7f6ad00047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6aeeb8f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6aeeb8f640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806fac0 0x7f6ae81da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6aeeb8f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8125820 con 0x7f6ae806fac0 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6adffff640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806fac0 0x7f6ae81da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6adffff640 10 monclient: get_auth_request con 0x7f6ae806fac0 auth_method 0 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6adffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6adffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.327+0000 7f6adffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.328+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.328+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.328+0000 7f6adffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.328+0000 7f6adffff640 10 monclient(hunting): handle_auth_done global_id 214237 payload 824 2026-02-17T06:05:14.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.328+0000 7f6adffff640 1 -- 10.20.193.78:0/3160170452 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6ae8125020 con 0x7f6ae806fac0 2026-02-17T06:05:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6adffff640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806fac0 msgr2=0x7f6ae81da7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6adffff640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806fac0 0x7f6ae81da7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00310e0 tx=0x7f6ad0008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad003e040 con 0x7f6ae806fac0 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 10 monclient: dump: 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.332 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f6ad0003c80 con 0x7f6ae806fac0 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806fac0 msgr2=0x7f6ae81da7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806fac0 0x7f6ae81da7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00310e0 tx=0x7f6ad0008030 comp rx=0 tx=0).stop 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806fac0 msgr2=0x7f6ae81da7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:14.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.330+0000 7f6add7fa640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6ae806fac0 0x7f6ae81da7d0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00310e0 tx=0x7f6ad0008030 comp rx=0 tx=0).stop 2026-02-17T06:05:14.334 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:05:14.334 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:14.334 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:14.334 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:05:14.334 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 220, 2026-02-17T06:05:14.334 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 117179, 2026-02-17T06:05:14.334 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.335 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.336 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:05:14.337 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.338 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.339 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:05:14.340 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.341 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.342 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:14.343 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:14.344 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:05:14.344 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ab803deb0 msgr2=0x7f6ab8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6ab803deb0 0x7f6ab8040370 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80a23f0 tx=0x7f6ae4009040 comp rx=0 tx=0).stop 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 10 monclient: shutdown 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ae806c890 msgr2=0x7f6ae8134b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 1 --2- 10.20.193.78:0/3160170452 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6ae806c890 0x7f6ae8134b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800c600 tx=0x7f6ad800cad0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aed105640 1 -- 10.20.193.78:0/3160170452 reap_dead start 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aed105640 1 -- 10.20.193.78:0/3160170452 reap_dead start 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 shutdown_connections 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 >> 10.20.193.78:0/3160170452 conn(0x7f6ae80a6620 msgr2=0x7f6ae8138c50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 shutdown_connections 2026-02-17T06:05:14.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.331+0000 7f6aeeb8f640 1 -- 10.20.193.78:0/3160170452 wait complete. 2026-02-17T06:05:14.346 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:05:14.346 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:05:14.346 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:05:14.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient: get_monmap_and_config 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient: build_initial_monmap 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient: monmap: 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:14.413346+0000 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:14.413346+0000 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:14.413 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 1 Processor -- start 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 1 -- start start 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient: init 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f749c06c890 0x7f749c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient(hunting): picked mon.noname-a con 0x7f749c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f749c09d170 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f749c09b200 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f749c1396d0 con 0x7f749c09d170 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f749c09bf60 con 0x7f749c09b200 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f749c09c520 con 0x7f749c06c890 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.412+0000 7f74a127c640 10 monclient(hunting): authenticate will time out at 1759.966975s 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f749c06c890 0x7f749c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749a575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f749c09d170 0x7f749c1388b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:51772/0 (socket says 10.20.193.78:51772) 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f749c06c890 0x7f749c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57036/0 (socket says 10.20.193.78:57036) 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749a575640 1 -- 10.20.193.78:0/3243091492 learned_addr learned my addr 10.20.193.78:0/3243091492 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient(hunting): get_auth_request con 0x7f749c06c890 auth_method 0 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient(hunting): handle_auth_done global_id 184587 payload 274 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 1 -- 10.20.193.78:0/3243091492 >> v1:10.20.193.115:6789/0 conn(0x7f749c09b200 legacy=0x7f749c138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 1 -- 10.20.193.78:0/3243091492 >> v1:10.20.193.78:6789/0 conn(0x7f749c09d170 legacy=0x7f749c1388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:05:14.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 1 monclient: found mon.noname-a 2026-02-17T06:05:14.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:14.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.413+0000 7f749ad76640 1 -- 10.20.193.78:0/3243091492 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c09c800 con 0x7f749c06c890 2026-02-17T06:05:14.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f749ad76640 1 -- 10.20.193.78:0/3243091492 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 msgr2=0x7f749c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f749ad76640 1 --2- 10.20.193.78:0/3243091492 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c06cc70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f74900098a0 tx=0x7f749002ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8703710549d6907a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 1 -- 10.20.193.78:0/3243091492 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749003c070 con 0x7f749c06c890 2026-02-17T06:05:14.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:14.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: dump: 2026-02-17T06:05:14.416 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.416 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: _finish_auth 0 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: _check_auth_tickets 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f74a127c640 5 monclient: authenticate success, global_id 184587 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f74a127c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 1 -- 10.20.193.78:0/3243091492 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749002fce0 con 0x7f749c06c890 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 1 -- 10.20.193.78:0/3243091492 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490037660 con 0x7f749c06c890 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f74a127c640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f74a127c640 4 set_mon_vals no callback set 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.415+0000 7f7499d74640 10 monclient: dump: 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient: shutdown 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 -- 10.20.193.78:0/3243091492 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 msgr2=0x7f749c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 --2- 10.20.193.78:0/3243091492 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c06cc70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f74900098a0 tx=0x7f749002ed80 comp rx=0 tx=0).stop 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 -- 10.20.193.78:0/3243091492 shutdown_connections 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 --2- 10.20.193.78:0/3243091492 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c06cc70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 -- 10.20.193.78:0/3243091492 >> 10.20.193.78:0/3243091492 conn(0x7f749c0a4230 msgr2=0x7f749c0a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 -- 10.20.193.78:0/3243091492 shutdown_connections 2026-02-17T06:05:14.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 -- 10.20.193.78:0/3243091492 wait complete. 2026-02-17T06:05:14.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient: build_initial_monmap 2026-02-17T06:05:14.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient: monmap: 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 Processor -- start 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 -- start start 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient: init 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c1cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient(hunting): picked mon.noname-a con 0x7f749c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f749c09b200 0x7f749c1cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient(hunting): picked mon.noname-b con 0x7f749c09b200 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f749c09d170 0x7f749c1d0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient(hunting): picked mon.noname-c con 0x7f749c09d170 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f749c0a3530 con 0x7f749c06c890 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.416+0000 7f74a127c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f74a127c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f749c0a33b0 con 0x7f749c09b200 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f74a127c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f74a127c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f749c06d1b0 con 0x7f749c09d170 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f74a127c640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f74a127c640 10 monclient(hunting): authenticate will time out at 1759.971006s 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c1cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c1cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57040/0 (socket says 10.20.193.78:57040) 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 -- 10.20.193.78:0/1354945938 learned_addr learned my addr 10.20.193.78:0/1354945938 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749b577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f749c09d170 0x7f749c1d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient(hunting): get_auth_request con 0x7f749c06c890 auth_method 0 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749b577640 10 monclient(hunting): get_auth_request con 0x7f749c09d170 auth_method 0 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749b577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749a575640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f749c09b200 0x7f749c1cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749a575640 10 monclient(hunting): get_auth_request con 0x7f749c09b200 auth_method 0 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient(hunting): handle_auth_done global_id 184590 payload 824 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 -- 10.20.193.78:0/1354945938 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f749c09d170 msgr2=0x7f749c1d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f749c09d170 0x7f749c1d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 -- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f749c09b200 msgr2=0x7f749c1cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f749c09b200 0x7f749c1cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 monclient: found mon.noname-a 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.417+0000 7f749ad76640 1 -- 10.20.193.78:0/1354945938 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c12efa0 con 0x7f749c06c890 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f749ad76640 1 -- 10.20.193.78:0/1354945938 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 msgr2=0x7f749c1cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f749ad76640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c1cf810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f74900098a0 tx=0x7f74900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 1 -- 10.20.193.78:0/1354945938 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490046070 con 0x7f749c06c890 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: dump: 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 1 -- 10.20.193.78:0/1354945938 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7490041870 con 0x7f749c06c890 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f74a127c640 5 monclient: authenticate success, global_id 184590 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f74a127c640 10 monclient: _renew_subs 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f74a127c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c1388b0 con 0x7f749c06c890 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 1 -- 10.20.193.78:0/1354945938 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749003c040 con 0x7f749c06c890 2026-02-17T06:05:14.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f7499573640 4 set_mon_vals no callback set 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f747b7fe640 10 monclient: dump: 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f74a127c640 10 monclient: _renew_subs 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f74a127c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.419+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c12ecc0 con 0x7f749c06c890 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f747b7fe640 1 -- 10.20.193.78:0/1354945938 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7490041c60 con 0x7f749c06c890 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f74a127c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f74a127c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f74a127c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c06a2b0 con 0x7f749c06c890 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f747b7fe640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f746803deb0 0x7f7468040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f747b7fe640 1 -- 10.20.193.78:0/1354945938 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(673..673 src has 525..673) ==== 4871+0+0 (secure 0 0 0) 0x7f7490078100 con 0x7f749c06c890 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f749a575640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f746803deb0 0x7f7468040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:14.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.420+0000 7f749a575640 10 monclient: get_auth_request con 0x7f746803deb0 auth_method 0 2026-02-17T06:05:14.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.422+0000 7f747b7fe640 1 -- 10.20.193.78:0/1354945938 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f749c06a2b0 con 0x7f749c06c890 2026-02-17T06:05:14.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.422+0000 7f747b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:14.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.422+0000 7f747b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:14.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.423+0000 7f749a575640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f746803deb0 0x7f7468040370 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f749c12e910 tx=0x7f7484009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:14.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.537+0000 7f74a127c640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:05:14.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.537+0000 7f74a127c640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:05:14.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.537+0000 7f74a127c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:14.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.537+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f749c074980 con 0x7f749c06c890 2026-02-17T06:05:14.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.538+0000 7f747b7fe640 1 -- 10.20.193.78:0/1354945938 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f7490037e10 con 0x7f749c06c890 2026-02-17T06:05:14.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.538+0000 7f747b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:05:14.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.538+0000 7f747b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=11,kv=12,logm=50} crc {auth=1194722482,config=692619927,health=3918307125,kv=449989585,logm=978473200}) 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=11,kv=10,logm=50} crc {auth=818412797,config=692619927,health=4013660047,kv=1499085521,logm=4019599144}) 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=11,kv=12,logm=50} crc {auth=1194722482,config=692619927,health=3918307125,kv=449989585,logm=978473200}) 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=25,config=1,health=12,kv=12,logm=50} crc {auth=862697521,config=3678701417,health=1845395070,kv=449989585,logm=3188188935}) 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=99758208}) 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=618955278}) 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=99758208}) 2026-02-17T06:05:14.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.538+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696167629}) 2026-02-17T06:05:14.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3673047501}) 2026-02-17T06:05:14.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3514570198}) 2026-02-17T06:05:14.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3673047501}) 2026-02-17T06:05:14.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1168508552}) 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f746803deb0 msgr2=0x7f7468040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f746803deb0 0x7f7468040370 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f749c12e910 tx=0x7f7484009040 comp rx=0 tx=0).stop 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 10 monclient: shutdown 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 msgr2=0x7f749c1cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c1cf810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f74900098a0 tx=0x7f74900047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 shutdown_connections 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f749c09d170 0x7f749c1d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f746803deb0 0x7f7468040370 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f749c09b200 0x7f749c1cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 --2- 10.20.193.78:0/1354945938 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f749c06c890 0x7f749c1cf810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 >> 10.20.193.78:0/1354945938 conn(0x7f749c0a4230 msgr2=0x7f749c0a5530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 shutdown_connections 2026-02-17T06:05:14.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:14.539+0000 7f74a127c640 1 -- 10.20.193.78:0/1354945938 wait complete. 2026-02-17T06:05:14.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3976350557}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2568712965}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3976350557}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.539+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1235779943}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1920862693}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659321330}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1920862693}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1539724700}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436056811}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.540+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=673323442}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436056811}) 2026-02-17T06:05:14.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=281011228}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3943745724}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=35005191}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3943745724}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2769601304}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120276380}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3370085885}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120276380}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.541+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1190191604}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1940367850}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3692958622}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1940367850}) 2026-02-17T06:05:14.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3959417494}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3031649803}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1988112255}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3031649803}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=663333648}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2991557279}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4185682148}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2991557279}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.542+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1975290936}) 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.543 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1186119658}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=922621535}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1186119658}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=417232038}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3486513386}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2242307568}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3486513386}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.543+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2884657891}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2063922949}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=467803425}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2063922949}) 2026-02-17T06:05:14.544 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3547479742}) 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1905330397}) 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3984445561}) 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1905330397}) 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2645881256}) 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1544701747}) 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3381416365}) 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1544701747}) 2026-02-17T06:05:14.545 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.544+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2685500872}) 2026-02-17T06:05:14.547 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:05:14.552 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.552 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=890111301}) 2026-02-17T06:05:14.552 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1943323146}) 2026-02-17T06:05:14.552 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=890111301}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=377054084}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=987096282}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1053117484}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=987096282}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1483935008}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168040475}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3223795894}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168040475}) 2026-02-17T06:05:14.553 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.552+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3379966620}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3675668873}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2940310470}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3675668873}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3489372563}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692386223}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3766138978}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692386223}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1734672811}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=935939217}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=470436583}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=935939217}) 2026-02-17T06:05:14.554 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.553+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=383598419}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3114141569}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=878809871}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3114141569}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=335271939}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3751455621}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1278531797}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3751455621}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1157231304}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=161318686}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615808366}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=161318686}) 2026-02-17T06:05:14.555 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.554+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2145452923}) 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3696788377}) 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3843098259}) 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3696788377}) 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2632545655}) 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890382048}) 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2312866907}) 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890382048}) 2026-02-17T06:05:14.556 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.555+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2376093838}) 2026-02-17T06:05:14.638 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.638 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498761608}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=397943497}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498761608}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4192954448}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845491979}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970374914}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845491979}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3895022633}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334561066}) 2026-02-17T06:05:14.639 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3688907169}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334561066}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.622+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1212008287}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1754388722}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1442655148}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1754388722}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=479253702}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581447057}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2383245133}) 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.640 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581447057}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=628861978}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=788847963}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3518204118}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=788847963}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=543545512}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2315427735}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3138437293}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2315427735}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.623+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=921928699}) 2026-02-17T06:05:14.641 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561133637}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4092478302}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=561133637}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3299231051}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3480989271}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=510682949}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3480989271}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3574567707}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725808067}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1173916544}) 2026-02-17T06:05:14.642 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725808067}) 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.624+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4264458440}) 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2156746472}) 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2244080804}) 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2156746472}) 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=145726860}) 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1626305394}) 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=754206060}) 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.643 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1626305394}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422405036}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=41217826}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1765712615}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=41217826}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1632234806}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836818208}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3145729111}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836818208}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.625+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1459387673}) 2026-02-17T06:05:14.644 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1965770413}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1756709550}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1965770413}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=453258795}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2862485090}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=138465705}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2862485090}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3847769441}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3166764920}) 2026-02-17T06:05:14.645 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1205346097}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3166764920}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.626+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1330996874}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3248910916}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3233539561}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3248910916}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1051407901}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164348638}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=103274081}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164348638}) 2026-02-17T06:05:14.646 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3714848211}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1700618026}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=209192883}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1700618026}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.627+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2135353408}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=698180307}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3365976342}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=698180307}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1621750703}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317966023}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1872898270}) 2026-02-17T06:05:14.647 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317966023}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2359767212}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4113708816}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3906763319}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4113708816}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.628+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2257706479}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390755490}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4236745953}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390755490}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1410556360}) 2026-02-17T06:05:14.648 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=28794324}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3914669048}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=28794324}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602421392}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312664032}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1443051809}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312664032}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273482067}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3427493761}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2136132842}) 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.649 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3427493761}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.629+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1046200024}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2090483846}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3343492693}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2090483846}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2948993284}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=191087012}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93697863}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=191087012}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931174965}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2257376394}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=27744652}) 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.650 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2257376394}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.630+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855275605}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3559079122}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1953326694}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3559079122}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=807945685}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=277089860}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1317407937}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=277089860}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3759569172}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=323993277}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4142259757}) 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.651 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=323993277}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2198857654}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=956970257}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2566335490}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=956970257}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.631+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3455114392}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2060222197}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1828716266}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2060222197}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3275173220}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892785954}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3562038462}) 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.652 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892785954}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3539828165}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538056954}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=514674295}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538056954}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.632+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1509367878}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2804517830}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1189727114}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2804517830}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2507874716}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3347304224}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2395614816}) 2026-02-17T06:05:14.653 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3347304224}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=991964684}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787360648}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1135529}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787360648}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.633+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2630620147}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3844999708}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2911280161}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3844999708}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=478516408}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923686294}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737833917}) 2026-02-17T06:05:14.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923686294}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1447050766}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=818307945}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766987835}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=818307945}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=935147892}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1925263018}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3826208387}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1925263018}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.634+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1377233191}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1001496018}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1146118525}) 2026-02-17T06:05:14.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1001496018}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783942619}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1547163310}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2191558962}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1547163310}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=907331771}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=23,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=31} crc {logm=1606267497,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1779744759,mon_config_key=2291868850,monmap=1784110042,osd_metadata=3878803090,osd_pg_creating=3742456367,osd_snap=61498374}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=21,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=34} crc {logm=2384765920,mdsmap=3140514009,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=4065268724,mon_config_key=1987855637,monmap=1784110042,osd_metadata=2116872255,osd_pg_creating=3742456367,osd_snap=3598837858}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=23,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=31} crc {logm=1606267497,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1779744759,mon_config_key=2291868850,monmap=1784110042,osd_metadata=3878803090,osd_pg_creating=3742456367,osd_snap=61498374}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.635+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=22,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1360800053,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1779744759,mon_config_key=1987855637,monmap=1784110042,osd_metadata=278982410,osd_pg_creating=3742456367,osd_snap=3021333086}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=87,osdmap=13} crc {osd_snap=2800602181,osdmap=3386644832}) 2026-02-17T06:05:14.656 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=82,osdmap=18} crc {osd_snap=1125917036,osdmap=4202896014}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=87,osdmap=13} crc {osd_snap=2800602181,osdmap=3386644832}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=86,osdmap=14} crc {osd_snap=2563943722,osdmap=342324158}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2999304387}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3888640433}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2999304387}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.636+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=948192729}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.637+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.637+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1508244350}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.637+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2558981478}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.637+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.637+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1508244350}) 2026-02-17T06:05:14.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.637+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=300348389}) 2026-02-17T06:05:14.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.639+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.639+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=15} crc {osdmap=3084279203}) 2026-02-17T06:05:14.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.639+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=15} crc {osdmap=2969276866}) 2026-02-17T06:05:14.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.639+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:05:14.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.639+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=15} crc {osdmap=3084279203}) 2026-02-17T06:05:14.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:05:14.639+0000 7f5dc0678640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=15} crc {osdmap=42598321}) 2026-02-17T06:05:14.675 INFO:tasks.rados.rados.0.trial115.stdout:3128: done (8 left) 2026-02-17T06:05:14.675 INFO:tasks.rados.rados.0.trial115.stdout:3129: done (7 left) 2026-02-17T06:05:14.676 INFO:tasks.rados.rados.0.trial115.stdout:3130: done (6 left) 2026-02-17T06:05:14.676 INFO:tasks.rados.rados.0.trial115.stdout:3131: done (5 left) 2026-02-17T06:05:14.676 INFO:tasks.rados.rados.0.trial115.stdout:3132: done (4 left) 2026-02-17T06:05:14.676 INFO:tasks.rados.rados.0.trial115.stdout:3133: done (3 left) 2026-02-17T06:05:14.676 INFO:tasks.rados.rados.0.trial115.stdout:3134: done (2 left) 2026-02-17T06:05:14.676 INFO:tasks.rados.rados.0.trial115.stdout:3135: done (1 left) 2026-02-17T06:05:14.676 INFO:tasks.rados.rados.0.trial115.stdout:3136: done (0 left) 2026-02-17T06:05:14.676 INFO:tasks.rados.rados.0.trial115.stdout:3137: snap_remove snap 307 2026-02-17T06:05:15.548 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:05:15.549 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:15.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.617+0000 7f6e16e5b640 10 monclient: get_monmap_and_config 2026-02-17T06:05:15.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.617+0000 7f6e16e5b640 10 monclient: build_initial_monmap 2026-02-17T06:05:15.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.617+0000 7f6e16e5b640 10 monclient: monmap: 2026-02-17T06:05:15.621 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:15.621 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.621 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:15.618726+0000 2026-02-17T06:05:15.621 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:15.618726+0000 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.617+0000 7f6e16e5b640 1 Processor -- start 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.617+0000 7f6e16e5b640 1 -- start start 2026-02-17T06:05:15.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.617+0000 7f6e16e5b640 10 monclient: init 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006c890 0x7f6e1006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient(hunting): picked mon.noname-b con 0x7f6e1006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e1009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6e1006d1b0 0x7f6e10135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient(hunting): picked mon.noname-c con 0x7f6e1006d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1006daf0 con 0x7f6e1009c4a0 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6e100a0880 con 0x7f6e1006d1b0 2026-02-17T06:05:15.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1009cc80 con 0x7f6e1006c890 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e16e5b640 10 monclient(hunting): authenticate will time out at 1761.172301s 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006c890 0x7f6e1006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006c890 0x7f6e1006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48548/0 (socket says 10.20.193.78:48548) 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 1 -- 10.20.193.78:0/2548047891 learned_addr learned my addr 10.20.193.78:0/2548047891 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient(hunting): get_auth_request con 0x7f6e1006c890 auth_method 0 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e153d1640 1 --2- 10.20.193.78:0/2548047891 >> v2:10.20.193.115:3300/0 conn(0x7f6e1006d1b0 0x7f6e10135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e153d1640 10 monclient(hunting): get_auth_request con 0x7f6e1006d1b0 auth_method 0 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e153d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e153d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e153d1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient(hunting): handle_auth_done global_id 234128 payload 274 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 1 -- 10.20.193.78:0/2548047891 >> v2:10.20.193.115:3300/0 conn(0x7f6e1006d1b0 msgr2=0x7f6e10135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 1 --2- 10.20.193.78:0/2548047891 >> v2:10.20.193.115:3300/0 conn(0x7f6e1006d1b0 0x7f6e10135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 1 -- 10.20.193.78:0/2548047891 >> v1:10.20.193.115:6789/0 conn(0x7f6e1009c4a0 legacy=0x7f6e10134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient: _finish_hunting 0 2026-02-17T06:05:15.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 1 monclient: found mon.noname-b 2026-02-17T06:05:15.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:15.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:05:15.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e14bd0640 1 -- 10.20.193.78:0/2548047891 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1009cf60 con 0x7f6e1006c890 2026-02-17T06:05:15.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.618+0000 7f6e153d1640 1 --2- 10.20.193.78:0/2548047891 >> v2:10.20.193.115:3300/0 conn(0x7f6e1006d1b0 0x7f6e10135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T06:05:15.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e14bd0640 1 -- 10.20.193.78:0/2548047891 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006c890 msgr2=0x7f6e1006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e14bd0640 1 --2- 10.20.193.78:0/2548047891 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006c890 0x7f6e1006cc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009970 tx=0x7f6e0002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b05f7a5bd75a07c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 1 -- 10.20.193.78:0/2548047891 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0003c070 con 0x7f6e1006c890 2026-02-17T06:05:15.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: dump: 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.633 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: _finish_auth 0 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: _check_auth_tickets 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 1 -- 10.20.193.78:0/2548047891 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0002fcd0 con 0x7f6e1006c890 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 5 monclient: authenticate success, global_id 234128 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 1 -- 10.20.193.78:0/2548047891 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00037660 con 0x7f6e1006c890 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 4 set_mon_vals no callback set 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e077fe640 10 monclient: dump: 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 10 monclient: shutdown 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 1 -- 10.20.193.78:0/2548047891 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006c890 msgr2=0x7f6e1006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/2548047891 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006c890 0x7f6e1006cc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009970 tx=0x7f6e0002ee90 comp rx=0 tx=0).stop 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 1 -- 10.20.193.78:0/2548047891 shutdown_connections 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/2548047891 >> v2:10.20.193.115:3300/0 conn(0x7f6e1006d1b0 0x7f6e10135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/2548047891 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006c890 0x7f6e1006cc70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 1 -- 10.20.193.78:0/2548047891 >> 10.20.193.78:0/2548047891 conn(0x7f6e100a6620 msgr2=0x7f6e100a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 1 -- 10.20.193.78:0/2548047891 shutdown_connections 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.621+0000 7f6e16e5b640 1 -- 10.20.193.78:0/2548047891 wait complete. 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient: build_initial_monmap 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient: monmap: 2026-02-17T06:05:15.636 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 1 Processor -- start 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 1 -- start start 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient: init 2026-02-17T06:05:15.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6e1006c890 0x7f6e101cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient(hunting): picked mon.noname-b con 0x7f6e1006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006d1b0 0x7f6e101cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient(hunting): picked mon.noname-c con 0x7f6e1006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6e1009c4a0 0x7f6e101238c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient(hunting): picked mon.noname-a con 0x7f6e1009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100a01a0 con 0x7f6e1009c4a0 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100a0020 con 0x7f6e1006c890 2026-02-17T06:05:15.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1009be20 con 0x7f6e1006d1b0 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e16e5b640 10 monclient(hunting): authenticate will time out at 1761.176607s 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6e1009c4a0 0x7f6e101238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6e1009c4a0 0x7f6e101238c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57048/0 (socket says 10.20.193.78:57048) 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 -- 10.20.193.78:0/148080587 learned_addr learned my addr 10.20.193.78:0/148080587 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient(hunting): get_auth_request con 0x7f6e1009c4a0 auth_method 0 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e07fff640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006d1b0 0x7f6e101cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e07fff640 10 monclient(hunting): get_auth_request con 0x7f6e1006d1b0 auth_method 0 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e07fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient(hunting): handle_auth_done global_id 184593 payload 824 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 -- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006d1b0 msgr2=0x7f6e101cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006d1b0 0x7f6e101cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 -- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6e1006c890 msgr2=0x7f6e101cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6e1006c890 0x7f6e101cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient: _finish_hunting 0 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 monclient: found mon.noname-a 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.622+0000 7f6e153d1640 1 -- 10.20.193.78:0/148080587 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e10128230 con 0x7f6e1009c4a0 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.624+0000 7f6e153d1640 1 -- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6e1009c4a0 msgr2=0x7f6e101238c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.624+0000 7f6e153d1640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6e1009c4a0 0x7f6e101238c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c007e60 tx=0x7f6e0c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 1 -- 10.20.193.78:0/148080587 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c01a070 con 0x7f6e1009c4a0 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: dump: 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: _finish_auth 0 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: _check_auth_tickets 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 1 -- 10.20.193.78:0/148080587 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0c003cb0 con 0x7f6e1009c4a0 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 1 -- 10.20.193.78:0/148080587 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c003e60 con 0x7f6e1009c4a0 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e06ffd640 4 set_mon_vals no callback set 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 10 monclient: dump: 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.643 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 5 monclient: authenticate success, global_id 184593 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 10 monclient: _renew_subs 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e10127570 con 0x7f6e1009c4a0 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 10 monclient: _renew_subs 2026-02-17T06:05:15.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e10072fe0 con 0x7f6e1009c4a0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 1 -- 10.20.193.78:0/148080587 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6e0c0127e0 con 0x7f6e1009c4a0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1006f800 con 0x7f6e1009c4a0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.625+0000 7f6e057fa640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6de003df00 0x7f6de00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.626+0000 7f6e057fa640 1 -- 10.20.193.78:0/148080587 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(674..674 src has 525..674) ==== 4751+0+0 (secure 0 0 0) 0x7f6e0c052e90 con 0x7f6e1009c4a0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.626+0000 7f6e14bd0640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6de003df00 0x7f6de00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.626+0000 7f6e14bd0640 10 monclient: get_auth_request con 0x7f6de003df00 auth_method 0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.628+0000 7f6e057fa640 1 -- 10.20.193.78:0/148080587 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6e1006f800 con 0x7f6e1009c4a0 2026-02-17T06:05:15.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.628+0000 7f6e057fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:15.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.628+0000 7f6e057fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:15.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.628+0000 7f6e14bd0640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6de003df00 0x7f6de00403c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6e00036000 tx=0x7f6e00039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.677 INFO:tasks.rados.rados.0.trial115.stdout:3137: done (0 left) 2026-02-17T06:05:15.677 INFO:tasks.rados.rados.0.trial115.stdout:3138: copy_from oid 36 from oid 32 current snap is 329 2026-02-17T06:05:15.677 INFO:tasks.rados.rados.0.trial115.stdout:3139: write oid 46 current snap is 329 2026-02-17T06:05:15.677 INFO:tasks.rados.rados.0.trial115.stdout:3139: seq_num 1093 ranges {770051=485636,1776701=511047} 2026-02-17T06:05:15.681 INFO:tasks.rados.rados.0.trial115.stdout:3139: writing trial11545420-46 from 770051 to 1255687 tid 1 2026-02-17T06:05:15.686 INFO:tasks.rados.rados.0.trial115.stdout:3139: writing trial11545420-46 from 1776701 to 2287748 tid 2 2026-02-17T06:05:15.686 INFO:tasks.rados.rados.0.trial115.stdout:3140: write oid 30 current snap is 329 2026-02-17T06:05:15.686 INFO:tasks.rados.rados.0.trial115.stdout:3140: seq_num 1094 ranges {786858=560885,1835183=551675} 2026-02-17T06:05:15.691 INFO:tasks.rados.rados.0.trial115.stdout:3140: writing trial11545420-30 from 786858 to 1347743 tid 1 2026-02-17T06:05:15.695 INFO:tasks.rados.rados.0.trial115.stdout:3140: writing trial11545420-30 from 1835183 to 2386858 tid 2 2026-02-17T06:05:15.695 INFO:tasks.rados.rados.0.trial115.stdout:3141: write oid 26 current snap is 329 2026-02-17T06:05:15.695 INFO:tasks.rados.rados.0.trial115.stdout:3139: finishing write tid 1 to trial11545420-46 2026-02-17T06:05:15.695 INFO:tasks.rados.rados.0.trial115.stdout:3141: seq_num 1095 ranges {403665=752487,1610018=514716} 2026-02-17T06:05:15.699 INFO:tasks.rados.rados.0.trial115.stdout:3141: writing trial11545420-26 from 403665 to 1156152 tid 1 2026-02-17T06:05:15.703 INFO:tasks.rados.rados.0.trial115.stdout:3141: writing trial11545420-26 from 1610018 to 2124734 tid 2 2026-02-17T06:05:15.703 INFO:tasks.rados.rados.0.trial115.stdout:3142: write (excl) oid 10 current snap is 329 2026-02-17T06:05:15.703 INFO:tasks.rados.rados.0.trial115.stdout:3142: seq_num 1096 ranges {420472=427736,1268500=658374,2347953=72519} 2026-02-17T06:05:15.707 INFO:tasks.rados.rados.0.trial115.stdout:3142: writing trial11545420-10 from 420472 to 848208 tid 1 2026-02-17T06:05:15.711 INFO:tasks.rados.rados.0.trial115.stdout:3142: writing trial11545420-10 from 1268500 to 1926874 tid 2 2026-02-17T06:05:15.714 INFO:tasks.rados.rados.0.trial115.stdout:3142: writing trial11545420-10 from 2347953 to 2420472 tid 3 2026-02-17T06:05:15.714 INFO:tasks.rados.rados.0.trial115.stdout:3143: delete oid 9 current snap is 329 2026-02-17T06:05:15.714 INFO:tasks.rados.rados.0.trial115.stdout:3138: finishing copy_from racing read to trial11545420-36 2026-02-17T06:05:15.714 INFO:tasks.rados.rados.0.trial115.stdout:3138: finishing copy_from to trial11545420-36 2026-02-17T06:05:15.714 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 1452 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:05:15.714 INFO:tasks.rados.rados.0.trial115.stdout:3139: finishing write tid 2 to trial11545420-46 2026-02-17T06:05:15.714 INFO:tasks.rados.rados.0.trial115.stdout:3139: finishing write tid 3 to trial11545420-46 2026-02-17T06:05:15.714 INFO:tasks.rados.rados.0.trial115.stdout:3139: finishing write tid 4 to trial11545420-46 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3139: oid 46 updating version 0 to 1565 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3139: oid 46 version 1565 is already newer than 1564 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3139: oid 46 updating version 1565 to 1566 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1566 (ObjNum 1093 snap 329 seq_num 1093) dirty exists 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3139: left oid 46 (ObjNum 1093 snap 329 seq_num 1093) 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3140: finishing write tid 1 to trial11545420-30 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3140: finishing write tid 2 to trial11545420-30 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3140: finishing write tid 3 to trial11545420-30 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3140: finishing write tid 4 to trial11545420-30 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3140: oid 30 updating version 0 to 1356 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3140: oid 30 version 1356 is already newer than 1355 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:3140: oid 30 updating version 1356 to 1357 2026-02-17T06:05:15.715 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1357 (ObjNum 1094 snap 329 seq_num 1094) dirty exists 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3140: left oid 30 (ObjNum 1094 snap 329 seq_num 1094) 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3141: finishing write tid 1 to trial11545420-26 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3141: finishing write tid 2 to trial11545420-26 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3141: finishing write tid 3 to trial11545420-26 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3141: finishing write tid 4 to trial11545420-26 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3141: oid 26 updating version 0 to 1457 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3141: oid 26 version 1457 is already newer than 1456 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3141: oid 26 version 1457 is already newer than 1455 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1457 (ObjNum 1095 snap 329 seq_num 1095) dirty exists 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3141: left oid 26 (ObjNum 1095 snap 329 seq_num 1095) 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3142: finishing write tid 1 to trial11545420-10 2026-02-17T06:05:15.716 INFO:tasks.rados.rados.0.trial115.stdout:3142: finishing write tid 2 to trial11545420-10 2026-02-17T06:05:15.718 INFO:tasks.rados.rados.0.trial115.stdout:3142: finishing write tid 3 to trial11545420-10 2026-02-17T06:05:15.718 INFO:tasks.rados.rados.0.trial115.stdout:3142: finishing write tid 4 to trial11545420-10 2026-02-17T06:05:15.718 INFO:tasks.rados.rados.0.trial115.stdout:3142: finishing write tid 5 to trial11545420-10 2026-02-17T06:05:15.718 INFO:tasks.rados.rados.0.trial115.stdout:3142: oid 10 updating version 0 to 1569 2026-02-17T06:05:15.718 INFO:tasks.rados.rados.0.trial115.stdout:3142: oid 10 version 1569 is already newer than 1567 2026-02-17T06:05:15.718 INFO:tasks.rados.rados.0.trial115.stdout:3142: oid 10 version 1569 is already newer than 1568 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3142: oid 10 updating version 1569 to 1570 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1570 (ObjNum 1096 snap 329 seq_num 1096) dirty exists 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3142: left oid 10 (ObjNum 1096 snap 329 seq_num 1096) 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3138: done (5 left) 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3139: done (4 left) 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3140: done (3 left) 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3141: done (2 left) 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3142: done (1 left) 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3143: done (0 left) 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3144: read oid 34 snap -1 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3144: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:05:15.719 INFO:tasks.rados.rados.0.trial115.stdout:3145: delete oid 49 current snap is 329 2026-02-17T06:05:15.720 INFO:tasks.rados.rados.0.trial115.stdout:3145: done (1 left) 2026-02-17T06:05:15.720 INFO:tasks.rados.rados.0.trial115.stdout:3146: read oid 2 snap -1 2026-02-17T06:05:15.720 INFO:tasks.rados.rados.0.trial115.stdout:3146: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:05:15.720 INFO:tasks.rados.rados.0.trial115.stdout:3147: read oid 6 snap 309 2026-02-17T06:05:15.720 INFO:tasks.rados.rados.0.trial115.stdout:3147: expect (ObjNum 935 snap 274 seq_num 935) 2026-02-17T06:05:15.720 INFO:tasks.rados.rados.0.trial115.stdout:3148: write (excl) oid 50 current snap is 329 2026-02-17T06:05:15.720 INFO:tasks.rados.rados.0.trial115.stdout:3148: seq_num 1099 ranges {470893=653483,1727593=522054} 2026-02-17T06:05:15.722 INFO:tasks.rados.rados.0.trial115.stdout:3148: writing trial11545420-50 from 470893 to 1124376 tid 1 2026-02-17T06:05:15.726 INFO:tasks.rados.rados.0.trial115.stdout:3148: writing trial11545420-50 from 1727593 to 2249647 tid 2 2026-02-17T06:05:15.726 INFO:tasks.rados.rados.0.trial115.stdout:3149: snap_remove snap 327 2026-02-17T06:05:15.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.741+0000 7f6e16e5b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:05:15.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.741+0000 7f6e16e5b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:15.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.741+0000 7f6e16e5b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.741+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6e10074980 con 0x7f6e1009c4a0 2026-02-17T06:05:15.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.741+0000 7f6e057fa640 1 -- 10.20.193.78:0/148080587 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f6e10128230 con 0x7f6e1009c4a0 2026-02-17T06:05:15.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.741+0000 7f6e057fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:15.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.741+0000 7f6e057fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:15.743 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":220,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:05:15.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6de003df00 msgr2=0x7f6de00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6de003df00 0x7f6de00403c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6e00036000 tx=0x7f6e00039040 comp rx=0 tx=0).stop 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 10 monclient: shutdown 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6e1009c4a0 msgr2=0x7f6e101238c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6e1009c4a0 0x7f6e101238c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c007e60 tx=0x7f6e0c00d4a0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 shutdown_connections 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6e1009c4a0 0x7f6e101238c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6e1006d1b0 0x7f6e101cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6de003df00 0x7f6de00403c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 --2- 10.20.193.78:0/148080587 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6e1006c890 0x7f6e101cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 >> 10.20.193.78:0/148080587 conn(0x7f6e100a6620 msgr2=0x7f6e10070320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:15.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 shutdown_connections 2026-02-17T06:05:15.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.743+0000 7f6e16e5b640 1 -- 10.20.193.78:0/148080587 wait complete. 2026-02-17T06:05:15.752 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:05:15.753 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:05:15.753 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:05:15.753 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:05:15.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.820+0000 7f7f16122640 10 monclient: get_monmap_and_config 2026-02-17T06:05:15.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.820+0000 7f7f16122640 10 monclient: build_initial_monmap 2026-02-17T06:05:15.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.820+0000 7f7f16122640 10 monclient: monmap: 2026-02-17T06:05:15.824 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:15.824 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.824 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:15.821976+0000 2026-02-17T06:05:15.824 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:15.821976+0000 2026-02-17T06:05:15.824 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 1 Processor -- start 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 1 -- start start 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient: init 2026-02-17T06:05:15.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7f1006c890 0x7f7f1006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient(hunting): picked mon.noname-c con 0x7f7f1006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7f1009c4a0 0x7f7f1006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient(hunting): picked mon.noname-a con 0x7f7f1009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7f10134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f1006daf0 con 0x7f7f10134df0 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7f100a0880 con 0x7f7f1009c4a0 2026-02-17T06:05:15.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7f1006d6f0 con 0x7f7f1006c890 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f16122640 10 monclient(hunting): authenticate will time out at 1761.375520s 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7f1009c4a0 0x7f7f1006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7f1009c4a0 0x7f7f1006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57058/0 (socket says 10.20.193.78:57058) 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 1 -- 10.20.193.78:0/3104099542 learned_addr learned my addr 10.20.193.78:0/3104099542 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient(hunting): get_auth_request con 0x7f7f1009c4a0 auth_method 0 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient(hunting): handle_auth_done global_id 184596 payload 274 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 1 -- 10.20.193.78:0/3104099542 >> v2:10.20.193.115:3300/0 conn(0x7f7f1006c890 msgr2=0x7f7f1006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 1 --2- 10.20.193.78:0/3104099542 >> v2:10.20.193.115:3300/0 conn(0x7f7f1006c890 0x7f7f1006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 1 -- 10.20.193.78:0/3104099542 >> v1:10.20.193.115:6789/0 conn(0x7f7f10134df0 legacy=0x7f7f10135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient: _finish_hunting 0 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 1 monclient: found mon.noname-a 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.821+0000 7f7f0effd640 1 -- 10.20.193.78:0/3104099542 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1009cc80 con 0x7f7f1009c4a0 2026-02-17T06:05:15.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0effd640 1 -- 10.20.193.78:0/3104099542 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f1009c4a0 msgr2=0x7f7f1006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0effd640 1 --2- 10.20.193.78:0/3104099542 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f1009c4a0 0x7f7f1006d1b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f1006a480 tx=0x7f7f0002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb9decb60bf5713 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 1 -- 10.20.193.78:0/3104099542 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0003c070 con 0x7f7f1009c4a0 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: dump: 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.837 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 1 -- 10.20.193.78:0/3104099542 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0002fd80 con 0x7f7f1009c4a0 2026-02-17T06:05:15.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 1 -- 10.20.193.78:0/3104099542 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00037660 con 0x7f7f1009c4a0 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 5 monclient: authenticate success, global_id 184596 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 4 set_mon_vals no callback set 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f0e7fc640 10 monclient: dump: 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.839 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 10 monclient: shutdown 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 1 -- 10.20.193.78:0/3104099542 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f1009c4a0 msgr2=0x7f7f1006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 1 --2- 10.20.193.78:0/3104099542 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f1009c4a0 0x7f7f1006d1b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f1006a480 tx=0x7f7f0002ee60 comp rx=0 tx=0).stop 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 1 -- 10.20.193.78:0/3104099542 shutdown_connections 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 1 --2- 10.20.193.78:0/3104099542 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f1009c4a0 0x7f7f1006d1b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 1 --2- 10.20.193.78:0/3104099542 >> v2:10.20.193.115:3300/0 conn(0x7f7f1006c890 0x7f7f1006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 1 -- 10.20.193.78:0/3104099542 >> 10.20.193.78:0/3104099542 conn(0x7f7f100a6620 msgr2=0x7f7f100a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 1 -- 10.20.193.78:0/3104099542 shutdown_connections 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.824+0000 7f7f16122640 1 -- 10.20.193.78:0/3104099542 wait complete. 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient: build_initial_monmap 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient: monmap: 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.841 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 1 Processor -- start 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 1 -- start start 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient: init 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f1006c890 0x7f7f101cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient(hunting): picked mon.noname-c con 0x7f7f1006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f1009c4a0 0x7f7f10123a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient(hunting): picked mon.noname-b con 0x7f7f1009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10134df0 0x7f7f10123f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient(hunting): picked mon.noname-a con 0x7f7f10134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f100a01a0 con 0x7f7f10134df0 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7f100a0020 con 0x7f7f1009c4a0 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7f1009be20 con 0x7f7f1006c890 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f16122640 10 monclient(hunting): authenticate will time out at 1761.379610s 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10134df0 0x7f7f10123f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f1006c890 0x7f7f101cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10134df0 0x7f7f10123f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57068/0 (socket says 10.20.193.78:57068) 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 -- 10.20.193.78:0/2677281891 learned_addr learned my addr 10.20.193.78:0/2677281891 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient(hunting): get_auth_request con 0x7f7f10134df0 auth_method 0 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 10 monclient(hunting): get_auth_request con 0x7f7f1006c890 auth_method 0 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0effd640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f1009c4a0 0x7f7f10123a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0effd640 10 monclient(hunting): get_auth_request con 0x7f7f1009c4a0 auth_method 0 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient(hunting): handle_auth_done global_id 184599 payload 824 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f1006c890 msgr2=0x7f7f101cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f1006c890 0x7f7f101cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f1009c4a0 msgr2=0x7f7f10123a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f1009c4a0 0x7f7f10123a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 monclient: found mon.noname-a 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0ffff640 1 -- 10.20.193.78:0/2677281891 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f10126fa0 con 0x7f7f10134df0 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.825+0000 7f7f0f7fe640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f1006c890 0x7f7f101cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0ffff640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10134df0 msgr2=0x7f7f10123f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0ffff640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10134df0 0x7f7f10123f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400ba10 tx=0x7f7f0400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04006e60 con 0x7f7f10134df0 2026-02-17T06:05:15.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: dump: 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.847 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: _finish_auth 0 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: _check_auth_tickets 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f04010070 con 0x7f7f10134df0 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0400cb90 con 0x7f7f10134df0 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 10 monclient: dump: 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.848 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0dffb640 4 set_mon_vals no callback set 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 5 monclient: authenticate success, global_id 184599 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 10 monclient: _renew_subs 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f10127280 con 0x7f7f10134df0 2026-02-17T06:05:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 10 monclient: _renew_subs 2026-02-17T06:05:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f10072780 con 0x7f7f10134df0 2026-02-17T06:05:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f101cfc70 0x7f7f101244d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f100a17d0 con 0x7f7f101cfc70 2026-02-17T06:05:15.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0f7fe640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f101cfc70 0x7f7f101244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0f7fe640 10 monclient: get_auth_request con 0x7f7f101cfc70 auth_method 0 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7f04004070 con 0x7f7f10134df0 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.828+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.829+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_done global_id 184602 payload 824 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.829+0000 7f7f0f7fe640 1 -- 10.20.193.78:0/2677281891 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f10135a20 con 0x7f7f101cfc70 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.829+0000 7f7eebfff640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ee003deb0 0x7f7ee0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.829+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(675..675 src has 525..675) ==== 4795+0+0 (secure 0 0 0) 0x7f7f04052560 con 0x7f7f10134df0 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.829+0000 7f7f0effd640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ee003deb0 0x7f7ee0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.829+0000 7f7f0effd640 10 monclient: get_auth_request con 0x7f7ee003deb0 auth_method 0 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7f0f7fe640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f101cfc70 msgr2=0x7f7f101244d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7f0f7fe640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f101cfc70 0x7f7f101244d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0029b0 tx=0x7f7efc002e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc016070 con 0x7f7f101cfc70 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 10 monclient: dump: 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.852 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7efc00ee40 con 0x7f7f101cfc70 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f101cfc70 msgr2=0x7f7f101244d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f101cfc70 0x7f7f101244d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0029b0 tx=0x7f7efc002e80 comp rx=0 tx=0).stop 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f101cfc70 msgr2=0x7f7f101244d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7eebfff640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f101cfc70 0x7f7f101244d0 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0029b0 tx=0x7f7efc002e80 comp rx=0 tx=0).stop 2026-02-17T06:05:15.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.831+0000 7f7f0effd640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ee003deb0 0x7f7ee0040370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f00036000 tx=0x7f7f00033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f16122640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f16122640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10124a10 0x7f7f101da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f16122640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f10124e20 con 0x7f7f10124a10 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10124a10 0x7f7f101da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 10 monclient: get_auth_request con 0x7f7f10124a10 auth_method 0 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 10 monclient(hunting): handle_auth_done global_id 184605 payload 824 2026-02-17T06:05:15.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.835+0000 7f7f0f7fe640 1 -- 10.20.193.78:0/2677281891 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7f100744d0 con 0x7f7f10124a10 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7f0f7fe640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10124a10 msgr2=0x7f7f101da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7f0f7fe640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10124a10 0x7f7f101da3c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0029b0 tx=0x7f7efc0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc016070 con 0x7f7f10124a10 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 10 monclient: dump: 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.856 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f7efc006250 con 0x7f7f10124a10 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10124a10 msgr2=0x7f7f101da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10124a10 0x7f7f101da3c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0029b0 tx=0x7f7efc0048d0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10124a10 msgr2=0x7f7f101da3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:15.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.837+0000 7f7eebfff640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10124a10 0x7f7f101da3c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 220, 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 280617, 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:05:15.858 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:15.859 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:15.860 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:05:15.861 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.862 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.863 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.864 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.865 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:05:15.866 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:05:15.867 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:05:15.868 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:05:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ee003deb0 msgr2=0x7f7ee0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ee003deb0 0x7f7ee0040370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f00036000 tx=0x7f7f00033000 comp rx=0 tx=0).stop 2026-02-17T06:05:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 10 monclient: shutdown 2026-02-17T06:05:15.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:15.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10134df0 msgr2=0x7f7f10123f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10134df0 0x7f7f10123f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400ba10 tx=0x7f7f0400bee0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f0ffff640 1 -- 10.20.193.78:0/2677281891 reap_dead start 2026-02-17T06:05:15.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 shutdown_connections 2026-02-17T06:05:15.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 --2- 10.20.193.78:0/2677281891 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f10134df0 0x7f7f10123f90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 >> 10.20.193.78:0/2677281891 conn(0x7f7f100a6620 msgr2=0x7f7f1006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:15.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 shutdown_connections 2026-02-17T06:05:15.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.838+0000 7f7f16122640 1 -- 10.20.193.78:0/2677281891 wait complete. 2026-02-17T06:05:15.870 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:05:15.870 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:05:15.870 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:05:15.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.936+0000 7f2254353640 10 monclient: get_monmap_and_config 2026-02-17T06:05:15.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.936+0000 7f2254353640 10 monclient: build_initial_monmap 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.936+0000 7f2254353640 10 monclient: monmap: 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:15.938010+0000 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:15.938010+0000 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:15.940 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 1 Processor -- start 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 1 -- start start 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient: init 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c06c890 0x7f224c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient(hunting): picked mon.noname-b con 0x7f224c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f224c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f224c06d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224c135c10 con 0x7f224c06d1b0 2026-02-17T06:05:15.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224c06daf0 con 0x7f224c09c4a0 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0a0880 con 0x7f224c06c890 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f2254353640 10 monclient(hunting): authenticate will time out at 1761.491595s 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c06c890 0x7f224c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c06c890 0x7f224c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48580/0 (socket says 10.20.193.78:48580) 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 1 -- 10.20.193.78:0/3736567370 learned_addr learned my addr 10.20.193.78:0/3736567370 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient(hunting): get_auth_request con 0x7f224c06c890 auth_method 0 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient(hunting): handle_auth_done global_id 234137 payload 274 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 1 -- 10.20.193.78:0/3736567370 >> v1:10.20.193.115:6789/0 conn(0x7f224c09c4a0 legacy=0x7f224c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 1 -- 10.20.193.78:0/3736567370 >> v1:10.20.193.78:6789/0 conn(0x7f224c06d1b0 legacy=0x7f224c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient: _finish_hunting 0 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 1 monclient: found mon.noname-b 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:05:15.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.937+0000 7f22520c8640 1 -- 10.20.193.78:0/3736567370 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c09cc80 con 0x7f224c06c890 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22520c8640 1 -- 10.20.193.78:0/3736567370 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c06c890 msgr2=0x7f224c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22520c8640 1 --2- 10.20.193.78:0/3736567370 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c06c890 0x7f224c06cc70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f223c0099a0 tx=0x7f223c02ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2eda540944c9ed server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 1 -- 10.20.193.78:0/3736567370 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c03c070 con 0x7f224c06c890 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: dump: 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: _finish_auth 0 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: _check_auth_tickets 2026-02-17T06:05:15.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 5 monclient: authenticate success, global_id 234137 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 1 -- 10.20.193.78:0/3736567370 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f223c02fd90 con 0x7f224c06c890 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 1 -- 10.20.193.78:0/3736567370 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c037660 con 0x7f224c06c890 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 4 set_mon_vals no callback set 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f22510c6640 10 monclient: dump: 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 10 monclient: shutdown 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 1 -- 10.20.193.78:0/3736567370 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c06c890 msgr2=0x7f224c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 1 --2- 10.20.193.78:0/3736567370 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c06c890 0x7f224c06cc70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f223c0099a0 tx=0x7f223c02ee30 comp rx=0 tx=0).stop 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 1 -- 10.20.193.78:0/3736567370 shutdown_connections 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 1 --2- 10.20.193.78:0/3736567370 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c06c890 0x7f224c06cc70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 1 -- 10.20.193.78:0/3736567370 >> 10.20.193.78:0/3736567370 conn(0x7f224c0a6620 msgr2=0x7f224c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 1 -- 10.20.193.78:0/3736567370 shutdown_connections 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.940+0000 7f2254353640 1 -- 10.20.193.78:0/3736567370 wait complete. 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient: build_initial_monmap 2026-02-17T06:05:15.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient: monmap: 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 1 Processor -- start 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 1 -- start start 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient: init 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f224c06c890 0x7f224c134350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient(hunting): picked mon.noname-a con 0x7f224c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f224c06d1b0 0x7f224c134890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient(hunting): picked mon.noname-b con 0x7f224c06d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c09c4a0 0x7f224c1cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient(hunting): picked mon.noname-c con 0x7f224c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f224c0a0020 con 0x7f224c06c890 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f224c06d600 con 0x7f224c06d1b0 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0a01a0 con 0x7f224c09c4a0 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f2254353640 10 monclient(hunting): authenticate will time out at 1761.495770s 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f224c06c890 0x7f224c134350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f224c06c890 0x7f224c134350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57102/0 (socket says 10.20.193.78:57102) 2026-02-17T06:05:15.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 1 -- 10.20.193.78:0/3103187030 learned_addr learned my addr 10.20.193.78:0/3103187030 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 10 monclient(hunting): get_auth_request con 0x7f224c06c890 auth_method 0 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22528c9640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c09c4a0 0x7f224c1cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22528c9640 10 monclient(hunting): get_auth_request con 0x7f224c09c4a0 auth_method 0 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22528c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22528c9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22528c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22528c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22528c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22528c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22520c8640 10 monclient(hunting): handle_auth_done global_id 184608 payload 824 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.941+0000 7f22518c7640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f224c06d1b0 0x7f224c134890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c09c4a0 msgr2=0x7f224c1cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c09c4a0 0x7f224c1cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f224c06d1b0 msgr2=0x7f224c134890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f224c06d1b0 0x7f224c134890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 10 monclient: _finish_hunting 0 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 1 monclient: found mon.noname-a 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22520c8640 1 -- 10.20.193.78:0/3103187030 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c0a2640 con 0x7f224c06c890 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22518c7640 10 monclient: get_auth_request con 0x7f224c06d1b0 auth_method 0 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.942+0000 7f22518c7640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f224c06d1b0 0x7f224c134890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f22520c8640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f224c06c890 msgr2=0x7f224c134350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f22520c8640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f224c06c890 0x7f224c134350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f223c0099a0 tx=0x7f223c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c046020 con 0x7f224c06c890 2026-02-17T06:05:15.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: dump: 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: _finish_auth 0 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f223c042620 con 0x7f224c06c890 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 5 monclient: authenticate success, global_id 184608 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 10 monclient: _renew_subs 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c09d810 con 0x7f224c06c890 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c03c040 con 0x7f224c06c890 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2232ffd640 10 monclient: dump: 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.953 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f22508c5640 4 set_mon_vals no callback set 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 10 monclient: _renew_subs 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c127a90 con 0x7f224c06c890 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11cc70 0x7f224c11d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f224c069a00 con 0x7f224c11cc70 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f22518c7640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11cc70 0x7f224c11d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f22518c7640 10 monclient: get_auth_request con 0x7f224c11cc70 auth_method 0 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f22518c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f22518c7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.944+0000 7f22518c7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f22518c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f22518c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f22518c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f223c03f030 con 0x7f224c06c890 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f22518c7640 10 monclient(hunting): handle_auth_done global_id 234143 payload 824 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f22518c7640 1 -- 10.20.193.78:0/3103187030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f224c127640 con 0x7f224c11cc70 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f2232ffd640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f221c03dee0 0x7f221c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(675..675 src has 525..675) ==== 4795+0+0 (secure 0 0 0) 0x7f223c077720 con 0x7f224c06c890 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f22528c9640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f221c03dee0 0x7f221c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.945+0000 7f22528c9640 10 monclient: get_auth_request con 0x7f221c03dee0 auth_method 0 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f22518c7640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11cc70 msgr2=0x7f224c11d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f22518c7640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11cc70 0x7f224c11d050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f224c09f370 tx=0x7f224000cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240004480 con 0x7f224c11cc70 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 10 monclient: dump: 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.954 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2240016070 con 0x7f224c11cc70 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11cc70 msgr2=0x7f224c11d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11cc70 0x7f224c11d050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f224c09f370 tx=0x7f224000cb90 comp rx=0 tx=0).stop 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11cc70 msgr2=0x7f224c11d050 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f2232ffd640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11cc70 0x7f224c11d050 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f224c09f370 tx=0x7f224000cb90 comp rx=0 tx=0).stop 2026-02-17T06:05:15.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.947+0000 7f22528c9640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f221c03dee0 0x7f221c0403a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f224800f2e0 tx=0x7f2248004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f2254353640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f2254353640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11d590 0x7f224c11d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f2254353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f224c06cc70 con 0x7f224c11d590 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11d590 0x7f224c11d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 10 monclient: get_auth_request con 0x7f224c11d590 auth_method 0 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 10 monclient(hunting): handle_auth_done global_id 234146 payload 824 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.950+0000 7f22518c7640 1 -- 10.20.193.78:0/3103187030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f224c126160 con 0x7f224c11d590 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.952+0000 7f22518c7640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11d590 msgr2=0x7f224c11d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.952+0000 7f22518c7640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11d590 0x7f224c11d9a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f224c09f370 tx=0x7f2240004850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240002ba0 con 0x7f224c11d590 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 10 monclient: dump: 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:15.956 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f2240007690 con 0x7f224c11d590 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11d590 msgr2=0x7f224c11d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11d590 0x7f224c11d9a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f224c09f370 tx=0x7f2240004850 comp rx=0 tx=0).stop 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11d590 msgr2=0x7f224c11d9a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2232ffd640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f224c11d590 0x7f224c11d9a0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f224c09f370 tx=0x7f2240004850 comp rx=0 tx=0).stop 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 220, 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 24636, 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:15.957 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:05:15.958 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.959 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:15.960 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:15.961 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f221c03dee0 msgr2=0x7f221c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f221c03dee0 0x7f221c0403a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f224800f2e0 tx=0x7f2248004200 comp rx=0 tx=0).stop 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 10 monclient: shutdown 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f224c06c890 msgr2=0x7f224c134350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f224c06c890 0x7f224c134350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f223c0099a0 tx=0x7f223c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f22528c9640 1 -- 10.20.193.78:0/3103187030 reap_dead start 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 shutdown_connections 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 --2- 10.20.193.78:0/3103187030 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f224c06c890 0x7f224c134350 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 >> 10.20.193.78:0/3103187030 conn(0x7f224c0a6620 msgr2=0x7f224c135340 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 shutdown_connections 2026-02-17T06:05:15.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:15.953+0000 7f2254353640 1 -- 10.20.193.78:0/3103187030 wait complete. 2026-02-17T06:05:15.963 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:05:15.963 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:05:15.963 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:05:16.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.029+0000 7fd1d585b640 10 monclient: get_monmap_and_config 2026-02-17T06:05:16.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.029+0000 7fd1d585b640 10 monclient: build_initial_monmap 2026-02-17T06:05:16.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.029+0000 7fd1d585b640 10 monclient: monmap: 2026-02-17T06:05:16.032 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:16.032 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:16.031062+0000 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:16.031062+0000 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 1 Processor -- start 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 1 -- start start 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient: init 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd1d006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd1d009c4a0 0x7fd1d013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient(hunting): picked mon.noname-c con 0x7fd1d009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:16.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient(hunting): picked mon.noname-b con 0x7fd1d006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d006daf0 con 0x7fd1d006c890 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d00a0880 con 0x7fd1d009c4a0 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d009cc80 con 0x7fd1d006d380 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1d585b640 10 monclient(hunting): authenticate will time out at 1761.584580s 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48618/0 (socket says 10.20.193.78:48618) 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/3104342559 learned_addr learned my addr 10.20.193.78:0/3104342559 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient(hunting): get_auth_request con 0x7fd1d006d380 auth_method 0 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1ce7fc640 1 --2- 10.20.193.78:0/3104342559 >> v2:10.20.193.115:3300/0 conn(0x7fd1d009c4a0 0x7fd1d013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1ce7fc640 10 monclient(hunting): get_auth_request con 0x7fd1d009c4a0 auth_method 0 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient(hunting): handle_auth_done global_id 234149 payload 274 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/3104342559 >> v2:10.20.193.115:3300/0 conn(0x7fd1d009c4a0 msgr2=0x7fd1d013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 1 --2- 10.20.193.78:0/3104342559 >> v2:10.20.193.115:3300/0 conn(0x7fd1d009c4a0 0x7fd1d013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/3104342559 >> v1:10.20.193.115:6789/0 conn(0x7fd1d006c890 legacy=0x7fd1d006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 1 monclient: found mon.noname-b 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.030+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/3104342559 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d009cf60 con 0x7fd1d006d380 2026-02-17T06:05:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/3104342559 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 msgr2=0x7fd1d013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cf7fe640 1 --2- 10.20.193.78:0/3104342559 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d013d8f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009e50 tx=0x7fd1c002f260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c2e5168b90e5d0a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:16.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 1 -- 10.20.193.78:0/3104342559 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00373f0 con 0x7fd1d006d380 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: dump: 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: _finish_auth 0 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: _check_auth_tickets 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 1 -- 10.20.193.78:0/3104342559 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c0037590 con 0x7fd1d006d380 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 5 monclient: authenticate success, global_id 234149 2026-02-17T06:05:16.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 1 -- 10.20.193.78:0/3104342559 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0037960 con 0x7fd1d006d380 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 4 set_mon_vals no callback set 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1cdffb640 10 monclient: dump: 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 10 monclient: shutdown 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 1 -- 10.20.193.78:0/3104342559 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 msgr2=0x7fd1d013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 1 --2- 10.20.193.78:0/3104342559 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d013d8f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009e50 tx=0x7fd1c002f260 comp rx=0 tx=0).stop 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 1 -- 10.20.193.78:0/3104342559 shutdown_connections 2026-02-17T06:05:16.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 1 --2- 10.20.193.78:0/3104342559 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d013d8f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 1 --2- 10.20.193.78:0/3104342559 >> v2:10.20.193.115:3300/0 conn(0x7fd1d009c4a0 0x7fd1d013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 1 -- 10.20.193.78:0/3104342559 >> 10.20.193.78:0/3104342559 conn(0x7fd1d00a6620 msgr2=0x7fd1d00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 1 -- 10.20.193.78:0/3104342559 shutdown_connections 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.033+0000 7fd1d585b640 1 -- 10.20.193.78:0/3104342559 wait complete. 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient: build_initial_monmap 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient: monmap: 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 1 Processor -- start 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 1 -- start start 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient: init 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:16.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d006c890 0x7fd1d0132d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient(hunting): picked mon.noname-b con 0x7fd1d006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d0133270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient(hunting): picked mon.noname-c con 0x7fd1d006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1d009c4a0 0x7fd1d01ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient(hunting): picked mon.noname-a con 0x7fd1d009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d009bfa0 con 0x7fd1d009c4a0 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d009be20 con 0x7fd1d006c890 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d013de30 con 0x7fd1d006d380 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1d585b640 10 monclient(hunting): authenticate will time out at 1761.588654s 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d0133270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1cf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1d009c4a0 0x7fd1d01ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d0133270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48620/0 (socket says 10.20.193.78:48620) 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 -- 10.20.193.78:0/1423296992 learned_addr learned my addr 10.20.193.78:0/1423296992 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient(hunting): get_auth_request con 0x7fd1d006d380 auth_method 0 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1cf7fe640 10 monclient(hunting): get_auth_request con 0x7fd1d009c4a0 auth_method 0 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ceffd640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d006c890 0x7fd1d0132d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ceffd640 10 monclient(hunting): get_auth_request con 0x7fd1d006c890 auth_method 0 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient(hunting): handle_auth_done global_id 234152 payload 824 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d006c890 msgr2=0x7fd1d0132d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d006c890 0x7fd1d0132d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1d009c4a0 msgr2=0x7fd1d01ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd1d009c4a0 0x7fd1d01ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 monclient: found mon.noname-c 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.034+0000 7fd1ce7fc640 1 -- 10.20.193.78:0/1423296992 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d011da20 con 0x7fd1d006d380 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1ce7fc640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 msgr2=0x7fd1d0133270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1ce7fc640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d0133270 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800e7c0 tx=0x7fd1b800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8017070 con 0x7fd1d006d380 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: dump: 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: _finish_auth 0 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: _check_auth_tickets 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b800cae0 con 0x7fd1d006d380 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800f040 con 0x7fd1d006d380 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 10 monclient: dump: 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1cd7fa640 4 set_mon_vals no callback set 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 5 monclient: authenticate success, global_id 234152 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 10 monclient: _renew_subs 2026-02-17T06:05:16.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d006f6c0 con 0x7fd1d006d380 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 10 monclient: _renew_subs 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d009d810 con 0x7fd1d006d380 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d011d5d0 0x7fd1d01cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d011d3c0 con 0x7fd1d011d5d0 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd1b8012050 con 0x7fd1d006d380 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1ceffd640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d011d5d0 0x7fd1d01cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1ceffd640 10 monclient: get_auth_request con 0x7fd1d011d5d0 auth_method 0 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.037+0000 7fd1ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1a7fff640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd1a003deb0 0x7fd1a0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(675..675 src has 525..675) ==== 4795+0+0 (secure 0 0 0) 0x7fd1b8052690 con 0x7fd1d006d380 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1ceffd640 10 monclient(hunting): handle_auth_done global_id 214255 payload 824 2026-02-17T06:05:16.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1cf7fe640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd1a003deb0 0x7fd1a0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1ceffd640 1 -- 10.20.193.78:0/1423296992 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1d0074980 con 0x7fd1d011d5d0 2026-02-17T06:05:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.038+0000 7fd1cf7fe640 10 monclient: get_auth_request con 0x7fd1a003deb0 auth_method 0 2026-02-17T06:05:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.039+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd1a003deb0 msgr2=0x7fd1a0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:05:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.039+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd1a003deb0 msgr2=0x7fd1a0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:05:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.039+0000 7fd1cf7fe640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd1a003deb0 0x7fd1a0040370 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7fd1d006a480 tx=0x7fd1c0033000 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:16.048 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.039+0000 7fd1cf7fe640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd1a003deb0 0x7fd1a0040370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:16.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.040+0000 7fd1ceffd640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d011d5d0 msgr2=0x7fd1d01cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:16.049 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.040+0000 7fd1ceffd640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d011d5d0 0x7fd1d01cd440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400e730 tx=0x7fd1c400ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.040+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4006ae0 con 0x7fd1d011d5d0 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.040+0000 7fd1a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.040+0000 7fd1a7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.040+0000 7fd1a7fff640 10 monclient: dump: 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.041+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd1c4006ae0 con 0x7fd1d011d5d0 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.041+0000 7fd1a7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.041+0000 7fd1a7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.041+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d011d5d0 msgr2=0x7fd1d01cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.041+0000 7fd1a7fff640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d011d5d0 0x7fd1d01cd440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400e730 tx=0x7fd1c400ec00 comp rx=0 tx=0).stop 2026-02-17T06:05:16.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.041+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d011d5d0 msgr2=0x7fd1d01cd440 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.041+0000 7fd1a7fff640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d011d5d0 0x7fd1d01cd440 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400e730 tx=0x7fd1c400ec00 comp rx=0 tx=0).stop 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.043+0000 7fd1d585b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.043+0000 7fd1d585b640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d01cd980 0x7fd1d01d8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.043+0000 7fd1d585b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.043+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d01cdd90 con 0x7fd1d01cd980 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d01cd980 0x7fd1d01d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 10 monclient: get_auth_request con 0x7fd1d01cd980 auth_method 0 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 10 monclient(hunting): handle_auth_done global_id 214258 payload 824 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.044+0000 7fd1ceffd640 1 -- 10.20.193.78:0/1423296992 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd1d013c7e0 con 0x7fd1d01cd980 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1ceffd640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d01cd980 msgr2=0x7fd1d01d8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1ceffd640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d01cd980 0x7fd1d01d8250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400e730 tx=0x7fd1c4003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400c930 con 0x7fd1d01cd980 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:16.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 10 monclient: dump: 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7fd1c400c930 con 0x7fd1d01cd980 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d01cd980 msgr2=0x7fd1d01d8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d01cd980 0x7fd1d01d8250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400e730 tx=0x7fd1c4003860 comp rx=0 tx=0).stop 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d01cd980 msgr2=0x7fd1d01d8250 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:16.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.046+0000 7fd1a7fff640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd1d01cd980 0x7fd1d01d8250 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400e730 tx=0x7fd1c4003860 comp rx=0 tx=0).stop 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 220, 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 118894, 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 4, 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:16.053 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:16.054 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:05:16.055 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.056 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:16.057 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:16.058 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:05:16.059 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:16.060 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:16.061 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:16.061 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:16.061 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:16.061 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:16.061 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:16.061 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:05:16.061 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd1a003deb0 msgr2=0x7fd1a0040370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd1a003deb0 0x7fd1a0040370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 10 monclient: shutdown 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 msgr2=0x7fd1d0133270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 1 --2- 10.20.193.78:0/1423296992 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd1d006d380 0x7fd1d0133270 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800e7c0 tx=0x7fd1b800ec90 comp rx=0 tx=0).stop 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/1423296992 reap_dead start 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1cf7fe640 1 -- 10.20.193.78:0/1423296992 reap_dead start 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 shutdown_connections 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 >> 10.20.193.78:0/1423296992 conn(0x7fd1d00a6620 msgr2=0x7fd1d00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 shutdown_connections 2026-02-17T06:05:16.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.047+0000 7fd1d585b640 1 -- 10.20.193.78:0/1423296992 wait complete. 2026-02-17T06:05:16.062 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:05:16.063 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T06:05:16.063 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:05:16.063 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T06:05:16.063 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T06:05:16.063 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T06:05:16.063 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:05:16.075 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T06:05:16.075 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:05:16.075 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:16.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.180+0000 7fa805eb7640 10 monclient: get_monmap_and_config 2026-02-17T06:05:16.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.180+0000 7fa805eb7640 10 monclient: build_initial_monmap 2026-02-17T06:05:16.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.180+0000 7fa805eb7640 10 monclient: monmap: 2026-02-17T06:05:16.185 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:16.185 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.185 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:16.182066+0000 2026-02-17T06:05:16.185 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:16.182066+0000 2026-02-17T06:05:16.185 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 1 Processor -- start 2026-02-17T06:05:16.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 1 -- start start 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient: init 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80006c890 0x7fa80006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient(hunting): picked mon.noname-b con 0x7fa80006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa80009d430 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa80009bf40 0x7fa80006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient(hunting): picked mon.noname-a con 0x7fa80009bf40 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:16.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa800073600 con 0x7fa80009d430 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa80009ca50 con 0x7fa80009bf40 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa80006d6f0 con 0x7fa80006c890 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa805eb7640 10 monclient(hunting): authenticate will time out at 1761.735591s 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7fffff640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fa80009bf40 msgr2=0x7fa80006d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80006c890 0x7fa80006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7fffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa80009bf40 0x7fa80006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80006c890 0x7fa80006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48666/0 (socket says 10.20.193.78:48666) 2026-02-17T06:05:16.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 1 -- 10.20.193.78:0/404468441 learned_addr learned my addr 10.20.193.78:0/404468441 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient(hunting): get_auth_request con 0x7fa80006c890 auth_method 0 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient(hunting): handle_auth_done global_id 234155 payload 274 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 1 -- 10.20.193.78:0/404468441 >> v2:10.20.193.78:3300/0 conn(0x7fa80009bf40 msgr2=0x7fa80006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 1 --2- 10.20.193.78:0/404468441 >> v2:10.20.193.78:3300/0 conn(0x7fa80009bf40 0x7fa80006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 1 -- 10.20.193.78:0/404468441 >> v1:10.20.193.115:6789/0 conn(0x7fa80009d430 legacy=0x7fa800134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:16.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:05:16.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 1 monclient: found mon.noname-b 2026-02-17T06:05:16.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:16.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.181+0000 7fa7ff7fe640 1 -- 10.20.193.78:0/404468441 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80006dab0 con 0x7fa80006c890 2026-02-17T06:05:16.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7ff7fe640 1 -- 10.20.193.78:0/404468441 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80006c890 msgr2=0x7fa80006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:16.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7ff7fe640 1 --2- 10.20.193.78:0/404468441 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80006c890 0x7fa80006cc70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa80009c670 tx=0x7fa7ec02ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46969058f36493bb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:16.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 1 -- 10.20.193.78:0/404468441 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec03c070 con 0x7fa80006c890 2026-02-17T06:05:16.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: dump: 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.198 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: _finish_auth 0 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 1 -- 10.20.193.78:0/404468441 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ec02fda0 con 0x7fa80006c890 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 5 monclient: authenticate success, global_id 234155 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 1 -- 10.20.193.78:0/404468441 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec037660 con 0x7fa80006c890 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:16.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 4 set_mon_vals no callback set 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa7fe7fc640 10 monclient: dump: 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.200 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 10 monclient: shutdown 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 1 -- 10.20.193.78:0/404468441 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80006c890 msgr2=0x7fa80006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 1 --2- 10.20.193.78:0/404468441 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80006c890 0x7fa80006cc70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa80009c670 tx=0x7fa7ec02ee80 comp rx=0 tx=0).stop 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 1 -- 10.20.193.78:0/404468441 shutdown_connections 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 1 --2- 10.20.193.78:0/404468441 >> v2:10.20.193.78:3300/0 conn(0x7fa80009bf40 0x7fa80006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 1 --2- 10.20.193.78:0/404468441 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80006c890 0x7fa80006cc70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 1 -- 10.20.193.78:0/404468441 >> 10.20.193.78:0/404468441 conn(0x7fa8000a6620 msgr2=0x7fa8000a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 1 -- 10.20.193.78:0/404468441 shutdown_connections 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.184+0000 7fa805eb7640 1 -- 10.20.193.78:0/404468441 wait complete. 2026-02-17T06:05:16.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient: build_initial_monmap 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient: monmap: 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:16.202 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 1 Processor -- start 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 1 -- start start 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient: init 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa80006c890 0x7fa8001cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient(hunting): picked mon.noname-b con 0x7fa80006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa80009bf40 0x7fa8001cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient(hunting): picked mon.noname-a con 0x7fa80009bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80009d430 0x7fa800121750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient(hunting): picked mon.noname-c con 0x7fa80009d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000733c0 con 0x7fa80009bf40 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa800073240 con 0x7fa80006c890 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa800135500 con 0x7fa80009d430 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa805eb7640 10 monclient(hunting): authenticate will time out at 1761.739853s 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7feffd640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa80009bf40 msgr2=0x7fa8001cfaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7feffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa80009bf40 0x7fa8001cfaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80009d430 0x7fa800121750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80009d430 0x7fa800121750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48670/0 (socket says 10.20.193.78:48670) 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 1 -- 10.20.193.78:0/2460792623 learned_addr learned my addr 10.20.193.78:0/2460792623 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:16.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 10 monclient(hunting): get_auth_request con 0x7fa80009d430 auth_method 0 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.185+0000 7fa7fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 10 monclient(hunting): handle_auth_done global_id 234158 payload 824 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 1 -- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa80006c890 msgr2=0x7fa8001cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa80006c890 0x7fa8001cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 1 -- 10.20.193.78:0/2460792623 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa80009bf40 msgr2=0x7fa8001cfaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa80009bf40 0x7fa8001cfaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 10 monclient: _finish_hunting 0 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 1 monclient: found mon.noname-c 2026-02-17T06:05:16.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.186+0000 7fa7fffff640 1 -- 10.20.193.78:0/2460792623 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8001250d0 con 0x7fa80009d430 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7fffff640 1 -- 10.20.193.78:0/2460792623 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80009d430 msgr2=0x7fa800121750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7fffff640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80009d430 0x7fa800121750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400be90 tx=0x7fa7f400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 1 -- 10.20.193.78:0/2460792623 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4016430 con 0x7fa80009d430 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: dump: 2026-02-17T06:05:16.206 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: _finish_auth 0 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: _check_auth_tickets 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 1 -- 10.20.193.78:0/2460792623 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f4010030 con 0x7fa80009d430 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:16.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 1 -- 10.20.193.78:0/2460792623 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4016810 con 0x7fa80009d430 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7fdffb640 4 set_mon_vals no callback set 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa7dbfff640 10 monclient: dump: 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:16.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 5 monclient: authenticate success, global_id 234158 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 10 monclient: _renew_subs 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8001253b0 con 0x7fa80009d430 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 10 monclient: _renew_subs 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa800134e80 con 0x7fa80009d430 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.188+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa800135930 con 0x7fa80009d430 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.189+0000 7fa7dbfff640 1 -- 10.20.193.78:0/2460792623 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa7f4008070 con 0x7fa80009d430 2026-02-17T06:05:16.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.189+0000 7fa7dbfff640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7d003de60 0x7fa7d0040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:16.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.189+0000 7fa7dbfff640 1 -- 10.20.193.78:0/2460792623 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(675..675 src has 525..675) ==== 4795+0+0 (secure 0 0 0) 0x7fa7f4052710 con 0x7fa80009d430 2026-02-17T06:05:16.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.189+0000 7fa7ff7fe640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7d003de60 0x7fa7d0040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:16.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.189+0000 7fa7ff7fe640 10 monclient: get_auth_request con 0x7fa7d003de60 auth_method 0 2026-02-17T06:05:16.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.191+0000 7fa7dbfff640 1 -- 10.20.193.78:0/2460792623 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa7f4014dc0 con 0x7fa80009d430 2026-02-17T06:05:16.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.191+0000 7fa7dbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:16.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.191+0000 7fa7dbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:16.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.192+0000 7fa7ff7fe640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7d003de60 0x7fa7d0040320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec036000 tx=0x7fa7ec031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:16.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.308+0000 7fa805eb7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:05:16.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.308+0000 7fa805eb7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:16.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.308+0000 7fa805eb7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:16.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.308+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa8000744d0 con 0x7fa80009d430 2026-02-17T06:05:16.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.308+0000 7fa7dbfff640 1 -- 10.20.193.78:0/2460792623 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa7f4015c30 con 0x7fa80009d430 2026-02-17T06:05:16.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.308+0000 7fa7dbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:16.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.308+0000 7fa7dbfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:16.309 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":220,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7d003de60 msgr2=0x7fa7d0040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7d003de60 0x7fa7d0040320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec036000 tx=0x7fa7ec031000 comp rx=0 tx=0).stop 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 10 monclient: shutdown 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80009d430 msgr2=0x7fa800121750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80009d430 0x7fa800121750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400be90 tx=0x7fa7f400bf90 comp rx=0 tx=0).stop 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 shutdown_connections 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa80009d430 0x7fa800121750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa80009bf40 0x7fa8001cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7d003de60 0x7fa7d0040320 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 --2- 10.20.193.78:0/2460792623 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa80006c890 0x7fa8001cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 >> 10.20.193.78:0/2460792623 conn(0x7fa8000a6620 msgr2=0x7fa800136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 shutdown_connections 2026-02-17T06:05:16.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:16.309+0000 7fa805eb7640 1 -- 10.20.193.78:0/2460792623 wait complete. 2026-02-17T06:05:19.318 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient: get_monmap_and_config 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient: build_initial_monmap 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient: monmap: 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:19.387478+0000 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:19.387478+0000 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:19.387 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 1 Processor -- start 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 1 -- start start 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient: init 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc9d406c890 0x7fc9d406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient(hunting): picked mon.noname-a con 0x7fc9d406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc9d409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc9d406d1b0 0x7fc9d4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient(hunting): picked mon.noname-c con 0x7fc9d406d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9d406daf0 con 0x7fc9d409c4a0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d40a0880 con 0x7fc9d406c890 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d409cc80 con 0x7fc9d406d1b0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.386+0000 7fc9dc096640 10 monclient(hunting): authenticate will time out at 1764.940987s 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9d960a640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7fc9d409c4a0 legacy=0x7fc9d4134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9d9e0b640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fc9d406c890 msgr2=0x7fc9d406cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9d9e0b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc9d406c890 0x7fc9d406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc9d406d1b0 0x7fc9d4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc9d406d1b0 0x7fc9d4135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:36400/0 (socket says 10.20.193.78:36400) 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 1 -- 10.20.193.78:0/860598863 learned_addr learned my addr 10.20.193.78:0/860598863 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient(hunting): get_auth_request con 0x7fc9d406d1b0 auth_method 0 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:19.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient(hunting): handle_auth_done global_id 214261 payload 274 2026-02-17T06:05:19.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 1 -- 10.20.193.78:0/860598863 >> v2:10.20.193.78:3300/0 conn(0x7fc9d406c890 msgr2=0x7fc9d406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:19.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 1 --2- 10.20.193.78:0/860598863 >> v2:10.20.193.78:3300/0 conn(0x7fc9d406c890 0x7fc9d406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:19.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 1 -- 10.20.193.78:0/860598863 >> v1:10.20.193.78:6789/0 conn(0x7fc9d409c4a0 legacy=0x7fc9d4134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:19.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient: _finish_hunting 0 2026-02-17T06:05:19.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 1 monclient: found mon.noname-c 2026-02-17T06:05:19.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:19.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:05:19.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.387+0000 7fc9da60c640 1 -- 10.20.193.78:0/860598863 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d409cf60 con 0x7fc9d406d1b0 2026-02-17T06:05:19.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9da60c640 1 -- 10.20.193.78:0/860598863 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 msgr2=0x7fc9d4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:19.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9da60c640 1 --2- 10.20.193.78:0/860598863 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 0x7fc9d4135500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4074de0 tx=0x7fc9d002f200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d09fb89e09e7bb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 1 -- 10.20.193.78:0/860598863 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d00373f0 con 0x7fc9d406d1b0 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: dump: 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: _finish_auth 0 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: _check_auth_tickets 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 5 monclient: authenticate success, global_id 214261 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 1 -- 10.20.193.78:0/860598863 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d0037590 con 0x7fc9d406d1b0 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 1 -- 10.20.193.78:0/860598863 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0037960 con 0x7fc9d406d1b0 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 4 set_mon_vals no callback set 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9d8e09640 10 monclient: dump: 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:19.391 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 10 monclient: shutdown 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 1 -- 10.20.193.78:0/860598863 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 msgr2=0x7fc9d4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 1 --2- 10.20.193.78:0/860598863 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 0x7fc9d4135500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4074de0 tx=0x7fc9d002f200 comp rx=0 tx=0).stop 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 1 -- 10.20.193.78:0/860598863 shutdown_connections 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 1 --2- 10.20.193.78:0/860598863 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 0x7fc9d4135500 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 1 --2- 10.20.193.78:0/860598863 >> v2:10.20.193.78:3300/0 conn(0x7fc9d406c890 0x7fc9d406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 1 -- 10.20.193.78:0/860598863 >> 10.20.193.78:0/860598863 conn(0x7fc9d40a6620 msgr2=0x7fc9d40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 1 -- 10.20.193.78:0/860598863 shutdown_connections 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.390+0000 7fc9dc096640 1 -- 10.20.193.78:0/860598863 wait complete. 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient: build_initial_monmap 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient: monmap: 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 1 Processor -- start 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 1 -- start start 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient: init 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 0x7fc9d41cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient(hunting): picked mon.noname-c con 0x7fc9d406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 0x7fc9d41cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient(hunting): picked mon.noname-b con 0x7fc9d406d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc9d409c4a0 0x7fc9d4123580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient(hunting): picked mon.noname-a con 0x7fc9d409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d40a01a0 con 0x7fc9d409c4a0 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d40a0020 con 0x7fc9d406d1b0 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d409be20 con 0x7fc9d406c890 2026-02-17T06:05:19.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9dc096640 10 monclient(hunting): authenticate will time out at 1764.945624s 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9da60c640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc9d409c4a0 msgr2=0x7fc9d4123580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9da60c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc9d409c4a0 0x7fc9d4123580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 0x7fc9d41cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 0x7fc9d41cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47576/0 (socket says 10.20.193.78:47576) 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 -- 10.20.193.78:0/1913361329 learned_addr learned my addr 10.20.193.78:0/1913361329 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient(hunting): get_auth_request con 0x7fc9d406c890 auth_method 0 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d960a640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 0x7fc9d41cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient(hunting): handle_auth_done global_id 234161 payload 824 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 -- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 msgr2=0x7fc9d41cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 0x7fc9d41cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 -- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc9d409c4a0 msgr2=0x7fc9d4123580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc9d409c4a0 0x7fc9d4123580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient: _finish_hunting 0 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 monclient: found mon.noname-c 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d9e0b640 1 -- 10.20.193.78:0/1913361329 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d406fd70 con 0x7fc9d406c890 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d960a640 10 monclient: get_auth_request con 0x7fc9d406d1b0 auth_method 0 2026-02-17T06:05:19.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.391+0000 7fc9d960a640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 0x7fc9d41cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.393+0000 7fc9d9e0b640 1 -- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 msgr2=0x7fc9d41cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.393+0000 7fc9d9e0b640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 0x7fc9d41cd480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400b780 tx=0x7fc9c400bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9d9e0b640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 0x7fc9d41cd480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400b780 tx=0x7fc9c400bc50 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fc9c4005470 mon_map magic: 0 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 1 -- 10.20.193.78:0/1913361329 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4005070 con 0x7fc9d406c890 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: dump: 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: _finish_auth 0 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: _check_auth_tickets 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 1 -- 10.20.193.78:0/1913361329 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c4010070 con 0x7fc9d406c890 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 5 monclient: authenticate success, global_id 234161 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 10 monclient: _renew_subs 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d409d810 con 0x7fc9d406c890 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c3fff640 4 set_mon_vals no callback set 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 10 monclient: _renew_subs 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:19.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d41271e0 con 0x7fc9d406c890 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 1 -- 10.20.193.78:0/1913361329 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4005470 con 0x7fc9d406c890 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 10 monclient: dump: 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 1 -- 10.20.193.78:0/1913361329 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc9c40056d0 con 0x7fc9d406c890 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d406cc70 con 0x7fc9d406c890 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.394+0000 7fc9c27fc640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc9a403de90 0x7fc9a4040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.395+0000 7fc9d960a640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc9a403de90 0x7fc9a4040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:19.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.395+0000 7fc9d960a640 10 monclient: get_auth_request con 0x7fc9a403de90 auth_method 0 2026-02-17T06:05:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.397+0000 7fc9d960a640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc9a403de90 0x7fc9a4040350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8006fd0 tx=0x7fc9c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:19.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:19.397+0000 7fc9c27fc640 1 -- 10.20.193.78:0/1913361329 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(675..675 src has 525..675) ==== 4795+0+0 (secure 0 0 0) 0x7fc9c4052190 con 0x7fc9d406c890 2026-02-17T06:05:20.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.018+0000 7fc9c27fc640 1 -- 10.20.193.78:0/1913361329 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc9c4050b80 con 0x7fc9d406c890 2026-02-17T06:05:20.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.018+0000 7fc9c27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:20.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.018+0000 7fc9c27fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:20.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.132+0000 7fc9dc096640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:05:20.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.132+0000 7fc9dc096640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:20.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.132+0000 7fc9dc096640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:20.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.132+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc9d40744d0 con 0x7fc9d406c890 2026-02-17T06:05:20.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.132+0000 7fc9c27fc640 1 -- 10.20.193.78:0/1913361329 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fc9c400cdd0 con 0x7fc9d406c890 2026-02-17T06:05:20.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.132+0000 7fc9c27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:20.133 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.132+0000 7fc9c27fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:20.133 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":220,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc9a403de90 msgr2=0x7fc9a4040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc9a403de90 0x7fc9a4040350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8006fd0 tx=0x7fc9c8008040 comp rx=0 tx=0).stop 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 10 monclient: shutdown 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 msgr2=0x7fc9d41cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 0x7fc9d41cd480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400b780 tx=0x7fc9c400bc50 comp rx=0 tx=0).stop 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 shutdown_connections 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc9d409c4a0 0x7fc9d4123580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc9a403de90 0x7fc9a4040350 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc9d406d1b0 0x7fc9d41cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 --2- 10.20.193.78:0/1913361329 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc9d406c890 0x7fc9d41cd480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 >> 10.20.193.78:0/1913361329 conn(0x7fc9d40a6620 msgr2=0x7fc9d40722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 shutdown_connections 2026-02-17T06:05:20.134 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:20.133+0000 7fc9dc096640 1 -- 10.20.193.78:0/1913361329 wait complete. 2026-02-17T06:05:23.142 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient: get_monmap_and_config 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient: build_initial_monmap 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient: monmap: 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:23.212403+0000 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:23.212403+0000 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:23.212 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:23.213 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 1 Processor -- start 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 1 -- start start 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient: init 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient: _add_conns ranks=[0,3,4,1,2] 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff5a806c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff5a809c4a0 0x7ff5a813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient(hunting): picked mon.noname-a con 0x7ff5a809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff5a806d380 0x7ff5a813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:23.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient(hunting): picked mon.noname-c con 0x7ff5a806d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a806daf0 con 0x7ff5a806c890 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a80a0880 con 0x7ff5a809c4a0 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a809cc80 con 0x7ff5a806d380 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5b0333640 10 monclient(hunting): authenticate will time out at 1768.765932s 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5ae0a8640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7ff5a806c890 legacy=0x7ff5a806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5ad8a7640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7ff5a809c4a0 msgr2=0x7ff5a813b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.211+0000 7ff5ad8a7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff5a809c4a0 0x7ff5a813b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff5a806d380 0x7ff5a813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff5a806d380 0x7ff5a813d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:36406/0 (socket says 10.20.193.78:36406) 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 1 -- 10.20.193.78:0/1165116584 learned_addr learned my addr 10.20.193.78:0/1165116584 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient(hunting): get_auth_request con 0x7ff5a806d380 auth_method 0 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient(hunting): handle_auth_done global_id 214264 payload 274 2026-02-17T06:05:23.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 1 -- 10.20.193.78:0/1165116584 >> v2:10.20.193.78:3300/0 conn(0x7ff5a809c4a0 msgr2=0x7ff5a813b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:23.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 1 --2- 10.20.193.78:0/1165116584 >> v2:10.20.193.78:3300/0 conn(0x7ff5a809c4a0 0x7ff5a813b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 1 -- 10.20.193.78:0/1165116584 >> v1:10.20.193.78:6789/0 conn(0x7ff5a806c890 legacy=0x7ff5a806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:23.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient: _finish_hunting 0 2026-02-17T06:05:23.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 1 monclient: found mon.noname-c 2026-02-17T06:05:23.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:23.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:05:23.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.212+0000 7ff5ae8a9640 1 -- 10.20.193.78:0/1165116584 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a809cf60 con 0x7ff5a806d380 2026-02-17T06:05:23.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.214+0000 7ff5ae8a9640 1 -- 10.20.193.78:0/1165116584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806d380 msgr2=0x7ff5a813d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.214+0000 7ff5ae8a9640 1 --2- 10.20.193.78:0/1165116584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806d380 0x7ff5a813d8f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5a806dcc0 tx=0x7ff5a402f200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6df190c4e4ea60f7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 1 -- 10.20.193.78:0/1165116584 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40373f0 con 0x7ff5a806d380 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: dump: 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: _finish_auth 0 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: _check_auth_tickets 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:23.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 5 monclient: authenticate success, global_id 214264 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 1 -- 10.20.193.78:0/1165116584 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a4037590 con 0x7ff5a806d380 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 1 -- 10.20.193.78:0/1165116584 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4037960 con 0x7ff5a806d380 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 4 set_mon_vals no callback set 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5ad0a6640 10 monclient: dump: 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 10 monclient: shutdown 2026-02-17T06:05:23.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 1 -- 10.20.193.78:0/1165116584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806d380 msgr2=0x7ff5a813d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 1 --2- 10.20.193.78:0/1165116584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806d380 0x7ff5a813d8f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5a806dcc0 tx=0x7ff5a402f200 comp rx=0 tx=0).stop 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 1 -- 10.20.193.78:0/1165116584 shutdown_connections 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 1 --2- 10.20.193.78:0/1165116584 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806d380 0x7ff5a813d8f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 1 --2- 10.20.193.78:0/1165116584 >> v2:10.20.193.78:3300/0 conn(0x7ff5a809c4a0 0x7ff5a813b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 1 -- 10.20.193.78:0/1165116584 >> 10.20.193.78:0/1165116584 conn(0x7ff5a80a6620 msgr2=0x7ff5a80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 1 -- 10.20.193.78:0/1165116584 shutdown_connections 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 1 -- 10.20.193.78:0/1165116584 wait complete. 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 10 monclient: build_initial_monmap 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.215+0000 7ff5b0333640 10 monclient: monmap: 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:23.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 1 Processor -- start 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 1 -- start start 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient: init 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806c890 0x7ff5a8072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient(hunting): picked mon.noname-b con 0x7ff5a806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff5a806d380 0x7ff5a8133390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient(hunting): picked mon.noname-c con 0x7ff5a806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff5a809c4a0 0x7ff5a81338d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient(hunting): picked mon.noname-a con 0x7ff5a809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a809bfa0 con 0x7ff5a809c4a0 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a809be20 con 0x7ff5a806c890 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a813de30 con 0x7ff5a806d380 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5b0333640 10 monclient(hunting): authenticate will time out at 1768.770348s 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ae8a9640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff5a809c4a0 msgr2=0x7ff5a81338d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ae8a9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff5a809c4a0 0x7ff5a81338d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:23.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff5a806d380 0x7ff5a8133390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff5a806d380 0x7ff5a8133390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47590/0 (socket says 10.20.193.78:47590) 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 -- 10.20.193.78:0/627158165 learned_addr learned my addr 10.20.193.78:0/627158165 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient(hunting): get_auth_request con 0x7ff5a806d380 auth_method 0 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ae0a8640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806c890 0x7ff5a8072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ae0a8640 10 monclient(hunting): get_auth_request con 0x7ff5a806c890 auth_method 0 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ae0a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ae0a8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ae0a8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient(hunting): handle_auth_done global_id 234164 payload 824 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 -- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806c890 msgr2=0x7ff5a8072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806c890 0x7ff5a8072530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 -- 10.20.193.78:0/627158165 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff5a809c4a0 msgr2=0x7ff5a81338d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff5a809c4a0 0x7ff5a81338d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient: _finish_hunting 0 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 monclient: found mon.noname-c 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:23.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:23.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.216+0000 7ff5ad8a7640 1 -- 10.20.193.78:0/627158165 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a811e540 con 0x7ff5a806d380 2026-02-17T06:05:23.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5ad8a7640 1 -- 10.20.193.78:0/627158165 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff5a806d380 msgr2=0x7ff5a8133390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:23.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5ad8a7640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff5a806d380 0x7ff5a8133390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00b780 tx=0x7ff59c00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:23.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 1 -- 10.20.193.78:0/627158165 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c005270 con 0x7ff5a806d380 2026-02-17T06:05:23.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:23.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:23.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: dump: 2026-02-17T06:05:23.225 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: _finish_auth 0 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: _check_auth_tickets 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 1 -- 10.20.193.78:0/627158165 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59c010070 con 0x7ff5a806d380 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5b0333640 5 monclient: authenticate success, global_id 234164 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5b0333640 10 monclient: _renew_subs 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5b0333640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a811d130 con 0x7ff5a806d380 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 1 -- 10.20.193.78:0/627158165 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c005650 con 0x7ff5a806d380 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:23.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5ac8a5640 4 set_mon_vals no callback set 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff58effd640 10 monclient: dump: 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5b0333640 10 monclient: _renew_subs 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5b0333640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.218+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a809e4c0 con 0x7ff5a806d380 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.219+0000 7ff5b0333640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.219+0000 7ff58effd640 1 -- 10.20.193.78:0/627158165 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff59c01d030 con 0x7ff5a806d380 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.219+0000 7ff5b0333640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.219+0000 7ff5b0333640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.219+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a806cc70 con 0x7ff5a806d380 2026-02-17T06:05:23.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.219+0000 7ff58effd640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff57803de60 0x7ff578040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:23.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.219+0000 7ff58effd640 1 -- 10.20.193.78:0/627158165 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(675..675 src has 525..675) ==== 4795+0+0 (secure 0 0 0) 0x7ff59c051d20 con 0x7ff5a806d380 2026-02-17T06:05:23.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.220+0000 7ff5ae0a8640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff57803de60 0x7ff578040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:23.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.220+0000 7ff5ae0a8640 10 monclient: get_auth_request con 0x7ff57803de60 auth_method 0 2026-02-17T06:05:23.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.222+0000 7ff58effd640 1 -- 10.20.193.78:0/627158165 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff5a811d130 con 0x7ff5a806d380 2026-02-17T06:05:23.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.222+0000 7ff58effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:23.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.222+0000 7ff58effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:23.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.222+0000 7ff5ae0a8640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff57803de60 0x7ff578040320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff598003a80 tx=0x7ff598003ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.337+0000 7ff5b0333640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:05:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.337+0000 7ff5b0333640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.337+0000 7ff5b0333640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.337+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff5a811e280 con 0x7ff5a806d380 2026-02-17T06:05:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.337+0000 7ff58effd640 1 -- 10.20.193.78:0/627158165 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7ff59c050710 con 0x7ff5a806d380 2026-02-17T06:05:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.337+0000 7ff58effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:23.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.337+0000 7ff58effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:23.339 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":220,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:05:23.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff57803de60 msgr2=0x7ff578040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:23.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff57803de60 0x7ff578040320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff598003a80 tx=0x7ff598003ab0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 10 monclient: shutdown 2026-02-17T06:05:23.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:23.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff5a806d380 msgr2=0x7ff5a8133390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:23.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff5a806d380 0x7ff5a8133390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00b780 tx=0x7ff59c00bc50 comp rx=0 tx=0).stop 2026-02-17T06:05:23.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 shutdown_connections 2026-02-17T06:05:23.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff5a809c4a0 0x7ff5a81338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff5a806d380 0x7ff5a8133390 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff57803de60 0x7ff578040320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 --2- 10.20.193.78:0/627158165 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff5a806c890 0x7ff5a8072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:23.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 >> 10.20.193.78:0/627158165 conn(0x7ff5a80a6620 msgr2=0x7ff5a8134240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:23.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 shutdown_connections 2026-02-17T06:05:23.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:23.338+0000 7ff5b0333640 1 -- 10.20.193.78:0/627158165 wait complete. 2026-02-17T06:05:26.349 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:26.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.418+0000 7fda2e887640 10 monclient: get_monmap_and_config 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.418+0000 7fda2e887640 10 monclient: build_initial_monmap 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.418+0000 7fda2e887640 10 monclient: monmap: 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:26.420017+0000 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:26.420017+0000 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 1 Processor -- start 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 1 -- start start 2026-02-17T06:05:26.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient: init 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient: _add_conns ranks=[0,2,3,4,1] 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fda2806c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient(hunting): picked mon.noname-b con 0x7fda2809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 0x7fda2813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient(hunting): picked mon.noname-a con 0x7fda2806d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda2806daf0 con 0x7fda2806c890 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fda280a0880 con 0x7fda2806d380 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fda2809cc80 con 0x7fda2809c4a0 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2e887640 10 monclient(hunting): authenticate will time out at 1771.973584s 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda27fff640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7fda2806c890 legacy=0x7fda2806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2cdfd640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 msgr2=0x7fda2813d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda2cdfd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 0x7fda2813d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47594/0 (socket says 10.20.193.78:47594) 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 learned_addr learned my addr 10.20.193.78:0/1999763533 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 10 monclient(hunting): get_auth_request con 0x7fda2809c4a0 auth_method 0 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:26.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.419+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:26.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.619+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.78:6789/0 conn(0x7fda2806c890 legacy=0x7fda2806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:26.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.619+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 msgr2=0x7fda2813d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.619+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 0x7fda2813d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 10 monclient(hunting): get_auth_request con 0x7fda2809c4a0 auth_method 0 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:26.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:26.620+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:27.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.78:6789/0 conn(0x7fda2806c890 legacy=0x7fda2806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 msgr2=0x7fda2813d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 0x7fda2813d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 10 monclient(hunting): get_auth_request con 0x7fda2809c4a0 auth_method 0 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:27.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:27.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:27.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.021+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:05:27.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.78:6789/0 conn(0x7fda2806c890 legacy=0x7fda2806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:27.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 msgr2=0x7fda2813d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:27.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 0x7fda2813d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:05:27.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:27.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 10 monclient(hunting): get_auth_request con 0x7fda2809c4a0 auth_method 0 2026-02-17T06:05:27.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:27.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:27.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:27.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:27.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:27.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:27.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:27.821+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 10 monclient(hunting): tick 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 monclient(hunting): continuing hunt 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 msgr2=0x7fda2813b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda2809c4a0 0x7fda2813b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 msgr2=0x7fda2813d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda2806d380 0x7fda2813d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.78:6789/0 conn(0x7fda2806c890 legacy=0x7fda2806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 10 monclient: _add_conns ranks=[1,4] 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fda040037a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 10 monclient(hunting): picked mon.noname-c con 0x7fda040074f0 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:29.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda2809cf60 con 0x7fda040037a0 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fda2809cc80 con 0x7fda040074f0 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda25ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 10 monclient(hunting): get_auth_request con 0x7fda040074f0 auth_method 0 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:29.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.420+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:29.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.422+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.115:6789/0 conn(0x7fda040037a0 legacy=0x7fda04005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:29.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:29.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 10 monclient(hunting): get_auth_request con 0x7fda040074f0 auth_method 0 2026-02-17T06:05:29.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:29.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:29.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:29.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:05:29.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:29.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:29.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:29.621+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:30.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:30.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 10 monclient(hunting): get_auth_request con 0x7fda040074f0 auth_method 0 2026-02-17T06:05:30.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:30.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:30.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:30.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:05:30.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:30.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:30.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.022+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:05:30.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.822+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:30.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.822+0000 7fda277fe640 10 monclient(hunting): get_auth_request con 0x7fda040074f0 auth_method 0 2026-02-17T06:05:30.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.822+0000 7fda277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:30.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.822+0000 7fda277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:30.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.822+0000 7fda277fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:30.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.823+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:05:30.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.823+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:30.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.823+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:30.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:30.823+0000 7fda277fe640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): tick 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 monclient(hunting): continuing hunt 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda040074f0 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.115:6789/0 conn(0x7fda040037a0 legacy=0x7fda04005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 reap_dead start 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2026-02-17T06:05:32.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 0x7fda040099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): picked mon.noname-a con 0x7fda0400a140 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fda0400a5c0 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): picked mon.noname-c con 0x7fda0400ab70 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda2806daf0 con 0x7fda0400a5c0 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fda2809cc80 con 0x7fda0400a140 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fda0400e960 con 0x7fda0400ab70 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda25ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request con 0x7fda0400ab70 auth_method 0 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:32.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.420+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:32.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.422+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.115:6789/0 conn(0x7fda0400a5c0 legacy=0x7fda04005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:32.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request con 0x7fda0400ab70 auth_method 0 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:32.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:32.620+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:33.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:33.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:05:33.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:33.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request con 0x7fda0400ab70 auth_method 0 2026-02-17T06:05:33.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:33.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:33.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:33.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:33.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:33.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:33.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.021+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:05:33.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.822+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:33.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.822+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:05:33.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.822+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:33.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.822+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request con 0x7fda0400ab70 auth_method 0 2026-02-17T06:05:33.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.822+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:33.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.822+0000 7fda2cdfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:33.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.822+0000 7fda2cdfd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:33.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.823+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:33.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.823+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:33.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.823+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:33.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:33.823+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:05:35.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 10 monclient(hunting): tick 2026-02-17T06:05:35.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 monclient(hunting): continuing hunt 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 msgr2=0x7fda0400d2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.115:3300/0 conn(0x7fda0400ab70 0x7fda0400d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 msgr2=0x7fda040099b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> v2:10.20.193.78:3300/0 conn(0x7fda0400a140 0x7fda040099b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.115:6789/0 conn(0x7fda0400a5c0 legacy=0x7fda04005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 10 monclient: _add_conns ranks=[2,0] 2026-02-17T06:05:35.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 10 monclient(hunting): picked mon.noname-b con 0x7fda0400d9b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fda0400e4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0400eb60 con 0x7fda0400e4a0 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 1 -- 10.20.193.78:0/1999763533 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fda2809cc80 con 0x7fda0400d9b0 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda25ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.78:6789/0 conn(0x7fda0400e4a0 legacy=0x7fda04012df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 10 monclient(hunting): get_auth_request con 0x7fda0400d9b0 auth_method 0 2026-02-17T06:05:35.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:35.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:35.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:35.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 msgr2=0x7fda0400dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:35.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 msgr2=0x7fda0400dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:35.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:35.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.420+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.78:6789/0 conn(0x7fda0400e4a0 legacy=0x7fda04012df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 10 monclient(hunting): get_auth_request con 0x7fda0400d9b0 auth_method 0 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 msgr2=0x7fda0400dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 msgr2=0x7fda0400dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:35.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:35.621+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:35.824 INFO:tasks.rados.rados.0.trial115.stdout:3149: done (4 left) 2026-02-17T06:05:35.824 INFO:tasks.rados.rados.0.trial115.stdout:3150: copy_from oid 16 from oid 11 current snap is 329 2026-02-17T06:05:35.824 INFO:tasks.rados.rados.0.trial115.stdout:3144: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:05:35.942 INFO:tasks.rados.rados.0.trial115.stdout:3146: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:05:36.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda277fe640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.78:6789/0 conn(0x7fda0400e4a0 legacy=0x7fda04012df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient(hunting): get_auth_request con 0x7fda0400d9b0 auth_method 0 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient(hunting): handle_auth_done global_id 234167 payload 274 2026-02-17T06:05:36.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> v1:10.20.193.78:6789/0 conn(0x7fda0400e4a0 legacy=0x7fda04012df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:36.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient: _finish_hunting 0 2026-02-17T06:05:36.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 1 monclient: found mon.noname-b 2026-02-17T06:05:36.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:36.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.021+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda04015690 con 0x7fda0400d9b0 2026-02-17T06:05:36.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda27fff640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 msgr2=0x7fda0400dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda27fff640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda04007430 tx=0x7fda14030200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b85b1217e4b0eb8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 1 -- 10.20.193.78:0/1999763533 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1403d070 con 0x7fda0400d9b0 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 10 monclient: dump: 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 10 monclient: _finish_auth 0 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 1 -- 10.20.193.78:0/1999763533 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda14038440 con 0x7fda0400d9b0 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.023+0000 7fda26ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 5 monclient: authenticate success, global_id 234167 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda26ffd640 1 -- 10.20.193.78:0/1999763533 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14038830 con 0x7fda0400d9b0 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 4 set_mon_vals no callback set 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda26ffd640 10 monclient: dump: 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.026 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 10 monclient: shutdown 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 1 -- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 msgr2=0x7fda0400dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 1 --2- 10.20.193.78:0/1999763533 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda0400d9b0 0x7fda0400dd90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda04007430 tx=0x7fda14030200 comp rx=0 tx=0).stop 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2cdfd640 1 -- 10.20.193.78:0/1999763533 reap_dead start 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 1 -- 10.20.193.78:0/1999763533 shutdown_connections 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 1 -- 10.20.193.78:0/1999763533 >> 10.20.193.78:0/1999763533 conn(0x7fda280a6620 msgr2=0x7fda280a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 1 -- 10.20.193.78:0/1999763533 shutdown_connections 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.024+0000 7fda2e887640 1 -- 10.20.193.78:0/1999763533 wait complete. 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient: build_initial_monmap 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient: monmap: 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 1 Processor -- start 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 1 -- start start 2026-02-17T06:05:36.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient: init 2026-02-17T06:05:36.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:36.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:05:36.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda0400d9b0 0x7fda281cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient(hunting): picked mon.noname-a con 0x7fda0400d9b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fda281cd600 0x7fda281cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient(hunting): picked mon.noname-b con 0x7fda281cd600 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda28123310 0x7fda28123790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient(hunting): picked mon.noname-c con 0x7fda28123310 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda2809bfa0 con 0x7fda0400d9b0 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fda2809be20 con 0x7fda281cd600 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fda2806cc70 con 0x7fda28123310 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2e887640 10 monclient(hunting): authenticate will time out at 1781.579607s 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda27fff640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda0400d9b0 msgr2=0x7fda281cd0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda27fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda0400d9b0 0x7fda281cd0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda28123310 0x7fda28123790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda28123310 0x7fda28123790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36096/0 (socket says 10.20.193.78:36096) 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 -- 10.20.193.78:0/713529648 learned_addr learned my addr 10.20.193.78:0/713529648 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request con 0x7fda28123310 auth_method 0 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient(hunting): handle_auth_done global_id 234170 payload 824 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 -- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fda281cd600 msgr2=0x7fda281cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fda281cd600 0x7fda281cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 -- 10.20.193.78:0/713529648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda0400d9b0 msgr2=0x7fda281cd0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda0400d9b0 0x7fda281cd0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient: _finish_hunting 0 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 monclient: found mon.noname-c 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.025+0000 7fda2cdfd640 1 -- 10.20.193.78:0/713529648 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2806fc20 con 0x7fda28123310 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.027+0000 7fda2cdfd640 1 -- 10.20.193.78:0/713529648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda28123310 msgr2=0x7fda28123790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.027+0000 7fda2cdfd640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda28123310 0x7fda28123790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fda04007460 tx=0x7fda1c011e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 1 -- 10.20.193.78:0/713529648 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c008040 con 0x7fda28123310 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: dump: 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: _finish_auth 0 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 1 -- 10.20.193.78:0/713529648 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1c00fd40 con 0x7fda28123310 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 1 -- 10.20.193.78:0/713529648 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c01b630 con 0x7fda28123310 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 10 monclient: dump: 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda267fc640 4 set_mon_vals no callback set 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 5 monclient: authenticate success, global_id 234170 2026-02-17T06:05:36.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 10 monclient: _renew_subs 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda28125e40 con 0x7fda28123310 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 10 monclient: _renew_subs 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda28126fc0 con 0x7fda28123310 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 1 -- 10.20.193.78:0/713529648 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fda1c00e050 con 0x7fda28123310 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2806cfa0 con 0x7fda28123310 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9f803de90 0x7fd9f8040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.028+0000 7fda24ff9640 1 -- 10.20.193.78:0/713529648 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(676..676 src has 525..676) ==== 4811+0+0 (secure 0 0 0) 0x7fda1c052f30 con 0x7fda28123310 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.029+0000 7fda27fff640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9f803de90 0x7fd9f8040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.029+0000 7fda27fff640 10 monclient: get_auth_request con 0x7fd9f803de90 auth_method 0 2026-02-17T06:05:36.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.031+0000 7fda24ff9640 1 -- 10.20.193.78:0/713529648 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fda2806fc20 con 0x7fda28123310 2026-02-17T06:05:36.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.031+0000 7fda24ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:36.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.031+0000 7fda24ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:36.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.031+0000 7fda27fff640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9f803de90 0x7fd9f8040350 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e470 tx=0x7fda14034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.072 INFO:tasks.rados.rados.0.trial115.stdout:3147: expect (ObjNum 935 snap 274 seq_num 935) 2026-02-17T06:05:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.151+0000 7fda2e887640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:05:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.151+0000 7fda2e887640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.151+0000 7fda2e887640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.151+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fda2809d760 con 0x7fda28123310 2026-02-17T06:05:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.151+0000 7fda24ff9640 1 -- 10.20.193.78:0/713529648 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fda1c051910 con 0x7fda28123310 2026-02-17T06:05:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.151+0000 7fda24ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:36.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.151+0000 7fda24ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:36.154 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":224,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:05:36.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9f803de90 msgr2=0x7fd9f8040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9f803de90 0x7fd9f8040350 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e470 tx=0x7fda14034000 comp rx=0 tx=0).stop 2026-02-17T06:05:36.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 10 monclient: shutdown 2026-02-17T06:05:36.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:36.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda28123310 msgr2=0x7fda28123790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda28123310 0x7fda28123790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fda04007460 tx=0x7fda1c011e30 comp rx=0 tx=0).stop 2026-02-17T06:05:36.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 shutdown_connections 2026-02-17T06:05:36.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fda28123310 0x7fda28123790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd9f803de90 0x7fd9f8040350 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fda281cd600 0x7fda281cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 --2- 10.20.193.78:0/713529648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fda0400d9b0 0x7fda281cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 >> 10.20.193.78:0/713529648 conn(0x7fda280a6620 msgr2=0x7fda28070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:36.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 shutdown_connections 2026-02-17T06:05:36.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.152+0000 7fda2e887640 1 -- 10.20.193.78:0/713529648 wait complete. 2026-02-17T06:05:36.161 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:05:36.162 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:05:36.162 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:05:36.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.231+0000 7f7f4878f640 10 monclient: get_monmap_and_config 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.231+0000 7f7f4878f640 10 monclient: build_initial_monmap 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.231+0000 7f7f4878f640 10 monclient: monmap: 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:36.232345+0000 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:36.232345+0000 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:36.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.231+0000 7f7f4878f640 1 Processor -- start 2026-02-17T06:05:36.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.231+0000 7f7f4878f640 1 -- start start 2026-02-17T06:05:36.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.231+0000 7f7f4878f640 10 monclient: init 2026-02-17T06:05:36.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.231+0000 7f7f4878f640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:36.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.231+0000 7f7f4878f640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2026-02-17T06:05:36.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ea90 0x7f7f400a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 10 monclient(hunting): picked mon.noname-b con 0x7f7f4009ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7f4009ee70 0x7f7f4009f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 10 monclient(hunting): picked mon.noname-c con 0x7f7f4009ee70 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7f4009d980 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f400a0a20 con 0x7f7f4009d980 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7f400a0d00 con 0x7f7f4009ee70 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7f400a0460 con 0x7f7f4009ea90 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f4878f640 10 monclient(hunting): authenticate will time out at 1781.786054s 2026-02-17T06:05:36.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46d05640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f7f4009d980 legacy=0x7f7f4009ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ea90 0x7f7f400a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ea90 0x7f7f400a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36106/0 (socket says 10.20.193.78:36106) 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 1 -- 10.20.193.78:0/3594048897 learned_addr learned my addr 10.20.193.78:0/3594048897 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient(hunting): get_auth_request con 0x7f7f4009ea90 auth_method 0 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f45d03640 1 --2- 10.20.193.78:0/3594048897 >> v2:10.20.193.115:3300/0 conn(0x7f7f4009ee70 0x7f7f4009f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f45d03640 10 monclient(hunting): get_auth_request con 0x7f7f4009ee70 auth_method 0 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f45d03640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f45d03640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f45d03640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient(hunting): handle_auth_done global_id 234173 payload 274 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 1 -- 10.20.193.78:0/3594048897 >> v2:10.20.193.115:3300/0 conn(0x7f7f4009ee70 msgr2=0x7f7f4009f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 1 --2- 10.20.193.78:0/3594048897 >> v2:10.20.193.115:3300/0 conn(0x7f7f4009ee70 0x7f7f4009f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 1 -- 10.20.193.78:0/3594048897 >> v1:10.20.193.78:6789/0 conn(0x7f7f4009d980 legacy=0x7f7f4009ddd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient: _finish_hunting 0 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 1 monclient: found mon.noname-b 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.232+0000 7f7f46504640 1 -- 10.20.193.78:0/3594048897 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f400a0740 con 0x7f7f4009ea90 2026-02-17T06:05:36.248 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.234+0000 7f7f46504640 1 -- 10.20.193.78:0/3594048897 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ea90 msgr2=0x7f7f400a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.234+0000 7f7f46504640 1 --2- 10.20.193.78:0/3594048897 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ea90 0x7f7f400a13d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009970 tx=0x7f7f3002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1431ee3d65e71c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 1 -- 10.20.193.78:0/3594048897 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3003c070 con 0x7f7f4009ea90 2026-02-17T06:05:36.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:05:36.249 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: dump: 2026-02-17T06:05:36.250 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.250 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.250 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.250 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.250 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.250 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.250 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.250 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: _finish_auth 0 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: _check_auth_tickets 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 1 -- 10.20.193.78:0/3594048897 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3002fcd0 con 0x7f7f4009ea90 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 5 monclient: authenticate success, global_id 234173 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:36.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 1 -- 10.20.193.78:0/3594048897 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30037660 con 0x7f7f4009ea90 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 4 set_mon_vals no callback set 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f45502640 10 monclient: dump: 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.252 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 10 monclient: shutdown 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 1 -- 10.20.193.78:0/3594048897 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ea90 msgr2=0x7f7f400a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 1 --2- 10.20.193.78:0/3594048897 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ea90 0x7f7f400a13d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009970 tx=0x7f7f3002ee90 comp rx=0 tx=0).stop 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 1 -- 10.20.193.78:0/3594048897 shutdown_connections 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 1 --2- 10.20.193.78:0/3594048897 >> v2:10.20.193.115:3300/0 conn(0x7f7f4009ee70 0x7f7f4009f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 1 --2- 10.20.193.78:0/3594048897 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ea90 0x7f7f400a13d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 1 -- 10.20.193.78:0/3594048897 >> 10.20.193.78:0/3594048897 conn(0x7f7f400a7520 msgr2=0x7f7f400a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 1 -- 10.20.193.78:0/3594048897 shutdown_connections 2026-02-17T06:05:36.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.235+0000 7f7f4878f640 1 -- 10.20.193.78:0/3594048897 wait complete. 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient: build_initial_monmap 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient: monmap: 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:36.254 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 1 Processor -- start 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 1 -- start start 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient: init 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f4009d980 0x7f7f4012fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient(hunting): picked mon.noname-b con 0x7f7f4009d980 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f4009ea90 0x7f7f40130020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient(hunting): picked mon.noname-a con 0x7f7f4009ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ee70 0x7f7f40130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient(hunting): picked mon.noname-c con 0x7f7f4009ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f400a66a0 con 0x7f7f4009ea90 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4009f7d0 con 0x7f7f4009d980 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7f400a6820 con 0x7f7f4009ee70 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f4878f640 10 monclient(hunting): authenticate will time out at 1781.790566s 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f45d03640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f4009ea90 msgr2=0x7f7f40130020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f45d03640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f4009ea90 0x7f7f40130020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ee70 0x7f7f40130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ee70 0x7f7f40130560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36108/0 (socket says 10.20.193.78:36108) 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 -- 10.20.193.78:0/1674967463 learned_addr learned my addr 10.20.193.78:0/1674967463 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:36.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient(hunting): get_auth_request con 0x7f7f4009ee70 auth_method 0 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46504640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f4009d980 0x7f7f4012fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46504640 10 monclient(hunting): get_auth_request con 0x7f7f4009d980 auth_method 0 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46504640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46504640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46504640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient(hunting): handle_auth_done global_id 234176 payload 824 2026-02-17T06:05:36.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f4009d980 msgr2=0x7f7f4012fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f4009d980 0x7f7f4012fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f4009ea90 msgr2=0x7f7f40130020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7f4009ea90 0x7f7f40130020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient: _finish_hunting 0 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 monclient: found mon.noname-c 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46d05640 1 -- 10.20.193.78:0/1674967463 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f40073d60 con 0x7f7f4009ee70 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.236+0000 7f7f46504640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7f4009d980 0x7f7f4012fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.238+0000 7f7f46d05640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ee70 msgr2=0x7f7f40130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.238+0000 7f7f46d05640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ee70 0x7f7f40130560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00be60 tx=0x7f7f3c00be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c006e60 con 0x7f7f4009ee70 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: dump: 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.259 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: _finish_auth 0 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c010030 con 0x7f7f4009ee70 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 5 monclient: authenticate success, global_id 234176 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 10 monclient: _renew_subs 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f400a1c10 con 0x7f7f4009ee70 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c016870 con 0x7f7f4009ee70 2026-02-17T06:05:36.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 10 monclient: dump: 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f44d01640 4 set_mon_vals no callback set 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 10 monclient: _renew_subs 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f400739b0 con 0x7f7f4009ee70 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cf360 0x7f7f401cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7f40135520 con 0x7f7f401cf360 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cf360 0x7f7f401cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 10 monclient: get_auth_request con 0x7f7f401cf360 auth_method 0 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7f3c008070 con 0x7f7f4009ee70 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 10 monclient(hunting): handle_auth_done global_id 234179 payload 824 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.239+0000 7f7f46504640 1 -- 10.20.193.78:0/1674967463 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f400a13b0 con 0x7f7f401cf360 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.240+0000 7f7f26ffd640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7f1003deb0 0x7f7f10040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.240+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(676..676 src has 525..676) ==== 4811+0+0 (secure 0 0 0) 0x7f7f3c052ad0 con 0x7f7f4009ee70 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.240+0000 7f7f45d03640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7f1003deb0 0x7f7f10040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.240+0000 7f7f45d03640 10 monclient: get_auth_request con 0x7f7f1003deb0 auth_method 0 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f46504640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cf360 msgr2=0x7f7f401cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f46504640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cf360 0x7f7f401cf740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f400a0160 tx=0x7f7f30004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3003c040 con 0x7f7f401cf360 2026-02-17T06:05:36.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 10 monclient: dump: 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7f30041660 con 0x7f7f401cf360 2026-02-17T06:05:36.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cf360 msgr2=0x7f7f401cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cf360 0x7f7f401cf740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f400a0160 tx=0x7f7f30004830 comp rx=0 tx=0).stop 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cf360 msgr2=0x7f7f401cf740 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.242+0000 7f7f26ffd640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cf360 0x7f7f401cf740 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f400a0160 tx=0x7f7f30004830 comp rx=0 tx=0).stop 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.243+0000 7f7f45d03640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7f1003deb0 0x7f7f10040370 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7f401326c0 tx=0x7f7f34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f4878f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f4878f640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cfc80 0x7f7f401da640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f4878f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7f400744d0 con 0x7f7f401cfc80 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f46504640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cfc80 0x7f7f401da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f46504640 10 monclient: get_auth_request con 0x7f7f401cfc80 auth_method 0 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f46504640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f46504640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f46504640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.245+0000 7f7f46504640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.246+0000 7f7f46504640 10 monclient(hunting): handle_auth_done global_id 234182 payload 824 2026-02-17T06:05:36.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.246+0000 7f7f46504640 1 -- 10.20.193.78:0/1674967463 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7f401d0090 con 0x7f7f401cfc80 2026-02-17T06:05:36.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f46504640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cfc80 msgr2=0x7f7f401da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f46504640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cfc80 0x7f7f401da640 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7f30002410 tx=0x7f7f30008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3003c040 con 0x7f7f401cfc80 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 10 monclient: dump: 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.268 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7f7f3004ab80 con 0x7f7f401cfc80 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cfc80 msgr2=0x7f7f401da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cfc80 0x7f7f401da640 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7f30002410 tx=0x7f7f30008030 comp rx=0 tx=0).stop 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cfc80 msgr2=0x7f7f401da640 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f26ffd640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f401cfc80 0x7f7f401da640 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7f30002410 tx=0x7f7f30008030 comp rx=0 tx=0).stop 2026-02-17T06:05:36.269 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 224, 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 44931, 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:36.270 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:36.271 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:36.272 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:36.273 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.274 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.275 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:36.276 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:36.277 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:05:36.278 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:05:36.279 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.279 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.279 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.279 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:05:36.279 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:05:36.279 INFO:tasks.rados.rados.0.trial115.stdout:3148: finishing write tid 1 to trial11545420-50 2026-02-17T06:05:36.279 INFO:tasks.rados.rados.0.trial115.stdout:3148: finishing write tid 2 to trial11545420-50 2026-02-17T06:05:36.279 INFO:tasks.rados.rados.0.trial115.stdout:3148: finishing write tid 3 to trial11545420-50 2026-02-17T06:05:36.279 INFO:tasks.rados.rados.0.trial115.stdout:3148: finishing write tid 4 to trial11545420-50 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3148: oid 50 updating version 0 to 1359 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3148: oid 50 updating version 1359 to 1360 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3148: oid 50 version 1360 is already newer than 1358 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1360 (ObjNum 1099 snap 329 seq_num 1099) dirty exists 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3148: left oid 50 (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3150: finishing copy_from racing read to trial11545420-16 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3144: done (4 left) 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3146: done (3 left) 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3147: done (2 left) 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3148: done (1 left) 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3151: snap_remove snap 318 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3151: done (1 left) 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3152: read oid 19 snap -1 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3152: expect (ObjNum 1063 snap 319 seq_num 1063) 2026-02-17T06:05:36.280 INFO:tasks.rados.rados.0.trial115.stdout:3153: snap_remove snap 305 2026-02-17T06:05:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7f1003deb0 msgr2=0x7f7f10040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7f1003deb0 0x7f7f10040370 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7f401326c0 tx=0x7f7f34008040 comp rx=0 tx=0).stop 2026-02-17T06:05:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 10 monclient: shutdown 2026-02-17T06:05:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ee70 msgr2=0x7f7f40130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 1 --2- 10.20.193.78:0/1674967463 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7f4009ee70 0x7f7f40130560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00be60 tx=0x7f7f3c00be90 comp rx=0 tx=0).stop 2026-02-17T06:05:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f46d05640 1 -- 10.20.193.78:0/1674967463 reap_dead start 2026-02-17T06:05:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f46d05640 1 -- 10.20.193.78:0/1674967463 reap_dead start 2026-02-17T06:05:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 shutdown_connections 2026-02-17T06:05:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 >> 10.20.193.78:0/1674967463 conn(0x7f7f400a7520 msgr2=0x7f7f40138180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 shutdown_connections 2026-02-17T06:05:36.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.248+0000 7f7f4878f640 1 -- 10.20.193.78:0/1674967463 wait complete. 2026-02-17T06:05:36.282 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:05:36.282 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T06:05:36.282 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.347+0000 7f5f2f46c640 10 monclient: get_monmap_and_config 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.347+0000 7f5f2f46c640 10 monclient: build_initial_monmap 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.347+0000 7f5f2f46c640 10 monclient: monmap: 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:36.348852+0000 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:36.348852+0000 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:36.348 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:36.349 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:36.349 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:36.349 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 1 Processor -- start 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 1 -- start start 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient: init 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5f2806c890 0x7f5f2806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient(hunting): picked mon.noname-c con 0x7f5f2806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5f2809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5f2806d1b0 0x7f5f28135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient(hunting): picked mon.noname-a con 0x7f5f2806d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f2806daf0 con 0x7f5f2809c4a0 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5f280a0880 con 0x7f5f2806d1b0 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5f2809cc80 con 0x7f5f2806c890 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2f46c640 10 monclient(hunting): authenticate will time out at 1781.902599s 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2d9e2640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f5f2806d1b0 msgr2=0x7f5f28135500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2d9e2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5f2806d1b0 0x7f5f28135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2c9e0640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f5f2809c4a0 0x7f5f28134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:51400/0 (socket says 10.20.193.78:51400) 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2c9e0640 1 -- 10.20.193.78:0/3626459872 learned_addr learned my addr 10.20.193.78:0/3626459872 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2d1e1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5f2806c890 0x7f5f2806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2d1e1640 10 monclient(hunting): get_auth_request con 0x7f5f2806c890 auth_method 0 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2d1e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2d1e1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.348+0000 7f5f2d1e1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_done global_id 214276 payload 274 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 1 -- 10.20.193.78:0/3626459872 >> v2:10.20.193.78:3300/0 conn(0x7f5f2806d1b0 msgr2=0x7f5f28135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 1 --2- 10.20.193.78:0/3626459872 >> v2:10.20.193.78:3300/0 conn(0x7f5f2806d1b0 0x7f5f28135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 1 -- 10.20.193.78:0/3626459872 >> v1:10.20.193.115:6789/0 conn(0x7f5f2809c4a0 legacy=0x7f5f28134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 10 monclient: _finish_hunting 0 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 1 monclient: found mon.noname-c 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:05:36.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.349+0000 7f5f2d1e1640 1 -- 10.20.193.78:0/3626459872 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2809cf60 con 0x7f5f2806c890 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.351+0000 7f5f2d1e1640 1 -- 10.20.193.78:0/3626459872 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2806c890 msgr2=0x7f5f2806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.351+0000 7f5f2d1e1640 1 --2- 10.20.193.78:0/3626459872 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2806c890 0x7f5f2806cc70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5f2806a480 tx=0x7f5f1802ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e391d783e9745213 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.351+0000 7f5f17fff640 1 -- 10.20.193.78:0/3626459872 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1803c070 con 0x7f5f2806c890 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.351+0000 7f5f17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.351+0000 7f5f17fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.351+0000 7f5f17fff640 10 monclient: dump: 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.358 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 10 monclient: _finish_auth 0 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 10 monclient: _check_auth_tickets 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 1 -- 10.20.193.78:0/3626459872 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1802fcd0 con 0x7f5f2806c890 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 1 -- 10.20.193.78:0/3626459872 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18037660 con 0x7f5f2806c890 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:36.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f17fff640 10 monclient: dump: 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 5 monclient: authenticate success, global_id 214276 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 4 set_mon_vals no callback set 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 10 monclient: shutdown 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 1 -- 10.20.193.78:0/3626459872 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2806c890 msgr2=0x7f5f2806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 1 --2- 10.20.193.78:0/3626459872 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2806c890 0x7f5f2806cc70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5f2806a480 tx=0x7f5f1802ed70 comp rx=0 tx=0).stop 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 1 -- 10.20.193.78:0/3626459872 shutdown_connections 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 1 --2- 10.20.193.78:0/3626459872 >> v2:10.20.193.78:3300/0 conn(0x7f5f2806d1b0 0x7f5f28135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 1 --2- 10.20.193.78:0/3626459872 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2806c890 0x7f5f2806cc70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 1 -- 10.20.193.78:0/3626459872 >> 10.20.193.78:0/3626459872 conn(0x7f5f280a6620 msgr2=0x7f5f280a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 1 -- 10.20.193.78:0/3626459872 shutdown_connections 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 1 -- 10.20.193.78:0/3626459872 wait complete. 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 10 monclient: build_initial_monmap 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.352+0000 7f5f2f46c640 10 monclient: monmap: 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:36.361 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 1 Processor -- start 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 1 -- start start 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient: init 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f2806c890 0x7f5f281cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient(hunting): picked mon.noname-a con 0x7f5f2806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f2806d1b0 0x7f5f28135ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient(hunting): picked mon.noname-c con 0x7f5f2806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809c4a0 0x7f5f28136530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient(hunting): picked mon.noname-b con 0x7f5f2809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280a01a0 con 0x7f5f2806c890 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280a0020 con 0x7f5f2809c4a0 2026-02-17T06:05:36.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2809be20 con 0x7f5f2806d1b0 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2f46c640 10 monclient(hunting): authenticate will time out at 1781.907414s 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2d1e1640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f2806c890 msgr2=0x7f5f281cf2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2d1e1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f2806c890 0x7f5f281cf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f2806d1b0 0x7f5f28135ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f2806d1b0 0x7f5f28135ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36140/0 (socket says 10.20.193.78:36140) 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 -- 10.20.193.78:0/4219969404 learned_addr learned my addr 10.20.193.78:0/4219969404 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient(hunting): get_auth_request con 0x7f5f2806d1b0 auth_method 0 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2d9e2640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809c4a0 0x7f5f28136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2d9e2640 10 monclient(hunting): get_auth_request con 0x7f5f2809c4a0 auth_method 0 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2d9e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2d9e2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2d9e2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient(hunting): handle_auth_done global_id 234185 payload 824 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809c4a0 msgr2=0x7f5f28136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809c4a0 0x7f5f28136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f2806c890 msgr2=0x7f5f281cf2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5f2806c890 0x7f5f281cf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient: _finish_hunting 0 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 monclient: found mon.noname-c 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.353+0000 7f5f2c9e0640 1 -- 10.20.193.78:0/4219969404 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f28127e70 con 0x7f5f2806d1b0 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2c9e0640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f2806d1b0 msgr2=0x7f5f28135ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2c9e0640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f2806d1b0 0x7f5f28135ff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00c750 tx=0x7f5f1c00cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c005de0 con 0x7f5f2806d1b0 2026-02-17T06:05:36.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: dump: 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: _finish_auth 0 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1c007070 con 0x7f5f2806d1b0 2026-02-17T06:05:36.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2f46c640 5 monclient: authenticate success, global_id 234185 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2f46c640 10 monclient: _renew_subs 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2f46c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f28129450 con 0x7f5f2806d1b0 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c007440 con 0x7f5f2806d1b0 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f177fe640 4 set_mon_vals no callback set 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f15ffb640 10 monclient: dump: 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.366 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2f46c640 10 monclient: _renew_subs 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2f46c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.355+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281cf800 con 0x7f5f2806d1b0 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2f46c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2f46c640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e310 0x7f5f2811ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2f46c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28128280 con 0x7f5f2809e310 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2d1e1640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e310 0x7f5f2811ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5f1c007620 con 0x7f5f2806d1b0 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2d1e1640 10 monclient: get_auth_request con 0x7f5f2809e310 auth_method 0 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2d1e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2d1e1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2d1e1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.367 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.356+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.357+0000 7f5f15ffb640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ef803de60 0x7f5ef8040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.357+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(677..677 src has 525..677) ==== 4839+0+0 (secure 0 0 0) 0x7f5f1c052620 con 0x7f5f2806d1b0 2026-02-17T06:05:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.357+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_done global_id 214282 payload 824 2026-02-17T06:05:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.357+0000 7f5f2d1e1640 1 -- 10.20.193.78:0/4219969404 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f28068410 con 0x7f5f2809e310 2026-02-17T06:05:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.357+0000 7f5f2d9e2640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ef803de60 0x7f5ef8040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.368 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.357+0000 7f5f2d9e2640 10 monclient: get_auth_request con 0x7f5ef803de60 auth_method 0 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f2d1e1640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e310 msgr2=0x7f5f2811ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f2d1e1640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e310 0x7f5f2811ec80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f18039000 tx=0x7f5f180047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18047070 con 0x7f5f2809e310 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 10 monclient: dump: 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.370 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f5f1804c450 con 0x7f5f2809e310 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e310 msgr2=0x7f5f2811ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e310 0x7f5f2811ec80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f18039000 tx=0x7f5f180047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e310 msgr2=0x7f5f2811ec80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f15ffb640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e310 0x7f5f2811ec80 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f18039000 tx=0x7f5f180047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.359+0000 7f5f2d9e2640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ef803de60 0x7f5ef8040320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5f24003db0 tx=0x7f5f240094f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.371 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.362+0000 7f5f2f46c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.362+0000 7f5f2f46c640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e6f0 0x7f5f281da610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.362+0000 7f5f2f46c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.362+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28073190 con 0x7f5f2809e6f0 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e6f0 0x7f5f281da610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 10 monclient: get_auth_request con 0x7f5f2809e6f0 auth_method 0 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 10 monclient(hunting): handle_auth_done global_id 214285 payload 824 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.363+0000 7f5f2d1e1640 1 -- 10.20.193.78:0/4219969404 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5f28136ea0 con 0x7f5f2809e6f0 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.365+0000 7f5f2d1e1640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e6f0 msgr2=0x7f5f281da610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.365+0000 7f5f2d1e1640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e6f0 0x7f5f281da610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f18002410 tx=0x7f5f18009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.365+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1803c040 con 0x7f5f2809e6f0 2026-02-17T06:05:36.372 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.365+0000 7f5f15ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.365+0000 7f5f15ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.365+0000 7f5f15ffb640 10 monclient: dump: 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7f5f1804c670 con 0x7f5f2809e6f0 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f15ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f15ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e6f0 msgr2=0x7f5f281da610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f15ffb640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e6f0 0x7f5f281da610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f18002410 tx=0x7f5f18009290 comp rx=0 tx=0).stop 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f15ffb640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e6f0 msgr2=0x7f5f281da610 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f15ffb640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5f2809e6f0 0x7f5f281da610 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f18002410 tx=0x7f5f18009290 comp rx=0 tx=0).stop 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 224, 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 139213, 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:05:36.374 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:36.375 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:05:36.376 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:05:36.377 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:05:36.378 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:36.379 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:05:36.380 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:05:36.381 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.382 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:05:36.383 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:05:36.384 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:05:36.384 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:05:36.384 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:05:36.384 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:05:36.384 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ef803de60 msgr2=0x7f5ef8040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5ef803de60 0x7f5ef8040320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5f24003db0 tx=0x7f5f240094f0 comp rx=0 tx=0).stop 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 10 monclient: shutdown 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f2806d1b0 msgr2=0x7f5f28135ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 1 --2- 10.20.193.78:0/4219969404 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5f2806d1b0 0x7f5f28135ff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00c750 tx=0x7f5f1c00cc20 comp rx=0 tx=0).stop 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2d9e2640 1 -- 10.20.193.78:0/4219969404 reap_dead start 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2d9e2640 1 -- 10.20.193.78:0/4219969404 reap_dead start 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 shutdown_connections 2026-02-17T06:05:36.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 >> 10.20.193.78:0/4219969404 conn(0x7f5f280a6620 msgr2=0x7f5f280714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:36.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 shutdown_connections 2026-02-17T06:05:36.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:36.366+0000 7f5f2f46c640 1 -- 10.20.193.78:0/4219969404 wait complete. 2026-02-17T06:05:36.386 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:05:36.386 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:05:37.265 INFO:tasks.rados.rados.0.trial115.stdout:3153: done (2 left) 2026-02-17T06:05:37.266 INFO:tasks.rados.rados.0.trial115.stdout:3154: read oid 31 snap -1 2026-02-17T06:05:37.266 INFO:tasks.rados.rados.0.trial115.stdout:3154: expect (ObjNum 1089 snap 328 seq_num 1089) 2026-02-17T06:05:37.266 INFO:tasks.rados.rados.0.trial115.stdout:3155: write oid 44 current snap is 329 2026-02-17T06:05:37.266 INFO:tasks.rados.rados.0.trial115.stdout:3155: seq_num 1100 ranges {487700=728732,1786075=665712} 2026-02-17T06:05:37.270 INFO:tasks.rados.rados.0.trial115.stdout:3155: writing trial11545420-44 from 487700 to 1216432 tid 1 2026-02-17T06:05:37.275 INFO:tasks.rados.rados.0.trial115.stdout:3155: writing trial11545420-44 from 1786075 to 2451787 tid 2 2026-02-17T06:05:37.275 INFO:tasks.rados.rados.0.trial115.stdout:3156: write oid 46 current snap is 329 2026-02-17T06:05:37.275 INFO:tasks.rados.rados.0.trial115.stdout:3156: seq_num 1101 ranges {504507=403981,1328204=409370,2252362=252145} 2026-02-17T06:05:37.279 INFO:tasks.rados.rados.0.trial115.stdout:3156: writing trial11545420-46 from 504507 to 908488 tid 1 2026-02-17T06:05:37.282 INFO:tasks.rados.rados.0.trial115.stdout:3156: writing trial11545420-46 from 1328204 to 1737574 tid 2 2026-02-17T06:05:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3156: writing trial11545420-46 from 2252362 to 2504507 tid 3 2026-02-17T06:05:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3157: read oid 25 snap -1 2026-02-17T06:05:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3157: expect (ObjNum 1069 snap 322 seq_num 1069) 2026-02-17T06:05:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3158: copy_from oid 14 from oid 6 current snap is 329 2026-02-17T06:05:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3150: finishing copy_from to trial11545420-16 2026-02-17T06:05:37.285 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1453 (ObjNum 1027 snap 297 seq_num 1027) dirty exists 2026-02-17T06:05:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3150: done (6 left) 2026-02-17T06:05:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3159: snap_remove snap 323 2026-02-17T06:05:38.269 INFO:tasks.rados.rados.0.trial115.stdout:3159: done (6 left) 2026-02-17T06:05:38.270 INFO:tasks.rados.rados.0.trial115.stdout:3160: snap_create 2026-02-17T06:05:38.270 INFO:tasks.rados.rados.0.trial115.stdout:3152: expect (ObjNum 1063 snap 319 seq_num 1063) 2026-02-17T06:05:38.358 INFO:tasks.rados.rados.0.trial115.stdout:3154: expect (ObjNum 1089 snap 328 seq_num 1089) 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3155: finishing write tid 1 to trial11545420-44 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3155: finishing write tid 2 to trial11545420-44 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3155: finishing write tid 3 to trial11545420-44 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3155: finishing write tid 4 to trial11545420-44 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3155: oid 44 updating version 0 to 1454 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3155: oid 44 version 1454 is already newer than 1453 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3155: oid 44 updating version 1454 to 1455 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1455 (ObjNum 1100 snap 329 seq_num 1100) dirty exists 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3155: left oid 44 (ObjNum 1100 snap 329 seq_num 1100) 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: finishing write tid 1 to trial11545420-46 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: finishing write tid 2 to trial11545420-46 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: finishing write tid 3 to trial11545420-46 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: finishing write tid 4 to trial11545420-46 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: finishing write tid 5 to trial11545420-46 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: oid 46 updating version 0 to 1592 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: oid 46 version 1592 is already newer than 1590 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: oid 46 version 1592 is already newer than 1591 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: oid 46 version 1592 is already newer than 1589 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1592 (ObjNum 1101 snap 329 seq_num 1101) dirty exists 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3156: left oid 46 (ObjNum 1101 snap 329 seq_num 1101) 2026-02-17T06:05:38.429 INFO:tasks.rados.rados.0.trial115.stdout:3157: expect (ObjNum 1069 snap 322 seq_num 1069) 2026-02-17T06:05:38.601 INFO:tasks.rados.rados.0.trial115.stdout:3158: finishing copy_from racing read to trial11545420-14 2026-02-17T06:05:38.601 INFO:tasks.rados.rados.0.trial115.stdout:3158: finishing copy_from to trial11545420-14 2026-02-17T06:05:38.601 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1482 (ObjNum 948 snap 276 seq_num 948) dirty exists 2026-02-17T06:05:39.274 INFO:tasks.rados.rados.0.trial115.stdout:3152: done (6 left) 2026-02-17T06:05:39.274 INFO:tasks.rados.rados.0.trial115.stdout:3154: done (5 left) 2026-02-17T06:05:39.274 INFO:tasks.rados.rados.0.trial115.stdout:3155: done (4 left) 2026-02-17T06:05:39.274 INFO:tasks.rados.rados.0.trial115.stdout:3156: done (3 left) 2026-02-17T06:05:39.274 INFO:tasks.rados.rados.0.trial115.stdout:3157: done (2 left) 2026-02-17T06:05:39.274 INFO:tasks.rados.rados.0.trial115.stdout:3158: done (1 left) 2026-02-17T06:05:39.275 INFO:tasks.rados.rados.0.trial115.stdout:3160: done (0 left) 2026-02-17T06:05:39.275 INFO:tasks.rados.rados.0.trial115.stdout:3161: read oid 26 snap -1 2026-02-17T06:05:39.275 INFO:tasks.rados.rados.0.trial115.stdout:3161: expect (ObjNum 1095 snap 329 seq_num 1095) 2026-02-17T06:05:39.275 INFO:tasks.rados.rados.0.trial115.stdout:3162: delete oid 20 current snap is 330 2026-02-17T06:05:39.276 INFO:tasks.rados.rados.0.trial115.stdout:3162: done (1 left) 2026-02-17T06:05:39.276 INFO:tasks.rados.rados.0.trial115.stdout:3163: snap_remove snap 329 2026-02-17T06:05:40.274 INFO:tasks.rados.rados.0.trial115.stdout:3163: done (1 left) 2026-02-17T06:05:40.274 INFO:tasks.rados.rados.0.trial115.stdout:3164: read oid 11 snap -1 2026-02-17T06:05:40.274 INFO:tasks.rados.rados.0.trial115.stdout:3164: expect (ObjNum 1027 snap 297 seq_num 1027) 2026-02-17T06:05:40.274 INFO:tasks.rados.rados.0.trial115.stdout:3165: snap_remove snap 299 2026-02-17T06:05:41.267 INFO:tasks.rados.rados.0.trial115.stdout:3165: done (2 left) 2026-02-17T06:05:41.267 INFO:tasks.rados.rados.0.trial115.stdout:3166: read oid 45 snap -1 2026-02-17T06:05:41.267 INFO:tasks.rados.rados.0.trial115.stdout:3166: expect (ObjNum 1067 snap 322 seq_num 1067) 2026-02-17T06:05:41.267 INFO:tasks.rados.rados.0.trial115.stdout:3167: snap_create 2026-02-17T06:05:41.267 INFO:tasks.rados.rados.0.trial115.stdout:3161: expect (ObjNum 1095 snap 329 seq_num 1095) 2026-02-17T06:05:41.321 INFO:tasks.rados.rados.0.trial115.stdout:3164: expect (ObjNum 1027 snap 297 seq_num 1027) 2026-02-17T06:05:41.472 INFO:tasks.rados.rados.0.trial115.stdout:3166: expect (ObjNum 1067 snap 322 seq_num 1067) 2026-02-17T06:05:42.267 INFO:tasks.rados.rados.0.trial115.stdout:3161: done (3 left) 2026-02-17T06:05:42.268 INFO:tasks.rados.rados.0.trial115.stdout:3164: done (2 left) 2026-02-17T06:05:42.268 INFO:tasks.rados.rados.0.trial115.stdout:3166: done (1 left) 2026-02-17T06:05:42.268 INFO:tasks.rados.rados.0.trial115.stdout:3167: done (0 left) 2026-02-17T06:05:42.268 INFO:tasks.rados.rados.0.trial115.stdout:3168: delete oid 17 current snap is 331 2026-02-17T06:05:42.270 INFO:tasks.rados.rados.0.trial115.stdout:3168: done (0 left) 2026-02-17T06:05:42.270 INFO:tasks.rados.rados.0.trial115.stdout:3169: rollback oid 29 current snap is 331 2026-02-17T06:05:42.270 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 324 2026-02-17T06:05:42.270 INFO:tasks.rados.rados.0.trial115.stdout:3170: snap_create 2026-02-17T06:05:42.273 INFO:tasks.rados.rados.0.trial115.stdout:3169: finishing rollback tid 0 to trial11545420-29 2026-02-17T06:05:42.274 INFO:tasks.rados.rados.0.trial115.stdout:3169: finishing rollback tid 1 to trial11545420-29 2026-02-17T06:05:42.274 INFO:tasks.rados.rados.0.trial115.stdout:3169: finishing rollback tid 2 to trial11545420-29 2026-02-17T06:05:42.274 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1468 (ObjNum 917 snap 265 seq_num 917) dirty exists 2026-02-17T06:05:43.270 INFO:tasks.rados.rados.0.trial115.stdout:3169: done (1 left) 2026-02-17T06:05:43.270 INFO:tasks.rados.rados.0.trial115.stdout:3170: done (0 left) 2026-02-17T06:05:43.270 INFO:tasks.rados.rados.0.trial115.stdout:3171: read oid 21 snap -1 2026-02-17T06:05:43.270 INFO:tasks.rados.rados.0.trial115.stdout:3171: expect deleted 2026-02-17T06:05:43.270 INFO:tasks.rados.rados.0.trial115.stdout:3172: read oid 45 snap -1 2026-02-17T06:05:43.270 INFO:tasks.rados.rados.0.trial115.stdout:3172: expect (ObjNum 1067 snap 322 seq_num 1067) 2026-02-17T06:05:43.270 INFO:tasks.rados.rados.0.trial115.stdout:3173: snap_remove snap 330 2026-02-17T06:05:44.708 INFO:tasks.rados.rados.0.trial115.stdout:3173: done (2 left) 2026-02-17T06:05:44.708 INFO:tasks.rados.rados.0.trial115.stdout:3174: write (excl) oid 5 current snap is 332 2026-02-17T06:05:44.708 INFO:tasks.rados.rados.0.trial115.stdout:3174: seq_num 1104 ranges {554928=746081,2020003=534925} 2026-02-17T06:05:44.713 INFO:tasks.rados.rados.0.trial115.stdout:3174: writing trial11545420-5 from 554928 to 1301009 tid 1 2026-02-17T06:05:44.718 INFO:tasks.rados.rados.0.trial115.stdout:3174: writing trial11545420-5 from 2020003 to 2554928 tid 2 2026-02-17T06:05:44.718 INFO:tasks.rados.rados.0.trial115.stdout:3175: snap_remove snap 309 2026-02-17T06:05:45.710 INFO:tasks.rados.rados.0.trial115.stdout:3175: done (3 left) 2026-02-17T06:05:45.710 INFO:tasks.rados.rados.0.trial115.stdout:3176: copy_from oid 29 from oid 36 current snap is 332 2026-02-17T06:05:45.710 INFO:tasks.rados.rados.0.trial115.stdout:3172: expect (ObjNum 1067 snap 322 seq_num 1067) 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: finishing write tid 1 to trial11545420-5 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: finishing write tid 2 to trial11545420-5 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: finishing write tid 3 to trial11545420-5 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: finishing write tid 4 to trial11545420-5 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: oid 5 updating version 0 to 1457 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: oid 5 updating version 1457 to 1458 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: oid 5 version 1458 is already newer than 1456 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1458 (ObjNum 1104 snap 332 seq_num 1104) dirty exists 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: left oid 5 (ObjNum 1104 snap 332 seq_num 1104) 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3176: finishing copy_from racing read to trial11545420-29 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3171: done (3 left) 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3172: done (2 left) 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3174: done (1 left) 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3177: read oid 40 snap -1 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3177: expect (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3178: write (excl) oid 14 current snap is 332 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:3176: finishing copy_from to trial11545420-29 2026-02-17T06:05:45.884 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1452 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:05:45.885 INFO:tasks.rados.rados.0.trial115.stdout:3178: seq_num 1105 ranges {571735=421330,1562132=416708} 2026-02-17T06:05:45.888 INFO:tasks.rados.rados.0.trial115.stdout:3178: writing trial11545420-14 from 571735 to 993065 tid 1 2026-02-17T06:05:45.892 INFO:tasks.rados.rados.0.trial115.stdout:3178: writing trial11545420-14 from 1562132 to 1978840 tid 2 2026-02-17T06:05:45.892 INFO:tasks.rados.rados.0.trial115.stdout:3176: done (2 left) 2026-02-17T06:05:45.892 INFO:tasks.rados.rados.0.trial115.stdout:3179: delete oid 28 current snap is 332 2026-02-17T06:05:45.892 INFO:tasks.rados.rados.0.trial115.stdout:3177: expect (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: finishing write tid 1 to trial11545420-14 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: finishing write tid 2 to trial11545420-14 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: finishing write tid 3 to trial11545420-14 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: finishing write tid 4 to trial11545420-14 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: oid 14 updating version 0 to 1485 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: oid 14 version 1485 is already newer than 1484 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: oid 14 version 1485 is already newer than 1483 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1485 (ObjNum 1105 snap 332 seq_num 1105) dirty exists 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: left oid 14 (ObjNum 1105 snap 332 seq_num 1105) 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3177: done (2 left) 2026-02-17T06:05:46.028 INFO:tasks.rados.rados.0.trial115.stdout:3178: done (1 left) 2026-02-17T06:05:46.029 INFO:tasks.rados.rados.0.trial115.stdout:3179: done (0 left) 2026-02-17T06:05:46.029 INFO:tasks.rados.rados.0.trial115.stdout:3180: rollback oid 45 current snap is 332 2026-02-17T06:05:46.029 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 301 2026-02-17T06:05:46.029 INFO:tasks.rados.rados.0.trial115.stdout:3181: delete oid 36 current snap is 332 2026-02-17T06:05:46.030 INFO:tasks.rados.rados.0.trial115.stdout:3180: finishing rollback tid 0 to trial11545420-45 2026-02-17T06:05:46.030 INFO:tasks.rados.rados.0.trial115.stdout:3180: finishing rollback tid 1 to trial11545420-45 2026-02-17T06:05:46.030 INFO:tasks.rados.rados.0.trial115.stdout:3180: finishing rollback tid 2 to trial11545420-45 2026-02-17T06:05:46.030 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1641 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3180: done (1 left) 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3181: done (0 left) 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3182: read oid 24 snap -1 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3182: expect (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3183: read oid 30 snap 301 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3183: expect (ObjNum 1012 snap 294 seq_num 1012) 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3184: rollback oid 27 current snap is 332 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 331 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3185: read oid 46 snap -1 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3185: expect (ObjNum 1101 snap 329 seq_num 1101) 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3186: copy_from oid 49 from oid 2 current snap is 332 2026-02-17T06:05:46.036 INFO:tasks.rados.rados.0.trial115.stdout:3187: snap_remove snap 324 2026-02-17T06:05:46.268 INFO:tasks.rados.rados.0.trial115.stdout:3187: done (5 left) 2026-02-17T06:05:46.268 INFO:tasks.rados.rados.0.trial115.stdout:3188: read oid 35 snap -1 2026-02-17T06:05:46.268 INFO:tasks.rados.rados.0.trial115.stdout:3188: expect deleted 2026-02-17T06:05:46.268 INFO:tasks.rados.rados.0.trial115.stdout:3189: snap_remove snap 314 2026-02-17T06:05:47.268 INFO:tasks.rados.rados.0.trial115.stdout:3189: done (6 left) 2026-02-17T06:05:47.268 INFO:tasks.rados.rados.0.trial115.stdout:3190: rollback oid 10 current snap is 332 2026-02-17T06:05:47.268 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 10 to 287 2026-02-17T06:05:47.268 INFO:tasks.rados.rados.0.trial115.stdout:3182: expect (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:05:47.373 INFO:tasks.rados.rados.0.trial115.stdout:3185: expect (ObjNum 1101 snap 329 seq_num 1101) 2026-02-17T06:05:47.448 INFO:tasks.rados.rados.0.trial115.stdout:3183: expect (ObjNum 1012 snap 294 seq_num 1012) 2026-02-17T06:05:47.602 INFO:tasks.rados.rados.0.trial115.stdout:3184: finishing rollback tid 1 to trial11545420-27 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1347 (ObjNum 177 snap 0 seq_num 1333871248) dirty dne 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3186: finishing copy_from racing read to trial11545420-49 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3186: finishing copy_from to trial11545420-49 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1452 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3182: done (6 left) 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3183: done (5 left) 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3184: done (4 left) 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3185: done (3 left) 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3186: done (2 left) 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3188: done (1 left) 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3191: read oid 24 snap 326 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3191: expect (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3192: delete oid 3 current snap is 332 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3190: finishing rollback tid 0 to trial11545420-10 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3190: finishing rollback tid 1 to trial11545420-10 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:3190: finishing rollback tid 2 to trial11545420-10 2026-02-17T06:05:47.603 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1622 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:05:47.604 INFO:tasks.rados.rados.0.trial115.stdout:3190: done (2 left) 2026-02-17T06:05:47.604 INFO:tasks.rados.rados.0.trial115.stdout:3192: done (1 left) 2026-02-17T06:05:47.604 INFO:tasks.rados.rados.0.trial115.stdout:3193: snap_create 2026-02-17T06:05:47.605 INFO:tasks.rados.rados.0.trial115.stdout:3191: expect (ObjNum 923 snap 271 seq_num 923) 2026-02-17T06:05:48.272 INFO:tasks.rados.rados.0.trial115.stdout:3191: done (1 left) 2026-02-17T06:05:48.272 INFO:tasks.rados.rados.0.trial115.stdout:3193: done (0 left) 2026-02-17T06:05:48.272 INFO:tasks.rados.rados.0.trial115.stdout:3194: delete oid 2 current snap is 333 2026-02-17T06:05:48.274 INFO:tasks.rados.rados.0.trial115.stdout:3194: done (0 left) 2026-02-17T06:05:48.274 INFO:tasks.rados.rados.0.trial115.stdout:3195: write (excl) oid 24 current snap is 333 2026-02-17T06:05:48.274 INFO:tasks.rados.rados.0.trial115.stdout:3195: seq_num 1110 ranges {655770=513928,1854542=684057} 2026-02-17T06:05:48.278 INFO:tasks.rados.rados.0.trial115.stdout:3195: writing trial11545420-24 from 655770 to 1169698 tid 1 2026-02-17T06:05:48.285 INFO:tasks.rados.rados.0.trial115.stdout:3195: writing trial11545420-24 from 1854542 to 2538599 tid 2 2026-02-17T06:05:48.285 INFO:tasks.rados.rados.0.trial115.stdout:3196: rollback oid 35 current snap is 333 2026-02-17T06:05:48.285 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 35 to 301 2026-02-17T06:05:48.285 INFO:tasks.rados.rados.0.trial115.stdout:3195: finishing write tid 1 to trial11545420-24 2026-02-17T06:05:48.285 INFO:tasks.rados.rados.0.trial115.stdout:3197: write oid 42 current snap is 333 2026-02-17T06:05:48.285 INFO:tasks.rados.rados.0.trial115.stdout:3197: seq_num 1111 ranges {672577=589177,1913024=427715} 2026-02-17T06:05:48.289 INFO:tasks.rados.rados.0.trial115.stdout:3197: writing trial11545420-42 from 672577 to 1261754 tid 1 2026-02-17T06:05:48.294 INFO:tasks.rados.rados.0.trial115.stdout:3197: writing trial11545420-42 from 1913024 to 2340739 tid 2 2026-02-17T06:05:48.294 INFO:tasks.rados.rados.0.trial115.stdout:3198: read oid 40 snap -1 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3198: expect (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3199: delete oid 12 current snap is 333 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3195: finishing write tid 2 to trial11545420-24 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3195: finishing write tid 3 to trial11545420-24 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3196: finishing rollback tid 1 to trial11545420-35 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3196: finishing rollback tid 2 to trial11545420-35 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1677 (ObjNum 948 snap 276 seq_num 948) dirty exists 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3195: finishing write tid 4 to trial11545420-24 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3195: oid 24 updating version 0 to 1657 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3195: oid 24 version 1657 is already newer than 1655 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:3195: oid 24 version 1657 is already newer than 1656 2026-02-17T06:05:48.295 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1657 (ObjNum 1110 snap 333 seq_num 1110) dirty exists 2026-02-17T06:05:48.296 INFO:tasks.rados.rados.0.trial115.stdout:3195: left oid 24 (ObjNum 1110 snap 333 seq_num 1110) 2026-02-17T06:05:48.296 INFO:tasks.rados.rados.0.trial115.stdout:3197: finishing write tid 1 to trial11545420-42 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:3197: finishing write tid 2 to trial11545420-42 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:3197: finishing write tid 3 to trial11545420-42 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:3197: finishing write tid 4 to trial11545420-42 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:3197: oid 42 updating version 0 to 1659 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:3197: oid 42 updating version 1659 to 1660 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:3197: oid 42 version 1660 is already newer than 1658 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1660 (ObjNum 1111 snap 333 seq_num 1111) dirty exists 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:3197: left oid 42 (ObjNum 1111 snap 333 seq_num 1111) 2026-02-17T06:05:48.297 INFO:tasks.rados.rados.0.trial115.stdout:3195: done (4 left) 2026-02-17T06:05:48.298 INFO:tasks.rados.rados.0.trial115.stdout:3196: done (3 left) 2026-02-17T06:05:48.298 INFO:tasks.rados.rados.0.trial115.stdout:3197: done (2 left) 2026-02-17T06:05:48.298 INFO:tasks.rados.rados.0.trial115.stdout:3199: done (1 left) 2026-02-17T06:05:48.298 INFO:tasks.rados.rados.0.trial115.stdout:3200: read oid 42 snap -1 2026-02-17T06:05:48.298 INFO:tasks.rados.rados.0.trial115.stdout:3200: expect (ObjNum 1111 snap 333 seq_num 1111) 2026-02-17T06:05:48.298 INFO:tasks.rados.rados.0.trial115.stdout:3201: delete oid 9 current snap is 333 2026-02-17T06:05:48.298 INFO:tasks.rados.rados.0.trial115.stdout:3201: done (2 left) 2026-02-17T06:05:48.299 INFO:tasks.rados.rados.0.trial115.stdout:3202: snap_create 2026-02-17T06:05:48.299 INFO:tasks.rados.rados.0.trial115.stdout:3198: expect (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:05:48.434 INFO:tasks.rados.rados.0.trial115.stdout:3200: expect (ObjNum 1111 snap 333 seq_num 1111) 2026-02-17T06:05:49.275 INFO:tasks.rados.rados.0.trial115.stdout:3198: done (2 left) 2026-02-17T06:05:49.275 INFO:tasks.rados.rados.0.trial115.stdout:3200: done (1 left) 2026-02-17T06:05:49.275 INFO:tasks.rados.rados.0.trial115.stdout:3202: done (0 left) 2026-02-17T06:05:49.275 INFO:tasks.rados.rados.0.trial115.stdout:3203: copy_from oid 16 from oid 10 current snap is 334 2026-02-17T06:05:49.276 INFO:tasks.rados.rados.0.trial115.stdout:3204: snap_remove snap 331 2026-02-17T06:05:50.277 INFO:tasks.rados.rados.0.trial115.stdout:3204: done (1 left) 2026-02-17T06:05:50.277 INFO:tasks.rados.rados.0.trial115.stdout:3205: delete oid 8 current snap is 334 2026-02-17T06:05:50.277 INFO:tasks.rados.rados.0.trial115.stdout:3203: finishing copy_from racing read to trial11545420-16 2026-02-17T06:05:50.277 INFO:tasks.rados.rados.0.trial115.stdout:3203: finishing copy_from to trial11545420-16 2026-02-17T06:05:50.277 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1622 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:05:50.279 INFO:tasks.rados.rados.0.trial115.stdout:3203: done (1 left) 2026-02-17T06:05:50.279 INFO:tasks.rados.rados.0.trial115.stdout:3205: done (0 left) 2026-02-17T06:05:50.279 INFO:tasks.rados.rados.0.trial115.stdout:3206: read oid 44 snap -1 2026-02-17T06:05:50.279 INFO:tasks.rados.rados.0.trial115.stdout:3206: expect (ObjNum 1100 snap 329 seq_num 1100) 2026-02-17T06:05:50.279 INFO:tasks.rados.rados.0.trial115.stdout:3207: read oid 1 snap -1 2026-02-17T06:05:50.279 INFO:tasks.rados.rados.0.trial115.stdout:3207: expect deleted 2026-02-17T06:05:50.279 INFO:tasks.rados.rados.0.trial115.stdout:3208: rollback oid 36 current snap is 334 2026-02-17T06:05:50.280 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 36 to 287 2026-02-17T06:05:50.280 INFO:tasks.rados.rados.0.trial115.stdout:3209: write (excl) oid 11 current snap is 334 2026-02-17T06:05:50.280 INFO:tasks.rados.rados.0.trial115.stdout:3209: seq_num 1115 ranges {739805=490173,1630599=435053,2636284=103521} 2026-02-17T06:05:50.284 INFO:tasks.rados.rados.0.trial115.stdout:3209: writing trial11545420-11 from 739805 to 1229978 tid 1 2026-02-17T06:05:50.288 INFO:tasks.rados.rados.0.trial115.stdout:3209: writing trial11545420-11 from 1630599 to 2065652 tid 2 2026-02-17T06:05:50.291 INFO:tasks.rados.rados.0.trial115.stdout:3209: writing trial11545420-11 from 2636284 to 2739805 tid 3 2026-02-17T06:05:50.291 INFO:tasks.rados.rados.0.trial115.stdout:3210: read oid 39 snap -1 2026-02-17T06:05:50.291 INFO:tasks.rados.rados.0.trial115.stdout:3210: expect (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:05:50.291 INFO:tasks.rados.rados.0.trial115.stdout:3211: delete oid 21 current snap is 334 2026-02-17T06:05:50.291 INFO:tasks.rados.rados.0.trial115.stdout:3208: finishing rollback tid 1 to trial11545420-36 2026-02-17T06:05:50.291 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 1486 (ObjNum 81 snap 0 seq_num 1) dirty dne 2026-02-17T06:05:50.291 INFO:tasks.rados.rados.0.trial115.stdout:3206: expect (ObjNum 1100 snap 329 seq_num 1100) 2026-02-17T06:05:50.421 INFO:tasks.rados.rados.0.trial115.stdout:3209: finishing write tid 1 to trial11545420-11 2026-02-17T06:05:50.421 INFO:tasks.rados.rados.0.trial115.stdout:3209: finishing write tid 2 to trial11545420-11 2026-02-17T06:05:50.421 INFO:tasks.rados.rados.0.trial115.stdout:3209: finishing write tid 3 to trial11545420-11 2026-02-17T06:05:50.421 INFO:tasks.rados.rados.0.trial115.stdout:3209: finishing write tid 4 to trial11545420-11 2026-02-17T06:05:50.421 INFO:tasks.rados.rados.0.trial115.stdout:3209: finishing write tid 5 to trial11545420-11 2026-02-17T06:05:50.421 INFO:tasks.rados.rados.0.trial115.stdout:3209: oid 11 updating version 0 to 1636 2026-02-17T06:05:50.421 INFO:tasks.rados.rados.0.trial115.stdout:3209: oid 11 version 1636 is already newer than 1635 2026-02-17T06:05:50.422 INFO:tasks.rados.rados.0.trial115.stdout:3209: oid 11 updating version 1636 to 1637 2026-02-17T06:05:50.422 INFO:tasks.rados.rados.0.trial115.stdout:3209: oid 11 updating version 1637 to 1638 2026-02-17T06:05:50.422 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1638 (ObjNum 1115 snap 334 seq_num 1115) dirty exists 2026-02-17T06:05:50.422 INFO:tasks.rados.rados.0.trial115.stdout:3209: left oid 11 (ObjNum 1115 snap 334 seq_num 1115) 2026-02-17T06:05:50.422 INFO:tasks.rados.rados.0.trial115.stdout:3210: expect (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:05:50.537 INFO:tasks.rados.rados.0.trial115.stdout:3206: done (5 left) 2026-02-17T06:05:50.537 INFO:tasks.rados.rados.0.trial115.stdout:3207: done (4 left) 2026-02-17T06:05:50.537 INFO:tasks.rados.rados.0.trial115.stdout:3208: done (3 left) 2026-02-17T06:05:50.537 INFO:tasks.rados.rados.0.trial115.stdout:3209: done (2 left) 2026-02-17T06:05:50.537 INFO:tasks.rados.rados.0.trial115.stdout:3210: done (1 left) 2026-02-17T06:05:50.537 INFO:tasks.rados.rados.0.trial115.stdout:3211: done (0 left) 2026-02-17T06:05:50.537 INFO:tasks.rados.rados.0.trial115.stdout:3212: snap_create 2026-02-17T06:05:51.269 INFO:tasks.rados.rados.0.trial115.stdout:3212: done (0 left) 2026-02-17T06:05:51.270 INFO:tasks.rados.rados.0.trial115.stdout:3213: snap_create 2026-02-17T06:05:52.269 INFO:tasks.rados.rados.0.trial115.stdout:3213: done (0 left) 2026-02-17T06:05:52.270 INFO:tasks.rados.rados.0.trial115.stdout:3214: rollback oid 21 current snap is 336 2026-02-17T06:05:52.270 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 21 to 301 2026-02-17T06:05:52.270 INFO:tasks.rados.rados.0.trial115.stdout:3215: copy_from oid 10 from oid 19 current snap is 336 2026-02-17T06:05:52.270 INFO:tasks.rados.rados.0.trial115.stdout:3216: copy_from oid 11 from oid 14 current snap is 336 2026-02-17T06:05:52.270 INFO:tasks.rados.rados.0.trial115.stdout:3217: snap_remove snap 333 2026-02-17T06:05:53.272 INFO:tasks.rados.rados.0.trial115.stdout:3217: done (3 left) 2026-02-17T06:05:53.273 INFO:tasks.rados.rados.0.trial115.stdout:3218: write (excl) oid 7 current snap is 336 2026-02-17T06:05:53.273 INFO:tasks.rados.rados.0.trial115.stdout:3218: seq_num 1117 ranges {773419=640671,2147563=438722} 2026-02-17T06:05:53.278 INFO:tasks.rados.rados.0.trial115.stdout:3218: writing trial11545420-7 from 773419 to 1414090 tid 1 2026-02-17T06:05:53.283 INFO:tasks.rados.rados.0.trial115.stdout:3218: writing trial11545420-7 from 2147563 to 2586285 tid 2 2026-02-17T06:05:53.283 INFO:tasks.rados.rados.0.trial115.stdout:3219: write (excl) oid 35 current snap is 336 2026-02-17T06:05:53.283 INFO:tasks.rados.rados.0.trial115.stdout:3219: seq_num 1118 ranges {790226=432273,1806045=582380} 2026-02-17T06:05:53.288 INFO:tasks.rados.rados.0.trial115.stdout:3219: writing trial11545420-35 from 790226 to 1222499 tid 1 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3219: writing trial11545420-35 from 1806045 to 2388425 tid 2 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3220: delete oid 28 current snap is 336 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3214: finishing rollback tid 1 to trial11545420-21 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1412 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3216: finishing copy_from racing read to trial11545420-11 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3215: finishing copy_from racing read to trial11545420-10 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3215: finishing copy_from to trial11545420-10 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1581 (ObjNum 1063 snap 319 seq_num 1063) dirty exists 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3216: finishing copy_from to trial11545420-11 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1485 (ObjNum 1105 snap 332 seq_num 1105) dirty exists 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3218: finishing write tid 1 to trial11545420-7 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3218: finishing write tid 2 to trial11545420-7 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3218: finishing write tid 3 to trial11545420-7 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3218: finishing write tid 4 to trial11545420-7 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3218: oid 7 updating version 0 to 1471 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3218: oid 7 version 1471 is already newer than 1470 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3218: oid 7 updating version 1471 to 1472 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1472 (ObjNum 1117 snap 336 seq_num 1117) dirty exists 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3218: left oid 7 (ObjNum 1117 snap 336 seq_num 1117) 2026-02-17T06:05:53.293 INFO:tasks.rados.rados.0.trial115.stdout:3219: finishing write tid 1 to trial11545420-35 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:3219: finishing write tid 2 to trial11545420-35 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:3219: finishing write tid 3 to trial11545420-35 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:3219: finishing write tid 4 to trial11545420-35 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:3219: oid 35 updating version 0 to 1687 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:3219: oid 35 version 1687 is already newer than 1683 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:3219: oid 35 version 1687 is already newer than 1686 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1687 (ObjNum 1118 snap 336 seq_num 1118) dirty exists 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:3219: left oid 35 (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:05:53.295 INFO:tasks.rados.rados.0.trial115.stdout:3214: done (5 left) 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3215: done (4 left) 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3216: done (3 left) 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3218: done (2 left) 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3219: done (1 left) 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3220: done (0 left) 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3221: read oid 28 snap -1 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3221: expect deleted 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3222: write oid 6 current snap is 336 2026-02-17T06:05:53.296 INFO:tasks.rados.rados.0.trial115.stdout:3222: seq_num 1120 ranges {423840=582771,1523009=586049,2657046=166794} 2026-02-17T06:05:53.299 INFO:tasks.rados.rados.0.trial115.stdout:3222: writing trial11545420-6 from 423840 to 1006611 tid 1 2026-02-17T06:05:53.304 INFO:tasks.rados.rados.0.trial115.stdout:3222: writing trial11545420-6 from 1523009 to 2109058 tid 2 2026-02-17T06:05:53.307 INFO:tasks.rados.rados.0.trial115.stdout:3222: writing trial11545420-6 from 2657046 to 2823840 tid 3 2026-02-17T06:05:53.307 INFO:tasks.rados.rados.0.trial115.stdout:3223: read oid 10 snap -1 2026-02-17T06:05:53.307 INFO:tasks.rados.rados.0.trial115.stdout:3223: expect (ObjNum 1063 snap 319 seq_num 1063) 2026-02-17T06:05:53.308 INFO:tasks.rados.rados.0.trial115.stdout:3224: write (excl) oid 11 current snap is 336 2026-02-17T06:05:53.308 INFO:tasks.rados.rados.0.trial115.stdout:3224: seq_num 1121 ranges {440647=658020,1581491=446060,2800822=39825} 2026-02-17T06:05:53.311 INFO:tasks.rados.rados.0.trial115.stdout:3224: writing trial11545420-11 from 440647 to 1098667 tid 1 2026-02-17T06:05:53.314 INFO:tasks.rados.rados.0.trial115.stdout:3224: writing trial11545420-11 from 1581491 to 2027551 tid 2 2026-02-17T06:05:53.317 INFO:tasks.rados.rados.0.trial115.stdout:3224: writing trial11545420-11 from 2800822 to 2840647 tid 3 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3225: rollback oid 14 current snap is 336 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 317 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: finishing write tid 1 to trial11545420-6 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: finishing write tid 2 to trial11545420-6 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: finishing write tid 3 to trial11545420-6 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: finishing write tid 4 to trial11545420-6 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: finishing write tid 5 to trial11545420-6 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: oid 6 updating version 0 to 1486 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: oid 6 version 1486 is already newer than 1485 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: oid 6 version 1486 is already newer than 1483 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: oid 6 version 1486 is already newer than 1484 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 1486 (ObjNum 1120 snap 336 seq_num 1120) dirty exists 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3222: left oid 6 (ObjNum 1120 snap 336 seq_num 1120) 2026-02-17T06:05:53.318 INFO:tasks.rados.rados.0.trial115.stdout:3223: expect (ObjNum 1063 snap 319 seq_num 1063) 2026-02-17T06:05:53.407 INFO:tasks.rados.rados.0.trial115.stdout:3224: finishing write tid 1 to trial11545420-11 2026-02-17T06:05:53.407 INFO:tasks.rados.rados.0.trial115.stdout:3224: finishing write tid 2 to trial11545420-11 2026-02-17T06:05:53.407 INFO:tasks.rados.rados.0.trial115.stdout:3224: finishing write tid 3 to trial11545420-11 2026-02-17T06:05:53.407 INFO:tasks.rados.rados.0.trial115.stdout:3221: done (4 left) 2026-02-17T06:05:53.407 INFO:tasks.rados.rados.0.trial115.stdout:3222: done (3 left) 2026-02-17T06:05:53.407 INFO:tasks.rados.rados.0.trial115.stdout:3223: done (2 left) 2026-02-17T06:05:53.407 INFO:tasks.rados.rados.0.trial115.stdout:3226: read oid 39 snap 335 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3226: expect (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3227: write oid 10 current snap is 336 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3224: finishing write tid 4 to trial11545420-11 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3224: finishing write tid 5 to trial11545420-11 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3224: oid 11 updating version 0 to 1649 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3224: oid 11 updating version 1649 to 1651 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3224: oid 11 version 1651 is already newer than 1648 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3224: oid 11 version 1651 is already newer than 1650 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1651 (ObjNum 1121 snap 336 seq_num 1121) dirty exists 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3224: left oid 11 (ObjNum 1121 snap 336 seq_num 1121) 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3225: finishing rollback tid 0 to trial11545420-14 2026-02-17T06:05:53.408 INFO:tasks.rados.rados.0.trial115.stdout:3227: seq_num 1122 ranges {457454=733269,1923620=589718} 2026-02-17T06:05:53.411 INFO:tasks.rados.rados.0.trial115.stdout:3227: writing trial11545420-10 from 457454 to 1190723 tid 1 2026-02-17T06:05:53.416 INFO:tasks.rados.rados.0.trial115.stdout:3227: writing trial11545420-10 from 1923620 to 2513338 tid 2 2026-02-17T06:05:53.416 INFO:tasks.rados.rados.0.trial115.stdout:3224: done (3 left) 2026-02-17T06:05:53.416 INFO:tasks.rados.rados.0.trial115.stdout:3228: copy_from oid 25 from oid 6 current snap is 336 2026-02-17T06:05:53.417 INFO:tasks.rados.rados.0.trial115.stdout:3225: finishing rollback tid 1 to trial11545420-14 2026-02-17T06:05:53.417 INFO:tasks.rados.rados.0.trial115.stdout:3225: finishing rollback tid 2 to trial11545420-14 2026-02-17T06:05:53.417 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1489 (ObjNum 1049 snap 312 seq_num 1049) dirty exists 2026-02-17T06:05:53.417 INFO:tasks.rados.rados.0.trial115.stdout:3226: expect (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:05:53.533 INFO:tasks.rados.rados.0.trial115.stdout:3227: finishing write tid 1 to trial11545420-10 2026-02-17T06:05:53.533 INFO:tasks.rados.rados.0.trial115.stdout:3227: finishing write tid 2 to trial11545420-10 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3225: done (3 left) 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3226: done (2 left) 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3229: write oid 2 current snap is 336 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3227: finishing write tid 3 to trial11545420-10 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3227: finishing write tid 4 to trial11545420-10 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3227: oid 10 updating version 0 to 1654 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3227: oid 10 version 1654 is already newer than 1652 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3227: oid 10 version 1654 is already newer than 1653 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1654 (ObjNum 1122 snap 336 seq_num 1122) dirty exists 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3227: left oid 10 (ObjNum 1122 snap 336 seq_num 1122) 2026-02-17T06:05:53.534 INFO:tasks.rados.rados.0.trial115.stdout:3228: finishing copy_from racing read to trial11545420-25 2026-02-17T06:05:53.535 INFO:tasks.rados.rados.0.trial115.stdout:3228: finishing copy_from to trial11545420-25 2026-02-17T06:05:53.535 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1486 (ObjNum 1120 snap 336 seq_num 1120) dirty exists 2026-02-17T06:05:53.535 INFO:tasks.rados.rados.0.trial115.stdout:3229: seq_num 1123 ranges {474261=408518,1582102=449729,2739315=134946} 2026-02-17T06:05:53.536 INFO:tasks.rados.rados.0.trial115.stdout:3229: writing trial11545420-2 from 474261 to 882779 tid 1 2026-02-17T06:05:53.541 INFO:tasks.rados.rados.0.trial115.stdout:3229: writing trial11545420-2 from 1582102 to 2031831 tid 2 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3229: writing trial11545420-2 from 2739315 to 2874261 tid 3 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3227: done (2 left) 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3228: done (1 left) 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3230: copy_from oid 17 from oid 35 current snap is 336 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3229: finishing write tid 1 to trial11545420-2 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3229: finishing write tid 2 to trial11545420-2 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3231: read oid 39 snap -1 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3231: expect (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3232: read oid 26 snap -1 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3232: expect (ObjNum 1095 snap 329 seq_num 1095) 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3233: read oid 28 snap 301 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3233: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:05:53.544 INFO:tasks.rados.rados.0.trial115.stdout:3234: write (excl) oid 31 current snap is 336 2026-02-17T06:05:53.545 INFO:tasks.rados.rados.0.trial115.stdout:3234: seq_num 1124 ranges {491068=483767,1640584=593387,2650385=240683} 2026-02-17T06:05:53.547 INFO:tasks.rados.rados.0.trial115.stdout:3234: writing trial11545420-31 from 491068 to 974835 tid 1 2026-02-17T06:05:53.552 INFO:tasks.rados.rados.0.trial115.stdout:3234: writing trial11545420-31 from 1640584 to 2233971 tid 2 2026-02-17T06:05:53.555 INFO:tasks.rados.rados.0.trial115.stdout:3234: writing trial11545420-31 from 2650385 to 2891068 tid 3 2026-02-17T06:05:53.555 INFO:tasks.rados.rados.0.trial115.stdout:3235: copy_from oid 14 from oid 40 current snap is 336 2026-02-17T06:05:53.555 INFO:tasks.rados.rados.0.trial115.stdout:3231: expect (ObjNum 1049 snap 312 seq_num 1049) 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3229: finishing write tid 3 to trial11545420-2 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3229: finishing write tid 4 to trial11545420-2 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3231: done (6 left) 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3236: write (excl) oid 5 current snap is 336 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3229: finishing write tid 5 to trial11545420-2 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3229: oid 2 updating version 0 to 1461 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3229: oid 2 version 1461 is already newer than 1460 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3229: oid 2 updating version 1461 to 1463 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:3229: oid 2 version 1463 is already newer than 1462 2026-02-17T06:05:53.672 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1463 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2026-02-17T06:05:53.673 INFO:tasks.rados.rados.0.trial115.stdout:3229: left oid 2 (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:05:53.673 INFO:tasks.rados.rados.0.trial115.stdout:3232: expect (ObjNum 1095 snap 329 seq_num 1095) 2026-02-17T06:05:53.725 INFO:tasks.rados.rados.0.trial115.stdout:3233: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:05:53.856 INFO:tasks.rados.rados.0.trial115.stdout:3234: finishing write tid 1 to trial11545420-31 2026-02-17T06:05:53.856 INFO:tasks.rados.rados.0.trial115.stdout:3236: seq_num 1125 ranges {507875=559016,1699066=737045} 2026-02-17T06:05:53.860 INFO:tasks.rados.rados.0.trial115.stdout:3236: writing trial11545420-5 from 507875 to 1066891 tid 1 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3236: writing trial11545420-5 from 1699066 to 2436111 tid 2 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3229: done (6 left) 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3232: done (5 left) 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3233: done (4 left) 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3237: copy_from oid 44 from oid 48 current snap is 336 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3230: finishing copy_from racing read to trial11545420-17 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3230: finishing copy_from to trial11545420-17 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1687 (ObjNum 1118 snap 336 seq_num 1118) dirty exists 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: finishing write tid 2 to trial11545420-31 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: finishing write tid 3 to trial11545420-31 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: finishing write tid 4 to trial11545420-31 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: finishing write tid 5 to trial11545420-31 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: oid 31 updating version 0 to 1465 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: oid 31 version 1465 is already newer than 1464 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: oid 31 updating version 1465 to 1467 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: oid 31 version 1467 is already newer than 1466 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1467 (ObjNum 1124 snap 336 seq_num 1124) dirty exists 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3234: left oid 31 (ObjNum 1124 snap 336 seq_num 1124) 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3235: finishing copy_from racing read to trial11545420-14 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3235: finishing copy_from to trial11545420-14 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1417 (ObjNum 1026 snap 297 seq_num 1026) dirty exists 2026-02-17T06:05:53.865 INFO:tasks.rados.rados.0.trial115.stdout:3236: finishing write tid 1 to trial11545420-5 2026-02-17T06:05:53.866 INFO:tasks.rados.rados.0.trial115.stdout:3230: done (4 left) 2026-02-17T06:05:53.866 INFO:tasks.rados.rados.0.trial115.stdout:3234: done (3 left) 2026-02-17T06:05:53.866 INFO:tasks.rados.rados.0.trial115.stdout:3235: done (2 left) 2026-02-17T06:05:53.866 INFO:tasks.rados.rados.0.trial115.stdout:3238: snap_remove snap 326 2026-02-17T06:05:54.276 INFO:tasks.rados.rados.0.trial115.stdout:3238: done (2 left) 2026-02-17T06:05:54.276 INFO:tasks.rados.rados.0.trial115.stdout:3239: write oid 9 current snap is 336 2026-02-17T06:05:54.276 INFO:tasks.rados.rados.0.trial115.stdout:3239: seq_num 1126 ranges {524682=750618,1757548=597056} 2026-02-17T06:05:54.281 INFO:tasks.rados.rados.0.trial115.stdout:3239: writing trial11545420-9 from 524682 to 1275300 tid 1 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3239: writing trial11545420-9 from 1757548 to 2354604 tid 2 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3240: write oid 26 current snap is 336 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3236: finishing write tid 2 to trial11545420-5 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3236: finishing write tid 3 to trial11545420-5 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3236: finishing write tid 4 to trial11545420-5 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3236: oid 5 updating version 0 to 1469 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3236: oid 5 updating version 1469 to 1470 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3236: oid 5 version 1470 is already newer than 1468 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1470 (ObjNum 1125 snap 336 seq_num 1125) dirty exists 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3236: left oid 5 (ObjNum 1125 snap 336 seq_num 1125) 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3237: finishing copy_from racing read to trial11545420-44 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3237: finishing copy_from to trial11545420-44 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1345 (ObjNum 1050 snap 312 seq_num 1050) dirty exists 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3239: finishing write tid 1 to trial11545420-9 2026-02-17T06:05:54.286 INFO:tasks.rados.rados.0.trial115.stdout:3240: seq_num 1127 ranges {541489=425867,1416030=740714,2732654=208835} 2026-02-17T06:05:54.290 INFO:tasks.rados.rados.0.trial115.stdout:3240: writing trial11545420-26 from 541489 to 967356 tid 1 2026-02-17T06:05:54.294 INFO:tasks.rados.rados.0.trial115.stdout:3240: writing trial11545420-26 from 1416030 to 2156744 tid 2 2026-02-17T06:05:54.298 INFO:tasks.rados.rados.0.trial115.stdout:3240: writing trial11545420-26 from 2732654 to 2941489 tid 3 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3236: done (3 left) 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3237: done (2 left) 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3241: rollback oid 36 current snap is 336 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 36 to 332 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3239: finishing write tid 2 to trial11545420-9 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3239: finishing write tid 3 to trial11545420-9 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3239: finishing write tid 4 to trial11545420-9 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3239: oid 9 updating version 0 to 1655 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3239: oid 9 updating version 1655 to 1663 2026-02-17T06:05:54.299 INFO:tasks.rados.rados.0.trial115.stdout:3239: oid 9 version 1663 is already newer than 1662 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1663 (ObjNum 1126 snap 336 seq_num 1126) dirty exists 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:3239: left oid 9 (ObjNum 1126 snap 336 seq_num 1126) 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:3240: finishing write tid 1 to trial11545420-26 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:3240: finishing write tid 2 to trial11545420-26 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:3239: done (2 left) 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:3242: rollback oid 7 current snap is 336 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 301 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:3243: rollback oid 32 current snap is 336 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 335 2026-02-17T06:05:54.300 INFO:tasks.rados.rados.0.trial115.stdout:3244: snap_remove snap 287 2026-02-17T06:05:55.280 INFO:tasks.rados.rados.0.trial115.stdout:3244: done (4 left) 2026-02-17T06:05:55.280 INFO:tasks.rados.rados.0.trial115.stdout:3245: rollback oid 45 current snap is 336 2026-02-17T06:05:55.280 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 317 2026-02-17T06:05:55.280 INFO:tasks.rados.rados.0.trial115.stdout:3241: finishing rollback tid 1 to trial11545420-36 2026-02-17T06:05:55.280 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3240: finishing write tid 3 to trial11545420-26 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3240: finishing write tid 4 to trial11545420-26 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3240: finishing write tid 5 to trial11545420-26 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3240: oid 26 updating version 0 to 1458 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3240: oid 26 updating version 1458 to 1461 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3240: oid 26 version 1461 is already newer than 1459 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3240: oid 26 version 1461 is already newer than 1460 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1461 (ObjNum 1127 snap 336 seq_num 1127) dirty exists 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3240: left oid 26 (ObjNum 1127 snap 336 seq_num 1127) 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3242: finishing rollback tid 0 to trial11545420-7 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3242: finishing rollback tid 1 to trial11545420-7 2026-02-17T06:05:55.281 INFO:tasks.rados.rados.0.trial115.stdout:3243: finishing rollback tid 0 to trial11545420-32 2026-02-17T06:05:55.282 INFO:tasks.rados.rados.0.trial115.stdout:3243: finishing rollback tid 1 to trial11545420-32 2026-02-17T06:05:55.282 INFO:tasks.rados.rados.0.trial115.stdout:3240: done (4 left) 2026-02-17T06:05:55.282 INFO:tasks.rados.rados.0.trial115.stdout:3241: done (3 left) 2026-02-17T06:05:55.282 INFO:tasks.rados.rados.0.trial115.stdout:3246: snap_create 2026-02-17T06:05:55.282 INFO:tasks.rados.rados.0.trial115.stdout:3242: finishing rollback tid 2 to trial11545420-7 2026-02-17T06:05:55.282 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1475 (ObjNum 1019 snap 297 seq_num 1019) dirty exists 2026-02-17T06:05:55.282 INFO:tasks.rados.rados.0.trial115.stdout:3243: finishing rollback tid 2 to trial11545420-32 2026-02-17T06:05:55.282 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1458 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:05:55.283 INFO:tasks.rados.rados.0.trial115.stdout:3245: finishing rollback tid 0 to trial11545420-45 2026-02-17T06:05:55.283 INFO:tasks.rados.rados.0.trial115.stdout:3245: finishing rollback tid 1 to trial11545420-45 2026-02-17T06:05:55.284 INFO:tasks.rados.rados.0.trial115.stdout:3245: finishing rollback tid 2 to trial11545420-45 2026-02-17T06:05:55.285 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1675 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:05:56.270 INFO:tasks.rados.rados.0.trial115.stdout:3242: done (3 left) 2026-02-17T06:05:56.270 INFO:tasks.rados.rados.0.trial115.stdout:3243: done (2 left) 2026-02-17T06:05:56.270 INFO:tasks.rados.rados.0.trial115.stdout:3245: done (1 left) 2026-02-17T06:05:56.270 INFO:tasks.rados.rados.0.trial115.stdout:3246: done (0 left) 2026-02-17T06:05:56.270 INFO:tasks.rados.rados.0.trial115.stdout:3247: snap_remove snap 325 2026-02-17T06:05:56.387 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T06:05:56.387 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T06:05:56.387 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T06:05:56.387 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T06:05:56.391 INFO:tasks.ceph.mon.a:Started 2026-02-17T06:05:56.391 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:05:56.391 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:56.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.458+0000 7f1c90b6a640 10 monclient: get_monmap_and_config 2026-02-17T06:05:56.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.458+0000 7f1c90b6a640 10 monclient: build_initial_monmap 2026-02-17T06:05:56.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.458+0000 7f1c90b6a640 10 monclient: monmap: 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:56.460026+0000 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:56.460026+0000 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:56.470 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:56.471 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:56.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 1 Processor -- start 2026-02-17T06:05:56.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 1 -- start start 2026-02-17T06:05:56.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient: init 2026-02-17T06:05:56.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:56.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2026-02-17T06:05:56.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1c8c09ea90 addr v1:10.20.193.115:6789/0 2026-02-17T06:05:56.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f1c8c09ee70 0x7f1c8c09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient(hunting): picked mon.noname-c con 0x7f1c8c09ee70 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c13d640 0x7f1c8c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient(hunting): picked mon.noname-b con 0x7f1c8c13d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c8c0a0a20 con 0x7f1c8c09ea90 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f1c8c0a0d00 con 0x7f1c8c09ee70 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8c0a0460 con 0x7f1c8c13d640 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c90b6a640 10 monclient(hunting): authenticate will time out at 1802.013592s 2026-02-17T06:05:56.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c8a575640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f1c8c09ea90 0x7f1c8c0a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:50966/0 (socket says 10.20.193.78:50966) 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c8a575640 1 -- 10.20.193.78:0/2098747347 learned_addr learned my addr 10.20.193.78:0/2098747347 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c89d74640 1 --2- 10.20.193.78:0/2098747347 >> v2:10.20.193.115:3300/0 conn(0x7f1c8c09ee70 0x7f1c8c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c89d74640 10 monclient(hunting): get_auth_request con 0x7f1c8c09ee70 auth_method 0 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c89d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c89d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c89d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c89d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c89d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.459+0000 7f1c89d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 10 monclient(hunting): handle_auth_done global_id 214288 payload 274 2026-02-17T06:05:56.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 1 -- 10.20.193.78:0/2098747347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c13d640 msgr2=0x7f1c8c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 1 --2- 10.20.193.78:0/2098747347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c13d640 0x7f1c8c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 1 -- 10.20.193.78:0/2098747347 >> v1:10.20.193.115:6789/0 conn(0x7f1c8c09ea90 legacy=0x7f1c8c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 10 monclient: _finish_hunting 0 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 1 monclient: found mon.noname-c 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.460+0000 7f1c89d74640 1 -- 10.20.193.78:0/2098747347 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c0a0740 con 0x7f1c8c09ee70 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.462+0000 7f1c89d74640 1 -- 10.20.193.78:0/2098747347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ee70 msgr2=0x7f1c8c09f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.462+0000 7f1c89d74640 1 --2- 10.20.193.78:0/2098747347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ee70 0x7f1c8c09f2c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c06e270 tx=0x7f1c7402ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7244fe2055fdd45a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 1 -- 10.20.193.78:0/2098747347 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7403c070 con 0x7f1c8c09ee70 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:56.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: dump: 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: _finish_auth 0 2026-02-17T06:05:56.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: _check_auth_tickets 2026-02-17T06:05:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 1 -- 10.20.193.78:0/2098747347 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c7402fcb0 con 0x7f1c8c09ee70 2026-02-17T06:05:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 1 -- 10.20.193.78:0/2098747347 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7402fe60 con 0x7f1c8c09ee70 2026-02-17T06:05:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:05:56.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c89573640 10 monclient: dump: 2026-02-17T06:05:56.477 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:56.477 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:56.477 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:56.477 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:56.477 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:56.477 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:56.477 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:56.478 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:56.478 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 5 monclient: authenticate success, global_id 214288 2026-02-17T06:05:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 10 monclient: get_monmap_and_config success 2026-02-17T06:05:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 4 set_mon_vals no callback set 2026-02-17T06:05:56.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 10 monclient: shutdown 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 1 -- 10.20.193.78:0/2098747347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ee70 msgr2=0x7f1c8c09f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/2098747347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ee70 0x7f1c8c09f2c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c06e270 tx=0x7f1c7402ed50 comp rx=0 tx=0).stop 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 1 -- 10.20.193.78:0/2098747347 shutdown_connections 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/2098747347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c13d640 0x7f1c8c13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/2098747347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ee70 0x7f1c8c09f2c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 1 -- 10.20.193.78:0/2098747347 >> 10.20.193.78:0/2098747347 conn(0x7f1c8c0a7520 msgr2=0x7f1c8c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 1 -- 10.20.193.78:0/2098747347 shutdown_connections 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.463+0000 7f1c90b6a640 1 -- 10.20.193.78:0/2098747347 wait complete. 2026-02-17T06:05:56.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient: build_initial_monmap 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient: monmap: 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:56.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 1 Processor -- start 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 1 -- start start 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient: init 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ea90 0x7f1c8c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient(hunting): picked mon.noname-b con 0x7f1c8c09ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c09ee70 0x7f1c8c13bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient(hunting): picked mon.noname-c con 0x7f1c8c09ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 0x7f1c8c13c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient(hunting): picked mon.noname-a con 0x7f1c8c13d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:56.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c0a6820 con 0x7f1c8c13d640 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c0a66a0 con 0x7f1c8c09ea90 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8c13ff70 con 0x7f1c8c09ee70 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c90b6a640 10 monclient(hunting): authenticate will time out at 1802.018550s 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c09ee70 0x7f1c8c13bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c09ee70 0x7f1c8c13bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58896/0 (socket says 10.20.193.78:58896) 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 -- 10.20.193.78:0/3774055821 learned_addr learned my addr 10.20.193.78:0/3774055821 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient(hunting): get_auth_request con 0x7f1c8c09ee70 auth_method 0 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:56.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 0x7f1c8c13c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 10 monclient(hunting): get_auth_request con 0x7f1c8c13d640 auth_method 0 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 1 -- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 msgr2=0x7f1c8c13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 1 -- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 msgr2=0x7f1c8c13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 0x7f1c8c13c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:56.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8ad76640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 0x7f1c8c13c1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8a575640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ea90 0x7f1c8c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient(hunting): handle_auth_done global_id 234188 payload 824 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 -- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ea90 msgr2=0x7f1c8c13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ea90 0x7f1c8c13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 -- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 msgr2=0x7f1c8c13c1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 0x7f1c8c13c1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient: _finish_hunting 0 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 monclient: found mon.noname-c 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:05:56.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c89d74640 1 -- 10.20.193.78:0/3774055821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c070ce0 con 0x7f1c8c09ee70 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8a575640 10 monclient: get_auth_request con 0x7f1c8c09ea90 auth_method 0 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.464+0000 7f1c8a575640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ea90 0x7f1c8c13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.466+0000 7f1c89d74640 1 -- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c09ee70 msgr2=0x7f1c8c13bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.466+0000 7f1c89d74640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c09ee70 0x7f1c8c13bca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1c740099a0 tx=0x7f1c740047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.466+0000 7f1c72ffd640 1 -- 10.20.193.78:0/3774055821 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74047020 con 0x7f1c8c09ee70 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.466+0000 7f1c72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.466+0000 7f1c72ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.466+0000 7f1c72ffd640 10 monclient: dump: 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:56.485 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 10 monclient: _finish_auth 0 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 1 -- 10.20.193.78:0/3774055821 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c74042690 con 0x7f1c8c09ee70 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 1 -- 10.20.193.78:0/3774055821 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7403c040 con 0x7f1c8c09ee70 2026-02-17T06:05:56.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 10 monclient: dump: 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:05:56.487 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c88d72640 4 set_mon_vals no callback set 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 5 monclient: authenticate success, global_id 234188 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 10 monclient: _renew_subs 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8c0a9c50 con 0x7f1c8c09ee70 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 10 monclient: _renew_subs 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8c06f5b0 con 0x7f1c8c09ee70 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 1 -- 10.20.193.78:0/3774055821 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1c74042a80 con 0x7f1c8c09ee70 2026-02-17T06:05:56.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c8c09f2c0 con 0x7f1c8c09ee70 2026-02-17T06:05:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c5803deb0 0x7f1c58040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c72ffd640 1 -- 10.20.193.78:0/3774055821 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(697..697 src has 525..697) ==== 4811+0+0 (secure 0 0 0) 0x7f1c740780f0 con 0x7f1c8c09ee70 2026-02-17T06:05:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.467+0000 7f1c8a575640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c5803deb0 0x7f1c58040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:56.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.468+0000 7f1c8a575640 10 monclient: get_auth_request con 0x7f1c5803deb0 auth_method 0 2026-02-17T06:05:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.470+0000 7f1c72ffd640 1 -- 10.20.193.78:0/3774055821 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1c74051030 con 0x7f1c8c09ee70 2026-02-17T06:05:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.470+0000 7f1c72ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:05:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.470+0000 7f1c72ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:05:56.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.470+0000 7f1c8a575640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c5803deb0 0x7f1c58040370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000a8d0 tx=0x7f1c80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:05:56.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.582+0000 7f1c90b6a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:05:56.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.582+0000 7f1c90b6a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:56.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.582+0000 7f1c90b6a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:05:56.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.582+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1c8c0a1c10 con 0x7f1c8c09ee70 2026-02-17T06:05:56.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.583+0000 7f1c72ffd640 1 -- 10.20.193.78:0/3774055821 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f1c74053300 con 0x7f1c8c09ee70 2026-02-17T06:05:56.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.583+0000 7f1c72ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:05:56.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.583+0000 7f1c72ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:05:56.586 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":224,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:05:56.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c5803deb0 msgr2=0x7f1c58040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:56.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c5803deb0 0x7f1c58040370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000a8d0 tx=0x7f1c80008040 comp rx=0 tx=0).stop 2026-02-17T06:05:56.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 10 monclient: shutdown 2026-02-17T06:05:56.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:05:56.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c09ee70 msgr2=0x7f1c8c13bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:05:56.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c09ee70 0x7f1c8c13bca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1c740099a0 tx=0x7f1c740047c0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 shutdown_connections 2026-02-17T06:05:56.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f1c8c13d640 0x7f1c8c13c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f1c8c09ee70 0x7f1c8c13bca0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f1c5803deb0 0x7f1c58040370 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 --2- 10.20.193.78:0/3774055821 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1c8c09ea90 0x7f1c8c13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:05:56.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 >> 10.20.193.78:0/3774055821 conn(0x7f1c8c0a7520 msgr2=0x7f1c8c136dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:05:56.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 shutdown_connections 2026-02-17T06:05:56.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:56.584+0000 7f1c90b6a640 1 -- 10.20.193.78:0/3774055821 wait complete. 2026-02-17T06:05:59.594 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:05:59.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient: get_monmap_and_config 2026-02-17T06:05:59.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient: build_initial_monmap 2026-02-17T06:05:59.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient: monmap: 2026-02-17T06:05:59.664 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:05:59.664 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:05:59.662366+0000 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:05:59.662366+0000 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:05:59.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 1 Processor -- start 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 1 -- start start 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient: init 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient: _reopen_session rank -1 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient(hunting): picked mon.noname-a con 0x7f4b5406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4b5409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient(hunting): picked mon.noname-c con 0x7f4b5406d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5406daf0 con 0x7f4b5409c4a0 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:59.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f4b540a0880 con 0x7f4b5406c890 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient(hunting): start opening mon connection 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5409cc80 con 0x7f4b5406d1b0 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient(hunting): _renew_subs 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.661+0000 7f4b594ca640 10 monclient(hunting): authenticate will time out at 1805.215933s 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b527fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f4b5409c4a0 0x7f4b54134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38088/0 (socket says 10.20.193.78:38088) 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b527fc640 1 -- 10.20.193.78:0/2369201152 learned_addr learned my addr 10.20.193.78:0/2369201152 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request con 0x7f4b5406c890 auth_method 0 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:59.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 10 monclient(hunting): get_auth_request con 0x7f4b5406d1b0 auth_method 0 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:59.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.662+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:05:59.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.664+0000 7f4b527fc640 1 -- 10.20.193.78:0/2369201152 >> v1:10.20.193.78:6789/0 conn(0x7f4b5409c4a0 legacy=0x7f4b54134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:05:59.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:59.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request con 0x7f4b5406c890 auth_method 0 2026-02-17T06:05:59.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:59.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:59.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:59.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:05:59.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:59.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:59.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:05:59.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:05:59.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 10 monclient(hunting): get_auth_request con 0x7f4b5406d1b0 auth_method 0 2026-02-17T06:05:59.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:05:59.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:05:59.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:05:59.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:05:59.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:05:59.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:05:59.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:05:59.862+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:00.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:00.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request con 0x7f4b5406c890 auth_method 0 2026-02-17T06:06:00.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:00.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:00.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:00.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:00.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:00.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 10 monclient(hunting): get_auth_request con 0x7f4b5406d1b0 auth_method 0 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:00.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:00.263+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:01.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:01.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request con 0x7f4b5406c890 auth_method 0 2026-02-17T06:06:01.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:01.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 10 monclient(hunting): get_auth_request con 0x7f4b5406d1b0 auth_method 0 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:01.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:01.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:01.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:01.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:01.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:01.068 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:01.064+0000 7f4b537fe640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:02.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 10 monclient(hunting): tick 2026-02-17T06:06:02.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:06:02.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 monclient(hunting): continuing hunt 2026-02-17T06:06:02.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:06:02.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 msgr2=0x7f4b54135500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:02.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.115:3300/0 conn(0x7f4b5406d1b0 0x7f4b54135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 -- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 msgr2=0x7f4b5406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 --2- 10.20.193.78:0/2369201152 >> v2:10.20.193.78:3300/0 conn(0x7f4b5406c890 0x7f4b5406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 -- 10.20.193.78:0/2369201152 >> v1:10.20.193.78:6789/0 conn(0x7f4b5409c4a0 legacy=0x7f4b54134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 10 monclient: _add_conns ranks=[2,1] 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b380037f0 0x7f4b38005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f4b380037f0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4b38006350 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 -- 10.20.193.78:0/2369201152 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5409cf60 con 0x7f4b38006350 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 1 -- 10.20.193.78:0/2369201152 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5409cc80 con 0x7f4b380037f0 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b50ff9640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b380037f0 0x7f4b38005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request con 0x7f4b380037f0 auth_method 0 2026-02-17T06:06:02.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b380037f0 msgr2=0x7f4b38005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:02.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b380037f0 msgr2=0x7f4b38005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:02.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b380037f0 0x7f4b38005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:02.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.662+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b380037f0 0x7f4b38005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:02.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.664+0000 7f4b527fc640 1 -- 10.20.193.78:0/2369201152 >> v1:10.20.193.115:6789/0 conn(0x7f4b38006350 legacy=0x7f4b38008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:02.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.684+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3369613069 0 0) 0x7f4b5409cf60 con 0x7f4b38006350 2026-02-17T06:06:02.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.684+0000 7f4b51ffb640 10 monclient(hunting): _negotiate 2026-02-17T06:06:02.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.684+0000 7f4b51ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.684+0000 7f4b51ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.684+0000 7f4b51ffb640 10 monclient(hunting): my global_id is 214291 2026-02-17T06:06:02.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.684+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b40003620 con 0x7f4b38006350 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2111097458 0 0) 0x7f4b40003620 con 0x7f4b38006350 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b380037f0 msgr2=0x7f4b38005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b380037f0 0x7f4b38005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: _finish_hunting 0 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3800b1c0 con 0x7f4b38006350 2026-02-17T06:06:02.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f4b3c0031f0 con 0x7f4b38006350 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b594ca640 5 monclient: authenticate success, global_id 214291 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b594ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: dump: 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 >> v1:10.20.193.115:6789/0 conn(0x7f4b38006350 legacy=0x7f4b38008810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b537fe640 1 -- 10.20.193.78:0/2369201152 reap_dead start 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40003b50 0x7f4b40003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient(hunting): picked mon.c con 0x7f4b40003b50 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4b40004900 0x7f4b40004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient(hunting): picked mon.b con 0x7f4b40004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 0x7f4b4000bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient(hunting): picked mon.a con 0x7f4b400054c0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5409cc80 con 0x7f4b400054c0 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4000c3c0 con 0x7f4b40004900 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4000c5c0 con 0x7f4b40003b50 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b51ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b594ca640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40003b50 0x7f4b40003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request con 0x7f4b40003b50 auth_method 0 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:02.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_done global_id 214291 payload 293 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4b40004900 msgr2=0x7f4b40004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4b40004900 0x7f4b40004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 msgr2=0x7f4b4000bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 0x7f4b4000bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 1 monclient: found mon.c 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b40003620 con 0x7f4b40003b50 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient: _finish_auth 0 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.685+0000 7f4b52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.687+0000 7f4b52ffd640 1 -- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40003b50 msgr2=0x7f4b40003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.687+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40003b50 0x7f4b40003f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b48014a20 tx=0x7f4b48030390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1d497a1d145f2f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48036070 con 0x7f4b40003b50 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 10 monclient: dump: 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.688 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b48030dd0 con 0x7f4b40003b50 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 1 -- 10.20.193.78:0/2369201152 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800f040 con 0x7f4b40003b50 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b51ffb640 10 monclient: dump: 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 4 set_mon_vals no callback set 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 10 monclient: shutdown 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 -- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40003b50 msgr2=0x7f4b40003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40003b50 0x7f4b40003f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b48014a20 tx=0x7f4b48030390 comp rx=0 tx=0).stop 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 -- 10.20.193.78:0/2369201152 shutdown_connections 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 0x7f4b4000bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4b40004900 0x7f4b40004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 --2- 10.20.193.78:0/2369201152 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40003b50 0x7f4b40003f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 -- 10.20.193.78:0/2369201152 >> 10.20.193.78:0/2369201152 conn(0x7f4b540a6620 msgr2=0x7f4b540a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 -- 10.20.193.78:0/2369201152 shutdown_connections 2026-02-17T06:06:02.690 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.688+0000 7f4b594ca640 1 -- 10.20.193.78:0/2369201152 wait complete. 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient: build_initial_monmap 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient: monmap: 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 1 Processor -- start 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 1 -- start start 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient: init 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4b40003b50 0x7f4b541cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient(hunting): picked mon.noname-b con 0x7f4b40003b50 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40004900 0x7f4b541350a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient(hunting): picked mon.noname-c con 0x7f4b40004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 0x7f4b541355e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient(hunting): picked mon.noname-a con 0x7f4b400054c0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540a01a0 con 0x7f4b400054c0 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540a0020 con 0x7f4b40003b50 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5409be20 con 0x7f4b40004900 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b594ca640 10 monclient(hunting): authenticate will time out at 1808.243825s 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40004900 0x7f4b541350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b537fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 0x7f4b541355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40004900 0x7f4b541350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46930/0 (socket says 10.20.193.78:46930) 2026-02-17T06:06:02.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 1 -- 10.20.193.78:0/3151913427 learned_addr learned my addr 10.20.193.78:0/3151913427 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 10 monclient(hunting): get_auth_request con 0x7f4b40004900 auth_method 0 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b537fe640 10 monclient(hunting): get_auth_request con 0x7f4b400054c0 auth_method 0 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b537fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.689+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 10 monclient(hunting): handle_auth_done global_id 244098 payload 824 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 1 -- 10.20.193.78:0/3151913427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40004900 msgr2=0x7f4b541350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40004900 0x7f4b541350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 1 -- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4b40003b50 msgr2=0x7f4b541cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4b40003b50 0x7f4b541cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 1 monclient: found mon.noname-a 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.690+0000 7f4b537fe640 1 -- 10.20.193.78:0/3151913427 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b54122720 con 0x7f4b400054c0 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b537fe640 1 -- 10.20.193.78:0/3151913427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 msgr2=0x7f4b541355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:02.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b537fe640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 0x7f4b541355e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4b5406f280 tx=0x7f4b44004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 1 -- 10.20.193.78:0/3151913427 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4401a070 con 0x7f4b400054c0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: dump: 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: _finish_auth 0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: _check_auth_tickets 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 1 -- 10.20.193.78:0/3151913427 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b44012d50 con 0x7f4b400054c0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 1 -- 10.20.193.78:0/3151913427 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44016660 con 0x7f4b400054c0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 10 monclient: dump: 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b517fa640 4 set_mon_vals no callback set 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b594ca640 5 monclient: authenticate success, global_id 244098 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b594ca640 10 monclient: _renew_subs 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b594ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5406f7c0 con 0x7f4b400054c0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b594ca640 10 monclient: _renew_subs 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b594ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b54124a80 con 0x7f4b400054c0 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.692+0000 7f4b337fe640 1 -- 10.20.193.78:0/3151913427 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4b44016800 con 0x7f4b400054c0 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.693+0000 7f4b594ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.693+0000 7f4b594ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.693+0000 7f4b594ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.693+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b54074980 con 0x7f4b400054c0 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.693+0000 7f4b337fe640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4b2003deb0 0x7f4b20040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.693+0000 7f4b337fe640 1 -- 10.20.193.78:0/3151913427 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(697..697 src has 525..697) ==== 4811+0+0 (secure 0 0 0) 0x7f4b440535c0 con 0x7f4b400054c0 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.693+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4b2003deb0 0x7f4b20040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.693+0000 7f4b52ffd640 10 monclient: get_auth_request con 0x7f4b2003deb0 auth_method 0 2026-02-17T06:06:02.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.696+0000 7f4b337fe640 1 -- 10.20.193.78:0/3151913427 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4b44010050 con 0x7f4b400054c0 2026-02-17T06:06:02.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.696+0000 7f4b337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:02.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.696+0000 7f4b337fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:02.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.696+0000 7f4b52ffd640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4b2003deb0 0x7f4b20040370 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4b48006fb0 tx=0x7f4b48032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.810+0000 7f4b594ca640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.810+0000 7f4b594ca640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.810+0000 7f4b594ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.810+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4b5406cc70 con 0x7f4b400054c0 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.810+0000 7f4b337fe640 1 -- 10.20.193.78:0/3151913427 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f4b44051fa0 con 0x7f4b400054c0 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.810+0000 7f4b337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.810+0000 7f4b337fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":232,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4b2003deb0 msgr2=0x7f4b20040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4b2003deb0 0x7f4b20040370 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4b48006fb0 tx=0x7f4b48032000 comp rx=0 tx=0).stop 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 10 monclient: shutdown 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:02.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 msgr2=0x7f4b541355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 0x7f4b541355e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4b5406f280 tx=0x7f4b44004a80 comp rx=0 tx=0).stop 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 shutdown_connections 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4b400054c0 0x7f4b541355e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4b40004900 0x7f4b541350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4b2003deb0 0x7f4b20040370 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 --2- 10.20.193.78:0/3151913427 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4b40003b50 0x7f4b541cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 >> 10.20.193.78:0/3151913427 conn(0x7f4b540a6620 msgr2=0x7f4b540a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 shutdown_connections 2026-02-17T06:06:02.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.811+0000 7f4b594ca640 1 -- 10.20.193.78:0/3151913427 wait complete. 2026-02-17T06:06:02.820 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:06:02.820 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:06:02.820 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:06:02.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.887+0000 7f48d00e1640 10 monclient: get_monmap_and_config 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.887+0000 7f48d00e1640 10 monclient: build_initial_monmap 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.887+0000 7f48d00e1640 10 monclient: monmap: 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:02.888882+0000 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:02.888882+0000 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:02.889 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:02.890 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:02.890 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 1 Processor -- start 2026-02-17T06:06:02.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 1 -- start start 2026-02-17T06:06:02.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient: init 2026-02-17T06:06:02.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:02.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T06:06:02.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c806c890 0x7f48c80a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient(hunting): picked mon.noname-b con 0x7f48c806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:02.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f48c806cc70 0x7f48c806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient(hunting): picked mon.noname-a con 0x7f48c806cc70 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f48c8134df0 0x7f48c8135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient(hunting): picked mon.noname-c con 0x7f48c8134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f48c8074150 con 0x7f48c806cc70 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f48c809ef70 con 0x7f48c8134df0 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f48c809f250 con 0x7f48c806c890 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48d00e1640 10 monclient(hunting): authenticate will time out at 1808.442563s 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c806c890 0x7f48c80a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c806c890 0x7f48c80a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:46936/0 (socket says 10.20.193.78:46936) 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 1 -- 10.20.193.78:0/2740486475 learned_addr learned my addr 10.20.193.78:0/2740486475 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f48c806cc70 0x7f48c806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 10 monclient(hunting): get_auth_request con 0x7f48c806c890 auth_method 0 2026-02-17T06:06:02.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient(hunting): get_auth_request con 0x7f48c806cc70 auth_method 0 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48ce657640 1 --2- 10.20.193.78:0/2740486475 >> v2:10.20.193.115:3300/0 conn(0x7f48c8134df0 0x7f48c8135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48ce657640 10 monclient(hunting): get_auth_request con 0x7f48c8134df0 auth_method 0 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48ce657640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48ce657640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48ce657640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient(hunting): handle_auth_done global_id 244101 payload 274 2026-02-17T06:06:02.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 1 -- 10.20.193.78:0/2740486475 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c806c890 msgr2=0x7f48c80a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 1 --2- 10.20.193.78:0/2740486475 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c806c890 0x7f48c80a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 1 -- 10.20.193.78:0/2740486475 >> v2:10.20.193.115:3300/0 conn(0x7f48c8134df0 msgr2=0x7f48c8135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 1 --2- 10.20.193.78:0/2740486475 >> v2:10.20.193.115:3300/0 conn(0x7f48c8134df0 0x7f48c8135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient: _finish_hunting 0 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 1 monclient: found mon.noname-a 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.888+0000 7f48cd655640 1 -- 10.20.193.78:0/2740486475 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c809df40 con 0x7f48c806cc70 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cd655640 1 -- 10.20.193.78:0/2740486475 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 msgr2=0x7f48c806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cd655640 1 --2- 10.20.193.78:0/2740486475 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c806d050 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f48c806e270 tx=0x7f48bc02eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30f5e939fd995ff server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 1 -- 10.20.193.78:0/2740486475 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc03c070 con 0x7f48c806cc70 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: dump: 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.898 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: _finish_auth 0 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: _check_auth_tickets 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 1 -- 10.20.193.78:0/2740486475 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc02fcf0 con 0x7f48c806cc70 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 5 monclient: authenticate success, global_id 244101 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 1 -- 10.20.193.78:0/2740486475 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc037660 con 0x7f48c806cc70 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 4 set_mon_vals no callback set 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48cce54640 10 monclient: dump: 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 10 monclient: shutdown 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 -- 10.20.193.78:0/2740486475 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 msgr2=0x7f48c806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 --2- 10.20.193.78:0/2740486475 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c806d050 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f48c806e270 tx=0x7f48bc02eeb0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 -- 10.20.193.78:0/2740486475 shutdown_connections 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 --2- 10.20.193.78:0/2740486475 >> v2:10.20.193.115:3300/0 conn(0x7f48c8134df0 0x7f48c8135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 --2- 10.20.193.78:0/2740486475 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c806d050 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 --2- 10.20.193.78:0/2740486475 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c806c890 0x7f48c80a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 -- 10.20.193.78:0/2740486475 >> 10.20.193.78:0/2740486475 conn(0x7f48c80a6620 msgr2=0x7f48c80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 -- 10.20.193.78:0/2740486475 shutdown_connections 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.891+0000 7f48d00e1640 1 -- 10.20.193.78:0/2740486475 wait complete. 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient: build_initial_monmap 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient: monmap: 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 1 Processor -- start 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 1 -- start start 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient: init 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48c806c890 0x7f48c8133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient(hunting): picked mon.noname-b con 0x7f48c806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c8134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient(hunting): picked mon.noname-a con 0x7f48c806cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c8134df0 0x7f48c811cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient(hunting): picked mon.noname-c con 0x7f48c8134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48c809d360 con 0x7f48c806cc70 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48c809d1e0 con 0x7f48c806c890 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f48c8135780 con 0x7f48c8134df0 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48d00e1640 10 monclient(hunting): authenticate will time out at 1808.446724s 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c8134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c8134df0 0x7f48c811cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c8134400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51814/0 (socket says 10.20.193.78:51814) 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 -- 10.20.193.78:0/265195714 learned_addr learned my addr 10.20.193.78:0/265195714 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient(hunting): get_auth_request con 0x7f48c806cc70 auth_method 0 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 10 monclient(hunting): get_auth_request con 0x7f48c8134df0 auth_method 0 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:02.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient(hunting): handle_auth_done global_id 244104 payload 824 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c8134df0 msgr2=0x7f48c811cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48c806c890 0x7f48c8133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c8134df0 0x7f48c811cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48c806c890 msgr2=0x7f48c8133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48c806c890 0x7f48c8133ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient: _finish_hunting 0 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 monclient: found mon.noname-a 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cd655640 1 -- 10.20.193.78:0/265195714 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c80a2800 con 0x7f48c806cc70 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cde56640 10 monclient: get_auth_request con 0x7f48c806c890 auth_method 0 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48ce657640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48c8134df0 0x7f48c811cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:06:02.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.892+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48c806c890 0x7f48c8133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:06:02.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cd655640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 msgr2=0x7f48c8134400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:02.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cd655640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c8134400 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f48bc002410 tx=0x7f48bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:02.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc03c040 con 0x7f48c806cc70 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: dump: 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: _finish_auth 0 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: _check_auth_tickets 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc047070 con 0x7f48c806cc70 2026-02-17T06:06:02.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 5 monclient: authenticate success, global_id 244104 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 10 monclient: _renew_subs 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c80a3800 con 0x7f48c806cc70 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc040a60 con 0x7f48c806cc70 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 10 monclient: dump: 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:02.908 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b7fff640 4 set_mon_vals no callback set 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 10 monclient: _renew_subs 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c8120590 con 0x7f48c806cc70 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 0x7f48c8139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48c809e740 con 0x7f48c81cf4b0 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 0x7f48c8139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 10 monclient: get_auth_request con 0x7f48c81cf4b0 auth_method 0 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f48bc03f030 con 0x7f48c806cc70 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 10 monclient(hunting): handle_auth_done global_id 244107 payload 824 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 1 -- 10.20.193.78:0/265195714 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48c8074980 con 0x7f48c81cf4b0 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 msgr2=0x7f48c8139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 msgr2=0x7f48c8139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 0x7f48c8139740 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f48b800f780 tx=0x7f48b800fc50 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.895+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 0x7f48c8139740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.896+0000 7f48b67fc640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f489803deb0 0x7f4898040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.896+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(697..697 src has 525..697) ==== 4811+0+0 (secure 0 0 0) 0x7f48bc077a70 con 0x7f48c806cc70 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.896+0000 7f48ce657640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f489803deb0 0x7f4898040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.896+0000 7f48ce657640 10 monclient: get_auth_request con 0x7f489803deb0 auth_method 0 2026-02-17T06:06:02.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:02.898+0000 7f48ce657640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f489803deb0 0x7f4898040370 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009fd0 tx=0x7f48c4009770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 0x7f48c8139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 10 monclient: get_auth_request con 0x7f48c81cf4b0 auth_method 0 2026-02-17T06:06:03.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 10 monclient: _init_auth method 2 2026-02-17T06:06:03.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 10 monclient: _init_auth already have auth, reseting 2026-02-17T06:06:03.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 10 monclient: handle_auth_reply_more payload 9 2026-02-17T06:06:03.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 10 monclient: handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 10 monclient: handle_auth_reply_more responding with 132 bytes 2026-02-17T06:06:03.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.096+0000 7f48cde56640 10 monclient: handle_auth_done global_id 244107 payload 293 2026-02-17T06:06:03.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48cde56640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 msgr2=0x7f48c8139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 0x7f48c8139740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f48b800f780 tx=0x7f48b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8024070 con 0x7f48c81cf4b0 2026-02-17T06:06:03.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:03.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 10 monclient: dump: 2026-02-17T06:06:03.100 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.100 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f48b800fe70 con 0x7f48c81cf4b0 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:03.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:03.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 msgr2=0x7f48c8139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 0x7f48c8139740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f48b800f780 tx=0x7f48b8008040 comp rx=0 tx=0).stop 2026-02-17T06:06:03.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 msgr2=0x7f48c8139740 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:03.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.098+0000 7f48b67fc640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c81cf4b0 0x7f48c8139740 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f48b800f780 tx=0x7f48b8008040 comp rx=0 tx=0).stop 2026-02-17T06:06:03.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.101+0000 7f48d00e1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:03.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.101+0000 7f48d00e1640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c8139c80 0x7f48c81da590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.101+0000 7f48d00e1640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.101+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48c809d810 con 0x7f48c8139c80 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c8139c80 0x7f48c81da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 10 monclient: get_auth_request con 0x7f48c8139c80 auth_method 0 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 10 monclient(hunting): handle_auth_done global_id 244110 payload 824 2026-02-17T06:06:03.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.102+0000 7f48cde56640 1 -- 10.20.193.78:0/265195714 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f48c8135be0 con 0x7f48c8139c80 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48cde56640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c8139c80 msgr2=0x7f48c81da590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48cde56640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c8139c80 0x7f48c81da590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f48b800f780 tx=0x7f48b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8024050 con 0x7f48c8139c80 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 10 monclient: dump: 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.106 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f48b8025040 con 0x7f48c8139c80 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:03.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c8139c80 msgr2=0x7f48c81da590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c8139c80 0x7f48c81da590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f48b800f780 tx=0x7f48b80047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c8139c80 msgr2=0x7f48c81da590 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:03.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48b67fc640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c8139c80 0x7f48c81da590 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f48b800f780 tx=0x7f48b80047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.108 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:03.108 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:03.108 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:03.108 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 232, 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 6653, 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:03.109 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.110 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.111 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:03.112 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:03.113 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.114 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:03.115 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:03.116 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.117 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:03.118 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.119 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:03.120 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.121 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:03.122 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:03.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f489803deb0 msgr2=0x7f4898040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f489803deb0 0x7f4898040370 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009fd0 tx=0x7f48c4009770 comp rx=0 tx=0).stop 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 10 monclient: shutdown 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 msgr2=0x7f48c8134400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c8134400 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f48bc002410 tx=0x7f48bc0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48ce657640 1 -- 10.20.193.78:0/265195714 reap_dead start 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 shutdown_connections 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 --2- 10.20.193.78:0/265195714 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48c806cc70 0x7f48c8134400 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 >> 10.20.193.78:0/265195714 conn(0x7f48c80a6620 msgr2=0x7f48c8138020 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 shutdown_connections 2026-02-17T06:06:03.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.104+0000 7f48d00e1640 1 -- 10.20.193.78:0/265195714 wait complete. 2026-02-17T06:06:03.126 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:03.126 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:06:03.127 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.194+0000 7f73be838640 10 monclient: get_monmap_and_config 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.194+0000 7f73be838640 10 monclient: build_initial_monmap 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.194+0000 7f73be838640 10 monclient: monmap: 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:03.195590+0000 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:03.195590+0000 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:03.195 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.194+0000 7f73be838640 1 Processor -- start 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.194+0000 7f73be838640 1 -- start start 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.194+0000 7f73be838640 10 monclient: init 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b806c890 0x7f73b806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient(hunting): picked mon.noname-b con 0x7f73b806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f73b809c4a0 0x7f73b806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient(hunting): picked mon.noname-c con 0x7f73b809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f73b8134df0 0x7f73b8135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient(hunting): picked mon.noname-a con 0x7f73b8134df0 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f73b80a0880 con 0x7f73b8134df0 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f73b806d6f0 con 0x7f73b809c4a0 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f73b806daf0 con 0x7f73b806c890 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73be838640 10 monclient(hunting): authenticate will time out at 1808.749156s 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f73b8134df0 0x7f73b8135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73b7fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b806c890 0x7f73b806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f73b8134df0 0x7f73b8135240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51836/0 (socket says 10.20.193.78:51836) 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 -- 10.20.193.78:0/2591153276 learned_addr learned my addr 10.20.193.78:0/2591153276 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient(hunting): get_auth_request con 0x7f73b8134df0 auth_method 0 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73b7fff640 10 monclient(hunting): get_auth_request con 0x7f73b806c890 auth_method 0 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient(hunting): handle_auth_done global_id 244113 payload 274 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 -- 10.20.193.78:0/2591153276 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b806c890 msgr2=0x7f73b806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 --2- 10.20.193.78:0/2591153276 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b806c890 0x7f73b806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 -- 10.20.193.78:0/2591153276 >> v2:10.20.193.115:3300/0 conn(0x7f73b809c4a0 msgr2=0x7f73b806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 --2- 10.20.193.78:0/2591153276 >> v2:10.20.193.115:3300/0 conn(0x7f73b809c4a0 0x7f73b806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient: _finish_hunting 0 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 monclient: found mon.noname-a 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:03.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.195+0000 7f73bcdae640 1 -- 10.20.193.78:0/2591153276 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b809cc80 con 0x7f73b8134df0 2026-02-17T06:06:03.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.197+0000 7f73bcdae640 1 -- 10.20.193.78:0/2591153276 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b8134df0 msgr2=0x7f73b8135240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.197+0000 7f73bcdae640 1 --2- 10.20.193.78:0/2591153276 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b8134df0 0x7f73b8135240 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73b80a7600 tx=0x7f73ac02f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c96c100888c905 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 1 -- 10.20.193.78:0/2591153276 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac02fe60 con 0x7f73b8134df0 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: dump: 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 1 -- 10.20.193.78:0/2591153276 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ac0373f0 con 0x7f73b8134df0 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 5 monclient: authenticate success, global_id 244113 2026-02-17T06:06:03.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 1 -- 10.20.193.78:0/2591153276 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0377c0 con 0x7f73b8134df0 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 4 set_mon_vals no callback set 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73b6ffd640 10 monclient: dump: 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 10 monclient: shutdown 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 -- 10.20.193.78:0/2591153276 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b8134df0 msgr2=0x7f73b8135240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 --2- 10.20.193.78:0/2591153276 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b8134df0 0x7f73b8135240 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73b80a7600 tx=0x7f73ac02f230 comp rx=0 tx=0).stop 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 -- 10.20.193.78:0/2591153276 shutdown_connections 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 --2- 10.20.193.78:0/2591153276 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b8134df0 0x7f73b8135240 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 --2- 10.20.193.78:0/2591153276 >> v2:10.20.193.115:3300/0 conn(0x7f73b809c4a0 0x7f73b806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 --2- 10.20.193.78:0/2591153276 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b806c890 0x7f73b806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 -- 10.20.193.78:0/2591153276 >> 10.20.193.78:0/2591153276 conn(0x7f73b80a6620 msgr2=0x7f73b80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 -- 10.20.193.78:0/2591153276 shutdown_connections 2026-02-17T06:06:03.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 1 -- 10.20.193.78:0/2591153276 wait complete. 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 10 monclient: build_initial_monmap 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.198+0000 7f73be838640 10 monclient: monmap: 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 1 Processor -- start 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 1 -- start start 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient: init 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b806c890 0x7f73b81cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient(hunting): picked mon.noname-a con 0x7f73b806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b809c4a0 0x7f73b81cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient(hunting): picked mon.noname-c con 0x7f73b809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:03.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73b8134df0 0x7f73b8121710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient(hunting): picked mon.noname-b con 0x7f73b8134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80a01a0 con 0x7f73b806c890 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80a0020 con 0x7f73b8134df0 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f73b809be20 con 0x7f73b809c4a0 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73be838640 10 monclient(hunting): authenticate will time out at 1808.753571s 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b806c890 0x7f73b81cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b806c890 0x7f73b81cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51844/0 (socket says 10.20.193.78:51844) 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 -- 10.20.193.78:0/2032109177 learned_addr learned my addr 10.20.193.78:0/2032109177 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient(hunting): get_auth_request con 0x7f73b806c890 auth_method 0 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b77fe640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b809c4a0 0x7f73b81cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b77fe640 10 monclient(hunting): get_auth_request con 0x7f73b809c4a0 auth_method 0 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient(hunting): handle_auth_done global_id 244116 payload 824 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b809c4a0 msgr2=0x7f73b81cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b809c4a0 0x7f73b81cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73b8134df0 msgr2=0x7f73b8121710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f73b8134df0 0x7f73b8121710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient: _finish_hunting 0 2026-02-17T06:06:03.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 monclient: found mon.noname-a 2026-02-17T06:06:03.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:03.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.199+0000 7f73b7fff640 1 -- 10.20.193.78:0/2032109177 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b8125090 con 0x7f73b806c890 2026-02-17T06:06:03.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b7fff640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b806c890 msgr2=0x7f73b81cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b7fff640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b806c890 0x7f73b81cd330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e890 tx=0x7f73a400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4017070 con 0x7f73b806c890 2026-02-17T06:06:03.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:03.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 10 monclient: dump: 2026-02-17T06:06:03.205 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.205 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 10 monclient: _finish_auth 0 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a400cad0 con 0x7f73b806c890 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73be838640 5 monclient: authenticate success, global_id 244116 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73be838640 10 monclient: _renew_subs 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73be838640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b8123800 con 0x7f73b806c890 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4013470 con 0x7f73b806c890 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.201+0000 7f73b67fc640 4 set_mon_vals no callback set 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b4ff9640 10 monclient: dump: 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.206 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73be838640 10 monclient: _renew_subs 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73be838640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b8125370 con 0x7f73b806c890 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73be838640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73be838640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8121c50 0x7f73b8122030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73be838640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f73b8074980 con 0x7f73b8121c50 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8121c50 0x7f73b8122030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 10 monclient: get_auth_request con 0x7f73b8121c50 auth_method 0 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f73a4024060 con 0x7f73b806c890 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 10 monclient(hunting): handle_auth_done global_id 234206 payload 824 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b77fe640 1 -- 10.20.193.78:0/2032109177 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73b8072760 con 0x7f73b8121c50 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b4ff9640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f738803deb0 0x7f7388040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(697..697 src has 525..697) ==== 4811+0+0 (secure 0 0 0) 0x7f73a40527c0 con 0x7f73b806c890 2026-02-17T06:06:03.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73bcdae640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f738803deb0 0x7f7388040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.202+0000 7f73bcdae640 10 monclient: get_auth_request con 0x7f738803deb0 auth_method 0 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b77fe640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8121c50 msgr2=0x7f73b8122030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b77fe640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8121c50 0x7f73b8122030 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73a80096d0 tx=0x7f73a8009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8016070 con 0x7f73b8121c50 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b4ff9640 10 monclient: dump: 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f73a800ea30 con 0x7f73b8121c50 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.204+0000 7f73b4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.205+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8121c50 msgr2=0x7f73b8122030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.205+0000 7f73b4ff9640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8121c50 0x7f73b8122030 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73a80096d0 tx=0x7f73a8009ba0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.205+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8121c50 msgr2=0x7f73b8122030 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:03.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.205+0000 7f73b4ff9640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8121c50 0x7f73b8122030 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.205+0000 7f73bcdae640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f738803deb0 0x7f7388040370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f73b809c880 tx=0x7f73ac031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.207+0000 7f73be838640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.207+0000 7f73be838640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8122570 0x7f73b81d8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.207+0000 7f73be838640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.207+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f73b8135240 con 0x7f73b8122570 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.207+0000 7f73b77fe640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8122570 0x7f73b81d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 10 monclient: get_auth_request con 0x7f73b8122570 auth_method 0 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 10 monclient(hunting): handle_auth_done global_id 234209 payload 824 2026-02-17T06:06:03.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.208+0000 7f73b77fe640 1 -- 10.20.193.78:0/2032109177 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f73b80744d0 con 0x7f73b8122570 2026-02-17T06:06:03.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b77fe640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8122570 msgr2=0x7f73b81d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b77fe640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8122570 0x7f73b81d8540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009fd0 tx=0x7f73a80048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8016040 con 0x7f73b8122570 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 10 monclient: dump: 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.211 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f73a801de40 con 0x7f73b8122570 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8122570 msgr2=0x7f73b81d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8122570 0x7f73b81d8540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009fd0 tx=0x7f73a80048d0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8122570 msgr2=0x7f73b81d8540 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73b4ff9640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f73b8122570 0x7f73b81d8540 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009fd0 tx=0x7f73a80048d0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 232, 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 71893, 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.212 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.213 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:03.214 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.215 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.216 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.217 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.218 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.219 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:03.219 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.219 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.219 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.219 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:03.219 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f738803deb0 msgr2=0x7f7388040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f738803deb0 0x7f7388040370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f73b809c880 tx=0x7f73ac031000 comp rx=0 tx=0).stop 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 10 monclient: shutdown 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b806c890 msgr2=0x7f73b81cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b806c890 0x7f73b81cd330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e890 tx=0x7f73a400ed60 comp rx=0 tx=0).stop 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73bcdae640 1 -- 10.20.193.78:0/2032109177 reap_dead start 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 shutdown_connections 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 --2- 10.20.193.78:0/2032109177 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f73b806c890 0x7f73b81cd330 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 >> 10.20.193.78:0/2032109177 conn(0x7f73b80a6620 msgr2=0x7f73b8135810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 shutdown_connections 2026-02-17T06:06:03.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.210+0000 7f73be838640 1 -- 10.20.193.78:0/2032109177 wait complete. 2026-02-17T06:06:03.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:03.220 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:06:03.220 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient: get_monmap_and_config 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient: build_initial_monmap 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient: monmap: 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:03.288467+0000 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:03.288467+0000 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:03.288 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 1 Processor -- start 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 1 -- start start 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient: init 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4f1006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f4f1006d380 0x7f4f100a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient(hunting): picked mon.noname-c con 0x7f4f1006d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.287+0000 7f4f155bc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4f1013a240 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f155bc640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f10074150 con 0x7f4f1013a240 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f155bc640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f1009ef70 con 0x7f4f1006c890 2026-02-17T06:06:03.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f155bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f155bc640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f4f1009f250 con 0x7f4f1006d380 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f155bc640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f155bc640 10 monclient(hunting): authenticate will time out at 1808.842021s 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f0f7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f4f1013a240 0x7f4f1013c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38090/0 (socket says 10.20.193.78:38090) 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f0f7fe640 1 -- 10.20.193.78:0/3864887102 learned_addr learned my addr 10.20.193.78:0/3864887102 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f0e7fc640 1 --2- 10.20.193.78:0/3864887102 >> v2:10.20.193.115:3300/0 conn(0x7f4f1006d380 0x7f4f100a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f0e7fc640 10 monclient(hunting): get_auth_request con 0x7f4f1006d380 auth_method 0 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f0e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f0e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.288+0000 7f4f0e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0f7fe640 1 -- 10.20.193.78:0/3864887102 >> v1:10.20.193.78:6789/0 conn(0x7f4f1013a240 legacy=0x7f4f1013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0effd640 1 -- 10.20.193.78:0/3864887102 >> v1:10.20.193.115:6789/0 conn(0x7f4f1006c890 legacy=0x7f4f1006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0dffb640 1 -- 10.20.193.78:0/3864887102 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839397284 0 0) 0x7f4f10074150 con 0x7f4f1013a240 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0dffb640 10 monclient(hunting): _negotiate 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0dffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0dffb640 10 monclient(hunting): my global_id is 244119 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0dffb640 1 -- 10.20.193.78:0/3864887102 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4efc003620 con 0x7f4f1013a240 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_done global_id 214297 payload 274 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 1 -- 10.20.193.78:0/3864887102 >> v1:10.20.193.115:6789/0 conn(0x7f4f1006c890 legacy=0x7f4f1006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 1 -- 10.20.193.78:0/3864887102 >> v1:10.20.193.78:6789/0 conn(0x7f4f1013a240 legacy=0x7f4f1013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 1 monclient: found mon.noname-c 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:06:03.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.290+0000 7f4f0e7fc640 1 -- 10.20.193.78:0/3864887102 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1009df40 con 0x7f4f1006d380 2026-02-17T06:06:03.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0e7fc640 1 -- 10.20.193.78:0/3864887102 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1006d380 msgr2=0x7f4f100a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0e7fc640 1 --2- 10.20.193.78:0/3864887102 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1006d380 0x7f4f100a01f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80099a0 tx=0x7f4ef802efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ad2c6235c90c2a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 1 -- 10.20.193.78:0/3864887102 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef803c070 con 0x7f4f1006d380 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: dump: 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.294 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 1 -- 10.20.193.78:0/3864887102 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef8037440 con 0x7f4f1006d380 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 5 monclient: authenticate success, global_id 214297 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 1 -- 10.20.193.78:0/3864887102 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8037810 con 0x7f4f1006d380 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 4 set_mon_vals no callback set 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f0dffb640 10 monclient: dump: 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 10 monclient: shutdown 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 1 -- 10.20.193.78:0/3864887102 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1006d380 msgr2=0x7f4f100a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 1 --2- 10.20.193.78:0/3864887102 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1006d380 0x7f4f100a01f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80099a0 tx=0x7f4ef802efa0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 1 -- 10.20.193.78:0/3864887102 shutdown_connections 2026-02-17T06:06:03.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 1 --2- 10.20.193.78:0/3864887102 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1006d380 0x7f4f100a01f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 1 -- 10.20.193.78:0/3864887102 >> 10.20.193.78:0/3864887102 conn(0x7f4f100a6620 msgr2=0x7f4f100a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:03.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 1 -- 10.20.193.78:0/3864887102 shutdown_connections 2026-02-17T06:06:03.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.293+0000 7f4f155bc640 1 -- 10.20.193.78:0/3864887102 wait complete. 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient: build_initial_monmap 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient: monmap: 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 1 Processor -- start 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 1 -- start start 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient: init 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:03.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f1006c890 0x7f4f101ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient(hunting): picked mon.noname-a con 0x7f4f1006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f1006d380 0x7f4f101cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient(hunting): picked mon.noname-c con 0x7f4f1006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1013a240 0x7f4f101cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient(hunting): picked mon.noname-b con 0x7f4f1013a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1013ded0 con 0x7f4f1006c890 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1013dd50 con 0x7f4f1013a240 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1013e050 con 0x7f4f1006d380 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f155bc640 10 monclient(hunting): authenticate will time out at 1808.848674s 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f1006c890 0x7f4f101ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f1006c890 0x7f4f101ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51860/0 (socket says 10.20.193.78:51860) 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0e7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f1006d380 0x7f4f101cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 -- 10.20.193.78:0/1904548242 learned_addr learned my addr 10.20.193.78:0/1904548242 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient(hunting): get_auth_request con 0x7f4f1006c890 auth_method 0 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0e7fc640 10 monclient(hunting): get_auth_request con 0x7f4f1006d380 auth_method 0 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient(hunting): handle_auth_done global_id 244122 payload 824 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f1006d380 msgr2=0x7f4f101cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f1006d380 0x7f4f101cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1013a240 msgr2=0x7f4f101cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1013a240 0x7f4f101cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient: _finish_hunting 0 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 monclient: found mon.noname-a 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.294+0000 7f4f0effd640 1 -- 10.20.193.78:0/1904548242 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1009d810 con 0x7f4f1006c890 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0effd640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f1006c890 msgr2=0x7f4f101ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0effd640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f1006c890 0x7f4f101ccf10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c730 tx=0x7f4f0400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04017070 con 0x7f4f1006c890 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: dump: 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: _finish_auth 0 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f04005cd0 con 0x7f4f1006c890 2026-02-17T06:06:03.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 5 monclient: authenticate success, global_id 244122 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 10 monclient: _renew_subs 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1011f4d0 con 0x7f4f1006c890 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04007290 con 0x7f4f1006c890 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0d7fa640 4 set_mon_vals no callback set 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 10 monclient: dump: 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 10 monclient: _renew_subs 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f100a3d60 con 0x7f4f1006c890 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1011e300 0x7f4f10134990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f100a01d0 con 0x7f4f1011e300 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4f04007470 con 0x7f4f1006c890 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0e7fc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1011e300 0x7f4f10134990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0e7fc640 10 monclient: get_auth_request con 0x7f4f1011e300 auth_method 0 2026-02-17T06:06:03.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4eef7fe640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4edc03deb0 0x7f4edc040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.297+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.298+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.298+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(697..697 src has 525..697) ==== 4811+0+0 (secure 0 0 0) 0x7f4f04051ec0 con 0x7f4f1006c890 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.298+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_done global_id 214300 payload 824 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.298+0000 7f4f0e7fc640 1 -- 10.20.193.78:0/1904548242 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f100008d0 con 0x7f4f1011e300 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.298+0000 7f4f0f7fe640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4edc03deb0 0x7f4edc040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.298+0000 7f4f0f7fe640 10 monclient: get_auth_request con 0x7f4edc03deb0 auth_method 0 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4f0e7fc640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1011e300 msgr2=0x7f4f10134990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4f0e7fc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1011e300 0x7f4f10134990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8039000 tx=0x7f4ef80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef803c040 con 0x7f4f1011e300 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 10 monclient: dump: 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.303 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4ef8037e20 con 0x7f4f1011e300 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1011e300 msgr2=0x7f4f10134990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1011e300 0x7f4f10134990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8039000 tx=0x7f4ef80047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1011e300 msgr2=0x7f4f10134990 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.300+0000 7f4eef7fe640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f1011e300 0x7f4f10134990 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8039000 tx=0x7f4ef80047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.301+0000 7f4f0f7fe640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4edc03deb0 0x7f4edc040370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f00003030 tx=0x7f4f000032b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f155bc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:03.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f155bc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f10134ed0 0x7f4f101d8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f155bc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10072e70 con 0x7f4f10134ed0 2026-02-17T06:06:03.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f10134ed0 0x7f4f101d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 10 monclient: get_auth_request con 0x7f4f10134ed0 auth_method 0 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 10 monclient(hunting): handle_auth_done global_id 214303 payload 824 2026-02-17T06:06:03.306 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.304+0000 7f4f0e7fc640 1 -- 10.20.193.78:0/1904548242 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4f101352b0 con 0x7f4f10134ed0 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f0e7fc640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f10134ed0 msgr2=0x7f4f101d8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f0e7fc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f10134ed0 0x7f4f101d8470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4ef802efe0 tx=0x7f4ef8008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef803c040 con 0x7f4f10134ed0 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 10 monclient: dump: 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.308 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f4ef804dd20 con 0x7f4f10134ed0 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f10134ed0 msgr2=0x7f4f101d8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f10134ed0 0x7f4f101d8470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4ef802efe0 tx=0x7f4ef8008030 comp rx=0 tx=0).stop 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f10134ed0 msgr2=0x7f4f101d8470 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4eef7fe640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f10134ed0 0x7f4f101d8470 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4ef802efe0 tx=0x7f4ef8008030 comp rx=0 tx=0).stop 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:03.309 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 232, 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 166154, 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.310 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.311 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:03.312 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:03.313 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:03.314 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.315 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.316 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.317 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:03.318 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:03.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4edc03deb0 msgr2=0x7f4edc040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4edc03deb0 0x7f4edc040370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f00003030 tx=0x7f4f000032b0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 10 monclient: shutdown 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f1006c890 msgr2=0x7f4f101ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f0f7fe640 1 -- 10.20.193.78:0/1904548242 reap_dead start 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f1006c890 0x7f4f101ccf10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c730 tx=0x7f4f0400cc00 comp rx=0 tx=0).stop 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 shutdown_connections 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 --2- 10.20.193.78:0/1904548242 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f1006c890 0x7f4f101ccf10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 >> 10.20.193.78:0/1904548242 conn(0x7f4f100a6620 msgr2=0x7f4f1009c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 shutdown_connections 2026-02-17T06:06:03.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.307+0000 7f4f155bc640 1 -- 10.20.193.78:0/1904548242 wait complete. 2026-02-17T06:06:03.320 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:03.320 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:06:03.321 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:06:03.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.385+0000 7fa4b8da0640 10 monclient: get_monmap_and_config 2026-02-17T06:06:03.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.385+0000 7fa4b8da0640 10 monclient: build_initial_monmap 2026-02-17T06:06:03.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.385+0000 7fa4b8da0640 10 monclient: monmap: 2026-02-17T06:06:03.388 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:03.388 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.388 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:03.386766+0000 2026-02-17T06:06:03.388 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:03.386766+0000 2026-02-17T06:06:03.388 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.385+0000 7fa4b8da0640 1 Processor -- start 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 1 -- start start 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient: init 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa4b406c890 0x7fa4b406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient(hunting): picked mon.noname-c con 0x7fa4b406c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa4b409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa4b406d1b0 0x7fa4b4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient(hunting): picked mon.noname-a con 0x7fa4b406d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:03.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b406daf0 con 0x7fa4b409c4a0 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b40a0880 con 0x7fa4b406d1b0 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b409cc80 con 0x7fa4b406c890 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b8da0640 10 monclient(hunting): authenticate will time out at 1808.940338s 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa4b406d1b0 0x7fa4b4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b1d74640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fa4b409c4a0 0x7fa4b4134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38100/0 (socket says 10.20.193.78:38100) 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa4b406d1b0 0x7fa4b4135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51876/0 (socket says 10.20.193.78:51876) 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b1d74640 1 -- 10.20.193.78:0/4002496749 learned_addr learned my addr 10.20.193.78:0/4002496749 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient(hunting): get_auth_request con 0x7fa4b406d1b0 auth_method 0 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2575640 1 --2- 10.20.193.78:0/4002496749 >> v2:10.20.193.115:3300/0 conn(0x7fa4b406c890 0x7fa4b406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2575640 10 monclient(hunting): get_auth_request con 0x7fa4b406c890 auth_method 0 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient(hunting): handle_auth_done global_id 244125 payload 274 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 1 -- 10.20.193.78:0/4002496749 >> v2:10.20.193.115:3300/0 conn(0x7fa4b406c890 msgr2=0x7fa4b406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 1 --2- 10.20.193.78:0/4002496749 >> v2:10.20.193.115:3300/0 conn(0x7fa4b406c890 0x7fa4b406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 1 -- 10.20.193.78:0/4002496749 >> v1:10.20.193.78:6789/0 conn(0x7fa4b409c4a0 legacy=0x7fa4b4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient: _finish_hunting 0 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 1 monclient: found mon.noname-a 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.386+0000 7fa4b2d76640 1 -- 10.20.193.78:0/4002496749 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b409cf60 con 0x7fa4b406d1b0 2026-02-17T06:06:03.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b2d76640 1 -- 10.20.193.78:0/4002496749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b406d1b0 msgr2=0x7fa4b4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b2d76640 1 --2- 10.20.193.78:0/4002496749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b406d1b0 0x7fa4b4135500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa4b406a480 tx=0x7fa4a402f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5be63c8f63395b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 1 -- 10.20.193.78:0/4002496749 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a403c070 con 0x7fa4b406d1b0 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: dump: 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: _finish_auth 0 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: _check_auth_tickets 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:03.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 1 -- 10.20.193.78:0/4002496749 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4037440 con 0x7fa4b406d1b0 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 5 monclient: authenticate success, global_id 244125 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 1 -- 10.20.193.78:0/4002496749 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4037810 con 0x7fa4b406d1b0 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 4 set_mon_vals no callback set 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b1573640 10 monclient: dump: 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 10 monclient: shutdown 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 1 -- 10.20.193.78:0/4002496749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b406d1b0 msgr2=0x7fa4b4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/4002496749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b406d1b0 0x7fa4b4135500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa4b406a480 tx=0x7fa4a402f130 comp rx=0 tx=0).stop 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 1 -- 10.20.193.78:0/4002496749 shutdown_connections 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/4002496749 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b406d1b0 0x7fa4b4135500 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/4002496749 >> v2:10.20.193.115:3300/0 conn(0x7fa4b406c890 0x7fa4b406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 1 -- 10.20.193.78:0/4002496749 >> 10.20.193.78:0/4002496749 conn(0x7fa4b40a6620 msgr2=0x7fa4b40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 1 -- 10.20.193.78:0/4002496749 shutdown_connections 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.389+0000 7fa4b8da0640 1 -- 10.20.193.78:0/4002496749 wait complete. 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient: build_initial_monmap 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient: monmap: 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:03.398 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 1 Processor -- start 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 1 -- start start 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient: init 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4b406c890 0x7fa4b41cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient(hunting): picked mon.noname-b con 0x7fa4b406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4b406d1b0 0x7fa4b41cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient(hunting): picked mon.noname-c con 0x7fa4b406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b409c4a0 0x7fa4b4123560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient(hunting): picked mon.noname-a con 0x7fa4b409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b40a01a0 con 0x7fa4b409c4a0 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b40a0020 con 0x7fa4b406c890 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b409be20 con 0x7fa4b406d1b0 2026-02-17T06:06:03.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b8da0640 10 monclient(hunting): authenticate will time out at 1808.944488s 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b409c4a0 0x7fa4b4123560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b409c4a0 0x7fa4b4123560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51888/0 (socket says 10.20.193.78:51888) 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 -- 10.20.193.78:0/598370991 learned_addr learned my addr 10.20.193.78:0/598370991 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient(hunting): get_auth_request con 0x7fa4b409c4a0 auth_method 0 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b1d74640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4b406d1b0 0x7fa4b41cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b1d74640 10 monclient(hunting): get_auth_request con 0x7fa4b406d1b0 auth_method 0 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:03.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient(hunting): handle_auth_done global_id 244128 payload 824 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 -- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4b406d1b0 msgr2=0x7fa4b41cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4b406d1b0 0x7fa4b41cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 -- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4b406c890 msgr2=0x7fa4b41cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4b406c890 0x7fa4b41cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient: _finish_hunting 0 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 monclient: found mon.noname-a 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.390+0000 7fa4b2d76640 1 -- 10.20.193.78:0/598370991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b4126eb0 con 0x7fa4b409c4a0 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa4b2d76640 1 -- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b409c4a0 msgr2=0x7fa4b4123560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa4b2d76640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b409c4a0 0x7fa4b4123560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa4a403a040 tx=0x7fa4a4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa49affd640 1 -- 10.20.193.78:0/598370991 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a403c050 con 0x7fa4b409c4a0 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa49affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa49affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa49affd640 10 monclient: dump: 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.401 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa49affd640 10 monclient: _finish_auth 0 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa49affd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa49affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa4b8da0640 5 monclient: authenticate success, global_id 244128 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 10 monclient: _renew_subs 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.392+0000 7fa49affd640 1 -- 10.20.193.78:0/598370991 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4048070 con 0x7fa4b409c4a0 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b4125c70 con 0x7fa4b409c4a0 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa49affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa49affd640 1 -- 10.20.193.78:0/598370991 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4041a50 con 0x7fa4b409c4a0 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa49affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b0d72640 4 set_mon_vals no callback set 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa49affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa49affd640 10 monclient: dump: 2026-02-17T06:06:03.402 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 10 monclient: _renew_subs 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b406fe70 con 0x7fa4b409c4a0 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b4074980 con 0x7fa4b409c4a0 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa49affd640 1 -- 10.20.193.78:0/598370991 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa4a4041bf0 con 0x7fa4b409c4a0 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.393+0000 7fa49affd640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa48003df00 0x7fa4800403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:03.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.394+0000 7fa49affd640 1 -- 10.20.193.78:0/598370991 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(697..697 src has 525..697) ==== 4811+0+0 (secure 0 0 0) 0x7fa4a40784e0 con 0x7fa4b409c4a0 2026-02-17T06:06:03.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.394+0000 7fa4b2575640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa48003df00 0x7fa4800403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:03.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.394+0000 7fa4b2575640 10 monclient: get_auth_request con 0x7fa48003df00 auth_method 0 2026-02-17T06:06:03.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.396+0000 7fa49affd640 1 -- 10.20.193.78:0/598370991 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa4a4052020 con 0x7fa4b409c4a0 2026-02-17T06:06:03.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.396+0000 7fa49affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:03.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.396+0000 7fa49affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:03.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.396+0000 7fa4b2575640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa48003df00 0x7fa4800403c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8006b90 tx=0x7fa4a8006b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:03.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.509+0000 7fa4b8da0640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:06:03.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.509+0000 7fa4b8da0640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:06:03.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.509+0000 7fa4b8da0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:03.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.509+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa4b4136ea0 con 0x7fa4b409c4a0 2026-02-17T06:06:03.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.509+0000 7fa49affd640 1 -- 10.20.193.78:0/598370991 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa4a4040d90 con 0x7fa4b409c4a0 2026-02-17T06:06:03.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.509+0000 7fa49affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:06:03.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.509+0000 7fa49affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:03.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa48003df00 msgr2=0x7fa4800403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa48003df00 0x7fa4800403c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8006b90 tx=0x7fa4a8006b20 comp rx=0 tx=0).stop 2026-02-17T06:06:03.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 10 monclient: shutdown 2026-02-17T06:06:03.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:03.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b409c4a0 msgr2=0x7fa4b4123560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:03.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b409c4a0 0x7fa4b4123560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa4a403a040 tx=0x7fa4a4004b40 comp rx=0 tx=0).stop 2026-02-17T06:06:03.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 shutdown_connections 2026-02-17T06:06:03.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa4b409c4a0 0x7fa4b4123560 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4b406d1b0 0x7fa4b41cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa48003df00 0x7fa4800403c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 --2- 10.20.193.78:0/598370991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa4b406c890 0x7fa4b41cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:03.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 >> 10.20.193.78:0/598370991 conn(0x7fa4b40a6620 msgr2=0x7fa4b4071280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:03.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 shutdown_connections 2026-02-17T06:06:03.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:03.510+0000 7fa4b8da0640 1 -- 10.20.193.78:0/598370991 wait complete. 2026-02-17T06:06:03.513 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.511+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.514 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.511+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=3374416129,config=692619927,health=3041450585,kv=1187752174,logm=198633943}) 2026-02-17T06:06:03.514 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.512+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=2768635760,config=692619927,health=3041450585,kv=1187752174,logm=1696318813}) 2026-02-17T06:06:03.514 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.512+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.514 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.512+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=3374416129,config=692619927,health=3041450585,kv=1187752174,logm=198633943}) 2026-02-17T06:06:03.514 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.512+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=27,config=2,health=12,kv=12,logm=47} crc {auth=2647979858,config=692619927,health=3041450585,kv=449989585,logm=987700966}) 2026-02-17T06:06:03.515 INFO:tasks.rados.rados.0.trial115.stdout:3247: done (0 left) 2026-02-17T06:06:03.515 INFO:tasks.rados.rados.0.trial115.stdout:3248: snap_create 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.513+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.513+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674861095}) 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.513+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4275672178}) 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.513+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.513+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674861095}) 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.513+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=213263213}) 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.514+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.514+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=387446834}) 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.514+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=541068672}) 2026-02-17T06:06:03.516 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.514+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.517 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.514+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=387446834}) 2026-02-17T06:06:03.517 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.514+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1781224266}) 2026-02-17T06:06:03.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:06:03.630 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.629+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.629+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=426195603}) 2026-02-17T06:06:03.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.629+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=490769101}) 2026-02-17T06:06:03.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.629+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.629+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=426195603}) 2026-02-17T06:06:03.631 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.629+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2858750072}) 2026-02-17T06:06:03.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1727665567}) 2026-02-17T06:06:03.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4010032150}) 2026-02-17T06:06:03.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1727665567}) 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1396829482}) 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613678779}) 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3631403308}) 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613678779}) 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4132653687}) 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399483530}) 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3617108494}) 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399483530}) 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.657+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4109978232}) 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1073990912}) 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1813075758}) 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1073990912}) 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276485063}) 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3540186423}) 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=152136633}) 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3540186423}) 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.658+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1907329331}) 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4064294453}) 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1966732000}) 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4064294453}) 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027349710}) 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1315606195}) 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663807501}) 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1315606195}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.659+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=233547040}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3394179329}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=837810627}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3394179329}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1506126561}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882779305}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4203012587}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882779305}) 2026-02-17T06:06:03.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.660+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1620263351}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2989150546}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4045812120}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2989150546}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1807204554}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545922479}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1147684338}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545922479}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.661+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682874982}) 2026-02-17T06:06:03.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3349809342}) 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=879151866}) 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3349809342}) 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1392542910}) 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065343626}) 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1862742126}) 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065343626}) 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1598215824}) 2026-02-17T06:06:03.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3762860660}) 2026-02-17T06:06:03.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.662+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3864299304}) 2026-02-17T06:06:03.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.663+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.663+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3762860660}) 2026-02-17T06:06:03.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.663+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1856264967}) 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.669+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.669+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1577626535}) 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.669+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2754034163}) 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.669+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.669+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1577626535}) 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.669+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=56938278}) 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4003211518}) 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2686046483}) 2026-02-17T06:06:03.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4003211518}) 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1806460675}) 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875264430}) 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2940797244}) 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875264430}) 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.670+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=927722368}) 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616665272}) 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1957355871}) 2026-02-17T06:06:03.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616665272}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1108452606}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2625284581}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3437608617}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2625284581}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.671+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3465081438}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933662573}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=271894414}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933662573}) 2026-02-17T06:06:03.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3697391568}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2012223748}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3696198377}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2012223748}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.672+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1438379561}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3717347061}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335720446}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3717347061}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3646565918}) 2026-02-17T06:06:03.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1347868068}) 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=648982722}) 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1347868068}) 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2491060060}) 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965580587}) 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4117246138}) 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965580587}) 2026-02-17T06:06:03.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.673+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1721135051}) 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3588439919}) 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1236262525}) 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3588439919}) 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=138288130}) 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1997027148}) 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3292145390}) 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.684 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1997027148}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=837627611}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=115079561}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4019849160}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=115079561}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.681+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1341086139}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735892879}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=290997967}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735892879}) 2026-02-17T06:06:03.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=741265790}) 2026-02-17T06:06:03.686 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.686 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2660959679}) 2026-02-17T06:06:03.686 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=726289636}) 2026-02-17T06:06:03.686 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:03.686 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2660959679}) 2026-02-17T06:06:03.686 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:03.682+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1115556497}) 2026-02-17T06:06:04.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3127429493}) 2026-02-17T06:06:04.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1480898422}) 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3127429493}) 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4086058152}) 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=967916232}) 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2280236802}) 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=967916232}) 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3299906912}) 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561380034}) 2026-02-17T06:06:04.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129889402}) 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561380034}) 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2269748339}) 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4091269621}) 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2381197075}) 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4091269621}) 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.196+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=19921641}) 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=768505949}) 2026-02-17T06:06:04.217 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391414746}) 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=768505949}) 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3192368733}) 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=270414152}) 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436969400}) 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=270414152}) 2026-02-17T06:06:04.218 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.197+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1197475268}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1458038512}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3484844936}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1458038512}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1423035450}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4117613325}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3402668571}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4117613325}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2551612294}) 2026-02-17T06:06:04.219 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3517267186}) 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987579922}) 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3517267186}) 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.198+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=345085149}) 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=312211852}) 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3086804223}) 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=312211852}) 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1531727698}) 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.220 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4084412333}) 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=345998368}) 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4084412333}) 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.199+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1735980921}) 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402974885}) 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=259062596}) 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402974885}) 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2556569900}) 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.221 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834639105}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360536839}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834639105}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.200+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3589884249}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=592723577}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=95249244}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=592723577}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=496708512}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899279085}) 2026-02-17T06:06:04.222 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=427846614}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899279085}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2788766819}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1020713934}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3014342623}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1020713934}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.201+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2816861510}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2849927775}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2257722012}) 2026-02-17T06:06:04.223 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2849927775}) 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824454576}) 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2738634301}) 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=323274685}) 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2738634301}) 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2207523327}) 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034931181}) 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2022539507}) 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.224 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034931181}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.202+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3510571416}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65061433}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=984172423}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65061433}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=862085284}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=773210560}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=208701933}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=773210560}) 2026-02-17T06:06:04.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4111110822}) 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3929330191}) 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3942298136}) 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3929330191}) 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.203+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=881686129}) 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1461180428}) 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1361801264}) 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1461180428}) 2026-02-17T06:06:04.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215808981}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947422010}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=8159980}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947422010}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2126163648}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2348416771}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3655829269}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2348416771}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.204+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=203577508}) 2026-02-17T06:06:04.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=50817470}) 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4186134320}) 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=50817470}) 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2971169294}) 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3683109397}) 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161719681}) 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3683109397}) 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1845097893}) 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494541683}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1559796299}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.205+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494541683}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=525162013}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3414017921}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4277752882}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3414017921}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1965685188}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743514269}) 2026-02-17T06:06:04.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4034592870}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743514269}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2171875669}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.206+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625982598}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4082525968}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625982598}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=359897002}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1887785437}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=822908643}) 2026-02-17T06:06:04.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1887785437}) 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1177485362}) 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2337738298}) 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3887450950}) 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2337738298}) 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.207+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2889698565}) 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4212807106}) 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=789890075}) 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4212807106}) 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=878113565}) 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1093082657}) 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=506230309}) 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1093082657}) 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057151164}) 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2221013037}) 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1196975805}) 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2221013037}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.208+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1586490691}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4292169514}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1307211858}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4292169514}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1308868247}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400788448}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3452688183}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400788448}) 2026-02-17T06:06:04.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1167188458}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186998281}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=942120032}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186998281}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.209+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2030269018}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=558709376}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=782221048}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=558709376}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2627948056}) 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3227409904}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3536916472}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3227409904}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=126413201}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950156937}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1942255404}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950156937}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.210+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4255675523}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361443400}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3598582838}) 2026-02-17T06:06:04.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361443400}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4134955414}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1972521333}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070099757}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1972521333}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773499935}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549957277}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1606425978}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549957277}) 2026-02-17T06:06:04.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.211+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448785208}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=52,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=3835684398,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgrstat=4214789694,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=294099063,osd_snap=3862306161}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=52,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=7} crc {logm=4273727961,mdsmap=1160443688,mgr=787438293,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=900291204,mon_config_key=4017421564,monmap=1784110042,osd_metadata=2240493509,osd_pg_creating=294099063,osd_snap=785252622}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=52,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=3835684398,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgrstat=4214789694,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=294099063,osd_snap=3862306161}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=51,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=6} crc {logm=827199205,mdsmap=1160443688,mgr=3003080605,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=900291204,mon_config_key=2399717749,monmap=1784110042,osd_metadata=1246192198,osd_pg_creating=294099063,osd_snap=3881713549}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=4071909312}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=100} crc {osd_snap=4089601960}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=4071909312}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=100} crc {osd_snap=3282456327}) 2026-02-17T06:06:04.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.212+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=17,osdmap=83} crc {osd_snap=2867606727,osdmap=2660419160}) 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=1547383146,osdmap=2387633975}) 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=17,osdmap=83} crc {osd_snap=2867606727,osdmap=2660419160}) 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=16,osdmap=84} crc {osd_snap=280262431,osdmap=3839960951}) 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3394423566}) 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1835660998}) 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3394423566}) 2026-02-17T06:06:04.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.213+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1699840302}) 2026-02-17T06:06:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.216+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.216+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=82} crc {osdmap=140592406}) 2026-02-17T06:06:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.216+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=80} crc {osdmap=2498579732}) 2026-02-17T06:06:04.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.216+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.216+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=82} crc {osdmap=140592406}) 2026-02-17T06:06:04.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:04.216+0000 7ffbf0736640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=993165712}) 2026-02-17T06:06:04.522 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:06:04.522 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:04.529 INFO:tasks.rados.rados.0.trial115.stdout:3248: done (0 left) 2026-02-17T06:06:04.529 INFO:tasks.rados.rados.0.trial115.stdout:3249: rollback oid 28 current snap is 338 2026-02-17T06:06:04.530 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 28 to 317 2026-02-17T06:06:04.530 INFO:tasks.rados.rados.0.trial115.stdout:3250: write (excl) oid 29 current snap is 338 2026-02-17T06:06:04.530 INFO:tasks.rados.rados.0.trial115.stdout:3250: seq_num 1128 ranges {558296=501116,1474512=600725,2760077=198219} 2026-02-17T06:06:04.533 INFO:tasks.rados.rados.0.trial115.stdout:3250: writing trial11545420-29 from 558296 to 1059412 tid 1 2026-02-17T06:06:04.537 INFO:tasks.rados.rados.0.trial115.stdout:3250: writing trial11545420-29 from 1474512 to 2075237 tid 2 2026-02-17T06:06:04.541 INFO:tasks.rados.rados.0.trial115.stdout:3250: writing trial11545420-29 from 2760077 to 2958296 tid 3 2026-02-17T06:06:04.541 INFO:tasks.rados.rados.0.trial115.stdout:3251: rollback oid 48 current snap is 338 2026-02-17T06:06:04.541 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 48 to 336 2026-02-17T06:06:04.541 INFO:tasks.rados.rados.0.trial115.stdout:3249: finishing rollback tid 1 to trial11545420-28 2026-02-17T06:06:04.541 INFO:tasks.rados.rados.0.trial115.stdout:3249: finishing rollback tid 2 to trial11545420-28 2026-02-17T06:06:04.541 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1681 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:06:04.542 INFO:tasks.rados.rados.0.trial115.stdout:3250: finishing write tid 1 to trial11545420-29 2026-02-17T06:06:04.542 INFO:tasks.rados.rados.0.trial115.stdout:3250: finishing write tid 2 to trial11545420-29 2026-02-17T06:06:04.542 INFO:tasks.rados.rados.0.trial115.stdout:3249: done (2 left) 2026-02-17T06:06:04.542 INFO:tasks.rados.rados.0.trial115.stdout:3252: copy_from oid 39 from oid 35 current snap is 338 2026-02-17T06:06:04.542 INFO:tasks.rados.rados.0.trial115.stdout:3253: rollback oid 12 current snap is 338 2026-02-17T06:06:04.542 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 335 2026-02-17T06:06:04.542 INFO:tasks.rados.rados.0.trial115.stdout:3254: snap_create 2026-02-17T06:06:04.543 INFO:tasks.rados.rados.0.trial115.stdout:3250: finishing write tid 3 to trial11545420-29 2026-02-17T06:06:04.543 INFO:tasks.rados.rados.0.trial115.stdout:3253: finishing rollback tid 1 to trial11545420-12 2026-02-17T06:06:04.543 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1583 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:06:04.543 INFO:tasks.rados.rados.0.trial115.stdout:3250: finishing write tid 4 to trial11545420-29 2026-02-17T06:06:04.543 INFO:tasks.rados.rados.0.trial115.stdout:3250: finishing write tid 5 to trial11545420-29 2026-02-17T06:06:04.543 INFO:tasks.rados.rados.0.trial115.stdout:3250: oid 29 updating version 0 to 1472 2026-02-17T06:06:04.543 INFO:tasks.rados.rados.0.trial115.stdout:3250: oid 29 version 1472 is already newer than 1469 2026-02-17T06:06:04.544 INFO:tasks.rados.rados.0.trial115.stdout:3250: oid 29 version 1472 is already newer than 1470 2026-02-17T06:06:04.544 INFO:tasks.rados.rados.0.trial115.stdout:3250: oid 29 version 1472 is already newer than 1471 2026-02-17T06:06:04.544 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1472 (ObjNum 1128 snap 338 seq_num 1128) dirty exists 2026-02-17T06:06:04.544 INFO:tasks.rados.rados.0.trial115.stdout:3250: left oid 29 (ObjNum 1128 snap 338 seq_num 1128) 2026-02-17T06:06:04.544 INFO:tasks.rados.rados.0.trial115.stdout:3251: finishing rollback tid 0 to trial11545420-48 2026-02-17T06:06:04.545 INFO:tasks.rados.rados.0.trial115.stdout:3251: finishing rollback tid 1 to trial11545420-48 2026-02-17T06:06:04.545 INFO:tasks.rados.rados.0.trial115.stdout:3251: finishing rollback tid 2 to trial11545420-48 2026-02-17T06:06:04.545 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 1707 (ObjNum 1050 snap 312 seq_num 1050) dirty exists 2026-02-17T06:06:04.550 INFO:tasks.rados.rados.0.trial115.stdout:3252: finishing copy_from racing read to trial11545420-39 2026-02-17T06:06:04.550 INFO:tasks.rados.rados.0.trial115.stdout:3252: finishing copy_from to trial11545420-39 2026-02-17T06:06:04.550 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1687 (ObjNum 1118 snap 336 seq_num 1118) dirty exists 2026-02-17T06:06:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.588+0000 7fb84f953640 10 monclient: get_monmap_and_config 2026-02-17T06:06:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.588+0000 7fb84f953640 10 monclient: build_initial_monmap 2026-02-17T06:06:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient: monmap: 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:04.590222+0000 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:04.590222+0000 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:04.590 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 1 Processor -- start 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 1 -- start start 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient: init 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb84806c890 0x7fb84806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient(hunting): picked mon.noname-c con 0x7fb84806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb84806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient(hunting): picked mon.noname-b con 0x7fb84809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb848134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84806daf0 con 0x7fb848134df0 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb8480a0880 con 0x7fb84806c890 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb84806d6f0 con 0x7fb84809c4a0 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84f953640 10 monclient(hunting): authenticate will time out at 1810.143732s 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb84806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb84806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47032/0 (socket says 10.20.193.78:47032) 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 1 -- 10.20.193.78:0/634687599 learned_addr learned my addr 10.20.193.78:0/634687599 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84dec9640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb848134df0 0x7fb848135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38114/0 (socket says 10.20.193.78:38114) 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 10 monclient(hunting): get_auth_request con 0x7fb84809c4a0 auth_method 0 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.589+0000 7fb84cec7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 10 monclient(hunting): handle_auth_done global_id 234218 payload 274 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 1 -- 10.20.193.78:0/634687599 >> v2:10.20.193.115:3300/0 conn(0x7fb84806c890 msgr2=0x7fb84806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 1 --2- 10.20.193.78:0/634687599 >> v2:10.20.193.115:3300/0 conn(0x7fb84806c890 0x7fb84806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 1 -- 10.20.193.78:0/634687599 >> v1:10.20.193.78:6789/0 conn(0x7fb848134df0 legacy=0x7fb848135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 10 monclient: _finish_hunting 0 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 1 monclient: found mon.noname-b 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.590+0000 7fb84cec7640 1 -- 10.20.193.78:0/634687599 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84809cc80 con 0x7fb84809c4a0 2026-02-17T06:06:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84cec7640 1 -- 10.20.193.78:0/634687599 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 msgr2=0x7fb84806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84cec7640 1 --2- 10.20.193.78:0/634687599 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb84806d1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb84806d870 tx=0x7fb83c02ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f09ef809e6e499d8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 1 -- 10.20.193.78:0/634687599 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c03c070 con 0x7fb84809c4a0 2026-02-17T06:06:04.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: dump: 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: _finish_auth 0 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: _check_auth_tickets 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 1 -- 10.20.193.78:0/634687599 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83c02fda0 con 0x7fb84809c4a0 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 1 -- 10.20.193.78:0/634687599 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c037660 con 0x7fb84809c4a0 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb837fff640 10 monclient: dump: 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.594 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 5 monclient: authenticate success, global_id 234218 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 4 set_mon_vals no callback set 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 10 monclient: shutdown 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 1 -- 10.20.193.78:0/634687599 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 msgr2=0x7fb84806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 1 --2- 10.20.193.78:0/634687599 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb84806d1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb84806d870 tx=0x7fb83c02ee80 comp rx=0 tx=0).stop 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 1 -- 10.20.193.78:0/634687599 shutdown_connections 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 1 --2- 10.20.193.78:0/634687599 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb84806d1b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 1 --2- 10.20.193.78:0/634687599 >> v2:10.20.193.115:3300/0 conn(0x7fb84806c890 0x7fb84806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 1 -- 10.20.193.78:0/634687599 >> 10.20.193.78:0/634687599 conn(0x7fb8480a6620 msgr2=0x7fb8480a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 1 -- 10.20.193.78:0/634687599 shutdown_connections 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.592+0000 7fb84f953640 1 -- 10.20.193.78:0/634687599 wait complete. 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient: build_initial_monmap 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient: monmap: 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:04.595 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 1 Processor -- start 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 1 -- start start 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient: init 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb84806c890 0x7fb848133fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient(hunting): picked mon.noname-a con 0x7fb84806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb848134520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient(hunting): picked mon.noname-c con 0x7fb84809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb848134df0 0x7fb848121a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient(hunting): picked mon.noname-b con 0x7fb848134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480a01a0 con 0x7fb84806c890 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480a0020 con 0x7fb848134df0 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb84809be20 con 0x7fb84809c4a0 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84f953640 10 monclient(hunting): authenticate will time out at 1810.147766s 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb848134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84d6c8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb84806c890 0x7fb848133fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb848134520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47040/0 (socket says 10.20.193.78:47040) 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 -- 10.20.193.78:0/481844669 learned_addr learned my addr 10.20.193.78:0/481844669 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:04.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient(hunting): get_auth_request con 0x7fb84809c4a0 auth_method 0 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84d6c8640 10 monclient(hunting): get_auth_request con 0x7fb84806c890 auth_method 0 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84d6c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84d6c8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84d6c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84d6c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84d6c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84d6c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84dec9640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb848134df0 0x7fb848121a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84dec9640 10 monclient(hunting): get_auth_request con 0x7fb848134df0 auth_method 0 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84dec9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84dec9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84dec9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient(hunting): handle_auth_done global_id 234221 payload 824 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 -- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb848134df0 msgr2=0x7fb848121a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb848134df0 0x7fb848121a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 -- 10.20.193.78:0/481844669 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb84806c890 msgr2=0x7fb848133fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb84806c890 0x7fb848133fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient: _finish_hunting 0 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 monclient: found mon.noname-c 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.593+0000 7fb84cec7640 1 -- 10.20.193.78:0/481844669 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb848125700 con 0x7fb84809c4a0 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84cec7640 1 -- 10.20.193.78:0/481844669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 msgr2=0x7fb848134520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84cec7640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb848134520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb83c002410 tx=0x7fb83c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 1 -- 10.20.193.78:0/481844669 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c046020 con 0x7fb84809c4a0 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: dump: 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 1 -- 10.20.193.78:0/481844669 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83c042640 con 0x7fb84809c4a0 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 1 -- 10.20.193.78:0/481844669 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c03c040 con 0x7fb84809c4a0 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb8377fe640 4 set_mon_vals no callback set 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 10 monclient: dump: 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.600 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 5 monclient: authenticate success, global_id 234221 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 10 monclient: _renew_subs 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb848123b90 con 0x7fb84809c4a0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 10 monclient: _renew_subs 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8480726f0 con 0x7fb84809c4a0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8480a1750 con 0x7fb84809c4a0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.596+0000 7fb835ffb640 1 -- 10.20.193.78:0/481844669 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb83c03f030 con 0x7fb84809c4a0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.597+0000 7fb835ffb640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb81803df00 0x7fb8180403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.597+0000 7fb835ffb640 1 -- 10.20.193.78:0/481844669 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(699..699 src has 525..699) ==== 4767+0+0 (secure 0 0 0) 0x7fb83c0779c0 con 0x7fb84809c4a0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.597+0000 7fb84d6c8640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb81803df00 0x7fb8180403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.597+0000 7fb84d6c8640 10 monclient: get_auth_request con 0x7fb81803df00 auth_method 0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.599+0000 7fb835ffb640 1 -- 10.20.193.78:0/481844669 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb83c050030 con 0x7fb84809c4a0 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.599+0000 7fb835ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:04.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.599+0000 7fb835ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:04.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.600+0000 7fb84d6c8640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb81803df00 0x7fb8180403c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb838006fd0 tx=0x7fb838009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.717+0000 7fb84f953640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:04.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.717+0000 7fb84f953640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:04.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.717+0000 7fb84f953640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.717+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb848135a70 con 0x7fb84809c4a0 2026-02-17T06:06:04.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.717+0000 7fb835ffb640 1 -- 10.20.193.78:0/481844669 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fb83c0763d0 con 0x7fb84809c4a0 2026-02-17T06:06:04.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.717+0000 7fb835ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:04.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.717+0000 7fb835ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:04.720 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":232,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:04.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb81803df00 msgr2=0x7fb8180403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb81803df00 0x7fb8180403c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb838006fd0 tx=0x7fb838009190 comp rx=0 tx=0).stop 2026-02-17T06:06:04.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 10 monclient: shutdown 2026-02-17T06:06:04.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:04.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 msgr2=0x7fb848134520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb848134520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb83c002410 tx=0x7fb83c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 shutdown_connections 2026-02-17T06:06:04.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb81803df00 0x7fb8180403c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb848134df0 0x7fb848121a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb84809c4a0 0x7fb848134520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 --2- 10.20.193.78:0/481844669 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb84806c890 0x7fb848133fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 >> 10.20.193.78:0/481844669 conn(0x7fb8480a6620 msgr2=0x7fb848135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:04.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 shutdown_connections 2026-02-17T06:06:04.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.718+0000 7fb84f953640 1 -- 10.20.193.78:0/481844669 wait complete. 2026-02-17T06:06:04.728 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:06:04.728 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:06:04.728 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:06:04.728 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:06:04.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.794+0000 7f988fefa640 10 monclient: get_monmap_and_config 2026-02-17T06:06:04.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.794+0000 7f988fefa640 10 monclient: build_initial_monmap 2026-02-17T06:06:04.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.794+0000 7f988fefa640 10 monclient: monmap: 2026-02-17T06:06:04.818 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:04.818 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.818 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:04.795927+0000 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:04.795927+0000 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 1 Processor -- start 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 1 -- start start 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient: init 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f988806c890 0x7f988806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient(hunting): picked mon.noname-c con 0x7f988806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988809d430 0x7f988806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient(hunting): picked mon.noname-b con 0x7f988809d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:04.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f988809bf40 0x7f9888134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient(hunting): picked mon.noname-a con 0x7f988809bf40 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f988809ca50 con 0x7f988809bf40 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f988806d6f0 con 0x7f988806c890 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9888073600 con 0x7f988809d430 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988fefa640 10 monclient(hunting): authenticate will time out at 1810.349510s 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988809d430 0x7f988806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988809d430 0x7f988806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47044/0 (socket says 10.20.193.78:47044) 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f988809bf40 0x7f9888134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 -- 10.20.193.78:0/3985912308 learned_addr learned my addr 10.20.193.78:0/3985912308 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient(hunting): get_auth_request con 0x7f988809d430 auth_method 0 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 10 monclient(hunting): get_auth_request con 0x7f988809bf40 auth_method 0 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient(hunting): handle_auth_done global_id 234224 payload 274 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 -- 10.20.193.78:0/3985912308 >> v2:10.20.193.115:3300/0 conn(0x7f988806c890 msgr2=0x7f988806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988dc6f640 1 --2- 10.20.193.78:0/3985912308 >> v2:10.20.193.115:3300/0 conn(0x7f988806c890 0x7f988806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 --2- 10.20.193.78:0/3985912308 >> v2:10.20.193.115:3300/0 conn(0x7f988806c890 0x7f988806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 -- 10.20.193.78:0/3985912308 >> v2:10.20.193.78:3300/0 conn(0x7f988809bf40 msgr2=0x7f9888134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 --2- 10.20.193.78:0/3985912308 >> v2:10.20.193.78:3300/0 conn(0x7f988809bf40 0x7f9888134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient: _finish_hunting 0 2026-02-17T06:06:04.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 monclient: found mon.noname-b 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988d46e640 1 -- 10.20.193.78:0/3985912308 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988806dab0 con 0x7f988809d430 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988dc6f640 10 monclient: get_auth_request con 0x7f988806c890 auth_method 0 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988dc6f640 1 --2- 10.20.193.78:0/3985912308 >> v2:10.20.193.115:3300/0 conn(0x7f988806c890 0x7f988806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.795+0000 7f988e470640 1 --2- 10.20.193.78:0/3985912308 >> v2:10.20.193.78:3300/0 conn(0x7f988809bf40 0x7f9888134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988d46e640 1 -- 10.20.193.78:0/3985912308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988809d430 msgr2=0x7f988806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988d46e640 1 --2- 10.20.193.78:0/3985912308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988809d430 0x7f988806d1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f988806a480 tx=0x7f987c02ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54260dda4c182d93 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 1 -- 10.20.193.78:0/3985912308 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c03c070 con 0x7f988809d430 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: dump: 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.822 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: _finish_auth 0 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: _check_auth_tickets 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 1 -- 10.20.193.78:0/3985912308 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c02fcb0 con 0x7f988809d430 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 5 monclient: authenticate success, global_id 234224 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 1 -- 10.20.193.78:0/3985912308 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c02fe60 con 0x7f988809d430 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988cc6d640 10 monclient: dump: 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.823 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 4 set_mon_vals no callback set 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 10 monclient: shutdown 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 -- 10.20.193.78:0/3985912308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988809d430 msgr2=0x7f988806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 --2- 10.20.193.78:0/3985912308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988809d430 0x7f988806d1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f988806a480 tx=0x7f987c02ee70 comp rx=0 tx=0).stop 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 -- 10.20.193.78:0/3985912308 shutdown_connections 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 --2- 10.20.193.78:0/3985912308 >> v2:10.20.193.78:3300/0 conn(0x7f988809bf40 0x7f9888134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 --2- 10.20.193.78:0/3985912308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988809d430 0x7f988806d1b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 --2- 10.20.193.78:0/3985912308 >> v2:10.20.193.115:3300/0 conn(0x7f988806c890 0x7f988806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 -- 10.20.193.78:0/3985912308 >> 10.20.193.78:0/3985912308 conn(0x7f98880a6620 msgr2=0x7f98880a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 -- 10.20.193.78:0/3985912308 shutdown_connections 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.798+0000 7f988fefa640 1 -- 10.20.193.78:0/3985912308 wait complete. 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient: build_initial_monmap 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient: monmap: 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 1 Processor -- start 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 1 -- start start 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient: init 2026-02-17T06:06:04.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f98881cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient(hunting): picked mon.noname-c con 0x7f988806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 0x7f98881cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient(hunting): picked mon.noname-a con 0x7f988809bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f988809d430 0x7f9888121330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient(hunting): picked mon.noname-b con 0x7f988809d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9888073380 con 0x7f988809bf40 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9888073200 con 0x7f988809d430 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9888135330 con 0x7f988806c890 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988fefa640 10 monclient(hunting): authenticate will time out at 1810.353700s 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f98881cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 0x7f98881cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 0x7f98881cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51934/0 (socket says 10.20.193.78:51934) 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f98881cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47056/0 (socket says 10.20.193.78:47056) 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 learned_addr learned my addr 10.20.193.78:0/1514120357 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient(hunting): get_auth_request con 0x7f988809bf40 auth_method 0 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 10 monclient(hunting): get_auth_request con 0x7f988806c890 auth_method 0 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient(hunting): handle_auth_done global_id 244137 payload 824 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 msgr2=0x7f98881cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f98881cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f988809d430 msgr2=0x7f9888121330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f988809d430 0x7f9888121330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient: _finish_hunting 0 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 monclient: found mon.noname-a 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.799+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9888123b30 con 0x7f988809bf40 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 msgr2=0x7f98881cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 0x7f98881cfaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f987c002410 tx=0x7f987c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 msgr2=0x7f98881cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 msgr2=0x7f98881cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 0x7f98881cfaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f987c002410 tx=0x7f987c0047c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 0x7f98881cfaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f987c002410 tx=0x7f987c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient: ms_handle_reset current mon [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 msgr2=0x7f98881cfaa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f988809bf40 0x7f98881cfaa0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f988809d430 0x7f98580055d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient(hunting): picked mon.noname-b con 0x7f988809d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f9858008050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient(hunting): picked mon.noname-c con 0x7f988806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9858008ab0 0x7f985800af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient(hunting): picked mon.noname-a con 0x7f9858008ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f985800ba40 con 0x7f9858008ab0 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f985800bbc0 con 0x7f988809d430 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f985800bdc0 con 0x7f988806c890 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f98767fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f9858008050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9858008ab0 0x7f985800af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient(hunting): get_auth_request con 0x7f988806c890 auth_method 0 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 10 monclient(hunting): get_auth_request con 0x7f9858008ab0 auth_method 0 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient(hunting): handle_auth_done global_id 244137 payload 293 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988dc6f640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f988809d430 0x7f98580055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f988809d430 msgr2=0x7f98580055d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f988809d430 0x7f98580055d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9858008ab0 msgr2=0x7f985800af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9858008ab0 0x7f985800af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient: _finish_hunting 0 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 monclient: found mon.noname-c 2026-02-17T06:06:04.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985800c080 con 0x7f988806c890 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988dc6f640 10 monclient: get_auth_request con 0x7f988809d430 auth_method 0 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988dc6f640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f988809d430 0x7f98580055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.802+0000 7f988e470640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9858008ab0 0x7f985800af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988d46e640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 msgr2=0x7f9858008050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988d46e640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f9858008050 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9888122bd0 tx=0x7f987c033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c050070 con 0x7f988806c890 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: dump: 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.831 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: _finish_auth 0 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: _check_auth_tickets 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c003310 con 0x7f988806c890 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988fefa640 5 monclient: authenticate success, global_id 244137 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988fefa640 10 monclient: _renew_subs 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988fefa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98881252c0 con 0x7f988806c890 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c044020 con 0x7f988806c890 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f98767fc640 10 monclient: dump: 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.832 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f9877fff640 4 set_mon_vals no callback set 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988fefa640 10 monclient: _renew_subs 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988fefa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.804+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98880a1540 con 0x7f988806c890 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988fefa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988fefa640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98881cfd20 0x7f9888124c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988fefa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9888124a20 con 0x7f98881cfd20 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f987c03f050 con 0x7f988806c890 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98881cfd20 0x7f9888124c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 10 monclient: get_auth_request con 0x7f98881cfd20 auth_method 0 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 10 monclient(hunting): handle_auth_done global_id 244140 payload 824 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988dc6f640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9888074980 con 0x7f98881cfd20 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f988809bf40 0x7f98580481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(699..699 src has 525..699) ==== 4767+0+0 (secure 0 0 0) 0x7f987c078b10 con 0x7f988806c890 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988e470640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f988809bf40 0x7f98580481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.805+0000 7f988e470640 10 monclient: get_auth_request con 0x7f988809bf40 auth_method 0 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f988dc6f640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98881cfd20 msgr2=0x7f9888124c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f988dc6f640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98881cfd20 0x7f9888124c10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9878010b90 tx=0x7f987800e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987801a070 con 0x7f98881cfd20 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 10 monclient: dump: 2026-02-17T06:06:04.834 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f9878016460 con 0x7f98881cfd20 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98881cfd20 msgr2=0x7f9888124c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98881cfd20 0x7f9888124c10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9878010b90 tx=0x7f987800e5a0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98881cfd20 msgr2=0x7f9888124c10 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.807+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f98881cfd20 0x7f9888124c10 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9878010b90 tx=0x7f987800e5a0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.808+0000 7f988e470640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f988809bf40 0x7f98580481c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9884009bd0 tx=0x7f988400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988fefa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:04.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988fefa640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9888121330 0x7f98881d4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988fefa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98881d0100 con 0x7f9888121330 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9888121330 0x7f98881d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 10 monclient: get_auth_request con 0x7f9888121330 auth_method 0 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 10 monclient(hunting): handle_auth_done global_id 244143 payload 824 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.811+0000 7f988dc6f640 1 -- 10.20.193.78:0/1514120357 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f988809d810 con 0x7f9888121330 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988dc6f640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9888121330 msgr2=0x7f98881d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988dc6f640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9888121330 0x7f98881d4530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9878010b90 tx=0x7f98780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878016e60 con 0x7f9888121330 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 10 monclient: dump: 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.836 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f987801f450 con 0x7f9888121330 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9888121330 msgr2=0x7f98881d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9888121330 0x7f98881d4530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9878010b90 tx=0x7f98780047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9888121330 msgr2=0x7f98881d4530 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:04.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f98767fc640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9888121330 0x7f98881d4530 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9878010b90 tx=0x7f98780047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 232, 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 8362, 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:04.838 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:04.839 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:04.840 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.841 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:04.842 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:04.843 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.844 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:04.845 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f988809bf40 msgr2=0x7f98580481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f988809bf40 0x7f98580481c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9884009bd0 tx=0x7f988400b040 comp rx=0 tx=0).stop 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 10 monclient: shutdown 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 msgr2=0x7f9858008050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f9858008050 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9888122bd0 tx=0x7f987c033000 comp rx=0 tx=0).stop 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988e470640 1 -- 10.20.193.78:0/1514120357 reap_dead start 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 shutdown_connections 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 1 --2- 10.20.193.78:0/1514120357 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f988806c890 0x7f9858008050 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.813+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 >> 10.20.193.78:0/1514120357 conn(0x7f98880a6620 msgr2=0x7f98881369c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.814+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 shutdown_connections 2026-02-17T06:06:04.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.814+0000 7f988fefa640 1 -- 10.20.193.78:0/1514120357 wait complete. 2026-02-17T06:06:04.847 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:04.847 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:06:04.847 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:06:04.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.914+0000 7f0941efc640 10 monclient: get_monmap_and_config 2026-02-17T06:06:04.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.914+0000 7f0941efc640 10 monclient: build_initial_monmap 2026-02-17T06:06:04.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.914+0000 7f0941efc640 10 monclient: monmap: 2026-02-17T06:06:04.915 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:04.915 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:04.915589+0000 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:04.915589+0000 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.914+0000 7f0941efc640 1 Processor -- start 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.914+0000 7f0941efc640 1 -- start start 2026-02-17T06:06:04.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.914+0000 7f0941efc640 10 monclient: init 2026-02-17T06:06:04.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:04.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T06:06:04.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient(hunting): picked mon.noname-b con 0x7f093c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:04.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 0x7f093c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient(hunting): picked mon.noname-a con 0x7f093c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f093c134df0 0x7f093c135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient(hunting): picked mon.noname-c con 0x7f093c134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f093c0a0880 con 0x7f093c09c4a0 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f093c06d6f0 con 0x7f093c134df0 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f093c06daf0 con 0x7f093c06c890 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f0941efc640 10 monclient(hunting): authenticate will time out at 1810.469122s 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47088/0 (socket says 10.20.193.78:47088) 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 -- 10.20.193.78:0/1109394976 learned_addr learned my addr 10.20.193.78:0/1109394976 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 0x7f093c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient(hunting): get_auth_request con 0x7f093c06c890 auth_method 0 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 10 monclient(hunting): get_auth_request con 0x7f093c09c4a0 auth_method 0 2026-02-17T06:06:04.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 1 -- 10.20.193.78:0/1109394976 >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 msgr2=0x7f093c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 1 -- 10.20.193.78:0/1109394976 >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 msgr2=0x7f093c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 1 --2- 10.20.193.78:0/1109394976 >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 0x7f093c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093affd640 1 --2- 10.20.193.78:0/1109394976 >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 0x7f093c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093bfff640 1 --2- 10.20.193.78:0/1109394976 >> v2:10.20.193.115:3300/0 conn(0x7f093c134df0 0x7f093c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093bfff640 10 monclient(hunting): get_auth_request con 0x7f093c134df0 auth_method 0 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient(hunting): handle_auth_done global_id 234230 payload 274 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 -- 10.20.193.78:0/1109394976 >> v2:10.20.193.115:3300/0 conn(0x7f093c134df0 msgr2=0x7f093c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 --2- 10.20.193.78:0/1109394976 >> v2:10.20.193.115:3300/0 conn(0x7f093c134df0 0x7f093c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 -- 10.20.193.78:0/1109394976 >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 msgr2=0x7f093c06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 --2- 10.20.193.78:0/1109394976 >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 0x7f093c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:04.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 monclient: found mon.noname-b 2026-02-17T06:06:04.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:04.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.915+0000 7f093b7fe640 1 -- 10.20.193.78:0/1109394976 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c09cc80 con 0x7f093c06c890 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.917+0000 7f093b7fe640 1 -- 10.20.193.78:0/1109394976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 msgr2=0x7f093c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.917+0000 7f093b7fe640 1 --2- 10.20.193.78:0/1109394976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c06cc70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f093c09c880 tx=0x7f092802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28eb4b053b67ebb7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 1 -- 10.20.193.78:0/1109394976 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092803c070 con 0x7f093c06c890 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: dump: 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.925 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 5 monclient: authenticate success, global_id 234230 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 1 -- 10.20.193.78:0/1109394976 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092802fca0 con 0x7f093c06c890 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 1 -- 10.20.193.78:0/1109394976 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928037660 con 0x7f093c06c890 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f093a7fc640 10 monclient: dump: 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.926 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 4 set_mon_vals no callback set 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 10 monclient: shutdown 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 -- 10.20.193.78:0/1109394976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 msgr2=0x7f093c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 --2- 10.20.193.78:0/1109394976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c06cc70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f093c09c880 tx=0x7f092802ee60 comp rx=0 tx=0).stop 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 -- 10.20.193.78:0/1109394976 shutdown_connections 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 --2- 10.20.193.78:0/1109394976 >> v2:10.20.193.115:3300/0 conn(0x7f093c134df0 0x7f093c135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 --2- 10.20.193.78:0/1109394976 >> v2:10.20.193.78:3300/0 conn(0x7f093c09c4a0 0x7f093c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 --2- 10.20.193.78:0/1109394976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c06cc70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 -- 10.20.193.78:0/1109394976 >> 10.20.193.78:0/1109394976 conn(0x7f093c0a6620 msgr2=0x7f093c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 -- 10.20.193.78:0/1109394976 shutdown_connections 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 1 -- 10.20.193.78:0/1109394976 wait complete. 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 10 monclient: build_initial_monmap 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.918+0000 7f0941efc640 10 monclient: monmap: 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:04.927 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 1 Processor -- start 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 1 -- start start 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient: init 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient(hunting): picked mon.noname-c con 0x7f093c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f093c09c4a0 0x7f093c1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient(hunting): picked mon.noname-a con 0x7f093c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f093c134df0 0x7f093c09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient(hunting): picked mon.noname-b con 0x7f093c134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f093c0a01a0 con 0x7f093c09c4a0 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f093c0a0020 con 0x7f093c134df0 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f093c09be20 con 0x7f093c06c890 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f0941efc640 10 monclient(hunting): authenticate will time out at 1810.473397s 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f093c09c4a0 0x7f093c1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c1cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47098/0 (socket says 10.20.193.78:47098) 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 -- 10.20.193.78:0/3754658740 learned_addr learned my addr 10.20.193.78:0/3754658740 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient(hunting): get_auth_request con 0x7f093c06c890 auth_method 0 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093affd640 10 monclient(hunting): get_auth_request con 0x7f093c09c4a0 auth_method 0 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient(hunting): handle_auth_done global_id 234233 payload 824 2026-02-17T06:06:04.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f093c134df0 msgr2=0x7f093c09db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f093c134df0 0x7f093c09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f093c09c4a0 msgr2=0x7f093c1cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f093c09c4a0 0x7f093c1cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 monclient: found mon.noname-c 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.919+0000 7f093b7fe640 1 -- 10.20.193.78:0/3754658740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c0a14f0 con 0x7f093c06c890 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f093b7fe640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 msgr2=0x7f093c1cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f093b7fe640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c1cf350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0928002410 tx=0x7f0928004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928047070 con 0x7f093c06c890 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: dump: 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.930 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: _finish_auth 0 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: _check_auth_tickets 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092802fe50 con 0x7f093c06c890 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0941efc640 5 monclient: authenticate success, global_id 234233 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0941efc640 10 monclient: _renew_subs 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0941efc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c134750 con 0x7f093c06c890 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092803c040 con 0x7f093c06c890 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0939ffb640 4 set_mon_vals no callback set 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0917fff640 10 monclient: dump: 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.931 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0941efc640 10 monclient: _renew_subs 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0941efc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.921+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c126070 con 0x7f093c06c890 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f0941efc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f0941efc640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09e850 0x7f093c11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f0941efc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f093c127350 con 0x7f093c09e850 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09e850 0x7f093c11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 10 monclient: get_auth_request con 0x7f093c09e850 auth_method 0 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f092803fab0 con 0x7f093c06c890 2026-02-17T06:06:04.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 10 monclient(hunting): handle_auth_done global_id 234236 payload 824 2026-02-17T06:06:04.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f093affd640 1 -- 10.20.193.78:0/3754658740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f093c0a22f0 con 0x7f093c09e850 2026-02-17T06:06:04.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f0917fff640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f090c03dee0 0x7f090c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.922+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(699..699 src has 525..699) ==== 4767+0+0 (secure 0 0 0) 0x7f09280774d0 con 0x7f093c06c890 2026-02-17T06:06:04.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.923+0000 7f093bfff640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f090c03dee0 0x7f090c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.923+0000 7f093bfff640 10 monclient: get_auth_request con 0x7f090c03dee0 auth_method 0 2026-02-17T06:06:04.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.924+0000 7f093affd640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09e850 msgr2=0x7f093c11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.924+0000 7f093affd640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09e850 0x7f093c11cc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f092c006fb0 tx=0x7f092c011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.924+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c019020 con 0x7f093c09e850 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.924+0000 7f0917fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.924+0000 7f0917fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.924+0000 7f0917fff640 10 monclient: dump: 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.925+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f092c00f830 con 0x7f093c09e850 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.925+0000 7f0917fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.925+0000 7f0917fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:04.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.925+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09e850 msgr2=0x7f093c11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.925+0000 7f0917fff640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09e850 0x7f093c11cc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f092c006fb0 tx=0x7f092c011ca0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.925+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09e850 msgr2=0x7f093c11cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.925+0000 7f0917fff640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09e850 0x7f093c11cc70 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f092c006fb0 tx=0x7f092c011ca0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.925+0000 7f093bfff640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f090c03dee0 0x7f090c0403a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f093c1d01a0 tx=0x7f0930009630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f0941efc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f0941efc640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09ec30 0x7f093c1da640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f0941efc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f093c074980 con 0x7f093c09ec30 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09ec30 0x7f093c1da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 10 monclient: get_auth_request con 0x7f093c09ec30 auth_method 0 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 10 monclient(hunting): handle_auth_done global_id 234239 payload 824 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.928+0000 7f093affd640 1 -- 10.20.193.78:0/3754658740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f093c135240 con 0x7f093c09ec30 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f093affd640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09ec30 msgr2=0x7f093c1da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f093affd640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09ec30 0x7f093c1da640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f092c006fb0 tx=0x7f092c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c019070 con 0x7f093c09ec30 2026-02-17T06:06:04.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 10 monclient: dump: 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f092c020d90 con 0x7f093c09ec30 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09ec30 msgr2=0x7f093c1da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09ec30 0x7f093c1da640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f092c006fb0 tx=0x7f092c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09ec30 msgr2=0x7f093c1da640 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:04.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0917fff640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c09ec30 0x7f093c1da640 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f092c006fb0 tx=0x7f092c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 232, 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 73613, 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:04.938 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:04.939 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:04.940 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:04.941 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:04.942 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:04.943 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.944 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:04.945 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:04.946 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:04.946 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:04.946 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:04.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f090c03dee0 msgr2=0x7f090c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f090c03dee0 0x7f090c0403a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f093c1d01a0 tx=0x7f0930009630 comp rx=0 tx=0).stop 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 10 monclient: shutdown 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 msgr2=0x7f093c1cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c1cf350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0928002410 tx=0x7f0928004830 comp rx=0 tx=0).stop 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f093bfff640 1 -- 10.20.193.78:0/3754658740 reap_dead start 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 shutdown_connections 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 --2- 10.20.193.78:0/3754658740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f093c06c890 0x7f093c1cf350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 >> 10.20.193.78:0/3754658740 conn(0x7f093c0a6620 msgr2=0x7f093c0a1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 shutdown_connections 2026-02-17T06:06:04.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:04.930+0000 7f0941efc640 1 -- 10.20.193.78:0/3754658740 wait complete. 2026-02-17T06:06:04.947 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:04.947 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:06:04.948 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:06:05.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.013+0000 7f0bd7be6640 10 monclient: get_monmap_and_config 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.013+0000 7f0bd7be6640 10 monclient: build_initial_monmap 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.013+0000 7f0bd7be6640 10 monclient: monmap: 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:05.015026+0000 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:05.015026+0000 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:05.015 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 1 Processor -- start 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 1 -- start start 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient: init 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd006c890 0x7f0bd006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient(hunting): picked mon.noname-b con 0x7f0bd006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0bd009c4a0 0x7f0bd006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient(hunting): picked mon.noname-c con 0x7f0bd009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bd0134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd006daf0 con 0x7f0bd0134df0 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd00a0880 con 0x7f0bd009c4a0 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd006d6f0 con 0x7f0bd006c890 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd7be6640 10 monclient(hunting): authenticate will time out at 1810.568558s 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd006c890 0x7f0bd006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd615c640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f0bd0134df0 0x7f0bd0135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38116/0 (socket says 10.20.193.78:38116) 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd615c640 1 -- 10.20.193.78:0/244343915 learned_addr learned my addr 10.20.193.78:0/244343915 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient(hunting): get_auth_request con 0x7f0bd006c890 auth_method 0 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd515a640 1 --2- 10.20.193.78:0/244343915 >> v2:10.20.193.115:3300/0 conn(0x7f0bd009c4a0 0x7f0bd006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd515a640 10 monclient(hunting): get_auth_request con 0x7f0bd009c4a0 auth_method 0 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd515a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd515a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd515a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient(hunting): handle_auth_done global_id 234242 payload 274 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 1 -- 10.20.193.78:0/244343915 >> v2:10.20.193.115:3300/0 conn(0x7f0bd009c4a0 msgr2=0x7f0bd006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 1 --2- 10.20.193.78:0/244343915 >> v2:10.20.193.115:3300/0 conn(0x7f0bd009c4a0 0x7f0bd006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 1 -- 10.20.193.78:0/244343915 >> v1:10.20.193.78:6789/0 conn(0x7f0bd0134df0 legacy=0x7f0bd0135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient: _finish_hunting 0 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 1 monclient: found mon.noname-b 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.014+0000 7f0bd595b640 1 -- 10.20.193.78:0/244343915 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd009cc80 con 0x7f0bd006c890 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd595b640 1 -- 10.20.193.78:0/244343915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd006c890 msgr2=0x7f0bd006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd595b640 1 --2- 10.20.193.78:0/244343915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd006c890 0x7f0bd006cc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc002ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1326fb74fbf1e41e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 1 -- 10.20.193.78:0/244343915 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc003c070 con 0x7f0bd006c890 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: dump: 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.025 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: _finish_auth 0 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: _check_auth_tickets 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 1 -- 10.20.193.78:0/244343915 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc002fc00 con 0x7f0bd006c890 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 5 monclient: authenticate success, global_id 234242 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 1 -- 10.20.193.78:0/244343915 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0037440 con 0x7f0bd006c890 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 4 set_mon_vals no callback set 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd4959640 10 monclient: dump: 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.026 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 10 monclient: shutdown 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 1 -- 10.20.193.78:0/244343915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd006c890 msgr2=0x7f0bd006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 1 --2- 10.20.193.78:0/244343915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd006c890 0x7f0bd006cc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc002ed80 comp rx=0 tx=0).stop 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 1 -- 10.20.193.78:0/244343915 shutdown_connections 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 1 --2- 10.20.193.78:0/244343915 >> v2:10.20.193.115:3300/0 conn(0x7f0bd009c4a0 0x7f0bd006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 1 --2- 10.20.193.78:0/244343915 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd006c890 0x7f0bd006cc70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 1 -- 10.20.193.78:0/244343915 >> 10.20.193.78:0/244343915 conn(0x7f0bd00a6620 msgr2=0x7f0bd00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 1 -- 10.20.193.78:0/244343915 shutdown_connections 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.017+0000 7f0bd7be6640 1 -- 10.20.193.78:0/244343915 wait complete. 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient: build_initial_monmap 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient: monmap: 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:05.027 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 1 Processor -- start 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 1 -- start start 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient: init 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bd006c890 0x7f0bd01cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient(hunting): picked mon.noname-a con 0x7f0bd006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd009c4a0 0x7f0bd0121860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient(hunting): picked mon.noname-b con 0x7f0bd009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd0134df0 0x7f0bd0121da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient(hunting): picked mon.noname-c con 0x7f0bd0134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd00a01a0 con 0x7f0bd006c890 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd00a0020 con 0x7f0bd009c4a0 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd009be20 con 0x7f0bd0134df0 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd7be6640 10 monclient(hunting): authenticate will time out at 1810.572738s 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bd006c890 0x7f0bd01cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bd006c890 0x7f0bd01cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51986/0 (socket says 10.20.193.78:51986) 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd615c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd0134df0 0x7f0bd0121da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 1 -- 10.20.193.78:0/2381044343 learned_addr learned my addr 10.20.193.78:0/2381044343 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 10 monclient(hunting): get_auth_request con 0x7f0bd006c890 auth_method 0 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd615c640 10 monclient(hunting): get_auth_request con 0x7f0bd0134df0 auth_method 0 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd615c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd615c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd615c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd615c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd615c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd615c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd515a640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd009c4a0 0x7f0bd0121860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd515a640 10 monclient(hunting): get_auth_request con 0x7f0bd009c4a0 auth_method 0 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd515a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd515a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd515a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd515a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.018+0000 7f0bd515a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd515a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 10 monclient(hunting): handle_auth_done global_id 234245 payload 824 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd009c4a0 msgr2=0x7f0bd0121860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd009c4a0 0x7f0bd0121860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bd006c890 msgr2=0x7f0bd01cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bd006c890 0x7f0bd01cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 10 monclient: _finish_hunting 0 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 1 monclient: found mon.noname-c 2026-02-17T06:06:05.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd615c640 1 -- 10.20.193.78:0/2381044343 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd0126130 con 0x7f0bd0134df0 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.019+0000 7f0bd595b640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bd006c890 0x7f0bd01cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd615c640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd0134df0 msgr2=0x7f0bd0121da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd615c640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd0134df0 0x7f0bd0121da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00cb20 tx=0x7f0bcc005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00ad50 con 0x7f0bd0134df0 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: dump: 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: _finish_auth 0 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc012400 con 0x7f0bd0134df0 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0127f0 con 0x7f0bd0134df0 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 10 monclient: dump: 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbffff640 4 set_mon_vals no callback set 2026-02-17T06:06:05.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 5 monclient: authenticate success, global_id 234245 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 10 monclient: _renew_subs 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd0123fc0 con 0x7f0bd0134df0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 10 monclient: _renew_subs 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd0072660 con 0x7f0bd0134df0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01250f0 0x7f0bd01222e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd00a16a0 con 0x7f0bd01250f0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.021+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0bcc0129d0 con 0x7f0bd0134df0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bbe7fc640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ba003deb0 0x7f0ba0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(699..699 src has 525..699) ==== 4767+0+0 (secure 0 0 0) 0x7f0bcc052c60 con 0x7f0bd0134df0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01250f0 0x7f0bd01222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 10 monclient: get_auth_request con 0x7f0bd01250f0 auth_method 0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd515a640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ba003deb0 0x7f0ba0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd515a640 10 monclient: get_auth_request con 0x7f0ba003deb0 auth_method 0 2026-02-17T06:06:05.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 10 monclient(hunting): handle_auth_done global_id 214321 payload 824 2026-02-17T06:06:05.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.022+0000 7f0bd595b640 1 -- 10.20.193.78:0/2381044343 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bd0074980 con 0x7f0bd01250f0 2026-02-17T06:06:05.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bd595b640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01250f0 msgr2=0x7f0bd01222e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bd595b640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01250f0 0x7f0bd01222e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0048070 con 0x7f0bd01250f0 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 10 monclient: dump: 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bd515a640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ba003deb0 0x7f0ba0040370 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0134890 tx=0x7f0bc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f0bc003d040 con 0x7f0bd01250f0 2026-02-17T06:06:05.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01250f0 msgr2=0x7f0bd01222e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01250f0 0x7f0bd01222e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc00047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01250f0 msgr2=0x7f0bd01222e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.025+0000 7f0bbe7fc640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01250f0 0x7f0bd01222e0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc00047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd7be6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd7be6640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01cdbc0 0x7f0bd01d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd7be6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd006d4b0 con 0x7f0bd01cdbc0 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01cdbc0 0x7f0bd01d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 10 monclient: get_auth_request con 0x7f0bd01cdbc0 auth_method 0 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 10 monclient(hunting): handle_auth_done global_id 214324 payload 824 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.028+0000 7f0bd595b640 1 -- 10.20.193.78:0/2381044343 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0bd00744d0 con 0x7f0bd01cdbc0 2026-02-17T06:06:05.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd595b640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01cdbc0 msgr2=0x7f0bd01d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd595b640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01cdbc0 0x7f0bd01d8330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0047000 tx=0x7f0bc0009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc003c040 con 0x7f0bd01cdbc0 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 10 monclient: dump: 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f0bc0057390 con 0x7f0bd01cdbc0 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01cdbc0 msgr2=0x7f0bd01d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01cdbc0 0x7f0bd01d8330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0047000 tx=0x7f0bc0009290 comp rx=0 tx=0).stop 2026-02-17T06:06:05.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01cdbc0 msgr2=0x7f0bd01d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:05.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bbe7fc640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bd01cdbc0 0x7f0bd01d8330 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0047000 tx=0x7f0bc0009290 comp rx=0 tx=0).stop 2026-02-17T06:06:05.039 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:05.039 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:05.039 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:05.039 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 232, 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 167878, 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:05.040 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.041 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:05.042 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:05.043 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:05.044 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:05.045 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.046 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.047 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:05.048 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:05.049 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.050 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:05.051 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:05.052 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:05.052 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:05.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ba003deb0 msgr2=0x7f0ba0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0ba003deb0 0x7f0ba0040370 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0134890 tx=0x7f0bc4008040 comp rx=0 tx=0).stop 2026-02-17T06:06:05.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 10 monclient: shutdown 2026-02-17T06:06:05.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:05.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd0134df0 msgr2=0x7f0bd0121da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 1 --2- 10.20.193.78:0/2381044343 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bd0134df0 0x7f0bd0121da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00cb20 tx=0x7f0bcc005bc0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd615c640 1 -- 10.20.193.78:0/2381044343 reap_dead start 2026-02-17T06:06:05.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd615c640 1 -- 10.20.193.78:0/2381044343 reap_dead start 2026-02-17T06:06:05.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 shutdown_connections 2026-02-17T06:06:05.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 >> 10.20.193.78:0/2381044343 conn(0x7f0bd00a6620 msgr2=0x7f0bd0135bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:05.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 shutdown_connections 2026-02-17T06:06:05.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.031+0000 7f0bd7be6640 1 -- 10.20.193.78:0/2381044343 wait complete. 2026-02-17T06:06:05.054 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:05.055 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T06:06:05.055 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:06:05.055 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T06:06:05.055 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T06:06:05.055 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T06:06:05.055 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:06:05.064 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T06:06:05.065 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:06:05.065 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.170+0000 7f889f2b6640 10 monclient: get_monmap_and_config 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.170+0000 7f889f2b6640 10 monclient: build_initial_monmap 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.170+0000 7f889f2b6640 10 monclient: monmap: 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:05.171595+0000 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:05.171595+0000 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:05.171 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:05.172 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:05.172 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:05.172 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:05.172 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.170+0000 7f889f2b6640 1 Processor -- start 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.170+0000 7f889f2b6640 1 -- start start 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.170+0000 7f889f2b6640 10 monclient: init 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f88980734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient(hunting): picked mon.noname-b con 0x7f88980730c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f88980a2530 0x7f88980a0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient(hunting): picked mon.noname-a con 0x7f88980a2530 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f88980a0870 0x7f88980a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient(hunting): picked mon.noname-c con 0x7f88980a0870 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f8898138fd0 con 0x7f88980a2530 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f88980a2c20 con 0x7f88980a0870 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8898073b50 con 0x7f88980730c0 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:05.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889f2b6640 10 monclient(hunting): authenticate will time out at 1810.725157s 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889c82a640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f88980a2530 msgr2=0x7f88980a0330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889c82a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f88980a2530 0x7f88980a0330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f88980734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f88980734a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47160/0 (socket says 10.20.193.78:47160) 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 -- 10.20.193.78:0/3346206297 learned_addr learned my addr 10.20.193.78:0/3346206297 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient(hunting): get_auth_request con 0x7f88980730c0 auth_method 0 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d82c640 1 --2- 10.20.193.78:0/3346206297 >> v2:10.20.193.115:3300/0 conn(0x7f88980a0870 0x7f88980a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d82c640 10 monclient(hunting): get_auth_request con 0x7f88980a0870 auth_method 0 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d82c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d82c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d82c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d82c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d82c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d82c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient(hunting): handle_auth_done global_id 234248 payload 274 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 -- 10.20.193.78:0/3346206297 >> v2:10.20.193.115:3300/0 conn(0x7f88980a0870 msgr2=0x7f88980a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 --2- 10.20.193.78:0/3346206297 >> v2:10.20.193.115:3300/0 conn(0x7f88980a0870 0x7f88980a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 -- 10.20.193.78:0/3346206297 >> v2:10.20.193.78:3300/0 conn(0x7f88980a2530 msgr2=0x7f88980a0330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 --2- 10.20.193.78:0/3346206297 >> v2:10.20.193.78:3300/0 conn(0x7f88980a2530 0x7f88980a0330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient: _finish_hunting 0 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 monclient: found mon.noname-b 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.171+0000 7f889d02b640 1 -- 10.20.193.78:0/3346206297 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88980a35d0 con 0x7f88980730c0 2026-02-17T06:06:05.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.173+0000 7f889d02b640 1 -- 10.20.193.78:0/3346206297 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 msgr2=0x7f88980734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:05.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.173+0000 7f889d02b640 1 --2- 10.20.193.78:0/3346206297 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f88980734a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88980a4de0 tx=0x7f888802ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32c4fb3779baff5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:05.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 1 -- 10.20.193.78:0/3346206297 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888803c070 con 0x7f88980730c0 2026-02-17T06:06:05.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:05.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: dump: 2026-02-17T06:06:05.180 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: _finish_auth 0 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: _check_auth_tickets 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 1 -- 10.20.193.78:0/3346206297 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888802fcc0 con 0x7f88980730c0 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 5 monclient: authenticate success, global_id 234248 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 1 -- 10.20.193.78:0/3346206297 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888037660 con 0x7f88980730c0 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f8887fff640 10 monclient: dump: 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 4 set_mon_vals no callback set 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 10 monclient: shutdown 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 -- 10.20.193.78:0/3346206297 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 msgr2=0x7f88980734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 --2- 10.20.193.78:0/3346206297 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f88980734a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88980a4de0 tx=0x7f888802ee80 comp rx=0 tx=0).stop 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 -- 10.20.193.78:0/3346206297 shutdown_connections 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 --2- 10.20.193.78:0/3346206297 >> v2:10.20.193.115:3300/0 conn(0x7f88980a0870 0x7f88980a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 --2- 10.20.193.78:0/3346206297 >> v2:10.20.193.78:3300/0 conn(0x7f88980a2530 0x7f88980a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 --2- 10.20.193.78:0/3346206297 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f88980734a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 -- 10.20.193.78:0/3346206297 >> 10.20.193.78:0/3346206297 conn(0x7f88980a7370 msgr2=0x7f88980a7780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 -- 10.20.193.78:0/3346206297 shutdown_connections 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.174+0000 7f889f2b6640 1 -- 10.20.193.78:0/3346206297 wait complete. 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient: build_initial_monmap 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient: monmap: 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 1 Processor -- start 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 1 -- start start 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient: init 2026-02-17T06:06:05.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f8898131400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient(hunting): picked mon.noname-c con 0x7f88980730c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88980a0870 0x7f8898131940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient(hunting): picked mon.noname-b con 0x7f88980a0870 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88980a2530 0x7f889813b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient(hunting): picked mon.noname-a con 0x7f88980a2530 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f88980a2260 con 0x7f88980a2530 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8898068950 con 0x7f88980a0870 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8898068ad0 con 0x7f88980730c0 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889f2b6640 10 monclient(hunting): authenticate will time out at 1810.729518s 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d82c640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88980a2530 msgr2=0x7f889813b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d82c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88980a2530 0x7f889813b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f8898131400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f8898131400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:47166/0 (socket says 10.20.193.78:47166) 2026-02-17T06:06:05.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 -- 10.20.193.78:0/4154358598 learned_addr learned my addr 10.20.193.78:0/4154358598 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient(hunting): get_auth_request con 0x7f88980730c0 auth_method 0 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889c82a640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88980a0870 0x7f8898131940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889c82a640 10 monclient(hunting): get_auth_request con 0x7f88980a0870 auth_method 0 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889c82a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889c82a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889c82a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient(hunting): handle_auth_done global_id 234251 payload 824 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 -- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88980a0870 msgr2=0x7f8898131940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88980a0870 0x7f8898131940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 -- 10.20.193.78:0/4154358598 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88980a2530 msgr2=0x7f889813b060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88980a2530 0x7f889813b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient: _finish_hunting 0 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 monclient: found mon.noname-c 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:05.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.175+0000 7f889d02b640 1 -- 10.20.193.78:0/4154358598 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889813e9b0 con 0x7f88980730c0 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f889d02b640 1 -- 10.20.193.78:0/4154358598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 msgr2=0x7f8898131400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f889d02b640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f8898131400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8888009a70 tx=0x7f8888004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 1 -- 10.20.193.78:0/4154358598 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888047070 con 0x7f88980730c0 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: dump: 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:05.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 1 -- 10.20.193.78:0/4154358598 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888803f7b0 con 0x7f88980730c0 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f889f2b6640 5 monclient: authenticate success, global_id 234251 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 1 -- 10.20.193.78:0/4154358598 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888803c040 con 0x7f88980730c0 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f8885ffb640 10 monclient: dump: 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f889f2b6640 10 monclient: _renew_subs 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f88877fe640 4 set_mon_vals no callback set 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f889f2b6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.177+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889813d560 con 0x7f88980730c0 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f889f2b6640 10 monclient: _renew_subs 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f889f2b6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f889813ec90 con 0x7f88980730c0 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f8885ffb640 1 -- 10.20.193.78:0/4154358598 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f888803fba0 con 0x7f88980730c0 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f889f2b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f889f2b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f889f2b6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88980a4700 con 0x7f88980730c0 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f8885ffb640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f886803de60 0x7f8868040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.178+0000 7f8885ffb640 1 -- 10.20.193.78:0/4154358598 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(699..699 src has 525..699) ==== 4767+0+0 (secure 0 0 0) 0x7f8888077380 con 0x7f88980730c0 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.179+0000 7f889c82a640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f886803de60 0x7f8868040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.179+0000 7f889c82a640 10 monclient: get_auth_request con 0x7f886803de60 auth_method 0 2026-02-17T06:06:05.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.181+0000 7f8885ffb640 1 -- 10.20.193.78:0/4154358598 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8888075d90 con 0x7f88980730c0 2026-02-17T06:06:05.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.181+0000 7f8885ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:05.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.181+0000 7f8885ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:05.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.181+0000 7f889c82a640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f886803de60 0x7f8868040320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f888c006fd0 tx=0x7f888c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:05.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.296+0000 7f889f2b6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:05.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.296+0000 7f889f2b6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:05.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.296+0000 7f889f2b6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:05.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.296+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f88980a0d00 con 0x7f88980730c0 2026-02-17T06:06:05.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.296+0000 7f8885ffb640 1 -- 10.20.193.78:0/4154358598 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f8888049d90 con 0x7f88980730c0 2026-02-17T06:06:05.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.296+0000 7f8885ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:05.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.296+0000 7f8885ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:05.299 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":232,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.297+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f886803de60 msgr2=0x7f8868040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.297+0000 7f889f2b6640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f886803de60 0x7f8868040320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f888c006fd0 tx=0x7f888c008040 comp rx=0 tx=0).stop 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.297+0000 7f889f2b6640 10 monclient: shutdown 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.297+0000 7f889f2b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.297+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 msgr2=0x7f8898131400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.297+0000 7f889f2b6640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f8898131400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8888009a70 tx=0x7f8888004830 comp rx=0 tx=0).stop 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.298+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 shutdown_connections 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.298+0000 7f889f2b6640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f88980a2530 0x7f889813b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.298+0000 7f889f2b6640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f886803de60 0x7f8868040320 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.298+0000 7f889f2b6640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f88980a0870 0x7f8898131940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.298+0000 7f889f2b6640 1 --2- 10.20.193.78:0/4154358598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f88980730c0 0x7f8898131400 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:05.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.298+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 >> 10.20.193.78:0/4154358598 conn(0x7f88980a7370 msgr2=0x7f8898137b70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:05.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.298+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 shutdown_connections 2026-02-17T06:06:05.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:05.298+0000 7f889f2b6640 1 -- 10.20.193.78:0/4154358598 wait complete. 2026-02-17T06:06:08.307 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient: get_monmap_and_config 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient: build_initial_monmap 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient: monmap: 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:08.377518+0000 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:08.377518+0000 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:08.377 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 1 Processor -- start 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 1 -- start start 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient: init 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f337c06c890 0x7f337c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient(hunting): picked mon.noname-c con 0x7f337c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f337c06cc70 0x7f337c06d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient(hunting): picked mon.noname-a con 0x7f337c06cc70 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c134df0 0x7f337c135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient(hunting): picked mon.noname-b con 0x7f337c134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f337c074150 con 0x7f337c06cc70 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f337c09ef70 con 0x7f337c06c890 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f337c09f250 con 0x7f337c134df0 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.376+0000 7f3382829640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3382829640 10 monclient(hunting): authenticate will time out at 1813.931005s 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f337b7fe640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f337c06cc70 msgr2=0x7f337c06d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f337b7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f337c06cc70 0x7f337c06d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c134df0 0x7f337c135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c134df0 0x7f337c135220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:55144/0 (socket says 10.20.193.78:55144) 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 -- 10.20.193.78:0/3376829963 learned_addr learned my addr 10.20.193.78:0/3376829963 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:08.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient(hunting): get_auth_request con 0x7f337c134df0 auth_method 0 2026-02-17T06:06:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:08.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient(hunting): handle_auth_done global_id 234254 payload 274 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 -- 10.20.193.78:0/3376829963 >> v2:10.20.193.115:3300/0 conn(0x7f337c06c890 msgr2=0x7f337c0a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 --2- 10.20.193.78:0/3376829963 >> v2:10.20.193.115:3300/0 conn(0x7f337c06c890 0x7f337c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 -- 10.20.193.78:0/3376829963 >> v2:10.20.193.78:3300/0 conn(0x7f337c06cc70 msgr2=0x7f337c06d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 --2- 10.20.193.78:0/3376829963 >> v2:10.20.193.78:3300/0 conn(0x7f337c06cc70 0x7f337c06d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient: _finish_hunting 0 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 monclient: found mon.noname-b 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.377+0000 7f3380d9f640 1 -- 10.20.193.78:0/3376829963 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c09df40 con 0x7f337c134df0 2026-02-17T06:06:08.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f3380d9f640 1 -- 10.20.193.78:0/3376829963 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c134df0 msgr2=0x7f337c135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f3380d9f640 1 --2- 10.20.193.78:0/3376829963 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c134df0 0x7f337c135220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f337c06a480 tx=0x7f337002f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56304f7ba2a5d38 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 1 -- 10.20.193.78:0/3376829963 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700373f0 con 0x7f337c134df0 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: dump: 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: _finish_auth 0 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 1 -- 10.20.193.78:0/3376829963 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3370037590 con 0x7f337c134df0 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f3382829640 5 monclient: authenticate success, global_id 234254 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 1 -- 10.20.193.78:0/3376829963 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370037960 con 0x7f337c134df0 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f3382829640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f3382829640 4 set_mon_vals no callback set 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.379+0000 7f337affd640 10 monclient: dump: 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:08.381 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 10 monclient: shutdown 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 -- 10.20.193.78:0/3376829963 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c134df0 msgr2=0x7f337c135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 --2- 10.20.193.78:0/3376829963 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c134df0 0x7f337c135220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f337c06a480 tx=0x7f337002f330 comp rx=0 tx=0).stop 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 -- 10.20.193.78:0/3376829963 shutdown_connections 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 --2- 10.20.193.78:0/3376829963 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c134df0 0x7f337c135220 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 --2- 10.20.193.78:0/3376829963 >> v2:10.20.193.78:3300/0 conn(0x7f337c06cc70 0x7f337c06d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 --2- 10.20.193.78:0/3376829963 >> v2:10.20.193.115:3300/0 conn(0x7f337c06c890 0x7f337c0a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 -- 10.20.193.78:0/3376829963 >> 10.20.193.78:0/3376829963 conn(0x7f337c0a6620 msgr2=0x7f337c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 -- 10.20.193.78:0/3376829963 shutdown_connections 2026-02-17T06:06:08.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 -- 10.20.193.78:0/3376829963 wait complete. 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 10 monclient: build_initial_monmap 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 10 monclient: monmap: 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:08.383 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 Processor -- start 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 1 -- start start 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.380+0000 7f3382829640 10 monclient: init 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f337c06c890 0x7f337c121a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient(hunting): picked mon.noname-b con 0x7f337c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c06cc70 0x7f337c121fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient(hunting): picked mon.noname-c con 0x7f337c06cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f337c134df0 0x7f337c1cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient(hunting): picked mon.noname-a con 0x7f337c134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f337c09d360 con 0x7f337c134df0 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f337c09d1e0 con 0x7f337c06c890 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f337c135760 con 0x7f337c06cc70 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3382829640 10 monclient(hunting): authenticate will time out at 1813.935099s 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3380d9f640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f337c134df0 msgr2=0x7f337c1cf000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f3380d9f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f337c134df0 0x7f337c1cf000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c06cc70 0x7f337c121fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c06cc70 0x7f337c121fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:55154/0 (socket says 10.20.193.78:55154) 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 -- 10.20.193.78:0/1983504299 learned_addr learned my addr 10.20.193.78:0/1983504299 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient(hunting): get_auth_request con 0x7f337c06cc70 auth_method 0 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337bfff640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f337c06c890 0x7f337c121a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337bfff640 10 monclient(hunting): get_auth_request con 0x7f337c06c890 auth_method 0 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient(hunting): handle_auth_done global_id 234257 payload 824 2026-02-17T06:06:08.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 -- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f337c06c890 msgr2=0x7f337c121a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f337c06c890 0x7f337c121a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 -- 10.20.193.78:0/1983504299 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f337c134df0 msgr2=0x7f337c1cf000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f337c134df0 0x7f337c1cf000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 monclient: found mon.noname-c 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.381+0000 7f337b7fe640 1 -- 10.20.193.78:0/1983504299 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c122d10 con 0x7f337c06cc70 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f337b7fe640 1 -- 10.20.193.78:0/1983504299 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c06cc70 msgr2=0x7f337c121fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f337b7fe640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c06cc70 0x7f337c121fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f336c00b780 tx=0x7f336c00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:08.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 1 -- 10.20.193.78:0/1983504299 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c005270 con 0x7f337c06cc70 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: dump: 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: _finish_auth 0 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 1 -- 10.20.193.78:0/1983504299 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f336c010070 con 0x7f337c06cc70 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 1 -- 10.20.193.78:0/1983504299 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c005650 con 0x7f337c06cc70 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f337a7fc640 4 set_mon_vals no callback set 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3378ff9640 10 monclient: dump: 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3382829640 5 monclient: authenticate success, global_id 234257 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3382829640 10 monclient: _renew_subs 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3382829640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c09c9c0 con 0x7f337c06cc70 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3382829640 10 monclient: _renew_subs 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3382829640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:08.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.383+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c128370 con 0x7f337c06cc70 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f3382829640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f3382829640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f3382829640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337c135a40 con 0x7f337c06cc70 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f3378ff9640 1 -- 10.20.193.78:0/1983504299 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f336c00c520 con 0x7f337c06cc70 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f3378ff9640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f334c03de60 0x7f334c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f3378ff9640 1 -- 10.20.193.78:0/1983504299 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(699..699 src has 525..699) ==== 4767+0+0 (secure 0 0 0) 0x7f336c052050 con 0x7f337c06cc70 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f337bfff640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f334c03de60 0x7f334c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.384+0000 7f337bfff640 10 monclient: get_auth_request con 0x7f334c03de60 auth_method 0 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.386+0000 7f3378ff9640 1 -- 10.20.193.78:0/1983504299 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f336c005870 con 0x7f337c06cc70 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.386+0000 7f3378ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.386+0000 7f3378ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:08.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.387+0000 7f337bfff640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f334c03de60 0x7f334c040320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3368009990 tx=0x7f3368008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:08.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.500+0000 7f3382829640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:08.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.500+0000 7f3382829640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:08.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.500+0000 7f3382829640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:08.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.500+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f337c1286e0 con 0x7f337c06cc70 2026-02-17T06:06:08.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.500+0000 7f3378ff9640 1 -- 10.20.193.78:0/1983504299 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f336c013c20 con 0x7f337c06cc70 2026-02-17T06:06:08.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.500+0000 7f3378ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:08.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.500+0000 7f3378ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:08.502 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":232,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:08.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f334c03de60 msgr2=0x7f334c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:08.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f334c03de60 0x7f334c040320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3368009990 tx=0x7f3368008040 comp rx=0 tx=0).stop 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 10 monclient: shutdown 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c06cc70 msgr2=0x7f337c121fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c06cc70 0x7f337c121fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f336c00b780 tx=0x7f336c00bc50 comp rx=0 tx=0).stop 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 shutdown_connections 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f337c134df0 0x7f337c1cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f337c06cc70 0x7f337c121fd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f334c03de60 0x7f334c040320 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 --2- 10.20.193.78:0/1983504299 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f337c06c890 0x7f337c121a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 >> 10.20.193.78:0/1983504299 conn(0x7f337c0a6620 msgr2=0x7f337c073660 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:08.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 shutdown_connections 2026-02-17T06:06:08.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:08.501+0000 7f3382829640 1 -- 10.20.193.78:0/1983504299 wait complete. 2026-02-17T06:06:11.511 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:11.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.580+0000 7fafaf779640 10 monclient: get_monmap_and_config 2026-02-17T06:06:11.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.580+0000 7fafaf779640 10 monclient: build_initial_monmap 2026-02-17T06:06:11.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.580+0000 7fafaf779640 10 monclient: monmap: 2026-02-17T06:06:11.581 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:11.581 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:11.582145+0000 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:11.582145+0000 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:11.582 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 1 Processor -- start 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 1 -- start start 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient: init 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa806c890 0x7fafa80a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient(hunting): picked mon.noname-b con 0x7fafa806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fafa806cc70 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fafa813a240 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa8074150 con 0x7fafa806cc70 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa809ef70 con 0x7fafa813a240 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fafa809f250 con 0x7fafa806c890 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafaf779640 10 monclient(hunting): authenticate will time out at 1817.135666s 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafacced640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7fafa806cc70 legacy=0x7fafa806d050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa806c890 0x7fafa80a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa806c890 0x7fafa80a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:55162/0 (socket says 10.20.193.78:55162) 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 1 -- 10.20.193.78:0/460789941 learned_addr learned my addr 10.20.193.78:0/460789941 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient(hunting): get_auth_request con 0x7fafa806c890 auth_method 0 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:11.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient(hunting): handle_auth_done global_id 234260 payload 274 2026-02-17T06:06:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 1 -- 10.20.193.78:0/460789941 >> v1:10.20.193.115:6789/0 conn(0x7fafa813a240 legacy=0x7fafa813c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 1 -- 10.20.193.78:0/460789941 >> v1:10.20.193.78:6789/0 conn(0x7fafa806cc70 legacy=0x7fafa806d050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient: _finish_hunting 0 2026-02-17T06:06:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 1 monclient: found mon.noname-b 2026-02-17T06:06:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:11.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.581+0000 7fafad4ee640 1 -- 10.20.193.78:0/460789941 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa809df40 con 0x7fafa806c890 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafad4ee640 1 -- 10.20.193.78:0/460789941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa806c890 msgr2=0x7fafa80a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafad4ee640 1 --2- 10.20.193.78:0/460789941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa806c890 0x7fafa80a01f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fafa806e270 tx=0x7faf9802ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7eb005a6b51350 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 1 -- 10.20.193.78:0/460789941 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9803c070 con 0x7fafa806c890 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: dump: 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:11.585 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: _finish_auth 0 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: _check_auth_tickets 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 1 -- 10.20.193.78:0/460789941 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf98037440 con 0x7fafa806c890 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafaf779640 5 monclient: authenticate success, global_id 234260 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafaf779640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 1 -- 10.20.193.78:0/460789941 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98037810 con 0x7fafa806c890 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafaf779640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafaf779640 4 set_mon_vals no callback set 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7faf97fff640 10 monclient: dump: 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafaf779640 10 monclient: shutdown 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafaf779640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafaf779640 1 -- 10.20.193.78:0/460789941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa806c890 msgr2=0x7fafa80a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.584+0000 7fafaf779640 1 --2- 10.20.193.78:0/460789941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa806c890 0x7fafa80a01f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fafa806e270 tx=0x7faf9802ef90 comp rx=0 tx=0).stop 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 -- 10.20.193.78:0/460789941 shutdown_connections 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 --2- 10.20.193.78:0/460789941 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa806c890 0x7fafa80a01f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 -- 10.20.193.78:0/460789941 >> 10.20.193.78:0/460789941 conn(0x7fafa80a6620 msgr2=0x7fafa80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:11.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 -- 10.20.193.78:0/460789941 shutdown_connections 2026-02-17T06:06:11.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 -- 10.20.193.78:0/460789941 wait complete. 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient: build_initial_monmap 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient: monmap: 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 Processor -- start 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 -- start start 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient: init 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafa806c890 0x7fafa811cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient(hunting): picked mon.noname-b con 0x7fafa806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fafa806cc70 0x7fafa811d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient(hunting): picked mon.noname-a con 0x7fafa806cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 0x7fafa811d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient(hunting): picked mon.noname-c con 0x7fafa813a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fafa813ded0 con 0x7fafa806cc70 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fafa813dd50 con 0x7fafa806c890 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fafa813e050 con 0x7fafa813a240 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:11.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafaf779640 10 monclient(hunting): authenticate will time out at 1817.139912s 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafacced640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fafa806cc70 msgr2=0x7fafa811d1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafacced640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fafa806cc70 0x7fafa811d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafadcef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 0x7fafa811d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafadcef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 0x7fafa811d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:55174/0 (socket says 10.20.193.78:55174) 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafadcef640 1 -- 10.20.193.78:0/2314837113 learned_addr learned my addr 10.20.193.78:0/2314837113 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafadcef640 10 monclient(hunting): get_auth_request con 0x7fafa813a240 auth_method 0 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafadcef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafadcef640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.585+0000 7fafadcef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafad4ee640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafa806c890 0x7fafa811cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafad4ee640 10 monclient(hunting): get_auth_request con 0x7fafa806c890 auth_method 0 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafad4ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafad4ee640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafad4ee640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafad4ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafad4ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafad4ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 10 monclient(hunting): handle_auth_done global_id 234263 payload 824 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 1 -- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafa806c890 msgr2=0x7fafa811cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafa806c890 0x7fafa811cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 1 -- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fafa806cc70 msgr2=0x7fafa811d1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fafa806cc70 0x7fafa811d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 10 monclient: _finish_hunting 0 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 1 monclient: found mon.noname-c 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:11.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.586+0000 7fafadcef640 1 -- 10.20.193.78:0/2314837113 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa811f990 con 0x7fafa813a240 2026-02-17T06:06:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafadcef640 1 -- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 msgr2=0x7fafa811d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafadcef640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 0x7fafa811d6f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafa400cb20 tx=0x7fafa400a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 1 -- 10.20.193.78:0/2314837113 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4016020 con 0x7fafa813a240 2026-02-17T06:06:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:11.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: dump: 2026-02-17T06:06:11.590 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 1 -- 10.20.193.78:0/2314837113 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa4012420 con 0x7fafa813a240 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafaf779640 5 monclient: authenticate success, global_id 234263 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafaf779640 10 monclient: _renew_subs 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafaf779640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa80a20b0 con 0x7fafa813a240 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 1 -- 10.20.193.78:0/2314837113 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4012810 con 0x7fafa813a240 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf977fe640 4 set_mon_vals no callback set 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7faf95ffb640 10 monclient: dump: 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafaf779640 10 monclient: _renew_subs 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafaf779640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.588+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa8072d70 con 0x7fafa813a240 2026-02-17T06:06:11.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7faf95ffb640 1 -- 10.20.193.78:0/2314837113 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fafa40129b0 con 0x7fafa813a240 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7fafaf779640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7fafaf779640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7fafaf779640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa811f1f0 con 0x7fafa813a240 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7faf95ffb640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf7803deb0 0x7faf78040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7faf95ffb640 1 -- 10.20.193.78:0/2314837113 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(699..699 src has 525..699) ==== 4767+0+0 (secure 0 0 0) 0x7fafa40530c0 con 0x7fafa813a240 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7fafad4ee640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf7803deb0 0x7faf78040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.589+0000 7fafad4ee640 10 monclient: get_auth_request con 0x7faf7803deb0 auth_method 0 2026-02-17T06:06:11.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.591+0000 7fafadcef640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 0x7fafa811d6f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafa400cb20 tx=0x7fafa400a060 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fafa811f1f0 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) 2026-02-17T06:06:11.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:11.592+0000 7fafad4ee640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf7803deb0 0x7faf78040370 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faf98036000 tx=0x7faf98033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:12.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.420+0000 7faf95ffb640 1 -- 10.20.193.78:0/2314837113 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fafa811f1f0 con 0x7fafa813a240 2026-02-17T06:06:12.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.420+0000 7faf95ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:12.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.420+0000 7faf95ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:12.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.533+0000 7fafaf779640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:12.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.533+0000 7fafaf779640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:12.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.533+0000 7fafaf779640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:12.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.533+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fafa80a01f0 con 0x7fafa813a240 2026-02-17T06:06:12.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.533+0000 7faf95ffb640 1 -- 10.20.193.78:0/2314837113 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fafa4020020 con 0x7fafa813a240 2026-02-17T06:06:12.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.533+0000 7faf95ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:12.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.533+0000 7faf95ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:12.536 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":232,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:12.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf7803deb0 msgr2=0x7faf78040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:12.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf7803deb0 0x7faf78040370 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faf98036000 tx=0x7faf98033000 comp rx=0 tx=0).stop 2026-02-17T06:06:12.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 10 monclient: shutdown 2026-02-17T06:06:12.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:12.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 msgr2=0x7fafa811d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 0x7fafa811d6f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafa400cb20 tx=0x7fafa400a060 comp rx=0 tx=0).stop 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 shutdown_connections 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fafa813a240 0x7fafa811d6f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fafa806cc70 0x7fafa811d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7faf7803deb0 0x7faf78040370 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 --2- 10.20.193.78:0/2314837113 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafa806c890 0x7fafa811cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 >> 10.20.193.78:0/2314837113 conn(0x7fafa80a6620 msgr2=0x7fafa809be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 shutdown_connections 2026-02-17T06:06:12.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:12.534+0000 7fafaf779640 1 -- 10.20.193.78:0/2314837113 wait complete. 2026-02-17T06:06:15.544 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.611+0000 7f9520efa640 10 monclient: get_monmap_and_config 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.611+0000 7f9520efa640 10 monclient: build_initial_monmap 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.611+0000 7f9520efa640 10 monclient: monmap: 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:15.612753+0000 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:15.612753+0000 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:15.612 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:15.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.611+0000 7f9520efa640 1 Processor -- start 2026-02-17T06:06:15.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.611+0000 7f9520efa640 1 -- start start 2026-02-17T06:06:15.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.611+0000 7f9520efa640 10 monclient: init 2026-02-17T06:06:15.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:15.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient: _add_conns ranks=[3,4,1,2,0] 2026-02-17T06:06:15.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 0x7f951c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:15.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient(hunting): picked mon.noname-a con 0x7f951c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:15.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient(hunting): picked mon.noname-c con 0x7f951c09d430 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f951c09bf40 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f951c073600 con 0x7f951c09bf40 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f951c09ca50 con 0x7f951c06c890 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f951c06d6f0 con 0x7f951c09d430 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9520efa640 10 monclient(hunting): authenticate will time out at 1821.166344s 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f951a575640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 msgr2=0x7f951c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f951a575640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 0x7f951c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f951ad76640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f951c09bf40 0x7f951c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:37604/0 (socket says 10.20.193.78:37604) 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f951ad76640 1 -- 10.20.193.78:0/3745425645 learned_addr learned my addr 10.20.193.78:0/3745425645 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 10 monclient(hunting): get_auth_request con 0x7f951c09d430 auth_method 0 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:15.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.612+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:15.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.614+0000 7f951ad76640 1 -- 10.20.193.78:0/3745425645 >> v1:10.20.193.115:6789/0 conn(0x7f951c09bf40 legacy=0x7f951c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:15.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 msgr2=0x7f951c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:15.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 0x7f951c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:15.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:15.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 10 monclient(hunting): get_auth_request con 0x7f951c09d430 auth_method 0 2026-02-17T06:06:15.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:15.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:15.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:15.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:15.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:15.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:15.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:15.813+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:16.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.213+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 msgr2=0x7f951c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:16.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.213+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 0x7f951c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:16.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:16.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 10 monclient(hunting): get_auth_request con 0x7f951c09d430 auth_method 0 2026-02-17T06:06:16.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:16.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:16.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:16.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:16.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:16.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:16.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:16.214+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:17.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.014+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 msgr2=0x7f951c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:17.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.014+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 0x7f951c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:17.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:17.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 10 monclient(hunting): get_auth_request con 0x7f951c09d430 auth_method 0 2026-02-17T06:06:17.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:17.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:17.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:17.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:17.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:17.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:17.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:17.015+0000 7f9519d74640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 10 monclient(hunting): tick 2026-02-17T06:06:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:06:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 monclient(hunting): continuing hunt 2026-02-17T06:06:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 msgr2=0x7f951c06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.115:3300/0 conn(0x7f951c09d430 0x7f951c06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 -- 10.20.193.78:0/3745425645 >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 msgr2=0x7f951c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 --2- 10.20.193.78:0/3745425645 >> v2:10.20.193.78:3300/0 conn(0x7f951c06c890 0x7f951c06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 -- 10.20.193.78:0/3745425645 >> v1:10.20.193.115:6789/0 conn(0x7f951c09bf40 legacy=0x7f951c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 10 monclient: _add_conns ranks=[2,0] 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 10 monclient(hunting): picked mon.noname-b con 0x7f94f8003810 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f94f8006370 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 -- 10.20.193.78:0/3745425645 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f951c06dab0 con 0x7f94f8006370 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 1 -- 10.20.193.78:0/3745425645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f951c09ca50 con 0x7f94f8003810 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9503fff640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:18.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v1:10.20.193.78:6789/0 conn(0x7f94f8006370 legacy=0x7f94f8008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 10 monclient(hunting): get_auth_request con 0x7f94f8003810 auth_method 0 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 msgr2=0x7f94f8005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 msgr2=0x7f94f8005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:18.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.613+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:18.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v1:10.20.193.78:6789/0 conn(0x7f94f8006370 legacy=0x7f94f8008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:06:18.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:18.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 10 monclient(hunting): get_auth_request con 0x7f94f8003810 auth_method 0 2026-02-17T06:06:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 msgr2=0x7f94f8005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 msgr2=0x7f94f8005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:18.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:18.813+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:19.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v1:10.20.193.78:6789/0 conn(0x7f94f8006370 legacy=0x7f94f8008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 10 monclient(hunting): get_auth_request con 0x7f94f8003810 auth_method 0 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 msgr2=0x7f94f8005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 msgr2=0x7f94f8005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:19.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:19.213+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:19.590 INFO:tasks.rados.rados.0.trial115.stdout:3250: done (4 left) 2026-02-17T06:06:19.590 INFO:tasks.rados.rados.0.trial115.stdout:3251: done (3 left) 2026-02-17T06:06:19.590 INFO:tasks.rados.rados.0.trial115.stdout:3252: done (2 left) 2026-02-17T06:06:19.590 INFO:tasks.rados.rados.0.trial115.stdout:3253: done (1 left) 2026-02-17T06:06:19.590 INFO:tasks.rados.rados.0.trial115.stdout:3254: done (0 left) 2026-02-17T06:06:19.590 INFO:tasks.rados.rados.0.trial115.stdout:3255: read oid 45 snap 337 2026-02-17T06:06:19.590 INFO:tasks.rados.rados.0.trial115.stdout:3255: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:06:19.590 INFO:tasks.rados.rados.0.trial115.stdout:3256: read oid 12 snap -1 2026-02-17T06:06:19.591 INFO:tasks.rados.rados.0.trial115.stdout:3256: expect deleted 2026-02-17T06:06:19.591 INFO:tasks.rados.rados.0.trial115.stdout:3257: copy_from oid 33 from oid 11 current snap is 339 2026-02-17T06:06:19.591 INFO:tasks.rados.rados.0.trial115.stdout:3258: delete oid 23 current snap is 339 2026-02-17T06:06:19.592 INFO:tasks.rados.rados.0.trial115.stdout:3258: done (3 left) 2026-02-17T06:06:19.592 INFO:tasks.rados.rados.0.trial115.stdout:3259: snap_remove snap 338 2026-02-17T06:06:20.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f9519d74640 1 -- 10.20.193.78:0/3745425645 >> v1:10.20.193.78:6789/0 conn(0x7f94f8006370 legacy=0x7f94f8008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:06:20.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient(hunting): get_auth_request con 0x7f94f8003810 auth_method 0 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient(hunting): handle_auth_done global_id 234266 payload 274 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> v1:10.20.193.78:6789/0 conn(0x7f94f8006370 legacy=0x7f94f8008830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient: _finish_hunting 0 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 1 monclient: found mon.noname-b 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.014+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f800b1e0 con 0x7f94f8003810 2026-02-17T06:06:20.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f951a575640 1 -- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 msgr2=0x7f94f8005c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f951a575640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9510009d30 tx=0x7f951002fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1329fb4a05f5138 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 1 -- 10.20.193.78:0/3745425645 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951003d070 con 0x7f94f8003810 2026-02-17T06:06:20.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:20.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: dump: 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: _finish_auth 0 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: _check_auth_tickets 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 1 -- 10.20.193.78:0/3745425645 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9510030d10 con 0x7f94f8003810 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 1 -- 10.20.193.78:0/3745425645 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510038660 con 0x7f94f8003810 2026-02-17T06:06:20.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9520efa640 5 monclient: authenticate success, global_id 234266 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9520efa640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9520efa640 4 set_mon_vals no callback set 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.017+0000 7f9519573640 10 monclient: dump: 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 10 monclient: shutdown 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 1 -- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 msgr2=0x7f94f8005c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 1 --2- 10.20.193.78:0/3745425645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f94f8003810 0x7f94f8005c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9510009d30 tx=0x7f951002fdf0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f951ad76640 1 -- 10.20.193.78:0/3745425645 reap_dead start 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 1 -- 10.20.193.78:0/3745425645 shutdown_connections 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 1 -- 10.20.193.78:0/3745425645 >> 10.20.193.78:0/3745425645 conn(0x7f951c0a6620 msgr2=0x7f951c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 1 -- 10.20.193.78:0/3745425645 shutdown_connections 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 1 -- 10.20.193.78:0/3745425645 wait complete. 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 10 monclient: build_initial_monmap 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.018+0000 7f9520efa640 10 monclient: monmap: 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 1 Processor -- start 2026-02-17T06:06:20.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 1 -- start start 2026-02-17T06:06:20.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient: init 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94f8003810 0x7f951c121790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient(hunting): picked mon.noname-a con 0x7f94f8003810 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f951c121cd0 0x7f951c1220d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient(hunting): picked mon.noname-b con 0x7f951c121cd0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f951c122610 0x7f951c1d5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient(hunting): picked mon.noname-c con 0x7f951c122610 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f951c09c800 con 0x7f94f8003810 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f951c073410 con 0x7f951c121cd0 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f951c135500 con 0x7f951c122610 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951a575640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94f8003810 msgr2=0x7f951c121790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9520efa640 10 monclient(hunting): authenticate will time out at 1825.573420s 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951a575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94f8003810 0x7f951c121790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f951c122610 0x7f951c1d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f951c122610 0x7f951c1d5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43914/0 (socket says 10.20.193.78:43914) 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 -- 10.20.193.78:0/3625538851 learned_addr learned my addr 10.20.193.78:0/3625538851 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient(hunting): get_auth_request con 0x7f951c122610 auth_method 0 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9519d74640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f951c121cd0 0x7f951c1220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9519d74640 10 monclient(hunting): get_auth_request con 0x7f951c121cd0 auth_method 0 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9519d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9519d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f9519d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient(hunting): handle_auth_done global_id 234269 payload 824 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 -- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f951c121cd0 msgr2=0x7f951c1220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f951c121cd0 0x7f951c1220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 -- 10.20.193.78:0/3625538851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94f8003810 msgr2=0x7f951c121790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94f8003810 0x7f951c121790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 monclient: found mon.noname-c 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.019+0000 7f951ad76640 1 -- 10.20.193.78:0/3625538851 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c123bd0 con 0x7f951c122610 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.021+0000 7f951ad76640 1 -- 10.20.193.78:0/3625538851 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f951c122610 msgr2=0x7f951c1d5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.021+0000 7f951ad76640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f951c122610 0x7f951c1d5320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94f800a070 tx=0x7f950c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.021+0000 7f9502ffd640 1 -- 10.20.193.78:0/3625538851 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c017070 con 0x7f951c122610 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.021+0000 7f9502ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 10 monclient: dump: 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 10 monclient: _finish_auth 0 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 1 -- 10.20.193.78:0/3625538851 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f950c00ac60 con 0x7f951c122610 2026-02-17T06:06:20.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 5 monclient: authenticate success, global_id 234269 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 1 -- 10.20.193.78:0/3625538851 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c012400 con 0x7f951c122610 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9518d72640 4 set_mon_vals no callback set 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 10 monclient: dump: 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 10 monclient: _renew_subs 2026-02-17T06:06:20.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c137db0 con 0x7f951c122610 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 10 monclient: _renew_subs 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c125080 con 0x7f951c122610 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 1 -- 10.20.193.78:0/3625538851 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f950c01b3b0 con 0x7f951c122610 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951c0a0630 con 0x7f951c122610 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94e803de60 0x7f94e8040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.022+0000 7f9502ffd640 1 -- 10.20.193.78:0/3625538851 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(700..700 src has 525..700) ==== 4767+0+0 (secure 0 0 0) 0x7f950c052320 con 0x7f951c122610 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.023+0000 7f951a575640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94e803de60 0x7f94e8040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.023+0000 7f951a575640 10 monclient: get_auth_request con 0x7f94e803de60 auth_method 0 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.025+0000 7f9502ffd640 1 -- 10.20.193.78:0/3625538851 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f951c0a0630 con 0x7f951c122610 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.025+0000 7f9502ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.025+0000 7f9502ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:20.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.025+0000 7f951a575640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94e803de60 0x7f94e8040320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9510037000 tx=0x7f9510034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.138+0000 7f9520efa640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:20.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.138+0000 7f9520efa640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:20.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.138+0000 7f9520efa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.138+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f951c135890 con 0x7f951c122610 2026-02-17T06:06:20.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.138+0000 7f9502ffd640 1 -- 10.20.193.78:0/3625538851 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f950c050d30 con 0x7f951c122610 2026-02-17T06:06:20.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.138+0000 7f9502ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:20.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.138+0000 7f9502ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:20.141 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":234,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94e803de60 msgr2=0x7f94e8040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94e803de60 0x7f94e8040320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9510037000 tx=0x7f9510034000 comp rx=0 tx=0).stop 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 10 monclient: shutdown 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f951c122610 msgr2=0x7f951c1d5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f951c122610 0x7f951c1d5320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94f800a070 tx=0x7f950c00cd70 comp rx=0 tx=0).stop 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 shutdown_connections 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f951c122610 0x7f951c1d5320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f94e803de60 0x7f94e8040320 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f951c121cd0 0x7f951c1220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 --2- 10.20.193.78:0/3625538851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f94f8003810 0x7f951c121790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 >> 10.20.193.78:0/3625538851 conn(0x7f951c0a6620 msgr2=0x7f951c1372a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:20.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 shutdown_connections 2026-02-17T06:06:20.144 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.139+0000 7f9520efa640 1 -- 10.20.193.78:0/3625538851 wait complete. 2026-02-17T06:06:20.149 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:06:20.149 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:06:20.149 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:06:20.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 10 monclient: get_monmap_and_config 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 10 monclient: build_initial_monmap 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 10 monclient: monmap: 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:20.217453+0000 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:20.217453+0000 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:20.217 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 1 Processor -- start 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 1 -- start start 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 10 monclient: init 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0bb806c890 0x7f0bb806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 10 monclient(hunting): picked mon.noname-a con 0x7f0bb806c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.216+0000 7f0bc02f3640 10 monclient(hunting): picked mon.noname-c con 0x7f0bb809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bc02f3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bb8134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bc02f3640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb806daf0 con 0x7f0bb8134df0 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bc02f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bc02f3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb80a0880 con 0x7f0bb806c890 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bc02f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bc02f3640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb806d6f0 con 0x7f0bb809c4a0 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bc02f3640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bc02f3640 10 monclient(hunting): authenticate will time out at 1825.771021s 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbe068640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f0bb806c890 msgr2=0x7f0bb806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbe869640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f0bb8134df0 legacy=0x7f0bb8135280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbe068640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0bb806c890 0x7f0bb806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:48234/0 (socket says 10.20.193.78:48234) 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 1 -- 10.20.193.78:0/1489572383 learned_addr learned my addr 10.20.193.78:0/1489572383 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient(hunting): get_auth_request con 0x7f0bb809c4a0 auth_method 0 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient(hunting): handle_auth_done global_id 214342 payload 274 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 1 -- 10.20.193.78:0/1489572383 >> v2:10.20.193.78:3300/0 conn(0x7f0bb806c890 msgr2=0x7f0bb806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 1 --2- 10.20.193.78:0/1489572383 >> v2:10.20.193.78:3300/0 conn(0x7f0bb806c890 0x7f0bb806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 1 -- 10.20.193.78:0/1489572383 >> v1:10.20.193.78:6789/0 conn(0x7f0bb8134df0 legacy=0x7f0bb8135280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient: _finish_hunting 0 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 1 monclient: found mon.noname-c 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.217+0000 7f0bbd867640 1 -- 10.20.193.78:0/1489572383 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb809cc80 con 0x7f0bb809c4a0 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.218+0000 7f0bbd867640 1 -- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 msgr2=0x7f0bb806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.218+0000 7f0bbd867640 1 -- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 msgr2=0x7f0bb806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.218+0000 7f0bbd867640 1 --2- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f0bb809c980 tx=0x7f0bac02ee80 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:20.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.218+0000 7f0bbd867640 1 --2- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd867640 1 --2- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd867640 10 monclient: get_auth_request con 0x7f0bb809c4a0 auth_method 0 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd867640 0 --2- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd867640 1 --2- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 10 monclient: ms_handle_reset current mon v2:10.20.193.115:3300/0 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 1 -- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 msgr2=0x7f0bb806d1b0 unknown :-1 s=STATE_CLOSED l=0).mark_down 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 1 --2- 10.20.193.78:0/1489572383 >> v2:10.20.193.115:3300/0 conn(0x7f0bb809c4a0 0x7f0bb806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 10 monclient: _add_conns ranks=[2,1] 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 1 --2- 10.20.193.78:0/1489572383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ba0003600 0x7f0ba0005a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 10 monclient(hunting): picked mon.noname-b con 0x7f0ba0003600 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0ba0006320 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 1 -- 10.20.193.78:0/1489572383 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba000a290 con 0x7f0ba0006320 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 1 -- 10.20.193.78:0/1489572383 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba000b4e0 con 0x7f0ba0003600 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbd066640 10 monclient(hunting): ms_handle_reset stray mon v2:10.20.193.115:3300/0 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 1 --2- 10.20.193.78:0/1489572383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ba0003600 0x7f0ba0005a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient(hunting): get_auth_request con 0x7f0ba0003600 auth_method 0 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient(hunting): handle_auth_done global_id 214342 payload 293 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 1 -- 10.20.193.78:0/1489572383 >> v1:10.20.193.115:6789/0 conn(0x7f0ba0006320 legacy=0x7f0ba00087e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:20.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient: _finish_hunting 0 2026-02-17T06:06:20.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 1 monclient: found mon.noname-b 2026-02-17T06:06:20.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:20.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.419+0000 7f0bbe068640 1 -- 10.20.193.78:0/1489572383 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba000b7a0 con 0x7f0ba0003600 2026-02-17T06:06:20.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbe068640 1 -- 10.20.193.78:0/1489572383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ba0003600 msgr2=0x7f0ba0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbe068640 1 --2- 10.20.193.78:0/1489572383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ba0003600 0x7f0ba0005a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009db0 tx=0x7f0bac009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0aa438db1cbb1ce server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 1 -- 10.20.193.78:0/1489572383 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8009a30 con 0x7f0ba0003600 2026-02-17T06:06:20.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: dump: 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: _finish_auth 0 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: _check_auth_tickets 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bc02f3640 5 monclient: authenticate success, global_id 214342 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 1 -- 10.20.193.78:0/1489572383 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba8003cf0 con 0x7f0ba0003600 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bc02f3640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bc02f3640 4 set_mon_vals no callback set 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 1 -- 10.20.193.78:0/1489572383 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8005310 con 0x7f0ba0003600 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.421+0000 7f0bbd066640 10 monclient: dump: 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 10 monclient: shutdown 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:20.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 1 -- 10.20.193.78:0/1489572383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ba0003600 msgr2=0x7f0ba0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 1 --2- 10.20.193.78:0/1489572383 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0ba0003600 0x7f0ba0005a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009db0 tx=0x7f0bac009340 comp rx=0 tx=0).stop 2026-02-17T06:06:20.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bbe869640 1 -- 10.20.193.78:0/1489572383 reap_dead start 2026-02-17T06:06:20.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 1 -- 10.20.193.78:0/1489572383 shutdown_connections 2026-02-17T06:06:20.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 1 -- 10.20.193.78:0/1489572383 >> 10.20.193.78:0/1489572383 conn(0x7f0bb80a6620 msgr2=0x7f0bb80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:20.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 1 -- 10.20.193.78:0/1489572383 shutdown_connections 2026-02-17T06:06:20.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 1 -- 10.20.193.78:0/1489572383 wait complete. 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 10 monclient: build_initial_monmap 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 10 monclient: monmap: 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 1 Processor -- start 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 1 -- start start 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.422+0000 7f0bc02f3640 10 monclient: init 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ba0003600 0x7f0bb81cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient(hunting): picked mon.noname-b con 0x7f0ba0003600 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bb8134150 0x7f0bb81345b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient(hunting): picked mon.noname-a con 0x7f0bb8134150 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb8123be0 0x7f0bb8124060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient(hunting): picked mon.noname-c con 0x7f0bb8123be0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb80a01a0 con 0x7f0bb8134150 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb80a0020 con 0x7f0ba0003600 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb809be20 con 0x7f0bb8123be0 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bc02f3640 10 monclient(hunting): authenticate will time out at 1825.977310s 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbd867640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bb8134150 msgr2=0x7f0bb81345b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:20.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbd867640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bb8134150 0x7f0bb81345b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb8123be0 0x7f0bb8124060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb8123be0 0x7f0bb8124060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43936/0 (socket says 10.20.193.78:43936) 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 -- 10.20.193.78:0/4203979148 learned_addr learned my addr 10.20.193.78:0/4203979148 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient(hunting): get_auth_request con 0x7f0bb8123be0 auth_method 0 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe068640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ba0003600 0x7f0bb81cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe068640 10 monclient(hunting): get_auth_request con 0x7f0ba0003600 auth_method 0 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe068640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe068640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe068640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient(hunting): handle_auth_done global_id 234272 payload 824 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ba0003600 msgr2=0x7f0bb81cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0ba0003600 0x7f0bb81cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bb8134150 msgr2=0x7f0bb81345b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bb8134150 0x7f0bb81345b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient: _finish_hunting 0 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 monclient: found mon.noname-c 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.423+0000 7f0bbe869640 1 -- 10.20.193.78:0/4203979148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb8134af0 con 0x7f0bb8123be0 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bbe869640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb8123be0 msgr2=0x7f0bb8124060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bbe869640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb8123be0 0x7f0bb8124060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400bac0 tx=0x7f0bb400bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400ce60 con 0x7f0bb8123be0 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: dump: 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.427 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: _finish_auth 0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bc02f3640 5 monclient: authenticate success, global_id 234272 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb4010030 con 0x7f0bb8123be0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4015860 con 0x7f0bb8123be0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bbc865640 4 set_mon_vals no callback set 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0b9effd640 10 monclient: dump: 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bc02f3640 10 monclient: _renew_subs 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bc02f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb8127890 con 0x7f0bb8123be0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bc02f3640 10 monclient: _renew_subs 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bc02f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.425+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb8125880 con 0x7f0bb8123be0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bc02f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bc02f3640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb81245a0 0x7f0bb809dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bc02f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb80a6f60 con 0x7f0bb81245a0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb81245a0 0x7f0bb809dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 10 monclient: get_auth_request con 0x7f0bb81245a0 auth_method 0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0bb40143d0 con 0x7f0bb8123be0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 10 monclient(hunting): handle_auth_done global_id 234275 payload 824 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbe068640 1 -- 10.20.193.78:0/4203979148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bb8135a20 con 0x7f0bb81245a0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0b9effd640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b8803de60 0x7f0b88040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(700..700 src has 525..700) ==== 4767+0+0 (secure 0 0 0) 0x7f0bb4052620 con 0x7f0bb8123be0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.426+0000 7f0bbd867640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b8803de60 0x7f0b88040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.427+0000 7f0bbd867640 10 monclient: get_auth_request con 0x7f0b8803de60 auth_method 0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.428+0000 7f0bbe068640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb81245a0 msgr2=0x7f0bb809dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.428+0000 7f0bbe068640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb81245a0 0x7f0bb809dc00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8014000 tx=0x7f0ba8005b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.428+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80052d0 con 0x7f0bb81245a0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.428+0000 7f0b9effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.428+0000 7f0b9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.428+0000 7f0b9effd640 10 monclient: dump: 2026-02-17T06:06:20.429 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.429+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f0ba8026070 con 0x7f0bb81245a0 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.429+0000 7f0b9effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.429+0000 7f0b9effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.429+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb81245a0 msgr2=0x7f0bb809dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.429+0000 7f0b9effd640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb81245a0 0x7f0bb809dc00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8014000 tx=0x7f0ba8005b10 comp rx=0 tx=0).stop 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.429+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb81245a0 msgr2=0x7f0bb809dc00 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.429+0000 7f0b9effd640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb81245a0 0x7f0bb809dc00 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8014000 tx=0x7f0ba8005b10 comp rx=0 tx=0).stop 2026-02-17T06:06:20.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.429+0000 7f0bbd867640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b8803de60 0x7f0b88040320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8125480 tx=0x7f0bac033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bc02f3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:20.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bc02f3640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb809e140 0x7f0bb809e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bc02f3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb81cf910 con 0x7f0bb809e140 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb809e140 0x7f0bb809e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 10 monclient: get_auth_request con 0x7f0bb809e140 auth_method 0 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 10 monclient(hunting): handle_auth_done global_id 234278 payload 824 2026-02-17T06:06:20.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.433+0000 7f0bbe068640 1 -- 10.20.193.78:0/4203979148 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0bb8124980 con 0x7f0bb809e140 2026-02-17T06:06:20.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0bbe068640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb809e140 msgr2=0x7f0bb809e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0bbe068640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb809e140 0x7f0bb809e550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bb806c650 tx=0x7f0ba8010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8005970 con 0x7f0bb809e140 2026-02-17T06:06:20.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 10 monclient: dump: 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7f0ba8005970 con 0x7f0bb809e140 2026-02-17T06:06:20.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:20.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:20.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb809e140 msgr2=0x7f0bb809e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb809e140 0x7f0bb809e550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bb806c650 tx=0x7f0ba8010000 comp rx=0 tx=0).stop 2026-02-17T06:06:20.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb809e140 msgr2=0x7f0bb809e550 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:20.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0b9effd640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb809e140 0x7f0bb809e550 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bb806c650 tx=0x7f0ba8010000 comp rx=0 tx=0).stop 2026-02-17T06:06:20.439 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:20.439 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 234, 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 89118, 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:06:20.440 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:20.441 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:20.442 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:20.443 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:20.444 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:20.445 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.446 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:20.447 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:20.448 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:20.449 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:20.450 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:20.451 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:06:20.452 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.452 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.452 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.452 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:20.452 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:20.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b8803de60 msgr2=0x7f0b88040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.435+0000 7f0bc02f3640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b8803de60 0x7f0b88040320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8125480 tx=0x7f0bac033000 comp rx=0 tx=0).stop 2026-02-17T06:06:20.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 10 monclient: shutdown 2026-02-17T06:06:20.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:20.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb8123be0 msgr2=0x7f0bb8124060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb8123be0 0x7f0bb8124060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400bac0 tx=0x7f0bb400bec0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bbe869640 1 -- 10.20.193.78:0/4203979148 reap_dead start 2026-02-17T06:06:20.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 shutdown_connections 2026-02-17T06:06:20.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 1 --2- 10.20.193.78:0/4203979148 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bb8123be0 0x7f0bb8124060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 >> 10.20.193.78:0/4203979148 conn(0x7f0bb80a6620 msgr2=0x7f0bb8135c30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:20.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 shutdown_connections 2026-02-17T06:06:20.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.436+0000 7f0bc02f3640 1 -- 10.20.193.78:0/4203979148 wait complete. 2026-02-17T06:06:20.454 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:06:20.454 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T06:06:20.454 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:06:20.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient: get_monmap_and_config 2026-02-17T06:06:20.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient: build_initial_monmap 2026-02-17T06:06:20.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient: monmap: 2026-02-17T06:06:20.522 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:20.522 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.522 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:20.520260+0000 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:20.520260+0000 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 1 Processor -- start 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 1 -- start start 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient: init 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2026-02-17T06:06:20.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc06c890 0x7f6bbc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient(hunting): picked mon.noname-b con 0x7f6bbc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6bbc09c4a0 0x7f6bbc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient(hunting): picked mon.noname-a con 0x7f6bbc09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6bbc134df0 0x7f6bbc135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient(hunting): picked mon.noname-c con 0x7f6bbc134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6bbc0a0880 con 0x7f6bbc09c4a0 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6bbc06d6f0 con 0x7f6bbc134df0 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc06daf0 con 0x7f6bbc06c890 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc417e640 10 monclient(hunting): authenticate will time out at 1826.073762s 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc16f2640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f6bbc09c4a0 msgr2=0x7f6bbc06d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:20.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc16f2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6bbc09c4a0 0x7f6bbc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc06c890 0x7f6bbc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc06c890 0x7f6bbc06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43964/0 (socket says 10.20.193.78:43964) 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/3094280391 learned_addr learned my addr 10.20.193.78:0/3094280391 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 10 monclient(hunting): get_auth_request con 0x7f6bbc06c890 auth_method 0 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc26f4640 1 --2- 10.20.193.78:0/3094280391 >> v2:10.20.193.115:3300/0 conn(0x7f6bbc134df0 0x7f6bbc135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc26f4640 10 monclient(hunting): get_auth_request con 0x7f6bbc134df0 auth_method 0 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc26f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc26f4640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.519+0000 7f6bc26f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_done global_id 234281 payload 274 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/3094280391 >> v2:10.20.193.115:3300/0 conn(0x7f6bbc134df0 msgr2=0x7f6bbc135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/3094280391 >> v2:10.20.193.115:3300/0 conn(0x7f6bbc134df0 0x7f6bbc135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/3094280391 >> v2:10.20.193.78:3300/0 conn(0x7f6bbc09c4a0 msgr2=0x7f6bbc06d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/3094280391 >> v2:10.20.193.78:3300/0 conn(0x7f6bbc09c4a0 0x7f6bbc06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 10 monclient: _finish_hunting 0 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 1 monclient: found mon.noname-b 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.520+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/3094280391 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc09cc80 con 0x7f6bbc06c890 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/3094280391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc06c890 msgr2=0x7f6bbc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/3094280391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc06c890 0x7f6bbc06cc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc09c880 tx=0x7f6bac02ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a45ea9f5f617f4 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 1 -- 10.20.193.78:0/3094280391 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac03c070 con 0x7f6bbc06c890 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: dump: 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.533 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: _finish_auth 0 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: _check_auth_tickets 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 1 -- 10.20.193.78:0/3094280391 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bac02fcc0 con 0x7f6bbc06c890 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc417e640 5 monclient: authenticate success, global_id 234281 2026-02-17T06:06:20.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 1 -- 10.20.193.78:0/3094280391 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac037660 con 0x7f6bbc06c890 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc0ef1640 10 monclient: dump: 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.535 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc417e640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.522+0000 7f6bc417e640 4 set_mon_vals no callback set 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 10 monclient: shutdown 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 -- 10.20.193.78:0/3094280391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc06c890 msgr2=0x7f6bbc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 --2- 10.20.193.78:0/3094280391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc06c890 0x7f6bbc06cc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc09c880 tx=0x7f6bac02ee80 comp rx=0 tx=0).stop 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 -- 10.20.193.78:0/3094280391 shutdown_connections 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 --2- 10.20.193.78:0/3094280391 >> v2:10.20.193.115:3300/0 conn(0x7f6bbc134df0 0x7f6bbc135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 --2- 10.20.193.78:0/3094280391 >> v2:10.20.193.78:3300/0 conn(0x7f6bbc09c4a0 0x7f6bbc06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 --2- 10.20.193.78:0/3094280391 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc06c890 0x7f6bbc06cc70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 -- 10.20.193.78:0/3094280391 >> 10.20.193.78:0/3094280391 conn(0x7f6bbc0a6620 msgr2=0x7f6bbc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 -- 10.20.193.78:0/3094280391 shutdown_connections 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 -- 10.20.193.78:0/3094280391 wait complete. 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 10 monclient: build_initial_monmap 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 10 monclient: monmap: 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:20.537 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 Processor -- start 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 1 -- start start 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.523+0000 7f6bc417e640 10 monclient: init 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc06c890 0x7f6bbc1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient(hunting): picked mon.noname-b con 0x7f6bbc06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bbc09c4a0 0x7f6bbc1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient(hunting): picked mon.noname-a con 0x7f6bbc09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc134df0 0x7f6bbc1215f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient(hunting): picked mon.noname-c con 0x7f6bbc134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc0a01a0 con 0x7f6bbc09c4a0 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc0a0020 con 0x7f6bbc06c890 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc09be20 con 0x7f6bbc134df0 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc417e640 10 monclient(hunting): authenticate will time out at 1826.078063s 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc16f2640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bbc09c4a0 msgr2=0x7f6bbc1cd870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc16f2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bbc09c4a0 0x7f6bbc1cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc134df0 0x7f6bbc1215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc134df0 0x7f6bbc1215f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:43976/0 (socket says 10.20.193.78:43976) 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 -- 10.20.193.78:0/1104002320 learned_addr learned my addr 10.20.193.78:0/1104002320 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient(hunting): get_auth_request con 0x7f6bbc134df0 auth_method 0 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc06c890 0x7f6bbc1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient(hunting): handle_auth_done global_id 234284 payload 824 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc06c890 msgr2=0x7f6bbc1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc06c890 0x7f6bbc1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bbc09c4a0 msgr2=0x7f6bbc1cd870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6bbc09c4a0 0x7f6bbc1cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient: _finish_hunting 0 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 monclient: found mon.noname-c 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc26f4640 1 -- 10.20.193.78:0/1104002320 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc1252a0 con 0x7f6bbc134df0 2026-02-17T06:06:20.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc1ef3640 10 monclient: get_auth_request con 0x7f6bbc06c890 auth_method 0 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.524+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc06c890 0x7f6bbc1cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc26f4640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc134df0 msgr2=0x7f6bbc1215f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc26f4640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc134df0 0x7f6bbc1215f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007d90 tx=0x7f6bb800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8013790 con 0x7f6bbc134df0 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: dump: 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.541 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: _finish_auth 0 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc417e640 5 monclient: authenticate success, global_id 234284 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb8019070 con 0x7f6bbc134df0 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc417e640 10 monclient: _renew_subs 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc417e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc123df0 con 0x7f6bbc134df0 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8013b90 con 0x7f6bbc134df0 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:20.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6babfff640 4 set_mon_vals no callback set 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6baa7fc640 10 monclient: dump: 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc417e640 10 monclient: _renew_subs 2026-02-17T06:06:20.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc417e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.526+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc0725d0 con 0x7f6bbc134df0 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc417e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc417e640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1222f0 0x7f6bbc11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc417e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc074980 con 0x7f6bbc1222f0 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6bb801e460 con 0x7f6bbc134df0 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1222f0 0x7f6bbc11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 10 monclient: get_auth_request con 0x7f6bbc1222f0 auth_method 0 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6baa7fc640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b8c03de90 0x7f6b8c040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(700..700 src has 525..700) ==== 4767+0+0 (secure 0 0 0) 0x7f6bb80530f0 con 0x7f6bbc134df0 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc16f2640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b8c03de90 0x7f6b8c040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc16f2640 10 monclient: get_auth_request con 0x7f6b8c03de90 auth_method 0 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_done global_id 214351 payload 824 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.527+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/1104002320 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bbc0a15d0 con 0x7f6bbc1222f0 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1222f0 msgr2=0x7f6bbc11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1222f0 0x7f6bbc11cc70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0a7600 tx=0x7f6bac004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6bc16f2640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b8c03de90 0x7f6b8c040350 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0003a80 tx=0x7f6bb0009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac046020 con 0x7f6bbc1222f0 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 10 monclient: dump: 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f6bac042420 con 0x7f6bbc1222f0 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:20.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1222f0 msgr2=0x7f6bbc11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1222f0 0x7f6bbc11cc70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0a7600 tx=0x7f6bac004830 comp rx=0 tx=0).stop 2026-02-17T06:06:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1222f0 msgr2=0x7f6bbc11cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:20.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.530+0000 7f6baa7fc640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1222f0 0x7f6bbc11cc70 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0a7600 tx=0x7f6bac004830 comp rx=0 tx=0).stop 2026-02-17T06:06:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.533+0000 7f6bc417e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.533+0000 7f6bc417e640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1226d0 0x7f6bbc1d8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.533+0000 7f6bc417e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.533+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc135240 con 0x7f6bbc1226d0 2026-02-17T06:06:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1226d0 0x7f6bbc1d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 10 monclient: get_auth_request con 0x7f6bbc1226d0 auth_method 0 2026-02-17T06:06:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:20.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 10 monclient(hunting): handle_auth_done global_id 214354 payload 824 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.534+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/1104002320 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6bbc11e330 con 0x7f6bbc1226d0 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.536+0000 7f6bc1ef3640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1226d0 msgr2=0x7f6bbc1d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.536+0000 7f6bc1ef3640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1226d0 0x7f6bbc1d8540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02eec0 tx=0x7f6bac0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac046070 con 0x7f6bbc1226d0 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 10 monclient: dump: 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:20.550 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7f6bac04db80 con 0x7f6bbc1226d0 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1226d0 msgr2=0x7f6bbc1d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1226d0 0x7f6bbc1d8540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02eec0 tx=0x7f6bac0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1226d0 msgr2=0x7f6bbc1d8540 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:20.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6baa7fc640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6bbc1226d0 0x7f6bbc1d8540 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02eec0 tx=0x7f6bac0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 234, 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 183385, 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:20.552 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:20.553 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:20.554 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:20.555 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.556 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:20.557 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.558 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:20.559 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:20.560 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:20.561 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:20.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b8c03de90 msgr2=0x7f6b8c040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6b8c03de90 0x7f6b8c040350 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0003a80 tx=0x7f6bb0009270 comp rx=0 tx=0).stop 2026-02-17T06:06:20.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 10 monclient: shutdown 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc134df0 msgr2=0x7f6bbc1215f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 1 --2- 10.20.193.78:0/1104002320 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6bbc134df0 0x7f6bbc1215f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007d90 tx=0x7f6bb800e4b0 comp rx=0 tx=0).stop 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc26f4640 1 -- 10.20.193.78:0/1104002320 reap_dead start 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc26f4640 1 -- 10.20.193.78:0/1104002320 reap_dead start 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 shutdown_connections 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 >> 10.20.193.78:0/1104002320 conn(0x7f6bbc0a6620 msgr2=0x7f6bbc136460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 shutdown_connections 2026-02-17T06:06:20.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:20.537+0000 7f6bc417e640 1 -- 10.20.193.78:0/1104002320 wait complete. 2026-02-17T06:06:20.563 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:06:20.563 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:06:20.592 INFO:tasks.rados.rados.0.trial115.stdout:3259: done (3 left) 2026-02-17T06:06:20.592 INFO:tasks.rados.rados.0.trial115.stdout:3260: read oid 50 snap -1 2026-02-17T06:06:20.592 INFO:tasks.rados.rados.0.trial115.stdout:3260: expect (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:06:20.592 INFO:tasks.rados.rados.0.trial115.stdout:3261: rollback oid 15 current snap is 339 2026-02-17T06:06:20.592 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 15 to 334 2026-02-17T06:06:20.592 INFO:tasks.rados.rados.0.trial115.stdout:3255: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:06:20.710 INFO:tasks.rados.rados.0.trial115.stdout:3257: finishing copy_from racing read to trial11545420-33 2026-02-17T06:06:20.710 INFO:tasks.rados.rados.0.trial115.stdout:3257: finishing copy_from to trial11545420-33 2026-02-17T06:06:20.710 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1651 (ObjNum 1121 snap 336 seq_num 1121) dirty exists 2026-02-17T06:06:20.710 INFO:tasks.rados.rados.0.trial115.stdout:3261: finishing rollback tid 1 to trial11545420-15 2026-02-17T06:06:20.710 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:06:20.710 INFO:tasks.rados.rados.0.trial115.stdout:3260: expect (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:06:20.808 INFO:tasks.rados.rados.0.trial115.stdout:3255: done (4 left) 2026-02-17T06:06:20.808 INFO:tasks.rados.rados.0.trial115.stdout:3256: done (3 left) 2026-02-17T06:06:20.808 INFO:tasks.rados.rados.0.trial115.stdout:3257: done (2 left) 2026-02-17T06:06:20.808 INFO:tasks.rados.rados.0.trial115.stdout:3260: done (1 left) 2026-02-17T06:06:20.808 INFO:tasks.rados.rados.0.trial115.stdout:3261: done (0 left) 2026-02-17T06:06:20.808 INFO:tasks.rados.rados.0.trial115.stdout:3262: read oid 39 snap -1 2026-02-17T06:06:20.809 INFO:tasks.rados.rados.0.trial115.stdout:3262: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:06:20.809 INFO:tasks.rados.rados.0.trial115.stdout:3263: snap_create 2026-02-17T06:06:20.811 INFO:tasks.rados.rados.0.trial115.stdout:3262: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:06:21.272 INFO:tasks.rados.rados.0.trial115.stdout:3262: done (1 left) 2026-02-17T06:06:21.273 INFO:tasks.rados.rados.0.trial115.stdout:3263: done (0 left) 2026-02-17T06:06:21.273 INFO:tasks.rados.rados.0.trial115.stdout:3264: read oid 28 snap -1 2026-02-17T06:06:21.273 INFO:tasks.rados.rados.0.trial115.stdout:3264: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:06:21.273 INFO:tasks.rados.rados.0.trial115.stdout:3265: snap_remove snap 301 2026-02-17T06:06:22.272 INFO:tasks.rados.rados.0.trial115.stdout:3265: done (1 left) 2026-02-17T06:06:22.272 INFO:tasks.rados.rados.0.trial115.stdout:3266: delete oid 6 current snap is 340 2026-02-17T06:06:22.273 INFO:tasks.rados.rados.0.trial115.stdout:3264: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:06:22.403 INFO:tasks.rados.rados.0.trial115.stdout:3264: done (1 left) 2026-02-17T06:06:22.403 INFO:tasks.rados.rados.0.trial115.stdout:3266: done (0 left) 2026-02-17T06:06:22.403 INFO:tasks.rados.rados.0.trial115.stdout:3267: read oid 27 snap 336 2026-02-17T06:06:22.403 INFO:tasks.rados.rados.0.trial115.stdout:3267: expect deleted 2026-02-17T06:06:22.404 INFO:tasks.rados.rados.0.trial115.stdout:3268: read oid 16 snap 328 2026-02-17T06:06:22.404 INFO:tasks.rados.rados.0.trial115.stdout:3268: expect (ObjNum 1014 snap 294 seq_num 1014) 2026-02-17T06:06:22.404 INFO:tasks.rados.rados.0.trial115.stdout:3269: rollback oid 41 current snap is 340 2026-02-17T06:06:22.404 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 328 2026-02-17T06:06:22.404 INFO:tasks.rados.rados.0.trial115.stdout:3270: snap_create 2026-02-17T06:06:22.405 INFO:tasks.rados.rados.0.trial115.stdout:3269: finishing rollback tid 0 to trial11545420-41 2026-02-17T06:06:22.405 INFO:tasks.rados.rados.0.trial115.stdout:3269: finishing rollback tid 1 to trial11545420-41 2026-02-17T06:06:22.405 INFO:tasks.rados.rados.0.trial115.stdout:3269: finishing rollback tid 2 to trial11545420-41 2026-02-17T06:06:22.405 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1556 (ObjNum 1080 snap 325 seq_num 1080) dirty exists 2026-02-17T06:06:22.409 INFO:tasks.rados.rados.0.trial115.stdout:3268: expect (ObjNum 1014 snap 294 seq_num 1014) 2026-02-17T06:06:23.274 INFO:tasks.rados.rados.0.trial115.stdout:3267: done (3 left) 2026-02-17T06:06:23.274 INFO:tasks.rados.rados.0.trial115.stdout:3268: done (2 left) 2026-02-17T06:06:23.274 INFO:tasks.rados.rados.0.trial115.stdout:3269: done (1 left) 2026-02-17T06:06:23.274 INFO:tasks.rados.rados.0.trial115.stdout:3270: done (0 left) 2026-02-17T06:06:23.274 INFO:tasks.rados.rados.0.trial115.stdout:3271: read oid 46 snap -1 2026-02-17T06:06:23.274 INFO:tasks.rados.rados.0.trial115.stdout:3271: expect (ObjNum 1101 snap 329 seq_num 1101) 2026-02-17T06:06:23.274 INFO:tasks.rados.rados.0.trial115.stdout:3272: write oid 20 current snap is 341 2026-02-17T06:06:23.274 INFO:tasks.rados.rados.0.trial115.stdout:3272: seq_num 1131 ranges {608717=726863,1933605=748052} 2026-02-17T06:06:23.279 INFO:tasks.rados.rados.0.trial115.stdout:3272: writing trial11545420-20 from 608717 to 1335580 tid 1 2026-02-17T06:06:23.284 INFO:tasks.rados.rados.0.trial115.stdout:3272: writing trial11545420-20 from 1933605 to 2681657 tid 2 2026-02-17T06:06:23.284 INFO:tasks.rados.rados.0.trial115.stdout:3273: rollback oid 39 current snap is 341 2026-02-17T06:06:23.284 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 39 to 339 2026-02-17T06:06:23.284 INFO:tasks.rados.rados.0.trial115.stdout:3271: expect (ObjNum 1101 snap 329 seq_num 1101) 2026-02-17T06:06:23.361 INFO:tasks.rados.rados.0.trial115.stdout:3272: finishing write tid 1 to trial11545420-20 2026-02-17T06:06:23.361 INFO:tasks.rados.rados.0.trial115.stdout:3273: finishing rollback tid 0 to trial11545420-39 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3273: finishing rollback tid 1 to trial11545420-39 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3272: finishing write tid 2 to trial11545420-20 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3273: finishing rollback tid 2 to trial11545420-39 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1704 (ObjNum 1118 snap 336 seq_num 1118) dirty exists 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3272: finishing write tid 3 to trial11545420-20 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3272: finishing write tid 4 to trial11545420-20 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3272: oid 20 updating version 0 to 1478 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3272: oid 20 version 1478 is already newer than 1476 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3272: oid 20 version 1478 is already newer than 1477 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1478 (ObjNum 1131 snap 341 seq_num 1131) dirty exists 2026-02-17T06:06:23.362 INFO:tasks.rados.rados.0.trial115.stdout:3272: left oid 20 (ObjNum 1131 snap 341 seq_num 1131) 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:3271: done (2 left) 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:3272: done (1 left) 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:3273: done (0 left) 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:3274: rollback oid 17 current snap is 341 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 332 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:3275: read oid 43 snap -1 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:3275: expect (ObjNum 1092 snap 328 seq_num 1092) 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:3276: delete oid 28 current snap is 341 2026-02-17T06:06:23.363 INFO:tasks.rados.rados.0.trial115.stdout:3276: done (2 left) 2026-02-17T06:06:23.364 INFO:tasks.rados.rados.0.trial115.stdout:3277: snap_remove snap 334 2026-02-17T06:06:24.279 INFO:tasks.rados.rados.0.trial115.stdout:3277: done (2 left) 2026-02-17T06:06:24.279 INFO:tasks.rados.rados.0.trial115.stdout:3278: read oid 23 snap -1 2026-02-17T06:06:24.279 INFO:tasks.rados.rados.0.trial115.stdout:3278: expect deleted 2026-02-17T06:06:24.279 INFO:tasks.rados.rados.0.trial115.stdout:3279: read oid 14 snap -1 2026-02-17T06:06:24.279 INFO:tasks.rados.rados.0.trial115.stdout:3279: expect (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:06:24.279 INFO:tasks.rados.rados.0.trial115.stdout:3280: write oid 18 current snap is 341 2026-02-17T06:06:24.279 INFO:tasks.rados.rados.0.trial115.stdout:3280: seq_num 1133 ranges {642331=593714,1766922=751721} 2026-02-17T06:06:24.283 INFO:tasks.rados.rados.0.trial115.stdout:3280: writing trial11545420-18 from 642331 to 1236045 tid 1 2026-02-17T06:06:24.288 INFO:tasks.rados.rados.0.trial115.stdout:3280: writing trial11545420-18 from 1766922 to 2518643 tid 2 2026-02-17T06:06:24.289 INFO:tasks.rados.rados.0.trial115.stdout:3281: snap_remove snap 317 2026-02-17T06:06:25.279 INFO:tasks.rados.rados.0.trial115.stdout:3281: done (5 left) 2026-02-17T06:06:25.279 INFO:tasks.rados.rados.0.trial115.stdout:3282: write (excl) oid 25 current snap is 341 2026-02-17T06:06:25.280 INFO:tasks.rados.rados.0.trial115.stdout:3282: seq_num 1134 ranges {659138=668963,2109051=611732} 2026-02-17T06:06:25.285 INFO:tasks.rados.rados.0.trial115.stdout:3282: writing trial11545420-25 from 659138 to 1328101 tid 1 2026-02-17T06:06:25.290 INFO:tasks.rados.rados.0.trial115.stdout:3282: writing trial11545420-25 from 2109051 to 2720783 tid 2 2026-02-17T06:06:25.290 INFO:tasks.rados.rados.0.trial115.stdout:3283: rollback oid 5 current snap is 341 2026-02-17T06:06:25.291 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 5 to 340 2026-02-17T06:06:25.291 INFO:tasks.rados.rados.0.trial115.stdout:3274: finishing rollback tid 0 to trial11545420-17 2026-02-17T06:06:25.291 INFO:tasks.rados.rados.0.trial115.stdout:3274: finishing rollback tid 1 to trial11545420-17 2026-02-17T06:06:25.291 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1689 (ObjNum 177 snap 0 seq_num 1333676528) dirty dne 2026-02-17T06:06:25.291 INFO:tasks.rados.rados.0.trial115.stdout:3274: done (6 left) 2026-02-17T06:06:25.291 INFO:tasks.rados.rados.0.trial115.stdout:3284: snap_create 2026-02-17T06:06:25.291 INFO:tasks.rados.rados.0.trial115.stdout:3275: expect (ObjNum 1092 snap 328 seq_num 1092) 2026-02-17T06:06:25.392 INFO:tasks.rados.rados.0.trial115.stdout:3279: expect (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:06:25.528 INFO:tasks.rados.rados.0.trial115.stdout:3280: finishing write tid 1 to trial11545420-18 2026-02-17T06:06:25.528 INFO:tasks.rados.rados.0.trial115.stdout:3280: finishing write tid 2 to trial11545420-18 2026-02-17T06:06:25.528 INFO:tasks.rados.rados.0.trial115.stdout:3280: finishing write tid 3 to trial11545420-18 2026-02-17T06:06:25.528 INFO:tasks.rados.rados.0.trial115.stdout:3280: finishing write tid 4 to trial11545420-18 2026-02-17T06:06:25.528 INFO:tasks.rados.rados.0.trial115.stdout:3280: oid 18 updating version 0 to 1557 2026-02-17T06:06:25.528 INFO:tasks.rados.rados.0.trial115.stdout:3280: oid 18 updating version 1557 to 1559 2026-02-17T06:06:25.528 INFO:tasks.rados.rados.0.trial115.stdout:3280: oid 18 version 1559 is already newer than 1558 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1559 (ObjNum 1133 snap 341 seq_num 1133) dirty exists 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3280: left oid 18 (ObjNum 1133 snap 341 seq_num 1133) 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3282: finishing write tid 1 to trial11545420-25 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3282: finishing write tid 2 to trial11545420-25 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3282: finishing write tid 3 to trial11545420-25 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3282: finishing write tid 4 to trial11545420-25 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3282: oid 25 updating version 0 to 1560 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3282: oid 25 updating version 1560 to 1562 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3282: oid 25 version 1562 is already newer than 1561 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1562 (ObjNum 1134 snap 341 seq_num 1134) dirty exists 2026-02-17T06:06:25.529 INFO:tasks.rados.rados.0.trial115.stdout:3282: left oid 25 (ObjNum 1134 snap 341 seq_num 1134) 2026-02-17T06:06:25.530 INFO:tasks.rados.rados.0.trial115.stdout:3283: finishing rollback tid 0 to trial11545420-5 2026-02-17T06:06:25.530 INFO:tasks.rados.rados.0.trial115.stdout:3283: finishing rollback tid 1 to trial11545420-5 2026-02-17T06:06:25.530 INFO:tasks.rados.rados.0.trial115.stdout:3283: finishing rollback tid 2 to trial11545420-5 2026-02-17T06:06:25.530 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1484 (ObjNum 1125 snap 336 seq_num 1125) dirty exists 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3275: done (6 left) 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3278: done (5 left) 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3279: done (4 left) 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3280: done (3 left) 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3282: done (2 left) 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3283: done (1 left) 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3284: done (0 left) 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3285: write (excl) oid 3 current snap is 342 2026-02-17T06:06:26.274 INFO:tasks.rados.rados.0.trial115.stdout:3285: seq_num 1135 ranges {675945=744212,2167533=755390} 2026-02-17T06:06:26.279 INFO:tasks.rados.rados.0.trial115.stdout:3285: writing trial11545420-3 from 675945 to 1420157 tid 1 2026-02-17T06:06:26.285 INFO:tasks.rados.rados.0.trial115.stdout:3285: writing trial11545420-3 from 2167533 to 2922923 tid 2 2026-02-17T06:06:26.285 INFO:tasks.rados.rados.0.trial115.stdout:3286: copy_from oid 12 from oid 2 current snap is 342 2026-02-17T06:06:26.285 INFO:tasks.rados.rados.0.trial115.stdout:3285: finishing write tid 1 to trial11545420-3 2026-02-17T06:06:26.285 INFO:tasks.rados.rados.0.trial115.stdout:3287: write oid 22 current snap is 342 2026-02-17T06:06:26.285 INFO:tasks.rados.rados.0.trial115.stdout:3287: seq_num 1136 ranges {692752=419461,1826015=615401,2979461=113291} 2026-02-17T06:06:26.289 INFO:tasks.rados.rados.0.trial115.stdout:3287: writing trial11545420-22 from 692752 to 1112213 tid 1 2026-02-17T06:06:26.294 INFO:tasks.rados.rados.0.trial115.stdout:3287: writing trial11545420-22 from 1826015 to 2441416 tid 2 2026-02-17T06:06:26.296 INFO:tasks.rados.rados.0.trial115.stdout:3287: writing trial11545420-22 from 2979461 to 3092752 tid 3 2026-02-17T06:06:26.296 INFO:tasks.rados.rados.0.trial115.stdout:3288: snap_create 2026-02-17T06:06:26.296 INFO:tasks.rados.rados.0.trial115.stdout:3285: finishing write tid 2 to trial11545420-3 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3285: finishing write tid 3 to trial11545420-3 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3285: finishing write tid 4 to trial11545420-3 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3285: oid 3 updating version 0 to 1490 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3285: oid 3 version 1490 is already newer than 1489 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3285: oid 3 updating version 1490 to 1491 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1491 (ObjNum 1135 snap 342 seq_num 1135) dirty exists 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3285: left oid 3 (ObjNum 1135 snap 342 seq_num 1135) 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3286: finishing copy_from racing read to trial11545420-12 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3286: finishing copy_from to trial11545420-12 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1463 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3287: finishing write tid 1 to trial11545420-22 2026-02-17T06:06:26.297 INFO:tasks.rados.rados.0.trial115.stdout:3287: finishing write tid 2 to trial11545420-22 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:3287: finishing write tid 3 to trial11545420-22 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:3287: finishing write tid 4 to trial11545420-22 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:3287: finishing write tid 5 to trial11545420-22 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:3287: oid 22 updating version 0 to 1361 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:3287: oid 22 updating version 1361 to 1364 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:3287: oid 22 version 1364 is already newer than 1362 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:3287: oid 22 version 1364 is already newer than 1363 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1364 (ObjNum 1136 snap 342 seq_num 1136) dirty exists 2026-02-17T06:06:26.298 INFO:tasks.rados.rados.0.trial115.stdout:3287: left oid 22 (ObjNum 1136 snap 342 seq_num 1136) 2026-02-17T06:06:27.284 INFO:tasks.rados.rados.0.trial115.stdout:3285: done (3 left) 2026-02-17T06:06:27.284 INFO:tasks.rados.rados.0.trial115.stdout:3286: done (2 left) 2026-02-17T06:06:27.284 INFO:tasks.rados.rados.0.trial115.stdout:3287: done (1 left) 2026-02-17T06:06:27.284 INFO:tasks.rados.rados.0.trial115.stdout:3288: done (0 left) 2026-02-17T06:06:27.284 INFO:tasks.rados.rados.0.trial115.stdout:3289: snap_remove snap 337 2026-02-17T06:06:28.284 INFO:tasks.rados.rados.0.trial115.stdout:3289: done (0 left) 2026-02-17T06:06:28.285 INFO:tasks.rados.rados.0.trial115.stdout:3290: snap_create 2026-02-17T06:06:29.712 INFO:tasks.rados.rados.0.trial115.stdout:3290: done (0 left) 2026-02-17T06:06:29.712 INFO:tasks.rados.rados.0.trial115.stdout:3291: write (excl) oid 23 current snap is 344 2026-02-17T06:06:29.712 INFO:tasks.rados.rados.0.trial115.stdout:3291: seq_num 1137 ranges {709559=494710,1884497=759059} 2026-02-17T06:06:29.717 INFO:tasks.rados.rados.0.trial115.stdout:3291: writing trial11545420-23 from 709559 to 1204269 tid 1 2026-02-17T06:06:29.723 INFO:tasks.rados.rados.0.trial115.stdout:3291: writing trial11545420-23 from 1884497 to 2643556 tid 2 2026-02-17T06:06:29.723 INFO:tasks.rados.rados.0.trial115.stdout:3292: snap_create 2026-02-17T06:06:29.723 INFO:tasks.rados.rados.0.trial115.stdout:3291: finishing write tid 1 to trial11545420-23 2026-02-17T06:06:29.726 INFO:tasks.rados.rados.0.trial115.stdout:3291: finishing write tid 2 to trial11545420-23 2026-02-17T06:06:29.726 INFO:tasks.rados.rados.0.trial115.stdout:3291: finishing write tid 3 to trial11545420-23 2026-02-17T06:06:29.726 INFO:tasks.rados.rados.0.trial115.stdout:3291: finishing write tid 4 to trial11545420-23 2026-02-17T06:06:29.726 INFO:tasks.rados.rados.0.trial115.stdout:3291: oid 23 updating version 0 to 1692 2026-02-17T06:06:29.726 INFO:tasks.rados.rados.0.trial115.stdout:3291: oid 23 version 1692 is already newer than 1691 2026-02-17T06:06:29.726 INFO:tasks.rados.rados.0.trial115.stdout:3291: oid 23 version 1692 is already newer than 1690 2026-02-17T06:06:29.726 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1692 (ObjNum 1137 snap 344 seq_num 1137) dirty exists 2026-02-17T06:06:29.726 INFO:tasks.rados.rados.0.trial115.stdout:3291: left oid 23 (ObjNum 1137 snap 344 seq_num 1137) 2026-02-17T06:06:30.713 INFO:tasks.rados.rados.0.trial115.stdout:3291: done (1 left) 2026-02-17T06:06:30.714 INFO:tasks.rados.rados.0.trial115.stdout:3292: done (0 left) 2026-02-17T06:06:30.714 INFO:tasks.rados.rados.0.trial115.stdout:3293: copy_from oid 30 from oid 49 current snap is 345 2026-02-17T06:06:30.714 INFO:tasks.rados.rados.0.trial115.stdout:3294: delete oid 38 current snap is 345 2026-02-17T06:06:30.716 INFO:tasks.rados.rados.0.trial115.stdout:3294: done (1 left) 2026-02-17T06:06:30.716 INFO:tasks.rados.rados.0.trial115.stdout:3295: copy_from oid 20 from oid 16 current snap is 345 2026-02-17T06:06:30.716 INFO:tasks.rados.rados.0.trial115.stdout:3296: rollback oid 11 current snap is 345 2026-02-17T06:06:30.716 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 344 2026-02-17T06:06:30.717 INFO:tasks.rados.rados.0.trial115.stdout:3297: snap_create 2026-02-17T06:06:30.719 INFO:tasks.rados.rados.0.trial115.stdout:3296: finishing rollback tid 0 to trial11545420-11 2026-02-17T06:06:30.720 INFO:tasks.rados.rados.0.trial115.stdout:3296: finishing rollback tid 1 to trial11545420-11 2026-02-17T06:06:30.720 INFO:tasks.rados.rados.0.trial115.stdout:3296: finishing rollback tid 2 to trial11545420-11 2026-02-17T06:06:30.720 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1709 (ObjNum 1121 snap 336 seq_num 1121) dirty exists 2026-02-17T06:06:30.722 INFO:tasks.rados.rados.0.trial115.stdout:3293: finishing copy_from racing read to trial11545420-30 2026-02-17T06:06:30.724 INFO:tasks.rados.rados.0.trial115.stdout:3293: finishing copy_from to trial11545420-30 2026-02-17T06:06:30.724 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1452 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:06:30.724 INFO:tasks.rados.rados.0.trial115.stdout:3295: finishing copy_from racing read to trial11545420-20 2026-02-17T06:06:30.725 INFO:tasks.rados.rados.0.trial115.stdout:3295: finishing copy_from to trial11545420-20 2026-02-17T06:06:30.725 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1622 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:06:31.273 INFO:tasks.rados.rados.0.trial115.stdout:3293: done (3 left) 2026-02-17T06:06:31.273 INFO:tasks.rados.rados.0.trial115.stdout:3295: done (2 left) 2026-02-17T06:06:31.273 INFO:tasks.rados.rados.0.trial115.stdout:3296: done (1 left) 2026-02-17T06:06:31.273 INFO:tasks.rados.rados.0.trial115.stdout:3297: done (0 left) 2026-02-17T06:06:31.274 INFO:tasks.rados.rados.0.trial115.stdout:3298: snap_remove snap 332 2026-02-17T06:06:32.274 INFO:tasks.rados.rados.0.trial115.stdout:3298: done (0 left) 2026-02-17T06:06:32.274 INFO:tasks.rados.rados.0.trial115.stdout:3299: write oid 25 current snap is 346 2026-02-17T06:06:32.274 INFO:tasks.rados.rados.0.trial115.stdout:3299: seq_num 1139 ranges {743173=645208,1885108=762728} 2026-02-17T06:06:32.279 INFO:tasks.rados.rados.0.trial115.stdout:3299: writing trial11545420-25 from 743173 to 1388381 tid 1 2026-02-17T06:06:32.284 INFO:tasks.rados.rados.0.trial115.stdout:3299: writing trial11545420-25 from 1885108 to 2647836 tid 2 2026-02-17T06:06:32.284 INFO:tasks.rados.rados.0.trial115.stdout:3300: snap_remove snap 335 2026-02-17T06:06:33.277 INFO:tasks.rados.rados.0.trial115.stdout:3300: done (1 left) 2026-02-17T06:06:33.277 INFO:tasks.rados.rados.0.trial115.stdout:3301: read oid 4 snap -1 2026-02-17T06:06:33.277 INFO:tasks.rados.rados.0.trial115.stdout:3301: expect deleted 2026-02-17T06:06:33.277 INFO:tasks.rados.rados.0.trial115.stdout:3302: snap_create 2026-02-17T06:06:33.277 INFO:tasks.rados.rados.0.trial115.stdout:3299: finishing write tid 1 to trial11545420-25 2026-02-17T06:06:33.277 INFO:tasks.rados.rados.0.trial115.stdout:3299: finishing write tid 2 to trial11545420-25 2026-02-17T06:06:33.277 INFO:tasks.rados.rados.0.trial115.stdout:3299: finishing write tid 3 to trial11545420-25 2026-02-17T06:06:33.278 INFO:tasks.rados.rados.0.trial115.stdout:3299: finishing write tid 4 to trial11545420-25 2026-02-17T06:06:33.278 INFO:tasks.rados.rados.0.trial115.stdout:3299: oid 25 updating version 0 to 1565 2026-02-17T06:06:33.278 INFO:tasks.rados.rados.0.trial115.stdout:3299: oid 25 version 1565 is already newer than 1564 2026-02-17T06:06:33.278 INFO:tasks.rados.rados.0.trial115.stdout:3299: oid 25 version 1565 is already newer than 1563 2026-02-17T06:06:33.278 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1565 (ObjNum 1139 snap 346 seq_num 1139) dirty exists 2026-02-17T06:06:33.278 INFO:tasks.rados.rados.0.trial115.stdout:3299: left oid 25 (ObjNum 1139 snap 346 seq_num 1139) 2026-02-17T06:06:34.278 INFO:tasks.rados.rados.0.trial115.stdout:3299: done (2 left) 2026-02-17T06:06:34.278 INFO:tasks.rados.rados.0.trial115.stdout:3301: done (1 left) 2026-02-17T06:06:34.278 INFO:tasks.rados.rados.0.trial115.stdout:3302: done (0 left) 2026-02-17T06:06:34.278 INFO:tasks.rados.rados.0.trial115.stdout:3303: delete oid 49 current snap is 347 2026-02-17T06:06:34.281 INFO:tasks.rados.rados.0.trial115.stdout:3303: done (0 left) 2026-02-17T06:06:34.282 INFO:tasks.rados.rados.0.trial115.stdout:3304: snap_remove snap 346 2026-02-17T06:06:35.278 INFO:tasks.rados.rados.0.trial115.stdout:3304: done (0 left) 2026-02-17T06:06:35.278 INFO:tasks.rados.rados.0.trial115.stdout:3305: copy_from oid 23 from oid 29 current snap is 347 2026-02-17T06:06:35.278 INFO:tasks.rados.rados.0.trial115.stdout:3306: snap_remove snap 342 2026-02-17T06:06:36.274 INFO:tasks.rados.rados.0.trial115.stdout:3306: done (1 left) 2026-02-17T06:06:36.274 INFO:tasks.rados.rados.0.trial115.stdout:3307: write oid 43 current snap is 347 2026-02-17T06:06:36.275 INFO:tasks.rados.rados.0.trial115.stdout:3305: finishing copy_from racing read to trial11545420-23 2026-02-17T06:06:36.275 INFO:tasks.rados.rados.0.trial115.stdout:3305: finishing copy_from to trial11545420-23 2026-02-17T06:06:36.275 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1472 (ObjNum 1128 snap 338 seq_num 1128) dirty exists 2026-02-17T06:06:36.275 INFO:tasks.rados.rados.0.trial115.stdout:3307: seq_num 1141 ranges {776787=512059,1718425=766397,3000223=176564} 2026-02-17T06:06:36.279 INFO:tasks.rados.rados.0.trial115.stdout:3307: writing trial11545420-43 from 776787 to 1288846 tid 1 2026-02-17T06:06:36.284 INFO:tasks.rados.rados.0.trial115.stdout:3307: writing trial11545420-43 from 1718425 to 2484822 tid 2 2026-02-17T06:06:36.287 INFO:tasks.rados.rados.0.trial115.stdout:3307: writing trial11545420-43 from 3000223 to 3176787 tid 3 2026-02-17T06:06:36.287 INFO:tasks.rados.rados.0.trial115.stdout:3305: done (1 left) 2026-02-17T06:06:36.287 INFO:tasks.rados.rados.0.trial115.stdout:3308: read oid 28 snap -1 2026-02-17T06:06:36.287 INFO:tasks.rados.rados.0.trial115.stdout:3308: expect deleted 2026-02-17T06:06:36.287 INFO:tasks.rados.rados.0.trial115.stdout:3309: snap_create 2026-02-17T06:06:36.287 INFO:tasks.rados.rados.0.trial115.stdout:3307: finishing write tid 1 to trial11545420-43 2026-02-17T06:06:36.287 INFO:tasks.rados.rados.0.trial115.stdout:3307: finishing write tid 2 to trial11545420-43 2026-02-17T06:06:36.288 INFO:tasks.rados.rados.0.trial115.stdout:3307: finishing write tid 3 to trial11545420-43 2026-02-17T06:06:36.288 INFO:tasks.rados.rados.0.trial115.stdout:3307: finishing write tid 4 to trial11545420-43 2026-02-17T06:06:36.288 INFO:tasks.rados.rados.0.trial115.stdout:3307: finishing write tid 5 to trial11545420-43 2026-02-17T06:06:36.288 INFO:tasks.rados.rados.0.trial115.stdout:3307: oid 43 updating version 0 to 1626 2026-02-17T06:06:36.288 INFO:tasks.rados.rados.0.trial115.stdout:3307: oid 43 version 1626 is already newer than 1625 2026-02-17T06:06:36.288 INFO:tasks.rados.rados.0.trial115.stdout:3307: oid 43 version 1626 is already newer than 1623 2026-02-17T06:06:36.288 INFO:tasks.rados.rados.0.trial115.stdout:3307: oid 43 version 1626 is already newer than 1624 2026-02-17T06:06:36.289 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1626 (ObjNum 1141 snap 347 seq_num 1141) dirty exists 2026-02-17T06:06:36.289 INFO:tasks.rados.rados.0.trial115.stdout:3307: left oid 43 (ObjNum 1141 snap 347 seq_num 1141) 2026-02-17T06:06:37.284 INFO:tasks.rados.rados.0.trial115.stdout:3307: done (2 left) 2026-02-17T06:06:37.284 INFO:tasks.rados.rados.0.trial115.stdout:3308: done (1 left) 2026-02-17T06:06:37.284 INFO:tasks.rados.rados.0.trial115.stdout:3309: done (0 left) 2026-02-17T06:06:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3310: write (excl) oid 42 current snap is 348 2026-02-17T06:06:37.285 INFO:tasks.rados.rados.0.trial115.stdout:3310: seq_num 1142 ranges {793594=587308,2060554=510055} 2026-02-17T06:06:37.290 INFO:tasks.rados.rados.0.trial115.stdout:3310: writing trial11545420-42 from 793594 to 1380902 tid 1 2026-02-17T06:06:37.295 INFO:tasks.rados.rados.0.trial115.stdout:3310: writing trial11545420-42 from 2060554 to 2570609 tid 2 2026-02-17T06:06:37.295 INFO:tasks.rados.rados.0.trial115.stdout:3311: read oid 43 snap -1 2026-02-17T06:06:37.295 INFO:tasks.rados.rados.0.trial115.stdout:3311: expect (ObjNum 1141 snap 347 seq_num 1141) 2026-02-17T06:06:37.295 INFO:tasks.rados.rados.0.trial115.stdout:3312: write (excl) oid 30 current snap is 348 2026-02-17T06:06:37.295 INFO:tasks.rados.rados.0.trial115.stdout:3312: seq_num 1143 ranges {410401=662557,1719036=770066,2938716=271685} 2026-02-17T06:06:37.298 INFO:tasks.rados.rados.0.trial115.stdout:3312: writing trial11545420-30 from 410401 to 1072958 tid 1 2026-02-17T06:06:37.304 INFO:tasks.rados.rados.0.trial115.stdout:3312: writing trial11545420-30 from 1719036 to 2489102 tid 2 2026-02-17T06:06:37.307 INFO:tasks.rados.rados.0.trial115.stdout:3312: writing trial11545420-30 from 2938716 to 3210401 tid 3 2026-02-17T06:06:37.308 INFO:tasks.rados.rados.0.trial115.stdout:3313: write (excl) oid 4 current snap is 348 2026-02-17T06:06:37.308 INFO:tasks.rados.rados.0.trial115.stdout:3310: finishing write tid 1 to trial11545420-42 2026-02-17T06:06:37.308 INFO:tasks.rados.rados.0.trial115.stdout:3310: finishing write tid 2 to trial11545420-42 2026-02-17T06:06:37.308 INFO:tasks.rados.rados.0.trial115.stdout:3310: finishing write tid 3 to trial11545420-42 2026-02-17T06:06:37.308 INFO:tasks.rados.rados.0.trial115.stdout:3310: finishing write tid 4 to trial11545420-42 2026-02-17T06:06:37.309 INFO:tasks.rados.rados.0.trial115.stdout:3310: oid 42 updating version 0 to 1727 2026-02-17T06:06:37.309 INFO:tasks.rados.rados.0.trial115.stdout:3310: oid 42 version 1727 is already newer than 1726 2026-02-17T06:06:37.309 INFO:tasks.rados.rados.0.trial115.stdout:3310: oid 42 version 1727 is already newer than 1725 2026-02-17T06:06:37.309 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1727 (ObjNum 1142 snap 348 seq_num 1142) dirty exists 2026-02-17T06:06:37.309 INFO:tasks.rados.rados.0.trial115.stdout:3310: left oid 42 (ObjNum 1142 snap 348 seq_num 1142) 2026-02-17T06:06:37.309 INFO:tasks.rados.rados.0.trial115.stdout:3311: expect (ObjNum 1141 snap 347 seq_num 1141) 2026-02-17T06:06:37.453 INFO:tasks.rados.rados.0.trial115.stdout:3312: finishing write tid 1 to trial11545420-30 2026-02-17T06:06:37.454 INFO:tasks.rados.rados.0.trial115.stdout:3312: finishing write tid 2 to trial11545420-30 2026-02-17T06:06:37.454 INFO:tasks.rados.rados.0.trial115.stdout:3313: seq_num 1144 ranges {427208=737806,1777518=513724,2966139=261069} 2026-02-17T06:06:37.458 INFO:tasks.rados.rados.0.trial115.stdout:3313: writing trial11545420-4 from 427208 to 1165014 tid 1 2026-02-17T06:06:37.462 INFO:tasks.rados.rados.0.trial115.stdout:3313: writing trial11545420-4 from 1777518 to 2291242 tid 2 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3313: writing trial11545420-4 from 2966139 to 3227208 tid 3 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3310: done (3 left) 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3311: done (2 left) 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3314: copy_from oid 33 from oid 29 current snap is 348 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: finishing write tid 3 to trial11545420-30 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: finishing write tid 4 to trial11545420-30 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: finishing write tid 5 to trial11545420-30 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: oid 30 updating version 0 to 1453 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: oid 30 updating version 1453 to 1456 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: oid 30 version 1456 is already newer than 1455 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: oid 30 version 1456 is already newer than 1454 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1456 (ObjNum 1143 snap 348 seq_num 1143) dirty exists 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: left oid 30 (ObjNum 1143 snap 348 seq_num 1143) 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3312: done (2 left) 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3315: rollback oid 3 current snap is 348 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 3 to 344 2026-02-17T06:06:37.466 INFO:tasks.rados.rados.0.trial115.stdout:3316: snap_remove snap 345 2026-02-17T06:06:38.289 INFO:tasks.rados.rados.0.trial115.stdout:3316: done (3 left) 2026-02-17T06:06:38.289 INFO:tasks.rados.rados.0.trial115.stdout:3317: delete oid 9 current snap is 348 2026-02-17T06:06:38.289 INFO:tasks.rados.rados.0.trial115.stdout:3315: finishing rollback tid 0 to trial11545420-3 2026-02-17T06:06:38.289 INFO:tasks.rados.rados.0.trial115.stdout:3315: finishing rollback tid 1 to trial11545420-3 2026-02-17T06:06:38.289 INFO:tasks.rados.rados.0.trial115.stdout:3313: finishing write tid 1 to trial11545420-4 2026-02-17T06:06:38.289 INFO:tasks.rados.rados.0.trial115.stdout:3315: finishing rollback tid 2 to trial11545420-3 2026-02-17T06:06:38.289 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1503 (ObjNum 1135 snap 342 seq_num 1135) dirty exists 2026-02-17T06:06:38.290 INFO:tasks.rados.rados.0.trial115.stdout:3313: finishing write tid 2 to trial11545420-4 2026-02-17T06:06:38.290 INFO:tasks.rados.rados.0.trial115.stdout:3313: finishing write tid 3 to trial11545420-4 2026-02-17T06:06:38.290 INFO:tasks.rados.rados.0.trial115.stdout:3313: finishing write tid 4 to trial11545420-4 2026-02-17T06:06:38.290 INFO:tasks.rados.rados.0.trial115.stdout:3313: finishing write tid 5 to trial11545420-4 2026-02-17T06:06:38.290 INFO:tasks.rados.rados.0.trial115.stdout:3313: oid 4 updating version 0 to 1470 2026-02-17T06:06:38.290 INFO:tasks.rados.rados.0.trial115.stdout:3313: oid 4 updating version 1470 to 1471 2026-02-17T06:06:38.291 INFO:tasks.rados.rados.0.trial115.stdout:3313: oid 4 updating version 1471 to 1472 2026-02-17T06:06:38.291 INFO:tasks.rados.rados.0.trial115.stdout:3313: oid 4 version 1472 is already newer than 1469 2026-02-17T06:06:38.291 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1472 (ObjNum 1144 snap 348 seq_num 1144) dirty exists 2026-02-17T06:06:38.291 INFO:tasks.rados.rados.0.trial115.stdout:3313: left oid 4 (ObjNum 1144 snap 348 seq_num 1144) 2026-02-17T06:06:38.291 INFO:tasks.rados.rados.0.trial115.stdout:3314: finishing copy_from racing read to trial11545420-33 2026-02-17T06:06:38.291 INFO:tasks.rados.rados.0.trial115.stdout:3314: finishing copy_from to trial11545420-33 2026-02-17T06:06:38.292 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1472 (ObjNum 1128 snap 338 seq_num 1128) dirty exists 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3313: done (3 left) 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3314: done (2 left) 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3315: done (1 left) 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3317: done (0 left) 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3318: read oid 20 snap 328 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3318: expect (ObjNum 1090 snap 328 seq_num 1090) 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3319: read oid 32 snap -1 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3319: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:06:38.293 INFO:tasks.rados.rados.0.trial115.stdout:3320: snap_create 2026-02-17T06:06:38.294 INFO:tasks.rados.rados.0.trial115.stdout:3318: expect (ObjNum 1090 snap 328 seq_num 1090) 2026-02-17T06:06:38.351 INFO:tasks.rados.rados.0.trial115.stdout:3319: expect (ObjNum 918 snap 269 seq_num 918) 2026-02-17T06:06:39.290 INFO:tasks.rados.rados.0.trial115.stdout:3318: done (2 left) 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3319: done (1 left) 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3320: done (0 left) 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3321: rollback oid 14 current snap is 349 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 343 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3322: rollback oid 16 current snap is 349 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 16 to 339 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3323: rollback oid 34 current snap is 349 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 341 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3324: read oid 29 snap -1 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3324: expect (ObjNum 1128 snap 338 seq_num 1128) 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3325: write (excl) oid 27 current snap is 349 2026-02-17T06:06:39.291 INFO:tasks.rados.rados.0.trial115.stdout:3325: seq_num 1146 ranges {460822=488304,1378129=517393,2504632=405962} 2026-02-17T06:06:39.293 INFO:tasks.rados.rados.0.trial115.stdout:3325: writing trial11545420-27 from 460822 to 949126 tid 1 2026-02-17T06:06:39.297 INFO:tasks.rados.rados.0.trial115.stdout:3325: writing trial11545420-27 from 1378129 to 1895522 tid 2 2026-02-17T06:06:39.301 INFO:tasks.rados.rados.0.trial115.stdout:3325: writing trial11545420-27 from 2504632 to 2910594 tid 3 2026-02-17T06:06:39.301 INFO:tasks.rados.rados.0.trial115.stdout:3326: read oid 19 snap -1 2026-02-17T06:06:39.301 INFO:tasks.rados.rados.0.trial115.stdout:3326: expect (ObjNum 1063 snap 319 seq_num 1063) 2026-02-17T06:06:39.301 INFO:tasks.rados.rados.0.trial115.stdout:3327: copy_from oid 50 from oid 28 current snap is 349 2026-02-17T06:06:39.301 INFO:tasks.rados.rados.0.trial115.stdout:3324: expect (ObjNum 1128 snap 338 seq_num 1128) 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3323: finishing rollback tid 0 to trial11545420-34 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3324: done (6 left) 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3328: snap_create 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3323: finishing rollback tid 1 to trial11545420-34 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3323: finishing rollback tid 2 to trial11545420-34 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1490 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3321: finishing rollback tid 0 to trial11545420-14 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3321: finishing rollback tid 1 to trial11545420-14 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3321: finishing rollback tid 2 to trial11545420-14 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1654 (ObjNum 1026 snap 297 seq_num 1026) dirty exists 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3322: finishing rollback tid 0 to trial11545420-16 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3322: finishing rollback tid 1 to trial11545420-16 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3322: finishing rollback tid 2 to trial11545420-16 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1749 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: finishing write tid 1 to trial11545420-27 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: finishing write tid 2 to trial11545420-27 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: finishing write tid 3 to trial11545420-27 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: finishing write tid 4 to trial11545420-27 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: finishing write tid 5 to trial11545420-27 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: oid 27 updating version 0 to 1457 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: oid 27 updating version 1457 to 1458 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: oid 27 updating version 1458 to 1460 2026-02-17T06:06:39.433 INFO:tasks.rados.rados.0.trial115.stdout:3325: oid 27 version 1460 is already newer than 1459 2026-02-17T06:06:39.434 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1460 (ObjNum 1146 snap 349 seq_num 1146) dirty exists 2026-02-17T06:06:39.434 INFO:tasks.rados.rados.0.trial115.stdout:3325: left oid 27 (ObjNum 1146 snap 349 seq_num 1146) 2026-02-17T06:06:39.434 INFO:tasks.rados.rados.0.trial115.stdout:3327: finishing copy_from to trial11545420-50 2026-02-17T06:06:39.434 INFO:tasks.rados.rados.0.trial115.stdout:3327: got expected ENOENT (src dne) 2026-02-17T06:06:39.434 INFO:tasks.rados.rados.0.trial115.stdout:3327: finishing copy_from racing read to trial11545420-50 2026-02-17T06:06:39.434 INFO:tasks.rados.rados.0.trial115.stdout:3326: expect (ObjNum 1063 snap 319 seq_num 1063) 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:3321: done (6 left) 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:3322: done (5 left) 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:3323: done (4 left) 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:3325: done (3 left) 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:3326: done (2 left) 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:3327: done (1 left) 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:3328: done (0 left) 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:3329: rollback oid 29 current snap is 350 2026-02-17T06:06:40.294 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 343 2026-02-17T06:06:40.295 INFO:tasks.rados.rados.0.trial115.stdout:3330: write oid 41 current snap is 350 2026-02-17T06:06:40.295 INFO:tasks.rados.rados.0.trial115.stdout:3330: seq_num 1147 ranges {477629=563553,1836611=777404,3048408=229221} 2026-02-17T06:06:40.297 INFO:tasks.rados.rados.0.trial115.stdout:3330: writing trial11545420-41 from 477629 to 1041182 tid 1 2026-02-17T06:06:40.303 INFO:tasks.rados.rados.0.trial115.stdout:3330: writing trial11545420-41 from 1836611 to 2614015 tid 2 2026-02-17T06:06:40.306 INFO:tasks.rados.rados.0.trial115.stdout:3330: writing trial11545420-41 from 3048408 to 3277629 tid 3 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3331: read oid 33 snap -1 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3331: expect (ObjNum 1128 snap 338 seq_num 1128) 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3332: copy_from oid 20 from oid 35 current snap is 350 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3329: finishing rollback tid 0 to trial11545420-29 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3329: finishing rollback tid 1 to trial11545420-29 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3329: finishing rollback tid 2 to trial11545420-29 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1475 (ObjNum 1128 snap 338 seq_num 1128) dirty exists 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3330: finishing write tid 1 to trial11545420-41 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3329: done (3 left) 2026-02-17T06:06:40.307 INFO:tasks.rados.rados.0.trial115.stdout:3333: read oid 29 snap 343 2026-02-17T06:06:40.308 INFO:tasks.rados.rados.0.trial115.stdout:3333: expect (ObjNum 1128 snap 338 seq_num 1128) 2026-02-17T06:06:40.308 INFO:tasks.rados.rados.0.trial115.stdout:3334: snap_create 2026-02-17T06:06:40.308 INFO:tasks.rados.rados.0.trial115.stdout:3330: finishing write tid 2 to trial11545420-41 2026-02-17T06:06:40.308 INFO:tasks.rados.rados.0.trial115.stdout:3330: finishing write tid 3 to trial11545420-41 2026-02-17T06:06:40.308 INFO:tasks.rados.rados.0.trial115.stdout:3330: finishing write tid 4 to trial11545420-41 2026-02-17T06:06:40.310 INFO:tasks.rados.rados.0.trial115.stdout:3333: expect (ObjNum 1128 snap 338 seq_num 1128) 2026-02-17T06:06:40.443 INFO:tasks.rados.rados.0.trial115.stdout:3332: finishing copy_from racing read to trial11545420-20 2026-02-17T06:06:40.443 INFO:tasks.rados.rados.0.trial115.stdout:3330: finishing write tid 5 to trial11545420-41 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:3330: oid 41 updating version 0 to 1566 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:3330: oid 41 updating version 1566 to 1569 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:3330: oid 41 version 1569 is already newer than 1567 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:3330: oid 41 version 1569 is already newer than 1568 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1569 (ObjNum 1147 snap 350 seq_num 1147) dirty exists 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:3330: left oid 41 (ObjNum 1147 snap 350 seq_num 1147) 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:3332: finishing copy_from to trial11545420-20 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1687 (ObjNum 1118 snap 336 seq_num 1118) dirty exists 2026-02-17T06:06:40.444 INFO:tasks.rados.rados.0.trial115.stdout:3331: expect (ObjNum 1128 snap 338 seq_num 1128) 2026-02-17T06:06:40.564 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T06:06:40.564 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T06:06:40.564 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T06:06:40.564 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T06:06:40.567 INFO:tasks.ceph.mon.a:Started 2026-02-17T06:06:40.567 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:06:40.567 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:40.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.633+0000 7f277e64d640 10 monclient: get_monmap_and_config 2026-02-17T06:06:40.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.633+0000 7f277e64d640 10 monclient: build_initial_monmap 2026-02-17T06:06:40.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.633+0000 7f277e64d640 10 monclient: monmap: 2026-02-17T06:06:40.644 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:40.644 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:40.634946+0000 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:40.634946+0000 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 1 Processor -- start 2026-02-17T06:06:40.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 1 -- start start 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient: init 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient: _add_conns ranks=[2,3,4,0,1] 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277806c890 0x7f277806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient(hunting): picked mon.noname-b con 0x7f277806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 0x7f277806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient(hunting): picked mon.noname-a con 0x7f277809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2778134df0 0x7f2778135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient(hunting): picked mon.noname-c con 0x7f2778134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:40.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f27780a0880 con 0x7f277809c4a0 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f277806d6f0 con 0x7f2778134df0 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f277806daf0 con 0x7f277806c890 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277e64d640 10 monclient(hunting): authenticate will time out at 1846.188476s 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 0x7f277806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 0x7f277806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:33404/0 (socket says 10.20.193.78:33404) 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 1 -- 10.20.193.78:0/239111598 learned_addr learned my addr 10.20.193.78:0/239111598 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 10 monclient(hunting): get_auth_request con 0x7f277809c4a0 auth_method 0 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:40.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 1 --2- 10.20.193.78:0/239111598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277806c890 0x7f277806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient(hunting): get_auth_request con 0x7f277806c890 auth_method 0 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 1 -- 10.20.193.78:0/239111598 >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 msgr2=0x7f277806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 1 -- 10.20.193.78:0/239111598 >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 msgr2=0x7f277806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 1 --2- 10.20.193.78:0/239111598 >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 0x7f277806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f27777fe640 1 --2- 10.20.193.78:0/239111598 >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 0x7f277806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:40.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277cbc3640 1 --2- 10.20.193.78:0/239111598 >> v2:10.20.193.115:3300/0 conn(0x7f2778134df0 0x7f2778135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277cbc3640 10 monclient(hunting): get_auth_request con 0x7f2778134df0 auth_method 0 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277cbc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277cbc3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f277cbc3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient(hunting): handle_auth_done global_id 234287 payload 274 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 1 -- 10.20.193.78:0/239111598 >> v2:10.20.193.115:3300/0 conn(0x7f2778134df0 msgr2=0x7f2778135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 1 --2- 10.20.193.78:0/239111598 >> v2:10.20.193.115:3300/0 conn(0x7f2778134df0 0x7f2778135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 1 -- 10.20.193.78:0/239111598 >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 msgr2=0x7f277806d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 1 --2- 10.20.193.78:0/239111598 >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 0x7f277806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient: _finish_hunting 0 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 1 monclient: found mon.noname-b 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.634+0000 7f2777fff640 1 -- 10.20.193.78:0/239111598 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277809cc80 con 0x7f277806c890 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2777fff640 1 -- 10.20.193.78:0/239111598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277806c890 msgr2=0x7f277806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2777fff640 1 --2- 10.20.193.78:0/239111598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277806c890 0x7f277806cc70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f277809c880 tx=0x7f276402ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d8012be70d416e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 1 -- 10.20.193.78:0/239111598 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276403c070 con 0x7f277806c890 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: dump: 2026-02-17T06:06:40.650 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: _finish_auth 0 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:40.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 1 -- 10.20.193.78:0/239111598 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276402fbd0 con 0x7f277806c890 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 1 -- 10.20.193.78:0/239111598 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764037440 con 0x7f277806c890 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 5 monclient: authenticate success, global_id 234287 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 4 set_mon_vals no callback set 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f2776ffd640 10 monclient: dump: 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:40.652 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 10 monclient: shutdown 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 -- 10.20.193.78:0/239111598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277806c890 msgr2=0x7f277806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 --2- 10.20.193.78:0/239111598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277806c890 0x7f277806cc70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f277809c880 tx=0x7f276402ed50 comp rx=0 tx=0).stop 2026-02-17T06:06:40.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 -- 10.20.193.78:0/239111598 shutdown_connections 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 --2- 10.20.193.78:0/239111598 >> v2:10.20.193.115:3300/0 conn(0x7f2778134df0 0x7f2778135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 --2- 10.20.193.78:0/239111598 >> v2:10.20.193.78:3300/0 conn(0x7f277809c4a0 0x7f277806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 --2- 10.20.193.78:0/239111598 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277806c890 0x7f277806cc70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 -- 10.20.193.78:0/239111598 >> 10.20.193.78:0/239111598 conn(0x7f27780a6620 msgr2=0x7f27780a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 -- 10.20.193.78:0/239111598 shutdown_connections 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.637+0000 7f277e64d640 1 -- 10.20.193.78:0/239111598 wait complete. 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient: build_initial_monmap 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient: monmap: 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:40.654 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 1 Processor -- start 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 1 -- start start 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient: init 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:40.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f277806c890 0x7f27781cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient(hunting): picked mon.noname-b con 0x7f277806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277809c4a0 0x7f27781cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient(hunting): picked mon.noname-c con 0x7f277809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 0x7f27781214a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient(hunting): picked mon.noname-a con 0x7f2778134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27780a01a0 con 0x7f2778134df0 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f27780a0020 con 0x7f277806c890 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f277809be20 con 0x7f277809c4a0 2026-02-17T06:06:40.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277e64d640 10 monclient(hunting): authenticate will time out at 1846.192686s 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277809c4a0 0x7f27781cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 0x7f27781214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277809c4a0 0x7f27781cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:41050/0 (socket says 10.20.193.78:41050) 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 -- 10.20.193.78:0/650828740 learned_addr learned my addr 10.20.193.78:0/650828740 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 10 monclient(hunting): get_auth_request con 0x7f2778134df0 auth_method 0 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient(hunting): get_auth_request con 0x7f277809c4a0 auth_method 0 2026-02-17T06:06:40.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 1 -- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 msgr2=0x7f27781214a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 1 -- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 msgr2=0x7f27781214a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 0x7f27781214a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f277cbc3640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 0x7f27781214a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f2777fff640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f277806c890 0x7f27781cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f2777fff640 10 monclient(hunting): get_auth_request con 0x7f277806c890 auth_method 0 2026-02-17T06:06:40.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f2777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f2777fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f2777fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient(hunting): handle_auth_done global_id 234290 payload 824 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 -- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f277806c890 msgr2=0x7f27781cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f277806c890 0x7f27781cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 -- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 msgr2=0x7f27781214a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 0x7f27781214a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 monclient: found mon.noname-c 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.638+0000 7f27777fe640 1 -- 10.20.193.78:0/650828740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781235a0 con 0x7f277809c4a0 2026-02-17T06:06:40.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f27777fe640 1 -- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277809c4a0 msgr2=0x7f27781cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f27777fe640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277809c4a0 0x7f27781cd870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f276800d830 tx=0x7f276800dd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 1 -- 10.20.193.78:0/650828740 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768018030 con 0x7f277809c4a0 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: dump: 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:40.660 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: _finish_auth 0 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 1 -- 10.20.193.78:0/650828740 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276800bae0 con 0x7f277809c4a0 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 1 -- 10.20.193.78:0/650828740 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276800bcd0 con 0x7f277809c4a0 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f27767fc640 4 set_mon_vals no callback set 2026-02-17T06:06:40.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 10 monclient: dump: 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 5 monclient: authenticate success, global_id 234290 2026-02-17T06:06:40.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 10 monclient: _renew_subs 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2778123c60 con 0x7f277809c4a0 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 10 monclient: _renew_subs 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f27780725b0 con 0x7f277809c4a0 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2778074980 con 0x7f277809c4a0 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.641+0000 7f2774ff9640 1 -- 10.20.193.78:0/650828740 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2768013050 con 0x7f277809c4a0 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.642+0000 7f2774ff9640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f274803deb0 0x7f2748040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:40.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.642+0000 7f2774ff9640 1 -- 10.20.193.78:0/650828740 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(721..721 src has 525..721) ==== 4783+0+0 (secure 0 0 0) 0x7f2768052850 con 0x7f277809c4a0 2026-02-17T06:06:40.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.642+0000 7f2777fff640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f274803deb0 0x7f2748040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:40.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.642+0000 7f2777fff640 10 monclient: get_auth_request con 0x7f274803deb0 auth_method 0 2026-02-17T06:06:40.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.644+0000 7f2774ff9640 1 -- 10.20.193.78:0/650828740 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2768022070 con 0x7f277809c4a0 2026-02-17T06:06:40.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.644+0000 7f2774ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:40.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.644+0000 7f2774ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:40.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.645+0000 7f2777fff640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f274803deb0 0x7f2748040370 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f27780a7600 tx=0x7f2764033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:40.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.758+0000 7f277e64d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:40.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.758+0000 7f277e64d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:40.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.758+0000 7f277e64d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:40.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.758+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f27780a1610 con 0x7f277809c4a0 2026-02-17T06:06:40.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.758+0000 7f2774ff9640 1 -- 10.20.193.78:0/650828740 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f2768013c90 con 0x7f277809c4a0 2026-02-17T06:06:40.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.758+0000 7f2774ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:40.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f2774ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:40.761 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":234,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:40.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f274803deb0 msgr2=0x7f2748040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:40.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f274803deb0 0x7f2748040370 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f27780a7600 tx=0x7f2764033000 comp rx=0 tx=0).stop 2026-02-17T06:06:40.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 10 monclient: shutdown 2026-02-17T06:06:40.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277809c4a0 msgr2=0x7f27781cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277809c4a0 0x7f27781cd870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f276800d830 tx=0x7f276800dd00 comp rx=0 tx=0).stop 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 shutdown_connections 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2778134df0 0x7f27781214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f277809c4a0 0x7f27781cd870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f274803deb0 0x7f2748040370 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 --2- 10.20.193.78:0/650828740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f277806c890 0x7f27781cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 >> 10.20.193.78:0/650828740 conn(0x7f27780a6620 msgr2=0x7f2778135810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 shutdown_connections 2026-02-17T06:06:40.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:40.759+0000 7f277e64d640 1 -- 10.20.193.78:0/650828740 wait complete. 2026-02-17T06:06:41.275 INFO:tasks.rados.rados.0.trial115.stdout:3330: done (4 left) 2026-02-17T06:06:41.275 INFO:tasks.rados.rados.0.trial115.stdout:3331: done (3 left) 2026-02-17T06:06:41.275 INFO:tasks.rados.rados.0.trial115.stdout:3332: done (2 left) 2026-02-17T06:06:41.275 INFO:tasks.rados.rados.0.trial115.stdout:3333: done (1 left) 2026-02-17T06:06:41.276 INFO:tasks.rados.rados.0.trial115.stdout:3334: done (0 left) 2026-02-17T06:06:41.276 INFO:tasks.rados.rados.0.trial115.stdout:3335: snap_create 2026-02-17T06:06:42.275 INFO:tasks.rados.rados.0.trial115.stdout:3335: done (0 left) 2026-02-17T06:06:42.275 INFO:tasks.rados.rados.0.trial115.stdout:3336: snap_create 2026-02-17T06:06:43.276 INFO:tasks.rados.rados.0.trial115.stdout:3336: done (0 left) 2026-02-17T06:06:43.276 INFO:tasks.rados.rados.0.trial115.stdout:3337: write oid 13 current snap is 353 2026-02-17T06:06:43.276 INFO:tasks.rados.rados.0.trial115.stdout:3337: seq_num 1148 ranges {494436=755155,2011446=521062,3075831=218605} 2026-02-17T06:06:43.280 INFO:tasks.rados.rados.0.trial115.stdout:3337: writing trial11545420-13 from 494436 to 1249591 tid 1 2026-02-17T06:06:43.285 INFO:tasks.rados.rados.0.trial115.stdout:3337: writing trial11545420-13 from 2011446 to 2532508 tid 2 2026-02-17T06:06:43.288 INFO:tasks.rados.rados.0.trial115.stdout:3337: writing trial11545420-13 from 3075831 to 3294436 tid 3 2026-02-17T06:06:43.288 INFO:tasks.rados.rados.0.trial115.stdout:3338: snap_remove snap 328 2026-02-17T06:06:43.770 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.835+0000 7f72b710a640 10 monclient: get_monmap_and_config 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.835+0000 7f72b710a640 10 monclient: build_initial_monmap 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.835+0000 7f72b710a640 10 monclient: monmap: 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:43.836773+0000 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:43.836773+0000 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:43.836 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.835+0000 7f72b710a640 1 Processor -- start 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 1 -- start start 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient: init 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f72b006c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient(hunting): picked mon.noname-c con 0x7f72b009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f72b006d380 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b006daf0 con 0x7f72b006c890 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b009cc80 con 0x7f72b006d380 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f72b00a0880 con 0x7f72b009c4a0 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b710a640 10 monclient(hunting): authenticate will time out at 1849.390348s 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b4e7f640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f72b006c890 0x7f72b006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:42152/0 (socket says 10.20.193.78:42152) 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 learned_addr learned my addr 10.20.193.78:0/2649996550 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f72b009c4a0 auth_method 0 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:43.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:43.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.836+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:43.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.838+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v1:10.20.193.78:6789/0 conn(0x7f72b006c890 legacy=0x7f72b006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:43.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:43.838+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v1:10.20.193.115:6789/0 conn(0x7f72b006d380 legacy=0x7f72b013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:44.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:44.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f72b009c4a0 auth_method 0 2026-02-17T06:06:44.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:44.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:44.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:44.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:44.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:44.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:44.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.037+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:44.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:44.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f72b009c4a0 auth_method 0 2026-02-17T06:06:44.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:44.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:44.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:44.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:44.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:44.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:44.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:44.438+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:45.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:45.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f72b009c4a0 auth_method 0 2026-02-17T06:06:45.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:45.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:45.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:45.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:45.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:45.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:45.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:45.239+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:46.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient(hunting): tick 2026-02-17T06:06:46.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:06:46.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 monclient(hunting): continuing hunt 2026-02-17T06:06:46.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:06:46.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 msgr2=0x7f72b013b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72b009c4a0 0x7f72b013b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 >> v1:10.20.193.115:6789/0 conn(0x7f72b006d380 legacy=0x7f72b013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 >> v1:10.20.193.78:6789/0 conn(0x7f72b006c890 legacy=0x7f72b006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient: _add_conns ranks=[2,3] 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient(hunting): picked mon.noname-b con 0x7f7294003910 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient(hunting): picked mon.noname-a con 0x7f7294006450 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7294008fa0 con 0x7f7294006450 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7294009150 con 0x7f7294003910 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a67fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:46.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f7294003910 auth_method 0 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f7294006450 auth_method 0 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:46.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:46.836+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f7294006450 auth_method 0 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f7294003910 auth_method 0 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:47.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:47.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f7294006450 auth_method 0 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f7294003910 auth_method 0 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:47.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:47.438+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:48.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f7294006450 auth_method 0 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f7294003910 auth_method 0 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:48.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:48.239+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:49.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): tick 2026-02-17T06:06:49.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 monclient(hunting): continuing hunt 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 msgr2=0x7f7294005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7294003910 0x7f7294005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 msgr2=0x7f7294008910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294006450 0x7f7294008910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 reap_dead start 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): picked mon.noname-a con 0x7f7294003d30 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7294006dc0 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): picked mon.noname-c con 0x7f72940089f0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b009cf60 con 0x7f7294006dc0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7294008fa0 con 0x7f7294003d30 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7294009150 con 0x7f72940089f0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72a67fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f7294003d30 auth_method 0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 10 monclient(hunting): get_auth_request con 0x7f72940089f0 auth_method 0 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:49.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.837+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:49.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:49.838+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v1:10.20.193.78:6789/0 conn(0x7f7294006dc0 legacy=0x7f7294007240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f7294003d30 auth_method 0 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 10 monclient(hunting): get_auth_request con 0x7f72940089f0 auth_method 0 2026-02-17T06:06:50.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:50.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:50.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:50.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:50.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:50.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:50.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.037+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:50.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:50.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f7294003d30 auth_method 0 2026-02-17T06:06:50.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:50.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:50.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:50.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.437+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 10 monclient(hunting): get_auth_request con 0x7f72940089f0 auth_method 0 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:50.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:50.438+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:06:51.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f7294003d30 auth_method 0 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:51.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 10 monclient(hunting): get_auth_request con 0x7f72940089f0 auth_method 0 2026-02-17T06:06:51.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:51.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:51.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:51.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:51.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:51.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:51.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:51.239+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:06:52.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 10 monclient(hunting): tick 2026-02-17T06:06:52.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:06:52.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 monclient(hunting): continuing hunt 2026-02-17T06:06:52.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:06:52.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 msgr2=0x7f729400d6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:52.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.115:3300/0 conn(0x7f72940089f0 0x7f729400d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:52.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 msgr2=0x7f7294006880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:52.839 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> v2:10.20.193.78:3300/0 conn(0x7f7294003d30 0x7f7294006880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 >> v1:10.20.193.78:6789/0 conn(0x7f7294006dc0 legacy=0x7f7294007240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 10 monclient: _add_conns ranks=[2,1] 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 0x7f729400e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 10 monclient(hunting): picked mon.noname-b con 0x7f729400dfa0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f72940110d0 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f729400ee50 con 0x7f72940110d0 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 1 -- 10.20.193.78:0/2649996550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7294008fa0 con 0x7f729400dfa0 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72a67fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 0x7f729400e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:52.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f729400dfa0 auth_method 0 2026-02-17T06:06:52.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:52.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:52.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:52.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 msgr2=0x7f729400e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:52.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 msgr2=0x7f729400e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:52.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 0x7f729400e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:52.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.837+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 0x7f729400e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:52.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:52.839+0000 7f72a7fff640 1 -- 10.20.193.78:0/2649996550 >> v1:10.20.193.115:6789/0 conn(0x7f72940110d0 legacy=0x7f72940134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 0x7f729400e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f729400dfa0 auth_method 0 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 msgr2=0x7f729400e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 msgr2=0x7f729400e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 0x7f729400e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:06:53.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.037+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 0x7f729400e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:06:53.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.266+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916809866 0 0) 0x7f729400ee50 con 0x7f72940110d0 2026-02-17T06:06:53.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.266+0000 7f72a77fe640 10 monclient(hunting): _negotiate 2026-02-17T06:06:53.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.266+0000 7f72a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.266+0000 7f72a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.266+0000 7f72a77fe640 10 monclient(hunting): my global_id is 214363 2026-02-17T06:06:53.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.266+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f729c003620 con 0x7f72940110d0 2026-02-17T06:06:53.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 22365358 0 0) 0x7f729c003620 con 0x7f72940110d0 2026-02-17T06:06:53.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 msgr2=0x7f729400e380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:53.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729400dfa0 0x7f729400e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7294015bf0 con 0x7f72940110d0 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b710a640 5 monclient: authenticate success, global_id 214363 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b710a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f729800f040 con 0x7f72940110d0 2026-02-17T06:06:53.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: dump: 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.275 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 >> v1:10.20.193.115:6789/0 conn(0x7f72940110d0 legacy=0x7f72940134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b5680640 1 -- 10.20.193.78:0/2649996550 reap_dead start 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c003ab0 0x7f729c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient(hunting): picked mon.a con 0x7f729c003ab0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c004900 0x7f729c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient(hunting): picked mon.c con 0x7f729c004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c005390 0x7f729c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient(hunting): picked mon.b con 0x7f729c005390 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:53.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7294008fa0 con 0x7f729c003ab0 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f729c00c2e0 con 0x7f729c005390 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f729c00c4e0 con 0x7f729c004900 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a77fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b710a640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c003ab0 0x7f729c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a7fff640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c004900 0x7f729c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f729c003ab0 auth_method 0 2026-02-17T06:06:53.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f729c004900 auth_method 0 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b5680640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c005390 0x7f729c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b5680640 10 monclient(hunting): get_auth_request con 0x7f729c005390 auth_method 0 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b5680640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b5680640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b5680640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient(hunting): handle_auth_done global_id 214363 payload 293 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c004900 msgr2=0x7f729c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c004900 0x7f729c004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c005390 msgr2=0x7f729c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c005390 0x7f729c00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 1 monclient: found mon.a 2026-02-17T06:06:53.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c003620 con 0x7f729c003ab0 2026-02-17T06:06:53.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.268+0000 7f72b4e7f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.270+0000 7f72b4e7f640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c003ab0 msgr2=0x7f729c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.270+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c003ab0 0x7f729c003eb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f72a0007d00 tx=0x7f72a0031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517fc5a8f115f80c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a003e070 con 0x7f729c003ab0 2026-02-17T06:06:53.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 10 monclient: dump: 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.288 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a0031c50 con 0x7f729c003ab0 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 1 -- 10.20.193.78:0/2649996550 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0002db0 con 0x7f729c003ab0 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72a77fe640 10 monclient: dump: 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.289 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:53.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 4 set_mon_vals no callback set 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 10 monclient: shutdown 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 -- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c003ab0 msgr2=0x7f729c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c003ab0 0x7f729c003eb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f72a0007d00 tx=0x7f72a0031430 comp rx=0 tx=0).stop 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 -- 10.20.193.78:0/2649996550 shutdown_connections 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c005390 0x7f729c00bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c004900 0x7f729c004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 --2- 10.20.193.78:0/2649996550 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c003ab0 0x7f729c003eb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 -- 10.20.193.78:0/2649996550 >> 10.20.193.78:0/2649996550 conn(0x7f72b00a6620 msgr2=0x7f72b00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 -- 10.20.193.78:0/2649996550 shutdown_connections 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.273+0000 7f72b710a640 1 -- 10.20.193.78:0/2649996550 wait complete. 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient: monmap: 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:53.292 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 1 Processor -- start 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 1 -- start start 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient: init 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c003ab0 0x7f72b01cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient(hunting): picked mon.noname-b con 0x7f729c003ab0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c004900 0x7f72b01cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient(hunting): picked mon.noname-a con 0x7f729c004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c005390 0x7f72b01249a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient(hunting): picked mon.noname-c con 0x7f729c005390 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72b009bfa0 con 0x7f729c004900 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72b009be20 con 0x7f729c003ab0 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f72b006cc70 con 0x7f729c005390 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b710a640 10 monclient(hunting): authenticate will time out at 1858.828597s 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c004900 0x7f72b01cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c004900 0x7f72b01cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39108/0 (socket says 10.20.193.78:39108) 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 -- 10.20.193.78:0/1480129976 learned_addr learned my addr 10.20.193.78:0/1480129976 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient(hunting): get_auth_request con 0x7f729c004900 auth_method 0 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b5680640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c005390 0x7f72b01249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b5680640 10 monclient(hunting): get_auth_request con 0x7f729c005390 auth_method 0 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b5680640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b5680640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b5680640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c003ab0 0x7f72b01cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request con 0x7f729c003ab0 auth_method 0 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b4e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b4e7f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b4e7f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b5680640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b5680640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72b5680640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient(hunting): handle_auth_done global_id 254097 payload 824 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 -- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c005390 msgr2=0x7f72b01249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c005390 0x7f72b01249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 -- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c003ab0 msgr2=0x7f72b01cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c003ab0 0x7f72b01cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 monclient: found mon.noname-a 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.274+0000 7f72a7fff640 1 -- 10.20.193.78:0/1480129976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b0128050 con 0x7f729c004900 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.276+0000 7f72a7fff640 1 -- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c004900 msgr2=0x7f72b01cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.297 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.276+0000 7f72a7fff640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c004900 0x7f72b01cf890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7298003940 tx=0x7f7298003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.276+0000 7f72a57fa640 1 -- 10.20.193.78:0/1480129976 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298028070 con 0x7f729c004900 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: dump: 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.298 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 1 -- 10.20.193.78:0/1480129976 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7298019d80 con 0x7f729c004900 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 1 -- 10.20.193.78:0/1480129976 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298023620 con 0x7f729c004900 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:53.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a6ffd640 4 set_mon_vals no callback set 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 10 monclient: dump: 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 5 monclient: authenticate success, global_id 254097 2026-02-17T06:06:53.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 10 monclient: _renew_subs 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b0136ea0 con 0x7f729c004900 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 10 monclient: _renew_subs 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b01288f0 con 0x7f729c004900 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 1 -- 10.20.193.78:0/1480129976 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7298015040 con 0x7f729c004900 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f728003df00 0x7f72800403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b0128700 con 0x7f729c004900 2026-02-17T06:06:53.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.277+0000 7f72a57fa640 1 -- 10.20.193.78:0/1480129976 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(724..724 src has 525..724) ==== 4735+0+0 (secure 0 0 0) 0x7f7298053cc0 con 0x7f729c004900 2026-02-17T06:06:53.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.278+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f728003df00 0x7f72800403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.278+0000 7f72b4e7f640 10 monclient: get_auth_request con 0x7f728003df00 auth_method 0 2026-02-17T06:06:53.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.280+0000 7f72a57fa640 1 -- 10.20.193.78:0/1480129976 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f72980526f0 con 0x7f729c004900 2026-02-17T06:06:53.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.280+0000 7f72a57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.280+0000 7f72a57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:53.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.280+0000 7f72b4e7f640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f728003df00 0x7f72800403c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f72a0038000 tx=0x7f72a0035000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.315 INFO:tasks.rados.rados.0.trial115.stdout:3338: done (1 left) 2026-02-17T06:06:53.315 INFO:tasks.rados.rados.0.trial115.stdout:3339: copy_from oid 33 from oid 41 current snap is 353 2026-02-17T06:06:53.315 INFO:tasks.rados.rados.0.trial115.stdout:3337: finishing write tid 1 to trial11545420-13 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: finishing write tid 2 to trial11545420-13 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: finishing write tid 3 to trial11545420-13 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: finishing write tid 4 to trial11545420-13 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: finishing write tid 5 to trial11545420-13 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: oid 13 updating version 0 to 1477 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: oid 13 updating version 1477 to 1478 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: oid 13 updating version 1478 to 1479 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: oid 13 version 1479 is already newer than 1476 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1479 (ObjNum 1148 snap 353 seq_num 1148) dirty exists 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: left oid 13 (ObjNum 1148 snap 353 seq_num 1148) 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3337: done (1 left) 2026-02-17T06:06:53.316 INFO:tasks.rados.rados.0.trial115.stdout:3340: copy_from oid 17 from oid 16 current snap is 353 2026-02-17T06:06:53.317 INFO:tasks.rados.rados.0.trial115.stdout:3341: snap_remove snap 350 2026-02-17T06:06:53.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.397+0000 7f72b710a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:53.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.397+0000 7f72b710a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:53.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.397+0000 7f72b710a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.397+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f72b0074980 con 0x7f729c004900 2026-02-17T06:06:53.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.397+0000 7f72a57fa640 1 -- 10.20.193.78:0/1480129976 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f7298022de0 con 0x7f729c004900 2026-02-17T06:06:53.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.397+0000 7f72a57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:53.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.397+0000 7f72a57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:53.400 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":242,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:53.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f728003df00 msgr2=0x7f72800403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f728003df00 0x7f72800403c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f72a0038000 tx=0x7f72a0035000 comp rx=0 tx=0).stop 2026-02-17T06:06:53.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 10 monclient: shutdown 2026-02-17T06:06:53.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c004900 msgr2=0x7f72b01cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c004900 0x7f72b01cf890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7298003940 tx=0x7f7298003e10 comp rx=0 tx=0).stop 2026-02-17T06:06:53.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 shutdown_connections 2026-02-17T06:06:53.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f729c005390 0x7f72b01249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f729c004900 0x7f72b01cf890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f728003df00 0x7f72800403c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 --2- 10.20.193.78:0/1480129976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f729c003ab0 0x7f72b01cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 >> 10.20.193.78:0/1480129976 conn(0x7f72b00a6620 msgr2=0x7f72b00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.398+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 shutdown_connections 2026-02-17T06:06:53.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.399+0000 7f72b710a640 1 -- 10.20.193.78:0/1480129976 wait complete. 2026-02-17T06:06:53.408 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:06:53.408 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:06:53.409 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.474+0000 7f6323787640 10 monclient: get_monmap_and_config 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.474+0000 7f6323787640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.474+0000 7f6323787640 10 monclient: monmap: 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:53.475963+0000 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:53.475963+0000 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:53.475 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 1 Processor -- start 2026-02-17T06:06:53.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 1 -- start start 2026-02-17T06:06:53.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient: init 2026-02-17T06:06:53.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f631c09ea90 0x7f631c0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient(hunting): picked mon.noname-a con 0x7f631c09ea90 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f631c09ee70 0x7f631c09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient(hunting): picked mon.noname-c con 0x7f631c09ee70 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f631c09d980 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c0a0a20 con 0x7f631c09d980 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f631c0a0d00 con 0x7f631c09ea90 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f631c0a0460 con 0x7f631c09ee70 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6323787640 10 monclient(hunting): authenticate will time out at 1859.029672s 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f631c09ea90 0x7f631c0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f631c09ea90 0x7f631c0a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39122/0 (socket says 10.20.193.78:39122) 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 1 -- 10.20.193.78:0/2202852862 learned_addr learned my addr 10.20.193.78:0/2202852862 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 10 monclient(hunting): get_auth_request con 0x7f631c09ea90 auth_method 0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6320cfb640 1 --2- 10.20.193.78:0/2202852862 >> v2:10.20.193.115:3300/0 conn(0x7f631c09ee70 0x7f631c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6320cfb640 10 monclient(hunting): get_auth_request con 0x7f631c09ee70 auth_method 0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6320cfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6320cfb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f6320cfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 10 monclient(hunting): handle_auth_done global_id 254100 payload 274 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.475+0000 7f63214fc640 1 -- 10.20.193.78:0/2202852862 >> v2:10.20.193.115:3300/0 conn(0x7f631c09ee70 msgr2=0x7f631c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.476+0000 7f63214fc640 1 --2- 10.20.193.78:0/2202852862 >> v2:10.20.193.115:3300/0 conn(0x7f631c09ee70 0x7f631c09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.476+0000 7f63214fc640 1 -- 10.20.193.78:0/2202852862 >> v1:10.20.193.115:6789/0 conn(0x7f631c09d980 legacy=0x7f631c09dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.476+0000 7f63214fc640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.476+0000 7f63214fc640 1 monclient: found mon.noname-a 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.476+0000 7f63214fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.476+0000 7f63214fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.476+0000 7f63214fc640 1 -- 10.20.193.78:0/2202852862 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c0a0740 con 0x7f631c09ea90 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f63214fc640 1 -- 10.20.193.78:0/2202852862 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ea90 msgr2=0x7f631c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f63214fc640 1 --2- 10.20.193.78:0/2202852862 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ea90 0x7f631c0a13d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f631c06e270 tx=0x7f630c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2c76e8e5b60617 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 1 -- 10.20.193.78:0/2202852862 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c03c070 con 0x7f631c09ea90 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: dump: 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 1 -- 10.20.193.78:0/2202852862 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c02fca0 con 0x7f631c09ea90 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f6323787640 5 monclient: authenticate success, global_id 254100 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f6323787640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 1 -- 10.20.193.78:0/2202852862 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c037660 con 0x7f631c09ea90 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f6323787640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:53.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f6323787640 4 set_mon_vals no callback set 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.478+0000 7f630bfff640 10 monclient: dump: 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.480 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 10 monclient: shutdown 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 -- 10.20.193.78:0/2202852862 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ea90 msgr2=0x7f631c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 --2- 10.20.193.78:0/2202852862 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ea90 0x7f631c0a13d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f631c06e270 tx=0x7f630c02ee60 comp rx=0 tx=0).stop 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 -- 10.20.193.78:0/2202852862 shutdown_connections 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 --2- 10.20.193.78:0/2202852862 >> v2:10.20.193.115:3300/0 conn(0x7f631c09ee70 0x7f631c09f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 --2- 10.20.193.78:0/2202852862 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ea90 0x7f631c0a13d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 -- 10.20.193.78:0/2202852862 >> 10.20.193.78:0/2202852862 conn(0x7f631c0a7520 msgr2=0x7f631c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 -- 10.20.193.78:0/2202852862 shutdown_connections 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 -- 10.20.193.78:0/2202852862 wait complete. 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 10 monclient: monmap: 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:53.481 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 Processor -- start 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 1 -- start start 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.479+0000 7f6323787640 10 monclient: init 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f631c09d980 0x7f631c12fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient(hunting): picked mon.noname-b con 0x7f631c09d980 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f631c09ea90 0x7f631c130020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient(hunting): picked mon.noname-c con 0x7f631c09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ee70 0x7f631c130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient(hunting): picked mon.noname-a con 0x7f631c09ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0a66a0 con 0x7f631c09ee70 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0a1910 con 0x7f631c09d980 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f631c0a6820 con 0x7f631c09ea90 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6323787640 10 monclient(hunting): authenticate will time out at 1859.034127s 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ee70 0x7f631c130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6320cfb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f631c09ea90 0x7f631c130020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ee70 0x7f631c130560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39126/0 (socket says 10.20.193.78:39126) 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 -- 10.20.193.78:0/4211830809 learned_addr learned my addr 10.20.193.78:0/4211830809 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient(hunting): get_auth_request con 0x7f631c09ee70 auth_method 0 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6320cfb640 10 monclient(hunting): get_auth_request con 0x7f631c09ea90 auth_method 0 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6320cfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6320cfb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6320cfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6320cfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6320cfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6320cfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f63214fc640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f631c09d980 0x7f631c12fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f63214fc640 10 monclient(hunting): get_auth_request con 0x7f631c09d980 auth_method 0 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f63214fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f63214fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f63214fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient(hunting): handle_auth_done global_id 254103 payload 824 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f631c09ea90 msgr2=0x7f631c130020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f631c09ea90 0x7f631c130020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f631c09d980 msgr2=0x7f631c12fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f631c09d980 0x7f631c12fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 monclient: found mon.noname-a 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.480+0000 7f6321cfd640 1 -- 10.20.193.78:0/4211830809 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c135450 con 0x7f631c09ee70 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6321cfd640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ee70 msgr2=0x7f631c130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6321cfd640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ee70 0x7f631c130560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f631800cc10 tx=0x7f631800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318016020 con 0x7f631c09ee70 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: dump: 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6318012620 con 0x7f631c09ee70 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318012a10 con 0x7f631c09ee70 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6309ffb640 10 monclient: dump: 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.484 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f630b7fe640 4 set_mon_vals no callback set 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 5 monclient: authenticate success, global_id 254103 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 10 monclient: _renew_subs 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c130de0 con 0x7f631c09ee70 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 10 monclient: _renew_subs 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c0a6150 con 0x7f631c09ee70 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c131cd0 0x7f631c1cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.482+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1320b0 con 0x7f631c131cd0 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c131cd0 0x7f631c1cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 10 monclient: get_auth_request con 0x7f631c131cd0 auth_method 0 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6318012c30 con 0x7f631c09ee70 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 10 monclient(hunting): handle_auth_done global_id 254106 payload 824 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f63214fc640 1 -- 10.20.193.78:0/4211830809 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f631c0a5c00 con 0x7f631c131cd0 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f6309ffb640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03deb0 0x7f62ec040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(725..725 src has 525..725) ==== 4779+0+0 (secure 0 0 0) 0x7f6318053250 con 0x7f631c09ee70 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f6320cfb640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03deb0 0x7f62ec040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.483+0000 7f6320cfb640 10 monclient: get_auth_request con 0x7f62ec03deb0 auth_method 0 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f63214fc640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c131cd0 msgr2=0x7f631c1cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f63214fc640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c131cd0 0x7f631c1cf360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f631c074930 tx=0x7f630c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c03c040 con 0x7f631c131cd0 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 10 monclient: dump: 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f630c02fe50 con 0x7f631c131cd0 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c131cd0 msgr2=0x7f631c1cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c131cd0 0x7f631c1cf360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f631c074930 tx=0x7f630c004830 comp rx=0 tx=0).stop 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c131cd0 msgr2=0x7f631c1cf360 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.485+0000 7f6309ffb640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c131cd0 0x7f631c1cf360 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f631c074930 tx=0x7f630c004830 comp rx=0 tx=0).stop 2026-02-17T06:06:53.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.486+0000 7f6320cfb640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03deb0 0x7f62ec040370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f631c073570 tx=0x7f6310008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f6323787640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f6323787640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c1cf8a0 0x7f631c1da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f6323787640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f631c09e530 con 0x7f631c1cf8a0 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f63214fc640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c1cf8a0 0x7f631c1da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f63214fc640 10 monclient: get_auth_request con 0x7f631c1cf8a0 auth_method 0 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f63214fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f63214fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.488+0000 7f63214fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.489+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.489+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.489+0000 7f63214fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.489+0000 7f63214fc640 10 monclient(hunting): handle_auth_done global_id 254109 payload 824 2026-02-17T06:06:53.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.489+0000 7f63214fc640 1 -- 10.20.193.78:0/4211830809 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f631c0744d0 con 0x7f631c1cf8a0 2026-02-17T06:06:53.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.491+0000 7f63214fc640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c1cf8a0 msgr2=0x7f631c1da7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.491 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.491+0000 7f63214fc640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c1cf8a0 0x7f631c1da7e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f631c06c650 tx=0x7f630c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c03c040 con 0x7f631c1cf8a0 2026-02-17T06:06:53.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 10 monclient: dump: 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f630c003da0 con 0x7f631c1cf8a0 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c1cf8a0 msgr2=0x7f631c1da7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c1cf8a0 0x7f631c1da7e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f631c06c650 tx=0x7f630c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c1cf8a0 msgr2=0x7f631c1da7e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6309ffb640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c1cf8a0 0x7f631c1da7e0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f631c06c650 tx=0x7f630c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 242, 2026-02-17T06:06:53.494 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 12867, 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:53.495 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.496 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:53.497 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:53.498 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03deb0 msgr2=0x7f62ec040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f62ec03deb0 0x7f62ec040370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f631c073570 tx=0x7f6310008040 comp rx=0 tx=0).stop 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 10 monclient: shutdown 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ee70 msgr2=0x7f631c130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 1 --2- 10.20.193.78:0/4211830809 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f631c09ee70 0x7f631c130560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f631800cc10 tx=0x7f631800a430 comp rx=0 tx=0).stop 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6321cfd640 1 -- 10.20.193.78:0/4211830809 reap_dead start 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6321cfd640 1 -- 10.20.193.78:0/4211830809 reap_dead start 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 shutdown_connections 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 >> 10.20.193.78:0/4211830809 conn(0x7f631c0a7520 msgr2=0x7f631c0a3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 shutdown_connections 2026-02-17T06:06:53.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.493+0000 7f6323787640 1 -- 10.20.193.78:0/4211830809 wait complete. 2026-02-17T06:06:53.499 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:53.499 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:06:53.500 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:06:53.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.564+0000 7f499560e640 10 monclient: get_monmap_and_config 2026-02-17T06:06:53.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.564+0000 7f499560e640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.564+0000 7f499560e640 10 monclient: monmap: 2026-02-17T06:06:53.567 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.567 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:53.566139+0000 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:53.566139+0000 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 1 Processor -- start 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 1 -- start start 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient: init 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f499006c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f499009c4a0 0x7f499013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient(hunting): picked mon.noname-c con 0x7f499009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f499006d380 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499006daf0 con 0x7f499006c890 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499009cc80 con 0x7f499006d380 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f49900a0880 con 0x7f499009c4a0 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f499560e640 10 monclient(hunting): authenticate will time out at 1859.119750s 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f498effd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f499006c890 0x7f499006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56468/0 (socket says 10.20.193.78:56468) 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f498effd640 1 -- 10.20.193.78:0/3507689684 learned_addr learned my addr 10.20.193.78:0/3507689684 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.565+0000 7f498e7fc640 1 --2- 10.20.193.78:0/3507689684 >> v2:10.20.193.115:3300/0 conn(0x7f499009c4a0 0x7f499013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient(hunting): get_auth_request con 0x7f499009c4a0 auth_method 0 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient(hunting): handle_auth_done global_id 214375 payload 274 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 1 -- 10.20.193.78:0/3507689684 >> v1:10.20.193.115:6789/0 conn(0x7f499006d380 legacy=0x7f499013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 1 -- 10.20.193.78:0/3507689684 >> v1:10.20.193.78:6789/0 conn(0x7f499006c890 legacy=0x7f499006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 1 monclient: found mon.noname-c 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:06:53.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.566+0000 7f498e7fc640 1 -- 10.20.193.78:0/3507689684 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499009cf60 con 0x7f499009c4a0 2026-02-17T06:06:53.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.568+0000 7f498e7fc640 1 -- 10.20.193.78:0/3507689684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499009c4a0 msgr2=0x7f499013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.568+0000 7f498e7fc640 1 --2- 10.20.193.78:0/3507689684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499009c4a0 0x7f499013b3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f499009c980 tx=0x7f497802ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e61d76d39876a19 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 1 -- 10.20.193.78:0/3507689684 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497803c070 con 0x7f499009c4a0 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: dump: 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 5 monclient: authenticate success, global_id 214375 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 1 -- 10.20.193.78:0/3507689684 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497802fcb0 con 0x7f499009c4a0 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 1 -- 10.20.193.78:0/3507689684 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497802fe60 con 0x7f499009c4a0 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f498dffb640 10 monclient: dump: 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.578 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 4 set_mon_vals no callback set 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 10 monclient: shutdown 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 1 -- 10.20.193.78:0/3507689684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499009c4a0 msgr2=0x7f499013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 1 --2- 10.20.193.78:0/3507689684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499009c4a0 0x7f499013b3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f499009c980 tx=0x7f497802ed50 comp rx=0 tx=0).stop 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 1 -- 10.20.193.78:0/3507689684 shutdown_connections 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 1 --2- 10.20.193.78:0/3507689684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499009c4a0 0x7f499013b3a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 1 -- 10.20.193.78:0/3507689684 >> 10.20.193.78:0/3507689684 conn(0x7f49900a6620 msgr2=0x7f49900a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 1 -- 10.20.193.78:0/3507689684 shutdown_connections 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 1 -- 10.20.193.78:0/3507689684 wait complete. 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 10 monclient: monmap: 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.569+0000 7f499560e640 1 Processor -- start 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 1 -- start start 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient: init 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:53.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499006c890 0x7f49901cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient(hunting): picked mon.noname-b con 0x7f499006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499006d380 0x7f4990136290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient(hunting): picked mon.noname-c con 0x7f499006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f499009c4a0 0x7f49901367d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient(hunting): picked mon.noname-a con 0x7f499009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f499009bfa0 con 0x7f499009c4a0 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f499009be20 con 0x7f499006c890 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f499013f010 con 0x7f499006d380 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f499560e640 10 monclient(hunting): authenticate will time out at 1859.124323s 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499006d380 0x7f4990136290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f499009c4a0 0x7f49901367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499006d380 0x7f4990136290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44932/0 (socket says 10.20.193.78:44932) 2026-02-17T06:06:53.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f499009c4a0 0x7f49901367d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39156/0 (socket says 10.20.193.78:39156) 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 1 -- 10.20.193.78:0/1609172072 learned_addr learned my addr 10.20.193.78:0/1609172072 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 10 monclient(hunting): get_auth_request con 0x7f499006d380 auth_method 0 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient(hunting): get_auth_request con 0x7f499009c4a0 auth_method 0 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient(hunting): handle_auth_done global_id 254112 payload 824 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499006d380 msgr2=0x7f4990136290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499006d380 0x7f4990136290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498effd640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499006c890 0x7f49901cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499006c890 msgr2=0x7f49901cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499006c890 0x7f49901cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 1 monclient: found mon.noname-a 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498f7fe640 1 -- 10.20.193.78:0/1609172072 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4990121c50 con 0x7f499009c4a0 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498e7fc640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499006d380 0x7f4990136290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498effd640 10 monclient: get_auth_request con 0x7f499006c890 auth_method 0 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.570+0000 7f498effd640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f499006c890 0x7f49901cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:06:53.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f498f7fe640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f499009c4a0 msgr2=0x7f49901367d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f498f7fe640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f499009c4a0 0x7f49901367d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f498000caf0 tx=0x7f4980004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980016030 con 0x7f499009c4a0 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: dump: 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.584 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498000ac50 con 0x7f499009c4a0 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f499560e640 5 monclient: authenticate success, global_id 254112 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f499560e640 10 monclient: _renew_subs 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f499560e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499013e480 con 0x7f499009c4a0 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980004a80 con 0x7f499009c4a0 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f496f7fe640 10 monclient: dump: 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.572+0000 7f498d7fa640 4 set_mon_vals no callback set 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f499560e640 10 monclient: _renew_subs 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f499560e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4990121970 con 0x7f499009c4a0 2026-02-17T06:06:53.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f499560e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f499560e640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011dd60 0x7f499011e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f499560e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4990121410 con 0x7f499011dd60 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4980023060 con 0x7f499009c4a0 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011dd60 0x7f499011e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 10 monclient: get_auth_request con 0x7f499011dd60 auth_method 0 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 10 monclient(hunting): handle_auth_done global_id 234302 payload 824 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f498effd640 1 -- 10.20.193.78:0/1609172072 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4990121770 con 0x7f499011dd60 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f496f7fe640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f495c03deb0 0x7f495c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.573+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(725..725 src has 525..725) ==== 4779+0+0 (secure 0 0 0) 0x7f49800529f0 con 0x7f499009c4a0 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.574+0000 7f498e7fc640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f495c03deb0 0x7f495c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.574+0000 7f498e7fc640 10 monclient: get_auth_request con 0x7f495c03deb0 auth_method 0 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.575+0000 7f498effd640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011dd60 msgr2=0x7f499011e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.575+0000 7f498effd640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011dd60 0x7f499011e140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f498400d720 tx=0x7f498400dbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.575+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984005de0 con 0x7f499011dd60 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.575+0000 7f496f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.575+0000 7f496f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.575+0000 7f496f7fe640 10 monclient: dump: 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.576+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f4984017070 con 0x7f499011dd60 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.576+0000 7f496f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.576+0000 7f496f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.576+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011dd60 msgr2=0x7f499011e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.576+0000 7f496f7fe640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011dd60 0x7f499011e140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f498400d720 tx=0x7f498400dbf0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.576+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011dd60 msgr2=0x7f499011e140 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:53.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.576+0000 7f496f7fe640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011dd60 0x7f499011e140 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f498400d720 tx=0x7f498400dbf0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.576+0000 7f498e7fc640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f495c03deb0 0x7f495c040370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4990121e60 tx=0x7f4978033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f499560e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:53.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f499560e640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011e680 0x7f49901da2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f499560e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f499006fcb0 con 0x7f499011e680 2026-02-17T06:06:53.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011e680 0x7f49901da2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 10 monclient: get_auth_request con 0x7f499011e680 auth_method 0 2026-02-17T06:06:53.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 10 monclient(hunting): handle_auth_done global_id 234305 payload 824 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.579+0000 7f498effd640 1 -- 10.20.193.78:0/1609172072 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f499006cc70 con 0x7f499011e680 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f498effd640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011e680 msgr2=0x7f49901da2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f498effd640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011e680 0x7f49901da2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f498400d720 tx=0x7f4984004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400bb40 con 0x7f499011e680 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 10 monclient: dump: 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.591 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f498400bb40 con 0x7f499011e680 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011e680 msgr2=0x7f49901da2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011e680 0x7f49901da2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f498400d720 tx=0x7f4984004a40 comp rx=0 tx=0).stop 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011e680 msgr2=0x7f49901da2f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f496f7fe640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f499011e680 0x7f49901da2f0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f498400d720 tx=0x7f4984004a40 comp rx=0 tx=0).stop 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:53.592 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 242, 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 122264, 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.593 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.594 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:53.595 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.596 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.597 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.598 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:53.599 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:53.600 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:53.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f495c03deb0 msgr2=0x7f495c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f499560e640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f495c03deb0 0x7f495c040370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4990121e60 tx=0x7f4978033000 comp rx=0 tx=0).stop 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f499560e640 10 monclient: shutdown 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f499560e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f499009c4a0 msgr2=0x7f49901367d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f499560e640 1 --2- 10.20.193.78:0/1609172072 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f499009c4a0 0x7f49901367d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f498000caf0 tx=0x7f4980004a10 comp rx=0 tx=0).stop 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f498f7fe640 1 -- 10.20.193.78:0/1609172072 reap_dead start 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.581+0000 7f498f7fe640 1 -- 10.20.193.78:0/1609172072 reap_dead start 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.582+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 shutdown_connections 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.582+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 >> 10.20.193.78:0/1609172072 conn(0x7f49900a6620 msgr2=0x7f499009db20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.582+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 shutdown_connections 2026-02-17T06:06:53.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.582+0000 7f499560e640 1 -- 10.20.193.78:0/1609172072 wait complete. 2026-02-17T06:06:53.602 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:53.602 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:06:53.602 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:06:53.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 10 monclient: get_monmap_and_config 2026-02-17T06:06:53.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 10 monclient: monmap: 2026-02-17T06:06:53.671 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:53.670471+0000 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:53.670471+0000 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 1 Processor -- start 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 1 -- start start 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 10 monclient: init 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2026-02-17T06:06:53.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f135006c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.669+0000 7f13577f7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f135009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f13577f7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f135006d380 0x7f135013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f13577f7640 10 monclient(hunting): picked mon.noname-c con 0x7f135006d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f13577f7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f135006daf0 con 0x7f135006c890 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f13577f7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f135009cc80 con 0x7f135009c4a0 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f13577f7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f13577f7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f13500a0880 con 0x7f135006d380 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f13577f7640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f13577f7640 10 monclient(hunting): authenticate will time out at 1859.224033s 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f135556c640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f135006c890 0x7f135006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56470/0 (socket says 10.20.193.78:56470) 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f135556c640 1 -- 10.20.193.78:0/3282980533 learned_addr learned my addr 10.20.193.78:0/3282980533 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 1 --2- 10.20.193.78:0/3282980533 >> v2:10.20.193.115:3300/0 conn(0x7f135006d380 0x7f135013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient(hunting): get_auth_request con 0x7f135006d380 auth_method 0 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient(hunting): handle_auth_done global_id 214378 payload 274 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 1 -- 10.20.193.78:0/3282980533 >> v1:10.20.193.115:6789/0 conn(0x7f135009c4a0 legacy=0x7f135013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 1 -- 10.20.193.78:0/3282980533 >> v1:10.20.193.78:6789/0 conn(0x7f135006c890 legacy=0x7f135006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 1 monclient: found mon.noname-c 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:06:53.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.670+0000 7f1355d6d640 1 -- 10.20.193.78:0/3282980533 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135009cf60 con 0x7f135006d380 2026-02-17T06:06:53.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f1355d6d640 1 -- 10.20.193.78:0/3282980533 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 msgr2=0x7f135013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f1355d6d640 1 --2- 10.20.193.78:0/3282980533 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 0x7f135013ead0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1350074de0 tx=0x7f134c02f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a225f6ce4dbee3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 1 -- 10.20.193.78:0/3282980533 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c03c070 con 0x7f135006d380 2026-02-17T06:06:53.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: dump: 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 5 monclient: authenticate success, global_id 214378 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:53.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 1 -- 10.20.193.78:0/3282980533 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134c037640 con 0x7f135006d380 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 1 -- 10.20.193.78:0/3282980533 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c037a10 con 0x7f135006d380 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 4 set_mon_vals no callback set 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f133ffff640 10 monclient: dump: 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 10 monclient: shutdown 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 1 -- 10.20.193.78:0/3282980533 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 msgr2=0x7f135013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 1 --2- 10.20.193.78:0/3282980533 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 0x7f135013ead0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1350074de0 tx=0x7f134c02f130 comp rx=0 tx=0).stop 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 1 -- 10.20.193.78:0/3282980533 shutdown_connections 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 1 --2- 10.20.193.78:0/3282980533 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 0x7f135013ead0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 1 -- 10.20.193.78:0/3282980533 >> 10.20.193.78:0/3282980533 conn(0x7f13500a6620 msgr2=0x7f13500a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 1 -- 10.20.193.78:0/3282980533 shutdown_connections 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 1 -- 10.20.193.78:0/3282980533 wait complete. 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.673+0000 7f13577f7640 10 monclient: monmap: 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:53.681 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 1 Processor -- start 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 1 -- start start 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient: init 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f135006c890 0x7f13501321a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient(hunting): picked mon.noname-c con 0x7f135006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 0x7f13501326e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient(hunting): picked mon.noname-b con 0x7f135006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f135009c4a0 0x7f13501ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient(hunting): picked mon.noname-a con 0x7f135009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f135009bfa0 con 0x7f135009c4a0 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f135009be20 con 0x7f135006d380 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f135013f010 con 0x7f135006c890 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f13577f7640 10 monclient(hunting): authenticate will time out at 1859.228304s 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f135006c890 0x7f13501321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f135006c890 0x7f13501321a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44972/0 (socket says 10.20.193.78:44972) 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f135009c4a0 0x7f13501ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 -- 10.20.193.78:0/3954448757 learned_addr learned my addr 10.20.193.78:0/3954448757 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient(hunting): get_auth_request con 0x7f135006c890 auth_method 0 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 10 monclient(hunting): get_auth_request con 0x7f135009c4a0 auth_method 0 2026-02-17T06:06:53.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1354d6b640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 0x7f13501326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1354d6b640 10 monclient(hunting): get_auth_request con 0x7f135006d380 auth_method 0 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1354d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1354d6b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1354d6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient(hunting): handle_auth_done global_id 234308 payload 824 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 msgr2=0x7f13501326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 0x7f13501326e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f135009c4a0 msgr2=0x7f13501ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f135009c4a0 0x7f13501ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 monclient: found mon.noname-c 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f135556c640 1 -- 10.20.193.78:0/3954448757 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135009d810 con 0x7f135006c890 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1355d6d640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f135009c4a0 0x7f13501ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.674+0000 7f1354d6b640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f135006d380 0x7f13501326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f135556c640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f135006c890 msgr2=0x7f13501321a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f135556c640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f135006c890 0x7f13501321a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f134000c750 tx=0x7f134000cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340005de0 con 0x7f135006c890 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: dump: 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.686 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1340007070 con 0x7f135006c890 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f13577f7640 5 monclient: authenticate success, global_id 234308 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f13577f7640 10 monclient: _renew_subs 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f13577f7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135013ae10 con 0x7f135006c890 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340007440 con 0x7f135006c890 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133dffb640 10 monclient: dump: 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f133f7fe640 4 set_mon_vals no callback set 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f13577f7640 10 monclient: _renew_subs 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f13577f7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.676+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f135013bb90 con 0x7f135006c890 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f13577f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f13577f7640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350133ff0 0x7f13501343d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f13577f7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13500709f0 con 0x7f1350133ff0 2026-02-17T06:06:53.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350133ff0 0x7f13501343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 10 monclient: get_auth_request con 0x7f1350133ff0 auth_method 0 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f1340007660 con 0x7f135006c890 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f133dffb640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f132003deb0 0x7f1320040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 10 monclient(hunting): handle_auth_done global_id 214384 payload 824 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1354d6b640 1 -- 10.20.193.78:0/3954448757 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f135006fcc0 con 0x7f1350133ff0 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(725..725 src has 525..725) ==== 4779+0+0 (secure 0 0 0) 0x7f13400525e0 con 0x7f135006c890 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1355d6d640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f132003deb0 0x7f1320040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.677+0000 7f1355d6d640 10 monclient: get_auth_request con 0x7f132003deb0 auth_method 0 2026-02-17T06:06:53.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f1354d6b640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350133ff0 msgr2=0x7f13501343d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f1354d6b640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350133ff0 0x7f13501343d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f135013c790 tx=0x7f134400eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134400c910 con 0x7f1350133ff0 2026-02-17T06:06:53.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 10 monclient: dump: 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f134400c910 con 0x7f1350133ff0 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350133ff0 msgr2=0x7f13501343d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350133ff0 0x7f13501343d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f135013c790 tx=0x7f134400eba0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350133ff0 msgr2=0x7f13501343d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f133dffb640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350133ff0 0x7f13501343d0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f135013c790 tx=0x7f134400eba0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.680+0000 7f1355d6d640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f132003deb0 0x7f1320040370 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f135006a480 tx=0x7f134c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f13577f7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f13577f7640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350134910 0x7f1350134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f13577f7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1350132c20 con 0x7f1350134910 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350134910 0x7f1350134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 10 monclient: get_auth_request con 0x7f1350134910 auth_method 0 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 10 monclient(hunting): handle_auth_done global_id 214387 payload 824 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.683+0000 7f1354d6b640 1 -- 10.20.193.78:0/3954448757 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f135006cc70 con 0x7f1350134910 2026-02-17T06:06:53.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f1354d6b640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350134910 msgr2=0x7f1350134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f1354d6b640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350134910 0x7f1350134d20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1344018000 tx=0x7f1344003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134401ea80 con 0x7f1350134910 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 10 monclient: dump: 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f1344007030 con 0x7f1350134910 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:53.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350134910 msgr2=0x7f1350134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350134910 0x7f1350134d20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1344018000 tx=0x7f1344003860 comp rx=0 tx=0).stop 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350134910 msgr2=0x7f1350134d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f133dffb640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f1350134910 0x7f1350134d20 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1344018000 tx=0x7f1344003860 comp rx=0 tx=0).stop 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 242, 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 216534, 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:53.695 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.696 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:53.697 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:53.698 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.699 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.700 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:53.701 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:53.702 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:53.703 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:53.704 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:53.705 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:53.705 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f132003deb0 msgr2=0x7f1320040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f132003deb0 0x7f1320040370 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f135006a480 tx=0x7f134c033000 comp rx=0 tx=0).stop 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 10 monclient: shutdown 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f135006c890 msgr2=0x7f13501321a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f135006c890 0x7f13501321a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f134000c750 tx=0x7f134000cc20 comp rx=0 tx=0).stop 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f1355d6d640 1 -- 10.20.193.78:0/3954448757 reap_dead start 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 shutdown_connections 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 --2- 10.20.193.78:0/3954448757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f135006c890 0x7f13501321a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 >> 10.20.193.78:0/3954448757 conn(0x7f13500a6620 msgr2=0x7f13500a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 shutdown_connections 2026-02-17T06:06:53.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.686+0000 7f13577f7640 1 -- 10.20.193.78:0/3954448757 wait complete. 2026-02-17T06:06:53.707 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:53.707 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:06:53.707 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.772+0000 7f9b92353640 10 monclient: get_monmap_and_config 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.772+0000 7f9b92353640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.772+0000 7f9b92353640 10 monclient: monmap: 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:53.773707+0000 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:53.773707+0000 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:53.775 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.772+0000 7f9b92353640 1 Processor -- start 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.772+0000 7f9b92353640 1 -- start start 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.772+0000 7f9b92353640 10 monclient: init 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9b8c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c09c4a0 0x7f9b8c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient(hunting): picked mon.noname-b con 0x7f9b8c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9b8c06d380 0x7f9b8c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient(hunting): picked mon.noname-c con 0x7f9b8c06d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c06daf0 con 0x7f9b8c06c890 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c0a0880 con 0x7f9b8c06d380 2026-02-17T06:06:53.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c09cc80 con 0x7f9b8c09c4a0 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b92353640 10 monclient(hunting): authenticate will time out at 1859.327222s 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c09c4a0 0x7f9b8c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c09c4a0 0x7f9b8c13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44976/0 (socket says 10.20.193.78:44976) 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/645593397 learned_addr learned my addr 10.20.193.78:0/645593397 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient(hunting): get_auth_request con 0x7f9b8c09c4a0 auth_method 0 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b908c9640 1 --2- 10.20.193.78:0/645593397 >> v2:10.20.193.115:3300/0 conn(0x7f9b8c06d380 0x7f9b8c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient(hunting): handle_auth_done global_id 234311 payload 274 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/645593397 >> v2:10.20.193.115:3300/0 conn(0x7f9b8c06d380 msgr2=0x7f9b8c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 1 --2- 10.20.193.78:0/645593397 >> v2:10.20.193.115:3300/0 conn(0x7f9b8c06d380 0x7f9b8c13d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/645593397 >> v1:10.20.193.115:6789/0 conn(0x7f9b8c06c890 legacy=0x7f9b8c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:53.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 1 monclient: found mon.noname-b 2026-02-17T06:06:53.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.773+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/645593397 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c09cf60 con 0x7f9b8c09c4a0 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/645593397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c09c4a0 msgr2=0x7f9b8c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8b7fe640 1 --2- 10.20.193.78:0/645593397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c09c4a0 0x7f9b8c13b3a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009970 tx=0x7f9b7c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aabd5df7c1d58c5e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 1 -- 10.20.193.78:0/645593397 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c03c070 con 0x7f9b8c09c4a0 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: dump: 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.782 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 1 -- 10.20.193.78:0/645593397 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c02fdb0 con 0x7f9b8c09c4a0 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 1 -- 10.20.193.78:0/645593397 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c037660 con 0x7f9b8c09c4a0 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b8affd640 10 monclient: dump: 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.783 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b92353640 5 monclient: authenticate success, global_id 234311 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b92353640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.775+0000 7f9b92353640 4 set_mon_vals no callback set 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 10 monclient: shutdown 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 1 -- 10.20.193.78:0/645593397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c09c4a0 msgr2=0x7f9b8c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 1 --2- 10.20.193.78:0/645593397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c09c4a0 0x7f9b8c13b3a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009970 tx=0x7f9b7c02ee90 comp rx=0 tx=0).stop 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 1 -- 10.20.193.78:0/645593397 shutdown_connections 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 1 --2- 10.20.193.78:0/645593397 >> v2:10.20.193.115:3300/0 conn(0x7f9b8c06d380 0x7f9b8c13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 1 --2- 10.20.193.78:0/645593397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c09c4a0 0x7f9b8c13b3a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 1 -- 10.20.193.78:0/645593397 >> 10.20.193.78:0/645593397 conn(0x7f9b8c0a6620 msgr2=0x7f9b8c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 1 -- 10.20.193.78:0/645593397 shutdown_connections 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 1 -- 10.20.193.78:0/645593397 wait complete. 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 10 monclient: build_initial_monmap 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.776+0000 7f9b92353640 10 monclient: monmap: 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 1 Processor -- start 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 1 -- start start 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient: init 2026-02-17T06:06:53.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9b8c06c890 0x7f9b8c1ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient(hunting): picked mon.noname-b con 0x7f9b8c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c06d380 0x7f9b8c131f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient(hunting): picked mon.noname-c con 0x7f9b8c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9b8c09c4a0 0x7f9b8c139080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient(hunting): picked mon.noname-a con 0x7f9b8c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c09bfa0 con 0x7f9b8c09c4a0 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c09be20 con 0x7f9b8c06c890 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c13de30 con 0x7f9b8c06d380 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b92353640 10 monclient(hunting): authenticate will time out at 1859.331329s 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c06d380 0x7f9b8c131f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c06d380 0x7f9b8c131f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:44984/0 (socket says 10.20.193.78:44984) 2026-02-17T06:06:53.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/2175242247 learned_addr learned my addr 10.20.193.78:0/2175242247 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient(hunting): get_auth_request con 0x7f9b8c06d380 auth_method 0 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b908c9640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9b8c09c4a0 0x7f9b8c139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b908c9640 10 monclient(hunting): get_auth_request con 0x7f9b8c09c4a0 auth_method 0 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b908c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b908c9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b908c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b908c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b908c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b908c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient(hunting): handle_auth_done global_id 234314 payload 824 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9b8c06c890 msgr2=0x7f9b8c1ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9b8c06c890 0x7f9b8c1ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9b8c09c4a0 msgr2=0x7f9b8c139080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9b8c09c4a0 0x7f9b8c139080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 monclient: found mon.noname-c 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.777+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/2175242247 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c13da50 con 0x7f9b8c06d380 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b8b7fe640 1 -- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c06d380 msgr2=0x7f9b8c131f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b8b7fe640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c06d380 0x7f9b8c131f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009aa0 tx=0x7f9b7c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 1 -- 10.20.193.78:0/2175242247 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c03c050 con 0x7f9b8c06d380 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: dump: 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: _finish_auth 0 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 1 -- 10.20.193.78:0/2175242247 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c048070 con 0x7f9b8c06d380 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b92353640 5 monclient: authenticate success, global_id 234314 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b92353640 10 monclient: _renew_subs 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b92353640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c09d810 con 0x7f9b8c06d380 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 1 -- 10.20.193.78:0/2175242247 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c041a60 con 0x7f9b8c06d380 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b88ff9640 10 monclient: dump: 2026-02-17T06:06:53.789 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b8a7fc640 4 set_mon_vals no callback set 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b92353640 10 monclient: _renew_subs 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b92353640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.779+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c1cdc20 con 0x7f9b8c06d380 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b88ff9640 1 -- 10.20.193.78:0/2175242247 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9b7c041c00 con 0x7f9b8c06d380 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b88ff9640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b5c03deb0 0x7f9b5c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b92353640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b92353640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b92353640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b88ff9640 1 -- 10.20.193.78:0/2175242247 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(725..725 src has 525..725) ==== 4779+0+0 (secure 0 0 0) 0x7f9b7c077f00 con 0x7f9b8c06d380 2026-02-17T06:06:53.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c074980 con 0x7f9b8c06d380 2026-02-17T06:06:53.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b8bfff640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b5c03deb0 0x7f9b5c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:53.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.780+0000 7f9b8bfff640 10 monclient: get_auth_request con 0x7f9b5c03deb0 auth_method 0 2026-02-17T06:06:53.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.783+0000 7f9b88ff9640 1 -- 10.20.193.78:0/2175242247 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9b7c0423b0 con 0x7f9b8c06d380 2026-02-17T06:06:53.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.783+0000 7f9b88ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:53.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.783+0000 7f9b88ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:53.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.783+0000 7f9b8bfff640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b5c03deb0 0x7f9b5c040370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9b78003ec0 tx=0x7f9b78007ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:53.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.898+0000 7f9b92353640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:06:53.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.898+0000 7f9b92353640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:06:53.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.898+0000 7f9b92353640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:06:53.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.898+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f9b8c06cc70 con 0x7f9b8c06d380 2026-02-17T06:06:53.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9b88ff9640 1 -- 10.20.193.78:0/2175242247 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f9b7c037e10 con 0x7f9b8c06d380 2026-02-17T06:06:53.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9b88ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:06:53.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9b88ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=12,kv=11,logm=46} crc {auth=2899010365,config=692619927,health=3219210987,kv=2256994401,logm=4120988717}) 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=29,config=2,health=11,kv=12,logm=46} crc {auth=2916477484,config=692619927,health=1637326440,kv=449989585,logm=1561701655}) 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=12,kv=11,logm=46} crc {auth=2899010365,config=692619927,health=3219210987,kv=2256994401,logm=4120988717}) 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=29,config=2,health=9,kv=12,logm=48} crc {auth=2811020253,config=692619927,health=3829188006,kv=449989585,logm=2449400003}) 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888033296}) 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1408403959}) 2026-02-17T06:06:53.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888033296}) 2026-02-17T06:06:53.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.899+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2318391489}) 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b5c03deb0 msgr2=0x7f9b5c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b5c03deb0 0x7f9b5c040370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9b78003ec0 tx=0x7f9b78007ab0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 10 monclient: shutdown 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c06d380 msgr2=0x7f9b8c131f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c06d380 0x7f9b8c131f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009aa0 tx=0x7f9b7c004830 comp rx=0 tx=0).stop 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 shutdown_connections 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9b8c09c4a0 0x7f9b8c139080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9b8c06d380 0x7f9b8c131f70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9b5c03deb0 0x7f9b5c040370 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 --2- 10.20.193.78:0/2175242247 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9b8c06c890 0x7f9b8c1ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:53.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 >> 10.20.193.78:0/2175242247 conn(0x7f9b8c0a6620 msgr2=0x7f9b8c1361b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:53.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 shutdown_connections 2026-02-17T06:06:53.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9b92353640 1 -- 10.20.193.78:0/2175242247 wait complete. 2026-02-17T06:06:53.907 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.907 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1309124206}) 2026-02-17T06:06:53.907 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3152688664}) 2026-02-17T06:06:53.907 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.907 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1309124206}) 2026-02-17T06:06:53.907 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2858612765}) 2026-02-17T06:06:53.907 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.907 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3692579706}) 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3491706569}) 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3692579706}) 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.900+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4086384874}) 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.901+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.901+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3312409036}) 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.901+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3800227119}) 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.901+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.901+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3312409036}) 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.901+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1905208445}) 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.908 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3674669693}) 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3263538155}) 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3674669693}) 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=936005998}) 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2923213266}) 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2223034637}) 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2923213266}) 2026-02-17T06:06:53.909 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.902+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=474319626}) 2026-02-17T06:06:53.911 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:06:53.918 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.908+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.918 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.908+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2773097959}) 2026-02-17T06:06:53.918 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.908+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1997371827}) 2026-02-17T06:06:53.918 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.918 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2773097959}) 2026-02-17T06:06:53.918 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2900526452}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2274465138}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1680061953}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2274465138}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4014781475}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2358174093}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3400947241}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2358174093}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.909+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=105738300}) 2026-02-17T06:06:53.919 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1980768380}) 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1159924201}) 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1980768380}) 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290276725}) 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2095022550}) 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1104416538}) 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2095022550}) 2026-02-17T06:06:53.920 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1882712800}) 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1144563884}) 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.910+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2803546564}) 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1144563884}) 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2095988453}) 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3521797403}) 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1449951346}) 2026-02-17T06:06:53.921 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3521797403}) 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1790036815}) 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3974557577}) 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2128278584}) 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3974557577}) 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.911+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=440199288}) 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3360007021}) 2026-02-17T06:06:53.922 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619100941}) 2026-02-17T06:06:53.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3360007021}) 2026-02-17T06:06:53.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2348072139}) 2026-02-17T06:06:53.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=498407532}) 2026-02-17T06:06:53.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=194978813}) 2026-02-17T06:06:53.923 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.924 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=498407532}) 2026-02-17T06:06:53.924 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1138112793}) 2026-02-17T06:06:53.924 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.924 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2885563583}) 2026-02-17T06:06:53.924 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1318816022}) 2026-02-17T06:06:53.924 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.924 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2885563583}) 2026-02-17T06:06:53.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2884748347}) 2026-02-17T06:06:53.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=455319264}) 2026-02-17T06:06:53.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2574857120}) 2026-02-17T06:06:53.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.925 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=455319264}) 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.912+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=385944985}) 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4175975146}) 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4162870745}) 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4175975146}) 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2070631280}) 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3982876998}) 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2984363909}) 2026-02-17T06:06:53.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3982876998}) 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873471049}) 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=437248755}) 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1531548701}) 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=437248755}) 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.913+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3629850630}) 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2226383957}) 2026-02-17T06:06:53.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=720733620}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2226383957}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=964823407}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974955811}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2135051806}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974955811}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.914+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2516614734}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114207943}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=665184681}) 2026-02-17T06:06:53.928 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114207943}) 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3177110319}) 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1020752744}) 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2571893911}) 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1020752744}) 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=67422373}) 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1946485157}) 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2533056112}) 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.929 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1946485157}) 2026-02-17T06:06:53.930 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.915+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3033420419}) 2026-02-17T06:06:53.939 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.939 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4288650947}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=700420233}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4288650947}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=116618126}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4143600773}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3021890781}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4143600773}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1690684267}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.923+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3484118376}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1807812753}) 2026-02-17T06:06:53.940 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3484118376}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2705993802}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1796781683}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2872963758}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1796781683}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1621775385}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2222061382}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=829959071}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2222061382}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3027761414}) 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.941 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=474365415}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1628189085}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=474365415}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3037273008}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416105325}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4044538706}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416105325}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=583196914}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3534322840}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1232374361}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3534322840}) 2026-02-17T06:06:53.942 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.925+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1921643132}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2739720395}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2816512761}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2739720395}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=207998938}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1458884766}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3046766247}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1458884766}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.926+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=269546230}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3665175306}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3911165731}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3665175306}) 2026-02-17T06:06:53.943 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1680387701}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194779396}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1427466381}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.933+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194779396}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2296139702}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2082294788}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1845691013}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2082294788}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131362272}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=704526697}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=66220938}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=704526697}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.934+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1352359022}) 2026-02-17T06:06:53.944 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1340819845}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1446640583}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1340819845}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3481350124}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3957674333}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1905777385}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3957674333}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2765807280}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3878828585}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1095681892}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3878828585}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.935+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1828345649}) 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.945 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4164517277}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963211685}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4164517277}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=801107111}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3516154172}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2717408908}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3516154172}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3752208602}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4052712737}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3325890308}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4052712737}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.936+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1442888309}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=897100899}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2808057541}) 2026-02-17T06:06:53.946 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=897100899}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=101534278}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397314412}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1501416694}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397314412}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1201917969}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=700265686}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3279834252}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=700265686}) 2026-02-17T06:06:53.947 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:53.937+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3094674181}) 2026-02-17T06:06:54.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746367317}) 2026-02-17T06:06:54.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=124577508}) 2026-02-17T06:06:54.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746367317}) 2026-02-17T06:06:54.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=755326993}) 2026-02-17T06:06:54.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.315+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.315+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2489649847}) 2026-02-17T06:06:54.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.315+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1179012575}) 2026-02-17T06:06:54.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.315+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.315+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2489649847}) 2026-02-17T06:06:54.317 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.315+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=845146271}) 2026-02-17T06:06:54.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.321+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.321+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581445216}) 2026-02-17T06:06:54.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.321+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2459648861}) 2026-02-17T06:06:54.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.321+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.321+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581445216}) 2026-02-17T06:06:54.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.321+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2084529715}) 2026-02-17T06:06:54.325 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.323+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.325 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.323+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3387967300}) 2026-02-17T06:06:54.325 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.323+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3483184937}) 2026-02-17T06:06:54.325 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.323+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.325 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.323+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3387967300}) 2026-02-17T06:06:54.325 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.323+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3920537513}) 2026-02-17T06:06:54.326 INFO:tasks.rados.rados.0.trial115.stdout:3341: done (2 left) 2026-02-17T06:06:54.326 INFO:tasks.rados.rados.0.trial115.stdout:3342: write oid 11 current snap is 353 2026-02-17T06:06:54.327 INFO:tasks.rados.rados.0.trial115.stdout:3339: finishing copy_from racing read to trial11545420-33 2026-02-17T06:06:54.327 INFO:tasks.rados.rados.0.trial115.stdout:3342: seq_num 1149 ranges {511243=430404,1669928=664720,3103254=207989} 2026-02-17T06:06:54.327 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.325+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.327 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.325+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966179702}) 2026-02-17T06:06:54.328 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.325+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187171248}) 2026-02-17T06:06:54.328 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.325+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.328 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.325+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966179702}) 2026-02-17T06:06:54.328 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.325+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=944649489}) 2026-02-17T06:06:54.328 INFO:tasks.rados.rados.0.trial115.stdout:3342: writing trial11545420-11 from 511243 to 941647 tid 1 2026-02-17T06:06:54.334 INFO:tasks.rados.rados.0.trial115.stdout:3342: writing trial11545420-11 from 1669928 to 2334648 tid 2 2026-02-17T06:06:54.339 INFO:tasks.rados.rados.0.trial115.stdout:3342: writing trial11545420-11 from 3103254 to 3311243 tid 3 2026-02-17T06:06:54.339 INFO:tasks.rados.rados.0.trial115.stdout:3343: write (excl) oid 7 current snap is 353 2026-02-17T06:06:54.339 INFO:tasks.rados.rados.0.trial115.stdout:3343: seq_num 1150 ranges {528050=505653,1612057=524731,2614324=567403} 2026-02-17T06:06:54.343 INFO:tasks.rados.rados.0.trial115.stdout:3343: writing trial11545420-7 from 528050 to 1033703 tid 1 2026-02-17T06:06:54.347 INFO:tasks.rados.rados.0.trial115.stdout:3343: writing trial11545420-7 from 1612057 to 2136788 tid 2 2026-02-17T06:06:54.352 INFO:tasks.rados.rados.0.trial115.stdout:3343: writing trial11545420-7 from 2614324 to 3181727 tid 3 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:3344: rollback oid 18 current snap is 353 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 341 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:3340: finishing copy_from racing read to trial11545420-17 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:3339: finishing copy_from to trial11545420-33 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1569 (ObjNum 1147 snap 350 seq_num 1147) dirty exists 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:3340: finishing copy_from to trial11545420-17 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1749 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:3339: done (4 left) 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:3340: done (3 left) 2026-02-17T06:06:54.353 INFO:tasks.rados.rados.0.trial115.stdout:3345: delete oid 4 current snap is 353 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: finishing write tid 1 to trial11545420-11 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: finishing write tid 2 to trial11545420-11 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: finishing write tid 3 to trial11545420-11 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: finishing write tid 4 to trial11545420-11 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: finishing write tid 5 to trial11545420-11 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: oid 11 updating version 0 to 1728 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: oid 11 version 1728 is already newer than 1727 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: oid 11 updating version 1728 to 1729 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: oid 11 updating version 1729 to 1730 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1730 (ObjNum 1149 snap 353 seq_num 1149) dirty exists 2026-02-17T06:06:54.354 INFO:tasks.rados.rados.0.trial115.stdout:3342: left oid 11 (ObjNum 1149 snap 353 seq_num 1149) 2026-02-17T06:06:54.355 INFO:tasks.rados.rados.0.trial115.stdout:3343: finishing write tid 1 to trial11545420-7 2026-02-17T06:06:54.355 INFO:tasks.rados.rados.0.trial115.stdout:3343: finishing write tid 2 to trial11545420-7 2026-02-17T06:06:54.356 INFO:tasks.rados.rados.0.trial115.stdout:3343: finishing write tid 3 to trial11545420-7 2026-02-17T06:06:54.356 INFO:tasks.rados.rados.0.trial115.stdout:3343: finishing write tid 4 to trial11545420-7 2026-02-17T06:06:54.356 INFO:tasks.rados.rados.0.trial115.stdout:3343: finishing write tid 5 to trial11545420-7 2026-02-17T06:06:54.356 INFO:tasks.rados.rados.0.trial115.stdout:3343: oid 7 updating version 0 to 1691 2026-02-17T06:06:54.356 INFO:tasks.rados.rados.0.trial115.stdout:3343: oid 7 version 1691 is already newer than 1689 2026-02-17T06:06:54.356 INFO:tasks.rados.rados.0.trial115.stdout:3343: oid 7 version 1691 is already newer than 1690 2026-02-17T06:06:54.356 INFO:tasks.rados.rados.0.trial115.stdout:3343: oid 7 version 1691 is already newer than 1688 2026-02-17T06:06:54.356 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1691 (ObjNum 1150 snap 353 seq_num 1150) dirty exists 2026-02-17T06:06:54.357 INFO:tasks.rados.rados.0.trial115.stdout:3343: left oid 7 (ObjNum 1150 snap 353 seq_num 1150) 2026-02-17T06:06:54.357 INFO:tasks.rados.rados.0.trial115.stdout:3342: done (3 left) 2026-02-17T06:06:54.357 INFO:tasks.rados.rados.0.trial115.stdout:3343: done (2 left) 2026-02-17T06:06:54.357 INFO:tasks.rados.rados.0.trial115.stdout:3345: done (1 left) 2026-02-17T06:06:54.357 INFO:tasks.rados.rados.0.trial115.stdout:3346: copy_from oid 40 from oid 21 current snap is 353 2026-02-17T06:06:54.357 INFO:tasks.rados.rados.0.trial115.stdout:3347: snap_remove snap 343 2026-02-17T06:06:54.364 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.363+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.365 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.363+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2259251202}) 2026-02-17T06:06:54.365 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.363+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2501502763}) 2026-02-17T06:06:54.365 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.363+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.365 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.363+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2259251202}) 2026-02-17T06:06:54.365 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.363+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239589024}) 2026-02-17T06:06:54.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.654 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2949352494}) 2026-02-17T06:06:54.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3076986467}) 2026-02-17T06:06:54.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2949352494}) 2026-02-17T06:06:54.655 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=531405145}) 2026-02-17T06:06:54.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.660+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.660+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2000973541}) 2026-02-17T06:06:54.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.660+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1309346308}) 2026-02-17T06:06:54.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.660+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.660+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2000973541}) 2026-02-17T06:06:54.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.660+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540560696}) 2026-02-17T06:06:54.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173244489}) 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=345976366}) 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173244489}) 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1621623922}) 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1997205392}) 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=6734824}) 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1997205392}) 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.868+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1807208308}) 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3057487337}) 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3055422602}) 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3057487337}) 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062080267}) 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1616703341}) 2026-02-17T06:06:54.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2691730561}) 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1616703341}) 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2557732628}) 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=275229098}) 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1899506717}) 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=275229098}) 2026-02-17T06:06:54.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.869+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2365248697}) 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1734405683}) 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=226829668}) 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1734405683}) 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1633912776}) 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3039051636}) 2026-02-17T06:06:54.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=576569052}) 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3039051636}) 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1675458277}) 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=173800963}) 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1165842423}) 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=173800963}) 2026-02-17T06:06:54.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.870+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=228406058}) 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=833244702}) 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3653049488}) 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=833244702}) 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1348191322}) 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=974969125}) 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=367542530}) 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=974969125}) 2026-02-17T06:06:54.886 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2025359317}) 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2154469682}) 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1356159354}) 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2154469682}) 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4083998776}) 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=314343695}) 2026-02-17T06:06:54.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4032556785}) 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=314343695}) 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.871+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2469021648}) 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613907809}) 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3383929832}) 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613907809}) 2026-02-17T06:06:54.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4202581208}) 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2142283892}) 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3586689956}) 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2142283892}) 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1993362778}) 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2456307381}) 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2979846482}) 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2456307381}) 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.872+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2004450454}) 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634840535}) 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4186205285}) 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634840535}) 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3367649373}) 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3249684081}) 2026-02-17T06:06:54.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1748011362}) 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3249684081}) 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=365163292}) 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2021520283}) 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2994957149}) 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2021520283}) 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1398605518}) 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.873+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.891 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=474178194}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698389811}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=474178194}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1854373051}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3836777958}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1676053279}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3836777958}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2310884012}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3667670118}) 2026-02-17T06:06:54.892 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=340843678}) 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3667670118}) 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2445746546}) 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2435294735}) 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1058292093}) 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2435294735}) 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.874+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=148475895}) 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.893 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549769380}) 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=422954229}) 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549769380}) 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2264312410}) 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281376802}) 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=608884172}) 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281376802}) 2026-02-17T06:06:54.894 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.875+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3571059289}) 2026-02-17T06:06:54.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.882+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.882+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3709662982}) 2026-02-17T06:06:54.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.882+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1256089825}) 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.882+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.882+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3709662982}) 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.882+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3781813587}) 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3642051804}) 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2873173005}) 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3642051804}) 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1118143598}) 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2818837751}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622049131}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2818837751}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4267125950}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=490573691}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1390132721}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=490573691}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819802127}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900327986}) 2026-02-17T06:06:54.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1954001352}) 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900327986}) 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.883+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1174061671}) 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=12,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=2419392837,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3449690384,mon_config_key=3818802971,monmap=3287446063,osd_metadata=2997046373,osd_pg_creating=3615593850,osd_snap=937855741}) 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=34,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=20} crc {logm=4190063924,mdsmap=1160443688,mgr=1527436628,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1088898585,mon_config_key=1912222953,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=3615593850,osd_snap=219930284}) 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=12,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=25} crc {logm=2419392837,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3449690384,mon_config_key=3818802971,monmap=3287446063,osd_metadata=2997046373,osd_pg_creating=3615593850,osd_snap=937855741}) 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=33,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3527665044,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1621033545,mon_config_key=1705260644,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=3615593850,osd_snap=2107240897}) 2026-02-17T06:06:54.902 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=3269820557}) 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=99,osdmap=1} crc {osd_snap=498889120,osdmap=2832771375}) 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=3269820557}) 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=100} crc {osd_snap=1699252579}) 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=2,osdmap=98} crc {osd_snap=1735905899,osdmap=3948108954}) 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=2,osdmap=98} crc {osd_snap=1735905899,osdmap=2817809992}) 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=2,osdmap=98} crc {osd_snap=1735905899,osdmap=3948108954}) 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.884+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=2,osdmap=98} crc {osd_snap=1735905899,osdmap=4091625974}) 2026-02-17T06:06:54.903 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2779350181}) 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4183691813}) 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2779350181}) 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2411208711}) 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3590382760}) 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=79845721}) 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3590382760}) 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.885+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3974564122}) 2026-02-17T06:06:54.904 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.888+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.905 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.888+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=9} crc {osdmap=4193171291}) 2026-02-17T06:06:54.905 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.888+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=8} crc {osdmap=1504823896}) 2026-02-17T06:06:54.905 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.888+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:06:54.905 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.888+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=9} crc {osdmap=4193171291}) 2026-02-17T06:06:54.905 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:06:54.888+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=8} crc {osdmap=3399728128}) 2026-02-17T06:06:54.911 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:06:54.912 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:54.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.980+0000 7efdba722640 10 monclient: get_monmap_and_config 2026-02-17T06:06:54.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.980+0000 7efdba722640 10 monclient: build_initial_monmap 2026-02-17T06:06:54.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.980+0000 7efdba722640 10 monclient: monmap: 2026-02-17T06:06:54.985 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:54.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:54.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:54.982132+0000 2026-02-17T06:06:54.985 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:54.982132+0000 2026-02-17T06:06:54.985 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 1 Processor -- start 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 1 -- start start 2026-02-17T06:06:54.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient: init 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7efdb406c890 0x7efdb406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient(hunting): picked mon.noname-c con 0x7efdb406c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efdb409c4a0 0x7efdb406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient(hunting): picked mon.noname-a con 0x7efdb409c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efdb4134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb406daf0 con 0x7efdb4134df0 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7efdb40a0880 con 0x7efdb409c4a0 2026-02-17T06:06:54.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7efdb406d6f0 con 0x7efdb406c890 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdba722640 10 monclient(hunting): authenticate will time out at 1860.535705s 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efdb409c4a0 0x7efdb406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7efdb409c4a0 0x7efdb406d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39188/0 (socket says 10.20.193.78:39188) 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 1 -- 10.20.193.78:0/2339842094 learned_addr learned my addr 10.20.193.78:0/2339842094 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 10 monclient(hunting): get_auth_request con 0x7efdb409c4a0 auth_method 0 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:54.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb3fff640 1 --2- 10.20.193.78:0/2339842094 >> v2:10.20.193.115:3300/0 conn(0x7efdb406c890 0x7efdb406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb3fff640 10 monclient(hunting): get_auth_request con 0x7efdb406c890 auth_method 0 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.981+0000 7efdb37fe640 10 monclient(hunting): handle_auth_done global_id 254121 payload 274 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.982+0000 7efdb37fe640 1 -- 10.20.193.78:0/2339842094 >> v2:10.20.193.115:3300/0 conn(0x7efdb406c890 msgr2=0x7efdb406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:54.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.982+0000 7efdb37fe640 1 --2- 10.20.193.78:0/2339842094 >> v2:10.20.193.115:3300/0 conn(0x7efdb406c890 0x7efdb406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:54.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.982+0000 7efdb37fe640 1 -- 10.20.193.78:0/2339842094 >> v1:10.20.193.115:6789/0 conn(0x7efdb4134df0 legacy=0x7efdb4135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:54.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.982+0000 7efdb37fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:54.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.982+0000 7efdb37fe640 1 monclient: found mon.noname-a 2026-02-17T06:06:54.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.982+0000 7efdb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:54.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.982+0000 7efdb37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:54.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.982+0000 7efdb37fe640 1 -- 10.20.193.78:0/2339842094 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb409cc80 con 0x7efdb409c4a0 2026-02-17T06:06:54.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb37fe640 1 -- 10.20.193.78:0/2339842094 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 msgr2=0x7efdb406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:54.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb37fe640 1 --2- 10.20.193.78:0/2339842094 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb406d1b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efdb406a480 tx=0x7efda402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ace77cb66fe2da9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:54.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 1 -- 10.20.193.78:0/2339842094 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda403c070 con 0x7efdb409c4a0 2026-02-17T06:06:54.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:54.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: dump: 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:54.998 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 1 -- 10.20.193.78:0/2339842094 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda402fca0 con 0x7efdb409c4a0 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 1 -- 10.20.193.78:0/2339842094 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4037660 con 0x7efdb409c4a0 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 5 monclient: authenticate success, global_id 254121 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 4 set_mon_vals no callback set 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:54.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdb2ffd640 10 monclient: dump: 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.000 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 10 monclient: shutdown 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 1 -- 10.20.193.78:0/2339842094 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 msgr2=0x7efdb406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 1 --2- 10.20.193.78:0/2339842094 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb406d1b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efdb406a480 tx=0x7efda402ee60 comp rx=0 tx=0).stop 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 1 -- 10.20.193.78:0/2339842094 shutdown_connections 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 1 --2- 10.20.193.78:0/2339842094 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb406d1b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 1 --2- 10.20.193.78:0/2339842094 >> v2:10.20.193.115:3300/0 conn(0x7efdb406c890 0x7efdb406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 1 -- 10.20.193.78:0/2339842094 >> 10.20.193.78:0/2339842094 conn(0x7efdb40a6620 msgr2=0x7efdb40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 1 -- 10.20.193.78:0/2339842094 shutdown_connections 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.984+0000 7efdba722640 1 -- 10.20.193.78:0/2339842094 wait complete. 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.001 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient: monmap: 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 1 Processor -- start 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 1 -- start start 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient: init 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdb406c890 0x7efdb41cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient(hunting): picked mon.noname-b con 0x7efdb406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb4133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient(hunting): picked mon.noname-a con 0x7efdb409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdb4134df0 0x7efdb41344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient(hunting): picked mon.noname-c con 0x7efdb4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:55.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40a01a0 con 0x7efdb409c4a0 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40a0020 con 0x7efdb406c890 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7efdb409be20 con 0x7efdb4134df0 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdba722640 10 monclient(hunting): authenticate will time out at 1860.539640s 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb4133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb4133f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39192/0 (socket says 10.20.193.78:39192) 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 -- 10.20.193.78:0/1693804740 learned_addr learned my addr 10.20.193.78:0/1693804740 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:55.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient(hunting): get_auth_request con 0x7efdb409c4a0 auth_method 0 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb8c98640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdb4134df0 0x7efdb41344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb3fff640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdb406c890 0x7efdb41cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb8c98640 10 monclient(hunting): get_auth_request con 0x7efdb4134df0 auth_method 0 2026-02-17T06:06:55.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb8c98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb8c98640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb8c98640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb3fff640 10 monclient(hunting): get_auth_request con 0x7efdb406c890 auth_method 0 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient(hunting): handle_auth_done global_id 254124 payload 824 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 -- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdb4134df0 msgr2=0x7efdb41344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdb4134df0 0x7efdb41344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 -- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdb406c890 msgr2=0x7efdb41cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdb406c890 0x7efdb41cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 monclient: found mon.noname-a 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.985+0000 7efdb37fe640 1 -- 10.20.193.78:0/1693804740 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb40a1780 con 0x7efdb409c4a0 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.987+0000 7efdb37fe640 1 -- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 msgr2=0x7efdb4133f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.987+0000 7efdb37fe640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb4133f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efda4002410 tx=0x7efda4004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 1 -- 10.20.193.78:0/1693804740 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda403c040 con 0x7efdb409c4a0 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:55.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: dump: 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 1 -- 10.20.193.78:0/1693804740 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4047070 con 0x7efdb409c4a0 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 1 -- 10.20.193.78:0/1693804740 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4042890 con 0x7efdb409c4a0 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 10 monclient: dump: 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.009 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb27fc640 4 set_mon_vals no callback set 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 5 monclient: authenticate success, global_id 254124 2026-02-17T06:06:55.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 10 monclient: _renew_subs 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb409d810 con 0x7efdb409c4a0 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 10 monclient: _renew_subs 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb409f7a0 con 0x7efdb409c4a0 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb4074980 con 0x7efdb409c4a0 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.988+0000 7efdb0ff9640 1 -- 10.20.193.78:0/1693804740 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7efda4042a70 con 0x7efdb409c4a0 2026-02-17T06:06:55.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.989+0000 7efdb0ff9640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd8403df00 0x7efd840403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.989+0000 7efdb0ff9640 1 -- 10.20.193.78:0/1693804740 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(726..726 src has 525..726) ==== 4795+0+0 (secure 0 0 0) 0x7efda4077bf0 con 0x7efdb409c4a0 2026-02-17T06:06:55.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.989+0000 7efdb3fff640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd8403df00 0x7efd840403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.989+0000 7efdb3fff640 10 monclient: get_auth_request con 0x7efd8403df00 auth_method 0 2026-02-17T06:06:55.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.991+0000 7efdb0ff9640 1 -- 10.20.193.78:0/1693804740 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efda40363a0 con 0x7efdb409c4a0 2026-02-17T06:06:55.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.991+0000 7efdb0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.991+0000 7efdb0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:55.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:54.991+0000 7efdb3fff640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd8403df00 0x7efd840403c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efda000c980 tx=0x7efda0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.106+0000 7efdba722640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.106+0000 7efdba722640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.106+0000 7efdba722640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.106+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efdb4135a20 con 0x7efdb409c4a0 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.106+0000 7efdb0ff9640 1 -- 10.20.193.78:0/1693804740 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7efda403f380 con 0x7efdb409c4a0 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.106+0000 7efdb0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.106+0000 7efdb0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":242,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd8403df00 msgr2=0x7efd840403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd8403df00 0x7efd840403c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efda000c980 tx=0x7efda0008040 comp rx=0 tx=0).stop 2026-02-17T06:06:55.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 10 monclient: shutdown 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 msgr2=0x7efdb4133f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb4133f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efda4002410 tx=0x7efda4004830 comp rx=0 tx=0).stop 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 shutdown_connections 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7efdb4134df0 0x7efdb41344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7efdb409c4a0 0x7efdb4133f70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7efd8403df00 0x7efd840403c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 --2- 10.20.193.78:0/1693804740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7efdb406c890 0x7efdb41cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 >> 10.20.193.78:0/1693804740 conn(0x7efdb40a6620 msgr2=0x7efdb40a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 shutdown_connections 2026-02-17T06:06:55.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.107+0000 7efdba722640 1 -- 10.20.193.78:0/1693804740 wait complete. 2026-02-17T06:06:55.114 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:06:55.115 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:06:55.115 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:06:55.115 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:06:55.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.180+0000 7f78615d3640 10 monclient: get_monmap_and_config 2026-02-17T06:06:55.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.180+0000 7f78615d3640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.180+0000 7f78615d3640 10 monclient: monmap: 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:55.181625+0000 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:55.181625+0000 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:55.182 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:55.183 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:55.183 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.180+0000 7f78615d3640 1 Processor -- start 2026-02-17T06:06:55.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.180+0000 7f78615d3640 1 -- start start 2026-02-17T06:06:55.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.180+0000 7f78615d3640 10 monclient: init 2026-02-17T06:06:55.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2026-02-17T06:06:55.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f785c09ea90 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:55.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f785c09ee70 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:55.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f785c13e650 0x7f785c140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 10 monclient(hunting): picked mon.noname-a con 0x7f785c13e650 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f785c0a0a20 con 0x7f785c09ee70 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f785c0a0460 con 0x7f785c09ea90 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f785c0a0d00 con 0x7f785c13e650 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f78615d3640 10 monclient(hunting): authenticate will time out at 1860.735291s 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f785c13e650 0x7f785c140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f785c13e650 0x7f785c140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39204/0 (socket says 10.20.193.78:39204) 2026-02-17T06:06:55.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 1 -- 10.20.193.78:0/474434593 learned_addr learned my addr 10.20.193.78:0/474434593 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:55.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 10 monclient(hunting): get_auth_request con 0x7f785c13e650 auth_method 0 2026-02-17T06:06:55.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.181+0000 7f785b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785a7fc640 1 -- 10.20.193.78:0/474434593 >> v1:10.20.193.78:6789/0 conn(0x7f785c09ee70 legacy=0x7f785c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785b7fe640 10 monclient(hunting): handle_auth_done global_id 254127 payload 274 2026-02-17T06:06:55.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785b7fe640 1 -- 10.20.193.78:0/474434593 >> v1:10.20.193.115:6789/0 conn(0x7f785c09ea90 legacy=0x7f785c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:55.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785b7fe640 1 -- 10.20.193.78:0/474434593 >> v1:10.20.193.78:6789/0 conn(0x7f785c09ee70 legacy=0x7f785c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785b7fe640 1 monclient: found mon.noname-a 2026-02-17T06:06:55.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.183+0000 7f785b7fe640 1 -- 10.20.193.78:0/474434593 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c0a0740 con 0x7f785c13e650 2026-02-17T06:06:55.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f785b7fe640 1 -- 10.20.193.78:0/474434593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13e650 msgr2=0x7f785c140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f785b7fe640 1 --2- 10.20.193.78:0/474434593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13e650 0x7f785c140a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f784c009ed0 tx=0x7f784c02f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca30dd7173960f3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 1 -- 10.20.193.78:0/474434593 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c0373f0 con 0x7f785c13e650 2026-02-17T06:06:55.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:55.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: dump: 2026-02-17T06:06:55.201 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 5 monclient: authenticate success, global_id 254127 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 1 -- 10.20.193.78:0/474434593 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784c037590 con 0x7f785c13e650 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 1 -- 10.20.193.78:0/474434593 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c037960 con 0x7f785c13e650 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 4 set_mon_vals no callback set 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f7859ffb640 10 monclient: dump: 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.204 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 10 monclient: shutdown 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 1 -- 10.20.193.78:0/474434593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13e650 msgr2=0x7f785c140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 1 --2- 10.20.193.78:0/474434593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13e650 0x7f785c140a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f784c009ed0 tx=0x7f784c02f2d0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 1 -- 10.20.193.78:0/474434593 shutdown_connections 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 1 --2- 10.20.193.78:0/474434593 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13e650 0x7f785c140a40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 1 -- 10.20.193.78:0/474434593 >> 10.20.193.78:0/474434593 conn(0x7f785c0a7520 msgr2=0x7f785c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 1 -- 10.20.193.78:0/474434593 shutdown_connections 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.186+0000 7f78615d3640 1 -- 10.20.193.78:0/474434593 wait complete. 2026-02-17T06:06:55.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient: monmap: 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:55.206 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 1 Processor -- start 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 1 -- start start 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient: init 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c09ea90 0x7f785c09dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient(hunting): picked mon.noname-a con 0x7f785c09ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c09ee70 0x7f785c09e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient(hunting): picked mon.noname-c con 0x7f785c09ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:55.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c13e650 0x7f785c13b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient(hunting): picked mon.noname-b con 0x7f785c13e650 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f785c141100 con 0x7f785c09ea90 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f785c140f80 con 0x7f785c13e650 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f785c141280 con 0x7f785c09ee70 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f78615d3640 10 monclient(hunting): authenticate will time out at 1860.741643s 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c09ea90 0x7f785c09dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c09ea90 0x7f785c09dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39218/0 (socket says 10.20.193.78:39218) 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785a7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c09ee70 0x7f785c09e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 -- 10.20.193.78:0/1463220776 learned_addr learned my addr 10.20.193.78:0/1463220776 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient(hunting): get_auth_request con 0x7f785c09ea90 auth_method 0 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785a7fc640 10 monclient(hunting): get_auth_request con 0x7f785c09ee70 auth_method 0 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785b7fe640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c13e650 0x7f785c13b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785b7fe640 10 monclient(hunting): get_auth_request con 0x7f785c13e650 auth_method 0 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient(hunting): handle_auth_done global_id 254130 payload 824 2026-02-17T06:06:55.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c09ee70 msgr2=0x7f785c09e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f785c09ee70 0x7f785c09e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c13e650 msgr2=0x7f785c13b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f785c13e650 0x7f785c13b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 monclient: found mon.noname-a 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.187+0000 7f785affd640 1 -- 10.20.193.78:0/1463220776 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c06fb00 con 0x7f785c09ea90 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.189+0000 7f785affd640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c09ea90 msgr2=0x7f785c09dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.189+0000 7f785affd640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c09ea90 0x7f785c09dc80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f785000b550 tx=0x7f785000ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850013020 con 0x7f785c09ea90 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: dump: 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.212 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78500026e0 con 0x7f785c09ea90 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850002ad0 con 0x7f785c09ea90 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78597fa640 4 set_mon_vals no callback set 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 10 monclient: dump: 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 5 monclient: authenticate success, global_id 254130 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 10 monclient: _renew_subs 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785c06f820 con 0x7f785c09ea90 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 10 monclient: _renew_subs 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785c13d570 con 0x7f785c09ea90 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13ba10 0x7f785c13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f785c0a5eb0 con 0x7f785c13ba10 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13ba10 0x7f785c13bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 10 monclient: get_auth_request con 0x7f785c13ba10 auth_method 0 2026-02-17T06:06:55.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7850018400 con 0x7f785c09ea90 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 10 monclient(hunting): handle_auth_done global_id 254133 payload 824 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f785a7fc640 1 -- 10.20.193.78:0/1463220776 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f785c0a98c0 con 0x7f785c13ba10 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803deb0 0x7f7828040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.190+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(726..726 src has 525..726) ==== 4795+0+0 (secure 0 0 0) 0x7f7850052700 con 0x7f785c09ea90 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.191+0000 7f785b7fe640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803deb0 0x7f7828040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.191+0000 7f785b7fe640 10 monclient: get_auth_request con 0x7f782803deb0 auth_method 0 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.192+0000 7f785a7fc640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13ba10 msgr2=0x7f785c13bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.192+0000 7f785a7fc640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13ba10 0x7f785c13bdf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f785c13ce20 tx=0x7f784400fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844005ad0 con 0x7f785c13ba10 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 10 monclient: dump: 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.217 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7844018070 con 0x7f785c13ba10 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13ba10 msgr2=0x7f785c13bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13ba10 0x7f785c13bdf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f785c13ce20 tx=0x7f784400fc10 comp rx=0 tx=0).stop 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13ba10 msgr2=0x7f785c13bdf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13ba10 0x7f785c13bdf0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f785c13ce20 tx=0x7f784400fc10 comp rx=0 tx=0).stop 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.193+0000 7f785b7fe640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803deb0 0x7f7828040370 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f785c06e270 tx=0x7f784c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f78615d3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f78615d3640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13c330 0x7f785c1da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f78615d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f785c06f540 con 0x7f785c13c330 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13c330 0x7f785c1da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 10 monclient: get_auth_request con 0x7f785c13c330 auth_method 0 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 10 monclient(hunting): handle_auth_done global_id 254136 payload 824 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.196+0000 7f785a7fc640 1 -- 10.20.193.78:0/1463220776 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f785c0a13b0 con 0x7f785c13c330 2026-02-17T06:06:55.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f785a7fc640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13c330 msgr2=0x7f785c1da5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f785a7fc640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13c330 0x7f785c1da5b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7844017000 tx=0x7f78440023d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440070d0 con 0x7f785c13c330 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 10 monclient: dump: 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.220 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f78440070d0 con 0x7f785c13c330 2026-02-17T06:06:55.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:55.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:55.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13c330 msgr2=0x7f785c1da5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13c330 0x7f785c1da5b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7844017000 tx=0x7f78440023d0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13c330 msgr2=0x7f785c1da5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:55.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f783b7fe640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c13c330 0x7f785c1da5b0 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7844017000 tx=0x7f78440023d0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.221 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 242, 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 14572, 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:55.222 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:55.223 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:55.224 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:55.225 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.226 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.227 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.228 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.229 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.230 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:55.231 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:55.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803deb0 msgr2=0x7f7828040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f78615d3640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f782803deb0 0x7f7828040370 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f785c06e270 tx=0x7f784c033000 comp rx=0 tx=0).stop 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f78615d3640 10 monclient: shutdown 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f78615d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c09ea90 msgr2=0x7f785c09dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f78615d3640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c09ea90 0x7f785c09dc80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f785000b550 tx=0x7f785000ba20 comp rx=0 tx=0).stop 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f785b7fe640 1 -- 10.20.193.78:0/1463220776 reap_dead start 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 shutdown_connections 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.198+0000 7f78615d3640 1 --2- 10.20.193.78:0/1463220776 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f785c09ea90 0x7f785c09dc80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.199+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 >> 10.20.193.78:0/1463220776 conn(0x7f785c0a7520 msgr2=0x7f785c0a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.199+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 shutdown_connections 2026-02-17T06:06:55.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.199+0000 7f78615d3640 1 -- 10.20.193.78:0/1463220776 wait complete. 2026-02-17T06:06:55.233 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:55.234 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:06:55.234 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient: get_monmap_and_config 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient: monmap: 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:55.301406+0000 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:55.301406+0000 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:55.301 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:55.302 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.302 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 1 Processor -- start 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 1 -- start start 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient: init 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f687c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f687c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f687c06d380 0x7f687c13ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient(hunting): picked mon.noname-a con 0x7f687c06d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c06daf0 con 0x7f687c06c890 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c09cc80 con 0x7f687c09c4a0 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f687c0a0880 con 0x7f687c06d380 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6882dee640 10 monclient(hunting): authenticate will time out at 1860.854923s 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.300+0000 7f6880b63640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f687c06c890 0x7f687c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56488/0 (socket says 10.20.193.78:56488) 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.301+0000 7f6880b63640 1 -- 10.20.193.78:0/203610098 learned_addr learned my addr 10.20.193.78:0/203610098 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.301+0000 7f6881364640 1 --2- 10.20.193.78:0/203610098 >> v2:10.20.193.78:3300/0 conn(0x7f687c06d380 0x7f687c13ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.301+0000 7f6881364640 10 monclient(hunting): get_auth_request con 0x7f687c06d380 auth_method 0 2026-02-17T06:06:55.303 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.301+0000 7f6881364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.301+0000 7f6881364640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.301+0000 7f6881364640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6880b63640 1 -- 10.20.193.78:0/203610098 >> v1:10.20.193.78:6789/0 conn(0x7f687c06c890 legacy=0x7f687c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 10 monclient(hunting): handle_auth_done global_id 254139 payload 274 2026-02-17T06:06:55.304 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 1 -- 10.20.193.78:0/203610098 >> v1:10.20.193.115:6789/0 conn(0x7f687c09c4a0 legacy=0x7f687c13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:55.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 1 -- 10.20.193.78:0/203610098 >> v1:10.20.193.78:6789/0 conn(0x7f687c06c890 legacy=0x7f687c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 1 monclient: found mon.noname-a 2026-02-17T06:06:55.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.305 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.303+0000 7f6881364640 1 -- 10.20.193.78:0/203610098 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c09cf60 con 0x7f687c06d380 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f6881364640 1 -- 10.20.193.78:0/203610098 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06d380 msgr2=0x7f687c13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f6881364640 1 --2- 10.20.193.78:0/203610098 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06d380 0x7f687c13ead0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f687c074de0 tx=0x7f687802f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a449a95f2ef9482b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 1 -- 10.20.193.78:0/203610098 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687803c070 con 0x7f687c06d380 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: dump: 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.307 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f6882dee640 5 monclient: authenticate success, global_id 254139 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 1 -- 10.20.193.78:0/203610098 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6878037640 con 0x7f687c06d380 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f6882dee640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 1 -- 10.20.193.78:0/203610098 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878037a10 con 0x7f687c06d380 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f6882dee640 4 set_mon_vals no callback set 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.305+0000 7f68737fe640 10 monclient: dump: 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.308 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 10 monclient: shutdown 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 -- 10.20.193.78:0/203610098 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06d380 msgr2=0x7f687c13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 --2- 10.20.193.78:0/203610098 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06d380 0x7f687c13ead0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f687c074de0 tx=0x7f687802f1e0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 -- 10.20.193.78:0/203610098 shutdown_connections 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 --2- 10.20.193.78:0/203610098 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c06d380 0x7f687c13ead0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 -- 10.20.193.78:0/203610098 >> 10.20.193.78:0/203610098 conn(0x7f687c0a6620 msgr2=0x7f687c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 -- 10.20.193.78:0/203610098 shutdown_connections 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 -- 10.20.193.78:0/203610098 wait complete. 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 10 monclient: monmap: 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.309 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:55.310 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:55.310 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:55.310 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 Processor -- start 2026-02-17T06:06:55.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 1 -- start start 2026-02-17T06:06:55.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.306+0000 7f6882dee640 10 monclient: init 2026-02-17T06:06:55.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:06:55.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c06c890 0x7f687c13d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient(hunting): picked mon.noname-b con 0x7f687c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c06d380 0x7f687c132120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient(hunting): picked mon.noname-c con 0x7f687c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c09c4a0 0x7f687c1cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient(hunting): picked mon.noname-a con 0x7f687c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f687c09bfa0 con 0x7f687c09c4a0 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f687c09be20 con 0x7f687c06c890 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f687c13f010 con 0x7f687c06d380 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6882dee640 10 monclient(hunting): authenticate will time out at 1860.861227s 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c09c4a0 0x7f687c1cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c09c4a0 0x7f687c1cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39262/0 (socket says 10.20.193.78:39262) 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 -- 10.20.193.78:0/264092777 learned_addr learned my addr 10.20.193.78:0/264092777 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient(hunting): get_auth_request con 0x7f687c09c4a0 auth_method 0 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6873fff640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c06d380 0x7f687c132120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6873fff640 10 monclient(hunting): get_auth_request con 0x7f687c06d380 auth_method 0 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6873fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6873fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6873fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6873fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6873fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6873fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient(hunting): handle_auth_done global_id 254145 payload 824 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c06d380 msgr2=0x7f687c132120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c06d380 0x7f687c132120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c06c890 msgr2=0x7f687c13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f687c06c890 0x7f687c13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 monclient: found mon.noname-a 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.307+0000 7f6881364640 1 -- 10.20.193.78:0/264092777 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c11d7e0 con 0x7f687c09c4a0 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6881364640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c09c4a0 msgr2=0x7f687c1cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6881364640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c09c4a0 0x7f687c1cf180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6878009f10 tx=0x7f6878004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878047020 con 0x7f687c09c4a0 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: dump: 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6878042680 con 0x7f687c09c4a0 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6882dee640 5 monclient: authenticate success, global_id 254145 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6882dee640 10 monclient: _renew_subs 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6882dee640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c13e460 con 0x7f687c09c4a0 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687803c040 con 0x7f687c09c4a0 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6872ffd640 4 set_mon_vals no callback set 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f68717fa640 10 monclient: dump: 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6882dee640 10 monclient: _renew_subs 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6882dee640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.309+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c09d810 con 0x7f687c09c4a0 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6878054060 con 0x7f687c09c4a0 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6882dee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6882dee640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c132660 0x7f687c133ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6882dee640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f687c06cc70 con 0x7f687c132660 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f68717fa640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684c03df00 0x7f684c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(726..726 src has 525..726) ==== 4795+0+0 (secure 0 0 0) 0x7f68780783a0 con 0x7f687c09c4a0 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c132660 0x7f687c133ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 10 monclient: get_auth_request con 0x7f687c132660 auth_method 0 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.316 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 10 monclient(hunting): handle_auth_done global_id 234326 payload 824 2026-02-17T06:06:55.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6880b63640 1 -- 10.20.193.78:0/264092777 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f687c09f3f0 con 0x7f687c132660 2026-02-17T06:06:55.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6873fff640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684c03df00 0x7f684c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.317 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.310+0000 7f6873fff640 10 monclient: get_auth_request con 0x7f684c03df00 auth_method 0 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f6880b63640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c132660 msgr2=0x7f687c133ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f6880b63640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c132660 0x7f687c133ff0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f686c00c3b0 tx=0x7f686c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c017070 con 0x7f687c132660 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 10 monclient: dump: 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f686c0072d0 con 0x7f687c132660 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.318 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c132660 msgr2=0x7f687c133ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c132660 0x7f687c133ff0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f686c00c3b0 tx=0x7f686c00c880 comp rx=0 tx=0).stop 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c132660 msgr2=0x7f687c133ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.312+0000 7f68717fa640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c132660 0x7f687c133ff0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f686c00c3b0 tx=0x7f686c00c880 comp rx=0 tx=0).stop 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.313+0000 7f6873fff640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684c03df00 0x7f684c0403c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f687c13ddc0 tx=0x7f6864007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6882dee640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6882dee640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c134530 0x7f687c1da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6882dee640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f687c0744d0 con 0x7f687c134530 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c134530 0x7f687c1da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 10 monclient: get_auth_request con 0x7f687c134530 auth_method 0 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 10 monclient(hunting): handle_auth_done global_id 234329 payload 824 2026-02-17T06:06:55.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.316+0000 7f6880b63640 1 -- 10.20.193.78:0/264092777 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f687c1cfe80 con 0x7f687c134530 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.318+0000 7f6880b63640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c134530 msgr2=0x7f687c1da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.318+0000 7f6880b63640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c134530 0x7f687c1da4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f686c011000 tx=0x7f686c0026e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.318+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c017030 con 0x7f687c134530 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.318+0000 7f68717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.318+0000 7f68717fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.318+0000 7f68717fa640 10 monclient: dump: 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.320 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.318+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f686c019b60 con 0x7f687c134530 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f68717fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f68717fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c134530 msgr2=0x7f687c1da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f68717fa640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c134530 0x7f687c1da4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f686c011000 tx=0x7f686c0026e0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f68717fa640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c134530 msgr2=0x7f687c1da4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f68717fa640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f687c134530 0x7f687c1da4d0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f686c011000 tx=0x7f686c0026e0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 242, 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 124001, 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:55.321 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:55.322 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.323 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.324 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.325 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.326 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684c03df00 msgr2=0x7f684c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f684c03df00 0x7f684c0403c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f687c13ddc0 tx=0x7f6864007f30 comp rx=0 tx=0).stop 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 10 monclient: shutdown 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c09c4a0 msgr2=0x7f687c1cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c09c4a0 0x7f687c1cf180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6878009f10 tx=0x7f6878004b40 comp rx=0 tx=0).stop 2026-02-17T06:06:55.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6881364640 1 -- 10.20.193.78:0/264092777 reap_dead start 2026-02-17T06:06:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 shutdown_connections 2026-02-17T06:06:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 --2- 10.20.193.78:0/264092777 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f687c09c4a0 0x7f687c1cf180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 >> 10.20.193.78:0/264092777 conn(0x7f687c0a6620 msgr2=0x7f687c0a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 shutdown_connections 2026-02-17T06:06:55.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.319+0000 7f6882dee640 1 -- 10.20.193.78:0/264092777 wait complete. 2026-02-17T06:06:55.328 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:55.328 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:06:55.328 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:06:55.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.394+0000 7f2be8c32640 10 monclient: get_monmap_and_config 2026-02-17T06:06:55.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.394+0000 7f2be8c32640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.394+0000 7f2be8c32640 10 monclient: monmap: 2026-02-17T06:06:55.395 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.395 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.395 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:55.396078+0000 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:55.396078+0000 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:55.396 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 1 Processor -- start 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 1 -- start start 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient: init 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2be406c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f2be409c4a0 0x7f2be413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient(hunting): picked mon.noname-c con 0x7f2be409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2be406d380 addr v1:10.20.193.115:6789/0 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be406daf0 con 0x7f2be406c890 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be409cc80 con 0x7f2be406d380 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f2be40a0880 con 0x7f2be409c4a0 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be8c32640 10 monclient(hunting): authenticate will time out at 1860.949637s 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be2575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f2be406c890 0x7f2be406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56492/0 (socket says 10.20.193.78:56492) 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be2575640 1 -- 10.20.193.78:0/320806759 learned_addr learned my addr 10.20.193.78:0/320806759 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be1d74640 1 --2- 10.20.193.78:0/320806759 >> v2:10.20.193.115:3300/0 conn(0x7f2be409c4a0 0x7f2be413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be1d74640 10 monclient(hunting): get_auth_request con 0x7f2be409c4a0 auth_method 0 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.395+0000 7f2be1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.396+0000 7f2be1d74640 10 monclient(hunting): handle_auth_done global_id 214399 payload 274 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.396+0000 7f2be1d74640 1 -- 10.20.193.78:0/320806759 >> v1:10.20.193.115:6789/0 conn(0x7f2be406d380 legacy=0x7f2be413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.396+0000 7f2be1d74640 1 -- 10.20.193.78:0/320806759 >> v1:10.20.193.78:6789/0 conn(0x7f2be406c890 legacy=0x7f2be406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.396+0000 7f2be1d74640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.396+0000 7f2be1d74640 1 monclient: found mon.noname-c 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.396+0000 7f2be1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.396+0000 7f2be1d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:06:55.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.396+0000 7f2be1d74640 1 -- 10.20.193.78:0/320806759 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be409cf60 con 0x7f2be409c4a0 2026-02-17T06:06:55.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1d74640 1 -- 10.20.193.78:0/320806759 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be409c4a0 msgr2=0x7f2be413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1d74640 1 --2- 10.20.193.78:0/320806759 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be409c4a0 0x7f2be413b3a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2be409c980 tx=0x7f2bcc02ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935c1dd9bbe991b7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1573640 1 -- 10.20.193.78:0/320806759 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc03c070 con 0x7f2be409c4a0 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1573640 10 monclient: dump: 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1573640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1573640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be8c32640 5 monclient: authenticate success, global_id 214399 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.398+0000 7f2be8c32640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be1573640 1 -- 10.20.193.78:0/320806759 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bcc02fcb0 con 0x7f2be409c4a0 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be1573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be1573640 1 -- 10.20.193.78:0/320806759 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc02fe60 con 0x7f2be409c4a0 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be1573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be1573640 10 monclient: dump: 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 4 set_mon_vals no callback set 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 10 monclient: shutdown 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 -- 10.20.193.78:0/320806759 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be409c4a0 msgr2=0x7f2be413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 --2- 10.20.193.78:0/320806759 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be409c4a0 0x7f2be413b3a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2be409c980 tx=0x7f2bcc02ed50 comp rx=0 tx=0).stop 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 -- 10.20.193.78:0/320806759 shutdown_connections 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 --2- 10.20.193.78:0/320806759 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be409c4a0 0x7f2be413b3a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 -- 10.20.193.78:0/320806759 >> 10.20.193.78:0/320806759 conn(0x7f2be40a6620 msgr2=0x7f2be40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 -- 10.20.193.78:0/320806759 shutdown_connections 2026-02-17T06:06:55.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 -- 10.20.193.78:0/320806759 wait complete. 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 10 monclient: monmap: 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 Processor -- start 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 1 -- start start 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.399+0000 7f2be8c32640 10 monclient: init 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:06:55.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2be406c890 0x7f2be413d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient(hunting): picked mon.noname-c con 0x7f2be406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2be406d380 0x7f2be41cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient(hunting): picked mon.noname-a con 0x7f2be406d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be409c4a0 0x7f2be41cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient(hunting): picked mon.noname-b con 0x7f2be409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2be409bfa0 con 0x7f2be406d380 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2be409be20 con 0x7f2be409c4a0 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f2be413f010 con 0x7f2be406c890 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be8c32640 10 monclient(hunting): authenticate will time out at 1860.954327s 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2be406c890 0x7f2be413d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2be406d380 0x7f2be41cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2be406c890 0x7f2be413d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45042/0 (socket says 10.20.193.78:45042) 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 1 -- 10.20.193.78:0/1494845066 learned_addr learned my addr 10.20.193.78:0/1494845066 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 10 monclient(hunting): get_auth_request con 0x7f2be406c890 auth_method 0 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient(hunting): get_auth_request con 0x7f2be406d380 auth_method 0 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient(hunting): handle_auth_done global_id 254148 payload 824 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2be406c890 msgr2=0x7f2be413d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f2be406c890 0x7f2be413d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be409c4a0 msgr2=0x7f2be41cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be409c4a0 0x7f2be41cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 1 monclient: found mon.noname-a 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.405 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.400+0000 7f2be1d74640 1 -- 10.20.193.78:0/1494845066 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be4121990 con 0x7f2be406d380 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2be1d74640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2be406d380 msgr2=0x7f2be41cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2be1d74640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2be406d380 0x7f2be41cf2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc036000 tx=0x7f2bcc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc046070 con 0x7f2be406d380 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 10 monclient: dump: 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bcc037e50 con 0x7f2be406d380 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2be8c32640 5 monclient: authenticate success, global_id 254148 2026-02-17T06:06:55.407 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2be8c32640 10 monclient: _renew_subs 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2be8c32640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be4121c70 con 0x7f2be406d380 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.402+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc03c040 con 0x7f2be406d380 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2bcaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be0d72640 4 set_mon_vals no callback set 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2bcaffd640 10 monclient: dump: 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be8c32640 10 monclient: _renew_subs 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be8c32640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be409f100 con 0x7f2be406d380 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be8c32640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be8c32640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be41cfd30 0x7f2be41363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be8c32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41d0110 con 0x7f2be41cfd30 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be41cfd30 0x7f2be41363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 10 monclient: get_auth_request con 0x7f2be41cfd30 auth_method 0 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f2bcc052060 con 0x7f2be406d380 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 10 monclient(hunting): handle_auth_done global_id 214402 payload 824 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2be2575640 1 -- 10.20.193.78:0/1494845066 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2be4121210 con 0x7f2be41cfd30 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.403+0000 7f2bcaffd640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bb003deb0 0x7f2bb0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.404+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(726..726 src has 525..726) ==== 4795+0+0 (secure 0 0 0) 0x7f2bcc077430 con 0x7f2be406d380 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.404+0000 7f2be2d76640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bb003deb0 0x7f2bb0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.404+0000 7f2be2d76640 10 monclient: get_auth_request con 0x7f2bb003deb0 auth_method 0 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2be2575640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be41cfd30 msgr2=0x7f2be41363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2be2575640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be41cfd30 0x7f2be41363d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800ea70 tx=0x7f2bd800ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801a070 con 0x7f2be41cfd30 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 10 monclient: dump: 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.409 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.410 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f2bd800cba0 con 0x7f2be41cfd30 2026-02-17T06:06:55.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be41cfd30 msgr2=0x7f2be41363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be41cfd30 0x7f2be41363d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800ea70 tx=0x7f2bd800ef40 comp rx=0 tx=0).stop 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be41cfd30 msgr2=0x7f2be41363d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.406+0000 7f2bcaffd640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be41cfd30 0x7f2be41363d0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800ea70 tx=0x7f2bd800ef40 comp rx=0 tx=0).stop 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.407+0000 7f2be2d76640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bb003deb0 0x7f2bb0040370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2be4070440 tx=0x7f2bd4003fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be8c32640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be8c32640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be4138390 0x7f2be41da810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be8c32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2be406cc70 con 0x7f2be4138390 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be2575640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be4138390 0x7f2be41da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be2575640 10 monclient: get_auth_request con 0x7f2be4138390 auth_method 0 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.409+0000 7f2be2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.410+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.410+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.410+0000 7f2be2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.410+0000 7f2be2575640 10 monclient(hunting): handle_auth_done global_id 214405 payload 824 2026-02-17T06:06:55.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.410+0000 7f2be2575640 1 -- 10.20.193.78:0/1494845066 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2be40744d0 con 0x7f2be4138390 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2be2575640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be4138390 msgr2=0x7f2be41da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2be2575640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be4138390 0x7f2be41da810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8017000 tx=0x7f2bd8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801a070 con 0x7f2be4138390 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 10 monclient: dump: 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.413 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f2bd801b040 con 0x7f2be4138390 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be4138390 msgr2=0x7f2be41da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be4138390 0x7f2be41da810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8017000 tx=0x7f2bd8004a40 comp rx=0 tx=0).stop 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be4138390 msgr2=0x7f2be41da810 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.412+0000 7f2bcaffd640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f2be4138390 0x7f2be41da810 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8017000 tx=0x7f2bd8004a40 comp rx=0 tx=0).stop 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 242, 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 218260, 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:06:55.414 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.415 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:06:55.416 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.417 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:06:55.418 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:06:55.419 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:06:55.420 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:06:55.421 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:06:55.421 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:06:55.421 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:06:55.421 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:06:55.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bb003deb0 msgr2=0x7f2bb0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f2bb003deb0 0x7f2bb0040370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2be4070440 tx=0x7f2bd4003fc0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 10 monclient: shutdown 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2be406d380 msgr2=0x7f2be41cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 1 --2- 10.20.193.78:0/1494845066 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f2be406d380 0x7f2be41cf2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc036000 tx=0x7f2bcc0047c0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be2d76640 1 -- 10.20.193.78:0/1494845066 reap_dead start 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be2d76640 1 -- 10.20.193.78:0/1494845066 reap_dead start 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 shutdown_connections 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 >> 10.20.193.78:0/1494845066 conn(0x7f2be40a6620 msgr2=0x7f2be40a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 shutdown_connections 2026-02-17T06:06:55.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.413+0000 7f2be8c32640 1 -- 10.20.193.78:0/1494845066 wait complete. 2026-02-17T06:06:55.422 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:06:55.422 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-17T06:06:55.422 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:06:55.422 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-17T06:06:55.422 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T06:06:55.423 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-17T06:06:55.423 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:06:55.433 INFO:tasks.ceph.mon.c:Stopped 2026-02-17T06:06:55.433 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:06:55.433 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:55.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient: get_monmap_and_config 2026-02-17T06:06:55.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.542 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient: monmap: 2026-02-17T06:06:55.542 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.542 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.542 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:55.540260+0000 2026-02-17T06:06:55.542 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:55.540260+0000 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 1 Processor -- start 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 1 -- start start 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient: init 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 0x7f92000734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient(hunting): picked mon.noname-b con 0x7f92000730c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f92000a2530 0x7f92000a0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient(hunting): picked mon.noname-c con 0x7f92000a2530 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f92000a0870 0x7f92000a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient(hunting): picked mon.noname-a con 0x7f92000a0870 addr v2:10.20.193.78:3300/0 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f9200138fd0 con 0x7f92000a0870 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f92000a2c20 con 0x7f92000a2530 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9200073b50 con 0x7f92000730c0 2026-02-17T06:06:55.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f9205eb7640 10 monclient(hunting): authenticate will time out at 1861.093818s 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91ff7fe640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 msgr2=0x7f92000734a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91ff7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 0x7f92000734a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91fffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f92000a0870 0x7f92000a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91fffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f92000a0870 0x7f92000a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39304/0 (socket says 10.20.193.78:39304) 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91fffff640 1 -- 10.20.193.78:0/3031088164 learned_addr learned my addr 10.20.193.78:0/3031088164 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91fffff640 10 monclient(hunting): get_auth_request con 0x7f92000a0870 auth_method 0 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91fffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.539+0000 7f91fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91feffd640 1 --2- 10.20.193.78:0/3031088164 >> v2:10.20.193.115:3300/0 conn(0x7f92000a2530 0x7f92000a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91feffd640 10 monclient(hunting): get_auth_request con 0x7f92000a2530 auth_method 0 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91feffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91feffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 10 monclient(hunting): handle_auth_done global_id 254151 payload 274 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 1 -- 10.20.193.78:0/3031088164 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 msgr2=0x7f92000734a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 1 --2- 10.20.193.78:0/3031088164 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 0x7f92000734a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 1 -- 10.20.193.78:0/3031088164 >> v2:10.20.193.115:3300/0 conn(0x7f92000a2530 msgr2=0x7f92000a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 1 --2- 10.20.193.78:0/3031088164 >> v2:10.20.193.115:3300/0 conn(0x7f92000a2530 0x7f92000a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 1 monclient: found mon.noname-a 2026-02-17T06:06:55.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.540+0000 7f91fffff640 1 -- 10.20.193.78:0/3031088164 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92000a35d0 con 0x7f92000a0870 2026-02-17T06:06:55.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fffff640 1 -- 10.20.193.78:0/3031088164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a0870 msgr2=0x7f92000a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fffff640 1 --2- 10.20.193.78:0/3031088164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a0870 0x7f92000a0d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f920006d870 tx=0x7f91f402f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7a8e45463d812a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 1 -- 10.20.193.78:0/3031088164 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f40373f0 con 0x7f92000a0870 2026-02-17T06:06:55.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: dump: 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 1 -- 10.20.193.78:0/3031088164 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f4037590 con 0x7f92000a0870 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f9205eb7640 5 monclient: authenticate success, global_id 254151 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 1 -- 10.20.193.78:0/3031088164 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4037960 con 0x7f92000a0870 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f91fe7fc640 10 monclient: dump: 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.554 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f9205eb7640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.542+0000 7f9205eb7640 4 set_mon_vals no callback set 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 10 monclient: shutdown 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 -- 10.20.193.78:0/3031088164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a0870 msgr2=0x7f92000a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 --2- 10.20.193.78:0/3031088164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a0870 0x7f92000a0d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f920006d870 tx=0x7f91f402f200 comp rx=0 tx=0).stop 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 -- 10.20.193.78:0/3031088164 shutdown_connections 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 --2- 10.20.193.78:0/3031088164 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a0870 0x7f92000a0d00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 --2- 10.20.193.78:0/3031088164 >> v2:10.20.193.115:3300/0 conn(0x7f92000a2530 0x7f92000a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 --2- 10.20.193.78:0/3031088164 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 0x7f92000734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 -- 10.20.193.78:0/3031088164 >> 10.20.193.78:0/3031088164 conn(0x7f92000a7370 msgr2=0x7f92000a7780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 -- 10.20.193.78:0/3031088164 shutdown_connections 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 -- 10.20.193.78:0/3031088164 wait complete. 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 10 monclient: build_initial_monmap 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 10 monclient: monmap: 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:55.556 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 Processor -- start 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 1 -- start start 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.543+0000 7f9205eb7640 10 monclient: init 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 0x7f92001cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient(hunting): picked mon.noname-c con 0x7f92000730c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92000a0870 0x7f9200131970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient(hunting): picked mon.noname-b con 0x7f92000a0870 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a2530 0x7f9200131eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient(hunting): picked mon.noname-a con 0x7f92000a2530 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:55.557 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92000a2260 con 0x7f92000a2530 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9200138530 con 0x7f92000a0870 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9200068950 con 0x7f92000730c0 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91ff7fe640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 msgr2=0x7f92001cf010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f9205eb7640 10 monclient(hunting): authenticate will time out at 1861.098277s 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91ff7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 0x7f92001cf010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a2530 0x7f9200131eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a2530 0x7f9200131eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:39314/0 (socket says 10.20.193.78:39314) 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 -- 10.20.193.78:0/1585595467 learned_addr learned my addr 10.20.193.78:0/1585595467 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:55.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient(hunting): get_auth_request con 0x7f92000a2530 auth_method 0 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91feffd640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92000a0870 0x7f9200131970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient(hunting): handle_auth_done global_id 254154 payload 824 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 -- 10.20.193.78:0/1585595467 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 msgr2=0x7f92001cf010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 0x7f92001cf010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 -- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92000a0870 msgr2=0x7f9200131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92000a0870 0x7f9200131970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient: _finish_hunting 0 2026-02-17T06:06:55.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 monclient: found mon.noname-a 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91fffff640 1 -- 10.20.193.78:0/1585595467 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9200123090 con 0x7f92000a2530 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91feffd640 10 monclient: get_auth_request con 0x7f92000a0870 auth_method 0 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.544+0000 7f91feffd640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92000a0870 0x7f9200131970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91fffff640 1 -- 10.20.193.78:0/1585595467 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a2530 msgr2=0x7f9200131eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91fffff640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a2530 0x7f9200131eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f91f4039040 tx=0x7f91f4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 1 -- 10.20.193.78:0/1585595467 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4046020 con 0x7f92000a2530 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: dump: 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.560 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: _finish_auth 0 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: _check_auth_tickets 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 1 -- 10.20.193.78:0/1585595467 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f4041870 con 0x7f92000a2530 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 1 -- 10.20.193.78:0/1585595467 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4041c60 con 0x7f92000a2530 2026-02-17T06:06:55.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f9205eb7640 5 monclient: authenticate success, global_id 254154 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91fdffb640 4 set_mon_vals no callback set 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f9205eb7640 10 monclient: _renew_subs 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f9205eb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92001cf550 con 0x7f92000a2530 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f91dbfff640 10 monclient: dump: 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:55.562 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f9205eb7640 10 monclient: _renew_subs 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f9205eb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.546+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92000a4470 con 0x7f92000a2530 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f91dbfff640 1 -- 10.20.193.78:0/1585595467 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f91f4042400 con 0x7f92000a2530 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f9205eb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f9205eb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f9205eb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92000a0d00 con 0x7f92000a2530 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f91dbfff640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:55.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f91dbfff640 1 -- 10.20.193.78:0/1585595467 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(726..726 src has 525..726) ==== 4795+0+0 (secure 0 0 0) 0x7f91f40787e0 con 0x7f92000a2530 2026-02-17T06:06:55.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f91ff7fe640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:55.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.547+0000 7f91ff7fe640 10 monclient: get_auth_request con 0x7f91d003deb0 auth_method 0 2026-02-17T06:06:55.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.550+0000 7f91ff7fe640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f91ec003a80 tx=0x7f91ec009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:55.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.550+0000 7f91dbfff640 1 -- 10.20.193.78:0/1585595467 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f91f4050030 con 0x7f92000a2530 2026-02-17T06:06:55.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.550+0000 7f91dbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:55.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.550+0000 7f91dbfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:55.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.665+0000 7f9205eb7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:55.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.665+0000 7f9205eb7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:55.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.665+0000 7f9205eb7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:55.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.665+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f92001214d0 con 0x7f92000a2530 2026-02-17T06:06:55.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.666+0000 7f91dbfff640 1 -- 10.20.193.78:0/1585595467 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f91f40771d0 con 0x7f92000a2530 2026-02-17T06:06:55.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.666+0000 7f91dbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:55.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.666+0000 7f91dbfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:55.668 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":242,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:55.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 msgr2=0x7f91d0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f91ec003a80 tx=0x7f91ec009270 comp rx=0 tx=0).stop 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 10 monclient: shutdown 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a2530 msgr2=0x7f9200131eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a2530 0x7f9200131eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f91f4039040 tx=0x7f91f4004b40 comp rx=0 tx=0).stop 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 shutdown_connections 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f92000a2530 0x7f9200131eb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f91d003deb0 0x7f91d0040370 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f92000a0870 0x7f9200131970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 --2- 10.20.193.78:0/1585595467 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f92000730c0 0x7f92001cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:55.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 >> 10.20.193.78:0/1585595467 conn(0x7f92000a7370 msgr2=0x7f92000721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:55.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 shutdown_connections 2026-02-17T06:06:55.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:55.667+0000 7f9205eb7640 1 -- 10.20.193.78:0/1585595467 wait complete. 2026-02-17T06:06:58.677 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:06:58.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.742+0000 7ff138b5e640 10 monclient: get_monmap_and_config 2026-02-17T06:06:58.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.742+0000 7ff138b5e640 10 monclient: build_initial_monmap 2026-02-17T06:06:58.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.742+0000 7ff138b5e640 10 monclient: monmap: 2026-02-17T06:06:58.746 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:06:58.744011+0000 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:06:58.744011+0000 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:06:58.747 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 1 Processor -- start 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 1 -- start start 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient: init 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7ff13406c890 0x7ff13406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient(hunting): picked mon.noname-c con 0x7ff13406c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13409c4a0 0x7ff13406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient(hunting): picked mon.noname-b con 0x7ff13409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff134134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:06:58.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13406daf0 con 0x7ff134134df0 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7ff1340a0880 con 0x7ff13406c890 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff13406d6f0 con 0x7ff13409c4a0 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff138b5e640 10 monclient(hunting): authenticate will time out at 1864.297534s 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff131d74640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13409c4a0 msgr2=0x7ff13406d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff131d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13409c4a0 0x7ff13406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132d76640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff134134df0 0x7ff134135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:35368/0 (socket says 10.20.193.78:35368) 2026-02-17T06:06:58.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132d76640 1 -- 10.20.193.78:0/1815597079 learned_addr learned my addr 10.20.193.78:0/1815597079 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132575640 1 --2- 10.20.193.78:0/1815597079 >> v2:10.20.193.115:3300/0 conn(0x7ff13406c890 0x7ff13406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132575640 10 monclient(hunting): get_auth_request con 0x7ff13406c890 auth_method 0 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.743+0000 7ff132575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 10 monclient(hunting): handle_auth_done global_id 214411 payload 274 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 1 -- 10.20.193.78:0/1815597079 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13409c4a0 msgr2=0x7ff13406d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 1 --2- 10.20.193.78:0/1815597079 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13409c4a0 0x7ff13406d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 1 -- 10.20.193.78:0/1815597079 >> v1:10.20.193.78:6789/0 conn(0x7ff134134df0 legacy=0x7ff134135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:06:58.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 10 monclient: _finish_hunting 0 2026-02-17T06:06:58.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 1 monclient: found mon.noname-c 2026-02-17T06:06:58.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:58.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:06:58.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.744+0000 7ff132575640 1 -- 10.20.193.78:0/1815597079 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13409cc80 con 0x7ff13406c890 2026-02-17T06:06:58.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff132575640 1 -- 10.20.193.78:0/1815597079 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff13406c890 msgr2=0x7ff13406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:58.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff132575640 1 --2- 10.20.193.78:0/1815597079 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff13406c890 0x7ff13406cc70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff13406d870 tx=0x7ff12802ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfab85752ce28643 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:58.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 1 -- 10.20.193.78:0/1815597079 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12803c070 con 0x7ff13406c890 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: dump: 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:58.759 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: _finish_auth 0 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: _check_auth_tickets 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff138b5e640 5 monclient: authenticate success, global_id 214411 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff138b5e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 1 -- 10.20.193.78:0/1815597079 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12802fca0 con 0x7ff13406c890 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 1 -- 10.20.193.78:0/1815597079 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128037660 con 0x7ff13406c890 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:06:58.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff131573640 10 monclient: dump: 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff138b5e640 10 monclient: get_monmap_and_config success 2026-02-17T06:06:58.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.746+0000 7ff138b5e640 4 set_mon_vals no callback set 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 10 monclient: shutdown 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 -- 10.20.193.78:0/1815597079 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff13406c890 msgr2=0x7ff13406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 --2- 10.20.193.78:0/1815597079 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff13406c890 0x7ff13406cc70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff13406d870 tx=0x7ff12802ed40 comp rx=0 tx=0).stop 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 -- 10.20.193.78:0/1815597079 shutdown_connections 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 --2- 10.20.193.78:0/1815597079 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13409c4a0 0x7ff13406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 --2- 10.20.193.78:0/1815597079 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff13406c890 0x7ff13406cc70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 -- 10.20.193.78:0/1815597079 >> 10.20.193.78:0/1815597079 conn(0x7ff1340a6620 msgr2=0x7ff1340a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 -- 10.20.193.78:0/1815597079 shutdown_connections 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 -- 10.20.193.78:0/1815597079 wait complete. 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 10 monclient: build_initial_monmap 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 10 monclient: monmap: 2026-02-17T06:06:58.762 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 Processor -- start 2026-02-17T06:06:58.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 1 -- start start 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.747+0000 7ff138b5e640 10 monclient: init 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient: _reopen_session rank -1 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13406c890 0x7ff1341cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient(hunting): picked mon.noname-c con 0x7ff13406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff13409c4a0 0x7ff1341cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient(hunting): picked mon.noname-a con 0x7ff13409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff134134df0 0x7ff1341215b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient(hunting): picked mon.noname-b con 0x7ff134134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340a01a0 con 0x7ff13409c4a0 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:58.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340a0020 con 0x7ff134134df0 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff13409be20 con 0x7ff13406c890 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient(hunting): _renew_subs 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff138b5e640 10 monclient(hunting): authenticate will time out at 1864.302182s 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132575640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13406c890 msgr2=0x7ff1341cd510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13406c890 0x7ff1341cd510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff13409c4a0 0x7ff1341cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff13409c4a0 0x7ff1341cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44082/0 (socket says 10.20.193.78:44082) 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 -- 10.20.193.78:0/2835804271 learned_addr learned my addr 10.20.193.78:0/2835804271 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient(hunting): get_auth_request con 0x7ff13409c4a0 auth_method 0 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:58.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132d76640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff134134df0 0x7ff1341215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132d76640 10 monclient(hunting): get_auth_request con 0x7ff134134df0 auth_method 0 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff132d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:06:58.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient(hunting): handle_auth_done global_id 254157 payload 824 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 -- 10.20.193.78:0/2835804271 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13406c890 msgr2=0x7ff1341cd510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13406c890 0x7ff1341cd510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 -- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff134134df0 msgr2=0x7ff1341215b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff134134df0 0x7ff1341215b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient: _finish_hunting 0 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 monclient: found mon.noname-a 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.748+0000 7ff131d74640 1 -- 10.20.193.78:0/2835804271 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff134123db0 con 0x7ff13409c4a0 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff131d74640 1 -- 10.20.193.78:0/2835804271 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff13409c4a0 msgr2=0x7ff1341cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff131d74640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff13409c4a0 0x7ff1341cda50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00b780 tx=0x7ff11c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:58.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 1 -- 10.20.193.78:0/2835804271 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c005270 con 0x7ff13409c4a0 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: dump: 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:58.768 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: _finish_auth 0 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: _check_auth_tickets 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff138b5e640 5 monclient: authenticate success, global_id 254157 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 1 -- 10.20.193.78:0/2835804271 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c010070 con 0x7ff13409c4a0 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 1 -- 10.20.193.78:0/2835804271 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c005650 con 0x7ff13409c4a0 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff130d72640 4 set_mon_vals no callback set 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff11affd640 10 monclient: dump: 2026-02-17T06:06:58.769 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff138b5e640 10 monclient: _renew_subs 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff138b5e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff134125540 con 0x7ff13409c4a0 2026-02-17T06:06:58.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff138b5e640 10 monclient: _renew_subs 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff138b5e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.750+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1340a1730 con 0x7ff13409c4a0 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff11affd640 1 -- 10.20.193.78:0/2835804271 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff11c015470 con 0x7ff13409c4a0 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff138b5e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff138b5e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff138b5e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff134135a70 con 0x7ff13409c4a0 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff11affd640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff10003de60 0x7ff100040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff11affd640 1 -- 10.20.193.78:0/2835804271 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(726..726 src has 525..726) ==== 4795+0+0 (secure 0 0 0) 0x7ff11c051d20 con 0x7ff13409c4a0 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff132575640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff10003de60 0x7ff100040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:06:58.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.751+0000 7ff132575640 10 monclient: get_auth_request con 0x7ff10003de60 auth_method 0 2026-02-17T06:06:58.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.754+0000 7ff11affd640 1 -- 10.20.193.78:0/2835804271 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff11c050710 con 0x7ff13409c4a0 2026-02-17T06:06:58.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.754+0000 7ff11affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:06:58.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.754+0000 7ff11affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:06:58.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.754+0000 7ff132575640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff10003de60 0x7ff100040320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff1340a7490 tx=0x7ff128033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:06:58.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.872+0000 7ff138b5e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:06:58.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.872+0000 7ff138b5e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:58.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.872+0000 7ff138b5e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:06:58.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.873+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff1340744d0 con 0x7ff13409c4a0 2026-02-17T06:06:58.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.873+0000 7ff11affd640 1 -- 10.20.193.78:0/2835804271 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7ff11c005830 con 0x7ff13409c4a0 2026-02-17T06:06:58.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.873+0000 7ff11affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:06:58.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.873+0000 7ff11affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:06:58.875 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":242,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:06:58.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff10003de60 msgr2=0x7ff100040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:58.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff10003de60 0x7ff100040320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff1340a7490 tx=0x7ff128033000 comp rx=0 tx=0).stop 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 10 monclient: shutdown 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff13409c4a0 msgr2=0x7ff1341cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff13409c4a0 0x7ff1341cda50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00b780 tx=0x7ff11c00bc50 comp rx=0 tx=0).stop 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 shutdown_connections 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff10003de60 0x7ff100040320 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff134134df0 0x7ff1341215b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff13409c4a0 0x7ff1341cda50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 --2- 10.20.193.78:0/2835804271 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff13406c890 0x7ff1341cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 >> 10.20.193.78:0/2835804271 conn(0x7ff1340a6620 msgr2=0x7ff134071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:06:58.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 shutdown_connections 2026-02-17T06:06:58.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:06:58.874+0000 7ff138b5e640 1 -- 10.20.193.78:0/2835804271 wait complete. 2026-02-17T06:07:01.883 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:01.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.950+0000 7f5b59734640 10 monclient: get_monmap_and_config 2026-02-17T06:07:01.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.950+0000 7f5b59734640 10 monclient: build_initial_monmap 2026-02-17T06:07:01.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.950+0000 7f5b59734640 10 monclient: monmap: 2026-02-17T06:07:01.951 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:01.951 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:01.952111+0000 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:01.952111+0000 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:01.952 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:01.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 1 Processor -- start 2026-02-17T06:07:01.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 1 -- start start 2026-02-17T06:07:01.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient: init 2026-02-17T06:07:01.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:01.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient: _add_conns ranks=[1,2,4,0,3] 2026-02-17T06:07:01.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b5406c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:01.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 0x7f5b5413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:01.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient(hunting): picked mon.noname-b con 0x7f5b5409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5b5406d380 0x7f5b5413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient(hunting): picked mon.noname-c con 0x7f5b5406d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b5406daf0 con 0x7f5b5406c890 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5b540a0880 con 0x7f5b5406d380 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5409cc80 con 0x7f5b5409c4a0 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b59734640 10 monclient(hunting): authenticate will time out at 1867.505630s 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b527fc640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 msgr2=0x7f5b5413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b527fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 0x7f5b5413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b52ffd640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7f5b5406c890 0x7f5b5406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:53828/0 (socket says 10.20.193.78:53828) 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b52ffd640 1 -- 10.20.193.78:0/3711813129 learned_addr learned my addr 10.20.193.78:0/3711813129 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b537fe640 1 --2- 10.20.193.78:0/3711813129 >> v2:10.20.193.115:3300/0 conn(0x7f5b5406d380 0x7f5b5413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b537fe640 10 monclient(hunting): get_auth_request con 0x7f5b5406d380 auth_method 0 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b537fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.951+0000 7f5b537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 10 monclient(hunting): handle_auth_done global_id 214417 payload 274 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 1 -- 10.20.193.78:0/3711813129 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 msgr2=0x7f5b5413b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 1 --2- 10.20.193.78:0/3711813129 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 0x7f5b5413b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 1 -- 10.20.193.78:0/3711813129 >> v1:10.20.193.115:6789/0 conn(0x7f5b5406c890 legacy=0x7f5b5406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:01.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 10 monclient: _finish_hunting 0 2026-02-17T06:07:01.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 1 monclient: found mon.noname-c 2026-02-17T06:07:01.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:01.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:07:01.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.952+0000 7f5b537fe640 1 -- 10.20.193.78:0/3711813129 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5409cf60 con 0x7f5b5406d380 2026-02-17T06:07:01.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b537fe640 1 -- 10.20.193.78:0/3711813129 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 msgr2=0x7f5b5413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:01.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b537fe640 1 --2- 10.20.193.78:0/3711813129 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 0x7f5b5413d8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009d90 tx=0x7f5b4402f170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=645c4d07be1ac9db server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b51ffb640 1 -- 10.20.193.78:0/3711813129 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4403c070 con 0x7f5b5406d380 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b51ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b51ffb640 10 monclient: dump: 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:01.956 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b51ffb640 10 monclient: _finish_auth 0 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b51ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.954+0000 7f5b59734640 5 monclient: authenticate success, global_id 214417 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b51ffb640 1 -- 10.20.193.78:0/3711813129 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b44037640 con 0x7f5b5406d380 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b51ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b51ffb640 1 -- 10.20.193.78:0/3711813129 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44037a10 con 0x7f5b5406d380 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b51ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b51ffb640 10 monclient: dump: 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:01.957 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 4 set_mon_vals no callback set 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 10 monclient: shutdown 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 -- 10.20.193.78:0/3711813129 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 msgr2=0x7f5b5413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 --2- 10.20.193.78:0/3711813129 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 0x7f5b5413d8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009d90 tx=0x7f5b4402f170 comp rx=0 tx=0).stop 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 -- 10.20.193.78:0/3711813129 shutdown_connections 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 --2- 10.20.193.78:0/3711813129 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 0x7f5b5413d8f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 --2- 10.20.193.78:0/3711813129 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 0x7f5b5413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 -- 10.20.193.78:0/3711813129 >> 10.20.193.78:0/3711813129 conn(0x7f5b540a6620 msgr2=0x7f5b540a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 -- 10.20.193.78:0/3711813129 shutdown_connections 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 -- 10.20.193.78:0/3711813129 wait complete. 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 10 monclient: build_initial_monmap 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 10 monclient: monmap: 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:01.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.955+0000 7f5b59734640 1 Processor -- start 2026-02-17T06:07:01.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 1 -- start start 2026-02-17T06:07:01.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient: init 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b5406c890 0x7f5b541cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient(hunting): picked mon.noname-a con 0x7f5b5406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 0x7f5b541360d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient(hunting): picked mon.noname-b con 0x7f5b5406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 0x7f5b54136610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient(hunting): picked mon.noname-c con 0x7f5b5409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5409bfa0 con 0x7f5b5406c890 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5409be20 con 0x7f5b5406d380 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:01.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5413de30 con 0x7f5b5409c4a0 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b59734640 10 monclient(hunting): authenticate will time out at 1867.510279s 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b537fe640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 msgr2=0x7f5b54136610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b537fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 0x7f5b54136610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b5406c890 0x7f5b541cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b5406c890 0x7f5b541cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44098/0 (socket says 10.20.193.78:44098) 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 -- 10.20.193.78:0/378202534 learned_addr learned my addr 10.20.193.78:0/378202534 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient(hunting): get_auth_request con 0x7f5b5406c890 auth_method 0 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b527fc640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 0x7f5b541360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b527fc640 10 monclient(hunting): get_auth_request con 0x7f5b5406d380 auth_method 0 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b527fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b527fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient(hunting): handle_auth_done global_id 254160 payload 824 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 -- 10.20.193.78:0/378202534 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 msgr2=0x7f5b54136610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 0x7f5b54136610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 -- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 msgr2=0x7f5b541360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 0x7f5b541360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:01.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 monclient: found mon.noname-a 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.956+0000 7f5b52ffd640 1 -- 10.20.193.78:0/378202534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b54122aa0 con 0x7f5b5406c890 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b52ffd640 1 -- 10.20.193.78:0/378202534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b5406c890 msgr2=0x7f5b541cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b52ffd640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b5406c890 0x7f5b541cf100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c750 tx=0x7f5b4800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 1 -- 10.20.193.78:0/378202534 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48005de0 con 0x7f5b5406c890 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: dump: 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: _finish_auth 0 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 1 -- 10.20.193.78:0/378202534 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b48007070 con 0x7f5b5406c890 2026-02-17T06:07:01.963 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b59734640 5 monclient: authenticate success, global_id 254160 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b59734640 10 monclient: _renew_subs 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b59734640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5406f7a0 con 0x7f5b5406c890 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 1 -- 10.20.193.78:0/378202534 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48007440 con 0x7f5b5406c890 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b337fe640 10 monclient: dump: 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b517fa640 4 set_mon_vals no callback set 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b59734640 10 monclient: _renew_subs 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b59734640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.958+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5409d810 con 0x7f5b5406c890 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.959+0000 7f5b59734640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.959+0000 7f5b59734640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:01.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.959+0000 7f5b59734640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.959+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5406cc70 con 0x7f5b5406c890 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.959+0000 7f5b337fe640 1 -- 10.20.193.78:0/378202534 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5b48007620 con 0x7f5b5406c890 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.959+0000 7f5b337fe640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b2003de60 0x7f5b20040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.959+0000 7f5b337fe640 1 -- 10.20.193.78:0/378202534 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(726..726 src has 525..726) ==== 4795+0+0 (secure 0 0 0) 0x7f5b480525f0 con 0x7f5b5406c890 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.959+0000 7f5b527fc640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b2003de60 0x7f5b20040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.960+0000 7f5b527fc640 10 monclient: get_auth_request con 0x7f5b2003de60 auth_method 0 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.962+0000 7f5b337fe640 1 -- 10.20.193.78:0/378202534 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5b4800ab10 con 0x7f5b5406c890 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.962+0000 7f5b337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.962+0000 7f5b337fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:01.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:01.962+0000 7f5b527fc640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b2003de60 0x7f5b20040320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5b541203b0 tx=0x7f5b3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:02.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.075+0000 7f5b59734640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:02.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.075+0000 7f5b59734640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:02.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.075+0000 7f5b59734640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:02.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.075+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5b54121090 con 0x7f5b5406c890 2026-02-17T06:07:02.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.075+0000 7f5b337fe640 1 -- 10.20.193.78:0/378202534 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f5b480546e0 con 0x7f5b5406c890 2026-02-17T06:07:02.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.075+0000 7f5b337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:02.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.075+0000 7f5b337fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:02.077 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":242,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:02.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b2003de60 msgr2=0x7f5b20040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:02.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b2003de60 0x7f5b20040320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5b541203b0 tx=0x7f5b3c008040 comp rx=0 tx=0).stop 2026-02-17T06:07:02.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 10 monclient: shutdown 2026-02-17T06:07:02.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:02.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b5406c890 msgr2=0x7f5b541cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:02.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b5406c890 0x7f5b541cf100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c750 tx=0x7f5b4800cc20 comp rx=0 tx=0).stop 2026-02-17T06:07:02.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 shutdown_connections 2026-02-17T06:07:02.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b5409c4a0 0x7f5b54136610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:02.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b2003de60 0x7f5b20040320 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:02.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b5406d380 0x7f5b541360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:02.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 --2- 10.20.193.78:0/378202534 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b5406c890 0x7f5b541cf100 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:02.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 >> 10.20.193.78:0/378202534 conn(0x7f5b540a6620 msgr2=0x7f5b540a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:02.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 shutdown_connections 2026-02-17T06:07:02.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:02.076+0000 7f5b59734640 1 -- 10.20.193.78:0/378202534 wait complete. 2026-02-17T06:07:05.085 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:05.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.152+0000 7f6f4a333640 10 monclient: get_monmap_and_config 2026-02-17T06:07:05.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.152+0000 7f6f4a333640 10 monclient: build_initial_monmap 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.152+0000 7f6f4a333640 10 monclient: monmap: 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:05.153770+0000 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:05.153770+0000 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:05.164 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 1 Processor -- start 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 1 -- start start 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient: init 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f6f440a3140 0x7f6f4406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient(hunting): picked mon.noname-c con 0x7f6f440a3140 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6f440a3520 0x7f6f440a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient(hunting): picked mon.noname-a con 0x7f6f440a3520 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6f4413c410 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f440a48a0 con 0x7f6f4413c410 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4406ffc0 con 0x7f6f440a3520 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4406f970 con 0x7f6f440a3140 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f4a333640 10 monclient(hunting): authenticate will time out at 1870.707402s 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6f440a3520 0x7f6f440a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f488a9640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f6f4413c410 0x7f6f4413e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:35382/0 (socket says 10.20.193.78:35382) 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f6f440a3520 0x7f6f440a3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44100/0 (socket says 10.20.193.78:44100) 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f488a9640 1 -- 10.20.193.78:0/2808077590 learned_addr learned my addr 10.20.193.78:0/2808077590 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient(hunting): get_auth_request con 0x7f6f440a3520 auth_method 0 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:05.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f43fff640 1 --2- 10.20.193.78:0/2808077590 >> v2:10.20.193.115:3300/0 conn(0x7f6f440a3140 0x7f6f4406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f43fff640 10 monclient(hunting): get_auth_request con 0x7f6f440a3140 auth_method 0 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f43fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f43fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient(hunting): handle_auth_done global_id 254163 payload 274 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 1 -- 10.20.193.78:0/2808077590 >> v2:10.20.193.115:3300/0 conn(0x7f6f440a3140 msgr2=0x7f6f4406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 1 --2- 10.20.193.78:0/2808077590 >> v2:10.20.193.115:3300/0 conn(0x7f6f440a3140 0x7f6f4406eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 1 -- 10.20.193.78:0/2808077590 >> v1:10.20.193.78:6789/0 conn(0x7f6f4413c410 legacy=0x7f6f4413e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient: _finish_hunting 0 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 1 monclient: found mon.noname-a 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.153+0000 7f6f437fe640 1 -- 10.20.193.78:0/2808077590 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4406fc50 con 0x7f6f440a3520 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f437fe640 1 -- 10.20.193.78:0/2808077590 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f440a3520 msgr2=0x7f6f440a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f437fe640 1 --2- 10.20.193.78:0/2808077590 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f440a3520 0x7f6f440a3930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6f44070d40 tx=0x7f6f3402ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bcc6dec258b322 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 1 -- 10.20.193.78:0/2808077590 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3403c070 con 0x7f6f440a3520 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: dump: 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:05.166 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: _finish_auth 0 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 1 -- 10.20.193.78:0/2808077590 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f3402fbb0 con 0x7f6f440a3520 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 5 monclient: authenticate success, global_id 254163 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 4 set_mon_vals no callback set 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 1 -- 10.20.193.78:0/2808077590 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34037440 con 0x7f6f440a3520 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f42ffd640 10 monclient: dump: 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 10 monclient: shutdown 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 1 -- 10.20.193.78:0/2808077590 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f440a3520 msgr2=0x7f6f440a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 1 --2- 10.20.193.78:0/2808077590 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f440a3520 0x7f6f440a3930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6f44070d40 tx=0x7f6f3402ed30 comp rx=0 tx=0).stop 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 1 -- 10.20.193.78:0/2808077590 shutdown_connections 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 1 --2- 10.20.193.78:0/2808077590 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f440a3520 0x7f6f440a3930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 1 --2- 10.20.193.78:0/2808077590 >> v2:10.20.193.115:3300/0 conn(0x7f6f440a3140 0x7f6f4406eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 1 -- 10.20.193.78:0/2808077590 >> 10.20.193.78:0/2808077590 conn(0x7f6f4409c000 msgr2=0x7f6f440a8740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:05.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 1 -- 10.20.193.78:0/2808077590 shutdown_connections 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.156+0000 7f6f4a333640 1 -- 10.20.193.78:0/2808077590 wait complete. 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient: build_initial_monmap 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient: monmap: 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 1 Processor -- start 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 1 -- start start 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient: init 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f440a3140 0x7f6f44124810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient(hunting): picked mon.noname-c con 0x7f6f440a3140 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f440a3520 0x7f6f441232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient(hunting): picked mon.noname-b con 0x7f6f440a3520 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 0x7f6f4413b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient(hunting): picked mon.noname-a con 0x7f6f4413c410 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f440a9300 con 0x7f6f4413c410 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f440a3e70 con 0x7f6f440a3520 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f6f440a9480 con 0x7f6f440a3140 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f4a333640 10 monclient(hunting): authenticate will time out at 1870.711544s 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f43fff640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f440a3140 msgr2=0x7f6f44124810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:07:05.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f43fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f440a3140 0x7f6f44124810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 0x7f6f4413b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 0x7f6f4413b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44104/0 (socket says 10.20.193.78:44104) 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 -- 10.20.193.78:0/1255309845 learned_addr learned my addr 10.20.193.78:0/1255309845 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient(hunting): get_auth_request con 0x7f6f4413c410 auth_method 0 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient(hunting): handle_auth_done global_id 254166 payload 824 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 -- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f440a3140 msgr2=0x7f6f44124810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f440a3140 0x7f6f44124810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 -- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f440a3520 msgr2=0x7f6f441232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f440a3520 0x7f6f441232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient: _finish_hunting 0 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 monclient: found mon.noname-a 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.157+0000 7f6f488a9640 1 -- 10.20.193.78:0/1255309845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f44125700 con 0x7f6f4413c410 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f488a9640 1 -- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 msgr2=0x7f6f4413b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f488a9640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 0x7f6f4413b610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800cb20 tx=0x7f6f3800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 1 -- 10.20.193.78:0/1255309845 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38012400 con 0x7f6f4413c410 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 10 monclient: dump: 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 10 monclient: _finish_auth 0 2026-02-17T06:07:05.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 1 -- 10.20.193.78:0/1255309845 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f380125c0 con 0x7f6f4413c410 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f4a333640 5 monclient: authenticate success, global_id 254166 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.159+0000 7f6f40ff9640 1 -- 10.20.193.78:0/1255309845 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f380129b0 con 0x7f6f4413c410 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 10 monclient: _renew_subs 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f427fc640 4 set_mon_vals no callback set 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f441259e0 con 0x7f6f4413c410 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f40ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f40ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f40ff9640 10 monclient: dump: 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 10 monclient: _renew_subs 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f441242d0 con 0x7f6f4413c410 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f40ff9640 1 -- 10.20.193.78:0/1255309845 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6f38012b50 con 0x7f6f4413c410 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4406eb10 con 0x7f6f4413c410 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f40ff9640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f1403deb0 0x7f6f14040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.160+0000 7f6f40ff9640 1 -- 10.20.193.78:0/1255309845 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(726..726 src has 525..726) ==== 4795+0+0 (secure 0 0 0) 0x7f6f38052fd0 con 0x7f6f4413c410 2026-02-17T06:07:05.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.161+0000 7f6f43fff640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f1403deb0 0x7f6f14040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:05.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.161+0000 7f6f43fff640 10 monclient: get_auth_request con 0x7f6f1403deb0 auth_method 0 2026-02-17T06:07:05.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.163+0000 7f6f488a9640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 0x7f6f4413b610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800cb20 tx=0x7f6f3800a060 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f6f380519c0 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) 2026-02-17T06:07:05.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.163+0000 7f6f43fff640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f1403deb0 0x7f6f14040370 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009180 tx=0x7f6f30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:05.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.991+0000 7f6f40ff9640 1 -- 10.20.193.78:0/1255309845 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6f380519c0 con 0x7f6f4413c410 2026-02-17T06:07:05.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.991+0000 7f6f40ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:05.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:05.991+0000 7f6f40ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:06.105+0000 7f6f4a333640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:06.105+0000 7f6f4a333640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:06.105+0000 7f6f4a333640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:06.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:06.105+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6f44125bb0 con 0x7f6f4413c410 2026-02-17T06:07:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:08.158+0000 7f6f1f7fe640 10 monclient: tick 2026-02-17T06:07:08.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:08.158+0000 7f6f1f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:08.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:08.158+0000 7f6f1f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:08.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:08.158+0000 7f6f1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:09.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:09.158+0000 7f6f1f7fe640 10 monclient: tick 2026-02-17T06:07:09.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:09.158+0000 7f6f1f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:09.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:09.158+0000 7f6f1f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:10.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.158+0000 7f6f1f7fe640 10 monclient: tick 2026-02-17T06:07:10.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.158+0000 7f6f1f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:10.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.158+0000 7f6f1f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:10.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.326+0000 7f6f40ff9640 1 -- 10.20.193.78:0/1255309845 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f6f38019c20 con 0x7f6f4413c410 2026-02-17T06:07:10.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.326+0000 7f6f40ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:10.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.326+0000 7f6f40ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:10.328 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":244,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:10.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f1403deb0 msgr2=0x7f6f14040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f1403deb0 0x7f6f14040370 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009180 tx=0x7f6f30008040 comp rx=0 tx=0).stop 2026-02-17T06:07:10.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 10 monclient: shutdown 2026-02-17T06:07:10.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:10.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 msgr2=0x7f6f4413b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 0x7f6f4413b610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800cb20 tx=0x7f6f3800a060 comp rx=0 tx=0).stop 2026-02-17T06:07:10.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 shutdown_connections 2026-02-17T06:07:10.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f6f4413c410 0x7f6f4413b610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f6f1403deb0 0x7f6f14040370 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f6f440a3520 0x7f6f441232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 --2- 10.20.193.78:0/1255309845 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f6f440a3140 0x7f6f44124810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 >> 10.20.193.78:0/1255309845 conn(0x7f6f4409c000 msgr2=0x7f6f440a5990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:10.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 shutdown_connections 2026-02-17T06:07:10.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.330+0000 7f6f4a333640 1 -- 10.20.193.78:0/1255309845 wait complete. 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:3347: done (2 left) 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:3348: rollback oid 2 current snap is 353 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 2 to 344 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:3344: finishing rollback tid 0 to trial11545420-18 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:3346: finishing copy_from to trial11545420-40 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:3346: got expected ENOENT (src dne) 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:3346: finishing copy_from racing read to trial11545420-40 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:3344: finishing rollback tid 1 to trial11545420-18 2026-02-17T06:07:10.336 INFO:tasks.rados.rados.0.trial115.stdout:3344: finishing rollback tid 2 to trial11545420-18 2026-02-17T06:07:10.337 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1572 (ObjNum 1133 snap 341 seq_num 1133) dirty exists 2026-02-17T06:07:10.337 INFO:tasks.rados.rados.0.trial115.stdout:3344: done (2 left) 2026-02-17T06:07:10.337 INFO:tasks.rados.rados.0.trial115.stdout:3346: done (1 left) 2026-02-17T06:07:10.337 INFO:tasks.rados.rados.0.trial115.stdout:3349: snap_remove snap 348 2026-02-17T06:07:10.342 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:07:10.342 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:07:10.342 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:07:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.407+0000 7fb43f2f2640 10 monclient: get_monmap_and_config 2026-02-17T06:07:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.407+0000 7fb43f2f2640 10 monclient: build_initial_monmap 2026-02-17T06:07:10.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.407+0000 7fb43f2f2640 10 monclient: monmap: 2026-02-17T06:07:10.411 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:10.411 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.411 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:10.408623+0000 2026-02-17T06:07:10.411 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:10.408623+0000 2026-02-17T06:07:10.411 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:10.411 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.411 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:10.411 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.407+0000 7fb43f2f2640 1 Processor -- start 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.407+0000 7fb43f2f2640 1 -- start start 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.407+0000 7fb43f2f2640 10 monclient: init 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb43806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:10.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb43806d380 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb4380a01f0 0x7fb43813d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 10 monclient(hunting): picked mon.noname-c con 0x7fb4380a01f0 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4380a0600 con 0x7fb43806d380 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb438074150 con 0x7fb43806c890 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb43809ef70 con 0x7fb4380a01f0 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43f2f2640 10 monclient(hunting): authenticate will time out at 1875.962185s 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43c866640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fb43806d380 0x7fb43813a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:45570/0 (socket says 10.20.193.78:45570) 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43c866640 1 -- 10.20.193.78:0/1134996740 learned_addr learned my addr 10.20.193.78:0/1134996740 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 1 --2- 10.20.193.78:0/1134996740 >> v2:10.20.193.115:3300/0 conn(0x7fb4380a01f0 0x7fb43813d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient(hunting): get_auth_request con 0x7fb4380a01f0 auth_method 0 2026-02-17T06:07:10.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient(hunting): handle_auth_done global_id 214426 payload 274 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 1 -- 10.20.193.78:0/1134996740 >> v1:10.20.193.115:6789/0 conn(0x7fb43806c890 legacy=0x7fb43806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 1 -- 10.20.193.78:0/1134996740 >> v1:10.20.193.78:6789/0 conn(0x7fb43806d380 legacy=0x7fb43813a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient: _finish_hunting 0 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 1 monclient: found mon.noname-c 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:10.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:07:10.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.408+0000 7fb43d868640 1 -- 10.20.193.78:0/1134996740 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43809f250 con 0x7fb4380a01f0 2026-02-17T06:07:10.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb43d868640 1 -- 10.20.193.78:0/1134996740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 msgr2=0x7fb43813d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb43d868640 1 --2- 10.20.193.78:0/1134996740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 0x7fb43813d970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb434009d90 tx=0x7fb43402f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e33d04ef692d754f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 1 -- 10.20.193.78:0/1134996740 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43403c070 con 0x7fb4380a01f0 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: dump: 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.424 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: _finish_auth 0 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: _check_auth_tickets 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb43f2f2640 5 monclient: authenticate success, global_id 214426 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb43f2f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 1 -- 10.20.193.78:0/1134996740 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb434037640 con 0x7fb4380a01f0 2026-02-17T06:07:10.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 1 -- 10.20.193.78:0/1134996740 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434037a10 con 0x7fb4380a01f0 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb43f2f2640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb43f2f2640 4 set_mon_vals no callback set 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.411+0000 7fb427fff640 10 monclient: dump: 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.426 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 10 monclient: shutdown 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1134996740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 msgr2=0x7fb43813d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 --2- 10.20.193.78:0/1134996740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 0x7fb43813d970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb434009d90 tx=0x7fb43402f1e0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1134996740 shutdown_connections 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 --2- 10.20.193.78:0/1134996740 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 0x7fb43813d970 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1134996740 >> 10.20.193.78:0/1134996740 conn(0x7fb4380a6620 msgr2=0x7fb4380a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1134996740 shutdown_connections 2026-02-17T06:07:10.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1134996740 wait complete. 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 10 monclient: build_initial_monmap 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 10 monclient: monmap: 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:10.428 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 Processor -- start 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 1 -- start start 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.412+0000 7fb43f2f2640 10 monclient: init 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb43806c890 0x7fb4381cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient(hunting): picked mon.noname-a con 0x7fb43806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb43806d380 0x7fb4381354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient(hunting): picked mon.noname-c con 0x7fb43806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 0x7fb4381cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient(hunting): picked mon.noname-b con 0x7fb4380a01f0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb43809d360 con 0x7fb43806c890 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb43809d1e0 con 0x7fb4380a01f0 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb43813deb0 con 0x7fb43806d380 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43f2f2640 10 monclient(hunting): authenticate will time out at 1875.967199s 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43c866640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb43806d380 msgr2=0x7fb4381354a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43c866640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb43806d380 0x7fb4381354a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb43806c890 0x7fb4381cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb43806c890 0x7fb4381cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35786/0 (socket says 10.20.193.78:35786) 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 -- 10.20.193.78:0/1851043150 learned_addr learned my addr 10.20.193.78:0/1851043150 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:10.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient(hunting): get_auth_request con 0x7fb43806c890 auth_method 0 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d868640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 0x7fb4381cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d868640 10 monclient(hunting): get_auth_request con 0x7fb4380a01f0 auth_method 0 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d868640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d868640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d868640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d868640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d868640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d868640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient(hunting): handle_auth_done global_id 254169 payload 824 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb43806d380 msgr2=0x7fb4381354a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb43806d380 0x7fb4381354a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 msgr2=0x7fb4381cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb4380a01f0 0x7fb4381cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient: _finish_hunting 0 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 monclient: found mon.noname-a 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:10.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.413+0000 7fb43d067640 1 -- 10.20.193.78:0/1851043150 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43813d2b0 con 0x7fb43806c890 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.415+0000 7fb43d067640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb43806c890 msgr2=0x7fb4381cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.415+0000 7fb43d067640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb43806c890 0x7fb4381cf260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb42800c5d0 tx=0x7fb42800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.415+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 DispatchQueue::entry inject delay of 0.002000 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428016020 con 0x7fb43806c890 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: dump: 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.433 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: _finish_auth 0 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:10.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb428012a60 con 0x7fb43806c890 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb43f2f2640 5 monclient: authenticate success, global_id 254169 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb43f2f2640 10 monclient: _renew_subs 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb43f2f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43806f330 con 0x7fb43806c890 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428012e50 con 0x7fb43806c890 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb425ffb640 10 monclient: dump: 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.435 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb4277fe640 4 set_mon_vals no callback set 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb43f2f2640 10 monclient: _renew_subs 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb43f2f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.417+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb438121b90 con 0x7fb43806c890 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43f2f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:10.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43f2f2640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381359e0 0x7fb4381cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43f2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb43806cc70 con 0x7fb4381359e0 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381359e0 0x7fb4381cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 10 monclient: get_auth_request con 0x7fb4381359e0 auth_method 0 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 10 monclient(hunting): handle_auth_done global_id 254172 payload 824 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43c866640 1 -- 10.20.193.78:0/1851043150 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb438121f30 con 0x7fb4381359e0 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb42801b400 con 0x7fb43806c890 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb425ffb640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb40803e050 0x7fb408040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(727..727 src has 525..727) ==== 4811+0+0 (secure 0 0 0) 0x7fb428052800 con 0x7fb43806c890 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43d868640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb40803e050 0x7fb408040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.418+0000 7fb43d868640 10 monclient: get_auth_request con 0x7fb40803e050 auth_method 0 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb43c866640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381359e0 msgr2=0x7fb4381cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb43c866640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381359e0 0x7fb4381cc2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb42c004910 tx=0x7fb42c00cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c017070 con 0x7fb4381359e0 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 10 monclient: dump: 2026-02-17T06:07:10.438 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fb42c005d40 con 0x7fb4381359e0 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:10.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381359e0 msgr2=0x7fb4381cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381359e0 0x7fb4381cc2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb42c004910 tx=0x7fb42c00cb20 comp rx=0 tx=0).stop 2026-02-17T06:07:10.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381359e0 msgr2=0x7fb4381cc2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:10.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.420+0000 7fb425ffb640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381359e0 0x7fb4381cc2a0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb42c004910 tx=0x7fb42c00cb20 comp rx=0 tx=0).stop 2026-02-17T06:07:10.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.421+0000 7fb43d868640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb40803e050 0x7fb408040510 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb438074e00 tx=0x7fb434033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43f2f2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43f2f2640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381cc7e0 0x7fb4381da5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43f2f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381ccbf0 con 0x7fb4381cc7e0 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381cc7e0 0x7fb4381da5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 10 monclient: get_auth_request con 0x7fb4381cc7e0 auth_method 0 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 10 monclient(hunting): handle_auth_done global_id 254175 payload 824 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.424+0000 7fb43c866640 1 -- 10.20.193.78:0/1851043150 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb4381203f0 con 0x7fb4381cc7e0 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43c866640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381cc7e0 msgr2=0x7fb4381da5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43c866640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381cc7e0 0x7fb4381da5f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb43809e270 tx=0x7fb42c004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c01ae60 con 0x7fb4381cc7e0 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:10.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 10 monclient: dump: 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4699 (secure 0 0 0) 0x7fb42c007490 con 0x7fb4381cc7e0 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:10.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381cc7e0 msgr2=0x7fb4381da5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381cc7e0 0x7fb4381da5f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb43809e270 tx=0x7fb42c004b10 comp rx=0 tx=0).stop 2026-02-17T06:07:10.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381cc7e0 msgr2=0x7fb4381da5f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:10.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb425ffb640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb4381cc7e0 0x7fb4381da5f0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb43809e270 tx=0x7fb42c004b10 comp rx=0 tx=0).stop 2026-02-17T06:07:10.445 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:10.445 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:10.445 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:10.445 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 244, 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 29800, 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:10.446 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:10.447 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:10.448 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:10.449 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:10.450 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:10.451 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.452 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.453 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.454 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:10.455 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:10.456 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:07:10.456 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.456 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.456 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.456 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:10.456 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb40803e050 msgr2=0x7fb408040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb40803e050 0x7fb408040510 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb438074e00 tx=0x7fb434033000 comp rx=0 tx=0).stop 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 10 monclient: shutdown 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb43806c890 msgr2=0x7fb4381cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb43806c890 0x7fb4381cf260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb42800c5d0 tx=0x7fb42800caa0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43d868640 1 -- 10.20.193.78:0/1851043150 reap_dead start 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 shutdown_connections 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 1 --2- 10.20.193.78:0/1851043150 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb43806c890 0x7fb4381cf260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.426+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 >> 10.20.193.78:0/1851043150 conn(0x7fb4380a6620 msgr2=0x7fb4380a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:10.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.427+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 shutdown_connections 2026-02-17T06:07:10.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.427+0000 7fb43f2f2640 1 -- 10.20.193.78:0/1851043150 wait complete. 2026-02-17T06:07:10.458 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:07:10.458 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T06:07:10.458 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:07:10.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient: get_monmap_and_config 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient: build_initial_monmap 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient: monmap: 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:10.526338+0000 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:10.526338+0000 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:10.526 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 1 Processor -- start 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 1 -- start start 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient: init 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fccc006c890 0x7fccc006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient(hunting): picked mon.noname-c con 0x7fccc006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fccc009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fccc006d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:10.527 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccc0135c10 con 0x7fccc006d1b0 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccc006daf0 con 0x7fccc009c4a0 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fccc00a0880 con 0x7fccc006c890 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccc50b0640 10 monclient(hunting): authenticate will time out at 1876.079872s 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccbf577640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fccc006d1b0 0x7fccc0135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:45582/0 (socket says 10.20.193.78:45582) 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.525+0000 7fccbf577640 1 -- 10.20.193.78:0/3670590593 learned_addr learned my addr 10.20.193.78:0/3670590593 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 1 --2- 10.20.193.78:0/3670590593 >> v2:10.20.193.115:3300/0 conn(0x7fccc006c890 0x7fccc006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient(hunting): get_auth_request con 0x7fccc006c890 auth_method 0 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient(hunting): handle_auth_done global_id 214432 payload 274 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 1 -- 10.20.193.78:0/3670590593 >> v1:10.20.193.115:6789/0 conn(0x7fccc009c4a0 legacy=0x7fccc0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 1 -- 10.20.193.78:0/3670590593 >> v1:10.20.193.78:6789/0 conn(0x7fccc006d1b0 legacy=0x7fccc0135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient: _finish_hunting 0 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 1 monclient: found mon.noname-c 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:07:10.528 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.526+0000 7fccbed76640 1 -- 10.20.193.78:0/3670590593 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc009cc80 con 0x7fccc006c890 2026-02-17T06:07:10.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.528+0000 7fccbed76640 1 -- 10.20.193.78:0/3670590593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006c890 msgr2=0x7fccc006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:10.529 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.528+0000 7fccbed76640 1 --2- 10.20.193.78:0/3670590593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006c890 0x7fccc006cc70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fccb40098a0 tx=0x7fccb402ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e998d45ebaf0fde server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 1 -- 10.20.193.78:0/3670590593 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb403c070 con 0x7fccc006c890 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: dump: 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: _finish_auth 0 2026-02-17T06:07:10.530 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: _check_auth_tickets 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 5 monclient: authenticate success, global_id 214432 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 1 -- 10.20.193.78:0/3670590593 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb402fce0 con 0x7fccc006c890 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 1 -- 10.20.193.78:0/3670590593 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4037660 con 0x7fccc006c890 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 4 set_mon_vals no callback set 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccbdd74640 10 monclient: dump: 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 10 monclient: shutdown 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 -- 10.20.193.78:0/3670590593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006c890 msgr2=0x7fccc006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 --2- 10.20.193.78:0/3670590593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006c890 0x7fccc006cc70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fccb40098a0 tx=0x7fccb402ed80 comp rx=0 tx=0).stop 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 -- 10.20.193.78:0/3670590593 shutdown_connections 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 --2- 10.20.193.78:0/3670590593 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006c890 0x7fccc006cc70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 -- 10.20.193.78:0/3670590593 >> 10.20.193.78:0/3670590593 conn(0x7fccc00a6620 msgr2=0x7fccc00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 -- 10.20.193.78:0/3670590593 shutdown_connections 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 -- 10.20.193.78:0/3670590593 wait complete. 2026-02-17T06:07:10.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 10 monclient: build_initial_monmap 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 10 monclient: monmap: 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 Processor -- start 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 1 -- start start 2026-02-17T06:07:10.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.529+0000 7fccc50b0640 10 monclient: init 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fccc006c890 0x7fccc01cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient(hunting): picked mon.noname-a con 0x7fccc006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006d1b0 0x7fccc0123df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient(hunting): picked mon.noname-b con 0x7fccc006d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fccc009c4a0 0x7fccc0124330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient(hunting): picked mon.noname-c con 0x7fccc009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fccc00a0020 con 0x7fccc006c890 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fccc006d640 con 0x7fccc006d1b0 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fccc00a01a0 con 0x7fccc009c4a0 2026-02-17T06:07:10.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccc50b0640 10 monclient(hunting): authenticate will time out at 1876.084141s 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbf577640 1 -- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fccc009c4a0 msgr2=0x7fccc0124330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3301/0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbf577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fccc009c4a0 0x7fccc0124330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fccc006c890 0x7fccc01cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fccc006c890 0x7fccc01cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:35808/0 (socket says 10.20.193.78:35808) 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 -- 10.20.193.78:0/865451182 learned_addr learned my addr 10.20.193.78:0/865451182 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient(hunting): get_auth_request con 0x7fccc006c890 auth_method 0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient(hunting): handle_auth_done global_id 254178 payload 824 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbe575640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006d1b0 0x7fccc0123df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fccc009c4a0 msgr2=0x7fccc0124330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fccc009c4a0 0x7fccc0124330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006d1b0 msgr2=0x7fccc0123df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006d1b0 0x7fccc0123df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient: _finish_hunting 0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 monclient: found mon.noname-a 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbed76640 1 -- 10.20.193.78:0/865451182 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc0128760 con 0x7fccc006c890 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbe575640 10 monclient: get_auth_request con 0x7fccc006d1b0 auth_method 0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.530+0000 7fccbe575640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc006d1b0 0x7fccc0123df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccbed76640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fccc006c890 msgr2=0x7fccc01cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccbed76640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fccc006c890 0x7fccc01cf6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccb40098a0 tx=0x7fccb40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4046070 con 0x7fccc006c890 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: dump: 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccc50b0640 5 monclient: authenticate success, global_id 254178 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb4041870 con 0x7fccc006c890 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb403c040 con 0x7fccc006c890 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccc50b0640 10 monclient: _renew_subs 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccbd573640 4 set_mon_vals no callback set 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccc50b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc01277c0 con 0x7fccc006c890 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:10.536 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fcc9f7fe640 10 monclient: dump: 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccc50b0640 10 monclient: _renew_subs 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccc50b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.532+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc0126370 con 0x7fccc006c890 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccc50b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccc50b0640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc0125030 0x7fccc0071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccc50b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fccc00a28f0 con 0x7fccc0125030 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fccb4041c60 con 0x7fccc006c890 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc0125030 0x7fccc0071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 10 monclient: get_auth_request con 0x7fccc0125030 auth_method 0 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fcc9f7fe640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc8c03de60 0x7fcc8c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(727..727 src has 525..727) ==== 4811+0+0 (secure 0 0 0) 0x7fccb40781c0 con 0x7fccc006c890 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbf577640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc8c03de60 0x7fcc8c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbf577640 10 monclient: get_auth_request con 0x7fcc8c03de60 auth_method 0 2026-02-17T06:07:10.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 10 monclient(hunting): handle_auth_done global_id 214435 payload 824 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.533+0000 7fccbe575640 1 -- 10.20.193.78:0/865451182 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fccc009bf40 con 0x7fccc0125030 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fccbf577640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc8c03de60 0x7fcc8c040320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fccc01261b0 tx=0x7fccb0005220 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fccbe575640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc0125030 msgr2=0x7fccc0071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fccbe575640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc0125030 0x7fccc0071420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fccc0134ea0 tx=0x7fcca800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8005270 con 0x7fccc0125030 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 10 monclient: dump: 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.538 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fcca8005270 con 0x7fccc0125030 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc0125030 msgr2=0x7fccc0071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc0125030 0x7fccc0071420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fccc0134ea0 tx=0x7fcca800db90 comp rx=0 tx=0).stop 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc0125030 msgr2=0x7fccc0071420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:10.539 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.536+0000 7fcc9f7fe640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc0125030 0x7fccc0071420 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fccc0134ea0 tx=0x7fcca800db90 comp rx=0 tx=0).stop 2026-02-17T06:07:10.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccc50b0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:10.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccc50b0640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc01cfbe0 0x7fccc01da350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:10.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccc50b0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:10.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fccc0072ae0 con 0x7fccc01cfbe0 2026-02-17T06:07:10.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc01cfbe0 0x7fccc01da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 10 monclient: get_auth_request con 0x7fccc01cfbe0 auth_method 0 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 10 monclient(hunting): handle_auth_done global_id 214438 payload 824 2026-02-17T06:07:10.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.540+0000 7fccbe575640 1 -- 10.20.193.78:0/865451182 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fccc00744d0 con 0x7fccc01cfbe0 2026-02-17T06:07:10.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.542+0000 7fccbe575640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc01cfbe0 msgr2=0x7fccc01da350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:10.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.542+0000 7fccbe575640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc01cfbe0 0x7fccc01da350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcca8016000 tx=0x7fcca8003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:10.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.542+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8006cf0 con 0x7fccc01cfbe0 2026-02-17T06:07:10.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.542+0000 7fcc9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:10.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.542+0000 7fcc9f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:10.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.542+0000 7fcc9f7fe640 10 monclient: dump: 2026-02-17T06:07:10.543 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7fcca8006cf0 con 0x7fccc01cfbe0 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fcc9f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fcc9f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc01cfbe0 msgr2=0x7fccc01da350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fcc9f7fe640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc01cfbe0 0x7fccc01da350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcca8016000 tx=0x7fcca8003860 comp rx=0 tx=0).stop 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fcc9f7fe640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc01cfbe0 msgr2=0x7fccc01da350 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fcc9f7fe640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fccc01cfbe0 0x7fccc01da350 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcca8016000 tx=0x7fcca8003860 comp rx=0 tx=0).stop 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 244, 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 233390, 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:10.544 INFO:teuthology.orchestra.run.trial078.stdout: 1 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:10.545 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:10.546 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:10.547 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:10.548 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.549 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:10.550 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc8c03de60 msgr2=0x7fcc8c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcc8c03de60 0x7fcc8c040320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fccc01261b0 tx=0x7fccb0005220 comp rx=0 tx=0).stop 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 10 monclient: shutdown 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fccc006c890 msgr2=0x7fccc01cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 1 --2- 10.20.193.78:0/865451182 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fccc006c890 0x7fccc01cf6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccb40098a0 tx=0x7fccb40047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccbf577640 1 -- 10.20.193.78:0/865451182 reap_dead start 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccbf577640 1 -- 10.20.193.78:0/865451182 reap_dead start 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 shutdown_connections 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 >> 10.20.193.78:0/865451182 conn(0x7fccc00a6620 msgr2=0x7fccc00a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 shutdown_connections 2026-02-17T06:07:10.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:10.543+0000 7fccc50b0640 1 -- 10.20.193.78:0/865451182 wait complete. 2026-02-17T06:07:10.551 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:07:10.551 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3349: done (1 left) 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3350: copy_from oid 43 from oid 21 current snap is 353 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3348: finishing rollback tid 0 to trial11545420-2 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3348: finishing rollback tid 1 to trial11545420-2 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3348: finishing rollback tid 2 to trial11545420-2 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1515 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3348: done (1 left) 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3351: read oid 17 snap 349 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3351: expect deleted 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3352: write oid 33 current snap is 353 2026-02-17T06:07:10.634 INFO:tasks.rados.rados.0.trial115.stdout:3352: seq_num 1152 ranges {561664=656151,1729021=528400,2669170=692494} 2026-02-17T06:07:10.637 INFO:tasks.rados.rados.0.trial115.stdout:3352: writing trial11545420-33 from 561664 to 1217815 tid 1 2026-02-17T06:07:10.641 INFO:tasks.rados.rados.0.trial115.stdout:3352: writing trial11545420-33 from 1729021 to 2257421 tid 2 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3352: writing trial11545420-33 from 2669170 to 3361664 tid 3 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3353: rollback oid 39 current snap is 353 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 39 to 340 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3350: finishing copy_from to trial11545420-43 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3350: got expected ENOENT (src dne) 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3350: finishing copy_from racing read to trial11545420-43 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3352: finishing write tid 1 to trial11545420-33 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3352: finishing write tid 2 to trial11545420-33 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3350: done (3 left) 2026-02-17T06:07:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3351: done (2 left) 2026-02-17T06:07:10.647 INFO:tasks.rados.rados.0.trial115.stdout:3354: delete oid 9 current snap is 353 2026-02-17T06:07:10.648 INFO:tasks.rados.rados.0.trial115.stdout:3352: finishing write tid 3 to trial11545420-33 2026-02-17T06:07:10.648 INFO:tasks.rados.rados.0.trial115.stdout:3352: finishing write tid 4 to trial11545420-33 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3352: finishing write tid 5 to trial11545420-33 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3352: oid 33 updating version 0 to 1655 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3352: oid 33 updating version 1655 to 1656 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3352: oid 33 updating version 1656 to 1657 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3352: oid 33 updating version 1657 to 1658 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1658 (ObjNum 1152 snap 353 seq_num 1152) dirty exists 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3352: left oid 33 (ObjNum 1152 snap 353 seq_num 1152) 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3352: done (2 left) 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3354: done (1 left) 2026-02-17T06:07:10.649 INFO:tasks.rados.rados.0.trial115.stdout:3355: snap_remove snap 339 2026-02-17T06:07:11.663 INFO:tasks.rados.rados.0.trial115.stdout:3355: done (1 left) 2026-02-17T06:07:11.663 INFO:tasks.rados.rados.0.trial115.stdout:3356: write oid 19 current snap is 353 2026-02-17T06:07:11.664 INFO:tasks.rados.rados.0.trial115.stdout:3356: seq_num 1154 ranges {595278=406649,1729632=532069,3007663=387615} 2026-02-17T06:07:11.667 INFO:tasks.rados.rados.0.trial115.stdout:3356: writing trial11545420-19 from 595278 to 1001927 tid 1 2026-02-17T06:07:11.672 INFO:tasks.rados.rados.0.trial115.stdout:3356: writing trial11545420-19 from 1729632 to 2261701 tid 2 2026-02-17T06:07:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3356: writing trial11545420-19 from 3007663 to 3395278 tid 3 2026-02-17T06:07:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3357: snap_remove snap 336 2026-02-17T06:07:12.667 INFO:tasks.rados.rados.0.trial115.stdout:3357: done (2 left) 2026-02-17T06:07:12.667 INFO:tasks.rados.rados.0.trial115.stdout:3358: delete oid 48 current snap is 353 2026-02-17T06:07:12.667 INFO:tasks.rados.rados.0.trial115.stdout:3353: finishing rollback tid 0 to trial11545420-39 2026-02-17T06:07:12.667 INFO:tasks.rados.rados.0.trial115.stdout:3353: finishing rollback tid 1 to trial11545420-39 2026-02-17T06:07:12.667 INFO:tasks.rados.rados.0.trial115.stdout:3353: finishing rollback tid 2 to trial11545420-39 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1741 (ObjNum 1118 snap 336 seq_num 1118) dirty exists 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: finishing write tid 1 to trial11545420-19 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: finishing write tid 2 to trial11545420-19 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: finishing write tid 3 to trial11545420-19 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: finishing write tid 4 to trial11545420-19 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: finishing write tid 5 to trial11545420-19 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: oid 19 updating version 0 to 1770 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: oid 19 version 1770 is already newer than 1763 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: oid 19 version 1770 is already newer than 1769 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: oid 19 version 1770 is already newer than 1768 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1770 (ObjNum 1154 snap 353 seq_num 1154) dirty exists 2026-02-17T06:07:12.668 INFO:tasks.rados.rados.0.trial115.stdout:3356: left oid 19 (ObjNum 1154 snap 353 seq_num 1154) 2026-02-17T06:07:12.669 INFO:tasks.rados.rados.0.trial115.stdout:3353: done (2 left) 2026-02-17T06:07:12.669 INFO:tasks.rados.rados.0.trial115.stdout:3356: done (1 left) 2026-02-17T06:07:12.669 INFO:tasks.rados.rados.0.trial115.stdout:3358: done (0 left) 2026-02-17T06:07:12.669 INFO:tasks.rados.rados.0.trial115.stdout:3359: delete oid 46 current snap is 353 2026-02-17T06:07:12.672 INFO:tasks.rados.rados.0.trial115.stdout:3359: done (0 left) 2026-02-17T06:07:12.672 INFO:tasks.rados.rados.0.trial115.stdout:3360: delete oid 37 current snap is 353 2026-02-17T06:07:12.673 INFO:tasks.rados.rados.0.trial115.stdout:3360: done (0 left) 2026-02-17T06:07:12.673 INFO:tasks.rados.rados.0.trial115.stdout:3361: delete oid 30 current snap is 353 2026-02-17T06:07:12.675 INFO:tasks.rados.rados.0.trial115.stdout:3361: done (0 left) 2026-02-17T06:07:12.675 INFO:tasks.rados.rados.0.trial115.stdout:3362: snap_remove snap 351 2026-02-17T06:07:13.667 INFO:tasks.rados.rados.0.trial115.stdout:3362: done (0 left) 2026-02-17T06:07:13.667 INFO:tasks.rados.rados.0.trial115.stdout:3363: copy_from oid 3 from oid 22 current snap is 353 2026-02-17T06:07:13.667 INFO:tasks.rados.rados.0.trial115.stdout:3364: rollback oid 44 current snap is 353 2026-02-17T06:07:13.667 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 44 to 341 2026-02-17T06:07:13.668 INFO:tasks.rados.rados.0.trial115.stdout:3365: delete oid 24 current snap is 353 2026-02-17T06:07:13.670 INFO:tasks.rados.rados.0.trial115.stdout:3365: done (2 left) 2026-02-17T06:07:13.670 INFO:tasks.rados.rados.0.trial115.stdout:3366: read oid 27 snap -1 2026-02-17T06:07:13.670 INFO:tasks.rados.rados.0.trial115.stdout:3366: expect (ObjNum 1146 snap 349 seq_num 1146) 2026-02-17T06:07:13.670 INFO:tasks.rados.rados.0.trial115.stdout:3367: delete oid 42 current snap is 353 2026-02-17T06:07:13.671 INFO:tasks.rados.rados.0.trial115.stdout:3367: done (3 left) 2026-02-17T06:07:13.671 INFO:tasks.rados.rados.0.trial115.stdout:3368: copy_from oid 28 from oid 38 current snap is 353 2026-02-17T06:07:13.671 INFO:tasks.rados.rados.0.trial115.stdout:3369: snap_create 2026-02-17T06:07:13.674 INFO:tasks.rados.rados.0.trial115.stdout:3364: finishing rollback tid 0 to trial11545420-44 2026-02-17T06:07:13.674 INFO:tasks.rados.rados.0.trial115.stdout:3364: finishing rollback tid 1 to trial11545420-44 2026-02-17T06:07:13.674 INFO:tasks.rados.rados.0.trial115.stdout:3364: finishing rollback tid 2 to trial11545420-44 2026-02-17T06:07:13.674 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1461 (ObjNum 1050 snap 312 seq_num 1050) dirty exists 2026-02-17T06:07:13.677 INFO:tasks.rados.rados.0.trial115.stdout:3368: finishing copy_from to trial11545420-28 2026-02-17T06:07:13.677 INFO:tasks.rados.rados.0.trial115.stdout:3368: got expected ENOENT (src dne) 2026-02-17T06:07:13.677 INFO:tasks.rados.rados.0.trial115.stdout:3363: finishing copy_from racing read to trial11545420-3 2026-02-17T06:07:13.677 INFO:tasks.rados.rados.0.trial115.stdout:3368: finishing copy_from racing read to trial11545420-28 2026-02-17T06:07:13.678 INFO:tasks.rados.rados.0.trial115.stdout:3366: expect (ObjNum 1146 snap 349 seq_num 1146) 2026-02-17T06:07:13.755 INFO:tasks.rados.rados.0.trial115.stdout:3363: finishing copy_from to trial11545420-3 2026-02-17T06:07:13.755 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1364 (ObjNum 1136 snap 342 seq_num 1136) dirty exists 2026-02-17T06:07:14.671 INFO:tasks.rados.rados.0.trial115.stdout:3363: done (4 left) 2026-02-17T06:07:14.671 INFO:tasks.rados.rados.0.trial115.stdout:3364: done (3 left) 2026-02-17T06:07:14.672 INFO:tasks.rados.rados.0.trial115.stdout:3366: done (2 left) 2026-02-17T06:07:14.672 INFO:tasks.rados.rados.0.trial115.stdout:3368: done (1 left) 2026-02-17T06:07:14.672 INFO:tasks.rados.rados.0.trial115.stdout:3369: done (0 left) 2026-02-17T06:07:14.672 INFO:tasks.rados.rados.0.trial115.stdout:3370: snap_create 2026-02-17T06:07:15.633 INFO:tasks.rados.rados.0.trial115.stdout:3370: done (0 left) 2026-02-17T06:07:15.633 INFO:tasks.rados.rados.0.trial115.stdout:3371: write (excl) oid 13 current snap is 355 2026-02-17T06:07:15.633 INFO:tasks.rados.rados.0.trial115.stdout:3371: seq_num 1161 ranges {712927=649745,2139006=686734} 2026-02-17T06:07:15.639 INFO:tasks.rados.rados.0.trial115.stdout:3371: writing trial11545420-13 from 712927 to 1362672 tid 1 2026-02-17T06:07:15.645 INFO:tasks.rados.rados.0.trial115.stdout:3371: writing trial11545420-13 from 2139006 to 2825740 tid 2 2026-02-17T06:07:15.645 INFO:tasks.rados.rados.0.trial115.stdout:3372: copy_from oid 15 from oid 43 current snap is 355 2026-02-17T06:07:15.645 INFO:tasks.rados.rados.0.trial115.stdout:3371: finishing write tid 1 to trial11545420-13 2026-02-17T06:07:15.645 INFO:tasks.rados.rados.0.trial115.stdout:3373: write oid 31 current snap is 355 2026-02-17T06:07:15.645 INFO:tasks.rados.rados.0.trial115.stdout:3373: seq_num 1162 ranges {729734=724994,2197488=430392,3227047=302687} 2026-02-17T06:07:15.650 INFO:tasks.rados.rados.0.trial115.stdout:3373: writing trial11545420-31 from 729734 to 1454728 tid 1 2026-02-17T06:07:15.654 INFO:tasks.rados.rados.0.trial115.stdout:3373: writing trial11545420-31 from 2197488 to 2627880 tid 2 2026-02-17T06:07:15.658 INFO:tasks.rados.rados.0.trial115.stdout:3373: writing trial11545420-31 from 3227047 to 3529734 tid 3 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:3374: snap_create 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:3371: finishing write tid 2 to trial11545420-13 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:3371: finishing write tid 3 to trial11545420-13 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:3371: finishing write tid 4 to trial11545420-13 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:3371: oid 13 updating version 0 to 1481 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:3371: oid 13 version 1481 is already newer than 1480 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:3371: oid 13 updating version 1481 to 1482 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1482 (ObjNum 1161 snap 355 seq_num 1161) dirty exists 2026-02-17T06:07:15.659 INFO:tasks.rados.rados.0.trial115.stdout:3371: left oid 13 (ObjNum 1161 snap 355 seq_num 1161) 2026-02-17T06:07:15.660 INFO:tasks.rados.rados.0.trial115.stdout:3372: finishing copy_from racing read to trial11545420-15 2026-02-17T06:07:15.660 INFO:tasks.rados.rados.0.trial115.stdout:3372: finishing copy_from to trial11545420-15 2026-02-17T06:07:15.660 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1626 (ObjNum 1141 snap 347 seq_num 1141) dirty exists 2026-02-17T06:07:15.660 INFO:tasks.rados.rados.0.trial115.stdout:3373: finishing write tid 1 to trial11545420-31 2026-02-17T06:07:15.660 INFO:tasks.rados.rados.0.trial115.stdout:3373: finishing write tid 2 to trial11545420-31 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:3373: finishing write tid 3 to trial11545420-31 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:3373: finishing write tid 4 to trial11545420-31 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:3373: finishing write tid 5 to trial11545420-31 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:3373: oid 31 updating version 0 to 1534 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:3373: oid 31 version 1534 is already newer than 1533 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:3373: oid 31 version 1534 is already newer than 1532 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:3373: oid 31 version 1534 is already newer than 1531 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1534 (ObjNum 1162 snap 355 seq_num 1162) dirty exists 2026-02-17T06:07:15.661 INFO:tasks.rados.rados.0.trial115.stdout:3373: left oid 31 (ObjNum 1162 snap 355 seq_num 1162) 2026-02-17T06:07:16.636 INFO:tasks.rados.rados.0.trial115.stdout:3371: done (3 left) 2026-02-17T06:07:16.636 INFO:tasks.rados.rados.0.trial115.stdout:3372: done (2 left) 2026-02-17T06:07:16.636 INFO:tasks.rados.rados.0.trial115.stdout:3373: done (1 left) 2026-02-17T06:07:16.636 INFO:tasks.rados.rados.0.trial115.stdout:3374: done (0 left) 2026-02-17T06:07:16.636 INFO:tasks.rados.rados.0.trial115.stdout:3375: snap_create 2026-02-17T06:07:17.637 INFO:tasks.rados.rados.0.trial115.stdout:3375: done (0 left) 2026-02-17T06:07:17.637 INFO:tasks.rados.rados.0.trial115.stdout:3376: read oid 45 snap 344 2026-02-17T06:07:17.638 INFO:tasks.rados.rados.0.trial115.stdout:3376: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:07:17.638 INFO:tasks.rados.rados.0.trial115.stdout:3377: copy_from oid 1 from oid 21 current snap is 357 2026-02-17T06:07:17.638 INFO:tasks.rados.rados.0.trial115.stdout:3378: write oid 11 current snap is 357 2026-02-17T06:07:17.638 INFO:tasks.rados.rados.0.trial115.stdout:3378: seq_num 1163 ranges {746541=400243,1739617=690403,3138117=408424} 2026-02-17T06:07:17.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: writing trial11545420-11 from 746541 to 1146784 tid 1 2026-02-17T06:07:17.646 INFO:tasks.rados.rados.0.trial115.stdout:3378: writing trial11545420-11 from 1739617 to 2430020 tid 2 2026-02-17T06:07:17.651 INFO:tasks.rados.rados.0.trial115.stdout:3378: writing trial11545420-11 from 3138117 to 3546541 tid 3 2026-02-17T06:07:17.651 INFO:tasks.rados.rados.0.trial115.stdout:3379: rollback oid 4 current snap is 357 2026-02-17T06:07:17.651 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 355 2026-02-17T06:07:17.651 INFO:tasks.rados.rados.0.trial115.stdout:3377: finishing copy_from to trial11545420-1 2026-02-17T06:07:17.651 INFO:tasks.rados.rados.0.trial115.stdout:3377: got expected ENOENT (src dne) 2026-02-17T06:07:17.652 INFO:tasks.rados.rados.0.trial115.stdout:3380: read oid 20 snap 352 2026-02-17T06:07:17.652 INFO:tasks.rados.rados.0.trial115.stdout:3380: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:07:17.652 INFO:tasks.rados.rados.0.trial115.stdout:3381: rollback oid 30 current snap is 357 2026-02-17T06:07:17.652 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 30 to 356 2026-02-17T06:07:17.652 INFO:tasks.rados.rados.0.trial115.stdout:3377: finishing copy_from racing read to trial11545420-1 2026-02-17T06:07:17.652 INFO:tasks.rados.rados.0.trial115.stdout:3376: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:07:17.770 INFO:tasks.rados.rados.0.trial115.stdout:3378: finishing write tid 1 to trial11545420-11 2026-02-17T06:07:17.770 INFO:tasks.rados.rados.0.trial115.stdout:3378: finishing write tid 2 to trial11545420-11 2026-02-17T06:07:17.771 INFO:tasks.rados.rados.0.trial115.stdout:3376: done (5 left) 2026-02-17T06:07:17.771 INFO:tasks.rados.rados.0.trial115.stdout:3377: done (4 left) 2026-02-17T06:07:17.771 INFO:tasks.rados.rados.0.trial115.stdout:3382: snap_remove snap 347 2026-02-17T06:07:18.641 INFO:tasks.rados.rados.0.trial115.stdout:3382: done (4 left) 2026-02-17T06:07:18.641 INFO:tasks.rados.rados.0.trial115.stdout:3383: write oid 15 current snap is 357 2026-02-17T06:07:18.641 INFO:tasks.rados.rados.0.trial115.stdout:3379: finishing rollback tid 1 to trial11545420-4 2026-02-17T06:07:18.641 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1473 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:07:18.641 INFO:tasks.rados.rados.0.trial115.stdout:3381: finishing rollback tid 1 to trial11545420-30 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1461 (ObjNum 348 snap 0 seq_num 304) dirty dne 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: finishing write tid 3 to trial11545420-11 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: finishing write tid 4 to trial11545420-11 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: finishing write tid 5 to trial11545420-11 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: oid 11 updating version 0 to 1756 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: oid 11 updating version 1756 to 1757 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: oid 11 updating version 1757 to 1758 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: oid 11 updating version 1758 to 1759 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1759 (ObjNum 1163 snap 357 seq_num 1163) dirty exists 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3378: left oid 11 (ObjNum 1163 snap 357 seq_num 1163) 2026-02-17T06:07:18.642 INFO:tasks.rados.rados.0.trial115.stdout:3383: seq_num 1164 ranges {763348=591845,1914452=434061,2881893=674270} 2026-02-17T06:07:18.645 INFO:tasks.rados.rados.0.trial115.stdout:3383: writing trial11545420-15 from 763348 to 1355193 tid 1 2026-02-17T06:07:18.649 INFO:tasks.rados.rados.0.trial115.stdout:3383: writing trial11545420-15 from 1914452 to 2348513 tid 2 2026-02-17T06:07:18.655 INFO:tasks.rados.rados.0.trial115.stdout:3383: writing trial11545420-15 from 2881893 to 3556163 tid 3 2026-02-17T06:07:18.655 INFO:tasks.rados.rados.0.trial115.stdout:3378: done (4 left) 2026-02-17T06:07:18.655 INFO:tasks.rados.rados.0.trial115.stdout:3379: done (3 left) 2026-02-17T06:07:18.655 INFO:tasks.rados.rados.0.trial115.stdout:3381: done (2 left) 2026-02-17T06:07:18.655 INFO:tasks.rados.rados.0.trial115.stdout:3384: snap_create 2026-02-17T06:07:18.655 INFO:tasks.rados.rados.0.trial115.stdout:3380: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:07:18.800 INFO:tasks.rados.rados.0.trial115.stdout:3383: finishing write tid 1 to trial11545420-15 2026-02-17T06:07:18.800 INFO:tasks.rados.rados.0.trial115.stdout:3383: finishing write tid 2 to trial11545420-15 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:3383: finishing write tid 3 to trial11545420-15 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:3383: finishing write tid 4 to trial11545420-15 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:3383: finishing write tid 5 to trial11545420-15 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:3383: oid 15 updating version 0 to 1770 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:3383: oid 15 version 1770 is already newer than 1769 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:3383: oid 15 version 1770 is already newer than 1761 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:3383: oid 15 version 1770 is already newer than 1762 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1770 (ObjNum 1164 snap 357 seq_num 1164) dirty exists 2026-02-17T06:07:18.801 INFO:tasks.rados.rados.0.trial115.stdout:3383: left oid 15 (ObjNum 1164 snap 357 seq_num 1164) 2026-02-17T06:07:19.644 INFO:tasks.rados.rados.0.trial115.stdout:3380: done (2 left) 2026-02-17T06:07:19.644 INFO:tasks.rados.rados.0.trial115.stdout:3383: done (1 left) 2026-02-17T06:07:19.644 INFO:tasks.rados.rados.0.trial115.stdout:3384: done (0 left) 2026-02-17T06:07:19.644 INFO:tasks.rados.rados.0.trial115.stdout:3385: write oid 33 current snap is 358 2026-02-17T06:07:19.644 INFO:tasks.rados.rados.0.trial115.stdout:3385: seq_num 1165 ranges {780155=667094,1972934=694072,3309316=270839} 2026-02-17T06:07:19.649 INFO:tasks.rados.rados.0.trial115.stdout:3385: writing trial11545420-33 from 780155 to 1447249 tid 1 2026-02-17T06:07:19.654 INFO:tasks.rados.rados.0.trial115.stdout:3385: writing trial11545420-33 from 1972934 to 2667006 tid 2 2026-02-17T06:07:19.658 INFO:tasks.rados.rados.0.trial115.stdout:3385: writing trial11545420-33 from 3309316 to 3580155 tid 3 2026-02-17T06:07:19.658 INFO:tasks.rados.rados.0.trial115.stdout:3386: delete oid 9 current snap is 358 2026-02-17T06:07:19.658 INFO:tasks.rados.rados.0.trial115.stdout:3385: finishing write tid 1 to trial11545420-33 2026-02-17T06:07:19.658 INFO:tasks.rados.rados.0.trial115.stdout:3385: finishing write tid 2 to trial11545420-33 2026-02-17T06:07:19.659 INFO:tasks.rados.rados.0.trial115.stdout:3385: finishing write tid 3 to trial11545420-33 2026-02-17T06:07:19.660 INFO:tasks.rados.rados.0.trial115.stdout:3385: finishing write tid 4 to trial11545420-33 2026-02-17T06:07:19.660 INFO:tasks.rados.rados.0.trial115.stdout:3385: finishing write tid 5 to trial11545420-33 2026-02-17T06:07:19.660 INFO:tasks.rados.rados.0.trial115.stdout:3385: oid 33 updating version 0 to 1660 2026-02-17T06:07:19.660 INFO:tasks.rados.rados.0.trial115.stdout:3385: oid 33 updating version 1660 to 1662 2026-02-17T06:07:19.660 INFO:tasks.rados.rados.0.trial115.stdout:3385: oid 33 version 1662 is already newer than 1659 2026-02-17T06:07:19.660 INFO:tasks.rados.rados.0.trial115.stdout:3385: oid 33 version 1662 is already newer than 1661 2026-02-17T06:07:19.661 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1662 (ObjNum 1165 snap 358 seq_num 1165) dirty exists 2026-02-17T06:07:19.661 INFO:tasks.rados.rados.0.trial115.stdout:3385: left oid 33 (ObjNum 1165 snap 358 seq_num 1165) 2026-02-17T06:07:19.661 INFO:tasks.rados.rados.0.trial115.stdout:3385: done (1 left) 2026-02-17T06:07:19.661 INFO:tasks.rados.rados.0.trial115.stdout:3386: done (0 left) 2026-02-17T06:07:19.661 INFO:tasks.rados.rados.0.trial115.stdout:3387: rollback oid 32 current snap is 358 2026-02-17T06:07:19.661 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 32 to 354 2026-02-17T06:07:19.661 INFO:tasks.rados.rados.0.trial115.stdout:3388: snap_remove snap 340 2026-02-17T06:07:20.634 INFO:tasks.rados.rados.0.trial115.stdout:3388: done (1 left) 2026-02-17T06:07:20.635 INFO:tasks.rados.rados.0.trial115.stdout:3389: snap_remove snap 356 2026-02-17T06:07:21.637 INFO:tasks.rados.rados.0.trial115.stdout:3389: done (1 left) 2026-02-17T06:07:21.638 INFO:tasks.rados.rados.0.trial115.stdout:3390: delete oid 28 current snap is 358 2026-02-17T06:07:21.638 INFO:tasks.rados.rados.0.trial115.stdout:3387: finishing rollback tid 0 to trial11545420-32 2026-02-17T06:07:21.638 INFO:tasks.rados.rados.0.trial115.stdout:3387: finishing rollback tid 1 to trial11545420-32 2026-02-17T06:07:21.638 INFO:tasks.rados.rados.0.trial115.stdout:3387: finishing rollback tid 2 to trial11545420-32 2026-02-17T06:07:21.638 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1464 (ObjNum 918 snap 269 seq_num 918) dirty exists 2026-02-17T06:07:21.641 INFO:tasks.rados.rados.0.trial115.stdout:3387: done (1 left) 2026-02-17T06:07:21.641 INFO:tasks.rados.rados.0.trial115.stdout:3390: done (0 left) 2026-02-17T06:07:21.641 INFO:tasks.rados.rados.0.trial115.stdout:3391: snap_create 2026-02-17T06:07:22.639 INFO:tasks.rados.rados.0.trial115.stdout:3391: done (0 left) 2026-02-17T06:07:22.639 INFO:tasks.rados.rados.0.trial115.stdout:3392: read oid 7 snap -1 2026-02-17T06:07:22.639 INFO:tasks.rados.rados.0.trial115.stdout:3392: expect (ObjNum 1150 snap 353 seq_num 1150) 2026-02-17T06:07:22.639 INFO:tasks.rados.rados.0.trial115.stdout:3393: write (excl) oid 16 current snap is 359 2026-02-17T06:07:22.639 INFO:tasks.rados.rados.0.trial115.stdout:3393: seq_num 1168 ranges {430576=492841,1632027=441399,2475232=435711,3365397=265179} 2026-02-17T06:07:22.642 INFO:tasks.rados.rados.0.trial115.stdout:3393: writing trial11545420-16 from 430576 to 923417 tid 1 2026-02-17T06:07:22.646 INFO:tasks.rados.rados.0.trial115.stdout:3393: writing trial11545420-16 from 1632027 to 2073426 tid 2 2026-02-17T06:07:22.650 INFO:tasks.rados.rados.0.trial115.stdout:3393: writing trial11545420-16 from 2475232 to 2910943 tid 3 2026-02-17T06:07:22.652 INFO:tasks.rados.rados.0.trial115.stdout:3393: writing trial11545420-16 from 3365397 to 3630576 tid 4 2026-02-17T06:07:22.653 INFO:tasks.rados.rados.0.trial115.stdout:3394: snap_create 2026-02-17T06:07:22.653 INFO:tasks.rados.rados.0.trial115.stdout:3392: expect (ObjNum 1150 snap 353 seq_num 1150) 2026-02-17T06:07:22.780 INFO:tasks.rados.rados.0.trial115.stdout:3393: finishing write tid 1 to trial11545420-16 2026-02-17T06:07:22.780 INFO:tasks.rados.rados.0.trial115.stdout:3393: finishing write tid 2 to trial11545420-16 2026-02-17T06:07:22.780 INFO:tasks.rados.rados.0.trial115.stdout:3393: finishing write tid 3 to trial11545420-16 2026-02-17T06:07:22.780 INFO:tasks.rados.rados.0.trial115.stdout:3393: finishing write tid 4 to trial11545420-16 2026-02-17T06:07:22.780 INFO:tasks.rados.rados.0.trial115.stdout:3393: finishing write tid 5 to trial11545420-16 2026-02-17T06:07:22.780 INFO:tasks.rados.rados.0.trial115.stdout:3393: finishing write tid 6 to trial11545420-16 2026-02-17T06:07:22.781 INFO:tasks.rados.rados.0.trial115.stdout:3393: oid 16 updating version 0 to 1800 2026-02-17T06:07:22.781 INFO:tasks.rados.rados.0.trial115.stdout:3393: oid 16 updating version 1800 to 1803 2026-02-17T06:07:22.781 INFO:tasks.rados.rados.0.trial115.stdout:3393: oid 16 version 1803 is already newer than 1802 2026-02-17T06:07:22.781 INFO:tasks.rados.rados.0.trial115.stdout:3393: oid 16 version 1803 is already newer than 1801 2026-02-17T06:07:22.781 INFO:tasks.rados.rados.0.trial115.stdout:3393: oid 16 updating version 1803 to 1804 2026-02-17T06:07:22.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1804 (ObjNum 1168 snap 359 seq_num 1168) dirty exists 2026-02-17T06:07:22.781 INFO:tasks.rados.rados.0.trial115.stdout:3393: left oid 16 (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:07:23.642 INFO:tasks.rados.rados.0.trial115.stdout:3392: done (2 left) 2026-02-17T06:07:23.643 INFO:tasks.rados.rados.0.trial115.stdout:3393: done (1 left) 2026-02-17T06:07:23.643 INFO:tasks.rados.rados.0.trial115.stdout:3394: done (0 left) 2026-02-17T06:07:23.643 INFO:tasks.rados.rados.0.trial115.stdout:3395: write oid 38 current snap is 360 2026-02-17T06:07:23.643 INFO:tasks.rados.rados.0.trial115.stdout:3395: seq_num 1169 ranges {447383=568090,1690509=701410,2902655=646983} 2026-02-17T06:07:23.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: writing trial11545420-38 from 447383 to 1015473 tid 1 2026-02-17T06:07:23.652 INFO:tasks.rados.rados.0.trial115.stdout:3395: writing trial11545420-38 from 1690509 to 2391919 tid 2 2026-02-17T06:07:23.656 INFO:tasks.rados.rados.0.trial115.stdout:3395: writing trial11545420-38 from 2902655 to 3549638 tid 3 2026-02-17T06:07:23.656 INFO:tasks.rados.rados.0.trial115.stdout:3396: snap_remove snap 355 2026-02-17T06:07:24.645 INFO:tasks.rados.rados.0.trial115.stdout:3396: done (1 left) 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3397: rollback oid 1 current snap is 360 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 1 to 353 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: finishing write tid 1 to trial11545420-38 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: finishing write tid 2 to trial11545420-38 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: finishing write tid 3 to trial11545420-38 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: finishing write tid 4 to trial11545420-38 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: finishing write tid 5 to trial11545420-38 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: oid 38 updating version 0 to 1808 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: oid 38 version 1808 is already newer than 1806 2026-02-17T06:07:24.646 INFO:tasks.rados.rados.0.trial115.stdout:3395: oid 38 version 1808 is already newer than 1807 2026-02-17T06:07:24.647 INFO:tasks.rados.rados.0.trial115.stdout:3395: oid 38 version 1808 is already newer than 1805 2026-02-17T06:07:24.647 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1808 (ObjNum 1169 snap 360 seq_num 1169) dirty exists 2026-02-17T06:07:24.647 INFO:tasks.rados.rados.0.trial115.stdout:3395: left oid 38 (ObjNum 1169 snap 360 seq_num 1169) 2026-02-17T06:07:24.647 INFO:tasks.rados.rados.0.trial115.stdout:3395: done (1 left) 2026-02-17T06:07:24.647 INFO:tasks.rados.rados.0.trial115.stdout:3398: write (excl) oid 23 current snap is 360 2026-02-17T06:07:24.647 INFO:tasks.rados.rados.0.trial115.stdout:3398: seq_num 1170 ranges {464190=643339,1748991=445068,2930078=574608} 2026-02-17T06:07:24.649 INFO:tasks.rados.rados.0.trial115.stdout:3398: writing trial11545420-23 from 464190 to 1107529 tid 1 2026-02-17T06:07:24.653 INFO:tasks.rados.rados.0.trial115.stdout:3398: writing trial11545420-23 from 1748991 to 2194059 tid 2 2026-02-17T06:07:24.658 INFO:tasks.rados.rados.0.trial115.stdout:3398: writing trial11545420-23 from 2930078 to 3504686 tid 3 2026-02-17T06:07:24.658 INFO:tasks.rados.rados.0.trial115.stdout:3399: read oid 11 snap 344 2026-02-17T06:07:24.658 INFO:tasks.rados.rados.0.trial115.stdout:3399: expect (ObjNum 1121 snap 336 seq_num 1121) 2026-02-17T06:07:24.658 INFO:tasks.rados.rados.0.trial115.stdout:3400: rollback oid 45 current snap is 360 2026-02-17T06:07:24.658 INFO:tasks.rados.rados.0.trial115.stdout:3397: finishing rollback tid 1 to trial11545420-1 2026-02-17T06:07:24.659 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:07:24.659 INFO:tasks.rados.rados.0.trial115.stdout:3398: finishing write tid 1 to trial11545420-23 2026-02-17T06:07:24.659 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 357 2026-02-17T06:07:24.659 INFO:tasks.rados.rados.0.trial115.stdout:3398: finishing write tid 2 to trial11545420-23 2026-02-17T06:07:24.659 INFO:tasks.rados.rados.0.trial115.stdout:3397: done (3 left) 2026-02-17T06:07:24.659 INFO:tasks.rados.rados.0.trial115.stdout:3401: snap_remove snap 358 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3401: done (3 left) 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3402: copy_from oid 37 from oid 26 current snap is 360 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3398: finishing write tid 3 to trial11545420-23 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3398: finishing write tid 4 to trial11545420-23 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3398: finishing write tid 5 to trial11545420-23 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3398: oid 23 updating version 0 to 1751 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3398: oid 23 version 1751 is already newer than 1750 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3398: oid 23 updating version 1751 to 1753 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3398: oid 23 version 1753 is already newer than 1752 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1753 (ObjNum 1170 snap 360 seq_num 1170) dirty exists 2026-02-17T06:07:25.637 INFO:tasks.rados.rados.0.trial115.stdout:3398: left oid 23 (ObjNum 1170 snap 360 seq_num 1170) 2026-02-17T06:07:25.638 INFO:tasks.rados.rados.0.trial115.stdout:3400: finishing rollback tid 0 to trial11545420-45 2026-02-17T06:07:25.638 INFO:tasks.rados.rados.0.trial115.stdout:3400: finishing rollback tid 1 to trial11545420-45 2026-02-17T06:07:25.638 INFO:tasks.rados.rados.0.trial115.stdout:3400: finishing rollback tid 2 to trial11545420-45 2026-02-17T06:07:25.638 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1771 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:07:25.638 INFO:tasks.rados.rados.0.trial115.stdout:3399: expect (ObjNum 1121 snap 336 seq_num 1121) 2026-02-17T06:07:25.792 INFO:tasks.rados.rados.0.trial115.stdout:3402: finishing copy_from racing read to trial11545420-37 2026-02-17T06:07:25.792 INFO:tasks.rados.rados.0.trial115.stdout:3402: finishing copy_from to trial11545420-37 2026-02-17T06:07:25.792 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1461 (ObjNum 1127 snap 336 seq_num 1127) dirty exists 2026-02-17T06:07:25.792 INFO:tasks.rados.rados.0.trial115.stdout:3398: done (3 left) 2026-02-17T06:07:25.792 INFO:tasks.rados.rados.0.trial115.stdout:3399: done (2 left) 2026-02-17T06:07:25.792 INFO:tasks.rados.rados.0.trial115.stdout:3400: done (1 left) 2026-02-17T06:07:25.792 INFO:tasks.rados.rados.0.trial115.stdout:3402: done (0 left) 2026-02-17T06:07:25.793 INFO:tasks.rados.rados.0.trial115.stdout:3403: snap_remove snap 357 2026-02-17T06:07:26.649 INFO:tasks.rados.rados.0.trial115.stdout:3403: done (0 left) 2026-02-17T06:07:26.649 INFO:tasks.rados.rados.0.trial115.stdout:3404: snap_remove snap 344 2026-02-17T06:07:27.649 INFO:tasks.rados.rados.0.trial115.stdout:3404: done (0 left) 2026-02-17T06:07:27.649 INFO:tasks.rados.rados.0.trial115.stdout:3405: write (excl) oid 12 current snap is 360 2026-02-17T06:07:27.650 INFO:tasks.rados.rados.0.trial115.stdout:3405: seq_num 1171 ranges {480997=434941,1407473=705079,2557501=785880} 2026-02-17T06:07:27.653 INFO:tasks.rados.rados.0.trial115.stdout:3405: writing trial11545420-12 from 480997 to 915938 tid 1 2026-02-17T06:07:27.658 INFO:tasks.rados.rados.0.trial115.stdout:3405: writing trial11545420-12 from 1407473 to 2112552 tid 2 2026-02-17T06:07:27.663 INFO:tasks.rados.rados.0.trial115.stdout:3405: writing trial11545420-12 from 2557501 to 3343381 tid 3 2026-02-17T06:07:27.663 INFO:tasks.rados.rados.0.trial115.stdout:3406: delete oid 14 current snap is 360 2026-02-17T06:07:27.663 INFO:tasks.rados.rados.0.trial115.stdout:3405: finishing write tid 1 to trial11545420-12 2026-02-17T06:07:27.663 INFO:tasks.rados.rados.0.trial115.stdout:3405: finishing write tid 2 to trial11545420-12 2026-02-17T06:07:27.664 INFO:tasks.rados.rados.0.trial115.stdout:3406: done (1 left) 2026-02-17T06:07:27.664 INFO:tasks.rados.rados.0.trial115.stdout:3407: snap_create 2026-02-17T06:07:27.665 INFO:tasks.rados.rados.0.trial115.stdout:3405: finishing write tid 3 to trial11545420-12 2026-02-17T06:07:27.665 INFO:tasks.rados.rados.0.trial115.stdout:3405: finishing write tid 4 to trial11545420-12 2026-02-17T06:07:27.665 INFO:tasks.rados.rados.0.trial115.stdout:3405: finishing write tid 5 to trial11545420-12 2026-02-17T06:07:27.665 INFO:tasks.rados.rados.0.trial115.stdout:3405: oid 12 updating version 0 to 1777 2026-02-17T06:07:27.665 INFO:tasks.rados.rados.0.trial115.stdout:3405: oid 12 updating version 1777 to 1786 2026-02-17T06:07:27.665 INFO:tasks.rados.rados.0.trial115.stdout:3405: oid 12 updating version 1786 to 1788 2026-02-17T06:07:27.666 INFO:tasks.rados.rados.0.trial115.stdout:3405: oid 12 version 1788 is already newer than 1787 2026-02-17T06:07:27.666 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1788 (ObjNum 1171 snap 360 seq_num 1171) dirty exists 2026-02-17T06:07:27.666 INFO:tasks.rados.rados.0.trial115.stdout:3405: left oid 12 (ObjNum 1171 snap 360 seq_num 1171) 2026-02-17T06:07:28.653 INFO:tasks.rados.rados.0.trial115.stdout:3405: done (1 left) 2026-02-17T06:07:28.654 INFO:tasks.rados.rados.0.trial115.stdout:3407: done (0 left) 2026-02-17T06:07:28.654 INFO:tasks.rados.rados.0.trial115.stdout:3408: read oid 20 snap -1 2026-02-17T06:07:28.654 INFO:tasks.rados.rados.0.trial115.stdout:3408: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:07:28.654 INFO:tasks.rados.rados.0.trial115.stdout:3409: write (excl) oid 32 current snap is 361 2026-02-17T06:07:28.654 INFO:tasks.rados.rados.0.trial115.stdout:3409: seq_num 1173 ranges {514611=585439,1524437=592395,2895994=408424} 2026-02-17T06:07:28.658 INFO:tasks.rados.rados.0.trial115.stdout:3409: writing trial11545420-32 from 514611 to 1100050 tid 1 2026-02-17T06:07:28.662 INFO:tasks.rados.rados.0.trial115.stdout:3409: writing trial11545420-32 from 1524437 to 2116832 tid 2 2026-02-17T06:07:28.667 INFO:tasks.rados.rados.0.trial115.stdout:3409: writing trial11545420-32 from 2895994 to 3304418 tid 3 2026-02-17T06:07:28.667 INFO:tasks.rados.rados.0.trial115.stdout:3410: write (excl) oid 49 current snap is 361 2026-02-17T06:07:28.667 INFO:tasks.rados.rados.0.trial115.stdout:3410: seq_num 1174 ranges {531418=660688,1982919=452406,3039770=691648} 2026-02-17T06:07:28.671 INFO:tasks.rados.rados.0.trial115.stdout:3410: writing trial11545420-49 from 531418 to 1192106 tid 1 2026-02-17T06:07:28.676 INFO:tasks.rados.rados.0.trial115.stdout:3410: writing trial11545420-49 from 1982919 to 2435325 tid 2 2026-02-17T06:07:28.681 INFO:tasks.rados.rados.0.trial115.stdout:3410: writing trial11545420-49 from 3039770 to 3731418 tid 3 2026-02-17T06:07:28.681 INFO:tasks.rados.rados.0.trial115.stdout:3411: delete oid 3 current snap is 361 2026-02-17T06:07:28.681 INFO:tasks.rados.rados.0.trial115.stdout:3408: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:07:28.827 INFO:tasks.rados.rados.0.trial115.stdout:3409: finishing write tid 1 to trial11545420-32 2026-02-17T06:07:28.827 INFO:tasks.rados.rados.0.trial115.stdout:3409: finishing write tid 2 to trial11545420-32 2026-02-17T06:07:28.827 INFO:tasks.rados.rados.0.trial115.stdout:3409: finishing write tid 3 to trial11545420-32 2026-02-17T06:07:28.827 INFO:tasks.rados.rados.0.trial115.stdout:3409: finishing write tid 4 to trial11545420-32 2026-02-17T06:07:28.828 INFO:tasks.rados.rados.0.trial115.stdout:3409: finishing write tid 5 to trial11545420-32 2026-02-17T06:07:28.828 INFO:tasks.rados.rados.0.trial115.stdout:3409: oid 32 updating version 0 to 1466 2026-02-17T06:07:28.828 INFO:tasks.rados.rados.0.trial115.stdout:3409: oid 32 updating version 1466 to 1467 2026-02-17T06:07:28.828 INFO:tasks.rados.rados.0.trial115.stdout:3409: oid 32 updating version 1467 to 1468 2026-02-17T06:07:28.828 INFO:tasks.rados.rados.0.trial115.stdout:3409: oid 32 version 1468 is already newer than 1465 2026-02-17T06:07:28.829 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1468 (ObjNum 1173 snap 361 seq_num 1173) dirty exists 2026-02-17T06:07:28.829 INFO:tasks.rados.rados.0.trial115.stdout:3409: left oid 32 (ObjNum 1173 snap 361 seq_num 1173) 2026-02-17T06:07:28.829 INFO:tasks.rados.rados.0.trial115.stdout:3410: finishing write tid 1 to trial11545420-49 2026-02-17T06:07:28.829 INFO:tasks.rados.rados.0.trial115.stdout:3410: finishing write tid 2 to trial11545420-49 2026-02-17T06:07:28.829 INFO:tasks.rados.rados.0.trial115.stdout:3408: done (3 left) 2026-02-17T06:07:28.829 INFO:tasks.rados.rados.0.trial115.stdout:3409: done (2 left) 2026-02-17T06:07:28.829 INFO:tasks.rados.rados.0.trial115.stdout:3411: done (1 left) 2026-02-17T06:07:28.830 INFO:tasks.rados.rados.0.trial115.stdout:3412: snap_remove snap 352 2026-02-17T06:07:29.656 INFO:tasks.rados.rados.0.trial115.stdout:3412: done (1 left) 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3413: rollback oid 22 current snap is 361 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 22 to 341 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3410: finishing write tid 3 to trial11545420-49 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3410: finishing write tid 4 to trial11545420-49 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3410: finishing write tid 5 to trial11545420-49 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3410: oid 49 updating version 0 to 1477 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3410: oid 49 version 1477 is already newer than 1476 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3410: oid 49 version 1477 is already newer than 1475 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3410: oid 49 version 1477 is already newer than 1474 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1477 (ObjNum 1174 snap 361 seq_num 1174) dirty exists 2026-02-17T06:07:29.657 INFO:tasks.rados.rados.0.trial115.stdout:3410: left oid 49 (ObjNum 1174 snap 361 seq_num 1174) 2026-02-17T06:07:29.658 INFO:tasks.rados.rados.0.trial115.stdout:3410: done (1 left) 2026-02-17T06:07:29.658 INFO:tasks.rados.rados.0.trial115.stdout:3414: read oid 11 snap -1 2026-02-17T06:07:29.658 INFO:tasks.rados.rados.0.trial115.stdout:3414: expect (ObjNum 1163 snap 357 seq_num 1163) 2026-02-17T06:07:29.658 INFO:tasks.rados.rados.0.trial115.stdout:3415: write oid 7 current snap is 361 2026-02-17T06:07:29.658 INFO:tasks.rados.rados.0.trial115.stdout:3415: seq_num 1176 ranges {565032=411186,1583530=456075,2578263=758593} 2026-02-17T06:07:29.659 INFO:tasks.rados.rados.0.trial115.stdout:3415: writing trial11545420-7 from 565032 to 976218 tid 1 2026-02-17T06:07:29.663 INFO:tasks.rados.rados.0.trial115.stdout:3415: writing trial11545420-7 from 1583530 to 2039605 tid 2 2026-02-17T06:07:29.668 INFO:tasks.rados.rados.0.trial115.stdout:3415: writing trial11545420-7 from 2578263 to 3336856 tid 3 2026-02-17T06:07:29.669 INFO:tasks.rados.rados.0.trial115.stdout:3416: snap_remove snap 359 2026-02-17T06:07:30.552 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-17T06:07:30.552 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-17T06:07:30.552 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-17T06:07:30.552 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-17T06:07:30.555 INFO:tasks.ceph.mon.c:Started 2026-02-17T06:07:30.555 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:07:30.555 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.623+0000 7fa5ad66f640 10 monclient: get_monmap_and_config 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.623+0000 7fa5ad66f640 10 monclient: build_initial_monmap 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.623+0000 7fa5ad66f640 10 monclient: monmap: 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:30.624548+0000 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:30.624548+0000 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:30.624 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.623+0000 7fa5ad66f640 1 Processor -- start 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.623+0000 7fa5ad66f640 1 -- start start 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.623+0000 7fa5ad66f640 10 monclient: init 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa5a806c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa5a809c4a0 0x7fa5a813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 10 monclient(hunting): picked mon.noname-c con 0x7fa5a809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa5a806d380 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a806daf0 con 0x7fa5a806c890 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a809cc80 con 0x7fa5a806d380 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a80a0880 con 0x7fa5a809c4a0 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5ad66f640 10 monclient(hunting): authenticate will time out at 1896.178088s 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a6ffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fa5a806c890 0x7fa5a806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52568/0 (socket says 10.20.193.78:52568) 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a6ffd640 1 -- 10.20.193.78:0/1169129248 learned_addr learned my addr 10.20.193.78:0/1169129248 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 1 --2- 10.20.193.78:0/1169129248 >> v2:10.20.193.115:3300/0 conn(0x7fa5a809c4a0 0x7fa5a813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient(hunting): get_auth_request con 0x7fa5a809c4a0 auth_method 0 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient(hunting): handle_auth_done global_id 214441 payload 274 2026-02-17T06:07:30.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 1 -- 10.20.193.78:0/1169129248 >> v1:10.20.193.115:6789/0 conn(0x7fa5a806d380 legacy=0x7fa5a813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:30.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 1 -- 10.20.193.78:0/1169129248 >> v1:10.20.193.78:6789/0 conn(0x7fa5a806c890 legacy=0x7fa5a806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:30.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient: _finish_hunting 0 2026-02-17T06:07:30.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 1 monclient: found mon.noname-c 2026-02-17T06:07:30.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:30.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:07:30.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.624+0000 7fa5a67fc640 1 -- 10.20.193.78:0/1169129248 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a809cf60 con 0x7fa5a809c4a0 2026-02-17T06:07:30.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a67fc640 1 -- 10.20.193.78:0/1169129248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a809c4a0 msgr2=0x7fa5a813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:30.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a67fc640 1 --2- 10.20.193.78:0/1169129248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a809c4a0 0x7fa5a813b3a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa5a809c980 tx=0x7fa59002ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=903a58a60c3cf178 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 1 -- 10.20.193.78:0/1169129248 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59003c070 con 0x7fa5a809c4a0 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: dump: 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: _finish_auth 0 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 5 monclient: authenticate success, global_id 214441 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 1 -- 10.20.193.78:0/1169129248 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59002fcb0 con 0x7fa5a809c4a0 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:30.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 1 -- 10.20.193.78:0/1169129248 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59002fe60 con 0x7fa5a809c4a0 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 4 set_mon_vals no callback set 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5a5ffb640 10 monclient: dump: 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 10 monclient: shutdown 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1169129248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a809c4a0 msgr2=0x7fa5a813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 1 --2- 10.20.193.78:0/1169129248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a809c4a0 0x7fa5a813b3a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa5a809c980 tx=0x7fa59002ed50 comp rx=0 tx=0).stop 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1169129248 shutdown_connections 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 1 --2- 10.20.193.78:0/1169129248 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a809c4a0 0x7fa5a813b3a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:30.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1169129248 >> 10.20.193.78:0/1169129248 conn(0x7fa5a80a6620 msgr2=0x7fa5a80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1169129248 shutdown_connections 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.627+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1169129248 wait complete. 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient: build_initial_monmap 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient: monmap: 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 1 Processor -- start 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 1 -- start start 2026-02-17T06:07:30.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient: init 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 0x7fa5a813d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient(hunting): picked mon.noname-c con 0x7fa5a806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a806d380 0x7fa5a81cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient(hunting): picked mon.noname-b con 0x7fa5a806d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5a809c4a0 0x7fa5a81cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient(hunting): picked mon.noname-a con 0x7fa5a809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a809bfa0 con 0x7fa5a809c4a0 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a809be20 con 0x7fa5a806d380 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a813f010 con 0x7fa5a806c890 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5ad66f640 10 monclient(hunting): authenticate will time out at 1896.182823s 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 0x7fa5a813d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5a809c4a0 0x7fa5a81cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 0x7fa5a813d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54368/0 (socket says 10.20.193.78:54368) 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 1 -- 10.20.193.78:0/1741648684 learned_addr learned my addr 10.20.193.78:0/1741648684 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 10 monclient(hunting): get_auth_request con 0x7fa5a806c890 auth_method 0 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a77fe640 10 monclient(hunting): get_auth_request con 0x7fa5a809c4a0 auth_method 0 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 1 -- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 msgr2=0x7fa5a813d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 1 -- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 msgr2=0x7fa5a813d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 0x7fa5a813d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a6ffd640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 0x7fa5a813d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.628+0000 7fa5a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a67fc640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a806d380 0x7fa5a81cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a67fc640 10 monclient(hunting): get_auth_request con 0x7fa5a806d380 auth_method 0 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 10 monclient(hunting): handle_auth_done global_id 254181 payload 824 2026-02-17T06:07:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 1 -- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 msgr2=0x7fa5a813d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:30.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 0x7fa5a813d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:30.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 1 -- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a806d380 msgr2=0x7fa5a81cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:30.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a806d380 0x7fa5a81cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:30.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 10 monclient: _finish_hunting 0 2026-02-17T06:07:30.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 1 monclient: found mon.noname-a 2026-02-17T06:07:30.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:30.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:30.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.629+0000 7fa5a77fe640 1 -- 10.20.193.78:0/1741648684 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a8121990 con 0x7fa5a809c4a0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5a77fe640 1 -- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5a809c4a0 msgr2=0x7fa5a81cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5a77fe640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5a809c4a0 0x7fa5a81cf7f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa59800cb20 tx=0x7fa598004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 1 -- 10.20.193.78:0/1741648684 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598016030 con 0x7fa5a809c4a0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: dump: 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: _finish_auth 0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 1 -- 10.20.193.78:0/1741648684 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59800ac50 con 0x7fa5a809c4a0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 1 -- 10.20.193.78:0/1741648684 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598004a80 con 0x7fa5a809c4a0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 5 monclient: authenticate success, global_id 254181 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 10 monclient: _renew_subs 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5a57fa640 4 set_mon_vals no callback set 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a8121c70 con 0x7fa5a809c4a0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5877fe640 10 monclient: dump: 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:30.633 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 10 monclient: _renew_subs 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a809f100 con 0x7fa5a809c4a0 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.631+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a8121210 con 0x7fa5a809c4a0 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.632+0000 7fa5877fe640 1 -- 10.20.193.78:0/1741648684 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa598023060 con 0x7fa5a809c4a0 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.632+0000 7fa5877fe640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa57403deb0 0x7fa574040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.632+0000 7fa5877fe640 1 -- 10.20.193.78:0/1741648684 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(747..747 src has 525..747) ==== 4811+0+0 (secure 0 0 0) 0x7fa598052aa0 con 0x7fa5a809c4a0 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.632+0000 7fa5a6ffd640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa57403deb0 0x7fa574040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:30.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.632+0000 7fa5a6ffd640 10 monclient: get_auth_request con 0x7fa57403deb0 auth_method 0 2026-02-17T06:07:30.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.634+0000 7fa5877fe640 1 -- 10.20.193.78:0/1741648684 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa59801b410 con 0x7fa5a809c4a0 2026-02-17T06:07:30.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.634+0000 7fa5877fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:30.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.634+0000 7fa5877fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:30.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.635+0000 7fa5a6ffd640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa57403deb0 0x7fa574040370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa59c007f00 tx=0x7fa59c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:30.658 INFO:tasks.rados.rados.0.trial115.stdout:3416: done (3 left) 2026-02-17T06:07:30.659 INFO:tasks.rados.rados.0.trial115.stdout:3417: write (excl) oid 4 current snap is 361 2026-02-17T06:07:30.659 INFO:tasks.rados.rados.0.trial115.stdout:3417: seq_num 1177 ranges {581839=486435,1642012=599733,3005686=569865} 2026-02-17T06:07:30.663 INFO:tasks.rados.rados.0.trial115.stdout:3417: writing trial11545420-4 from 581839 to 1068274 tid 1 2026-02-17T06:07:30.667 INFO:tasks.rados.rados.0.trial115.stdout:3417: writing trial11545420-4 from 1642012 to 2241745 tid 2 2026-02-17T06:07:30.673 INFO:tasks.rados.rados.0.trial115.stdout:3417: writing trial11545420-4 from 3005686 to 3575551 tid 3 2026-02-17T06:07:30.673 INFO:tasks.rados.rados.0.trial115.stdout:3418: snap_create 2026-02-17T06:07:30.673 INFO:tasks.rados.rados.0.trial115.stdout:3414: expect (ObjNum 1163 snap 357 seq_num 1163) 2026-02-17T06:07:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.750+0000 7fa5ad66f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.750+0000 7fa5ad66f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.750+0000 7fa5ad66f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.750+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa5a806cc70 con 0x7fa5a809c4a0 2026-02-17T06:07:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.750+0000 7fa5877fe640 1 -- 10.20.193.78:0/1741648684 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fa598019360 con 0x7fa5a809c4a0 2026-02-17T06:07:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.750+0000 7fa5877fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.750+0000 7fa5877fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:30.753 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":244,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa57403deb0 msgr2=0x7fa574040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa57403deb0 0x7fa574040370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa59c007f00 tx=0x7fa59c009040 comp rx=0 tx=0).stop 2026-02-17T06:07:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 10 monclient: shutdown 2026-02-17T06:07:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5a809c4a0 msgr2=0x7fa5a81cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5a809c4a0 0x7fa5a81cf7f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa59800cb20 tx=0x7fa598004a10 comp rx=0 tx=0).stop 2026-02-17T06:07:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 shutdown_connections 2026-02-17T06:07:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa5a809c4a0 0x7fa5a81cf7f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa57403deb0 0x7fa574040370 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa5a806d380 0x7fa5a81cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.751+0000 7fa5ad66f640 1 --2- 10.20.193.78:0/1741648684 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa5a806c890 0x7fa5a813d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.752+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 >> 10.20.193.78:0/1741648684 conn(0x7fa5a80a6620 msgr2=0x7fa5a80a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.752+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 shutdown_connections 2026-02-17T06:07:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:30.752+0000 7fa5ad66f640 1 -- 10.20.193.78:0/1741648684 wait complete. 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3413: finishing rollback tid 0 to trial11545420-22 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3413: finishing rollback tid 1 to trial11545420-22 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3413: finishing rollback tid 2 to trial11545420-22 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1464 (ObjNum 1050 snap 312 seq_num 1050) dirty exists 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: finishing write tid 1 to trial11545420-7 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: finishing write tid 2 to trial11545420-7 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: finishing write tid 3 to trial11545420-7 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: finishing write tid 4 to trial11545420-7 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: finishing write tid 5 to trial11545420-7 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: oid 7 updating version 0 to 1695 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: oid 7 version 1695 is already newer than 1692 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: oid 7 version 1695 is already newer than 1694 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: oid 7 version 1695 is already newer than 1693 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1695 (ObjNum 1176 snap 361 seq_num 1176) dirty exists 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3415: left oid 7 (ObjNum 1176 snap 361 seq_num 1176) 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3417: finishing write tid 1 to trial11545420-4 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3417: finishing write tid 2 to trial11545420-4 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3417: finishing write tid 3 to trial11545420-4 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3417: finishing write tid 4 to trial11545420-4 2026-02-17T06:07:30.886 INFO:tasks.rados.rados.0.trial115.stdout:3417: finishing write tid 5 to trial11545420-4 2026-02-17T06:07:30.887 INFO:tasks.rados.rados.0.trial115.stdout:3417: oid 4 updating version 0 to 1479 2026-02-17T06:07:30.887 INFO:tasks.rados.rados.0.trial115.stdout:3417: oid 4 updating version 1479 to 1481 2026-02-17T06:07:30.887 INFO:tasks.rados.rados.0.trial115.stdout:3417: oid 4 version 1481 is already newer than 1478 2026-02-17T06:07:30.887 INFO:tasks.rados.rados.0.trial115.stdout:3417: oid 4 version 1481 is already newer than 1480 2026-02-17T06:07:30.887 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1481 (ObjNum 1177 snap 361 seq_num 1177) dirty exists 2026-02-17T06:07:30.887 INFO:tasks.rados.rados.0.trial115.stdout:3417: left oid 4 (ObjNum 1177 snap 361 seq_num 1177) 2026-02-17T06:07:31.662 INFO:tasks.rados.rados.0.trial115.stdout:3413: done (4 left) 2026-02-17T06:07:31.662 INFO:tasks.rados.rados.0.trial115.stdout:3414: done (3 left) 2026-02-17T06:07:31.663 INFO:tasks.rados.rados.0.trial115.stdout:3415: done (2 left) 2026-02-17T06:07:31.663 INFO:tasks.rados.rados.0.trial115.stdout:3417: done (1 left) 2026-02-17T06:07:31.663 INFO:tasks.rados.rados.0.trial115.stdout:3418: done (0 left) 2026-02-17T06:07:31.663 INFO:tasks.rados.rados.0.trial115.stdout:3419: read oid 35 snap -1 2026-02-17T06:07:31.663 INFO:tasks.rados.rados.0.trial115.stdout:3419: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:07:31.663 INFO:tasks.rados.rados.0.trial115.stdout:3420: read oid 33 snap -1 2026-02-17T06:07:31.663 INFO:tasks.rados.rados.0.trial115.stdout:3420: expect (ObjNum 1165 snap 358 seq_num 1165) 2026-02-17T06:07:31.663 INFO:tasks.rados.rados.0.trial115.stdout:3421: delete oid 21 current snap is 362 2026-02-17T06:07:31.665 INFO:tasks.rados.rados.0.trial115.stdout:3421: done (2 left) 2026-02-17T06:07:31.665 INFO:tasks.rados.rados.0.trial115.stdout:3422: copy_from oid 44 from oid 42 current snap is 362 2026-02-17T06:07:31.665 INFO:tasks.rados.rados.0.trial115.stdout:3423: write oid 23 current snap is 362 2026-02-17T06:07:31.665 INFO:tasks.rados.rados.0.trial115.stdout:3423: seq_num 1179 ranges {615453=753286,2158976=603402,3460532=354921} 2026-02-17T06:07:31.670 INFO:tasks.rados.rados.0.trial115.stdout:3423: writing trial11545420-23 from 615453 to 1368739 tid 1 2026-02-17T06:07:31.675 INFO:tasks.rados.rados.0.trial115.stdout:3423: writing trial11545420-23 from 2158976 to 2762378 tid 2 2026-02-17T06:07:31.679 INFO:tasks.rados.rados.0.trial115.stdout:3423: writing trial11545420-23 from 3460532 to 3815453 tid 3 2026-02-17T06:07:31.679 INFO:tasks.rados.rados.0.trial115.stdout:3424: write oid 37 current snap is 362 2026-02-17T06:07:31.679 INFO:tasks.rados.rados.0.trial115.stdout:3424: seq_num 1180 ranges {632260=428535,1817458=463413,2687955=520034,3709324=122936} 2026-02-17T06:07:31.683 INFO:tasks.rados.rados.0.trial115.stdout:3424: writing trial11545420-37 from 632260 to 1060795 tid 1 2026-02-17T06:07:31.688 INFO:tasks.rados.rados.0.trial115.stdout:3424: writing trial11545420-37 from 1817458 to 2280871 tid 2 2026-02-17T06:07:31.691 INFO:tasks.rados.rados.0.trial115.stdout:3424: writing trial11545420-37 from 2687955 to 3207989 tid 3 2026-02-17T06:07:31.694 INFO:tasks.rados.rados.0.trial115.stdout:3424: writing trial11545420-37 from 3709324 to 3832260 tid 4 2026-02-17T06:07:31.694 INFO:tasks.rados.rados.0.trial115.stdout:3425: delete oid 36 current snap is 362 2026-02-17T06:07:31.694 INFO:tasks.rados.rados.0.trial115.stdout:3422: finishing copy_from to trial11545420-44 2026-02-17T06:07:31.694 INFO:tasks.rados.rados.0.trial115.stdout:3422: got expected ENOENT (src dne) 2026-02-17T06:07:31.694 INFO:tasks.rados.rados.0.trial115.stdout:3422: finishing copy_from racing read to trial11545420-44 2026-02-17T06:07:31.695 INFO:tasks.rados.rados.0.trial115.stdout:3420: expect (ObjNum 1165 snap 358 seq_num 1165) 2026-02-17T06:07:31.839 INFO:tasks.rados.rados.0.trial115.stdout:3419: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:07:31.984 INFO:tasks.rados.rados.0.trial115.stdout:3423: finishing write tid 1 to trial11545420-23 2026-02-17T06:07:31.984 INFO:tasks.rados.rados.0.trial115.stdout:3423: finishing write tid 2 to trial11545420-23 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: finishing write tid 3 to trial11545420-23 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: finishing write tid 4 to trial11545420-23 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: finishing write tid 5 to trial11545420-23 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: oid 23 updating version 0 to 1756 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: oid 23 version 1756 is already newer than 1755 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: oid 23 updating version 1756 to 1757 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: oid 23 version 1757 is already newer than 1754 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1757 (ObjNum 1179 snap 362 seq_num 1179) dirty exists 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: left oid 23 (ObjNum 1179 snap 362 seq_num 1179) 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: finishing write tid 1 to trial11545420-37 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: finishing write tid 2 to trial11545420-37 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: finishing write tid 3 to trial11545420-37 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: finishing write tid 4 to trial11545420-37 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3419: done (5 left) 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3420: done (4 left) 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3422: done (3 left) 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3423: done (2 left) 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3425: done (1 left) 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3426: snap_create 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: finishing write tid 5 to trial11545420-37 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: finishing write tid 6 to trial11545420-37 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: oid 37 updating version 0 to 1699 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: oid 37 version 1699 is already newer than 1698 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: oid 37 updating version 1699 to 1700 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: oid 37 version 1700 is already newer than 1697 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: oid 37 version 1700 is already newer than 1696 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1700 (ObjNum 1180 snap 362 seq_num 1180) dirty exists 2026-02-17T06:07:31.985 INFO:tasks.rados.rados.0.trial115.stdout:3424: left oid 37 (ObjNum 1180 snap 362 seq_num 1180) 2026-02-17T06:07:32.668 INFO:tasks.rados.rados.0.trial115.stdout:3424: done (1 left) 2026-02-17T06:07:32.668 INFO:tasks.rados.rados.0.trial115.stdout:3426: done (0 left) 2026-02-17T06:07:32.668 INFO:tasks.rados.rados.0.trial115.stdout:3427: delete oid 14 current snap is 363 2026-02-17T06:07:32.669 INFO:tasks.rados.rados.0.trial115.stdout:3427: done (0 left) 2026-02-17T06:07:32.669 INFO:tasks.rados.rados.0.trial115.stdout:3428: snap_create 2026-02-17T06:07:33.669 INFO:tasks.rados.rados.0.trial115.stdout:3428: done (0 left) 2026-02-17T06:07:33.669 INFO:tasks.rados.rados.0.trial115.stdout:3429: delete oid 41 current snap is 364 2026-02-17T06:07:33.672 INFO:tasks.rados.rados.0.trial115.stdout:3429: done (0 left) 2026-02-17T06:07:33.672 INFO:tasks.rados.rados.0.trial115.stdout:3430: read oid 34 snap -1 2026-02-17T06:07:33.673 INFO:tasks.rados.rados.0.trial115.stdout:3430: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:07:33.673 INFO:tasks.rados.rados.0.trial115.stdout:3431: read oid 38 snap -1 2026-02-17T06:07:33.673 INFO:tasks.rados.rados.0.trial115.stdout:3431: expect (ObjNum 1169 snap 360 seq_num 1169) 2026-02-17T06:07:33.673 INFO:tasks.rados.rados.0.trial115.stdout:3432: delete oid 14 current snap is 364 2026-02-17T06:07:33.673 INFO:tasks.rados.rados.0.trial115.stdout:3432: done (2 left) 2026-02-17T06:07:33.674 INFO:tasks.rados.rados.0.trial115.stdout:3433: snap_remove snap 362 2026-02-17T06:07:33.762 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:33.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient: get_monmap_and_config 2026-02-17T06:07:33.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient: build_initial_monmap 2026-02-17T06:07:33.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient: monmap: 2026-02-17T06:07:33.831 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:33.831 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:33.831 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:33.828303+0000 2026-02-17T06:07:33.831 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:33.828303+0000 2026-02-17T06:07:33.831 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 1 Processor -- start 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 1 -- start start 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient: init 2026-02-17T06:07:33.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 0x7f8a0406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient(hunting): picked mon.noname-b con 0x7f8a0406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8a0409c4a0 0x7f8a0406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient(hunting): picked mon.noname-c con 0x7f8a0409c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8a04134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0406daf0 con 0x7f8a04134df0 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f8a040a0880 con 0x7f8a0409c4a0 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:33.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0406d6f0 con 0x7f8a0406c890 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0c64d640 10 monclient(hunting): authenticate will time out at 1899.381832s 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0a3c2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 0x7f8a0406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0a3c2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 0x7f8a0406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54382/0 (socket says 10.20.193.78:54382) 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0a3c2640 1 -- 10.20.193.78:0/399607791 learned_addr learned my addr 10.20.193.78:0/399607791 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0a3c2640 10 monclient(hunting): get_auth_request con 0x7f8a0406c890 auth_method 0 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0a3c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0a3c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0a3c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.827+0000 7f8a0a3c2640 1 -- 10.20.193.78:0/399607791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 msgr2=0x7f8a0406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a0a3c2640 1 -- 10.20.193.78:0/399607791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 msgr2=0x7f8a0406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:33.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a0a3c2640 1 --2- 10.20.193.78:0/399607791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 0x7f8a0406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a0a3c2640 1 --2- 10.20.193.78:0/399607791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 0x7f8a0406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 1 --2- 10.20.193.78:0/399607791 >> v2:10.20.193.115:3300/0 conn(0x7f8a0409c4a0 0x7f8a0406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient(hunting): get_auth_request con 0x7f8a0409c4a0 auth_method 0 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient(hunting): handle_auth_done global_id 214447 payload 274 2026-02-17T06:07:33.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 1 -- 10.20.193.78:0/399607791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 msgr2=0x7f8a0406cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:33.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 1 --2- 10.20.193.78:0/399607791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 0x7f8a0406cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 1 -- 10.20.193.78:0/399607791 >> v1:10.20.193.78:6789/0 conn(0x7f8a04134df0 legacy=0x7f8a04135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:33.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient: _finish_hunting 0 2026-02-17T06:07:33.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 1 monclient: found mon.noname-c 2026-02-17T06:07:33.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:33.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:07:33.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.828+0000 7f8a09bc1640 1 -- 10.20.193.78:0/399607791 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0409cc80 con 0x7f8a0409c4a0 2026-02-17T06:07:33.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a09bc1640 1 -- 10.20.193.78:0/399607791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0409c4a0 msgr2=0x7f8a0406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:33.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a09bc1640 1 --2- 10.20.193.78:0/399607791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0409c4a0 0x7f8a0406d1b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f89f80098a0 tx=0x7f89f802ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c156b1693fe181c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 1 -- 10.20.193.78:0/399607791 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f803c070 con 0x7f8a0409c4a0 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: dump: 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:33.844 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: _finish_auth 0 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: _check_auth_tickets 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 5 monclient: authenticate success, global_id 214447 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 1 -- 10.20.193.78:0/399607791 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f802fba0 con 0x7f8a0409c4a0 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 1 -- 10.20.193.78:0/399607791 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8037440 con 0x7f8a0409c4a0 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:33.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a093c0640 10 monclient: dump: 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:33.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 4 set_mon_vals no callback set 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 10 monclient: shutdown 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 1 -- 10.20.193.78:0/399607791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0409c4a0 msgr2=0x7f8a0406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/399607791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0409c4a0 0x7f8a0406d1b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f89f80098a0 tx=0x7f89f802ed20 comp rx=0 tx=0).stop 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 1 -- 10.20.193.78:0/399607791 shutdown_connections 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/399607791 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0409c4a0 0x7f8a0406d1b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/399607791 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0406c890 0x7f8a0406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 1 -- 10.20.193.78:0/399607791 >> 10.20.193.78:0/399607791 conn(0x7f8a040a6620 msgr2=0x7f8a040a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 1 -- 10.20.193.78:0/399607791 shutdown_connections 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.831+0000 7f8a0c64d640 1 -- 10.20.193.78:0/399607791 wait complete. 2026-02-17T06:07:33.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient: build_initial_monmap 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient: monmap: 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 1 Processor -- start 2026-02-17T06:07:33.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 1 -- start start 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient: init 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0406c890 0x7f8a041cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient(hunting): picked mon.noname-b con 0x7f8a0406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 0x7f8a04133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient(hunting): picked mon.noname-c con 0x7f8a0409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8a04134df0 0x7f8a04134500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient(hunting): picked mon.noname-a con 0x7f8a04134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a040a01a0 con 0x7f8a04134df0 2026-02-17T06:07:33.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a040a0020 con 0x7f8a0406c890 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0409be20 con 0x7f8a0409c4a0 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0c64d640 10 monclient(hunting): authenticate will time out at 1899.386813s 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8a04134df0 0x7f8a04134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 0x7f8a04133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 0x7f8a04133fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:54388/0 (socket says 10.20.193.78:54388) 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8a04134df0 0x7f8a04134500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44590/0 (socket says 10.20.193.78:44590) 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 1 -- 10.20.193.78:0/721813648 learned_addr learned my addr 10.20.193.78:0/721813648 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 10 monclient(hunting): get_auth_request con 0x7f8a0409c4a0 auth_method 0 2026-02-17T06:07:33.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 10 monclient(hunting): get_auth_request con 0x7f8a04134df0 auth_method 0 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 1 -- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 msgr2=0x7f8a04133fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 1 -- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 msgr2=0x7f8a04133fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 0x7f8a04133fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a09bc1640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 0x7f8a04133fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:33.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.832+0000 7f8a0abc3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0a3c2640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0406c890 0x7f8a041cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0a3c2640 10 monclient(hunting): get_auth_request con 0x7f8a0406c890 auth_method 0 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0a3c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0a3c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0a3c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 10 monclient(hunting): handle_auth_done global_id 254184 payload 824 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 1 -- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 msgr2=0x7f8a04133fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 0x7f8a04133fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 1 -- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0406c890 msgr2=0x7f8a041cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0406c890 0x7f8a041cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 10 monclient: _finish_hunting 0 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 1 monclient: found mon.noname-a 2026-02-17T06:07:33.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.833+0000 7f8a0abc3640 1 -- 10.20.193.78:0/721813648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a04127700 con 0x7f8a04134df0 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0abc3640 1 -- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8a04134df0 msgr2=0x7f8a04134500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0abc3640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8a04134df0 0x7f8a04134500 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000caf0 tx=0x7f8a00004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 1 -- 10.20.193.78:0/721813648 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00016030 con 0x7f8a04134df0 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: dump: 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:33.853 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: _finish_auth 0 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: _check_auth_tickets 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 1 -- 10.20.193.78:0/721813648 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0000ac50 con 0x7f8a04134df0 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 5 monclient: authenticate success, global_id 254184 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 10 monclient: _renew_subs 2026-02-17T06:07:33.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0409d810 con 0x7f8a04134df0 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 1 -- 10.20.193.78:0/721813648 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00004a80 con 0x7f8a04134df0 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 10 monclient: dump: 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:33.855 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a08bbf640 4 set_mon_vals no callback set 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 10 monclient: _renew_subs 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a040a1710 con 0x7f8a04134df0 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f89eaffd640 1 -- 10.20.193.78:0/721813648 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8a00023060 con 0x7f8a04134df0 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:33.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.835+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0406d000 con 0x7f8a04134df0 2026-02-17T06:07:33.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.836+0000 7f89eaffd640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f89d403deb0 0x7f89d4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:33.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.836+0000 7f89eaffd640 1 -- 10.20.193.78:0/721813648 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(751..751 src has 525..751) ==== 4783+0+0 (secure 0 0 0) 0x7f8a00052ae0 con 0x7f8a04134df0 2026-02-17T06:07:33.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.836+0000 7f8a0a3c2640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f89d403deb0 0x7f89d4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:33.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.836+0000 7f8a0a3c2640 10 monclient: get_auth_request con 0x7f89d403deb0 auth_method 0 2026-02-17T06:07:33.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.838+0000 7f89eaffd640 1 -- 10.20.193.78:0/721813648 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8a0406d000 con 0x7f8a04134df0 2026-02-17T06:07:33.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.838+0000 7f89eaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:33.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.838+0000 7f89eaffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:33.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.839+0000 7f8a0a3c2640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f89d403deb0 0x7f89d4040370 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f89f4006fd0 tx=0x7f89f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:33.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.957+0000 7f8a0c64d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:33.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.957+0000 7f8a0c64d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:33.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.957+0000 7f8a0c64d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:33.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.957+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8a04074980 con 0x7f8a04134df0 2026-02-17T06:07:33.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.957+0000 7f89eaffd640 1 -- 10.20.193.78:0/721813648 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f8a000514e0 con 0x7f8a04134df0 2026-02-17T06:07:33.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.957+0000 7f89eaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:33.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.957+0000 7f89eaffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:33.959 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":244,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f89d403deb0 msgr2=0x7f89d4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f89d403deb0 0x7f89d4040370 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f89f4006fd0 tx=0x7f89f4008040 comp rx=0 tx=0).stop 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 10 monclient: shutdown 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8a04134df0 msgr2=0x7f8a04134500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8a04134df0 0x7f8a04134500 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000caf0 tx=0x7f8a00004a10 comp rx=0 tx=0).stop 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 shutdown_connections 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8a04134df0 0x7f8a04134500 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8a0409c4a0 0x7f8a04133fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f89d403deb0 0x7f89d4040370 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 --2- 10.20.193.78:0/721813648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8a0406c890 0x7f8a041cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:33.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 >> 10.20.193.78:0/721813648 conn(0x7f8a040a6620 msgr2=0x7f8a04071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:33.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 shutdown_connections 2026-02-17T06:07:33.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:33.958+0000 7f8a0c64d640 1 -- 10.20.193.78:0/721813648 wait complete. 2026-02-17T06:07:34.874 INFO:tasks.rados.rados.0.trial115.stdout:3433: done (2 left) 2026-02-17T06:07:34.874 INFO:tasks.rados.rados.0.trial115.stdout:3434: copy_from oid 15 from oid 20 current snap is 364 2026-02-17T06:07:34.874 INFO:tasks.rados.rados.0.trial115.stdout:3430: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:07:34.991 INFO:tasks.rados.rados.0.trial115.stdout:3431: expect (ObjNum 1169 snap 360 seq_num 1169) 2026-02-17T06:07:35.074 INFO:tasks.rados.rados.0.trial115.stdout:3434: finishing copy_from racing read to trial11545420-15 2026-02-17T06:07:35.075 INFO:tasks.rados.rados.0.trial115.stdout:3434: finishing copy_from to trial11545420-15 2026-02-17T06:07:35.075 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1687 (ObjNum 1118 snap 336 seq_num 1118) dirty exists 2026-02-17T06:07:35.075 INFO:tasks.rados.rados.0.trial115.stdout:3430: done (2 left) 2026-02-17T06:07:35.075 INFO:tasks.rados.rados.0.trial115.stdout:3431: done (1 left) 2026-02-17T06:07:35.076 INFO:tasks.rados.rados.0.trial115.stdout:3434: done (0 left) 2026-02-17T06:07:35.076 INFO:tasks.rados.rados.0.trial115.stdout:3435: read oid 31 snap -1 2026-02-17T06:07:35.076 INFO:tasks.rados.rados.0.trial115.stdout:3435: expect (ObjNum 1162 snap 355 seq_num 1162) 2026-02-17T06:07:35.076 INFO:tasks.rados.rados.0.trial115.stdout:3436: snap_remove snap 354 2026-02-17T06:07:35.640 INFO:tasks.rados.rados.0.trial115.stdout:3436: done (1 left) 2026-02-17T06:07:35.641 INFO:tasks.rados.rados.0.trial115.stdout:3437: copy_from oid 47 from oid 4 current snap is 364 2026-02-17T06:07:35.641 INFO:tasks.rados.rados.0.trial115.stdout:3435: expect (ObjNum 1162 snap 355 seq_num 1162) 2026-02-17T06:07:35.791 INFO:tasks.rados.rados.0.trial115.stdout:3437: finishing copy_from racing read to trial11545420-47 2026-02-17T06:07:35.791 INFO:tasks.rados.rados.0.trial115.stdout:3437: finishing copy_from to trial11545420-47 2026-02-17T06:07:35.791 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1481 (ObjNum 1177 snap 361 seq_num 1177) dirty exists 2026-02-17T06:07:35.791 INFO:tasks.rados.rados.0.trial115.stdout:3435: done (1 left) 2026-02-17T06:07:35.791 INFO:tasks.rados.rados.0.trial115.stdout:3437: done (0 left) 2026-02-17T06:07:35.791 INFO:tasks.rados.rados.0.trial115.stdout:3438: write (excl) oid 24 current snap is 364 2026-02-17T06:07:35.791 INFO:tasks.rados.rados.0.trial115.stdout:3438: seq_num 1185 ranges {716295=404780,1593515=614409,2708717=492747,3640519=275776} 2026-02-17T06:07:35.795 INFO:tasks.rados.rados.0.trial115.stdout:3438: writing trial11545420-24 from 716295 to 1121075 tid 1 2026-02-17T06:07:35.799 INFO:tasks.rados.rados.0.trial115.stdout:3438: writing trial11545420-24 from 1593515 to 2207924 tid 2 2026-02-17T06:07:35.803 INFO:tasks.rados.rados.0.trial115.stdout:3438: writing trial11545420-24 from 2708717 to 3201464 tid 3 2026-02-17T06:07:35.806 INFO:tasks.rados.rados.0.trial115.stdout:3438: writing trial11545420-24 from 3640519 to 3916295 tid 4 2026-02-17T06:07:35.806 INFO:tasks.rados.rados.0.trial115.stdout:3439: rollback oid 12 current snap is 364 2026-02-17T06:07:35.806 INFO:tasks.rados.rados.0.trial115.stdout:3438: finishing write tid 1 to trial11545420-24 2026-02-17T06:07:35.806 INFO:tasks.rados.rados.0.trial115.stdout:3438: finishing write tid 2 to trial11545420-24 2026-02-17T06:07:35.806 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 12 to 353 2026-02-17T06:07:35.806 INFO:tasks.rados.rados.0.trial115.stdout:3440: read oid 3 snap -1 2026-02-17T06:07:35.806 INFO:tasks.rados.rados.0.trial115.stdout:3440: expect deleted 2026-02-17T06:07:35.806 INFO:tasks.rados.rados.0.trial115.stdout:3441: snap_remove snap 361 2026-02-17T06:07:36.643 INFO:tasks.rados.rados.0.trial115.stdout:3441: done (3 left) 2026-02-17T06:07:36.644 INFO:tasks.rados.rados.0.trial115.stdout:3442: snap_remove snap 363 2026-02-17T06:07:36.967 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:37.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.033+0000 7f8db1914640 10 monclient: get_monmap_and_config 2026-02-17T06:07:37.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.033+0000 7f8db1914640 10 monclient: build_initial_monmap 2026-02-17T06:07:37.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient: monmap: 2026-02-17T06:07:37.037 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:37.037 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.037 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:37.035217+0000 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:37.035217+0000 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 1 Processor -- start 2026-02-17T06:07:37.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 1 -- start start 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient: init 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient: _add_conns ranks=[3,4,2,0,1] 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient(hunting): picked mon.noname-a con 0x7f8dac06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient(hunting): picked mon.noname-c con 0x7f8dac09d430 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient(hunting): picked mon.noname-b con 0x7f8dac09bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f8dac09ca50 con 0x7f8dac06c890 2026-02-17T06:07:37.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f8dac06d6f0 con 0x7f8dac09d430 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8dac073600 con 0x7f8dac09bf40 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8db1914640 10 monclient(hunting): authenticate will time out at 1902.588745s 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44600/0 (socket says 10.20.193.78:44600) 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 1 -- 10.20.193.78:0/2196458202 learned_addr learned my addr 10.20.193.78:0/2196458202 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 10 monclient(hunting): get_auth_request con 0x7f8dac06c890 auth_method 0 2026-02-17T06:07:37.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 10 monclient(hunting): get_auth_request con 0x7f8dac09bf40 auth_method 0 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 msgr2=0x7f8dac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 msgr2=0x7f8dac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 msgr2=0x7f8dac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 msgr2=0x7f8dac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:37.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:37.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daa7fc640 10 monclient(hunting): get_auth_request con 0x7f8dac09d430 auth_method 0 2026-02-17T06:07:37.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.034+0000 7f8daa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.035+0000 7f8daa7fc640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 msgr2=0x7f8dac06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:07:37.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.035+0000 7f8daa7fc640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 msgr2=0x7f8dac06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.035+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.035+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:37.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 10 monclient(hunting): get_auth_request con 0x7f8dac06c890 auth_method 0 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 10 monclient(hunting): get_auth_request con 0x7f8dac09bf40 auth_method 0 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:37.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 msgr2=0x7f8dac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 msgr2=0x7f8dac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 msgr2=0x7f8dac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 msgr2=0x7f8dac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 10 monclient(hunting): get_auth_request con 0x7f8dac09d430 auth_method 0 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:37.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 msgr2=0x7f8dac06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:07:37.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 msgr2=0x7f8dac06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.235+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:07:37.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 10 monclient(hunting): get_auth_request con 0x7f8dac06c890 auth_method 0 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 10 monclient(hunting): get_auth_request con 0x7f8dac09bf40 auth_method 0 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 msgr2=0x7f8dac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 msgr2=0x7f8dac06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:07:37.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 msgr2=0x7f8dac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 msgr2=0x7f8dac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.115:3300/0 conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient(hunting): get_auth_request con 0x7f8dac09d430 auth_method 0 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:37.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient(hunting): handle_auth_done global_id 214453 payload 274 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 msgr2=0x7f8dac134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 1 -- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 msgr2=0x7f8dac06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 1 monclient: found mon.noname-c 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:07:37.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.636+0000 7f8daa7fc640 1 -- 10.20.193.78:0/2196458202 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac06dab0 con 0x7f8dac09d430 2026-02-17T06:07:37.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8daa7fc640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09d430 msgr2=0x7f8dac06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:37.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09d430 0x7f8dac06d1b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0a0ac0 tx=0x7f8d9402f400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73cbdcda4af365af server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 1 -- 10.20.193.78:0/2196458202 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94035070 con 0x7f8dac09d430 2026-02-17T06:07:37.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:37.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: dump: 2026-02-17T06:07:37.650 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.651 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8db1914640 5 monclient: authenticate success, global_id 214453 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8db1914640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 1 -- 10.20.193.78:0/2196458202 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d94035620 con 0x7f8dac09d430 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 1 -- 10.20.193.78:0/2196458202 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400f040 con 0x7f8dac09d430 2026-02-17T06:07:37.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8db1914640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8db1914640 4 set_mon_vals no callback set 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.639+0000 7f8da9ffb640 10 monclient: dump: 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.653 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 10 monclient: shutdown 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 -- 10.20.193.78:0/2196458202 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09d430 msgr2=0x7f8dac06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09d430 0x7f8dac06d1b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0a0ac0 tx=0x7f8d9402f400 comp rx=0 tx=0).stop 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 -- 10.20.193.78:0/2196458202 shutdown_connections 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09bf40 0x7f8dac134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 --2- 10.20.193.78:0/2196458202 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09d430 0x7f8dac06d1b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 --2- 10.20.193.78:0/2196458202 >> v2:10.20.193.78:3300/0 conn(0x7f8dac06c890 0x7f8dac06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 -- 10.20.193.78:0/2196458202 >> 10.20.193.78:0/2196458202 conn(0x7f8dac0a6620 msgr2=0x7f8dac0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:37.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 -- 10.20.193.78:0/2196458202 shutdown_connections 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 1 -- 10.20.193.78:0/2196458202 wait complete. 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 10 monclient: build_initial_monmap 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.640+0000 7f8db1914640 10 monclient: monmap: 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:37.655 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 1 Processor -- start 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 1 -- start start 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient: init 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8dac06c890 0x7f8dac121640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient(hunting): picked mon.noname-a con 0x7f8dac06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09bf40 0x7f8dac121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient(hunting): picked mon.noname-b con 0x7f8dac09bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:37.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 0x7f8dac1220c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient(hunting): picked mon.noname-c con 0x7f8dac09d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac073380 con 0x7f8dac06c890 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac073200 con 0x7f8dac09bf40 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8dac135330 con 0x7f8dac09d430 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8db1914640 10 monclient(hunting): authenticate will time out at 1903.195741s 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8dac06c890 0x7f8dac121640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 0x7f8dac1220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8dac06c890 0x7f8dac121640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40824/0 (socket says 10.20.193.78:40824) 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 1 -- 10.20.193.78:0/2310807397 learned_addr learned my addr 10.20.193.78:0/2310807397 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 10 monclient(hunting): get_auth_request con 0x7f8dac06c890 auth_method 0 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 10 monclient(hunting): get_auth_request con 0x7f8dac09d430 auth_method 0 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:37.658 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 1 -- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 msgr2=0x7f8dac1220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 1 -- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 msgr2=0x7f8dac1220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 0x7f8dac1220c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8dab7fe640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 0x7f8dac1220c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09bf40 0x7f8dac121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daa7fc640 10 monclient(hunting): get_auth_request con 0x7f8dac09bf40 auth_method 0 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.641+0000 7f8daa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 10 monclient(hunting): handle_auth_done global_id 254190 payload 824 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 1 -- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 msgr2=0x7f8dac1220c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:37.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 0x7f8dac1220c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 1 -- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09bf40 msgr2=0x7f8dac121b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09bf40 0x7f8dac121b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 10 monclient: _finish_hunting 0 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 1 monclient: found mon.noname-a 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.642+0000 7f8daaffd640 1 -- 10.20.193.78:0/2310807397 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac124d50 con 0x7f8dac06c890 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8daaffd640 1 -- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8dac06c890 msgr2=0x7f8dac121640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8daaffd640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8dac06c890 0x7f8dac121640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8da0011510 tx=0x7f8da00119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 1 -- 10.20.193.78:0/2310807397 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0019070 con 0x7f8dac06c890 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: dump: 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: _finish_auth 0 2026-02-17T06:07:37.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: _check_auth_tickets 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8db1914640 5 monclient: authenticate success, global_id 254190 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8db1914640 10 monclient: _renew_subs 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 1 -- 10.20.193.78:0/2310807397 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da0015860 con 0x7f8dac06c890 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8db1914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dac125030 con 0x7f8dac06c890 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 1 -- 10.20.193.78:0/2310807397 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0015c50 con 0x7f8dac06c890 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8d83fff640 10 monclient: dump: 2026-02-17T06:07:37.662 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8da97fa640 4 set_mon_vals no callback set 2026-02-17T06:07:37.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8db1914640 10 monclient: _renew_subs 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8db1914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.644+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dac137470 con 0x7f8dac06c890 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8d83fff640 1 -- 10.20.193.78:0/2310807397 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8da0009420 con 0x7f8dac06c890 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8db1914640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8db1914640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8db1914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dac074980 con 0x7f8dac06c890 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8d83fff640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d7c03deb0 0x7f8d7c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8d83fff640 1 -- 10.20.193.78:0/2310807397 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(754..754 src has 525..754) ==== 4779+0+0 (secure 0 0 0) 0x7f8da0053030 con 0x7f8dac06c890 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d7c03deb0 0x7f8d7c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.645+0000 7f8daa7fc640 10 monclient: get_auth_request con 0x7f8d7c03deb0 auth_method 0 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.647+0000 7f8d83fff640 1 -- 10.20.193.78:0/2310807397 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8da0015df0 con 0x7f8dac06c890 2026-02-17T06:07:37.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.647+0000 7f8d83fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:37.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.647+0000 7f8d83fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:37.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.648+0000 7f8daa7fc640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d7c03deb0 0x7f8d7c040370 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8d94037000 tx=0x7f8d94035000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.666 INFO:tasks.rados.rados.0.trial115.stdout:3442: done (3 left) 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3443: rollback oid 7 current snap is 364 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 349 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: finishing write tid 3 to trial11545420-24 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: finishing write tid 4 to trial11545420-24 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: finishing write tid 5 to trial11545420-24 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: finishing write tid 6 to trial11545420-24 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: oid 24 updating version 0 to 1807 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: oid 24 updating version 1807 to 1810 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: oid 24 version 1810 is already newer than 1809 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: oid 24 updating version 1810 to 1811 2026-02-17T06:07:37.667 INFO:tasks.rados.rados.0.trial115.stdout:3438: oid 24 version 1811 is already newer than 1808 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1811 (ObjNum 1185 snap 364 seq_num 1185) dirty exists 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:3438: left oid 24 (ObjNum 1185 snap 364 seq_num 1185) 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:3439: finishing rollback tid 0 to trial11545420-12 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:3439: finishing rollback tid 1 to trial11545420-12 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:3439: finishing rollback tid 2 to trial11545420-12 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1815 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:3438: done (3 left) 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:3439: done (2 left) 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:3440: done (1 left) 2026-02-17T06:07:37.668 INFO:tasks.rados.rados.0.trial115.stdout:3444: copy_from oid 13 from oid 40 current snap is 364 2026-02-17T06:07:37.669 INFO:tasks.rados.rados.0.trial115.stdout:3445: rollback oid 17 current snap is 364 2026-02-17T06:07:37.669 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 360 2026-02-17T06:07:37.669 INFO:tasks.rados.rados.0.trial115.stdout:3446: write oid 41 current snap is 364 2026-02-17T06:07:37.669 INFO:tasks.rados.rados.0.trial115.stdout:3446: seq_num 1186 ranges {733102=596382,1768350=758067,3136140=704019} 2026-02-17T06:07:37.669 INFO:tasks.rados.rados.0.trial115.stdout:3446: writing trial11545420-41 from 733102 to 1329484 tid 1 2026-02-17T06:07:37.669 INFO:tasks.rados.rados.0.trial115.stdout:3446: writing trial11545420-41 from 1768350 to 2526417 tid 2 2026-02-17T06:07:37.672 INFO:tasks.rados.rados.0.trial115.stdout:3446: writing trial11545420-41 from 3136140 to 3840159 tid 3 2026-02-17T06:07:37.672 INFO:tasks.rados.rados.0.trial115.stdout:3447: read oid 22 snap -1 2026-02-17T06:07:37.672 INFO:tasks.rados.rados.0.trial115.stdout:3447: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:07:37.672 INFO:tasks.rados.rados.0.trial115.stdout:3448: rollback oid 10 current snap is 364 2026-02-17T06:07:37.672 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 10 to 360 2026-02-17T06:07:37.672 INFO:tasks.rados.rados.0.trial115.stdout:3443: finishing rollback tid 0 to trial11545420-7 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3445: finishing rollback tid 0 to trial11545420-17 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3443: finishing rollback tid 1 to trial11545420-7 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3443: finishing rollback tid 2 to trial11545420-7 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1703 (ObjNum 1019 snap 297 seq_num 1019) dirty exists 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3445: finishing rollback tid 1 to trial11545420-17 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3445: finishing rollback tid 2 to trial11545420-17 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1760 (ObjNum 886 snap 254 seq_num 886) dirty exists 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: finishing write tid 1 to trial11545420-41 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: finishing write tid 2 to trial11545420-41 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3444: finishing copy_from racing read to trial11545420-13 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3443: done (5 left) 2026-02-17T06:07:37.673 INFO:tasks.rados.rados.0.trial115.stdout:3445: done (4 left) 2026-02-17T06:07:37.674 INFO:tasks.rados.rados.0.trial115.stdout:3449: snap_remove snap 341 2026-02-17T06:07:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.767+0000 7f8db1914640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.767+0000 7f8db1914640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.767+0000 7f8db1914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.767+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8dac134dc0 con 0x7f8dac06c890 2026-02-17T06:07:37.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.768+0000 7f8d83fff640 1 -- 10.20.193.78:0/2310807397 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f8da0051a30 con 0x7f8dac06c890 2026-02-17T06:07:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.768+0000 7f8d83fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:37.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.768+0000 7f8d83fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:37.770 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:37.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d7c03deb0 msgr2=0x7f8d7c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d7c03deb0 0x7f8d7c040370 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8d94037000 tx=0x7f8d94035000 comp rx=0 tx=0).stop 2026-02-17T06:07:37.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 10 monclient: shutdown 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8dac06c890 msgr2=0x7f8dac121640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8dac06c890 0x7f8dac121640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8da0011510 tx=0x7f8da00119e0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 shutdown_connections 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8dac09d430 0x7f8dac1220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8d7c03deb0 0x7f8d7c040370 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8dac09bf40 0x7f8dac121b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 --2- 10.20.193.78:0/2310807397 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8dac06c890 0x7f8dac121640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 >> 10.20.193.78:0/2310807397 conn(0x7f8dac0a6620 msgr2=0x7f8dac135de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 shutdown_connections 2026-02-17T06:07:37.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.769+0000 7f8db1914640 1 -- 10.20.193.78:0/2310807397 wait complete. 2026-02-17T06:07:37.778 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:07:37.778 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:07:37.778 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:07:37.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient: get_monmap_and_config 2026-02-17T06:07:37.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient: build_initial_monmap 2026-02-17T06:07:37.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient: monmap: 2026-02-17T06:07:37.848 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:37.848 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.848 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:37.845322+0000 2026-02-17T06:07:37.848 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:37.845322+0000 2026-02-17T06:07:37.848 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:37.848 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.848 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 1 Processor -- start 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 1 -- start start 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient: init 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:37.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f05a806c890 0x7f05a806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient(hunting): picked mon.noname-a con 0x7f05a806c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f05a809c4a0 0x7f05a806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient(hunting): picked mon.noname-c con 0x7f05a809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f05a8134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a806daf0 con 0x7f05a8134df0 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f05a80a0880 con 0x7f05a806c890 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f05a806d6f0 con 0x7f05a809c4a0 2026-02-17T06:07:37.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05af424640 10 monclient(hunting): authenticate will time out at 1903.398869s 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05ad199640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f05a806c890 0x7f05a806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05ad199640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f05a806c890 0x7f05a806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40830/0 (socket says 10.20.193.78:40830) 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05ad199640 1 -- 10.20.193.78:0/3602975677 learned_addr learned my addr 10.20.193.78:0/3602975677 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05ad199640 10 monclient(hunting): get_auth_request con 0x7f05a806c890 auth_method 0 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.844+0000 7f05ad199640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ac998640 1 --2- 10.20.193.78:0/3602975677 >> v2:10.20.193.115:3300/0 conn(0x7f05a809c4a0 0x7f05a806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:37.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ac998640 10 monclient(hunting): get_auth_request con 0x7f05a809c4a0 auth_method 0 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ac998640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ac998640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ac998640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 10 monclient(hunting): handle_auth_done global_id 254193 payload 274 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 1 -- 10.20.193.78:0/3602975677 >> v2:10.20.193.115:3300/0 conn(0x7f05a809c4a0 msgr2=0x7f05a806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 1 --2- 10.20.193.78:0/3602975677 >> v2:10.20.193.115:3300/0 conn(0x7f05a809c4a0 0x7f05a806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 1 -- 10.20.193.78:0/3602975677 >> v1:10.20.193.115:6789/0 conn(0x7f05a8134df0 legacy=0x7f05a8135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 10 monclient: _finish_hunting 0 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 1 monclient: found mon.noname-a 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:37.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.845+0000 7f05ad199640 1 -- 10.20.193.78:0/3602975677 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a809cc80 con 0x7f05a806c890 2026-02-17T06:07:37.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f05ad199640 1 -- 10.20.193.78:0/3602975677 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a806c890 msgr2=0x7f05a806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:37.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f05ad199640 1 --2- 10.20.193.78:0/3602975677 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a806c890 0x7f05a806cc70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05a809c980 tx=0x7f059802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687d0cdc31bbff04 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 1 -- 10.20.193.78:0/3602975677 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059803c070 con 0x7f05a806c890 2026-02-17T06:07:37.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: dump: 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.863 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: _finish_auth 0 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: _check_auth_tickets 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f05af424640 5 monclient: authenticate success, global_id 254193 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 1 -- 10.20.193.78:0/3602975677 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059802fca0 con 0x7f05a806c890 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f05af424640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 1 -- 10.20.193.78:0/3602975677 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598037660 con 0x7f05a806c890 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f05af424640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f05af424640 4 set_mon_vals no callback set 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.847+0000 7f0597fff640 10 monclient: dump: 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 10 monclient: shutdown 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 -- 10.20.193.78:0/3602975677 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a806c890 msgr2=0x7f05a806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 --2- 10.20.193.78:0/3602975677 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a806c890 0x7f05a806cc70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05a809c980 tx=0x7f059802ee60 comp rx=0 tx=0).stop 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 -- 10.20.193.78:0/3602975677 shutdown_connections 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 --2- 10.20.193.78:0/3602975677 >> v2:10.20.193.115:3300/0 conn(0x7f05a809c4a0 0x7f05a806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 --2- 10.20.193.78:0/3602975677 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a806c890 0x7f05a806cc70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 -- 10.20.193.78:0/3602975677 >> 10.20.193.78:0/3602975677 conn(0x7f05a80a6620 msgr2=0x7f05a80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 -- 10.20.193.78:0/3602975677 shutdown_connections 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 -- 10.20.193.78:0/3602975677 wait complete. 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 10 monclient: build_initial_monmap 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 10 monclient: monmap: 2026-02-17T06:07:37.866 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 Processor -- start 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 1 -- start start 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.848+0000 7f05af424640 10 monclient: init 2026-02-17T06:07:37.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f05a806c890 0x7f05a8133f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient(hunting): picked mon.noname-b con 0x7f05a806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05a809c4a0 0x7f05a81344c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient(hunting): picked mon.noname-c con 0x7f05a809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a8134df0 0x7f05a809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient(hunting): picked mon.noname-a con 0x7f05a8134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05a80a01a0 con 0x7f05a8134df0 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f05a80a0020 con 0x7f05a806c890 2026-02-17T06:07:37.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f05a809be20 con 0x7f05a809c4a0 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05af424640 10 monclient(hunting): authenticate will time out at 1903.403110s 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a8134df0 0x7f05a809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a8134df0 0x7f05a809db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40834/0 (socket says 10.20.193.78:40834) 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 -- 10.20.193.78:0/2653453110 learned_addr learned my addr 10.20.193.78:0/2653453110 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient(hunting): get_auth_request con 0x7f05a8134df0 auth_method 0 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05a809c4a0 0x7f05a81344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 10 monclient(hunting): get_auth_request con 0x7f05a809c4a0 auth_method 0 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05a809c4a0 msgr2=0x7f05a81344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05a809c4a0 msgr2=0x7f05a81344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:37.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05a809c4a0 0x7f05a81344c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ac998640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05a809c4a0 0x7f05a81344c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient(hunting): handle_auth_done global_id 254196 payload 824 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05a809c4a0 msgr2=0x7f05a81344c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f05a809c4a0 0x7f05a81344c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f05a806c890 msgr2=0x7f05a8133f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f05a806c890 0x7f05a8133f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient: _finish_hunting 0 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 monclient: found mon.noname-a 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.849+0000 7f05ad99a640 1 -- 10.20.193.78:0/2653453110 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a809d810 con 0x7f05a8134df0 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05ad99a640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a8134df0 msgr2=0x7f05a809db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05ad99a640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a8134df0 0x7f05a809db50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f05a400cb60 tx=0x7f05a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4016020 con 0x7f05a8134df0 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: dump: 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.872 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: _finish_auth 0 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a4012420 con 0x7f05a8134df0 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05af424640 5 monclient: authenticate success, global_id 254196 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05af424640 10 monclient: _renew_subs 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05af424640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a8072700 con 0x7f05a8134df0 2026-02-17T06:07:37.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4012810 con 0x7f05a8134df0 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f0595ffb640 10 monclient: dump: 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.874 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05977fe640 4 set_mon_vals no callback set 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05af424640 10 monclient: _renew_subs 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05af424640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a809f7f0 con 0x7f05a8134df0 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.851+0000 7f05af424640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05af424640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a81cf4c0 0x7f05a81275e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05af424640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8074980 con 0x7f05a81cf4c0 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a81cf4c0 0x7f05a81275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 10 monclient: get_auth_request con 0x7f05a81cf4c0 auth_method 0 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 10 monclient(hunting): handle_auth_done global_id 254199 payload 824 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ad199640 1 -- 10.20.193.78:0/2653453110 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05a8135a20 con 0x7f05a81cf4c0 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f05a40129f0 con 0x7f05a8134df0 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f0595ffb640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f057803dee0 0x7f05780403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(755..755 src has 525..755) ==== 4779+0+0 (secure 0 0 0) 0x7f05a40533c0 con 0x7f05a8134df0 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ac998640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f057803dee0 0x7f05780403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.852+0000 7f05ac998640 10 monclient: get_auth_request con 0x7f057803dee0 auth_method 0 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f05ad199640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a81cf4c0 msgr2=0x7f05a81275e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f05ad199640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a81cf4c0 0x7f05a81275e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05a8074de0 tx=0x7f05980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598046020 con 0x7f05a81cf4c0 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 10 monclient: dump: 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.877 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f0598037bb0 con 0x7f05a81cf4c0 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a81cf4c0 msgr2=0x7f05a81275e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a81cf4c0 0x7f05a81275e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05a8074de0 tx=0x7f05980047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a81cf4c0 msgr2=0x7f05a81275e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.854+0000 7f0595ffb640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a81cf4c0 0x7f05a81275e0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05a8074de0 tx=0x7f05980047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.855+0000 7f05ac998640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f057803dee0 0x7f05780403a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f05a809f2d0 tx=0x7f059c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05af424640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05af424640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a811cc70 0x7f05a81da5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05af424640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8126210 con 0x7f05a811cc70 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a811cc70 0x7f05a81da5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 10 monclient: get_auth_request con 0x7f05a811cc70 auth_method 0 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 10 monclient(hunting): handle_auth_done global_id 254202 payload 824 2026-02-17T06:07:37.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.857+0000 7f05ad199640 1 -- 10.20.193.78:0/2653453110 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f05a80744d0 con 0x7f05a811cc70 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05ad199640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a811cc70 msgr2=0x7f05a81da5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05ad199640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a811cc70 0x7f05a81da5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f059802eea0 tx=0x7f0598008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598051070 con 0x7f05a811cc70 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 10 monclient: dump: 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.880 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f0598003e30 con 0x7f05a811cc70 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a811cc70 msgr2=0x7f05a81da5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a811cc70 0x7f05a81da5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f059802eea0 tx=0x7f0598008030 comp rx=0 tx=0).stop 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a811cc70 msgr2=0x7f05a81da5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:37.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f0595ffb640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a811cc70 0x7f05a81da5a0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f059802eea0 tx=0x7f0598008030 comp rx=0 tx=0).stop 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 246, 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 57234, 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:07:37.882 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:37.883 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:37.884 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:37.885 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:37.886 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:37.887 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:37.888 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:37.889 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:37.890 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:37.891 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:37.892 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:37.892 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:07:37.892 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:37.892 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:37.892 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:37.892 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:37.892 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f057803dee0 msgr2=0x7f05780403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f057803dee0 0x7f05780403a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f05a809f2d0 tx=0x7f059c008040 comp rx=0 tx=0).stop 2026-02-17T06:07:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 10 monclient: shutdown 2026-02-17T06:07:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:37.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a8134df0 msgr2=0x7f05a809db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 1 --2- 10.20.193.78:0/2653453110 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f05a8134df0 0x7f05a809db50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f05a400cb60 tx=0x7f05a400a430 comp rx=0 tx=0).stop 2026-02-17T06:07:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05ad99a640 1 -- 10.20.193.78:0/2653453110 reap_dead start 2026-02-17T06:07:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05ad99a640 1 -- 10.20.193.78:0/2653453110 reap_dead start 2026-02-17T06:07:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 shutdown_connections 2026-02-17T06:07:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 >> 10.20.193.78:0/2653453110 conn(0x7f05a80a6620 msgr2=0x7f05a80a1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 shutdown_connections 2026-02-17T06:07:37.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.860+0000 7f05af424640 1 -- 10.20.193.78:0/2653453110 wait complete. 2026-02-17T06:07:37.894 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:07:37.894 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:07:37.894 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient: get_monmap_and_config 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient: build_initial_monmap 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient: monmap: 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:37.962333+0000 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:37.962333+0000 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:37.964 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 1 Processor -- start 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 1 -- start start 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient: init 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f51d409ea90 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:37.965 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d409f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient(hunting): picked mon.noname-b con 0x7f51d409ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f51d413d640 0x7f51d413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient(hunting): picked mon.noname-c con 0x7f51d413d640 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d40a0a20 con 0x7f51d409ea90 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f51d40a0d00 con 0x7f51d413d640 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f51d40a0460 con 0x7f51d409ee70 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51db8c4640 10 monclient(hunting): authenticate will time out at 1903.515927s 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.961+0000 7f51d9639640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f51d409ea90 0x7f51d40a13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38114/0 (socket says 10.20.193.78:38114) 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9639640 1 -- 10.20.193.78:0/2493571115 learned_addr learned my addr 10.20.193.78:0/2493571115 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 1 --2- 10.20.193.78:0/2493571115 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d409f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 10 monclient(hunting): get_auth_request con 0x7f51d409ee70 auth_method 0 2026-02-17T06:07:37.966 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 1 -- 10.20.193.78:0/2493571115 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 msgr2=0x7f51d409f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 1 -- 10.20.193.78:0/2493571115 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 msgr2=0x7f51d409f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 1 --2- 10.20.193.78:0/2493571115 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d409f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 1 --2- 10.20.193.78:0/2493571115 >> v2:10.20.193.115:3300/0 conn(0x7f51d413d640 0x7f51d413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d8e38640 1 --2- 10.20.193.78:0/2493571115 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d409f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient(hunting): get_auth_request con 0x7f51d413d640 auth_method 0 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:37.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient(hunting): handle_auth_done global_id 214462 payload 274 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 1 -- 10.20.193.78:0/2493571115 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 msgr2=0x7f51d409f290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 1 --2- 10.20.193.78:0/2493571115 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d409f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 1 -- 10.20.193.78:0/2493571115 >> v1:10.20.193.78:6789/0 conn(0x7f51d409ea90 legacy=0x7f51d40a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient: _finish_hunting 0 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 1 monclient: found mon.noname-c 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:07:37.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.962+0000 7f51d9e3a640 1 -- 10.20.193.78:0/2493571115 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d40a0740 con 0x7f51d413d640 2026-02-17T06:07:37.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51d9e3a640 1 -- 10.20.193.78:0/2493571115 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d413d640 msgr2=0x7f51d413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:37.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51d9e3a640 1 --2- 10.20.193.78:0/2493571115 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d413d640 0x7f51d413fa30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009d90 tx=0x7f51d002f190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efc93480c60af863 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 1 -- 10.20.193.78:0/2493571115 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d003c070 con 0x7f51d413d640 2026-02-17T06:07:37.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: dump: 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: _finish_auth 0 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: _check_auth_tickets 2026-02-17T06:07:37.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 5 monclient: authenticate success, global_id 214462 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 1 -- 10.20.193.78:0/2493571115 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d0037440 con 0x7f51d413d640 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 1 -- 10.20.193.78:0/2493571115 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0037810 con 0x7f51d413d640 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51c3fff640 10 monclient: dump: 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.976 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 4 set_mon_vals no callback set 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 10 monclient: shutdown 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 1 -- 10.20.193.78:0/2493571115 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d413d640 msgr2=0x7f51d413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 1 --2- 10.20.193.78:0/2493571115 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d413d640 0x7f51d413fa30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009d90 tx=0x7f51d002f190 comp rx=0 tx=0).stop 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 1 -- 10.20.193.78:0/2493571115 shutdown_connections 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 1 --2- 10.20.193.78:0/2493571115 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d413d640 0x7f51d413fa30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 1 --2- 10.20.193.78:0/2493571115 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d409f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 1 -- 10.20.193.78:0/2493571115 >> 10.20.193.78:0/2493571115 conn(0x7f51d40a7520 msgr2=0x7f51d40a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:37.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 1 -- 10.20.193.78:0/2493571115 shutdown_connections 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.965+0000 7f51db8c4640 1 -- 10.20.193.78:0/2493571115 wait complete. 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient: build_initial_monmap 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient: monmap: 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 1 Processor -- start 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 1 -- start start 2026-02-17T06:07:37.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient: init 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d409ea90 0x7f51d413b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient(hunting): picked mon.noname-b con 0x7f51d409ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d413bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient(hunting): picked mon.noname-c con 0x7f51d409ee70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f51d413d640 0x7f51d413c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient(hunting): picked mon.noname-a con 0x7f51d413d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40a6820 con 0x7f51d413d640 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40a66a0 con 0x7f51d409ea90 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f51d413ff70 con 0x7f51d409ee70 2026-02-17T06:07:37.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51db8c4640 10 monclient(hunting): authenticate will time out at 1903.520953s 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.966+0000 7f51d8e38640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d413bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d413bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48854/0 (socket says 10.20.193.78:48854) 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f51d413d640 0x7f51d413c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 1 -- 10.20.193.78:0/975268336 learned_addr learned my addr 10.20.193.78:0/975268336 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 10 monclient(hunting): get_auth_request con 0x7f51d409ee70 auth_method 0 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient(hunting): get_auth_request con 0x7f51d413d640 auth_method 0 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:37.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 msgr2=0x7f51d413bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 msgr2=0x7f51d413bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d413bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d8e38640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d413bdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d409ea90 0x7f51d413b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9639640 10 monclient(hunting): get_auth_request con 0x7f51d409ea90 auth_method 0 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9639640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9639640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9639640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient(hunting): handle_auth_done global_id 254205 payload 824 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 msgr2=0x7f51d413bdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d409ee70 0x7f51d413bdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d409ea90 msgr2=0x7f51d413b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f51d409ea90 0x7f51d413b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:37.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient: _finish_hunting 0 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 1 monclient: found mon.noname-a 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.967+0000 7f51d9e3a640 1 -- 10.20.193.78:0/975268336 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d413d380 con 0x7f51d413d640 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51d9e3a640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f51d413d640 msgr2=0x7f51d413c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51d9e3a640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f51d413d640 0x7f51d413c300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51d0036000 tx=0x7f51d0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d003c050 con 0x7f51d413d640 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: dump: 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.982 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: _finish_auth 0 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d0048070 con 0x7f51d413d640 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0041c60 con 0x7f51d413d640 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c1ffb640 10 monclient: dump: 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:37.983 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51c37fe640 4 set_mon_vals no callback set 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51db8c4640 5 monclient: authenticate success, global_id 254205 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51db8c4640 10 monclient: _renew_subs 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51db8c4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d413fbb0 con 0x7f51d413d640 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51db8c4640 10 monclient: _renew_subs 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51db8c4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:37.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d406ef90 con 0x7f51d413d640 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.969+0000 7f51db8c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51db8c4640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51db8c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f51d40a1c10 con 0x7f51d41cef30 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 10 monclient: get_auth_request con 0x7f51d41cef30 auth_method 0 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f51d004d460 con 0x7f51d413d640 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51c1ffb640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51a403deb0 0x7f51a4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:37.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(755..755 src has 525..755) ==== 4779+0+0 (secure 0 0 0) 0x7f51d0078760 con 0x7f51d413d640 2026-02-17T06:07:37.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d8e38640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51a403deb0 0x7f51a4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:37.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.970+0000 7f51d8e38640 10 monclient: get_auth_request con 0x7f51a403deb0 auth_method 0 2026-02-17T06:07:37.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:37.973+0000 7f51d8e38640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51a403deb0 0x7f51a4040370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f51d40a9ed0 tx=0x7f51c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 10 monclient: get_auth_request con 0x7f51d41cef30 auth_method 0 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:38.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.170+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:07:38.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:38.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 10 monclient: get_auth_request con 0x7f51d41cef30 auth_method 0 2026-02-17T06:07:38.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:38.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:38.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:38.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:07:38.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:38.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:38.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:38.571+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:07:38.672 INFO:tasks.rados.rados.0.trial115.stdout:3449: done (4 left) 2026-02-17T06:07:38.672 INFO:tasks.rados.rados.0.trial115.stdout:3450: delete oid 37 current snap is 364 2026-02-17T06:07:38.672 INFO:tasks.rados.rados.0.trial115.stdout:3444: finishing copy_from to trial11545420-13 2026-02-17T06:07:38.672 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1417 (ObjNum 1026 snap 297 seq_num 1026) dirty exists 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: finishing write tid 3 to trial11545420-41 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: finishing write tid 4 to trial11545420-41 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: finishing write tid 5 to trial11545420-41 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: oid 41 updating version 0 to 1575 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: oid 41 updating version 1575 to 1577 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: oid 41 version 1577 is already newer than 1574 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: oid 41 version 1577 is already newer than 1576 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1577 (ObjNum 1186 snap 364 seq_num 1186) dirty exists 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3446: left oid 41 (ObjNum 1186 snap 364 seq_num 1186) 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3448: finishing rollback tid 0 to trial11545420-10 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3448: finishing rollback tid 1 to trial11545420-10 2026-02-17T06:07:38.673 INFO:tasks.rados.rados.0.trial115.stdout:3448: finishing rollback tid 2 to trial11545420-10 2026-02-17T06:07:38.674 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1808 (ObjNum 1122 snap 336 seq_num 1122) dirty exists 2026-02-17T06:07:38.674 INFO:tasks.rados.rados.0.trial115.stdout:3447: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:07:38.835 INFO:tasks.rados.rados.0.trial115.stdout:3444: done (4 left) 2026-02-17T06:07:38.835 INFO:tasks.rados.rados.0.trial115.stdout:3446: done (3 left) 2026-02-17T06:07:38.836 INFO:tasks.rados.rados.0.trial115.stdout:3447: done (2 left) 2026-02-17T06:07:38.836 INFO:tasks.rados.rados.0.trial115.stdout:3448: done (1 left) 2026-02-17T06:07:38.836 INFO:tasks.rados.rados.0.trial115.stdout:3450: done (0 left) 2026-02-17T06:07:38.836 INFO:tasks.rados.rados.0.trial115.stdout:3451: copy_from oid 6 from oid 34 current snap is 364 2026-02-17T06:07:38.836 INFO:tasks.rados.rados.0.trial115.stdout:3452: rollback oid 37 current snap is 364 2026-02-17T06:07:38.836 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 349 2026-02-17T06:07:38.836 INFO:tasks.rados.rados.0.trial115.stdout:3453: delete oid 26 current snap is 364 2026-02-17T06:07:38.837 INFO:tasks.rados.rados.0.trial115.stdout:3452: finishing rollback tid 1 to trial11545420-37 2026-02-17T06:07:38.837 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1704 (ObjNum 129 snap 0 seq_num 1334318744) dirty dne 2026-02-17T06:07:38.839 INFO:tasks.rados.rados.0.trial115.stdout:3452: done (2 left) 2026-02-17T06:07:38.839 INFO:tasks.rados.rados.0.trial115.stdout:3453: done (1 left) 2026-02-17T06:07:38.839 INFO:tasks.rados.rados.0.trial115.stdout:3454: snap_remove snap 349 2026-02-17T06:07:39.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:39.373 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 10 monclient: get_auth_request con 0x7f51d41cef30 auth_method 0 2026-02-17T06:07:39.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:39.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:39.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:39.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:07:39.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:07:39.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:07:39.374 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:39.372+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:07:39.681 INFO:tasks.rados.rados.0.trial115.stdout:3454: done (1 left) 2026-02-17T06:07:39.681 INFO:tasks.rados.rados.0.trial115.stdout:3455: rollback oid 8 current snap is 364 2026-02-17T06:07:39.681 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 8 to 353 2026-02-17T06:07:39.681 INFO:tasks.rados.rados.0.trial115.stdout:3451: finishing copy_from racing read to trial11545420-6 2026-02-17T06:07:39.681 INFO:tasks.rados.rados.0.trial115.stdout:3451: finishing copy_from to trial11545420-6 2026-02-17T06:07:39.681 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 1490 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:07:39.681 INFO:tasks.rados.rados.0.trial115.stdout:3451: done (1 left) 2026-02-17T06:07:39.682 INFO:tasks.rados.rados.0.trial115.stdout:3456: rollback oid 47 current snap is 364 2026-02-17T06:07:39.682 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 47 to 360 2026-02-17T06:07:39.682 INFO:tasks.rados.rados.0.trial115.stdout:3457: write (excl) oid 18 current snap is 364 2026-02-17T06:07:39.682 INFO:tasks.rados.rados.0.trial115.stdout:3457: seq_num 1189 ranges {783523=422129,1827443=621747,3218409=654188} 2026-02-17T06:07:39.685 INFO:tasks.rados.rados.0.trial115.stdout:3457: writing trial11545420-18 from 783523 to 1205652 tid 1 2026-02-17T06:07:39.689 INFO:tasks.rados.rados.0.trial115.stdout:3457: writing trial11545420-18 from 1827443 to 2449190 tid 2 2026-02-17T06:07:39.695 INFO:tasks.rados.rados.0.trial115.stdout:3457: writing trial11545420-18 from 3218409 to 3872597 tid 3 2026-02-17T06:07:39.695 INFO:tasks.rados.rados.0.trial115.stdout:3458: write oid 21 current snap is 364 2026-02-17T06:07:39.695 INFO:tasks.rados.rados.0.trial115.stdout:3458: seq_num 1190 ranges {400330=497378,1485925=514405} 2026-02-17T06:07:39.698 INFO:tasks.rados.rados.0.trial115.stdout:3458: writing trial11545420-21 from 400330 to 897708 tid 1 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:3458: writing trial11545420-21 from 1485925 to 2000330 tid 2 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:3459: read oid 26 snap -1 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:3459: expect deleted 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:3460: copy_from oid 14 from oid 5 current snap is 364 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:3455: finishing rollback tid 1 to trial11545420-8 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:3456: finishing rollback tid 0 to trial11545420-47 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:3456: finishing rollback tid 1 to trial11545420-47 2026-02-17T06:07:39.704 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1573 (ObjNum 817 snap 0 seq_num 1715449888) dirty dne 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3457: finishing write tid 1 to trial11545420-18 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3455: done (5 left) 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3456: done (4 left) 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3461: rollback oid 37 current snap is 364 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 353 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3457: finishing write tid 2 to trial11545420-18 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3457: finishing write tid 3 to trial11545420-18 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3457: finishing write tid 4 to trial11545420-18 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3457: finishing write tid 5 to trial11545420-18 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3457: oid 18 updating version 0 to 1581 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3457: oid 18 version 1581 is already newer than 1578 2026-02-17T06:07:39.705 INFO:tasks.rados.rados.0.trial115.stdout:3457: oid 18 version 1581 is already newer than 1580 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3457: oid 18 version 1581 is already newer than 1579 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1581 (ObjNum 1189 snap 364 seq_num 1189) dirty exists 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3457: left oid 18 (ObjNum 1189 snap 364 seq_num 1189) 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3458: finishing write tid 1 to trial11545420-21 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3457: done (4 left) 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3462: rollback oid 27 current snap is 364 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 353 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3463: read oid 22 snap -1 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3463: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3464: snap_create 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:3461: finishing rollback tid 1 to trial11545420-37 2026-02-17T06:07:39.706 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1704 (ObjNum 129 snap 0 seq_num 1334318744) dirty dne 2026-02-17T06:07:39.707 INFO:tasks.rados.rados.0.trial115.stdout:3458: finishing write tid 2 to trial11545420-21 2026-02-17T06:07:39.707 INFO:tasks.rados.rados.0.trial115.stdout:3458: finishing write tid 3 to trial11545420-21 2026-02-17T06:07:39.707 INFO:tasks.rados.rados.0.trial115.stdout:3458: finishing write tid 4 to trial11545420-21 2026-02-17T06:07:39.707 INFO:tasks.rados.rados.0.trial115.stdout:3458: oid 21 updating version 0 to 1582 2026-02-17T06:07:39.707 INFO:tasks.rados.rados.0.trial115.stdout:3458: oid 21 updating version 1582 to 1583 2026-02-17T06:07:39.708 INFO:tasks.rados.rados.0.trial115.stdout:3458: oid 21 updating version 1583 to 1584 2026-02-17T06:07:39.708 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1584 (ObjNum 1190 snap 364 seq_num 1190) dirty exists 2026-02-17T06:07:39.708 INFO:tasks.rados.rados.0.trial115.stdout:3458: left oid 21 (ObjNum 1190 snap 364 seq_num 1190) 2026-02-17T06:07:39.708 INFO:tasks.rados.rados.0.trial115.stdout:3462: finishing rollback tid 0 to trial11545420-27 2026-02-17T06:07:39.710 INFO:tasks.rados.rados.0.trial115.stdout:3462: finishing rollback tid 1 to trial11545420-27 2026-02-17T06:07:39.710 INFO:tasks.rados.rados.0.trial115.stdout:3462: finishing rollback tid 2 to trial11545420-27 2026-02-17T06:07:39.710 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1467 (ObjNum 1146 snap 349 seq_num 1146) dirty exists 2026-02-17T06:07:39.711 INFO:tasks.rados.rados.0.trial115.stdout:3463: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:07:39.877 INFO:tasks.rados.rados.0.trial115.stdout:3460: finishing copy_from racing read to trial11545420-14 2026-02-17T06:07:39.877 INFO:tasks.rados.rados.0.trial115.stdout:3460: finishing copy_from to trial11545420-14 2026-02-17T06:07:39.877 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1484 (ObjNum 1125 snap 336 seq_num 1125) dirty exists 2026-02-17T06:07:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.967+0000 7f51c0ff9640 10 monclient: tick 2026-02-17T06:07:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.967+0000 7f51c0ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:07:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.967+0000 7f51c0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:40.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.967+0000 7f51c0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 10 monclient: get_auth_request con 0x7f51d41cef30 auth_method 0 2026-02-17T06:07:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:07:40.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:40.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:40.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:40.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 10 monclient(hunting): handle_auth_done global_id 264101 payload 824 2026-02-17T06:07:40.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.974+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51d409f290 con 0x7f51d41cef30 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51c4007d90 tx=0x7f51c4012d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c401c400 con 0x7f51d41cef30 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 10 monclient: dump: 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:40.978 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f51c4014400 con 0x7f51d41cef30 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:40.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51c4007d90 tx=0x7f51c4012d60 comp rx=0 tx=0).stop 2026-02-17T06:07:40.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 msgr2=0x7f51d41cf310 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:40.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.976+0000 7f51c1ffb640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cef30 0x7f51d41cf310 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51c4007d90 tx=0x7f51c4012d60 comp rx=0 tx=0).stop 2026-02-17T06:07:40.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51db8c4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:40.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51db8c4640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cf850 0x7f51d41da280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:40.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51db8c4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:40.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f51d40a9af0 con 0x7f51d41cf850 2026-02-17T06:07:40.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cf850 0x7f51d41da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:40.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 10 monclient: get_auth_request con 0x7f51d41cf850 auth_method 0 2026-02-17T06:07:40.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:40.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:40.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:40.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:40.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:40.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:40.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 10 monclient(hunting): handle_auth_done global_id 264104 payload 824 2026-02-17T06:07:40.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.980+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f51d40744d0 con 0x7f51d41cf850 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51d9639640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cf850 msgr2=0x7f51d41da280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51d9639640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cf850 0x7f51d41da280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f51c4007d90 tx=0x7f51c4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c40145b0 con 0x7f51d41cf850 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 10 monclient: dump: 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:40.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f51c40145b0 con 0x7f51d41cf850 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:40.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cf850 msgr2=0x7f51d41da280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:40.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cf850 0x7f51d41da280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f51c4007d90 tx=0x7f51c4004a40 comp rx=0 tx=0).stop 2026-02-17T06:07:40.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cf850 msgr2=0x7f51d41da280 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:40.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51c1ffb640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f51d41cf850 0x7f51d41da280 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f51c4007d90 tx=0x7f51c4004a40 comp rx=0 tx=0).stop 2026-02-17T06:07:40.987 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:40.987 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:40.987 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:40.987 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 246, 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 10368, 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:40.988 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:40.989 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:40.990 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:40.991 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:40.992 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:40.993 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:40.994 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.995 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:40.996 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:40.997 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:40.998 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:40.999 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:41.000 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:41.001 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:41.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51a403deb0 msgr2=0x7f51a4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f51a403deb0 0x7f51a4040370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f51d40a9ed0 tx=0x7f51c8008040 comp rx=0 tx=0).stop 2026-02-17T06:07:41.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 10 monclient: shutdown 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f51d413d640 msgr2=0x7f51d413c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f51d413d640 0x7f51d413c300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51d0036000 tx=0x7f51d0004b40 comp rx=0 tx=0).stop 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51d9e3a640 1 -- 10.20.193.78:0/975268336 reap_dead start 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 shutdown_connections 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 --2- 10.20.193.78:0/975268336 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f51d413d640 0x7f51d413c300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 >> 10.20.193.78:0/975268336 conn(0x7f51d40a7520 msgr2=0x7f51d4136d20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 shutdown_connections 2026-02-17T06:07:41.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:40.983+0000 7f51db8c4640 1 -- 10.20.193.78:0/975268336 wait complete. 2026-02-17T06:07:41.005 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:07:41.005 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:07:41.005 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:07:41.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.072+0000 7fede33a9640 10 monclient: get_monmap_and_config 2026-02-17T06:07:41.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.072+0000 7fede33a9640 10 monclient: build_initial_monmap 2026-02-17T06:07:41.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.072+0000 7fede33a9640 10 monclient: monmap: 2026-02-17T06:07:41.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:41.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:41.073729+0000 2026-02-17T06:07:41.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:41.073729+0000 2026-02-17T06:07:41.075 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:41.075 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.072+0000 7fede33a9640 1 Processor -- start 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.072+0000 7fede33a9640 1 -- start start 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.072+0000 7fede33a9640 10 monclient: init 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:41.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 10 monclient: _add_conns ranks=[0,3,1,4,2] 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feddc06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7feddc09c4a0 0x7feddc13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 10 monclient(hunting): picked mon.noname-a con 0x7feddc09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7feddc06d380 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feddc06daf0 con 0x7feddc06c890 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feddc09cc80 con 0x7feddc06d380 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7feddc0a0880 con 0x7feddc09c4a0 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede33a9640 10 monclient(hunting): authenticate will time out at 1906.627331s 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede111e640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7feddc06c890 0x7feddc06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38128/0 (socket says 10.20.193.78:38128) 2026-02-17T06:07:41.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede111e640 1 -- 10.20.193.78:0/3012840220 learned_addr learned my addr 10.20.193.78:0/3012840220 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede091d640 1 --2- 10.20.193.78:0/3012840220 >> v2:10.20.193.78:3300/0 conn(0x7feddc09c4a0 0x7feddc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede091d640 10 monclient(hunting): get_auth_request con 0x7feddc09c4a0 auth_method 0 2026-02-17T06:07:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede091d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede091d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.073+0000 7fede091d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede111e640 1 -- 10.20.193.78:0/3012840220 >> v1:10.20.193.78:6789/0 conn(0x7feddc06c890 legacy=0x7feddc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 10 monclient(hunting): handle_auth_done global_id 254208 payload 274 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 1 -- 10.20.193.78:0/3012840220 >> v1:10.20.193.115:6789/0 conn(0x7feddc06d380 legacy=0x7feddc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 1 -- 10.20.193.78:0/3012840220 >> v1:10.20.193.78:6789/0 conn(0x7feddc06c890 legacy=0x7feddc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 10 monclient: _finish_hunting 0 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 1 monclient: found mon.noname-a 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:41.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.075+0000 7fede091d640 1 -- 10.20.193.78:0/3012840220 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc09cf60 con 0x7feddc09c4a0 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede091d640 1 -- 10.20.193.78:0/3012840220 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 msgr2=0x7feddc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede091d640 1 --2- 10.20.193.78:0/3012840220 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 0x7feddc13b3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feddc09c980 tx=0x7fedd002ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4b23e361168b68 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 1 -- 10.20.193.78:0/3012840220 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd003c070 con 0x7feddc09c4a0 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: dump: 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.088 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: _finish_auth 0 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: _check_auth_tickets 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 1 -- 10.20.193.78:0/3012840220 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd002fd90 con 0x7feddc09c4a0 2026-02-17T06:07:41.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 5 monclient: authenticate success, global_id 254208 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 1 -- 10.20.193.78:0/3012840220 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0037660 con 0x7feddc09c4a0 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fedcbfff640 10 monclient: dump: 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.090 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 4 set_mon_vals no callback set 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 10 monclient: shutdown 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 1 -- 10.20.193.78:0/3012840220 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 msgr2=0x7feddc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.091 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 1 --2- 10.20.193.78:0/3012840220 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 0x7feddc13b3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feddc09c980 tx=0x7fedd002ede0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 1 -- 10.20.193.78:0/3012840220 shutdown_connections 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 1 --2- 10.20.193.78:0/3012840220 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 0x7feddc13b3a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 1 -- 10.20.193.78:0/3012840220 >> 10.20.193.78:0/3012840220 conn(0x7feddc0a6620 msgr2=0x7feddc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 1 -- 10.20.193.78:0/3012840220 shutdown_connections 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.078+0000 7fede33a9640 1 -- 10.20.193.78:0/3012840220 wait complete. 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient: build_initial_monmap 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient: monmap: 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:41.092 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 1 Processor -- start 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 1 -- start start 2026-02-17T06:07:41.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient: init 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feddc06c890 0x7feddc13da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient(hunting): picked mon.noname-c con 0x7feddc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc06d380 0x7feddc132570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient(hunting): picked mon.noname-b con 0x7feddc06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 0x7feddc09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient(hunting): picked mon.noname-a con 0x7feddc09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7feddc09bfa0 con 0x7feddc09c4a0 2026-02-17T06:07:41.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7feddc09be20 con 0x7feddc06d380 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7feddc13f010 con 0x7feddc06c890 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede33a9640 10 monclient(hunting): authenticate will time out at 1906.633700s 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 0x7feddc09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede111e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feddc06c890 0x7feddc13da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 0x7feddc09db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40886/0 (socket says 10.20.193.78:40886) 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 -- 10.20.193.78:0/2964526656 learned_addr learned my addr 10.20.193.78:0/2964526656 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient(hunting): get_auth_request con 0x7feddc09c4a0 auth_method 0 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede111e640 10 monclient(hunting): get_auth_request con 0x7feddc06c890 auth_method 0 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede111e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede111e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede111e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient(hunting): handle_auth_done global_id 254214 payload 824 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feddc06c890 msgr2=0x7feddc13da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feddc06c890 0x7feddc13da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc06d380 msgr2=0x7feddc132570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc06d380 0x7feddc132570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient: _finish_hunting 0 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 monclient: found mon.noname-a 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.079+0000 7fede191f640 1 -- 10.20.193.78:0/2964526656 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc09ed10 con 0x7feddc09c4a0 2026-02-17T06:07:41.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede191f640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 msgr2=0x7feddc09db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede191f640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 0x7feddc09db20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fedd800cb60 tx=0x7fedd800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8016020 con 0x7feddc09c4a0 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: dump: 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.098 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8012420 con 0x7feddc09c4a0 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8012810 con 0x7feddc09c4a0 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedc9ffb640 10 monclient: dump: 2026-02-17T06:07:41.099 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fedcb7fe640 4 set_mon_vals no callback set 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 5 monclient: authenticate success, global_id 254214 2026-02-17T06:07:41.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 10 monclient: _renew_subs 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc09d810 con 0x7feddc09c4a0 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 10 monclient: _renew_subs 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc06fd00 con 0x7feddc09c4a0 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc132ab0 0x7feddc11dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7feddc0a3fb0 con 0x7feddc132ab0 2026-02-17T06:07:41.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede111e640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc132ab0 0x7feddc11dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede111e640 10 monclient: get_auth_request con 0x7feddc132ab0 auth_method 0 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede111e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede111e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede111e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.082+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.083+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fedd8012a30 con 0x7feddc09c4a0 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.083+0000 7fede111e640 10 monclient(hunting): handle_auth_done global_id 214468 payload 824 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.083+0000 7fede111e640 1 -- 10.20.193.78:0/2964526656 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feddc06d1a0 con 0x7feddc132ab0 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.083+0000 7fedc9ffb640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fedac03df00 0x7fedac0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.083+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(758..758 src has 525..758) ==== 4783+0+0 (secure 0 0 0) 0x7fedd8053120 con 0x7feddc09c4a0 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.083+0000 7fede091d640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fedac03df00 0x7fedac0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.083+0000 7fede091d640 10 monclient: get_auth_request con 0x7fedac03df00 auth_method 0 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fede111e640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc132ab0 msgr2=0x7feddc11dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fede111e640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc132ab0 0x7feddc11dbb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00f770 tx=0x7fedcc00fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc005bc0 con 0x7feddc132ab0 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 10 monclient: dump: 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.103 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fedcc018070 con 0x7feddc132ab0 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc132ab0 msgr2=0x7feddc11dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc132ab0 0x7feddc11dbb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00f770 tx=0x7fedcc00fc40 comp rx=0 tx=0).stop 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc132ab0 msgr2=0x7feddc11dbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.085+0000 7fedc9ffb640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc132ab0 0x7feddc11dbb0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00f770 tx=0x7fedcc00fc40 comp rx=0 tx=0).stop 2026-02-17T06:07:41.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.086+0000 7fede091d640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fedac03df00 0x7fedac0403c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feddc06a480 tx=0x7fedd0031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:41.107 INFO:tasks.rados.rados.0.trial115.stdout:3458: done (6 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3459: done (5 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3460: done (4 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3461: done (3 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3462: done (2 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3463: done (1 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3464: done (0 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3465: delete oid 3 current snap is 365 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3465: done (0 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3466: delete oid 29 current snap is 365 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3466: done (0 left) 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3467: read oid 46 snap -1 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3467: expect deleted 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3468: read oid 30 snap -1 2026-02-17T06:07:41.108 INFO:tasks.rados.rados.0.trial115.stdout:3468: expect deleted 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3469: read oid 43 snap -1 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3469: expect (ObjNum 1141 snap 347 seq_num 1141) 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3470: read oid 26 snap -1 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3470: expect deleted 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3471: delete oid 1 current snap is 365 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3467: done (4 left) 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3468: done (3 left) 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3470: done (2 left) 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3471: done (1 left) 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3472: write (excl) oid 37 current snap is 365 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3472: seq_num 1194 ranges {467558=514727,1719853=347705} 2026-02-17T06:07:41.109 INFO:tasks.rados.rados.0.trial115.stdout:3472: writing trial11545420-37 from 467558 to 982285 tid 1 2026-02-17T06:07:41.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.088+0000 7fede33a9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:41.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.088+0000 7fede33a9640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc134110 0x7feddc1da870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.088+0000 7fede33a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.088+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7feddc06cc70 con 0x7feddc134110 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc134110 0x7feddc1da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 10 monclient: get_auth_request con 0x7feddc134110 auth_method 0 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 10 monclient(hunting): handle_auth_done global_id 214471 payload 824 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.089+0000 7fede111e640 1 -- 10.20.193.78:0/2964526656 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feddc11d790 con 0x7feddc134110 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.091+0000 7fede111e640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc134110 msgr2=0x7feddc1da870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.091+0000 7fede111e640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc134110 0x7feddc1da870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fedcc017000 tx=0x7fedcc007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:41.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.091+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01ee60 con 0x7feddc134110 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.091+0000 7fedc9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.091+0000 7fedc9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.091+0000 7fedc9ffb640 10 monclient: dump: 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7fedcc0072e0 con 0x7feddc134110 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fedc9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fedc9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc134110 msgr2=0x7feddc1da870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fedc9ffb640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc134110 0x7feddc1da870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fedcc017000 tx=0x7fedcc007f90 comp rx=0 tx=0).stop 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fedc9ffb640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc134110 msgr2=0x7feddc1da870 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fedc9ffb640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feddc134110 0x7feddc1da870 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fedcc017000 tx=0x7fedcc007f90 comp rx=0 tx=0).stop 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:41.112 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 246, 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 263939, 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:41.113 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:41.114 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:41.115 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:41.116 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.117 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:41.118 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.119 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:41.120 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.121 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:41.122 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:41.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fedac03df00 msgr2=0x7fedac0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fedac03df00 0x7fedac0403c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feddc06a480 tx=0x7fedd0031000 comp rx=0 tx=0).stop 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 10 monclient: shutdown 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 msgr2=0x7feddc09db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 1 --2- 10.20.193.78:0/2964526656 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feddc09c4a0 0x7feddc09db20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fedd800cb60 tx=0x7fedd800a430 comp rx=0 tx=0).stop 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede191f640 1 -- 10.20.193.78:0/2964526656 reap_dead start 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede191f640 1 -- 10.20.193.78:0/2964526656 reap_dead start 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 shutdown_connections 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 >> 10.20.193.78:0/2964526656 conn(0x7feddc0a6620 msgr2=0x7feddc0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 shutdown_connections 2026-02-17T06:07:41.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.092+0000 7fede33a9640 1 -- 10.20.193.78:0/2964526656 wait complete. 2026-02-17T06:07:41.124 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:07:41.124 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:07:41.125 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:07:41.125 INFO:tasks.rados.rados.0.trial115.stdout:3472: writing trial11545420-37 from 1719853 to 2067558 tid 2 2026-02-17T06:07:41.125 INFO:tasks.rados.rados.0.trial115.stdout:3473: snap_create 2026-02-17T06:07:41.125 INFO:tasks.rados.rados.0.trial115.stdout:3469: expect (ObjNum 1141 snap 347 seq_num 1141) 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient: get_monmap_and_config 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient: build_initial_monmap 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient: monmap: 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:41.193244+0000 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:41.193244+0000 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.193 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:41.194 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:41.194 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:41.194 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:41.194 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:41.194 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.195 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 1 Processor -- start 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 1 -- start start 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient: init 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f48e409ea90 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f48e409ee70 0x7f48e409f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient(hunting): picked mon.noname-c con 0x7f48e409ee70 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient(hunting): picked mon.noname-b con 0x7f48e413d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48e40a0a20 con 0x7f48e409ea90 2026-02-17T06:07:41.196 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f48e40a0d00 con 0x7f48e409ee70 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f48e40a0460 con 0x7f48e413d640 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48ec86c640 10 monclient(hunting): authenticate will time out at 1906.746834s 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48eade2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48eade2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e413fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48944/0 (socket says 10.20.193.78:48944) 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48eade2640 1 -- 10.20.193.78:0/2875310774 learned_addr learned my addr 10.20.193.78:0/2875310774 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient(hunting): get_auth_request con 0x7f48e413d640 auth_method 0 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.192+0000 7f48e9de0640 1 --2- 10.20.193.78:0/2875310774 >> v2:10.20.193.115:3300/0 conn(0x7f48e409ee70 0x7f48e409f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48e9de0640 10 monclient(hunting): get_auth_request con 0x7f48e409ee70 auth_method 0 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48e9de0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48e9de0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48e9de0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48e9de0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48e9de0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48e9de0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient(hunting): handle_auth_done global_id 264110 payload 274 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 1 -- 10.20.193.78:0/2875310774 >> v2:10.20.193.115:3300/0 conn(0x7f48e409ee70 msgr2=0x7f48e409f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 1 --2- 10.20.193.78:0/2875310774 >> v2:10.20.193.115:3300/0 conn(0x7f48e409ee70 0x7f48e409f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 1 -- 10.20.193.78:0/2875310774 >> v1:10.20.193.115:6789/0 conn(0x7f48e409ea90 legacy=0x7f48e40a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient: _finish_hunting 0 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 1 monclient: found mon.noname-b 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:41.198 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:07:41.199 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.193+0000 7f48eade2640 1 -- 10.20.193.78:0/2875310774 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e40a0740 con 0x7f48e413d640 2026-02-17T06:07:41.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48eade2640 1 -- 10.20.193.78:0/2875310774 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 msgr2=0x7f48e413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48eade2640 1 --2- 10.20.193.78:0/2875310774 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e413fa30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f48e406e270 tx=0x7f48d802f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a564ebd773de29a3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 1 -- 10.20.193.78:0/2875310774 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d802fe60 con 0x7f48e413d640 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: dump: 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.204 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: _finish_auth 0 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: _check_auth_tickets 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48ec86c640 5 monclient: authenticate success, global_id 264110 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48ec86c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 1 -- 10.20.193.78:0/2875310774 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d80373f0 con 0x7f48e413d640 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 1 -- 10.20.193.78:0/2875310774 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d80377c0 con 0x7f48e413d640 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48e95df640 10 monclient: dump: 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.205 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48ec86c640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.195+0000 7f48ec86c640 4 set_mon_vals no callback set 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 10 monclient: shutdown 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 -- 10.20.193.78:0/2875310774 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 msgr2=0x7f48e413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 --2- 10.20.193.78:0/2875310774 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e413fa30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f48e406e270 tx=0x7f48d802f230 comp rx=0 tx=0).stop 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 -- 10.20.193.78:0/2875310774 shutdown_connections 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 --2- 10.20.193.78:0/2875310774 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e413fa30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 --2- 10.20.193.78:0/2875310774 >> v2:10.20.193.115:3300/0 conn(0x7f48e409ee70 0x7f48e409f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 -- 10.20.193.78:0/2875310774 >> 10.20.193.78:0/2875310774 conn(0x7f48e40a7520 msgr2=0x7f48e40a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 -- 10.20.193.78:0/2875310774 shutdown_connections 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 -- 10.20.193.78:0/2875310774 wait complete. 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 10 monclient: build_initial_monmap 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 10 monclient: monmap: 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:41.207 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 Processor -- start 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 1 -- start start 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.196+0000 7f48ec86c640 10 monclient: init 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48e409ea90 0x7f48e41cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient(hunting): picked mon.noname-b con 0x7f48e409ea90 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48e409ee70 0x7f48e40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient(hunting): picked mon.noname-a con 0x7f48e409ee70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e4073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient(hunting): picked mon.noname-c con 0x7f48e413d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48e40a6820 con 0x7f48e409ee70 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48e40a66a0 con 0x7f48e409ea90 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:41.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f48e413ff70 con 0x7f48e413d640 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ec86c640 10 monclient(hunting): authenticate will time out at 1906.751223s 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48e9de0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48e409ee70 0x7f48e40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e4073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e4073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48948/0 (socket says 10.20.193.78:48948) 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 -- 10.20.193.78:0/1046816308 learned_addr learned my addr 10.20.193.78:0/1046816308 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient(hunting): get_auth_request con 0x7f48e413d640 auth_method 0 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48e9de0640 10 monclient(hunting): get_auth_request con 0x7f48e409ee70 auth_method 0 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48e9de0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48e9de0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48e9de0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48e9de0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48e9de0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48e9de0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ea5e1640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48e409ea90 0x7f48e41cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ea5e1640 10 monclient(hunting): get_auth_request con 0x7f48e409ea90 auth_method 0 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ea5e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ea5e1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48ea5e1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient(hunting): handle_auth_done global_id 264113 payload 824 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 -- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48e409ea90 msgr2=0x7f48e41cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48e409ea90 0x7f48e41cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 -- 10.20.193.78:0/1046816308 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48e409ee70 msgr2=0x7f48e40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48e409ee70 0x7f48e40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient: _finish_hunting 0 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 monclient: found mon.noname-c 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:41.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.197+0000 7f48eade2640 1 -- 10.20.193.78:0/1046816308 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e4070ec0 con 0x7f48e413d640 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48eade2640 1 -- 10.20.193.78:0/1046816308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 msgr2=0x7f48e4073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48eade2640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e4073c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009f50 tx=0x7f48d8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 1 -- 10.20.193.78:0/1046816308 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8046020 con 0x7f48e413d640 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: dump: 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: _finish_auth 0 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: _check_auth_tickets 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 1 -- 10.20.193.78:0/1046816308 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d8041870 con 0x7f48e413d640 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ec86c640 5 monclient: authenticate success, global_id 264113 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 1 -- 10.20.193.78:0/1046816308 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8041c40 con 0x7f48e413d640 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ceffd640 10 monclient: dump: 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:41.212 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48e8dde640 4 set_mon_vals no callback set 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ec86c640 10 monclient: _renew_subs 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ec86c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e406fe60 con 0x7f48e413d640 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ec86c640 10 monclient: _renew_subs 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ec86c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.199+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e4070be0 con 0x7f48e413d640 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ceffd640 1 -- 10.20.193.78:0/1046816308 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f48d8042400 con 0x7f48e413d640 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ec86c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ec86c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ec86c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e409f2c0 con 0x7f48e413d640 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ceffd640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48b403deb0 0x7f48b4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ceffd640 1 -- 10.20.193.78:0/1046816308 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(758..758 src has 525..758) ==== 4783+0+0 (secure 0 0 0) 0x7f48d8078620 con 0x7f48e413d640 2026-02-17T06:07:41.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ea5e1640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48b403deb0 0x7f48b4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.200+0000 7f48ea5e1640 10 monclient: get_auth_request con 0x7f48b403deb0 auth_method 0 2026-02-17T06:07:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.203+0000 7f48ea5e1640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48b403deb0 0x7f48b4040370 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f48dc005020 tx=0x7f48dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.203+0000 7f48ceffd640 1 -- 10.20.193.78:0/1046816308 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f48d8050030 con 0x7f48e413d640 2026-02-17T06:07:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.203+0000 7f48ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:41.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.203+0000 7f48ceffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:41.257 INFO:tasks.rados.rados.0.trial115.stdout:3472: finishing write tid 1 to trial11545420-37 2026-02-17T06:07:41.257 INFO:tasks.rados.rados.0.trial115.stdout:3472: finishing write tid 2 to trial11545420-37 2026-02-17T06:07:41.258 INFO:tasks.rados.rados.0.trial115.stdout:3472: finishing write tid 3 to trial11545420-37 2026-02-17T06:07:41.258 INFO:tasks.rados.rados.0.trial115.stdout:3472: finishing write tid 4 to trial11545420-37 2026-02-17T06:07:41.258 INFO:tasks.rados.rados.0.trial115.stdout:3472: oid 37 updating version 0 to 1706 2026-02-17T06:07:41.258 INFO:tasks.rados.rados.0.trial115.stdout:3472: oid 37 updating version 1706 to 1707 2026-02-17T06:07:41.258 INFO:tasks.rados.rados.0.trial115.stdout:3472: oid 37 version 1707 is already newer than 1705 2026-02-17T06:07:41.258 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1707 (ObjNum 1194 snap 365 seq_num 1194) dirty exists 2026-02-17T06:07:41.258 INFO:tasks.rados.rados.0.trial115.stdout:3472: left oid 37 (ObjNum 1194 snap 365 seq_num 1194) 2026-02-17T06:07:41.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.318+0000 7f48ec86c640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:07:41.319 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.318+0000 7f48ec86c640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:07:41.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.318+0000 7f48ec86c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:41.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.318+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f48e406f520 con 0x7f48e413d640 2026-02-17T06:07:41.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.319+0000 7f48ceffd640 1 -- 10.20.193.78:0/1046816308 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f48d8041e20 con 0x7f48e413d640 2026-02-17T06:07:41.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.319+0000 7f48ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:07:41.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.319+0000 7f48ceffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:41.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48b403deb0 msgr2=0x7f48b4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48b403deb0 0x7f48b4040370 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f48dc005020 tx=0x7f48dc009040 comp rx=0 tx=0).stop 2026-02-17T06:07:41.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 10 monclient: shutdown 2026-02-17T06:07:41.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:41.322 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 msgr2=0x7f48e4073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e4073c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009f50 tx=0x7f48d8004b40 comp rx=0 tx=0).stop 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 shutdown_connections 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f48e413d640 0x7f48e4073c20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f48e409ee70 0x7f48e40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f48b403deb0 0x7f48b4040370 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 --2- 10.20.193.78:0/1046816308 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f48e409ea90 0x7f48e41cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 >> 10.20.193.78:0/1046816308 conn(0x7f48e40a7520 msgr2=0x7f48e4136d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 shutdown_connections 2026-02-17T06:07:41.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:41.320+0000 7f48ec86c640 1 -- 10.20.193.78:0/1046816308 wait complete. 2026-02-17T06:07:41.329 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:07:41.635 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.633+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.635 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.633+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=12,kv=12,logm=49} crc {auth=776483463,config=692619927,health=3219210987,kv=449989585,logm=3248039913}) 2026-02-17T06:07:41.635 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.633+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=25,config=2,health=12,kv=12,logm=49} crc {auth=25427001,config=692619927,health=3219210987,kv=2165459515,logm=3236802371}) 2026-02-17T06:07:41.635 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.633+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.635 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.633+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=12,kv=12,logm=49} crc {auth=776483463,config=692619927,health=3219210987,kv=449989585,logm=3248039913}) 2026-02-17T06:07:41.635 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.633+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=26,config=2,health=12,kv=11,logm=49} crc {auth=1950317538,config=692619927,health=3219210987,kv=3497084452,logm=3604614511}) 2026-02-17T06:07:41.635 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.636 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934053389}) 2026-02-17T06:07:41.636 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=177873419}) 2026-02-17T06:07:41.636 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.636 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934053389}) 2026-02-17T06:07:41.636 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654086042}) 2026-02-17T06:07:41.657 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1509026265}) 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2459830473}) 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1509026265}) 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.634+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2657672635}) 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269794817}) 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=289701153}) 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.658 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269794817}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2201037557}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2217285495}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=925100217}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2217285495}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.635+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3458732371}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2631741029}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=299938157}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2631741029}) 2026-02-17T06:07:41.659 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=491257018}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735124228}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3345781735}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735124228}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.636+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854871266}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1868006723}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511232104}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1868006723}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=754428014}) 2026-02-17T06:07:41.660 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=726827917}) 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1760643304}) 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=726827917}) 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1956486533}) 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926971883}) 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470893367}) 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926971883}) 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.637+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053209700}) 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.661 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1713795639}) 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3931653259}) 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1713795639}) 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=908797912}) 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2257382963}) 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1613257999}) 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2257382963}) 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.638+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3968673420}) 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.662 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3043996370}) 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=474646789}) 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3043996370}) 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2362641345}) 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1232135394}) 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239603893}) 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1232135394}) 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=366929234}) 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.663 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=38270417}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=613222895}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=38270417}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.639+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1871486554}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4132068157}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3600728836}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4132068157}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2594086307}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323795326}) 2026-02-17T06:07:41.664 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3316655224}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323795326}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.640+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2010884854}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459356116}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1734817542}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459356116}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2494776519}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2243469994}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4075564131}) 2026-02-17T06:07:41.665 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2243469994}) 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=27256227}) 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1812474111}) 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1511286672}) 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1812474111}) 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.641+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3780211699}) 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4215594283}) 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1688834108}) 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.666 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4215594283}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3480249041}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1934267426}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1660139520}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1934267426}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.642+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1268589810}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279524579}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1083725426}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279524579}) 2026-02-17T06:07:41.667 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2321025465}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=399666408}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=880638892}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=399666408}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4164226777}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3560212633}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3326871331}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3560212633}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.643+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2140396482}) 2026-02-17T06:07:41.668 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931269689}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3075354724}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931269689}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4108550608}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2188108295}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4238100547}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2188108295}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.644+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2439986832}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=758055233}) 2026-02-17T06:07:41.669 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2186404314}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=758055233}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3199701516}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2272022986}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3132463744}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2272022986}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1191882938}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3998145498}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=698500230}) 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.670 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3998145498}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.645+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=52565661}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208609118}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1220521383}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208609118}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321227577}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662517033}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3633998521}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662517033}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.646+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3676448045}) 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.671 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2775473716}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1988804197}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2775473716}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2038140811}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1287118710}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2416338059}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1287118710}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3716681533}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1041088859}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2282091345}) 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1041088859}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.647+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3177399702}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1521213290}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=74485944}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1521213290}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3862193569}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727967104}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4031371904}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727967104}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2359932625}) 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=241330883}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=703695006}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.648+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=241330883}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272011805}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2330080218}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2163260245}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2330080218}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284437090}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2916601427}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1636409720}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2916601427}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.649+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=95863655}) 2026-02-17T06:07:41.674 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681450182}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4076745340}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681450182}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664870300}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424930780}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=915214185}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424930780}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.650+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=445544855}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4172375849}) 2026-02-17T06:07:41.675 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3988755092}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4172375849}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=389528586}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3876601981}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465897312}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3876601981}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3237919389}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2463879850}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=595489724}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2463879850}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.651+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2721945725}) 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.676 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3078321198}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3282794944}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3078321198}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=565050122}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3870298309}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2249139234}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3870298309}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.652+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=757780963}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474207689}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513176818}) 2026-02-17T06:07:41.677 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474207689}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3888978735}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950952768}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=635149836}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950952768}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2193525712}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3792399740}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1525861150}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3792399740}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.653+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2809068865}) 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.678 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399964168}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3077881689}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399964168}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180313066}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265855487}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1443080316}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265855487}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=526799333}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1368830749}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1610579402}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1368830749}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.654+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=447616382}) 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.679 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493261297}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3510469173}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493261297}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3910767807}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3453641564}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=997797840}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3453641564}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4011751607}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1959552097}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2713854003}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1959552097}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.655+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1037583398}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.656+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.656+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=763036689}) 2026-02-17T06:07:41.680 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.656+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2709619693}) 2026-02-17T06:07:41.681 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.656+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.681 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.656+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=763036689}) 2026-02-17T06:07:41.681 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.656+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1337046431}) 2026-02-17T06:07:41.681 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.681 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499304061}) 2026-02-17T06:07:41.681 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3441689836}) 2026-02-17T06:07:41.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499304061}) 2026-02-17T06:07:41.682 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2628769330}) 2026-02-17T06:07:41.683 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.682+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.683 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.682+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2157595476}) 2026-02-17T06:07:41.683 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.682+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3391887082}) 2026-02-17T06:07:41.683 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.682+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.683 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.682+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2157595476}) 2026-02-17T06:07:41.683 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.682+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3360269517}) 2026-02-17T06:07:41.689 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.688+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.689 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.688+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589260329}) 2026-02-17T06:07:41.689 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.688+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=160245656}) 2026-02-17T06:07:41.689 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.688+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.689 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.688+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589260329}) 2026-02-17T06:07:41.689 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.688+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3261219717}) 2026-02-17T06:07:41.690 INFO:tasks.rados.rados.0.trial115.stdout:3469: done (2 left) 2026-02-17T06:07:41.690 INFO:tasks.rados.rados.0.trial115.stdout:3472: done (1 left) 2026-02-17T06:07:41.690 INFO:tasks.rados.rados.0.trial115.stdout:3473: done (0 left) 2026-02-17T06:07:41.690 INFO:tasks.rados.rados.0.trial115.stdout:3474: snap_remove snap 353 2026-02-17T06:07:41.691 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.690+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.691 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.690+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2346703356}) 2026-02-17T06:07:41.691 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.690+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3026092617}) 2026-02-17T06:07:41.691 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.690+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:41.691 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.690+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2346703356}) 2026-02-17T06:07:41.692 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:41.690+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=885108518}) 2026-02-17T06:07:42.208 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.208 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2460423642}) 2026-02-17T06:07:42.208 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2194015477}) 2026-02-17T06:07:42.208 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2460423642}) 2026-02-17T06:07:42.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375406829}) 2026-02-17T06:07:42.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2185923059}) 2026-02-17T06:07:42.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3241723433}) 2026-02-17T06:07:42.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2185923059}) 2026-02-17T06:07:42.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=881219042}) 2026-02-17T06:07:42.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1475948469}) 2026-02-17T06:07:42.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1909064441}) 2026-02-17T06:07:42.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1475948469}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1523888066}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713373856}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2941803488}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=713373856}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1950963474}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393248060}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1067903297}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393248060}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3626390734}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3637170988}) 2026-02-17T06:07:42.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=419514242}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3637170988}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3199494336}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=981627799}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154925789}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=981627799}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3918901484}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3881343561}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2077676027}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3881343561}) 2026-02-17T06:07:42.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1851959285}) 2026-02-17T06:07:42.330 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:07:42.331 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.397+0000 7f21cd91b640 10 monclient: get_monmap_and_config 2026-02-17T06:07:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.397+0000 7f21cd91b640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.397+0000 7f21cd91b640 10 monclient: monmap: 2026-02-17T06:07:42.400 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.400 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.400 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:42.398559+0000 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:42.398559+0000 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.397+0000 7f21cd91b640 1 Processor -- start 2026-02-17T06:07:42.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.397+0000 7f21cd91b640 1 -- start start 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.397+0000 7f21cd91b640 10 monclient: init 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 10 monclient: _add_conns ranks=[2,1,0,4,3] 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806c890 0x7f21c806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 10 monclient(hunting): picked mon.noname-b con 0x7f21c806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f21c809c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f21c806d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c8135c10 con 0x7f21c806d1b0 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c806daf0 con 0x7f21c809c4a0 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f21c80a0880 con 0x7f21c806c890 2026-02-17T06:07:42.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21cd91b640 10 monclient(hunting): authenticate will time out at 1907.952089s 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806c890 0x7f21c806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806c890 0x7f21c806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48962/0 (socket says 10.20.193.78:48962) 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 1 -- 10.20.193.78:0/2206469001 learned_addr learned my addr 10.20.193.78:0/2206469001 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient(hunting): get_auth_request con 0x7f21c806c890 auth_method 0 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c77fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f21c806d1b0 0x7f21c8135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38140/0 (socket says 10.20.193.78:38140) 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient(hunting): handle_auth_done global_id 264116 payload 274 2026-02-17T06:07:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 1 -- 10.20.193.78:0/2206469001 >> v1:10.20.193.115:6789/0 conn(0x7f21c809c4a0 legacy=0x7f21c8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 1 -- 10.20.193.78:0/2206469001 >> v1:10.20.193.78:6789/0 conn(0x7f21c806d1b0 legacy=0x7f21c8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 1 monclient: found mon.noname-b 2026-02-17T06:07:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.404 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.398+0000 7f21c6ffd640 1 -- 10.20.193.78:0/2206469001 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c809cc80 con 0x7f21c806c890 2026-02-17T06:07:42.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.400+0000 7f21c6ffd640 1 -- 10.20.193.78:0/2206469001 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806c890 msgr2=0x7f21c806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.400+0000 7f21c6ffd640 1 --2- 10.20.193.78:0/2206469001 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806c890 0x7f21c806cc70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0099a0 tx=0x7f21bc02ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78fe53d91e6e4dba server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 1 -- 10.20.193.78:0/2206469001 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc03c070 con 0x7f21c806c890 2026-02-17T06:07:42.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: dump: 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.412 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: _finish_auth 0 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 1 -- 10.20.193.78:0/2206469001 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc02fd90 con 0x7f21c806c890 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 5 monclient: authenticate success, global_id 264116 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 1 -- 10.20.193.78:0/2206469001 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc037660 con 0x7f21c806c890 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21c5ffb640 10 monclient: dump: 2026-02-17T06:07:42.413 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 4 set_mon_vals no callback set 2026-02-17T06:07:42.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 10 monclient: shutdown 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 1 -- 10.20.193.78:0/2206469001 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806c890 msgr2=0x7f21c806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 1 --2- 10.20.193.78:0/2206469001 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806c890 0x7f21c806cc70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0099a0 tx=0x7f21bc02ee30 comp rx=0 tx=0).stop 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 1 -- 10.20.193.78:0/2206469001 shutdown_connections 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 1 --2- 10.20.193.78:0/2206469001 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806c890 0x7f21c806cc70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 1 -- 10.20.193.78:0/2206469001 >> 10.20.193.78:0/2206469001 conn(0x7f21c80a6620 msgr2=0x7f21c80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 1 -- 10.20.193.78:0/2206469001 shutdown_connections 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 1 -- 10.20.193.78:0/2206469001 wait complete. 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.401+0000 7f21cd91b640 10 monclient: monmap: 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.415 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 1 Processor -- start 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 1 -- start start 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient: init 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.416 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21c806c890 0x7f21c81341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient(hunting): picked mon.noname-b con 0x7f21c806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806d1b0 0x7f21c8134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient(hunting): picked mon.noname-c con 0x7f21c806d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21c809c4a0 0x7f21c809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient(hunting): picked mon.noname-a con 0x7f21c809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21c80a0020 con 0x7f21c809c4a0 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f21c806d600 con 0x7f21c806c890 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f21c80a01a0 con 0x7f21c806d1b0 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21cd91b640 10 monclient(hunting): authenticate will time out at 1907.956421s 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21c809c4a0 0x7f21c809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806d1b0 0x7f21c8134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21c809c4a0 0x7f21c809db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40906/0 (socket says 10.20.193.78:40906) 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 1 -- 10.20.193.78:0/2367170261 learned_addr learned my addr 10.20.193.78:0/2367170261 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 10 monclient(hunting): get_auth_request con 0x7f21c809c4a0 auth_method 0 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient(hunting): get_auth_request con 0x7f21c806d1b0 auth_method 0 2026-02-17T06:07:42.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c6ffd640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21c806c890 0x7f21c81341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c6ffd640 10 monclient(hunting): get_auth_request con 0x7f21c806c890 auth_method 0 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient(hunting): handle_auth_done global_id 264119 payload 824 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 1 -- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21c806c890 msgr2=0x7f21c81341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21c806c890 0x7f21c81341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 1 -- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21c809c4a0 msgr2=0x7f21c809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21c809c4a0 0x7f21c809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 1 monclient: found mon.noname-c 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c67fc640 1 -- 10.20.193.78:0/2367170261 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c809d810 con 0x7f21c806d1b0 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.402+0000 7f21c77fe640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21c809c4a0 0x7f21c809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:07:42.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21c67fc640 1 -- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806d1b0 msgr2=0x7f21c8134730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21c67fc640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806d1b0 0x7f21c8134730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f21b000c450 tx=0x7f21b000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 1 -- 10.20.193.78:0/2367170261 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0016020 con 0x7f21c806d1b0 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: dump: 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.421 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: _finish_auth 0 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 1 -- 10.20.193.78:0/2367170261 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b0012a40 con 0x7f21c806d1b0 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21cd91b640 5 monclient: authenticate success, global_id 264119 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21cd91b640 10 monclient: _renew_subs 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21cd91b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c80a26d0 con 0x7f21c806d1b0 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:42.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 1 -- 10.20.193.78:0/2367170261 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0011380 con 0x7f21c806d1b0 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f219ffff640 10 monclient: dump: 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.423 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21c57fa640 4 set_mon_vals no callback set 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21cd91b640 10 monclient: _renew_subs 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.404+0000 7f21cd91b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.405+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c8127930 con 0x7f21c806d1b0 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.405+0000 7f21cd91b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.405+0000 7f21cd91b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.405+0000 7f21cd91b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.405+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c8068410 con 0x7f21c806d1b0 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.405+0000 7f219ffff640 1 -- 10.20.193.78:0/2367170261 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f21b00115e0 con 0x7f21c806d1b0 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.405+0000 7f219ffff640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f219803deb0 0x7f2198040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.405+0000 7f219ffff640 1 -- 10.20.193.78:0/2367170261 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(759..759 src has 525..759) ==== 4783+0+0 (secure 0 0 0) 0x7f21b0052af0 con 0x7f21c806d1b0 2026-02-17T06:07:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.406+0000 7f21c6ffd640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f219803deb0 0x7f2198040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.406+0000 7f21c6ffd640 10 monclient: get_auth_request con 0x7f219803deb0 auth_method 0 2026-02-17T06:07:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.408+0000 7f219ffff640 1 -- 10.20.193.78:0/2367170261 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f21c8068410 con 0x7f21c806d1b0 2026-02-17T06:07:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.408+0000 7f219ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.408+0000 7f219ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.408+0000 7f21c6ffd640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f219803deb0 0x7f2198040370 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f21bc036000 tx=0x7f21bc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.488+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.488+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1502647078}) 2026-02-17T06:07:42.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.488+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=461448925}) 2026-02-17T06:07:42.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.488+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.488+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1502647078}) 2026-02-17T06:07:42.490 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.488+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=917543758}) 2026-02-17T06:07:42.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.520+0000 7f21cd91b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.520+0000 7f21cd91b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.520+0000 7f21cd91b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.520+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f21c806cc70 con 0x7f21c806d1b0 2026-02-17T06:07:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.520+0000 7f219ffff640 1 -- 10.20.193.78:0/2367170261 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f21b0020020 con 0x7f21c806d1b0 2026-02-17T06:07:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.520+0000 7f219ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:42.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.520+0000 7f219ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:42.522 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:42.523 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.520+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.523 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.520+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=807980873}) 2026-02-17T06:07:42.523 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.521+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1794794533}) 2026-02-17T06:07:42.523 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.521+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.523 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.521+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=807980873}) 2026-02-17T06:07:42.524 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.521+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3711500901}) 2026-02-17T06:07:42.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f219803deb0 msgr2=0x7f2198040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f219803deb0 0x7f2198040370 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f21bc036000 tx=0x7f21bc033000 comp rx=0 tx=0).stop 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 10 monclient: shutdown 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806d1b0 msgr2=0x7f21c8134730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806d1b0 0x7f21c8134730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f21b000c450 tx=0x7f21b000c920 comp rx=0 tx=0).stop 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 shutdown_connections 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f21c809c4a0 0x7f21c809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f21c806d1b0 0x7f21c8134730 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f219803deb0 0x7f2198040370 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 --2- 10.20.193.78:0/2367170261 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f21c806c890 0x7f21c81341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.525 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 >> 10.20.193.78:0/2367170261 conn(0x7f21c80a6620 msgr2=0x7f21c8135460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:42.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 shutdown_connections 2026-02-17T06:07:42.526 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.521+0000 7f21cd91b640 1 -- 10.20.193.78:0/2367170261 wait complete. 2026-02-17T06:07:42.530 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:07:42.531 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:07:42.531 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:07:42.531 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:07:42.600 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient: get_monmap_and_config 2026-02-17T06:07:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient: monmap: 2026-02-17T06:07:42.601 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.601 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.601 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:42.598412+0000 2026-02-17T06:07:42.601 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:42.598412+0000 2026-02-17T06:07:42.601 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.601 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 1 Processor -- start 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 1 -- start start 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient: init 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2026-02-17T06:07:42.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b406c890 0x7f64b406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient(hunting): picked mon.noname-b con 0x7f64b406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f64b409d430 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f64b409bf40 0x7f64b406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient(hunting): picked mon.noname-a con 0x7f64b409bf40 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b4073600 con 0x7f64b409d430 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f64b409ca50 con 0x7f64b409bf40 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f64b406d6f0 con 0x7f64b406c890 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.597+0000 7f64ba464640 10 monclient(hunting): authenticate will time out at 1908.151975s 2026-02-17T06:07:42.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b406c890 0x7f64b406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b89da640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f64b409bf40 0x7f64b406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b406c890 0x7f64b406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48972/0 (socket says 10.20.193.78:48972) 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 1 -- 10.20.193.78:0/2363550371 learned_addr learned my addr 10.20.193.78:0/2363550371 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient(hunting): get_auth_request con 0x7f64b406c890 auth_method 0 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b89da640 10 monclient(hunting): get_auth_request con 0x7f64b409bf40 auth_method 0 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b89da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b89da640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b89da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b89da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b89da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b89da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient(hunting): handle_auth_done global_id 264122 payload 274 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 1 -- 10.20.193.78:0/2363550371 >> v2:10.20.193.78:3300/0 conn(0x7f64b409bf40 msgr2=0x7f64b406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 1 --2- 10.20.193.78:0/2363550371 >> v2:10.20.193.78:3300/0 conn(0x7f64b409bf40 0x7f64b406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 1 -- 10.20.193.78:0/2363550371 >> v1:10.20.193.78:6789/0 conn(0x7f64b409d430 legacy=0x7f64b4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 1 monclient: found mon.noname-b 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.598+0000 7f64b3fff640 1 -- 10.20.193.78:0/2363550371 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b406dab0 con 0x7f64b406c890 2026-02-17T06:07:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b3fff640 1 -- 10.20.193.78:0/2363550371 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b406c890 msgr2=0x7f64b406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b3fff640 1 --2- 10.20.193.78:0/2363550371 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b406c890 0x7f64b406cc70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f64b409c670 tx=0x7f64a002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f836a623b6d44326 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 1 -- 10.20.193.78:0/2363550371 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a003c070 con 0x7f64b406c890 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: dump: 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.616 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 1 -- 10.20.193.78:0/2363550371 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a002fca0 con 0x7f64b406c890 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64ba464640 5 monclient: authenticate success, global_id 264122 2026-02-17T06:07:42.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64ba464640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 1 -- 10.20.193.78:0/2363550371 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0037660 con 0x7f64b406c890 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64ba464640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64ba464640 4 set_mon_vals no callback set 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.600+0000 7f64b2ffd640 10 monclient: dump: 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.618 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 10 monclient: shutdown 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 -- 10.20.193.78:0/2363550371 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b406c890 msgr2=0x7f64b406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 --2- 10.20.193.78:0/2363550371 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b406c890 0x7f64b406cc70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f64b409c670 tx=0x7f64a002ee60 comp rx=0 tx=0).stop 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 -- 10.20.193.78:0/2363550371 shutdown_connections 2026-02-17T06:07:42.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 --2- 10.20.193.78:0/2363550371 >> v2:10.20.193.78:3300/0 conn(0x7f64b409bf40 0x7f64b406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 --2- 10.20.193.78:0/2363550371 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b406c890 0x7f64b406cc70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 -- 10.20.193.78:0/2363550371 >> 10.20.193.78:0/2363550371 conn(0x7f64b40a6620 msgr2=0x7f64b40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 -- 10.20.193.78:0/2363550371 shutdown_connections 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 -- 10.20.193.78:0/2363550371 wait complete. 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 10 monclient: monmap: 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.620 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 Processor -- start 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 1 -- start start 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.601+0000 7f64ba464640 10 monclient: init 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64b406c890 0x7f64b41cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient(hunting): picked mon.noname-b con 0x7f64b406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:42.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b409bf40 0x7f64b41cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient(hunting): picked mon.noname-a con 0x7f64b409bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b409d430 0x7f64b41215c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient(hunting): picked mon.noname-c con 0x7f64b409d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64b40733c0 con 0x7f64b409bf40 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f64b4073240 con 0x7f64b406c890 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f64b4135500 con 0x7f64b409d430 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64ba464640 10 monclient(hunting): authenticate will time out at 1908.156126s 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b409bf40 0x7f64b41cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b409d430 0x7f64b41215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b409bf40 0x7f64b41cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40922/0 (socket says 10.20.193.78:40922) 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 1 -- 10.20.193.78:0/2345985612 learned_addr learned my addr 10.20.193.78:0/2345985612 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient(hunting): get_auth_request con 0x7f64b409d430 auth_method 0 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 10 monclient(hunting): get_auth_request con 0x7f64b409bf40 auth_method 0 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient(hunting): handle_auth_done global_id 264125 payload 824 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64b406c890 msgr2=0x7f64b41cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f64b406c890 0x7f64b41cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b409bf40 msgr2=0x7f64b41cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b409bf40 0x7f64b41cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 1 monclient: found mon.noname-c 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.602+0000 7f64b89da640 1 -- 10.20.193.78:0/2345985612 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b4125f20 con 0x7f64b409d430 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b89da640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b409d430 msgr2=0x7f64b41215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b89da640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b409d430 0x7f64b41215c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f64a800efc0 tx=0x7f64a800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8018020 con 0x7f64b409d430 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: dump: 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.625 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: _finish_auth 0 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a80146a0 con 0x7f64b409d430 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64ba464640 5 monclient: authenticate success, global_id 264125 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64ba464640 10 monclient: _renew_subs 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64ba464640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b4125550 con 0x7f64b409d430 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8009b10 con 0x7f64b409d430 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b27fc640 4 set_mon_vals no callback set 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64b0ff9640 10 monclient: dump: 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.627 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64ba464640 10 monclient: _renew_subs 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64ba464640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.604+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b40a19d0 con 0x7f64b409d430 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64ba464640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64ba464640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4124ef0 0x7f64b4121b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64ba464640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64b4074980 con 0x7f64b4124ef0 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4124ef0 0x7f64b4121b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 10 monclient: get_auth_request con 0x7f64b4124ef0 auth_method 0 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f64a8014a90 con 0x7f64b409d430 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 10 monclient(hunting): handle_auth_done global_id 254229 payload 824 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b3fff640 1 -- 10.20.193.78:0/2345985612 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64b409d810 con 0x7f64b4124ef0 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b0ff9640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f648403deb0 0x7f6484040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(759..759 src has 525..759) ==== 4783+0+0 (secure 0 0 0) 0x7f64a8053750 con 0x7f64b409d430 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b37fe640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f648403deb0 0x7f6484040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.605+0000 7f64b37fe640 10 monclient: get_auth_request con 0x7f648403deb0 auth_method 0 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b3fff640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4124ef0 msgr2=0x7f64b4121b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b3fff640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4124ef0 0x7f64b4121b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f64b40a7600 tx=0x7f64a00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0046020 con 0x7f64b4124ef0 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 10 monclient: dump: 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.630 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f64a0037b60 con 0x7f64b4124ef0 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4124ef0 msgr2=0x7f64b4121b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4124ef0 0x7f64b4121b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f64b40a7600 tx=0x7f64a00047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4124ef0 msgr2=0x7f64b4121b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.607+0000 7f64b0ff9640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4124ef0 0x7f64b4121b00 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f64b40a7600 tx=0x7f64a00047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.608+0000 7f64b37fe640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f648403deb0 0x7f6484040370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f64b4124ec0 tx=0x7f64a4003960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.610+0000 7f64ba464640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.610+0000 7f64ba464640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4122040 0x7f64b41da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.610+0000 7f64ba464640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.610+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64b4135930 con 0x7f64b4122040 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4122040 0x7f64b41da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 10 monclient: get_auth_request con 0x7f64b4122040 auth_method 0 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 10 monclient(hunting): handle_auth_done global_id 254232 payload 824 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.611+0000 7f64b3fff640 1 -- 10.20.193.78:0/2345985612 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64b41252d0 con 0x7f64b4122040 2026-02-17T06:07:42.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b3fff640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4122040 msgr2=0x7f64b41da8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b3fff640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4122040 0x7f64b41da8c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64a002eea0 tx=0x7f64a0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0051070 con 0x7f64b4122040 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 10 monclient: dump: 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.633 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f64a004ae50 con 0x7f64b4122040 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4122040 msgr2=0x7f64b41da8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4122040 0x7f64b41da8c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64a002eea0 tx=0x7f64a0008030 comp rx=0 tx=0).stop 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4122040 msgr2=0x7f64b41da8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:42.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b0ff9640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f64b4122040 0x7f64b41da8c0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64a002eea0 tx=0x7f64a0008030 comp rx=0 tx=0).stop 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 246, 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 61987, 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:07:42.635 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 5, 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.636 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.637 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:42.638 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.639 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.640 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.641 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:42.642 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.643 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:07:42.644 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:42.645 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f648403deb0 msgr2=0x7f6484040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64ba464640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f648403deb0 0x7f6484040370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f64b4124ec0 tx=0x7f64a4003960 comp rx=0 tx=0).stop 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64ba464640 10 monclient: shutdown 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64ba464640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b409d430 msgr2=0x7f64b41215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64ba464640 1 --2- 10.20.193.78:0/2345985612 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f64b409d430 0x7f64b41215c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f64a800efc0 tx=0x7f64a800c490 comp rx=0 tx=0).stop 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b89da640 1 -- 10.20.193.78:0/2345985612 reap_dead start 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64b89da640 1 -- 10.20.193.78:0/2345985612 reap_dead start 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 shutdown_connections 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.613+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 >> 10.20.193.78:0/2345985612 conn(0x7f64b40a6620 msgr2=0x7f64b4137970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.614+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 shutdown_connections 2026-02-17T06:07:42.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.614+0000 7f64ba464640 1 -- 10.20.193.78:0/2345985612 wait complete. 2026-02-17T06:07:42.648 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:07:42.648 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:07:42.648 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:07:42.672 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3770645766}) 2026-02-17T06:07:42.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2602789755}) 2026-02-17T06:07:42.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.671+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3770645766}) 2026-02-17T06:07:42.673 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.672+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1428898951}) 2026-02-17T06:07:42.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.684+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.684+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4245965686}) 2026-02-17T06:07:42.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.684+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3114708564}) 2026-02-17T06:07:42.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.684+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.684+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4245965686}) 2026-02-17T06:07:42.685 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.684+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=369366350}) 2026-02-17T06:07:42.688 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.687+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.688 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.687+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1335245426}) 2026-02-17T06:07:42.688 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.687+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2288518267}) 2026-02-17T06:07:42.688 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.687+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.688 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.687+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1335245426}) 2026-02-17T06:07:42.688 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.687+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1581138531}) 2026-02-17T06:07:42.690 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.689+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.690 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.689+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977222431}) 2026-02-17T06:07:42.690 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.689+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1905052919}) 2026-02-17T06:07:42.690 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.689+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.690 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.689+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977222431}) 2026-02-17T06:07:42.690 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.689+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1134322079}) 2026-02-17T06:07:42.692 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.691+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.692 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.691+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1055598400}) 2026-02-17T06:07:42.692 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.691+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4022853646}) 2026-02-17T06:07:42.692 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.691+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.692 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.691+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1055598400}) 2026-02-17T06:07:42.692 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.691+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=884274630}) 2026-02-17T06:07:42.693 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.692+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.693 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.692+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399374872}) 2026-02-17T06:07:42.693 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.692+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=420681520}) 2026-02-17T06:07:42.694 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.692+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.694 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.692+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1399374872}) 2026-02-17T06:07:42.694 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.692+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=902975179}) 2026-02-17T06:07:42.694 INFO:tasks.rados.rados.0.trial115.stdout:3474: done (0 left) 2026-02-17T06:07:42.694 INFO:tasks.rados.rados.0.trial115.stdout:3475: write (excl) oid 32 current snap is 366 2026-02-17T06:07:42.695 INFO:tasks.rados.rados.0.trial115.stdout:3475: seq_num 1195 ranges {484365=589976,1661982=422383} 2026-02-17T06:07:42.698 INFO:tasks.rados.rados.0.trial115.stdout:3475: writing trial11545420-32 from 484365 to 1074341 tid 1 2026-02-17T06:07:42.702 INFO:tasks.rados.rados.0.trial115.stdout:3475: writing trial11545420-32 from 1661982 to 2084365 tid 2 2026-02-17T06:07:42.702 INFO:tasks.rados.rados.0.trial115.stdout:3476: snap_remove snap 364 2026-02-17T06:07:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient: get_monmap_and_config 2026-02-17T06:07:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient: monmap: 2026-02-17T06:07:42.720 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.720 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.720 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:42.718322+0000 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:42.718322+0000 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 1 Processor -- start 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 1 -- start start 2026-02-17T06:07:42.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient: init 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa08406c890 0x7fa08406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient(hunting): picked mon.noname-c con 0x7fa08406c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa08409d430 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409bf40 0x7fa08406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient(hunting): picked mon.noname-b con 0x7fa08409bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa084073600 con 0x7fa08409d430 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa08409ca50 con 0x7fa08406c890 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa08406d6f0 con 0x7fa08409bf40 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa089ce8640 10 monclient(hunting): authenticate will time out at 1908.271856s 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa083fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409bf40 0x7fa08406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa083fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409bf40 0x7fa08406d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:48986/0 (socket says 10.20.193.78:48986) 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa083fff640 1 -- 10.20.193.78:0/892887641 learned_addr learned my addr 10.20.193.78:0/892887641 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa083fff640 10 monclient(hunting): get_auth_request con 0x7fa08409bf40 auth_method 0 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa083fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa083fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.717+0000 7fa083fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa0837fe640 1 --2- 10.20.193.78:0/892887641 >> v2:10.20.193.115:3300/0 conn(0x7fa08406c890 0x7fa08406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa0837fe640 10 monclient(hunting): get_auth_request con 0x7fa08406c890 auth_method 0 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa0837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa0837fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa0837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 10 monclient(hunting): handle_auth_done global_id 264128 payload 274 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 1 -- 10.20.193.78:0/892887641 >> v2:10.20.193.115:3300/0 conn(0x7fa08406c890 msgr2=0x7fa08406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 1 --2- 10.20.193.78:0/892887641 >> v2:10.20.193.115:3300/0 conn(0x7fa08406c890 0x7fa08406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 1 -- 10.20.193.78:0/892887641 >> v1:10.20.193.115:6789/0 conn(0x7fa08409d430 legacy=0x7fa084134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 1 monclient: found mon.noname-b 2026-02-17T06:07:42.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.718+0000 7fa083fff640 1 -- 10.20.193.78:0/892887641 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08406dab0 con 0x7fa08409bf40 2026-02-17T06:07:42.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa083fff640 1 -- 10.20.193.78:0/892887641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409bf40 msgr2=0x7fa08406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa083fff640 1 --2- 10.20.193.78:0/892887641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409bf40 0x7fa08406d1b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa08409cbd0 tx=0x7fa07802f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=157d210e3e165163 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 1 -- 10.20.193.78:0/892887641 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0780373f0 con 0x7fa08409bf40 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: dump: 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.734 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: _finish_auth 0 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: _check_auth_tickets 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa089ce8640 5 monclient: authenticate success, global_id 264128 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 1 -- 10.20.193.78:0/892887641 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa078037590 con 0x7fa08409bf40 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa089ce8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:42.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa089ce8640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 1 -- 10.20.193.78:0/892887641 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078037960 con 0x7fa08409bf40 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa089ce8640 4 set_mon_vals no callback set 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa0827fc640 10 monclient: dump: 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.736 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa089ce8640 10 monclient: shutdown 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa089ce8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa089ce8640 1 -- 10.20.193.78:0/892887641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409bf40 msgr2=0x7fa08406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.720+0000 7fa089ce8640 1 --2- 10.20.193.78:0/892887641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409bf40 0x7fa08406d1b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa08409cbd0 tx=0x7fa07802f200 comp rx=0 tx=0).stop 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 -- 10.20.193.78:0/892887641 shutdown_connections 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 --2- 10.20.193.78:0/892887641 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409bf40 0x7fa08406d1b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 --2- 10.20.193.78:0/892887641 >> v2:10.20.193.115:3300/0 conn(0x7fa08406c890 0x7fa08406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 -- 10.20.193.78:0/892887641 >> 10.20.193.78:0/892887641 conn(0x7fa0840a6620 msgr2=0x7fa0840a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 -- 10.20.193.78:0/892887641 shutdown_connections 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 -- 10.20.193.78:0/892887641 wait complete. 2026-02-17T06:07:42.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient: monmap: 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 Processor -- start 2026-02-17T06:07:42.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 -- start start 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient: init 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa08406c890 0x7fa0841cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient(hunting): picked mon.noname-a con 0x7fa08406c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa08409bf40 0x7fa0841cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient(hunting): picked mon.noname-b con 0x7fa08409bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409d430 0x7fa084121660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient(hunting): picked mon.noname-c con 0x7fa08409d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa08409c7f0 con 0x7fa08406c890 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa084073410 con 0x7fa08409bf40 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa084135500 con 0x7fa08409d430 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.721+0000 7fa089ce8640 10 monclient(hunting): authenticate will time out at 1908.275997s 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409d430 0x7fa084121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa0837fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa08406c890 0x7fa0841cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409d430 0x7fa084121660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49000/0 (socket says 10.20.193.78:49000) 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 -- 10.20.193.78:0/2470803465 learned_addr learned my addr 10.20.193.78:0/2470803465 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient(hunting): get_auth_request con 0x7fa08409d430 auth_method 0 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa0837fe640 10 monclient(hunting): get_auth_request con 0x7fa08406c890 auth_method 0 2026-02-17T06:07:42.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa0837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa0837fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa0837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient(hunting): handle_auth_done global_id 264131 payload 824 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa082ffd640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa08409bf40 0x7fa0841cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa08409bf40 msgr2=0x7fa0841cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa08409bf40 0x7fa0841cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa08406c890 msgr2=0x7fa0841cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa08406c890 0x7fa0841cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 monclient: found mon.noname-c 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa083fff640 1 -- 10.20.193.78:0/2470803465 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa084125030 con 0x7fa08409d430 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa082ffd640 10 monclient: get_auth_request con 0x7fa08409bf40 auth_method 0 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.722+0000 7fa082ffd640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa08409bf40 0x7fa0841cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa083fff640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409d430 msgr2=0x7fa084121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa083fff640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409d430 0x7fa084121660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa078009f50 tx=0x7fa078004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078046020 con 0x7fa08409d430 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: dump: 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: _finish_auth 0 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa078041870 con 0x7fa08409d430 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078041c60 con 0x7fa08409d430 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa081ffb640 4 set_mon_vals no callback set 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa05ffff640 10 monclient: dump: 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.744 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa089ce8640 5 monclient: authenticate success, global_id 264131 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa089ce8640 10 monclient: _renew_subs 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa089ce8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0841237a0 con 0x7fa08409d430 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa089ce8640 10 monclient: _renew_subs 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa089ce8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.724+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa084125310 con 0x7fa08409d430 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa089ce8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa089ce8640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122360 0x7fa08411cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa089ce8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa0840a1990 con 0x7fa084122360 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa078042400 con 0x7fa08409d430 2026-02-17T06:07:42.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122360 0x7fa08411cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 10 monclient: get_auth_request con 0x7fa084122360 auth_method 0 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa05ffff640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa05403deb0 0x7fa054040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(760..760 src has 525..760) ==== 4811+0+0 (secure 0 0 0) 0x7fa0780786d0 con 0x7fa08409d430 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 10 monclient(hunting): handle_auth_done global_id 264134 payload 824 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa0837fe640 1 -- 10.20.193.78:0/2470803465 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa084074980 con 0x7fa084122360 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa082ffd640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa05403deb0 0x7fa054040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.725+0000 7fa082ffd640 10 monclient: get_auth_request con 0x7fa05403deb0 auth_method 0 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa0837fe640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122360 msgr2=0x7fa08411cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa0837fe640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122360 0x7fa08411cc70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa070011830 tx=0x7fa070011d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000f040 con 0x7fa084122360 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 10 monclient: dump: 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa07000fe50 con 0x7fa084122360 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122360 msgr2=0x7fa08411cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122360 0x7fa08411cc70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa070011830 tx=0x7fa070011d00 comp rx=0 tx=0).stop 2026-02-17T06:07:42.747 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122360 msgr2=0x7fa08411cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.727+0000 7fa05ffff640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122360 0x7fa08411cc70 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa070011830 tx=0x7fa070011d00 comp rx=0 tx=0).stop 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.728+0000 7fa082ffd640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa05403deb0 0x7fa054040370 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa084124c80 tx=0x7fa074009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa089ce8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa089ce8640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122740 0x7fa0841da770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa089ce8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa08409d870 con 0x7fa084122740 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122740 0x7fa0841da770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 10 monclient: get_auth_request con 0x7fa084122740 auth_method 0 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 10 monclient(hunting): handle_auth_done global_id 264137 payload 824 2026-02-17T06:07:42.748 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.731+0000 7fa0837fe640 1 -- 10.20.193.78:0/2470803465 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa0840a1400 con 0x7fa084122740 2026-02-17T06:07:42.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.733+0000 7fa0837fe640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122740 msgr2=0x7fa0841da770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.733+0000 7fa0837fe640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122740 0x7fa0841da770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa070011830 tx=0x7fa070004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.733+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070015660 con 0x7fa084122740 2026-02-17T06:07:42.749 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.733+0000 7fa05ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.733+0000 7fa05ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.733+0000 7fa05ffff640 10 monclient: dump: 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7fa070015660 con 0x7fa084122740 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa05ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa05ffff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122740 msgr2=0x7fa0841da770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa05ffff640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122740 0x7fa0841da770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa070011830 tx=0x7fa070004830 comp rx=0 tx=0).stop 2026-02-17T06:07:42.750 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa05ffff640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122740 msgr2=0x7fa0841da770 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa05ffff640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa084122740 0x7fa0841da770 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa070011830 tx=0x7fa070004830 comp rx=0 tx=0).stop 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 246, 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 12118, 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.751 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 5, 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.752 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:42.753 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.754 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.755 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:42.756 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:42.757 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:07:42.758 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.759 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.759 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.759 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:42.759 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:42.759 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.734+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.759 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.734+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=199123736}) 2026-02-17T06:07:42.759 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.734+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642467426}) 2026-02-17T06:07:42.760 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.734+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.760 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.734+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=199123736}) 2026-02-17T06:07:42.760 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.734+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=709960804}) 2026-02-17T06:07:42.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa05403deb0 msgr2=0x7fa054040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa05403deb0 0x7fa054040370 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa084124c80 tx=0x7fa074009040 comp rx=0 tx=0).stop 2026-02-17T06:07:42.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 10 monclient: shutdown 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409d430 msgr2=0x7fa084121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409d430 0x7fa084121660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa078009f50 tx=0x7fa078004b40 comp rx=0 tx=0).stop 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa083fff640 1 -- 10.20.193.78:0/2470803465 reap_dead start 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 shutdown_connections 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 --2- 10.20.193.78:0/2470803465 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa08409d430 0x7fa084121660 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 >> 10.20.193.78:0/2470803465 conn(0x7fa0840a6620 msgr2=0x7fa084136a90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 shutdown_connections 2026-02-17T06:07:42.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.734+0000 7fa089ce8640 1 -- 10.20.193.78:0/2470803465 wait complete. 2026-02-17T06:07:42.761 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:07:42.761 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:07:42.761 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.870+0000 7f0bc673e640 10 monclient: get_monmap_and_config 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.870+0000 7f0bc673e640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.870+0000 7f0bc673e640 10 monclient: monmap: 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:42.871740+0000 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:42.871740+0000 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.874 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.870+0000 7f0bc673e640 1 Processor -- start 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 1 -- start start 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient: init 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient: _add_conns ranks=[4,2,3,0,1] 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f0bc006c890 0x7f0bc006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient(hunting): picked mon.noname-c con 0x7f0bc006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009d170 0x7f0bc006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient(hunting): picked mon.noname-b con 0x7f0bc009d170 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0bc009b200 0x7f0bc01388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient(hunting): picked mon.noname-a con 0x7f0bc009b200 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc006d6f0 con 0x7f0bc009b200 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc009bf60 con 0x7f0bc006c890 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc009c520 con 0x7f0bc009d170 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc673e640 10 monclient(hunting): authenticate will time out at 1908.425307s 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0bc009b200 0x7f0bc01388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009d170 0x7f0bc006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f0bc009b200 0x7f0bc01388b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:40960/0 (socket says 10.20.193.78:40960) 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009d170 0x7f0bc006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49018/0 (socket says 10.20.193.78:49018) 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 -- 10.20.193.78:0/1813322833 learned_addr learned my addr 10.20.193.78:0/1813322833 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient(hunting): get_auth_request con 0x7f0bc009b200 auth_method 0 2026-02-17T06:07:42.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 10 monclient(hunting): get_auth_request con 0x7f0bc009d170 auth_method 0 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbffff640 1 --2- 10.20.193.78:0/1813322833 >> v2:10.20.193.115:3300/0 conn(0x7f0bc006c890 0x7f0bc006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbffff640 10 monclient(hunting): get_auth_request con 0x7f0bc006c890 auth_method 0 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient(hunting): handle_auth_done global_id 254238 payload 274 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 -- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009d170 msgr2=0x7f0bc006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 --2- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009d170 0x7f0bc006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 -- 10.20.193.78:0/1813322833 >> v2:10.20.193.115:3300/0 conn(0x7f0bc006c890 msgr2=0x7f0bc006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 --2- 10.20.193.78:0/1813322833 >> v2:10.20.193.115:3300/0 conn(0x7f0bc006c890 0x7f0bc006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 monclient: found mon.noname-a 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bc4cb4640 1 -- 10.20.193.78:0/1813322833 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc009c800 con 0x7f0bc009b200 2026-02-17T06:07:42.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.871+0000 7f0bbf7fe640 1 --2- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009d170 0x7f0bc006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:07:42.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc4cb4640 1 -- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc009b200 msgr2=0x7f0bc01388b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc4cb4640 1 --2- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc009b200 0x7f0bc01388b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0068b60 tx=0x7f0bb402f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883e2cd1b2ea805a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 1 -- 10.20.193.78:0/1813322833 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb402fe50 con 0x7f0bc009b200 2026-02-17T06:07:42.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: dump: 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: _finish_auth 0 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: _check_auth_tickets 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:42.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 1 -- 10.20.193.78:0/1813322833 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb40373f0 con 0x7f0bc009b200 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 5 monclient: authenticate success, global_id 254238 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 1 -- 10.20.193.78:0/1813322833 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40377c0 con 0x7f0bc009b200 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bbeffd640 10 monclient: dump: 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 4 set_mon_vals no callback set 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 10 monclient: shutdown 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 -- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc009b200 msgr2=0x7f0bc01388b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 --2- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc009b200 0x7f0bc01388b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0068b60 tx=0x7f0bb402f1e0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 -- 10.20.193.78:0/1813322833 shutdown_connections 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 --2- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc009b200 0x7f0bc01388b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 --2- 10.20.193.78:0/1813322833 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009d170 0x7f0bc006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 --2- 10.20.193.78:0/1813322833 >> v2:10.20.193.115:3300/0 conn(0x7f0bc006c890 0x7f0bc006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 -- 10.20.193.78:0/1813322833 >> 10.20.193.78:0/1813322833 conn(0x7f0bc00a4230 msgr2=0x7f0bc00a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 -- 10.20.193.78:0/1813322833 shutdown_connections 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.874+0000 7f0bc673e640 1 -- 10.20.193.78:0/1813322833 wait complete. 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient: monmap: 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:42.887 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 1 Processor -- start 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 1 -- start start 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient: init 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc006c890 0x7f0bc0073890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient(hunting): picked mon.noname-a con 0x7f0bc006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009b200 0x7f0bc00918c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient(hunting): picked mon.noname-c con 0x7f0bc009b200 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc009d170 0x7f0bc0091e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient(hunting): picked mon.noname-b con 0x7f0bc009d170 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc00a3530 con 0x7f0bc006c890 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc00a33b0 con 0x7f0bc009d170 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc0138df0 con 0x7f0bc009b200 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bc673e640 10 monclient(hunting): authenticate will time out at 1908.429620s 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009b200 0x7f0bc00918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc006c890 0x7f0bc0073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009b200 0x7f0bc00918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49032/0 (socket says 10.20.193.78:49032) 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 -- 10.20.193.78:0/3458126998 learned_addr learned my addr 10.20.193.78:0/3458126998 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient(hunting): get_auth_request con 0x7f0bc009b200 auth_method 0 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbffff640 10 monclient(hunting): get_auth_request con 0x7f0bc006c890 auth_method 0 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient(hunting): handle_auth_done global_id 264143 payload 824 2026-02-17T06:07:42.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc009d170 msgr2=0x7f0bc0091e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc009d170 0x7f0bc0091e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc006c890 msgr2=0x7f0bc0073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f0bc006c890 0x7f0bc0073890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 monclient: found mon.noname-c 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.875+0000 7f0bbf7fe640 1 -- 10.20.193.78:0/3458126998 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc00a59b0 con 0x7f0bc009b200 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.877+0000 7f0bbf7fe640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009b200 msgr2=0x7f0bc00918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.877+0000 7f0bbf7fe640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009b200 0x7f0bc00918c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000eb60 tx=0x7f0bb000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0017070 con 0x7f0bc009b200 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: dump: 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.891 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: _finish_auth 0 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: _check_auth_tickets 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 5 monclient: authenticate success, global_id 264143 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 10 monclient: _renew_subs 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc009e920 con 0x7f0bc009b200 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb000ce50 con 0x7f0bc009b200 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0013690 con 0x7f0bc009b200 2026-02-17T06:07:42.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 10 monclient: dump: 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbe7fc640 4 set_mon_vals no callback set 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 10 monclient: _renew_subs 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc0094bd0 con 0x7f0bc009b200 2026-02-17T06:07:42.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0092340 0x7f0bc00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0077110 con 0x7f0bc0092340 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbffff640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0092340 0x7f0bc00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0bb0024060 con 0x7f0bc009b200 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbffff640 10 monclient: get_auth_request con 0x7f0bc0092340 auth_method 0 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.878+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.879+0000 7f0bbcff9640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9003deb0 0x7f0b90040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.879+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(760..760 src has 525..760) ==== 4811+0+0 (secure 0 0 0) 0x7f0bb0052770 con 0x7f0bc009b200 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.879+0000 7f0bbffff640 10 monclient(hunting): handle_auth_done global_id 214489 payload 824 2026-02-17T06:07:42.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.879+0000 7f0bbffff640 1 -- 10.20.193.78:0/3458126998 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bc0092720 con 0x7f0bc0092340 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.879+0000 7f0bc4cb4640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9003deb0 0x7f0b90040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.879+0000 7f0bc4cb4640 10 monclient: get_auth_request con 0x7f0b9003deb0 auth_method 0 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbffff640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0092340 msgr2=0x7f0bc00700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbffff640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0092340 0x7f0bc00700f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0bac011c00 tx=0x7f0bac00f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac019070 con 0x7f0bc0092340 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 10 monclient: dump: 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.895 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f0bac00fe80 con 0x7f0bc0092340 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0092340 msgr2=0x7f0bc00700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0092340 0x7f0bc00700f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0bac011c00 tx=0x7f0bac00f610 comp rx=0 tx=0).stop 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bc4cb4640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9003deb0 0x7f0b90040370 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00693c0 tx=0x7f0bb4031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0092340 msgr2=0x7f0bc00700f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:42.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.881+0000 7f0bbcff9640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0092340 0x7f0bc00700f0 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0bac011c00 tx=0x7f0bac00f610 comp rx=0 tx=0).stop 2026-02-17T06:07:42.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bc673e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:42.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bc673e640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0070630 0x7f0bc0070a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bc673e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01378a0 con 0x7f0bc0070630 2026-02-17T06:07:42.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bbffff640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0070630 0x7f0bc0070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bbffff640 10 monclient: get_auth_request con 0x7f0bc0070630 auth_method 0 2026-02-17T06:07:42.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bbffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.884+0000 7f0bbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.885+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.885+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.885+0000 7f0bbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.885+0000 7f0bbffff640 10 monclient(hunting): handle_auth_done global_id 214492 payload 824 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.885+0000 7f0bbffff640 1 -- 10.20.193.78:0/3458126998 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0bc0074980 con 0x7f0bc0070630 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbffff640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0070630 msgr2=0x7f0bc0070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbffff640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0070630 0x7f0bc0070a40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0bac011c00 tx=0x7f0bac0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac019040 con 0x7f0bc0070630 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 10 monclient: dump: 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.899 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f0bac003e10 con 0x7f0bc0070630 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0070630 msgr2=0x7f0bc0070a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0070630 0x7f0bc0070a40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0bac011c00 tx=0x7f0bac0047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0070630 msgr2=0x7f0bc0070a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:42.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bbcff9640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f0bc0070630 0x7f0bc0070a40 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0bac011c00 tx=0x7f0bac0047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 246, 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 265734, 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 5, 2026-02-17T06:07:42.901 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:42.902 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:42.903 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:42.904 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:42.905 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.906 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.907 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:42.908 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.909 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:42.910 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:42.911 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:42.911 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:42.911 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:42.911 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:42.911 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:42.911 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:42.911 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:42.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9003deb0 msgr2=0x7f0b90040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bc673e640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f0b9003deb0 0x7f0b90040370 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00693c0 tx=0x7f0bb4031000 comp rx=0 tx=0).stop 2026-02-17T06:07:42.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bc673e640 10 monclient: shutdown 2026-02-17T06:07:42.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bc673e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:42.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009b200 msgr2=0x7f0bc00918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:42.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bc673e640 1 --2- 10.20.193.78:0/3458126998 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f0bc009b200 0x7f0bc00918c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000eb60 tx=0x7f0bb000c520 comp rx=0 tx=0).stop 2026-02-17T06:07:42.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bc4cb4640 1 -- 10.20.193.78:0/3458126998 reap_dead start 2026-02-17T06:07:42.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.887+0000 7f0bc4cb4640 1 -- 10.20.193.78:0/3458126998 reap_dead start 2026-02-17T06:07:42.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.888+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 shutdown_connections 2026-02-17T06:07:42.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.888+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 >> 10.20.193.78:0/3458126998 conn(0x7f0bc00a4230 msgr2=0x7f0bc0137110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:42.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.888+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 shutdown_connections 2026-02-17T06:07:42.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.888+0000 7f0bc673e640 1 -- 10.20.193.78:0/3458126998 wait complete. 2026-02-17T06:07:42.913 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:07:42.913 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T06:07:42.914 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:07:42.914 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T06:07:42.914 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:07:42.914 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T06:07:42.914 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:07:42.925 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T06:07:42.925 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:07:42.926 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:42.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=714245676}) 2026-02-17T06:07:42.926 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2399277843}) 2026-02-17T06:07:42.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:07:42.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=714245676}) 2026-02-17T06:07:42.927 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:07:42.924+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2918189659}) 2026-02-17T06:07:42.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.992+0000 7f43100a7640 10 monclient: get_monmap_and_config 2026-02-17T06:07:42.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.992+0000 7f43100a7640 10 monclient: build_initial_monmap 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.992+0000 7f43100a7640 10 monclient: monmap: 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:42.994130+0000 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:42.994130+0000 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:42.994 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:42.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 1 Processor -- start 2026-02-17T06:07:42.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 1 -- start start 2026-02-17T06:07:42.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient: init 2026-02-17T06:07:42.996 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f430806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient(hunting): picked mon.noname-b con 0x7f430806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f430809c4a0 0x7f430806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient(hunting): picked mon.noname-c con 0x7f430809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4308134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f430806daf0 con 0x7f4308134df0 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f43080a0880 con 0x7f430809c4a0 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f430806d6f0 con 0x7f430806c890 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f43100a7640 10 monclient(hunting): authenticate will time out at 1908.547699s 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430d61b640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f430809c4a0 msgr2=0x7f430806d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:42.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f430806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430e61d640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f4308134df0 0x7f4308135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38162/0 (socket says 10.20.193.78:38162) 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430e61d640 1 -- 10.20.193.78:0/1680889924 learned_addr learned my addr 10.20.193.78:0/1680889924 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430d61b640 1 --2- 10.20.193.78:0/1680889924 >> v2:10.20.193.115:3300/0 conn(0x7f430809c4a0 0x7f430806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient(hunting): get_auth_request con 0x7f430806c890 auth_method 0 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient(hunting): handle_auth_done global_id 264146 payload 274 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 1 -- 10.20.193.78:0/1680889924 >> v2:10.20.193.115:3300/0 conn(0x7f430809c4a0 msgr2=0x7f430806d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 1 --2- 10.20.193.78:0/1680889924 >> v2:10.20.193.115:3300/0 conn(0x7f430809c4a0 0x7f430806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 1 -- 10.20.193.78:0/1680889924 >> v1:10.20.193.78:6789/0 conn(0x7f4308134df0 legacy=0x7f4308135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:42.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient: _finish_hunting 0 2026-02-17T06:07:42.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 1 monclient: found mon.noname-b 2026-02-17T06:07:42.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:42.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:07:42.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.993+0000 7f430de1c640 1 -- 10.20.193.78:0/1680889924 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430809cc80 con 0x7f430806c890 2026-02-17T06:07:43.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430de1c640 1 -- 10.20.193.78:0/1680889924 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 msgr2=0x7f430806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430de1c640 1 --2- 10.20.193.78:0/1680889924 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f430806cc70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f42f80098a0 tx=0x7f42f802eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1abf44899b91e04 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 1 -- 10.20.193.78:0/1680889924 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f803c070 con 0x7f430806c890 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: dump: 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:43.004 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: _finish_auth 0 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: _check_auth_tickets 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 1 -- 10.20.193.78:0/1680889924 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f802fd00 con 0x7f430806c890 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f43100a7640 5 monclient: authenticate success, global_id 264146 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f43100a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 1 -- 10.20.193.78:0/1680889924 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8037660 con 0x7f430806c890 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f43100a7640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f43100a7640 4 set_mon_vals no callback set 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f430ce1a640 10 monclient: dump: 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:43.005 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f43100a7640 10 monclient: shutdown 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f43100a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:43.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f43100a7640 1 -- 10.20.193.78:0/1680889924 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 msgr2=0x7f430806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.996+0000 7f43100a7640 1 --2- 10.20.193.78:0/1680889924 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f430806cc70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f42f80098a0 tx=0x7f42f802eda0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 -- 10.20.193.78:0/1680889924 shutdown_connections 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 --2- 10.20.193.78:0/1680889924 >> v2:10.20.193.115:3300/0 conn(0x7f430809c4a0 0x7f430806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 --2- 10.20.193.78:0/1680889924 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f430806cc70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 -- 10.20.193.78:0/1680889924 >> 10.20.193.78:0/1680889924 conn(0x7f43080a6620 msgr2=0x7f43080a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 -- 10.20.193.78:0/1680889924 shutdown_connections 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 -- 10.20.193.78:0/1680889924 wait complete. 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient: build_initial_monmap 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient: monmap: 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:43.007 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 Processor -- start 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 -- start start 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient: init 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f43081cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient(hunting): picked mon.noname-c con 0x7f430806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f430809c4a0 0x7f43081cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient(hunting): picked mon.noname-a con 0x7f430809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4308134df0 0x7f430809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient(hunting): picked mon.noname-b con 0x7f4308134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:43.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43080a01a0 con 0x7f430809c4a0 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f43080a0020 con 0x7f4308134df0 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f430809be20 con 0x7f430806c890 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.997+0000 7f43100a7640 10 monclient(hunting): authenticate will time out at 1908.551978s 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f43081cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f43081cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49056/0 (socket says 10.20.193.78:49056) 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f430809c4a0 0x7f43081cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 -- 10.20.193.78:0/2859565500 learned_addr learned my addr 10.20.193.78:0/2859565500 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient(hunting): get_auth_request con 0x7f430806c890 auth_method 0 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 10 monclient(hunting): get_auth_request con 0x7f430809c4a0 auth_method 0 2026-02-17T06:07:43.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430e61d640 1 -- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4308134df0 msgr2=0x7f430809db50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430e61d640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4308134df0 0x7f430809db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient(hunting): handle_auth_done global_id 264149 payload 824 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 -- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4308134df0 msgr2=0x7f430809db50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4308134df0 0x7f430809db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 -- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f430809c4a0 msgr2=0x7f43081cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f430809c4a0 0x7f43081cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.010 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient: _finish_hunting 0 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 monclient: found mon.noname-c 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430de1c640 1 -- 10.20.193.78:0/2859565500 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430809f7f0 con 0x7f430806c890 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:42.998+0000 7f430d61b640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f430809c4a0 0x7f43081cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f430de1c640 1 -- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 msgr2=0x7f43081cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f430de1c640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f43081cf710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42f80099d0 tx=0x7f42f80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 1 -- 10.20.193.78:0/2859565500 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8046070 con 0x7f430806c890 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: dump: 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:43.011 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: _finish_auth 0 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: _check_auth_tickets 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 5 monclient: authenticate success, global_id 264149 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 1 -- 10.20.193.78:0/2859565500 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f8041870 con 0x7f430806c890 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 1 -- 10.20.193.78:0/2859565500 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f803c040 con 0x7f430806c890 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 10 monclient: _renew_subs 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f7fff640 4 set_mon_vals no callback set 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4308127510 con 0x7f430806c890 2026-02-17T06:07:43.012 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f42f67fc640 10 monclient: dump: 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 10 monclient: _renew_subs 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4308126f50 con 0x7f430806c890 2026-02-17T06:07:43.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:43.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:43.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:43.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.000+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43081359e0 con 0x7f430806c890 2026-02-17T06:07:43.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.001+0000 7f42f67fc640 1 -- 10.20.193.78:0/2859565500 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f42f8041c60 con 0x7f430806c890 2026-02-17T06:07:43.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.001+0000 7f42f67fc640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f42d803de60 0x7f42d8040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:43.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.001+0000 7f42f67fc640 1 -- 10.20.193.78:0/2859565500 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(760..760 src has 525..760) ==== 4811+0+0 (secure 0 0 0) 0x7f42f80779d0 con 0x7f430806c890 2026-02-17T06:07:43.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.001+0000 7f430d61b640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f42d803de60 0x7f42d8040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:43.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.001+0000 7f430d61b640 10 monclient: get_auth_request con 0x7f42d803de60 auth_method 0 2026-02-17T06:07:43.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.003+0000 7f42f67fc640 1 -- 10.20.193.78:0/2859565500 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f42f8050020 con 0x7f430806c890 2026-02-17T06:07:43.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.003+0000 7f42f67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:43.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.003+0000 7f42f67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:43.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.004+0000 7f430d61b640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f42d803de60 0x7f42d8040320 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f42fc006f70 tx=0x7f42fc006ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:43.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.120+0000 7f43100a7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:43.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.120+0000 7f43100a7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:43.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.120+0000 7f43100a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:43.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.120+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4308127120 con 0x7f430806c890 2026-02-17T06:07:43.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.120+0000 7f42f67fc640 1 -- 10.20.193.78:0/2859565500 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f42f802fc70 con 0x7f430806c890 2026-02-17T06:07:43.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.120+0000 7f42f67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:43.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.120+0000 7f42f67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:43.122 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:43.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f42d803de60 msgr2=0x7f42d8040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:43.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f42d803de60 0x7f42d8040320 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f42fc006f70 tx=0x7f42fc006ec0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 10 monclient: shutdown 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 msgr2=0x7f43081cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f43081cf710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42f80099d0 tx=0x7f42f80047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 shutdown_connections 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f42d803de60 0x7f42d8040320 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4308134df0 0x7f430809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f430809c4a0 0x7f43081cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 --2- 10.20.193.78:0/2859565500 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f430806c890 0x7f43081cf710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 >> 10.20.193.78:0/2859565500 conn(0x7f43080a6620 msgr2=0x7f43080a1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:43.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 shutdown_connections 2026-02-17T06:07:43.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:43.121+0000 7f43100a7640 1 -- 10.20.193.78:0/2859565500 wait complete. 2026-02-17T06:07:46.131 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:46.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient: get_monmap_and_config 2026-02-17T06:07:46.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient: build_initial_monmap 2026-02-17T06:07:46.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient: monmap: 2026-02-17T06:07:46.205 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:46.205 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:46.205 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:46.202331+0000 2026-02-17T06:07:46.205 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:46.202331+0000 2026-02-17T06:07:46.205 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:46.205 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 1 Processor -- start 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 1 -- start start 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient: init 2026-02-17T06:07:46.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient(hunting): picked mon.noname-b con 0x7f3f8c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3f8c09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3f8c06d1b0 0x7f3f8c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient(hunting): picked mon.noname-a con 0x7f3f8c06d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f8c06daf0 con 0x7f3f8c09c4a0 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:46.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8c0a0880 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8c09cc80 con 0x7f3f8c06c890 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f94076640 10 monclient(hunting): authenticate will time out at 1911.755860s 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f925ec640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3f8c06d1b0 0x7f3f8c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f91deb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f915ea640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f3f8c09c4a0 0x7f3f8c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:38170/0 (socket says 10.20.193.78:38170) 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f91deb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49058/0 (socket says 10.20.193.78:49058) 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f925ec640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3f8c06d1b0 0x7f3f8c135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:41004/0 (socket says 10.20.193.78:41004) 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f915ea640 1 -- 10.20.193.78:0/1167515334 learned_addr learned my addr 10.20.193.78:0/1167515334 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:07:46.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f91deb640 10 monclient(hunting): get_auth_request con 0x7f3f8c06c890 auth_method 0 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f91deb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f91deb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f91deb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f925ec640 10 monclient(hunting): get_auth_request con 0x7f3f8c06d1b0 auth_method 0 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f925ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f925ec640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.201+0000 7f3f925ec640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f91deb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:46.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f91deb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f91deb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 10 monclient(hunting): handle_auth_done global_id 254247 payload 274 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 1 -- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 msgr2=0x7f3f8c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 1 --2- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 1 -- 10.20.193.78:0/1167515334 >> v1:10.20.193.78:6789/0 conn(0x7f3f8c09c4a0 legacy=0x7f3f8c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 10 monclient: _finish_hunting 0 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 1 monclient: found mon.noname-a 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f925ec640 1 -- 10.20.193.78:0/1167515334 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c09cf60 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.202+0000 7f3f91deb640 1 --2- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:07:46.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f925ec640 1 -- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 msgr2=0x7f3f8c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f925ec640 1 --2- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 0x7f3f8c135500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009d90 tx=0x7f3f8802f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51cf7c18d2a58fd server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 1 -- 10.20.193.78:0/1167515334 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8803c070 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: dump: 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:46.218 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: _finish_auth 0 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: _check_auth_tickets 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 1 -- 10.20.193.78:0/1167515334 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f88037440 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f94076640 5 monclient: authenticate success, global_id 254247 2026-02-17T06:07:46.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f94076640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 1 -- 10.20.193.78:0/1167515334 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88037810 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f94076640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f94076640 4 set_mon_vals no callback set 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f90de9640 10 monclient: dump: 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:46.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f94076640 10 monclient: shutdown 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.204+0000 7f3f94076640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 msgr2=0x7f3f8c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 --2- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 0x7f3f8c135500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009d90 tx=0x7f3f8802f150 comp rx=0 tx=0).stop 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- 10.20.193.78:0/1167515334 shutdown_connections 2026-02-17T06:07:46.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 --2- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 0x7f3f8c135500 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 --2- 10.20.193.78:0/1167515334 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- 10.20.193.78:0/1167515334 >> 10.20.193.78:0/1167515334 conn(0x7f3f8c0a6620 msgr2=0x7f3f8c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- 10.20.193.78:0/1167515334 shutdown_connections 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- 10.20.193.78:0/1167515334 wait complete. 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient: build_initial_monmap 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient: monmap: 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:46.222 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 Processor -- start 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- start start 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient: init 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:46.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient(hunting): picked mon.noname-c con 0x7f3f8c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 0x7f3f8c1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient(hunting): picked mon.noname-a con 0x7f3f8c06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3f8c09c4a0 0x7f3f8c123660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient(hunting): picked mon.noname-b con 0x7f3f8c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c0a01a0 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c0a0020 con 0x7f3f8c09c4a0 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8c09be20 con 0x7f3f8c06c890 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:46.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.205+0000 7f3f94076640 10 monclient(hunting): authenticate will time out at 1911.759958s 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f925ec640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3f8c09c4a0 msgr2=0x7f3f8c123660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 0x7f3f8c1cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f925ec640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3f8c09c4a0 0x7f3f8c123660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c1cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:49072/0 (socket says 10.20.193.78:49072) 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 1 -- 10.20.193.78:0/1444794193 learned_addr learned my addr 10.20.193.78:0/1444794193 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 10 monclient(hunting): get_auth_request con 0x7f3f8c06c890 auth_method 0 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient(hunting): get_auth_request con 0x7f3f8c06d1b0 auth_method 0 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f91deb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient(hunting): handle_auth_done global_id 254250 payload 824 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 1 -- 10.20.193.78:0/1444794193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 msgr2=0x7f3f8c1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 1 -- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3f8c09c4a0 msgr2=0x7f3f8c123660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3f8c09c4a0 0x7f3f8c123660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient: _finish_hunting 0 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 1 monclient: found mon.noname-a 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.206+0000 7f3f915ea640 1 -- 10.20.193.78:0/1444794193 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c125e60 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f915ea640 1 -- 10.20.193.78:0/1444794193 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 msgr2=0x7f3f8c1cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f915ea640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 0x7f3f8c1cd870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c4a0 tx=0x7f3f8000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 1 -- 10.20.193.78:0/1444794193 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007070 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: dump: 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:46.228 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 1 -- 10.20.193.78:0/1444794193 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f80007210 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f94076640 5 monclient: authenticate success, global_id 254250 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f94076640 10 monclient: _renew_subs 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f94076640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c1275f0 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 1 -- 10.20.193.78:0/1444794193 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007600 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:46.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7bfff640 4 set_mon_vals no callback set 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f7a7fc640 10 monclient: dump: 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:46.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f94076640 10 monclient: _renew_subs 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f94076640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.208+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c136e10 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f94076640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f94076640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f94076640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f7a7fc640 1 -- 10.20.193.78:0/1444794193 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3f8001b400 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c074980 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f7a7fc640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3f5c03de60 0x7f3f5c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f7a7fc640 1 -- 10.20.193.78:0/1444794193 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(760..760 src has 525..760) ==== 4811+0+0 (secure 0 0 0) 0x7f3f800524d0 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f91deb640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3f5c03de60 0x7f3f5c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:46.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.209+0000 7f3f91deb640 10 monclient: get_auth_request con 0x7f3f5c03de60 auth_method 0 2026-02-17T06:07:46.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.211+0000 7f3f7a7fc640 1 -- 10.20.193.78:0/1444794193 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3f80050eb0 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.211+0000 7f3f7a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:46.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.211+0000 7f3f7a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:46.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.212+0000 7f3f91deb640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3f5c03de60 0x7f3f5c040320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009660 tx=0x7f3f7c009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:46.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.328+0000 7f3f94076640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:46.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.328+0000 7f3f94076640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:46.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.328+0000 7f3f94076640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:46.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.328+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3f8c06cc70 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.328+0000 7f3f7a7fc640 1 -- 10.20.193.78:0/1444794193 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f3f8001a9c0 con 0x7f3f8c06d1b0 2026-02-17T06:07:46.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.328+0000 7f3f7a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:46.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.328+0000 7f3f7a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:46.329 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3f5c03de60 msgr2=0x7f3f5c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3f5c03de60 0x7f3f5c040320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009660 tx=0x7f3f7c009290 comp rx=0 tx=0).stop 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 10 monclient: shutdown 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 msgr2=0x7f3f8c1cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 0x7f3f8c1cd870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c4a0 tx=0x7f3f8000c970 comp rx=0 tx=0).stop 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 shutdown_connections 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3f5c03de60 0x7f3f5c040320 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3f8c09c4a0 0x7f3f8c123660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3f8c06d1b0 0x7f3f8c1cd870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 --2- 10.20.193.78:0/1444794193 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3f8c06c890 0x7f3f8c1cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 >> 10.20.193.78:0/1444794193 conn(0x7f3f8c0a6620 msgr2=0x7f3f8c071130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:46.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 shutdown_connections 2026-02-17T06:07:46.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:46.329+0000 7f3f94076640 1 -- 10.20.193.78:0/1444794193 wait complete. 2026-02-17T06:07:49.339 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.407+0000 7f9066a3b640 10 monclient: get_monmap_and_config 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.407+0000 7f9066a3b640 10 monclient: build_initial_monmap 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.407+0000 7f9066a3b640 10 monclient: monmap: 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:49.408766+0000 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:49.408766+0000 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:49.408 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:49.409 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:49.409 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:49.409 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:49.409 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.407+0000 7f9066a3b640 1 Processor -- start 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 1 -- start start 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient: init 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f906006c890 0x7f906006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient(hunting): picked mon.noname-a con 0x7f906006c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f906009d430 0x7f906006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient(hunting): picked mon.noname-c con 0x7f906009d430 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:49.410 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f906009bf40 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9060073600 con 0x7f906009bf40 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f906009ca50 con 0x7f906006c890 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f906006d6f0 con 0x7f906009d430 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9066a3b640 10 monclient(hunting): authenticate will time out at 1914.962346s 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f906006c890 0x7f906006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905f7fe640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f906009d430 msgr2=0x7f906006d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905f7fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f906009d430 0x7f906006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f906006c890 0x7f906006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49600/0 (socket says 10.20.193.78:49600) 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f9064fb1640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f906009bf40 0x7f9060134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:58204/0 (socket says 10.20.193.78:58204) 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 1 -- 10.20.193.78:0/2250496799 learned_addr learned my addr 10.20.193.78:0/2250496799 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient(hunting): get_auth_request con 0x7f906006c890 auth_method 0 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient(hunting): handle_auth_done global_id 254253 payload 274 2026-02-17T06:07:49.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 1 -- 10.20.193.78:0/2250496799 >> v2:10.20.193.115:3300/0 conn(0x7f906009d430 msgr2=0x7f906006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:49.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 1 --2- 10.20.193.78:0/2250496799 >> v2:10.20.193.115:3300/0 conn(0x7f906009d430 0x7f906006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 1 -- 10.20.193.78:0/2250496799 >> v1:10.20.193.78:6789/0 conn(0x7f906009bf40 legacy=0x7f9060134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:49.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:07:49.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 1 monclient: found mon.noname-a 2026-02-17T06:07:49.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:49.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:49.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.408+0000 7f905ffff640 1 -- 10.20.193.78:0/2250496799 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906006dab0 con 0x7f906006c890 2026-02-17T06:07:49.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905ffff640 1 -- 10.20.193.78:0/2250496799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 msgr2=0x7f906006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:49.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905ffff640 1 --2- 10.20.193.78:0/2250496799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f906006cc70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f90600a0ac0 tx=0x7f904c02ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a19624a68c780d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:49.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 1 -- 10.20.193.78:0/2250496799 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c03c070 con 0x7f906006c890 2026-02-17T06:07:49.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:49.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:49.417 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: dump: 2026-02-17T06:07:49.417 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:49.417 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: _finish_auth 0 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: _check_auth_tickets 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 1 -- 10.20.193.78:0/2250496799 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904c02fda0 con 0x7f906006c890 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 5 monclient: authenticate success, global_id 254253 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:49.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 1 -- 10.20.193.78:0/2250496799 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c037660 con 0x7f906006c890 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 4 set_mon_vals no callback set 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f905effd640 10 monclient: dump: 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:49.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 10 monclient: shutdown 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 1 -- 10.20.193.78:0/2250496799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 msgr2=0x7f906006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 1 --2- 10.20.193.78:0/2250496799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f906006cc70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f90600a0ac0 tx=0x7f904c02ee80 comp rx=0 tx=0).stop 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 1 -- 10.20.193.78:0/2250496799 shutdown_connections 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 1 --2- 10.20.193.78:0/2250496799 >> v2:10.20.193.115:3300/0 conn(0x7f906009d430 0x7f906006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 1 --2- 10.20.193.78:0/2250496799 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f906006cc70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 1 -- 10.20.193.78:0/2250496799 >> 10.20.193.78:0/2250496799 conn(0x7f90600a6620 msgr2=0x7f90600a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 1 -- 10.20.193.78:0/2250496799 shutdown_connections 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.411+0000 7f9066a3b640 1 -- 10.20.193.78:0/2250496799 wait complete. 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient: build_initial_monmap 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient: monmap: 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:49.420 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 1 Processor -- start 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 1 -- start start 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient: init 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f90601cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient(hunting): picked mon.noname-a con 0x7f906006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f906009bf40 0x7f90601cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient(hunting): picked mon.noname-c con 0x7f906009bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f906009d430 0x7f9060121500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient(hunting): picked mon.noname-b con 0x7f906009d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f906009c800 con 0x7f906006c890 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9060073410 con 0x7f906009d430 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9060135500 con 0x7f906009bf40 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:49.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9066a3b640 10 monclient(hunting): authenticate will time out at 1914.966677s 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f90601cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f90601cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49608/0 (socket says 10.20.193.78:49608) 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 -- 10.20.193.78:0/802415861 learned_addr learned my addr 10.20.193.78:0/802415861 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient(hunting): get_auth_request con 0x7f906006c890 auth_method 0 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9064fb1640 1 -- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f906009d430 msgr2=0x7f9060121500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905f7fe640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f906009bf40 0x7f90601cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f9064fb1640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f906009d430 0x7f9060121500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905f7fe640 10 monclient(hunting): get_auth_request con 0x7f906009bf40 auth_method 0 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient(hunting): handle_auth_done global_id 254256 payload 824 2026-02-17T06:07:49.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 -- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f906009bf40 msgr2=0x7f90601cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f906009bf40 0x7f90601cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 -- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f906009d430 msgr2=0x7f9060121500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f906009d430 0x7f9060121500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 monclient: found mon.noname-a 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.412+0000 7f905ffff640 1 -- 10.20.193.78:0/802415861 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9060125490 con 0x7f906006c890 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905ffff640 1 -- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 msgr2=0x7f90601cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905ffff640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f90601cf560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f904c002410 tx=0x7f904c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 1 -- 10.20.193.78:0/802415861 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c046020 con 0x7f906006c890 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: dump: 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:49.423 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: _finish_auth 0 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: _check_auth_tickets 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 1 -- 10.20.193.78:0/802415861 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904c042640 con 0x7f906006c890 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 5 monclient: authenticate success, global_id 254256 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 10 monclient: _renew_subs 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9060123d00 con 0x7f906006c890 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 1 -- 10.20.193.78:0/802415861 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c03c040 con 0x7f906006c890 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 10 monclient: dump: 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:49.424 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905e7fc640 4 set_mon_vals no callback set 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 10 monclient: _renew_subs 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906009d810 con 0x7f906006c890 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 1 -- 10.20.193.78:0/802415861 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f904c03f030 con 0x7f906006c890 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90601358d0 con 0x7f906006c890 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.415+0000 7f905cff9640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f903003dee0 0x7f90300403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.416+0000 7f905cff9640 1 -- 10.20.193.78:0/802415861 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(760..760 src has 525..760) ==== 4811+0+0 (secure 0 0 0) 0x7f904c077710 con 0x7f906006c890 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.416+0000 7f905f7fe640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f903003dee0 0x7f90300403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:49.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.416+0000 7f905f7fe640 10 monclient: get_auth_request con 0x7f903003dee0 auth_method 0 2026-02-17T06:07:49.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.418+0000 7f905cff9640 1 -- 10.20.193.78:0/802415861 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f904c050070 con 0x7f906006c890 2026-02-17T06:07:49.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.418+0000 7f905cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:49.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.418+0000 7f905cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:49.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.418+0000 7f905f7fe640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f903003dee0 0x7f90300403a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f90500094d0 tx=0x7f9050009460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:49.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.531+0000 7f9066a3b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:49.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.531+0000 7f9066a3b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:49.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.531+0000 7f9066a3b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:49.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.531+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f90600744d0 con 0x7f906006c890 2026-02-17T06:07:49.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.531+0000 7f905cff9640 1 -- 10.20.193.78:0/802415861 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f904c0760f0 con 0x7f906006c890 2026-02-17T06:07:49.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.531+0000 7f905cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:49.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.531+0000 7f905cff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:49.532 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f903003dee0 msgr2=0x7f90300403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f903003dee0 0x7f90300403a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f90500094d0 tx=0x7f9050009460 comp rx=0 tx=0).stop 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 10 monclient: shutdown 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 msgr2=0x7f90601cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f90601cf560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f904c002410 tx=0x7f904c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 shutdown_connections 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f903003dee0 0x7f90300403a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f906009d430 0x7f9060121500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f906009bf40 0x7f90601cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 --2- 10.20.193.78:0/802415861 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f906006c890 0x7f90601cf560 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:49.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 >> 10.20.193.78:0/802415861 conn(0x7f90600a6620 msgr2=0x7f9060136770 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:49.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 shutdown_connections 2026-02-17T06:07:49.534 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:49.532+0000 7f9066a3b640 1 -- 10.20.193.78:0/802415861 wait complete. 2026-02-17T06:07:52.540 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:07:52.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient: get_monmap_and_config 2026-02-17T06:07:52.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient: build_initial_monmap 2026-02-17T06:07:52.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient: monmap: 2026-02-17T06:07:52.609 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:52.609 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:52.609 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:52.609276+0000 2026-02-17T06:07:52.609 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:52.609276+0000 2026-02-17T06:07:52.609 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:52.609 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:52.610 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:52.610 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:52.610 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:52.610 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:52.610 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:52.610 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:52.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 1 Processor -- start 2026-02-17T06:07:52.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 1 -- start start 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient: init 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient: _add_conns ranks=[2,0,3,1,4] 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e006c890 0x7f29e006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient(hunting): picked mon.noname-b con 0x7f29e006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f29e009d430 addr v1:10.20.193.78:6789/0 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f29e009bf40 0x7f29e006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient(hunting): picked mon.noname-a con 0x7f29e009bf40 addr v2:10.20.193.78:3300/0 2026-02-17T06:07:52.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e0073600 con 0x7f29e009d430 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f29e009ca50 con 0x7f29e009bf40 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f29e006d6f0 con 0x7f29e006c890 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e628f640 10 monclient(hunting): authenticate will time out at 1918.162832s 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29dffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e006c890 0x7f29e006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29dffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e006c890 0x7f29e006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36958/0 (socket says 10.20.193.78:36958) 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29dffff640 1 -- 10.20.193.78:0/3260169665 learned_addr learned my addr 10.20.193.78:0/3260169665 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:52.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29dffff640 10 monclient(hunting): get_auth_request con 0x7f29e006c890 auth_method 0 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29dffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29dffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e4805640 1 --2- 10.20.193.78:0/3260169665 >> v2:10.20.193.78:3300/0 conn(0x7f29e009bf40 0x7f29e006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e4805640 10 monclient(hunting): get_auth_request con 0x7f29e009bf40 auth_method 0 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e4805640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e4805640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.608+0000 7f29e4805640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:52.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29e4805640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29e4805640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29e4805640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 10 monclient(hunting): handle_auth_done global_id 264161 payload 274 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 1 -- 10.20.193.78:0/3260169665 >> v2:10.20.193.78:3300/0 conn(0x7f29e009bf40 msgr2=0x7f29e006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 1 --2- 10.20.193.78:0/3260169665 >> v2:10.20.193.78:3300/0 conn(0x7f29e009bf40 0x7f29e006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 1 -- 10.20.193.78:0/3260169665 >> v1:10.20.193.78:6789/0 conn(0x7f29e009d430 legacy=0x7f29e0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 10 monclient: _finish_hunting 0 2026-02-17T06:07:52.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 1 monclient: found mon.noname-b 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29dffff640 1 -- 10.20.193.78:0/3260169665 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e006dab0 con 0x7f29e006c890 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.609+0000 7f29e4805640 1 --2- 10.20.193.78:0/3260169665 >> v2:10.20.193.78:3300/0 conn(0x7f29e009bf40 0x7f29e006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29dffff640 1 -- 10.20.193.78:0/3260169665 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e006c890 msgr2=0x7f29e006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29dffff640 1 --2- 10.20.193.78:0/3260169665 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e006c890 0x7f29e006cc70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29e009c670 tx=0x7f29cc02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cb257c26e773afe server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 1 -- 10.20.193.78:0/3260169665 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc03c070 con 0x7f29e006c890 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: dump: 2026-02-17T06:07:52.625 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:52.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: _finish_auth 0 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: _check_auth_tickets 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 1 -- 10.20.193.78:0/3260169665 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29cc02fca0 con 0x7f29e006c890 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 1 -- 10.20.193.78:0/3260169665 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc037660 con 0x7f29e006c890 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29deffd640 10 monclient: dump: 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:52.627 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29e628f640 5 monclient: authenticate success, global_id 264161 2026-02-17T06:07:52.628 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29e628f640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.611+0000 7f29e628f640 4 set_mon_vals no callback set 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 10 monclient: shutdown 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 -- 10.20.193.78:0/3260169665 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e006c890 msgr2=0x7f29e006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 --2- 10.20.193.78:0/3260169665 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e006c890 0x7f29e006cc70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29e009c670 tx=0x7f29cc02ee60 comp rx=0 tx=0).stop 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 -- 10.20.193.78:0/3260169665 shutdown_connections 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 --2- 10.20.193.78:0/3260169665 >> v2:10.20.193.78:3300/0 conn(0x7f29e009bf40 0x7f29e006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 --2- 10.20.193.78:0/3260169665 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e006c890 0x7f29e006cc70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 -- 10.20.193.78:0/3260169665 >> 10.20.193.78:0/3260169665 conn(0x7f29e00a6620 msgr2=0x7f29e00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:52.629 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 -- 10.20.193.78:0/3260169665 shutdown_connections 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 -- 10.20.193.78:0/3260169665 wait complete. 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 10 monclient: build_initial_monmap 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 10 monclient: monmap: 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:52.630 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 Processor -- start 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 1 -- start start 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.612+0000 7f29e628f640 10 monclient: init 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f29e006c890 0x7f29e01cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient(hunting): picked mon.noname-b con 0x7f29e006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:52.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e009bf40 0x7f29e01cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient(hunting): picked mon.noname-c con 0x7f29e009bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f29e009d430 0x7f29e01215c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient(hunting): picked mon.noname-a con 0x7f29e009d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29e00733c0 con 0x7f29e009d430 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29e0073240 con 0x7f29e006c890 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f29e0135500 con 0x7f29e009bf40 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:52.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e628f640 10 monclient(hunting): authenticate will time out at 1918.167108s 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f29e009d430 0x7f29e01215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f29e009d430 0x7f29e01215c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:49632/0 (socket says 10.20.193.78:49632) 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 -- 10.20.193.78:0/1190401796 learned_addr learned my addr 10.20.193.78:0/1190401796 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29df7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e009bf40 0x7f29e01cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient(hunting): get_auth_request con 0x7f29e009d430 auth_method 0 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29dffff640 1 -- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f29e006c890 msgr2=0x7f29e01cf850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:52.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29dffff640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f29e006c890 0x7f29e01cf850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29df7fe640 10 monclient(hunting): get_auth_request con 0x7f29e009bf40 auth_method 0 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29df7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29df7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:52.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient(hunting): handle_auth_done global_id 254262 payload 824 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 -- 10.20.193.78:0/1190401796 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e009bf40 msgr2=0x7f29e01cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e009bf40 0x7f29e01cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 -- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f29e006c890 msgr2=0x7f29e01cf850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f29e006c890 0x7f29e01cf850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient: _finish_hunting 0 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 monclient: found mon.noname-a 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.613+0000 7f29e4805640 1 -- 10.20.193.78:0/1190401796 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e0125f20 con 0x7f29e009d430 2026-02-17T06:07:52.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e4805640 1 -- 10.20.193.78:0/1190401796 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f29e009d430 msgr2=0x7f29e01215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e4805640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f29e009d430 0x7f29e01215c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29d400ef90 tx=0x7f29d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 1 -- 10.20.193.78:0/1190401796 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4018020 con 0x7f29e009d430 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: dump: 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:52.636 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: _finish_auth 0 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: _check_auth_tickets 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 1 -- 10.20.193.78:0/1190401796 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d4014690 con 0x7f29e009d430 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 1 -- 10.20.193.78:0/1190401796 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4014a80 con 0x7f29e009d430 2026-02-17T06:07:52.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29dcff9640 10 monclient: dump: 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:52.638 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29de7fc640 4 set_mon_vals no callback set 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e628f640 5 monclient: authenticate success, global_id 254262 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e628f640 10 monclient: _renew_subs 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e628f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e0125550 con 0x7f29e009d430 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e628f640 10 monclient: _renew_subs 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e628f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:52.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.615+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e00a19d0 con 0x7f29e009d430 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29e628f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29e628f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29e628f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e0074980 con 0x7f29e009d430 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29dcff9640 1 -- 10.20.193.78:0/1190401796 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f29d4014c20 con 0x7f29e009d430 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29dcff9640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f29b003de60 0x7f29b0040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29dcff9640 1 -- 10.20.193.78:0/1190401796 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(760..760 src has 525..760) ==== 4811+0+0 (secure 0 0 0) 0x7f29d4053570 con 0x7f29e009d430 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29dffff640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f29b003de60 0x7f29b0040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.616+0000 7f29dffff640 10 monclient: get_auth_request con 0x7f29b003de60 auth_method 0 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.618+0000 7f29dcff9640 1 -- 10.20.193.78:0/1190401796 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f29d4022070 con 0x7f29e009d430 2026-02-17T06:07:52.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.618+0000 7f29dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:52.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.618+0000 7f29dcff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:52.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.619+0000 7f29dffff640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f29b003de60 0x7f29b0040320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f29cc036000 tx=0x7f29cc039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:52.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.732+0000 7f29e628f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:07:52.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.732+0000 7f29e628f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:52.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.732+0000 7f29e628f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:07:52.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:52.732+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f29e009d810 con 0x7f29e009d430 2026-02-17T06:07:55.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:55.613+0000 7f29bb7fe640 10 monclient: tick 2026-02-17T06:07:55.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:55.613+0000 7f29bb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:55.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:55.613+0000 7f29bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:55.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:55.613+0000 7f29bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:56.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:56.613+0000 7f29bb7fe640 10 monclient: tick 2026-02-17T06:07:56.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:56.613+0000 7f29bb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:56.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:56.613+0000 7f29bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:57.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.613+0000 7f29bb7fe640 10 monclient: tick 2026-02-17T06:07:57.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.613+0000 7f29bb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:07:57.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.613+0000 7f29bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:57.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.695+0000 7f29dcff9640 1 -- 10.20.193.78:0/1190401796 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f29d4010040 con 0x7f29e009d430 2026-02-17T06:07:57.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.695+0000 7f29dcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:07:57.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.695+0000 7f29dcff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:57.697 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":250,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:07:57.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f29b003de60 msgr2=0x7f29b0040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f29b003de60 0x7f29b0040320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f29cc036000 tx=0x7f29cc039040 comp rx=0 tx=0).stop 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 10 monclient: shutdown 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f29e009d430 msgr2=0x7f29e01215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f29e009d430 0x7f29e01215c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29d400ef90 tx=0x7f29d400c490 comp rx=0 tx=0).stop 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 shutdown_connections 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f29e009d430 0x7f29e01215c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f29e009bf40 0x7f29e01cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f29b003de60 0x7f29b0040320 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 --2- 10.20.193.78:0/1190401796 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f29e006c890 0x7f29e01cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 >> 10.20.193.78:0/1190401796 conn(0x7f29e00a6620 msgr2=0x7f29e0137970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:57.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 shutdown_connections 2026-02-17T06:07:57.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.698+0000 7f29e628f640 1 -- 10.20.193.78:0/1190401796 wait complete. 2026-02-17T06:07:57.709 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:07:57.709 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:07:57.709 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:07:57.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient: get_monmap_and_config 2026-02-17T06:07:57.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient: build_initial_monmap 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient: monmap: 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:57.779235+0000 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:57.779235+0000 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:57.791 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 1 Processor -- start 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 1 -- start start 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient: init 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb9ac06c890 0x7fb9ac0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient(hunting): picked mon.noname-c con 0x7fb9ac06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb9ac06cc70 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac13a240 0x7fb9ac13c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient(hunting): picked mon.noname-b con 0x7fb9ac13a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9ac074150 con 0x7fb9ac06cc70 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb9ac09ef70 con 0x7fb9ac06c890 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ac09f250 con 0x7fb9ac13a240 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9b217e640 10 monclient(hunting): authenticate will time out at 1923.332777s 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9ab7fe640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fb9ac06c890 msgr2=0x7fb9ac0a01f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9aaffd640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fb9ac06cc70 legacy=0x7fb9ac06d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9ab7fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb9ac06c890 0x7fb9ac0a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:57.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac13a240 0x7fb9ac13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac13a240 0x7fb9ac13c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45216/0 (socket says 10.20.193.78:45216) 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 1 -- 10.20.193.78:0/449417996 learned_addr learned my addr 10.20.193.78:0/449417996 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 10 monclient(hunting): get_auth_request con 0x7fb9ac13a240 auth_method 0 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.778+0000 7fb9abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 10 monclient(hunting): handle_auth_done global_id 264167 payload 274 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 1 -- 10.20.193.78:0/449417996 >> v2:10.20.193.115:3300/0 conn(0x7fb9ac06c890 msgr2=0x7fb9ac0a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:57.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 1 --2- 10.20.193.78:0/449417996 >> v2:10.20.193.115:3300/0 conn(0x7fb9ac06c890 0x7fb9ac0a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 1 -- 10.20.193.78:0/449417996 >> v1:10.20.193.115:6789/0 conn(0x7fb9ac06cc70 legacy=0x7fb9ac06d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 10 monclient: _finish_hunting 0 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 1 monclient: found mon.noname-b 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.779+0000 7fb9abfff640 1 -- 10.20.193.78:0/449417996 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac09df40 con 0x7fb9ac13a240 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9abfff640 1 -- 10.20.193.78:0/449417996 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac13a240 msgr2=0x7fb9ac13c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9abfff640 1 --2- 10.20.193.78:0/449417996 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac13a240 0x7fb9ac13c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009e50 tx=0x7fb9a002f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5986e03c65bd296b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 1 -- 10.20.193.78:0/449417996 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00373f0 con 0x7fb9ac13a240 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:07:57.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: dump: 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: _finish_auth 0 2026-02-17T06:07:57.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9b217e640 5 monclient: authenticate success, global_id 264167 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 1 -- 10.20.193.78:0/449417996 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a0037590 con 0x7fb9ac13a240 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9b217e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 1 -- 10.20.193.78:0/449417996 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0037960 con 0x7fb9ac13a240 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9b217e640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9b217e640 4 set_mon_vals no callback set 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:57.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9aa7fc640 10 monclient: dump: 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.781+0000 7fb9b217e640 10 monclient: shutdown 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 -- 10.20.193.78:0/449417996 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac13a240 msgr2=0x7fb9ac13c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 --2- 10.20.193.78:0/449417996 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac13a240 0x7fb9ac13c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009e50 tx=0x7fb9a002f360 comp rx=0 tx=0).stop 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 -- 10.20.193.78:0/449417996 shutdown_connections 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 --2- 10.20.193.78:0/449417996 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac13a240 0x7fb9ac13c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 --2- 10.20.193.78:0/449417996 >> v2:10.20.193.115:3300/0 conn(0x7fb9ac06c890 0x7fb9ac0a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 -- 10.20.193.78:0/449417996 >> 10.20.193.78:0/449417996 conn(0x7fb9ac0a6620 msgr2=0x7fb9ac0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 -- 10.20.193.78:0/449417996 shutdown_connections 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 -- 10.20.193.78:0/449417996 wait complete. 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 10 monclient: build_initial_monmap 2026-02-17T06:07:57.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 10 monclient: monmap: 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 Processor -- start 2026-02-17T06:07:57.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 -- start start 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 10 monclient: init 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac06c890 0x7fb9ac072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 10 monclient(hunting): picked mon.noname-c con 0x7fb9ac06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.782+0000 7fb9b217e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb9ac06cc70 0x7fb9ac1cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 10 monclient(hunting): picked mon.noname-b con 0x7fb9ac06cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13a240 0x7fb9ac1cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 10 monclient(hunting): picked mon.noname-a con 0x7fb9ac13a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac13ccf0 con 0x7fb9ac13a240 2026-02-17T06:07:57.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac13cb70 con 0x7fb9ac06cc70 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ac13ce70 con 0x7fb9ac06c890 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9b217e640 10 monclient(hunting): authenticate will time out at 1923.337020s 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac06c890 0x7fb9ac072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9aaffd640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb9ac06cc70 msgr2=0x7fb9ac1cf500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9aaffd640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb9ac06cc70 0x7fb9ac1cf500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac06c890 0x7fb9ac072fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45226/0 (socket says 10.20.193.78:45226) 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 -- 10.20.193.78:0/1910829975 learned_addr learned my addr 10.20.193.78:0/1910829975 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:57.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient(hunting): get_auth_request con 0x7fb9ac06c890 auth_method 0 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9abfff640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13a240 0x7fb9ac1cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9abfff640 10 monclient(hunting): get_auth_request con 0x7fb9ac13a240 auth_method 0 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9abfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient(hunting): handle_auth_done global_id 264170 payload 824 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb9ac06cc70 msgr2=0x7fb9ac1cf500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb9ac06cc70 0x7fb9ac1cf500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13a240 msgr2=0x7fb9ac1cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13a240 0x7fb9ac1cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 monclient: found mon.noname-c 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:57.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.783+0000 7fb9ab7fe640 1 -- 10.20.193.78:0/1910829975 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac0a1340 con 0x7fb9ac06c890 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9ab7fe640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac06c890 msgr2=0x7fb9ac072fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9ab7fe640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac06c890 0x7fb9ac072fc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b780 tx=0x7fb99800bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980052d0 con 0x7fb9ac06c890 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: dump: 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.805 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: _finish_auth 0 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: _check_auth_tickets 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb998010070 con 0x7fb9ac06c890 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9b217e640 5 monclient: authenticate success, global_id 264170 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:57.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980056d0 con 0x7fb9ac06c890 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb987fff640 10 monclient: dump: 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9a9ffb640 4 set_mon_vals no callback set 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9b217e640 10 monclient: _renew_subs 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9b217e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ac13c960 con 0x7fb9ac06c890 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9b217e640 10 monclient: _renew_subs 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9b217e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ac133b30 con 0x7fb9ac06c890 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.785+0000 7fb9b217e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9b217e640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13db80 0x7fb9ac13df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9b217e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac1cff80 con 0x7fb9ac13db80 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13db80 0x7fb9ac13df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 10 monclient: get_auth_request con 0x7fb9ac13db80 auth_method 0 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb99801d390 con 0x7fb9ac06c890 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 10 monclient(hunting): handle_auth_done global_id 254268 payload 824 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9aaffd640 1 -- 10.20.193.78:0/1910829975 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9ac0a01f0 con 0x7fb9ac13db80 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb987fff640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97c03dee0 0x7fb97c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(760..760 src has 525..760) ==== 4811+0+0 (secure 0 0 0) 0x7fb998052190 con 0x7fb9ac06c890 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9abfff640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97c03dee0 0x7fb97c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.786+0000 7fb9abfff640 10 monclient: get_auth_request con 0x7fb97c03dee0 auth_method 0 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb9aaffd640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13db80 msgr2=0x7fb9ac13df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb9aaffd640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13db80 0x7fb9ac13df60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac13f6a0 tx=0x7fb99c00bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c005270 con 0x7fb9ac13db80 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 10 monclient: dump: 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.810 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fb99c010070 con 0x7fb9ac13db80 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13db80 msgr2=0x7fb9ac13df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13db80 0x7fb9ac13df60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac13f6a0 tx=0x7fb99c00bb20 comp rx=0 tx=0).stop 2026-02-17T06:07:57.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13db80 msgr2=0x7fb9ac13df60 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:57.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.788+0000 7fb987fff640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13db80 0x7fb9ac13df60 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac13f6a0 tx=0x7fb99c00bb20 comp rx=0 tx=0).stop 2026-02-17T06:07:57.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.789+0000 7fb9abfff640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97c03dee0 0x7fb97c0403a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac074e00 tx=0x7fb9a0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9b217e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:57.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9b217e640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13e4a0 0x7fb9ac1da370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9b217e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac13f250 con 0x7fb9ac13e4a0 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9aaffd640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13e4a0 0x7fb9ac1da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9aaffd640 10 monclient: get_auth_request con 0x7fb9ac13e4a0 auth_method 0 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.791+0000 7fb9aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.792+0000 7fb9aaffd640 10 monclient(hunting): handle_auth_done global_id 254271 payload 824 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.792+0000 7fb9aaffd640 1 -- 10.20.193.78:0/1910829975 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb9ac0008d0 con 0x7fb9ac13e4a0 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9aaffd640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13e4a0 msgr2=0x7fb9ac1da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9aaffd640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13e4a0 0x7fb9ac1da370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb99c014000 tx=0x7fb99c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c01a890 con 0x7fb9ac13e4a0 2026-02-17T06:07:57.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 10 monclient: dump: 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4699 (secure 0 0 0) 0x7fb99c01a890 con 0x7fb9ac13e4a0 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13e4a0 msgr2=0x7fb9ac1da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13e4a0 0x7fb9ac1da370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb99c014000 tx=0x7fb99c005a90 comp rx=0 tx=0).stop 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13e4a0 msgr2=0x7fb9ac1da370 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb987fff640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb9ac13e4a0 0x7fb9ac1da370 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:57.817 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 250, 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 77168, 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:57.818 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:57.819 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.820 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:57.821 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:57.822 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.823 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.824 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:57.825 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:57.826 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:57.827 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:57.828 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97c03dee0 msgr2=0x7fb97c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb97c03dee0 0x7fb97c0403a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac074e00 tx=0x7fb9a0033000 comp rx=0 tx=0).stop 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 10 monclient: shutdown 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac06c890 msgr2=0x7fb9ac072fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 1 --2- 10.20.193.78:0/1910829975 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb9ac06c890 0x7fb9ac072fc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b780 tx=0x7fb99800bc50 comp rx=0 tx=0).stop 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9abfff640 1 -- 10.20.193.78:0/1910829975 reap_dead start 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9abfff640 1 -- 10.20.193.78:0/1910829975 reap_dead start 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 shutdown_connections 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 >> 10.20.193.78:0/1910829975 conn(0x7fb9ac0a6620 msgr2=0x7fb9ac09be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 shutdown_connections 2026-02-17T06:07:57.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.794+0000 7fb9b217e640 1 -- 10.20.193.78:0/1910829975 wait complete. 2026-02-17T06:07:57.831 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:07:57.831 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:07:57.831 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:07:57.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.896+0000 7f9ce7577640 10 monclient: get_monmap_and_config 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.896+0000 7f9ce7577640 10 monclient: build_initial_monmap 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient: monmap: 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:07:57.898222+0000 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:07:57.898222+0000 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:07:57.900 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 1 Processor -- start 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 1 -- start start 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient: init 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9ce806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806d380 0x7f9ce80a0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient(hunting): picked mon.noname-b con 0x7f9ce806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9ce813a0a0 0x7f9ce813c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient(hunting): picked mon.noname-c con 0x7f9ce813a0a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce809dd50 con 0x7f9ce806c890 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce80741c0 con 0x7f9ce813a0a0 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce809dad0 con 0x7f9ce806d380 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce7577640 10 monclient(hunting): authenticate will time out at 1923.451794s 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce6575640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f9ce806c890 legacy=0x7f9ce806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce6d76640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f9ce813a0a0 msgr2=0x7f9ce813c490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806d380 0x7f9ce80a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce6d76640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9ce813a0a0 0x7f9ce813c490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806d380 0x7f9ce80a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45242/0 (socket says 10.20.193.78:45242) 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 1 -- 10.20.193.78:0/140426987 learned_addr learned my addr 10.20.193.78:0/140426987 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 10 monclient(hunting): get_auth_request con 0x7f9ce806d380 auth_method 0 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.897+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_done global_id 264173 payload 274 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 1 -- 10.20.193.78:0/140426987 >> v2:10.20.193.115:3300/0 conn(0x7f9ce813a0a0 msgr2=0x7f9ce813c490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 1 --2- 10.20.193.78:0/140426987 >> v2:10.20.193.115:3300/0 conn(0x7f9ce813a0a0 0x7f9ce813c490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 1 -- 10.20.193.78:0/140426987 >> v1:10.20.193.115:6789/0 conn(0x7f9ce806c890 legacy=0x7f9ce806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 10 monclient: _finish_hunting 0 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 1 monclient: found mon.noname-b 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:07:57.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.898+0000 7f9ce5d74640 1 -- 10.20.193.78:0/140426987 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce809edf0 con 0x7f9ce806d380 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5d74640 1 -- 10.20.193.78:0/140426987 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806d380 msgr2=0x7f9ce80a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5d74640 1 --2- 10.20.193.78:0/140426987 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806d380 0x7f9ce80a0070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009970 tx=0x7f9cd002eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a27fe9646d8608d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 1 -- 10.20.193.78:0/140426987 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd003c070 con 0x7f9ce806d380 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: dump: 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.909 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: _finish_auth 0 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: _check_auth_tickets 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 1 -- 10.20.193.78:0/140426987 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd002fdd0 con 0x7f9ce806d380 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 5 monclient: authenticate success, global_id 264173 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 1 -- 10.20.193.78:0/140426987 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0037660 con 0x7f9ce806d380 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 10 monclient: get_monmap_and_config success 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 4 set_mon_vals no callback set 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce5573640 10 monclient: dump: 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.910 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 10 monclient: shutdown 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 1 -- 10.20.193.78:0/140426987 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806d380 msgr2=0x7f9ce80a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 1 --2- 10.20.193.78:0/140426987 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806d380 0x7f9ce80a0070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009970 tx=0x7f9cd002eeb0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 1 -- 10.20.193.78:0/140426987 shutdown_connections 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 1 --2- 10.20.193.78:0/140426987 >> v2:10.20.193.115:3300/0 conn(0x7f9ce813a0a0 0x7f9ce813c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 1 --2- 10.20.193.78:0/140426987 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806d380 0x7f9ce80a0070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.900+0000 7f9ce7577640 1 -- 10.20.193.78:0/140426987 >> 10.20.193.78:0/140426987 conn(0x7f9ce80a65f0 msgr2=0x7f9ce80a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 1 -- 10.20.193.78:0/140426987 shutdown_connections 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 1 -- 10.20.193.78:0/140426987 wait complete. 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 10 monclient: build_initial_monmap 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 10 monclient: monmap: 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 1 Processor -- start 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 1 -- start start 2026-02-17T06:07:57.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 10 monclient: init 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 10 monclient: _reopen_session rank -1 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.901+0000 7f9ce7577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806c890 0x7f9ce8135220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 10 monclient(hunting): picked mon.noname-c con 0x7f9ce806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ce806d380 0x7f9ce81cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 10 monclient(hunting): picked mon.noname-b con 0x7f9ce806d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ce813a0a0 0x7f9ce81cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 10 monclient(hunting): picked mon.noname-a con 0x7f9ce813a0a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce813cb50 con 0x7f9ce813a0a0 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce813c9d0 con 0x7f9ce806d380 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce813ccd0 con 0x7f9ce806c890 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 10 monclient(hunting): _renew_subs 2026-02-17T06:07:57.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce7577640 10 monclient(hunting): authenticate will time out at 1923.456063s 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806c890 0x7f9ce8135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806c890 0x7f9ce8135220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:45248/0 (socket says 10.20.193.78:45248) 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 -- 10.20.193.78:0/2627190288 learned_addr learned my addr 10.20.193.78:0/2627190288 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient(hunting): get_auth_request con 0x7f9ce806c890 auth_method 0 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6d76640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ce813a0a0 0x7f9ce81cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce5d74640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ce806d380 msgr2=0x7f9ce81cbda0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6d76640 10 monclient(hunting): get_auth_request con 0x7f9ce813a0a0 auth_method 0 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce5d74640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ce806d380 0x7f9ce81cbda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:07:57.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient(hunting): handle_auth_done global_id 264176 payload 824 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ce806d380 msgr2=0x7f9ce81cbda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9ce806d380 0x7f9ce81cbda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ce813a0a0 msgr2=0x7f9ce81cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9ce813a0a0 0x7f9ce81cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient: _finish_hunting 0 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 monclient: found mon.noname-c 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.902+0000 7f9ce6575640 1 -- 10.20.193.78:0/2627190288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce8120120 con 0x7f9ce806c890 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce6575640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806c890 msgr2=0x7f9ce8135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:57.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce6575640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806c890 0x7f9ce8135220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b7b0 tx=0x7f9cdc00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0052d0 con 0x7f9ce806c890 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: dump: 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: _finish_auth 0 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: _check_auth_tickets 2026-02-17T06:07:57.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cdc010070 con 0x7f9ce806c890 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce7577640 5 monclient: authenticate success, global_id 264176 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0056d0 con 0x7f9ce806c890 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9cceffd640 10 monclient: dump: 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.917 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce4d72640 4 set_mon_vals no callback set 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce7577640 10 monclient: _renew_subs 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce7577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce8122900 con 0x7f9ce806c890 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce7577640 10 monclient: _renew_subs 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce7577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce8072fc0 con 0x7f9ce806c890 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.904+0000 7f9ce7577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce7577640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8122510 0x7f9ce81cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce7577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce80a0070 con 0x7f9ce8122510 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8122510 0x7f9ce81cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 10 monclient: get_auth_request con 0x7f9ce8122510 auth_method 0 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9cdc00c520 con 0x7f9ce806c890 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_done global_id 264179 payload 824 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce5d74640 1 -- 10.20.193.78:0/2627190288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ce81cfd20 con 0x7f9ce8122510 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9cceffd640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cb403de90 0x7f9cb4040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(760..760 src has 525..760) ==== 4811+0+0 (secure 0 0 0) 0x7f9cdc052070 con 0x7f9ce806c890 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce6d76640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cb403de90 0x7f9cb4040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.905+0000 7f9ce6d76640 10 monclient: get_auth_request con 0x7f9cb403de90 auth_method 0 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9ce5d74640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8122510 msgr2=0x7f9ce81cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:57.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9ce5d74640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8122510 0x7f9ce81cc820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ce809d5a0 tx=0x7f9cd00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd003c040 con 0x7f9ce8122510 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 10 monclient: dump: 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f9cd0037d70 con 0x7f9ce8122510 2026-02-17T06:07:57.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:07:57.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:07:57.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8122510 msgr2=0x7f9ce81cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8122510 0x7f9ce81cc820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ce809d5a0 tx=0x7f9cd00047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8122510 msgr2=0x7f9ce81cc820 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:57.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.907+0000 7f9cceffd640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8122510 0x7f9ce81cc820 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ce809d5a0 tx=0x7f9cd00047c0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.908+0000 7f9ce6d76640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cb403de90 0x7f9cb4040350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ce81213e0 tx=0x7f9cd80095f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce7577640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce7577640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8135760 0x7f9ce8137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce7577640 10 monclient(hunting): start opening mon connection 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce8137740 con 0x7f9ce8135760 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8135760 0x7f9ce8137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 10 monclient: get_auth_request con 0x7f9ce8135760 auth_method 0 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:07:57.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 10 monclient(hunting): handle_auth_done global_id 264182 payload 824 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.910+0000 7f9ce5d74640 1 -- 10.20.193.78:0/2627190288 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ce80744d0 con 0x7f9ce8135760 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.912+0000 7f9ce5d74640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8135760 msgr2=0x7f9ce8137200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.912+0000 7f9ce5d74640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8135760 0x7f9ce8137200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00090b0 tx=0x7f9cd0009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.912+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd003c040 con 0x7f9ce8135760 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.912+0000 7f9cceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.912+0000 7f9cceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.912+0000 7f9cceffd640 10 monclient: dump: 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:07:57.924 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7f9cd0003710 con 0x7f9ce8135760 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9cceffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9cceffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8135760 msgr2=0x7f9ce8137200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9cceffd640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8135760 0x7f9ce8137200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00090b0 tx=0x7f9cd0009340 comp rx=0 tx=0).stop 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9cceffd640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8135760 msgr2=0x7f9ce8137200 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9cceffd640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce8135760 0x7f9ce8137200 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00090b0 tx=0x7f9cd0009340 comp rx=0 tx=0).stop 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:07:57.925 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 250, 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 27297, 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:57.926 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:57.927 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:57.928 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:07:57.929 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:57.930 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:07:57.931 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:57.932 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.933 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:07:57.934 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:07:57.935 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:07:57.936 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:07:57.936 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:07:57.936 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cb403de90 msgr2=0x7f9cb4040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9cb403de90 0x7f9cb4040350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ce81213e0 tx=0x7f9cd80095f0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 10 monclient: shutdown 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806c890 msgr2=0x7f9ce8135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806c890 0x7f9ce8135220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b7b0 tx=0x7f9cdc00bc80 comp rx=0 tx=0).stop 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce6d76640 1 -- 10.20.193.78:0/2627190288 reap_dead start 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 shutdown_connections 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 --2- 10.20.193.78:0/2627190288 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9ce806c890 0x7f9ce8135220 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 >> 10.20.193.78:0/2627190288 conn(0x7f9ce80a65f0 msgr2=0x7f9ce8139760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 shutdown_connections 2026-02-17T06:07:57.937 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:07:57.913+0000 7f9ce7577640 1 -- 10.20.193.78:0/2627190288 wait complete. 2026-02-17T06:07:57.938 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:07:57.938 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:07:58.706 INFO:tasks.rados.rados.0.trial115.stdout:3476: done (1 left) 2026-02-17T06:07:58.706 INFO:tasks.rados.rados.0.trial115.stdout:3477: read oid 30 snap 360 2026-02-17T06:07:58.706 INFO:tasks.rados.rados.0.trial115.stdout:3477: expect deleted 2026-02-17T06:07:58.707 INFO:tasks.rados.rados.0.trial115.stdout:3478: write (excl) oid 46 current snap is 366 2026-02-17T06:07:58.707 INFO:tasks.rados.rados.0.trial115.stdout:3475: finishing write tid 1 to trial11545420-32 2026-02-17T06:07:58.707 INFO:tasks.rados.rados.0.trial115.stdout:3478: seq_num 1196 ranges {501172=665225,1720464=380708} 2026-02-17T06:07:58.710 INFO:tasks.rados.rados.0.trial115.stdout:3478: writing trial11545420-46 from 501172 to 1166397 tid 1 2026-02-17T06:07:58.714 INFO:tasks.rados.rados.0.trial115.stdout:3478: writing trial11545420-46 from 1720464 to 2101172 tid 2 2026-02-17T06:07:58.714 INFO:tasks.rados.rados.0.trial115.stdout:3479: write oid 36 current snap is 366 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3475: finishing write tid 2 to trial11545420-32 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3475: finishing write tid 3 to trial11545420-32 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3475: finishing write tid 4 to trial11545420-32 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3475: oid 32 updating version 0 to 1469 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3475: oid 32 updating version 1469 to 1471 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3475: oid 32 version 1471 is already newer than 1470 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1471 (ObjNum 1195 snap 366 seq_num 1195) dirty exists 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3475: left oid 32 (ObjNum 1195 snap 366 seq_num 1195) 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3478: finishing write tid 1 to trial11545420-46 2026-02-17T06:07:58.715 INFO:tasks.rados.rados.0.trial115.stdout:3479: seq_num 1197 ranges {517979=740474,1778946=339033} 2026-02-17T06:07:58.718 INFO:tasks.rados.rados.0.trial115.stdout:3479: writing trial11545420-36 from 517979 to 1258453 tid 1 2026-02-17T06:07:58.722 INFO:tasks.rados.rados.0.trial115.stdout:3479: writing trial11545420-36 from 1778946 to 2117979 tid 2 2026-02-17T06:07:58.722 INFO:tasks.rados.rados.0.trial115.stdout:3475: done (3 left) 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3477: done (2 left) 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3480: rollback oid 27 current snap is 366 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 365 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3478: finishing write tid 2 to trial11545420-46 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3478: finishing write tid 3 to trial11545420-46 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3478: finishing write tid 4 to trial11545420-46 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3478: oid 46 updating version 0 to 1830 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3478: oid 46 version 1830 is already newer than 1829 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3478: oid 46 updating version 1830 to 1831 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1831 (ObjNum 1196 snap 366 seq_num 1196) dirty exists 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3478: left oid 46 (ObjNum 1196 snap 366 seq_num 1196) 2026-02-17T06:07:58.723 INFO:tasks.rados.rados.0.trial115.stdout:3478: done (2 left) 2026-02-17T06:07:58.724 INFO:tasks.rados.rados.0.trial115.stdout:3481: rollback oid 3 current snap is 366 2026-02-17T06:07:58.724 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 3 to 360 2026-02-17T06:07:58.724 INFO:tasks.rados.rados.0.trial115.stdout:3482: delete oid 8 current snap is 366 2026-02-17T06:07:58.724 INFO:tasks.rados.rados.0.trial115.stdout:3479: finishing write tid 1 to trial11545420-36 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:3482: done (3 left) 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:3483: snap_create 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:3481: finishing rollback tid 1 to trial11545420-3 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:3481: finishing rollback tid 2 to trial11545420-3 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1586 (ObjNum 1136 snap 342 seq_num 1136) dirty exists 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:3479: finishing write tid 2 to trial11545420-36 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:3479: finishing write tid 3 to trial11545420-36 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:3479: finishing write tid 4 to trial11545420-36 2026-02-17T06:07:58.725 INFO:tasks.rados.rados.0.trial115.stdout:3479: oid 36 updating version 0 to 1666 2026-02-17T06:07:58.726 INFO:tasks.rados.rados.0.trial115.stdout:3479: oid 36 version 1666 is already newer than 1664 2026-02-17T06:07:58.726 INFO:tasks.rados.rados.0.trial115.stdout:3479: oid 36 version 1666 is already newer than 1665 2026-02-17T06:07:58.726 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 1666 (ObjNum 1197 snap 366 seq_num 1197) dirty exists 2026-02-17T06:07:58.726 INFO:tasks.rados.rados.0.trial115.stdout:3479: left oid 36 (ObjNum 1197 snap 366 seq_num 1197) 2026-02-17T06:07:58.732 INFO:tasks.rados.rados.0.trial115.stdout:3480: finishing rollback tid 0 to trial11545420-27 2026-02-17T06:07:58.733 INFO:tasks.rados.rados.0.trial115.stdout:3480: finishing rollback tid 1 to trial11545420-27 2026-02-17T06:07:58.733 INFO:tasks.rados.rados.0.trial115.stdout:3480: finishing rollback tid 2 to trial11545420-27 2026-02-17T06:07:58.734 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1470 (ObjNum 1146 snap 349 seq_num 1146) dirty exists 2026-02-17T06:07:59.709 INFO:tasks.rados.rados.0.trial115.stdout:3479: done (3 left) 2026-02-17T06:07:59.709 INFO:tasks.rados.rados.0.trial115.stdout:3480: done (2 left) 2026-02-17T06:07:59.709 INFO:tasks.rados.rados.0.trial115.stdout:3481: done (1 left) 2026-02-17T06:07:59.709 INFO:tasks.rados.rados.0.trial115.stdout:3483: done (0 left) 2026-02-17T06:07:59.709 INFO:tasks.rados.rados.0.trial115.stdout:3484: snap_create 2026-02-17T06:08:00.640 INFO:tasks.rados.rados.0.trial115.stdout:3484: done (0 left) 2026-02-17T06:08:00.640 INFO:tasks.rados.rados.0.trial115.stdout:3485: snap_remove snap 360 2026-02-17T06:08:01.642 INFO:tasks.rados.rados.0.trial115.stdout:3485: done (0 left) 2026-02-17T06:08:01.643 INFO:tasks.rados.rados.0.trial115.stdout:3486: snap_create 2026-02-17T06:08:02.644 INFO:tasks.rados.rados.0.trial115.stdout:3486: done (0 left) 2026-02-17T06:08:02.644 INFO:tasks.rados.rados.0.trial115.stdout:3487: read oid 50 snap -1 2026-02-17T06:08:02.644 INFO:tasks.rados.rados.0.trial115.stdout:3487: expect (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:08:02.644 INFO:tasks.rados.rados.0.trial115.stdout:3488: rollback oid 22 current snap is 369 2026-02-17T06:08:02.644 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 22 to 365 2026-02-17T06:08:02.645 INFO:tasks.rados.rados.0.trial115.stdout:3489: read oid 15 snap -1 2026-02-17T06:08:02.645 INFO:tasks.rados.rados.0.trial115.stdout:3489: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:08:02.645 INFO:tasks.rados.rados.0.trial115.stdout:3490: delete oid 7 current snap is 369 2026-02-17T06:08:02.649 INFO:tasks.rados.rados.0.trial115.stdout:3489: expect (ObjNum 1118 snap 336 seq_num 1118) 2026-02-17T06:08:02.795 INFO:tasks.rados.rados.0.trial115.stdout:3487: expect (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:08:02.893 INFO:tasks.rados.rados.0.trial115.stdout:3488: finishing rollback tid 0 to trial11545420-22 2026-02-17T06:08:02.893 INFO:tasks.rados.rados.0.trial115.stdout:3488: finishing rollback tid 1 to trial11545420-22 2026-02-17T06:08:02.893 INFO:tasks.rados.rados.0.trial115.stdout:3488: finishing rollback tid 2 to trial11545420-22 2026-02-17T06:08:02.893 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1473 (ObjNum 1050 snap 312 seq_num 1050) dirty exists 2026-02-17T06:08:02.893 INFO:tasks.rados.rados.0.trial115.stdout:3487: done (3 left) 2026-02-17T06:08:02.893 INFO:tasks.rados.rados.0.trial115.stdout:3488: done (2 left) 2026-02-17T06:08:02.894 INFO:tasks.rados.rados.0.trial115.stdout:3489: done (1 left) 2026-02-17T06:08:02.894 INFO:tasks.rados.rados.0.trial115.stdout:3490: done (0 left) 2026-02-17T06:08:02.894 INFO:tasks.rados.rados.0.trial115.stdout:3491: snap_remove snap 368 2026-02-17T06:08:03.674 INFO:tasks.rados.rados.0.trial115.stdout:3491: done (0 left) 2026-02-17T06:08:03.674 INFO:tasks.rados.rados.0.trial115.stdout:3492: snap_create 2026-02-17T06:08:04.659 INFO:tasks.rados.rados.0.trial115.stdout:3492: done (0 left) 2026-02-17T06:08:04.659 INFO:tasks.rados.rados.0.trial115.stdout:3493: write (excl) oid 26 current snap is 370 2026-02-17T06:08:04.659 INFO:tasks.rados.rados.0.trial115.stdout:3493: seq_num 1200 ranges {568400=566221,1838039=330361} 2026-02-17T06:08:04.663 INFO:tasks.rados.rados.0.trial115.stdout:3493: writing trial11545420-26 from 568400 to 1134621 tid 1 2026-02-17T06:08:04.667 INFO:tasks.rados.rados.0.trial115.stdout:3493: writing trial11545420-26 from 1838039 to 2168400 tid 2 2026-02-17T06:08:04.668 INFO:tasks.rados.rados.0.trial115.stdout:3494: write (excl) oid 20 current snap is 370 2026-02-17T06:08:04.668 INFO:tasks.rados.rados.0.trial115.stdout:3494: seq_num 1201 ranges {585207=641470,1896521=288686} 2026-02-17T06:08:04.672 INFO:tasks.rados.rados.0.trial115.stdout:3494: writing trial11545420-20 from 585207 to 1226677 tid 1 2026-02-17T06:08:04.676 INFO:tasks.rados.rados.0.trial115.stdout:3494: writing trial11545420-20 from 1896521 to 2185207 tid 2 2026-02-17T06:08:04.676 INFO:tasks.rados.rados.0.trial115.stdout:3495: write oid 30 current snap is 370 2026-02-17T06:08:04.676 INFO:tasks.rados.rados.0.trial115.stdout:3495: seq_num 1202 ranges {602014=433072,1671356=530658} 2026-02-17T06:08:04.679 INFO:tasks.rados.rados.0.trial115.stdout:3495: writing trial11545420-30 from 602014 to 1035086 tid 1 2026-02-17T06:08:04.684 INFO:tasks.rados.rados.0.trial115.stdout:3495: writing trial11545420-30 from 1671356 to 2202014 tid 2 2026-02-17T06:08:04.684 INFO:tasks.rados.rados.0.trial115.stdout:3496: write oid 28 current snap is 370 2026-02-17T06:08:04.684 INFO:tasks.rados.rados.0.trial115.stdout:3496: seq_num 1203 ranges {618821=508321,1613485=531077} 2026-02-17T06:08:04.688 INFO:tasks.rados.rados.0.trial115.stdout:3496: writing trial11545420-28 from 618821 to 1127142 tid 1 2026-02-17T06:08:04.692 INFO:tasks.rados.rados.0.trial115.stdout:3496: writing trial11545420-28 from 1613485 to 2144562 tid 2 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3497: read oid 42 snap -1 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3497: expect deleted 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3498: write oid 25 current snap is 370 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3493: finishing write tid 1 to trial11545420-26 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3493: finishing write tid 2 to trial11545420-26 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3493: finishing write tid 3 to trial11545420-26 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3493: finishing write tid 4 to trial11545420-26 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3493: oid 26 updating version 0 to 1464 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3493: oid 26 version 1464 is already newer than 1463 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:3493: oid 26 updating version 1464 to 1465 2026-02-17T06:08:04.693 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1465 (ObjNum 1200 snap 370 seq_num 1200) dirty exists 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3493: left oid 26 (ObjNum 1200 snap 370 seq_num 1200) 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3494: finishing write tid 1 to trial11545420-20 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3494: finishing write tid 2 to trial11545420-20 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3494: finishing write tid 3 to trial11545420-20 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3494: finishing write tid 4 to trial11545420-20 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3494: oid 20 updating version 0 to 1709 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3494: oid 20 updating version 1709 to 1710 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3494: oid 20 updating version 1710 to 1711 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1711 (ObjNum 1201 snap 370 seq_num 1201) dirty exists 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3494: left oid 20 (ObjNum 1201 snap 370 seq_num 1201) 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3495: finishing write tid 1 to trial11545420-30 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3495: finishing write tid 2 to trial11545420-30 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3495: finishing write tid 3 to trial11545420-30 2026-02-17T06:08:04.694 INFO:tasks.rados.rados.0.trial115.stdout:3495: finishing write tid 4 to trial11545420-30 2026-02-17T06:08:04.695 INFO:tasks.rados.rados.0.trial115.stdout:3495: oid 30 updating version 0 to 1476 2026-02-17T06:08:04.695 INFO:tasks.rados.rados.0.trial115.stdout:3495: oid 30 version 1476 is already newer than 1474 2026-02-17T06:08:04.695 INFO:tasks.rados.rados.0.trial115.stdout:3495: oid 30 version 1476 is already newer than 1475 2026-02-17T06:08:04.695 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1476 (ObjNum 1202 snap 370 seq_num 1202) dirty exists 2026-02-17T06:08:04.695 INFO:tasks.rados.rados.0.trial115.stdout:3495: left oid 30 (ObjNum 1202 snap 370 seq_num 1202) 2026-02-17T06:08:04.695 INFO:tasks.rados.rados.0.trial115.stdout:3496: finishing write tid 1 to trial11545420-28 2026-02-17T06:08:04.695 INFO:tasks.rados.rados.0.trial115.stdout:3498: seq_num 1204 ranges {635628=583570,1671967=563661} 2026-02-17T06:08:04.696 INFO:tasks.rados.rados.0.trial115.stdout:3498: writing trial11545420-25 from 635628 to 1219198 tid 1 2026-02-17T06:08:04.700 INFO:tasks.rados.rados.0.trial115.stdout:3498: writing trial11545420-25 from 1671967 to 2235628 tid 2 2026-02-17T06:08:04.700 INFO:tasks.rados.rados.0.trial115.stdout:3493: done (5 left) 2026-02-17T06:08:04.700 INFO:tasks.rados.rados.0.trial115.stdout:3494: done (4 left) 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3495: done (3 left) 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3499: snap_create 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3496: finishing write tid 2 to trial11545420-28 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3496: finishing write tid 3 to trial11545420-28 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3496: finishing write tid 4 to trial11545420-28 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3496: oid 28 updating version 0 to 1836 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3496: oid 28 updating version 1836 to 1837 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3496: oid 28 updating version 1837 to 1838 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1838 (ObjNum 1203 snap 370 seq_num 1203) dirty exists 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3496: left oid 28 (ObjNum 1203 snap 370 seq_num 1203) 2026-02-17T06:08:04.701 INFO:tasks.rados.rados.0.trial115.stdout:3498: finishing write tid 1 to trial11545420-25 2026-02-17T06:08:04.702 INFO:tasks.rados.rados.0.trial115.stdout:3498: finishing write tid 2 to trial11545420-25 2026-02-17T06:08:04.702 INFO:tasks.rados.rados.0.trial115.stdout:3498: finishing write tid 3 to trial11545420-25 2026-02-17T06:08:04.702 INFO:tasks.rados.rados.0.trial115.stdout:3498: finishing write tid 4 to trial11545420-25 2026-02-17T06:08:04.702 INFO:tasks.rados.rados.0.trial115.stdout:3498: oid 25 updating version 0 to 1585 2026-02-17T06:08:04.702 INFO:tasks.rados.rados.0.trial115.stdout:3498: oid 25 updating version 1585 to 1587 2026-02-17T06:08:04.703 INFO:tasks.rados.rados.0.trial115.stdout:3498: oid 25 version 1587 is already newer than 1586 2026-02-17T06:08:04.703 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1587 (ObjNum 1204 snap 370 seq_num 1204) dirty exists 2026-02-17T06:08:04.703 INFO:tasks.rados.rados.0.trial115.stdout:3498: left oid 25 (ObjNum 1204 snap 370 seq_num 1204) 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:3496: done (3 left) 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:3497: done (2 left) 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:3498: done (1 left) 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:3499: done (0 left) 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:3500: rollback oid 25 current snap is 371 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 370 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:3501: read oid 44 snap -1 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:3501: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:08:05.642 INFO:tasks.rados.rados.0.trial115.stdout:3502: read oid 23 snap 370 2026-02-17T06:08:05.643 INFO:tasks.rados.rados.0.trial115.stdout:3502: expect (ObjNum 1179 snap 362 seq_num 1179) 2026-02-17T06:08:05.643 INFO:tasks.rados.rados.0.trial115.stdout:3503: delete oid 49 current snap is 371 2026-02-17T06:08:05.645 INFO:tasks.rados.rados.0.trial115.stdout:3503: done (3 left) 2026-02-17T06:08:05.645 INFO:tasks.rados.rados.0.trial115.stdout:3504: copy_from oid 19 from oid 12 current snap is 371 2026-02-17T06:08:05.645 INFO:tasks.rados.rados.0.trial115.stdout:3505: read oid 47 snap -1 2026-02-17T06:08:05.645 INFO:tasks.rados.rados.0.trial115.stdout:3505: expect deleted 2026-02-17T06:08:05.645 INFO:tasks.rados.rados.0.trial115.stdout:3506: rollback oid 48 current snap is 371 2026-02-17T06:08:05.645 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 48 to 365 2026-02-17T06:08:05.645 INFO:tasks.rados.rados.0.trial115.stdout:3507: snap_remove snap 367 2026-02-17T06:08:06.646 INFO:tasks.rados.rados.0.trial115.stdout:3507: done (6 left) 2026-02-17T06:08:06.646 INFO:tasks.rados.rados.0.trial115.stdout:3508: read oid 50 snap -1 2026-02-17T06:08:06.646 INFO:tasks.rados.rados.0.trial115.stdout:3508: expect (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:08:06.647 INFO:tasks.rados.rados.0.trial115.stdout:3500: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:08:06.647 INFO:tasks.rados.rados.0.trial115.stdout:3509: snap_create 2026-02-17T06:08:06.647 INFO:tasks.rados.rados.0.trial115.stdout:3500: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:08:06.647 INFO:tasks.rados.rados.0.trial115.stdout:3506: finishing rollback tid 1 to trial11545420-48 2026-02-17T06:08:06.647 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-17T06:08:06.647 INFO:tasks.rados.rados.0.trial115.stdout:3500: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:08:06.647 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1590 (ObjNum 1204 snap 370 seq_num 1204) dirty exists 2026-02-17T06:08:06.648 INFO:tasks.rados.rados.0.trial115.stdout:3501: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:08:06.809 INFO:tasks.rados.rados.0.trial115.stdout:3502: expect (ObjNum 1179 snap 362 seq_num 1179) 2026-02-17T06:08:06.969 INFO:tasks.rados.rados.0.trial115.stdout:3504: finishing copy_from racing read to trial11545420-19 2026-02-17T06:08:06.969 INFO:tasks.rados.rados.0.trial115.stdout:3504: finishing copy_from to trial11545420-19 2026-02-17T06:08:06.969 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1815 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2026-02-17T06:08:06.969 INFO:tasks.rados.rados.0.trial115.stdout:3508: expect (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:08:07.644 INFO:tasks.rados.rados.0.trial115.stdout:3500: done (7 left) 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3501: done (6 left) 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3502: done (5 left) 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3504: done (4 left) 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3505: done (3 left) 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3506: done (2 left) 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3508: done (1 left) 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3509: done (0 left) 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3510: write (excl) oid 4 current snap is 372 2026-02-17T06:08:07.645 INFO:tasks.rados.rados.0.trial115.stdout:3510: seq_num 1206 ranges {669242=734068,2188931=80311} 2026-02-17T06:08:07.649 INFO:tasks.rados.rados.0.trial115.stdout:3510: writing trial11545420-4 from 669242 to 1403310 tid 1 2026-02-17T06:08:07.653 INFO:tasks.rados.rados.0.trial115.stdout:3510: writing trial11545420-4 from 2188931 to 2269242 tid 2 2026-02-17T06:08:07.653 INFO:tasks.rados.rados.0.trial115.stdout:3511: write oid 46 current snap is 372 2026-02-17T06:08:07.653 INFO:tasks.rados.rados.0.trial115.stdout:3511: seq_num 1207 ranges {686049=409317,1847413=438636} 2026-02-17T06:08:07.657 INFO:tasks.rados.rados.0.trial115.stdout:3511: writing trial11545420-46 from 686049 to 1095366 tid 1 2026-02-17T06:08:07.662 INFO:tasks.rados.rados.0.trial115.stdout:3511: writing trial11545420-46 from 1847413 to 2286049 tid 2 2026-02-17T06:08:07.662 INFO:tasks.rados.rados.0.trial115.stdout:3512: read oid 1 snap -1 2026-02-17T06:08:07.662 INFO:tasks.rados.rados.0.trial115.stdout:3512: expect deleted 2026-02-17T06:08:07.662 INFO:tasks.rados.rados.0.trial115.stdout:3513: write (excl) oid 14 current snap is 372 2026-02-17T06:08:07.662 INFO:tasks.rados.rados.0.trial115.stdout:3510: finishing write tid 1 to trial11545420-4 2026-02-17T06:08:07.662 INFO:tasks.rados.rados.0.trial115.stdout:3510: finishing write tid 2 to trial11545420-4 2026-02-17T06:08:07.662 INFO:tasks.rados.rados.0.trial115.stdout:3510: finishing write tid 3 to trial11545420-4 2026-02-17T06:08:07.662 INFO:tasks.rados.rados.0.trial115.stdout:3513: seq_num 1208 ranges {702856=484566,1789542=513314} 2026-02-17T06:08:07.666 INFO:tasks.rados.rados.0.trial115.stdout:3513: writing trial11545420-14 from 702856 to 1187422 tid 1 2026-02-17T06:08:07.671 INFO:tasks.rados.rados.0.trial115.stdout:3513: writing trial11545420-14 from 1789542 to 2302856 tid 2 2026-02-17T06:08:07.671 INFO:tasks.rados.rados.0.trial115.stdout:3514: read oid 50 snap -1 2026-02-17T06:08:07.671 INFO:tasks.rados.rados.0.trial115.stdout:3514: expect (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:08:07.671 INFO:tasks.rados.rados.0.trial115.stdout:3515: write (excl) oid 27 current snap is 372 2026-02-17T06:08:07.671 INFO:tasks.rados.rados.0.trial115.stdout:3510: finishing write tid 4 to trial11545420-4 2026-02-17T06:08:07.671 INFO:tasks.rados.rados.0.trial115.stdout:3510: oid 4 updating version 0 to 1485 2026-02-17T06:08:07.671 INFO:tasks.rados.rados.0.trial115.stdout:3510: oid 4 version 1485 is already newer than 1483 2026-02-17T06:08:07.671 INFO:tasks.rados.rados.0.trial115.stdout:3510: oid 4 version 1485 is already newer than 1484 2026-02-17T06:08:07.672 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1485 (ObjNum 1206 snap 372 seq_num 1206) dirty exists 2026-02-17T06:08:07.672 INFO:tasks.rados.rados.0.trial115.stdout:3510: left oid 4 (ObjNum 1206 snap 372 seq_num 1206) 2026-02-17T06:08:07.672 INFO:tasks.rados.rados.0.trial115.stdout:3511: finishing write tid 1 to trial11545420-46 2026-02-17T06:08:07.672 INFO:tasks.rados.rados.0.trial115.stdout:3515: seq_num 1209 ranges {719663=676168,1964377=355286} 2026-02-17T06:08:07.676 INFO:tasks.rados.rados.0.trial115.stdout:3515: writing trial11545420-27 from 719663 to 1395831 tid 1 2026-02-17T06:08:07.680 INFO:tasks.rados.rados.0.trial115.stdout:3515: writing trial11545420-27 from 1964377 to 2319663 tid 2 2026-02-17T06:08:07.680 INFO:tasks.rados.rados.0.trial115.stdout:3510: done (5 left) 2026-02-17T06:08:07.680 INFO:tasks.rados.rados.0.trial115.stdout:3516: rollback oid 26 current snap is 372 2026-02-17T06:08:07.680 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 26 to 370 2026-02-17T06:08:07.680 INFO:tasks.rados.rados.0.trial115.stdout:3511: finishing write tid 2 to trial11545420-46 2026-02-17T06:08:07.680 INFO:tasks.rados.rados.0.trial115.stdout:3511: finishing write tid 3 to trial11545420-46 2026-02-17T06:08:07.680 INFO:tasks.rados.rados.0.trial115.stdout:3511: finishing write tid 4 to trial11545420-46 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3511: oid 46 updating version 0 to 1842 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3511: oid 46 version 1842 is already newer than 1841 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3511: oid 46 version 1842 is already newer than 1840 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1842 (ObjNum 1207 snap 372 seq_num 1207) dirty exists 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3511: left oid 46 (ObjNum 1207 snap 372 seq_num 1207) 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3513: finishing write tid 1 to trial11545420-14 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3511: done (5 left) 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3512: done (4 left) 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3517: copy_from oid 23 from oid 16 current snap is 372 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3513: finishing write tid 2 to trial11545420-14 2026-02-17T06:08:07.681 INFO:tasks.rados.rados.0.trial115.stdout:3514: expect (ObjNum 1099 snap 329 seq_num 1099) 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:3513: finishing write tid 3 to trial11545420-14 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:3514: done (4 left) 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:3518: copy_from oid 39 from oid 18 current snap is 372 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:3513: finishing write tid 4 to trial11545420-14 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:3513: oid 14 updating version 0 to 1669 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:3513: oid 14 version 1669 is already newer than 1667 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:3513: oid 14 version 1669 is already newer than 1668 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1669 (ObjNum 1208 snap 372 seq_num 1208) dirty exists 2026-02-17T06:08:07.779 INFO:tasks.rados.rados.0.trial115.stdout:3513: left oid 14 (ObjNum 1208 snap 372 seq_num 1208) 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3515: finishing write tid 1 to trial11545420-27 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3516: finishing rollback tid 0 to trial11545420-26 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3516: finishing rollback tid 1 to trial11545420-26 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3516: finishing rollback tid 2 to trial11545420-26 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1468 (ObjNum 1200 snap 370 seq_num 1200) dirty exists 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3517: finishing copy_from racing read to trial11545420-23 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3515: finishing write tid 2 to trial11545420-27 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3515: finishing write tid 3 to trial11545420-27 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3515: finishing write tid 4 to trial11545420-27 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3515: oid 27 updating version 0 to 1478 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3515: oid 27 version 1478 is already newer than 1477 2026-02-17T06:08:07.780 INFO:tasks.rados.rados.0.trial115.stdout:3515: oid 27 updating version 1478 to 1479 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1479 (ObjNum 1209 snap 372 seq_num 1209) dirty exists 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:3515: left oid 27 (ObjNum 1209 snap 372 seq_num 1209) 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:3517: finishing copy_from to trial11545420-23 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1804 (ObjNum 1168 snap 359 seq_num 1168) dirty exists 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:3513: done (4 left) 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:3515: done (3 left) 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:3516: done (2 left) 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:3517: done (1 left) 2026-02-17T06:08:07.781 INFO:tasks.rados.rados.0.trial115.stdout:3519: snap_remove snap 366 2026-02-17T06:08:08.648 INFO:tasks.rados.rados.0.trial115.stdout:3519: done (1 left) 2026-02-17T06:08:08.648 INFO:tasks.rados.rados.0.trial115.stdout:3520: delete oid 17 current snap is 372 2026-02-17T06:08:08.649 INFO:tasks.rados.rados.0.trial115.stdout:3518: finishing copy_from racing read to trial11545420-39 2026-02-17T06:08:08.649 INFO:tasks.rados.rados.0.trial115.stdout:3518: finishing copy_from to trial11545420-39 2026-02-17T06:08:08.649 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1581 (ObjNum 1189 snap 364 seq_num 1189) dirty exists 2026-02-17T06:08:08.652 INFO:tasks.rados.rados.0.trial115.stdout:3518: done (1 left) 2026-02-17T06:08:08.652 INFO:tasks.rados.rados.0.trial115.stdout:3520: done (0 left) 2026-02-17T06:08:08.652 INFO:tasks.rados.rados.0.trial115.stdout:3521: snap_create 2026-02-17T06:08:09.649 INFO:tasks.rados.rados.0.trial115.stdout:3521: done (0 left) 2026-02-17T06:08:09.649 INFO:tasks.rados.rados.0.trial115.stdout:3522: read oid 9 snap -1 2026-02-17T06:08:09.649 INFO:tasks.rados.rados.0.trial115.stdout:3522: expect deleted 2026-02-17T06:08:09.649 INFO:tasks.rados.rados.0.trial115.stdout:3523: read oid 47 snap -1 2026-02-17T06:08:09.650 INFO:tasks.rados.rados.0.trial115.stdout:3523: expect deleted 2026-02-17T06:08:09.650 INFO:tasks.rados.rados.0.trial115.stdout:3524: read oid 18 snap 370 2026-02-17T06:08:09.650 INFO:tasks.rados.rados.0.trial115.stdout:3524: expect (ObjNum 1189 snap 364 seq_num 1189) 2026-02-17T06:08:09.650 INFO:tasks.rados.rados.0.trial115.stdout:3525: snap_remove snap 365 2026-02-17T06:08:10.642 INFO:tasks.rados.rados.0.trial115.stdout:3525: done (3 left) 2026-02-17T06:08:10.643 INFO:tasks.rados.rados.0.trial115.stdout:3526: write (excl) oid 33 current snap is 373 2026-02-17T06:08:10.643 INFO:tasks.rados.rados.0.trial115.stdout:3526: seq_num 1211 ranges {753277=426666,1681341=545753} 2026-02-17T06:08:10.646 INFO:tasks.rados.rados.0.trial115.stdout:3526: writing trial11545420-33 from 753277 to 1179943 tid 1 2026-02-17T06:08:10.651 INFO:tasks.rados.rados.0.trial115.stdout:3526: writing trial11545420-33 from 1681341 to 2227094 tid 2 2026-02-17T06:08:10.651 INFO:tasks.rados.rados.0.trial115.stdout:3527: snap_create 2026-02-17T06:08:10.651 INFO:tasks.rados.rados.0.trial115.stdout:3524: expect (ObjNum 1189 snap 364 seq_num 1189) 2026-02-17T06:08:10.827 INFO:tasks.rados.rados.0.trial115.stdout:3526: finishing write tid 1 to trial11545420-33 2026-02-17T06:08:10.828 INFO:tasks.rados.rados.0.trial115.stdout:3526: finishing write tid 2 to trial11545420-33 2026-02-17T06:08:10.828 INFO:tasks.rados.rados.0.trial115.stdout:3526: finishing write tid 3 to trial11545420-33 2026-02-17T06:08:10.828 INFO:tasks.rados.rados.0.trial115.stdout:3526: finishing write tid 4 to trial11545420-33 2026-02-17T06:08:10.828 INFO:tasks.rados.rados.0.trial115.stdout:3526: oid 33 updating version 0 to 1672 2026-02-17T06:08:10.828 INFO:tasks.rados.rados.0.trial115.stdout:3526: oid 33 version 1672 is already newer than 1671 2026-02-17T06:08:10.828 INFO:tasks.rados.rados.0.trial115.stdout:3526: oid 33 version 1672 is already newer than 1670 2026-02-17T06:08:10.828 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1672 (ObjNum 1211 snap 373 seq_num 1211) dirty exists 2026-02-17T06:08:10.828 INFO:tasks.rados.rados.0.trial115.stdout:3526: left oid 33 (ObjNum 1211 snap 373 seq_num 1211) 2026-02-17T06:08:11.652 INFO:tasks.rados.rados.0.trial115.stdout:3522: done (4 left) 2026-02-17T06:08:11.653 INFO:tasks.rados.rados.0.trial115.stdout:3523: done (3 left) 2026-02-17T06:08:11.653 INFO:tasks.rados.rados.0.trial115.stdout:3524: done (2 left) 2026-02-17T06:08:11.653 INFO:tasks.rados.rados.0.trial115.stdout:3526: done (1 left) 2026-02-17T06:08:11.653 INFO:tasks.rados.rados.0.trial115.stdout:3527: done (0 left) 2026-02-17T06:08:11.653 INFO:tasks.rados.rados.0.trial115.stdout:3528: rollback oid 11 current snap is 374 2026-02-17T06:08:11.653 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 370 2026-02-17T06:08:11.653 INFO:tasks.rados.rados.0.trial115.stdout:3529: read oid 49 snap -1 2026-02-17T06:08:11.653 INFO:tasks.rados.rados.0.trial115.stdout:3529: expect deleted 2026-02-17T06:08:11.654 INFO:tasks.rados.rados.0.trial115.stdout:3530: delete oid 41 current snap is 374 2026-02-17T06:08:11.656 INFO:tasks.rados.rados.0.trial115.stdout:3529: done (2 left) 2026-02-17T06:08:11.656 INFO:tasks.rados.rados.0.trial115.stdout:3530: done (1 left) 2026-02-17T06:08:11.656 INFO:tasks.rados.rados.0.trial115.stdout:3531: write (excl) oid 10 current snap is 374 2026-02-17T06:08:11.656 INFO:tasks.rados.rados.0.trial115.stdout:3531: seq_num 1213 ranges {786891=577164,2081952=304939} 2026-02-17T06:08:11.661 INFO:tasks.rados.rados.0.trial115.stdout:3531: writing trial11545420-10 from 786891 to 1364055 tid 1 2026-02-17T06:08:11.666 INFO:tasks.rados.rados.0.trial115.stdout:3531: writing trial11545420-10 from 2081952 to 2386891 tid 2 2026-02-17T06:08:11.666 INFO:tasks.rados.rados.0.trial115.stdout:3532: read oid 45 snap -1 2026-02-17T06:08:11.666 INFO:tasks.rados.rados.0.trial115.stdout:3532: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:08:11.666 INFO:tasks.rados.rados.0.trial115.stdout:3533: rollback oid 27 current snap is 374 2026-02-17T06:08:11.666 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 372 2026-02-17T06:08:11.666 INFO:tasks.rados.rados.0.trial115.stdout:3528: finishing rollback tid 0 to trial11545420-11 2026-02-17T06:08:11.666 INFO:tasks.rados.rados.0.trial115.stdout:3528: finishing rollback tid 1 to trial11545420-11 2026-02-17T06:08:11.666 INFO:tasks.rados.rados.0.trial115.stdout:3528: finishing rollback tid 2 to trial11545420-11 2026-02-17T06:08:11.667 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1848 (ObjNum 1163 snap 357 seq_num 1163) dirty exists 2026-02-17T06:08:11.667 INFO:tasks.rados.rados.0.trial115.stdout:3531: finishing write tid 1 to trial11545420-10 2026-02-17T06:08:11.667 INFO:tasks.rados.rados.0.trial115.stdout:3528: done (3 left) 2026-02-17T06:08:11.667 INFO:tasks.rados.rados.0.trial115.stdout:3534: write oid 3 current snap is 374 2026-02-17T06:08:11.667 INFO:tasks.rados.rados.0.trial115.stdout:3534: seq_num 1214 ranges {403698=652413,1740434=663264} 2026-02-17T06:08:11.669 INFO:tasks.rados.rados.0.trial115.stdout:3534: writing trial11545420-3 from 403698 to 1056111 tid 1 2026-02-17T06:08:11.675 INFO:tasks.rados.rados.0.trial115.stdout:3534: writing trial11545420-3 from 1740434 to 2403698 tid 2 2026-02-17T06:08:11.675 INFO:tasks.rados.rados.0.trial115.stdout:3535: rollback oid 21 current snap is 374 2026-02-17T06:08:11.675 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 21 to 369 2026-02-17T06:08:11.675 INFO:tasks.rados.rados.0.trial115.stdout:3531: finishing write tid 2 to trial11545420-10 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3531: finishing write tid 3 to trial11545420-10 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3531: finishing write tid 4 to trial11545420-10 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3531: oid 10 updating version 0 to 1850 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3531: oid 10 updating version 1850 to 1852 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3531: oid 10 version 1852 is already newer than 1851 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1852 (ObjNum 1213 snap 374 seq_num 1213) dirty exists 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3531: left oid 10 (ObjNum 1213 snap 374 seq_num 1213) 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3533: finishing rollback tid 0 to trial11545420-27 2026-02-17T06:08:11.676 INFO:tasks.rados.rados.0.trial115.stdout:3533: finishing rollback tid 1 to trial11545420-27 2026-02-17T06:08:11.677 INFO:tasks.rados.rados.0.trial115.stdout:3533: finishing rollback tid 2 to trial11545420-27 2026-02-17T06:08:11.677 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1482 (ObjNum 1209 snap 372 seq_num 1209) dirty exists 2026-02-17T06:08:11.677 INFO:tasks.rados.rados.0.trial115.stdout:3532: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:08:11.792 INFO:tasks.rados.rados.0.trial115.stdout:3534: finishing write tid 1 to trial11545420-3 2026-02-17T06:08:11.792 INFO:tasks.rados.rados.0.trial115.stdout:3531: done (4 left) 2026-02-17T06:08:11.792 INFO:tasks.rados.rados.0.trial115.stdout:3532: done (3 left) 2026-02-17T06:08:11.792 INFO:tasks.rados.rados.0.trial115.stdout:3533: done (2 left) 2026-02-17T06:08:11.792 INFO:tasks.rados.rados.0.trial115.stdout:3536: snap_remove snap 373 2026-02-17T06:08:12.655 INFO:tasks.rados.rados.0.trial115.stdout:3536: done (2 left) 2026-02-17T06:08:12.656 INFO:tasks.rados.rados.0.trial115.stdout:3537: copy_from oid 1 from oid 26 current snap is 374 2026-02-17T06:08:12.656 INFO:tasks.rados.rados.0.trial115.stdout:3535: finishing rollback tid 0 to trial11545420-21 2026-02-17T06:08:12.656 INFO:tasks.rados.rados.0.trial115.stdout:3535: finishing rollback tid 1 to trial11545420-21 2026-02-17T06:08:12.656 INFO:tasks.rados.rados.0.trial115.stdout:3535: finishing rollback tid 2 to trial11545420-21 2026-02-17T06:08:12.656 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1594 (ObjNum 1190 snap 364 seq_num 1190) dirty exists 2026-02-17T06:08:12.656 INFO:tasks.rados.rados.0.trial115.stdout:3534: finishing write tid 2 to trial11545420-3 2026-02-17T06:08:12.657 INFO:tasks.rados.rados.0.trial115.stdout:3534: finishing write tid 3 to trial11545420-3 2026-02-17T06:08:12.657 INFO:tasks.rados.rados.0.trial115.stdout:3534: finishing write tid 4 to trial11545420-3 2026-02-17T06:08:12.657 INFO:tasks.rados.rados.0.trial115.stdout:3534: oid 3 updating version 0 to 1590 2026-02-17T06:08:12.657 INFO:tasks.rados.rados.0.trial115.stdout:3534: oid 3 updating version 1590 to 1592 2026-02-17T06:08:12.657 INFO:tasks.rados.rados.0.trial115.stdout:3534: oid 3 version 1592 is already newer than 1591 2026-02-17T06:08:12.657 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1592 (ObjNum 1214 snap 374 seq_num 1214) dirty exists 2026-02-17T06:08:12.658 INFO:tasks.rados.rados.0.trial115.stdout:3534: left oid 3 (ObjNum 1214 snap 374 seq_num 1214) 2026-02-17T06:08:12.658 INFO:tasks.rados.rados.0.trial115.stdout:3534: done (2 left) 2026-02-17T06:08:12.658 INFO:tasks.rados.rados.0.trial115.stdout:3535: done (1 left) 2026-02-17T06:08:12.658 INFO:tasks.rados.rados.0.trial115.stdout:3538: snap_create 2026-02-17T06:08:12.662 INFO:tasks.rados.rados.0.trial115.stdout:3537: finishing copy_from racing read to trial11545420-1 2026-02-17T06:08:12.665 INFO:tasks.rados.rados.0.trial115.stdout:3537: finishing copy_from to trial11545420-1 2026-02-17T06:08:12.666 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 1468 (ObjNum 1200 snap 370 seq_num 1200) dirty exists 2026-02-17T06:08:13.654 INFO:tasks.rados.rados.0.trial115.stdout:3537: done (1 left) 2026-02-17T06:08:13.655 INFO:tasks.rados.rados.0.trial115.stdout:3538: done (0 left) 2026-02-17T06:08:13.655 INFO:tasks.rados.rados.0.trial115.stdout:3539: snap_create 2026-02-17T06:08:14.656 INFO:tasks.rados.rados.0.trial115.stdout:3539: done (0 left) 2026-02-17T06:08:14.657 INFO:tasks.rados.rados.0.trial115.stdout:3540: snap_remove snap 371 2026-02-17T06:08:15.642 INFO:tasks.rados.rados.0.trial115.stdout:3540: done (0 left) 2026-02-17T06:08:15.643 INFO:tasks.rados.rados.0.trial115.stdout:3541: write oid 3 current snap is 376 2026-02-17T06:08:15.643 INFO:tasks.rados.rados.0.trial115.stdout:3541: seq_num 1215 ranges {420505=727662,1798916=553091} 2026-02-17T06:08:15.648 INFO:tasks.rados.rados.0.trial115.stdout:3541: writing trial11545420-3 from 420505 to 1148167 tid 1 2026-02-17T06:08:15.653 INFO:tasks.rados.rados.0.trial115.stdout:3541: writing trial11545420-3 from 1798916 to 2352007 tid 2 2026-02-17T06:08:15.653 INFO:tasks.rados.rados.0.trial115.stdout:3542: read oid 13 snap 372 2026-02-17T06:08:15.653 INFO:tasks.rados.rados.0.trial115.stdout:3542: expect (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:08:15.653 INFO:tasks.rados.rados.0.trial115.stdout:3543: delete oid 23 current snap is 376 2026-02-17T06:08:15.653 INFO:tasks.rados.rados.0.trial115.stdout:3541: finishing write tid 1 to trial11545420-3 2026-02-17T06:08:15.654 INFO:tasks.rados.rados.0.trial115.stdout:3541: finishing write tid 2 to trial11545420-3 2026-02-17T06:08:15.654 INFO:tasks.rados.rados.0.trial115.stdout:3541: finishing write tid 3 to trial11545420-3 2026-02-17T06:08:15.654 INFO:tasks.rados.rados.0.trial115.stdout:3541: finishing write tid 4 to trial11545420-3 2026-02-17T06:08:15.654 INFO:tasks.rados.rados.0.trial115.stdout:3541: oid 3 updating version 0 to 1598 2026-02-17T06:08:15.655 INFO:tasks.rados.rados.0.trial115.stdout:3541: oid 3 updating version 1598 to 1600 2026-02-17T06:08:15.655 INFO:tasks.rados.rados.0.trial115.stdout:3541: oid 3 version 1600 is already newer than 1599 2026-02-17T06:08:15.655 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1600 (ObjNum 1215 snap 376 seq_num 1215) dirty exists 2026-02-17T06:08:15.655 INFO:tasks.rados.rados.0.trial115.stdout:3541: left oid 3 (ObjNum 1215 snap 376 seq_num 1215) 2026-02-17T06:08:15.656 INFO:tasks.rados.rados.0.trial115.stdout:3542: expect (ObjNum 1026 snap 297 seq_num 1026) 2026-02-17T06:08:15.792 INFO:tasks.rados.rados.0.trial115.stdout:3541: done (2 left) 2026-02-17T06:08:15.793 INFO:tasks.rados.rados.0.trial115.stdout:3542: done (1 left) 2026-02-17T06:08:15.793 INFO:tasks.rados.rados.0.trial115.stdout:3543: done (0 left) 2026-02-17T06:08:15.793 INFO:tasks.rados.rados.0.trial115.stdout:3544: copy_from oid 15 from oid 16 current snap is 376 2026-02-17T06:08:15.793 INFO:tasks.rados.rados.0.trial115.stdout:3545: copy_from oid 43 from oid 6 current snap is 376 2026-02-17T06:08:15.793 INFO:tasks.rados.rados.0.trial115.stdout:3546: write oid 5 current snap is 376 2026-02-17T06:08:15.793 INFO:tasks.rados.rados.0.trial115.stdout:3546: seq_num 1217 ranges {454119=594513,1515880=556760} 2026-02-17T06:08:15.797 INFO:tasks.rados.rados.0.trial115.stdout:3546: writing trial11545420-5 from 454119 to 1048632 tid 1 2026-02-17T06:08:15.801 INFO:tasks.rados.rados.0.trial115.stdout:3546: writing trial11545420-5 from 1515880 to 2072640 tid 2 2026-02-17T06:08:15.801 INFO:tasks.rados.rados.0.trial115.stdout:3547: snap_remove snap 375 2026-02-17T06:08:16.663 INFO:tasks.rados.rados.0.trial115.stdout:3547: done (3 left) 2026-02-17T06:08:16.663 INFO:tasks.rados.rados.0.trial115.stdout:3548: snap_create 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3546: finishing write tid 1 to trial11545420-5 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3545: finishing copy_from racing read to trial11545420-43 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3544: finishing copy_from racing read to trial11545420-15 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3544: finishing copy_from to trial11545420-15 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1804 (ObjNum 1168 snap 359 seq_num 1168) dirty exists 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3545: finishing copy_from to trial11545420-43 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1490 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3546: finishing write tid 2 to trial11545420-5 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3546: finishing write tid 3 to trial11545420-5 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3546: finishing write tid 4 to trial11545420-5 2026-02-17T06:08:16.664 INFO:tasks.rados.rados.0.trial115.stdout:3546: oid 5 updating version 0 to 1603 2026-02-17T06:08:16.665 INFO:tasks.rados.rados.0.trial115.stdout:3546: oid 5 version 1603 is already newer than 1602 2026-02-17T06:08:16.665 INFO:tasks.rados.rados.0.trial115.stdout:3546: oid 5 updating version 1603 to 1604 2026-02-17T06:08:16.665 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1604 (ObjNum 1217 snap 376 seq_num 1217) dirty exists 2026-02-17T06:08:16.665 INFO:tasks.rados.rados.0.trial115.stdout:3546: left oid 5 (ObjNum 1217 snap 376 seq_num 1217) 2026-02-17T06:08:17.664 INFO:tasks.rados.rados.0.trial115.stdout:3544: done (3 left) 2026-02-17T06:08:17.664 INFO:tasks.rados.rados.0.trial115.stdout:3545: done (2 left) 2026-02-17T06:08:17.665 INFO:tasks.rados.rados.0.trial115.stdout:3546: done (1 left) 2026-02-17T06:08:17.665 INFO:tasks.rados.rados.0.trial115.stdout:3548: done (0 left) 2026-02-17T06:08:17.665 INFO:tasks.rados.rados.0.trial115.stdout:3549: snap_create 2026-02-17T06:08:17.938 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:08:17.938 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T06:08:17.939 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T06:08:17.939 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T06:08:17.941 INFO:tasks.ceph.mon.b:Started 2026-02-17T06:08:17.941 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:08:17.942 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:08:18.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient: get_monmap_and_config 2026-02-17T06:08:18.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient: build_initial_monmap 2026-02-17T06:08:18.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient: monmap: 2026-02-17T06:08:18.028 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:18.028 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:18.028 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:18.010473+0000 2026-02-17T06:08:18.028 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:18.010473+0000 2026-02-17T06:08:18.028 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:18.028 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 1 Processor -- start 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 1 -- start start 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient: init 2026-02-17T06:08:18.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f43ec06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f43ec09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec06d380 0x7f43ec13ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient(hunting): picked mon.noname-b con 0x7f43ec06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43ec06daf0 con 0x7f43ec09c4a0 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43ec09cc80 con 0x7f43ec06c890 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f43ec0a0880 con 0x7f43ec06d380 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.009+0000 7f43f2951640 10 monclient(hunting): authenticate will time out at 1943.563994s 2026-02-17T06:08:18.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43eb7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f43ec09c4a0 0x7f43ec13b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55852/0 (socket says 10.20.193.78:55852) 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec06d380 0x7f43ec13ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43eb7fe640 1 -- 10.20.193.78:0/3663702030 learned_addr learned my addr 10.20.193.78:0/3663702030 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient(hunting): get_auth_request con 0x7f43ec06d380 auth_method 0 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient(hunting): handle_auth_done global_id 264185 payload 274 2026-02-17T06:08:18.031 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 1 -- 10.20.193.78:0/3663702030 >> v1:10.20.193.115:6789/0 conn(0x7f43ec06c890 legacy=0x7f43ec06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 1 -- 10.20.193.78:0/3663702030 >> v1:10.20.193.78:6789/0 conn(0x7f43ec09c4a0 legacy=0x7f43ec13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient: _finish_hunting 0 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 1 monclient: found mon.noname-b 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.010+0000 7f43f0ec7640 1 -- 10.20.193.78:0/3663702030 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec09cf60 con 0x7f43ec06d380 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43f0ec7640 1 -- 10.20.193.78:0/3663702030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec06d380 msgr2=0x7f43ec13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43f0ec7640 1 --2- 10.20.193.78:0/3663702030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec06d380 0x7f43ec13ead0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009dc0 tx=0x7f43e002f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61f2bb1685e844b8 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 1 -- 10.20.193.78:0/3663702030 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e003c070 con 0x7f43ec06d380 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: dump: 2026-02-17T06:08:18.032 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: _finish_auth 0 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:18.033 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 1 -- 10.20.193.78:0/3663702030 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e0037640 con 0x7f43ec06d380 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43f2951640 5 monclient: authenticate success, global_id 264185 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43f2951640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 1 -- 10.20.193.78:0/3663702030 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0037a10 con 0x7f43ec06d380 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43eaffd640 10 monclient: dump: 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:18.034 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43f2951640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.012+0000 7f43f2951640 4 set_mon_vals no callback set 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 10 monclient: shutdown 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 1 -- 10.20.193.78:0/3663702030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec06d380 msgr2=0x7f43ec13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 1 --2- 10.20.193.78:0/3663702030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec06d380 0x7f43ec13ead0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009dc0 tx=0x7f43e002f1c0 comp rx=0 tx=0).stop 2026-02-17T06:08:18.035 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 1 -- 10.20.193.78:0/3663702030 shutdown_connections 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 1 --2- 10.20.193.78:0/3663702030 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec06d380 0x7f43ec13ead0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 1 -- 10.20.193.78:0/3663702030 >> 10.20.193.78:0/3663702030 conn(0x7f43ec0a6620 msgr2=0x7f43ec0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 1 -- 10.20.193.78:0/3663702030 shutdown_connections 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 1 -- 10.20.193.78:0/3663702030 wait complete. 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 10 monclient: build_initial_monmap 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 10 monclient: monmap: 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:18.036 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.013+0000 7f43f2951640 1 Processor -- start 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 1 -- start start 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient: init 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f43ec06c890 0x7f43ec1ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient(hunting): picked mon.noname-a con 0x7f43ec06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43ec06d380 0x7f43ec1cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:18.037 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient(hunting): picked mon.noname-b con 0x7f43ec06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec09c4a0 0x7f43ec1cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient(hunting): picked mon.noname-c con 0x7f43ec09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec09bfa0 con 0x7f43ec06c890 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec09be20 con 0x7f43ec06d380 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f43ec13f010 con 0x7f43ec09c4a0 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f2951640 10 monclient(hunting): authenticate will time out at 1943.568279s 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f43ec06c890 0x7f43ec1ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:18.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec09c4a0 0x7f43ec1cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f43ec06c890 0x7f43ec1ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44782/0 (socket says 10.20.193.78:44782) 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 1 -- 10.20.193.78:0/1631882959 learned_addr learned my addr 10.20.193.78:0/1631882959 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec09c4a0 0x7f43ec1cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58078/0 (socket says 10.20.193.78:58078) 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 10 monclient(hunting): get_auth_request con 0x7f43ec06c890 auth_method 0 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient(hunting): get_auth_request con 0x7f43ec09c4a0 auth_method 0 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:18.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient(hunting): handle_auth_done global_id 264188 payload 824 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 1 -- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43ec06d380 msgr2=0x7f43ec1cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43ec06d380 0x7f43ec1cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 1 -- 10.20.193.78:0/1631882959 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f43ec06c890 msgr2=0x7f43ec1ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f43ec06c890 0x7f43ec1ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient: _finish_hunting 0 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 1 monclient: found mon.noname-c 2026-02-17T06:08:18.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.014+0000 7f43f0ec7640 1 -- 10.20.193.78:0/1631882959 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec09d810 con 0x7f43ec09c4a0 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f0ec7640 1 -- 10.20.193.78:0/1631882959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec09c4a0 msgr2=0x7f43ec1cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f0ec7640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec09c4a0 0x7f43ec1cda40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f43e0036000 tx=0x7f43e0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 1 -- 10.20.193.78:0/1631882959 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0047020 con 0x7f43ec09c4a0 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: dump: 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:18.041 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: _finish_auth 0 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 1 -- 10.20.193.78:0/1631882959 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e0042880 con 0x7f43ec09c4a0 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f2951640 5 monclient: authenticate success, global_id 264188 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f2951640 10 monclient: _renew_subs 2026-02-17T06:08:18.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f2951640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec11f980 con 0x7f43ec09c4a0 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 1 -- 10.20.193.78:0/1631882959 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e003c040 con 0x7f43ec09c4a0 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43e8ff9640 10 monclient: dump: 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:18.043 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43ea7fc640 4 set_mon_vals no callback set 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f2951640 10 monclient: _renew_subs 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f2951640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.016+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec072960 con 0x7f43ec09c4a0 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43f2951640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43f2951640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43f2951640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ec06cc70 con 0x7f43ec09c4a0 2026-02-17T06:08:18.044 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43e8ff9640 1 -- 10.20.193.78:0/1631882959 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f43e0042cb0 con 0x7f43ec09c4a0 2026-02-17T06:08:18.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43e8ff9640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43bc03df00 0x7f43bc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:18.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43e8ff9640 1 -- 10.20.193.78:0/1631882959 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(780..780 src has 525..780) ==== 4767+0+0 (secure 0 0 0) 0x7f43e00785b0 con 0x7f43ec09c4a0 2026-02-17T06:08:18.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43ebfff640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43bc03df00 0x7f43bc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:18.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.017+0000 7f43ebfff640 10 monclient: get_auth_request con 0x7f43bc03df00 auth_method 0 2026-02-17T06:08:18.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.020+0000 7f43e8ff9640 1 -- 10.20.193.78:0/1631882959 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f43e0051030 con 0x7f43ec09c4a0 2026-02-17T06:08:18.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.020+0000 7f43e8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:18.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.020+0000 7f43e8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:18.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.020+0000 7f43ebfff640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43bc03df00 0x7f43bc0403c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f43d80052b0 tx=0x7f43d8003960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:18.136 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.135+0000 7f43f2951640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:08:18.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.135+0000 7f43f2951640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:18.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.135+0000 7f43f2951640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:18.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.135+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f43ec13ecb0 con 0x7f43ec09c4a0 2026-02-17T06:08:18.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.135+0000 7f43e8ff9640 1 -- 10.20.193.78:0/1631882959 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f43ec09d810 con 0x7f43ec09c4a0 2026-02-17T06:08:18.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.135+0000 7f43e8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:18.137 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.135+0000 7f43e8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:18.138 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":250,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43bc03df00 msgr2=0x7f43bc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43bc03df00 0x7f43bc0403c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f43d80052b0 tx=0x7f43d8003960 comp rx=0 tx=0).stop 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 10 monclient: shutdown 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec09c4a0 msgr2=0x7f43ec1cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec09c4a0 0x7f43ec1cda40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f43e0036000 tx=0x7f43e0004b40 comp rx=0 tx=0).stop 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 shutdown_connections 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f43ec09c4a0 0x7f43ec1cda40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:18.139 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f43bc03df00 0x7f43bc0403c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:18.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f43ec06d380 0x7f43ec1cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:18.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 --2- 10.20.193.78:0/1631882959 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f43ec06c890 0x7f43ec1ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:18.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 >> 10.20.193.78:0/1631882959 conn(0x7f43ec0a6620 msgr2=0x7f43ec0715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:18.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 shutdown_connections 2026-02-17T06:08:18.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:18.136+0000 7f43f2951640 1 -- 10.20.193.78:0/1631882959 wait complete. 2026-02-17T06:08:18.668 INFO:tasks.rados.rados.0.trial115.stdout:3549: done (0 left) 2026-02-17T06:08:18.669 INFO:tasks.rados.rados.0.trial115.stdout:3550: write (excl) oid 14 current snap is 378 2026-02-17T06:08:18.669 INFO:tasks.rados.rados.0.trial115.stdout:3550: seq_num 1218 ranges {470926=669762,1574362=700418} 2026-02-17T06:08:18.673 INFO:tasks.rados.rados.0.trial115.stdout:3550: writing trial11545420-14 from 470926 to 1140688 tid 1 2026-02-17T06:08:18.677 INFO:tasks.rados.rados.0.trial115.stdout:3550: writing trial11545420-14 from 1574362 to 2274780 tid 2 2026-02-17T06:08:18.678 INFO:tasks.rados.rados.0.trial115.stdout:3551: delete oid 26 current snap is 378 2026-02-17T06:08:18.678 INFO:tasks.rados.rados.0.trial115.stdout:3550: finishing write tid 1 to trial11545420-14 2026-02-17T06:08:18.679 INFO:tasks.rados.rados.0.trial115.stdout:3551: done (1 left) 2026-02-17T06:08:18.679 INFO:tasks.rados.rados.0.trial115.stdout:3552: snap_remove snap 372 2026-02-17T06:08:19.676 INFO:tasks.rados.rados.0.trial115.stdout:3552: done (1 left) 2026-02-17T06:08:19.676 INFO:tasks.rados.rados.0.trial115.stdout:3553: read oid 16 snap -1 2026-02-17T06:08:19.676 INFO:tasks.rados.rados.0.trial115.stdout:3553: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:08:19.676 INFO:tasks.rados.rados.0.trial115.stdout:3550: finishing write tid 2 to trial11545420-14 2026-02-17T06:08:19.676 INFO:tasks.rados.rados.0.trial115.stdout:3550: finishing write tid 3 to trial11545420-14 2026-02-17T06:08:19.676 INFO:tasks.rados.rados.0.trial115.stdout:3550: finishing write tid 4 to trial11545420-14 2026-02-17T06:08:19.676 INFO:tasks.rados.rados.0.trial115.stdout:3550: oid 14 updating version 0 to 1674 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:3550: oid 14 updating version 1674 to 1675 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:3550: oid 14 version 1675 is already newer than 1673 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1675 (ObjNum 1218 snap 378 seq_num 1218) dirty exists 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:3550: left oid 14 (ObjNum 1218 snap 378 seq_num 1218) 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:3554: read oid 45 snap -1 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:3554: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:3550: done (2 left) 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:3555: write (excl) oid 1 current snap is 378 2026-02-17T06:08:19.677 INFO:tasks.rados.rados.0.trial115.stdout:3555: seq_num 1220 ranges {504540=420260,1574973=704087} 2026-02-17T06:08:19.679 INFO:tasks.rados.rados.0.trial115.stdout:3555: writing trial11545420-1 from 504540 to 924800 tid 1 2026-02-17T06:08:19.685 INFO:tasks.rados.rados.0.trial115.stdout:3555: writing trial11545420-1 from 1574973 to 2279060 tid 2 2026-02-17T06:08:19.685 INFO:tasks.rados.rados.0.trial115.stdout:3556: write (excl) oid 40 current snap is 378 2026-02-17T06:08:19.685 INFO:tasks.rados.rados.0.trial115.stdout:3556: seq_num 1221 ranges {521347=495509,1633455=564098} 2026-02-17T06:08:19.688 INFO:tasks.rados.rados.0.trial115.stdout:3556: writing trial11545420-40 from 521347 to 1016856 tid 1 2026-02-17T06:08:19.693 INFO:tasks.rados.rados.0.trial115.stdout:3556: writing trial11545420-40 from 1633455 to 2197553 tid 2 2026-02-17T06:08:19.693 INFO:tasks.rados.rados.0.trial115.stdout:3557: snap_remove snap 370 2026-02-17T06:08:20.645 INFO:tasks.rados.rados.0.trial115.stdout:3557: done (4 left) 2026-02-17T06:08:20.645 INFO:tasks.rados.rados.0.trial115.stdout:3558: write (excl) oid 28 current snap is 378 2026-02-17T06:08:20.645 INFO:tasks.rados.rados.0.trial115.stdout:3558: seq_num 1222 ranges {538154=570758,1691937=707756} 2026-02-17T06:08:20.650 INFO:tasks.rados.rados.0.trial115.stdout:3558: writing trial11545420-28 from 538154 to 1108912 tid 1 2026-02-17T06:08:20.655 INFO:tasks.rados.rados.0.trial115.stdout:3558: writing trial11545420-28 from 1691937 to 2399693 tid 2 2026-02-17T06:08:20.655 INFO:tasks.rados.rados.0.trial115.stdout:3559: snap_remove snap 374 2026-02-17T06:08:21.146 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:08:21.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.214+0000 7fe67397e640 10 monclient: get_monmap_and_config 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.214+0000 7fe67397e640 10 monclient: build_initial_monmap 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.214+0000 7fe67397e640 10 monclient: monmap: 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:21.215820+0000 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:21.215820+0000 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:21.218 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.214+0000 7fe67397e640 1 Processor -- start 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 1 -- start start 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient: init 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fe66c06c890 0x7fe66c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:21.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient(hunting): picked mon.noname-c con 0x7fe66c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe66c09c4a0 0x7fe66c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient(hunting): picked mon.noname-a con 0x7fe66c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe66c134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c06daf0 con 0x7fe66c134df0 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c0a0880 con 0x7fe66c09c4a0 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c06d6f0 con 0x7fe66c06c890 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe67397e640 10 monclient(hunting): authenticate will time out at 1946.769356s 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe66c09c4a0 0x7fe66c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:21.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe66c09c4a0 0x7fe66c06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44794/0 (socket says 10.20.193.78:44794) 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 1 -- 10.20.193.78:0/875924556 learned_addr learned my addr 10.20.193.78:0/875924556 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient(hunting): get_auth_request con 0x7fe66c09c4a0 auth_method 0 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe6716f3640 1 --2- 10.20.193.78:0/875924556 >> v2:10.20.193.115:3300/0 conn(0x7fe66c06c890 0x7fe66c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient(hunting): handle_auth_done global_id 254280 payload 274 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 1 -- 10.20.193.78:0/875924556 >> v2:10.20.193.115:3300/0 conn(0x7fe66c06c890 msgr2=0x7fe66c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 1 --2- 10.20.193.78:0/875924556 >> v2:10.20.193.115:3300/0 conn(0x7fe66c06c890 0x7fe66c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 1 -- 10.20.193.78:0/875924556 >> v1:10.20.193.78:6789/0 conn(0x7fe66c134df0 legacy=0x7fe66c135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:21.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient: _finish_hunting 0 2026-02-17T06:08:21.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 1 monclient: found mon.noname-a 2026-02-17T06:08:21.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:21.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:08:21.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.215+0000 7fe670ef2640 1 -- 10.20.193.78:0/875924556 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c09cc80 con 0x7fe66c09c4a0 2026-02-17T06:08:21.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.217+0000 7fe670ef2640 1 -- 10.20.193.78:0/875924556 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c09c4a0 msgr2=0x7fe66c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:21.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe670ef2640 1 --2- 10.20.193.78:0/875924556 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c09c4a0 0x7fe66c06d1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe66c06a480 tx=0x7fe66002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905975e081305401 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:21.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 1 -- 10.20.193.78:0/875924556 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66003c070 con 0x7fe66c09c4a0 2026-02-17T06:08:21.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:21.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:08:21.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: dump: 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:21.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: _finish_auth 0 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 1 -- 10.20.193.78:0/875924556 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66002fd80 con 0x7fe66c09c4a0 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 1 -- 10.20.193.78:0/875924556 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660037660 con 0x7fe66c09c4a0 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 5 monclient: authenticate success, global_id 254280 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 4 set_mon_vals no callback set 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe65bfff640 10 monclient: dump: 2026-02-17T06:08:21.231 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 10 monclient: shutdown 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:21.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 1 -- 10.20.193.78:0/875924556 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c09c4a0 msgr2=0x7fe66c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 1 --2- 10.20.193.78:0/875924556 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c09c4a0 0x7fe66c06d1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe66c06a480 tx=0x7fe66002ee60 comp rx=0 tx=0).stop 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 1 -- 10.20.193.78:0/875924556 shutdown_connections 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 1 --2- 10.20.193.78:0/875924556 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c09c4a0 0x7fe66c06d1b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 1 --2- 10.20.193.78:0/875924556 >> v2:10.20.193.115:3300/0 conn(0x7fe66c06c890 0x7fe66c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 1 -- 10.20.193.78:0/875924556 >> 10.20.193.78:0/875924556 conn(0x7fe66c0a6620 msgr2=0x7fe66c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 1 -- 10.20.193.78:0/875924556 shutdown_connections 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 1 -- 10.20.193.78:0/875924556 wait complete. 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 10 monclient: build_initial_monmap 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.218+0000 7fe67397e640 10 monclient: monmap: 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:21.233 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 1 Processor -- start 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 1 -- start start 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient: init 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:08:21.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe66c06c890 0x7fe66c1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient(hunting): picked mon.noname-c con 0x7fe66c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe66c09c4a0 0x7fe66c1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient(hunting): picked mon.noname-b con 0x7fe66c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c134df0 0x7fe66c121780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient(hunting): picked mon.noname-a con 0x7fe66c134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0a01a0 con 0x7fe66c134df0 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0a0020 con 0x7fe66c09c4a0 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c09be20 con 0x7fe66c06c890 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:21.235 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe67397e640 10 monclient(hunting): authenticate will time out at 1946.773502s 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c134df0 0x7fe66c121780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe6716f3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe66c06c890 0x7fe66c1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c134df0 0x7fe66c121780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44800/0 (socket says 10.20.193.78:44800) 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 -- 10.20.193.78:0/189205045 learned_addr learned my addr 10.20.193.78:0/189205045 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient(hunting): get_auth_request con 0x7fe66c134df0 auth_method 0 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe6716f3640 10 monclient(hunting): get_auth_request con 0x7fe66c06c890 auth_method 0 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe6716f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:21.236 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe6716f3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe6716f3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe6716f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe6716f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe6716f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe670ef2640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe66c09c4a0 0x7fe66c1cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe670ef2640 10 monclient(hunting): get_auth_request con 0x7fe66c09c4a0 auth_method 0 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe670ef2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe670ef2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe670ef2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:21.237 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient(hunting): handle_auth_done global_id 254283 payload 824 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 -- 10.20.193.78:0/189205045 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe66c06c890 msgr2=0x7fe66c1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe66c06c890 0x7fe66c1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 -- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe66c09c4a0 msgr2=0x7fe66c1cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe66c09c4a0 0x7fe66c1cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient: _finish_hunting 0 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 monclient: found mon.noname-a 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.219+0000 7fe671ef4640 1 -- 10.20.193.78:0/189205045 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c072770 con 0x7fe66c134df0 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe671ef4640 1 -- 10.20.193.78:0/189205045 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c134df0 msgr2=0x7fe66c121780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe671ef4640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c134df0 0x7fe66c121780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe66800cb60 tx=0x7fe66800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:21.238 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 1 -- 10.20.193.78:0/189205045 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668016020 con 0x7fe66c134df0 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: dump: 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:21.239 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: _finish_auth 0 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 1 -- 10.20.193.78:0/189205045 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe668012420 con 0x7fe66c134df0 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 1 -- 10.20.193.78:0/189205045 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668012810 con 0x7fe66c134df0 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe659ffb640 10 monclient: dump: 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:21.240 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe67397e640 5 monclient: authenticate success, global_id 254283 2026-02-17T06:08:21.241 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe65b7fe640 4 set_mon_vals no callback set 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe67397e640 10 monclient: _renew_subs 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe67397e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.221+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c125100 con 0x7fe66c134df0 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.222+0000 7fe67397e640 10 monclient: _renew_subs 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.222+0000 7fe67397e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.222+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c123f30 con 0x7fe66c134df0 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.222+0000 7fe67397e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.222+0000 7fe67397e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.222+0000 7fe67397e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.222+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c074980 con 0x7fe66c134df0 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.222+0000 7fe659ffb640 1 -- 10.20.193.78:0/189205045 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe6680129f0 con 0x7fe66c134df0 2026-02-17T06:08:21.242 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.223+0000 7fe659ffb640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe63c03deb0 0x7fe63c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:21.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.223+0000 7fe659ffb640 1 -- 10.20.193.78:0/189205045 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(783..783 src has 525..783) ==== 4807+0+0 (secure 0 0 0) 0x7fe6680531a0 con 0x7fe66c134df0 2026-02-17T06:08:21.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.223+0000 7fe6716f3640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe63c03deb0 0x7fe63c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:21.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.223+0000 7fe6716f3640 10 monclient: get_auth_request con 0x7fe63c03deb0 auth_method 0 2026-02-17T06:08:21.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.225+0000 7fe659ffb640 1 -- 10.20.193.78:0/189205045 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe668020020 con 0x7fe66c134df0 2026-02-17T06:08:21.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.225+0000 7fe659ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:21.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.225+0000 7fe659ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:21.243 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.225+0000 7fe6716f3640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe63c03deb0 0x7fe63c040370 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00ab20 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:21.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.337+0000 7fe67397e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:08:21.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.337+0000 7fe67397e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:21.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.337+0000 7fe67397e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:21.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.337+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe66c135a20 con 0x7fe66c134df0 2026-02-17T06:08:21.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe659ffb640 1 -- 10.20.193.78:0/189205045 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fe66c072770 con 0x7fe66c134df0 2026-02-17T06:08:21.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe659ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:21.340 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe659ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:21.340 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":250,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:08:21.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe63c03deb0 msgr2=0x7fe63c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:21.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe63c03deb0 0x7fe63c040370 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00ab20 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2026-02-17T06:08:21.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 10 monclient: shutdown 2026-02-17T06:08:21.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:21.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c134df0 msgr2=0x7fe66c121780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c134df0 0x7fe66c121780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe66800cb60 tx=0x7fe66800a430 comp rx=0 tx=0).stop 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 shutdown_connections 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe66c134df0 0x7fe66c121780 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe63c03deb0 0x7fe63c040370 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe66c09c4a0 0x7fe66c1cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 --2- 10.20.193.78:0/189205045 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe66c06c890 0x7fe66c1cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.338+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 >> 10.20.193.78:0/189205045 conn(0x7fe66c0a6620 msgr2=0x7fe66c0a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.339+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 shutdown_connections 2026-02-17T06:08:21.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:21.339+0000 7fe67397e640 1 -- 10.20.193.78:0/189205045 wait complete. 2026-02-17T06:08:21.648 INFO:tasks.rados.rados.0.trial115.stdout:3559: done (5 left) 2026-02-17T06:08:21.648 INFO:tasks.rados.rados.0.trial115.stdout:3560: delete oid 48 current snap is 378 2026-02-17T06:08:21.649 INFO:tasks.rados.rados.0.trial115.stdout:3554: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:08:21.766 INFO:tasks.rados.rados.0.trial115.stdout:3553: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:08:21.966 INFO:tasks.rados.rados.0.trial115.stdout:3555: finishing write tid 1 to trial11545420-1 2026-02-17T06:08:21.966 INFO:tasks.rados.rados.0.trial115.stdout:3555: finishing write tid 2 to trial11545420-1 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3555: finishing write tid 3 to trial11545420-1 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3555: finishing write tid 4 to trial11545420-1 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3555: oid 1 updating version 0 to 1678 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3555: oid 1 version 1678 is already newer than 1677 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3555: oid 1 version 1678 is already newer than 1676 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 1678 (ObjNum 1220 snap 378 seq_num 1220) dirty exists 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3555: left oid 1 (ObjNum 1220 snap 378 seq_num 1220) 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3556: finishing write tid 1 to trial11545420-40 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3556: finishing write tid 2 to trial11545420-40 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3556: finishing write tid 3 to trial11545420-40 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3556: finishing write tid 4 to trial11545420-40 2026-02-17T06:08:21.967 INFO:tasks.rados.rados.0.trial115.stdout:3556: oid 40 updating version 0 to 1487 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3556: oid 40 updating version 1487 to 1488 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3556: oid 40 version 1488 is already newer than 1486 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1488 (ObjNum 1221 snap 378 seq_num 1221) dirty exists 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3556: left oid 40 (ObjNum 1221 snap 378 seq_num 1221) 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3558: finishing write tid 1 to trial11545420-28 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3558: finishing write tid 2 to trial11545420-28 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3553: done (5 left) 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3554: done (4 left) 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3555: done (3 left) 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3556: done (2 left) 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3560: done (1 left) 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3561: read oid 40 snap -1 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3561: expect (ObjNum 1221 snap 378 seq_num 1221) 2026-02-17T06:08:21.968 INFO:tasks.rados.rados.0.trial115.stdout:3562: write (excl) oid 37 current snap is 378 2026-02-17T06:08:21.969 INFO:tasks.rados.rados.0.trial115.stdout:3558: finishing write tid 3 to trial11545420-28 2026-02-17T06:08:21.969 INFO:tasks.rados.rados.0.trial115.stdout:3558: finishing write tid 4 to trial11545420-28 2026-02-17T06:08:21.969 INFO:tasks.rados.rados.0.trial115.stdout:3558: oid 28 updating version 0 to 1883 2026-02-17T06:08:21.969 INFO:tasks.rados.rados.0.trial115.stdout:3558: oid 28 version 1883 is already newer than 1882 2026-02-17T06:08:21.969 INFO:tasks.rados.rados.0.trial115.stdout:3558: oid 28 updating version 1883 to 1884 2026-02-17T06:08:21.969 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1884 (ObjNum 1222 snap 378 seq_num 1222) dirty exists 2026-02-17T06:08:21.969 INFO:tasks.rados.rados.0.trial115.stdout:3558: left oid 28 (ObjNum 1222 snap 378 seq_num 1222) 2026-02-17T06:08:21.969 INFO:tasks.rados.rados.0.trial115.stdout:3562: seq_num 1224 ranges {571768=437609,1808901=711425} 2026-02-17T06:08:21.970 INFO:tasks.rados.rados.0.trial115.stdout:3562: writing trial11545420-37 from 571768 to 1009377 tid 1 2026-02-17T06:08:21.975 INFO:tasks.rados.rados.0.trial115.stdout:3562: writing trial11545420-37 from 1808901 to 2520326 tid 2 2026-02-17T06:08:21.975 INFO:tasks.rados.rados.0.trial115.stdout:3558: done (2 left) 2026-02-17T06:08:21.976 INFO:tasks.rados.rados.0.trial115.stdout:3563: read oid 31 snap -1 2026-02-17T06:08:21.976 INFO:tasks.rados.rados.0.trial115.stdout:3563: expect (ObjNum 1162 snap 355 seq_num 1162) 2026-02-17T06:08:21.976 INFO:tasks.rados.rados.0.trial115.stdout:3564: snap_create 2026-02-17T06:08:21.976 INFO:tasks.rados.rados.0.trial115.stdout:3561: expect (ObjNum 1221 snap 378 seq_num 1221) 2026-02-17T06:08:22.097 INFO:tasks.rados.rados.0.trial115.stdout:3562: finishing write tid 1 to trial11545420-37 2026-02-17T06:08:22.098 INFO:tasks.rados.rados.0.trial115.stdout:3562: finishing write tid 2 to trial11545420-37 2026-02-17T06:08:22.098 INFO:tasks.rados.rados.0.trial115.stdout:3562: finishing write tid 3 to trial11545420-37 2026-02-17T06:08:22.098 INFO:tasks.rados.rados.0.trial115.stdout:3563: expect (ObjNum 1162 snap 355 seq_num 1162) 2026-02-17T06:08:22.248 INFO:tasks.rados.rados.0.trial115.stdout:3562: finishing write tid 4 to trial11545420-37 2026-02-17T06:08:22.248 INFO:tasks.rados.rados.0.trial115.stdout:3562: oid 37 updating version 0 to 1713 2026-02-17T06:08:22.248 INFO:tasks.rados.rados.0.trial115.stdout:3562: oid 37 updating version 1713 to 1714 2026-02-17T06:08:22.248 INFO:tasks.rados.rados.0.trial115.stdout:3562: oid 37 version 1714 is already newer than 1712 2026-02-17T06:08:22.248 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1714 (ObjNum 1224 snap 378 seq_num 1224) dirty exists 2026-02-17T06:08:22.248 INFO:tasks.rados.rados.0.trial115.stdout:3562: left oid 37 (ObjNum 1224 snap 378 seq_num 1224) 2026-02-17T06:08:22.682 INFO:tasks.rados.rados.0.trial115.stdout:3561: done (3 left) 2026-02-17T06:08:22.682 INFO:tasks.rados.rados.0.trial115.stdout:3562: done (2 left) 2026-02-17T06:08:22.683 INFO:tasks.rados.rados.0.trial115.stdout:3563: done (1 left) 2026-02-17T06:08:22.683 INFO:tasks.rados.rados.0.trial115.stdout:3564: done (0 left) 2026-02-17T06:08:22.683 INFO:tasks.rados.rados.0.trial115.stdout:3565: read oid 42 snap -1 2026-02-17T06:08:22.683 INFO:tasks.rados.rados.0.trial115.stdout:3565: expect deleted 2026-02-17T06:08:22.683 INFO:tasks.rados.rados.0.trial115.stdout:3566: copy_from oid 20 from oid 23 current snap is 379 2026-02-17T06:08:22.683 INFO:tasks.rados.rados.0.trial115.stdout:3567: write oid 47 current snap is 379 2026-02-17T06:08:22.683 INFO:tasks.rados.rados.0.trial115.stdout:3567: seq_num 1225 ranges {588575=512858,1867383=455083} 2026-02-17T06:08:22.686 INFO:tasks.rados.rados.0.trial115.stdout:3567: writing trial11545420-47 from 588575 to 1101433 tid 1 2026-02-17T06:08:22.691 INFO:tasks.rados.rados.0.trial115.stdout:3567: writing trial11545420-47 from 1867383 to 2322466 tid 2 2026-02-17T06:08:22.691 INFO:tasks.rados.rados.0.trial115.stdout:3568: write oid 46 current snap is 379 2026-02-17T06:08:22.691 INFO:tasks.rados.rados.0.trial115.stdout:3568: seq_num 1226 ranges {605382=588107,1925865=679517} 2026-02-17T06:08:22.695 INFO:tasks.rados.rados.0.trial115.stdout:3568: writing trial11545420-46 from 605382 to 1193489 tid 1 2026-02-17T06:08:22.701 INFO:tasks.rados.rados.0.trial115.stdout:3568: writing trial11545420-46 from 1925865 to 2605382 tid 2 2026-02-17T06:08:22.701 INFO:tasks.rados.rados.0.trial115.stdout:3569: write (excl) oid 40 current snap is 379 2026-02-17T06:08:22.701 INFO:tasks.rados.rados.0.trial115.stdout:3566: finishing copy_from to trial11545420-20 2026-02-17T06:08:22.701 INFO:tasks.rados.rados.0.trial115.stdout:3566: got expected ENOENT (src dne) 2026-02-17T06:08:22.701 INFO:tasks.rados.rados.0.trial115.stdout:3566: finishing copy_from racing read to trial11545420-20 2026-02-17T06:08:22.701 INFO:tasks.rados.rados.0.trial115.stdout:3567: finishing write tid 1 to trial11545420-47 2026-02-17T06:08:22.702 INFO:tasks.rados.rados.0.trial115.stdout:3567: finishing write tid 2 to trial11545420-47 2026-02-17T06:08:22.702 INFO:tasks.rados.rados.0.trial115.stdout:3567: finishing write tid 3 to trial11545420-47 2026-02-17T06:08:22.702 INFO:tasks.rados.rados.0.trial115.stdout:3569: seq_num 1227 ranges {622189=663356,1984347=458752} 2026-02-17T06:08:22.705 INFO:tasks.rados.rados.0.trial115.stdout:3569: writing trial11545420-40 from 622189 to 1285545 tid 1 2026-02-17T06:08:22.711 INFO:tasks.rados.rados.0.trial115.stdout:3569: writing trial11545420-40 from 1984347 to 2443099 tid 2 2026-02-17T06:08:22.711 INFO:tasks.rados.rados.0.trial115.stdout:3565: done (4 left) 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3566: done (3 left) 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3570: read oid 36 snap -1 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3570: expect (ObjNum 1197 snap 366 seq_num 1197) 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3571: read oid 20 snap -1 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3571: expect (ObjNum 1201 snap 370 seq_num 1201) 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3572: rollback oid 43 current snap is 379 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 376 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3567: finishing write tid 4 to trial11545420-47 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3567: oid 47 updating version 0 to 1621 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3567: oid 47 updating version 1621 to 1623 2026-02-17T06:08:22.712 INFO:tasks.rados.rados.0.trial115.stdout:3567: oid 47 version 1623 is already newer than 1622 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1623 (ObjNum 1225 snap 379 seq_num 1225) dirty exists 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3567: left oid 47 (ObjNum 1225 snap 379 seq_num 1225) 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3568: finishing write tid 1 to trial11545420-46 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3568: finishing write tid 2 to trial11545420-46 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3568: finishing write tid 3 to trial11545420-46 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3568: finishing write tid 4 to trial11545420-46 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3568: oid 46 updating version 0 to 1890 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3568: oid 46 updating version 1890 to 1892 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3568: oid 46 version 1892 is already newer than 1891 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1892 (ObjNum 1226 snap 379 seq_num 1226) dirty exists 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3568: left oid 46 (ObjNum 1226 snap 379 seq_num 1226) 2026-02-17T06:08:22.713 INFO:tasks.rados.rados.0.trial115.stdout:3569: finishing write tid 1 to trial11545420-40 2026-02-17T06:08:22.714 INFO:tasks.rados.rados.0.trial115.stdout:3567: done (5 left) 2026-02-17T06:08:22.714 INFO:tasks.rados.rados.0.trial115.stdout:3568: done (4 left) 2026-02-17T06:08:22.714 INFO:tasks.rados.rados.0.trial115.stdout:3573: rollback oid 28 current snap is 379 2026-02-17T06:08:22.714 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 28 to 377 2026-02-17T06:08:22.714 INFO:tasks.rados.rados.0.trial115.stdout:3574: snap_create 2026-02-17T06:08:22.715 INFO:tasks.rados.rados.0.trial115.stdout:3569: finishing write tid 2 to trial11545420-40 2026-02-17T06:08:22.715 INFO:tasks.rados.rados.0.trial115.stdout:3569: finishing write tid 3 to trial11545420-40 2026-02-17T06:08:22.715 INFO:tasks.rados.rados.0.trial115.stdout:3569: finishing write tid 4 to trial11545420-40 2026-02-17T06:08:22.715 INFO:tasks.rados.rados.0.trial115.stdout:3569: oid 40 updating version 0 to 1491 2026-02-17T06:08:22.715 INFO:tasks.rados.rados.0.trial115.stdout:3569: oid 40 version 1491 is already newer than 1490 2026-02-17T06:08:22.715 INFO:tasks.rados.rados.0.trial115.stdout:3569: oid 40 version 1491 is already newer than 1489 2026-02-17T06:08:22.715 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1491 (ObjNum 1227 snap 379 seq_num 1227) dirty exists 2026-02-17T06:08:22.716 INFO:tasks.rados.rados.0.trial115.stdout:3569: left oid 40 (ObjNum 1227 snap 379 seq_num 1227) 2026-02-17T06:08:22.716 INFO:tasks.rados.rados.0.trial115.stdout:3573: finishing rollback tid 0 to trial11545420-28 2026-02-17T06:08:22.716 INFO:tasks.rados.rados.0.trial115.stdout:3573: finishing rollback tid 1 to trial11545420-28 2026-02-17T06:08:22.716 INFO:tasks.rados.rados.0.trial115.stdout:3573: finishing rollback tid 2 to trial11545420-28 2026-02-17T06:08:22.716 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 28 v 1896 (ObjNum 1203 snap 370 seq_num 1203) dirty exists 2026-02-17T06:08:22.716 INFO:tasks.rados.rados.0.trial115.stdout:3570: expect (ObjNum 1197 snap 366 seq_num 1197) 2026-02-17T06:08:22.765 INFO:tasks.rados.rados.0.trial115.stdout:3571: expect (ObjNum 1201 snap 370 seq_num 1201) 2026-02-17T06:08:22.890 INFO:tasks.rados.rados.0.trial115.stdout:3572: finishing rollback tid 0 to trial11545420-43 2026-02-17T06:08:22.890 INFO:tasks.rados.rados.0.trial115.stdout:3572: finishing rollback tid 1 to trial11545420-43 2026-02-17T06:08:22.890 INFO:tasks.rados.rados.0.trial115.stdout:3572: finishing rollback tid 2 to trial11545420-43 2026-02-17T06:08:22.890 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1717 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:08:23.683 INFO:tasks.rados.rados.0.trial115.stdout:3569: done (5 left) 2026-02-17T06:08:23.683 INFO:tasks.rados.rados.0.trial115.stdout:3570: done (4 left) 2026-02-17T06:08:23.683 INFO:tasks.rados.rados.0.trial115.stdout:3571: done (3 left) 2026-02-17T06:08:23.683 INFO:tasks.rados.rados.0.trial115.stdout:3572: done (2 left) 2026-02-17T06:08:23.683 INFO:tasks.rados.rados.0.trial115.stdout:3573: done (1 left) 2026-02-17T06:08:23.683 INFO:tasks.rados.rados.0.trial115.stdout:3574: done (0 left) 2026-02-17T06:08:23.683 INFO:tasks.rados.rados.0.trial115.stdout:3575: rollback oid 50 current snap is 380 2026-02-17T06:08:23.684 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 50 to 377 2026-02-17T06:08:23.684 INFO:tasks.rados.rados.0.trial115.stdout:3576: read oid 22 snap -1 2026-02-17T06:08:23.684 INFO:tasks.rados.rados.0.trial115.stdout:3576: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:08:23.684 INFO:tasks.rados.rados.0.trial115.stdout:3577: delete oid 39 current snap is 380 2026-02-17T06:08:23.687 INFO:tasks.rados.rados.0.trial115.stdout:3577: done (2 left) 2026-02-17T06:08:23.687 INFO:tasks.rados.rados.0.trial115.stdout:3578: write oid 13 current snap is 380 2026-02-17T06:08:23.687 INFO:tasks.rados.rados.0.trial115.stdout:3578: seq_num 1229 ranges {655803=413854,1584958=462421} 2026-02-17T06:08:23.691 INFO:tasks.rados.rados.0.trial115.stdout:3578: writing trial11545420-13 from 655803 to 1069657 tid 1 2026-02-17T06:08:23.696 INFO:tasks.rados.rados.0.trial115.stdout:3578: writing trial11545420-13 from 1584958 to 2047379 tid 2 2026-02-17T06:08:23.696 INFO:tasks.rados.rados.0.trial115.stdout:3578: finishing write tid 1 to trial11545420-13 2026-02-17T06:08:23.696 INFO:tasks.rados.rados.0.trial115.stdout:3579: read oid 46 snap -1 2026-02-17T06:08:23.696 INFO:tasks.rados.rados.0.trial115.stdout:3579: expect (ObjNum 1226 snap 379 seq_num 1226) 2026-02-17T06:08:23.696 INFO:tasks.rados.rados.0.trial115.stdout:3580: read oid 16 snap -1 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:3580: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:3581: rollback oid 44 current snap is 380 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 44 to 379 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:3575: finishing rollback tid 0 to trial11545420-50 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:3575: finishing rollback tid 1 to trial11545420-50 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:3575: finishing rollback tid 2 to trial11545420-50 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1485 (ObjNum 1099 snap 329 seq_num 1099) dirty exists 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:3575: done (5 left) 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:3582: write (excl) oid 29 current snap is 380 2026-02-17T06:08:23.697 INFO:tasks.rados.rados.0.trial115.stdout:3582: seq_num 1230 ranges {672610=489103,1643440=722432} 2026-02-17T06:08:23.700 INFO:tasks.rados.rados.0.trial115.stdout:3582: writing trial11545420-29 from 672610 to 1161713 tid 1 2026-02-17T06:08:23.705 INFO:tasks.rados.rados.0.trial115.stdout:3582: writing trial11545420-29 from 1643440 to 2365872 tid 2 2026-02-17T06:08:23.705 INFO:tasks.rados.rados.0.trial115.stdout:3583: write oid 35 current snap is 380 2026-02-17T06:08:23.705 INFO:tasks.rados.rados.0.trial115.stdout:3583: seq_num 1231 ranges {689417=564352,1701922=466090} 2026-02-17T06:08:23.709 INFO:tasks.rados.rados.0.trial115.stdout:3583: writing trial11545420-35 from 689417 to 1253769 tid 1 2026-02-17T06:08:23.713 INFO:tasks.rados.rados.0.trial115.stdout:3583: writing trial11545420-35 from 1701922 to 2168012 tid 2 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3584: rollback oid 15 current snap is 380 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 15 to 376 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3578: finishing write tid 2 to trial11545420-13 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3578: finishing write tid 3 to trial11545420-13 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3578: finishing write tid 4 to trial11545420-13 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3578: oid 13 updating version 0 to 1485 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3578: oid 13 version 1485 is already newer than 1484 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3578: oid 13 updating version 1485 to 1486 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 13 v 1486 (ObjNum 1229 snap 380 seq_num 1229) dirty exists 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3578: left oid 13 (ObjNum 1229 snap 380 seq_num 1229) 2026-02-17T06:08:23.714 INFO:tasks.rados.rados.0.trial115.stdout:3581: finishing rollback tid 0 to trial11545420-44 2026-02-17T06:08:23.715 INFO:tasks.rados.rados.0.trial115.stdout:3581: finishing rollback tid 1 to trial11545420-44 2026-02-17T06:08:23.715 INFO:tasks.rados.rados.0.trial115.stdout:3581: finishing rollback tid 2 to trial11545420-44 2026-02-17T06:08:23.715 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1474 (ObjNum 1050 snap 312 seq_num 1050) dirty exists 2026-02-17T06:08:23.715 INFO:tasks.rados.rados.0.trial115.stdout:3579: expect (ObjNum 1226 snap 379 seq_num 1226) 2026-02-17T06:08:23.811 INFO:tasks.rados.rados.0.trial115.stdout:3576: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:08:23.974 INFO:tasks.rados.rados.0.trial115.stdout:3582: finishing write tid 1 to trial11545420-29 2026-02-17T06:08:23.974 INFO:tasks.rados.rados.0.trial115.stdout:3580: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:08:24.174 INFO:tasks.rados.rados.0.trial115.stdout:3582: finishing write tid 2 to trial11545420-29 2026-02-17T06:08:24.174 INFO:tasks.rados.rados.0.trial115.stdout:3576: done (7 left) 2026-02-17T06:08:24.175 INFO:tasks.rados.rados.0.trial115.stdout:3578: done (6 left) 2026-02-17T06:08:24.175 INFO:tasks.rados.rados.0.trial115.stdout:3579: done (5 left) 2026-02-17T06:08:24.175 INFO:tasks.rados.rados.0.trial115.stdout:3580: done (4 left) 2026-02-17T06:08:24.175 INFO:tasks.rados.rados.0.trial115.stdout:3581: done (3 left) 2026-02-17T06:08:24.175 INFO:tasks.rados.rados.0.trial115.stdout:3585: write (excl) oid 22 current snap is 380 2026-02-17T06:08:24.175 INFO:tasks.rados.rados.0.trial115.stdout:3585: seq_num 1232 ranges {706224=755954,2160404=545820} 2026-02-17T06:08:24.179 INFO:tasks.rados.rados.0.trial115.stdout:3585: writing trial11545420-22 from 706224 to 1462178 tid 1 2026-02-17T06:08:24.184 INFO:tasks.rados.rados.0.trial115.stdout:3585: writing trial11545420-22 from 2160404 to 2706224 tid 2 2026-02-17T06:08:24.184 INFO:tasks.rados.rados.0.trial115.stdout:3586: snap_remove snap 377 2026-02-17T06:08:24.348 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:08:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.415+0000 7fa2ea33b640 10 monclient: get_monmap_and_config 2026-02-17T06:08:24.418 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.415+0000 7fa2ea33b640 10 monclient: build_initial_monmap 2026-02-17T06:08:24.419 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.415+0000 7fa2ea33b640 10 monclient: monmap: 2026-02-17T06:08:24.419 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:24.419 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.419 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:24.416642+0000 2026-02-17T06:08:24.419 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:24.416642+0000 2026-02-17T06:08:24.419 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:24.419 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.420 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:24.420 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:24.420 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:24.420 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:24.420 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:24.420 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.420 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.415+0000 7fa2ea33b640 1 Processor -- start 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.415+0000 7fa2ea33b640 1 -- start start 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.415+0000 7fa2ea33b640 10 monclient: init 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 10 monclient: _add_conns ranks=[0,1,3,2,4] 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa2e406c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa2e409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fa2e406d380 0x7fa2e413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 10 monclient(hunting): picked mon.noname-a con 0x7fa2e406d380 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:24.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e406daf0 con 0x7fa2e406c890 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e409cc80 con 0x7fa2e409c4a0 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e40a0880 con 0x7fa2e406d380 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2ea33b640 10 monclient(hunting): authenticate will time out at 1949.970237s 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e3fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fa2e406c890 0x7fa2e406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55878/0 (socket says 10.20.193.78:55878) 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e3fff640 1 -- 10.20.193.78:0/1539918194 learned_addr learned my addr 10.20.193.78:0/1539918194 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 1 --2- 10.20.193.78:0/1539918194 >> v2:10.20.193.78:3300/0 conn(0x7fa2e406d380 0x7fa2e413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient(hunting): get_auth_request con 0x7fa2e406d380 auth_method 0 2026-02-17T06:08:24.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient(hunting): handle_auth_done global_id 254286 payload 274 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1539918194 >> v1:10.20.193.115:6789/0 conn(0x7fa2e409c4a0 legacy=0x7fa2e413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1539918194 >> v1:10.20.193.78:6789/0 conn(0x7fa2e406c890 legacy=0x7fa2e406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient: _finish_hunting 0 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 1 monclient: found mon.noname-a 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:24.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:08:24.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.416+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1539918194 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e409cf60 con 0x7fa2e406d380 2026-02-17T06:08:24.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1539918194 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e406d380 msgr2=0x7fa2e413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e88b1640 1 --2- 10.20.193.78:0/1539918194 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e406d380 0x7fa2e413ead0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4074de0 tx=0x7fa2d802f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2d5e5c9a68faa5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 1 -- 10.20.193.78:0/1539918194 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d803c070 con 0x7fa2e406d380 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: dump: 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.431 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 5 monclient: authenticate success, global_id 254286 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 1 -- 10.20.193.78:0/1539918194 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d8037640 con 0x7fa2e406d380 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 1 -- 10.20.193.78:0/1539918194 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8037a10 con 0x7fa2e406d380 2026-02-17T06:08:24.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2e2ffd640 10 monclient: dump: 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.433 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 4 set_mon_vals no callback set 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 10 monclient: shutdown 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1539918194 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e406d380 msgr2=0x7fa2e413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 1 --2- 10.20.193.78:0/1539918194 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e406d380 0x7fa2e413ead0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4074de0 tx=0x7fa2d802f130 comp rx=0 tx=0).stop 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1539918194 shutdown_connections 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 1 --2- 10.20.193.78:0/1539918194 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e406d380 0x7fa2e413ead0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1539918194 >> 10.20.193.78:0/1539918194 conn(0x7fa2e40a6620 msgr2=0x7fa2e40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1539918194 shutdown_connections 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.419+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1539918194 wait complete. 2026-02-17T06:08:24.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient: build_initial_monmap 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient: monmap: 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 1 Processor -- start 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 1 -- start start 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient: init 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa2e406c890 0x7fa2e413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient(hunting): picked mon.noname-c con 0x7fa2e406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa2e406d380 0x7fa2e4132190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient(hunting): picked mon.noname-b con 0x7fa2e406d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e409c4a0 0x7fa2e409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient(hunting): picked mon.noname-a con 0x7fa2e409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e409bfa0 con 0x7fa2e409c4a0 2026-02-17T06:08:24.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e409be20 con 0x7fa2e406d380 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e413f010 con 0x7fa2e406c890 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2ea33b640 10 monclient(hunting): authenticate will time out at 1949.974615s 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e409c4a0 0x7fa2e409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e409c4a0 0x7fa2e409db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:44806/0 (socket says 10.20.193.78:44806) 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1283619173 learned_addr learned my addr 10.20.193.78:0/1283619173 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient(hunting): get_auth_request con 0x7fa2e409c4a0 auth_method 0 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e3fff640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa2e406c890 0x7fa2e413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e3fff640 10 monclient(hunting): get_auth_request con 0x7fa2e406c890 auth_method 0 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient(hunting): handle_auth_done global_id 254289 payload 824 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e37fe640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa2e406d380 0x7fa2e4132190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa2e406c890 msgr2=0x7fa2e413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa2e406c890 0x7fa2e413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa2e406d380 msgr2=0x7fa2e4132190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa2e406d380 0x7fa2e4132190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient: _finish_hunting 0 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 monclient: found mon.noname-a 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1283619173 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e411d850 con 0x7fa2e409c4a0 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e37fe640 10 monclient: get_auth_request con 0x7fa2e406d380 auth_method 0 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.420+0000 7fa2e37fe640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa2e406d380 0x7fa2e4132190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:08:24.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.422+0000 7fa2e88b1640 1 -- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e409c4a0 msgr2=0x7fa2e409db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.422+0000 7fa2e88b1640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e409c4a0 0x7fa2e409db20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009e90 tx=0x7fa2d8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 1 -- 10.20.193.78:0/1283619173 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8047020 con 0x7fa2e409c4a0 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: dump: 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.440 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: _finish_auth 0 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 1 -- 10.20.193.78:0/1283619173 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d8042880 con 0x7fa2e409c4a0 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 5 monclient: authenticate success, global_id 254289 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 10 monclient: _renew_subs 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:24.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e409f4e0 con 0x7fa2e409c4a0 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 1 -- 10.20.193.78:0/1283619173 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d803c040 con 0x7fa2e409c4a0 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e27fc640 4 set_mon_vals no callback set 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 10 monclient: dump: 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.442 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 10 monclient: _renew_subs 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e411d570 con 0x7fa2e409c4a0 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2e0ff9640 1 -- 10.20.193.78:0/1283619173 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa2d8043420 con 0x7fa2e409c4a0 2026-02-17T06:08:24.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.423+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e406cc70 con 0x7fa2e409c4a0 2026-02-17T06:08:24.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.424+0000 7fa2e0ff9640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa2b403deb0 0x7fa2b4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.424+0000 7fa2e0ff9640 1 -- 10.20.193.78:0/1283619173 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(786..786 src has 525..786) ==== 4751+0+0 (secure 0 0 0) 0x7fa2d8078680 con 0x7fa2e409c4a0 2026-02-17T06:08:24.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.424+0000 7fa2e3fff640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa2b403deb0 0x7fa2b4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.424+0000 7fa2e3fff640 10 monclient: get_auth_request con 0x7fa2b403deb0 auth_method 0 2026-02-17T06:08:24.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.426+0000 7fa2e0ff9640 1 -- 10.20.193.78:0/1283619173 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa2d8051070 con 0x7fa2e409c4a0 2026-02-17T06:08:24.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.426+0000 7fa2e0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:24.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.426+0000 7fa2e0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:24.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.426+0000 7fa2e3fff640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa2b403deb0 0x7fa2b4040370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0007d90 tx=0x7fa2d0007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.542+0000 7fa2ea33b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:08:24.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.542+0000 7fa2ea33b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:24.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.542+0000 7fa2ea33b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:24.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.542+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa2e409f260 con 0x7fa2e409c4a0 2026-02-17T06:08:24.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.542+0000 7fa2e0ff9640 1 -- 10.20.193.78:0/1283619173 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa2d80770a0 con 0x7fa2e409c4a0 2026-02-17T06:08:24.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.542+0000 7fa2e0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:24.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.542+0000 7fa2e0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:24.543 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":252,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa2b403deb0 msgr2=0x7fa2b4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa2b403deb0 0x7fa2b4040370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0007d90 tx=0x7fa2d0007ce0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 10 monclient: shutdown 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e409c4a0 msgr2=0x7fa2e409db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e409c4a0 0x7fa2e409db20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009e90 tx=0x7fa2d8004b40 comp rx=0 tx=0).stop 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 shutdown_connections 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa2e409c4a0 0x7fa2e409db20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa2b403deb0 0x7fa2b4040370 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa2e406d380 0x7fa2e4132190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 --2- 10.20.193.78:0/1283619173 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa2e406c890 0x7fa2e413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 >> 10.20.193.78:0/1283619173 conn(0x7fa2e40a6620 msgr2=0x7fa2e40a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 shutdown_connections 2026-02-17T06:08:24.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.543+0000 7fa2ea33b640 1 -- 10.20.193.78:0/1283619173 wait complete. 2026-02-17T06:08:24.551 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:08:24.552 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:08:24.552 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:08:24.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient: get_monmap_and_config 2026-02-17T06:08:24.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient: build_initial_monmap 2026-02-17T06:08:24.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient: monmap: 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:24.620412+0000 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:24.620412+0000 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:24.623 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 1 Processor -- start 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 1 -- start start 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient: init 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa7e006c890 0x7fa7e006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient(hunting): picked mon.noname-c con 0x7fa7e006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa7e009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:24.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006d1b0 0x7fa7e0135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient(hunting): picked mon.noname-b con 0x7fa7e006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e006daf0 con 0x7fa7e009c4a0 2026-02-17T06:08:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e00a0880 con 0x7fa7e006c890 2026-02-17T06:08:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e009cc80 con 0x7fa7e006d1b0 2026-02-17T06:08:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:24.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.619+0000 7fa7e5187640 10 monclient(hunting): authenticate will time out at 1950.173977s 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006d1b0 0x7fa7e0135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006d1b0 0x7fa7e0135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58114/0 (socket says 10.20.193.78:58114) 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 1 -- 10.20.193.78:0/1372604035 learned_addr learned my addr 10.20.193.78:0/1372604035 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient(hunting): get_auth_request con 0x7fa7e006d1b0 auth_method 0 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7ded76640 1 --2- 10.20.193.78:0/1372604035 >> v2:10.20.193.115:3300/0 conn(0x7fa7e006c890 0x7fa7e006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7ded76640 10 monclient(hunting): get_auth_request con 0x7fa7e006c890 auth_method 0 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7ded76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7ded76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7ded76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient(hunting): handle_auth_done global_id 264197 payload 274 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 1 -- 10.20.193.78:0/1372604035 >> v2:10.20.193.115:3300/0 conn(0x7fa7e006c890 msgr2=0x7fa7e006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 1 --2- 10.20.193.78:0/1372604035 >> v2:10.20.193.115:3300/0 conn(0x7fa7e006c890 0x7fa7e006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 1 -- 10.20.193.78:0/1372604035 >> v1:10.20.193.115:6789/0 conn(0x7fa7e009c4a0 legacy=0x7fa7e0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient: _finish_hunting 0 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 1 monclient: found mon.noname-b 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.627 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.620+0000 7fa7df577640 1 -- 10.20.193.78:0/1372604035 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e009cf60 con 0x7fa7e006d1b0 2026-02-17T06:08:24.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7df577640 1 -- 10.20.193.78:0/1372604035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006d1b0 msgr2=0x7fa7e0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7df577640 1 --2- 10.20.193.78:0/1372604035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006d1b0 0x7fa7e0135500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa7e006dcc0 tx=0x7fa7d002f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f8c50844bf18e2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 1 -- 10.20.193.78:0/1372604035 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00373f0 con 0x7fa7e006d1b0 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: dump: 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.638 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: _finish_auth 0 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: _check_auth_tickets 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 1 -- 10.20.193.78:0/1372604035 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d0037590 con 0x7fa7e006d1b0 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7e5187640 5 monclient: authenticate success, global_id 264197 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 1 -- 10.20.193.78:0/1372604035 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0037960 con 0x7fa7e006d1b0 2026-02-17T06:08:24.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7e5187640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7e5187640 4 set_mon_vals no callback set 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.622+0000 7fa7ddd74640 10 monclient: dump: 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.640 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 10 monclient: shutdown 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 -- 10.20.193.78:0/1372604035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006d1b0 msgr2=0x7fa7e0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 --2- 10.20.193.78:0/1372604035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006d1b0 0x7fa7e0135500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa7e006dcc0 tx=0x7fa7d002f200 comp rx=0 tx=0).stop 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 -- 10.20.193.78:0/1372604035 shutdown_connections 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 --2- 10.20.193.78:0/1372604035 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006d1b0 0x7fa7e0135500 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 --2- 10.20.193.78:0/1372604035 >> v2:10.20.193.115:3300/0 conn(0x7fa7e006c890 0x7fa7e006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 -- 10.20.193.78:0/1372604035 >> 10.20.193.78:0/1372604035 conn(0x7fa7e00a6620 msgr2=0x7fa7e00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:24.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 -- 10.20.193.78:0/1372604035 shutdown_connections 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 -- 10.20.193.78:0/1372604035 wait complete. 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 10 monclient: build_initial_monmap 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 10 monclient: monmap: 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:24.642 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 Processor -- start 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 1 -- start start 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.623+0000 7fa7e5187640 10 monclient: init 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006c890 0x7fa7e01cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient(hunting): picked mon.noname-c con 0x7fa7e006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e006d1b0 0x7fa7e01cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient(hunting): picked mon.noname-a con 0x7fa7e006d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7e009c4a0 0x7fa7e0124cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient(hunting): picked mon.noname-b con 0x7fa7e009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e00a01a0 con 0x7fa7e006d1b0 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e00a0020 con 0x7fa7e009c4a0 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e009be20 con 0x7fa7e006c890 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7e5187640 10 monclient(hunting): authenticate will time out at 1950.178260s 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006c890 0x7fa7e01cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006c890 0x7fa7e01cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58124/0 (socket says 10.20.193.78:58124) 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 -- 10.20.193.78:0/1194697648 learned_addr learned my addr 10.20.193.78:0/1194697648 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient(hunting): get_auth_request con 0x7fa7e006c890 auth_method 0 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7de575640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e006d1b0 0x7fa7e01cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7de575640 10 monclient(hunting): get_auth_request con 0x7fa7e006d1b0 auth_method 0 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7de575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7de575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7de575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient(hunting): handle_auth_done global_id 264200 payload 824 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7e009c4a0 msgr2=0x7fa7e0124cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7df577640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7e009c4a0 0x7fa7e0124cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7e009c4a0 0x7fa7e0124cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e006d1b0 msgr2=0x7fa7e01cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e006d1b0 0x7fa7e01cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient: _finish_hunting 0 2026-02-17T06:08:24.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 monclient: found mon.noname-c 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7ded76640 1 -- 10.20.193.78:0/1194697648 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e0128600 con 0x7fa7e006c890 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7df577640 10 monclient: get_auth_request con 0x7fa7e009c4a0 auth_method 0 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.624+0000 7fa7df577640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa7e009c4a0 0x7fa7e0124cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7ded76640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006c890 msgr2=0x7fa7e01cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7ded76640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006c890 0x7fa7e01cf350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400e7c0 tx=0x7fa7d400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4017070 con 0x7fa7e006c890 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:24.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: dump: 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: _finish_auth 0 2026-02-17T06:08:24.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d400cae0 con 0x7fa7e006c890 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400f040 con 0x7fa7e006c890 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7dd573640 4 set_mon_vals no callback set 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7bf7fe640 10 monclient: dump: 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.649 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7e5187640 5 monclient: authenticate success, global_id 264200 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7e5187640 10 monclient: _renew_subs 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7e5187640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e01288e0 con 0x7fa7e006c890 2026-02-17T06:08:24.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7e5187640 10 monclient: _renew_subs 2026-02-17T06:08:24.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7e5187640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.626+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e006fd20 con 0x7fa7e006c890 2026-02-17T06:08:24.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7e5187640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:24.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7e5187640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e01251f0 0x7fa7e01255d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7e5187640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0074980 con 0x7fa7e01251f0 2026-02-17T06:08:24.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e01251f0 0x7fa7e01255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 10 monclient: get_auth_request con 0x7fa7e01251f0 auth_method 0 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa7d4012050 con 0x7fa7e006c890 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 10 monclient(hunting): handle_auth_done global_id 254295 payload 824 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7de575640 1 -- 10.20.193.78:0/1194697648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7e006cc70 con 0x7fa7e01251f0 2026-02-17T06:08:24.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7bf7fe640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7ac03df30 0x7fa7ac0403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.627+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(786..786 src has 525..786) ==== 4751+0+0 (secure 0 0 0) 0x7fa7d40525a0 con 0x7fa7e006c890 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.628+0000 7fa7df577640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7ac03df30 0x7fa7ac0403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.628+0000 7fa7df577640 10 monclient: get_auth_request con 0x7fa7ac03df30 auth_method 0 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.629+0000 7fa7de575640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e01251f0 msgr2=0x7fa7e01255d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.629+0000 7fa7de575640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e01251f0 0x7fa7e01255d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800f770 tx=0x7fa7c800fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.629+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8018030 con 0x7fa7e01251f0 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.629+0000 7fa7bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.629+0000 7fa7bf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.629+0000 7fa7bf7fe640 10 monclient: dump: 2026-02-17T06:08:24.653 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.630+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa7c800dbc0 con 0x7fa7e01251f0 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.630+0000 7fa7bf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.630+0000 7fa7bf7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:24.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.630+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e01251f0 msgr2=0x7fa7e01255d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.630+0000 7fa7bf7fe640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e01251f0 0x7fa7e01255d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800f770 tx=0x7fa7c800fc40 comp rx=0 tx=0).stop 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.630+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e01251f0 msgr2=0x7fa7e01255d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.630+0000 7fa7bf7fe640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e01251f0 0x7fa7e01255d0 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800f770 tx=0x7fa7c800fc40 comp rx=0 tx=0).stop 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.630+0000 7fa7df577640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7ac03df30 0x7fa7ac0403f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa7e009c980 tx=0x7fa7d0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7e5187640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7e5187640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e0125b10 0x7fa7e01da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7e5187640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e01271b0 con 0x7fa7e0125b10 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e0125b10 0x7fa7e01da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 10 monclient: get_auth_request con 0x7fa7e0125b10 auth_method 0 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 10 monclient(hunting): handle_auth_done global_id 254298 payload 824 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.633+0000 7fa7de575640 1 -- 10.20.193.78:0/1194697648 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa7e00744d0 con 0x7fa7e0125b10 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7de575640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e0125b10 msgr2=0x7fa7e01da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7de575640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e0125b10 0x7fa7e01da560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800f770 tx=0x7fa7c8003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8005260 con 0x7fa7e0125b10 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 10 monclient: dump: 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.656 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7fa7c8005260 con 0x7fa7e0125b10 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e0125b10 msgr2=0x7fa7e01da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e0125b10 0x7fa7e01da560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800f770 tx=0x7fa7c8003860 comp rx=0 tx=0).stop 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e0125b10 msgr2=0x7fa7e01da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:24.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.635+0000 7fa7bf7fe640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa7e0125b10 0x7fa7e01da560 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800f770 tx=0x7fa7c8003860 comp rx=0 tx=0).stop 2026-02-17T06:08:24.658 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:08:24.658 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:24.658 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:24.658 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:08:24.658 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 252, 2026-02-17T06:08:24.658 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 104009, 2026-02-17T06:08:24.658 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:08:24.658 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:24.659 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:24.660 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:24.661 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:24.662 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.663 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.665 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:08:24.666 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.667 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:08:24.668 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:08:24.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7ac03df30 msgr2=0x7fa7ac0403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa7ac03df30 0x7fa7ac0403f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa7e009c980 tx=0x7fa7d0033000 comp rx=0 tx=0).stop 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 10 monclient: shutdown 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006c890 msgr2=0x7fa7e01cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006c890 0x7fa7e01cf350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400e7c0 tx=0x7fa7d400ec90 comp rx=0 tx=0).stop 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7df577640 1 -- 10.20.193.78:0/1194697648 reap_dead start 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 shutdown_connections 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 --2- 10.20.193.78:0/1194697648 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa7e006c890 0x7fa7e01cf350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 >> 10.20.193.78:0/1194697648 conn(0x7fa7e00a6620 msgr2=0x7fa7e00a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 shutdown_connections 2026-02-17T06:08:24.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.636+0000 7fa7e5187640 1 -- 10.20.193.78:0/1194697648 wait complete. 2026-02-17T06:08:24.670 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:08:24.670 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:08:24.671 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:08:24.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient: get_monmap_and_config 2026-02-17T06:08:24.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient: build_initial_monmap 2026-02-17T06:08:24.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient: monmap: 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:24.739405+0000 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:24.739405+0000 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:24.741 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 1 Processor -- start 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 1 -- start start 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient: init 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient: _add_conns ranks=[1,2,0,3,4] 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f655c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c09c4a0 0x7f655c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient(hunting): picked mon.noname-b con 0x7f655c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f655c06d380 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c06daf0 con 0x7f655c06d380 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c09cc80 con 0x7f655c06c890 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0a0880 con 0x7f655c09c4a0 2026-02-17T06:08:24.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f6561fd3640 10 monclient(hunting): authenticate will time out at 1950.292913s 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.738+0000 7f655affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c09c4a0 0x7f655c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c09c4a0 0x7f655c13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58134/0 (socket says 10.20.193.78:58134) 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 1 -- 10.20.193.78:0/1648461626 learned_addr learned my addr 10.20.193.78:0/1648461626 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient(hunting): get_auth_request con 0x7f655c09c4a0 auth_method 0 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655bfff640 1 --1- 10.20.193.78:0/1648461626 >> v1:10.20.193.78:6789/0 conn(0x7f655c06d380 0x7f655c13d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55880/0 (socket says 10.20.193.78:55880) 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient(hunting): handle_auth_done global_id 264203 payload 274 2026-02-17T06:08:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 1 -- 10.20.193.78:0/1648461626 >> v1:10.20.193.115:6789/0 conn(0x7f655c06c890 legacy=0x7f655c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 1 -- 10.20.193.78:0/1648461626 >> v1:10.20.193.78:6789/0 conn(0x7f655c06d380 legacy=0x7f655c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient: _finish_hunting 0 2026-02-17T06:08:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 1 monclient: found mon.noname-b 2026-02-17T06:08:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.739+0000 7f655affd640 1 -- 10.20.193.78:0/1648461626 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c09cf60 con 0x7f655c09c4a0 2026-02-17T06:08:24.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655affd640 1 -- 10.20.193.78:0/1648461626 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c09c4a0 msgr2=0x7f655c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655affd640 1 --2- 10.20.193.78:0/1648461626 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c09c4a0 0x7f655c13b3a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f654c0099a0 tx=0x7f654c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a84383fb30105157 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 1 -- 10.20.193.78:0/1648461626 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c03c070 con 0x7f655c09c4a0 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: dump: 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.752 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 1 -- 10.20.193.78:0/1648461626 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c02fdf0 con 0x7f655c09c4a0 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f6561fd3640 5 monclient: authenticate success, global_id 264203 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f6561fd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 1 -- 10.20.193.78:0/1648461626 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c037660 con 0x7f655c09c4a0 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f6561fd3640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:24.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f6561fd3640 4 set_mon_vals no callback set 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.741+0000 7f655a7fc640 10 monclient: dump: 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 10 monclient: shutdown 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 -- 10.20.193.78:0/1648461626 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c09c4a0 msgr2=0x7f655c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 --2- 10.20.193.78:0/1648461626 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c09c4a0 0x7f655c13b3a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f654c0099a0 tx=0x7f654c02ee90 comp rx=0 tx=0).stop 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 -- 10.20.193.78:0/1648461626 shutdown_connections 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 --2- 10.20.193.78:0/1648461626 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c09c4a0 0x7f655c13b3a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 -- 10.20.193.78:0/1648461626 >> 10.20.193.78:0/1648461626 conn(0x7f655c0a6620 msgr2=0x7f655c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 -- 10.20.193.78:0/1648461626 shutdown_connections 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 -- 10.20.193.78:0/1648461626 wait complete. 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 10 monclient: build_initial_monmap 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 10 monclient: monmap: 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:24.755 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 Processor -- start 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 1 -- start start 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.742+0000 7f6561fd3640 10 monclient: init 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c06c890 0x7f655c132210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient(hunting): picked mon.noname-c con 0x7f655c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:24.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f655c06d380 0x7f655c132750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient(hunting): picked mon.noname-b con 0x7f655c06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f655c09c4a0 0x7f655c139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient(hunting): picked mon.noname-a con 0x7f655c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f655c09bfa0 con 0x7f655c09c4a0 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f655c09be20 con 0x7f655c06d380 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f655c13f010 con 0x7f655c06c890 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f6561fd3640 10 monclient(hunting): authenticate will time out at 1950.297120s 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c06c890 0x7f655c132210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.757 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f655c09c4a0 0x7f655c139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c06c890 0x7f655c132210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58144/0 (socket says 10.20.193.78:58144) 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 -- 10.20.193.78:0/1611989518 learned_addr learned my addr 10.20.193.78:0/1611989518 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient(hunting): get_auth_request con 0x7f655c06c890 auth_method 0 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655bfff640 10 monclient(hunting): get_auth_request con 0x7f655c09c4a0 auth_method 0 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.758 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655affd640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f655c06d380 0x7f655c132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655affd640 10 monclient(hunting): get_auth_request con 0x7f655c06d380 auth_method 0 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient(hunting): handle_auth_done global_id 264206 payload 824 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f655c06d380 msgr2=0x7f655c132750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f655c06d380 0x7f655c132750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f655c09c4a0 msgr2=0x7f655c139000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f655c09c4a0 0x7f655c139000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 monclient: found mon.noname-c 2026-02-17T06:08:24.759 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.743+0000 7f655b7fe640 1 -- 10.20.193.78:0/1611989518 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c09d810 con 0x7f655c06c890 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f655b7fe640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c06c890 msgr2=0x7f655c132210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f655b7fe640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c06c890 0x7f655c132210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f654800b5a0 tx=0x7f654800ba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548012070 con 0x7f655c06c890 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: dump: 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.760 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: _finish_auth 0 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: _check_auth_tickets 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6548002780 con 0x7f655c06c890 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6561fd3640 5 monclient: authenticate success, global_id 264206 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6561fd3640 10 monclient: _renew_subs 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6561fd3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c13aca0 con 0x7f655c06c890 2026-02-17T06:08:24.761 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548002b70 con 0x7f655c06c890 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6559ffb640 4 set_mon_vals no callback set 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6537fff640 10 monclient: dump: 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.762 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6561fd3640 10 monclient: _renew_subs 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6561fd3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.745+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c13b580 con 0x7f655c06c890 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f6561fd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f6561fd3640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139540 0x7f655c139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f6561fd3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f655c132c90 con 0x7f655c139540 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139540 0x7f655c139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f654800d460 con 0x7f655c06c890 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 10 monclient: get_auth_request con 0x7f655c139540 auth_method 0 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 10 monclient(hunting): handle_auth_done global_id 264209 payload 824 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f6537fff640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f652c03df00 0x7f652c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655affd640 1 -- 10.20.193.78:0/1611989518 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f655c06cc70 con 0x7f655c139540 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(786..786 src has 525..786) ==== 4751+0+0 (secure 0 0 0) 0x7f6548052420 con 0x7f655c06c890 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655bfff640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f652c03df00 0x7f652c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.746+0000 7f655bfff640 10 monclient: get_auth_request con 0x7f652c03df00 auth_method 0 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f655affd640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139540 msgr2=0x7f655c139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f655affd640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139540 0x7f655c139920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f654c036000 tx=0x7f654c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c048070 con 0x7f655c139540 2026-02-17T06:08:24.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 10 monclient: dump: 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f654c041640 con 0x7f655c139540 2026-02-17T06:08:24.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:24.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:24.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139540 msgr2=0x7f655c139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139540 0x7f655c139920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f654c036000 tx=0x7f654c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139540 msgr2=0x7f655c139920 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:24.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.748+0000 7f6537fff640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139540 0x7f655c139920 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f654c036000 tx=0x7f654c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.749+0000 7f655bfff640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f652c03df00 0x7f652c0403c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f655c13c7a0 tx=0x7f6550005380 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f6561fd3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f6561fd3640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139e60 0x7f655c1d8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f6561fd3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f655c13e350 con 0x7f655c139e60 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139e60 0x7f655c1d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 10 monclient: get_auth_request con 0x7f655c139e60 auth_method 0 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 10 monclient(hunting): handle_auth_done global_id 264212 payload 824 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.752+0000 7f655affd640 1 -- 10.20.193.78:0/1611989518 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f655c13c7d0 con 0x7f655c139e60 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f655affd640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139e60 msgr2=0x7f655c1d8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f655affd640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139e60 0x7f655c1d8230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f654c02eed0 tx=0x7f654c008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c03c040 con 0x7f655c139e60 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 10 monclient: dump: 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.769 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f654c003920 con 0x7f655c139e60 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139e60 msgr2=0x7f655c1d8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139e60 0x7f655c1d8230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f654c02eed0 tx=0x7f654c008030 comp rx=0 tx=0).stop 2026-02-17T06:08:24.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139e60 msgr2=0x7f655c1d8230 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.754+0000 7f6537fff640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c139e60 0x7f655c1d8230 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f654c02eed0 tx=0x7f654c008030 comp rx=0 tx=0).stop 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 252, 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 54139, 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:08:24.771 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:24.772 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:24.773 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:08:24.774 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.775 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.776 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.777 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.778 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:08:24.779 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:24.780 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:08:24.781 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:08:24.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f652c03df00 msgr2=0x7f652c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f652c03df00 0x7f652c0403c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f655c13c7a0 tx=0x7f6550005380 comp rx=0 tx=0).stop 2026-02-17T06:08:24.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 10 monclient: shutdown 2026-02-17T06:08:24.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:24.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c06c890 msgr2=0x7f655c132210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c06c890 0x7f655c132210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f654800b5a0 tx=0x7f654800ba70 comp rx=0 tx=0).stop 2026-02-17T06:08:24.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f655bfff640 1 -- 10.20.193.78:0/1611989518 reap_dead start 2026-02-17T06:08:24.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 shutdown_connections 2026-02-17T06:08:24.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 --2- 10.20.193.78:0/1611989518 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f655c06c890 0x7f655c132210 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 >> 10.20.193.78:0/1611989518 conn(0x7f655c0a6620 msgr2=0x7f655c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:24.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 shutdown_connections 2026-02-17T06:08:24.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.755+0000 7f6561fd3640 1 -- 10.20.193.78:0/1611989518 wait complete. 2026-02-17T06:08:24.783 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:08:24.783 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:08:24.783 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3586: done (4 left) 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3587: snap_create 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3582: finishing write tid 3 to trial11545420-29 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3582: finishing write tid 4 to trial11545420-29 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3582: oid 29 updating version 0 to 1488 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3582: oid 29 version 1488 is already newer than 1487 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3582: oid 29 updating version 1488 to 1489 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1489 (ObjNum 1230 snap 380 seq_num 1230) dirty exists 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3582: left oid 29 (ObjNum 1230 snap 380 seq_num 1230) 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3583: finishing write tid 1 to trial11545420-35 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3583: finishing write tid 2 to trial11545420-35 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3583: finishing write tid 3 to trial11545420-35 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3583: finishing write tid 4 to trial11545420-35 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3583: oid 35 updating version 0 to 1860 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3583: oid 35 updating version 1860 to 1861 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3583: oid 35 version 1861 is already newer than 1859 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1861 (ObjNum 1231 snap 380 seq_num 1231) dirty exists 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3583: left oid 35 (ObjNum 1231 snap 380 seq_num 1231) 2026-02-17T06:08:24.787 INFO:tasks.rados.rados.0.trial115.stdout:3584: finishing rollback tid 0 to trial11545420-15 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3584: finishing rollback tid 1 to trial11545420-15 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3584: finishing rollback tid 2 to trial11545420-15 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 1900 (ObjNum 1168 snap 359 seq_num 1168) dirty exists 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3585: finishing write tid 1 to trial11545420-22 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3585: finishing write tid 2 to trial11545420-22 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3585: finishing write tid 3 to trial11545420-22 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3585: finishing write tid 4 to trial11545420-22 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3585: oid 22 updating version 0 to 1486 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3585: oid 22 updating version 1486 to 1488 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3585: oid 22 version 1488 is already newer than 1487 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1488 (ObjNum 1232 snap 380 seq_num 1232) dirty exists 2026-02-17T06:08:24.788 INFO:tasks.rados.rados.0.trial115.stdout:3585: left oid 22 (ObjNum 1232 snap 380 seq_num 1232) 2026-02-17T06:08:24.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.850+0000 7fb00263b640 10 monclient: get_monmap_and_config 2026-02-17T06:08:24.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.850+0000 7fb00263b640 10 monclient: build_initial_monmap 2026-02-17T06:08:24.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.850+0000 7fb00263b640 10 monclient: monmap: 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:24.852169+0000 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:24.852169+0000 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:24.852 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 1 Processor -- start 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 1 -- start start 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient: init 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient: _add_conns ranks=[1,0,2,3,4] 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faffc06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faffc09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06d380 0x7faffc13ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient(hunting): picked mon.noname-b con 0x7faffc06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faffc06daf0 con 0x7faffc09c4a0 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faffc09cc80 con 0x7faffc06c890 2026-02-17T06:08:24.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faffc0a0880 con 0x7faffc06d380 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb00263b640 10 monclient(hunting): authenticate will time out at 1950.405708s 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06d380 0x7faffc13ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7faffb7fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7faffc09c4a0 0x7faffc13b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55882/0 (socket says 10.20.193.78:55882) 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7faffb7fe640 1 -- 10.20.193.78:0/867293978 learned_addr learned my addr 10.20.193.78:0/867293978 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient(hunting): get_auth_request con 0x7faffc06d380 auth_method 0 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient(hunting): handle_auth_done global_id 264215 payload 274 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 1 -- 10.20.193.78:0/867293978 >> v1:10.20.193.115:6789/0 conn(0x7faffc06c890 legacy=0x7faffc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:24.857 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 1 -- 10.20.193.78:0/867293978 >> v1:10.20.193.78:6789/0 conn(0x7faffc09c4a0 legacy=0x7faffc13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 10 monclient: _finish_hunting 0 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.851+0000 7fb000bb1640 1 monclient: found mon.noname-b 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.852+0000 7fb000bb1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.852+0000 7fb000bb1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.852+0000 7fb000bb1640 1 -- 10.20.193.78:0/867293978 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc09cf60 con 0x7faffc06d380 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb000bb1640 1 -- 10.20.193.78:0/867293978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06d380 msgr2=0x7faffc13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb000bb1640 1 --2- 10.20.193.78:0/867293978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06d380 0x7faffc13ead0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faff0009dc0 tx=0x7faff002f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0f4de0b1486d8a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 1 -- 10.20.193.78:0/867293978 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff003c070 con 0x7faffc06d380 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: dump: 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.858 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: _finish_auth 0 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 1 -- 10.20.193.78:0/867293978 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff0037640 con 0x7faffc06d380 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 5 monclient: authenticate success, global_id 264215 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 1 -- 10.20.193.78:0/867293978 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0037a10 con 0x7faffc06d380 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 4 set_mon_vals no callback set 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:24.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7faffaffd640 10 monclient: dump: 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 10 monclient: shutdown 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 1 -- 10.20.193.78:0/867293978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06d380 msgr2=0x7faffc13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 1 --2- 10.20.193.78:0/867293978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06d380 0x7faffc13ead0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faff0009dc0 tx=0x7faff002f1c0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 1 -- 10.20.193.78:0/867293978 shutdown_connections 2026-02-17T06:08:24.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 1 --2- 10.20.193.78:0/867293978 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06d380 0x7faffc13ead0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 1 -- 10.20.193.78:0/867293978 >> 10.20.193.78:0/867293978 conn(0x7faffc0a6620 msgr2=0x7faffc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 1 -- 10.20.193.78:0/867293978 shutdown_connections 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.854+0000 7fb00263b640 1 -- 10.20.193.78:0/867293978 wait complete. 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient: build_initial_monmap 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient: monmap: 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 1 Processor -- start 2026-02-17T06:08:24.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 1 -- start start 2026-02-17T06:08:24.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient: init 2026-02-17T06:08:24.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06c890 0x7faffc1ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient(hunting): picked mon.noname-c con 0x7faffc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faffc06d380 0x7faffc1cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient(hunting): picked mon.noname-a con 0x7faffc06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc09c4a0 0x7faffc1cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient(hunting): picked mon.noname-b con 0x7faffc09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7faffc09bfa0 con 0x7faffc06d380 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faffc09be20 con 0x7faffc09c4a0 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7faffc13f010 con 0x7faffc06c890 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:24.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb00263b640 10 monclient(hunting): authenticate will time out at 1950.409793s 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06c890 0x7faffc1ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06c890 0x7faffc1ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:58188/0 (socket says 10.20.193.78:58188) 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faffc06d380 0x7faffc1cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 1 -- 10.20.193.78:0/800675353 learned_addr learned my addr 10.20.193.78:0/800675353 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 10 monclient(hunting): get_auth_request con 0x7faffc06c890 auth_method 0 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffb7fe640 10 monclient(hunting): get_auth_request con 0x7faffc06d380 auth_method 0 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:24.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7faffbfff640 10 monclient(hunting): handle_auth_done global_id 264218 payload 824 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.855+0000 7fb000bb1640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc09c4a0 0x7faffc1cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc09c4a0 msgr2=0x7faffc1cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc09c4a0 0x7faffc1cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faffc06d380 msgr2=0x7faffc1cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7faffc06d380 0x7faffc1cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 10 monclient: _finish_hunting 0 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 1 monclient: found mon.noname-c 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7faffbfff640 1 -- 10.20.193.78:0/800675353 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc09d810 con 0x7faffc06c890 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7fb000bb1640 10 monclient: get_auth_request con 0x7faffc09c4a0 auth_method 0 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.856+0000 7fb000bb1640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc09c4a0 0x7faffc1cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faffbfff640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06c890 msgr2=0x7faffc1ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faffbfff640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06c890 0x7faffc1ccfc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fafe800c5d0 tx=0x7fafe800caa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8016020 con 0x7faffc06c890 2026-02-17T06:08:24.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: dump: 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: _finish_auth 0 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:24.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe8012a60 con 0x7faffc06c890 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 5 monclient: authenticate success, global_id 264218 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8012e50 con 0x7faffc06c890 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faffa7fc640 4 set_mon_vals no callback set 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7faff8ff9640 10 monclient: dump: 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:24.870 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 10 monclient: _renew_subs 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc11f980 con 0x7faffc06c890 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 10 monclient: _renew_subs 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc072960 con 0x7faffc06c890 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.858+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faffc06fdd0 con 0x7faffc136200 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fafe801b400 con 0x7faffc06c890 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 10 monclient: get_auth_request con 0x7faffc136200 auth_method 0 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:24.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faff8ff9640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fafcc03e050 0x7fafcc040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(787..787 src has 525..787) ==== 4795+0+0 (secure 0 0 0) 0x7fafe8052870 con 0x7faffc06c890 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7fb000bb1640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fafcc03e050 0x7fafcc040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7fb000bb1640 10 monclient: get_auth_request con 0x7fafcc03e050 auth_method 0 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.859+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:24.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:24.862+0000 7fb000bb1640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fafcc03e050 0x7fafcc040510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faffc09c980 tx=0x7faff0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:25.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.059+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:25.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.059+0000 7faffb7fe640 10 monclient: get_auth_request con 0x7faffc136200 auth_method 0 2026-02-17T06:08:25.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.059+0000 7faffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:25.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.059+0000 7faffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:25.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.059+0000 7faffb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:25.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.060+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:08:25.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.060+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:08:25.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.060+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:25.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.060+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:08:25.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:25.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 10 monclient: get_auth_request con 0x7faffc136200 auth_method 0 2026-02-17T06:08:25.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:25.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:25.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:25.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:08:25.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:08:25.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:25.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:25.460+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:08:25.647 INFO:tasks.rados.rados.0.trial115.stdout:3582: done (4 left) 2026-02-17T06:08:25.647 INFO:tasks.rados.rados.0.trial115.stdout:3583: done (3 left) 2026-02-17T06:08:25.648 INFO:tasks.rados.rados.0.trial115.stdout:3584: done (2 left) 2026-02-17T06:08:25.648 INFO:tasks.rados.rados.0.trial115.stdout:3585: done (1 left) 2026-02-17T06:08:25.648 INFO:tasks.rados.rados.0.trial115.stdout:3587: done (0 left) 2026-02-17T06:08:25.648 INFO:tasks.rados.rados.0.trial115.stdout:3588: write (excl) oid 27 current snap is 381 2026-02-17T06:08:25.648 INFO:tasks.rados.rados.0.trial115.stdout:3588: seq_num 1233 ranges {723031=431203,1818886=469759} 2026-02-17T06:08:25.651 INFO:tasks.rados.rados.0.trial115.stdout:3588: writing trial11545420-27 from 723031 to 1154234 tid 1 2026-02-17T06:08:25.655 INFO:tasks.rados.rados.0.trial115.stdout:3588: writing trial11545420-27 from 1818886 to 2288645 tid 2 2026-02-17T06:08:25.656 INFO:tasks.rados.rados.0.trial115.stdout:3589: delete oid 18 current snap is 381 2026-02-17T06:08:25.658 INFO:tasks.rados.rados.0.trial115.stdout:3588: finishing write tid 1 to trial11545420-27 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3588: finishing write tid 2 to trial11545420-27 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3588: finishing write tid 3 to trial11545420-27 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3588: finishing write tid 4 to trial11545420-27 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3588: oid 27 updating version 0 to 1490 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3588: oid 27 updating version 1490 to 1491 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3588: oid 27 version 1491 is already newer than 1489 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1491 (ObjNum 1233 snap 381 seq_num 1233) dirty exists 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3588: left oid 27 (ObjNum 1233 snap 381 seq_num 1233) 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3588: done (1 left) 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3589: done (0 left) 2026-02-17T06:08:25.659 INFO:tasks.rados.rados.0.trial115.stdout:3590: copy_from oid 16 from oid 14 current snap is 381 2026-02-17T06:08:25.660 INFO:tasks.rados.rados.0.trial115.stdout:3591: write oid 18 current snap is 381 2026-02-17T06:08:25.660 INFO:tasks.rados.rados.0.trial115.stdout:3591: seq_num 1235 ranges {756645=581701,1935850=473428} 2026-02-17T06:08:25.663 INFO:tasks.rados.rados.0.trial115.stdout:3591: writing trial11545420-18 from 756645 to 1338346 tid 1 2026-02-17T06:08:25.668 INFO:tasks.rados.rados.0.trial115.stdout:3591: writing trial11545420-18 from 1935850 to 2409278 tid 2 2026-02-17T06:08:25.668 INFO:tasks.rados.rados.0.trial115.stdout:3592: read oid 3 snap -1 2026-02-17T06:08:25.668 INFO:tasks.rados.rados.0.trial115.stdout:3592: expect (ObjNum 1215 snap 376 seq_num 1215) 2026-02-17T06:08:25.668 INFO:tasks.rados.rados.0.trial115.stdout:3593: copy_from oid 20 from oid 48 current snap is 381 2026-02-17T06:08:25.668 INFO:tasks.rados.rados.0.trial115.stdout:3590: finishing copy_from racing read to trial11545420-16 2026-02-17T06:08:25.669 INFO:tasks.rados.rados.0.trial115.stdout:3591: finishing write tid 1 to trial11545420-18 2026-02-17T06:08:25.669 INFO:tasks.rados.rados.0.trial115.stdout:3590: finishing copy_from to trial11545420-16 2026-02-17T06:08:25.669 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1675 (ObjNum 1218 snap 378 seq_num 1218) dirty exists 2026-02-17T06:08:25.669 INFO:tasks.rados.rados.0.trial115.stdout:3590: done (3 left) 2026-02-17T06:08:25.669 INFO:tasks.rados.rados.0.trial115.stdout:3594: write oid 45 current snap is 381 2026-02-17T06:08:25.669 INFO:tasks.rados.rados.0.trial115.stdout:3594: seq_num 1236 ranges {773452=656950,1877979=617086} 2026-02-17T06:08:25.673 INFO:tasks.rados.rados.0.trial115.stdout:3594: writing trial11545420-45 from 773452 to 1430402 tid 1 2026-02-17T06:08:25.677 INFO:tasks.rados.rados.0.trial115.stdout:3594: writing trial11545420-45 from 1877979 to 2495065 tid 2 2026-02-17T06:08:25.677 INFO:tasks.rados.rados.0.trial115.stdout:3595: delete oid 38 current snap is 381 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3593: finishing copy_from to trial11545420-20 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3593: got expected ENOENT (src dne) 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3593: finishing copy_from racing read to trial11545420-20 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3591: finishing write tid 2 to trial11545420-18 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3591: finishing write tid 3 to trial11545420-18 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3591: finishing write tid 4 to trial11545420-18 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3591: oid 18 updating version 0 to 1598 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3591: oid 18 version 1598 is already newer than 1596 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3591: oid 18 version 1598 is already newer than 1597 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1598 (ObjNum 1235 snap 381 seq_num 1235) dirty exists 2026-02-17T06:08:25.678 INFO:tasks.rados.rados.0.trial115.stdout:3591: left oid 18 (ObjNum 1235 snap 381 seq_num 1235) 2026-02-17T06:08:25.679 INFO:tasks.rados.rados.0.trial115.stdout:3592: expect (ObjNum 1215 snap 376 seq_num 1215) 2026-02-17T06:08:25.817 INFO:tasks.rados.rados.0.trial115.stdout:3594: finishing write tid 1 to trial11545420-45 2026-02-17T06:08:25.817 INFO:tasks.rados.rados.0.trial115.stdout:3594: finishing write tid 2 to trial11545420-45 2026-02-17T06:08:25.817 INFO:tasks.rados.rados.0.trial115.stdout:3591: done (4 left) 2026-02-17T06:08:25.817 INFO:tasks.rados.rados.0.trial115.stdout:3592: done (3 left) 2026-02-17T06:08:25.817 INFO:tasks.rados.rados.0.trial115.stdout:3593: done (2 left) 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3595: done (1 left) 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3596: rollback oid 11 current snap is 381 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 380 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3594: finishing write tid 3 to trial11545420-45 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3594: finishing write tid 4 to trial11545420-45 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3594: oid 45 updating version 0 to 1859 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3594: oid 45 updating version 1859 to 1860 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3594: oid 45 updating version 1860 to 1861 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1861 (ObjNum 1236 snap 381 seq_num 1236) dirty exists 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3594: left oid 45 (ObjNum 1236 snap 381 seq_num 1236) 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3594: done (1 left) 2026-02-17T06:08:25.818 INFO:tasks.rados.rados.0.trial115.stdout:3597: rollback oid 25 current snap is 381 2026-02-17T06:08:25.819 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 379 2026-02-17T06:08:25.819 INFO:tasks.rados.rados.0.trial115.stdout:3598: rollback oid 7 current snap is 381 2026-02-17T06:08:25.819 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 369 2026-02-17T06:08:25.819 INFO:tasks.rados.rados.0.trial115.stdout:3599: read oid 35 snap -1 2026-02-17T06:08:25.819 INFO:tasks.rados.rados.0.trial115.stdout:3599: expect (ObjNum 1231 snap 380 seq_num 1231) 2026-02-17T06:08:25.819 INFO:tasks.rados.rados.0.trial115.stdout:3600: snap_create 2026-02-17T06:08:25.819 INFO:tasks.rados.rados.0.trial115.stdout:3598: finishing rollback tid 1 to trial11545420-7 2026-02-17T06:08:25.819 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 0 (ObjNum 33 snap 0 seq_num 1330673496) dirty dne 2026-02-17T06:08:25.820 INFO:tasks.rados.rados.0.trial115.stdout:3596: finishing rollback tid 0 to trial11545420-11 2026-02-17T06:08:25.820 INFO:tasks.rados.rados.0.trial115.stdout:3596: finishing rollback tid 1 to trial11545420-11 2026-02-17T06:08:25.820 INFO:tasks.rados.rados.0.trial115.stdout:3596: finishing rollback tid 2 to trial11545420-11 2026-02-17T06:08:25.820 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1910 (ObjNum 1163 snap 357 seq_num 1163) dirty exists 2026-02-17T06:08:25.820 INFO:tasks.rados.rados.0.trial115.stdout:3597: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:08:25.821 INFO:tasks.rados.rados.0.trial115.stdout:3597: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:08:25.821 INFO:tasks.rados.rados.0.trial115.stdout:3597: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:08:25.821 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1601 (ObjNum 1204 snap 370 seq_num 1204) dirty exists 2026-02-17T06:08:25.822 INFO:tasks.rados.rados.0.trial115.stdout:3599: expect (ObjNum 1231 snap 380 seq_num 1231) 2026-02-17T06:08:26.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:26.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 10 monclient: get_auth_request con 0x7faffc136200 auth_method 0 2026-02-17T06:08:26.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:26.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:26.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:26.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:08:26.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:08:26.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:26.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:26.262+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:08:27.347 INFO:tasks.rados.rados.0.trial115.stdout:3596: done (4 left) 2026-02-17T06:08:27.348 INFO:tasks.rados.rados.0.trial115.stdout:3597: done (3 left) 2026-02-17T06:08:27.348 INFO:tasks.rados.rados.0.trial115.stdout:3598: done (2 left) 2026-02-17T06:08:27.348 INFO:tasks.rados.rados.0.trial115.stdout:3599: done (1 left) 2026-02-17T06:08:27.348 INFO:tasks.rados.rados.0.trial115.stdout:3600: done (0 left) 2026-02-17T06:08:27.348 INFO:tasks.rados.rados.0.trial115.stdout:3601: snap_create 2026-02-17T06:08:27.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.856+0000 7fafd77fe640 10 monclient: tick 2026-02-17T06:08:27.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.856+0000 7fafd77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:08:27.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.856+0000 7fafd77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:27.856 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.856+0000 7fafd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:27.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:27.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 10 monclient: get_auth_request con 0x7faffc136200 auth_method 0 2026-02-17T06:08:27.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:27.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:08:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:08:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:27.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:27.864+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2026-02-17T06:08:28.354 INFO:tasks.rados.rados.0.trial115.stdout:3601: done (0 left) 2026-02-17T06:08:28.354 INFO:tasks.rados.rados.0.trial115.stdout:3602: snap_remove snap 378 2026-02-17T06:08:28.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 10 monclient: tick 2026-02-17T06:08:28.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:08:28.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:28.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 5 monclient: _check_tell_commands timeout tell command 1 2026-02-17T06:08:28.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:08:28.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafc8003640 0x7fafc8005ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:28.858 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 msgr2=0x7faffc139390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc136200 0x7faffc139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.855+0000 7fafd77fe640 1 -- 10.20.193.78:0/800675353 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faffc06fdd0 con 0x7fafc8003640 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafc8003640 0x7fafc8005ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 10 monclient: get_auth_request con 0x7fafc8003640 auth_method 0 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:28.859 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:28.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:28.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 10 monclient(hunting): handle_auth_done global_id 274099 payload 824 2026-02-17T06:08:28.860 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.856+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafc80061b0 con 0x7fafc8003640 2026-02-17T06:08:28.861 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafc8003640 msgr2=0x7fafc8005ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafc8003640 0x7fafc8005ad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faffc13efe0 tx=0x7fafec018f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec019450 con 0x7fafc8003640 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 10 monclient: dump: 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:28.862 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:28.863 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:28.863 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:28.863 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:28.863 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:28.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fafec019450 con 0x7fafc8003640 2026-02-17T06:08:28.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:28.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:28.863 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafc8003640 msgr2=0x7fafc8005ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:28.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafc8003640 0x7fafc8005ad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faffc13efe0 tx=0x7fafec018f60 comp rx=0 tx=0).stop 2026-02-17T06:08:28.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafc8003640 msgr2=0x7fafc8005ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:28.864 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.859+0000 7faff8ff9640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fafc8003640 0x7fafc8005ad0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.865 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7fb00263b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7fb00263b640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc1365e0 0x7faffc1d8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7fb00263b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7faffc13ecb0 con 0x7faffc1365e0 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc1365e0 0x7faffc1d8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 10 monclient: get_auth_request con 0x7faffc1365e0 auth_method 0 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:28.866 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:28.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 10 monclient(hunting): handle_auth_done global_id 274102 payload 824 2026-02-17T06:08:28.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.863+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faffc13aa50 con 0x7faffc1365e0 2026-02-17T06:08:28.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faffb7fe640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc1365e0 msgr2=0x7faffc1d8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:28.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faffb7fe640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc1365e0 0x7faffc1d8520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafec022000 tx=0x7fafec004b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:28.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec028cb0 con 0x7faffc1365e0 2026-02-17T06:08:28.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:28.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:08:28.868 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 10 monclient: dump: 2026-02-17T06:08:28.868 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7fafec019e10 con 0x7faffc1365e0 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:08:28.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:28.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc1365e0 msgr2=0x7faffc1d8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:28.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc1365e0 0x7faffc1d8520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafec022000 tx=0x7fafec004b90 comp rx=0 tx=0).stop 2026-02-17T06:08:28.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc1365e0 msgr2=0x7faffc1d8520 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:28.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7faff8ff9640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7faffc1365e0 0x7faffc1d8520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb000bb1640 1 -- 10.20.193.78:0/800675353 reap_dead start 2026-02-17T06:08:28.870 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:08:28.870 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 252, 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 10868, 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 5, 2026-02-17T06:08:28.871 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:28.872 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:28.873 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:28.874 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:28.874 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:28.874 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:28.874 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:28.874 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:28.874 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.875 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:08:28.875 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:08:28.875 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:08:28.875 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:08:28.875 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:08:28.875 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:08:28.875 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:28.875 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:28.876 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.877 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:28.878 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.879 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:28.880 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:28.880 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:28.880 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:28.880 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.880 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:28.880 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:08:28.880 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:28.880 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.881 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.881 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:28.881 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:08:28.881 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:28.881 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:28.881 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:28.881 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.882 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:08:28.882 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:08:28.882 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:28.882 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:28.882 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:28.882 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.883 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.883 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:28.883 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:28.883 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:28.883 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:28.883 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.883 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:28.884 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:08:28.884 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:28.884 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.884 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.884 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:28.884 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:08:28.884 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:28.885 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:08:28.886 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:28.887 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:28.887 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:28.887 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:28.887 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:28.887 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:28.887 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:28.887 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:08:28.887 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fafcc03e050 msgr2=0x7fafcc040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fafcc03e050 0x7fafcc040510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faffc09c980 tx=0x7faff0033000 comp rx=0 tx=0).stop 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 10 monclient: shutdown 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06c890 msgr2=0x7faffc1ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06c890 0x7faffc1ccfc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fafe800c5d0 tx=0x7fafe800caa0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 shutdown_connections 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fafcc03e050 0x7fafcc040510 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 --2- 10.20.193.78:0/800675353 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7faffc06c890 0x7faffc1ccfc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 >> 10.20.193.78:0/800675353 conn(0x7faffc0a6620 msgr2=0x7faffc0715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:28.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 shutdown_connections 2026-02-17T06:08:28.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.866+0000 7fb00263b640 1 -- 10.20.193.78:0/800675353 wait complete. 2026-02-17T06:08:28.890 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:08:28.890 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:08:28.890 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:08:28.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.958+0000 7fd310b22640 10 monclient: get_monmap_and_config 2026-02-17T06:08:28.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.958+0000 7fd310b22640 10 monclient: build_initial_monmap 2026-02-17T06:08:28.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.958+0000 7fd310b22640 10 monclient: monmap: 2026-02-17T06:08:28.959 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:28.959 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:28.959 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:28.959556+0000 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:28.959556+0000 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:28.960 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:28.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.958+0000 7fd310b22640 1 Processor -- start 2026-02-17T06:08:28.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.958+0000 7fd310b22640 1 -- start start 2026-02-17T06:08:28.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.958+0000 7fd310b22640 10 monclient: init 2026-02-17T06:08:28.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:28.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2026-02-17T06:08:28.972 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 0x7fd30c0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient(hunting): picked mon.noname-c con 0x7fd30c09ea90 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd30c09ee70 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd30c13d640 0x7fd30c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient(hunting): picked mon.noname-a con 0x7fd30c13d640 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd30c0a0a20 con 0x7fd30c09ee70 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd30c0a0d00 con 0x7fd30c13d640 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd30c0a0460 con 0x7fd30c09ea90 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd310b22640 10 monclient(hunting): authenticate will time out at 1954.513293s 2026-02-17T06:08:28.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd30c13d640 0x7fd30c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd309d74640 1 --1- >> v1:10.20.193.115:6789/0 conn(0x7fd30c09ee70 0x7fd30c09f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.115:6789/0 says I am v1:10.20.193.78:41974/0 (socket says 10.20.193.78:41974) 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd30c13d640 0x7fd30c13fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47750/0 (socket says 10.20.193.78:47750) 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd309d74640 1 -- 10.20.193.78:0/4000098695 learned_addr learned my addr 10.20.193.78:0/4000098695 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient(hunting): get_auth_request con 0x7fd30c13d640 auth_method 0 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:28.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30a575640 1 --2- 10.20.193.78:0/4000098695 >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 0x7fd30c0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30a575640 10 monclient(hunting): get_auth_request con 0x7fd30c09ea90 auth_method 0 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient(hunting): handle_auth_done global_id 254307 payload 274 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 1 -- 10.20.193.78:0/4000098695 >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 msgr2=0x7fd30c0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 1 --2- 10.20.193.78:0/4000098695 >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 0x7fd30c0a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 1 -- 10.20.193.78:0/4000098695 >> v1:10.20.193.115:6789/0 conn(0x7fd30c09ee70 legacy=0x7fd30c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 1 monclient: found mon.noname-a 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:28.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.959+0000 7fd30ad76640 1 -- 10.20.193.78:0/4000098695 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c0a0740 con 0x7fd30c13d640 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd30ad76640 1 -- 10.20.193.78:0/4000098695 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 msgr2=0x7fd30c13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd30ad76640 1 --2- 10.20.193.78:0/4000098695 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c13fa30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06ecd0 tx=0x7fd2fc02f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b93a1eb6489a4b2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 1 -- 10.20.193.78:0/4000098695 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc03c070 con 0x7fd30c13d640 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: dump: 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:28.976 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: _finish_auth 0 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: _check_auth_tickets 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 5 monclient: authenticate success, global_id 254307 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 1 -- 10.20.193.78:0/4000098695 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2fc037440 con 0x7fd30c13d640 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:28.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 1 -- 10.20.193.78:0/4000098695 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc037810 con 0x7fd30c13d640 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd309573640 10 monclient: dump: 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:28.978 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 4 set_mon_vals no callback set 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 10 monclient: shutdown 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 1 -- 10.20.193.78:0/4000098695 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 msgr2=0x7fd30c13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 1 --2- 10.20.193.78:0/4000098695 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c13fa30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06ecd0 tx=0x7fd2fc02f130 comp rx=0 tx=0).stop 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 1 -- 10.20.193.78:0/4000098695 shutdown_connections 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 1 --2- 10.20.193.78:0/4000098695 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c13fa30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 1 --2- 10.20.193.78:0/4000098695 >> v2:10.20.193.115:3300/0 conn(0x7fd30c09ea90 0x7fd30c0a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 1 -- 10.20.193.78:0/4000098695 >> 10.20.193.78:0/4000098695 conn(0x7fd30c0a7520 msgr2=0x7fd30c0a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 1 -- 10.20.193.78:0/4000098695 shutdown_connections 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 1 -- 10.20.193.78:0/4000098695 wait complete. 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 10 monclient: build_initial_monmap 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.962+0000 7fd310b22640 10 monclient: monmap: 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:28.980 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 1 Processor -- start 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 1 -- start start 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient: init 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ea90 0x7fd30c0a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient(hunting): picked mon.noname-c con 0x7fd30c09ea90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c09ee70 0x7fd30c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient(hunting): picked mon.noname-b con 0x7fd30c09ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c0a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient(hunting): picked mon.noname-a con 0x7fd30c13d640 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:28.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c0a6820 con 0x7fd30c13d640 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c0a66a0 con 0x7fd30c09ee70 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd30c13ff70 con 0x7fd30c09ea90 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd310b22640 10 monclient(hunting): authenticate will time out at 1954.517451s 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c0a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ea90 0x7fd30c0a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c0a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47758/0 (socket says 10.20.193.78:47758) 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 -- 10.20.193.78:0/2486188623 learned_addr learned my addr 10.20.193.78:0/2486188623 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:28.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient(hunting): get_auth_request con 0x7fd30c13d640 auth_method 0 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30a575640 10 monclient(hunting): get_auth_request con 0x7fd30c09ea90 auth_method 0 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:28.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd309d74640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c09ee70 0x7fd30c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd309d74640 10 monclient(hunting): get_auth_request con 0x7fd30c09ee70 auth_method 0 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd309d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd309d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd309d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient(hunting): handle_auth_done global_id 254310 payload 824 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 -- 10.20.193.78:0/2486188623 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ea90 msgr2=0x7fd30c0a32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ea90 0x7fd30c0a32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 -- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c09ee70 msgr2=0x7fd30c0a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c09ee70 0x7fd30c0a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:08:28.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 monclient: found mon.noname-a 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.963+0000 7fd30ad76640 1 -- 10.20.193.78:0/2486188623 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c06f1b0 con 0x7fd30c13d640 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd30ad76640 1 -- 10.20.193.78:0/2486188623 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 msgr2=0x7fd30c0a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd30ad76640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c0a3d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc03a040 tx=0x7fd2fc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 1 -- 10.20.193.78:0/2486188623 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc03c050 con 0x7fd30c13d640 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: dump: 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:28.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 1 -- 10.20.193.78:0/2486188623 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2fc048070 con 0x7fd30c13d640 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd310b22640 5 monclient: authenticate success, global_id 254310 2026-02-17T06:08:28.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 1 -- 10.20.193.78:0/2486188623 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc041c60 con 0x7fd30c13d640 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd308d72640 4 set_mon_vals no callback set 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd2f2ffd640 10 monclient: dump: 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:28.987 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd310b22640 10 monclient: _renew_subs 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd310b22640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.965+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30c0a9c10 con 0x7fd30c13d640 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd310b22640 10 monclient: _renew_subs 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd310b22640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30c070a40 con 0x7fd30c13d640 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd310b22640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd310b22640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd310b22640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd30c070840 con 0x7fd30c13d640 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd2f2ffd640 1 -- 10.20.193.78:0/2486188623 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd2fc04d460 con 0x7fd30c13d640 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd2f2ffd640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2d803deb0 0x7fd2d8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.966+0000 7fd2f2ffd640 1 -- 10.20.193.78:0/2486188623 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(790..790 src has 525..790) ==== 4735+0+0 (secure 0 0 0) 0x7fd2fc078850 con 0x7fd30c13d640 2026-02-17T06:08:28.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.967+0000 7fd30a575640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2d803deb0 0x7fd2d8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:28.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.967+0000 7fd30a575640 10 monclient: get_auth_request con 0x7fd2d803deb0 auth_method 0 2026-02-17T06:08:28.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.969+0000 7fd2f2ffd640 1 -- 10.20.193.78:0/2486188623 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd2fc035d80 con 0x7fd30c13d640 2026-02-17T06:08:28.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.969+0000 7fd2f2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:28.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.969+0000 7fd2f2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:28.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:28.969+0000 7fd30a575640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2d803deb0 0x7fd2d8040370 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd300009a50 tx=0x7fd300008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:29.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.083+0000 7fd310b22640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:08:29.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.083+0000 7fd310b22640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:08:29.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.083+0000 7fd310b22640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:29.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.083+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd30c0a13d0 con 0x7fd30c13d640 2026-02-17T06:08:29.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.083+0000 7fd2f2ffd640 1 -- 10.20.193.78:0/2486188623 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd2fc04d780 con 0x7fd30c13d640 2026-02-17T06:08:29.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.083+0000 7fd2f2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:08:29.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.083+0000 7fd2f2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:29.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2d803deb0 msgr2=0x7fd2d8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:29.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2d803deb0 0x7fd2d8040370 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd300009a50 tx=0x7fd300008040 comp rx=0 tx=0).stop 2026-02-17T06:08:29.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 10 monclient: shutdown 2026-02-17T06:08:29.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:29.086 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 msgr2=0x7fd30c0a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c0a3d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc03a040 tx=0x7fd2fc004b40 comp rx=0 tx=0).stop 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 shutdown_connections 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd30c13d640 0x7fd30c0a3d30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd2d803deb0 0x7fd2d8040370 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd30c09ee70 0x7fd30c0a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 --2- 10.20.193.78:0/2486188623 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd30c09ea90 0x7fd30c0a32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 >> 10.20.193.78:0/2486188623 conn(0x7fd30c0a7520 msgr2=0x7fd30c13c860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 shutdown_connections 2026-02-17T06:08:29.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:29.084+0000 7fd310b22640 1 -- 10.20.193.78:0/2486188623 wait complete. 2026-02-17T06:08:29.088 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.085+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.088 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.085+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=10,kv=11,logm=50} crc {auth=857428664,config=692619927,health=838888936,kv=449989585,logm=649326231}) 2026-02-17T06:08:29.088 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.085+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=11,kv=12,logm=48} crc {auth=3784777907,config=692619927,health=2774463547,kv=449989585,logm=2402883424}) 2026-02-17T06:08:29.088 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.085+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.088 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.085+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=10,kv=11,logm=50} crc {auth=857428664,config=692619927,health=838888936,kv=449989585,logm=649326231}) 2026-02-17T06:08:29.088 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.085+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=26,config=2,health=12,kv=11,logm=49} crc {auth=3034788103,config=692619927,health=2443276415,kv=2589905713,logm=2181451140}) 2026-02-17T06:08:29.093 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:08:29.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=881464110}) 2026-02-17T06:08:29.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3541290551}) 2026-02-17T06:08:29.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=881464110}) 2026-02-17T06:08:29.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2870435782}) 2026-02-17T06:08:29.352 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=117561501}) 2026-02-17T06:08:29.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=699684819}) 2026-02-17T06:08:29.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=117561501}) 2026-02-17T06:08:29.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4255068315}) 2026-02-17T06:08:29.354 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.352+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.354 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.352+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=25107498}) 2026-02-17T06:08:29.354 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.352+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=957343385}) 2026-02-17T06:08:29.354 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.352+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.354 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.352+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=25107498}) 2026-02-17T06:08:29.354 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.352+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1099820840}) 2026-02-17T06:08:29.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.355+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.356 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.355+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494704645}) 2026-02-17T06:08:29.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.355+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1620365535}) 2026-02-17T06:08:29.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.355+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.355+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3494704645}) 2026-02-17T06:08:29.357 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.355+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1158071257}) 2026-02-17T06:08:29.357 INFO:tasks.rados.rados.0.trial115.stdout:3602: done (0 left) 2026-02-17T06:08:29.357 INFO:tasks.rados.rados.0.trial115.stdout:3603: snap_remove snap 369 2026-02-17T06:08:29.358 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3953467531}) 2026-02-17T06:08:29.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3241259643}) 2026-02-17T06:08:29.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:29.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3953467531}) 2026-02-17T06:08:29.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:29.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2629916616}) 2026-02-17T06:08:30.038 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.038 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3500701045}) 2026-02-17T06:08:30.038 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2459581690}) 2026-02-17T06:08:30.038 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.038 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3500701045}) 2026-02-17T06:08:30.038 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1656715447}) 2026-02-17T06:08:30.038 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=855619039}) 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=299171769}) 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=855619039}) 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.037+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3052830568}) 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4141765822}) 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2395254935}) 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4141765822}) 2026-02-17T06:08:30.039 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3173299483}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4033144588}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321401549}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4033144588}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.038+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=257034601}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2267035287}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2829772193}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2267035287}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258383127}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2835870276}) 2026-02-17T06:08:30.040 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2429083902}) 2026-02-17T06:08:30.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2835870276}) 2026-02-17T06:08:30.041 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=19847403}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.039+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2664861713}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2452847636}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2664861713}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=423550736}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=840523851}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1964102659}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=840523851}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1737843672}) 2026-02-17T06:08:30.042 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284849965}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4010690103}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284849965}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.040+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2380157677}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2085173064}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072211102}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2085173064}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1568279305}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=683280906}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3966585878}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=683280906}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=997486519}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4176598465}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1959166486}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4176598465}) 2026-02-17T06:08:30.043 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.041+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2153115481}) 2026-02-17T06:08:30.044 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.044 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=210191912}) 2026-02-17T06:08:30.044 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=179991205}) 2026-02-17T06:08:30.044 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.044 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=210191912}) 2026-02-17T06:08:30.044 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2923916927}) 2026-02-17T06:08:30.044 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4237306266}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1493720264}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4237306266}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1300030643}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297176014}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1245967212}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297176014}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.042+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=12742161}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2865671615}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=310652799}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2865671615}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555773755}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=920213075}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1758667207}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=920213075}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.043+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536609856}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3910307415}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3057421691}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3910307415}) 2026-02-17T06:08:30.045 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1139409876}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649673445}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=447929556}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649673445}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2857362086}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1733856950}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=609247856}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1733856950}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.044+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4029480739}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2388672860}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3280226517}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2388672860}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2605473456}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=146752342}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3610814024}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=146752342}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2918475869}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=993546889}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=388129746}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=993546889}) 2026-02-17T06:08:30.046 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.045+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=907093412}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2874550424}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2308287640}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2874550424}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2549364137}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1782884447}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1236554453}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1782884447}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1532413473}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1992189721}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1596239963}) 2026-02-17T06:08:30.047 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1992189721}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.046+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1542487720}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777249679}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59313772}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777249679}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3615290441}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=752585351}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1745889900}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=752585351}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3001098341}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2946805512}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3136862429}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2946805512}) 2026-02-17T06:08:30.048 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.047+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1758296718}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1598532806}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4054523493}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1598532806}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4790450}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146491627}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=408998236}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146491627}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=274264510}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=776147462}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1542521664}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=776147462}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4025566959}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.048+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1264700893}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2252783990}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1264700893}) 2026-02-17T06:08:30.049 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2326541124}) 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162480868}) 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2997604926}) 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162480868}) 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2557475479}) 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827261031}) 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619020157}) 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827261031}) 2026-02-17T06:08:30.050 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.049+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4277775209}) 2026-02-17T06:08:30.051 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.050+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.051 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.050+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383235224}) 2026-02-17T06:08:30.051 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.050+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4063481236}) 2026-02-17T06:08:30.051 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.050+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.051 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.050+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383235224}) 2026-02-17T06:08:30.051 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.050+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=97258397}) 2026-02-17T06:08:30.094 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:08:30.094 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:08:30.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.162+0000 7f5455ec9640 10 monclient: get_monmap_and_config 2026-02-17T06:08:30.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.162+0000 7f5455ec9640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.162+0000 7f5455ec9640 10 monclient: monmap: 2026-02-17T06:08:30.163 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:30.163833+0000 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:30.163833+0000 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:30.164 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.162+0000 7f5455ec9640 1 Processor -- start 2026-02-17T06:08:30.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 1 -- start start 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient: init 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient: _add_conns ranks=[1,3,2,4,0] 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f545006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f545009d170 0x7f545013daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient(hunting): picked mon.noname-a con 0x7f545009d170 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545009b200 0x7f5450140040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient(hunting): picked mon.noname-b con 0x7f545009b200 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f545009bf60 con 0x7f545006c890 2026-02-17T06:08:30.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f545009c520 con 0x7f545009d170 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f545009c800 con 0x7f545009b200 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f5455ec9640 10 monclient(hunting): authenticate will time out at 1955.717339s 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f545009d170 0x7f545013daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545009b200 0x7f5450140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f545009d170 0x7f545013daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47774/0 (socket says 10.20.193.78:47774) 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 1 -- 10.20.193.78:0/2623626192 learned_addr learned my addr 10.20.193.78:0/2623626192 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient(hunting): get_auth_request con 0x7f545009d170 auth_method 0 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544ffff640 10 monclient(hunting): get_auth_request con 0x7f545009b200 auth_method 0 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient(hunting): handle_auth_done global_id 254313 payload 274 2026-02-17T06:08:30.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 1 -- 10.20.193.78:0/2623626192 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545009b200 msgr2=0x7f5450140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 1 --2- 10.20.193.78:0/2623626192 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545009b200 0x7f5450140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 1 -- 10.20.193.78:0/2623626192 >> v1:10.20.193.115:6789/0 conn(0x7f545006c890 legacy=0x7f545006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 1 monclient: found mon.noname-a 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.163+0000 7f544effd640 1 -- 10.20.193.78:0/2623626192 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545009c240 con 0x7f545009d170 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.165+0000 7f544effd640 1 -- 10.20.193.78:0/2623626192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 msgr2=0x7f545013daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544effd640 1 --2- 10.20.193.78:0/2623626192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f545013daf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5440009970 tx=0x7f544002ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226d90cb84c4a3f0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 1 -- 10.20.193.78:0/2623626192 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544003c070 con 0x7f545009d170 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: dump: 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 1 -- 10.20.193.78:0/2623626192 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544002fcd0 con 0x7f545009d170 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 1 -- 10.20.193.78:0/2623626192 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440037660 con 0x7f545009d170 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 5 monclient: authenticate success, global_id 254313 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 4 set_mon_vals no callback set 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f544e7fc640 10 monclient: dump: 2026-02-17T06:08:30.179 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 10 monclient: shutdown 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 -- 10.20.193.78:0/2623626192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 msgr2=0x7f545013daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 --2- 10.20.193.78:0/2623626192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f545013daf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5440009970 tx=0x7f544002ee90 comp rx=0 tx=0).stop 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 -- 10.20.193.78:0/2623626192 shutdown_connections 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 --2- 10.20.193.78:0/2623626192 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545009b200 0x7f5450140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 --2- 10.20.193.78:0/2623626192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f545013daf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 -- 10.20.193.78:0/2623626192 >> 10.20.193.78:0/2623626192 conn(0x7f54500a4230 msgr2=0x7f54500a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 -- 10.20.193.78:0/2623626192 shutdown_connections 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 -- 10.20.193.78:0/2623626192 wait complete. 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 10 monclient: monmap: 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:30.181 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 Processor -- start 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 1 -- start start 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.166+0000 7f5455ec9640 10 monclient: init 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545006c890 0x7f54500946f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient(hunting): picked mon.noname-c con 0x7f545006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f545009b200 0x7f5450094c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient(hunting): picked mon.noname-b con 0x7f545009b200 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f5450095170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient(hunting): picked mon.noname-a con 0x7f545009d170 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f545006d500 con 0x7f545009d170 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f545006d380 con 0x7f545009b200 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f545006d680 con 0x7f545006c890 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f5455ec9640 10 monclient(hunting): authenticate will time out at 1955.721250s 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545006c890 0x7f54500946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f5450095170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f5450095170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47788/0 (socket says 10.20.193.78:47788) 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545006c890 0x7f54500946f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38208/0 (socket says 10.20.193.78:38208) 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 -- 10.20.193.78:0/1699536337 learned_addr learned my addr 10.20.193.78:0/1699536337 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient(hunting): get_auth_request con 0x7f545009d170 auth_method 0 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 10 monclient(hunting): get_auth_request con 0x7f545006c890 auth_method 0 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient(hunting): handle_auth_done global_id 254316 payload 824 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544effd640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f545009b200 0x7f5450094c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 -- 10.20.193.78:0/1699536337 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545006c890 msgr2=0x7f54500946f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545006c890 0x7f54500946f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 -- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f545009b200 msgr2=0x7f5450094c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f545009b200 0x7f5450094c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 monclient: found mon.noname-a 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544ffff640 1 -- 10.20.193.78:0/1699536337 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54500a0d90 con 0x7f545009d170 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544effd640 10 monclient: get_auth_request con 0x7f545009b200 auth_method 0 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.167+0000 7f544effd640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f545009b200 0x7f5450094c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f544ffff640 1 -- 10.20.193.78:0/1699536337 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 msgr2=0x7f5450095170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f544ffff640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f5450095170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5444009d80 tx=0x7f544400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 1 -- 10.20.193.78:0/1699536337 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444018020 con 0x7f545009d170 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: dump: 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 1 -- 10.20.193.78:0/1699536337 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54440146a0 con 0x7f545009d170 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 1 -- 10.20.193.78:0/1699536337 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444014a90 con 0x7f545009d170 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f544dffb640 4 set_mon_vals no callback set 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f542bfff640 10 monclient: dump: 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.188 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f5455ec9640 5 monclient: authenticate success, global_id 254316 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f5455ec9640 10 monclient: _renew_subs 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f5455ec9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54500967e0 con 0x7f545009d170 2026-02-17T06:08:30.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f5455ec9640 10 monclient: _renew_subs 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f5455ec9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.169+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5450097c90 con 0x7f545009d170 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f5455ec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f5455ec9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f5455ec9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5450074980 con 0x7f545009d170 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f542bfff640 1 -- 10.20.193.78:0/1699536337 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5444014c30 con 0x7f545009d170 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f542bfff640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f542003deb0 0x7f5420040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f542bfff640 1 -- 10.20.193.78:0/1699536337 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7f5444053870 con 0x7f545009d170 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f544f7fe640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f542003deb0 0x7f5420040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.170+0000 7f544f7fe640 10 monclient: get_auth_request con 0x7f542003deb0 auth_method 0 2026-02-17T06:08:30.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.172+0000 7f542bfff640 1 -- 10.20.193.78:0/1699536337 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5444022030 con 0x7f545009d170 2026-02-17T06:08:30.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.172+0000 7f542bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.172+0000 7f542bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:30.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.173+0000 7f544f7fe640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f542003deb0 0x7f5420040370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f543c005160 tx=0x7f543c007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4124132383}) 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1999955691}) 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4124132383}) 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4260472556}) 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=530472760}) 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190248465}) 2026-02-17T06:08:30.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=530472760}) 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.205+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1218303660}) 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=504903995}) 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2543515732}) 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=504903995}) 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1446680889}) 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1171638732}) 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2650832272}) 2026-02-17T06:08:30.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1171638732}) 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1817103315}) 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765127157}) 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2741929244}) 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765127157}) 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.206+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=504854246}) 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=439661875}) 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=963414115}) 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=439661875}) 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2440336483}) 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2865330409}) 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3795625557}) 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2865330409}) 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.207+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3832350855}) 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875893938}) 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3558886072}) 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875893938}) 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=763867223}) 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1616252738}) 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2736350209}) 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1616252738}) 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3116650549}) 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197072370}) 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=818075870}) 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4197072370}) 2026-02-17T06:08:30.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.208+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2218372649}) 2026-02-17T06:08:30.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3160636614}) 2026-02-17T06:08:30.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4235684680}) 2026-02-17T06:08:30.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3160636614}) 2026-02-17T06:08:30.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1341851144}) 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2224908711}) 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=289235740}) 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2224908711}) 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.209+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=966153391}) 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1061334734}) 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1843935827}) 2026-02-17T06:08:30.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1061334734}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1466777141}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=12142043}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=736659078}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=12142043}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=607156005}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=484427780}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=452531786}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=484427780}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.210+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289548606}) 2026-02-17T06:08:30.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2600202044}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=441991513}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2600202044}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2146057331}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=506365395}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2278610634}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=506365395}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2485973789}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3837797238}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2775186338}) 2026-02-17T06:08:30.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3837797238}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.211+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1437715423}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594681356}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956378802}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594681356}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730039399}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681072754}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1064696729}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681072754}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2829823024}) 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.235 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1072204662}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=877060814}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1072204662}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.212+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023485108}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2351782044}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3231179641}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2351782044}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1467066215}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1977740854}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3043421837}) 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.236 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1977740854}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2736836996}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1844878477}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2542921613}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1844878477}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.213+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3070702368}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570019602}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4156982379}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570019602}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=305153055}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483009635}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=119044133}) 2026-02-17T06:08:30.237 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483009635}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3482660673}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2470915020}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=395730121}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2470915020}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3012712884}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3026947998}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266288888}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3026947998}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.214+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=279606160}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=585849505}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468783866}) 2026-02-17T06:08:30.238 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=585849505}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1117664892}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1550753046}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2691521376}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1550753046}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3681670555}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231219412}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932984570}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231219412}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.215+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251879829}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=845555155}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270515187}) 2026-02-17T06:08:30.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=845555155}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=437444928}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3912066969}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2707616877}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3912066969}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2106022105}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2998120284}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=12153861}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2998120284}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.216+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048245235}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3910281269}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3492192837}) 2026-02-17T06:08:30.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3910281269}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=501511306}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=952357915}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=649320944}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=952357915}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3810001907}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=881068533}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3739384208}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=881068533}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1815436817}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130876294}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1637032584}) 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130876294}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.217+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1577914253}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1076579456}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1309746046}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1076579456}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=414339955}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1251950825}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2897696344}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1251950825}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2869242885}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4236723583}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3413371684}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4236723583}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.218+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3032499868}) 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3591122526}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2201568261}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3591122526}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2382513668}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2209556897}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=422856715}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2209556897}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=464864297}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622170067}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1740594504}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622170067}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3169259932}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.219+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2016296716}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2944870716}) 2026-02-17T06:08:30.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2016296716}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3605336634}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732929808}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=25862877}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732929808}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2340545575}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3870368641}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3203048693}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3870368641}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3962061859}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2968617361}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1704964467}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2968617361}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.220+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1749611600}) 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3535482451}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1454088429}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3535482451}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=121335320}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9088925}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3984549185}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9088925}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1964520167}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931106245}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3283226212}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931106245}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.221+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1266572342}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2877069440}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094123991}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2877069440}) 2026-02-17T06:08:30.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1973647804}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3881842550}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3077148267}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3881842550}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3674484995}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1407478839}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3227220382}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1407478839}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.222+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3686999988}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=2} crc {logm=2511800767,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3258734097,mon_config_key=1943263243}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=79,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=2576588564,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3826097715}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=2} crc {logm=2511800767,mdsmap=1160443688,mgr=3987721425,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=3258734097,mon_config_key=1943263243}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=79,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=1} crc {logm=255350561,mdsmap=1160443688,mgr=787438293,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=1895597338,mon_config_key=2343521820}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=80} crc {mon_config_key=1093141065,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=241526288,osd_snap=4276326829}) 2026-02-17T06:08:30.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=10,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=81} crc {mon_config_key=344415991,monmap=1784110042,osd_metadata=3878803090,osd_pg_creating=241526288,osd_snap=1027723056}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=80} crc {mon_config_key=1093141065,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=241526288,osd_snap=4276326829}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=80} crc {mon_config_key=1593018701,monmap=1784110042,osd_metadata=2997046373,osd_pg_creating=241526288,osd_snap=202049972}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=52,osdmap=48} crc {osd_snap=4191187500,osdmap=2768265848}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=52,osdmap=48} crc {osd_snap=3523027286,osdmap=360683720}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=52,osdmap=48} crc {osd_snap=4191187500,osdmap=2768265848}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.223+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2029264917,osdmap=937453932}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.224+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.224+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2952272708}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.224+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1828260583}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.224+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.224+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2952272708}) 2026-02-17T06:08:30.247 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.224+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2864028726}) 2026-02-17T06:08:30.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.285+0000 7f5455ec9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:08:30.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.285+0000 7f5455ec9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:30.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.285+0000 7f5455ec9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:30.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.285+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f54500976d0 con 0x7f545009d170 2026-02-17T06:08:30.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.285+0000 7f542bfff640 1 -- 10.20.193.78:0/1699536337 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f54500a0d90 con 0x7f545009d170 2026-02-17T06:08:30.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.285+0000 7f542bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:30.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.285+0000 7f542bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:30.287 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":252,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:08:30.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f542003deb0 msgr2=0x7f5420040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f542003deb0 0x7f5420040370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f543c005160 tx=0x7f543c007f30 comp rx=0 tx=0).stop 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 10 monclient: shutdown 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 msgr2=0x7f5450095170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f5450095170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5444009d80 tx=0x7f544400c490 comp rx=0 tx=0).stop 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 shutdown_connections 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f545009d170 0x7f5450095170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f542003deb0 0x7f5420040370 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f545009b200 0x7f5450094c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 --2- 10.20.193.78:0/1699536337 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f545006c890 0x7f54500946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 >> 10.20.193.78:0/1699536337 conn(0x7f54500a4230 msgr2=0x7f54500a5530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 shutdown_connections 2026-02-17T06:08:30.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.286+0000 7f5455ec9640 1 -- 10.20.193.78:0/1699536337 wait complete. 2026-02-17T06:08:30.295 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:08:30.295 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:08:30.295 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:08:30.296 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:08:30.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4076577316}) 2026-02-17T06:08:30.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=739002163}) 2026-02-17T06:08:30.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4076577316}) 2026-02-17T06:08:30.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.342+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1822617981}) 2026-02-17T06:08:30.352 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1345748640}) 2026-02-17T06:08:30.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1538925370}) 2026-02-17T06:08:30.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1345748640}) 2026-02-17T06:08:30.353 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.351+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=277002770}) 2026-02-17T06:08:30.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=43} crc {osdmap=4173230406}) 2026-02-17T06:08:30.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=43} crc {osdmap=1637960474}) 2026-02-17T06:08:30.359 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:08:30.360 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=43} crc {osdmap=4173230406}) 2026-02-17T06:08:30.360 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:08:30.358+0000 7f9f153cf640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=44} crc {osdmap=3559227322}) 2026-02-17T06:08:30.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.362+0000 7fd92ac4c640 10 monclient: get_monmap_and_config 2026-02-17T06:08:30.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.362+0000 7fd92ac4c640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.362+0000 7fd92ac4c640 10 monclient: monmap: 2026-02-17T06:08:30.363 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.363 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.363 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:30.363876+0000 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:30.363876+0000 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:30.364 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.375 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 1 Processor -- start 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 1 -- start start 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient: init 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient: _add_conns ranks=[1,0,4,2,3] 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd92409ddd0 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd924073f60 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd92413e990 0x7fd924140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient(hunting): picked mon.noname-c con 0x7fd92413e990 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9240a2850 con 0x7fd924073f60 2026-02-17T06:08:30.376 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9240a2560 con 0x7fd92409ddd0 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd9240a3810 con 0x7fd92413e990 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd92ac4c640 10 monclient(hunting): authenticate will time out at 1955.917742s 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd91bfff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fd924073f60 0x7fd92413d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56944/0 (socket says 10.20.193.78:56944) 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd91bfff640 1 -- 10.20.193.78:0/807508063 learned_addr learned my addr 10.20.193.78:0/807508063 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd9291c2640 1 --2- 10.20.193.78:0/807508063 >> v2:10.20.193.115:3300/0 conn(0x7fd92413e990 0x7fd924140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd9291c2640 10 monclient(hunting): get_auth_request con 0x7fd92413e990 auth_method 0 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd9291c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd9291c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.377 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.363+0000 7fd9291c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.364+0000 7fd9291c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.364+0000 7fd9291c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.364+0000 7fd9291c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.365+0000 7fd91bfff640 1 -- 10.20.193.78:0/807508063 >> v1:10.20.193.78:6789/0 conn(0x7fd924073f60 legacy=0x7fd92413d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569408500 0 0) 0x7fd9240a2850 con 0x7fd924073f60 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): _negotiate 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): my global_id is 254319 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd910003620 con 0x7fd924073f60 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 1 -- 10.20.193.78:0/807508063 >> v1:10.20.193.115:6789/0 conn(0x7fd92409ddd0 legacy=0x7fd92409e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.378 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1836861967 0 0) 0x7fd910003620 con 0x7fd924073f60 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 >> v2:10.20.193.115:3300/0 conn(0x7fd92413e990 msgr2=0x7fd924140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 --2- 10.20.193.78:0/807508063 >> v2:10.20.193.115:3300/0 conn(0x7fd92413e990 0x7fd924140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 >> v1:10.20.193.115:6789/0 conn(0x7fd92409ddd0 legacy=0x7fd92409e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9240a3af0 con 0x7fd924073f60 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fd90c0031b0 con 0x7fd924073f60 2026-02-17T06:08:30.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: dump: 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.380 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 >> v1:10.20.193.78:6789/0 conn(0x7fd924073f60 legacy=0x7fd92413d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd910004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): picked mon.a con 0x7fd910003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd910004900 0x7fd910004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): picked mon.c con 0x7fd910004900 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9100054c0 0x7fd91000bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): picked mon.b con 0x7fd9100054c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:30.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9240a3810 con 0x7fd910003c20 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd91000c400 con 0x7fd9100054c0 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd91000c600 con 0x7fd910004900 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd910004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient(hunting): get_auth_request con 0x7fd910003c20 auth_method 0 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:30.382 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91bfff640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd910004900 0x7fd910004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91bfff640 10 monclient(hunting): get_auth_request con 0x7fd910004900 auth_method 0 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd91bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient(hunting): handle_auth_done global_id 254319 payload 293 2026-02-17T06:08:30.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9291c2640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9100054c0 0x7fd91000bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 1 -- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd910004900 msgr2=0x7fd910004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd910004900 0x7fd910004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 1 -- 10.20.193.78:0/807508063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9100054c0 msgr2=0x7fd91000bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9100054c0 0x7fd91000bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 1 monclient: found mon.a 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 1 -- 10.20.193.78:0/807508063 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd910003620 con 0x7fd910003c20 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9289c1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd92ac4c640 5 monclient: authenticate success, global_id 254319 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd92ac4c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9291c2640 10 monclient: get_auth_request con 0x7fd9100054c0 auth_method 0 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9291c2640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9100054c0 0x7fd91000bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.366+0000 7fd9291c2640 1 -- 10.20.193.78:0/807508063 reap_dead start 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.368+0000 7fd9289c1640 1 -- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 msgr2=0x7fd910004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.368+0000 7fd9289c1640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd910004040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9140032a0 tx=0x7fd91402efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13c35ec8b036572 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914002fa0 con 0x7fd910003c20 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 10 monclient: dump: 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.385 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd914033030 con 0x7fd910003c20 2026-02-17T06:08:30.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 1 -- 10.20.193.78:0/807508063 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914005d20 con 0x7fd910003c20 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd91b7fe640 10 monclient: dump: 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.387 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 4 set_mon_vals no callback set 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 10 monclient: shutdown 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 1 -- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 msgr2=0x7fd910004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd910004040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9140032a0 tx=0x7fd91402efa0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 1 -- 10.20.193.78:0/807508063 shutdown_connections 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 1 --2- 10.20.193.78:0/807508063 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd910004040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 1 -- 10.20.193.78:0/807508063 >> 10.20.193.78:0/807508063 conn(0x7fd9240aacd0 msgr2=0x7fd9240ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 1 -- 10.20.193.78:0/807508063 shutdown_connections 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 1 -- 10.20.193.78:0/807508063 wait complete. 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.369+0000 7fd92ac4c640 10 monclient: monmap: 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:30.389 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 1 Processor -- start 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 1 -- start start 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient: init 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd924073930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient(hunting): picked mon.noname-a con 0x7fd910003c20 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd9241cf120 0x7fd9241cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient(hunting): picked mon.noname-c con 0x7fd9241cf120 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9241cfa60 0x7fd92413c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient(hunting): picked mon.noname-b con 0x7fd9241cfa60 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:30.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9240a4840 con 0x7fd910003c20 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9240a57e0 con 0x7fd9241cfa60 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd9240a49c0 con 0x7fd9241cf120 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd92ac4c640 10 monclient(hunting): authenticate will time out at 1955.924356s 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd924073930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd9241cf120 0x7fd9241cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd924073930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:47808/0 (socket says 10.20.193.78:47808) 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 1 -- 10.20.193.78:0/2878005154 learned_addr learned my addr 10.20.193.78:0/2878005154 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient(hunting): get_auth_request con 0x7fd9241cf120 auth_method 0 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 10 monclient(hunting): get_auth_request con 0x7fd910003c20 auth_method 0 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9291c2640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9241cfa60 0x7fd92413c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9291c2640 10 monclient(hunting): get_auth_request con 0x7fd9241cfa60 auth_method 0 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9291c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9291c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9291c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient(hunting): handle_auth_done global_id 264230 payload 824 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9241cfa60 msgr2=0x7fd92413c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd9241cfa60 0x7fd92413c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 msgr2=0x7fd924073930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd924073930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 1 monclient: found mon.noname-c 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd91bfff640 1 -- 10.20.193.78:0/2878005154 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd924070d30 con 0x7fd9241cf120 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.370+0000 7fd9289c1640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd910003c20 0x7fd924073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd91bfff640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd9241cf120 msgr2=0x7fd9241cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd91bfff640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd9241cf120 0x7fd9241cf520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd924074380 tx=0x7fd90c0077b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01a070 con 0x7fd9241cf120 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: dump: 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd90c00fb40 con 0x7fd9241cf120 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd92ac4c640 5 monclient: authenticate success, global_id 264230 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd92ac4c640 10 monclient: _renew_subs 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd92ac4c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92406fef0 con 0x7fd9241cf120 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c012350 con 0x7fd9241cf120 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd91affd640 4 set_mon_vals no callback set 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd9197fa640 10 monclient: dump: 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.395 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd92ac4c640 10 monclient: _renew_subs 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd92ac4c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.372+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92406f400 con 0x7fd9241cf120 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd92ac4c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd92ac4c640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd92413c890 0x7fd92413cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd92ac4c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd924069d50 con 0x7fd92413c890 2026-02-17T06:08:30.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd92413c890 0x7fd92413cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 10 monclient: get_auth_request con 0x7fd92413c890 auth_method 0 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 10 monclient(hunting): handle_auth_done global_id 254325 payload 824 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9289c1640 1 -- 10.20.193.78:0/2878005154 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9240a1c80 con 0x7fd92413c890 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd90c0124f0 con 0x7fd9241cf120 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9197fa640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8f403deb0 0x7fd8f4040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.373+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7fd90c052350 con 0x7fd9241cf120 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.374+0000 7fd9291c2640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8f403deb0 0x7fd8f4040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.374+0000 7fd9291c2640 10 monclient: get_auth_request con 0x7fd8f403deb0 auth_method 0 2026-02-17T06:08:30.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.375+0000 7fd9289c1640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd92413c890 msgr2=0x7fd92413cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.375+0000 7fd9289c1640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd92413c890 0x7fd92413cc70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd91402efe0 tx=0x7fd914002cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.375+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914007270 con 0x7fd92413c890 2026-02-17T06:08:30.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.375+0000 7fd9197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.375+0000 7fd9197fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.375+0000 7fd9197fa640 10 monclient: dump: 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.376+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd914007270 con 0x7fd92413c890 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.376+0000 7fd9197fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.376+0000 7fd9197fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:30.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.376+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd92413c890 msgr2=0x7fd92413cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.376+0000 7fd9197fa640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd92413c890 0x7fd92413cc70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd91402efe0 tx=0x7fd914002cc0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.376+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd92413c890 msgr2=0x7fd92413cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.376+0000 7fd9197fa640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd92413c890 0x7fd92413cc70 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd91402efe0 tx=0x7fd914002cc0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.376+0000 7fd9291c2640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8f403deb0 0x7fd8f4040370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd9240a2f00 tx=0x7fd92000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd92ac4c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd92ac4c640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd924139040 0x7fd9241da200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd92ac4c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd92413d1b0 con 0x7fd924139040 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd924139040 0x7fd9241da200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 10 monclient: get_auth_request con 0x7fd924139040 auth_method 0 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 10 monclient(hunting): handle_auth_done global_id 254328 payload 824 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.379+0000 7fd9289c1640 1 -- 10.20.193.78:0/2878005154 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd92409e2d0 con 0x7fd924139040 2026-02-17T06:08:30.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9289c1640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd924139040 msgr2=0x7fd9241da200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9289c1640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd924139040 0x7fd9241da200 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd92406c6a0 tx=0x7fd914009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914043480 con 0x7fd924139040 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 10 monclient: dump: 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7fd9140373d0 con 0x7fd924139040 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:08:30.402 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd924139040 msgr2=0x7fd9241da200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd924139040 0x7fd9241da200 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd92406c6a0 tx=0x7fd914009070 comp rx=0 tx=0).stop 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd924139040 msgr2=0x7fd9241da200 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9197fa640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd924139040 0x7fd9241da200 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd92406c6a0 tx=0x7fd914009070 comp rx=0 tx=0).stop 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 252, 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 109755, 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:08:30.403 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 7, 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.404 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:08:30.405 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.406 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.407 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.408 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.409 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:08:30.410 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.411 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.412 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:08:30.412 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8f403deb0 msgr2=0x7fd8f4040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd92ac4c640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd8f403deb0 0x7fd8f4040370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd9240a2f00 tx=0x7fd92000b040 comp rx=0 tx=0).stop 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd92ac4c640 10 monclient: shutdown 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd92ac4c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd9241cf120 msgr2=0x7fd9241cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd92ac4c640 1 --2- 10.20.193.78:0/2878005154 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd9241cf120 0x7fd9241cf520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd924074380 tx=0x7fd90c0077b0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9291c2640 1 -- 10.20.193.78:0/2878005154 reap_dead start 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.381+0000 7fd9291c2640 1 -- 10.20.193.78:0/2878005154 reap_dead start 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.382+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 shutdown_connections 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.382+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 >> 10.20.193.78:0/2878005154 conn(0x7fd9240aacd0 msgr2=0x7fd9240a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.382+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 shutdown_connections 2026-02-17T06:08:30.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.382+0000 7fd92ac4c640 1 -- 10.20.193.78:0/2878005154 wait complete. 2026-02-17T06:08:30.413 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:08:30.414 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:08:30.414 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.479+0000 7fc17ec24640 10 monclient: get_monmap_and_config 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.479+0000 7fc17ec24640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.479+0000 7fc17ec24640 10 monclient: monmap: 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:30.480595+0000 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:30.480595+0000 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:30.480 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.479+0000 7fc17ec24640 1 Processor -- start 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.479+0000 7fc17ec24640 1 -- start start 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.479+0000 7fc17ec24640 10 monclient: init 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc17806c890 0x7fc1780a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient(hunting): picked mon.noname-c con 0x7fc17806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc17806cc70 0x7fc17806d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient(hunting): picked mon.noname-a con 0x7fc17806cc70 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178134df0 0x7fc178135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient(hunting): picked mon.noname-b con 0x7fc178134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc178074150 con 0x7fc17806cc70 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc17809ef70 con 0x7fc17806c890 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc17809f250 con 0x7fc178134df0 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17ec24640 10 monclient(hunting): authenticate will time out at 1956.034219s 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178134df0 0x7fc178135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc16ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc17806cc70 0x7fc17806d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178134df0 0x7fc178135220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38250/0 (socket says 10.20.193.78:38250) 2026-02-17T06:08:30.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 -- 10.20.193.78:0/1438607575 learned_addr learned my addr 10.20.193.78:0/1438607575 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient(hunting): get_auth_request con 0x7fc178134df0 auth_method 0 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc16ffff640 10 monclient(hunting): get_auth_request con 0x7fc17806cc70 auth_method 0 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc16ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc16ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc16ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17c999640 1 --2- 10.20.193.78:0/1438607575 >> v2:10.20.193.115:3300/0 conn(0x7fc17806c890 0x7fc1780a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17c999640 10 monclient(hunting): get_auth_request con 0x7fc17806c890 auth_method 0 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17c999640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17c999640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17c999640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient(hunting): handle_auth_done global_id 264233 payload 274 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 -- 10.20.193.78:0/1438607575 >> v2:10.20.193.115:3300/0 conn(0x7fc17806c890 msgr2=0x7fc1780a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 --2- 10.20.193.78:0/1438607575 >> v2:10.20.193.115:3300/0 conn(0x7fc17806c890 0x7fc1780a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 -- 10.20.193.78:0/1438607575 >> v2:10.20.193.78:3300/0 conn(0x7fc17806cc70 msgr2=0x7fc17806d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 --2- 10.20.193.78:0/1438607575 >> v2:10.20.193.78:3300/0 conn(0x7fc17806cc70 0x7fc17806d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 monclient: found mon.noname-b 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.480+0000 7fc17d19a640 1 -- 10.20.193.78:0/1438607575 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17809df40 con 0x7fc178134df0 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17d19a640 1 -- 10.20.193.78:0/1438607575 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178134df0 msgr2=0x7fc178135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17d19a640 1 --2- 10.20.193.78:0/1438607575 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178134df0 0x7fc178135220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc17806a480 tx=0x7fc17402f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87692226f0e02ab7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 1 -- 10.20.193.78:0/1438607575 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740373f0 con 0x7fc178134df0 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: dump: 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.493 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 1 -- 10.20.193.78:0/1438607575 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc174037590 con 0x7fc178134df0 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 5 monclient: authenticate success, global_id 264233 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 1 -- 10.20.193.78:0/1438607575 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174037960 con 0x7fc178134df0 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 4 set_mon_vals no callback set 2026-02-17T06:08:30.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc16f7fe640 10 monclient: dump: 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 10 monclient: shutdown 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 -- 10.20.193.78:0/1438607575 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178134df0 msgr2=0x7fc178135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 --2- 10.20.193.78:0/1438607575 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178134df0 0x7fc178135220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc17806a480 tx=0x7fc17402f330 comp rx=0 tx=0).stop 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 -- 10.20.193.78:0/1438607575 shutdown_connections 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 --2- 10.20.193.78:0/1438607575 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178134df0 0x7fc178135220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 --2- 10.20.193.78:0/1438607575 >> v2:10.20.193.78:3300/0 conn(0x7fc17806cc70 0x7fc17806d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 --2- 10.20.193.78:0/1438607575 >> v2:10.20.193.115:3300/0 conn(0x7fc17806c890 0x7fc1780a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 -- 10.20.193.78:0/1438607575 >> 10.20.193.78:0/1438607575 conn(0x7fc1780a6620 msgr2=0x7fc1780a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 -- 10.20.193.78:0/1438607575 shutdown_connections 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 1 -- 10.20.193.78:0/1438607575 wait complete. 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.483+0000 7fc17ec24640 10 monclient: monmap: 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:30.496 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 1 Processor -- start 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 1 -- start start 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient: init 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17806c890 0x7fc17812d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient(hunting): picked mon.noname-c con 0x7fc17806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc17806cc70 0x7fc17812d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient(hunting): picked mon.noname-b con 0x7fc17806cc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:30.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc178134df0 0x7fc17813c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient(hunting): picked mon.noname-a con 0x7fc178134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc17809d360 con 0x7fc178134df0 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc17809d1e0 con 0x7fc17806cc70 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc178135760 con 0x7fc17806c890 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17ec24640 10 monclient(hunting): authenticate will time out at 1956.038417s 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17806c890 0x7fc17812d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc178134df0 0x7fc17813c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17806c890 0x7fc17812d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38254/0 (socket says 10.20.193.78:38254) 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 -- 10.20.193.78:0/550543120 learned_addr learned my addr 10.20.193.78:0/550543120 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient(hunting): get_auth_request con 0x7fc17806c890 auth_method 0 2026-02-17T06:08:30.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 10 monclient(hunting): get_auth_request con 0x7fc178134df0 auth_method 0 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient(hunting): handle_auth_done global_id 264236 payload 824 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc16ffff640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc17806cc70 0x7fc17812d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.499 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc17806cc70 msgr2=0x7fc17812d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc17806cc70 0x7fc17812d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc178134df0 msgr2=0x7fc17813c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc178134df0 0x7fc17813c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 monclient: found mon.noname-c 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17c999640 1 -- 10.20.193.78:0/550543120 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17809d810 con 0x7fc17806c890 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc17d19a640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc178134df0 0x7fc17813c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc16ffff640 10 monclient: get_auth_request con 0x7fc17806cc70 auth_method 0 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.484+0000 7fc16ffff640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc17806cc70 0x7fc17812d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17c999640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17806c890 msgr2=0x7fc17812d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17c999640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17806c890 0x7fc17812d2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc16800e7f0 tx=0x7fc16800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168017070 con 0x7fc17806c890 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: dump: 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc16800cae0 con 0x7fc17806c890 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16800f040 con 0x7fc17806c890 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16effd640 4 set_mon_vals no callback set 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc16d7fa640 10 monclient: dump: 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.502 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17ec24640 5 monclient: authenticate success, global_id 264236 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17ec24640 10 monclient: _renew_subs 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17ec24640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17813aeb0 con 0x7fc17806c890 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17ec24640 10 monclient: _renew_subs 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17ec24640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.486+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc178137ba0 con 0x7fc17806c890 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc17ec24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc17ec24640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17813c790 0x7fc17813cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc17ec24640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc178137270 con 0x7fc17813c790 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17813c790 0x7fc17813cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 10 monclient: get_auth_request con 0x7fc17813c790 auth_method 0 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc168012050 con 0x7fc17806c890 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 10 monclient(hunting): handle_auth_done global_id 264239 payload 824 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16ffff640 1 -- 10.20.193.78:0/550543120 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc17813d0b0 con 0x7fc17813c790 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16d7fa640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc14803deb0 0x7fc148040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.487+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7fc168052670 con 0x7fc17806c890 2026-02-17T06:08:30.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.488+0000 7fc17d19a640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc14803deb0 0x7fc148040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.488+0000 7fc17d19a640 10 monclient: get_auth_request con 0x7fc14803deb0 auth_method 0 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16ffff640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17813c790 msgr2=0x7fc17813cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16ffff640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17813c790 0x7fc17813cb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1780735d0 tx=0x7fc160010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16000f040 con 0x7fc17813c790 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16d7fa640 10 monclient: dump: 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.505 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fc160004020 con 0x7fc17813c790 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.489+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17813c790 msgr2=0x7fc17813cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.490+0000 7fc16d7fa640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17813c790 0x7fc17813cb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1780735d0 tx=0x7fc160010d70 comp rx=0 tx=0).stop 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.490+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17813c790 msgr2=0x7fc17813cb70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.490+0000 7fc16d7fa640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17813c790 0x7fc17813cb70 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1780735d0 tx=0x7fc160010d70 comp rx=0 tx=0).stop 2026-02-17T06:08:30.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.490+0000 7fc17d19a640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc14803deb0 0x7fc148040370 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc178074e00 tx=0x7fc174033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.492+0000 7fc17ec24640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:08:30.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.492+0000 7fc17ec24640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178131730 0x7fc1781da440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc17ec24640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc178137820 con 0x7fc178131730 2026-02-17T06:08:30.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178131730 0x7fc1781da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 10 monclient: get_auth_request con 0x7fc178131730 auth_method 0 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 10 monclient(hunting): handle_auth_done global_id 264242 payload 824 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.493+0000 7fc16ffff640 1 -- 10.20.193.78:0/550543120 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc178074980 con 0x7fc178131730 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16ffff640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178131730 msgr2=0x7fc1781da440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16ffff640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178131730 0x7fc1781da440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1780735d0 tx=0x7fc1600047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160027470 con 0x7fc178131730 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 10 monclient: dump: 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7fc16001c400 con 0x7fc178131730 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178131730 msgr2=0x7fc1781da440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178131730 0x7fc1781da440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1780735d0 tx=0x7fc1600047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178131730 msgr2=0x7fc1781da440 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc16d7fa640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc178131730 0x7fc1781da440 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1780735d0 tx=0x7fc1600047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 252, 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 59880, 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:08:30.511 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 7, 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.512 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.513 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:08:30.514 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.515 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:08:30.516 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.517 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.518 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.519 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:08:30.520 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.521 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.521 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.521 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:30.521 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.521 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.521 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.521 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:08:30.521 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc14803deb0 msgr2=0x7fc148040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc14803deb0 0x7fc148040370 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc178074e00 tx=0x7fc174033000 comp rx=0 tx=0).stop 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 10 monclient: shutdown 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17806c890 msgr2=0x7fc17812d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17806c890 0x7fc17812d2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc16800e7f0 tx=0x7fc16800ecc0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17d19a640 1 -- 10.20.193.78:0/550543120 reap_dead start 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 shutdown_connections 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 1 --2- 10.20.193.78:0/550543120 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc17806c890 0x7fc17812d2c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.495+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 >> 10.20.193.78:0/550543120 conn(0x7fc1780a6620 msgr2=0x7fc1781303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.496+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 shutdown_connections 2026-02-17T06:08:30.523 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.496+0000 7fc17ec24640 1 -- 10.20.193.78:0/550543120 wait complete. 2026-02-17T06:08:30.523 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:08:30.523 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:08:30.523 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:08:30.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.589+0000 7f37f7e9f640 10 monclient: get_monmap_and_config 2026-02-17T06:08:30.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.589+0000 7f37f7e9f640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.589+0000 7f37f7e9f640 10 monclient: monmap: 2026-02-17T06:08:30.590 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.590 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.590 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:30.590513+0000 2026-02-17T06:08:30.590 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:30.590513+0000 2026-02-17T06:08:30.590 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.591 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.591 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:30.591 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:30.591 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:30.591 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:30.591 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:30.591 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.589+0000 7f37f7e9f640 1 Processor -- start 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.589+0000 7f37f7e9f640 1 -- start start 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.589+0000 7f37f7e9f640 10 monclient: init 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f37f006c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006d380 0x7f37f00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient(hunting): picked mon.noname-b con 0x7f37f006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f37f013a240 0x7f37f013c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient(hunting): picked mon.noname-c con 0x7f37f013a240 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f0074150 con 0x7f37f006c890 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f37f009ef70 con 0x7f37f013a240 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f37f009f250 con 0x7f37f006d380 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f7e9f640 10 monclient(hunting): authenticate will time out at 1956.144215s 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006d380 0x7f37f00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5c14640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f37f006c890 0x7f37f006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:56956/0 (socket says 10.20.193.78:56956) 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5c14640 1 -- 10.20.193.78:0/748366051 learned_addr learned my addr 10.20.193.78:0/748366051 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient(hunting): get_auth_request con 0x7f37f006d380 auth_method 0 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient(hunting): handle_auth_done global_id 264245 payload 274 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 1 -- 10.20.193.78:0/748366051 >> v2:10.20.193.115:3300/0 conn(0x7f37f013a240 msgr2=0x7f37f013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 1 --2- 10.20.193.78:0/748366051 >> v2:10.20.193.115:3300/0 conn(0x7f37f013a240 0x7f37f013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 1 -- 10.20.193.78:0/748366051 >> v1:10.20.193.78:6789/0 conn(0x7f37f006c890 legacy=0x7f37f006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 1 monclient: found mon.noname-b 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.590+0000 7f37f5413640 1 -- 10.20.193.78:0/748366051 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f009df40 con 0x7f37f006d380 2026-02-17T06:08:30.601 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.592+0000 7f37f5413640 1 -- 10.20.193.78:0/748366051 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006d380 msgr2=0x7f37f00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.592+0000 7f37f5413640 1 --2- 10.20.193.78:0/748366051 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006d380 0x7f37f00a01f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37f006e270 tx=0x7f37e402eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d97a4fd1f436f845 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 1 -- 10.20.193.78:0/748366051 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e403c070 con 0x7f37f006d380 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: dump: 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.602 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 1 -- 10.20.193.78:0/748366051 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e402fdd0 con 0x7f37f006d380 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 1 -- 10.20.193.78:0/748366051 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4037660 con 0x7f37f006d380 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f4c12640 10 monclient: dump: 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.603 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 5 monclient: authenticate success, global_id 264245 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 4 set_mon_vals no callback set 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 10 monclient: shutdown 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 1 -- 10.20.193.78:0/748366051 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006d380 msgr2=0x7f37f00a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.604 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 1 --2- 10.20.193.78:0/748366051 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006d380 0x7f37f00a01f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37f006e270 tx=0x7f37e402eeb0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 1 -- 10.20.193.78:0/748366051 shutdown_connections 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 1 --2- 10.20.193.78:0/748366051 >> v2:10.20.193.115:3300/0 conn(0x7f37f013a240 0x7f37f013c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 1 --2- 10.20.193.78:0/748366051 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006d380 0x7f37f00a01f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 1 -- 10.20.193.78:0/748366051 >> 10.20.193.78:0/748366051 conn(0x7f37f00a6620 msgr2=0x7f37f00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 1 -- 10.20.193.78:0/748366051 shutdown_connections 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 1 -- 10.20.193.78:0/748366051 wait complete. 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.593+0000 7f37f7e9f640 10 monclient: monmap: 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:30.605 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 1 Processor -- start 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 1 -- start start 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient: init 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006c890 0x7f37f01cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient(hunting): picked mon.noname-c con 0x7f37f006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f006d380 0x7f37f0134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient(hunting): picked mon.noname-b con 0x7f37f006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37f013a240 0x7f37f011dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient(hunting): picked mon.noname-a con 0x7f37f013a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37f013ccf0 con 0x7f37f013a240 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37f013cb70 con 0x7f37f006d380 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f37f013ce70 con 0x7f37f006c890 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f7e9f640 10 monclient(hunting): authenticate will time out at 1956.148491s 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006c890 0x7f37f01cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006c890 0x7f37f01cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38296/0 (socket says 10.20.193.78:38296) 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 -- 10.20.193.78:0/1089214790 learned_addr learned my addr 10.20.193.78:0/1089214790 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient(hunting): get_auth_request con 0x7f37f006c890 auth_method 0 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f6415640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37f013a240 0x7f37f011dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f6415640 10 monclient(hunting): get_auth_request con 0x7f37f013a240 auth_method 0 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f6415640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f6415640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f6415640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f6415640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f6415640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f6415640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient(hunting): handle_auth_done global_id 264248 payload 824 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f006d380 msgr2=0x7f37f0134190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5413640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f006d380 0x7f37f0134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f006d380 0x7f37f0134190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37f013a240 msgr2=0x7f37f011dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.608 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37f013a240 0x7f37f011dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 monclient: found mon.noname-c 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5c14640 1 -- 10.20.193.78:0/1089214790 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f0120480 con 0x7f37f006c890 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5413640 10 monclient: get_auth_request con 0x7f37f006d380 auth_method 0 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.594+0000 7f37f5413640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f006d380 0x7f37f0134190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37f5c14640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006c890 msgr2=0x7f37f01cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37f5c14640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006c890 0x7f37f01cefe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f37e000c780 tx=0x7f37e000cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0005de0 con 0x7f37f006c890 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 10 monclient: dump: 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.609 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e0007070 con 0x7f37f006c890 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.596+0000 7f37de7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0007440 con 0x7f37f006c890 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37dffff640 4 set_mon_vals no callback set 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37de7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37de7fc640 10 monclient: dump: 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 5 monclient: authenticate success, global_id 264248 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 10 monclient: _renew_subs 2026-02-17T06:08:30.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f01208a0 con 0x7f37f006c890 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 10 monclient: _renew_subs 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f0121890 con 0x7f37f006c890 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f01346d0 0x7f37f0136220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37f00a01d0 con 0x7f37f01346d0 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f37e0007620 con 0x7f37f006c890 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f5413640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f01346d0 0x7f37f0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f5413640 10 monclient: get_auth_request con 0x7f37f01346d0 auth_method 0 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f5413640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f5413640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f5413640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.597+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.598+0000 7f37de7fc640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37c003df00 0x7f37c00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.598+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7f37e0052530 con 0x7f37f006c890 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.598+0000 7f37f5413640 10 monclient(hunting): handle_auth_done global_id 274120 payload 824 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.598+0000 7f37f5413640 1 -- 10.20.193.78:0/1089214790 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37f0134b20 con 0x7f37f01346d0 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.598+0000 7f37f6415640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37c003df00 0x7f37c00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.598+0000 7f37f6415640 10 monclient: get_auth_request con 0x7f37c003df00 auth_method 0 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37f5413640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f01346d0 msgr2=0x7f37f0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37f5413640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f01346d0 0x7f37f0136220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37f006a480 tx=0x7f37e40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4047070 con 0x7f37f01346d0 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 10 monclient: dump: 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.613 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f37e4037de0 con 0x7f37f01346d0 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f01346d0 msgr2=0x7f37f0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f01346d0 0x7f37f0136220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37f006a480 tx=0x7f37e40047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f01346d0 msgr2=0x7f37f0136220 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:30.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37de7fc640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f01346d0 0x7f37f0136220 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37f006a480 tx=0x7f37e40047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.600+0000 7f37f6415640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37c003df00 0x7f37c00403c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f37f011f570 tx=0x7f37ec00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.603+0000 7f37f7e9f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.603+0000 7f37f7e9f640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f0136760 0x7f37f01da390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.603+0000 7f37f7e9f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.603+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37f00744d0 con 0x7f37f0136760 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f0136760 0x7f37f01da390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 10 monclient: get_auth_request con 0x7f37f0136760 auth_method 0 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 10 monclient(hunting): handle_auth_done global_id 274123 payload 824 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.604+0000 7f37f5413640 1 -- 10.20.193.78:0/1089214790 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f37f011e8e0 con 0x7f37f0136760 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f5413640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f0136760 msgr2=0x7f37f01da390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f5413640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f0136760 0x7f37f01da390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006de0 tx=0x7f37e4008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e403c040 con 0x7f37f0136760 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 10 monclient: dump: 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.618 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f37e4003dc0 con 0x7f37f0136760 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f0136760 msgr2=0x7f37f01da390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f0136760 0x7f37f01da390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006de0 tx=0x7f37e4008030 comp rx=0 tx=0).stop 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f0136760 msgr2=0x7f37f01da390 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37de7fc640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37f0136760 0x7f37f01da390 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006de0 tx=0x7f37e4008030 comp rx=0 tx=0).stop 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 252, 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 12608, 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:08:30.620 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 7, 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.621 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.622 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:08:30.623 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:08:30.624 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.625 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.626 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:30.627 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:30.628 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:30.629 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:30.630 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:08:30.630 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37c003df00 msgr2=0x7f37c00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f7e9f640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37c003df00 0x7f37c00403c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f37f011f570 tx=0x7f37ec00d040 comp rx=0 tx=0).stop 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f7e9f640 10 monclient: shutdown 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f7e9f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006c890 msgr2=0x7f37f01cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f7e9f640 1 --2- 10.20.193.78:0/1089214790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37f006c890 0x7f37f01cefe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f37e000c780 tx=0x7f37e000cc50 comp rx=0 tx=0).stop 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f6415640 1 -- 10.20.193.78:0/1089214790 reap_dead start 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f6415640 1 -- 10.20.193.78:0/1089214790 reap_dead start 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 shutdown_connections 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.606+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 >> 10.20.193.78:0/1089214790 conn(0x7f37f00a6620 msgr2=0x7f37f0135190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.607+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 shutdown_connections 2026-02-17T06:08:30.631 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.607+0000 7f37f7e9f640 1 -- 10.20.193.78:0/1089214790 wait complete. 2026-02-17T06:08:30.632 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:08:30.632 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-17T06:08:30.632 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:08:30.632 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-17T06:08:30.632 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T06:08:30.632 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T06:08:30.632 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:08:30.644 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T06:08:30.644 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:08:30.644 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:08:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.748+0000 7fc3ac44c640 10 monclient: get_monmap_and_config 2026-02-17T06:08:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.748+0000 7fc3ac44c640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.748+0000 7fc3ac44c640 10 monclient: monmap: 2026-02-17T06:08:30.752 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.752 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.752 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:30.750175+0000 2026-02-17T06:08:30.752 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:30.750175+0000 2026-02-17T06:08:30.752 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 1 Processor -- start 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 1 -- start start 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient: init 2026-02-17T06:08:30.753 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc3a406c890 0x7fc3a40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient(hunting): picked mon.noname-a con 0x7fc3a406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc3a406cc70 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a413a240 0x7fc3a413c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient(hunting): picked mon.noname-b con 0x7fc3a413a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a4074150 con 0x7fc3a406cc70 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a409ef70 con 0x7fc3a406c890 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a409f250 con 0x7fc3a413a240 2026-02-17T06:08:30.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3ac44c640 10 monclient(hunting): authenticate will time out at 1956.303707s 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa1c1640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7fc3a406c890 msgr2=0x7fc3a40a01f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3a99c0640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7fc3a406cc70 legacy=0x7fc3a406d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa1c1640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc3a406c890 0x7fc3a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a413a240 0x7fc3a413c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a413a240 0x7fc3a413c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38338/0 (socket says 10.20.193.78:38338) 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 1 -- 10.20.193.78:0/2871417958 learned_addr learned my addr 10.20.193.78:0/2871417958 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient(hunting): get_auth_request con 0x7fc3a413a240 auth_method 0 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient(hunting): handle_auth_done global_id 264251 payload 274 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 1 -- 10.20.193.78:0/2871417958 >> v2:10.20.193.78:3300/0 conn(0x7fc3a406c890 msgr2=0x7fc3a40a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 1 --2- 10.20.193.78:0/2871417958 >> v2:10.20.193.78:3300/0 conn(0x7fc3a406c890 0x7fc3a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 1 -- 10.20.193.78:0/2871417958 >> v1:10.20.193.78:6789/0 conn(0x7fc3a406cc70 legacy=0x7fc3a406d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 1 monclient: found mon.noname-b 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.756 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.749+0000 7fc3aa9c2640 1 -- 10.20.193.78:0/2871417958 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a409df40 con 0x7fc3a413a240 2026-02-17T06:08:30.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3aa9c2640 1 -- 10.20.193.78:0/2871417958 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a413a240 msgr2=0x7fc3a413c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3aa9c2640 1 --2- 10.20.193.78:0/2871417958 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a413a240 0x7fc3a413c630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009e50 tx=0x7fc3a002f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd9141f760db8f0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.762 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 1 -- 10.20.193.78:0/2871417958 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00373f0 con 0x7fc3a413a240 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: dump: 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.763 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 5 monclient: authenticate success, global_id 264251 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 1 -- 10.20.193.78:0/2871417958 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a0037590 con 0x7fc3a413a240 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 1 -- 10.20.193.78:0/2871417958 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0037960 con 0x7fc3a413a240 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 4 set_mon_vals no callback set 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3a91bf640 10 monclient: dump: 2026-02-17T06:08:30.764 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 10 monclient: shutdown 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2871417958 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a413a240 msgr2=0x7fc3a413c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.765 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2871417958 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a413a240 0x7fc3a413c630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009e50 tx=0x7fc3a002f360 comp rx=0 tx=0).stop 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2871417958 shutdown_connections 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2871417958 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a413a240 0x7fc3a413c630 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2871417958 >> v2:10.20.193.78:3300/0 conn(0x7fc3a406c890 0x7fc3a40a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.752+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2871417958 >> 10.20.193.78:0/2871417958 conn(0x7fc3a40a6620 msgr2=0x7fc3a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2871417958 shutdown_connections 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2871417958 wait complete. 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient: build_initial_monmap 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient: monmap: 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:30.766 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 Processor -- start 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 -- start start 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient: init 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3a406c890 0x7fc3a41cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient(hunting): picked mon.noname-b con 0x7fc3a406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 0x7fc3a41cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient(hunting): picked mon.noname-c con 0x7fc3a406cc70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3a413a240 0x7fc3a4121680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient(hunting): picked mon.noname-a con 0x7fc3a413a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a409bfa0 con 0x7fc3a413a240 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a409be20 con 0x7fc3a406c890 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a409c120 con 0x7fc3a406cc70 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3ac44c640 10 monclient(hunting): authenticate will time out at 1956.307791s 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3aa9c2640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3a413a240 msgr2=0x7fc3a4121680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3aa9c2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3a413a240 0x7fc3a4121680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:30.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 0x7fc3a41cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 0x7fc3a41cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38350/0 (socket says 10.20.193.78:38350) 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 1 -- 10.20.193.78:0/2623171111 learned_addr learned my addr 10.20.193.78:0/2623171111 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 10 monclient(hunting): get_auth_request con 0x7fc3a406cc70 auth_method 0 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3a99c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.753+0000 7fc3aa1c1640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3a406c890 0x7fc3a41cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3aa1c1640 10 monclient(hunting): get_auth_request con 0x7fc3a406c890 auth_method 0 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3aa1c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3aa1c1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:30.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3aa1c1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 10 monclient(hunting): handle_auth_done global_id 264254 payload 824 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 1 -- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3a406c890 msgr2=0x7fc3a41cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3a406c890 0x7fc3a41cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 1 -- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3a413a240 msgr2=0x7fc3a4121680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3a413a240 0x7fc3a4121680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 10 monclient: _finish_hunting 0 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 1 monclient: found mon.noname-c 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.754+0000 7fc3a99c0640 1 -- 10.20.193.78:0/2623171111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a4125ff0 con 0x7fc3a406cc70 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3a99c0640 1 -- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 msgr2=0x7fc3a41cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3a99c0640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 0x7fc3a41cfb70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc39800b780 tx=0x7fc39800bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3a99c0640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 0x7fc3a41cfb70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc39800b780 tx=0x7fc39800bc50 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7fc398005650 mon_map magic: 0 2026-02-17T06:08:30.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 1 -- 10.20.193.78:0/2623171111 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398005270 con 0x7fc3a406cc70 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 10 monclient: dump: 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 10 monclient: _finish_auth 0 2026-02-17T06:08:30.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 1 -- 10.20.193.78:0/2623171111 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc398010070 con 0x7fc3a406cc70 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 5 monclient: authenticate success, global_id 264254 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc38affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 10 monclient: _renew_subs 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3a89be640 4 set_mon_vals no callback set 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a4125330 con 0x7fc3a406cc70 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 10 monclient: _renew_subs 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a4139780 con 0x7fc3a406cc70 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:30.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.756+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a409d810 con 0x7fc3a406cc70 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.757+0000 7fc38affd640 1 -- 10.20.193.78:0/2623171111 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398005650 con 0x7fc3a406cc70 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.757+0000 7fc38affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.757+0000 7fc38affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.757+0000 7fc38affd640 10 monclient: dump: 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:30.773 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:30.774 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:30.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.757+0000 7fc38affd640 1 -- 10.20.193.78:0/2623171111 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc39801d070 con 0x7fc3a406cc70 2026-02-17T06:08:30.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.757+0000 7fc38affd640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc37403de90 0x7fc374040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:30.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.757+0000 7fc3aa1c1640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc37403de90 0x7fc374040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:30.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.757+0000 7fc3aa1c1640 10 monclient: get_auth_request con 0x7fc37403de90 auth_method 0 2026-02-17T06:08:30.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.760+0000 7fc3aa1c1640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc37403de90 0x7fc374040350 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc394003a80 tx=0x7fc394003ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:30.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:30.760+0000 7fc38affd640 1 -- 10.20.193.78:0/2623171111 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7fc398052220 con 0x7fc3a406cc70 2026-02-17T06:08:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.379+0000 7fc38affd640 1 -- 10.20.193.78:0/2623171111 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc39801a060 con 0x7fc3a406cc70 2026-02-17T06:08:31.379 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.379+0000 7fc38affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:31.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.379+0000 7fc38affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:31.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.493+0000 7fc3ac44c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:08:31.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.493+0000 7fc3ac44c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:31.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.493+0000 7fc3ac44c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:31.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.493+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc3a40a01f0 con 0x7fc3a406cc70 2026-02-17T06:08:31.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.493+0000 7fc38affd640 1 -- 10.20.193.78:0/2623171111 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fc3a4125ff0 con 0x7fc3a406cc70 2026-02-17T06:08:31.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.493+0000 7fc38affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:31.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.493+0000 7fc38affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:31.496 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":252,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:08:31.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc37403de90 msgr2=0x7fc374040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:31.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc37403de90 0x7fc374040350 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc394003a80 tx=0x7fc394003ab0 comp rx=0 tx=0).stop 2026-02-17T06:08:31.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 10 monclient: shutdown 2026-02-17T06:08:31.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:31.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 msgr2=0x7fc3a41cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:31.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 0x7fc3a41cfb70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc39800b780 tx=0x7fc39800bc50 comp rx=0 tx=0).stop 2026-02-17T06:08:31.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 shutdown_connections 2026-02-17T06:08:31.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc3a413a240 0x7fc3a4121680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:31.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc3a406cc70 0x7fc3a41cfb70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:31.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc37403de90 0x7fc374040350 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:31.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 --2- 10.20.193.78:0/2623171111 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc3a406c890 0x7fc3a41cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:31.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 >> 10.20.193.78:0/2623171111 conn(0x7fc3a40a6620 msgr2=0x7fc3a4138230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:31.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 shutdown_connections 2026-02-17T06:08:31.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:31.494+0000 7fc3ac44c640 1 -- 10.20.193.78:0/2623171111 wait complete. 2026-02-17T06:08:34.503 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:08:34.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.571+0000 7f849a385640 10 monclient: get_monmap_and_config 2026-02-17T06:08:34.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.571+0000 7f849a385640 10 monclient: build_initial_monmap 2026-02-17T06:08:34.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.571+0000 7f849a385640 10 monclient: monmap: 2026-02-17T06:08:34.575 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:34.575 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:34.575 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:34.572826+0000 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:34.572826+0000 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 1 Processor -- start 2026-02-17T06:08:34.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 1 -- start start 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient: init 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84940a1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient(hunting): picked mon.noname-b con 0x7f84940a6470 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f84940a6850 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f84940a6c30 0x7f8494138660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient(hunting): picked mon.noname-c con 0x7f84940a6c30 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84940a0730 con 0x7f84940a6850 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:34.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f849409fa00 con 0x7f84940a6c30 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f84940a19c0 con 0x7f84940a6470 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f849a385640 10 monclient(hunting): authenticate will time out at 1960.126370s 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f84937fe640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f84940a6850 legacy=0x7f8494137f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84940a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84940a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38362/0 (socket says 10.20.193.78:38362) 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 1 -- 10.20.193.78:0/3853008050 learned_addr learned my addr 10.20.193.78:0/3853008050 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient(hunting): get_auth_request con 0x7f84940a6470 auth_method 0 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:34.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f84988fb640 1 --2- 10.20.193.78:0/3853008050 >> v2:10.20.193.115:3300/0 conn(0x7f84940a6c30 0x7f8494138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f84988fb640 10 monclient(hunting): get_auth_request con 0x7f84940a6c30 auth_method 0 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f84988fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f84988fb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f84988fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient(hunting): handle_auth_done global_id 264257 payload 274 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 1 -- 10.20.193.78:0/3853008050 >> v2:10.20.193.115:3300/0 conn(0x7f84940a6c30 msgr2=0x7f8494138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 1 --2- 10.20.193.78:0/3853008050 >> v2:10.20.193.115:3300/0 conn(0x7f84940a6c30 0x7f8494138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 1 -- 10.20.193.78:0/3853008050 >> v1:10.20.193.78:6789/0 conn(0x7f84940a6850 legacy=0x7f8494137f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:34.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient: _finish_hunting 0 2026-02-17T06:08:34.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 1 monclient: found mon.noname-b 2026-02-17T06:08:34.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:34.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:34.580 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.572+0000 7f8493fff640 1 -- 10.20.193.78:0/3853008050 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84940a0450 con 0x7f84940a6470 2026-02-17T06:08:34.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8493fff640 1 -- 10.20.193.78:0/3853008050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 msgr2=0x7f84940a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:34.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8493fff640 1 --2- 10.20.193.78:0/3853008050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84940a1480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8480009970 tx=0x7f848002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=691f507ae93de8a1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:34.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 1 -- 10.20.193.78:0/3853008050 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848003c070 con 0x7f84940a6470 2026-02-17T06:08:34.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:34.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:34.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: dump: 2026-02-17T06:08:34.586 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: _finish_auth 0 2026-02-17T06:08:34.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 1 -- 10.20.193.78:0/3853008050 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848002fcd0 con 0x7f84940a6470 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 1 -- 10.20.193.78:0/3853008050 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480037660 con 0x7f84940a6470 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 5 monclient: authenticate success, global_id 264257 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 4 set_mon_vals no callback set 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f8492ffd640 10 monclient: dump: 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:34.588 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 10 monclient: shutdown 2026-02-17T06:08:34.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 1 -- 10.20.193.78:0/3853008050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 msgr2=0x7f84940a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 1 --2- 10.20.193.78:0/3853008050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84940a1480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8480009970 tx=0x7f848002ee90 comp rx=0 tx=0).stop 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 1 -- 10.20.193.78:0/3853008050 shutdown_connections 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 1 --2- 10.20.193.78:0/3853008050 >> v2:10.20.193.115:3300/0 conn(0x7f84940a6c30 0x7f8494138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 1 --2- 10.20.193.78:0/3853008050 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84940a1480 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 1 -- 10.20.193.78:0/3853008050 >> 10.20.193.78:0/3853008050 conn(0x7f84940aae40 msgr2=0x7f84940ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 1 -- 10.20.193.78:0/3853008050 shutdown_connections 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 1 -- 10.20.193.78:0/3853008050 wait complete. 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 10 monclient: build_initial_monmap 2026-02-17T06:08:34.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.575+0000 7f849a385640 10 monclient: monmap: 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:34.591 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 1 Processor -- start 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 1 -- start start 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient: init 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84941cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient(hunting): picked mon.noname-c con 0x7f84940a6470 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f84940a6850 0x7f8494135190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient(hunting): picked mon.noname-a con 0x7f84940a6850 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f84940a6c30 0x7f84941356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:34.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient(hunting): picked mon.noname-b con 0x7f84940a6c30 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8494138ba0 con 0x7f84940a6850 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8494138d20 con 0x7f84940a6c30 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8494140290 con 0x7f84940a6470 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f849a385640 10 monclient(hunting): authenticate will time out at 1960.130386s 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f84937fe640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f84940a6850 msgr2=0x7f8494135190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84941cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f84937fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f84940a6850 0x7f8494135190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:34.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84941cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:38364/0 (socket says 10.20.193.78:38364) 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 -- 10.20.193.78:0/1661058703 learned_addr learned my addr 10.20.193.78:0/1661058703 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient(hunting): get_auth_request con 0x7f84940a6470 auth_method 0 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient(hunting): handle_auth_done global_id 264260 payload 824 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 -- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f84940a6c30 msgr2=0x7f84941356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f84940a6c30 0x7f84941356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.594 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 -- 10.20.193.78:0/1661058703 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f84940a6850 msgr2=0x7f8494135190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f84940a6850 0x7f8494135190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient: _finish_hunting 0 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 monclient: found mon.noname-c 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.576+0000 7f8493fff640 1 -- 10.20.193.78:0/1661058703 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8494133980 con 0x7f84940a6470 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8493fff640 1 -- 10.20.193.78:0/1661058703 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 msgr2=0x7f84941cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8493fff640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84941cf5b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8480009970 tx=0x7f8480004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 1 -- 10.20.193.78:0/1661058703 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848003c040 con 0x7f84940a6470 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:34.595 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: dump: 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: _finish_auth 0 2026-02-17T06:08:34.596 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 1 -- 10.20.193.78:0/1661058703 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8480047070 con 0x7f84940a6470 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f849a385640 5 monclient: authenticate success, global_id 264260 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f849a385640 10 monclient: _renew_subs 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f849a385640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8494138860 con 0x7f84940a6470 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 1 -- 10.20.193.78:0/1661058703 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480042890 con 0x7f84940a6470 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f84927fc640 4 set_mon_vals no callback set 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f8490ff9640 10 monclient: dump: 2026-02-17T06:08:34.597 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f849a385640 10 monclient: _renew_subs 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f849a385640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.578+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8494131970 con 0x7f84940a6470 2026-02-17T06:08:34.598 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.579+0000 7f849a385640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.579+0000 7f849a385640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.579+0000 7f849a385640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.579+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8494074980 con 0x7f84940a6470 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.579+0000 7f8490ff9640 1 -- 10.20.193.78:0/1661058703 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8480042a30 con 0x7f84940a6470 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.579+0000 7f8490ff9640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846403dee0 0x7f84640403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.579+0000 7f8490ff9640 1 -- 10.20.193.78:0/1661058703 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7f8480077560 con 0x7f84940a6470 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.579+0000 7f84937fe640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846403dee0 0x7f84640403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.580+0000 7f84937fe640 10 monclient: get_auth_request con 0x7f846403dee0 auth_method 0 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.582+0000 7f8490ff9640 1 -- 10.20.193.78:0/1661058703 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8480050020 con 0x7f84940a6470 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.582+0000 7f8490ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.582+0000 7f8490ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:34.599 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.582+0000 7f84937fe640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846403dee0 0x7f84640403a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f84941315b0 tx=0x7f8484009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:34.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.705+0000 7f849a385640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:08:34.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.705+0000 7f849a385640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:34.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.705+0000 7f849a385640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:34.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.705+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f84940a1480 con 0x7f84940a6470 2026-02-17T06:08:34.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.705+0000 7f8490ff9640 1 -- 10.20.193.78:0/1661058703 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f84940a0730 con 0x7f84940a6470 2026-02-17T06:08:34.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.705+0000 7f8490ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:34.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.705+0000 7f8490ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:34.707 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":252,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:08:34.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846403dee0 msgr2=0x7f84640403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:34.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846403dee0 0x7f84640403a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f84941315b0 tx=0x7f8484009270 comp rx=0 tx=0).stop 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 10 monclient: shutdown 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 msgr2=0x7f84941cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84941cf5b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8480009970 tx=0x7f8480004830 comp rx=0 tx=0).stop 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 shutdown_connections 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f846403dee0 0x7f84640403a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f84940a6c30 0x7f84941356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f84940a6850 0x7f8494135190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 --2- 10.20.193.78:0/1661058703 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f84940a6470 0x7f84941cf5b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 >> 10.20.193.78:0/1661058703 conn(0x7f84940aae40 msgr2=0x7f84940a97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:34.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 shutdown_connections 2026-02-17T06:08:34.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:34.706+0000 7f849a385640 1 -- 10.20.193.78:0/1661058703 wait complete. 2026-02-17T06:08:37.715 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:08:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.783+0000 7f1610a76640 10 monclient: get_monmap_and_config 2026-02-17T06:08:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.783+0000 7f1610a76640 10 monclient: build_initial_monmap 2026-02-17T06:08:37.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.783+0000 7f1610a76640 10 monclient: monmap: 2026-02-17T06:08:37.787 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:37.787 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:37.784882+0000 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:37.784882+0000 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:37.788 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 1 Processor -- start 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 1 -- start start 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient: init 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f160c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f160c09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c13ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient(hunting): picked mon.noname-b con 0x7f160c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f160c06daf0 con 0x7f160c09c4a0 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f160c09cc80 con 0x7f160c06c890 2026-02-17T06:08:37.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f160c0a0880 con 0x7f160c06d380 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1610a76640 10 monclient(hunting): authenticate will time out at 1963.338405s 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f1609d74640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f160c09c4a0 legacy=0x7f160c13b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c13ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c13ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57092/0 (socket says 10.20.193.78:57092) 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 1 -- 10.20.193.78:0/3245439214 learned_addr learned my addr 10.20.193.78:0/3245439214 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient(hunting): get_auth_request con 0x7f160c06d380 auth_method 0 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:37.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient(hunting): handle_auth_done global_id 264263 payload 274 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 1 -- 10.20.193.78:0/3245439214 >> v1:10.20.193.115:6789/0 conn(0x7f160c06c890 legacy=0x7f160c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 1 -- 10.20.193.78:0/3245439214 >> v1:10.20.193.78:6789/0 conn(0x7f160c09c4a0 legacy=0x7f160c13b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient: _finish_hunting 0 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 1 monclient: found mon.noname-b 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:08:37.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.784+0000 7f160ad76640 1 -- 10.20.193.78:0/3245439214 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c09cf60 con 0x7f160c06d380 2026-02-17T06:08:37.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f160ad76640 1 -- 10.20.193.78:0/3245439214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 msgr2=0x7f160c13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:37.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f160ad76640 1 --2- 10.20.193.78:0/3245439214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c13ead0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009e80 tx=0x7f15fc02f250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46eaecd8b71c8f6c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:37.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 1 -- 10.20.193.78:0/3245439214 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0373f0 con 0x7f160c06d380 2026-02-17T06:08:37.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: dump: 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:37.799 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: _finish_auth 0 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: _check_auth_tickets 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 1 -- 10.20.193.78:0/3245439214 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc037590 con 0x7f160c06d380 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 5 monclient: authenticate success, global_id 264263 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:37.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 1 -- 10.20.193.78:0/3245439214 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc037960 con 0x7f160c06d380 2026-02-17T06:08:37.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:37.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:37.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1609573640 10 monclient: dump: 2026-02-17T06:08:37.801 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:37.801 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:37.801 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:37.801 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:37.801 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 4 set_mon_vals no callback set 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 10 monclient: shutdown 2026-02-17T06:08:37.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:37.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 1 -- 10.20.193.78:0/3245439214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 msgr2=0x7f160c13ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:37.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 1 --2- 10.20.193.78:0/3245439214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c13ead0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009e80 tx=0x7f15fc02f250 comp rx=0 tx=0).stop 2026-02-17T06:08:37.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 1 -- 10.20.193.78:0/3245439214 shutdown_connections 2026-02-17T06:08:37.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 1 --2- 10.20.193.78:0/3245439214 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c13ead0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:37.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 1 -- 10.20.193.78:0/3245439214 >> 10.20.193.78:0/3245439214 conn(0x7f160c0a6620 msgr2=0x7f160c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:37.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 1 -- 10.20.193.78:0/3245439214 shutdown_connections 2026-02-17T06:08:37.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.787+0000 7f1610a76640 1 -- 10.20.193.78:0/3245439214 wait complete. 2026-02-17T06:08:37.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient: build_initial_monmap 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient: monmap: 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:37.804 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 1 Processor -- start 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 1 -- start start 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient: init 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f160c06c890 0x7f160c136100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient(hunting): picked mon.noname-a con 0x7f160c06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient(hunting): picked mon.noname-c con 0x7f160c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f160c09c4a0 0x7f160c09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:37.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient(hunting): picked mon.noname-b con 0x7f160c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f160c09bfa0 con 0x7f160c06c890 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f160c09be20 con 0x7f160c09c4a0 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f160c13f010 con 0x7f160c06d380 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1610a76640 10 monclient(hunting): authenticate will time out at 1963.342532s 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f160a575640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f160c06c890 msgr2=0x7f160c136100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f160a575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f160c06c890 0x7f160c136100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:37.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c136640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57096/0 (socket says 10.20.193.78:57096) 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 -- 10.20.193.78:0/1079177565 learned_addr learned my addr 10.20.193.78:0/1079177565 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient(hunting): get_auth_request con 0x7f160c06d380 auth_method 0 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient(hunting): handle_auth_done global_id 264266 payload 824 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 -- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f160c09c4a0 msgr2=0x7f160c09db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f160c09c4a0 0x7f160c09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:37.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 -- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f160c06c890 msgr2=0x7f160c136100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f160c06c890 0x7f160c136100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient: _finish_hunting 0 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 monclient: found mon.noname-c 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.788+0000 7f1609d74640 1 -- 10.20.193.78:0/1079177565 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c09d810 con 0x7f160c06d380 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.790+0000 7f1609d74640 1 -- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 msgr2=0x7f160c136640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.790+0000 7f1609d74640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c136640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b700 tx=0x7f15f400bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 1 -- 10.20.193.78:0/1079177565 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4005270 con 0x7f160c06d380 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:37.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: dump: 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:08:37.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 5 monclient: authenticate success, global_id 264266 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 1 -- 10.20.193.78:0/1079177565 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f4010070 con 0x7f160c06d380 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 10 monclient: _renew_subs 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c121840 con 0x7f160c06d380 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 1 -- 10.20.193.78:0/1079177565 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4005650 con 0x7f160c06d380 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 10 monclient: dump: 2026-02-17T06:08:37.810 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1608d72640 4 set_mon_vals no callback set 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 10 monclient: _renew_subs 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:37.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c121180 con 0x7f160c06d380 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160c0728f0 con 0x7f160c06d380 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.791+0000 7f15f2ffd640 1 -- 10.20.193.78:0/1079177565 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f15f401d030 con 0x7f160c06d380 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.792+0000 7f15f2ffd640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f15d803deb0 0x7f15d8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.792+0000 7f15f2ffd640 1 -- 10.20.193.78:0/1079177565 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7f15f4051d90 con 0x7f160c06d380 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.792+0000 7f160a575640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f15d803deb0 0x7f15d8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.792+0000 7f160a575640 10 monclient: get_auth_request con 0x7f15d803deb0 auth_method 0 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.794+0000 7f15f2ffd640 1 -- 10.20.193.78:0/1079177565 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f160c09d810 con 0x7f160c06d380 2026-02-17T06:08:37.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.794+0000 7f15f2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:37.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.794+0000 7f15f2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:37.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.795+0000 7f160a575640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f15d803deb0 0x7f15d8040370 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f16000037d0 tx=0x7f1600007b10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:37.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.914+0000 7f1610a76640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:08:37.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.914+0000 7f1610a76640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:37.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.914+0000 7f1610a76640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:37.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.914+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f160c06fd20 con 0x7f160c06d380 2026-02-17T06:08:37.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:37.914+0000 7f1609d74640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c136640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b700 tx=0x7f15f400bbd0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f15f400cd80 mon_command_ack([{"prefix": "quorum_status"}]=0 v0) 2026-02-17T06:08:38.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.742+0000 7f15f2ffd640 1 -- 10.20.193.78:0/1079177565 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f15f400cd80 con 0x7f160c06d380 2026-02-17T06:08:38.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.742+0000 7f15f2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:38.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.742+0000 7f15f2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:38.744 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":252,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f15d803deb0 msgr2=0x7f15d8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f15d803deb0 0x7f15d8040370 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f16000037d0 tx=0x7f1600007b10 comp rx=0 tx=0).stop 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 10 monclient: shutdown 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 msgr2=0x7f160c136640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c136640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b700 tx=0x7f15f400bbd0 comp rx=0 tx=0).stop 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 shutdown_connections 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f15d803deb0 0x7f15d8040370 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:38.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f160c09c4a0 0x7f160c09db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:38.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f160c06d380 0x7f160c136640 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:38.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 --2- 10.20.193.78:0/1079177565 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f160c06c890 0x7f160c136100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:38.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 >> 10.20.193.78:0/1079177565 conn(0x7f160c0a6620 msgr2=0x7f160c072e40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:38.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 shutdown_connections 2026-02-17T06:08:38.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:38.743+0000 7f1610a76640 1 -- 10.20.193.78:0/1079177565 wait complete. 2026-02-17T06:08:41.754 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:08:41.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient: get_monmap_and_config 2026-02-17T06:08:41.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient: build_initial_monmap 2026-02-17T06:08:41.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient: monmap: 2026-02-17T06:08:41.827 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:41.827 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:41.827 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:41.823454+0000 2026-02-17T06:08:41.827 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:41.823454+0000 2026-02-17T06:08:41.827 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:41.827 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:41.827 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:41.827 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:41.828 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:41.828 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:41.828 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:41.828 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:41.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 1 Processor -- start 2026-02-17T06:08:41.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 1 -- start start 2026-02-17T06:08:41.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient: init 2026-02-17T06:08:41.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:41.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2026-02-17T06:08:41.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f878c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:41.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 0x7f878c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:41.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient(hunting): picked mon.noname-a con 0x7f878c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:41.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:41.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient(hunting): picked mon.noname-b con 0x7f878c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:41.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f878c06daf0 con 0x7f878c06c890 2026-02-17T06:08:41.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:41.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f878c0a0880 con 0x7f878c09c4a0 2026-02-17T06:08:41.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:41.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f878c09cc80 con 0x7f878c06d380 2026-02-17T06:08:41.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:41.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.822+0000 7f8793a32640 10 monclient(hunting): authenticate will time out at 1967.376992s 2026-02-17T06:08:41.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8790fa6640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 msgr2=0x7f878c13b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:41.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8790fa6640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 0x7f878c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:41.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:41.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57104/0 (socket says 10.20.193.78:57104) 2026-02-17T06:08:41.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 learned_addr learned my addr 10.20.193.78:0/2154001263 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:41.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 10 monclient(hunting): get_auth_request con 0x7f878c06d380 auth_method 0 2026-02-17T06:08:41.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:41.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:41.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:41.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:08:41.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:41.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:41.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.823+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:41.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:41.825+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.115:6789/0 conn(0x7f878c06c890 legacy=0x7f878c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:42.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 msgr2=0x7f878c13b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:42.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 0x7f878c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:08:42.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:42.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 10 monclient(hunting): get_auth_request con 0x7f878c06d380 auth_method 0 2026-02-17T06:08:42.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:42.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:42.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:42.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:08:42.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:42.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:42.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.023+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:08:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.423+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 msgr2=0x7f878c13b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:42.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.423+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 0x7f878c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 10 monclient(hunting): get_auth_request con 0x7f878c06d380 auth_method 0 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:42.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:42.424+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:08:43.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.224+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 msgr2=0x7f878c13b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:43.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.224+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 0x7f878c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:08:43.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:43.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 10 monclient(hunting): get_auth_request con 0x7f878c06d380 auth_method 0 2026-02-17T06:08:43.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:43.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:43.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:43.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:08:43.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:43.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:43.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:43.225+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:08:44.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 10 monclient(hunting): tick 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 monclient(hunting): continuing hunt 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 msgr2=0x7f878c13d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c06d380 0x7f878c13d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 msgr2=0x7f878c13b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.78:3300/0 conn(0x7f878c09c4a0 0x7f878c13b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.115:6789/0 conn(0x7f878c06c890 legacy=0x7f878c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:44.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 10 monclient: _add_conns ranks=[0,4] 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8770003840 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 10 monclient(hunting): picked mon.noname-c con 0x7f8770007590 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f878c09cf60 con 0x7f8770003840 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f878c09cc80 con 0x7f8770007590 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f877affd640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f8770003840 legacy=0x7f8770005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 10 monclient(hunting): get_auth_request con 0x7f8770007590 auth_method 0 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:44.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:44.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:44.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:08:44.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:44.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:44.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:44.823+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:45.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.023+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f8770003840 legacy=0x7f8770005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:45.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:45.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 10 monclient(hunting): get_auth_request con 0x7f8770007590 auth_method 0 2026-02-17T06:08:45.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:45.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:45.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:45.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:08:45.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:45.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:45.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.024+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:08:45.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.423+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f8770003840 legacy=0x7f8770005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:45.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.424+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:45.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.424+0000 7f8790fa6640 10 monclient(hunting): get_auth_request con 0x7f8770007590 auth_method 0 2026-02-17T06:08:45.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.424+0000 7f8790fa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:45.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.424+0000 7f8790fa6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:45.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.424+0000 7f8790fa6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:45.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.425+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:08:45.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.425+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:45.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.425+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:45.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:45.425+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:08:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.224+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f8770003840 legacy=0x7f8770005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:46.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.224+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 DispatchQueue::entry inject delay of 0.002000 2026-02-17T06:08:46.226 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 10 monclient(hunting): get_auth_request con 0x7f8770007590 auth_method 0 2026-02-17T06:08:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:08:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:46.227 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:46.226+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient(hunting): tick 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 1 monclient(hunting): continuing hunt 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 msgr2=0x7f8770009a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 1 --2- 10.20.193.78:0/2154001263 >> v2:10.20.193.115:3300/0 conn(0x7f8770007590 0x7f8770009a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f8770003840 legacy=0x7f8770005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 reap_dead start 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f877000a0f0 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f877000ac80 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:47.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient(hunting): picked mon.noname-b con 0x7f8770009af0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f877000b1c0 con 0x7f877000ac80 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f878c09cf60 con 0x7f877000a0f0 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 1 -- 10.20.193.78:0/2154001263 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f878c09cc80 con 0x7f8770009af0 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f877affd640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f877000ac80 legacy=0x7f8770005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 10 monclient(hunting): get_auth_request con 0x7f8770009af0 auth_method 0 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f877000e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f877000e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:47.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.823+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:47.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:47.825+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.115:6789/0 conn(0x7f877000a0f0 legacy=0x7f877000a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f877000ac80 legacy=0x7f8770005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:48.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 10 monclient(hunting): get_auth_request con 0x7f8770009af0 auth_method 0 2026-02-17T06:08:48.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:48.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f877000e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:08:48.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f877000e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:48.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:48.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.023+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:08:48.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8790fa6640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f877000ac80 legacy=0x7f8770005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:48.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 10 monclient(hunting): get_auth_request con 0x7f8770009af0 auth_method 0 2026-02-17T06:08:48.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:48.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f877000e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:08:48.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f877000e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:08:48.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:08:48.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.424+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:08:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.453+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246175022 0 0) 0x7f878c09cf60 con 0x7f877000a0f0 2026-02-17T06:08:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.453+0000 7f877bfff640 10 monclient(hunting): _negotiate 2026-02-17T06:08:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.453+0000 7f877bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.453+0000 7f877bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.453+0000 7f877bfff640 10 monclient(hunting): my global_id is 274132 2026-02-17T06:08:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.453+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8774003620 con 0x7f877000a0f0 2026-02-17T06:08:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2992838315 0 0) 0x7f8774003620 con 0x7f877000a0f0 2026-02-17T06:08:48.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f877000e610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f877000e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.78:6789/0 conn(0x7f877000ac80 legacy=0x7f8770005c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8770010e10 con 0x7f877000a0f0 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: _finish_auth 0 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8793a32640 5 monclient: authenticate success, global_id 274132 2026-02-17T06:08:48.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8793a32640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f877c003250 con 0x7f877000a0f0 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: dump: 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.460 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 >> v1:10.20.193.115:6789/0 conn(0x7f877000a0f0 legacy=0x7f877000a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f8774003ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient(hunting): picked mon.c con 0x7f8770009af0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f87740040c0 0x7f8774004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient(hunting): picked mon.b con 0x7f87740040c0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:48.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8774004e40 0x7f87740052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient(hunting): picked mon.a con 0x7f8774004e40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f878c09cc80 con 0x7f8774004e40 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f877400bcb0 con 0x7f87740040c0 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f877400beb0 con 0x7f8770009af0 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f877bfff640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8793a32640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8774004e40 msgr2=0x7f87740052c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:48.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8791fa8640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8774004e40 0x7f87740052c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f8774003ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient(hunting): get_auth_request con 0x7f8770009af0 auth_method 0 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8790fa6640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f87740040c0 0x7f8774004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8790fa6640 10 monclient(hunting): get_auth_request con 0x7f87740040c0 auth_method 0 2026-02-17T06:08:48.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8790fa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8790fa6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8790fa6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8790fa6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8790fa6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f8790fa6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient(hunting): handle_auth_done global_id 274132 payload 293 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f87740040c0 msgr2=0x7f8774004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f87740040c0 0x7f8774004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8774004e40 msgr2=0x7f87740052c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8774004e40 0x7f87740052c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient: _finish_hunting 0 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 1 monclient: found mon.c 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8774003620 con 0x7f8770009af0 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient: _finish_auth 0 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 10 monclient: _check_auth_tickets 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.454+0000 7f87917a7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f87917a7640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f8774003ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f87917a7640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f8774003ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f877c002920 tx=0x7f877c02f5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2b333795b2e41f server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c00c0e0 con 0x7f8770009af0 2026-02-17T06:08:48.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:48.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 10 monclient: dump: 2026-02-17T06:08:48.472 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.472 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877c005ac0 con 0x7f8770009af0 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f8793a32640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:48.473 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 1 -- 10.20.193.78:0/2154001263 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c007070 con 0x7f8770009af0 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f8793a32640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f8793a32640 4 set_mon_vals no callback set 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.457+0000 7f877bfff640 10 monclient: dump: 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.474 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 10 monclient: shutdown 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 1 -- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 msgr2=0x7f8774003ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 1 --2- 10.20.193.78:0/2154001263 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8770009af0 0x7f8774003ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f877c002920 tx=0x7f877c02f5c0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8791fa8640 1 -- 10.20.193.78:0/2154001263 reap_dead start 2026-02-17T06:08:48.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 1 -- 10.20.193.78:0/2154001263 shutdown_connections 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 1 -- 10.20.193.78:0/2154001263 >> 10.20.193.78:0/2154001263 conn(0x7f878c0a6620 msgr2=0x7f878c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 1 -- 10.20.193.78:0/2154001263 shutdown_connections 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 1 -- 10.20.193.78:0/2154001263 wait complete. 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 10 monclient: build_initial_monmap 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 10 monclient: monmap: 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.476 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 1 Processor -- start 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 1 -- start start 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.458+0000 7f8793a32640 10 monclient: init 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f878c1cf350 0x7f878c1cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient(hunting): picked mon.noname-a con 0x7f878c1cf350 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f878c1cfc70 0x7f878c09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.477 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient(hunting): picked mon.noname-b con 0x7f878c1cfc70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c09e060 0x7f878c09e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient(hunting): picked mon.noname-c con 0x7f878c09e060 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f878c09bfa0 con 0x7f878c1cf350 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f878c09be20 con 0x7f878c1cfc70 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f878c06cc70 con 0x7f878c09e060 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8793a32640 10 monclient(hunting): authenticate will time out at 1974.013157s 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f87917a7640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f878c1cf350 msgr2=0x7f878c1cf730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:48.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f87917a7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f878c1cf350 0x7f878c1cf730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c09e060 0x7f878c09e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c09e060 0x7f878c09e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57098/0 (socket says 10.20.193.78:57098) 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 -- 10.20.193.78:0/1630260055 learned_addr learned my addr 10.20.193.78:0/1630260055 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient(hunting): get_auth_request con 0x7f878c09e060 auth_method 0 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8790fa6640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f878c1cfc70 0x7f878c09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8790fa6640 10 monclient(hunting): get_auth_request con 0x7f878c1cfc70 auth_method 0 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8790fa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8790fa6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8790fa6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient(hunting): handle_auth_done global_id 264269 payload 824 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 -- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f878c1cfc70 msgr2=0x7f878c09db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f878c1cfc70 0x7f878c09db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 -- 10.20.193.78:0/1630260055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f878c1cf350 msgr2=0x7f878c1cf730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f878c1cf350 0x7f878c1cf730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient: _finish_hunting 0 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 monclient: found mon.noname-c 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.459+0000 7f8791fa8640 1 -- 10.20.193.78:0/1630260055 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c129170 con 0x7f878c09e060 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8791fa8640 1 -- 10.20.193.78:0/1630260055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c09e060 msgr2=0x7f878c09e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8791fa8640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c09e060 0x7f878c09e4c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8770007500 tx=0x7f8788018e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 1 -- 10.20.193.78:0/1630260055 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878800b4f0 con 0x7f878c09e060 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: dump: 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.481 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: _finish_auth 0 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 1 -- 10.20.193.78:0/1630260055 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788016870 con 0x7f878c09e060 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 1 -- 10.20.193.78:0/1630260055 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788016c20 con 0x7f878c09e060 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8779ffb640 10 monclient: dump: 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f877b7fe640 4 set_mon_vals no callback set 2026-02-17T06:08:48.483 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8793a32640 5 monclient: authenticate success, global_id 264269 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8793a32640 10 monclient: _renew_subs 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8793a32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c129730 con 0x7f878c09e060 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8793a32640 10 monclient: _renew_subs 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8793a32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.461+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c128e90 con 0x7f878c09e060 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f8779ffb640 1 -- 10.20.193.78:0/1630260055 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8788009700 con 0x7f878c09e060 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f8793a32640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f8793a32640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f8793a32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c136ea0 con 0x7f878c09e060 2026-02-17T06:08:48.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f8779ffb640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f875c03deb0 0x7f875c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f8779ffb640 1 -- 10.20.193.78:0/1630260055 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7f8788053af0 con 0x7f878c09e060 2026-02-17T06:08:48.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f87917a7640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f875c03deb0 0x7f875c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.462+0000 7f87917a7640 10 monclient: get_auth_request con 0x7f875c03deb0 auth_method 0 2026-02-17T06:08:48.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.464+0000 7f8779ffb640 1 -- 10.20.193.78:0/1630260055 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f878c136ea0 con 0x7f878c09e060 2026-02-17T06:08:48.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.464+0000 7f8779ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:48.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.464+0000 7f8779ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:48.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.465+0000 7f87917a7640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f875c03deb0 0x7f875c040370 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f877c033000 tx=0x7f877c009070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.577+0000 7f8793a32640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:08:48.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.577+0000 7f8793a32640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:48.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.577+0000 7f8793a32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.577+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f878c06cfa0 con 0x7f878c09e060 2026-02-17T06:08:48.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.577+0000 7f8779ffb640 1 -- 10.20.193.78:0/1630260055 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f8788052500 con 0x7f878c09e060 2026-02-17T06:08:48.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.577+0000 7f8779ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:08:48.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.577+0000 7f8779ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:48.579 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":254,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:08:48.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f875c03deb0 msgr2=0x7f875c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f875c03deb0 0x7f875c040370 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f877c033000 tx=0x7f877c009070 comp rx=0 tx=0).stop 2026-02-17T06:08:48.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 10 monclient: shutdown 2026-02-17T06:08:48.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:48.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c09e060 msgr2=0x7f878c09e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c09e060 0x7f878c09e4c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8770007500 tx=0x7f8788018e00 comp rx=0 tx=0).stop 2026-02-17T06:08:48.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 shutdown_connections 2026-02-17T06:08:48.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f878c09e060 0x7f878c09e4c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f875c03deb0 0x7f875c040370 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f878c1cfc70 0x7f878c09db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 --2- 10.20.193.78:0/1630260055 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f878c1cf350 0x7f878c1cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 >> 10.20.193.78:0/1630260055 conn(0x7f878c0a6620 msgr2=0x7f878c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:48.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 shutdown_connections 2026-02-17T06:08:48.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.578+0000 7f8793a32640 1 -- 10.20.193.78:0/1630260055 wait complete. 2026-02-17T06:08:48.587 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:08:48.588 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:08:48.588 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient: get_monmap_and_config 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient: build_initial_monmap 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient: monmap: 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:48.667429+0000 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:48.667429+0000 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:48.667 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 1 Processor -- start 2026-02-17T06:08:48.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 1 -- start start 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient: init 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f814806c890 0x7f814806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient(hunting): picked mon.noname-c con 0x7f814806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f814809c4a0 0x7f814806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient(hunting): picked mon.noname-a con 0x7f814809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8148134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f814806daf0 con 0x7f8148134df0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.666+0000 7f814efb6640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f81480a0880 con 0x7f814809c4a0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814efb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814efb6640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f814806d6f0 con 0x7f814806c890 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814efb6640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814efb6640 10 monclient(hunting): authenticate will time out at 1974.221014s 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f813ffff640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f814809c4a0 msgr2=0x7f814806d1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f813ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f814809c4a0 0x7f814806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f814806c890 0x7f814806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f814806c890 0x7f814806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:47884/0 (socket says 10.20.193.78:47884) 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 1 -- 10.20.193.78:0/380949448 learned_addr learned my addr 10.20.193.78:0/380949448 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient(hunting): get_auth_request con 0x7f814806c890 auth_method 0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient(hunting): handle_auth_done global_id 274138 payload 274 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 1 -- 10.20.193.78:0/380949448 >> v2:10.20.193.78:3300/0 conn(0x7f814809c4a0 msgr2=0x7f814806d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 1 --2- 10.20.193.78:0/380949448 >> v2:10.20.193.78:3300/0 conn(0x7f814809c4a0 0x7f814806d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 1 -- 10.20.193.78:0/380949448 >> v1:10.20.193.115:6789/0 conn(0x7f8148134df0 legacy=0x7f8148135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient: _finish_hunting 0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 1 monclient: found mon.noname-c 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:08:48.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.667+0000 7f814cd2b640 1 -- 10.20.193.78:0/380949448 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814809cc80 con 0x7f814806c890 2026-02-17T06:08:48.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814cd2b640 1 -- 10.20.193.78:0/380949448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f814806c890 msgr2=0x7f814806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814cd2b640 1 --2- 10.20.193.78:0/380949448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f814806c890 0x7f814806cc70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f814806a480 tx=0x7f813802ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=522fa774a4786013 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 1 -- 10.20.193.78:0/380949448 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813803c070 con 0x7f814806c890 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: dump: 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 1 -- 10.20.193.78:0/380949448 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813802fda0 con 0x7f814806c890 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 5 monclient: authenticate success, global_id 274138 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:48.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 1 -- 10.20.193.78:0/380949448 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138037660 con 0x7f814806c890 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 4 set_mon_vals no callback set 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f813f7fe640 10 monclient: dump: 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 10 monclient: shutdown 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 1 -- 10.20.193.78:0/380949448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f814806c890 msgr2=0x7f814806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 1 --2- 10.20.193.78:0/380949448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f814806c890 0x7f814806cc70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f814806a480 tx=0x7f813802ee80 comp rx=0 tx=0).stop 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 1 -- 10.20.193.78:0/380949448 shutdown_connections 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 1 --2- 10.20.193.78:0/380949448 >> v2:10.20.193.78:3300/0 conn(0x7f814809c4a0 0x7f814806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 1 --2- 10.20.193.78:0/380949448 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f814806c890 0x7f814806cc70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 1 -- 10.20.193.78:0/380949448 >> 10.20.193.78:0/380949448 conn(0x7f81480a6620 msgr2=0x7f81480a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 1 -- 10.20.193.78:0/380949448 shutdown_connections 2026-02-17T06:08:48.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.670+0000 7f814efb6640 1 -- 10.20.193.78:0/380949448 wait complete. 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient: build_initial_monmap 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient: monmap: 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 1 Processor -- start 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 1 -- start start 2026-02-17T06:08:48.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient: init 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f814806c890 0x7f81481cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient(hunting): picked mon.noname-a con 0x7f814806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809c4a0 0x7f814809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient(hunting): picked mon.noname-c con 0x7f814809c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8148134df0 0x7f814809e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient(hunting): picked mon.noname-b con 0x7f8148134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81480a01a0 con 0x7f814806c890 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f81480a0020 con 0x7f8148134df0 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f814809be20 con 0x7f814809c4a0 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814efb6640 10 monclient(hunting): authenticate will time out at 1974.225735s 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814cd2b640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f814806c890 msgr2=0x7f81481cf7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814cd2b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f814806c890 0x7f81481cf7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809c4a0 0x7f814809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809c4a0 0x7f814809db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57108/0 (socket says 10.20.193.78:57108) 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 1 -- 10.20.193.78:0/1900199645 learned_addr learned my addr 10.20.193.78:0/1900199645 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 10 monclient(hunting): get_auth_request con 0x7f814809c4a0 auth_method 0 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f813ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814d52c640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8148134df0 0x7f814809e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814d52c640 10 monclient(hunting): get_auth_request con 0x7f8148134df0 auth_method 0 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814d52c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814d52c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.671+0000 7f814d52c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 10 monclient(hunting): handle_auth_done global_id 264272 payload 824 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8148134df0 msgr2=0x7f814809e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8148134df0 0x7f814809e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f814806c890 msgr2=0x7f81481cf7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f814806c890 0x7f81481cf7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:08:48.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 1 monclient: found mon.noname-c 2026-02-17T06:08:48.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:48.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.672+0000 7f813ffff640 1 -- 10.20.193.78:0/1900199645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8148127560 con 0x7f814809c4a0 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813ffff640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809c4a0 msgr2=0x7f814809db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813ffff640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809c4a0 0x7f814809db50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f813000b780 tx=0x7f813000bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130005270 con 0x7f814809c4a0 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: dump: 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: _finish_auth 0 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 5 monclient: authenticate success, global_id 264272 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8130010070 con 0x7f814809c4a0 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130005650 con 0x7f814809c4a0 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813effd640 4 set_mon_vals no callback set 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 10 monclient: dump: 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.676 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 10 monclient: _renew_subs 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81480a16a0 con 0x7f814809c4a0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 10 monclient: _renew_subs 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f814809f7f0 con 0x7f814809c4a0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81481cfce0 0x7f814809e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8148135a20 con 0x7f81481cfce0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f813000c520 con 0x7f814809c4a0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814cd2b640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81481cfce0 0x7f814809e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814cd2b640 10 monclient: get_auth_request con 0x7f81481cfce0 auth_method 0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814cd2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814cd2b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814cd2b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.674+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.675+0000 7f814cd2b640 10 monclient(hunting): handle_auth_done global_id 264275 payload 824 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.675+0000 7f814cd2b640 1 -- 10.20.193.78:0/1900199645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8148127120 con 0x7f81481cfce0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.675+0000 7f813d7fa640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f811803de90 0x7f8118040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.675+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (secure 0 0 0) 0x7f8130051fd0 con 0x7f814809c4a0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.675+0000 7f814d52c640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f811803de90 0x7f8118040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.675+0000 7f814d52c640 10 monclient: get_auth_request con 0x7f811803de90 auth_method 0 2026-02-17T06:08:48.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f814cd2b640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81481cfce0 msgr2=0x7f814809e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f814cd2b640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81481cfce0 0x7f814809e5d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8148074de0 tx=0x7f81380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138047020 con 0x7f81481cfce0 2026-02-17T06:08:48.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 10 monclient: dump: 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f8138037d70 con 0x7f81481cfce0 2026-02-17T06:08:48.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:48.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:48.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81481cfce0 msgr2=0x7f814809e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81481cfce0 0x7f814809e5d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8148074de0 tx=0x7f81380047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81481cfce0 msgr2=0x7f814809e5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:48.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f813d7fa640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f81481cfce0 0x7f814809e5d0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8148074de0 tx=0x7f81380047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.677+0000 7f814d52c640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f811803de90 0x7f8118040350 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f814400e160 tx=0x7f8144009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814efb6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:08:48.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814efb6640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809eb10 0x7f81481da430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814efb6640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f814809ef20 con 0x7f814809eb10 2026-02-17T06:08:48.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809eb10 0x7f81481da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 10 monclient: get_auth_request con 0x7f814809eb10 auth_method 0 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 10 monclient(hunting): handle_auth_done global_id 264278 payload 824 2026-02-17T06:08:48.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.680+0000 7f814cd2b640 1 -- 10.20.193.78:0/1900199645 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f81480008d0 con 0x7f814809eb10 2026-02-17T06:08:48.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f814cd2b640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809eb10 msgr2=0x7f81481da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f814cd2b640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809eb10 0x7f81481da430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8148135ba0 tx=0x7f8138009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138047040 con 0x7f814809eb10 2026-02-17T06:08:48.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:48.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 10 monclient: dump: 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7f813804eb80 con 0x7f814809eb10 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809eb10 msgr2=0x7f81481da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809eb10 0x7f81481da430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8148135ba0 tx=0x7f8138009340 comp rx=0 tx=0).stop 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809eb10 msgr2=0x7f81481da430 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.682+0000 7f813d7fa640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809eb10 0x7f81481da430 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8148135ba0 tx=0x7f8138009340 comp rx=0 tx=0).stop 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 254, 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 78067, 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:08:48.686 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:08:48.687 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.687 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:08:48.687 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:48.687 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:08:48.687 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:08:48.687 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:08:48.688 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:08:48.689 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:48.690 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.691 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:48.692 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.693 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.694 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "num": 6 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.695 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:48.696 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:48.696 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:48.696 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.696 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.696 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.696 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:08:48.696 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f811803de90 msgr2=0x7f8118040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f811803de90 0x7f8118040350 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f814400e160 tx=0x7f8144009040 comp rx=0 tx=0).stop 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 10 monclient: shutdown 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809c4a0 msgr2=0x7f814809db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 1 --2- 10.20.193.78:0/1900199645 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f814809c4a0 0x7f814809db50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f813000b780 tx=0x7f813000bc50 comp rx=0 tx=0).stop 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814d52c640 1 -- 10.20.193.78:0/1900199645 reap_dead start 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814d52c640 1 -- 10.20.193.78:0/1900199645 reap_dead start 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 shutdown_connections 2026-02-17T06:08:48.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 >> 10.20.193.78:0/1900199645 conn(0x7f81480a6620 msgr2=0x7f8148071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:48.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 shutdown_connections 2026-02-17T06:08:48.698 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.683+0000 7f814efb6640 1 -- 10.20.193.78:0/1900199645 wait complete. 2026-02-17T06:08:48.698 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:08:48.698 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-17T06:08:48.698 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:08:48.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.768+0000 7f969836a640 10 monclient: get_monmap_and_config 2026-02-17T06:08:48.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.768+0000 7f969836a640 10 monclient: build_initial_monmap 2026-02-17T06:08:48.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.768+0000 7f969836a640 10 monclient: monmap: 2026-02-17T06:08:48.769 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:48.769 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:08:48.769571+0000 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:08:48.769571+0000 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.768+0000 7f969836a640 1 Processor -- start 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.768+0000 7f969836a640 1 -- start start 2026-02-17T06:08:48.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.768+0000 7f969836a640 10 monclient: init 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient: _add_conns ranks=[3,4,0,2,1] 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f969006c890 0x7f969006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient(hunting): picked mon.noname-a con 0x7f969006c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f969009c4a0 0x7f969006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient(hunting): picked mon.noname-c con 0x7f969009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9690134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f969006daf0 con 0x7f9690134df0 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f96900a0880 con 0x7f969006c890 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f969006d6f0 con 0x7f969009c4a0 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f969836a640 10 monclient(hunting): authenticate will time out at 1974.323112s 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96960df640 1 -- >> v2:10.20.193.78:3300/0 conn(0x7f969006c890 msgr2=0x7f969006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96960df640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f969006c890 0x7f969006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96968e0640 1 -- >> v1:10.20.193.78:6789/0 conn(0x7f9690134df0 legacy=0x7f9690135280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.78:6789/0 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f969009c4a0 0x7f969006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f969009c4a0 0x7f969006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.115:3300/0 says I am v2:10.20.193.78:47910/0 (socket says 10.20.193.78:47910) 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 1 -- 10.20.193.78:0/3488050619 learned_addr learned my addr 10.20.193.78:0/3488050619 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient(hunting): get_auth_request con 0x7f969009c4a0 auth_method 0 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient(hunting): handle_auth_done global_id 274144 payload 274 2026-02-17T06:08:48.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 1 -- 10.20.193.78:0/3488050619 >> v2:10.20.193.78:3300/0 conn(0x7f969006c890 msgr2=0x7f969006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:48.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 1 --2- 10.20.193.78:0/3488050619 >> v2:10.20.193.78:3300/0 conn(0x7f969006c890 0x7f969006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 1 -- 10.20.193.78:0/3488050619 >> v1:10.20.193.78:6789/0 conn(0x7f9690134df0 legacy=0x7f9690135280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:08:48.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient: _finish_hunting 0 2026-02-17T06:08:48.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 1 monclient: found mon.noname-c 2026-02-17T06:08:48.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:48.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:08:48.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.769+0000 7f96958de640 1 -- 10.20.193.78:0/3488050619 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969009cc80 con 0x7f969009c4a0 2026-02-17T06:08:48.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.771+0000 7f96958de640 1 -- 10.20.193.78:0/3488050619 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009c4a0 msgr2=0x7f969006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.771+0000 7f96958de640 1 --2- 10.20.193.78:0/3488050619 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009c4a0 0x7f969006d1b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f969009c980 tx=0x7f968402ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74282cdba2930957 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 1 -- 10.20.193.78:0/3488050619 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968403c070 con 0x7f969009c4a0 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: dump: 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: _finish_auth 0 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 1 -- 10.20.193.78:0/3488050619 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968402fda0 con 0x7f969009c4a0 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 1 -- 10.20.193.78:0/3488050619 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684037660 con 0x7f969009c4a0 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f96950dd640 10 monclient: dump: 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.780 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 5 monclient: authenticate success, global_id 274144 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 10 monclient: get_monmap_and_config success 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 4 set_mon_vals no callback set 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 10 monclient: shutdown 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 1 -- 10.20.193.78:0/3488050619 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009c4a0 msgr2=0x7f969006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 1 --2- 10.20.193.78:0/3488050619 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009c4a0 0x7f969006d1b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f969009c980 tx=0x7f968402ee80 comp rx=0 tx=0).stop 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 1 -- 10.20.193.78:0/3488050619 shutdown_connections 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 1 --2- 10.20.193.78:0/3488050619 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009c4a0 0x7f969006d1b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 1 --2- 10.20.193.78:0/3488050619 >> v2:10.20.193.78:3300/0 conn(0x7f969006c890 0x7f969006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 1 -- 10.20.193.78:0/3488050619 >> 10.20.193.78:0/3488050619 conn(0x7f96900a6620 msgr2=0x7f96900a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:48.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 1 -- 10.20.193.78:0/3488050619 shutdown_connections 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 1 -- 10.20.193.78:0/3488050619 wait complete. 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 10 monclient: build_initial_monmap 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.772+0000 7f969836a640 10 monclient: monmap: 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 1 Processor -- start 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 1 -- start start 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient: init 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient: _reopen_session rank -1 2026-02-17T06:08:48.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f969006c890 0x7f96901cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient(hunting): picked mon.noname-a con 0x7f969006c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f969009c4a0 0x7f969009db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient(hunting): picked mon.noname-c con 0x7f969009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9690134df0 0x7f969009e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient(hunting): picked mon.noname-b con 0x7f9690134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96900a01a0 con 0x7f969006c890 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f96900a0020 con 0x7f9690134df0 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f969009be20 con 0x7f969009c4a0 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient(hunting): _renew_subs 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f969836a640 10 monclient(hunting): authenticate will time out at 1974.327423s 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96960df640 1 -- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f969006c890 msgr2=0x7f96901cf7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.78:3300/0 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96960df640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f969006c890 0x7f96901cf7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f969009c4a0 0x7f969009db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f969009c4a0 0x7f969009db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:57132/0 (socket says 10.20.193.78:57132) 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 -- 10.20.193.78:0/2772121033 learned_addr learned my addr 10.20.193.78:0/2772121033 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient(hunting): get_auth_request con 0x7f969009c4a0 auth_method 0 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96968e0640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9690134df0 0x7f969009e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient(hunting): handle_auth_done global_id 264281 payload 824 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9690134df0 msgr2=0x7f969009e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9690134df0 0x7f969009e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f969006c890 msgr2=0x7f96901cf7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f969006c890 0x7f96901cf7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient: _finish_hunting 0 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 monclient: found mon.noname-c 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96958de640 1 -- 10.20.193.78:0/2772121033 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96900727c0 con 0x7f969009c4a0 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96968e0640 10 monclient: get_auth_request con 0x7f9690134df0 auth_method 0 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.773+0000 7f96968e0640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9690134df0 0x7f969009e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f96958de640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f969009c4a0 msgr2=0x7f969009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f96958de640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f969009c4a0 0x7f969009db50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9684009a70 tx=0x7f96840047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684046020 con 0x7f969009c4a0 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f96958de640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f969009c4a0 0x7f969009db50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9684009a70 tx=0x7f96840047c0 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f968403c040 mon_map magic: 0 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 10 monclient: dump: 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 10 monclient: _finish_auth 0 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:08:48.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f969836a640 5 monclient: authenticate success, global_id 264281 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9684042640 con 0x7f969009c4a0 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f9676ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f969836a640 10 monclient: _renew_subs 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f969836a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96900a1840 con 0x7f969009c4a0 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.775+0000 7f96948dc640 4 set_mon_vals no callback set 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f969836a640 10 monclient: _renew_subs 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f969836a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9690127520 con 0x7f969009c4a0 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f969836a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f969836a640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f96901cfce0 0x7f969009e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f969836a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f969009f800 con 0x7f96901cfce0 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968403c040 con 0x7f969009c4a0 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f9676ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f9676ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f9676ffd640 10 monclient: dump: 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.786 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f968403f030 con 0x7f969009c4a0 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f96960df640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f96901cfce0 0x7f969009e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f96960df640 10 monclient: get_auth_request con 0x7f96901cfce0 auth_method 0 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f96960df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f96960df640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f96960df640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f9676ffd640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f966003de60 0x7f9660040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f96960df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f96960df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.776+0000 7f96960df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.777+0000 7f96960df640 10 monclient(hunting): handle_auth_done global_id 274147 payload 824 2026-02-17T06:08:48.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.777+0000 7f96968e0640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f966003de60 0x7f9660040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.777+0000 7f96960df640 1 -- 10.20.193.78:0/2772121033 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9690135a20 con 0x7f96901cfce0 2026-02-17T06:08:48.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.777+0000 7f96968e0640 10 monclient: get_auth_request con 0x7f966003de60 auth_method 0 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f96960df640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f96901cfce0 msgr2=0x7f969009e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f96960df640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f96901cfce0 0x7f969009e5d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f968000bcc0 tx=0x7f968000bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680005070 con 0x7f96901cfce0 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 10 monclient: dump: 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f9680005070 con 0x7f96901cfce0 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f96968e0640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f966003de60 0x7f9660040320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f96900736b0 tx=0x7f968c0073c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f96901cfce0 msgr2=0x7f969009e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f96901cfce0 0x7f969009e5d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f968000bcc0 tx=0x7f968000bcf0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f96901cfce0 msgr2=0x7f969009e5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.779+0000 7f9676ffd640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f96901cfce0 0x7f969009e5d0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f968000bcc0 tx=0x7f968000bcf0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f969836a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f969836a640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009eb10 0x7f96901da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f969836a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f969009d760 con 0x7f969009eb10 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f96960df640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009eb10 0x7f96901da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f96960df640 10 monclient: get_auth_request con 0x7f969009eb10 auth_method 0 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f96960df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f96960df640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:08:48.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.782+0000 7f96960df640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.783+0000 7f96960df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.783+0000 7f96960df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.783+0000 7f96960df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.783+0000 7f96960df640 10 monclient(hunting): handle_auth_done global_id 274150 payload 824 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.783+0000 7f96960df640 1 -- 10.20.193.78:0/2772121033 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f96900744d0 con 0x7f969009eb10 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f96960df640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009eb10 msgr2=0x7f96901da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f96960df640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009eb10 0x7f96901da3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f968000bcc0 tx=0x7f9680005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968001a730 con 0x7f969009eb10 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 10 monclient: dump: 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4699 (secure 0 0 0) 0x7f9680014430 con 0x7f969009eb10 2026-02-17T06:08:48.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009eb10 msgr2=0x7f96901da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009eb10 0x7f96901da3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f968000bcc0 tx=0x7f9680005ad0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009eb10 msgr2=0x7f96901da3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f969009eb10 0x7f96901da3c0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f968000bcc0 tx=0x7f9680005ad0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 254, 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 30787, 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.793 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:48.794 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:08:48.795 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:48.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.797 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.798 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.799 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:08:48.800 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f966003de60 msgr2=0x7f9660040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f969836a640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f966003de60 0x7f9660040320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f96900736b0 tx=0x7f968c0073c0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f969836a640 10 monclient: shutdown 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f969836a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f969009c4a0 msgr2=0x7f969009db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f969836a640 1 --2- 10.20.193.78:0/2772121033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f969009c4a0 0x7f969009db50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9684009a70 tx=0x7f96840047c0 comp rx=0 tx=0).stop 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f96968e0640 1 -- 10.20.193.78:0/2772121033 reap_dead start 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f9676ffd640 1 -- 10.20.193.78:0/2772121033 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(791..791 src has 525..791) ==== 4763+0+0 (unknown 0 0 0) 0x7f9684077bf0 con 0x7f969009c4a0 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.785+0000 7f96968e0640 1 -- 10.20.193.78:0/2772121033 reap_dead start 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.786+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 shutdown_connections 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.786+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 >> 10.20.193.78:0/2772121033 conn(0x7f96900a6620 msgr2=0x7f96900a1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.786+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 shutdown_connections 2026-02-17T06:08:48.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:08:48.786+0000 7f969836a640 1 -- 10.20.193.78:0/2772121033 wait complete. 2026-02-17T06:08:48.803 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:08:48.803 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:08:49.455 INFO:tasks.rados.rados.0.trial115.stdout:3603: done (0 left) 2026-02-17T06:08:49.455 INFO:tasks.rados.rados.0.trial115.stdout:3604: write oid 49 current snap is 383 2026-02-17T06:08:49.456 INFO:tasks.rados.rados.0.trial115.stdout:3604: seq_num 1238 ranges {407066=407448,1594943=620755} 2026-02-17T06:08:49.458 INFO:tasks.rados.rados.0.trial115.stdout:3604: writing trial11545420-49 from 407066 to 814514 tid 1 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:3604: writing trial11545420-49 from 1594943 to 2215698 tid 2 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:3605: rollback oid 48 current snap is 383 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 48 to 382 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:3604: finishing write tid 1 to trial11545420-49 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:3606: read oid 32 snap 376 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:3606: expect (ObjNum 1195 snap 366 seq_num 1195) 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:3607: rollback oid 29 current snap is 383 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 380 2026-02-17T06:08:49.465 INFO:tasks.rados.rados.0.trial115.stdout:3608: copy_from oid 8 from oid 31 current snap is 383 2026-02-17T06:08:49.466 INFO:tasks.rados.rados.0.trial115.stdout:3609: write (excl) oid 30 current snap is 383 2026-02-17T06:08:49.466 INFO:tasks.rados.rados.0.trial115.stdout:3609: seq_num 1239 ranges {423873=482697,1653425=480766,2691441=132432} 2026-02-17T06:08:49.467 INFO:tasks.rados.rados.0.trial115.stdout:3609: writing trial11545420-30 from 423873 to 906570 tid 1 2026-02-17T06:08:49.472 INFO:tasks.rados.rados.0.trial115.stdout:3609: writing trial11545420-30 from 1653425 to 2134191 tid 2 2026-02-17T06:08:49.475 INFO:tasks.rados.rados.0.trial115.stdout:3609: writing trial11545420-30 from 2691441 to 2823873 tid 3 2026-02-17T06:08:49.475 INFO:tasks.rados.rados.0.trial115.stdout:3610: snap_create 2026-02-17T06:08:49.475 INFO:tasks.rados.rados.0.trial115.stdout:3607: finishing rollback tid 0 to trial11545420-29 2026-02-17T06:08:49.475 INFO:tasks.rados.rados.0.trial115.stdout:3607: finishing rollback tid 1 to trial11545420-29 2026-02-17T06:08:49.475 INFO:tasks.rados.rados.0.trial115.stdout:3607: finishing rollback tid 2 to trial11545420-29 2026-02-17T06:08:49.475 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1492 (ObjNum 1230 snap 380 seq_num 1230) dirty exists 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3605: finishing rollback tid 1 to trial11545420-48 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3604: finishing write tid 2 to trial11545420-49 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3604: finishing write tid 3 to trial11545420-49 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3604: finishing write tid 4 to trial11545420-49 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3604: oid 49 updating version 0 to 1492 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3604: oid 49 updating version 1492 to 1493 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3604: oid 49 updating version 1493 to 1494 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1494 (ObjNum 1238 snap 383 seq_num 1238) dirty exists 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3604: left oid 49 (ObjNum 1238 snap 383 seq_num 1238) 2026-02-17T06:08:49.476 INFO:tasks.rados.rados.0.trial115.stdout:3606: expect (ObjNum 1195 snap 366 seq_num 1195) 2026-02-17T06:08:49.568 INFO:tasks.rados.rados.0.trial115.stdout:3608: finishing copy_from racing read to trial11545420-8 2026-02-17T06:08:49.568 INFO:tasks.rados.rados.0.trial115.stdout:3609: finishing write tid 1 to trial11545420-30 2026-02-17T06:08:49.568 INFO:tasks.rados.rados.0.trial115.stdout:3608: finishing copy_from to trial11545420-8 2026-02-17T06:08:49.568 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1534 (ObjNum 1162 snap 355 seq_num 1162) dirty exists 2026-02-17T06:08:49.568 INFO:tasks.rados.rados.0.trial115.stdout:3609: finishing write tid 2 to trial11545420-30 2026-02-17T06:08:49.568 INFO:tasks.rados.rados.0.trial115.stdout:3609: finishing write tid 3 to trial11545420-30 2026-02-17T06:08:49.568 INFO:tasks.rados.rados.0.trial115.stdout:3609: finishing write tid 4 to trial11545420-30 2026-02-17T06:08:49.568 INFO:tasks.rados.rados.0.trial115.stdout:3609: finishing write tid 5 to trial11545420-30 2026-02-17T06:08:49.569 INFO:tasks.rados.rados.0.trial115.stdout:3609: oid 30 updating version 0 to 1495 2026-02-17T06:08:49.569 INFO:tasks.rados.rados.0.trial115.stdout:3609: oid 30 version 1495 is already newer than 1492 2026-02-17T06:08:49.569 INFO:tasks.rados.rados.0.trial115.stdout:3609: oid 30 version 1495 is already newer than 1493 2026-02-17T06:08:49.569 INFO:tasks.rados.rados.0.trial115.stdout:3609: oid 30 version 1495 is already newer than 1494 2026-02-17T06:08:49.569 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1495 (ObjNum 1239 snap 383 seq_num 1239) dirty exists 2026-02-17T06:08:49.569 INFO:tasks.rados.rados.0.trial115.stdout:3609: left oid 30 (ObjNum 1239 snap 383 seq_num 1239) 2026-02-17T06:08:50.461 INFO:tasks.rados.rados.0.trial115.stdout:3604: done (6 left) 2026-02-17T06:08:50.462 INFO:tasks.rados.rados.0.trial115.stdout:3605: done (5 left) 2026-02-17T06:08:50.462 INFO:tasks.rados.rados.0.trial115.stdout:3606: done (4 left) 2026-02-17T06:08:50.462 INFO:tasks.rados.rados.0.trial115.stdout:3607: done (3 left) 2026-02-17T06:08:50.462 INFO:tasks.rados.rados.0.trial115.stdout:3608: done (2 left) 2026-02-17T06:08:50.462 INFO:tasks.rados.rados.0.trial115.stdout:3609: done (1 left) 2026-02-17T06:08:50.463 INFO:tasks.rados.rados.0.trial115.stdout:3610: done (0 left) 2026-02-17T06:08:50.463 INFO:tasks.rados.rados.0.trial115.stdout:3611: rollback oid 4 current snap is 384 2026-02-17T06:08:50.463 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 4 to 376 2026-02-17T06:08:50.463 INFO:tasks.rados.rados.0.trial115.stdout:3612: write (excl) oid 50 current snap is 384 2026-02-17T06:08:50.463 INFO:tasks.rados.rados.0.trial115.stdout:3612: seq_num 1240 ranges {440680=674299,1711907=624424} 2026-02-17T06:08:50.465 INFO:tasks.rados.rados.0.trial115.stdout:3612: writing trial11545420-50 from 440680 to 1114979 tid 1 2026-02-17T06:08:50.470 INFO:tasks.rados.rados.0.trial115.stdout:3612: writing trial11545420-50 from 1711907 to 2336331 tid 2 2026-02-17T06:08:50.470 INFO:tasks.rados.rados.0.trial115.stdout:3613: write (excl) oid 19 current snap is 384 2026-02-17T06:08:50.470 INFO:tasks.rados.rados.0.trial115.stdout:3611: finishing rollback tid 0 to trial11545420-4 2026-02-17T06:08:50.470 INFO:tasks.rados.rados.0.trial115.stdout:3611: finishing rollback tid 1 to trial11545420-4 2026-02-17T06:08:50.470 INFO:tasks.rados.rados.0.trial115.stdout:3611: finishing rollback tid 2 to trial11545420-4 2026-02-17T06:08:50.470 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1497 (ObjNum 1206 snap 372 seq_num 1206) dirty exists 2026-02-17T06:08:50.470 INFO:tasks.rados.rados.0.trial115.stdout:3612: finishing write tid 1 to trial11545420-50 2026-02-17T06:08:50.471 INFO:tasks.rados.rados.0.trial115.stdout:3613: seq_num 1241 ranges {457487=749548,1770389=484435,2746287=111200} 2026-02-17T06:08:50.474 INFO:tasks.rados.rados.0.trial115.stdout:3613: writing trial11545420-19 from 457487 to 1207035 tid 1 2026-02-17T06:08:50.478 INFO:tasks.rados.rados.0.trial115.stdout:3613: writing trial11545420-19 from 1770389 to 2254824 tid 2 2026-02-17T06:08:50.480 INFO:tasks.rados.rados.0.trial115.stdout:3613: writing trial11545420-19 from 2746287 to 2857487 tid 3 2026-02-17T06:08:50.481 INFO:tasks.rados.rados.0.trial115.stdout:3611: done (2 left) 2026-02-17T06:08:50.481 INFO:tasks.rados.rados.0.trial115.stdout:3614: write (excl) oid 14 current snap is 384 2026-02-17T06:08:50.481 INFO:tasks.rados.rados.0.trial115.stdout:3612: finishing write tid 2 to trial11545420-50 2026-02-17T06:08:50.481 INFO:tasks.rados.rados.0.trial115.stdout:3612: finishing write tid 3 to trial11545420-50 2026-02-17T06:08:50.481 INFO:tasks.rados.rados.0.trial115.stdout:3614: seq_num 1242 ranges {474294=424797,1428871=628093,2657357=216937} 2026-02-17T06:08:50.483 INFO:tasks.rados.rados.0.trial115.stdout:3614: writing trial11545420-14 from 474294 to 899091 tid 1 2026-02-17T06:08:50.488 INFO:tasks.rados.rados.0.trial115.stdout:3614: writing trial11545420-14 from 1428871 to 2056964 tid 2 2026-02-17T06:08:50.491 INFO:tasks.rados.rados.0.trial115.stdout:3614: writing trial11545420-14 from 2657357 to 2874294 tid 3 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3615: snap_create 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3612: finishing write tid 4 to trial11545420-50 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3612: oid 50 updating version 0 to 1496 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3612: oid 50 updating version 1496 to 1497 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3612: oid 50 updating version 1497 to 1498 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 50 v 1498 (ObjNum 1240 snap 384 seq_num 1240) dirty exists 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3612: left oid 50 (ObjNum 1240 snap 384 seq_num 1240) 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3613: finishing write tid 1 to trial11545420-19 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3613: finishing write tid 2 to trial11545420-19 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3613: finishing write tid 3 to trial11545420-19 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3613: finishing write tid 4 to trial11545420-19 2026-02-17T06:08:50.492 INFO:tasks.rados.rados.0.trial115.stdout:3613: finishing write tid 5 to trial11545420-19 2026-02-17T06:08:50.493 INFO:tasks.rados.rados.0.trial115.stdout:3613: oid 19 updating version 0 to 1887 2026-02-17T06:08:50.493 INFO:tasks.rados.rados.0.trial115.stdout:3613: oid 19 version 1887 is already newer than 1885 2026-02-17T06:08:50.493 INFO:tasks.rados.rados.0.trial115.stdout:3613: oid 19 version 1887 is already newer than 1886 2026-02-17T06:08:50.493 INFO:tasks.rados.rados.0.trial115.stdout:3613: oid 19 version 1887 is already newer than 1884 2026-02-17T06:08:50.493 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1887 (ObjNum 1241 snap 384 seq_num 1241) dirty exists 2026-02-17T06:08:50.493 INFO:tasks.rados.rados.0.trial115.stdout:3613: left oid 19 (ObjNum 1241 snap 384 seq_num 1241) 2026-02-17T06:08:50.493 INFO:tasks.rados.rados.0.trial115.stdout:3614: finishing write tid 1 to trial11545420-14 2026-02-17T06:08:50.497 INFO:tasks.rados.rados.0.trial115.stdout:3614: finishing write tid 2 to trial11545420-14 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:3614: finishing write tid 3 to trial11545420-14 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:3614: finishing write tid 4 to trial11545420-14 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:3614: finishing write tid 5 to trial11545420-14 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:3614: oid 14 updating version 0 to 1680 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:3614: oid 14 version 1680 is already newer than 1679 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:3614: oid 14 updating version 1680 to 1682 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:3614: oid 14 version 1682 is already newer than 1681 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1682 (ObjNum 1242 snap 384 seq_num 1242) dirty exists 2026-02-17T06:08:50.498 INFO:tasks.rados.rados.0.trial115.stdout:3614: left oid 14 (ObjNum 1242 snap 384 seq_num 1242) 2026-02-17T06:08:51.464 INFO:tasks.rados.rados.0.trial115.stdout:3612: done (3 left) 2026-02-17T06:08:51.465 INFO:tasks.rados.rados.0.trial115.stdout:3613: done (2 left) 2026-02-17T06:08:51.465 INFO:tasks.rados.rados.0.trial115.stdout:3614: done (1 left) 2026-02-17T06:08:51.465 INFO:tasks.rados.rados.0.trial115.stdout:3615: done (0 left) 2026-02-17T06:08:51.465 INFO:tasks.rados.rados.0.trial115.stdout:3616: read oid 44 snap -1 2026-02-17T06:08:51.465 INFO:tasks.rados.rados.0.trial115.stdout:3616: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:08:51.465 INFO:tasks.rados.rados.0.trial115.stdout:3617: snap_remove snap 381 2026-02-17T06:08:52.468 INFO:tasks.rados.rados.0.trial115.stdout:3617: done (1 left) 2026-02-17T06:08:52.468 INFO:tasks.rados.rados.0.trial115.stdout:3618: read oid 27 snap 382 2026-02-17T06:08:52.469 INFO:tasks.rados.rados.0.trial115.stdout:3618: expect (ObjNum 1233 snap 381 seq_num 1233) 2026-02-17T06:08:52.469 INFO:tasks.rados.rados.0.trial115.stdout:3619: read oid 1 snap -1 2026-02-17T06:08:52.469 INFO:tasks.rados.rados.0.trial115.stdout:3619: expect (ObjNum 1220 snap 378 seq_num 1220) 2026-02-17T06:08:52.469 INFO:tasks.rados.rados.0.trial115.stdout:3620: write (excl) oid 17 current snap is 385 2026-02-17T06:08:52.469 INFO:tasks.rados.rados.0.trial115.stdout:3616: expect (ObjNum 1050 snap 312 seq_num 1050) 2026-02-17T06:08:52.631 INFO:tasks.rados.rados.0.trial115.stdout:3619: expect (ObjNum 1220 snap 378 seq_num 1220) 2026-02-17T06:08:52.712 INFO:tasks.rados.rados.0.trial115.stdout:3618: expect (ObjNum 1233 snap 381 seq_num 1233) 2026-02-17T06:08:52.815 INFO:tasks.rados.rados.0.trial115.stdout:3620: seq_num 1243 ranges {491101=500046,1487353=488104,2401133=489968} 2026-02-17T06:08:52.819 INFO:tasks.rados.rados.0.trial115.stdout:3620: writing trial11545420-17 from 491101 to 991147 tid 1 2026-02-17T06:08:52.822 INFO:tasks.rados.rados.0.trial115.stdout:3620: writing trial11545420-17 from 1487353 to 1975457 tid 2 2026-02-17T06:08:52.827 INFO:tasks.rados.rados.0.trial115.stdout:3620: writing trial11545420-17 from 2401133 to 2891101 tid 3 2026-02-17T06:08:52.827 INFO:tasks.rados.rados.0.trial115.stdout:3616: done (3 left) 2026-02-17T06:08:52.827 INFO:tasks.rados.rados.0.trial115.stdout:3618: done (2 left) 2026-02-17T06:08:52.827 INFO:tasks.rados.rados.0.trial115.stdout:3619: done (1 left) 2026-02-17T06:08:52.827 INFO:tasks.rados.rados.0.trial115.stdout:3621: read oid 15 snap 379 2026-02-17T06:08:52.827 INFO:tasks.rados.rados.0.trial115.stdout:3621: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:08:52.827 INFO:tasks.rados.rados.0.trial115.stdout:3622: rollback oid 18 current snap is 385 2026-02-17T06:08:52.828 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 18 to 382 2026-02-17T06:08:52.828 INFO:tasks.rados.rados.0.trial115.stdout:3620: finishing write tid 1 to trial11545420-17 2026-02-17T06:08:52.828 INFO:tasks.rados.rados.0.trial115.stdout:3620: finishing write tid 2 to trial11545420-17 2026-02-17T06:08:52.828 INFO:tasks.rados.rados.0.trial115.stdout:3623: rollback oid 40 current snap is 385 2026-02-17T06:08:52.828 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 40 to 382 2026-02-17T06:08:52.828 INFO:tasks.rados.rados.0.trial115.stdout:3624: read oid 45 snap -1 2026-02-17T06:08:52.828 INFO:tasks.rados.rados.0.trial115.stdout:3624: expect (ObjNum 1236 snap 381 seq_num 1236) 2026-02-17T06:08:52.828 INFO:tasks.rados.rados.0.trial115.stdout:3625: delete oid 28 current snap is 385 2026-02-17T06:08:52.830 INFO:tasks.rados.rados.0.trial115.stdout:3620: finishing write tid 3 to trial11545420-17 2026-02-17T06:08:52.830 INFO:tasks.rados.rados.0.trial115.stdout:3620: finishing write tid 4 to trial11545420-17 2026-02-17T06:08:52.830 INFO:tasks.rados.rados.0.trial115.stdout:3620: finishing write tid 5 to trial11545420-17 2026-02-17T06:08:52.830 INFO:tasks.rados.rados.0.trial115.stdout:3620: oid 17 updating version 0 to 1807 2026-02-17T06:08:52.830 INFO:tasks.rados.rados.0.trial115.stdout:3620: oid 17 updating version 1807 to 1808 2026-02-17T06:08:52.830 INFO:tasks.rados.rados.0.trial115.stdout:3620: oid 17 updating version 1808 to 1809 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:3620: oid 17 updating version 1809 to 1810 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1810 (ObjNum 1243 snap 385 seq_num 1243) dirty exists 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:3620: left oid 17 (ObjNum 1243 snap 385 seq_num 1243) 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:3622: finishing rollback tid 0 to trial11545420-18 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:3622: finishing rollback tid 1 to trial11545420-18 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:3623: finishing rollback tid 0 to trial11545420-40 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:3622: finishing rollback tid 2 to trial11545420-18 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 1604 (ObjNum 1235 snap 381 seq_num 1235) dirty exists 2026-02-17T06:08:52.831 INFO:tasks.rados.rados.0.trial115.stdout:3623: finishing rollback tid 1 to trial11545420-40 2026-02-17T06:08:52.832 INFO:tasks.rados.rados.0.trial115.stdout:3623: finishing rollback tid 2 to trial11545420-40 2026-02-17T06:08:52.832 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1500 (ObjNum 1227 snap 379 seq_num 1227) dirty exists 2026-02-17T06:08:52.832 INFO:tasks.rados.rados.0.trial115.stdout:3624: expect (ObjNum 1236 snap 381 seq_num 1236) 2026-02-17T06:08:52.957 INFO:tasks.rados.rados.0.trial115.stdout:3621: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:08:53.158 INFO:tasks.rados.rados.0.trial115.stdout:3620: done (5 left) 2026-02-17T06:08:53.158 INFO:tasks.rados.rados.0.trial115.stdout:3621: done (4 left) 2026-02-17T06:08:53.158 INFO:tasks.rados.rados.0.trial115.stdout:3622: done (3 left) 2026-02-17T06:08:53.158 INFO:tasks.rados.rados.0.trial115.stdout:3623: done (2 left) 2026-02-17T06:08:53.158 INFO:tasks.rados.rados.0.trial115.stdout:3624: done (1 left) 2026-02-17T06:08:53.158 INFO:tasks.rados.rados.0.trial115.stdout:3625: done (0 left) 2026-02-17T06:08:53.158 INFO:tasks.rados.rados.0.trial115.stdout:3626: delete oid 23 current snap is 385 2026-02-17T06:08:53.159 INFO:tasks.rados.rados.0.trial115.stdout:3626: done (0 left) 2026-02-17T06:08:53.159 INFO:tasks.rados.rados.0.trial115.stdout:3627: snap_remove snap 376 2026-02-17T06:08:54.010 INFO:tasks.rados.rados.0.trial115.stdout:3627: done (0 left) 2026-02-17T06:08:54.011 INFO:tasks.rados.rados.0.trial115.stdout:3628: rollback oid 14 current snap is 385 2026-02-17T06:08:54.011 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 14 to 379 2026-02-17T06:08:54.011 INFO:tasks.rados.rados.0.trial115.stdout:3629: rollback oid 45 current snap is 385 2026-02-17T06:08:54.011 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 383 2026-02-17T06:08:54.011 INFO:tasks.rados.rados.0.trial115.stdout:3630: snap_create 2026-02-17T06:08:54.015 INFO:tasks.rados.rados.0.trial115.stdout:3629: finishing rollback tid 0 to trial11545420-45 2026-02-17T06:08:54.015 INFO:tasks.rados.rados.0.trial115.stdout:3629: finishing rollback tid 1 to trial11545420-45 2026-02-17T06:08:54.015 INFO:tasks.rados.rados.0.trial115.stdout:3629: finishing rollback tid 2 to trial11545420-45 2026-02-17T06:08:54.015 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1877 (ObjNum 1236 snap 381 seq_num 1236) dirty exists 2026-02-17T06:08:54.016 INFO:tasks.rados.rados.0.trial115.stdout:3628: finishing rollback tid 0 to trial11545420-14 2026-02-17T06:08:54.017 INFO:tasks.rados.rados.0.trial115.stdout:3628: finishing rollback tid 1 to trial11545420-14 2026-02-17T06:08:54.017 INFO:tasks.rados.rados.0.trial115.stdout:3628: finishing rollback tid 2 to trial11545420-14 2026-02-17T06:08:54.017 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1685 (ObjNum 1218 snap 378 seq_num 1218) dirty exists 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:3628: done (2 left) 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:3629: done (1 left) 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:3630: done (0 left) 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:3631: copy_from oid 42 from oid 3 current snap is 386 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:3632: rollback oid 34 current snap is 386 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 384 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:3633: read oid 22 snap 383 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:3633: expect (ObjNum 1232 snap 380 seq_num 1232) 2026-02-17T06:08:55.012 INFO:tasks.rados.rados.0.trial115.stdout:3634: snap_remove snap 385 2026-02-17T06:08:56.015 INFO:tasks.rados.rados.0.trial115.stdout:3634: done (3 left) 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3635: read oid 12 snap -1 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3635: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3636: delete oid 26 current snap is 386 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3632: finishing rollback tid 0 to trial11545420-34 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3632: finishing rollback tid 1 to trial11545420-34 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3632: finishing rollback tid 2 to trial11545420-34 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1493 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3631: finishing copy_from racing read to trial11545420-42 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3631: finishing copy_from to trial11545420-42 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1600 (ObjNum 1215 snap 376 seq_num 1215) dirty exists 2026-02-17T06:08:56.016 INFO:tasks.rados.rados.0.trial115.stdout:3633: expect (ObjNum 1232 snap 380 seq_num 1232) 2026-02-17T06:08:56.152 INFO:tasks.rados.rados.0.trial115.stdout:3635: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:08:56.223 INFO:tasks.rados.rados.0.trial115.stdout:3631: done (4 left) 2026-02-17T06:08:56.223 INFO:tasks.rados.rados.0.trial115.stdout:3632: done (3 left) 2026-02-17T06:08:56.223 INFO:tasks.rados.rados.0.trial115.stdout:3633: done (2 left) 2026-02-17T06:08:56.223 INFO:tasks.rados.rados.0.trial115.stdout:3635: done (1 left) 2026-02-17T06:08:56.223 INFO:tasks.rados.rados.0.trial115.stdout:3636: done (0 left) 2026-02-17T06:08:56.223 INFO:tasks.rados.rados.0.trial115.stdout:3637: snap_create 2026-02-17T06:08:57.018 INFO:tasks.rados.rados.0.trial115.stdout:3637: done (0 left) 2026-02-17T06:08:57.019 INFO:tasks.rados.rados.0.trial115.stdout:3638: read oid 14 snap -1 2026-02-17T06:08:57.019 INFO:tasks.rados.rados.0.trial115.stdout:3638: expect (ObjNum 1218 snap 378 seq_num 1218) 2026-02-17T06:08:57.019 INFO:tasks.rados.rados.0.trial115.stdout:3639: write oid 41 current snap is 387 2026-02-17T06:08:57.019 INFO:tasks.rados.rados.0.trial115.stdout:3639: seq_num 1247 ranges {558329=517395,1721281=779089} 2026-02-17T06:08:57.022 INFO:tasks.rados.rados.0.trial115.stdout:3639: writing trial11545420-41 from 558329 to 1075724 tid 1 2026-02-17T06:08:57.028 INFO:tasks.rados.rados.0.trial115.stdout:3639: writing trial11545420-41 from 1721281 to 2500370 tid 2 2026-02-17T06:08:57.028 INFO:tasks.rados.rados.0.trial115.stdout:3640: snap_create 2026-02-17T06:08:57.028 INFO:tasks.rados.rados.0.trial115.stdout:3638: expect (ObjNum 1218 snap 378 seq_num 1218) 2026-02-17T06:08:57.120 INFO:tasks.rados.rados.0.trial115.stdout:3639: finishing write tid 1 to trial11545420-41 2026-02-17T06:08:57.120 INFO:tasks.rados.rados.0.trial115.stdout:3639: finishing write tid 2 to trial11545420-41 2026-02-17T06:08:57.120 INFO:tasks.rados.rados.0.trial115.stdout:3639: finishing write tid 3 to trial11545420-41 2026-02-17T06:08:57.120 INFO:tasks.rados.rados.0.trial115.stdout:3639: finishing write tid 4 to trial11545420-41 2026-02-17T06:08:57.120 INFO:tasks.rados.rados.0.trial115.stdout:3639: oid 41 updating version 0 to 1607 2026-02-17T06:08:57.120 INFO:tasks.rados.rados.0.trial115.stdout:3639: oid 41 version 1607 is already newer than 1606 2026-02-17T06:08:57.121 INFO:tasks.rados.rados.0.trial115.stdout:3639: oid 41 version 1607 is already newer than 1605 2026-02-17T06:08:57.121 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1607 (ObjNum 1247 snap 387 seq_num 1247) dirty exists 2026-02-17T06:08:57.121 INFO:tasks.rados.rados.0.trial115.stdout:3639: left oid 41 (ObjNum 1247 snap 387 seq_num 1247) 2026-02-17T06:08:58.008 INFO:tasks.rados.rados.0.trial115.stdout:3638: done (2 left) 2026-02-17T06:08:58.008 INFO:tasks.rados.rados.0.trial115.stdout:3639: done (1 left) 2026-02-17T06:08:58.008 INFO:tasks.rados.rados.0.trial115.stdout:3640: done (0 left) 2026-02-17T06:08:58.008 INFO:tasks.rados.rados.0.trial115.stdout:3641: delete oid 36 current snap is 388 2026-02-17T06:08:58.011 INFO:tasks.rados.rados.0.trial115.stdout:3641: done (0 left) 2026-02-17T06:08:58.011 INFO:tasks.rados.rados.0.trial115.stdout:3642: read oid 48 snap -1 2026-02-17T06:08:58.012 INFO:tasks.rados.rados.0.trial115.stdout:3642: expect deleted 2026-02-17T06:08:58.012 INFO:tasks.rados.rados.0.trial115.stdout:3643: write oid 5 current snap is 388 2026-02-17T06:08:58.012 INFO:tasks.rados.rados.0.trial115.stdout:3643: seq_num 1249 ranges {591943=667893,1721892=782758} 2026-02-17T06:08:58.016 INFO:tasks.rados.rados.0.trial115.stdout:3643: writing trial11545420-5 from 591943 to 1259836 tid 1 2026-02-17T06:08:58.023 INFO:tasks.rados.rados.0.trial115.stdout:3643: writing trial11545420-5 from 1721892 to 2504650 tid 2 2026-02-17T06:08:58.023 INFO:tasks.rados.rados.0.trial115.stdout:3644: read oid 2 snap 379 2026-02-17T06:08:58.023 INFO:tasks.rados.rados.0.trial115.stdout:3644: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:08:58.023 INFO:tasks.rados.rados.0.trial115.stdout:3645: write oid 19 current snap is 388 2026-02-17T06:08:58.023 INFO:tasks.rados.rados.0.trial115.stdout:3643: finishing write tid 1 to trial11545420-5 2026-02-17T06:08:58.023 INFO:tasks.rados.rados.0.trial115.stdout:3645: seq_num 1250 ranges {608750=743142,1780374=642769} 2026-02-17T06:08:58.027 INFO:tasks.rados.rados.0.trial115.stdout:3645: writing trial11545420-19 from 608750 to 1351892 tid 1 2026-02-17T06:08:58.032 INFO:tasks.rados.rados.0.trial115.stdout:3645: writing trial11545420-19 from 1780374 to 2423143 tid 2 2026-02-17T06:08:58.032 INFO:tasks.rados.rados.0.trial115.stdout:3642: done (3 left) 2026-02-17T06:08:58.032 INFO:tasks.rados.rados.0.trial115.stdout:3646: read oid 33 snap -1 2026-02-17T06:08:58.032 INFO:tasks.rados.rados.0.trial115.stdout:3646: expect (ObjNum 1211 snap 373 seq_num 1211) 2026-02-17T06:08:58.032 INFO:tasks.rados.rados.0.trial115.stdout:3647: read oid 6 snap -1 2026-02-17T06:08:58.032 INFO:tasks.rados.rados.0.trial115.stdout:3647: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:08:58.032 INFO:tasks.rados.rados.0.trial115.stdout:3648: snap_remove snap 383 2026-02-17T06:08:59.020 INFO:tasks.rados.rados.0.trial115.stdout:3648: done (5 left) 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3649: read oid 26 snap -1 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3649: expect deleted 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3650: snap_create 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3643: finishing write tid 2 to trial11545420-5 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3643: finishing write tid 3 to trial11545420-5 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3643: finishing write tid 4 to trial11545420-5 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3643: oid 5 updating version 0 to 1630 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3643: oid 5 version 1630 is already newer than 1629 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3643: oid 5 updating version 1630 to 1631 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1631 (ObjNum 1249 snap 388 seq_num 1249) dirty exists 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3643: left oid 5 (ObjNum 1249 snap 388 seq_num 1249) 2026-02-17T06:08:59.021 INFO:tasks.rados.rados.0.trial115.stdout:3644: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:08:59.093 INFO:tasks.rados.rados.0.trial115.stdout:3645: finishing write tid 1 to trial11545420-19 2026-02-17T06:08:59.093 INFO:tasks.rados.rados.0.trial115.stdout:3645: finishing write tid 2 to trial11545420-19 2026-02-17T06:08:59.093 INFO:tasks.rados.rados.0.trial115.stdout:3645: finishing write tid 3 to trial11545420-19 2026-02-17T06:08:59.093 INFO:tasks.rados.rados.0.trial115.stdout:3645: finishing write tid 4 to trial11545420-19 2026-02-17T06:08:59.094 INFO:tasks.rados.rados.0.trial115.stdout:3645: oid 19 updating version 0 to 1900 2026-02-17T06:08:59.094 INFO:tasks.rados.rados.0.trial115.stdout:3645: oid 19 version 1900 is already newer than 1899 2026-02-17T06:08:59.094 INFO:tasks.rados.rados.0.trial115.stdout:3645: oid 19 updating version 1900 to 1901 2026-02-17T06:08:59.094 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1901 (ObjNum 1250 snap 388 seq_num 1250) dirty exists 2026-02-17T06:08:59.094 INFO:tasks.rados.rados.0.trial115.stdout:3645: left oid 19 (ObjNum 1250 snap 388 seq_num 1250) 2026-02-17T06:08:59.094 INFO:tasks.rados.rados.0.trial115.stdout:3647: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:08:59.210 INFO:tasks.rados.rados.0.trial115.stdout:3646: expect (ObjNum 1211 snap 373 seq_num 1211) 2026-02-17T06:09:00.022 INFO:tasks.rados.rados.0.trial115.stdout:3643: done (6 left) 2026-02-17T06:09:00.023 INFO:tasks.rados.rados.0.trial115.stdout:3644: done (5 left) 2026-02-17T06:09:00.023 INFO:tasks.rados.rados.0.trial115.stdout:3645: done (4 left) 2026-02-17T06:09:00.023 INFO:tasks.rados.rados.0.trial115.stdout:3646: done (3 left) 2026-02-17T06:09:00.023 INFO:tasks.rados.rados.0.trial115.stdout:3647: done (2 left) 2026-02-17T06:09:00.023 INFO:tasks.rados.rados.0.trial115.stdout:3649: done (1 left) 2026-02-17T06:09:00.023 INFO:tasks.rados.rados.0.trial115.stdout:3650: done (0 left) 2026-02-17T06:09:00.023 INFO:tasks.rados.rados.0.trial115.stdout:3651: snap_create 2026-02-17T06:09:01.036 INFO:tasks.rados.rados.0.trial115.stdout:3651: done (0 left) 2026-02-17T06:09:01.036 INFO:tasks.rados.rados.0.trial115.stdout:3652: read oid 2 snap 382 2026-02-17T06:09:01.036 INFO:tasks.rados.rados.0.trial115.stdout:3652: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:09:01.036 INFO:tasks.rados.rados.0.trial115.stdout:3653: rollback oid 41 current snap is 390 2026-02-17T06:09:01.036 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 41 to 388 2026-02-17T06:09:01.036 INFO:tasks.rados.rados.0.trial115.stdout:3654: snap_create 2026-02-17T06:09:01.037 INFO:tasks.rados.rados.0.trial115.stdout:3652: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:09:01.108 INFO:tasks.rados.rados.0.trial115.stdout:3653: finishing rollback tid 0 to trial11545420-41 2026-02-17T06:09:01.108 INFO:tasks.rados.rados.0.trial115.stdout:3653: finishing rollback tid 1 to trial11545420-41 2026-02-17T06:09:01.108 INFO:tasks.rados.rados.0.trial115.stdout:3653: finishing rollback tid 2 to trial11545420-41 2026-02-17T06:09:01.109 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 1610 (ObjNum 1247 snap 387 seq_num 1247) dirty exists 2026-02-17T06:09:02.036 INFO:tasks.rados.rados.0.trial115.stdout:3652: done (2 left) 2026-02-17T06:09:02.037 INFO:tasks.rados.rados.0.trial115.stdout:3653: done (1 left) 2026-02-17T06:09:02.037 INFO:tasks.rados.rados.0.trial115.stdout:3654: done (0 left) 2026-02-17T06:09:02.037 INFO:tasks.rados.rados.0.trial115.stdout:3655: copy_from oid 25 from oid 4 current snap is 391 2026-02-17T06:09:02.037 INFO:tasks.rados.rados.0.trial115.stdout:3656: snap_remove snap 380 2026-02-17T06:09:03.008 INFO:tasks.rados.rados.0.trial115.stdout:3656: done (1 left) 2026-02-17T06:09:03.008 INFO:tasks.rados.rados.0.trial115.stdout:3657: write oid 29 current snap is 391 2026-02-17T06:09:03.008 INFO:tasks.rados.rados.0.trial115.stdout:3655: finishing copy_from racing read to trial11545420-25 2026-02-17T06:09:03.008 INFO:tasks.rados.rados.0.trial115.stdout:3655: finishing copy_from to trial11545420-25 2026-02-17T06:09:03.008 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1497 (ObjNum 1206 snap 372 seq_num 1206) dirty exists 2026-02-17T06:09:03.009 INFO:tasks.rados.rados.0.trial115.stdout:3657: seq_num 1251 ranges {625557=418391,1838856=786427} 2026-02-17T06:09:03.012 INFO:tasks.rados.rados.0.trial115.stdout:3657: writing trial11545420-29 from 625557 to 1043948 tid 1 2026-02-17T06:09:03.018 INFO:tasks.rados.rados.0.trial115.stdout:3657: writing trial11545420-29 from 1838856 to 2625283 tid 2 2026-02-17T06:09:03.018 INFO:tasks.rados.rados.0.trial115.stdout:3655: done (1 left) 2026-02-17T06:09:03.018 INFO:tasks.rados.rados.0.trial115.stdout:3658: snap_create 2026-02-17T06:09:03.018 INFO:tasks.rados.rados.0.trial115.stdout:3657: finishing write tid 1 to trial11545420-29 2026-02-17T06:09:03.020 INFO:tasks.rados.rados.0.trial115.stdout:3657: finishing write tid 2 to trial11545420-29 2026-02-17T06:09:03.021 INFO:tasks.rados.rados.0.trial115.stdout:3657: finishing write tid 3 to trial11545420-29 2026-02-17T06:09:03.021 INFO:tasks.rados.rados.0.trial115.stdout:3657: finishing write tid 4 to trial11545420-29 2026-02-17T06:09:03.021 INFO:tasks.rados.rados.0.trial115.stdout:3657: oid 29 updating version 0 to 1493 2026-02-17T06:09:03.021 INFO:tasks.rados.rados.0.trial115.stdout:3657: oid 29 updating version 1493 to 1494 2026-02-17T06:09:03.021 INFO:tasks.rados.rados.0.trial115.stdout:3657: oid 29 updating version 1494 to 1495 2026-02-17T06:09:03.021 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1495 (ObjNum 1251 snap 391 seq_num 1251) dirty exists 2026-02-17T06:09:03.021 INFO:tasks.rados.rados.0.trial115.stdout:3657: left oid 29 (ObjNum 1251 snap 391 seq_num 1251) 2026-02-17T06:09:04.012 INFO:tasks.rados.rados.0.trial115.stdout:3657: done (1 left) 2026-02-17T06:09:04.012 INFO:tasks.rados.rados.0.trial115.stdout:3658: done (0 left) 2026-02-17T06:09:04.012 INFO:tasks.rados.rados.0.trial115.stdout:3659: rollback oid 11 current snap is 392 2026-02-17T06:09:04.013 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 11 to 382 2026-02-17T06:09:04.013 INFO:tasks.rados.rados.0.trial115.stdout:3660: copy_from oid 16 from oid 49 current snap is 392 2026-02-17T06:09:04.013 INFO:tasks.rados.rados.0.trial115.stdout:3661: write (excl) oid 45 current snap is 392 2026-02-17T06:09:04.013 INFO:tasks.rados.rados.0.trial115.stdout:3661: seq_num 1252 ranges {642364=493640,1897338=646438} 2026-02-17T06:09:04.016 INFO:tasks.rados.rados.0.trial115.stdout:3661: writing trial11545420-45 from 642364 to 1136004 tid 1 2026-02-17T06:09:04.022 INFO:tasks.rados.rados.0.trial115.stdout:3661: writing trial11545420-45 from 1897338 to 2543776 tid 2 2026-02-17T06:09:04.022 INFO:tasks.rados.rados.0.trial115.stdout:3662: write oid 24 current snap is 392 2026-02-17T06:09:04.022 INFO:tasks.rados.rados.0.trial115.stdout:3659: finishing rollback tid 0 to trial11545420-11 2026-02-17T06:09:04.022 INFO:tasks.rados.rados.0.trial115.stdout:3659: finishing rollback tid 1 to trial11545420-11 2026-02-17T06:09:04.022 INFO:tasks.rados.rados.0.trial115.stdout:3662: seq_num 1253 ranges {659171=568889,1839467=790096} 2026-02-17T06:09:04.026 INFO:tasks.rados.rados.0.trial115.stdout:3662: writing trial11545420-24 from 659171 to 1228060 tid 1 2026-02-17T06:09:04.031 INFO:tasks.rados.rados.0.trial115.stdout:3662: writing trial11545420-24 from 1839467 to 2629563 tid 2 2026-02-17T06:09:04.032 INFO:tasks.rados.rados.0.trial115.stdout:3663: write oid 35 current snap is 392 2026-02-17T06:09:04.032 INFO:tasks.rados.rados.0.trial115.stdout:3663: seq_num 1254 ranges {675978=644138,1897949=650107} 2026-02-17T06:09:04.036 INFO:tasks.rados.rados.0.trial115.stdout:3663: writing trial11545420-35 from 675978 to 1320116 tid 1 2026-02-17T06:09:04.042 INFO:tasks.rados.rados.0.trial115.stdout:3663: writing trial11545420-35 from 1897949 to 2548056 tid 2 2026-02-17T06:09:04.042 INFO:tasks.rados.rados.0.trial115.stdout:3664: read oid 2 snap -1 2026-02-17T06:09:04.042 INFO:tasks.rados.rados.0.trial115.stdout:3664: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:09:04.042 INFO:tasks.rados.rados.0.trial115.stdout:3665: read oid 5 snap 386 2026-02-17T06:09:04.042 INFO:tasks.rados.rados.0.trial115.stdout:3665: expect (ObjNum 1217 snap 376 seq_num 1217) 2026-02-17T06:09:04.042 INFO:tasks.rados.rados.0.trial115.stdout:3666: read oid 21 snap -1 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3666: expect (ObjNum 1190 snap 364 seq_num 1190) 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3667: copy_from oid 33 from oid 22 current snap is 392 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3659: finishing rollback tid 2 to trial11545420-11 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1946 (ObjNum 1163 snap 357 seq_num 1163) dirty exists 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3661: finishing write tid 1 to trial11545420-45 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3660: finishing copy_from racing read to trial11545420-16 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3659: done (8 left) 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3668: read oid 1 snap 386 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3668: expect (ObjNum 1220 snap 378 seq_num 1220) 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3669: write oid 38 current snap is 392 2026-02-17T06:09:04.043 INFO:tasks.rados.rados.0.trial115.stdout:3669: seq_num 1255 ranges {692785=435740,1672784=793765,2897503=195282} 2026-02-17T06:09:04.046 INFO:tasks.rados.rados.0.trial115.stdout:3669: writing trial11545420-38 from 692785 to 1128525 tid 1 2026-02-17T06:09:04.051 INFO:tasks.rados.rados.0.trial115.stdout:3669: writing trial11545420-38 from 1672784 to 2466549 tid 2 2026-02-17T06:09:04.054 INFO:tasks.rados.rados.0.trial115.stdout:3669: writing trial11545420-38 from 2897503 to 3092785 tid 3 2026-02-17T06:09:04.054 INFO:tasks.rados.rados.0.trial115.stdout:3670: copy_from oid 25 from oid 12 current snap is 392 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3660: finishing copy_from to trial11545420-16 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1494 (ObjNum 1238 snap 383 seq_num 1238) dirty exists 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3661: finishing write tid 2 to trial11545420-45 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3661: finishing write tid 3 to trial11545420-45 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3661: finishing write tid 4 to trial11545420-45 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3661: oid 45 updating version 0 to 1886 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3661: oid 45 version 1886 is already newer than 1884 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3661: oid 45 version 1886 is already newer than 1885 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1886 (ObjNum 1252 snap 392 seq_num 1252) dirty exists 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3661: left oid 45 (ObjNum 1252 snap 392 seq_num 1252) 2026-02-17T06:09:04.055 INFO:tasks.rados.rados.0.trial115.stdout:3662: finishing write tid 1 to trial11545420-24 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3662: finishing write tid 2 to trial11545420-24 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3660: done (10 left) 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3661: done (9 left) 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3671: read oid 14 snap -1 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3671: expect (ObjNum 1218 snap 378 seq_num 1218) 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3672: snap_remove snap 390 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3662: finishing write tid 3 to trial11545420-24 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3662: finishing write tid 4 to trial11545420-24 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3662: oid 24 updating version 0 to 1888 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3662: oid 24 updating version 1888 to 1889 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:3662: oid 24 updating version 1889 to 1890 2026-02-17T06:09:04.056 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1890 (ObjNum 1253 snap 392 seq_num 1253) dirty exists 2026-02-17T06:09:04.057 INFO:tasks.rados.rados.0.trial115.stdout:3662: left oid 24 (ObjNum 1253 snap 392 seq_num 1253) 2026-02-17T06:09:04.057 INFO:tasks.rados.rados.0.trial115.stdout:3663: finishing write tid 1 to trial11545420-35 2026-02-17T06:09:05.016 INFO:tasks.rados.rados.0.trial115.stdout:3662: done (10 left) 2026-02-17T06:09:05.016 INFO:tasks.rados.rados.0.trial115.stdout:3672: done (9 left) 2026-02-17T06:09:05.016 INFO:tasks.rados.rados.0.trial115.stdout:3673: write (excl) oid 26 current snap is 392 2026-02-17T06:09:05.017 INFO:tasks.rados.rados.0.trial115.stdout:3673: seq_num 1256 ranges {709592=510989,1731266=653776,3041279=68313} 2026-02-17T06:09:05.021 INFO:tasks.rados.rados.0.trial115.stdout:3673: writing trial11545420-26 from 709592 to 1220581 tid 1 2026-02-17T06:09:05.025 INFO:tasks.rados.rados.0.trial115.stdout:3673: writing trial11545420-26 from 1731266 to 2385042 tid 2 2026-02-17T06:09:05.028 INFO:tasks.rados.rados.0.trial115.stdout:3673: writing trial11545420-26 from 3041279 to 3109592 tid 3 2026-02-17T06:09:05.028 INFO:tasks.rados.rados.0.trial115.stdout:3674: copy_from oid 49 from oid 50 current snap is 392 2026-02-17T06:09:05.028 INFO:tasks.rados.rados.0.trial115.stdout:3664: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:09:05.099 INFO:tasks.rados.rados.0.trial115.stdout:3668: expect (ObjNum 1220 snap 378 seq_num 1220) 2026-02-17T06:09:05.180 INFO:tasks.rados.rados.0.trial115.stdout:3663: finishing write tid 2 to trial11545420-35 2026-02-17T06:09:05.181 INFO:tasks.rados.rados.0.trial115.stdout:3665: expect (ObjNum 1217 snap 376 seq_num 1217) 2026-02-17T06:09:05.284 INFO:tasks.rados.rados.0.trial115.stdout:3663: finishing write tid 3 to trial11545420-35 2026-02-17T06:09:05.284 INFO:tasks.rados.rados.0.trial115.stdout:3663: finishing write tid 4 to trial11545420-35 2026-02-17T06:09:05.284 INFO:tasks.rados.rados.0.trial115.stdout:3663: oid 35 updating version 0 to 1913 2026-02-17T06:09:05.284 INFO:tasks.rados.rados.0.trial115.stdout:3663: oid 35 updating version 1913 to 1914 2026-02-17T06:09:05.284 INFO:tasks.rados.rados.0.trial115.stdout:3663: oid 35 updating version 1914 to 1915 2026-02-17T06:09:05.285 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 1915 (ObjNum 1254 snap 392 seq_num 1254) dirty exists 2026-02-17T06:09:05.285 INFO:tasks.rados.rados.0.trial115.stdout:3663: left oid 35 (ObjNum 1254 snap 392 seq_num 1254) 2026-02-17T06:09:05.285 INFO:tasks.rados.rados.0.trial115.stdout:3666: expect (ObjNum 1190 snap 364 seq_num 1190) 2026-02-17T06:09:05.330 INFO:tasks.rados.rados.0.trial115.stdout:3667: finishing copy_from racing read to trial11545420-33 2026-02-17T06:09:05.330 INFO:tasks.rados.rados.0.trial115.stdout:3669: finishing write tid 1 to trial11545420-38 2026-02-17T06:09:05.330 INFO:tasks.rados.rados.0.trial115.stdout:3667: finishing copy_from to trial11545420-33 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1488 (ObjNum 1232 snap 380 seq_num 1232) dirty exists 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3669: finishing write tid 2 to trial11545420-38 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3669: finishing write tid 3 to trial11545420-38 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3663: done (10 left) 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3664: done (9 left) 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3665: done (8 left) 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3666: done (7 left) 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3667: done (6 left) 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3668: done (5 left) 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3675: delete oid 18 current snap is 392 2026-02-17T06:09:05.331 INFO:tasks.rados.rados.0.trial115.stdout:3669: finishing write tid 4 to trial11545420-38 2026-02-17T06:09:05.332 INFO:tasks.rados.rados.0.trial115.stdout:3671: expect (ObjNum 1218 snap 378 seq_num 1218) 2026-02-17T06:09:05.423 INFO:tasks.rados.rados.0.trial115.stdout:3669: finishing write tid 5 to trial11545420-38 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:3669: oid 38 updating version 0 to 1918 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:3669: oid 38 version 1918 is already newer than 1917 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:3669: oid 38 updating version 1918 to 1919 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:3669: oid 38 version 1919 is already newer than 1916 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1919 (ObjNum 1255 snap 392 seq_num 1255) dirty exists 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:3669: left oid 38 (ObjNum 1255 snap 392 seq_num 1255) 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:3670: finishing copy_from racing read to trial11545420-25 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:3670: finishing copy_from to trial11545420-25 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 1815 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2026-02-17T06:09:05.424 INFO:tasks.rados.rados.0.trial115.stdout:3673: finishing write tid 1 to trial11545420-26 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3673: finishing write tid 2 to trial11545420-26 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3669: done (5 left) 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3670: done (4 left) 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3671: done (3 left) 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3675: done (2 left) 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3676: rollback oid 5 current snap is 392 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 5 to 379 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3673: finishing write tid 3 to trial11545420-26 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3673: finishing write tid 4 to trial11545420-26 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3673: finishing write tid 5 to trial11545420-26 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3673: oid 26 updating version 0 to 1470 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3673: oid 26 updating version 1470 to 1471 2026-02-17T06:09:05.425 INFO:tasks.rados.rados.0.trial115.stdout:3673: oid 26 updating version 1471 to 1473 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3673: oid 26 version 1473 is already newer than 1472 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1473 (ObjNum 1256 snap 392 seq_num 1256) dirty exists 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3673: left oid 26 (ObjNum 1256 snap 392 seq_num 1256) 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3674: finishing copy_from racing read to trial11545420-49 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3674: finishing copy_from to trial11545420-49 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1498 (ObjNum 1240 snap 384 seq_num 1240) dirty exists 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3673: done (2 left) 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3674: done (1 left) 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3677: read oid 41 snap -1 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3677: expect (ObjNum 1247 snap 387 seq_num 1247) 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:3678: rollback oid 38 current snap is 392 2026-02-17T06:09:05.426 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 38 to 382 2026-02-17T06:09:05.427 INFO:tasks.rados.rados.0.trial115.stdout:3679: read oid 49 snap 384 2026-02-17T06:09:05.427 INFO:tasks.rados.rados.0.trial115.stdout:3679: expect (ObjNum 1238 snap 383 seq_num 1238) 2026-02-17T06:09:05.427 INFO:tasks.rados.rados.0.trial115.stdout:3680: read oid 16 snap 379 2026-02-17T06:09:05.427 INFO:tasks.rados.rados.0.trial115.stdout:3680: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:09:05.427 INFO:tasks.rados.rados.0.trial115.stdout:3681: delete oid 48 current snap is 392 2026-02-17T06:09:05.428 INFO:tasks.rados.rados.0.trial115.stdout:3676: finishing rollback tid 0 to trial11545420-5 2026-02-17T06:09:05.428 INFO:tasks.rados.rados.0.trial115.stdout:3676: finishing rollback tid 1 to trial11545420-5 2026-02-17T06:09:05.428 INFO:tasks.rados.rados.0.trial115.stdout:3676: finishing rollback tid 2 to trial11545420-5 2026-02-17T06:09:05.428 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1639 (ObjNum 1217 snap 376 seq_num 1217) dirty exists 2026-02-17T06:09:05.428 INFO:tasks.rados.rados.0.trial115.stdout:3679: expect (ObjNum 1238 snap 383 seq_num 1238) 2026-02-17T06:09:05.496 INFO:tasks.rados.rados.0.trial115.stdout:3677: expect (ObjNum 1247 snap 387 seq_num 1247) 2026-02-17T06:09:05.566 INFO:tasks.rados.rados.0.trial115.stdout:3678: finishing rollback tid 0 to trial11545420-38 2026-02-17T06:09:05.566 INFO:tasks.rados.rados.0.trial115.stdout:3678: finishing rollback tid 1 to trial11545420-38 2026-02-17T06:09:05.566 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1925 (ObjNum 145 snap 0 seq_num 1334218072) dirty dne 2026-02-17T06:09:05.566 INFO:tasks.rados.rados.0.trial115.stdout:3680: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:3676: done (5 left) 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:3677: done (4 left) 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:3678: done (3 left) 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:3679: done (2 left) 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:3680: done (1 left) 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:3681: done (0 left) 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:3682: rollback oid 34 current snap is 392 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 386 2026-02-17T06:09:05.766 INFO:tasks.rados.rados.0.trial115.stdout:3683: read oid 38 snap -1 2026-02-17T06:09:05.767 INFO:tasks.rados.rados.0.trial115.stdout:3683: expect deleted 2026-02-17T06:09:05.767 INFO:tasks.rados.rados.0.trial115.stdout:3684: snap_remove snap 382 2026-02-17T06:09:06.017 INFO:tasks.rados.rados.0.trial115.stdout:3684: done (2 left) 2026-02-17T06:09:06.017 INFO:tasks.rados.rados.0.trial115.stdout:3685: delete oid 19 current snap is 392 2026-02-17T06:09:06.018 INFO:tasks.rados.rados.0.trial115.stdout:3682: finishing rollback tid 0 to trial11545420-34 2026-02-17T06:09:06.018 INFO:tasks.rados.rados.0.trial115.stdout:3682: finishing rollback tid 1 to trial11545420-34 2026-02-17T06:09:06.018 INFO:tasks.rados.rados.0.trial115.stdout:3682: finishing rollback tid 2 to trial11545420-34 2026-02-17T06:09:06.018 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1496 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:09:06.022 INFO:tasks.rados.rados.0.trial115.stdout:3682: done (2 left) 2026-02-17T06:09:06.022 INFO:tasks.rados.rados.0.trial115.stdout:3683: done (1 left) 2026-02-17T06:09:06.022 INFO:tasks.rados.rados.0.trial115.stdout:3685: done (0 left) 2026-02-17T06:09:06.022 INFO:tasks.rados.rados.0.trial115.stdout:3686: rollback oid 33 current snap is 392 2026-02-17T06:09:06.022 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 379 2026-02-17T06:09:06.022 INFO:tasks.rados.rados.0.trial115.stdout:3687: snap_remove snap 391 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3687: done (1 left) 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3688: read oid 24 snap -1 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3688: expect (ObjNum 1253 snap 392 seq_num 1253) 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3689: rollback oid 17 current snap is 392 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3686: finishing rollback tid 0 to trial11545420-33 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3686: finishing rollback tid 1 to trial11545420-33 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3686: finishing rollback tid 2 to trial11545420-33 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1689 (ObjNum 1211 snap 373 seq_num 1211) dirty exists 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 17 to 379 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3686: done (2 left) 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3690: rollback oid 23 current snap is 392 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 23 to 384 2026-02-17T06:09:07.022 INFO:tasks.rados.rados.0.trial115.stdout:3691: snap_create 2026-02-17T06:09:07.025 INFO:tasks.rados.rados.0.trial115.stdout:3690: finishing rollback tid 1 to trial11545420-23 2026-02-17T06:09:07.025 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 0 (ObjNum 65 snap 0 seq_num 0) dirty dne 2026-02-17T06:09:07.026 INFO:tasks.rados.rados.0.trial115.stdout:3688: expect (ObjNum 1253 snap 392 seq_num 1253) 2026-02-17T06:09:07.124 INFO:tasks.rados.rados.0.trial115.stdout:3689: finishing rollback tid 0 to trial11545420-17 2026-02-17T06:09:07.125 INFO:tasks.rados.rados.0.trial115.stdout:3689: finishing rollback tid 1 to trial11545420-17 2026-02-17T06:09:07.125 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1812 (ObjNum 305 snap 0 seq_num 0) dirty dne 2026-02-17T06:09:08.009 INFO:tasks.rados.rados.0.trial115.stdout:3688: done (3 left) 2026-02-17T06:09:08.009 INFO:tasks.rados.rados.0.trial115.stdout:3689: done (2 left) 2026-02-17T06:09:08.009 INFO:tasks.rados.rados.0.trial115.stdout:3690: done (1 left) 2026-02-17T06:09:08.009 INFO:tasks.rados.rados.0.trial115.stdout:3691: done (0 left) 2026-02-17T06:09:08.009 INFO:tasks.rados.rados.0.trial115.stdout:3692: snap_create 2026-02-17T06:09:08.803 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-17T06:09:08.803 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-17T06:09:08.803 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T06:09:08.803 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T06:09:08.805 INFO:tasks.ceph.mon.a:Started 2026-02-17T06:09:08.805 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:09:08.806 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:08.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.873+0000 7f76b6a66640 10 monclient: get_monmap_and_config 2026-02-17T06:09:08.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.873+0000 7f76b6a66640 10 monclient: build_initial_monmap 2026-02-17T06:09:08.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.873+0000 7f76b6a66640 10 monclient: monmap: 2026-02-17T06:09:08.878 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:08.878 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:08.878 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:08.874888+0000 2026-02-17T06:09:08.878 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:08.874888+0000 2026-02-17T06:09:08.878 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:08.878 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 1 Processor -- start 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 1 -- start start 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient: init 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2026-02-17T06:09:08.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f76b006c890 0x7f76b006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient(hunting): picked mon.noname-c con 0x7f76b006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f76b009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006d1b0 0x7f76b0135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient(hunting): picked mon.noname-b con 0x7f76b006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b006daf0 con 0x7f76b009c4a0 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f76b00a0880 con 0x7f76b006c890 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f76b009cc80 con 0x7f76b006d1b0 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:08.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b6a66640 10 monclient(hunting): authenticate will time out at 1994.428501s 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006d1b0 0x7f76b0135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006d1b0 0x7f76b0135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40214/0 (socket says 10.20.193.78:40214) 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 1 -- 10.20.193.78:0/2768859187 learned_addr learned my addr 10.20.193.78:0/2768859187 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient(hunting): get_auth_request con 0x7f76b006d1b0 auth_method 0 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76affff640 1 --2- 10.20.193.78:0/2768859187 >> v2:10.20.193.115:3300/0 conn(0x7f76b006c890 0x7f76b006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:08.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76affff640 10 monclient(hunting): get_auth_request con 0x7f76b006c890 auth_method 0 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76affff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient(hunting): handle_auth_done global_id 264284 payload 274 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 1 -- 10.20.193.78:0/2768859187 >> v2:10.20.193.115:3300/0 conn(0x7f76b006c890 msgr2=0x7f76b006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 1 --2- 10.20.193.78:0/2768859187 >> v2:10.20.193.115:3300/0 conn(0x7f76b006c890 0x7f76b006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 1 -- 10.20.193.78:0/2768859187 >> v1:10.20.193.115:6789/0 conn(0x7f76b009c4a0 legacy=0x7f76b0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient: _finish_hunting 0 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 1 monclient: found mon.noname-b 2026-02-17T06:09:08.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:08.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:09:08.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.874+0000 7f76b4fdc640 1 -- 10.20.193.78:0/2768859187 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b009cf60 con 0x7f76b006d1b0 2026-02-17T06:09:08.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b4fdc640 1 -- 10.20.193.78:0/2768859187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006d1b0 msgr2=0x7f76b0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:08.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b4fdc640 1 --2- 10.20.193.78:0/2768859187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006d1b0 0x7f76b0135500 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76b006dcc0 tx=0x7f76a402f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb2cef2c996df78 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:08.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 1 -- 10.20.193.78:0/2768859187 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40373f0 con 0x7f76b006d1b0 2026-02-17T06:09:08.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:08.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: dump: 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: _finish_auth 0 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: _check_auth_tickets 2026-02-17T06:09:08.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 1 -- 10.20.193.78:0/2768859187 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a4037590 con 0x7f76b006d1b0 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 5 monclient: authenticate success, global_id 264284 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 1 -- 10.20.193.78:0/2768859187 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4037960 con 0x7f76b006d1b0 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76aeffd640 10 monclient: dump: 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:08.891 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 4 set_mon_vals no callback set 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 10 monclient: shutdown 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 1 -- 10.20.193.78:0/2768859187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006d1b0 msgr2=0x7f76b0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 1 --2- 10.20.193.78:0/2768859187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006d1b0 0x7f76b0135500 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76b006dcc0 tx=0x7f76a402f200 comp rx=0 tx=0).stop 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 1 -- 10.20.193.78:0/2768859187 shutdown_connections 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 1 --2- 10.20.193.78:0/2768859187 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006d1b0 0x7f76b0135500 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 1 --2- 10.20.193.78:0/2768859187 >> v2:10.20.193.115:3300/0 conn(0x7f76b006c890 0x7f76b006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 1 -- 10.20.193.78:0/2768859187 >> 10.20.193.78:0/2768859187 conn(0x7f76b00a6620 msgr2=0x7f76b00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 1 -- 10.20.193.78:0/2768859187 shutdown_connections 2026-02-17T06:09:08.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.877+0000 7f76b6a66640 1 -- 10.20.193.78:0/2768859187 wait complete. 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient: build_initial_monmap 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient: monmap: 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 1 Processor -- start 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 1 -- start start 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient: init 2026-02-17T06:09:08.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006c890 0x7f76b011cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient(hunting): picked mon.noname-c con 0x7f76b006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 0x7f76b011d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient(hunting): picked mon.noname-a con 0x7f76b006d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f76b009c4a0 0x7f76b011d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient(hunting): picked mon.noname-b con 0x7f76b009c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76b00a01a0 con 0x7f76b006d1b0 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76b00a0020 con 0x7f76b009c4a0 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f76b009be20 con 0x7f76b006c890 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76b6a66640 10 monclient(hunting): authenticate will time out at 1994.432910s 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76affff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006c890 0x7f76b011cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76af7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 0x7f76b011d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76affff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006c890 0x7f76b011cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40224/0 (socket says 10.20.193.78:40224) 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76af7fe640 10 monclient(hunting): get_auth_request con 0x7f76b006d1b0 auth_method 0 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76affff640 1 -- 10.20.193.78:0/1102995873 learned_addr learned my addr 10.20.193.78:0/1102995873 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76af7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76af7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76affff640 10 monclient(hunting): get_auth_request con 0x7f76b006c890 auth_method 0 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76affff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.878+0000 7f76affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76af7fe640 1 -- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 msgr2=0x7f76b011d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76af7fe640 1 -- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 msgr2=0x7f76b011d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76af7fe640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 0x7f76b011d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:08.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76af7fe640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 0x7f76b011d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76b4fdc640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f76b009c4a0 0x7f76b011d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76b4fdc640 10 monclient(hunting): get_auth_request con 0x7f76b009c4a0 auth_method 0 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76b4fdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76b4fdc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76b4fdc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 10 monclient(hunting): handle_auth_done global_id 264287 payload 824 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 1 -- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f76b009c4a0 msgr2=0x7f76b011d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f76b009c4a0 0x7f76b011d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 1 -- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 msgr2=0x7f76b011d1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 0x7f76b011d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 10 monclient: _finish_hunting 0 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 1 monclient: found mon.noname-c 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.879+0000 7f76affff640 1 -- 10.20.193.78:0/1102995873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b0120370 con 0x7f76b006c890 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76affff640 1 -- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006c890 msgr2=0x7f76b011cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76affff640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006c890 0x7f76b011cc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e7c0 tx=0x7f769c00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 1 -- 10.20.193.78:0/1102995873 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c017070 con 0x7f76b006c890 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: dump: 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: _finish_auth 0 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: _check_auth_tickets 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 1 -- 10.20.193.78:0/1102995873 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f769c00cae0 con 0x7f76b006c890 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 1 -- 10.20.193.78:0/1102995873 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c00f040 con 0x7f76b006c890 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76acff9640 10 monclient: dump: 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76ae7fc640 4 set_mon_vals no callback set 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76b6a66640 5 monclient: authenticate success, global_id 264287 2026-02-17T06:09:08.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76b6a66640 10 monclient: _renew_subs 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76b6a66640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b009d810 con 0x7f76b006c890 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76b6a66640 10 monclient: _renew_subs 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76b6a66640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.881+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b011fdb0 con 0x7f76b006c890 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76b6a66640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76b6a66640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76b6a66640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b0068410 con 0x7f76b006c890 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76acff9640 1 -- 10.20.193.78:0/1102995873 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f769c012050 con 0x7f76b006c890 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76acff9640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f768003deb0 0x7f7680040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76acff9640 1 -- 10.20.193.78:0/1102995873 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(811..811 src has 525..811) ==== 4811+0+0 (secure 0 0 0) 0x7f769c052690 con 0x7f76b006c890 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76af7fe640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f768003deb0 0x7f7680040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.882+0000 7f76af7fe640 10 monclient: get_auth_request con 0x7f768003deb0 auth_method 0 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.884+0000 7f76acff9640 1 -- 10.20.193.78:0/1102995873 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f76b0068410 con 0x7f76b006c890 2026-02-17T06:09:08.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.884+0000 7f76acff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:08.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.884+0000 7f76acff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:08.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.885+0000 7f76af7fe640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f768003deb0 0x7f7680040370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f76b0051310 tx=0x7f76a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:08.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.996+0000 7f76b6a66640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:08.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.996+0000 7f76b6a66640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:08.997 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.996+0000 7f76b6a66640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:08.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.996+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f76b006cc70 con 0x7f76b006c890 2026-02-17T06:09:08.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.997+0000 7f76acff9640 1 -- 10.20.193.78:0/1102995873 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f769c023d80 con 0x7f76b006c890 2026-02-17T06:09:08.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.997+0000 7f76acff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:08.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.997+0000 7f76acff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:08.998 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":254,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:08.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f768003deb0 msgr2=0x7f7680040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:08.998 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f768003deb0 0x7f7680040370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f76b0051310 tx=0x7f76a0009040 comp rx=0 tx=0).stop 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 10 monclient: shutdown 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006c890 msgr2=0x7f76b011cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006c890 0x7f76b011cc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e7c0 tx=0x7f769c00ec90 comp rx=0 tx=0).stop 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 shutdown_connections 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f768003deb0 0x7f7680040370 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f76b009c4a0 0x7f76b011d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f76b006d1b0 0x7f76b011d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 --2- 10.20.193.78:0/1102995873 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f76b006c890 0x7f76b011cc70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 >> 10.20.193.78:0/1102995873 conn(0x7f76b00a6620 msgr2=0x7f76b009e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 shutdown_connections 2026-02-17T06:09:08.999 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:08.998+0000 7f76b6a66640 1 -- 10.20.193.78:0/1102995873 wait complete. 2026-02-17T06:09:09.009 INFO:tasks.rados.rados.0.trial115.stdout:3692: done (0 left) 2026-02-17T06:09:09.009 INFO:tasks.rados.rados.0.trial115.stdout:3693: read oid 25 snap -1 2026-02-17T06:09:09.009 INFO:tasks.rados.rados.0.trial115.stdout:3693: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:09:09.009 INFO:tasks.rados.rados.0.trial115.stdout:3694: copy_from oid 43 from oid 47 current snap is 394 2026-02-17T06:09:09.009 INFO:tasks.rados.rados.0.trial115.stdout:3695: snap_create 2026-02-17T06:09:09.015 INFO:tasks.rados.rados.0.trial115.stdout:3693: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:09:09.087 INFO:tasks.rados.rados.0.trial115.stdout:3694: finishing copy_from racing read to trial11545420-43 2026-02-17T06:09:09.087 INFO:tasks.rados.rados.0.trial115.stdout:3694: finishing copy_from to trial11545420-43 2026-02-17T06:09:09.087 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1623 (ObjNum 1225 snap 379 seq_num 1225) dirty exists 2026-02-17T06:09:10.010 INFO:tasks.rados.rados.0.trial115.stdout:3693: done (2 left) 2026-02-17T06:09:10.011 INFO:tasks.rados.rados.0.trial115.stdout:3694: done (1 left) 2026-02-17T06:09:10.011 INFO:tasks.rados.rados.0.trial115.stdout:3695: done (0 left) 2026-02-17T06:09:10.011 INFO:tasks.rados.rados.0.trial115.stdout:3696: snap_remove snap 388 2026-02-17T06:09:11.012 INFO:tasks.rados.rados.0.trial115.stdout:3696: done (0 left) 2026-02-17T06:09:11.012 INFO:tasks.rados.rados.0.trial115.stdout:3697: delete oid 35 current snap is 395 2026-02-17T06:09:11.017 INFO:tasks.rados.rados.0.trial115.stdout:3697: done (0 left) 2026-02-17T06:09:11.017 INFO:tasks.rados.rados.0.trial115.stdout:3698: read oid 13 snap 389 2026-02-17T06:09:11.017 INFO:tasks.rados.rados.0.trial115.stdout:3698: expect (ObjNum 1229 snap 380 seq_num 1229) 2026-02-17T06:09:11.018 INFO:tasks.rados.rados.0.trial115.stdout:3699: delete oid 33 current snap is 395 2026-02-17T06:09:11.019 INFO:tasks.rados.rados.0.trial115.stdout:3699: done (1 left) 2026-02-17T06:09:11.019 INFO:tasks.rados.rados.0.trial115.stdout:3700: copy_from oid 9 from oid 14 current snap is 395 2026-02-17T06:09:11.019 INFO:tasks.rados.rados.0.trial115.stdout:3701: write oid 10 current snap is 395 2026-02-17T06:09:11.020 INFO:tasks.rados.rados.0.trial115.stdout:3701: seq_num 1262 ranges {410434=678836,1565805=548430,2573111=637323} 2026-02-17T06:09:11.023 INFO:tasks.rados.rados.0.trial115.stdout:3701: writing trial11545420-10 from 410434 to 1089270 tid 1 2026-02-17T06:09:11.026 INFO:tasks.rados.rados.0.trial115.stdout:3701: writing trial11545420-10 from 1565805 to 2114235 tid 2 2026-02-17T06:09:11.031 INFO:tasks.rados.rados.0.trial115.stdout:3701: writing trial11545420-10 from 2573111 to 3210434 tid 3 2026-02-17T06:09:11.031 INFO:tasks.rados.rados.0.trial115.stdout:3702: read oid 2 snap -1 2026-02-17T06:09:11.031 INFO:tasks.rados.rados.0.trial115.stdout:3702: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:09:11.032 INFO:tasks.rados.rados.0.trial115.stdout:3703: snap_create 2026-02-17T06:09:11.032 INFO:tasks.rados.rados.0.trial115.stdout:3698: expect (ObjNum 1229 snap 380 seq_num 1229) 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3700: finishing copy_from racing read to trial11545420-9 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3700: finishing copy_from to trial11545420-9 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1685 (ObjNum 1218 snap 378 seq_num 1218) dirty exists 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: finishing write tid 1 to trial11545420-10 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: finishing write tid 2 to trial11545420-10 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: finishing write tid 3 to trial11545420-10 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: finishing write tid 4 to trial11545420-10 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: finishing write tid 5 to trial11545420-10 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: oid 10 updating version 0 to 1961 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: oid 10 version 1961 is already newer than 1959 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: oid 10 updating version 1961 to 1962 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: oid 10 version 1962 is already newer than 1960 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 1962 (ObjNum 1262 snap 395 seq_num 1262) dirty exists 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3701: left oid 10 (ObjNum 1262 snap 395 seq_num 1262) 2026-02-17T06:09:11.157 INFO:tasks.rados.rados.0.trial115.stdout:3702: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:09:12.007 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:12.017 INFO:tasks.rados.rados.0.trial115.stdout:3698: done (4 left) 2026-02-17T06:09:12.017 INFO:tasks.rados.rados.0.trial115.stdout:3700: done (3 left) 2026-02-17T06:09:12.018 INFO:tasks.rados.rados.0.trial115.stdout:3701: done (2 left) 2026-02-17T06:09:12.018 INFO:tasks.rados.rados.0.trial115.stdout:3702: done (1 left) 2026-02-17T06:09:12.018 INFO:tasks.rados.rados.0.trial115.stdout:3703: done (0 left) 2026-02-17T06:09:12.018 INFO:tasks.rados.rados.0.trial115.stdout:3704: read oid 10 snap -1 2026-02-17T06:09:12.018 INFO:tasks.rados.rados.0.trial115.stdout:3704: expect (ObjNum 1262 snap 395 seq_num 1262) 2026-02-17T06:09:12.018 INFO:tasks.rados.rados.0.trial115.stdout:3705: delete oid 14 current snap is 396 2026-02-17T06:09:12.020 INFO:tasks.rados.rados.0.trial115.stdout:3705: done (1 left) 2026-02-17T06:09:12.020 INFO:tasks.rados.rados.0.trial115.stdout:3706: copy_from oid 19 from oid 49 current snap is 396 2026-02-17T06:09:12.020 INFO:tasks.rados.rados.0.trial115.stdout:3707: write oid 22 current snap is 396 2026-02-17T06:09:12.020 INFO:tasks.rados.rados.0.trial115.stdout:3707: seq_num 1264 ranges {444048=429334,1282769=552099,2627957=616091} 2026-02-17T06:09:12.023 INFO:tasks.rados.rados.0.trial115.stdout:3707: writing trial11545420-22 from 444048 to 873382 tid 1 2026-02-17T06:09:12.026 INFO:tasks.rados.rados.0.trial115.stdout:3707: writing trial11545420-22 from 1282769 to 1834868 tid 2 2026-02-17T06:09:12.032 INFO:tasks.rados.rados.0.trial115.stdout:3707: writing trial11545420-22 from 2627957 to 3244048 tid 3 2026-02-17T06:09:12.032 INFO:tasks.rados.rados.0.trial115.stdout:3708: rollback oid 29 current snap is 396 2026-02-17T06:09:12.032 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 379 2026-02-17T06:09:12.032 INFO:tasks.rados.rados.0.trial115.stdout:3704: expect (ObjNum 1262 snap 395 seq_num 1262) 2026-02-17T06:09:12.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.077+0000 7fc7a9a85640 10 monclient: get_monmap_and_config 2026-02-17T06:09:12.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.077+0000 7fc7a9a85640 10 monclient: build_initial_monmap 2026-02-17T06:09:12.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.077+0000 7fc7a9a85640 10 monclient: monmap: 2026-02-17T06:09:12.080 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:12.080 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:12.078694+0000 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:12.078694+0000 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:12.081 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.077+0000 7fc7a9a85640 1 Processor -- start 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.077+0000 7fc7a9a85640 1 -- start start 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.077+0000 7fc7a9a85640 10 monclient: init 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 10 monclient: _add_conns ranks=[3,1,0,4,2] 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 0x7fc7a40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 10 monclient(hunting): picked mon.noname-a con 0x7fc7a406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc7a406cc70 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc7a413a240 addr v1:10.20.193.78:6789/0 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a4074150 con 0x7fc7a413a240 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a409ef70 con 0x7fc7a406cc70 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:12.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a409f250 con 0x7fc7a406c890 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a9a85640 10 monclient(hunting): authenticate will time out at 1997.632298s 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 0x7fc7a40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 0x7fc7a40a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55236/0 (socket says 10.20.193.78:55236) 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a37fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc7a413a240 0x7fc7a413c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:45972/0 (socket says 10.20.193.78:45972) 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 1 -- 10.20.193.78:0/1974797374 learned_addr learned my addr 10.20.193.78:0/1974797374 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 10 monclient(hunting): get_auth_request con 0x7fc7a406c890 auth_method 0 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 1 -- 10.20.193.78:0/1974797374 >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 msgr2=0x7fc7a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:09:12.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 1 -- 10.20.193.78:0/1974797374 >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 msgr2=0x7fc7a40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:09:12.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 1 --2- 10.20.193.78:0/1974797374 >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 0x7fc7a40a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:09:12.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.078+0000 7fc7a2ffd640 1 --2- 10.20.193.78:0/1974797374 >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 0x7fc7a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:12.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.080+0000 7fc7a37fe640 1 -- 10.20.193.78:0/1974797374 >> v1:10.20.193.78:6789/0 conn(0x7fc7a413a240 legacy=0x7fc7a413c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:12.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.080+0000 7fc7a27fc640 1 -- 10.20.193.78:0/1974797374 >> v1:10.20.193.115:6789/0 conn(0x7fc7a406cc70 legacy=0x7fc7a406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:12.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 <== mon.1 v1:10.20.193.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246390733 0 0) 0x7fc7a409ef70 con 0x7fc7a406cc70 2026-02-17T06:09:12.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): _negotiate 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): my global_id is 274159 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 --> v1:10.20.193.115:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7900058b0 con 0x7fc7a406cc70 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 <== mon.1 v1:10.20.193.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1723928045 0 0) 0x7fc7900058b0 con 0x7fc7a406cc70 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 msgr2=0x7fc7a40a01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 --2- 10.20.193.78:0/1974797374 >> v2:10.20.193.78:3300/0 conn(0x7fc7a406c890 0x7fc7a40a01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 >> v1:10.20.193.78:6789/0 conn(0x7fc7a413a240 legacy=0x7fc7a413c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: _finish_hunting 0 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 monclient: found mon.noname-c-legacy 2026-02-17T06:09:12.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.115:6789/0 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 --> v1:10.20.193.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a409df40 con 0x7fc7a406cc70 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: _finish_auth 0 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 <== mon.1 v1:10.20.193.115:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fc78c003200 con 0x7fc7a406cc70 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: dump: 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:12.095 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] but i'm connected to v1:10.20.193.115:6789/0, reconnecting 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 >> v1:10.20.193.115:6789/0 conn(0x7fc7a406cc70 legacy=0x7fc7a406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:12.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc790005eb0 0x7fc7900062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): picked mon.a con 0x7fc790005eb0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc790007010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): picked mon.c con 0x7fc790006b90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7900077a0 0x7fc79000ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): picked mon.b con 0x7fc7900077a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a409f250 con 0x7fc790005eb0 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc79000e6b0 con 0x7fc7900077a0 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:12.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc79000e8b0 con 0x7fc790006b90 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a1ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc790005eb0 0x7fc7900062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 10 monclient(hunting): get_auth_request con 0x7fc790005eb0 auth_method 0 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc790007010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient(hunting): get_auth_request con 0x7fc790006b90 auth_method 0 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:12.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7900077a0 0x7fc79000ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 10 monclient(hunting): get_auth_request con 0x7fc7900077a0 auth_method 0 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:12.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient(hunting): handle_auth_done global_id 274159 payload 293 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 1 -- 10.20.193.78:0/1974797374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7900077a0 msgr2=0x7fc79000ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7900077a0 0x7fc79000ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 1 -- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc790005eb0 msgr2=0x7fc7900062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc790005eb0 0x7fc7900062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient: _finish_hunting 0 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 1 monclient: found mon.c 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a37fe640 1 -- 10.20.193.78:0/1974797374 reap_dead start 2026-02-17T06:09:12.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 1 -- 10.20.193.78:0/1974797374 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7900058b0 con 0x7fc790006b90 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient: _finish_auth 0 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 10 monclient: _check_auth_tickets 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a9a85640 5 monclient: authenticate success, global_id 274159 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a9a85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.081+0000 7fc7a2ffd640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc790005eb0 0x7fc7900062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a27fc640 1 -- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 msgr2=0x7fc790007010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a27fc640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc790007010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc78c002bf0 tx=0x7fc78c02f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0dd8ca6df07946b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00c0e0 con 0x7fc790006b90 2026-02-17T06:09:12.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 10 monclient: dump: 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:12.102 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c005ac0 con 0x7fc790006b90 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 1 -- 10.20.193.78:0/1974797374 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c007070 con 0x7fc790006b90 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 4 set_mon_vals no callback set 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a1ffb640 10 monclient: dump: 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:12.103 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 10 monclient: shutdown 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 1 -- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 msgr2=0x7fc790007010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:12.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc790007010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc78c002bf0 tx=0x7fc78c02f660 comp rx=0 tx=0).stop 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 1 -- 10.20.193.78:0/1974797374 shutdown_connections 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 1 --2- 10.20.193.78:0/1974797374 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc790007010 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 1 -- 10.20.193.78:0/1974797374 >> 10.20.193.78:0/1974797374 conn(0x7fc7a40a6620 msgr2=0x7fc7a40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 1 -- 10.20.193.78:0/1974797374 shutdown_connections 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.084+0000 7fc7a9a85640 1 -- 10.20.193.78:0/1974797374 wait complete. 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient: build_initial_monmap 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient: monmap: 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:12.105 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 1 Processor -- start 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 1 -- start start 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient: init 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:09:12.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc7a411cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient(hunting): picked mon.noname-c con 0x7fc790006b90 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7a411d1b0 0x7fc7a411d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient(hunting): picked mon.noname-b con 0x7fc7a411d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 0x7fc7a41d5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient(hunting): picked mon.noname-a con 0x7fc7a41d3370 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a40a0370 con 0x7fc7a41d3370 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a40a01f0 con 0x7fc7a411d1b0 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:12.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a40a04f0 con 0x7fc790006b90 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a9a85640 10 monclient(hunting): authenticate will time out at 1997.639648s 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc7a411cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 0x7fc7a41d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc7a411cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40294/0 (socket says 10.20.193.78:40294) 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 -- 10.20.193.78:0/3783733652 learned_addr learned my addr 10.20.193.78:0/3783733652 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 0x7fc7a41d5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:55250/0 (socket says 10.20.193.78:55250) 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient(hunting): get_auth_request con 0x7fc790006b90 auth_method 0 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:12.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 10 monclient(hunting): get_auth_request con 0x7fc7a41d3370 auth_method 0 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 1 -- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 msgr2=0x7fc7a41d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 1 -- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 msgr2=0x7fc7a41d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 0x7fc7a41d5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a37fe640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 0x7fc7a41d5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:12.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a27fc640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7a411d1b0 0x7fc7a411d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a27fc640 10 monclient(hunting): get_auth_request con 0x7fc7a411d1b0 auth_method 0 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient(hunting): handle_auth_done global_id 264290 payload 824 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 -- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7a411d1b0 msgr2=0x7fc7a411d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7a411d1b0 0x7fc7a411d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 -- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 msgr2=0x7fc7a41d5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 0x7fc7a41d5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:09:12.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 monclient: found mon.noname-c 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.085+0000 7fc7a2ffd640 1 -- 10.20.193.78:0/3783733652 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a409d810 con 0x7fc790006b90 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a2ffd640 1 -- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 msgr2=0x7fc7a411cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a2ffd640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc7a411cc70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc798010b60 tx=0x7fc79800e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 1 -- 10.20.193.78:0/3783733652 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79801a020 con 0x7fc790006b90 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: dump: 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:12.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: _finish_auth 0 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:09:12.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 5 monclient: authenticate success, global_id 264290 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 1 -- 10.20.193.78:0/3783733652 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc798010e70 con 0x7fc790006b90 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 1 -- 10.20.193.78:0/3783733652 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798009780 con 0x7fc790006b90 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a17fa640 4 set_mon_vals no callback set 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 10 monclient: _renew_subs 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a411f010 con 0x7fc790006b90 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 10 monclient: dump: 2026-02-17T06:09:12.113 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 10 monclient: _renew_subs 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a411ea50 con 0x7fc790006b90 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:12.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a406d080 con 0x7fc790006b90 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.088+0000 7fc77bfff640 1 -- 10.20.193.78:0/3783733652 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc798016700 con 0x7fc790006b90 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.089+0000 7fc77bfff640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc77403deb0 0x7fc774040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.089+0000 7fc77bfff640 1 -- 10.20.193.78:0/3783733652 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(815..815 src has 525..815) ==== 4799+0+0 (secure 0 0 0) 0x7fc7980535b0 con 0x7fc790006b90 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.089+0000 7fc7a27fc640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc77403deb0 0x7fc774040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.089+0000 7fc7a27fc640 10 monclient: get_auth_request con 0x7fc77403deb0 auth_method 0 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.091+0000 7fc77bfff640 1 -- 10.20.193.78:0/3783733652 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc798024020 con 0x7fc790006b90 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.091+0000 7fc77bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.091+0000 7fc77bfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:12.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.092+0000 7fc7a27fc640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc77403deb0 0x7fc774040370 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc78c007fd0 tx=0x7fc78c009070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:12.124 INFO:tasks.rados.rados.0.trial115.stdout:3706: finishing copy_from racing read to trial11545420-19 2026-02-17T06:09:12.124 INFO:tasks.rados.rados.0.trial115.stdout:3706: finishing copy_from to trial11545420-19 2026-02-17T06:09:12.124 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 19 v 1498 (ObjNum 1240 snap 384 seq_num 1240) dirty exists 2026-02-17T06:09:12.124 INFO:tasks.rados.rados.0.trial115.stdout:3707: finishing write tid 1 to trial11545420-22 2026-02-17T06:09:12.124 INFO:tasks.rados.rados.0.trial115.stdout:3707: finishing write tid 2 to trial11545420-22 2026-02-17T06:09:12.124 INFO:tasks.rados.rados.0.trial115.stdout:3704: done (3 left) 2026-02-17T06:09:12.124 INFO:tasks.rados.rados.0.trial115.stdout:3706: done (2 left) 2026-02-17T06:09:12.124 INFO:tasks.rados.rados.0.trial115.stdout:3709: snap_remove snap 395 2026-02-17T06:09:12.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.205+0000 7fc7a9a85640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:12.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.205+0000 7fc7a9a85640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:12.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.205+0000 7fc7a9a85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:12.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.205+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc7a40730e0 con 0x7fc790006b90 2026-02-17T06:09:12.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.205+0000 7fc77bfff640 1 -- 10.20.193.78:0/3783733652 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fc79801fe20 con 0x7fc790006b90 2026-02-17T06:09:12.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.205+0000 7fc77bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:12.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.205+0000 7fc77bfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:12.207 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":254,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:12.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc77403deb0 msgr2=0x7fc774040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:12.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc77403deb0 0x7fc774040370 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc78c007fd0 tx=0x7fc78c009070 comp rx=0 tx=0).stop 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 10 monclient: shutdown 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 msgr2=0x7fc7a411cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc7a411cc70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc798010b60 tx=0x7fc79800e5a0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 shutdown_connections 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc7a41d3370 0x7fc7a41d5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc77403deb0 0x7fc774040370 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc7a411d1b0 0x7fc7a411d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 --2- 10.20.193.78:0/3783733652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc790006b90 0x7fc7a411cc70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 >> 10.20.193.78:0/3783733652 conn(0x7fc7a40a6620 msgr2=0x7fc7a409c720 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:12.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 shutdown_connections 2026-02-17T06:09:12.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:12.206+0000 7fc7a9a85640 1 -- 10.20.193.78:0/3783733652 wait complete. 2026-02-17T06:09:13.011 INFO:tasks.rados.rados.0.trial115.stdout:3709: done (2 left) 2026-02-17T06:09:13.011 INFO:tasks.rados.rados.0.trial115.stdout:3710: write oid 14 current snap is 396 2026-02-17T06:09:13.012 INFO:tasks.rados.rados.0.trial115.stdout:3707: finishing write tid 3 to trial11545420-22 2026-02-17T06:09:13.012 INFO:tasks.rados.rados.0.trial115.stdout:3707: finishing write tid 4 to trial11545420-22 2026-02-17T06:09:13.012 INFO:tasks.rados.rados.0.trial115.stdout:3710: seq_num 1265 ranges {460855=504583,1624898=412110,2539027=572508} 2026-02-17T06:09:13.015 INFO:tasks.rados.rados.0.trial115.stdout:3710: writing trial11545420-14 from 460855 to 965438 tid 1 2026-02-17T06:09:13.019 INFO:tasks.rados.rados.0.trial115.stdout:3710: writing trial11545420-14 from 1624898 to 2037008 tid 2 2026-02-17T06:09:13.023 INFO:tasks.rados.rados.0.trial115.stdout:3710: writing trial11545420-14 from 2539027 to 3111535 tid 3 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3711: copy_from oid 27 from oid 16 current snap is 396 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3707: finishing write tid 5 to trial11545420-22 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3707: oid 22 updating version 0 to 1499 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3707: oid 22 updating version 1499 to 1502 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3707: oid 22 version 1502 is already newer than 1501 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3707: oid 22 version 1502 is already newer than 1500 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1502 (ObjNum 1264 snap 396 seq_num 1264) dirty exists 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3707: left oid 22 (ObjNum 1264 snap 396 seq_num 1264) 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3708: finishing rollback tid 0 to trial11545420-29 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:3708: finishing rollback tid 1 to trial11545420-29 2026-02-17T06:09:13.024 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1497 (ObjNum 49 snap 0 seq_num 311903389) dirty dne 2026-02-17T06:09:13.025 INFO:tasks.rados.rados.0.trial115.stdout:3707: done (3 left) 2026-02-17T06:09:13.025 INFO:tasks.rados.rados.0.trial115.stdout:3708: done (2 left) 2026-02-17T06:09:13.025 INFO:tasks.rados.rados.0.trial115.stdout:3712: snap_create 2026-02-17T06:09:13.025 INFO:tasks.rados.rados.0.trial115.stdout:3710: finishing write tid 1 to trial11545420-14 2026-02-17T06:09:13.025 INFO:tasks.rados.rados.0.trial115.stdout:3710: finishing write tid 2 to trial11545420-14 2026-02-17T06:09:13.026 INFO:tasks.rados.rados.0.trial115.stdout:3710: finishing write tid 3 to trial11545420-14 2026-02-17T06:09:13.026 INFO:tasks.rados.rados.0.trial115.stdout:3710: finishing write tid 4 to trial11545420-14 2026-02-17T06:09:13.026 INFO:tasks.rados.rados.0.trial115.stdout:3710: finishing write tid 5 to trial11545420-14 2026-02-17T06:09:13.026 INFO:tasks.rados.rados.0.trial115.stdout:3710: oid 14 updating version 0 to 1695 2026-02-17T06:09:13.026 INFO:tasks.rados.rados.0.trial115.stdout:3710: oid 14 version 1695 is already newer than 1694 2026-02-17T06:09:13.026 INFO:tasks.rados.rados.0.trial115.stdout:3710: oid 14 version 1695 is already newer than 1693 2026-02-17T06:09:13.026 INFO:tasks.rados.rados.0.trial115.stdout:3710: oid 14 version 1695 is already newer than 1692 2026-02-17T06:09:13.027 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1695 (ObjNum 1265 snap 396 seq_num 1265) dirty exists 2026-02-17T06:09:13.027 INFO:tasks.rados.rados.0.trial115.stdout:3710: left oid 14 (ObjNum 1265 snap 396 seq_num 1265) 2026-02-17T06:09:13.030 INFO:tasks.rados.rados.0.trial115.stdout:3711: finishing copy_from racing read to trial11545420-27 2026-02-17T06:09:13.031 INFO:tasks.rados.rados.0.trial115.stdout:3711: finishing copy_from to trial11545420-27 2026-02-17T06:09:13.031 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1494 (ObjNum 1238 snap 383 seq_num 1238) dirty exists 2026-02-17T06:09:14.020 INFO:tasks.rados.rados.0.trial115.stdout:3710: done (2 left) 2026-02-17T06:09:14.020 INFO:tasks.rados.rados.0.trial115.stdout:3711: done (1 left) 2026-02-17T06:09:14.020 INFO:tasks.rados.rados.0.trial115.stdout:3712: done (0 left) 2026-02-17T06:09:14.020 INFO:tasks.rados.rados.0.trial115.stdout:3713: write oid 17 current snap is 397 2026-02-17T06:09:14.020 INFO:tasks.rados.rados.0.trial115.stdout:3713: seq_num 1266 ranges {477662=579832,1683380=555768,2966450=311212} 2026-02-17T06:09:14.024 INFO:tasks.rados.rados.0.trial115.stdout:3713: writing trial11545420-17 from 477662 to 1057494 tid 1 2026-02-17T06:09:14.029 INFO:tasks.rados.rados.0.trial115.stdout:3713: writing trial11545420-17 from 1683380 to 2239148 tid 2 2026-02-17T06:09:14.033 INFO:tasks.rados.rados.0.trial115.stdout:3713: writing trial11545420-17 from 2966450 to 3277662 tid 3 2026-02-17T06:09:14.033 INFO:tasks.rados.rados.0.trial115.stdout:3714: read oid 6 snap -1 2026-02-17T06:09:14.033 INFO:tasks.rados.rados.0.trial115.stdout:3714: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:09:14.033 INFO:tasks.rados.rados.0.trial115.stdout:3715: snap_remove snap 379 2026-02-17T06:09:15.030 INFO:tasks.rados.rados.0.trial115.stdout:3715: done (2 left) 2026-02-17T06:09:15.030 INFO:tasks.rados.rados.0.trial115.stdout:3716: write (excl) oid 32 current snap is 397 2026-02-17T06:09:15.030 INFO:tasks.rados.rados.0.trial115.stdout:3716: seq_num 1267 ranges {494469=655081,1741862=415779,2710226=584243} 2026-02-17T06:09:15.034 INFO:tasks.rados.rados.0.trial115.stdout:3716: writing trial11545420-32 from 494469 to 1149550 tid 1 2026-02-17T06:09:15.038 INFO:tasks.rados.rados.0.trial115.stdout:3716: writing trial11545420-32 from 1741862 to 2157641 tid 2 2026-02-17T06:09:15.043 INFO:tasks.rados.rados.0.trial115.stdout:3716: writing trial11545420-32 from 2710226 to 3294469 tid 3 2026-02-17T06:09:15.043 INFO:tasks.rados.rados.0.trial115.stdout:3717: read oid 16 snap -1 2026-02-17T06:09:15.043 INFO:tasks.rados.rados.0.trial115.stdout:3717: expect (ObjNum 1238 snap 383 seq_num 1238) 2026-02-17T06:09:15.043 INFO:tasks.rados.rados.0.trial115.stdout:3718: snap_remove snap 386 2026-02-17T06:09:15.215 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.280+0000 7feb30d14640 10 monclient: get_monmap_and_config 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.280+0000 7feb30d14640 10 monclient: build_initial_monmap 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.280+0000 7feb30d14640 10 monclient: monmap: 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:15.281854+0000 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:15.281854+0000 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:15.281 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:15.282 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 1 Processor -- start 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 1 -- start start 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient: init 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feb2c06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c09d430 0x7feb2c13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient(hunting): picked mon.noname-b con 0x7feb2c09d430 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7feb2c09bf40 0x7feb2c13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient(hunting): picked mon.noname-c con 0x7feb2c09bf40 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb2c073600 con 0x7feb2c06c890 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7feb2c09ca50 con 0x7feb2c09bf40 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c06dab0 con 0x7feb2c09d430 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb30d14640 10 monclient(hunting): authenticate will time out at 2000.835501s 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb2a575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7feb2c06c890 0x7feb2c06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:45984/0 (socket says 10.20.193.78:45984) 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb2a575640 1 -- 10.20.193.78:0/1541485757 learned_addr learned my addr 10.20.193.78:0/1541485757 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 1 --2- 10.20.193.78:0/1541485757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c09d430 0x7feb2c13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient(hunting): get_auth_request con 0x7feb2c09d430 auth_method 0 2026-02-17T06:09:15.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb2ad76640 1 --2- 10.20.193.78:0/1541485757 >> v2:10.20.193.115:3300/0 conn(0x7feb2c09bf40 0x7feb2c13fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb2ad76640 10 monclient(hunting): get_auth_request con 0x7feb2c09bf40 auth_method 0 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb2ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb2ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb2ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient(hunting): handle_auth_done global_id 264293 payload 274 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 1 -- 10.20.193.78:0/1541485757 >> v2:10.20.193.115:3300/0 conn(0x7feb2c09bf40 msgr2=0x7feb2c13fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 1 --2- 10.20.193.78:0/1541485757 >> v2:10.20.193.115:3300/0 conn(0x7feb2c09bf40 0x7feb2c13fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 1 -- 10.20.193.78:0/1541485757 >> v1:10.20.193.78:6789/0 conn(0x7feb2c06c890 legacy=0x7feb2c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient: _finish_hunting 0 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 1 monclient: found mon.noname-b 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:09:15.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.281+0000 7feb29d74640 1 -- 10.20.193.78:0/1541485757 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c09c770 con 0x7feb2c09d430 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29d74640 1 -- 10.20.193.78:0/1541485757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c09d430 msgr2=0x7feb2c13d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29d74640 1 --2- 10.20.193.78:0/1541485757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c09d430 0x7feb2c13d5d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feb2c09c670 tx=0x7feb1402ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19791583d3031ff0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 1 -- 10.20.193.78:0/1541485757 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1403c070 con 0x7feb2c09d430 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: dump: 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:15.285 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: _finish_auth 0 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: _check_auth_tickets 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 1 -- 10.20.193.78:0/1541485757 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1402fbd0 con 0x7feb2c09d430 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 5 monclient: authenticate success, global_id 264293 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 1 -- 10.20.193.78:0/1541485757 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14037440 con 0x7feb2c09d430 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 4 set_mon_vals no callback set 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb29573640 10 monclient: dump: 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 10 monclient: shutdown 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 1 -- 10.20.193.78:0/1541485757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c09d430 msgr2=0x7feb2c13d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 1 --2- 10.20.193.78:0/1541485757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c09d430 0x7feb2c13d5d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feb2c09c670 tx=0x7feb1402ed50 comp rx=0 tx=0).stop 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 1 -- 10.20.193.78:0/1541485757 shutdown_connections 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 1 --2- 10.20.193.78:0/1541485757 >> v2:10.20.193.115:3300/0 conn(0x7feb2c09bf40 0x7feb2c13fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 1 --2- 10.20.193.78:0/1541485757 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c09d430 0x7feb2c13d5d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 1 -- 10.20.193.78:0/1541485757 >> 10.20.193.78:0/1541485757 conn(0x7feb2c0a6620 msgr2=0x7feb2c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:15.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 1 -- 10.20.193.78:0/1541485757 shutdown_connections 2026-02-17T06:09:15.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 1 -- 10.20.193.78:0/1541485757 wait complete. 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 10 monclient: build_initial_monmap 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.284+0000 7feb30d14640 10 monclient: monmap: 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 1 Processor -- start 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 1 -- start start 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient: init 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:15.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c06c890 0x7feb2c13e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient(hunting): picked mon.noname-c con 0x7feb2c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feb2c09bf40 0x7feb2c13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient(hunting): picked mon.noname-b con 0x7feb2c09bf40 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 0x7feb2c1cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient(hunting): picked mon.noname-a con 0x7feb2c09d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c06d500 con 0x7feb2c09d430 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c06d380 con 0x7feb2c09bf40 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c06d680 con 0x7feb2c06c890 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb30d14640 10 monclient(hunting): authenticate will time out at 2000.839492s 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c06c890 0x7feb2c13e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c06c890 0x7feb2c13e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:40316/0 (socket says 10.20.193.78:40316) 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 -- 10.20.193.78:0/3218144244 learned_addr learned my addr 10.20.193.78:0/3218144244 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient(hunting): get_auth_request con 0x7feb2c06c890 auth_method 0 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 0x7feb2c1cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 10 monclient(hunting): get_auth_request con 0x7feb2c09d430 auth_method 0 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:15.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 1 -- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 msgr2=0x7feb2c1cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 1 -- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 msgr2=0x7feb2c1cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 0x7feb2c1cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2ad76640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 0x7feb2c1cef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb29d74640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feb2c09bf40 0x7feb2c13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient(hunting): handle_auth_done global_id 264296 payload 824 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 -- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feb2c09bf40 msgr2=0x7feb2c13eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feb2c09bf40 0x7feb2c13eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 -- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 msgr2=0x7feb2c1cef30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 0x7feb2c1cef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient: _finish_hunting 0 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 monclient: found mon.noname-c 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb2a575640 1 -- 10.20.193.78:0/3218144244 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c13fa50 con 0x7feb2c06c890 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb29d74640 10 monclient: get_auth_request con 0x7feb2c09bf40 auth_method 0 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.285+0000 7feb29d74640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feb2c09bf40 0x7feb2c13eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb2a575640 1 -- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c06c890 msgr2=0x7feb2c13e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:15.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb2a575640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c06c890 0x7feb2c13e860 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c750 tx=0x7feb2000cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 1 -- 10.20.193.78:0/3218144244 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20005de0 con 0x7feb2c06c890 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 10 monclient: dump: 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:15.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 10 monclient: _finish_auth 0 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb30d14640 5 monclient: authenticate success, global_id 264296 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 1 -- 10.20.193.78:0/3218144244 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb20007070 con 0x7feb2c06c890 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb30d14640 10 monclient: _renew_subs 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb30d14640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c09d810 con 0x7feb2c06c890 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.287+0000 7feb12ffd640 1 -- 10.20.193.78:0/3218144244 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20007440 con 0x7feb2c06c890 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb12ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb28d72640 4 set_mon_vals no callback set 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb12ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb12ffd640 10 monclient: dump: 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb30d14640 10 monclient: _renew_subs 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb30d14640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c13c320 con 0x7feb2c06c890 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb30d14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb30d14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:15.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb30d14640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c13bed0 con 0x7feb2c06c890 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb12ffd640 1 -- 10.20.193.78:0/3218144244 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7feb20007660 con 0x7feb2c06c890 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb12ffd640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feaf803deb0 0x7feaf8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.288+0000 7feb12ffd640 1 -- 10.20.193.78:0/3218144244 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(818..818 src has 525..818) ==== 4795+0+0 (secure 0 0 0) 0x7feb200526c0 con 0x7feb2c06c890 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.289+0000 7feb29d74640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feaf803deb0 0x7feaf8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.289+0000 7feb29d74640 10 monclient: get_auth_request con 0x7feaf803deb0 auth_method 0 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.291+0000 7feb12ffd640 1 -- 10.20.193.78:0/3218144244 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7feb2c13bed0 con 0x7feb2c06c890 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.291+0000 7feb12ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.291+0000 7feb12ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:15.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.291+0000 7feb29d74640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feaf803deb0 0x7feaf8040370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feb2c06a480 tx=0x7feb14033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:15.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.410+0000 7feb30d14640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:15.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.410+0000 7feb30d14640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:15.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.410+0000 7feb30d14640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:15.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.410+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7feb2c074980 con 0x7feb2c06c890 2026-02-17T06:09:15.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.410+0000 7feb12ffd640 1 -- 10.20.193.78:0/3218144244 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7feb2000a5e0 con 0x7feb2c06c890 2026-02-17T06:09:15.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.410+0000 7feb12ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:15.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.410+0000 7feb12ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:15.411 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":254,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":26,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feaf803deb0 msgr2=0x7feaf8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feaf803deb0 0x7feaf8040370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feb2c06a480 tx=0x7feb14033000 comp rx=0 tx=0).stop 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 10 monclient: shutdown 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c06c890 msgr2=0x7feb2c13e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c06c890 0x7feb2c13e860 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c750 tx=0x7feb2000cc20 comp rx=0 tx=0).stop 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 shutdown_connections 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7feb2c09d430 0x7feb2c1cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7feaf803deb0 0x7feaf8040370 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7feb2c09bf40 0x7feb2c13eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 --2- 10.20.193.78:0/3218144244 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7feb2c06c890 0x7feb2c13e860 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.411+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 >> 10.20.193.78:0/3218144244 conn(0x7feb2c0a6620 msgr2=0x7feb2c1374c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:15.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.412+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 shutdown_connections 2026-02-17T06:09:15.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:15.412+0000 7feb30d14640 1 -- 10.20.193.78:0/3218144244 wait complete. 2026-02-17T06:09:16.034 INFO:tasks.rados.rados.0.trial115.stdout:3718: done (4 left) 2026-02-17T06:09:16.034 INFO:tasks.rados.rados.0.trial115.stdout:3719: rollback oid 48 current snap is 397 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 48 to 392 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: finishing write tid 1 to trial11545420-17 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: finishing write tid 2 to trial11545420-17 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: finishing write tid 3 to trial11545420-17 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: finishing write tid 4 to trial11545420-17 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: finishing write tid 5 to trial11545420-17 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: oid 17 updating version 0 to 1815 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: oid 17 updating version 1815 to 1816 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: oid 17 version 1816 is already newer than 1814 2026-02-17T06:09:16.035 INFO:tasks.rados.rados.0.trial115.stdout:3713: oid 17 version 1816 is already newer than 1813 2026-02-17T06:09:16.036 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1816 (ObjNum 1266 snap 397 seq_num 1266) dirty exists 2026-02-17T06:09:16.036 INFO:tasks.rados.rados.0.trial115.stdout:3713: left oid 17 (ObjNum 1266 snap 397 seq_num 1266) 2026-02-17T06:09:16.036 INFO:tasks.rados.rados.0.trial115.stdout:3714: expect (ObjNum 1022 snap 297 seq_num 1022) 2026-02-17T06:09:16.151 INFO:tasks.rados.rados.0.trial115.stdout:3716: finishing write tid 1 to trial11545420-32 2026-02-17T06:09:16.151 INFO:tasks.rados.rados.0.trial115.stdout:3713: done (4 left) 2026-02-17T06:09:16.151 INFO:tasks.rados.rados.0.trial115.stdout:3714: done (3 left) 2026-02-17T06:09:16.151 INFO:tasks.rados.rados.0.trial115.stdout:3720: write oid 20 current snap is 397 2026-02-17T06:09:16.152 INFO:tasks.rados.rados.0.trial115.stdout:3720: seq_num 1268 ranges {511276=730330,1800344=559437,3021296=289980} 2026-02-17T06:09:16.156 INFO:tasks.rados.rados.0.trial115.stdout:3720: writing trial11545420-20 from 511276 to 1241606 tid 1 2026-02-17T06:09:16.160 INFO:tasks.rados.rados.0.trial115.stdout:3720: writing trial11545420-20 from 1800344 to 2359781 tid 2 2026-02-17T06:09:16.164 INFO:tasks.rados.rados.0.trial115.stdout:3720: writing trial11545420-20 from 3021296 to 3311276 tid 3 2026-02-17T06:09:16.164 INFO:tasks.rados.rados.0.trial115.stdout:3721: delete oid 45 current snap is 397 2026-02-17T06:09:16.164 INFO:tasks.rados.rados.0.trial115.stdout:3716: finishing write tid 2 to trial11545420-32 2026-02-17T06:09:16.164 INFO:tasks.rados.rados.0.trial115.stdout:3716: finishing write tid 3 to trial11545420-32 2026-02-17T06:09:16.164 INFO:tasks.rados.rados.0.trial115.stdout:3716: finishing write tid 4 to trial11545420-32 2026-02-17T06:09:16.164 INFO:tasks.rados.rados.0.trial115.stdout:3716: finishing write tid 5 to trial11545420-32 2026-02-17T06:09:16.164 INFO:tasks.rados.rados.0.trial115.stdout:3716: oid 32 updating version 0 to 1478 2026-02-17T06:09:16.165 INFO:tasks.rados.rados.0.trial115.stdout:3716: oid 32 version 1478 is already newer than 1475 2026-02-17T06:09:16.165 INFO:tasks.rados.rados.0.trial115.stdout:3716: oid 32 version 1478 is already newer than 1477 2026-02-17T06:09:16.165 INFO:tasks.rados.rados.0.trial115.stdout:3716: oid 32 version 1478 is already newer than 1476 2026-02-17T06:09:16.165 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1478 (ObjNum 1267 snap 397 seq_num 1267) dirty exists 2026-02-17T06:09:16.165 INFO:tasks.rados.rados.0.trial115.stdout:3716: left oid 32 (ObjNum 1267 snap 397 seq_num 1267) 2026-02-17T06:09:16.165 INFO:tasks.rados.rados.0.trial115.stdout:3717: expect (ObjNum 1238 snap 383 seq_num 1238) 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3719: finishing rollback tid 1 to trial11545420-48 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 0 (ObjNum 289 snap 0 seq_num 311579051) dirty dne 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3720: finishing write tid 1 to trial11545420-20 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3720: finishing write tid 2 to trial11545420-20 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3720: finishing write tid 3 to trial11545420-20 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3716: done (4 left) 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3717: done (3 left) 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3719: done (2 left) 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3721: done (1 left) 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3722: snap_create 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3720: finishing write tid 4 to trial11545420-20 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3720: finishing write tid 5 to trial11545420-20 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3720: oid 20 updating version 0 to 1718 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3720: oid 20 updating version 1718 to 1719 2026-02-17T06:09:16.232 INFO:tasks.rados.rados.0.trial115.stdout:3720: oid 20 updating version 1719 to 1721 2026-02-17T06:09:16.233 INFO:tasks.rados.rados.0.trial115.stdout:3720: oid 20 version 1721 is already newer than 1720 2026-02-17T06:09:16.233 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1721 (ObjNum 1268 snap 397 seq_num 1268) dirty exists 2026-02-17T06:09:16.233 INFO:tasks.rados.rados.0.trial115.stdout:3720: left oid 20 (ObjNum 1268 snap 397 seq_num 1268) 2026-02-17T06:09:17.037 INFO:tasks.rados.rados.0.trial115.stdout:3720: done (1 left) 2026-02-17T06:09:17.037 INFO:tasks.rados.rados.0.trial115.stdout:3722: done (0 left) 2026-02-17T06:09:17.037 INFO:tasks.rados.rados.0.trial115.stdout:3723: snap_remove snap 394 2026-02-17T06:09:18.011 INFO:tasks.rados.rados.0.trial115.stdout:3723: done (0 left) 2026-02-17T06:09:18.011 INFO:tasks.rados.rados.0.trial115.stdout:3724: rollback oid 16 current snap is 398 2026-02-17T06:09:18.011 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 16 to 387 2026-02-17T06:09:18.012 INFO:tasks.rados.rados.0.trial115.stdout:3725: rollback oid 29 current snap is 398 2026-02-17T06:09:18.012 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 29 to 397 2026-02-17T06:09:18.012 INFO:tasks.rados.rados.0.trial115.stdout:3726: snap_remove snap 384 2026-02-17T06:09:18.420 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:18.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.489+0000 7f7dd6f6a640 10 monclient: get_monmap_and_config 2026-02-17T06:09:18.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.489+0000 7f7dd6f6a640 10 monclient: build_initial_monmap 2026-02-17T06:09:18.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.489+0000 7f7dd6f6a640 10 monclient: monmap: 2026-02-17T06:09:18.490 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:18.490 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:18.490 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:18.490784+0000 2026-02-17T06:09:18.490 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:18.490784+0000 2026-02-17T06:09:18.491 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:18.491 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:18.491 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:18.491 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:18.491 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:18.491 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:18.491 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:18.491 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:18.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.489+0000 7f7dd6f6a640 1 Processor -- start 2026-02-17T06:09:18.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 1 -- start start 2026-02-17T06:09:18.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient: init 2026-02-17T06:09:18.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:18.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7dd006c890 0x7f7dd006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient(hunting): picked mon.noname-c con 0x7f7dd006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 0x7f7dd006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient(hunting): picked mon.noname-a con 0x7f7dd009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7dd0134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd006daf0 con 0x7f7dd0134df0 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd00a0880 con 0x7f7dd009c4a0 2026-02-17T06:09:18.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd006d6f0 con 0x7f7dd006c890 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd6f6a640 10 monclient(hunting): authenticate will time out at 2004.044382s 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 0x7f7dd006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 0x7f7dd006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51974/0 (socket says 10.20.193.78:51974) 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 1 -- 10.20.193.78:0/2428996289 learned_addr learned my addr 10.20.193.78:0/2428996289 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 10 monclient(hunting): get_auth_request con 0x7f7dd009c4a0 auth_method 0 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:18.503 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 1 -- 10.20.193.78:0/2428996289 >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 msgr2=0x7f7dd006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 1 -- 10.20.193.78:0/2428996289 >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 msgr2=0x7f7dd006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 1 --2- 10.20.193.78:0/2428996289 >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 0x7f7dd006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 1 --2- 10.20.193.78:0/2428996289 >> v2:10.20.193.115:3300/0 conn(0x7f7dd006c890 0x7f7dd006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dc7fff640 1 --2- 10.20.193.78:0/2428996289 >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 0x7f7dd006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient(hunting): get_auth_request con 0x7f7dd006c890 auth_method 0 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:18.504 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient(hunting): handle_auth_done global_id 274168 payload 274 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 1 -- 10.20.193.78:0/2428996289 >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 msgr2=0x7f7dd006d1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 1 --2- 10.20.193.78:0/2428996289 >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 0x7f7dd006d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 1 -- 10.20.193.78:0/2428996289 >> v1:10.20.193.115:6789/0 conn(0x7f7dd0134df0 legacy=0x7f7dd0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient: _finish_hunting 0 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 1 monclient: found mon.noname-c 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:18.505 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.490+0000 7f7dd4cdf640 1 -- 10.20.193.78:0/2428996289 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd009cc80 con 0x7f7dd006c890 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd4cdf640 1 -- 10.20.193.78:0/2428996289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 msgr2=0x7f7dd006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd4cdf640 1 --2- 10.20.193.78:0/2428996289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 0x7f7dd006cc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7dd006a480 tx=0x7f7dc002ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b949c68fcff1e0e0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 1 -- 10.20.193.78:0/2428996289 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc003c070 con 0x7f7dd006c890 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: dump: 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:18.506 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: _finish_auth 0 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: _check_auth_tickets 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:18.507 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 5 monclient: authenticate success, global_id 274168 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 1 -- 10.20.193.78:0/2428996289 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc002fbd0 con 0x7f7dd006c890 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 1 -- 10.20.193.78:0/2428996289 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0037440 con 0x7f7dd006c890 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dc77fe640 10 monclient: dump: 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:18.508 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 4 set_mon_vals no callback set 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 10 monclient: shutdown 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2428996289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 msgr2=0x7f7dd006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2428996289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 0x7f7dd006cc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7dd006a480 tx=0x7f7dc002ed50 comp rx=0 tx=0).stop 2026-02-17T06:09:18.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2428996289 shutdown_connections 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2428996289 >> v2:10.20.193.78:3300/0 conn(0x7f7dd009c4a0 0x7f7dd006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2428996289 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 0x7f7dd006cc70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2428996289 >> 10.20.193.78:0/2428996289 conn(0x7f7dd00a6620 msgr2=0x7f7dd00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.493+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2428996289 shutdown_connections 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2428996289 wait complete. 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 10 monclient: build_initial_monmap 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 10 monclient: monmap: 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:18.510 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 1 Processor -- start 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 1 -- start start 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 10 monclient: init 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:09:18.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 0x7f7dd01cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.494+0000 7f7dd6f6a640 10 monclient(hunting): picked mon.noname-b con 0x7f7dd006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 0x7f7dd01cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 10 monclient(hunting): picked mon.noname-a con 0x7f7dd009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7dd0134df0 0x7f7dd01217a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 10 monclient(hunting): picked mon.noname-c con 0x7f7dd0134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00a01a0 con 0x7f7dd009c4a0 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00a0020 con 0x7f7dd006c890 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd009be20 con 0x7f7dd0134df0 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:18.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd6f6a640 10 monclient(hunting): authenticate will time out at 2004.049029s 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 0x7f7dd01cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7dd0134df0 0x7f7dd01217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 0x7f7dd01cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:51976/0 (socket says 10.20.193.78:51976) 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 1 -- 10.20.193.78:0/2729769095 learned_addr learned my addr 10.20.193.78:0/2729769095 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 10 monclient(hunting): get_auth_request con 0x7f7dd009c4a0 auth_method 0 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient(hunting): get_auth_request con 0x7f7dd0134df0 auth_method 0 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:18.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 1 -- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 msgr2=0x7f7dd01cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 1 -- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 msgr2=0x7f7dd01cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 0x7f7dd01cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dc7fff640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 0x7f7dd01cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd4cdf640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 0x7f7dd01cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd4cdf640 10 monclient(hunting): get_auth_request con 0x7f7dd006c890 auth_method 0 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd4cdf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd4cdf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:18.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd4cdf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient(hunting): handle_auth_done global_id 264299 payload 824 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 1 -- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 msgr2=0x7f7dd01cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 0x7f7dd01cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 1 -- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 msgr2=0x7f7dd01cd870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 0x7f7dd01cd870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient: _finish_hunting 0 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 1 monclient: found mon.noname-c 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.495+0000 7f7dd54e0640 1 -- 10.20.193.78:0/2729769095 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd0125450 con 0x7f7dd0134df0 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd54e0640 1 -- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7dd0134df0 msgr2=0x7f7dd01217a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:18.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd54e0640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7dd0134df0 0x7f7dd01217a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00caf0 tx=0x7f7dcc004a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 1 -- 10.20.193.78:0/2729769095 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc016030 con 0x7f7dd0134df0 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: dump: 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:18.516 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: _finish_auth 0 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: _check_auth_tickets 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd6f6a640 5 monclient: authenticate success, global_id 264299 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 1 -- 10.20.193.78:0/2729769095 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dcc00ac50 con 0x7f7dd0134df0 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 1 -- 10.20.193.78:0/2729769095 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc004a80 con 0x7f7dd0134df0 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc6ffd640 4 set_mon_vals no callback set 2026-02-17T06:09:18.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dc57fa640 10 monclient: dump: 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd6f6a640 10 monclient: _renew_subs 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd6f6a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:18.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd0123fa0 con 0x7f7dd0134df0 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd6f6a640 10 monclient: _renew_subs 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd6f6a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.497+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd00726c0 con 0x7f7dd0134df0 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dd6f6a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dd6f6a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dd6f6a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd0074980 con 0x7f7dd0134df0 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dc57fa640 1 -- 10.20.193.78:0/2729769095 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7dcc023060 con 0x7f7dd0134df0 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dc57fa640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7da003deb0 0x7f7da0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:18.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dc57fa640 1 -- 10.20.193.78:0/2729769095 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(821..821 src has 525..821) ==== 4823+0+0 (secure 0 0 0) 0x7f7dcc052ab0 con 0x7f7dd0134df0 2026-02-17T06:09:18.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dd4cdf640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7da003deb0 0x7f7da0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:18.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.498+0000 7f7dd4cdf640 10 monclient: get_auth_request con 0x7f7da003deb0 auth_method 0 2026-02-17T06:09:18.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.500+0000 7f7dc57fa640 1 -- 10.20.193.78:0/2729769095 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7dcc01b410 con 0x7f7dd0134df0 2026-02-17T06:09:18.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.500+0000 7f7dc57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:18.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.500+0000 7f7dc57fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:18.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.501+0000 7f7dd4cdf640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7da003deb0 0x7f7da0040370 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0074de0 tx=0x7f7dc0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:18.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.613+0000 7f7dd6f6a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:18.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.613+0000 7f7dd6f6a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:18.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.613+0000 7f7dd6f6a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:18.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.613+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7dd00a1730 con 0x7f7dd0134df0 2026-02-17T06:09:18.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.613+0000 7f7dc57fa640 1 -- 10.20.193.78:0/2729769095 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f7dcc019360 con 0x7f7dd0134df0 2026-02-17T06:09:18.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.613+0000 7f7dc57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:18.614 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.613+0000 7f7dc57fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:18.614 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":254,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":30,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7da003deb0 msgr2=0x7f7da0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7da003deb0 0x7f7da0040370 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0074de0 tx=0x7f7dc0033000 comp rx=0 tx=0).stop 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 10 monclient: shutdown 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7dd0134df0 msgr2=0x7f7dd01217a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7dd0134df0 0x7f7dd01217a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00caf0 tx=0x7f7dcc004a10 comp rx=0 tx=0).stop 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 shutdown_connections 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7dd0134df0 0x7f7dd01217a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7dd009c4a0 0x7f7dd01cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7da003deb0 0x7f7da0040370 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 --2- 10.20.193.78:0/2729769095 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7dd006c890 0x7f7dd01cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 >> 10.20.193.78:0/2729769095 conn(0x7f7dd00a6620 msgr2=0x7f7dd00a1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 shutdown_connections 2026-02-17T06:09:18.615 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:18.614+0000 7f7dd6f6a640 1 -- 10.20.193.78:0/2729769095 wait complete. 2026-02-17T06:09:19.014 INFO:tasks.rados.rados.0.trial115.stdout:3726: done (2 left) 2026-02-17T06:09:19.014 INFO:tasks.rados.rados.0.trial115.stdout:3727: delete oid 35 current snap is 398 2026-02-17T06:09:19.014 INFO:tasks.rados.rados.0.trial115.stdout:3725: finishing rollback tid 1 to trial11545420-29 2026-02-17T06:09:19.014 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1497 (ObjNum 65 snap 0 seq_num 311572794) dirty dne 2026-02-17T06:09:19.014 INFO:tasks.rados.rados.0.trial115.stdout:3724: finishing rollback tid 0 to trial11545420-16 2026-02-17T06:09:19.014 INFO:tasks.rados.rados.0.trial115.stdout:3724: finishing rollback tid 1 to trial11545420-16 2026-02-17T06:09:19.014 INFO:tasks.rados.rados.0.trial115.stdout:3724: finishing rollback tid 2 to trial11545420-16 2026-02-17T06:09:19.014 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1968 (ObjNum 1218 snap 378 seq_num 1218) dirty exists 2026-02-17T06:09:19.016 INFO:tasks.rados.rados.0.trial115.stdout:3724: done (2 left) 2026-02-17T06:09:19.016 INFO:tasks.rados.rados.0.trial115.stdout:3725: done (1 left) 2026-02-17T06:09:19.016 INFO:tasks.rados.rados.0.trial115.stdout:3727: done (0 left) 2026-02-17T06:09:19.017 INFO:tasks.rados.rados.0.trial115.stdout:3728: delete oid 6 current snap is 398 2026-02-17T06:09:19.020 INFO:tasks.rados.rados.0.trial115.stdout:3728: done (0 left) 2026-02-17T06:09:19.020 INFO:tasks.rados.rados.0.trial115.stdout:3729: write (excl) oid 32 current snap is 398 2026-02-17T06:09:19.020 INFO:tasks.rados.rados.0.trial115.stdout:3729: seq_num 1272 ranges {578504=747679,2034272=566775,3130988=247516} 2026-02-17T06:09:19.025 INFO:tasks.rados.rados.0.trial115.stdout:3729: writing trial11545420-32 from 578504 to 1326183 tid 1 2026-02-17T06:09:19.029 INFO:tasks.rados.rados.0.trial115.stdout:3729: writing trial11545420-32 from 2034272 to 2601047 tid 2 2026-02-17T06:09:19.033 INFO:tasks.rados.rados.0.trial115.stdout:3729: writing trial11545420-32 from 3130988 to 3378504 tid 3 2026-02-17T06:09:19.033 INFO:tasks.rados.rados.0.trial115.stdout:3730: rollback oid 43 current snap is 398 2026-02-17T06:09:19.033 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 389 2026-02-17T06:09:19.033 INFO:tasks.rados.rados.0.trial115.stdout:3729: finishing write tid 1 to trial11545420-32 2026-02-17T06:09:19.033 INFO:tasks.rados.rados.0.trial115.stdout:3729: finishing write tid 2 to trial11545420-32 2026-02-17T06:09:19.033 INFO:tasks.rados.rados.0.trial115.stdout:3731: write oid 24 current snap is 398 2026-02-17T06:09:19.033 INFO:tasks.rados.rados.0.trial115.stdout:3731: seq_num 1273 ranges {595311=422928,1576401=710433,3042058=353253} 2026-02-17T06:09:19.036 INFO:tasks.rados.rados.0.trial115.stdout:3731: writing trial11545420-24 from 595311 to 1018239 tid 1 2026-02-17T06:09:19.041 INFO:tasks.rados.rados.0.trial115.stdout:3731: writing trial11545420-24 from 1576401 to 2286834 tid 2 2026-02-17T06:09:19.046 INFO:tasks.rados.rados.0.trial115.stdout:3731: writing trial11545420-24 from 3042058 to 3395311 tid 3 2026-02-17T06:09:19.046 INFO:tasks.rados.rados.0.trial115.stdout:3732: write (excl) oid 40 current snap is 398 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3729: finishing write tid 3 to trial11545420-32 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3729: finishing write tid 4 to trial11545420-32 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3729: finishing write tid 5 to trial11545420-32 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3729: oid 32 updating version 0 to 1481 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3729: oid 32 updating version 1481 to 1482 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3729: oid 32 version 1482 is already newer than 1480 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3729: oid 32 version 1482 is already newer than 1479 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1482 (ObjNum 1272 snap 398 seq_num 1272) dirty exists 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3729: left oid 32 (ObjNum 1272 snap 398 seq_num 1272) 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3730: finishing rollback tid 0 to trial11545420-43 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3730: finishing rollback tid 1 to trial11545420-43 2026-02-17T06:09:19.047 INFO:tasks.rados.rados.0.trial115.stdout:3732: seq_num 1274 ranges {612118=498177,1634883=570444,2669481=706662} 2026-02-17T06:09:19.050 INFO:tasks.rados.rados.0.trial115.stdout:3732: writing trial11545420-40 from 612118 to 1110295 tid 1 2026-02-17T06:09:19.054 INFO:tasks.rados.rados.0.trial115.stdout:3732: writing trial11545420-40 from 1634883 to 2205327 tid 2 2026-02-17T06:09:19.059 INFO:tasks.rados.rados.0.trial115.stdout:3732: writing trial11545420-40 from 2669481 to 3376143 tid 3 2026-02-17T06:09:19.059 INFO:tasks.rados.rados.0.trial115.stdout:3729: done (3 left) 2026-02-17T06:09:19.059 INFO:tasks.rados.rados.0.trial115.stdout:3733: write oid 8 current snap is 398 2026-02-17T06:09:19.059 INFO:tasks.rados.rados.0.trial115.stdout:3733: seq_num 1275 ranges {628925=573426,1693365=714102,3096904=332021} 2026-02-17T06:09:19.063 INFO:tasks.rados.rados.0.trial115.stdout:3733: writing trial11545420-8 from 628925 to 1202351 tid 1 2026-02-17T06:09:19.068 INFO:tasks.rados.rados.0.trial115.stdout:3733: writing trial11545420-8 from 1693365 to 2407467 tid 2 2026-02-17T06:09:19.072 INFO:tasks.rados.rados.0.trial115.stdout:3733: writing trial11545420-8 from 3096904 to 3428925 tid 3 2026-02-17T06:09:19.072 INFO:tasks.rados.rados.0.trial115.stdout:3734: write oid 34 current snap is 398 2026-02-17T06:09:19.072 INFO:tasks.rados.rados.0.trial115.stdout:3730: finishing rollback tid 2 to trial11545420-43 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1724 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: finishing write tid 1 to trial11545420-24 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: finishing write tid 2 to trial11545420-24 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: finishing write tid 3 to trial11545420-24 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: finishing write tid 4 to trial11545420-24 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: finishing write tid 5 to trial11545420-24 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: oid 24 updating version 0 to 1925 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: oid 24 version 1925 is already newer than 1924 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: oid 24 updating version 1925 to 1926 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: oid 24 updating version 1926 to 1927 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1927 (ObjNum 1273 snap 398 seq_num 1273) dirty exists 2026-02-17T06:09:19.073 INFO:tasks.rados.rados.0.trial115.stdout:3731: left oid 24 (ObjNum 1273 snap 398 seq_num 1273) 2026-02-17T06:09:19.074 INFO:tasks.rados.rados.0.trial115.stdout:3732: finishing write tid 1 to trial11545420-40 2026-02-17T06:09:19.074 INFO:tasks.rados.rados.0.trial115.stdout:3734: seq_num 1276 ranges {645732=648675,1751847=574113,3124327=321405} 2026-02-17T06:09:19.076 INFO:tasks.rados.rados.0.trial115.stdout:3734: writing trial11545420-34 from 645732 to 1294407 tid 1 2026-02-17T06:09:19.080 INFO:tasks.rados.rados.0.trial115.stdout:3734: writing trial11545420-34 from 1751847 to 2325960 tid 2 2026-02-17T06:09:19.085 INFO:tasks.rados.rados.0.trial115.stdout:3734: writing trial11545420-34 from 3124327 to 3445732 tid 3 2026-02-17T06:09:19.085 INFO:tasks.rados.rados.0.trial115.stdout:3730: done (4 left) 2026-02-17T06:09:19.085 INFO:tasks.rados.rados.0.trial115.stdout:3731: done (3 left) 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3735: copy_from oid 44 from oid 15 current snap is 398 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: finishing write tid 2 to trial11545420-40 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: finishing write tid 3 to trial11545420-40 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: finishing write tid 4 to trial11545420-40 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: finishing write tid 5 to trial11545420-40 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: oid 40 updating version 0 to 1503 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: oid 40 version 1503 is already newer than 1502 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: oid 40 updating version 1503 to 1504 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: oid 40 version 1504 is already newer than 1501 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1504 (ObjNum 1274 snap 398 seq_num 1274) dirty exists 2026-02-17T06:09:19.086 INFO:tasks.rados.rados.0.trial115.stdout:3732: left oid 40 (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: finishing write tid 1 to trial11545420-8 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: finishing write tid 2 to trial11545420-8 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: finishing write tid 3 to trial11545420-8 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: finishing write tid 4 to trial11545420-8 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: finishing write tid 5 to trial11545420-8 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: oid 8 updating version 0 to 1535 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: oid 8 updating version 1535 to 1537 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: oid 8 version 1537 is already newer than 1536 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: oid 8 updating version 1537 to 1538 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1538 (ObjNum 1275 snap 398 seq_num 1275) dirty exists 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3733: left oid 8 (ObjNum 1275 snap 398 seq_num 1275) 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3734: finishing write tid 1 to trial11545420-34 2026-02-17T06:09:19.087 INFO:tasks.rados.rados.0.trial115.stdout:3734: finishing write tid 2 to trial11545420-34 2026-02-17T06:09:19.088 INFO:tasks.rados.rados.0.trial115.stdout:3732: done (3 left) 2026-02-17T06:09:19.088 INFO:tasks.rados.rados.0.trial115.stdout:3733: done (2 left) 2026-02-17T06:09:19.088 INFO:tasks.rados.rados.0.trial115.stdout:3736: snap_remove snap 397 2026-02-17T06:09:20.043 INFO:tasks.rados.rados.0.trial115.stdout:3736: done (2 left) 2026-02-17T06:09:20.043 INFO:tasks.rados.rados.0.trial115.stdout:3737: snap_remove snap 387 2026-02-17T06:09:21.046 INFO:tasks.rados.rados.0.trial115.stdout:3737: done (2 left) 2026-02-17T06:09:21.046 INFO:tasks.rados.rados.0.trial115.stdout:3738: delete oid 12 current snap is 398 2026-02-17T06:09:21.046 INFO:tasks.rados.rados.0.trial115.stdout:3734: finishing write tid 3 to trial11545420-34 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3734: finishing write tid 4 to trial11545420-34 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3734: finishing write tid 5 to trial11545420-34 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3734: oid 34 updating version 0 to 1498 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3734: oid 34 updating version 1498 to 1499 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3734: oid 34 updating version 1499 to 1500 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3734: oid 34 updating version 1500 to 1501 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1501 (ObjNum 1276 snap 398 seq_num 1276) dirty exists 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3734: left oid 34 (ObjNum 1276 snap 398 seq_num 1276) 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3735: finishing copy_from racing read to trial11545420-44 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:3735: finishing copy_from to trial11545420-44 2026-02-17T06:09:21.047 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1900 (ObjNum 1168 snap 359 seq_num 1168) dirty exists 2026-02-17T06:09:21.048 INFO:tasks.rados.rados.0.trial115.stdout:3734: done (2 left) 2026-02-17T06:09:21.048 INFO:tasks.rados.rados.0.trial115.stdout:3735: done (1 left) 2026-02-17T06:09:21.048 INFO:tasks.rados.rados.0.trial115.stdout:3738: done (0 left) 2026-02-17T06:09:21.048 INFO:tasks.rados.rados.0.trial115.stdout:3739: delete oid 21 current snap is 398 2026-02-17T06:09:21.050 INFO:tasks.rados.rados.0.trial115.stdout:3739: done (0 left) 2026-02-17T06:09:21.050 INFO:tasks.rados.rados.0.trial115.stdout:3740: rollback oid 8 current snap is 398 2026-02-17T06:09:21.050 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 8 to 396 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3741: read oid 26 snap -1 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3741: expect (ObjNum 1256 snap 392 seq_num 1256) 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3742: read oid 41 snap -1 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3742: expect (ObjNum 1247 snap 387 seq_num 1247) 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3743: rollback oid 9 current snap is 398 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 9 to 392 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3744: read oid 27 snap 396 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3744: expect (ObjNum 1238 snap 383 seq_num 1238) 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3745: write oid 5 current snap is 398 2026-02-17T06:09:21.051 INFO:tasks.rados.rados.0.trial115.stdout:3745: seq_num 1279 ranges {696153=590775,1927293=721440,3206596=289557} 2026-02-17T06:09:21.054 INFO:tasks.rados.rados.0.trial115.stdout:3745: writing trial11545420-5 from 696153 to 1286928 tid 1 2026-02-17T06:09:21.059 INFO:tasks.rados.rados.0.trial115.stdout:3745: writing trial11545420-5 from 1927293 to 2648733 tid 2 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3745: writing trial11545420-5 from 3206596 to 3496153 tid 3 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3740: finishing rollback tid 0 to trial11545420-8 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3743: finishing rollback tid 0 to trial11545420-9 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3740: finishing rollback tid 1 to trial11545420-8 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3746: read oid 28 snap -1 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3746: expect deleted 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3747: copy_from oid 43 from oid 34 current snap is 398 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3743: finishing rollback tid 1 to trial11545420-9 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1987 (ObjNum 529 snap 0 seq_num 308354476) dirty dne 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3740: finishing rollback tid 2 to trial11545420-8 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1541 (ObjNum 1162 snap 355 seq_num 1162) dirty exists 2026-02-17T06:09:21.063 INFO:tasks.rados.rados.0.trial115.stdout:3741: expect (ObjNum 1256 snap 392 seq_num 1256) 2026-02-17T06:09:21.138 INFO:tasks.rados.rados.0.trial115.stdout:3744: expect (ObjNum 1238 snap 383 seq_num 1238) 2026-02-17T06:09:21.206 INFO:tasks.rados.rados.0.trial115.stdout:3742: expect (ObjNum 1247 snap 387 seq_num 1247) 2026-02-17T06:09:21.276 INFO:tasks.rados.rados.0.trial115.stdout:3745: finishing write tid 1 to trial11545420-5 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3745: finishing write tid 2 to trial11545420-5 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3740: done (7 left) 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3741: done (6 left) 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3742: done (5 left) 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3743: done (4 left) 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3744: done (3 left) 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3748: read oid 33 snap -1 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3748: expect deleted 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3749: rollback oid 24 current snap is 398 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 396 2026-02-17T06:09:21.277 INFO:tasks.rados.rados.0.trial115.stdout:3745: finishing write tid 3 to trial11545420-5 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3745: finishing write tid 4 to trial11545420-5 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3745: finishing write tid 5 to trial11545420-5 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3745: oid 5 updating version 0 to 1655 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3745: oid 5 version 1655 is already newer than 1654 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3745: oid 5 updating version 1655 to 1656 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3745: oid 5 version 1656 is already newer than 1653 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1656 (ObjNum 1279 snap 398 seq_num 1279) dirty exists 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3745: left oid 5 (ObjNum 1279 snap 398 seq_num 1279) 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3747: finishing copy_from racing read to trial11545420-43 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3747: finishing copy_from to trial11545420-43 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1501 (ObjNum 1276 snap 398 seq_num 1276) dirty exists 2026-02-17T06:09:21.278 INFO:tasks.rados.rados.0.trial115.stdout:3745: done (4 left) 2026-02-17T06:09:21.279 INFO:tasks.rados.rados.0.trial115.stdout:3746: done (3 left) 2026-02-17T06:09:21.279 INFO:tasks.rados.rados.0.trial115.stdout:3747: done (2 left) 2026-02-17T06:09:21.279 INFO:tasks.rados.rados.0.trial115.stdout:3750: snap_remove snap 393 2026-02-17T06:09:21.569 INFO:tasks.rados.rados.0.trial115.stdout:3750: done (2 left) 2026-02-17T06:09:21.569 INFO:tasks.rados.rados.0.trial115.stdout:3751: write oid 46 current snap is 398 2026-02-17T06:09:21.569 INFO:tasks.rados.rados.0.trial115.stdout:3751: seq_num 1280 ranges {712960=666024,1985775=581451,3234019=278941} 2026-02-17T06:09:21.574 INFO:tasks.rados.rados.0.trial115.stdout:3751: writing trial11545420-46 from 712960 to 1378984 tid 1 2026-02-17T06:09:21.579 INFO:tasks.rados.rados.0.trial115.stdout:3751: writing trial11545420-46 from 1985775 to 2567226 tid 2 2026-02-17T06:09:21.582 INFO:tasks.rados.rados.0.trial115.stdout:3751: writing trial11545420-46 from 3234019 to 3512960 tid 3 2026-02-17T06:09:21.582 INFO:tasks.rados.rados.0.trial115.stdout:3752: snap_create 2026-02-17T06:09:21.582 INFO:tasks.rados.rados.0.trial115.stdout:3749: finishing rollback tid 0 to trial11545420-24 2026-02-17T06:09:21.582 INFO:tasks.rados.rados.0.trial115.stdout:3749: finishing rollback tid 1 to trial11545420-24 2026-02-17T06:09:21.582 INFO:tasks.rados.rados.0.trial115.stdout:3749: finishing rollback tid 2 to trial11545420-24 2026-02-17T06:09:21.582 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1938 (ObjNum 1253 snap 392 seq_num 1253) dirty exists 2026-02-17T06:09:21.583 INFO:tasks.rados.rados.0.trial115.stdout:3751: finishing write tid 1 to trial11545420-46 2026-02-17T06:09:21.583 INFO:tasks.rados.rados.0.trial115.stdout:3751: finishing write tid 2 to trial11545420-46 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:3751: finishing write tid 3 to trial11545420-46 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:3751: finishing write tid 4 to trial11545420-46 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:3751: finishing write tid 5 to trial11545420-46 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:3751: oid 46 updating version 0 to 1992 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:3751: oid 46 updating version 1992 to 1994 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:3751: oid 46 version 1994 is already newer than 1993 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:3751: oid 46 version 1994 is already newer than 1991 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 46 v 1994 (ObjNum 1280 snap 398 seq_num 1280) dirty exists 2026-02-17T06:09:21.584 INFO:tasks.rados.rados.0.trial115.stdout:3751: left oid 46 (ObjNum 1280 snap 398 seq_num 1280) 2026-02-17T06:09:21.622 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:21.691 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient: get_monmap_and_config 2026-02-17T06:09:21.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient: build_initial_monmap 2026-02-17T06:09:21.692 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient: monmap: 2026-02-17T06:09:21.692 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:21.692 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.692 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:21.689299+0000 2026-02-17T06:09:21.692 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:21.689299+0000 2026-02-17T06:09:21.692 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:21.692 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 1 Processor -- start 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 1 -- start start 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient: init 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:21.693 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f37281384d0 0x7f3728070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient(hunting): picked mon.noname-c con 0x7f37281384d0 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f37281388b0 0x7f3728138d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient(hunting): picked mon.noname-a con 0x7f37281388b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f37280a3220 addr v1:10.20.193.78:6789/0 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3728073f70 con 0x7f37280a3220 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f37280a4b20 con 0x7f37281388b0 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f37280a4f70 con 0x7f37281384d0 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:21.694 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f372d6f2640 10 monclient(hunting): authenticate will time out at 2007.242892s 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f37267fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f37281388b0 0x7f3728138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f37267fc640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f37281388b0 0x7f3728138d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52006/0 (socket says 10.20.193.78:52006) 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f37277fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f37280a3220 0x7f37280a3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54460/0 (socket says 10.20.193.78:54460) 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.688+0000 7f37267fc640 1 -- 10.20.193.78:0/366445851 learned_addr learned my addr 10.20.193.78:0/366445851 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient(hunting): get_auth_request con 0x7f37281388b0 auth_method 0 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f3726ffd640 1 --2- 10.20.193.78:0/366445851 >> v2:10.20.193.115:3300/0 conn(0x7f37281384d0 0x7f3728070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f3726ffd640 10 monclient(hunting): get_auth_request con 0x7f37281384d0 auth_method 0 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f3726ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f3726ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f3726ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient(hunting): handle_auth_done global_id 284097 payload 274 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 1 -- 10.20.193.78:0/366445851 >> v2:10.20.193.115:3300/0 conn(0x7f37281384d0 msgr2=0x7f3728070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 1 --2- 10.20.193.78:0/366445851 >> v2:10.20.193.115:3300/0 conn(0x7f37281384d0 0x7f3728070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 1 -- 10.20.193.78:0/366445851 >> v1:10.20.193.78:6789/0 conn(0x7f37280a3220 legacy=0x7f37280a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient: _finish_hunting 0 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 1 monclient: found mon.noname-a 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:21.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:21.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.689+0000 7f37267fc640 1 -- 10.20.193.78:0/366445851 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372806f870 con 0x7f37281388b0 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f37267fc640 1 -- 10.20.193.78:0/366445851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37281388b0 msgr2=0x7f3728138d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f37267fc640 1 --2- 10.20.193.78:0/366445851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37281388b0 0x7f3728138d00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3728073270 tx=0x7f371002ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9997a60dffd19323 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 1 -- 10.20.193.78:0/366445851 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371003c070 con 0x7f37281388b0 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: dump: 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.704 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: _finish_auth 0 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:21.705 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 1 -- 10.20.193.78:0/366445851 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371002fd10 con 0x7f37281388b0 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f372d6f2640 5 monclient: authenticate success, global_id 284097 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f372d6f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 1 -- 10.20.193.78:0/366445851 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710037660 con 0x7f37281388b0 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f372d6f2640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f372d6f2640 4 set_mon_vals no callback set 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f3725ffb640 10 monclient: dump: 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.706 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f372d6f2640 10 monclient: shutdown 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f372d6f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f372d6f2640 1 -- 10.20.193.78:0/366445851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37281388b0 msgr2=0x7f3728138d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.691+0000 7f372d6f2640 1 --2- 10.20.193.78:0/366445851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37281388b0 0x7f3728138d00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3728073270 tx=0x7f371002ee90 comp rx=0 tx=0).stop 2026-02-17T06:09:21.707 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 1 -- 10.20.193.78:0/366445851 shutdown_connections 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 1 --2- 10.20.193.78:0/366445851 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37281388b0 0x7f3728138d00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 1 --2- 10.20.193.78:0/366445851 >> v2:10.20.193.115:3300/0 conn(0x7f37281384d0 0x7f3728070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 1 -- 10.20.193.78:0/366445851 >> 10.20.193.78:0/366445851 conn(0x7f372809c000 msgr2=0x7f37280a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 1 -- 10.20.193.78:0/366445851 shutdown_connections 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 1 -- 10.20.193.78:0/366445851 wait complete. 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 10 monclient: build_initial_monmap 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 10 monclient: monmap: 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:21.708 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 1 Processor -- start 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 1 -- start start 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.692+0000 7f372d6f2640 10 monclient: init 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:09:21.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37280a3220 0x7f37281cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient(hunting): picked mon.noname-a con 0x7f37280a3220 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37281384d0 0x7f37281cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient(hunting): picked mon.noname-c con 0x7f37281384d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37281388b0 0x7f37281249f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient(hunting): picked mon.noname-b con 0x7f37281388b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37280a94e0 con 0x7f37280a3220 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3728139240 con 0x7f37281388b0 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f37280a9660 con 0x7f37281384d0 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:21.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f372d6f2640 10 monclient(hunting): authenticate will time out at 2007.247089s 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37280a3220 0x7f37281cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37280a3220 0x7f37281cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52010/0 (socket says 10.20.193.78:52010) 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 -- 10.20.193.78:0/1671292486 learned_addr learned my addr 10.20.193.78:0/1671292486 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient(hunting): get_auth_request con 0x7f37280a3220 auth_method 0 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37267fc640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37281384d0 0x7f37281cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37267fc640 10 monclient(hunting): get_auth_request con 0x7f37281384d0 auth_method 0 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37267fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37267fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37267fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37277fe640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37281388b0 0x7f37281249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37267fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37267fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37267fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37277fe640 10 monclient(hunting): get_auth_request con 0x7f37281388b0 auth_method 0 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f37277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient(hunting): handle_auth_done global_id 284100 payload 824 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 -- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37281384d0 msgr2=0x7f37281cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37281384d0 0x7f37281cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 -- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37281388b0 msgr2=0x7f37281249f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37281388b0 0x7f37281249f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 monclient: found mon.noname-a 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.693+0000 7f3726ffd640 1 -- 10.20.193.78:0/1671292486 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37281286a0 con 0x7f37280a3220 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f3726ffd640 1 -- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37280a3220 msgr2=0x7f37281cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f3726ffd640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37280a3220 0x7f37281cf6a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f371c00e7c0 tx=0x7f371c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:21.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 1 -- 10.20.193.78:0/1671292486 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c017070 con 0x7f37280a3220 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: dump: 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.714 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: _finish_auth 0 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: _check_auth_tickets 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 1 -- 10.20.193.78:0/1671292486 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c00cae0 con 0x7f37280a3220 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f372d6f2640 5 monclient: authenticate success, global_id 284100 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 1 -- 10.20.193.78:0/1671292486 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c00f040 con 0x7f37280a3220 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f372d6f2640 10 monclient: _renew_subs 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f372d6f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37257fa640 4 set_mon_vals no callback set 2026-02-17T06:09:21.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3728128980 con 0x7f37280a3220 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f37077fe640 10 monclient: dump: 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.716 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f372d6f2640 10 monclient: _renew_subs 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f372d6f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.695+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37280729c0 con 0x7f37280a3220 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f372d6f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f372d6f2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f372d6f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3728070530 con 0x7f37280a3220 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f37077fe640 1 -- 10.20.193.78:0/1671292486 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f371c012050 con 0x7f37280a3220 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f37077fe640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f36f403df00 0x7f36f40403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f37077fe640 1 -- 10.20.193.78:0/1671292486 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(825..825 src has 525..825) ==== 4827+0+0 (secure 0 0 0) 0x7f371c052610 con 0x7f37280a3220 2026-02-17T06:09:21.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f37267fc640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f36f403df00 0x7f36f40403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.696+0000 7f37267fc640 10 monclient: get_auth_request con 0x7f36f403df00 auth_method 0 2026-02-17T06:09:21.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.698+0000 7f37077fe640 1 -- 10.20.193.78:0/1671292486 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f371c023d80 con 0x7f37280a3220 2026-02-17T06:09:21.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.698+0000 7f37077fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:21.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.698+0000 7f37077fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:21.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.699+0000 7f37267fc640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f36f403df00 0x7f36f40403c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3710036000 tx=0x7f3710033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:21.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.812+0000 7f372d6f2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:21.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.812+0000 7f372d6f2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:21.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.812+0000 7f372d6f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:21.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.812+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f37280744d0 con 0x7f37280a3220 2026-02-17T06:09:21.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.812+0000 7f37077fe640 1 -- 10.20.193.78:0/1671292486 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f371c0236a0 con 0x7f37280a3220 2026-02-17T06:09:21.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.812+0000 7f37077fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:21.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.812+0000 7f37077fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:21.815 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":258,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f36f403df00 msgr2=0x7f36f40403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f36f403df00 0x7f36f40403c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3710036000 tx=0x7f3710033000 comp rx=0 tx=0).stop 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 10 monclient: shutdown 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37280a3220 msgr2=0x7f37281cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37280a3220 0x7f37281cf6a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f371c00e7c0 tx=0x7f371c00ec90 comp rx=0 tx=0).stop 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 shutdown_connections 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f36f403df00 0x7f36f40403c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37281388b0 0x7f37281249f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37281384d0 0x7f37281cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 --2- 10.20.193.78:0/1671292486 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37280a3220 0x7f37281cf6a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 >> 10.20.193.78:0/1671292486 conn(0x7f372809c000 msgr2=0x7f372809f070 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:21.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 shutdown_connections 2026-02-17T06:09:21.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.813+0000 7f372d6f2640 1 -- 10.20.193.78:0/1671292486 wait complete. 2026-02-17T06:09:21.822 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:09:21.822 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:09:21.822 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:09:21.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.889+0000 7f03bfda0640 10 monclient: get_monmap_and_config 2026-02-17T06:09:21.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.889+0000 7f03bfda0640 10 monclient: build_initial_monmap 2026-02-17T06:09:21.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient: monmap: 2026-02-17T06:09:21.894 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:21.894 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.894 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:21.891227+0000 2026-02-17T06:09:21.894 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:21.891227+0000 2026-02-17T06:09:21.895 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:21.895 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.895 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:21.895 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:21.895 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:21.895 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:21.895 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:21.896 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 1 Processor -- start 2026-02-17T06:09:21.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 1 -- start start 2026-02-17T06:09:21.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient: init 2026-02-17T06:09:21.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:21.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2026-02-17T06:09:21.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f03b806c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:09:21.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f03b809c4a0 0x7f03b813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient(hunting): picked mon.noname-c con 0x7f03b809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:21.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient(hunting): picked mon.noname-b con 0x7f03b806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:21.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b806daf0 con 0x7f03b806c890 2026-02-17T06:09:21.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f03b80a0880 con 0x7f03b809c4a0 2026-02-17T06:09:21.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f03b809cc80 con 0x7f03b806d380 2026-02-17T06:09:21.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:21.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bfda0640 10 monclient(hunting): authenticate will time out at 2007.444776s 2026-02-17T06:09:21.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bdb15640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f03b806c890 0x7f03b806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54468/0 (socket says 10.20.193.78:54468) 2026-02-17T06:09:21.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bdb15640 1 -- 10.20.193.78:0/1041927431 learned_addr learned my addr 10.20.193.78:0/1041927431 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:09:21.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03be316640 1 --2- 10.20.193.78:0/1041927431 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03be316640 10 monclient(hunting): get_auth_request con 0x7f03b806d380 auth_method 0 2026-02-17T06:09:21.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03be316640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03be316640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03be316640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bd314640 1 --2- 10.20.193.78:0/1041927431 >> v2:10.20.193.115:3300/0 conn(0x7f03b809c4a0 0x7f03b813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bd314640 10 monclient(hunting): get_auth_request con 0x7f03b809c4a0 auth_method 0 2026-02-17T06:09:21.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bd314640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bd314640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03bd314640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03be316640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.890+0000 7f03be316640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03bd314640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03bd314640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03bd314640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 10 monclient(hunting): handle_auth_done global_id 264305 payload 274 2026-02-17T06:09:21.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 1 -- 10.20.193.78:0/1041927431 >> v2:10.20.193.115:3300/0 conn(0x7f03b809c4a0 msgr2=0x7f03b813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:21.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 1 --2- 10.20.193.78:0/1041927431 >> v2:10.20.193.115:3300/0 conn(0x7f03b809c4a0 0x7f03b813b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 1 -- 10.20.193.78:0/1041927431 >> v1:10.20.193.78:6789/0 conn(0x7f03b806c890 legacy=0x7f03b806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:21.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 10 monclient: _finish_hunting 0 2026-02-17T06:09:21.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 1 monclient: found mon.noname-b 2026-02-17T06:09:21.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:21.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:09:21.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.891+0000 7f03be316640 1 -- 10.20.193.78:0/1041927431 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b809cf60 con 0x7f03b806d380 2026-02-17T06:09:21.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03be316640 1 -- 10.20.193.78:0/1041927431 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 msgr2=0x7f03b813d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:21.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03be316640 1 --2- 10.20.193.78:0/1041927431 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813d8f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009d90 tx=0x7f03b402f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ab3b216cc404bf server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:21.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 1 -- 10.20.193.78:0/1041927431 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b403c070 con 0x7f03b806d380 2026-02-17T06:09:21.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:09:21.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: dump: 2026-02-17T06:09:21.918 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.918 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.918 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.919 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.919 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.919 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.919 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.919 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.919 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.920 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: _finish_auth 0 2026-02-17T06:09:21.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: _check_auth_tickets 2026-02-17T06:09:21.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:21.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bfda0640 5 monclient: authenticate success, global_id 264305 2026-02-17T06:09:21.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 1 -- 10.20.193.78:0/1041927431 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b4037440 con 0x7f03b806d380 2026-02-17T06:09:21.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bfda0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:21.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:21.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 1 -- 10.20.193.78:0/1041927431 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4037810 con 0x7f03b806d380 2026-02-17T06:09:21.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bfda0640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:21.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bfda0640 4 set_mon_vals no callback set 2026-02-17T06:09:21.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:21.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bcb13640 10 monclient: dump: 2026-02-17T06:09:21.921 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bfda0640 10 monclient: shutdown 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bfda0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:21.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bfda0640 1 -- 10.20.193.78:0/1041927431 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 msgr2=0x7f03b813d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.893+0000 7f03bfda0640 1 --2- 10.20.193.78:0/1041927431 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813d8f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009d90 tx=0x7f03b402f190 comp rx=0 tx=0).stop 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 -- 10.20.193.78:0/1041927431 shutdown_connections 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 --2- 10.20.193.78:0/1041927431 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813d8f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 --2- 10.20.193.78:0/1041927431 >> v2:10.20.193.115:3300/0 conn(0x7f03b809c4a0 0x7f03b813b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 -- 10.20.193.78:0/1041927431 >> 10.20.193.78:0/1041927431 conn(0x7f03b80a6620 msgr2=0x7f03b80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 -- 10.20.193.78:0/1041927431 shutdown_connections 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 -- 10.20.193.78:0/1041927431 wait complete. 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient: build_initial_monmap 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient: monmap: 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:21.923 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 Processor -- start 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 -- start start 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient: init 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:09:21.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03b806c890 0x7f03b813c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient(hunting): picked mon.noname-b con 0x7f03b806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient(hunting): picked mon.noname-c con 0x7f03b806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b809c4a0 0x7f03b8137380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient(hunting): picked mon.noname-a con 0x7f03b809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03b809bfa0 con 0x7f03b809c4a0 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03b809be20 con 0x7f03b806c890 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f03b813de30 con 0x7f03b806d380 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:21.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bfda0640 10 monclient(hunting): authenticate will time out at 2007.448870s 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03be316640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b809c4a0 0x7f03b8137380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03be316640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b809c4a0 0x7f03b8137380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52026/0 (socket says 10.20.193.78:52026) 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53796/0 (socket says 10.20.193.78:53796) 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03be316640 1 -- 10.20.193.78:0/2583258392 learned_addr learned my addr 10.20.193.78:0/2583258392 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03be316640 10 monclient(hunting): get_auth_request con 0x7f03b809c4a0 auth_method 0 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03be316640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03be316640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03be316640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 10 monclient(hunting): get_auth_request con 0x7f03b806d380 auth_method 0 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.894+0000 7f03bd314640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03be316640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03be316640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03be316640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 10 monclient(hunting): handle_auth_done global_id 264308 payload 824 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03b806c890 msgr2=0x7f03b813c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bdb15640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03b806c890 0x7f03b813c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03b806c890 0x7f03b813c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b809c4a0 msgr2=0x7f03b8137380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b809c4a0 0x7f03b8137380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 10 monclient: _finish_hunting 0 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 1 monclient: found mon.noname-c 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:21.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bd314640 1 -- 10.20.193.78:0/2583258392 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b813afd0 con 0x7f03b806d380 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03be316640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b809c4a0 0x7f03b8137380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bdb15640 10 monclient: get_auth_request con 0x7f03b806c890 auth_method 0 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.895+0000 7f03bdb15640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f03b806c890 0x7f03b813c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bd314640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 msgr2=0x7f03b813cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bd314640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813cd20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00e9b0 tx=0x7f03ac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac017070 con 0x7f03b806d380 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: dump: 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.928 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: _finish_auth 0 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: _check_auth_tickets 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac00cbb0 con 0x7f03b806d380 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 5 monclient: authenticate success, global_id 264308 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 10 monclient: _renew_subs 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b813b2b0 con 0x7f03b806d380 2026-02-17T06:09:21.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac013470 con 0x7f03b806d380 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a67fc640 10 monclient: dump: 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03a7fff640 4 set_mon_vals no callback set 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 10 monclient: _renew_subs 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b813da60 con 0x7f03b806d380 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b81cd170 0x7f03b81378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80a3ed0 con 0x7f03b81cd170 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bdb15640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b81cd170 0x7f03b81378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bdb15640 10 monclient: get_auth_request con 0x7f03b81cd170 auth_method 0 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bdb15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bdb15640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bdb15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bdb15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.897+0000 7f03bdb15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.898+0000 7f03bdb15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.898+0000 7f03bdb15640 10 monclient(hunting): handle_auth_done global_id 284106 payload 824 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.898+0000 7f03bdb15640 1 -- 10.20.193.78:0/2583258392 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03b8074980 con 0x7f03b81cd170 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.898+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f03ac024060 con 0x7f03b806d380 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.898+0000 7f03a67fc640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f038803deb0 0x7f0388040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.898+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(825..825 src has 525..825) ==== 4827+0+0 (secure 0 0 0) 0x7f03ac0528d0 con 0x7f03b806d380 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.898+0000 7f03be316640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f038803deb0 0x7f0388040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.898+0000 7f03be316640 10 monclient: get_auth_request con 0x7f038803deb0 auth_method 0 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03bdb15640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b81cd170 msgr2=0x7f03b81378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03bdb15640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b81cd170 0x7f03b81378c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d720 tx=0x7f03a800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8005de0 con 0x7f03b81cd170 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 10 monclient: dump: 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.932 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f03a8017070 con 0x7f03b81cd170 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b81cd170 msgr2=0x7f03b81378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b81cd170 0x7f03b81378c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d720 tx=0x7f03a800dbf0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b81cd170 msgr2=0x7f03b81378c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.900+0000 7f03a67fc640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b81cd170 0x7f03b81378c0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d720 tx=0x7f03a800dbf0 comp rx=0 tx=0).stop 2026-02-17T06:09:21.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.901+0000 7f03be316640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f038803deb0 0x7f0388040370 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f03b8139840 tx=0x7f03b4033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bfda0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bfda0640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b8137e00 0x7f03b81d8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bfda0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03b8138210 con 0x7f03b8137e00 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b8137e00 0x7f03b81d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 10 monclient: get_auth_request con 0x7f03b8137e00 auth_method 0 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 10 monclient(hunting): handle_auth_done global_id 284109 payload 824 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.903+0000 7f03bdb15640 1 -- 10.20.193.78:0/2583258392 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f03b81398a0 con 0x7f03b8137e00 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bdb15640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b8137e00 msgr2=0x7f03b81d8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:21.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bdb15640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b8137e00 0x7f03b81d8270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d720 tx=0x7f03a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a800bb40 con 0x7f03b8137e00 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 10 monclient: dump: 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f03a800bb40 con 0x7f03b8137e00 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:09:21.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b8137e00 msgr2=0x7f03b81d8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b8137e00 0x7f03b81d8270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d720 tx=0x7f03a8004a40 comp rx=0 tx=0).stop 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b8137e00 msgr2=0x7f03b81d8270 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03a67fc640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f03b8137e00 0x7f03b81d8270 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d720 tx=0x7f03a8004a40 comp rx=0 tx=0).stop 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 258, 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 13044, 2026-02-17T06:09:21.936 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:21.937 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:21.938 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:09:21.939 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:21.940 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:21.941 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:21.942 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:21.943 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:09:21.944 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:09:21.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f038803deb0 msgr2=0x7f0388040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f038803deb0 0x7f0388040370 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f03b8139840 tx=0x7f03b4033000 comp rx=0 tx=0).stop 2026-02-17T06:09:21.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 10 monclient: shutdown 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 msgr2=0x7f03b813cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 1 --2- 10.20.193.78:0/2583258392 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f03b806d380 0x7f03b813cd20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00e9b0 tx=0x7f03ac00ee80 comp rx=0 tx=0).stop 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03be316640 1 -- 10.20.193.78:0/2583258392 reap_dead start 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03be316640 1 -- 10.20.193.78:0/2583258392 reap_dead start 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 shutdown_connections 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 >> 10.20.193.78:0/2583258392 conn(0x7f03b80a6620 msgr2=0x7f03b80a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 shutdown_connections 2026-02-17T06:09:21.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:21.906+0000 7f03bfda0640 1 -- 10.20.193.78:0/2583258392 wait complete. 2026-02-17T06:09:21.946 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:09:21.946 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:09:21.946 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.010+0000 7fc7237a9640 10 monclient: get_monmap_and_config 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.010+0000 7fc7237a9640 10 monclient: build_initial_monmap 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.010+0000 7fc7237a9640 10 monclient: monmap: 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:22.011879+0000 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:22.011879+0000 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.011 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:22.012 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:22.012 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:22.012 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:22.012 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:22.012 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 1 Processor -- start 2026-02-17T06:09:22.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 1 -- start start 2026-02-17T06:09:22.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient: init 2026-02-17T06:09:22.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:22.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2026-02-17T06:09:22.013 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc71c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc71c09c4a0 0x7fc71c13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient(hunting): picked mon.noname-a con 0x7fc71c09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient(hunting): picked mon.noname-b con 0x7fc71c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71c06daf0 con 0x7fc71c06c890 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fc71c0a0880 con 0x7fc71c09c4a0 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c09cc80 con 0x7fc71c06d380 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc7237a9640 10 monclient(hunting): authenticate will time out at 2007.565387s 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc71c09c4a0 0x7fc71c13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fc71c09c4a0 0x7fc71c13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52046/0 (socket says 10.20.193.78:52046) 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 1 -- 10.20.193.78:0/7384033 learned_addr learned my addr 10.20.193.78:0/7384033 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient(hunting): get_auth_request con 0x7fc71c09c4a0 auth_method 0 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc721d1f640 1 --2- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc721d1f640 10 monclient(hunting): get_auth_request con 0x7fc71c06d380 auth_method 0 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc721d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.014 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc721d1f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc721d1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc721d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc721d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc721d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient(hunting): handle_auth_done global_id 284112 payload 274 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 1 -- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 msgr2=0x7fc71c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 1 --2- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 1 -- 10.20.193.78:0/7384033 >> v1:10.20.193.115:6789/0 conn(0x7fc71c06c890 legacy=0x7fc71c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient: _finish_hunting 0 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 1 monclient: found mon.noname-a 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:22.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.011+0000 7fc720d1d640 1 -- 10.20.193.78:0/7384033 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c09cf60 con 0x7fc71c09c4a0 2026-02-17T06:09:22.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc720d1d640 1 -- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 msgr2=0x7fc71c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc720d1d640 1 --2- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 0x7fc71c13b3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc710009970 tx=0x7fc71002ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8376d2444df3e60c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 1 -- 10.20.193.78:0/7384033 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71003c070 con 0x7fc71c09c4a0 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: dump: 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: _finish_auth 0 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 1 -- 10.20.193.78:0/7384033 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71002fcd0 con 0x7fc71c09c4a0 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 1 -- 10.20.193.78:0/7384033 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710037660 con 0x7fc71c09c4a0 2026-02-17T06:09:22.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc70bfff640 10 monclient: dump: 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 5 monclient: authenticate success, global_id 284112 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 4 set_mon_vals no callback set 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 10 monclient: shutdown 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 1 -- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 msgr2=0x7fc71c13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 1 --2- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 0x7fc71c13b3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc710009970 tx=0x7fc71002ee90 comp rx=0 tx=0).stop 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 1 -- 10.20.193.78:0/7384033 shutdown_connections 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 1 --2- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 1 --2- 10.20.193.78:0/7384033 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 0x7fc71c13b3a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 1 -- 10.20.193.78:0/7384033 >> 10.20.193.78:0/7384033 conn(0x7fc71c0a6620 msgr2=0x7fc71c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:22.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 1 -- 10.20.193.78:0/7384033 shutdown_connections 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.014+0000 7fc7237a9640 1 -- 10.20.193.78:0/7384033 wait complete. 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient: build_initial_monmap 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient: monmap: 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 1 Processor -- start 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 1 -- start start 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient: init 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc71c06c890 0x7fc71c072a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient(hunting): picked mon.noname-b con 0x7fc71c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c1336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient(hunting): picked mon.noname-c con 0x7fc71c06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 0x7fc71c11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient(hunting): picked mon.noname-a con 0x7fc71c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c09bfa0 con 0x7fc71c09c4a0 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c09be20 con 0x7fc71c06c890 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c13de30 con 0x7fc71c06d380 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc7237a9640 10 monclient(hunting): authenticate will time out at 2007.569575s 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c1336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c1336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53808/0 (socket says 10.20.193.78:53808) 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 -- 10.20.193.78:0/1742027078 learned_addr learned my addr 10.20.193.78:0/1742027078 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient(hunting): get_auth_request con 0x7fc71c06d380 auth_method 0 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc721d1f640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 0x7fc71c11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc721d1f640 10 monclient(hunting): get_auth_request con 0x7fc71c09c4a0 auth_method 0 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc721d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc721d1f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc721d1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc721d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc721d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc721d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient(hunting): handle_auth_done global_id 264314 payload 824 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc71c06c890 msgr2=0x7fc71c072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc71c06c890 0x7fc71c072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 msgr2=0x7fc71c11cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc71c09c4a0 0x7fc71c11cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient: _finish_hunting 0 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 monclient: found mon.noname-c 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.015+0000 7fc720d1d640 1 -- 10.20.193.78:0/1742027078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c11fd90 con 0x7fc71c06d380 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.017+0000 7fc720d1d640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 msgr2=0x7fc71c1336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.017+0000 7fc720d1d640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c1336c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc710009970 tx=0x7fc710004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.017+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71003c040 con 0x7fc71c06d380 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: dump: 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: _finish_auth 0 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc710047070 con 0x7fc71c06d380 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 5 monclient: authenticate success, global_id 264314 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710042890 con 0x7fc71c06d380 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 10 monclient: dump: 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc70b7fe640 4 set_mon_vals no callback set 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 10 monclient: _renew_subs 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c120070 con 0x7fc71c06d380 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 10 monclient: _renew_subs 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c11dd10 con 0x7fc71c06d380 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c11e870 0x7fc71c11d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c074980 con 0x7fc71c11e870 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc710042a30 con 0x7fc71c06d380 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c11e870 0x7fc71c11d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 10 monclient: get_auth_request con 0x7fc71c11e870 auth_method 0 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 10 monclient(hunting): handle_auth_done global_id 264317 payload 824 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc72151e640 1 -- 10.20.193.78:0/1742027078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc71c06cc70 con 0x7fc71c11e870 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6ec03deb0 0x7fc6ec040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(825..825 src has 525..825) ==== 4827+0+0 (secure 0 0 0) 0x7fc710077a50 con 0x7fc71c06d380 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc721d1f640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6ec03deb0 0x7fc6ec040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.018+0000 7fc721d1f640 10 monclient: get_auth_request con 0x7fc6ec03deb0 auth_method 0 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.020+0000 7fc72151e640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c11e870 msgr2=0x7fc71c11d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.020+0000 7fc72151e640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c11e870 0x7fc71c11d1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00c390 tx=0x7fc70c00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.020+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c011a80 con 0x7fc71c11e870 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.020+0000 7fc709ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.020+0000 7fc709ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.020+0000 7fc709ffb640 10 monclient: dump: 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.021+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fc70c017070 con 0x7fc71c11e870 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.021+0000 7fc709ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.021+0000 7fc709ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.021+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c11e870 msgr2=0x7fc71c11d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.021+0000 7fc709ffb640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c11e870 0x7fc71c11d1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00c390 tx=0x7fc70c00c860 comp rx=0 tx=0).stop 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.021+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c11e870 msgr2=0x7fc71c11d1b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.021+0000 7fc709ffb640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c11e870 0x7fc71c11d1b0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00c390 tx=0x7fc70c00c860 comp rx=0 tx=0).stop 2026-02-17T06:09:22.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.021+0000 7fc721d1f640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6ec03deb0 0x7fc6ec040370 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc7180113e0 tx=0x7fc71800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc7237a9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc7237a9640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c133c00 0x7fc71c1da630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc7237a9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c0744d0 con 0x7fc71c133c00 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c133c00 0x7fc71c1da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 10 monclient: get_auth_request con 0x7fc71c133c00 auth_method 0 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 10 monclient(hunting): handle_auth_done global_id 264320 payload 824 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.024+0000 7fc72151e640 1 -- 10.20.193.78:0/1742027078 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc71c11f920 con 0x7fc71c133c00 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc72151e640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c133c00 msgr2=0x7fc71c1da630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc72151e640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c133c00 0x7fc71c1da630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00c390 tx=0x7fc70c003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0055b0 con 0x7fc71c133c00 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 10 monclient: dump: 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.029 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fc70c0046e0 con 0x7fc71c133c00 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c133c00 msgr2=0x7fc71c1da630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c133c00 0x7fc71c1da630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00c390 tx=0x7fc70c003860 comp rx=0 tx=0).stop 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c133c00 msgr2=0x7fc71c1da630 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc709ffb640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c133c00 0x7fc71c1da630 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00c390 tx=0x7fc70c003860 comp rx=0 tx=0).stop 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:09:22.030 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 258, 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 111411, 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.031 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:22.032 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.033 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.034 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.035 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.036 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:09:22.037 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:09:22.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6ec03deb0 msgr2=0x7fc6ec040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc6ec03deb0 0x7fc6ec040370 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc7180113e0 tx=0x7fc71800b040 comp rx=0 tx=0).stop 2026-02-17T06:09:22.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 10 monclient: shutdown 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 msgr2=0x7fc71c1336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c1336c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc710009970 tx=0x7fc710004830 comp rx=0 tx=0).stop 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc721d1f640 1 -- 10.20.193.78:0/1742027078 reap_dead start 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 shutdown_connections 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 --2- 10.20.193.78:0/1742027078 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc71c06d380 0x7fc71c1336c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 >> 10.20.193.78:0/1742027078 conn(0x7fc71c0a6620 msgr2=0x7fc71c1344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 shutdown_connections 2026-02-17T06:09:22.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.026+0000 7fc7237a9640 1 -- 10.20.193.78:0/1742027078 wait complete. 2026-02-17T06:09:22.039 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:09:22.039 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:09:22.039 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:09:22.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.105+0000 7f9cca785640 10 monclient: get_monmap_and_config 2026-02-17T06:09:22.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.105+0000 7f9cca785640 10 monclient: build_initial_monmap 2026-02-17T06:09:22.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.105+0000 7f9cca785640 10 monclient: monmap: 2026-02-17T06:09:22.106 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:22.106 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.106 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:22.107146+0000 2026-02-17T06:09:22.106 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:22.107146+0000 2026-02-17T06:09:22.106 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:22.107 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.107 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:22.107 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:22.107 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:22.107 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:22.107 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:22.107 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 1 Processor -- start 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 1 -- start start 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient: init 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f9cc406c890 0x7f9cc406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient(hunting): picked mon.noname-a con 0x7f9cc406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc409c4a0 0x7f9cc406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient(hunting): picked mon.noname-b con 0x7f9cc409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f9cc4134df0 0x7f9cc4135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient(hunting): picked mon.noname-c con 0x7f9cc4134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc40a0880 con 0x7f9cc406c890 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc406d6f0 con 0x7f9cc4134df0 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc406daf0 con 0x7f9cc409c4a0 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cca785640 10 monclient(hunting): authenticate will time out at 2007.660700s 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f9cc406c890 0x7f9cc406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc37fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc409c4a0 0x7f9cc406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f9cc406c890 0x7f9cc406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52058/0 (socket says 10.20.193.78:52058) 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 1 -- 10.20.193.78:0/347175424 learned_addr learned my addr 10.20.193.78:0/347175424 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient(hunting): get_auth_request con 0x7f9cc406c890 auth_method 0 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc37fe640 10 monclient(hunting): get_auth_request con 0x7f9cc409c4a0 auth_method 0 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc8cfb640 1 --2- 10.20.193.78:0/347175424 >> v2:10.20.193.115:3300/0 conn(0x7f9cc4134df0 0x7f9cc4135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc8cfb640 10 monclient(hunting): get_auth_request con 0x7f9cc4134df0 auth_method 0 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc8cfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc8cfb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc8cfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_done global_id 284118 payload 274 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 1 -- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc409c4a0 msgr2=0x7f9cc406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 1 --2- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc409c4a0 0x7f9cc406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 1 -- 10.20.193.78:0/347175424 >> v2:10.20.193.115:3300/0 conn(0x7f9cc4134df0 msgr2=0x7f9cc4135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 1 --2- 10.20.193.78:0/347175424 >> v2:10.20.193.115:3300/0 conn(0x7f9cc4134df0 0x7f9cc4135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 10 monclient: _finish_hunting 0 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.106+0000 7f9cc3fff640 1 monclient: found mon.noname-a 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.107+0000 7f9cc3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.107+0000 7f9cc3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.107+0000 7f9cc3fff640 1 -- 10.20.193.78:0/347175424 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc409cc80 con 0x7f9cc406c890 2026-02-17T06:09:22.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.107+0000 7f9cc37fe640 1 --2- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc409c4a0 0x7f9cc406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc3fff640 1 -- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc406c890 msgr2=0x7f9cc406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc3fff640 1 --2- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc406c890 0x7f9cc406cc70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40a7490 tx=0x7f9cb002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b3399a8a4908f1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 1 -- 10.20.193.78:0/347175424 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb003c070 con 0x7f9cc406c890 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: dump: 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.110 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: _finish_auth 0 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 1 -- 10.20.193.78:0/347175424 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb002fca0 con 0x7f9cc406c890 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cca785640 5 monclient: authenticate success, global_id 284118 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cca785640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 1 -- 10.20.193.78:0/347175424 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0037660 con 0x7f9cc406c890 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cc2ffd640 10 monclient: dump: 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cca785640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.109+0000 7f9cca785640 4 set_mon_vals no callback set 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 10 monclient: shutdown 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 -- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc406c890 msgr2=0x7f9cc406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 --2- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc406c890 0x7f9cc406cc70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40a7490 tx=0x7f9cb002ee60 comp rx=0 tx=0).stop 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 -- 10.20.193.78:0/347175424 shutdown_connections 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 --2- 10.20.193.78:0/347175424 >> v2:10.20.193.115:3300/0 conn(0x7f9cc4134df0 0x7f9cc4135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 --2- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc409c4a0 0x7f9cc406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 --2- 10.20.193.78:0/347175424 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc406c890 0x7f9cc406cc70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 -- 10.20.193.78:0/347175424 >> 10.20.193.78:0/347175424 conn(0x7f9cc40a6620 msgr2=0x7f9cc40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:22.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 -- 10.20.193.78:0/347175424 shutdown_connections 2026-02-17T06:09:22.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 -- 10.20.193.78:0/347175424 wait complete. 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 10 monclient: build_initial_monmap 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 10 monclient: monmap: 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 Processor -- start 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 -- start start 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 10 monclient: init 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc406c890 0x7f9cc41cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 10 monclient(hunting): picked mon.noname-b con 0x7f9cc406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc409c4a0 0x7f9cc41cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 10 monclient(hunting): picked mon.noname-a con 0x7f9cc409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.110+0000 7f9cca785640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc4134df0 0x7f9cc409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 10 monclient(hunting): picked mon.noname-c con 0x7f9cc4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc40a01a0 con 0x7f9cc409c4a0 2026-02-17T06:09:22.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc40a0020 con 0x7f9cc406c890 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc409be20 con 0x7f9cc4134df0 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cca785640 10 monclient(hunting): authenticate will time out at 2007.665006s 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc409c4a0 0x7f9cc41cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc409c4a0 0x7f9cc41cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52074/0 (socket says 10.20.193.78:52074) 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 -- 10.20.193.78:0/1188984501 learned_addr learned my addr 10.20.193.78:0/1188984501 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient(hunting): get_auth_request con 0x7f9cc409c4a0 auth_method 0 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc8cfb640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc4134df0 0x7f9cc409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc8cfb640 10 monclient(hunting): get_auth_request con 0x7f9cc4134df0 auth_method 0 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc8cfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc8cfb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc8cfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc3fff640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc406c890 0x7f9cc41cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc3fff640 10 monclient(hunting): get_auth_request con 0x7f9cc406c890 auth_method 0 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc8cfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc8cfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc8cfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient(hunting): handle_auth_done global_id 284121 payload 824 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc4134df0 msgr2=0x7f9cc409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f9cc4134df0 0x7f9cc409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc406c890 msgr2=0x7f9cc41cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc406c890 0x7f9cc41cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:22.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 monclient: found mon.noname-a 2026-02-17T06:09:22.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:22.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:22.115 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.111+0000 7f9cc37fe640 1 -- 10.20.193.78:0/1188984501 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc41276e0 con 0x7f9cc409c4a0 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc37fe640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc409c4a0 msgr2=0x7f9cc41cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc37fe640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc409c4a0 0x7f9cc41cf890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400ebe0 tx=0x7f9cb400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4017070 con 0x7f9cc409c4a0 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: dump: 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: _finish_auth 0 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb400ce50 con 0x7f9cc409c4a0 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4013690 con 0x7f9cc409c4a0 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc0ff9640 10 monclient: dump: 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.116 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cc27fc640 4 set_mon_vals no callback set 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cca785640 5 monclient: authenticate success, global_id 284121 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cca785640 10 monclient: _renew_subs 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cca785640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc40a1680 con 0x7f9cc409c4a0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cca785640 10 monclient: _renew_subs 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cca785640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.113+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc4125f50 con 0x7f9cc409c4a0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cca785640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cca785640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e090 0x7f9cc409e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cca785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc4074980 con 0x7f9cc409e090 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9cb4024060 con 0x7f9cc409c4a0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e090 0x7f9cc409e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 10 monclient: get_auth_request con 0x7f9cc409e090 auth_method 0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc0ff9640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9c9403df00 0x7f9c940403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(825..825 src has 525..825) ==== 4827+0+0 (secure 0 0 0) 0x7f9cb4052820 con 0x7f9cc409c4a0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc8cfb640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9c9403df00 0x7f9c940403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc8cfb640 10 monclient: get_auth_request con 0x7f9c9403df00 auth_method 0 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_done global_id 274189 payload 824 2026-02-17T06:09:22.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.114+0000 7f9cc3fff640 1 -- 10.20.193.78:0/1188984501 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cc4135240 con 0x7f9cc409e090 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.116+0000 7f9cc3fff640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e090 msgr2=0x7f9cc409e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.116+0000 7f9cc3fff640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e090 0x7f9cc409e470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9cc406a480 tx=0x7f9cb0004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.116+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0046070 con 0x7f9cc409e090 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.116+0000 7f9cc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.116+0000 7f9cc0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.116+0000 7f9cc0ff9640 10 monclient: dump: 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.117+0000 7f9cc8cfb640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9c9403df00 0x7f9c940403c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4126c70 tx=0x7f9cb8009770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.117+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f9cb002fe50 con 0x7f9cc409e090 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.117+0000 7f9cc0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.117+0000 7f9cc0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.117+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e090 msgr2=0x7f9cc409e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.117+0000 7f9cc0ff9640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e090 0x7f9cc409e470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9cc406a480 tx=0x7f9cb0004830 comp rx=0 tx=0).stop 2026-02-17T06:09:22.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.117+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e090 msgr2=0x7f9cc409e470 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:22.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.117+0000 7f9cc0ff9640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e090 0x7f9cc409e470 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9cc406a480 tx=0x7f9cb0004830 comp rx=0 tx=0).stop 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cca785640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cca785640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e9b0 0x7f9cc41da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cca785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc409f7b0 con 0x7f9cc409e9b0 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cc3fff640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e9b0 0x7f9cc41da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cc3fff640 10 monclient: get_auth_request con 0x7f9cc409e9b0 auth_method 0 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cc3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cc3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.120+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.121+0000 7f9cc3fff640 10 monclient(hunting): handle_auth_done global_id 274192 payload 824 2026-02-17T06:09:22.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.121+0000 7f9cc3fff640 1 -- 10.20.193.78:0/1188984501 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9cc40744d0 con 0x7f9cc409e9b0 2026-02-17T06:09:22.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc3fff640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e9b0 msgr2=0x7f9cc41da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc3fff640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e9b0 0x7f9cc41da560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0002410 tx=0x7f9cb00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0046040 con 0x7f9cc409e9b0 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 10 monclient: dump: 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f9cb0003d30 con 0x7f9cc409e9b0 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e9b0 msgr2=0x7f9cc41da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e9b0 0x7f9cc41da560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0002410 tx=0x7f9cb00047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e9b0 msgr2=0x7f9cc41da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc0ff9640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9cc409e9b0 0x7f9cc41da560 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0002410 tx=0x7f9cb00047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.124 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 258, 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 64124, 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:22.125 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:22.126 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:22.127 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:22.128 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:09:22.129 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9c9403df00 msgr2=0x7f9c940403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f9c9403df00 0x7f9c940403c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4126c70 tx=0x7f9cb8009770 comp rx=0 tx=0).stop 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 10 monclient: shutdown 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc409c4a0 msgr2=0x7f9cc41cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 1 --2- 10.20.193.78:0/1188984501 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f9cc409c4a0 0x7f9cc41cf890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400ebe0 tx=0x7f9cb400c520 comp rx=0 tx=0).stop 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc8cfb640 1 -- 10.20.193.78:0/1188984501 reap_dead start 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cc8cfb640 1 -- 10.20.193.78:0/1188984501 reap_dead start 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 shutdown_connections 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 >> 10.20.193.78:0/1188984501 conn(0x7f9cc40a6620 msgr2=0x7f9cc4071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 shutdown_connections 2026-02-17T06:09:22.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.123+0000 7f9cca785640 1 -- 10.20.193.78:0/1188984501 wait complete. 2026-02-17T06:09:22.131 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:09:22.131 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:09:22.131 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.196+0000 7f0822054640 10 monclient: get_monmap_and_config 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.196+0000 7f0822054640 10 monclient: build_initial_monmap 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.196+0000 7f0822054640 10 monclient: monmap: 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:22.197491+0000 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:22.197491+0000 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.197 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:22.198 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:22.198 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:22.198 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:22.198 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:22.198 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.196+0000 7f0822054640 1 Processor -- start 2026-02-17T06:09:22.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.196+0000 7f0822054640 1 -- start start 2026-02-17T06:09:22.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.196+0000 7f0822054640 10 monclient: init 2026-02-17T06:09:22.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.196+0000 7f0822054640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:22.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.196+0000 7f0822054640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f081c06c890 0x7f081c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 10 monclient(hunting): picked mon.noname-a con 0x7f081c06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c09c4a0 0x7f081c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 10 monclient(hunting): picked mon.noname-b con 0x7f081c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f081c134df0 0x7f081c135240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 10 monclient(hunting): picked mon.noname-c con 0x7f081c134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f081c0a0880 con 0x7f081c06c890 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f081c06d6f0 con 0x7f081c134df0 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f081c06daf0 con 0x7f081c09c4a0 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f0822054640 10 monclient(hunting): authenticate will time out at 2007.751038s 2026-02-17T06:09:22.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c09c4a0 0x7f081c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081b7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f081c06c890 0x7f081c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c09c4a0 0x7f081c06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53870/0 (socket says 10.20.193.78:53870) 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 -- 10.20.193.78:0/492435248 learned_addr learned my addr 10.20.193.78:0/492435248 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient(hunting): get_auth_request con 0x7f081c09c4a0 auth_method 0 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081b7fe640 10 monclient(hunting): get_auth_request con 0x7f081c06c890 auth_method 0 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.205 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081bfff640 1 --2- 10.20.193.78:0/492435248 >> v2:10.20.193.115:3300/0 conn(0x7f081c134df0 0x7f081c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081bfff640 10 monclient(hunting): get_auth_request con 0x7f081c134df0 auth_method 0 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient(hunting): handle_auth_done global_id 264329 payload 274 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 -- 10.20.193.78:0/492435248 >> v2:10.20.193.115:3300/0 conn(0x7f081c134df0 msgr2=0x7f081c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 --2- 10.20.193.78:0/492435248 >> v2:10.20.193.115:3300/0 conn(0x7f081c134df0 0x7f081c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 -- 10.20.193.78:0/492435248 >> v2:10.20.193.78:3300/0 conn(0x7f081c06c890 msgr2=0x7f081c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 --2- 10.20.193.78:0/492435248 >> v2:10.20.193.78:3300/0 conn(0x7f081c06c890 0x7f081c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient: _finish_hunting 0 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 monclient: found mon.noname-b 2026-02-17T06:09:22.206 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.197+0000 7f081affd640 1 -- 10.20.193.78:0/492435248 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c09cc80 con 0x7f081c09c4a0 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081affd640 1 -- 10.20.193.78:0/492435248 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c09c4a0 msgr2=0x7f081c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081affd640 1 --2- 10.20.193.78:0/492435248 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c09c4a0 0x7f081c06d1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f081c0a7490 tx=0x7f080c02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb93429893b110e server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 1 -- 10.20.193.78:0/492435248 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c03c070 con 0x7f081c09c4a0 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: dump: 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.207 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f0822054640 5 monclient: authenticate success, global_id 264329 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 1 -- 10.20.193.78:0/492435248 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c02fca0 con 0x7f081c09c4a0 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f0822054640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 1 -- 10.20.193.78:0/492435248 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c037660 con 0x7f081c09c4a0 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f0822054640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f0822054640 4 set_mon_vals no callback set 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.208 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.199+0000 7f081a7fc640 10 monclient: dump: 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 10 monclient: shutdown 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 -- 10.20.193.78:0/492435248 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c09c4a0 msgr2=0x7f081c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 --2- 10.20.193.78:0/492435248 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c09c4a0 0x7f081c06d1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f081c0a7490 tx=0x7f080c02ee60 comp rx=0 tx=0).stop 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 -- 10.20.193.78:0/492435248 shutdown_connections 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 --2- 10.20.193.78:0/492435248 >> v2:10.20.193.115:3300/0 conn(0x7f081c134df0 0x7f081c135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 --2- 10.20.193.78:0/492435248 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c09c4a0 0x7f081c06d1b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 --2- 10.20.193.78:0/492435248 >> v2:10.20.193.78:3300/0 conn(0x7f081c06c890 0x7f081c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 -- 10.20.193.78:0/492435248 >> 10.20.193.78:0/492435248 conn(0x7f081c0a6620 msgr2=0x7f081c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 -- 10.20.193.78:0/492435248 shutdown_connections 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 -- 10.20.193.78:0/492435248 wait complete. 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 10 monclient: build_initial_monmap 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 10 monclient: monmap: 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 Processor -- start 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 1 -- start start 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.200+0000 7f0822054640 10 monclient: init 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f081c06c890 0x7f081c072bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient(hunting): picked mon.noname-b con 0x7f081c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f081c09c4a0 0x7f081c11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient(hunting): picked mon.noname-a con 0x7f081c09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c134df0 0x7f081c11d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient(hunting): picked mon.noname-c con 0x7f081c134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0a01a0 con 0x7f081c09c4a0 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0a0020 con 0x7f081c06c890 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f081c09be20 con 0x7f081c134df0 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f0822054640 10 monclient(hunting): authenticate will time out at 2007.755208s 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f081c09c4a0 0x7f081c11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c134df0 0x7f081c11d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c134df0 0x7f081c11d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53878/0 (socket says 10.20.193.78:53878) 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f081c09c4a0 0x7f081c11cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52086/0 (socket says 10.20.193.78:52086) 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 -- 10.20.193.78:0/3632681541 learned_addr learned my addr 10.20.193.78:0/3632681541 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient(hunting): get_auth_request con 0x7f081c134df0 auth_method 0 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 10 monclient(hunting): get_auth_request con 0x7f081c09c4a0 auth_method 0 2026-02-17T06:09:22.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient(hunting): handle_auth_done global_id 264332 payload 824 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 -- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f081c06c890 msgr2=0x7f081c072bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f081c06c890 0x7f081c072bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 -- 10.20.193.78:0/3632681541 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f081c09c4a0 msgr2=0x7f081c11cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f081c09c4a0 0x7f081c11cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient: _finish_hunting 0 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 monclient: found mon.noname-c 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.201+0000 7f081bfff640 1 -- 10.20.193.78:0/3632681541 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c131ca0 con 0x7f081c134df0 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f081bfff640 1 -- 10.20.193.78:0/3632681541 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c134df0 msgr2=0x7f081c11d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f081bfff640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c134df0 0x7f081c11d1b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0810007df0 tx=0x7f081000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 1 -- 10.20.193.78:0/3632681541 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081000eea0 con 0x7f081c134df0 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: dump: 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: _finish_auth 0 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: _check_auth_tickets 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:22.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 1 -- 10.20.193.78:0/3632681541 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810019070 con 0x7f081c134df0 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f0822054640 5 monclient: authenticate success, global_id 264332 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 1 -- 10.20.193.78:0/3632681541 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08100139f0 con 0x7f081c134df0 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f0819ffb640 4 set_mon_vals no callback set 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f07f7fff640 10 monclient: dump: 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f0822054640 10 monclient: _renew_subs 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f0822054640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c11ddb0 con 0x7f081c134df0 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f0822054640 10 monclient: _renew_subs 2026-02-17T06:09:22.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f0822054640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.203+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c131990 con 0x7f081c134df0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f0822054640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f0822054640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f0822054640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c074980 con 0x7f081c134df0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f07f7fff640 1 -- 10.20.193.78:0/3632681541 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f0810013b90 con 0x7f081c134df0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f07f7fff640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f07ec03deb0 0x7f07ec040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f07f7fff640 1 -- 10.20.193.78:0/3632681541 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(825..825 src has 525..825) ==== 4827+0+0 (secure 0 0 0) 0x7f0810052fd0 con 0x7f081c134df0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f081b7fe640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f07ec03deb0 0x7f07ec040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.204+0000 7f081b7fe640 10 monclient: get_auth_request con 0x7f07ec03deb0 auth_method 0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.207+0000 7f07f7fff640 1 -- 10.20.193.78:0/3632681541 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f081001cc80 con 0x7f081c134df0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.207+0000 7f07f7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.207+0000 7f07f7fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:22.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.207+0000 7f081b7fe640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f07ec03deb0 0x7f07ec040370 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0808006b90 tx=0x7f0808006b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:22.320 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.319+0000 7f0822054640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:09:22.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.319+0000 7f0822054640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:09:22.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.319+0000 7f0822054640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:22.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.319+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f081c135240 con 0x7f081c134df0 2026-02-17T06:09:22.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.320+0000 7f07f7fff640 1 -- 10.20.193.78:0/3632681541 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f0810012b90 con 0x7f081c134df0 2026-02-17T06:09:22.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.320+0000 7f07f7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:09:22.321 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.320+0000 7f07f7fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:22.322 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.320+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.320+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=12,kv=11,logm=46} crc {auth=1278583840,config=692619927,health=4185752094,kv=2049492640,logm=3147171918}) 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.320+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=12,kv=12,logm=47} crc {auth=1528445809,config=692619927,health=4185752094,kv=449989585,logm=3615700841}) 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.320+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.320+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=12,kv=11,logm=46} crc {auth=1278583840,config=692619927,health=4185752094,kv=2049492640,logm=3147171918}) 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.320+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=29,config=2,health=11,kv=11,logm=47} crc {auth=1278583840,config=692619927,health=1019750300,kv=2165459515,logm=3861236300}) 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.321+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.321+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763966982}) 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.321+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2835361333}) 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.321+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.321+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763966982}) 2026-02-17T06:09:22.323 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.321+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1704168902}) 2026-02-17T06:09:22.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f07ec03deb0 msgr2=0x7f07ec040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f07ec03deb0 0x7f07ec040370 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0808006b90 tx=0x7f0808006b20 comp rx=0 tx=0).stop 2026-02-17T06:09:22.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 10 monclient: shutdown 2026-02-17T06:09:22.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:22.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c134df0 msgr2=0x7f081c11d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c134df0 0x7f081c11d1b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0810007df0 tx=0x7f081000e4b0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 shutdown_connections 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f081c134df0 0x7f081c11d1b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f081c09c4a0 0x7f081c11cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f07ec03deb0 0x7f07ec040370 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 --2- 10.20.193.78:0/3632681541 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f081c06c890 0x7f081c072bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 >> 10.20.193.78:0/3632681541 conn(0x7f081c0a6620 msgr2=0x7f081c0a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 shutdown_connections 2026-02-17T06:09:22.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:22.321+0000 7f0822054640 1 -- 10.20.193.78:0/3632681541 wait complete. 2026-02-17T06:09:22.331 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.331 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3125275375}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=878219112}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3125275375}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2185501439}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213801643}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=609428388}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213801643}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.322+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1465212256}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=638581329}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1698165126}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=638581329}) 2026-02-17T06:09:22.332 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544688472}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2268037593}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2514364421}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2268037593}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.323+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3256844797}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=475793943}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=107834543}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=475793943}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1011927952}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743165332}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3593052837}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743165332}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.324+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3680637318}) 2026-02-17T06:09:22.333 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232234073}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3806870328}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232234073}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=884341044}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4078552572}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=254630646}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4078552572}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.325+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1028807732}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598261154}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=335073899}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598261154}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3345706060}) 2026-02-17T06:09:22.334 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=888068535}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3464150443}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=888068535}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1955368710}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3550261857}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273597269}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3550261857}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.326+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1271423696}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3581772302}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2114510559}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3581772302}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2747632939}) 2026-02-17T06:09:22.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3438375078}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956892265}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3438375078}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=983587983}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1458495778}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=47036125}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1458495778}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.327+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=64484334}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=748296605}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2993262424}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=748296605}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2827591457}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1600085054}) 2026-02-17T06:09:22.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=783305510}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1600085054}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2735609117}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1228940023}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2630860716}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1228940023}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.328+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1897808875}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=576769088}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2050341179}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=576769088}) 2026-02-17T06:09:22.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2534021580}) 2026-02-17T06:09:22.338 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:09:22.342 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.342 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2049704702}) 2026-02-17T06:09:22.342 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1488732252}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2049704702}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=260959707}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2809269142}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=216178435}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2809269142}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.329+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910615952}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=235398829}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3865072935}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=235398829}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1553619582}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2191032654}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3895782039}) 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2191032654}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.330+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3865842155}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3205828691}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737754260}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3205828691}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1058989143}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838899679}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2880582747}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838899679}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=454570593}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2745763315}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=279361187}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2745763315}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.331+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1957978007}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=703035498}) 2026-02-17T06:09:22.344 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4015127413}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=703035498}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232623819}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3192396235}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2062568077}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3192396235}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1052000002}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1743804538}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3482808754}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1743804538}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3491921749}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1461503991}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1540961767}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1461503991}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.332+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1651076793}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=227851137}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=218960833}) 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=227851137}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102377421}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56192236}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=418532876}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56192236}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.333+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531271349}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1733811983}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=438764567}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1733811983}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2411992030}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062096287}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2846389395}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062096287}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=688652869}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766890082}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2087503862}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766890082}) 2026-02-17T06:09:22.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2221902129}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2800205698}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4141714709}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2800205698}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1786367646}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=392835144}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2651376913}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=392835144}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3513977000}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1216613959}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=817838377}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1216613959}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4211159082}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.335+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636901505}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=699852533}) 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.347 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636901505}) 2026-02-17T06:09:22.348 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122867526}) 2026-02-17T06:09:22.375 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.373+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.375 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.373+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=107738551}) 2026-02-17T06:09:22.375 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.373+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3965367116}) 2026-02-17T06:09:22.375 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.373+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.375 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.373+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=107738551}) 2026-02-17T06:09:22.375 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.373+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=165345543}) 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=147129934}) 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=572263559}) 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=147129934}) 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097915734}) 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=935997065}) 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3811619748}) 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=935997065}) 2026-02-17T06:09:22.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4193010707}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1144770280}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1799796327}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1144770280}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.385+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1768869545}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3271631539}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1132544042}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3271631539}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3789029319}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3373520024}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239794521}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3373520024}) 2026-02-17T06:09:22.387 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=337569249}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663675979}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1855493872}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663675979}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.386+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1944364331}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384091000}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3321829546}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384091000}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2259679733}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=859183948}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1118909803}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=859183948}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2540115736}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2139430457}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=11647609}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2139430457}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.387+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3096376621}) 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.389 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483430498}) 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3549393662}) 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483430498}) 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2704221255}) 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4086351704}) 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1850919326}) 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4086351704}) 2026-02-17T06:09:22.390 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3089351214}) 2026-02-17T06:09:22.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2827048297}) 2026-02-17T06:09:22.391 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1551009390}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2827048297}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=638944832}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3277938124}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=333591325}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3277938124}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.388+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1837870205}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=185871584}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=598654026}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=185871584}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=17603182}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=256085963}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2634321213}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=256085963}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.389+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4044230787}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3201337687}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387522379}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3201337687}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1883002166}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2250232928}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2150750037}) 2026-02-17T06:09:22.392 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2250232928}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2077792790}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1139984968}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2240814506}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1139984968}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.390+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=700940391}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=650472268}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=633036907}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=650472268}) 2026-02-17T06:09:22.393 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=297097236}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2560254145}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1175450570}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2560254145}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3812957574}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4215953424}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1038745667}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4215953424}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.391+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2802466970}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2077442253}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1981618347}) 2026-02-17T06:09:22.394 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2077442253}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3038575214}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=315672690}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773455948}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=315672690}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1701423358}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=877689415}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2029480955}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=877689415}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.392+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2469395494}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.393+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.393+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2154776969}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.393+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3375901967}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.393+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.393+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2154776969}) 2026-02-17T06:09:22.395 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.393+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1167825914}) 2026-02-17T06:09:22.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4208609756}) 2026-02-17T06:09:22.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1469722949}) 2026-02-17T06:09:22.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4208609756}) 2026-02-17T06:09:22.497 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2342806703}) 2026-02-17T06:09:22.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.507+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.507+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=40560367}) 2026-02-17T06:09:22.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.507+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4241308448}) 2026-02-17T06:09:22.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.507+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.507+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=40560367}) 2026-02-17T06:09:22.509 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.507+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=443665214}) 2026-02-17T06:09:22.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.569+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.569+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1302902405}) 2026-02-17T06:09:22.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.569+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=958380241}) 2026-02-17T06:09:22.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.569+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.569+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1302902405}) 2026-02-17T06:09:22.570 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.569+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2295315697}) 2026-02-17T06:09:22.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1481707840}) 2026-02-17T06:09:22.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=120840222}) 2026-02-17T06:09:22.574 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.574 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1481707840}) 2026-02-17T06:09:22.574 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=972929973}) 2026-02-17T06:09:22.579 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.578+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.579 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.578+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129540160}) 2026-02-17T06:09:22.579 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.578+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=557596890}) 2026-02-17T06:09:22.579 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.578+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.579 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.578+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129540160}) 2026-02-17T06:09:22.580 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.578+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3088989859}) 2026-02-17T06:09:22.580 INFO:tasks.rados.rados.0.trial115.stdout:3748: done (3 left) 2026-02-17T06:09:22.580 INFO:tasks.rados.rados.0.trial115.stdout:3749: done (2 left) 2026-02-17T06:09:22.580 INFO:tasks.rados.rados.0.trial115.stdout:3751: done (1 left) 2026-02-17T06:09:22.580 INFO:tasks.rados.rados.0.trial115.stdout:3752: done (0 left) 2026-02-17T06:09:22.580 INFO:tasks.rados.rados.0.trial115.stdout:3753: write oid 38 current snap is 399 2026-02-17T06:09:22.580 INFO:tasks.rados.rados.0.trial115.stdout:3753: seq_num 1281 ranges {729767=741273,1927904=725109,3145089=384678} 2026-02-17T06:09:22.585 INFO:tasks.rados.rados.0.trial115.stdout:3753: writing trial11545420-38 from 729767 to 1471040 tid 1 2026-02-17T06:09:22.590 INFO:tasks.rados.rados.0.trial115.stdout:3753: writing trial11545420-38 from 1927904 to 2653013 tid 2 2026-02-17T06:09:22.593 INFO:tasks.rados.rados.0.trial115.stdout:3753: writing trial11545420-38 from 3145089 to 3529767 tid 3 2026-02-17T06:09:22.593 INFO:tasks.rados.rados.0.trial115.stdout:3754: write (excl) oid 8 current snap is 399 2026-02-17T06:09:22.593 INFO:tasks.rados.rados.0.trial115.stdout:3754: seq_num 1282 ranges {746574=416522,1586386=468767,2772512=629544} 2026-02-17T06:09:22.597 INFO:tasks.rados.rados.0.trial115.stdout:3754: writing trial11545420-8 from 746574 to 1163096 tid 1 2026-02-17T06:09:22.601 INFO:tasks.rados.rados.0.trial115.stdout:3754: writing trial11545420-8 from 1586386 to 2055153 tid 2 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:3754: writing trial11545420-8 from 2772512 to 3402056 tid 3 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:3755: rollback oid 1 current snap is 399 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 1 to 392 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:3753: finishing write tid 1 to trial11545420-38 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:3753: finishing write tid 2 to trial11545420-38 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:3753: finishing write tid 3 to trial11545420-38 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:3753: finishing write tid 4 to trial11545420-38 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:3753: finishing write tid 5 to trial11545420-38 2026-02-17T06:09:22.606 INFO:tasks.rados.rados.0.trial115.stdout:3753: oid 38 updating version 0 to 1993 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:3753: oid 38 version 1993 is already newer than 1992 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:3753: oid 38 version 1993 is already newer than 1991 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:3753: oid 38 version 1993 is already newer than 1990 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1993 (ObjNum 1281 snap 399 seq_num 1281) dirty exists 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:3753: left oid 38 (ObjNum 1281 snap 399 seq_num 1281) 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:3754: finishing write tid 1 to trial11545420-8 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:3754: finishing write tid 2 to trial11545420-8 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:3753: done (2 left) 2026-02-17T06:09:22.607 INFO:tasks.rados.rados.0.trial115.stdout:3756: snap_create 2026-02-17T06:09:22.608 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.605+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.608 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.605+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662634245}) 2026-02-17T06:09:22.608 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.605+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=406745610}) 2026-02-17T06:09:22.608 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.605+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:22.608 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.605+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662634245}) 2026-02-17T06:09:22.608 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:22.605+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=278649300}) 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3754: finishing write tid 3 to trial11545420-8 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3754: finishing write tid 4 to trial11545420-8 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3754: finishing write tid 5 to trial11545420-8 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3754: oid 8 updating version 0 to 1544 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3754: oid 8 version 1544 is already newer than 1542 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3754: oid 8 version 1544 is already newer than 1543 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3754: oid 8 updating version 1544 to 1545 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1545 (ObjNum 1282 snap 399 seq_num 1282) dirty exists 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3754: left oid 8 (ObjNum 1282 snap 399 seq_num 1282) 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3755: finishing rollback tid 0 to trial11545420-1 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3755: finishing rollback tid 1 to trial11545420-1 2026-02-17T06:09:22.609 INFO:tasks.rados.rados.0.trial115.stdout:3755: finishing rollback tid 2 to trial11545420-1 2026-02-17T06:09:22.610 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 1698 (ObjNum 1220 snap 378 seq_num 1220) dirty exists 2026-02-17T06:09:23.339 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:09:23.339 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:23.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.406+0000 7f5371654640 10 monclient: get_monmap_and_config 2026-02-17T06:09:23.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.406+0000 7f5371654640 10 monclient: build_initial_monmap 2026-02-17T06:09:23.409 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.406+0000 7f5371654640 10 monclient: monmap: 2026-02-17T06:09:23.409 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:23.409 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:23.407692+0000 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:23.407692+0000 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:23.410 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.406+0000 7f5371654640 1 Processor -- start 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.406+0000 7f5371654640 1 -- start start 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.406+0000 7f5371654640 10 monclient: init 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f536c06c890 0x7f536c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient(hunting): picked mon.noname-c con 0x7f536c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f536c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c06d1b0 0x7f536c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.411 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient(hunting): picked mon.noname-b con 0x7f536c06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f536c06daf0 con 0x7f536c09c4a0 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f536c0a0880 con 0x7f536c06c890 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f536c09cc80 con 0x7f536c06d1b0 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f5371654640 10 monclient(hunting): authenticate will time out at 2008.961259s 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c06d1b0 0x7f536c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c06d1b0 0x7f536c135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53880/0 (socket says 10.20.193.78:53880) 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 1 -- 10.20.193.78:0/2819447571 learned_addr learned my addr 10.20.193.78:0/2819447571 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:23.412 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient(hunting): get_auth_request con 0x7f536c06d1b0 auth_method 0 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient(hunting): handle_auth_done global_id 264335 payload 274 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 1 -- 10.20.193.78:0/2819447571 >> v2:10.20.193.115:3300/0 conn(0x7f536c06c890 msgr2=0x7f536c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 1 --2- 10.20.193.78:0/2819447571 >> v2:10.20.193.115:3300/0 conn(0x7f536c06c890 0x7f536c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.413 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 1 -- 10.20.193.78:0/2819447571 >> v1:10.20.193.115:6789/0 conn(0x7f536c09c4a0 legacy=0x7f536c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:23.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:23.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 1 monclient: found mon.noname-b 2026-02-17T06:09:23.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:23.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.414 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.407+0000 7f536b7fe640 1 -- 10.20.193.78:0/2819447571 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c09cf60 con 0x7f536c06d1b0 2026-02-17T06:09:23.421 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f536b7fe640 1 -- 10.20.193.78:0/2819447571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c06d1b0 msgr2=0x7f536c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f536b7fe640 1 --2- 10.20.193.78:0/2819447571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c06d1b0 0x7f536c135500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f536c06dcc0 tx=0x7f535c02f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b7c3f1c865eb61 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 1 -- 10.20.193.78:0/2819447571 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c0373f0 con 0x7f536c06d1b0 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: dump: 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.422 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: _finish_auth 0 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:23.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 1 -- 10.20.193.78:0/2819447571 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c037590 con 0x7f536c06d1b0 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 5 monclient: authenticate success, global_id 264335 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 1 -- 10.20.193.78:0/2819447571 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c037960 con 0x7f536c06d1b0 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 4 set_mon_vals no callback set 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5369ffb640 10 monclient: dump: 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.424 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 10 monclient: shutdown 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:23.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 1 -- 10.20.193.78:0/2819447571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c06d1b0 msgr2=0x7f536c135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 1 --2- 10.20.193.78:0/2819447571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c06d1b0 0x7f536c135500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f536c06dcc0 tx=0x7f535c02f200 comp rx=0 tx=0).stop 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 1 -- 10.20.193.78:0/2819447571 shutdown_connections 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 1 --2- 10.20.193.78:0/2819447571 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c06d1b0 0x7f536c135500 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 1 --2- 10.20.193.78:0/2819447571 >> v2:10.20.193.115:3300/0 conn(0x7f536c06c890 0x7f536c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 1 -- 10.20.193.78:0/2819447571 >> 10.20.193.78:0/2819447571 conn(0x7f536c0a6620 msgr2=0x7f536c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 1 -- 10.20.193.78:0/2819447571 shutdown_connections 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.410+0000 7f5371654640 1 -- 10.20.193.78:0/2819447571 wait complete. 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient: build_initial_monmap 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient: monmap: 2026-02-17T06:09:23.426 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 1 Processor -- start 2026-02-17T06:09:23.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 1 -- start start 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient: init 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f536c06c890 0x7f536c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient(hunting): picked mon.noname-b con 0x7f536c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f536c06d1b0 0x7f536c1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient(hunting): picked mon.noname-a con 0x7f536c06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c09c4a0 0x7f536c09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient(hunting): picked mon.noname-c con 0x7f536c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:23.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f536c0a01a0 con 0x7f536c06d1b0 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f536c0a0020 con 0x7f536c06c890 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f536c09be20 con 0x7f536c09c4a0 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f5371654640 10 monclient(hunting): authenticate will time out at 2008.965560s 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c09c4a0 0x7f536c09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c09c4a0 0x7f536c09db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53894/0 (socket says 10.20.193.78:53894) 2026-02-17T06:09:23.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 -- 10.20.193.78:0/4264473632 learned_addr learned my addr 10.20.193.78:0/4264473632 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient(hunting): get_auth_request con 0x7f536c09c4a0 auth_method 0 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536a7fc640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f536c06d1b0 0x7f536c1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536affd640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f536c06c890 0x7f536c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.430 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536a7fc640 10 monclient(hunting): get_auth_request con 0x7f536c06d1b0 auth_method 0 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536affd640 10 monclient(hunting): get_auth_request con 0x7f536c06c890 auth_method 0 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient(hunting): handle_auth_done global_id 264338 payload 824 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 -- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f536c06c890 msgr2=0x7f536c1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f536c06c890 0x7f536c1cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 -- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f536c06d1b0 msgr2=0x7f536c1cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f536c06d1b0 0x7f536c1cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 monclient: found mon.noname-c 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.411+0000 7f536b7fe640 1 -- 10.20.193.78:0/4264473632 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c06fdb0 con 0x7f536c09c4a0 2026-02-17T06:09:23.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.413+0000 7f536b7fe640 1 -- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c09c4a0 msgr2=0x7f536c09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.413+0000 7f536b7fe640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c09c4a0 0x7f536c09db50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f535c0093e0 tx=0x7f535c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 1 -- 10.20.193.78:0/4264473632 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c046020 con 0x7f536c09c4a0 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: dump: 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.433 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 5 monclient: authenticate success, global_id 264338 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 10 monclient: _renew_subs 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 1 -- 10.20.193.78:0/4264473632 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c041870 con 0x7f536c09c4a0 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c09d810 con 0x7f536c09c4a0 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:23.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 1 -- 10.20.193.78:0/4264473632 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c041c40 con 0x7f536c09c4a0 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f53697fa640 4 set_mon_vals no callback set 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 10 monclient: dump: 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.435 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 10 monclient: _renew_subs 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c129620 con 0x7f536c09c4a0 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536c06cc70 con 0x7f536c09c4a0 2026-02-17T06:09:23.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.414+0000 7f534b7fe640 1 -- 10.20.193.78:0/4264473632 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f535c042400 con 0x7f536c09c4a0 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.415+0000 7f534b7fe640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 0x7f5338040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.415+0000 7f534b7fe640 1 -- 10.20.193.78:0/4264473632 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(826..826 src has 525..826) ==== 4799+0+0 (secure 0 0 0) 0x7f535c0784b0 con 0x7f536c09c4a0 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.415+0000 7f536affd640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 0x7f5338040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.415+0000 7f536affd640 10 monclient: get_auth_request con 0x7f533803deb0 auth_method 0 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.415+0000 7f536affd640 1 -- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 msgr2=0x7f5338040370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.415+0000 7f536affd640 1 -- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 msgr2=0x7f5338040370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.415+0000 7f536affd640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 0x7f5338040370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.415+0000 7f536affd640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 0x7f5338040370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.417+0000 7f534b7fe640 1 -- 10.20.193.78:0/4264473632 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f535c050070 con 0x7f536c09c4a0 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.417+0000 7f534b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.417+0000 7f534b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:23.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2711280675}) 2026-02-17T06:09:23.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4160996971}) 2026-02-17T06:09:23.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2711280675}) 2026-02-17T06:09:23.498 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.496+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3913210561}) 2026-02-17T06:09:23.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.506+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.506+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549848963}) 2026-02-17T06:09:23.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.506+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3280367169}) 2026-02-17T06:09:23.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.506+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.506+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549848963}) 2026-02-17T06:09:23.508 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.506+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2180090478}) 2026-02-17T06:09:23.531 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.531 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2034656426}) 2026-02-17T06:09:23.531 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1052365882}) 2026-02-17T06:09:23.531 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2034656426}) 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1239087500}) 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489985568}) 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4178156892}) 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489985568}) 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1386483971}) 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3125173680}) 2026-02-17T06:09:23.532 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1258423943}) 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3125173680}) 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.528+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3214240837}) 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3579257243}) 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2366289927}) 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3579257243}) 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1032434350}) 2026-02-17T06:09:23.533 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703569507}) 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3300872748}) 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703569507}) 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.529+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2791451229}) 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1424902425}) 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2055905285}) 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1424902425}) 2026-02-17T06:09:23.534 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4122511157}) 2026-02-17T06:09:23.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.530+0000 7f5371654640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:23.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.530+0000 7f5371654640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:23.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.530+0000 7f5371654640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.530+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f536c0744d0 con 0x7f536c09c4a0 2026-02-17T06:09:23.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029206402}) 2026-02-17T06:09:23.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4149493070}) 2026-02-17T06:09:23.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029206402}) 2026-02-17T06:09:23.536 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=172202648}) 2026-02-17T06:09:23.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.530+0000 7f534b7fe640 1 -- 10.20.193.78:0/4264473632 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f535c041e20 con 0x7f536c09c4a0 2026-02-17T06:09:23.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.530+0000 7f534b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:23.537 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.530+0000 7f534b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:23.537 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":258,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:23.538 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1147860507}) 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2497981986}) 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1147860507}) 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.530+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1160267058}) 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504135042}) 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=264239126}) 2026-02-17T06:09:23.539 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504135042}) 2026-02-17T06:09:23.540 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1645874846}) 2026-02-17T06:09:23.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 msgr2=0x7f5338040370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:23.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 0x7f5338040370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.540 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 10 monclient: shutdown 2026-02-17T06:09:23.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:23.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c09c4a0 msgr2=0x7f536c09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.541 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c09c4a0 0x7f536c09db50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f535c0093e0 tx=0x7f535c004b40 comp rx=0 tx=0).stop 2026-02-17T06:09:23.541 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=572827087}) 2026-02-17T06:09:23.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3597534213}) 2026-02-17T06:09:23.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=572827087}) 2026-02-17T06:09:23.542 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4092113183}) 2026-02-17T06:09:23.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 shutdown_connections 2026-02-17T06:09:23.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f536c09c4a0 0x7f536c09db50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f536c06d1b0 0x7f536c1cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f533803deb0 0x7f5338040370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 --2- 10.20.193.78:0/4264473632 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f536c06c890 0x7f536c1cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 >> 10.20.193.78:0/4264473632 conn(0x7f536c0a6620 msgr2=0x7f536c0a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:23.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 shutdown_connections 2026-02-17T06:09:23.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.531+0000 7f5371654640 1 -- 10.20.193.78:0/4264473632 wait complete. 2026-02-17T06:09:23.546 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.546 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1527725783}) 2026-02-17T06:09:23.546 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1835614518}) 2026-02-17T06:09:23.546 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.546 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1527725783}) 2026-02-17T06:09:23.546 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.531+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1298899034}) 2026-02-17T06:09:23.546 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1870917774}) 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=316448430}) 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1870917774}) 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1836118804}) 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2898201317}) 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872358619}) 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2898201317}) 2026-02-17T06:09:23.547 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2812781476}) 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4141189320}) 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=776187860}) 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4141189320}) 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.532+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3472746196}) 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.533+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.533+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1425364777}) 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.533+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3522332680}) 2026-02-17T06:09:23.548 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.533+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.549 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.533+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1425364777}) 2026-02-17T06:09:23.549 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.533+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1230322029}) 2026-02-17T06:09:23.550 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:09:23.550 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:09:23.550 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:09:23.550 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:09:23.577 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.576+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.577 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.576+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2679926529}) 2026-02-17T06:09:23.577 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.576+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=251943317}) 2026-02-17T06:09:23.577 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.576+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.577 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.576+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2679926529}) 2026-02-17T06:09:23.577 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.576+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2295271087}) 2026-02-17T06:09:23.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.582+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.582+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3557723380}) 2026-02-17T06:09:23.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.582+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564947285}) 2026-02-17T06:09:23.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.582+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.582+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3557723380}) 2026-02-17T06:09:23.584 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.582+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2314736222}) 2026-02-17T06:09:23.585 INFO:tasks.rados.rados.0.trial115.stdout:3754: done (2 left) 2026-02-17T06:09:23.585 INFO:tasks.rados.rados.0.trial115.stdout:3755: done (1 left) 2026-02-17T06:09:23.585 INFO:tasks.rados.rados.0.trial115.stdout:3756: done (0 left) 2026-02-17T06:09:23.585 INFO:tasks.rados.rados.0.trial115.stdout:3757: delete oid 18 current snap is 400 2026-02-17T06:09:23.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.584+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.584+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2098575261}) 2026-02-17T06:09:23.586 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.584+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1595778272}) 2026-02-17T06:09:23.587 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.584+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.587 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.584+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2098575261}) 2026-02-17T06:09:23.587 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.584+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1882613177}) 2026-02-17T06:09:23.588 INFO:tasks.rados.rados.0.trial115.stdout:3757: done (0 left) 2026-02-17T06:09:23.588 INFO:tasks.rados.rados.0.trial115.stdout:3758: read oid 5 snap -1 2026-02-17T06:09:23.588 INFO:tasks.rados.rados.0.trial115.stdout:3758: expect (ObjNum 1279 snap 398 seq_num 1279) 2026-02-17T06:09:23.588 INFO:tasks.rados.rados.0.trial115.stdout:3759: read oid 21 snap -1 2026-02-17T06:09:23.589 INFO:tasks.rados.rados.0.trial115.stdout:3759: expect deleted 2026-02-17T06:09:23.589 INFO:tasks.rados.rados.0.trial115.stdout:3760: read oid 36 snap -1 2026-02-17T06:09:23.589 INFO:tasks.rados.rados.0.trial115.stdout:3760: expect deleted 2026-02-17T06:09:23.589 INFO:tasks.rados.rados.0.trial115.stdout:3761: copy_from oid 30 from oid 40 current snap is 400 2026-02-17T06:09:23.589 INFO:tasks.rados.rados.0.trial115.stdout:3762: read oid 24 snap 389 2026-02-17T06:09:23.589 INFO:tasks.rados.rados.0.trial115.stdout:3762: expect (ObjNum 1185 snap 364 seq_num 1185) 2026-02-17T06:09:23.589 INFO:tasks.rados.rados.0.trial115.stdout:3763: snap_remove snap 392 2026-02-17T06:09:23.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient: get_monmap_and_config 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient: build_initial_monmap 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient: monmap: 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:23.619290+0000 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:23.619290+0000 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:23.622 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 1 Processor -- start 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 1 -- start start 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient: init 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7b1806c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:09:23.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b180a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient(hunting): picked mon.noname-b con 0x7f7b1806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7b1813a240 0x7f7b1813c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient(hunting): picked mon.noname-c con 0x7f7b1813a240 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b18074150 con 0x7f7b1806c890 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7b1809ef70 con 0x7f7b1813a240 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1809f250 con 0x7f7b1806d380 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b1e98f640 10 monclient(hunting): authenticate will time out at 2009.172829s 2026-02-17T06:09:23.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b17fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f7b1806c890 0x7f7b1806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:54470/0 (socket says 10.20.193.78:54470) 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b17fff640 1 -- 10.20.193.78:0/1248382376 learned_addr learned my addr 10.20.193.78:0/1248382376 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b177fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b180a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b177fe640 10 monclient(hunting): get_auth_request con 0x7f7b1806d380 auth_method 0 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.618+0000 7f7b177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b1cf05640 1 --2- 10.20.193.78:0/1248382376 >> v2:10.20.193.115:3300/0 conn(0x7f7b1813a240 0x7f7b1813c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b1cf05640 10 monclient(hunting): get_auth_request con 0x7f7b1813a240 auth_method 0 2026-02-17T06:09:23.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b1cf05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b1cf05640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b1cf05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 10 monclient(hunting): handle_auth_done global_id 264341 payload 274 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 1 -- 10.20.193.78:0/1248382376 >> v2:10.20.193.115:3300/0 conn(0x7f7b1813a240 msgr2=0x7f7b1813c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 1 --2- 10.20.193.78:0/1248382376 >> v2:10.20.193.115:3300/0 conn(0x7f7b1813a240 0x7f7b1813c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 1 -- 10.20.193.78:0/1248382376 >> v1:10.20.193.78:6789/0 conn(0x7f7b1806c890 legacy=0x7f7b1806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 1 monclient: found mon.noname-b 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.619+0000 7f7b177fe640 1 -- 10.20.193.78:0/1248382376 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1809df40 con 0x7f7b1806d380 2026-02-17T06:09:23.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b177fe640 1 -- 10.20.193.78:0/1248382376 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 msgr2=0x7f7b180a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b177fe640 1 --2- 10.20.193.78:0/1248382376 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b180a01f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b1806e270 tx=0x7f7b0802eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=542d0b13e1b1ccb9 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 1 -- 10.20.193.78:0/1248382376 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0803c070 con 0x7f7b1806d380 2026-02-17T06:09:23.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:09:23.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: dump: 2026-02-17T06:09:23.636 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: _finish_auth 0 2026-02-17T06:09:23.637 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 5 monclient: authenticate success, global_id 264341 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 1 -- 10.20.193.78:0/1248382376 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0802fcf0 con 0x7f7b1806d380 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 1 -- 10.20.193.78:0/1248382376 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08037660 con 0x7f7b1806d380 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 4 set_mon_vals no callback set 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b16ffd640 10 monclient: dump: 2026-02-17T06:09:23.638 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 10 monclient: shutdown 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:23.639 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1248382376 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 msgr2=0x7f7b180a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 1 --2- 10.20.193.78:0/1248382376 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b180a01f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b1806e270 tx=0x7f7b0802eeb0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1248382376 shutdown_connections 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 1 --2- 10.20.193.78:0/1248382376 >> v2:10.20.193.115:3300/0 conn(0x7f7b1813a240 0x7f7b1813c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.621+0000 7f7b1e98f640 1 --2- 10.20.193.78:0/1248382376 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b180a01f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1248382376 >> 10.20.193.78:0/1248382376 conn(0x7f7b180a6620 msgr2=0x7f7b180a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1248382376 shutdown_connections 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1248382376 wait complete. 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient: build_initial_monmap 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient: monmap: 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:23.640 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 Processor -- start 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 -- start start 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient: init 2026-02-17T06:09:23.641 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b1806c890 0x7f7b18135a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient(hunting): picked mon.noname-b con 0x7f7b1806c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b18131380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient(hunting): picked mon.noname-c con 0x7f7b1806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813a240 0x7f7b181318c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient(hunting): picked mon.noname-a con 0x7f7b1813a240 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1813ccf0 con 0x7f7b1813a240 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1813cb70 con 0x7f7b1806c890 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1813ce70 con 0x7f7b1806d380 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1e98f640 10 monclient(hunting): authenticate will time out at 2009.176903s 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b177fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b18131380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1cf05640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813a240 0x7f7b181318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b177fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b18131380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53900/0 (socket says 10.20.193.78:53900) 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b177fe640 1 -- 10.20.193.78:0/1994734162 learned_addr learned my addr 10.20.193.78:0/1994734162 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1cf05640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813a240 0x7f7b181318c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52098/0 (socket says 10.20.193.78:52098) 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b177fe640 10 monclient(hunting): get_auth_request con 0x7f7b1806d380 auth_method 0 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1cf05640 10 monclient(hunting): get_auth_request con 0x7f7b1813a240 auth_method 0 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1cf05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1cf05640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.622+0000 7f7b1cf05640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b1cf05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b1cf05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b1cf05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 10 monclient(hunting): handle_auth_done global_id 264344 payload 824 2026-02-17T06:09:23.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b17fff640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b1806c890 0x7f7b18135a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b1806c890 msgr2=0x7f7b18135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b1806c890 0x7f7b18135a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813a240 msgr2=0x7f7b181318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813a240 0x7f7b181318c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 1 monclient: found mon.noname-c 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b177fe640 1 -- 10.20.193.78:0/1994734162 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b180a16c0 con 0x7f7b1806d380 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b17fff640 10 monclient: get_auth_request con 0x7f7b1806c890 auth_method 0 2026-02-17T06:09:23.645 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.623+0000 7f7b17fff640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7b1806c890 0x7f7b18135a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b177fe640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 msgr2=0x7f7b18131380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b177fe640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b18131380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b080099a0 tx=0x7f7b080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0803c040 con 0x7f7b1806d380 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: dump: 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.646 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: _finish_auth 0 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b08047070 con 0x7f7b1806d380 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08040a60 con 0x7f7b1806d380 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b167fc640 4 set_mon_vals no callback set 2026-02-17T06:09:23.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b14ff9640 10 monclient: dump: 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 5 monclient: authenticate success, global_id 264344 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 10 monclient: _renew_subs 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b180a1190 con 0x7f7b1806d380 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 10 monclient: _renew_subs 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b18132ac0 con 0x7f7b1806d380 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b18131e00 0x7f7b1813a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.625+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18134150 con 0x7f7b18131e00 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b18131e00 0x7f7b1813a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 10 monclient: get_auth_request con 0x7f7b18131e00 auth_method 0 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7b0803f030 con 0x7f7b1806d380 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 10 monclient(hunting): handle_auth_done global_id 284136 payload 824 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b17fff640 1 -- 10.20.193.78:0/1994734162 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b181321e0 con 0x7f7b18131e00 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b14ff9640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ae803dee0 0x7f7ae80403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(827..827 src has 525..827) ==== 4799+0+0 (secure 0 0 0) 0x7f7b08077a10 con 0x7f7b1806d380 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b1cf05640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ae803dee0 0x7f7ae80403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.626+0000 7f7b1cf05640 10 monclient: get_auth_request con 0x7f7ae803dee0 auth_method 0 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b17fff640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b18131e00 msgr2=0x7f7b1813a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b17fff640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b18131e00 0x7f7b1813a620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d6f0 tx=0x7f7b0400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.650 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04005d40 con 0x7f7b18131e00 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 10 monclient: dump: 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.651 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7b04017070 con 0x7f7b18131e00 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b18131e00 msgr2=0x7f7b1813a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b18131e00 0x7f7b1813a620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d6f0 tx=0x7f7b0400dbc0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b18131e00 msgr2=0x7f7b1813a620 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.628+0000 7f7b14ff9640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b18131e00 0x7f7b1813a620 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d6f0 tx=0x7f7b0400dbc0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.629+0000 7f7b1cf05640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ae803dee0 0x7f7ae80403a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7b18133a40 tx=0x7f7b0c009770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b1e98f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b1e98f640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813ab60 0x7f7b1813af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b1e98f640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.652 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18132db0 con 0x7f7b1813ab60 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813ab60 0x7f7b1813af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 10 monclient: get_auth_request con 0x7f7b1813ab60 auth_method 0 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 10 monclient(hunting): handle_auth_done global_id 284139 payload 824 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.631+0000 7f7b17fff640 1 -- 10.20.193.78:0/1994734162 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7b18074980 con 0x7f7b1813ab60 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.633+0000 7f7b17fff640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813ab60 msgr2=0x7f7b1813af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.633+0000 7f7b17fff640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813ab60 0x7f7b1813af70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d6f0 tx=0x7f7b04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400bc20 con 0x7f7b1813ab60 2026-02-17T06:09:23.653 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 10 monclient: dump: 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f7b0400bc20 con 0x7f7b1813ab60 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:09:23.654 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813ab60 msgr2=0x7f7b1813af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813ab60 0x7f7b1813af70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d6f0 tx=0x7f7b04004a40 comp rx=0 tx=0).stop 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813ab60 msgr2=0x7f7b1813af70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b14ff9640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7b1813ab60 0x7f7b1813af70 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d6f0 tx=0x7f7b04004a40 comp rx=0 tx=0).stop 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:09:23.655 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 258, 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 14772, 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.656 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.657 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:23.658 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.659 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.660 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.661 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:23.662 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.663 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.664 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:23.665 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.665 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.665 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.665 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:09:23.665 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:09:23.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ae803dee0 msgr2=0x7f7ae80403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7ae803dee0 0x7f7ae80403a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7b18133a40 tx=0x7f7b0c009770 comp rx=0 tx=0).stop 2026-02-17T06:09:23.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 10 monclient: shutdown 2026-02-17T06:09:23.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:23.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 msgr2=0x7f7b18131380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b18131380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b080099a0 tx=0x7f7b080047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1cf05640 1 -- 10.20.193.78:0/1994734162 reap_dead start 2026-02-17T06:09:23.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 shutdown_connections 2026-02-17T06:09:23.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 --2- 10.20.193.78:0/1994734162 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7b1806d380 0x7f7b18131380 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 >> 10.20.193.78:0/1994734162 conn(0x7f7b180a6620 msgr2=0x7f7b181393e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:23.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 shutdown_connections 2026-02-17T06:09:23.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.634+0000 7f7b1e98f640 1 -- 10.20.193.78:0/1994734162 wait complete. 2026-02-17T06:09:23.667 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:09:23.667 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:09:23.667 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.774+0000 7ff8c7961640 10 monclient: get_monmap_and_config 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.774+0000 7ff8c7961640 10 monclient: build_initial_monmap 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.774+0000 7ff8c7961640 10 monclient: monmap: 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:23.775625+0000 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:23.775625+0000 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:23.775 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.774+0000 7ff8c7961640 1 Processor -- start 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.774+0000 7ff8c7961640 1 -- start start 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.774+0000 7ff8c7961640 10 monclient: init 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff8c006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff8c009c4a0 0x7ff8c013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient(hunting): picked mon.noname-a con 0x7ff8c009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006d380 0x7ff8c013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient(hunting): picked mon.noname-b con 0x7ff8c006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c006daf0 con 0x7ff8c006c890 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c00a0880 con 0x7ff8c009c4a0 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c009cc80 con 0x7ff8c006d380 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c7961640 10 monclient(hunting): authenticate will time out at 2009.329160s 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff8c009c4a0 0x7ff8c013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006d380 0x7ff8c013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff8c009c4a0 0x7ff8c013b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52124/0 (socket says 10.20.193.78:52124) 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/4242922801 learned_addr learned my addr 10.20.193.78:0/4242922801 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:23.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient(hunting): get_auth_request con 0x7ff8c009c4a0 auth_method 0 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 10 monclient(hunting): get_auth_request con 0x7ff8c006d380 auth_method 0 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_done global_id 284142 payload 274 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006d380 msgr2=0x7ff8c013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 1 --2- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006d380 0x7ff8c013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/4242922801 >> v1:10.20.193.115:6789/0 conn(0x7ff8c006c890 legacy=0x7ff8c006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient: _finish_hunting 0 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 1 monclient: found mon.noname-a 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/4242922801 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c009cf60 con 0x7ff8c009c4a0 2026-02-17T06:09:23.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.775+0000 7ff8c5ed7640 1 --2- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006d380 0x7ff8c013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:23.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.777+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 msgr2=0x7ff8c013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.777+0000 7ff8c4ed5640 1 --2- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 0x7ff8c013b3a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009970 tx=0x7ff8b402ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6eb04e683751d21 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 1 -- 10.20.193.78:0/4242922801 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b403c070 con 0x7ff8c009c4a0 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: dump: 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.788 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: _finish_auth 0 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: _check_auth_tickets 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 1 -- 10.20.193.78:0/4242922801 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b402fcd0 con 0x7ff8c009c4a0 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 1 -- 10.20.193.78:0/4242922801 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4037660 con 0x7ff8c009c4a0 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8affff640 10 monclient: dump: 2026-02-17T06:09:23.789 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 5 monclient: authenticate success, global_id 284142 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 4 set_mon_vals no callback set 2026-02-17T06:09:23.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 10 monclient: shutdown 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 1 -- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 msgr2=0x7ff8c013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 1 --2- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 0x7ff8c013b3a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009970 tx=0x7ff8b402ee90 comp rx=0 tx=0).stop 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 1 -- 10.20.193.78:0/4242922801 shutdown_connections 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 1 --2- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006d380 0x7ff8c013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 1 --2- 10.20.193.78:0/4242922801 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 0x7ff8c013b3a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 1 -- 10.20.193.78:0/4242922801 >> 10.20.193.78:0/4242922801 conn(0x7ff8c00a6620 msgr2=0x7ff8c00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 1 -- 10.20.193.78:0/4242922801 shutdown_connections 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 1 -- 10.20.193.78:0/4242922801 wait complete. 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 10 monclient: build_initial_monmap 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.778+0000 7ff8c7961640 10 monclient: monmap: 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:23.791 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 1 Processor -- start 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 1 -- start start 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient: init 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:23.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006c890 0x7ff8c01cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient(hunting): picked mon.noname-c con 0x7ff8c006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8c006d380 0x7ff8c01333e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient(hunting): picked mon.noname-b con 0x7ff8c006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 0x7ff8c013a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient(hunting): picked mon.noname-a con 0x7ff8c009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c009bfa0 con 0x7ff8c009c4a0 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c009be20 con 0x7ff8c006d380 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c013de30 con 0x7ff8c006c890 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c7961640 10 monclient(hunting): authenticate will time out at 2009.333371s 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006c890 0x7ff8c01cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 0x7ff8c013a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 0x7ff8c013a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52136/0 (socket says 10.20.193.78:52136) 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 1 -- 10.20.193.78:0/114708423 learned_addr learned my addr 10.20.193.78:0/114708423 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient(hunting): get_auth_request con 0x7ff8c006c890 auth_method 0 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 10 monclient(hunting): get_auth_request con 0x7ff8c009c4a0 auth_method 0 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c4ed5640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8c006d380 0x7ff8c01333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c4ed5640 10 monclient(hunting): get_auth_request con 0x7ff8c006d380 auth_method 0 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c4ed5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c4ed5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c4ed5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient(hunting): handle_auth_done global_id 264350 payload 824 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8c006d380 msgr2=0x7ff8c01333e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff8c006d380 0x7ff8c01333e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 msgr2=0x7ff8c013a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 0x7ff8c013a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient: _finish_hunting 0 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 1 monclient: found mon.noname-c 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c56d6640 1 -- 10.20.193.78:0/114708423 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c011dab0 con 0x7ff8c006c890 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.779+0000 7ff8c5ed7640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff8c009c4a0 0x7ff8c013a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:23.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c56d6640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006c890 msgr2=0x7ff8c01cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c56d6640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006c890 0x7ff8c01cd150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c450 tx=0x7ff8b000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00119a0 con 0x7ff8c006c890 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: dump: 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.797 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: _finish_auth 0 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: _check_auth_tickets 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b0017070 con 0x7ff8c006c890 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c7961640 5 monclient: authenticate success, global_id 264350 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c7961640 10 monclient: _renew_subs 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c7961640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c011dd90 con 0x7ff8c006c890 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0011dc0 con 0x7ff8c006c890 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8af7fe640 4 set_mon_vals no callback set 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8adffb640 10 monclient: dump: 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c7961640 10 monclient: _renew_subs 2026-02-17T06:09:23.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c7961640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.781+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c013da40 con 0x7ff8c006c890 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c7961640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c7961640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c0133920 0x7ff8c013ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c7961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c009d870 con 0x7ff8c0133920 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c0133920 0x7ff8c013ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 10 monclient: get_auth_request con 0x7ff8c0133920 auth_method 0 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_done global_id 264353 payload 824 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/114708423 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8c0070700 con 0x7ff8c0133920 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff8b001c460 con 0x7ff8c006c890 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.782+0000 7ff8adffb640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff89003deb0 0x7ff890040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.783+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(827..827 src has 525..827) ==== 4799+0+0 (secure 0 0 0) 0x7ff8b00526b0 con 0x7ff8c006c890 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.783+0000 7ff8c5ed7640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff89003deb0 0x7ff890040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.783+0000 7ff8c5ed7640 10 monclient: get_auth_request con 0x7ff89003deb0 auth_method 0 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.784+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c0133920 msgr2=0x7ff8c013ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.784+0000 7ff8c4ed5640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c0133920 0x7ff8c013ab80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8c006a480 tx=0x7ff8b4004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.784+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b403c040 con 0x7ff8c0133920 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.784+0000 7ff8adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.784+0000 7ff8adffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.784+0000 7ff8adffb640 10 monclient: dump: 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.785+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff8b4037d70 con 0x7ff8c0133920 2026-02-17T06:09:23.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.785+0000 7ff8adffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.785+0000 7ff8adffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:23.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.785+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c0133920 msgr2=0x7ff8c013ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.785+0000 7ff8adffb640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c0133920 0x7ff8c013ab80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8c006a480 tx=0x7ff8b4004830 comp rx=0 tx=0).stop 2026-02-17T06:09:23.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.785+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c0133920 msgr2=0x7ff8c013ab80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:23.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.785+0000 7ff8adffb640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c0133920 0x7ff8c013ab80 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8c006a480 tx=0x7ff8b4004830 comp rx=0 tx=0).stop 2026-02-17T06:09:23.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.785+0000 7ff8c5ed7640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff89003deb0 0x7ff890040370 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc003d00 tx=0x7ff8bc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c7961640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:09:23.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c7961640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c013b0c0 0x7ff8c01d84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c7961640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c013b4a0 con 0x7ff8c013b0c0 2026-02-17T06:09:23.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c013b0c0 0x7ff8c01d84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 10 monclient: get_auth_request con 0x7ff8c013b0c0 auth_method 0 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 10 monclient(hunting): handle_auth_done global_id 264356 payload 824 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.788+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/114708423 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff8c011d730 con 0x7ff8c013b0c0 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8c4ed5640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c013b0c0 msgr2=0x7ff8c01d84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8c4ed5640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c013b0c0 0x7ff8c01d84e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4002410 tx=0x7ff8b40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b403c040 con 0x7ff8c013b0c0 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:23.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 10 monclient: dump: 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7ff8b40423f0 con 0x7ff8c013b0c0 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:23.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c013b0c0 msgr2=0x7ff8c01d84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c013b0c0 0x7ff8c01d84e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4002410 tx=0x7ff8b40047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c013b0c0 msgr2=0x7ff8c01d84e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.790+0000 7ff8adffb640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c013b0c0 0x7ff8c01d84e0 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4002410 tx=0x7ff8b40047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 258, 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 113175, 2026-02-17T06:09:23.808 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.809 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.810 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:09:23.811 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.812 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.813 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.814 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.815 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.816 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.817 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: "num": 4 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:09:23.818 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff89003deb0 msgr2=0x7ff890040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff89003deb0 0x7ff890040370 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc003d00 tx=0x7ff8bc00b040 comp rx=0 tx=0).stop 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 10 monclient: shutdown 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006c890 msgr2=0x7ff8c01cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006c890 0x7ff8c01cd150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c450 tx=0x7ff8b000c920 comp rx=0 tx=0).stop 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c5ed7640 1 -- 10.20.193.78:0/114708423 reap_dead start 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 shutdown_connections 2026-02-17T06:09:23.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 --2- 10.20.193.78:0/114708423 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff8c006c890 0x7ff8c01cd150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 >> 10.20.193.78:0/114708423 conn(0x7ff8c00a6620 msgr2=0x7ff8c00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:23.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 shutdown_connections 2026-02-17T06:09:23.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.791+0000 7ff8c7961640 1 -- 10.20.193.78:0/114708423 wait complete. 2026-02-17T06:09:23.820 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.791+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.820 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.791+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416992244}) 2026-02-17T06:09:23.820 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.791+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4137546630}) 2026-02-17T06:09:23.820 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.791+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.820 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.791+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416992244}) 2026-02-17T06:09:23.820 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.791+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1696628955}) 2026-02-17T06:09:23.821 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:09:23.821 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:09:23.821 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:09:23.867 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.866+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.867 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.866+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3320938090}) 2026-02-17T06:09:23.868 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.866+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2845228142}) 2026-02-17T06:09:23.868 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.866+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.868 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.866+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3320938090}) 2026-02-17T06:09:23.868 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.866+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729848445}) 2026-02-17T06:09:23.869 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.868+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.870 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.868+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1897256748}) 2026-02-17T06:09:23.870 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.868+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=581267743}) 2026-02-17T06:09:23.870 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.868+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.870 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.868+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1897256748}) 2026-02-17T06:09:23.870 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.868+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3293815448}) 2026-02-17T06:09:23.871 INFO:tasks.rados.rados.0.trial115.stdout:3763: done (5 left) 2026-02-17T06:09:23.871 INFO:tasks.rados.rados.0.trial115.stdout:3764: write oid 39 current snap is 400 2026-02-17T06:09:23.871 INFO:tasks.rados.rados.0.trial115.stdout:3764: seq_num 1284 ranges {780188=567020,2103350=472436,3227358=352830} 2026-02-17T06:09:23.872 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.871+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.872 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.871+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2402483300}) 2026-02-17T06:09:23.872 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.871+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=605694810}) 2026-02-17T06:09:23.873 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.871+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.873 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.871+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2402483300}) 2026-02-17T06:09:23.873 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.871+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2707220350}) 2026-02-17T06:09:23.875 INFO:tasks.rados.rados.0.trial115.stdout:3764: writing trial11545420-39 from 780188 to 1347208 tid 1 2026-02-17T06:09:23.879 INFO:tasks.rados.rados.0.trial115.stdout:3764: writing trial11545420-39 from 2103350 to 2575786 tid 2 2026-02-17T06:09:23.884 INFO:tasks.rados.rados.0.trial115.stdout:3764: writing trial11545420-39 from 3227358 to 3580188 tid 3 2026-02-17T06:09:23.884 INFO:tasks.rados.rados.0.trial115.stdout:3765: snap_remove snap 396 2026-02-17T06:09:23.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.890+0000 7fd41e67e640 10 monclient: get_monmap_and_config 2026-02-17T06:09:23.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.890+0000 7fd41e67e640 10 monclient: build_initial_monmap 2026-02-17T06:09:23.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.890+0000 7fd41e67e640 10 monclient: monmap: 2026-02-17T06:09:23.913 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:23.913 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.913 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:23.891738+0000 2026-02-17T06:09:23.913 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:23.891738+0000 2026-02-17T06:09:23.913 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:23.913 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.913 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:23.914 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:23.914 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:23.914 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:23.914 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:23.914 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.890+0000 7fd41e67e640 1 Processor -- start 2026-02-17T06:09:23.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 1 -- start start 2026-02-17T06:09:23.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient: init 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd41806c890 0x7fd41806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient(hunting): picked mon.noname-a con 0x7fd41806c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd41809d430 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 0x7fd41806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient(hunting): picked mon.noname-b con 0x7fd41809bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd418073600 con 0x7fd41809d430 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd41809ca50 con 0x7fd41806c890 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd41806d6f0 con 0x7fd41809bf40 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:23.915 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41e67e640 10 monclient(hunting): authenticate will time out at 2009.445368s 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd41806c890 0x7fd41806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd41806c890 0x7fd41806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52138/0 (socket says 10.20.193.78:52138) 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 1 -- 10.20.193.78:0/2263324496 learned_addr learned my addr 10.20.193.78:0/2263324496 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 0x7fd41806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient(hunting): get_auth_request con 0x7fd41806c890 auth_method 0 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 10 monclient(hunting): get_auth_request con 0x7fd41809bf40 auth_method 0 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient(hunting): handle_auth_done global_id 284148 payload 274 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 1 -- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 msgr2=0x7fd41806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 1 --2- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 0x7fd41806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 1 -- 10.20.193.78:0/2263324496 >> v1:10.20.193.115:6789/0 conn(0x7fd41809d430 legacy=0x7fd418134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient: _finish_hunting 0 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 1 monclient: found mon.noname-a 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd417fff640 1 -- 10.20.193.78:0/2263324496 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41806dab0 con 0x7fd41806c890 2026-02-17T06:09:23.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.891+0000 7fd41cbf4640 1 --2- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 0x7fd41806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd417fff640 1 -- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 msgr2=0x7fd41806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd417fff640 1 --2- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 0x7fd41806cc70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd41809cbd0 tx=0x7fd40402ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b87c7fdb0a355a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 1 -- 10.20.193.78:0/2263324496 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40403c070 con 0x7fd41806c890 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: dump: 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.918 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: _finish_auth 0 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 1 -- 10.20.193.78:0/2263324496 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40402fcd0 con 0x7fd41806c890 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 5 monclient: authenticate success, global_id 284148 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 1 -- 10.20.193.78:0/2263324496 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404037660 con 0x7fd41806c890 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd416ffd640 10 monclient: dump: 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.919 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 4 set_mon_vals no callback set 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 10 monclient: shutdown 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 1 -- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 msgr2=0x7fd41806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 1 --2- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 0x7fd41806cc70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd41809cbd0 tx=0x7fd40402ee90 comp rx=0 tx=0).stop 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 1 -- 10.20.193.78:0/2263324496 shutdown_connections 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 1 --2- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 0x7fd41806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 1 --2- 10.20.193.78:0/2263324496 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 0x7fd41806cc70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 1 -- 10.20.193.78:0/2263324496 >> 10.20.193.78:0/2263324496 conn(0x7fd4180a6620 msgr2=0x7fd4180a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 1 -- 10.20.193.78:0/2263324496 shutdown_connections 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.894+0000 7fd41e67e640 1 -- 10.20.193.78:0/2263324496 wait complete. 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient: build_initial_monmap 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient: monmap: 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 1 Processor -- start 2026-02-17T06:09:23.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 1 -- start start 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient: init 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 0x7fd4181cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient(hunting): picked mon.noname-a con 0x7fd41806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 0x7fd4181cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient(hunting): picked mon.noname-c con 0x7fd41809bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd41809d430 0x7fd4181218a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient(hunting): picked mon.noname-b con 0x7fd41809d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd41809c7f0 con 0x7fd41806c890 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd418073410 con 0x7fd41809d430 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd418135500 con 0x7fd41809bf40 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:23.922 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd41e67e640 10 monclient(hunting): authenticate will time out at 2009.449779s 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 0x7fd4181cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd4177fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 0x7fd4181cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 0x7fd4181cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52140/0 (socket says 10.20.193.78:52140) 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 -- 10.20.193.78:0/1379691505 learned_addr learned my addr 10.20.193.78:0/1379691505 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient(hunting): get_auth_request con 0x7fd41806c890 auth_method 0 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd4177fe640 10 monclient(hunting): get_auth_request con 0x7fd41809bf40 auth_method 0 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd4177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd4177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd4177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient(hunting): handle_auth_done global_id 284151 payload 824 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 msgr2=0x7fd4181cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd41809bf40 0x7fd4181cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd41809d430 msgr2=0x7fd4181218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd41809d430 0x7fd4181218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient: _finish_hunting 0 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 monclient: found mon.noname-a 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.895+0000 7fd417fff640 1 -- 10.20.193.78:0/1379691505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd418123540 con 0x7fd41806c890 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd417fff640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 msgr2=0x7fd4181cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd417fff640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 0x7fd4181cf560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd404002410 tx=0x7fd404004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40403c040 con 0x7fd41806c890 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: dump: 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.924 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: _finish_auth 0 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd404047070 con 0x7fd41806c890 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 5 monclient: authenticate success, global_id 284151 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 10 monclient: _renew_subs 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:23.925 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd418125830 con 0x7fd41806c890 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040428b0 con 0x7fd41806c890 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd414ff9640 10 monclient: dump: 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd4167fc640 4 set_mon_vals no callback set 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 10 monclient: _renew_subs 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4180a1ff0 con 0x7fd41806c890 2026-02-17T06:09:23.926 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd4181225a0 0x7fd41811cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd418074980 con 0x7fd4181225a0 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.898+0000 7fd4177fe640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd4181225a0 0x7fd41811cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 10 monclient: get_auth_request con 0x7fd4181225a0 auth_method 0 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd404042a50 con 0x7fd41806c890 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd414ff9640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3e803df00 0x7fd3e80403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(828..828 src has 525..828) ==== 4871+0+0 (secure 0 0 0) 0x7fd404077a10 con 0x7fd41806c890 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 10 monclient(hunting): handle_auth_done global_id 274207 payload 824 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd4177fe640 1 -- 10.20.193.78:0/1379691505 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd4181358d0 con 0x7fd4181225a0 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd41cbf4640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3e803df00 0x7fd3e80403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.899+0000 7fd41cbf4640 10 monclient: get_auth_request con 0x7fd3e803df00 auth_method 0 2026-02-17T06:09:23.927 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd4177fe640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd4181225a0 msgr2=0x7fd41811cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd4177fe640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd4181225a0 0x7fd41811cc70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd418123140 tx=0x7fd40800fc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408018030 con 0x7fd4181225a0 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 10 monclient: dump: 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fd40800dbc0 con 0x7fd4181225a0 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:23.928 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd4181225a0 msgr2=0x7fd41811cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd4181225a0 0x7fd41811cc70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd418123140 tx=0x7fd40800fc10 comp rx=0 tx=0).stop 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd4181225a0 msgr2=0x7fd41811cc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd414ff9640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd4181225a0 0x7fd41811cc70 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd418123140 tx=0x7fd40800fc10 comp rx=0 tx=0).stop 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.901+0000 7fd41cbf4640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3e803df00 0x7fd3e80403c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e250 tx=0x7fd40c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.904+0000 7fd41e67e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.904+0000 7fd41e67e640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd418122980 0x7fd4181da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.904+0000 7fd41e67e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.904+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4180a1a80 con 0x7fd418122980 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.904+0000 7fd4177fe640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd418122980 0x7fd4181da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 10 monclient: get_auth_request con 0x7fd418122980 auth_method 0 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 10 monclient(hunting): handle_auth_done global_id 274210 payload 824 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.905+0000 7fd4177fe640 1 -- 10.20.193.78:0/1379691505 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd41811e330 con 0x7fd418122980 2026-02-17T06:09:23.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd4177fe640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd418122980 msgr2=0x7fd4181da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd4177fe640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd418122980 0x7fd4181da560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd408018000 tx=0x7fd408003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408004000 con 0x7fd418122980 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 10 monclient: dump: 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7fd408004000 con 0x7fd418122980 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd418122980 msgr2=0x7fd4181da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd418122980 0x7fd4181da560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd408018000 tx=0x7fd408003860 comp rx=0 tx=0).stop 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd418122980 msgr2=0x7fd4181da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.907+0000 7fd414ff9640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd418122980 0x7fd4181da560 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd408018000 tx=0x7fd408003860 comp rx=0 tx=0).stop 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 258, 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 65909, 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:09:23.931 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 2, 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:09:23.932 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:09:23.933 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:09:23.934 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.935 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.936 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.937 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:09:23.938 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:23.939 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:23.939 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:23.939 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:23.939 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:23.939 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:23.939 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:23.939 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:09:23.939 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3e803df00 msgr2=0x7fd3e80403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd3e803df00 0x7fd3e80403c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e250 tx=0x7fd40c00b040 comp rx=0 tx=0).stop 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 10 monclient: shutdown 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 msgr2=0x7fd4181cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 1 --2- 10.20.193.78:0/1379691505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd41806c890 0x7fd4181cf560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd404002410 tx=0x7fd404004830 comp rx=0 tx=0).stop 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41cbf4640 1 -- 10.20.193.78:0/1379691505 reap_dead start 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41cbf4640 1 -- 10.20.193.78:0/1379691505 reap_dead start 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 shutdown_connections 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 >> 10.20.193.78:0/1379691505 conn(0x7fd4180a6620 msgr2=0x7fd4181389a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 shutdown_connections 2026-02-17T06:09:23.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:23.908+0000 7fd41e67e640 1 -- 10.20.193.78:0/1379691505 wait complete. 2026-02-17T06:09:23.940 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:09:23.940 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T06:09:23.940 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-17T06:09:23.941 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T06:09:23.941 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:09:23.941 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T06:09:23.941 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:09:23.949 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.948+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.949 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.948+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014195639}) 2026-02-17T06:09:23.949 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.948+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1345560529}) 2026-02-17T06:09:23.949 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.948+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:09:23.949 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.948+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014195639}) 2026-02-17T06:09:23.949 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:09:23.948+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=59999990}) 2026-02-17T06:09:23.950 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T06:09:23.950 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:09:23.950 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:24.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.014+0000 7f8b57361640 10 monclient: get_monmap_and_config 2026-02-17T06:09:24.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.014+0000 7f8b57361640 10 monclient: build_initial_monmap 2026-02-17T06:09:24.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.014+0000 7f8b57361640 10 monclient: monmap: 2026-02-17T06:09:24.015 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:24.015 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:24.015 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:24.016142+0000 2026-02-17T06:09:24.015 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:24.016142+0000 2026-02-17T06:09:24.016 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:24.016 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:24.016 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:24.016 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:24.016 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:24.016 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:24.016 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:24.016 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 1 Processor -- start 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 1 -- start start 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient: init 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b500a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient(hunting): picked mon.noname-b con 0x7f8b5006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8b5006cc70 0x7f8b5006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient(hunting): picked mon.noname-a con 0x7f8b5006cc70 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8b50134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b50074150 con 0x7f8b50134df0 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f8b5009ef70 con 0x7f8b5006cc70 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5009f250 con 0x7f8b5006c890 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.015+0000 7f8b57361640 10 monclient(hunting): authenticate will time out at 2009.569992s 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b500a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b558d7640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f8b50134df0 legacy=0x7f8b50135240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8b5006cc70 0x7f8b5006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b500a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53972/0 (socket says 10.20.193.78:53972) 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f8b5006cc70 0x7f8b5006d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52144/0 (socket says 10.20.193.78:52144) 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 1 -- 10.20.193.78:0/423344797 learned_addr learned my addr 10.20.193.78:0/423344797 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient(hunting): get_auth_request con 0x7f8b5006c890 auth_method 0 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 10 monclient(hunting): get_auth_request con 0x7f8b5006cc70 auth_method 0 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:24.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b548d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient(hunting): handle_auth_done global_id 264365 payload 274 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 1 -- 10.20.193.78:0/423344797 >> v2:10.20.193.78:3300/0 conn(0x7f8b5006cc70 msgr2=0x7f8b5006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 1 --2- 10.20.193.78:0/423344797 >> v2:10.20.193.78:3300/0 conn(0x7f8b5006cc70 0x7f8b5006d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 1 -- 10.20.193.78:0/423344797 >> v1:10.20.193.115:6789/0 conn(0x7f8b50134df0 legacy=0x7f8b50135240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient: _finish_hunting 0 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 1 monclient: found mon.noname-b 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:09:24.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.016+0000 7f8b550d6640 1 -- 10.20.193.78:0/423344797 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5009df40 con 0x7f8b5006c890 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b550d6640 1 -- 10.20.193.78:0/423344797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 msgr2=0x7f8b500a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b550d6640 1 --2- 10.20.193.78:0/423344797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b500a01f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009970 tx=0x7f8b4002efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633b3b987e9ff19d server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 1 -- 10.20.193.78:0/423344797 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4003c070 con 0x7f8b5006c890 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: dump: 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:24.019 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: _finish_auth 0 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b57361640 5 monclient: authenticate success, global_id 264365 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 1 -- 10.20.193.78:0/423344797 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4002fdf0 con 0x7f8b5006c890 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b57361640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 1 -- 10.20.193.78:0/423344797 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40037660 con 0x7f8b5006c890 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b57361640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b57361640 4 set_mon_vals no callback set 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.018+0000 7f8b3ffff640 10 monclient: dump: 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 10 monclient: shutdown 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 -- 10.20.193.78:0/423344797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 msgr2=0x7f8b500a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 --2- 10.20.193.78:0/423344797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b500a01f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009970 tx=0x7f8b4002efb0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 -- 10.20.193.78:0/423344797 shutdown_connections 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 --2- 10.20.193.78:0/423344797 >> v2:10.20.193.78:3300/0 conn(0x7f8b5006cc70 0x7f8b5006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 --2- 10.20.193.78:0/423344797 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b500a01f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 -- 10.20.193.78:0/423344797 >> 10.20.193.78:0/423344797 conn(0x7f8b500a6620 msgr2=0x7f8b500a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:24.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 -- 10.20.193.78:0/423344797 shutdown_connections 2026-02-17T06:09:24.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 -- 10.20.193.78:0/423344797 wait complete. 2026-02-17T06:09:24.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 10 monclient: build_initial_monmap 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 10 monclient: monmap: 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 Processor -- start 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 1 -- start start 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.019+0000 7f8b57361640 10 monclient: init 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b50133f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient(hunting): picked mon.noname-c con 0x7f8b5006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8b5006cc70 0x7f8b50120820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient(hunting): picked mon.noname-a con 0x7f8b5006cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8b50134df0 0x7f8b50120d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient(hunting): picked mon.noname-b con 0x7f8b50134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5009d360 con 0x7f8b5006cc70 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5009d1e0 con 0x7f8b50134df0 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8b50135950 con 0x7f8b5006c890 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b57361640 10 monclient(hunting): authenticate will time out at 2009.574090s 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b50133f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8b5006cc70 0x7f8b50120820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b50133f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:53980/0 (socket says 10.20.193.78:53980) 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 1 -- 10.20.193.78:0/1661942234 learned_addr learned my addr 10.20.193.78:0/1661942234 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 10 monclient(hunting): get_auth_request con 0x7f8b5006c890 auth_method 0 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:24.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b558d7640 1 -- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8b50134df0 msgr2=0x7f8b50120d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient(hunting): get_auth_request con 0x7f8b5006cc70 auth_method 0 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b558d7640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8b50134df0 0x7f8b50120d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient(hunting): handle_auth_done global_id 284157 payload 824 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 1 -- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 msgr2=0x7f8b50133f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b50133f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 1 -- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8b50134df0 msgr2=0x7f8b50120d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8b50134df0 0x7f8b50120d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient: _finish_hunting 0 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 1 monclient: found mon.noname-a 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b548d5640 1 -- 10.20.193.78:0/1661942234 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b501bd330 con 0x7f8b5006cc70 2026-02-17T06:09:24.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.020+0000 7f8b550d6640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b50133f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b548d5640 1 -- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8b5006cc70 msgr2=0x7f8b50120820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b548d5640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8b5006cc70 0x7f8b50120820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b44010fc0 tx=0x7f8b4400e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 1 -- 10.20.193.78:0/1661942234 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4401d070 con 0x7f8b5006cc70 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: dump: 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: _finish_auth 0 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 1 -- 10.20.193.78:0/1661942234 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4400d7f0 con 0x7f8b5006cc70 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 1 -- 10.20.193.78:0/1661942234 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400dbe0 con 0x7f8b5006cc70 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3f7fe640 4 set_mon_vals no callback set 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b3dffb640 10 monclient: dump: 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:24.024 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b57361640 5 monclient: authenticate success, global_id 284157 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b57361640 10 monclient: _renew_subs 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b57361640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b501228c0 con 0x7f8b5006cc70 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b57361640 10 monclient: _renew_subs 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b57361640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.022+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5009d810 con 0x7f8b5006cc70 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b57361640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b57361640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b57361640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b50074980 con 0x7f8b5006cc70 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b3dffb640 1 -- 10.20.193.78:0/1661942234 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8b4400c040 con 0x7f8b5006cc70 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b3dffb640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8b2003de60 0x7f8b20040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b3dffb640 1 -- 10.20.193.78:0/1661942234 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(828..828 src has 525..828) ==== 4871+0+0 (secure 0 0 0) 0x7f8b440551f0 con 0x7f8b5006cc70 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b550d6640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8b2003de60 0x7f8b20040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:24.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.023+0000 7f8b550d6640 10 monclient: get_auth_request con 0x7f8b2003de60 auth_method 0 2026-02-17T06:09:24.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.025+0000 7f8b3dffb640 1 -- 10.20.193.78:0/1661942234 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8b501bd330 con 0x7f8b5006cc70 2026-02-17T06:09:24.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.025+0000 7f8b3dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:24.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.025+0000 7f8b3dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:24.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.026+0000 7f8b550d6640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8b2003de60 0x7f8b20040320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8b40002410 tx=0x7f8b40033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:24.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.138+0000 7f8b57361640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:24.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.138+0000 7f8b57361640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:24.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.138+0000 7f8b57361640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:24.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.138+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8b50072db0 con 0x7f8b5006cc70 2026-02-17T06:09:24.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.138+0000 7f8b3dffb640 1 -- 10.20.193.78:0/1661942234 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f8b4400dd80 con 0x7f8b5006cc70 2026-02-17T06:09:24.140 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.138+0000 7f8b3dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:24.141 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.138+0000 7f8b3dffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:24.141 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":258,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:24.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8b2003de60 msgr2=0x7f8b20040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:24.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8b2003de60 0x7f8b20040320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8b40002410 tx=0x7f8b40033000 comp rx=0 tx=0).stop 2026-02-17T06:09:24.142 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 10 monclient: shutdown 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8b5006cc70 msgr2=0x7f8b50120820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8b5006cc70 0x7f8b50120820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b44010fc0 tx=0x7f8b4400e880 comp rx=0 tx=0).stop 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 shutdown_connections 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8b2003de60 0x7f8b20040320 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8b50134df0 0x7f8b50120d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8b5006cc70 0x7f8b50120820 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 --2- 10.20.193.78:0/1661942234 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8b5006c890 0x7f8b50133f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 >> 10.20.193.78:0/1661942234 conn(0x7f8b500a6620 msgr2=0x7f8b50073290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 shutdown_connections 2026-02-17T06:09:24.143 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:24.139+0000 7f8b57361640 1 -- 10.20.193.78:0/1661942234 wait complete. 2026-02-17T06:09:27.148 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:27.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.216+0000 7f4e16205640 10 monclient: get_monmap_and_config 2026-02-17T06:09:27.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.216+0000 7f4e16205640 10 monclient: build_initial_monmap 2026-02-17T06:09:27.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.216+0000 7f4e16205640 10 monclient: monmap: 2026-02-17T06:09:27.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:27.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:27.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:27.217707+0000 2026-02-17T06:09:27.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:27.217707+0000 2026-02-17T06:09:27.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.216+0000 7f4e16205640 1 Processor -- start 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.216+0000 7f4e16205640 1 -- start start 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.216+0000 7f4e16205640 10 monclient: init 2026-02-17T06:09:27.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient: _add_conns ranks=[2,0,4,1,3] 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e100a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient(hunting): picked mon.noname-b con 0x7f4e1006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4e1006cc70 addr v1:10.20.193.78:6789/0 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f4e1013a240 0x7f4e1013c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient(hunting): picked mon.noname-c con 0x7f4e1013a240 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e10074150 con 0x7f4e1006cc70 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f4e1009ef70 con 0x7f4e1013a240 2026-02-17T06:09:27.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1009f250 con 0x7f4e1006c890 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e16205640 10 monclient(hunting): authenticate will time out at 2012.771233s 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e100a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e100a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36514/0 (socket says 10.20.193.78:36514) 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/3136537235 learned_addr learned my addr 10.20.193.78:0/3136537235 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient(hunting): get_auth_request con 0x7f4e1006c890 auth_method 0 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0ffff640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f4e1013a240 msgr2=0x7f4e1013c630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:27.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0effd640 1 --1- 10.20.193.78:0/3136537235 >> v1:10.20.193.78:6789/0 conn(0x7f4e1006cc70 0x7f4e1006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55190/0 (socket says 10.20.193.78:55190) 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0ffff640 1 --2- 10.20.193.78:0/3136537235 >> v2:10.20.193.115:3300/0 conn(0x7f4e1013a240 0x7f4e1013c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient(hunting): handle_auth_done global_id 264371 payload 274 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/3136537235 >> v2:10.20.193.115:3300/0 conn(0x7f4e1013a240 msgr2=0x7f4e1013c630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 1 --2- 10.20.193.78:0/3136537235 >> v2:10.20.193.115:3300/0 conn(0x7f4e1013a240 0x7f4e1013c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/3136537235 >> v1:10.20.193.78:6789/0 conn(0x7f4e1006cc70 legacy=0x7f4e1006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:27.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:27.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 1 monclient: found mon.noname-b 2026-02-17T06:09:27.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:27.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:09:27.225 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.217+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/3136537235 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1009df40 con 0x7f4e1006c890 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.219+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/3136537235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 msgr2=0x7f4e100a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.219+0000 7f4e0f7fe640 1 --2- 10.20.193.78:0/3136537235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e100a01f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e1006e270 tx=0x7f4dfc02efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28f5a1cb8904bdb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 1 -- 10.20.193.78:0/3136537235 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc03c070 con 0x7f4e1006c890 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: dump: 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 1 -- 10.20.193.78:0/3136537235 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc037440 con 0x7f4e1006c890 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 5 monclient: authenticate success, global_id 264371 2026-02-17T06:09:27.229 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 1 -- 10.20.193.78:0/3136537235 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc037810 con 0x7f4e1006c890 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 4 set_mon_vals no callback set 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e0e7fc640 10 monclient: dump: 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 10 monclient: shutdown 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 1 -- 10.20.193.78:0/3136537235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 msgr2=0x7f4e100a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 1 --2- 10.20.193.78:0/3136537235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e100a01f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e1006e270 tx=0x7f4dfc02efb0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 1 -- 10.20.193.78:0/3136537235 shutdown_connections 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 1 --2- 10.20.193.78:0/3136537235 >> v2:10.20.193.115:3300/0 conn(0x7f4e1013a240 0x7f4e1013c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 1 --2- 10.20.193.78:0/3136537235 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e100a01f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 1 -- 10.20.193.78:0/3136537235 >> 10.20.193.78:0/3136537235 conn(0x7f4e100a6620 msgr2=0x7f4e100a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 1 -- 10.20.193.78:0/3136537235 shutdown_connections 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 1 -- 10.20.193.78:0/3136537235 wait complete. 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 10 monclient: build_initial_monmap 2026-02-17T06:09:27.230 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.220+0000 7f4e16205640 10 monclient: monmap: 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 1 Processor -- start 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 1 -- start start 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient: init 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e10135090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient(hunting): picked mon.noname-c con 0x7f4e1006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e1006cc70 0x7f4e101355d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient(hunting): picked mon.noname-a con 0x7f4e1006cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e1013a240 0x7f4e1011db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient(hunting): picked mon.noname-b con 0x7f4e1013a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1009bfa0 con 0x7f4e1006cc70 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1009be20 con 0x7f4e1013a240 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1009c120 con 0x7f4e1006c890 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e16205640 10 monclient(hunting): authenticate will time out at 2012.775501s 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e10135090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:27.231 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0effd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e1006cc70 0x7f4e101355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e10135090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:36520/0 (socket says 10.20.193.78:36520) 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/1492101310 learned_addr learned my addr 10.20.193.78:0/1492101310 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient(hunting): get_auth_request con 0x7f4e1006c890 auth_method 0 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0effd640 10 monclient(hunting): get_auth_request con 0x7f4e1006cc70 auth_method 0 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0effd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0ffff640 1 -- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e1013a240 msgr2=0x7f4e1011db00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0ffff640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e1013a240 0x7f4e1011db00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient(hunting): handle_auth_done global_id 264374 payload 824 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e1013a240 msgr2=0x7f4e1011db00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e1013a240 0x7f4e1011db00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/1492101310 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e1006cc70 msgr2=0x7f4e101355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e1006cc70 0x7f4e101355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 monclient: found mon.noname-c 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.221+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/1492101310 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e100a25e0 con 0x7f4e1006c890 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4e0f7fe640 1 -- 10.20.193.78:0/1492101310 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 msgr2=0x7f4e10135090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:27.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4e0f7fe640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e10135090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009a70 tx=0x7f4dfc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 1 -- 10.20.193.78:0/1492101310 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc046020 con 0x7f4e1006c890 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 10 monclient: dump: 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 10 monclient: _finish_auth 0 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 10 monclient: _check_auth_tickets 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 1 -- 10.20.193.78:0/1492101310 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc042830 con 0x7f4e1006c890 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4e16205640 5 monclient: authenticate success, global_id 264374 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4e16205640 10 monclient: _renew_subs 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4e16205640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1009d810 con 0x7f4e1006c890 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.223+0000 7f4debfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4debfff640 1 -- 10.20.193.78:0/1492101310 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc03c040 con 0x7f4e1006c890 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4debfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4debfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4e0dffb640 4 set_mon_vals no callback set 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4debfff640 10 monclient: dump: 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:27.233 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4e16205640 10 monclient: _renew_subs 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4e16205640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e10121480 con 0x7f4e1006c890 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4e16205640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4e16205640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4e16205640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1006d050 con 0x7f4e1006c890 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4debfff640 1 -- 10.20.193.78:0/1492101310 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4dfc03f030 con 0x7f4e1006c890 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4debfff640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4de003de60 0x7f4de0040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.224+0000 7f4debfff640 1 -- 10.20.193.78:0/1492101310 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(828..828 src has 525..828) ==== 4871+0+0 (secure 0 0 0) 0x7f4dfc077b60 con 0x7f4e1006c890 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.225+0000 7f4e0effd640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4de003de60 0x7f4de0040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.225+0000 7f4e0effd640 10 monclient: get_auth_request con 0x7f4de003de60 auth_method 0 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.227+0000 7f4debfff640 1 -- 10.20.193.78:0/1492101310 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4e1006d050 con 0x7f4e1006c890 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.227+0000 7f4debfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.227+0000 7f4debfff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:27.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.227+0000 7f4e0effd640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4de003de60 0x7f4de0040320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4e10135fe0 tx=0x7f4e00007f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:27.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.343+0000 7f4e16205640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:27.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.343+0000 7f4e16205640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:27.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.343+0000 7f4e16205640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:09:27.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.343+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4e100a01d0 con 0x7f4e1006c890 2026-02-17T06:09:27.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.344+0000 7f4debfff640 1 -- 10.20.193.78:0/1492101310 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f4dfc050070 con 0x7f4e1006c890 2026-02-17T06:09:27.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.344+0000 7f4debfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:27.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.344+0000 7f4debfff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:27.345 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":258,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:27.345 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4de003de60 msgr2=0x7f4de0040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4de003de60 0x7f4de0040320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4e10135fe0 tx=0x7f4e00007f60 comp rx=0 tx=0).stop 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 10 monclient: shutdown 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 msgr2=0x7f4e10135090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e10135090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009a70 tx=0x7f4dfc0047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 shutdown_connections 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4de003de60 0x7f4de0040320 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4e1013a240 0x7f4e1011db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4e1006cc70 0x7f4e101355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 --2- 10.20.193.78:0/1492101310 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4e1006c890 0x7f4e10135090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 >> 10.20.193.78:0/1492101310 conn(0x7f4e100a6620 msgr2=0x7f4e100a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 shutdown_connections 2026-02-17T06:09:27.346 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:27.345+0000 7f4e16205640 1 -- 10.20.193.78:0/1492101310 wait complete. 2026-02-17T06:09:30.354 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:30.423 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient: get_monmap_and_config 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient: build_initial_monmap 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient: monmap: 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:30.422350+0000 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:30.422350+0000 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:30.424 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 1 Processor -- start 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 1 -- start start 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient: init 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f13d406c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f13d409c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:30.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 0x7f13d413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient(hunting): picked mon.noname-c con 0x7f13d406d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d406daf0 con 0x7f13d406c890 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d409cc80 con 0x7f13d409c4a0 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f13d40a0880 con 0x7f13d406d380 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d9ff7640 10 monclient(hunting): authenticate will time out at 2015.975889s 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d37fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f13d406c890 0x7f13d406cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55192/0 (socket says 10.20.193.78:55192) 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d3fff640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 msgr2=0x7f13d413ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d37fe640 1 -- 10.20.193.78:0/1250523427 learned_addr learned my addr 10.20.193.78:0/1250523427 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d2ffd640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f13d409c4a0 legacy=0x7f13d413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:30.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.421+0000 7f13d3fff640 1 --2- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 0x7f13d413ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:30.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.424+0000 7f13d37fe640 1 -- 10.20.193.78:0/1250523427 >> v1:10.20.193.78:6789/0 conn(0x7f13d406c890 legacy=0x7f13d406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:30.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.622+0000 7f13d3fff640 1 -- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 msgr2=0x7f13d413ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:30.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.622+0000 7f13d3fff640 1 --2- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 0x7f13d413ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:09:30.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:30.622+0000 7f13d2ffd640 1 -- 10.20.193.78:0/1250523427 >> v1:10.20.193.115:6789/0 conn(0x7f13d409c4a0 legacy=0x7f13d413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:31.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:31.022+0000 7f13d3fff640 1 -- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 msgr2=0x7f13d413ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:31.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:31.022+0000 7f13d3fff640 1 --2- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 0x7f13d413ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:09:31.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:31.022+0000 7f13d2ffd640 1 -- 10.20.193.78:0/1250523427 >> v1:10.20.193.115:6789/0 conn(0x7f13d409c4a0 legacy=0x7f13d413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:31.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:31.823+0000 7f13d3fff640 1 -- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 msgr2=0x7f13d413ead0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:31.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:31.823+0000 7f13d3fff640 1 --2- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 0x7f13d413ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:09:31.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:31.823+0000 7f13d2ffd640 1 -- 10.20.193.78:0/1250523427 >> v1:10.20.193.115:6789/0 conn(0x7f13d409c4a0 legacy=0x7f13d413b3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:33.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient(hunting): tick 2026-02-17T06:09:33.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:09:33.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 monclient(hunting): continuing hunt 2026-02-17T06:09:33.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:09:33.425 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 -- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 msgr2=0x7f13d413ead0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 --2- 10.20.193.78:0/1250523427 >> v2:10.20.193.115:3300/0 conn(0x7f13d406d380 0x7f13d413ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 -- 10.20.193.78:0/1250523427 >> v1:10.20.193.115:6789/0 conn(0x7f13d409c4a0 legacy=0x7f13d413b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 -- 10.20.193.78:0/1250523427 >> v1:10.20.193.78:6789/0 conn(0x7f13d406c890 legacy=0x7f13d406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient: _add_conns ranks=[2,3] 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 --2- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13b0003830 0x7f13b0005c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient(hunting): picked mon.noname-b con 0x7f13b0003830 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 --2- 10.20.193.78:0/1250523427 >> v2:10.20.193.78:3300/0 conn(0x7f13b0006340 0x7f13b0008800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient(hunting): picked mon.noname-a con 0x7f13b0006340 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 -- 10.20.193.78:0/1250523427 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f13b0008e90 con 0x7f13b0006340 2026-02-17T06:09:33.426 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 1 -- 10.20.193.78:0/1250523427 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f13b0009040 con 0x7f13b0003830 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d17fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 1 --2- 10.20.193.78:0/1250523427 >> v2:10.20.193.78:3300/0 conn(0x7f13b0006340 0x7f13b0008800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 1 --2- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13b0003830 0x7f13b0005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient(hunting): get_auth_request con 0x7f13b0006340 auth_method 0 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 10 monclient(hunting): get_auth_request con 0x7f13b0003830 auth_method 0 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:33.427 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient(hunting): handle_auth_done global_id 284163 payload 274 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 1 -- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13b0003830 msgr2=0x7f13b0005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 1 --2- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13b0003830 0x7f13b0005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 1 monclient: found mon.noname-a 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:33.428 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:33.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d2ffd640 1 -- 10.20.193.78:0/1250523427 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d409cf60 con 0x7f13b0006340 2026-02-17T06:09:33.429 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.422+0000 7f13d37fe640 1 --2- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13b0003830 0x7f13b0005c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:33.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.424+0000 7f13d2ffd640 1 -- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 msgr2=0x7f13b0008800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:33.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.424+0000 7f13d2ffd640 1 --2- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13b0008800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f13c40039b0 tx=0x7f13c402f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deeddbb52832d820 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:33.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 1 -- 10.20.193.78:0/1250523427 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4037630 con 0x7f13b0006340 2026-02-17T06:09:33.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: dump: 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:33.437 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: _finish_auth 0 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: _check_auth_tickets 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 1 -- 10.20.193.78:0/1250523427 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c40377f0 con 0x7f13b0006340 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 5 monclient: authenticate success, global_id 284163 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 1 -- 10.20.193.78:0/1250523427 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4037be0 con 0x7f13b0006340 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 4 set_mon_vals no callback set 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:33.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d27fc640 10 monclient: dump: 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:33.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 10 monclient: shutdown 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 msgr2=0x7f13b0008800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 1 --2- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13b0008800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f13c40039b0 tx=0x7f13c402f7d0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d3fff640 1 -- 10.20.193.78:0/1250523427 reap_dead start 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1250523427 shutdown_connections 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 1 --2- 10.20.193.78:0/1250523427 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13b0008800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1250523427 >> 10.20.193.78:0/1250523427 conn(0x7f13d40a6620 msgr2=0x7f13d40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1250523427 shutdown_connections 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.425+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1250523427 wait complete. 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient: build_initial_monmap 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient: monmap: 2026-02-17T06:09:33.440 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 1 Processor -- start 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 1 -- start start 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient: init 2026-02-17T06:09:33.441 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13d41362b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient(hunting): picked mon.noname-a con 0x7f13b0006340 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13d41cf1c0 0x7f13d411dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient(hunting): picked mon.noname-c con 0x7f13d41cf1c0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13d41367f0 0x7f13d411e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient(hunting): picked mon.noname-b con 0x7f13d41367f0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13d409bfa0 con 0x7f13b0006340 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:33.442 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13d409be20 con 0x7f13d41367f0 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f13d413f010 con 0x7f13d41cf1c0 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d9ff7640 10 monclient(hunting): authenticate will time out at 2018.980753s 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13d41362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13d41362b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57388/0 (socket says 10.20.193.78:57388) 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 1 -- 10.20.193.78:0/1561399569 learned_addr learned my addr 10.20.193.78:0/1561399569 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 10 monclient(hunting): get_auth_request con 0x7f13b0006340 auth_method 0 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:33.443 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d2ffd640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13d41cf1c0 0x7f13d411dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d3fff640 1 -- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13d41367f0 msgr2=0x7f13d411e2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d3fff640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13d41367f0 0x7f13d411e2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d2ffd640 10 monclient(hunting): get_auth_request con 0x7f13d41cf1c0 auth_method 0 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:33.444 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 10 monclient(hunting): handle_auth_done global_id 284166 payload 824 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 1 -- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13d41cf1c0 msgr2=0x7f13d411dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13d41cf1c0 0x7f13d411dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 1 -- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13d41367f0 msgr2=0x7f13d411e2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.426+0000 7f13d37fe640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13d41367f0 0x7f13d411e2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.427+0000 7f13d37fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.427+0000 7f13d37fe640 1 monclient: found mon.noname-a 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.427+0000 7f13d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.427+0000 7f13d37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.427+0000 7f13d37fe640 1 -- 10.20.193.78:0/1561399569 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d41cfe70 con 0x7f13b0006340 2026-02-17T06:09:33.445 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d37fe640 1 -- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 msgr2=0x7f13d41362b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d37fe640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13d41362b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13c0007340 tx=0x7f13c0007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 1 -- 10.20.193.78:0/1561399569 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c000fbd0 con 0x7f13b0006340 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: dump: 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:33.446 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: _finish_auth 0 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: _check_auth_tickets 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 1 -- 10.20.193.78:0/1561399569 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c0018070 con 0x7f13b0006340 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 1 -- 10.20.193.78:0/1561399569 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00133f0 con 0x7f13b0006340 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d1ffb640 4 set_mon_vals no callback set 2026-02-17T06:09:33.447 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13affff640 10 monclient: dump: 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 5 monclient: authenticate success, global_id 284166 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 10 monclient: _renew_subs 2026-02-17T06:09:33.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d4121c50 con 0x7f13b0006340 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 10 monclient: _renew_subs 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d413e4f0 con 0x7f13b0006340 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.429+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d406cc70 con 0x7f13b0006340 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.430+0000 7f13affff640 1 -- 10.20.193.78:0/1561399569 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f13c00135d0 con 0x7f13b0006340 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.430+0000 7f13affff640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f13a403de90 0x7f13a4040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:33.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.430+0000 7f13affff640 1 -- 10.20.193.78:0/1561399569 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(828..828 src has 525..828) ==== 4871+0+0 (secure 0 0 0) 0x7f13c0052820 con 0x7f13b0006340 2026-02-17T06:09:33.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.430+0000 7f13d2ffd640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f13a403de90 0x7f13a4040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:33.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.430+0000 7f13d2ffd640 10 monclient: get_auth_request con 0x7f13a403de90 auth_method 0 2026-02-17T06:09:33.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.432+0000 7f13affff640 1 -- 10.20.193.78:0/1561399569 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f13c001cc70 con 0x7f13b0006340 2026-02-17T06:09:33.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.432+0000 7f13affff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:33.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.432+0000 7f13affff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:33.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.432+0000 7f13d2ffd640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f13a403de90 0x7f13a4040350 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f13b0006310 tx=0x7f13c4032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:33.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.551+0000 7f13d9ff7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:33.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.551+0000 7f13d9ff7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:33.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.551+0000 7f13d9ff7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:33.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.551+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f13d411f960 con 0x7f13b0006340 2026-02-17T06:09:33.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.551+0000 7f13affff640 1 -- 10.20.193.78:0/1561399569 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f13c0054c70 con 0x7f13b0006340 2026-02-17T06:09:33.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.551+0000 7f13affff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:33.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.551+0000 7f13affff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:33.552 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":258,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f13a403de90 msgr2=0x7f13a4040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f13a403de90 0x7f13a4040350 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f13b0006310 tx=0x7f13c4032000 comp rx=0 tx=0).stop 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 10 monclient: shutdown 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 msgr2=0x7f13d41362b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13d41362b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13c0007340 tx=0x7f13c0007810 comp rx=0 tx=0).stop 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 shutdown_connections 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f13a403de90 0x7f13a4040350 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f13d41367f0 0x7f13d411e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f13d41cf1c0 0x7f13d411dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 --2- 10.20.193.78:0/1561399569 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f13b0006340 0x7f13d41362b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 >> 10.20.193.78:0/1561399569 conn(0x7f13d40a6620 msgr2=0x7f13d40a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 shutdown_connections 2026-02-17T06:09:33.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:33.552+0000 7f13d9ff7640 1 -- 10.20.193.78:0/1561399569 wait complete. 2026-02-17T06:09:36.561 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:36.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.629+0000 7f5b8bfff640 10 monclient: get_monmap_and_config 2026-02-17T06:09:36.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.629+0000 7f5b8bfff640 10 monclient: build_initial_monmap 2026-02-17T06:09:36.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.629+0000 7f5b8bfff640 10 monclient: monmap: 2026-02-17T06:09:36.630 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:36.630 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:36.630 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:36.630346+0000 2026-02-17T06:09:36.630 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:36.630346+0000 2026-02-17T06:09:36.631 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:36.631 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:36.631 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:36.631 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:36.631 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:36.631 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:36.631 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:36.631 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:36.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.629+0000 7f5b8bfff640 1 Processor -- start 2026-02-17T06:09:36.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.629+0000 7f5b8bfff640 1 -- start start 2026-02-17T06:09:36.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.629+0000 7f5b8bfff640 10 monclient: init 2026-02-17T06:09:36.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.629+0000 7f5b8bfff640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:36.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 10 monclient: _add_conns ranks=[4,0,1,2,3] 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 0x7f5b8c0a0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 10 monclient(hunting): picked mon.noname-c con 0x7f5b8c06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5b8c06cc70 addr v1:10.20.193.78:6789/0 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b8c13a0a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c09dd50 con 0x7f5b8c06cc70 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c09dad0 con 0x7f5b8c13a0a0 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c0741c0 con 0x7f5b8c06c890 2026-02-17T06:09:36.634 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:36.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8bfff640 10 monclient(hunting): authenticate will time out at 2022.184099s 2026-02-17T06:09:36.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8affd640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 msgr2=0x7f5b8c0a0070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:36.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8affd640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 0x7f5b8c0a0070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:36.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8b7fe640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f5b8c13a0a0 legacy=0x7f5b8c13c490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:36.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8a7fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f5b8c06cc70 0x7f5b8c06d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55204/0 (socket says 10.20.193.78:55204) 2026-02-17T06:09:36.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.630+0000 7f5b8a7fc640 1 -- 10.20.193.78:0/2829877192 learned_addr learned my addr 10.20.193.78:0/2829877192 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:09:36.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.632+0000 7f5b8a7fc640 1 -- 10.20.193.78:0/2829877192 >> v1:10.20.193.78:6789/0 conn(0x7f5b8c06cc70 legacy=0x7f5b8c06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:36.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.830+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2829877192 >> v1:10.20.193.115:6789/0 conn(0x7f5b8c13a0a0 legacy=0x7f5b8c13c490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:36.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.830+0000 7f5b8affd640 1 -- 10.20.193.78:0/2829877192 >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 msgr2=0x7f5b8c0a0070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:36.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:36.830+0000 7f5b8affd640 1 --2- 10.20.193.78:0/2829877192 >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 0x7f5b8c0a0070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:09:37.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:37.230+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2829877192 >> v1:10.20.193.115:6789/0 conn(0x7f5b8c13a0a0 legacy=0x7f5b8c13c490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:37.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:37.230+0000 7f5b8affd640 1 -- 10.20.193.78:0/2829877192 >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 msgr2=0x7f5b8c0a0070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:37.232 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:37.230+0000 7f5b8affd640 1 --2- 10.20.193.78:0/2829877192 >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 0x7f5b8c0a0070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:09:38.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.031+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2829877192 >> v1:10.20.193.115:6789/0 conn(0x7f5b8c13a0a0 legacy=0x7f5b8c13c490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:38.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.031+0000 7f5b8affd640 1 -- 10.20.193.78:0/2829877192 >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 msgr2=0x7f5b8c0a0070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:38.032 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.031+0000 7f5b8affd640 1 --2- 10.20.193.78:0/2829877192 >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 0x7f5b8c0a0070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:09:38.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.878+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299916044 0 0) 0x7f5b8c09dd50 con 0x7f5b8c06cc70 2026-02-17T06:09:38.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.879+0000 7f5b89ffb640 10 monclient(hunting): _negotiate 2026-02-17T06:09:38.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.879+0000 7f5b89ffb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:38.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.879+0000 7f5b89ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:38.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.879+0000 7f5b89ffb640 10 monclient(hunting): my global_id is 284169 2026-02-17T06:09:38.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.879+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b78003620 con 0x7f5b8c06cc70 2026-02-17T06:09:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2300389804 0 0) 0x7f5b78003620 con 0x7f5b8c06cc70 2026-02-17T06:09:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 msgr2=0x7f5b8c0a0070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 --2- 10.20.193.78:0/2829877192 >> v2:10.20.193.115:3300/0 conn(0x7f5b8c06c890 0x7f5b8c0a0070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 >> v1:10.20.193.115:6789/0 conn(0x7f5b8c13a0a0 legacy=0x7f5b8c13c490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:38.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: _finish_hunting 0 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c09edf0 con 0x7f5b8c06cc70 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: _finish_auth 0 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7f5b740030d0 con 0x7f5b8c06cc70 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: dump: 2026-02-17T06:09:38.893 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:09:38.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 >> v1:10.20.193.78:6789/0 conn(0x7f5b8c06cc70 legacy=0x7f5b8c06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b78003ab0 0x7f5b78003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient(hunting): picked mon.c con 0x7f5b78003ab0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b78004900 0x7f5b78004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient(hunting): picked mon.a con 0x7f5b78004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b780053e0 0x7f5b7800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient(hunting): picked mon.b con 0x7f5b780053e0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7800c300 con 0x7f5b78004900 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7800c4b0 con 0x7f5b780053e0 2026-02-17T06:09:38.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7800c6b0 con 0x7f5b78003ab0 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b89ffb640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b78004900 0x7f5b78004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b78003ab0 0x7f5b78003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient(hunting): get_auth_request con 0x7f5b78004900 auth_method 0 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2829877192 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b780053e0 msgr2=0x7f5b7800bac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 10 monclient(hunting): get_auth_request con 0x7f5b78003ab0 auth_method 0 2026-02-17T06:09:38.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8b7fe640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b780053e0 0x7f5b7800bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient(hunting): handle_auth_done global_id 284169 payload 293 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 1 -- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b78003ab0 msgr2=0x7f5b78003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:38.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b78003ab0 0x7f5b78003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 1 -- 10.20.193.78:0/2829877192 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b780053e0 msgr2=0x7f5b7800bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b780053e0 0x7f5b7800bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 1 monclient: found mon.a 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2829877192 reap_dead start 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 1 -- 10.20.193.78:0/2829877192 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b78003620 con 0x7f5b78004900 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:09:38.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8bfff640 5 monclient: authenticate success, global_id 284169 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8bfff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.880+0000 7f5b8affd640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b78003ab0 0x7f5b78003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8a7fc640 1 -- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b78004900 msgr2=0x7f5b78004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8a7fc640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b78004900 0x7f5b78004d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b740036c0 tx=0x7f5b7402f590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0a2d36c4154be6 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400c0e0 con 0x7f5b78004900 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 10 monclient: dump: 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.899 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b74005bf0 con 0x7f5b78004900 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 1 -- 10.20.193.78:0/2829877192 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74007070 con 0x7f5b78004900 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:38.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b89ffb640 10 monclient: dump: 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:38.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 4 set_mon_vals no callback set 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 10 monclient: shutdown 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b78004900 msgr2=0x7f5b78004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b78004900 0x7f5b78004d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b740036c0 tx=0x7f5b7402f590 comp rx=0 tx=0).stop 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2829877192 shutdown_connections 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 1 --2- 10.20.193.78:0/2829877192 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b78004900 0x7f5b78004d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2829877192 >> 10.20.193.78:0/2829877192 conn(0x7f5b8c0a65f0 msgr2=0x7f5b8c0a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2829877192 shutdown_connections 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.883+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2829877192 wait complete. 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient: build_initial_monmap 2026-02-17T06:09:38.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient: monmap: 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 1 Processor -- start 2026-02-17T06:09:38.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 1 -- start start 2026-02-17T06:09:38.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient: init 2026-02-17T06:09:38.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:38.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:09:38.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78004900 0x7f5b8c1ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:38.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient(hunting): picked mon.noname-b con 0x7f5b78004900 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:38.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b8c132350 0x7f5b8c070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:38.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient(hunting): picked mon.noname-c con 0x7f5b8c132350 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:38.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b8c1327d0 0x7f5b8c0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:38.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient(hunting): picked mon.noname-a con 0x7f5b8c1327d0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:38.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:38.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0a01f0 con 0x7f5b8c1327d0 2026-02-17T06:09:38.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:38.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0a0070 con 0x7f5b78004900 2026-02-17T06:09:38.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:38.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c0a0370 con 0x7f5b8c132350 2026-02-17T06:09:38.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:38.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8bfff640 10 monclient(hunting): authenticate will time out at 2024.438953s 2026-02-17T06:09:38.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8affd640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78004900 msgr2=0x7f5b8c1ca630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:38.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8affd640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78004900 0x7f5b8c1ca630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:38.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.884+0000 7f5b8b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b8c1327d0 0x7f5b8c0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:38.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8a7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b8c132350 0x7f5b8c070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:38.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b8c1327d0 0x7f5b8c0713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58190/0 (socket says 10.20.193.78:58190) 2026-02-17T06:09:38.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2409213302 learned_addr learned my addr 10.20.193.78:0/2409213302 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:38.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8a7fc640 10 monclient(hunting): get_auth_request con 0x7f5b8c132350 auth_method 0 2026-02-17T06:09:38.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:38.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:38.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:38.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient(hunting): get_auth_request con 0x7f5b8c1327d0 auth_method 0 2026-02-17T06:09:38.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient(hunting): handle_auth_done global_id 284172 payload 824 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2409213302 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b8c132350 msgr2=0x7f5b8c070e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b8c132350 0x7f5b8c070e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:38.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78004900 msgr2=0x7f5b8c1ca630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78004900 0x7f5b8c1ca630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 1 monclient: found mon.noname-a 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.885+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2409213302 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c11cc70 con 0x7f5b8c1327d0 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8b7fe640 1 -- 10.20.193.78:0/2409213302 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b8c1327d0 msgr2=0x7f5b8c0713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8b7fe640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b8c1327d0 0x7f5b8c0713d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0a2e70 tx=0x7f5b7c00e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 1 -- 10.20.193.78:0/2409213302 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00ecf0 con 0x7f5b8c1327d0 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:38.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: dump: 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: _finish_auth 0 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:09:38.910 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 1 -- 10.20.193.78:0/2409213302 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c00c0d0 con 0x7f5b8c1327d0 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 5 monclient: authenticate success, global_id 284172 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 10 monclient: _renew_subs 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c11d450 con 0x7f5b8c1327d0 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 1 -- 10.20.193.78:0/2409213302 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00c280 con 0x7f5b8c1327d0 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b897fa640 4 set_mon_vals no callback set 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:38.911 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b6b7fe640 10 monclient: dump: 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 10 monclient: _renew_subs 2026-02-17T06:09:38.912 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c072b30 con 0x7f5b8c1327d0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.887+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c06d080 con 0x7f5b8c1327d0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.888+0000 7f5b6b7fe640 1 -- 10.20.193.78:0/2409213302 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5b7c00a400 con 0x7f5b8c1327d0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.888+0000 7f5b6b7fe640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b5803de60 0x7f5b58040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.888+0000 7f5b6b7fe640 1 -- 10.20.193.78:0/2409213302 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(828..828 src has 525..828) ==== 4871+0+0 (secure 0 0 0) 0x7f5b7c054a10 con 0x7f5b8c1327d0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.888+0000 7f5b8affd640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b5803de60 0x7f5b58040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.888+0000 7f5b8affd640 10 monclient: get_auth_request con 0x7f5b5803de60 auth_method 0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.890+0000 7f5b8affd640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b5803de60 0x7f5b58040320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5b80003c50 tx=0x7f5b800094e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:38.913 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.891+0000 7f5b6b7fe640 1 -- 10.20.193.78:0/2409213302 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5b8c11d450 con 0x7f5b8c1327d0 2026-02-17T06:09:38.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.891+0000 7f5b6b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:38.914 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:38.891+0000 7f5b6b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.004+0000 7f5b8bfff640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.004+0000 7f5b8bfff640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.004+0000 7f5b8bfff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.004+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5b8c11d790 con 0x7f5b8c1327d0 2026-02-17T06:09:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.004+0000 7f5b6b7fe640 1 -- 10.20.193.78:0/2409213302 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f5b7c006c80 con 0x7f5b8c1327d0 2026-02-17T06:09:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.004+0000 7f5b6b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:39.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.004+0000 7f5b6b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:39.006 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":262,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:39.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b5803de60 msgr2=0x7f5b58040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b5803de60 0x7f5b58040320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5b80003c50 tx=0x7f5b800094e0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 10 monclient: shutdown 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b8c1327d0 msgr2=0x7f5b8c0713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b8c1327d0 0x7f5b8c0713d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0a2e70 tx=0x7f5b7c00e420 comp rx=0 tx=0).stop 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 shutdown_connections 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5b8c1327d0 0x7f5b8c0713d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5b8c132350 0x7f5b8c070e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5b5803de60 0x7f5b58040320 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 --2- 10.20.193.78:0/2409213302 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5b78004900 0x7f5b8c1ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 >> 10.20.193.78:0/2409213302 conn(0x7f5b8c0a65f0 msgr2=0x7f5b8c138710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:39.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 shutdown_connections 2026-02-17T06:09:39.009 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.005+0000 7f5b8bfff640 1 -- 10.20.193.78:0/2409213302 wait complete. 2026-02-17T06:09:39.014 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:09:39.014 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:09:39.014 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:09:39.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.079+0000 7fae0a8a0640 10 monclient: get_monmap_and_config 2026-02-17T06:09:39.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.079+0000 7fae0a8a0640 10 monclient: build_initial_monmap 2026-02-17T06:09:39.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.079+0000 7fae0a8a0640 10 monclient: monmap: 2026-02-17T06:09:39.080 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:39.080615+0000 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:39.080615+0000 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:39.081 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.079+0000 7fae0a8a0640 1 Processor -- start 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.079+0000 7fae0a8a0640 1 -- start start 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.079+0000 7fae0a8a0640 10 monclient: init 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae0409ea90 0x7fae040a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient(hunting): picked mon.noname-a con 0x7fae0409ea90 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fae0409ee70 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:39.092 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fae0413d640 0x7fae0413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient(hunting): picked mon.noname-c con 0x7fae0413d640 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae040a0a20 con 0x7fae0409ee70 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fae040a0d00 con 0x7fae0409ea90 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fae040a0460 con 0x7fae0413d640 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae0a8a0640 10 monclient(hunting): authenticate will time out at 2024.634262s 2026-02-17T06:09:39.093 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae08e16640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fae0413d640 msgr2=0x7fae0413fa30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae037fe640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fae0409ee70 legacy=0x7fae0409f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae0409ea90 0x7fae040a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae08e16640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fae0413d640 0x7fae0413fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae0409ea90 0x7fae040a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58206/0 (socket says 10.20.193.78:58206) 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 1 -- 10.20.193.78:0/1185205082 learned_addr learned my addr 10.20.193.78:0/1185205082 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient(hunting): get_auth_request con 0x7fae0409ea90 auth_method 0 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient(hunting): handle_auth_done global_id 284175 payload 274 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 1 -- 10.20.193.78:0/1185205082 >> v2:10.20.193.115:3300/0 conn(0x7fae0413d640 msgr2=0x7fae0413fa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 1 --2- 10.20.193.78:0/1185205082 >> v2:10.20.193.115:3300/0 conn(0x7fae0413d640 0x7fae0413fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 1 -- 10.20.193.78:0/1185205082 >> v1:10.20.193.115:6789/0 conn(0x7fae0409ee70 legacy=0x7fae0409f2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient: _finish_hunting 0 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 1 monclient: found mon.noname-a 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:39.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.080+0000 7fae03fff640 1 -- 10.20.193.78:0/1185205082 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae040a0740 con 0x7fae0409ea90 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae03fff640 1 -- 10.20.193.78:0/1185205082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 msgr2=0x7fae040a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae03fff640 1 --2- 10.20.193.78:0/1185205082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 0x7fae040a13d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae04135c40 tx=0x7fadf002ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db4a727e7c42676 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 1 -- 10.20.193.78:0/1185205082 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf003c070 con 0x7fae0409ea90 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: dump: 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.096 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: _finish_auth 0 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:39.097 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 5 monclient: authenticate success, global_id 284175 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 1 -- 10.20.193.78:0/1185205082 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf002fda0 con 0x7fae0409ea90 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 1 -- 10.20.193.78:0/1185205082 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0037660 con 0x7fae0409ea90 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 4 set_mon_vals no callback set 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae02ffd640 10 monclient: dump: 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.098 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 10 monclient: shutdown 2026-02-17T06:09:39.099 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 1 -- 10.20.193.78:0/1185205082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 msgr2=0x7fae040a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 1 --2- 10.20.193.78:0/1185205082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 0x7fae040a13d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae04135c40 tx=0x7fadf002ee80 comp rx=0 tx=0).stop 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 1 -- 10.20.193.78:0/1185205082 shutdown_connections 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 1 --2- 10.20.193.78:0/1185205082 >> v2:10.20.193.115:3300/0 conn(0x7fae0413d640 0x7fae0413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 1 --2- 10.20.193.78:0/1185205082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 0x7fae040a13d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 1 -- 10.20.193.78:0/1185205082 >> 10.20.193.78:0/1185205082 conn(0x7fae040a7520 msgr2=0x7fae040a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 1 -- 10.20.193.78:0/1185205082 shutdown_connections 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.083+0000 7fae0a8a0640 1 -- 10.20.193.78:0/1185205082 wait complete. 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient: build_initial_monmap 2026-02-17T06:09:39.100 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient: monmap: 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 1 Processor -- start 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 1 -- start start 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient: init 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 0x7fae04072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient(hunting): picked mon.noname-a con 0x7fae0409ea90 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0409ee70 0x7fae040a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient(hunting): picked mon.noname-b con 0x7fae0409ee70 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0413d640 0x7fae040a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.102 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient(hunting): picked mon.noname-c con 0x7fae0413d640 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae040a6820 con 0x7fae0409ea90 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae040a66a0 con 0x7fae0409ee70 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae0413ff70 con 0x7fae0413d640 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae0a8a0640 10 monclient(hunting): authenticate will time out at 2024.638547s 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 0x7fae04072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 0x7fae04072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58208/0 (socket says 10.20.193.78:58208) 2026-02-17T06:09:39.103 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 -- 10.20.193.78:0/586963692 learned_addr learned my addr 10.20.193.78:0/586963692 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient(hunting): get_auth_request con 0x7fae0409ea90 auth_method 0 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae037fe640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0409ee70 msgr2=0x7fae040a3560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae08e16640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0413d640 0x7fae040a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae037fe640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0409ee70 0x7fae040a3560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.104 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae08e16640 10 monclient(hunting): get_auth_request con 0x7fae0413d640 auth_method 0 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae08e16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae08e16640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae08e16640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae08e16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae08e16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae08e16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient(hunting): handle_auth_done global_id 284178 payload 824 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0413d640 msgr2=0x7fae040a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae0413d640 0x7fae040a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.105 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0409ee70 msgr2=0x7fae040a3560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae0409ee70 0x7fae040a3560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient: _finish_hunting 0 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 monclient: found mon.noname-a 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.084+0000 7fae03fff640 1 -- 10.20.193.78:0/586963692 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae04070ef0 con 0x7fae0409ea90 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.086+0000 7fae03fff640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 msgr2=0x7fae04072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.086+0000 7fae03fff640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 0x7fae04072df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fadf002f390 tx=0x7fadf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.086+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0046020 con 0x7fae0409ea90 2026-02-17T06:09:39.106 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: dump: 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.107 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: _finish_auth 0 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 5 monclient: authenticate success, global_id 284178 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 10 monclient: _renew_subs 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf0042620 con 0x7fae0409ea90 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae040711d0 con 0x7fae0409ea90 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:39.108 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf003c040 con 0x7fae0409ea90 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae027fc640 4 set_mon_vals no callback set 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 10 monclient: dump: 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.109 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 10 monclient: _renew_subs 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0406fd20 con 0x7fae0409ea90 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae040a3fe0 0x7fae0413a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.110 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae040a43c0 con 0x7fae040a3fe0 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae040a3fe0 0x7fae0413a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fadf003f030 con 0x7fae0409ea90 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 10 monclient: get_auth_request con 0x7fae040a3fe0 auth_method 0 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 10 monclient(hunting): handle_auth_done global_id 284181 payload 824 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae037fe640 1 -- 10.20.193.78:0/586963692 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae040a13d0 con 0x7fae040a3fe0 2026-02-17T06:09:39.111 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.087+0000 7fae00ff9640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadd403de90 0x7fadd4040350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.088+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(828..828 src has 525..828) ==== 4871+0+0 (secure 0 0 0) 0x7fadf00779e0 con 0x7fae0409ea90 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.088+0000 7fae08e16640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadd403de90 0x7fadd4040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.088+0000 7fae08e16640 10 monclient: get_auth_request con 0x7fadd403de90 auth_method 0 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae037fe640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae040a3fe0 msgr2=0x7fae0413a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae037fe640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae040a3fe0 0x7fae0413a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b680 tx=0x7fadf400bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4005270 con 0x7fae040a3fe0 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 10 monclient: dump: 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.112 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fadf4010070 con 0x7fae040a3fe0 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae040a3fe0 msgr2=0x7fae0413a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae040a3fe0 0x7fae0413a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b680 tx=0x7fadf400bb50 comp rx=0 tx=0).stop 2026-02-17T06:09:39.113 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae040a3fe0 msgr2=0x7fae0413a7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:39.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae00ff9640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae040a3fe0 0x7fae0413a7a0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b680 tx=0x7fadf400bb50 comp rx=0 tx=0).stop 2026-02-17T06:09:39.114 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.090+0000 7fae08e16640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadd403de90 0x7fadd4040350 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fadf8003db0 tx=0x7fadf80095f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae0a8a0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae0a8a0640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae04063880 0x7fae04063c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae0a8a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae0406f8c0 con 0x7fae04063880 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae04063880 0x7fae04063c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 10 monclient: get_auth_request con 0x7fae04063880 auth_method 0 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.116 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 10 monclient(hunting): handle_auth_done global_id 284184 payload 824 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.093+0000 7fae037fe640 1 -- 10.20.193.78:0/586963692 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fae0413ace0 con 0x7fae04063880 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae037fe640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae04063880 msgr2=0x7fae04063c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae037fe640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae04063880 0x7fae04063c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009fd0 tx=0x7fadf4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf40058f0 con 0x7fae04063880 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 10 monclient: dump: 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.117 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4699 (secure 0 0 0) 0x7fadf401aaa0 con 0x7fae04063880 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae04063880 msgr2=0x7fae04063c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae04063880 0x7fae04063c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009fd0 tx=0x7fadf4005a90 comp rx=0 tx=0).stop 2026-02-17T06:09:39.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae04063880 msgr2=0x7fae04063c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.095+0000 7fae00ff9640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae04063880 0x7fae04063c90 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009fd0 tx=0x7fadf4005a90 comp rx=0 tx=0).stop 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 262, 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 30233, 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:09:39.119 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:39.120 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:39.121 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:09:39.122 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.123 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.124 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.125 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:39.126 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.127 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:09:39.128 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.129 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.129 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.129 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:09:39.129 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadd403de90 msgr2=0x7fadd4040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fadd403de90 0x7fadd4040350 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fadf8003db0 tx=0x7fadf80095f0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 10 monclient: shutdown 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 msgr2=0x7fae04072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 1 --2- 10.20.193.78:0/586963692 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae0409ea90 0x7fae04072df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fadf002f390 tx=0x7fadf00047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae08e16640 1 -- 10.20.193.78:0/586963692 reap_dead start 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae08e16640 1 -- 10.20.193.78:0/586963692 reap_dead start 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 shutdown_connections 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 >> 10.20.193.78:0/586963692 conn(0x7fae040a7520 msgr2=0x7fae0413c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 shutdown_connections 2026-02-17T06:09:39.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.096+0000 7fae0a8a0640 1 -- 10.20.193.78:0/586963692 wait complete. 2026-02-17T06:09:39.131 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:09:39.131 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:09:39.131 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:09:39.200 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.197+0000 7fae507ba640 10 monclient: get_monmap_and_config 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.197+0000 7fae507ba640 10 monclient: build_initial_monmap 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.197+0000 7fae507ba640 10 monclient: monmap: 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:39.198829+0000 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:39.198829+0000 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:39.201 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.197+0000 7fae507ba640 1 Processor -- start 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 1 -- start start 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient: init 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fae4806c890 0x7fae4806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient(hunting): picked mon.noname-c con 0x7fae4806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae4809d430 0x7fae4806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient(hunting): picked mon.noname-a con 0x7fae4809d430 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fae4809bf40 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae48073600 con 0x7fae4809bf40 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fae4809ca50 con 0x7fae4809d430 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.202 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fae4806d6f0 con 0x7fae4806c890 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae507ba640 10 monclient(hunting): authenticate will time out at 2024.752408s 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4e52f640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fae4806c890 msgr2=0x7fae4806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4e52f640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fae4806c890 0x7fae4806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae4809d430 0x7fae4806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4ed30640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7fae4809bf40 legacy=0x7fae48134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fae4809d430 0x7fae4806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58232/0 (socket says 10.20.193.78:58232) 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 1 -- 10.20.193.78:0/1762343786 learned_addr learned my addr 10.20.193.78:0/1762343786 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient(hunting): get_auth_request con 0x7fae4809d430 auth_method 0 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_done global_id 284187 payload 274 2026-02-17T06:09:39.203 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 1 -- 10.20.193.78:0/1762343786 >> v2:10.20.193.115:3300/0 conn(0x7fae4806c890 msgr2=0x7fae4806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:39.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 1 --2- 10.20.193.78:0/1762343786 >> v2:10.20.193.115:3300/0 conn(0x7fae4806c890 0x7fae4806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 1 -- 10.20.193.78:0/1762343786 >> v1:10.20.193.115:6789/0 conn(0x7fae4809bf40 legacy=0x7fae48134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:09:39.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient: _finish_hunting 0 2026-02-17T06:09:39.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 1 monclient: found mon.noname-a 2026-02-17T06:09:39.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:39.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.204 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.198+0000 7fae4dd2e640 1 -- 10.20.193.78:0/1762343786 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4806dab0 con 0x7fae4809d430 2026-02-17T06:09:39.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4dd2e640 1 -- 10.20.193.78:0/1762343786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4809d430 msgr2=0x7fae4806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:39.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4dd2e640 1 --2- 10.20.193.78:0/1762343786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4809d430 0x7fae4806d1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae4806a480 tx=0x7fae3c02ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fb679e96b35cac server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.209 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 1 -- 10.20.193.78:0/1762343786 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c03c070 con 0x7fae4809d430 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: dump: 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: _finish_auth 0 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: _check_auth_tickets 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:39.210 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 1 -- 10.20.193.78:0/1762343786 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3c02fda0 con 0x7fae4809d430 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 5 monclient: authenticate success, global_id 284187 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 1 -- 10.20.193.78:0/1762343786 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c037660 con 0x7fae4809d430 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 4 set_mon_vals no callback set 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae4d52d640 10 monclient: dump: 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.211 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 10 monclient: shutdown 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 1 -- 10.20.193.78:0/1762343786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4809d430 msgr2=0x7fae4806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 1 --2- 10.20.193.78:0/1762343786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4809d430 0x7fae4806d1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae4806a480 tx=0x7fae3c02ee80 comp rx=0 tx=0).stop 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 1 -- 10.20.193.78:0/1762343786 shutdown_connections 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 1 --2- 10.20.193.78:0/1762343786 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4809d430 0x7fae4806d1b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 1 --2- 10.20.193.78:0/1762343786 >> v2:10.20.193.115:3300/0 conn(0x7fae4806c890 0x7fae4806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 1 -- 10.20.193.78:0/1762343786 >> 10.20.193.78:0/1762343786 conn(0x7fae480a6620 msgr2=0x7fae480a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 1 -- 10.20.193.78:0/1762343786 shutdown_connections 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.201+0000 7fae507ba640 1 -- 10.20.193.78:0/1762343786 wait complete. 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient: build_initial_monmap 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient: monmap: 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:39.212 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 1 Processor -- start 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 1 -- start start 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient: init 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4806c890 0x7fae481cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient(hunting): picked mon.noname-a con 0x7fae4806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae4809bf40 0x7fae481cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient(hunting): picked mon.noname-c con 0x7fae4809bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae4809d430 0x7fae481215f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient(hunting): picked mon.noname-b con 0x7fae4809d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae4809c800 con 0x7fae4806c890 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae48073410 con 0x7fae4809d430 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae48135500 con 0x7fae4809bf40 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae507ba640 10 monclient(hunting): authenticate will time out at 2024.756599s 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4806c890 0x7fae481cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4806c890 0x7fae481cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58234/0 (socket says 10.20.193.78:58234) 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 -- 10.20.193.78:0/3710280621 learned_addr learned my addr 10.20.193.78:0/3710280621 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4dd2e640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae4809bf40 0x7fae481cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient(hunting): get_auth_request con 0x7fae4806c890 auth_method 0 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4ed30640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae4809d430 msgr2=0x7fae481215f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4dd2e640 10 monclient(hunting): get_auth_request con 0x7fae4809bf40 auth_method 0 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4dd2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4dd2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4dd2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4ed30640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae4809d430 0x7fae481215f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient(hunting): handle_auth_done global_id 284190 payload 824 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae4809bf40 msgr2=0x7fae481cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae4809bf40 0x7fae481cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae4809d430 msgr2=0x7fae481215f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fae4809d430 0x7fae481215f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient: _finish_hunting 0 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 monclient: found mon.noname-a 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.202+0000 7fae4e52f640 1 -- 10.20.193.78:0/3710280621 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae48123680 con 0x7fae4806c890 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.204+0000 7fae4e52f640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4806c890 msgr2=0x7fae481cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.204+0000 7fae4e52f640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4806c890 0x7fae481cf560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b7e0 tx=0x7fae3800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380052d0 con 0x7fae4806c890 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: dump: 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.216 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: _finish_auth 0 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: _check_auth_tickets 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae38010070 con 0x7fae4806c890 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 5 monclient: authenticate success, global_id 284190 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 10 monclient: _renew_subs 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae48134e80 con 0x7fae4806c890 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380056d0 con 0x7fae4806c890 2026-02-17T06:09:39.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 10 monclient: dump: 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4cd2c640 4 set_mon_vals no callback set 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 10 monclient: _renew_subs 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae48123d10 con 0x7fae4806c890 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:39.218 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48125270 0x7fae48121b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae481358f0 con 0x7fae48125270 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fae3801d030 con 0x7fae4806c890 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48125270 0x7fae48121b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 10 monclient: get_auth_request con 0x7fae48125270 auth_method 0 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_done global_id 264392 payload 824 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae4dd2e640 1 -- 10.20.193.78:0/3710280621 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae480008d0 con 0x7fae48125270 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae1803de60 0x7fae18040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.205+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(828..828 src has 525..828) ==== 4871+0+0 (secure 0 0 0) 0x7fae38051e80 con 0x7fae4806c890 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.206+0000 7fae4ed30640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae1803de60 0x7fae18040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.206+0000 7fae4ed30640 10 monclient: get_auth_request con 0x7fae1803de60 auth_method 0 2026-02-17T06:09:39.219 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae4dd2e640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48125270 msgr2=0x7fae48121b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae4dd2e640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48125270 0x7fae48121b30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae48138100 tx=0x7fae3c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c048070 con 0x7fae48125270 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 10 monclient: dump: 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fae3c0376d0 con 0x7fae48125270 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:39.220 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48125270 msgr2=0x7fae48121b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48125270 0x7fae48121b30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae48138100 tx=0x7fae3c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48125270 msgr2=0x7fae48121b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:39.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae2effd640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48125270 0x7fae48121b30 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae48138100 tx=0x7fae3c0047c0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.221 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.208+0000 7fae4ed30640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae1803de60 0x7fae18040320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae44003db0 tx=0x7fae44007180 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae507ba640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:09:39.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae507ba640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48122070 0x7fae481da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:39.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae507ba640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:39.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fae48124c00 con 0x7fae48122070 2026-02-17T06:09:39.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48122070 0x7fae481da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:39.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 10 monclient: get_auth_request con 0x7fae48122070 auth_method 0 2026-02-17T06:09:39.222 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 10 monclient(hunting): handle_auth_done global_id 264395 payload 824 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.211+0000 7fae4dd2e640 1 -- 10.20.193.78:0/3710280621 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fae481cffe0 con 0x7fae48122070 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae4dd2e640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48122070 msgr2=0x7fae481da5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae4dd2e640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48122070 0x7fae481da5d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae3c02eec0 tx=0x7fae3c009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c03c040 con 0x7fae48122070 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 10 monclient: dump: 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:39.223 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7fae3c041440 con 0x7fae48122070 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48122070 msgr2=0x7fae481da5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48122070 0x7fae481da5d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae3c02eec0 tx=0x7fae3c009340 comp rx=0 tx=0).stop 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48122070 msgr2=0x7fae481da5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.213+0000 7fae2effd640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fae48122070 0x7fae481da5d0 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae3c02eec0 tx=0x7fae3c009340 comp rx=0 tx=0).stop 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:09:39.224 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 262, 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 128598, 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:39.225 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:09:39.226 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:09:39.227 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:39.228 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:09:39.229 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.230 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.231 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:09:39.232 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:09:39.233 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:09:39.233 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:09:39.233 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:09:39.233 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:09:39.233 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:09:39.233 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae1803de60 msgr2=0x7fae18040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fae1803de60 0x7fae18040320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae44003db0 tx=0x7fae44007180 comp rx=0 tx=0).stop 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 10 monclient: shutdown 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4806c890 msgr2=0x7fae481cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4806c890 0x7fae481cf560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b7e0 tx=0x7fae3800bcb0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae4ed30640 1 -- 10.20.193.78:0/3710280621 reap_dead start 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 shutdown_connections 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 --2- 10.20.193.78:0/3710280621 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fae4806c890 0x7fae481cf560 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 >> 10.20.193.78:0/3710280621 conn(0x7fae480a6620 msgr2=0x7fae481379c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 shutdown_connections 2026-02-17T06:09:39.234 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:39.214+0000 7fae507ba640 1 -- 10.20.193.78:0/3710280621 wait complete. 2026-02-17T06:09:39.234 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:09:39.234 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:09:39.880 INFO:tasks.rados.rados.0.trial115.stdout:3765: done (6 left) 2026-02-17T06:09:39.881 INFO:tasks.rados.rados.0.trial115.stdout:3766: write oid 34 current snap is 400 2026-02-17T06:09:39.881 INFO:tasks.rados.rados.0.trial115.stdout:3766: seq_num 1285 ranges {796995=758622,2161832=732447} 2026-02-17T06:09:39.886 INFO:tasks.rados.rados.0.trial115.stdout:3766: writing trial11545420-34 from 796995 to 1555617 tid 1 2026-02-17T06:09:39.892 INFO:tasks.rados.rados.0.trial115.stdout:3766: writing trial11545420-34 from 2161832 to 2894279 tid 2 2026-02-17T06:09:39.892 INFO:tasks.rados.rados.0.trial115.stdout:3767: read oid 13 snap -1 2026-02-17T06:09:39.892 INFO:tasks.rados.rados.0.trial115.stdout:3767: expect (ObjNum 1229 snap 380 seq_num 1229) 2026-02-17T06:09:39.892 INFO:tasks.rados.rados.0.trial115.stdout:3768: write oid 49 current snap is 400 2026-02-17T06:09:39.892 INFO:tasks.rados.rados.0.trial115.stdout:3768: seq_num 1286 ranges {413802=433871,1420314=476105,2482204=790985} 2026-02-17T06:09:39.895 INFO:tasks.rados.rados.0.trial115.stdout:3768: writing trial11545420-49 from 413802 to 847673 tid 1 2026-02-17T06:09:39.899 INFO:tasks.rados.rados.0.trial115.stdout:3768: writing trial11545420-49 from 1420314 to 1896419 tid 2 2026-02-17T06:09:39.905 INFO:tasks.rados.rados.0.trial115.stdout:3768: writing trial11545420-49 from 2482204 to 3273189 tid 3 2026-02-17T06:09:39.905 INFO:tasks.rados.rados.0.trial115.stdout:3769: delete oid 9 current snap is 400 2026-02-17T06:09:39.905 INFO:tasks.rados.rados.0.trial115.stdout:3758: expect (ObjNum 1279 snap 398 seq_num 1279) 2026-02-17T06:09:40.023 INFO:tasks.rados.rados.0.trial115.stdout:3762: expect (ObjNum 1185 snap 364 seq_num 1185) 2026-02-17T06:09:40.115 INFO:tasks.rados.rados.0.trial115.stdout:3761: finishing copy_from racing read to trial11545420-30 2026-02-17T06:09:40.115 INFO:tasks.rados.rados.0.trial115.stdout:3758: done (9 left) 2026-02-17T06:09:40.115 INFO:tasks.rados.rados.0.trial115.stdout:3759: done (8 left) 2026-02-17T06:09:40.115 INFO:tasks.rados.rados.0.trial115.stdout:3760: done (7 left) 2026-02-17T06:09:40.115 INFO:tasks.rados.rados.0.trial115.stdout:3762: done (6 left) 2026-02-17T06:09:40.116 INFO:tasks.rados.rados.0.trial115.stdout:3769: done (5 left) 2026-02-17T06:09:40.116 INFO:tasks.rados.rados.0.trial115.stdout:3770: write (excl) oid 24 current snap is 400 2026-02-17T06:09:40.116 INFO:tasks.rados.rados.0.trial115.stdout:3761: finishing copy_from to trial11545420-30 2026-02-17T06:09:40.116 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 30 v 1504 (ObjNum 1274 snap 398 seq_num 1274) dirty exists 2026-02-17T06:09:40.116 INFO:tasks.rados.rados.0.trial115.stdout:3764: finishing write tid 1 to trial11545420-39 2026-02-17T06:09:40.116 INFO:tasks.rados.rados.0.trial115.stdout:3764: finishing write tid 2 to trial11545420-39 2026-02-17T06:09:40.116 INFO:tasks.rados.rados.0.trial115.stdout:3770: seq_num 1288 ranges {447416=584369,1537278=479774,2537050=529882,3524784=122632} 2026-02-17T06:09:40.118 INFO:tasks.rados.rados.0.trial115.stdout:3770: writing trial11545420-24 from 447416 to 1031785 tid 1 2026-02-17T06:09:40.122 INFO:tasks.rados.rados.0.trial115.stdout:3770: writing trial11545420-24 from 1537278 to 2017052 tid 2 2026-02-17T06:09:40.126 INFO:tasks.rados.rados.0.trial115.stdout:3770: writing trial11545420-24 from 2537050 to 3066932 tid 3 2026-02-17T06:09:40.128 INFO:tasks.rados.rados.0.trial115.stdout:3770: writing trial11545420-24 from 3524784 to 3647416 tid 4 2026-02-17T06:09:40.128 INFO:tasks.rados.rados.0.trial115.stdout:3761: done (5 left) 2026-02-17T06:09:40.129 INFO:tasks.rados.rados.0.trial115.stdout:3771: snap_remove snap 398 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3771: done (5 left) 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3772: copy_from oid 8 from oid 32 current snap is 400 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3764: finishing write tid 3 to trial11545420-39 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3764: finishing write tid 4 to trial11545420-39 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3764: finishing write tid 5 to trial11545420-39 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3764: oid 39 updating version 0 to 1952 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3764: oid 39 updating version 1952 to 1953 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3764: oid 39 version 1953 is already newer than 1951 2026-02-17T06:09:40.884 INFO:tasks.rados.rados.0.trial115.stdout:3764: oid 39 updating version 1953 to 1954 2026-02-17T06:09:40.885 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1954 (ObjNum 1284 snap 400 seq_num 1284) dirty exists 2026-02-17T06:09:40.885 INFO:tasks.rados.rados.0.trial115.stdout:3764: left oid 39 (ObjNum 1284 snap 400 seq_num 1284) 2026-02-17T06:09:40.885 INFO:tasks.rados.rados.0.trial115.stdout:3766: finishing write tid 1 to trial11545420-34 2026-02-17T06:09:40.885 INFO:tasks.rados.rados.0.trial115.stdout:3764: done (5 left) 2026-02-17T06:09:40.885 INFO:tasks.rados.rados.0.trial115.stdout:3773: write oid 23 current snap is 400 2026-02-17T06:09:40.885 INFO:tasks.rados.rados.0.trial115.stdout:3773: seq_num 1289 ranges {464223=659618,1879407=739785,3248120=416103} 2026-02-17T06:09:40.888 INFO:tasks.rados.rados.0.trial115.stdout:3773: writing trial11545420-23 from 464223 to 1123841 tid 1 2026-02-17T06:09:40.894 INFO:tasks.rados.rados.0.trial115.stdout:3773: writing trial11545420-23 from 1879407 to 2619192 tid 2 2026-02-17T06:09:40.899 INFO:tasks.rados.rados.0.trial115.stdout:3773: writing trial11545420-23 from 3248120 to 3664223 tid 3 2026-02-17T06:09:40.899 INFO:tasks.rados.rados.0.trial115.stdout:3774: snap_create 2026-02-17T06:09:40.899 INFO:tasks.rados.rados.0.trial115.stdout:3766: finishing write tid 2 to trial11545420-34 2026-02-17T06:09:40.899 INFO:tasks.rados.rados.0.trial115.stdout:3766: finishing write tid 3 to trial11545420-34 2026-02-17T06:09:40.899 INFO:tasks.rados.rados.0.trial115.stdout:3766: finishing write tid 4 to trial11545420-34 2026-02-17T06:09:40.900 INFO:tasks.rados.rados.0.trial115.stdout:3766: oid 34 updating version 0 to 1503 2026-02-17T06:09:40.900 INFO:tasks.rados.rados.0.trial115.stdout:3766: oid 34 updating version 1503 to 1504 2026-02-17T06:09:40.900 INFO:tasks.rados.rados.0.trial115.stdout:3766: oid 34 version 1504 is already newer than 1502 2026-02-17T06:09:40.900 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1504 (ObjNum 1285 snap 400 seq_num 1285) dirty exists 2026-02-17T06:09:40.900 INFO:tasks.rados.rados.0.trial115.stdout:3766: left oid 34 (ObjNum 1285 snap 400 seq_num 1285) 2026-02-17T06:09:40.900 INFO:tasks.rados.rados.0.trial115.stdout:3767: expect (ObjNum 1229 snap 380 seq_num 1229) 2026-02-17T06:09:41.025 INFO:tasks.rados.rados.0.trial115.stdout:3768: finishing write tid 1 to trial11545420-49 2026-02-17T06:09:41.025 INFO:tasks.rados.rados.0.trial115.stdout:3768: finishing write tid 2 to trial11545420-49 2026-02-17T06:09:41.025 INFO:tasks.rados.rados.0.trial115.stdout:3768: finishing write tid 3 to trial11545420-49 2026-02-17T06:09:41.025 INFO:tasks.rados.rados.0.trial115.stdout:3768: finishing write tid 4 to trial11545420-49 2026-02-17T06:09:41.025 INFO:tasks.rados.rados.0.trial115.stdout:3768: finishing write tid 5 to trial11545420-49 2026-02-17T06:09:41.025 INFO:tasks.rados.rados.0.trial115.stdout:3768: oid 49 updating version 0 to 1506 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3768: oid 49 updating version 1506 to 1508 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3768: oid 49 version 1508 is already newer than 1507 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3768: oid 49 version 1508 is already newer than 1505 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1508 (ObjNum 1286 snap 400 seq_num 1286) dirty exists 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3768: left oid 49 (ObjNum 1286 snap 400 seq_num 1286) 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: finishing write tid 1 to trial11545420-24 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: finishing write tid 2 to trial11545420-24 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: finishing write tid 3 to trial11545420-24 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: finishing write tid 4 to trial11545420-24 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: finishing write tid 5 to trial11545420-24 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: finishing write tid 6 to trial11545420-24 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: oid 24 updating version 0 to 1966 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: oid 24 version 1966 is already newer than 1964 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: oid 24 version 1966 is already newer than 1963 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: oid 24 version 1966 is already newer than 1962 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: oid 24 version 1966 is already newer than 1965 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1966 (ObjNum 1288 snap 400 seq_num 1288) dirty exists 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3770: left oid 24 (ObjNum 1288 snap 400 seq_num 1288) 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3773: finishing write tid 1 to trial11545420-23 2026-02-17T06:09:41.026 INFO:tasks.rados.rados.0.trial115.stdout:3772: finishing copy_from racing read to trial11545420-8 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: finishing write tid 2 to trial11545420-23 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3772: finishing copy_from to trial11545420-8 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1482 (ObjNum 1272 snap 398 seq_num 1272) dirty exists 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: finishing write tid 3 to trial11545420-23 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: finishing write tid 4 to trial11545420-23 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: finishing write tid 5 to trial11545420-23 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: oid 23 updating version 0 to 1819 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: oid 23 updating version 1819 to 1820 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: oid 23 version 1820 is already newer than 1818 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: oid 23 version 1820 is already newer than 1817 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1820 (ObjNum 1289 snap 400 seq_num 1289) dirty exists 2026-02-17T06:09:41.027 INFO:tasks.rados.rados.0.trial115.stdout:3773: left oid 23 (ObjNum 1289 snap 400 seq_num 1289) 2026-02-17T06:09:41.885 INFO:tasks.rados.rados.0.trial115.stdout:3766: done (6 left) 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3767: done (5 left) 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3768: done (4 left) 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3770: done (3 left) 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3772: done (2 left) 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3773: done (1 left) 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3774: done (0 left) 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3775: read oid 8 snap -1 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3775: expect (ObjNum 1272 snap 398 seq_num 1272) 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3776: write (excl) oid 15 current snap is 401 2026-02-17T06:09:41.886 INFO:tasks.rados.rados.0.trial115.stdout:3776: seq_num 1290 ranges {481030=734867,1937889=483443,2875543=552426} 2026-02-17T06:09:41.891 INFO:tasks.rados.rados.0.trial115.stdout:3776: writing trial11545420-15 from 481030 to 1215897 tid 1 2026-02-17T06:09:41.895 INFO:tasks.rados.rados.0.trial115.stdout:3776: writing trial11545420-15 from 1937889 to 2421332 tid 2 2026-02-17T06:09:41.900 INFO:tasks.rados.rados.0.trial115.stdout:3776: writing trial11545420-15 from 2875543 to 3427969 tid 3 2026-02-17T06:09:41.900 INFO:tasks.rados.rados.0.trial115.stdout:3777: delete oid 20 current snap is 401 2026-02-17T06:09:41.900 INFO:tasks.rados.rados.0.trial115.stdout:3775: expect (ObjNum 1272 snap 398 seq_num 1272) 2026-02-17T06:09:42.065 INFO:tasks.rados.rados.0.trial115.stdout:3776: finishing write tid 1 to trial11545420-15 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: finishing write tid 2 to trial11545420-15 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: finishing write tid 3 to trial11545420-15 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: finishing write tid 4 to trial11545420-15 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: finishing write tid 5 to trial11545420-15 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: oid 15 updating version 0 to 2008 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: oid 15 version 2008 is already newer than 2005 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: oid 15 version 2008 is already newer than 2007 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: oid 15 version 2008 is already newer than 2006 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 2008 (ObjNum 1290 snap 401 seq_num 1290) dirty exists 2026-02-17T06:09:42.066 INFO:tasks.rados.rados.0.trial115.stdout:3776: left oid 15 (ObjNum 1290 snap 401 seq_num 1290) 2026-02-17T06:09:42.067 INFO:tasks.rados.rados.0.trial115.stdout:3775: done (2 left) 2026-02-17T06:09:42.067 INFO:tasks.rados.rados.0.trial115.stdout:3776: done (1 left) 2026-02-17T06:09:42.067 INFO:tasks.rados.rados.0.trial115.stdout:3777: done (0 left) 2026-02-17T06:09:42.067 INFO:tasks.rados.rados.0.trial115.stdout:3778: read oid 15 snap 389 2026-02-17T06:09:42.067 INFO:tasks.rados.rados.0.trial115.stdout:3778: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:09:42.067 INFO:tasks.rados.rados.0.trial115.stdout:3779: snap_remove snap 399 2026-02-17T06:09:42.886 INFO:tasks.rados.rados.0.trial115.stdout:3779: done (1 left) 2026-02-17T06:09:42.886 INFO:tasks.rados.rados.0.trial115.stdout:3780: write oid 47 current snap is 401 2026-02-17T06:09:42.886 INFO:tasks.rados.rados.0.trial115.stdout:3780: seq_num 1292 ranges {514644=485365,1654853=487112,2930389=574970} 2026-02-17T06:09:42.891 INFO:tasks.rados.rados.0.trial115.stdout:3780: writing trial11545420-47 from 514644 to 1000009 tid 1 2026-02-17T06:09:42.895 INFO:tasks.rados.rados.0.trial115.stdout:3780: writing trial11545420-47 from 1654853 to 2141965 tid 2 2026-02-17T06:09:42.900 INFO:tasks.rados.rados.0.trial115.stdout:3780: writing trial11545420-47 from 2930389 to 3505359 tid 3 2026-02-17T06:09:42.900 INFO:tasks.rados.rados.0.trial115.stdout:3781: delete oid 38 current snap is 401 2026-02-17T06:09:42.900 INFO:tasks.rados.rados.0.trial115.stdout:3778: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: finishing write tid 1 to trial11545420-47 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: finishing write tid 2 to trial11545420-47 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: finishing write tid 3 to trial11545420-47 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3778: done (2 left) 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3781: done (1 left) 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3782: read oid 32 snap -1 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3782: expect (ObjNum 1272 snap 398 seq_num 1272) 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3783: delete oid 29 current snap is 401 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: finishing write tid 4 to trial11545420-47 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: finishing write tid 5 to trial11545420-47 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: oid 47 updating version 0 to 1667 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: oid 47 version 1667 is already newer than 1666 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: oid 47 version 1667 is already newer than 1665 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: oid 47 version 1667 is already newer than 1664 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1667 (ObjNum 1292 snap 401 seq_num 1292) dirty exists 2026-02-17T06:09:43.101 INFO:tasks.rados.rados.0.trial115.stdout:3780: left oid 47 (ObjNum 1292 snap 401 seq_num 1292) 2026-02-17T06:09:43.103 INFO:tasks.rados.rados.0.trial115.stdout:3780: done (2 left) 2026-02-17T06:09:43.103 INFO:tasks.rados.rados.0.trial115.stdout:3783: done (1 left) 2026-02-17T06:09:43.103 INFO:tasks.rados.rados.0.trial115.stdout:3784: snap_remove snap 389 2026-02-17T06:09:43.871 INFO:tasks.rados.rados.0.trial115.stdout:3784: done (1 left) 2026-02-17T06:09:43.871 INFO:tasks.rados.rados.0.trial115.stdout:3785: snap_remove snap 400 2026-02-17T06:09:44.874 INFO:tasks.rados.rados.0.trial115.stdout:3785: done (1 left) 2026-02-17T06:09:44.874 INFO:tasks.rados.rados.0.trial115.stdout:3786: rollback oid 31 current snap is 401 2026-02-17T06:09:44.874 INFO:tasks.rados.rados.0.trial115.stdout:3786: done (1 left) 2026-02-17T06:09:44.874 INFO:tasks.rados.rados.0.trial115.stdout:3787: delete oid 45 current snap is 401 2026-02-17T06:09:44.875 INFO:tasks.rados.rados.0.trial115.stdout:3782: expect (ObjNum 1272 snap 398 seq_num 1272) 2026-02-17T06:09:45.037 INFO:tasks.rados.rados.0.trial115.stdout:3782: done (1 left) 2026-02-17T06:09:45.038 INFO:tasks.rados.rados.0.trial115.stdout:3787: done (0 left) 2026-02-17T06:09:45.038 INFO:tasks.rados.rados.0.trial115.stdout:3788: delete oid 47 current snap is 401 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3788: done (0 left) 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3789: read oid 12 snap -1 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3789: expect deleted 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3790: rollback oid 3 current snap is 401 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3790: done (1 left) 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3791: rollback oid 10 current snap is 401 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3791: done (1 left) 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3792: copy_from oid 37 from oid 30 current snap is 401 2026-02-17T06:09:45.039 INFO:tasks.rados.rados.0.trial115.stdout:3793: delete oid 41 current snap is 401 2026-02-17T06:09:45.040 INFO:tasks.rados.rados.0.trial115.stdout:3789: done (2 left) 2026-02-17T06:09:45.041 INFO:tasks.rados.rados.0.trial115.stdout:3793: done (1 left) 2026-02-17T06:09:45.041 INFO:tasks.rados.rados.0.trial115.stdout:3794: write oid 48 current snap is 401 2026-02-17T06:09:45.041 INFO:tasks.rados.rados.0.trial115.stdout:3794: seq_num 1298 ranges {615486=653212,1889392=498119,2978574=475308} 2026-02-17T06:09:45.045 INFO:tasks.rados.rados.0.trial115.stdout:3794: writing trial11545420-48 from 615486 to 1268698 tid 1 2026-02-17T06:09:45.050 INFO:tasks.rados.rados.0.trial115.stdout:3794: writing trial11545420-48 from 1889392 to 2387511 tid 2 2026-02-17T06:09:45.054 INFO:tasks.rados.rados.0.trial115.stdout:3794: writing trial11545420-48 from 2978574 to 3453882 tid 3 2026-02-17T06:09:45.054 INFO:tasks.rados.rados.0.trial115.stdout:3795: copy_from oid 26 from oid 22 current snap is 401 2026-02-17T06:09:45.054 INFO:tasks.rados.rados.0.trial115.stdout:3792: finishing copy_from racing read to trial11545420-37 2026-02-17T06:09:45.054 INFO:tasks.rados.rados.0.trial115.stdout:3794: finishing write tid 1 to trial11545420-48 2026-02-17T06:09:45.054 INFO:tasks.rados.rados.0.trial115.stdout:3792: finishing copy_from to trial11545420-37 2026-02-17T06:09:45.054 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1504 (ObjNum 1274 snap 398 seq_num 1274) dirty exists 2026-02-17T06:09:45.054 INFO:tasks.rados.rados.0.trial115.stdout:3792: done (2 left) 2026-02-17T06:09:45.055 INFO:tasks.rados.rados.0.trial115.stdout:3796: write oid 21 current snap is 401 2026-02-17T06:09:45.055 INFO:tasks.rados.rados.0.trial115.stdout:3796: seq_num 1299 ranges {632293=728461,1947874=641777,3005997=686580} 2026-02-17T06:09:45.059 INFO:tasks.rados.rados.0.trial115.stdout:3796: writing trial11545420-21 from 632293 to 1360754 tid 1 2026-02-17T06:09:45.064 INFO:tasks.rados.rados.0.trial115.stdout:3796: writing trial11545420-21 from 1947874 to 2589651 tid 2 2026-02-17T06:09:45.068 INFO:tasks.rados.rados.0.trial115.stdout:3796: writing trial11545420-21 from 3005997 to 3692577 tid 3 2026-02-17T06:09:45.068 INFO:tasks.rados.rados.0.trial115.stdout:3797: write (excl) oid 12 current snap is 401 2026-02-17T06:09:45.068 INFO:tasks.rados.rados.0.trial115.stdout:3797: seq_num 1300 ranges {649100=403710,1606356=501788,2633420=497852,3636005=213095} 2026-02-17T06:09:45.072 INFO:tasks.rados.rados.0.trial115.stdout:3797: writing trial11545420-12 from 649100 to 1052810 tid 1 2026-02-17T06:09:45.076 INFO:tasks.rados.rados.0.trial115.stdout:3797: writing trial11545420-12 from 1606356 to 2108144 tid 2 2026-02-17T06:09:45.080 INFO:tasks.rados.rados.0.trial115.stdout:3797: writing trial11545420-12 from 2633420 to 3131272 tid 3 2026-02-17T06:09:45.084 INFO:tasks.rados.rados.0.trial115.stdout:3797: writing trial11545420-12 from 3636005 to 3849100 tid 4 2026-02-17T06:09:45.084 INFO:tasks.rados.rados.0.trial115.stdout:3798: read oid 33 snap -1 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3798: expect deleted 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3799: delete oid 9 current snap is 401 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3794: finishing write tid 2 to trial11545420-48 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3794: finishing write tid 3 to trial11545420-48 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3794: finishing write tid 4 to trial11545420-48 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3794: finishing write tid 5 to trial11545420-48 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3794: oid 48 updating version 0 to 2011 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3794: oid 48 updating version 2011 to 2012 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3794: oid 48 updating version 2012 to 2013 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:3794: oid 48 version 2013 is already newer than 2010 2026-02-17T06:09:45.085 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 48 v 2013 (ObjNum 1298 snap 401 seq_num 1298) dirty exists 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3794: left oid 48 (ObjNum 1298 snap 401 seq_num 1298) 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: finishing write tid 1 to trial11545420-21 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3795: finishing copy_from racing read to trial11545420-26 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3795: finishing copy_from to trial11545420-26 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1502 (ObjNum 1264 snap 396 seq_num 1264) dirty exists 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: finishing write tid 2 to trial11545420-21 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: finishing write tid 3 to trial11545420-21 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: finishing write tid 4 to trial11545420-21 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: finishing write tid 5 to trial11545420-21 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: oid 21 updating version 0 to 1819 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: oid 21 updating version 1819 to 1821 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: oid 21 version 1821 is already newer than 1820 2026-02-17T06:09:45.086 INFO:tasks.rados.rados.0.trial115.stdout:3796: oid 21 updating version 1821 to 1822 2026-02-17T06:09:45.087 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 1822 (ObjNum 1299 snap 401 seq_num 1299) dirty exists 2026-02-17T06:09:45.087 INFO:tasks.rados.rados.0.trial115.stdout:3796: left oid 21 (ObjNum 1299 snap 401 seq_num 1299) 2026-02-17T06:09:45.087 INFO:tasks.rados.rados.0.trial115.stdout:3797: finishing write tid 1 to trial11545420-12 2026-02-17T06:09:45.087 INFO:tasks.rados.rados.0.trial115.stdout:3797: finishing write tid 2 to trial11545420-12 2026-02-17T06:09:45.087 INFO:tasks.rados.rados.0.trial115.stdout:3797: finishing write tid 3 to trial11545420-12 2026-02-17T06:09:45.089 INFO:tasks.rados.rados.0.trial115.stdout:3797: finishing write tid 4 to trial11545420-12 2026-02-17T06:09:45.089 INFO:tasks.rados.rados.0.trial115.stdout:3794: done (5 left) 2026-02-17T06:09:45.089 INFO:tasks.rados.rados.0.trial115.stdout:3795: done (4 left) 2026-02-17T06:09:45.089 INFO:tasks.rados.rados.0.trial115.stdout:3796: done (3 left) 2026-02-17T06:09:45.089 INFO:tasks.rados.rados.0.trial115.stdout:3798: done (2 left) 2026-02-17T06:09:45.089 INFO:tasks.rados.rados.0.trial115.stdout:3799: done (1 left) 2026-02-17T06:09:45.089 INFO:tasks.rados.rados.0.trial115.stdout:3800: read oid 26 snap -1 2026-02-17T06:09:45.089 INFO:tasks.rados.rados.0.trial115.stdout:3800: expect (ObjNum 1264 snap 396 seq_num 1264) 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3801: read oid 6 snap -1 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3801: expect deleted 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3802: read oid 48 snap -1 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3802: expect (ObjNum 1298 snap 401 seq_num 1298) 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3803: copy_from oid 45 from oid 5 current snap is 401 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3797: finishing write tid 5 to trial11545420-12 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3797: finishing write tid 6 to trial11545420-12 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3797: oid 12 updating version 0 to 1980 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3797: oid 12 version 1980 is already newer than 1978 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3797: oid 12 updating version 1980 to 1981 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3797: oid 12 updating version 1981 to 1982 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:3797: oid 12 version 1982 is already newer than 1979 2026-02-17T06:09:45.090 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 12 v 1982 (ObjNum 1300 snap 401 seq_num 1300) dirty exists 2026-02-17T06:09:45.091 INFO:tasks.rados.rados.0.trial115.stdout:3797: left oid 12 (ObjNum 1300 snap 401 seq_num 1300) 2026-02-17T06:09:45.091 INFO:tasks.rados.rados.0.trial115.stdout:3797: done (4 left) 2026-02-17T06:09:45.091 INFO:tasks.rados.rados.0.trial115.stdout:3804: read oid 3 snap -1 2026-02-17T06:09:45.091 INFO:tasks.rados.rados.0.trial115.stdout:3804: expect (ObjNum 1215 snap 376 seq_num 1215) 2026-02-17T06:09:45.091 INFO:tasks.rados.rados.0.trial115.stdout:3805: copy_from oid 16 from oid 46 current snap is 401 2026-02-17T06:09:45.091 INFO:tasks.rados.rados.0.trial115.stdout:3806: copy_from oid 38 from oid 22 current snap is 401 2026-02-17T06:09:45.091 INFO:tasks.rados.rados.0.trial115.stdout:3807: snap_create 2026-02-17T06:09:45.091 INFO:tasks.rados.rados.0.trial115.stdout:3804: expect (ObjNum 1215 snap 376 seq_num 1215) 2026-02-17T06:09:45.225 INFO:tasks.rados.rados.0.trial115.stdout:3800: expect (ObjNum 1264 snap 396 seq_num 1264) 2026-02-17T06:09:45.383 INFO:tasks.rados.rados.0.trial115.stdout:3802: expect (ObjNum 1298 snap 401 seq_num 1298) 2026-02-17T06:09:45.474 INFO:tasks.rados.rados.0.trial115.stdout:3803: finishing copy_from racing read to trial11545420-45 2026-02-17T06:09:45.474 INFO:tasks.rados.rados.0.trial115.stdout:3803: finishing copy_from to trial11545420-45 2026-02-17T06:09:45.474 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 1656 (ObjNum 1279 snap 398 seq_num 1279) dirty exists 2026-02-17T06:09:45.474 INFO:tasks.rados.rados.0.trial115.stdout:3805: finishing copy_from racing read to trial11545420-16 2026-02-17T06:09:45.475 INFO:tasks.rados.rados.0.trial115.stdout:3806: finishing copy_from to trial11545420-38 2026-02-17T06:09:45.475 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 1502 (ObjNum 1264 snap 396 seq_num 1264) dirty exists 2026-02-17T06:09:45.475 INFO:tasks.rados.rados.0.trial115.stdout:3806: finishing copy_from racing read to trial11545420-38 2026-02-17T06:09:45.475 INFO:tasks.rados.rados.0.trial115.stdout:3805: finishing copy_from to trial11545420-16 2026-02-17T06:09:45.475 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 1994 (ObjNum 1280 snap 398 seq_num 1280) dirty exists 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3800: done (7 left) 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3801: done (6 left) 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3802: done (5 left) 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3803: done (4 left) 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3804: done (3 left) 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3805: done (2 left) 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3806: done (1 left) 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3807: done (0 left) 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3808: rollback oid 24 current snap is 402 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 24 to 401 2026-02-17T06:09:45.874 INFO:tasks.rados.rados.0.trial115.stdout:3809: write (excl) oid 2 current snap is 402 2026-02-17T06:09:45.875 INFO:tasks.rados.rados.0.trial115.stdout:3809: seq_num 1302 ranges {682714=670561,2123320=505457,3088266=636749} 2026-02-17T06:09:45.879 INFO:tasks.rados.rados.0.trial115.stdout:3809: writing trial11545420-2 from 682714 to 1353275 tid 1 2026-02-17T06:09:45.885 INFO:tasks.rados.rados.0.trial115.stdout:3809: writing trial11545420-2 from 2123320 to 2628777 tid 2 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3809: writing trial11545420-2 from 3088266 to 3725015 tid 3 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3810: rollback oid 27 current snap is 402 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 27 to 401 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3808: finishing rollback tid 0 to trial11545420-24 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3808: finishing rollback tid 1 to trial11545420-24 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3808: finishing rollback tid 2 to trial11545420-24 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 1987 (ObjNum 1288 snap 400 seq_num 1288) dirty exists 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3809: finishing write tid 1 to trial11545420-2 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3809: finishing write tid 2 to trial11545420-2 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3808: done (2 left) 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3811: write oid 26 current snap is 402 2026-02-17T06:09:45.889 INFO:tasks.rados.rados.0.trial115.stdout:3811: seq_num 1303 ranges {699521=745810,2181802=649115,3515689=383832} 2026-02-17T06:09:45.894 INFO:tasks.rados.rados.0.trial115.stdout:3811: writing trial11545420-26 from 699521 to 1445331 tid 1 2026-02-17T06:09:45.899 INFO:tasks.rados.rados.0.trial115.stdout:3811: writing trial11545420-26 from 2181802 to 2830917 tid 2 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3811: writing trial11545420-26 from 3515689 to 3899521 tid 3 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3812: rollback oid 7 current snap is 402 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 401 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: finishing write tid 3 to trial11545420-2 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: finishing write tid 4 to trial11545420-2 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3813: copy_from oid 9 from oid 49 current snap is 402 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: finishing write tid 5 to trial11545420-2 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: oid 2 updating version 0 to 1678 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: oid 2 version 1678 is already newer than 1676 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: oid 2 version 1678 is already newer than 1677 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: oid 2 updating version 1678 to 1679 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1679 (ObjNum 1302 snap 402 seq_num 1302) dirty exists 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: left oid 2 (ObjNum 1302 snap 402 seq_num 1302) 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3810: finishing rollback tid 0 to trial11545420-27 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3810: finishing rollback tid 1 to trial11545420-27 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3810: finishing rollback tid 2 to trial11545420-27 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 27 v 1541 (ObjNum 1238 snap 383 seq_num 1238) dirty exists 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3811: finishing write tid 1 to trial11545420-26 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3809: done (4 left) 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3810: done (3 left) 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3814: write oid 38 current snap is 402 2026-02-17T06:09:45.904 INFO:tasks.rados.rados.0.trial115.stdout:3814: seq_num 1304 ranges {716328=421059,1840284=509126,3143112=659293} 2026-02-17T06:09:45.908 INFO:tasks.rados.rados.0.trial115.stdout:3814: writing trial11545420-38 from 716328 to 1137387 tid 1 2026-02-17T06:09:45.912 INFO:tasks.rados.rados.0.trial115.stdout:3814: writing trial11545420-38 from 1840284 to 2349410 tid 2 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3814: writing trial11545420-38 from 3143112 to 3802405 tid 3 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3815: snap_create 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: finishing write tid 2 to trial11545420-26 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: finishing write tid 3 to trial11545420-26 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: finishing write tid 4 to trial11545420-26 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: finishing write tid 5 to trial11545420-26 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: oid 26 updating version 0 to 1504 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: oid 26 updating version 1504 to 1505 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: oid 26 version 1505 is already newer than 1503 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: oid 26 updating version 1505 to 1506 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1506 (ObjNum 1303 snap 402 seq_num 1303) dirty exists 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3811: left oid 26 (ObjNum 1303 snap 402 seq_num 1303) 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3812: finishing rollback tid 1 to trial11545420-7 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 0 (ObjNum 33 snap 0 seq_num 1334285504) dirty dne 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3813: finishing copy_from racing read to trial11545420-9 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3814: finishing write tid 1 to trial11545420-38 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3813: finishing copy_from to trial11545420-9 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 1508 (ObjNum 1286 snap 400 seq_num 1286) dirty exists 2026-02-17T06:09:45.918 INFO:tasks.rados.rados.0.trial115.stdout:3814: finishing write tid 2 to trial11545420-38 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:3814: finishing write tid 3 to trial11545420-38 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:3814: finishing write tid 4 to trial11545420-38 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:3814: finishing write tid 5 to trial11545420-38 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:3814: oid 38 updating version 0 to 2020 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:3814: oid 38 version 2020 is already newer than 2018 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:3814: oid 38 version 2020 is already newer than 2019 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:3814: oid 38 version 2020 is already newer than 2017 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 2020 (ObjNum 1304 snap 402 seq_num 1304) dirty exists 2026-02-17T06:09:45.920 INFO:tasks.rados.rados.0.trial115.stdout:3814: left oid 38 (ObjNum 1304 snap 402 seq_num 1304) 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3811: done (4 left) 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3812: done (3 left) 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3813: done (2 left) 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3814: done (1 left) 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3815: done (0 left) 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3816: read oid 29 snap 402 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3816: expect deleted 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3817: read oid 33 snap -1 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3817: expect deleted 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3818: write oid 38 current snap is 403 2026-02-17T06:09:46.877 INFO:tasks.rados.rados.0.trial115.stdout:3818: seq_num 1305 ranges {733135=496308,1898766=652784,3170535=470565} 2026-02-17T06:09:46.881 INFO:tasks.rados.rados.0.trial115.stdout:3818: writing trial11545420-38 from 733135 to 1229443 tid 1 2026-02-17T06:09:46.886 INFO:tasks.rados.rados.0.trial115.stdout:3818: writing trial11545420-38 from 1898766 to 2551550 tid 2 2026-02-17T06:09:46.891 INFO:tasks.rados.rados.0.trial115.stdout:3818: writing trial11545420-38 from 3170535 to 3641100 tid 3 2026-02-17T06:09:46.891 INFO:tasks.rados.rados.0.trial115.stdout:3819: rollback oid 34 current snap is 403 2026-02-17T06:09:46.891 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 402 2026-02-17T06:09:46.891 INFO:tasks.rados.rados.0.trial115.stdout:3818: finishing write tid 1 to trial11545420-38 2026-02-17T06:09:46.891 INFO:tasks.rados.rados.0.trial115.stdout:3818: finishing write tid 2 to trial11545420-38 2026-02-17T06:09:46.891 INFO:tasks.rados.rados.0.trial115.stdout:3816: done (3 left) 2026-02-17T06:09:46.891 INFO:tasks.rados.rados.0.trial115.stdout:3817: done (2 left) 2026-02-17T06:09:46.892 INFO:tasks.rados.rados.0.trial115.stdout:3820: snap_remove snap 401 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3820: done (2 left) 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3821: snap_create 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3818: finishing write tid 3 to trial11545420-38 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3818: finishing write tid 4 to trial11545420-38 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3818: finishing write tid 5 to trial11545420-38 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3818: oid 38 updating version 0 to 2024 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3818: oid 38 version 2024 is already newer than 2022 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3818: oid 38 version 2024 is already newer than 2023 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3818: oid 38 updating version 2024 to 2025 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 38 v 2025 (ObjNum 1305 snap 403 seq_num 1305) dirty exists 2026-02-17T06:09:47.879 INFO:tasks.rados.rados.0.trial115.stdout:3818: left oid 38 (ObjNum 1305 snap 403 seq_num 1305) 2026-02-17T06:09:47.880 INFO:tasks.rados.rados.0.trial115.stdout:3819: finishing rollback tid 0 to trial11545420-34 2026-02-17T06:09:47.880 INFO:tasks.rados.rados.0.trial115.stdout:3819: finishing rollback tid 1 to trial11545420-34 2026-02-17T06:09:47.880 INFO:tasks.rados.rados.0.trial115.stdout:3819: finishing rollback tid 2 to trial11545420-34 2026-02-17T06:09:47.880 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1507 (ObjNum 1285 snap 400 seq_num 1285) dirty exists 2026-02-17T06:09:48.881 INFO:tasks.rados.rados.0.trial115.stdout:3818: done (2 left) 2026-02-17T06:09:48.881 INFO:tasks.rados.rados.0.trial115.stdout:3819: done (1 left) 2026-02-17T06:09:48.881 INFO:tasks.rados.rados.0.trial115.stdout:3821: done (0 left) 2026-02-17T06:09:48.881 INFO:tasks.rados.rados.0.trial115.stdout:3822: write oid 14 current snap is 404 2026-02-17T06:09:48.881 INFO:tasks.rados.rados.0.trial115.stdout:3822: seq_num 1306 ranges {749942=571557,1840895=796442,3365252=584690} 2026-02-17T06:09:48.886 INFO:tasks.rados.rados.0.trial115.stdout:3822: writing trial11545420-14 from 749942 to 1321499 tid 1 2026-02-17T06:09:48.890 INFO:tasks.rados.rados.0.trial115.stdout:3822: writing trial11545420-14 from 1840895 to 2637337 tid 2 2026-02-17T06:09:48.896 INFO:tasks.rados.rados.0.trial115.stdout:3822: writing trial11545420-14 from 3365252 to 3949942 tid 3 2026-02-17T06:09:48.896 INFO:tasks.rados.rados.0.trial115.stdout:3823: delete oid 28 current snap is 404 2026-02-17T06:09:48.896 INFO:tasks.rados.rados.0.trial115.stdout:3822: finishing write tid 1 to trial11545420-14 2026-02-17T06:09:48.896 INFO:tasks.rados.rados.0.trial115.stdout:3822: finishing write tid 2 to trial11545420-14 2026-02-17T06:09:48.898 INFO:tasks.rados.rados.0.trial115.stdout:3822: finishing write tid 3 to trial11545420-14 2026-02-17T06:09:48.898 INFO:tasks.rados.rados.0.trial115.stdout:3822: finishing write tid 4 to trial11545420-14 2026-02-17T06:09:48.898 INFO:tasks.rados.rados.0.trial115.stdout:3822: finishing write tid 5 to trial11545420-14 2026-02-17T06:09:48.898 INFO:tasks.rados.rados.0.trial115.stdout:3822: oid 14 updating version 0 to 1700 2026-02-17T06:09:48.898 INFO:tasks.rados.rados.0.trial115.stdout:3822: oid 14 version 1700 is already newer than 1699 2026-02-17T06:09:48.898 INFO:tasks.rados.rados.0.trial115.stdout:3822: oid 14 updating version 1700 to 1702 2026-02-17T06:09:48.898 INFO:tasks.rados.rados.0.trial115.stdout:3822: oid 14 version 1702 is already newer than 1701 2026-02-17T06:09:48.899 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1702 (ObjNum 1306 snap 404 seq_num 1306) dirty exists 2026-02-17T06:09:48.899 INFO:tasks.rados.rados.0.trial115.stdout:3822: left oid 14 (ObjNum 1306 snap 404 seq_num 1306) 2026-02-17T06:09:48.899 INFO:tasks.rados.rados.0.trial115.stdout:3822: done (1 left) 2026-02-17T06:09:48.900 INFO:tasks.rados.rados.0.trial115.stdout:3823: done (0 left) 2026-02-17T06:09:48.900 INFO:tasks.rados.rados.0.trial115.stdout:3824: copy_from oid 31 from oid 41 current snap is 404 2026-02-17T06:09:48.900 INFO:tasks.rados.rados.0.trial115.stdout:3825: rollback oid 3 current snap is 404 2026-02-17T06:09:48.900 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 3 to 403 2026-02-17T06:09:48.900 INFO:tasks.rados.rados.0.trial115.stdout:3826: snap_remove snap 402 2026-02-17T06:09:49.884 INFO:tasks.rados.rados.0.trial115.stdout:3826: done (2 left) 2026-02-17T06:09:49.884 INFO:tasks.rados.rados.0.trial115.stdout:3827: write oid 22 current snap is 404 2026-02-17T06:09:49.884 INFO:tasks.rados.rados.0.trial115.stdout:3827: seq_num 1308 ranges {783556=438408,1674212=400111,2736451=420734,3882270=101286} 2026-02-17T06:09:49.888 INFO:tasks.rados.rados.0.trial115.stdout:3827: writing trial11545420-22 from 783556 to 1221964 tid 1 2026-02-17T06:09:49.891 INFO:tasks.rados.rados.0.trial115.stdout:3827: writing trial11545420-22 from 1674212 to 2074323 tid 2 2026-02-17T06:09:49.895 INFO:tasks.rados.rados.0.trial115.stdout:3827: writing trial11545420-22 from 2736451 to 3157185 tid 3 2026-02-17T06:09:49.898 INFO:tasks.rados.rados.0.trial115.stdout:3827: writing trial11545420-22 from 3882270 to 3983556 tid 4 2026-02-17T06:09:49.899 INFO:tasks.rados.rados.0.trial115.stdout:3828: write oid 39 current snap is 404 2026-02-17T06:09:49.899 INFO:tasks.rados.rados.0.trial115.stdout:3828: seq_num 1309 ranges {400363=513657,1332694=660122} 2026-02-17T06:09:49.902 INFO:tasks.rados.rados.0.trial115.stdout:3828: writing trial11545420-39 from 400363 to 914020 tid 1 2026-02-17T06:09:49.906 INFO:tasks.rados.rados.0.trial115.stdout:3828: writing trial11545420-39 from 1332694 to 1992816 tid 2 2026-02-17T06:09:49.906 INFO:tasks.rados.rados.0.trial115.stdout:3829: read oid 45 snap 403 2026-02-17T06:09:49.906 INFO:tasks.rados.rados.0.trial115.stdout:3829: expect (ObjNum 1279 snap 398 seq_num 1279) 2026-02-17T06:09:49.906 INFO:tasks.rados.rados.0.trial115.stdout:3830: write oid 36 current snap is 404 2026-02-17T06:09:49.906 INFO:tasks.rados.rados.0.trial115.stdout:3824: finishing copy_from to trial11545420-31 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3824: got expected ENOENT (src dne) 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3824: finishing copy_from racing read to trial11545420-31 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3825: finishing rollback tid 0 to trial11545420-3 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3825: finishing rollback tid 1 to trial11545420-3 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3825: finishing rollback tid 2 to trial11545420-3 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 3 v 1685 (ObjNum 1215 snap 376 seq_num 1215) dirty exists 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3827: finishing write tid 1 to trial11545420-22 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3827: finishing write tid 2 to trial11545420-22 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3827: finishing write tid 3 to trial11545420-22 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3827: finishing write tid 4 to trial11545420-22 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3827: finishing write tid 5 to trial11545420-22 2026-02-17T06:09:49.907 INFO:tasks.rados.rados.0.trial115.stdout:3830: seq_num 1310 ranges {417170=588906,1674823=342347} 2026-02-17T06:09:49.910 INFO:tasks.rados.rados.0.trial115.stdout:3830: writing trial11545420-36 from 417170 to 1006076 tid 1 2026-02-17T06:09:49.914 INFO:tasks.rados.rados.0.trial115.stdout:3830: writing trial11545420-36 from 1674823 to 2017170 tid 2 2026-02-17T06:09:49.914 INFO:tasks.rados.rados.0.trial115.stdout:3824: done (5 left) 2026-02-17T06:09:49.914 INFO:tasks.rados.rados.0.trial115.stdout:3825: done (4 left) 2026-02-17T06:09:49.914 INFO:tasks.rados.rados.0.trial115.stdout:3831: write oid 15 current snap is 404 2026-02-17T06:09:49.914 INFO:tasks.rados.rados.0.trial115.stdout:3831: seq_num 1311 ranges {433977=664155,1733305=300672} 2026-02-17T06:09:49.918 INFO:tasks.rados.rados.0.trial115.stdout:3831: writing trial11545420-15 from 433977 to 1098132 tid 1 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3831: writing trial11545420-15 from 1733305 to 2033977 tid 2 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3832: copy_from oid 7 from oid 24 current snap is 404 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3827: finishing write tid 6 to trial11545420-22 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3827: oid 22 updating version 0 to 1546 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3827: oid 22 updating version 1546 to 1548 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3827: oid 22 version 1548 is already newer than 1545 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3827: oid 22 updating version 1548 to 1549 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3827: oid 22 version 1549 is already newer than 1547 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1549 (ObjNum 1308 snap 404 seq_num 1308) dirty exists 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3827: left oid 22 (ObjNum 1308 snap 404 seq_num 1308) 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3828: finishing write tid 1 to trial11545420-39 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3827: done (5 left) 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3833: copy_from oid 11 from oid 2 current snap is 404 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3828: finishing write tid 2 to trial11545420-39 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3828: finishing write tid 3 to trial11545420-39 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3828: finishing write tid 4 to trial11545420-39 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3828: oid 39 updating version 0 to 1993 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3828: oid 39 version 1993 is already newer than 1992 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3828: oid 39 version 1993 is already newer than 1991 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1993 (ObjNum 1309 snap 404 seq_num 1309) dirty exists 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3828: left oid 39 (ObjNum 1309 snap 404 seq_num 1309) 2026-02-17T06:09:49.922 INFO:tasks.rados.rados.0.trial115.stdout:3829: expect (ObjNum 1279 snap 398 seq_num 1279) 2026-02-17T06:09:50.041 INFO:tasks.rados.rados.0.trial115.stdout:3830: finishing write tid 1 to trial11545420-36 2026-02-17T06:09:50.041 INFO:tasks.rados.rados.0.trial115.stdout:3830: finishing write tid 2 to trial11545420-36 2026-02-17T06:09:50.041 INFO:tasks.rados.rados.0.trial115.stdout:3830: finishing write tid 3 to trial11545420-36 2026-02-17T06:09:50.041 INFO:tasks.rados.rados.0.trial115.stdout:3828: done (5 left) 2026-02-17T06:09:50.041 INFO:tasks.rados.rados.0.trial115.stdout:3829: done (4 left) 2026-02-17T06:09:50.041 INFO:tasks.rados.rados.0.trial115.stdout:3834: snap_remove snap 403 2026-02-17T06:09:50.887 INFO:tasks.rados.rados.0.trial115.stdout:3834: done (4 left) 2026-02-17T06:09:50.887 INFO:tasks.rados.rados.0.trial115.stdout:3835: write oid 42 current snap is 404 2026-02-17T06:09:50.887 INFO:tasks.rados.rados.0.trial115.stdout:3835: seq_num 1312 ranges {450784=739404,1791787=258997} 2026-02-17T06:09:50.892 INFO:tasks.rados.rados.0.trial115.stdout:3835: writing trial11545420-42 from 450784 to 1190188 tid 1 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3835: writing trial11545420-42 from 1791787 to 2050784 tid 2 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3836: read oid 1 snap -1 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3836: expect (ObjNum 1220 snap 378 seq_num 1220) 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3837: read oid 22 snap -1 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3837: expect (ObjNum 1308 snap 404 seq_num 1308) 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3838: copy_from oid 39 from oid 8 current snap is 404 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3830: finishing write tid 4 to trial11545420-36 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3830: oid 36 updating version 0 to 1703 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3830: oid 36 updating version 1703 to 1705 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3830: oid 36 version 1705 is already newer than 1704 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 36 v 1705 (ObjNum 1310 snap 404 seq_num 1310) dirty exists 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3830: left oid 36 (ObjNum 1310 snap 404 seq_num 1310) 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3831: finishing write tid 1 to trial11545420-15 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3830: done (7 left) 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3839: delete oid 33 current snap is 404 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3831: finishing write tid 2 to trial11545420-15 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3831: finishing write tid 3 to trial11545420-15 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3831: finishing write tid 4 to trial11545420-15 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3831: oid 15 updating version 0 to 2025 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3831: oid 15 updating version 2025 to 2027 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3831: oid 15 version 2027 is already newer than 2026 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 15 v 2027 (ObjNum 1311 snap 404 seq_num 1311) dirty exists 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3831: left oid 15 (ObjNum 1311 snap 404 seq_num 1311) 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3833: finishing copy_from racing read to trial11545420-11 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3832: finishing copy_from racing read to trial11545420-7 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3833: finishing copy_from to trial11545420-11 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 1679 (ObjNum 1302 snap 402 seq_num 1302) dirty exists 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:3832: finishing copy_from to trial11545420-7 2026-02-17T06:09:50.896 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1987 (ObjNum 1288 snap 400 seq_num 1288) dirty exists 2026-02-17T06:09:50.901 INFO:tasks.rados.rados.0.trial115.stdout:3835: finishing write tid 1 to trial11545420-42 2026-02-17T06:09:50.901 INFO:tasks.rados.rados.0.trial115.stdout:3835: finishing write tid 2 to trial11545420-42 2026-02-17T06:09:50.903 INFO:tasks.rados.rados.0.trial115.stdout:3835: finishing write tid 3 to trial11545420-42 2026-02-17T06:09:50.903 INFO:tasks.rados.rados.0.trial115.stdout:3835: finishing write tid 4 to trial11545420-42 2026-02-17T06:09:50.903 INFO:tasks.rados.rados.0.trial115.stdout:3835: oid 42 updating version 0 to 1996 2026-02-17T06:09:50.903 INFO:tasks.rados.rados.0.trial115.stdout:3835: oid 42 version 1996 is already newer than 1995 2026-02-17T06:09:50.903 INFO:tasks.rados.rados.0.trial115.stdout:3835: oid 42 version 1996 is already newer than 1994 2026-02-17T06:09:50.903 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 1996 (ObjNum 1312 snap 404 seq_num 1312) dirty exists 2026-02-17T06:09:50.903 INFO:tasks.rados.rados.0.trial115.stdout:3835: left oid 42 (ObjNum 1312 snap 404 seq_num 1312) 2026-02-17T06:09:50.903 INFO:tasks.rados.rados.0.trial115.stdout:3836: expect (ObjNum 1220 snap 378 seq_num 1220) 2026-02-17T06:09:50.985 INFO:tasks.rados.rados.0.trial115.stdout:3838: finishing copy_from racing read to trial11545420-39 2026-02-17T06:09:50.985 INFO:tasks.rados.rados.0.trial115.stdout:3837: expect (ObjNum 1308 snap 404 seq_num 1308) 2026-02-17T06:09:51.154 INFO:tasks.rados.rados.0.trial115.stdout:3838: finishing copy_from to trial11545420-39 2026-02-17T06:09:51.154 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1482 (ObjNum 1272 snap 398 seq_num 1272) dirty exists 2026-02-17T06:09:51.154 INFO:tasks.rados.rados.0.trial115.stdout:3831: done (7 left) 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3832: done (6 left) 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3833: done (5 left) 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3835: done (4 left) 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3836: done (3 left) 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3837: done (2 left) 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3838: done (1 left) 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3839: done (0 left) 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3840: copy_from oid 23 from oid 28 current snap is 404 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3841: write oid 8 current snap is 404 2026-02-17T06:09:51.155 INFO:tasks.rados.rados.0.trial115.stdout:3841: seq_num 1314 ranges {484398=489902,1392398=411118} 2026-02-17T06:09:51.158 INFO:tasks.rados.rados.0.trial115.stdout:3841: writing trial11545420-8 from 484398 to 974300 tid 1 2026-02-17T06:09:51.162 INFO:tasks.rados.rados.0.trial115.stdout:3841: writing trial11545420-8 from 1392398 to 1803516 tid 2 2026-02-17T06:09:51.162 INFO:tasks.rados.rados.0.trial115.stdout:3842: rollback oid 10 current snap is 404 2026-02-17T06:09:51.162 INFO:tasks.rados.rados.0.trial115.stdout:3842: done (2 left) 2026-02-17T06:09:51.162 INFO:tasks.rados.rados.0.trial115.stdout:3843: snap_create 2026-02-17T06:09:51.162 INFO:tasks.rados.rados.0.trial115.stdout:3840: finishing copy_from to trial11545420-23 2026-02-17T06:09:51.162 INFO:tasks.rados.rados.0.trial115.stdout:3840: got expected ENOENT (src dne) 2026-02-17T06:09:51.162 INFO:tasks.rados.rados.0.trial115.stdout:3840: finishing copy_from racing read to trial11545420-23 2026-02-17T06:09:51.167 INFO:tasks.rados.rados.0.trial115.stdout:3841: finishing write tid 1 to trial11545420-8 2026-02-17T06:09:51.167 INFO:tasks.rados.rados.0.trial115.stdout:3841: finishing write tid 2 to trial11545420-8 2026-02-17T06:09:51.167 INFO:tasks.rados.rados.0.trial115.stdout:3841: finishing write tid 3 to trial11545420-8 2026-02-17T06:09:51.167 INFO:tasks.rados.rados.0.trial115.stdout:3841: finishing write tid 4 to trial11545420-8 2026-02-17T06:09:51.168 INFO:tasks.rados.rados.0.trial115.stdout:3841: oid 8 updating version 0 to 1548 2026-02-17T06:09:51.168 INFO:tasks.rados.rados.0.trial115.stdout:3841: oid 8 version 1548 is already newer than 1546 2026-02-17T06:09:51.168 INFO:tasks.rados.rados.0.trial115.stdout:3841: oid 8 version 1548 is already newer than 1547 2026-02-17T06:09:51.168 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 1548 (ObjNum 1314 snap 404 seq_num 1314) dirty exists 2026-02-17T06:09:51.168 INFO:tasks.rados.rados.0.trial115.stdout:3841: left oid 8 (ObjNum 1314 snap 404 seq_num 1314) 2026-02-17T06:09:51.890 INFO:tasks.rados.rados.0.trial115.stdout:3840: done (2 left) 2026-02-17T06:09:51.891 INFO:tasks.rados.rados.0.trial115.stdout:3841: done (1 left) 2026-02-17T06:09:51.891 INFO:tasks.rados.rados.0.trial115.stdout:3843: done (0 left) 2026-02-17T06:09:51.891 INFO:tasks.rados.rados.0.trial115.stdout:3844: copy_from oid 20 from oid 44 current snap is 405 2026-02-17T06:09:51.891 INFO:tasks.rados.rados.0.trial115.stdout:3845: snap_create 2026-02-17T06:09:51.900 INFO:tasks.rados.rados.0.trial115.stdout:3844: finishing copy_from racing read to trial11545420-20 2026-02-17T06:09:51.903 INFO:tasks.rados.rados.0.trial115.stdout:3844: finishing copy_from to trial11545420-20 2026-02-17T06:09:51.903 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 20 v 1900 (ObjNum 1168 snap 359 seq_num 1168) dirty exists 2026-02-17T06:09:52.893 INFO:tasks.rados.rados.0.trial115.stdout:3844: done (1 left) 2026-02-17T06:09:52.893 INFO:tasks.rados.rados.0.trial115.stdout:3845: done (0 left) 2026-02-17T06:09:52.893 INFO:tasks.rados.rados.0.trial115.stdout:3846: delete oid 50 current snap is 406 2026-02-17T06:09:52.896 INFO:tasks.rados.rados.0.trial115.stdout:3846: done (0 left) 2026-02-17T06:09:52.896 INFO:tasks.rados.rados.0.trial115.stdout:3847: read oid 24 snap 404 2026-02-17T06:09:52.896 INFO:tasks.rados.rados.0.trial115.stdout:3847: expect (ObjNum 1288 snap 400 seq_num 1288) 2026-02-17T06:09:52.897 INFO:tasks.rados.rados.0.trial115.stdout:3848: write oid 49 current snap is 406 2026-02-17T06:09:52.897 INFO:tasks.rados.rados.0.trial115.stdout:3848: seq_num 1316 ranges {518012=756753,2025715=92297} 2026-02-17T06:09:52.900 INFO:tasks.rados.rados.0.trial115.stdout:3848: writing trial11545420-49 from 518012 to 1274765 tid 1 2026-02-17T06:09:52.904 INFO:tasks.rados.rados.0.trial115.stdout:3848: writing trial11545420-49 from 2025715 to 2118012 tid 2 2026-02-17T06:09:52.904 INFO:tasks.rados.rados.0.trial115.stdout:3849: read oid 14 snap 404 2026-02-17T06:09:52.904 INFO:tasks.rados.rados.0.trial115.stdout:3849: expect (ObjNum 1306 snap 404 seq_num 1306) 2026-02-17T06:09:52.904 INFO:tasks.rados.rados.0.trial115.stdout:3850: rollback oid 5 current snap is 406 2026-02-17T06:09:52.904 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 5 to 405 2026-02-17T06:09:52.904 INFO:tasks.rados.rados.0.trial115.stdout:3847: expect (ObjNum 1288 snap 400 seq_num 1288) 2026-02-17T06:09:53.014 INFO:tasks.rados.rados.0.trial115.stdout:3848: finishing write tid 1 to trial11545420-49 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:3848: finishing write tid 2 to trial11545420-49 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:3848: finishing write tid 3 to trial11545420-49 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:3848: finishing write tid 4 to trial11545420-49 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:3848: oid 49 updating version 0 to 1510 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:3848: oid 49 version 1510 is already newer than 1509 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:3848: oid 49 updating version 1510 to 1511 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1511 (ObjNum 1316 snap 406 seq_num 1316) dirty exists 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:3848: left oid 49 (ObjNum 1316 snap 406 seq_num 1316) 2026-02-17T06:09:53.015 INFO:tasks.rados.rados.0.trial115.stdout:3850: finishing rollback tid 0 to trial11545420-5 2026-02-17T06:09:53.016 INFO:tasks.rados.rados.0.trial115.stdout:3850: finishing rollback tid 1 to trial11545420-5 2026-02-17T06:09:53.016 INFO:tasks.rados.rados.0.trial115.stdout:3850: finishing rollback tid 2 to trial11545420-5 2026-02-17T06:09:53.016 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 5 v 1693 (ObjNum 1279 snap 398 seq_num 1279) dirty exists 2026-02-17T06:09:53.016 INFO:tasks.rados.rados.0.trial115.stdout:3849: expect (ObjNum 1306 snap 404 seq_num 1306) 2026-02-17T06:09:53.130 INFO:tasks.rados.rados.0.trial115.stdout:3847: done (3 left) 2026-02-17T06:09:53.130 INFO:tasks.rados.rados.0.trial115.stdout:3848: done (2 left) 2026-02-17T06:09:53.130 INFO:tasks.rados.rados.0.trial115.stdout:3849: done (1 left) 2026-02-17T06:09:53.130 INFO:tasks.rados.rados.0.trial115.stdout:3850: done (0 left) 2026-02-17T06:09:53.130 INFO:tasks.rados.rados.0.trial115.stdout:3851: snap_remove snap 405 2026-02-17T06:09:53.873 INFO:tasks.rados.rados.0.trial115.stdout:3851: done (0 left) 2026-02-17T06:09:53.874 INFO:tasks.rados.rados.0.trial115.stdout:3852: snap_remove snap 404 2026-02-17T06:09:54.874 INFO:tasks.rados.rados.0.trial115.stdout:3852: done (0 left) 2026-02-17T06:09:54.874 INFO:tasks.rados.rados.0.trial115.stdout:3853: read oid 20 snap -1 2026-02-17T06:09:54.874 INFO:tasks.rados.rados.0.trial115.stdout:3853: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:09:54.874 INFO:tasks.rados.rados.0.trial115.stdout:3854: rollback oid 12 current snap is 406 2026-02-17T06:09:54.874 INFO:tasks.rados.rados.0.trial115.stdout:3854: done (1 left) 2026-02-17T06:09:54.874 INFO:tasks.rados.rados.0.trial115.stdout:3855: read oid 5 snap -1 2026-02-17T06:09:54.874 INFO:tasks.rados.rados.0.trial115.stdout:3855: expect (ObjNum 1279 snap 398 seq_num 1279) 2026-02-17T06:09:54.875 INFO:tasks.rados.rados.0.trial115.stdout:3856: read oid 6 snap -1 2026-02-17T06:09:54.875 INFO:tasks.rados.rados.0.trial115.stdout:3856: expect deleted 2026-02-17T06:09:54.875 INFO:tasks.rados.rados.0.trial115.stdout:3857: delete oid 8 current snap is 406 2026-02-17T06:09:54.876 INFO:tasks.rados.rados.0.trial115.stdout:3856: done (3 left) 2026-02-17T06:09:54.876 INFO:tasks.rados.rados.0.trial115.stdout:3857: done (2 left) 2026-02-17T06:09:54.877 INFO:tasks.rados.rados.0.trial115.stdout:3858: rollback oid 14 current snap is 406 2026-02-17T06:09:54.877 INFO:tasks.rados.rados.0.trial115.stdout:3858: done (2 left) 2026-02-17T06:09:54.877 INFO:tasks.rados.rados.0.trial115.stdout:3859: write oid 22 current snap is 406 2026-02-17T06:09:54.877 INFO:tasks.rados.rados.0.trial115.stdout:3859: seq_num 1318 ranges {551626=507251,1626326=418456} 2026-02-17T06:09:54.880 INFO:tasks.rados.rados.0.trial115.stdout:3859: writing trial11545420-22 from 551626 to 1058877 tid 1 2026-02-17T06:09:54.884 INFO:tasks.rados.rados.0.trial115.stdout:3859: writing trial11545420-22 from 1626326 to 2044782 tid 2 2026-02-17T06:09:54.885 INFO:tasks.rados.rados.0.trial115.stdout:3860: rollback oid 49 current snap is 406 2026-02-17T06:09:54.885 INFO:tasks.rados.rados.0.trial115.stdout:3860: done (3 left) 2026-02-17T06:09:54.885 INFO:tasks.rados.rados.0.trial115.stdout:3861: read oid 28 snap -1 2026-02-17T06:09:54.885 INFO:tasks.rados.rados.0.trial115.stdout:3861: expect deleted 2026-02-17T06:09:54.885 INFO:tasks.rados.rados.0.trial115.stdout:3862: read oid 36 snap -1 2026-02-17T06:09:54.885 INFO:tasks.rados.rados.0.trial115.stdout:3862: expect (ObjNum 1310 snap 404 seq_num 1310) 2026-02-17T06:09:54.885 INFO:tasks.rados.rados.0.trial115.stdout:3863: snap_create 2026-02-17T06:09:54.885 INFO:tasks.rados.rados.0.trial115.stdout:3855: expect (ObjNum 1279 snap 398 seq_num 1279) 2026-02-17T06:09:55.002 INFO:tasks.rados.rados.0.trial115.stdout:3853: expect (ObjNum 1168 snap 359 seq_num 1168) 2026-02-17T06:09:55.202 INFO:tasks.rados.rados.0.trial115.stdout:3859: finishing write tid 1 to trial11545420-22 2026-02-17T06:09:55.202 INFO:tasks.rados.rados.0.trial115.stdout:3862: expect (ObjNum 1310 snap 404 seq_num 1310) 2026-02-17T06:09:55.250 INFO:tasks.rados.rados.0.trial115.stdout:3859: finishing write tid 2 to trial11545420-22 2026-02-17T06:09:55.250 INFO:tasks.rados.rados.0.trial115.stdout:3859: finishing write tid 3 to trial11545420-22 2026-02-17T06:09:55.250 INFO:tasks.rados.rados.0.trial115.stdout:3859: finishing write tid 4 to trial11545420-22 2026-02-17T06:09:55.250 INFO:tasks.rados.rados.0.trial115.stdout:3859: oid 22 updating version 0 to 1559 2026-02-17T06:09:55.250 INFO:tasks.rados.rados.0.trial115.stdout:3859: oid 22 version 1559 is already newer than 1556 2026-02-17T06:09:55.250 INFO:tasks.rados.rados.0.trial115.stdout:3859: oid 22 updating version 1559 to 1560 2026-02-17T06:09:55.250 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1560 (ObjNum 1318 snap 406 seq_num 1318) dirty exists 2026-02-17T06:09:55.250 INFO:tasks.rados.rados.0.trial115.stdout:3859: left oid 22 (ObjNum 1318 snap 406 seq_num 1318) 2026-02-17T06:09:56.337 INFO:tasks.rados.rados.0.trial115.stdout:3853: done (5 left) 2026-02-17T06:09:56.337 INFO:tasks.rados.rados.0.trial115.stdout:3855: done (4 left) 2026-02-17T06:09:56.337 INFO:tasks.rados.rados.0.trial115.stdout:3859: done (3 left) 2026-02-17T06:09:56.337 INFO:tasks.rados.rados.0.trial115.stdout:3861: done (2 left) 2026-02-17T06:09:56.338 INFO:tasks.rados.rados.0.trial115.stdout:3862: done (1 left) 2026-02-17T06:09:56.338 INFO:tasks.rados.rados.0.trial115.stdout:3863: done (0 left) 2026-02-17T06:09:56.338 INFO:tasks.rados.rados.0.trial115.stdout:3864: delete oid 8 current snap is 407 2026-02-17T06:09:56.339 INFO:tasks.rados.rados.0.trial115.stdout:3864: done (0 left) 2026-02-17T06:09:56.339 INFO:tasks.rados.rados.0.trial115.stdout:3865: write oid 22 current snap is 407 2026-02-17T06:09:56.339 INFO:tasks.rados.rados.0.trial115.stdout:3865: seq_num 1320 ranges {585240=657749,1743290=422125} 2026-02-17T06:09:56.344 INFO:tasks.rados.rados.0.trial115.stdout:3865: writing trial11545420-22 from 585240 to 1242989 tid 1 2026-02-17T06:09:56.348 INFO:tasks.rados.rados.0.trial115.stdout:3865: writing trial11545420-22 from 1743290 to 2165415 tid 2 2026-02-17T06:09:56.348 INFO:tasks.rados.rados.0.trial115.stdout:3866: read oid 10 snap -1 2026-02-17T06:09:56.348 INFO:tasks.rados.rados.0.trial115.stdout:3866: expect (ObjNum 1262 snap 395 seq_num 1262) 2026-02-17T06:09:56.348 INFO:tasks.rados.rados.0.trial115.stdout:3867: read oid 11 snap -1 2026-02-17T06:09:56.348 INFO:tasks.rados.rados.0.trial115.stdout:3867: expect (ObjNum 1302 snap 402 seq_num 1302) 2026-02-17T06:09:56.348 INFO:tasks.rados.rados.0.trial115.stdout:3868: delete oid 19 current snap is 407 2026-02-17T06:09:56.350 INFO:tasks.rados.rados.0.trial115.stdout:3866: expect (ObjNum 1262 snap 395 seq_num 1262) 2026-02-17T06:09:56.443 INFO:tasks.rados.rados.0.trial115.stdout:3865: finishing write tid 1 to trial11545420-22 2026-02-17T06:09:56.443 INFO:tasks.rados.rados.0.trial115.stdout:3865: finishing write tid 2 to trial11545420-22 2026-02-17T06:09:56.444 INFO:tasks.rados.rados.0.trial115.stdout:3866: done (3 left) 2026-02-17T06:09:56.444 INFO:tasks.rados.rados.0.trial115.stdout:3868: done (2 left) 2026-02-17T06:09:56.444 INFO:tasks.rados.rados.0.trial115.stdout:3869: read oid 23 snap -1 2026-02-17T06:09:56.444 INFO:tasks.rados.rados.0.trial115.stdout:3869: expect (ObjNum 1289 snap 400 seq_num 1289) 2026-02-17T06:09:56.444 INFO:tasks.rados.rados.0.trial115.stdout:3870: read oid 49 snap -1 2026-02-17T06:09:56.444 INFO:tasks.rados.rados.0.trial115.stdout:3870: expect (ObjNum 1316 snap 406 seq_num 1316) 2026-02-17T06:09:56.445 INFO:tasks.rados.rados.0.trial115.stdout:3871: read oid 42 snap 406 2026-02-17T06:09:56.445 INFO:tasks.rados.rados.0.trial115.stdout:3871: expect (ObjNum 1312 snap 404 seq_num 1312) 2026-02-17T06:09:56.445 INFO:tasks.rados.rados.0.trial115.stdout:3872: rollback oid 45 current snap is 407 2026-02-17T06:09:56.445 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 45 to 406 2026-02-17T06:09:56.445 INFO:tasks.rados.rados.0.trial115.stdout:3865: finishing write tid 3 to trial11545420-22 2026-02-17T06:09:56.445 INFO:tasks.rados.rados.0.trial115.stdout:3865: finishing write tid 4 to trial11545420-22 2026-02-17T06:09:56.445 INFO:tasks.rados.rados.0.trial115.stdout:3865: oid 22 updating version 0 to 1563 2026-02-17T06:09:56.446 INFO:tasks.rados.rados.0.trial115.stdout:3865: oid 22 version 1563 is already newer than 1562 2026-02-17T06:09:56.446 INFO:tasks.rados.rados.0.trial115.stdout:3865: oid 22 updating version 1563 to 1564 2026-02-17T06:09:56.446 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 22 v 1564 (ObjNum 1320 snap 407 seq_num 1320) dirty exists 2026-02-17T06:09:56.446 INFO:tasks.rados.rados.0.trial115.stdout:3865: left oid 22 (ObjNum 1320 snap 407 seq_num 1320) 2026-02-17T06:09:56.446 INFO:tasks.rados.rados.0.trial115.stdout:3867: expect (ObjNum 1302 snap 402 seq_num 1302) 2026-02-17T06:09:56.564 INFO:tasks.rados.rados.0.trial115.stdout:3871: expect (ObjNum 1312 snap 404 seq_num 1312) 2026-02-17T06:09:56.695 INFO:tasks.rados.rados.0.trial115.stdout:3872: finishing rollback tid 0 to trial11545420-45 2026-02-17T06:09:56.695 INFO:tasks.rados.rados.0.trial115.stdout:3865: done (5 left) 2026-02-17T06:09:56.695 INFO:tasks.rados.rados.0.trial115.stdout:3867: done (4 left) 2026-02-17T06:09:56.695 INFO:tasks.rados.rados.0.trial115.stdout:3871: done (3 left) 2026-02-17T06:09:56.696 INFO:tasks.rados.rados.0.trial115.stdout:3873: write oid 47 current snap is 407 2026-02-17T06:09:56.696 INFO:tasks.rados.rados.0.trial115.stdout:3872: finishing rollback tid 1 to trial11545420-45 2026-02-17T06:09:56.696 INFO:tasks.rados.rados.0.trial115.stdout:3872: finishing rollback tid 2 to trial11545420-45 2026-02-17T06:09:56.696 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 45 v 2003 (ObjNum 1279 snap 398 seq_num 1279) dirty exists 2026-02-17T06:09:56.696 INFO:tasks.rados.rados.0.trial115.stdout:3870: expect (ObjNum 1316 snap 406 seq_num 1316) 2026-02-17T06:09:56.795 INFO:tasks.rados.rados.0.trial115.stdout:3869: expect (ObjNum 1289 snap 400 seq_num 1289) 2026-02-17T06:09:56.878 INFO:tasks.rados.rados.0.trial115.stdout:3873: seq_num 1322 ranges {618854=408247,1743901=425794} 2026-02-17T06:09:56.882 INFO:tasks.rados.rados.0.trial115.stdout:3873: writing trial11545420-47 from 618854 to 1027101 tid 1 2026-02-17T06:09:56.887 INFO:tasks.rados.rados.0.trial115.stdout:3873: writing trial11545420-47 from 1743901 to 2169695 tid 2 2026-02-17T06:09:56.887 INFO:tasks.rados.rados.0.trial115.stdout:3869: done (3 left) 2026-02-17T06:09:56.888 INFO:tasks.rados.rados.0.trial115.stdout:3870: done (2 left) 2026-02-17T06:09:56.888 INFO:tasks.rados.rados.0.trial115.stdout:3872: done (1 left) 2026-02-17T06:09:56.888 INFO:tasks.rados.rados.0.trial115.stdout:3874: copy_from oid 4 from oid 50 current snap is 407 2026-02-17T06:09:56.888 INFO:tasks.rados.rados.0.trial115.stdout:3873: finishing write tid 1 to trial11545420-47 2026-02-17T06:09:56.888 INFO:tasks.rados.rados.0.trial115.stdout:3875: write (excl) oid 32 current snap is 407 2026-02-17T06:09:56.888 INFO:tasks.rados.rados.0.trial115.stdout:3875: seq_num 1323 ranges {635661=599849,1918736=316925} 2026-02-17T06:09:56.891 INFO:tasks.rados.rados.0.trial115.stdout:3875: writing trial11545420-32 from 635661 to 1235510 tid 1 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3875: writing trial11545420-32 from 1918736 to 2235661 tid 2 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3876: snap_create 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3874: finishing copy_from to trial11545420-4 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3874: got expected ENOENT (src dne) 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3874: finishing copy_from racing read to trial11545420-4 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3873: finishing write tid 2 to trial11545420-47 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3873: finishing write tid 3 to trial11545420-47 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3873: finishing write tid 4 to trial11545420-47 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3873: oid 47 updating version 0 to 1700 2026-02-17T06:09:56.896 INFO:tasks.rados.rados.0.trial115.stdout:3873: oid 47 version 1700 is already newer than 1698 2026-02-17T06:09:56.897 INFO:tasks.rados.rados.0.trial115.stdout:3873: oid 47 version 1700 is already newer than 1699 2026-02-17T06:09:56.897 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1700 (ObjNum 1322 snap 407 seq_num 1322) dirty exists 2026-02-17T06:09:56.897 INFO:tasks.rados.rados.0.trial115.stdout:3873: left oid 47 (ObjNum 1322 snap 407 seq_num 1322) 2026-02-17T06:09:56.897 INFO:tasks.rados.rados.0.trial115.stdout:3875: finishing write tid 1 to trial11545420-32 2026-02-17T06:09:56.898 INFO:tasks.rados.rados.0.trial115.stdout:3875: finishing write tid 2 to trial11545420-32 2026-02-17T06:09:56.898 INFO:tasks.rados.rados.0.trial115.stdout:3875: finishing write tid 3 to trial11545420-32 2026-02-17T06:09:56.898 INFO:tasks.rados.rados.0.trial115.stdout:3875: finishing write tid 4 to trial11545420-32 2026-02-17T06:09:56.898 INFO:tasks.rados.rados.0.trial115.stdout:3875: oid 32 updating version 0 to 1901 2026-02-17T06:09:56.898 INFO:tasks.rados.rados.0.trial115.stdout:3875: oid 32 updating version 1901 to 1903 2026-02-17T06:09:56.898 INFO:tasks.rados.rados.0.trial115.stdout:3875: oid 32 version 1903 is already newer than 1902 2026-02-17T06:09:56.898 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1903 (ObjNum 1323 snap 407 seq_num 1323) dirty exists 2026-02-17T06:09:56.898 INFO:tasks.rados.rados.0.trial115.stdout:3875: left oid 32 (ObjNum 1323 snap 407 seq_num 1323) 2026-02-17T06:09:57.877 INFO:tasks.rados.rados.0.trial115.stdout:3873: done (3 left) 2026-02-17T06:09:57.877 INFO:tasks.rados.rados.0.trial115.stdout:3874: done (2 left) 2026-02-17T06:09:57.878 INFO:tasks.rados.rados.0.trial115.stdout:3875: done (1 left) 2026-02-17T06:09:57.878 INFO:tasks.rados.rados.0.trial115.stdout:3876: done (0 left) 2026-02-17T06:09:57.878 INFO:tasks.rados.rados.0.trial115.stdout:3877: snap_create 2026-02-17T06:09:58.872 INFO:tasks.rados.rados.0.trial115.stdout:3877: done (0 left) 2026-02-17T06:09:58.873 INFO:tasks.rados.rados.0.trial115.stdout:3878: snap_create 2026-02-17T06:09:59.235 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:09:59.236 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T06:09:59.236 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T06:09:59.236 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T06:09:59.239 INFO:tasks.ceph.mon.b:Started 2026-02-17T06:09:59.239 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:09:59.239 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:09:59.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient: get_monmap_and_config 2026-02-17T06:09:59.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient: build_initial_monmap 2026-02-17T06:09:59.310 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient: monmap: 2026-02-17T06:09:59.310 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:09:59.308427+0000 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:09:59.308427+0000 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:59.311 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 1 Processor -- start 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 1 -- start start 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient: init 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient: _add_conns ranks=[3,1,4,0,2] 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3a3c0a6470 0x7f3a3c0a1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient(hunting): picked mon.noname-a con 0x7f3a3c0a6470 addr v2:10.20.193.78:3300/0 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3a3c0a6850 addr v1:10.20.193.115:6789/0 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f3a3c0a6c60 0x7f3a3c138660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient(hunting): picked mon.noname-c con 0x7f3a3c0a6c60 addr v2:10.20.193.115:3300/0 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a3c0a0730 con 0x7f3a3c0a6850 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:59.312 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f3a3c09fa00 con 0x7f3a3c0a6470 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f3a3c0a19c0 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.307+0000 7f3a42bff640 10 monclient(hunting): authenticate will time out at 2044.861989s 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3a3c0a6470 0x7f3a3c0a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3a3c0a6470 0x7f3a3c0a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42574/0 (socket says 10.20.193.78:42574) 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 1 -- 10.20.193.78:0/1694196462 learned_addr learned my addr 10.20.193.78:0/1694196462 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient(hunting): get_auth_request con 0x7f3a3c0a6470 auth_method 0 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:59.313 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a41175640 1 --2- 10.20.193.78:0/1694196462 >> v2:10.20.193.115:3300/0 conn(0x7f3a3c0a6c60 0x7f3a3c138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a41175640 10 monclient(hunting): get_auth_request con 0x7f3a3c0a6c60 auth_method 0 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a41175640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a41175640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a41175640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient(hunting): handle_auth_done global_id 284193 payload 274 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 1 -- 10.20.193.78:0/1694196462 >> v2:10.20.193.115:3300/0 conn(0x7f3a3c0a6c60 msgr2=0x7f3a3c138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 1 --2- 10.20.193.78:0/1694196462 >> v2:10.20.193.115:3300/0 conn(0x7f3a3c0a6c60 0x7f3a3c138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 1 -- 10.20.193.78:0/1694196462 >> v1:10.20.193.115:6789/0 conn(0x7f3a3c0a6850 legacy=0x7f3a3c137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:09:59.314 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient: _finish_hunting 0 2026-02-17T06:09:59.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 1 monclient: found mon.noname-a 2026-02-17T06:09:59.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:59.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:59.315 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.308+0000 7f3a40974640 1 -- 10.20.193.78:0/1694196462 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c0a0450 con 0x7f3a3c0a6470 2026-02-17T06:09:59.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a40974640 1 -- 10.20.193.78:0/1694196462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6470 msgr2=0x7f3a3c0a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:59.323 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a40974640 1 --2- 10.20.193.78:0/1694196462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6470 0x7f3a3c0a1480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0a7090 tx=0x7f3a2c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569d471e84dc89c2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 1 -- 10.20.193.78:0/1694196462 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c03c070 con 0x7f3a3c0a6470 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: dump: 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:59.324 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: _finish_auth 0 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: _check_auth_tickets 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 1 -- 10.20.193.78:0/1694196462 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2c02fca0 con 0x7f3a3c0a6470 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a42bff640 5 monclient: authenticate success, global_id 284193 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 1 -- 10.20.193.78:0/1694196462 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c037660 con 0x7f3a3c0a6470 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a42bff640 10 monclient: get_monmap_and_config success 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a42bff640 4 set_mon_vals no callback set 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.310+0000 7f3a337fe640 10 monclient: dump: 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:59.325 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 10 monclient: shutdown 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 -- 10.20.193.78:0/1694196462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6470 msgr2=0x7f3a3c0a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 --2- 10.20.193.78:0/1694196462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6470 0x7f3a3c0a1480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0a7090 tx=0x7f3a2c02ee60 comp rx=0 tx=0).stop 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 -- 10.20.193.78:0/1694196462 shutdown_connections 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 --2- 10.20.193.78:0/1694196462 >> v2:10.20.193.115:3300/0 conn(0x7f3a3c0a6c60 0x7f3a3c138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 --2- 10.20.193.78:0/1694196462 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6470 0x7f3a3c0a1480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 -- 10.20.193.78:0/1694196462 >> 10.20.193.78:0/1694196462 conn(0x7f3a3c0aae40 msgr2=0x7f3a3c0ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 -- 10.20.193.78:0/1694196462 shutdown_connections 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 -- 10.20.193.78:0/1694196462 wait complete. 2026-02-17T06:09:59.326 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 10 monclient: build_initial_monmap 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 10 monclient: monmap: 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.311+0000 7f3a42bff640 1 Processor -- start 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 1 -- start start 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient: init 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient: _reopen_session rank -1 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:09:59.327 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a3c0a6470 0x7f3a3c12fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient(hunting): picked mon.noname-b con 0x7f3a3c0a6470 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a3c0a6850 0x7f3a3c130020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient(hunting): picked mon.noname-c con 0x7f3a3c0a6850 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6c60 0x7f3a3c130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient(hunting): picked mon.noname-a con 0x7f3a3c0a6c60 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c138ba0 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c138d20 con 0x7f3a3c0a6470 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient(hunting): start opening mon connection 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3c140290 con 0x7f3a3c0a6850 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient(hunting): _renew_subs 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a42bff640 10 monclient(hunting): authenticate will time out at 2044.866297s 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6c60 0x7f3a3c130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a33fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a3c0a6850 0x7f3a3c130020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6c60 0x7f3a3c130560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42582/0 (socket says 10.20.193.78:42582) 2026-02-17T06:09:59.328 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 -- 10.20.193.78:0/2932472161 learned_addr learned my addr 10.20.193.78:0/2932472161 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient(hunting): get_auth_request con 0x7f3a3c0a6c60 auth_method 0 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a33fff640 10 monclient(hunting): get_auth_request con 0x7f3a3c0a6850 auth_method 0 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a33fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a40974640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a3c0a6470 0x7f3a3c12fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a40974640 10 monclient(hunting): get_auth_request con 0x7f3a3c0a6470 auth_method 0 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a40974640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:09:59.329 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a40974640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a40974640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient(hunting): handle_auth_done global_id 284196 payload 824 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 -- 10.20.193.78:0/2932472161 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a3c0a6850 msgr2=0x7f3a3c130020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a3c0a6850 0x7f3a3c130020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 -- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a3c0a6470 msgr2=0x7f3a3c12fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a3c0a6470 0x7f3a3c12fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient: _finish_hunting 0 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 monclient: found mon.noname-a 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.312+0000 7f3a41175640 1 -- 10.20.193.78:0/2932472161 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c1319e0 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a41175640 1 -- 10.20.193.78:0/2932472161 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6c60 msgr2=0x7f3a3c130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a41175640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6c60 0x7f3a3c130560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a38007e70 tx=0x7f3a3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 1 -- 10.20.193.78:0/2932472161 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3800eea0 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: dump: 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:59.330 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: _finish_auth 0 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: _check_auth_tickets 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 1 -- 10.20.193.78:0/2932472161 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a38019070 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 1 -- 10.20.193.78:0/2932472161 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a380139f0 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a42bff640 5 monclient: authenticate success, global_id 284196 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a32ffd640 4 set_mon_vals no callback set 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:09:59.331 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a317fa640 10 monclient: dump: 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a42bff640 10 monclient: _renew_subs 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a42bff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3c136720 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a42bff640 10 monclient: _renew_subs 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a42bff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.314+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3c1320a0 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.315+0000 7f3a42bff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.315+0000 7f3a42bff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.315+0000 7f3a42bff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:59.332 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.315+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3c1322d0 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.315+0000 7f3a317fa640 1 -- 10.20.193.78:0/2932472161 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3a38013b90 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.315+0000 7f3a317fa640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a0c03deb0 0x7f3a0c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.315+0000 7f3a317fa640 1 -- 10.20.193.78:0/2932472161 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(847..847 src has 525..847) ==== 4735+0+0 (secure 0 0 0) 0x7f3a38053060 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.315+0000 7f3a40974640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a0c03deb0 0x7f3a0c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.316+0000 7f3a40974640 10 monclient: get_auth_request con 0x7f3a0c03deb0 auth_method 0 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.318+0000 7f3a317fa640 1 -- 10.20.193.78:0/2932472161 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3a3c1322d0 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.318+0000 7f3a317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.318+0000 7f3a317fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:09:59.333 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.318+0000 7f3a40974640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a0c03deb0 0x7f3a0c040370 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c036000 tx=0x7f3a2c039040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:09:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.431+0000 7f3a42bff640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:09:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.431+0000 7f3a42bff640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:59.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.431+0000 7f3a42bff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:09:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.431+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3a3c074980 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.431+0000 7f3a317fa640 1 -- 10.20.193.78:0/2932472161 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f3a3801b310 con 0x7f3a3c0a6c60 2026-02-17T06:09:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.431+0000 7f3a317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:09:59.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.431+0000 7f3a317fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:09:59.435 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":262,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:09:59.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a0c03deb0 msgr2=0x7f3a0c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:59.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a0c03deb0 0x7f3a0c040370 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c036000 tx=0x7f3a2c039040 comp rx=0 tx=0).stop 2026-02-17T06:09:59.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 10 monclient: shutdown 2026-02-17T06:09:59.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:09:59.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6c60 msgr2=0x7f3a3c130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:09:59.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6c60 0x7f3a3c130560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a38007e70 tx=0x7f3a3800e4b0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 shutdown_connections 2026-02-17T06:09:59.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3a3c0a6c60 0x7f3a3c130560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3a3c0a6850 0x7f3a3c130020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3a0c03deb0 0x7f3a0c040370 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 --2- 10.20.193.78:0/2932472161 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3a3c0a6470 0x7f3a3c12fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:09:59.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 >> 10.20.193.78:0/2932472161 conn(0x7f3a3c0aae40 msgr2=0x7f3a3c0a97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:09:59.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 shutdown_connections 2026-02-17T06:09:59.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:09:59.432+0000 7f3a42bff640 1 -- 10.20.193.78:0/2932472161 wait complete. 2026-02-17T06:09:59.876 INFO:tasks.rados.rados.0.trial115.stdout:3878: done (0 left) 2026-02-17T06:09:59.876 INFO:tasks.rados.rados.0.trial115.stdout:3879: delete oid 7 current snap is 410 2026-02-17T06:09:59.879 INFO:tasks.rados.rados.0.trial115.stdout:3879: done (0 left) 2026-02-17T06:09:59.879 INFO:tasks.rados.rados.0.trial115.stdout:3880: write oid 11 current snap is 410 2026-02-17T06:09:59.879 INFO:tasks.rados.rados.0.trial115.stdout:3880: seq_num 1325 ranges {669275=750347,2035700=233575} 2026-02-17T06:09:59.885 INFO:tasks.rados.rados.0.trial115.stdout:3880: writing trial11545420-11 from 669275 to 1419622 tid 1 2026-02-17T06:09:59.889 INFO:tasks.rados.rados.0.trial115.stdout:3880: writing trial11545420-11 from 2035700 to 2269275 tid 2 2026-02-17T06:09:59.889 INFO:tasks.rados.rados.0.trial115.stdout:3881: read oid 46 snap -1 2026-02-17T06:09:59.889 INFO:tasks.rados.rados.0.trial115.stdout:3881: expect (ObjNum 1280 snap 398 seq_num 1280) 2026-02-17T06:09:59.889 INFO:tasks.rados.rados.0.trial115.stdout:3882: snap_remove snap 407 2026-02-17T06:10:00.889 INFO:tasks.rados.rados.0.trial115.stdout:3882: done (2 left) 2026-02-17T06:10:00.889 INFO:tasks.rados.rados.0.trial115.stdout:3883: write oid 47 current snap is 410 2026-02-17T06:10:00.889 INFO:tasks.rados.rados.0.trial115.stdout:3880: finishing write tid 1 to trial11545420-11 2026-02-17T06:10:00.889 INFO:tasks.rados.rados.0.trial115.stdout:3880: finishing write tid 2 to trial11545420-11 2026-02-17T06:10:00.889 INFO:tasks.rados.rados.0.trial115.stdout:3880: finishing write tid 3 to trial11545420-11 2026-02-17T06:10:00.889 INFO:tasks.rados.rados.0.trial115.stdout:3880: finishing write tid 4 to trial11545420-11 2026-02-17T06:10:00.889 INFO:tasks.rados.rados.0.trial115.stdout:3880: oid 11 updating version 0 to 2037 2026-02-17T06:10:00.889 INFO:tasks.rados.rados.0.trial115.stdout:3880: oid 11 version 2037 is already newer than 2035 2026-02-17T06:10:00.890 INFO:tasks.rados.rados.0.trial115.stdout:3880: oid 11 version 2037 is already newer than 2036 2026-02-17T06:10:00.890 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 2037 (ObjNum 1325 snap 410 seq_num 1325) dirty exists 2026-02-17T06:10:00.890 INFO:tasks.rados.rados.0.trial115.stdout:3880: left oid 11 (ObjNum 1325 snap 410 seq_num 1325) 2026-02-17T06:10:00.890 INFO:tasks.rados.rados.0.trial115.stdout:3881: expect (ObjNum 1280 snap 398 seq_num 1280) 2026-02-17T06:10:01.036 INFO:tasks.rados.rados.0.trial115.stdout:3883: seq_num 1326 ranges {686082=425596,1577829=433132} 2026-02-17T06:10:01.040 INFO:tasks.rados.rados.0.trial115.stdout:3883: writing trial11545420-47 from 686082 to 1111678 tid 1 2026-02-17T06:10:01.044 INFO:tasks.rados.rados.0.trial115.stdout:3883: writing trial11545420-47 from 1577829 to 2010961 tid 2 2026-02-17T06:10:01.044 INFO:tasks.rados.rados.0.trial115.stdout:3880: done (2 left) 2026-02-17T06:10:01.044 INFO:tasks.rados.rados.0.trial115.stdout:3881: done (1 left) 2026-02-17T06:10:01.045 INFO:tasks.rados.rados.0.trial115.stdout:3884: copy_from oid 44 from oid 18 current snap is 410 2026-02-17T06:10:01.045 INFO:tasks.rados.rados.0.trial115.stdout:3883: finishing write tid 1 to trial11545420-47 2026-02-17T06:10:01.045 INFO:tasks.rados.rados.0.trial115.stdout:3885: write (excl) oid 10 current snap is 410 2026-02-17T06:10:01.045 INFO:tasks.rados.rados.0.trial115.stdout:3885: seq_num 1327 ranges {702889=500845,1636311=576790} 2026-02-17T06:10:01.049 INFO:tasks.rados.rados.0.trial115.stdout:3885: writing trial11545420-10 from 702889 to 1203734 tid 1 2026-02-17T06:10:01.053 INFO:tasks.rados.rados.0.trial115.stdout:3885: writing trial11545420-10 from 1636311 to 2213101 tid 2 2026-02-17T06:10:01.053 INFO:tasks.rados.rados.0.trial115.stdout:3883: finishing write tid 2 to trial11545420-47 2026-02-17T06:10:01.053 INFO:tasks.rados.rados.0.trial115.stdout:3883: finishing write tid 3 to trial11545420-47 2026-02-17T06:10:01.053 INFO:tasks.rados.rados.0.trial115.stdout:3886: read oid 7 snap -1 2026-02-17T06:10:01.054 INFO:tasks.rados.rados.0.trial115.stdout:3886: expect deleted 2026-02-17T06:10:01.054 INFO:tasks.rados.rados.0.trial115.stdout:3887: snap_remove snap 408 2026-02-17T06:10:01.889 INFO:tasks.rados.rados.0.trial115.stdout:3883: finishing write tid 4 to trial11545420-47 2026-02-17T06:10:01.889 INFO:tasks.rados.rados.0.trial115.stdout:3883: oid 47 updating version 0 to 1702 2026-02-17T06:10:01.889 INFO:tasks.rados.rados.0.trial115.stdout:3883: oid 47 updating version 1702 to 1704 2026-02-17T06:10:01.889 INFO:tasks.rados.rados.0.trial115.stdout:3883: oid 47 version 1704 is already newer than 1703 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1704 (ObjNum 1326 snap 410 seq_num 1326) dirty exists 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3883: left oid 47 (ObjNum 1326 snap 410 seq_num 1326) 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3884: finishing copy_from to trial11545420-44 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3884: got expected ENOENT (src dne) 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3884: finishing copy_from racing read to trial11545420-44 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3885: finishing write tid 1 to trial11545420-10 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3883: done (4 left) 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3884: done (3 left) 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3887: done (2 left) 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3888: read oid 2 snap -1 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3888: expect (ObjNum 1302 snap 402 seq_num 1302) 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3889: write oid 14 current snap is 410 2026-02-17T06:10:01.890 INFO:tasks.rados.rados.0.trial115.stdout:3889: seq_num 1328 ranges {719696=576094,2094793=224903} 2026-02-17T06:10:01.894 INFO:tasks.rados.rados.0.trial115.stdout:3889: writing trial11545420-14 from 719696 to 1295790 tid 1 2026-02-17T06:10:01.898 INFO:tasks.rados.rados.0.trial115.stdout:3889: writing trial11545420-14 from 2094793 to 2319696 tid 2 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3890: snap_create 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3885: finishing write tid 2 to trial11545420-10 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3885: finishing write tid 3 to trial11545420-10 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3885: finishing write tid 4 to trial11545420-10 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3885: oid 10 updating version 0 to 2042 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3885: oid 10 version 2042 is already newer than 2041 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3885: oid 10 updating version 2042 to 2043 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 2043 (ObjNum 1327 snap 410 seq_num 1327) dirty exists 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3885: left oid 10 (ObjNum 1327 snap 410 seq_num 1327) 2026-02-17T06:10:01.899 INFO:tasks.rados.rados.0.trial115.stdout:3888: expect (ObjNum 1302 snap 402 seq_num 1302) 2026-02-17T06:10:02.020 INFO:tasks.rados.rados.0.trial115.stdout:3889: finishing write tid 1 to trial11545420-14 2026-02-17T06:10:02.021 INFO:tasks.rados.rados.0.trial115.stdout:3889: finishing write tid 2 to trial11545420-14 2026-02-17T06:10:02.021 INFO:tasks.rados.rados.0.trial115.stdout:3889: finishing write tid 3 to trial11545420-14 2026-02-17T06:10:02.021 INFO:tasks.rados.rados.0.trial115.stdout:3889: finishing write tid 4 to trial11545420-14 2026-02-17T06:10:02.021 INFO:tasks.rados.rados.0.trial115.stdout:3889: oid 14 updating version 0 to 1708 2026-02-17T06:10:02.021 INFO:tasks.rados.rados.0.trial115.stdout:3889: oid 14 version 1708 is already newer than 1707 2026-02-17T06:10:02.021 INFO:tasks.rados.rados.0.trial115.stdout:3889: oid 14 version 1708 is already newer than 1706 2026-02-17T06:10:02.021 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1708 (ObjNum 1328 snap 410 seq_num 1328) dirty exists 2026-02-17T06:10:02.021 INFO:tasks.rados.rados.0.trial115.stdout:3889: left oid 14 (ObjNum 1328 snap 410 seq_num 1328) 2026-02-17T06:10:02.442 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient: get_monmap_and_config 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient: build_initial_monmap 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient: monmap: 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:02.510465+0000 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:02.510465+0000 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:02.510 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 1 Processor -- start 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 1 -- start start 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient: init 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff4cc06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff4cc09c4a0 0x7ff4cc13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient(hunting): picked mon.noname-a con 0x7ff4cc09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff4cc06d380 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4cc06daf0 con 0x7ff4cc06d380 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4cc09cc80 con 0x7ff4cc06c890 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff4cc0a0880 con 0x7ff4cc09c4a0 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.509+0000 7ff4d32de640 10 monclient(hunting): authenticate will time out at 2048.063994s 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff4cc09c4a0 0x7ff4cc13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff4cc09c4a0 0x7ff4cc13b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42600/0 (socket says 10.20.193.78:42600) 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d1854640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff4cc06d380 0x7ff4cc13d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52466/0 (socket says 10.20.193.78:52466) 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 1 -- 10.20.193.78:0/1161777628 learned_addr learned my addr 10.20.193.78:0/1161777628 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient(hunting): get_auth_request con 0x7ff4cc09c4a0 auth_method 0 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:02.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient(hunting): handle_auth_done global_id 284199 payload 274 2026-02-17T06:10:02.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 1 -- 10.20.193.78:0/1161777628 >> v1:10.20.193.115:6789/0 conn(0x7ff4cc06c890 legacy=0x7ff4cc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:02.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 1 -- 10.20.193.78:0/1161777628 >> v1:10.20.193.78:6789/0 conn(0x7ff4cc06d380 legacy=0x7ff4cc13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:02.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient: _finish_hunting 0 2026-02-17T06:10:02.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 1 monclient: found mon.noname-a 2026-02-17T06:10:02.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:02.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:02.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.510+0000 7ff4d0852640 1 -- 10.20.193.78:0/1161777628 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc09cf60 con 0x7ff4cc09c4a0 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4d0852640 1 -- 10.20.193.78:0/1161777628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc09c4a0 msgr2=0x7ff4cc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4d0852640 1 --2- 10.20.193.78:0/1161777628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc09c4a0 0x7ff4cc13b3a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc09c980 tx=0x7ff4c002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a2234742ee91b2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 1 -- 10.20.193.78:0/1161777628 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c003c070 con 0x7ff4cc09c4a0 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: dump: 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:02.513 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: _finish_auth 0 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: _check_auth_tickets 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 1 -- 10.20.193.78:0/1161777628 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c002fd80 con 0x7ff4cc09c4a0 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 1 -- 10.20.193.78:0/1161777628 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0037660 con 0x7ff4cc09c4a0 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4d32de640 5 monclient: authenticate success, global_id 284199 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4d32de640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4d32de640 4 set_mon_vals no callback set 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.512+0000 7ff4bbfff640 10 monclient: dump: 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 10 monclient: shutdown 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 -- 10.20.193.78:0/1161777628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc09c4a0 msgr2=0x7ff4cc13b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 --2- 10.20.193.78:0/1161777628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc09c4a0 0x7ff4cc13b3a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc09c980 tx=0x7ff4c002ee60 comp rx=0 tx=0).stop 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 -- 10.20.193.78:0/1161777628 shutdown_connections 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 --2- 10.20.193.78:0/1161777628 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc09c4a0 0x7ff4cc13b3a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:02.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 -- 10.20.193.78:0/1161777628 >> 10.20.193.78:0/1161777628 conn(0x7ff4cc0a6620 msgr2=0x7ff4cc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 -- 10.20.193.78:0/1161777628 shutdown_connections 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 -- 10.20.193.78:0/1161777628 wait complete. 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 10 monclient: build_initial_monmap 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 10 monclient: monmap: 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 Processor -- start 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 1 -- start start 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.513+0000 7ff4d32de640 10 monclient: init 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:10:02.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4cc06c890 0x7ff4cc13d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient(hunting): picked mon.noname-c con 0x7ff4cc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc06d380 0x7ff4cc1362c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient(hunting): picked mon.noname-a con 0x7ff4cc06d380 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4cc09c4a0 0x7ff4cc1cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient(hunting): picked mon.noname-b con 0x7ff4cc09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc09bfa0 con 0x7ff4cc06d380 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc09be20 con 0x7ff4cc09c4a0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc13f010 con 0x7ff4cc06c890 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d32de640 10 monclient(hunting): authenticate will time out at 2048.068103s 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4cc06c890 0x7ff4cc13d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc06d380 0x7ff4cc1362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc06d380 0x7ff4cc1362c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42616/0 (socket says 10.20.193.78:42616) 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 -- 10.20.193.78:0/1314824605 learned_addr learned my addr 10.20.193.78:0/1314824605 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4cc06c890 0x7ff4cc13d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:39248/0 (socket says 10.20.193.78:39248) 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient(hunting): get_auth_request con 0x7ff4cc06d380 auth_method 0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 10 monclient(hunting): get_auth_request con 0x7ff4cc06c890 auth_method 0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1053640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient(hunting): handle_auth_done global_id 284202 payload 824 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 -- 10.20.193.78:0/1314824605 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4cc06c890 msgr2=0x7ff4cc13d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4cc06c890 0x7ff4cc13d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 -- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4cc09c4a0 msgr2=0x7ff4cc1cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1854640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4cc09c4a0 0x7ff4cc1cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4cc09c4a0 0x7ff4cc1cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient: _finish_hunting 0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 monclient: found mon.noname-a 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d0852640 1 -- 10.20.193.78:0/1314824605 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc11e810 con 0x7ff4cc06d380 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1854640 10 monclient: get_auth_request con 0x7ff4cc09c4a0 auth_method 0 2026-02-17T06:10:02.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.514+0000 7ff4d1854640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4cc09c4a0 0x7ff4cc1cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d0852640 1 -- 10.20.193.78:0/1314824605 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc06d380 msgr2=0x7ff4cc1362c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d0852640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc06d380 0x7ff4cc1362c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0002410 tx=0x7ff4c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 1 -- 10.20.193.78:0/1314824605 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0046020 con 0x7ff4cc06d380 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: dump: 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 1 -- 10.20.193.78:0/1314824605 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c0042640 con 0x7ff4cc06d380 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 1 -- 10.20.193.78:0/1314824605 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c003c040 con 0x7ff4cc06d380 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d32de640 5 monclient: authenticate success, global_id 284202 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d32de640 10 monclient: _renew_subs 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4bb7fe640 4 set_mon_vals no callback set 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d32de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc11fc60 con 0x7ff4cc06d380 2026-02-17T06:10:02.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4b9ffb640 10 monclient: dump: 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d32de640 10 monclient: _renew_subs 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d32de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc09d810 con 0x7ff4cc06d380 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.516+0000 7ff4d32de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.517+0000 7ff4d32de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.517+0000 7ff4d32de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.517+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc06cc70 con 0x7ff4cc06d380 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.517+0000 7ff4b9ffb640 1 -- 10.20.193.78:0/1314824605 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff4c003f030 con 0x7ff4cc06d380 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.517+0000 7ff4b9ffb640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49c03df00 0x7ff49c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.517+0000 7ff4b9ffb640 1 -- 10.20.193.78:0/1314824605 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(850..850 src has 525..850) ==== 4779+0+0 (secure 0 0 0) 0x7ff4c0077700 con 0x7ff4cc06d380 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.517+0000 7ff4d1053640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49c03df00 0x7ff49c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:02.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.517+0000 7ff4d1053640 10 monclient: get_auth_request con 0x7ff49c03df00 auth_method 0 2026-02-17T06:10:02.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.519+0000 7ff4b9ffb640 1 -- 10.20.193.78:0/1314824605 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff4c0050070 con 0x7ff4cc06d380 2026-02-17T06:10:02.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.519+0000 7ff4b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:02.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.519+0000 7ff4b9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:02.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.521+0000 7ff4d1053640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49c03df00 0x7ff49c0403c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc005160 tx=0x7ff4bc007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:02.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.632+0000 7ff4d32de640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:10:02.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.632+0000 7ff4d32de640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:10:02.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.632+0000 7ff4d32de640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:02.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.632+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff4cc0744d0 con 0x7ff4cc06d380 2026-02-17T06:10:02.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.632+0000 7ff4b9ffb640 1 -- 10.20.193.78:0/1314824605 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7ff4c002fd20 con 0x7ff4cc06d380 2026-02-17T06:10:02.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.632+0000 7ff4b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:10:02.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.632+0000 7ff4b9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:02.634 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":262,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:10:02.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49c03df00 msgr2=0x7ff49c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:02.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49c03df00 0x7ff49c0403c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc005160 tx=0x7ff4bc007f30 comp rx=0 tx=0).stop 2026-02-17T06:10:02.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 10 monclient: shutdown 2026-02-17T06:10:02.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:02.635 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc06d380 msgr2=0x7ff4cc1362c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc06d380 0x7ff4cc1362c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0002410 tx=0x7ff4c00047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 shutdown_connections 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff49c03df00 0x7ff49c0403c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff4cc09c4a0 0x7ff4cc1cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff4cc06d380 0x7ff4cc1362c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 --2- 10.20.193.78:0/1314824605 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff4cc06c890 0x7ff4cc13d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 >> 10.20.193.78:0/1314824605 conn(0x7ff4cc0a6620 msgr2=0x7ff4cc0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 shutdown_connections 2026-02-17T06:10:02.636 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:02.633+0000 7ff4d32de640 1 -- 10.20.193.78:0/1314824605 wait complete. 2026-02-17T06:10:03.331 INFO:tasks.rados.rados.0.trial115.stdout:3885: done (4 left) 2026-02-17T06:10:03.331 INFO:tasks.rados.rados.0.trial115.stdout:3886: done (3 left) 2026-02-17T06:10:03.331 INFO:tasks.rados.rados.0.trial115.stdout:3888: done (2 left) 2026-02-17T06:10:03.331 INFO:tasks.rados.rados.0.trial115.stdout:3889: done (1 left) 2026-02-17T06:10:03.331 INFO:tasks.rados.rados.0.trial115.stdout:3890: done (0 left) 2026-02-17T06:10:03.331 INFO:tasks.rados.rados.0.trial115.stdout:3891: copy_from oid 26 from oid 36 current snap is 411 2026-02-17T06:10:03.331 INFO:tasks.rados.rados.0.trial115.stdout:3892: write (excl) oid 11 current snap is 411 2026-02-17T06:10:03.331 INFO:tasks.rados.rados.0.trial115.stdout:3892: seq_num 1329 ranges {736503=651343,2153275=183228} 2026-02-17T06:10:03.337 INFO:tasks.rados.rados.0.trial115.stdout:3892: writing trial11545420-11 from 736503 to 1387846 tid 1 2026-02-17T06:10:03.340 INFO:tasks.rados.rados.0.trial115.stdout:3892: writing trial11545420-11 from 2153275 to 2336503 tid 2 2026-02-17T06:10:03.340 INFO:tasks.rados.rados.0.trial115.stdout:3893: read oid 17 snap -1 2026-02-17T06:10:03.340 INFO:tasks.rados.rados.0.trial115.stdout:3893: expect (ObjNum 1266 snap 397 seq_num 1266) 2026-02-17T06:10:03.340 INFO:tasks.rados.rados.0.trial115.stdout:3894: snap_create 2026-02-17T06:10:03.340 INFO:tasks.rados.rados.0.trial115.stdout:3891: finishing copy_from racing read to trial11545420-26 2026-02-17T06:10:03.340 INFO:tasks.rados.rados.0.trial115.stdout:3891: finishing copy_from to trial11545420-26 2026-02-17T06:10:03.341 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 26 v 1705 (ObjNum 1310 snap 404 seq_num 1310) dirty exists 2026-02-17T06:10:03.342 INFO:tasks.rados.rados.0.trial115.stdout:3892: finishing write tid 1 to trial11545420-11 2026-02-17T06:10:03.342 INFO:tasks.rados.rados.0.trial115.stdout:3892: finishing write tid 2 to trial11545420-11 2026-02-17T06:10:03.342 INFO:tasks.rados.rados.0.trial115.stdout:3892: finishing write tid 3 to trial11545420-11 2026-02-17T06:10:03.342 INFO:tasks.rados.rados.0.trial115.stdout:3892: finishing write tid 4 to trial11545420-11 2026-02-17T06:10:03.342 INFO:tasks.rados.rados.0.trial115.stdout:3892: oid 11 updating version 0 to 2050 2026-02-17T06:10:03.342 INFO:tasks.rados.rados.0.trial115.stdout:3892: oid 11 version 2050 is already newer than 2049 2026-02-17T06:10:03.343 INFO:tasks.rados.rados.0.trial115.stdout:3892: oid 11 updating version 2050 to 2051 2026-02-17T06:10:03.343 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 11 v 2051 (ObjNum 1329 snap 411 seq_num 1329) dirty exists 2026-02-17T06:10:03.343 INFO:tasks.rados.rados.0.trial115.stdout:3892: left oid 11 (ObjNum 1329 snap 411 seq_num 1329) 2026-02-17T06:10:03.344 INFO:tasks.rados.rados.0.trial115.stdout:3893: expect (ObjNum 1266 snap 397 seq_num 1266) 2026-02-17T06:10:03.874 INFO:tasks.rados.rados.0.trial115.stdout:3891: done (3 left) 2026-02-17T06:10:03.874 INFO:tasks.rados.rados.0.trial115.stdout:3892: done (2 left) 2026-02-17T06:10:03.874 INFO:tasks.rados.rados.0.trial115.stdout:3893: done (1 left) 2026-02-17T06:10:03.874 INFO:tasks.rados.rados.0.trial115.stdout:3894: done (0 left) 2026-02-17T06:10:03.874 INFO:tasks.rados.rados.0.trial115.stdout:3895: copy_from oid 21 from oid 45 current snap is 412 2026-02-17T06:10:03.875 INFO:tasks.rados.rados.0.trial115.stdout:3896: read oid 47 snap -1 2026-02-17T06:10:03.875 INFO:tasks.rados.rados.0.trial115.stdout:3896: expect (ObjNum 1326 snap 410 seq_num 1326) 2026-02-17T06:10:03.875 INFO:tasks.rados.rados.0.trial115.stdout:3897: delete oid 18 current snap is 412 2026-02-17T06:10:03.880 INFO:tasks.rados.rados.0.trial115.stdout:3897: done (2 left) 2026-02-17T06:10:03.880 INFO:tasks.rados.rados.0.trial115.stdout:3898: read oid 12 snap -1 2026-02-17T06:10:03.880 INFO:tasks.rados.rados.0.trial115.stdout:3898: expect (ObjNum 1300 snap 401 seq_num 1300) 2026-02-17T06:10:03.880 INFO:tasks.rados.rados.0.trial115.stdout:3899: read oid 38 snap -1 2026-02-17T06:10:03.880 INFO:tasks.rados.rados.0.trial115.stdout:3899: expect (ObjNum 1305 snap 403 seq_num 1305) 2026-02-17T06:10:03.881 INFO:tasks.rados.rados.0.trial115.stdout:3900: read oid 17 snap 410 2026-02-17T06:10:03.881 INFO:tasks.rados.rados.0.trial115.stdout:3900: expect (ObjNum 1266 snap 397 seq_num 1266) 2026-02-17T06:10:03.881 INFO:tasks.rados.rados.0.trial115.stdout:3901: read oid 30 snap 411 2026-02-17T06:10:03.881 INFO:tasks.rados.rados.0.trial115.stdout:3901: expect (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:10:03.881 INFO:tasks.rados.rados.0.trial115.stdout:3902: snap_create 2026-02-17T06:10:03.881 INFO:tasks.rados.rados.0.trial115.stdout:3896: expect (ObjNum 1326 snap 410 seq_num 1326) 2026-02-17T06:10:03.958 INFO:tasks.rados.rados.0.trial115.stdout:3898: expect (ObjNum 1300 snap 401 seq_num 1300) 2026-02-17T06:10:04.048 INFO:tasks.rados.rados.0.trial115.stdout:3901: expect (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:10:04.253 INFO:tasks.rados.rados.0.trial115.stdout:3900: expect (ObjNum 1266 snap 397 seq_num 1266) 2026-02-17T06:10:04.330 INFO:tasks.rados.rados.0.trial115.stdout:3899: expect (ObjNum 1305 snap 403 seq_num 1305) 2026-02-17T06:10:04.540 INFO:tasks.rados.rados.0.trial115.stdout:3895: finishing copy_from racing read to trial11545420-21 2026-02-17T06:10:04.541 INFO:tasks.rados.rados.0.trial115.stdout:3895: finishing copy_from to trial11545420-21 2026-02-17T06:10:04.541 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 2003 (ObjNum 1279 snap 398 seq_num 1279) dirty exists 2026-02-17T06:10:05.335 INFO:tasks.rados.rados.0.trial115.stdout:3895: done (6 left) 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:3896: done (5 left) 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:3898: done (4 left) 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:3899: done (3 left) 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:3900: done (2 left) 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:3901: done (1 left) 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:3902: done (0 left) 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:3903: rollback oid 23 current snap is 413 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 23 to 412 2026-02-17T06:10:05.336 INFO:tasks.rados.rados.0.trial115.stdout:3904: delete oid 9 current snap is 413 2026-02-17T06:10:05.338 INFO:tasks.rados.rados.0.trial115.stdout:3903: finishing rollback tid 0 to trial11545420-23 2026-02-17T06:10:05.338 INFO:tasks.rados.rados.0.trial115.stdout:3903: finishing rollback tid 1 to trial11545420-23 2026-02-17T06:10:05.338 INFO:tasks.rados.rados.0.trial115.stdout:3903: finishing rollback tid 2 to trial11545420-23 2026-02-17T06:10:05.338 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1823 (ObjNum 1289 snap 400 seq_num 1289) dirty exists 2026-02-17T06:10:05.339 INFO:tasks.rados.rados.0.trial115.stdout:3903: done (1 left) 2026-02-17T06:10:05.339 INFO:tasks.rados.rados.0.trial115.stdout:3904: done (0 left) 2026-02-17T06:10:05.339 INFO:tasks.rados.rados.0.trial115.stdout:3905: rollback oid 8 current snap is 413 2026-02-17T06:10:05.339 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 8 to 412 2026-02-17T06:10:05.340 INFO:tasks.rados.rados.0.trial115.stdout:3906: rollback oid 25 current snap is 413 2026-02-17T06:10:05.340 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 406 2026-02-17T06:10:05.340 INFO:tasks.rados.rados.0.trial115.stdout:3907: delete oid 47 current snap is 413 2026-02-17T06:10:05.340 INFO:tasks.rados.rados.0.trial115.stdout:3905: finishing rollback tid 1 to trial11545420-8 2026-02-17T06:10:05.340 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 0 (ObjNum 33 snap 0 seq_num 1330677968) dirty dne 2026-02-17T06:10:05.341 INFO:tasks.rados.rados.0.trial115.stdout:3905: done (2 left) 2026-02-17T06:10:05.341 INFO:tasks.rados.rados.0.trial115.stdout:3907: done (1 left) 2026-02-17T06:10:05.341 INFO:tasks.rados.rados.0.trial115.stdout:3908: read oid 39 snap -1 2026-02-17T06:10:05.341 INFO:tasks.rados.rados.0.trial115.stdout:3908: expect (ObjNum 1272 snap 398 seq_num 1272) 2026-02-17T06:10:05.341 INFO:tasks.rados.rados.0.trial115.stdout:3909: snap_create 2026-02-17T06:10:05.342 INFO:tasks.rados.rados.0.trial115.stdout:3906: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:10:05.342 INFO:tasks.rados.rados.0.trial115.stdout:3906: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:10:05.342 INFO:tasks.rados.rados.0.trial115.stdout:3906: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:10:05.342 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 2006 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2026-02-17T06:10:05.345 INFO:tasks.rados.rados.0.trial115.stdout:3908: expect (ObjNum 1272 snap 398 seq_num 1272) 2026-02-17T06:10:05.642 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:10:05.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.708+0000 7fc836c5b640 10 monclient: get_monmap_and_config 2026-02-17T06:10:05.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.708+0000 7fc836c5b640 10 monclient: build_initial_monmap 2026-02-17T06:10:05.711 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.708+0000 7fc836c5b640 10 monclient: monmap: 2026-02-17T06:10:05.711 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:05.711 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.711 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:05.709579+0000 2026-02-17T06:10:05.711 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:05.709579+0000 2026-02-17T06:10:05.711 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:05.711 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.708+0000 7fc836c5b640 1 Processor -- start 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.708+0000 7fc836c5b640 1 -- start start 2026-02-17T06:10:05.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.708+0000 7fc836c5b640 10 monclient: init 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fc83009ddd0 0x7fc83009e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 10 monclient(hunting): picked mon.noname-c con 0x7fc83009ddd0 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc830073f60 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc830138660 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8300a2850 con 0x7fc830073f60 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8300a2560 con 0x7fc830138660 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fc8300a3810 con 0x7fc83009ddd0 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:05.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc836c5b640 10 monclient(hunting): authenticate will time out at 2051.263157s 2026-02-17T06:10:05.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc827fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fc830073f60 0x7fc830137f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52478/0 (socket says 10.20.193.78:52478) 2026-02-17T06:10:05.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc827fff640 1 -- 10.20.193.78:0/1987374923 learned_addr learned my addr 10.20.193.78:0/1987374923 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:10:05.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.709+0000 7fc8349d0640 1 --2- 10.20.193.78:0/1987374923 >> v2:10.20.193.115:3300/0 conn(0x7fc83009ddd0 0x7fc83009e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc827fff640 1 -- 10.20.193.78:0/1987374923 >> v1:10.20.193.78:6789/0 conn(0x7fc830073f60 legacy=0x7fc830137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:05.724 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8349d0640 10 monclient(hunting): get_auth_request con 0x7fc83009ddd0 auth_method 0 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8351d1640 1 -- 10.20.193.78:0/1987374923 >> v1:10.20.193.115:6789/0 conn(0x7fc830138660 legacy=0x7fc830140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8349d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8349d0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8349d0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8349d0640 1 -- 10.20.193.78:0/1987374923 >> v2:10.20.193.115:3300/0 conn(0x7fc83009ddd0 msgr2=0x7fc83009e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8349d0640 1 -- 10.20.193.78:0/1987374923 >> v2:10.20.193.115:3300/0 conn(0x7fc83009ddd0 msgr2=0x7fc83009e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8349d0640 1 --2- 10.20.193.78:0/1987374923 >> v2:10.20.193.115:3300/0 conn(0x7fc83009ddd0 0x7fc83009e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.711+0000 7fc8349d0640 1 --2- 10.20.193.78:0/1987374923 >> v2:10.20.193.115:3300/0 conn(0x7fc83009ddd0 0x7fc83009e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.713+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1250224386 0 0) 0x7fc8300a2850 con 0x7fc830073f60 2026-02-17T06:10:05.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.713+0000 7fc8277fe640 10 monclient(hunting): _negotiate 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.713+0000 7fc8277fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.713+0000 7fc8277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.713+0000 7fc8277fe640 10 monclient(hunting): my global_id is 284205 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.713+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc81c003620 con 0x7fc830073f60 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 <== mon.0 v1:10.20.193.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216221269 0 0) 0x7fc81c003620 con 0x7fc830073f60 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 >> v2:10.20.193.115:3300/0 conn(0x7fc83009ddd0 msgr2=0x7fc83009e1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 --2- 10.20.193.78:0/1987374923 >> v2:10.20.193.115:3300/0 conn(0x7fc83009ddd0 0x7fc83009e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 >> v1:10.20.193.115:6789/0 conn(0x7fc830138660 legacy=0x7fc830140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: _finish_hunting 0 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 monclient: found mon.noname-a-legacy 2026-02-17T06:10:05.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.78:6789/0 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 --> v1:10.20.193.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8300a3af0 con 0x7fc830073f60 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: _finish_auth 0 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: _check_auth_tickets 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 <== mon.0 v1:10.20.193.78:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 684720633 0 0) 0x7fc818003270 con 0x7fc830073f60 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: dump: 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.727 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] but i'm connected to v1:10.20.193.78:6789/0, reconnecting 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 >> v1:10.20.193.78:6789/0 conn(0x7fc830073f60 legacy=0x7fc830137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc81c003b50 0x7fc81c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient(hunting): picked mon.b con 0x7fc81c003b50 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc81c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient(hunting): picked mon.a con 0x7fc81c004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc81c0054a0 0x7fc81c00bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient(hunting): picked mon.c con 0x7fc81c0054a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8300a3810 con 0x7fc81c004900 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c00c3e0 con 0x7fc81c003b50 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc81c00c5e0 con 0x7fc81c0054a0 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:05.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8277fe640 10 monclient(hunting): _finish_auth 0 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc81c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient(hunting): get_auth_request con 0x7fc81c004900 auth_method 0 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc81c0054a0 0x7fc81c00bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 10 monclient(hunting): get_auth_request con 0x7fc81c0054a0 auth_method 0 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:05.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient(hunting): handle_auth_done global_id 284205 payload 293 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 1 -- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc81c0054a0 msgr2=0x7fc81c00bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc81c0054a0 0x7fc81c00bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 1 -- 10.20.193.78:0/1987374923 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc81c003b50 msgr2=0x7fc81c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc81c003b50 0x7fc81c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient: _finish_hunting 0 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 1 monclient: found mon.a 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 1 -- 10.20.193.78:0/1987374923 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c003620 con 0x7fc81c004900 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient: _finish_auth 0 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 10 monclient: _check_auth_tickets 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc827fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc836c5b640 5 monclient: authenticate success, global_id 284205 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc836c5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:05.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.714+0000 7fc8351d1640 1 -- 10.20.193.78:0/1987374923 reap_dead start 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.716+0000 7fc827fff640 1 -- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 msgr2=0x7fc81c004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.716+0000 7fc827fff640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc81c004d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc830074380 tx=0x7fc81802f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e638f566b20e216a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.716+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81800c220 con 0x7fc81c004900 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.716+0000 7fc8277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.716+0000 7fc8277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.716+0000 7fc8277fe640 10 monclient: dump: 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.733 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc818005cb0 con 0x7fc81c004900 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc8277fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc8277fe640 1 -- 10.20.193.78:0/1987374923 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818005e60 con 0x7fc81c004900 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc8277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc8277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc8277fe640 10 monclient: dump: 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:05.735 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 4 set_mon_vals no callback set 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 10 monclient: shutdown 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 -- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 msgr2=0x7fc81c004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc81c004d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc830074380 tx=0x7fc81802f7b0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 -- 10.20.193.78:0/1987374923 shutdown_connections 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 --2- 10.20.193.78:0/1987374923 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc81c004d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 -- 10.20.193.78:0/1987374923 >> 10.20.193.78:0/1987374923 conn(0x7fc8300aacd0 msgr2=0x7fc8300ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 -- 10.20.193.78:0/1987374923 shutdown_connections 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 -- 10.20.193.78:0/1987374923 wait complete. 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 10 monclient: build_initial_monmap 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 10 monclient: monmap: 2026-02-17T06:10:05.736 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 Processor -- start 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 1 -- start start 2026-02-17T06:10:05.737 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.717+0000 7fc836c5b640 10 monclient: init 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc8300735a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient(hunting): picked mon.noname-a con 0x7fc81c004900 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc83012fb50 0x7fc83012ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient(hunting): picked mon.noname-b con 0x7fc83012fb50 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc830130470 0x7fc8301d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient(hunting): picked mon.noname-c con 0x7fc830130470 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8300a4840 con 0x7fc81c004900 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.738 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8300a57e0 con 0x7fc83012fb50 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fc8300a49c0 con 0x7fc830130470 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc836c5b640 10 monclient(hunting): authenticate will time out at 2051.272183s 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc8300735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc8300735a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42634/0 (socket says 10.20.193.78:42634) 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc830130470 0x7fc8301d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 1 -- 10.20.193.78:0/3134662832 learned_addr learned my addr 10.20.193.78:0/3134662832 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 10 monclient(hunting): get_auth_request con 0x7fc81c004900 auth_method 0 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.739 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient(hunting): get_auth_request con 0x7fc830130470 auth_method 0 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8349d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc827fff640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc83012fb50 0x7fc83012ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient(hunting): handle_auth_done global_id 264404 payload 824 2026-02-17T06:10:05.740 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 1 -- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc83012fb50 msgr2=0x7fc83012ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc83012fb50 0x7fc83012ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 1 -- 10.20.193.78:0/3134662832 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 msgr2=0x7fc8300735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc8300735a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient: _finish_hunting 0 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 1 monclient: found mon.noname-c 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc8351d1640 1 -- 10.20.193.78:0/3134662832 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8300aa3c0 con 0x7fc830130470 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc827fff640 10 monclient: get_auth_request con 0x7fc83012fb50 auth_method 0 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.718+0000 7fc827fff640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc83012fb50 0x7fc83012ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8351d1640 1 -- 10.20.193.78:0/3134662832 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc830130470 msgr2=0x7fc8301d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:05.741 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8351d1640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc830130470 0x7fc8301d5dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc83006df60 tx=0x7fc82c00d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 1 -- 10.20.193.78:0/3134662832 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0070d0 con 0x7fc830130470 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: dump: 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.742 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: _finish_auth 0 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: _check_auth_tickets 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 1 -- 10.20.193.78:0/3134662832 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c018070 con 0x7fc830130470 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc836c5b640 5 monclient: authenticate success, global_id 264404 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc836c5b640 10 monclient: _renew_subs 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc836c5b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8300a1c80 con 0x7fc830130470 2026-02-17T06:10:05.743 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 1 -- 10.20.193.78:0/3134662832 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0074f0 con 0x7fc830130470 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc826ffd640 4 set_mon_vals no callback set 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc8257fa640 10 monclient: dump: 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.744 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc836c5b640 10 monclient: _renew_subs 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc836c5b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.720+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8300a11b0 con 0x7fc830130470 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc8257fa640 1 -- 10.20.193.78:0/3134662832 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fc82c007690 con 0x7fc830130470 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc836c5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc836c5b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc836c5b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8300a6880 con 0x7fc830130470 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc8257fa640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc80003df00 0x7fc8000403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc8257fa640 1 -- 10.20.193.78:0/3134662832 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(853..853 src has 525..853) ==== 4735+0+0 (secure 0 0 0) 0x7fc82c052d00 con 0x7fc830130470 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc8349d0640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc80003df00 0x7fc8000403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.721+0000 7fc8349d0640 10 monclient: get_auth_request con 0x7fc80003df00 auth_method 0 2026-02-17T06:10:05.745 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.723+0000 7fc8257fa640 1 -- 10.20.193.78:0/3134662832 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc8300a6880 con 0x7fc830130470 2026-02-17T06:10:05.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.723+0000 7fc8257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:05.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.723+0000 7fc8257fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:05.746 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.724+0000 7fc8349d0640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc80003df00 0x7fc8000403c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc820006b90 tx=0x7fc820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:05.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.836+0000 7fc836c5b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:10:05.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.836+0000 7fc836c5b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:10:05.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.836+0000 7fc836c5b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:05.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.836+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc830137020 con 0x7fc830130470 2026-02-17T06:10:05.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.836+0000 7fc8257fa640 1 -- 10.20.193.78:0/3134662832 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fc830137020 con 0x7fc830130470 2026-02-17T06:10:05.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.836+0000 7fc8257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:10:05.838 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.836+0000 7fc8257fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:05.839 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":264,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc80003df00 msgr2=0x7fc8000403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc80003df00 0x7fc8000403c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc820006b90 tx=0x7fc820008040 comp rx=0 tx=0).stop 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 10 monclient: shutdown 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc830130470 msgr2=0x7fc8301d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc830130470 0x7fc8301d5dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc83006df60 tx=0x7fc82c00d520 comp rx=0 tx=0).stop 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 shutdown_connections 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fc830130470 0x7fc8301d5dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fc80003df00 0x7fc8000403c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fc83012fb50 0x7fc83012ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.840 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 --2- 10.20.193.78:0/3134662832 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fc81c004900 0x7fc8300735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 >> 10.20.193.78:0/3134662832 conn(0x7fc8300aacd0 msgr2=0x7fc8300a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:05.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 shutdown_connections 2026-02-17T06:10:05.841 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.837+0000 7fc836c5b640 1 -- 10.20.193.78:0/3134662832 wait complete. 2026-02-17T06:10:05.846 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:10:05.846 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:10:05.847 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:10:05.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.913+0000 7f50c3785640 10 monclient: get_monmap_and_config 2026-02-17T06:10:05.916 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.913+0000 7f50c3785640 10 monclient: build_initial_monmap 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.913+0000 7f50c3785640 10 monclient: monmap: 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:05.914853+0000 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:05.914853+0000 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:05.917 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.913+0000 7f50c3785640 1 Processor -- start 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 1 -- start start 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient: init 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f50bc06c890 0x7f50bc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient(hunting): picked mon.noname-c con 0x7f50bc06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f50bc09c4a0 0x7f50bc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.918 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient(hunting): picked mon.noname-a con 0x7f50bc09c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f50bc134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50bc06daf0 con 0x7f50bc134df0 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f50bc0a0880 con 0x7f50bc09c4a0 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f50bc06d6f0 con 0x7f50bc06c890 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c3785640 10 monclient(hunting): authenticate will time out at 2051.468370s 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f50bc09c4a0 0x7f50bc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f50bc09c4a0 0x7f50bc06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42648/0 (socket says 10.20.193.78:42648) 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 1 -- 10.20.193.78:0/570729082 learned_addr learned my addr 10.20.193.78:0/570729082 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient(hunting): get_auth_request con 0x7f50bc09c4a0 auth_method 0 2026-02-17T06:10:05.919 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c14fa640 1 --2- 10.20.193.78:0/570729082 >> v2:10.20.193.115:3300/0 conn(0x7f50bc06c890 0x7f50bc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient(hunting): handle_auth_done global_id 284211 payload 274 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 1 -- 10.20.193.78:0/570729082 >> v2:10.20.193.115:3300/0 conn(0x7f50bc06c890 msgr2=0x7f50bc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 1 --2- 10.20.193.78:0/570729082 >> v2:10.20.193.115:3300/0 conn(0x7f50bc06c890 0x7f50bc06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 1 -- 10.20.193.78:0/570729082 >> v1:10.20.193.115:6789/0 conn(0x7f50bc134df0 legacy=0x7f50bc135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient: _finish_hunting 0 2026-02-17T06:10:05.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 1 monclient: found mon.noname-a 2026-02-17T06:10:05.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:05.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:05.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.914+0000 7f50c0cf9640 1 -- 10.20.193.78:0/570729082 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc09cc80 con 0x7f50bc09c4a0 2026-02-17T06:10:05.929 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c0cf9640 1 -- 10.20.193.78:0/570729082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 msgr2=0x7f50bc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c0cf9640 1 --2- 10.20.193.78:0/570729082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 0x7f50bc06d1b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50bc06a480 tx=0x7f50b002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9c8f58029e35af server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 1 -- 10.20.193.78:0/570729082 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b003c070 con 0x7f50bc09c4a0 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: dump: 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.930 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.931 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.931 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.931 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.931 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: _finish_auth 0 2026-02-17T06:10:05.931 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: _check_auth_tickets 2026-02-17T06:10:05.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:05.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 1 -- 10.20.193.78:0/570729082 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b002fd80 con 0x7f50bc09c4a0 2026-02-17T06:10:05.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 5 monclient: authenticate success, global_id 284211 2026-02-17T06:10:05.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:05.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:05.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 1 -- 10.20.193.78:0/570729082 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0037660 con 0x7f50bc09c4a0 2026-02-17T06:10:05.932 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 4 set_mon_vals no callback set 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50abfff640 10 monclient: dump: 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.933 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.934 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.934 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.934 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 10 monclient: shutdown 2026-02-17T06:10:05.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:05.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 -- 10.20.193.78:0/570729082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 msgr2=0x7f50bc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.934 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 --2- 10.20.193.78:0/570729082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 0x7f50bc06d1b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50bc06a480 tx=0x7f50b002ee60 comp rx=0 tx=0).stop 2026-02-17T06:10:05.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 -- 10.20.193.78:0/570729082 shutdown_connections 2026-02-17T06:10:05.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 --2- 10.20.193.78:0/570729082 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 0x7f50bc06d1b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 --2- 10.20.193.78:0/570729082 >> v2:10.20.193.115:3300/0 conn(0x7f50bc06c890 0x7f50bc06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 -- 10.20.193.78:0/570729082 >> 10.20.193.78:0/570729082 conn(0x7f50bc0a6620 msgr2=0x7f50bc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:05.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 -- 10.20.193.78:0/570729082 shutdown_connections 2026-02-17T06:10:05.935 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 -- 10.20.193.78:0/570729082 wait complete. 2026-02-17T06:10:05.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 10 monclient: build_initial_monmap 2026-02-17T06:10:05.936 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 10 monclient: monmap: 2026-02-17T06:10:05.936 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:05.936 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:05.936 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:05.936 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:05.937 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:05.937 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.937 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:05.937 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:05.937 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:05.937 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 Processor -- start 2026-02-17T06:10:05.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 1 -- start start 2026-02-17T06:10:05.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.917+0000 7f50c3785640 10 monclient: init 2026-02-17T06:10:05.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:05.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:10:05.938 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50bc06c890 0x7f50bc1cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient(hunting): picked mon.noname-c con 0x7f50bc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:05.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 0x7f50bc133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient(hunting): picked mon.noname-a con 0x7f50bc09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:05.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50bc134df0 0x7f50bc1344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient(hunting): picked mon.noname-b con 0x7f50bc134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:05.939 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc0a01a0 con 0x7f50bc09c4a0 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc0a0020 con 0x7f50bc134df0 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f50bc09be20 con 0x7f50bc06c890 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c3785640 10 monclient(hunting): authenticate will time out at 2051.472257s 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 0x7f50bc133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 0x7f50bc133f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42662/0 (socket says 10.20.193.78:42662) 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 -- 10.20.193.78:0/2389416858 learned_addr learned my addr 10.20.193.78:0/2389416858 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:05.940 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient(hunting): get_auth_request con 0x7f50bc09c4a0 auth_method 0 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c14fa640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50bc06c890 0x7f50bc1cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c14fa640 10 monclient(hunting): get_auth_request con 0x7f50bc06c890 auth_method 0 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c14fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c14fa640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c14fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.941 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c1cfb640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50bc134df0 0x7f50bc1344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c1cfb640 10 monclient(hunting): get_auth_request con 0x7f50bc134df0 auth_method 0 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c1cfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c1cfb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c1cfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient(hunting): handle_auth_done global_id 284214 payload 824 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50bc06c890 msgr2=0x7f50bc1cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50bc06c890 0x7f50bc1cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50bc134df0 msgr2=0x7f50bc1344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.942 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50bc134df0 0x7f50bc1344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient: _finish_hunting 0 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 monclient: found mon.noname-a 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.918+0000 7f50c0cf9640 1 -- 10.20.193.78:0/2389416858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc0a1780 con 0x7f50bc09c4a0 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c0cf9640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 msgr2=0x7f50bc133f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c0cf9640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 0x7f50bc133f70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50b0002410 tx=0x7f50b0004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b003c040 con 0x7f50bc09c4a0 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: dump: 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.943 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b0047070 con 0x7f50bc09c4a0 2026-02-17T06:10:05.944 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0042890 con 0x7f50bc09c4a0 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c3785640 5 monclient: authenticate success, global_id 284214 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50ab7fe640 4 set_mon_vals no callback set 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c3785640 10 monclient: _renew_subs 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c3785640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc09d810 con 0x7f50bc09c4a0 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50a9ffb640 10 monclient: dump: 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.945 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c3785640 10 monclient: _renew_subs 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c3785640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.920+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc09f7a0 con 0x7f50bc09c4a0 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c3785640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:05.946 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c3785640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc126200 0x7f50bc09e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c3785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc074980 con 0x7f50bc126200 2026-02-17T06:10:05.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f50b0042a70 con 0x7f50bc09c4a0 2026-02-17T06:10:05.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc126200 0x7f50bc09e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 10 monclient: get_auth_request con 0x7f50bc126200 auth_method 0 2026-02-17T06:10:05.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.947 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:05.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 10 monclient(hunting): handle_auth_done global_id 284217 payload 824 2026-02-17T06:10:05.948 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c14fa640 1 -- 10.20.193.78:0/2389416858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50bc135a20 con 0x7f50bc126200 2026-02-17T06:10:05.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50a9ffb640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f508c03df00 0x7f508c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(853..853 src has 525..853) ==== 4735+0+0 (secure 0 0 0) 0x7f50b0077b30 con 0x7f50bc09c4a0 2026-02-17T06:10:05.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c1cfb640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f508c03df00 0x7f508c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.921+0000 7f50c1cfb640 10 monclient: get_auth_request con 0x7f508c03df00 auth_method 0 2026-02-17T06:10:05.949 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50c14fa640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc126200 msgr2=0x7f50bc09e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50c14fa640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc126200 0x7f50bc09e2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00f880 tx=0x7f50ac00fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac00de30 con 0x7f50bc126200 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 10 monclient: dump: 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.950 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.951 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.951 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.951 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f50ac00de30 con 0x7f50bc126200 2026-02-17T06:10:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:05.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc126200 msgr2=0x7f50bc09e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc126200 0x7f50bc09e2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00f880 tx=0x7f50ac00fd50 comp rx=0 tx=0).stop 2026-02-17T06:10:05.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc126200 msgr2=0x7f50bc09e2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:05.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.923+0000 7f50a9ffb640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc126200 0x7f50bc09e2e0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00f880 tx=0x7f50ac00fd50 comp rx=0 tx=0).stop 2026-02-17T06:10:05.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.924+0000 7f50c1cfb640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f508c03df00 0x7f508c0403c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f50bc125f90 tx=0x7f50b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:05.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.926+0000 7f50c3785640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:10:05.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.926+0000 7f50c3785640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc11cc70 0x7f50bc1da530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:05.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c3785640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:05.952 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc0744d0 con 0x7f50bc11cc70 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc11cc70 0x7f50bc1da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 10 monclient: get_auth_request con 0x7f50bc11cc70 auth_method 0 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 10 monclient(hunting): handle_auth_done global_id 284220 payload 824 2026-02-17T06:10:05.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.927+0000 7f50c14fa640 1 -- 10.20.193.78:0/2389416858 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f50bc09e820 con 0x7f50bc11cc70 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c14fa640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc11cc70 msgr2=0x7f50bc1da530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c14fa640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc11cc70 0x7f50bc1da530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00f880 tx=0x7f50ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac003b30 con 0x7f50bc11cc70 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 10 monclient: dump: 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:05.955 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7f50ac003b30 con 0x7f50bc11cc70 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc11cc70 msgr2=0x7f50bc1da530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc11cc70 0x7f50bc1da530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00f880 tx=0x7f50ac0047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc11cc70 msgr2=0x7f50bc1da530 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50a9ffb640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc11cc70 0x7f50bc1da530 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00f880 tx=0x7f50ac0047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:05.956 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 264, 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 57067, 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:10:05.957 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:10:05.958 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:10:05.959 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:05.960 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:05.961 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.962 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:05.963 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:10:05.964 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:05.965 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:10:05.966 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:10:05.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f508c03df00 msgr2=0x7f508c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f508c03df00 0x7f508c0403c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f50bc125f90 tx=0x7f50b8009040 comp rx=0 tx=0).stop 2026-02-17T06:10:05.967 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 10 monclient: shutdown 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 msgr2=0x7f50bc133f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 1 --2- 10.20.193.78:0/2389416858 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50bc09c4a0 0x7f50bc133f70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50b0002410 tx=0x7f50b0004830 comp rx=0 tx=0).stop 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c1cfb640 1 -- 10.20.193.78:0/2389416858 reap_dead start 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c1cfb640 1 -- 10.20.193.78:0/2389416858 reap_dead start 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 shutdown_connections 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 >> 10.20.193.78:0/2389416858 conn(0x7f50bc0a6620 msgr2=0x7f50bc0a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 shutdown_connections 2026-02-17T06:10:05.968 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:05.929+0000 7f50c3785640 1 -- 10.20.193.78:0/2389416858 wait complete. 2026-02-17T06:10:05.968 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:10:05.968 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:10:05.968 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:10:06.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.034+0000 7f7ff2cfa640 10 monclient: get_monmap_and_config 2026-02-17T06:10:06.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.034+0000 7f7ff2cfa640 10 monclient: build_initial_monmap 2026-02-17T06:10:06.038 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.034+0000 7f7ff2cfa640 10 monclient: monmap: 2026-02-17T06:10:06.038 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:06.038 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.038 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:06.035729+0000 2026-02-17T06:10:06.038 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:06.035729+0000 2026-02-17T06:10:06.038 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.034+0000 7f7ff2cfa640 1 Processor -- start 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.034+0000 7f7ff2cfa640 1 -- start start 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.034+0000 7f7ff2cfa640 10 monclient: init 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient: _add_conns ranks=[4,0,3,2,1] 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f7fec06c890 0x7f7fec06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient(hunting): picked mon.noname-c con 0x7f7fec06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:06.039 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7fec09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:06.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7fec06d1b0 0x7f7fec135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient(hunting): picked mon.noname-a con 0x7f7fec06d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:06.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fec06daf0 con 0x7f7fec09c4a0 2026-02-17T06:10:06.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f7fec0a0880 con 0x7f7fec06d1b0 2026-02-17T06:10:06.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f7fec09cc80 con 0x7f7fec06c890 2026-02-17T06:10:06.040 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff2cfa640 10 monclient(hunting): authenticate will time out at 2051.589272s 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7fe3fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f7fec09c4a0 0x7f7fec134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52480/0 (socket says 10.20.193.78:52480) 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f7fec06d1b0 0x7f7fec135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7fe3fff640 1 -- 10.20.193.78:0/1393443058 learned_addr learned my addr 10.20.193.78:0/1393443058 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient(hunting): get_auth_request con 0x7f7fec06d1b0 auth_method 0 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff0a6f640 1 --2- 10.20.193.78:0/1393443058 >> v2:10.20.193.115:3300/0 conn(0x7f7fec06c890 0x7f7fec06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff0a6f640 10 monclient(hunting): get_auth_request con 0x7f7fec06c890 auth_method 0 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff0a6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff0a6f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff0a6f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.041 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient(hunting): handle_auth_done global_id 284223 payload 274 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 1 -- 10.20.193.78:0/1393443058 >> v2:10.20.193.115:3300/0 conn(0x7f7fec06c890 msgr2=0x7f7fec06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 1 --2- 10.20.193.78:0/1393443058 >> v2:10.20.193.115:3300/0 conn(0x7f7fec06c890 0x7f7fec06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 1 -- 10.20.193.78:0/1393443058 >> v1:10.20.193.78:6789/0 conn(0x7f7fec09c4a0 legacy=0x7f7fec134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient: _finish_hunting 0 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 1 monclient: found mon.noname-a 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.035+0000 7f7ff1270640 1 -- 10.20.193.78:0/1393443058 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec09cf60 con 0x7f7fec06d1b0 2026-02-17T06:10:06.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.037+0000 7f7ff1270640 1 -- 10.20.193.78:0/1393443058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec06d1b0 msgr2=0x7f7fec135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.037+0000 7f7ff1270640 1 --2- 10.20.193.78:0/1393443058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec06d1b0 0x7f7fec135500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7fec06a480 tx=0x7f7fe802f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574f7a7768a5ed33 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 1 -- 10.20.193.78:0/1393443058 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe803c070 con 0x7f7fec06d1b0 2026-02-17T06:10:06.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:06.050 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: dump: 2026-02-17T06:10:06.050 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: _finish_auth 0 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: _check_auth_tickets 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 5 monclient: authenticate success, global_id 284223 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 1 -- 10.20.193.78:0/1393443058 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe8037440 con 0x7f7fec06d1b0 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:06.051 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 1 -- 10.20.193.78:0/1393443058 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8037810 con 0x7f7fec06d1b0 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 4 set_mon_vals no callback set 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7fe37fe640 10 monclient: dump: 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.052 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 10 monclient: shutdown 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/1393443058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec06d1b0 msgr2=0x7f7fec135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 1 --2- 10.20.193.78:0/1393443058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec06d1b0 0x7f7fec135500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7fec06a480 tx=0x7f7fe802f130 comp rx=0 tx=0).stop 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/1393443058 shutdown_connections 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 1 --2- 10.20.193.78:0/1393443058 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec06d1b0 0x7f7fec135500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 1 --2- 10.20.193.78:0/1393443058 >> v2:10.20.193.115:3300/0 conn(0x7f7fec06c890 0x7f7fec06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/1393443058 >> 10.20.193.78:0/1393443058 conn(0x7f7fec0a6620 msgr2=0x7f7fec0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/1393443058 shutdown_connections 2026-02-17T06:10:06.053 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/1393443058 wait complete. 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 10 monclient: build_initial_monmap 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.038+0000 7f7ff2cfa640 10 monclient: monmap: 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 1 Processor -- start 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 1 -- start start 2026-02-17T06:10:06.054 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient: init 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7fec06c890 0x7f7fec1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient(hunting): picked mon.noname-b con 0x7f7fec06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec06d1b0 0x7f7fec1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient(hunting): picked mon.noname-c con 0x7f7fec06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec09c4a0 0x7f7fec123560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient(hunting): picked mon.noname-a con 0x7f7fec09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.055 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0a01a0 con 0x7f7fec09c4a0 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0a0020 con 0x7f7fec06c890 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7fec09be20 con 0x7f7fec06d1b0 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff2cfa640 10 monclient(hunting): authenticate will time out at 2051.593588s 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec09c4a0 0x7f7fec123560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec09c4a0 0x7f7fec123560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42712/0 (socket says 10.20.193.78:42712) 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7fe3fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec06d1b0 0x7f7fec1cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 -- 10.20.193.78:0/524993505 learned_addr learned my addr 10.20.193.78:0/524993505 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient(hunting): get_auth_request con 0x7f7fec09c4a0 auth_method 0 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7fe3fff640 10 monclient(hunting): get_auth_request con 0x7f7fec06d1b0 auth_method 0 2026-02-17T06:10:06.056 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7fe3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7fe3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7fe3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7fe3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7fe3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff0a6f640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7fec06c890 0x7f7fec1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7fe3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff0a6f640 10 monclient(hunting): get_auth_request con 0x7f7fec06c890 auth_method 0 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff0a6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff0a6f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff0a6f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient(hunting): handle_auth_done global_id 284226 payload 824 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec06d1b0 msgr2=0x7f7fec1cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec06d1b0 0x7f7fec1cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7fec06c890 msgr2=0x7f7fec1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.057 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f7fec06c890 0x7f7fec1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient: _finish_hunting 0 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 monclient: found mon.noname-a 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.039+0000 7f7ff1270640 1 -- 10.20.193.78:0/524993505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec126eb0 con 0x7f7fec09c4a0 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.041+0000 7f7ff1270640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec09c4a0 msgr2=0x7f7fec123560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.041+0000 7f7ff1270640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec09c4a0 0x7f7fec123560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7fe803a040 tx=0x7f7fe8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe803c050 con 0x7f7fec09c4a0 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: dump: 2026-02-17T06:10:06.058 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.059 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: _finish_auth 0 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: _check_auth_tickets 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe8048070 con 0x7f7fec09c4a0 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8041a50 con 0x7f7fec09c4a0 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 10 monclient: dump: 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.060 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe2ffd640 4 set_mon_vals no callback set 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 5 monclient: authenticate success, global_id 284226 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 10 monclient: _renew_subs 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec125c70 con 0x7f7fec09c4a0 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 10 monclient: _renew_subs 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec06fe70 con 0x7f7fec09c4a0 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124260 0x7f7fec11ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7fec074980 con 0x7f7fec124260 2026-02-17T06:10:06.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124260 0x7f7fec11ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 10 monclient: get_auth_request con 0x7f7fec124260 auth_method 0 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff1270640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec09c4a0 0x7f7fec123560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7fe803a040 tx=0x7f7fe8004b40 comp rx=0 tx=0).queue_received will delay after 1.77131e+09 on 0x7f7fe8041bf0 mgrmap(e 5) 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f7fe8041bf0 con 0x7f7fec09c4a0 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_done global_id 264413 payload 824 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7ff0a6f640 1 -- 10.20.193.78:0/524993505 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fec136ea0 con 0x7f7fec124260 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.042+0000 7f7fe17fa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7fbc03df00 0x7f7fbc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.043+0000 7f7fe3fff640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7fbc03df00 0x7f7fbc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.043+0000 7f7fe3fff640 10 monclient: get_auth_request con 0x7f7fbc03df00 auth_method 0 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.044+0000 7f7ff0a6f640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124260 msgr2=0x7f7fec11ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.062 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.044+0000 7f7ff0a6f640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124260 0x7f7fec11ec80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc011830 tx=0x7f7fdc011d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00fe50 con 0x7f7fec124260 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 10 monclient: dump: 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f7fdc00fe50 con 0x7f7fec124260 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:06.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124260 msgr2=0x7f7fec11ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124260 0x7f7fec11ec80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc011830 tx=0x7f7fdc011d00 comp rx=0 tx=0).stop 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124260 msgr2=0x7f7fec11ec80 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe17fa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124260 0x7f7fec11ec80 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc011830 tx=0x7f7fdc011d00 comp rx=0 tx=0).stop 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.045+0000 7f7fe3fff640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7fbc03df00 0x7f7fbc0403c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4007d90 tx=0x7f7fd4007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff2cfa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff2cfa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124640 0x7f7fec1d8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff2cfa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f7fec06cc70 con 0x7f7fec124640 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124640 0x7f7fec1d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 10 monclient: get_auth_request con 0x7f7fec124640 auth_method 0 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 10 monclient(hunting): handle_auth_done global_id 264416 payload 824 2026-02-17T06:10:06.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.048+0000 7f7ff0a6f640 1 -- 10.20.193.78:0/524993505 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7fec120340 con 0x7f7fec124640 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.050+0000 7f7ff0a6f640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124640 msgr2=0x7f7fec1d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.050+0000 7f7ff0a6f640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124640 0x7f7fec1d8330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc011830 tx=0x7f7fdc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.050+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc003920 con 0x7f7fec124640 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.050+0000 7f7fe17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.050+0000 7f7fe17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.050+0000 7f7fe17fa640 10 monclient: dump: 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.066 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f7fdc003920 con 0x7f7fec124640 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7fe17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7fe17fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124640 msgr2=0x7f7fec1d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7fe17fa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124640 0x7f7fec1d8330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc011830 tx=0x7f7fdc0047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124640 msgr2=0x7f7fec1d8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7fe17fa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f7fec124640 0x7f7fec1d8330 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc011830 tx=0x7f7fdc0047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:06.067 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 264, 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 155435, 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:06.068 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:06.069 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:10:06.070 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:06.071 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:10:06.072 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.073 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:06.074 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.075 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.076 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:10:06.077 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.078 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:06.078 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:06.078 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:10:06.078 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.078 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.078 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.078 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:10:06.078 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7fbc03df00 msgr2=0x7f7fbc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f7fbc03df00 0x7f7fbc0403c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4007d90 tx=0x7f7fd4007ce0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 10 monclient: shutdown 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec09c4a0 msgr2=0x7f7fec123560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 1 --2- 10.20.193.78:0/524993505 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f7fec09c4a0 0x7f7fec123560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7fe803a040 tx=0x7f7fe8004b40 comp rx=0 tx=0).stop 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff1270640 1 -- 10.20.193.78:0/524993505 reap_dead start 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7fe17fa640 1 -- 10.20.193.78:0/524993505 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(853..853 src has 525..853) ==== 4735+0+0 (unknown 0 0 0) 0x7f7fe8078040 con 0x7f7fec09c4a0 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff1270640 1 -- 10.20.193.78:0/524993505 reap_dead start 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 shutdown_connections 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 >> 10.20.193.78:0/524993505 conn(0x7f7fec0a6620 msgr2=0x7f7fec071280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 shutdown_connections 2026-02-17T06:10:06.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.051+0000 7f7ff2cfa640 1 -- 10.20.193.78:0/524993505 wait complete. 2026-02-17T06:10:06.080 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:10:06.080 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:10:06.080 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:10:06.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient: get_monmap_and_config 2026-02-17T06:10:06.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient: build_initial_monmap 2026-02-17T06:10:06.145 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient: monmap: 2026-02-17T06:10:06.145 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:06.145498+0000 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:06.145498+0000 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 1 Processor -- start 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 1 -- start start 2026-02-17T06:10:06.146 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient: init 2026-02-17T06:10:06.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:06.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2026-02-17T06:10:06.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f54d806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:06.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f54d806d380 0x7f54d80a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient(hunting): picked mon.noname-c con 0x7f54d806d380 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:06.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f54d813a240 0x7f54d813c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.151 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient(hunting): picked mon.noname-a con 0x7f54d813a240 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d8074150 con 0x7f54d806c890 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f54d809ef70 con 0x7f54d813a240 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f54d809f250 con 0x7f54d806d380 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.144+0000 7f54de3e4640 10 monclient(hunting): authenticate will time out at 2051.698992s 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f54d813a240 0x7f54d813c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f54d813a240 0x7f54d813c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42724/0 (socket says 10.20.193.78:42724) 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 1 -- 10.20.193.78:0/3104071298 learned_addr learned my addr 10.20.193.78:0/3104071298 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient(hunting): get_auth_request con 0x7f54d813a240 auth_method 0 2026-02-17T06:10:06.152 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54d77fe640 1 --2- 10.20.193.78:0/3104071298 >> v2:10.20.193.115:3300/0 conn(0x7f54d806d380 0x7f54d80a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54d77fe640 10 monclient(hunting): get_auth_request con 0x7f54d806d380 auth_method 0 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient(hunting): handle_auth_done global_id 284229 payload 274 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 1 -- 10.20.193.78:0/3104071298 >> v2:10.20.193.115:3300/0 conn(0x7f54d806d380 msgr2=0x7f54d80a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 1 --2- 10.20.193.78:0/3104071298 >> v2:10.20.193.115:3300/0 conn(0x7f54d806d380 0x7f54d80a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 1 -- 10.20.193.78:0/3104071298 >> v1:10.20.193.115:6789/0 conn(0x7f54d806c890 legacy=0x7f54d806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient: _finish_hunting 0 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 1 monclient: found mon.noname-a 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.145+0000 7f54dc95a640 1 -- 10.20.193.78:0/3104071298 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d809df40 con 0x7f54d813a240 2026-02-17T06:10:06.153 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.147+0000 7f54dc95a640 1 -- 10.20.193.78:0/3104071298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d813a240 msgr2=0x7f54d813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.147+0000 7f54dc95a640 1 --2- 10.20.193.78:0/3104071298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d813a240 0x7f54d813c630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f54d8074e00 tx=0x7f54cc02f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2eba20dac4522 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.147+0000 7f54d6ffd640 1 -- 10.20.193.78:0/3104071298 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0373f0 con 0x7f54d813a240 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: dump: 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:10:06.154 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 1 -- 10.20.193.78:0/3104071298 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc037590 con 0x7f54d813a240 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 5 monclient: authenticate success, global_id 284229 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 1 -- 10.20.193.78:0/3104071298 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc037960 con 0x7f54d813a240 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 4 set_mon_vals no callback set 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54d6ffd640 10 monclient: dump: 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.155 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 10 monclient: shutdown 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 -- 10.20.193.78:0/3104071298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d813a240 msgr2=0x7f54d813c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 --2- 10.20.193.78:0/3104071298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d813a240 0x7f54d813c630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f54d8074e00 tx=0x7f54cc02f270 comp rx=0 tx=0).stop 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 -- 10.20.193.78:0/3104071298 shutdown_connections 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 --2- 10.20.193.78:0/3104071298 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d813a240 0x7f54d813c630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 --2- 10.20.193.78:0/3104071298 >> v2:10.20.193.115:3300/0 conn(0x7f54d806d380 0x7f54d80a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 -- 10.20.193.78:0/3104071298 >> 10.20.193.78:0/3104071298 conn(0x7f54d80a6620 msgr2=0x7f54d80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 -- 10.20.193.78:0/3104071298 shutdown_connections 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 -- 10.20.193.78:0/3104071298 wait complete. 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 10 monclient: build_initial_monmap 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 10 monclient: monmap: 2026-02-17T06:10:06.156 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.148+0000 7f54de3e4640 1 Processor -- start 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 1 -- start start 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient: init 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d806c890 0x7f54d8138580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient(hunting): picked mon.noname-a con 0x7f54d806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54d806d380 0x7f54d8135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient(hunting): picked mon.noname-c con 0x7f54d806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:06.157 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d813a240 0x7f54d81cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient(hunting): picked mon.noname-b con 0x7f54d813a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54d813ccf0 con 0x7f54d806c890 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54d813cb70 con 0x7f54d813a240 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f54d813ce70 con 0x7f54d806d380 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54de3e4640 10 monclient(hunting): authenticate will time out at 2051.703302s 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d806c890 0x7f54d8138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54d806d380 0x7f54d8135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d806c890 0x7f54d8138580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42730/0 (socket says 10.20.193.78:42730) 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 -- 10.20.193.78:0/4096465011 learned_addr learned my addr 10.20.193.78:0/4096465011 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient(hunting): get_auth_request con 0x7f54d806c890 auth_method 0 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 10 monclient(hunting): get_auth_request con 0x7f54d806d380 auth_method 0 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.158 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54dc95a640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d813a240 0x7f54d81cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54dc95a640 10 monclient(hunting): get_auth_request con 0x7f54d813a240 auth_method 0 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54dc95a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54dc95a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54dc95a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient(hunting): handle_auth_done global_id 284232 payload 824 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54d806d380 msgr2=0x7f54d8135510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54d806d380 0x7f54d8135510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d813a240 msgr2=0x7f54d81cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d813a240 0x7f54d81cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient: _finish_hunting 0 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 monclient: found mon.noname-a 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.159 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d7fff640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d811fea0 con 0x7f54d806c890 2026-02-17T06:10:06.160 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.149+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f54d806d380 0x7f54d8135510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:10:06.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d7fff640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d806c890 msgr2=0x7f54d8138580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d7fff640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d806c890 0x7f54d8138580 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54c400c5d0 tx=0x7f54c400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.165 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4016020 con 0x7f54d806c890 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: dump: 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: _finish_auth 0 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c4012a60 con 0x7f54d806c890 2026-02-17T06:10:06.166 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4012e50 con 0x7f54d806c890 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d4ff9640 10 monclient: dump: 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54d67fc640 4 set_mon_vals no callback set 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54de3e4640 5 monclient: authenticate success, global_id 284232 2026-02-17T06:10:06.167 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54de3e4640 10 monclient: _renew_subs 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54de3e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d80a2730 con 0x7f54d806c890 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54de3e4640 10 monclient: _renew_subs 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54de3e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.151+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d809d810 con 0x7f54d806c890 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54de3e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54de3e4640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d8135a50 0x7f54d81cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54de3e4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8074980 con 0x7f54d8135a50 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d8135a50 0x7f54d81cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 10 monclient: get_auth_request con 0x7f54d8135a50 auth_method 0 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.168 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f54c401b400 con 0x7f54d806c890 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d4ff9640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54a803e050 0x7f54a8040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(853..853 src has 525..853) ==== 4735+0+0 (secure 0 0 0) 0x7f54c40528a0 con 0x7f54d806c890 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 10 monclient(hunting): handle_auth_done global_id 294115 payload 824 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54d77fe640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54d80a01d0 con 0x7f54d8135a50 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54dc95a640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54a803e050 0x7f54a8040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.152+0000 7f54dc95a640 10 monclient: get_auth_request con 0x7f54a803e050 auth_method 0 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d77fe640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d8135a50 msgr2=0x7f54d81cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d8135a50 0x7f54d81cf450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54c8011a60 tx=0x7f54c8011f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8019070 con 0x7f54d8135a50 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 10 monclient: dump: 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.169 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54dc95a640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54a803e050 0x7f54a8040510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f54d806a480 tx=0x7f54cc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f54c8015660 con 0x7f54d8135a50 2026-02-17T06:10:06.170 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d8135a50 msgr2=0x7f54d81cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d8135a50 0x7f54d81cf450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54c8011a60 tx=0x7f54c8011f30 comp rx=0 tx=0).stop 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d8135a50 msgr2=0x7f54d81cf450 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.155+0000 7f54d4ff9640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d8135a50 0x7f54d81cf450 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54c8011a60 tx=0x7f54c8011f30 comp rx=0 tx=0).stop 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54de3e4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54de3e4640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 0x7f54d81da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54de3e4640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81cfda0 con 0x7f54d81cf990 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 0x7f54d81da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 10 monclient: get_auth_request con 0x7f54d81cf990 auth_method 0 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.171 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 10 monclient(hunting): handle_auth_done global_id 294118 payload 824 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.158+0000 7f54d77fe640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f54d811f6e0 con 0x7f54d81cf990 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 msgr2=0x7f54d81da2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 0x7f54d81da2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f54c8011a60 tx=0x7f54c8004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 msgr2=0x7f54d81da2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 msgr2=0x7f54d81da2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 0x7f54d81da2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f54c8011a60 tx=0x7f54c8004830 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:06.172 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 0x7f54d81da2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f54c8011a60 tx=0x7f54c8004830 comp rx=0 tx=0).stop 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 msgr2=0x7f54d81da2c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d4ff9640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 0x7f54d81da2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d4ff9640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54a8044280 0x7f54a8046670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 msgr2=0x7f54d81da2c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d4ff9640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54d81cf990 0x7f54d81da2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d4ff9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8046c80 con 0x7f54a8044280 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54a8044280 0x7f54a8046670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 10 monclient: get_auth_request con 0x7f54a8044280 auth_method 0 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 10 monclient(hunting): handle_auth_done global_id 294121 payload 824 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.161+0000 7f54d77fe640 1 -- 10.20.193.78:0/4096465011 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f54c8015660 con 0x7f54a8044280 2026-02-17T06:10:06.173 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d77fe640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54a8044280 msgr2=0x7f54a8046670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d77fe640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54a8044280 0x7f54a8046670 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54c8002410 tx=0x7f54c80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8038020 con 0x7f54a8044280 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 10 monclient: dump: 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f54c8003cd0 con 0x7f54a8044280 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54a8044280 msgr2=0x7f54a8046670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54a8044280 0x7f54a8046670 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54c8002410 tx=0x7f54c80047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54a8044280 msgr2=0x7f54a8046670 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54d4ff9640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f54a8044280 0x7f54a8046670 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54c8002410 tx=0x7f54c80047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54dc95a640 1 -- 10.20.193.78:0/4096465011 reap_dead start 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 264, 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 6866, 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:10:06.175 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:06.176 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:10:06.177 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:06.178 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:06.179 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:06.180 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:06.181 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:10:06.182 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:10:06.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54a803e050 msgr2=0x7f54a8040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54a803e050 0x7f54a8040510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f54d806a480 tx=0x7f54cc033000 comp rx=0 tx=0).stop 2026-02-17T06:10:06.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 10 monclient: shutdown 2026-02-17T06:10:06.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:06.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d806c890 msgr2=0x7f54d8138580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d806c890 0x7f54d8138580 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54c400c5d0 tx=0x7f54c400caa0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 shutdown_connections 2026-02-17T06:10:06.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f54a803e050 0x7f54a8040510 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 --2- 10.20.193.78:0/4096465011 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f54d806c890 0x7f54d8138580 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 >> 10.20.193.78:0/4096465011 conn(0x7f54d80a6620 msgr2=0x7f54d8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:06.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 shutdown_connections 2026-02-17T06:10:06.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.164+0000 7f54de3e4640 1 -- 10.20.193.78:0/4096465011 wait complete. 2026-02-17T06:10:06.184 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:10:06.184 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:10:06.184 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient: get_monmap_and_config 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient: build_initial_monmap 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient: monmap: 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:06.252305+0000 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:06.252305+0000 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 1 Processor -- start 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 1 -- start start 2026-02-17T06:10:06.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient: init 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fa9ac06c890 0x7fa9ac06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient(hunting): picked mon.noname-c con 0x7fa9ac06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa9ac09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa9ac06d1b0 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ac135c10 con 0x7fa9ac06d1b0 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ac06daf0 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ac0a0880 con 0x7fa9ac06c890 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b27c3640 10 monclient(hunting): authenticate will time out at 2051.805818s 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b0d39640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fa9ac06d1b0 0x7fa9ac135500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:52494/0 (socket says 10.20.193.78:52494) 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9b0d39640 1 -- 10.20.193.78:0/736206560 learned_addr learned my addr 10.20.193.78:0/736206560 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9abfff640 1 --2- 10.20.193.78:0/736206560 >> v2:10.20.193.115:3300/0 conn(0x7fa9ac06c890 0x7fa9ac06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9abfff640 10 monclient(hunting): get_auth_request con 0x7fa9ac06c890 auth_method 0 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9abfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.251+0000 7fa9abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.252+0000 7fa9abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.252+0000 7fa9abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.252+0000 7fa9abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9b0d39640 1 -- 10.20.193.78:0/736206560 >> v1:10.20.193.78:6789/0 conn(0x7fa9ac06d1b0 legacy=0x7fa9ac135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9aaffd640 1 -- 10.20.193.78:0/736206560 <== mon.0 v1:10.20.193.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803404507 0 0) 0x7fa9ac135c10 con 0x7fa9ac06d1b0 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9aaffd640 10 monclient(hunting): _negotiate 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9aaffd640 10 monclient(hunting): my global_id is 284235 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9aaffd640 1 -- 10.20.193.78:0/736206560 --> v1:10.20.193.78:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa990003620 con 0x7fa9ac06d1b0 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9ab7fe640 1 -- 10.20.193.78:0/736206560 >> v1:10.20.193.115:6789/0 conn(0x7fa9ac09c4a0 legacy=0x7fa9ac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9abfff640 10 monclient(hunting): handle_auth_done global_id 294124 payload 274 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9abfff640 1 -- 10.20.193.78:0/736206560 >> v1:10.20.193.115:6789/0 conn(0x7fa9ac09c4a0 legacy=0x7fa9ac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9abfff640 1 -- 10.20.193.78:0/736206560 >> v1:10.20.193.78:6789/0 conn(0x7fa9ac06d1b0 legacy=0x7fa9ac135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9abfff640 10 monclient: _finish_hunting 0 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9abfff640 1 monclient: found mon.noname-c 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9abfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:10:06.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.254+0000 7fa9abfff640 1 -- 10.20.193.78:0/736206560 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac09cc80 con 0x7fa9ac06c890 2026-02-17T06:10:06.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.256+0000 7fa9abfff640 1 -- 10.20.193.78:0/736206560 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 msgr2=0x7fa9ac06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.256+0000 7fa9abfff640 1 --2- 10.20.193.78:0/736206560 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 0x7fa9ac06cc70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa9980099a0 tx=0x7fa99802eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c62cce3637a8a3 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 1 -- 10.20.193.78:0/736206560 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99803c070 con 0x7fa9ac06c890 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: dump: 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: _finish_auth 0 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: _check_auth_tickets 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 1 -- 10.20.193.78:0/736206560 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99802fe00 con 0x7fa9ac06c890 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 5 monclient: authenticate success, global_id 294124 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 1 -- 10.20.193.78:0/736206560 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998037660 con 0x7fa9ac06c890 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9aaffd640 10 monclient: dump: 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 4 set_mon_vals no callback set 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 10 monclient: shutdown 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 1 -- 10.20.193.78:0/736206560 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 msgr2=0x7fa9ac06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 1 --2- 10.20.193.78:0/736206560 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 0x7fa9ac06cc70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa9980099a0 tx=0x7fa99802eee0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 1 -- 10.20.193.78:0/736206560 shutdown_connections 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 1 --2- 10.20.193.78:0/736206560 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 0x7fa9ac06cc70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 1 -- 10.20.193.78:0/736206560 >> 10.20.193.78:0/736206560 conn(0x7fa9ac0a6620 msgr2=0x7fa9ac0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 1 -- 10.20.193.78:0/736206560 shutdown_connections 2026-02-17T06:10:06.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 1 -- 10.20.193.78:0/736206560 wait complete. 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 10 monclient: build_initial_monmap 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.257+0000 7fa9b27c3640 10 monclient: monmap: 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 1 Processor -- start 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 1 -- start start 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient: init 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 0x7fa9ac1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient(hunting): picked mon.noname-b con 0x7fa9ac06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa9ac06d1b0 0x7fa9ac1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient(hunting): picked mon.noname-c con 0x7fa9ac06d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa9ac09c4a0 0x7fa9ac122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient(hunting): picked mon.noname-a con 0x7fa9ac09c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac0a0020 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac06d640 con 0x7fa9ac06c890 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ac0a01a0 con 0x7fa9ac06d1b0 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:06.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b27c3640 10 monclient(hunting): authenticate will time out at 2051.812420s 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa9ac09c4a0 0x7fa9ac122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9ab7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa9ac06d1b0 0x7fa9ac1cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa9ac09c4a0 0x7fa9ac122680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:42732/0 (socket says 10.20.193.78:42732) 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 -- 10.20.193.78:0/462335503 learned_addr learned my addr 10.20.193.78:0/462335503 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient(hunting): get_auth_request con 0x7fa9ac09c4a0 auth_method 0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9ab7fe640 10 monclient(hunting): get_auth_request con 0x7fa9ac06d1b0 auth_method 0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9abfff640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 0x7fa9ac1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient(hunting): handle_auth_done global_id 284238 payload 824 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 -- 10.20.193.78:0/462335503 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa9ac06d1b0 msgr2=0x7fa9ac1cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa9ac06d1b0 0x7fa9ac1cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 -- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 msgr2=0x7fa9ac1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 0x7fa9ac1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient: _finish_hunting 0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 monclient: found mon.noname-a 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9b0d39640 1 -- 10.20.193.78:0/462335503 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac125f90 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9abfff640 10 monclient: get_auth_request con 0x7fa9ac06c890 auth_method 0 2026-02-17T06:10:06.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.258+0000 7fa9abfff640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 0x7fa9ac1cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:10:06.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b0d39640 1 -- 10.20.193.78:0/462335503 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa9ac09c4a0 msgr2=0x7fa9ac122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:06.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b0d39640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa9ac09c4a0 0x7fa9ac122680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000cde0 tx=0x7fa9a000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 1 -- 10.20.193.78:0/462335503 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a000ad50 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:06.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: dump: 2026-02-17T06:10:06.263 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.263 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: _finish_auth 0 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 1 -- 10.20.193.78:0/462335503 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a0012400 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 1 -- 10.20.193.78:0/462335503 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0012830 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b27c3640 5 monclient: authenticate success, global_id 284238 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b27c3640 10 monclient: _renew_subs 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b27c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ac1246a0 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9aa7fc640 4 set_mon_vals no callback set 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9a8ff9640 10 monclient: dump: 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b27c3640 10 monclient: _renew_subs 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b27c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.260+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ac0a2590 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.261+0000 7fa9b27c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.261+0000 7fa9b27c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.261+0000 7fa9b27c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.261+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ac09bf40 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.261+0000 7fa9a8ff9640 1 -- 10.20.193.78:0/462335503 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa9a0012a90 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.261+0000 7fa9a8ff9640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa97c03deb0 0x7fa97c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.261+0000 7fa9a8ff9640 1 -- 10.20.193.78:0/462335503 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(853..853 src has 525..853) ==== 4735+0+0 (secure 0 0 0) 0x7fa9a0052bf0 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.262+0000 7fa9abfff640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa97c03deb0 0x7fa97c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.262+0000 7fa9abfff640 10 monclient: get_auth_request con 0x7fa97c03deb0 auth_method 0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.264+0000 7fa9a8ff9640 1 -- 10.20.193.78:0/462335503 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa9a001bc00 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.264+0000 7fa9a8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.264+0000 7fa9a8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:06.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.264+0000 7fa9abfff640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa97c03deb0 0x7fa97c040370 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac074de0 tx=0x7fa998033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:06.336 INFO:tasks.rados.rados.0.trial115.stdout:3906: done (2 left) 2026-02-17T06:10:06.336 INFO:tasks.rados.rados.0.trial115.stdout:3908: done (1 left) 2026-02-17T06:10:06.337 INFO:tasks.rados.rados.0.trial115.stdout:3909: done (0 left) 2026-02-17T06:10:06.337 INFO:tasks.rados.rados.0.trial115.stdout:3910: snap_remove snap 411 2026-02-17T06:10:06.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.379+0000 7fa9b27c3640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:10:06.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.379+0000 7fa9b27c3640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:10:06.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.379+0000 7fa9b27c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:06.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.379+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa9ac09d760 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.380 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.379+0000 7fa9a8ff9640 1 -- 10.20.193.78:0/462335503 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa9a0007070 con 0x7fa9ac09c4a0 2026-02-17T06:10:06.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.379+0000 7fa9a8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:10:06.381 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.379+0000 7fa9a8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:06.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.379+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.379+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=3870698283,config=692619927,health=4185752094,kv=449989585,logm=2432546020}) 2026-02-17T06:10:06.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.379+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=28,config=2,health=11,kv=12,logm=47} crc {auth=3628276407,config=692619927,health=3319432238,kv=449989585,logm=306776842}) 2026-02-17T06:10:06.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.379+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.382 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.379+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=3870698283,config=692619927,health=4185752094,kv=449989585,logm=2432546020}) 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.379+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=30,config=2,health=12,kv=12,logm=44} crc {auth=3030411438,config=692619927,health=4185752094,kv=449989585,logm=4059520827}) 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1671860567}) 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1725934944}) 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1671860567}) 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2671579188}) 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.383 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187311361}) 2026-02-17T06:10:06.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa97c03deb0 msgr2=0x7fa97c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa97c03deb0 0x7fa97c040370 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac074de0 tx=0x7fa998033000 comp rx=0 tx=0).stop 2026-02-17T06:10:06.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 10 monclient: shutdown 2026-02-17T06:10:06.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:06.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa9ac09c4a0 msgr2=0x7fa9ac122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:06.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2444802092}) 2026-02-17T06:10:06.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa9ac09c4a0 0x7fa9ac122680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000cde0 tx=0x7fa9a000a430 comp rx=0 tx=0).stop 2026-02-17T06:10:06.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187311361}) 2026-02-17T06:10:06.384 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.380+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4109247785}) 2026-02-17T06:10:06.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 shutdown_connections 2026-02-17T06:10:06.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa9ac09c4a0 0x7fa9ac122680 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa9ac06d1b0 0x7fa9ac1cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa97c03deb0 0x7fa97c040370 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 --2- 10.20.193.78:0/462335503 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa9ac06c890 0x7fa9ac1cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:06.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 >> 10.20.193.78:0/462335503 conn(0x7fa9ac0a6620 msgr2=0x7fa9ac11cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:06.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 shutdown_connections 2026-02-17T06:10:06.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:06.380+0000 7fa9b27c3640 1 -- 10.20.193.78:0/462335503 wait complete. 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2864914069}) 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1748458584}) 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2864914069}) 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3349140075}) 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1006792738}) 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2514658335}) 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.385 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1006792738}) 2026-02-17T06:10:06.386 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.381+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=148969652}) 2026-02-17T06:10:06.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:10:06.716 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.715+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.716 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.715+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2410868388}) 2026-02-17T06:10:06.717 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.715+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=656115764}) 2026-02-17T06:10:06.717 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.715+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.717 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.715+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2410868388}) 2026-02-17T06:10:06.717 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.715+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=950347744}) 2026-02-17T06:10:06.718 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.716+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.718 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.716+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2991834326}) 2026-02-17T06:10:06.718 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.716+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=269793023}) 2026-02-17T06:10:06.718 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.716+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:06.718 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.716+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2991834326}) 2026-02-17T06:10:06.718 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:06.716+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2943590979}) 2026-02-17T06:10:07.016 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1067039658}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=69735863}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1067039658}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4196005788}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=744764126}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4260588}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=744764126}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.016+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3139552723}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570817277}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4123483173}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570817277}) 2026-02-17T06:10:07.017 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1100752637}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3208287096}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3812334788}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3208287096}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3694063741}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=998892060}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1744699815}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=998892060}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.017+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1695082095}) 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.018 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969434731}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099834516}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969434731}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196081974}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2994332723}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3851171887}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2994332723}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3710683750}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2605153679}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3925680963}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2605153679}) 2026-02-17T06:10:07.019 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.018+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4293016732}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3544241802}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3560745014}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3544241802}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1134040764}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436970160}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887589822}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436970160}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1695124215}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=73722628}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1167050687}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=73722628}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.019+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2548936457}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.020+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.020+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2159611596}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.020+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3377404299}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.020+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.020+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2159611596}) 2026-02-17T06:10:07.020 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.020+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4196481671}) 2026-02-17T06:10:07.027 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.026+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.027 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.026+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122225682}) 2026-02-17T06:10:07.027 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.027+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=742996209}) 2026-02-17T06:10:07.027 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.027+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.027 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.027+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122225682}) 2026-02-17T06:10:07.027 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.027+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1176296074}) 2026-02-17T06:10:07.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.335 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3629646912}) 2026-02-17T06:10:07.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=176389302}) 2026-02-17T06:10:07.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3629646912}) 2026-02-17T06:10:07.336 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.334+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4125807783}) 2026-02-17T06:10:07.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2697840811}) 2026-02-17T06:10:07.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=293907147}) 2026-02-17T06:10:07.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2697840811}) 2026-02-17T06:10:07.337 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.336+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2273237180}) 2026-02-17T06:10:07.338 INFO:tasks.rados.rados.0.trial115.stdout:3910: done (0 left) 2026-02-17T06:10:07.338 INFO:tasks.rados.rados.0.trial115.stdout:3911: rollback oid 33 current snap is 414 2026-02-17T06:10:07.338 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 410 2026-02-17T06:10:07.338 INFO:tasks.rados.rados.0.trial115.stdout:3912: write (excl) oid 32 current snap is 414 2026-02-17T06:10:07.339 INFO:tasks.rados.rados.0.trial115.stdout:3912: seq_num 1333 ranges {403731=668692,1587203=587797} 2026-02-17T06:10:07.342 INFO:tasks.rados.rados.0.trial115.stdout:3912: writing trial11545420-32 from 403731 to 1072423 tid 1 2026-02-17T06:10:07.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.342+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.342+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209876606}) 2026-02-17T06:10:07.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.342+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=189836968}) 2026-02-17T06:10:07.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.342+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.342+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209876606}) 2026-02-17T06:10:07.343 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.342+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1084883640}) 2026-02-17T06:10:07.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.344+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.345 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.344+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=184909498}) 2026-02-17T06:10:07.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.344+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2977585893}) 2026-02-17T06:10:07.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.344+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.344+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=184909498}) 2026-02-17T06:10:07.346 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.344+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=772985497}) 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:3912: writing trial11545420-32 from 1587203 to 2175000 tid 2 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:3913: read oid 9 snap -1 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:3913: expect deleted 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:3914: rollback oid 7 current snap is 414 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 406 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:3911: finishing rollback tid 1 to trial11545420-33 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:3912: finishing write tid 1 to trial11545420-32 2026-02-17T06:10:07.348 INFO:tasks.rados.rados.0.trial115.stdout:3911: done (3 left) 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3915: read oid 41 snap -1 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3915: expect deleted 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3916: rollback oid 33 current snap is 414 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 33 to 413 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3917: copy_from oid 40 from oid 37 current snap is 414 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3918: copy_from oid 8 from oid 38 current snap is 414 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3919: rollback oid 49 current snap is 414 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 49 to 413 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3920: copy_from oid 20 from oid 19 current snap is 414 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3921: write (excl) oid 10 current snap is 414 2026-02-17T06:10:07.349 INFO:tasks.rados.rados.0.trial115.stdout:3921: seq_num 1334 ranges {420538=743941,1929332=491206} 2026-02-17T06:10:07.351 INFO:tasks.rados.rados.0.trial115.stdout:3921: writing trial11545420-10 from 420538 to 1164479 tid 1 2026-02-17T06:10:07.356 INFO:tasks.rados.rados.0.trial115.stdout:3921: writing trial11545420-10 from 1929332 to 2420538 tid 2 2026-02-17T06:10:07.356 INFO:tasks.rados.rados.0.trial115.stdout:3922: write (excl) oid 29 current snap is 414 2026-02-17T06:10:07.356 INFO:tasks.rados.rados.0.trial115.stdout:3922: seq_num 1335 ranges {437345=419190,1587814=591466} 2026-02-17T06:10:07.359 INFO:tasks.rados.rados.0.trial115.stdout:3922: writing trial11545420-29 from 437345 to 856535 tid 1 2026-02-17T06:10:07.364 INFO:tasks.rados.rados.0.trial115.stdout:3922: writing trial11545420-29 from 1587814 to 2179280 tid 2 2026-02-17T06:10:07.365 INFO:tasks.rados.rados.0.trial115.stdout:3923: write (excl) oid 24 current snap is 414 2026-02-17T06:10:07.365 INFO:tasks.rados.rados.0.trial115.stdout:3923: seq_num 1336 ranges {454152=494439,1646296=735124} 2026-02-17T06:10:07.368 INFO:tasks.rados.rados.0.trial115.stdout:3923: writing trial11545420-24 from 454152 to 948591 tid 1 2026-02-17T06:10:07.373 INFO:tasks.rados.rados.0.trial115.stdout:3923: writing trial11545420-24 from 1646296 to 2381420 tid 2 2026-02-17T06:10:07.373 INFO:tasks.rados.rados.0.trial115.stdout:3924: snap_remove snap 410 2026-02-17T06:10:07.390 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:10:07.390 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient: get_monmap_and_config 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient: build_initial_monmap 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient: monmap: 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:07.458348+0000 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:07.458348+0000 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:07.458 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:07.459 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:07.459 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 1 Processor -- start 2026-02-17T06:10:07.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 1 -- start start 2026-02-17T06:10:07.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient: init 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f974006c890 0x7f974006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient(hunting): picked mon.noname-c con 0x7f974006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974009c4a0 0x7f974006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient(hunting): picked mon.noname-b con 0x7f974009c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9740134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f974006daf0 con 0x7f9740134df0 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f97400a0880 con 0x7f974006c890 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f974006d6f0 con 0x7f974009c4a0 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f9744ada640 10 monclient(hunting): authenticate will time out at 2053.011857s 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f973dd74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974009c4a0 0x7f974006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f973dd74640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974009c4a0 0x7f974006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37294/0 (socket says 10.20.193.78:37294) 2026-02-17T06:10:07.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f973dd74640 1 -- 10.20.193.78:0/861274265 learned_addr learned my addr 10.20.193.78:0/861274265 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f973dd74640 10 monclient(hunting): get_auth_request con 0x7f974009c4a0 auth_method 0 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f973dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f973dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.457+0000 7f973dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973e575640 1 --2- 10.20.193.78:0/861274265 >> v2:10.20.193.115:3300/0 conn(0x7f974006c890 0x7f974006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973e575640 10 monclient(hunting): get_auth_request con 0x7f974006c890 auth_method 0 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 10 monclient(hunting): handle_auth_done global_id 264425 payload 274 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 1 -- 10.20.193.78:0/861274265 >> v2:10.20.193.115:3300/0 conn(0x7f974006c890 msgr2=0x7f974006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 1 --2- 10.20.193.78:0/861274265 >> v2:10.20.193.115:3300/0 conn(0x7f974006c890 0x7f974006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 1 -- 10.20.193.78:0/861274265 >> v1:10.20.193.115:6789/0 conn(0x7f9740134df0 legacy=0x7f9740135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 10 monclient: _finish_hunting 0 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 1 monclient: found mon.noname-b 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.458+0000 7f973dd74640 1 -- 10.20.193.78:0/861274265 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974009cc80 con 0x7f974009c4a0 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973dd74640 1 -- 10.20.193.78:0/861274265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974009c4a0 msgr2=0x7f974006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973dd74640 1 --2- 10.20.193.78:0/861274265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974009c4a0 0x7f974006d1b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f974006d870 tx=0x7f972802ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4279a87d789094c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 1 -- 10.20.193.78:0/861274265 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972803c070 con 0x7f974009c4a0 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: dump: 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.464 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: _finish_auth 0 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: _check_auth_tickets 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f9744ada640 5 monclient: authenticate success, global_id 264425 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 1 -- 10.20.193.78:0/861274265 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972802fcc0 con 0x7f974009c4a0 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f9744ada640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 1 -- 10.20.193.78:0/861274265 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728037660 con 0x7f974009c4a0 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f9744ada640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f9744ada640 4 set_mon_vals no callback set 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f973d573640 10 monclient: dump: 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.465 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f9744ada640 10 monclient: shutdown 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f9744ada640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f9744ada640 1 -- 10.20.193.78:0/861274265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974009c4a0 msgr2=0x7f974006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.460+0000 7f9744ada640 1 --2- 10.20.193.78:0/861274265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974009c4a0 0x7f974006d1b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f974006d870 tx=0x7f972802ee80 comp rx=0 tx=0).stop 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 1 -- 10.20.193.78:0/861274265 shutdown_connections 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 1 --2- 10.20.193.78:0/861274265 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974009c4a0 0x7f974006d1b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 1 --2- 10.20.193.78:0/861274265 >> v2:10.20.193.115:3300/0 conn(0x7f974006c890 0x7f974006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 1 -- 10.20.193.78:0/861274265 >> 10.20.193.78:0/861274265 conn(0x7f97400a6620 msgr2=0x7f97400a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 1 -- 10.20.193.78:0/861274265 shutdown_connections 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 1 -- 10.20.193.78:0/861274265 wait complete. 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 10 monclient: build_initial_monmap 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 10 monclient: monmap: 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 1 Processor -- start 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 1 -- start start 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.461+0000 7f9744ada640 10 monclient: init 2026-02-17T06:10:07.466 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974006c890 0x7f97401cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient(hunting): picked mon.noname-c con 0x7f974006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f974009c4a0 0x7f9740121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient(hunting): picked mon.noname-a con 0x7f974009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9740134df0 0x7f9740121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient(hunting): picked mon.noname-b con 0x7f9740134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97400a01a0 con 0x7f974009c4a0 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f97400a0020 con 0x7f9740134df0 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f974009be20 con 0x7f974006c890 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f9744ada640 10 monclient(hunting): authenticate will time out at 2053.016297s 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974006c890 0x7f97401cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974006c890 0x7f97401cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37306/0 (socket says 10.20.193.78:37306) 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 -- 10.20.193.78:0/190859517 learned_addr learned my addr 10.20.193.78:0/190859517 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient(hunting): get_auth_request con 0x7f974006c890 auth_method 0 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973dd74640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f974009c4a0 0x7f9740121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973dd74640 10 monclient(hunting): get_auth_request con 0x7f974009c4a0 auth_method 0 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973dd74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient(hunting): handle_auth_done global_id 264428 payload 824 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 -- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9740134df0 msgr2=0x7f9740121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973ed76640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9740134df0 0x7f9740121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9740134df0 0x7f9740121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 -- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f974009c4a0 msgr2=0x7f9740121760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f974009c4a0 0x7f9740121760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient: _finish_hunting 0 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 monclient: found mon.noname-c 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973e575640 1 -- 10.20.193.78:0/190859517 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97401260d0 con 0x7f974006c890 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973ed76640 10 monclient: get_auth_request con 0x7f9740134df0 auth_method 0 2026-02-17T06:10:07.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.462+0000 7f973ed76640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9740134df0 0x7f9740121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:10:07.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f973e575640 1 -- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974006c890 msgr2=0x7f97401cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f973e575640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974006c890 0x7f97401cd680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f973400e840 tx=0x7f973400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 1 -- 10.20.193.78:0/190859517 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734017070 con 0x7f974006c890 2026-02-17T06:10:07.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:10:07.469 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: dump: 2026-02-17T06:10:07.469 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: _finish_auth 0 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 1 -- 10.20.193.78:0/190859517 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973400cae0 con 0x7f974006c890 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9744ada640 5 monclient: authenticate success, global_id 264428 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9744ada640 10 monclient: _renew_subs 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9744ada640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9740125410 con 0x7f974006c890 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 1 -- 10.20.193.78:0/190859517 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973400f040 con 0x7f974006c890 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9726ffd640 10 monclient: dump: 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.470 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f973cd72640 4 set_mon_vals no callback set 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9744ada640 10 monclient: _renew_subs 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9744ada640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.464+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9740072760 con 0x7f974006c890 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f9744ada640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f9744ada640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f9744ada640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9740074980 con 0x7f974006c890 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f9726ffd640 1 -- 10.20.193.78:0/190859517 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f9734012050 con 0x7f974006c890 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f9726ffd640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f970c03deb0 0x7f970c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f9726ffd640 1 -- 10.20.193.78:0/190859517 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(855..855 src has 525..855) ==== 4779+0+0 (secure 0 0 0) 0x7f97340525c0 con 0x7f974006c890 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f973dd74640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f970c03deb0 0x7f970c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.465+0000 7f973dd74640 10 monclient: get_auth_request con 0x7f970c03deb0 auth_method 0 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.468+0000 7f9726ffd640 1 -- 10.20.193.78:0/190859517 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f97401260d0 con 0x7f974006c890 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.468+0000 7f9726ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.468+0000 7f9726ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:07.471 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.468+0000 7f973dd74640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f970c03deb0 0x7f970c040370 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9728036000 tx=0x7f9728033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.505 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.503+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.505 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.503+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=621734670}) 2026-02-17T06:10:07.505 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.503+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2279808987}) 2026-02-17T06:10:07.505 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.503+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.505 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.503+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=621734670}) 2026-02-17T06:10:07.505 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.503+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4014454777}) 2026-02-17T06:10:07.511 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.510+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.511 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.510+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1173074390}) 2026-02-17T06:10:07.511 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.510+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2592963353}) 2026-02-17T06:10:07.512 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.510+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.512 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.510+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1173074390}) 2026-02-17T06:10:07.512 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.510+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3778399566}) 2026-02-17T06:10:07.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.580+0000 7f9744ada640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:10:07.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.580+0000 7f9744ada640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:10:07.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.580+0000 7f9744ada640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.581 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.580+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f97400a17b0 con 0x7f974006c890 2026-02-17T06:10:07.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.580+0000 7f9726ffd640 1 -- 10.20.193.78:0/190859517 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f9734023d80 con 0x7f974006c890 2026-02-17T06:10:07.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.580+0000 7f9726ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:10:07.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.580+0000 7f9726ffd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:07.582 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":264,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:10:07.582 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.582 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3979356277}) 2026-02-17T06:10:07.583 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1920745035}) 2026-02-17T06:10:07.583 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.583 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3979356277}) 2026-02-17T06:10:07.583 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1724723721}) 2026-02-17T06:10:07.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f970c03deb0 msgr2=0x7f970c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f970c03deb0 0x7f970c040370 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9728036000 tx=0x7f9728033000 comp rx=0 tx=0).stop 2026-02-17T06:10:07.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 10 monclient: shutdown 2026-02-17T06:10:07.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:07.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974006c890 msgr2=0x7f97401cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974006c890 0x7f97401cd680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f973400e840 tx=0x7f973400ed10 comp rx=0 tx=0).stop 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 shutdown_connections 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f970c03deb0 0x7f970c040370 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9740134df0 0x7f9740121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f974009c4a0 0x7f9740121760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 --2- 10.20.193.78:0/190859517 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f974006c890 0x7f97401cd680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 >> 10.20.193.78:0/190859517 conn(0x7f97400a6620 msgr2=0x7f9740135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 shutdown_connections 2026-02-17T06:10:07.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.581+0000 7f9744ada640 1 -- 10.20.193.78:0/190859517 wait complete. 2026-02-17T06:10:07.589 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:10:07.589 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-17T06:10:07.589 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:10:07.589 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:10:07.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient: get_monmap_and_config 2026-02-17T06:10:07.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient: build_initial_monmap 2026-02-17T06:10:07.656 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient: monmap: 2026-02-17T06:10:07.656 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:07.656 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.656 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:07.656321+0000 2026-02-17T06:10:07.656 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:07.656321+0000 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 1 Processor -- start 2026-02-17T06:10:07.657 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 1 -- start start 2026-02-17T06:10:07.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient: init 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient: _add_conns ranks=[2,0,4,3,1] 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c06c890 0x7f694c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient(hunting): picked mon.noname-b con 0x7f694c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f694c09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f694c06d1b0 0x7f694c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient(hunting): picked mon.noname-c con 0x7f694c06d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f694c06daf0 con 0x7f694c09c4a0 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f694c0a0880 con 0x7f694c06d1b0 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f694c09cc80 con 0x7f694c06c890 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f695360c640 10 monclient(hunting): authenticate will time out at 2053.209825s 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f6951381640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c06c890 0x7f694c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f6951381640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c06c890 0x7f694c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37308/0 (socket says 10.20.193.78:37308) 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f6951381640 1 -- 10.20.193.78:0/1046664262 learned_addr learned my addr 10.20.193.78:0/1046664262 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f6951381640 10 monclient(hunting): get_auth_request con 0x7f694c06c890 auth_method 0 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f6951381640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f6951381640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.655+0000 7f6951381640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951b82640 1 --2- 10.20.193.78:0/1046664262 >> v2:10.20.193.115:3300/0 conn(0x7f694c06d1b0 0x7f694c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951b82640 10 monclient(hunting): get_auth_request con 0x7f694c06d1b0 auth_method 0 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951b82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951b82640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951b82640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 10 monclient(hunting): handle_auth_done global_id 264431 payload 274 2026-02-17T06:10:07.671 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 1 -- 10.20.193.78:0/1046664262 >> v2:10.20.193.115:3300/0 conn(0x7f694c06d1b0 msgr2=0x7f694c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 1 --2- 10.20.193.78:0/1046664262 >> v2:10.20.193.115:3300/0 conn(0x7f694c06d1b0 0x7f694c135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 1 -- 10.20.193.78:0/1046664262 >> v1:10.20.193.78:6789/0 conn(0x7f694c09c4a0 legacy=0x7f694c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 10 monclient: _finish_hunting 0 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 1 monclient: found mon.noname-b 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.656+0000 7f6951381640 1 -- 10.20.193.78:0/1046664262 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c09cf60 con 0x7f694c06c890 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f6951381640 1 -- 10.20.193.78:0/1046664262 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c06c890 msgr2=0x7f694c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f6951381640 1 --2- 10.20.193.78:0/1046664262 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c06c890 0x7f694c06cc70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f693c009970 tx=0x7f693c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1bdbac7f76f1952 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 1 -- 10.20.193.78:0/1046664262 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c03c070 con 0x7f694c06c890 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.672 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: dump: 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.673 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: _finish_auth 0 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 1 -- 10.20.193.78:0/1046664262 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693c02fcd0 con 0x7f694c06c890 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f695360c640 5 monclient: authenticate success, global_id 264431 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f695360c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 1 -- 10.20.193.78:0/1046664262 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c037660 con 0x7f694c06c890 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f693bfff640 10 monclient: dump: 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.674 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f695360c640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.658+0000 7f695360c640 4 set_mon_vals no callback set 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 10 monclient: shutdown 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 -- 10.20.193.78:0/1046664262 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c06c890 msgr2=0x7f694c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.675 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 --2- 10.20.193.78:0/1046664262 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c06c890 0x7f694c06cc70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f693c009970 tx=0x7f693c02ee90 comp rx=0 tx=0).stop 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 -- 10.20.193.78:0/1046664262 shutdown_connections 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 --2- 10.20.193.78:0/1046664262 >> v2:10.20.193.115:3300/0 conn(0x7f694c06d1b0 0x7f694c135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 --2- 10.20.193.78:0/1046664262 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c06c890 0x7f694c06cc70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 -- 10.20.193.78:0/1046664262 >> 10.20.193.78:0/1046664262 conn(0x7f694c0a6620 msgr2=0x7f694c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 -- 10.20.193.78:0/1046664262 shutdown_connections 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 -- 10.20.193.78:0/1046664262 wait complete. 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 10 monclient: build_initial_monmap 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 10 monclient: monmap: 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:07.676 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 Processor -- start 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 1 -- start start 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.659+0000 7f695360c640 10 monclient: init 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f694c06c890 0x7f694c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient(hunting): picked mon.noname-b con 0x7f694c06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c06d1b0 0x7f694c1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient(hunting): picked mon.noname-a con 0x7f694c06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c09c4a0 0x7f694c09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.677 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient(hunting): picked mon.noname-c con 0x7f694c09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f694c0a01a0 con 0x7f694c06d1b0 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f694c0a0020 con 0x7f694c06c890 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f694c09be20 con 0x7f694c09c4a0 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f695360c640 10 monclient(hunting): authenticate will time out at 2053.214229s 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c06d1b0 0x7f694c1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c09c4a0 0x7f694c09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c06d1b0 0x7f694c1cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38070/0 (socket says 10.20.193.78:38070) 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 1 -- 10.20.193.78:0/874725991 learned_addr learned my addr 10.20.193.78:0/874725991 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 10 monclient(hunting): get_auth_request con 0x7f694c06d1b0 auth_method 0 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.678 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient(hunting): get_auth_request con 0x7f694c09c4a0 auth_method 0 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951381640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f694c06c890 0x7f694c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6950b80640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951381640 10 monclient(hunting): get_auth_request con 0x7f694c06c890 auth_method 0 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951381640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951381640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.679 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951381640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient(hunting): handle_auth_done global_id 264434 payload 824 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f694c06c890 msgr2=0x7f694c1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f694c06c890 0x7f694c1cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c06d1b0 msgr2=0x7f694c1cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c06d1b0 0x7f694c1cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient: _finish_hunting 0 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 1 monclient: found mon.noname-c 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.660+0000 7f6951b82640 1 -- 10.20.193.78:0/874725991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c136ea0 con 0x7f694c09c4a0 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6951b82640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c09c4a0 msgr2=0x7f694c09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6951b82640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c09c4a0 0x7f694c09db50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6948007d40 tx=0x7f694800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948003a90 con 0x7f694c09c4a0 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.680 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: dump: 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: _finish_auth 0 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f695360c640 5 monclient: authenticate success, global_id 264434 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6948003c50 con 0x7f694c09c4a0 2026-02-17T06:10:07.681 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948004020 con 0x7f694c09c4a0 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f693b7fe640 4 set_mon_vals no callback set 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f6939ffb640 10 monclient: dump: 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f695360c640 10 monclient: _renew_subs 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f695360c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.682 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694c06fe40 con 0x7f694c09c4a0 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f695360c640 10 monclient: _renew_subs 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f695360c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.662+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f694c129090 con 0x7f694c09c4a0 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f695360c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f695360c640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e090 0x7f694c09e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f695360c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f694c074980 con 0x7f694c09e090 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e090 0x7f694c09e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 10 monclient: get_auth_request con 0x7f694c09e090 auth_method 0 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.683 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6948004240 con 0x7f694c09c4a0 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 10 monclient(hunting): handle_auth_done global_id 284247 payload 824 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6951381640 1 -- 10.20.193.78:0/874725991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f694c127ec0 con 0x7f694c09e090 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6939ffb640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f691c03deb0 0x7f691c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(855..855 src has 525..855) ==== 4779+0+0 (secure 0 0 0) 0x7f6948052b50 con 0x7f694c09c4a0 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6950b80640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f691c03deb0 0x7f691c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.663+0000 7f6950b80640 10 monclient: get_auth_request con 0x7f691c03deb0 auth_method 0 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6951381640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e090 msgr2=0x7f694c09e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6951381640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e090 0x7f694c09e470 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f694c0a7600 tx=0x7f693c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c03c040 con 0x7f694c09e090 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 10 monclient: dump: 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.684 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f693c041660 con 0x7f694c09e090 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e090 msgr2=0x7f694c09e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e090 0x7f694c09e470 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f694c0a7600 tx=0x7f693c004830 comp rx=0 tx=0).stop 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e090 msgr2=0x7f694c09e470 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.665+0000 7f6939ffb640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e090 0x7f694c09e470 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f694c0a7600 tx=0x7f693c004830 comp rx=0 tx=0).stop 2026-02-17T06:10:07.685 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.666+0000 7f6950b80640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f691c03deb0 0x7f691c040370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f694c09f310 tx=0x7f6940003960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f695360c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f695360c640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e9b0 0x7f694c1da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f695360c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f694c127c40 con 0x7f694c09e9b0 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f6951381640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e9b0 0x7f694c1da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f6951381640 10 monclient: get_auth_request con 0x7f694c09e9b0 auth_method 0 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f6951381640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f6951381640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f6951381640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.687 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.668+0000 7f6951381640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.669+0000 7f6951381640 10 monclient(hunting): handle_auth_done global_id 284250 payload 824 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.669+0000 7f6951381640 1 -- 10.20.193.78:0/874725991 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f694c0744d0 con 0x7f694c09e9b0 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6951381640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e9b0 msgr2=0x7f694c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6951381640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e9b0 0x7f694c1da560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f693c002410 tx=0x7f693c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c03c040 con 0x7f694c09e9b0 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 10 monclient: dump: 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.688 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7f693c0423f0 con 0x7f694c09e9b0 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e9b0 msgr2=0x7f694c1da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e9b0 0x7f694c1da560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f693c002410 tx=0x7f693c008030 comp rx=0 tx=0).stop 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e9b0 msgr2=0x7f694c1da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6939ffb640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f694c09e9b0 0x7f694c1da560 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f693c002410 tx=0x7f693c008030 comp rx=0 tx=0).stop 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:10:07.689 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 264, 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 58809, 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:07.690 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.691 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:07.692 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:10:07.693 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:10:07.694 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:07.695 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:07.696 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.697 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.698 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.698 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:10:07.698 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f691c03deb0 msgr2=0x7f691c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f691c03deb0 0x7f691c040370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f694c09f310 tx=0x7f6940003960 comp rx=0 tx=0).stop 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 10 monclient: shutdown 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c09c4a0 msgr2=0x7f694c09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c09c4a0 0x7f694c09db50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6948007d40 tx=0x7f694800e4b0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f6951b82640 1 -- 10.20.193.78:0/874725991 reap_dead start 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 shutdown_connections 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 --2- 10.20.193.78:0/874725991 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f694c09c4a0 0x7f694c09db50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 >> 10.20.193.78:0/874725991 conn(0x7f694c0a6620 msgr2=0x7f694c072240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 shutdown_connections 2026-02-17T06:10:07.699 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.671+0000 7f695360c640 1 -- 10.20.193.78:0/874725991 wait complete. 2026-02-17T06:10:07.699 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:10:07.699 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:10:07.700 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:10:07.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient: get_monmap_and_config 2026-02-17T06:10:07.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient: build_initial_monmap 2026-02-17T06:10:07.767 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient: monmap: 2026-02-17T06:10:07.767 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:07.767 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.767 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:07.765418+0000 2026-02-17T06:10:07.767 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:07.765418+0000 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 1 Processor -- start 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 1 -- start start 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient: init 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fbaa406c890 0x7fbaa406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.768 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient(hunting): picked mon.noname-a con 0x7fbaa406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbaa409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa406d1b0 0x7fbaa4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient(hunting): picked mon.noname-b con 0x7fbaa406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa406daf0 con 0x7fbaa409c4a0 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa40a0880 con 0x7fbaa406c890 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa409cc80 con 0x7fbaa406d1b0 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.764+0000 7fbaa93ce640 10 monclient(hunting): authenticate will time out at 2053.318945s 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fbaa406c890 0x7fbaa406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa27fc640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fbaa409c4a0 0x7fbaa4134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:50832/0 (socket says 10.20.193.78:50832) 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa27fc640 1 -- 10.20.193.78:0/1946946788 learned_addr learned my addr 10.20.193.78:0/1946946788 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient(hunting): get_auth_request con 0x7fbaa406c890 auth_method 0 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.769 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 1 --2- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa406d1b0 0x7fbaa4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 10 monclient(hunting): get_auth_request con 0x7fbaa406d1b0 auth_method 0 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_done global_id 284253 payload 274 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 1 -- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa406d1b0 msgr2=0x7fbaa4135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 1 --2- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa406d1b0 0x7fbaa4135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 1 -- 10.20.193.78:0/1946946788 >> v1:10.20.193.78:6789/0 conn(0x7fbaa409c4a0 legacy=0x7fbaa4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 1 monclient: found mon.noname-a 2026-02-17T06:10:07.770 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:07.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:07.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa2ffd640 1 -- 10.20.193.78:0/1946946788 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa409cf60 con 0x7fbaa406c890 2026-02-17T06:10:07.771 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.765+0000 7fbaa37fe640 1 --2- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa406d1b0 0x7fbaa4135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa2ffd640 1 -- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406c890 msgr2=0x7fbaa406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa2ffd640 1 --2- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406c890 0x7fbaa406cc70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbaa406dcc0 tx=0x7fba9802ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2decf4d287c72ec1 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 1 -- 10.20.193.78:0/1946946788 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9803c070 con 0x7fbaa406c890 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: dump: 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.776 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: _finish_auth 0 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa93ce640 5 monclient: authenticate success, global_id 284253 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 1 -- 10.20.193.78:0/1946946788 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9802fc00 con 0x7fbaa406c890 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa93ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 1 -- 10.20.193.78:0/1946946788 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98037440 con 0x7fbaa406c890 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa93ce640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa93ce640 4 set_mon_vals no callback set 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:07.777 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.767+0000 7fbaa1ffb640 10 monclient: dump: 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 10 monclient: shutdown 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406c890 msgr2=0x7fbaa406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 --2- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406c890 0x7fbaa406cc70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbaa406dcc0 tx=0x7fba9802ed80 comp rx=0 tx=0).stop 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1946946788 shutdown_connections 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 --2- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa406d1b0 0x7fbaa4135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.778 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 --2- 10.20.193.78:0/1946946788 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406c890 0x7fbaa406cc70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1946946788 >> 10.20.193.78:0/1946946788 conn(0x7fbaa40a6620 msgr2=0x7fbaa40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1946946788 shutdown_connections 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1946946788 wait complete. 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 10 monclient: build_initial_monmap 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 10 monclient: monmap: 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 Processor -- start 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 1 -- start start 2026-02-17T06:10:07.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 10 monclient: init 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.768+0000 7fbaa93ce640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbaa406c890 0x7fbaa41cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient(hunting): picked mon.noname-b con 0x7fbaa406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406d1b0 0x7fbaa41cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient(hunting): picked mon.noname-a con 0x7fbaa406d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409c4a0 0x7fbaa409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient(hunting): picked mon.noname-c con 0x7fbaa409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40a01a0 con 0x7fbaa406d1b0 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40a0020 con 0x7fbaa406c890 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa409be20 con 0x7fbaa409c4a0 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa93ce640 10 monclient(hunting): authenticate will time out at 2053.323073s 2026-02-17T06:10:07.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406d1b0 0x7fbaa41cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406d1b0 0x7fbaa41cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38102/0 (socket says 10.20.193.78:38102) 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 -- 10.20.193.78:0/1046408296 learned_addr learned my addr 10.20.193.78:0/1046408296 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient(hunting): get_auth_request con 0x7fbaa406d1b0 auth_method 0 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa37fe640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409c4a0 0x7fbaa409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa37fe640 10 monclient(hunting): get_auth_request con 0x7fbaa409c4a0 auth_method 0 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa37fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient(hunting): handle_auth_done global_id 284256 payload 824 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409c4a0 msgr2=0x7fbaa409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409c4a0 0x7fbaa409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbaa406c890 msgr2=0x7fbaa41cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbaa406c890 0x7fbaa41cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient: _finish_hunting 0 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 monclient: found mon.noname-a 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.769+0000 7fbaa27fc640 1 -- 10.20.193.78:0/1046408296 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa4128320 con 0x7fbaa406d1b0 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa27fc640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406d1b0 msgr2=0x7fbaa41cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa27fc640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406d1b0 0x7fbaa41cf920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c780 tx=0x7fba8c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c005de0 con 0x7fbaa406d1b0 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: dump: 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.782 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: _finish_auth 0 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: _check_auth_tickets 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8c007070 con 0x7fbaa406d1b0 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c007440 con 0x7fbaa406d1b0 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa93ce640 5 monclient: authenticate success, global_id 284256 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:07.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa17fa640 4 set_mon_vals no callback set 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fba837fe640 10 monclient: dump: 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa93ce640 10 monclient: _renew_subs 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa93ce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa40a2680 con 0x7fbaa406d1b0 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa93ce640 10 monclient: _renew_subs 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa93ce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.771+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa409f7f0 con 0x7fbaa406d1b0 2026-02-17T06:10:07.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa93ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa93ce640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409e850 0x7fbaa409ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa41271a0 con 0x7fbaa409e850 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409e850 0x7fbaa409ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 10 monclient: get_auth_request con 0x7fbaa409e850 auth_method 0 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fba8c007620 con 0x7fbaa406d1b0 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_done global_id 264443 payload 824 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa2ffd640 1 -- 10.20.193.78:0/1046408296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbaa4074980 con 0x7fbaa409e850 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fba837fe640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba7003deb0 0x7fba70040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(855..855 src has 525..855) ==== 4779+0+0 (secure 0 0 0) 0x7fba8c052620 con 0x7fbaa406d1b0 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa37fe640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba7003deb0 0x7fba70040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.772+0000 7fbaa37fe640 10 monclient: get_auth_request con 0x7fba7003deb0 auth_method 0 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fbaa2ffd640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409e850 msgr2=0x7fbaa409ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fbaa2ffd640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409e850 0x7fbaa409ec30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba98039000 tx=0x7fba98037670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98046020 con 0x7fbaa409e850 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 10 monclient: dump: 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fba98037af0 con 0x7fbaa409e850 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:07.786 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:07.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409e850 msgr2=0x7fbaa409ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409e850 0x7fbaa409ec30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba98039000 tx=0x7fba98037670 comp rx=0 tx=0).stop 2026-02-17T06:10:07.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409e850 msgr2=0x7fbaa409ec30 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:07.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.774+0000 7fba837fe640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa409e850 0x7fbaa409ec30 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba98039000 tx=0x7fba98037670 comp rx=0 tx=0).stop 2026-02-17T06:10:07.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.775+0000 7fbaa37fe640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba7003deb0 0x7fba70040370 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fba94011420 tx=0x7fba9400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa93ce640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:10:07.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa93ce640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa411cc70 0x7fbaa41da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa93ce640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa40744d0 con 0x7fbaa411cc70 2026-02-17T06:10:07.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa2ffd640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa411cc70 0x7fbaa41da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa2ffd640 10 monclient: get_auth_request con 0x7fbaa411cc70 auth_method 0 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.777+0000 7fbaa2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.778+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.778+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.778+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.778+0000 7fbaa2ffd640 10 monclient(hunting): handle_auth_done global_id 264446 payload 824 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.778+0000 7fbaa2ffd640 1 -- 10.20.193.78:0/1046408296 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbaa41cfe60 con 0x7fbaa411cc70 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa2ffd640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa411cc70 msgr2=0x7fbaa41da6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa2ffd640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa411cc70 0x7fbaa41da6b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba9802edc0 tx=0x7fba98033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98046030 con 0x7fbaa411cc70 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 10 monclient: dump: 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.789 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fba9804aa30 con 0x7fbaa411cc70 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa411cc70 msgr2=0x7fbaa41da6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa411cc70 0x7fbaa41da6b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba9802edc0 tx=0x7fba98033000 comp rx=0 tx=0).stop 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa411cc70 msgr2=0x7fbaa41da6b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fba837fe640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbaa411cc70 0x7fbaa41da6b0 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba9802edc0 tx=0x7fba98033000 comp rx=0 tx=0).stop 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:07.790 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 264, 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 157165, 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:07.791 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:10:07.792 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:07.793 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.794 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.795 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.796 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.797 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba7003deb0 msgr2=0x7fba70040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fba7003deb0 0x7fba70040370 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fba94011420 tx=0x7fba9400b040 comp rx=0 tx=0).stop 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 10 monclient: shutdown 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406d1b0 msgr2=0x7fbaa41cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 1 --2- 10.20.193.78:0/1046408296 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbaa406d1b0 0x7fbaa41cf920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c780 tx=0x7fba8c00cc50 comp rx=0 tx=0).stop 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa37fe640 1 -- 10.20.193.78:0/1046408296 reap_dead start 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa37fe640 1 -- 10.20.193.78:0/1046408296 reap_dead start 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 shutdown_connections 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 >> 10.20.193.78:0/1046408296 conn(0x7fbaa40a6620 msgr2=0x7fbaa40a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 shutdown_connections 2026-02-17T06:10:07.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.780+0000 7fbaa93ce640 1 -- 10.20.193.78:0/1046408296 wait complete. 2026-02-17T06:10:07.799 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.780+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.799 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.781+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039003852}) 2026-02-17T06:10:07.799 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.781+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3521376682}) 2026-02-17T06:10:07.799 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.781+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.799 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.781+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039003852}) 2026-02-17T06:10:07.800 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.781+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=454620896}) 2026-02-17T06:10:07.800 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:10:07.800 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:10:07.800 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:10:07.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.866+0000 7f9126ae8640 10 monclient: get_monmap_and_config 2026-02-17T06:10:07.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.866+0000 7f9126ae8640 10 monclient: build_initial_monmap 2026-02-17T06:10:07.867 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.866+0000 7f9126ae8640 10 monclient: monmap: 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:07.868143+0000 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:07.868143+0000 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:07.868 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 1 Processor -- start 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 1 -- start start 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient: init 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f912006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f912009c4a0 0x7f912013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient(hunting): picked mon.noname-a con 0x7f912009c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f912013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient(hunting): picked mon.noname-b con 0x7f912006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912006daf0 con 0x7f912006c890 2026-02-17T06:10:07.869 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f91200a0880 con 0x7f912009c4a0 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f912009cc80 con 0x7f912006d380 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9126ae8640 10 monclient(hunting): authenticate will time out at 2053.421653s 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f912009c4a0 0x7f912013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f912013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f912009c4a0 0x7f912013b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38106/0 (socket says 10.20.193.78:38106) 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f912013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37352/0 (socket says 10.20.193.78:37352) 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 1 -- 10.20.193.78:0/159644142 learned_addr learned my addr 10.20.193.78:0/159644142 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient(hunting): get_auth_request con 0x7f912009c4a0 auth_method 0 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 10 monclient(hunting): get_auth_request con 0x7f912006d380 auth_method 0 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f912505e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient(hunting): handle_auth_done global_id 284259 payload 274 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 1 -- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 msgr2=0x7f912013d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 1 --2- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f912013d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 1 -- 10.20.193.78:0/159644142 >> v1:10.20.193.115:6789/0 conn(0x7f912006c890 legacy=0x7f912006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:07.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient: _finish_hunting 0 2026-02-17T06:10:07.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 1 monclient: found mon.noname-a 2026-02-17T06:10:07.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:07.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:07.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.867+0000 7f9117fff640 1 -- 10.20.193.78:0/159644142 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912009cf60 con 0x7f912009c4a0 2026-02-17T06:10:07.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.868+0000 7f912505e640 1 --2- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f912013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9117fff640 1 -- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 msgr2=0x7f912013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9117fff640 1 --2- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f912013b3a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9108009970 tx=0x7f910802ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670bbb18ca235086 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 1 -- 10.20.193.78:0/159644142 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910803c070 con 0x7f912009c4a0 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: dump: 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: _finish_auth 0 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: _check_auth_tickets 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 5 monclient: authenticate success, global_id 284259 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 1 -- 10.20.193.78:0/159644142 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910802fcd0 con 0x7f912009c4a0 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:07.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 1 -- 10.20.193.78:0/159644142 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108037660 con 0x7f912009c4a0 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 4 set_mon_vals no callback set 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f91177fe640 10 monclient: dump: 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 10 monclient: shutdown 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 1 -- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 msgr2=0x7f912013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 1 --2- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f912013b3a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9108009970 tx=0x7f910802ee90 comp rx=0 tx=0).stop 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 1 -- 10.20.193.78:0/159644142 shutdown_connections 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 1 --2- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f912013d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 1 --2- 10.20.193.78:0/159644142 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f912013b3a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 1 -- 10.20.193.78:0/159644142 >> 10.20.193.78:0/159644142 conn(0x7f91200a6620 msgr2=0x7f91200a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 1 -- 10.20.193.78:0/159644142 shutdown_connections 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.870+0000 7f9126ae8640 1 -- 10.20.193.78:0/159644142 wait complete. 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient: build_initial_monmap 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient: monmap: 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:07.873 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 1 Processor -- start 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 1 -- start start 2026-02-17T06:10:07.874 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient: init 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006c890 0x7f91201cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient(hunting): picked mon.noname-b con 0x7f912006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient(hunting): picked mon.noname-c con 0x7f912006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f912013a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient(hunting): picked mon.noname-a con 0x7f912009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f912009bfa0 con 0x7f912009c4a0 2026-02-17T06:10:07.875 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f912009be20 con 0x7f912006c890 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f912013de30 con 0x7f912006d380 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9126ae8640 10 monclient(hunting): authenticate will time out at 2053.425708s 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f912013a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:37366/0 (socket says 10.20.193.78:37366) 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f912013a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:38120/0 (socket says 10.20.193.78:38120) 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 learned_addr learned my addr 10.20.193.78:0/3332919834 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient(hunting): get_auth_request con 0x7f912006d380 auth_method 0 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 10 monclient(hunting): get_auth_request con 0x7f912009c4a0 auth_method 0 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient(hunting): handle_auth_done global_id 264452 payload 824 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006c890 msgr2=0x7f91201cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006c890 0x7f91201cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 msgr2=0x7f912013a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f912013a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient: _finish_hunting 0 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 monclient: found mon.noname-c 2026-02-17T06:10:07.876 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:07.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:10:07.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912011dab0 con 0x7f912006d380 2026-02-17T06:10:07.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f912505e640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f912013a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:10:07.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 msgr2=0x7f91201333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:10:07.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 msgr2=0x7f91201333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:10:07.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f9108009970 tx=0x7f9108004830 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:07.877 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:07.871+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.877+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.877+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=290059826}) 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.877+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3051628463}) 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.877+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.877+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=290059826}) 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.877+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=178915216}) 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=40489297}) 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094987180}) 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=40489297}) 2026-02-17T06:10:07.878 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=59408988}) 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4267159846}) 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=21917188}) 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4267159846}) 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2639452832}) 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2032589344}) 2026-02-17T06:10:07.879 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1648514408}) 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2032589344}) 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.878+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3004295462}) 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3845767308}) 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3216930381}) 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3845767308}) 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4069970310}) 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.880 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3300471643}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2019884290}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3300471643}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229030204}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2146983805}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1521477449}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2146983805}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.879+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435044031}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1741323645}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=662084507}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1741323645}) 2026-02-17T06:10:07.881 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1661818162}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823634986}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3382580341}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823634986}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1341528110}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129262483}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1170275355}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129262483}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.880+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4101676035}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1563993761}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=141796403}) 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.882 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1563993761}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2511528722}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2519939843}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2507770073}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2519939843}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2582974847}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=981441968}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700111722}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=981441968}) 2026-02-17T06:10:07.883 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.881+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=618009733}) 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=386796912}) 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3189937627}) 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=386796912}) 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1924780611}) 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2220197565}) 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4093158480}) 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.884 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2220197565}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2976588550}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2136439329}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3188411715}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2136439329}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=440767435}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434074794}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3386559622}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434074794}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.882+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1166084002}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3334308955}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3861529887}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3334308955}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2741429168}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537572812}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2868313087}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537572812}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1184999373}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3697030523}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83454694}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3697030523}) 2026-02-17T06:10:07.885 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.883+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2551326646}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4078555958}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3598451749}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4078555958}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3842610726}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=309264635}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=278514224}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=309264635}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2318653663}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1750477194}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=569165854}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1750477194}) 2026-02-17T06:10:07.887 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.884+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3687673878}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1941020296}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1436696314}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1941020296}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1768957092}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156495970}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2276061117}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156495970}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=909879326}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186704196}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3551870303}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186704196}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4253787473}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4280685293}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.885+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=400130531}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4280685293}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1371527439}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2507423971}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=285676024}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2507423971}) 2026-02-17T06:10:07.888 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4079714725}) 2026-02-17T06:10:07.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131788278}) 2026-02-17T06:10:07.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=944953005}) 2026-02-17T06:10:07.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131788278}) 2026-02-17T06:10:07.889 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2188046255}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4094616034}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1830155200}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4094616034}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.886+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2665627653}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4198892749}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2391688627}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4198892749}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2308997051}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3083135082}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3046565464}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3083135082}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2725743071}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3725716979}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2609314383}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3725716979}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.887+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1858549246}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.888+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.888+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=977486598}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.888+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3153255581}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.888+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.888+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=977486598}) 2026-02-17T06:10:07.890 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.888+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=561775163}) 2026-02-17T06:10:07.895 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.894+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.895 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.894+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1065302850}) 2026-02-17T06:10:07.895 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.894+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3946433129}) 2026-02-17T06:10:07.895 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.894+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.895 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.894+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1065302850}) 2026-02-17T06:10:07.895 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.894+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2167738274}) 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1764383688}) 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2092643760}) 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1764383688}) 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=339860740}) 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3346115156}) 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2798683268}) 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3346115156}) 2026-02-17T06:10:07.896 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=482307485}) 2026-02-17T06:10:07.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1410263668}) 2026-02-17T06:10:07.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.895+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3827124760}) 2026-02-17T06:10:07.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.898 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1410263668}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1804368225}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765066490}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3681141711}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765066490}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2339262067}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3779739327}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3553370640}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3779739327}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1428028481}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2970132459}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2378987355}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2970132459}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.896+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1377241684}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=880549278}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3700622899}) 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.899 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=880549278}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=971846131}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86952870}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1739952577}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86952870}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=964131009}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362438378}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=106926485}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362438378}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.897+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3018073078}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3006754373}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3321185721}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3006754373}) 2026-02-17T06:10:07.900 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2540734537}) 2026-02-17T06:10:07.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2539837594}) 2026-02-17T06:10:07.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825723624}) 2026-02-17T06:10:07.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:10:07.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2539837594}) 2026-02-17T06:10:07.901 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:10:07.898+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=390195218}) 2026-02-17T06:10:08.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:08.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 10 monclient: get_auth_request con 0x7f912006d380 auth_method 0 2026-02-17T06:10:08.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 0 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-17T06:10:08.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:08.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient: ms_handle_reset current mon [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:08.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 msgr2=0x7f91201333e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f912006d380 0x7f91201333e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90f00035a0 0x7f90f00059f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient(hunting): picked mon.noname-c con 0x7f90f00035a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f90f0008570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient(hunting): picked mon.noname-a con 0x7f912009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006c890 0x7f90f000b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient(hunting): picked mon.noname-b con 0x7f912006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90f000ba50 con 0x7f912009c4a0 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90f000bc00 con 0x7f912006c890 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f90f000be00 con 0x7f90f00035a0 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f91157fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90f00035a0 0x7f90f00059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f90f0008570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient(hunting): get_auth_request con 0x7f90f00035a0 auth_method 0 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 10 monclient(hunting): get_auth_request con 0x7f912009c4a0 auth_method 0 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912505e640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006c890 0x7f90f000b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912505e640 10 monclient(hunting): get_auth_request con 0x7f912006c890 auth_method 0 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912505e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912505e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912505e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient(hunting): handle_auth_done global_id 264452 payload 293 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006c890 msgr2=0x7f90f000b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f912006c890 0x7f90f000b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 msgr2=0x7f90f0008570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:08.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f90f0008570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient: _finish_hunting 0 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 1 monclient: found mon.noname-c 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f912485d640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f000c0c0 con 0x7f90f00035a0 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.072+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f912009c4a0 0x7f90f0008570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f912485d640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90f00035a0 msgr2=0x7f90f00059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f912485d640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90f00035a0 0x7f90f00059f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9110003f80 tx=0x7f9108039040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110003b50 con 0x7f90f00035a0 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: dump: 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:08.075 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: _finish_auth 0 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: _check_auth_tickets 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9110005460 con 0x7f90f00035a0 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911000c8e0 con 0x7f90f00035a0 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f9116ffd640 4 set_mon_vals no callback set 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f91157fa640 10 monclient: dump: 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f9126ae8640 5 monclient: authenticate success, global_id 264452 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f9126ae8640 10 monclient: _renew_subs 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f9126ae8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912011dd90 con 0x7f90f00035a0 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f9126ae8640 10 monclient: _renew_subs 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f9126ae8640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:08.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.074+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f912013da40 con 0x7f90f00035a0 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9126ae8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9126ae8640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91201cd150 0x7f91201333e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9126ae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f912011d730 con 0x7f91201cd150 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f911000cb40 con 0x7f90f00035a0 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91201cd150 0x7f91201333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 10 monclient: get_auth_request con 0x7f91201cd150 auth_method 0 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f912006d380 0x7f90f0048750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(855..855 src has 525..855) ==== 4779+0+0 (secure 0 0 0) 0x7f9110050df0 con 0x7f90f00035a0 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f912505e640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f912006d380 0x7f90f0048750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f912505e640 10 monclient: get_auth_request con 0x7f912006d380 auth_method 0 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 10 monclient(hunting): handle_auth_done global_id 294139 payload 824 2026-02-17T06:10:08.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.075+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9120070700 con 0x7f91201cd150 2026-02-17T06:10:08.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f912505e640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f912006d380 0x7f90f0048750 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f911c009bd0 tx=0x7f911c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:08.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91201cd150 msgr2=0x7f91201333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:08.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91201cd150 0x7f91201333e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f910802eed0 tx=0x7f910803f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910803fe20 con 0x7f91201cd150 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 10 monclient: dump: 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f910802fcb0 con 0x7f91201cd150 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91201cd150 msgr2=0x7f91201333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91201cd150 0x7f91201333e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f910802eed0 tx=0x7f910803f690 comp rx=0 tx=0).stop 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91201cd150 msgr2=0x7f91201333e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:08.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.078+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f91201cd150 0x7f91201333e0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f910802eed0 tx=0x7f910803f690 comp rx=0 tx=0).stop 2026-02-17T06:10:08.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.081+0000 7f9126ae8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:10:08.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.081+0000 7f9126ae8640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9120133920 0x7f91201d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:08.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.081+0000 7f9126ae8640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.081+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9120051260 con 0x7f9120133920 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9120133920 0x7f91201d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 10 monclient: get_auth_request con 0x7f9120133920 auth_method 0 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 10 monclient(hunting): handle_auth_done global_id 294142 payload 824 2026-02-17T06:10:08.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.082+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9120074980 con 0x7f9120133920 2026-02-17T06:10:08.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.084+0000 7f9117fff640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9120133920 msgr2=0x7f91201d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:08.085 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.084+0000 7f9117fff640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9120133920 0x7f91201d24b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91201cad10 tx=0x7f9108004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:08.087 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108009590 con 0x7f9120133920 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 10 monclient: dump: 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:08.088 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f9108009590 con 0x7f9120133920 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9120133920 msgr2=0x7f91201d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9120133920 0x7f91201d24b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91201cad10 tx=0x7f9108004830 comp rx=0 tx=0).stop 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9120133920 msgr2=0x7f91201d24b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f91157fa640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f9120133920 0x7f91201d24b0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91201cad10 tx=0x7f9108004830 comp rx=0 tx=0).stop 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:08.089 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 264, 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 8788, 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 3, 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:08.090 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:10:08.091 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:08.092 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:10:08.093 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:08.094 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:08.095 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:08.096 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:08.097 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:08.098 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:10:08.099 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f912006d380 msgr2=0x7f90f0048750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f9126ae8640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f912006d380 0x7f90f0048750 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f911c009bd0 tx=0x7f911c00b040 comp rx=0 tx=0).stop 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f9126ae8640 10 monclient: shutdown 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f9126ae8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90f00035a0 msgr2=0x7f90f00059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f9126ae8640 1 --2- 10.20.193.78:0/3332919834 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f90f00035a0 0x7f90f00059f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9110003f80 tx=0x7f9108039040 comp rx=0 tx=0).stop 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.086+0000 7f912505e640 1 -- 10.20.193.78:0/3332919834 reap_dead start 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.087+0000 7f912505e640 1 -- 10.20.193.78:0/3332919834 reap_dead start 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.087+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 shutdown_connections 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.087+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 >> 10.20.193.78:0/3332919834 conn(0x7f91200a6620 msgr2=0x7f91200a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.087+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 shutdown_connections 2026-02-17T06:10:08.101 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:08.087+0000 7f9126ae8640 1 -- 10.20.193.78:0/3332919834 wait complete. 2026-02-17T06:10:08.101 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:10:08.102 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-17T06:10:08.102 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2026-02-17T06:10:08.102 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-17T06:10:08.102 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:10:08.102 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T06:10:08.102 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:10:08.110 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T06:10:08.110 INFO:tasks.mon_thrash:Sending STOP to mon a 2026-02-17T06:10:08.111 INFO:tasks.ceph.mon.a:Sent signal 19 2026-02-17T06:10:08.111 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T06:10:23.111 INFO:tasks.mon_thrash:Sending CONT to mon a 2026-02-17T06:10:23.112 INFO:tasks.ceph.mon.a:Sent signal 18 2026-02-17T06:10:23.112 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-17T06:10:23.112 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:10:23.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient: get_monmap_and_config 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient: build_initial_monmap 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient: monmap: 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:23.182314+0000 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:23.182314+0000 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:23.182 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 1 Processor -- start 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 1 -- start start 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient: init 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f270c06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f270c06d380 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 0x7f270c13d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient(hunting): picked mon.noname-c con 0x7f270c0a01f0 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270c0a0600 con 0x7f270c06d380 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270c074150 con 0x7f270c06c890 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f270c09ef70 con 0x7f270c0a01f0 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271370a640 10 monclient(hunting): authenticate will time out at 2068.735841s 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f271147f640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f270c06c890 legacy=0x7f270c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f2710c7e640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f270c06d380 0x7f270c13a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:39812/0 (socket says 10.20.193.78:39812) 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 learned_addr learned my addr 10.20.193.78:0/3940802839 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f2711c80640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 msgr2=0x7f270c13d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:23.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.182+0000 7f2711c80640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 0x7f270c13d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:23.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.185+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v1:10.20.193.78:6789/0 conn(0x7f270c06d380 legacy=0x7f270c13a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:23.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.383+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> v1:10.20.193.115:6789/0 conn(0x7f270c06c890 legacy=0x7f270c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:10:23.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.383+0000 7f2711c80640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 msgr2=0x7f270c13d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:23.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.383+0000 7f2711c80640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 0x7f270c13d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:10:23.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.783+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> v1:10.20.193.115:6789/0 conn(0x7f270c06c890 legacy=0x7f270c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:10:23.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.783+0000 7f2711c80640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 msgr2=0x7f270c13d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:23.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:23.783+0000 7f2711c80640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 0x7f270c13d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:10:24.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:24.584+0000 7f2711c80640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 msgr2=0x7f270c13d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:24.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:24.584+0000 7f2711c80640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 0x7f270c13d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:10:24.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:24.584+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> v1:10.20.193.115:6789/0 conn(0x7f270c06c890 legacy=0x7f270c06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:10:26.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient(hunting): tick 2026-02-17T06:10:26.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:10:26.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 monclient(hunting): continuing hunt 2026-02-17T06:10:26.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:10:26.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 msgr2=0x7f270c13d970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.115:3300/0 conn(0x7f270c0a01f0 0x7f270c13d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 >> v1:10.20.193.115:6789/0 conn(0x7f270c06c890 legacy=0x7f270c06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 >> v1:10.20.193.78:6789/0 conn(0x7f270c06d380 legacy=0x7f270c13a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient: _add_conns ranks=[2,3] 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient(hunting): picked mon.noname-b con 0x7f26f0003840 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient(hunting): picked mon.noname-a con 0x7f26f00062d0 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:26.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:26.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f270c09ef70 con 0x7f26f00062d0 2026-02-17T06:10:26.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:26.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f26f0008e50 con 0x7f26f0003840 2026-02-17T06:10:26.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f26faffd640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:26.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.182+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:26.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f2710c7e640 10 monclient(hunting): get_auth_request con 0x7f26f00062d0 auth_method 0 2026-02-17T06:10:26.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:26.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f2710c7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:26.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f2710c7e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:26.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f2710c7e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:26.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 10 monclient(hunting): get_auth_request con 0x7f26f0003840 auth_method 0 2026-02-17T06:10:26.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:26.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:26.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.183+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:26.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:26.384 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 10 monclient(hunting): get_auth_request con 0x7f26f00062d0 auth_method 0 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 10 monclient(hunting): get_auth_request con 0x7f26f0003840 auth_method 0 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:26.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:26.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:26.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:10:26.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:26.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:26.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.383+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 10 monclient(hunting): get_auth_request con 0x7f26f00062d0 auth_method 0 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 10 monclient(hunting): get_auth_request con 0x7f26f0003840 auth_method 0 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:26.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:26.784+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-17T06:10:27.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.584+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.584+0000 7f2710c7e640 10 monclient(hunting): get_auth_request con 0x7f26f00062d0 auth_method 0 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.584+0000 7f2710c7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.584+0000 7f2710c7e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.584+0000 7f2710c7e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.584+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.584+0000 7f2710c7e640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.584+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:27.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 10 monclient(hunting): get_auth_request con 0x7f26f0003840 auth_method 0 2026-02-17T06:10:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:10:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-17T06:10:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-17T06:10:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:10:27.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:27.585+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): tick 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): _check_auth_tickets 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 monclient(hunting): continuing hunt 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): _reopen_session rank -1 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 msgr2=0x7f26f0005c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0003840 0x7f26f0005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 msgr2=0x7f26f0008790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f00062d0 0x7f26f0008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2711c80640 1 -- 10.20.193.78:0/3940802839 reap_dead start 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f0003c80 0x7f26f0006780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): picked mon.noname-a con 0x7f26f0003c80 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f26f00070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): picked mon.noname-b con 0x7f26f0006cc0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f26f0007610 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270c09f250 con 0x7f26f0007610 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f270c09ef70 con 0x7f26f0003c80 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 1 -- 10.20.193.78:0/3940802839 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f26f0008e50 con 0x7f26f0006cc0 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f26faffd640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f26f00070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> v2:10.20.193.78:3300/0 conn(0x7f26f0003c80 0x7f26f0006780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 10 monclient(hunting): get_auth_request con 0x7f26f0006cc0 auth_method 0 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient(hunting): get_auth_request con 0x7f26f0003c80 auth_method 0 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient(hunting): handle_auth_done global_id 284265 payload 274 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 msgr2=0x7f26f00070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f26f00070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> v1:10.20.193.78:6789/0 conn(0x7f26f0007610 legacy=0x7f26f0007ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient: _finish_hunting 0 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 1 monclient: found mon.noname-a 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c074150 con 0x7f26f0003c80 2026-02-17T06:10:29.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.183+0000 7f2710c7e640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f26f00070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:10:29.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.185+0000 7f271147f640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 msgr2=0x7f26f0006780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.185+0000 7f271147f640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 0x7f26f0006780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f26fc003a30 tx=0x7f26fc030080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34a2be627ea172b server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 1 -- 10.20.193.78:0/3940802839 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc03d070 con 0x7f26f0003c80 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: dump: 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: _finish_auth 0 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: _check_auth_tickets 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 1 -- 10.20.193.78:0/3940802839 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26fc038440 con 0x7f26f0003c80 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 5 monclient: authenticate success, global_id 284265 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 1 -- 10.20.193.78:0/3940802839 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc038830 con 0x7f26f0003c80 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 4 set_mon_vals no callback set 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f26fbfff640 10 monclient: dump: 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 10 monclient: shutdown 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 1 -- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 msgr2=0x7f26f0006780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 0x7f26f0006780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f26fc003a30 tx=0x7f26fc030080 comp rx=0 tx=0).stop 2026-02-17T06:10:29.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 1 -- 10.20.193.78:0/3940802839 shutdown_connections 2026-02-17T06:10:29.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f26f00070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 1 --2- 10.20.193.78:0/3940802839 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 0x7f26f0006780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 1 -- 10.20.193.78:0/3940802839 >> 10.20.193.78:0/3940802839 conn(0x7f270c0a6620 msgr2=0x7f270c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:29.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 1 -- 10.20.193.78:0/3940802839 shutdown_connections 2026-02-17T06:10:29.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 1 -- 10.20.193.78:0/3940802839 wait complete. 2026-02-17T06:10:29.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 10 monclient: build_initial_monmap 2026-02-17T06:10:29.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.186+0000 7f271370a640 10 monclient: monmap: 2026-02-17T06:10:29.187 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 1 Processor -- start 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 1 -- start start 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient: init 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 0x7f270c1cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient(hunting): picked mon.noname-a con 0x7f26f0003c80 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f270c1350f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient(hunting): picked mon.noname-c con 0x7f26f0006cc0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26f0007610 0x7f270c135630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient(hunting): picked mon.noname-b con 0x7f26f0007610 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f270c09d360 con 0x7f26f0003c80 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f270c09d1e0 con 0x7f26f0007610 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f270c06cc70 con 0x7f26f0006cc0 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271370a640 10 monclient(hunting): authenticate will time out at 2074.740394s 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f270c1350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 0x7f270c1cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f270c1350f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51378/0 (socket says 10.20.193.78:51378) 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 1 -- 10.20.193.78:0/1819509997 learned_addr learned my addr 10.20.193.78:0/1819509997 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 10 monclient(hunting): get_auth_request con 0x7f26f0006cc0 auth_method 0 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2711c80640 1 -- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26f0007610 msgr2=0x7f270c135630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient(hunting): get_auth_request con 0x7f26f0003c80 auth_method 0 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2711c80640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26f0007610 0x7f270c135630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f2710c7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient(hunting): handle_auth_done global_id 284268 payload 824 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 1 -- 10.20.193.78:0/1819509997 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 msgr2=0x7f270c1350f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f270c1350f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 1 -- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26f0007610 msgr2=0x7f270c135630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26f0007610 0x7f270c135630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient: _finish_hunting 0 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 1 monclient: found mon.noname-a 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.187+0000 7f271147f640 1 -- 10.20.193.78:0/1819509997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c129840 con 0x7f26f0003c80 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f271147f640 1 -- 10.20.193.78:0/1819509997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 msgr2=0x7f270c1cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f271147f640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 0x7f270c1cf270 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26fc03a000 tx=0x7f26fc005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.189 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 1 -- 10.20.193.78:0/1819509997 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc048020 con 0x7f26f0003c80 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: dump: 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 1 -- 10.20.193.78:0/1819509997 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26fc043880 con 0x7f26f0003c80 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f271370a640 5 monclient: authenticate success, global_id 284268 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f271370a640 10 monclient: _renew_subs 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f271370a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270c129b20 con 0x7f26f0003c80 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 1 -- 10.20.193.78:0/1819509997 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc03d040 con 0x7f26f0003c80 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26fb7fe640 4 set_mon_vals no callback set 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.189+0000 7f26f9ffb640 10 monclient: dump: 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f271370a640 10 monclient: _renew_subs 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f271370a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270c072f90 con 0x7f26f0003c80 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f26f9ffb640 1 -- 10.20.193.78:0/1819509997 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f26fc043c50 con 0x7f26f0003c80 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f271370a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f271370a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f271370a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270c074980 con 0x7f26f0003c80 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f26f9ffb640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f26dc03de60 0x7f26dc040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.190+0000 7f26f9ffb640 1 -- 10.20.193.78:0/1819509997 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(855..855 src has 525..855) ==== 4779+0+0 (secure 0 0 0) 0x7f26fc078fe0 con 0x7f26f0003c80 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.191+0000 7f2710c7e640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f26dc03de60 0x7f26dc040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.191+0000 7f2710c7e640 10 monclient: get_auth_request con 0x7f26dc03de60 auth_method 0 2026-02-17T06:10:29.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.193+0000 7f26f9ffb640 1 -- 10.20.193.78:0/1819509997 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f26fc052030 con 0x7f26f0003c80 2026-02-17T06:10:29.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.193+0000 7f26f9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:29.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.193+0000 7f26f9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:29.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.193+0000 7f2710c7e640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f26dc03de60 0x7f26dc040320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f270c129a50 tx=0x7f2700009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.307+0000 7f271370a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:10:29.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.307+0000 7f271370a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:10:29.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.307+0000 7f271370a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.307+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f270c0a2920 con 0x7f26f0003c80 2026-02-17T06:10:29.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.307+0000 7f26f9ffb640 1 -- 10.20.193.78:0/1819509997 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f26fc042df0 con 0x7f26f0003c80 2026-02-17T06:10:29.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.307+0000 7f26f9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:10:29.307 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.307+0000 7f26f9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:29.307 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":268,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f26dc03de60 msgr2=0x7f26dc040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f26dc03de60 0x7f26dc040320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f270c129a50 tx=0x7f2700009040 comp rx=0 tx=0).stop 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 10 monclient: shutdown 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 msgr2=0x7f270c1cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 0x7f270c1cf270 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26fc03a000 tx=0x7f26fc005d70 comp rx=0 tx=0).stop 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 shutdown_connections 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f26dc03de60 0x7f26dc040320 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f26f0007610 0x7f270c135630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f26f0006cc0 0x7f270c1350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 --2- 10.20.193.78:0/1819509997 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f26f0003c80 0x7f270c1cf270 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 >> 10.20.193.78:0/1819509997 conn(0x7f270c0a6620 msgr2=0x7f270c0734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 shutdown_connections 2026-02-17T06:10:29.308 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.308+0000 7f271370a640 1 -- 10.20.193.78:0/1819509997 wait complete. 2026-02-17T06:10:29.317 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-17T06:10:29.317 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-17T06:10:29.317 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.383+0000 7fdfc56fa640 10 monclient: get_monmap_and_config 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.383+0000 7fdfc56fa640 10 monclient: build_initial_monmap 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.383+0000 7fdfc56fa640 10 monclient: monmap: 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:29.383872+0000 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:29.383872+0000 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:29.383 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 1 Processor -- start 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 1 -- start start 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient: init 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient: _add_conns ranks=[2,0,3,1,4] 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006c890 0x7fdfc006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient(hunting): picked mon.noname-b con 0x7fdfc006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdfc009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdfc006d1b0 0x7fdfc0135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient(hunting): picked mon.noname-a con 0x7fdfc006d1b0 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfc006daf0 con 0x7fdfc009c4a0 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fdfc00a0880 con 0x7fdfc006d1b0 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc009cc80 con 0x7fdfc006c890 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfc56fa640 10 monclient(hunting): authenticate will time out at 2074.937509s 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdfc006d1b0 0x7fdfc0135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbeffd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006c890 0x7fdfc006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdfc006d1b0 0x7fdfc0135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36210/0 (socket says 10.20.193.78:36210) 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/2134625716 learned_addr learned my addr 10.20.193.78:0/2134625716 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient(hunting): get_auth_request con 0x7fdfc006d1b0 auth_method 0 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbe7fc640 1 --1- 10.20.193.78:0/2134625716 >> v1:10.20.193.78:6789/0 conn(0x7fdfc009c4a0 0x7fdfc0134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55436/0 (socket says 10.20.193.78:55436) 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbeffd640 10 monclient(hunting): get_auth_request con 0x7fdfc006c890 auth_method 0 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.385 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient(hunting): handle_auth_done global_id 284271 payload 274 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/2134625716 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006c890 msgr2=0x7fdfc006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 1 --2- 10.20.193.78:0/2134625716 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006c890 0x7fdfc006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/2134625716 >> v1:10.20.193.78:6789/0 conn(0x7fdfc009c4a0 legacy=0x7fdfc0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 1 monclient: found mon.noname-a 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.384+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/2134625716 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc009cf60 con 0x7fdfc006d1b0 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/2134625716 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc006d1b0 msgr2=0x7fdfc0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.386 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbf7fe640 1 --2- 10.20.193.78:0/2134625716 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc006d1b0 0x7fdfc0135500 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009e50 tx=0x7fdfb002f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76a290997b326b2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 1 -- 10.20.193.78:0/2134625716 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00373f0 con 0x7fdfc006d1b0 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: dump: 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.387 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: _finish_auth 0 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: _check_auth_tickets 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 5 monclient: authenticate success, global_id 284271 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 1 -- 10.20.193.78:0/2134625716 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb0037590 con 0x7fdfc006d1b0 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 1 -- 10.20.193.78:0/2134625716 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037960 con 0x7fdfc006d1b0 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfbdffb640 10 monclient: dump: 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 4 set_mon_vals no callback set 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 10 monclient: shutdown 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 1 -- 10.20.193.78:0/2134625716 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc006d1b0 msgr2=0x7fdfc0135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 1 --2- 10.20.193.78:0/2134625716 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc006d1b0 0x7fdfc0135500 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009e50 tx=0x7fdfb002f210 comp rx=0 tx=0).stop 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 1 -- 10.20.193.78:0/2134625716 shutdown_connections 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 1 --2- 10.20.193.78:0/2134625716 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc006d1b0 0x7fdfc0135500 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 1 --2- 10.20.193.78:0/2134625716 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006c890 0x7fdfc006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 1 -- 10.20.193.78:0/2134625716 >> 10.20.193.78:0/2134625716 conn(0x7fdfc00a6620 msgr2=0x7fdfc00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 1 -- 10.20.193.78:0/2134625716 shutdown_connections 2026-02-17T06:10:29.388 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.387+0000 7fdfc56fa640 1 -- 10.20.193.78:0/2134625716 wait complete. 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient: build_initial_monmap 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient: monmap: 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.390 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 1 Processor -- start 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 1 -- start start 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient: init 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfc006c890 0x7fdfc01cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient(hunting): picked mon.noname-b con 0x7fdfc006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006d1b0 0x7fdfc01cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient(hunting): picked mon.noname-c con 0x7fdfc006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc009c4a0 0x7fdfc0125a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient(hunting): picked mon.noname-a con 0x7fdfc009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc00a01a0 con 0x7fdfc009c4a0 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc00a0020 con 0x7fdfc006c890 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc009be20 con 0x7fdfc006d1b0 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfc56fa640 10 monclient(hunting): authenticate will time out at 2074.941655s 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006d1b0 0x7fdfc01cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.391 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc009c4a0 0x7fdfc0125a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbeffd640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfc006c890 msgr2=0x7fdfc01cf350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc009c4a0 0x7fdfc0125a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36214/0 (socket says 10.20.193.78:36214) 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006d1b0 0x7fdfc01cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:51402/0 (socket says 10.20.193.78:51402) 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbeffd640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfc006c890 0x7fdfc01cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/795323926 learned_addr learned my addr 10.20.193.78:0/795323926 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient(hunting): get_auth_request con 0x7fdfc009c4a0 auth_method 0 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 10 monclient(hunting): get_auth_request con 0x7fdfc006d1b0 auth_method 0 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient(hunting): handle_auth_done global_id 284274 payload 824 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006d1b0 msgr2=0x7fdfc01cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfc006d1b0 0x7fdfc01cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfc006c890 msgr2=0x7fdfc01cf350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfc006c890 0x7fdfc01cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.392 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:10:29.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 monclient: found mon.noname-a 2026-02-17T06:10:29.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:29.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.388+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/795323926 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc0129430 con 0x7fdfc009c4a0 2026-02-17T06:10:29.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.390+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc009c4a0 msgr2=0x7fdfc0125a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.393 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.390+0000 7fdfbf7fe640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc009c4a0 0x7fdfc0125a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0039040 tx=0x7fdfb0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0046020 con 0x7fdfc009c4a0 2026-02-17T06:10:29.394 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: dump: 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: _finish_auth 0 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb0041870 con 0x7fdfc009c4a0 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00373f0 con 0x7fdfc009c4a0 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 10 monclient: dump: 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.395 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 5 monclient: authenticate success, global_id 284274 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 10 monclient: _renew_subs 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc0128260 con 0x7fdfc009c4a0 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbd7fa640 4 set_mon_vals no callback set 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 10 monclient: _renew_subs 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc00725f0 con 0x7fdfc009c4a0 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126760 0x7fdfc0121180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc0074980 con 0x7fdfc0126760 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126760 0x7fdfc0121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 10 monclient: get_auth_request con 0x7fdfc0126760 auth_method 0 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.396 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fdfb0041cc0 con 0x7fdfc009c4a0 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_done global_id 284277 payload 824 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.391+0000 7fdfbeffd640 1 -- 10.20.193.78:0/795323926 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdfc006cc70 con 0x7fdfc0126760 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.392+0000 7fdf9f7fe640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdf8c03de60 0x7fdf8c040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.392+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(855..855 src has 525..855) ==== 4779+0+0 (secure 0 0 0) 0x7fdfb00782a0 con 0x7fdfc009c4a0 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.392+0000 7fdfbe7fc640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdf8c03de60 0x7fdf8c040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.392+0000 7fdfbe7fc640 10 monclient: get_auth_request con 0x7fdf8c03de60 auth_method 0 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdfbeffd640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126760 msgr2=0x7fdfc0121180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdfbeffd640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126760 0x7fdfc0121180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400e8b0 tx=0x7fdfb400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4017070 con 0x7fdfc0126760 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 10 monclient: dump: 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.397 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fdfb400cbc0 con 0x7fdfc0126760 2026-02-17T06:10:29.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:29.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:29.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126760 msgr2=0x7fdfc0121180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126760 0x7fdfc0121180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400e8b0 tx=0x7fdfb400ed80 comp rx=0 tx=0).stop 2026-02-17T06:10:29.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126760 msgr2=0x7fdfc0121180 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:29.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdf9f7fe640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126760 0x7fdfc0121180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.398 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.394+0000 7fdfbe7fc640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdf8c03de60 0x7fdf8c040320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0129640 tx=0x7fdfa8007f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfc56fa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfc56fa640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126b40 0x7fdfc01da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfc56fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc0127fe0 con 0x7fdfc0126b40 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126b40 0x7fdfc01da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 10 monclient: get_auth_request con 0x7fdfc0126b40 auth_method 0 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 10 monclient(hunting): handle_auth_done global_id 284280 payload 824 2026-02-17T06:10:29.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.397+0000 7fdfbeffd640 1 -- 10.20.193.78:0/795323926 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdfc0122840 con 0x7fdfc0126b40 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.399+0000 7fdfbeffd640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126b40 msgr2=0x7fdfc01da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.399+0000 7fdfbeffd640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126b40 0x7fdfc01da560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400e8b0 tx=0x7fdfb4004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4017050 con 0x7fdfc0126b40 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 10 monclient: dump: 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7fdfb400ce50 con 0x7fdfc0126b40 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126b40 msgr2=0x7fdfc01da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126b40 0x7fdfc01da560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400e8b0 tx=0x7fdfb4004830 comp rx=0 tx=0).stop 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126b40 msgr2=0x7fdfc01da560 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdf9f7fe640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc0126b40 0x7fdfc01da560 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400e8b0 tx=0x7fdfb4004830 comp rx=0 tx=0).stop 2026-02-17T06:10:29.400 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 268, 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 80537, 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:29.401 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:29.402 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.403 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.404 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "num": 3 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.405 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdf8c03de60 msgr2=0x7fdf8c040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdf8c03de60 0x7fdf8c040320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0129640 tx=0x7fdfa8007f60 comp rx=0 tx=0).stop 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 10 monclient: shutdown 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc009c4a0 msgr2=0x7fdfc0125a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfbf7fe640 1 -- 10.20.193.78:0/795323926 reap_dead start 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc009c4a0 0x7fdfc0125a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0039040 tx=0x7fdfb0004b40 comp rx=0 tx=0).stop 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 shutdown_connections 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 --2- 10.20.193.78:0/795323926 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfc009c4a0 0x7fdfc0125a60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 >> 10.20.193.78:0/795323926 conn(0x7fdfc00a6620 msgr2=0x7fdfc0072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 shutdown_connections 2026-02-17T06:10:29.406 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.400+0000 7fdfc56fa640 1 -- 10.20.193.78:0/795323926 wait complete. 2026-02-17T06:10:29.407 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:10:29.407 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-17T06:10:29.407 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:10:29.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.472+0000 7f37e3ec9640 10 monclient: get_monmap_and_config 2026-02-17T06:10:29.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.472+0000 7f37e3ec9640 10 monclient: build_initial_monmap 2026-02-17T06:10:29.472 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.472+0000 7f37e3ec9640 10 monclient: monmap: 2026-02-17T06:10:29.472 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:29.472 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.472 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:29.472924+0000 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:29.472924+0000 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:29.473 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 1 Processor -- start 2026-02-17T06:10:29.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 1 -- start start 2026-02-17T06:10:29.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient: init 2026-02-17T06:10:29.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:29.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2026-02-17T06:10:29.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f37dc06c890 0x7f37dc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient(hunting): picked mon.noname-c con 0x7f37dc06c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:29.474 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc09c4a0 0x7f37dc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient(hunting): picked mon.noname-b con 0x7f37dc09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f37dc134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37dc06daf0 con 0x7f37dc134df0 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f37dc0a0880 con 0x7f37dc06c890 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f37dc06d6f0 con 0x7f37dc09c4a0 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e3ec9640 10 monclient(hunting): authenticate will time out at 2075.026485s 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e1c3e640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f37dc06c890 msgr2=0x7f37dc06cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e1c3e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f37dc06c890 0x7f37dc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc09c4a0 0x7f37dc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e243f640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f37dc134df0 0x7f37dc135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:55448/0 (socket says 10.20.193.78:55448) 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e243f640 1 -- 10.20.193.78:0/3624167258 learned_addr learned my addr 10.20.193.78:0/3624167258 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient(hunting): get_auth_request con 0x7f37dc09c4a0 auth_method 0 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient(hunting): handle_auth_done global_id 264467 payload 274 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 1 -- 10.20.193.78:0/3624167258 >> v2:10.20.193.115:3300/0 conn(0x7f37dc06c890 msgr2=0x7f37dc06cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 1 --2- 10.20.193.78:0/3624167258 >> v2:10.20.193.115:3300/0 conn(0x7f37dc06c890 0x7f37dc06cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.475 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 1 -- 10.20.193.78:0/3624167258 >> v1:10.20.193.78:6789/0 conn(0x7f37dc134df0 legacy=0x7f37dc135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:29.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient: _finish_hunting 0 2026-02-17T06:10:29.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 1 monclient: found mon.noname-b 2026-02-17T06:10:29.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:29.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:10:29.476 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.473+0000 7f37e143d640 1 -- 10.20.193.78:0/3624167258 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc09cc80 con 0x7f37dc09c4a0 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e143d640 1 -- 10.20.193.78:0/3624167258 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc09c4a0 msgr2=0x7f37dc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e143d640 1 --2- 10.20.193.78:0/3624167258 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc09c4a0 0x7f37dc06d1b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f37dc06d870 tx=0x7f37d002eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72652728d4735a7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 1 -- 10.20.193.78:0/3624167258 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d003c070 con 0x7f37dc09c4a0 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: dump: 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.478 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: _finish_auth 0 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: _check_auth_tickets 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 1 -- 10.20.193.78:0/3624167258 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d002fd00 con 0x7f37dc09c4a0 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 5 monclient: authenticate success, global_id 264467 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 1 -- 10.20.193.78:0/3624167258 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0037660 con 0x7f37dc09c4a0 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 4 set_mon_vals no callback set 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e0c3c640 10 monclient: dump: 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.479 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 10 monclient: shutdown 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3624167258 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc09c4a0 msgr2=0x7f37dc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 1 --2- 10.20.193.78:0/3624167258 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc09c4a0 0x7f37dc06d1b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f37dc06d870 tx=0x7f37d002eda0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3624167258 shutdown_connections 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 1 --2- 10.20.193.78:0/3624167258 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc09c4a0 0x7f37dc06d1b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 1 --2- 10.20.193.78:0/3624167258 >> v2:10.20.193.115:3300/0 conn(0x7f37dc06c890 0x7f37dc06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3624167258 >> 10.20.193.78:0/3624167258 conn(0x7f37dc0a6620 msgr2=0x7f37dc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3624167258 shutdown_connections 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3624167258 wait complete. 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 10 monclient: build_initial_monmap 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.476+0000 7f37e3ec9640 10 monclient: monmap: 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.480 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 1 Processor -- start 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 1 -- start start 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient: init 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37dc06c890 0x7f37dc1cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient(hunting): picked mon.noname-a con 0x7f37dc06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37dc09c4a0 0x7f37dc1333e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient(hunting): picked mon.noname-b con 0x7f37dc09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc134df0 0x7f37dc133920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient(hunting): picked mon.noname-c con 0x7f37dc134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc0a01a0 con 0x7f37dc06c890 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc0a0020 con 0x7f37dc09c4a0 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f37dc09be20 con 0x7f37dc134df0 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e3ec9640 10 monclient(hunting): authenticate will time out at 2075.030407s 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37dc06c890 0x7f37dc1cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e243f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc134df0 0x7f37dc133920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37dc06c890 0x7f37dc1cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:36238/0 (socket says 10.20.193.78:36238) 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 -- 10.20.193.78:0/3441377435 learned_addr learned my addr 10.20.193.78:0/3441377435 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient(hunting): get_auth_request con 0x7f37dc06c890 auth_method 0 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e143d640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37dc09c4a0 msgr2=0x7f37dc1333e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.481 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e143d640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37dc09c4a0 0x7f37dc1333e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e243f640 10 monclient(hunting): get_auth_request con 0x7f37dc134df0 auth_method 0 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e243f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e243f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e243f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e243f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e243f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e243f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient(hunting): handle_auth_done global_id 284283 payload 824 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc134df0 msgr2=0x7f37dc133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc134df0 0x7f37dc133920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37dc09c4a0 msgr2=0x7f37dc1333e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f37dc09c4a0 0x7f37dc1333e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient: _finish_hunting 0 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 monclient: found mon.noname-a 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.482 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.477+0000 7f37e1c3e640 1 -- 10.20.193.78:0/3441377435 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc126b20 con 0x7f37dc06c890 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37e1c3e640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37dc06c890 msgr2=0x7f37dc1cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37e1c3e640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37dc06c890 0x7f37dc1cf1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00b7b0 tx=0x7f37cc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0052d0 con 0x7f37dc06c890 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: dump: 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.484 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: _finish_auth 0 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37e3ec9640 5 monclient: authenticate success, global_id 284283 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37cc010070 con 0x7f37dc06c890 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0056d0 con 0x7f37dc06c890 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37cbfff640 4 set_mon_vals no callback set 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37ca7fc640 10 monclient: dump: 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.485 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37e3ec9640 10 monclient: _renew_subs 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37e3ec9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.479+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37dc1cf700 con 0x7f37dc06c890 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e3ec9640 10 monclient: _renew_subs 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e3ec9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37dc124b10 con 0x7f37dc06c890 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e3ec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e3ec9640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123630 0x7f37dc070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e3ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f37dc0a1730 con 0x7f37dc123630 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123630 0x7f37dc070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 10 monclient: get_auth_request con 0x7f37dc123630 auth_method 0 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f37cc013370 con 0x7f37dc06c890 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 10 monclient(hunting): handle_auth_done global_id 264473 payload 824 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e143d640 1 -- 10.20.193.78:0/3441377435 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37dc06d0a0 con 0x7f37dc123630 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37ca7fc640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37ac03de60 0x7f37ac040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(855..855 src has 525..855) ==== 4779+0+0 (secure 0 0 0) 0x7f37cc051dd0 con 0x7f37dc06c890 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.480+0000 7f37e243f640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37ac03de60 0x7f37ac040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.481+0000 7f37e243f640 10 monclient: get_auth_request con 0x7f37ac03de60 auth_method 0 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.482+0000 7f37e143d640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123630 msgr2=0x7f37dc070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.482+0000 7f37e143d640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123630 0x7f37dc070380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37d0039000 tx=0x7f37d00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.482+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0047070 con 0x7f37dc123630 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.482+0000 7f37ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.482+0000 7f37ca7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.482+0000 7f37ca7fc640 10 monclient: dump: 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.483+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7f37d004c450 con 0x7f37dc123630 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.483+0000 7f37ca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.483+0000 7f37ca7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.483+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123630 msgr2=0x7f37dc070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.483+0000 7f37ca7fc640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123630 0x7f37dc070380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37d0039000 tx=0x7f37d00047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.483+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123630 msgr2=0x7f37dc070380 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.483+0000 7f37ca7fc640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123630 0x7f37dc070380 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37d0039000 tx=0x7f37d00047c0 comp rx=0 tx=0).stop 2026-02-17T06:10:29.487 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.483+0000 7f37e243f640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37ac03de60 0x7f37ac040320 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f37d80055f0 tx=0x7f37d8005580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e3ec9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:10:29.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e3ec9640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123a10 0x7f37dc1da510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:29.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e3ec9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f37dc135a70 con 0x7f37dc123a10 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123a10 0x7f37dc1da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 10 monclient: get_auth_request con 0x7f37dc123a10 auth_method 0 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 10 monclient(hunting): handle_auth_done global_id 264476 payload 824 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.486+0000 7f37e143d640 1 -- 10.20.193.78:0/3441377435 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f37dc071a40 con 0x7f37dc123a10 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37e143d640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123a10 msgr2=0x7f37dc1da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37e143d640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123a10 0x7f37dc1da510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f37d0002410 tx=0x7f37d0009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d003c040 con 0x7f37dc123a10 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 10 monclient: dump: 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:29.489 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4698 (secure 0 0 0) 0x7f37d004c670 con 0x7f37dc123a10 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123a10 msgr2=0x7f37dc1da510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123a10 0x7f37dc1da510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f37d0002410 tx=0x7f37d0009290 comp rx=0 tx=0).stop 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123a10 msgr2=0x7f37dc1da510 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.488+0000 7f37ca7fc640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f37dc123a10 0x7f37dc1da510 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f37d0002410 tx=0x7f37d0009290 comp rx=0 tx=0).stop 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:10:29.490 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 268, 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 178872, 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 1, 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:29.491 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:10:29.492 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.493 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:10:29.494 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:10:29.495 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37ac03de60 msgr2=0x7f37ac040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f37ac03de60 0x7f37ac040320 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f37d80055f0 tx=0x7f37d8005580 comp rx=0 tx=0).stop 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 10 monclient: shutdown 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37dc06c890 msgr2=0x7f37dc1cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 1 --2- 10.20.193.78:0/3441377435 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f37dc06c890 0x7f37dc1cf1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00b7b0 tx=0x7f37cc00bc80 comp rx=0 tx=0).stop 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e243f640 1 -- 10.20.193.78:0/3441377435 reap_dead start 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e243f640 1 -- 10.20.193.78:0/3441377435 reap_dead start 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 shutdown_connections 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 >> 10.20.193.78:0/3441377435 conn(0x7f37dc0a6620 msgr2=0x7f37dc0a1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 shutdown_connections 2026-02-17T06:10:29.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:29.489+0000 7f37e3ec9640 1 -- 10.20.193.78:0/3441377435 wait complete. 2026-02-17T06:10:29.497 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-17T06:10:29.497 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2026-02-17T06:10:29.497 INFO:tasks.rados.rados.0.trial115.stdout:3924: done (12 left) 2026-02-17T06:10:29.497 INFO:tasks.rados.rados.0.trial115.stdout:3925: delete oid 12 current snap is 414 2026-02-17T06:10:29.497 INFO:tasks.rados.rados.0.trial115.stdout:3916: finishing rollback tid 1 to trial11545420-33 2026-02-17T06:10:29.497 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T06:10:29.497 INFO:tasks.rados.rados.0.trial115.stdout:3912: finishing write tid 2 to trial11545420-32 2026-02-17T06:10:29.497 INFO:tasks.rados.rados.0.trial115.stdout:3912: finishing write tid 3 to trial11545420-32 2026-02-17T06:10:29.497 INFO:tasks.rados.rados.0.trial115.stdout:3912: finishing write tid 4 to trial11545420-32 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3912: oid 32 updating version 0 to 1904 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3912: oid 32 updating version 1904 to 1906 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3912: oid 32 version 1906 is already newer than 1905 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 32 v 1906 (ObjNum 1333 snap 414 seq_num 1333) dirty exists 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3912: left oid 32 (ObjNum 1333 snap 414 seq_num 1333) 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3919: finishing rollback tid 0 to trial11545420-49 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3919: finishing rollback tid 1 to trial11545420-49 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3919: finishing rollback tid 2 to trial11545420-49 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 49 v 1514 (ObjNum 1316 snap 406 seq_num 1316) dirty exists 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3920: finishing copy_from to trial11545420-20 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3920: got expected ENOENT (src dne) 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3920: finishing copy_from racing read to trial11545420-20 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3914: finishing rollback tid 1 to trial11545420-7 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3914: finishing rollback tid 2 to trial11545420-7 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1990 (ObjNum 1288 snap 400 seq_num 1288) dirty exists 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3921: finishing write tid 1 to trial11545420-10 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3918: finishing copy_from racing read to trial11545420-8 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3917: finishing copy_from racing read to trial11545420-40 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3921: finishing write tid 2 to trial11545420-10 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3921: finishing write tid 3 to trial11545420-10 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3921: finishing write tid 4 to trial11545420-10 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3921: oid 10 updating version 0 to 2058 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3921: oid 10 updating version 2058 to 2059 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3921: oid 10 version 2059 is already newer than 2057 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 2059 (ObjNum 1334 snap 414 seq_num 1334) dirty exists 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3921: left oid 10 (ObjNum 1334 snap 414 seq_num 1334) 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3917: finishing copy_from to trial11545420-40 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1504 (ObjNum 1274 snap 398 seq_num 1274) dirty exists 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3918: finishing copy_from to trial11545420-8 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 2025 (ObjNum 1305 snap 403 seq_num 1305) dirty exists 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3922: finishing write tid 1 to trial11545420-29 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3922: finishing write tid 2 to trial11545420-29 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3922: finishing write tid 3 to trial11545420-29 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3922: finishing write tid 4 to trial11545420-29 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3922: oid 29 updating version 0 to 1499 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3922: oid 29 updating version 1499 to 1500 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3922: oid 29 version 1500 is already newer than 1498 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1500 (ObjNum 1335 snap 414 seq_num 1335) dirty exists 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3922: left oid 29 (ObjNum 1335 snap 414 seq_num 1335) 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3923: finishing write tid 1 to trial11545420-24 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3923: finishing write tid 2 to trial11545420-24 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3923: finishing write tid 3 to trial11545420-24 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3923: finishing write tid 4 to trial11545420-24 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3923: oid 24 updating version 0 to 2005 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3923: oid 24 updating version 2005 to 2006 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:3923: oid 24 updating version 2006 to 2007 2026-02-17T06:10:29.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 2007 (ObjNum 1336 snap 414 seq_num 1336) dirty exists 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3923: left oid 24 (ObjNum 1336 snap 414 seq_num 1336) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3912: done (12 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3913: done (11 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3914: done (10 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3915: done (9 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3916: done (8 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3917: done (7 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3918: done (6 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3919: done (5 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3920: done (4 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3921: done (3 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3922: done (2 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3923: done (1 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3925: done (0 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3926: copy_from oid 39 from oid 23 current snap is 414 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3927: read oid 3 snap -1 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3927: expect (ObjNum 1215 snap 376 seq_num 1215) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3928: read oid 49 snap -1 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3928: expect (ObjNum 1316 snap 406 seq_num 1316) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3929: rollback oid 2 current snap is 414 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 2 to 409 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3930: delete oid 14 current snap is 414 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3930: done (4 left) 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3931: write oid 16 current snap is 414 2026-02-17T06:10:29.499 INFO:tasks.rados.rados.0.trial115.stdout:3931: seq_num 1339 ranges {504573=436539,1421742=598804,2437505=67068} 2026-02-17T06:10:29.501 INFO:tasks.rados.rados.0.trial115.stdout:3931: writing trial11545420-16 from 504573 to 941112 tid 1 2026-02-17T06:10:29.505 INFO:tasks.rados.rados.0.trial115.stdout:3931: writing trial11545420-16 from 1421742 to 2020546 tid 2 2026-02-17T06:10:29.507 INFO:tasks.rados.rados.0.trial115.stdout:3931: writing trial11545420-16 from 2437505 to 2504573 tid 3 2026-02-17T06:10:29.507 INFO:tasks.rados.rados.0.trial115.stdout:3932: rollback oid 37 current snap is 414 2026-02-17T06:10:29.507 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 37 to 406 2026-02-17T06:10:29.507 INFO:tasks.rados.rados.0.trial115.stdout:3927: expect (ObjNum 1215 snap 376 seq_num 1215) 2026-02-17T06:10:29.647 INFO:tasks.rados.rados.0.trial115.stdout:3928: expect (ObjNum 1316 snap 406 seq_num 1316) 2026-02-17T06:10:29.748 INFO:tasks.rados.rados.0.trial115.stdout:3929: finishing rollback tid 0 to trial11545420-2 2026-02-17T06:10:29.748 INFO:tasks.rados.rados.0.trial115.stdout:3929: finishing rollback tid 1 to trial11545420-2 2026-02-17T06:10:29.748 INFO:tasks.rados.rados.0.trial115.stdout:3927: done (5 left) 2026-02-17T06:10:29.748 INFO:tasks.rados.rados.0.trial115.stdout:3928: done (4 left) 2026-02-17T06:10:29.748 INFO:tasks.rados.rados.0.trial115.stdout:3933: copy_from oid 8 from oid 15 current snap is 414 2026-02-17T06:10:29.748 INFO:tasks.rados.rados.0.trial115.stdout:3929: finishing rollback tid 2 to trial11545420-2 2026-02-17T06:10:29.748 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1714 (ObjNum 1302 snap 402 seq_num 1302) dirty exists 2026-02-17T06:10:29.748 INFO:tasks.rados.rados.0.trial115.stdout:3931: finishing write tid 1 to trial11545420-16 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3926: finishing copy_from racing read to trial11545420-39 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3926: finishing copy_from to trial11545420-39 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 1823 (ObjNum 1289 snap 400 seq_num 1289) dirty exists 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: finishing write tid 2 to trial11545420-16 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: finishing write tid 3 to trial11545420-16 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: finishing write tid 4 to trial11545420-16 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: finishing write tid 5 to trial11545420-16 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: oid 16 updating version 0 to 2035 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: oid 16 version 2035 is already newer than 2033 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: oid 16 version 2035 is already newer than 2034 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: oid 16 updating version 2035 to 2036 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 16 v 2036 (ObjNum 1339 snap 414 seq_num 1339) dirty exists 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3931: left oid 16 (ObjNum 1339 snap 414 seq_num 1339) 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3932: finishing rollback tid 0 to trial11545420-37 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3932: finishing rollback tid 1 to trial11545420-37 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3932: finishing rollback tid 2 to trial11545420-37 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 37 v 1993 (ObjNum 1274 snap 398 seq_num 1274) dirty exists 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3926: done (4 left) 2026-02-17T06:10:29.749 INFO:tasks.rados.rados.0.trial115.stdout:3929: done (3 left) 2026-02-17T06:10:29.750 INFO:tasks.rados.rados.0.trial115.stdout:3931: done (2 left) 2026-02-17T06:10:29.750 INFO:tasks.rados.rados.0.trial115.stdout:3932: done (1 left) 2026-02-17T06:10:29.750 INFO:tasks.rados.rados.0.trial115.stdout:3934: rollback oid 43 current snap is 414 2026-02-17T06:10:29.750 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 43 to 412 2026-02-17T06:10:29.750 INFO:tasks.rados.rados.0.trial115.stdout:3935: write oid 39 current snap is 414 2026-02-17T06:10:29.750 INFO:tasks.rados.rados.0.trial115.stdout:3935: seq_num 1340 ranges {521380=511788,1480224=742462} 2026-02-17T06:10:29.751 INFO:tasks.rados.rados.0.trial115.stdout:3935: writing trial11545420-39 from 521380 to 1033168 tid 1 2026-02-17T06:10:29.756 INFO:tasks.rados.rados.0.trial115.stdout:3935: writing trial11545420-39 from 1480224 to 2222686 tid 2 2026-02-17T06:10:29.756 INFO:tasks.rados.rados.0.trial115.stdout:3936: write oid 29 current snap is 414 2026-02-17T06:10:29.756 INFO:tasks.rados.rados.0.trial115.stdout:3936: seq_num 1341 ranges {538187=587037,1538706=602473} 2026-02-17T06:10:29.760 INFO:tasks.rados.rados.0.trial115.stdout:3936: writing trial11545420-29 from 538187 to 1125224 tid 1 2026-02-17T06:10:29.764 INFO:tasks.rados.rados.0.trial115.stdout:3936: writing trial11545420-29 from 1538706 to 2141179 tid 2 2026-02-17T06:10:29.764 INFO:tasks.rados.rados.0.trial115.stdout:3937: write oid 44 current snap is 414 2026-02-17T06:10:29.764 INFO:tasks.rados.rados.0.trial115.stdout:3937: seq_num 1342 ranges {554994=662286,1997188=557806} 2026-02-17T06:10:29.768 INFO:tasks.rados.rados.0.trial115.stdout:3937: writing trial11545420-44 from 554994 to 1217280 tid 1 2026-02-17T06:10:29.773 INFO:tasks.rados.rados.0.trial115.stdout:3937: writing trial11545420-44 from 1997188 to 2554994 tid 2 2026-02-17T06:10:29.773 INFO:tasks.rados.rados.0.trial115.stdout:3938: read oid 9 snap -1 2026-02-17T06:10:29.773 INFO:tasks.rados.rados.0.trial115.stdout:3938: expect deleted 2026-02-17T06:10:29.773 INFO:tasks.rados.rados.0.trial115.stdout:3939: snap_create 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3934: finishing rollback tid 0 to trial11545420-43 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3934: finishing rollback tid 1 to trial11545420-43 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3934: finishing rollback tid 2 to trial11545420-43 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 43 v 1996 (ObjNum 1276 snap 398 seq_num 1276) dirty exists 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3933: finishing copy_from racing read to trial11545420-8 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3935: finishing write tid 1 to trial11545420-39 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3933: finishing copy_from to trial11545420-8 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 2027 (ObjNum 1311 snap 404 seq_num 1311) dirty exists 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3935: finishing write tid 2 to trial11545420-39 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3935: finishing write tid 3 to trial11545420-39 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3935: finishing write tid 4 to trial11545420-39 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3935: oid 39 updating version 0 to 2014 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3935: oid 39 updating version 2014 to 2015 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3935: oid 39 updating version 2015 to 2016 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 2016 (ObjNum 1340 snap 414 seq_num 1340) dirty exists 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3935: left oid 39 (ObjNum 1340 snap 414 seq_num 1340) 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3936: finishing write tid 1 to trial11545420-29 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3936: finishing write tid 2 to trial11545420-29 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3936: finishing write tid 3 to trial11545420-29 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3936: finishing write tid 4 to trial11545420-29 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3936: oid 29 updating version 0 to 1503 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3936: oid 29 version 1503 is already newer than 1502 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3936: oid 29 version 1503 is already newer than 1501 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1503 (ObjNum 1341 snap 414 seq_num 1341) dirty exists 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3936: left oid 29 (ObjNum 1341 snap 414 seq_num 1341) 2026-02-17T06:10:29.774 INFO:tasks.rados.rados.0.trial115.stdout:3937: finishing write tid 1 to trial11545420-44 2026-02-17T06:10:29.775 INFO:tasks.rados.rados.0.trial115.stdout:3937: finishing write tid 2 to trial11545420-44 2026-02-17T06:10:29.775 INFO:tasks.rados.rados.0.trial115.stdout:3937: finishing write tid 3 to trial11545420-44 2026-02-17T06:10:29.775 INFO:tasks.rados.rados.0.trial115.stdout:3937: finishing write tid 4 to trial11545420-44 2026-02-17T06:10:29.775 INFO:tasks.rados.rados.0.trial115.stdout:3937: oid 44 updating version 0 to 1907 2026-02-17T06:10:29.775 INFO:tasks.rados.rados.0.trial115.stdout:3937: oid 44 updating version 1907 to 1909 2026-02-17T06:10:29.775 INFO:tasks.rados.rados.0.trial115.stdout:3937: oid 44 version 1909 is already newer than 1908 2026-02-17T06:10:29.775 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 44 v 1909 (ObjNum 1342 snap 414 seq_num 1342) dirty exists 2026-02-17T06:10:29.775 INFO:tasks.rados.rados.0.trial115.stdout:3937: left oid 44 (ObjNum 1342 snap 414 seq_num 1342) 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3933: done (6 left) 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3934: done (5 left) 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3935: done (4 left) 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3936: done (3 left) 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3937: done (2 left) 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3938: done (1 left) 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3939: done (0 left) 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3940: rollback oid 7 current snap is 415 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 7 to 412 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3941: write oid 33 current snap is 415 2026-02-17T06:10:30.499 INFO:tasks.rados.rados.0.trial115.stdout:3941: seq_num 1343 ranges {571801=737535,1939317=489789} 2026-02-17T06:10:30.505 INFO:tasks.rados.rados.0.trial115.stdout:3941: writing trial11545420-33 from 571801 to 1309336 tid 1 2026-02-17T06:10:30.509 INFO:tasks.rados.rados.0.trial115.stdout:3941: writing trial11545420-33 from 1939317 to 2429106 tid 2 2026-02-17T06:10:30.509 INFO:tasks.rados.rados.0.trial115.stdout:3942: rollback oid 10 current snap is 415 2026-02-17T06:10:30.509 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 10 to 412 2026-02-17T06:10:30.509 INFO:tasks.rados.rados.0.trial115.stdout:3940: finishing rollback tid 0 to trial11545420-7 2026-02-17T06:10:30.510 INFO:tasks.rados.rados.0.trial115.stdout:3940: finishing rollback tid 1 to trial11545420-7 2026-02-17T06:10:30.510 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1998 (ObjNum 1715449088 snap 32570 seq_num 304) dirty dne 2026-02-17T06:10:30.510 INFO:tasks.rados.rados.0.trial115.stdout:3940: done (2 left) 2026-02-17T06:10:30.510 INFO:tasks.rados.rados.0.trial115.stdout:3943: delete oid 11 current snap is 415 2026-02-17T06:10:30.510 INFO:tasks.rados.rados.0.trial115.stdout:3941: finishing write tid 1 to trial11545420-33 2026-02-17T06:10:30.511 INFO:tasks.rados.rados.0.trial115.stdout:3941: finishing write tid 2 to trial11545420-33 2026-02-17T06:10:30.511 INFO:tasks.rados.rados.0.trial115.stdout:3941: finishing write tid 3 to trial11545420-33 2026-02-17T06:10:30.511 INFO:tasks.rados.rados.0.trial115.stdout:3941: finishing write tid 4 to trial11545420-33 2026-02-17T06:10:30.511 INFO:tasks.rados.rados.0.trial115.stdout:3941: oid 33 updating version 0 to 1710 2026-02-17T06:10:30.511 INFO:tasks.rados.rados.0.trial115.stdout:3941: oid 33 updating version 1710 to 1712 2026-02-17T06:10:30.512 INFO:tasks.rados.rados.0.trial115.stdout:3941: oid 33 version 1712 is already newer than 1711 2026-02-17T06:10:30.512 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 33 v 1712 (ObjNum 1343 snap 415 seq_num 1343) dirty exists 2026-02-17T06:10:30.512 INFO:tasks.rados.rados.0.trial115.stdout:3941: left oid 33 (ObjNum 1343 snap 415 seq_num 1343) 2026-02-17T06:10:30.512 INFO:tasks.rados.rados.0.trial115.stdout:3942: finishing rollback tid 0 to trial11545420-10 2026-02-17T06:10:30.513 INFO:tasks.rados.rados.0.trial115.stdout:3942: finishing rollback tid 1 to trial11545420-10 2026-02-17T06:10:30.513 INFO:tasks.rados.rados.0.trial115.stdout:3942: finishing rollback tid 2 to trial11545420-10 2026-02-17T06:10:30.513 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 10 v 2069 (ObjNum 1327 snap 410 seq_num 1327) dirty exists 2026-02-17T06:10:30.513 INFO:tasks.rados.rados.0.trial115.stdout:3941: done (2 left) 2026-02-17T06:10:30.514 INFO:tasks.rados.rados.0.trial115.stdout:3942: done (1 left) 2026-02-17T06:10:30.514 INFO:tasks.rados.rados.0.trial115.stdout:3943: done (0 left) 2026-02-17T06:10:30.514 INFO:tasks.rados.rados.0.trial115.stdout:3944: read oid 9 snap -1 2026-02-17T06:10:30.514 INFO:tasks.rados.rados.0.trial115.stdout:3944: expect deleted 2026-02-17T06:10:30.514 INFO:tasks.rados.rados.0.trial115.stdout:3945: write oid 8 current snap is 415 2026-02-17T06:10:30.514 INFO:tasks.rados.rados.0.trial115.stdout:3945: seq_num 1345 ranges {605415=488033,1656281=493458} 2026-02-17T06:10:30.518 INFO:tasks.rados.rados.0.trial115.stdout:3945: writing trial11545420-8 from 605415 to 1093448 tid 1 2026-02-17T06:10:30.522 INFO:tasks.rados.rados.0.trial115.stdout:3945: writing trial11545420-8 from 1656281 to 2149739 tid 2 2026-02-17T06:10:30.522 INFO:tasks.rados.rados.0.trial115.stdout:3946: read oid 43 snap -1 2026-02-17T06:10:30.522 INFO:tasks.rados.rados.0.trial115.stdout:3946: expect (ObjNum 1276 snap 398 seq_num 1276) 2026-02-17T06:10:30.522 INFO:tasks.rados.rados.0.trial115.stdout:3947: delete oid 18 current snap is 415 2026-02-17T06:10:30.522 INFO:tasks.rados.rados.0.trial115.stdout:3945: finishing write tid 1 to trial11545420-8 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3945: finishing write tid 2 to trial11545420-8 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3945: finishing write tid 3 to trial11545420-8 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3944: done (3 left) 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3947: done (2 left) 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3948: read oid 26 snap 406 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3948: expect (ObjNum 1303 snap 402 seq_num 1303) 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3949: snap_create 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3945: finishing write tid 4 to trial11545420-8 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3945: oid 8 updating version 0 to 2029 2026-02-17T06:10:30.524 INFO:tasks.rados.rados.0.trial115.stdout:3945: oid 8 updating version 2029 to 2030 2026-02-17T06:10:30.525 INFO:tasks.rados.rados.0.trial115.stdout:3945: oid 8 version 2030 is already newer than 2028 2026-02-17T06:10:30.525 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 8 v 2030 (ObjNum 1345 snap 415 seq_num 1345) dirty exists 2026-02-17T06:10:30.525 INFO:tasks.rados.rados.0.trial115.stdout:3945: left oid 8 (ObjNum 1345 snap 415 seq_num 1345) 2026-02-17T06:10:30.527 INFO:tasks.rados.rados.0.trial115.stdout:3946: expect (ObjNum 1276 snap 398 seq_num 1276) 2026-02-17T06:10:30.685 INFO:tasks.rados.rados.0.trial115.stdout:3948: expect (ObjNum 1303 snap 402 seq_num 1303) 2026-02-17T06:10:31.500 INFO:tasks.rados.rados.0.trial115.stdout:3945: done (3 left) 2026-02-17T06:10:31.500 INFO:tasks.rados.rados.0.trial115.stdout:3946: done (2 left) 2026-02-17T06:10:31.500 INFO:tasks.rados.rados.0.trial115.stdout:3948: done (1 left) 2026-02-17T06:10:31.500 INFO:tasks.rados.rados.0.trial115.stdout:3949: done (0 left) 2026-02-17T06:10:31.501 INFO:tasks.rados.rados.0.trial115.stdout:3950: delete oid 45 current snap is 416 2026-02-17T06:10:31.503 INFO:tasks.rados.rados.0.trial115.stdout:3950: done (0 left) 2026-02-17T06:10:31.503 INFO:tasks.rados.rados.0.trial115.stdout:3951: read oid 44 snap -1 2026-02-17T06:10:31.503 INFO:tasks.rados.rados.0.trial115.stdout:3951: expect (ObjNum 1342 snap 414 seq_num 1342) 2026-02-17T06:10:31.503 INFO:tasks.rados.rados.0.trial115.stdout:3952: snap_create 2026-02-17T06:10:31.508 INFO:tasks.rados.rados.0.trial115.stdout:3951: expect (ObjNum 1342 snap 414 seq_num 1342) 2026-02-17T06:10:32.501 INFO:tasks.rados.rados.0.trial115.stdout:3951: done (1 left) 2026-02-17T06:10:32.501 INFO:tasks.rados.rados.0.trial115.stdout:3952: done (0 left) 2026-02-17T06:10:32.501 INFO:tasks.rados.rados.0.trial115.stdout:3953: delete oid 7 current snap is 417 2026-02-17T06:10:32.505 INFO:tasks.rados.rados.0.trial115.stdout:3953: done (0 left) 2026-02-17T06:10:32.505 INFO:tasks.rados.rados.0.trial115.stdout:3954: delete oid 41 current snap is 417 2026-02-17T06:10:32.507 INFO:tasks.rados.rados.0.trial115.stdout:3954: done (0 left) 2026-02-17T06:10:32.507 INFO:tasks.rados.rados.0.trial115.stdout:3955: copy_from oid 18 from oid 24 current snap is 417 2026-02-17T06:10:32.507 INFO:tasks.rados.rados.0.trial115.stdout:3956: snap_create 2026-02-17T06:10:32.517 INFO:tasks.rados.rados.0.trial115.stdout:3955: finishing copy_from racing read to trial11545420-18 2026-02-17T06:10:32.518 INFO:tasks.rados.rados.0.trial115.stdout:3955: finishing copy_from to trial11545420-18 2026-02-17T06:10:32.518 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 2007 (ObjNum 1336 snap 414 seq_num 1336) dirty exists 2026-02-17T06:10:33.118 INFO:tasks.rados.rados.0.trial115.stdout:3955: done (1 left) 2026-02-17T06:10:33.118 INFO:tasks.rados.rados.0.trial115.stdout:3956: done (0 left) 2026-02-17T06:10:33.118 INFO:tasks.rados.rados.0.trial115.stdout:3957: rollback oid 25 current snap is 418 2026-02-17T06:10:33.118 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 25 to 415 2026-02-17T06:10:33.118 INFO:tasks.rados.rados.0.trial115.stdout:3958: write (excl) oid 47 current snap is 418 2026-02-17T06:10:33.118 INFO:tasks.rados.rados.0.trial115.stdout:3958: seq_num 1350 ranges {689450=580631,1948691=740759} 2026-02-17T06:10:33.124 INFO:tasks.rados.rados.0.trial115.stdout:3958: writing trial11545420-47 from 689450 to 1270081 tid 1 2026-02-17T06:10:33.129 INFO:tasks.rados.rados.0.trial115.stdout:3958: writing trial11545420-47 from 1948691 to 2689450 tid 2 2026-02-17T06:10:33.129 INFO:tasks.rados.rados.0.trial115.stdout:3959: write (excl) oid 24 current snap is 418 2026-02-17T06:10:33.129 INFO:tasks.rados.rados.0.trial115.stdout:3957: finishing rollback tid 0 to trial11545420-25 2026-02-17T06:10:33.129 INFO:tasks.rados.rados.0.trial115.stdout:3957: finishing rollback tid 1 to trial11545420-25 2026-02-17T06:10:33.129 INFO:tasks.rados.rados.0.trial115.stdout:3957: finishing rollback tid 2 to trial11545420-25 2026-02-17T06:10:33.129 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 25 v 2010 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2026-02-17T06:10:33.129 INFO:tasks.rados.rados.0.trial115.stdout:3958: finishing write tid 1 to trial11545420-47 2026-02-17T06:10:33.129 INFO:tasks.rados.rados.0.trial115.stdout:3959: seq_num 1351 ranges {706257=655880,1890820=504465} 2026-02-17T06:10:33.134 INFO:tasks.rados.rados.0.trial115.stdout:3959: writing trial11545420-24 from 706257 to 1362137 tid 1 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3959: writing trial11545420-24 from 1890820 to 2395285 tid 2 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3957: done (2 left) 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3960: rollback oid 35 current snap is 418 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 35 to 416 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3958: finishing write tid 2 to trial11545420-47 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3958: finishing write tid 3 to trial11545420-47 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3958: finishing write tid 4 to trial11545420-47 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3958: oid 47 updating version 0 to 1718 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3958: oid 47 updating version 1718 to 1719 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3958: oid 47 version 1719 is already newer than 1717 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 47 v 1719 (ObjNum 1350 snap 418 seq_num 1350) dirty exists 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3958: left oid 47 (ObjNum 1350 snap 418 seq_num 1350) 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3959: finishing write tid 1 to trial11545420-24 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3958: done (2 left) 2026-02-17T06:10:33.138 INFO:tasks.rados.rados.0.trial115.stdout:3961: snap_create 2026-02-17T06:10:33.139 INFO:tasks.rados.rados.0.trial115.stdout:3960: finishing rollback tid 1 to trial11545420-35 2026-02-17T06:10:33.139 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-17T06:10:33.140 INFO:tasks.rados.rados.0.trial115.stdout:3959: finishing write tid 2 to trial11545420-24 2026-02-17T06:10:33.140 INFO:tasks.rados.rados.0.trial115.stdout:3959: finishing write tid 3 to trial11545420-24 2026-02-17T06:10:33.140 INFO:tasks.rados.rados.0.trial115.stdout:3959: finishing write tid 4 to trial11545420-24 2026-02-17T06:10:33.140 INFO:tasks.rados.rados.0.trial115.stdout:3959: oid 24 updating version 0 to 2020 2026-02-17T06:10:33.140 INFO:tasks.rados.rados.0.trial115.stdout:3959: oid 24 updating version 2020 to 2022 2026-02-17T06:10:33.140 INFO:tasks.rados.rados.0.trial115.stdout:3959: oid 24 version 2022 is already newer than 2021 2026-02-17T06:10:33.140 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 24 v 2022 (ObjNum 1351 snap 418 seq_num 1351) dirty exists 2026-02-17T06:10:33.140 INFO:tasks.rados.rados.0.trial115.stdout:3959: left oid 24 (ObjNum 1351 snap 418 seq_num 1351) 2026-02-17T06:10:34.120 INFO:tasks.rados.rados.0.trial115.stdout:3959: done (2 left) 2026-02-17T06:10:34.121 INFO:tasks.rados.rados.0.trial115.stdout:3960: done (1 left) 2026-02-17T06:10:34.121 INFO:tasks.rados.rados.0.trial115.stdout:3961: done (0 left) 2026-02-17T06:10:34.121 INFO:tasks.rados.rados.0.trial115.stdout:3962: read oid 40 snap -1 2026-02-17T06:10:34.121 INFO:tasks.rados.rados.0.trial115.stdout:3962: expect (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:10:34.121 INFO:tasks.rados.rados.0.trial115.stdout:3963: read oid 31 snap -1 2026-02-17T06:10:34.121 INFO:tasks.rados.rados.0.trial115.stdout:3963: expect (ObjNum 1162 snap 355 seq_num 1162) 2026-02-17T06:10:34.121 INFO:tasks.rados.rados.0.trial115.stdout:3964: write oid 29 current snap is 419 2026-02-17T06:10:34.121 INFO:tasks.rados.rados.0.trial115.stdout:3964: seq_num 1352 ranges {723064=731129,1949302=764476} 2026-02-17T06:10:34.126 INFO:tasks.rados.rados.0.trial115.stdout:3964: writing trial11545420-29 from 723064 to 1454193 tid 1 2026-02-17T06:10:34.131 INFO:tasks.rados.rados.0.trial115.stdout:3964: writing trial11545420-29 from 1949302 to 2713778 tid 2 2026-02-17T06:10:34.131 INFO:tasks.rados.rados.0.trial115.stdout:3965: write (excl) oid 21 current snap is 419 2026-02-17T06:10:34.131 INFO:tasks.rados.rados.0.trial115.stdout:3965: seq_num 1353 ranges {739871=406378,1607784=508134} 2026-02-17T06:10:34.135 INFO:tasks.rados.rados.0.trial115.stdout:3965: writing trial11545420-21 from 739871 to 1146249 tid 1 2026-02-17T06:10:34.139 INFO:tasks.rados.rados.0.trial115.stdout:3965: writing trial11545420-21 from 1607784 to 2115918 tid 2 2026-02-17T06:10:34.139 INFO:tasks.rados.rados.0.trial115.stdout:3966: read oid 30 snap -1 2026-02-17T06:10:34.139 INFO:tasks.rados.rados.0.trial115.stdout:3966: expect (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:10:34.139 INFO:tasks.rados.rados.0.trial115.stdout:3967: rollback oid 42 current snap is 419 2026-02-17T06:10:34.139 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 42 to 418 2026-02-17T06:10:34.139 INFO:tasks.rados.rados.0.trial115.stdout:3963: expect (ObjNum 1162 snap 355 seq_num 1162) 2026-02-17T06:10:34.290 INFO:tasks.rados.rados.0.trial115.stdout:3962: expect (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:10:34.497 INFO:tasks.rados.rados.0.trial115.stdout:3964: finishing write tid 1 to trial11545420-29 2026-02-17T06:10:34.497 INFO:tasks.rados.rados.0.trial115.stdout:3964: finishing write tid 2 to trial11545420-29 2026-02-17T06:10:34.497 INFO:tasks.rados.rados.0.trial115.stdout:3964: finishing write tid 3 to trial11545420-29 2026-02-17T06:10:34.497 INFO:tasks.rados.rados.0.trial115.stdout:3962: done (5 left) 2026-02-17T06:10:34.497 INFO:tasks.rados.rados.0.trial115.stdout:3963: done (4 left) 2026-02-17T06:10:34.497 INFO:tasks.rados.rados.0.trial115.stdout:3968: delete oid 11 current snap is 419 2026-02-17T06:10:34.498 INFO:tasks.rados.rados.0.trial115.stdout:3964: finishing write tid 4 to trial11545420-29 2026-02-17T06:10:34.498 INFO:tasks.rados.rados.0.trial115.stdout:3964: oid 29 updating version 0 to 1504 2026-02-17T06:10:34.498 INFO:tasks.rados.rados.0.trial115.stdout:3964: oid 29 updating version 1504 to 1505 2026-02-17T06:10:34.498 INFO:tasks.rados.rados.0.trial115.stdout:3964: oid 29 updating version 1505 to 1506 2026-02-17T06:10:34.498 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 29 v 1506 (ObjNum 1352 snap 419 seq_num 1352) dirty exists 2026-02-17T06:10:34.498 INFO:tasks.rados.rados.0.trial115.stdout:3964: left oid 29 (ObjNum 1352 snap 419 seq_num 1352) 2026-02-17T06:10:34.498 INFO:tasks.rados.rados.0.trial115.stdout:3967: finishing rollback tid 0 to trial11545420-42 2026-02-17T06:10:34.498 INFO:tasks.rados.rados.0.trial115.stdout:3967: finishing rollback tid 1 to trial11545420-42 2026-02-17T06:10:34.499 INFO:tasks.rados.rados.0.trial115.stdout:3967: finishing rollback tid 2 to trial11545420-42 2026-02-17T06:10:34.499 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 42 v 2026 (ObjNum 1312 snap 404 seq_num 1312) dirty exists 2026-02-17T06:10:34.499 INFO:tasks.rados.rados.0.trial115.stdout:3965: finishing write tid 1 to trial11545420-21 2026-02-17T06:10:34.499 INFO:tasks.rados.rados.0.trial115.stdout:3965: finishing write tid 2 to trial11545420-21 2026-02-17T06:10:34.499 INFO:tasks.rados.rados.0.trial115.stdout:3966: expect (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3965: finishing write tid 3 to trial11545420-21 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3965: finishing write tid 4 to trial11545420-21 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3965: oid 21 updating version 0 to 2011 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3965: oid 21 updating version 2011 to 2013 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3965: oid 21 version 2013 is already newer than 2012 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 21 v 2013 (ObjNum 1353 snap 419 seq_num 1353) dirty exists 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3965: left oid 21 (ObjNum 1353 snap 419 seq_num 1353) 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3964: done (4 left) 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3965: done (3 left) 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3966: done (2 left) 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3967: done (1 left) 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3968: done (0 left) 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3969: read oid 44 snap -1 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3969: expect (ObjNum 1342 snap 414 seq_num 1342) 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3970: write (excl) oid 35 current snap is 419 2026-02-17T06:10:34.701 INFO:tasks.rados.rados.0.trial115.stdout:3970: seq_num 1355 ranges {773485=673229,2124748=511803} 2026-02-17T06:10:34.706 INFO:tasks.rados.rados.0.trial115.stdout:3970: writing trial11545420-35 from 773485 to 1446714 tid 1 2026-02-17T06:10:34.710 INFO:tasks.rados.rados.0.trial115.stdout:3970: writing trial11545420-35 from 2124748 to 2636551 tid 2 2026-02-17T06:10:34.710 INFO:tasks.rados.rados.0.trial115.stdout:3971: copy_from oid 23 from oid 20 current snap is 419 2026-02-17T06:10:34.711 INFO:tasks.rados.rados.0.trial115.stdout:3969: expect (ObjNum 1342 snap 414 seq_num 1342) 2026-02-17T06:10:34.832 INFO:tasks.rados.rados.0.trial115.stdout:3970: finishing write tid 1 to trial11545420-35 2026-02-17T06:10:34.832 INFO:tasks.rados.rados.0.trial115.stdout:3970: finishing write tid 2 to trial11545420-35 2026-02-17T06:10:34.833 INFO:tasks.rados.rados.0.trial115.stdout:3970: finishing write tid 3 to trial11545420-35 2026-02-17T06:10:34.833 INFO:tasks.rados.rados.0.trial115.stdout:3970: finishing write tid 4 to trial11545420-35 2026-02-17T06:10:34.833 INFO:tasks.rados.rados.0.trial115.stdout:3970: oid 35 updating version 0 to 2038 2026-02-17T06:10:34.833 INFO:tasks.rados.rados.0.trial115.stdout:3970: oid 35 updating version 2038 to 2039 2026-02-17T06:10:34.833 INFO:tasks.rados.rados.0.trial115.stdout:3970: oid 35 version 2039 is already newer than 2037 2026-02-17T06:10:34.833 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 35 v 2039 (ObjNum 1355 snap 419 seq_num 1355) dirty exists 2026-02-17T06:10:34.834 INFO:tasks.rados.rados.0.trial115.stdout:3970: left oid 35 (ObjNum 1355 snap 419 seq_num 1355) 2026-02-17T06:10:34.834 INFO:tasks.rados.rados.0.trial115.stdout:3971: finishing copy_from racing read to trial11545420-23 2026-02-17T06:10:34.834 INFO:tasks.rados.rados.0.trial115.stdout:3971: finishing copy_from to trial11545420-23 2026-02-17T06:10:34.834 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1900 (ObjNum 1168 snap 359 seq_num 1168) dirty exists 2026-02-17T06:10:34.834 INFO:tasks.rados.rados.0.trial115.stdout:3969: done (2 left) 2026-02-17T06:10:34.835 INFO:tasks.rados.rados.0.trial115.stdout:3970: done (1 left) 2026-02-17T06:10:34.835 INFO:tasks.rados.rados.0.trial115.stdout:3971: done (0 left) 2026-02-17T06:10:34.835 INFO:tasks.rados.rados.0.trial115.stdout:3972: delete oid 17 current snap is 419 2026-02-17T06:10:34.836 INFO:tasks.rados.rados.0.trial115.stdout:3972: done (0 left) 2026-02-17T06:10:34.836 INFO:tasks.rados.rados.0.trial115.stdout:3973: write (excl) oid 18 current snap is 419 2026-02-17T06:10:34.836 INFO:tasks.rados.rados.0.trial115.stdout:3973: seq_num 1357 ranges {407099=423727,1441712=515472,2582060=225039} 2026-02-17T06:10:34.837 INFO:tasks.rados.rados.0.trial115.stdout:3973: writing trial11545420-18 from 407099 to 830826 tid 1 2026-02-17T06:10:34.841 INFO:tasks.rados.rados.0.trial115.stdout:3973: writing trial11545420-18 from 1441712 to 1957184 tid 2 2026-02-17T06:10:34.844 INFO:tasks.rados.rados.0.trial115.stdout:3973: writing trial11545420-18 from 2582060 to 2807099 tid 3 2026-02-17T06:10:34.844 INFO:tasks.rados.rados.0.trial115.stdout:3974: write (excl) oid 1 current snap is 419 2026-02-17T06:10:34.845 INFO:tasks.rados.rados.0.trial115.stdout:3974: seq_num 1358 ranges {423906=498976,1500194=659130,2609483=214423} 2026-02-17T06:10:34.847 INFO:tasks.rados.rados.0.trial115.stdout:3974: writing trial11545420-1 from 423906 to 922882 tid 1 2026-02-17T06:10:34.852 INFO:tasks.rados.rados.0.trial115.stdout:3974: writing trial11545420-1 from 1500194 to 2159324 tid 2 2026-02-17T06:10:34.855 INFO:tasks.rados.rados.0.trial115.stdout:3974: writing trial11545420-1 from 2609483 to 2823906 tid 3 2026-02-17T06:10:34.856 INFO:tasks.rados.rados.0.trial115.stdout:3975: snap_create 2026-02-17T06:10:34.856 INFO:tasks.rados.rados.0.trial115.stdout:3973: finishing write tid 1 to trial11545420-18 2026-02-17T06:10:34.856 INFO:tasks.rados.rados.0.trial115.stdout:3973: finishing write tid 2 to trial11545420-18 2026-02-17T06:10:34.856 INFO:tasks.rados.rados.0.trial115.stdout:3973: finishing write tid 3 to trial11545420-18 2026-02-17T06:10:34.856 INFO:tasks.rados.rados.0.trial115.stdout:3973: finishing write tid 4 to trial11545420-18 2026-02-17T06:10:34.857 INFO:tasks.rados.rados.0.trial115.stdout:3973: finishing write tid 5 to trial11545420-18 2026-02-17T06:10:34.857 INFO:tasks.rados.rados.0.trial115.stdout:3973: oid 18 updating version 0 to 2016 2026-02-17T06:10:34.857 INFO:tasks.rados.rados.0.trial115.stdout:3973: oid 18 version 2016 is already newer than 2014 2026-02-17T06:10:34.857 INFO:tasks.rados.rados.0.trial115.stdout:3973: oid 18 version 2016 is already newer than 2015 2026-02-17T06:10:34.857 INFO:tasks.rados.rados.0.trial115.stdout:3973: oid 18 updating version 2016 to 2017 2026-02-17T06:10:34.857 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 18 v 2017 (ObjNum 1357 snap 419 seq_num 1357) dirty exists 2026-02-17T06:10:34.857 INFO:tasks.rados.rados.0.trial115.stdout:3973: left oid 18 (ObjNum 1357 snap 419 seq_num 1357) 2026-02-17T06:10:34.858 INFO:tasks.rados.rados.0.trial115.stdout:3974: finishing write tid 1 to trial11545420-1 2026-02-17T06:10:34.858 INFO:tasks.rados.rados.0.trial115.stdout:3974: finishing write tid 2 to trial11545420-1 2026-02-17T06:10:34.859 INFO:tasks.rados.rados.0.trial115.stdout:3974: finishing write tid 3 to trial11545420-1 2026-02-17T06:10:34.859 INFO:tasks.rados.rados.0.trial115.stdout:3974: finishing write tid 4 to trial11545420-1 2026-02-17T06:10:34.859 INFO:tasks.rados.rados.0.trial115.stdout:3974: finishing write tid 5 to trial11545420-1 2026-02-17T06:10:34.859 INFO:tasks.rados.rados.0.trial115.stdout:3974: oid 1 updating version 0 to 1716 2026-02-17T06:10:34.860 INFO:tasks.rados.rados.0.trial115.stdout:3974: oid 1 version 1716 is already newer than 1715 2026-02-17T06:10:34.860 INFO:tasks.rados.rados.0.trial115.stdout:3974: oid 1 version 1716 is already newer than 1713 2026-02-17T06:10:34.860 INFO:tasks.rados.rados.0.trial115.stdout:3974: oid 1 version 1716 is already newer than 1714 2026-02-17T06:10:34.860 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 1 v 1716 (ObjNum 1358 snap 419 seq_num 1358) dirty exists 2026-02-17T06:10:34.860 INFO:tasks.rados.rados.0.trial115.stdout:3974: left oid 1 (ObjNum 1358 snap 419 seq_num 1358) 2026-02-17T06:10:35.505 INFO:tasks.rados.rados.0.trial115.stdout:3973: done (2 left) 2026-02-17T06:10:35.505 INFO:tasks.rados.rados.0.trial115.stdout:3974: done (1 left) 2026-02-17T06:10:35.505 INFO:tasks.rados.rados.0.trial115.stdout:3975: done (0 left) 2026-02-17T06:10:35.506 INFO:tasks.rados.rados.0.trial115.stdout:3976: snap_remove snap 406 2026-02-17T06:10:36.506 INFO:tasks.rados.rados.0.trial115.stdout:3976: done (0 left) 2026-02-17T06:10:36.506 INFO:tasks.rados.rados.0.trial115.stdout:3977: snap_create 2026-02-17T06:10:37.507 INFO:tasks.rados.rados.0.trial115.stdout:3977: done (0 left) 2026-02-17T06:10:37.507 INFO:tasks.rados.rados.0.trial115.stdout:3978: snap_create 2026-02-17T06:10:38.119 INFO:tasks.rados.rados.0.trial115.stdout:3978: done (0 left) 2026-02-17T06:10:38.119 INFO:tasks.rados.rados.0.trial115.stdout:3979: write oid 17 current snap is 422 2026-02-17T06:10:38.119 INFO:tasks.rados.rados.0.trial115.stdout:3979: seq_num 1359 ranges {440713=574225,1442323=519141,2636906=203807} 2026-02-17T06:10:38.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: writing trial11545420-17 from 440713 to 1014938 tid 1 2026-02-17T06:10:38.127 INFO:tasks.rados.rados.0.trial115.stdout:3979: writing trial11545420-17 from 1442323 to 1961464 tid 2 2026-02-17T06:10:38.131 INFO:tasks.rados.rados.0.trial115.stdout:3979: writing trial11545420-17 from 2636906 to 2840713 tid 3 2026-02-17T06:10:38.131 INFO:tasks.rados.rados.0.trial115.stdout:3980: snap_remove snap 414 2026-02-17T06:10:39.122 INFO:tasks.rados.rados.0.trial115.stdout:3980: done (1 left) 2026-02-17T06:10:39.122 INFO:tasks.rados.rados.0.trial115.stdout:3981: read oid 37 snap -1 2026-02-17T06:10:39.122 INFO:tasks.rados.rados.0.trial115.stdout:3981: expect (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:10:39.122 INFO:tasks.rados.rados.0.trial115.stdout:3982: delete oid 3 current snap is 422 2026-02-17T06:10:39.122 INFO:tasks.rados.rados.0.trial115.stdout:3979: finishing write tid 1 to trial11545420-17 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: finishing write tid 2 to trial11545420-17 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: finishing write tid 3 to trial11545420-17 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: finishing write tid 4 to trial11545420-17 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: finishing write tid 5 to trial11545420-17 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: oid 17 updating version 0 to 1902 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: oid 17 updating version 1902 to 1903 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: oid 17 updating version 1903 to 1905 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: oid 17 version 1905 is already newer than 1904 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1905 (ObjNum 1359 snap 422 seq_num 1359) dirty exists 2026-02-17T06:10:39.123 INFO:tasks.rados.rados.0.trial115.stdout:3979: left oid 17 (ObjNum 1359 snap 422 seq_num 1359) 2026-02-17T06:10:39.127 INFO:tasks.rados.rados.0.trial115.stdout:3979: done (2 left) 2026-02-17T06:10:39.127 INFO:tasks.rados.rados.0.trial115.stdout:3982: done (1 left) 2026-02-17T06:10:39.127 INFO:tasks.rados.rados.0.trial115.stdout:3983: write (excl) oid 9 current snap is 422 2026-02-17T06:10:39.128 INFO:tasks.rados.rados.0.trial115.stdout:3983: seq_num 1361 ranges {474327=441076,1675640=522810,2808105=66222} 2026-02-17T06:10:39.131 INFO:tasks.rados.rados.0.trial115.stdout:3983: writing trial11545420-9 from 474327 to 915403 tid 1 2026-02-17T06:10:39.136 INFO:tasks.rados.rados.0.trial115.stdout:3983: writing trial11545420-9 from 1675640 to 2198450 tid 2 2026-02-17T06:10:39.138 INFO:tasks.rados.rados.0.trial115.stdout:3983: writing trial11545420-9 from 2808105 to 2874327 tid 3 2026-02-17T06:10:39.139 INFO:tasks.rados.rados.0.trial115.stdout:3984: copy_from oid 26 from oid 12 current snap is 422 2026-02-17T06:10:39.139 INFO:tasks.rados.rados.0.trial115.stdout:3981: expect (ObjNum 1274 snap 398 seq_num 1274) 2026-02-17T06:10:39.342 INFO:tasks.rados.rados.0.trial115.stdout:3983: finishing write tid 1 to trial11545420-9 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3983: finishing write tid 2 to trial11545420-9 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3984: finishing copy_from to trial11545420-26 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3984: got expected ENOENT (src dne) 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3984: finishing copy_from racing read to trial11545420-26 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3983: finishing write tid 3 to trial11545420-9 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3983: finishing write tid 4 to trial11545420-9 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3983: finishing write tid 5 to trial11545420-9 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3983: oid 9 updating version 0 to 2083 2026-02-17T06:10:39.343 INFO:tasks.rados.rados.0.trial115.stdout:3983: oid 9 updating version 2083 to 2085 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:3983: oid 9 updating version 2085 to 2086 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:3983: oid 9 version 2086 is already newer than 2084 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 9 v 2086 (ObjNum 1361 snap 422 seq_num 1361) dirty exists 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:3983: left oid 9 (ObjNum 1361 snap 422 seq_num 1361) 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:3981: done (2 left) 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:3983: done (1 left) 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:3984: done (0 left) 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:3985: write oid 4 current snap is 422 2026-02-17T06:10:39.344 INFO:tasks.rados.rados.0.trial115.stdout:3985: seq_num 1362 ranges {491134=516325,1734122=666468} 2026-02-17T06:10:39.346 INFO:tasks.rados.rados.0.trial115.stdout:3985: writing trial11545420-4 from 491134 to 1007459 tid 1 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:3985: writing trial11545420-4 from 1734122 to 2400590 tid 2 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:3986: rollback oid 34 current snap is 422 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 34 to 421 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:3987: rollback oid 31 current snap is 422 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 31 to 421 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:3988: copy_from oid 40 from oid 45 current snap is 422 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:3989: copy_from oid 14 from oid 32 current snap is 422 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:3990: rollback oid 23 current snap is 422 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:rollback oid 23 to 416 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:3991: write (excl) oid 39 current snap is 422 2026-02-17T06:10:39.351 INFO:tasks.rados.rados.0.trial115.stdout:3991: seq_num 1363 ranges {507941=591574,1676251=526479,2746598=161343} 2026-02-17T06:10:39.355 INFO:tasks.rados.rados.0.trial115.stdout:3991: writing trial11545420-39 from 507941 to 1099515 tid 1 2026-02-17T06:10:39.359 INFO:tasks.rados.rados.0.trial115.stdout:3991: writing trial11545420-39 from 1676251 to 2202730 tid 2 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3991: writing trial11545420-39 from 2746598 to 2907941 tid 3 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3992: copy_from oid 7 from oid 22 current snap is 422 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3985: finishing write tid 1 to trial11545420-4 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3987: finishing rollback tid 0 to trial11545420-31 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3985: finishing write tid 2 to trial11545420-4 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3987: finishing rollback tid 1 to trial11545420-31 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3985: finishing write tid 3 to trial11545420-4 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3987: finishing rollback tid 2 to trial11545420-31 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 31 v 1727 (ObjNum 1162 snap 355 seq_num 1162) dirty exists 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3987: done (7 left) 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3993: write (excl) oid 41 current snap is 422 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3985: finishing write tid 4 to trial11545420-4 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3985: oid 4 updating version 0 to 1515 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3985: oid 4 updating version 1515 to 1516 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3985: oid 4 updating version 1516 to 1517 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 4 v 1517 (ObjNum 1362 snap 422 seq_num 1362) dirty exists 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3985: left oid 4 (ObjNum 1362 snap 422 seq_num 1362) 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3988: finishing copy_from to trial11545420-40 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3988: got expected ENOENT (src dne) 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3988: finishing copy_from racing read to trial11545420-40 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3986: finishing rollback tid 0 to trial11545420-34 2026-02-17T06:10:39.362 INFO:tasks.rados.rados.0.trial115.stdout:3986: finishing rollback tid 1 to trial11545420-34 2026-02-17T06:10:39.363 INFO:tasks.rados.rados.0.trial115.stdout:3986: finishing rollback tid 2 to trial11545420-34 2026-02-17T06:10:39.363 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 34 v 1510 (ObjNum 1285 snap 400 seq_num 1285) dirty exists 2026-02-17T06:10:39.363 INFO:tasks.rados.rados.0.trial115.stdout:3990: finishing rollback tid 0 to trial11545420-23 2026-02-17T06:10:39.363 INFO:tasks.rados.rados.0.trial115.stdout:3993: seq_num 1364 ranges {524748=666823,1734733=670137} 2026-02-17T06:10:39.366 INFO:tasks.rados.rados.0.trial115.stdout:3993: writing trial11545420-41 from 524748 to 1191571 tid 1 2026-02-17T06:10:39.371 INFO:tasks.rados.rados.0.trial115.stdout:3993: writing trial11545420-41 from 1734733 to 2404870 tid 2 2026-02-17T06:10:39.371 INFO:tasks.rados.rados.0.trial115.stdout:3985: done (7 left) 2026-02-17T06:10:39.371 INFO:tasks.rados.rados.0.trial115.stdout:3986: done (6 left) 2026-02-17T06:10:39.371 INFO:tasks.rados.rados.0.trial115.stdout:3988: done (5 left) 2026-02-17T06:10:39.371 INFO:tasks.rados.rados.0.trial115.stdout:3994: write oid 17 current snap is 422 2026-02-17T06:10:39.371 INFO:tasks.rados.rados.0.trial115.stdout:3994: seq_num 1365 ranges {541555=742072,1793215=530148,2801444=140111} 2026-02-17T06:10:39.375 INFO:tasks.rados.rados.0.trial115.stdout:3994: writing trial11545420-17 from 541555 to 1283627 tid 1 2026-02-17T06:10:39.379 INFO:tasks.rados.rados.0.trial115.stdout:3994: writing trial11545420-17 from 1793215 to 2323363 tid 2 2026-02-17T06:10:39.382 INFO:tasks.rados.rados.0.trial115.stdout:3994: writing trial11545420-17 from 2801444 to 2941555 tid 3 2026-02-17T06:10:39.382 INFO:tasks.rados.rados.0.trial115.stdout:3995: write oid 40 current snap is 422 2026-02-17T06:10:39.382 INFO:tasks.rados.rados.0.trial115.stdout:3990: finishing rollback tid 1 to trial11545420-23 2026-02-17T06:10:39.382 INFO:tasks.rados.rados.0.trial115.stdout:3990: finishing rollback tid 2 to trial11545420-23 2026-02-17T06:10:39.382 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 23 v 1908 (ObjNum 1289 snap 400 seq_num 1289) dirty exists 2026-02-17T06:10:39.383 INFO:tasks.rados.rados.0.trial115.stdout:3995: seq_num 1366 ranges {558362=417321,1451697=673806,2712514=245848} 2026-02-17T06:10:39.386 INFO:tasks.rados.rados.0.trial115.stdout:3995: writing trial11545420-40 from 558362 to 975683 tid 1 2026-02-17T06:10:39.390 INFO:tasks.rados.rados.0.trial115.stdout:3995: writing trial11545420-40 from 1451697 to 2125503 tid 2 2026-02-17T06:10:39.393 INFO:tasks.rados.rados.0.trial115.stdout:3995: writing trial11545420-40 from 2712514 to 2958362 tid 3 2026-02-17T06:10:39.393 INFO:tasks.rados.rados.0.trial115.stdout:3990: done (6 left) 2026-02-17T06:10:39.393 INFO:tasks.rados.rados.0.trial115.stdout:3996: write (excl) oid 2 current snap is 422 2026-02-17T06:10:39.393 INFO:tasks.rados.rados.0.trial115.stdout:3996: seq_num 1367 ranges {575169=492570,1793826=417464,2623584=351585} 2026-02-17T06:10:39.397 INFO:tasks.rados.rados.0.trial115.stdout:3996: writing trial11545420-2 from 575169 to 1067739 tid 1 2026-02-17T06:10:39.401 INFO:tasks.rados.rados.0.trial115.stdout:3996: writing trial11545420-2 from 1793826 to 2211290 tid 2 2026-02-17T06:10:39.404 INFO:tasks.rados.rados.0.trial115.stdout:3996: writing trial11545420-2 from 2623584 to 2975169 tid 3 2026-02-17T06:10:39.404 INFO:tasks.rados.rados.0.trial115.stdout:3997: copy_from oid 6 from oid 4 current snap is 422 2026-02-17T06:10:39.404 INFO:tasks.rados.rados.0.trial115.stdout:3991: finishing write tid 1 to trial11545420-39 2026-02-17T06:10:39.404 INFO:tasks.rados.rados.0.trial115.stdout:3989: finishing copy_from racing read to trial11545420-14 2026-02-17T06:10:39.404 INFO:tasks.rados.rados.0.trial115.stdout:3989: finishing copy_from to trial11545420-14 2026-02-17T06:10:39.404 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 14 v 1906 (ObjNum 1333 snap 414 seq_num 1333) dirty exists 2026-02-17T06:10:39.404 INFO:tasks.rados.rados.0.trial115.stdout:3991: finishing write tid 2 to trial11545420-39 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3989: done (7 left) 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3998: delete oid 30 current snap is 422 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3991: finishing write tid 3 to trial11545420-39 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3991: finishing write tid 4 to trial11545420-39 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3991: finishing write tid 5 to trial11545420-39 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3991: oid 39 updating version 0 to 2046 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3991: oid 39 version 2046 is already newer than 2044 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3991: oid 39 version 2046 is already newer than 2045 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3991: oid 39 updating version 2046 to 2047 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 39 v 2047 (ObjNum 1363 snap 422 seq_num 1363) dirty exists 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3991: left oid 39 (ObjNum 1363 snap 422 seq_num 1363) 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3992: finishing copy_from racing read to trial11545420-7 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3992: finishing copy_from to trial11545420-7 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 7 v 1564 (ObjNum 1320 snap 407 seq_num 1320) dirty exists 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3993: finishing write tid 1 to trial11545420-41 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3993: finishing write tid 2 to trial11545420-41 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3993: finishing write tid 3 to trial11545420-41 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3993: finishing write tid 4 to trial11545420-41 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3993: oid 41 updating version 0 to 2020 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3993: oid 41 version 2020 is already newer than 2018 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3993: oid 41 version 2020 is already newer than 2019 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 41 v 2020 (ObjNum 1364 snap 422 seq_num 1364) dirty exists 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3993: left oid 41 (ObjNum 1364 snap 422 seq_num 1364) 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: finishing write tid 1 to trial11545420-17 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: finishing write tid 2 to trial11545420-17 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: finishing write tid 3 to trial11545420-17 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: finishing write tid 4 to trial11545420-17 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: finishing write tid 5 to trial11545420-17 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: oid 17 updating version 0 to 1912 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: oid 17 version 1912 is already newer than 1911 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: oid 17 version 1912 is already newer than 1909 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: oid 17 version 1912 is already newer than 1910 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 17 v 1912 (ObjNum 1365 snap 422 seq_num 1365) dirty exists 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3994: left oid 17 (ObjNum 1365 snap 422 seq_num 1365) 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3995: finishing write tid 1 to trial11545420-40 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3995: finishing write tid 2 to trial11545420-40 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3995: finishing write tid 3 to trial11545420-40 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3995: finishing write tid 4 to trial11545420-40 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3995: finishing write tid 5 to trial11545420-40 2026-02-17T06:10:39.405 INFO:tasks.rados.rados.0.trial115.stdout:3995: oid 40 updating version 0 to 1521 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3995: oid 40 version 1521 is already newer than 1520 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3995: oid 40 version 1521 is already newer than 1518 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3995: oid 40 version 1521 is already newer than 1519 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 40 v 1521 (ObjNum 1366 snap 422 seq_num 1366) dirty exists 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3995: left oid 40 (ObjNum 1366 snap 422 seq_num 1366) 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: finishing write tid 1 to trial11545420-2 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: finishing write tid 2 to trial11545420-2 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: finishing write tid 3 to trial11545420-2 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: finishing write tid 4 to trial11545420-2 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: finishing write tid 5 to trial11545420-2 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: oid 2 updating version 0 to 1731 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: oid 2 version 1731 is already newer than 1729 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: oid 2 version 1731 is already newer than 1730 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: oid 2 updating version 1731 to 1732 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 2 v 1732 (ObjNum 1367 snap 422 seq_num 1367) dirty exists 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3996: left oid 2 (ObjNum 1367 snap 422 seq_num 1367) 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3991: done (7 left) 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3992: done (6 left) 2026-02-17T06:10:39.406 INFO:tasks.rados.rados.0.trial115.stdout:3993: done (5 left) 2026-02-17T06:10:39.407 INFO:tasks.rados.rados.0.trial115.stdout:3994: done (4 left) 2026-02-17T06:10:39.407 INFO:tasks.rados.rados.0.trial115.stdout:3995: done (3 left) 2026-02-17T06:10:39.407 INFO:tasks.rados.rados.0.trial115.stdout:3996: done (2 left) 2026-02-17T06:10:39.407 INFO:tasks.rados.rados.0.trial115.stdout:3998: done (1 left) 2026-02-17T06:10:39.407 INFO:tasks.rados.rados.0.trial115.stdout:3999: read oid 25 snap 409 2026-02-17T06:10:39.407 INFO:tasks.rados.rados.0.trial115.stdout:3999: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:10:39.407 INFO:tasks.rados.rados.0.trial115.stdout: waiting on 2 2026-02-17T06:10:39.410 INFO:tasks.rados.rados.0.trial115.stdout:3997: finishing copy_from racing read to trial11545420-6 2026-02-17T06:10:39.411 INFO:tasks.rados.rados.0.trial115.stdout:3999: expect (ObjNum 1123 snap 336 seq_num 1123) 2026-02-17T06:10:39.481 INFO:tasks.rados.rados.0.trial115.stdout:3997: finishing copy_from to trial11545420-6 2026-02-17T06:10:39.481 INFO:tasks.rados.rados.0.trial115.stdout:update_object_version oid 6 v 1517 (ObjNum 1362 snap 422 seq_num 1362) dirty exists 2026-02-17T06:10:39.481 INFO:tasks.rados.rados.0.trial115.stdout:3997: done (1 left) 2026-02-17T06:10:39.481 INFO:tasks.rados.rados.0.trial115.stdout:3999: done (0 left) 2026-02-17T06:10:39.482 INFO:tasks.rados.rados.0.trial115.stderr:0 errors. 2026-02-17T06:10:39.482 INFO:tasks.rados.rados.0.trial115.stderr: 2026-02-17T06:10:39.484 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.549+0000 7fd48e21a640 10 monclient: get_monmap_and_config 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.549+0000 7fd48e21a640 10 monclient: build_initial_monmap 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.549+0000 7fd48e21a640 10 monclient: monmap: 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:39.550146+0000 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:39.550146+0000 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:39.559 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 1 Processor -- start 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 1 -- start start 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient: init 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient: _add_conns ranks=[4,3,2,1,0] 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd48806c890 0x7fd48806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient(hunting): picked mon.noname-c con 0x7fd48806c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd48809c4a0 0x7fd48806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient(hunting): picked mon.noname-a con 0x7fd48809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd488135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient(hunting): picked mon.noname-b con 0x7fd488134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fd4880a0880 con 0x7fd48809c4a0 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd48806d6f0 con 0x7fd48806c890 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd48806daf0 con 0x7fd488134df0 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd48e21a640 10 monclient(hunting): authenticate will time out at 2085.103687s 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd4877fe640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7fd48806c890 msgr2=0x7fd48806cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:39.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd4877fe640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd48806c890 0x7fd48806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd48809c4a0 0x7fd48806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd487fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd488135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fd48809c4a0 0x7fd48806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57826/0 (socket says 10.20.193.78:57826) 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 1 -- 10.20.193.78:0/447975530 learned_addr learned my addr 10.20.193.78:0/447975530 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 10 monclient(hunting): get_auth_request con 0x7fd48809c4a0 auth_method 0 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd487fff640 10 monclient(hunting): get_auth_request con 0x7fd488134df0 auth_method 0 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd487fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd487fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd487fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd487fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd487fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd487fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 10 monclient(hunting): handle_auth_done global_id 284286 payload 274 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 1 -- 10.20.193.78:0/447975530 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 msgr2=0x7fd488135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 1 --2- 10.20.193.78:0/447975530 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd488135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 1 -- 10.20.193.78:0/447975530 >> v2:10.20.193.115:3300/0 conn(0x7fd48806c890 msgr2=0x7fd48806cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:39.563 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.550+0000 7fd486ffd640 1 --2- 10.20.193.78:0/447975530 >> v2:10.20.193.115:3300/0 conn(0x7fd48806c890 0x7fd48806cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.551+0000 7fd486ffd640 10 monclient: _finish_hunting 0 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.551+0000 7fd486ffd640 1 monclient: found mon.noname-a 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.551+0000 7fd486ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.551+0000 7fd486ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.551+0000 7fd486ffd640 1 -- 10.20.193.78:0/447975530 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48809cc80 con 0x7fd48809c4a0 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd486ffd640 1 -- 10.20.193.78:0/447975530 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48809c4a0 msgr2=0x7fd48806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd486ffd640 1 --2- 10.20.193.78:0/447975530 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48809c4a0 0x7fd48806d1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd48806a480 tx=0x7fd47802ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2df991f802337c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 1 -- 10.20.193.78:0/447975530 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47803c070 con 0x7fd48809c4a0 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:10:39.564 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: dump: 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: _finish_auth 0 2026-02-17T06:10:39.565 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: _check_auth_tickets 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 1 -- 10.20.193.78:0/447975530 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47802fcc0 con 0x7fd48809c4a0 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 5 monclient: authenticate success, global_id 284286 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 1 -- 10.20.193.78:0/447975530 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478037660 con 0x7fd48809c4a0 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 4 set_mon_vals no callback set 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd4867fc640 10 monclient: dump: 2026-02-17T06:10:39.566 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 10 monclient: shutdown 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:39.567 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 -- 10.20.193.78:0/447975530 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48809c4a0 msgr2=0x7fd48806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 --2- 10.20.193.78:0/447975530 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48809c4a0 0x7fd48806d1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd48806a480 tx=0x7fd47802ee80 comp rx=0 tx=0).stop 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 -- 10.20.193.78:0/447975530 shutdown_connections 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 --2- 10.20.193.78:0/447975530 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd488135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 --2- 10.20.193.78:0/447975530 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48809c4a0 0x7fd48806d1b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 --2- 10.20.193.78:0/447975530 >> v2:10.20.193.115:3300/0 conn(0x7fd48806c890 0x7fd48806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 -- 10.20.193.78:0/447975530 >> 10.20.193.78:0/447975530 conn(0x7fd4880a6620 msgr2=0x7fd4880a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 -- 10.20.193.78:0/447975530 shutdown_connections 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.553+0000 7fd48e21a640 1 -- 10.20.193.78:0/447975530 wait complete. 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient: build_initial_monmap 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient: monmap: 2026-02-17T06:10:39.568 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 1 Processor -- start 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 1 -- start start 2026-02-17T06:10:39.569 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient: init 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48806c890 0x7fd4881cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient(hunting): picked mon.noname-a con 0x7fd48806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd48809c4a0 0x7fd488122ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient(hunting): picked mon.noname-b con 0x7fd48809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd4881231e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient(hunting): picked mon.noname-c con 0x7fd488134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.570 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880a01a0 con 0x7fd48806c890 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880a0020 con 0x7fd48809c4a0 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd48809be20 con 0x7fd488134df0 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.554+0000 7fd48e21a640 10 monclient(hunting): authenticate will time out at 2085.107961s 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48806c890 0x7fd4881cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd486ffd640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd48809c4a0 msgr2=0x7fd488122ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd4881231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd486ffd640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd48809c4a0 0x7fd488122ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd4881231e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59668/0 (socket says 10.20.193.78:59668) 2026-02-17T06:10:39.571 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48806c890 0x7fd4881cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:57832/0 (socket says 10.20.193.78:57832) 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 -- 10.20.193.78:0/220474135 learned_addr learned my addr 10.20.193.78:0/220474135 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient(hunting): get_auth_request con 0x7fd488134df0 auth_method 0 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 10 monclient(hunting): get_auth_request con 0x7fd48806c890 auth_method 0 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient(hunting): handle_auth_done global_id 264482 payload 824 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 -- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd48809c4a0 msgr2=0x7fd488122ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd48809c4a0 0x7fd488122ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 -- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48806c890 msgr2=0x7fd4881cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48806c890 0x7fd4881cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient: _finish_hunting 0 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 monclient: found mon.noname-c 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:39.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd487fff640 1 -- 10.20.193.78:0/220474135 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd488126b90 con 0x7fd488134df0 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.555+0000 7fd4877fe640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48806c890 0x7fd4881cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd487fff640 1 -- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 msgr2=0x7fd4881231e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd487fff640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd4881231e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009d20 tx=0x7fd47c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 1 -- 10.20.193.78:0/220474135 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c018020 con 0x7fd488134df0 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: dump: 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.574 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: _finish_auth 0 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: _check_auth_tickets 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 1 -- 10.20.193.78:0/220474135 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c014680 con 0x7fd488134df0 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd48e21a640 5 monclient: authenticate success, global_id 264482 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd48e21a640 10 monclient: _renew_subs 2026-02-17T06:10:39.575 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd48e21a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4880725f0 con 0x7fd488134df0 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 1 -- 10.20.193.78:0/220474135 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c014a70 con 0x7fd488134df0 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd485ffb640 4 set_mon_vals no callback set 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd463fff640 10 monclient: dump: 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.576 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd48e21a640 10 monclient: _renew_subs 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd48e21a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.557+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4881265d0 con 0x7fd488134df0 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd48e21a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd48e21a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd48e21a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd488074980 con 0x7fd488134df0 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd463fff640 1 -- 10.20.193.78:0/220474135 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd47c014c10 con 0x7fd488134df0 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd463fff640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd45803de60 0x7fd458040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.577 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd463fff640 1 -- 10.20.193.78:0/220474135 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(866..866 src has 525..866) ==== 4779+0+0 (secure 0 0 0) 0x7fd47c053460 con 0x7fd488134df0 2026-02-17T06:10:39.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd4877fe640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd45803de60 0x7fd458040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.558+0000 7fd4877fe640 10 monclient: get_auth_request con 0x7fd45803de60 auth_method 0 2026-02-17T06:10:39.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.560+0000 7fd463fff640 1 -- 10.20.193.78:0/220474135 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd47c022070 con 0x7fd488134df0 2026-02-17T06:10:39.578 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.560+0000 7fd463fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.560+0000 7fd463fff640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:39.579 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.561+0000 7fd4877fe640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd45803de60 0x7fd458040320 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd474006f70 tx=0x7fd474006ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.642 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.641+0000 7fd48e21a640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:10:39.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.641+0000 7fd48e21a640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:10:39.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.641+0000 7fd48e21a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.641+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd488135280 con 0x7fd488134df0 2026-02-17T06:10:39.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.642+0000 7fd463fff640 1 -- 10.20.193.78:0/220474135 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v866) ==== 74+0+14325 (secure 0 0 0) 0x7fd47c010040 con 0x7fd488134df0 2026-02-17T06:10:39.643 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.642+0000 7fd463fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:10:39.644 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.642+0000 7fd463fff640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:10:39.644 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T06:10:39.645 INFO:teuthology.orchestra.run.trial078.stdout:{"epoch":866,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T06:10:39.119968+0000","last_up_change":"2026-02-17T05:46:02.153660+0000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:46:03.931310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T05:46:08.458376+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2026-02-17T05:46:17.031199+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"866","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":429,"snap_epoch":866,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6800","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6801","nonce":3807268031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6802","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6803","nonce":3807268031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6806","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6807","nonce":3807268031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6804","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6805","nonce":3807268031}]},"public_addr":"10.20.193.78:6801/3807268031","cluster_addr":"10.20.193.78:6803/3807268031","heartbeat_back_addr":"10.20.193.78:6807/3807268031","heartbeat_front_addr":"10.20.193.78:6805/3807268031","state":["exists","up"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6808","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6809","nonce":1094981820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6810","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6811","nonce":1094981820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6814","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6815","nonce":1094981820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6812","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6813","nonce":1094981820}]},"public_addr":"10.20.193.78:6809/1094981820","cluster_addr":"10.20.193.78:6811/1094981820","heartbeat_back_addr":"10.20.193.78:6815/1094981820","heartbeat_front_addr":"10.20.193.78:6813/1094981820","state":["exists","up"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6816","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6817","nonce":707008561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6818","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6819","nonce":707008561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6822","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6823","nonce":707008561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6820","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6821","nonce":707008561}]},"public_addr":"10.20.193.78:6817/707008561","cluster_addr":"10.20.193.78:6819/707008561","heartbeat_back_addr":"10.20.193.78:6823/707008561","heartbeat_front_addr":"10.20.193.78:6821/707008561","state":["exists","up"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6810","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6812","nonce":2268846355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6814","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6816","nonce":2268846355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6822","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6824","nonce":2268846355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6818","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6820","nonce":2268846355}]},"public_addr":"10.20.193.115:6812/2268846355","cluster_addr":"10.20.193.115:6816/2268846355","heartbeat_back_addr":"10.20.193.115:6824/2268846355","heartbeat_front_addr":"10.20.193.115:6820/2268846355","state":["exists","up"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6802","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6803","nonce":2725813951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6804","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6805","nonce":2725813951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6808","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6809","nonce":2725813951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6806","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6807","nonce":2725813951}]},"public_addr":"10.20.193.115:6803/2725813951","cluster_addr":"10.20.193.115:6805/2725813951","heartbeat_back_addr":"10.20.193.115:6809/2725813951","heartbeat_front_addr":"10.20.193.115:6807/2725813951","state":["exists","up"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6811","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6813","nonce":2171295268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6815","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6817","nonce":2171295268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6823","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6825","nonce":2171295268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6819","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6821","nonce":2171295268}]},"public_addr":"10.20.193.115:6813/2171295268","cluster_addr":"10.20.193.115:6817/2171295268","heartbeat_back_addr":"10.20.193.115:6825/2171295268","heartbeat_front_addr":"10.20.193.115:6821/2171295268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.280056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.641140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.705253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:01.037254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.075730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.553045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":414,"length":1},{"begin":422,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":422,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T06:10:39.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd45803de60 msgr2=0x7fd458040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd45803de60 0x7fd458040320 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd474006f70 tx=0x7fd474006ec0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 10 monclient: shutdown 2026-02-17T06:10:39.647 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:39.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 msgr2=0x7fd4881231e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd4881231e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009d20 tx=0x7fd47c00c490 comp rx=0 tx=0).stop 2026-02-17T06:10:39.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 shutdown_connections 2026-02-17T06:10:39.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd488134df0 0x7fd4881231e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd45803de60 0x7fd458040320 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.648 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd48809c4a0 0x7fd488122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 --2- 10.20.193.78:0/220474135 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd48806c890 0x7fd4881cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 >> 10.20.193.78:0/220474135 conn(0x7fd4880a6620 msgr2=0x7fd4880a1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:39.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 shutdown_connections 2026-02-17T06:10:39.649 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.643+0000 7fd48e21a640 1 -- 10.20.193.78:0/220474135 wait complete. 2026-02-17T06:10:39.652 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient: get_monmap_and_config 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient: build_initial_monmap 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient: monmap: 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:39.720308+0000 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:39.720308+0000 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:39.720 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 1 Processor -- start 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 1 -- start start 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient: init 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient: _add_conns ranks=[4,1,2,3,0] 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f108006c890 0x7f108006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient(hunting): picked mon.noname-c con 0x7f108006c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f108009c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f1080135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient(hunting): picked mon.noname-b con 0x7f108006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f108006daf0 con 0x7f108009c4a0 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f10800a0880 con 0x7f108006c890 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f108009cc80 con 0x7f108006d1b0 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1087d79640 10 monclient(hunting): authenticate will time out at 2085.273868s 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1085aee640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f108006c890 msgr2=0x7f108006cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f1085aee640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f108006c890 0x7f108006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:39.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f10852ed640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f108009c4a0 legacy=0x7f1080134df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f10862ef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f1080135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f10862ef640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f1080135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59684/0 (socket says 10.20.193.78:59684) 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f10862ef640 1 -- 10.20.193.78:0/4259421854 learned_addr learned my addr 10.20.193.78:0/4259421854 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.720+0000 7f10862ef640 10 monclient(hunting): get_auth_request con 0x7f108006d1b0 auth_method 0 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient(hunting): handle_auth_done global_id 264485 payload 274 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 1 -- 10.20.193.78:0/4259421854 >> v2:10.20.193.115:3300/0 conn(0x7f108006c890 msgr2=0x7f108006cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 1 --2- 10.20.193.78:0/4259421854 >> v2:10.20.193.115:3300/0 conn(0x7f108006c890 0x7f108006cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 1 -- 10.20.193.78:0/4259421854 >> v1:10.20.193.115:6789/0 conn(0x7f108009c4a0 legacy=0x7f1080134df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient: _finish_hunting 0 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 1 monclient: found mon.noname-b 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.723 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.721+0000 7f10862ef640 1 -- 10.20.193.78:0/4259421854 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108009cf60 con 0x7f108006d1b0 2026-02-17T06:10:39.725 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f10862ef640 1 -- 10.20.193.78:0/4259421854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 msgr2=0x7f1080135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f10862ef640 1 --2- 10.20.193.78:0/4259421854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f1080135500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f108006dcc0 tx=0x7f107c02f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec1b499036bf435 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 1 -- 10.20.193.78:0/4259421854 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c0373f0 con 0x7f108006d1b0 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: dump: 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: _finish_auth 0 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: _check_auth_tickets 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1087d79640 5 monclient: authenticate success, global_id 264485 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 1 -- 10.20.193.78:0/4259421854 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107c037590 con 0x7f108006d1b0 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1087d79640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 1 -- 10.20.193.78:0/4259421854 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c037960 con 0x7f108006d1b0 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1087d79640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1087d79640 4 set_mon_vals no callback set 2026-02-17T06:10:39.726 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.723+0000 7f1084aec640 10 monclient: dump: 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient: shutdown 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- 10.20.193.78:0/4259421854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 msgr2=0x7f1080135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 --2- 10.20.193.78:0/4259421854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f1080135500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f108006dcc0 tx=0x7f107c02f230 comp rx=0 tx=0).stop 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- 10.20.193.78:0/4259421854 shutdown_connections 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 --2- 10.20.193.78:0/4259421854 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f1080135500 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 --2- 10.20.193.78:0/4259421854 >> v2:10.20.193.115:3300/0 conn(0x7f108006c890 0x7f108006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- 10.20.193.78:0/4259421854 >> 10.20.193.78:0/4259421854 conn(0x7f10800a6620 msgr2=0x7f10800a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- 10.20.193.78:0/4259421854 shutdown_connections 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- 10.20.193.78:0/4259421854 wait complete. 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient: build_initial_monmap 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient: monmap: 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:39.727 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 Processor -- start 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- start start 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient: init 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f108006c890 0x7f10801cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient(hunting): picked mon.noname-b con 0x7f108006c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f10801cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient(hunting): picked mon.noname-c con 0x7f108006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f108009c4a0 0x7f1080124d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient(hunting): picked mon.noname-a con 0x7f108009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10800a01a0 con 0x7f108009c4a0 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f10800a0020 con 0x7f108006c890 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f108009be20 con 0x7f108006d1b0 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.724+0000 7f1087d79640 10 monclient(hunting): authenticate will time out at 2085.277992s 2026-02-17T06:10:39.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f1085aee640 1 -- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f108006c890 msgr2=0x7f10801cf350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f1085aee640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f108006c890 0x7f10801cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f10801cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f10801cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59694/0 (socket says 10.20.193.78:59694) 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 -- 10.20.193.78:0/2719896758 learned_addr learned my addr 10.20.193.78:0/2719896758 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient(hunting): get_auth_request con 0x7f108006d1b0 auth_method 0 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10862ef640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f108009c4a0 0x7f1080124d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10862ef640 10 monclient(hunting): get_auth_request con 0x7f108009c4a0 auth_method 0 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10862ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10862ef640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10862ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10862ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10862ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10862ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient(hunting): handle_auth_done global_id 264488 payload 824 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 -- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f108006c890 msgr2=0x7f10801cf350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f108006c890 0x7f10801cf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 -- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f108009c4a0 msgr2=0x7f1080124d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f108009c4a0 0x7f1080124d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient: _finish_hunting 0 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 monclient: found mon.noname-c 2026-02-17T06:10:39.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:39.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.730 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.725+0000 7f10852ed640 1 -- 10.20.193.78:0/2719896758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1080128670 con 0x7f108006d1b0 2026-02-17T06:10:39.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f10852ed640 1 -- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 msgr2=0x7f10801cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:39.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f10852ed640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f10801cf890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f107400b780 tx=0x7f107400bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 1 -- 10.20.193.78:0/2719896758 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074005270 con 0x7f108006d1b0 2026-02-17T06:10:39.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:10:39.731 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: dump: 2026-02-17T06:10:39.731 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: _finish_auth 0 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 1 -- 10.20.193.78:0/2719896758 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1074010070 con 0x7f108006d1b0 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:39.732 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 1 -- 10.20.193.78:0/2719896758 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074005650 con 0x7f108006d1b0 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 5 monclient: authenticate success, global_id 264488 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 10 monclient: _renew_subs 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106ffff640 4 set_mon_vals no callback set 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1080128c30 con 0x7f108006d1b0 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f106e7fc640 10 monclient: dump: 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.733 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 10 monclient: _renew_subs 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1080128390 con 0x7f108006d1b0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.727+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108006cc70 con 0x7f108006d1b0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.728+0000 7f106e7fc640 1 -- 10.20.193.78:0/2719896758 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f107400c520 con 0x7f108006d1b0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.728+0000 7f106e7fc640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f105003deb0 0x7f1050040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.728+0000 7f106e7fc640 1 -- 10.20.193.78:0/2719896758 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(866..866 src has 525..866) ==== 4779+0+0 (secure 0 0 0) 0x7f1074052110 con 0x7f108006d1b0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.728+0000 7f1085aee640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f105003deb0 0x7f1050040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.728+0000 7f1085aee640 10 monclient: get_auth_request con 0x7f105003deb0 auth_method 0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.730+0000 7f106e7fc640 1 -- 10.20.193.78:0/2719896758 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f107401a060 con 0x7f108006d1b0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.730+0000 7f106e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.730+0000 7f106e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.730+0000 7f1085aee640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f105003deb0 0x7f1050040370 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1070003a30 tx=0x7f1070009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.734 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.730+0000 7f106e7fc640 1 -- 10.20.193.78:0/2719896758 DispatchQueue::entry inject delay of 0.002000 2026-02-17T06:10:39.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.811+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1080127220 con 0x7f105003deb0 2026-02-17T06:10:39.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.812+0000 7f106e7fc640 1 -- 10.20.193.78:0/2719896758 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+77192 (secure 0 0 0) 0x7f1080127220 con 0x7f105003deb0 2026-02-17T06:10:39.813 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T06:10:39.813 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f105003deb0 msgr2=0x7f1050040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f105003deb0 0x7f1050040370 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1070003a30 tx=0x7f1070009270 comp rx=0 tx=0).stop 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 10 monclient: shutdown 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 msgr2=0x7f10801cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f10801cf890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f107400b780 tx=0x7f107400bc50 comp rx=0 tx=0).stop 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 shutdown_connections 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f108009c4a0 0x7f1080124d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f108006d1b0 0x7f10801cf890 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f105003deb0 0x7f1050040370 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 --2- 10.20.193.78:0/2719896758 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f108006c890 0x7f10801cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 >> 10.20.193.78:0/2719896758 conn(0x7f10800a6620 msgr2=0x7f10800a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:39.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 shutdown_connections 2026-02-17T06:10:39.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.813+0000 7f1087d79640 1 -- 10.20.193.78:0/2719896758 wait complete. 2026-02-17T06:10:39.821 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":1593,"stamp":"2026-02-17T06:10:39.121098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184404281,"num_objects":87,"num_object_clones":38,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":6,"num_read":7840,"num_read_kb":7681682,"num_write":5710,"num_write_kb":2280260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19030,"ondisk_log_size":19030,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":532836,"kb_used_data":261400,"kb_used_omap":9,"kb_used_meta":271286,"kb_avail":565698204,"statfs":{"total":579820584960,"available":579274960896,"internally_reserved":0,"allocated":267673600,"data_stored":318744208,"data_compressed":9046386,"data_compressed_allocated":6651904,"data_compressed_original":19466582,"omap_allocated":9562,"internal_metadata":277797542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12569313,"num_objects":4,"num_object_clones":4,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":-2,"num_read":49,"num_read_kb":54514,"num_write":39,"num_write_kb":13233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.617353"},"pg_stats":[{"pgid":"3.f","version":"863'2043","reported_seq":6198,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.126512+0000","last_change":"2026-02-17T06:10:36.515874+0000","last_active":"2026-02-17T06:10:38.126512+0000","last_peered":"2026-02-17T06:10:38.126512+0000","last_clean":"2026-02-17T06:10:38.126512+0000","last_became_active":"2026-02-17T05:46:37.557090+0000","last_became_peered":"2026-02-17T05:46:37.557090+0000","last_unstale":"2026-02-17T06:10:38.126512+0000","last_undegraded":"2026-02-17T06:10:38.126512+0000","last_fullsized":"2026-02-17T06:10:38.126512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":2043,"log_dups_size":0,"ondisk_log_size":2043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:07:15.988555+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0041408310000000002,"stat_sum":{"num_bytes":15137477,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":889,"num_read_kb":950641,"num_write":591,"num_write_kb":254077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19e","length":"1"}]},{"pgid":"3.e","version":"863'878","reported_seq":5020,"reported_epoch":864,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.075467+0000","last_change":"2026-02-17T06:10:36.513233+0000","last_active":"2026-02-17T06:10:38.075467+0000","last_peered":"2026-02-17T06:10:38.075467+0000","last_clean":"2026-02-17T06:10:38.075467+0000","last_became_active":"2026-02-17T05:46:37.554917+0000","last_became_peered":"2026-02-17T05:46:37.554917+0000","last_unstale":"2026-02-17T06:10:38.075467+0000","last_undegraded":"2026-02-17T06:10:38.075467+0000","last_fullsized":"2026-02-17T06:10:38.075467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:59:25.835050+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003049118,"stat_sum":{"num_bytes":8510888,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":333,"num_read_kb":341637,"num_write":235,"num_write_kb":97860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"3.d","version":"863'1082","reported_seq":5192,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.119300+0000","last_change":"2026-02-17T06:10:36.514560+0000","last_active":"2026-02-17T06:10:38.119300+0000","last_peered":"2026-02-17T06:10:38.119300+0000","last_clean":"2026-02-17T06:10:38.119300+0000","last_became_active":"2026-02-17T05:46:37.557109+0000","last_became_peered":"2026-02-17T05:46:37.557109+0000","last_unstale":"2026-02-17T06:10:38.119300+0000","last_undegraded":"2026-02-17T06:10:38.119300+0000","last_fullsized":"2026-02-17T06:10:38.119300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":1082,"log_dups_size":0,"ondisk_log_size":1082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:58:38.894911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016473740000000001,"stat_sum":{"num_bytes":11211490,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":364,"num_read_kb":295378,"num_write":313,"num_write_kb":125529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"19e","length":"1"}]},{"pgid":"3.c","version":"865'744","reported_seq":4914,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.143391+0000","last_change":"2026-02-17T06:10:36.514921+0000","last_active":"2026-02-17T06:10:38.143391+0000","last_peered":"2026-02-17T06:10:38.143391+0000","last_clean":"2026-02-17T06:10:38.143391+0000","last_became_active":"2026-02-17T05:46:37.558441+0000","last_became_peered":"2026-02-17T05:46:37.558441+0000","last_unstale":"2026-02-17T06:10:38.143391+0000","last_undegraded":"2026-02-17T06:10:38.143391+0000","last_fullsized":"2026-02-17T06:10:38.143391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T16:49:14.125498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032420690000000002,"stat_sum":{"num_bytes":13413174,"num_objects":5,"num_object_clones":3,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":311,"num_read_kb":303307,"num_write":242,"num_write_kb":103642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19e","length":"1"}]},{"pgid":"3.b","version":"839'624","reported_seq":4803,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.507605+0000","last_change":"2026-02-17T06:10:36.507814+0000","last_active":"2026-02-17T06:10:36.507605+0000","last_peered":"2026-02-17T06:10:36.507605+0000","last_clean":"2026-02-17T06:10:36.507605+0000","last_became_active":"2026-02-17T05:46:37.557956+0000","last_became_peered":"2026-02-17T05:46:37.557956+0000","last_unstale":"2026-02-17T06:10:36.507605+0000","last_undegraded":"2026-02-17T06:10:36.507605+0000","last_fullsized":"2026-02-17T06:10:36.507605+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:40:56.415551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000581019,"stat_sum":{"num_bytes":3596995,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":257,"num_read_kb":272814,"num_write":194,"num_write_kb":75131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"3.a","version":"863'1566","reported_seq":5714,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.126492+0000","last_change":"2026-02-17T06:10:36.513736+0000","last_active":"2026-02-17T06:10:38.126492+0000","last_peered":"2026-02-17T06:10:38.126492+0000","last_clean":"2026-02-17T06:10:38.126492+0000","last_became_active":"2026-02-17T05:46:37.557102+0000","last_became_peered":"2026-02-17T05:46:37.557102+0000","last_unstale":"2026-02-17T06:10:38.126492+0000","last_undegraded":"2026-02-17T06:10:38.126492+0000","last_fullsized":"2026-02-17T06:10:38.126492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:38:32.648187+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.002072469,"stat_sum":{"num_bytes":8404424,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":661,"num_read_kb":654977,"num_write":454,"num_write_kb":158854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19e","length":"1"}]},{"pgid":"3.9","version":"863'1350","reported_seq":5519,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.126374+0000","last_change":"2026-02-17T06:10:36.513534+0000","last_active":"2026-02-17T06:10:38.126374+0000","last_peered":"2026-02-17T06:10:38.126374+0000","last_clean":"2026-02-17T06:10:38.126374+0000","last_became_active":"2026-02-17T05:46:37.558435+0000","last_became_peered":"2026-02-17T05:46:37.558435+0000","last_unstale":"2026-02-17T06:10:38.126374+0000","last_undegraded":"2026-02-17T06:10:38.126374+0000","last_fullsized":"2026-02-17T06:10:38.126374+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":1350,"log_dups_size":0,"ondisk_log_size":1350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T10:54:30.268674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001964407,"stat_sum":{"num_bytes":13321922,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":571,"num_read_kb":518877,"num_write":392,"num_write_kb":183923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19e","length":"1"}]},{"pgid":"3.8","version":"0'0","reported_seq":4087,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.505844+0000","last_change":"2026-02-17T06:10:36.505976+0000","last_active":"2026-02-17T06:10:36.505844+0000","last_peered":"2026-02-17T06:10:36.505844+0000","last_clean":"2026-02-17T06:10:36.505844+0000","last_became_active":"2026-02-17T05:46:37.559533+0000","last_became_peered":"2026-02-17T05:46:37.559533+0000","last_unstale":"2026-02-17T06:10:36.505844+0000","last_undegraded":"2026-02-17T06:10:36.505844+0000","last_fullsized":"2026-02-17T06:10:36.505844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:03:12.654644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031299099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"1.0","version":"16'112","reported_seq":1815,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.119216+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T06:10:38.119216+0000","last_peered":"2026-02-17T06:10:38.119216+0000","last_clean":"2026-02-17T06:10:38.119216+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T06:10:38.119216+0000","last_undegraded":"2026-02-17T06:10:38.119216+0000","last_fullsized":"2026-02-17T06:10:38.119216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":1678,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.119245+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T06:10:38.119245+0000","last_peered":"2026-02-17T06:10:38.119245+0000","last_clean":"2026-02-17T06:10:38.119245+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T06:10:38.119245+0000","last_undegraded":"2026-02-17T06:10:38.119245+0000","last_fullsized":"2026-02-17T06:10:38.119245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"863'454","reported_seq":4653,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.508961+0000","last_change":"2026-02-17T06:10:36.509134+0000","last_active":"2026-02-17T06:10:36.508961+0000","last_peered":"2026-02-17T06:10:36.508961+0000","last_clean":"2026-02-17T06:10:36.508961+0000","last_became_active":"2026-02-17T05:46:37.557758+0000","last_became_peered":"2026-02-17T05:46:37.557758+0000","last_unstale":"2026-02-17T06:10:36.508961+0000","last_undegraded":"2026-02-17T06:10:36.508961+0000","last_fullsized":"2026-02-17T06:10:36.508961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:09:31.022980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001911439,"stat_sum":{"num_bytes":5916691,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":218,"num_read_kb":197132,"num_write":135,"num_write_kb":54608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":1687,"reported_epoch":864,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.075393+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T06:10:38.075393+0000","last_peered":"2026-02-17T06:10:38.075393+0000","last_clean":"2026-02-17T06:10:38.075393+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T06:10:38.075393+0000","last_undegraded":"2026-02-17T06:10:38.075393+0000","last_fullsized":"2026-02-17T06:10:38.075393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"857'386","reported_seq":4538,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.507804+0000","last_change":"2026-02-17T06:10:36.508130+0000","last_active":"2026-02-17T06:10:36.507804+0000","last_peered":"2026-02-17T06:10:36.507804+0000","last_clean":"2026-02-17T06:10:36.507804+0000","last_became_active":"2026-02-17T05:46:37.559556+0000","last_became_peered":"2026-02-17T05:46:37.559556+0000","last_unstale":"2026-02-17T06:10:36.507804+0000","last_undegraded":"2026-02-17T06:10:36.507804+0000","last_fullsized":"2026-02-17T06:10:36.507804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:54:31.510708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023614339999999999,"stat_sum":{"num_bytes":3471144,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":184240,"num_write":149,"num_write_kb":61869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":1694,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.507066+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T06:10:36.507066+0000","last_peered":"2026-02-17T06:10:36.507066+0000","last_clean":"2026-02-17T06:10:36.507066+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T06:10:36.507066+0000","last_undegraded":"2026-02-17T06:10:36.507066+0000","last_fullsized":"2026-02-17T06:10:36.507066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"861'802","reported_seq":4929,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.505557+0000","last_change":"2026-02-17T06:10:36.505801+0000","last_active":"2026-02-17T06:10:36.505557+0000","last_peered":"2026-02-17T06:10:36.505557+0000","last_clean":"2026-02-17T06:10:36.505557+0000","last_became_active":"2026-02-17T05:46:37.557410+0000","last_became_peered":"2026-02-17T05:46:37.557410+0000","last_unstale":"2026-02-17T06:10:36.505557+0000","last_undegraded":"2026-02-17T06:10:36.505557+0000","last_fullsized":"2026-02-17T06:10:36.505557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:39:59.923590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040373900000000002,"stat_sum":{"num_bytes":6698881,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":336,"num_read_kb":356470,"num_write":245,"num_write_kb":107474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":1692,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.126323+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T06:10:38.126323+0000","last_peered":"2026-02-17T06:10:38.126323+0000","last_clean":"2026-02-17T06:10:38.126323+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T06:10:38.126323+0000","last_undegraded":"2026-02-17T06:10:38.126323+0000","last_fullsized":"2026-02-17T06:10:38.126323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"863'1583","reported_seq":5701,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.126404+0000","last_change":"2026-02-17T06:10:36.516464+0000","last_active":"2026-02-17T06:10:38.126404+0000","last_peered":"2026-02-17T06:10:38.126404+0000","last_clean":"2026-02-17T06:10:38.126404+0000","last_became_active":"2026-02-17T05:46:37.554661+0000","last_became_peered":"2026-02-17T05:46:37.554661+0000","last_unstale":"2026-02-17T06:10:38.126404+0000","last_undegraded":"2026-02-17T06:10:38.126404+0000","last_fullsized":"2026-02-17T06:10:38.126404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":1583,"log_dups_size":0,"ondisk_log_size":1583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:36:11.593303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047025419999999997,"stat_sum":{"num_bytes":14863517,"num_objects":8,"num_object_clones":5,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":567,"num_read_kb":548495,"num_write":443,"num_write_kb":186634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19e","length":"1"}]},{"pgid":"3.5","version":"863'1568","reported_seq":5701,"reported_epoch":864,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.075308+0000","last_change":"2026-02-17T06:10:36.513290+0000","last_active":"2026-02-17T06:10:38.075308+0000","last_peered":"2026-02-17T06:10:38.075308+0000","last_clean":"2026-02-17T06:10:38.075308+0000","last_became_active":"2026-02-17T05:46:37.558482+0000","last_became_peered":"2026-02-17T05:46:37.558482+0000","last_unstale":"2026-02-17T06:10:38.075308+0000","last_undegraded":"2026-02-17T06:10:38.075308+0000","last_fullsized":"2026-02-17T06:10:38.075308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":1568,"log_dups_size":0,"ondisk_log_size":1568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:08:55.305327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031103419999999999,"stat_sum":{"num_bytes":17783258,"num_objects":8,"num_object_clones":4,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":601,"num_read_kb":574302,"num_write":436,"num_write_kb":196676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":1671,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.505399+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T06:10:36.505399+0000","last_peered":"2026-02-17T06:10:36.505399+0000","last_clean":"2026-02-17T06:10:36.505399+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T06:10:36.505399+0000","last_undegraded":"2026-02-17T06:10:36.505399+0000","last_fullsized":"2026-02-17T06:10:36.505399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"863'2036","reported_seq":6107,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.511121+0000","last_change":"2026-02-17T06:10:36.511240+0000","last_active":"2026-02-17T06:10:36.511121+0000","last_peered":"2026-02-17T06:10:36.511121+0000","last_clean":"2026-02-17T06:10:36.511121+0000","last_became_active":"2026-02-17T05:46:37.559495+0000","last_became_peered":"2026-02-17T05:46:37.559495+0000","last_unstale":"2026-02-17T06:10:36.511121+0000","last_undegraded":"2026-02-17T06:10:36.511121+0000","last_fullsized":"2026-02-17T06:10:36.511121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":2036,"log_dups_size":0,"ondisk_log_size":2036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T10:49:29.411404+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0055217410000000001,"stat_sum":{"num_bytes":21585350,"num_objects":11,"num_object_clones":6,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":817,"num_read_kb":921149,"num_write":562,"num_write_kb":239934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":1677,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.119298+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T06:10:38.119298+0000","last_peered":"2026-02-17T06:10:38.119298+0000","last_clean":"2026-02-17T06:10:38.119298+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T06:10:38.119298+0000","last_undegraded":"2026-02-17T06:10:38.119298+0000","last_fullsized":"2026-02-17T06:10:38.119298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"863'2078","reported_seq":6178,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.511532+0000","last_change":"2026-02-17T06:10:36.511629+0000","last_active":"2026-02-17T06:10:36.511532+0000","last_peered":"2026-02-17T06:10:36.511532+0000","last_clean":"2026-02-17T06:10:36.511532+0000","last_became_active":"2026-02-17T05:46:37.559946+0000","last_became_peered":"2026-02-17T05:46:37.559946+0000","last_unstale":"2026-02-17T06:10:36.511532+0000","last_undegraded":"2026-02-17T06:10:36.511532+0000","last_fullsized":"2026-02-17T06:10:36.511532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":2078,"log_dups_size":0,"ondisk_log_size":2078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:41:15.910502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058476159999999999,"stat_sum":{"num_bytes":20735317,"num_objects":11,"num_object_clones":6,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":893,"num_read_kb":894524,"num_write":604,"num_write_kb":243443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":1670,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.505692+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T06:10:36.505692+0000","last_peered":"2026-02-17T06:10:36.505692+0000","last_clean":"2026-02-17T06:10:36.505692+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T06:10:36.505692+0000","last_undegraded":"2026-02-17T06:10:36.505692+0000","last_fullsized":"2026-02-17T06:10:36.505692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"863'1721","reported_seq":5874,"reported_epoch":863,"state":"active+clean","last_fresh":"2026-02-17T06:10:36.507583+0000","last_change":"2026-02-17T06:10:36.507755+0000","last_active":"2026-02-17T06:10:36.507583+0000","last_peered":"2026-02-17T06:10:36.507583+0000","last_clean":"2026-02-17T06:10:36.507583+0000","last_became_active":"2026-02-17T05:46:37.560062+0000","last_became_peered":"2026-02-17T05:46:37.560062+0000","last_unstale":"2026-02-17T06:10:36.507583+0000","last_undegraded":"2026-02-17T06:10:36.507583+0000","last_fullsized":"2026-02-17T06:10:36.507583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:17.202789+0000","last_clean_scrub_stamp":"2026-02-17T05:46:17.202789+0000","objects_scrubbed":0,"log_size":1721,"log_dups_size":0,"ondisk_log_size":1721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T08:28:54.894426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019990110000000002,"stat_sum":{"num_bytes":19163366,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":747,"num_read_kb":667666,"num_write":568,"num_write_kb":187932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19e","length":"1"},{"start":"1a2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":1677,"reported_epoch":865,"state":"active+clean","last_fresh":"2026-02-17T06:10:38.119295+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T06:10:38.119295+0000","last_peered":"2026-02-17T06:10:38.119295+0000","last_clean":"2026-02-17T06:10:38.119295+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T06:10:38.119295+0000","last_undegraded":"2026-02-17T06:10:38.119295+0000","last_fullsized":"2026-02-17T06:10:38.119295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":183813894,"num_objects":83,"num_object_clones":38,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":6,"num_read":7754,"num_read_kb":7681609,"num_write":5563,"num_write_kb":2277586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":243806208,"data_stored":308111193,"data_compressed":8982276,"data_compressed_allocated":6569984,"data_compressed_original":18442582,"omap_allocated":0,"internal_metadata":0},"log_size":18915,"ondisk_log_size":18915,"up":32,"acting":32,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50504,"kb_used_data":23536,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94321336,"statfs":{"total":96636764160,"available":96585048064,"internally_reserved":0,"allocated":24100864,"data_stored":25950334,"data_compressed":548397,"data_compressed_allocated":462848,"data_compressed_original":1544192,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640557,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105096,"kb_used_data":47604,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94266744,"statfs":{"total":96636764160,"available":96529145856,"internally_reserved":0,"allocated":48746496,"data_stored":56988391,"data_compressed":1304396,"data_compressed_allocated":1044480,"data_compressed_original":2576384,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607850,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":110864,"kb_used_data":61688,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94260976,"statfs":{"total":96636764160,"available":96523239424,"internally_reserved":0,"allocated":63168512,"data_stored":78979059,"data_compressed":2050483,"data_compressed_allocated":1556480,"data_compressed_original":4165632,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607850,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98604,"kb_used_data":41108,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94273236,"statfs":{"total":96636764160,"available":96535793664,"internally_reserved":0,"allocated":42094592,"data_stored":56079722,"data_compressed":2486872,"data_compressed_allocated":1478656,"data_compressed_original":5111808,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58448,"kb_used_data":27320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94313392,"statfs":{"total":96636764160,"available":96576913408,"internally_reserved":0,"allocated":27975680,"data_stored":29096861,"data_compressed":694503,"data_compressed_allocated":688128,"data_compressed_original":1914027,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109320,"kb_used_data":60144,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94262520,"statfs":{"total":96636764160,"available":96524820480,"internally_reserved":0,"allocated":61587456,"data_stored":71649841,"data_compressed":1961735,"data_compressed_allocated":1421312,"data_compressed_original":4154539,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58716160,"data_stored":71134946,"data_compressed":1961735,"data_compressed_allocated":1421312,"data_compressed_original":4154539,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23740416,"data_stored":26755884,"data_compressed":662448,"data_compressed_allocated":647168,"data_compressed_original":1402027,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37920768,"data_stored":54267673,"data_compressed":2486872,"data_compressed_allocated":1478656,"data_compressed_original":5111808,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58994688,"data_stored":77167010,"data_compressed":2050483,"data_compressed_allocated":1556480,"data_compressed_original":4165632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":44568576,"data_stored":55176323,"data_compressed":1304396,"data_compressed_allocated":1044480,"data_compressed_original":2576384,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19865600,"data_stored":23609357,"data_compressed":516342,"data_compressed_allocated":421888,"data_compressed_original":1032192,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:10:39.822 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2026-02-17T06:10:39.822 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.889+0000 7f5dac7d3640 10 monclient: get_monmap_and_config 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.889+0000 7f5dac7d3640 10 monclient: build_initial_monmap 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.889+0000 7f5dac7d3640 10 monclient: monmap: 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:10:39.890080+0000 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:10:39.890080+0000 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:10:39.890 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 1 Processor -- start 2026-02-17T06:10:39.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 1 -- start start 2026-02-17T06:10:39.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient: init 2026-02-17T06:10:39.895 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5da40a1020 0x7f5da40a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient(hunting): picked mon.noname-c con 0x7f5da40a1020 addr v2:10.20.193.115:3300/0 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1940 0x7f5da4135e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient(hunting): picked mon.noname-b con 0x7f5da40a1940 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5da4136340 addr v1:10.20.193.115:6789/0 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da409fed0 con 0x7f5da4136340 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f5da40a2980 con 0x7f5da40a1020 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5da40a0590 con 0x7f5da40a1940 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:39.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5dac7d3640 10 monclient(hunting): authenticate will time out at 2085.443615s 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5daa548640 1 -- >> v2:10.20.193.115:3300/0 conn(0x7f5da40a1020 msgr2=0x7f5da40a1400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5daa548640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f5da40a1020 0x7f5da40a1400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5daad49640 1 -- >> v1:10.20.193.115:6789/0 conn(0x7f5da4136340 legacy=0x7f5da413d260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.115:6789/0 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1940 0x7f5da4135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1940 0x7f5da4135e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59710/0 (socket says 10.20.193.78:59710) 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 1 -- 10.20.193.78:0/3639541652 learned_addr learned my addr 10.20.193.78:0/3639541652 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient(hunting): get_auth_request con 0x7f5da40a1940 auth_method 0 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient(hunting): handle_auth_done global_id 264491 payload 274 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 1 -- 10.20.193.78:0/3639541652 >> v2:10.20.193.115:3300/0 conn(0x7f5da40a1020 msgr2=0x7f5da40a1400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 1 --2- 10.20.193.78:0/3639541652 >> v2:10.20.193.115:3300/0 conn(0x7f5da40a1020 0x7f5da40a1400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 1 -- 10.20.193.78:0/3639541652 >> v1:10.20.193.115:6789/0 conn(0x7f5da4136340 legacy=0x7f5da413d260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient: _finish_hunting 0 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 1 monclient: found mon.noname-b 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.890+0000 7f5da9d47640 1 -- 10.20.193.78:0/3639541652 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da409d0a0 con 0x7f5da40a1940 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9d47640 1 -- 10.20.193.78:0/3639541652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1940 msgr2=0x7f5da4135e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9d47640 1 --2- 10.20.193.78:0/3639541652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1940 0x7f5da4135e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009970 tx=0x7f5d9802ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc9bab6f6df63f2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 1 -- 10.20.193.78:0/3639541652 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9803c070 con 0x7f5da40a1940 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: dump: 2026-02-17T06:10:39.898 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: _finish_auth 0 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: _check_auth_tickets 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 5 monclient: authenticate success, global_id 264491 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 1 -- 10.20.193.78:0/3639541652 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98037440 con 0x7f5da40a1940 2026-02-17T06:10:39.899 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 1 -- 10.20.193.78:0/3639541652 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98037810 con 0x7f5da40a1940 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 10 monclient: get_monmap_and_config success 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 4 set_mon_vals no callback set 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5da9546640 10 monclient: dump: 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.900 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 10 monclient: shutdown 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 1 -- 10.20.193.78:0/3639541652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1940 msgr2=0x7f5da4135e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/3639541652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1940 0x7f5da4135e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009970 tx=0x7f5d9802ef10 comp rx=0 tx=0).stop 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 1 -- 10.20.193.78:0/3639541652 shutdown_connections 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/3639541652 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1940 0x7f5da4135e00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/3639541652 >> v2:10.20.193.115:3300/0 conn(0x7f5da40a1020 0x7f5da40a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 1 -- 10.20.193.78:0/3639541652 >> 10.20.193.78:0/3639541652 conn(0x7f5da40a3300 msgr2=0x7f5da40a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 1 -- 10.20.193.78:0/3639541652 shutdown_connections 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.893+0000 7f5dac7d3640 1 -- 10.20.193.78:0/3639541652 wait complete. 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient: build_initial_monmap 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient: monmap: 2026-02-17T06:10:39.901 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 1 Processor -- start 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 1 -- start start 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient: init 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient: _reopen_session rank -1 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:10:39.902 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1020 0x7f5da4135400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient(hunting): picked mon.noname-c con 0x7f5da40a1020 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5da40a1940 0x7f5da41cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient(hunting): picked mon.noname-b con 0x7f5da40a1940 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5da4136340 0x7f5da41cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient(hunting): picked mon.noname-a con 0x7f5da4136340 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5da40a1d50 con 0x7f5da4136340 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.903 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5da40a1ed0 con 0x7f5da40a1940 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient(hunting): start opening mon connection 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5da40a0180 con 0x7f5da40a1020 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient(hunting): _renew_subs 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5dac7d3640 10 monclient(hunting): authenticate will time out at 2085.447789s 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1020 0x7f5da4135400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1020 0x7f5da4135400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:59712/0 (socket says 10.20.193.78:59712) 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 1 -- 10.20.193.78:0/1684187373 learned_addr learned my addr 10.20.193.78:0/1684187373 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 10 monclient(hunting): get_auth_request con 0x7f5da40a1020 auth_method 0 2026-02-17T06:10:39.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5da9d47640 1 -- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5da40a1940 msgr2=0x7f5da41cf4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:10.20.193.115:3300/0 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5da9d47640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5da40a1940 0x7f5da41cf4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daad49640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5da4136340 0x7f5da41cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daad49640 10 monclient(hunting): get_auth_request con 0x7f5da4136340 auth_method 0 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daad49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daad49640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daad49640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daad49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daad49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daad49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.894+0000 7f5daa548640 10 monclient(hunting): handle_auth_done global_id 264494 payload 824 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 1 -- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5da40a1940 msgr2=0x7f5da41cf4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5da40a1940 0x7f5da41cf4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 1 -- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5da4136340 msgr2=0x7f5da41cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:39.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5da4136340 0x7f5da41cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:39.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 10 monclient: _finish_hunting 0 2026-02-17T06:10:39.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 1 monclient: found mon.noname-c 2026-02-17T06:10:39.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:10:39.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.906 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.895+0000 7f5daa548640 1 -- 10.20.193.78:0/1684187373 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da4128e20 con 0x7f5da40a1020 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5daa548640 1 -- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1020 msgr2=0x7f5da4135400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5daa548640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1020 0x7f5da4135400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400b780 tx=0x7f5d9400bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 1 -- 10.20.193.78:0/1684187373 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940052d0 con 0x7f5da40a1020 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: dump: 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.907 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: _finish_auth 0 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: _check_auth_tickets 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 1 -- 10.20.193.78:0/1684187373 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d94010070 con 0x7f5da40a1020 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 5 monclient: authenticate success, global_id 264494 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 10 monclient: _renew_subs 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da409d810 con 0x7f5da40a1020 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 1 -- 10.20.193.78:0/1684187373 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940056d0 con 0x7f5da40a1020 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5da8d45640 4 set_mon_vals no callback set 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5d8affd640 10 monclient: dump: 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:10:39.908 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 10 monclient: _renew_subs 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da4128860 con 0x7f5da40a1020 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.897+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da4074440 con 0x7f5da40a1020 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.898+0000 7f5d8affd640 1 -- 10.20.193.78:0/1684187373 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5d9401d030 con 0x7f5da40a1020 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.898+0000 7f5d8affd640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5d7403de60 0x7f5d74040320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.898+0000 7f5d8affd640 1 -- 10.20.193.78:0/1684187373 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(866..866 src has 525..866) ==== 4779+0+0 (secure 0 0 0) 0x7f5d94051d60 con 0x7f5da40a1020 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.898+0000 7f5da9d47640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5d7403de60 0x7f5d74040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.898+0000 7f5da9d47640 10 monclient: get_auth_request con 0x7f5d7403de60 auth_method 0 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.900+0000 7f5d8affd640 1 -- 10.20.193.78:0/1684187373 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5da4074440 con 0x7f5da40a1020 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.900+0000 7f5d8affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.900+0000 7f5d8affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:10:39.909 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.901+0000 7f5da9d47640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5d7403de60 0x7f5d74040320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5d98036000 tx=0x7f5d98033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:10:39.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.981+0000 7f5dac7d3640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2026-02-17T06:10:39.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.981+0000 7f5dac7d3640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2026-02-17T06:10:39.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.981+0000 7f5dac7d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:10:39.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:39.981+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5da40a1400 con 0x7f5da40a1020 2026-02-17T06:10:40.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.127+0000 7f5d8affd640 1 -- 10.20.193.78:0/1684187373 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]=0 pool 'unique_pool_0' removed v867) ==== 175+0+0 (secure 0 0 0) 0x7f5d94057020 con 0x7f5da40a1020 2026-02-17T06:10:40.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.127+0000 7f5d8affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2026-02-17T06:10:40.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.127+0000 7f5d8affd640 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' removed 2026-02-17T06:10:40.127 INFO:teuthology.orchestra.run.trial078.stderr:pool 'unique_pool_0' removed 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5d7403de60 msgr2=0x7f5d74040320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5d7403de60 0x7f5d74040320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5d98036000 tx=0x7f5d98033000 comp rx=0 tx=0).stop 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 10 monclient: shutdown 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1020 msgr2=0x7f5da4135400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1020 0x7f5da4135400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400b780 tx=0x7f5d9400bc50 comp rx=0 tx=0).stop 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 shutdown_connections 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5da4136340 0x7f5da41cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5d7403de60 0x7f5d74040320 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5da40a1940 0x7f5da41cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:40.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 --2- 10.20.193.78:0/1684187373 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5da40a1020 0x7f5da4135400 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:10:40.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 >> 10.20.193.78:0/1684187373 conn(0x7f5da40a3300 msgr2=0x7f5da40a5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:10:40.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 shutdown_connections 2026-02-17T06:10:40.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:10:40.128+0000 7f5dac7d3640 1 -- 10.20.193.78:0/1684187373 wait complete. 2026-02-17T06:10:40.137 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2026-02-17T06:10:40.156 INFO:tasks.mon_thrash:joining mon_thrasher 2026-02-17T06:10:49.497 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-17T06:10:49.497 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-17T06:10:49.497 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-17T06:10:49.497 DEBUG:teuthology.orchestra.run.trial115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-17T06:10:49.500 INFO:tasks.ceph.mon.b:Started 2026-02-17T06:10:49.500 INFO:tasks.mon_thrash:Sending STOP to mon a 2026-02-17T06:10:49.500 INFO:tasks.ceph.mon.a:Sent signal 19 2026-02-17T06:10:49.500 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-17T06:11:04.501 INFO:tasks.mon_thrash:Sending CONT to mon a 2026-02-17T06:11:04.502 INFO:tasks.ceph.mon.a:Sent signal 18 2026-02-17T06:11:04.502 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:11:04.502 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:11:04.572 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.572+0000 7f606880e640 10 monclient: get_monmap_and_config 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.572+0000 7f606880e640 10 monclient: build_initial_monmap 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.572+0000 7f606880e640 10 monclient: monmap: 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:04.572901+0000 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:04.572901+0000 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:04.573 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:04.574 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:04.574 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:04.574 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 1 Processor -- start 2026-02-17T06:11:04.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 1 -- start start 2026-02-17T06:11:04.574 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient: init 2026-02-17T06:11:04.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:04.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f606006c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f606009d170 0x7f606013daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient(hunting): picked mon.noname-c con 0x7f606009d170 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606009b200 0x7f6060140040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient(hunting): picked mon.noname-b con 0x7f606009b200 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606009bf60 con 0x7f606006c890 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f606009c520 con 0x7f606009d170 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f606009c800 con 0x7f606009b200 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f606880e640 10 monclient(hunting): authenticate will time out at 2110.126479s 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066583640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f606006c890 0x7f606006cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46332/0 (socket says 10.20.193.78:46332) 2026-02-17T06:11:04.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066583640 1 -- 10.20.193.78:0/3378825949 learned_addr learned my addr 10.20.193.78:0/3378825949 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 1 --2- 10.20.193.78:0/3378825949 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606009b200 0x7f6060140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient(hunting): get_auth_request con 0x7f606009b200 auth_method 0 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6065d82640 1 --2- 10.20.193.78:0/3378825949 >> v2:10.20.193.115:3300/0 conn(0x7f606009d170 0x7f606013daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6065d82640 10 monclient(hunting): get_auth_request con 0x7f606009d170 auth_method 0 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6065d82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6065d82640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6065d82640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient(hunting): handle_auth_done global_id 264497 payload 274 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 1 -- 10.20.193.78:0/3378825949 >> v2:10.20.193.115:3300/0 conn(0x7f606009d170 msgr2=0x7f606013daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 1 --2- 10.20.193.78:0/3378825949 >> v2:10.20.193.115:3300/0 conn(0x7f606009d170 0x7f606013daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 1 -- 10.20.193.78:0/3378825949 >> v1:10.20.193.78:6789/0 conn(0x7f606006c890 legacy=0x7f606006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:04.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient: _finish_hunting 0 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 1 monclient: found mon.noname-b 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.573+0000 7f6066d84640 1 -- 10.20.193.78:0/3378825949 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606009c240 con 0x7f606009b200 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6066d84640 1 -- 10.20.193.78:0/3378825949 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606009b200 msgr2=0x7f6060140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6066d84640 1 --2- 10.20.193.78:0/3378825949 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606009b200 0x7f6060140040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6054009d90 tx=0x7f605402f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a6104130522ac44 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 1 -- 10.20.193.78:0/3378825949 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605403c070 con 0x7f606009b200 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: dump: 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.585 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: _finish_auth 0 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: _check_auth_tickets 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 1 -- 10.20.193.78:0/3378825949 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6054037440 con 0x7f606009b200 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 1 -- 10.20.193.78:0/3378825949 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054037810 con 0x7f606009b200 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 5 monclient: authenticate success, global_id 264497 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 4 set_mon_vals no callback set 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f6065581640 10 monclient: dump: 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.586 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 10 monclient: shutdown 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 1 -- 10.20.193.78:0/3378825949 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606009b200 msgr2=0x7f6060140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 1 --2- 10.20.193.78:0/3378825949 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606009b200 0x7f6060140040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6054009d90 tx=0x7f605402f190 comp rx=0 tx=0).stop 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 1 -- 10.20.193.78:0/3378825949 shutdown_connections 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 1 --2- 10.20.193.78:0/3378825949 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606009b200 0x7f6060140040 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 1 --2- 10.20.193.78:0/3378825949 >> v2:10.20.193.115:3300/0 conn(0x7f606009d170 0x7f606013daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 1 -- 10.20.193.78:0/3378825949 >> 10.20.193.78:0/3378825949 conn(0x7f60600a4230 msgr2=0x7f60600a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 1 -- 10.20.193.78:0/3378825949 shutdown_connections 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.576+0000 7f606880e640 1 -- 10.20.193.78:0/3378825949 wait complete. 2026-02-17T06:11:04.587 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient: build_initial_monmap 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient: monmap: 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 1 Processor -- start 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 1 -- start start 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient: init 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606006c890 0x7f60601cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient(hunting): picked mon.noname-c con 0x7f606006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:04.588 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f606009b200 0x7f60601cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient(hunting): picked mon.noname-b con 0x7f606009b200 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f606009d170 0x7f6060094640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient(hunting): picked mon.noname-a con 0x7f606009d170 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f606006d500 con 0x7f606009d170 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f606006d380 con 0x7f606009b200 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f606006d680 con 0x7f606006c890 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f606880e640 10 monclient(hunting): authenticate will time out at 2110.130720s 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606006c890 0x7f60601cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606006c890 0x7f60601cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60468/0 (socket says 10.20.193.78:60468) 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 -- 10.20.193.78:0/1470685622 learned_addr learned my addr 10.20.193.78:0/1470685622 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient(hunting): get_auth_request con 0x7f606006c890 auth_method 0 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066d84640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f606009d170 0x7f6060094640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066d84640 10 monclient(hunting): get_auth_request con 0x7f606009d170 auth_method 0 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066d84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066d84640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066d84640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.589 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066d84640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066d84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066d84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6065d82640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f606009b200 0x7f60601cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6065d82640 10 monclient(hunting): get_auth_request con 0x7f606009b200 auth_method 0 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6065d82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6065d82640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6065d82640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient(hunting): handle_auth_done global_id 264500 payload 824 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 -- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f606009b200 msgr2=0x7f60601cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f606009b200 0x7f60601cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 -- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f606009d170 msgr2=0x7f6060094640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f606009d170 0x7f6060094640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient: _finish_hunting 0 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 monclient: found mon.noname-c 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.577+0000 7f6066583640 1 -- 10.20.193.78:0/1470685622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60600a6d30 con 0x7f606006c890 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f6066583640 1 -- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606006c890 msgr2=0x7f60601cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f6066583640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606006c890 0x7f60601cfa80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f605800c780 tx=0x7f605800cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 1 -- 10.20.193.78:0/1470685622 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058005de0 con 0x7f606006c890 2026-02-17T06:11:04.590 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: dump: 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: _finish_auth 0 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: _check_auth_tickets 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 1 -- 10.20.193.78:0/1470685622 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6058007070 con 0x7f606006c890 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 1 -- 10.20.193.78:0/1470685622 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058007440 con 0x7f606006c890 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 10 monclient: dump: 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.591 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f6064d80640 4 set_mon_vals no callback set 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 5 monclient: authenticate success, global_id 264500 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 10 monclient: _renew_subs 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6060096bc0 con 0x7f606006c890 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 10 monclient: _renew_subs 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f606009e920 con 0x7f606006c890 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 1 -- 10.20.193.78:0/1470685622 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f6058007620 con 0x7f606006c890 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6060074980 con 0x7f606006c890 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003df00 0x7f60300403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.580+0000 7f604affd640 1 -- 10.20.193.78:0/1470685622 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f60580523c0 con 0x7f606006c890 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.581+0000 7f6065d82640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003df00 0x7f60300403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.592 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.581+0000 7f6065d82640 10 monclient: get_auth_request con 0x7f603003df00 auth_method 0 2026-02-17T06:11:04.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.583+0000 7f604affd640 1 -- 10.20.193.78:0/1470685622 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f60600a6d30 con 0x7f606006c890 2026-02-17T06:11:04.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.583+0000 7f604affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.583+0000 7f604affd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:04.593 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.583+0000 7f6065d82640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003df00 0x7f60300403c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6060095d20 tx=0x7f604c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.695+0000 7f606880e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:11:04.695 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.695+0000 7f606880e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:11:04.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.695+0000 7f606880e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:04.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.695+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f606006cc70 con 0x7f606006c890 2026-02-17T06:11:04.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f604affd640 1 -- 10.20.193.78:0/1470685622 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f6058050f40 con 0x7f606006c890 2026-02-17T06:11:04.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f604affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:11:04.696 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f604affd640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:11:04.696 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":276,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003df00 msgr2=0x7f60300403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003df00 0x7f60300403c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6060095d20 tx=0x7f604c008040 comp rx=0 tx=0).stop 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 10 monclient: shutdown 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606006c890 msgr2=0x7f60601cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606006c890 0x7f60601cfa80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f605800c780 tx=0x7f605800cc50 comp rx=0 tx=0).stop 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 shutdown_connections 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f606009d170 0x7f6060094640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f603003df00 0x7f60300403c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f606009b200 0x7f60601cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 --2- 10.20.193.78:0/1470685622 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f606006c890 0x7f60601cfa80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 >> 10.20.193.78:0/1470685622 conn(0x7f60600a4230 msgr2=0x7f60600a0520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.696+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 shutdown_connections 2026-02-17T06:11:04.697 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.697+0000 7f606880e640 1 -- 10.20.193.78:0/1470685622 wait complete. 2026-02-17T06:11:04.704 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:11:04.704 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-17T06:11:04.704 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-17T06:11:04.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient: get_monmap_and_config 2026-02-17T06:11:04.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient: build_initial_monmap 2026-02-17T06:11:04.772 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient: monmap: 2026-02-17T06:11:04.772 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:04.772 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.772 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:04.770396+0000 2026-02-17T06:11:04.772 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:04.770396+0000 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 1 Processor -- start 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 1 -- start start 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient: init 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2026-02-17T06:11:04.773 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcce406c890 0x7fcce406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient(hunting): picked mon.noname-a con 0x7fcce406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce409c4a0 0x7fcce406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient(hunting): picked mon.noname-b con 0x7fcce409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcce4134df0 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce406daf0 con 0x7fcce4134df0 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fcce40a0880 con 0x7fcce406c890 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcce406d6f0 con 0x7fcce409c4a0 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.770+0000 7fcceaf33640 10 monclient(hunting): authenticate will time out at 2110.323952s 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcce406c890 0x7fcce406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fccdbfff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce409c4a0 0x7fcce406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fcce406c890 0x7fcce406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58240/0 (socket says 10.20.193.78:58240) 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 1 -- 10.20.193.78:0/2192485372 learned_addr learned my addr 10.20.193.78:0/2192485372 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient(hunting): get_auth_request con 0x7fcce406c890 auth_method 0 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.774 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fccdbfff640 10 monclient(hunting): get_auth_request con 0x7fcce409c4a0 auth_method 0 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fccdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fccdbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fccdbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fccdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fccdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fccdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_done global_id 284301 payload 274 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 1 -- 10.20.193.78:0/2192485372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce409c4a0 msgr2=0x7fcce406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 1 --2- 10.20.193.78:0/2192485372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce409c4a0 0x7fcce406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 1 -- 10.20.193.78:0/2192485372 >> v1:10.20.193.115:6789/0 conn(0x7fcce4134df0 legacy=0x7fcce4135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient: _finish_hunting 0 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 1 monclient: found mon.noname-a 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.775 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.771+0000 7fcce8ca8640 1 -- 10.20.193.78:0/2192485372 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce409cc80 con 0x7fcce406c890 2026-02-17T06:11:04.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fcce8ca8640 1 -- 10.20.193.78:0/2192485372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce406c890 msgr2=0x7fcce406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fcce8ca8640 1 --2- 10.20.193.78:0/2192485372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce406c890 0x7fcce406cc70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcce406d870 tx=0x7fccd402ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2427f95ee82a1933 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 1 -- 10.20.193.78:0/2192485372 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd403c070 con 0x7fcce406c890 2026-02-17T06:11:04.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:04.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: dump: 2026-02-17T06:11:04.783 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: _finish_auth 0 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:11:04.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 1 -- 10.20.193.78:0/2192485372 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd402fc60 con 0x7fcce406c890 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fcceaf33640 5 monclient: authenticate success, global_id 284301 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fcceaf33640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 1 -- 10.20.193.78:0/2192485372 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4037660 con 0x7fcce406c890 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fcceaf33640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fcceaf33640 4 set_mon_vals no callback set 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.773+0000 7fccdb7fe640 10 monclient: dump: 2026-02-17T06:11:04.785 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.786 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 10 monclient: shutdown 2026-02-17T06:11:04.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:04.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 1 -- 10.20.193.78:0/2192485372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce406c890 msgr2=0x7fcce406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 1 --2- 10.20.193.78:0/2192485372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce406c890 0x7fcce406cc70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcce406d870 tx=0x7fccd402ee20 comp rx=0 tx=0).stop 2026-02-17T06:11:04.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 1 -- 10.20.193.78:0/2192485372 shutdown_connections 2026-02-17T06:11:04.787 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 1 --2- 10.20.193.78:0/2192485372 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce409c4a0 0x7fcce406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 1 --2- 10.20.193.78:0/2192485372 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce406c890 0x7fcce406cc70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 1 -- 10.20.193.78:0/2192485372 >> 10.20.193.78:0/2192485372 conn(0x7fcce40a6620 msgr2=0x7fcce40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 1 -- 10.20.193.78:0/2192485372 shutdown_connections 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 1 -- 10.20.193.78:0/2192485372 wait complete. 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 10 monclient: build_initial_monmap 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.774+0000 7fcceaf33640 10 monclient: monmap: 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:04.788 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 1 Processor -- start 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 1 -- start start 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient: init 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcce406c890 0x7fcce41cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.789 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient(hunting): picked mon.noname-b con 0x7fcce406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce409c4a0 0x7fcce4123910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient(hunting): picked mon.noname-a con 0x7fcce409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce4134df0 0x7fcce4123e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient(hunting): picked mon.noname-c con 0x7fcce4134df0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcce40a01a0 con 0x7fcce409c4a0 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcce40a0020 con 0x7fcce406c890 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcce409be20 con 0x7fcce4134df0 2026-02-17T06:11:04.790 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcceaf33640 10 monclient(hunting): authenticate will time out at 2110.328342s 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce409c4a0 0x7fcce4123910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce409c4a0 0x7fcce4123910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58254/0 (socket says 10.20.193.78:58254) 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcce94a9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce4134df0 0x7fcce4123e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 -- 10.20.193.78:0/233496845 learned_addr learned my addr 10.20.193.78:0/233496845 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient(hunting): get_auth_request con 0x7fcce409c4a0 auth_method 0 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcce94a9640 10 monclient(hunting): get_auth_request con 0x7fcce4134df0 auth_method 0 2026-02-17T06:11:04.791 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcce94a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcce94a9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcce94a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcce94a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcce94a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fcce94a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient(hunting): handle_auth_done global_id 284304 payload 824 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce4134df0 msgr2=0x7fcce4123e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcce4134df0 0x7fcce4123e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcce406c890 msgr2=0x7fcce41cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.792 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcce406c890 0x7fcce41cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient: _finish_hunting 0 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 monclient: found mon.noname-a 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.775+0000 7fccdbfff640 1 -- 10.20.193.78:0/233496845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce4128230 con 0x7fcce409c4a0 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccdbfff640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce409c4a0 msgr2=0x7fcce4123910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccdbfff640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce409c4a0 0x7fcce4123910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00ebe0 tx=0x7fcccc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc017070 con 0x7fcce409c4a0 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: dump: 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.793 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: _finish_auth 0 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: _check_auth_tickets 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcccc00ce50 con 0x7fcce409c4a0 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0136b0 con 0x7fcce409c4a0 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccd97fa640 10 monclient: dump: 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.794 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fcceaf33640 5 monclient: authenticate success, global_id 284304 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fccdaffd640 4 set_mon_vals no callback set 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fcceaf33640 10 monclient: _renew_subs 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fcceaf33640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce4127850 con 0x7fcce409c4a0 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fcceaf33640 10 monclient: _renew_subs 2026-02-17T06:11:04.795 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fcceaf33640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.777+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce409d810 con 0x7fcce409c4a0 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcceaf33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcceaf33640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce41cfde0 0x7fcce4127180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcceaf33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcce41276c0 con 0x7fcce41cfde0 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce41cfde0 0x7fcce4127180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 10 monclient: get_auth_request con 0x7fcce41cfde0 auth_method 0 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.796 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_done global_id 284307 payload 824 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce8ca8640 1 -- 10.20.193.78:0/233496845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcce4074980 con 0x7fcce41cfde0 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fcccc024060 con 0x7fcce409c4a0 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fccd97fa640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fccb403df00 0x7fccb40403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fcccc0527c0 con 0x7fcce409c4a0 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce94a9640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fccb403df00 0x7fccb40403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.778+0000 7fcce94a9640 10 monclient: get_auth_request con 0x7fccb403df00 auth_method 0 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fcce8ca8640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce41cfde0 msgr2=0x7fcce4127180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fcce8ca8640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce41cfde0 0x7fcce4127180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcce40a7490 tx=0x7fccd40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4046020 con 0x7fcce41cfde0 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 10 monclient: dump: 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.797 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fccd4037ad0 con 0x7fcce41cfde0 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce41cfde0 msgr2=0x7fcce4127180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce41cfde0 0x7fcce4127180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcce40a7490 tx=0x7fccd40047c0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce41cfde0 msgr2=0x7fcce4127180 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.780+0000 7fccd97fa640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce41cfde0 0x7fcce4127180 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcce40a7490 tx=0x7fccd40047c0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.798 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.781+0000 7fcce94a9640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fccb403df00 0x7fccb40403c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcce4126990 tx=0x7fcce0004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.799 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcceaf33640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcceaf33640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce4124390 0x7fcce41da430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcceaf33640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4135a70 con 0x7fcce4124390 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce4124390 0x7fcce41da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 10 monclient: get_auth_request con 0x7fcce4124390 auth_method 0 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 10 monclient(hunting): handle_auth_done global_id 284310 payload 824 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.783+0000 7fcce8ca8640 1 -- 10.20.193.78:0/233496845 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcce4125e40 con 0x7fcce4124390 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcce8ca8640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce4124390 msgr2=0x7fcce41da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcce8ca8640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce4124390 0x7fcce41da430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccd402ee60 tx=0x7fccd4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd403c040 con 0x7fcce4124390 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 10 monclient: dump: 2026-02-17T06:11:04.800 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 <== mon.0 v2:10.20.193.78:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7fccd4042450 con 0x7fcce4124390 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce4124390 msgr2=0x7fcce41da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce4124390 0x7fcce41da430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccd402ee60 tx=0x7fccd4008030 comp rx=0 tx=0).stop 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce4124390 msgr2=0x7fcce41da430 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fccd97fa640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce4124390 0x7fcce41da430 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccd402ee60 tx=0x7fccd4008030 comp rx=0 tx=0).stop 2026-02-17T06:11:04.801 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "state": "leader", 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 276, 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 115923, 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.802 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.803 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:11:04.804 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.805 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.806 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.807 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.808 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "num": 2 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "mgr": [ 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:11:04.809 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:04.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fccb403df00 msgr2=0x7fccb40403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fccb403df00 0x7fccb40403c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcce4126990 tx=0x7fcce0004200 comp rx=0 tx=0).stop 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 10 monclient: shutdown 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce409c4a0 msgr2=0x7fcce4123910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 1 --2- 10.20.193.78:0/233496845 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcce409c4a0 0x7fcce4123910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00ebe0 tx=0x7fcccc00c520 comp rx=0 tx=0).stop 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcce94a9640 1 -- 10.20.193.78:0/233496845 reap_dead start 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcce94a9640 1 -- 10.20.193.78:0/233496845 reap_dead start 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 shutdown_connections 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 >> 10.20.193.78:0/233496845 conn(0x7fcce40a6620 msgr2=0x7fcce406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 shutdown_connections 2026-02-17T06:11:04.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.786+0000 7fcceaf33640 1 -- 10.20.193.78:0/233496845 wait complete. 2026-02-17T06:11:04.811 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:11:04.811 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-17T06:11:04.811 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 10 monclient: get_monmap_and_config 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 10 monclient: build_initial_monmap 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 10 monclient: monmap: 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:04.878479+0000 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:04.878479+0000 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:04.878 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 1 Processor -- start 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 1 -- start start 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 10 monclient: init 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49406c890 0x7fa49406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 10 monclient(hunting): picked mon.noname-b con 0x7fa49406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.878+0000 7fa4990bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa49409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa4990bf640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa49406d1b0 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa4990bf640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa494135c10 con 0x7fa49409c4a0 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa4990bf640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa49406daf0 con 0x7fa49406d1b0 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa4990bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa4990bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa4940a0880 con 0x7fa49406c890 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa4990bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa4990bf640 10 monclient(hunting): authenticate will time out at 2110.432024s 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492575640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fa49409c4a0 0x7fa494134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46346/0 (socket says 10.20.193.78:46346) 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49406c890 0x7fa49406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492575640 1 -- 10.20.193.78:0/1027800895 learned_addr learned my addr 10.20.193.78:0/1027800895 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient(hunting): get_auth_request con 0x7fa49406c890 auth_method 0 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient(hunting): handle_auth_done global_id 264509 payload 274 2026-02-17T06:11:04.879 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 1 -- 10.20.193.78:0/1027800895 >> v1:10.20.193.115:6789/0 conn(0x7fa49406d1b0 legacy=0x7fa494135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 1 -- 10.20.193.78:0/1027800895 >> v1:10.20.193.78:6789/0 conn(0x7fa49409c4a0 legacy=0x7fa494134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient: _finish_hunting 0 2026-02-17T06:11:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 1 monclient: found mon.noname-b 2026-02-17T06:11:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:04.880 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.879+0000 7fa492d76640 1 -- 10.20.193.78:0/1027800895 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49409cc80 con 0x7fa49406c890 2026-02-17T06:11:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa492d76640 1 -- 10.20.193.78:0/1027800895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49406c890 msgr2=0x7fa49406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa492d76640 1 --2- 10.20.193.78:0/1027800895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49406c890 0x7fa49406cc70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa4880098d0 tx=0x7fa48802ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d839a5cc0084bf40 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 1 -- 10.20.193.78:0/1027800895 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48803c070 con 0x7fa49406c890 2026-02-17T06:11:04.881 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: dump: 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: _finish_auth 0 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: _check_auth_tickets 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 1 -- 10.20.193.78:0/1027800895 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48802fcc0 con 0x7fa49406c890 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 1 -- 10.20.193.78:0/1027800895 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488037660 con 0x7fa49406c890 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa4990bf640 5 monclient: authenticate success, global_id 264509 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.881+0000 7fa491d74640 10 monclient: dump: 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 4 set_mon_vals no callback set 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 10 monclient: shutdown 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:04.882 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 1 -- 10.20.193.78:0/1027800895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49406c890 msgr2=0x7fa49406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 1 --2- 10.20.193.78:0/1027800895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49406c890 0x7fa49406cc70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa4880098d0 tx=0x7fa48802ed60 comp rx=0 tx=0).stop 2026-02-17T06:11:04.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 1 -- 10.20.193.78:0/1027800895 shutdown_connections 2026-02-17T06:11:04.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 1 --2- 10.20.193.78:0/1027800895 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49406c890 0x7fa49406cc70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 1 -- 10.20.193.78:0/1027800895 >> 10.20.193.78:0/1027800895 conn(0x7fa4940a6620 msgr2=0x7fa4940a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:04.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 1 -- 10.20.193.78:0/1027800895 shutdown_connections 2026-02-17T06:11:04.883 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 1 -- 10.20.193.78:0/1027800895 wait complete. 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 10 monclient: build_initial_monmap 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 10 monclient: monmap: 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.882+0000 7fa4990bf640 1 Processor -- start 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 1 -- start start 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient: init 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa49406c890 0x7fa494133f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient(hunting): picked mon.noname-b con 0x7fa49406c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa49406d1b0 0x7fa4941344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient(hunting): picked mon.noname-a con 0x7fa49406d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49409c4a0 0x7fa494121a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient(hunting): picked mon.noname-c con 0x7fa49409c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4940a0020 con 0x7fa49406d1b0 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa49406d600 con 0x7fa49406c890 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa4940a01a0 con 0x7fa49409c4a0 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa4990bf640 10 monclient(hunting): authenticate will time out at 2110.436295s 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa49406d1b0 0x7fa4941344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa49406d1b0 0x7fa4941344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58272/0 (socket says 10.20.193.78:58272) 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 -- 10.20.193.78:0/1900923725 learned_addr learned my addr 10.20.193.78:0/1900923725 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:04.884 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient(hunting): get_auth_request con 0x7fa49406d1b0 auth_method 0 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49409c4a0 0x7fa494121a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 10 monclient(hunting): get_auth_request con 0x7fa49409c4a0 auth_method 0 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492d76640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa49406c890 0x7fa494133f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492d76640 10 monclient(hunting): get_auth_request con 0x7fa49406c890 auth_method 0 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient(hunting): handle_auth_done global_id 284313 payload 824 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49409c4a0 msgr2=0x7fa494121a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49409c4a0 0x7fa494121a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa49406c890 msgr2=0x7fa494133f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fa49406c890 0x7fa494133f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient: _finish_hunting 0 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 monclient: found mon.noname-a 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa492575640 1 -- 10.20.193.78:0/1900923725 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa494134f10 con 0x7fa49406d1b0 2026-02-17T06:11:04.885 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.883+0000 7fa493577640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa49409c4a0 0x7fa494121a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa492575640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa49406d1b0 msgr2=0x7fa4941344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa492575640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa49406d1b0 0x7fa4941344d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00c780 tx=0x7fa47c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c005de0 con 0x7fa49406d1b0 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: dump: 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.886 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: _finish_auth 0 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: _check_auth_tickets 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c007070 con 0x7fa49406d1b0 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c007440 con 0x7fa49406d1b0 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa491573640 4 set_mon_vals no callback set 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4737fe640 10 monclient: dump: 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4990bf640 5 monclient: authenticate success, global_id 284313 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4990bf640 10 monclient: _renew_subs 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4990bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa494124420 con 0x7fa49406d1b0 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4990bf640 10 monclient: _renew_subs 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4990bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.885+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49409d810 con 0x7fa49406d1b0 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa4990bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.887 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa4990bf640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4941cd3a0 0x7fa494121f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa4990bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa494125af0 con 0x7fa4941cd3a0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fa47c007620 con 0x7fa49406d1b0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4941cd3a0 0x7fa494121f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 10 monclient: get_auth_request con 0x7fa4941cd3a0 auth_method 0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 10 monclient(hunting): handle_auth_done global_id 264515 payload 824 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa492d76640 1 -- 10.20.193.78:0/1900923725 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa494068410 con 0x7fa4941cd3a0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa4737fe640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa46003deb0 0x7fa460040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fa47c052460 con 0x7fa49406d1b0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa493577640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa46003deb0 0x7fa460040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.886+0000 7fa493577640 10 monclient: get_auth_request con 0x7fa46003deb0 auth_method 0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.888+0000 7fa492d76640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4941cd3a0 msgr2=0x7fa494121f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.888+0000 7fa492d76640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4941cd3a0 0x7fa494121f40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa488039000 tx=0x7fa4880047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.888+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488048070 con 0x7fa4941cd3a0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.888+0000 7fa4737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.888+0000 7fa4737fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.888+0000 7fa4737fe640 10 monclient: dump: 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.888 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.889+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7fa48804d450 con 0x7fa4941cd3a0 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.889+0000 7fa4737fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.889+0000 7fa4737fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.889+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4941cd3a0 msgr2=0x7fa494121f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.889+0000 7fa4737fe640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4941cd3a0 0x7fa494121f40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa488039000 tx=0x7fa4880047c0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.889+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4941cd3a0 msgr2=0x7fa494121f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.889+0000 7fa4737fe640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa4941cd3a0 0x7fa494121f40 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa488039000 tx=0x7fa4880047c0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.889 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.889+0000 7fa493577640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa46003deb0 0x7fa460040370 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa494073e30 tx=0x7fa484004200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa4990bf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:11:04.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa4990bf640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa494122480 0x7fa4941d8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.891 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa4990bf640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fa494122890 con 0x7fa494122480 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa494122480 0x7fa4941d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 10 monclient: get_auth_request con 0x7fa494122480 auth_method 0 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 10 monclient(hunting): handle_auth_done global_id 264518 payload 824 2026-02-17T06:11:04.892 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.892+0000 7fa492d76640 1 -- 10.20.193.78:0/1900923725 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa49406cc70 con 0x7fa494122480 2026-02-17T06:11:04.893 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa492d76640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa494122480 msgr2=0x7fa4941d8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa492d76640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa494122480 0x7fa4941d8480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48802eda0 tx=0x7fa488008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48803c040 con 0x7fa494122480 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 10 monclient: dump: 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 <== mon.2 v2:10.20.193.78:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fa488041650 con 0x7fa494122480 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa494122480 msgr2=0x7fa4941d8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa494122480 0x7fa4941d8480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48802eda0 tx=0x7fa488008030 comp rx=0 tx=0).stop 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa494122480 msgr2=0x7fa4941d8480 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:11:04.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4737fe640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fa494122480 0x7fa4941d8480 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48802eda0 tx=0x7fa488008030 comp rx=0 tx=0).stop 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 276, 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 214278, 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.895 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.896 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:11:04.897 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.898 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.899 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.900 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:11:04.901 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:11:04.902 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: "num": 5 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:11:04.903 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:04.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa46003deb0 msgr2=0x7fa460040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fa46003deb0 0x7fa460040370 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa494073e30 tx=0x7fa484004200 comp rx=0 tx=0).stop 2026-02-17T06:11:04.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 10 monclient: shutdown 2026-02-17T06:11:04.904 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:04.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa49406d1b0 msgr2=0x7fa4941344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 1 --2- 10.20.193.78:0/1900923725 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fa49406d1b0 0x7fa4941344d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00c780 tx=0x7fa47c00cc50 comp rx=0 tx=0).stop 2026-02-17T06:11:04.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa493577640 1 -- 10.20.193.78:0/1900923725 reap_dead start 2026-02-17T06:11:04.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa493577640 1 -- 10.20.193.78:0/1900923725 reap_dead start 2026-02-17T06:11:04.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 shutdown_connections 2026-02-17T06:11:04.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 >> 10.20.193.78:0/1900923725 conn(0x7fa4940a6620 msgr2=0x7fa494135460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:04.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 shutdown_connections 2026-02-17T06:11:04.905 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.894+0000 7fa4990bf640 1 -- 10.20.193.78:0/1900923725 wait complete. 2026-02-17T06:11:04.905 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:11:04.905 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-17T06:11:04.905 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.973+0000 7ff66beb9640 10 monclient: get_monmap_and_config 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.973+0000 7ff66beb9640 10 monclient: build_initial_monmap 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.973+0000 7ff66beb9640 10 monclient: monmap: 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:04.973600+0000 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:04.973600+0000 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:04.973 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.973+0000 7ff66beb9640 1 Processor -- start 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.973+0000 7ff66beb9640 1 -- start start 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.973+0000 7ff66beb9640 10 monclient: init 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient: _add_conns ranks=[3,0,2,4,1] 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff66406c890 0x7ff66406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient(hunting): picked mon.noname-a con 0x7ff66406c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff66409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406d1b0 0x7ff664135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient(hunting): picked mon.noname-b con 0x7ff66406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66406daf0 con 0x7ff66409c4a0 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7ff6640a0880 con 0x7ff66406c890 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff66409cc80 con 0x7ff66406d1b0 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66beb9640 10 monclient(hunting): authenticate will time out at 2110.527123s 2026-02-17T06:11:04.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7ff66406c890 0x7ff66406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66942d640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7ff66409c4a0 0x7ff664134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46358/0 (socket says 10.20.193.78:46358) 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66942d640 1 -- 10.20.193.78:0/858392864 learned_addr learned my addr 10.20.193.78:0/858392864 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 1 --2- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406d1b0 0x7ff664135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient(hunting): get_auth_request con 0x7ff66406c890 auth_method 0 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 10 monclient(hunting): get_auth_request con 0x7ff66406d1b0 auth_method 0 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient(hunting): handle_auth_done global_id 284316 payload 274 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 1 -- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406d1b0 msgr2=0x7ff664135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 1 --2- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406d1b0 0x7ff664135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 1 -- 10.20.193.78:0/858392864 >> v1:10.20.193.78:6789/0 conn(0x7ff66409c4a0 legacy=0x7ff664134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient: _finish_hunting 0 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 1 monclient: found mon.noname-a 2026-02-17T06:11:04.976 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:04.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff669c2e640 1 -- 10.20.193.78:0/858392864 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66409cf60 con 0x7ff66406c890 2026-02-17T06:11:04.977 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.974+0000 7ff66a42f640 1 --2- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406d1b0 0x7ff664135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.976+0000 7ff669c2e640 1 -- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66406c890 msgr2=0x7ff66406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.976+0000 7ff669c2e640 1 --2- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66406c890 0x7ff66406cc70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff66406dcc0 tx=0x7ff65402ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755bc517e266d3c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 1 -- 10.20.193.78:0/858392864 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65403c070 con 0x7ff66406c890 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: dump: 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.979 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: _finish_auth 0 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: _check_auth_tickets 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 1 -- 10.20.193.78:0/858392864 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65402fc00 con 0x7ff66406c890 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 5 monclient: authenticate success, global_id 284316 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 1 -- 10.20.193.78:0/858392864 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654037440 con 0x7ff66406c890 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff668c2c640 10 monclient: dump: 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.980 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 4 set_mon_vals no callback set 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 10 monclient: shutdown 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 1 -- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66406c890 msgr2=0x7ff66406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 1 --2- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66406c890 0x7ff66406cc70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff66406dcc0 tx=0x7ff65402ed80 comp rx=0 tx=0).stop 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 1 -- 10.20.193.78:0/858392864 shutdown_connections 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 1 --2- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406d1b0 0x7ff664135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 1 --2- 10.20.193.78:0/858392864 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66406c890 0x7ff66406cc70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 1 -- 10.20.193.78:0/858392864 >> 10.20.193.78:0/858392864 conn(0x7ff6640a6620 msgr2=0x7ff6640a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 1 -- 10.20.193.78:0/858392864 shutdown_connections 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 1 -- 10.20.193.78:0/858392864 wait complete. 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 10 monclient: build_initial_monmap 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.977+0000 7ff66beb9640 10 monclient: monmap: 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:04.981 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 1 Processor -- start 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 1 -- start start 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient: init 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406c890 0x7ff664133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient(hunting): picked mon.noname-c con 0x7ff66406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff66406d1b0 0x7ff664134510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient(hunting): picked mon.noname-b con 0x7ff66406d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66409c4a0 0x7ff6641c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient(hunting): picked mon.noname-a con 0x7ff66409c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6640a01a0 con 0x7ff66409c4a0 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6640a0020 con 0x7ff66406d1b0 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7ff66409be20 con 0x7ff66406c890 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66beb9640 10 monclient(hunting): authenticate will time out at 2110.531420s 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406c890 0x7ff664133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66409c4a0 0x7ff6641c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406c890 0x7ff664133fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60546/0 (socket says 10.20.193.78:60546) 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66409c4a0 0x7ff6641c0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58286/0 (socket says 10.20.193.78:58286) 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 1 -- 10.20.193.78:0/656746402 learned_addr learned my addr 10.20.193.78:0/656746402 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 10 monclient(hunting): get_auth_request con 0x7ff66406c890 auth_method 0 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient(hunting): get_auth_request con 0x7ff66409c4a0 auth_method 0 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient(hunting): handle_auth_done global_id 284319 payload 824 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406c890 msgr2=0x7ff664133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406c890 0x7ff664133fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff66406d1b0 msgr2=0x7ff664134510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff66406d1b0 0x7ff664134510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient: _finish_hunting 0 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 1 monclient: found mon.noname-a 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff66a42f640 1 -- 10.20.193.78:0/656746402 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6640a2610 con 0x7ff66409c4a0 2026-02-17T06:11:04.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.978+0000 7ff669c2e640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7ff66406c890 0x7ff664133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:04.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66a42f640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66409c4a0 msgr2=0x7ff6641c0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66a42f640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66409c4a0 0x7ff6641c0d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff66000efc0 tx=0x7ff66000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660018020 con 0x7ff66409c4a0 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: dump: 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: _finish_auth 0 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: _check_auth_tickets 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff660014680 con 0x7ff66409c4a0 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660014a70 con 0x7ff66409c4a0 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff653fff640 4 set_mon_vals no callback set 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff6527fc640 10 monclient: dump: 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.985 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66beb9640 5 monclient: authenticate success, global_id 284319 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66beb9640 10 monclient: _renew_subs 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66beb9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff664121180 con 0x7ff66409c4a0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66beb9640 10 monclient: _renew_subs 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66beb9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.980+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff664121480 con 0x7ff66409c4a0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff66beb9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff66beb9640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1a10 0x7ff664071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff66beb9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff664121b90 con 0x7ff6641c1a10 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7ff660014c10 con 0x7ff66409c4a0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1a10 0x7ff664071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff6527fc640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff63403df00 0x7ff6340403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 10 monclient: get_auth_request con 0x7ff6641c1a10 auth_method 0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7ff660053530 con 0x7ff66409c4a0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff66942d640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff63403df00 0x7ff6340403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff66942d640 10 monclient: get_auth_request con 0x7ff63403df00 auth_method 0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 10 monclient(hunting): handle_auth_done global_id 304108 payload 824 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.981+0000 7ff669c2e640 1 -- 10.20.193.78:0/656746402 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff664074980 con 0x7ff6641c1a10 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff669c2e640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1a10 msgr2=0x7ff664071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff669c2e640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1a10 0x7ff664071420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff65402edc0 tx=0x7ff6540047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654048070 con 0x7ff6641c1a10 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 10 monclient: dump: 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.986 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff66942d640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff63403df00 0x7ff6340403c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff658007fd0 tx=0x7ff658007ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9607 (secure 0 0 0) 0x7ff654041640 con 0x7ff6641c1a10 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1a10 msgr2=0x7ff664071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1a10 0x7ff664071420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff65402edc0 tx=0x7ff6540047c0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1a10 msgr2=0x7ff664071420 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.984+0000 7ff6527fc640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1a10 0x7ff664071420 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff65402edc0 tx=0x7ff6540047c0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.987+0000 7ff66beb9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.987+0000 7ff66beb9640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1df0 0x7ff6641da770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.987+0000 7ff66beb9640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.987+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff664072ae0 con 0x7ff6641c1df0 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.987+0000 7ff669c2e640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1df0 0x7ff6641da770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 10 monclient: get_auth_request con 0x7ff6641c1df0 auth_method 0 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:04.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:04.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:04.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:04.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:04.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 10 monclient(hunting): handle_auth_done global_id 304111 payload 824 2026-02-17T06:11:04.988 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.988+0000 7ff669c2e640 1 -- 10.20.193.78:0/656746402 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff6640744d0 con 0x7ff6641c1df0 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff669c2e640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1df0 msgr2=0x7ff6641da770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff669c2e640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1df0 0x7ff6641da770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff6640a7630 tx=0x7ff654009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65403c040 con 0x7ff6641c1df0 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 10 monclient: dump: 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 <== mon.1 v2:10.20.193.115:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7ff65404ddb0 con 0x7ff6641c1df0 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:11:04.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1df0 msgr2=0x7ff6641da770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1df0 0x7ff6641da770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff6640a7630 tx=0x7ff654009290 comp rx=0 tx=0).stop 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1df0 msgr2=0x7ff6641da770 secure :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.990+0000 7ff6527fc640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7ff6641c1df0 0x7ff6641da770 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff6640a7630 tx=0x7ff654009290 comp rx=0 tx=0).stop 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "state": "peon", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "election_epoch": 276, 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "uptime": 15433, 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "quorum": [ 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: 0, 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: 1, 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: 2 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_age": 0, 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "required_con": "2450521708007866372", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "required_mon": [ 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_con": "4540701547738038271", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "quorum_mon": [ 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.991 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "outside_quorum": [], 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "extra_probe_peers": [], 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "sync_provider": [], 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "monmap": { 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "epoch": 1, 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "fsid": "5df7382e-3ebf-45af-b855-6719898a3119", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "modified": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "created": "2026-02-17T05:45:23.713551Z", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release": 19, 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "min_mon_release_name": "squid", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "election_strategy": 1, 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "disallowed_leaders": "", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false, 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "tiebreaker_mon": "", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "removed_ranks": "", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "features": { 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "persistent": [ 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "kraken", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "luminous", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "mimic", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "osdmap-prune", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "nautilus", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "octopus", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "pacific", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "elector-pinging", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "quincy", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "reef", 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "squid" 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "optional": [] 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: "mons": [ 2026-02-17T06:11:04.992 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 0, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "name": "a", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3300", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6789/0", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6789/0", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 1, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "name": "b", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:3300", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.115:6789/0", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.115:6789/0", 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.993 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "rank": 2, 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "name": "c", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "public_addrs": { 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "addrvec": [ 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v2", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:3301", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "type": "v1", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "nonce": 0 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "addr": "10.20.193.78:6790/0", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "public_addr": "10.20.193.78:6790/0", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "priority": 0, 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "weight": 0, 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "crush_location": "{}" 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "feature_map": { 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "mon": [ 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "osd": [ 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: ], 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "client": [ 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: { 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "features": "0x3f03cffffffdffff", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "release": "squid", 2026-02-17T06:11:04.994 INFO:teuthology.orchestra.run.trial078.stdout: "num": 1 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stdout: } 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stdout: ] 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stdout: }, 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stdout: "stretch_mode": false 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff63403df00 msgr2=0x7ff6340403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7ff63403df00 0x7ff6340403c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff658007fd0 tx=0x7ff658007ba0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 10 monclient: shutdown 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66409c4a0 msgr2=0x7ff6641c0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66409c4a0 0x7ff6641c0d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff66000efc0 tx=0x7ff66000c490 comp rx=0 tx=0).stop 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66a42f640 1 -- 10.20.193.78:0/656746402 reap_dead start 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 shutdown_connections 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 --2- 10.20.193.78:0/656746402 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7ff66409c4a0 0x7ff6641c0d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 >> 10.20.193.78:0/656746402 conn(0x7ff6640a6620 msgr2=0x7ff6640a1940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 shutdown_connections 2026-02-17T06:11:04.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:04.991+0000 7ff66beb9640 1 -- 10.20.193.78:0/656746402 wait complete. 2026-02-17T06:11:04.996 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-17T06:11:04.996 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-17T06:11:04.996 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.061+0000 7f8de7a96640 10 monclient: get_monmap_and_config 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.061+0000 7f8de7a96640 10 monclient: build_initial_monmap 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.061+0000 7f8de7a96640 10 monclient: monmap: 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:05.061893+0000 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:05.061893+0000 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:05.061 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:05.062 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:05.062 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:05.062 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 1 Processor -- start 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 1 -- start start 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient: init 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient(hunting): picked mon.noname-b con 0x7f8de006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f8de009d430 0x7f8de006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient(hunting): picked mon.noname-c con 0x7f8de009d430 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8de009bf40 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de0073600 con 0x7f8de009bf40 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f8de009ca50 con 0x7f8de009d430 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8de006d6f0 con 0x7f8de006c890 2026-02-17T06:11:05.063 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de7a96640 10 monclient(hunting): authenticate will time out at 2110.615434s 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60548/0 (socket says 10.20.193.78:60548) 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 1 -- 10.20.193.78:0/2992105499 learned_addr learned my addr 10.20.193.78:0/2992105499 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient(hunting): get_auth_request con 0x7f8de006c890 auth_method 0 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de500a640 1 --2- 10.20.193.78:0/2992105499 >> v2:10.20.193.115:3300/0 conn(0x7f8de009d430 0x7f8de006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de500a640 10 monclient(hunting): get_auth_request con 0x7f8de009d430 auth_method 0 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de500a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de500a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de500a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient(hunting): handle_auth_done global_id 264527 payload 274 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 1 -- 10.20.193.78:0/2992105499 >> v2:10.20.193.115:3300/0 conn(0x7f8de009d430 msgr2=0x7f8de006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 1 --2- 10.20.193.78:0/2992105499 >> v2:10.20.193.115:3300/0 conn(0x7f8de009d430 0x7f8de006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 1 -- 10.20.193.78:0/2992105499 >> v1:10.20.193.115:6789/0 conn(0x7f8de009bf40 legacy=0x7f8de0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient: _finish_hunting 0 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 1 monclient: found mon.noname-b 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:05.064 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.062+0000 7f8de580b640 1 -- 10.20.193.78:0/2992105499 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de006dab0 con 0x7f8de006c890 2026-02-17T06:11:05.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de580b640 1 -- 10.20.193.78:0/2992105499 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 msgr2=0x7f8de006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:05.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de580b640 1 --2- 10.20.193.78:0/2992105499 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de006cc70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8de00a7600 tx=0x7f8dd002ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b9dd34a1ec1947 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:05.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 1 -- 10.20.193.78:0/2992105499 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd003c070 con 0x7f8de006c890 2026-02-17T06:11:05.065 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: dump: 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: _finish_auth 0 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: _check_auth_tickets 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 1 -- 10.20.193.78:0/2992105499 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd002fcc0 con 0x7f8de006c890 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 1 -- 10.20.193.78:0/2992105499 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0037660 con 0x7f8de006c890 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 5 monclient: authenticate success, global_id 264527 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de4809640 10 monclient: dump: 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:05.066 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 4 set_mon_vals no callback set 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 10 monclient: shutdown 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 1 -- 10.20.193.78:0/2992105499 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 msgr2=0x7f8de006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 1 --2- 10.20.193.78:0/2992105499 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de006cc70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8de00a7600 tx=0x7f8dd002ee80 comp rx=0 tx=0).stop 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 1 -- 10.20.193.78:0/2992105499 shutdown_connections 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 1 --2- 10.20.193.78:0/2992105499 >> v2:10.20.193.115:3300/0 conn(0x7f8de009d430 0x7f8de006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 1 --2- 10.20.193.78:0/2992105499 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de006cc70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 1 -- 10.20.193.78:0/2992105499 >> 10.20.193.78:0/2992105499 conn(0x7f8de00a6620 msgr2=0x7f8de00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 1 -- 10.20.193.78:0/2992105499 shutdown_connections 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.065+0000 7f8de7a96640 1 -- 10.20.193.78:0/2992105499 wait complete. 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient: build_initial_monmap 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient: monmap: 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:05.067 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:05.068 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 1 Processor -- start 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 1 -- start start 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient: init 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de01cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient(hunting): picked mon.noname-c con 0x7f8de006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8de009bf40 0x7f8de01cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:05.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient(hunting): picked mon.noname-a con 0x7f8de009bf40 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8de009d430 0x7f8de0121540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient(hunting): picked mon.noname-b con 0x7f8de009d430 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8de00733c0 con 0x7f8de009bf40 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0073240 con 0x7f8de009d430 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f8de0135500 con 0x7f8de006c890 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de7a96640 10 monclient(hunting): authenticate will time out at 2110.619679s 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de01cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de01cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60552/0 (socket says 10.20.193.78:60552) 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 -- 10.20.193.78:0/3320357199 learned_addr learned my addr 10.20.193.78:0/3320357199 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient(hunting): get_auth_request con 0x7f8de006c890 auth_method 0 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de500a640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8de009bf40 0x7f8de01cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de500a640 10 monclient(hunting): get_auth_request con 0x7f8de009bf40 auth_method 0 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de500a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de500a640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de500a640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de500a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de500a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:05.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de500a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient(hunting): handle_auth_done global_id 264530 payload 824 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 -- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8de009d430 msgr2=0x7f8de0121540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8de009d430 0x7f8de0121540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 -- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8de009bf40 msgr2=0x7f8de01cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8de009bf40 0x7f8de01cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient: _finish_hunting 0 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 monclient: found mon.noname-c 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.066+0000 7f8de580b640 1 -- 10.20.193.78:0/3320357199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de01231a0 con 0x7f8de006c890 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.068+0000 7f8de580b640 1 -- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 msgr2=0x7f8de01cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.069+0000 7f8de580b640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de01cf560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009a70 tx=0x7f8dd0004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:05.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.069+0000 7f8dce7fc640 1 -- 10.20.193.78:0/3320357199 DispatchQueue::entry inject delay of 0.002000 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 1 -- 10.20.193.78:0/3320357199 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd003c040 con 0x7f8de006c890 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: dump: 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: _finish_auth 0 2026-02-17T06:11:05.072 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 1 -- 10.20.193.78:0/3320357199 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd0047070 con 0x7f8de006c890 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 1 -- 10.20.193.78:0/3320357199 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0042890 con 0x7f8de006c890 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 10 monclient: dump: 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dcffff640 4 set_mon_vals no callback set 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 5 monclient: authenticate success, global_id 264530 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 10 monclient: _renew_subs 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de0125490 con 0x7f8de006c890 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 10 monclient: _renew_subs 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de00a1a50 con 0x7f8de006c890 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:05.073 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de0074980 con 0x7f8de006c890 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 1 -- 10.20.193.78:0/3320357199 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f8dd0042a30 con 0x7f8de006c890 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8db003df00 0x7f8db00403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8dce7fc640 1 -- 10.20.193.78:0/3320357199 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f8dd0077a30 con 0x7f8de006c890 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.071+0000 7f8de500a640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8db003df00 0x7f8db00403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.072+0000 7f8de500a640 10 monclient: get_auth_request con 0x7f8db003df00 auth_method 0 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.074+0000 7f8de500a640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8db003df00 0x7f8db00403c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8de0122da0 tx=0x7f8dd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.074+0000 7f8dce7fc640 1 -- 10.20.193.78:0/3320357199 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8dd0035340 con 0x7f8de006c890 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.074+0000 7f8dce7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:05.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.074+0000 7f8dce7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.187+0000 7f8de7a96640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-17T06:11:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.187+0000 7f8de7a96640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-17T06:11:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.187+0000 7f8de7a96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:05.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.187+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f8de00744d0 con 0x7f8de006c890 2026-02-17T06:11:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.188+0000 7f8dce7fc640 1 -- 10.20.193.78:0/3320357199 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f8dd0042e00 con 0x7f8de006c890 2026-02-17T06:11:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.188+0000 7f8dce7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-17T06:11:05.188 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.188+0000 7f8dce7fc640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=1,health=11,kv=12,logm=49} crc {auth=2091129960,config=1228700967,health=378698282,kv=449989585,logm=529938895}) 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=26,config=2,health=10,kv=11,logm=51} crc {auth=4050860900,config=692619927,health=1939044355,kv=570842137,logm=3260187324}) 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=1,health=11,kv=12,logm=49} crc {auth=2091129960,config=1228700967,health=378698282,kv=449989585,logm=529938895}) 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=25,config=2,health=12,kv=12,logm=49} crc {auth=3010359124,config=692619927,health=577633902,kv=449989585,logm=2697338432}) 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3437192659}) 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=589221468}) 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3437192659}) 2026-02-17T06:11:05.189 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.188+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3233453867}) 2026-02-17T06:11:05.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8db003df00 msgr2=0x7f8db00403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:05.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8db003df00 0x7f8db00403c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8de0122da0 tx=0x7f8dd4008040 comp rx=0 tx=0).stop 2026-02-17T06:11:05.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 10 monclient: shutdown 2026-02-17T06:11:05.190 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 msgr2=0x7f8de01cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de01cf560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009a70 tx=0x7f8dd0004830 comp rx=0 tx=0).stop 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 shutdown_connections 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f8db003df00 0x7f8db00403c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f8de009d430 0x7f8de0121540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f8de009bf40 0x7f8de01cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 --2- 10.20.193.78:0/3320357199 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f8de006c890 0x7f8de01cf560 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 >> 10.20.193.78:0/3320357199 conn(0x7f8de00a6620 msgr2=0x7f8de01367b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 shutdown_connections 2026-02-17T06:11:05.191 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:05.189+0000 7f8de7a96640 1 -- 10.20.193.78:0/3320357199 wait complete. 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.189+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.189+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2830448131}) 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.189+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=88841750}) 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.189+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.189+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2830448131}) 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.189+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=407949026}) 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1043747464}) 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=32505441}) 2026-02-17T06:11:05.196 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1043747464}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3270251587}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3310005718}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1600899018}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3310005718}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.190+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3415606874}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2024430610}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3998542210}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2024430610}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4279006594}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813420997}) 2026-02-17T06:11:05.197 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131999620}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813420997}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=360786353}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.191+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2788315960}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3868896579}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2788315960}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2044013178}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3152678119}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2554551524}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3152678119}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=57939250}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=778768850}) 2026-02-17T06:11:05.198 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3806741536}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=778768850}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=269966394}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463281490}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3996535525}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463281490}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.192+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1591661490}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4126669603}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=383297034}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4126669603}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=367304095}) 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.199 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=942382087}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3711972473}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=942382087}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1195272492}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3972790728}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39875512}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3972790728}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.193+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2645840053}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698441378}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2544786213}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698441378}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2126626044}) 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.200 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3907467565}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2303905670}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3907467565}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2585195019}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214512682}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1217224531}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214512682}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.194+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1615589794}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=426639110}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174519300}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=426639110}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2164150923}) 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.201 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=53783005}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3818108834}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=53783005}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1329740163}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=377373117}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2394453239}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=377373117}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.195+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2880953137}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1733102714}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1701800542}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1733102714}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=450939993}) 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.202 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1702841176}) 2026-02-17T06:11:05.203 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=181677184}) 2026-02-17T06:11:05.203 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.203 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1702841176}) 2026-02-17T06:11:05.203 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4245336825}) 2026-02-17T06:11:05.204 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=618334926}) 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2479721031}) 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=618334926}) 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.196+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=376063127}) 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683862148}) 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1089888036}) 2026-02-17T06:11:05.209 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683862148}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1926770737}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2225837547}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1795512827}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2225837547}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4193338352}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=484265495}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1851569452}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=484265495}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.197+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=694170822}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3964464228}) 2026-02-17T06:11:05.210 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=349571559}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3964464228}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3812641331}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1538629437}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3584755944}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1538629437}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1008409247}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2712201421}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4246190582}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2712201421}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.198+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2451351733}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2204435011}) 2026-02-17T06:11:05.211 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1687017316}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2204435011}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=799798047}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1605116226}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1227621864}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1605116226}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2776590452}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=441773519}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1655444117}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=441773519}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.199+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1898108609}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4193902189}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3874605972}) 2026-02-17T06:11:05.212 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4193902189}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2127032972}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093138936}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1145928037}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093138936}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.200+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2189137915}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=115601148}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=11368900}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=115601148}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4245935896}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111082131}) 2026-02-17T06:11:05.213 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1042086570}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111082131}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=657119907}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4114262027}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1168410692}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4114262027}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.201+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1404941192}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.202+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.202+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2428281396}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.202+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3709358662}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.202+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.202+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2428281396}) 2026-02-17T06:11:05.214 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.202+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=860999783}) 2026-02-17T06:11:05.215 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.215+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.215+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1166628183}) 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.215+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2269665416}) 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.215+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.215+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1166628183}) 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.215+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908086807}) 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3695754132}) 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=114198786}) 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3695754132}) 2026-02-17T06:11:05.216 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1630204403}) 2026-02-17T06:11:05.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007471935}) 2026-02-17T06:11:05.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3510834776}) 2026-02-17T06:11:05.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.225 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007471935}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=481495868}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3041200401}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1487437692}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3041200401}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.216+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=883649432}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693269906}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457212666}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693269906}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1879022525}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236398739}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1461863327}) 2026-02-17T06:11:05.226 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236398739}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=757136958}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=426208510}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4033957406}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=426208510}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.217+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2567612798}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1369098250}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1642937326}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1369098250}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2421701220}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1999015717}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2010721282}) 2026-02-17T06:11:05.227 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1999015717}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.218+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4213653955}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776096991}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=353777607}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776096991}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=479765714}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1804904130}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1973372826}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1804904130}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1408587431}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=450057594}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933488279}) 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.228 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=450057594}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.219+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756475015}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502395299}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1631405153}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502395299}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3133014062}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252872447}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=813124209}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252872447}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3310799362}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1700495273}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017075991}) 2026-02-17T06:11:05.229 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1700495273}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.220+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1688621205}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2426575269}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=769212825}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2426575269}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3743629557}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1771930173}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3216253889}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1771930173}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=211463905}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326166740}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002668765}) 2026-02-17T06:11:05.230 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326166740}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.221+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2316531381}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149503641}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1827027500}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149503641}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430467628}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2346392053}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=386484616}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2346392053}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2770552171}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4151853148}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4178665031}) 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.231 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4151853148}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.222+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3999408992}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=30943244}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3834289753}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=30943244}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3266881541}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493937130}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1874885398}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493937130}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.223+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=545711101}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1473909174}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4094354798}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1473909174}) 2026-02-17T06:11:05.232 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1358400560}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2356311078}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=289271566}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2356311078}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1407495443}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1048322189}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3213446705}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1048322189}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.224+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=427438534}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2276324262}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2975770662}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2276324262}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=505032577}) 2026-02-17T06:11:05.233 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955761267}) 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273493769}) 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955761267}) 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1456559680}) 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934252183}) 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2446585994}) 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934252183}) 2026-02-17T06:11:05.234 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.225+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1894414646}) 2026-02-17T06:11:05.239 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3639489910}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3098261037}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3639489910}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3518418554}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514539662}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4225492191}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514539662}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078374866}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2723175021}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1965639302}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2723175021}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.226+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1404865128}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1665731794}) 2026-02-17T06:11:05.240 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1484574127}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1665731794}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=986152307}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062769649}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3724056946}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062769649}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2153345734}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1739068851}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=522821985}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1739068851}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.227+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1951135120}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2191919457}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2537341888}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2191919457}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1912251951}) 2026-02-17T06:11:05.241 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3386584259}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=673800051}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3386584259}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1788318165}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.228+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655104571}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956746175}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655104571}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3060934648}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1980646763}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2497394612}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1980646763}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2528481256}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2119327027}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=149395405}) 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.242 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2119327027}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.229+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2583708413}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3091837936}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220461708}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3091837936}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4267342770}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=313395201}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3476175235}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=313395201}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3058576886}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2673011191}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2240441262}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2673011191}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.230+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1650396733}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2347414192}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663478785}) 2026-02-17T06:11:05.243 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2347414192}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200562495}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187623231}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1598659350}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187623231}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3405668849}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2083366551}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2721102335}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2083366551}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.231+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750368941}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2287303109}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3894054226}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2287303109}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=568486156}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677476191}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1474001167}) 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.244 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677476191}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.232+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4202596973}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1760065976}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3726699535}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1760065976}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3884362874}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=442878323}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1300706522}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=442878323}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1906046223}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3094064600}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=861740006}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3094064600}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.233+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3273803156}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441571501}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=99730630}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441571501}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1979413922}) 2026-02-17T06:11:05.245 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2402470066}) 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=293316184}) 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2402470066}) 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.234+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4000171420}) 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.235+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.235+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2549728183}) 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.235+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2087798382}) 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.235+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.235+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2549728183}) 2026-02-17T06:11:05.246 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.235+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3441336283}) 2026-02-17T06:11:05.513 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.512+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.513 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.512+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2751631547}) 2026-02-17T06:11:05.513 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.512+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2359182326}) 2026-02-17T06:11:05.513 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.512+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.514 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.512+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2751631547}) 2026-02-17T06:11:05.514 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.512+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3671023207}) 2026-02-17T06:11:05.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1538312508}) 2026-02-17T06:11:05.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2225842507}) 2026-02-17T06:11:05.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1538312508}) 2026-02-17T06:11:05.573 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.573+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3705549703}) 2026-02-17T06:11:05.581 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.581 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891358293}) 2026-02-17T06:11:05.581 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174210878}) 2026-02-17T06:11:05.581 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.581 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891358293}) 2026-02-17T06:11:05.581 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.581+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2493958893}) 2026-02-17T06:11:05.967 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.967 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=826396541}) 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1681198368}) 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=826396541}) 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4140750827}) 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=456733099}) 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2881084939}) 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=456733099}) 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3493416707}) 2026-02-17T06:11:05.968 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=819810141}) 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1913211856}) 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=819810141}) 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.959+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1847947535}) 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=926312192}) 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2241877309}) 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=926312192}) 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=944108988}) 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.969 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=801526943}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3478368702}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=801526943}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2937966734}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1508114836}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834496224}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1508114836}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.960+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=77192496}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732406578}) 2026-02-17T06:11:05.970 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3000112410}) 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732406578}) 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2400719450}) 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330774410}) 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615319995}) 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330774410}) 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4160932179}) 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=708403626}) 2026-02-17T06:11:05.971 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3617978188}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=708403626}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.961+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1847137438}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2496589628}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=248201147}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2496589628}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1708868190}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=55377832}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3688691038}) 2026-02-17T06:11:05.972 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.973 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=55377832}) 2026-02-17T06:11:05.973 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2453790602}) 2026-02-17T06:11:05.973 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.973 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3521701307}) 2026-02-17T06:11:05.973 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1355275918}) 2026-02-17T06:11:05.973 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.973 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3521701307}) 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.962+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=309257988}) 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876798581}) 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1795876527}) 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876798581}) 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=837304898}) 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1189802347}) 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4166119088}) 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.974 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1189802347}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4075348766}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363420010}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4185700176}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363420010}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1367165303}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3032499797}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1617921597}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3032499797}) 2026-02-17T06:11:05.975 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3275673116}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.963+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2693525364}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1005723694}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2693525364}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1068831736}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2167645938}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=91676880}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2167645938}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2316303432}) 2026-02-17T06:11:05.976 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=62,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=1} crc {logm=1136701503,mdsmap=1160443688,mgr=2723955629,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=354724421,mon_config_key=1765318643,monmap=1784110042,osd_metadata=795255734}) 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=62,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=1} crc {logm=280040501,mdsmap=1160443688,mgr=267573432,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=354724421,mon_config_key=3847793152,monmap=1784110042,osd_metadata=795255734}) 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=62,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=12,monmap=3,osd_metadata=1} crc {logm=1136701503,mdsmap=1160443688,mgr=2723955629,mgr_command_descs=1238205435,mgr_metadata=1258482571,mgrstat=354724421,mon_config_key=1765318643,monmap=1784110042,osd_metadata=795255734}) 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.964+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=63,mdsmap=3,mgr=7,mgr_command_descs=1,mgrstat=10,mon_config_key=13,monmap=3} crc {logm=1612847658,mdsmap=1160443688,mgr=1039804757,mgr_command_descs=1238205435,mgrstat=354724421,mon_config_key=1912222953,monmap=1784110042}) 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=94} crc {osd_metadata=1501630441,osd_pg_creating=2054283389,osd_snap=2924313975}) 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=5,osd_snap=95} crc {osd_metadata=1501630441,osd_snap=1469638788}) 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=94} crc {osd_metadata=1501630441,osd_pg_creating=2054283389,osd_snap=2924313975}) 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=94} crc {osd_metadata=1619905305,osd_pg_creating=2054283389,osd_snap=385582993}) 2026-02-17T06:11:05.977 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1403112313,osdmap=2268710430}) 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=51,osdmap=49} crc {osd_snap=2591583967,osdmap=932216909}) 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=50,osdmap=50} crc {osd_snap=1403112313,osdmap=2268710430}) 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=53,osdmap=47} crc {osd_snap=2678731307,osdmap=2883769886}) 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2712919260}) 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1592851137}) 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2712919260}) 2026-02-17T06:11:05.978 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.965+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3638106667}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=904239976}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2658335328}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=904239976}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1665036870}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=973446181}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3615710719}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=973446181}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.966+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2279420365}) 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.967+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.979 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.967+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3209134151}) 2026-02-17T06:11:05.980 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.967+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3113127567}) 2026-02-17T06:11:05.980 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.967+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:05.980 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.967+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3209134151}) 2026-02-17T06:11:05.980 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:05.967+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4036280876}) 2026-02-17T06:11:06.204 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-17T06:11:06.205 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-17T06:11:06.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient: get_monmap_and_config 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient: build_initial_monmap 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient: monmap: 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:06.274383+0000 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:06.274383+0000 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:06.274 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 1 Processor -- start 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 1 -- start start 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient: init 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdecc06c890 0x7fdecc06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient(hunting): picked mon.noname-a con 0x7fdecc06c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdecc09c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fdecc06d1b0 0x7fdecc135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient(hunting): picked mon.noname-c con 0x7fdecc06d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdecc06daf0 con 0x7fdecc09c4a0 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fdecc0a0880 con 0x7fdecc06c890 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fdecc09cc80 con 0x7fdecc06d1b0 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.274+0000 7fded2e7c640 10 monclient(hunting): authenticate will time out at 2111.827912s 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdecc06c890 0x7fdecc06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fdec3fff640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fdecc09c4a0 0x7fdecc134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46374/0 (socket says 10.20.193.78:46374) 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fdec3fff640 1 -- 10.20.193.78:0/2598368463 learned_addr learned my addr 10.20.193.78:0/2598368463 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient(hunting): get_auth_request con 0x7fdecc06c890 auth_method 0 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.275 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded13f2640 1 --2- 10.20.193.78:0/2598368463 >> v2:10.20.193.115:3300/0 conn(0x7fdecc06d1b0 0x7fdecc135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded13f2640 10 monclient(hunting): get_auth_request con 0x7fdecc06d1b0 auth_method 0 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded13f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded13f2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded13f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded13f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded13f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded13f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient(hunting): handle_auth_done global_id 284325 payload 274 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 1 -- 10.20.193.78:0/2598368463 >> v2:10.20.193.115:3300/0 conn(0x7fdecc06d1b0 msgr2=0x7fdecc135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 1 --2- 10.20.193.78:0/2598368463 >> v2:10.20.193.115:3300/0 conn(0x7fdecc06d1b0 0x7fdecc135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 1 -- 10.20.193.78:0/2598368463 >> v1:10.20.193.78:6789/0 conn(0x7fdecc09c4a0 legacy=0x7fdecc134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient: _finish_hunting 0 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 1 monclient: found mon.noname-a 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.276 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.275+0000 7fded0bf1640 1 -- 10.20.193.78:0/2598368463 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc09cf60 con 0x7fdecc06c890 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fded0bf1640 1 -- 10.20.193.78:0/2598368463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06c890 msgr2=0x7fdecc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fded0bf1640 1 --2- 10.20.193.78:0/2598368463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06c890 0x7fdecc06cc70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdecc074de0 tx=0x7fdebc02ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5d7013e15f10cb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 1 -- 10.20.193.78:0/2598368463 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc03c070 con 0x7fdecc06c890 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: dump: 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.277 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: _finish_auth 0 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: _check_auth_tickets 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 1 -- 10.20.193.78:0/2598368463 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdebc02fbd0 con 0x7fdecc06c890 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 1 -- 10.20.193.78:0/2598368463 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc037440 con 0x7fdecc06c890 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fdec37fe640 10 monclient: dump: 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fded2e7c640 5 monclient: authenticate success, global_id 284325 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fded2e7c640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:06.278 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.277+0000 7fded2e7c640 4 set_mon_vals no callback set 2026-02-17T06:11:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 10 monclient: shutdown 2026-02-17T06:11:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 1 -- 10.20.193.78:0/2598368463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06c890 msgr2=0x7fdecc06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.279 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 1 --2- 10.20.193.78:0/2598368463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06c890 0x7fdecc06cc70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdecc074de0 tx=0x7fdebc02ed50 comp rx=0 tx=0).stop 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 1 -- 10.20.193.78:0/2598368463 shutdown_connections 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 1 --2- 10.20.193.78:0/2598368463 >> v2:10.20.193.115:3300/0 conn(0x7fdecc06d1b0 0x7fdecc135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 1 --2- 10.20.193.78:0/2598368463 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06c890 0x7fdecc06cc70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 1 -- 10.20.193.78:0/2598368463 >> 10.20.193.78:0/2598368463 conn(0x7fdecc0a6620 msgr2=0x7fdecc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 1 -- 10.20.193.78:0/2598368463 shutdown_connections 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 1 -- 10.20.193.78:0/2598368463 wait complete. 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 10 monclient: build_initial_monmap 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.278+0000 7fded2e7c640 10 monclient: monmap: 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 1 Processor -- start 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 1 -- start start 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient: init 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdecc06c890 0x7fdecc1354d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient(hunting): picked mon.noname-c con 0x7fdecc06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06d1b0 0x7fdecc135a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient(hunting): picked mon.noname-a con 0x7fdecc06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdecc09c4a0 0x7fdecc124f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient(hunting): picked mon.noname-b con 0x7fdecc09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc0a01a0 con 0x7fdecc06d1b0 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc0a0020 con 0x7fdecc09c4a0 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc09be20 con 0x7fdecc06c890 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded2e7c640 10 monclient(hunting): authenticate will time out at 2111.832438s 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06d1b0 0x7fdecc135a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.280 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06d1b0 0x7fdecc135a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58296/0 (socket says 10.20.193.78:58296) 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 -- 10.20.193.78:0/285392721 learned_addr learned my addr 10.20.193.78:0/285392721 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient(hunting): get_auth_request con 0x7fdecc06d1b0 auth_method 0 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded0bf1640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdecc06c890 0x7fdecc1354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded0bf1640 10 monclient(hunting): get_auth_request con 0x7fdecc06c890 auth_method 0 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded0bf1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded0bf1640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded0bf1640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded0bf1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded0bf1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded0bf1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded13f2640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdecc09c4a0 0x7fdecc124f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded13f2640 10 monclient(hunting): get_auth_request con 0x7fdecc09c4a0 auth_method 0 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded13f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded13f2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fded13f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient(hunting): handle_auth_done global_id 284328 payload 824 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 -- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdecc06c890 msgr2=0x7fdecc1354d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdecc06c890 0x7fdecc1354d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 -- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdecc09c4a0 msgr2=0x7fdecc124f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdecc09c4a0 0x7fdecc124f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient: _finish_hunting 0 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 monclient: found mon.noname-a 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.281 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.279+0000 7fdec3fff640 1 -- 10.20.193.78:0/285392721 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc128c10 con 0x7fdecc06d1b0 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec3fff640 1 -- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06d1b0 msgr2=0x7fdecc135a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec3fff640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06d1b0 0x7fdecc135a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400c780 tx=0x7fdeb400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 1 -- 10.20.193.78:0/285392721 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4005de0 con 0x7fdecc06d1b0 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: dump: 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: _finish_auth 0 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: _check_auth_tickets 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 1 -- 10.20.193.78:0/285392721 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4007070 con 0x7fdecc06d1b0 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:06.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 1 -- 10.20.193.78:0/285392721 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4007440 con 0x7fdecc06d1b0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec2ffd640 4 set_mon_vals no callback set 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fdec17fa640 10 monclient: dump: 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fded2e7c640 5 monclient: authenticate success, global_id 284328 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fded2e7c640 10 monclient: _renew_subs 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fded2e7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.281+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc1270a0 con 0x7fdecc06d1b0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fded2e7c640 10 monclient: _renew_subs 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fded2e7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc136ea0 con 0x7fdecc06d1b0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fded2e7c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fded2e7c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fded2e7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdecc074980 con 0x7fdecc06d1b0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fdec17fa640 1 -- 10.20.193.78:0/285392721 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fdeb4007620 con 0x7fdecc06d1b0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fdec17fa640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde9c03df00 0x7fde9c0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.282+0000 7fdec17fa640 1 -- 10.20.193.78:0/285392721 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fdeb40523c0 con 0x7fdecc06d1b0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.283+0000 7fded0bf1640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde9c03df00 0x7fde9c0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.283+0000 7fded0bf1640 10 monclient: get_auth_request con 0x7fde9c03df00 auth_method 0 2026-02-17T06:11:06.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.285+0000 7fded0bf1640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde9c03df00 0x7fde9c0403c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdecc09c980 tx=0x7fdebc033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.285+0000 7fdec17fa640 1 -- 10.20.193.78:0/285392721 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdeb400aac0 con 0x7fdecc06d1b0 2026-02-17T06:11:06.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.285+0000 7fdec17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.285+0000 7fdec17fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:06.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.399+0000 7fded2e7c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-17T06:11:06.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.399+0000 7fded2e7c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-17T06:11:06.399 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.399+0000 7fded2e7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.399+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdecc073000 con 0x7fdecc06d1b0 2026-02-17T06:11:06.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fdec17fa640 1 -- 10.20.193.78:0/285392721 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fdeb40520c0 con 0x7fdecc06d1b0 2026-02-17T06:11:06.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fdec17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-17T06:11:06.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fdec17fa640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:11:06.400 INFO:teuthology.orchestra.run.trial078.stdout:{"election_epoch":276,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","modified":"2026-02-17T05:45:23.713551Z","created":"2026-02-17T05:45:23.713551Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3300","nonce":0},{"type":"v1","addr":"10.20.193.78:6789","nonce":0}]},"addr":"10.20.193.78:6789/0","public_addr":"10.20.193.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:3300","nonce":0},{"type":"v1","addr":"10.20.193.115:6789","nonce":0}]},"addr":"10.20.193.115:6789/0","public_addr":"10.20.193.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:3301","nonce":0},{"type":"v1","addr":"10.20.193.78:6790","nonce":0}]},"addr":"10.20.193.78:6790/0","public_addr":"10.20.193.78:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2026-02-17T06:11:06.400 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde9c03df00 msgr2=0x7fde9c0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde9c03df00 0x7fde9c0403c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdecc09c980 tx=0x7fdebc033000 comp rx=0 tx=0).stop 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 10 monclient: shutdown 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06d1b0 msgr2=0x7fdecc135a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06d1b0 0x7fdecc135a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400c780 tx=0x7fdeb400cc50 comp rx=0 tx=0).stop 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 shutdown_connections 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fde9c03df00 0x7fde9c0403c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdecc09c4a0 0x7fdecc124f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdecc06d1b0 0x7fdecc135a10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 --2- 10.20.193.78:0/285392721 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdecc06c890 0x7fdecc1354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 >> 10.20.193.78:0/285392721 conn(0x7fdecc0a6620 msgr2=0x7fdecc0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.400+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 shutdown_connections 2026-02-17T06:11:06.401 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.401+0000 7fded2e7c640 1 -- 10.20.193.78:0/285392721 wait complete. 2026-02-17T06:11:06.407 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-17T06:11:06.407 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-17T06:11:06.420 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-17T06:11:06.420 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:11:06.422 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:06.422+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:06.422 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:06.422+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3352696162}) 2026-02-17T06:11:06.422 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:06.422+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=54} crc {osdmap=2610078022}) 2026-02-17T06:11:06.422 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:06.422+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-17T06:11:06.423 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:06.422+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3352696162}) 2026-02-17T06:11:06.423 INFO:tasks.ceph.mon.a.trial078.stderr:2026-02-17T06:11:06.422+0000 7fc1a5b85640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=52} crc {osdmap=3126042249}) 2026-02-17T06:11:06.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.486+0000 7f28c8bf7640 10 monclient: get_monmap_and_config 2026-02-17T06:11:06.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.486+0000 7f28c8bf7640 10 monclient: build_initial_monmap 2026-02-17T06:11:06.486 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.486+0000 7f28c8bf7640 10 monclient: monmap: 2026-02-17T06:11:06.486 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:06.486 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.486 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:06.487009+0000 2026-02-17T06:11:06.486 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:06.487009+0000 2026-02-17T06:11:06.486 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:06.487 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.487 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:06.487 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:06.487 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:06.487 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:06.487 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:06.487 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 1 Processor -- start 2026-02-17T06:11:06.488 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 1 -- start start 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient: init 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f28c406c890 0x7f28c40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient(hunting): picked mon.noname-c con 0x7f28c406c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f28c406cc70 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f28c413a240 0x7f28c413c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient(hunting): picked mon.noname-a con 0x7f28c413a240 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c4074150 con 0x7f28c406cc70 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f28c409ef70 con 0x7f28c413a240 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f28c409f250 con 0x7f28c406c890 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c8bf7640 10 monclient(hunting): authenticate will time out at 2112.040874s 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c2d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f28c413a240 0x7f28c413c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c2d76640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f28c413a240 0x7f28c413c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58304/0 (socket says 10.20.193.78:58304) 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c2d76640 1 -- 10.20.193.78:0/265261152 learned_addr learned my addr 10.20.193.78:0/265261152 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:06.489 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c2d76640 10 monclient(hunting): get_auth_request con 0x7f28c413a240 auth_method 0 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c2d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.487+0000 7f28c2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2575640 1 --2- 10.20.193.78:0/265261152 >> v2:10.20.193.115:3300/0 conn(0x7f28c406c890 0x7f28c40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2575640 10 monclient(hunting): get_auth_request con 0x7f28c406c890 auth_method 0 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 10 monclient(hunting): handle_auth_done global_id 284331 payload 274 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 1 -- 10.20.193.78:0/265261152 >> v2:10.20.193.115:3300/0 conn(0x7f28c406c890 msgr2=0x7f28c40a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 1 --2- 10.20.193.78:0/265261152 >> v2:10.20.193.115:3300/0 conn(0x7f28c406c890 0x7f28c40a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 1 -- 10.20.193.78:0/265261152 >> v1:10.20.193.115:6789/0 conn(0x7f28c406cc70 legacy=0x7f28c406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 10 monclient: _finish_hunting 0 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 1 monclient: found mon.noname-a 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.490 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.488+0000 7f28c2d76640 1 -- 10.20.193.78:0/265261152 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c409df40 con 0x7f28c413a240 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c2d76640 1 -- 10.20.193.78:0/265261152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c413a240 msgr2=0x7f28c413c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c2d76640 1 --2- 10.20.193.78:0/265261152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c413a240 0x7f28c413c630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28c406a480 tx=0x7f28b402f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c840fd2ea45b58d7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 1 -- 10.20.193.78:0/265261152 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b40373f0 con 0x7f28c413a240 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: dump: 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.493 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: _finish_auth 0 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: _check_auth_tickets 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 1 -- 10.20.193.78:0/265261152 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b4037590 con 0x7f28c413a240 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c8bf7640 5 monclient: authenticate success, global_id 284331 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c8bf7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 1 -- 10.20.193.78:0/265261152 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4037960 con 0x7f28c413a240 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c1573640 10 monclient: dump: 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.494 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c8bf7640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.490+0000 7f28c8bf7640 4 set_mon_vals no callback set 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 10 monclient: shutdown 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 -- 10.20.193.78:0/265261152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c413a240 msgr2=0x7f28c413c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/265261152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c413a240 0x7f28c413c630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28c406a480 tx=0x7f28b402f270 comp rx=0 tx=0).stop 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 -- 10.20.193.78:0/265261152 shutdown_connections 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/265261152 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c413a240 0x7f28c413c630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/265261152 >> v2:10.20.193.115:3300/0 conn(0x7f28c406c890 0x7f28c40a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 -- 10.20.193.78:0/265261152 >> 10.20.193.78:0/265261152 conn(0x7f28c40a6620 msgr2=0x7f28c40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 -- 10.20.193.78:0/265261152 shutdown_connections 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 -- 10.20.193.78:0/265261152 wait complete. 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 10 monclient: build_initial_monmap 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 10 monclient: monmap: 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:06.495 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 Processor -- start 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 1 -- start start 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.491+0000 7f28c8bf7640 10 monclient: init 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28c406c890 0x7f28c4130100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient(hunting): picked mon.noname-c con 0x7f28c406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c406cc70 0x7f28c4130640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient(hunting): picked mon.noname-a con 0x7f28c406cc70 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28c413a240 0x7f28c413a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient(hunting): picked mon.noname-b con 0x7f28c413a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.496 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28c413ccf0 con 0x7f28c406cc70 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f28c413cb70 con 0x7f28c413a240 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f28c413ce70 con 0x7f28c406c890 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c8bf7640 10 monclient(hunting): authenticate will time out at 2112.045066s 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c406cc70 0x7f28c4130640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28c406c890 0x7f28c4130100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c406cc70 0x7f28c4130640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:58314/0 (socket says 10.20.193.78:58314) 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 -- 10.20.193.78:0/3531131355 learned_addr learned my addr 10.20.193.78:0/3531131355 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient(hunting): get_auth_request con 0x7f28c406cc70 auth_method 0 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28c406c890 0x7f28c4130100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60586/0 (socket says 10.20.193.78:60586) 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 10 monclient(hunting): get_auth_request con 0x7f28c406c890 auth_method 0 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.497 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient(hunting): handle_auth_done global_id 284334 payload 824 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 -- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28c406c890 msgr2=0x7f28c4130100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28c406c890 0x7f28c4130100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 -- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28c413a240 msgr2=0x7f28c413a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28c413a240 0x7f28c413a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient: _finish_hunting 0 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 monclient: found mon.noname-a 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.498 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.492+0000 7f28c1d74640 1 -- 10.20.193.78:0/3531131355 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c4139b00 con 0x7f28c406cc70 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.494+0000 7f28c1d74640 1 -- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c406cc70 msgr2=0x7f28c4130640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.494+0000 7f28c1d74640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c406cc70 0x7f28c4130640 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b570 tx=0x7f28ac00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.494+0000 7f28aaffd640 1 -- 10.20.193.78:0/3531131355 DispatchQueue::entry inject delay of 0.002000 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 1 -- 10.20.193.78:0/3531131355 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac002780 con 0x7f28c406cc70 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: dump: 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.500 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: _finish_auth 0 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: _check_auth_tickets 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 1 -- 10.20.193.78:0/3531131355 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ac015070 con 0x7f28c406cc70 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 1 -- 10.20.193.78:0/3531131355 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac002b80 con 0x7f28c406cc70 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c0d72640 4 set_mon_vals no callback set 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28aaffd640 10 monclient: dump: 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.501 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c8bf7640 5 monclient: authenticate success, global_id 284334 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c8bf7640 10 monclient: _renew_subs 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c8bf7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c40a3d40 con 0x7f28c406cc70 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c8bf7640 10 monclient: _renew_subs 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c8bf7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c4073b60 con 0x7f28c406cc70 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.496+0000 7f28c8bf7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.497+0000 7f28c8bf7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.497+0000 7f28c8bf7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.497+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c4074980 con 0x7f28c406cc70 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.497+0000 7f28aaffd640 1 -- 10.20.193.78:0/3531131355 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f28ac010480 con 0x7f28c406cc70 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.497+0000 7f28aaffd640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f289003df00 0x7f28900403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.497+0000 7f28aaffd640 1 -- 10.20.193.78:0/3531131355 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f28ac0524e0 con 0x7f28c406cc70 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.497+0000 7f28c2575640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f289003df00 0x7f28900403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.497+0000 7f28c2575640 10 monclient: get_auth_request con 0x7f289003df00 auth_method 0 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.499+0000 7f28aaffd640 1 -- 10.20.193.78:0/3531131355 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f28ac019ce0 con 0x7f28c406cc70 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.499+0000 7f28aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.499+0000 7f28aaffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:06.502 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.500+0000 7f28c2575640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f289003df00 0x7f28900403c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009a50 tx=0x7f28b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.582 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.582+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28c411d2b0 con 0x7f289003df00 2026-02-17T06:11:06.583 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.583+0000 7f28aaffd640 1 -- 10.20.193.78:0/3531131355 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34010 (secure 0 0 0) 0x7f28c411d2b0 con 0x7f289003df00 2026-02-17T06:11:06.583 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T06:11:06.583 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T06:11:06.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f289003df00 msgr2=0x7f28900403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f289003df00 0x7f28900403c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009a50 tx=0x7f28b8008040 comp rx=0 tx=0).stop 2026-02-17T06:11:06.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 10 monclient: shutdown 2026-02-17T06:11:06.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:06.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c406cc70 msgr2=0x7f28c4130640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c406cc70 0x7f28c4130640 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b570 tx=0x7f28ac00ba40 comp rx=0 tx=0).stop 2026-02-17T06:11:06.584 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 shutdown_connections 2026-02-17T06:11:06.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f289003df00 0x7f28900403c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f28c413a240 0x7f28c413a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f28c406cc70 0x7f28c4130640 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 --2- 10.20.193.78:0/3531131355 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f28c406c890 0x7f28c4130100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 >> 10.20.193.78:0/3531131355 conn(0x7f28c40a6620 msgr2=0x7f28c409be20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:06.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 shutdown_connections 2026-02-17T06:11:06.585 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.584+0000 7f28c8bf7640 1 -- 10.20.193.78:0/3531131355 wait complete. 2026-02-17T06:11:06.592 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":1608,"stamp":"2026-02-17T06:11:06.065408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":296304,"kb_used_data":24896,"kb_used_omap":9,"kb_used_meta":271286,"kb_avail":565934736,"statfs":{"total":579820584960,"available":579517169664,"internally_reserved":0,"allocated":25493504,"data_stored":12055756,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":277797542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001034"},"pg_stats":[{"pgid":"1.0","version":"16'112","reported_seq":1819,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773900+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T06:10:40.773900+0000","last_peered":"2026-02-17T06:10:40.773900+0000","last_clean":"2026-02-17T06:10:40.773900+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T06:10:40.773900+0000","last_undegraded":"2026-02-17T06:10:40.773900+0000","last_fullsized":"2026-02-17T06:10:40.773900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":1682,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773927+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T06:10:40.773927+0000","last_peered":"2026-02-17T06:10:40.773927+0000","last_clean":"2026-02-17T06:10:40.773927+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T06:10:40.773927+0000","last_undegraded":"2026-02-17T06:10:40.773927+0000","last_fullsized":"2026-02-17T06:10:40.773927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1693,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.449963+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T06:10:40.449963+0000","last_peered":"2026-02-17T06:10:40.449963+0000","last_clean":"2026-02-17T06:10:40.449963+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T06:10:40.449963+0000","last_undegraded":"2026-02-17T06:10:40.449963+0000","last_fullsized":"2026-02-17T06:10:40.449963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1702,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.772884+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T06:10:40.772884+0000","last_peered":"2026-02-17T06:10:40.772884+0000","last_clean":"2026-02-17T06:10:40.772884+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T06:10:40.772884+0000","last_undegraded":"2026-02-17T06:10:40.772884+0000","last_fullsized":"2026-02-17T06:10:40.772884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":1696,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.127960+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T06:10:40.127960+0000","last_peered":"2026-02-17T06:10:40.127960+0000","last_clean":"2026-02-17T06:10:40.127960+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T06:10:40.127960+0000","last_undegraded":"2026-02-17T06:10:40.127960+0000","last_fullsized":"2026-02-17T06:10:40.127960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1679,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.126045+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T06:10:40.126045+0000","last_peered":"2026-02-17T06:10:40.126045+0000","last_clean":"2026-02-17T06:10:40.126045+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T06:10:40.126045+0000","last_undegraded":"2026-02-17T06:10:40.126045+0000","last_fullsized":"2026-02-17T06:10:40.126045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1681,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.774004+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T06:10:40.774004+0000","last_peered":"2026-02-17T06:10:40.774004+0000","last_clean":"2026-02-17T06:10:40.774004+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T06:10:40.774004+0000","last_undegraded":"2026-02-17T06:10:40.774004+0000","last_fullsized":"2026-02-17T06:10:40.774004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1678,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773012+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T06:10:40.773012+0000","last_peered":"2026-02-17T06:10:40.773012+0000","last_clean":"2026-02-17T06:10:40.773012+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T06:10:40.773012+0000","last_undegraded":"2026-02-17T06:10:40.773012+0000","last_fullsized":"2026-02-17T06:10:40.773012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1681,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.774000+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T06:10:40.774000+0000","last_peered":"2026-02-17T06:10:40.774000+0000","last_clean":"2026-02-17T06:10:40.774000+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T06:10:40.774000+0000","last_undegraded":"2026-02-17T06:10:40.774000+0000","last_fullsized":"2026-02-17T06:10:40.774000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607858,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31156,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94340684,"statfs":{"total":96636764160,"available":96604860416,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640562,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53296,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318544,"statfs":{"total":96636764160,"available":96582189056,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61616,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310224,"statfs":{"total":96636764160,"available":96573669376,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35308,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94336532,"statfs":{"total":96636764160,"available":96600608768,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607858,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53300,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318540,"statfs":{"total":96636764160,"available":96582184960,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:11:06.593 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:11:06.659 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient: get_monmap_and_config 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient: build_initial_monmap 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient: monmap: 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:06.660338+0000 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:06.660338+0000 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:06.660 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 1 Processor -- start 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 1 -- start start 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient: init 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbdec06c890 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fbdec09c4a0 0x7fbdec13b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient(hunting): picked mon.noname-c con 0x7fbdec09c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec06d380 0x7fbdec13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient(hunting): picked mon.noname-b con 0x7fbdec06d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdec06daf0 con 0x7fbdec06c890 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fbdec0a0880 con 0x7fbdec09c4a0 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbdec09cc80 con 0x7fbdec06d380 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdf14fa640 10 monclient(hunting): authenticate will time out at 2112.213909s 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.660+0000 7fbdeaffd640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fbdec06c890 0x7fbdec06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:46388/0 (socket says 10.20.193.78:46388) 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeaffd640 1 -- 10.20.193.78:0/472008065 learned_addr learned my addr 10.20.193.78:0/472008065 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 1 --2- 10.20.193.78:0/472008065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec06d380 0x7fbdec13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient(hunting): get_auth_request con 0x7fbdec06d380 auth_method 0 2026-02-17T06:11:06.661 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdea7fc640 1 --2- 10.20.193.78:0/472008065 >> v2:10.20.193.115:3300/0 conn(0x7fbdec09c4a0 0x7fbdec13b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdea7fc640 10 monclient(hunting): get_auth_request con 0x7fbdec09c4a0 auth_method 0 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient(hunting): handle_auth_done global_id 264539 payload 274 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/472008065 >> v2:10.20.193.115:3300/0 conn(0x7fbdec09c4a0 msgr2=0x7fbdec13b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 1 --2- 10.20.193.78:0/472008065 >> v2:10.20.193.115:3300/0 conn(0x7fbdec09c4a0 0x7fbdec13b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/472008065 >> v1:10.20.193.78:6789/0 conn(0x7fbdec06c890 legacy=0x7fbdec06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 1 monclient: found mon.noname-b 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:06.662 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.661+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/472008065 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec09cf60 con 0x7fbdec06d380 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/472008065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec06d380 msgr2=0x7fbdec13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbdeb7fe640 1 --2- 10.20.193.78:0/472008065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec06d380 0x7fbdec13d8f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009d90 tx=0x7fbddc02f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7739fb692294735 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 1 -- 10.20.193.78:0/472008065 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc03c070 con 0x7fbdec06d380 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: dump: 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.663 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 1 -- 10.20.193.78:0/472008065 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbddc037440 con 0x7fbdec06d380 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbdf14fa640 5 monclient: authenticate success, global_id 264539 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbdf14fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 1 -- 10.20.193.78:0/472008065 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc037810 con 0x7fbdec06d380 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbde9ffb640 10 monclient: dump: 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbdf14fa640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:06.664 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.663+0000 7fbdf14fa640 4 set_mon_vals no callback set 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 10 monclient: shutdown 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 -- 10.20.193.78:0/472008065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec06d380 msgr2=0x7fbdec13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/472008065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec06d380 0x7fbdec13d8f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009d90 tx=0x7fbddc02f190 comp rx=0 tx=0).stop 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 -- 10.20.193.78:0/472008065 shutdown_connections 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/472008065 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec06d380 0x7fbdec13d8f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/472008065 >> v2:10.20.193.115:3300/0 conn(0x7fbdec09c4a0 0x7fbdec13b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 -- 10.20.193.78:0/472008065 >> 10.20.193.78:0/472008065 conn(0x7fbdec0a6620 msgr2=0x7fbdec0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 -- 10.20.193.78:0/472008065 shutdown_connections 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 -- 10.20.193.78:0/472008065 wait complete. 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 10 monclient: build_initial_monmap 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 10 monclient: monmap: 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:06.665 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 Processor -- start 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 1 -- start start 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.664+0000 7fbdf14fa640 10 monclient: init 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbdec06c890 0x7fbdec133170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient(hunting): picked mon.noname-a con 0x7fbdec06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbdec06d380 0x7fbdec1336b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient(hunting): picked mon.noname-b con 0x7fbdec06d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec09c4a0 0x7fbdec11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient(hunting): picked mon.noname-c con 0x7fbdec09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec09bfa0 con 0x7fbdec06c890 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec09be20 con 0x7fbdec06d380 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbdec13de30 con 0x7fbdec09c4a0 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdf14fa640 10 monclient(hunting): authenticate will time out at 2112.218205s 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec09c4a0 0x7fbdec11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeaffd640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbdec06c890 0x7fbdec133170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec09c4a0 0x7fbdec11cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60608/0 (socket says 10.20.193.78:60608) 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/1120489421 learned_addr learned my addr 10.20.193.78:0/1120489421 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient(hunting): get_auth_request con 0x7fbdec09c4a0 auth_method 0 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeaffd640 10 monclient(hunting): get_auth_request con 0x7fbdec06c890 auth_method 0 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.666 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdea7fc640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbdec06d380 0x7fbdec1336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdea7fc640 10 monclient(hunting): get_auth_request con 0x7fbdec06d380 auth_method 0 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient(hunting): handle_auth_done global_id 264542 payload 824 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbdec06d380 msgr2=0x7fbdec1336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbdec06d380 0x7fbdec1336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/1120489421 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbdec06c890 msgr2=0x7fbdec133170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbdec06c890 0x7fbdec133170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 monclient: found mon.noname-c 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.665+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/1120489421 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec1cd350 con 0x7fbdec09c4a0 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdeb7fe640 1 -- 10.20.193.78:0/1120489421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec09c4a0 msgr2=0x7fbdec11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdeb7fe640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec09c4a0 0x7fbdec11cc70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009d90 tx=0x7fbddc004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 1 -- 10.20.193.78:0/1120489421 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc03c050 con 0x7fbdec09c4a0 2026-02-17T06:11:06.667 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: dump: 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: _finish_auth 0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 1 -- 10.20.193.78:0/1120489421 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbddc048070 con 0x7fbdec09c4a0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdf14fa640 5 monclient: authenticate success, global_id 264542 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdf14fa640 10 monclient: _renew_subs 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdf14fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec09de20 con 0x7fbdec09c4a0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbde97fa640 4 set_mon_vals no callback set 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdf14fa640 10 monclient: _renew_subs 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdf14fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec11f860 con 0x7fbdec09c4a0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 1 -- 10.20.193.78:0/1120489421 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc041a30 con 0x7fbdec09c4a0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.667+0000 7fbdcb7fe640 10 monclient: dump: 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdf14fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdf14fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.668 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdf14fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdec074980 con 0x7fbdec09c4a0 2026-02-17T06:11:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdcb7fe640 1 -- 10.20.193.78:0/1120489421 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fbddc041bd0 con 0x7fbdec09c4a0 2026-02-17T06:11:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdcb7fe640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbdb803df00 0x7fbdb80403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdcb7fe640 1 -- 10.20.193.78:0/1120489421 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fbddc078600 con 0x7fbdec09c4a0 2026-02-17T06:11:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdeaffd640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbdb803df00 0x7fbdb80403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.669 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.668+0000 7fbdeaffd640 10 monclient: get_auth_request con 0x7fbdb803df00 auth_method 0 2026-02-17T06:11:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.671+0000 7fbdeaffd640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbdb803df00 0x7fbdb80403c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbde0007d90 tx=0x7fbde0007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.671+0000 7fbdcb7fe640 1 -- 10.20.193.78:0/1120489421 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbdec09de20 con 0x7fbdec09c4a0 2026-02-17T06:11:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.671+0000 7fbdcb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.670 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.671+0000 7fbdcb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:06.751 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.751+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbdec06cc70 con 0x7fbdb803df00 2026-02-17T06:11:06.752 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.752+0000 7fbdcb7fe640 1 -- 10.20.193.78:0/1120489421 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34010 (secure 0 0 0) 0x7fbdec06cc70 con 0x7fbdb803df00 2026-02-17T06:11:06.752 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbdb803df00 msgr2=0x7fbdb80403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbdb803df00 0x7fbdb80403c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbde0007d90 tx=0x7fbde0007ce0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 10 monclient: shutdown 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec09c4a0 msgr2=0x7fbdec11cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec09c4a0 0x7fbdec11cc70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009d90 tx=0x7fbddc004b40 comp rx=0 tx=0).stop 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 shutdown_connections 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbdec09c4a0 0x7fbdec11cc70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbdb803df00 0x7fbdb80403c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.754 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbdec06d380 0x7fbdec1336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 --2- 10.20.193.78:0/1120489421 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbdec06c890 0x7fbdec133170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 >> 10.20.193.78:0/1120489421 conn(0x7fbdec0a6620 msgr2=0x7fbdec134110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:06.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 shutdown_connections 2026-02-17T06:11:06.755 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.753+0000 7fbdf14fa640 1 -- 10.20.193.78:0/1120489421 wait complete. 2026-02-17T06:11:06.760 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":1608,"stamp":"2026-02-17T06:11:06.065408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":296304,"kb_used_data":24896,"kb_used_omap":9,"kb_used_meta":271286,"kb_avail":565934736,"statfs":{"total":579820584960,"available":579517169664,"internally_reserved":0,"allocated":25493504,"data_stored":12055756,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":277797542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001034"},"pg_stats":[{"pgid":"1.0","version":"16'112","reported_seq":1819,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773900+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T06:10:40.773900+0000","last_peered":"2026-02-17T06:10:40.773900+0000","last_clean":"2026-02-17T06:10:40.773900+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T06:10:40.773900+0000","last_undegraded":"2026-02-17T06:10:40.773900+0000","last_fullsized":"2026-02-17T06:10:40.773900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":1682,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773927+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T06:10:40.773927+0000","last_peered":"2026-02-17T06:10:40.773927+0000","last_clean":"2026-02-17T06:10:40.773927+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T06:10:40.773927+0000","last_undegraded":"2026-02-17T06:10:40.773927+0000","last_fullsized":"2026-02-17T06:10:40.773927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1693,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.449963+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T06:10:40.449963+0000","last_peered":"2026-02-17T06:10:40.449963+0000","last_clean":"2026-02-17T06:10:40.449963+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T06:10:40.449963+0000","last_undegraded":"2026-02-17T06:10:40.449963+0000","last_fullsized":"2026-02-17T06:10:40.449963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1702,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.772884+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T06:10:40.772884+0000","last_peered":"2026-02-17T06:10:40.772884+0000","last_clean":"2026-02-17T06:10:40.772884+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T06:10:40.772884+0000","last_undegraded":"2026-02-17T06:10:40.772884+0000","last_fullsized":"2026-02-17T06:10:40.772884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":1696,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.127960+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T06:10:40.127960+0000","last_peered":"2026-02-17T06:10:40.127960+0000","last_clean":"2026-02-17T06:10:40.127960+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T06:10:40.127960+0000","last_undegraded":"2026-02-17T06:10:40.127960+0000","last_fullsized":"2026-02-17T06:10:40.127960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1679,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.126045+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T06:10:40.126045+0000","last_peered":"2026-02-17T06:10:40.126045+0000","last_clean":"2026-02-17T06:10:40.126045+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T06:10:40.126045+0000","last_undegraded":"2026-02-17T06:10:40.126045+0000","last_fullsized":"2026-02-17T06:10:40.126045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1681,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.774004+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T06:10:40.774004+0000","last_peered":"2026-02-17T06:10:40.774004+0000","last_clean":"2026-02-17T06:10:40.774004+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T06:10:40.774004+0000","last_undegraded":"2026-02-17T06:10:40.774004+0000","last_fullsized":"2026-02-17T06:10:40.774004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1678,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773012+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T06:10:40.773012+0000","last_peered":"2026-02-17T06:10:40.773012+0000","last_clean":"2026-02-17T06:10:40.773012+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T06:10:40.773012+0000","last_undegraded":"2026-02-17T06:10:40.773012+0000","last_fullsized":"2026-02-17T06:10:40.773012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1681,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.774000+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T06:10:40.774000+0000","last_peered":"2026-02-17T06:10:40.774000+0000","last_clean":"2026-02-17T06:10:40.774000+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T06:10:40.774000+0000","last_undegraded":"2026-02-17T06:10:40.774000+0000","last_fullsized":"2026-02-17T06:10:40.774000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607858,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31156,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94340684,"statfs":{"total":96636764160,"available":96604860416,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640562,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53296,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318544,"statfs":{"total":96636764160,"available":96582189056,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61616,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310224,"statfs":{"total":96636764160,"available":96573669376,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35308,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94336532,"statfs":{"total":96636764160,"available":96600608768,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607858,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53300,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318540,"statfs":{"total":96636764160,"available":96582184960,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:11:06.762 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-17T06:11:06.762 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:11:06.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.829+0000 7fdfe06c7640 10 monclient: get_monmap_and_config 2026-02-17T06:11:06.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.829+0000 7fdfe06c7640 10 monclient: build_initial_monmap 2026-02-17T06:11:06.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.829+0000 7fdfe06c7640 10 monclient: monmap: 2026-02-17T06:11:06.832 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:06.832 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:06.829985+0000 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:06.829985+0000 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 1 Processor -- start 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 1 -- start start 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient: init 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient(hunting): picked mon.noname-b con 0x7fdfd806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fdfd809c4a0 0x7fdfd806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient(hunting): picked mon.noname-a con 0x7fdfd809c4a0 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdfd8134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd806daf0 con 0x7fdfd8134df0 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.834 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd80a0880 con 0x7fdfd809c4a0 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd806d6f0 con 0x7fdfd806c890 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfe06c7640 10 monclient(hunting): authenticate will time out at 2112.383532s 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60614/0 (socket says 10.20.193.78:60614) 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 1 -- 10.20.193.78:0/4087692419 learned_addr learned my addr 10.20.193.78:0/4087692419 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient(hunting): get_auth_request con 0x7fdfd806c890 auth_method 0 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.835 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfddc3b640 1 --2- 10.20.193.78:0/4087692419 >> v2:10.20.193.78:3300/0 conn(0x7fdfd809c4a0 0x7fdfd806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfddc3b640 10 monclient(hunting): get_auth_request con 0x7fdfd809c4a0 auth_method 0 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfddc3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfddc3b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfddc3b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfddc3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfddc3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfddc3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient(hunting): handle_auth_done global_id 264545 payload 274 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 1 -- 10.20.193.78:0/4087692419 >> v2:10.20.193.78:3300/0 conn(0x7fdfd809c4a0 msgr2=0x7fdfd806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 1 --2- 10.20.193.78:0/4087692419 >> v2:10.20.193.78:3300/0 conn(0x7fdfd809c4a0 0x7fdfd806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.836 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 1 -- 10.20.193.78:0/4087692419 >> v1:10.20.193.78:6789/0 conn(0x7fdfd8134df0 legacy=0x7fdfd8135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient: _finish_hunting 0 2026-02-17T06:11:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 1 monclient: found mon.noname-b 2026-02-17T06:11:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:06.837 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.830+0000 7fdfde43c640 1 -- 10.20.193.78:0/4087692419 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd809cc80 con 0x7fdfd806c890 2026-02-17T06:11:06.842 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfde43c640 1 -- 10.20.193.78:0/4087692419 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 msgr2=0x7fdfd806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfde43c640 1 --2- 10.20.193.78:0/4087692419 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd806cc70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009970 tx=0x7fdfc802ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2cc3d9bb2942ca2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 1 -- 10.20.193.78:0/4087692419 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc803c070 con 0x7fdfd806c890 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: dump: 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.843 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: _finish_auth 0 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: _check_auth_tickets 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 1 -- 10.20.193.78:0/4087692419 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc802fcd0 con 0x7fdfd806c890 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 5 monclient: authenticate success, global_id 264545 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 1 -- 10.20.193.78:0/4087692419 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8037660 con 0x7fdfd806c890 2026-02-17T06:11:06.844 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 4 set_mon_vals no callback set 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfdd43a640 10 monclient: dump: 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.845 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 10 monclient: shutdown 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 1 -- 10.20.193.78:0/4087692419 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 msgr2=0x7fdfd806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/4087692419 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd806cc70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009970 tx=0x7fdfc802ee90 comp rx=0 tx=0).stop 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 1 -- 10.20.193.78:0/4087692419 shutdown_connections 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/4087692419 >> v2:10.20.193.78:3300/0 conn(0x7fdfd809c4a0 0x7fdfd806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/4087692419 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd806cc70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 1 -- 10.20.193.78:0/4087692419 >> 10.20.193.78:0/4087692419 conn(0x7fdfd80a6620 msgr2=0x7fdfd80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 1 -- 10.20.193.78:0/4087692419 shutdown_connections 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.833+0000 7fdfe06c7640 1 -- 10.20.193.78:0/4087692419 wait complete. 2026-02-17T06:11:06.846 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient: build_initial_monmap 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient: monmap: 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 1 Processor -- start 2026-02-17T06:11:06.847 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 1 -- start start 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient: init 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd81cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient(hunting): picked mon.noname-c con 0x7fdfd806c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfd809c4a0 0x7fdfd81cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient(hunting): picked mon.noname-a con 0x7fdfd809c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfd8134df0 0x7fdfd81215c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient(hunting): picked mon.noname-b con 0x7fdfd8134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd80a01a0 con 0x7fdfd809c4a0 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.848 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd80a0020 con 0x7fdfd8134df0 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd809be20 con 0x7fdfd806c890 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfe06c7640 10 monclient(hunting): authenticate will time out at 2112.387613s 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd81cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfd809c4a0 0x7fdfd81cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd81cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60628/0 (socket says 10.20.193.78:60628) 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 1 -- 10.20.193.78:0/532247790 learned_addr learned my addr 10.20.193.78:0/532247790 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 10 monclient(hunting): get_auth_request con 0x7fdfd806c890 auth_method 0 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.849 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient(hunting): get_auth_request con 0x7fdfd809c4a0 auth_method 0 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient(hunting): handle_auth_done global_id 284343 payload 824 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfdec3d640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfd8134df0 0x7fdfd81215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 1 -- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 msgr2=0x7fdfd81cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.850 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd81cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 1 -- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfd8134df0 msgr2=0x7fdfd81215c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfd8134df0 0x7fdfd81215c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient: _finish_hunting 0 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 1 monclient: found mon.noname-a 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfddc3b640 1 -- 10.20.193.78:0/532247790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd8123d70 con 0x7fdfd809c4a0 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.834+0000 7fdfde43c640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd81cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.836+0000 7fdfddc3b640 1 -- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfd809c4a0 msgr2=0x7fdfd81cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.836+0000 7fdfddc3b640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfd809c4a0 0x7fdfd81cd950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00e9b0 tx=0x7fdfcc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 1 -- 10.20.193.78:0/532247790 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc017070 con 0x7fdfd809c4a0 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.851 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: dump: 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: _finish_auth 0 2026-02-17T06:11:06.852 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: _check_auth_tickets 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 1 -- 10.20.193.78:0/532247790 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfcc00cc10 con 0x7fdfd809c4a0 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 1 -- 10.20.193.78:0/532247790 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc013470 con 0x7fdfd809c4a0 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 10 monclient: dump: 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:06.853 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfdcc39640 4 set_mon_vals no callback set 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 5 monclient: authenticate success, global_id 284343 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 10 monclient: _renew_subs 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd8125500 con 0x7fdfd809c4a0 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 10 monclient: _renew_subs 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.854 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd80a16d0 con 0x7fdfd809c4a0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd8074980 con 0x7fdfd809c4a0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.837+0000 7fdfbeffd640 1 -- 10.20.193.78:0/532247790 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fdfcc024060 con 0x7fdfd809c4a0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.838+0000 7fdfbeffd640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdfa803deb0 0x7fdfa8040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.838+0000 7fdfbeffd640 1 -- 10.20.193.78:0/532247790 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fdfcc052760 con 0x7fdfd809c4a0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.838+0000 7fdfde43c640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdfa803deb0 0x7fdfa8040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.838+0000 7fdfde43c640 10 monclient: get_auth_request con 0x7fdfa803deb0 auth_method 0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.840+0000 7fdfbeffd640 1 -- 10.20.193.78:0/532247790 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdfcc0123d0 con 0x7fdfd809c4a0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.840+0000 7fdfbeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.840+0000 7fdfbeffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:06.855 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.840+0000 7fdfde43c640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdfa803deb0 0x7fdfa8040370 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdfc802eed0 tx=0x7fdfc8033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:06.920 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.920+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfd8125f00 con 0x7fdfa803deb0 2026-02-17T06:11:06.921 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.921+0000 7fdfbeffd640 1 -- 10.20.193.78:0/532247790 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34010 (secure 0 0 0) 0x7fdfc804e4f0 con 0x7fdfa803deb0 2026-02-17T06:11:06.922 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T06:11:06.923 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T06:11:06.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdfa803deb0 msgr2=0x7fdfa8040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdfa803deb0 0x7fdfa8040370 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdfc802eed0 tx=0x7fdfc8033000 comp rx=0 tx=0).stop 2026-02-17T06:11:06.923 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 10 monclient: shutdown 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfd809c4a0 msgr2=0x7fdfd81cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfd809c4a0 0x7fdfd81cd950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00e9b0 tx=0x7fdfcc00ee80 comp rx=0 tx=0).stop 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 shutdown_connections 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fdfa803deb0 0x7fdfa8040370 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fdfd8134df0 0x7fdfd81215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fdfd809c4a0 0x7fdfd81cd950 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 --2- 10.20.193.78:0/532247790 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fdfd806c890 0x7fdfd81cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 >> 10.20.193.78:0/532247790 conn(0x7fdfd80a6620 msgr2=0x7fdfd8071420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 shutdown_connections 2026-02-17T06:11:06.924 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.922+0000 7fdfe06c7640 1 -- 10.20.193.78:0/532247790 wait complete. 2026-02-17T06:11:06.930 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":1608,"stamp":"2026-02-17T06:11:06.065408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":296304,"kb_used_data":24896,"kb_used_omap":9,"kb_used_meta":271286,"kb_avail":565934736,"statfs":{"total":579820584960,"available":579517169664,"internally_reserved":0,"allocated":25493504,"data_stored":12055756,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":277797542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001034"},"pg_stats":[{"pgid":"1.0","version":"16'112","reported_seq":1819,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773900+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T06:10:40.773900+0000","last_peered":"2026-02-17T06:10:40.773900+0000","last_clean":"2026-02-17T06:10:40.773900+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T06:10:40.773900+0000","last_undegraded":"2026-02-17T06:10:40.773900+0000","last_fullsized":"2026-02-17T06:10:40.773900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":1682,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773927+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T06:10:40.773927+0000","last_peered":"2026-02-17T06:10:40.773927+0000","last_clean":"2026-02-17T06:10:40.773927+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T06:10:40.773927+0000","last_undegraded":"2026-02-17T06:10:40.773927+0000","last_fullsized":"2026-02-17T06:10:40.773927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1693,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.449963+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T06:10:40.449963+0000","last_peered":"2026-02-17T06:10:40.449963+0000","last_clean":"2026-02-17T06:10:40.449963+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T06:10:40.449963+0000","last_undegraded":"2026-02-17T06:10:40.449963+0000","last_fullsized":"2026-02-17T06:10:40.449963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1702,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.772884+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T06:10:40.772884+0000","last_peered":"2026-02-17T06:10:40.772884+0000","last_clean":"2026-02-17T06:10:40.772884+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T06:10:40.772884+0000","last_undegraded":"2026-02-17T06:10:40.772884+0000","last_fullsized":"2026-02-17T06:10:40.772884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":1696,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.127960+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T06:10:40.127960+0000","last_peered":"2026-02-17T06:10:40.127960+0000","last_clean":"2026-02-17T06:10:40.127960+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T06:10:40.127960+0000","last_undegraded":"2026-02-17T06:10:40.127960+0000","last_fullsized":"2026-02-17T06:10:40.127960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1679,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.126045+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T06:10:40.126045+0000","last_peered":"2026-02-17T06:10:40.126045+0000","last_clean":"2026-02-17T06:10:40.126045+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T06:10:40.126045+0000","last_undegraded":"2026-02-17T06:10:40.126045+0000","last_fullsized":"2026-02-17T06:10:40.126045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1681,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.774004+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T06:10:40.774004+0000","last_peered":"2026-02-17T06:10:40.774004+0000","last_clean":"2026-02-17T06:10:40.774004+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T06:10:40.774004+0000","last_undegraded":"2026-02-17T06:10:40.774004+0000","last_fullsized":"2026-02-17T06:10:40.774004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1678,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773012+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T06:10:40.773012+0000","last_peered":"2026-02-17T06:10:40.773012+0000","last_clean":"2026-02-17T06:10:40.773012+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T06:10:40.773012+0000","last_undegraded":"2026-02-17T06:10:40.773012+0000","last_fullsized":"2026-02-17T06:10:40.773012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1681,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.774000+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T06:10:40.774000+0000","last_peered":"2026-02-17T06:10:40.774000+0000","last_clean":"2026-02-17T06:10:40.774000+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T06:10:40.774000+0000","last_undegraded":"2026-02-17T06:10:40.774000+0000","last_fullsized":"2026-02-17T06:10:40.774000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607858,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31156,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94340684,"statfs":{"total":96636764160,"available":96604860416,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640562,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53296,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318544,"statfs":{"total":96636764160,"available":96582189056,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61616,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310224,"statfs":{"total":96636764160,"available":96573669376,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35308,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94336532,"statfs":{"total":96636764160,"available":96600608768,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607858,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53300,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318540,"statfs":{"total":96636764160,"available":96582184960,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:11:06.932 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T06:11:07.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.999+0000 7f42536fa640 10 monclient: get_monmap_and_config 2026-02-17T06:11:07.002 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.999+0000 7f42536fa640 10 monclient: build_initial_monmap 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:06.999+0000 7f42536fa640 10 monclient: monmap: 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:07.000076+0000 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:07.000076+0000 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:07.003 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 1 Processor -- start 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 1 -- start start 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient: init 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:07.004 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient(hunting): picked mon.noname-b con 0x7f424c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f424c09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f424c06d1b0 0x7f424c135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient(hunting): picked mon.noname-c con 0x7f424c06d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f424c06daf0 con 0x7f424c09c4a0 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f424c0a0880 con 0x7f424c06d1b0 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f424c09cc80 con 0x7f424c06c890 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f42536fa640 10 monclient(hunting): authenticate will time out at 2112.553662s 2026-02-17T06:11:07.005 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60638/0 (socket says 10.20.193.78:60638) 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 1 -- 10.20.193.78:0/3355371521 learned_addr learned my addr 10.20.193.78:0/3355371521 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 10 monclient(hunting): get_auth_request con 0x7f424c06c890 auth_method 0 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f4251c70640 1 --2- 10.20.193.78:0/3355371521 >> v2:10.20.193.115:3300/0 conn(0x7f424c06d1b0 0x7f424c135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f4251c70640 10 monclient(hunting): get_auth_request con 0x7f424c06d1b0 auth_method 0 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f4251c70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f4251c70640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:07.006 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f4251c70640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f425146f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f4251c70640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f4251c70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.000+0000 7f4251c70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 10 monclient(hunting): handle_auth_done global_id 264551 payload 274 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 1 -- 10.20.193.78:0/3355371521 >> v2:10.20.193.115:3300/0 conn(0x7f424c06d1b0 msgr2=0x7f424c135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 1 --2- 10.20.193.78:0/3355371521 >> v2:10.20.193.115:3300/0 conn(0x7f424c06d1b0 0x7f424c135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 1 -- 10.20.193.78:0/3355371521 >> v1:10.20.193.115:6789/0 conn(0x7f424c09c4a0 legacy=0x7f424c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:07.007 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 10 monclient: _finish_hunting 0 2026-02-17T06:11:07.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 1 monclient: found mon.noname-b 2026-02-17T06:11:07.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:07.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:07.008 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.001+0000 7f425146f640 1 -- 10.20.193.78:0/3355371521 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c09cf60 con 0x7f424c06c890 2026-02-17T06:11:07.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f425146f640 1 -- 10.20.193.78:0/3355371521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 msgr2=0x7f424c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:07.015 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f425146f640 1 --2- 10.20.193.78:0/3355371521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c06cc70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f423c009970 tx=0x7f423c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9650e572d906bb0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 1 -- 10.20.193.78:0/3355371521 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c03c070 con 0x7f424c06c890 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: dump: 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:07.016 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: _finish_auth 0 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 1 -- 10.20.193.78:0/3355371521 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423c02fcd0 con 0x7f424c06c890 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 5 monclient: authenticate success, global_id 264551 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:07.017 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 1 -- 10.20.193.78:0/3355371521 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c037660 con 0x7f424c06c890 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 4 set_mon_vals no callback set 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f423bfff640 10 monclient: dump: 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:07.018 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 10 monclient: shutdown 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 1 -- 10.20.193.78:0/3355371521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 msgr2=0x7f424c06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 1 --2- 10.20.193.78:0/3355371521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c06cc70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f423c009970 tx=0x7f423c02ee90 comp rx=0 tx=0).stop 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 1 -- 10.20.193.78:0/3355371521 shutdown_connections 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 1 --2- 10.20.193.78:0/3355371521 >> v2:10.20.193.115:3300/0 conn(0x7f424c06d1b0 0x7f424c135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.019 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 1 --2- 10.20.193.78:0/3355371521 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c06cc70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 1 -- 10.20.193.78:0/3355371521 >> 10.20.193.78:0/3355371521 conn(0x7f424c0a6620 msgr2=0x7f424c0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 1 -- 10.20.193.78:0/3355371521 shutdown_connections 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.003+0000 7f42536fa640 1 -- 10.20.193.78:0/3355371521 wait complete. 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient: build_initial_monmap 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient: monmap: 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:07.020 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 1 Processor -- start 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 1 -- start start 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient: init 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient(hunting): picked mon.noname-c con 0x7f424c06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:07.021 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f424c06d1b0 0x7f424c1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient(hunting): picked mon.noname-a con 0x7f424c06d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f424c09c4a0 0x7f424c124d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient(hunting): picked mon.noname-b con 0x7f424c09c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f424c0a01a0 con 0x7f424c06d1b0 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f424c0a0020 con 0x7f424c09c4a0 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f424c09be20 con 0x7f424c06c890 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f42536fa640 10 monclient(hunting): authenticate will time out at 2112.557798s 2026-02-17T06:11:07.022 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:07.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c1cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:60642/0 (socket says 10.20.193.78:60642) 2026-02-17T06:11:07.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 -- 10.20.193.78:0/4026883747 learned_addr learned my addr 10.20.193.78:0/4026883747 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:07.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient(hunting): get_auth_request con 0x7f424c06c890 auth_method 0 2026-02-17T06:11:07.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:07.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:07.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:07.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4250c6e640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f424c06d1b0 0x7f424c1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:07.023 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4250c6e640 10 monclient(hunting): get_auth_request con 0x7f424c06d1b0 auth_method 0 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4250c6e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4250c6e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4250c6e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4250c6e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4250c6e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4250c6e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient(hunting): handle_auth_done global_id 264554 payload 824 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4251c70640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f424c09c4a0 0x7f424c124d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:07.024 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 -- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f424c09c4a0 msgr2=0x7f424c124d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f424c09c4a0 0x7f424c124d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 -- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f424c06d1b0 msgr2=0x7f424c1cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f424c06d1b0 0x7f424c1cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient: _finish_hunting 0 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 monclient: found mon.noname-c 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f425146f640 1 -- 10.20.193.78:0/4026883747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c136ea0 con 0x7f424c06c890 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4251c70640 10 monclient: get_auth_request con 0x7f424c09c4a0 auth_method 0 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.004+0000 7f4251c70640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f424c09c4a0 0x7f424c124d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:11:07.025 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f425146f640 1 -- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 msgr2=0x7f424c1cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f425146f640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c1cf350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f423c009aa0 tx=0x7f423c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 1 -- 10.20.193.78:0/4026883747 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c047070 con 0x7f424c06c890 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: dump: 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:07.026 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: _finish_auth 0 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 1 -- 10.20.193.78:0/4026883747 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423c03f790 con 0x7f424c06c890 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 5 monclient: authenticate success, global_id 264554 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 10 monclient: _renew_subs 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:07.027 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424c09d810 con 0x7f424c06c890 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 1 -- 10.20.193.78:0/4026883747 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c03c040 con 0x7f424c06c890 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f423b7fe640 4 set_mon_vals no callback set 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f4239ffb640 10 monclient: dump: 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:07.028 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 10 monclient: _renew_subs 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f424c128b00 con 0x7f424c06c890 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:07.029 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.007+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424c074980 con 0x7f424c06c890 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.008+0000 7f4239ffb640 1 -- 10.20.193.78:0/4026883747 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f423c03fb80 con 0x7f424c06c890 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.008+0000 7f4239ffb640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f421c03deb0 0x7f421c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.008+0000 7f4239ffb640 1 -- 10.20.193.78:0/4026883747 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f423c0773e0 con 0x7f424c06c890 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.008+0000 7f4250c6e640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f421c03deb0 0x7f421c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.008+0000 7f4250c6e640 10 monclient: get_auth_request con 0x7f421c03deb0 auth_method 0 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.010+0000 7f4239ffb640 1 -- 10.20.193.78:0/4026883747 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f423c036e20 con 0x7f424c06c890 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.010+0000 7f4239ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.010+0000 7f4239ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:07.030 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.010+0000 7f4250c6e640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f421c03deb0 0x7f421c040370 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f424c128190 tx=0x7f4240007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:07.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.094+0000 7f42536fa640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:11:07.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.094+0000 7f42536fa640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:11:07.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.094+0000 7f42536fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:07.094 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.094+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f424c1285f0 con 0x7f424c06c890 2026-02-17T06:11:07.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f4239ffb640 1 -- 10.20.193.78:0/4026883747 <== mon.2 v2:10.20.193.78:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v867) ==== 74+0+12388 (secure 0 0 0) 0x7f423c077080 con 0x7f424c06c890 2026-02-17T06:11:07.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f4239ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-17T06:11:07.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f4239ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:11:07.095 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T06:11:07.095 INFO:teuthology.orchestra.run.trial078.stdout:{"epoch":867,"fsid":"5df7382e-3ebf-45af-b855-6719898a3119","created":"2026-02-17T05:45:51.259427+0000","modified":"2026-02-17T06:10:40.122311+0000","last_up_change":"2026-02-17T05:46:02.153660+0000","last_in_change":"2026-02-17T05:45:53.905739+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T05:46:03.931310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T05:46:08.458376+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e0537ff-96b9-44ed-87db-8e297f066844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6800","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6801","nonce":3807268031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6802","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6803","nonce":3807268031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6806","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6807","nonce":3807268031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6804","nonce":3807268031},{"type":"v1","addr":"10.20.193.78:6805","nonce":3807268031}]},"public_addr":"10.20.193.78:6801/3807268031","cluster_addr":"10.20.193.78:6803/3807268031","heartbeat_back_addr":"10.20.193.78:6807/3807268031","heartbeat_front_addr":"10.20.193.78:6805/3807268031","state":["exists","up"]},{"osd":1,"uuid":"f0bf3bb1-eef0-47f7-adad-b802b3ccbe8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6808","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6809","nonce":1094981820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6810","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6811","nonce":1094981820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6814","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6815","nonce":1094981820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6812","nonce":1094981820},{"type":"v1","addr":"10.20.193.78:6813","nonce":1094981820}]},"public_addr":"10.20.193.78:6809/1094981820","cluster_addr":"10.20.193.78:6811/1094981820","heartbeat_back_addr":"10.20.193.78:6815/1094981820","heartbeat_front_addr":"10.20.193.78:6813/1094981820","state":["exists","up"]},{"osd":2,"uuid":"9f613d91-6a9c-4d90-bd09-a2c84fc35597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6816","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6817","nonce":707008561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6818","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6819","nonce":707008561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6822","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6823","nonce":707008561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.78:6820","nonce":707008561},{"type":"v1","addr":"10.20.193.78:6821","nonce":707008561}]},"public_addr":"10.20.193.78:6817/707008561","cluster_addr":"10.20.193.78:6819/707008561","heartbeat_back_addr":"10.20.193.78:6823/707008561","heartbeat_front_addr":"10.20.193.78:6821/707008561","state":["exists","up"]},{"osd":3,"uuid":"4fe4aca4-d018-4166-953a-0bef54c014f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6810","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6812","nonce":2268846355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6814","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6816","nonce":2268846355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6822","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6824","nonce":2268846355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6818","nonce":2268846355},{"type":"v1","addr":"10.20.193.115:6820","nonce":2268846355}]},"public_addr":"10.20.193.115:6812/2268846355","cluster_addr":"10.20.193.115:6816/2268846355","heartbeat_back_addr":"10.20.193.115:6824/2268846355","heartbeat_front_addr":"10.20.193.115:6820/2268846355","state":["exists","up"]},{"osd":4,"uuid":"faad6f1c-d3bf-43dd-b7be-66d0c85bab5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6802","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6803","nonce":2725813951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6804","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6805","nonce":2725813951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6808","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6809","nonce":2725813951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6806","nonce":2725813951},{"type":"v1","addr":"10.20.193.115:6807","nonce":2725813951}]},"public_addr":"10.20.193.115:6803/2725813951","cluster_addr":"10.20.193.115:6805/2725813951","heartbeat_back_addr":"10.20.193.115:6809/2725813951","heartbeat_front_addr":"10.20.193.115:6807/2725813951","state":["exists","up"]},{"osd":5,"uuid":"907eb455-f636-4e41-8761-c13502c9c730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6811","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6813","nonce":2171295268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6815","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6817","nonce":2171295268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6823","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6825","nonce":2171295268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.115:6819","nonce":2171295268},{"type":"v1","addr":"10.20.193.115:6821","nonce":2171295268}]},"public_addr":"10.20.193.115:6813/2171295268","cluster_addr":"10.20.193.115:6817/2171295268","heartbeat_back_addr":"10.20.193.115:6825/2171295268","heartbeat_front_addr":"10.20.193.115:6821/2171295268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.280056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.641140+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.705253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:01.037254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.075730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-17T05:46:00.553045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":414,"length":1},{"begin":422,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T06:11:07.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f421c03deb0 msgr2=0x7f421c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:07.095 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f421c03deb0 0x7f421c040370 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f424c128190 tx=0x7f4240007f30 comp rx=0 tx=0).stop 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 10 monclient: shutdown 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 msgr2=0x7f424c1cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c1cf350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f423c009aa0 tx=0x7f423c004830 comp rx=0 tx=0).stop 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 shutdown_connections 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f421c03deb0 0x7f421c040370 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f424c09c4a0 0x7f424c124d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f424c06d1b0 0x7f424c1cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 --2- 10.20.193.78:0/4026883747 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f424c06c890 0x7f424c1cf350 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 >> 10.20.193.78:0/4026883747 conn(0x7f424c0a6620 msgr2=0x7f424c0a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 shutdown_connections 2026-02-17T06:11:07.096 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:07.095+0000 7f42536fa640 1 -- 10.20.193.78:0/4026883747 wait complete. 2026-02-17T06:11:08.103 INFO:tasks.ceph:Scrubbing osd.0 2026-02-17T06:11:08.103 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.174+0000 7f5d33b5e640 10 monclient: get_monmap_and_config 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.174+0000 7f5d33b5e640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.174+0000 7f5d33b5e640 10 monclient: monmap: 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:08.174825+0000 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:08.174825+0000 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:08.174 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:08.175 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:08.175 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:08.175 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:08.175 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.174+0000 7f5d33b5e640 1 Processor -- start 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 1 -- start start 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient: init 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5d2c134df0 0x7f5d2c0a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient(hunting): picked mon.noname-a con 0x7f5d2c134df0 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5d2c1351d0 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c13a240 0x7f5d2c13c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient(hunting): picked mon.noname-b con 0x7f5d2c13a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d2c0a26d0 con 0x7f5d2c1351d0 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.176 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f5d2c09f480 con 0x7f5d2c134df0 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2c09f760 con 0x7f5d2c13a240 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d33b5e640 10 monclient(hunting): authenticate will time out at 2113.728356s 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5d2c134df0 0x7f5d2c0a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c13a240 0x7f5d2c13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d310d2640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f5d2c1351d0 0x7f5d2c135640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33104/0 (socket says 10.20.193.78:33104) 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f5d2c134df0 0x7f5d2c0a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52212/0 (socket says 10.20.193.78:52212) 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d310d2640 1 -- 10.20.193.78:0/1689632753 learned_addr learned my addr 10.20.193.78:0/1689632753 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c13a240 0x7f5d2c13c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34630/0 (socket says 10.20.193.78:34630) 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient(hunting): get_auth_request con 0x7f5d2c13a240 auth_method 0 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 10 monclient(hunting): get_auth_request con 0x7f5d2c134df0 auth_method 0 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient(hunting): handle_auth_done global_id 264557 payload 274 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 1 -- 10.20.193.78:0/1689632753 >> v2:10.20.193.78:3300/0 conn(0x7f5d2c134df0 msgr2=0x7f5d2c0a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 1 --2- 10.20.193.78:0/1689632753 >> v2:10.20.193.78:3300/0 conn(0x7f5d2c134df0 0x7f5d2c0a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 1 -- 10.20.193.78:0/1689632753 >> v1:10.20.193.78:6789/0 conn(0x7f5d2c1351d0 legacy=0x7f5d2c135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.177 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 1 monclient: found mon.noname-b 2026-02-17T06:11:08.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d320d4640 1 -- 10.20.193.78:0/1689632753 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c09fa40 con 0x7f5d2c13a240 2026-02-17T06:11:08.178 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.175+0000 7f5d318d3640 1 --2- 10.20.193.78:0/1689632753 >> v2:10.20.193.78:3300/0 conn(0x7f5d2c134df0 0x7f5d2c0a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d320d4640 1 -- 10.20.193.78:0/1689632753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c13a240 msgr2=0x7f5d2c13c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d320d4640 1 --2- 10.20.193.78:0/1689632753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c13a240 0x7f5d2c13c630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c06dc10 tx=0x7f5d2802f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cd2b6d428a07aa server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 1 -- 10.20.193.78:0/1689632753 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2803c070 con 0x7f5d2c13a240 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: dump: 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.180 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 1 -- 10.20.193.78:0/1689632753 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d28037440 con 0x7f5d2c13a240 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 5 monclient: authenticate success, global_id 264557 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 4 set_mon_vals no callback set 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 1 -- 10.20.193.78:0/1689632753 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28037810 con 0x7f5d2c13a240 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d308d1640 10 monclient: dump: 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 10 monclient: shutdown 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 1 -- 10.20.193.78:0/1689632753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c13a240 msgr2=0x7f5d2c13c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 1 --2- 10.20.193.78:0/1689632753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c13a240 0x7f5d2c13c630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c06dc10 tx=0x7f5d2802f130 comp rx=0 tx=0).stop 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 1 -- 10.20.193.78:0/1689632753 shutdown_connections 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 1 --2- 10.20.193.78:0/1689632753 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c13a240 0x7f5d2c13c630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.181 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 1 --2- 10.20.193.78:0/1689632753 >> v2:10.20.193.78:3300/0 conn(0x7f5d2c134df0 0x7f5d2c0a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 1 -- 10.20.193.78:0/1689632753 >> 10.20.193.78:0/1689632753 conn(0x7f5d2c0a3300 msgr2=0x7f5d2c0a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 1 -- 10.20.193.78:0/1689632753 shutdown_connections 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 1 -- 10.20.193.78:0/1689632753 wait complete. 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.178+0000 7f5d33b5e640 10 monclient: monmap: 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 1 Processor -- start 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 1 -- start start 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient: init 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5d2c134df0 0x7f5d2c072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient(hunting): picked mon.noname-a con 0x7f5d2c134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c1351d0 0x7f5d2c13dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient(hunting): picked mon.noname-c con 0x7f5d2c1351d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5d2c13a240 0x7f5d2c13e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient(hunting): picked mon.noname-b con 0x7f5d2c13a240 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c09d220 con 0x7f5d2c134df0 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.182 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c09d0a0 con 0x7f5d2c13a240 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2c13cb70 con 0x7f5d2c1351d0 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d33b5e640 10 monclient(hunting): authenticate will time out at 2113.732473s 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c1351d0 0x7f5d2c13dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5d2c134df0 0x7f5d2c072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c1351d0 0x7f5d2c13dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34634/0 (socket says 10.20.193.78:34634) 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5d2c134df0 0x7f5d2c072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52226/0 (socket says 10.20.193.78:52226) 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 -- 10.20.193.78:0/452335976 learned_addr learned my addr 10.20.193.78:0/452335976 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient(hunting): get_auth_request con 0x7f5d2c1351d0 auth_method 0 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 10 monclient(hunting): get_auth_request con 0x7f5d2c134df0 auth_method 0 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient(hunting): handle_auth_done global_id 264560 payload 824 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 -- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5d2c13a240 msgr2=0x7f5d2c13e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d320d4640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5d2c13a240 0x7f5d2c13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5d2c13a240 0x7f5d2c13e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 -- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5d2c134df0 msgr2=0x7f5d2c072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5d2c134df0 0x7f5d2c072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.183 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 monclient: found mon.noname-c 2026-02-17T06:11:08.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d310d2640 1 -- 10.20.193.78:0/452335976 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c0733d0 con 0x7f5d2c1351d0 2026-02-17T06:11:08.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d318d3640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f5d2c134df0 0x7f5d2c072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:08.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d320d4640 10 monclient: get_auth_request con 0x7f5d2c13a240 auth_method 0 2026-02-17T06:11:08.184 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.179+0000 7f5d320d4640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f5d2c13a240 0x7f5d2c13e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d310d2640 1 -- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c1351d0 msgr2=0x7f5d2c13dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d310d2640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c1351d0 0x7f5d2c13dc30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000c780 tx=0x7f5d2000cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20005de0 con 0x7f5d2c1351d0 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: dump: 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.185 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d20007070 con 0x7f5d2c1351d0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d33b5e640 5 monclient: authenticate success, global_id 264560 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d33b5e640 10 monclient: _renew_subs 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d33b5e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c070ce0 con 0x7f5d2c1351d0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20007440 con 0x7f5d2c1351d0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1a7fc640 10 monclient: dump: 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d1bfff640 4 set_mon_vals no callback set 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d33b5e640 10 monclient: _renew_subs 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d33b5e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.181+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c0a13b0 con 0x7f5d2c1351d0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d33b5e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d2c13c960 con 0x7f5d2c1351d0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5d20007620 con 0x7f5d2c1351d0 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d1a7fc640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cfc03df00 0x7f5cfc0403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.186 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f5d200523c0 con 0x7f5d2c1351d0 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d1a7fc640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f5cfc043e20 0x7f5cfc046240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 --> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5cfc0468f0 con 0x7f5cfc043e20 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_get_version_reply(handle=1 version=867) ==== 24+0+0 (secure 0 0 0) 0x7f5d20059050 con 0x7f5d2c1351d0 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d1a7fc640 10 monclient: handle_get_version_reply finishing 1 version 867 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d320d4640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f5cfc043e20 0x7f5cfc046240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d320d4640 10 monclient: get_auth_request con 0x7f5cfc043e20 auth_method 0 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.182+0000 7f5d318d3640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cfc03df00 0x7f5cfc0403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.183+0000 7f5d318d3640 10 monclient: get_auth_request con 0x7f5cfc03df00 auth_method 0 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.185+0000 7f5d318d3640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cfc03df00 0x7f5cfc0403c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009660 tx=0x7f5d1c009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.187+0000 7f5d320d4640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f5cfc043e20 0x7f5cfc046240 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.187 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.187+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 <== osd.0 v2:10.20.193.78:6800/3807268031 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f5cfc0468f0 con 0x7f5cfc043e20 2026-02-17T06:11:08.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.193+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 --> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5d2c0a0140 con 0x7f5cfc043e20 2026-02-17T06:11:08.193 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.193+0000 7f5d1a7fc640 1 -- 10.20.193.78:0/452335976 <== osd.0 v2:10.20.193.78:6800/3807268031 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5d2c0a0140 con 0x7f5cfc043e20 2026-02-17T06:11:08.193 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:08.193 INFO:teuthology.orchestra.run.trial078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:11:08.193 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f5cfc043e20 msgr2=0x7f5cfc046240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:6800/3807268031,v1:10.20.193.78:6801/3807268031] conn(0x7f5cfc043e20 0x7f5cfc046240 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cfc03df00 msgr2=0x7f5cfc0403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f5cfc03df00 0x7f5cfc0403c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009660 tx=0x7f5d1c009290 comp rx=0 tx=0).stop 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 10 monclient: shutdown 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c1351d0 msgr2=0x7f5d2c13dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 --2- 10.20.193.78:0/452335976 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f5d2c1351d0 0x7f5d2c13dc30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000c780 tx=0x7f5d2000cc50 comp rx=0 tx=0).stop 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d320d4640 1 -- 10.20.193.78:0/452335976 reap_dead start 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 shutdown_connections 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 >> 10.20.193.78:0/452335976 conn(0x7f5d2c0a3300 msgr2=0x7f5d2c0a4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 shutdown_connections 2026-02-17T06:11:08.194 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.194+0000 7f5d33b5e640 1 -- 10.20.193.78:0/452335976 wait complete. 2026-02-17T06:11:08.198 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-17T06:11:08.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.265+0000 7fcddc699640 10 monclient: get_monmap_and_config 2026-02-17T06:11:08.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.265+0000 7fcddc699640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.268 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.265+0000 7fcddc699640 10 monclient: monmap: 2026-02-17T06:11:08.268 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.268 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:08.265577+0000 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:08.265577+0000 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.265+0000 7fcddc699640 1 Processor -- start 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.265+0000 7fcddc699640 1 -- start start 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.265+0000 7fcddc699640 10 monclient: init 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fcdd406c890 0x7fcdd406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient(hunting): picked mon.noname-c con 0x7fcdd406c890 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcdd409c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406d1b0 0x7fcdd4135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient(hunting): picked mon.noname-b con 0x7fcdd406d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd406daf0 con 0x7fcdd409c4a0 2026-02-17T06:11:08.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd40a0880 con 0x7fcdd406c890 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd409cc80 con 0x7fcdd406d1b0 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddc699640 10 monclient(hunting): authenticate will time out at 2113.819130s 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406d1b0 0x7fcdd4135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcdd9c0d640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fcdd409c4a0 0x7fcdd4134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33114/0 (socket says 10.20.193.78:33114) 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcdd9c0d640 1 -- 10.20.193.78:0/1073814863 learned_addr learned my addr 10.20.193.78:0/1073814863 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406d1b0 0x7fcdd4135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34648/0 (socket says 10.20.193.78:34648) 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient(hunting): get_auth_request con 0x7fcdd406d1b0 auth_method 0 2026-02-17T06:11:08.271 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcdda40e640 1 --2- 10.20.193.78:0/1073814863 >> v2:10.20.193.115:3300/0 conn(0x7fcdd406c890 0x7fcdd406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcdda40e640 10 monclient(hunting): get_auth_request con 0x7fcdd406c890 auth_method 0 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcdda40e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcdda40e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcdda40e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.272 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient(hunting): handle_auth_done global_id 264563 payload 274 2026-02-17T06:11:08.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 1 -- 10.20.193.78:0/1073814863 >> v2:10.20.193.115:3300/0 conn(0x7fcdd406c890 msgr2=0x7fcdd406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 1 --2- 10.20.193.78:0/1073814863 >> v2:10.20.193.115:3300/0 conn(0x7fcdd406c890 0x7fcdd406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 1 -- 10.20.193.78:0/1073814863 >> v1:10.20.193.78:6789/0 conn(0x7fcdd409c4a0 legacy=0x7fcdd4134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 1 monclient: found mon.noname-b 2026-02-17T06:11:08.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.273 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.266+0000 7fcddac0f640 1 -- 10.20.193.78:0/1073814863 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd409cf60 con 0x7fcdd406d1b0 2026-02-17T06:11:08.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcddac0f640 1 -- 10.20.193.78:0/1073814863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406d1b0 msgr2=0x7fcdd4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcddac0f640 1 --2- 10.20.193.78:0/1073814863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406d1b0 0x7fcdd4135500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcdd406dcc0 tx=0x7fcdd002f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d9702c3d435a2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 1 -- 10.20.193.78:0/1073814863 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd003c070 con 0x7fcdd406d1b0 2026-02-17T06:11:08.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.282 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: dump: 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.283 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcddc699640 5 monclient: authenticate success, global_id 264563 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 1 -- 10.20.193.78:0/1073814863 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd0037440 con 0x7fcdd406d1b0 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcddc699640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 1 -- 10.20.193.78:0/1073814863 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0037810 con 0x7fcdd406d1b0 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcddc699640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcddc699640 4 set_mon_vals no callback set 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.284 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.268+0000 7fcdd940c640 10 monclient: dump: 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.285 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 10 monclient: shutdown 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 1 -- 10.20.193.78:0/1073814863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406d1b0 msgr2=0x7fcdd4135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 1 --2- 10.20.193.78:0/1073814863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406d1b0 0x7fcdd4135500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcdd406dcc0 tx=0x7fcdd002f150 comp rx=0 tx=0).stop 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 1 -- 10.20.193.78:0/1073814863 shutdown_connections 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 1 --2- 10.20.193.78:0/1073814863 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406d1b0 0x7fcdd4135500 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 1 --2- 10.20.193.78:0/1073814863 >> v2:10.20.193.115:3300/0 conn(0x7fcdd406c890 0x7fcdd406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 1 -- 10.20.193.78:0/1073814863 >> 10.20.193.78:0/1073814863 conn(0x7fcdd40a6620 msgr2=0x7fcdd40a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 1 -- 10.20.193.78:0/1073814863 shutdown_connections 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 1 -- 10.20.193.78:0/1073814863 wait complete. 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.286 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.269+0000 7fcddc699640 10 monclient: monmap: 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 1 Processor -- start 2026-02-17T06:11:08.287 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 1 -- start start 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient: init 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406c890 0x7fcdd41cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient(hunting): picked mon.noname-c con 0x7fcdd406c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcdd406d1b0 0x7fcdd41217b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient(hunting): picked mon.noname-a con 0x7fcdd406d1b0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcdd409c4a0 0x7fcdd4121cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient(hunting): picked mon.noname-b con 0x7fcdd409c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd40a01a0 con 0x7fcdd406d1b0 2026-02-17T06:11:08.288 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd40a0020 con 0x7fcdd409c4a0 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd409be20 con 0x7fcdd406c890 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddc699640 10 monclient(hunting): authenticate will time out at 2113.823456s 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406c890 0x7fcdd41cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcdd406d1b0 0x7fcdd41217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406c890 0x7fcdd41cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34664/0 (socket says 10.20.193.78:34664) 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 -- 10.20.193.78:0/575771509 learned_addr learned my addr 10.20.193.78:0/575771509 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.289 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient(hunting): get_auth_request con 0x7fcdd406c890 auth_method 0 2026-02-17T06:11:08.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 10 monclient(hunting): get_auth_request con 0x7fcdd406d1b0 auth_method 0 2026-02-17T06:11:08.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.290 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddac0f640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcdd409c4a0 0x7fcdd4121cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.291 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient(hunting): handle_auth_done global_id 264566 payload 824 2026-02-17T06:11:08.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 -- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcdd409c4a0 msgr2=0x7fcdd4121cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcdd409c4a0 0x7fcdd4121cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 -- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcdd406d1b0 msgr2=0x7fcdd41217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcdd406d1b0 0x7fcdd41217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 monclient: found mon.noname-c 2026-02-17T06:11:08.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.292 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdda40e640 1 -- 10.20.193.78:0/575771509 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd4123fb0 con 0x7fcdd406c890 2026-02-17T06:11:08.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddac0f640 10 monclient: get_auth_request con 0x7fcdd409c4a0 auth_method 0 2026-02-17T06:11:08.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcddac0f640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcdd409c4a0 0x7fcdd4121cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:11:08.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.270+0000 7fcdd9c0d640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcdd406d1b0 0x7fcdd41217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:08.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdda40e640 1 -- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406c890 msgr2=0x7fcdd41cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdda40e640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406c890 0x7fcdd41cd780 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400e870 tx=0x7fcdc400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.293 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 1 -- 10.20.193.78:0/575771509 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4017070 con 0x7fcdd406c890 2026-02-17T06:11:08.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:08.294 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: dump: 2026-02-17T06:11:08.294 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.294 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.294 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.294 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.294 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.295 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.295 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.295 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.295 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.295 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.295 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 1 -- 10.20.193.78:0/575771509 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc400cae0 con 0x7fcdd406c890 2026-02-17T06:11:08.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 1 -- 10.20.193.78:0/575771509 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400f040 con 0x7fcdd406c890 2026-02-17T06:11:08.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:08.296 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdbaffd640 10 monclient: dump: 2026-02-17T06:11:08.296 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.296 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.297 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.297 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.297 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.297 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.297 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.297 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.297 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.297 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcdd8c0b640 4 set_mon_vals no callback set 2026-02-17T06:11:08.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcddc699640 5 monclient: authenticate success, global_id 264566 2026-02-17T06:11:08.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcddc699640 10 monclient: _renew_subs 2026-02-17T06:11:08.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcddc699640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.272+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd4125740 con 0x7fcdd406c890 2026-02-17T06:11:08.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcddc699640 10 monclient: _renew_subs 2026-02-17T06:11:08.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcddc699640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.298 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd40a20c0 con 0x7fcdd406c890 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcddc699640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcddc699640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcddc699640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcdbaffd640 1 -- 10.20.193.78:0/575771509 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fcdc4012050 con 0x7fcdd406c890 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd4074980 con 0x7fcdd406c890 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcdbaffd640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcda403df00 0x7fcda40403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcdbaffd640 1 -- 10.20.193.78:0/575771509 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fcdc40523f0 con 0x7fcdd406c890 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcdd9c0d640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcda403df00 0x7fcda40403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.299 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.273+0000 7fcdd9c0d640 10 monclient: get_auth_request con 0x7fcda403df00 auth_method 0 2026-02-17T06:11:08.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.276+0000 7fcdbaffd640 1 -- 10.20.193.78:0/575771509 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcdc4050f70 con 0x7fcdd406c890 2026-02-17T06:11:08.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.276+0000 7fcdbaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.276+0000 7fcdbaffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:08.300 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.276+0000 7fcdd9c0d640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcda403df00 0x7fcda40403c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8007d90 tx=0x7fcdc8007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.357+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fcdd4126140 con 0x7fcda403df00 2026-02-17T06:11:08.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.357+0000 7fcdbaffd640 1 -- 10.20.193.78:0/575771509 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcdc8006d40 con 0x7fcda403df00 2026-02-17T06:11:08.357 INFO:teuthology.orchestra.run.trial078.stderr:instructed osd(s) 0 to deep-scrub 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcda403df00 msgr2=0x7fcda40403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcda403df00 0x7fcda40403c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8007d90 tx=0x7fcdc8007ce0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 10 monclient: shutdown 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406c890 msgr2=0x7fcdd41cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406c890 0x7fcdd41cd780 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400e870 tx=0x7fcdc400ed40 comp rx=0 tx=0).stop 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 shutdown_connections 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fcda403df00 0x7fcda40403c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fcdd409c4a0 0x7fcdd4121cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fcdd406d1b0 0x7fcdd41217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 --2- 10.20.193.78:0/575771509 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fcdd406c890 0x7fcdd41cd780 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 >> 10.20.193.78:0/575771509 conn(0x7fcdd40a6620 msgr2=0x7fcdd40a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 shutdown_connections 2026-02-17T06:11:08.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.358+0000 7fcddc699640 1 -- 10.20.193.78:0/575771509 wait complete. 2026-02-17T06:11:08.365 INFO:tasks.ceph:Scrubbing osd.1 2026-02-17T06:11:08.365 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:11:08.435 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.433+0000 7f77ebc66640 10 monclient: get_monmap_and_config 2026-02-17T06:11:08.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.433+0000 7f77ebc66640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.436 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.433+0000 7f77ebc66640 10 monclient: monmap: 2026-02-17T06:11:08.436 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.436 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.436 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:08.433686+0000 2026-02-17T06:11:08.436 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:08.433686+0000 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.437 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.433+0000 7f77ebc66640 1 Processor -- start 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.433+0000 7f77ebc66640 1 -- start start 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.433+0000 7f77ebc66640 10 monclient: init 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 10 monclient: _add_conns ranks=[0,1,3,2,4] 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f77e40a1020 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f77e40a1b10 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f77e413c970 0x7f77e413ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 10 monclient(hunting): picked mon.noname-a con 0x7f77e413c970 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e40a1f20 con 0x7f77e40a1020 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e409fed0 con 0x7f77e40a1b10 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.438 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f77e40a2980 con 0x7f77e413c970 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ebc66640 10 monclient(hunting): authenticate will time out at 2113.987236s 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77e99db640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f77e40a1020 0x7f77e40a1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33120/0 (socket says 10.20.193.78:33120) 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77e99db640 1 -- 10.20.193.78:0/4237750968 learned_addr learned my addr 10.20.193.78:0/4237750968 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 1 --2- 10.20.193.78:0/4237750968 >> v2:10.20.193.78:3300/0 conn(0x7f77e413c970 0x7f77e413ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient(hunting): get_auth_request con 0x7f77e413c970 auth_method 0 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.439 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient(hunting): handle_auth_done global_id 284358 payload 274 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 1 -- 10.20.193.78:0/4237750968 >> v1:10.20.193.115:6789/0 conn(0x7f77e40a1b10 legacy=0x7f77e413b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 1 -- 10.20.193.78:0/4237750968 >> v1:10.20.193.78:6789/0 conn(0x7f77e40a1020 legacy=0x7f77e40a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 1 monclient: found mon.noname-a 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.440 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.434+0000 7f77ea1dc640 1 -- 10.20.193.78:0/4237750968 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e40a0590 con 0x7f77e413c970 2026-02-17T06:11:08.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.436+0000 7f77ea1dc640 1 -- 10.20.193.78:0/4237750968 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e413c970 msgr2=0x7f77e413ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.448 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.436+0000 7f77ea1dc640 1 --2- 10.20.193.78:0/4237750968 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e413c970 0x7f77e413ed60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009d90 tx=0x7f77e002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10bd660dc23f83 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 1 -- 10.20.193.78:0/4237750968 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e003c070 con 0x7f77e413c970 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: dump: 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.449 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 5 monclient: authenticate success, global_id 284358 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 1 -- 10.20.193.78:0/4237750968 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e0037640 con 0x7f77e413c970 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.450 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 4 set_mon_vals no callback set 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 1 -- 10.20.193.78:0/4237750968 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0037a10 con 0x7f77e413c970 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77e89d9640 10 monclient: dump: 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.451 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 10 monclient: shutdown 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 1 -- 10.20.193.78:0/4237750968 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e413c970 msgr2=0x7f77e413ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 1 --2- 10.20.193.78:0/4237750968 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e413c970 0x7f77e413ed60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009d90 tx=0x7f77e002f1e0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 1 -- 10.20.193.78:0/4237750968 shutdown_connections 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 1 --2- 10.20.193.78:0/4237750968 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e413c970 0x7f77e413ed60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.452 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 1 -- 10.20.193.78:0/4237750968 >> 10.20.193.78:0/4237750968 conn(0x7f77e40a3300 msgr2=0x7f77e40a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 1 -- 10.20.193.78:0/4237750968 shutdown_connections 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 1 -- 10.20.193.78:0/4237750968 wait complete. 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.437+0000 7f77ebc66640 10 monclient: monmap: 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:08.453 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 1 Processor -- start 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 1 -- start start 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient: init 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77e40a1020 0x7f77e41cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient(hunting): picked mon.noname-b con 0x7f77e40a1020 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e40a1b10 0x7f77e41312d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient(hunting): picked mon.noname-a con 0x7f77e40a1b10 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:08.454 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77e413c970 0x7f77e4131810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient(hunting): picked mon.noname-c con 0x7f77e413c970 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77e40a0a50 con 0x7f77e40a1b10 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77e40a08d0 con 0x7f77e40a1020 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f77e40a0180 con 0x7f77e413c970 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ebc66640 10 monclient(hunting): authenticate will time out at 2113.991547s 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e40a1b10 0x7f77e41312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.455 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e40a1b10 0x7f77e41312d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52258/0 (socket says 10.20.193.78:52258) 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 -- 10.20.193.78:0/2724548347 learned_addr learned my addr 10.20.193.78:0/2724548347 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient(hunting): get_auth_request con 0x7f77e40a1b10 auth_method 0 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ea1dc640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77e413c970 0x7f77e4131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ea1dc640 10 monclient(hunting): get_auth_request con 0x7f77e413c970 auth_method 0 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ea1dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.456 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ea1dc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ea1dc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e99db640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77e40a1020 0x7f77e41cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e99db640 10 monclient(hunting): get_auth_request con 0x7f77e40a1020 auth_method 0 2026-02-17T06:11:08.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e99db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e99db640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e99db640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ea1dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.457 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ea1dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77ea1dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient(hunting): handle_auth_done global_id 284361 payload 824 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 -- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77e413c970 msgr2=0x7f77e4131810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f77e413c970 0x7f77e4131810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 -- 10.20.193.78:0/2724548347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77e40a1020 msgr2=0x7f77e41cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f77e40a1020 0x7f77e41cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 monclient: found mon.noname-a 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.438+0000 7f77e91da640 1 -- 10.20.193.78:0/2724548347 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e413bd60 con 0x7f77e40a1b10 2026-02-17T06:11:08.458 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.440+0000 7f77e91da640 1 -- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e40a1b10 msgr2=0x7f77e41312d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.440+0000 7f77e91da640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e40a1b10 0x7f77e41312d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77d800c5d0 tx=0x7f77d800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.440+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8016020 con 0x7f77e40a1b10 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: dump: 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.459 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d8012a60 con 0x7f77e40a1b10 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8012e50 con 0x7f77e40a1b10 2026-02-17T06:11:08.460 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 5 monclient: authenticate success, global_id 284361 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d3fff640 4 set_mon_vals no callback set 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 10 monclient: _renew_subs 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e406f7d0 con 0x7f77e40a1b10 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 10 monclient: dump: 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.461 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 10 monclient: _renew_subs 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e409d810 con 0x7f77e40a1b10 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77e40a1400 con 0x7f77e40a1b10 2026-02-17T06:11:08.462 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f77d801b400 con 0x7f77e40a1b10 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.441+0000 7f77d27fc640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77b403e0d0 0x7f77b4040590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f77d80526b0 con 0x7f77e40a1b10 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77d27fc640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f77b4043ff0 0x7f77b4046410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 --> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77d801eeb0 con 0x7f77b4043ff0 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=867) ==== 24+0+0 (secure 0 0 0) 0x7f77d8059050 con 0x7f77e40a1b10 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77d27fc640 10 monclient: handle_get_version_reply finishing 1 version 867 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77e99db640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77b403e0d0 0x7f77b4040590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77e99db640 10 monclient: get_auth_request con 0x7f77b403e0d0 auth_method 0 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77ea1dc640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f77b4043ff0 0x7f77b4046410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.442+0000 7f77ea1dc640 10 monclient: get_auth_request con 0x7f77b4043ff0 auth_method 0 2026-02-17T06:11:08.463 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.444+0000 7f77e99db640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77b403e0d0 0x7f77b4040590 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009910 tx=0x7f77d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.446+0000 7f77ea1dc640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f77b4043ff0 0x7f77b4046410 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.446+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 <== osd.1 v2:10.20.193.78:6808/1094981820 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f77d801eeb0 con 0x7f77b4043ff0 2026-02-17T06:11:08.464 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 --> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f77e411cd30 con 0x7f77b4043ff0 2026-02-17T06:11:08.465 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77d27fc640 1 -- 10.20.193.78:0/2724548347 <== osd.1 v2:10.20.193.78:6808/1094981820 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f77e411cd30 con 0x7f77b4043ff0 2026-02-17T06:11:08.465 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:08.465 INFO:teuthology.orchestra.run.trial078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:11:08.465 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:08.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f77b4043ff0 msgr2=0x7f77b4046410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:6808/1094981820,v1:10.20.193.78:6809/1094981820] conn(0x7f77b4043ff0 0x7f77b4046410 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77b403e0d0 msgr2=0x7f77b4040590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f77b403e0d0 0x7f77b4040590 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009910 tx=0x7f77d4008040 comp rx=0 tx=0).stop 2026-02-17T06:11:08.467 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 10 monclient: shutdown 2026-02-17T06:11:08.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e40a1b10 msgr2=0x7f77e41312d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 --2- 10.20.193.78:0/2724548347 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f77e40a1b10 0x7f77e41312d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77d800c5d0 tx=0x7f77d800caa0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ea1dc640 1 -- 10.20.193.78:0/2724548347 reap_dead start 2026-02-17T06:11:08.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 shutdown_connections 2026-02-17T06:11:08.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 >> 10.20.193.78:0/2724548347 conn(0x7f77e40a3300 msgr2=0x7f77e40a53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 shutdown_connections 2026-02-17T06:11:08.468 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.453+0000 7f77ebc66640 1 -- 10.20.193.78:0/2724548347 wait complete. 2026-02-17T06:11:08.468 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-17T06:11:08.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.535+0000 7f4f39287640 10 monclient: get_monmap_and_config 2026-02-17T06:11:08.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.535+0000 7f4f39287640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.535 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.535+0000 7f4f39287640 10 monclient: monmap: 2026-02-17T06:11:08.535 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:08.535503+0000 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:08.535503+0000 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:08.536 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.543 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.535+0000 7f4f39287640 1 Processor -- start 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.535+0000 7f4f39287640 1 -- start start 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.535+0000 7f4f39287640 10 monclient: init 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient: _add_conns ranks=[2,1,4,0,3] 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6470 0x7f4f340a1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient(hunting): picked mon.noname-b con 0x7f4f340a6470 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4f340a6850 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f4f340a6c30 0x7f4f34138660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient(hunting): picked mon.noname-c con 0x7f4f340a6c30 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f340a0730 con 0x7f4f340a6850 2026-02-17T06:11:08.544 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f4f3409fa00 con 0x7f4f340a6c30 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4f340a19c0 con 0x7f4f340a6470 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f39287640 10 monclient(hunting): authenticate will time out at 2114.089217s 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6470 0x7f4f340a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6470 0x7f4f340a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34680/0 (socket says 10.20.193.78:34680) 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 1 -- 10.20.193.78:0/1392754839 learned_addr learned my addr 10.20.193.78:0/1392754839 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient(hunting): get_auth_request con 0x7f4f340a6470 auth_method 0 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.545 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f33577640 1 --2- 10.20.193.78:0/1392754839 >> v2:10.20.193.115:3300/0 conn(0x7f4f340a6c30 0x7f4f34138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f33577640 10 monclient(hunting): get_auth_request con 0x7f4f340a6c30 auth_method 0 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f33577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f33577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f33577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient(hunting): handle_auth_done global_id 264572 payload 274 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 1 -- 10.20.193.78:0/1392754839 >> v2:10.20.193.115:3300/0 conn(0x7f4f340a6c30 msgr2=0x7f4f34138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 1 --2- 10.20.193.78:0/1392754839 >> v2:10.20.193.115:3300/0 conn(0x7f4f340a6c30 0x7f4f34138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.546 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 1 -- 10.20.193.78:0/1392754839 >> v1:10.20.193.115:6789/0 conn(0x7f4f340a6850 legacy=0x7f4f34137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 1 monclient: found mon.noname-b 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.536+0000 7f4f32d76640 1 -- 10.20.193.78:0/1392754839 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f340a0450 con 0x7f4f340a6470 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f32d76640 1 -- 10.20.193.78:0/1392754839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6470 msgr2=0x7f4f340a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f32d76640 1 --2- 10.20.193.78:0/1392754839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6470 0x7f4f340a1480 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009970 tx=0x7f4f2802ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=326195687faa60af server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 1 -- 10.20.193.78:0/1392754839 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2803c070 con 0x7f4f340a6470 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: dump: 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.547 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 1 -- 10.20.193.78:0/1392754839 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f2802fcd0 con 0x7f4f340a6470 2026-02-17T06:11:08.548 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 5 monclient: authenticate success, global_id 264572 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 1 -- 10.20.193.78:0/1392754839 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28037660 con 0x7f4f340a6470 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 4 set_mon_vals no callback set 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f31d74640 10 monclient: dump: 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.549 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 10 monclient: shutdown 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 1 -- 10.20.193.78:0/1392754839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6470 msgr2=0x7f4f340a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 1 --2- 10.20.193.78:0/1392754839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6470 0x7f4f340a1480 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009970 tx=0x7f4f2802ee90 comp rx=0 tx=0).stop 2026-02-17T06:11:08.550 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 1 -- 10.20.193.78:0/1392754839 shutdown_connections 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 1 --2- 10.20.193.78:0/1392754839 >> v2:10.20.193.115:3300/0 conn(0x7f4f340a6c30 0x7f4f34138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 1 --2- 10.20.193.78:0/1392754839 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6470 0x7f4f340a1480 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 1 -- 10.20.193.78:0/1392754839 >> 10.20.193.78:0/1392754839 conn(0x7f4f340aae40 msgr2=0x7f4f340ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 1 -- 10.20.193.78:0/1392754839 shutdown_connections 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 1 -- 10.20.193.78:0/1392754839 wait complete. 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.539+0000 7f4f39287640 10 monclient: monmap: 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.551 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 1 Processor -- start 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 1 -- start start 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient: init 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f340a6470 0x7f4f341cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient(hunting): picked mon.noname-a con 0x7f4f340a6470 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6850 0x7f4f34134de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient(hunting): picked mon.noname-c con 0x7f4f340a6850 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.552 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f340a6c30 0x7f4f34135320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient(hunting): picked mon.noname-b con 0x7f4f340a6c30 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34138ba0 con 0x7f4f340a6470 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34138d20 con 0x7f4f340a6c30 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f4f34140290 con 0x7f4f340a6850 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f39287640 10 monclient(hunting): authenticate will time out at 2114.093566s 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f340a6470 0x7f4f341cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f340a6470 0x7f4f341cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52274/0 (socket says 10.20.193.78:52274) 2026-02-17T06:11:08.553 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 -- 10.20.193.78:0/2802967408 learned_addr learned my addr 10.20.193.78:0/2802967408 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient(hunting): get_auth_request con 0x7f4f340a6470 auth_method 0 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32575640 1 -- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6850 msgr2=0x7f4f34134de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32575640 1 -- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6850 msgr2=0x7f4f34134de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32575640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6850 0x7f4f34134de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32575640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6850 0x7f4f34134de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f33577640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f340a6c30 0x7f4f34135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.554 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f33577640 10 monclient(hunting): get_auth_request con 0x7f4f340a6c30 auth_method 0 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f33577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f33577640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f33577640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient(hunting): handle_auth_done global_id 284364 payload 824 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 -- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6850 msgr2=0x7f4f34134de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6850 0x7f4f34134de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 -- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f340a6c30 msgr2=0x7f4f34135320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f340a6c30 0x7f4f34135320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 monclient: found mon.noname-a 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.555 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.540+0000 7f4f32d76640 1 -- 10.20.193.78:0/2802967408 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f340a1c10 con 0x7f4f340a6470 2026-02-17T06:11:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.542+0000 7f4f32d76640 1 -- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f340a6470 msgr2=0x7f4f341cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.542+0000 7f4f32d76640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f340a6470 0x7f4f341cf250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009970 tx=0x7f4f28004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.556 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.542+0000 7f4f137fe640 1 -- 10.20.193.78:0/2802967408 DispatchQueue::entry inject delay of 0.002000 2026-02-17T06:11:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 1 -- 10.20.193.78:0/2802967408 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2803c040 con 0x7f4f340a6470 2026-02-17T06:11:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:08.558 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: dump: 2026-02-17T06:11:08.558 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.558 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.559 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 1 -- 10.20.193.78:0/2802967408 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f28047070 con 0x7f4f340a6470 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 1 -- 10.20.193.78:0/2802967408 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28042890 con 0x7f4f340a6470 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 10 monclient: dump: 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.560 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f31573640 4 set_mon_vals no callback set 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 5 monclient: authenticate success, global_id 284364 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 10 monclient: _renew_subs 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f34138810 con 0x7f4f340a6470 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 10 monclient: _renew_subs 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f341335d0 con 0x7f4f340a6470 2026-02-17T06:11:08.561 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f34074980 con 0x7f4f340a6470 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.545+0000 7f4f137fe640 1 -- 10.20.193.78:0/2802967408 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f4f28042a30 con 0x7f4f340a6470 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.546+0000 7f4f137fe640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f0003df00 0x7f4f000403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.546+0000 7f4f137fe640 1 -- 10.20.193.78:0/2802967408 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f4f28077a10 con 0x7f4f340a6470 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.546+0000 7f4f32575640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f0003df00 0x7f4f000403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.546+0000 7f4f32575640 10 monclient: get_auth_request con 0x7f4f0003df00 auth_method 0 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.548+0000 7f4f137fe640 1 -- 10.20.193.78:0/2802967408 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4f340a0730 con 0x7f4f340a6470 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.548+0000 7f4f137fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.548+0000 7f4f137fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:08.562 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.548+0000 7f4f32575640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f0003df00 0x7f4f000403c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4f34131200 tx=0x7f4f1c007a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.630+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f4f340a1480 con 0x7f4f0003df00 2026-02-17T06:11:08.630 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.630+0000 7f4f137fe640 1 -- 10.20.193.78:0/2802967408 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4f340a1480 con 0x7f4f0003df00 2026-02-17T06:11:08.630 INFO:teuthology.orchestra.run.trial078.stderr:instructed osd(s) 1 to deep-scrub 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f0003df00 msgr2=0x7f4f000403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f0003df00 0x7f4f000403c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4f34131200 tx=0x7f4f1c007a90 comp rx=0 tx=0).stop 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 10 monclient: shutdown 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f340a6470 msgr2=0x7f4f341cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f340a6470 0x7f4f341cf250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009970 tx=0x7f4f28004830 comp rx=0 tx=0).stop 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 shutdown_connections 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f4f0003df00 0x7f4f000403c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.632 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f4f340a6c30 0x7f4f34135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f4f340a6850 0x7f4f34134de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 --2- 10.20.193.78:0/2802967408 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f4f340a6470 0x7f4f341cf250 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 >> 10.20.193.78:0/2802967408 conn(0x7f4f340aae40 msgr2=0x7f4f340a97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 shutdown_connections 2026-02-17T06:11:08.633 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.631+0000 7f4f39287640 1 -- 10.20.193.78:0/2802967408 wait complete. 2026-02-17T06:11:08.638 INFO:tasks.ceph:Scrubbing osd.2 2026-02-17T06:11:08.639 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.708+0000 7f3b16c66640 10 monclient: get_monmap_and_config 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.708+0000 7f3b16c66640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.708+0000 7f3b16c66640 10 monclient: monmap: 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:08.709150+0000 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:08.709150+0000 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.708 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:08.709 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:08.709 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:08.709 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:08.709 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:08.709 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 1 Processor -- start 2026-02-17T06:11:08.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 1 -- start start 2026-02-17T06:11:08.709 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient: init 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3b10134df0 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f3b101358e0 0x7f3b100a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient(hunting): picked mon.noname-a con 0x7f3b101358e0 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1013c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient(hunting): picked mon.noname-b con 0x7f3b1013a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b100a26d0 con 0x7f3b10134df0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f3b1009f480 con 0x7f3b101358e0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1009f760 con 0x7f3b1013a240 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b16c66640 10 monclient(hunting): authenticate will time out at 2114.262689s 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b149db640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f3b10134df0 0x7f3b101351d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33130/0 (socket says 10.20.193.78:33130) 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b149db640 1 -- 10.20.193.78:0/2463687158 learned_addr learned my addr 10.20.193.78:0/2463687158 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 1 --2- 10.20.193.78:0/2463687158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b07fff640 1 --2- 10.20.193.78:0/2463687158 >> v2:10.20.193.78:3300/0 conn(0x7f3b101358e0 0x7f3b100a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient(hunting): get_auth_request con 0x7f3b1013a240 auth_method 0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b07fff640 10 monclient(hunting): get_auth_request con 0x7f3b101358e0 auth_method 0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b07fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient(hunting): handle_auth_done global_id 264575 payload 274 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 1 -- 10.20.193.78:0/2463687158 >> v2:10.20.193.78:3300/0 conn(0x7f3b101358e0 msgr2=0x7f3b100a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 1 --2- 10.20.193.78:0/2463687158 >> v2:10.20.193.78:3300/0 conn(0x7f3b101358e0 0x7f3b100a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 1 -- 10.20.193.78:0/2463687158 >> v1:10.20.193.78:6789/0 conn(0x7f3b10134df0 legacy=0x7f3b101351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.709+0000 7f3b151dc640 1 monclient: found mon.noname-b 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.710+0000 7f3b151dc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.710+0000 7f3b151dc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.710+0000 7f3b151dc640 1 -- 10.20.193.78:0/2463687158 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1009fa40 con 0x7f3b1013a240 2026-02-17T06:11:08.710 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.710+0000 7f3b07fff640 1 --2- 10.20.193.78:0/2463687158 >> v2:10.20.193.78:3300/0 conn(0x7f3b101358e0 0x7f3b100a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b151dc640 1 -- 10.20.193.78:0/2463687158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 msgr2=0x7f3b1013c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b151dc640 1 --2- 10.20.193.78:0/2463687158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1013c630 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b100743b0 tx=0x7f3b0c02f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07d842fbeef881a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 1 -- 10.20.193.78:0/2463687158 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c03c070 con 0x7f3b1013a240 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: dump: 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 1 -- 10.20.193.78:0/2463687158 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0c037440 con 0x7f3b1013a240 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b16c66640 5 monclient: authenticate success, global_id 264575 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b16c66640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 1 -- 10.20.193.78:0/2463687158 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c037810 con 0x7f3b1013a240 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b077fe640 10 monclient: dump: 2026-02-17T06:11:08.712 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b16c66640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:08.713 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b16c66640 4 set_mon_vals no callback set 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b16c66640 10 monclient: shutdown 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b16c66640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b16c66640 1 -- 10.20.193.78:0/2463687158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 msgr2=0x7f3b1013c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.712+0000 7f3b16c66640 1 --2- 10.20.193.78:0/2463687158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1013c630 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b100743b0 tx=0x7f3b0c02f130 comp rx=0 tx=0).stop 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 1 -- 10.20.193.78:0/2463687158 shutdown_connections 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 1 --2- 10.20.193.78:0/2463687158 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1013c630 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 1 --2- 10.20.193.78:0/2463687158 >> v2:10.20.193.78:3300/0 conn(0x7f3b101358e0 0x7f3b100a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 1 -- 10.20.193.78:0/2463687158 >> 10.20.193.78:0/2463687158 conn(0x7f3b100a3300 msgr2=0x7f3b100a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 1 -- 10.20.193.78:0/2463687158 shutdown_connections 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 1 -- 10.20.193.78:0/2463687158 wait complete. 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 10 monclient: monmap: 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:08.714 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 1 Processor -- start 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 1 -- start start 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.713+0000 7f3b16c66640 10 monclient: init 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b10134df0 0x7f3b1006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient(hunting): picked mon.noname-b con 0x7f3b10134df0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b101358e0 0x7f3b1006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient(hunting): picked mon.noname-a con 0x7f3b101358e0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1006f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient(hunting): picked mon.noname-c con 0x7f3b1013a240 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1009d220 con 0x7f3b101358e0 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1009d0a0 con 0x7f3b10134df0 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f3b100a1020 con 0x7f3b1013a240 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b16c66640 10 monclient(hunting): authenticate will time out at 2114.267292s 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b101358e0 0x7f3b1006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b101358e0 0x7f3b1006eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52288/0 (socket says 10.20.193.78:52288) 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 -- 10.20.193.78:0/3214132745 learned_addr learned my addr 10.20.193.78:0/3214132745 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient(hunting): get_auth_request con 0x7f3b101358e0 auth_method 0 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1006f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 10 monclient(hunting): get_auth_request con 0x7f3b1013a240 auth_method 0 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b149db640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b10134df0 0x7f3b1006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b149db640 10 monclient(hunting): get_auth_request con 0x7f3b10134df0 auth_method 0 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b149db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b149db640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.715 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b149db640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient(hunting): handle_auth_done global_id 284370 payload 824 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 -- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 msgr2=0x7f3b1006f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1006f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 -- 10.20.193.78:0/3214132745 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b10134df0 msgr2=0x7f3b1006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f3b10134df0 0x7f3b1006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 monclient: found mon.noname-a 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b07fff640 1 -- 10.20.193.78:0/3214132745 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b10072080 con 0x7f3b101358e0 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.714+0000 7f3b151dc640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f3b1013a240 0x7f3b1006f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b07fff640 1 -- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b101358e0 msgr2=0x7f3b1006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b07fff640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b101358e0 0x7f3b1006eff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3af800e710 tx=0x7f3af800ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af801d070 con 0x7f3b101358e0 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.716 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: dump: 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af800dbc0 con 0x7f3b101358e0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b16c66640 5 monclient: authenticate success, global_id 284370 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b16c66640 10 monclient: _renew_subs 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b16c66640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b10072360 con 0x7f3b101358e0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8018410 con 0x7f3b101358e0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b057fa640 10 monclient: dump: 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b06ffd640 4 set_mon_vals no callback set 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b16c66640 10 monclient: _renew_subs 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b16c66640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.716+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b100a4420 con 0x7f3b101358e0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b16c66640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.717 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b10073ed0 con 0x7f3b101358e0 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f3af800c040 con 0x7f3b101358e0 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b057fa640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae003deb0 0x7f3ae0040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f3af8055030 con 0x7f3b101358e0 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b057fa640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f3ae0043d50 0x7f3ae0046170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 --> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ae0046820 con 0x7f3ae0043d50 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=867) ==== 24+0+0 (secure 0 0 0) 0x7f3af8057600 con 0x7f3b101358e0 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b057fa640 10 monclient: handle_get_version_reply finishing 1 version 867 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b149db640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae003deb0 0x7f3ae0040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b149db640 10 monclient: get_auth_request con 0x7f3ae003deb0 auth_method 0 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b151dc640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f3ae0043d50 0x7f3ae0046170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.718 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.717+0000 7f3b151dc640 10 monclient: get_auth_request con 0x7f3ae0043d50 auth_method 0 2026-02-17T06:11:08.719 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.720+0000 7f3b149db640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae003deb0 0x7f3ae0040370 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3b00007340 tx=0x7f3b00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.721 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.722+0000 7f3b151dc640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f3ae0043d50 0x7f3ae0046170 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.722 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.722+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 <== osd.2 v2:10.20.193.78:6816/707008561 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f3ae0046820 con 0x7f3ae0043d50 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 --> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3b100a0140 con 0x7f3ae0043d50 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b057fa640 1 -- 10.20.193.78:0/3214132745 <== osd.2 v2:10.20.193.78:6816/707008561 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3b100a0140 con 0x7f3ae0043d50 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f3ae0043d50 msgr2=0x7f3ae0046170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:6816/707008561,v1:10.20.193.78:6817/707008561] conn(0x7f3ae0043d50 0x7f3ae0046170 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae003deb0 msgr2=0x7f3ae0040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f3ae003deb0 0x7f3ae0040370 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3b00007340 tx=0x7f3b00009040 comp rx=0 tx=0).stop 2026-02-17T06:11:08.728 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 10 monclient: shutdown 2026-02-17T06:11:08.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b101358e0 msgr2=0x7f3b1006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 --2- 10.20.193.78:0/3214132745 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f3b101358e0 0x7f3b1006eff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3af800e710 tx=0x7f3af800ebe0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b151dc640 1 -- 10.20.193.78:0/3214132745 reap_dead start 2026-02-17T06:11:08.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 shutdown_connections 2026-02-17T06:11:08.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.728+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 >> 10.20.193.78:0/3214132745 conn(0x7f3b100a3300 msgr2=0x7f3b10139750 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.729+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 shutdown_connections 2026-02-17T06:11:08.729 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.729+0000 7f3b16c66640 1 -- 10.20.193.78:0/3214132745 wait complete. 2026-02-17T06:11:08.734 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-17T06:11:08.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.801+0000 7f53f1585640 10 monclient: get_monmap_and_config 2026-02-17T06:11:08.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.801+0000 7f53f1585640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.801+0000 7f53f1585640 10 monclient: monmap: 2026-02-17T06:11:08.804 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.804 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:08.801998+0000 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:08.801998+0000 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 1 Processor -- start 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 1 -- start start 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient: init 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f53ec06c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f53ec09d430 0x7f53ec13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient(hunting): picked mon.noname-c con 0x7f53ec09d430 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f53ec09bf40 0x7f53ec13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient(hunting): picked mon.noname-a con 0x7f53ec09bf40 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:08.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53ec073600 con 0x7f53ec06c890 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f53ec09ca50 con 0x7f53ec09bf40 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f53ec06dab0 con 0x7f53ec09d430 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53f1585640 10 monclient(hunting): authenticate will time out at 2114.355531s 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f53ec09bf40 0x7f53ec13fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f53ec09bf40 0x7f53ec13fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52290/0 (socket says 10.20.193.78:52290) 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 1 -- 10.20.193.78:0/4202482649 learned_addr learned my addr 10.20.193.78:0/4202482649 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient(hunting): get_auth_request con 0x7f53ec09bf40 auth_method 0 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53ea7fc640 1 --2- 10.20.193.78:0/4202482649 >> v2:10.20.193.115:3300/0 conn(0x7f53ec09d430 0x7f53ec13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53ea7fc640 10 monclient(hunting): get_auth_request con 0x7f53ec09d430 auth_method 0 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53ea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53ea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient(hunting): handle_auth_done global_id 284373 payload 274 2026-02-17T06:11:08.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 1 -- 10.20.193.78:0/4202482649 >> v2:10.20.193.115:3300/0 conn(0x7f53ec09d430 msgr2=0x7f53ec13d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 1 --2- 10.20.193.78:0/4202482649 >> v2:10.20.193.115:3300/0 conn(0x7f53ec09d430 0x7f53ec13d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 1 -- 10.20.193.78:0/4202482649 >> v1:10.20.193.115:6789/0 conn(0x7f53ec06c890 legacy=0x7f53ec06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 1 monclient: found mon.noname-a 2026-02-17T06:11:08.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.802+0000 7f53eb7fe640 1 -- 10.20.193.78:0/4202482649 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec09c770 con 0x7f53ec09bf40 2026-02-17T06:11:08.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53eb7fe640 1 -- 10.20.193.78:0/4202482649 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09bf40 msgr2=0x7f53ec13fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53eb7fe640 1 --2- 10.20.193.78:0/4202482649 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09bf40 0x7f53ec13fb20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0a0ac0 tx=0x7f53dc02f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a8cb42f0d1d35c server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 1 -- 10.20.193.78:0/4202482649 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc02fe60 con 0x7f53ec09bf40 2026-02-17T06:11:08.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: dump: 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.817 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 1 -- 10.20.193.78:0/4202482649 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc0373f0 con 0x7f53ec09bf40 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 5 monclient: authenticate success, global_id 284373 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 1 -- 10.20.193.78:0/4202482649 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0377c0 con 0x7f53ec09bf40 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 4 set_mon_vals no callback set 2026-02-17T06:11:08.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:08.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53e9ffb640 10 monclient: dump: 2026-02-17T06:11:08.819 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.819 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.819 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.819 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.819 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.819 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.820 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.820 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.820 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.820 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 10 monclient: shutdown 2026-02-17T06:11:08.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 1 -- 10.20.193.78:0/4202482649 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09bf40 msgr2=0x7f53ec13fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 1 --2- 10.20.193.78:0/4202482649 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09bf40 0x7f53ec13fb20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0a0ac0 tx=0x7f53dc02f230 comp rx=0 tx=0).stop 2026-02-17T06:11:08.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 1 -- 10.20.193.78:0/4202482649 shutdown_connections 2026-02-17T06:11:08.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 1 --2- 10.20.193.78:0/4202482649 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09bf40 0x7f53ec13fb20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 1 --2- 10.20.193.78:0/4202482649 >> v2:10.20.193.115:3300/0 conn(0x7f53ec09d430 0x7f53ec13d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 1 -- 10.20.193.78:0/4202482649 >> 10.20.193.78:0/4202482649 conn(0x7f53ec0a6620 msgr2=0x7f53ec0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 1 -- 10.20.193.78:0/4202482649 shutdown_connections 2026-02-17T06:11:08.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.805+0000 7f53f1585640 1 -- 10.20.193.78:0/4202482649 wait complete. 2026-02-17T06:11:08.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient: monmap: 2026-02-17T06:11:08.822 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.822 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:08.822 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:08.822 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:08.823 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.823 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.823 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:08.823 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:08.823 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:08.823 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 1 Processor -- start 2026-02-17T06:11:08.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 1 -- start start 2026-02-17T06:11:08.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient: init 2026-02-17T06:11:08.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-17T06:11:08.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f53ec06c890 0x7f53ec13ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient(hunting): picked mon.noname-b con 0x7f53ec06c890 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f53ec09bf40 0x7f53ec135710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient(hunting): picked mon.noname-c con 0x7f53ec09bf40 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09d430 0x7f53ec11cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient(hunting): picked mon.noname-a con 0x7f53ec09d430 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec06d380 con 0x7f53ec09d430 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec073410 con 0x7f53ec06c890 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f53ec06d500 con 0x7f53ec09bf40 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53f1585640 10 monclient(hunting): authenticate will time out at 2114.359681s 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09d430 0x7f53ec11cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f53ec09bf40 0x7f53ec135710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09d430 0x7f53ec11cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52298/0 (socket says 10.20.193.78:52298) 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 1 -- 10.20.193.78:0/4138193638 learned_addr learned my addr 10.20.193.78:0/4138193638 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient(hunting): get_auth_request con 0x7f53ec09bf40 auth_method 0 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 10 monclient(hunting): get_auth_request con 0x7f53ec09d430 auth_method 0 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eaffd640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f53ec06c890 0x7f53ec13ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eaffd640 10 monclient(hunting): get_auth_request con 0x7f53ec06c890 auth_method 0 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient(hunting): handle_auth_done global_id 264581 payload 824 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 1 -- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f53ec06c890 msgr2=0x7f53ec13ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f53ec06c890 0x7f53ec13ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 1 -- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09d430 msgr2=0x7f53ec11cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09d430 0x7f53ec11cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 1 monclient: found mon.noname-c 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53ea7fc640 1 -- 10.20.193.78:0/4138193638 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec11fdc0 con 0x7f53ec09bf40 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.806+0000 7f53eb7fe640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09d430 0x7f53ec11cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:08.828 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53ea7fc640 1 -- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f53ec09bf40 msgr2=0x7f53ec135710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53ea7fc640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f53ec09bf40 0x7f53ec135710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f53d400e810 tx=0x7f53d400ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 1 -- 10.20.193.78:0/4138193638 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4017070 con 0x7f53ec09bf40 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: dump: 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.829 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 1 -- 10.20.193.78:0/4138193638 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d400cae0 con 0x7f53ec09bf40 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 1 -- 10.20.193.78:0/4138193638 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400f040 con 0x7f53ec09bf40 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 10 monclient: dump: 2026-02-17T06:11:08.830 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53e97fa640 4 set_mon_vals no callback set 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 5 monclient: authenticate success, global_id 264581 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 10 monclient: _renew_subs 2026-02-17T06:11:08.831 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec0a22f0 con 0x7f53ec09bf40 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 10 monclient: _renew_subs 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec09d810 con 0x7f53ec09bf40 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ec074980 con 0x7f53ec09bf40 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 1 -- 10.20.193.78:0/4138193638 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f53d4012050 con 0x7f53ec09bf40 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f53b803dee0 0x7f53b80403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.809+0000 7f53cb7fe640 1 -- 10.20.193.78:0/4138193638 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f53d4052430 con 0x7f53ec09bf40 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.810+0000 7f53eaffd640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f53b803dee0 0x7f53b80403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.832 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.810+0000 7f53eaffd640 10 monclient: get_auth_request con 0x7f53b803dee0 auth_method 0 2026-02-17T06:11:08.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.812+0000 7f53cb7fe640 1 -- 10.20.193.78:0/4138193638 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f53d4023d80 con 0x7f53ec09bf40 2026-02-17T06:11:08.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.812+0000 7f53cb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:08.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.812+0000 7f53cb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:08.833 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.812+0000 7f53eaffd640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f53b803dee0 0x7f53b80403a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.894+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f53ec11ddb0 con 0x7f53b803dee0 2026-02-17T06:11:08.894 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.894+0000 7f53cb7fe640 1 -- 10.20.193.78:0/4138193638 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f53ec11ddb0 con 0x7f53b803dee0 2026-02-17T06:11:08.894 INFO:teuthology.orchestra.run.trial078.stderr:instructed osd(s) 2 to deep-scrub 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f53b803dee0 msgr2=0x7f53b80403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f53b803dee0 0x7f53b80403a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0008040 comp rx=0 tx=0).stop 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 10 monclient: shutdown 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f53ec09bf40 msgr2=0x7f53ec135710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f53ec09bf40 0x7f53ec135710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f53d400e810 tx=0x7f53d400ece0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 shutdown_connections 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f53ec09d430 0x7f53ec11cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.896 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f53ec09bf40 0x7f53ec135710 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f53b803dee0 0x7f53b80403a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 --2- 10.20.193.78:0/4138193638 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f53ec06c890 0x7f53ec13ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 >> 10.20.193.78:0/4138193638 conn(0x7f53ec0a6620 msgr2=0x7f53ec09f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 shutdown_connections 2026-02-17T06:11:08.897 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.895+0000 7f53f1585640 1 -- 10.20.193.78:0/4138193638 wait complete. 2026-02-17T06:11:08.903 INFO:tasks.ceph:Scrubbing osd.3 2026-02-17T06:11:08.903 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.973+0000 7f50ae41a640 10 monclient: get_monmap_and_config 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.973+0000 7f50ae41a640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.973+0000 7f50ae41a640 10 monclient: monmap: 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:08.974168+0000 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:08.974168+0000 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.974 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:08.975 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:08.975 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:08.975 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:08.975 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:08.975 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 1 Processor -- start 2026-02-17T06:11:08.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 1 -- start start 2026-02-17T06:11:08.975 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient: init 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f50a80a1020 0x7f50a80a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient(hunting): picked mon.noname-c con 0x7f50a80a1020 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f50a80a1940 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f50a8136510 0x7f50a813e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient(hunting): picked mon.noname-a con 0x7f50a8136510 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a809fed0 con 0x7f50a80a1940 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f50a80a2980 con 0x7f50a8136510 2026-02-17T06:11:08.980 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f50a80a0590 con 0x7f50a80a1020 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ae41a640 10 monclient(hunting): authenticate will time out at 2114.527736s 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ac990640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f50a8136510 0x7f50a813e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50a77fe640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f50a80a1940 0x7f50a8135e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33138/0 (socket says 10.20.193.78:33138) 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50a77fe640 1 -- 10.20.193.78:0/269629360 learned_addr learned my addr 10.20.193.78:0/269629360 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ac990640 10 monclient(hunting): get_auth_request con 0x7f50a8136510 auth_method 0 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ac990640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ac990640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ac990640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ac990640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ac990640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50a7fff640 1 --2- 10.20.193.78:0/269629360 >> v2:10.20.193.115:3300/0 conn(0x7f50a80a1020 0x7f50a80a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50ac990640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50a7fff640 10 monclient(hunting): get_auth_request con 0x7f50a80a1020 auth_method 0 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.974+0000 7f50a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 10 monclient(hunting): handle_auth_done global_id 284379 payload 274 2026-02-17T06:11:08.981 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 1 -- 10.20.193.78:0/269629360 >> v2:10.20.193.115:3300/0 conn(0x7f50a80a1020 msgr2=0x7f50a80a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 1 --2- 10.20.193.78:0/269629360 >> v2:10.20.193.115:3300/0 conn(0x7f50a80a1020 0x7f50a80a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 1 -- 10.20.193.78:0/269629360 >> v1:10.20.193.78:6789/0 conn(0x7f50a80a1940 legacy=0x7f50a8135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 1 monclient: found mon.noname-a 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.975+0000 7f50ac990640 1 -- 10.20.193.78:0/269629360 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a809d0a0 con 0x7f50a8136510 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ac990640 1 -- 10.20.193.78:0/269629360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a8136510 msgr2=0x7f50a813e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ac990640 1 --2- 10.20.193.78:0/269629360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a8136510 0x7f50a813e270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50a806dc40 tx=0x7f509c02f250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507405d620878afb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 1 -- 10.20.193.78:0/269629360 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c03c070 con 0x7f50a8136510 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: dump: 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.982 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 1 -- 10.20.193.78:0/269629360 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509c037640 con 0x7f50a8136510 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ae41a640 5 monclient: authenticate success, global_id 284379 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ae41a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 1 -- 10.20.193.78:0/269629360 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c037a10 con 0x7f50a8136510 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ae41a640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ae41a640 4 set_mon_vals no callback set 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50a6ffd640 10 monclient: dump: 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.983 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ae41a640 10 monclient: shutdown 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ae41a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ae41a640 1 -- 10.20.193.78:0/269629360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a8136510 msgr2=0x7f50a813e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.977+0000 7f50ae41a640 1 --2- 10.20.193.78:0/269629360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a8136510 0x7f50a813e270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50a806dc40 tx=0x7f509c02f250 comp rx=0 tx=0).stop 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 1 -- 10.20.193.78:0/269629360 shutdown_connections 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 1 --2- 10.20.193.78:0/269629360 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a8136510 0x7f50a813e270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 1 --2- 10.20.193.78:0/269629360 >> v2:10.20.193.115:3300/0 conn(0x7f50a80a1020 0x7f50a80a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 1 -- 10.20.193.78:0/269629360 >> 10.20.193.78:0/269629360 conn(0x7f50a80a3300 msgr2=0x7f50a80a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 1 -- 10.20.193.78:0/269629360 shutdown_connections 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 1 -- 10.20.193.78:0/269629360 wait complete. 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 10 monclient: build_initial_monmap 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 10 monclient: monmap: 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:08.984 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 1 Processor -- start 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 1 -- start start 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.978+0000 7f50ae41a640 10 monclient: init 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a80a1020 0x7f50a81cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient(hunting): picked mon.noname-a con 0x7f50a80a1020 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a80a1940 0x7f50a8134f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient(hunting): picked mon.noname-c con 0x7f50a80a1940 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a8136510 0x7f50a81354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient(hunting): picked mon.noname-b con 0x7f50a8136510 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50a80a1d50 con 0x7f50a80a1020 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50a80a1ed0 con 0x7f50a8136510 2026-02-17T06:11:08.985 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f50a80a0180 con 0x7f50a80a1940 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ae41a640 10 monclient(hunting): authenticate will time out at 2114.532076s 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a80a1020 0x7f50a81cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a80a1020 0x7f50a81cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52322/0 (socket says 10.20.193.78:52322) 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 -- 10.20.193.78:0/1413074292 learned_addr learned my addr 10.20.193.78:0/1413074292 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient(hunting): get_auth_request con 0x7f50a80a1020 auth_method 0 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a77fe640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a80a1940 0x7f50a8134f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a77fe640 10 monclient(hunting): get_auth_request con 0x7f50a80a1940 auth_method 0 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ac990640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a8136510 0x7f50a81354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:08.986 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ac990640 10 monclient(hunting): get_auth_request con 0x7f50a8136510 auth_method 0 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ac990640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ac990640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50ac990640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient(hunting): handle_auth_done global_id 284382 payload 824 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 -- 10.20.193.78:0/1413074292 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a80a1940 msgr2=0x7f50a8134f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f50a80a1940 0x7f50a8134f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 -- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a8136510 msgr2=0x7f50a81354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f50a8136510 0x7f50a81354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient: _finish_hunting 0 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 monclient: found mon.noname-a 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.987 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.979+0000 7f50a7fff640 1 -- 10.20.193.78:0/1413074292 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a809ca00 con 0x7f50a80a1020 2026-02-17T06:11:08.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a7fff640 1 -- 10.20.193.78:0/1413074292 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a80a1020 msgr2=0x7f50a81cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:08.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a7fff640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a80a1020 0x7f50a81cf1e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f509400e7c0 tx=0x7f509400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094017070 con 0x7f50a80a1020 2026-02-17T06:11:08.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:08.989 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: dump: 2026-02-17T06:11:08.989 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: _finish_auth 0 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: _check_auth_tickets 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509400cae0 con 0x7f50a80a1020 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 5 monclient: authenticate success, global_id 284382 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 10 monclient: _renew_subs 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a8072f60 con 0x7f50a80a1020 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509400f040 con 0x7f50a80a1020 2026-02-17T06:11:08.990 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a67fc640 4 set_mon_vals no callback set 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50a4ff9640 10 monclient: dump: 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 10 monclient: _renew_subs 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a81283c0 con 0x7f50a80a1020 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.981+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50a8136920 con 0x7f50a80a1020 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f5094012050 con 0x7f50a80a1020 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a4ff9640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f507803deb0 0x7f5078040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.991 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f5094052860 con 0x7f50a80a1020 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a4ff9640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f5078043d50 0x7f5078046170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 --> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5094011ea0 con 0x7f5078043d50 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=867) ==== 24+0+0 (secure 0 0 0) 0x7f5094052b40 con 0x7f50a80a1020 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a4ff9640 10 monclient: handle_get_version_reply finishing 1 version 867 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a77fe640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f507803deb0 0x7f5078040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50a77fe640 10 monclient: get_auth_request con 0x7f507803deb0 auth_method 0 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50ac990640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f5078043d50 0x7f5078046170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.982+0000 7f50ac990640 10 monclient: get_auth_request con 0x7f5078043d50 auth_method 0 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.985+0000 7f50a77fe640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f507803deb0 0x7f5078040370 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5098007d90 tx=0x7f5098007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.987+0000 7f50ac990640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f5078043d50 0x7f5078046170 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:08.992 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.988+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 <== osd.3 v2:10.20.193.115:6810/2268846355 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f5094011ea0 con 0x7f5078043d50 2026-02-17T06:11:08.993 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 --> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f50a8128950 con 0x7f5078043d50 2026-02-17T06:11:08.994 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50a4ff9640 1 -- 10.20.193.78:0/1413074292 <== osd.3 v2:10.20.193.115:6810/2268846355 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f50a8128950 con 0x7f5078043d50 2026-02-17T06:11:08.994 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:08.994 INFO:teuthology.orchestra.run.trial078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:11:08.994 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f5078043d50 msgr2=0x7f5078046170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6810/2268846355,v1:10.20.193.115:6812/2268846355] conn(0x7f5078043d50 0x7f5078046170 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f507803deb0 msgr2=0x7f5078040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f507803deb0 0x7f5078040370 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5098007d90 tx=0x7f5098007ce0 comp rx=0 tx=0).stop 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 10 monclient: shutdown 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a80a1020 msgr2=0x7f50a81cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 --2- 10.20.193.78:0/1413074292 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f50a80a1020 0x7f50a81cf1e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f509400e7c0 tx=0x7f509400ec90 comp rx=0 tx=0).stop 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ac990640 1 -- 10.20.193.78:0/1413074292 reap_dead start 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 shutdown_connections 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 >> 10.20.193.78:0/1413074292 conn(0x7f50a80a3300 msgr2=0x7f50a80a5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 shutdown_connections 2026-02-17T06:11:08.995 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:08.994+0000 7f50ae41a640 1 -- 10.20.193.78:0/1413074292 wait complete. 2026-02-17T06:11:09.001 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-17T06:11:09.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.067+0000 7fbedeb49640 10 monclient: get_monmap_and_config 2026-02-17T06:11:09.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.067+0000 7fbedeb49640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.067 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.067+0000 7fbedeb49640 10 monclient: monmap: 2026-02-17T06:11:09.067 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.067 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.067 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:09.067790+0000 2026-02-17T06:11:09.067 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:09.067790+0000 2026-02-17T06:11:09.068 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.068 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.068 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:09.068 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:09.068 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:09.068 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:09.068 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:09.068 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.069 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.067+0000 7fbedeb49640 1 Processor -- start 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 1 -- start start 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient: init 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbed806c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fbed809c4a0 0x7fbed813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient(hunting): picked mon.noname-c con 0x7fbed809c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbed806d380 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbed806daf0 con 0x7fbed806d380 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbed809cc80 con 0x7fbed806c890 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fbed80a0880 con 0x7fbed809c4a0 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedeb49640 10 monclient(hunting): authenticate will time out at 2114.621429s 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedd0bf640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7fbed806d380 0x7fbed813d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33142/0 (socket says 10.20.193.78:33142) 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbedd0bf640 1 -- 10.20.193.78:0/3821196526 learned_addr learned my addr 10.20.193.78:0/3821196526 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbecffff640 1 --2- 10.20.193.78:0/3821196526 >> v2:10.20.193.115:3300/0 conn(0x7fbed809c4a0 0x7fbed813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.070 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbecffff640 10 monclient(hunting): get_auth_request con 0x7fbed809c4a0 auth_method 0 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbecffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbecffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.068+0000 7fbecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.069+0000 7fbecffff640 10 monclient(hunting): handle_auth_done global_id 304162 payload 274 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.069+0000 7fbecffff640 1 -- 10.20.193.78:0/3821196526 >> v1:10.20.193.115:6789/0 conn(0x7fbed806c890 legacy=0x7fbed806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.069+0000 7fbecffff640 1 -- 10.20.193.78:0/3821196526 >> v1:10.20.193.78:6789/0 conn(0x7fbed806d380 legacy=0x7fbed813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.069+0000 7fbecffff640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.069+0000 7fbecffff640 1 monclient: found mon.noname-c 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.069+0000 7fbecffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.069+0000 7fbecffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:11:09.071 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.069+0000 7fbecffff640 1 -- 10.20.193.78:0/3821196526 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed809cf60 con 0x7fbed809c4a0 2026-02-17T06:11:09.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecffff640 1 -- 10.20.193.78:0/3821196526 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 msgr2=0x7fbed813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecffff640 1 --2- 10.20.193.78:0/3821196526 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 0x7fbed813b3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbed809c980 tx=0x7fbec002ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9df049b39d667d5 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 1 -- 10.20.193.78:0/3821196526 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec003c070 con 0x7fbed809c4a0 2026-02-17T06:11:09.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:09.074 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: dump: 2026-02-17T06:11:09.074 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.074 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbedeb49640 5 monclient: authenticate success, global_id 304162 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbedeb49640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 1 -- 10.20.193.78:0/3821196526 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec002fcb0 con 0x7fbed809c4a0 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 1 -- 10.20.193.78:0/3821196526 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec002fe60 con 0x7fbed809c4a0 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbedeb49640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.075 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbedeb49640 4 set_mon_vals no callback set 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.071+0000 7fbecf7fe640 10 monclient: dump: 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 10 monclient: shutdown 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 -- 10.20.193.78:0/3821196526 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 msgr2=0x7fbed813b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 --2- 10.20.193.78:0/3821196526 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 0x7fbed813b3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbed809c980 tx=0x7fbec002ed50 comp rx=0 tx=0).stop 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 -- 10.20.193.78:0/3821196526 shutdown_connections 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 --2- 10.20.193.78:0/3821196526 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 0x7fbed813b3a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 -- 10.20.193.78:0/3821196526 >> 10.20.193.78:0/3821196526 conn(0x7fbed80a6620 msgr2=0x7fbed80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.076 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 -- 10.20.193.78:0/3821196526 shutdown_connections 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 -- 10.20.193.78:0/3821196526 wait complete. 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 10 monclient: monmap: 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 Processor -- start 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 1 -- start start 2026-02-17T06:11:09.077 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.072+0000 7fbedeb49640 10 monclient: init 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbed806c890 0x7fbed813d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient(hunting): picked mon.noname-a con 0x7fbed806c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbed806d380 0x7fbed81362c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient(hunting): picked mon.noname-c con 0x7fbed806d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 0x7fbed81cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient(hunting): picked mon.noname-b con 0x7fbed809c4a0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbed809bfa0 con 0x7fbed806c890 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbed809be20 con 0x7fbed809c4a0 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fbed813f010 con 0x7fbed806d380 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedeb49640 10 monclient(hunting): authenticate will time out at 2114.626089s 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbed806c890 0x7fbed813d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbed806c890 0x7fbed813d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52328/0 (socket says 10.20.193.78:52328) 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 -- 10.20.193.78:0/1428376693 learned_addr learned my addr 10.20.193.78:0/1428376693 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.078 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient(hunting): get_auth_request con 0x7fbed806c890 auth_method 0 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbecffff640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbed806d380 0x7fbed81362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbecffff640 10 monclient(hunting): get_auth_request con 0x7fbed806d380 auth_method 0 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbecffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbecffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedd0bf640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 0x7fbed81cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedd0bf640 10 monclient(hunting): get_auth_request con 0x7fbed809c4a0 auth_method 0 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedd0bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedd0bf640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedd0bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient(hunting): handle_auth_done global_id 284385 payload 824 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 -- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbed806d380 msgr2=0x7fbed81362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.079 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbed806d380 0x7fbed81362c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 -- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 msgr2=0x7fbed81cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 0x7fbed81cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 monclient: found mon.noname-a 2026-02-17T06:11:09.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.080 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.073+0000 7fbedc8be640 1 -- 10.20.193.78:0/1428376693 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed811e810 con 0x7fbed806c890 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedc8be640 1 -- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbed806c890 msgr2=0x7fbed813d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedc8be640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbed806c890 0x7fbed813d440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbec800c5d0 tx=0x7fbec800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 1 -- 10.20.193.78:0/1428376693 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8016020 con 0x7fbed806c890 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: dump: 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.082 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 1 -- 10.20.193.78:0/1428376693 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec8012a60 con 0x7fbed806c890 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedeb49640 5 monclient: authenticate success, global_id 284385 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 1 -- 10.20.193.78:0/1428376693 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8012e50 con 0x7fbed806c890 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbeceffd640 4 set_mon_vals no callback set 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbecd7fa640 10 monclient: dump: 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.083 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedeb49640 10 monclient: _renew_subs 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedeb49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed811fc60 con 0x7fbed806c890 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedeb49640 10 monclient: _renew_subs 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedeb49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.075+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed809d810 con 0x7fbed806c890 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbedeb49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbedeb49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbedeb49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed806cc70 con 0x7fbed806c890 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbecd7fa640 1 -- 10.20.193.78:0/1428376693 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fbec801b400 con 0x7fbed806c890 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbecd7fa640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbea803e050 0x7fbea8040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbecd7fa640 1 -- 10.20.193.78:0/1428376693 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fbec8052a60 con 0x7fbed806c890 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbecffff640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbea803e050 0x7fbea8040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.076+0000 7fbecffff640 10 monclient: get_auth_request con 0x7fbea803e050 auth_method 0 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.078+0000 7fbecd7fa640 1 -- 10.20.193.78:0/1428376693 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbec8020020 con 0x7fbed806c890 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.078+0000 7fbecd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.078+0000 7fbecd7fa640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:09.084 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.079+0000 7fbecffff640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbea803e050 0x7fbea8040510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbed8074de0 tx=0x7fbec0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.161+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fbed80744d0 con 0x7fbea803e050 2026-02-17T06:11:09.161 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.161+0000 7fbecd7fa640 1 -- 10.20.193.78:0/1428376693 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbed80744d0 con 0x7fbea803e050 2026-02-17T06:11:09.161 INFO:teuthology.orchestra.run.trial078.stderr:instructed osd(s) 3 to deep-scrub 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbea803e050 msgr2=0x7fbea8040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbea803e050 0x7fbea8040510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbed8074de0 tx=0x7fbec0033000 comp rx=0 tx=0).stop 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 10 monclient: shutdown 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbed806c890 msgr2=0x7fbed813d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbed806c890 0x7fbed813d440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbec800c5d0 tx=0x7fbec800caa0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 shutdown_connections 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fbea803e050 0x7fbea8040510 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fbed809c4a0 0x7fbed81cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.162 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fbed806d380 0x7fbed81362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 --2- 10.20.193.78:0/1428376693 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fbed806c890 0x7fbed813d440 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 >> 10.20.193.78:0/1428376693 conn(0x7fbed80a6620 msgr2=0x7fbed80a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 shutdown_connections 2026-02-17T06:11:09.163 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.162+0000 7fbedeb49640 1 -- 10.20.193.78:0/1428376693 wait complete. 2026-02-17T06:11:09.169 INFO:tasks.ceph:Scrubbing osd.4 2026-02-17T06:11:09.170 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:11:09.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.239+0000 7f56a1987640 10 monclient: get_monmap_and_config 2026-02-17T06:11:09.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.239+0000 7f56a1987640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.251 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.239+0000 7f56a1987640 10 monclient: monmap: 2026-02-17T06:11:09.251 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.251 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:09.239384+0000 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:09.239384+0000 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.239+0000 7f56a1987640 1 Processor -- start 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.239+0000 7f56a1987640 1 -- start start 2026-02-17T06:11:09.252 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.239+0000 7f56a1987640 10 monclient: init 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 10 monclient: _add_conns ranks=[3,0,1,2,4] 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f569c0a1020 0x7f569c0a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 10 monclient(hunting): picked mon.noname-a con 0x7f569c0a1020 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f569c0a1940 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f569c136510 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c09fed0 con 0x7f569c0a1940 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c0a2980 con 0x7f569c136510 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7f569c0a0590 con 0x7f569c0a1020 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f56a1987640 10 monclient(hunting): authenticate will time out at 2114.793231s 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f569c0a1020 0x7f569c0a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7f569c0a1020 0x7f569c0a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52336/0 (socket says 10.20.193.78:52336) 2026-02-17T06:11:09.253 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 1 -- 10.20.193.78:0/1835935640 learned_addr learned my addr 10.20.193.78:0/1835935640 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient(hunting): get_auth_request con 0x7f569c0a1020 auth_method 0 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient(hunting): handle_auth_done global_id 284388 payload 274 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 1 -- 10.20.193.78:0/1835935640 >> v1:10.20.193.115:6789/0 conn(0x7f569c136510 legacy=0x7f569c13e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 1 -- 10.20.193.78:0/1835935640 >> v1:10.20.193.78:6789/0 conn(0x7f569c0a1940 legacy=0x7f569c135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 1 monclient: found mon.noname-a 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.254 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.240+0000 7f569affd640 1 -- 10.20.193.78:0/1835935640 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c09d0a0 con 0x7f569c0a1020 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.242+0000 7f569affd640 1 -- 10.20.193.78:0/1835935640 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1020 msgr2=0x7f569c0a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.242+0000 7f569affd640 1 --2- 10.20.193.78:0/1835935640 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1020 0x7f569c0a1400 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5690009970 tx=0x7f569002efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e2bdda70b2fdeb server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.242+0000 7f5699ffb640 1 -- 10.20.193.78:0/1835935640 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569003c070 con 0x7f569c0a1020 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: dump: 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.255 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 1 -- 10.20.193.78:0/1835935640 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690037440 con 0x7f569c0a1020 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 5 monclient: authenticate success, global_id 284388 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 1 -- 10.20.193.78:0/1835935640 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690037810 con 0x7f569c0a1020 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f5699ffb640 10 monclient: dump: 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.256 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 4 set_mon_vals no callback set 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 10 monclient: shutdown 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 1 -- 10.20.193.78:0/1835935640 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1020 msgr2=0x7f569c0a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 1 --2- 10.20.193.78:0/1835935640 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1020 0x7f569c0a1400 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5690009970 tx=0x7f569002efb0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 1 -- 10.20.193.78:0/1835935640 shutdown_connections 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 1 --2- 10.20.193.78:0/1835935640 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1020 0x7f569c0a1400 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.257 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 1 -- 10.20.193.78:0/1835935640 >> 10.20.193.78:0/1835935640 conn(0x7f569c0a3300 msgr2=0x7f569c0a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 1 -- 10.20.193.78:0/1835935640 shutdown_connections 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.243+0000 7f56a1987640 1 -- 10.20.193.78:0/1835935640 wait complete. 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient: monmap: 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.258 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 1 Processor -- start 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 1 -- start start 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient: init 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f569c0a1020 0x7f569c135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient(hunting): picked mon.noname-c con 0x7f569c0a1020 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1940 0x7f569c135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient(hunting): picked mon.noname-a con 0x7f569c0a1940 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f569c136510 0x7f569c122bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient(hunting): picked mon.noname-b con 0x7f569c136510 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f569c0a1d50 con 0x7f569c0a1940 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f569c0a1ed0 con 0x7f569c136510 2026-02-17T06:11:09.259 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f569c0a0180 con 0x7f569c0a1020 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f56a1987640 10 monclient(hunting): authenticate will time out at 2114.797564s 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1940 0x7f569c135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1940 0x7f569c135640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52348/0 (socket says 10.20.193.78:52348) 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569affd640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f569c0a1020 0x7f569c135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 -- 10.20.193.78:0/505713752 learned_addr learned my addr 10.20.193.78:0/505713752 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient(hunting): get_auth_request con 0x7f569c0a1940 auth_method 0 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569affd640 10 monclient(hunting): get_auth_request con 0x7f569c0a1020 auth_method 0 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.260 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569affd640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569b7fe640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f569c136510 0x7f569c122bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569b7fe640 10 monclient(hunting): get_auth_request con 0x7f569c136510 auth_method 0 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.261 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient(hunting): handle_auth_done global_id 284391 payload 824 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 -- 10.20.193.78:0/505713752 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f569c0a1020 msgr2=0x7f569c135100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f569c0a1020 0x7f569c135100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 -- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f569c136510 msgr2=0x7f569c122bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f569c136510 0x7f569c122bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 monclient: found mon.noname-a 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.244+0000 7f569a7fc640 1 -- 10.20.193.78:0/505713752 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c124cb0 con 0x7f569c0a1940 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.246+0000 7f569a7fc640 1 -- 10.20.193.78:0/505713752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1940 msgr2=0x7f569c135640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.246+0000 7f569a7fc640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1940 0x7f569c135640 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f568400c600 tx=0x7f568400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684012a40 con 0x7f569c0a1940 2026-02-17T06:11:09.262 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: dump: 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.263 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5684017070 con 0x7f569c0a1940 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684012e20 con 0x7f569c0a1940 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 10 monclient: dump: 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.264 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56997fa640 4 set_mon_vals no callback set 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 5 monclient: authenticate success, global_id 284391 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 10 monclient: _renew_subs 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569c126b00 con 0x7f569c0a1940 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 10 monclient: _renew_subs 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569c09d810 con 0x7f569c0a1940 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f569c06f630 con 0x7f569c0a1940 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.247+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f568401c460 con 0x7f569c0a1940 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f5673fff640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f566c03deb0 0x7f566c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f56840524c0 con 0x7f569c0a1940 2026-02-17T06:11:09.265 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f5673fff640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7f566c043d50 0x7f566c046170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 --> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5684011e50 con 0x7f566c043d50 2026-02-17T06:11:09.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=867) ==== 24+0+0 (secure 0 0 0) 0x7f5684059050 con 0x7f569c0a1940 2026-02-17T06:11:09.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f5673fff640 10 monclient: handle_get_version_reply finishing 1 version 867 2026-02-17T06:11:09.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f569affd640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f566c03deb0 0x7f566c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f569affd640 10 monclient: get_auth_request con 0x7f566c03deb0 auth_method 0 2026-02-17T06:11:09.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f569b7fe640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7f566c043d50 0x7f566c046170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.248+0000 7f569b7fe640 10 monclient: get_auth_request con 0x7f566c043d50 auth_method 0 2026-02-17T06:11:09.266 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.250+0000 7f569affd640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f566c03deb0 0x7f566c040370 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5690036000 tx=0x7f5690033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.253+0000 7f569b7fe640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7f566c043d50 0x7f566c046170 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.253+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 <== osd.4 v2:10.20.193.115:6802/2725813951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f5684011e50 con 0x7f566c043d50 2026-02-17T06:11:09.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 --> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f569c1cd6b0 con 0x7f566c043d50 2026-02-17T06:11:09.267 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f5673fff640 1 -- 10.20.193.78:0/505713752 <== osd.4 v2:10.20.193.115:6802/2725813951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f569c1cd6b0 con 0x7f566c043d50 2026-02-17T06:11:09.267 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:09.268 INFO:teuthology.orchestra.run.trial078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:11:09.268 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:09.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7f566c043d50 msgr2=0x7f566c046170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6802/2725813951,v1:10.20.193.115:6803/2725813951] conn(0x7f566c043d50 0x7f566c046170 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f566c03deb0 msgr2=0x7f566c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f566c03deb0 0x7f566c040370 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5690036000 tx=0x7f5690033000 comp rx=0 tx=0).stop 2026-02-17T06:11:09.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 10 monclient: shutdown 2026-02-17T06:11:09.269 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1940 msgr2=0x7f569c135640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 --2- 10.20.193.78:0/505713752 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f569c0a1940 0x7f569c135640 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f568400c600 tx=0x7f568400cad0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f569b7fe640 1 -- 10.20.193.78:0/505713752 reap_dead start 2026-02-17T06:11:09.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 shutdown_connections 2026-02-17T06:11:09.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 >> 10.20.193.78:0/505713752 conn(0x7f569c0a3300 msgr2=0x7f569c0a5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 shutdown_connections 2026-02-17T06:11:09.270 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.260+0000 7f56a1987640 1 -- 10.20.193.78:0/505713752 wait complete. 2026-02-17T06:11:09.270 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-17T06:11:09.338 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.336+0000 7fe75636a640 10 monclient: get_monmap_and_config 2026-02-17T06:11:09.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.336+0000 7fe75636a640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.339 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.336+0000 7fe75636a640 10 monclient: monmap: 2026-02-17T06:11:09.339 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.339 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.339 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:09.336944+0000 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:09.336944+0000 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:09.340 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 1 Processor -- start 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 1 -- start start 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient: init 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe75006c890 0x7fe75006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient(hunting): picked mon.noname-a con 0x7fe75006c890 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe75009c4a0 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006d1b0 0x7fe750135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient(hunting): picked mon.noname-b con 0x7fe75006d1b0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.341 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75006daf0 con 0x7fe75009c4a0 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fe7500a0880 con 0x7fe75006c890 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe75009cc80 con 0x7fe75006d1b0 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe75636a640 10 monclient(hunting): authenticate will time out at 2114.890513s 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe75006c890 0x7fe75006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fe75006c890 0x7fe75006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52350/0 (socket says 10.20.193.78:52350) 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 1 -- 10.20.193.78:0/2950567062 learned_addr learned my addr 10.20.193.78:0/2950567062 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient(hunting): get_auth_request con 0x7fe75006c890 auth_method 0 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.342 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe7548e0640 1 --2- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006d1b0 0x7fe750135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe7548e0640 10 monclient(hunting): get_auth_request con 0x7fe75006d1b0 auth_method 0 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe7548e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe7548e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe7548e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe7548e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.343 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe7548e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe7548e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient(hunting): handle_auth_done global_id 284394 payload 274 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 1 -- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006d1b0 msgr2=0x7fe750135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 1 --2- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006d1b0 0x7fe750135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 1 -- 10.20.193.78:0/2950567062 >> v1:10.20.193.78:6789/0 conn(0x7fe75009c4a0 legacy=0x7fe750134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 1 monclient: found mon.noname-a 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.344 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.337+0000 7fe74ffff640 1 -- 10.20.193.78:0/2950567062 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75009cf60 con 0x7fe75006c890 2026-02-17T06:11:09.351 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74ffff640 1 -- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75006c890 msgr2=0x7fe75006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74ffff640 1 --2- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75006c890 0x7fe75006cc70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe75006dcc0 tx=0x7fe73c02ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a73af6f786d8f7 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 1 -- 10.20.193.78:0/2950567062 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c03c070 con 0x7fe75006c890 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: dump: 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.352 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 1 -- 10.20.193.78:0/2950567062 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73c02fcc0 con 0x7fe75006c890 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 5 monclient: authenticate success, global_id 284394 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:09.353 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 1 -- 10.20.193.78:0/2950567062 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c037660 con 0x7fe75006c890 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe74effd640 10 monclient: dump: 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.354 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 4 set_mon_vals no callback set 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 10 monclient: shutdown 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 1 -- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75006c890 msgr2=0x7fe75006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 1 --2- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75006c890 0x7fe75006cc70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe75006dcc0 tx=0x7fe73c02ee80 comp rx=0 tx=0).stop 2026-02-17T06:11:09.355 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 1 -- 10.20.193.78:0/2950567062 shutdown_connections 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 1 --2- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006d1b0 0x7fe750135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 1 --2- 10.20.193.78:0/2950567062 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75006c890 0x7fe75006cc70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 1 -- 10.20.193.78:0/2950567062 >> 10.20.193.78:0/2950567062 conn(0x7fe7500a6620 msgr2=0x7fe7500a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 1 -- 10.20.193.78:0/2950567062 shutdown_connections 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.340+0000 7fe75636a640 1 -- 10.20.193.78:0/2950567062 wait complete. 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient: monmap: 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:09.356 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 1 Processor -- start 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 1 -- start start 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient: init 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:11:09.357 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006c890 0x7fe7501cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient(hunting): picked mon.noname-c con 0x7fe75006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe75006d1b0 0x7fe750123f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient(hunting): picked mon.noname-b con 0x7fe75006d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75009c4a0 0x7fe750124440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient(hunting): picked mon.noname-a con 0x7fe75009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7500a01a0 con 0x7fe75009c4a0 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe7500a0020 con 0x7fe75006d1b0 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe75009be20 con 0x7fe75006c890 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.358 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe75636a640 10 monclient(hunting): authenticate will time out at 2114.894506s 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006c890 0x7fe7501cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75009c4a0 0x7fe750124440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75009c4a0 0x7fe750124440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52360/0 (socket says 10.20.193.78:52360) 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 -- 10.20.193.78:0/968251908 learned_addr learned my addr 10.20.193.78:0/968251908 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006c890 0x7fe7501cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34754/0 (socket says 10.20.193.78:34754) 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient(hunting): get_auth_request con 0x7fe75009c4a0 auth_method 0 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 10 monclient(hunting): get_auth_request con 0x7fe75006c890 auth_method 0 2026-02-17T06:11:09.359 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74f7fe640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe75006d1b0 0x7fe750123f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient(hunting): handle_auth_done global_id 284397 payload 824 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 -- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006c890 msgr2=0x7fe7501cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.360 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006c890 0x7fe7501cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 -- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe75006d1b0 msgr2=0x7fe750123f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe75006d1b0 0x7fe750123f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 monclient: found mon.noname-a 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe7548e0640 1 -- 10.20.193.78:0/968251908 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe750128870 con 0x7fe75009c4a0 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.341+0000 7fe74ffff640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006c890 0x7fe7501cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe7548e0640 1 -- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75009c4a0 msgr2=0x7fe750124440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe7548e0640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75009c4a0 0x7fe750124440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe74400efc0 tx=0x7fe74400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 1 -- 10.20.193.78:0/968251908 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744018020 con 0x7fe75009c4a0 2026-02-17T06:11:09.361 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: dump: 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.362 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe75636a640 5 monclient: authenticate success, global_id 284397 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 1 -- 10.20.193.78:0/968251908 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe744014680 con 0x7fe75009c4a0 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 1 -- 10.20.193.78:0/968251908 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744014a70 con 0x7fe75009c4a0 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe75636a640 10 monclient: _renew_subs 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe75636a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe750127bb0 con 0x7fe75009c4a0 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74e7fc640 4 set_mon_vals no callback set 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:09.363 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.343+0000 7fe74cff9640 10 monclient: dump: 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe75636a640 10 monclient: _renew_subs 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe75636a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7500a2030 con 0x7fe75009c4a0 2026-02-17T06:11:09.364 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe75636a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe75636a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe75636a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe750074980 con 0x7fe75009c4a0 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe74cff9640 1 -- 10.20.193.78:0/968251908 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe744014c10 con 0x7fe75009c4a0 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe74cff9640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe72003deb0 0x7fe720040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe74cff9640 1 -- 10.20.193.78:0/968251908 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fe744053650 con 0x7fe75009c4a0 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe74ffff640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe72003deb0 0x7fe720040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.344+0000 7fe74ffff640 10 monclient: get_auth_request con 0x7fe72003deb0 auth_method 0 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.347+0000 7fe74cff9640 1 -- 10.20.193.78:0/968251908 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe744022030 con 0x7fe75009c4a0 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.347+0000 7fe74cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.347+0000 7fe74cff9640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:09.365 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.347+0000 7fe74ffff640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe72003deb0 0x7fe720040370 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe73c036000 tx=0x7fe73c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.431 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.431+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe750126480 con 0x7fe72003deb0 2026-02-17T06:11:09.432 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.431+0000 7fe74cff9640 1 -- 10.20.193.78:0/968251908 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe750126480 con 0x7fe72003deb0 2026-02-17T06:11:09.432 INFO:teuthology.orchestra.run.trial078.stderr:instructed osd(s) 4 to deep-scrub 2026-02-17T06:11:09.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe72003deb0 msgr2=0x7fe720040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe72003deb0 0x7fe720040370 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe73c036000 tx=0x7fe73c033000 comp rx=0 tx=0).stop 2026-02-17T06:11:09.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 10 monclient: shutdown 2026-02-17T06:11:09.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.433 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75009c4a0 msgr2=0x7fe750124440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75009c4a0 0x7fe750124440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe74400efc0 tx=0x7fe74400c490 comp rx=0 tx=0).stop 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 shutdown_connections 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe75009c4a0 0x7fe750124440 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe72003deb0 0x7fe720040370 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe75006d1b0 0x7fe750123f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 --2- 10.20.193.78:0/968251908 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe75006c890 0x7fe7501cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 >> 10.20.193.78:0/968251908 conn(0x7fe7500a6620 msgr2=0x7fe7500a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 shutdown_connections 2026-02-17T06:11:09.434 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.432+0000 7fe75636a640 1 -- 10.20.193.78:0/968251908 wait complete. 2026-02-17T06:11:09.440 INFO:tasks.ceph:Scrubbing osd.5 2026-02-17T06:11:09.440 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.509+0000 7fb253e38640 10 monclient: get_monmap_and_config 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.509+0000 7fb253e38640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.509+0000 7fb253e38640 10 monclient: monmap: 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:09.509791+0000 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:09.509791+0000 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:09.509 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.510+0000 7fb253e38640 1 Processor -- start 2026-02-17T06:11:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 1 -- start start 2026-02-17T06:11:09.510 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient: init 2026-02-17T06:11:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2026-02-17T06:11:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb24c1382f0 0x7fb24c1386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient(hunting): picked mon.noname-a con 0x7fb24c1382f0 addr v2:10.20.193.78:3300/0 2026-02-17T06:11:09.511 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fb24c138c10 0x7fb24c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient(hunting): picked mon.noname-c con 0x7fb24c138c10 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb24c0a5a70 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c0a3540 con 0x7fb24c0a5a70 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 1 -- --> v2:10.20.193.78:3300/0 -- mon_getmap magic: 0 -- 0x7fb24c0a5000 con 0x7fb24c1382f0 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fb24c0a37e0 con 0x7fb24c138c10 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb253e38640 10 monclient(hunting): authenticate will time out at 2115.064440s 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb24c1382f0 0x7fb24c1386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 1 --2- >> v2:10.20.193.78:3300/0 conn(0x7fb24c1382f0 0x7fb24c1386d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52362/0 (socket says 10.20.193.78:52362) 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 1 -- 10.20.193.78:0/730580046 learned_addr learned my addr 10.20.193.78:0/730580046 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient(hunting): get_auth_request con 0x7fb24c1382f0 auth_method 0 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb2513ac640 1 --2- 10.20.193.78:0/730580046 >> v2:10.20.193.115:3300/0 conn(0x7fb24c138c10 0x7fb24c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb2513ac640 10 monclient(hunting): get_auth_request con 0x7fb24c138c10 auth_method 0 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb2513ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb2513ac640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb2513ac640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient(hunting): handle_auth_done global_id 284400 payload 274 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 1 -- 10.20.193.78:0/730580046 >> v2:10.20.193.115:3300/0 conn(0x7fb24c138c10 msgr2=0x7fb24c0a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 1 --2- 10.20.193.78:0/730580046 >> v2:10.20.193.115:3300/0 conn(0x7fb24c138c10 0x7fb24c0a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 1 -- 10.20.193.78:0/730580046 >> v1:10.20.193.78:6789/0 conn(0x7fb24c0a5a70 legacy=0x7fb24c0a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.512 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 1 monclient: found mon.noname-a 2026-02-17T06:11:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.511+0000 7fb251bad640 1 -- 10.20.193.78:0/730580046 --> v2:10.20.193.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c0a21d0 con 0x7fb24c1382f0 2026-02-17T06:11:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb251bad640 1 -- 10.20.193.78:0/730580046 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c1382f0 msgr2=0x7fb24c1386d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.513 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb251bad640 1 --2- 10.20.193.78:0/730580046 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c1382f0 0x7fb24c1386d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb24c06e460 tx=0x7fb23c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d61e3a2d5db6f2 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 1 -- 10.20.193.78:0/730580046 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c03c070 con 0x7fb24c1382f0 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: dump: 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.514 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 1 -- 10.20.193.78:0/730580046 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c02fca0 con 0x7fb24c1382f0 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 1 -- 10.20.193.78:0/730580046 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c037660 con 0x7fb24c1382f0 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 5 monclient: authenticate success, global_id 284400 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 4 set_mon_vals no callback set 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb250bab640 10 monclient: dump: 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 10 monclient: shutdown 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 1 -- 10.20.193.78:0/730580046 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c1382f0 msgr2=0x7fb24c1386d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.515 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 1 --2- 10.20.193.78:0/730580046 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c1382f0 0x7fb24c1386d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb24c06e460 tx=0x7fb23c02ee60 comp rx=0 tx=0).stop 2026-02-17T06:11:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 1 -- 10.20.193.78:0/730580046 shutdown_connections 2026-02-17T06:11:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 1 --2- 10.20.193.78:0/730580046 >> v2:10.20.193.115:3300/0 conn(0x7fb24c138c10 0x7fb24c0a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 1 --2- 10.20.193.78:0/730580046 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c1382f0 0x7fb24c1386d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 1 -- 10.20.193.78:0/730580046 >> 10.20.193.78:0/730580046 conn(0x7fb24c0a7a50 msgr2=0x7fb24c0a7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 1 -- 10.20.193.78:0/730580046 shutdown_connections 2026-02-17T06:11:09.516 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.514+0000 7fb253e38640 1 -- 10.20.193.78:0/730580046 wait complete. 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient: monmap: 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 1 Processor -- start 2026-02-17T06:11:09.517 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 1 -- start start 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient: init 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb24c0a5a70 0x7fb24c1cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient(hunting): picked mon.noname-c con 0x7fb24c0a5a70 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb24c1382f0 0x7fb24c1cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient(hunting): picked mon.noname-b con 0x7fb24c1382f0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c138c10 0x7fb24c124c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient(hunting): picked mon.noname-a con 0x7fb24c138c10 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c0a1970 con 0x7fb24c138c10 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c139020 con 0x7fb24c1382f0 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c0a1af0 con 0x7fb24c0a5a70 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb253e38640 10 monclient(hunting): authenticate will time out at 2115.068778s 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb24c0a5a70 0x7fb24c1cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2523ae640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c138c10 0x7fb24c124c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb24c0a5a70 0x7fb24c1cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34770/0 (socket says 10.20.193.78:34770) 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 1 -- 10.20.193.78:0/577813954 learned_addr learned my addr 10.20.193.78:0/577813954 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 10 monclient(hunting): get_auth_request con 0x7fb24c0a5a70 auth_method 0 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2523ae640 10 monclient(hunting): get_auth_request con 0x7fb24c138c10 auth_method 0 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2523ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2523ae640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2523ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.518 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2513ac640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb24c1382f0 0x7fb24c1cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb251bad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2523ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2523ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2523ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2513ac640 10 monclient(hunting): get_auth_request con 0x7fb24c1382f0 auth_method 0 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2513ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2513ac640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.515+0000 7fb2513ac640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 10 monclient(hunting): handle_auth_done global_id 284403 payload 824 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 1 -- 10.20.193.78:0/577813954 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb24c0a5a70 msgr2=0x7fb24c1cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb24c0a5a70 0x7fb24c1cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 1 -- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb24c1382f0 msgr2=0x7fb24c1cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fb24c1382f0 0x7fb24c1cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 1 monclient: found mon.noname-a 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb2523ae640 1 -- 10.20.193.78:0/577813954 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c137ad0 con 0x7fb24c138c10 2026-02-17T06:11:09.519 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.516+0000 7fb251bad640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fb24c0a5a70 0x7fb24c1cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb2523ae640 1 -- 10.20.193.78:0/577813954 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c138c10 msgr2=0x7fb24c124c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb2523ae640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c138c10 0x7fb24c124c20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb24800be00 tx=0x7fb24800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24800ca60 con 0x7fb24c138c10 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: dump: 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.520 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb248010070 con 0x7fb24c138c10 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 5 monclient: authenticate success, global_id 284403 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 10 monclient: _renew_subs 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c128570 con 0x7fb24c138c10 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2480040f0 con 0x7fb24c138c10 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23a7fc640 10 monclient: dump: 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.521 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb23bfff640 4 set_mon_vals no callback set 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 10 monclient: _renew_subs 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c1273a0 con 0x7fb24c138c10 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.518+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb24c074f50 con 0x7fb24c138c10 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fb248004310 con 0x7fb24c138c10 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb23a7fc640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb22403deb0 0x7fb224040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fb248052620 con 0x7fb24c138c10 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb23a7fc640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fb224043d50 0x7fb224046170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 --> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb224046820 con 0x7fb224043d50 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=867) ==== 24+0+0 (secure 0 0 0) 0x7fb248059050 con 0x7fb24c138c10 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb23a7fc640 10 monclient: handle_get_version_reply finishing 1 version 867 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb251bad640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb22403deb0 0x7fb224040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb251bad640 10 monclient: get_auth_request con 0x7fb22403deb0 auth_method 0 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb2513ac640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fb224043d50 0x7fb224046170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.519+0000 7fb2513ac640 10 monclient: get_auth_request con 0x7fb224043d50 auth_method 0 2026-02-17T06:11:09.522 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.521+0000 7fb251bad640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb22403deb0 0x7fb224040370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009730 tx=0x7fb23c033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.524+0000 7fb2513ac640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fb224043d50 0x7fb224046170 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.524 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.525+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 <== osd.5 v2:10.20.193.115:6811/2171295268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fb224046820 con 0x7fb224043d50 2026-02-17T06:11:09.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 --> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb24c0735a0 con 0x7fb224043d50 2026-02-17T06:11:09.531 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb23a7fc640 1 -- 10.20.193.78:0/577813954 <== osd.5 v2:10.20.193.115:6811/2171295268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb24c0735a0 con 0x7fb224043d50 2026-02-17T06:11:09.531 INFO:teuthology.orchestra.run.trial078.stdout:{ 2026-02-17T06:11:09.531 INFO:teuthology.orchestra.run.trial078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T06:11:09.531 INFO:teuthology.orchestra.run.trial078.stdout:} 2026-02-17T06:11:09.532 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fb224043d50 msgr2=0x7fb224046170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6811/2171295268,v1:10.20.193.115:6813/2171295268] conn(0x7fb224043d50 0x7fb224046170 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb22403deb0 msgr2=0x7fb224040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fb22403deb0 0x7fb224040370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009730 tx=0x7fb23c033000 comp rx=0 tx=0).stop 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 10 monclient: shutdown 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c138c10 msgr2=0x7fb24c124c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 --2- 10.20.193.78:0/577813954 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fb24c138c10 0x7fb24c124c20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb24800be00 tx=0x7fb24800bf00 comp rx=0 tx=0).stop 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb2523ae640 1 -- 10.20.193.78:0/577813954 reap_dead start 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 shutdown_connections 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 >> 10.20.193.78:0/577813954 conn(0x7fb24c0a7a50 msgr2=0x7fb24c072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 shutdown_connections 2026-02-17T06:11:09.533 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.531+0000 7fb253e38640 1 -- 10.20.193.78:0/577813954 wait complete. 2026-02-17T06:11:09.538 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-17T06:11:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.606+0000 7fd3b0bde640 10 monclient: get_monmap_and_config 2026-02-17T06:11:09.606 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.606+0000 7fd3b0bde640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.606+0000 7fd3b0bde640 10 monclient: monmap: 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:09.607184+0000 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:09.607184+0000 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:09.607 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 1 Processor -- start 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 1 -- start start 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient: init 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac06c890 0x7fd3ac06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient(hunting): picked mon.noname-b con 0x7fd3ac06c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd3ac09c4a0 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fd3ac06d1b0 0x7fd3ac135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient(hunting): picked mon.noname-c con 0x7fd3ac06d1b0 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3ac06daf0 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fd3ac0a0880 con 0x7fd3ac06d1b0 2026-02-17T06:11:09.610 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ac09cc80 con 0x7fd3ac06c890 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3b0bde640 10 monclient(hunting): authenticate will time out at 2115.160699s 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac06c890 0x7fd3ac06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac06c890 0x7fd3ac06cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34774/0 (socket says 10.20.193.78:34774) 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 1 -- 10.20.193.78:0/3038788959 learned_addr learned my addr 10.20.193.78:0/3038788959 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 10 monclient(hunting): get_auth_request con 0x7fd3ac06c890 auth_method 0 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aad76640 1 --2- 10.20.193.78:0/3038788959 >> v2:10.20.193.115:3300/0 conn(0x7fd3ac06d1b0 0x7fd3ac135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aad76640 10 monclient(hunting): get_auth_request con 0x7fd3ac06d1b0 auth_method 0 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.611 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 10 monclient(hunting): handle_auth_done global_id 264602 payload 274 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 1 -- 10.20.193.78:0/3038788959 >> v2:10.20.193.115:3300/0 conn(0x7fd3ac06d1b0 msgr2=0x7fd3ac135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 1 --2- 10.20.193.78:0/3038788959 >> v2:10.20.193.115:3300/0 conn(0x7fd3ac06d1b0 0x7fd3ac135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.607+0000 7fd3aa575640 1 -- 10.20.193.78:0/3038788959 >> v1:10.20.193.115:6789/0 conn(0x7fd3ac09c4a0 legacy=0x7fd3ac134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.608+0000 7fd3aa575640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.608+0000 7fd3aa575640 1 monclient: found mon.noname-b 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.608+0000 7fd3aa575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.608+0000 7fd3aa575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.608+0000 7fd3aa575640 1 -- 10.20.193.78:0/3038788959 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac09cf60 con 0x7fd3ac06c890 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3aa575640 1 -- 10.20.193.78:0/3038788959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac06c890 msgr2=0x7fd3ac06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.612 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3aa575640 1 --2- 10.20.193.78:0/3038788959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac06c890 0x7fd3ac06cc70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009970 tx=0x7fd3a002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f91aaa48a6dd2ac server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 1 -- 10.20.193.78:0/3038788959 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a003c070 con 0x7fd3ac06c890 2026-02-17T06:11:09.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:09.616 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: dump: 2026-02-17T06:11:09.616 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 1 -- 10.20.193.78:0/3038788959 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a002fcd0 con 0x7fd3ac06c890 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 5 monclient: authenticate success, global_id 264602 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:09.617 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 1 -- 10.20.193.78:0/3038788959 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0037660 con 0x7fd3ac06c890 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 4 set_mon_vals no callback set 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3a9573640 10 monclient: dump: 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 10 monclient: shutdown 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 1 -- 10.20.193.78:0/3038788959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac06c890 msgr2=0x7fd3ac06cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.618 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/3038788959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac06c890 0x7fd3ac06cc70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009970 tx=0x7fd3a002ee90 comp rx=0 tx=0).stop 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 1 -- 10.20.193.78:0/3038788959 shutdown_connections 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.610+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/3038788959 >> v2:10.20.193.115:3300/0 conn(0x7fd3ac06d1b0 0x7fd3ac135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/3038788959 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac06c890 0x7fd3ac06cc70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 1 -- 10.20.193.78:0/3038788959 >> 10.20.193.78:0/3038788959 conn(0x7fd3ac0a6620 msgr2=0x7fd3ac0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 1 -- 10.20.193.78:0/3038788959 shutdown_connections 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 1 -- 10.20.193.78:0/3038788959 wait complete. 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 10 monclient: monmap: 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.619 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 1 Processor -- start 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 1 -- start start 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.611+0000 7fd3b0bde640 10 monclient: init 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd3ac06c890 0x7fd3ac1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient(hunting): picked mon.noname-a con 0x7fd3ac06c890 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd3ac06d1b0 0x7fd3ac1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient(hunting): picked mon.noname-b con 0x7fd3ac06d1b0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac09c4a0 0x7fd3ac09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient(hunting): picked mon.noname-c con 0x7fd3ac09c4a0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac0a01a0 con 0x7fd3ac06c890 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac0a0020 con 0x7fd3ac06d1b0 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ac09be20 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3b0bde640 10 monclient(hunting): authenticate will time out at 2115.165126s 2026-02-17T06:11:09.620 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac09c4a0 0x7fd3ac09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd3ac06c890 0x7fd3ac1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac09c4a0 0x7fd3ac09db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:34780/0 (socket says 10.20.193.78:34780) 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 -- 10.20.193.78:0/4014570582 learned_addr learned my addr 10.20.193.78:0/4014570582 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 10 monclient(hunting): get_auth_request con 0x7fd3ac06c890 auth_method 0 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient(hunting): get_auth_request con 0x7fd3ac09c4a0 auth_method 0 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3a9d74640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd3ac06d1b0 0x7fd3ac1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.621 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3a9d74640 10 monclient(hunting): get_auth_request con 0x7fd3ac06d1b0 auth_method 0 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3a9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3a9d74640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3a9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient(hunting): handle_auth_done global_id 264605 payload 824 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 -- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd3ac06d1b0 msgr2=0x7fd3ac1cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd3ac06d1b0 0x7fd3ac1cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 -- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd3ac06c890 msgr2=0x7fd3ac1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd3ac06c890 0x7fd3ac1cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 monclient: found mon.noname-c 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aad76640 1 -- 10.20.193.78:0/4014570582 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac136ea0 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.612+0000 7fd3aa575640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd3ac06c890 0x7fd3ac1cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3aad76640 1 -- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac09c4a0 msgr2=0x7fd3ac09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3aad76640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac09c4a0 0x7fd3ac09db50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd39c007d90 tx=0x7fd39c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 1 -- 10.20.193.78:0/4014570582 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c014620 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.622 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: dump: 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 1 -- 10.20.193.78:0/4014570582 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39c019070 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.623 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 1 -- 10.20.193.78:0/4014570582 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c014a20 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd392ffd640 10 monclient: dump: 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3a8d72640 4 set_mon_vals no callback set 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3b0bde640 5 monclient: authenticate success, global_id 264605 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3b0bde640 10 monclient: _renew_subs 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3b0bde640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac06fe40 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.624 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3b0bde640 10 monclient: _renew_subs 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3b0bde640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.614+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac129090 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd3b0bde640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd3b0bde640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd3b0bde640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac074980 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd392ffd640 1 -- 10.20.193.78:0/4014570582 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fd39c01e460 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd392ffd640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd37803df00 0x7fd3780403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd392ffd640 1 -- 10.20.193.78:0/4014570582 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fd39c052fc0 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd3aa575640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd37803df00 0x7fd3780403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.625 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.615+0000 7fd3aa575640 10 monclient: get_auth_request con 0x7fd37803df00 auth_method 0 2026-02-17T06:11:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.617+0000 7fd392ffd640 1 -- 10.20.193.78:0/4014570582 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd3ac136ea0 con 0x7fd3ac09c4a0 2026-02-17T06:11:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.617+0000 7fd392ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.617+0000 7fd392ffd640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:09.626 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.618+0000 7fd3aa575640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd37803df00 0x7fd3780403c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0036000 tx=0x7fd3a0033000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.700 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.701+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fd3ac06cc70 con 0x7fd37803df00 2026-02-17T06:11:09.701 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.701+0000 7fd392ffd640 1 -- 10.20.193.78:0/4014570582 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd3ac06cc70 con 0x7fd37803df00 2026-02-17T06:11:09.701 INFO:teuthology.orchestra.run.trial078.stderr:instructed osd(s) 5 to deep-scrub 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd37803df00 msgr2=0x7fd3780403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd37803df00 0x7fd3780403c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0036000 tx=0x7fd3a0033000 comp rx=0 tx=0).stop 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 10 monclient: shutdown 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac09c4a0 msgr2=0x7fd3ac09db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac09c4a0 0x7fd3ac09db50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd39c007d90 tx=0x7fd39c00e4b0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 shutdown_connections 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fd3ac09c4a0 0x7fd3ac09db50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fd37803df00 0x7fd3780403c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fd3ac06d1b0 0x7fd3ac1cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 --2- 10.20.193.78:0/4014570582 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fd3ac06c890 0x7fd3ac1cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 >> 10.20.193.78:0/4014570582 conn(0x7fd3ac0a6620 msgr2=0x7fd3ac072240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 shutdown_connections 2026-02-17T06:11:09.702 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.702+0000 7fd3b0bde640 1 -- 10.20.193.78:0/4014570582 wait complete. 2026-02-17T06:11:09.709 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:11:09.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.776+0000 7f38c41a0640 10 monclient: get_monmap_and_config 2026-02-17T06:11:09.779 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.776+0000 7f38c41a0640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient: monmap: 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:09.777226+0000 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:09.777226+0000 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:09.780 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 1 Processor -- start 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 1 -- start start 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient: init 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f38bc0a3140 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:09.781 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f38bc0a3c30 0x7f38bc06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient(hunting): picked mon.noname-c con 0x7f38bc0a3c30 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:09.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f38bc13d420 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:09.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc0a48a0 con 0x7f38bc0a3140 2026-02-17T06:11:09.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc06f970 con 0x7f38bc13d420 2026-02-17T06:11:09.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc06ffc0 con 0x7f38bc0a3c30 2026-02-17T06:11:09.782 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.777+0000 7f38c41a0640 10 monclient(hunting): authenticate will time out at 2115.330958s 2026-02-17T06:11:09.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1f15640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f38bc0a3140 0x7f38bc0a3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:33184/0 (socket says 10.20.193.78:33184) 2026-02-17T06:11:09.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1f15640 1 -- 10.20.193.78:0/2722381994 learned_addr learned my addr 10.20.193.78:0/2722381994 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:09.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 1 --2- 10.20.193.78:0/2722381994 >> v2:10.20.193.115:3300/0 conn(0x7f38bc0a3c30 0x7f38bc06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient(hunting): get_auth_request con 0x7f38bc0a3c30 auth_method 0 2026-02-17T06:11:09.783 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient(hunting): handle_auth_done global_id 304183 payload 274 2026-02-17T06:11:09.784 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 1 -- 10.20.193.78:0/2722381994 >> v1:10.20.193.115:6789/0 conn(0x7f38bc13d420 legacy=0x7f38bc13f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 1 -- 10.20.193.78:0/2722381994 >> v1:10.20.193.78:6789/0 conn(0x7f38bc0a3140 legacy=0x7f38bc0a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:09.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 1 monclient: found mon.noname-c 2026-02-17T06:11:09.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.115:3300/0 2026-02-17T06:11:09.785 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.778+0000 7f38c1714640 1 -- 10.20.193.78:0/2722381994 --> v2:10.20.193.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc06fc50 con 0x7f38bc0a3c30 2026-02-17T06:11:09.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.780+0000 7f38c1714640 1 -- 10.20.193.78:0/2722381994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3c30 msgr2=0x7f38bc06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.780+0000 7f38c1714640 1 --2- 10.20.193.78:0/2722381994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3c30 0x7f38bc06eb10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f38bc06f0e0 tx=0x7f38b002ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d456bf0fc4eecef server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.802 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.780+0000 7f38c0f13640 1 -- 10.20.193.78:0/2722381994 <== mon.1 v2:10.20.193.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b003c070 con 0x7f38bc0a3c30 2026-02-17T06:11:09.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.780+0000 7f38c0f13640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:09.803 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 10 monclient: dump: 2026-02-17T06:11:09.803 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.803 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.803 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.803 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.804 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.804 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.804 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.804 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.804 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.804 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.804 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 1 -- 10.20.193.78:0/2722381994 <== mon.1 v2:10.20.193.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b002fcb0 con 0x7f38bc0a3c30 2026-02-17T06:11:09.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 5 monclient: authenticate success, global_id 304183 2026-02-17T06:11:09.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 1 -- 10.20.193.78:0/2722381994 <== mon.1 v2:10.20.193.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b002fe60 con 0x7f38bc0a3c30 2026-02-17T06:11:09.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:09.805 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 4 set_mon_vals no callback set 2026-02-17T06:11:09.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-17T06:11:09.806 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c0f13640 10 monclient: dump: 2026-02-17T06:11:09.806 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.806 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.806 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.806 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.807 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.807 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.807 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.807 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.807 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.807 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.807 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 10 monclient: shutdown 2026-02-17T06:11:09.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 -- 10.20.193.78:0/2722381994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3c30 msgr2=0x7f38bc06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 --2- 10.20.193.78:0/2722381994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3c30 0x7f38bc06eb10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f38bc06f0e0 tx=0x7f38b002ed50 comp rx=0 tx=0).stop 2026-02-17T06:11:09.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 -- 10.20.193.78:0/2722381994 shutdown_connections 2026-02-17T06:11:09.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 --2- 10.20.193.78:0/2722381994 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3c30 0x7f38bc06eb10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 -- 10.20.193.78:0/2722381994 >> 10.20.193.78:0/2722381994 conn(0x7f38bc09c000 msgr2=0x7f38bc0a8740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 -- 10.20.193.78:0/2722381994 shutdown_connections 2026-02-17T06:11:09.808 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 -- 10.20.193.78:0/2722381994 wait complete. 2026-02-17T06:11:09.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 10 monclient: build_initial_monmap 2026-02-17T06:11:09.809 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 10 monclient: monmap: 2026-02-17T06:11:09.809 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:09.809 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:09.809 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:09.809 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:09.809 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 Processor -- start 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 1 -- start start 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.781+0000 7f38c41a0640 10 monclient: init 2026-02-17T06:11:09.810 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-17T06:11:09.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 0x7f38bc1232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient(hunting): picked mon.noname-b con 0x7f38bc0a3140 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:09.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 0x7f38bc1237e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient(hunting): picked mon.noname-a con 0x7f38bc0a3c30 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:09.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 0x7f38bc123d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient(hunting): picked mon.noname-c con 0x7f38bc13d420 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.811 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc0a9480 con 0x7f38bc0a3c30 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc0a9300 con 0x7f38bc0a3140 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc140f30 con 0x7f38bc13d420 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c41a0640 10 monclient(hunting): authenticate will time out at 2115.335179s 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 0x7f38bc1237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.812 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 0x7f38bc1237e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:52394/0 (socket says 10.20.193.78:52394) 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 -- 10.20.193.78:0/3537041186 learned_addr learned my addr 10.20.193.78:0/3537041186 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient(hunting): get_auth_request con 0x7f38bc0a3c30 auth_method 0 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c2716640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 0x7f38bc123d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.813 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c2716640 10 monclient(hunting): get_auth_request con 0x7f38bc13d420 auth_method 0 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c2716640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c2716640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c2716640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1f15640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 0x7f38bc1232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c2716640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c2716640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c2716640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:09.814 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1f15640 10 monclient(hunting): get_auth_request con 0x7f38bc0a3140 auth_method 0 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1f15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1f15640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1f15640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient(hunting): handle_auth_done global_id 284409 payload 824 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 msgr2=0x7f38bc123d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 0x7f38bc123d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 msgr2=0x7f38bc1232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 0x7f38bc1232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.815 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 monclient: found mon.noname-a 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.782+0000 7f38c1714640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc125c70 con 0x7f38bc0a3c30 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c1714640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 msgr2=0x7f38bc1237e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c1714640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 0x7f38bc1237e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f38b0002410 tx=0x7f38b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0046070 con 0x7f38bc0a3c30 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:09.816 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: dump: 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.817 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c41a0640 5 monclient: authenticate success, global_id 284409 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b0041890 con 0x7f38bc0a3c30 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b003c040 con 0x7f38bc0a3c30 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: dump: 2026-02-17T06:11:09.818 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c41a0640 10 monclient: _renew_subs 2026-02-17T06:11:09.819 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38abfff640 4 set_mon_vals no callback set 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c41a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc125f50 con 0x7f38bc0a3c30 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c41a0640 10 monclient: _renew_subs 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c41a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc0a5fa0 con 0x7f38bc0a3c30 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c1714640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 msgr2=0x7f38bc1237e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c1714640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 msgr2=0x7f38bc1237e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c1714640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 0x7f38bc1237e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f38b0002410 tx=0x7f38b00047c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38c1714640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 0x7f38bc1237e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f38b0002410 tx=0x7f38b00047c0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: ms_handle_reset current mon [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:09.820 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 msgr2=0x7f38bc1237e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.784+0000 7f38aa7fc640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f38bc0a3c30 0x7f38bc1237e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 0x7f388c005970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 10 monclient(hunting): picked mon.c con 0x7f38bc13d420 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 0x7f388c0083b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 10 monclient(hunting): picked mon.b con 0x7f38bc0a3140 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f388c008dc0 0x7f388c00b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 10 monclient(hunting): picked mon.a con 0x7f388c008dc0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.821 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f388c00bd00 con 0x7f388c008dc0 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f388c00be80 con 0x7f38bc0a3140 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f388c00c000 con 0x7f38bc13d420 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38aa7fc640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 0x7f388c005970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c2716640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f388c008dc0 0x7f388c00b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient(hunting): get_auth_request con 0x7f38bc13d420 auth_method 0 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.822 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c2716640 10 monclient(hunting): get_auth_request con 0x7f388c008dc0 auth_method 0 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c2716640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c2716640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c2716640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1714640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 0x7f388c0083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c2716640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:09.823 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c2716640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c2716640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1714640 10 monclient(hunting): get_auth_request con 0x7f38bc0a3140 auth_method 0 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1714640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1714640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1714640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient(hunting): handle_auth_done global_id 284409 payload 293 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c41a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 msgr2=0x7f388c0083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 0x7f388c0083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f388c008dc0 msgr2=0x7f388c00b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f388c008dc0 0x7f388c00b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient: _finish_hunting 0 2026-02-17T06:11:09.824 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 1 monclient: found mon.c 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7f388c00c330 con 0x7f38bc13d420 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient: _finish_auth 0 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 10 monclient: _check_auth_tickets 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c1f15640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c41a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c41a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.785+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc06eb10 con 0x7f38bc13d420 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38c1f15640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 msgr2=0x7f388c005970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38c1f15640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 0x7f388c005970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00c9b0 tx=0x7f38b0033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.825 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00c580 con 0x7f38bc13d420 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38aa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38aa7fc640 10 monclient: dump: 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:09.826 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac01a020 con 0x7f38bc13d420 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38aa7fc640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.787+0000 7f38ab7fe640 4 set_mon_vals no callback set 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.788+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f38ac00e800 con 0x7f38bc13d420 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.788+0000 7f38aa7fc640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38bc0a3c30 0x7f388c048720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.788+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mon.2 v2:10.20.193.78:3301/0 4 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f38ac010040 con 0x7f38bc13d420 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.788+0000 7f38c1714640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38bc0a3c30 0x7f388c048720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.788+0000 7f38c1714640 10 monclient: get_auth_request con 0x7f38bc0a3c30 auth_method 0 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.790+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mon.2 v2:10.20.193.78:3301/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f38ac00c340 con 0x7f38bc13d420 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.790+0000 7f38aa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.790+0000 7f38aa7fc640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:09.827 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.791+0000 7f38c1714640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38bc0a3c30 0x7f388c048720 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f38bc09c830 tx=0x7f38b0041000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:09.870 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.870+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38bc09f930 con 0x7f38bc0a3c30 2026-02-17T06:11:09.871 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.871+0000 7f38aa7fc640 1 -- 10.20.193.78:0/3537041186 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34010 (secure 0 0 0) 0x7f38bc09f930 con 0x7f38bc0a3c30 2026-02-17T06:11:09.871 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T06:11:09.871 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38bc0a3c30 msgr2=0x7f388c048720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38bc0a3c30 0x7f388c048720 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f38bc09c830 tx=0x7f38b0041000 comp rx=0 tx=0).stop 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 10 monclient: shutdown 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 msgr2=0x7f388c005970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 0x7f388c005970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00c9b0 tx=0x7f38b0033000 comp rx=0 tx=0).stop 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 shutdown_connections 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f388c008dc0 0x7f388c00b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f38bc0a3c30 0x7f388c048720 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f38bc0a3140 0x7f388c0083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 --2- 10.20.193.78:0/3537041186 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f38bc13d420 0x7f388c005970 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 >> 10.20.193.78:0/3537041186 conn(0x7f38bc09c000 msgr2=0x7f38bc09e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 shutdown_connections 2026-02-17T06:11:09.872 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:09.872+0000 7f38c41a0640 1 -- 10.20.193.78:0/3537041186 wait complete. 2026-02-17T06:11:09.879 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":1609,"stamp":"2026-02-17T06:11:08.065569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":296304,"kb_used_data":24896,"kb_used_omap":9,"kb_used_meta":271286,"kb_avail":565934736,"statfs":{"total":579820584960,"available":579517169664,"internally_reserved":0,"allocated":25493504,"data_stored":12055756,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":277797542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000986"},"pg_stats":[{"pgid":"1.0","version":"16'112","reported_seq":1819,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773900+0000","last_change":"2026-02-17T05:46:05.167613+0000","last_active":"2026-02-17T06:10:40.773900+0000","last_peered":"2026-02-17T06:10:40.773900+0000","last_clean":"2026-02-17T06:10:40.773900+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T06:10:40.773900+0000","last_undegraded":"2026-02-17T06:10:40.773900+0000","last_fullsized":"2026-02-17T06:10:40.773900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:04.161551+0000","last_clean_scrub_stamp":"2026-02-17T05:46:04.161551+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T11:13:58.006467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":1682,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773927+0000","last_change":"2026-02-17T05:46:12.183095+0000","last_active":"2026-02-17T06:10:40.773927+0000","last_peered":"2026-02-17T06:10:40.773927+0000","last_clean":"2026-02-17T06:10:40.773927+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T06:10:40.773927+0000","last_undegraded":"2026-02-17T06:10:40.773927+0000","last_fullsized":"2026-02-17T06:10:40.773927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:04:04.403821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1693,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.449963+0000","last_change":"2026-02-17T05:46:12.183042+0000","last_active":"2026-02-17T06:10:40.449963+0000","last_peered":"2026-02-17T06:10:40.449963+0000","last_clean":"2026-02-17T06:10:40.449963+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T06:10:40.449963+0000","last_undegraded":"2026-02-17T06:10:40.449963+0000","last_fullsized":"2026-02-17T06:10:40.449963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:15:40.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1702,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.772884+0000","last_change":"2026-02-17T05:46:12.182571+0000","last_active":"2026-02-17T06:10:40.772884+0000","last_peered":"2026-02-17T06:10:40.772884+0000","last_clean":"2026-02-17T06:10:40.772884+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T06:10:40.772884+0000","last_undegraded":"2026-02-17T06:10:40.772884+0000","last_fullsized":"2026-02-17T06:10:40.772884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:18:54.353167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":1696,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.127960+0000","last_change":"2026-02-17T05:46:12.184021+0000","last_active":"2026-02-17T06:10:40.127960+0000","last_peered":"2026-02-17T06:10:40.127960+0000","last_clean":"2026-02-17T06:10:40.127960+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T06:10:40.127960+0000","last_undegraded":"2026-02-17T06:10:40.127960+0000","last_fullsized":"2026-02-17T06:10:40.127960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:38:14.986294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1679,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.126045+0000","last_change":"2026-02-17T05:46:12.217740+0000","last_active":"2026-02-17T06:10:40.126045+0000","last_peered":"2026-02-17T06:10:40.126045+0000","last_clean":"2026-02-17T06:10:40.126045+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T06:10:40.126045+0000","last_undegraded":"2026-02-17T06:10:40.126045+0000","last_fullsized":"2026-02-17T06:10:40.126045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:23:05.440393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1681,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.774004+0000","last_change":"2026-02-17T05:46:12.183276+0000","last_active":"2026-02-17T06:10:40.774004+0000","last_peered":"2026-02-17T06:10:40.774004+0000","last_clean":"2026-02-17T06:10:40.774004+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T06:10:40.774004+0000","last_undegraded":"2026-02-17T06:10:40.774004+0000","last_fullsized":"2026-02-17T06:10:40.774004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T09:23:44.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1678,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.773012+0000","last_change":"2026-02-17T05:46:12.184736+0000","last_active":"2026-02-17T06:10:40.773012+0000","last_peered":"2026-02-17T06:10:40.773012+0000","last_clean":"2026-02-17T06:10:40.773012+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T06:10:40.773012+0000","last_undegraded":"2026-02-17T06:10:40.773012+0000","last_fullsized":"2026-02-17T06:10:40.773012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T15:25:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1681,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:10:40.774000+0000","last_change":"2026-02-17T05:46:12.183122+0000","last_active":"2026-02-17T06:10:40.774000+0000","last_peered":"2026-02-17T06:10:40.774000+0000","last_clean":"2026-02-17T06:10:40.774000+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T06:10:40.774000+0000","last_undegraded":"2026-02-17T06:10:40.774000+0000","last_fullsized":"2026-02-17T06:10:40.774000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T05:46:09.171002+0000","last_clean_scrub_stamp":"2026-02-17T05:46:09.171002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:12:49.022806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607859,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31156,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94340684,"statfs":{"total":96636764160,"available":96604860416,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640563,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53296,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318544,"statfs":{"total":96636764160,"available":96582189056,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61616,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310224,"statfs":{"total":96636764160,"available":96573669376,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35308,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94336532,"statfs":{"total":96636764160,"available":96600608768,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607858,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53300,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318540,"statfs":{"total":96636764160,"available":96582184960,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:11:09.882 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-17T05:46:04.161551+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=4, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.882 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-17T05:46:09.171002+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=9, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.882 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-17T05:46:09.171002+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=9, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.882 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-17T05:46:09.171002+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=9, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.882 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-17T05:46:09.171002+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=9, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.882 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-17T05:46:09.171002+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=9, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.882 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-17T05:46:09.171002+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=9, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.882 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-17T05:46:09.171002+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=9, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.883 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-17T05:46:09.171002+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=9, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T06:11:09.883 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-17T06:11:29.883 DEBUG:teuthology.orchestra.run.trial078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.951+0000 7fe4a7775640 10 monclient: get_monmap_and_config 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.951+0000 7fe4a7775640 10 monclient: build_initial_monmap 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.951+0000 7fe4a7775640 10 monclient: monmap: 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:29.952039+0000 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:29.952039+0000 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:29.951 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:29.952 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:29.952 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:29.952 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:29.952 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:29.952 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 1 Processor -- start 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 1 -- start start 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient: init 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe4a006c890 addr v1:10.20.193.115:6789/0 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7fe4a009c4a0 0x7fe4a013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient(hunting): picked mon.noname-c con 0x7fe4a009c4a0 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006d380 0x7fe4a013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient(hunting): picked mon.noname-b con 0x7fe4a006d380 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 1 -- --> v1:10.20.193.115:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a006daf0 con 0x7fe4a006c890 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a00a0880 con 0x7fe4a009c4a0 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a009cc80 con 0x7fe4a006d380 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a7775640 10 monclient(hunting): authenticate will time out at 2135.505628s 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006d380 0x7fe4a013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006d380 0x7fe4a013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:35304/0 (socket says 10.20.193.78:35304) 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 1 -- 10.20.193.78:0/3412172011 learned_addr learned my addr 10.20.193.78:0/3412172011 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient(hunting): get_auth_request con 0x7fe4a006d380 auth_method 0 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:29.953 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a4ce9640 1 --2- 10.20.193.78:0/3412172011 >> v2:10.20.193.115:3300/0 conn(0x7fe4a009c4a0 0x7fe4a013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a4ce9640 10 monclient(hunting): get_auth_request con 0x7fe4a009c4a0 auth_method 0 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a4ce9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a4ce9640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a4ce9640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient(hunting): handle_auth_done global_id 264611 payload 274 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 1 -- 10.20.193.78:0/3412172011 >> v2:10.20.193.115:3300/0 conn(0x7fe4a009c4a0 msgr2=0x7fe4a013b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 1 --2- 10.20.193.78:0/3412172011 >> v2:10.20.193.115:3300/0 conn(0x7fe4a009c4a0 0x7fe4a013b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 1 -- 10.20.193.78:0/3412172011 >> v1:10.20.193.115:6789/0 conn(0x7fe4a006c890 legacy=0x7fe4a006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient: _finish_hunting 0 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 1 monclient: found mon.noname-b 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.952+0000 7fe4a5ceb640 1 -- 10.20.193.78:0/3412172011 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a009cf60 con 0x7fe4a006d380 2026-02-17T06:11:29.954 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a5ceb640 1 -- 10.20.193.78:0/3412172011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006d380 msgr2=0x7fe4a013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a5ceb640 1 --2- 10.20.193.78:0/3412172011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006d380 0x7fe4a013d8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009e50 tx=0x7fe49c02f260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e11224ff5b9992 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 1 -- 10.20.193.78:0/3412172011 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0373f0 con 0x7fe4a006d380 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: dump: 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:29.955 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: _finish_auth 0 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: _check_auth_tickets 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 1 -- 10.20.193.78:0/3412172011 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49c037590 con 0x7fe4a006d380 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 5 monclient: authenticate success, global_id 264611 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 1 -- 10.20.193.78:0/3412172011 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c037960 con 0x7fe4a006d380 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 4 set_mon_vals no callback set 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe48ffff640 10 monclient: dump: 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 10 monclient: shutdown 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 1 -- 10.20.193.78:0/3412172011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006d380 msgr2=0x7fe4a013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 1 --2- 10.20.193.78:0/3412172011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006d380 0x7fe4a013d8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009e50 tx=0x7fe49c02f260 comp rx=0 tx=0).stop 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 1 -- 10.20.193.78:0/3412172011 shutdown_connections 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 1 --2- 10.20.193.78:0/3412172011 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006d380 0x7fe4a013d8f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 1 --2- 10.20.193.78:0/3412172011 >> v2:10.20.193.115:3300/0 conn(0x7fe4a009c4a0 0x7fe4a013b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 1 -- 10.20.193.78:0/3412172011 >> 10.20.193.78:0/3412172011 conn(0x7fe4a00a6620 msgr2=0x7fe4a00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 1 -- 10.20.193.78:0/3412172011 shutdown_connections 2026-02-17T06:11:29.956 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.955+0000 7fe4a7775640 1 -- 10.20.193.78:0/3412172011 wait complete. 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient: build_initial_monmap 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient: monmap: 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 1 Processor -- start 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 1 -- start start 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient: init 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006c890 0x7fe4a01cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient(hunting): picked mon.noname-c con 0x7fe4a006c890 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe4a006d380 0x7fe4a0136380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient(hunting): picked mon.noname-b con 0x7fe4a006d380 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4a009c4a0 0x7fe4a01368c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient(hunting): picked mon.noname-a con 0x7fe4a009c4a0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a009bfa0 con 0x7fe4a009c4a0 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a009be20 con 0x7fe4a006d380 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a013de30 con 0x7fe4a006c890 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a7775640 10 monclient(hunting): authenticate will time out at 2135.509836s 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006c890 0x7fe4a01cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:29.958 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006c890 0x7fe4a01cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3301/0 says I am v2:10.20.193.78:35316/0 (socket says 10.20.193.78:35316) 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 1 -- 10.20.193.78:0/754454144 learned_addr learned my addr 10.20.193.78:0/754454144 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 10 monclient(hunting): get_auth_request con 0x7fe4a006c890 auth_method 0 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a5ceb640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4a009c4a0 0x7fe4a01368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a5ceb640 10 monclient(hunting): get_auth_request con 0x7fe4a009c4a0 auth_method 0 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a5ceb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a5ceb640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a5ceb640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a54ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a5ceb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a5ceb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.956+0000 7fe4a5ceb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 10 monclient(hunting): handle_auth_done global_id 264614 payload 824 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 1 -- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe4a006d380 msgr2=0x7fe4a0136380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe4a006d380 0x7fe4a0136380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 1 -- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4a009c4a0 msgr2=0x7fe4a01368c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4a009c4a0 0x7fe4a01368c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 10 monclient: _finish_hunting 0 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 1 monclient: found mon.noname-c 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.78:3301/0 2026-02-17T06:11:29.959 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.957+0000 7fe4a54ea640 1 -- 10.20.193.78:0/754454144 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a006f750 con 0x7fe4a006c890 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a54ea640 1 -- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006c890 msgr2=0x7fe4a01cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a54ea640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006c890 0x7fe4a01cf160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c600 tx=0x7fe49000cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 1 -- 10.20.193.78:0/754454144 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490012a40 con 0x7fe4a006c890 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: dump: 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:29.960 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: _finish_auth 0 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: _check_auth_tickets 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 1 -- 10.20.193.78:0/754454144 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe490017070 con 0x7fe4a006c890 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 5 monclient: authenticate success, global_id 264614 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 1 -- 10.20.193.78:0/754454144 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490012e20 con 0x7fe4a006c890 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48f7fe640 4 set_mon_vals no callback set 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe48dffb640 10 monclient: dump: 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 10 monclient: _renew_subs 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a013da80 con 0x7fe4a006c890 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 10 monclient: _renew_subs 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a01224b0 con 0x7fe4a006c890 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.78:3301/0 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.959+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a0074980 con 0x7fe4a006c890 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.960+0000 7fe48dffb640 1 -- 10.20.193.78:0/754454144 <== mon.2 v2:10.20.193.78:3301/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7fe49001c460 con 0x7fe4a006c890 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.960+0000 7fe48dffb640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe47003deb0 0x7fe470040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.960+0000 7fe48dffb640 1 -- 10.20.193.78:0/754454144 <== mon.2 v2:10.20.193.78:3301/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7fe490052490 con 0x7fe4a006c890 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.960+0000 7fe4a4ce9640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe47003deb0 0x7fe470040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:29.961 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.960+0000 7fe4a4ce9640 10 monclient: get_auth_request con 0x7fe47003deb0 auth_method 0 2026-02-17T06:11:29.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.962+0000 7fe48dffb640 1 -- 10.20.193.78:0/754454144 <== mon.2 v2:10.20.193.78:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe49001ac60 con 0x7fe4a006c890 2026-02-17T06:11:29.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.962+0000 7fe48dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:29.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.962+0000 7fe48dffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:29.962 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:29.963+0000 7fe4a4ce9640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe47003deb0 0x7fe470040370 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe494006fd0 tx=0x7fe494008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:30.042 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.042+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 --> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4a006cc70 con 0x7fe47003deb0 2026-02-17T06:11:30.043 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.043+0000 7fe48dffb640 1 -- 10.20.193.78:0/754454144 <== mgr.4105 v2:10.20.193.115:6800/2827515952 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34029 (secure 0 0 0) 0x7fe4a006cc70 con 0x7fe47003deb0 2026-02-17T06:11:30.043 INFO:teuthology.orchestra.run.trial078.stdout: 2026-02-17T06:11:30.045 INFO:teuthology.orchestra.run.trial078.stderr:dumped all 2026-02-17T06:11:30.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe47003deb0 msgr2=0x7fe470040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:30.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe47003deb0 0x7fe470040370 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe494006fd0 tx=0x7fe494008040 comp rx=0 tx=0).stop 2026-02-17T06:11:30.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 10 monclient: shutdown 2026-02-17T06:11:30.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:30.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006c890 msgr2=0x7fe4a01cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:30.045 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006c890 0x7fe4a01cf160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c600 tx=0x7fe49000cad0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 shutdown_connections 2026-02-17T06:11:30.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7fe4a009c4a0 0x7fe4a01368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7fe47003deb0 0x7fe470040370 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7fe4a006d380 0x7fe4a0136380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 --2- 10.20.193.78:0/754454144 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7fe4a006c890 0x7fe4a01cf160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.046 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 >> 10.20.193.78:0/754454144 conn(0x7fe4a00a6620 msgr2=0x7fe4a00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:30.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 shutdown_connections 2026-02-17T06:11:30.047 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.044+0000 7fe4a7775640 1 -- 10.20.193.78:0/754454144 wait complete. 2026-02-17T06:11:30.051 INFO:teuthology.orchestra.run.trial078.stdout:{"pg_ready":true,"pg_map":{"version":1619,"stamp":"2026-02-17T06:11:28.067288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":296288,"kb_used_data":24896,"kb_used_omap":9,"kb_used_meta":271286,"kb_avail":565934752,"statfs":{"total":579820584960,"available":579517186048,"internally_reserved":0,"allocated":25493504,"data_stored":12055756,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":277797542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000985"},"pg_stats":[{"pgid":"1.0","version":"867'114","reported_seq":1830,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:09.601860+0000","last_change":"2026-02-17T06:11:09.601880+0000","last_active":"2026-02-17T06:11:09.601860+0000","last_peered":"2026-02-17T06:11:09.601860+0000","last_clean":"2026-02-17T06:11:09.601860+0000","last_became_active":"2026-02-17T05:46:05.167482+0000","last_became_peered":"2026-02-17T05:46:05.167482+0000","last_unstale":"2026-02-17T06:11:09.601860+0000","last_undegraded":"2026-02-17T06:11:09.601860+0000","last_fullsized":"2026-02-17T06:11:09.601860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"867'114","last_scrub_stamp":"2026-02-17T06:11:09.601821+0000","last_deep_scrub":"867'114","last_deep_scrub_stamp":"2026-02-17T06:11:09.601821+0000","last_clean_scrub_stamp":"2026-02-17T06:11:09.601821+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:48:15.790558+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"867'2","reported_seq":1693,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:12.663727+0000","last_change":"2026-02-17T06:11:12.663727+0000","last_active":"2026-02-17T06:11:12.663727+0000","last_peered":"2026-02-17T06:11:12.663727+0000","last_clean":"2026-02-17T06:11:12.663727+0000","last_became_active":"2026-02-17T05:46:10.183915+0000","last_became_peered":"2026-02-17T05:46:10.183915+0000","last_unstale":"2026-02-17T06:11:12.663727+0000","last_undegraded":"2026-02-17T06:11:12.663727+0000","last_fullsized":"2026-02-17T06:11:12.663727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"867'2","last_scrub_stamp":"2026-02-17T06:11:12.663679+0000","last_deep_scrub":"867'2","last_deep_scrub_stamp":"2026-02-17T06:11:12.663679+0000","last_clean_scrub_stamp":"2026-02-17T06:11:12.663679+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T12:23:36.537788+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00022683299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1702,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:09.131303+0000","last_change":"2026-02-17T06:11:09.131313+0000","last_active":"2026-02-17T06:11:09.131303+0000","last_peered":"2026-02-17T06:11:09.131303+0000","last_clean":"2026-02-17T06:11:09.131303+0000","last_became_active":"2026-02-17T05:46:10.179745+0000","last_became_peered":"2026-02-17T05:46:10.179745+0000","last_unstale":"2026-02-17T06:11:09.131303+0000","last_undegraded":"2026-02-17T06:11:09.131303+0000","last_fullsized":"2026-02-17T06:11:09.131303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:11:09.131257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:11:09.131257+0000","last_clean_scrub_stamp":"2026-02-17T06:11:09.131257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T07:09:19.591083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00017645900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1712,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:09.486333+0000","last_change":"2026-02-17T06:11:09.486333+0000","last_active":"2026-02-17T06:11:09.486333+0000","last_peered":"2026-02-17T06:11:09.486333+0000","last_clean":"2026-02-17T06:11:09.486333+0000","last_became_active":"2026-02-17T05:46:10.179943+0000","last_became_peered":"2026-02-17T05:46:10.179943+0000","last_unstale":"2026-02-17T06:11:09.486333+0000","last_undegraded":"2026-02-17T06:11:09.486333+0000","last_fullsized":"2026-02-17T06:11:09.486333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:11:09.486278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:11:09.486278+0000","last_clean_scrub_stamp":"2026-02-17T06:11:09.486278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T10:01:47.825950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294758,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"867'3","reported_seq":1707,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:09.345157+0000","last_change":"2026-02-17T06:11:09.345157+0000","last_active":"2026-02-17T06:11:09.345157+0000","last_peered":"2026-02-17T06:11:09.345157+0000","last_clean":"2026-02-17T06:11:09.345157+0000","last_became_active":"2026-02-17T05:46:10.181200+0000","last_became_peered":"2026-02-17T05:46:10.181200+0000","last_unstale":"2026-02-17T06:11:09.345157+0000","last_undegraded":"2026-02-17T06:11:09.345157+0000","last_fullsized":"2026-02-17T06:11:09.345157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"867'3","last_scrub_stamp":"2026-02-17T06:11:09.345120+0000","last_deep_scrub":"867'3","last_deep_scrub_stamp":"2026-02-17T06:11:09.345120+0000","last_clean_scrub_stamp":"2026-02-17T06:11:09.345120+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:10:45.634202+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000398484,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1689,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:10.296251+0000","last_change":"2026-02-17T06:11:10.296251+0000","last_active":"2026-02-17T06:11:10.296251+0000","last_peered":"2026-02-17T06:11:10.296251+0000","last_clean":"2026-02-17T06:11:10.296251+0000","last_became_active":"2026-02-17T05:46:10.180707+0000","last_became_peered":"2026-02-17T05:46:10.180707+0000","last_unstale":"2026-02-17T06:11:10.296251+0000","last_undegraded":"2026-02-17T06:11:10.296251+0000","last_fullsized":"2026-02-17T06:11:10.296251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:11:10.296211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:11:10.296211+0000","last_clean_scrub_stamp":"2026-02-17T06:11:10.296211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T06:27:53.377031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021927700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1691,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:11.626029+0000","last_change":"2026-02-17T06:11:11.626029+0000","last_active":"2026-02-17T06:11:11.626029+0000","last_peered":"2026-02-17T06:11:11.626029+0000","last_clean":"2026-02-17T06:11:11.626029+0000","last_became_active":"2026-02-17T05:46:10.180654+0000","last_became_peered":"2026-02-17T05:46:10.180654+0000","last_unstale":"2026-02-17T06:11:11.626029+0000","last_undegraded":"2026-02-17T06:11:11.626029+0000","last_fullsized":"2026-02-17T06:11:11.626029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:11:11.625964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:11:11.625964+0000","last_clean_scrub_stamp":"2026-02-17T06:11:11.625964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T14:59:11.173721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000368568,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1687,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:09.535102+0000","last_change":"2026-02-17T06:11:09.535114+0000","last_active":"2026-02-17T06:11:09.535102+0000","last_peered":"2026-02-17T06:11:09.535102+0000","last_clean":"2026-02-17T06:11:09.535102+0000","last_became_active":"2026-02-17T05:46:10.180302+0000","last_became_peered":"2026-02-17T05:46:10.180302+0000","last_unstale":"2026-02-17T06:11:09.535102+0000","last_undegraded":"2026-02-17T06:11:09.535102+0000","last_fullsized":"2026-02-17T06:11:09.535102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:11:09.535059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:11:09.535059+0000","last_clean_scrub_stamp":"2026-02-17T06:11:09.535059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T13:46:21.169994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025251899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1690,"reported_epoch":867,"state":"active+clean","last_fresh":"2026-02-17T06:11:10.612296+0000","last_change":"2026-02-17T06:11:10.612321+0000","last_active":"2026-02-17T06:11:10.612296+0000","last_peered":"2026-02-17T06:11:10.612296+0000","last_clean":"2026-02-17T06:11:10.612296+0000","last_became_active":"2026-02-17T05:46:10.180272+0000","last_became_peered":"2026-02-17T05:46:10.180272+0000","last_unstale":"2026-02-17T06:11:10.612296+0000","last_undegraded":"2026-02-17T06:11:10.612296+0000","last_fullsized":"2026-02-17T06:11:10.612296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T06:11:10.612181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T06:11:10.612181+0000","last_clean_scrub_stamp":"2026-02-17T06:11:10.612181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T17:09:45.635083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00021717500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":64110,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31148,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94340692,"statfs":{"total":96636764160,"available":96604868608,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640567,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61620,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310220,"statfs":{"total":96636764160,"available":96573665280,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53296,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318544,"statfs":{"total":96636764160,"available":96582189056,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61616,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94310224,"statfs":{"total":96636764160,"available":96573669376,"internally_reserved":0,"allocated":4227072,"data_stored":1832977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607860,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35308,"kb_used_data":4188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94336532,"statfs":{"total":96636764160,"available":96600608768,"internally_reserved":0,"allocated":4288512,"data_stored":2361905,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607862,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53300,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318540,"statfs":{"total":96636764160,"available":96582184960,"internally_reserved":0,"allocated":4231168,"data_stored":1832996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":32055,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T06:11:30.053 DEBUG:teuthology.orchestra.run.trial078:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.118+0000 7f25534a0640 10 monclient: get_monmap_and_config 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.118+0000 7f25534a0640 10 monclient: build_initial_monmap 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.118+0000 7f25534a0640 10 monclient: monmap: 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T06:11:30.119124+0000 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T06:11:30.119124+0000 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:30.118 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:30.119 INFO:teuthology.orchestra.run.trial078.stderr:0: v1:10.20.193.78:6789/0 mon.noname-a-legacy 2026-02-17T06:11:30.119 INFO:teuthology.orchestra.run.trial078.stderr:1: v1:10.20.193.115:6789/0 mon.noname-c-legacy 2026-02-17T06:11:30.119 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-b 2026-02-17T06:11:30.119 INFO:teuthology.orchestra.run.trial078.stderr:3: v2:10.20.193.78:3300/0 mon.noname-a 2026-02-17T06:11:30.119 INFO:teuthology.orchestra.run.trial078.stderr:4: v2:10.20.193.115:3300/0 mon.noname-c 2026-02-17T06:11:30.119 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 1 Processor -- start 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 1 -- start start 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient: init 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 1 --2- >> v2:10.20.193.115:3300/0 conn(0x7f254c134df0 0x7f254c0a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient(hunting): picked mon.noname-c con 0x7f254c134df0 addr v2:10.20.193.115:3300/0 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c1351d0 0x7f254c135640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient(hunting): picked mon.noname-b con 0x7f254c1351d0 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f254c09db20 addr v1:10.20.193.78:6789/0 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 1 -- --> v1:10.20.193.78:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254c0a26d0 con 0x7f254c09db20 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 1 -- --> v2:10.20.193.115:3300/0 -- mon_getmap magic: 0 -- 0x7f254c09f480 con 0x7f254c134df0 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f254c09f760 con 0x7f254c1351d0 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f25534a0640 10 monclient(hunting): authenticate will time out at 2135.672922s 2026-02-17T06:11:30.120 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f2551a16640 1 --1- >> v1:10.20.193.78:6789/0 conn(0x7f254c09db20 0x7f254c09df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.78:6789/0 says I am v1:10.20.193.78:35612/0 (socket says 10.20.193.78:35612) 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.119+0000 7f2550a14640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c1351d0 0x7f254c135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2551a16640 1 -- 10.20.193.78:0/723653865 learned_addr learned my addr 10.20.193.78:0/723653865 (peer_addr_for_me v1:10.20.193.78:0/0) 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient(hunting): get_auth_request con 0x7f254c1351d0 auth_method 0 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2551215640 1 --2- 10.20.193.78:0/723653865 >> v2:10.20.193.115:3300/0 conn(0x7f254c134df0 0x7f254c0a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2551215640 10 monclient(hunting): get_auth_request con 0x7f254c134df0 auth_method 0 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2551215640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2551215640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2551215640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient(hunting): handle_auth_done global_id 264617 payload 274 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 1 -- 10.20.193.78:0/723653865 >> v2:10.20.193.115:3300/0 conn(0x7f254c134df0 msgr2=0x7f254c0a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 1 --2- 10.20.193.78:0/723653865 >> v2:10.20.193.115:3300/0 conn(0x7f254c134df0 0x7f254c0a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 1 -- 10.20.193.78:0/723653865 >> v1:10.20.193.78:6789/0 conn(0x7f254c09db20 legacy=0x7f254c09df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient: _finish_hunting 0 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 1 monclient: found mon.noname-b 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.78:3301/0 2026-02-17T06:11:30.121 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.120+0000 7f2550a14640 1 -- 10.20.193.78:0/723653865 --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c09fa40 con 0x7f254c1351d0 2026-02-17T06:11:30.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f2550a14640 1 -- 10.20.193.78:0/723653865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c1351d0 msgr2=0x7f254c135640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:30.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f2550a14640 1 --2- 10.20.193.78:0/723653865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c1351d0 0x7f254c135640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f25400098a0 tx=0x7f254002ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa4d8544059e248a server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:30.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 1 -- 10.20.193.78:0/723653865 <== mon.2 v2:10.20.193.78:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254003c070 con 0x7f254c1351d0 2026-02-17T06:11:30.122 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: dump: 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: _finish_auth 0 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: _check_auth_tickets 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 1 -- 10.20.193.78:0/723653865 <== mon.2 v2:10.20.193.78:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f254002fc00 con 0x7f254c1351d0 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f25534a0640 5 monclient: authenticate success, global_id 264617 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f25534a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f25534a0640 10 monclient: get_monmap_and_config success 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 1 -- 10.20.193.78:0/723653865 <== mon.2 v2:10.20.193.78:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540037440 con 0x7f254c1351d0 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f25534a0640 4 set_mon_vals no callback set 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.122+0000 7f253bfff640 10 monclient: dump: 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:30.123 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 10 monclient: shutdown 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 -- 10.20.193.78:0/723653865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c1351d0 msgr2=0x7f254c135640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 --2- 10.20.193.78:0/723653865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c1351d0 0x7f254c135640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f25400098a0 tx=0x7f254002ed80 comp rx=0 tx=0).stop 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 -- 10.20.193.78:0/723653865 shutdown_connections 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 --2- 10.20.193.78:0/723653865 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c1351d0 0x7f254c135640 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 --2- 10.20.193.78:0/723653865 >> v2:10.20.193.115:3300/0 conn(0x7f254c134df0 0x7f254c0a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 -- 10.20.193.78:0/723653865 >> 10.20.193.78:0/723653865 conn(0x7f254c0a3300 msgr2=0x7f254c0a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 -- 10.20.193.78:0/723653865 shutdown_connections 2026-02-17T06:11:30.124 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 -- 10.20.193.78:0/723653865 wait complete. 2026-02-17T06:11:30.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 10 monclient: build_initial_monmap 2026-02-17T06:11:30.125 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 10 monclient: monmap: 2026-02-17T06:11:30.125 INFO:teuthology.orchestra.run.trial078.stderr:epoch 0 2026-02-17T06:11:30.125 INFO:teuthology.orchestra.run.trial078.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-17T06:11:30.125 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 0.000000 2026-02-17T06:11:30.125 INFO:teuthology.orchestra.run.trial078.stderr:created 0.000000 2026-02-17T06:11:30.125 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 0 (unknown) 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.noname-a 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.noname-b 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.noname-c 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 Processor -- start 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 1 -- start start 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.123+0000 7f25534a0640 10 monclient: init 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient: _reopen_session rank -1 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 1 --2- >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c09db20 0x7f254c1cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient(hunting): picked mon.noname-c con 0x7f254c09db20 addr [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f254c134df0 0x7f254c1cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient(hunting): picked mon.noname-a con 0x7f254c134df0 addr [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 1 --2- >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f254c1351d0 0x7f254c1248c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient(hunting): picked mon.noname-b con 0x7f254c1351d0 addr [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 1 -- --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_getmap magic: 0 -- 0x7f254c09d220 con 0x7f254c134df0 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 1 -- --> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] -- mon_getmap magic: 0 -- 0x7f254c09d0a0 con 0x7f254c1351d0 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient(hunting): start opening mon connection 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 1 -- --> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] -- mon_getmap magic: 0 -- 0x7f254c09e6a0 con 0x7f254c09db20 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient(hunting): _renew_subs 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f25534a0640 10 monclient(hunting): authenticate will time out at 2135.677105s 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f254c134df0 0x7f254c1cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 --2- >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f254c134df0 0x7f254c1cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.78:3300/0 says I am v2:10.20.193.78:34320/0 (socket says 10.20.193.78:34320) 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 -- 10.20.193.78:0/2764614932 learned_addr learned my addr 10.20.193.78:0/2764614932 (peer_addr_for_me v2:10.20.193.78:0/0) 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient(hunting): get_auth_request con 0x7f254c134df0 auth_method 0 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:30.126 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551215640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c09db20 0x7f254c1cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551215640 10 monclient(hunting): get_auth_request con 0x7f254c09db20 auth_method 0 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551215640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551215640 10 monclient(hunting): _init_auth method 2 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551215640 10 monclient(hunting): _init_auth creating new auth 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551215640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551215640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551215640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient(hunting): handle_auth_done global_id 284415 payload 824 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 -- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c09db20 msgr2=0x7f254c1cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c09db20 0x7f254c1cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551a16640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f254c1351d0 0x7f254c1248c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 -- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f254c1351d0 msgr2=0x7f254c1248c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f254c1351d0 0x7f254c1248c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient: _finish_hunting 0 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 monclient: found mon.noname-a 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.78:3300/0 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2550a14640 1 -- 10.20.193.78:0/2764614932 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c129230 con 0x7f254c134df0 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551a16640 10 monclient: get_auth_request con 0x7f254c1351d0 auth_method 0 2026-02-17T06:11:30.127 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.124+0000 7f2551a16640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f254c1351d0 0x7f254c1248c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-17T06:11:30.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2550a14640 1 -- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f254c134df0 msgr2=0x7f254c1cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-17T06:11:30.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2550a14640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f254c134df0 0x7f254c1cfb00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2540002410 tx=0x7f25400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:30.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 1 -- 10.20.193.78:0/2764614932 <== mon.0 v2:10.20.193.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254003c040 con 0x7f254c134df0 2026-02-17T06:11:30.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:30.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-17T06:11:30.128 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: dump: 2026-02-17T06:11:30.128 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:30.128 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: _finish_auth 0 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: _check_auth_tickets 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 1 -- 10.20.193.78:0/2764614932 <== mon.0 v2:10.20.193.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2540047070 con 0x7f254c134df0 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f25534a0640 5 monclient: authenticate success, global_id 284415 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f25534a0640 10 monclient: _renew_subs 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f25534a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c128570 con 0x7f254c134df0 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: handle_config config(0 keys) 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 1 -- 10.20.193.78:0/2764614932 <== mon.0 v2:10.20.193.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540040a40 con 0x7f254c134df0 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f253b7fe640 4 set_mon_vals no callback set 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f2539ffb640 10 monclient: dump: 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:epoch 1 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:fsid 5df7382e-3ebf-45af-b855-6719898a3119 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:last_changed 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:created 2026-02-17T05:45:23.713551+0000 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:min_mon_release 19 (squid) 2026-02-17T06:11:30.129 INFO:teuthology.orchestra.run.trial078.stderr:election_strategy: 1 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:0: [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] mon.a 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:1: [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] mon.b 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2: [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] mon.c 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f25534a0640 10 monclient: _renew_subs 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f25534a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.126+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c0a5ea0 con 0x7f254c134df0 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f25534a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f25534a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f25534a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254c09ea80 con 0x7f254c134df0 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f2539ffb640 1 -- 10.20.193.78:0/2764614932 <== mon.0 v2:10.20.193.78:3300/0 4 ==== mgrmap(e 5) ==== 50338+0+0 (secure 0 0 0) 0x7f254003f030 con 0x7f254c134df0 2026-02-17T06:11:30.130 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f2539ffb640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f251c03deb0 0x7f251c040370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-17T06:11:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f2539ffb640 1 -- 10.20.193.78:0/2764614932 <== mon.0 v2:10.20.193.78:3300/0 5 ==== osd_map(867..867 src has 525..867) ==== 4386+0+0 (secure 0 0 0) 0x7f2540077a50 con 0x7f254c134df0 2026-02-17T06:11:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f2551215640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f251c03deb0 0x7f251c040370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-17T06:11:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.127+0000 7f2551215640 10 monclient: get_auth_request con 0x7f251c03deb0 auth_method 0 2026-02-17T06:11:30.131 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.130+0000 7f2539ffb640 1 -- 10.20.193.78:0/2764614932 <== mon.0 v2:10.20.193.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f254c129230 con 0x7f254c134df0 2026-02-17T06:11:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.130+0000 7f2539ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-17T06:11:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.130+0000 7f2539ffb640 10 monclient: _finish_command 1 = system:0 2026-02-17T06:11:30.132 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.130+0000 7f2551215640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f251c03deb0 0x7f251c040370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f253c009a50 tx=0x7f253c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-17T06:11:30.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.212+0000 7f25534a0640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-17T06:11:30.212 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.212+0000 7f25534a0640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-17T06:11:30.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.212+0000 7f25534a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.78:3300/0 2026-02-17T06:11:30.213 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.212+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 --> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f254c126e40 con 0x7f254c134df0 2026-02-17T06:11:30.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.214+0000 7f2539ffb640 1 -- 10.20.193.78:0/2764614932 <== mon.0 v2:10.20.193.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v4)=0 v4) ==== 123+0+0 (secure 0 0 0) 0x7f254002fba0 con 0x7f254c134df0 2026-02-17T06:11:30.214 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.214+0000 7f2539ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-17T06:11:30.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.214+0000 7f2539ffb640 10 monclient: _finish_command 2 = system:0 2026-02-17T06:11:30.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.214+0000 7f2539ffb640 1 -- 10.20.193.78:0/2764614932 <== mon.0 v2:10.20.193.78:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f254007d080 con 0x7f254c134df0 2026-02-17T06:11:30.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.214+0000 7f2539ffb640 10 monclient: handle_config config(1 keys) 2026-02-17T06:11:30.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.214+0000 7f253affd640 4 set_mon_vals no callback set 2026-02-17T06:11:30.215 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f253affd640 10 set_mon_vals mon_health_to_clog = false 2026-02-17T06:11:30.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f251c03deb0 msgr2=0x7f251c040370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:30.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f251c03deb0 0x7f251c040370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f253c009a50 tx=0x7f253c008040 comp rx=0 tx=0).stop 2026-02-17T06:11:30.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 10 monclient: shutdown 2026-02-17T06:11:30.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-17T06:11:30.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f254c134df0 msgr2=0x7f254c1cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-17T06:11:30.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f254c134df0 0x7f254c1cfb00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2540002410 tx=0x7f25400047c0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.216 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 shutdown_connections 2026-02-17T06:11:30.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:6800/2827515952,v1:10.20.193.115:6801/2827515952] conn(0x7f251c03deb0 0x7f251c040370 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.115:3300/0,v1:10.20.193.115:6789/0] conn(0x7f254c1351d0 0x7f254c1248c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3300/0,v1:10.20.193.78:6789/0] conn(0x7f254c134df0 0x7f254c1cfb00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 --2- 10.20.193.78:0/2764614932 >> [v2:10.20.193.78:3301/0,v1:10.20.193.78:6790/0] conn(0x7f254c09db20 0x7f254c1cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-17T06:11:30.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 >> 10.20.193.78:0/2764614932 conn(0x7f254c0a3300 msgr2=0x7f254c0a0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-17T06:11:30.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 shutdown_connections 2026-02-17T06:11:30.217 INFO:teuthology.orchestra.run.trial078.stderr:2026-02-17T06:11:30.215+0000 7f25534a0640 1 -- 10.20.193.78:0/2764614932 wait complete. 2026-02-17T06:11:30.223 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-17T06:11:30.223 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-17T06:11:30.223 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-17T06:11:30.223 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.258 INFO:tasks.ceph.osd.0:Stopped 2026-02-17T06:11:30.258 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-17T06:11:30.258 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.334 INFO:tasks.ceph.osd.1:Stopped 2026-02-17T06:11:30.334 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-17T06:11:30.334 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.405 INFO:tasks.ceph.osd.2:Stopped 2026-02-17T06:11:30.406 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-17T06:11:30.406 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.442 INFO:tasks.ceph.osd.3:Stopped 2026-02-17T06:11:30.442 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-17T06:11:30.442 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.517 INFO:tasks.ceph.osd.4:Stopped 2026-02-17T06:11:30.517 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-17T06:11:30.517 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.587 INFO:tasks.ceph.osd.5:Stopped 2026-02-17T06:11:30.587 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-17T06:11:30.587 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-17T06:11:30.587 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.658 INFO:tasks.ceph.mgr.x:Stopped 2026-02-17T06:11:30.658 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-17T06:11:30.658 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T06:11:30.658 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.669 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T06:11:30.669 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-17T06:11:30.669 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.720 INFO:tasks.ceph.mon.c:Stopped 2026-02-17T06:11:30.721 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-17T06:11:30.721 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T06:11:30.730 INFO:tasks.ceph.mon.b:Stopped 2026-02-17T06:11:30.730 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-17T06:11:30.731 DEBUG:teuthology.orchestra.run.trial078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-17T06:11:30.789 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial078.front.sepia.ceph.com 2026-02-17T06:11:30.790 DEBUG:teuthology.orchestra.run.trial078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-17T06:11:30.865 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial078.front.sepia.ceph.com 2026-02-17T06:11:30.866 DEBUG:teuthology.orchestra.run.trial078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-17T06:11:30.940 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial078.front.sepia.ceph.com 2026-02-17T06:11:30.940 DEBUG:teuthology.orchestra.run.trial078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-17T06:11:31.011 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial115.front.sepia.ceph.com 2026-02-17T06:11:31.011 DEBUG:teuthology.orchestra.run.trial115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-17T06:11:31.052 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@trial115.front.sepia.ceph.com 2026-02-17T06:11:31.052 DEBUG:teuthology.orchestra.run.trial115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2026-02-17T06:11:31.121 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@trial115.front.sepia.ceph.com 2026-02-17T06:11:31.122 DEBUG:teuthology.orchestra.run.trial115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2026-02-17T06:11:31.191 INFO:tasks.ceph:Archiving mon data... 2026-02-17T06:11:31.192 DEBUG:teuthology.misc:Transferring archived files from trial078:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/data/mon.a.tgz 2026-02-17T06:11:31.193 DEBUG:teuthology.orchestra.run.trial078:> mktemp 2026-02-17T06:11:31.212 INFO:teuthology.orchestra.run.trial078.stdout:/tmp/tmp.1teOB0QSA2 2026-02-17T06:11:31.212 DEBUG:teuthology.orchestra.run.trial078:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.1teOB0QSA2 2026-02-17T06:11:31.357 DEBUG:teuthology.orchestra.run.trial078:> sudo chmod 0666 /tmp/tmp.1teOB0QSA2 2026-02-17T06:11:31.437 DEBUG:teuthology.orchestra.remote:trial078:/tmp/tmp.1teOB0QSA2 is 830KB 2026-02-17T06:11:31.543 DEBUG:teuthology.orchestra.run.trial078:> rm -fr /tmp/tmp.1teOB0QSA2 2026-02-17T06:11:31.560 DEBUG:teuthology.misc:Transferring archived files from trial078:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/data/mon.c.tgz 2026-02-17T06:11:31.561 DEBUG:teuthology.orchestra.run.trial078:> mktemp 2026-02-17T06:11:31.617 INFO:teuthology.orchestra.run.trial078.stdout:/tmp/tmp.FcvrKbLPBU 2026-02-17T06:11:31.618 DEBUG:teuthology.orchestra.run.trial078:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.FcvrKbLPBU 2026-02-17T06:11:31.761 DEBUG:teuthology.orchestra.run.trial078:> sudo chmod 0666 /tmp/tmp.FcvrKbLPBU 2026-02-17T06:11:31.841 DEBUG:teuthology.orchestra.remote:trial078:/tmp/tmp.FcvrKbLPBU is 830KB 2026-02-17T06:11:31.937 DEBUG:teuthology.orchestra.run.trial078:> rm -fr /tmp/tmp.FcvrKbLPBU 2026-02-17T06:11:31.954 DEBUG:teuthology.misc:Transferring archived files from trial115:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/data/mon.b.tgz 2026-02-17T06:11:31.954 DEBUG:teuthology.orchestra.run.trial115:> mktemp 2026-02-17T06:11:31.972 INFO:teuthology.orchestra.run.trial115.stdout:/tmp/tmp.JX8xsQkgyQ 2026-02-17T06:11:31.973 DEBUG:teuthology.orchestra.run.trial115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.JX8xsQkgyQ 2026-02-17T06:11:32.116 DEBUG:teuthology.orchestra.run.trial115:> sudo chmod 0666 /tmp/tmp.JX8xsQkgyQ 2026-02-17T06:11:32.198 DEBUG:teuthology.orchestra.remote:trial115:/tmp/tmp.JX8xsQkgyQ is 829KB 2026-02-17T06:11:32.302 DEBUG:teuthology.orchestra.run.trial115:> rm -fr /tmp/tmp.JX8xsQkgyQ 2026-02-17T06:11:32.321 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-17T06:11:32.321 DEBUG:teuthology.orchestra.run.trial078:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-17T06:11:32.324 DEBUG:teuthology.orchestra.run.trial115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-17T06:11:32.404 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-17T06:11:32.441 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-17T06:11:32.441 INFO:tasks.ceph:Archiving crash dumps... 2026-02-17T06:11:32.443 DEBUG:teuthology.misc:Transferring archived files from trial078:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/remote/trial078/crash 2026-02-17T06:11:32.444 DEBUG:teuthology.orchestra.run.trial078:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-17T06:11:32.469 DEBUG:teuthology.misc:Transferring archived files from trial115:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/remote/trial115/crash 2026-02-17T06:11:32.470 DEBUG:teuthology.orchestra.run.trial115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-17T06:11:32.506 INFO:tasks.ceph:Compressing logs... 2026-02-17T06:11:32.506 DEBUG:teuthology.orchestra.run.trial078:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:11:32.514 DEBUG:teuthology.orchestra.run.trial115:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:11:32.534 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53121.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42665.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45420.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-client.admin.42665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42607.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-client.0.45420.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42665.log.gz 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-client.admin.42607.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-osd.4.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42607.log.gz 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42466.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42513.log 2026-02-17T06:11:32.580 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-client.admin.42466.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-17T06:11:32.581 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-client.admin.42513.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42466.log.gz 2026-02-17T06:11:32.581 INFO:teuthology.orchestra.run.trial115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42560.log 2026-02-17T06:11:32.581 INFO:teuthology.orchestra.run.trial115.stderr: 90.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-17T06:11:32.581 INFO:teuthology.orchestra.run.trial115.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42513.log.gz 2026-02-17T06:11:32.581 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-17T06:11:32.581 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-client.admin.42560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42712.log 2026-02-17T06:11:32.581 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42560.log.gz 2026-02-17T06:11:32.581 INFO:teuthology.orchestra.run.trial115.stderr:/var/log/ceph/ceph-client.admin.42712.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42712.log.gz 2026-02-17T06:11:32.588 INFO:teuthology.orchestra.run.trial115.stderr: 87.6% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-17T06:11:32.592 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64312.log 2026-02-17T06:11:32.592 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53121.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-17T06:11:32.592 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64312.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53121.log.gz 2026-02-17T06:11:32.592 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46284.log 2026-02-17T06:11:32.592 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48976.log 2026-02-17T06:11:32.592 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.46284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61676.log 2026-02-17T06:11:32.592 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64312.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46284.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55269.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66867.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48976.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61725.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61676.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45587.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55269.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61725.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66867.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57679.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56021.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61725.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45587.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51580.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56021.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57679.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62525.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56021.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51580.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47965.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61331.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47965.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.62525.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67766.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47965.log.gz 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56168.log 2026-02-17T06:11:32.593 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47425.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61331.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48632.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67766.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64116.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56168.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48632.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47425.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63214.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60415.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64116.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48632.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45588.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64116.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60415.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63214.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68212.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54221.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60415.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45588.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46069.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47572.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68212.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51284.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.54221.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68556.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46069.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47572.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68507.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57023.log 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51284.log.gz 2026-02-17T06:11:32.594 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47670.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68556.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57023.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.68507.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64819.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47670.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57023.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64819.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47670.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48014.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59464.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64819.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61267.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59464.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48014.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56778.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61267.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59464.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58778.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51334.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58778.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61267.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69066.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56778.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58778.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56119.log/var/log/ceph/ceph-client.admin.51334.log: 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59366.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69066.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51334.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63165.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69066.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59366.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.56119.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46756.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63821.log 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63165.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59366.log.gz 2026-02-17T06:11:32.595 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49630.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63165.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63821.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46756.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68161.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54828.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49630.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63821.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51482.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49630.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60826.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.68161.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48406.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.54828.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51482.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54613.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57826.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48406.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.60826.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65624.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48406.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71729.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57826.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54613.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48455.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57826.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71729.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65624.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59826.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70112.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71729.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59826.log: 89.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.48455.log.gz 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46155.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57464.log 2026-02-17T06:11:32.596 INFO:teuthology.orchestra.run.trial078.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.59826.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50679.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70112.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46155.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67717.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52530.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57464.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67717.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50679.log.gzgzip 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49319.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67717.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49973.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70651.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52530.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66621.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49319.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50728.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49973.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64165.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.70651.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50728.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66621.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66670.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64165.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50728.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59170.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64165.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63871.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62771.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66670.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63871.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.59170.log.gz 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61022.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46153.log 2026-02-17T06:11:32.597 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63871.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56464.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.62771.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63116.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61022.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56464.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46153.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49221.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54690.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56464.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53778.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.63116.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54690.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49221.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69458.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53778.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54690.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67521.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70161.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67521.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53778.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64721.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69458.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51777.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.67521.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64721.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.70161.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45882.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51777.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64721.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55923.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51777.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57973.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45882.log: /var/log/ceph/ceph-client.admin.55923.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45882.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66572.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57973.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55923.log.gz 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53876.log 2026-02-17T06:11:32.598 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66572.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57973.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61218.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.66572.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61169.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61218.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53876.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49368.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61169.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61218.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51120.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49368.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61169.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50973.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51120.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49368.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63733.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50973.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51120.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59219.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50973.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45059.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59219.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63733.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68967.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45059.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59219.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65015.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68967.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45059.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53337.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65015.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68967.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71533.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65015.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51071.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57415.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.53337.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71533.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61921.log 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51071.log.gz 2026-02-17T06:11:32.599 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42337.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54319.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.42337.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.57415.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61921.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55825.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:gzip/var/log/ceph/ceph-client.admin.54319.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57777.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.42337.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58365.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55825.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.54319.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53484.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57777.log.gz/var/log/ceph/ceph-client.admin.58365.log: 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50875.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46154.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58365.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65477.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53484.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45877.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.50875.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65477.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.46154.log.gz --verbose 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: -- /var/log/ceph/ceph-client.admin.48544.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53927.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.65477.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48259.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45877.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53927.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48544.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67815.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48259.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53927.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48259.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70406.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67915.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46755.log 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67815.log.gz 2026-02-17T06:11:32.600 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57219.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70406.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46755.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67915.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69360.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46755.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57219.log: /var/log/ceph/ceph-client.admin.69360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66916.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57219.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51678.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64067.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.69360.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65113.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66916.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64067.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51678.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64067.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54515.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65113.log: /var/log/ceph/ceph-client.admin.48210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54515.log: gzip -5 --verbose 88.3% -- /var/log/ceph/ceph-client.admin.58071.log -- replaced with /var/log/ceph/ceph-client.admin.65113.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69262.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54515.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58071.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71435.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56316.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58071.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50581.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69262.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56316.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71435.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61578.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56316.log.gz 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50826.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71680.log 2026-02-17T06:11:32.601 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51629.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50581.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61578.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71680.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50826.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65162.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71680.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58827.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51629.log: /var/log/ceph/ceph-client.admin.65162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49777.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45883.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51629.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49777.log: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.65162.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58827.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65869.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49777.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68311.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45883.log: /var/log/ceph/ceph-client.admin.65869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46337.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45883.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68311.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65869.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55576.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46337.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.68311.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54417.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46337.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55972.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55576.log: /var/log/ceph/ceph-client.admin.54417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55527.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54417.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51196.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55576.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58267.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55972.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55527.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62280.log 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58267.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51196.log.gz 2026-02-17T06:11:32.602 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58974.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62280.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58267.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52332.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58974.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.62280.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57121.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52332.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58974.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57121.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52332.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64263.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46680.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57121.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66376.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56513.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66376.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64263.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62476.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56513.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66376.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63459.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62476.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56513.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45590.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63459.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.62476.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52382.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45590.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63459.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52120.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52382.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45590.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61480.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52827.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.52382.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61480.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52120.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45406.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61480.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46631.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52827.log: /var/log/ceph/ceph-client.admin.45406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62969.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.52827.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45406.log.gz 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67178.log 2026-02-17T06:11:32.603 INFO:teuthology.orchestra.run.trial078.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46631.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66523.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59317.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62969.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66523.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67178.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64410.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66523.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61823.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53072.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59317.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61823.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64410.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70210.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53072.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61823.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65722.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67864.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.53072.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65722.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.70210.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57513.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.65722.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54270.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65967.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67864.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54270.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57513.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63920.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65967.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.54270.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45217.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr: 88.3%/var/log/ceph/ceph-client.admin.63920.log: -- replaced with /var/log/ceph/ceph-client.admin.65967.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53170.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45217.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.63920.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64361.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45217.log.gz 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68779.log 2026-02-17T06:11:32.604 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64361.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53170.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48780.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68779.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64361.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51875.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48780.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68779.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66719.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51875.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48780.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52925.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66719.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51875.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60120.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52925.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66719.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70945.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60120.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52925.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58316.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70945.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.60120.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62231.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58316.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70945.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52677.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58316.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63018.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52677.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.62231.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69569.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52677.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65575.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69569.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.63018.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52579.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65575.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69569.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69867.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52579.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65575.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62120.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69867.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52579.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49443.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62120.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69867.log.gz 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42540.log 2026-02-17T06:11:32.605 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54368.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.62120.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.42540.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.49443.log.gz --verbose -- 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr: /var/log/ceph/ceph-client.admin.60491.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58414.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42540.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54368.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59973.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58414.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60491.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55675.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59973.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58414.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62672.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr: 88.6%/var/log/ceph/ceph-client.admin.55675.log: -- replaced with /var/log/ceph/ceph-client.admin.59973.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69818.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62672.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55675.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66817.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69818.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62672.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50121.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69818.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56827.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50121.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66817.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59589.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56827.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50121.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70455.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr: 88.2%/var/log/ceph/ceph-client.admin.59589.log: -- replaced with /var/log/ceph/ceph-client.admin.56827.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57875.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr: 88.2%/var/log/ceph/ceph-client.admin.70455.log: -- replaced with /var/log/ceph/ceph-client.admin.59589.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51973.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57875.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70455.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59777.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51973.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57875.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46906.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59777.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51973.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68458.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46906.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.59777.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54877.log 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68458.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46906.log.gz 2026-02-17T06:11:32.606 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56876.log 2026-02-17T06:11:32.607 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54877.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68458.log.gz 2026-02-17T06:11:32.607 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69718.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52071.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.54877.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69718.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56876.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58218.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52071.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69718.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46828.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr: 88.9%/var/log/ceph/ceph-client.admin.58218.log: -- replaced with /var/log/ceph/ceph-client.admin.52071.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49924.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46828.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58218.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60924.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49924.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46828.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57072.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60924.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49924.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58542.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57072.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60924.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50071.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58542.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57072.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67227.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50071.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58542.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69115.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50071.log.gz 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68063.log 2026-02-17T06:11:32.614 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69115.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.67227.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67668.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68063.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69115.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66114.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67668.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.68063.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62329.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66114.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67668.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63263.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62329.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.66114.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56266.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62329.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66189.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56266.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63263.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62427.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56266.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59513.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62427.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66189.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69620.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62427.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62068.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56562.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59513.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62068.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69620.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49875.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60219.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62068.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54778.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56562.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60219.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.49875.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54466.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54778.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60219.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60777.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54466.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54778.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56640.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60777.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54466.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49123.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56640.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.60777.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66965.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49123.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56640.log.gz 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61071.log 2026-02-17T06:11:32.615 INFO:teuthology.orchestra.run.trial078.stderr: 87.9%/var/log/ceph/ceph-client.admin.66965.log: -- replaced with /var/log/ceph/ceph-client.admin.49123.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48357.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61071.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66965.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50433.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48357.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.61071.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57728.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46976.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48357.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57728.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50433.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63410.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47075.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57728.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49074.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46976.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63410.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49025.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.47075.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55122.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60022.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49074.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55122.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49025.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52628.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60022.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55122.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53023.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52628.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60022.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58120.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53023.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52628.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59415.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58120.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53023.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50630.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59415.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58120.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42607.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64525.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59415.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.42607.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50630.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55776.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42607.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56070.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64525.log: /var/log/ceph/ceph-client.admin.55776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58169.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 88.4%/var/log/ceph/ceph-client.admin.56070.log: -- replaced with /var/log/ceph/ceph-client.admin.55776.log.gz 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65918.log 2026-02-17T06:11:32.616 INFO:teuthology.orchestra.run.trial078.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64525.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58169.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56070.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69409.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65918.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58169.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45147.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69409.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65918.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51433.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45147.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69409.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54172.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51433.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45147.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47719.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47818.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.51433.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54172.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67570.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47719.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71141.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47916.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47818.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71141.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.67570.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49728.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71141.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48927.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49728.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.47916.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61774.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48927.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49728.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46905.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61774.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48927.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66065.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61774.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51383.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46905.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51383.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.66065.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66474.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51383.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60317.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66474.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46070.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66474.log.gz 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61120.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65339.log 2026-02-17T06:11:32.617 INFO:teuthology.orchestra.run.trial078.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60317.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46070.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70896.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61120.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45884.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68262.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65339.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr: 88.3%/var/log/ceph/ceph-client.admin.45884.log: -- replaced with /var/log/ceph/ceph-client.admin.70896.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63067.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45884.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57317.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63067.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68262.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63558.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57317.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63067.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53386.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63558.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57317.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64574.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52022.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.63558.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64574.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53386.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67423.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64574.log.gz 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65771.log 2026-02-17T06:11:32.618 INFO:teuthology.orchestra.run.trial115.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.0.45420.log.gz 2026-02-17T06:11:32.620 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48829.log 2026-02-17T06:11:32.620 INFO:teuthology.orchestra.run.trial078.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52022.log.gz 2026-02-17T06:11:32.620 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65771.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67423.log.gz 2026-02-17T06:11:32.620 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60875.log 2026-02-17T06:11:32.620 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48829.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.65771.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62920.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48829.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53247.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62920.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60875.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47523.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53247.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62920.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52432.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53247.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60678.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64868.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.47523.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60678.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52432.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45357.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60579.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60678.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45357.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.64868.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63361.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45357.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67040.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63969.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60579.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67040.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63361.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49270.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67040.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48112.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70602.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.63969.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68014.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70602.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.49270.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48112.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68869.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70602.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61431.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68014.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68869.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61431.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68869.log.gzgzip 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58680.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61431.log.gz 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50219.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46435.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54975.log 2026-02-17T06:11:32.621 INFO:teuthology.orchestra.run.trial078.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.58680.log.gz 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46435.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50219.log.gz 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66327.log 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46435.log.gz 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59677.log 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54975.log.gz 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70553.log 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59677.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.66327.log.gz 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57268.log 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr: 94.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59677.log.gz 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68112.log 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70553.log.gz 2026-02-17T06:11:32.622 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60268.log 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57268.log.gz 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62574.log 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55318.log 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68112.log.gz 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60268.log.gz 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63656.log 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55318.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62574.log.gz 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69213.log 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63656.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55318.log.gz 2026-02-17T06:11:32.623 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59023.log 2026-02-17T06:11:32.624 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69213.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63656.log.gz 2026-02-17T06:11:32.624 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53680.log 2026-02-17T06:11:32.624 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59023.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69213.log.gz 2026-02-17T06:11:32.624 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63312.log 2026-02-17T06:11:32.624 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53680.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59023.log.gz 2026-02-17T06:11:32.624 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64018.log 2026-02-17T06:11:32.624 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63312.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53680.log.gz 2026-02-17T06:11:32.624 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51022.log 2026-02-17T06:11:32.625 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64018.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.63312.log.gz 2026-02-17T06:11:32.625 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57170.log 2026-02-17T06:11:32.625 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55416.log 2026-02-17T06:11:32.625 INFO:teuthology.orchestra.run.trial078.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.64018.log.gz 2026-02-17T06:11:32.625 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45242.log 2026-02-17T06:11:32.625 INFO:teuthology.orchestra.run.trial078.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51022.log.gz 2026-02-17T06:11:32.625 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55220.log 2026-02-17T06:11:32.625 INFO:teuthology.orchestra.run.trial078.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57170.log.gz 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45242.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55416.log.gz 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55220.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45242.log.gz 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47769.log 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50022.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55220.log.gz 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71337.log 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47769.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53533.log 2026-02-17T06:11:32.626 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71337.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47769.log.gz 2026-02-17T06:11:32.627 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49679.log 2026-02-17T06:11:32.627 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71337.log.gz/var/log/ceph/ceph-client.admin.53533.log: 2026-02-17T06:11:32.627 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45879.log 2026-02-17T06:11:32.627 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49679.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53533.log.gz 2026-02-17T06:11:32.627 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60366.log 2026-02-17T06:11:32.627 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45879.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49679.log.gz 2026-02-17T06:11:32.627 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-17T06:11:32.628 INFO:teuthology.orchestra.run.trial078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45879.log.gz 2026-02-17T06:11:32.628 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70259.log 2026-02-17T06:11:32.628 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-osd.1.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60366.log.gz 2026-02-17T06:11:32.628 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62623.log 2026-02-17T06:11:32.628 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2026-02-17T06:11:32.628 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62623.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.70259.log.gz 2026-02-17T06:11:32.628 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71631.log 2026-02-17T06:11:32.628 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62623.log.gz 2026-02-17T06:11:32.629 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54025.log 2026-02-17T06:11:32.629 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50532.log: /var/log/ceph/ceph-client.admin.71631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50345.log 2026-02-17T06:11:32.629 INFO:teuthology.orchestra.run.trial078.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2026-02-17T06:11:32.629 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54025.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71631.log.gz 2026-02-17T06:11:32.629 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70798.log 2026-02-17T06:11:32.629 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50345.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54025.log.gz 2026-02-17T06:11:32.629 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61380.log 2026-02-17T06:11:32.630 INFO:teuthology.orchestra.run.trial078.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50345.log.gz/var/log/ceph/ceph-client.admin.70798.log: 2026-02-17T06:11:32.630 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70504.log 2026-02-17T06:11:32.630 INFO:teuthology.orchestra.run.trial078.stderr: 88.7%/var/log/ceph/ceph-client.admin.61380.log: -- replaced with /var/log/ceph/ceph-client.admin.70798.log.gz 2026-02-17T06:11:32.630 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47278.log 2026-02-17T06:11:32.630 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47621.log 2026-02-17T06:11:32.630 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61380.log.gz 2026-02-17T06:11:32.631 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47278.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70504.log.gz 2026-02-17T06:11:32.631 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61872.log 2026-02-17T06:11:32.631 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47621.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.47278.log.gz 2026-02-17T06:11:32.631 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50777.log 2026-02-17T06:11:32.631 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68360.log 2026-02-17T06:11:32.631 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47621.log.gz 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50777.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61872.log.gz 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59268.log 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50777.log.gz 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53827.log 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59268.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.68360.log.gz 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52481.log 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53827.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59268.log.gz 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45589.log 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52481.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53827.log.gz 2026-02-17T06:11:32.632 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58925.log 2026-02-17T06:11:32.633 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49581.log 2026-02-17T06:11:32.633 INFO:teuthology.orchestra.run.trial078.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52481.log.gz 2026-02-17T06:11:32.633 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58925.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45589.log.gz 2026-02-17T06:11:32.633 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56728.log 2026-02-17T06:11:32.633 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58925.log.gz 2026-02-17T06:11:32.633 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71092.log 2026-02-17T06:11:32.633 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2026-02-17T06:11:32.634 INFO:teuthology.orchestra.run.trial078.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49581.log.gz 2026-02-17T06:11:32.634 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71092.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56728.log.gz 2026-02-17T06:11:32.634 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71582.log 2026-02-17T06:11:32.634 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47376.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71092.log.gz 2026-02-17T06:11:32.634 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67619.log 2026-02-17T06:11:32.634 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71582.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67472.log 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45586.log 88.5% 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71582.log.gz 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr: 88.3%/var/log/ceph/ceph-client.admin.67472.log: -- replaced with /var/log/ceph/ceph-client.admin.67619.log.gz 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65263.log 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45586.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.67472.log.gz 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65263.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45586.log.gz 2026-02-17T06:11:32.635 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57924.log 2026-02-17T06:11:32.636 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50268.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65263.log.gz 2026-02-17T06:11:32.636 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69520.log 2026-02-17T06:11:32.636 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57924.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2026-02-17T06:11:32.636 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61627.log 2026-02-17T06:11:32.636 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69520.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57924.log.gz 2026-02-17T06:11:32.636 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67128.log 2026-02-17T06:11:32.636 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61627.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69520.log.gz 2026-02-17T06:11:32.636 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46533.log 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67128.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61627.log.gz 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71484.log 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46533.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67128.log.gz 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47867.log 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71484.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46533.log.gz 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48731.log 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47867.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71484.log.gz 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69965.log 2026-02-17T06:11:32.637 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48731.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47867.log.gz 2026-02-17T06:11:32.638 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56974.log 2026-02-17T06:11:32.638 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48731.log.gz 2026-02-17T06:11:32.638 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60728.log 2026-02-17T06:11:32.638 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56974.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69965.log.gz 2026-02-17T06:11:32.638 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70063.log 2026-02-17T06:11:32.638 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60728.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56974.log.gz 2026-02-17T06:11:32.638 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64214.log 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70063.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.60728.log.gz 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70357.log 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64214.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70063.log.gz 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48063.log 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70357.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64214.log.gz 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53631.log 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48063.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.70357.log.gz 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65214.log 2026-02-17T06:11:32.639 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53631.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48063.log.gz 2026-02-17T06:11:32.640 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66278.log 2026-02-17T06:11:32.640 INFO:teuthology.orchestra.run.trial078.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53631.log.gz 2026-02-17T06:11:32.640 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59875.log 2026-02-17T06:11:32.640 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66278.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.65214.log.gz 2026-02-17T06:11:32.640 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65526.log 2026-02-17T06:11:32.640 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52778.log 2026-02-17T06:11:32.640 INFO:teuthology.orchestra.run.trial078.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.66278.log.gz 2026-02-17T06:11:32.641 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65526.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.59875.log.gz 2026-02-17T06:11:32.641 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47229.log 2026-02-17T06:11:32.641 INFO:teuthology.orchestra.run.trial078.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.65526.log.gz 2026-02-17T06:11:32.641 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59728.log 2026-02-17T06:11:32.641 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64770.log 2026-02-17T06:11:32.641 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52778.log.gz 2026-02-17T06:11:32.641 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59728.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.47229.log.gz 2026-02-17T06:11:32.641 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53582.log 2026-02-17T06:11:32.642 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71190.log 2026-02-17T06:11:32.642 INFO:teuthology.orchestra.run.trial078.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.59728.log.gz 2026-02-17T06:11:32.642 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53582.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64770.log.gz 2026-02-17T06:11:32.642 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56925.log 2026-02-17T06:11:32.642 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71190.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53582.log.gz 2026-02-17T06:11:32.642 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54926.log 2026-02-17T06:11:32.642 INFO:teuthology.orchestra.run.trial078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71190.log.gz 2026-02-17T06:11:32.643 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61970.log 2026-02-17T06:11:32.643 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51924.log 2026-02-17T06:11:32.643 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61970.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.56925.log.gz 2026-02-17T06:11:32.643 INFO:teuthology.orchestra.run.trial078.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.54926.log.gz 2026-02-17T06:11:32.643 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64672.log 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59924.log 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61970.log.gz 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51924.log.gz 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71386.log 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58729.log 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64672.log.gz 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59924.log.gz 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61529.log 2026-02-17T06:11:32.644 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58729.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71386.log.gz 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62869.log 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.61529.log: gzip -5 --verbose 91.2% -- -- replaced with /var/log/ceph/ceph-client.admin.58729.log.gz 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr: /var/log/ceph/ceph-client.admin.69164.log 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61529.log.gz 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46582.log 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69164.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62869.log.gz 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69669.log 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46582.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.69164.log.gz 2026-02-17T06:11:32.645 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67325.log 2026-02-17T06:11:32.646 INFO:teuthology.orchestra.run.trial078.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46582.log.gz 2026-02-17T06:11:32.646 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70994.log 2026-02-17T06:11:32.646 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67325.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69669.log.gz 2026-02-17T06:11:32.646 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62820.log 2026-02-17T06:11:32.646 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70994.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67325.log.gz 2026-02-17T06:11:32.646 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60628.log 2026-02-17T06:11:32.646 INFO:teuthology.orchestra.run.trial078.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70994.log.gz 2026-02-17T06:11:32.646 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55874.log 2026-02-17T06:11:32.647 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60628.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62820.log.gz 2026-02-17T06:11:32.647 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48878.log 2026-02-17T06:11:32.647 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55874.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60628.log.gz 2026-02-17T06:11:32.647 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67374.log 2026-02-17T06:11:32.647 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48878.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.55874.log.gz 2026-02-17T06:11:32.647 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49826.log 2026-02-17T06:11:32.647 INFO:teuthology.orchestra.run.trial078.stderr: 88.7%/var/log/ceph/ceph-client.admin.67374.log: -- replaced with /var/log/ceph/ceph-client.admin.48878.log.gz 2026-02-17T06:11:32.647 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50170.log 2026-02-17T06:11:32.648 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49826.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67374.log.gz 2026-02-17T06:11:32.648 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60169.log 2026-02-17T06:11:32.648 INFO:teuthology.orchestra.run.trial078.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49826.log.gz/var/log/ceph/ceph-client.admin.50170.log: 2026-02-17T06:11:32.648 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65673.log 2026-02-17T06:11:32.648 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60169.log: gzip -5 --verbose -- 90.4% /var/log/ceph/ceph-client.admin.58876.log -- replaced with /var/log/ceph/ceph-client.admin.50170.log.gz 2026-02-17T06:11:32.648 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:32.648 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65673.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60169.log.gz 2026-02-17T06:11:32.648 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68654.log 2026-02-17T06:11:32.649 INFO:teuthology.orchestra.run.trial078.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65673.log.gz 2026-02-17T06:11:32.649 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62182.log 2026-02-17T06:11:32.649 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68654.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.58876.log.gz 2026-02-17T06:11:32.649 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68409.log 2026-02-17T06:11:32.649 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62182.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68654.log.gz 2026-02-17T06:11:32.649 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70308.log 2026-02-17T06:11:32.649 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68409.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62182.log.gz 2026-02-17T06:11:32.649 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71288.log 2026-02-17T06:11:32.650 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70308.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68409.log.gz 2026-02-17T06:11:32.650 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63508.log 2026-02-17T06:11:32.650 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47026.log 2026-02-17T06:11:32.650 INFO:teuthology.orchestra.run.trial078.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70308.log.gz 2026-02-17T06:11:32.650 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63508.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71288.log.gz 2026-02-17T06:11:32.650 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70700.log 2026-02-17T06:11:32.650 INFO:teuthology.orchestra.run.trial078.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63508.log.gz 2026-02-17T06:11:32.650 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69017.log 2026-02-17T06:11:32.651 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56365.log 2026-02-17T06:11:32.651 INFO:teuthology.orchestra.run.trial078.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47026.log.gz 2026-02-17T06:11:32.651 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69017.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70700.log.gz 2026-02-17T06:11:32.651 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62019.log 2026-02-17T06:11:32.651 INFO:teuthology.orchestra.run.trial078.stderr: 88.5%/var/log/ceph/ceph-client.admin.56365.log: -- replaced with /var/log/ceph/ceph-client.admin.69017.log.gz 2026-02-17T06:11:32.651 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45880.log 2026-02-17T06:11:32.651 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62019.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56365.log.gz 2026-02-17T06:11:32.652 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52876.log 2026-02-17T06:11:32.652 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45880.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62019.log.gz 2026-02-17T06:11:32.652 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66016.log 2026-02-17T06:11:32.652 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52876.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45880.log.gz 2026-02-17T06:11:32.652 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52974.log 2026-02-17T06:11:32.652 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66016.log: gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.52876.log.gz --verbose 2026-02-17T06:11:32.652 INFO:teuthology.orchestra.run.trial078.stderr: -- /var/log/ceph/ceph-client.admin.53435.log 2026-02-17T06:11:32.652 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52974.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.66016.log.gz 2026-02-17T06:11:32.653 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64623.log 2026-02-17T06:11:32.653 INFO:teuthology.orchestra.run.trial078.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52974.log.gz 2026-02-17T06:11:32.653 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55024.log 2026-02-17T06:11:32.653 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64623.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53435.log.gz 2026-02-17T06:11:32.653 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65820.log 2026-02-17T06:11:32.653 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55024.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64623.log.gz 2026-02-17T06:11:32.653 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64966.log 2026-02-17T06:11:32.653 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55024.log.gz 2026-02-17T06:11:32.654 INFO:teuthology.orchestra.run.trial078.stderr:gzip/var/log/ceph/ceph-client.admin.65820.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62722.log 2026-02-17T06:11:32.654 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68703.log 2026-02-17T06:11:32.654 INFO:teuthology.orchestra.run.trial078.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.65820.log.gz 2026-02-17T06:11:32.654 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62722.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64966.log.gz 2026-02-17T06:11:32.654 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48161.log 2026-02-17T06:11:32.654 INFO:teuthology.orchestra.run.trial078.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62722.log.gz 2026-02-17T06:11:32.654 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62378.log 2026-02-17T06:11:32.655 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48161.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68703.log.gz 2026-02-17T06:11:32.655 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60973.log 2026-02-17T06:11:32.655 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.62378.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48161.log.gz 2026-02-17T06:11:32.655 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55626.log 2026-02-17T06:11:32.655 INFO:teuthology.orchestra.run.trial078.stderr: 88.5%/var/log/ceph/ceph-client.admin.60973.log: -- replaced with /var/log/ceph/ceph-client.admin.62378.log.gz 2026-02-17T06:11:32.655 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48682.log 2026-02-17T06:11:32.655 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55626.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60973.log.gz 2026-02-17T06:11:32.655 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54564.log 2026-02-17T06:11:32.656 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69311.log 2026-02-17T06:11:32.656 INFO:teuthology.orchestra.run.trial078.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.55626.log.gz 2026-02-17T06:11:32.656 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58463.log 2026-02-17T06:11:32.656 INFO:teuthology.orchestra.run.trial078.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.48682.log.gz 2026-02-17T06:11:32.656 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58630.log 2026-02-17T06:11:32.656 INFO:teuthology.orchestra.run.trial078.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.54564.log.gz 2026-02-17T06:11:32.656 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58463.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69311.log.gz 2026-02-17T06:11:32.656 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57366.log 2026-02-17T06:11:32.657 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58630.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58463.log.gz 2026-02-17T06:11:32.657 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70749.log 2026-02-17T06:11:32.657 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57366.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58630.log.gz 2026-02-17T06:11:32.657 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56415.log 2026-02-17T06:11:32.657 INFO:teuthology.orchestra.run.trial078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57366.log.gz 2026-02-17T06:11:32.657 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47179.log 2026-02-17T06:11:32.657 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56415.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.70749.log.gz 2026-02-17T06:11:32.657 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46484.log 2026-02-17T06:11:32.658 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47179.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56415.log.gz 2026-02-17T06:11:32.658 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-17T06:11:32.658 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.46484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63607.log 2026-02-17T06:11:32.658 INFO:teuthology.orchestra.run.trial078.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.47179.log.gz 2026-02-17T06:11:32.658 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46484.log.gz 2026-02-17T06:11:32.658 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55727.log 2026-02-17T06:11:32.658 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.63607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56217.log 2026-02-17T06:11:32.659 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55727.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63607.log.gz 2026-02-17T06:11:32.659 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55171.log 2026-02-17T06:11:32.659 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.56217.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55727.log.gz 2026-02-17T06:11:32.659 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52169.log 2026-02-17T06:11:32.659 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55171.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.56217.log.gz 2026-02-17T06:11:32.659 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51531.log 2026-02-17T06:11:32.659 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55367.log 2026-02-17T06:11:32.659 INFO:teuthology.orchestra.run.trial078.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55171.log.gz 2026-02-17T06:11:32.660 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51531.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52169.log.gz 2026-02-17T06:11:32.660 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64917.log 2026-02-17T06:11:32.660 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55367.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51531.log.gz 2026-02-17T06:11:32.660 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52728.log 2026-02-17T06:11:32.660 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.64917.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55367.log.gz 2026-02-17T06:11:32.660 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55073.log 2026-02-17T06:11:32.660 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52728.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64917.log.gz 2026-02-17T06:11:32.660 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71776.log 2026-02-17T06:11:32.661 INFO:teuthology.orchestra.run.trial078.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52728.log.gz 2026-02-17T06:11:32.661 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50924.log 2026-02-17T06:11:32.661 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71776.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55073.log.gz 2026-02-17T06:11:32.661 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66425.log 2026-02-17T06:11:32.661 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50924.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71776.log.gz 2026-02-17T06:11:32.661 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59072.log 2026-02-17T06:11:32.661 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66425.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50924.log.gz 2026-02-17T06:11:32.661 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58022.log 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59072.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66425.log.gz 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51826.log 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.58022.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59072.log.gz 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47474.log 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51826.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58022.log.gz 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65064.log 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47474.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.51826.log.gz 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45308.log 2026-02-17T06:11:32.662 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65064.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47474.log.gz 2026-02-17T06:11:32.663 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57590.log 2026-02-17T06:11:32.663 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45308.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65064.log.gz 2026-02-17T06:11:32.663 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71043.log 2026-02-17T06:11:32.663 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.57590.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45308.log.gz 2026-02-17T06:11:32.663 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70014.log 2026-02-17T06:11:32.663 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71043.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57590.log.gz 2026-02-17T06:11:32.663 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60071.log 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70014.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71043.log.gz 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47124.log 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.60071.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.70014.log.gz 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49531.log 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.47124.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.60071.log.gz 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53976.log 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49531.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47124.log.gz 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48308.log 2026-02-17T06:11:32.664 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53976.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49531.log.gz 2026-02-17T06:11:32.665 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54123.log 2026-02-17T06:11:32.665 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.48308.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53976.log.gz 2026-02-17T06:11:32.665 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51728.log 2026-02-17T06:11:32.665 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54123.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48308.log.gz 2026-02-17T06:11:32.665 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55478.log 2026-02-17T06:11:32.665 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.51728.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54123.log.gz 2026-02-17T06:11:32.665 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67276.log 2026-02-17T06:11:32.665 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.55478.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51728.log.gz 2026-02-17T06:11:32.666 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71239.log 2026-02-17T06:11:32.666 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67276.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55478.log.gz 2026-02-17T06:11:32.666 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66768.log 2026-02-17T06:11:32.666 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.71239.log: gzip -5 --verbose -- 90.6% /var/log/ceph/ceph.tmp-client.admin.37554.log 2026-02-17T06:11:32.666 INFO:teuthology.orchestra.run.trial078.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67276.log.gz 2026-02-17T06:11:32.666 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.66768.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71239.log.gz 2026-02-17T06:11:32.666 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70847.log 2026-02-17T06:11:32.666 INFO:teuthology.orchestra.run.trial078.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66768.log.gz 2026-02-17T06:11:32.667 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph.tmp-client.admin.37554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68605.log 2026-02-17T06:11:32.667 INFO:teuthology.orchestra.run.trial078.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.37554.log.gz 2026-02-17T06:11:32.667 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.70847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67965.log 2026-02-17T06:11:32.667 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68605.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70847.log.gz 2026-02-17T06:11:32.667 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68918.log 2026-02-17T06:11:32.667 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.67965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65428.log 2026-02-17T06:11:32.667 INFO:teuthology.orchestra.run.trial078.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.68605.log.gz 2026-02-17T06:11:32.667 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.68918.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67965.log.gz 2026-02-17T06:11:32.668 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49172.log 2026-02-17T06:11:32.668 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.65428.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68918.log.gz 2026-02-17T06:11:32.668 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69916.log 2026-02-17T06:11:32.668 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.49172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52244.log 2026-02-17T06:11:32.668 INFO:teuthology.orchestra.run.trial078.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.65428.log.gz 2026-02-17T06:11:32.668 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69916.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49172.log.gz 2026-02-17T06:11:32.668 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50483.log 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69916.log.gz 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.52244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53729.log 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.50483.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.52244.log.gz -5 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45585.log 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.53729.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50483.log.gz 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69767.log 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.45585.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53729.log.gz 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59121.log 2026-02-17T06:11:32.669 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.69767.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45585.log.gz 2026-02-17T06:11:32.670 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54074.log 2026-02-17T06:11:32.670 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.59121.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69767.log.gz 2026-02-17T06:11:32.670 INFO:teuthology.orchestra.run.trial078.stderr:/var/log/ceph/ceph-client.admin.54074.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59121.log.gz 2026-02-17T06:11:32.670 INFO:teuthology.orchestra.run.trial078.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54074.log.gz 2026-02-17T06:11:32.673 INFO:teuthology.orchestra.run.trial078.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-17T06:11:32.710 INFO:teuthology.orchestra.run.trial115.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-17T06:11:32.856 INFO:teuthology.orchestra.run.trial078.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-17T06:11:32.933 INFO:teuthology.orchestra.run.trial115.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-17T06:11:33.107 INFO:teuthology.orchestra.run.trial078.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-17T06:11:34.098 INFO:teuthology.orchestra.run.trial115.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-17T06:11:34.360 INFO:teuthology.orchestra.run.trial078.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-17T06:11:34.761 INFO:teuthology.orchestra.run.trial078.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-17T06:11:35.184 INFO:teuthology.orchestra.run.trial115.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-17T06:11:35.521 INFO:teuthology.orchestra.run.trial115.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-17T06:11:35.523 INFO:teuthology.orchestra.run.trial115.stderr: 2026-02-17T06:11:35.523 INFO:teuthology.orchestra.run.trial115.stderr:real 0m2.958s 2026-02-17T06:11:35.523 INFO:teuthology.orchestra.run.trial115.stderr:user 0m7.278s 2026-02-17T06:11:35.523 INFO:teuthology.orchestra.run.trial115.stderr:sys 0m0.357s 2026-02-17T06:11:36.223 INFO:teuthology.orchestra.run.trial078.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-17T06:11:36.224 INFO:teuthology.orchestra.run.trial078.stderr: 2026-02-17T06:11:36.224 INFO:teuthology.orchestra.run.trial078.stderr:real 0m3.698s 2026-02-17T06:11:36.224 INFO:teuthology.orchestra.run.trial078.stderr:user 0m8.284s 2026-02-17T06:11:36.225 INFO:teuthology.orchestra.run.trial078.stderr:sys 0m0.659s 2026-02-17T06:11:36.225 INFO:tasks.ceph:Archiving logs... 2026-02-17T06:11:36.226 DEBUG:teuthology.misc:Transferring archived files from trial078:/var/log/ceph to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/remote/trial078/log 2026-02-17T06:11:36.227 DEBUG:teuthology.orchestra.run.trial078:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-17T06:11:37.318 DEBUG:teuthology.misc:Transferring archived files from trial115:/var/log/ceph to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/remote/trial115/log 2026-02-17T06:11:37.320 DEBUG:teuthology.orchestra.run.trial115:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-17T06:11:38.020 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T06:11:38.031 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T06:11:38.031 DEBUG:teuthology.orchestra.run.trial078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T06:11:38.033 DEBUG:teuthology.orchestra.run.trial115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T06:11:38.188 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T06:11:38.199 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T06:11:38.199 DEBUG:teuthology.orchestra.run.trial078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T06:11:38.202 DEBUG:teuthology.orchestra.run.trial115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T06:11:38.216 INFO:teuthology.orchestra.run.trial078.stderr:bash: line 1: ntpq: command not found 2026-02-17T06:11:38.218 INFO:teuthology.orchestra.run.trial115.stderr:bash: line 1: ntpq: command not found 2026-02-17T06:11:38.241 INFO:teuthology.orchestra.run.trial078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T06:11:38.241 INFO:teuthology.orchestra.run.trial078.stdout:=============================================================================== 2026-02-17T06:11:38.241 INFO:teuthology.orchestra.run.trial078.stdout:^- li1210-167.members.linod> 2 6 377 22 -51us[ -51us] +/- 96ms 2026-02-17T06:11:38.241 INFO:teuthology.orchestra.run.trial078.stdout:^* 209.205.228.50 2 7 377 80 -358us[ -413us] +/- 26ms 2026-02-17T06:11:38.241 INFO:teuthology.orchestra.run.trial078.stdout:^- time1.lshiy.com 2 7 377 16 +1493us[+1493us] +/- 123ms 2026-02-17T06:11:38.241 INFO:teuthology.orchestra.run.trial078.stdout:^- 170-187-142-180.ip.linod> 2 8 377 11 -7892us[-7892us] +/- 82ms 2026-02-17T06:11:38.254 INFO:teuthology.orchestra.run.trial115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T06:11:38.254 INFO:teuthology.orchestra.run.trial115.stdout:=============================================================================== 2026-02-17T06:11:38.254 INFO:teuthology.orchestra.run.trial115.stdout:^- lithium.constant.com 2 8 377 14 -133us[ -133us] +/- 51ms 2026-02-17T06:11:38.255 INFO:teuthology.orchestra.run.trial115.stdout:^- 172-232-15-202.ip.linode> 2 6 377 78 -3197us[-3209us] +/- 79ms 2026-02-17T06:11:38.255 INFO:teuthology.orchestra.run.trial115.stdout:^- 72-46-53-234.dia-static.> 3 8 377 16 +641us[ +641us] +/- 74ms 2026-02-17T06:11:38.255 INFO:teuthology.orchestra.run.trial115.stdout:^* nyc3.us.ntp.li 2 8 377 17 -69us[ -81us] +/- 3265us 2026-02-17T06:11:38.255 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T06:11:38.266 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T06:11:38.266 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T06:11:38.279 DEBUG:teuthology.orchestra.run.trial078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T06:11:38.342 DEBUG:teuthology.orchestra.run.trial115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T06:11:38.395 DEBUG:teuthology.orchestra.run.trial078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T06:11:38.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:11:38.424 DEBUG:teuthology.orchestra.run.trial115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T06:11:38.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:11:38.450 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T06:11:38.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T06:11:38.476 INFO:teuthology.task.internal:Duration was 1814.983617 seconds 2026-02-17T06:11:38.476 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T06:11:38.496 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T06:11:38.496 DEBUG:teuthology.orchestra.run.trial078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T06:11:38.499 DEBUG:teuthology.orchestra.run.trial115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T06:11:38.534 INFO:teuthology.orchestra.run.trial078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T06:11:38.536 INFO:teuthology.orchestra.run.trial115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T06:11:38.769 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T06:11:38.769 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial078.front.sepia.ceph.com 2026-02-17T06:11:38.769 DEBUG:teuthology.orchestra.run.trial078:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T06:11:38.820 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial115.front.sepia.ceph.com 2026-02-17T06:11:38.821 DEBUG:teuthology.orchestra.run.trial115:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T06:11:38.840 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T06:11:38.840 DEBUG:teuthology.orchestra.run.trial078:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:11:38.863 DEBUG:teuthology.orchestra.run.trial115:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:11:38.942 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T06:11:38.942 DEBUG:teuthology.orchestra.run.trial078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:11:38.944 DEBUG:teuthology.orchestra.run.trial115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:11:38.964 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T06:11:38.965 INFO:teuthology.orchestra.run.trial078.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T06:11:38.965 INFO:teuthology.orchestra.run.trial078.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:11:38.965 INFO:teuthology.orchestra.run.trial078.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T06:11:38.965 INFO:teuthology.orchestra.run.trial078.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T06:11:38.967 INFO:teuthology.orchestra.run.trial115.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T06:11:38.967 INFO:teuthology.orchestra.run.trial115.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T06:11:38.968 INFO:teuthology.orchestra.run.trial115.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T06:11:38.968 INFO:teuthology.orchestra.run.trial115.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:11:38.968 INFO:teuthology.orchestra.run.trial115.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T06:11:38.977 INFO:teuthology.orchestra.run.trial078.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 85.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T06:11:38.978 INFO:teuthology.orchestra.run.trial115.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T06:11:38.980 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T06:11:38.996 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T06:11:38.996 DEBUG:teuthology.orchestra.run.trial078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T06:11:39.043 DEBUG:teuthology.orchestra.run.trial115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T06:11:39.066 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T06:11:39.083 DEBUG:teuthology.orchestra.run.trial078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:11:39.086 DEBUG:teuthology.orchestra.run.trial115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:11:39.107 INFO:teuthology.orchestra.run.trial078.stdout:kernel.core_pattern = core 2026-02-17T06:11:39.129 INFO:teuthology.orchestra.run.trial115.stdout:kernel.core_pattern = core 2026-02-17T06:11:39.147 DEBUG:teuthology.orchestra.run.trial078:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:11:39.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:11:39.179 DEBUG:teuthology.orchestra.run.trial115:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:11:39.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:11:39.203 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T06:11:39.218 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T06:11:39.219 DEBUG:teuthology.misc:Transferring archived files from trial078:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/remote/trial078 2026-02-17T06:11:39.219 DEBUG:teuthology.orchestra.run.trial078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T06:11:39.256 DEBUG:teuthology.misc:Transferring archived files from trial115:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51059/remote/trial115 2026-02-17T06:11:39.256 DEBUG:teuthology.orchestra.run.trial115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T06:11:39.291 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T06:11:39.291 DEBUG:teuthology.orchestra.run.trial078:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T06:11:39.294 DEBUG:teuthology.orchestra.run.trial115:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T06:11:39.339 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T06:11:39.359 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T06:11:39.359 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T06:11:39.377 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T06:11:39.378 DEBUG:teuthology.orchestra.run.trial078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T06:11:39.381 DEBUG:teuthology.orchestra.run.trial115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T06:11:39.395 INFO:teuthology.orchestra.run.trial078.stdout: 791885 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 06:11 /home/ubuntu/cephtest 2026-02-17T06:11:39.397 INFO:teuthology.orchestra.run.trial115.stdout: 791886 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 06:11 /home/ubuntu/cephtest 2026-02-17T06:11:39.398 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T06:11:39.416 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T06:11:39.455 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/one workloads/snaps-few-objects} duration: 1814.983617067337 flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com success: true 2026-02-17T06:11:39.455 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T06:11:39.570 INFO:teuthology.run:pass